Aleksei Magusev
683a3ad3c5
Correct module name for grapheme tests
2017-02-23 23:58:20 +01:00
José Valim
636c49fb5c
Add grapheme tests
2017-02-13 15:44:59 +01:00
José Valim
3e394e8ef0
Incorporate new grapheme rules in Unicode 9
2017-02-13 14:34:04 +01:00
Andrea Leopardi
3604554aea
Remove leftover trailing whitespace from the code base
...
[ci skip]
2017-01-13 15:01:47 +01:00
José Valim
bd7b4847bd
Speed up String.split/1
2016-11-21 19:03:16 +01:00
José Valim
1606c40453
Update to Unicode 9.0.0
...
The update instructions have also been outlined
on top of the unicode.ex file.
Closes #4864 .
2016-11-20 00:13:12 +01:00
Miles Starkenburg
551a9ac14e
Fix nfd normalization bug ( #4606 )
...
* Fix nfd normalization bug
* Change String.normalize return type to binary
2016-05-13 09:28:50 +02:00
eksperimental
6da34e0225
Standardize unicode and friends ( #4569 )
...
* Unicode is a proper noun, so it should be capitalized
* Replace 'char data' with 'chardata'
* Correct use of a/an articles
2016-05-05 09:21:05 +02:00
Aleksei Magusev
7e20b6ef99
Merge pull request #4535 from lexmag/string-trim-pad
...
Introduce String.pad_{leading,trailing}/3 and String.trim{,_leading,_trailing}/2
2016-04-25 20:02:19 +02:00
Aleksei Magusev
24ab310950
Introduce String.trim{,_leading,_trailing}/2
2016-04-25 19:59:24 +02:00
eksperimental
4ceb41e71b
Formmating: Add white space around vertical bar ( #4507 )
2016-04-25 00:55:49 +02:00
José Valim
8fe87ffa3c
Fold decomposition recursions at compile time
...
Signed-off-by: José Valim <jose.valim@plataformatec.com.br >
2016-03-29 16:55:29 +02:00
José Valim
79b132a665
Do not reorder starting classes
...
Signed-off-by: José Valim <jose.valim@plataformatec.com.br >
2016-03-29 13:38:14 +02:00
José Valim
cd953f5883
Handle compositios with non-zero combining class
...
Signed-off-by: José Valim <jose.valim@plataformatec.com.br >
2016-03-29 13:38:10 +02:00
José Valim
107a185209
More optimizations and improvements to normalization
...
Signed-off-by: José Valim <jose.valim@plataformatec.com.br >
2016-03-29 13:38:07 +02:00
José Valim
c07f2b2820
Make decomposition recursive and consider exclusion list
...
Signed-off-by: José Valim <jose.valim@plataformatec.com.br >
2016-03-29 13:38:00 +02:00
José Valim
ec027fa421
Use combining_classes from UnicodeData
...
Signed-off-by: José Valim <jose.valim@plataformatec.com.br >
2016-03-29 13:37:56 +02:00
José Valim
664feb1802
Rely only on UnicodeData for composition/decomposition
2016-03-29 00:54:20 +02:00
José Valim
8199b81f7c
Clean up unicode range parsing
2016-03-18 16:59:13 +01:00
José Valim
e1a1065ee9
Update Unicode to 8.0.0 (equivalent pending)
2016-03-15 15:58:01 +01:00
José Valim
646ae4d63f
Avoid regular expressions
2016-03-15 15:43:32 +01:00
Abel Muiño
98a4ee5da6
New definition of whitespace & breakable whitespace
...
All whitespace can be removed by `strip` but only breakable whitespace
can be used as a delimiter by `split`, with updated docs for String.split/1
2016-03-15 14:54:11 +01:00
José Valim
ea4102355c
Remove debug_info from unicode
2016-02-01 09:21:49 +01:00
José Valim
e65b01b870
Do not include debug_info in metadata String modules
2016-01-08 10:11:07 +01:00
Aleksei Magusev
40096e0ff1
Eliminate spaces in bitstring definitions
2015-11-28 23:48:48 +01:00
José Valim
94a07df8d3
Remove uses of soon to be deprecated Dict
2015-10-23 15:04:24 +02:00
Bryan Endersstocker
07fedf66d0
Add NFC support to String.normalize/2
2015-10-03 11:08:55 -04:00
Bryan Endersstocker
945f8b6466
Optimize Unicode normalization
...
Also extracted String.Normalizer module from String.Unicode.
2015-09-26 10:59:04 -04:00
Bryan Endersstocker
bf02047280
Add String.normalize/2
2015-09-26 00:12:44 -04:00
Bryan Enders
dab2a632b6
Rename String.is_equivalent/2 to String.equivalent?/2
2015-09-25 15:01:29 -04:00
Bryan Enders
f83cc7de5f
Add String.is_equivalent/2 to test canonical Unicode equivalence
2015-09-25 13:35:16 -04:00
José Valim
ff51926fe8
Move away from HashDict and HashSet
2015-09-25 12:46:22 +02:00
eksperimental
a251ccc722
Format raise message consistently
...
Kernel.raise messages should not start with uppercase, and should not
have trailing punctuation.
Mix.raise messages should start with uppercase, and should not have a trailing
puntuation.
# starting with uppercase
# exclude (Mix.raise, and any message that starts with all uppecase such as: I or IO
ag '(?<!Mix\.)raise\s+\"+(?!([A-Z]+\b))[A-Z]'
# Mix.raise should start with upper case
# except when `mix` or `rebar` are mentioned
ag 'Mix\.raise\s+\"(?!(mix|rebar))+[a-z]'
# ending in period
ag "raise\s+\"+.+\.\"\s*\n"
2015-09-08 00:44:17 +07:00
José Valim
0e7232671b
Optimize many functions in String
...
Many functions relied on next_grapheme, which would traverse
the whole string generating a lot of graphemes as garbage along
the way.
This commit introduces next_grapheme_size, which returns the
size in bytes of the next grapheme, avoiding the generation
of the garbage and optimizing functions that need just the
byte_size.
Functions optimized are: String.at, String.split_at, String.slice
and String.length.
2015-08-02 13:49:50 +02:00
José Valim
a52bf5f40e
Speed up upcase and downcase for large strings
2015-06-05 13:12:25 +02:00
José Valim
6efdf35f41
Handle corner cases for small strings in rstrp
2015-06-02 10:13:45 +02:00
José Valim
8574c7b568
Optimize rstrip
...
This new implementation is no longer linear without affecting
smaller samples. For a string that is 100 bytes long, it is
25x faster than the previous implementation.
2015-06-02 09:52:13 +02:00
eksperimental
50ab29ffb1
fix typo in previous commit. bodyless defmacro added to show proper arg in b/1, t/1
2015-03-28 19:19:37 +07:00
eksperimental
3592151b3e
rename function arg. when "other" was used
2015-03-27 02:37:36 +07:00
jw2013
f2863de304
fix wrong CRLF grapheme
2014-11-08 12:45:41 -08:00
José Valim
9717a63604
Do not consider subpatterns on Regex.split/3
2014-07-30 11:13:18 +02:00
José Valim
182cbfe541
Update to Unicode 7.0.0
2014-06-17 17:17:32 +02:00
José Valim
1038018e84
Migrate from size to byte_size and tuple_size
2014-06-06 13:33:02 +02:00
José Valim
540c763c85
Move binary and list conversions to String and List
2014-05-17 17:02:46 +02:00
José Valim
3bb8e17e9e
Add iodata_to_binary and chardata_to_string to IO
2014-05-17 13:06:40 +02:00
José Valim
56f15d9f4e
Do not add spaces after { and before }
...
This makes the source code consistent with the result
returned by inspect/2.
2014-04-21 19:06:35 +02:00
José Valim
024edf3d40
Rename iolist_* to iodata_*
2014-02-23 11:32:03 +01:00
José Valim
3512c1a860
Start migrating lc to for comprehensions
2014-02-17 14:45:04 +01:00
José Valim
ba54afbf45
Move to new sigils syntax
2014-02-05 14:39:04 +01:00
José Valim
0802014779
No longer compile unicode in parallel
...
Compiling in parallel only bought us 2s (out of 24s) since
compiling graphemes is what takes most of the time but it
considerably increases memory usage, by about 30%, since both
compilation targets are loaded into memory at once.
2014-01-22 17:51:24 +01:00