Xavier Noria
b784c9315d
Clarify String.next_codepoint/1 re invalid UTF-8 ( #8489 )
2018-12-08 16:41:57 +01:00
Sun Yaozhu
6760ba44c7
Fix ZWJ handling in Unicode grapheme clusters ( #8361 )
2018-11-04 19:56:36 +01:00
José Valim
df26e1bcbd
Print number of tests from graphemes tests
2018-10-02 12:25:50 +02:00
José Valim
99868597b1
Rely on Erlang/OTP normalization which is faster
2018-07-12 13:28:58 +02:00
José Valim
c43136216b
Update to Unicode 11
2018-06-06 11:22:36 +02:00
Michał Muskała
7cfb63c636
Use binary_part/3 instead of :binary.part/3
...
binary_part is a guard BIF and is slightly more efficient to use then
:binary.part which does a fully qualified external call.
2018-04-30 13:22:19 +02:00
José Valim
0db0b85f52
Consider case ignorable characters on Greek downcasing, closes #7149
...
Note this does not impact the runtime cost of other downcasing
operations. The final beam file grew only in 8kb.
2018-01-09 13:59:13 +01:00
José Valim
3f95b4a17d
Improvements to greek handling of downcase, closes #7149
2017-12-26 23:11:17 +01:00
José Valim
8fd0141297
Optimize String.downcase/1
...
Thanks to @michalmuskala for feedback and review.
2017-12-20 12:58:17 +01:00
José Valim
2a6a545b8b
Provide :greek and :ascii mappings in String upcase/downcase/capitalize
...
Closes #7105 .
2017-12-20 11:54:06 +01:00
José Valim
e43f5f53b2
Do not indent right-side of pipelines in the formatter ( #7103 )
2017-12-11 21:07:30 +01:00
José Valim
27aa867b2e
Minimize generated beam size by using ranges
...
This approach is less performant when processing
sigma but compiles twice faster and takes two times
less space in disk.
2017-10-30 18:45:16 +01:00
Ben Olive
6f40fae504
Handle final sigma unicode exception
...
Signed-off-by: José Valim <jose.valim@plataformatec.com.br >
2017-10-30 18:45:16 +01:00
José Valim
dec9e6f0a8
Improvements to previously formatted files
2017-10-11 22:59:30 +02:00
José Valim
940d2f61c0
Make sure to not break guards on clauses with comments
2017-10-11 21:25:21 +02:00
José Valim
c6233421e5
Format more files with little formatting requirement
2017-10-11 00:28:10 +02:00
Frank McGeough
24acb605ba
Run formatter on lib/elixir/unicode/tokenizer.ex ( #6813 )
2017-10-10 18:10:35 +02:00
Guilherme Pasqualino
5c21265cc8
Run code formatter on lib/elixir/unicode/unicode.ex ( #6795 )
2017-10-10 12:24:15 +02:00
Michał Muskała
dae60dbea4
Add is_binary guard to String.length ( #6430 )
...
This produces a bit more readable message if not a string is passed into `String.length`, then failing in the `next_grapheme_size` function.
2017-08-03 21:04:12 +02:00
Michał Muskała
c71926f8db
Optimise binary pattern matching in String.Break ( #6340 )
...
In downcase and upcase pattern match tail as ::bits instead of ::binary. This
saves us a check on each iteration if the tail is a proper binary. We only do
this check in the very beginning.
In trim_leading additionally make sure every head of the recursive function
pattern matches on a binary in first argument so the single binary match context
optimisation can be applied.
2017-07-16 14:29:45 +02:00
José Valim
e8690e412a
Update to Unicode 10 ( #6293 )
2017-07-04 09:57:07 +02:00
Andrea Leopardi
7d873750b1
Use :binary.split/3 in String.Break.split/1 ( #6257 )
2017-06-25 18:53:11 +02:00
José Valim
19b1668cfd
Do not rely on on_load because it breaks releases
2017-05-29 15:50:06 +02:00
José Valim
2f663ce3a2
Add non-quoted Unicode atoms and variables ( #6158 )
...
It follows Unicode Annex #31 .
See the Unicode Syntax document for a more in-depth reference.
2017-05-27 22:43:05 +02:00
José Valim
50d16830fd
Properly slice titlecase_once binaries
2017-05-19 12:37:05 +02:00
Aleksei Magusev
6898fd98d7
Correct module name for grapheme tests
2017-02-23 23:58:20 +01:00
José Valim
de1b731afb
Add grapheme tests
2017-02-13 15:44:59 +01:00
José Valim
b1c567b5ef
Incorporate new grapheme rules in Unicode 9
2017-02-13 14:34:04 +01:00
Andrea Leopardi
7222c1bce3
Remove leftover trailing whitespace from the code base
...
[ci skip]
2017-01-13 15:01:47 +01:00
José Valim
cae33236ad
Speed up String.split/1
2016-11-21 19:03:16 +01:00
José Valim
1b3efa9625
Update to Unicode 9.0.0
...
The update instructions have also been outlined
on top of the unicode.ex file.
Closes #4864 .
2016-11-20 00:13:12 +01:00
Miles Starkenburg
52ecfb0754
Fix nfd normalization bug ( #4606 )
...
* Fix nfd normalization bug
* Change String.normalize return type to binary
2016-05-13 09:28:50 +02:00
eksperimental
bc8c2a55e9
Standardize unicode and friends ( #4569 )
...
* Unicode is a proper noun, so it should be capitalized
* Replace 'char data' with 'chardata'
* Correct use of a/an articles
2016-05-05 09:21:05 +02:00
Aleksei Magusev
6c96174a0b
Merge pull request #4535 from lexmag/string-trim-pad
...
Introduce String.pad_{leading,trailing}/3 and String.trim{,_leading,_trailing}/2
2016-04-25 20:02:19 +02:00
Aleksei Magusev
0ea04744bc
Introduce String.trim{,_leading,_trailing}/2
2016-04-25 19:59:24 +02:00
eksperimental
685e87bdf0
Formmating: Add white space around vertical bar ( #4507 )
2016-04-25 00:55:49 +02:00
José Valim
7da99878cb
Fold decomposition recursions at compile time
...
Signed-off-by: José Valim <jose.valim@plataformatec.com.br >
2016-03-29 16:55:29 +02:00
José Valim
3693966251
Do not reorder starting classes
...
Signed-off-by: José Valim <jose.valim@plataformatec.com.br >
2016-03-29 13:38:14 +02:00
José Valim
d1b6a8d553
Handle compositios with non-zero combining class
...
Signed-off-by: José Valim <jose.valim@plataformatec.com.br >
2016-03-29 13:38:10 +02:00
José Valim
0de30c7af5
More optimizations and improvements to normalization
...
Signed-off-by: José Valim <jose.valim@plataformatec.com.br >
2016-03-29 13:38:07 +02:00
José Valim
196d225489
Make decomposition recursive and consider exclusion list
...
Signed-off-by: José Valim <jose.valim@plataformatec.com.br >
2016-03-29 13:38:00 +02:00
José Valim
a9e246ecc6
Use combining_classes from UnicodeData
...
Signed-off-by: José Valim <jose.valim@plataformatec.com.br >
2016-03-29 13:37:56 +02:00
José Valim
60d7771f41
Rely only on UnicodeData for composition/decomposition
2016-03-29 00:54:20 +02:00
José Valim
0bf5293aab
Clean up unicode range parsing
2016-03-18 16:59:13 +01:00
José Valim
1d5ca3beed
Update Unicode to 8.0.0 (equivalent pending)
2016-03-15 15:58:01 +01:00
José Valim
d3ff4691b8
Avoid regular expressions
2016-03-15 15:43:32 +01:00
Abel Muiño
c465f80a6d
New definition of whitespace & breakable whitespace
...
All whitespace can be removed by `strip` but only breakable whitespace
can be used as a delimiter by `split`, with updated docs for String.split/1
2016-03-15 14:54:11 +01:00
José Valim
5b77dd9a14
Remove debug_info from unicode
2016-02-01 09:21:49 +01:00
José Valim
0c4a1f189f
Do not include debug_info in metadata String modules
2016-01-08 10:11:07 +01:00
Aleksei Magusev
612fcc4760
Eliminate spaces in bitstring definitions
2015-11-28 23:48:48 +01:00