Commit Graph
72 Commits
Author SHA1 Message Date
Cẩm Huỳnh 5b59ce40da Optimize String.codepoints (#10633) 2021-01-08 19:20:27 +01:00
José Valim 0820b21a0d Consider CRLF on Windows, closes #10470 2020-11-01 17:25:06 +01:00
Eksperimental 0acab05f53 Use titlecase in Elixir, Erlang, EEx, Unicode, and Dialyzer (#9557) 2019-11-19 09:10:29 +01:00
José Valim f410cb13d1 Update to Unicode 12.1 2019-07-05 11:04:08 +02:00
José Valim 45ae6faeb5 Improve docs for built-in data types 2019-07-05 11:04:08 +02:00
José Valim 3f976942c2 Avoid unicode codepoint computation for lower bytes 2019-07-04 09:38:55 +02:00
Eric Meadows-Jönsson 1f974f869e Move unreachable function check from xref to group pass (#9168) 2019-06-28 10:25:49 +02:00
José Valim 6a05455c2e Optimize Unicode graphemes
We only check for Pictographic once we find a ZWJ.
2019-03-27 01:46:34 +01:00
Eksperimental 266caa2f2f Use "code point" instead of "codepoint" (#8774)
That's the term it is used in the Unicode standard.
2019-02-06 11:21:03 +01:00
Xavier Noria b784c9315d Clarify String.next_codepoint/1 re invalid UTF-8 (#8489) 2018-12-08 16:41:57 +01:00
Sun Yaozhu 6760ba44c7 Fix ZWJ handling in Unicode grapheme clusters (#8361) 2018-11-04 19:56:36 +01:00
José Valim 99868597b1 Rely on Erlang/OTP normalization which is faster 2018-07-12 13:28:58 +02:00
José Valim c43136216b Update to Unicode 11 2018-06-06 11:22:36 +02:00
Michał Muskała 7cfb63c636 Use binary_part/3 instead of :binary.part/3
binary_part is a guard BIF and is slightly more efficient to use then
:binary.part which does a fully qualified external call.
2018-04-30 13:22:19 +02:00
Guilherme Pasqualino 5c21265cc8 Run code formatter on lib/elixir/unicode/unicode.ex (#6795) 2017-10-10 12:24:15 +02:00
Michał Muskała dae60dbea4 Add is_binary guard to String.length (#6430)
This produces a bit more readable message if not a string is passed into `String.length`, then failing in the `next_grapheme_size` function.
2017-08-03 21:04:12 +02:00
José Valim e8690e412a Update to Unicode 10 (#6293) 2017-07-04 09:57:07 +02:00
José Valim 2f663ce3a2 Add non-quoted Unicode atoms and variables (#6158)
It follows Unicode Annex #31.

See the Unicode Syntax document for a more in-depth reference.
2017-05-27 22:43:05 +02:00
José Valim 50d16830fd Properly slice titlecase_once binaries 2017-05-19 12:37:05 +02:00
José Valim de1b731afb Add grapheme tests 2017-02-13 15:44:59 +01:00
José Valim b1c567b5ef Incorporate new grapheme rules in Unicode 9 2017-02-13 14:34:04 +01:00
Andrea Leopardi 7222c1bce3 Remove leftover trailing whitespace from the code base
[ci skip]
2017-01-13 15:01:47 +01:00
José Valim cae33236ad Speed up String.split/1 2016-11-21 19:03:16 +01:00
José Valim 1b3efa9625 Update to Unicode 9.0.0
The update instructions have also been outlined
on top of the unicode.ex file.

Closes #4864.
2016-11-20 00:13:12 +01:00
Miles Starkenburg 52ecfb0754 Fix nfd normalization bug (#4606)
* Fix nfd normalization bug

* Change String.normalize return type to binary
2016-05-13 09:28:50 +02:00
eksperimental bc8c2a55e9 Standardize unicode and friends (#4569)
* Unicode is a proper noun, so it should be capitalized

* Replace 'char data' with 'chardata'

* Correct use of a/an articles
2016-05-05 09:21:05 +02:00
Aleksei Magusev 6c96174a0b Merge pull request #4535 from lexmag/string-trim-pad
Introduce String.pad_{leading,trailing}/3 and String.trim{,_leading,_trailing}/2
2016-04-25 20:02:19 +02:00
Aleksei Magusev 0ea04744bc Introduce String.trim{,_leading,_trailing}/2 2016-04-25 19:59:24 +02:00
eksperimental 685e87bdf0 Formmating: Add white space around vertical bar (#4507) 2016-04-25 00:55:49 +02:00
José Valim 7da99878cb Fold decomposition recursions at compile time
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 16:55:29 +02:00
José Valim 3693966251 Do not reorder starting classes
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 13:38:14 +02:00
José Valim d1b6a8d553 Handle compositios with non-zero combining class
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 13:38:10 +02:00
José Valim 0de30c7af5 More optimizations and improvements to normalization
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 13:38:07 +02:00
José Valim 196d225489 Make decomposition recursive and consider exclusion list
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 13:38:00 +02:00
José Valim a9e246ecc6 Use combining_classes from UnicodeData
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 13:37:56 +02:00
José Valim 60d7771f41 Rely only on UnicodeData for composition/decomposition 2016-03-29 00:54:20 +02:00
José Valim 0bf5293aab Clean up unicode range parsing 2016-03-18 16:59:13 +01:00
José Valim 1d5ca3beed Update Unicode to 8.0.0 (equivalent pending) 2016-03-15 15:58:01 +01:00
José Valim d3ff4691b8 Avoid regular expressions 2016-03-15 15:43:32 +01:00
Abel Muiño c465f80a6d New definition of whitespace & breakable whitespace
All whitespace can be removed by `strip` but only breakable whitespace
can be used as a delimiter by `split`, with updated docs for String.split/1
2016-03-15 14:54:11 +01:00
José Valim 5b77dd9a14 Remove debug_info from unicode 2016-02-01 09:21:49 +01:00
José Valim 0c4a1f189f Do not include debug_info in metadata String modules 2016-01-08 10:11:07 +01:00
Aleksei Magusev 612fcc4760 Eliminate spaces in bitstring definitions 2015-11-28 23:48:48 +01:00
José Valim cfc4bc20d1 Remove uses of soon to be deprecated Dict 2015-10-23 15:04:24 +02:00
Bryan Endersstocker e12adea1d4 Add NFC support to String.normalize/2 2015-10-03 11:08:55 -04:00
Bryan Endersstocker 180f3184c6 Optimize Unicode normalization
Also extracted String.Normalizer module from String.Unicode.
2015-09-26 10:59:04 -04:00
Bryan Endersstocker a65b8c27da Add String.normalize/2 2015-09-26 00:12:44 -04:00
Bryan Enders 4fb6446145 Rename String.is_equivalent/2 to String.equivalent?/2 2015-09-25 15:01:29 -04:00
Bryan Enders f01ad6df36 Add String.is_equivalent/2 to test canonical Unicode equivalence 2015-09-25 13:35:16 -04:00
José Valim 5ed087ef50 Move away from HashDict and HashSet 2015-09-25 12:46:22 +02:00