Commit Graph
86 Commits
Author SHA1 Message Date
José Valim bffb79e553 Update to Unicode 12.1 2019-07-05 11:04:08 +02:00
José Valim 0e6b8732ba Improve docs for built-in data types 2019-07-05 11:04:08 +02:00
José Valim 076dece778 Avoid unicode codepoint computation for lower bytes 2019-07-04 09:38:55 +02:00
Eric Meadows-Jönsson 9b6ec7bae7 Move unreachable function check from xref to group pass (#9168) 2019-06-28 10:25:49 +02:00
Eksperimental d4f8d419e4 Use https instead of http when servers are redirecting (#9094)
The servers are automatically redirecting from http:// to https://
2019-05-29 07:37:54 +02:00
José Valim 93fb75d793 Optimize Unicode graphemes
We only check for Pictographic once we find a ZWJ.
2019-03-27 01:46:34 +01:00
Eksperimental 99d919e0c8 Use "code point" instead of "codepoint" (#8774)
That's the term it is used in the Unicode standard.
2019-02-06 11:21:03 +01:00
Xavier Noria ee5a7fa0a4 Clarify String.next_codepoint/1 re invalid UTF-8 (#8489) 2018-12-08 16:41:57 +01:00
Sun Yaozhu 1967a06abe Fix ZWJ handling in Unicode grapheme clusters (#8361) 2018-11-04 19:56:36 +01:00
José Valim 67b7ffce61 Print number of tests from graphemes tests 2018-10-02 12:25:50 +02:00
José Valim 54cb02c240 Rely on Erlang/OTP normalization which is faster 2018-07-12 13:28:58 +02:00
José Valim c995ba968f Update to Unicode 11 2018-06-06 11:22:36 +02:00
Michał Muskała 722aef3791 Use binary_part/3 instead of :binary.part/3
binary_part is a guard BIF and is slightly more efficient to use then
:binary.part which does a fully qualified external call.
2018-04-30 13:22:19 +02:00
José Valim a7eb9adfe9 Consider case ignorable characters on Greek downcasing, closes #7149
Note this does not impact the runtime cost of other downcasing
operations. The final beam file grew only in 8kb.
2018-01-09 13:59:13 +01:00
José Valim 44a8f665b1 Improvements to greek handling of downcase, closes #7149 2017-12-26 23:11:17 +01:00
José Valim 6ecfbc546c Optimize String.downcase/1
Thanks to @michalmuskala for feedback and review.
2017-12-20 12:58:17 +01:00
José Valim 18288881c2 Provide :greek and :ascii mappings in String upcase/downcase/capitalize
Closes #7105.
2017-12-20 11:54:06 +01:00
José Valim b4aac2d7be Do not indent right-side of pipelines in the formatter (#7103) 2017-12-11 21:07:30 +01:00
José Valim b3992aa574 Minimize generated beam size by using ranges
This approach is less performant when processing
sigma but compiles twice faster and takes two times
less space in disk.
2017-10-30 18:45:16 +01:00
Ben Olive dfcc88fc33 Handle final sigma unicode exception
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2017-10-30 18:45:16 +01:00
José Valim c217203558 Improvements to previously formatted files 2017-10-11 22:59:30 +02:00
José Valim 97ad07b7ab Make sure to not break guards on clauses with comments 2017-10-11 21:25:21 +02:00
José Valim 660838b399 Format more files with little formatting requirement 2017-10-11 00:28:10 +02:00
Frank McGeough ba9d08a54b Run formatter on lib/elixir/unicode/tokenizer.ex (#6813) 2017-10-10 18:10:35 +02:00
Guilherme Pasqualino e08df95e86 Run code formatter on lib/elixir/unicode/unicode.ex (#6795) 2017-10-10 12:24:15 +02:00
Michał Muskała c7a7a3625a Add is_binary guard to String.length (#6430)
This produces a bit more readable message if not a string is passed into `String.length`, then failing in the `next_grapheme_size` function.
2017-08-03 21:04:12 +02:00
Michał Muskała df33881ceb Optimise binary pattern matching in String.Break (#6340)
In downcase and upcase pattern match tail as ::bits instead of ::binary. This
saves us a check on each iteration if the tail is a proper binary. We only do
this check in the very beginning.

In trim_leading additionally make sure every head of the recursive function
pattern matches on a binary in first argument so the single binary match context
optimisation can be applied.
2017-07-16 14:29:45 +02:00
José Valim a311ae403e Update to Unicode 10 (#6293) 2017-07-04 09:57:07 +02:00
Andrea Leopardi 3b365510e5 Use :binary.split/3 in String.Break.split/1 (#6257) 2017-06-25 18:53:11 +02:00
José Valim 3c9b191ec1 Do not rely on on_load because it breaks releases 2017-05-29 15:50:06 +02:00
José Valim 9873e4239f Add non-quoted Unicode atoms and variables (#6158)
It follows Unicode Annex #31.

See the Unicode Syntax document for a more in-depth reference.
2017-05-27 22:43:05 +02:00
José Valim 7ca7139bb4 Properly slice titlecase_once binaries 2017-05-19 12:37:05 +02:00
Aleksei Magusev 683a3ad3c5 Correct module name for grapheme tests 2017-02-23 23:58:20 +01:00
José Valim 636c49fb5c Add grapheme tests 2017-02-13 15:44:59 +01:00
José Valim 3e394e8ef0 Incorporate new grapheme rules in Unicode 9 2017-02-13 14:34:04 +01:00
Andrea Leopardi 3604554aea Remove leftover trailing whitespace from the code base
[ci skip]
2017-01-13 15:01:47 +01:00
José Valim bd7b4847bd Speed up String.split/1 2016-11-21 19:03:16 +01:00
José Valim 1606c40453 Update to Unicode 9.0.0
The update instructions have also been outlined
on top of the unicode.ex file.

Closes #4864.
2016-11-20 00:13:12 +01:00
Miles Starkenburg 551a9ac14e Fix nfd normalization bug (#4606)
* Fix nfd normalization bug

* Change String.normalize return type to binary
2016-05-13 09:28:50 +02:00
eksperimental 6da34e0225 Standardize unicode and friends (#4569)
* Unicode is a proper noun, so it should be capitalized

* Replace 'char data' with 'chardata'

* Correct use of a/an articles
2016-05-05 09:21:05 +02:00
Aleksei Magusev 7e20b6ef99 Merge pull request #4535 from lexmag/string-trim-pad
Introduce String.pad_{leading,trailing}/3 and String.trim{,_leading,_trailing}/2
2016-04-25 20:02:19 +02:00
Aleksei Magusev 24ab310950 Introduce String.trim{,_leading,_trailing}/2 2016-04-25 19:59:24 +02:00
eksperimental 4ceb41e71b Formmating: Add white space around vertical bar (#4507) 2016-04-25 00:55:49 +02:00
José Valim 8fe87ffa3c Fold decomposition recursions at compile time
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 16:55:29 +02:00
José Valim 79b132a665 Do not reorder starting classes
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 13:38:14 +02:00
José Valim cd953f5883 Handle compositios with non-zero combining class
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 13:38:10 +02:00
José Valim 107a185209 More optimizations and improvements to normalization
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 13:38:07 +02:00
José Valim c07f2b2820 Make decomposition recursive and consider exclusion list
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 13:38:00 +02:00
José Valim ec027fa421 Use combining_classes from UnicodeData
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 13:37:56 +02:00
José Valim 664feb1802 Rely only on UnicodeData for composition/decomposition 2016-03-29 00:54:20 +02:00