Commit Graph
67 Commits
Author SHA1 Message Date
Ben Olive 6f40fae504 Handle final sigma unicode exception
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2017-10-30 18:45:16 +01:00
José Valim dec9e6f0a8 Improvements to previously formatted files 2017-10-11 22:59:30 +02:00
José Valim 940d2f61c0 Make sure to not break guards on clauses with comments 2017-10-11 21:25:21 +02:00
José Valim c6233421e5 Format more files with little formatting requirement 2017-10-11 00:28:10 +02:00
Frank McGeough 24acb605ba Run formatter on lib/elixir/unicode/tokenizer.ex (#6813) 2017-10-10 18:10:35 +02:00
Guilherme Pasqualino 5c21265cc8 Run code formatter on lib/elixir/unicode/unicode.ex (#6795) 2017-10-10 12:24:15 +02:00
Michał Muskała dae60dbea4 Add is_binary guard to String.length (#6430)
This produces a bit more readable message if not a string is passed into `String.length`, then failing in the `next_grapheme_size` function.
2017-08-03 21:04:12 +02:00
Michał Muskała c71926f8db Optimise binary pattern matching in String.Break (#6340)
In downcase and upcase pattern match tail as ::bits instead of ::binary. This
saves us a check on each iteration if the tail is a proper binary. We only do
this check in the very beginning.

In trim_leading additionally make sure every head of the recursive function
pattern matches on a binary in first argument so the single binary match context
optimisation can be applied.
2017-07-16 14:29:45 +02:00
José Valim e8690e412a Update to Unicode 10 (#6293) 2017-07-04 09:57:07 +02:00
Andrea Leopardi 7d873750b1 Use :binary.split/3 in String.Break.split/1 (#6257) 2017-06-25 18:53:11 +02:00
José Valim 19b1668cfd Do not rely on on_load because it breaks releases 2017-05-29 15:50:06 +02:00
José Valim 2f663ce3a2 Add non-quoted Unicode atoms and variables (#6158)
It follows Unicode Annex #31.

See the Unicode Syntax document for a more in-depth reference.
2017-05-27 22:43:05 +02:00
José Valim 50d16830fd Properly slice titlecase_once binaries 2017-05-19 12:37:05 +02:00
Aleksei Magusev 6898fd98d7 Correct module name for grapheme tests 2017-02-23 23:58:20 +01:00
José Valim de1b731afb Add grapheme tests 2017-02-13 15:44:59 +01:00
José Valim b1c567b5ef Incorporate new grapheme rules in Unicode 9 2017-02-13 14:34:04 +01:00
Andrea Leopardi 7222c1bce3 Remove leftover trailing whitespace from the code base
[ci skip]
2017-01-13 15:01:47 +01:00
José Valim cae33236ad Speed up String.split/1 2016-11-21 19:03:16 +01:00
José Valim 1b3efa9625 Update to Unicode 9.0.0
The update instructions have also been outlined
on top of the unicode.ex file.

Closes #4864.
2016-11-20 00:13:12 +01:00
Miles Starkenburg 52ecfb0754 Fix nfd normalization bug (#4606)
* Fix nfd normalization bug

* Change String.normalize return type to binary
2016-05-13 09:28:50 +02:00
eksperimental bc8c2a55e9 Standardize unicode and friends (#4569)
* Unicode is a proper noun, so it should be capitalized

* Replace 'char data' with 'chardata'

* Correct use of a/an articles
2016-05-05 09:21:05 +02:00
Aleksei Magusev 6c96174a0b Merge pull request #4535 from lexmag/string-trim-pad
Introduce String.pad_{leading,trailing}/3 and String.trim{,_leading,_trailing}/2
2016-04-25 20:02:19 +02:00
Aleksei Magusev 0ea04744bc Introduce String.trim{,_leading,_trailing}/2 2016-04-25 19:59:24 +02:00
eksperimental 685e87bdf0 Formmating: Add white space around vertical bar (#4507) 2016-04-25 00:55:49 +02:00
José Valim 7da99878cb Fold decomposition recursions at compile time
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 16:55:29 +02:00
José Valim 3693966251 Do not reorder starting classes
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 13:38:14 +02:00
José Valim d1b6a8d553 Handle compositios with non-zero combining class
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 13:38:10 +02:00
José Valim 0de30c7af5 More optimizations and improvements to normalization
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 13:38:07 +02:00
José Valim 196d225489 Make decomposition recursive and consider exclusion list
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 13:38:00 +02:00
José Valim a9e246ecc6 Use combining_classes from UnicodeData
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 13:37:56 +02:00
José Valim 60d7771f41 Rely only on UnicodeData for composition/decomposition 2016-03-29 00:54:20 +02:00
José Valim 0bf5293aab Clean up unicode range parsing 2016-03-18 16:59:13 +01:00
José Valim 1d5ca3beed Update Unicode to 8.0.0 (equivalent pending) 2016-03-15 15:58:01 +01:00
José Valim d3ff4691b8 Avoid regular expressions 2016-03-15 15:43:32 +01:00
Abel Muiño c465f80a6d New definition of whitespace & breakable whitespace
All whitespace can be removed by `strip` but only breakable whitespace
can be used as a delimiter by `split`, with updated docs for String.split/1
2016-03-15 14:54:11 +01:00
José Valim 5b77dd9a14 Remove debug_info from unicode 2016-02-01 09:21:49 +01:00
José Valim 0c4a1f189f Do not include debug_info in metadata String modules 2016-01-08 10:11:07 +01:00
Aleksei Magusev 612fcc4760 Eliminate spaces in bitstring definitions 2015-11-28 23:48:48 +01:00
José Valim cfc4bc20d1 Remove uses of soon to be deprecated Dict 2015-10-23 15:04:24 +02:00
Bryan Endersstocker e12adea1d4 Add NFC support to String.normalize/2 2015-10-03 11:08:55 -04:00
Bryan Endersstocker 180f3184c6 Optimize Unicode normalization
Also extracted String.Normalizer module from String.Unicode.
2015-09-26 10:59:04 -04:00
Bryan Endersstocker a65b8c27da Add String.normalize/2 2015-09-26 00:12:44 -04:00
Bryan Enders 4fb6446145 Rename String.is_equivalent/2 to String.equivalent?/2 2015-09-25 15:01:29 -04:00
Bryan Enders f01ad6df36 Add String.is_equivalent/2 to test canonical Unicode equivalence 2015-09-25 13:35:16 -04:00
José Valim 5ed087ef50 Move away from HashDict and HashSet 2015-09-25 12:46:22 +02:00
eksperimental 8787e7019e Format raise message consistently
Kernel.raise messages should not start with uppercase, and should not
have trailing punctuation.

Mix.raise messages should start with uppercase, and should not have a trailing
puntuation.

    # starting with uppercase
    # exclude (Mix.raise, and any message that starts with all uppecase such as: I or IO
    ag '(?<!Mix\.)raise\s+\"+(?!([A-Z]+\b))[A-Z]'

    # Mix.raise should start with upper case
    # except when `mix` or `rebar` are mentioned
    ag 'Mix\.raise\s+\"(?!(mix|rebar))+[a-z]'

    # ending in period
    ag "raise\s+\"+.+\.\"\s*\n"
2015-09-08 00:44:17 +07:00
José Valim 34bbb3d83e Optimize many functions in String
Many functions relied on next_grapheme, which would traverse
the whole string generating a lot of graphemes as garbage along
the way.

This commit introduces next_grapheme_size, which returns the
size in bytes of the next grapheme, avoiding the generation
of the garbage and optimizing functions that need just the
byte_size.

Functions optimized are: String.at, String.split_at, String.slice
and String.length.
2015-08-02 13:49:50 +02:00
José Valim 5dafe3ab7b Speed up upcase and downcase for large strings 2015-06-05 13:12:25 +02:00
José Valim 4d8c6cbb57 Handle corner cases for small strings in rstrp 2015-06-02 10:13:45 +02:00
José Valim dbfbac4c86 Optimize rstrip
This new implementation is no longer linear without affecting
smaller samples. For a string that is 100 bytes long, it is
25x faster than the previous implementation.
2015-06-02 09:52:13 +02:00