Commit Graph
58 Commits
Author SHA1 Message Date
Guilherme Pasqualino 5c21265cc8 Run code formatter on lib/elixir/unicode/unicode.ex (#6795) 2017-10-10 12:24:15 +02:00
Michał Muskała dae60dbea4 Add is_binary guard to String.length (#6430)
This produces a bit more readable message if not a string is passed into `String.length`, then failing in the `next_grapheme_size` function.
2017-08-03 21:04:12 +02:00
José Valim e8690e412a Update to Unicode 10 (#6293) 2017-07-04 09:57:07 +02:00
José Valim 2f663ce3a2 Add non-quoted Unicode atoms and variables (#6158)
It follows Unicode Annex #31.

See the Unicode Syntax document for a more in-depth reference.
2017-05-27 22:43:05 +02:00
José Valim 50d16830fd Properly slice titlecase_once binaries 2017-05-19 12:37:05 +02:00
José Valim de1b731afb Add grapheme tests 2017-02-13 15:44:59 +01:00
José Valim b1c567b5ef Incorporate new grapheme rules in Unicode 9 2017-02-13 14:34:04 +01:00
Andrea Leopardi 7222c1bce3 Remove leftover trailing whitespace from the code base
[ci skip]
2017-01-13 15:01:47 +01:00
José Valim cae33236ad Speed up String.split/1 2016-11-21 19:03:16 +01:00
José Valim 1b3efa9625 Update to Unicode 9.0.0
The update instructions have also been outlined
on top of the unicode.ex file.

Closes #4864.
2016-11-20 00:13:12 +01:00
Miles Starkenburg 52ecfb0754 Fix nfd normalization bug (#4606)
* Fix nfd normalization bug

* Change String.normalize return type to binary
2016-05-13 09:28:50 +02:00
eksperimental bc8c2a55e9 Standardize unicode and friends (#4569)
* Unicode is a proper noun, so it should be capitalized

* Replace 'char data' with 'chardata'

* Correct use of a/an articles
2016-05-05 09:21:05 +02:00
Aleksei Magusev 6c96174a0b Merge pull request #4535 from lexmag/string-trim-pad
Introduce String.pad_{leading,trailing}/3 and String.trim{,_leading,_trailing}/2
2016-04-25 20:02:19 +02:00
Aleksei Magusev 0ea04744bc Introduce String.trim{,_leading,_trailing}/2 2016-04-25 19:59:24 +02:00
eksperimental 685e87bdf0 Formmating: Add white space around vertical bar (#4507) 2016-04-25 00:55:49 +02:00
José Valim 7da99878cb Fold decomposition recursions at compile time
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 16:55:29 +02:00
José Valim 3693966251 Do not reorder starting classes
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 13:38:14 +02:00
José Valim d1b6a8d553 Handle compositios with non-zero combining class
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 13:38:10 +02:00
José Valim 0de30c7af5 More optimizations and improvements to normalization
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 13:38:07 +02:00
José Valim 196d225489 Make decomposition recursive and consider exclusion list
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 13:38:00 +02:00
José Valim a9e246ecc6 Use combining_classes from UnicodeData
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 13:37:56 +02:00
José Valim 60d7771f41 Rely only on UnicodeData for composition/decomposition 2016-03-29 00:54:20 +02:00
José Valim 0bf5293aab Clean up unicode range parsing 2016-03-18 16:59:13 +01:00
José Valim 1d5ca3beed Update Unicode to 8.0.0 (equivalent pending) 2016-03-15 15:58:01 +01:00
José Valim d3ff4691b8 Avoid regular expressions 2016-03-15 15:43:32 +01:00
Abel Muiño c465f80a6d New definition of whitespace & breakable whitespace
All whitespace can be removed by `strip` but only breakable whitespace
can be used as a delimiter by `split`, with updated docs for String.split/1
2016-03-15 14:54:11 +01:00
José Valim 5b77dd9a14 Remove debug_info from unicode 2016-02-01 09:21:49 +01:00
José Valim 0c4a1f189f Do not include debug_info in metadata String modules 2016-01-08 10:11:07 +01:00
Aleksei Magusev 612fcc4760 Eliminate spaces in bitstring definitions 2015-11-28 23:48:48 +01:00
José Valim cfc4bc20d1 Remove uses of soon to be deprecated Dict 2015-10-23 15:04:24 +02:00
Bryan Endersstocker e12adea1d4 Add NFC support to String.normalize/2 2015-10-03 11:08:55 -04:00
Bryan Endersstocker 180f3184c6 Optimize Unicode normalization
Also extracted String.Normalizer module from String.Unicode.
2015-09-26 10:59:04 -04:00
Bryan Endersstocker a65b8c27da Add String.normalize/2 2015-09-26 00:12:44 -04:00
Bryan Enders 4fb6446145 Rename String.is_equivalent/2 to String.equivalent?/2 2015-09-25 15:01:29 -04:00
Bryan Enders f01ad6df36 Add String.is_equivalent/2 to test canonical Unicode equivalence 2015-09-25 13:35:16 -04:00
José Valim 5ed087ef50 Move away from HashDict and HashSet 2015-09-25 12:46:22 +02:00
eksperimental 8787e7019e Format raise message consistently
Kernel.raise messages should not start with uppercase, and should not
have trailing punctuation.

Mix.raise messages should start with uppercase, and should not have a trailing
puntuation.

    # starting with uppercase
    # exclude (Mix.raise, and any message that starts with all uppecase such as: I or IO
    ag '(?<!Mix\.)raise\s+\"+(?!([A-Z]+\b))[A-Z]'

    # Mix.raise should start with upper case
    # except when `mix` or `rebar` are mentioned
    ag 'Mix\.raise\s+\"(?!(mix|rebar))+[a-z]'

    # ending in period
    ag "raise\s+\"+.+\.\"\s*\n"
2015-09-08 00:44:17 +07:00
José Valim 34bbb3d83e Optimize many functions in String
Many functions relied on next_grapheme, which would traverse
the whole string generating a lot of graphemes as garbage along
the way.

This commit introduces next_grapheme_size, which returns the
size in bytes of the next grapheme, avoiding the generation
of the garbage and optimizing functions that need just the
byte_size.

Functions optimized are: String.at, String.split_at, String.slice
and String.length.
2015-08-02 13:49:50 +02:00
José Valim 5dafe3ab7b Speed up upcase and downcase for large strings 2015-06-05 13:12:25 +02:00
José Valim 4d8c6cbb57 Handle corner cases for small strings in rstrp 2015-06-02 10:13:45 +02:00
José Valim dbfbac4c86 Optimize rstrip
This new implementation is no longer linear without affecting
smaller samples. For a string that is 100 bytes long, it is
25x faster than the previous implementation.
2015-06-02 09:52:13 +02:00
eksperimental 8cc6ed995f fix typo in previous commit. bodyless defmacro added to show proper arg in b/1, t/1 2015-03-28 19:19:37 +07:00
eksperimental fdc0221517 rename function arg. when "other" was used 2015-03-27 02:37:36 +07:00
jw2013 3dd9df184e fix wrong CRLF grapheme 2014-11-08 12:45:41 -08:00
José Valim 64b68edd45 Do not consider subpatterns on Regex.split/3 2014-07-30 11:13:18 +02:00
José Valim 182cbfe541 Update to Unicode 7.0.0 2014-06-17 17:17:32 +02:00
José Valim 1038018e84 Migrate from size to byte_size and tuple_size 2014-06-06 13:33:02 +02:00
José Valim 540c763c85 Move binary and list conversions to String and List 2014-05-17 17:02:46 +02:00
José Valim 3bb8e17e9e Add iodata_to_binary and chardata_to_string to IO 2014-05-17 13:06:40 +02:00
José Valim 56f15d9f4e Do not add spaces after { and before }
This makes the source code consistent with the result
returned by inspect/2.
2014-04-21 19:06:35 +02:00