Commit Graph
54 Commits
Author SHA1 Message Date
Aleksei Magusev 683a3ad3c5 Correct module name for grapheme tests 2017-02-23 23:58:20 +01:00
José Valim 636c49fb5c Add grapheme tests 2017-02-13 15:44:59 +01:00
José Valim 3e394e8ef0 Incorporate new grapheme rules in Unicode 9 2017-02-13 14:34:04 +01:00
Andrea Leopardi 3604554aea Remove leftover trailing whitespace from the code base
[ci skip]
2017-01-13 15:01:47 +01:00
José Valim bd7b4847bd Speed up String.split/1 2016-11-21 19:03:16 +01:00
José Valim 1606c40453 Update to Unicode 9.0.0
The update instructions have also been outlined
on top of the unicode.ex file.

Closes #4864.
2016-11-20 00:13:12 +01:00
Miles Starkenburg 551a9ac14e Fix nfd normalization bug (#4606)
* Fix nfd normalization bug

* Change String.normalize return type to binary
2016-05-13 09:28:50 +02:00
eksperimental 6da34e0225 Standardize unicode and friends (#4569)
* Unicode is a proper noun, so it should be capitalized

* Replace 'char data' with 'chardata'

* Correct use of a/an articles
2016-05-05 09:21:05 +02:00
Aleksei Magusev 7e20b6ef99 Merge pull request #4535 from lexmag/string-trim-pad
Introduce String.pad_{leading,trailing}/3 and String.trim{,_leading,_trailing}/2
2016-04-25 20:02:19 +02:00
Aleksei Magusev 24ab310950 Introduce String.trim{,_leading,_trailing}/2 2016-04-25 19:59:24 +02:00
eksperimental 4ceb41e71b Formmating: Add white space around vertical bar (#4507) 2016-04-25 00:55:49 +02:00
José Valim 8fe87ffa3c Fold decomposition recursions at compile time
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 16:55:29 +02:00
José Valim 79b132a665 Do not reorder starting classes
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 13:38:14 +02:00
José Valim cd953f5883 Handle compositios with non-zero combining class
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 13:38:10 +02:00
José Valim 107a185209 More optimizations and improvements to normalization
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 13:38:07 +02:00
José Valim c07f2b2820 Make decomposition recursive and consider exclusion list
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 13:38:00 +02:00
José Valim ec027fa421 Use combining_classes from UnicodeData
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-03-29 13:37:56 +02:00
José Valim 664feb1802 Rely only on UnicodeData for composition/decomposition 2016-03-29 00:54:20 +02:00
José Valim 8199b81f7c Clean up unicode range parsing 2016-03-18 16:59:13 +01:00
José Valim e1a1065ee9 Update Unicode to 8.0.0 (equivalent pending) 2016-03-15 15:58:01 +01:00
José Valim 646ae4d63f Avoid regular expressions 2016-03-15 15:43:32 +01:00
Abel Muiño 98a4ee5da6 New definition of whitespace & breakable whitespace
All whitespace can be removed by `strip` but only breakable whitespace
can be used as a delimiter by `split`, with updated docs for String.split/1
2016-03-15 14:54:11 +01:00
José Valim ea4102355c Remove debug_info from unicode 2016-02-01 09:21:49 +01:00
José Valim e65b01b870 Do not include debug_info in metadata String modules 2016-01-08 10:11:07 +01:00
Aleksei Magusev 40096e0ff1 Eliminate spaces in bitstring definitions 2015-11-28 23:48:48 +01:00
José Valim 94a07df8d3 Remove uses of soon to be deprecated Dict 2015-10-23 15:04:24 +02:00
Bryan Endersstocker 07fedf66d0 Add NFC support to String.normalize/2 2015-10-03 11:08:55 -04:00
Bryan Endersstocker 945f8b6466 Optimize Unicode normalization
Also extracted String.Normalizer module from String.Unicode.
2015-09-26 10:59:04 -04:00
Bryan Endersstocker bf02047280 Add String.normalize/2 2015-09-26 00:12:44 -04:00
Bryan Enders dab2a632b6 Rename String.is_equivalent/2 to String.equivalent?/2 2015-09-25 15:01:29 -04:00
Bryan Enders f83cc7de5f Add String.is_equivalent/2 to test canonical Unicode equivalence 2015-09-25 13:35:16 -04:00
José Valim ff51926fe8 Move away from HashDict and HashSet 2015-09-25 12:46:22 +02:00
eksperimental a251ccc722 Format raise message consistently
Kernel.raise messages should not start with uppercase, and should not
have trailing punctuation.

Mix.raise messages should start with uppercase, and should not have a trailing
puntuation.

    # starting with uppercase
    # exclude (Mix.raise, and any message that starts with all uppecase such as: I or IO
    ag '(?<!Mix\.)raise\s+\"+(?!([A-Z]+\b))[A-Z]'

    # Mix.raise should start with upper case
    # except when `mix` or `rebar` are mentioned
    ag 'Mix\.raise\s+\"(?!(mix|rebar))+[a-z]'

    # ending in period
    ag "raise\s+\"+.+\.\"\s*\n"
2015-09-08 00:44:17 +07:00
José Valim 0e7232671b Optimize many functions in String
Many functions relied on next_grapheme, which would traverse
the whole string generating a lot of graphemes as garbage along
the way.

This commit introduces next_grapheme_size, which returns the
size in bytes of the next grapheme, avoiding the generation
of the garbage and optimizing functions that need just the
byte_size.

Functions optimized are: String.at, String.split_at, String.slice
and String.length.
2015-08-02 13:49:50 +02:00
José Valim a52bf5f40e Speed up upcase and downcase for large strings 2015-06-05 13:12:25 +02:00
José Valim 6efdf35f41 Handle corner cases for small strings in rstrp 2015-06-02 10:13:45 +02:00
José Valim 8574c7b568 Optimize rstrip
This new implementation is no longer linear without affecting
smaller samples. For a string that is 100 bytes long, it is
25x faster than the previous implementation.
2015-06-02 09:52:13 +02:00
eksperimental 50ab29ffb1 fix typo in previous commit. bodyless defmacro added to show proper arg in b/1, t/1 2015-03-28 19:19:37 +07:00
eksperimental 3592151b3e rename function arg. when "other" was used 2015-03-27 02:37:36 +07:00
jw2013 f2863de304 fix wrong CRLF grapheme 2014-11-08 12:45:41 -08:00
José Valim 9717a63604 Do not consider subpatterns on Regex.split/3 2014-07-30 11:13:18 +02:00
José Valim 182cbfe541 Update to Unicode 7.0.0 2014-06-17 17:17:32 +02:00
José Valim 1038018e84 Migrate from size to byte_size and tuple_size 2014-06-06 13:33:02 +02:00
José Valim 540c763c85 Move binary and list conversions to String and List 2014-05-17 17:02:46 +02:00
José Valim 3bb8e17e9e Add iodata_to_binary and chardata_to_string to IO 2014-05-17 13:06:40 +02:00
José Valim 56f15d9f4e Do not add spaces after { and before }
This makes the source code consistent with the result
returned by inspect/2.
2014-04-21 19:06:35 +02:00
José Valim 024edf3d40 Rename iolist_* to iodata_* 2014-02-23 11:32:03 +01:00
José Valim 3512c1a860 Start migrating lc to for comprehensions 2014-02-17 14:45:04 +01:00
José Valim ba54afbf45 Move to new sigils syntax 2014-02-05 14:39:04 +01:00
José Valim 0802014779 No longer compile unicode in parallel
Compiling in parallel only bought us 2s (out of 24s) since
compiling graphemes is what takes most of the time but it
considerably increases memory usage, by about 30%, since both
compilation targets are loaded into memory at once.
2014-01-22 17:51:24 +01:00