Commit Graph
185 Commits
Author SHA1 Message Date
José Valim f38895cebb Add literal encoder option to Code.string_to_quoted 2019-06-19 00:04:08 +02:00
José Valim 0229358f88 Ensure third element in location tuple is either nil or an integer 2019-05-09 10:59:13 +02:00
Tomáš Janoušek 8d079862dc Fix columns in tokenize of dot_call_op (#8995)
handle_dot already gets the column of `(` from strip_dot_space.
2019-04-29 23:34:08 +02:00
Eksperimental 99d919e0c8 Use "code point" instead of "codepoint" (#8774)
That's the term it is used in the Unicode standard.
2019-02-06 11:21:03 +01:00
Glauber Campinho 3fcc9241db Optimizes multiple :maps put and merge operations (#8223)
Optimizes :maps.put/3 to a VM operation instead of a remote call, including nested calls to it.
It does the same for :maps.merge/2 with a simple map on the right side.

Closes #7353
2018-09-28 08:40:27 +02:00
Andrea Leopardi 9e302cbd01 Forbid using only the "else" clause in "try" (#8219)
Closes #8146.

Now using "else" without a "rescue", "catch", or "after" in "try" 
expressions is not allowed anymore.
2018-09-23 12:09:47 +02:00
José Valim beb596d37c Warn on unnecessary quotes (#7838) 2018-07-10 10:27:02 +02:00
José Valim 9bf4826210 Fix erlang-based tests 2018-07-09 11:22:23 +02:00
Michał Muskała d225f4a373 Restore and/or optimisation (#7764)
The optimize_boolean option is very sensitive to the order of
clauses and the optimisation was not happening for and/2 and or/2.

Add regression tests to ensure that won't happen again.

Removes tests that dialyzer does not emit warnings for optimised
and and or - those warnings could actually be helpful.
2018-06-23 09:21:15 +02:00
José Valim 5edb1d2739 Add hints on mismatched do/end and others pairs
When you are working on large files and you add new code,
you may forget to add a `do` or an `end`. In those cases,
the error message usually points to the first `do` or the
last `end` in the file, which are usually far away from
the source of the error.

This pull request adds a simple heuristic based on the
indentation of the tokens, to try to provide hints of
where the source may be. Those hints are not deterministic
but they may be able to point users to the source of the
problem.

For example, in this case:

    defmodule MyApp do
      def one do
      # end

      def two do
      end
    end

we know that we now that `def two do` is happening on the
same indentation as `def one do`, which may mean that
`def one do` was not closed properly. We store this as a
hint in case the terminators do not match later.

Similarly, in the case below:

    defmodule MyApp do
      def one
      end

      def two do
      end
    end

The `end` on line 3 will end-up closing the defmodule `do`,
on line 1. Because their indentation do not match, it may
be that there is a missing `do`, where the `end` was supposed
to align.

Some basic testing show those heuristics work on the majority
of the cases, but we will only be sure when we have enough
feedback from the community.
2018-06-10 13:55:49 +02:00
René Föhring 1ade3c0dd9 Fix column in :elixir_tokenizer for double quotes in keyword identifiers (#7722)
This fixes cases where `:elixir_tokenizer` does not account for double
quotes in keyword identifiers when determining the column for the next
token.
2018-06-04 16:06:42 +02:00
René Föhring 901752a4fe Fix column in :elixir_tokenizer for sigils with modifiers (#7529)
This fixes cases where :elixir_tokenizer does not account for sigil modifiers
when determining the column for the next token.
2018-04-03 13:16:02 +02:00
Glauber Campinho add3a9666f Fix column count on heredoc tokenizer (#7486) 2018-03-25 12:37:15 +02:00
José Valim 219c6cc867 Move unused_vars check to Elixir land
This gives us better control over when and how unused variables are printed.

As a result, we are able to emit unused variable warnings in situations
we could not before. This also opens up the way for us to remove a
dependency on erl_lint and track types information, which allows us to
speed up compilation times about 5% and allow us to emit more performant
code in some situations.
2018-01-20 19:45:48 +01:00
José Valim c270e7211f Remove deprecation warnings 2017-12-26 23:18:18 +01:00
José Valim d3a85bfd21 Remove end column information from tokens
If we want to store this information, we need to store it
alongside the end line, otherwise it is ambiguous.
2017-10-25 11:56:31 +02:00
José Valim 1ac735d9ba Fix tokenizer tests 2017-10-16 17:11:23 +02:00
José Valim 34651f1034 Format errors_test.exs 2017-10-12 09:35:58 +02:00
José Valim 82e93512e9 Do not raise on clauses with newlines before and after -> 2017-10-07 10:13:53 +02:00
José Valim 4387576126 Add more metadata to containers 2017-10-07 00:18:22 +02:00
José Valim a6ccda9370 Update tokenizer and interpolation tests 2017-10-05 19:29:47 +02:00
José Valim 455c2c3f3b Rely on side effects for preserving comments
The previous implementation would not lead to parsing
errors in interpolations since code inside interpolation
is parsed immediately.
2017-10-05 13:27:25 +02:00
Artur Cygan c0fd3d2655 Improve error message on oversized atom (#6612) 2017-09-28 14:04:29 +02:00
José Valim 93f79543ae Do not use __block__ and __aliases__ as literal identifiers 2017-09-01 01:45:08 +02:00
José Valim 97ece16b11 Add more formatter metadata 2017-08-30 17:21:17 +02:00
José Valim 6030be5ae5 Add literal annotation for aliases 2017-08-28 23:59:45 +02:00
José Valim 9a33445c6c Do not unescape interpolation 2017-08-28 14:17:12 +02:00
José Valim a378154652 Make terminator binaries 2017-08-27 12:39:40 +02:00
Andrea Leopardi 4cfce073c1 Keep the representation of "float" tokens like we do for "int" tokens (#6501) 2017-08-23 12:57:10 +02:00
Andrea Leopardi bd1508cc23 Fix integer token representation for decimal integers 2017-08-23 12:07:46 +02:00
Andrea Leopardi 2f9b651011 Merge integer tokens into the "int" token (#6499) 2017-08-23 11:55:12 +02:00
Andrea Leopardi 8cd9b8a75a Change the format of tokens (#6495)
Right now, tokens are "{Token, Location}" or "{Token, Location, Value}".
This commit changes "Location" from "{Line, StartColumn, EndColumn}" to
"{Line, {StartColumn, EndColumn}, Meta}" where "Meta" can be anything.
This will be used for things such as storing the format of integers.
2017-08-22 21:12:30 +02:00
Andrea Leopardi 1dd4c16958 Rename the "hexadecimal" token to "hex" 2017-08-22 00:07:03 +02:00
Alex Jiao c004527139 Augment integer literals with representation info (#6344) 2017-07-18 14:40:59 +02:00
Michał Muskała 92a3bf76a3 Use orelse instead of or when expanding in operator (#6314)
orelse may lead to better code in many occasions - Since we're comparing
to literals, there's no semantic difference.

This also means, the compiler is able to figure out more "will always match"
checks. For example in a situation like the following a warning will be emitted.
This pull request also fixes such warnings in the test suite.

    if true do
      IO.puts("true")
    end

One problem with using `orelse` is that it will short-circuit and not evaluate
some side-effecting things, for example:

    1 in [1, send(self(), :foo)]

To remedy this, we'll traverse the right side of `in` and extract any things
that are not simple literals to variables, so that the above example expands
to something like the following:

    var0 = send(seld(), :foo)
    var = 1
    var === 1 or var === var0

This guarantees all side-effects are performed and we still retain benefits of
the short-circuiting comparisons in the end.
2017-07-16 12:50:21 +02:00
Alex Jiao d74bd047df Add terminator info to sigil AST node metadata (#6305) 2017-07-08 01:05:37 +02:00
Alex Jiao 1652c1f852 Fix comment preservation bug (#6284) 2017-07-02 17:31:55 +02:00
Michał Muskała ceb29aec41 Fix binaries optimisation from #6240 (#6272) 2017-06-30 13:39:06 +02:00
Alex Jiao 8e9ea26ddf Add a preserve_comments option to the tokenizer (#6222) 2017-06-21 10:09:52 +02:00
José Valim 9873e4239f Add non-quoted Unicode atoms and variables (#6158)
It follows Unicode Annex #31.

See the Unicode Syntax document for a more in-depth reference.
2017-05-27 22:43:05 +02:00
Andrea Leopardi 7ebd30b431 Give a proper syntax error when invalid sigil delimiters are used (#6133)
Before this commit, we would have:

    iex> ~s\
    ** (SyntaxError) iex:1: unexpected token: "~" (column 1, codepoint U+007E)

After this commit, we have:

    iex> ~s\
    ** (SyntaxError) iex:1: invalid sigil delimiter: "\" (column 3, codepoint U+005C)
2017-05-23 00:58:38 +02:00
José Valim 9747e58db7 Fix precedence of unary operators and operator capture 2017-05-14 19:31:50 +02:00
Gal Tsubery c6766f4b2b Add float overflow error message to Tokenizer (#5893) 2017-03-16 17:20:56 +00:00
Eric Entin 21c0e80076 Support "left not in right" syntax (#5620) 2017-01-07 10:23:13 +01:00
José Valim 6b8db410b3 Raise BadBooleanError on and/or, closes #5297 2016-11-20 13:30:50 +01:00
Michał Muskała 0aa8f16077 Make tokenizer recognize VC merge conflict marker (#5416)
The "<<<<<<<" is used by all major VC systems including git,
subversion and mercurial.
2016-11-06 17:55:59 +01:00
Mike Stok 05431f6010 keep tokenizer's column counts in sync for numbers with _ characters
For providing better feedback in credo and tools like it we should be
able to map a token back to its original source.  This makes sure that
`_` characters in numbers are properly accounted for so that they stay
in sync after we've encountered something like 123_456_789.

Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-06-30 10:25:43 +01:00
José Valim 1aca21c876 Ensure variables in function clauses are not reused, closes #4815
Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2016-06-15 16:14:48 +02:00
eksperimental 4f31dda2e2 Remove spaces between "fn" and "(" (#4747) 2016-06-01 22:41:46 +02:00
eksperimental 0aece49e2d Replace 'char_list' and 'char list' with 'charlist' 2016-05-07 03:20:57 +07:00