Release v1.16.3

The exclamation mark was missing in the result of the concatenation.

.formatter.exs

@@ -13,7 +13,10 @@
     assert_same: 2,
 
     # Errors tests
-    assert_eval_raise: 3
+    assert_eval_raise: 3,
+
+    # Float tests
+    float_assert: 1
   ],
   normalize_bitstring_modifiers: false
 ]

.github/workflows/ci.yml

@@ -3,10 +3,10 @@ name: CI
 on:
   push:
     paths-ignore:
-      - 'lib/**/*.md'
+      - "lib/**/*.md"
   pull_request:
     paths-ignore:
-      - 'lib/**/*.md'
+      - "lib/**/*.md"
 
 env:
   ELIXIR_ASSERT_TIMEOUT: 2000
@@ -24,12 +24,12 @@ jobs:
       fail-fast: false
       matrix:
         include:
-          - otp_version: '26.0'
+          - otp_version: "26.0"
             otp_latest: true
-          - otp_version: '25.3'
+          - otp_version: "25.3"
-          - otp_version: '25.0'
+          - otp_version: "25.0"
-          - otp_version: '24.3'
+          - otp_version: "24.3"
-          - otp_version: '24.0'
+          - otp_version: "24.0"
           - otp_version: master
             development: true
           - otp_version: maint
@@ -77,7 +77,7 @@ jobs:
     name: Windows Server 2019, Erlang/OTP ${{ matrix.otp_version }}
     strategy:
       matrix:
-        otp_version: ['24', '25', '26']
+        otp_version: ["24", "25", "26.0"]
     runs-on: windows-2019
     steps:
       - name: Configure Git

CHANGELOG.md

@@ -29,6 +29,21 @@ For mismatched delimiters, it now shows both delimiters:
     └─ lib/my_app.ex:1:18
 ```
 
+For unclosed delimiters, it now shows where the unclosed delimiter starts:
+
+```
+** (TokenMissingError) token missing on lib/my_app:8:23:
+    error: missing terminator: )
+    │
+  1 │ my_numbers = (1, 2, 3, 4, 5, 6
+    │              └ unclosed delimiter
+ ...
+  8 │ IO.inspect(my_numbers)
+    │                       └ missing closing delimiter (expected ")")
+    │
+    └─ lib/my_app:8:23
+```
+
 Errors and warnings diagnostics also include code snippets. When possible, we will show precise spans, such as on undefined variables:
 
 ```
@@ -51,19 +66,91 @@ error: function names should start with lowercase characters or underscore, inva
   └─ lib/sample.ex:3
 ```
 
-A huge thank you to Vinícius Muller for working on the new diagnostics.
+A huge thank you to Vinícius Müller for working on the new diagnostics.
 
 ## Revamped documentation
 
-Elixir's Getting Started guided has been made part of the Elixir repository and incorporated into ExDoc. This was an opportunity to revisit and unify all official guides and references.
+Elixir's Getting Started guides have been made part of the Elixir repository and incorporated into ExDoc. This was an opportunity to revisit and unify all official guides and references.
 
-We have also incorporated and extended the work on [Understanding Code Smells in Elixir Functional Language](https://github.com/lucasvegi/Elixir-Code-Smells/blob/main/etc/2023-emse-code-smells-elixir.pdf), by Lucas Vegi and Marco Tulio Valente, from [ASERG/DCC/UFMG](http://aserg.labsoft.dcc.ufmg.br/), into the official document in the form of anti-patterns. The anti-patterns are divided into four categories: code-related, design-related, process-related, and meta-programming. Our goal is to give all developers with both positive and negative examples of Elixir code, with context and examples on how to improve their codebases.
+We have also incorporated and extended the work on [Understanding Code Smells in Elixir Functional Language](https://github.com/lucasvegi/Elixir-Code-Smells/blob/main/etc/2023-emse-code-smells-elixir.pdf), by Lucas Vegi and Marco Tulio Valente, from [ASERG/DCC/UFMG](http://aserg.labsoft.dcc.ufmg.br/), into the official document in the form of anti-patterns. The anti-patterns are divided into four categories: code-related, design-related, process-related, and meta-programming. Our goal is to give all developers examples of potential anti-patterns, with context and examples on how to improve their codebases.
 
 Another [ExDoc](https://github.com/elixir-lang/ex_doc) feature we have incorporated in this release is the addition of cheatsheets, starting with [a cheatsheet for the Enum module](https://hexdocs.pm/elixir/main/enum-cheat.html). If you would like to contribute future cheatsheets to Elixir itself, feel free to start a discussion with an issue.
 
 Finally, we have started enriching our documentation with [Mermaid.js](https://mermaid.js.org/) diagrams. You can find examples in the [GenServer](https://hexdocs.pm/elixir/main/GenServer.html) and [Supervisor](https://hexdocs.pm/elixir/main/Supervisor.html) docs.
 
-## v1.16.0-dev
+## v1.16.3 (2024-05-21)
+
+### 1. Bug fixes
+
+#### Elixir
+
+  * [bin/elixir] Properly handle the `--dbg` flag in Elixir's CLI
+  * [Code.Formatter] Add brackets around keyword lists when formatting the left-hand side of `when`
+  * [Kernel] Only infer size in pinned variable in binary strings when needed
+  * [System] Add a note that arguments are unsafe when invoking .bat/.com scripts on Windows via `System.cmd/3`
+  * [Port] Add a note that arguments are unsafe when invoking .bat/.com scripts on Windows
+  * [URI] Ensure `:undefined` fields are properly converted to `nil` when invoking Erlang's API
+
+#### Logger
+
+  * [Logger] Ensure translators are persisted across logger restarts
+
+#### Mix
+
+  * [mix compile] Ensure compile paths are accessible during compilation
+
+## v1.16.2 (2024-03-10)
+
+### 1. Enhancements
+
+#### Elixir
+
+  * [Code] Emit `:defmodule` tracing event on module definition
+
+#### Mix
+
+  * [Mix] Add `Mix.install_project_dir/0`
+  * [Mix] Add environment variable for reusing `Mix.install/2` installation
+  * [Mix.SCM] Add `Mix.SCM.delete/1`
+
+### 2. Bug fixes
+
+#### Elixir
+
+  * [Code] Fix charlist formatting issue when a single-quoted charlist escapes a double-quote character
+  * [Path] Fix regression on how `Path.relative_to/2` dealt with "." as input
+
+#### IEx
+
+  * [IEx.Helpers] Discard mermaid fenced blocks from ansi docs
+
+#### ExUnit
+
+  * [ExUnit] Properly compared pinned values when building diffs
+
+## v1.16.1 (2024-01-31)
+
+### 1. Bug fixes
+
+#### Elixir
+
+  * [Code] Fix `Code.quoted_to_algebra/2` for operator with :do key as operand
+  * [Kernel.ParallelCompiler] Do not crash parallel compiler when it receives diagnostics from additional code evaluation
+  * [Kernel.ParallelCompiler] Always log errors at the end of compilation
+  * [String] Fix `String.capitalize/1` with a single codepoint
+
+#### IEx
+
+  * [IEx] Fix autocompletion of function signatures on Erlang/OTP 26
+  * [IEx] Do not assume `$HOME` is set
+
+#### Mix
+
+  * [mix deps.compile] Handle compilation of rebar3 dependencies when rebar3 is on a path with spaces on Unix
+  * [mix test] Properly resolve relative paths when running tests from individual files
+  * [mix test] Properly resolve Windows paths when running tests from individual files
+
+## v1.16.0 (2023-12-22)
 
 ### 1. Enhancements
 
@@ -73,9 +160,14 @@ Finally, we have started enriching our documentation with [Mermaid.js](https://m
 
 #### Elixir
 
+  * [Code] Add `:emit_warnings` for `Code.string_to_quoted/2`
   * [Code] Automatically include columns in parsing options
   * [Code] Introduce `MismatchedDelimiterError` for handling mismatched delimiter exceptions
   * [Code.Fragment] Handle anonymous calls in fragments
+  * [Code.Formatter] Trim trailing whitespace on heredocs with `\r\n`
+  * [File] Add `:offset` option to `File.stream!/2`
+  * [Kernel] Auto infer size of matched variable in bitstrings
+  * [Kernel] Preserve column information when translating typespecs
   * [Kernel] Suggest module names based on suffix and casing errors when the module does not exist in `UndefinedFunctionError`
   * [Kernel.ParallelCompiler] Introduce `Kernel.ParallelCompiler.pmap/2` to compile multiple additional entries in parallel
   * [Kernel.SpecialForms] Warn if `True`/`False`/`Nil` are used as aliases and there is no such alias
@@ -83,34 +175,57 @@ Finally, we have started enriching our documentation with [Mermaid.js](https://m
   * [Module] Add support for `@nifs` annotation from Erlang/OTP 25
   * [Module] Add support for missing `@dialyzer` configuration
   * [String] Update to Unicode 15.1.0
+  * [String] Add `String.replace_invalid/2`
   * [Task] Add `:limit` option to `Task.yield_many/2`
 
+#### Logger
+
+  * [Logger] Add `Logger.levels/0`
+
 #### Mix
 
   * [mix] Add `MIX_PROFILE` to profile a list of comma separated tasks
+  * [mix archive.install] Support `--sparse` option
+  * [mix compile.app] Warn if both `:applications` and `:extra_applications` are used
+  * [mix compile.elixir] Pass original exception down to diagnostic `:details` when possible
   * [mix compile.elixir] Optimize scenario where there are thousands of files in `lib/` and one of them is changed
+  * [mix deps.clean] Emit a warning instead of crashing when a dependency cannot be removed
+  * [mix escript.build] Escripts now strip .beam files by default, which leads to smaller escripts. However, if you are using escripts to access Elixir docs or compile Elixir code, documentation and deprecation metadata is no longer available. Set `strip_beams: false` in your escript configuration in your `mix.exs` to keep all metadata
+  * [mix escript.install] Support `--sparse` option
+  * [mix release] Include `include/` directory in releases
   * [mix test] Allow testing multiple file:line at once, such as `mix test test/foo_test.exs:13 test/bar_test.exs:27`
 
 ### 2. Bug fixes
 
 #### Elixir
 
+  * [Code] Keep quotes for atom keys in formatter
   * [Code.Fragment] Fix crash in `Code.Fragment.surround_context/2` when matching on `->`
   * [IO] Raise when using `IO.binwrite/2` on terminated device (mirroring `IO.write/2`)
   * [Kernel] Do not expand aliases recursively (the alias stored in Macro.Env is already expanded)
   * [Kernel] Ensure `dbg` module is a compile-time dependency
   * [Kernel] Warn when a private function or macro uses `unquote/1` and the function/macro itself is unused
+  * [Kernel] Re-enabled compiler optimizations for top level functions in scripts (disabled in v1.14.0 but shouldn't impact most programs)
   * [Kernel] Do not define an alias for nested modules starting with `Elixir.` in their definition
   * [Kernel.ParallelCompiler] Consider a module has been defined in `@after_compile` callbacks to avoid deadlocks
+  * [Macro] Address exception on `Macro.to_string/1` for certain ASTs
+  * [Path] Lazily evaluate `File.cwd!/0` in `Path.expand/1` and `Path.absname/1`
   * [Path] Ensure `Path.relative_to/2` returns a relative path when the given argument does not share a common prefix with `cwd`
 
 #### ExUnit
 
   * [ExUnit] Raise on incorrectly dedented doctests
 
+#### IEx
+
+  * [IEx.Pry] Fix prying functions with only literals in their body
+
 #### Mix
 
-  * [Mix] Ensure files with duplicate modules are recompiled whenever any of the files change
+  * [mix archive.install] Restore code paths after `mix archive.install`
+  * [mix compile] Ensure files with duplicate modules are recompiled whenever any of the files change
+  * [mix compile] Update Mix compiler diagnostics documentation and typespecs to match the Elixir compiler behaviour where both lines and columns start from one (before it inaccurately said that columns started from zero)
+  * [mix escript.install] Restore code paths after `mix escript.install`
 
 ### 3. Soft deprecations (no warnings emitted)
 
@@ -120,6 +235,10 @@ Finally, we have started enriching our documentation with [Mermaid.js](https://m
   * [Kernel.ParallelCompiler] Deprecate `Kernel.ParallelCompiler.async/1` in favor of `Kernel.ParallelCompiler.pmap/2`
   * [Path] Deprecate `Path.safe_relative_to/2` in favor of `Path.safe_relative/2`
 
+#### Mix
+
+  * [mix compile] Returning a four-element tuple as a position in `Mix.Task.Compiler.Diagnostic`
+
 ### 4. Hard deprecations
 
 #### Elixir

Makefile

@@ -2,7 +2,7 @@ PREFIX ?= /usr/local
 TEST_FILES ?= "*_test.exs"
 SHARE_PREFIX ?= $(PREFIX)/share
 MAN_PREFIX ?= $(SHARE_PREFIX)/man
-CANONICAL := main/
+# CANONICAL := main/
 ELIXIRC := bin/elixirc --ignore-module-conflict $(ELIXIRC_OPTS)
 ERLC := erlc -I lib/elixir/include
 ERL_MAKE := if [ -n "$(ERLC_OPTS)" ]; then ERL_COMPILER_OPTIONS=$(ERLC_OPTS) erl -make; else erl -make; fi
@@ -189,7 +189,6 @@ docs_elixir: compile ../ex_doc/bin/ex_doc
 	$(Q) rm -rf doc/elixir
 	$(call DOCS_COMPILE,Elixir,elixir,Kernel,--config "lib/elixir/scripts/elixir_docs.exs")
 	$(call DOCS_CONFIG,elixir)
-	cp -R lib/elixir/pages/images doc/elixir
 
 docs_eex: compile ../ex_doc/bin/ex_doc
 	@ echo "==> ex_doc (eex)"

SECURITY.md

@@ -6,12 +6,11 @@ Elixir applies bug fixes only to the latest minor branch. Security patches are a
 
 Elixir version | Support
 :------------- | :-----------------------------
-1.16           | Development
+1.16           | Bug fixes and security patches
-1.15           | Bug fixes and security patches
+1.15           | Security patches only
 1.14           | Security patches only
 1.13           | Security patches only
 1.12           | Security patches only
-1.11           | Security patches only
 
 ## Announcements
 

VERSION

@@ -1 +1 @@
-1.16.0-dev
+1.16.3

bin/elixir

@@ -1,7 +1,7 @@
 #!/bin/sh
 set -e
 
-ELIXIR_VERSION=1.16.0-dev
+ELIXIR_VERSION=1.16.3
 
 if [ $# -eq 0 ] || { [ $# -eq 1 ] && { [ "$1" = "--help" ] || [ "$1" = "-h" ]; }; }; then
   cat <<USAGE >&2
@@ -112,10 +112,10 @@ while [ $I -le $LENGTH ]; do
         C=1
         MODE="iex"
         ;;
-    -v|--no-halt|--dbg)
+    -v|--no-halt)
         C=1
         ;;
-    -e|-r|-pr|-pa|-pz|--eval|--remsh|--dot-iex)
+    -e|-r|-pr|-pa|-pz|--eval|--remsh|--dot-iex|--dbg)
         C=2
         ;;
     --rpc-eval)

bin/elixir.bat

@@ -1,6 +1,6 @@
 @if defined ELIXIR_CLI_ECHO (@echo on) else (@echo off)
 
-set ELIXIR_VERSION=1.16.0-dev
+set ELIXIR_VERSION=1.16.3
 
 setlocal enabledelayedexpansion
 if    ""%1""==""""                if ""%2""=="""" goto documentation
@@ -140,16 +140,16 @@ if ""==!par:--remsh=!     (set "parsElixir=!parsElixir! --remsh %~1" && shift &&
 if ""==!par:--dot-iex=!   (set "parsElixir=!parsElixir! --dot-iex %~1" && shift && goto startloop)
 if ""==!par:--dbg=!       (set "parsElixir=!parsElixir! --dbg %~1" && shift && goto startloop)
 rem ******* ERLANG PARAMETERS **********************
-if ""==!par:--boot=!                (set "parsErlang=!parsErlang! -boot %~1" && shift && goto startloop)
+if ""==!par:--boot=!                (set "parsErlang=!parsErlang! -boot "%~1"" && shift && goto startloop)
-if ""==!par:--boot-var=!            (set "parsErlang=!parsErlang! -boot_var %~1 %~2" && shift && shift && goto startloop)
+if ""==!par:--boot-var=!            (set "parsErlang=!parsErlang! -boot_var "%~1" "%~2"" && shift && shift && goto startloop)
-if ""==!par:--cookie=!              (set "parsErlang=!parsErlang! -setcookie %~1" && shift && goto startloop)
+if ""==!par:--cookie=!              (set "parsErlang=!parsErlang! -setcookie "%~1"" && shift && goto startloop)
 if ""==!par:--hidden=!              (set "parsErlang=!parsErlang! -hidden" && goto startloop)
-if ""==!par:--erl-config=!          (set "parsErlang=!parsErlang! -config %~1" && shift && goto startloop)
+if ""==!par:--erl-config=!          (set "parsErlang=!parsErlang! -config "%~1"" && shift && goto startloop)
 if ""==!par:--logger-otp-reports=!  (set "parsErlang=!parsErlang! -logger handle_otp_reports %1" && shift && goto startloop)
 if ""==!par:--logger-sasl-reports=! (set "parsErlang=!parsErlang! -logger handle_sasl_reports %1" && shift && goto startloop)
-if ""==!par:--name=!                (set "parsErlang=!parsErlang! -name %~1" && shift && goto startloop)
+if ""==!par:--name=!                (set "parsErlang=!parsErlang! -name "%~1"" && shift && goto startloop)
-if ""==!par:--sname=!               (set "parsErlang=!parsErlang! -sname %~1" && shift && goto startloop)
+if ""==!par:--sname=!               (set "parsErlang=!parsErlang! -sname "%~1"" && shift && goto startloop)
-if ""==!par:--vm-args=!             (set "parsErlang=!parsErlang! -args_file %~1" && shift && goto startloop)
+if ""==!par:--vm-args=!             (set "parsErlang=!parsErlang! -args_file "%~1"" && shift && goto startloop)
 if ""==!par:--erl=!                 (set "beforeExtra=!beforeExtra! %~1" && shift && goto startloop)
 if ""==!par:--pipe-to=!             (echo --pipe-to : Option is not supported on Windows && goto end)
 set endLoop=1

lib/eex/test/eex_test.exs

@@ -761,38 +761,31 @@ defmodule EExTest do
     end
 
     test "line and column meta" do
-      parser_options = Code.get_compiler_option(:parser_options)
+      indentation = 12
-      Code.put_compiler_option(:parser_options, columns: true)
 
-      try do
+      ast =
-        indentation = 12
+        EEx.compile_string(
+          """
+          <%= f() %> <% f() %>
+            <%= f fn -> %>
+              <%= f() %>
+            <% end %>
+          """,
+          indentation: indentation
+        )
 
-        ast =
+      {_, calls} =
-          EEx.compile_string(
+        Macro.prewalk(ast, [], fn
-            """
+          {:f, meta, _args} = expr, acc -> {expr, [meta | acc]}
-            <%= f() %> <% f() %>
+          other, acc -> {other, acc}
-              <%= f fn -> %>
+        end)
-                <%= f() %>
-              <% end %>
-            """,
-            indentation: indentation
-          )
 
-        {_, calls} =
+      assert Enum.reverse(calls) == [
-          Macro.prewalk(ast, [], fn
+               [line: 1, column: indentation + 5],
-            {:f, meta, _args} = expr, acc -> {expr, [meta | acc]}
+               [line: 1, column: indentation + 15],
-            other, acc -> {other, acc}
+               [line: 2, column: indentation + 7],
-          end)
+               [line: 3, column: indentation + 9]
-
+             ]
-        assert Enum.reverse(calls) == [
-                 [line: 1, column: indentation + 5],
-                 [line: 1, column: indentation + 15],
-                 [line: 2, column: indentation + 7],
-                 [line: 3, column: indentation + 9]
-               ]
-      after
-        Code.put_compiler_option(:parser_options, parser_options)
-      end
     end
   end
 

lib/elixir/lib/code.ex

@@ -158,6 +158,10 @@ defmodule Code do
       of keys to traverse in the application environment and `return` is either
       `{:ok, value}` or `:error`.
 
+    * `:defmodule` - (since v1.16.2) traced as soon as the definition of a module
+      starts. This is invoked early on in the module life-cycle, `Module.open?/1`
+      still returns `false` for such traces
+
     * `{:on_module, bytecode, _ignore}` - (since v1.13.0) traced whenever a module
       is defined. This is equivalent to the `@after_compile` callback and invoked
       after any `@after_compile` in the given module. The third element is currently
@@ -196,20 +200,44 @@ defmodule Code do
 
   @typedoc """
   Diagnostics returned by the compiler and code evaluation.
+
+  The file and position relate to where the diagnostic should be shown.
+  If there is a file and position, then the diagnostic is precise
+  and you can use the given file and position for generating snippets,
+  IDEs annotations, and so on. An optional span is available with
+  the line and column the diagnostic ends.
+
+  Otherwise, a stacktrace may be given, which you can place your own
+  heuristics to provide better reporting.
+
+  The source field points to the source file the compiler tracked
+  the error to. For example, a file `lib/foo.ex` may embed `.eex`
+  templates from `lib/foo/bar.eex`. A syntax error on the EEx template
+  will point to file `lib/foo/bar.eex` but the source is `lib/foo.ex`.
   """
   @type diagnostic(severity) :: %{
-          required(:file) => Path.t(),
+          required(:source) => Path.t() | nil,
+          required(:file) => Path.t() | nil,
           required(:severity) => severity,
           required(:message) => String.t(),
-          required(:position) => position,
+          required(:position) => position(),
           required(:stacktrace) => Exception.stacktrace(),
-          required(:span) => {non_neg_integer, non_neg_integer} | nil,
+          required(:span) => {line :: pos_integer(), column :: pos_integer()} | nil,
+          optional(:details) => term(),
           optional(any()) => any()
         }
 
   @typedoc "The line. 0 indicates no line."
   @type line() :: non_neg_integer()
-  @type position() :: line() | {pos_integer(), column :: non_neg_integer}
+
+  @typedoc """
+  The position of the diagnostic.
+
+  Can be either a line number or a `{line, column}`.
+  Line and columns numbers are one-based.
+  A position of `0` represents unknown.
+  """
+  @type position() :: line() | {line :: pos_integer(), column :: pos_integer()}
 
   @boolean_compiler_options [
     :docs,
@@ -962,10 +990,9 @@ defmodule Code do
     to_quoted_opts =
       [
         unescape: false,
-        warn_on_unnecessary_quotes: false,
         literal_encoder: &{:ok, {:__block__, &2, [&1]}},
         token_metadata: true,
-        warnings: false
+        emit_warnings: false
       ] ++ opts
 
     {forms, comments} = string_to_quoted_with_comments!(string, to_quoted_opts)
@@ -1126,9 +1153,8 @@ defmodule Code do
       atoms but `:existing_atoms_only` is still used for dynamic atoms,
       such as atoms with interpolations.
 
-    * `:warn_on_unnecessary_quotes` - when `false`, does not warn
+    * `:emit_warnings` (since v1.16.0) - when `false`, does not emit
-      when atoms, keywords or calls have unnecessary quotes on
+      tokenizing/parsing related warnings. Defaults to `true`.
-      them. Defaults to `true`.
 
   ## `Macro.to_string/2`
 
@@ -2057,7 +2083,8 @@ defmodule Code do
 
   defp get_beam_and_path(module) do
     with {^module, beam, filename} <- :code.get_object_code(module),
-         {:ok, ^module} <- beam |> :beam_lib.info() |> Keyword.fetch(:module) do
+         info_pairs when is_list(info_pairs) <- :beam_lib.info(beam),
+         {:ok, ^module} <- Keyword.fetch(info_pairs, :module) do
       {beam, filename}
     else
       _ -> :error

lib/elixir/lib/code/formatter.ex

@@ -6,7 +6,8 @@ defmodule Code.Formatter do
   @double_heredoc "\"\"\""
   @single_quote "'"
   @single_heredoc "'''"
-  @sigil_c "~c\""
+  @sigil_c_double "~c\""
+  @sigil_c_single "~c'"
   @sigil_c_heredoc "~c\"\"\""
   @newlines 2
   @min_line 0
@@ -299,7 +300,7 @@ defmodule Code.Formatter do
         remote_to_algebra(quoted, context, state)
 
       meta[:delimiter] == ~s['''] ->
-        {opener, quotes} = get_charlist_quotes(true, state)
+        {opener, quotes} = get_charlist_quotes(:heredoc, state)
 
         {doc, state} =
           entries
@@ -309,7 +310,7 @@ defmodule Code.Formatter do
         {force_unfit(doc), state}
 
       true ->
-        {opener, quotes} = get_charlist_quotes(false, state)
+        {opener, quotes} = get_charlist_quotes({:regular, entries}, state)
         list_interpolation_to_algebra(entries, quotes, state, opener, quotes)
     end
   end
@@ -368,13 +369,14 @@ defmodule Code.Formatter do
   defp quoted_to_algebra({:__block__, meta, [list]}, _context, state) when is_list(list) do
     case meta[:delimiter] do
       ~s['''] ->
-        {opener, quotes} = get_charlist_quotes(true, state)
+        {opener, quotes} = get_charlist_quotes(:heredoc, state)
         string = list |> List.to_string() |> escape_heredoc(quotes)
         {opener |> concat(string) |> concat(quotes) |> force_unfit(), state}
 
       ~s['] ->
-        {opener, quotes} = get_charlist_quotes(false, state)
+        string = list |> List.to_string()
-        string = list |> List.to_string() |> escape_string(quotes)
+        {opener, quotes} = get_charlist_quotes({:regular, [string]}, state)
+        string = escape_string(string, quotes)
         {opener |> concat(string) |> concat(quotes), state}
 
       _other ->
@@ -520,14 +522,15 @@ defmodule Code.Formatter do
               {string(~S{"..//":}), state}
 
             {:__block__, _, [atom]} when is_atom(atom) ->
-              key =
+              iodata =
                 if Macro.classify_atom(atom) in [:identifier, :unquoted] do
-                  IO.iodata_to_binary([Atom.to_string(atom), ?:])
+                  [Atom.to_string(atom), ?:]
                 else
-                  IO.iodata_to_binary([?", Atom.to_string(atom), ?", ?:])
+                  [?", atom |> Atom.to_string() |> String.replace("\"", "\\\""), ?", ?:]
                 end
 
-              {string(key) |> color(:atom, state.inspect_opts), state}
+              {iodata |> IO.iodata_to_binary() |> string() |> color(:atom, state.inspect_opts),
+               state}
 
             {{:., _, [:erlang, :binary_to_atom]}, _, [{:<<>>, _, entries}, :utf8]} ->
               interpolation_to_algebra(entries, @double_quote, state, "\"", "\":")
@@ -1956,6 +1959,14 @@ defmodule Code.Formatter do
   # fn a, b, c when d -> e end
   defp clause_args_to_algebra([{:when, meta, args}], state) do
     {args, right} = split_last(args)
+
+    # If there are any keywords, wrap them in lists
+    args =
+      Enum.map(args, fn
+        [_ | _] = keyword -> {:__block__, [], [keyword]}
+        other -> other
+      end)
+
     left = {{:special, :clause_args}, meta, [args]}
     binary_op_to_algebra(:when, "when", meta, left, right, :no_parens_arg, state)
   end
@@ -2409,19 +2420,23 @@ defmodule Code.Formatter do
     {left, right}
   end
 
-  defp get_charlist_quotes(_heredoc = false, state) do
+  defp get_charlist_quotes(:heredoc, state) do
-    if state.normalize_charlists_as_sigils do
-      {@sigil_c, @double_quote}
-    else
-      {@single_quote, @single_quote}
-    end
-  end
-
-  defp get_charlist_quotes(_heredoc = true, state) do
     if state.normalize_charlists_as_sigils do
       {@sigil_c_heredoc, @double_heredoc}
     else
       {@single_heredoc, @single_heredoc}
     end
   end
+
+  defp get_charlist_quotes({:regular, chunks}, state) do
+    cond do
+      !state.normalize_charlists_as_sigils -> {@single_quote, @single_quote}
+      Enum.any?(chunks, &has_double_quote?/1) -> {@sigil_c_single, @single_quote}
+      true -> {@sigil_c_double, @double_quote}
+    end
+  end
+
+  defp has_double_quote?(chunk) do
+    is_binary(chunk) and chunk =~ @double_quote
+  end
 end

lib/elixir/lib/code/fragment.ex

@@ -636,7 +636,7 @@ defmodule Code.Fragment do
     {reversed_pre, post} = adjust_position(reversed_pre, post)
 
     case take_identifier(post, []) do
-      :none ->
+      {_, [], _} ->
         maybe_operator(reversed_pre, post, line, opts)
 
       {:identifier, reversed_post, rest} ->
@@ -644,7 +644,7 @@ defmodule Code.Fragment do
         reversed = reversed_post ++ reversed_pre
 
         case codepoint_cursor_context(reversed, opts) do
-          {{:struct, acc}, offset} when acc != [] ->
+          {{:struct, acc}, offset} ->
             build_surround({:struct, acc}, reversed, line, offset)
 
           {{:alias, acc}, offset} ->
@@ -749,27 +749,11 @@ defmodule Code.Fragment do
     do: take_identifier(t, [h | acc])
 
   defp take_identifier(rest, acc) do
-    {stripped, _} = strip_spaces(rest, 0)
+    with {[?. | t], _} <- strip_spaces(rest, 0),
-
-    with [?. | t] <- stripped,
          {[h | _], _} when h in ?A..?Z <- strip_spaces(t, 0) do
       take_alias(rest, acc)
     else
-      # Consider it an identifier if we are at the end of line
+      _ -> {:identifier, acc, rest}
-      # or if we have spaces not followed by . (call) or / (arity)
-      _ when acc == [] and (rest == [] or (hd(rest) in @space and hd(stripped) not in ~c"/.")) ->
-        {:identifier, acc, rest}
-
-      # If we are immediately followed by a container, we are still part of the identifier.
-      # We don't consider << as it _may_ be an operator.
-      _ when acc == [] and hd(stripped) in ~c"({[" ->
-        {:identifier, acc, rest}
-
-      _ when acc == [] ->
-        :none
-
-      _ ->
-        {:identifier, acc, rest}
     end
   end
 
@@ -1107,6 +1091,6 @@ defmodule Code.Fragment do
     opts =
       Keyword.take(opts, [:file, :line, :column, :columns, :token_metadata, :literal_encoder])
 
-    Code.string_to_quoted(fragment, [cursor_completion: true, warnings: false] ++ opts)
+    Code.string_to_quoted(fragment, [cursor_completion: true, emit_warnings: false] ++ opts)
   end
 end

lib/elixir/lib/code/normalizer.ex

@@ -349,19 +349,25 @@ defmodule Code.Normalizer do
         meta
       end
 
+    last = List.last(args)
+
     cond do
-      Keyword.has_key?(meta, :do) or match?([{{:__block__, _, [:do]}, _} | _], List.last(args)) ->
+      not allow_keyword?(form, arity) ->
+        args = normalize_args(args, %{state | parent_meta: meta})
+        {form, meta, args}
+
+      Keyword.has_key?(meta, :do) or match?([{{:__block__, _, [:do]}, _} | _], last) ->
         # def foo do :ok end
         # def foo, do: :ok
         normalize_kw_blocks(form, meta, args, state)
 
-      match?([{:do, _} | _], List.last(args)) ->
+      match?([{:do, _} | _], last) and Keyword.keyword?(last) ->
         # Non normalized kw blocks
         line = state.parent_meta[:line]
         meta = meta ++ [do: [line: line], end: [line: line]]
         normalize_kw_blocks(form, meta, args, state)
 
-      allow_keyword?(form, arity) ->
+      true ->
         args = normalize_args(args, %{state | parent_meta: meta})
         {last_arg, leading_args} = List.pop_at(args, -1, [])
 
@@ -382,10 +388,6 @@ defmodule Code.Normalizer do
           end
 
         {form, meta, leading_args ++ last_args}
-
-      true ->
-        args = normalize_args(args, %{state | parent_meta: meta})
-        {form, meta, args}
     end
   end
 

lib/elixir/lib/code/typespec.ex

@@ -175,7 +175,8 @@ defmodule Code.Typespec do
 
   defp get_module_and_beam(module) when is_atom(module) do
     with {^module, beam, _filename} <- :code.get_object_code(module),
-         {:ok, ^module} <- beam |> :beam_lib.info() |> Keyword.fetch(:module) do
+         info_pairs when is_list(info_pairs) <- :beam_lib.info(beam),
+         {:ok, ^module} <- Keyword.fetch(info_pairs, :module) do
       {module, beam}
     else
       _ -> :error
@@ -419,5 +420,13 @@ defmodule Code.Typespec do
     :error
   end
 
-  defp meta(anno), do: [line: :erl_anno.line(anno)]
+  defp meta(anno) do
+    case :erl_anno.location(anno) do
+      {line, column} ->
+        [line: line, column: column]
+
+      line when is_integer(line) ->
+        [line: line]
+    end
+  end
 end

lib/elixir/lib/config.ex

@@ -46,9 +46,9 @@ defmodule Config do
   ## Migrating from `use Mix.Config`
 
   The `Config` module in Elixir was introduced in v1.9 as a replacement to
-  `Mix.Config`, which was specific to Mix and has been deprecated.
+  `use Mix.Config`, which was specific to Mix and has been deprecated.
 
-  You can leverage `Config` instead of `Mix.Config` in three steps. The first
+  You can leverage `Config` instead of `use Mix.Config` in three steps. The first
   step is to replace `use Mix.Config` at the top of your config files by
   `import Config`.
 

lib/elixir/lib/enum.ex

@@ -261,7 +261,7 @@ defmodule Enum do
   traversed as if it was an enumerable.
 
   For a general overview of all functions in the `Enum` module, see
-  [the `Enum` cheatsheet](enum-cheat.html).
+  [the `Enum` cheatsheet](enum-cheat.cheatmd).
 
   The functions in this module work in linear time. This means that, the
   time it takes to perform an operation grows at the same rate as the length
@@ -3793,9 +3793,6 @@ defmodule Enum do
       iex> Enum.unzip([{:a, 1}, {:b, 2}, {:c, 3}])
       {[:a, :b, :c], [1, 2, 3]}
 
-      iex> Enum.unzip(%{a: 1, b: 2})
-      {[:a, :b], [1, 2]}
-
   """
   @spec unzip(t) :: {[element], [element]}
 
@@ -4051,7 +4048,7 @@ defmodule Enum do
       ...> end)
       [{1, 2, 3}, {1, 2, 3}]
 
-      iex> enums = [[1, 2], %{a: 3, b: 4}, [5, 6]]
+      iex> enums = [[1, 2], [a: 3, b: 4], [5, 6]]
       ...> Enum.zip_reduce(enums, [], fn elements, acc ->
       ...>   [List.to_tuple(elements) | acc]
       ...> end)

lib/elixir/lib/exception.ex

@@ -31,7 +31,21 @@ defmodule Exception do
   @type arity_or_args :: non_neg_integer | list
   @type location :: keyword
 
+  @doc """
+  Receives the arguments given to `raise/2` and returns the exception struct.
+
+  The default implementation accepts either a set of keyword arguments
+  that is merged into the struct or a string to be used as the exception's message.
+  """
   @callback exception(term) :: t
+
+  @doc """
+  Receives the exception struct and must return its message.
+
+  Most commonly exceptions have a message field which by default is accessed
+  by this function. However, if an exception does not have a message field,
+  this function must be explicitly implemented.
+  """
   @callback message(t) :: String.t()
 
   @doc """
@@ -793,6 +807,194 @@ defmodule Exception do
       col -> format_file_line_column(Keyword.get(opts, :file), Keyword.get(opts, :line), col, " ")
     end
   end
+
+  @doc false
+  def format_delimiter(delimiter) do
+    if delimiter |> Atom.to_string() |> String.contains?(["\"", "'"]),
+      do: delimiter,
+      else: ~s("#{delimiter}")
+  end
+
+  @doc false
+  def format_snippet(
+        {start_line, _start_column} = start_pos,
+        {end_line, end_column} = end_pos,
+        description,
+        file,
+        lines,
+        start_message,
+        end_message
+      )
+      when start_line < end_line do
+    max_digits = digits(end_line)
+    general_padding = max(2, max_digits) + 1
+    padding = n_spaces(general_padding)
+
+    relevant_lines =
+      if end_line - start_line < 5 do
+        line_range(lines, start_pos, end_pos, padding, max_digits, start_message, end_message)
+      else
+        trimmed_inbetween_lines(
+          lines,
+          start_pos,
+          end_pos,
+          padding,
+          max_digits,
+          start_message,
+          end_message
+        )
+      end
+
+    """
+     #{padding}#{red("error:")} #{pad_message(description, padding)}
+     #{padding}│
+    #{relevant_lines}
+     #{padding}│
+     #{padding}└─ #{Path.relative_to_cwd(file)}:#{end_line}:#{end_column}\
+    """
+  end
+
+  def format_snippet(
+        {start_line, start_column},
+        {end_line, end_column},
+        description,
+        file,
+        lines,
+        start_message,
+        end_message
+      )
+      when start_line == end_line do
+    max_digits = digits(end_line)
+    general_padding = max(2, max_digits) + 1
+    padding = n_spaces(general_padding)
+    formatted_line = [line_padding(end_line, max_digits), to_string(end_line), " │ ", hd(lines)]
+
+    mismatched_closing_line =
+      [
+        n_spaces(start_column - 1),
+        red("│"),
+        format_end_message(end_column - start_column, end_message)
+      ]
+
+    unclosed_delimiter_line =
+      [padding, " │ ", format_start_message(start_column, start_message)]
+
+    below_line = [padding, " │ ", mismatched_closing_line, "\n", unclosed_delimiter_line]
+
+    """
+     #{padding}#{red("error:")} #{pad_message(description, padding)}
+     #{padding}│
+    #{formatted_line}
+    #{below_line}
+     #{padding}│
+     #{padding}└─ #{Path.relative_to_cwd(file)}:#{end_line}:#{end_column}\
+    """
+  end
+
+  defp line_padding(line_number, max_digits) do
+    line_digits = digits(line_number)
+
+    spacing =
+      if line_digits == 1 do
+        max(2, max_digits)
+      else
+        max_digits - line_digits + 1
+      end
+
+    n_spaces(spacing)
+  end
+
+  defp n_spaces(n), do: String.duplicate(" ", n)
+
+  defp digits(number, acc \\ 1)
+  defp digits(number, acc) when number < 10, do: acc
+  defp digits(number, acc), do: digits(div(number, 10), acc + 1)
+
+  defp trimmed_inbetween_lines(
+         lines,
+         {start_line, start_column},
+         {end_line, end_column},
+         padding,
+         max_digits,
+         start_message,
+         end_message
+       ) do
+    start_padding = line_padding(start_line, max_digits)
+    end_padding = line_padding(end_line, max_digits)
+    first_line = hd(lines)
+    last_line = List.last(lines)
+
+    """
+    #{start_padding}#{start_line} │ #{first_line}
+     #{padding}│ #{format_start_message(start_column, start_message)}
+     ...
+    #{end_padding}#{end_line} │ #{last_line}
+     #{padding}│ #{format_end_message(end_column, end_message)}\
+    """
+  end
+
+  defp line_range(
+         lines,
+         {start_line, start_column},
+         {end_line, end_column},
+         padding,
+         max_digits,
+         start_message,
+         end_message
+       ) do
+    Enum.zip_with(lines, start_line..end_line, fn line, line_number ->
+      line_padding = line_padding(line_number, max_digits)
+
+      cond do
+        line_number == start_line ->
+          [
+            line_padding,
+            to_string(line_number),
+            " │ ",
+            line,
+            "\n",
+            padding,
+            " │ ",
+            format_start_message(start_column, start_message)
+          ]
+
+        line_number == end_line ->
+          [
+            line_padding,
+            to_string(line_number),
+            " │ ",
+            line,
+            "\n",
+            padding,
+            " │ ",
+            format_end_message(end_column, end_message)
+          ]
+
+        true ->
+          [line_padding, to_string(line_number), " │ ", line]
+      end
+    end)
+    |> Enum.intersperse("\n")
+  end
+
+  defp format_end_message(end_column, message),
+    do: [
+      n_spaces(end_column - 1),
+      red(message)
+    ]
+
+  defp format_start_message(start_column, message),
+    do: [n_spaces(start_column - 1), red(message)]
+
+  defp pad_message(message, padding), do: String.replace(message, "\n", "\n #{padding}")
+
+  defp red(string) do
+    if IO.ANSI.enabled?() do
+      [IO.ANSI.red(), string, IO.ANSI.reset()]
+    else
+      string
+    end
+  end
 end
 
 # Some exceptions implement "message/1" instead of "exception/1" mostly
@@ -944,11 +1146,24 @@ defmodule MismatchedDelimiterError do
   An exception raised when a mismatched delimiter is found when parsing code.
 
   For example:
-  - `[1, 2, 3}`
-  - `fn a -> )`
-  """
 
-  @max_lines_shown 5
+    * `[1, 2, 3}`
+    * `fn a -> )`
+
+  The following fields of this exceptions are public and can be accessed freely:
+
+    * `:file` (`t:Path.t/0` or `nil`) - the file where the error occurred, or `nil` if
+      the error occurred in code that did not come from a file
+    * `:line` - the line for the opening delimiter
+    * `:column` - the column for the opening delimiter
+    * `:end_line` - the line for the mismatched closing delimiter
+    * `:end_column` - the column for the mismatched closing delimiter
+    * `:opening_delimiter` - an atom representing the opening delimiter
+    * `:closing_delimiter` - an atom representing the mismatched closing delimiter
+    * `:expected_delimiter` - an atom representing the closing delimiter
+    * `:description` - a description of the mismatched delimiter error
+
+  """
 
   defexception [
     :file,
@@ -958,6 +1173,7 @@ defmodule MismatchedDelimiterError do
     :end_column,
     :opening_delimiter,
     :closing_delimiter,
+    :expected_delimiter,
     :snippet,
     description: "mismatched delimiter error"
   ]
@@ -969,206 +1185,32 @@ defmodule MismatchedDelimiterError do
         end_line: end_line,
         end_column: end_column,
         description: description,
-        opening_delimiter: opening_delimiter,
+        expected_delimiter: expected_delimiter,
-        closing_delimiter: _closing_delimiter,
         file: file,
         snippet: snippet
       }) do
     start_pos = {start_line, start_column}
     end_pos = {end_line, end_column}
     lines = String.split(snippet, "\n")
-    expected_delimiter = :elixir_tokenizer.terminator(opening_delimiter)
+    expected_delimiter = Exception.format_delimiter(expected_delimiter)
+
+    start_message = "└ unclosed delimiter"
+    end_message = ~s/└ mismatched closing delimiter (expected #{expected_delimiter})/
+
+    snippet =
+      Exception.format_snippet(
+        start_pos,
+        end_pos,
+        description,
+        file,
+        lines,
+        start_message,
+        end_message
+      )
 
-    snippet = format_snippet(start_pos, end_pos, description, file, lines, expected_delimiter)
     format_message(file, end_line, end_column, snippet)
   end
 
-  defp format_snippet(
-         {start_line, _start_column} = start_pos,
-         {end_line, end_column} = end_pos,
-         description,
-         file,
-         lines,
-         expected_delimiter
-       )
-       when start_line < end_line do
-    max_digits = digits(end_line)
-    general_padding = max(2, max_digits) + 1
-    padding = n_spaces(general_padding)
-
-    relevant_lines =
-      if end_line - start_line < @max_lines_shown do
-        line_range(lines, start_pos, end_pos, padding, max_digits, expected_delimiter)
-      else
-        trimmed_inbetween_lines(
-          lines,
-          start_pos,
-          end_pos,
-          padding,
-          max_digits,
-          expected_delimiter
-        )
-      end
-
-    """
-     #{padding}#{red("error:")} #{pad_message(description, padding)}
-     #{padding}│
-    #{relevant_lines}
-     #{padding}│
-     #{padding}└─ #{Path.relative_to_cwd(file)}:#{end_line}:#{end_column}\
-    """
-  end
-
-  defp format_snippet(
-         {start_line, start_column},
-         {end_line, end_column},
-         description,
-         file,
-         lines,
-         expected_delimiter
-       )
-       when start_line == end_line do
-    max_digits = digits(end_line)
-    general_padding = max(2, max_digits) + 1
-    padding = n_spaces(general_padding)
-
-    line = Enum.fetch!(lines, end_line - 1)
-    formatted_line = [line_padding(end_line, max_digits), to_string(end_line), " │ ", line]
-
-    mismatched_closing_line =
-      [
-        n_spaces(start_column - 1),
-        red("│"),
-        mismatched_closing_delimiter(end_column - start_column, expected_delimiter)
-      ]
-
-    unclosed_delimiter_line =
-      [padding, " │ ", unclosed_delimiter(start_column)]
-
-    below_line = [padding, " │ ", mismatched_closing_line, "\n", unclosed_delimiter_line]
-
-    """
-     #{padding}#{red("error:")} #{pad_message(description, padding)}
-     #{padding}│
-    #{formatted_line}
-    #{below_line}
-     #{padding}│
-     #{padding}└─ #{Path.relative_to_cwd(file)}:#{end_line}:#{end_column}\
-    """
-  end
-
-  defp line_padding(line_number, max_digits) do
-    line_digits = digits(line_number)
-
-    spacing =
-      if line_digits == 1 do
-        max(2, max_digits)
-      else
-        max_digits - line_digits + 1
-      end
-
-    n_spaces(spacing)
-  end
-
-  defp n_spaces(n), do: String.duplicate(" ", n)
-
-  defp digits(number, acc \\ 1)
-  defp digits(number, acc) when number < 10, do: acc
-  defp digits(number, acc), do: digits(div(number, 10), acc + 1)
-
-  defp trimmed_inbetween_lines(
-         lines,
-         {start_line, start_column},
-         {end_line, end_column},
-         padding,
-         max_digits,
-         expected_delimiter
-       ) do
-    start_padding = line_padding(start_line, max_digits)
-    end_padding = line_padding(end_line, max_digits)
-    first_line = Enum.fetch!(lines, start_line - 1)
-    last_line = Enum.fetch!(lines, end_line - 1)
-
-    """
-    #{start_padding}#{start_line} │ #{first_line}
-     #{padding}│ #{unclosed_delimiter(start_column)}
-     ...
-    #{end_padding}#{end_line} │ #{last_line}
-     #{padding}│ #{mismatched_closing_delimiter(end_column, expected_delimiter)}\
-    """
-  end
-
-  defp line_range(
-         lines,
-         {start_line, start_column},
-         {end_line, end_column},
-         padding,
-         max_digits,
-         expected_delimiter
-       ) do
-    start_line = start_line - 1
-    end_line = end_line - 1
-
-    lines
-    |> Enum.slice(start_line..end_line)
-    |> Enum.zip_with(start_line..end_line, fn line, line_number ->
-      line_number = line_number + 1
-      start_line = start_line + 1
-      end_line = end_line + 1
-
-      line_padding = line_padding(line_number, max_digits)
-
-      cond do
-        line_number == start_line ->
-          [
-            line_padding,
-            to_string(line_number),
-            " │ ",
-            line,
-            "\n",
-            padding,
-            " │ ",
-            unclosed_delimiter(start_column)
-          ]
-
-        line_number == end_line ->
-          [
-            line_padding,
-            to_string(line_number),
-            " │ ",
-            line,
-            "\n",
-            padding,
-            " │ ",
-            mismatched_closing_delimiter(end_column, expected_delimiter)
-          ]
-
-        true ->
-          [line_padding, to_string(line_number), " │ ", line]
-      end
-    end)
-    |> Enum.intersperse("\n")
-  end
-
-  defp mismatched_closing_delimiter(end_column, expected_closing_delimiter),
-    do: [
-      n_spaces(end_column - 1),
-      red(~s/└ mismatched closing delimiter (expected "#{expected_closing_delimiter}")/)
-    ]
-
-  defp unclosed_delimiter(start_column),
-    do: [n_spaces(start_column - 1), red("└ unclosed delimiter")]
-
-  defp pad_message(message, padding), do: String.replace(message, "\n", "\n #{padding}")
-
-  defp red(string) do
-    if IO.ANSI.enabled?() do
-      [IO.ANSI.red(), string, IO.ANSI.reset()]
-    else
-      string
-    end
-  end
-
   defp format_message(file, line, column, message) do
     location = Exception.format_file_line_column(Path.relative_to_cwd(file), line, column)
     "mismatched delimiter found on " <> location <> "\n" <> message
@@ -1183,8 +1225,9 @@ defmodule SyntaxError do
 
     * `:file` (`t:Path.t/0` or `nil`) - the file where the error occurred, or `nil` if
       the error occurred in code that did not come from a file
-    * `:line` (`t:non_neg_integer/0`) - the line where the error occurred
+    * `:line` - the line where the error occurred
-    * `:column` (`t:non_neg_integer/0`) - the column where the error occurred
+    * `:column` - the column where the error occurred
+    * `:description` - a description of the syntax error
 
   """
 
@@ -1233,17 +1276,24 @@ defmodule TokenMissingError do
 
     * `:file` (`t:Path.t/0` or `nil`) - the file where the error occurred, or `nil` if
       the error occurred in code that did not come from a file
-    * `:line` (`t:non_neg_integer/0`) - the line where the error occurred
+    * `:line` - the line for the opening delimiter
-    * `:column` (`t:non_neg_integer/0`) - the column where the error occurred
+    * `:column` - the column for the opening delimiter
-
+    * `:end_line` - the line for the end of the string
+    * `:end_column` - the column for the end of the string
+    * `:opening_delimiter` - an atom representing the opening delimiter
+    * `:expected_delimiter` - an atom representing the expected delimiter
+    * `:description` - a description of the missing token error
   """
 
   defexception [
     :file,
     :line,
-    :snippet,
     :column,
+    :end_line,
+    :end_column,
+    :snippet,
     :opening_delimiter,
+    :expected_delimiter,
     description: "expression is incomplete"
   ]
 
@@ -1252,14 +1302,40 @@ defmodule TokenMissingError do
         file: file,
         line: line,
         column: column,
+        end_line: end_line,
         description: description,
+        expected_delimiter: expected_delimiter,
         snippet: snippet
       })
-      when not is_nil(snippet) and not is_nil(column) do
+      when not is_nil(snippet) and not is_nil(column) and not is_nil(end_line) do
-    snippet =
+    {trimmed, [last_line | _] = reversed_lines} =
-      :elixir_errors.format_snippet({line, column}, file, description, snippet, :error, [], nil)
+      snippet
+      |> String.split("\n")
+      |> Enum.reverse()
+      |> Enum.split_while(&(&1 == ""))
 
-    format_message(file, line, column, snippet)
+    end_line = end_line - length(trimmed)
+    end_column = String.length(last_line) + 1
+
+    start_pos = {line, column}
+    end_pos = {end_line, end_column}
+    expected_delimiter = Exception.format_delimiter(expected_delimiter)
+
+    start_message = ~s/└ unclosed delimiter/
+    end_message = ~s/└ missing closing delimiter (expected #{expected_delimiter})/
+
+    snippet =
+      Exception.format_snippet(
+        start_pos,
+        end_pos,
+        description,
+        file,
+        Enum.reverse(reversed_lines),
+        start_message,
+        end_message
+      )
+
+    format_message(file, end_line, end_column, snippet)
   end
 
   @impl true
@@ -1267,13 +1343,13 @@ defmodule TokenMissingError do
         file: file,
         line: line,
         column: column,
+        snippet: snippet,
         description: description
       }) do
     snippet =
-      :elixir_errors.format_snippet({line, column}, file, description, nil, :error, [], nil)
+      :elixir_errors.format_snippet({line, column}, file, description, snippet, :error, [], nil)
 
-    padded = "   " <> String.replace(snippet, "\n", "\n   ")
+    format_message(file, line, column, snippet)
-    format_message(file, line, column, padded)
   end
 
   defp format_message(file, line, column, message) do

lib/elixir/lib/file.ex

@@ -133,6 +133,7 @@ defmodule File do
 
   @type stream_mode ::
           encoding_mode()
+          | read_offset_mode()
           | :append
           | :compressed
           | :delayed_write
@@ -140,6 +141,8 @@ defmodule File do
           | {:read_ahead, pos_integer | false}
           | {:delayed_write, non_neg_integer, non_neg_integer}
 
+  @type read_offset_mode :: {:read_offset, non_neg_integer()}
+
   @type erlang_time ::
           {{year :: non_neg_integer(), month :: 1..12, day :: 1..31},
            {hour :: 0..23, minute :: 0..59, second :: 0..59}}
@@ -1748,18 +1751,20 @@ defmodule File do
   One may also consider passing the `:delayed_write` option if the stream
   is meant to be written to under a tight loop.
 
-  ## Byte order marks
+  ## Byte order marks and read offset
 
   If you pass `:trim_bom` in the modes parameter, the stream will
   trim UTF-8, UTF-16 and UTF-32 byte order marks when reading from file.
 
   Note that this function does not try to discover the file encoding
-  based on BOM.
+  based on BOM. From Elixir v1.16.0, you may also pass a `:read_offset`
+  that is skipped whenever enumerating the stream (if both `:read_offset`
+  and `:trim_bom` are given, the offset is skipped after the BOM).
 
   ## Examples
 
       # Read a utf8 text file which may include BOM
-      File.stream!("./test/test.txt", encoding: :utf8, trim_bom: true)
+      File.stream!("./test/test.txt", [:trim_bom, encoding: :utf8])
 
       # Read in 2048 byte chunks rather than lines
       File.stream!("./test/test.data", 2048)

lib/elixir/lib/file/stream.ex

@@ -18,6 +18,12 @@ defmodule File.Stream do
 
   @doc false
   def __build__(path, line_or_bytes, modes) do
+    with {:read_offset, offset} <- :lists.keyfind(:read_offset, 1, modes),
+         false <- is_integer(offset) and offset >= 0 do
+      raise ArgumentError,
+            "expected :read_offset to be a non-negative integer, got: #{inspect(offset)}"
+    end
+
     raw = :lists.keyfind(:encoding, 1, modes) == false
 
     modes =
@@ -88,7 +94,7 @@ defmodule File.Stream do
       start_fun = fn ->
         case File.Stream.__open__(stream, read_modes(modes)) do
           {:ok, device} ->
-            if :trim_bom in modes, do: trim_bom(device, raw) |> elem(0), else: device
+            skip_bom_and_offset(device, raw, modes)
 
           {:error, reason} ->
             raise File.Error, reason: reason, action: "stream", path: stream.path
@@ -104,9 +110,14 @@ defmodule File.Stream do
       Stream.resource(start_fun, next_fun, &:file.close/1).(acc, fun)
     end
 
-    def count(%{modes: modes, line_or_bytes: :line, path: path} = stream) do
+    def count(%{modes: modes, line_or_bytes: :line, path: path, raw: raw} = stream) do
       pattern = :binary.compile_pattern("\n")
-      counter = &count_lines(&1, path, pattern, read_function(stream), 0)
+
+      counter = fn device ->
+        device = skip_bom_and_offset(device, raw, modes)
+        count_lines(device, path, pattern, read_function(stream), 0)
+      end
+
       {:ok, open!(stream, modes, counter)}
     end
 
@@ -116,8 +127,11 @@ defmodule File.Stream do
           {:error, __MODULE__}
 
         {:ok, %{size: size}} ->
+          bom_offset = count_raw_bom(stream, modes)
+          offset = get_read_offset(modes)
+          size = max(size - bom_offset - offset, 0)
           remainder = if rem(size, bytes) == 0, do: 0, else: 1
-          {:ok, div(size, bytes) + remainder - count_raw_bom(stream, modes)}
+          {:ok, div(size, bytes) + remainder}
 
         {:error, reason} ->
           raise File.Error, reason: reason, action: "stream", path: path
@@ -158,6 +172,23 @@ defmodule File.Stream do
       end
     end
 
+    defp skip_bom_and_offset(device, raw, modes) do
+      device =
+        if :trim_bom in modes do
+          device |> trim_bom(raw) |> elem(0)
+        else
+          device
+        end
+
+      offset = get_read_offset(modes)
+
+      if offset > 0 do
+        {:ok, _} = :file.position(device, {:cur, offset})
+      end
+
+      device
+    end
+
     defp trim_bom(device, true) do
       bom_length = device |> IO.binread(4) |> bom_length()
       {:ok, new_pos} = :file.position(device, bom_length)
@@ -183,6 +214,13 @@ defmodule File.Stream do
     defp bom_length(<<254, 255, 0, 0, _rest::binary>>), do: 4
     defp bom_length(_binary), do: 0
 
+    def get_read_offset(modes) do
+      case :lists.keyfind(:read_offset, 1, modes) do
+        {:read_offset, offset} -> offset
+        false -> 0
+      end
+    end
+
     defp read_modes(modes) do
       for mode <- modes, mode not in [:write, :append, :trim_bom], do: mode
     end

lib/elixir/lib/float.ex

@@ -198,7 +198,7 @@ defmodule Float do
   defp add_dot(acc, false), do: acc <> ".0"
 
   @doc """
-  Rounds a float to the largest number less than or equal to `num`.
+  Rounds a float to the largest float less than or equal to `number`.
 
   `floor/2` also accepts a precision to round a floating-point value down
   to an arbitrary number of fractional digits (between 0 and 15).
@@ -246,7 +246,7 @@ defmodule Float do
   end
 
   @doc """
-  Rounds a float to the smallest integer greater than or equal to `num`.
+  Rounds a float to the smallest float greater than or equal to `number`.
 
   `ceil/2` also accepts a precision to round a floating-point value down
   to an arbitrary number of fractional digits (between 0 and 15).
@@ -275,6 +275,8 @@ defmodule Float do
       -56.0
       iex> Float.ceil(34.251, 2)
       34.26
+      iex> Float.ceil(-0.01)
+      -0.0
 
   """
   @spec ceil(float, precision_range) :: float
@@ -332,6 +334,8 @@ defmodule Float do
       -6.0
       iex> Float.round(12.341444444444441, 15)
       12.341444444444441
+      iex> Float.round(-0.01)
+      -0.0
 
   """
   @spec round(float, precision_range) :: float
@@ -340,8 +344,13 @@ defmodule Float do
   # and could be implemented in the future.
   def round(float, precision \\ 0)
 
+  def round(float, 0) when float == 0.0, do: float
+
   def round(float, 0) when is_float(float) do
-    float |> :erlang.round() |> :erlang.float()
+    case float |> :erlang.round() |> :erlang.float() do
+      zero when zero == 0.0 and float < 0.0 -> -0.0
+      rounded -> rounded
+    end
   end
 
   def round(float, precision) when is_float(float) and precision in @precision_range do
@@ -365,6 +374,8 @@ defmodule Float do
         case rounding do
           :ceil when sign === 0 -> 1 / power_of_10(precision)
           :floor when sign === 1 -> -1 / power_of_10(precision)
+          :ceil when sign === 1 -> minus_zero()
+          :half_up when sign === 1 -> minus_zero()
           _ -> 0.0
         end
 
@@ -394,6 +405,9 @@ defmodule Float do
         boundary = den <<< 52
 
         cond do
+          num == 0 and sign == 1 ->
+            minus_zero()
+
           num == 0 ->
             0.0
 
@@ -408,6 +422,11 @@ defmodule Float do
     end
   end
 
+  # TODO remove once we require Erlang/OTP 27+
+  # This function tricks the compiler to avoid this bug in previous versions:
+  # https://github.com/elixir-lang/elixir/blob/main/lib/elixir/lib/float.ex#L408-L412
+  defp minus_zero, do: -0.0
+
   defp decompose(significant, initial) do
     decompose(significant, 1, 0, initial)
   end

lib/elixir/lib/gen_server.ex

@@ -9,14 +9,10 @@ defmodule GenServer do
   tracing and error reporting. It will also fit into a supervision tree.
 
   ```mermaid
-  graph TD
+  graph BT
-      GenServer
+      C(Client #3) ~~~ B(Client #2) ~~~ A(Client #1)
-      GenServer -. reply -.-> A
+      A & B & C -->|request| GenServer
-      GenServer -. reply -.-> B
+      GenServer -.->|reply| A & B & C
-      GenServer -. reply -.-> C
-      A(Client #1) -- request --> GenServer
-      B(Client #2) -- request --> GenServer
-      C(Client #3) -- request --> GenServer
   ```
 
   ## Example
@@ -475,7 +471,7 @@ defmodule GenServer do
   guide provides a tutorial-like introduction. The documentation and links
   in Erlang can also provide extra insight.
 
-    * [GenServer - Elixir's Getting Started Guide](https://elixir-lang.org/getting-started/mix-otp/genserver.html)
+    * [GenServer - Elixir's Getting Started Guide](genservers.md)
     * [`:gen_server` module documentation](`:gen_server`)
     * [gen_server Behaviour - OTP Design Principles](https://www.erlang.org/doc/design_principles/gen_server_concepts.html)
     * [Clients and Servers - Learn You Some Erlang for Great Good!](http://learnyousomeerlang.com/clients-and-servers)

lib/elixir/lib/inspect.ex

@@ -535,7 +535,8 @@ end
 
 defimpl Inspect, for: Any do
   defmacro __deriving__(module, struct, options) do
-    fields = Map.keys(struct) -- [:__exception__, :__struct__]
+    fields = Enum.sort(Map.keys(struct) -- [:__exception__, :__struct__])
+
     only = Keyword.get(options, :only, fields)
     except = Keyword.get(options, :except, [])
     optional = Keyword.get(options, :optional, [])
@@ -545,7 +546,7 @@ defimpl Inspect, for: Any do
     :ok = validate_option(:optional, optional, fields, module)
 
     inspect_module =
-      if fields == only and except == [] do
+      if fields == Enum.sort(only) and except == [] do
         Inspect.Map
       else
         Inspect.Any

lib/elixir/lib/inspect/algebra.ex

@@ -143,9 +143,8 @@ defmodule Inspect.Opts do
   function as this must be controlled by applications. Libraries
   should instead define their own structs with custom inspect
   implementations. If a library must change the default inspect
-  function, then it is best to define to ask users of your library
+  function, then it is best to ask users of your library to explicitly
-  to explicitly call `default_inspect_fun/1` with your function of
+  call `default_inspect_fun/1` with your function of choice.
-  choice.
 
   The default is `Inspect.inspect/2`.
 

lib/elixir/lib/io.ex

@@ -308,17 +308,20 @@ defmodule IO do
       entry from the compilation environment will be used
 
     * a keyword list with at least the `:file` option representing
-      a single stacktrace entry (since v1.14.0). The `:line`, `:module`,
+      a single stacktrace entry (since v1.14.0). The `:line`, `:column`,
-      `:function` options are also supported
+      `:module`, and `:function` options are also supported
 
-  This function also notifies the compiler a warning was printed
+  This function notifies the compiler a warning was printed
-  (in case --warnings-as-errors was enabled). It returns `:ok`
+  and emits a compiler diagnostic (`t:Code.diagnostic/1`).
-  if it succeeds.
+  The diagnostic will include precise file and location information
+  if a `Macro.Env` is given or those values have been passed as
+  keyword list, but not for stacktraces, as they are often imprecise.
+
+  It returns `:ok` if it succeeds.
 
   ## Examples
 
-      stacktrace = [{MyApp, :main, 1, [file: 'my_app.ex', line: 4]}]
+      IO.warn("variable bar is unused", module: MyApp, function: {:main, 1}, line: 4, file: "my_app.ex")
-      IO.warn("variable bar is unused", stacktrace)
       #=> warning: variable bar is unused
       #=>   my_app.ex:4: MyApp.main/1
 
@@ -337,15 +340,22 @@ defmodule IO do
 
   def warn(message, [{_, _} | _] = keyword) do
     if file = keyword[:file] do
-      warn(
+      line = keyword[:line]
-        message,
+      column = keyword[:column]
-        %{
+      position = if line && column, do: {line, column}, else: line
+      message = to_chardata(message)
+
+      stacktrace =
+        Macro.Env.stacktrace(%{
           __ENV__
           | module: keyword[:module],
             function: keyword[:function],
-            line: keyword[:line],
+            line: line,
             file: file
-        }
+        })
+
+      :elixir_errors.emit_diagnostic(:warning, position, file, message, stacktrace,
+        read_snippet: true
       )
     else
       warn(message, [])

lib/elixir/lib/io/ansi/docs.ex

@@ -176,12 +176,17 @@ defmodule IO.ANSI.Docs do
     process_code(rest, [line], indent, options)
   end
 
-  defp process(["```" <> _line | rest], text, indent, options) do
+  defp process(["```mermaid" <> _line | rest], text, indent, options) do
-    process_fenced_code_block(rest, text, indent, options, _delimiter = "```")
+    write_text(text, indent, options)
+
+    rest
+    |> Enum.drop_while(&(&1 != "```"))
+    |> Enum.drop(1)
+    |> process([], indent, options)
   end
 
-  defp process(["~~~" <> _line | rest], text, indent, options) do
+  defp process(["```" <> _line | rest], text, indent, options) do
-    process_fenced_code_block(rest, text, indent, options, _delimiter = "~~~")
+    process_fenced_code_block(rest, text, indent, options, _delimiter = "```")
   end
 
   defp process(["<!--" <> line | rest], text, indent, options) do
@@ -388,7 +393,7 @@ defmodule IO.ANSI.Docs do
   end
 
   defp process_fenced_code([line | rest], code, indent, options, delimiter) do
-    if line === delimiter do
+    if line == delimiter do
       process_code(rest, code, indent, options)
     else
       process_fenced_code(rest, [line | code], indent, options, delimiter)

lib/elixir/lib/kernel.ex

@@ -2064,9 +2064,6 @@ defmodule Kernel do
       {var, _, nil} when is_atom(var) ->
         invalid_concat_left_argument_error(Atom.to_string(var))
 
-      {:^, _, [{var, _, nil}]} when is_atom(var) ->
-        invalid_concat_left_argument_error("^#{Atom.to_string(var)}")
-
       _ ->
         expanded_arg
     end
@@ -2079,8 +2076,9 @@ defmodule Kernel do
   defp invalid_concat_left_argument_error(arg) do
     :erlang.error(
       ArgumentError.exception(
-        "the left argument of <> operator inside a match should always be a literal " <>
+        "cannot perform prefix match because the left operand of <> has unknown size. " <>
-          "binary because its size can't be verified. Got: #{arg}"
+          "The left operand of <> inside a match should either be a literal binary or " <>
+          "an existing variable with the pin operator (such as ^some_var). Got: #{arg}"
       )
     )
   end
@@ -3625,7 +3623,7 @@ defmodule Kernel do
   defp do_at([], meta, name, function?, env) do
     IO.warn(
       "the @#{name}() notation (with parentheses) is deprecated, please use @#{name} (without parentheses) instead",
-      Macro.Env.stacktrace(env)
+      env
     )
 
     do_at(nil, meta, name, function?, env)

lib/elixir/lib/kernel/parallel_compiler.ex

@@ -455,7 +455,11 @@ defmodule Kernel.ParallelCompiler do
   # No more queue, nothing waiting, this cycle is done
   defp spawn_workers([], spawned, waiting, files, result, warnings, errors, state)
        when map_size(spawned) == 0 and map_size(waiting) == 0 do
-    [] = errors
+    # Print any spurious error that we may have found
+    Enum.map(errors, fn {diagnostic, read_snippet} ->
+      :elixir_errors.print_diagnostic(diagnostic, read_snippet)
+    end)
+
     [] = files
     cycle_return = each_cycle_return(state.each_cycle.())
     state = cycle_timing(result, state)
@@ -510,8 +514,9 @@ defmodule Kernel.ParallelCompiler do
     if deadlocked do
       spawn_workers(deadlocked, spawned, waiting, files, result, warnings, errors, state)
     else
-      deadlock_errors = handle_deadlock(waiting, files)
+      return_error(warnings, errors, state, fn ->
-      {return_error(deadlock_errors ++ errors, warnings), state}
+        handle_deadlock(waiting, files)
+      end)
     end
   end
 
@@ -681,12 +686,13 @@ defmodule Kernel.ParallelCompiler do
         state = %{state | timer_ref: timer_ref}
         spawn_workers(queue, spawned, waiting, files, result, warnings, errors, state)
 
-      {:diagnostic, %{severity: :warning} = diagnostic} ->
+      {:diagnostic, %{severity: :warning, file: file} = diagnostic, read_snippet} ->
-        warnings = [Module.ParallelChecker.format_diagnostic_file(diagnostic) | warnings]
+        :elixir_errors.print_diagnostic(diagnostic, read_snippet)
+        warnings = [%{diagnostic | file: file && Path.absname(file)} | warnings]
         wait_for_messages(queue, spawned, waiting, files, result, warnings, errors, state)
 
-      {:diagnostic, %{severity: :error} = diagnostic} ->
+      {:diagnostic, %{severity: :error} = diagnostic, read_snippet} ->
-        errors = [Module.ParallelChecker.format_diagnostic_file(diagnostic) | errors]
+        errors = [{diagnostic, read_snippet} | errors]
         wait_for_messages(queue, spawned, waiting, files, result, warnings, errors, state)
 
       {:file_ok, child_pid, ref, file, lexical} ->
@@ -706,10 +712,13 @@ defmodule Kernel.ParallelCompiler do
         spawn_workers(queue, new_spawned, waiting, new_files, result, warnings, errors, state)
 
       {:file_error, child_pid, file, {kind, reason, stack}} ->
-        print_error(file, kind, reason, stack)
         {_file, _new_spawned, new_files} = discard_file_pid(spawned, files, child_pid)
         terminate(new_files)
-        {return_error([to_error(file, kind, reason, stack) | errors], warnings), state}
+
+        return_error(warnings, errors, state, fn ->
+          print_error(file, kind, reason, stack)
+          [to_error(file, kind, reason, stack)]
+        end)
 
       {:DOWN, ref, :process, pid, reason} when is_map_key(spawned, ref) ->
         # async spawned processes have no file, so we always have to delete the ref directly
@@ -718,18 +727,27 @@ defmodule Kernel.ParallelCompiler do
         {file, spawned, files} = discard_file_pid(spawned, files, pid)
 
         if file do
-          print_error(file.file, :exit, reason, [])
           terminate(files)
-          {return_error([to_error(file.file, :exit, reason, []) | errors], warnings), state}
+
+          return_error(warnings, errors, state, fn ->
+            print_error(file.file, :exit, reason, [])
+            [to_error(file.file, :exit, reason, [])]
+          end)
         else
           wait_for_messages(queue, spawned, waiting, files, result, warnings, errors, state)
         end
     end
   end
 
-  defp return_error(errors, warnings) do
+  defp return_error(warnings, errors, state, fun) do
+    errors =
+      Enum.map(errors, fn {%{file: file} = diagnostic, read_snippet} ->
+        :elixir_errors.print_diagnostic(diagnostic, read_snippet)
+        %{diagnostic | file: file && Path.absname(file)}
+      end)
+
     info = %{compile_warnings: Enum.reverse(warnings), runtime_warnings: []}
-    {:error, Enum.reverse(errors), info}
+    {{:error, Enum.reverse(errors, fun.()), info}, state}
   end
 
   defp update_result(result, kind, module, value) do
@@ -859,9 +877,12 @@ defmodule Kernel.ParallelCompiler do
     )
 
     for {file, _, description, stacktrace} <- deadlock do
+      file = Path.absname(file)
+
       %{
         severity: :error,
-        file: Path.absname(file),
+        file: file,
+        source: file,
         position: nil,
         message: description,
         stacktrace: stacktrace,
@@ -889,50 +910,64 @@ defmodule Kernel.ParallelCompiler do
     ])
   end
 
-  defp to_error(file, kind, reason, stack) do
+  defp to_error(source, kind, reason, stack) do
-    line = get_line(file, reason, stack)
+    {file, line, span} = get_snippet_info(source, reason, stack)
-    file = Path.absname(file)
+    source = Path.absname(source)
     message = :unicode.characters_to_binary(Kernel.CLI.format_error(kind, reason, stack))
 
     %{
-      file: file,
+      file: file || source,
+      source: source,
       position: line || 0,
       message: message,
       severity: :error,
       stacktrace: stack,
-      span: nil
+      span: span,
+      details: {kind, reason}
     }
   end
 
-  defp get_line(_file, %{line: line, column: column}, _stack)
+  defp get_snippet_info(
+         _file,
+         %{file: file, line: line, column: column, end_line: end_line, end_column: end_column},
+         _stack
+       )
+       when is_integer(line) and line > 0 and is_integer(column) and column >= 0 and
+              is_integer(end_line) and end_line > 0 and is_integer(end_column) and end_column >= 0 do
+    {Path.absname(file), {line, column}, {end_line, end_column}}
+  end
+
+  defp get_snippet_info(_file, %{file: file, line: line, column: column}, _stack)
        when is_integer(line) and line > 0 and is_integer(column) and column >= 0 do
-    {line, column}
+    {Path.absname(file), {line, column}, nil}
   end
 
-  defp get_line(_file, %{line: line}, _stack) when is_integer(line) and line > 0 do
+  defp get_snippet_info(_file, %{line: line}, _stack) when is_integer(line) and line > 0 do
-    line
+    {nil, line, nil}
   end
 
-  defp get_line(file, :undef, [{_, _, _, []}, {_, _, _, info} | _]) do
+  defp get_snippet_info(file, :undef, [{_, _, _, []}, {_, _, _, info} | _]) do
-    if Keyword.get(info, :file) == to_charlist(Path.relative_to_cwd(file)) do
+    get_snippet_info_from_stacktrace_info(info, file)
-      Keyword.get(info, :line)
-    end
   end
 
-  defp get_line(file, _reason, [{_, _, _, [file: expanding]}, {_, _, _, info} | _])
+  defp get_snippet_info(file, _reason, [{_, _, _, [file: expanding]}, {_, _, _, info} | _])
        when expanding in [~c"expanding macro", ~c"expanding struct"] do
-    if Keyword.get(info, :file) == to_charlist(Path.relative_to_cwd(file)) do
+    get_snippet_info_from_stacktrace_info(info, file)
-      Keyword.get(info, :line)
-    end
   end
 
-  defp get_line(file, _reason, [{_, _, _, info} | _]) do
+  defp get_snippet_info(file, _reason, [{_, _, _, info} | _]) do
-    if Keyword.get(info, :file) == to_charlist(Path.relative_to_cwd(file)) do
+    get_snippet_info_from_stacktrace_info(info, file)
-      Keyword.get(info, :line)
-    end
   end
 
-  defp get_line(_, _, _) do
+  defp get_snippet_info(_, _, _) do
-    nil
+    {nil, nil, nil}
+  end
+
+  defp get_snippet_info_from_stacktrace_info(info, file) do
+    if Keyword.get(info, :file) == to_charlist(Path.relative_to_cwd(file)) do
+      {nil, Keyword.get(info, :line), nil}
+    else
+      {nil, nil, nil}
+    end
   end
 end

lib/elixir/lib/kernel/typespec.ex

@@ -385,17 +385,17 @@ defmodule Kernel.Typespec do
       compile_error(caller, error)
     end
 
-    line = line(meta)
+    location = location(meta)
     vars = Keyword.keys(guard)
 
     {args, state} = :lists.mapfoldl(&typespec(&1, vars, caller, &2), state, args)
     {return, state} = typespec(return, vars, caller, state)
-    spec = {:type, line, :fun, [{:type, line, :product, args}, return]}
+    spec = {:type, location, :fun, [{:type, location, :product, args}, return]}
 
     {spec, state} =
       case guard_to_constraints(guard, vars, meta, caller, state) do
         {[], state} -> {spec, state}
-        {constraints, state} -> {{:type, line, :bounded_fun, [spec, constraints]}, state}
+        {constraints, state} -> {{:type, location, :bounded_fun, [spec, constraints]}, state}
       end
 
     ensure_no_unused_local_vars!(caller, state.local_vars)
@@ -437,7 +437,7 @@ defmodule Kernel.Typespec do
   defp ensure_not_default(_), do: :ok
 
   defp guard_to_constraints(guard, vars, meta, caller, state) do
-    line = line(meta)
+    location = location(meta)
 
     fun = fn
       {_name, {:var, _, context}}, {constraints, state} when is_atom(context) ->
@@ -445,9 +445,9 @@ defmodule Kernel.Typespec do
 
       {name, type}, {constraints, state} ->
         {spec, state} = typespec(type, vars, caller, state)
-        constraint = [{:atom, line, :is_subtype}, [{:var, line, name}, spec]]
+        constraint = [{:atom, location, :is_subtype}, [{:var, location, name}, spec]]
         state = update_local_vars(state, name)
-        {[{:type, line, :constraint, constraint} | constraints], state}
+        {[{:type, location, :constraint, constraint} | constraints], state}
     end
 
     {constraints, state} = :lists.foldl(fun, {[], state}, guard)
@@ -456,21 +456,27 @@ defmodule Kernel.Typespec do
 
   ## To typespec conversion
 
-  defp line(meta) do
+  defp location(meta) do
-    Keyword.get(meta, :line, 0)
+    line = Keyword.get(meta, :line, 0)
+
+    if column = Keyword.get(meta, :column) do
+      {line, column}
+    else
+      line
+    end
   end
 
   # Handle unions
   defp typespec({:|, meta, [_, _]} = exprs, vars, caller, state) do
     exprs = collect_union(exprs)
     {union, state} = :lists.mapfoldl(&typespec(&1, vars, caller, &2), state, exprs)
-    {{:type, line(meta), :union, union}, state}
+    {{:type, location(meta), :union, union}, state}
   end
 
   # Handle binaries
   defp typespec({:<<>>, meta, []}, _, _, state) do
-    line = line(meta)
+    location = location(meta)
-    {{:type, line, :binary, [{:integer, line, 0}, {:integer, line, 0}]}, state}
+    {{:type, location, :binary, [{:integer, location, 0}, {:integer, location, 0}]}, state}
   end
 
   defp typespec(
@@ -480,14 +486,18 @@ defmodule Kernel.Typespec do
          state
        )
        when is_atom(ctx1) and is_atom(ctx2) and unit in 1..256 do
-    line = line(meta)
+    location = location(meta)
-    {{:type, line, :binary, [{:integer, line, 0}, {:integer, line(unit_meta), unit}]}, state}
+
+    {{:type, location, :binary, [{:integer, location, 0}, {:integer, location(unit_meta), unit}]},
+     state}
   end
 
   defp typespec({:<<>>, meta, [{:"::", size_meta, [{:_, _, ctx}, size]}]}, _, _, state)
        when is_atom(ctx) and is_integer(size) and size >= 0 do
-    line = line(meta)
+    location = location(meta)
-    {{:type, line, :binary, [{:integer, line(size_meta), size}, {:integer, line, 0}]}, state}
+
+    {{:type, location, :binary, [{:integer, location(size_meta), size}, {:integer, location, 0}]},
+     state}
   end
 
   defp typespec(
@@ -505,8 +515,8 @@ defmodule Kernel.Typespec do
        )
        when is_atom(ctx1) and is_atom(ctx2) and is_atom(ctx3) and is_integer(size) and
               size >= 0 and unit in 1..256 do
-    args = [{:integer, line(size_meta), size}, {:integer, line(unit_meta), unit}]
+    args = [{:integer, location(size_meta), size}, {:integer, location(unit_meta), unit}]
-    {{:type, line(meta), :binary, args}, state}
+    {{:type, location(meta), :binary, args}, state}
   end
 
   defp typespec({:<<>>, _meta, _args}, _vars, caller, _state) do
@@ -519,7 +529,7 @@ defmodule Kernel.Typespec do
 
   ## Handle maps and structs
   defp typespec({:map, meta, args}, _vars, _caller, state) when args == [] or is_atom(args) do
-    {{:type, line(meta), :map, :any}, state}
+    {{:type, location(meta), :map, :any}, state}
   end
 
   defp typespec({:%{}, meta, fields} = map, vars, caller, state) do
@@ -527,17 +537,17 @@ defmodule Kernel.Typespec do
       {{:required, meta2, [k]}, v}, state ->
         {arg1, state} = typespec(k, vars, caller, state)
         {arg2, state} = typespec(v, vars, caller, state)
-        {{:type, line(meta2), :map_field_exact, [arg1, arg2]}, state}
+        {{:type, location(meta2), :map_field_exact, [arg1, arg2]}, state}
 
       {{:optional, meta2, [k]}, v}, state ->
         {arg1, state} = typespec(k, vars, caller, state)
         {arg2, state} = typespec(v, vars, caller, state)
-        {{:type, line(meta2), :map_field_assoc, [arg1, arg2]}, state}
+        {{:type, location(meta2), :map_field_assoc, [arg1, arg2]}, state}
 
       {k, v}, state ->
         {arg1, state} = typespec(k, vars, caller, state)
         {arg2, state} = typespec(v, vars, caller, state)
-        {{:type, line(meta), :map_field_exact, [arg1, arg2]}, state}
+        {{:type, location(meta), :map_field_exact, [arg1, arg2]}, state}
 
       {:|, _, [_, _]}, _state ->
         error =
@@ -551,7 +561,7 @@ defmodule Kernel.Typespec do
     end
 
     {fields, state} = :lists.mapfoldl(fun, state, fields)
-    {{:type, line(meta), :map, fields}, state}
+    {{:type, location(meta), :map, fields}, state}
   end
 
   defp typespec({:%, _, [name, {:%{}, meta, fields}]} = node, vars, caller, state) do
@@ -644,7 +654,7 @@ defmodule Kernel.Typespec do
     {right, state} = typespec(right, vars, caller, state)
     :ok = validate_range(left, right, caller)
 
-    {{:type, line(meta), :range, [left, right]}, state}
+    {{:type, location(meta), :range, [left, right]}, state}
   end
 
   # Handle special forms
@@ -668,7 +678,7 @@ defmodule Kernel.Typespec do
         pair -> pair
       end
 
-    {{:type, line(meta), :fun, fun_args}, state}
+    {{:type, location(meta), :fun, fun_args}, state}
   end
 
   # Handle type operator
@@ -691,10 +701,10 @@ defmodule Kernel.Typespec do
         # This may be generating an invalid typespec but we need to generate it
         # to avoid breaking existing code that was valid but only broke Dialyzer
         {right, state} = typespec(expr, vars, caller, state)
-        {{:ann_type, line(meta), [{:var, line(var_meta), var_name}, right]}, state}
+        {{:ann_type, location(meta), [{:var, location(var_meta), var_name}, right]}, state}
 
       {right, state} ->
-        {{:ann_type, line(meta), [{:var, line(var_meta), var_name}, right]}, state}
+        {{:ann_type, location(meta), [{:var, location(var_meta), var_name}, right]}, state}
     end
   end
 
@@ -723,13 +733,13 @@ defmodule Kernel.Typespec do
     {left, state} = typespec(left, vars, caller, state)
     state = %{state | undefined_type_error_enabled?: true}
     {right, state} = typespec(right, vars, caller, state)
-    {{:ann_type, line(meta), [left, right]}, state}
+    {{:ann_type, location(meta), [left, right]}, state}
   end
 
   # Handle unary ops
   defp typespec({op, meta, [integer]}, _, _, state) when op in [:+, :-] and is_integer(integer) do
-    line = line(meta)
+    location = location(meta)
-    {{:op, line, op, {:integer, line, integer}}, state}
+    {{:op, location, op, {:integer, location, integer}}, state}
   end
 
   # Handle remote calls in the form of @module_attribute.type.
@@ -778,12 +788,12 @@ defmodule Kernel.Typespec do
 
   # Handle tuples
   defp typespec({:tuple, meta, []}, _vars, _caller, state) do
-    {{:type, line(meta), :tuple, :any}, state}
+    {{:type, location(meta), :tuple, :any}, state}
   end
 
   defp typespec({:{}, meta, t}, vars, caller, state) when is_list(t) do
     {args, state} = :lists.mapfoldl(&typespec(&1, vars, caller, &2), state, t)
-    {{:type, line(meta), :tuple, args}, state}
+    {{:type, location(meta), :tuple, args}, state}
   end
 
   defp typespec({left, right}, vars, caller, state) do
@@ -799,7 +809,7 @@ defmodule Kernel.Typespec do
   defp typespec({name, meta, atom}, vars, caller, state) when is_atom(atom) do
     if :lists.member(name, vars) do
       state = update_local_vars(state, name)
-      {{:var, line(meta), name}, state}
+      {{:var, location(meta), name}, state}
     else
       typespec({name, meta, []}, vars, caller, state)
     end
@@ -814,7 +824,7 @@ defmodule Kernel.Typespec do
 
     IO.warn(warning, caller)
     {args, state} = :lists.mapfoldl(&typespec(&1, vars, caller, &2), state, args)
-    {{:type, line(meta), :string, args}, state}
+    {{:type, location(meta), :string, args}, state}
   end
 
   defp typespec({:nonempty_string, meta, args}, vars, caller, state) do
@@ -825,7 +835,7 @@ defmodule Kernel.Typespec do
 
     IO.warn(warning, caller)
     {args, state} = :lists.mapfoldl(&typespec(&1, vars, caller, &2), state, args)
-    {{:type, line(meta), :nonempty_string, args}, state}
+    {{:type, location(meta), :nonempty_string, args}, state}
   end
 
   defp typespec({type, _meta, []}, vars, caller, state) when type in [:charlist, :char_list] do
@@ -855,7 +865,7 @@ defmodule Kernel.Typespec do
 
   defp typespec({:fun, meta, args}, vars, caller, state) do
     {args, state} = :lists.mapfoldl(&typespec(&1, vars, caller, &2), state, args)
-    {{:type, line(meta), :fun, args}, state}
+    {{:type, location(meta), :fun, args}, state}
   end
 
   defp typespec({:..., _meta, _args}, _vars, caller, _state) do
@@ -872,7 +882,7 @@ defmodule Kernel.Typespec do
 
     case :erl_internal.is_type(name, arity) do
       true ->
-        {{:type, line(meta), name, args}, state}
+        {{:type, location(meta), name, args}, state}
 
       false ->
         if state.undefined_type_error_enabled? and
@@ -890,7 +900,7 @@ defmodule Kernel.Typespec do
             %{state | used_type_pairs: [{name, arity} | state.used_type_pairs]}
           end
 
-        {{:user_type, line(meta), name, args}, state}
+        {{:user_type, location(meta), name, args}, state}
     end
   end
 
@@ -963,7 +973,7 @@ defmodule Kernel.Typespec do
 
   defp remote_type({remote, meta, name, args}, vars, caller, state) do
     {args, state} = :lists.mapfoldl(&typespec(&1, vars, caller, &2), state, args)
-    {{:remote_type, line(meta), [remote, name, args]}, state}
+    {{:remote_type, location(meta), [remote, name, args]}, state}
   end
 
   defp collect_union({:|, _, [a, b]}), do: [a | collect_union(b)]
@@ -996,16 +1006,16 @@ defmodule Kernel.Typespec do
   end
 
   defp fn_args(meta, [{:..., _, _}], _vars, _caller, state) do
-    {{:type, line(meta), :any}, state}
+    {{:type, location(meta), :any}, state}
   end
 
   defp fn_args(meta, args, vars, caller, state) do
     {args, state} = :lists.mapfoldl(&typespec(&1, vars, caller, &2), state, args)
-    {{:type, line(meta), :product, args}, state}
+    {{:type, location(meta), :product, args}, state}
   end
 
   defp variable({name, meta, args}) when is_atom(name) and is_atom(args) do
-    {:var, line(meta), name}
+    {:var, location(meta), name}
   end
 
   defp variable(expr), do: expr

lib/elixir/lib/kernel/utils.ex

@@ -36,14 +36,14 @@ defmodule Kernel.Utils do
     if is_list(funs) do
       IO.warn(
         "passing a list to Kernel.defdelegate/2 is deprecated, please define each delegate separately",
-        Macro.Env.stacktrace(env)
+        env
       )
     end
 
     if Keyword.has_key?(opts, :append_first) do
       IO.warn(
         "Kernel.defdelegate/2 :append_first option is deprecated",
-        Macro.Env.stacktrace(env)
+        env
       )
     end
 

lib/elixir/lib/macro.ex

@@ -115,7 +115,7 @@ defmodule Macro do
     * `:from_brackets` - Used to determine whether a call to `Access.get/3` is from
       bracket syntax.
 
-    * `:from_interpolation` - Used to determine whether a call to `Access.get/3` is
+    * `:from_interpolation` - Used to determine whether a call to `Kernel.to_string/1` is
       from interpolation.
 
     * `:generated` - Whether the code should be considered as generated by
@@ -136,8 +136,9 @@ defmodule Macro do
 
     * `:closing` - contains metadata about the closing pair, such as a `}`
       in a tuple or in a map, or such as the closing `)` in a function call
-      with parens. The `:closing` does not delimit the end of expression if
+      with parens (when `:token_metadata` is true). If the function call
-      there are `:do` and `:end` metadata  (when `:token_metadata` is true)
+      has a do-end block attached to it, its metadata is found under the
+      `:do` and `:end` metadata
 
     * `:column` - the column number of the AST node (when `:columns` is true).
       Note column information is always discarded from quoted code.
@@ -154,8 +155,10 @@ defmodule Macro do
       `do`-`end` blocks (when `:token_metadata` is true)
 
     * `:end_of_expression` - denotes when the end of expression effectively
-      happens. Available for all expressions except the last one inside a
+      happens (when `:token_metadata` is true). This is only available for
-      `__block__` (when `:token_metadata` is true)
+      direct children of a `__block__`, and it is either the location of a
+      newline or of the `;` character. The last expression of `__block__`
+      does not have this metadata.
 
     * `:indentation` - indentation of a sigil heredoc
 

lib/elixir/lib/map.ex

@@ -147,7 +147,7 @@ defmodule Map do
 
   ## Examples
 
-      iex> Map.keys(%{a: 1, b: 2})
+      Map.keys(%{a: 1, b: 2})
       [:a, :b]
 
   """
@@ -161,7 +161,7 @@ defmodule Map do
 
   ## Examples
 
-      iex> Map.values(%{a: 1, b: 2})
+      Map.values(%{a: 1, b: 2})
       [1, 2]
 
   """

lib/elixir/lib/module.ex

@@ -326,8 +326,17 @@ defmodule Module do
   ### `@nifs` (since v1.16.0)
 
   A list of functions and their arities which will be overridden
-  by a native implementation (NIF). See the Erlang documentation
+  by a native implementation (NIF).
-  for more information: https://www.erlang.org/doc/man/erl_nif
+
+      defmodule MyLibrary.MyModule do
+        @nifs [foo: 1, bar: 2]
+
+        def foo(arg1), do: :erlang.nif_error(:not_loaded)
+        def bar(arg1, arg2), do: :erlang.nif_error(:not_loaded)
+      end
+
+  See the Erlang documentation for more information:
+  https://www.erlang.org/doc/man/erl_nif
 
   ### `@on_definition`
 
@@ -2124,8 +2133,7 @@ defmodule Module do
   end
 
   defp attribute_stack(module, line) do
-    file = String.to_charlist(Path.relative_to_cwd(:elixir_module.file(module)))
+    struct!(Macro.Env, module: module, file: :elixir_module.file(module), line: line)
-    [{module, :__MODULE__, 0, file: file, line: line}]
   end
 
   ## Helpers

lib/elixir/lib/module/parallel_checker.ex

@@ -167,8 +167,9 @@ defmodule Module.ParallelChecker do
 
   defp collect_results(count, diagnostics) do
     receive do
-      {:diagnostic, diagnostic} ->
+      {:diagnostic, %{file: file} = diagnostic, read_snippet} ->
-        diagnostic = format_diagnostic_file(diagnostic)
+        :elixir_errors.print_diagnostic(diagnostic, read_snippet)
+        diagnostic = %{diagnostic | file: file && Path.absname(file)}
         collect_results(count, [diagnostic | diagnostics])
 
       {__MODULE__, _module, new_diagnostics} ->
@@ -287,11 +288,6 @@ defmodule Module.ParallelChecker do
     end
   end
 
-  @doc false
-  def format_diagnostic_file(%{file: file} = diagnostic) do
-    %{diagnostic | file: file && Path.absname(file)}
-  end
-
   ## Warning helpers
 
   defp group_warnings(warnings) do
@@ -324,6 +320,7 @@ defmodule Module.ParallelChecker do
   defp to_diagnostic(message, {file, position, mfa}) when is_list(position) do
     %{
       severity: :warning,
+      source: file,
       file: file,
       position: position_to_tuple(position),
       message: IO.iodata_to_binary(message),
@@ -334,7 +331,7 @@ defmodule Module.ParallelChecker do
 
   defp position_to_tuple(position) do
     case position[:column] do
-      nil -> position[:line]
+      nil -> position[:line] || 0
       col -> {position[:line], col}
     end
   end

lib/elixir/lib/path.ex

@@ -44,17 +44,17 @@ defmodule Path do
   """
   @spec absname(t) :: binary
   def absname(path) do
-    absname(path, File.cwd!())
+    absname(path, &File.cwd!/0)
   end
 
   @doc """
   Builds a path from `relative_to` to `path`.
 
   If `path` is already an absolute path, `relative_to` is ignored. See also
-  `relative_to/3`.
+  `relative_to/3`. `relative_to` is either a path or an anonymous function,
+  which is invoked only when necessary, that returns a path.
 
-  Unlike `expand/2`, no attempt is made to
+  Unlike `expand/2`, no attempt is made to resolve `..`, `.` or `~`.
-  resolve `..`, `.` or `~`.
 
   ## Examples
 
@@ -65,19 +65,33 @@ defmodule Path do
       "bar/../x"
 
   """
-  @spec absname(t, t) :: binary
+  @spec absname(t, t | (-> t)) :: binary
   def absname(path, relative_to) do
     path = IO.chardata_to_string(path)
 
     case type(path) do
       :relative ->
+        relative_to =
+          if is_function(relative_to, 0) do
+            relative_to.()
+          else
+            relative_to
+          end
+
         absname_join([relative_to, path])
 
       :absolute ->
         absname_join([path])
 
       :volumerelative ->
-        relative_to = IO.chardata_to_string(relative_to)
+        relative_to =
+          if is_function(relative_to, 0) do
+            relative_to.()
+          else
+            relative_to
+          end
+          |> IO.chardata_to_string()
+
         absname_vr(split(path), split(relative_to), relative_to)
     end
   end
@@ -163,7 +177,7 @@ defmodule Path do
   """
   @spec expand(t) :: binary
   def expand(path) do
-    expand_dot(absname(expand_home(path), File.cwd!()))
+    expand_dot(absname(expand_home(path), &File.cwd!/0))
   end
 
   @doc """
@@ -192,7 +206,7 @@ defmodule Path do
   """
   @spec expand(t, t) :: binary
   def expand(path, relative_to) do
-    expand_dot(absname(absname(expand_home(path), expand_home(relative_to)), File.cwd!()))
+    expand_dot(absname(absname(expand_home(path), expand_home(relative_to)), &File.cwd!/0))
   end
 
   @doc """
@@ -410,6 +424,8 @@ defmodule Path do
   defp relative_to_unforced(_, _, original), do: join(original)
 
   defp relative_to_forced(path, path, _original), do: "."
+  defp relative_to_forced(["."], _path, _original), do: "."
+  defp relative_to_forced(path, ["."], _original), do: join(path)
   defp relative_to_forced([h | t1], [h | t2], original), do: relative_to_forced(t1, t2, original)
 
   # this should only happen if we have two paths on different drives on windows
@@ -745,9 +761,9 @@ defmodule Path do
   You may call `Path.expand/1` to normalize the path before invoking
   this function.
 
-  A character preceded by \ loses its special meaning.
+  A character preceded by `\\` loses its special meaning.
-  Note that \ must be written as \\ in a string literal.
+  Note that `\\` must be written as `\\\\` in a string literal.
-  For example, "\\?*" will match any filename starting with ?.
+  For example, `"\\\\?*"` will match any filename starting with `?.`.
 
   By default, the patterns `*` and `?` do not match files starting
   with a dot `.`. See the `:match_dot` option in the "Options" section

lib/elixir/lib/port.ex

@@ -79,6 +79,27 @@ defmodule Port do
   are for advanced usage within the VM. Also consider using `System.cmd/3`
   if all you want is to execute a program and retrieve its return value.
 
+  > #### Windows argument splitting and untrusted arguments {: .warning}
+  >
+  > On Unix systems, arguments are passed to a new operating system
+  > process as an array of strings but on Windows it is up to the child
+  > process to parse them and some Windows programs may apply their own
+  > rules, which are inconsistent with the standard C runtime `argv` parsing
+  >
+  > This is particularly troublesome when invoking `.bat` or `.com` files
+  > as these run implicitly through `cmd.exe`, whose argument parsing is
+  > vulnerable to malicious input and can be used to run arbitrary shell
+  > commands.
+  >
+  > Therefore, if you are running on Windows and you execute batch
+  > files or `.com` applications, you must not pass untrusted input as
+  > arguments to the program. You may avoid accidentally executing them
+  > by explicitly passing the extension of the program you want to run,
+  > such as `.exe`, and double check the program is indeed not a batch
+  > file or `.com` application.
+  >
+  > This affects both `spawn` and `spawn_executable`.
+
   ### spawn
 
   The `:spawn` tuple receives a binary that is going to be executed as a

lib/elixir/lib/process.ex

@@ -504,7 +504,7 @@ defmodule Process do
   If the process is already dead when calling `Process.monitor/1`, a
   `:DOWN` message is delivered immediately.
 
-  See ["The need for monitoring"](https://elixir-lang.org/getting-started/mix-otp/genserver.html#the-need-for-monitoring)
+  See ["The need for monitoring"](genservers.md#the-need-for-monitoring)
   for an example. See `:erlang.monitor/2` for more information.
 
   Inlined by the compiler.

lib/elixir/lib/protocol.ex

@@ -448,7 +448,7 @@ defmodule Protocol do
   ## Examples
 
       # Get Elixir's ebin directory path and retrieve all protocols
-      iex> path = :code.lib_dir(:elixir, :ebin)
+      iex> path = Application.app_dir(:elixir, "ebin")
       iex> mods = Protocol.extract_protocols([path])
       iex> Enumerable in mods
       true
@@ -477,7 +477,7 @@ defmodule Protocol do
   ## Examples
 
       # Get Elixir's ebin directory path and retrieve all protocols
-      iex> path = :code.lib_dir(:elixir, :ebin)
+      iex> path = Application.app_dir(:elixir, "ebin")
       iex> mods = Protocol.extract_impls(Enumerable, [path])
       iex> List in mods
       true

lib/elixir/lib/registry.ex

@@ -27,8 +27,8 @@ defmodule Registry do
   `Registry.start_link/1`, it can be used to register and access named
   processes using the `{:via, Registry, {registry, key}}` tuple:
 
-      {:ok, _} = Registry.start_link(keys: :unique, name: Registry.ViaTest)
+      {:ok, _} = Registry.start_link(keys: :unique, name: MyApp.Registry)
-      name = {:via, Registry, {Registry.ViaTest, "agent"}}
+      name = {:via, Registry, {MyApp.Registry, "agent"}}
       {:ok, _} = Agent.start_link(fn -> 0 end, name: name)
       Agent.get(name, & &1)
       #=> 0
@@ -39,22 +39,22 @@ defmodule Registry do
   In the previous example, we were not interested in associating a value to the
   process:
 
-      Registry.lookup(Registry.ViaTest, "agent")
+      Registry.lookup(MyApp.Registry, "agent")
       #=> [{self(), nil}]
 
   However, in some cases it may be desired to associate a value to the process
   using the alternate `{:via, Registry, {registry, key, value}}` tuple:
 
-      {:ok, _} = Registry.start_link(keys: :unique, name: Registry.ViaTest)
+      {:ok, _} = Registry.start_link(keys: :unique, name: MyApp.Registry)
-      name = {:via, Registry, {Registry.ViaTest, "agent", :hello}}
+      name = {:via, Registry, {MyApp.Registry, "agent", :hello}}
       {:ok, agent_pid} = Agent.start_link(fn -> 0 end, name: name)
-      Registry.lookup(Registry.ViaTest, "agent")
+      Registry.lookup(MyApp.Registry, "agent")
       #=> [{agent_pid, :hello}]
 
   To this point, we have been starting `Registry` using `start_link/1`.
   Typically the registry is started as part of a supervision tree though:
 
-      {Registry, keys: :unique, name: Registry.ViaTest}
+      {Registry, keys: :unique, name: MyApp.Registry}
 
   Only registries with unique keys can be used in `:via`. If the name is
   already taken, the case-specific `start_link` function (`Agent.start_link/2`
@@ -1301,16 +1301,16 @@ defmodule Registry do
       iex> Registry.start_link(keys: :unique, name: Registry.SelectAllTest)
       iex> {:ok, _} = Registry.register(Registry.SelectAllTest, "hello", :value)
       iex> {:ok, _} = Registry.register(Registry.SelectAllTest, "world", :value)
-      iex> Registry.select(Registry.SelectAllTest, [{{:"$1", :"$2", :"$3"}, [], [{{:"$1", :"$2", :"$3"}}]}])
+      iex> Registry.select(Registry.SelectAllTest, [{{:"$1", :"$2", :"$3"}, [], [{{:"$1", :"$2", :"$3"}}]}]) |> Enum.sort()
-      [{"world", self(), :value}, {"hello", self(), :value}]
+      [{"hello", self(), :value}, {"world", self(), :value}]
 
   Get all keys in the registry:
 
       iex> Registry.start_link(keys: :unique, name: Registry.SelectAllTest)
       iex> {:ok, _} = Registry.register(Registry.SelectAllTest, "hello", :value)
       iex> {:ok, _} = Registry.register(Registry.SelectAllTest, "world", :value)
-      iex> Registry.select(Registry.SelectAllTest, [{{:"$1", :_, :_}, [], [:"$1"]}])
+      iex> Registry.select(Registry.SelectAllTest, [{{:"$1", :_, :_}, [], [:"$1"]}]) |> Enum.sort()
-      ["world", "hello"]
+      ["hello", "world"]
 
   """
   @doc since: "1.9.0"

lib/elixir/lib/string.ex

@@ -966,6 +966,7 @@ defmodule String do
 
   def capitalize(string, mode) when is_binary(string) do
     case :unicode_util.gc(string) do
+      [gc] -> grapheme_to_binary(:string.titlecase([gc]))
       [gc, rest] -> grapheme_to_binary(:string.titlecase([gc])) <> downcase(rest, mode)
       [gc | rest] -> grapheme_to_binary(:string.titlecase([gc])) <> downcase(rest, mode)
       [] -> ""
@@ -1871,6 +1872,104 @@ defmodule String do
     end
   end
 
+  defguardp replace_invalid_ii_of_iii(i, ii)
+            when Bitwise.bor(Bitwise.bsl(i, 6), ii) in 32..863 or
+                   Bitwise.bor(Bitwise.bsl(i, 6), ii) in 896..1023
+
+  defguardp replace_invalid_ii_of_iv(i, ii)
+            when Bitwise.bor(Bitwise.bsl(i, 6), ii) in 16..271
+
+  defguardp replace_invalid_iii_of_iv(i, ii, iii)
+            when Bitwise.bor(Bitwise.bor(Bitwise.bsl(i, 12), Bitwise.bsl(ii, 6)), iii) in 1024..17407
+
+  defguardp replace_invalid_is_next(next) when Bitwise.bsr(next, 6) !== 0b10
+
+  @doc ~S"""
+  Returns a new string created by replacing all invalid bytes with `replacement` (`"�"` by default).
+
+  ## Examples
+
+      iex> String.replace_invalid("asd" <> <<0xFF::8>>)
+      "asd�"
+
+      iex> String.replace_invalid("nem rán bề bề")
+      "nem rán bề bề"
+
+      iex> String.replace_invalid("nem rán b" <> <<225, 187>> <> " bề")
+      "nem rán b� bề"
+
+      iex> String.replace_invalid("nem rán b" <> <<225, 187>> <> " bề", "ERROR!")
+      "nem rán bERROR! bề"
+  """
+  @doc since: "1.16.0"
+  def replace_invalid(bytes, replacement \\ "�")
+      when is_binary(bytes) and is_binary(replacement) do
+    do_replace_invalid(bytes, replacement, <<>>)
+  end
+
+  # Valid ASCII (for better average speed)
+  defp do_replace_invalid(<<ascii::8, next::8, _::bytes>> = rest, rep, acc)
+       when ascii in 0..127 and replace_invalid_is_next(next) do
+    <<_::8, rest::bytes>> = rest
+    do_replace_invalid(rest, rep, acc <> <<ascii::8>>)
+  end
+
+  # Valid UTF-8
+  defp do_replace_invalid(<<grapheme::utf8, rest::bytes>>, rep, acc) do
+    do_replace_invalid(rest, rep, acc <> <<grapheme::utf8>>)
+  end
+
+  # 2/3 truncated sequence
+  defp do_replace_invalid(<<0b1110::4, i::4, 0b10::2, ii::6>>, rep, acc)
+       when replace_invalid_ii_of_iii(i, ii) do
+    acc <> rep
+  end
+
+  defp do_replace_invalid(<<0b1110::4, i::4, 0b10::2, ii::6, next::8, _::bytes>> = rest, rep, acc)
+       when replace_invalid_ii_of_iii(i, ii) and replace_invalid_is_next(next) do
+    <<_::16, rest::bytes>> = rest
+    do_replace_invalid(rest, rep, acc <> rep)
+  end
+
+  # 2/4
+  defp do_replace_invalid(<<0b11110::5, i::3, 0b10::2, ii::6>>, rep, acc)
+       when replace_invalid_ii_of_iv(i, ii) do
+    acc <> rep
+  end
+
+  defp do_replace_invalid(
+         <<0b11110::5, i::3, 0b10::2, ii::6, next::8, _::bytes>> = rest,
+         rep,
+         acc
+       )
+       when replace_invalid_ii_of_iv(i, ii) and replace_invalid_is_next(next) do
+    <<_::16, rest::bytes>> = rest
+    do_replace_invalid(rest, rep, acc <> rep)
+  end
+
+  # 3/4
+  defp do_replace_invalid(<<0b11110::5, i::3, 0b10::2, ii::6, 0b10::2, iii::6>>, rep, acc)
+       when replace_invalid_iii_of_iv(i, ii, iii) do
+    acc <> rep
+  end
+
+  defp do_replace_invalid(
+         <<0b11110::5, i::3, 0b10::2, ii::6, 0b10::2, iii::6, next::8, _::bytes>> = rest,
+         rep,
+         acc
+       )
+       when replace_invalid_iii_of_iv(i, ii, iii) and replace_invalid_is_next(next) do
+    <<_::24, rest::bytes>> = rest
+    do_replace_invalid(rest, rep, acc <> rep)
+  end
+
+  # Everything else
+  defp do_replace_invalid(<<_, rest::bytes>>, rep, acc),
+    do: do_replace_invalid(rest, rep, acc <> rep)
+
+  # Final
+  defp do_replace_invalid(<<>>, _, acc), do: acc
+
   @doc ~S"""
   Splits the string into chunks of characters that share a common trait.
 

lib/elixir/lib/supervisor.ex

@@ -413,10 +413,10 @@ defmodule Supervisor do
   The difference between the two approaches is that a module-based
   supervisor gives you more direct control over how the supervisor
   is initialized. Instead of calling `Supervisor.start_link/2` with
-  a list of child specifications that are automatically initialized, we manually
+  a list of child specifications that are implicitly initialized for us,
-  initialize the children by calling `Supervisor.init/2` inside its
+  we must explicitly initialize the children by calling `Supervisor.init/2`
-  `c:init/1` callback. `Supervisor.init/2` accepts the same `:strategy`,
+  inside its `c:init/1` callback. `Supervisor.init/2` accepts the same
-  `:max_restarts`, and `:max_seconds` options as `start_link/2`.
+  `:strategy`, `:max_restarts`, and `:max_seconds` options as `start_link/2`.
 
   > #### `use Supervisor` {: .info}
   >

lib/elixir/lib/system.ex

@@ -1005,6 +1005,25 @@ defmodule System do
   `Port` module describes this problem and possible solutions under
   the "Zombie processes" section.
 
+  > #### Windows argument splitting and untrusted arguments {: .warning}
+  >
+  > On Unix systems, arguments are passed to a new operating system
+  > process as an array of strings but on Windows it is up to the child
+  > process to parse them and some Windows programs may apply their own
+  > rules, which are inconsistent with the standard C runtime `argv` parsing
+  >
+  > This is particularly troublesome when invoking `.bat` or `.com` files
+  > as these run implicitly through `cmd.exe`, whose argument parsing is
+  > vulnerable to malicious input and can be used to run arbitrary shell
+  > commands.
+  >
+  > Therefore, if you are running on Windows and you execute batch
+  > files or `.com` applications, you must not pass untrusted input as
+  > arguments to the program. You may avoid accidentally executing them
+  > by explicitly passing the extension of the program you want to run,
+  > such as `.exe`, and double check the program is indeed not a batch
+  > file or `.com` application.
+
   ## Examples
 
       iex> System.cmd("echo", ["hello"])

lib/elixir/lib/task.ex

@@ -40,14 +40,42 @@ defmodule Task do
        as they are *always* sent. If you are not expecting a reply,
        consider using `Task.start_link/1` as detailed below.
 
-    2. async tasks link the caller and the spawned process. This
+    2. Async tasks link the caller and the spawned process. This
        means that, if the caller crashes, the task will crash
        too and vice-versa. This is on purpose: if the process
        meant to receive the result no longer exists, there is
-       no purpose in completing the computation.
+       no purpose in completing the computation. If this is not
+       desired, you will want to use supervised tasks, described
+       in a subsequent section.
 
-       If this is not desired, you will want to use supervised
+  ## Tasks are processes
-       tasks, described next.
+
+  Tasks are processes and so data will need to be completely copied
+  to them. Take the following code as an example:
+
+      large_data = fetch_large_data()
+      task = Task.async(fn -> do_some_work(large_data) end)
+      res = do_some_other_work()
+      res + Task.await(task)
+
+  The code above copies over all of `large_data`, which can be
+  resource intensive depending on the size of the data.
+  There are two ways to address this.
+
+  First, if you need to access only part of `large_data`,
+  consider extracting it before the task:
+
+      large_data = fetch_large_data()
+      subset_data = large_data.some_field
+      task = Task.async(fn -> do_some_work(subset_data) end)
+
+  Alternatively, if you can move the data loading altogether
+  to the task, it may be even better:
+
+      task = Task.async(fn ->
+        large_data = fetch_large_data()
+        do_some_work(large_data)
+      end)
 
   ## Dynamically supervised tasks
 

lib/elixir/lib/task/supervisor.ex

@@ -152,7 +152,7 @@ defmodule Task.Supervisor do
   Starts a task that can be awaited on.
 
   The `supervisor` must be a reference as defined in `Supervisor`.
-  The task will still be linked to the caller, see `Task.async/3` for
+  The task will still be linked to the caller, see `Task.async/1` for
   more information and `async_nolink/3` for a non-linked variant.
 
   Raises an error if `supervisor` has reached the maximum number of
@@ -174,7 +174,7 @@ defmodule Task.Supervisor do
   Starts a task that can be awaited on.
 
   The `supervisor` must be a reference as defined in `Supervisor`.
-  The task will still be linked to the caller, see `Task.async/3` for
+  The task will still be linked to the caller, see `Task.async/1` for
   more information and `async_nolink/3` for a non-linked variant.
 
   Raises an error if `supervisor` has reached the maximum number of
@@ -196,7 +196,7 @@ defmodule Task.Supervisor do
   Starts a task that can be awaited on.
 
   The `supervisor` must be a reference as defined in `Supervisor`.
-  The task won't be linked to the caller, see `Task.async/3` for
+  The task won't be linked to the caller, see `Task.async/1` for
   more information.
 
   Raises an error if `supervisor` has reached the maximum number of
@@ -283,7 +283,7 @@ defmodule Task.Supervisor do
   Starts a task that can be awaited on.
 
   The `supervisor` must be a reference as defined in `Supervisor`.
-  The task won't be linked to the caller, see `Task.async/3` for
+  The task won't be linked to the caller, see `Task.async/1` for
   more information.
 
   Raises an error if `supervisor` has reached the maximum number of

lib/elixir/lib/uri.ex

@@ -362,22 +362,24 @@ defmodule URI do
   end
 
   @doc """
-  Percent-escapes all characters that require escaping in `string`.
+  Percent-encodes all characters that require escaping in `string`.
 
-  This means reserved characters, such as `:` and `/`, and the
+  By default, this function is meant to escape the whole URI, and
-  so-called unreserved characters, which have the same meaning both
+  therefore it will escape all characters which are foreign to the
-  escaped and unescaped, won't be escaped by default.
+  URI specification. Reserved characters (such as `:` and `/`) or
+  unreserved (such as letters and numbers) are not escaped.
+
+  Because different components of a URI require different escaping
+  rules, this function also accepts a `predicate` function as an optional
+  argument. If passed, this function will be called with each byte
+  in `string` as its argument and should return a truthy value (anything other
+  than `false` or `nil`) if the given byte should be left as is, or
+  return a falsy value (`false` or `nil`) if the character should be
+  escaped. Defaults to `URI.char_unescaped?/1`.
 
   See `encode_www_form/1` if you are interested in escaping reserved
   characters too.
 
-  This function also accepts a `predicate` function as an optional
-  argument. If passed, this function will be called with each byte
-  in `string` as its argument and should return a truthy value (anything other
-  than `false` or `nil`) if the given byte should be left as is, or return a
-  falsy value (`false` or `nil`) if the character should be escaped. Defaults
-  to `URI.char_unescaped?/1`.
-
   ## Examples
 
       iex> URI.encode("ftp://s-ite.tld/?value=put it+й")
@@ -654,16 +656,16 @@ defmodule URI do
         scheme = String.downcase(scheme, :ascii)
 
         case map do
-          %{port: port} when port != :undefined ->
+          %{port: port} when is_integer(port) ->
             %{uri | scheme: scheme}
 
           %{} ->
-            case default_port(scheme) do
+            %{uri | scheme: scheme, port: default_port(scheme)}
-              nil -> %{uri | scheme: scheme}
-              port -> %{uri | scheme: scheme, port: port}
-            end
         end
 
+      %{port: :undefined} ->
+        %{uri | port: nil}
+
       %{} ->
         uri
     end

lib/elixir/pages/anti-patterns/code-anti-patterns.md

@@ -1,8 +1,8 @@
 # Code-related anti-patterns
 
-This document outlines anti-patterns related to your code and particular Elixir idioms and features.
+This document outlines potential anti-patterns related to your code and particular Elixir idioms and features.
 
-## Comments
+## Comments overuse
 
 #### Problem
 
@@ -34,7 +34,8 @@ You could refactor the code above like this:
 @five_min_in_seconds 60 * 5
 
 defp unix_five_min_from_now do
-  unix_now = DateTime.to_unix(DateTime.utc_now(), :second)
+  now = DateTime.utc_now()
+  unix_now = DateTime.to_unix(now, :second)
   unix_now + @five_min_in_seconds
 end
 ```
@@ -45,75 +46,6 @@ We removed the unnecessary comments. We also added a `@five_min_in_seconds` modu
 
 Elixir makes a clear distinction between **documentation** and code comments. The language has built-in first-class support for documentation through `@doc`, `@moduledoc`, and more. See the ["Writing documentation"](../getting-started/writing-documentation.md) guide for more information.
 
-## Complex branching
-
-#### Problem
-
-When a function assumes the responsibility of handling multiple errors alone, it can increase its cyclomatic complexity (metric of control-flow) and become incomprehensible. This situation can configure a specific instance of "Long function", a traditional anti-pattern, but has implications of its own. Under these circumstances, this function could get very confusing, difficult to maintain and test, and therefore bug-proneness.
-
-#### Example
-
-An example of this anti-pattern is when a function uses the `case` control-flow structure or other similar constructs (for example, `cond` or `receive`) to handle variations of a return type. This practice can make the function more complex, long, and difficult to understand, as shown next.
-
-```elixir
-def get_customer(customer_id) do
-  case SomeHTTPClient.get("/customers/#{customer_id}") do
-    {:ok, %{status: 200, body: body}} ->
-      case Jason.decode(body) do
-        {:ok, decoded} ->
-          %{
-            "first_name" => first_name,
-            "last_name" => last_name,
-            "company" => company
-          } = decoded
-
-          customer =
-            %Customer{
-              id: customer_id,
-              name: "#{first_name} #{last_name}",
-              company: company
-            }
-
-          {:ok, customer}
-
-        {:error, _} ->
-          {:error, "invalid response body"}
-      end
-
-    {:error, %{status: status, body: body}} ->
-      case Jason.decode(body) do
-        %{"error" => message} when is_binary(message) ->
-          {:error, message}
-
-        %{} ->
-          {:error, "invalid response with status #{status}"}
-      end
-  end
-end
-```
-
-The code above is complex because the `case` clauses are long and often have their own branching logic in them. With the clauses spread out, it is hard to understand what each clause does individually and it is hard to see all of the different scenarios your code pattern matches on.
-
-#### Refactoring
-
-As shown below, in this situation, instead of concentrating all error handling within the same function, creating complex branches, it is better to delegate each branch to a different private function. In this way, the code will be cleaner and more readable.
-
-```elixir
-def get_customer(customer_id) do
-  case SomeHTTPClient.get("/customers/#{customer_id}") do
-    {:ok, %{status: 200, body: body}} ->
-      http_customer_to_struct(customer_id, body)
-
-    {:error, %{status: status, body: body}} ->
-      http_error(status, body)
-  end
-end
-```
-
-Both `http_customer_to_struct(customer_id, body)` and `http_error(status, body)` above contain the previous branches refactored into private functions.
-
-It is worth noting that this refactoring is trivial to perform in Elixir because clauses cannot define variables or otherwise affect their parent scope. Therefore, extracting any clause or branch to a private function is a matter of gathering all variables used in that branch and passing them as arguments to the new function.
-
 ## Complex `else` clauses in `with`
 
 #### Problem
@@ -169,11 +101,11 @@ end
 
 #### Problem
 
-When we use multi-clause functions, it is possible to extract values in the clauses for further usage and for pattern matching/guard checking. This extraction itself does not represent an anti-pattern, but when you have too many clauses or too many arguments, it becomes hard to know which extracted parts are used for pattern/guards and what is used only inside the function body. This anti-pattern is related to [Unrelated multi-clause function](design-anti-patterns.md#unrelated-multi-clause-function), but with implications of its own. It impairs the code readability in a different way.
+When we use multi-clause functions, it is possible to extract values in the clauses for further usage and for pattern matching/guard checking. This extraction itself does not represent an anti-pattern, but when you have *extractions made across several clauses and several arguments of the same function*, it becomes hard to know which extracted parts are used for pattern/guards and what is used only inside the function body. This anti-pattern is related to [Unrelated multi-clause function](design-anti-patterns.md#unrelated-multi-clause-function), but with implications of its own. It impairs the code readability in a different way.
 
 #### Example
 
-The multi-clause function `drive/1` is extracting fields of an `%User{}` struct for usage in the clause expression (`age`) and for usage in the function body (`name`). Ideally, a function should not mix pattern matching extractions for usage in its guard expressions and also in its body.
+The multi-clause function `drive/1` is extracting fields of an `%User{}` struct for usage in the clause expression (`age`) and for usage in the function body (`name`):
 
 ```elixir
 def drive(%User{name: name, age: age}) when age >= 18 do
@@ -185,7 +117,7 @@ def drive(%User{name: name, age: age}) when age < 18 do
 end
 ```
 
-While the example is small and looks like a clear code, try to imagine a situation where `drive/1` was more complex, having many more clauses, arguments, and extractions.
+While the example above is small and does not configure an anti-pattern, it is an example of mixed extraction and pattern matching. A situation where `drive/1` was more complex, having many more clauses, arguments, and extractions, would make it hard to know at a glance which variables are used for pattern/guards and which ones are not.
 
 #### Refactoring
 
@@ -288,7 +220,7 @@ defmodule MyRequestHandler do
 end
 ```
 
-All valid statuses all defined as atoms within the same module, and that's enough. If you want to be explicit, you could also have a function that lists them:
+All valid statuses are defined as atoms within the same module, and that's enough. If you want to be explicit, you could also have a function that lists them:
 
 ```elixir
 def valid_statuses do
@@ -319,7 +251,9 @@ end
 
 #### Refactoring
 
-To address this anti-pattern, related arguments can be grouped using maps, structs, or even tuples. This effectively reduces the number of arguments, simplifying the function's interface. In the case of `loan/6`, its arguments were grouped into two different maps, thereby reducing its arity to `loan/2`:
+To address this anti-pattern, related arguments can be grouped using key-value data structures, such as maps, structs, or even keyword lists in the case of optional arguments. This effectively reduces the number of arguments and the key-value data structures adds clarity to the caller.
+
+For this particular example, the arguments to `loan/6` can be grouped into two different maps, thereby reducing its arity to `loan/2`:
 
 ```elixir
 defmodule Library do
@@ -363,11 +297,13 @@ defmodule PlugAuth do
 end
 ```
 
+#### Additional remarks
+
 There are few known exceptions to this anti-pattern:
 
   * [Protocol implementations](`Kernel.defimpl/2`) are, by design, defined under the protocol namespace
 
-  * [Custom Mix tasks](`Mix.Task`) are always defined under the `Mix.Tasks` namespace, such as `Mix.Tasks.PlugAuth`
+  * In some scenarios, the namespace owner may allow exceptions to this rule. For example, in Elixir itself, you defined [custom Mix tasks](`Mix.Task`) by placing them under the `Mix.Tasks` namespace, such as `Mix.Tasks.PlugAuth`
 
   * If you are the maintainer for both `plug` and `plug_auth`, then you may allow `plug_auth` to define modules with the `Plug` namespace, such as `Plug.Auth`. However, you are responsible for avoiding or managing any conflicts that may arise in the future
 
@@ -375,9 +311,13 @@ There are few known exceptions to this anti-pattern:
 
 #### Problem
 
-In Elixir, it is possible to access values from `Map`s, which are key-value data structures, either statically or dynamically. When a key is expected to exist in the map, it must be accessed using the `map.key` notation, which asserts the key exists. If the key does not exist, an exception is raised (and in some situations also compiler warnings), allowing developers to catch bugs early on.
+In Elixir, it is possible to access values from `Map`s, which are key-value data structures, either statically or dynamically.
 
-`map[:key]` must be used with optional keys. This way, if the informed key does not exist, `nil` is returned. When used with required keys, this return can be confusing and allow `nil` values to pass through the system, while `map.key` would raise upfront. In this way, this anti-pattern may cause bugs in the code.
+When a key is expected to exist in a map, it must be accessed using the `map.key` notation, making it clear to developers (and the compiler) that the key must exist. If the key does not exist, an exception is raised (and in some cases also compiler warnings). This is also known as the static notation, as the key is known at the time of writing the code.
+
+When a key is optional, the `map[:key]` notation must be used instead. This way, if the informed key does not exist, `nil` is returned. This is the dynamic notation, as it also supports dynamic key access, such as `map[some_var]`.
+
+When you use `map[:key]` to access a key that always exists in the map, you are making the code less clear for developers and for the compiler, as they now need to work with the assumption the key may not be there. This mismatch may also make it harder to track certain bugs. If the key is unexpectedly missing, you will have a `nil` value propagate through the system, instead of raising on map access.
 
 #### Example
 
@@ -387,8 +327,6 @@ The function `plot/1` tries to draw a graphic to represent the position of a poi
 defmodule Graphics do
   def plot(point) do
     # Some other code...
-
-    # Dynamic access to use point values
     {point[:x], point[:y], point[:z]}
   end
 end
@@ -397,90 +335,77 @@ end
 ```elixir
 iex> point_2d = %{x: 2, y: 3}
 %{x: 2, y: 3}
-iex> point_3d = %{x: 5, y: 6, z: nil}
+iex> point_3d = %{x: 5, y: 6, z: 7}
-%{x: 5, y: 6, z: nil}
+%{x: 5, y: 6, z: 7}
 iex> Graphics.plot(point_2d)
-{2, 3, nil}   # <= ambiguous return
+{2, 3, nil}
 iex> Graphics.plot(point_3d)
-{5, 6, nil}
+{5, 6, 7}
 ```
 
-As can be seen in the example above, even when the key `:z` does not exist in the map (`point_2d`), dynamic access returns the value `nil`. This return can be dangerous because of its ambiguity. It is not possible to conclude from it whether the map has the key `:z` or not. If the function relies on the return value to make decisions about how to plot a point, this can be problematic and even cause errors when testing the code.
+Given we want to plot both 2D and 3D points, the behaviour above is expected. But what happens if we forget to pass a point with either `:x` or `:y`?
+
+```elixir
+iex> bad_point = %{y: 3, z: 4}
+%{y: 3, z: 4}
+iex> Graphics.plot(bad_point)
+{nil, 3, 4}
+```
+
+The behaviour above is unexpected because our function should not work with points without a `:x` key. This leads to subtle bugs, as we may now pass `nil` to another function, instead of raising early on.
 
 #### Refactoring
 
-To remove this anti-pattern, whenever accessing an existing key of `Atom` type in the map, replace the dynamic `map[:key]` syntax by the static `map.key` notation. This way, when a non-existent key is accessed, Elixir raises an error immediately, allowing developers to find bugs faster. The next code illustrates the refactoring of `plot/1`, removing this anti-pattern:
+To remove this anti-pattern, we must use the dynamic `map[:key]` syntax and the static `map.key` notation according to our requirements. We expect `:x` and `:y` to always exist, but not `:z`. The next code illustrates the refactoring of `plot/1`, removing this anti-pattern:
 
 ```elixir
 defmodule Graphics do
   def plot(point) do
     # Some other code...
-
+    {point.x, point.y, point[:z]}
-    # Strict access to use point values
-    {point.x, point.y, point.z}
   end
 end
 ```
 
 ```elixir
-iex> point_2d = %{x: 2, y: 3}
-%{x: 2, y: 3}
-iex> point_3d = %{x: 5, y: 6, z: nil}
-%{x: 5, y: 6, z: nil}
 iex> Graphics.plot(point_2d)
-** (KeyError) key :z not found in: %{x: 2, y: 3} # <= explicitly warns that
+{2, 3, nil}
-  graphic.ex:6: Graphics.plot/1                  # <= the :z key does not exist!
+iex> Graphics.plot(bad_point)
-iex> Graphics.plot(point_3d)
+** (KeyError) key :x not found in: %{y: 3, z: 4} # <= explicitly warns that
-{5, 6, nil}
+  graphic.ex:4: Graphics.plot/1                  # <= the :x key does not exist!
 ```
 
 Overall, the usage of `map.key` and `map[:key]` encode important information about your data structure, allowing developers to be clear about their intent. See both `Map` and `Access` module documentation for more information and examples.
 
-Another alternative to refactor this anti-pattern is to use pattern matching:
+An alternative to refactor this anti-pattern is to use pattern matching, defining explicit clauses for 2d vs 3d points:
 
 ```elixir
 defmodule Graphics do
+  # 3d
   def plot(%{x: x, y: y, z: z}) do
     # Some other code...
-
-    # Strict access to use point values
     {x, y, z}
   end
+
+  # 2d
+  def plot(%{x: x, y: y}) do
+    # Some other code...
+    {x, y}
+  end
 end
 ```
 
-```elixir
+Pattern-matching is specially useful when matching over multiple keys as well as on the values themselves at once.
-iex> point_2d = %{x: 2, y: 3}
-%{x: 2, y: 3}
-iex> point_3d = %{x: 5, y: 6, z: nil}
-%{x: 5, y: 6, z: nil}
-iex> Graphics.plot(point_2d)
-** (FunctionClauseError)  no function clause matching in Graphics.plot/1
-  graphic.ex:2: Graphics.plot/1                  # <= the :z key does not exist!
-iex> Graphics.plot(point_3d)
-{5, 6, nil}
-```
 
-Another alternative is to use structs. By default, structs only support static access to its fields:
+Another option is to use structs. By default, structs only support static access to its fields. In such scenarios, you may consider defining structs for both 2D and 3D points:
 
 ```elixir
-defmodule Point.2D do
+defmodule Point2D do
   @enforce_keys [:x, :y]
   defstruct [x: nil, y: nil]
 end
 ```
 
-```elixir
-iex> point = %Point.2D{x: 2, y: 3}
-%Point.2D{x: 2, y: 3}
-iex> point.x   # <= strict access to use point values
-2
-iex> point.z   # <= trying to access a non-existent key
-** (KeyError) key :z not found in: %Point{x: 2, y: 3}
-iex> point[:x] # <= by default, struct does not support dynamic access
-** (UndefinedFunctionError) ... (Point does not implement the Access behaviour)
-```
-
 Generally speaking, structs are useful when sharing data structures across modules, at the cost of adding a compile time dependency between these modules. If module `A` uses a struct defined in module `B`, `A` must be recompiled if the fields in the struct `B` change.
 
 #### Additional remarks
@@ -562,7 +487,7 @@ case some_function(arg) do
 end
 ```
 
-In particular, avoid matching solely on `_`, as shown below, as it is less clear in intent and it may hide bugs if `some_function/1` adds new return values in the future:
+In particular, avoid matching solely on `_`, as shown below:
 
 ```elixir
 case some_function(arg) do
@@ -571,6 +496,8 @@ case some_function(arg) do
 end
 ```
 
+ Matching on `_` is less clear in intent and it may hide bugs if `some_function/1` adds new return values in the future.
+
 #### Additional remarks
 
 This anti-pattern was formerly known as [Speculative assumptions](https://github.com/lucasvegi/Elixir-Code-Smells#speculative-assumptions).

lib/elixir/pages/anti-patterns/design-anti-patterns.md

@@ -1,7 +1,6 @@
 # Design-related anti-patterns
 
-This document outlines anti-patterns related to your modules, functions, and the role they
+This document outlines potential anti-patterns related to your modules, functions, and the role they play within a codebase.
-play within a codebase.
 
 ## Alternative return types
 
@@ -31,7 +30,7 @@ end
 
 ```elixir
 iex> AlternativeInteger.parse("13")
-{13, ""}
+13
 iex> AlternativeInteger.parse("13", discard_rest: true)
 13
 iex> AlternativeInteger.parse("13", discard_rest: false)
@@ -126,7 +125,7 @@ Remember booleans are internally represented as atoms. Therefore there is no per
 
 #### Problem
 
-This anti-pattern refers to code that uses exceptions for control flow. Exception handling itself does not represent an anti-pattern, but developers must prefer to use `case` and pattern matching to change the flow of their code, instead of `try/rescue`. In turn, library authors should provide developers with APIs to handle errors without relying on exception handling. When developers have no freedom to decide if an error is exceptional or not, this is considered an anti-pattern.
+This anti-pattern refers to code that uses `Exception`s for control flow. Exception handling itself does not represent an anti-pattern, but developers must prefer to use `case` and pattern matching to change the flow of their code, instead of `try/rescue`. In turn, library authors should provide developers with APIs to handle errors without relying on exception handling. When developers have no freedom to decide if an error is exceptional or not, this is considered an anti-pattern.
 
 #### Example
 
@@ -168,7 +167,7 @@ defmodule MyModule do
 end
 ```
 
-This is only possible because the `File` module provides APIs for reading files with tuples as results (`File.read/1`), as well as a version that raises an exception (`File.read!/1`). The bang (exclamation point) is effectively part of [Elixir's naming conventions](naming-conventions.html#trailing-bang-foo).
+This is only possible because the `File` module provides APIs for reading files with tuples as results (`File.read/1`), as well as a version that raises an exception (`File.read!/1`). The bang (exclamation point) is effectively part of [Elixir's naming conventions](naming-conventions.md#trailing-bang-foo).
 
 Library authors are encouraged to follow the same practices. In practice, the bang variant is implemented on top of the non-raising version of the code. For example, `File.read!/1` is implemented as:
 
@@ -186,63 +185,15 @@ end
 
 A common practice followed by the community is to make the non-raising version return `{:ok, result}` or `{:error, Exception.t}`. For example, an HTTP client may return `{:ok, %HTTP.Response{}}` on success cases and `{:error, %HTTP.Error{}}` for failures, where `HTTP.Error` is [implemented as an exception](`Kernel.defexception/1`). This makes it convenient for anyone to raise an exception by simply calling `Kernel.raise/1`.
 
-## Feature envy
+#### Additional remarks
 
-#### Problem
+This anti-pattern was formerly known as [Using exceptions for control-flow](https://github.com/lucasvegi/Elixir-Code-Smells#using-exceptions-for-control-flow).
-
-This anti-pattern occurs when a function accesses more data or calls more functions from another module than from its own. The presence of this anti-pattern can make a module less cohesive and increase code coupling.
-
-#### Example
-
-In the following code, all the data used in the `calculate_total_item/1` function of the module `Order` comes from the `OrderItem` module. This increases coupling and decreases code cohesion unnecessarily.
-
-```elixir
-defmodule Order do
-  # Some functions...
-
-  def calculate_total_item(id) do
-    item = OrderItem.find_item(id)
-    total = (item.price + item.taxes) * item.amount
-
-    if discount = OrderItem.find_discount(item) do
-      total - total * discount
-    else
-      total
-    end
-  end
-end
-```
-
-#### Refactoring
-
-To remove this anti-pattern we can move `calculate_total_item/1` to `OrderItem`, decreasing coupling:
-
-```elixir
-defmodule OrderItem do
-  def find_item(id)
-  def find_discount(item)
-
-  def calculate_total_item(id) do   # <= function moved from Order!
-    item = find_item(id)
-    total = (item.price + item.taxes) * item.amount
-    discount = find_discount(item)
-
-    unless is_nil(discount) do
-      total - total * discount
-    else
-      total
-    end
-  end
-end
-```
-
-This refactoring is only possible when you own both modules. If the module you are invoking belongs to another application, then it is not possible to add new functions to it, and your only option is to define an additional module that augments the third-party module.
 
 ## Primitive obsession
 
 #### Problem
 
-This anti-pattern happens when Elixir basic types (for example, *integer*, *float*, and *string*) are abusively used in function parameters and code variables, rather than creating specific composite data types (for example, *tuples*, *maps*, and *structs*) that can better represent a domain.
+This anti-pattern happens when Elixir basic types (for example, *integer*, *float*, and *string*) are excessively used to carry structured information, rather than creating specific composite data types (for example, *tuples*, *maps*, and *structs*) that can better represent a domain.
 
 #### Example
 
@@ -250,17 +201,23 @@ An example of this anti-pattern is the use of a single *string* to represent an
 
 ```elixir
 defmodule MyApp do
-  def process_address(address) when is_binary(address) do
+  def extract_postal_code(address) when is_binary(address) do
-    # Do something with address...
+    # Extract postal code with address...
+  end
+
+  def fill_in_country(address) when is_binary(address) do
+    # Fill in missing country...
   end
 end
 ```
 
+While you may receive the `address` as a string from a database, web request, or a third-party, if you find yourself frequently manipulating or extracting information from the string, it is a good indicator you should convert the address into structured data:
+
 Another example of this anti-pattern is using floating numbers to model money and currency, when [richer data structures should be preferred](https://hexdocs.pm/ex_money/).
 
 #### Refactoring
 
-Possible solutions to this anti-pattern is to use maps or structs to model our address. The example below creates an `Address` struct, better representing this domain through a composite type. Additionally, we can modify the `process_address/1` function to accept a parameter of type `Address` instead of a *string*. With this modification, we can extract each field of this composite type individually when needed.
+Possible solutions to this anti-pattern is to use maps or structs to model our address. The example below creates an `Address` struct, better representing this domain through a composite type. Additionally, we introduce a `parse/1` function, that converts the string into an `Address`, which will simplify the logic of remainng functions. With this modification, we can extract each field of this composite type individually when needed.
 
 ```elixir
 defmodule Address do
@@ -270,129 +227,62 @@ end
 
 ```elixir
 defmodule MyApp do
-  def process_address(%Address{} = address) do
+  def parse(address) when is_binary(address) do
-    # Do something with address...
+    # Returns %Address{}
+  end
+
+  def extract_postal_code(%Address{} = address) do
+    # Extract postal code with address...
+  end
+
+  def fill_in_country(%Address{} = address) do
+    # Fill in missing country...
   end
 end
 ```
 
-## Propagating invalid data
-
-#### Problem
-
-This anti-pattern refers to a function that does not validate its parameters and propagates them to other functions, which can produce internal unexpected behavior. When an error is raised inside a function due to an invalid parameter value, it can be confusing for developers and make it harder to locate and fix the error.
-
-#### Example
-
-An example of this anti-pattern is when a function receives an invalid parameter and then passes it to other functions, either in the same library or in a third-party library. This can cause an error to be raised deep inside the call stack, which may be confusing for the developer who is working with invalid data. As shown next, the function `foo/1` is a user-facing API which doesn't validate its parameters at the boundary. In this way, it is possible that invalid data will be passed through, causing an error that is obscure and hard to debug.
-
-```elixir
-defmodule MyLibrary do
-  def foo(invalid_data) do
-    # Some other code...
-
-    MyLibrary.Internal.sum(1, invalid_data)
-  end
-end
-```
-
-```elixir
-iex> MyLibrary.foo(2)
-3
-iex> MyLibrary.foo("José") # With invalid data
-** (ArithmeticError) bad argument in arithmetic expression: 1 + "José"
-  :erlang.+(1, "José")
-  my_library.ex:4: MyLibrary.Internal.sum/2
-```
-
-#### Refactoring
-
-To remove this anti-pattern, the client code must validate input parameters at the boundary with the user, via guard clauses, pattern matching, or conditionals. This prevents errors from occurring elsewhere in the call stack, making them easier to understand and debug. This refactoring also allows libraries to be implemented without worrying about creating internal protection mechanisms. The next code snippet illustrates the refactoring of `foo/1`, removing this anti-pattern:
-
-```elixir
-defmodule MyLibrary do
-  def foo(data) when is_integer(data) do
-    # Some other code
-
-    MyLibrary.Internal.sum(1, data)
-  end
-end
-```
-
-```elixir
-iex> MyLibrary.foo(2) # With valid data
-3
-iex> MyLibrary.foo("José") # With invalid data
-** (FunctionClauseError) no function clause matching in MyLibrary.foo/1.
-The following arguments were given to MyLibrary.foo/1:
-
-      # 1
-      "José"
-
-  my_library.ex:2: MyLibrary.foo/1
-```
-
 ## Unrelated multi-clause function
 
 #### Problem
 
-Using multi-clause functions in Elixir, to group functions of the same name, is not an anti-pattern in itself. However, due to the great flexibility provided by this programming feature, some developers may abuse the number of guard clauses and pattern matches to group *unrelated* functionality.
+Using multi-clause functions is a powerful Elixir feature. However, some developers may abuse this feature to group *unrelated* functionality, which is an anti-pattern.
 
 #### Example
 
-A frequent example of this usage of multi-clause functions is when developers mix unrelated business logic into the same function definition. Such functions often have generic names or too broad specifications, making it difficult for maintainers and users of said functions to maintain and understand them.
+A frequent example of this usage of multi-clause functions occurs when developers mix unrelated business logic into the same function definition, in a way that the behaviour of each clause becomes completely distinct from the others. Such functions often have too broad specifications, making it difficult for other developers to understand and maintain them.
 
-Some developers may use documentation mechanisms such as `@doc` annotations to compensate for poor code readability, however the documentation itself may end-up full of conditionals to describe how the function behaves for each different argument combination.
+Some developers may use documentation mechanisms such as `@doc` annotations to compensate for poor code readability, however the documentation itself may end-up full of conditionals to describe how the function behaves for each different argument combination. This is a good indicator that the clauses are ultimately unrelated.
 
 ```elixir
 @doc """
 Updates a struct.
 
-If given a "sharp" product (metal or glass with empty count),
+If given a product, it will...
-it will...
-
-If given a blunt product, it will...
 
 If given an animal, it will...
 """
-def update(%Product{count: nil, material: material})
+def update(%Product{count: count, material: material})  do
-    when material in ["metal", "glass"] do
   # ...
 end
 
-def update(%Product{count: count, material: material})
+def update(%Animal{count: count, skin: skin})  do
-    when count > 0 and material not in ["metal", "glass"] do
-  # ...
-end
-
-def update(%Animal{count: 1, skin: skin})
-    when skin in ["fur", "hairy"] do
   # ...
 end
 ```
 
+If updating an animal is completely different from updating a product and requires a different set of rules, it may be worth splitting those over different functions or even different modules.
+
 #### Refactoring
 
-As shown below, a possible solution to this anti-pattern is to break the business rules that are mixed up in a single unrelated multi-clause function in several different simple functions. More precise names make the scope of the function clear. Each function can have a specific `@doc`, describing its behavior and parameters received. While this refactoring sounds simple, it can have a lot of impact on the function's current users, so be careful!
+As shown below, a possible solution to this anti-pattern is to break the business rules that are mixed up in a single unrelated multi-clause function in simple functions. Each function can have a specific name and `@doc`, describing its behavior and parameters received. While this refactoring sounds simple, it can impact the function's callers, so be careful!
 
 ```elixir
 @doc """
-Updates a "sharp" product.
+Updates a product.
 
 It will...
 """
-def update_sharp_product(%Product{count: nil, material: material})
+def update_product(%Product{count: count, material: material}) do
-    when material in ["metal", "glass"] do
-  # ...
-end
-
-@doc """
-Updates a "blunt" product.
-
-It will...
-"""
-def update_blunt_product(%Product{count: count, material: material})
-    when count > 0 and material not in ["metal", "glass"] do
   # ...
 end
 
@@ -401,12 +291,48 @@ Updates an animal.
 
 It will...
 """
-def update_animal(%Animal{count: 1, skin: skin})
+def update_animal(%Animal{count: count, skin: skin}) do
-    when skin in ["fur", "hairy"] do
   # ...
 end
 ```
 
+These functions may still be implemented with multiple clauses, as long as the clauses group related funtionality. For example, `update_product` could be in practice implemented as follows:
+
+```elixir
+def update_product(%Product{count: 0}) do
+  # ...
+end
+
+def update_product(%Product{material: material})
+    when material in ["metal", "glass"] do
+  # ...
+end
+
+def update_product(%Product{material: material})
+    when material not in ["metal", "glass"] do
+  # ...
+end
+```
+
+You can see this pattern in practice within Elixir itself. The `+/2` operator can add `Integer`s and `Float`s together, but not `String`s, which instead use the `<>/2` operator. In this sense, it is reasonable to handle integers and floats in the same operation, but strings are unrelated enough to deserve their own function.
+
+You will also find examples in Elixir of functions that work with any struct, which would seemingly be an occurrence of this anti-pattern, such as `struct/2`:
+
+```elixir
+iex> struct(URI.parse("/foo/bar"), path: "/bar/baz")
+%URI{
+  scheme: nil,
+  userinfo: nil,
+  host: nil,
+  port: nil,
+  path: "/bar/baz",
+  query: nil,
+  fragment: nil
+}
+```
+
+The difference here is that the `struct/2` function behaves precisely the same for any struct given, therefore there is no question of how the function handles different inputs. If the behaviour is clear and consistent for all inputs, then the anti-pattern does not take place.
+
 ## Using application configuration for libraries
 
 #### Problem
@@ -448,7 +374,7 @@ iex> DashSplitter.split("Lucas-Francisco-da-Matta-Vegi")
 
 #### Refactoring
 
-To remove this anti-pattern and make the library more adaptable and flexible, this type of configuration must be performed via parameters in function calls. The code shown below performs the refactoring of the `split/1` function by accepting [keyword lists](`Keyword`) as a new optional parameter. With this new parameter, it is possible to modify the default behavior of the function at the time of its call, allowing multiple different ways of using `split/2` within the same application:
+To remove this anti-pattern, this type of configuration should be performed using a parameter passed to the function. The code shown below performs the refactoring of the `split/1` function by accepting [keyword lists](`Keyword`) as a new optional parameter. With this new parameter, it is possible to modify the default behavior of the function at the time of its call, allowing multiple different ways of using `split/2` within the same application:
 
 ```elixir
 defmodule DashSplitter do
@@ -465,3 +391,83 @@ iex> DashSplitter.split("Lucas-Francisco-da-Matta-Vegi", [parts: 5])
 iex> DashSplitter.split("Lucas-Francisco-da-Matta-Vegi") #<= default config is used!
 ["Lucas", "Francisco-da-Matta-Vegi"]
 ```
+
+Of course, not all uses of the application environment by libraries are incorrect. One example is using configuration to replace a component (or dependency) of a library by another that must behave the exact same. Consider a library that needs to parse CSV files. The library author may pick one package to use as default parser but allow its users to swap to different implementations via the application environment. At the end of the day, choosing a different CSV parser should not change the outcome, and library authors can even enforce this by [defining behaviours](../references/typespecs.md#behaviours) with the exact semantics they expect.
+
+#### Additional remarks: Supervision trees
+
+In practice, libraries may require additional configuration beyond keyword lists. For example, if a library needs to start a supervision tree, how can the user of said library customize its supervision tree? Given the supervision tree itself is global (as it belongs to the library), library authors may be tempted to use the application configuration once more.
+
+One solution is for the library to provide its own child specification, instead of starting the supervision tree itself. This allows the user to start all necessary processes under its own supervision tree, potentially passing custom configuration options during initialization.
+
+You can see this pattern in practice in projects like [Nx](https://github.com/elixir-nx/nx) and [DNS Cluster](https://github.com/phoenixframework/dns_cluster). These libraries require that you list processes under your own supervision tree:
+
+```elixir
+children = [
+  {DNSCluster, query: "my.subdomain"}
+]
+```
+
+In such cases, if the users of `DNSCluster` need to configure DNSCluster per environment, they can be the ones reading from the application environment, without the library forcing them to:
+
+```elixir
+children = [
+  {DNSCluster, query: Application.get_env(:my_app, :dns_cluster_query) || :ignore}
+]
+```
+
+Some libraries, such as [Ecto](https://github.com/elixir-ecto/ecto), allow you to pass your application name as an option (called `:otp_app` or similar) and then automatically read the environment from *your* application. While this addresses the issue with the application environment being global, as they read from each individual application, it comes at the cost of some indirection, compared to the example above where users explicitly read their application environment from their own code, whenever desired.
+
+#### Additional remarks: Compile-time configuration
+
+A similar discussion entails compile-time configuration. What if a library author requires some configuration to be provided at compilation time?
+
+Once again, instead of forcing users of your library to provide compile-time configuration, you may want to allow users of your library to generate the code themselves. That's the approach taken by libraries such as [Ecto](https://github.com/elixir-ecto/ecto):
+
+```elixir
+defmodule MyApp.Repo do
+  use Ecto.Repo, adapter: Ecto.Adapters.Postgres
+end
+```
+
+Instead of forcing developers to share a single repository, Ecto allows its users to define as many repositories as they want. Given the `:adapter` configuration is required at compile-time, it is a required value on `use Ecto.Repo`. If developers want to configure the adapter per environment, then it is their choice:
+
+```elixir
+defmodule MyApp.Repo do
+  use Ecto.Repo, adapter: Application.compile_env(:my_app, :repo_adapter)
+end
+```
+
+On the other hand, [code generation comes with its own anti-patterns](macro-anti-patterns.md), and must be considered carefully. That's to say: while using the application environment for libraries is discouraged, especially compile-time configuration, in some cases they may be the best option. For example, consider a library needs to parse CSV or JSON files to generate code based on data files. In such cases, it is best to provide reasonable defaults and make them customizable via the application environment, instead of asking each user of your library to generate the exact same code.
+
+#### Additional remarks: Mix tasks
+
+For Mix tasks and related tools, it may be necessary to provide per-project configuration. For example, imagine you have a `:linter` project, which supports setting the output file and the verbosity level. You may choose to configure it through application environment:
+
+```elixir
+config :linter,
+  output_file: "/path/to/output.json",
+  verbosity: 3
+```
+
+However, `Mix` allows tasks to read per-project configuration via `Mix.Project.config/0`. In this case, you can configure the `:linter` directly in the `mix.exs` file:
+
+```elixir
+def project do
+  [
+    app: :my_app,
+    version: "1.0.0",
+    linter: [
+      output_file: "/path/to/output.json",
+      verbosity: 3
+    ],
+    ...
+  ]
+end
+```
+
+Additionally, if a Mix task is available, you can also accept these options as command line arguments (see `OptionParser`):
+
+```bash
+mix linter --output-file /path/to/output.json --verbosity 3
+```

lib/elixir/pages/anti-patterns/macro-anti-patterns.md

@@ -1,8 +1,8 @@
 # Meta-programming anti-patterns
 
-This document outlines anti-patterns related to meta-programming.
+This document outlines potential anti-patterns related to meta-programming.
 
-## Large code generation by macros
+## Large code generation
 
 #### Problem
 
@@ -24,7 +24,7 @@ defmodule Routes do
       end
 
       if not is_atom(handler) do
-        raise ArgumentError, "route must be a module"
+        raise ArgumentError, "handler must be a module"
       end
 
       @store_route_for_compilation {route, handler}
@@ -51,7 +51,7 @@ defmodule Routes do
     end
 
     if not is_atom(handler) do
-      raise ArgumentError, "route must be a module"
+      raise ArgumentError, "handler must be a module"
     end
 
     Module.put_attribute(module, :store_route_for_compilation, {route, handler})
@@ -63,7 +63,7 @@ end
 
 #### Problem
 
-**Macros** are powerful meta-programming mechanisms that can be used in Elixir to extend the language. While using macros is not an anti-pattern in itself, this meta-programming mechanism should only be used when absolutely necessary. Whenever a macro is used, but it would have been possible to solve the same problem using functions or other existing Elixir structures, the code becomes unnecessarily more complex and less readable. Because macros are more difficult to implement and reason about, their indiscriminate use can compromise the evolution of a system, reducing its maintainability.
+*Macros* are powerful meta-programming mechanisms that can be used in Elixir to extend the language. While using macros is not an anti-pattern in itself, this meta-programming mechanism should only be used when absolutely necessary. Whenever a macro is used, but it would have been possible to solve the same problem using functions or other existing Elixir structures, the code becomes unnecessarily more complex and less readable. Because macros are more difficult to implement and reason about, their indiscriminate use can compromise the evolution of a system, reducing its maintainability.
 
 #### Example
 
@@ -165,7 +165,7 @@ error: imported ModuleA.foo/0 conflicts with local function
 
 #### Refactoring
 
-To remove this anti-pattern, we recommend library authors to avoid providing `__using__/1` callbacks whenever it can be replaced by `alias/1` or `import/1` directives. In the following code, we assume `use Library` is no longer available and `ClientApp` was refactored in this way, and with that, the code is clearer and the conflict as previously shown no longer exists:
+To remove this anti-pattern, we recommend library authors avoid providing `__using__/1` callbacks whenever it can be replaced by `alias/1` or `import/1` directives. In the following code, we assume `use Library` is no longer available and `ClientApp` was refactored in this way, and with that, the code is clearer and the conflict as previously shown no longer exists:
 
 ```elixir
 defmodule ClientApp do

lib/elixir/pages/anti-patterns/process-anti-patterns.md

@@ -1,12 +1,12 @@
 # Process-related anti-patterns
 
-This document outlines anti-patterns related to processes and process-based abstractions.
+This document outlines potential anti-patterns related to processes and process-based abstractions.
 
 ## Code organization by process
 
 #### Problem
 
-This anti-pattern refers to code that is unnecessarily organized by processes. A process itself does not represent an anti-pattern, but it should only be used to model runtime properties (such as concurrency, access to shared resources, and event scheduling). When you use a process for code organization, it can create bottlenecks in the system.
+This anti-pattern refers to code that is unnecessarily organized by processes. A process itself does not represent an anti-pattern, but it should only be used to model runtime properties (such as concurrency, access to shared resources, error isolation, etc). When you use a process for code organization, it can create bottlenecks in the system.
 
 #### Example
 
@@ -188,6 +188,60 @@ iex> Foo.Bucket.get(bucket, "milk")
 
 This anti-pattern was formerly known as [Agent obsession](https://github.com/lucasvegi/Elixir-Code-Smells/tree/main#agent-obsession).
 
+## Sending unnecessary data
+
+#### Problem
+
+Sending a message to a process can be an expensive operation if the message is big enough. That's because that message will be fully copied to the receiving process, which may be CPU and memory intensive. This is due to Erlang's "share nothing" architecture, where each process has its own memory, which simplifies and speeds up garbage collection.
+
+This is more obvious when using `send/2`, `GenServer.call/3`, or the initial data in `GenServer.start_link/3`. Notably this also happens when using `spawn/1`, `Task.async/1`, `Task.async_stream/3`, and so on. It is more subtle here as the anonymous function passed to these functions captures the variables it references, and all captured variables will be copied over. By doing this, you can accidentally send way more data to a process than you actually need.
+
+#### Example
+
+Imagine you were to implement some simple reporting of IP addresses that made requests against your application. You want to do this asynchronously and not block processing, so you decide to use `spawn/1`. It may seem like a good idea to hand over the whole connection because we might need more data later. However passing the connection results in copying a lot of unnecessary data like the request body, params, etc.
+
+```elixir
+# log_request_ip send the ip to some external service
+spawn(fn -> log_request_ip(conn) end)
+```
+
+This problem also occurs when accessing only the relevant parts:
+
+```elixir
+spawn(fn -> log_request_ip(conn.remote_ip) end)
+```
+
+This will still copy over all of `conn`, because the `conn` variable is being captured inside the spawned function. The function then extracts the `remote_ip` field, but only after the whole `conn` has been copied over.
+
+`send/2` and the `GenServer` APIs also rely on message passing. In the example below, the `conn` is once again copied to the underlying `GenServer`:
+
+```elixir
+GenServer.cast(pid, {:report_ip_address, conn})
+```
+
+#### Refactoring
+
+This anti-pattern has many potential remedies:
+
+  * Limit the data you send to the absolute necessary minimum instead of sending an entire struct. For example, don't send an entire `conn` struct if all you need is a couple of fields.
+
+  * If the only process that needs data is the one you are sending to, consider making the process fetch that data instead of passing it.
+
+  * Some abstractions, such as [`:persistent_term`](https://www.erlang.org/doc/man/persistent_term.html), allows you to share data between processes, as long as such data changes infrequently.
+
+In our case, limiting the input data is a reasonable strategy. If all we need *right now* is the IP address, then let's only work with that and make sure we're only passing the IP address into the closure, like so:
+
+```elixir
+ip_address = conn.remote_ip
+spawn(fn -> log_request_ip(ip_address) end)
+```
+
+Or in the `GenServer` case:
+
+```elixir
+GenServer.cast(pid, {:report_ip_address, conn.remote_ip})
+```
+
 ## Unsupervised processes
 
 #### Problem
@@ -207,7 +261,9 @@ defmodule Counter do
   use GenServer
 
   @doc "Starts a counter process."
-  def start(initial_value, name \\ __MODULE__) when is_integer(initial_value) do
+  def start_link(opts \\ []) do
+    initial_value = Keyword.get(opts, :initial_value, 0)
+    name = Keyword.get(opts, :name, __MODULE__)
     GenServer.start(__MODULE__, initial_value, name: name)
   end
 
@@ -217,7 +273,7 @@ defmodule Counter do
   end
 
   @doc "Bumps the value of the given counter."
-  def bump(value, pid_name \\ __MODULE__) do
+  def bump(pid_name \\ __MODULE__, value) do
     GenServer.call(pid_name, {:bump, value})
   end
 
@@ -238,23 +294,23 @@ end
 ```
 
 ```elixir
-iex> Counter.start(0)
+iex> Counter.start_link()
 {:ok, #PID<0.115.0>}
 iex> Counter.get()
 0
-iex> Counter.start(15, :other_counter)
+iex> Counter.start_link(initial_value: 15, name: :other_counter)
 {:ok, #PID<0.120.0>}
 iex> Counter.get(:other_counter)
 15
-iex> Counter.bump(-3, :other_counter)
+iex> Counter.bump(:other_counter, -3)
 12
-iex> Counter.bump(7)
+iex> Counter.bump(Counter, 7)
 7
 ```
 
 #### Refactoring
 
-To ensure that clients of a library have full control over their systems, regardless of the number of processes used and the lifetime of each one, all processes must be started inside a supervision tree. As shown below, this code uses a `Supervisor` as a supervision tree. When this Elixir application is started, two different counters (`Counter` and `:other_counter`) are also started as child processes of the `Supervisor` named `App.Supervisor`. Both are initialized to `0`. By means of this supervision tree, it is possible to manage the lifecycle of all child processes (stopping or restarting each one), improving the visibility of the entire app.
+To ensure that clients of a library have full control over their systems, regardless of the number of processes used and the lifetime of each one, all processes must be started inside a supervision tree. As shown below, this code uses a `Supervisor` as a supervision tree. When this Elixir application is started, two different counters (`Counter` and `:other_counter`) are also started as child processes of the `Supervisor` named `App.Supervisor`. One is initialized with `0`, the other with `15`. By means of this supervision tree, it is possible to manage the lifecycle of all child processes (stopping or restarting each one), improving the visibility of the entire app.
 
 ```elixir
 defmodule SupervisedProcess.Application do
@@ -263,8 +319,13 @@ defmodule SupervisedProcess.Application do
   @impl true
   def start(_type, _args) do
     children = [
-      %{id: Counter, start: {Counter, :start, [0]}},
+      # With the default values for counter and name
-      %{id: :other_counter, start: {Counter, :start, [0, :other_counter]}}
+      Counter,
+      # With custom values for counter, name, and a custom ID
+      Supervisor.child_spec(
+        {Counter, name: :other_counter, initial_value: 15},
+        id: :other_counter
+      )
     ]
 
     Supervisor.start_link(children, strategy: :one_for_one, name: App.Supervisor)
@@ -278,8 +339,8 @@ iex> Supervisor.count_children(App.Supervisor)
 iex> Counter.get(Counter)
 0
 iex> Counter.get(:other_counter)
-0
+15
-iex> Counter.bump(7, Counter)
+iex> Counter.bump(Counter, 7)
 7
 iex> Supervisor.terminate_child(App.Supervisor, Counter)
 iex> Supervisor.count_children(App.Supervisor) # Only one active child

lib/elixir/pages/anti-patterns/what-anti-patterns.md

@@ -1,12 +1,14 @@
 # What are anti-patterns?
 
-Anti-patterns describe common mistakes or indicators of potential problems in code.
+Anti-patterns describe common mistakes or indicators of problems in code.
 They are also known as "code smells".
 
-The goal of these guides is to document known anti-patterns found in Elixir software
+The goal of these guides is to document potential anti-patterns found in Elixir software
-and teach developers how to identify and correct them. If an existing piece of code
+and teach developers how to identify them and their pitfalls. If an existing piece
-matches an anti-pattern, it does not mean your code must be rewritten. However, you
+of code matches an anti-pattern, it does not mean your code must be rewritten.
-should take its potential pitfalls and alternatives into consideration.
+Sometimes, even if a snippet matches a potential anti-pattern and its limitations,
+it may be the best approach to the problem at hand. No codebase is free of anti-patterns
+and one should not aim to remove all of them.
 
 The anti-patterns in these guides are broken into 4 main categories:
 
@@ -34,6 +36,8 @@ Each anti-pattern is documented using the following structure:
   * **Refactoring:** Ways to change your code to improve its qualities. Examples of refactored
     code are presented to illustrate these changes.
 
+An additional section with "Additional Remarks" may be provided. Those may include known scenarios where the anti-pattern does not apply.
+
 The initial catalog of anti-patterns was proposed by Lucas Vegi and Marco Tulio Valente, from [ASERG/DCC/UFMG](http://aserg.labsoft.dcc.ufmg.br/). For more info, see [Understanding Code Smells in Elixir Functional Language](https://github.com/lucasvegi/Elixir-Code-Smells/blob/main/etc/2023-emse-code-smells-elixir.pdf) and [the associated code repository](https://github.com/lucasvegi/Elixir-Code-Smells).
 
-Additionally, the Security Working Group of the [Erlang Ecosystem Foundation](https://erlef.github.io/security-wg/) publishes [documents with security resources and best-practices of both Erland and Elixir, including detailed guides for web applications](https://erlef.github.io/security-wg/).
+Additionally, the Security Working Group of the [Erlang Ecosystem Foundation](https://erlef.github.io/security-wg/) publishes [documents with security resources and best-practices of both Erlang and Elixir, including detailed guides for web applications](https://erlef.github.io/security-wg/).

lib/elixir/pages/cheatsheets/enum-cheat.cheatmd

@@ -707,11 +707,11 @@ fruits = ["apple", "banana", "grape", "orange", "pear"]
 iex> Enum.slide(fruits, 2, 0)
 ["grape", "apple", "banana", "orange", "pear"]
 iex> Enum.slide(fruits, 2, 4)
-["apple", "banana", "orange", "pear", "grape", ]
+["apple", "banana", "orange", "pear", "grape"]
 iex> Enum.slide(fruits, 1..3, 0)
 ["banana", "grape", "orange", "apple", "pear"]
 iex> Enum.slide(fruits, 1..3, 4)
-["banana", "pear", "grape", "orange", "apple"]
+["apple", "pear", "banana", "grape", "orange"]
 ```
 
 ## Reversing
@@ -820,7 +820,10 @@ Negative indexes count from the back:
 
 ```elixir
 iex> Enum.drop(cart, -1)
-[%{fruit: "orange", count: 6}]
+[
+  %{fruit: "apple", count: 3},
+  %{fruit: "banana", count: 1}
+]
 ```
 
 ### [`drop_every(enum, nth)`](`Enum.drop_every/2`)
@@ -851,10 +854,7 @@ Negative indexes count from the back:
 
 ```elixir
 iex> Enum.take(cart, -1)
-[
+[%{fruit: "orange", count: 6}]
-  %{fruit: "apple", count: 3},
-  %{fruit: "banana", count: 1}
-]
 ```
 
 ### [`take_every(enum, nth)`](`Enum.take_every/2`)

lib/elixir/pages/getting-started/alias-require-and-import.md

@@ -112,7 +112,7 @@ end
 
 In the example above, the imported `List.duplicate/2` is only visible within that specific function. `duplicate/2` won't be available in any other function in that module (or any other module for that matter).
 
-Note that `import`s are generally discouraged in the language. When working on your own code, prefer `alias` to `import`.
+While `import`s can be a useful for frameworks and libraries to build abstractions, developers should generally prefer `alias` to `import` on their own codebases, as aliases make the origin of the function being invoked clearer.
 
 ## use
 

lib/elixir/pages/getting-started/anonymous-functions.md

@@ -17,7 +17,7 @@ true
 
 In the example above, we defined an anonymous function that receives two arguments, `a` and `b`, and returns the result of `a + b`. The arguments are always on the left-hand side of `->` and the code to be executed on the right-hand side. The anonymous function is stored in the variable `add`.
 
-We can invoke anonymous functions by passing arguments to it. Note that a dot (`.`) between the variable and parentheses is required to invoke an anonymous function. The dot ensures there is no ambiguity between calling the anonymous function matched to a variable `add` and a named function `add/2`. We will write our own named functions when dealing with [Modules and Functions](modules-and-functions.md). For now, just remember that Elixir makes a clear distinction between anonymous functions and named functions.
+We can invoke anonymous functions by passing arguments to it. Note that a dot (`.`) between the variable and parentheses is required to invoke an anonymous function. The dot makes it clear when you are calling an anonymous function, stored in the variable `add`, opposed to a function named `add/2`. For example, if you have an anonymous function stored in the variable `is_atom`, there is no ambiguity between `is_atom.(:foo)` and `is_atom(:foo)`. If both used the same `is_atom(:foo)` syntax, the only way to know the actual behaviour of `is_atom(:foo)` would be by scanning all code thus far for a possible definition of the `is_atom` variable. This scanning hurts maintainability as it requires developers to track additional context in their head when reading and writing code.
 
 Anonymous functions in Elixir are also identified by the number of arguments they receive. We can check if a function is of any given arity by using `is_function/2`:
 

lib/elixir/pages/getting-started/basic-types.md

@@ -83,7 +83,7 @@ You can also use `is_float` or `is_number` to check, respectively, if an argumen
 
 ## Identifying functions and documentation
 
-Before we move on to the next data type, let's talk about how Elixir identity functions.
+Before we move on to the next data type, let's talk about how Elixir identifies functions.
 
 Functions in Elixir are identified by both their name and their arity. The arity of a function describes the number of arguments that the function takes. From this point on we will use both the function name and its arity to describe functions throughout the documentation. `trunc/1` identifies the function which is named `trunc` and takes `1` argument, whereas `trunc/2` identifies a different (nonexistent) function with the same name but with an arity of `2`.
 
@@ -227,7 +227,7 @@ Elixir also supports string interpolation:
 ```elixir
 iex> string = "world"
 iex> "hello #{string}!"
-"hello world"
+"hello world!"
 ```
 
 String concatenation requires both sides to be strings but interpolation supports any data type that may be converted to a string:
@@ -325,6 +325,6 @@ iex> 1 === 1.0
 false
 ```
 
-The comparison operators in Elixir can compare across any data type. We say these operators perform _structural comparison_. For more information, you can read our documentation on [Structural vs Semantic comparisons](Kernel.html#module-structural-comparison).
+The comparison operators in Elixir can compare across any data type. We say these operators perform _structural comparison_. For more information, you can read our documentation on [Structural vs Semantic comparisons](`Kernel#module-structural-comparison`).
 
 Elixir also provides data-types for expressing collections, such as lists and tuples, which we learn next. When we talk about concurrency and fault-tolerance via processes, we will also discuss ports, pids, and references, but that will come on later chapters. Let's move forward.

lib/elixir/pages/getting-started/binaries-strings-and-charlists.md

@@ -257,7 +257,7 @@ iex> heartbeats_per_minute = [99, 97, 116]
 ~c"cat"
 ```
 
-You can always for charlists to be printed in their list representation by calling the `inspect/2` function:
+You can always force charlists to be printed in their list representation by calling the `inspect/2` function:
 
 ```elixir
 iex> inspect(heartbeats_per_minute, charlists: :as_list)

lib/elixir/pages/getting-started/case-cond-and-if.md

@@ -65,7 +65,7 @@ iex> case :ok do
 ** (CaseClauseError) no case clause matching: :ok
 ```
 
-The documentation for the `Kernel` module lists all available guards in its sidebar. You can also consult the complete [Patterns and Guards](../references/patterns-and-guards.html#guards) reference for in-depth documentation.
+The documentation for the `Kernel` module lists all available guards in its sidebar. You can also consult the complete [Patterns and Guards](../references/patterns-and-guards.md#guards) reference for in-depth documentation.
 
 ## cond
 

lib/elixir/pages/getting-started/enumerable-and-streams.md

@@ -13,7 +13,7 @@ iex> Enum.map(%{1 => 2, 3 => 4}, fn {k, v} -> k * v end)
 [2, 12]
 ```
 
-The `Enum` module provides a huge range of functions to transform, sort, group, filter and retrieve items from enumerables. It is one of the modules developers use frequently in their Elixir code. For a general overview of all functions in the `Enum` module, see [the `Enum` cheatsheet](enum-cheat.html).
+The `Enum` module provides a huge range of functions to transform, sort, group, filter and retrieve items from enumerables. It is one of the modules developers use frequently in their Elixir code. For a general overview of all functions in the `Enum` module, see [the `Enum` cheatsheet](enum-cheat.cheatmd).
 
 Elixir also provides ranges (see `Range`), which are also enumerable:
 
@@ -119,6 +119,6 @@ iex> Enum.take(stream, 10)
 
 The example above will fetch the first 10 lines of the file you have selected. This means streams can be very useful for handling large files or even slow resources like network resources.
 
-The `Enum` and `Stream` modules provide a wide-range functions but know all of them at heart. Familiarize yourself with `Enum.map/2`, `Enum.reduce/3` and other functions with either `map` or `reduce` in their names, and you will naturally build an intuition around the most important use cases. You may also focus on the `Enum` module first and only move to `Stream` for the particular scenarios where laziness is required, to either deal with slow resources or large, possibly infinite, collections.
+The `Enum` and `Stream` modules provide a wide range of functions, but you don't have to know all of them by heart. Familiarize yourself with `Enum.map/2`, `Enum.reduce/3` and other functions with either `map` or `reduce` in their names, and you will naturally build an intuition around the most important use cases. You may also focus on the `Enum` module first and only move to `Stream` for the particular scenarios where laziness is required, to either deal with slow resources or large, possibly infinite, collections.
 
 Next, we'll look at a feature central to Elixir, Processes, which allows us to write concurrent, parallel and distributed programs in an easy and understandable way.

lib/elixir/pages/getting-started/io-and-the-file-system.md

@@ -129,7 +129,7 @@ iex> IO.write(pid, "hello")
 
 After `IO.write/2`, we can see the request sent by the `IO` module printed out (a four-elements tuple). Soon after that, we see that it fails since the `IO` module expected some kind of result, which we did not supply.
 
-By modeling IO devices with processes, the Erlang VM allows is to even read and write to files across nodes. Neat!
+By modeling IO devices with processes, the Erlang VM allows us to even read and write to files across nodes. Neat!
 
 ## `iodata` and `chardata`
 

lib/elixir/pages/getting-started/keywords-and-maps.md

@@ -127,7 +127,7 @@ iex> if true, do: "This will be seen", else: "This won't"
 
 Pay close attention to both syntaxes. In the keyword list format, we separate each key-value pair with commas, and each key is followed by `:`. In the `do`-blocks, we get rid of the colons, the commas, and separate each keyword by a newline. They are useful exactly because they remove the verbosity when writing blocks of code. Most of the time, you will use the block syntax, but it is good to know they are equivalent.
 
-This plays an important role in the language as it allows Elixir syntax to stay small but still expressive. We only need few data structures to represent the language, a topic we will come back to when talking about [optional syntax](optional-syntax.md) and go in-depth when discussing [meta-programming](../quote-and-unquote.md).
+This plays an important role in the language as it allows Elixir syntax to stay small but still expressive. We only need few data structures to represent the language, a topic we will come back to when talking about [optional syntax](optional-syntax.md) and go in-depth when discussing [meta-programming](../meta-programming/quote-and-unquote.md).
 
 With this out of the way, let's talk about maps.
 

lib/elixir/pages/getting-started/sigils.md

@@ -2,7 +2,7 @@
 
 Elixir provides double-quoted strings as well as a concept called charlists, which are defined using the `~c"hello world"` sigil syntax. In this chapter, we will learn more about sigils and how to define our own.
 
-One of Elixir's goals is extensibility: developers should be able to extend the language to fit any particular domain. Sigils provide the foundation for extending the language with custom textual representations. Sigils start with the tilde (`~`) character which is followed by a one lower-case letter or several upper-case ones and then a delimiter. Optionally, modifiers can be added after the final delimiter.
+One of Elixir's goals is extensibility: developers should be able to extend the language to fit any particular domain. Sigils provide the foundation for extending the language with custom textual representations. Sigils start with the tilde (`~`) character which is followed by either a single lower-case letter or one or more upper-case letters, and then a delimiter. Optional modifiers are added after the final delimiter.
 
 ## Regular expressions
 

lib/elixir/pages/meta-programming/macros.md

@@ -26,7 +26,7 @@ defmodule Unless do
 end
 ```
 
-The function receives the arguments and passes them to `if/2`. However, as we learned in the [previous guide](quote-and-unquote.html), the macro will receive quoted expressions, inject them into the quote, and finally return another quoted expression.
+The function receives the arguments and passes them to `if/2`. However, as we learned in the [previous guide](quote-and-unquote.md), the macro will receive quoted expressions, inject them into the quote, and finally return another quoted expression.
 
 Let's start `iex` with the module above:
 
@@ -223,7 +223,7 @@ It is important that a macro is defined before its usage. Failing to define a ma
 
 ```elixir
 iex> defmodule Sample do
-...>  def four, do: two + two
+...>  def four, do: two() + two()
 ...>  defmacrop two, do: 2
 ...> end
 ** (CompileError) iex:2: function two/0 undefined

lib/elixir/pages/mix-and-otp/agents.md

@@ -124,7 +124,7 @@ The first step in our implementation is to call `use Agent`. Most of the functio
 
 Then we define a `start_link/1` function, which will effectively start the agent. It is a convention to define a `start_link/1` function that always accepts a list of options. We don't plan on using any options right now, but we might later on. We then proceed to call `Agent.start_link/1`, which receives an anonymous function that returns the Agent's initial state.
 
-We are keeping a map inside the agent to store our keys and values. Getting and putting values on the map is done with the Agent API and the capture operator `&`, introduced in [the Getting Started guide](../getting-started/modules-and-functions.md#function-capturing). The agent passes its state to the anonymous function via the `&1` argument when `Agent.get/2` and `Agent.update/2` are called.
+We are keeping a map inside the agent to store our keys and values. Getting and putting values on the map is done with the Agent API and the capture operator `&`, introduced in [the Getting Started guide](../getting-started/anonymous-functions.md#the-capture-operator). The agent passes its state to the anonymous function via the `&1` argument when `Agent.get/2` and `Agent.update/2` are called.
 
 Now that the `KV.Bucket` module has been defined, our test should pass! You can try it yourself by running: `mix test`.
 

lib/elixir/pages/mix-and-otp/distributed-tasks.md

@@ -90,7 +90,7 @@ There are three better alternatives to `Node.spawn_link/2` that we could use in
 
 2. We could have a server running on the other node and send requests to that node via the `GenServer` API. For example, you can call a server on a remote node by using `GenServer.call({name, node}, arg)` or passing the remote process PID as the first argument
 
-3. We could use [tasks](`Task`), which we have learned about in [a previous chapter](../getting-started/mix-otp/task-and-gen-tcp.md), as they can be spawned on both local and remote nodes
+3. We could use [tasks](`Task`), which we have learned about in [a previous chapter](task-and-gen-tcp.md), as they can be spawned on both local and remote nodes
 
 The options above have different properties. The GenServer would serialize your requests on a single server, while tasks are effectively running asynchronously on the remote node, with the only serialization point being the spawning done by the supervisor.
 

lib/elixir/pages/mix-and-otp/dynamic-supervisor.md

@@ -183,7 +183,7 @@ A GUI should pop up containing all sorts of information about our system, from g
 
 In the Applications tab, you will see all applications currently running in your system alongside their supervision tree. You can select the `kv` application to explore it further:
 
-<img src="images/kv-observer.png" alt="Observer GUI screenshot" />
+<img src="assets/kv-observer.png" alt="Observer GUI screenshot" />
 
 Not only that, as you create new buckets on the terminal, you should see new processes spawned in the supervision tree shown in Observer:
 

lib/elixir/pages/mix-and-otp/genservers.md

@@ -1,6 +1,6 @@
 # Client-server communication with GenServer
 
-In the [previous chapter](../agents.md), we used agents to represent our buckets. In the [introduction to mix](../introduction-to-mix.md), we specified we would like to name each bucket so we can do the following:
+In the [previous chapter](agents.md), we used agents to represent our buckets. In the [introduction to mix](introduction-to-mix.md), we specified we would like to name each bucket so we can do the following:
 
 ```elixir
 CREATE shopping
@@ -320,7 +320,7 @@ So far we have used three callbacks: `handle_call/3`, `handle_cast/2` and `handl
 
 Since any message, including the ones sent via `send/2`, go to `handle_info/2`, there is a chance that unexpected messages will arrive to the server. Therefore, if we don't define the catch-all clause, those messages could cause our registry to crash, because no clause would match. We don't need to worry about such cases for `handle_call/3` and `handle_cast/2` though. Calls and casts are only done via the `GenServer` API, so an unknown message is quite likely a developer mistake.
 
-To help developers remember the differences between call, cast and info, the supported return values and more, we have a tiny [GenServer cheat sheet](/downloads/cheatsheets/gen-server.pdf).
+To help developers remember the differences between call, cast and info, the supported return values and more, we have a tiny [GenServer cheat sheet](https://elixir-lang.org/downloads/cheatsheets/gen-server.pdf).
 
 ## Monitors or links?
 

lib/elixir/pages/references/compatibility-and-deprecations.md

@@ -8,12 +8,11 @@ Elixir applies bug fixes only to the latest minor branch. Security patches are a
 
 Elixir version | Support
 :------------- | :-----------------------------
-1.16           | Development
+1.16           | Bug fixes and security patches
-1.15           | Bug fixes and security patches
+1.15           | Security patches only
 1.14           | Security patches only
 1.13           | Security patches only
 1.12           | Security patches only
-1.11           | Security patches only
 
 New releases are announced in the read-only [announcements mailing list](https://groups.google.com/group/elixir-lang-ann). All security releases [will be tagged with `[security]`](https://groups.google.com/forum/#!searchin/elixir-lang-ann/%5Bsecurity%5D%7Csort:date).
 
@@ -43,8 +42,9 @@ Erlang/OTP versioning is independent from the versioning of Elixir. Erlang relea
 
 Elixir version | Supported Erlang/OTP versions
 :------------- | :-------------------------------
+1.16           | 24 - 26
 1.15           | 24 - 26
-1.14           | 23 - 25
+1.14           | 23 - 25 (and Erlang/OTP 26 from v1.14.5)
 1.13           | 22 - 24 (and Erlang/OTP 25 from v1.13.4)
 1.12           | 22 - 24
 1.11           | 21 - 23 (and Erlang/OTP 24 from v1.11.4)

lib/elixir/pages/references/operators.md

@@ -64,7 +64,7 @@ Finally, these operators appear in the precedence table above but are only meani
   * `=>` - see [`%{}`](`%{}/1`)
   * `when` - see [Guards](patterns-and-guards.md#guards)
   * `<-` - see [`for`](`for/1`) and [`with`](`with/1`)
-  * `\\` - see [Default arguments](Kernel.html#def/2-default-arguments)
+  * `\\` - see [Default arguments](`Kernel#def/2-default-arguments`)
 
 ## Comparison operators
 

lib/elixir/pages/references/patterns-and-guards.md

@@ -159,13 +159,13 @@ iex> [head | tail] = []
 Given charlists are represented as a list of integers, one can also perform prefix matches on charlists using the list concatenation operator ([`++`](`++/2`)):
 
 ```elixir
-iex> 'hello ' ++ world = 'hello world'
+iex> ~c"hello " ++ world = ~c"hello world"
-'hello world'
+~c"hello world"
 iex> world
-'world'
+~c"world"
 ```
 
-Which is equivalent to matching on `[?h, ?e, ?l, ?l, ?o, ?\s | world]`. Suffix matches (`hello ++ ' world'`) are not valid patterns.
+Which is equivalent to matching on `[?h, ?e, ?l, ?l, ?o, ?\s | world]`. Suffix matches (`hello ++ ~c" world"`) are not valid patterns.
 
 ### Maps
 
@@ -274,7 +274,7 @@ Not all expressions are allowed in guard clauses, but only a handful of them. Th
 
 ### List of allowed functions and operators
 
-You can find the built-in list of guards [in the `Kernel` module](Kernel.html#guards). Here is an overview:
+You can find the built-in list of guards [in the `Kernel` module](`Kernel#guards`). Here is an overview:
 
   * comparison operators ([`==`](`==/2`), [`!=`](`!=/2`), [`===`](`===/2`), [`!==`](`!==/2`),
     [`<`](`</2`), [`<=`](`<=/2`), [`>`](`>/2`), [`>=`](`>=/2`))

lib/elixir/pages/references/syntax-reference.md

@@ -92,7 +92,7 @@ Variables in Elixir must start with an underscore or a Unicode letter that is no
 
 ### Non-qualified calls (local calls)
 
-Non-qualified calls, such as `add(1, 2)`, must start with characters and then follow the same rules as as variables, which are optionally followed by parentheses, and then arguments.
+Non-qualified calls, such as `add(1, 2)`, must start with characters and then follow the same rules as variables, which are optionally followed by parentheses, and then arguments.
 
 Parentheses are required for zero-arity calls (i.e. calls without arguments), to avoid ambiguity with variables. If parentheses are used, they must immediately follow the function name *without spaces*. For example, `add (1, 2)` is a syntax error, since `(1, 2)` is treated as an invalid block which is attempted to be given as a single argument to `add`.
 
@@ -104,7 +104,7 @@ As many programming languages, Elixir also support operators as non-qualified ca
 
 ### Qualified calls (remote calls)
 
-Qualified calls, such as `Math.add(1, 2)`, must start with characters and then follow the same rules as as variables, which are optionally followed by parentheses, and then arguments. Qualified calls also support operators, such as `Kernel.+(1, 2)`. Elixir also allows the function name to be written between double- or single-quotes, allowing any character in between the quotes, such as `Math."++add++"(1, 2)`.
+Qualified calls, such as `Math.add(1, 2)`, must start with characters and then follow the same rules as variables, which are optionally followed by parentheses, and then arguments. Qualified calls also support operators, such as `Kernel.+(1, 2)`. Elixir also allows the function name to be written between double- or single-quotes, allowing any character in between the quotes, such as `Math."++add++"(1, 2)`.
 
 Similar to non-qualified calls, parentheses have different meaning for zero-arity calls (i.e. calls without arguments). If parentheses are used, such as `mod.fun()`, it means a function call. If parenthesis are skipped, such as `map.field`, it means accessing a field of a map.
 
@@ -395,7 +395,7 @@ end
 
 All of the constructs above are part of Elixir's syntax and have their own representation as part of the Elixir AST. This section will discuss the remaining constructs that are alternative representations of the constructs above. In other words, the constructs below can be represented in more than one way in your Elixir code and retain AST equivalence. We call this "Optional Syntax".
 
-For a lightweight introduction to Elixir's Optional Syntax, [see this document](https://elixir-lang.org/getting-started/optional-syntax.html). Below we continue with a more complete reference.
+For a lightweight introduction to Elixir's Optional Syntax, [see this document](../getting-started/optional-syntax.md). Below we continue with a more complete reference.
 
 ### Integers in other bases and Unicode code points
 

lib/elixir/pages/references/typespecs.md

@@ -17,7 +17,7 @@ Type specifications (most often referred to as *typespecs*) are defined in diffe
   * `@callback`
   * `@macrocallback`
 
-In addition, you can use `@typedoc` to describe a custom `@type` definition.
+In addition, you can use `@typedoc` to document a custom `@type` definition.
 
 See the "User-defined types" and "Defining a specification" sub-sections below for more information on defining types and typespecs.
 
@@ -250,19 +250,114 @@ Behaviours in Elixir (and Erlang) are a way to separate and abstract the generic
 
 A behaviour module defines a set of functions and macros (referred to as *callbacks*) that callback modules implementing that behaviour must export. This "interface" identifies the specific part of the component. For example, the `GenServer` behaviour and functions abstract away all the message-passing (sending and receiving) and error reporting that a "server" process will likely want to implement from the specific parts such as the actions that this server process has to perform.
 
-To define a behaviour module, it's enough to define one or more callbacks in that module. To define callbacks, the `@callback` and `@macrocallback` module attributes can be used (for function callbacks and macro callbacks respectively).
+Say we want to implement a bunch of parsers, each parsing structured data: for example, a JSON parser and a MessagePack parser. Each of these two parsers will *behave* the same way: both will provide a `parse/1` function and an `extensions/0` function. The `parse/1` function will return an Elixir representation of the structured data, while the `extensions/0` function will return a list of file extensions that can be used for each type of data (e.g., `.json` for JSON files).
 
-    defmodule MyBehaviour do
+We can create a `Parser` behaviour:
-      @callback my_fun(arg :: any) :: any
+
-      @macrocallback my_macro(arg :: any) :: Macro.t
+```elixir
-    end
+defmodule Parser do
+  @doc """
+  Parses a string.
+  """
+  @callback parse(String.t) :: {:ok, term} | {:error, atom}
+
+  @doc """
+  Lists all supported file extensions.
+  """
+  @callback extensions() :: [String.t]
+end
+```
 
 As seen in the example above, defining a callback is a matter of defining a specification for that callback, made of:
 
-  * the callback name (`my_fun` or `my_macro` in the example)
+  * the callback name (`parse` or `extensions` in the example)
-  * the arguments that the callback must accept (`arg :: any` in the example)
+  * the arguments that the callback must accept (`String.t`)
   * the *expected* type of the callback return value
 
+Modules adopting the `Parser` behaviour will have to implement all the functions defined with the `@callback` attribute. As you can see, `@callback` expects a function name but also a function specification like the ones used with the `@spec` attribute we saw above.
+
+### Implementing behaviours
+
+Implementing a behaviour is straightforward:
+
+```elixir
+defmodule JSONParser do
+  @behaviour Parser
+
+  @impl Parser
+  def parse(str), do: {:ok, "some json " <> str} # ... parse JSON
+
+  @impl Parser
+  def extensions, do: [".json"]
+end
+```
+
+```elixir
+defmodule CSVParser do
+  @behaviour Parser
+
+  @impl Parser
+  def parse(str), do: {:ok, "some csv " <> str} # ... parse CSV
+
+  @impl Parser
+  def extensions, do: [".csv"]
+end
+```
+
+If a module adopting a given behaviour doesn't implement one of the callbacks required by that behaviour, a compile-time warning will be generated.
+
+Furthermore, with `@impl` you can also make sure that you are implementing the **correct** callbacks from the given behaviour in an explicit manner. For example, the following parser implements both `parse` and `extensions`. However, thanks to a typo, `BADParser` is implementing `parse/0` instead of `parse/1`.
+
+```elixir
+defmodule BADParser do
+  @behaviour Parser
+
+  @impl Parser
+  def parse, do: {:ok, "something bad"}
+
+  @impl Parser
+  def extensions, do: ["bad"]
+end
+```
+
+This code generates a warning letting you know that you are mistakenly implementing `parse/0` instead of `parse/1`.
+You can read more about `@impl` in the [module documentation](`Module#module-impl`).
+
+### Using behaviours
+
+Behaviours are useful because you can pass modules around as arguments and you can then *call back* to any of the functions specified in the behaviour. For example, we can have a function that receives a filename, several parsers, and parses the file based on its extension:
+
+```elixir
+@spec parse_path(Path.t(), [module()]) :: {:ok, term} | {:error, atom}
+def parse_path(filename, parsers) do
+  with {:ok, ext} <- parse_extension(filename),
+       {:ok, parser} <- find_parser(ext, parsers),
+       {:ok, contents} <- File.read(filename) do
+    parser.parse(contents)
+  end
+end
+
+defp parse_extension(filename) do
+  if ext = Path.extname(filename) do
+    {:ok, ext}
+  else
+    {:error, :no_extension}
+  end
+end
+
+defp find_parser(ext, parsers) do
+  if parser = Enum.find(parsers, fn parser -> ext in parser.extensions() end) do
+    {:ok, parser}
+  else
+    {:error, :no_matching_parser}
+  end
+end
+```
+
+You could also invoke any parser directly: `CSVParser.parse(...)`.
+
+Note you don't need to define a behaviour in order to dynamically dispatch on a module, but those features often go hand in hand.
+
 ### Optional callbacks
 
 Optional callbacks are callbacks that callback modules may implement if they want to, but are not required to. Usually, behaviour modules know if they should call those callbacks based on configuration, or they check if the callbacks are defined with `function_exported?/3` or `macro_exported?/3`.
@@ -278,34 +373,6 @@ Optional callbacks can be defined through the `@optional_callbacks` module attri
 
 One example of optional callback in Elixir's standard library is `c:GenServer.format_status/2`.
 
-### Implementing behaviours
-
-To specify that a module implements a given behaviour, the `@behaviour` attribute must be used:
-
-    defmodule MyBehaviour do
-      @callback my_fun(arg :: any) :: any
-    end
-
-    defmodule MyCallbackModule do
-      @behaviour MyBehaviour
-      def my_fun(arg), do: arg
-    end
-
-If a callback module that implements a given behaviour doesn't export all the functions and macros defined by that behaviour, the user will be notified through warnings during the compilation process (no errors will happen).
-
-You can also use the `@impl` attribute before a function to denote that particular function is implementation a behaviour:
-
-    defmodule MyCallbackModule do
-      @behaviour MyBehaviour
-
-      @impl true
-      def my_fun(arg), do: arg
-    end
-
-You can also use `@impl MyBehaviour` to make clearer from which behaviour the callbacks comes from, providing even more context for future readers of your code.
-
-Elixir's standard library contains a few frequently used behaviours such as `GenServer`, `Supervisor`, and `Application`.
-
 ### Inspecting behaviours
 
 The `@callback` and `@optional_callbacks` attributes are used to create a `behaviour_info/1` function available on the defining module. This function can be used to retrieve the callbacks and optional callbacks defined by that module.
@@ -319,6 +386,10 @@ For example, for the `MyBehaviour` module defined in "Optional callbacks" above:
 
 When using `iex`, the `IEx.Helpers.b/1` helper is also available.
 
+## Pitfalls
+
+There are some known pitfalls when using typespecs, they are documented next.
+
 ## The `string()` type
 
 Elixir discourages the use of the `string()` type. The `string()` type refers to Erlang strings, which are known as "charlists" in Elixir. They do not refer to Elixir strings, which are UTF-8 encoded binaries. To avoid confusion, if you attempt to use the type `string()`, Elixir will emit a warning. You should use `charlist()`, `nonempty_charlist()`, `binary()` or `String.t()` accordingly, or any of the several literal representations for these types.

lib/elixir/pages/references/unicode-syntax.md

@@ -81,7 +81,7 @@ Unicode atoms in Elixir follow the identifier rule above with the following modi
   * `<Start>` additionally includes the code point `_` (005F)
   * `<Continue>` additionally includes the code point `@` (0040)
 
-Note atoms can also be quoted, which allows any characters, such as `:"hello elixir"`. All Elixir operators are also valid atoms, such as `:+`, `:@`, `:|>`, and others. The full description of valid atoms is available in the ["Atoms" section in the syntax reference](syntax-reference.html#atoms).
+Note atoms can also be quoted, which allows any characters, such as `:"hello elixir"`. All Elixir operators are also valid atoms, such as `:+`, `:@`, `:|>`, and others. The full description of valid atoms is available in the ["Atoms" section in the syntax reference](syntax-reference.md#atoms).
 
 #### Variables, local calls, and remote calls
 

lib/elixir/scripts/elixir_docs.exs

@@ -2,6 +2,7 @@
 canonical = System.fetch_env!("CANONICAL")
 
 [
+  assets: "lib/elixir/pages/images",
   extras: [
     "lib/elixir/pages/getting-started/introduction.md",
     "lib/elixir/pages/getting-started/basic-types.md",

lib/elixir/scripts/windows_installer/build.sh

@@ -4,43 +4,27 @@
 # With Elixir archive:
 #
 #     ELIXIR_ZIP=Precompiled.zip OTP_VERSION=25.3.2.2 ./build.sh
-#
-# With Elixir version:
-#
-#     ELIXIR_VERSION=1.14.5 OTP_VERSION=25.3.2.2 ./build.sh
 set -euo pipefail
 
-OTP_VERSION="${OTP_VERSION:-26.0}"
-otp_release=`echo "${OTP_VERSION}" | cut -d. -f1`
-
 mkdir -p tmp
+rm -rf tmp/elixir
+unzip -d "tmp/elixir" "${ELIXIR_ZIP}"
 
-ELIXIR_VERSION="${ELIXIR_VERSION:-}"
+elixir_version=`cat tmp/elixir/VERSION`
-if [ -n "${ELIXIR_VERSION}" ]; then
+otp_release=`erl -noshell -eval 'io:put_chars(erlang:system_info(otp_release)), halt().'`
-  ELIXIR_ZIP="tmp/elixir-${ELIXIR_VERSION}-otp-${otp_release}.zip"
+otp_version=`erl -noshell -eval '{ok, Vsn} = file:read_file(code:root_dir() ++ "/releases/" ++ erlang:system_info(otp_release) ++ "/OTP_VERSION"), io:put_chars(Vsn), halt().'`
-  if [ ! -f "${ELIXIR_ZIP}" ]; then
+elixir_exe=elixir-otp-${otp_release}.exe
-    url="https://github.com/elixir-lang/elixir/releases/download/v${ELIXIR_VERSION}/elixir-otp-${otp_release}.zip"
-    echo "downloading ${url}"
-    curl --fail -L -o "${ELIXIR_ZIP}" "${url}"
-  fi
-  basename=elixir-${ELIXIR_VERSION}-otp-${otp_release}
-else
-  basename=elixir-otp-${otp_release}
-fi
-
-if [ ! -d "tmp/${basename}" ]; then
-  unzip -d "tmp/${basename}" "${ELIXIR_ZIP}"
-fi
 
 # brew install makensis
 # apt install -y nsis
 # choco install -y nsis
 export PATH="/c/Program Files (x86)/NSIS:${PATH}"
 makensis \
-  -X"OutFile tmp\\${basename}.exe" \
+  -X"OutFile tmp\\${elixir_exe}" \
-  -DOTP_VERSION=${OTP_VERSION} \
   -DOTP_RELEASE="${otp_release}" \
-  -DELIXIR_DIR=tmp\\${basename} \
+  -DOTP_VERSION=${otp_version} \
+  -DELIXIR_DIR=tmp\\elixir \
+  -DELIXIR_VERSION=${elixir_version} \
   installer.nsi
 
-echo "Installer path: tmp/${basename}.exe"
+echo "Installer path: tmp/${elixir_exe}"

lib/elixir/scripts/windows_installer/installer.nsi

@@ -6,7 +6,8 @@ Name "Elixir"
 ManifestDPIAware true
 Unicode True
 InstallDir "$PROGRAMFILES64\Elixir"
-!define MUI_ICON "assets\drop.ico"
+!define MUI_ICON "assets\Elixir.ico"
+!define MUI_UNICON "assets\Elixir.ico"
 
 ; Install Page: Install Erlang/OTP
 
@@ -191,7 +192,17 @@ FunctionEnd
 Section "Install Elixir" SectionElixir
   SetOutPath "$INSTDIR"
   File /r "${ELIXIR_DIR}\"
+  File "assets\Elixir.ico"
   File "update_system_path.erl"
+  WriteRegStr   HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\Elixir" "DisplayName" "Elixir"
+  WriteRegStr   HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\Elixir" "DisplayVersion" "${ELIXIR_VERSION}"
+  WriteRegStr   HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\Elixir" "DisplayIcon" "$INSTDIR\Elixir.ico"
+  WriteRegStr   HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\Elixir" "Publisher" "The Elixir Team"
+  WriteRegStr   HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\Elixir" "UninstallString" '"$INSTDIR\Uninstall.exe"'
+  WriteRegDWORD HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\Elixir" "NoModify" 1
+  WriteRegDWORD HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\Elixir" "NoRepair" 1
+
+  WriteRegStr   HKLM "Software\Elixir\Elixir" "InstallRoot" "$INSTDIR"
 
   WriteUninstaller "Uninstall.exe"
 SectionEnd
@@ -270,6 +281,9 @@ UninstPage custom un.FinishPageShow un.FinishPageLeave
 
 Section "Uninstall"
   RMDir /r "$INSTDIR"
+  DeleteRegKey HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\Elixir"
+  DeleteRegKey HKLM "Software\Elixir\Elixir"
+  DeleteRegKey /ifempty HKLM "Software\Elixir"
 SectionEnd
 
 !insertmacro MUI_LANGUAGE "English"

lib/elixir/src/elixir.erl

@@ -486,7 +486,7 @@ string_to_tokens(String, StartLine, StartColumn, File, Opts) when is_integer(Sta
     {ok, _Line, _Column, [], Tokens} ->
       {ok, Tokens};
     {ok, _Line, _Column, Warnings, Tokens} ->
-      (lists:keyfind(warnings, 1, Opts) /= {warnings, false}) andalso
+      (lists:keyfind(emit_warnings, 1, Opts) /= {emit_warnings, false}) andalso
         [elixir_errors:erl_warn(L, File, M) || {L, M} <- lists:reverse(Warnings)],
       {ok, Tokens};
     {error, {Location, {ErrorPrefix, ErrorSuffix}, Token}, _Rest, _Warnings, _SoFar} ->
@@ -544,8 +544,8 @@ to_binary(Atom) when is_atom(Atom) -> atom_to_binary(Atom).
 
 handle_parsing_opts(File, Opts) ->
   WarningFile =
-    case lists:keyfind(warnings, 1, Opts) of
+    case lists:keyfind(emit_warnings, 1, Opts) of
-      {warnings, false} -> nil;
+      {emit_warnings, false} -> nil;
       _ -> File
     end,
   LiteralEncoder =

lib/elixir/src/elixir.hrl

@@ -35,7 +35,7 @@
   identifier_tokenizer=elixir_tokenizer,
   ascii_identifiers_only=true,
   indentation=0,
+  column=1,
   mismatch_hints=[],
-  warn_on_unnecessary_quotes=true,
   warnings=[]
 }).

lib/elixir/src/elixir_bitstring.erl

@@ -35,7 +35,12 @@ expand(BitstrMeta, Fun, [{'::', Meta, [Left, Right]} | T], Acc, S, E, Alignment,
 
   MatchOrRequireSize = RequireSize or is_match_size(T, EL),
   EType = expr_type(ELeft),
-  {ERight, EAlignment, SS, ES} = expand_specs(EType, Meta, Right, SL, OriginalS, EL, MatchOrRequireSize),
+  ExpectSize = case ELeft of
+    _ when not MatchOrRequireSize -> optional;
+    {'^', _, [{_, _, _}]} -> {infer, ELeft};
+    _ -> required
+  end,
+  {ERight, EAlignment, SS, ES} = expand_specs(EType, Meta, Right, SL, OriginalS, EL, ExpectSize),
 
   EAcc = concat_or_prepend_bitstring(Meta, ELeft, ERight, Acc, ES, MatchOrRequireSize),
   expand(BitstrMeta, Fun, T, EAcc, {SS, OriginalS}, ES, alignment(Alignment, EAlignment), RequireSize);
@@ -147,7 +152,7 @@ expand_expr(Meta, Component, Fun, S, E) ->
 
 %% Expands and normalizes types of a bitstring.
 
-expand_specs(ExprType, Meta, Info, S, OriginalS, E, RequireSize) ->
+expand_specs(ExprType, Meta, Info, S, OriginalS, E, ExpectSize) ->
   Default =
     #{size => default,
       unit => default,
@@ -158,11 +163,17 @@ expand_specs(ExprType, Meta, Info, S, OriginalS, E, RequireSize) ->
     expand_each_spec(Meta, unpack_specs(Info, []), Default, S, OriginalS, E),
 
   MergedType = type(Meta, ExprType, Type, E),
-  validate_size_required(Meta, RequireSize, ExprType, MergedType, Size, ES),
+  validate_size_required(Meta, ExpectSize, ExprType, MergedType, Size, ES),
   SizeAndUnit = size_and_unit(Meta, ExprType, Size, Unit, ES),
   Alignment = compute_alignment(MergedType, Size, Unit),
 
-  [H | T] = build_spec(Meta, Size, Unit, MergedType, Endianness, Sign, SizeAndUnit, ES),
+  MaybeInferredSize = case {ExpectSize, MergedType, SizeAndUnit} of
+    {{infer, PinnedVar}, binary, []} -> [{size, Meta, [{{'.', Meta, [erlang, byte_size]}, Meta, [PinnedVar]}]}];
+    {{infer, PinnedVar}, bitstring, []} -> [{size, Meta, [{{'.', Meta, [erlang, bit_size]}, Meta, [PinnedVar]}]}];
+    _ -> SizeAndUnit
+  end,
+
+  [H | T] = build_spec(Meta, Size, Unit, MergedType, Endianness, Sign, MaybeInferredSize, ES),
   {lists:foldl(fun(I, Acc) -> {'-', Meta, [Acc, I]} end, H, T), Alignment, SS, ES}.
 
 type(_, default, default, _) ->
@@ -276,7 +287,7 @@ validate_spec_arg(Meta, unit, Value, _S, _OriginalS, E) when not is_integer(Valu
 validate_spec_arg(_Meta, _Key, _Value, _S, _OriginalS, _E) ->
   ok.
 
-validate_size_required(Meta, true, default, Type, default, E) when Type == binary; Type == bitstring ->
+validate_size_required(Meta, required, default, Type, default, E) when Type == binary; Type == bitstring ->
   function_error(Meta, E, ?MODULE, unsized_binary),
   ok;
 validate_size_required(_, _, _, _, _, _) ->

lib/elixir/src/elixir_compiler.erl

@@ -1,6 +1,6 @@
 %% Elixir compiler front-end to the Erlang backend.
 -module(elixir_compiler).
--export([string/3, quoted/3, bootstrap/0, file/2, compile/3]).
+-export([string/3, quoted/3, bootstrap/0, file/2, compile/4]).
 -include("elixir.hrl").
 
 string(Contents, File, Callback) ->
@@ -36,22 +36,22 @@ maybe_fast_compile(Forms, Args, E) ->
   case (?key(E, module) == nil) andalso allows_fast_compilation(Forms) andalso
         (not elixir_config:is_bootstrap()) of
     true  -> fast_compile(Forms, E);
-    false -> compile(Forms, Args, E)
+    false -> compile(Forms, Args, [], E)
   end,
   ok.
 
-compile(Quoted, ArgsList, #{line := Line} = E) ->
+compile(Quoted, ArgsList, CompilerOpts, #{line := Line} = E) ->
   Block = no_tail_optimize([{line, Line}], Quoted),
   {Expanded, SE, EE} = elixir_expand:expand(Block, elixir_env:env_to_ex(E), E),
   elixir_env:check_unused_vars(SE, EE),
 
   {Module, Fun, Purgeable} =
-    elixir_erl_compiler:spawn(fun() -> spawned_compile(Expanded, E) end),
+    elixir_erl_compiler:spawn(fun() -> spawned_compile(Expanded, CompilerOpts, E) end),
 
   Args = list_to_tuple(ArgsList),
   {dispatch(Module, Fun, Args, Purgeable), SE, EE}.
 
-spawned_compile(ExExprs, #{line := Line, file := File} = E) ->
+spawned_compile(ExExprs, CompilerOpts, #{line := Line, file := File} = E) ->
   {Vars, S} = elixir_erl_var:from_env(E),
   {ErlExprs, _} = elixir_erl_pass:translate(ExExprs, erl_anno:new(Line), S),
 
@@ -59,7 +59,7 @@ spawned_compile(ExExprs, #{line := Line, file := File} = E) ->
   Fun = code_fun(?key(E, module)),
   Forms = code_mod(Fun, ErlExprs, Line, File, Module, Vars),
 
-  {Module, Binary} = elixir_erl_compiler:noenv_forms(Forms, File, [nowarn_nomatch, no_bool_opt, no_ssa_opt]),
+  {Module, Binary} = elixir_erl_compiler:noenv_forms(Forms, File, [nowarn_nomatch | CompilerOpts]),
   code:load_binary(Module, "", Binary),
   {Module, Fun, is_purgeable(Module, Binary)}.
 

lib/elixir/src/elixir_errors.erl

@@ -26,7 +26,7 @@ print_warning(Position, File, Message) ->
 %% Called by Module.ParallelChecker.
 print_warning(Message, Diagnostic) ->
   #{file := File, position := Position, stacktrace := S} = Diagnostic,
-  Snippet = get_snippet(File, Position),
+  Snippet = read_snippet(File, Position),
   Span = get_span(Diagnostic),
   Output = format_snippet(Position, File, Message, Snippet, warning, S, Span),
   io:put_chars(standard_error, [Output, $\n, $\n]).
@@ -34,7 +34,7 @@ print_warning(Message, Diagnostic) ->
 %% Called by Module.ParallelChecker.
 print_warning_group(Message, [Diagnostic | Others]) ->
   #{file := File, position := Position, stacktrace := S} = Diagnostic,
-  Snippet = get_snippet(File, Position),
+  Snippet = read_snippet(File, Position),
   Span = get_span(Diagnostic),
   Formatted = format_snippet(Position, File, Message, Snippet, warning, S, Span),
   LineNumber = extract_line(Position),
@@ -49,16 +49,15 @@ print_warning_group(Message, [Diagnostic | Others]) ->
 get_span(#{span := nil}) -> nil;
 get_span(#{span := Span}) -> Span.
 
-get_snippet(nil, _Position) ->
+read_snippet(nil, _Position) ->
   nil;
-get_snippet(<<"nofile">>, _Position) ->
+read_snippet(<<"nofile">>, _Position) ->
   nil;
-get_snippet(File, Position) ->
+read_snippet(File, Position) ->
   LineNumber = extract_line(Position),
   get_file_line(File, LineNumber).
 
-get_file_line(_, 0) -> nil;
+get_file_line(File, LineNumber) when is_integer(LineNumber), LineNumber > 0 ->
-get_file_line(File, LineNumber) ->
   case file:open(File, [read, binary]) of
     {ok, IoDevice} ->
       Line = traverse_file_line(IoDevice, LineNumber),
@@ -66,7 +65,8 @@ get_file_line(File, LineNumber) ->
       Line;
     {error, _} ->
       nil
-  end.
+  end;
+get_file_line(_, _) -> nil.
 
 traverse_file_line(IoDevice, 1) ->
   case file:read_line(IoDevice) of
@@ -80,7 +80,7 @@ traverse_file_line(IoDevice, N) ->
 print_diagnostic(#{severity := Severity, message := M, stacktrace := Stacktrace, position := P, file := F} = Diagnostic, ReadSnippet) ->
   Snippet =
     case ReadSnippet of
-      true -> get_snippet(F, P);
+      true -> read_snippet(F, P);
       false -> nil
     end,
 
@@ -106,6 +106,7 @@ emit_diagnostic(Severity, Position, File, Message, Stacktrace, Options) ->
 
   Diagnostic = #{
     severity => Severity,
+    source => File,
     file => File,
     position => Position,
     message => unicode:characters_to_binary(Message),
@@ -114,14 +115,17 @@ emit_diagnostic(Severity, Position, File, Message, Stacktrace, Options) ->
   },
 
   case get(elixir_code_diagnostics) of
-    undefined -> print_diagnostic(Diagnostic, ReadSnippet);
+    undefined ->
-    {Tail, true} -> put(elixir_code_diagnostics, {[print_diagnostic(Diagnostic, ReadSnippet) | Tail], true});
+      case get(elixir_compiler_info) of
-    {Tail, false} -> put(elixir_code_diagnostics, {[Diagnostic | Tail], false})
+        undefined -> print_diagnostic(Diagnostic, ReadSnippet);
-  end,
+        {CompilerPid, _} -> CompilerPid ! {diagnostic, Diagnostic, ReadSnippet}
+      end;
 
-  case get(elixir_compiler_info) of
+    {Tail, true} ->
-    undefined -> ok;
+      put(elixir_code_diagnostics, {[print_diagnostic(Diagnostic, ReadSnippet) | Tail], true});
-    {CompilerPid, _} -> CompilerPid ! {diagnostic, Diagnostic}
+
+    {Tail, false} ->
+      put(elixir_code_diagnostics, {[Diagnostic | Tail], false})
   end,
 
   ok.
@@ -319,6 +323,7 @@ parse_error(Location, File, Error, <<>>, Input) ->
     <<"syntax error before: ">> -> <<"syntax error: expression is incomplete">>;
     _ -> <<Error/binary>>
   end,
+
   raise_snippet(Location, File, Input, 'Elixir.TokenMissingError', Message);
 
 %% Show a nicer message for end of line
@@ -394,8 +399,10 @@ parse_error(Location, File, <<"syntax error before: ">>, <<$$, Char/binary>>, In
 parse_error(Location, File, Error, Token, Input) when is_binary(Error), is_binary(Token) ->
   Message = <<Error/binary, Token/binary>>,
   case lists:keytake(error_type, 1, Location) of
-    {value, {error_type, mismatched_delimiter}, Loc} -> raise_mismatched_delimiter(Loc, File, Input, Message);
+    {value, {error_type, mismatched_delimiter}, Loc} ->
-    _ -> raise_snippet(Location, File, Input, 'Elixir.SyntaxError', Message)
+      raise_snippet(Loc, File, Input, 'Elixir.MismatchedDelimiterError', Message);
+    _ ->
+      raise_snippet(Location, File, Input, 'Elixir.SyntaxError', Message)
   end.
 
 parse_erl_term(Term) ->
@@ -403,11 +410,6 @@ parse_erl_term(Term) ->
   {ok, Parsed} = erl_parse:parse_term(Tokens ++ [{dot, 1}]),
   Parsed.
 
-raise_mismatched_delimiter(Location, File, Input, Message) ->
-  {InputString, _, _} = Input,
-  InputBinary = elixir_utils:characters_to_binary(InputString),
-  raise('Elixir.MismatchedDelimiterError', Message,  [{file, File}, {snippet, InputBinary} | Location]).
-
 raise_reserved(Location, File, Input, Keyword) ->
   raise_snippet(Location, File, Input, 'Elixir.SyntaxError',
         <<"syntax error before: ", Keyword/binary, ". \"", Keyword/binary, "\" is a "
@@ -415,25 +417,43 @@ raise_reserved(Location, File, Input, Keyword) ->
           "it can't be used as a variable or be defined nor invoked as a regular function">>).
 
 raise_snippet(Location, File, Input, Kind, Message) when is_binary(File) ->
-  {InputString, StartLine, _} = Input,
+  Snippet = cut_snippet(Location, Input),
-  Snippet = snippet_line(InputString, Location, StartLine),
   raise(Kind, Message, [{file, File}, {snippet, Snippet} | Location]).
 
-snippet_line(InputString, Location, StartLine) ->
+cut_snippet(Location, Input) ->
-  {line, Line} = lists:keyfind(line, 1, Location),
   case lists:keyfind(column, 1, Location) of
     {column, _} ->
-      Lines = string:split(InputString, "\n", all),
+      {line, Line} = lists:keyfind(line, 1, Location),
-      Snippet = (lists:nth(Line - StartLine + 1, Lines)),
+
-      case string:trim(Snippet, leading) of
+      case lists:keyfind(end_line, 1, Location) of
-        [] -> nil;
+        {end_line, EndLine} ->
-        _ -> elixir_utils:characters_to_binary(Snippet)
+          cut_snippet(Input, Line, EndLine - Line + 1);
+
+        false ->
+          Snippet = cut_snippet(Input, Line, 1),
+          case string:trim(Snippet, leading) of
+            <<>> -> nil;
+            _ -> Snippet
+          end
       end;
 
     false ->
       nil
   end.
 
+cut_snippet({InputString, StartLine, StartColumn}, Line, Span) ->
+  %% In case the code is indented, we need to add the indentation back
+  %% for the snippets to match the reported columns.
+  Indent = binary:copy(<<" ">>, StartColumn - 1),
+  Lines = string:split(InputString, "\n", all),
+  [Head | Tail] = lists:nthtail(Line - StartLine, Lines),
+  IndentedTail = indent_n(Tail, Span - 1, <<"\n", Indent/binary>>),
+  elixir_utils:characters_to_binary([Indent, Head, IndentedTail]).
+
+indent_n([], _Count, _Indent) -> [];
+indent_n(_Lines, 0, _Indent) -> [];
+indent_n([H | T], Count, Indent) -> [Indent, H | indent_n(T, Count - 1, Indent)].
+
 %% Helpers
 
 prefix(warning) -> highlight(<<"warning:">>, warning);

lib/elixir/src/elixir_expand.erl

@@ -402,16 +402,12 @@ expand({Atom, Meta, Args}, S, E) when is_atom(Atom), is_list(Meta), is_list(Args
 
 %% Remote calls
 
-expand({{'.', DotMeta, [Left, Right]}, Meta, Args} = Expr, S, E)
+expand({{'.', DotMeta, [Left, Right]}, Meta, Args}, S, E)
     when (is_tuple(Left) orelse is_atom(Left)), is_atom(Right), is_list(Meta), is_list(Args) ->
   {ELeft, SL, EL} = expand(Left, elixir_env:prepare_write(S), E),
-  NoParens = lists:keyfind(no_parens, 1, Meta),
-
-  (is_atom(ELeft) and (Args =:= []) and (NoParens =:= {no_parens, true})) andalso
-    elixir_errors:file_warn(Meta, E, ?MODULE, {remote_nullary_no_parens, Expr}),
 
   elixir_dispatch:dispatch_require(Meta, ELeft, Right, Args, S, EL, fun(AR, AF, AA) ->
-    expand_remote(AR, DotMeta, AF, Meta, NoParens, AA, S, SL, EL)
+    expand_remote(AR, DotMeta, AF, Meta, AA, S, SL, EL)
   end);
 
 %% Anonymous calls
@@ -858,12 +854,13 @@ expand_local(Meta, Name, Args, _S, #{function := nil} = E) ->
 
 %% Remote
 
-expand_remote(Receiver, DotMeta, Right, Meta, NoParens, Args, S, SL, #{context := Context} = E) when is_atom(Receiver) or is_tuple(Receiver) ->
+expand_remote(Receiver, DotMeta, Right, Meta, Args, S, SL, #{context := Context} = E)
+    when is_atom(Receiver) or is_tuple(Receiver) ->
   assert_no_clauses(Right, Meta, Args, E),
 
   if
     Context =:= guard, is_tuple(Receiver) ->
-      (NoParens /= {no_parens, true}) andalso
+      (lists:keyfind(no_parens, 1, Meta) /= {no_parens, true}) andalso
         function_error(Meta, E, ?MODULE, {parens_map_lookup, Receiver, Right, guard_context(S)}),
 
       {{{'.', DotMeta, [Receiver, Right]}, Meta, []}, SL, E};
@@ -881,7 +878,7 @@ expand_remote(Receiver, DotMeta, Right, Meta, NoParens, Args, S, SL, #{context :
           file_error(Meta, E, elixir_rewrite, Error)
       end
   end;
-expand_remote(Receiver, DotMeta, Right, Meta, _NoParens, Args, _, _, E) ->
+expand_remote(Receiver, DotMeta, Right, Meta, Args, _, _, E) ->
   Call = {{'.', DotMeta, [Receiver, Right]}, Meta, Args},
   file_error(Meta, E, ?MODULE, {invalid_call, Call}).
 
@@ -1171,9 +1168,6 @@ assert_no_underscore_clause_in_cond(_Other, _E) ->
 guard_context(#elixir_ex{prematch={_, _, {bitsize, _}}}) -> "bitstring size specifier";
 guard_context(_) -> "guards".
 
-format_error({remote_nullary_no_parens, Expr}) ->
-  String = 'Elixir.String':replace_suffix('Elixir.Macro':to_string(Expr), <<"()">>, <<>>),
-  io_lib:format("parentheses are required for function calls with no arguments, got: ~ts", [String]);
 format_error(invalid_match_on_zero_float) ->
   "pattern matching on 0.0 is equivalent to matching only on +0.0 from Erlang/OTP 27+. Instead you must match on +0.0 or -0.0";
 format_error({useless_literal, Term}) ->

lib/elixir/src/elixir_import.erl

@@ -9,24 +9,26 @@ import(Meta, Ref, Opts, E) ->
   {Functions, Macros, Added} =
     case keyfind(only, Opts) of
       {only, functions} ->
-        {Added1, Funs} = import_functions(Meta, Ref, Opts, E),
+        {Added1, _Used1, Funs} = import_functions(Meta, Ref, Opts, E),
         {Funs, keydelete(Ref, ?key(E, macros)), Added1};
       {only, macros} ->
-        {Added2, Macs} = import_macros(true, Meta, Ref, Opts, E),
+        {Added2, _Used2, Macs} = import_macros(true, Meta, Ref, Opts, E),
         {keydelete(Ref, ?key(E, functions)), Macs, Added2};
       {only, sigils} ->
-        {Added1, Funs} = import_sigil_functions(Meta, Ref, Opts, E),
+        {Added1, _Used1, Funs} = import_sigil_functions(Meta, Ref, Opts, E),
-        {Added2, Macs} = import_sigil_macros(Meta, Ref, Opts, E),
+        {Added2, _Used2, Macs} = import_sigil_macros(Meta, Ref, Opts, E),
         {Funs, Macs, Added1 or Added2};
       {only, List} when is_list(List) ->
-        {Added1, Funs} = import_functions(Meta, Ref, Opts, E),
+        {Added1, Used1, Funs} = import_functions(Meta, Ref, Opts, E),
-        {Added2, Macs} = import_macros(false, Meta, Ref, Opts, E),
+        {Added2, Used2, Macs} = import_macros(false, Meta, Ref, Opts, E),
+        [elixir_errors:file_warn(Meta, ?key(E, file), ?MODULE, {invalid_import, {Ref, Name, Arity}}) ||
+         {Name, Arity} <- (List -- Used1) -- Used2],
         {Funs, Macs, Added1 or Added2};
       {only, Other} ->
         elixir_errors:file_error(Meta, E, ?MODULE, {invalid_option, only, Other});
       false ->
-        {Added1, Funs} = import_functions(Meta, Ref, Opts, E),
+        {Added1, _Used1, Funs} = import_functions(Meta, Ref, Opts, E),
-        {Added2, Macs} = import_macros(false, Meta, Ref, Opts, E),
+        {Added2, _Used2, Macs} = import_macros(false, Meta, Ref, Opts, E),
         {Funs, Macs, Added1 or Added2}
     end,
 
@@ -95,9 +97,6 @@ calculate(Meta, Key, Opts, Old, File, Existing) ->
         _ -> elixir_errors:file_error(Meta, File, ?MODULE, only_and_except_given)
       end,
 
-      [elixir_errors:file_warn(Meta, File, ?MODULE, {invalid_import, {Key, Name, Arity}}) ||
-       {Name, Arity} <- Only -- get_exports(Key)],
-
       intersection(Only, Existing());
 
     _ ->
@@ -125,17 +124,14 @@ calculate(Meta, Key, Opts, Old, File, Existing) ->
   %% Normalize the data before storing it
   case ordsets:from_list(New) of
     [] ->
-      {false, keydelete(Key, Old)};
+      {false, [], keydelete(Key, Old)};
     Set  ->
       ensure_no_special_form_conflict(Meta, File, Key, Set),
-      {true, [{Key, Set} | keydelete(Key, Old)]}
+      {true, Set, [{Key, Set} | keydelete(Key, Old)]}
   end.
 
 %% Retrieve functions and macros from modules
 
-get_exports(Module) ->
-  get_functions(Module) ++ get_macros(Module).
-
 get_functions(Module) ->
   try
     Module:'__info__'(functions)
@@ -143,14 +139,6 @@ get_functions(Module) ->
     error:undef -> remove_internals(Module:module_info(exports))
   end.
 
-get_macros(Module) ->
-  case fetch_macros(Module) of
-    {ok, Macros} ->
-      Macros;
-    error ->
-      []
-  end.
-
 fetch_macros(Module) ->
   try
     {ok, Module:'__info__'(macros)}

lib/elixir/src/elixir_interpolation.erl

@@ -97,7 +97,7 @@ extract_nl(Rest, Buffer, Output, Line, Scope, Interpol, [H,H,H] = Last) ->
       extract(NewRest, NewBuffer, Output, Line + 1, Column, Scope, Interpol, Last)
   end;
 extract_nl(Rest, Buffer, Output, Line, Scope, Interpol, Last) ->
-  extract(Rest, Buffer, Output, Line + 1, 1, Scope, Interpol, Last).
+  extract(Rest, Buffer, Output, Line + 1, Scope#elixir_tokenizer.column, Scope, Interpol, Last).
 
 strip_horizontal_space([H | T], Buffer, Counter) when H =:= $\s; H =:= $\t ->
   strip_horizontal_space(T, [H | Buffer], Counter + 1);

lib/elixir/src/elixir_module.erl

@@ -118,6 +118,7 @@ invalid_module_name(Module) ->
 compile(Line, Module, ModuleAsCharlist, Block, Vars, Prune, E) ->
   File = ?key(E, file),
   check_module_availability(Module, Line, E),
+  elixir_env:trace(defmodule, E),
 
   CompilerModules = compiler_modules(),
   {Tables, Ref} = build(Module, Line, File, E),
@@ -421,7 +422,9 @@ build(Module, Line, File, E) ->
 %% Handles module and callback evaluations.
 
 eval_form(Line, Module, DataBag, Block, Vars, Prune, E) ->
-  {Value, ExS, EE} = elixir_compiler:compile(Block, Vars, E),
+  %% Given Elixir modules can get very long to compile due to metaprogramming,
+  %% we disable expansions that take linear time to code size.
+  {Value, ExS, EE} = elixir_compiler:compile(Block, Vars, [no_bool_opt, no_ssa_opt], E),
   elixir_overridable:store_not_overridden(Module),
   EV = (elixir_env:reset_vars(EE))#{line := Line},
   EC = eval_callbacks(Line, DataBag, before_compile, [EV], EV),

lib/elixir/src/elixir_parser.yrl

@@ -292,13 +292,13 @@ bracket_arg -> open_bracket container_expr close_bracket : build_access_arg('$1'
 bracket_arg -> open_bracket container_expr ',' close_bracket : build_access_arg('$1', '$2', '$4').
 bracket_arg -> open_bracket container_expr ',' container_args close_bracket : error_too_many_access_syntax('$3').
 
-bracket_expr -> dot_bracket_identifier bracket_arg : build_access(build_no_parens('$1', nil), '$2').
+bracket_expr -> dot_bracket_identifier bracket_arg : build_access(build_no_parens('$1', nil), meta_with_from_brackets('$2')).
 bracket_expr -> access_expr bracket_arg : build_access('$1', meta_with_from_brackets('$2')).
 
 bracket_at_expr -> at_op_eol dot_bracket_identifier bracket_arg :
-                     build_access(build_unary_op('$1', build_no_parens('$2', nil)), '$3').
+                     build_access(build_unary_op('$1', build_no_parens('$2', nil)), meta_with_from_brackets('$3')).
 bracket_at_expr -> at_op_eol access_expr bracket_arg :
-                     build_access(build_unary_op('$1', '$2'), '$3').
+                     build_access(build_unary_op('$1', '$2'), meta_with_from_brackets('$3')).
 
 %% Blocks
 
@@ -910,11 +910,6 @@ build_identifier({'.', Meta, _} = Dot, Args) ->
 build_identifier({op_identifier, Location, Identifier}, [Arg]) ->
   {Identifier, [{ambiguous_op, nil} | meta_from_location(Location)], [Arg]};
 
-%% TODO: Either remove ... or make it an operator on v2.0
-build_identifier({_, {Line, Column, _} = Location, '...'}, Args) when is_list(Args) ->
-  warn({Line, Column}, "... is no longer supported as a function call and it must receive no arguments"),
-  {'...', meta_from_location(Location), Args};
-
 build_identifier({_, Location, Identifier}, Args) ->
   {Identifier, meta_from_location(Location), Args}.
 

lib/elixir/src/elixir_tokenizer.erl

@@ -123,11 +123,9 @@ tokenize(String, Line, Column, Opts) ->
         Acc#elixir_tokenizer{preserve_comments=PreserveComments};
       ({unescape, Unescape}, Acc) when is_boolean(Unescape) ->
         Acc#elixir_tokenizer{unescape=Unescape};
-      ({warn_on_unnecessary_quotes, Unnecessary}, Acc) when is_boolean(Unnecessary) ->
-        Acc#elixir_tokenizer{warn_on_unnecessary_quotes=Unnecessary};
       (_, Acc) ->
         Acc
-    end, #elixir_tokenizer{identifier_tokenizer=IdentifierTokenizer}, Opts),
+    end, #elixir_tokenizer{identifier_tokenizer=IdentifierTokenizer, column=Column}, Opts),
 
   tokenize(String, Line, Column, Scope, []).
 
@@ -144,12 +142,19 @@ tokenize([], Line, Column, #elixir_tokenizer{cursor_completion=Cursor} = Scope,
   AccTokens = cursor_complete(Line, CursorColumn, CursorTerminators, CursorTokens),
   {ok, Line, Column, AllWarnings, AccTokens};
 
-tokenize([], EndLine, Column, #elixir_tokenizer{terminators=[{Start, {StartLine, _, _}, _} | _]} = Scope, Tokens) ->
+tokenize([], EndLine, EndColumn, #elixir_tokenizer{terminators=[{Start, {StartLine, StartColumn, _}, _} | _]} = Scope, Tokens) ->
   End = terminator(Start),
   Hint = missing_terminator_hint(Start, End, Scope),
-  Message = "missing terminator: ~ts (for \"~ts\" starting at line ~B)",
+  Message = "missing terminator: ~ts",
-  Formatted = io_lib:format(Message, [End, Start, StartLine]),
+  Formatted = io_lib:format(Message, [End]),
-  Meta = [{opening_delimiter, Start} | ?LOC(EndLine, Column)],
+  Meta = [
+    {opening_delimiter, Start},
+    {expected_delimiter, End},
+    {line, StartLine},
+    {column, StartColumn},
+    {end_line, EndLine},
+    {end_column, EndColumn}
+  ],
   error({Meta, [Formatted, Hint], []}, [], Scope, Tokens);
 
 tokenize([], Line, Column, #elixir_tokenizer{} = Scope, Tokens) ->
@@ -527,7 +532,7 @@ tokenize([$:, H | T] = Original, Line, Column, Scope, Tokens) when ?is_quote(H)
 
     {error, Reason} ->
       Message = " (for atom starting at line ~B)",
-      interpolation_error(Reason, Original, Scope, Tokens, Message, [Line])
+      interpolation_error(Reason, Original, Scope, Tokens, Message, [Line], Line, Column + 1, [H], [H])
   end;
 
 tokenize([$: | String] = Original, Line, Column, Scope, Tokens) ->
@@ -706,9 +711,9 @@ unexpected_token([T | Rest], Line, Column, Scope, Tokens) ->
   error({?LOC(Line, Column), "unexpected token: ", Message}, Rest, Scope, Tokens).
 
 tokenize_eol(Rest, Line, Scope, Tokens) ->
-  {StrippedRest, Indentation} = strip_horizontal_space(Rest, 0),
+  {StrippedRest, Column} = strip_horizontal_space(Rest, Scope#elixir_tokenizer.column),
-  IndentedScope = Scope#elixir_tokenizer{indentation=Indentation},
+  IndentedScope = Scope#elixir_tokenizer{indentation=Column-1},
-  tokenize(StrippedRest, Line + 1, Indentation + 1, IndentedScope, Tokens).
+  tokenize(StrippedRest, Line + 1, Column, IndentedScope, Tokens).
 
 strip_horizontal_space([H | T], Counter) when ?is_horizontal_space(H) ->
   strip_horizontal_space(T, Counter + 1);
@@ -724,12 +729,12 @@ tokenize_dot(T, Line, Column, DotInfo, Scope, Tokens) ->
 
         {Rest, Comment} ->
           preserve_comments(Line, Column, Tokens, Comment, Rest, Scope),
-          tokenize_dot(Rest, Line, 1, DotInfo, Scope, Tokens)
+          tokenize_dot(Rest, Line, Scope#elixir_tokenizer.column, DotInfo, Scope, Tokens)
       end;
     {"\r\n" ++ Rest, _} ->
-      tokenize_dot(Rest, Line + 1, 1, DotInfo, Scope, Tokens);
+      tokenize_dot(Rest, Line + 1, Scope#elixir_tokenizer.column, DotInfo, Scope, Tokens);
     {"\n" ++ Rest, _} ->
-      tokenize_dot(Rest, Line + 1, 1, DotInfo, Scope, Tokens);
+      tokenize_dot(Rest, Line + 1, Scope#elixir_tokenizer.column, DotInfo, Scope, Tokens);
     {Rest, Length} ->
       handle_dot([$. | Rest], Line, Column + Length, DotInfo, Scope, Tokens)
   end.
@@ -768,7 +773,7 @@ handle_heredocs(T, Line, Column, H, Scope, Tokens) ->
 handle_strings(T, Line, Column, H, Scope, Tokens) ->
   case elixir_interpolation:extract(Line, Column, Scope, true, T, H) of
     {error, Reason} ->
-      interpolation_error(Reason, [H | T], Scope, Tokens, " (for string starting at line ~B)", [Line]);
+      interpolation_error(Reason, [H | T], Scope, Tokens, " (for string starting at line ~B)", [Line], Line, Column-1, [H], [H]);
 
     {NewLine, NewColumn, Parts, [$: | Rest], InterScope} when ?is_space(hd(Rest)) ->
       NewScope = case is_unnecessary_quote(Parts, InterScope) of
@@ -794,7 +799,7 @@ handle_strings(T, Line, Column, H, Scope, Tokens) ->
               Token = {kw_identifier, {Line, Column - 1, nil}, Atom},
               tokenize(Rest, NewLine, NewColumn + 1, NewScope, [Token | Tokens]);
             {error, Reason} ->
-              {error, Reason, Rest, Tokens}
+              error(Reason, Rest, NewScope, Tokens)
           end;
 
         {ok, Unescaped} ->
@@ -921,7 +926,7 @@ handle_dot([$., H | T] = Original, Line, Column, DotInfo, Scope, Tokens) when ?i
       Message = "interpolation is not allowed when calling function/macro. Found interpolation in a call starting with: ",
       error({?LOC(Line, Column), Message, [H]}, Rest, NewScope, Tokens);
     {error, Reason} ->
-      interpolation_error(Reason, Original, Scope, Tokens, " (for function name starting at line ~B)", [Line])
+      interpolation_error(Reason, Original, Scope, Tokens, " (for function name starting at line ~B)", [Line], Line, Column, [H], [H])
   end;
 
 handle_dot([$. | Rest], Line, Column, DotInfo, Scope, Tokens) ->
@@ -964,12 +969,11 @@ eol(_Line, _Column, [{eol, {Line, Column, Count}} | Tokens]) ->
 eol(Line, Column, Tokens) ->
   [{eol, {Line, Column, 1}} | Tokens].
 
-is_unnecessary_quote([Part], #elixir_tokenizer{warn_on_unnecessary_quotes=true} = Scope) when is_list(Part) ->
+is_unnecessary_quote([Part], Scope) when is_list(Part) ->
   case (Scope#elixir_tokenizer.identifier_tokenizer):tokenize(Part) of
     {identifier, _, [], _, true, Special} -> not lists:member(at, Special);
     _ -> false
   end;
-
 is_unnecessary_quote(_Parts, _Scope) ->
   false.
 
@@ -1027,7 +1031,7 @@ extract_heredoc_with_interpolation(Line, Column, Scope, Interpol, T, H) ->
           {ok, NewLine, NewColumn, tokens_to_binary(Parts2), Rest, NewScope};
 
         {error, Reason} ->
-          {error, interpolation_format(Reason, " (for heredoc starting at line ~B)", [Line])}
+          {error, interpolation_format(Reason, " (for heredoc starting at line ~B)", [Line], Line, Column, [H, H, H], [H, H, H])}
       end;
 
     error ->
@@ -1340,12 +1344,20 @@ previous_was_eol(_) -> nil.
 
 %% Error handling
 
-interpolation_error(Reason, Rest, Scope, Tokens, Extension, Args) ->
+interpolation_error(Reason, Rest, Scope, Tokens, Extension, Args, Line, Column, Opening, Closing) ->
-  error(interpolation_format(Reason, Extension, Args), Rest, Scope, Tokens).
+  error(interpolation_format(Reason, Extension, Args, Line, Column, Opening, Closing), Rest, Scope, Tokens).
 
-interpolation_format({string, Line, Column, Message, Token}, Extension, Args) ->
+interpolation_format({string, EndLine, EndColumn, Message, Token}, Extension, Args, Line, Column, Opening, Closing) ->
-  {?LOC(Line, Column), [Message, io_lib:format(Extension, Args)], Token};
+  Meta = [
-interpolation_format({_, _, _} = Reason, _Extension, _Args) ->
+    {opening_delimiter, list_to_atom(Opening)},
+    {expected_delimiter, list_to_atom(Closing)},
+    {line, Line},
+    {column, Column},
+    {end_line, EndLine},
+    {end_column, EndColumn}
+  ],
+  {Meta, [Message, io_lib:format(Extension, Args)], Token};
+interpolation_format({_, _, _} = Reason, _Extension, _Args, _Line, _Column, _Opening, _Closing) ->
   Reason.
 
 %% Terminators
@@ -1417,7 +1429,7 @@ check_terminator({End, {EndLine, EndColumn, _}}, [{Start, {StartLine, StartColum
     End ->
       {ok, Scope#elixir_tokenizer{terminators=Terminators}};
 
-    _ExpectedEnd ->
+    ExpectedEnd ->
       Meta = [
         {line, StartLine},
         {column, StartColumn},
@@ -1425,7 +1437,8 @@ check_terminator({End, {EndLine, EndColumn, _}}, [{Start, {StartLine, StartColum
         {end_column, EndColumn},
         {error_type, mismatched_delimiter},
         {opening_delimiter, Start},
-        {closing_delimiter, End}
+        {closing_delimiter, End},
+        {expected_delimiter, ExpectedEnd}
      ],
      {error, {Meta, unexpected_token_or_reserved(End), [atom_to_list(End)]}}
   end;
@@ -1583,7 +1596,7 @@ tokenize_sigil_contents([H | T] = Original, [S | _] = SigilName, Line, Column, S
     {error, Reason} ->
       Sigil = [$~, S, H],
       Message = " (for sigil ~ts starting at line ~B)",
-      interpolation_error(Reason, [$~] ++ SigilName ++ Original, Scope, Tokens, Message, [Sigil, Line])
+      interpolation_error(Reason, [$~] ++ SigilName ++ Original, Scope, Tokens, Message, [Sigil, Line], Line, Column, [H], [sigil_terminator(H)])
   end;
 
 tokenize_sigil_contents([H | _] = Original, SigilName, Line, Column, Scope, Tokens) ->

lib/elixir/test/elixir/code_formatter/containers_test.exs

@@ -203,6 +203,7 @@ defmodule Code.Formatter.ContainersTest do
       assert_same ~S(["\w": 1, "\\w": 2])
       assert_same ~S(["Elixir.Foo": 1, "Elixir.Bar": 2])
       assert_format ~S(["Foo": 1, "Bar": 2]), ~S([Foo: 1, Bar: 2])
+      assert_same ~S(["with \"scare quotes\"": 1])
     end
 
     test "with operators keyword lists" do

lib/elixir/test/elixir/code_formatter/general_test.exs

@@ -296,6 +296,28 @@ defmodule Code.Formatter.GeneralTest do
       assert_same code, @short_length
     end
 
+    test "keeps parens if argument includes keyword list" do
+      assert_same """
+      fn [] when is_integer(x) ->
+        x + 42
+      end
+      """
+
+      bad = """
+      fn (input: x) when is_integer(x) ->
+        x + 42
+      end
+      """
+
+      good = """
+      fn [input: x] when is_integer(x) ->
+        x + 42
+      end
+      """
+
+      assert_format bad, good
+    end
+
     test "with a single clause, followed by a newline, and can fit in one line" do
       assert_same """
       fn

lib/elixir/test/elixir/code_formatter/literals_test.exs

@@ -210,10 +210,15 @@ defmodule Code.Formatter.LiteralsTest do
     test "with escapes" do
       assert_format ~S['f\a\b\ro'], ~S[~c"f\a\b\ro"]
       assert_format ~S['single \' quote'], ~S[~c"single ' quote"]
-      assert_format ~S['double " quote'], ~S[~c"double \" quote"]
+      assert_format ~S['double " quote'], ~S[~c'double " quote']
+      assert_format ~S['escaped \" quote'], ~S[~c'escaped \" quote']
+      assert_format ~S['\\"'], ~S[~c'\\"']
 
       assert_same ~S['f\a\b\ro'], @keep_charlists
       assert_same ~S['single \' quote'], @keep_charlists
+      assert_same ~S['double " quote'], @keep_charlists
+      assert_same ~S['escaped \" quote'], @keep_charlists
+      assert_same ~S['\\"'], @keep_charlists
     end
 
     test "keeps literal new lines" do
@@ -235,13 +240,15 @@ defmodule Code.Formatter.LiteralsTest do
 
     test "with interpolation" do
       assert_format ~S['one #{2} three'], ~S[~c"one #{2} three"]
+      assert_format ~S['#{1}\n \\ " \"'], ~S[~c'#{1}\n \\ " \"']
 
       assert_same ~S['one #{2} three'], @keep_charlists
+      assert_same ~S['#{1}\n \\ " \"'], @keep_charlists
     end
 
     test "with escape and interpolation" do
       assert_format ~S['one\n\'#{2}\'\nthree'], ~S[~c"one\n'#{2}'\nthree"]
-      assert_format ~S['one\n"#{2}"\nthree'], ~S[~c"one\n\"#{2}\"\nthree"]
+      assert_format ~S['one\n"#{2}"\nthree'], ~S[~c'one\n"#{2}"\nthree']
 
       assert_same ~S['one\n\'#{2}\'\nthree'], @keep_charlists
     end

lib/elixir/test/elixir/code_fragment_test.exs

@@ -443,12 +443,11 @@ defmodule CodeFragmentTest do
     end
 
     test "column out of range" do
-      assert CF.surround_context("hello", {1, 20}) ==
+      assert CF.surround_context("hello", {1, 20}) == :none
-               %{begin: {1, 1}, context: {:local_or_var, ~c"hello"}, end: {1, 6}}
     end
 
     test "local_or_var" do
-      for i <- 1..9 do
+      for i <- 1..8 do
         assert CF.surround_context("hello_wo", {1, i}) == %{
                  context: {:local_or_var, ~c"hello_wo"},
                  begin: {1, 1},
@@ -456,9 +455,9 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context("hello_wo ", {1, 10}) == :none
+      assert CF.surround_context("hello_wo", {1, 9}) == :none
 
-      for i <- 2..10 do
+      for i <- 2..9 do
         assert CF.surround_context(" hello_wo", {1, i}) == %{
                  context: {:local_or_var, ~c"hello_wo"},
                  begin: {1, 2},
@@ -466,9 +465,9 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context(" hello_wo ", {1, 11}) == :none
+      assert CF.surround_context(" hello_wo", {1, 10}) == :none
 
-      for i <- 1..7 do
+      for i <- 1..6 do
         assert CF.surround_context("hello!", {1, i}) == %{
                  context: {:local_or_var, ~c"hello!"},
                  begin: {1, 1},
@@ -476,9 +475,9 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context("hello! ", {1, 8}) == :none
+      assert CF.surround_context("hello!", {1, 7}) == :none
 
-      for i <- 1..6 do
+      for i <- 1..5 do
         assert CF.surround_context("안녕_세상", {1, i}) == %{
                  context: {:local_or_var, ~c"안녕_세상"},
                  begin: {1, 1},
@@ -486,7 +485,7 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context("안녕_세상 ", {1, 6}) == :none
+      assert CF.surround_context("안녕_세상", {1, 6}) == :none
 
       # Keywords are not local or var
       for keyword <- ~w(do end after catch else rescue fn true false nil)c do
@@ -500,38 +499,8 @@ defmodule CodeFragmentTest do
       end
     end
 
-    test "local + operator" do
-      for i <- 1..8 do
-        assert CF.surround_context("hello_wo+", {1, i}) == %{
-                 context: {:local_or_var, ~c"hello_wo"},
-                 begin: {1, 1},
-                 end: {1, 9}
-               }
-      end
-
-      assert CF.surround_context("hello_wo+", {1, 9}) == %{
-               begin: {1, 9},
-               context: {:operator, ~c"+"},
-               end: {1, 10}
-             }
-
-      for i <- 1..9 do
-        assert CF.surround_context("hello_wo +", {1, i}) == %{
-                 context: {:local_or_var, ~c"hello_wo"},
-                 begin: {1, 1},
-                 end: {1, 9}
-               }
-      end
-
-      assert CF.surround_context("hello_wo +", {1, 10}) == %{
-               begin: {1, 10},
-               context: {:operator, ~c"+"},
-               end: {1, 11}
-             }
-    end
-
     test "local call" do
-      for i <- 1..9 do
+      for i <- 1..8 do
         assert CF.surround_context("hello_wo(", {1, i}) == %{
                  context: {:local_call, ~c"hello_wo"},
                  begin: {1, 1},
@@ -539,9 +508,9 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context("hello_wo(", {1, 10}) == :none
+      assert CF.surround_context("hello_wo(", {1, 9}) == :none
 
-      for i <- 1..9 do
+      for i <- 1..8 do
         assert CF.surround_context("hello_wo (", {1, i}) == %{
                  context: {:local_call, ~c"hello_wo"},
                  begin: {1, 1},
@@ -549,10 +518,9 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context("hello_wo (", {1, 10}) == :none
+      assert CF.surround_context("hello_wo (", {1, 9}) == :none
-      assert CF.surround_context("hello_wo (", {1, 11}) == :none
 
-      for i <- 1..7 do
+      for i <- 1..6 do
         assert CF.surround_context("hello!(", {1, i}) == %{
                  context: {:local_call, ~c"hello!"},
                  begin: {1, 1},
@@ -560,9 +528,9 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context("hello!(", {1, 8}) == :none
+      assert CF.surround_context("hello!(", {1, 7}) == :none
 
-      for i <- 1..6 do
+      for i <- 1..5 do
         assert CF.surround_context("안녕_세상(", {1, i}) == %{
                  context: {:local_call, ~c"안녕_세상"},
                  begin: {1, 1},
@@ -570,7 +538,7 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context("안녕_세상(", {1, 7}) == :none
+      assert CF.surround_context("안녕_세상(", {1, 6}) == :none
     end
 
     test "local arity" do
@@ -698,7 +666,7 @@ defmodule CodeFragmentTest do
     end
 
     test "alias" do
-      for i <- 1..9 do
+      for i <- 1..8 do
         assert CF.surround_context("HelloWor", {1, i}) == %{
                  context: {:alias, ~c"HelloWor"},
                  begin: {1, 1},
@@ -706,9 +674,9 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context("HelloWor ", {1, 10}) == :none
+      assert CF.surround_context("HelloWor", {1, 9}) == :none
 
-      for i <- 2..10 do
+      for i <- 2..9 do
         assert CF.surround_context(" HelloWor", {1, i}) == %{
                  context: {:alias, ~c"HelloWor"},
                  begin: {1, 2},
@@ -716,9 +684,9 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context(" HelloWor ", {1, 11}) == :none
+      assert CF.surround_context(" HelloWor", {1, 10}) == :none
 
-      for i <- 1..10 do
+      for i <- 1..9 do
         assert CF.surround_context("Hello.Wor", {1, i}) == %{
                  context: {:alias, ~c"Hello.Wor"},
                  begin: {1, 1},
@@ -726,9 +694,9 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context("Hello.Wor ", {1, 11}) == :none
+      assert CF.surround_context("Hello.Wor", {1, 10}) == :none
 
-      for i <- 1..12 do
+      for i <- 1..11 do
         assert CF.surround_context("Hello . Wor", {1, i}) == %{
                  context: {:alias, ~c"Hello.Wor"},
                  begin: {1, 1},
@@ -736,9 +704,9 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context("Hello . Wor ", {1, 13}) == :none
+      assert CF.surround_context("Hello . Wor", {1, 12}) == :none
 
-      for i <- 1..16 do
+      for i <- 1..15 do
         assert CF.surround_context("Foo . Bar . Baz", {1, i}) == %{
                  context: {:alias, ~c"Foo.Bar.Baz"},
                  begin: {1, 1},
@@ -770,7 +738,7 @@ defmodule CodeFragmentTest do
                end: {1, 11}
              }
 
-      for i <- 1..15 do
+      for i <- 1..14 do
         assert CF.surround_context("__MODULE__.Foo", {1, i}) == %{
                  context: {:alias, {:local_or_var, ~c"__MODULE__"}, ~c"Foo"},
                  begin: {1, 1},
@@ -778,7 +746,7 @@ defmodule CodeFragmentTest do
                }
       end
 
-      for i <- 1..19 do
+      for i <- 1..18 do
         assert CF.surround_context("__MODULE__.Foo.Sub", {1, i}) == %{
                  context: {:alias, {:local_or_var, ~c"__MODULE__"}, ~c"Foo.Sub"},
                  begin: {1, 1},
@@ -830,7 +798,7 @@ defmodule CodeFragmentTest do
     end
 
     test "attribute submodules" do
-      for i <- 1..10 do
+      for i <- 1..9 do
         assert CF.surround_context("@some.Foo", {1, i}) == %{
                  context: {:alias, {:module_attribute, ~c"some"}, ~c"Foo"},
                  begin: {1, 1},
@@ -838,7 +806,7 @@ defmodule CodeFragmentTest do
                }
       end
 
-      for i <- 1..14 do
+      for i <- 1..13 do
         assert CF.surround_context("@some.Foo.Sub", {1, i}) == %{
                  context: {:alias, {:module_attribute, ~c"some"}, ~c"Foo.Sub"},
                  begin: {1, 1},
@@ -921,7 +889,7 @@ defmodule CodeFragmentTest do
                end: {1, 15}
              }
 
-      for i <- 2..10 do
+      for i <- 2..9 do
         assert CF.surround_context("%HelloWor", {1, i}) == %{
                  context: {:struct, ~c"HelloWor"},
                  begin: {1, 1},
@@ -929,7 +897,7 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context("%HelloWor ", {1, 11}) == :none
+      assert CF.surround_context("%HelloWor", {1, 10}) == :none
 
       # With dot
       assert CF.surround_context("%Hello.Wor", {1, 1}) == %{
@@ -938,7 +906,7 @@ defmodule CodeFragmentTest do
                end: {1, 11}
              }
 
-      for i <- 2..11 do
+      for i <- 2..10 do
         assert CF.surround_context("%Hello.Wor", {1, i}) == %{
                  context: {:struct, ~c"Hello.Wor"},
                  begin: {1, 1},
@@ -946,7 +914,7 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context("%Hello.Wor ", {1, 12}) == :none
+      assert CF.surround_context("%Hello.Wor", {1, 11}) == :none
 
       # With spaces
       assert CF.surround_context("% Hello . Wor", {1, 1}) == %{
@@ -955,7 +923,7 @@ defmodule CodeFragmentTest do
                end: {1, 14}
              }
 
-      for i <- 2..14 do
+      for i <- 2..13 do
         assert CF.surround_context("% Hello . Wor", {1, i}) == %{
                  context: {:struct, ~c"Hello.Wor"},
                  begin: {1, 1},
@@ -963,7 +931,7 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context("% Hello . Wor ", {1, 15}) == :none
+      assert CF.surround_context("% Hello . Wor", {1, 14}) == :none
     end
 
     test "module attributes" do

lib/elixir/test/elixir/code_normalizer/quoted_ast_test.exs

@@ -229,6 +229,12 @@ defmodule Code.Normalizer.QuotedASTTest do
     test "invalid block" do
       assert quoted_to_string({:__block__, [], {:bar, [], []}}) ==
                "{:__block__, [], {:bar, [], []}}"
+
+      assert quoted_to_string({:foo, [], [{:do, :ok}, :not_keyword]}) ==
+               "foo({:do, :ok}, :not_keyword)"
+
+      assert quoted_to_string({:foo, [], [[{:do, :ok}, :not_keyword]]}) ==
+               "foo([{:do, :ok}, :not_keyword])"
     end
 
     test "not in" do
@@ -533,6 +539,10 @@ defmodule Code.Normalizer.QuotedASTTest do
                "\e[34m[\e[0m\e[32ma:\e[0m \e[33m1\e[0m, \e[32mb:\e[0m \e[33m2\e[0m\e[34m]\e[0m"
     end
 
+    test "keyword list with :do as operand" do
+      assert quoted_to_string(quote(do: a = [do: 1])) == "a = [do: 1]"
+    end
+
     test "interpolation" do
       assert quoted_to_string(quote(do: "foo#{bar}baz")) == ~S["foo#{bar}baz"]
     end
@@ -698,10 +708,10 @@ defmodule Code.Normalizer.QuotedASTTest do
                ~S[~c"one\n'#{2}'\nthree"]
 
       assert Code.string_to_quoted!(~S/'one\n"#{2}"\nthree'/) |> quoted_to_string(escape: false) ==
-               ~s[~c"one\n\\"\#{2}\\"\nthree"]
+               ~s[~c'one\n"\#{2}"\nthree']
 
       assert Code.string_to_quoted!(~S/'one\n"#{2}"\nthree'/) |> quoted_to_string() ==
-               ~S[~c"one\n\"#{2}\"\nthree"]
+               ~S[~c'one\n"#{2}"\nthree']
     end
 
     test "atoms" do

lib/elixir/test/elixir/code_test.exs

@@ -252,6 +252,7 @@ defmodule CodeTest do
                  message: "undefined variable \"x\"",
                  position: 1,
                  file: "nofile",
+                 source: "nofile",
                  stacktrace: [],
                  severity: :error
                }
@@ -483,7 +484,7 @@ defmodule CodeTest do
 
     assert_exception(
       SyntaxError,
-      ["nofile:11:5:", "syntax error before:", "1 + * 3", "^"],
+      ["nofile:11:7:", "syntax error before:", "1 + * 3", "^"],
       fn ->
         Code.string_to_quoted!(":ok\n1 + * 3", line: 10, column: 3)
       end
@@ -498,9 +499,13 @@ defmodule CodeTest do
   end
 
   test "string_to_quoted returns error on incomplete escaped string" do
-    assert Code.string_to_quoted("\"\\") ==
+    assert {:error, {meta, "missing terminator: \" (for string starting at line 1)", ""}} =
-             {:error,
+             Code.string_to_quoted("\"\\")
-              {[line: 1, column: 3], "missing terminator: \" (for string starting at line 1)", ""}}
+
+    assert meta[:line] == 1
+    assert meta[:column] == 1
+    assert meta[:end_line] == 1
+    assert meta[:end_column] == 3
   end
 
   test "compile source" do

lib/elixir/test/elixir/enum_test.exs

@@ -56,9 +56,9 @@ defmodule EnumTest do
     end
 
     test "mix and match" do
-      enums = [[1, 2], %{a: 3, b: 4}, [5, 6]]
+      enums = [[1, 2], 3..4, [5, 6]]
       result = Enum.zip_reduce(enums, [], fn elements, acc -> [List.to_tuple(elements) | acc] end)
-      assert result == [{2, {:b, 4}, 6}, {1, {:a, 3}, 5}]
+      assert result == [{2, 4, 6}, {1, 3, 5}]
     end
   end
 
@@ -412,7 +412,7 @@ defmodule EnumTest do
     assert Enum.into([a: 1, b: 2], %{c: 3}) == %{a: 1, b: 2, c: 3}
     assert Enum.into(MapSet.new(a: 1, b: 2), %{}) == %{a: 1, b: 2}
     assert Enum.into(MapSet.new(a: 1, b: 2), %{c: 3}) == %{a: 1, b: 2, c: 3}
-    assert Enum.into(%{a: 1, b: 2}, []) == [a: 1, b: 2]
+    assert Enum.into(%{a: 1, b: 2}, []) |> Enum.sort() == [a: 1, b: 2]
     assert Enum.into(1..3, []) == [1, 2, 3]
     assert Enum.into(["H", "i"], "") == "Hi"
   end
@@ -1430,7 +1430,7 @@ defmodule EnumTest do
   test "unzip/1" do
     assert Enum.unzip([{:a, 1}, {:b, 2}, {:c, 3}]) == {[:a, :b, :c], [1, 2, 3]}
     assert Enum.unzip([]) == {[], []}
-    assert Enum.unzip(%{a: 1, b: 2}) == {[:a, :b], [1, 2]}
+    assert Enum.unzip(%{a: 1}) == {[:a], [1]}
     assert Enum.unzip(foo: "a", bar: "b") == {[:foo, :bar], ["a", "b"]}
 
     assert_raise FunctionClauseError, fn -> Enum.unzip([{:a, 1}, {:b, 2, "foo"}]) end

lib/elixir/test/elixir/file/stream_test.exs

@@ -141,6 +141,35 @@ defmodule File.StreamTest do
         end
       end
 
+      test "supports byte offset" do
+        src = fixture_path("file.txt")
+
+        assert @node
+               |> stream!(src, [{:read_offset, 0}])
+               |> Enum.take(1) == ["FOO\n"]
+
+        assert @node
+               |> stream!(src, [{:read_offset, 1}])
+               |> Enum.take(1) == ["OO\n"]
+
+        assert @node
+               |> stream!(src, [{:read_offset, 4}])
+               |> Enum.take(1) == []
+
+        assert @node |> stream!(src, 1, [{:read_offset, 1}]) |> Enum.count() == 3
+        assert @node |> stream!(src, 1, [{:read_offset, 4}]) |> Enum.count() == 0
+      end
+
+      test "applies offset after trimming BOM" do
+        src = fixture_path("utf8_bom.txt")
+
+        assert @node
+               |> stream!(src, [:trim_bom, {:read_offset, 4}])
+               |> Enum.take(1) == ["сский\n"]
+
+        assert @node |> stream!(src, 1, [:trim_bom, {:read_offset, 4}]) |> Enum.count() == 15
+      end
+
       test "keeps BOM when raw" do
         src = fixture_path("utf8_bom.txt")
 
@@ -169,6 +198,7 @@ defmodule File.StreamTest do
 
         assert @node |> stream!(src, [:trim_bom]) |> Enum.count() == 2
         assert @node |> stream!(src, 1, [:trim_bom]) |> Enum.count() == 19
+        assert @node |> stream!(src, 2, [:trim_bom]) |> Enum.count() == 10
       end
 
       test "keeps BOM with utf8 encoding" do

lib/elixir/test/elixir/float_test.exs

@@ -5,6 +5,17 @@ defmodule FloatTest do
 
   doctest Float
 
+  # TODO remove and replace by assert once we require Erlang/OTP 27+
+  # We can't easily distinguish between -0.0 and +0.0 on previous version
+  defmacrop float_assert({:===, _, [left, right]}) do
+    quote do
+      # note: these are pure functions so no need to use bind_quoted
+      # we favor a useful error message instead
+      assert unquote(left) === unquote(right)
+      assert to_string(unquote(left)) === to_string(unquote(right))
+    end
+  end
+
   test "parse/1" do
     assert Float.parse("12") === {12.0, ""}
     assert Float.parse("-12") === {-12.0, ""}
@@ -45,39 +56,39 @@ defmodule FloatTest do
   end
 
   test "floor/1" do
-    assert Float.floor(12.524235) === 12.0
+    float_assert Float.floor(12.524235) === 12.0
-    assert Float.floor(-12.5) === -13.0
+    float_assert Float.floor(-12.5) === -13.0
-    assert Float.floor(-12.524235) === -13.0
+    float_assert Float.floor(-12.524235) === -13.0
-    assert Float.floor(7.5e3) === 7500.0
+    float_assert Float.floor(7.5e3) === 7500.0
-    assert Float.floor(7.5432e3) === 7543.0
+    float_assert Float.floor(7.5432e3) === 7543.0
-    assert Float.floor(7.5e-3) === 0.0
+    float_assert Float.floor(7.5e-3) === 0.0
-    assert Float.floor(-12.32453e4) === -123_246.0
+    float_assert Float.floor(-12.32453e4) === -123_246.0
-    assert Float.floor(-12.32453e-10) === -1.0
+    float_assert Float.floor(-12.32453e-10) === -1.0
-    assert Float.floor(0.32453e-10) === 0.0
+    float_assert Float.floor(0.32453e-10) === 0.0
-    assert Float.floor(-0.32453e-10) === -1.0
+    float_assert Float.floor(-0.32453e-10) === -1.0
-    assert Float.floor(1.32453e-10) === 0.0
+    float_assert Float.floor(1.32453e-10) === 0.0
   end
 
   describe "floor/2" do
     test "with 0.0" do
       for precision <- 0..15 do
-        assert Float.floor(0.0, precision) === 0.0
+        float_assert Float.floor(0.0, precision) === 0.0
-        assert Float.floor(-0.0, precision) === -0.0
+        float_assert Float.floor(-0.0, precision) === -0.0
       end
     end
 
     test "floor/2 with precision" do
-      assert Float.floor(12.524235, 0) === 12.0
+      float_assert Float.floor(12.524235, 0) === 12.0
-      assert Float.floor(-12.524235, 0) === -13.0
+      float_assert Float.floor(-12.524235, 0) === -13.0
 
-      assert Float.floor(12.52, 2) === 12.51
+      float_assert Float.floor(12.52, 2) === 12.51
-      assert Float.floor(-12.52, 2) === -12.52
+      float_assert Float.floor(-12.52, 2) === -12.52
 
-      assert Float.floor(12.524235, 2) === 12.52
+      float_assert Float.floor(12.524235, 2) === 12.52
-      assert Float.floor(-12.524235, 3) === -12.525
+      float_assert Float.floor(-12.524235, 3) === -12.525
 
-      assert Float.floor(12.32453e-20, 2) === 0.0
+      float_assert Float.floor(12.32453e-20, 2) === 0.0
-      assert Float.floor(-12.32453e-20, 2) === -0.01
+      float_assert Float.floor(-12.32453e-20, 2) === -0.01
 
       assert_raise ArgumentError, "precision 16 is out of valid range of 0..15", fn ->
         Float.floor(1.1, 16)
@@ -85,68 +96,73 @@ defmodule FloatTest do
     end
 
     test "with subnormal floats" do
-      assert Float.floor(-5.0e-324, 0) === -1.0
+      float_assert Float.floor(-5.0e-324, 0) === -1.0
-      assert Float.floor(-5.0e-324, 1) === -0.1
+      float_assert Float.floor(-5.0e-324, 1) === -0.1
-      assert Float.floor(-5.0e-324, 2) === -0.01
+      float_assert Float.floor(-5.0e-324, 2) === -0.01
-      assert Float.floor(-5.0e-324, 15) === -0.000000000000001
+      float_assert Float.floor(-5.0e-324, 15) === -0.000000000000001
 
       for precision <- 0..15 do
-        assert Float.floor(5.0e-324, precision) === 0.0
+        float_assert Float.floor(5.0e-324, precision) === 0.0
       end
     end
   end
 
   test "ceil/1" do
-    assert Float.ceil(12.524235) === 13.0
+    float_assert Float.ceil(12.524235) === 13.0
-    assert Float.ceil(-12.5) === -12.0
+    float_assert Float.ceil(-12.5) === -12.0
-    assert Float.ceil(-12.524235) === -12.0
+    float_assert Float.ceil(-12.524235) === -12.0
-    assert Float.ceil(7.5e3) === 7500.0
+    float_assert Float.ceil(7.5e3) === 7500.0
-    assert Float.ceil(7.5432e3) === 7544.0
+    float_assert Float.ceil(7.5432e3) === 7544.0
-    assert Float.ceil(7.5e-3) === 1.0
+    float_assert Float.ceil(7.5e-3) === 1.0
-    assert Float.ceil(-12.32453e4) === -123_245.0
+    float_assert Float.ceil(-12.32453e4) === -123_245.0
-    assert Float.ceil(-12.32453e-10) === 0.0
+    float_assert Float.ceil(-12.32453e-10) === -0.0
-    assert Float.ceil(0.32453e-10) === 1.0
+    float_assert Float.ceil(0.32453e-10) === 1.0
-    assert Float.ceil(-0.32453e-10) === 0.0
+    float_assert Float.ceil(-0.32453e-10) === -0.0
-    assert Float.ceil(1.32453e-10) === 1.0
+    float_assert Float.ceil(1.32453e-10) === 1.0
-    assert Float.ceil(0.0) === 0.0
+    float_assert Float.ceil(0.0) === 0.0
   end
 
   describe "ceil/2" do
     test "with 0.0" do
       for precision <- 0..15 do
-        assert Float.ceil(0.0, precision) === 0.0
+        float_assert Float.ceil(0.0, precision) === 0.0
-        assert Float.ceil(-0.0, precision) === -0.0
+        float_assert Float.ceil(-0.0, precision) === -0.0
       end
     end
 
     test "with regular floats" do
-      assert Float.ceil(12.524235, 0) === 13.0
+      float_assert Float.ceil(12.524235, 0) === 13.0
-      assert Float.ceil(-12.524235, 0) === -12.0
+      float_assert Float.ceil(-12.524235, 0) === -12.0
 
-      assert Float.ceil(12.52, 2) === 12.52
+      float_assert Float.ceil(12.52, 2) === 12.52
-      assert Float.ceil(-12.52, 2) === -12.51
+      float_assert Float.ceil(-12.52, 2) === -12.51
 
-      assert Float.ceil(12.524235, 2) === 12.53
+      float_assert Float.ceil(12.524235, 2) === 12.53
-      assert Float.ceil(-12.524235, 3) === -12.524
+      float_assert Float.ceil(-12.524235, 3) === -12.524
 
-      assert Float.ceil(12.32453e-20, 2) === 0.01
+      float_assert Float.ceil(12.32453e-20, 2) === 0.01
-      assert Float.ceil(-12.32453e-20, 2) === 0.0
+      float_assert Float.ceil(-12.32453e-20, 2) === -0.0
 
-      assert Float.ceil(0.0, 2) === 0.0
+      float_assert Float.ceil(0.0, 2) === 0.0
 
       assert_raise ArgumentError, "precision 16 is out of valid range of 0..15", fn ->
         Float.ceil(1.1, 16)
       end
     end
 
+    test "with small floats rounded up to -0.0" do
+      float_assert Float.ceil(-0.1, 0) === -0.0
+      float_assert Float.ceil(-0.01, 1) === -0.0
+    end
+
     test "with subnormal floats" do
-      assert Float.ceil(5.0e-324, 0) === 1.0
+      float_assert Float.ceil(5.0e-324, 0) === 1.0
-      assert Float.ceil(5.0e-324, 1) === 0.1
+      float_assert Float.ceil(5.0e-324, 1) === 0.1
-      assert Float.ceil(5.0e-324, 2) === 0.01
+      float_assert Float.ceil(5.0e-324, 2) === 0.01
-      assert Float.ceil(5.0e-324, 15) === 0.000000000000001
+      float_assert Float.ceil(5.0e-324, 15) === 0.000000000000001
 
       for precision <- 0..15 do
-        assert Float.ceil(-5.0e-324, precision) === -0.0
+        float_assert Float.ceil(-5.0e-324, precision) === -0.0
       end
     end
   end
@@ -154,28 +170,39 @@ defmodule FloatTest do
   describe "round/2" do
     test "with 0.0" do
       for precision <- 0..15 do
-        assert Float.round(0.0, precision) === 0.0
+        float_assert Float.round(0.0, precision) === 0.0
-        assert Float.round(-0.0, precision) === -0.0
+        float_assert Float.round(-0.0, precision) === -0.0
       end
     end
 
     test "with regular floats" do
-      assert Float.round(5.5675, 3) === 5.567
+      float_assert Float.round(5.5675, 3) === 5.567
-      assert Float.round(-5.5674, 3) === -5.567
+      float_assert Float.round(-5.5674, 3) === -5.567
-      assert Float.round(5.5, 3) === 5.5
+      float_assert Float.round(5.5, 3) === 5.5
-      assert Float.round(5.5e-10, 10) === 5.0e-10
+      float_assert Float.round(5.5e-10, 10) === 5.0e-10
-      assert Float.round(5.5e-10, 8) === 0.0
+      float_assert Float.round(5.5e-10, 8) === 0.0
-      assert Float.round(5.0, 0) === 5.0
+      float_assert Float.round(5.0, 0) === 5.0
 
       assert_raise ArgumentError, "precision 16 is out of valid range of 0..15", fn ->
         Float.round(1.1, 16)
       end
     end
 
+    test "with small floats rounded to +0.0 / -0.0" do
+      float_assert Float.round(0.01, 0) === 0.0
+      float_assert Float.round(0.01, 1) === 0.0
+
+      float_assert Float.round(-0.01, 0) === -0.0
+      float_assert Float.round(-0.01, 1) === -0.0
+
+      float_assert Float.round(-0.49999, 0) === -0.0
+      float_assert Float.round(-0.049999, 1) === -0.0
+    end
+
     test "with subnormal floats" do
       for precision <- 0..15 do
-        assert Float.round(5.0e-324, precision) === 0.0
+        float_assert Float.round(5.0e-324, precision) === 0.0
-        assert Float.round(-5.0e-324, precision) === -0.0
+        float_assert Float.round(-5.0e-324, precision) === -0.0
       end
     end
   end

lib/elixir/test/elixir/io/ansi/docs_test.exs

@@ -187,8 +187,11 @@ defmodule IO.ANSI.DocsTest do
       assert result == "line\n\e[0m\n\e[36m    code\n    code2\e[0m\n\e[0m\nline2\n\e[0m"
       result = format_markdown("line\n```elixir\ncode\ncode2\n```\nline2\n")
       assert result == "line\n\e[0m\n\e[36m    code\n    code2\e[0m\n\e[0m\nline2\n\e[0m"
-      result = format_markdown("line\n~~~elixir\ncode\n```\n~~~\nline2\n")
+    end
-      assert result == "line\n\e[0m\n\e[36m    code\n    ```\e[0m\n\e[0m\nline2\n\e[0m"
+
+    test "mermaid fenced code block is discarded" do
+      result = format_markdown("line\n```mermaid\ncode\ncode2\n```\nline2\n")
+      assert result == "line\n\e[0m\nline2\n\e[0m"
     end
 
     test "* list is converted" do

lib/elixir/test/elixir/kernel/binary_test.exs

@@ -119,7 +119,7 @@ defmodule Kernel.BinaryTest do
       Code.eval_string(~s["foo" <> 1])
     end
 
-    message = ~r"left argument of <> operator inside a match"
+    message = ~r"cannot perform prefix match because the left operand of <> has unknown size."
 
     assert_raise ArgumentError, message, fn ->
       Code.eval_string(~s[a <> "b" = "ab"])
@@ -128,20 +128,6 @@ defmodule Kernel.BinaryTest do
     assert_raise ArgumentError, message, fn ->
       Code.eval_string(~s["a" <> b <> "c" = "abc"])
     end
-
-    assert_raise ArgumentError, message, fn ->
-      Code.eval_string(~s[
-        a = "a"
-        ^a <> "b" = "ab"
-      ])
-    end
-
-    assert_raise ArgumentError, message, fn ->
-      Code.eval_string(~s[
-        b = "b"
-        "a" <> ^b <> "c" = "abc"
-      ])
-    end
   end
 
   test "hex" do
@@ -269,6 +255,34 @@ defmodule Kernel.BinaryTest do
     assert <<1::size((^foo).bar)>> = <<1::5>>
   end
 
+  test "bitsyntax size with pinned integer" do
+    a = 1
+    b = <<2, 3>>
+    assert <<^a, ^b::binary>> = <<1, 2, 3>>
+  end
+
+  test "automatic size computation of matched bitsyntax variable" do
+    var = "foo"
+    <<^var::binary, rest::binary>> = "foobar"
+    assert rest == "bar"
+
+    <<^var::bytes, rest::bytes>> = "foobar"
+    assert rest == "bar"
+
+    ^var <> rest = "foobar"
+    assert rest == "bar"
+
+    var = <<0, 1>>
+    <<^var::bitstring, rest::bitstring>> = <<0, 1, 2, 3>>
+    assert rest == <<2, 3>>
+
+    <<^var::bits, rest::bits>> = <<0, 1, 2, 3>>
+    assert rest == <<2, 3>>
+
+    ^var <> rest = <<0, 1, 2, 3>>
+    assert rest == <<2, 3>>
+  end
+
   defmacro signed_16 do
     quote do
       big - signed - integer - unit(16)

lib/elixir/test/elixir/kernel/diagnostics_test.exs

@@ -53,6 +53,28 @@ defmodule Kernel.DiagnosticsTest do
              """
     end
 
+    test "same line with offset" do
+      output =
+        capture_raise(
+          """
+          [1, 2, 3, 4, 5, 6)
+          """,
+          MismatchedDelimiterError,
+          line: 3
+        )
+
+      assert output == """
+             ** (MismatchedDelimiterError) mismatched delimiter found on nofile:3:18:
+                 error: unexpected token: )
+                 │
+               3 │ [1, 2, 3, 4, 5, 6)
+                 │ │                └ mismatched closing delimiter (expected "]")
+                 │ └ unclosed delimiter
+                 │
+                 └─ nofile:3:18\
+             """
+    end
+
     test "two-line span" do
       output =
         capture_raise(
@@ -76,6 +98,30 @@ defmodule Kernel.DiagnosticsTest do
              """
     end
 
+    test "two-line span with offset" do
+      output =
+        capture_raise(
+          """
+          [a, b, c
+           d, f, g}
+          """,
+          MismatchedDelimiterError,
+          line: 3
+        )
+
+      assert output == """
+             ** (MismatchedDelimiterError) mismatched delimiter found on nofile:4:9:
+                 error: unexpected token: }
+                 │
+               3 │ [a, b, c
+                 │ └ unclosed delimiter
+               4 │  d, f, g}
+                 │         └ mismatched closing delimiter (expected "]")
+                 │
+                 └─ nofile:4:9\
+             """
+    end
+
     test "many-line span" do
       output =
         capture_raise(
@@ -103,7 +149,9 @@ defmodule Kernel.DiagnosticsTest do
                  │
                  └─ nofile:5:5\
              """
+    end
 
+    test "many-line span with offset" do
       output =
         capture_raise(
           """
@@ -111,20 +159,21 @@ defmodule Kernel.DiagnosticsTest do
             IO.inspect(2 + 2) + 2
           )
           """,
-          MismatchedDelimiterError
+          MismatchedDelimiterError,
+          line: 3
         )
 
       assert output == """
-             ** (MismatchedDelimiterError) mismatched delimiter found on nofile:3:1:
+             ** (MismatchedDelimiterError) mismatched delimiter found on nofile:5:1:
                  error: unexpected token: )
                  │
-               1 │ fn always_forget_end ->
+               3 │ fn always_forget_end ->
                  │ └ unclosed delimiter
-               2 │   IO.inspect(2 + 2) + 2
+               4 │   IO.inspect(2 + 2) + 2
-               3 │ )
+               5 │ )
                  │ └ mismatched closing delimiter (expected "end")
                  │
-                 └─ nofile:3:1\
+                 └─ nofile:5:1\
              """
     end
 
@@ -153,7 +202,7 @@ defmodule Kernel.DiagnosticsTest do
              """
     end
 
-    test "trim inbetween lines if too many" do
+    test "trim in between lines if too many" do
       output =
         capture_raise(
           """
@@ -288,18 +337,247 @@ defmodule Kernel.DiagnosticsTest do
     end
   end
 
+  describe "token missing error" do
+    test "missing parens terminator" do
+      output =
+        capture_raise(
+          """
+          my_numbers = [1, 2, 3, 4, 5, 6
+          IO.inspect(my_numbers)
+          """,
+          TokenMissingError
+        )
+
+      assert output == """
+             ** (TokenMissingError) token missing on nofile:2:23:
+                 error: missing terminator: ]
+                 │
+               1 │ my_numbers = [1, 2, 3, 4, 5, 6
+                 │              └ unclosed delimiter
+               2 │ IO.inspect(my_numbers)
+                 │                       └ missing closing delimiter (expected "]")
+                 │
+                 └─ nofile:2:23\
+             """
+    end
+
+    test "missing heredoc terminator" do
+      output =
+        capture_raise(
+          """
+          a = \"""
+          test string
+
+          IO.inspect(10 + 20)
+          """,
+          TokenMissingError
+        )
+
+      assert output == """
+             ** (TokenMissingError) token missing on nofile:4:20:
+                 error: missing terminator: \""" (for heredoc starting at line 1)
+                 │
+               1 │ a = \"""
+                 │     └ unclosed delimiter
+               2 │ test string
+               3 │\s
+               4 │ IO.inspect(10 + 20)
+                 │                    └ missing closing delimiter (expected \""")
+                 │
+                 └─ nofile:4:20\
+             """
+    end
+
+    test "missing sigil terminator" do
+      output =
+        capture_raise("~s<foobar", TokenMissingError)
+
+      assert output == """
+             ** (TokenMissingError) token missing on nofile:1:10:
+                 error: missing terminator: > (for sigil ~s< starting at line 1)
+                 │
+               1 │ ~s<foobar
+                 │   │      └ missing closing delimiter (expected ">")
+                 │   └ unclosed delimiter
+                 │
+                 └─ nofile:1:10\
+             """
+
+      output =
+        capture_raise("~s|foobar", TokenMissingError)
+
+      assert output == """
+             ** (TokenMissingError) token missing on nofile:1:10:
+                 error: missing terminator: | (for sigil ~s| starting at line 1)
+                 │
+               1 │ ~s|foobar
+                 │   │      └ missing closing delimiter (expected "|")
+                 │   └ unclosed delimiter
+                 │
+                 └─ nofile:1:10\
+             """
+    end
+
+    test "missing string terminator" do
+      output =
+        capture_raise("\"foobar", TokenMissingError)
+
+      assert output == """
+             ** (TokenMissingError) token missing on nofile:1:8:
+                 error: missing terminator: " (for string starting at line 1)
+                 │
+               1 │ "foobar
+                 │ │      └ missing closing delimiter (expected ")
+                 │ └ unclosed delimiter
+                 │
+                 └─ nofile:1:8\
+             """
+    end
+
+    test "missing atom terminator" do
+      output =
+        capture_raise(":\"foobar", TokenMissingError)
+
+      assert output == """
+             ** (TokenMissingError) token missing on nofile:1:9:
+                 error: missing terminator: " (for atom starting at line 1)
+                 │
+               1 │ :"foobar
+                 │  │      └ missing closing delimiter (expected ")
+                 │  └ unclosed delimiter
+                 │
+                 └─ nofile:1:9\
+             """
+    end
+
+    test "missing function terminator" do
+      output =
+        capture_raise("K.\"foobar", TokenMissingError)
+
+      assert output == """
+             ** (TokenMissingError) token missing on nofile:1:10:
+                 error: missing terminator: " (for function name starting at line 1)
+                 │
+               1 │ K."foobar
+                 │   │      └ missing closing delimiter (expected ")
+                 │   └ unclosed delimiter
+                 │
+                 └─ nofile:1:10\
+             """
+    end
+
+    test "shows in between lines if EOL is not far below" do
+      output =
+        capture_raise(
+          """
+          my_numbers = [1, 2, 3, 4, 5, 6
+          my_numbers
+          |> Enum.map(&(&1 + 1))
+          |> Enum.map(&(&1 * &1))
+          |> IO.inspect()
+          """,
+          TokenMissingError
+        )
+
+      assert output == """
+             ** (TokenMissingError) token missing on nofile:5:16:
+                 error: missing terminator: ]
+                 │
+               1 │ my_numbers = [1, 2, 3, 4, 5, 6
+                 │              └ unclosed delimiter
+               2 │ my_numbers
+               3 │ |> Enum.map(&(&1 + 1))
+               4 │ |> Enum.map(&(&1 * &1))
+               5 │ |> IO.inspect()
+                 │                └ missing closing delimiter (expected "]")
+                 │
+                 └─ nofile:5:16\
+             """
+    end
+
+    test "trims lines" do
+      output =
+        capture_raise(
+          """
+          my_numbers = (1, 2, 3, 4, 5, 6
+
+
+
+
+
+
+          IO.inspect(my_numbers)
+          """,
+          TokenMissingError
+        )
+
+      assert output == """
+             ** (TokenMissingError) token missing on nofile:8:23:
+                 error: missing terminator: )
+                 │
+               1 │ my_numbers = (1, 2, 3, 4, 5, 6
+                 │              └ unclosed delimiter
+              ...
+               8 │ IO.inspect(my_numbers)
+                 │                       └ missing closing delimiter (expected ")")
+                 │
+                 └─ nofile:8:23\
+             """
+    end
+
+    test "shows the last non-empty line of a file" do
+      output =
+        capture_raise(
+          """
+          my_numbers = {1, 2, 3, 4, 5, 6
+          IO.inspect(my_numbers)
+
+
+
+
+          """,
+          TokenMissingError
+        )
+
+      assert output == """
+             ** (TokenMissingError) token missing on nofile:2:23:
+                 error: missing terminator: }
+                 │
+               1 │ my_numbers = {1, 2, 3, 4, 5, 6
+                 │              └ unclosed delimiter
+               2 │ IO.inspect(my_numbers)
+                 │                       └ missing closing delimiter (expected "}")
+                 │
+                 └─ nofile:2:23\
+             """
+    end
+
+    test "supports unicode" do
+      output =
+        capture_raise(
+          """
+          my_emojis = [1, 2, 3, 4 # ⚗️
+          IO.inspect(my_numbers)
+          """,
+          TokenMissingError
+        )
+
+      assert output == """
+             ** (TokenMissingError) token missing on nofile:2:23:
+                 error: missing terminator: ]
+                 │
+               1 │ my_emojis = [1, 2, 3, 4 # ⚗️
+                 │             └ unclosed delimiter
+               2 │ IO.inspect(my_numbers)
+                 │                       └ missing closing delimiter (expected "]")
+                 │
+                 └─ nofile:2:23\
+             """
+    end
+  end
+
   describe "compile-time exceptions" do
     test "SyntaxError (snippet)" do
-      expected = """
-      ** (SyntaxError) invalid syntax found on nofile:1:17:
-          error: syntax error before: '*'
-          │
-        1 │ [1, 2, 3, 4, 5, *]
-          │                 ^
-          │
-          └─ nofile:1:17\
-      """
-
       output =
         capture_raise(
           """
@@ -308,20 +586,39 @@ defmodule Kernel.DiagnosticsTest do
           SyntaxError
         )
 
-      assert output == expected
+      assert output == """
+             ** (SyntaxError) invalid syntax found on nofile:1:17:
+                 error: syntax error before: '*'
+                 │
+               1 │ [1, 2, 3, 4, 5, *]
+                 │                 ^
+                 │
+                 └─ nofile:1:17\
+             """
+    end
+
+    test "SyntaxError (snippet) with offset" do
+      output =
+        capture_raise(
+          """
+          [1, 2, 3, 4, 5, *]
+          """,
+          SyntaxError,
+          line: 3
+        )
+
+      assert output == """
+             ** (SyntaxError) invalid syntax found on nofile:3:17:
+                 error: syntax error before: '*'
+                 │
+               3 │ [1, 2, 3, 4, 5, *]
+                 │                 ^
+                 │
+                 └─ nofile:3:17\
+             """
     end
 
     test "TokenMissingError (snippet)" do
-      expected = """
-      ** (TokenMissingError) token missing on nofile:1:4:
-          error: syntax error: expression is incomplete
-          │
-        1 │ 1 +
-          │    ^
-          │
-          └─ nofile:1:4\
-      """
-
       output =
         capture_raise(
           """
@@ -330,14 +627,49 @@ defmodule Kernel.DiagnosticsTest do
           TokenMissingError
         )
 
-      assert output == expected
+      assert output == """
+             ** (TokenMissingError) token missing on nofile:1:4:
+                 error: syntax error: expression is incomplete
+                 │
+               1 │ 1 +
+                 │    ^
+                 │
+                 └─ nofile:1:4\
+             """
     end
 
-    test "TokenMissingError (no snippet)" do
+    test "TokenMissingError (snippet) with offset and column" do
+      output =
+        capture_raise(
+          """
+          1 +
+          """,
+          TokenMissingError,
+          line: 3,
+          column: 3
+        )
+
+      assert output == """
+             ** (TokenMissingError) token missing on nofile:3:6:
+                 error: syntax error: expression is incomplete
+                 │
+               3 │   1 +
+                 │      ^
+                 │
+                 └─ nofile:3:6\
+             """
+    end
+
+    test "TokenMissingError (unclosed delimiter)" do
       expected = """
-      ** (TokenMissingError) token missing on nofile:2:1:
+      ** (TokenMissingError) token missing on nofile:1:5:
-         error: missing terminator: end (for "fn" starting at line 1)
+          error: missing terminator: end
-         └─ nofile:2:1\
+          │
+        1 │ fn a
+          │ │   └ missing closing delimiter (expected "end")
+          │ └ unclosed delimiter
+          │
+          └─ nofile:1:5\
       """
 
       output =
@@ -419,7 +751,7 @@ defmodule Kernel.DiagnosticsTest do
           1 -
           """,
           TokenMissingError,
-          fake_stacktrace
+          stacktrace: fake_stacktrace
         )
 
       assert output == expected
@@ -448,7 +780,7 @@ defmodule Kernel.DiagnosticsTest do
           1 -
           """,
           TokenMissingError,
-          fake_stacktrace
+          stacktrace: fake_stacktrace
         )
 
       assert output == expected
@@ -521,7 +853,7 @@ defmodule Kernel.DiagnosticsTest do
           └─ #{path}:3: Sample.a/0
       """
 
-      assert capture_eval(source, false) =~ expected
+      assert capture_eval(source, columns: false) =~ expected
     after
       purge(Sample)
     end
@@ -544,101 +876,65 @@ defmodule Kernel.DiagnosticsTest do
     end
 
     @tag :tmp_dir
-    test "long message (file)", %{tmp_dir: tmp_dir} do
+    test "IO.warn file+line", %{tmp_dir: tmp_dir} do
-      path = make_relative_tmp(tmp_dir, "long-warning.ex")
+      path = make_relative_tmp(tmp_dir, "io-warn-file-line.ex")
 
       source = """
-      defmodule Sample do
+      IO.warn("oops\\nmulti\\nline", file: __ENV__.file, line: __ENV__.line)
-        @file "#{path}"
-
-        def atom_case do
-          v = "bc"
-
-          case v do
-            _ when is_atom(v) -> :ok
-            _ -> :fail
-          end
-        end
-      end
       """
 
       File.write!(path, source)
 
       expected = """
-          warning: incompatible types:
+          warning: oops
-
+          multi
-              binary() !~ atom()
+          line
-
-          in expression:
-
-              # #{path}:8
-              is_atom(v)
-
-          where "v" was given the type binary() in:
-
-              # #{path}:5
-              v = "bc"
-
-          where "v" was given the type atom() in:
-
-              # #{path}:8
-              is_atom(v)
-
-          Conflict found at
           │
-        8 │       _ when is_atom(v) -> :ok
+        1 │ IO.warn("oops\\nmulti\\nline", file: __ENV__.file, line: __ENV__.line)
-          │              ~
+          │ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           │
-          └─ #{path}:8:14: Sample.atom_case/0
+          └─ tmp\
       """
 
-      assert capture_eval(source) =~ expected
+      assert capture_io(:stderr, fn -> Code.eval_file(path) end) =~ expected
-    after
-      purge(Sample)
     end
 
-    test "long message (nofile)" do
+    @tag :tmp_dir
-      source = """
+    test "IO.warn file+line+column", %{tmp_dir: tmp_dir} do
-      defmodule Sample do
+      path = make_relative_tmp(tmp_dir, "io-warn-file-line-column.ex")
-        def atom_case do
-          v = "bc"
 
-          case v do
+      source = """
-            _ when is_atom(v) -> :ok
+      IO.warn("oops\\nmulti\\nline", file: __ENV__.file, line: __ENV__.line, column: 4)
-            _ -> :fail
-          end
-        end
-      end
       """
 
+      File.write!(path, source)
+
       expected = """
-      warning: incompatible types:
+          warning: oops
-
+          multi
-          binary() !~ atom()
+          line
-
+          │
-      in expression:
+        1 │ IO.warn("oops\\nmulti\\nline", file: __ENV__.file, line: __ENV__.line, column: 4)
-
+          │    ~
-          # nofile:6
+          │
-          is_atom(v)
+          └─ tmp\
-
-      where "v" was given the type binary() in:
-
-          # nofile:3
-          v = "bc"
-
-      where "v" was given the type atom() in:
-
-          # nofile:6
-          is_atom(v)
-
-      Conflict found at
-      └─ nofile:6:14: Sample.atom_case/0
-
       """
 
-      assert capture_eval(source) =~ expected
+      assert capture_io(:stderr, fn -> Code.eval_file(path) end) =~ expected
-    after
+    end
-      purge(Sample)
+
+    test "IO.warn with missing data" do
+      assert capture_eval("""
+             IO.warn("oops-bad", file: #{inspect(__ENV__.file)}, line: 3, column: nil)
+             """) =~ "warning: oops-bad"
+
+      assert capture_eval("""
+             IO.warn("oops-bad", file: #{inspect(__ENV__.file)}, line: nil)
+             """) =~ "oops-bad"
+
+      assert capture_eval("""
+             IO.warn("oops-bad", file: nil)
+             """) =~ "oops-bad"
     end
 
     @tag :tmp_dir
@@ -791,7 +1087,7 @@ defmodule Kernel.DiagnosticsTest do
 
       """
 
-      assert capture_eval(source, false) == expected
+      assert capture_eval(source, columns: false) == expected
     after
       purge(Sample)
     end
@@ -822,7 +1118,7 @@ defmodule Kernel.DiagnosticsTest do
 
       """
 
-      assert capture_compile(source, false) == expected
+      assert capture_compile(source, columns: false) == expected
     after
       purge(Sample)
     end
@@ -995,7 +1291,7 @@ defmodule Kernel.DiagnosticsTest do
 
       """
 
-      assert capture_eval(source, false) == expected
+      assert capture_eval(source, columns: false) == expected
     after
       purge(Sample)
     end
@@ -1148,30 +1444,32 @@ defmodule Kernel.DiagnosticsTest do
     |> Path.relative_to_cwd()
   end
 
-  defp capture_eval(source, columns? \\ true) do
+  defp capture_eval(source, opts \\ [columns: true]) do
     capture_io(:stderr, fn ->
-      quoted = Code.string_to_quoted!(source, columns: columns?)
+      quoted = Code.string_to_quoted!(source, opts)
       Code.eval_quoted(quoted)
     end)
   end
 
-  defp capture_compile(source, columns? \\ true) do
+  defp capture_compile(source, opts \\ [columns: true]) do
     capture_io(:stderr, fn ->
       assert_raise CompileError, fn ->
-        ast = Code.string_to_quoted!(source, columns: columns?)
+        ast = Code.string_to_quoted!(source, opts)
         Code.eval_quoted(ast)
       end
     end)
   end
 
-  defp capture_raise(source, exception, mock_stacktrace \\ []) do
+  defp capture_raise(source, exception, opts \\ []) do
+    {stacktrace, opts} = Keyword.pop(opts, :stacktrace, [])
+
     e =
       assert_raise exception, fn ->
-        ast = Code.string_to_quoted!(source, columns: true)
+        ast = Code.string_to_quoted!(source, [columns: true] ++ opts)
         Code.eval_quoted(ast)
       end
 
-    Exception.format(:error, e, mock_stacktrace)
+    Exception.format(:error, e, stacktrace)
   end
 
   defp purge(module) when is_atom(module) do

lib/elixir/test/elixir/kernel/errors_test.exs

@@ -473,7 +473,7 @@ defmodule Kernel.ErrorsTest do
       assert_eval_raise TokenMissingError,
                         [
                           "nofile:1:5:",
-                          ~r/missing terminator: end \(for "fn" starting at line 1\)/
+                          ~r/missing terminator: end/
                         ],
                         ~c"fn 1"
 
@@ -490,16 +490,16 @@ defmodule Kernel.ErrorsTest do
     assert_eval_raise TokenMissingError,
                       [
                         "nofile:1:25:",
-                        "missing terminator: end (for \"do\" starting at line 1)",
+                        "missing terminator: end",
                         "defmodule ShowSnippet do\n",
-                        "^"
+                        "└ unclosed delimiter"
                       ],
                       ~c"defmodule ShowSnippet do"
   end
 
   test "don't show snippet when error line is empty" do
     assert_eval_raise TokenMissingError,
-                      ["nofile:3:1:", "missing terminator: end (for \"do\" starting at line 1)"],
+                      ["nofile:1:25:", "missing terminator: end"],
                       ~c"defmodule ShowSnippet do\n\n"
   end
 

lib/elixir/test/elixir/kernel/parallel_compiler_test.exs

@@ -147,7 +147,7 @@ defmodule Kernel.ParallelCompilerTest do
       expected_msg = "Undef.__struct__/1 is undefined, cannot expand struct Undef"
 
       assert capture_io(:stderr, fn ->
-               assert {:error, [{^fixture, 3, msg}, {^fixture, 0, compile_msg}], []} =
+               assert {:error, [{^fixture, {3, 5}, msg}, {^fixture, 0, compile_msg}], []} =
                         Kernel.ParallelCompiler.compile([fixture])
 
                assert msg =~ expected_msg
@@ -196,6 +196,28 @@ defmodule Kernel.ParallelCompilerTest do
       purge([QuickExample])
     end
 
+    test "does not crash on external reports" do
+      [fixture] =
+        write_tmp(
+          "compile_quoted",
+          quick_example: """
+          defmodule CompileQuoted do
+            try do
+              Code.compile_quoted({:fn, [], [{:->, [], [[], quote(do: unknown_var)]}]})
+            rescue
+              _ -> :ok
+            end
+          end
+          """
+        )
+
+      assert capture_io(:stderr, fn ->
+               assert {:ok, [CompileQuoted], []} = Kernel.ParallelCompiler.compile([fixture])
+             end) =~ "undefined variable \"unknown_var\""
+    after
+      purge([CompileQuoted])
+    end
+
     test "does not hang on missing dependencies" do
       [fixture] =
         write_tmp(
@@ -216,7 +238,7 @@ defmodule Kernel.ParallelCompilerTest do
         "ThisModuleWillNeverBeAvailable.__struct__/1 is undefined, cannot expand struct ThisModuleWillNeverBeAvailable"
 
       assert capture_io(:stderr, fn ->
-               assert {:error, [{^fixture, 7, msg}, {^fixture, 0, compile_msg}], []} =
+               assert {:error, [{^fixture, {7, 3}, msg}, {^fixture, 0, compile_msg}], []} =
                         Kernel.ParallelCompiler.compile([fixture])
 
                assert msg =~ expected_msg
@@ -245,7 +267,9 @@ defmodule Kernel.ParallelCompilerTest do
         "ThisModuleWillNeverBeAvailable.__struct__/1 is undefined, cannot expand struct ThisModuleWillNeverBeAvailable"
 
       assert capture_io(:stderr, fn ->
-               assert {:error, [{^missing_struct, 2, msg}, {^missing_struct, 0, compile_msg}], []} =
+               assert {:error,
+                       [{^missing_struct, {2, 3}, msg}, {^missing_struct, 0, compile_msg}],
+                       []} =
                         Kernel.ParallelCompiler.compile([missing_struct, depends_on])
 
                assert msg =~ expected_msg
@@ -272,7 +296,9 @@ defmodule Kernel.ParallelCompilerTest do
       expected_msg = "module Unknown.Module is not loaded and could not be found"
 
       assert capture_io(:stderr, fn ->
-               assert {:error, [{^missing_import, 2, msg}, {^missing_import, 0, compile_msg}], []} =
+               assert {:error,
+                       [{^missing_import, {2, 3}, msg}, {^missing_import, 0, compile_msg}],
+                       []} =
                         Kernel.ParallelCompiler.compile([missing_import, depends_on])
 
                assert msg =~ expected_msg
@@ -301,7 +327,7 @@ defmodule Kernel.ParallelCompilerTest do
       msg =
         capture_io(:stderr, fn ->
           fixtures = [foo, bar]
-          assert {:error, [foo_error, bar_error], []} = Kernel.ParallelCompiler.compile(fixtures)
+          assert {:error, [bar_error, foo_error], []} = Kernel.ParallelCompiler.compile(fixtures)
           assert bar_error == {bar, nil, "deadlocked waiting on module FooDeadlock"}
           assert foo_error == {foo, nil, "deadlocked waiting on module BarDeadlock"}
         end)
@@ -389,7 +415,7 @@ defmodule Kernel.ParallelCompilerTest do
 
       capture_io(:stderr, fn ->
         fixtures = [foo, bar]
-        assert {:error, [foo_error, bar_error], []} = Kernel.ParallelCompiler.compile(fixtures)
+        assert {:error, [bar_error, foo_error], []} = Kernel.ParallelCompiler.compile(fixtures)
         assert {^bar, nil, "deadlocked waiting on module FooAsyncDeadlock"} = bar_error
         assert {^foo, nil, "deadlocked waiting on pmap [#PID<" <> _} = foo_error
       end)
@@ -476,6 +502,24 @@ defmodule Kernel.ParallelCompilerTest do
       end)
     end
 
+    test "gets correct file+line+column number for SyntaxError" do
+      File.mkdir_p!(tmp_path())
+
+      [fixture] =
+        write_tmp("error",
+          error: """
+          raise SyntaxError, file: "foo/bar.ex", line: 3, column: 10
+          """
+        )
+
+      file = Path.absname("foo/bar.ex")
+
+      capture_io(:stderr, fn ->
+        assert {:error, [%{file: ^file, source: ^fixture, position: {3, 10}}], _} =
+                 Kernel.ParallelCompiler.compile([fixture], return_diagnostics: true)
+      end)
+    end
+
     test "gets proper beam destinations from dynamic modules" do
       fixtures =
         write_tmp(
@@ -509,7 +553,7 @@ defmodule Kernel.ParallelCompilerTest do
       expected_msg = "Undef.__struct__/1 is undefined, cannot expand struct Undef"
 
       assert capture_io(:stderr, fn ->
-               assert {:error, [{^fixture, 3, msg}, {^fixture, 0, compile_msg}], []} =
+               assert {:error, [{^fixture, {3, 5}, msg}, {^fixture, 0, compile_msg}], []} =
                         Kernel.ParallelCompiler.require([fixture])
 
                assert msg =~ expected_msg
@@ -537,7 +581,7 @@ defmodule Kernel.ParallelCompilerTest do
         "ThisModuleWillNeverBeAvailable.__struct__/1 is undefined, cannot expand struct ThisModuleWillNeverBeAvailable"
 
       assert capture_io(:stderr, fn ->
-               assert {:error, [{^fixture, 7, msg}, {^fixture, 0, compile_msg}], []} =
+               assert {:error, [{^fixture, {7, 3}, msg}, {^fixture, 0, compile_msg}], []} =
                         Kernel.ParallelCompiler.require([fixture])
 
                assert msg =~ expected_msg

lib/elixir/test/elixir/kernel/parser_test.exs

@@ -491,83 +491,6 @@ defmodule Kernel.ParserTest do
     end
   end
 
-  describe "token missing errors" do
-    test "missing paren" do
-      assert_token_missing(
-        ["nofile:1:9:", "missing terminator: ) (for \"(\" starting at line 1)"],
-        ~c"case 1 ("
-      )
-    end
-
-    test "dot terminator" do
-      assert_token_missing(
-        ["nofile:1:9:", "missing terminator: \" (for function name starting at line 1)"],
-        ~c"foo.\"bar"
-      )
-    end
-
-    test "sigil terminator" do
-      assert_token_missing(
-        ["nofile:3:1:", "missing terminator: \" (for sigil ~r\" starting at line 1)"],
-        ~c"~r\"foo\n\n"
-      )
-
-      assert_token_missing(
-        ["nofile:3:1:", "missing terminator: } (for sigil ~r{ starting at line 1)"],
-        ~c"~r{foo\n\n"
-      )
-    end
-
-    test "string terminator" do
-      assert_token_missing(
-        ["nofile:1:5:", "missing terminator: \" (for string starting at line 1)"],
-        ~c"\"bar"
-      )
-    end
-
-    test "heredoc with incomplete interpolation" do
-      assert_token_missing(
-        [
-          "nofile:2:1:",
-          ~s/missing interpolation terminator: "}" (for heredoc starting at line 1)/
-        ],
-        ~c"\"\"\"\n\#{\n"
-      )
-    end
-
-    test "heredoc terminator" do
-      assert_token_missing(
-        ["nofile:2:4:", ~s/missing terminator: """ (for heredoc starting at line 1)/],
-        ~c"\"\"\"\nbar"
-      )
-
-      assert_token_missing(
-        ["nofile:2:7:", ~s/missing terminator: """ (for heredoc starting at line 1)/],
-        ~c"\"\"\"\nbar\"\"\""
-      )
-    end
-
-    test "missing end" do
-      assert_token_missing(
-        ["nofile:1:9:", "missing terminator: end \(for \"do\" starting at line 1\)"],
-        ~c"foo do 1"
-      )
-
-      assert_token_missing(
-        ["hint:", ~s/it looks like the "do" on line 2 does not have a matching "end"/],
-        ~c"""
-        defmodule MyApp do
-          def one do
-          # end
-
-          def two do
-          end
-        end
-        """
-      )
-    end
-  end
-
   describe "syntax errors" do
     test "invalid heredoc start" do
       assert_syntax_error(
@@ -1136,18 +1059,18 @@ defmodule Kernel.ParserTest do
 
       assert_syntax_error(
         ["atom length must be less than system limit: "],
-        ~s[:"#{atom}"]
+        ~s{:"#{atom}"}
+      )
+
+      assert_syntax_error(
+        ["atom length must be less than system limit: "],
+        ~s{["#{atom}": 123]}
       )
     end
   end
 
   defp parse!(string), do: Code.string_to_quoted!(string)
 
-  defp assert_token_missing(given_messages, string) do
-    e = assert_raise TokenMissingError, fn -> parse!(string) end
-    assert_exception_msg(e, given_messages)
-  end
-
   defp assert_syntax_error(given_messages, source) do
     e = assert_raise SyntaxError, fn -> parse!(source) end
     assert_exception_msg(e, given_messages)