Release v1.16.0-rc.1

Closes #13162.
Closes #13164.

.github/workflows/ci.yml

@@ -3,10 +3,10 @@ name: CI
 on:
   push:
     paths-ignore:
-      - 'lib/**/*.md'
+      - "lib/**/*.md"
   pull_request:
     paths-ignore:
-      - 'lib/**/*.md'
+      - "lib/**/*.md"
 
 env:
   ELIXIR_ASSERT_TIMEOUT: 2000
@@ -24,12 +24,12 @@ jobs:
       fail-fast: false
       matrix:
         include:
-          - otp_version: '26.0'
+          - otp_version: "26.0"
             otp_latest: true
-          - otp_version: '25.3'
+          - otp_version: "25.3"
-          - otp_version: '25.0'
+          - otp_version: "25.0"
-          - otp_version: '24.3'
+          - otp_version: "24.3"
-          - otp_version: '24.0'
+          - otp_version: "24.0"
           - otp_version: master
             development: true
           - otp_version: maint
@@ -77,7 +77,7 @@ jobs:
     name: Windows Server 2019, Erlang/OTP ${{ matrix.otp_version }}
     strategy:
       matrix:
-        otp_version: ['24', '25', '26']
+        otp_version: ["24", "25", "26.0"]
     runs-on: windows-2019
     steps:
       - name: Configure Git

CHANGELOG.md

@@ -57,13 +57,56 @@ A huge thank you to Vinícius Muller for working on the new diagnostics.
 
 Elixir's Getting Started guided has been made part of the Elixir repository and incorporated into ExDoc. This was an opportunity to revisit and unify all official guides and references.
 
-We have also incorporated and extended the work on [Understanding Code Smells in Elixir Functional Language](https://github.com/lucasvegi/Elixir-Code-Smells/blob/main/etc/2023-emse-code-smells-elixir.pdf), by Lucas Vegi and Marco Tulio Valente, from [ASERG/DCC/UFMG](http://aserg.labsoft.dcc.ufmg.br/), into the official document in the form of anti-patterns. The anti-patterns are divided into four categories: code-related, design-related, process-related, and meta-programming. Our goal is to give all developers with both positive and negative examples of Elixir code, with context and examples on how to improve their codebases.
+We have also incorporated and extended the work on [Understanding Code Smells in Elixir Functional Language](https://github.com/lucasvegi/Elixir-Code-Smells/blob/main/etc/2023-emse-code-smells-elixir.pdf), by Lucas Vegi and Marco Tulio Valente, from [ASERG/DCC/UFMG](http://aserg.labsoft.dcc.ufmg.br/), into the official document in the form of anti-patterns. The anti-patterns are divided into four categories: code-related, design-related, process-related, and meta-programming. Our goal is to give all developers examples of potential anti-patterns, with context and examples on how to improve their codebases.
 
 Another [ExDoc](https://github.com/elixir-lang/ex_doc) feature we have incorporated in this release is the addition of cheatsheets, starting with [a cheatsheet for the Enum module](https://hexdocs.pm/elixir/main/enum-cheat.html). If you would like to contribute future cheatsheets to Elixir itself, feel free to start a discussion with an issue.
 
 Finally, we have started enriching our documentation with [Mermaid.js](https://mermaid.js.org/) diagrams. You can find examples in the [GenServer](https://hexdocs.pm/elixir/main/GenServer.html) and [Supervisor](https://hexdocs.pm/elixir/main/Supervisor.html) docs.
 
-## v1.16.0-dev
+## v1.16.0-rc.1 (2023-12-12)
+
+### 1. Enhancements
+
+#### Elixir
+
+  * [Code] Add `:emit_warnings` for `Code.string_to_quoted/2`
+  * [File] Add `:offset` option to `File.stream!/2`
+  * [Kernel] Auto infer size of matched variable in bitstrings
+  * [Kernel] Preserve column information when translating typespecs
+  * [String] Add `String.replace_invalid/2`
+
+#### Logger
+
+  * [Logger] Add `Logger.levels/0`
+
+#### Mix
+
+  * [mix archive.install] Support `--sparse` option
+  * [mix compile.app] Warn if both `:applications` and `:extra_applications` are used
+  * [mix compile.elixir] Pass original exception down to diagnostic `:details` when possible
+  * [mix deps.clean] Emit a warning instead of crashing when a dependency cannot be removed
+  * [mix escript.install] Support `--sparse` option
+  * [mix release] Include `include/` directory in releases
+
+### 2. Bug fixes
+
+#### Elixir
+
+  * [Code] Keep quotes for atom keys in formatter
+  * [Macro] Address exception on `Macro.to_string/1` for certain ASTs
+  * [Module] Make sure file and position information is included in several module warnings (regression)
+  * [Path] Lazily evaluate `File.cwd!/0` in `Path.expand/1` and `Path.absname/1`
+
+#### IEx
+
+  * [IEx.Pry] Fix prying functions with only literals in their body
+
+#### Mix
+
+  * [mix archive.install] Restore code paths after archive.install
+  * [mix escript.install] Restore code paths after archive.install
+
+## v1.16.0-rc.0 (2023-10-31)
 
 ### 1. Enhancements
 
@@ -76,6 +119,7 @@ Finally, we have started enriching our documentation with [Mermaid.js](https://m
   * [Code] Automatically include columns in parsing options
   * [Code] Introduce `MismatchedDelimiterError` for handling mismatched delimiter exceptions
   * [Code.Fragment] Handle anonymous calls in fragments
+  * [Code.Formatter] Trim trailing whitespace on heredocs with `\r\n`
   * [Kernel] Suggest module names based on suffix and casing errors when the module does not exist in `UndefinedFunctionError`
   * [Kernel.ParallelCompiler] Introduce `Kernel.ParallelCompiler.pmap/2` to compile multiple additional entries in parallel
   * [Kernel.SpecialForms] Warn if `True`/`False`/`Nil` are used as aliases and there is no such alias

Makefile

@@ -2,7 +2,7 @@ PREFIX ?= /usr/local
 TEST_FILES ?= "*_test.exs"
 SHARE_PREFIX ?= $(PREFIX)/share
 MAN_PREFIX ?= $(SHARE_PREFIX)/man
-CANONICAL := main/
+# CANONICAL := main/
 ELIXIRC := bin/elixirc --ignore-module-conflict $(ELIXIRC_OPTS)
 ERLC := erlc -I lib/elixir/include
 ERL_MAKE := if [ -n "$(ERLC_OPTS)" ]; then ERL_COMPILER_OPTIONS=$(ERLC_OPTS) erl -make; else erl -make; fi

SECURITY.md

@@ -6,12 +6,11 @@ Elixir applies bug fixes only to the latest minor branch. Security patches are a
 
 Elixir version | Support
 :------------- | :-----------------------------
-1.16           | Development
+1.16           | Bug fixes and security patches
-1.15           | Bug fixes and security patches
+1.15           | Security patches only
 1.14           | Security patches only
 1.13           | Security patches only
 1.12           | Security patches only
-1.11           | Security patches only
 
 ## Announcements
 

VERSION

@@ -1 +1 @@
-1.16.0-dev
+1.16.0-rc.1

bin/elixir

@@ -1,7 +1,7 @@
 #!/bin/sh
 set -e
 
-ELIXIR_VERSION=1.16.0-dev
+ELIXIR_VERSION=1.16.0-rc.1
 
 if [ $# -eq 0 ] || { [ $# -eq 1 ] && { [ "$1" = "--help" ] || [ "$1" = "-h" ]; }; }; then
   cat <<USAGE >&2

bin/elixir.bat

@@ -1,6 +1,6 @@
 @if defined ELIXIR_CLI_ECHO (@echo on) else (@echo off)
 
-set ELIXIR_VERSION=1.16.0-dev
+set ELIXIR_VERSION=1.16.0-rc.1
 
 setlocal enabledelayedexpansion
 if    ""%1""==""""                if ""%2""=="""" goto documentation

lib/elixir/lib/code.ex

@@ -204,6 +204,7 @@ defmodule Code do
           required(:position) => position,
           required(:stacktrace) => Exception.stacktrace(),
           required(:span) => {non_neg_integer, non_neg_integer} | nil,
+          optional(:exception) => Exception.t() | nil,
           optional(any()) => any()
         }
 
@@ -962,10 +963,9 @@ defmodule Code do
     to_quoted_opts =
       [
         unescape: false,
-        warn_on_unnecessary_quotes: false,
         literal_encoder: &{:ok, {:__block__, &2, [&1]}},
         token_metadata: true,
-        warnings: false
+        emit_warnings: false
       ] ++ opts
 
     {forms, comments} = string_to_quoted_with_comments!(string, to_quoted_opts)
@@ -1126,9 +1126,8 @@ defmodule Code do
       atoms but `:existing_atoms_only` is still used for dynamic atoms,
       such as atoms with interpolations.
 
-    * `:warn_on_unnecessary_quotes` - when `false`, does not warn
+    * `:emit_warnings` (since v1.16.0) - when `false`, does not emit
-      when atoms, keywords or calls have unnecessary quotes on
+      tokenizing/parsing related warnings. Defaults to `true`.
-      them. Defaults to `true`.
 
   ## `Macro.to_string/2`
 
@@ -2057,7 +2056,8 @@ defmodule Code do
 
   defp get_beam_and_path(module) do
     with {^module, beam, filename} <- :code.get_object_code(module),
-         {:ok, ^module} <- beam |> :beam_lib.info() |> Keyword.fetch(:module) do
+         info_pairs when is_list(info_pairs) <- :beam_lib.info(beam),
+         {:ok, ^module} <- Keyword.fetch(info_pairs, :module) do
       {beam, filename}
     else
       _ -> :error

lib/elixir/lib/code/formatter.ex

@@ -520,14 +520,15 @@ defmodule Code.Formatter do
               {string(~S{"..//":}), state}
 
             {:__block__, _, [atom]} when is_atom(atom) ->
-              key =
+              iodata =
                 if Macro.classify_atom(atom) in [:identifier, :unquoted] do
-                  IO.iodata_to_binary([Atom.to_string(atom), ?:])
+                  [Atom.to_string(atom), ?:]
                 else
-                  IO.iodata_to_binary([?", Atom.to_string(atom), ?", ?:])
+                  [?", atom |> Atom.to_string() |> String.replace("\"", "\\\""), ?", ?:]
                 end
 
-              {string(key) |> color(:atom, state.inspect_opts), state}
+              {iodata |> IO.iodata_to_binary() |> string() |> color(:atom, state.inspect_opts),
+               state}
 
             {{:., _, [:erlang, :binary_to_atom]}, _, [{:<<>>, _, entries}, :utf8]} ->
               interpolation_to_algebra(entries, @double_quote, state, "\"", "\":")

lib/elixir/lib/code/fragment.ex

@@ -636,7 +636,7 @@ defmodule Code.Fragment do
     {reversed_pre, post} = adjust_position(reversed_pre, post)
 
     case take_identifier(post, []) do
-      :none ->
+      {_, [], _} ->
         maybe_operator(reversed_pre, post, line, opts)
 
       {:identifier, reversed_post, rest} ->
@@ -644,7 +644,7 @@ defmodule Code.Fragment do
         reversed = reversed_post ++ reversed_pre
 
         case codepoint_cursor_context(reversed, opts) do
-          {{:struct, acc}, offset} when acc != [] ->
+          {{:struct, acc}, offset} ->
             build_surround({:struct, acc}, reversed, line, offset)
 
           {{:alias, acc}, offset} ->
@@ -749,27 +749,11 @@ defmodule Code.Fragment do
     do: take_identifier(t, [h | acc])
 
   defp take_identifier(rest, acc) do
-    {stripped, _} = strip_spaces(rest, 0)
+    with {[?. | t], _} <- strip_spaces(rest, 0),
-
-    with [?. | t] <- stripped,
          {[h | _], _} when h in ?A..?Z <- strip_spaces(t, 0) do
       take_alias(rest, acc)
     else
-      # Consider it an identifier if we are at the end of line
+      _ -> {:identifier, acc, rest}
-      # or if we have spaces not followed by . (call) or / (arity)
-      _ when acc == [] and (rest == [] or (hd(rest) in @space and hd(stripped) not in ~c"/.")) ->
-        {:identifier, acc, rest}
-
-      # If we are immediately followed by a container, we are still part of the identifier.
-      # We don't consider << as it _may_ be an operator.
-      _ when acc == [] and hd(stripped) in ~c"({[" ->
-        {:identifier, acc, rest}
-
-      _ when acc == [] ->
-        :none
-
-      _ ->
-        {:identifier, acc, rest}
     end
   end
 
@@ -1107,6 +1091,6 @@ defmodule Code.Fragment do
     opts =
       Keyword.take(opts, [:file, :line, :column, :columns, :token_metadata, :literal_encoder])
 
-    Code.string_to_quoted(fragment, [cursor_completion: true, warnings: false] ++ opts)
+    Code.string_to_quoted(fragment, [cursor_completion: true, emit_warnings: false] ++ opts)
   end
 end

lib/elixir/lib/code/normalizer.ex

@@ -349,13 +349,15 @@ defmodule Code.Normalizer do
         meta
       end
 
+    last = List.last(args)
+
     cond do
-      Keyword.has_key?(meta, :do) or match?([{{:__block__, _, [:do]}, _} | _], List.last(args)) ->
+      Keyword.has_key?(meta, :do) or match?([{{:__block__, _, [:do]}, _} | _], last) ->
         # def foo do :ok end
         # def foo, do: :ok
         normalize_kw_blocks(form, meta, args, state)
 
-      match?([{:do, _} | _], List.last(args)) ->
+      match?([{:do, _} | _], last) and Keyword.keyword?(last) ->
         # Non normalized kw blocks
         line = state.parent_meta[:line]
         meta = meta ++ [do: [line: line], end: [line: line]]

lib/elixir/lib/code/typespec.ex

@@ -175,7 +175,8 @@ defmodule Code.Typespec do
 
   defp get_module_and_beam(module) when is_atom(module) do
     with {^module, beam, _filename} <- :code.get_object_code(module),
-         {:ok, ^module} <- beam |> :beam_lib.info() |> Keyword.fetch(:module) do
+         info_pairs when is_list(info_pairs) <- :beam_lib.info(beam),
+         {:ok, ^module} <- Keyword.fetch(info_pairs, :module) do
       {module, beam}
     else
       _ -> :error
@@ -419,5 +420,13 @@ defmodule Code.Typespec do
     :error
   end
 
-  defp meta(anno), do: [line: :erl_anno.line(anno)]
+  defp meta(anno) do
+    case :erl_anno.location(anno) do
+      {line, column} ->
+        [line: line, column: column]
+
+      line when is_integer(line) ->
+        [line: line]
+    end
+  end
 end

lib/elixir/lib/config.ex

@@ -46,9 +46,9 @@ defmodule Config do
   ## Migrating from `use Mix.Config`
 
   The `Config` module in Elixir was introduced in v1.9 as a replacement to
-  `Mix.Config`, which was specific to Mix and has been deprecated.
+  `use Mix.Config`, which was specific to Mix and has been deprecated.
 
-  You can leverage `Config` instead of `Mix.Config` in three steps. The first
+  You can leverage `Config` instead of `use Mix.Config` in three steps. The first
   step is to replace `use Mix.Config` at the top of your config files by
   `import Config`.
 

lib/elixir/lib/exception.ex

@@ -793,6 +793,212 @@ defmodule Exception do
       col -> format_file_line_column(Keyword.get(opts, :file), Keyword.get(opts, :line), col, " ")
     end
   end
+
+  @doc false
+  def format_snippet(
+        {start_line, _start_column} = start_pos,
+        {end_line, end_column} = end_pos,
+        line_offset,
+        description,
+        file,
+        lines,
+        start_message,
+        end_message
+      )
+      when start_line < end_line do
+    max_digits = digits(end_line)
+    general_padding = max(2, max_digits) + 1
+    padding = n_spaces(general_padding)
+
+    relevant_lines =
+      if end_line - start_line < 5 do
+        line_range(
+          lines,
+          start_pos,
+          end_pos,
+          line_offset,
+          padding,
+          max_digits,
+          start_message,
+          end_message
+        )
+      else
+        trimmed_inbetween_lines(
+          lines,
+          start_pos,
+          end_pos,
+          line_offset,
+          padding,
+          max_digits,
+          start_message,
+          end_message
+        )
+      end
+
+    """
+     #{padding}#{red("error:")} #{pad_message(description, padding)}
+     #{padding}│
+    #{relevant_lines}
+     #{padding}│
+     #{padding}└─ #{Path.relative_to_cwd(file)}:#{end_line}:#{end_column}\
+    """
+  end
+
+  def format_snippet(
+        {start_line, start_column},
+        {end_line, end_column},
+        line_offset,
+        description,
+        file,
+        lines,
+        start_message,
+        end_message
+      )
+      when start_line == end_line do
+    max_digits = digits(end_line)
+    general_padding = max(2, max_digits) + 1
+    padding = n_spaces(general_padding)
+
+    line = Enum.fetch!(lines, end_line - 1 - line_offset)
+    formatted_line = [line_padding(end_line, max_digits), to_string(end_line), " │ ", line]
+
+    mismatched_closing_line =
+      [
+        n_spaces(start_column - 1),
+        red("│"),
+        format_end_message(end_column - start_column, end_message)
+      ]
+
+    unclosed_delimiter_line =
+      [padding, " │ ", format_start_message(start_column, start_message)]
+
+    below_line = [padding, " │ ", mismatched_closing_line, "\n", unclosed_delimiter_line]
+
+    """
+     #{padding}#{red("error:")} #{pad_message(description, padding)}
+     #{padding}│
+    #{formatted_line}
+    #{below_line}
+     #{padding}│
+     #{padding}└─ #{Path.relative_to_cwd(file)}:#{end_line}:#{end_column}\
+    """
+  end
+
+  defp line_padding(line_number, max_digits) do
+    line_digits = digits(line_number)
+
+    spacing =
+      if line_digits == 1 do
+        max(2, max_digits)
+      else
+        max_digits - line_digits + 1
+      end
+
+    n_spaces(spacing)
+  end
+
+  defp n_spaces(n), do: String.duplicate(" ", n)
+
+  defp digits(number, acc \\ 1)
+  defp digits(number, acc) when number < 10, do: acc
+  defp digits(number, acc), do: digits(div(number, 10), acc + 1)
+
+  defp trimmed_inbetween_lines(
+         lines,
+         {start_line, start_column},
+         {end_line, end_column},
+         line_offset,
+         padding,
+         max_digits,
+         start_message,
+         end_message
+       ) do
+    start_padding = line_padding(start_line, max_digits)
+    end_padding = line_padding(end_line, max_digits)
+    first_line = Enum.fetch!(lines, start_line - 1 - line_offset)
+    last_line = Enum.fetch!(lines, end_line - 1 - line_offset)
+
+    """
+    #{start_padding}#{start_line} │ #{first_line}
+     #{padding}│ #{format_start_message(start_column, start_message)}
+     ...
+    #{end_padding}#{end_line} │ #{last_line}
+     #{padding}│ #{format_end_message(end_column, end_message)}\
+    """
+  end
+
+  defp line_range(
+         lines,
+         {start_line, start_column},
+         {end_line, end_column},
+         line_offset,
+         padding,
+         max_digits,
+         start_message,
+         end_message
+       ) do
+    start_line = start_line - 1
+    end_line = end_line - 1
+
+    lines
+    |> Enum.slice((start_line - line_offset)..(end_line - line_offset))
+    |> Enum.zip_with(start_line..end_line, fn line, line_number ->
+      line_number = line_number + 1
+      start_line = start_line + 1
+      end_line = end_line + 1
+
+      line_padding = line_padding(line_number, max_digits)
+
+      cond do
+        line_number == start_line ->
+          [
+            line_padding,
+            to_string(line_number),
+            " │ ",
+            line,
+            "\n",
+            padding,
+            " │ ",
+            format_start_message(start_column, start_message)
+          ]
+
+        line_number == end_line ->
+          [
+            line_padding,
+            to_string(line_number),
+            " │ ",
+            line,
+            "\n",
+            padding,
+            " │ ",
+            format_end_message(end_column, end_message)
+          ]
+
+        true ->
+          [line_padding, to_string(line_number), " │ ", line]
+      end
+    end)
+    |> Enum.intersperse("\n")
+  end
+
+  defp format_end_message(end_column, message),
+    do: [
+      n_spaces(end_column - 1),
+      red(message)
+    ]
+
+  defp format_start_message(start_column, message),
+    do: [n_spaces(start_column - 1), red(message)]
+
+  defp pad_message(message, padding), do: String.replace(message, "\n", "\n #{padding}")
+
+  defp red(string) do
+    if IO.ANSI.enabled?() do
+      [IO.ANSI.red(), string, IO.ANSI.reset()]
+    else
+      string
+    end
+  end
 end
 
 # Some exceptions implement "message/1" instead of "exception/1" mostly
@@ -948,12 +1154,11 @@ defmodule MismatchedDelimiterError do
   - `fn a -> )`
   """
 
-  @max_lines_shown 5
-
   defexception [
     :file,
     :line,
     :column,
+    :line_offset,
     :end_line,
     :end_column,
     :opening_delimiter,
@@ -968,6 +1173,7 @@ defmodule MismatchedDelimiterError do
         column: start_column,
         end_line: end_line,
         end_column: end_column,
+        line_offset: line_offset,
         description: description,
         opening_delimiter: opening_delimiter,
         closing_delimiter: _closing_delimiter,
@@ -979,196 +1185,24 @@ defmodule MismatchedDelimiterError do
     lines = String.split(snippet, "\n")
     expected_delimiter = :elixir_tokenizer.terminator(opening_delimiter)
 
-    snippet = format_snippet(start_pos, end_pos, description, file, lines, expected_delimiter)
+    start_message = "└ unclosed delimiter"
+    end_message = ~s/└ mismatched closing delimiter (expected "#{expected_delimiter}")/
+
+    snippet =
+      Exception.format_snippet(
+        start_pos,
+        end_pos,
+        line_offset,
+        description,
+        file,
+        lines,
+        start_message,
+        end_message
+      )
+
     format_message(file, end_line, end_column, snippet)
   end
 
-  defp format_snippet(
-         {start_line, _start_column} = start_pos,
-         {end_line, end_column} = end_pos,
-         description,
-         file,
-         lines,
-         expected_delimiter
-       )
-       when start_line < end_line do
-    max_digits = digits(end_line)
-    general_padding = max(2, max_digits) + 1
-    padding = n_spaces(general_padding)
-
-    relevant_lines =
-      if end_line - start_line < @max_lines_shown do
-        line_range(lines, start_pos, end_pos, padding, max_digits, expected_delimiter)
-      else
-        trimmed_inbetween_lines(
-          lines,
-          start_pos,
-          end_pos,
-          padding,
-          max_digits,
-          expected_delimiter
-        )
-      end
-
-    """
-     #{padding}#{red("error:")} #{pad_message(description, padding)}
-     #{padding}│
-    #{relevant_lines}
-     #{padding}│
-     #{padding}└─ #{Path.relative_to_cwd(file)}:#{end_line}:#{end_column}\
-    """
-  end
-
-  defp format_snippet(
-         {start_line, start_column},
-         {end_line, end_column},
-         description,
-         file,
-         lines,
-         expected_delimiter
-       )
-       when start_line == end_line do
-    max_digits = digits(end_line)
-    general_padding = max(2, max_digits) + 1
-    padding = n_spaces(general_padding)
-
-    line = Enum.fetch!(lines, end_line - 1)
-    formatted_line = [line_padding(end_line, max_digits), to_string(end_line), " │ ", line]
-
-    mismatched_closing_line =
-      [
-        n_spaces(start_column - 1),
-        red("│"),
-        mismatched_closing_delimiter(end_column - start_column, expected_delimiter)
-      ]
-
-    unclosed_delimiter_line =
-      [padding, " │ ", unclosed_delimiter(start_column)]
-
-    below_line = [padding, " │ ", mismatched_closing_line, "\n", unclosed_delimiter_line]
-
-    """
-     #{padding}#{red("error:")} #{pad_message(description, padding)}
-     #{padding}│
-    #{formatted_line}
-    #{below_line}
-     #{padding}│
-     #{padding}└─ #{Path.relative_to_cwd(file)}:#{end_line}:#{end_column}\
-    """
-  end
-
-  defp line_padding(line_number, max_digits) do
-    line_digits = digits(line_number)
-
-    spacing =
-      if line_digits == 1 do
-        max(2, max_digits)
-      else
-        max_digits - line_digits + 1
-      end
-
-    n_spaces(spacing)
-  end
-
-  defp n_spaces(n), do: String.duplicate(" ", n)
-
-  defp digits(number, acc \\ 1)
-  defp digits(number, acc) when number < 10, do: acc
-  defp digits(number, acc), do: digits(div(number, 10), acc + 1)
-
-  defp trimmed_inbetween_lines(
-         lines,
-         {start_line, start_column},
-         {end_line, end_column},
-         padding,
-         max_digits,
-         expected_delimiter
-       ) do
-    start_padding = line_padding(start_line, max_digits)
-    end_padding = line_padding(end_line, max_digits)
-    first_line = Enum.fetch!(lines, start_line - 1)
-    last_line = Enum.fetch!(lines, end_line - 1)
-
-    """
-    #{start_padding}#{start_line} │ #{first_line}
-     #{padding}│ #{unclosed_delimiter(start_column)}
-     ...
-    #{end_padding}#{end_line} │ #{last_line}
-     #{padding}│ #{mismatched_closing_delimiter(end_column, expected_delimiter)}\
-    """
-  end
-
-  defp line_range(
-         lines,
-         {start_line, start_column},
-         {end_line, end_column},
-         padding,
-         max_digits,
-         expected_delimiter
-       ) do
-    start_line = start_line - 1
-    end_line = end_line - 1
-
-    lines
-    |> Enum.slice(start_line..end_line)
-    |> Enum.zip_with(start_line..end_line, fn line, line_number ->
-      line_number = line_number + 1
-      start_line = start_line + 1
-      end_line = end_line + 1
-
-      line_padding = line_padding(line_number, max_digits)
-
-      cond do
-        line_number == start_line ->
-          [
-            line_padding,
-            to_string(line_number),
-            " │ ",
-            line,
-            "\n",
-            padding,
-            " │ ",
-            unclosed_delimiter(start_column)
-          ]
-
-        line_number == end_line ->
-          [
-            line_padding,
-            to_string(line_number),
-            " │ ",
-            line,
-            "\n",
-            padding,
-            " │ ",
-            mismatched_closing_delimiter(end_column, expected_delimiter)
-          ]
-
-        true ->
-          [line_padding, to_string(line_number), " │ ", line]
-      end
-    end)
-    |> Enum.intersperse("\n")
-  end
-
-  defp mismatched_closing_delimiter(end_column, expected_closing_delimiter),
-    do: [
-      n_spaces(end_column - 1),
-      red(~s/└ mismatched closing delimiter (expected "#{expected_closing_delimiter}")/)
-    ]
-
-  defp unclosed_delimiter(start_column),
-    do: [n_spaces(start_column - 1), red("└ unclosed delimiter")]
-
-  defp pad_message(message, padding), do: String.replace(message, "\n", "\n #{padding}")
-
-  defp red(string) do
-    if IO.ANSI.enabled?() do
-      [IO.ANSI.red(), string, IO.ANSI.reset()]
-    else
-      string
-    end
-  end
-
   defp format_message(file, line, column, message) do
     location = Exception.format_file_line_column(Path.relative_to_cwd(file), line, column)
     "mismatched delimiter found on " <> location <> "\n" <> message
@@ -1241,8 +1275,10 @@ defmodule TokenMissingError do
   defexception [
     :file,
     :line,
-    :snippet,
     :column,
+    :end_line,
+    :line_offset,
+    :snippet,
     :opening_delimiter,
     description: "expression is incomplete"
   ]
@@ -1252,14 +1288,42 @@ defmodule TokenMissingError do
         file: file,
         line: line,
         column: column,
+        end_line: end_line,
+        line_offset: line_offset,
         description: description,
+        opening_delimiter: opening_delimiter,
         snippet: snippet
       })
-      when not is_nil(snippet) and not is_nil(column) do
+      when not is_nil(snippet) and not is_nil(column) and not is_nil(end_line) do
-    snippet =
+    {lines, total_trimmed_lines} = handle_trailing_newlines(snippet)
-      :elixir_errors.format_snippet({line, column}, file, description, snippet, :error, [], nil)
+    end_line = end_line - total_trimmed_lines
 
-    format_message(file, line, column, snippet)
+    end_column =
+      lines
+      |> Enum.fetch!(end_line - line_offset - 1)
+      |> String.length()
+      |> Kernel.+(1)
+
+    start_pos = {line, column}
+    end_pos = {end_line, end_column}
+    expected_delimiter = :elixir_tokenizer.terminator(opening_delimiter)
+
+    start_message = ~s/└ unclosed delimiter/
+    end_message = ~s/└ missing closing delimiter (expected "#{expected_delimiter}")/
+
+    snippet =
+      Exception.format_snippet(
+        start_pos,
+        end_pos,
+        line_offset,
+        description,
+        file,
+        lines,
+        start_message,
+        end_message
+      )
+
+    format_message(file, end_line, end_column, snippet)
   end
 
   @impl true
@@ -1267,13 +1331,20 @@ defmodule TokenMissingError do
         file: file,
         line: line,
         column: column,
+        snippet: snippet,
         description: description
       }) do
     snippet =
-      :elixir_errors.format_snippet({line, column}, file, description, nil, :error, [], nil)
+      :elixir_errors.format_snippet({line, column}, file, description, snippet, :error, [], nil)
 
-    padded = "   " <> String.replace(snippet, "\n", "\n   ")
+    format_message(file, line, column, snippet)
-    format_message(file, line, column, padded)
+  end
+
+  defp handle_trailing_newlines(snippet) do
+    trimmed_snippet = String.trim_trailing(snippet, "\n")
+    total_trimmed_newlines = String.length(snippet) - String.length(trimmed_snippet)
+    lines = String.split(trimmed_snippet, "\n")
+    {lines, total_trimmed_newlines}
   end
 
   defp format_message(file, line, column, message) do

lib/elixir/lib/file.ex

@@ -133,6 +133,7 @@ defmodule File do
 
   @type stream_mode ::
           encoding_mode()
+          | read_offset_mode()
           | :append
           | :compressed
           | :delayed_write
@@ -140,6 +141,8 @@ defmodule File do
           | {:read_ahead, pos_integer | false}
           | {:delayed_write, non_neg_integer, non_neg_integer}
 
+  @type read_offset_mode :: {:read_offset, non_neg_integer()}
+
   @type erlang_time ::
           {{year :: non_neg_integer(), month :: 1..12, day :: 1..31},
            {hour :: 0..23, minute :: 0..59, second :: 0..59}}
@@ -1748,13 +1751,15 @@ defmodule File do
   One may also consider passing the `:delayed_write` option if the stream
   is meant to be written to under a tight loop.
 
-  ## Byte order marks
+  ## Byte order marks and read offset
 
   If you pass `:trim_bom` in the modes parameter, the stream will
   trim UTF-8, UTF-16 and UTF-32 byte order marks when reading from file.
 
   Note that this function does not try to discover the file encoding
-  based on BOM.
+  based on BOM. From Elixir v1.16.0, you may also pass a `:read_offset`
+  that is skipped whenever enumerating the stream (if both `:read_offset`
+  and `:trim_bom` are given, the offset is skipped after the BOM).
 
   ## Examples
 

lib/elixir/lib/file/stream.ex

@@ -18,6 +18,12 @@ defmodule File.Stream do
 
   @doc false
   def __build__(path, line_or_bytes, modes) do
+    with {:read_offset, offset} <- :lists.keyfind(:read_offset, 1, modes),
+         false <- is_integer(offset) and offset >= 0 do
+      raise ArgumentError,
+            "expected :read_offset to be a non-negative integer, got: #{inspect(offset)}"
+    end
+
     raw = :lists.keyfind(:encoding, 1, modes) == false
 
     modes =
@@ -88,7 +94,7 @@ defmodule File.Stream do
       start_fun = fn ->
         case File.Stream.__open__(stream, read_modes(modes)) do
           {:ok, device} ->
-            if :trim_bom in modes, do: trim_bom(device, raw) |> elem(0), else: device
+            skip_bom_and_offset(device, raw, modes)
 
           {:error, reason} ->
             raise File.Error, reason: reason, action: "stream", path: stream.path
@@ -104,9 +110,14 @@ defmodule File.Stream do
       Stream.resource(start_fun, next_fun, &:file.close/1).(acc, fun)
     end
 
-    def count(%{modes: modes, line_or_bytes: :line, path: path} = stream) do
+    def count(%{modes: modes, line_or_bytes: :line, path: path, raw: raw} = stream) do
       pattern = :binary.compile_pattern("\n")
-      counter = &count_lines(&1, path, pattern, read_function(stream), 0)
+
+      counter = fn device ->
+        device = skip_bom_and_offset(device, raw, modes)
+        count_lines(device, path, pattern, read_function(stream), 0)
+      end
+
       {:ok, open!(stream, modes, counter)}
     end
 
@@ -116,8 +127,11 @@ defmodule File.Stream do
           {:error, __MODULE__}
 
         {:ok, %{size: size}} ->
+          bom_offset = count_raw_bom(stream, modes)
+          offset = get_read_offset(modes)
+          size = max(size - bom_offset - offset, 0)
           remainder = if rem(size, bytes) == 0, do: 0, else: 1
-          {:ok, div(size, bytes) + remainder - count_raw_bom(stream, modes)}
+          {:ok, div(size, bytes) + remainder}
 
         {:error, reason} ->
           raise File.Error, reason: reason, action: "stream", path: path
@@ -158,6 +172,23 @@ defmodule File.Stream do
       end
     end
 
+    defp skip_bom_and_offset(device, raw, modes) do
+      device =
+        if :trim_bom in modes do
+          device |> trim_bom(raw) |> elem(0)
+        else
+          device
+        end
+
+      offset = get_read_offset(modes)
+
+      if offset > 0 do
+        {:ok, _} = :file.position(device, {:cur, offset})
+      end
+
+      device
+    end
+
     defp trim_bom(device, true) do
       bom_length = device |> IO.binread(4) |> bom_length()
       {:ok, new_pos} = :file.position(device, bom_length)
@@ -183,6 +214,13 @@ defmodule File.Stream do
     defp bom_length(<<254, 255, 0, 0, _rest::binary>>), do: 4
     defp bom_length(_binary), do: 0
 
+    def get_read_offset(modes) do
+      case :lists.keyfind(:read_offset, 1, modes) do
+        {:read_offset, offset} -> offset
+        false -> 0
+      end
+    end
+
     defp read_modes(modes) do
       for mode <- modes, mode not in [:write, :append, :trim_bom], do: mode
     end

lib/elixir/lib/float.ex

@@ -198,7 +198,7 @@ defmodule Float do
   defp add_dot(acc, false), do: acc <> ".0"
 
   @doc """
-  Rounds a float to the largest number less than or equal to `num`.
+  Rounds a float to the largest float less than or equal to `number`.
 
   `floor/2` also accepts a precision to round a floating-point value down
   to an arbitrary number of fractional digits (between 0 and 15).
@@ -246,7 +246,7 @@ defmodule Float do
   end
 
   @doc """
-  Rounds a float to the smallest integer greater than or equal to `num`.
+  Rounds a float to the smallest float greater than or equal to `number`.
 
   `ceil/2` also accepts a precision to round a floating-point value down
   to an arbitrary number of fractional digits (between 0 and 15).

lib/elixir/lib/gen_server.ex

@@ -9,14 +9,10 @@ defmodule GenServer do
   tracing and error reporting. It will also fit into a supervision tree.
 
   ```mermaid
-  graph TD
+  graph BT
-      GenServer
+      C(Client #3) ~~~ B(Client #2) ~~~ A(Client #1)
-      GenServer -. reply -.-> A
+      A & B & C -->|request| GenServer
-      GenServer -. reply -.-> B
+      GenServer -.->|reply| A & B & C
-      GenServer -. reply -.-> C
-      A(Client #1) -- request --> GenServer
-      B(Client #2) -- request --> GenServer
-      C(Client #3) -- request --> GenServer
   ```
 
   ## Example
@@ -475,7 +471,7 @@ defmodule GenServer do
   guide provides a tutorial-like introduction. The documentation and links
   in Erlang can also provide extra insight.
 
-    * [GenServer - Elixir's Getting Started Guide](https://elixir-lang.org/getting-started/mix-otp/genserver.html)
+    * [GenServer - Elixir's Getting Started Guide](genservers.md)
     * [`:gen_server` module documentation](`:gen_server`)
     * [gen_server Behaviour - OTP Design Principles](https://www.erlang.org/doc/design_principles/gen_server_concepts.html)
     * [Clients and Servers - Learn You Some Erlang for Great Good!](http://learnyousomeerlang.com/clients-and-servers)

lib/elixir/lib/inspect/algebra.ex

@@ -143,9 +143,8 @@ defmodule Inspect.Opts do
   function as this must be controlled by applications. Libraries
   should instead define their own structs with custom inspect
   implementations. If a library must change the default inspect
-  function, then it is best to define to ask users of your library
+  function, then it is best to ask users of your library to explicitly
-  to explicitly call `default_inspect_fun/1` with your function of
+  call `default_inspect_fun/1` with your function of choice.
-  choice.
 
   The default is `Inspect.inspect/2`.
 

lib/elixir/lib/kernel.ex

@@ -2064,9 +2064,6 @@ defmodule Kernel do
       {var, _, nil} when is_atom(var) ->
         invalid_concat_left_argument_error(Atom.to_string(var))
 
-      {:^, _, [{var, _, nil}]} when is_atom(var) ->
-        invalid_concat_left_argument_error("^#{Atom.to_string(var)}")
-
       _ ->
         expanded_arg
     end
@@ -3625,7 +3622,7 @@ defmodule Kernel do
   defp do_at([], meta, name, function?, env) do
     IO.warn(
       "the @#{name}() notation (with parentheses) is deprecated, please use @#{name} (without parentheses) instead",
-      Macro.Env.stacktrace(env)
+      env
     )
 
     do_at(nil, meta, name, function?, env)

lib/elixir/lib/kernel/parallel_compiler.ex

@@ -900,10 +900,14 @@ defmodule Kernel.ParallelCompiler do
       message: message,
       severity: :error,
       stacktrace: stack,
-      span: nil
+      span: nil,
+      exception: get_exception(reason)
     }
   end
 
+  defp get_exception(exception) when is_exception(exception), do: exception
+  defp get_exception(_reason), do: nil
+
   defp get_line(_file, %{line: line, column: column}, _stack)
        when is_integer(line) and line > 0 and is_integer(column) and column >= 0 do
     {line, column}

lib/elixir/lib/kernel/typespec.ex

@@ -385,17 +385,17 @@ defmodule Kernel.Typespec do
       compile_error(caller, error)
     end
 
-    line = line(meta)
+    location = location(meta)
     vars = Keyword.keys(guard)
 
     {args, state} = :lists.mapfoldl(&typespec(&1, vars, caller, &2), state, args)
     {return, state} = typespec(return, vars, caller, state)
-    spec = {:type, line, :fun, [{:type, line, :product, args}, return]}
+    spec = {:type, location, :fun, [{:type, location, :product, args}, return]}
 
     {spec, state} =
       case guard_to_constraints(guard, vars, meta, caller, state) do
         {[], state} -> {spec, state}
-        {constraints, state} -> {{:type, line, :bounded_fun, [spec, constraints]}, state}
+        {constraints, state} -> {{:type, location, :bounded_fun, [spec, constraints]}, state}
       end
 
     ensure_no_unused_local_vars!(caller, state.local_vars)
@@ -437,7 +437,7 @@ defmodule Kernel.Typespec do
   defp ensure_not_default(_), do: :ok
 
   defp guard_to_constraints(guard, vars, meta, caller, state) do
-    line = line(meta)
+    location = location(meta)
 
     fun = fn
       {_name, {:var, _, context}}, {constraints, state} when is_atom(context) ->
@@ -445,9 +445,9 @@ defmodule Kernel.Typespec do
 
       {name, type}, {constraints, state} ->
         {spec, state} = typespec(type, vars, caller, state)
-        constraint = [{:atom, line, :is_subtype}, [{:var, line, name}, spec]]
+        constraint = [{:atom, location, :is_subtype}, [{:var, location, name}, spec]]
         state = update_local_vars(state, name)
-        {[{:type, line, :constraint, constraint} | constraints], state}
+        {[{:type, location, :constraint, constraint} | constraints], state}
     end
 
     {constraints, state} = :lists.foldl(fun, {[], state}, guard)
@@ -456,21 +456,27 @@ defmodule Kernel.Typespec do
 
   ## To typespec conversion
 
-  defp line(meta) do
+  defp location(meta) do
-    Keyword.get(meta, :line, 0)
+    line = Keyword.get(meta, :line, 0)
+
+    if column = Keyword.get(meta, :column) do
+      {line, column}
+    else
+      line
+    end
   end
 
   # Handle unions
   defp typespec({:|, meta, [_, _]} = exprs, vars, caller, state) do
     exprs = collect_union(exprs)
     {union, state} = :lists.mapfoldl(&typespec(&1, vars, caller, &2), state, exprs)
-    {{:type, line(meta), :union, union}, state}
+    {{:type, location(meta), :union, union}, state}
   end
 
   # Handle binaries
   defp typespec({:<<>>, meta, []}, _, _, state) do
-    line = line(meta)
+    location = location(meta)
-    {{:type, line, :binary, [{:integer, line, 0}, {:integer, line, 0}]}, state}
+    {{:type, location, :binary, [{:integer, location, 0}, {:integer, location, 0}]}, state}
   end
 
   defp typespec(
@@ -480,14 +486,18 @@ defmodule Kernel.Typespec do
          state
        )
        when is_atom(ctx1) and is_atom(ctx2) and unit in 1..256 do
-    line = line(meta)
+    location = location(meta)
-    {{:type, line, :binary, [{:integer, line, 0}, {:integer, line(unit_meta), unit}]}, state}
+
+    {{:type, location, :binary, [{:integer, location, 0}, {:integer, location(unit_meta), unit}]},
+     state}
   end
 
   defp typespec({:<<>>, meta, [{:"::", size_meta, [{:_, _, ctx}, size]}]}, _, _, state)
        when is_atom(ctx) and is_integer(size) and size >= 0 do
-    line = line(meta)
+    location = location(meta)
-    {{:type, line, :binary, [{:integer, line(size_meta), size}, {:integer, line, 0}]}, state}
+
+    {{:type, location, :binary, [{:integer, location(size_meta), size}, {:integer, location, 0}]},
+     state}
   end
 
   defp typespec(
@@ -505,8 +515,8 @@ defmodule Kernel.Typespec do
        )
        when is_atom(ctx1) and is_atom(ctx2) and is_atom(ctx3) and is_integer(size) and
               size >= 0 and unit in 1..256 do
-    args = [{:integer, line(size_meta), size}, {:integer, line(unit_meta), unit}]
+    args = [{:integer, location(size_meta), size}, {:integer, location(unit_meta), unit}]
-    {{:type, line(meta), :binary, args}, state}
+    {{:type, location(meta), :binary, args}, state}
   end
 
   defp typespec({:<<>>, _meta, _args}, _vars, caller, _state) do
@@ -519,7 +529,7 @@ defmodule Kernel.Typespec do
 
   ## Handle maps and structs
   defp typespec({:map, meta, args}, _vars, _caller, state) when args == [] or is_atom(args) do
-    {{:type, line(meta), :map, :any}, state}
+    {{:type, location(meta), :map, :any}, state}
   end
 
   defp typespec({:%{}, meta, fields} = map, vars, caller, state) do
@@ -527,17 +537,17 @@ defmodule Kernel.Typespec do
       {{:required, meta2, [k]}, v}, state ->
         {arg1, state} = typespec(k, vars, caller, state)
         {arg2, state} = typespec(v, vars, caller, state)
-        {{:type, line(meta2), :map_field_exact, [arg1, arg2]}, state}
+        {{:type, location(meta2), :map_field_exact, [arg1, arg2]}, state}
 
       {{:optional, meta2, [k]}, v}, state ->
         {arg1, state} = typespec(k, vars, caller, state)
         {arg2, state} = typespec(v, vars, caller, state)
-        {{:type, line(meta2), :map_field_assoc, [arg1, arg2]}, state}
+        {{:type, location(meta2), :map_field_assoc, [arg1, arg2]}, state}
 
       {k, v}, state ->
         {arg1, state} = typespec(k, vars, caller, state)
         {arg2, state} = typespec(v, vars, caller, state)
-        {{:type, line(meta), :map_field_exact, [arg1, arg2]}, state}
+        {{:type, location(meta), :map_field_exact, [arg1, arg2]}, state}
 
       {:|, _, [_, _]}, _state ->
         error =
@@ -551,7 +561,7 @@ defmodule Kernel.Typespec do
     end
 
     {fields, state} = :lists.mapfoldl(fun, state, fields)
-    {{:type, line(meta), :map, fields}, state}
+    {{:type, location(meta), :map, fields}, state}
   end
 
   defp typespec({:%, _, [name, {:%{}, meta, fields}]} = node, vars, caller, state) do
@@ -644,7 +654,7 @@ defmodule Kernel.Typespec do
     {right, state} = typespec(right, vars, caller, state)
     :ok = validate_range(left, right, caller)
 
-    {{:type, line(meta), :range, [left, right]}, state}
+    {{:type, location(meta), :range, [left, right]}, state}
   end
 
   # Handle special forms
@@ -668,7 +678,7 @@ defmodule Kernel.Typespec do
         pair -> pair
       end
 
-    {{:type, line(meta), :fun, fun_args}, state}
+    {{:type, location(meta), :fun, fun_args}, state}
   end
 
   # Handle type operator
@@ -691,10 +701,10 @@ defmodule Kernel.Typespec do
         # This may be generating an invalid typespec but we need to generate it
         # to avoid breaking existing code that was valid but only broke Dialyzer
         {right, state} = typespec(expr, vars, caller, state)
-        {{:ann_type, line(meta), [{:var, line(var_meta), var_name}, right]}, state}
+        {{:ann_type, location(meta), [{:var, location(var_meta), var_name}, right]}, state}
 
       {right, state} ->
-        {{:ann_type, line(meta), [{:var, line(var_meta), var_name}, right]}, state}
+        {{:ann_type, location(meta), [{:var, location(var_meta), var_name}, right]}, state}
     end
   end
 
@@ -723,13 +733,13 @@ defmodule Kernel.Typespec do
     {left, state} = typespec(left, vars, caller, state)
     state = %{state | undefined_type_error_enabled?: true}
     {right, state} = typespec(right, vars, caller, state)
-    {{:ann_type, line(meta), [left, right]}, state}
+    {{:ann_type, location(meta), [left, right]}, state}
   end
 
   # Handle unary ops
   defp typespec({op, meta, [integer]}, _, _, state) when op in [:+, :-] and is_integer(integer) do
-    line = line(meta)
+    location = location(meta)
-    {{:op, line, op, {:integer, line, integer}}, state}
+    {{:op, location, op, {:integer, location, integer}}, state}
   end
 
   # Handle remote calls in the form of @module_attribute.type.
@@ -778,12 +788,12 @@ defmodule Kernel.Typespec do
 
   # Handle tuples
   defp typespec({:tuple, meta, []}, _vars, _caller, state) do
-    {{:type, line(meta), :tuple, :any}, state}
+    {{:type, location(meta), :tuple, :any}, state}
   end
 
   defp typespec({:{}, meta, t}, vars, caller, state) when is_list(t) do
     {args, state} = :lists.mapfoldl(&typespec(&1, vars, caller, &2), state, t)
-    {{:type, line(meta), :tuple, args}, state}
+    {{:type, location(meta), :tuple, args}, state}
   end
 
   defp typespec({left, right}, vars, caller, state) do
@@ -799,7 +809,7 @@ defmodule Kernel.Typespec do
   defp typespec({name, meta, atom}, vars, caller, state) when is_atom(atom) do
     if :lists.member(name, vars) do
       state = update_local_vars(state, name)
-      {{:var, line(meta), name}, state}
+      {{:var, location(meta), name}, state}
     else
       typespec({name, meta, []}, vars, caller, state)
     end
@@ -814,7 +824,7 @@ defmodule Kernel.Typespec do
 
     IO.warn(warning, caller)
     {args, state} = :lists.mapfoldl(&typespec(&1, vars, caller, &2), state, args)
-    {{:type, line(meta), :string, args}, state}
+    {{:type, location(meta), :string, args}, state}
   end
 
   defp typespec({:nonempty_string, meta, args}, vars, caller, state) do
@@ -825,7 +835,7 @@ defmodule Kernel.Typespec do
 
     IO.warn(warning, caller)
     {args, state} = :lists.mapfoldl(&typespec(&1, vars, caller, &2), state, args)
-    {{:type, line(meta), :nonempty_string, args}, state}
+    {{:type, location(meta), :nonempty_string, args}, state}
   end
 
   defp typespec({type, _meta, []}, vars, caller, state) when type in [:charlist, :char_list] do
@@ -855,7 +865,7 @@ defmodule Kernel.Typespec do
 
   defp typespec({:fun, meta, args}, vars, caller, state) do
     {args, state} = :lists.mapfoldl(&typespec(&1, vars, caller, &2), state, args)
-    {{:type, line(meta), :fun, args}, state}
+    {{:type, location(meta), :fun, args}, state}
   end
 
   defp typespec({:..., _meta, _args}, _vars, caller, _state) do
@@ -872,7 +882,7 @@ defmodule Kernel.Typespec do
 
     case :erl_internal.is_type(name, arity) do
       true ->
-        {{:type, line(meta), name, args}, state}
+        {{:type, location(meta), name, args}, state}
 
       false ->
         if state.undefined_type_error_enabled? and
@@ -890,7 +900,7 @@ defmodule Kernel.Typespec do
             %{state | used_type_pairs: [{name, arity} | state.used_type_pairs]}
           end
 
-        {{:user_type, line(meta), name, args}, state}
+        {{:user_type, location(meta), name, args}, state}
     end
   end
 
@@ -963,7 +973,7 @@ defmodule Kernel.Typespec do
 
   defp remote_type({remote, meta, name, args}, vars, caller, state) do
     {args, state} = :lists.mapfoldl(&typespec(&1, vars, caller, &2), state, args)
-    {{:remote_type, line(meta), [remote, name, args]}, state}
+    {{:remote_type, location(meta), [remote, name, args]}, state}
   end
 
   defp collect_union({:|, _, [a, b]}), do: [a | collect_union(b)]
@@ -996,16 +1006,16 @@ defmodule Kernel.Typespec do
   end
 
   defp fn_args(meta, [{:..., _, _}], _vars, _caller, state) do
-    {{:type, line(meta), :any}, state}
+    {{:type, location(meta), :any}, state}
   end
 
   defp fn_args(meta, args, vars, caller, state) do
     {args, state} = :lists.mapfoldl(&typespec(&1, vars, caller, &2), state, args)
-    {{:type, line(meta), :product, args}, state}
+    {{:type, location(meta), :product, args}, state}
   end
 
   defp variable({name, meta, args}) when is_atom(name) and is_atom(args) do
-    {:var, line(meta), name}
+    {:var, location(meta), name}
   end
 
   defp variable(expr), do: expr

lib/elixir/lib/kernel/utils.ex

@@ -36,14 +36,14 @@ defmodule Kernel.Utils do
     if is_list(funs) do
       IO.warn(
         "passing a list to Kernel.defdelegate/2 is deprecated, please define each delegate separately",
-        Macro.Env.stacktrace(env)
+        env
       )
     end
 
     if Keyword.has_key?(opts, :append_first) do
       IO.warn(
         "Kernel.defdelegate/2 :append_first option is deprecated",
-        Macro.Env.stacktrace(env)
+        env
       )
     end
 

lib/elixir/lib/module.ex

@@ -326,8 +326,17 @@ defmodule Module do
   ### `@nifs` (since v1.16.0)
 
   A list of functions and their arities which will be overridden
-  by a native implementation (NIF). See the Erlang documentation
+  by a native implementation (NIF).
-  for more information: https://www.erlang.org/doc/man/erl_nif
+
+      defmodule MyLibrary.MyModule do
+        @nifs [foo: 1, bar: 2]
+
+        def foo(arg1), do: :erlang.nif_error(:not_loaded)
+        def bar(arg1, arg2), do: :erlang.nif_error(:not_loaded)
+      end
+
+  See the Erlang documentation for more information:
+  https://www.erlang.org/doc/man/erl_nif
 
   ### `@on_definition`
 
@@ -2124,8 +2133,7 @@ defmodule Module do
   end
 
   defp attribute_stack(module, line) do
-    file = String.to_charlist(Path.relative_to_cwd(:elixir_module.file(module)))
+    struct!(Macro.Env, module: module, file: :elixir_module.file(module), line: line)
-    [{module, :__MODULE__, 0, file: file, line: line}]
   end
 
   ## Helpers

lib/elixir/lib/module/parallel_checker.ex

@@ -334,7 +334,7 @@ defmodule Module.ParallelChecker do
 
   defp position_to_tuple(position) do
     case position[:column] do
-      nil -> position[:line]
+      nil -> position[:line] || 0
       col -> {position[:line], col}
     end
   end

lib/elixir/lib/path.ex

@@ -44,17 +44,17 @@ defmodule Path do
   """
   @spec absname(t) :: binary
   def absname(path) do
-    absname(path, File.cwd!())
+    absname(path, &File.cwd!/0)
   end
 
   @doc """
   Builds a path from `relative_to` to `path`.
 
   If `path` is already an absolute path, `relative_to` is ignored. See also
-  `relative_to/3`.
+  `relative_to/3`. `relative_to` is either a path or an anonymous function,
+  which is invoked only when necessary, that returns a path.
 
-  Unlike `expand/2`, no attempt is made to
+  Unlike `expand/2`, no attempt is made to resolve `..`, `.` or `~`.
-  resolve `..`, `.` or `~`.
 
   ## Examples
 
@@ -65,19 +65,33 @@ defmodule Path do
       "bar/../x"
 
   """
-  @spec absname(t, t) :: binary
+  @spec absname(t, t | (-> t)) :: binary
   def absname(path, relative_to) do
     path = IO.chardata_to_string(path)
 
     case type(path) do
       :relative ->
+        relative_to =
+          if is_function(relative_to, 0) do
+            relative_to.()
+          else
+            relative_to
+          end
+
         absname_join([relative_to, path])
 
       :absolute ->
         absname_join([path])
 
       :volumerelative ->
-        relative_to = IO.chardata_to_string(relative_to)
+        relative_to =
+          if is_function(relative_to, 0) do
+            relative_to.()
+          else
+            relative_to
+          end
+          |> IO.chardata_to_string()
+
         absname_vr(split(path), split(relative_to), relative_to)
     end
   end
@@ -163,7 +177,7 @@ defmodule Path do
   """
   @spec expand(t) :: binary
   def expand(path) do
-    expand_dot(absname(expand_home(path), File.cwd!()))
+    expand_dot(absname(expand_home(path), &File.cwd!/0))
   end
 
   @doc """
@@ -192,7 +206,7 @@ defmodule Path do
   """
   @spec expand(t, t) :: binary
   def expand(path, relative_to) do
-    expand_dot(absname(absname(expand_home(path), expand_home(relative_to)), File.cwd!()))
+    expand_dot(absname(absname(expand_home(path), expand_home(relative_to)), &File.cwd!/0))
   end
 
   @doc """
@@ -745,9 +759,9 @@ defmodule Path do
   You may call `Path.expand/1` to normalize the path before invoking
   this function.
 
-  A character preceded by \ loses its special meaning.
+  A character preceded by `\\` loses its special meaning.
-  Note that \ must be written as \\ in a string literal.
+  Note that `\\` must be written as `\\\\` in a string literal.
-  For example, "\\?*" will match any filename starting with ?.
+  For example, `"\\\\?*"` will match any filename starting with `?.`.
 
   By default, the patterns `*` and `?` do not match files starting
   with a dot `.`. See the `:match_dot` option in the "Options" section

lib/elixir/lib/process.ex

@@ -504,7 +504,7 @@ defmodule Process do
   If the process is already dead when calling `Process.monitor/1`, a
   `:DOWN` message is delivered immediately.
 
-  See ["The need for monitoring"](https://elixir-lang.org/getting-started/mix-otp/genserver.html#the-need-for-monitoring)
+  See ["The need for monitoring"](genservers.md#the-need-for-monitoring)
   for an example. See `:erlang.monitor/2` for more information.
 
   Inlined by the compiler.

lib/elixir/lib/registry.ex

@@ -27,8 +27,8 @@ defmodule Registry do
   `Registry.start_link/1`, it can be used to register and access named
   processes using the `{:via, Registry, {registry, key}}` tuple:
 
-      {:ok, _} = Registry.start_link(keys: :unique, name: Registry.ViaTest)
+      {:ok, _} = Registry.start_link(keys: :unique, name: MyApp.Registry)
-      name = {:via, Registry, {Registry.ViaTest, "agent"}}
+      name = {:via, Registry, {MyApp.Registry, "agent"}}
       {:ok, _} = Agent.start_link(fn -> 0 end, name: name)
       Agent.get(name, & &1)
       #=> 0
@@ -39,22 +39,22 @@ defmodule Registry do
   In the previous example, we were not interested in associating a value to the
   process:
 
-      Registry.lookup(Registry.ViaTest, "agent")
+      Registry.lookup(MyApp.Registry, "agent")
       #=> [{self(), nil}]
 
   However, in some cases it may be desired to associate a value to the process
   using the alternate `{:via, Registry, {registry, key, value}}` tuple:
 
-      {:ok, _} = Registry.start_link(keys: :unique, name: Registry.ViaTest)
+      {:ok, _} = Registry.start_link(keys: :unique, name: MyApp.Registry)
-      name = {:via, Registry, {Registry.ViaTest, "agent", :hello}}
+      name = {:via, Registry, {MyApp.Registry, "agent", :hello}}
       {:ok, agent_pid} = Agent.start_link(fn -> 0 end, name: name)
-      Registry.lookup(Registry.ViaTest, "agent")
+      Registry.lookup(MyApp.Registry, "agent")
       #=> [{agent_pid, :hello}]
 
   To this point, we have been starting `Registry` using `start_link/1`.
   Typically the registry is started as part of a supervision tree though:
 
-      {Registry, keys: :unique, name: Registry.ViaTest}
+      {Registry, keys: :unique, name: MyApp.Registry}
 
   Only registries with unique keys can be used in `:via`. If the name is
   already taken, the case-specific `start_link` function (`Agent.start_link/2`

lib/elixir/lib/string.ex

@@ -1871,6 +1871,104 @@ defmodule String do
     end
   end
 
+  defguardp replace_invalid_ii_of_iii(i, ii)
+            when Bitwise.bor(Bitwise.bsl(i, 6), ii) in 32..863 or
+                   Bitwise.bor(Bitwise.bsl(i, 6), ii) in 896..1023
+
+  defguardp replace_invalid_ii_of_iv(i, ii)
+            when Bitwise.bor(Bitwise.bsl(i, 6), ii) in 16..271
+
+  defguardp replace_invalid_iii_of_iv(i, ii, iii)
+            when Bitwise.bor(Bitwise.bor(Bitwise.bsl(i, 12), Bitwise.bsl(ii, 6)), iii) in 1024..17407
+
+  defguardp replace_invalid_is_next(next) when Bitwise.bsr(next, 6) !== 0b10
+
+  @doc ~S"""
+  Returns a new string created by replacing all invalid bytes with `replacement` (`"�"` by default).
+
+  ## Examples
+
+      iex> String.replace_invalid("asd" <> <<0xFF::8>>)
+      "asd�"
+
+      iex> String.replace_invalid("nem rán bề bề")
+      "nem rán bề bề"
+
+      iex> String.replace_invalid("nem rán b" <> <<225, 187>> <> " bề")
+      "nem rán b� bề"
+
+      iex> String.replace_invalid("nem rán b" <> <<225, 187>> <> " bề", "ERROR!")
+      "nem rán bERROR! bề"
+  """
+  @doc since: "1.16.0"
+  def replace_invalid(bytes, replacement \\ "�")
+      when is_binary(bytes) and is_binary(replacement) do
+    do_replace_invalid(bytes, replacement, <<>>)
+  end
+
+  # Valid ASCII (for better average speed)
+  defp do_replace_invalid(<<ascii::8, next::8, _::bytes>> = rest, rep, acc)
+       when ascii in 0..127 and replace_invalid_is_next(next) do
+    <<_::8, rest::bytes>> = rest
+    do_replace_invalid(rest, rep, acc <> <<ascii::8>>)
+  end
+
+  # Valid UTF-8
+  defp do_replace_invalid(<<grapheme::utf8, rest::bytes>>, rep, acc) do
+    do_replace_invalid(rest, rep, acc <> <<grapheme::utf8>>)
+  end
+
+  # 2/3 truncated sequence
+  defp do_replace_invalid(<<0b1110::4, i::4, 0b10::2, ii::6>>, rep, acc)
+       when replace_invalid_ii_of_iii(i, ii) do
+    acc <> rep
+  end
+
+  defp do_replace_invalid(<<0b1110::4, i::4, 0b10::2, ii::6, next::8, _::bytes>> = rest, rep, acc)
+       when replace_invalid_ii_of_iii(i, ii) and replace_invalid_is_next(next) do
+    <<_::16, rest::bytes>> = rest
+    do_replace_invalid(rest, rep, acc <> rep)
+  end
+
+  # 2/4
+  defp do_replace_invalid(<<0b11110::5, i::3, 0b10::2, ii::6>>, rep, acc)
+       when replace_invalid_ii_of_iv(i, ii) do
+    acc <> rep
+  end
+
+  defp do_replace_invalid(
+         <<0b11110::5, i::3, 0b10::2, ii::6, next::8, _::bytes>> = rest,
+         rep,
+         acc
+       )
+       when replace_invalid_ii_of_iv(i, ii) and replace_invalid_is_next(next) do
+    <<_::16, rest::bytes>> = rest
+    do_replace_invalid(rest, rep, acc <> rep)
+  end
+
+  # 3/4
+  defp do_replace_invalid(<<0b11110::5, i::3, 0b10::2, ii::6, 0b10::2, iii::6>>, rep, acc)
+       when replace_invalid_iii_of_iv(i, ii, iii) do
+    acc <> rep
+  end
+
+  defp do_replace_invalid(
+         <<0b11110::5, i::3, 0b10::2, ii::6, 0b10::2, iii::6, next::8, _::bytes>> = rest,
+         rep,
+         acc
+       )
+       when replace_invalid_iii_of_iv(i, ii, iii) and replace_invalid_is_next(next) do
+    <<_::24, rest::bytes>> = rest
+    do_replace_invalid(rest, rep, acc <> rep)
+  end
+
+  # Everything else
+  defp do_replace_invalid(<<_, rest::bytes>>, rep, acc),
+    do: do_replace_invalid(rest, rep, acc <> rep)
+
+  # Final
+  defp do_replace_invalid(<<>>, _, acc), do: acc
+
   @doc ~S"""
   Splits the string into chunks of characters that share a common trait.
 

lib/elixir/lib/supervisor.ex

@@ -413,10 +413,10 @@ defmodule Supervisor do
   The difference between the two approaches is that a module-based
   supervisor gives you more direct control over how the supervisor
   is initialized. Instead of calling `Supervisor.start_link/2` with
-  a list of child specifications that are automatically initialized, we manually
+  a list of child specifications that are implicitly initialized for us,
-  initialize the children by calling `Supervisor.init/2` inside its
+  we must explicitly initialize the children by calling `Supervisor.init/2`
-  `c:init/1` callback. `Supervisor.init/2` accepts the same `:strategy`,
+  inside its `c:init/1` callback. `Supervisor.init/2` accepts the same
-  `:max_restarts`, and `:max_seconds` options as `start_link/2`.
+  `:strategy`, `:max_restarts`, and `:max_seconds` options as `start_link/2`.
 
   > #### `use Supervisor` {: .info}
   >

lib/elixir/pages/anti-patterns/code-anti-patterns.md

@@ -1,6 +1,6 @@
 # Code-related anti-patterns
 
-This document outlines anti-patterns related to your code and particular Elixir idioms and features.
+This document outlines potential anti-patterns related to your code and particular Elixir idioms and features.
 
 ## Comments
 
@@ -34,7 +34,8 @@ You could refactor the code above like this:
 @five_min_in_seconds 60 * 5
 
 defp unix_five_min_from_now do
-  unix_now = DateTime.to_unix(DateTime.utc_now(), :second)
+  now = DateTime.utc_now()
+  unix_now = DateTime.to_unix(now, :second)
   unix_now + @five_min_in_seconds
 end
 ```
@@ -45,75 +46,6 @@ We removed the unnecessary comments. We also added a `@five_min_in_seconds` modu
 
 Elixir makes a clear distinction between **documentation** and code comments. The language has built-in first-class support for documentation through `@doc`, `@moduledoc`, and more. See the ["Writing documentation"](../getting-started/writing-documentation.md) guide for more information.
 
-## Complex branching
-
-#### Problem
-
-When a function assumes the responsibility of handling multiple errors alone, it can increase its cyclomatic complexity (metric of control-flow) and become incomprehensible. This situation can configure a specific instance of "Long function", a traditional anti-pattern, but has implications of its own. Under these circumstances, this function could get very confusing, difficult to maintain and test, and therefore bug-proneness.
-
-#### Example
-
-An example of this anti-pattern is when a function uses the `case` control-flow structure or other similar constructs (for example, `cond` or `receive`) to handle variations of a return type. This practice can make the function more complex, long, and difficult to understand, as shown next.
-
-```elixir
-def get_customer(customer_id) do
-  case SomeHTTPClient.get("/customers/#{customer_id}") do
-    {:ok, %{status: 200, body: body}} ->
-      case Jason.decode(body) do
-        {:ok, decoded} ->
-          %{
-            "first_name" => first_name,
-            "last_name" => last_name,
-            "company" => company
-          } = decoded
-
-          customer =
-            %Customer{
-              id: customer_id,
-              name: "#{first_name} #{last_name}",
-              company: company
-            }
-
-          {:ok, customer}
-
-        {:error, _} ->
-          {:error, "invalid response body"}
-      end
-
-    {:error, %{status: status, body: body}} ->
-      case Jason.decode(body) do
-        %{"error" => message} when is_binary(message) ->
-          {:error, message}
-
-        %{} ->
-          {:error, "invalid response with status #{status}"}
-      end
-  end
-end
-```
-
-The code above is complex because the `case` clauses are long and often have their own branching logic in them. With the clauses spread out, it is hard to understand what each clause does individually and it is hard to see all of the different scenarios your code pattern matches on.
-
-#### Refactoring
-
-As shown below, in this situation, instead of concentrating all error handling within the same function, creating complex branches, it is better to delegate each branch to a different private function. In this way, the code will be cleaner and more readable.
-
-```elixir
-def get_customer(customer_id) do
-  case SomeHTTPClient.get("/customers/#{customer_id}") do
-    {:ok, %{status: 200, body: body}} ->
-      http_customer_to_struct(customer_id, body)
-
-    {:error, %{status: status, body: body}} ->
-      http_error(status, body)
-  end
-end
-```
-
-Both `http_customer_to_struct(customer_id, body)` and `http_error(status, body)` above contain the previous branches refactored into private functions.
-
-It is worth noting that this refactoring is trivial to perform in Elixir because clauses cannot define variables or otherwise affect their parent scope. Therefore, extracting any clause or branch to a private function is a matter of gathering all variables used in that branch and passing them as arguments to the new function.
-
 ## Complex `else` clauses in `with`
 
 #### Problem
@@ -169,11 +101,11 @@ end
 
 #### Problem
 
-When we use multi-clause functions, it is possible to extract values in the clauses for further usage and for pattern matching/guard checking. This extraction itself does not represent an anti-pattern, but when you have too many clauses or too many arguments, it becomes hard to know which extracted parts are used for pattern/guards and what is used only inside the function body. This anti-pattern is related to [Unrelated multi-clause function](design-anti-patterns.md#unrelated-multi-clause-function), but with implications of its own. It impairs the code readability in a different way.
+When we use multi-clause functions, it is possible to extract values in the clauses for further usage and for pattern matching/guard checking. This extraction itself does not represent an anti-pattern, but when you have *extractions made across several clauses and several arguments of the same function*, it becomes hard to know which extracted parts are used for pattern/guards and what is used only inside the function body. This anti-pattern is related to [Unrelated multi-clause function](design-anti-patterns.md#unrelated-multi-clause-function), but with implications of its own. It impairs the code readability in a different way.
 
 #### Example
 
-The multi-clause function `drive/1` is extracting fields of an `%User{}` struct for usage in the clause expression (`age`) and for usage in the function body (`name`). Ideally, a function should not mix pattern matching extractions for usage in its guard expressions and also in its body.
+The multi-clause function `drive/1` is extracting fields of an `%User{}` struct for usage in the clause expression (`age`) and for usage in the function body (`name`):
 
 ```elixir
 def drive(%User{name: name, age: age}) when age >= 18 do
@@ -185,7 +117,7 @@ def drive(%User{name: name, age: age}) when age < 18 do
 end
 ```
 
-While the example is small and looks like a clear code, try to imagine a situation where `drive/1` was more complex, having many more clauses, arguments, and extractions.
+While the example above is small and does not configure an anti-pattern, it is an example of mixed extraction and pattern matching. A situation where `drive/1` was more complex, having many more clauses, arguments, and extractions, would make it hard to know at a glance which variables are used for pattern/guards and which ones are not.
 
 #### Refactoring
 
@@ -288,7 +220,7 @@ defmodule MyRequestHandler do
 end
 ```
 
-All valid statuses all defined as atoms within the same module, and that's enough. If you want to be explicit, you could also have a function that lists them:
+All valid statuses are defined as atoms within the same module, and that's enough. If you want to be explicit, you could also have a function that lists them:
 
 ```elixir
 def valid_statuses do
@@ -363,11 +295,13 @@ defmodule PlugAuth do
 end
 ```
 
+#### Additional remarks
+
 There are few known exceptions to this anti-pattern:
 
   * [Protocol implementations](`Kernel.defimpl/2`) are, by design, defined under the protocol namespace
 
-  * [Custom Mix tasks](`Mix.Task`) are always defined under the `Mix.Tasks` namespace, such as `Mix.Tasks.PlugAuth`
+  * In some scenarios, the namespace owner may allow exceptions to this rule. For example, in Elixir itself, you defined [custom Mix tasks](`Mix.Task`) by placing them under the `Mix.Tasks` namespace, such as `Mix.Tasks.PlugAuth`
 
   * If you are the maintainer for both `plug` and `plug_auth`, then you may allow `plug_auth` to define modules with the `Plug` namespace, such as `Plug.Auth`. However, you are responsible for avoiding or managing any conflicts that may arise in the future
 
@@ -436,7 +370,7 @@ iex> Graphics.plot(point_3d)
 
 Overall, the usage of `map.key` and `map[:key]` encode important information about your data structure, allowing developers to be clear about their intent. See both `Map` and `Access` module documentation for more information and examples.
 
-Another alternative to refactor this anti-pattern is to use pattern matching:
+An even simpler alternative to refactor this anti-pattern is to use pattern matching:
 
 ```elixir
 defmodule Graphics do
@@ -461,6 +395,8 @@ iex> Graphics.plot(point_3d)
 {5, 6, nil}
 ```
 
+Pattern-matching is specially useful when matching over multiple keys at once and also when you want to match and assert on the values of a map.
+
 Another alternative is to use structs. By default, structs only support static access to its fields:
 
 ```elixir

lib/elixir/pages/anti-patterns/design-anti-patterns.md

@@ -1,6 +1,6 @@
 # Design-related anti-patterns
 
-This document outlines anti-patterns related to your modules, functions, and the role they
+This document outlines potential anti-patterns related to your modules, functions, and the role they
 play within a codebase.
 
 ## Alternative return types
@@ -126,7 +126,7 @@ Remember booleans are internally represented as atoms. Therefore there is no per
 
 #### Problem
 
-This anti-pattern refers to code that uses exceptions for control flow. Exception handling itself does not represent an anti-pattern, but developers must prefer to use `case` and pattern matching to change the flow of their code, instead of `try/rescue`. In turn, library authors should provide developers with APIs to handle errors without relying on exception handling. When developers have no freedom to decide if an error is exceptional or not, this is considered an anti-pattern.
+This anti-pattern refers to code that uses `Exception`s for control flow. Exception handling itself does not represent an anti-pattern, but developers must prefer to use `case` and pattern matching to change the flow of their code, instead of `try/rescue`. In turn, library authors should provide developers with APIs to handle errors without relying on exception handling. When developers have no freedom to decide if an error is exceptional or not, this is considered an anti-pattern.
 
 #### Example
 
@@ -186,63 +186,15 @@ end
 
 A common practice followed by the community is to make the non-raising version return `{:ok, result}` or `{:error, Exception.t}`. For example, an HTTP client may return `{:ok, %HTTP.Response{}}` on success cases and `{:error, %HTTP.Error{}}` for failures, where `HTTP.Error` is [implemented as an exception](`Kernel.defexception/1`). This makes it convenient for anyone to raise an exception by simply calling `Kernel.raise/1`.
 
-## Feature envy
+#### Additional remarks
 
-#### Problem
+This anti-pattern was formerly known as [Using exceptions for control-flow](https://github.com/lucasvegi/Elixir-Code-Smells#using-exceptions-for-control-flow).
-
-This anti-pattern occurs when a function accesses more data or calls more functions from another module than from its own. The presence of this anti-pattern can make a module less cohesive and increase code coupling.
-
-#### Example
-
-In the following code, all the data used in the `calculate_total_item/1` function of the module `Order` comes from the `OrderItem` module. This increases coupling and decreases code cohesion unnecessarily.
-
-```elixir
-defmodule Order do
-  # Some functions...
-
-  def calculate_total_item(id) do
-    item = OrderItem.find_item(id)
-    total = (item.price + item.taxes) * item.amount
-
-    if discount = OrderItem.find_discount(item) do
-      total - total * discount
-    else
-      total
-    end
-  end
-end
-```
-
-#### Refactoring
-
-To remove this anti-pattern we can move `calculate_total_item/1` to `OrderItem`, decreasing coupling:
-
-```elixir
-defmodule OrderItem do
-  def find_item(id)
-  def find_discount(item)
-
-  def calculate_total_item(id) do   # <= function moved from Order!
-    item = find_item(id)
-    total = (item.price + item.taxes) * item.amount
-    discount = find_discount(item)
-
-    unless is_nil(discount) do
-      total - total * discount
-    else
-      total
-    end
-  end
-end
-```
-
-This refactoring is only possible when you own both modules. If the module you are invoking belongs to another application, then it is not possible to add new functions to it, and your only option is to define an additional module that augments the third-party module.
 
 ## Primitive obsession
 
 #### Problem
 
-This anti-pattern happens when Elixir basic types (for example, *integer*, *float*, and *string*) are abusively used in function parameters and code variables, rather than creating specific composite data types (for example, *tuples*, *maps*, and *structs*) that can better represent a domain.
+This anti-pattern happens when Elixir basic types (for example, *integer*, *float*, and *string*) are excessively used to carry structured information, rather than creating specific composite data types (for example, *tuples*, *maps*, and *structs*) that can better represent a domain.
 
 #### Example
 
@@ -250,17 +202,23 @@ An example of this anti-pattern is the use of a single *string* to represent an
 
 ```elixir
 defmodule MyApp do
-  def process_address(address) when is_binary(address) do
+  def extract_postal_code(address) when is_binary(address) do
-    # Do something with address...
+    # Extract postal code with address...
+  end
+
+  def fill_in_country(address) when is_binary(address) do
+    # Fill in missing country...
   end
 end
 ```
 
+While you may receive the `address` as a string from a database, web request, or a third-party, if you find yourself frequently manipulating or extracting information from the string, it is a good indicator you should convert the address into structured data:
+
 Another example of this anti-pattern is using floating numbers to model money and currency, when [richer data structures should be preferred](https://hexdocs.pm/ex_money/).
 
 #### Refactoring
 
-Possible solutions to this anti-pattern is to use maps or structs to model our address. The example below creates an `Address` struct, better representing this domain through a composite type. Additionally, we can modify the `process_address/1` function to accept a parameter of type `Address` instead of a *string*. With this modification, we can extract each field of this composite type individually when needed.
+Possible solutions to this anti-pattern is to use maps or structs to model our address. The example below creates an `Address` struct, better representing this domain through a composite type. Additionally, we introduce a `parse/1` function, that converts the string into an `Address`, which will simplify the logic of remainng functions. With this modification, we can extract each field of this composite type individually when needed.
 
 ```elixir
 defmodule Address do
@@ -270,8 +228,16 @@ end
 
 ```elixir
 defmodule MyApp do
-  def process_address(%Address{} = address) do
+  def parse(address) when is_binary(address) do
-    # Do something with address...
+    # Returns %Address{}
+  end
+
+  def extract_postal_code(%Address{} = address) do
+    # Extract postal code with address...
+  end
+
+  def fill_in_country(%Address{} = address) do
+    # Fill in missing country...
   end
 end
 ```
@@ -332,67 +298,52 @@ The following arguments were given to MyLibrary.foo/1:
   my_library.ex:2: MyLibrary.foo/1
 ```
 
+#### Additional remarks
+
+This anti-pattern was formerly known as [Working with invalid data](https://github.com/lucasvegi/Elixir-Code-Smells#working-with-invalid-data).
+
 ## Unrelated multi-clause function
 
 #### Problem
 
-Using multi-clause functions in Elixir, to group functions of the same name, is not an anti-pattern in itself. However, due to the great flexibility provided by this programming feature, some developers may abuse the number of guard clauses and pattern matches to group *unrelated* functionality.
+Using multi-clause functions in Elixir, to group functions of the same name, is a powerful Elixir feature. However, some developers may abuse this feature to group *unrelated* functionality, which configures an anti-pattern.
 
 #### Example
 
-A frequent example of this usage of multi-clause functions is when developers mix unrelated business logic into the same function definition. Such functions often have generic names or too broad specifications, making it difficult for maintainers and users of said functions to maintain and understand them.
+A frequent example of this usage of multi-clause functions is when developers mix unrelated business logic into the same function definition, in a way the behaviour of each clause is completely distinct from the other ones. Such functions often have too broad specifications, making it difficult for other developers to understand and maintain them.
 
-Some developers may use documentation mechanisms such as `@doc` annotations to compensate for poor code readability, however the documentation itself may end-up full of conditionals to describe how the function behaves for each different argument combination.
+Some developers may use documentation mechanisms such as `@doc` annotations to compensate for poor code readability, however the documentation itself may end-up full of conditionals to describe how the function behaves for each different argument combination. This is a good indicator that the clauses are ultimately unrelated.
 
 ```elixir
 @doc """
 Updates a struct.
 
-If given a "sharp" product (metal or glass with empty count),
+If given a product, it will...
-it will...
-
-If given a blunt product, it will...
 
 If given an animal, it will...
 """
-def update(%Product{count: nil, material: material})
+def update(%Product{count: count, material: material})  do
-    when material in ["metal", "glass"] do
   # ...
 end
 
-def update(%Product{count: count, material: material})
+def update(%Animal{count: count, skin: skin})  do
-    when count > 0 and material not in ["metal", "glass"] do
-  # ...
-end
-
-def update(%Animal{count: 1, skin: skin})
-    when skin in ["fur", "hairy"] do
   # ...
 end
 ```
 
+If updating an animal is completely different from updating a product and requires a different set of rules, it may be worth splitting those over different functions or even different modules.
+
 #### Refactoring
 
-As shown below, a possible solution to this anti-pattern is to break the business rules that are mixed up in a single unrelated multi-clause function in several different simple functions. More precise names make the scope of the function clear. Each function can have a specific `@doc`, describing its behavior and parameters received. While this refactoring sounds simple, it can have a lot of impact on the function's current users, so be careful!
+As shown below, a possible solution to this anti-pattern is to break the business rules that are mixed up in a single unrelated multi-clause function in simple functions. Each function can have a specific name and `@doc`, describing its behavior and parameters received. While this refactoring sounds simple, it can impact the function's current users, so be careful!
 
 ```elixir
 @doc """
-Updates a "sharp" product.
+Updates a product.
 
 It will...
 """
-def update_sharp_product(%Product{count: nil, material: material})
+def update_product(%Product{count: count, material: material}) do
-    when material in ["metal", "glass"] do
-  # ...
-end
-
-@doc """
-Updates a "blunt" product.
-
-It will...
-"""
-def update_blunt_product(%Product{count: count, material: material})
-    when count > 0 and material not in ["metal", "glass"] do
   # ...
 end
 
@@ -401,12 +352,48 @@ Updates an animal.
 
 It will...
 """
-def update_animal(%Animal{count: 1, skin: skin})
+def update_animal(%Animal{count: count, skin: skin}) do
-    when skin in ["fur", "hairy"] do
   # ...
 end
 ```
 
+These functions may still be implemented with multiple clauses, as long as the clauses group related funtionality. For example, `update_product` could be in practice implemented as follows:
+
+```elixir
+def update_product(%Product{count: 0}) do
+  # ...
+end
+
+def update_product(%Product{material: material})
+    when material in ["metal", "glass"] do
+  # ...
+end
+
+def update_product(%Product{material: material})
+    when material not in ["metal", "glass"] do
+  # ...
+end
+```
+
+You can see this pattern in practice within Elixir itself. The `+/2` operator can add `Integer`s and `Float`s together, but not `String`s, which instead use the `<>/2` operator. In this sense, it is reasonable to handle integers and floats in the same operation, but strings are unrelated enough to deserve their own function.
+
+You will also find examples in Elixir of functions that work with any struct, such as `struct/2`:
+
+```elixir
+iex> struct(URI.parse("/foo/bar"), path: "/bar/baz")
+%URI{
+  scheme: nil,
+  userinfo: nil,
+  host: nil,
+  port: nil,
+  path: "/bar/baz",
+  query: nil,
+  fragment: nil
+}
+```
+
+The difference here is that the `struct/2` function behaves precisely the same for any struct given, therefore there is no question of how the function handles different inputs. If the behaviour is clear and consistent for all inputs, then the anti-pattern does not take place.
+
 ## Using application configuration for libraries
 
 #### Problem
@@ -465,3 +452,35 @@ iex> DashSplitter.split("Lucas-Francisco-da-Matta-Vegi", [parts: 5])
 iex> DashSplitter.split("Lucas-Francisco-da-Matta-Vegi") #<= default config is used!
 ["Lucas", "Francisco-da-Matta-Vegi"]
 ```
+
+#### Additional Remarks
+
+For Mix tasks and related tools, it may be necessary to provide per-project configuration. For example, imagine you have a `:linter` project, which supports setting the output file and the verbosity level. You may choose to configure it through application environment:
+
+```elixir
+config :linter,
+  output_file: "/path/to/output.json",
+  verbosity: 3
+```
+
+However, `Mix` allows tasks to read per-project configuration via `Mix.Project.config/0`. In this case, you can configure the `:linter` directly in the `mix.exs` file:
+
+```elixir
+def project do
+  [
+    app: :my_app,
+    version: "1.0.0",
+    linter: [
+      output_file: "/path/to/output.json",
+      verbosity: 3
+    ],
+    ...
+  ]
+end
+```
+
+Additonally, if a Mix task is available, you can also accept these options as command line arguments (see `OptionParser`):
+
+```bash
+mix linter --output-file /path/to/output.json --verbosity 3
+```

lib/elixir/pages/anti-patterns/macro-anti-patterns.md

@@ -1,6 +1,6 @@
 # Meta-programming anti-patterns
 
-This document outlines anti-patterns related to meta-programming.
+This document outlines potential anti-patterns related to meta-programming.
 
 ## Large code generation by macros
 

lib/elixir/pages/anti-patterns/process-anti-patterns.md

@@ -1,6 +1,6 @@
 # Process-related anti-patterns
 
-This document outlines anti-patterns related to processes and process-based abstractions.
+This document outlines potential anti-patterns related to processes and process-based abstractions.
 
 ## Code organization by process
 

lib/elixir/pages/anti-patterns/what-anti-patterns.md

@@ -1,12 +1,14 @@
 # What are anti-patterns?
 
-Anti-patterns describe common mistakes or indicators of potential problems in code.
+Anti-patterns describe common mistakes or indicators of problems in code.
 They are also known as "code smells".
 
-The goal of these guides is to document known anti-patterns found in Elixir software
+The goal of these guides is to document potential anti-patterns found in Elixir software
-and teach developers how to identify and correct them. If an existing piece of code
+and teach developers how to identify them and their pitfalls. If an existing piece
-matches an anti-pattern, it does not mean your code must be rewritten. However, you
+of code matches an anti-pattern, it does not mean your code must be rewritten.
-should take its potential pitfalls and alternatives into consideration.
+Sometimes, even if a snippet matches a potential anti-pattern and its limitations,
+it may be the best approach to the problem at hand. No codebase is free of anti-patterns
+and one should not aim to remove all of them.
 
 The anti-patterns in these guides are broken into 4 main categories:
 
@@ -34,6 +36,8 @@ Each anti-pattern is documented using the following structure:
   * **Refactoring:** Ways to change your code to improve its qualities. Examples of refactored
     code are presented to illustrate these changes.
 
+An additional section with "Additional Remarks" may be provided. Those may include known scenarios where the anti-pattern does not apply.
+
 The initial catalog of anti-patterns was proposed by Lucas Vegi and Marco Tulio Valente, from [ASERG/DCC/UFMG](http://aserg.labsoft.dcc.ufmg.br/). For more info, see [Understanding Code Smells in Elixir Functional Language](https://github.com/lucasvegi/Elixir-Code-Smells/blob/main/etc/2023-emse-code-smells-elixir.pdf) and [the associated code repository](https://github.com/lucasvegi/Elixir-Code-Smells).
 
-Additionally, the Security Working Group of the [Erlang Ecosystem Foundation](https://erlef.github.io/security-wg/) publishes [documents with security resources and best-practices of both Erland and Elixir, including detailed guides for web applications](https://erlef.github.io/security-wg/).
+Additionally, the Security Working Group of the [Erlang Ecosystem Foundation](https://erlef.github.io/security-wg/) publishes [documents with security resources and best-practices of both Erlang and Elixir, including detailed guides for web applications](https://erlef.github.io/security-wg/).

lib/elixir/pages/cheatsheets/enum-cheat.cheatmd

@@ -820,7 +820,10 @@ Negative indexes count from the back:
 
 ```elixir
 iex> Enum.drop(cart, -1)
-[%{fruit: "orange", count: 6}]
+[
+  %{fruit: "apple", count: 3},
+  %{fruit: "banana", count: 1}
+]
 ```
 
 ### [`drop_every(enum, nth)`](`Enum.drop_every/2`)
@@ -851,10 +854,7 @@ Negative indexes count from the back:
 
 ```elixir
 iex> Enum.take(cart, -1)
-[
+[%{fruit: "orange", count: 6}]
-  %{fruit: "apple", count: 3},
-  %{fruit: "banana", count: 1}
-]
 ```
 
 ### [`take_every(enum, nth)`](`Enum.take_every/2`)

lib/elixir/pages/getting-started/alias-require-and-import.md

@@ -112,7 +112,7 @@ end
 
 In the example above, the imported `List.duplicate/2` is only visible within that specific function. `duplicate/2` won't be available in any other function in that module (or any other module for that matter).
 
-Note that `import`s are generally discouraged in the language. When working on your own code, prefer `alias` to `import`.
+While `import`s can be a useful for frameworks and libraries to build abstractions, developers should generally prefer `alias` to `import` on their own codebases, as aliases make the origin of the function being invoked clearer.
 
 ## use
 

lib/elixir/pages/getting-started/basic-types.md

@@ -83,7 +83,7 @@ You can also use `is_float` or `is_number` to check, respectively, if an argumen
 
 ## Identifying functions and documentation
 
-Before we move on to the next data type, let's talk about how Elixir identity functions.
+Before we move on to the next data type, let's talk about how Elixir identifies functions.
 
 Functions in Elixir are identified by both their name and their arity. The arity of a function describes the number of arguments that the function takes. From this point on we will use both the function name and its arity to describe functions throughout the documentation. `trunc/1` identifies the function which is named `trunc` and takes `1` argument, whereas `trunc/2` identifies a different (nonexistent) function with the same name but with an arity of `2`.
 

lib/elixir/pages/getting-started/case-cond-and-if.md

@@ -65,7 +65,7 @@ iex> case :ok do
 ** (CaseClauseError) no case clause matching: :ok
 ```
 
-The documentation for the `Kernel` module lists all available guards in its sidebar. You can also consult the complete [Patterns and Guards](../references/patterns-and-guards.html#guards) reference for in-depth documentation.
+The documentation for the `Kernel` module lists all available guards in its sidebar. You can also consult the complete [Patterns and Guards](../references/patterns-and-guards.md#guards) reference for in-depth documentation.
 
 ## cond
 

lib/elixir/pages/getting-started/enumerable-and-streams.md

@@ -119,6 +119,6 @@ iex> Enum.take(stream, 10)
 
 The example above will fetch the first 10 lines of the file you have selected. This means streams can be very useful for handling large files or even slow resources like network resources.
 
-The `Enum` and `Stream` modules provide a wide-range functions but know all of them at heart. Familiarize yourself with `Enum.map/2`, `Enum.reduce/3` and other functions with either `map` or `reduce` in their names, and you will naturally build an intuition around the most important use cases. You may also focus on the `Enum` module first and only move to `Stream` for the particular scenarios where laziness is required, to either deal with slow resources or large, possibly infinite, collections.
+The `Enum` and `Stream` modules provide a wide range of functions, but you don't have to know all of them by heart. Familiarize yourself with `Enum.map/2`, `Enum.reduce/3` and other functions with either `map` or `reduce` in their names, and you will naturally build an intuition around the most important use cases. You may also focus on the `Enum` module first and only move to `Stream` for the particular scenarios where laziness is required, to either deal with slow resources or large, possibly infinite, collections.
 
 Next, we'll look at a feature central to Elixir, Processes, which allows us to write concurrent, parallel and distributed programs in an easy and understandable way.

lib/elixir/pages/getting-started/io-and-the-file-system.md

@@ -129,7 +129,7 @@ iex> IO.write(pid, "hello")
 
 After `IO.write/2`, we can see the request sent by the `IO` module printed out (a four-elements tuple). Soon after that, we see that it fails since the `IO` module expected some kind of result, which we did not supply.
 
-By modeling IO devices with processes, the Erlang VM allows is to even read and write to files across nodes. Neat!
+By modeling IO devices with processes, the Erlang VM allows us to even read and write to files across nodes. Neat!
 
 ## `iodata` and `chardata`
 

lib/elixir/pages/getting-started/keywords-and-maps.md

@@ -127,7 +127,7 @@ iex> if true, do: "This will be seen", else: "This won't"
 
 Pay close attention to both syntaxes. In the keyword list format, we separate each key-value pair with commas, and each key is followed by `:`. In the `do`-blocks, we get rid of the colons, the commas, and separate each keyword by a newline. They are useful exactly because they remove the verbosity when writing blocks of code. Most of the time, you will use the block syntax, but it is good to know they are equivalent.
 
-This plays an important role in the language as it allows Elixir syntax to stay small but still expressive. We only need few data structures to represent the language, a topic we will come back to when talking about [optional syntax](optional-syntax.md) and go in-depth when discussing [meta-programming](../quote-and-unquote.md).
+This plays an important role in the language as it allows Elixir syntax to stay small but still expressive. We only need few data structures to represent the language, a topic we will come back to when talking about [optional syntax](optional-syntax.md) and go in-depth when discussing [meta-programming](../meta-programming/quote-and-unquote.md).
 
 With this out of the way, let's talk about maps.
 

lib/elixir/pages/getting-started/sigils.md

@@ -2,7 +2,7 @@
 
 Elixir provides double-quoted strings as well as a concept called charlists, which are defined using the `~c"hello world"` sigil syntax. In this chapter, we will learn more about sigils and how to define our own.
 
-One of Elixir's goals is extensibility: developers should be able to extend the language to fit any particular domain. Sigils provide the foundation for extending the language with custom textual representations. Sigils start with the tilde (`~`) character which is followed by a one lower-case letter or several upper-case ones and then a delimiter. Optionally, modifiers can be added after the final delimiter.
+One of Elixir's goals is extensibility: developers should be able to extend the language to fit any particular domain. Sigils provide the foundation for extending the language with custom textual representations. Sigils start with the tilde (`~`) character which is followed by either a single lower-case letter or one or more upper-case letters, and then a delimiter. Optional modifiers are added after the final delimiter.
 
 ## Regular expressions
 

lib/elixir/pages/mix-and-otp/agents.md

@@ -124,7 +124,7 @@ The first step in our implementation is to call `use Agent`. Most of the functio
 
 Then we define a `start_link/1` function, which will effectively start the agent. It is a convention to define a `start_link/1` function that always accepts a list of options. We don't plan on using any options right now, but we might later on. We then proceed to call `Agent.start_link/1`, which receives an anonymous function that returns the Agent's initial state.
 
-We are keeping a map inside the agent to store our keys and values. Getting and putting values on the map is done with the Agent API and the capture operator `&`, introduced in [the Getting Started guide](../getting-started/modules-and-functions.md#function-capturing). The agent passes its state to the anonymous function via the `&1` argument when `Agent.get/2` and `Agent.update/2` are called.
+We are keeping a map inside the agent to store our keys and values. Getting and putting values on the map is done with the Agent API and the capture operator `&`, introduced in [the Getting Started guide](../getting-started/anonymous-functions.md#the-capture-operator). The agent passes its state to the anonymous function via the `&1` argument when `Agent.get/2` and `Agent.update/2` are called.
 
 Now that the `KV.Bucket` module has been defined, our test should pass! You can try it yourself by running: `mix test`.
 

lib/elixir/pages/mix-and-otp/distributed-tasks.md

@@ -90,7 +90,7 @@ There are three better alternatives to `Node.spawn_link/2` that we could use in
 
 2. We could have a server running on the other node and send requests to that node via the `GenServer` API. For example, you can call a server on a remote node by using `GenServer.call({name, node}, arg)` or passing the remote process PID as the first argument
 
-3. We could use [tasks](`Task`), which we have learned about in [a previous chapter](../getting-started/mix-otp/task-and-gen-tcp.md), as they can be spawned on both local and remote nodes
+3. We could use [tasks](`Task`), which we have learned about in [a previous chapter](task-and-gen-tcp.md), as they can be spawned on both local and remote nodes
 
 The options above have different properties. The GenServer would serialize your requests on a single server, while tasks are effectively running asynchronously on the remote node, with the only serialization point being the spawning done by the supervisor.
 

lib/elixir/pages/mix-and-otp/genservers.md

@@ -1,6 +1,6 @@
 # Client-server communication with GenServer
 
-In the [previous chapter](../agents.md), we used agents to represent our buckets. In the [introduction to mix](../introduction-to-mix.md), we specified we would like to name each bucket so we can do the following:
+In the [previous chapter](agents.md), we used agents to represent our buckets. In the [introduction to mix](introduction-to-mix.md), we specified we would like to name each bucket so we can do the following:
 
 ```elixir
 CREATE shopping
@@ -320,7 +320,7 @@ So far we have used three callbacks: `handle_call/3`, `handle_cast/2` and `handl
 
 Since any message, including the ones sent via `send/2`, go to `handle_info/2`, there is a chance that unexpected messages will arrive to the server. Therefore, if we don't define the catch-all clause, those messages could cause our registry to crash, because no clause would match. We don't need to worry about such cases for `handle_call/3` and `handle_cast/2` though. Calls and casts are only done via the `GenServer` API, so an unknown message is quite likely a developer mistake.
 
-To help developers remember the differences between call, cast and info, the supported return values and more, we have a tiny [GenServer cheat sheet](/downloads/cheatsheets/gen-server.pdf).
+To help developers remember the differences between call, cast and info, the supported return values and more, we have a tiny [GenServer cheat sheet](https://elixir-lang.org/downloads/cheatsheets/gen-server.pdf).
 
 ## Monitors or links?
 

lib/elixir/pages/references/compatibility-and-deprecations.md

@@ -8,12 +8,11 @@ Elixir applies bug fixes only to the latest minor branch. Security patches are a
 
 Elixir version | Support
 :------------- | :-----------------------------
-1.16           | Development
+1.16           | Bug fixes and security patches
-1.15           | Bug fixes and security patches
+1.15           | Security patches only
 1.14           | Security patches only
 1.13           | Security patches only
 1.12           | Security patches only
-1.11           | Security patches only
 
 New releases are announced in the read-only [announcements mailing list](https://groups.google.com/group/elixir-lang-ann). All security releases [will be tagged with `[security]`](https://groups.google.com/forum/#!searchin/elixir-lang-ann/%5Bsecurity%5D%7Csort:date).
 
@@ -43,8 +42,9 @@ Erlang/OTP versioning is independent from the versioning of Elixir. Erlang relea
 
 Elixir version | Supported Erlang/OTP versions
 :------------- | :-------------------------------
+1.16           | 24 - 26
 1.15           | 24 - 26
-1.14           | 23 - 25
+1.14           | 23 - 25 (and Erlang/OTP 26 from v1.14.5)
 1.13           | 22 - 24 (and Erlang/OTP 25 from v1.13.4)
 1.12           | 22 - 24
 1.11           | 21 - 23 (and Erlang/OTP 24 from v1.11.4)

lib/elixir/pages/references/syntax-reference.md

@@ -395,7 +395,7 @@ end
 
 All of the constructs above are part of Elixir's syntax and have their own representation as part of the Elixir AST. This section will discuss the remaining constructs that are alternative representations of the constructs above. In other words, the constructs below can be represented in more than one way in your Elixir code and retain AST equivalence. We call this "Optional Syntax".
 
-For a lightweight introduction to Elixir's Optional Syntax, [see this document](https://elixir-lang.org/getting-started/optional-syntax.html). Below we continue with a more complete reference.
+For a lightweight introduction to Elixir's Optional Syntax, [see this document](../getting-started/optional-syntax.md). Below we continue with a more complete reference.
 
 ### Integers in other bases and Unicode code points
 

lib/elixir/pages/references/typespecs.md

@@ -17,7 +17,7 @@ Type specifications (most often referred to as *typespecs*) are defined in diffe
   * `@callback`
   * `@macrocallback`
 
-In addition, you can use `@typedoc` to describe a custom `@type` definition.
+In addition, you can use `@typedoc` to document a custom `@type` definition.
 
 See the "User-defined types" and "Defining a specification" sub-sections below for more information on defining types and typespecs.
 
@@ -250,19 +250,114 @@ Behaviours in Elixir (and Erlang) are a way to separate and abstract the generic
 
 A behaviour module defines a set of functions and macros (referred to as *callbacks*) that callback modules implementing that behaviour must export. This "interface" identifies the specific part of the component. For example, the `GenServer` behaviour and functions abstract away all the message-passing (sending and receiving) and error reporting that a "server" process will likely want to implement from the specific parts such as the actions that this server process has to perform.
 
-To define a behaviour module, it's enough to define one or more callbacks in that module. To define callbacks, the `@callback` and `@macrocallback` module attributes can be used (for function callbacks and macro callbacks respectively).
+Say we want to implement a bunch of parsers, each parsing structured data: for example, a JSON parser and a MessagePack parser. Each of these two parsers will *behave* the same way: both will provide a `parse/1` function and an `extensions/0` function. The `parse/1` function will return an Elixir representation of the structured data, while the `extensions/0` function will return a list of file extensions that can be used for each type of data (e.g., `.json` for JSON files).
 
-    defmodule MyBehaviour do
+We can create a `Parser` behaviour:
-      @callback my_fun(arg :: any) :: any
+
-      @macrocallback my_macro(arg :: any) :: Macro.t
+```elixir
-    end
+defmodule Parser do
+  @doc """
+  Parses a string.
+  """
+  @callback parse(String.t) :: {:ok, term} | {:error, atom}
+
+  @doc """
+  Lists all supported file extensions.
+  """
+  @callback extensions() :: [String.t]
+end
+```
 
 As seen in the example above, defining a callback is a matter of defining a specification for that callback, made of:
 
-  * the callback name (`my_fun` or `my_macro` in the example)
+  * the callback name (`parse` or `extensions` in the example)
-  * the arguments that the callback must accept (`arg :: any` in the example)
+  * the arguments that the callback must accept (`String.t`)
   * the *expected* type of the callback return value
 
+Modules adopting the `Parser` behaviour will have to implement all the functions defined with the `@callback` attribute. As you can see, `@callback` expects a function name but also a function specification like the ones used with the `@spec` attribute we saw above.
+
+### Implementing behaviours
+
+Implementing a behaviour is straightforward:
+
+```elixir
+defmodule JSONParser do
+  @behaviour Parser
+
+  @impl Parser
+  def parse(str), do: {:ok, "some json " <> str} # ... parse JSON
+
+  @impl Parser
+  def extensions, do: [".json"]
+end
+```
+
+```elixir
+defmodule CSVParser do
+  @behaviour Parser
+
+  @impl Parser
+  def parse(str), do: {:ok, "some csv " <> str} # ... parse CSV
+
+  @impl Parser
+  def extensions, do: [".csv"]
+end
+```
+
+If a module adopting a given behaviour doesn't implement one of the callbacks required by that behaviour, a compile-time warning will be generated.
+
+Furthermore, with `@impl` you can also make sure that you are implementing the **correct** callbacks from the given behaviour in an explicit manner. For example, the following parser implements both `parse` and `extensions`. However, thanks to a typo, `BADParser` is implementing `parse/0` instead of `parse/1`.
+
+```elixir
+defmodule BADParser do
+  @behaviour Parser
+
+  @impl Parser
+  def parse, do: {:ok, "something bad"}
+
+  @impl Parser
+  def extensions, do: ["bad"]
+end
+```
+
+This code generates a warning letting you know that you are mistakenly implementing `parse/0` instead of `parse/1`.
+You can read more about `@impl` in the [module documentation](Module.html#module-impl).
+
+### Using behaviours
+
+Behaviours are useful because you can pass modules around as arguments and you can then *call back* to any of the functions specified in the behaviour. For example, we can have a function that receives a filename, several parsers, and parses the file based on its extension:
+
+```elixir
+@spec parse_path(Path.t(), [module()]) :: {:ok, term} | {:error, atom}
+def parse_path(filename, parsers) do
+  with {:ok, ext} <- parse_extension(filename),
+       {:ok, parser} <- find_parser(ext, parsers),
+       {:ok, contents} <- File.read(filename) do
+    parser.parse(contents)
+  end
+end
+
+defp parse_extension(filename) do
+  if ext = Path.extname(filename) do
+    {:ok, ext}
+  else
+    {:error, :no_extension}
+  end
+end
+
+defp find_parser(ext, parsers) do
+  if parser = Enum.find(parsers, fn parser -> ext in parser.extensions() end) do
+    {:ok, parser}
+  else
+    {:error, :no_matching_parser}
+  end
+end
+```
+
+You could also invoke any parser directly: `CSVParser.parse(...)`.
+
+Note you don't need to define a behaviour in order to dynamically dispatch on a module, but those features often go hand in hand.
+
 ### Optional callbacks
 
 Optional callbacks are callbacks that callback modules may implement if they want to, but are not required to. Usually, behaviour modules know if they should call those callbacks based on configuration, or they check if the callbacks are defined with `function_exported?/3` or `macro_exported?/3`.
@@ -278,34 +373,6 @@ Optional callbacks can be defined through the `@optional_callbacks` module attri
 
 One example of optional callback in Elixir's standard library is `c:GenServer.format_status/2`.
 
-### Implementing behaviours
-
-To specify that a module implements a given behaviour, the `@behaviour` attribute must be used:
-
-    defmodule MyBehaviour do
-      @callback my_fun(arg :: any) :: any
-    end
-
-    defmodule MyCallbackModule do
-      @behaviour MyBehaviour
-      def my_fun(arg), do: arg
-    end
-
-If a callback module that implements a given behaviour doesn't export all the functions and macros defined by that behaviour, the user will be notified through warnings during the compilation process (no errors will happen).
-
-You can also use the `@impl` attribute before a function to denote that particular function is implementation a behaviour:
-
-    defmodule MyCallbackModule do
-      @behaviour MyBehaviour
-
-      @impl true
-      def my_fun(arg), do: arg
-    end
-
-You can also use `@impl MyBehaviour` to make clearer from which behaviour the callbacks comes from, providing even more context for future readers of your code.
-
-Elixir's standard library contains a few frequently used behaviours such as `GenServer`, `Supervisor`, and `Application`.
-
 ### Inspecting behaviours
 
 The `@callback` and `@optional_callbacks` attributes are used to create a `behaviour_info/1` function available on the defining module. This function can be used to retrieve the callbacks and optional callbacks defined by that module.
@@ -319,6 +386,10 @@ For example, for the `MyBehaviour` module defined in "Optional callbacks" above:
 
 When using `iex`, the `IEx.Helpers.b/1` helper is also available.
 
+## Pitfalls
+
+There are some known pitfalls when using typespecs, they are documented next.
+
 ## The `string()` type
 
 Elixir discourages the use of the `string()` type. The `string()` type refers to Erlang strings, which are known as "charlists" in Elixir. They do not refer to Elixir strings, which are UTF-8 encoded binaries. To avoid confusion, if you attempt to use the type `string()`, Elixir will emit a warning. You should use `charlist()`, `nonempty_charlist()`, `binary()` or `String.t()` accordingly, or any of the several literal representations for these types.

lib/elixir/scripts/windows_installer/build.sh

@@ -4,43 +4,27 @@
 # With Elixir archive:
 #
 #     ELIXIR_ZIP=Precompiled.zip OTP_VERSION=25.3.2.2 ./build.sh
-#
-# With Elixir version:
-#
-#     ELIXIR_VERSION=1.14.5 OTP_VERSION=25.3.2.2 ./build.sh
 set -euo pipefail
 
-OTP_VERSION="${OTP_VERSION:-26.0}"
-otp_release=`echo "${OTP_VERSION}" | cut -d. -f1`
-
 mkdir -p tmp
+rm -rf tmp/elixir
+unzip -d "tmp/elixir" "${ELIXIR_ZIP}"
 
-ELIXIR_VERSION="${ELIXIR_VERSION:-}"
+elixir_version=`cat tmp/elixir/VERSION`
-if [ -n "${ELIXIR_VERSION}" ]; then
+otp_release=`erl -noshell -eval 'io:put_chars(erlang:system_info(otp_release)), halt().'`
-  ELIXIR_ZIP="tmp/elixir-${ELIXIR_VERSION}-otp-${otp_release}.zip"
+otp_version=`erl -noshell -eval '{ok, Vsn} = file:read_file(code:root_dir() ++ "/releases/" ++ erlang:system_info(otp_release) ++ "/OTP_VERSION"), io:put_chars(Vsn), halt().'`
-  if [ ! -f "${ELIXIR_ZIP}" ]; then
+elixir_exe=elixir-otp-${otp_release}.exe
-    url="https://github.com/elixir-lang/elixir/releases/download/v${ELIXIR_VERSION}/elixir-otp-${otp_release}.zip"
-    echo "downloading ${url}"
-    curl --fail -L -o "${ELIXIR_ZIP}" "${url}"
-  fi
-  basename=elixir-${ELIXIR_VERSION}-otp-${otp_release}
-else
-  basename=elixir-otp-${otp_release}
-fi
-
-if [ ! -d "tmp/${basename}" ]; then
-  unzip -d "tmp/${basename}" "${ELIXIR_ZIP}"
-fi
 
 # brew install makensis
 # apt install -y nsis
 # choco install -y nsis
 export PATH="/c/Program Files (x86)/NSIS:${PATH}"
 makensis \
-  -X"OutFile tmp\\${basename}.exe" \
+  -X"OutFile tmp\\${elixir_exe}" \
-  -DOTP_VERSION=${OTP_VERSION} \
   -DOTP_RELEASE="${otp_release}" \
-  -DELIXIR_DIR=tmp\\${basename} \
+  -DOTP_VERSION=${otp_version} \
+  -DELIXIR_DIR=tmp\\elixir \
+  -DELIXIR_VERSION=${elixir_version} \
   installer.nsi
 
-echo "Installer path: tmp/${basename}.exe"
+echo "Installer path: tmp/${elixir_exe}"

lib/elixir/scripts/windows_installer/installer.nsi

@@ -6,7 +6,8 @@ Name "Elixir"
 ManifestDPIAware true
 Unicode True
 InstallDir "$PROGRAMFILES64\Elixir"
-!define MUI_ICON "assets\drop.ico"
+!define MUI_ICON "assets\Elixir.ico"
+!define MUI_UNICON "assets\Elixir.ico"
 
 ; Install Page: Install Erlang/OTP
 
@@ -191,7 +192,17 @@ FunctionEnd
 Section "Install Elixir" SectionElixir
   SetOutPath "$INSTDIR"
   File /r "${ELIXIR_DIR}\"
+  File "assets\Elixir.ico"
   File "update_system_path.erl"
+  WriteRegStr   HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\Elixir" "DisplayName" "Elixir"
+  WriteRegStr   HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\Elixir" "DisplayVersion" "${ELIXIR_VERSION}"
+  WriteRegStr   HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\Elixir" "DisplayIcon" "$INSTDIR\Elixir.ico"
+  WriteRegStr   HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\Elixir" "Publisher" "The Elixir Team"
+  WriteRegStr   HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\Elixir" "UninstallString" '"$INSTDIR\Uninstall.exe"'
+  WriteRegDWORD HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\Elixir" "NoModify" 1
+  WriteRegDWORD HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\Elixir" "NoRepair" 1
+
+  WriteRegStr   HKLM "Software\Elixir\Elixir" "InstallRoot" "$INSTDIR"
 
   WriteUninstaller "Uninstall.exe"
 SectionEnd
@@ -270,6 +281,9 @@ UninstPage custom un.FinishPageShow un.FinishPageLeave
 
 Section "Uninstall"
   RMDir /r "$INSTDIR"
+  DeleteRegKey HKLM "Software\Microsoft\Windows\CurrentVersion\Uninstall\Elixir"
+  DeleteRegKey HKLM "Software\Elixir\Elixir"
+  DeleteRegKey /ifempty HKLM "Software\Elixir"
 SectionEnd
 
 !insertmacro MUI_LANGUAGE "English"

lib/elixir/src/elixir.erl

@@ -486,7 +486,7 @@ string_to_tokens(String, StartLine, StartColumn, File, Opts) when is_integer(Sta
     {ok, _Line, _Column, [], Tokens} ->
       {ok, Tokens};
     {ok, _Line, _Column, Warnings, Tokens} ->
-      (lists:keyfind(warnings, 1, Opts) /= {warnings, false}) andalso
+      (lists:keyfind(emit_warnings, 1, Opts) /= {emit_warnings, false}) andalso
         [elixir_errors:erl_warn(L, File, M) || {L, M} <- lists:reverse(Warnings)],
       {ok, Tokens};
     {error, {Location, {ErrorPrefix, ErrorSuffix}, Token}, _Rest, _Warnings, _SoFar} ->
@@ -544,8 +544,8 @@ to_binary(Atom) when is_atom(Atom) -> atom_to_binary(Atom).
 
 handle_parsing_opts(File, Opts) ->
   WarningFile =
-    case lists:keyfind(warnings, 1, Opts) of
+    case lists:keyfind(emit_warnings, 1, Opts) of
-      {warnings, false} -> nil;
+      {emit_warnings, false} -> nil;
       _ -> File
     end,
   LiteralEncoder =

lib/elixir/src/elixir.hrl

@@ -36,6 +36,5 @@
   ascii_identifiers_only=true,
   indentation=0,
   mismatch_hints=[],
-  warn_on_unnecessary_quotes=true,
   warnings=[]
 }).

lib/elixir/src/elixir_bitstring.erl

@@ -35,7 +35,12 @@ expand(BitstrMeta, Fun, [{'::', Meta, [Left, Right]} | T], Acc, S, E, Alignment,
 
   MatchOrRequireSize = RequireSize or is_match_size(T, EL),
   EType = expr_type(ELeft),
-  {ERight, EAlignment, SS, ES} = expand_specs(EType, Meta, Right, SL, OriginalS, EL, MatchOrRequireSize),
+  ExpectSize = case ELeft of
+    {'^', _, [{_, _, _}]} -> {infer, ELeft};
+    _ when MatchOrRequireSize -> required;
+    _ -> optional
+  end,
+  {ERight, EAlignment, SS, ES} = expand_specs(EType, Meta, Right, SL, OriginalS, EL, ExpectSize),
 
   EAcc = concat_or_prepend_bitstring(Meta, ELeft, ERight, Acc, ES, MatchOrRequireSize),
   expand(BitstrMeta, Fun, T, EAcc, {SS, OriginalS}, ES, alignment(Alignment, EAlignment), RequireSize);
@@ -147,7 +152,7 @@ expand_expr(Meta, Component, Fun, S, E) ->
 
 %% Expands and normalizes types of a bitstring.
 
-expand_specs(ExprType, Meta, Info, S, OriginalS, E, RequireSize) ->
+expand_specs(ExprType, Meta, Info, S, OriginalS, E, ExpectSize) ->
   Default =
     #{size => default,
       unit => default,
@@ -158,11 +163,17 @@ expand_specs(ExprType, Meta, Info, S, OriginalS, E, RequireSize) ->
     expand_each_spec(Meta, unpack_specs(Info, []), Default, S, OriginalS, E),
 
   MergedType = type(Meta, ExprType, Type, E),
-  validate_size_required(Meta, RequireSize, ExprType, MergedType, Size, ES),
+  validate_size_required(Meta, ExpectSize, ExprType, MergedType, Size, ES),
   SizeAndUnit = size_and_unit(Meta, ExprType, Size, Unit, ES),
   Alignment = compute_alignment(MergedType, Size, Unit),
 
-  [H | T] = build_spec(Meta, Size, Unit, MergedType, Endianness, Sign, SizeAndUnit, ES),
+  MaybeInferredSize = case {ExpectSize, MergedType, SizeAndUnit} of
+    {{infer, PinnedVar}, binary, []} -> [{size, Meta, [{{'.', Meta, [erlang, byte_size]}, Meta, [PinnedVar]}]}];
+    {{infer, PinnedVar}, bitstring, []} -> [{size, Meta, [{{'.', Meta, [erlang, bit_size]}, Meta, [PinnedVar]}]}];
+    _ -> SizeAndUnit
+  end,
+
+  [H | T] = build_spec(Meta, Size, Unit, MergedType, Endianness, Sign, MaybeInferredSize, ES),
   {lists:foldl(fun(I, Acc) -> {'-', Meta, [Acc, I]} end, H, T), Alignment, SS, ES}.
 
 type(_, default, default, _) ->
@@ -276,7 +287,7 @@ validate_spec_arg(Meta, unit, Value, _S, _OriginalS, E) when not is_integer(Valu
 validate_spec_arg(_Meta, _Key, _Value, _S, _OriginalS, _E) ->
   ok.
 
-validate_size_required(Meta, true, default, Type, default, E) when Type == binary; Type == bitstring ->
+validate_size_required(Meta, required, default, Type, default, E) when Type == binary; Type == bitstring ->
   function_error(Meta, E, ?MODULE, unsized_binary),
   ok;
 validate_size_required(_, _, _, _, _, _) ->

lib/elixir/src/elixir_compiler.erl

@@ -1,6 +1,6 @@
 %% Elixir compiler front-end to the Erlang backend.
 -module(elixir_compiler).
--export([string/3, quoted/3, bootstrap/0, file/2, compile/3]).
+-export([string/3, quoted/3, bootstrap/0, file/2, compile/4]).
 -include("elixir.hrl").
 
 string(Contents, File, Callback) ->
@@ -36,22 +36,22 @@ maybe_fast_compile(Forms, Args, E) ->
   case (?key(E, module) == nil) andalso allows_fast_compilation(Forms) andalso
         (not elixir_config:is_bootstrap()) of
     true  -> fast_compile(Forms, E);
-    false -> compile(Forms, Args, E)
+    false -> compile(Forms, Args, [], E)
   end,
   ok.
 
-compile(Quoted, ArgsList, #{line := Line} = E) ->
+compile(Quoted, ArgsList, CompilerOpts, #{line := Line} = E) ->
   Block = no_tail_optimize([{line, Line}], Quoted),
   {Expanded, SE, EE} = elixir_expand:expand(Block, elixir_env:env_to_ex(E), E),
   elixir_env:check_unused_vars(SE, EE),
 
   {Module, Fun, Purgeable} =
-    elixir_erl_compiler:spawn(fun() -> spawned_compile(Expanded, E) end),
+    elixir_erl_compiler:spawn(fun() -> spawned_compile(Expanded, CompilerOpts, E) end),
 
   Args = list_to_tuple(ArgsList),
   {dispatch(Module, Fun, Args, Purgeable), SE, EE}.
 
-spawned_compile(ExExprs, #{line := Line, file := File} = E) ->
+spawned_compile(ExExprs, CompilerOpts, #{line := Line, file := File} = E) ->
   {Vars, S} = elixir_erl_var:from_env(E),
   {ErlExprs, _} = elixir_erl_pass:translate(ExExprs, erl_anno:new(Line), S),
 
@@ -59,7 +59,7 @@ spawned_compile(ExExprs, #{line := Line, file := File} = E) ->
   Fun = code_fun(?key(E, module)),
   Forms = code_mod(Fun, ErlExprs, Line, File, Module, Vars),
 
-  {Module, Binary} = elixir_erl_compiler:noenv_forms(Forms, File, [nowarn_nomatch, no_bool_opt, no_ssa_opt]),
+  {Module, Binary} = elixir_erl_compiler:noenv_forms(Forms, File, [nowarn_nomatch | CompilerOpts]),
   code:load_binary(Module, "", Binary),
   {Module, Fun, is_purgeable(Module, Binary)}.
 

lib/elixir/src/elixir_errors.erl

@@ -57,8 +57,7 @@ get_snippet(File, Position) ->
   LineNumber = extract_line(Position),
   get_file_line(File, LineNumber).
 
-get_file_line(_, 0) -> nil;
+get_file_line(File, LineNumber) when is_integer(LineNumber), LineNumber > 0 ->
-get_file_line(File, LineNumber) ->
   case file:open(File, [read, binary]) of
     {ok, IoDevice} ->
       Line = traverse_file_line(IoDevice, LineNumber),
@@ -66,7 +65,8 @@ get_file_line(File, LineNumber) ->
       Line;
     {error, _} ->
       nil
-  end.
+  end;
+get_file_line(_, _) -> nil.
 
 traverse_file_line(IoDevice, 1) ->
   case file:read_line(IoDevice) of
@@ -319,7 +319,10 @@ parse_error(Location, File, Error, <<>>, Input) ->
     <<"syntax error before: ">> -> <<"syntax error: expression is incomplete">>;
     _ -> <<Error/binary>>
   end,
-  raise_snippet(Location, File, Input, 'Elixir.TokenMissingError', Message);
+  case lists:keytake(error_type, 1, Location) of
+    {value, {error_type, unclosed_delimiter}, Loc} -> raise_token_missing(Loc, File, Input, Message);
+    _ -> raise_snippet(Location, File, Input, 'Elixir.TokenMissingError', Message)
+  end;
 
 %% Show a nicer message for end of line
 parse_error(Location, File, <<"syntax error before: ">>, <<"eol">>, Input) ->
@@ -404,9 +407,15 @@ parse_erl_term(Term) ->
   Parsed.
 
 raise_mismatched_delimiter(Location, File, Input, Message) ->
-  {InputString, _, _} = Input,
+  {InputString, StartLine, _} = Input,
   InputBinary = elixir_utils:characters_to_binary(InputString),
-  raise('Elixir.MismatchedDelimiterError', Message,  [{file, File}, {snippet, InputBinary} | Location]).
+  KV = [{file, File}, {line_offset, StartLine - 1}, {snippet, InputBinary} | Location],
+  raise('Elixir.MismatchedDelimiterError', Message, KV).
+
+raise_token_missing(Location, File, Input, Message) ->
+  {InputString, StartLine, _} = Input,
+  InputBinary = elixir_utils:characters_to_binary(InputString),
+  raise('Elixir.TokenMissingError', Message,  [{line_offset, StartLine - 1}, {file, File}, {snippet, InputBinary} | Location]).
 
 raise_reserved(Location, File, Input, Keyword) ->
   raise_snippet(Location, File, Input, 'Elixir.SyntaxError',

lib/elixir/src/elixir_expand.erl

@@ -402,16 +402,12 @@ expand({Atom, Meta, Args}, S, E) when is_atom(Atom), is_list(Meta), is_list(Args
 
 %% Remote calls
 
-expand({{'.', DotMeta, [Left, Right]}, Meta, Args} = Expr, S, E)
+expand({{'.', DotMeta, [Left, Right]}, Meta, Args}, S, E)
     when (is_tuple(Left) orelse is_atom(Left)), is_atom(Right), is_list(Meta), is_list(Args) ->
   {ELeft, SL, EL} = expand(Left, elixir_env:prepare_write(S), E),
-  NoParens = lists:keyfind(no_parens, 1, Meta),
-
-  (is_atom(ELeft) and (Args =:= []) and (NoParens =:= {no_parens, true})) andalso
-    elixir_errors:file_warn(Meta, E, ?MODULE, {remote_nullary_no_parens, Expr}),
 
   elixir_dispatch:dispatch_require(Meta, ELeft, Right, Args, S, EL, fun(AR, AF, AA) ->
-    expand_remote(AR, DotMeta, AF, Meta, NoParens, AA, S, SL, EL)
+    expand_remote(AR, DotMeta, AF, Meta, AA, S, SL, EL)
   end);
 
 %% Anonymous calls
@@ -858,12 +854,13 @@ expand_local(Meta, Name, Args, _S, #{function := nil} = E) ->
 
 %% Remote
 
-expand_remote(Receiver, DotMeta, Right, Meta, NoParens, Args, S, SL, #{context := Context} = E) when is_atom(Receiver) or is_tuple(Receiver) ->
+expand_remote(Receiver, DotMeta, Right, Meta, Args, S, SL, #{context := Context} = E)
+    when is_atom(Receiver) or is_tuple(Receiver) ->
   assert_no_clauses(Right, Meta, Args, E),
 
   if
     Context =:= guard, is_tuple(Receiver) ->
-      (NoParens /= {no_parens, true}) andalso
+      (lists:keyfind(no_parens, 1, Meta) /= {no_parens, true}) andalso
         function_error(Meta, E, ?MODULE, {parens_map_lookup, Receiver, Right, guard_context(S)}),
 
       {{{'.', DotMeta, [Receiver, Right]}, Meta, []}, SL, E};
@@ -881,7 +878,7 @@ expand_remote(Receiver, DotMeta, Right, Meta, NoParens, Args, S, SL, #{context :
           file_error(Meta, E, elixir_rewrite, Error)
       end
   end;
-expand_remote(Receiver, DotMeta, Right, Meta, _NoParens, Args, _, _, E) ->
+expand_remote(Receiver, DotMeta, Right, Meta, Args, _, _, E) ->
   Call = {{'.', DotMeta, [Receiver, Right]}, Meta, Args},
   file_error(Meta, E, ?MODULE, {invalid_call, Call}).
 
@@ -1171,9 +1168,6 @@ assert_no_underscore_clause_in_cond(_Other, _E) ->
 guard_context(#elixir_ex{prematch={_, _, {bitsize, _}}}) -> "bitstring size specifier";
 guard_context(_) -> "guards".
 
-format_error({remote_nullary_no_parens, Expr}) ->
-  String = 'Elixir.String':replace_suffix('Elixir.Macro':to_string(Expr), <<"()">>, <<>>),
-  io_lib:format("parentheses are required for function calls with no arguments, got: ~ts", [String]);
 format_error(invalid_match_on_zero_float) ->
   "pattern matching on 0.0 is equivalent to matching only on +0.0 from Erlang/OTP 27+. Instead you must match on +0.0 or -0.0";
 format_error({useless_literal, Term}) ->

lib/elixir/src/elixir_module.erl

@@ -421,7 +421,9 @@ build(Module, Line, File, E) ->
 %% Handles module and callback evaluations.
 
 eval_form(Line, Module, DataBag, Block, Vars, Prune, E) ->
-  {Value, ExS, EE} = elixir_compiler:compile(Block, Vars, E),
+  %% Given Elixir modules can get very long to compile due to metaprogramming,
+  %% we disable expansions that take linear time to code size.
+  {Value, ExS, EE} = elixir_compiler:compile(Block, Vars, [no_bool_opt, no_ssa_opt], E),
   elixir_overridable:store_not_overridden(Module),
   EV = (elixir_env:reset_vars(EE))#{line := Line},
   EC = eval_callbacks(Line, DataBag, before_compile, [EV], EV),

lib/elixir/src/elixir_parser.yrl

@@ -910,11 +910,6 @@ build_identifier({'.', Meta, _} = Dot, Args) ->
 build_identifier({op_identifier, Location, Identifier}, [Arg]) ->
   {Identifier, [{ambiguous_op, nil} | meta_from_location(Location)], [Arg]};
 
-%% TODO: Either remove ... or make it an operator on v2.0
-build_identifier({_, {Line, Column, _} = Location, '...'}, Args) when is_list(Args) ->
-  warn({Line, Column}, "... is no longer supported as a function call and it must receive no arguments"),
-  {'...', meta_from_location(Location), Args};
-
 build_identifier({_, Location, Identifier}, Args) ->
   {Identifier, meta_from_location(Location), Args}.
 

lib/elixir/src/elixir_tokenizer.erl

@@ -123,8 +123,6 @@ tokenize(String, Line, Column, Opts) ->
         Acc#elixir_tokenizer{preserve_comments=PreserveComments};
       ({unescape, Unescape}, Acc) when is_boolean(Unescape) ->
         Acc#elixir_tokenizer{unescape=Unescape};
-      ({warn_on_unnecessary_quotes, Unnecessary}, Acc) when is_boolean(Unnecessary) ->
-        Acc#elixir_tokenizer{warn_on_unnecessary_quotes=Unnecessary};
       (_, Acc) ->
         Acc
     end, #elixir_tokenizer{identifier_tokenizer=IdentifierTokenizer}, Opts),
@@ -144,12 +142,18 @@ tokenize([], Line, Column, #elixir_tokenizer{cursor_completion=Cursor} = Scope,
   AccTokens = cursor_complete(Line, CursorColumn, CursorTerminators, CursorTokens),
   {ok, Line, Column, AllWarnings, AccTokens};
 
-tokenize([], EndLine, Column, #elixir_tokenizer{terminators=[{Start, {StartLine, _, _}, _} | _]} = Scope, Tokens) ->
+tokenize([], EndLine, _, #elixir_tokenizer{terminators=[{Start, {StartLine, StartColumn, _}, _} | _]} = Scope, Tokens) ->
   End = terminator(Start),
   Hint = missing_terminator_hint(Start, End, Scope),
-  Message = "missing terminator: ~ts (for \"~ts\" starting at line ~B)",
+  Message = "missing terminator: ~ts",
-  Formatted = io_lib:format(Message, [End, Start, StartLine]),
+  Formatted = io_lib:format(Message, [End]),
-  Meta = [{opening_delimiter, Start} | ?LOC(EndLine, Column)],
+  Meta = [
+          {error_type, unclosed_delimiter},
+          {opening_delimiter, Start},
+          {line, StartLine},
+          {column, StartColumn},
+          {end_line, EndLine}
+         ],
   error({Meta, [Formatted, Hint], []}, [], Scope, Tokens);
 
 tokenize([], Line, Column, #elixir_tokenizer{} = Scope, Tokens) ->
@@ -794,7 +798,7 @@ handle_strings(T, Line, Column, H, Scope, Tokens) ->
               Token = {kw_identifier, {Line, Column - 1, nil}, Atom},
               tokenize(Rest, NewLine, NewColumn + 1, NewScope, [Token | Tokens]);
             {error, Reason} ->
-              {error, Reason, Rest, Tokens}
+              error(Reason, Rest, NewScope, Tokens)
           end;
 
         {ok, Unescaped} ->
@@ -964,12 +968,11 @@ eol(_Line, _Column, [{eol, {Line, Column, Count}} | Tokens]) ->
 eol(Line, Column, Tokens) ->
   [{eol, {Line, Column, 1}} | Tokens].
 
-is_unnecessary_quote([Part], #elixir_tokenizer{warn_on_unnecessary_quotes=true} = Scope) when is_list(Part) ->
+is_unnecessary_quote([Part], Scope) when is_list(Part) ->
   case (Scope#elixir_tokenizer.identifier_tokenizer):tokenize(Part) of
     {identifier, _, [], _, true, Special} -> not lists:member(at, Special);
     _ -> false
   end;
-
 is_unnecessary_quote(_Parts, _Scope) ->
   false.
 

lib/elixir/test/elixir/code_formatter/containers_test.exs

@@ -203,6 +203,7 @@ defmodule Code.Formatter.ContainersTest do
       assert_same ~S(["\w": 1, "\\w": 2])
       assert_same ~S(["Elixir.Foo": 1, "Elixir.Bar": 2])
       assert_format ~S(["Foo": 1, "Bar": 2]), ~S([Foo: 1, Bar: 2])
+      assert_same ~S(["with \"scare quotes\"": 1])
     end
 
     test "with operators keyword lists" do

lib/elixir/test/elixir/code_fragment_test.exs

@@ -443,12 +443,11 @@ defmodule CodeFragmentTest do
     end
 
     test "column out of range" do
-      assert CF.surround_context("hello", {1, 20}) ==
+      assert CF.surround_context("hello", {1, 20}) == :none
-               %{begin: {1, 1}, context: {:local_or_var, ~c"hello"}, end: {1, 6}}
     end
 
     test "local_or_var" do
-      for i <- 1..9 do
+      for i <- 1..8 do
         assert CF.surround_context("hello_wo", {1, i}) == %{
                  context: {:local_or_var, ~c"hello_wo"},
                  begin: {1, 1},
@@ -456,9 +455,9 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context("hello_wo ", {1, 10}) == :none
+      assert CF.surround_context("hello_wo", {1, 9}) == :none
 
-      for i <- 2..10 do
+      for i <- 2..9 do
         assert CF.surround_context(" hello_wo", {1, i}) == %{
                  context: {:local_or_var, ~c"hello_wo"},
                  begin: {1, 2},
@@ -466,9 +465,9 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context(" hello_wo ", {1, 11}) == :none
+      assert CF.surround_context(" hello_wo", {1, 10}) == :none
 
-      for i <- 1..7 do
+      for i <- 1..6 do
         assert CF.surround_context("hello!", {1, i}) == %{
                  context: {:local_or_var, ~c"hello!"},
                  begin: {1, 1},
@@ -476,9 +475,9 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context("hello! ", {1, 8}) == :none
+      assert CF.surround_context("hello!", {1, 7}) == :none
 
-      for i <- 1..6 do
+      for i <- 1..5 do
         assert CF.surround_context("안녕_세상", {1, i}) == %{
                  context: {:local_or_var, ~c"안녕_세상"},
                  begin: {1, 1},
@@ -486,7 +485,7 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context("안녕_세상 ", {1, 6}) == :none
+      assert CF.surround_context("안녕_세상", {1, 6}) == :none
 
       # Keywords are not local or var
       for keyword <- ~w(do end after catch else rescue fn true false nil)c do
@@ -500,38 +499,8 @@ defmodule CodeFragmentTest do
       end
     end
 
-    test "local + operator" do
-      for i <- 1..8 do
-        assert CF.surround_context("hello_wo+", {1, i}) == %{
-                 context: {:local_or_var, ~c"hello_wo"},
-                 begin: {1, 1},
-                 end: {1, 9}
-               }
-      end
-
-      assert CF.surround_context("hello_wo+", {1, 9}) == %{
-               begin: {1, 9},
-               context: {:operator, ~c"+"},
-               end: {1, 10}
-             }
-
-      for i <- 1..9 do
-        assert CF.surround_context("hello_wo +", {1, i}) == %{
-                 context: {:local_or_var, ~c"hello_wo"},
-                 begin: {1, 1},
-                 end: {1, 9}
-               }
-      end
-
-      assert CF.surround_context("hello_wo +", {1, 10}) == %{
-               begin: {1, 10},
-               context: {:operator, ~c"+"},
-               end: {1, 11}
-             }
-    end
-
     test "local call" do
-      for i <- 1..9 do
+      for i <- 1..8 do
         assert CF.surround_context("hello_wo(", {1, i}) == %{
                  context: {:local_call, ~c"hello_wo"},
                  begin: {1, 1},
@@ -539,9 +508,9 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context("hello_wo(", {1, 10}) == :none
+      assert CF.surround_context("hello_wo(", {1, 9}) == :none
 
-      for i <- 1..9 do
+      for i <- 1..8 do
         assert CF.surround_context("hello_wo (", {1, i}) == %{
                  context: {:local_call, ~c"hello_wo"},
                  begin: {1, 1},
@@ -549,10 +518,9 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context("hello_wo (", {1, 10}) == :none
+      assert CF.surround_context("hello_wo (", {1, 9}) == :none
-      assert CF.surround_context("hello_wo (", {1, 11}) == :none
 
-      for i <- 1..7 do
+      for i <- 1..6 do
         assert CF.surround_context("hello!(", {1, i}) == %{
                  context: {:local_call, ~c"hello!"},
                  begin: {1, 1},
@@ -560,9 +528,9 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context("hello!(", {1, 8}) == :none
+      assert CF.surround_context("hello!(", {1, 7}) == :none
 
-      for i <- 1..6 do
+      for i <- 1..5 do
         assert CF.surround_context("안녕_세상(", {1, i}) == %{
                  context: {:local_call, ~c"안녕_세상"},
                  begin: {1, 1},
@@ -570,7 +538,7 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context("안녕_세상(", {1, 7}) == :none
+      assert CF.surround_context("안녕_세상(", {1, 6}) == :none
     end
 
     test "local arity" do
@@ -698,7 +666,7 @@ defmodule CodeFragmentTest do
     end
 
     test "alias" do
-      for i <- 1..9 do
+      for i <- 1..8 do
         assert CF.surround_context("HelloWor", {1, i}) == %{
                  context: {:alias, ~c"HelloWor"},
                  begin: {1, 1},
@@ -706,9 +674,9 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context("HelloWor ", {1, 10}) == :none
+      assert CF.surround_context("HelloWor", {1, 9}) == :none
 
-      for i <- 2..10 do
+      for i <- 2..9 do
         assert CF.surround_context(" HelloWor", {1, i}) == %{
                  context: {:alias, ~c"HelloWor"},
                  begin: {1, 2},
@@ -716,9 +684,9 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context(" HelloWor ", {1, 11}) == :none
+      assert CF.surround_context(" HelloWor", {1, 10}) == :none
 
-      for i <- 1..10 do
+      for i <- 1..9 do
         assert CF.surround_context("Hello.Wor", {1, i}) == %{
                  context: {:alias, ~c"Hello.Wor"},
                  begin: {1, 1},
@@ -726,9 +694,9 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context("Hello.Wor ", {1, 11}) == :none
+      assert CF.surround_context("Hello.Wor", {1, 10}) == :none
 
-      for i <- 1..12 do
+      for i <- 1..11 do
         assert CF.surround_context("Hello . Wor", {1, i}) == %{
                  context: {:alias, ~c"Hello.Wor"},
                  begin: {1, 1},
@@ -736,9 +704,9 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context("Hello . Wor ", {1, 13}) == :none
+      assert CF.surround_context("Hello . Wor", {1, 12}) == :none
 
-      for i <- 1..16 do
+      for i <- 1..15 do
         assert CF.surround_context("Foo . Bar . Baz", {1, i}) == %{
                  context: {:alias, ~c"Foo.Bar.Baz"},
                  begin: {1, 1},
@@ -770,7 +738,7 @@ defmodule CodeFragmentTest do
                end: {1, 11}
              }
 
-      for i <- 1..15 do
+      for i <- 1..14 do
         assert CF.surround_context("__MODULE__.Foo", {1, i}) == %{
                  context: {:alias, {:local_or_var, ~c"__MODULE__"}, ~c"Foo"},
                  begin: {1, 1},
@@ -778,7 +746,7 @@ defmodule CodeFragmentTest do
                }
       end
 
-      for i <- 1..19 do
+      for i <- 1..18 do
         assert CF.surround_context("__MODULE__.Foo.Sub", {1, i}) == %{
                  context: {:alias, {:local_or_var, ~c"__MODULE__"}, ~c"Foo.Sub"},
                  begin: {1, 1},
@@ -830,7 +798,7 @@ defmodule CodeFragmentTest do
     end
 
     test "attribute submodules" do
-      for i <- 1..10 do
+      for i <- 1..9 do
         assert CF.surround_context("@some.Foo", {1, i}) == %{
                  context: {:alias, {:module_attribute, ~c"some"}, ~c"Foo"},
                  begin: {1, 1},
@@ -838,7 +806,7 @@ defmodule CodeFragmentTest do
                }
       end
 
-      for i <- 1..14 do
+      for i <- 1..13 do
         assert CF.surround_context("@some.Foo.Sub", {1, i}) == %{
                  context: {:alias, {:module_attribute, ~c"some"}, ~c"Foo.Sub"},
                  begin: {1, 1},
@@ -921,7 +889,7 @@ defmodule CodeFragmentTest do
                end: {1, 15}
              }
 
-      for i <- 2..10 do
+      for i <- 2..9 do
         assert CF.surround_context("%HelloWor", {1, i}) == %{
                  context: {:struct, ~c"HelloWor"},
                  begin: {1, 1},
@@ -929,7 +897,7 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context("%HelloWor ", {1, 11}) == :none
+      assert CF.surround_context("%HelloWor", {1, 10}) == :none
 
       # With dot
       assert CF.surround_context("%Hello.Wor", {1, 1}) == %{
@@ -938,7 +906,7 @@ defmodule CodeFragmentTest do
                end: {1, 11}
              }
 
-      for i <- 2..11 do
+      for i <- 2..10 do
         assert CF.surround_context("%Hello.Wor", {1, i}) == %{
                  context: {:struct, ~c"Hello.Wor"},
                  begin: {1, 1},
@@ -946,7 +914,7 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context("%Hello.Wor ", {1, 12}) == :none
+      assert CF.surround_context("%Hello.Wor", {1, 11}) == :none
 
       # With spaces
       assert CF.surround_context("% Hello . Wor", {1, 1}) == %{
@@ -955,7 +923,7 @@ defmodule CodeFragmentTest do
                end: {1, 14}
              }
 
-      for i <- 2..14 do
+      for i <- 2..13 do
         assert CF.surround_context("% Hello . Wor", {1, i}) == %{
                  context: {:struct, ~c"Hello.Wor"},
                  begin: {1, 1},
@@ -963,7 +931,7 @@ defmodule CodeFragmentTest do
                }
       end
 
-      assert CF.surround_context("% Hello . Wor ", {1, 15}) == :none
+      assert CF.surround_context("% Hello . Wor", {1, 14}) == :none
     end
 
     test "module attributes" do

lib/elixir/test/elixir/code_normalizer/quoted_ast_test.exs

@@ -229,6 +229,12 @@ defmodule Code.Normalizer.QuotedASTTest do
     test "invalid block" do
       assert quoted_to_string({:__block__, [], {:bar, [], []}}) ==
                "{:__block__, [], {:bar, [], []}}"
+
+      assert quoted_to_string({:foo, [], [{:do, :ok}, :not_keyword]}) ==
+               "foo({:do, :ok}, :not_keyword)"
+
+      assert quoted_to_string({:foo, [], [[{:do, :ok}, :not_keyword]]}) ==
+               "foo([{:do, :ok}, :not_keyword])"
     end
 
     test "not in" do

lib/elixir/test/elixir/file/stream_test.exs

@@ -141,6 +141,35 @@ defmodule File.StreamTest do
         end
       end
 
+      test "supports byte offset" do
+        src = fixture_path("file.txt")
+
+        assert @node
+               |> stream!(src, [{:read_offset, 0}])
+               |> Enum.take(1) == ["FOO\n"]
+
+        assert @node
+               |> stream!(src, [{:read_offset, 1}])
+               |> Enum.take(1) == ["OO\n"]
+
+        assert @node
+               |> stream!(src, [{:read_offset, 4}])
+               |> Enum.take(1) == []
+
+        assert @node |> stream!(src, 1, [{:read_offset, 1}]) |> Enum.count() == 3
+        assert @node |> stream!(src, 1, [{:read_offset, 4}]) |> Enum.count() == 0
+      end
+
+      test "applies offset after trimming BOM" do
+        src = fixture_path("utf8_bom.txt")
+
+        assert @node
+               |> stream!(src, [:trim_bom, {:read_offset, 4}])
+               |> Enum.take(1) == ["сский\n"]
+
+        assert @node |> stream!(src, 1, [:trim_bom, {:read_offset, 4}]) |> Enum.count() == 15
+      end
+
       test "keeps BOM when raw" do
         src = fixture_path("utf8_bom.txt")
 
@@ -169,6 +198,7 @@ defmodule File.StreamTest do
 
         assert @node |> stream!(src, [:trim_bom]) |> Enum.count() == 2
         assert @node |> stream!(src, 1, [:trim_bom]) |> Enum.count() == 19
+        assert @node |> stream!(src, 2, [:trim_bom]) |> Enum.count() == 10
       end
 
       test "keeps BOM with utf8 encoding" do

lib/elixir/test/elixir/kernel/binary_test.exs

@@ -128,20 +128,6 @@ defmodule Kernel.BinaryTest do
     assert_raise ArgumentError, message, fn ->
       Code.eval_string(~s["a" <> b <> "c" = "abc"])
     end
-
-    assert_raise ArgumentError, message, fn ->
-      Code.eval_string(~s[
-        a = "a"
-        ^a <> "b" = "ab"
-      ])
-    end
-
-    assert_raise ArgumentError, message, fn ->
-      Code.eval_string(~s[
-        b = "b"
-        "a" <> ^b <> "c" = "abc"
-      ])
-    end
   end
 
   test "hex" do
@@ -269,6 +255,28 @@ defmodule Kernel.BinaryTest do
     assert <<1::size((^foo).bar)>> = <<1::5>>
   end
 
+  test "automatic size computation of matched bitsyntax variable" do
+    var = "foo"
+    <<^var::binary, rest::binary>> = "foobar"
+    assert rest == "bar"
+
+    <<^var::bytes, rest::bytes>> = "foobar"
+    assert rest == "bar"
+
+    ^var <> rest = "foobar"
+    assert rest == "bar"
+
+    var = <<0, 1>>
+    <<^var::bitstring, rest::bitstring>> = <<0, 1, 2, 3>>
+    assert rest == <<2, 3>>
+
+    <<^var::bits, rest::bits>> = <<0, 1, 2, 3>>
+    assert rest == <<2, 3>>
+
+    ^var <> rest = <<0, 1, 2, 3>>
+    assert rest == <<2, 3>>
+  end
+
   defmacro signed_16 do
     quote do
       big - signed - integer - unit(16)

lib/elixir/test/elixir/kernel/diagnostics_test.exs

@@ -53,6 +53,28 @@ defmodule Kernel.DiagnosticsTest do
              """
     end
 
+    test "same line with offset" do
+      output =
+        capture_raise(
+          """
+          [1, 2, 3, 4, 5, 6)
+          """,
+          MismatchedDelimiterError,
+          line: 3
+        )
+
+      assert output == """
+             ** (MismatchedDelimiterError) mismatched delimiter found on nofile:3:18:
+                 error: unexpected token: )
+                 │
+               3 │ [1, 2, 3, 4, 5, 6)
+                 │ │                └ mismatched closing delimiter (expected "]")
+                 │ └ unclosed delimiter
+                 │
+                 └─ nofile:3:18\
+             """
+    end
+
     test "two-line span" do
       output =
         capture_raise(
@@ -76,6 +98,30 @@ defmodule Kernel.DiagnosticsTest do
              """
     end
 
+    test "two-line span with offset" do
+      output =
+        capture_raise(
+          """
+          [a, b, c
+           d, f, g}
+          """,
+          MismatchedDelimiterError,
+          line: 3
+        )
+
+      assert output == """
+             ** (MismatchedDelimiterError) mismatched delimiter found on nofile:4:9:
+                 error: unexpected token: }
+                 │
+               3 │ [a, b, c
+                 │ └ unclosed delimiter
+               4 │  d, f, g}
+                 │         └ mismatched closing delimiter (expected "]")
+                 │
+                 └─ nofile:4:9\
+             """
+    end
+
     test "many-line span" do
       output =
         capture_raise(
@@ -103,7 +149,9 @@ defmodule Kernel.DiagnosticsTest do
                  │
                  └─ nofile:5:5\
              """
+    end
 
+    test "many-line span with offset" do
       output =
         capture_raise(
           """
@@ -111,20 +159,21 @@ defmodule Kernel.DiagnosticsTest do
             IO.inspect(2 + 2) + 2
           )
           """,
-          MismatchedDelimiterError
+          MismatchedDelimiterError,
+          line: 3
         )
 
       assert output == """
-             ** (MismatchedDelimiterError) mismatched delimiter found on nofile:3:1:
+             ** (MismatchedDelimiterError) mismatched delimiter found on nofile:5:1:
                  error: unexpected token: )
                  │
-               1 │ fn always_forget_end ->
+               3 │ fn always_forget_end ->
                  │ └ unclosed delimiter
-               2 │   IO.inspect(2 + 2) + 2
+               4 │   IO.inspect(2 + 2) + 2
-               3 │ )
+               5 │ )
                  │ └ mismatched closing delimiter (expected "end")
                  │
-                 └─ nofile:3:1\
+                 └─ nofile:5:1\
              """
     end
 
@@ -153,7 +202,7 @@ defmodule Kernel.DiagnosticsTest do
              """
     end
 
-    test "trim inbetween lines if too many" do
+    test "trim in between lines if too many" do
       output =
         capture_raise(
           """
@@ -288,18 +337,142 @@ defmodule Kernel.DiagnosticsTest do
     end
   end
 
+  describe "token missing error" do
+    test "missing parens terminator" do
+      output =
+        capture_raise(
+          """
+          my_numbers = [1, 2, 3, 4, 5, 6
+          IO.inspect(my_numbers)
+          """,
+          TokenMissingError
+        )
+
+      assert output == """
+             ** (TokenMissingError) token missing on nofile:2:23:
+                 error: missing terminator: ]
+                 │
+               1 │ my_numbers = [1, 2, 3, 4, 5, 6
+                 │              └ unclosed delimiter
+               2 │ IO.inspect(my_numbers)
+                 │                       └ missing closing delimiter (expected "]")
+                 │
+                 └─ nofile:2:23\
+             """
+    end
+
+    test "shows in between lines if EOL is not far below" do
+      output =
+        capture_raise(
+          """
+          my_numbers = [1, 2, 3, 4, 5, 6
+          my_numbers
+          |> Enum.map(&(&1 + 1))
+          |> Enum.map(&(&1 * &1))
+          |> IO.inspect()
+          """,
+          TokenMissingError
+        )
+
+      assert output == """
+             ** (TokenMissingError) token missing on nofile:5:16:
+                 error: missing terminator: ]
+                 │
+               1 │ my_numbers = [1, 2, 3, 4, 5, 6
+                 │              └ unclosed delimiter
+               2 │ my_numbers
+               3 │ |> Enum.map(&(&1 + 1))
+               4 │ |> Enum.map(&(&1 * &1))
+               5 │ |> IO.inspect()
+                 │                └ missing closing delimiter (expected "]")
+                 │
+                 └─ nofile:5:16\
+             """
+    end
+
+    test "trims lines" do
+      output =
+        capture_raise(
+          """
+          my_numbers = (1, 2, 3, 4, 5, 6
+
+
+
+
+
+
+          IO.inspect(my_numbers)
+          """,
+          TokenMissingError
+        )
+
+      assert output == """
+             ** (TokenMissingError) token missing on nofile:8:23:
+                 error: missing terminator: )
+                 │
+               1 │ my_numbers = (1, 2, 3, 4, 5, 6
+                 │              └ unclosed delimiter
+              ...
+               8 │ IO.inspect(my_numbers)
+                 │                       └ missing closing delimiter (expected ")")
+                 │
+                 └─ nofile:8:23\
+             """
+    end
+
+    test "shows the last non-empty line of a file" do
+      output =
+        capture_raise(
+          """
+          my_numbers = {1, 2, 3, 4, 5, 6
+          IO.inspect(my_numbers)
+
+
+
+
+          """,
+          TokenMissingError
+        )
+
+      assert output == """
+             ** (TokenMissingError) token missing on nofile:2:23:
+                 error: missing terminator: }
+                 │
+               1 │ my_numbers = {1, 2, 3, 4, 5, 6
+                 │              └ unclosed delimiter
+               2 │ IO.inspect(my_numbers)
+                 │                       └ missing closing delimiter (expected "}")
+                 │
+                 └─ nofile:2:23\
+             """
+    end
+
+    test "supports unicode" do
+      output =
+        capture_raise(
+          """
+          my_emojis = [1, 2, 3, 4 # ⚗️
+          IO.inspect(my_numbers)
+          """,
+          TokenMissingError
+        )
+
+      assert output == """
+             ** (TokenMissingError) token missing on nofile:2:23:
+                 error: missing terminator: ]
+                 │
+               1 │ my_emojis = [1, 2, 3, 4 # ⚗️
+                 │             └ unclosed delimiter
+               2 │ IO.inspect(my_numbers)
+                 │                       └ missing closing delimiter (expected "]")
+                 │
+                 └─ nofile:2:23\
+             """
+    end
+  end
+
   describe "compile-time exceptions" do
     test "SyntaxError (snippet)" do
-      expected = """
-      ** (SyntaxError) invalid syntax found on nofile:1:17:
-          error: syntax error before: '*'
-          │
-        1 │ [1, 2, 3, 4, 5, *]
-          │                 ^
-          │
-          └─ nofile:1:17\
-      """
-
       output =
         capture_raise(
           """
@@ -308,20 +481,39 @@ defmodule Kernel.DiagnosticsTest do
           SyntaxError
         )
 
-      assert output == expected
+      assert output == """
+             ** (SyntaxError) invalid syntax found on nofile:1:17:
+                 error: syntax error before: '*'
+                 │
+               1 │ [1, 2, 3, 4, 5, *]
+                 │                 ^
+                 │
+                 └─ nofile:1:17\
+             """
+    end
+
+    test "SyntaxError (snippet) with offset" do
+      output =
+        capture_raise(
+          """
+          [1, 2, 3, 4, 5, *]
+          """,
+          SyntaxError,
+          line: 3
+        )
+
+      assert output == """
+             ** (SyntaxError) invalid syntax found on nofile:3:17:
+                 error: syntax error before: '*'
+                 │
+               3 │ [1, 2, 3, 4, 5, *]
+                 │                 ^
+                 │
+                 └─ nofile:3:17\
+             """
     end
 
     test "TokenMissingError (snippet)" do
-      expected = """
-      ** (TokenMissingError) token missing on nofile:1:4:
-          error: syntax error: expression is incomplete
-          │
-        1 │ 1 +
-          │    ^
-          │
-          └─ nofile:1:4\
-      """
-
       output =
         capture_raise(
           """
@@ -330,14 +522,48 @@ defmodule Kernel.DiagnosticsTest do
           TokenMissingError
         )
 
-      assert output == expected
+      assert output == """
+             ** (TokenMissingError) token missing on nofile:1:4:
+                 error: syntax error: expression is incomplete
+                 │
+               1 │ 1 +
+                 │    ^
+                 │
+                 └─ nofile:1:4\
+             """
     end
 
-    test "TokenMissingError (no snippet)" do
+    test "TokenMissingError (snippet) with offset" do
+      output =
+        capture_raise(
+          """
+          1 +
+          """,
+          TokenMissingError,
+          line: 3
+        )
+
+      assert output == """
+             ** (TokenMissingError) token missing on nofile:3:4:
+                 error: syntax error: expression is incomplete
+                 │
+               3 │ 1 +
+                 │    ^
+                 │
+                 └─ nofile:3:4\
+             """
+    end
+
+    test "TokenMissingError (unclosed delimiter)" do
       expected = """
-      ** (TokenMissingError) token missing on nofile:2:1:
+      ** (TokenMissingError) token missing on nofile:1:5:
-         error: missing terminator: end (for "fn" starting at line 1)
+          error: missing terminator: end
-         └─ nofile:2:1\
+          │
+        1 │ fn a
+          │ │   └ missing closing delimiter (expected "end")
+          │ └ unclosed delimiter
+          │
+          └─ nofile:1:5\
       """
 
       output =
@@ -419,7 +645,7 @@ defmodule Kernel.DiagnosticsTest do
           1 -
           """,
           TokenMissingError,
-          fake_stacktrace
+          stacktrace: fake_stacktrace
         )
 
       assert output == expected
@@ -448,7 +674,7 @@ defmodule Kernel.DiagnosticsTest do
           1 -
           """,
           TokenMissingError,
-          fake_stacktrace
+          stacktrace: fake_stacktrace
         )
 
       assert output == expected
@@ -544,101 +770,41 @@ defmodule Kernel.DiagnosticsTest do
     end
 
     @tag :tmp_dir
-    test "long message (file)", %{tmp_dir: tmp_dir} do
+    test "IO.warn file+line+column", %{tmp_dir: tmp_dir} do
-      path = make_relative_tmp(tmp_dir, "long-warning.ex")
+      path = make_relative_tmp(tmp_dir, "io-warn-file-line-column.ex")
 
       source = """
-      defmodule Sample do
+      IO.warn("oops\\nmulti\\nline", file: __ENV__.file, line: __ENV__.line, column: 4)
-        @file "#{path}"
-
-        def atom_case do
-          v = "bc"
-
-          case v do
-            _ when is_atom(v) -> :ok
-            _ -> :fail
-          end
-        end
-      end
       """
 
       File.write!(path, source)
 
       expected = """
-          warning: incompatible types:
+          warning: oops
-
+          multi
-              binary() !~ atom()
+          line
-
-          in expression:
-
-              # #{path}:8
-              is_atom(v)
-
-          where "v" was given the type binary() in:
-
-              # #{path}:5
-              v = "bc"
-
-          where "v" was given the type atom() in:
-
-              # #{path}:8
-              is_atom(v)
-
-          Conflict found at
           │
-        8 │       _ when is_atom(v) -> :ok
+        1 │ IO.warn("oops\\nmulti\\nline", file: __ENV__.file, line: __ENV__.line, column: 4)
-          │              ~
+          │ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           │
-          └─ #{path}:8:14: Sample.atom_case/0
+          └─ tmp\
       """
 
-      assert capture_eval(source) =~ expected
+      assert capture_io(:stderr, fn -> Code.eval_file(path) end) =~ expected
-    after
-      purge(Sample)
     end
 
-    test "long message (nofile)" do
+    test "IO.warn with missing data" do
-      source = """
+      assert capture_eval("""
-      defmodule Sample do
+             IO.warn("oops-bad", file: #{inspect(__ENV__.file)}, line: 3, column: nil)
-        def atom_case do
+             """) =~ "warning: oops-bad"
-          v = "bc"
 
-          case v do
+      assert capture_eval("""
-            _ when is_atom(v) -> :ok
+             IO.warn("oops-bad", file: #{inspect(__ENV__.file)}, line: nil)
-            _ -> :fail
+             """) =~ "oops-bad"
-          end
-        end
-      end
-      """
 
-      expected = """
+      assert capture_eval("""
-      warning: incompatible types:
+             IO.warn("oops-bad", file: nil)
-
+             """) =~ "oops-bad"
-          binary() !~ atom()
-
-      in expression:
-
-          # nofile:6
-          is_atom(v)
-
-      where "v" was given the type binary() in:
-
-          # nofile:3
-          v = "bc"
-
-      where "v" was given the type atom() in:
-
-          # nofile:6
-          is_atom(v)
-
-      Conflict found at
-      └─ nofile:6:14: Sample.atom_case/0
-
-      """
-
-      assert capture_eval(source) =~ expected
-    after
-      purge(Sample)
     end
 
     @tag :tmp_dir
@@ -1164,14 +1330,16 @@ defmodule Kernel.DiagnosticsTest do
     end)
   end
 
-  defp capture_raise(source, exception, mock_stacktrace \\ []) do
+  defp capture_raise(source, exception, opts \\ []) do
+    {stacktrace, opts} = Keyword.pop(opts, :stacktrace, [])
+
     e =
       assert_raise exception, fn ->
-        ast = Code.string_to_quoted!(source, columns: true)
+        ast = Code.string_to_quoted!(source, [columns: true] ++ opts)
         Code.eval_quoted(ast)
       end
 
-    Exception.format(:error, e, mock_stacktrace)
+    Exception.format(:error, e, stacktrace)
   end
 
   defp purge(module) when is_atom(module) do

lib/elixir/test/elixir/kernel/errors_test.exs

@@ -473,7 +473,7 @@ defmodule Kernel.ErrorsTest do
       assert_eval_raise TokenMissingError,
                         [
                           "nofile:1:5:",
-                          ~r/missing terminator: end \(for "fn" starting at line 1\)/
+                          ~r/missing terminator: end/
                         ],
                         ~c"fn 1"
 
@@ -490,16 +490,16 @@ defmodule Kernel.ErrorsTest do
     assert_eval_raise TokenMissingError,
                       [
                         "nofile:1:25:",
-                        "missing terminator: end (for \"do\" starting at line 1)",
+                        "missing terminator: end",
                         "defmodule ShowSnippet do\n",
-                        "^"
+                        "└ unclosed delimiter"
                       ],
                       ~c"defmodule ShowSnippet do"
   end
 
   test "don't show snippet when error line is empty" do
     assert_eval_raise TokenMissingError,
-                      ["nofile:3:1:", "missing terminator: end (for \"do\" starting at line 1)"],
+                      ["nofile:1:25:", "missing terminator: end"],
                       ~c"defmodule ShowSnippet do\n\n"
   end
 

lib/elixir/test/elixir/kernel/parser_test.exs

@@ -494,7 +494,7 @@ defmodule Kernel.ParserTest do
   describe "token missing errors" do
     test "missing paren" do
       assert_token_missing(
-        ["nofile:1:9:", "missing terminator: ) (for \"(\" starting at line 1)"],
+        ["nofile:1:9:", "missing terminator: )"],
         ~c"case 1 ("
       )
     end
@@ -549,7 +549,7 @@ defmodule Kernel.ParserTest do
 
     test "missing end" do
       assert_token_missing(
-        ["nofile:1:9:", "missing terminator: end \(for \"do\" starting at line 1\)"],
+        ["nofile:1:9:", "missing terminator: end"],
         ~c"foo do 1"
       )
 
@@ -1136,7 +1136,12 @@ defmodule Kernel.ParserTest do
 
       assert_syntax_error(
         ["atom length must be less than system limit: "],
-        ~s[:"#{atom}"]
+        ~s{:"#{atom}"}
+      )
+
+      assert_syntax_error(
+        ["atom length must be less than system limit: "],
+        ~s{["#{atom}": 123]}
       )
     end
   end

lib/elixir/test/elixir/kernel/warning_test.exs

@@ -1175,18 +1175,6 @@ defmodule Kernel.WarningTest do
     purge(Sample)
   end
 
-  test "parens on nullary remote call" do
-    assert_warn_eval(
-      [
-        "nofile:1:8",
-        "parentheses are required for function calls with no arguments, got: System.version"
-      ],
-      "System.version"
-    )
-  after
-    purge(Sample)
-  end
-
   test "parens with module attribute" do
     assert_warn_eval(
       [
@@ -1652,22 +1640,39 @@ defmodule Kernel.WarningTest do
   end
 
   test "reserved doc metadata keys" do
-    output =
+    {output, diagnostics} =
-      capture_eval("""
+      Code.with_diagnostics([log: true], fn ->
-      defmodule Sample do
+        capture_eval("""
-        @typedoc opaque: false
+        defmodule Sample do
-        @type t :: binary
+          @typedoc opaque: false
+          @type t :: binary
 
-        @doc defaults: 3, since: "1.2.3"
+          @doc defaults: 3, since: "1.2.3"
-        def foo(a), do: a
+          def foo(a), do: a
-      end
+        end
-      """)
+        """)
+      end)
 
     assert output =~ "ignoring reserved documentation metadata key: :opaque"
     assert output =~ "nofile:2: "
     assert output =~ "ignoring reserved documentation metadata key: :defaults"
     assert output =~ "nofile:5: "
     refute output =~ ":since"
+
+    assert [
+             %{
+               message: "ignoring reserved documentation metadata key: :opaque",
+               position: 2,
+               file: "nofile",
+               severity: :warning
+             },
+             %{
+               message: "ignoring reserved documentation metadata key: :defaults",
+               position: 5,
+               file: "nofile",
+               severity: :warning
+             }
+           ] = diagnostics
   after
     purge(Sample)
   end

lib/elixir/test/elixir/module/types/integration_test.exs

@@ -695,6 +695,29 @@ defmodule Module.Types.IntegrationTest do
 
       assert_warnings(files, warnings)
     end
+
+    test "reports unquote functions" do
+      files = %{
+        "a.ex" => """
+        defmodule A do
+          @deprecated "oops"
+          def a, do: :ok
+        end
+        """,
+        "b.ex" => """
+        defmodule B do
+          def b, do: unquote(&A.a/0)
+        end
+        """
+      }
+
+      warnings = [
+        "A.a/0 is deprecated. oops",
+        "b.ex: B.b/0"
+      ]
+
+      assert_warnings(files, warnings)
+    end
   end
 
   defp assert_warnings(files, expected) when is_binary(expected) do

lib/elixir/test/elixir/protocol_test.exs

@@ -154,8 +154,12 @@ defmodule ProtocolTest do
   end
 
   test "protocol defines callbacks" do
-    assert [{:type, 13, :fun, args}] = get_callbacks(@sample_binary, :ok, 1)
+    assert [{:type, {13, 19}, :fun, args}] = get_callbacks(@sample_binary, :ok, 1)
-    assert args == [{:type, 13, :product, [{:user_type, 13, :t, []}]}, {:type, 13, :boolean, []}]
+
+    assert args == [
+             {:type, {13, 19}, :product, [{:user_type, {13, 16}, :t, []}]},
+             {:type, {13, 22}, :boolean, []}
+           ]
 
     assert [{:type, 23, :fun, args}] = get_callbacks(@with_any_binary, :ok, 1)
     assert args == [{:type, 23, :product, [{:user_type, 23, :t, []}]}, {:type, 23, :term, []}]

lib/elixir/test/elixir/string_test.exs

@@ -826,6 +826,33 @@ defmodule StringTest do
     refute String.valid?("asdasdasd" <> <<0xFFFF::16>>, :fast_ascii)
   end
 
+  test "replace_invalid" do
+    assert String.replace_invalid("") === ""
+    assert String.replace_invalid(<<0xFF>>) === "�"
+    assert String.replace_invalid(<<0xFF, 0xFF, 0xFF>>) === "���"
+
+    # Valid ASCII
+    assert String.replace_invalid("hello") === "hello"
+
+    # Valid UTF-8
+    assert String.replace_invalid("こんにちは") === "こんにちは"
+
+    # 2/3 byte truncated "ề"
+    assert String.replace_invalid(<<225, 187>>) === "�"
+    assert String.replace_invalid("nem rán b" <> <<225, 187>> <> " bề") === "nem rán b� bề"
+
+    # 2/4 byte truncated "😔"
+    assert String.replace_invalid(<<240, 159>>) === "�"
+    assert String.replace_invalid("It's so over " <> <<240, 159>>) === "It's so over �"
+
+    # 3/4 byte truncated "😃"
+    assert String.replace_invalid(<<240, 159, 152>>) === "�"
+    assert String.replace_invalid("We're so back " <> <<240, 159, 152>>) === "We're so back �"
+
+    # 3 byte overlong "e"
+    assert String.replace_invalid(<<0b11100000, 0b10000001, 0b10100101>>) === "���"
+  end
+
   test "chunk/2 with :valid trait" do
     assert String.chunk("", :valid) == []
 

lib/elixir/test/elixir/typespec_test.exs

@@ -1582,23 +1582,29 @@ defmodule TypespecTest do
     """)
 
     [type: type] = types(:typespec_test_mod)
-    line = 5
 
     assert Code.Typespec.type_to_quoted(type) ==
-             {:"::", [], [{:t, [], [{:x, [line: line], nil}]}, [{:x, [line: line], nil}]]}
+             {:"::", [],
+              [
+                {:t, [], [{:x, meta(5, 9), nil}]},
+                [{:x, meta(5, 20), nil}]
+              ]}
 
     [{{:f, 1}, [spec]}] = specs(:typespec_test_mod)
-    line = 7
 
     assert Code.Typespec.spec_to_quoted(:f, spec) ==
-             {:when, [line: line],
+             {:when, meta(7, 8),
               [
-                {:"::", [line: line],
+                {:"::", meta(7, 8),
-                 [{:f, [line: line], [{:x, [line: line], nil}]}, {:x, [line: line], nil}]},
+                 [{:f, meta(7, 8), [{:x, meta(7, 9), nil}]}, {:x, meta(7, 15), nil}]},
-                [x: {:var, [line: line], nil}]
+                [x: {:var, meta(7, 8), nil}]
               ]}
   end
 
+  defp meta(line, column) do
+    [line: line, column: column]
+  end
+
   defp erlc(context, module, code) do
     dir = context.tmp_dir
 

lib/ex_unit/lib/ex_unit.ex

@@ -148,6 +148,16 @@ defmodule ExUnit do
   end
 
   defmodule TimeoutError do
+    @moduledoc """
+    Exception raised when a test times out.
+    """
+
+    @typedoc since: "1.16.0"
+    @type t :: %__MODULE__{
+            timeout: non_neg_integer,
+            type: String.t()
+          }
+
     defexception [:timeout, :type]
 
     @impl true

lib/ex_unit/lib/ex_unit/assertions.ex

@@ -1,10 +1,23 @@
 defmodule ExUnit.AssertionError do
   @moduledoc """
   Raised to signal an assertion error.
+
+  This is used by macros such as `ExUnit.Assertions.assert/1`.
   """
 
   @no_value :ex_unit_no_meaningful_value
 
+  @typedoc since: "1.16.0"
+  @type t :: %__MODULE__{
+          left: any,
+          right: any,
+          message: any,
+          expr: any,
+          args: any,
+          doctest: any,
+          context: any
+        }
+
   defexception left: @no_value,
                right: @no_value,
                message: @no_value,
@@ -16,6 +29,7 @@ defmodule ExUnit.AssertionError do
   @doc """
   Indicates no meaningful value for a field.
   """
+  @spec no_value :: atom
   def no_value do
     @no_value
   end
@@ -31,6 +45,11 @@ defmodule ExUnit.MultiError do
   Raised to signal multiple errors happened in a test case.
   """
 
+  @typedoc since: "1.16.0"
+  @type t :: %__MODULE__{
+          errors: [{Exception.kind(), any, Exception.stacktrace()}]
+        }
+
   defexception errors: []
 
   @impl true

lib/ex_unit/lib/ex_unit/case.ex

@@ -1,8 +1,23 @@
 defmodule ExUnit.DuplicateTestError do
+  @moduledoc """
+  Exception raised to indicate two or more tests with the same name.
+  """
+
+  @typedoc since: "1.16.0"
+  @type t :: %__MODULE__{message: String.t()}
+
   defexception [:message]
 end
 
 defmodule ExUnit.DuplicateDescribeError do
+  @moduledoc """
+  Exception raised to indicate two or more `describe` blocks with
+  the same name.
+  """
+
+  @typedoc since: "1.16.0"
+  @type t :: %__MODULE__{message: String.t()}
+
   defexception [:message]
 end
 

lib/ex_unit/lib/ex_unit/doc_test.ex

@@ -152,6 +152,12 @@ defmodule ExUnit.DocTest do
   @opaque_type_regex ~r/#[\w\.]+</
 
   defmodule Error do
+    @moduledoc """
+    Exception raised when there's an error with the syntax or semantics of a doctest.
+    """
+
+    @typedoc since: "1.16.0"
+    @type t :: %__MODULE__{message: String.t()}
     defexception [:message]
 
     @impl true

lib/ex_unit/lib/ex_unit/formatter.ex

@@ -67,6 +67,97 @@ defmodule ExUnit.Formatter do
           load: pos_integer | nil
         }
 
+  @typedoc """
+  Key passed to a formatter callback to format a diff.
+
+  See `t:formatter_callback/0`.
+  """
+  @typedoc since: "1.16.0"
+  @type formatter_callback_diff_key ::
+          :diff_delete
+          | :diff_delete_whitespace
+          | :diff_insert
+          | :diff_insert_whitespace
+
+  @typedoc """
+  Key passed to a formatter callback to format information.
+
+  See `t:formatter_callback/0`.
+  """
+  @typedoc since: "1.16.0"
+  @type formatter_callback_info_key ::
+          :extra_info
+          | :error_info
+          | :test_module_info
+          | :test_info
+          | :location_info
+          | :stacktrace_info
+          | :blame_diff
+
+  @typedoc """
+  A function that this module calls to format various things.
+
+  You can pass this functions to various functions in this module, and use it
+  to customize the formatting of the output. For example, ExUnit's CLI formatter
+  uses this callback to colorize output.
+
+  ## Keys
+
+  The possible keys are:
+
+    * `:diff_enabled?` - whether diffing is enabled. It receives a boolean
+      indicating whether diffing is enabled by default and returns a boolean
+      indicating whether diffing should be enabled for the current test.
+
+    * `:diff_delete` and `:diff_delete_whitespace` - Should format a diff deletion,
+      with or without whitespace respectively.
+
+    * `:diff_insert` and `:diff_insert_whitespace` - Should format a diff insertion,
+      with or without whitespace respectively.
+
+    * `:extra_info` - Should format extra information, such as the `"code: "` label
+      that precedes code to show.
+
+    * `:error_info` - Should format error information.
+
+    * `:error_info` - Should format error information.
+
+    * `:test_module_info` - Should format test module information. The message returned
+    when this key is passed precedes messages such as `"failure on setup_all callback [...]"`.
+
+    * `:test_info` - Should format test information.
+
+    * `:location_info` - Should format test location information.
+
+    * `:stacktrace_info` - Should format stacktrace information.
+
+    * `:blame_diff` - Should format a string of code.
+
+  ## Examples
+
+  For example, to format errors as *red strings* and everything else as is, you could define
+  a formatter callback function like this:
+
+      formatter_callback = fn
+        :error_info, msg -> [:red, msg, :reset] |> IO.ANSI.format() |> IO.iodata_to_binary()
+        _key, value -> value
+      end
+
+  """
+  @typedoc since: "1.16.0"
+  @type formatter_callback ::
+          (:diff_enabled?, boolean -> boolean)
+          | (formatter_callback_diff_key, Inspect.Algebra.t() -> Inspect.Algebra.t())
+          | (formatter_callback_info_key, String.t() -> String.t())
+
+  @typedoc """
+  Width for formatting.
+
+  For example, see `format_assertion_diff/4`.
+  """
+  @typedoc since: "1.16.0"
+  @type width :: non_neg_integer | :infinity
+
   import Exception, only: [format_stacktrace_entry: 1, format_file_line: 3]
 
   alias ExUnit.Diff
@@ -152,9 +243,26 @@ defmodule ExUnit.Formatter do
     end
   end
 
-  @doc """
+  @doc ~S"""
-  Receives a test and formats its failure.
+  Receives a test and formats its failures.
+
+  ## Examples
+
+      iex> failure = {:error, catch_error(raise "oops"), _stacktrace = []}
+      iex> formatter_cb = fn _key, value -> value end
+      iex> test = %ExUnit.Test{name: :"it works", module: MyTest, tags: %{file: "file.ex", line: 7}}
+      iex> format_test_failure(test, [failure], 1, 80, formatter_cb)
+      "  1) it works (MyTest)\n     file.ex:7\n     ** (RuntimeError) oops\n"
+
   """
+  @spec format_test_failure(
+          test,
+          [failure],
+          non_neg_integer,
+          width,
+          formatter_callback
+        ) :: String.t()
+        when failure: {atom, term, Exception.stacktrace()}
   def format_test_failure(test, failures, counter, width, formatter) do
     %ExUnit.Test{name: name, module: module, tags: tags} = test
 
@@ -174,9 +282,26 @@ defmodule ExUnit.Formatter do
     format_test_all_failure(test_case, failures, counter, width, formatter)
   end
 
-  @doc """
+  @doc ~S"""
   Receives a test module and formats its failure.
+
+  ## Examples
+
+      iex> failure = {:error, catch_error(raise "oops"), _stacktrace = []}
+      iex> formatter_cb = fn _key, value -> value end
+      iex> test_module = %ExUnit.TestModule{name: Hello}
+      iex> format_test_all_failure(test_module, [failure], 1, 80, formatter_cb)
+      "  1) Hello: failure on setup_all callback, all tests have been invalidated\n     ** (RuntimeError) oops\n"
+
   """
+  @spec format_test_all_failure(
+          ExUnit.TestModule.t(),
+          [failure],
+          non_neg_integer,
+          width,
+          formatter_callback
+        ) :: String.t()
+        when failure: {atom, term, Exception.stacktrace()}
   def format_test_all_failure(test_module, failures, counter, width, formatter) do
     name = test_module.name
 
@@ -279,7 +404,25 @@ defmodule ExUnit.Formatter do
   It expects the assertion error, the `padding_size`
   for formatted content, the width (may be `:infinity`),
   and the formatter callback function.
+
+  ## Examples
+
+      iex> error = assert_raise ExUnit.AssertionError, fn -> assert [1, 2] == [1, 3] end
+      iex> formatter_cb = fn
+      ...>   :diff_enabled?, _ -> true
+      ...>   _key, value -> value
+      ...> end
+      iex> keyword = format_assertion_diff(error, 5, 80, formatter_cb)
+      iex> for {key, val} <- keyword, do: {key, IO.iodata_to_binary(val)}
+      [left: "[1, 2]", right: "[1, 3]"]
+
   """
+  @spec format_assertion_diff(
+          ExUnit.AssertionError.t(),
+          non_neg_integer,
+          width,
+          formatter_callback
+        ) :: keyword
   def format_assertion_diff(assert_error, padding_size, width, formatter)
 
   def format_assertion_diff(%ExUnit.AssertionError{context: {:mailbox, _pins, []}}, _, _, _) do

lib/ex_unit/test/ex_unit/doc_test_test.exs

@@ -790,15 +790,16 @@ defmodule ExUnit.DocTestTest do
     line = starting_line + 35
 
     assert output =~ """
-             6) doctest ExUnit.DocTestTest.Invalid.misplaced_opaque_type/0 (6) (ExUnit.DocTestTest.InvalidCompiled)
+           6) doctest ExUnit.DocTestTest.Invalid.misplaced_opaque_type/0 (6) (ExUnit.DocTestTest.InvalidCompiled)
                 test/ex_unit/doc_test_test.exs:#{doctest_line}
-                Doctest did not compile, got: (TokenMissingError) token missing on test/ex_unit/doc_test_test.exs:#{line}:7:
+                Doctest did not compile, got: (TokenMissingError) token missing on test/ex_unit/doc_test_test.exs:#{line}:20:
-                     error: missing terminator: } (for "{" starting at line #{line})
+                     error: missing terminator: }
                      │
-                 #{line} │ {:ok, #Inspect<[]>}
+                 228 │ {:ok, #Inspect<[]>}
-                     │       ^
+                     │ │                  └ missing closing delimiter (expected "}")
+                     │ └ unclosed delimiter
                      │
-                     └─ test/ex_unit/doc_test_test.exs:#{line}:7
+                     └─ test/ex_unit/doc_test_test.exs:#{line}:20
                 If you are planning to assert on the result of an iex> expression which contains a value inspected as #Name<...>, please make sure the inspected value is placed at the beginning of the expression; otherwise Elixir will treat it as a comment due to the leading sign #.
                 doctest:
                   iex> {:ok, :oops}

lib/iex/lib/iex/helpers.ex

@@ -1476,7 +1476,8 @@ defmodule IEx.Helpers do
 
   defp get_beam_and_path(module) do
     with {^module, beam, filename} <- :code.get_object_code(module),
-         {:ok, ^module} <- beam |> :beam_lib.info() |> Keyword.fetch(:module) do
+         info_pairs when is_list(info_pairs) <- :beam_lib.info(beam),
+         {:ok, ^module} <- Keyword.fetch(info_pairs, :module) do
       {beam, filename}
     else
       _ -> :error

lib/iex/lib/iex/info.ex

@@ -89,7 +89,7 @@ defimpl IEx.Info, for: Atom do
 
       {^atom, beam, _path} ->
         info = :beam_lib.info(beam)
-        Keyword.fetch(info, :module) == {:ok, atom}
+        is_list(info) and Keyword.fetch(info, :module) == {:ok, atom}
     end
   end
 

lib/iex/lib/iex/pry.ex

@@ -519,7 +519,7 @@ defmodule IEx.Pry do
     next_binding = binding(expr, binding)
     {min_line, max_line} = line_range(expr, line)
 
-    if force? or (min_line > line and min_line != :infinity) do
+    if force? or min_line > line do
       pry_var = next_var(version)
       pry_binding = Map.to_list(binding)
       pry_opts = [line: min_line] ++ opts
@@ -545,7 +545,9 @@ defmodule IEx.Pry do
   end
 
   defp line_range(ast, line) do
-    {_, min_max} =
+    # We want min_line to start from infinity because
+    # if it starts from line it will always just return line.
+    {_, {min, max}} =
       Macro.prewalk(ast, {:infinity, line}, fn
         {_, meta, _} = ast, {min_line, max_line} when is_list(meta) ->
           line = meta[:line]
@@ -560,7 +562,7 @@ defmodule IEx.Pry do
           {ast, acc}
       end)
 
-    min_max
+    if min == :infinity, do: {line, max}, else: {min, max}
   end
 
   defp binding(ast, binding) do

lib/logger/lib/logger.ex

@@ -13,16 +13,16 @@ defmodule Logger do
 
     * Supports both message-based and structural logging.
 
+    * Integrate with Erlang's [`:logger`](`:logger`) and
+      support custom filters and handlers.
+
     * Formats and truncates messages on the client
-      to avoid clogging `Logger` backends.
+      to avoid clogging `Logger` handlers.
 
     * Alternates between sync and async modes to remain
       performant when required but also apply back-pressure
       when under stress.
 
-    * Support for custom filters and handlers as provided by
-      Erlang's `:logger`.
-
     * Allows overriding the logging level for a specific module,
       application or process.
 
@@ -65,7 +65,7 @@ defmodule Logger do
 
   For example, `:info` takes precedence over `:debug`. If your log
   level is set to `:info`, then all `:info`, `:notice` and above will
-  be passed to backends. If your log level is set to `:alert`, only
+  be passed to handlers. If your log level is set to `:alert`, only
   `:alert` and `:emergency` will be printed.
 
   ## Message
@@ -126,8 +126,8 @@ defmodule Logger do
     * `:crash_reason` - a two-element tuple with the throw/error/exit reason
       as first argument and the stacktrace as second. A throw will always be
       `{:nocatch, term}`. An error is always an `Exception` struct. All other
-      entries are exits. The console backend ignores this metadata by default
+      entries are exits. The default formatter ignores this metadata by default
-      but it can be useful to other backends, such as the ones that report
+      but it can be useful to certain handlers, such as the ones that report
       errors to third-party services
 
   There are two special metadata keys, `:module` and `:function`, which
@@ -275,8 +275,8 @@ defmodule Logger do
       Remember that if you want to purge log calls from a dependency, the
       dependency must be recompiled.
 
-  For example, to configure the `:backends` and purge all calls that happen
+  For example, to purge all calls that happen at compile time with level
-  at compile time with level lower than `:info` in a `config/config.exs` file:
+  lower than `:info` in a `config/config.exs` file:
 
       config :logger,
         compile_time_purge_matching: [
@@ -300,7 +300,7 @@ defmodule Logger do
 
     * `:level` - the logging level. Attempting to log any message
       with severity less than the configured level will simply
-      cause the message to be ignored. Keep in mind that each backend
+      cause the message to be ignored. Keep in mind that each handler
       may have its specific level, too. In addition to levels mentioned
       above it also supports 2 "meta-levels":
 
@@ -397,7 +397,7 @@ defmodule Logger do
   Prior to Elixir v1.15, custom logging could be achieved with Logger
   backends. The main API for writing Logger backends have been moved to
   the [`:logger_backends`](https://github.com/elixir-lang/logger_backends)
-  project. However, the backends are still part of Elixir for backwards
+  project. However, the backends API are still part of Elixir for backwards
   compatibility.
 
   Important remarks:
@@ -428,9 +428,12 @@ defmodule Logger do
       Backends, you can still set `backends: [Logger.Backends.Console]` and place
       the configuration under `config :logger, Logger.Backends.Console`. Although
       consider using the [`:logger_backends`](https://github.com/elixir-lang/logger_backends)
-      project in such case, as `Logger.Backends.Console` itself will be deprecated
+      project in such cases, as `Logger.Backends.Console` itself will be deprecated
       in future releases
 
+    * `Logger.Backends` only receive `:debug`, `:info`, `:warning`, and `:error`
+      messages. `:notice` maps to `:info`. `:warn` amps to `:warnings`.
+      All others map to `:error`
   """
 
   @type level ::
@@ -442,6 +445,13 @@ defmodule Logger do
   @levels [:error, :info, :debug] ++ @new_erlang_levels
   @metadata :logger_level
 
+  @doc ~S"""
+  Returns all the available levels.
+  """
+  @doc since: "1.16.0"
+  @spec levels() :: [level(), ...]
+  def levels(), do: @levels
+
   @doc ~S"""
   Returns the default formatter used by Logger.
 
@@ -933,37 +943,21 @@ defmodule Logger do
 
   defp add_elixir_domain(metadata), do: Map.put(metadata, :domain, [:elixir])
 
-  translations = %{
-    emergency: :error,
-    alert: :error,
-    critical: :error,
-    notice: :info
-  }
-
   for level <- @levels do
     report = [something: :reported, this: level]
-
+    metadata = [user_id: 42, request_id: "xU32kFa"]
-    extra =
+    article = if level in [:info, :error, :alert, :emergency], do: "an", else: "a"
-      if translation = translations[level] do
-        """
-
-
-        This is reported as \"#{translation}\" in Elixir's
-        logger backends for backwards compatibility reasons.
-
-        """
-      end
 
     @doc """
-    Logs a #{level} message.
+    Logs #{article} #{level} message.
 
-    Returns `:ok`.#{extra}
+    Returns `:ok`.
 
     ## Examples
 
     Logging a message (string or iodata):
 
-        Logger.#{level}("this is a #{level} message")
+        Logger.#{level}("this is #{article} #{level} message")
 
     Report message (maps or keywords):
 
@@ -973,6 +967,14 @@ defmodule Logger do
         # as map
         Logger.#{level}(#{inspect(Map.new(report))})
 
+    Report message with metadata (maps or keywords):
+
+        # as a keyword list
+        Logger.#{level}("this is #{article} #{level} message", #{inspect(metadata)})
+
+        # as map
+        Logger.#{level}("this is #{article} #{level} message", #{inspect(Map.new(metadata))})
+
     """
 
     # Only macros generated for the "new" Erlang levels are available since 1.11.0. Other

lib/logger/test/logger_test.exs

@@ -16,6 +16,11 @@ defmodule LoggerTest do
     msg("module=LoggerTest #{text}")
   end
 
+  test "levels/0" do
+    assert [_ | _] = Logger.levels()
+    assert :info in Logger.levels()
+  end
+
   test "level/0" do
     assert Logger.level() == :debug
 

lib/mix/lib/mix/compilers/elixir.ex

@@ -767,14 +767,16 @@ defmodule Mix.Compilers.Elixir do
     end
   end
 
-  defp diagnostic(%{
+  defp diagnostic(
-         file: file,
+         %{
-         position: position,
+           file: file,
-         message: message,
+           position: position,
-         severity: severity,
+           message: message,
-         stacktrace: stacktrace,
+           severity: severity,
-         span: span
+           stacktrace: stacktrace,
-       }) do
+           span: span
+         } = diagnostic
+       ) do
     %Mix.Task.Compiler.Diagnostic{
       file: file,
       position: position,
@@ -782,7 +784,8 @@ defmodule Mix.Compilers.Elixir do
       severity: severity,
       compiler_name: "Elixir",
       stacktrace: stacktrace,
-      span: span
+      span: span,
+      details: Map.get(diagnostic, :exception, nil)
     }
   end
 

lib/mix/lib/mix/dep/lock.ex

@@ -11,7 +11,7 @@ defmodule Mix.Dep.Lock do
   """
   @spec read(Path.t()) :: map()
   def read(lockfile \\ lockfile()) do
-    opts = [file: lockfile, warn_on_unnecessary_quotes: false]
+    opts = [file: lockfile, emit_warnings: false]
 
     with {:ok, contents} <- File.read(lockfile),
          assert_no_merge_conflicts_in_lockfile(lockfile, contents),

lib/mix/lib/mix/local/installer.ex

@@ -245,7 +245,7 @@ defmodule Mix.Local.Installer do
   end
 
   defp git_fetcher(url, git_config, opts) do
-    git_opts = git_config ++ [git: url, submodules: opts[:submodules]]
+    git_opts = git_config ++ [git: url, submodules: opts[:submodules], sparse: opts[:sparse]]
 
     app_name =
       if opts[:app] do
@@ -319,8 +319,15 @@ defmodule Mix.Local.Installer do
   """
   @spec fetch(tuple, (atom -> any), (atom -> any)) :: any
   def fetch(dep_spec, in_fetcher \\ &in_fetcher/1, in_package) do
-    with_tmp_dir(fn tmp_path ->
+    tmp_path = tmp_path()
+    previous_env = Mix.env()
+    deps_path = System.get_env("MIX_DEPS_PATH")
+    code_path = :code.get_path()
+
+    try do
       File.mkdir_p!(tmp_path)
+      System.delete_env("MIX_DEPS_PATH")
+      Mix.env(:prod)
 
       File.write!(Path.join(tmp_path, "mix.exs"), """
       defmodule Mix.Local.Installer.MixProject do
@@ -336,64 +343,62 @@ defmodule Mix.Local.Installer do
       end
       """)
 
-      with_reset_prod_env(fn ->
+      Mix.ProjectStack.on_clean_slate(fn ->
-        Mix.ProjectStack.on_clean_slate(fn ->
+        tmp_path =
-          tmp_path =
+          Mix.Project.in_project(:mix_local_installer, tmp_path, [], fn mix_exs ->
-            Mix.Project.in_project(:mix_local_installer, tmp_path, [], fn mix_exs ->
-              in_fetcher.(mix_exs)
-
-              # The tmp_dir may have symlinks in it, so we properly resolve
-              # the directory before customizing deps_path and lockfile.
-              File.cwd!()
-            end)
-
-          package_name = elem(dep_spec, 0)
-          package_name_string = Atom.to_string(package_name)
-          package_path = Path.join([tmp_path, "deps", package_name_string])
-
-          post_config = [
-            deps_path: Path.join(tmp_path, "deps"),
-            lockfile: Path.join(tmp_path, "mix.lock")
-          ]
-
-          Mix.Project.in_project(package_name, package_path, post_config, fn mix_exs ->
             in_fetcher.(mix_exs)
-            in_package.(mix_exs)
+
+            # The tmp_dir may have symlinks in it, so we properly resolve
+            # the directory before customizing deps_path and lockfile.
+            File.cwd!()
           end)
+
+        {package_name, package_path} = package_name_path(dep_spec, tmp_path)
+
+        post_config = [
+          deps_path: Path.join(tmp_path, "deps"),
+          lockfile: Path.join(tmp_path, "mix.lock")
+        ]
+
+        Mix.Project.in_project(package_name, package_path, post_config, fn mix_exs ->
+          in_fetcher.(mix_exs)
+          in_package.(mix_exs)
         end)
       end)
-    end)
+    after
-  after
+      File.rm_rf(tmp_path)
-    :code.purge(Mix.Local.Installer.Fetcher)
+      Mix.env(previous_env)
-    :code.delete(Mix.Local.Installer.Fetcher)
+      deps_path && System.put_env("MIX_DEPS_PATH", deps_path)
+      :code.set_path(code_path)
+      :code.purge(Mix.Local.Installer.Fetcher)
+      :code.delete(Mix.Local.Installer.Fetcher)
+    end
   end
 
+  defp package_name_path(dep_spec, tmp_path) do
+    package_name = elem(dep_spec, 0)
+    package_name_string = Atom.to_string(package_name)
+    package_path = Path.join([tmp_path, "deps", package_name_string, maybe_sparse_dir(dep_spec)])
+
+    {package_name, package_path}
+  end
+
+  defp maybe_sparse_dir({_app, opts}) when is_list(opts) do
+    if opts[:git] do
+      opts[:sparse] || ""
+    else
+      ""
+    end
+  end
+
+  defp maybe_sparse_dir(_dep_spec), do: ""
+
   defp in_fetcher(_mix_exs) do
     Mix.Task.run("deps.get", ["--only", Atom.to_string(Mix.env())])
   end
 
-  defp with_tmp_dir(fun) do
+  defp tmp_path do
     unique = :crypto.strong_rand_bytes(4) |> Base.url_encode64(padding: false)
-    tmp_path = Path.join(System.tmp_dir!(), "mix-local-installer-fetcher-" <> unique)
+    Path.join(System.tmp_dir!(), "mix-local-installer-fetcher-" <> unique)
-
-    try do
-      fun.(tmp_path)
-    after
-      File.rm_rf(tmp_path)
-    end
-  end
-
-  defp with_reset_prod_env(fun) do
-    previous_env = Mix.env()
-    deps_path = System.get_env("MIX_DEPS_PATH")
-
-    try do
-      System.delete_env("MIX_DEPS_PATH")
-      Mix.env(:prod)
-      fun.()
-    after
-      Mix.env(previous_env)
-      deps_path && System.put_env("MIX_DEPS_PATH", deps_path)
-    end
   end
 end

lib/mix/lib/mix/release.ex

@@ -68,7 +68,7 @@ defmodule Mix.Release do
   @safe_modes [:permanent, :temporary, :transient]
   @unsafe_modes [:load, :none]
   @additional_chunks ~w(Attr)c
-  @copy_app_dirs ["priv"]
+  @copy_app_dirs ["priv", "include"]
 
   @doc false
   @spec from_config!(atom, keyword, keyword) :: t

lib/mix/lib/mix/scm/git.ex

@@ -201,18 +201,24 @@ defmodule Mix.SCM.Git do
   ## Helpers
 
   defp validate_git_options(opts) do
-    err =
+    case Keyword.take(opts, [:branch, :ref, :tag]) do
-      "You should specify only one of branch, ref or tag, and only once. " <>
+      [] ->
-        "Error on Git dependency: #{redact_uri(opts[:git])}"
+        opts
 
-    validate_single_uniq(opts, [:branch, :ref, :tag], err)
+      [{_refspec, value}] when is_binary(value) ->
-  end
+        opts
 
-  defp validate_single_uniq(opts, take, error) do
+      [{refspec, value}] ->
-    case Keyword.take(opts, take) do
+        Mix.raise(
-      [] -> opts
+          "A dependency's #{refspec} must be a string, got: #{inspect(value)}. " <>
-      [_] -> opts
+            "Error on Git dependency: #{redact_uri(opts[:git])}"
-      _ -> Mix.raise(error)
+        )
+
+      _ ->
+        Mix.raise(
+          "You should specify only one of branch, ref or tag, and only once. " <>
+            "Error on Git dependency: #{redact_uri(opts[:git])}"
+        )
     end
   end
 

lib/mix/lib/mix/task.compiler.ex

@@ -39,7 +39,7 @@ defmodule Mix.Task.Compiler do
             message: IO.chardata(),
             position: position,
             compiler_name: String.t(),
-            details: any,
+            details: Exception.t() | any,
             stacktrace: Exception.stacktrace(),
             span: {non_neg_integer, non_neg_integer} | nil
           }

lib/mix/lib/mix/task.ex

@@ -239,6 +239,9 @@ defmodule Mix.Task do
     Mix.ProjectStack.recursing() != nil
   end
 
+  @doc """
+  Available for backwards compatibility.
+  """
   @deprecated "Configure the environment in your mix.exs"
   defdelegate preferred_cli_env(task), to: Mix.CLI
 

lib/mix/lib/mix/tasks/archive.install.ex

@@ -47,6 +47,9 @@ defmodule Mix.Tasks.Archive.Install do
     * `--submodules` - fetches repository submodules before building archive from
       Git or GitHub
 
+    * `--sparse` - checkout a single directory inside the Git repository and use
+      it as the archive root directory
+
     * `--app` - specifies a custom app name to be used for building the archive
       from Git, GitHub, or Hex
 
@@ -63,6 +66,7 @@ defmodule Mix.Tasks.Archive.Install do
     force: :boolean,
     sha512: :string,
     submodules: :boolean,
+    sparse: :string,
     app: :string,
     organization: :string,
     repo: :string,

lib/mix/lib/mix/tasks/compile.app.ex

@@ -344,6 +344,13 @@ defmodule Mix.Tasks.Compile.App do
   defp handle_extra_applications(properties, config) do
     {extra, properties} = Keyword.pop(properties, :extra_applications, [])
 
+    if extra != [] and Keyword.has_key?(properties, :applications) do
+      Mix.shell().error(
+        "both :extra_applications and :applications was found in your mix.exs. " <>
+          "You most likely want to remove the :applications key, as all applications are derived from your dependencies"
+      )
+    end
+
     {all, optional} =
       project_apps(properties, config, extra, fn ->
         apps_from_runtime_prod_deps(properties, config)

lib/mix/lib/mix/tasks/deps.clean.ex

@@ -99,6 +99,15 @@ defmodule Mix.Tasks.Deps.Clean do
     paths
   end
 
+  defp maybe_warn_failed_file_deletion(result) do
+    with {:error, reason, file} <- result do
+      Mix.shell().error(
+        "warning: could not delete file #{Path.relative_to_cwd(file)}, " <>
+          "reason: #{:file.format_error(reason)}"
+      )
+    end
+  end
+
   defp do_clean(apps, deps, build_path, deps_path, build_only?) do
     shell = Mix.shell()
 
@@ -112,7 +121,7 @@ defmodule Mix.Tasks.Deps.Clean do
       |> Path.join(to_string(app))
       |> Path.wildcard()
       |> maybe_warn_for_invalid_path(app)
-      |> Enum.each(&File.rm_rf!/1)
+      |> Enum.map(&(&1 |> File.rm_rf() |> maybe_warn_failed_file_deletion()))
 
       # Remove everything from the source directory of dependencies.
       # Skip this step if --build option is specified or if
@@ -122,7 +131,8 @@ defmodule Mix.Tasks.Deps.Clean do
       else
         deps_path
         |> Path.join(to_string(app))
-        |> File.rm_rf!()
+        |> File.rm_rf()
+        |> maybe_warn_failed_file_deletion()
       end
     end)
   end

lib/mix/lib/mix/tasks/escript.install.ex

@@ -47,6 +47,9 @@ defmodule Mix.Tasks.Escript.Install do
     * `--submodules` - fetches repository submodules before building escript from
       Git or GitHub
 
+    * `--sparse` - checkout a single directory inside the Git repository and use
+      it as the escript project directory
+
     * `--app` - specifies a custom app name to be used for building the escript
       from Git, GitHub, or Hex
 
@@ -66,6 +69,7 @@ defmodule Mix.Tasks.Escript.Install do
     force: :boolean,
     sha512: :string,
     submodules: :boolean,
+    sparse: :string,
     app: :string,
     organization: :string,
     repo: :string,

lib/mix/test/mix/local/installer_test.exs

@@ -30,50 +30,76 @@ defmodule Mix.Local.InstallerTest do
 
   test "parse_args Git" do
     args = ["git", "https://example.com/user/repo.git"]
-    opts = [git: "https://example.com/user/repo.git", submodules: nil]
+    opts = [git: "https://example.com/user/repo.git", submodules: nil, sparse: nil]
 
     assert Mix.Local.Installer.parse_args(args, []) == {:fetcher, {:"new package", opts}}
   end
 
   test "parse_args Git branch" do
     args = ["git", "https://example.com/user/repo.git", "branch", "not_main"]
-    opts = [branch: "not_main", git: "https://example.com/user/repo.git", submodules: nil]
+
+    opts = [
+      branch: "not_main",
+      git: "https://example.com/user/repo.git",
+      submodules: nil,
+      sparse: nil
+    ]
 
     assert Mix.Local.Installer.parse_args(args, []) == {:fetcher, {:"new package", opts}}
   end
 
   test "parse_args Git ref" do
     args = ["git", "https://example.com/user/repo.git", "ref", "not_main"]
-    opts = [ref: "not_main", git: "https://example.com/user/repo.git", submodules: nil]
+
+    opts = [
+      ref: "not_main",
+      git: "https://example.com/user/repo.git",
+      submodules: nil,
+      sparse: nil
+    ]
 
     assert Mix.Local.Installer.parse_args(args, []) == {:fetcher, {:"new package", opts}}
   end
 
   test "parse_args Git tag" do
     args = ["git", "https://example.com/user/repo.git", "tag", "not_main"]
-    opts = [tag: "not_main", git: "https://example.com/user/repo.git", submodules: nil]
+
+    opts = [
+      tag: "not_main",
+      git: "https://example.com/user/repo.git",
+      submodules: nil,
+      sparse: nil
+    ]
 
     assert Mix.Local.Installer.parse_args(args, []) == {:fetcher, {:"new package", opts}}
   end
 
   test "parse_args Git submodules" do
     args = ["git", "https://example.com/user/repo.git"]
-    opts = [git: "https://example.com/user/repo.git", submodules: true]
+    opts = [git: "https://example.com/user/repo.git", submodules: true, sparse: nil]
 
     assert Mix.Local.Installer.parse_args(args, submodules: true) ==
              {:fetcher, {:"new package", opts}}
   end
 
+  test "parse_args Git sparse" do
+    args = ["git", "https://example.com/user/repo.git"]
+    opts = [git: "https://example.com/user/repo.git", submodules: nil, sparse: "foo"]
+
+    assert Mix.Local.Installer.parse_args(args, sparse: "foo") ==
+             {:fetcher, {:"new package", opts}}
+  end
+
   test "parse_args Git app" do
     args = ["git", "https://example.com/user/repo.git"]
-    opts = [git: "https://example.com/user/repo.git", submodules: nil]
+    opts = [git: "https://example.com/user/repo.git", submodules: nil, sparse: nil]
 
     assert Mix.Local.Installer.parse_args(args, app: "my_app") == {:fetcher, {:my_app, opts}}
   end
 
   test "parse_args GitHub" do
     args = ["github", "user/repo"]
-    opts = [git: "https://github.com/user/repo.git", submodules: nil]
+    opts = [git: "https://github.com/user/repo.git", submodules: nil, sparse: nil]
 
     assert Mix.Local.Installer.parse_args(args, []) == {:fetcher, {:"new package", opts}}
   end

lib/mix/test/mix/scm/git_test.exs

@@ -50,7 +50,7 @@ defmodule Mix.SCM.GitTest do
     end
   end
 
-  test "raises about conflicting Git checkout options" do
+  test "raises about conflicting Git refspec options" do
     assert_raise Mix.Error, ~r/You should specify only one of branch, ref or tag/, fn ->
       Mix.SCM.Git.accepts_options(nil, git: "/repo", branch: "main", tag: "0.1.0")
     end
@@ -60,6 +60,20 @@ defmodule Mix.SCM.GitTest do
     end
   end
 
+  test "raises about non-binary Git refspec options" do
+    assert_raise Mix.Error, ~r/A dependency's branch must be a string/, fn ->
+      Mix.SCM.Git.accepts_options(nil, git: "/repo", branch: :main)
+    end
+
+    assert_raise Mix.Error, ~r/A dependency's tag must be a string/, fn ->
+      Mix.SCM.Git.accepts_options(nil, git: "/repo", tag: :stable)
+    end
+
+    assert_raise Mix.Error, ~r/A dependency's ref must be a string/, fn ->
+      Mix.SCM.Git.accepts_options(nil, git: "/repo", ref: :abcdef0123456789)
+    end
+  end
+
   defp lock(opts \\ []) do
     [lock: {:git, "/repo", "abcdef0123456789", opts}]
   end

lib/mix/test/mix/tasks/compile_test.exs

@@ -182,7 +182,8 @@ defmodule Mix.Tasks.CompileTest do
                  severity: :error,
                  position: {2, 20},
                  message: "** (SyntaxError) invalid syntax found on lib/a.ex:2:" <> _,
-                 compiler_name: "Elixir"
+                 compiler_name: "Elixir",
+                 details: %SyntaxError{}
                } = diagnostic
       end)
     end)