Release v1.13.2

They are already being used in projects, which
means it is too late for a deprecation. The docs
instead discuss when to use them.

CHANGELOG.md

@@ -128,43 +128,51 @@ Now any application can use your formatter as follows:
 
 Finally, the `Code` module has also been augmented with two functions: `Code.string_to_quoted_with_comments/2` and `Code.quoted_to_algebra/2`. Those functions allow someone to retrieve the Elixir AST with their original source code comments, and then convert this AST to formatted code. In other words, those functions provide a wrapper around the Elixir Code Formatter, supporting developers who wish to create tools that directly manipulate and custom format Elixir source code.
 
-## v1.13.0-rc.1 (2021-11-16)
+## v1.13.2 (2022-01-13)
 
 ### 1. Enhancements
 
 #### Mix
 
-  * [Mix] Add `Mix.installed?/0`
+  * [mix format] Allow plugins to also format `.ex` and `.exs` files
+  * [mix release] Allow bypassing application mode validation in release spec
+  * [mix test] Print a message when the suite fails due to the coverage threshold
 
 ### 2. Bug fixes
 
 #### Elixir
 
-  * [Application] Allow any expression as first argument of `compile_env`
-  * [Kernel] Reject bidirectional formatting characters in strings and comments
-  * [Kernel] Support escaping of terminators in uppercase sigils heredocs for consistency
+  * [Code] Do not emit warnings on `Code.Fragment.container_cursor_to_quoted/2`
+  * [Kernel] Fix a crash when a for-comprehension with `:uniq` was used inside another comprehension with `:uniq`
+  * [Kernel] Ensure `env.context_modules` is properly set inside optimized `defmodule`
+  * [Keyword] Deprecate the recently added `Keyword.map/2` as it is equivalent to `Keyword.new/2`
+  * [Map] Deprecate the recently added `Map.map/2` as it is equivalent to `Map.new/2`
+  * [Protocol] Warn on zero arity callbacks inside protocols
 
-#### IEx
+## v1.13.1 (2021-12-14)
 
-  * [IEx] Stop evaluator before exiting IEx server
-
-#### Mix
-
-  * [mix release] Raise proper error message when non-serializable values are in configs
-
-### 3. Regressions
+### 1. Bug fixes
 
 #### Elixir
 
-  * [Code] Do not crash on duplicate bindings during eval
-  * [Macro] Do not raise on variables that look like an empty tuple in `Macro.to_string/1`
-  * [URI] Make sure parsed URIs with empty paths have the path field set to nil
+  * [Code] Do not show code snippets in `SyntaxError` and `TokenMissingError` if line is empty
+  * [Exception] Do not fail blaming `ArgumentError` for improper lists on `apply/3`
+  * [Macro] Set a max `line_length` for `Macro.to_string/1`
+  * [Macro] Fix formatting of lists on module attributes for `Macro.to_string/1`
+  * [String] Fix incorrect codepoint byte counting in `slice` with negative positions in ranges
+  * [Task] Ensure async streams can be consumed from another process than the one that creates them
+  * [URI] Undeprecate `URI.parse/1` as `URI.new/1` is too strict in many common cases
+  * [URI] Make sure `URI.new/1` returns nil for empty paths
+
+#### IEx
+
+  * [IEx] Make sure the `--version` flag halts IEx
 
 #### Mix
 
-  * [Mix.Shell] Re-add `yes?/1` as a callback
+  * [Mix] Make protocol consolidation part of the `Mix.install/2` cache
 
-## v1.13.0-rc.0 (2021-11-01)
+## v1.13.0 (2021-12-03)
 
 ### 1. Enhancements
 
@@ -187,7 +195,7 @@ Finally, the `Code` module has also been augmented with two functions: `Code.str
   * [Inspect] Allow default inspect fun to be set globally with `Inspect.Opts.default_inspect_fun/1`
   * [IO] Allow `:eof` to be given as limit to `IO.getn/2`
   * [Kernel] Support the `:sigils` option in `import Mod, only: :sigils` and allow the sigil modifiers to be also digits
-  * [Kernel] Make `get_in` consistently abort when `nil` values are found
+  * [Kernel] Make `get_in` consistently abort and return `nil` when `nil` values are found (previously Elixir would raise an error in this case). This allows a user to use `get_in` as a safe navigation operator.
   * [Kernel] Improve compilation times by reducing the amount of copies of the AST across compiler processes
   * [Kernel] Raise if trying to define a module with a slash in its name
   * [Kernel] Warn when `?\` is used and there is no need for a escape character
@@ -223,11 +231,13 @@ Finally, the `Code` module has also been augmented with two functions: `Code.str
 #### Logger
 
   * [Logger] Add `Logger.put_application_level/2`
+  * [Logger] Print all log levels in accordance to Erlang/OTP. This also means `[warn]` is now shown as `[warning]`
 
 #### Mix
 
   * [Mix] Add `MIX_INSTALL_FORCE` environment variable support
   * [Mix] Support `:config` and `:system_env` in `Mix.install/2`
+  * [Mix] Add `Mix.installed?/0`
   * [Mix.Shell] Add `:default` option to `Mix.Shell.yes?`
   * [mix archive.install] Run `loadconfig` before building archive
   * [mix compile] Move Elixir version check to before deps are compiled, in order to give feedback earlier
@@ -263,10 +273,13 @@ Finally, the `Code` module has also been augmented with two functions: `Code.str
 
 #### Elixir
 
+  * [Application] Allow any expression as first argument of `compile_env`
   * [Application] Warn if `Application.compile_env` or `Application.compile_env!` are called without a require
   * [Code] Make sure `:static_atoms_encoder` in `Code.string_to_quoted/2` also applies to quoted keyword keys
   * [Code] Ensure bindings with no context are returned as atoms instead of `{binding, nil}` in eval operations
   * [Inspect] Fix a bug when inspecting a non-binary bitstring with colors
+  * [Kernel] Reject bidirectional formatting characters in strings and comments
+  * [Kernel] Support escaping of terminators in uppercase sigils heredocs for consistency
   * [Kernel] Raise if `__CALLER__` or `__ENV__` or `__STACKTRACE__` are used in match
   * [Kernel] Improve error message on invalid argument for `byte_size` from binary concat
   * [Kernel] Raise when aliasing non-Elixir modules without `:as`
@@ -293,6 +306,7 @@ Finally, the `Code` module has also been augmented with two functions: `Code.str
 #### IEx
 
   * [IEx] Fix the loss of `.iex.exs` context after a pry session
+  * [IEx] Stop evaluator before exiting IEx server to avoid evaluators leaking
 
 #### Logger
 
@@ -301,6 +315,7 @@ Finally, the `Code` module has also been augmented with two functions: `Code.str
 
 #### Mix
 
+  * [mix compile.elixir] Track transitive runtime dependencies coming from local/path dependencies
   * [mix compile.elixir] Recompile file if `@external_resource` is deleted
   * [mix compile.elixir] Print number of compiling files on all compiler cycles. This will make the `Compiling N files (.ex)` show up multiple times if necessary
   * [mix deps] Raise if local dep is unavailable while compiling
@@ -309,6 +324,7 @@ Finally, the `Code` module has also been augmented with two functions: `Code.str
   * [mix release] Improve release scripts by making sure shell errors cascade (this is done by avoiding exporting and defining variables in a single step)
   * [mix release] Do not boot release if `RELEASE_COOKIE` is empty
   * [mix release] Allow releases running as a daemon to be restarted
+  * [mix release] Raise proper error message when non-serializable values are in configs
   * [mix test] Fix coverage engine to also tag `case`, `cond`, and `receive` branches where the right side is a literal
 
 ### 3. Soft-deprecations (no warnings emitted)

VERSION

@@ -1 +1 @@
-1.13.0-rc.1
+1.13.2

bin/elixir

@@ -1,7 +1,7 @@
 #!/bin/sh
 set -e
 
-ELIXIR_VERSION=1.13.0-rc.1
+ELIXIR_VERSION=1.13.2
 
 if [ $# -eq 0 ] || { [ $# -eq 1 ] && { [ "$1" = "--help" ] || [ "$1" = "-h" ]; }; }; then
   cat <<USAGE >&2
@@ -16,7 +16,7 @@ Usage: $(basename "$0") [options] [.exs file] [data]
   -pr "FILE"                   Requires the given files/patterns in parallel (*)
   -pa "PATH"                   Prepends the given path to Erlang code path (*)
   -pz "PATH"                   Appends the given path to Erlang code path (*)
-  -v, --version                Prints Erlang/OTP and Elixir versions
+  -v, --version                Prints Erlang/OTP and Elixir versions (standalone)
 
   --app APP                    Starts the given app and its dependencies (*)
   --erl "SWITCHES"             Switches to be passed down to Erlang (*)

bin/elixir.bat

@@ -1,6 +1,6 @@
 @if defined ELIXIR_CLI_ECHO (@echo on) else (@echo off)
 
-set ELIXIR_VERSION=1.13.0-rc.1
+set ELIXIR_VERSION=1.13.2
 
 setlocal enabledelayedexpansion
 if    ""%1""==""""                if ""%2""=="""" goto documentation
@@ -23,7 +23,7 @@ echo   -S SCRIPT                    Finds and executes the given script in $PATH
 echo   -pr "FILE"                   Requires the given files/patterns in parallel (*)
 echo   -pa "PATH"                   Prepends the given path to Erlang code path (*)
 echo   -pz "PATH"                   Appends the given path to Erlang code path (*)
-echo   -v, --version                Prints Erlang/OTP and Elixir versions
+echo   -v, --version                Prints Erlang/OTP and Elixir versions (standalone)
 echo.
 echo   --app APP                    Starts the given app and its dependencies (*)
 echo   --erl "SWITCHES"             Switches to be passed down to Erlang (*)

bin/elixirc

@@ -7,7 +7,7 @@ Usage: $(basename "$0") [elixir switches] [compiler switches] [.ex files]
 
   -h, --help                Prints this message and exits
   -o                        The directory to output compiled files
-  -v, --version             Prints Elixir version and exits
+  -v, --version             Prints Elixir version and exits (standalone)
 
   --ignore-module-conflict  Does not emit warnings if a module was previously defined
   --no-debug-info           Does not attach debug info to compiled modules

bin/elixirc.bat

@@ -16,7 +16,7 @@ echo Usage: %~nx0 [elixir switches] [compiler switches] [.ex files]
 echo.
 echo   -h, --help                Prints this message and exits
 echo   -o                        The directory to output compiled files
-echo   -v, --version             Prints Elixir version and exits
+echo   -v, --version             Prints Elixir version and exits (standalone)
 echo.
 echo   --ignore-module-conflict  Does not emit warnings if a module was previously defined
 echo   --no-debug-info           Does not attach debug info to compiled modules

lib/elixir/lib/base.ex

@@ -92,6 +92,9 @@ defmodule Base do
 
   """
 
+  @type encode_case :: :upper | :lower
+  @type decode_case :: :upper | :lower | :mixed
+
   b16_alphabet = '0123456789ABCDEF'
   b64_alphabet = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/'
   b64url_alphabet = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_'
@@ -267,7 +270,7 @@ defmodule Base do
       "666f6f626172"
 
   """
-  @spec encode16(binary, keyword) :: binary
+  @spec encode16(binary, case: encode_case) :: binary
   def encode16(data, opts \\ []) when is_binary(data) do
     case = Keyword.get(opts, :case, :upper)
     do_encode16(case, data)
@@ -300,7 +303,7 @@ defmodule Base do
       {:ok, "foobar"}
 
   """
-  @spec decode16(binary, keyword) :: {:ok, binary} | :error
+  @spec decode16(binary, case: decode_case) :: {:ok, binary} | :error
   def decode16(string, opts \\ []) do
     {:ok, decode16!(string, opts)}
   rescue
@@ -337,7 +340,7 @@ defmodule Base do
       "foobar"
 
   """
-  @spec decode16!(binary, keyword) :: binary
+  @spec decode16!(binary, case: encode_case) :: binary
   def decode16!(string, opts \\ [])
 
   def decode16!(string, opts) when is_binary(string) and rem(byte_size(string), 2) == 0 do
@@ -367,7 +370,7 @@ defmodule Base do
       "Zm9vYg"
 
   """
-  @spec encode64(binary, keyword) :: binary
+  @spec encode64(binary, padding: boolean) :: binary
   def encode64(data, opts \\ []) when is_binary(data) do
     pad? = Keyword.get(opts, :padding, true)
     do_encode64(data, pad?)
@@ -397,7 +400,7 @@ defmodule Base do
       {:ok, "foob"}
 
   """
-  @spec decode64(binary, keyword) :: {:ok, binary} | :error
+  @spec decode64(binary, ignore: :whitespace, padding: boolean) :: {:ok, binary} | :error
   def decode64(string, opts \\ []) when is_binary(string) do
     {:ok, decode64!(string, opts)}
   rescue
@@ -431,7 +434,7 @@ defmodule Base do
       "foob"
 
   """
-  @spec decode64!(binary, keyword) :: binary
+  @spec decode64!(binary, ignore: :whitespace, padding: boolean) :: binary
   def decode64!(string, opts \\ []) when is_binary(string) do
     pad? = Keyword.get(opts, :padding, true)
     string |> remove_ignored(opts[:ignore]) |> do_decode64(pad?)
@@ -453,7 +456,7 @@ defmodule Base do
       "_3_-_A"
 
   """
-  @spec url_encode64(binary, keyword) :: binary
+  @spec url_encode64(binary, padding: boolean) :: binary
   def url_encode64(data, opts \\ []) when is_binary(data) do
     pad? = Keyword.get(opts, :padding, true)
     do_encode64url(data, pad?)
@@ -481,7 +484,7 @@ defmodule Base do
       {:ok, <<255, 127, 254, 252>>}
 
   """
-  @spec url_decode64(binary, keyword) :: {:ok, binary} | :error
+  @spec url_decode64(binary, ignore: :whitespace, padding: boolean) :: {:ok, binary} | :error
   def url_decode64(string, opts \\ []) when is_binary(string) do
     {:ok, url_decode64!(string, opts)}
   rescue
@@ -513,7 +516,7 @@ defmodule Base do
       <<255, 127, 254, 252>>
 
   """
-  @spec url_decode64!(binary, keyword) :: binary
+  @spec url_decode64!(binary, ignore: :whitespace, padding: boolean) :: binary
   def url_decode64!(string, opts \\ []) when is_binary(string) do
     pad? = Keyword.get(opts, :padding, true)
     string |> remove_ignored(opts[:ignore]) |> do_decode64url(pad?)
@@ -551,7 +554,7 @@ defmodule Base do
       "MZXW6YTBOI"
 
   """
-  @spec encode32(binary, keyword) :: binary
+  @spec encode32(binary, case: encode_case, padding: boolean) :: binary
   def encode32(data, opts \\ []) when is_binary(data) do
     case = Keyword.get(opts, :case, :upper)
     pad? = Keyword.get(opts, :padding, true)
@@ -594,7 +597,7 @@ defmodule Base do
       {:ok, "foobar"}
 
   """
-  @spec decode32(binary, keyword) :: {:ok, binary} | :error
+  @spec decode32(binary, case: decode_case, padding: boolean) :: {:ok, binary} | :error
   def decode32(string, opts \\ []) do
     {:ok, decode32!(string, opts)}
   rescue
@@ -640,7 +643,7 @@ defmodule Base do
       "foobar"
 
   """
-  @spec decode32!(binary, keyword) :: binary
+  @spec decode32!(binary, case: decode_case, padding: boolean) :: binary
   def decode32!(string, opts \\ []) when is_binary(string) do
     case = Keyword.get(opts, :case, :upper)
     pad? = Keyword.get(opts, :padding, true)
@@ -680,7 +683,7 @@ defmodule Base do
       "CPNMUOJ1E8"
 
   """
-  @spec hex_encode32(binary, keyword) :: binary
+  @spec hex_encode32(binary, case: encode_case, padding: boolean) :: binary
   def hex_encode32(data, opts \\ []) when is_binary(data) do
     case = Keyword.get(opts, :case, :upper)
     pad? = Keyword.get(opts, :padding, true)
@@ -724,7 +727,7 @@ defmodule Base do
       {:ok, "foobar"}
 
   """
-  @spec hex_decode32(binary, keyword) :: {:ok, binary} | :error
+  @spec hex_decode32(binary, case: decode_case, padding: boolean) :: {:ok, binary} | :error
   def hex_decode32(string, opts \\ []) do
     {:ok, hex_decode32!(string, opts)}
   rescue
@@ -771,7 +774,7 @@ defmodule Base do
       "foobar"
 
   """
-  @spec hex_decode32!(binary, keyword) :: binary
+  @spec hex_decode32!(binary, case: decode_case, padding: boolean) :: binary
   def hex_decode32!(string, opts \\ []) when is_binary(string) do
     case = Keyword.get(opts, :case, :upper)
     pad? = Keyword.get(opts, :padding, true)

lib/elixir/lib/calendar/datetime.ex

@@ -2,11 +2,14 @@ defmodule DateTime do
   @moduledoc """
   A datetime implementation with a time zone.
 
-  This datetime can be seen as an ephemeral snapshot
-  of a datetime at a given time zone. For such purposes,
-  it also includes both UTC and Standard offsets, as
-  well as the zone abbreviation field used exclusively
-  for formatting purposes.
+  This datetime can be seen as a snapshot of a date and time
+  at a given time zone. For such purposes, it also includes both
+  UTC and Standard offsets, as well as the zone abbreviation
+  field used exclusively for formatting purposes. Note future
+  datetimes are not necessarily guaranteed to exist, as time
+  zones may change any time in the future due to geopolitical
+  reasons. See the "Datetimes as snapshots" section for more
+  information.
 
   Remember, comparisons in Elixir using `==/2`, `>/2`, `</2` and friends
   are structural and based on the DateTime struct fields. For proper
@@ -41,6 +44,56 @@ defmodule DateTime do
       Calendar.put_time_zone_database(Tzdata.TimeZoneDatabase)
 
   See the proper names in the library installation instructions.
+
+  ## Datetimes as snapshots
+
+  In the first section, we described datetimes as a "snapshot of
+  a date and time at a given time zone". To understand precisely
+  what we mean, let's see an example.
+
+  Imagine someone in Poland wants to schedule a meeting with someone
+  in Brazil in the next year. The meeting will happen at 2:30 AM
+  in the Polish time zone. At what time will the meeting happen in
+  Brazil?
+
+  You can consult the time zone database today, one year before,
+  using the API in this module and it will give you an answer that
+  is valid right now. However, this answer may not be valid in the
+  future. Why? Because both Brazil and Poland may change their timezone
+  rules, ultimately affecting the result. For example, a country may
+  choose to enter or abandon "Daylight Saving Time", which is a
+  process where we adjust the clock one hour forward or one hour
+  back once per year. Whenener the rules change, the exact instant
+  that 2:30 AM in Polish time will be in Brazil may change.
+
+  In other words, whenever working with future DateTimes, there is
+  no guarantee the results you get will always be correct, until
+  the event actually happens. Therefore, when you ask for a future
+  time, the answers you get are a snapshot that reflects the current
+  state of the time zone rules. For datetimes in the past, this is
+  not a problem, because time zone rules do not change for past
+  events.
+
+  To make matters worse, it may be that the 2:30 AM in Polish time
+  does not actually even exist or it is ambiguous. If a certain
+  time zone observes "Daylight Saving Time", they will move their
+  clock forward once a year. When this happens, there is a whole
+  hour that does not exist. Then, when they move the clock back,
+  there is a certain hour that will happen twice. So if you want
+  to schedule a meeting when this shift back happens, you would
+  need to explicitly say which of the 2:30 AM you precisely mean.
+  Applications that are date and time sensitive, need to take
+  these scenarios into account and correctly communicate them to
+  users.
+
+  The good news is: Elixir contains all of the building blocks
+  necessary to tackle those problems. The default timezone database
+  used by Elixir, `Calendar.UTCOnlyTimeZoneDatabase`, only works
+  with UTC, which does not observe those issues. Once you bring
+  a proper time zone database, the functions in this module will
+  query the database and return the relevant information. For
+  example, look at how `DateTime.new/4` returns different results
+  based on the scenarios described in this section.
   """
 
   @enforce_keys [:year, :month, :day, :hour, :minute, :second] ++

lib/elixir/lib/code.ex

@@ -1599,7 +1599,7 @@ defmodule Code do
   file.
 
   It returns the term stored in the documentation chunk in the format defined by
-  [EEP 48](https://erlang.org/eep/eeps/eep-0048.html) or `{:error, reason}` if
+  [EEP 48](https://www.erlang.org/eeps/eep-0048.html) or `{:error, reason}` if
   the chunk is not available.
 
   ## Examples
@@ -1702,7 +1702,7 @@ defmodule Code do
   @doc ~S"""
   Deprecated function to retrieve old documentation format.
 
-  Elixir v1.7 adopts [EEP 48](https://erlang.org/eep/eeps/eep-0048.html)
+  Elixir v1.7 adopts [EEP 48](https://www.erlang.org/eeps/eep-0048.html)
   which is a new documentation format meant to be shared across all
   BEAM languages. The old format, used by `Code.get_docs/2`, is no
   longer available, and therefore this function always returns `nil`.

lib/elixir/lib/code/fragment.ex

@@ -525,7 +525,7 @@ defmodule Code.Fragment do
     |> position_surround_context(line, column, opts)
   end
 
-  def surround_context(other, position, opts) do
+  def surround_context(other, {_, _} = position, opts) do
     surround_context(to_charlist(other), position, opts)
   end
 
@@ -827,7 +827,7 @@ defmodule Code.Fragment do
 
     case :elixir_tokenizer.tokenize(fragment, line, column, tokenizer_opts) do
       {:ok, _, _, _warnings, tokens} ->
-        :elixir.tokens_to_quoted(tokens, file, columns: columns, token_metadata: token_metadata)
+        :elixir.tokens_to_quoted(tokens, nil, columns: columns, token_metadata: token_metadata)
 
       {:error, {line, column, {prefix, suffix}, token}, _rest, _warnings, _so_far} ->
         location = [line: line, column: column]

lib/elixir/lib/code/normalizer.ex

@@ -197,6 +197,28 @@ defmodule Code.Normalizer do
     end
   end
 
+  # Module attributes
+  defp do_normalize({:@, meta, [{name, name_meta, [value]}]}, state) do
+    value =
+      cond do
+        keyword?(value) ->
+          normalize_kw_args(value, state, true)
+
+        is_list(value) ->
+          normalize_literal(value, meta, state)
+
+        true ->
+          do_normalize(value, state)
+      end
+
+    {:@, meta, [{name, name_meta, [value]}]}
+  end
+
+  # Regular blocks
+  defp do_normalize({:__block__, meta, args}, state) when is_list(args) do
+    {:__block__, meta, normalize_args(args, state)}
+  end
+
   # Calls
   defp do_normalize({_, _, args} = quoted, state) when is_list(args) do
     normalize_call(quoted, state)

lib/elixir/lib/enum.ex

@@ -2578,6 +2578,7 @@ defmodule Enum do
       [:a, :b, :c, :e, :f, :g, :d]
 
   """
+  @doc since: "1.13.0"
   def slide(enumerable, range_or_single_index, insertion_index)
 
   def slide(enumerable, single_index, insertion_index) when is_integer(single_index) do

lib/elixir/lib/exception.ex

@@ -732,6 +732,10 @@ defmodule ArgumentError do
       ) do
     message =
       cond do
+        not proper_list?(args) ->
+          "you attempted to apply a function named #{inspect(function)} on module #{inspect(module)} " <>
+            "with arguments #{inspect(args)}. Arguments (the third argument of apply) must always be a proper list"
+
         # Note that args may be an empty list even if they were supplied
         not is_atom(module) and is_atom(function) and args == [] ->
           "you attempted to apply a function named #{inspect(function)} on #{inspect(module)}. " <>
@@ -747,10 +751,6 @@ defmodule ArgumentError do
           "you attempted to apply a function named #{inspect(function)} on module #{inspect(module)}. " <>
             "However #{inspect(function)} is not a valid function name. Function names (the second argument " <>
             "of apply) must always be an atom"
-
-        not is_list(args) ->
-          "you attempted to apply a function named #{inspect(function)} on module #{inspect(module)} " <>
-            "with arguments #{inspect(args)}. Arguments (the third argument of apply) must always be a list"
       end
 
     {%{exception | message: message}, stacktrace}
@@ -759,6 +759,9 @@ defmodule ArgumentError do
   def blame(exception, stacktrace) do
     {exception, stacktrace}
   end
+
+  defp proper_list?(list) when length(list) >= 0, do: true
+  defp proper_list?(_), do: false
 end
 
 defmodule ArithmeticError do

lib/elixir/lib/kernel/cli.ex

@@ -218,12 +218,16 @@ defmodule Kernel.CLI do
 
   # Parse shared options
 
-  defp parse_shared([opt | _], _config) when opt in @standalone_opts do
+  defp warn_standalone(opt) do
     IO.puts(:stderr, "#{opt} : Standalone options can't be combined with other options")
+  end
+
+  defp parse_shared([opt | _], _config) when opt in @standalone_opts do
+    warn_standalone(opt)
     System.halt(1)
   end
 
-  defp parse_shared([opt | t], config) when opt in ["-v", "--version"] do
+  defp parse_shared([opt | t], _config) when opt in ["-v", "--version"] do
     if function_exported?(IEx, :started?, 0) and IEx.started?() do
       IO.puts("IEx " <> System.build_info()[:build])
     else
@@ -231,7 +235,8 @@ defmodule Kernel.CLI do
       IO.puts("Elixir " <> System.build_info()[:build])
     end
 
-    parse_shared(t, config)
+    t != [] && warn_standalone(opt)
+    System.halt(0)
   end
 
   defp parse_shared(["-pa", h | t], config) do

lib/elixir/lib/keyword.ex

@@ -1374,27 +1374,9 @@ defmodule Keyword do
     end
   end
 
-  @doc """
-  Maps the function `fun` over all key-value pairs in `keywords`,
-  returning a keyword list with all the values replaced with
-  the result of the function.
-
-  ## Examples
-
-      iex> Keyword.map([one: 1, two: 2, three: 3], fn {_key, val} -> to_string(val) end)
-      [one: "1", two: "2", three: "3"]
-
-  """
-  @doc since: "1.13.0"
-  @spec map(t, ({key, value} -> value)) :: t
-  def map(keywords, fun) when is_list(keywords) and is_function(fun, 1) do
-    do_map(keywords, fun)
-  end
-
-  defp do_map([], _fun), do: []
-
-  defp do_map([{key, value} | rest], fun) do
-    new_value = fun.({key, value})
-    [{key, new_value} | do_map(rest, fun)]
+  @doc false
+  @deprecated "Use Keyword.new/2 instead"
+  def map(keywords, fun) when is_list(keywords) do
+    Enum.map(keywords, fn {k, v} -> {k, fun.({k, v})} end)
   end
 end

lib/elixir/lib/macro.ex

@@ -661,7 +661,9 @@ defmodule Macro do
   dependency on the given struct.
   """
   @doc since: "1.8.0"
-  @spec struct!(module, Macro.Env.t()) :: %{__struct__: module} when module: module()
+  @spec struct!(module, Macro.Env.t()) ::
+          %{required(:__struct__) => module, optional(atom) => any}
+        when module: module()
   def struct!(module, env) when is_atom(module) do
     if module == env.module do
       Module.get_attribute(module, :__struct__)
@@ -943,7 +945,7 @@ defmodule Macro do
   """
   @spec to_string(t()) :: String.t()
   def to_string(tree) do
-    doc = Inspect.Algebra.format(Code.quoted_to_algebra(tree), :infinity)
+    doc = Inspect.Algebra.format(Code.quoted_to_algebra(tree), 98)
     IO.iodata_to_binary(doc)
   end
 

lib/elixir/lib/map.ex

@@ -979,12 +979,14 @@ defmodule Map do
   `fun` receives the key and value of each of the
   elements in the map as a key-value pair.
 
-  `Map.filter/2` is faster than using `map |> Enum.filter(fun) |> Enum.into(%{})`,
-  as no intermediate list is being built.
-
   See also `reject/2` which discards all elements where the
   function returns a truthy value.
 
+  > Note: if you find yourself doing multiple calls to `Map.filter/2`
+  > and `Map.reject/2` in a pipeline, it is likely more efficient
+  > to use `Enum.map/2` and `Enum.filter/2` instead and convert to
+  > a map at the end using `Map.new/1`.
+
   ## Examples
 
       iex> Map.filter(%{one: 1, two: 2, three: 3}, fn {_key, val} -> rem(val, 2) == 1 end)
@@ -1010,9 +1012,8 @@ defmodule Map do
   end
 
   @doc """
-  Returns map excluding the pairs from `map` for which `fun` returns a truthy value.
-  `Map.reject/2` is faster than using `map |> Enum.reject(fun) |> Enum.into(%{})`,
-  as no intermediate list is being built.
+  Returns map excluding the pairs from `map` for which `fun` returns
+  a truthy value.
 
   See also `filter/2`.
 
@@ -1040,28 +1041,9 @@ defmodule Map do
     end
   end
 
-  @doc """
-  Maps the function `fun` over all key-value pairs in `map`, returning a map
-  with all the values replaced with the result of the function.
-
-  ## Examples
-
-      iex> Map.map(%{1 => "joe", 2 => "mike", 3 => "robert"}, fn {_key, val} -> String.capitalize(val) end)
-      %{1 => "Joe", 2 => "Mike", 3 => "Robert"}
-
-  """
-  @doc since: "1.13.0"
-  @spec map(map, ({key, value} -> value)) :: map
-  def map(map, fun) when is_map(map) and is_function(fun, 1) do
-    iter = :maps.iterator(map)
-    next = :maps.next(iter)
-    :maps.from_list(do_map(next, fun))
-  end
-
-  defp do_map(:none, _fun), do: []
-
-  defp do_map({key, value, iter}, fun) do
-    new_value = fun.({key, value})
-    [{key, new_value} | do_map(:maps.next(iter), fun)]
+  @doc false
+  @deprecated "Use Map.new/2 instead"
+  def map(map, fun) when is_map(map) do
+    new(map, fn {k, v} -> {k, fun.({k, v})} end)
   end
 end

lib/elixir/lib/module/types/expr.ex

@@ -469,12 +469,19 @@ defmodule Module.Types.Expr do
   end
 
   defp of_clauses(clauses, stack, context) do
-    reduce_ok(clauses, context, fn {:->, _meta, [head, body]}, context = acc ->
+    reduce_ok(clauses, context, fn {:->, meta, [head, body]}, context = acc ->
       {patterns, guards} = extract_head(head)
 
-      with {:ok, _, context} <- Pattern.of_head(patterns, guards, stack, context),
-           {:ok, _expr_type, context} <- of_expr(body, :dynamic, stack, context),
-           do: {:ok, keep_warnings(acc, context)}
+      case Pattern.of_head(patterns, guards, stack, context) do
+        {:ok, _, context} ->
+          with {:ok, _expr_type, context} <- of_expr(body, :dynamic, stack, context) do
+            {:ok, keep_warnings(acc, context)}
+          end
+
+        error ->
+          # Skip the clause if it the head has an error
+          if meta[:generated], do: {:ok, acc}, else: error
+      end
     end)
   end
 

lib/elixir/lib/process.ex

@@ -383,7 +383,7 @@ defmodule Process do
   @type spawn_opt ::
           :link
           | :monitor
-          | {:monitor, :erlang.monitor_option()}
+          | {:monitor, monitor_option()}
           | {:priority, :low | :normal | :high}
           | {:fullsweep_after, non_neg_integer}
           | {:min_heap_size, non_neg_integer}
@@ -392,6 +392,10 @@ defmodule Process do
           | {:message_queue_data, :off_heap | :on_heap}
   @type spawn_opts :: [spawn_opt]
 
+  # TODO: Use :erlang.monitor_option() on Erlang/OTP 24+
+  @typep monitor_option ::
+           [alias: :explicit_unalias | :demonitor | :reply_demonitor, tag: term()]
+
   @doc """
   Spawns the given function according to the given options.
 

lib/elixir/lib/protocol.ex

@@ -723,14 +723,14 @@ defmodule Protocol do
 
   defp callback_ast_to_fa({kind, {:"::", meta, [{name, _, args}, _return]}, _pos})
        when kind in [:callback, :macrocallback] do
-    [{{name, length(args)}, meta}]
+    [{{name, length(List.wrap(args))}, meta}]
   end
 
   defp callback_ast_to_fa(
          {kind, {:when, _, [{:"::", meta, [{name, _, args}, _return]}, _vars]}, _pos}
        )
        when kind in [:callback, :macrocallback] do
-    [{{name, length(args)}, meta}]
+    [{{name, length(List.wrap(args))}, meta}]
   end
 
   defp callback_ast_to_fa({kind, _, _pos}) when kind in [:callback, :macrocallback] do

lib/elixir/lib/string.ex

@@ -2736,7 +2736,7 @@ defmodule String do
     end
   end
 
-  defp codepoint_byte_size(cp) when cp <= 0x00FF, do: 1
+  defp codepoint_byte_size(cp) when cp <= 0x007F, do: 1
   defp codepoint_byte_size(cp) when cp <= 0x07FF, do: 2
   defp codepoint_byte_size(cp) when cp <= 0xFFFF, do: 3
   defp codepoint_byte_size(_), do: 4

lib/elixir/lib/task.ex

@@ -657,12 +657,14 @@ defmodule Task do
   end
 
   defp build_stream(enumerable, fun, options) do
-    owner = self()
+    fn acc, acc_fun ->
+      owner = get_owner(self())
 
-    &Task.Supervised.stream(enumerable, &1, &2, get_callers(owner), fun, options, fn ->
-      {:ok, pid} = Task.Supervised.start_link(get_owner(owner), :nomonitor)
-      {:ok, :link, pid}
-    end)
+      Task.Supervised.stream(enumerable, acc, acc_fun, get_callers(self()), fun, options, fn ->
+        {:ok, pid} = Task.Supervised.start_link(owner, :nomonitor)
+        {:ok, :link, pid}
+      end)
+    end
   end
 
   # Returns a tuple with the node where this is executed and either the

lib/elixir/lib/task/supervisor.ex

@@ -484,20 +484,22 @@ defmodule Task.Supervisor do
   end
 
   defp build_stream(supervisor, link_type, enumerable, fun, options) do
-    shutdown = options[:shutdown]
-    owner = self()
+    fn acc, acc_fun ->
+      shutdown = options[:shutdown]
+      owner = get_owner(self())
 
-    &Task.Supervised.stream(enumerable, &1, &2, get_callers(owner), fun, options, fn ->
-      args = [get_owner(owner), :monitor]
+      Task.Supervised.stream(enumerable, acc, acc_fun, get_callers(self()), fun, options, fn ->
+        args = [owner, :monitor]
 
-      case start_child_with_spec(supervisor, args, :temporary, shutdown) do
-        {:ok, pid} ->
-          if link_type == :link, do: Process.link(pid)
-          {:ok, link_type, pid}
+        case start_child_with_spec(supervisor, args, :temporary, shutdown) do
+          {:ok, pid} ->
+            if link_type == :link, do: Process.link(pid)
+            {:ok, link_type, pid}
 
-        {:error, :max_children} ->
-          {:error, :max_children}
-      end
-    end)
+          {:error, :max_children} ->
+            {:error, :max_children}
+        end
+      end)
+    end
   end
 end

lib/elixir/lib/uri.ex

@@ -12,7 +12,7 @@ defmodule URI do
   """
 
   defstruct scheme: nil,
-            path: "",
+            path: nil,
             query: nil,
             fragment: nil,
             authority: nil,
@@ -24,7 +24,7 @@ defmodule URI do
           authority: authority,
           fragment: nil | binary,
           host: nil | binary,
-          path: binary,
+          path: nil | binary,
           port: nil | :inet.port_number(),
           query: nil | binary,
           scheme: nil | binary,
@@ -479,12 +479,13 @@ defmodule URI do
   Creates a new URI struct from a URI or a string.
 
   If a `%URI{}` struct is given, it returns `{:ok, uri}`. If a string is
-  given, it will parse it and returns `{:ok, uri}`. If the string is
-  invalid, it returns `{:error, part}` instead, with the invalid part of the URI.
+  given, it will parse and validate it. If the string is valid, it returns
+  `{:ok, uri}`, otherwise it returns `{:error, part}` with the invalid part
+  of the URI. For parsing URIs without further validation, see `parse/1`.
 
   This function can parse both absolute and relative URLs. You can check
   if a URI is absolute or relative by checking if the `scheme` field is
-  `nil` or not. All fields may be `nil`, except for the `path`.
+  `nil` or not.
 
   When a URI is given without a port, the value returned by `URI.default_port/1`
   for the URI's scheme is used for the `:port` field. The scheme is also
@@ -551,7 +552,7 @@ defmodule URI do
       {:ok, %URI{
         fragment: nil,
         host: nil,
-        path: "",
+        path: nil,
         port: 443,
         query: "query",
         scheme: "https",
@@ -633,6 +634,8 @@ defmodule URI do
     end
   end
 
+  defp uri_from_map(%{path: ""} = map), do: uri_from_map(%{map | path: nil})
+
   defp uri_from_map(map) do
     uri = Map.merge(%URI{}, map)
 
@@ -657,29 +660,106 @@ defmodule URI do
   end
 
   @doc """
-  Parses a well-formed URI into its components.
+  Parses a URI into its components, without further validation.
 
-  This function is deprecated as it fails to raise in case of invalid URIs.
-  Use `URI.new!/1` or `URI.new/1` instead. In case you want to mimic the
-  behaviour of this function, you can do:
+  This function can parse both absolute and relative URLs. You can check
+  if a URI is absolute or relative by checking if the `scheme` field is
+  nil or not. Furthermore, this function expects both absolute and
+  relative URIs to be well-formed and does not perform any validation.
+  See the "Examples" section below. Use `new/1` if you want more strict
+  validation.
 
-      case URI.new(path) do
-        {:ok, uri} -> uri
-        {:error, _} -> %URI{path: path}
-      end
+  When a URI is given without a port, the value returned by `URI.default_port/1`
+  for the URI's scheme is used for the `:port` field. The scheme is also
+  normalized to lowercase.
 
-  There are two differencws in the behaviour of this function compared to
-  `URI.new/1`:
+  If a `%URI{}` struct is given to this function, this function returns it
+  unmodified.
 
-    * This function sets the deprecated authority field
+  > Note: this function sets the field :authority for backwards
+  > compatibility reasons but it is deprecated.
 
-    * This function sets the path to `nil` when it is empty,
-      while `new/1` consider the path always exists and sets it
-      to an empty string
+  ## Examples
+
+      iex> URI.parse("https://elixir-lang.org/")
+      %URI{
+        authority: "elixir-lang.org",
+        fragment: nil,
+        host: "elixir-lang.org",
+        path: "/",
+        port: 443,
+        query: nil,
+        scheme: "https",
+        userinfo: nil
+      }
+
+      iex> URI.parse("//elixir-lang.org/")
+      %URI{
+        authority: "elixir-lang.org",
+        fragment: nil,
+        host: "elixir-lang.org",
+        path: "/",
+        port: nil,
+        query: nil,
+        scheme: nil,
+        userinfo: nil
+      }
+
+      iex> URI.parse("/foo/bar")
+      %URI{
+        authority: nil,
+        fragment: nil,
+        host: nil,
+        path: "/foo/bar",
+        port: nil,
+        query: nil,
+        scheme: nil,
+        userinfo: nil
+      }
+
+      iex> URI.parse("foo/bar")
+      %URI{
+        authority: nil,
+        fragment: nil,
+        host: nil,
+        path: "foo/bar",
+        port: nil,
+        query: nil,
+        scheme: nil,
+        userinfo: nil
+      }
+
+  In contrast to `URI.new/1`, this function will parse poorly-formed
+  URIs, for example:
+
+      iex> URI.parse("/invalid_greater_than_in_path/>")
+      %URI{
+        authority: nil,
+        fragment: nil,
+        host: nil,
+        path: "/invalid_greater_than_in_path/>",
+        port: nil,
+        query: nil,
+        scheme: nil,
+        userinfo: nil
+      }
+
+  Another example is a URI with brackets in query strings. It is accepted
+  by `parse/1` but it will be refused by `new/1`:
+
+      iex> URI.parse("/?foo[bar]=baz")
+      %URI{
+        authority: nil,
+        fragment: nil,
+        host: nil,
+        path: "/",
+        port: nil,
+        query: "foo[bar]=baz",
+        scheme: nil,
+        userinfo: nil
+      }
 
   """
-  # TODO: Deprecate me at least on v1.17
-  @doc deprecated: "Use URI.new/1 or URI.new!/1 instead"
   @spec parse(t | binary) :: t
   def parse(%URI{} = uri), do: uri
 
@@ -812,6 +892,7 @@ defmodule URI do
     %{rel | scheme: base.scheme, path: remove_dot_segments_from_path(rel.path)}
   end
 
+  # TODO: Check only for nils in future versions
   def merge(%URI{} = base, %URI{path: rel_path} = rel) when rel_path in ["", nil] do
     %{base | query: rel.query || base.query, fragment: rel.fragment}
   end
@@ -825,9 +906,7 @@ defmodule URI do
     merge(parse(base), parse(rel))
   end
 
-  # TODO: Deprecate me on Elixir v1.19
   defp merge_paths(nil, rel_path), do: merge_paths("/", rel_path)
-  defp merge_paths("", rel_path), do: merge_paths("/", rel_path)
   defp merge_paths(_, "/" <> _ = rel_path), do: remove_dot_segments_from_path(rel_path)
 
   defp merge_paths(base_path, rel_path) do
@@ -867,7 +946,8 @@ end
 
 defimpl String.Chars, for: URI do
   def to_string(%{host: host, path: path} = uri)
-      when host != nil and path != "" and binary_part(path, 0, 1) != "/" do
+      when host != nil and is_binary(path) and
+             path != "" and binary_part(path, 0, 1) != "/" do
     raise ArgumentError,
           ":path in URI must be empty or an absolute path if URL has a :host, got: #{inspect(uri)}"
   end

lib/elixir/lib/version.ex

@@ -440,7 +440,7 @@ defmodule Version do
   The internal representation is opaque.
   """
   @spec compile_requirement(Requirement.t()) :: Requirement.t()
-  def compile_requirement(requirement) do
+  def compile_requirement(%Requirement{} = requirement) do
     requirement
   end
 

lib/elixir/src/elixir.erl

@@ -359,8 +359,8 @@ string_to_tokens(String, StartLine, StartColumn, File, Opts) when is_integer(Sta
       {error, {Location, to_binary(Error), to_binary(Token)}}
   end.
 
-tokens_to_quoted(Tokens, File, Opts) ->
-  handle_parsing_opts(File, Opts),
+tokens_to_quoted(Tokens, WarningFile, Opts) ->
+  handle_parsing_opts(WarningFile, Opts),
 
   try elixir_parser:parse(Tokens) of
     {ok, Forms} ->
@@ -370,7 +370,7 @@ tokens_to_quoted(Tokens, File, Opts) ->
     {error, {Line, _, [Error, Token]}} ->
       {error, {parser_location(Line), to_binary(Error), to_binary(Token)}}
   after
-    erase(elixir_parser_file),
+    erase(elixir_parser_warning_file),
     erase(elixir_parser_columns),
     erase(elixir_token_metadata),
     erase(elixir_literal_encoder)
@@ -406,7 +406,7 @@ parser_location(Meta) ->
 to_binary(List) when is_list(List) -> elixir_utils:characters_to_binary(List);
 to_binary(Atom) when is_atom(Atom) -> atom_to_binary(Atom, utf8).
 
-handle_parsing_opts(File, Opts) ->
+handle_parsing_opts(WarningFile, Opts) ->
   LiteralEncoder =
     case lists:keyfind(literal_encoder, 1, Opts) of
       {literal_encoder, Fun} -> Fun;
@@ -414,7 +414,7 @@ handle_parsing_opts(File, Opts) ->
     end,
   TokenMetadata = lists:keyfind(token_metadata, 1, Opts) == {token_metadata, true},
   Columns = lists:keyfind(columns, 1, Opts) == {columns, true},
-  put(elixir_parser_file, File),
+  put(elixir_parser_warning_file, WarningFile),
   put(elixir_parser_columns, Columns),
   put(elixir_token_metadata, TokenMetadata),
   put(elixir_literal_encoder, LiteralEncoder).

lib/elixir/src/elixir_compiler.erl

@@ -133,7 +133,8 @@ fast_compile({defmodule, Meta, [Mod, [{do, TailBlock}]]}, NoLineE) ->
       'Elixir.Macro':expand(Mod, E)
   end,
 
-  elixir_module:compile(Expanded, Block, [], E).
+  ContextModules = [Expanded | ?key(E, context_modules)],
+  elixir_module:compile(Expanded, Block, [], E#{context_modules := ContextModules}).
 
 %% Bootstrapper
 

lib/elixir/src/elixir_erl_for.erl

@@ -21,7 +21,7 @@ translate_reduce(Meta, Cases, Expr, Reduce, S) ->
     ({'case', CaseAnn, _, CaseBlock}, InnerAcc) -> {'case', CaseAnn, InnerAcc, CaseBlock}
   end,
 
-  {build_reduce(Ann, TCases, InnerFun, TExpr, TReduce, false, SE), SE}.
+  build_reduce(Ann, TCases, InnerFun, TExpr, TReduce, false, SE).
 
 translate_into(Meta, Cases, Expr, Opts, Return, S) ->
   Ann = ?ann(Meta),
@@ -115,11 +115,11 @@ build_inline(Ann, Clauses, Expr, Into, Uniq, S) ->
 
 build_inline_each(Ann, Clauses, Expr, false, Uniq, S) ->
   InnerFun = fun(InnerExpr, _InnerAcc) -> InnerExpr end,
-  {build_reduce(Ann, Clauses, InnerFun, Expr, {nil, Ann}, Uniq, S), S};
+  build_reduce(Ann, Clauses, InnerFun, Expr, {nil, Ann}, Uniq, S);
 build_inline_each(Ann, Clauses, Expr, {nil, _} = Into, Uniq, S) ->
   InnerFun = fun(InnerExpr, InnerAcc) -> {cons, Ann, InnerExpr, InnerAcc} end,
-  ReduceExpr = build_reduce(Ann, Clauses, InnerFun, Expr, Into, Uniq, S),
-  {?remote(Ann, lists, reverse, [ReduceExpr]), S};
+  {ReduceExpr, SR} = build_reduce(Ann, Clauses, InnerFun, Expr, Into, Uniq, S),
+  {?remote(Ann, lists, reverse, [ReduceExpr]), SR};
 build_inline_each(Ann, Clauses, Expr, {bin, _, []}, Uniq, S) ->
   {InnerValue, SV} = build_var(Ann, S),
   Generated = erl_anno:set_generated(true, Ann),
@@ -138,8 +138,8 @@ build_inline_each(Ann, Clauses, Expr, {bin, _, []}, Uniq, S) ->
     ]}
   end,
 
-  ReduceExpr = build_reduce(Ann, Clauses, InnerFun, Expr, {nil, Ann}, Uniq, SV),
-  {?remote(Ann, erlang, list_to_bitstring, [ReduceExpr]), SV}.
+  {ReduceExpr, SR} = build_reduce(Ann, Clauses, InnerFun, Expr, {nil, Ann}, Uniq, SV),
+  {?remote(Ann, erlang, list_to_bitstring, [ReduceExpr]), SR}.
 
 build_into(Ann, Clauses, Expr, {map, _, []}, Uniq, S) ->
   {ReduceExpr, SR} = build_inline_each(Ann, Clauses, Expr, {nil, Ann}, Uniq, S),
@@ -161,7 +161,7 @@ build_into(Ann, Clauses, Expr, Into, Uniq, S) ->
     ?remote(Ann, 'Elixir.Collectable', into, [Into])
   },
 
-  IntoReduceExpr = build_reduce(Ann, Clauses, InnerFun, Expr, Acc, Uniq, SD),
+  {IntoReduceExpr, SN} = build_reduce(Ann, Clauses, InnerFun, Expr, Acc, Uniq, SD),
 
   TryExpr =
     {'try', Ann,
@@ -173,7 +173,7 @@ build_into(Ann, Clauses, Expr, Into, Uniq, S) ->
       [stacktrace_clause(Ann, Fun, Acc, Kind, Reason, Stack)],
       []},
 
-  {{block, Ann, [MatchExpr, TryExpr]}, SD}.
+  {{block, Ann, [MatchExpr, TryExpr]}, SN}.
 
 stacktrace_clause(Ann, Fun, Acc, Kind, Reason, Stack) ->
   {clause, Ann,
@@ -186,7 +186,7 @@ stacktrace_clause(Ann, Fun, Acc, Kind, Reason, Stack) ->
 
 build_reduce(Ann, Clauses, InnerFun, Expr, Into, false, S) ->
   {Acc, SA} = build_var(Ann, S),
-  build_reduce_each(Clauses, InnerFun(Expr, Acc), Into, Acc, SA);
+  {build_reduce_each(Clauses, InnerFun(Expr, Acc), Into, Acc, SA), SA};
 build_reduce(Ann, Clauses, InnerFun, Expr, Into, true, S) ->
   %% Those variables are used only inside the anonymous function
   %% so we don't need to worry about returning the scope.
@@ -209,7 +209,7 @@ build_reduce(Ann, Clauses, InnerFun, Expr, Into, true, S) ->
   ]},
 
   EnumReduceCall = build_reduce_each(Clauses, InnerExpr, NewInto, Acc, SU),
-  ?remote(Ann, erlang, element, [{integer, Ann, 1}, EnumReduceCall]).
+  {?remote(Ann, erlang, element, [{integer, Ann, 1}, EnumReduceCall]), SU}.
 
 build_reduce_each([{enum, Meta, Left, Right, Filters} | T], Expr, Arg, Acc, S) ->
   Ann = ?ann(Meta),

lib/elixir/src/elixir_errors.erl

@@ -87,9 +87,12 @@ snippet(InputString, Location, StartLine, StartColumn) ->
   case lists:keyfind(column, 1, Location) of
     {column, Column} ->
       Lines = string:split(InputString, "\n", all),
-      Snippet = elixir_utils:characters_to_binary(lists:nth(Line - StartLine + 1, Lines)),
+      Snippet = (lists:nth(Line - StartLine + 1, Lines)),
       Offset = if Line == StartLine -> Column - StartColumn; true -> Column - 1 end,
-      #{content => Snippet, offset => Offset};
+      case string:trim(Snippet, leading) of
+        [] -> nil;
+        _ -> #{content => elixir_utils:characters_to_binary(Snippet), offset => Offset}
+      end;
 
     false ->
       nil

lib/elixir/src/elixir_parser.yrl

@@ -552,6 +552,7 @@ kw_base -> kw_eol container_expr : [{'$1', '$2'}].
 kw_base -> kw_base ',' kw_eol container_expr : [{'$3', '$4'} | '$1'].
 
 kw_call -> kw_base : reverse('$1').
+kw_call -> kw_base ',' : warn_trailing_comma('$2'), reverse('$1').
 kw_call -> kw_base ',' matched_expr : maybe_bad_keyword_call_follow_up('$2', '$1', '$3').
 
 kw_data -> kw_base : reverse('$1').
@@ -639,7 +640,6 @@ map -> struct_op struct_expr eol map_args : {'%', meta_from_token('$1'), ['$2',
 
 Erlang code.
 
--define(file(), get(elixir_parser_file)).
 -define(columns(), get(elixir_parser_columns)).
 -define(token_metadata(), get(elixir_token_metadata)).
 
@@ -728,10 +728,12 @@ build_op(AST, {_Kind, Location, '//'}, Right) ->
 
 build_op({UOp, _, [Left]}, {_Kind, {Line, Column, _} = Location, 'in'}, Right) when ?rearrange_uop(UOp) ->
   %% TODO: Remove "not left in right" rearrangement on v2.0
-  elixir_errors:erl_warn({Line, Column}, ?file(),
+  warn(
+    {Line, Column},
     "\"not expr1 in expr2\" is deprecated. "
     "Instead use \"expr1 not in expr2\" if you require Elixir v1.5+, "
-    "or \"not(expr1 in expr2)\" if you have to support earlier Elixir versions"),
+    "or \"not(expr1 in expr2)\" if you have to support earlier Elixir versions"
+  ),
   Meta = meta_from_location(Location),
   {UOp, Meta, [{'in', Meta, [Left, Right]}]};
 
@@ -1166,17 +1168,24 @@ error_invalid_kw_identifier({_, Location, do}) ->
 error_invalid_kw_identifier({_, Location, KW}) ->
   return_error(Location, "syntax error before: ", "'" ++ atom_to_list(KW) ++ ":'").
 
+%% TODO: Make this an error on v2.0
+warn_trailing_comma({',', {Line, Column, _}}) ->
+  warn({Line, Column}, "trailing commas are not allowed inside function/macro call arguments").
+
 %% TODO: Make this an error on v2.0
 warn_empty_paren({_, {Line, Column, _}}) ->
-  elixir_errors:erl_warn({Line, Column}, ?file(),
+  warn(
+    {Line, Column},
     "invalid expression (). "
     "If you want to invoke or define a function, make sure there are "
     "no spaces between the function name and its arguments. If you wanted "
-    "to pass an empty block or code, pass a value instead, such as a nil or an atom").
+    "to pass an empty block or code, pass a value instead, such as a nil or an atom"
+  ).
 
 %% TODO: Make this an error on v2.0
 warn_pipe({arrow_op, {Line, Column, _}, Op}, {_, [_ | _], [_ | _]}) ->
-  elixir_errors:erl_warn({Line, Column}, ?file(),
+  warn(
+    {Line, Column},
     io_lib:format(
       "parentheses are required when piping into a function call. For example:\n\n"
       "    foo 1 ~ts bar 2 ~ts baz 3\n\n"
@@ -1190,6 +1199,14 @@ warn_pipe(_Token, _) ->
   ok.
 
 warn_empty_stab_clause({stab_op, {Line, Column, _}, '->'}) ->
-  elixir_errors:erl_warn({Line, Column}, ?file(),
+  warn(
+    {Line, Column},
     "an expression is always required on the right side of ->. "
-    "Please provide a value after ->").
+    "Please provide a value after ->"
+  ).
+
+warn(LineColumn, Message) ->
+  case get(elixir_parser_warning_file) of
+    nil -> ok;
+    File -> elixir_errors:erl_warn(LineColumn, File, Message)
+  end.

lib/elixir/test/elixir/base_test.exs

@@ -799,10 +799,21 @@ defmodule BaseTest do
         |> Enum.shuffle()
         |> IO.iodata_to_binary()
 
+      allowed_opts =
+        encode
+        |> Function.info()
+        |> Keyword.fetch!(:name)
+        |> case do
+          :encode16 -> [:case]
+          :encode64 -> [:padding]
+          :url_encode64 -> [:padding]
+          _ -> [:case, :padding]
+        end
+
       expected =
         data
-        |> encode.(case: encode_case, pad: pad?)
-        |> decode.(case: decode_case, pad: pad?)
+        |> encode.(Keyword.take([case: encode_case, padding: pad?], allowed_opts))
+        |> decode.(Keyword.take([case: decode_case, padding: pad?], allowed_opts))
 
       assert data == expected,
              "identity did not match for #{inspect(data)} when #{inspect(encode)} (#{encode_case})"

lib/elixir/test/elixir/code_fragment_test.exs

@@ -915,6 +915,11 @@ defmodule CodeFragmentTest do
       assert cc2q("foo(123, ~r/") == s2q("foo(123, __cursor__())")
     end
 
+    test "no warnings" do
+      assert cc2q(~s"?\\ ") == s2q("__cursor__()")
+      assert cc2q(~s"{fn -> end, ") == s2q("{fn -> nil end, __cursor__()}")
+    end
+
     test "options" do
       opts = [columns: true]
       assert cc2q("foo(", opts) == s2q("foo(__cursor__())", opts)

lib/elixir/test/elixir/code_normalizer/formatted_ast_test.exs

@@ -164,6 +164,12 @@ defmodule Code.Normalizer.FormatterASTTest do
     end
   end
 
+  describe "lists" do
+    test "on module attribute" do
+      assert_same ~S"@foo [1]"
+    end
+  end
+
   describe "charlists" do
     test "without escapes" do
       assert_same ~S['']
@@ -352,6 +358,18 @@ defmodule Code.Normalizer.FormatterASTTest do
       assert_same ~S"{:wrapped, 1, [opt1: true, opt2: false]}"
       assert_same ~S"{:unwrapped, 1, opt1: true, opt2: false}"
     end
+
+    test "on module attribute" do
+      assert_same ~S"""
+      @foo a: b,
+           c: d
+      """
+
+      assert_same ~S"@foo [
+        a: b,
+        c: d
+      ]"
+    end
   end
 
   describe "preserves user choice on parenthesis" do
@@ -542,5 +560,12 @@ defmodule Code.Normalizer.FormatterASTTest do
       end
       """
     end
+
+    test "keyword literals with variable values" do
+      assert_same(~S"""
+      foo = foo()
+      [foo: foo]
+      """)
+    end
   end
 end

lib/elixir/test/elixir/code_normalizer/quoted_ast_test.exs

@@ -542,6 +542,26 @@ defmodule Code.Normalizer.QuotedASTTest do
       assert quoted_to_string(quote(do: foo(catch: a))) == "foo(catch: a)"
       assert quoted_to_string(quote(do: foo |> [bar: :baz])) == "foo |> [bar: :baz]"
     end
+
+    test "list in module attribute" do
+      assert quoted_to_string(
+               quote do
+                 @foo [1]
+               end
+             ) == "@foo [1]"
+
+      assert quoted_to_string(
+               quote do
+                 @foo [foo: :bar]
+               end
+             ) == "@foo foo: :bar"
+
+      assert quoted_to_string(
+               quote do
+                 @foo [1, foo: :bar]
+               end
+             ) == "@foo [1, foo: :bar]"
+    end
   end
 
   describe "quoted_to_algebra/2 escapes" do

lib/elixir/test/elixir/exception_test.exs

@@ -423,7 +423,11 @@ defmodule ExceptionTest do
 
       assert blame_message(123, &apply(Kernel, :+, &1)) ==
                "you attempted to apply a function named :+ on module Kernel with arguments 123. " <>
-                 "Arguments (the third argument of apply) must always be a list"
+                 "Arguments (the third argument of apply) must always be a proper list"
+
+      assert blame_message(123, &apply(Kernel, :+, [&1 | 456])) ==
+               "you attempted to apply a function named :+ on module Kernel with arguments [123 | 456]. " <>
+                 "Arguments (the third argument of apply) must always be a proper list"
     end
 
     test "annotates function clause errors" do

lib/elixir/test/elixir/kernel/cli_test.exs

@@ -71,10 +71,14 @@ defmodule Kernel.CLITest do
     assert output =~ "Erlang/OTP #{System.otp_release()}"
     assert output =~ "Elixir #{System.version()}"
 
+    output = iex('--version')
+    assert output =~ "Erlang/OTP #{System.otp_release()}"
+    assert output =~ "IEx #{System.version()}"
+
     output = elixir('--version -e "IO.puts(:test_output)"')
     assert output =~ "Erlang/OTP #{System.otp_release()}"
     assert output =~ "Elixir #{System.version()}"
-    assert output =~ "test_output"
+    assert output =~ "Standalone options can't be combined with other options"
   end
 
   test "--short-version smoke test" do

lib/elixir/test/elixir/kernel/comprehension_test.exs

@@ -98,6 +98,19 @@ defmodule Kernel.ComprehensionTest do
     assert Process.get(:into_halt)
   end
 
+  test "nested for comprehensions with unique values" do
+    assert for(x <- [1, 1, 2], uniq: true, do: for(y <- [3, 3], uniq: true, do: x * y)) == [
+             [3],
+             [6]
+           ]
+
+    assert for(<<x <- "abcabc">>,
+             uniq: true,
+             into: "",
+             do: for(<<y <- "zz">>, uniq: true, into: "", do: to_bin(x) <> to_bin(y))
+           ) == "azbzcz"
+  end
+
   test "for comprehensions with nilly filters" do
     assert for(x <- 1..3, nilly(), do: x * 2) == []
   end

lib/elixir/test/elixir/kernel/errors_test.exs

@@ -491,6 +491,21 @@ defmodule Kernel.ErrorsTest do
     assert_eval_raise TokenMissingError, ~r/nofile:1:3: invalid escape \\ at end of file/, '1 \\'
   end
 
+  test "show snippet on missing tokens" do
+    assert_eval_raise TokenMissingError,
+                      "nofile:1:25: missing terminator: end (for \"do\" starting at line 1)\n" <>
+                        "    |\n" <>
+                        "  1 | defmodule ShowSnippet do\n" <>
+                        "    |                         ^",
+                      'defmodule ShowSnippet do'
+  end
+
+  test "don't show snippet when error line is empty" do
+    assert_eval_raise TokenMissingError,
+                      "nofile:3:1: missing terminator: end (for \"do\" starting at line 1)",
+                      'defmodule ShowSnippet do\n\n'
+  end
+
   test "function local conflict" do
     assert_eval_raise CompileError,
                       "nofile:3: imported Kernel.&&/2 conflicts with local function",

lib/elixir/test/elixir/kernel/warning_test.exs

@@ -1163,9 +1163,11 @@ defmodule Kernel.WarningTest do
 
             def without_specs(term, options \\ [])
 
+            @callback foo :: {:ok, term}
             @callback foo(term) :: {:ok, term}
             @callback foo(term, keyword) :: {:ok, term, keyword}
 
+            @callback foo_when :: {:ok, x} when x: term
             @callback foo_when(x) :: {:ok, x} when x: term
             @callback foo_when(x, opts) :: {:ok, x, opts} when x: term, opts: keyword
 
@@ -1178,12 +1180,18 @@ defmodule Kernel.WarningTest do
         """)
       end)
 
+    assert message =~
+             "cannot define @callback foo/0 inside protocol, use def/1 to outline your protocol definition\n  nofile:1"
+
     assert message =~
              "cannot define @callback foo/1 inside protocol, use def/1 to outline your protocol definition\n  nofile:1"
 
     assert message =~
              "cannot define @callback foo/2 inside protocol, use def/1 to outline your protocol definition\n  nofile:1"
 
+    assert message =~
+             "cannot define @callback foo_when/0 inside protocol, use def/1 to outline your protocol definition\n  nofile:1"
+
     assert message =~
              "cannot define @callback foo_when/1 inside protocol, use def/1 to outline your protocol definition\n  nofile:1"
 
@@ -1901,6 +1909,11 @@ defmodule Kernel.WarningTest do
     end
   end
 
+  test "warnings on trailing comma on call" do
+    assert capture_err(fn -> Code.eval_string("Keyword.merge([], foo: 1,)") end) =~
+             "trailing commas are not allowed inside function/macro call arguments"
+  end
+
   test "defstruct warns with duplicate keys" do
     assert capture_err(fn ->
              Code.eval_string("""

lib/elixir/test/elixir/kernel_test.exs

@@ -723,8 +723,12 @@ defmodule KernelTest do
       result = expand_to_string(quote(do: rand() in 1..2))
       assert result =~ "var = rand()"
 
-      assert result =~
-               ":erlang.andalso(:erlang.is_integer(var), :erlang.andalso(:erlang.>=(var, 1), :erlang.\"=<\"(var, 2)))"
+      assert result =~ """
+             :erlang.andalso(
+               :erlang.is_integer(var),
+               :erlang.andalso(:erlang.>=(var, 1), :erlang.\"=<\"(var, 2))
+             )\
+             """
 
       # Empty list
       assert expand_to_string(quote(do: :x in [])) =~ "_ = :x\nfalse"
@@ -754,14 +758,26 @@ defmodule KernelTest do
       assert expand_to_string(quote(do: foo in [])) ==
                "_ = foo\nfalse"
 
-      assert expand_to_string(quote(do: foo in [1, 2, 3])) ==
-               ":erlang.orelse(:erlang.orelse(:erlang.\"=:=\"(foo, 1), :erlang.\"=:=\"(foo, 2)), :erlang.\"=:=\"(foo, 3))"
+      assert expand_to_string(quote(do: foo in [1, 2, 3])) == """
+             :erlang.orelse(
+               :erlang.orelse(:erlang.\"=:=\"(foo, 1), :erlang.\"=:=\"(foo, 2)),
+               :erlang.\"=:=\"(foo, 3)
+             )\
+             """
 
-      assert expand_to_string(quote(do: foo in 0..1)) ==
-               ":erlang.andalso(:erlang.is_integer(foo), :erlang.andalso(:erlang.>=(foo, 0), :erlang.\"=<\"(foo, 1)))"
+      assert expand_to_string(quote(do: foo in 0..1)) == """
+             :erlang.andalso(
+               :erlang.is_integer(foo),
+               :erlang.andalso(:erlang.>=(foo, 0), :erlang.\"=<\"(foo, 1))
+             )\
+             """
 
-      assert expand_to_string(quote(do: foo in -1..0)) ==
-               ":erlang.andalso(:erlang.is_integer(foo), :erlang.andalso(:erlang.>=(foo, -1), :erlang.\"=<\"(foo, 0)))"
+      assert expand_to_string(quote(do: foo in -1..0)) == """
+             :erlang.andalso(
+               :erlang.is_integer(foo),
+               :erlang.andalso(:erlang.>=(foo, -1), :erlang.\"=<\"(foo, 0))
+             )\
+             """
 
       assert expand_to_string(quote(do: foo in 1..1)) ==
                ":erlang.\"=:=\"(foo, 1)"

lib/elixir/test/elixir/macro_test.exs

@@ -810,6 +810,14 @@ defmodule MacroTest do
       defmodule Foo.Bar do
         assert __MODULE__ in __ENV__.context_modules
       end
+
+      assert Foo.Bar in __ENV__.context_modules
+
+      Code.compile_string("""
+      defmodule Foo.Bar.Compiled do
+        true = __MODULE__ in __ENV__.context_modules
+      end
+      """)
     end
 
     test "to_match/1" do

lib/elixir/test/elixir/module/types/types_test.exs

@@ -17,6 +17,10 @@ defmodule Module.Types.TypesTest do
     end
   end
 
+  defmacro generated(ast) do
+    Macro.prewalk(ast, fn node -> Macro.update_meta(node, &([generated: true] ++ &1)) end)
+  end
+
   def __expr__({patterns, guards, body}) do
     with {:ok, _types, context} <-
            Pattern.of_head(patterns, guards, TypeHelper.new_stack(), TypeHelper.new_context()),
@@ -213,6 +217,32 @@ defmodule Module.Types.TypesTest do
              """
     end
 
+    test "warns on guards from cases unless generated" do
+      string =
+        warning(
+          [var],
+          [is_integer(var)],
+          case var do
+            _ when is_binary(var) -> :ok
+          end
+        )
+
+      assert is_binary(string)
+
+      string =
+        generated(
+          warning(
+            [var],
+            [is_integer(var)],
+            case var do
+              _ when is_binary(var) -> :ok
+            end
+          )
+        )
+
+      assert string == :none
+    end
+
     test "only show relevant traces in warning" do
       string = warning([x = y, z], [is_integer(x) and is_binary(y) and is_boolean(z)], {x, y, z})
 

lib/elixir/test/elixir/string_test.exs

@@ -652,6 +652,7 @@ defmodule StringTest do
     assert String.slice("あいうえお", -10..-15) == ""
     assert String.slice("hello あいうえお Unicode", 8..-1) == "うえお Unicode"
     assert String.slice("abc", -1..14) == "c"
+    assert String.slice("a·̀ͯ‿.⁀:", 0..-2) == "a·̀ͯ‿.⁀"
 
     assert_raise FunctionClauseError, fn ->
       String.slice(nil, 0..1)

lib/elixir/test/elixir/task/supervisor_test.exs

@@ -450,6 +450,15 @@ defmodule Task.SupervisorTest do
 
       assert_receive :done
     end
+
+    test "consuming from another process", config do
+      parent = self()
+      stream = Task.Supervisor.async_stream(config[:supervisor], [1, 2, 3], &send(parent, &1))
+      Task.start(Stream, :run, [stream])
+      assert_receive 1
+      assert_receive 2
+      assert_receive 3
+    end
   end
 
   describe "async_stream_nolink" do

lib/elixir/test/elixir/task_test.exs

@@ -829,6 +829,15 @@ defmodule TaskTest do
 
       assert_receive :done
     end
+
+    test "consuming from another process" do
+      parent = self()
+      stream = Task.async_stream([1, 2, 3], &send(parent, &1))
+      Task.start(Stream, :run, [stream])
+      assert_receive 1
+      assert_receive 2
+      assert_receive 3
+    end
   end
 
   for {desc, concurrency} <- [==: 4, <: 2, >: 8] do

lib/elixir/test/elixir/test_helper.exs

@@ -39,6 +39,14 @@ defmodule PathHelpers do
     executable_path("elixirc")
   end
 
+  def iex(args) do
+    run_cmd(iex_executable(), args)
+  end
+
+  def iex_executable do
+    executable_path("iex")
+  end
+
   def write_beam({:module, name, bin, _} = res) do
     File.mkdir_p!(unquote(path))
     beam_path = Path.join(unquote(path), Atom.to_string(name) <> ".beam")

lib/elixir/test/elixir/uri_test.exs

@@ -94,7 +94,7 @@ defmodule URITest do
 
   describe "new/1" do
     test "empty" do
-      assert URI.new("") == {:ok, %URI{path: ""}}
+      assert URI.new("") == {:ok, %URI{}}
     end
 
     test "errors on bad URIs" do
@@ -131,7 +131,7 @@ defmodule URITest do
         query: nil,
         fragment: nil,
         port: 443,
-        path: "",
+        path: nil,
         userinfo: nil
       }
 
@@ -349,9 +349,9 @@ defmodule URITest do
     assert URI.merge(base, "") |> to_string == "http://example.com/foo/bar"
     assert URI.merge(base, "#fragment") |> to_string == "http://example.com/foo/bar#fragment"
     assert URI.merge(base, "?query") |> to_string == "http://example.com/foo/bar?query"
-    assert URI.merge(base, %URI{path: ""}) |> to_string == "http://example.com/foo/bar"
+    assert URI.merge(base, %URI{}) |> to_string == "http://example.com/foo/bar"
 
-    assert URI.merge(base, %URI{path: "", fragment: "fragment"})
+    assert URI.merge(base, %URI{fragment: "fragment"})
            |> to_string == "http://example.com/foo/bar#fragment"
 
     base = URI.new!("http://example.com")

lib/elixir/test/elixir/version_test.exs

@@ -308,6 +308,15 @@ defmodule VersionTest do
     assert Version.match?("0.7.0", req)
   end
 
+  test "compile_requirement/1" do
+    {:ok, req} = Version.parse_requirement("1.2.3")
+    assert req == Version.compile_requirement(req)
+
+    assert_raise(FunctionClauseError, fn ->
+      Version.compile_requirement("~> 1.2.3")
+    end)
+  end
+
   test "compile requirement" do
     {:ok, req} = Version.parse_requirement("1.2.3")
     req = Version.compile_requirement(req)

lib/mix/lib/mix.ex

@@ -663,9 +663,10 @@ defmodule Mix do
 
     config = Keyword.get(opts, :config, [])
     system_env = Keyword.get(opts, :system_env, [])
+    consolidate_protocols? = Keyword.get(opts, :consolidate_protocols, true)
 
     id =
-      {deps, config, system_env}
+      {deps, config, system_env, consolidate_protocols?}
       |> :erlang.term_to_binary()
       |> :erlang.md5()
       |> Base.encode16(case: :lower)
@@ -703,7 +704,7 @@ defmodule Mix do
           erlc_paths: ["src"],
           elixirc_paths: ["lib"],
           compilers: [],
-          consolidate_protocols: Keyword.get(opts, :consolidate_protocols, true)
+          consolidate_protocols: consolidate_protocols?
         ]
 
         started_apps = Application.started_applications()

lib/mix/lib/mix/compilers/elixir.ex

@@ -725,37 +725,78 @@ defmodule Mix.Compilers.Elixir do
 
     for %{scm: scm, opts: opts} = dep <- Mix.Dep.cached(),
         not scm.fetchable?,
-        Mix.Utils.last_modified(Path.join([opts[:build], ".mix", base])) > modified,
-        path <- Mix.Dep.load_paths(dep),
-        beam <- Path.wildcard(Path.join(path, "*.beam")),
-        Mix.Utils.last_modified(beam) > modified,
+        manifest = Path.join([opts[:build], ".mix", base]),
+        Mix.Utils.last_modified(manifest) > modified,
         reduce: {stale_modules, %{}, old_exports} do
       {modules, exports, new_exports} ->
-        module = beam |> Path.basename() |> Path.rootname() |> String.to_atom()
-        export = exports_md5(module, false)
-        modules = Map.put(modules, module, [])
+        {_manifest_modules, dep_sources} = read_manifest(manifest)
 
-        # If the exports are the same, then the API did not change,
-        # so we do not mark the export as stale. Note this has to
-        # be very conservative. If the module is not loaded or if
-        # the exports were not there, we need to consider it a stale
-        # export.
-        exports =
-          if export && old_exports[module] == export,
-            do: exports,
-            else: Map.put(exports, module, [])
+        # TODO: Use :maps.from_keys/2 on Erlang/OTP 24+
+        dep_modules =
+          for path <- Mix.Dep.load_paths(dep),
+              beam <- Path.wildcard(Path.join(path, "*.beam")),
+              Mix.Utils.last_modified(beam) > modified,
+              do: {beam |> Path.basename() |> Path.rootname() |> String.to_atom(), []},
+              into: %{}
 
-        # In any case, we always store it as the most update export
-        # that we have, otherwise we delete it.
-        new_exports =
-          if export,
-            do: Map.put(new_exports, module, export),
-            else: Map.delete(new_exports, module)
+        # If any module has a compile time dependency on a changed module
+        # within the dependnecy, they will be recompiled. However, export
+        # and runtime dependencies won't have recompiled so we need to
+        # propagate them to the parent app.
+        dep_modules = fixpoint_dep_modules(dep_sources, dep_modules, false, [])
 
-        {modules, exports, new_exports}
+        # Update exports
+        {exports, new_exports} =
+          for {module, _} <- dep_modules, reduce: {exports, new_exports} do
+            {exports, new_exports} ->
+              export = exports_md5(module, false)
+
+              # If the exports are the same, then the API did not change,
+              # so we do not mark the export as stale. Note this has to
+              # be very conservative. If the module is not loaded or if
+              # the exports were not there, we need to consider it a stale
+              # export.
+              exports =
+                if export && old_exports[module] == export,
+                  do: exports,
+                  else: Map.put(exports, module, [])
+
+              # In any case, we always store it as the most update export
+              # that we have, otherwise we delete it.
+              new_exports =
+                if export,
+                  do: Map.put(new_exports, module, export),
+                  else: Map.delete(new_exports, module)
+
+              {exports, new_exports}
+          end
+
+        {Map.merge(modules, dep_modules), exports, new_exports}
     end
   end
 
+  defp fixpoint_dep_modules([source | sources], modules, new_modules?, acc_sources) do
+    source(export_references: export_refs, runtime_references: runtime_refs) = source
+
+    if has_any_key?(modules, export_refs) or has_any_key?(modules, runtime_refs) do
+      new_modules = Enum.reject(source(source, :modules), &Map.has_key?(modules, &1))
+      new_modules? = new_modules? or new_modules != []
+      modules = Enum.reduce(new_modules, modules, &Map.put(&2, &1, []))
+      fixpoint_dep_modules(sources, modules, new_modules?, acc_sources)
+    else
+      fixpoint_dep_modules(sources, modules, new_modules?, [source | acc_sources])
+    end
+  end
+
+  defp fixpoint_dep_modules([], modules, false, _),
+    do: modules
+
+  defp fixpoint_dep_modules([], modules, true, []),
+    do: modules
+
+  defp fixpoint_dep_modules([], modules, true, sources),
+    do: fixpoint_dep_modules(sources, modules, false, [])
+
   defp exports_md5(module, use_attributes?) do
     cond do
       function_exported?(module, :__info__, 1) ->

lib/mix/lib/mix/compilers/test.ex

@@ -25,52 +25,57 @@ defmodule Mix.Compilers.Test do
   def require_and_run(matched_test_files, test_paths, opts) do
     stale = opts[:stale]
 
-    {test_files, stale_manifest_pid, parallel_require_opts} =
+    {test_files, stale_manifest_pid, parallel_require_callbacks} =
       if stale do
         set_up_stale(matched_test_files, test_paths, opts)
       else
         {matched_test_files, nil, []}
       end
 
-    if test_files == [] do
-      :noop
-    else
-      task = ExUnit.async_run()
-      parallel_require_opts = profile_opts(parallel_require_opts, opts)
-      warnings_as_errors? = Keyword.get(opts, :warnings_as_errors, false)
+    cond do
+      test_files == [] ->
+        :noop
 
-      try do
-        failed? =
-          case Kernel.ParallelCompiler.require(test_files, parallel_require_opts) do
-            {:ok, _, [_ | _]} when warnings_as_errors? -> true
-            {:ok, _, _} -> false
-            {:error, _, _} -> exit({:shutdown, 1})
+      Keyword.get(opts, :profile_require) == "time" ->
+        Kernel.ParallelCompiler.require(test_files, profile: :time)
+        :noop
+
+      true ->
+        task = ExUnit.async_run()
+        warnings_as_errors? = Keyword.get(opts, :warnings_as_errors, false)
+
+        try do
+          failed? =
+            case Kernel.ParallelCompiler.require(test_files, parallel_require_callbacks) do
+              {:ok, _, [_ | _]} when warnings_as_errors? -> true
+              {:ok, _, _} -> false
+              {:error, _, _} -> exit({:shutdown, 1})
+            end
+
+          %{failures: failures} = results = ExUnit.await_run(task)
+
+          if failures == 0 do
+            if failed? do
+              message =
+                "\nERROR! Test suite aborted after successful execution due to warnings while using the --warnings-as-errors option"
+
+              IO.puts(:stderr, IO.ANSI.format([:red, message]))
+              exit({:shutdown, 1})
+            end
+
+            agent_write_manifest(stale_manifest_pid)
           end
 
-        %{failures: failures} = results = ExUnit.await_run(task)
-
-        if failures == 0 do
-          if failed? do
-            message =
-              "\nERROR! Test suite aborted after successful execution due to warnings while using the --warnings-as-errors option"
-
-            IO.puts(:stderr, IO.ANSI.format([:red, message]))
-            exit({:shutdown, 1})
-          end
-
-          agent_write_manifest(stale_manifest_pid)
+          {:ok, results}
+        catch
+          kind, reason ->
+            # In case there is an error, shut down the runner task
+            # before the error propagates up and trigger links.
+            Task.shutdown(task)
+            :erlang.raise(kind, reason, __STACKTRACE__)
+        after
+          agent_stop(stale_manifest_pid)
         end
-
-        {:ok, results}
-      catch
-        kind, reason ->
-          # In case there is an error, shut down the runner task
-          # before the error propagates up and trigger links.
-          Task.shutdown(task)
-          :erlang.raise(kind, reason, __STACKTRACE__)
-      after
-        agent_stop(stale_manifest_pid)
-      end
     end
   end
 
@@ -167,14 +172,6 @@ defmodule Mix.Compilers.Test do
     sources
   end
 
-  defp profile_opts(target, opts) do
-    if Keyword.get(opts, :profile_require) == "time" do
-      Keyword.put(target, :profile, :time)
-    else
-      target
-    end
-  end
-
   ## Manifest
 
   defp manifest, do: Path.join(Mix.Project.manifest_path(), @stale_manifest)

lib/mix/lib/mix/release.ex

@@ -609,13 +609,24 @@ defmodule Mix.Release do
   end
 
   defp build_release_spec(release, modes) do
-    %{name: name, version: version, erts_version: erts_version, applications: apps} = release
+    %{
+      name: name,
+      version: version,
+      erts_version: erts_version,
+      applications: apps,
+      options: options
+    } = release
+
+    skip_mode_validation_for =
+      options
+      |> Keyword.get(:skip_mode_validation_for, [])
+      |> MapSet.new()
 
     rel_apps =
       for {app, mode} <- modes do
         properties = Map.get(apps, app) || throw({:error, "Unknown application #{inspect(app)}"})
         children = Keyword.get(properties, :applications, [])
-        validate_mode!(app, mode, modes, children)
+        app in skip_mode_validation_for || validate_mode!(app, mode, modes, children)
         build_app_for_release(app, mode, properties)
       end
 

lib/mix/lib/mix/tasks/format.ex

@@ -213,6 +213,7 @@ defmodule Mix.Tasks.Format do
   to be formatted. The options are returned for reflection
   purposes.
   """
+  @doc since: "1.13.0"
   def formatter_for_file(file, opts \\ []) do
     {dot_formatter, formatter_opts} = eval_dot_formatter(opts)
 
@@ -494,12 +495,12 @@ defmodule Mix.Tasks.Format do
     ext = Path.extname(file)
 
     cond do
-      ext in ~w(.ex .exs) ->
-        &elixir_format(&1, [file: file] ++ formatter_opts)
-
       plugin = find_plugin_for_extension(formatter_opts, ext) ->
         &plugin.format(&1, [extension: ext] ++ formatter_opts)
 
+      ext in ~w(.ex .exs) ->
+        &elixir_format(&1, [file: file] ++ formatter_opts)
+
       true ->
         & &1
     end

lib/mix/lib/mix/tasks/release.ex

@@ -455,6 +455,14 @@ defmodule Mix.Tasks.Release do
     * `:steps` - a list of steps to execute when assembling the release. See
       the "Steps" section for more information.
 
+    * `:skip_mode_validation_for` - a list of application names
+      (atoms) specifying applications to skip strict validation of
+      "unsafe" modes. An "unsafe" case is when a parent application
+      mode is `:permanent` but one of the applications it depends on
+      is set to `:load`. Use this with care, as a release with
+      invalid modes may no longer boot without additional tweaks.
+      Defaults to `[]`.
+
   Note each release definition can be given as an anonymous function. This
   is useful if some release attributes are expensive to compute:
 

lib/mix/lib/mix/tasks/test.coverage.ex

@@ -299,6 +299,7 @@ defmodule Mix.Tasks.Test.Coverage do
     print_summary(module_results, totals, summary_opts)
 
     if totals < get_threshold(summary_opts) do
+      print_failed_threshold(totals, get_threshold(summary_opts))
       System.at_exit(fn _ -> exit({:shutdown, 3}) end)
     end
 
@@ -353,6 +354,13 @@ defmodule Mix.Tasks.Test.Coverage do
     Mix.shell().info("")
   end
 
+  defp print_failed_threshold(totals, threshold) do
+    Mix.shell().info("Coverage test failed, threshold not met:\n")
+    Mix.shell().info("    Coverage:  #{format_number(totals, 6)}%")
+    Mix.shell().info("    Threshold: #{format_number(threshold, 6)}%")
+    Mix.shell().info("")
+  end
+
   defp display({percentage, name}, threshold) do
     Mix.shell().info([
       color(percentage, threshold),
@@ -369,6 +377,9 @@ defmodule Mix.Tasks.Test.Coverage do
   defp color(percentage, threshold) when percentage >= threshold, do: :green
   defp color(_, _), do: :red
 
+  defp format_number(number, length) when is_integer(number),
+    do: format_number(number / 1, length)
+
   defp format_number(number, length), do: :io_lib.format("~#{length}.2f", [number])
 
   defp format_name(name) when is_binary(name), do: name

lib/mix/lib/mix/tasks/test.ex

@@ -160,7 +160,8 @@ defmodule Mix.Tasks.Test do
 
     * `--preload-modules` - preloads all modules defined in applications
 
-    * `--profile-require` - profiles the time spent to require test files
+    * `--profile-require` - profiles the time spent to require test files.
+      Used only for debugging. The test suite does not run.
 
     * `--raise` - raises if the test suite failed
 

lib/mix/test/mix/release_test.exs

@@ -479,6 +479,13 @@ defmodule Mix.ReleaseTest do
       assert message =~
                "Application :mix has mode :permanent but it depends on :elixir which is set to :none"
     end
+
+    test "does not raise on child unsafe mode if parent is in `skip_mode_validation_for`" do
+      # `:mix` is a parent that depends on `:elixir`
+      release = release(applications: [elixir: :load], skip_mode_validation_for: [:mix])
+      assert make_boot_script(release, @boot_script_path, release.boot_scripts.start) == :ok
+      assert make_boot_script(release, @boot_script_path, release.boot_scripts.start_clean) == :ok
+    end
   end
 
   describe "make_cookie/1" do

lib/mix/test/mix/tasks/test_test.exs

@@ -168,7 +168,14 @@ defmodule Mix.Tasks.TestTest do
 
         # We skip a test in bar to force coverage below the default threshold
         # which should result in an exit status of 1.
-        assert {_, code} = mix_code(["test", "--cover", "--exclude", "maybe_skip"])
+        assert {output, code} = mix_code(["test", "--cover", "--exclude", "maybe_skip"])
+
+        assert output =~ """
+               Coverage test failed, threshold not met:
+
+                   Coverage:    0.00%
+                   Threshold:  90.00%
+               """
 
         unless windows?() do
           assert code == 3

lib/mix/test/mix/umbrella_test.exs

@@ -439,12 +439,9 @@ defmodule Mix.UmbrellaTest do
       Mix.Project.in_project(:bar, "bar", fn _ ->
         File.write!("../foo/lib/foo.ex", "defmodule Foo, do: defstruct [:bar]")
 
-        Mix.Task.run("compile", ["--verbose"])
-
         # Add struct dependency
         File.write!("lib/bar.ex", "defmodule Bar, do: %Foo{bar: true}")
-
-        assert Mix.Tasks.Compile.Elixir.run(["--verbose"]) == {:ok, []}
+        Mix.Task.run("compile", ["--verbose"])
         assert_receive {:mix_shell, :info, ["Compiled lib/bar.ex"]}
 
         # Recompiles for struct dependencies
@@ -458,6 +455,37 @@ defmodule Mix.UmbrellaTest do
     end)
   end
 
+  test "recompiles after compile through runtime path dependency changes" do
+    in_fixture("umbrella_dep/deps/umbrella/apps", fn ->
+      Mix.Project.in_project(:bar, "bar", fn _ ->
+        File.write!("../foo/lib/foo.bar.ex", """
+        defmodule Foo.Bar do
+          def hello, do: Foo.Baz.hello()
+        end
+        """)
+
+        File.write!("../foo/lib/foo.baz.ex", """
+        defmodule Foo.Baz do
+          def hello, do: "from bar"
+        end
+        """)
+
+        # Add compile time to Foo.Bar
+        File.write!("lib/bar.ex", "defmodule Bar, do: Foo.Bar.hello()")
+        Mix.Task.run("compile", ["--verbose"])
+        assert_receive {:mix_shell, :info, ["Compiled lib/bar.ex"]}
+
+        # Recompiles for due to compile dependency via runtime dependencies
+        mtime = File.stat!("_build/dev/lib/bar/.mix/compile.elixir").mtime
+        ensure_touched("_build/dev/lib/foo/ebin/Elixir.Foo.Baz.beam", mtime)
+        ensure_touched("_build/dev/lib/foo/.mix/compile.elixir", mtime)
+
+        assert Mix.Tasks.Compile.Elixir.run(["--verbose"]) == {:ok, []}
+        assert_receive {:mix_shell, :info, ["Compiled lib/bar.ex"]}
+      end)
+    end)
+  end
+
   test "reloads app in app tracer if .app changes" do
     in_fixture("umbrella_dep/deps/umbrella/apps", fn ->
       deps = [{:foo, in_umbrella: true}]