Release v1.13.2

They are already being used in projects, which
means it is too late for a deprecation. The docs
instead discuss when to use them.

CHANGELOG.md

@@ -74,7 +74,7 @@ authority:    fragment:     host:         path:         port:
 query:        scheme:       userinfo:
 ```
 
-As you fill them in, the already filled structs no longer show up:
+As you fill a field in, the already filled fields no longer show up:
 
 ```elixir
 iex(1)> %URI{path: "/example",
@@ -82,6 +82,16 @@ authority:    fragment:     host:         port:         query:
 scheme:       userinfo:
 ```
 
+Along the same lines, `SyntaxError` and `TokenMissingError` were improved to show a code snippet whenever possible:
+
+```elixir
+$ elixir -e "hello + * world"
+** (SyntaxError) nofile:1:9: syntax error before: '*'
+    |
+  1 | hello + * world
+    |         ^
+```
+
 Finally, new compilation tracers have been added, alongside a handful of functions in `Module` to retrieve module metadata, which can be used to enrich suggestions in programming environments.
 
 ## Extended code formatting
@@ -118,7 +128,51 @@ Now any application can use your formatter as follows:
 
 Finally, the `Code` module has also been augmented with two functions: `Code.string_to_quoted_with_comments/2` and `Code.quoted_to_algebra/2`. Those functions allow someone to retrieve the Elixir AST with their original source code comments, and then convert this AST to formatted code. In other words, those functions provide a wrapper around the Elixir Code Formatter, supporting developers who wish to create tools that directly manipulate and custom format Elixir source code.
 
-## v1.13.0-dev
+## v1.13.2 (2022-01-13)
+
+### 1. Enhancements
+
+#### Mix
+
+  * [mix format] Allow plugins to also format `.ex` and `.exs` files
+  * [mix release] Allow bypassing application mode validation in release spec
+  * [mix test] Print a message when the suite fails due to the coverage threshold
+
+### 2. Bug fixes
+
+#### Elixir
+
+  * [Code] Do not emit warnings on `Code.Fragment.container_cursor_to_quoted/2`
+  * [Kernel] Fix a crash when a for-comprehension with `:uniq` was used inside another comprehension with `:uniq`
+  * [Kernel] Ensure `env.context_modules` is properly set inside optimized `defmodule`
+  * [Keyword] Deprecate the recently added `Keyword.map/2` as it is equivalent to `Keyword.new/2`
+  * [Map] Deprecate the recently added `Map.map/2` as it is equivalent to `Map.new/2`
+  * [Protocol] Warn on zero arity callbacks inside protocols
+
+## v1.13.1 (2021-12-14)
+
+### 1. Bug fixes
+
+#### Elixir
+
+  * [Code] Do not show code snippets in `SyntaxError` and `TokenMissingError` if line is empty
+  * [Exception] Do not fail blaming `ArgumentError` for improper lists on `apply/3`
+  * [Macro] Set a max `line_length` for `Macro.to_string/1`
+  * [Macro] Fix formatting of lists on module attributes for `Macro.to_string/1`
+  * [String] Fix incorrect codepoint byte counting in `slice` with negative positions in ranges
+  * [Task] Ensure async streams can be consumed from another process than the one that creates them
+  * [URI] Undeprecate `URI.parse/1` as `URI.new/1` is too strict in many common cases
+  * [URI] Make sure `URI.new/1` returns nil for empty paths
+
+#### IEx
+
+  * [IEx] Make sure the `--version` flag halts IEx
+
+#### Mix
+
+  * [Mix] Make protocol consolidation part of the `Mix.install/2` cache
+
+## v1.13.0 (2021-12-03)
 
 ### 1. Enhancements
 
@@ -136,11 +190,12 @@ Finally, the `Code` module has also been augmented with two functions: `Code.str
   * [Code] Allow custom sigil formatting on `Code.format_string!/2`
   * [Code] Add `{:on_module, bytecode, :none}` trace to compilation tracers
   * [Enum] Optimize `Enum.concat/1` for lists of lists
+  * [Enum] Add `Enum.slide/3`
   * [Exception] Better format Elixir exceptions in Erlang
   * [Inspect] Allow default inspect fun to be set globally with `Inspect.Opts.default_inspect_fun/1`
   * [IO] Allow `:eof` to be given as limit to `IO.getn/2`
   * [Kernel] Support the `:sigils` option in `import Mod, only: :sigils` and allow the sigil modifiers to be also digits
-  * [Kernel] Make `get_in` consistently abort when `nil` values are found
+  * [Kernel] Make `get_in` consistently abort and return `nil` when `nil` values are found (previously Elixir would raise an error in this case). This allows a user to use `get_in` as a safe navigation operator.
   * [Kernel] Improve compilation times by reducing the amount of copies of the AST across compiler processes
   * [Kernel] Raise if trying to define a module with a slash in its name
   * [Kernel] Warn when `?\` is used and there is no need for a escape character
@@ -159,9 +214,11 @@ Finally, the `Code` module has also been augmented with two functions: `Code.str
   * [Task] Add `Task.completed/1`
   * [Task] Add `Task.ignore/1` to keep a task running but ignoring all of its results
   * [Task] Reduce the amount of copying `Task.async*` functions
+  * [URI] Add `URI.new/1` and `URI.new!/1`
 
 #### ExUnit
 
+  * [ExUnit] Show hint if comparing different but equivalent strings
   * [ExUnit.CaptureIO] Add `with_io/3` to return result with captured io
   * [ExUnit.CaptureLog] Add `with_log/2` to return result with captured logs
 
@@ -174,9 +231,14 @@ Finally, the `Code` module has also been augmented with two functions: `Code.str
 #### Logger
 
   * [Logger] Add `Logger.put_application_level/2`
+  * [Logger] Print all log levels in accordance to Erlang/OTP. This also means `[warn]` is now shown as `[warning]`
 
 #### Mix
 
+  * [Mix] Add `MIX_INSTALL_FORCE` environment variable support
+  * [Mix] Support `:config` and `:system_env` in `Mix.install/2`
+  * [Mix] Add `Mix.installed?/0`
+  * [Mix.Shell] Add `:default` option to `Mix.Shell.yes?`
   * [mix archive.install] Run `loadconfig` before building archive
   * [mix compile] Move Elixir version check to before deps are compiled, in order to give feedback earlier
   * [mix compile.elixir] Do not recompile files if their modification time change but their contents are still the same and the .beam files are still on disk
@@ -206,13 +268,18 @@ Finally, the `Code` module has also been augmented with two functions: `Code.str
 
 #### EEx
 
+  * [EEx] Accept comments in EEx between do and the first clause
   * [EEx] Accept EEx expressions where `->` is followed by newline
 
 #### Elixir
 
+  * [Application] Allow any expression as first argument of `compile_env`
   * [Application] Warn if `Application.compile_env` or `Application.compile_env!` are called without a require
   * [Code] Make sure `:static_atoms_encoder` in `Code.string_to_quoted/2` also applies to quoted keyword keys
   * [Code] Ensure bindings with no context are returned as atoms instead of `{binding, nil}` in eval operations
+  * [Inspect] Fix a bug when inspecting a non-binary bitstring with colors
+  * [Kernel] Reject bidirectional formatting characters in strings and comments
+  * [Kernel] Support escaping of terminators in uppercase sigils heredocs for consistency
   * [Kernel] Raise if `__CALLER__` or `__ENV__` or `__STACKTRACE__` are used in match
   * [Kernel] Improve error message on invalid argument for `byte_size` from binary concat
   * [Kernel] Raise when aliasing non-Elixir modules without `:as`
@@ -239,30 +306,34 @@ Finally, the `Code` module has also been augmented with two functions: `Code.str
 #### IEx
 
   * [IEx] Fix the loss of `.iex.exs` context after a pry session
+  * [IEx] Stop evaluator before exiting IEx server to avoid evaluators leaking
 
 #### Logger
 
   * [Logger] Raise clear error message for invalid `:compile_time_purge_matching` configuration
+  * [Logger] Fix a bug where Logger would not reset its discard counter under some scenarios
 
 #### Mix
 
+  * [mix compile.elixir] Track transitive runtime dependencies coming from local/path dependencies
   * [mix compile.elixir] Recompile file if `@external_resource` is deleted
   * [mix compile.elixir] Print number of compiling files on all compiler cycles. This will make the `Compiling N files (.ex)` show up multiple times if necessary
   * [mix deps] Raise if local dep is unavailable while compiling
-  * [mix deps.unlock] Fix blank output when dependency is not locked
+  * [mix deps.unlock] Fix blank output when unlocking a dependency that is not locked
   * [mix local.install] Do not respect `MIX_DEPS_PATH` for install commands
-  * [mix release] Improve release scripts to make sure shell errors cascade by avoiding exporting and defining variables at once
-  * [mix release] Do not boot release if RELEASE_COOKIE is empty
-  * [mix release] Allow release running as a daemon to be restarted
-  * [mix test] Allow coverage engine to also tag `case`, `cond`, and `receive` branches where the right side is a literal
-  * [Mix.Shell] Add `default` option to `Mix.Shell.yes?`
+  * [mix release] Improve release scripts by making sure shell errors cascade (this is done by avoiding exporting and defining variables in a single step)
+  * [mix release] Do not boot release if `RELEASE_COOKIE` is empty
+  * [mix release] Allow releases running as a daemon to be restarted
+  * [mix release] Raise proper error message when non-serializable values are in configs
+  * [mix test] Fix coverage engine to also tag `case`, `cond`, and `receive` branches where the right side is a literal
 
 ### 3. Soft-deprecations (no warnings emitted)
 
 #### Elixir
 
-  * [IO] `:all` on `IO.getn` is deprecated in favor of `:eof`
   * [Code] Environment options in `Code.eval_quoted/3` and `Code.eval_string/3`, such as `:aliases` and `:tracers`, have been deprecated in favor of passing an environment
+  * [IO] `:all` on `IO.getn` is deprecated in favor of `:eof`
+  * [URI] `URI.parse/1` is deprecated in favor of `URI.new/1` and `URI.new!/1`
 
 #### Mix
 

Makefile

@@ -2,7 +2,7 @@ PREFIX ?= /usr/local
 TEST_FILES ?= "*_test.exs"
 SHARE_PREFIX ?= $(PREFIX)/share
 MAN_PREFIX ?= $(SHARE_PREFIX)/man
-#CANONICAL := MAJOR.MINOR/
+CANONICAL := 1.13/
 CANONICAL ?= master/
 ELIXIRC := bin/elixirc --ignore-module-conflict $(ELIXIRC_OPTS)
 ERLC := erlc -I lib/elixir/include

RELEASE.md

@@ -32,7 +32,7 @@
 
 2. Update tables in /SECURITY.md and "Compatibility and Deprecations"
 
-3. Commit "Prepare vMAJOR.MINOR for release"
+3. Commit "Branch out vMAJOR.MINOR"
 
 ### Back in master
 

SECURITY.md

@@ -6,12 +6,11 @@ Elixir applies bug fixes only to the latest minor branch. Security patches are a
 
 Elixir version | Support
 :------------- | :-----------------------------
-1.13           | Development
-1.12           | Bug fixes and security patches
+1.13           | Bug fixes and security patches
+1.12           | Security patches only
 1.11           | Security patches only
 1.10           | Security patches only
 1.9            | Security patches only
-1.8            | Security patches only
 
 ## Announcements
 

VERSION

@@ -1 +1 @@
-1.13.0-dev
+1.13.2

bin/elixir

@@ -1,7 +1,7 @@
 #!/bin/sh
 set -e
 
-ELIXIR_VERSION=1.13.0-dev
+ELIXIR_VERSION=1.13.2
 
 if [ $# -eq 0 ] || { [ $# -eq 1 ] && { [ "$1" = "--help" ] || [ "$1" = "-h" ]; }; }; then
   cat <<USAGE >&2
@@ -16,7 +16,7 @@ Usage: $(basename "$0") [options] [.exs file] [data]
   -pr "FILE"                   Requires the given files/patterns in parallel (*)
   -pa "PATH"                   Prepends the given path to Erlang code path (*)
   -pz "PATH"                   Appends the given path to Erlang code path (*)
-  -v, --version                Prints Erlang/OTP and Elixir versions
+  -v, --version                Prints Erlang/OTP and Elixir versions (standalone)
 
   --app APP                    Starts the given app and its dependencies (*)
   --erl "SWITCHES"             Switches to be passed down to Erlang (*)

bin/elixir.bat

@@ -1,6 +1,6 @@
 @if defined ELIXIR_CLI_ECHO (@echo on) else (@echo off)
 
-set ELIXIR_VERSION=1.13.0-dev
+set ELIXIR_VERSION=1.13.2
 
 setlocal enabledelayedexpansion
 if    ""%1""==""""                if ""%2""=="""" goto documentation
@@ -23,7 +23,7 @@ echo   -S SCRIPT                    Finds and executes the given script in $PATH
 echo   -pr "FILE"                   Requires the given files/patterns in parallel (*)
 echo   -pa "PATH"                   Prepends the given path to Erlang code path (*)
 echo   -pz "PATH"                   Appends the given path to Erlang code path (*)
-echo   -v, --version                Prints Erlang/OTP and Elixir versions
+echo   -v, --version                Prints Erlang/OTP and Elixir versions (standalone)
 echo.
 echo   --app APP                    Starts the given app and its dependencies (*)
 echo   --erl "SWITCHES"             Switches to be passed down to Erlang (*)

bin/elixirc

@@ -7,7 +7,7 @@ Usage: $(basename "$0") [elixir switches] [compiler switches] [.ex files]
 
   -h, --help                Prints this message and exits
   -o                        The directory to output compiled files
-  -v, --version             Prints Elixir version and exits
+  -v, --version             Prints Elixir version and exits (standalone)
 
   --ignore-module-conflict  Does not emit warnings if a module was previously defined
   --no-debug-info           Does not attach debug info to compiled modules

bin/elixirc.bat

@@ -16,7 +16,7 @@ echo Usage: %~nx0 [elixir switches] [compiler switches] [.ex files]
 echo.
 echo   -h, --help                Prints this message and exits
 echo   -o                        The directory to output compiled files
-echo   -v, --version             Prints Elixir version and exits
+echo   -v, --version             Prints Elixir version and exits (standalone)
 echo.
 echo   --ignore-module-conflict  Does not emit warnings if a module was previously defined
 echo   --no-debug-info           Does not attach debug info to compiled modules

lib/elixir/lib/application.ex

@@ -508,7 +508,7 @@ defmodule Application do
   # compile time instead of compile_env
   @doc since: "1.10.0"
   @spec compile_env(app, key | list, value) :: value
-  defmacro compile_env(app, key_or_path, default \\ nil) when is_atom(app) do
+  defmacro compile_env(app, key_or_path, default \\ nil) do
     if __CALLER__.function do
       raise "Application.compile_env/3 cannot be called inside functions, only in the module body"
     end
@@ -545,7 +545,7 @@ defmodule Application do
   """
   @doc since: "1.10.0"
   @spec compile_env!(app, key | list) :: value
-  defmacro compile_env!(app, key_or_path) when is_atom(app) do
+  defmacro compile_env!(app, key_or_path) do
     if __CALLER__.function do
       raise "Application.compile_env!/2 cannot be called inside functions, only in the module body"
     end

lib/elixir/lib/base.ex

@@ -92,6 +92,9 @@ defmodule Base do
 
   """
 
+  @type encode_case :: :upper | :lower
+  @type decode_case :: :upper | :lower | :mixed
+
   b16_alphabet = '0123456789ABCDEF'
   b64_alphabet = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/'
   b64url_alphabet = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_'
@@ -267,7 +270,7 @@ defmodule Base do
       "666f6f626172"
 
   """
-  @spec encode16(binary, keyword) :: binary
+  @spec encode16(binary, case: encode_case) :: binary
   def encode16(data, opts \\ []) when is_binary(data) do
     case = Keyword.get(opts, :case, :upper)
     do_encode16(case, data)
@@ -300,7 +303,7 @@ defmodule Base do
       {:ok, "foobar"}
 
   """
-  @spec decode16(binary, keyword) :: {:ok, binary} | :error
+  @spec decode16(binary, case: decode_case) :: {:ok, binary} | :error
   def decode16(string, opts \\ []) do
     {:ok, decode16!(string, opts)}
   rescue
@@ -337,7 +340,7 @@ defmodule Base do
       "foobar"
 
   """
-  @spec decode16!(binary, keyword) :: binary
+  @spec decode16!(binary, case: encode_case) :: binary
   def decode16!(string, opts \\ [])
 
   def decode16!(string, opts) when is_binary(string) and rem(byte_size(string), 2) == 0 do
@@ -367,7 +370,7 @@ defmodule Base do
       "Zm9vYg"
 
   """
-  @spec encode64(binary, keyword) :: binary
+  @spec encode64(binary, padding: boolean) :: binary
   def encode64(data, opts \\ []) when is_binary(data) do
     pad? = Keyword.get(opts, :padding, true)
     do_encode64(data, pad?)
@@ -397,7 +400,7 @@ defmodule Base do
       {:ok, "foob"}
 
   """
-  @spec decode64(binary, keyword) :: {:ok, binary} | :error
+  @spec decode64(binary, ignore: :whitespace, padding: boolean) :: {:ok, binary} | :error
   def decode64(string, opts \\ []) when is_binary(string) do
     {:ok, decode64!(string, opts)}
   rescue
@@ -431,7 +434,7 @@ defmodule Base do
       "foob"
 
   """
-  @spec decode64!(binary, keyword) :: binary
+  @spec decode64!(binary, ignore: :whitespace, padding: boolean) :: binary
   def decode64!(string, opts \\ []) when is_binary(string) do
     pad? = Keyword.get(opts, :padding, true)
     string |> remove_ignored(opts[:ignore]) |> do_decode64(pad?)
@@ -453,7 +456,7 @@ defmodule Base do
       "_3_-_A"
 
   """
-  @spec url_encode64(binary, keyword) :: binary
+  @spec url_encode64(binary, padding: boolean) :: binary
   def url_encode64(data, opts \\ []) when is_binary(data) do
     pad? = Keyword.get(opts, :padding, true)
     do_encode64url(data, pad?)
@@ -481,7 +484,7 @@ defmodule Base do
       {:ok, <<255, 127, 254, 252>>}
 
   """
-  @spec url_decode64(binary, keyword) :: {:ok, binary} | :error
+  @spec url_decode64(binary, ignore: :whitespace, padding: boolean) :: {:ok, binary} | :error
   def url_decode64(string, opts \\ []) when is_binary(string) do
     {:ok, url_decode64!(string, opts)}
   rescue
@@ -513,7 +516,7 @@ defmodule Base do
       <<255, 127, 254, 252>>
 
   """
-  @spec url_decode64!(binary, keyword) :: binary
+  @spec url_decode64!(binary, ignore: :whitespace, padding: boolean) :: binary
   def url_decode64!(string, opts \\ []) when is_binary(string) do
     pad? = Keyword.get(opts, :padding, true)
     string |> remove_ignored(opts[:ignore]) |> do_decode64url(pad?)
@@ -551,7 +554,7 @@ defmodule Base do
       "MZXW6YTBOI"
 
   """
-  @spec encode32(binary, keyword) :: binary
+  @spec encode32(binary, case: encode_case, padding: boolean) :: binary
   def encode32(data, opts \\ []) when is_binary(data) do
     case = Keyword.get(opts, :case, :upper)
     pad? = Keyword.get(opts, :padding, true)
@@ -594,7 +597,7 @@ defmodule Base do
       {:ok, "foobar"}
 
   """
-  @spec decode32(binary, keyword) :: {:ok, binary} | :error
+  @spec decode32(binary, case: decode_case, padding: boolean) :: {:ok, binary} | :error
   def decode32(string, opts \\ []) do
     {:ok, decode32!(string, opts)}
   rescue
@@ -640,7 +643,7 @@ defmodule Base do
       "foobar"
 
   """
-  @spec decode32!(binary, keyword) :: binary
+  @spec decode32!(binary, case: decode_case, padding: boolean) :: binary
   def decode32!(string, opts \\ []) when is_binary(string) do
     case = Keyword.get(opts, :case, :upper)
     pad? = Keyword.get(opts, :padding, true)
@@ -680,7 +683,7 @@ defmodule Base do
       "CPNMUOJ1E8"
 
   """
-  @spec hex_encode32(binary, keyword) :: binary
+  @spec hex_encode32(binary, case: encode_case, padding: boolean) :: binary
   def hex_encode32(data, opts \\ []) when is_binary(data) do
     case = Keyword.get(opts, :case, :upper)
     pad? = Keyword.get(opts, :padding, true)
@@ -724,7 +727,7 @@ defmodule Base do
       {:ok, "foobar"}
 
   """
-  @spec hex_decode32(binary, keyword) :: {:ok, binary} | :error
+  @spec hex_decode32(binary, case: decode_case, padding: boolean) :: {:ok, binary} | :error
   def hex_decode32(string, opts \\ []) do
     {:ok, hex_decode32!(string, opts)}
   rescue
@@ -771,7 +774,7 @@ defmodule Base do
       "foobar"
 
   """
-  @spec hex_decode32!(binary, keyword) :: binary
+  @spec hex_decode32!(binary, case: decode_case, padding: boolean) :: binary
   def hex_decode32!(string, opts \\ []) when is_binary(string) do
     case = Keyword.get(opts, :case, :upper)
     pad? = Keyword.get(opts, :padding, true)

lib/elixir/lib/calendar/datetime.ex

@@ -2,11 +2,14 @@ defmodule DateTime do
   @moduledoc """
   A datetime implementation with a time zone.
 
-  This datetime can be seen as an ephemeral snapshot
-  of a datetime at a given time zone. For such purposes,
-  it also includes both UTC and Standard offsets, as
-  well as the zone abbreviation field used exclusively
-  for formatting purposes.
+  This datetime can be seen as a snapshot of a date and time
+  at a given time zone. For such purposes, it also includes both
+  UTC and Standard offsets, as well as the zone abbreviation
+  field used exclusively for formatting purposes. Note future
+  datetimes are not necessarily guaranteed to exist, as time
+  zones may change any time in the future due to geopolitical
+  reasons. See the "Datetimes as snapshots" section for more
+  information.
 
   Remember, comparisons in Elixir using `==/2`, `>/2`, `</2` and friends
   are structural and based on the DateTime struct fields. For proper
@@ -41,6 +44,56 @@ defmodule DateTime do
       Calendar.put_time_zone_database(Tzdata.TimeZoneDatabase)
 
   See the proper names in the library installation instructions.
+
+  ## Datetimes as snapshots
+
+  In the first section, we described datetimes as a "snapshot of
+  a date and time at a given time zone". To understand precisely
+  what we mean, let's see an example.
+
+  Imagine someone in Poland wants to schedule a meeting with someone
+  in Brazil in the next year. The meeting will happen at 2:30 AM
+  in the Polish time zone. At what time will the meeting happen in
+  Brazil?
+
+  You can consult the time zone database today, one year before,
+  using the API in this module and it will give you an answer that
+  is valid right now. However, this answer may not be valid in the
+  future. Why? Because both Brazil and Poland may change their timezone
+  rules, ultimately affecting the result. For example, a country may
+  choose to enter or abandon "Daylight Saving Time", which is a
+  process where we adjust the clock one hour forward or one hour
+  back once per year. Whenener the rules change, the exact instant
+  that 2:30 AM in Polish time will be in Brazil may change.
+
+  In other words, whenever working with future DateTimes, there is
+  no guarantee the results you get will always be correct, until
+  the event actually happens. Therefore, when you ask for a future
+  time, the answers you get are a snapshot that reflects the current
+  state of the time zone rules. For datetimes in the past, this is
+  not a problem, because time zone rules do not change for past
+  events.
+
+  To make matters worse, it may be that the 2:30 AM in Polish time
+  does not actually even exist or it is ambiguous. If a certain
+  time zone observes "Daylight Saving Time", they will move their
+  clock forward once a year. When this happens, there is a whole
+  hour that does not exist. Then, when they move the clock back,
+  there is a certain hour that will happen twice. So if you want
+  to schedule a meeting when this shift back happens, you would
+  need to explicitly say which of the 2:30 AM you precisely mean.
+  Applications that are date and time sensitive, need to take
+  these scenarios into account and correctly communicate them to
+  users.
+
+  The good news is: Elixir contains all of the building blocks
+  necessary to tackle those problems. The default timezone database
+  used by Elixir, `Calendar.UTCOnlyTimeZoneDatabase`, only works
+  with UTC, which does not observe those issues. Once you bring
+  a proper time zone database, the functions in this module will
+  query the database and return the relevant information. For
+  example, look at how `DateTime.new/4` returns different results
+  based on the scenarios described in this section.
   """
 
   @enforce_keys [:year, :month, :day, :hour, :minute, :second] ++

lib/elixir/lib/code.ex

@@ -1599,7 +1599,7 @@ defmodule Code do
   file.
 
   It returns the term stored in the documentation chunk in the format defined by
-  [EEP 48](https://erlang.org/eep/eeps/eep-0048.html) or `{:error, reason}` if
+  [EEP 48](https://www.erlang.org/eeps/eep-0048.html) or `{:error, reason}` if
   the chunk is not available.
 
   ## Examples
@@ -1702,7 +1702,7 @@ defmodule Code do
   @doc ~S"""
   Deprecated function to retrieve old documentation format.
 
-  Elixir v1.7 adopts [EEP 48](https://erlang.org/eep/eeps/eep-0048.html)
+  Elixir v1.7 adopts [EEP 48](https://www.erlang.org/eeps/eep-0048.html)
   which is a new documentation format meant to be shared across all
   BEAM languages. The old format, used by `Code.get_docs/2`, is no
   longer available, and therefore this function always returns `nil`.

lib/elixir/lib/code/formatter.ex

@@ -270,7 +270,7 @@ defmodule Code.Formatter do
         {doc, state} =
           entries
           |> prepend_heredoc_line()
-          |> interpolation_to_algebra(:heredoc, state, @double_heredoc, @double_heredoc)
+          |> interpolation_to_algebra(~s["""], state, @double_heredoc, @double_heredoc)
 
         {force_unfit(doc), state}
 
@@ -292,7 +292,7 @@ defmodule Code.Formatter do
         {doc, state} =
           entries
           |> prepend_heredoc_line()
-          |> list_interpolation_to_algebra(:heredoc, state, @single_heredoc, @single_heredoc)
+          |> list_interpolation_to_algebra(~s['''], state, @single_heredoc, @single_heredoc)
 
         {force_unfit(doc), state}
 
@@ -355,7 +355,7 @@ defmodule Code.Formatter do
   defp quoted_to_algebra({:__block__, meta, [list]}, _context, state) when is_list(list) do
     case meta[:delimiter] do
       ~s['''] ->
-        string = list |> List.to_string() |> escape_heredoc()
+        string = list |> List.to_string() |> escape_heredoc(~s['''])
         {@single_heredoc |> concat(string) |> concat(@single_heredoc) |> force_unfit(), state}
 
       ~s['] ->
@@ -369,7 +369,7 @@ defmodule Code.Formatter do
 
   defp quoted_to_algebra({:__block__, meta, [string]}, _context, state) when is_binary(string) do
     if meta[:delimiter] == ~s["""] do
-      string = escape_heredoc(string)
+      string = escape_heredoc(string, ~s["""])
       {@double_heredoc |> concat(string) |> concat(@double_heredoc) |> force_unfit(), state}
     else
       string = escape_string(string, @double_quote)
@@ -1339,7 +1339,7 @@ defmodule Code.Formatter do
         {doc, state} =
           entries
           |> prepend_heredoc_line()
-          |> interpolation_to_algebra(:heredoc, state, doc, closing_delimiter)
+          |> interpolation_to_algebra(opening_delimiter, state, doc, closing_delimiter)
 
         {force_unfit(doc), state}
       else
@@ -1548,11 +1548,13 @@ defmodule Code.Formatter do
     end
   end
 
-  defp escape_heredoc(string) do
+  defp escape_heredoc(string, escape) do
+    string = String.replace(string, escape, "\\" <> escape)
     heredoc_to_algebra(["" | String.split(string, "\n")])
   end
 
-  defp escape_string(string, :heredoc) do
+  defp escape_string(string, <<_, _, _>> = escape) do
+    string = String.replace(string, escape, "\\" <> escape)
     heredoc_to_algebra(String.split(string, "\n"))
   end
 
@@ -1873,19 +1875,25 @@ defmodule Code.Formatter do
   end
 
   defp each_quoted_to_algebra_with_comments([arg | args], acc, max_line, state, comments?, fun) do
-    {doc_start, doc_end} = traverse_line(arg, {@max_line, @min_line})
+    case traverse_line(arg, {@max_line, @min_line}) do
+      {@max_line, @min_line} ->
+        {doc_triplet, state} = fun.(arg, args, state)
+        acc = [doc_triplet | acc]
+        each_quoted_to_algebra_with_comments(args, acc, max_line, state, comments?, fun)
 
-    {acc, comments, comments?} =
-      extract_comments_before(doc_start, acc, state.comments, comments?)
+      {doc_start, doc_end} ->
+        {acc, comments, comments?} =
+          extract_comments_before(doc_start, acc, state.comments, comments?)
 
-    {doc_triplet, state} = fun.(arg, args, %{state | comments: comments})
+        {doc_triplet, state} = fun.(arg, args, %{state | comments: comments})
 
-    {acc, comments, comments?} =
-      extract_comments_trailing(doc_start, doc_end, acc, state.comments, comments?)
+        {acc, comments, comments?} =
+          extract_comments_trailing(doc_start, doc_end, acc, state.comments, comments?)
 
-    acc = [adjust_trailing_newlines(doc_triplet, doc_end, comments) | acc]
-    state = %{state | comments: comments}
-    each_quoted_to_algebra_with_comments(args, acc, max_line, state, comments?, fun)
+        acc = [adjust_trailing_newlines(doc_triplet, doc_end, comments) | acc]
+        state = %{state | comments: comments}
+        each_quoted_to_algebra_with_comments(args, acc, max_line, state, comments?, fun)
+    end
   end
 
   defp extract_comments_before(max, acc, [%{line: line} = comment | rest], _) when line < max do

lib/elixir/lib/code/fragment.ex

@@ -525,7 +525,7 @@ defmodule Code.Fragment do
     |> position_surround_context(line, column, opts)
   end
 
-  def surround_context(other, position, opts) do
+  def surround_context(other, {_, _} = position, opts) do
     surround_context(to_charlist(other), position, opts)
   end
 
@@ -827,7 +827,7 @@ defmodule Code.Fragment do
 
     case :elixir_tokenizer.tokenize(fragment, line, column, tokenizer_opts) do
       {:ok, _, _, _warnings, tokens} ->
-        :elixir.tokens_to_quoted(tokens, file, columns: columns, token_metadata: token_metadata)
+        :elixir.tokens_to_quoted(tokens, nil, columns: columns, token_metadata: token_metadata)
 
       {:error, {line, column, {prefix, suffix}, token}, _rest, _warnings, _so_far} ->
         location = [line: line, column: column]

lib/elixir/lib/code/normalizer.ex

@@ -63,15 +63,17 @@ defmodule Code.Normalizer do
   end
 
   # Bit containers
-  defp do_normalize({:<<>>, _, _} = quoted, state) do
+  defp do_normalize({:<<>>, _, args} = quoted, state) when is_list(args) do
     normalize_bitstring(quoted, state)
   end
 
   # Atoms with interpolations
   defp do_normalize(
-         {{:., dot_meta, [:erlang, :binary_to_atom]}, call_meta, [{:<<>>, _, _} = string, :utf8]},
+         {{:., dot_meta, [:erlang, :binary_to_atom]}, call_meta,
+          [{:<<>>, _, args} = string, :utf8]},
          state
-       ) do
+       )
+       when is_list(args) do
     dot_meta = patch_meta_line(dot_meta, state.parent_meta)
     call_meta = patch_meta_line(call_meta, dot_meta)
 
@@ -166,8 +168,8 @@ defmodule Code.Normalizer do
   end
 
   # Sigils
-  defp do_normalize({sigil, meta, [{:<<>>, _, _} = string, modifiers]} = quoted, state)
-       when is_atom(sigil) do
+  defp do_normalize({sigil, meta, [{:<<>>, _, args} = string, modifiers]} = quoted, state)
+       when is_list(args) and is_atom(sigil) do
     case Atom.to_string(sigil) do
       <<"sigil_", _name>> ->
         meta =
@@ -183,19 +185,40 @@ defmodule Code.Normalizer do
   end
 
   # Tuples
-  defp do_normalize({:{}, meta, args} = quoted, state) do
+  defp do_normalize({:{}, meta, args} = quoted, state) when is_list(args) do
     {last_arg, args} = List.pop_at(args, -1)
 
-    with [{{:__block__, key_meta, _}, _} | _] <- last_arg, :keyword <- key_meta[:format] do
-      args = normalize_kw_args(args, state)
-      kw_list = normalize_kw_args(last_arg, state)
+    if args != [] and match?([_ | _], last_arg) and keyword?(last_arg) do
+      args = normalize_args(args, state)
+      kw_list = normalize_kw_args(last_arg, state, true)
       {:{}, meta, args ++ kw_list}
     else
-      _ ->
-        normalize_call(quoted, state)
+      normalize_call(quoted, state)
     end
   end
 
+  # Module attributes
+  defp do_normalize({:@, meta, [{name, name_meta, [value]}]}, state) do
+    value =
+      cond do
+        keyword?(value) ->
+          normalize_kw_args(value, state, true)
+
+        is_list(value) ->
+          normalize_literal(value, meta, state)
+
+        true ->
+          do_normalize(value, state)
+      end
+
+    {:@, meta, [{name, name_meta, [value]}]}
+  end
+
+  # Regular blocks
+  defp do_normalize({:__block__, meta, args}, state) when is_list(args) do
+    {:__block__, meta, normalize_args(args, state)}
+  end
+
   # Calls
   defp do_normalize({_, _, args} = quoted, state) when is_list(args) do
     normalize_call(quoted, state)
@@ -246,11 +269,10 @@ defmodule Code.Normalizer do
     meta = patch_meta_line(meta, state.parent_meta)
     state = %{state | parent_meta: meta}
 
-    with [{{:__block__, key_meta, _}, _} | _] <- right, :keyword <- key_meta[:format] do
-      {:__block__, meta, [{do_normalize(left, state), normalize_kw_args(right, state)}]}
+    if match?([_ | _], right) and keyword?(right) do
+      {:__block__, meta, [{do_normalize(left, state), normalize_kw_args(right, state, true)}]}
     else
-      _ ->
-        {:__block__, meta, [{do_normalize(left, state), do_normalize(right, state)}]}
+      {:__block__, meta, [{do_normalize(left, state), do_normalize(right, state)}]}
     end
   end
 
@@ -282,7 +304,7 @@ defmodule Code.Normalizer do
           meta
         end
 
-      {:__block__, meta, [normalize_kw_args(list, state)]}
+      {:__block__, meta, [normalize_kw_args(list, state, false)]}
     end
   end
 
@@ -402,7 +424,7 @@ defmodule Code.Normalizer do
   end
 
   defp normalize_map_args(args, state) do
-    Enum.map(normalize_kw_args(args, state), fn
+    Enum.map(normalize_kw_args(args, state, false), fn
       {:__block__, _, [{_, _} = pair]} -> pair
       pair -> pair
     end)
@@ -435,7 +457,7 @@ defmodule Code.Normalizer do
     {form, meta, leading_args ++ [kw_blocks]}
   end
 
-  defp normalize_kw_args(elems, state, keyword? \\ false)
+  defp normalize_kw_args(elems, state, keyword?)
 
   defp normalize_kw_args(
          [{{:__block__, key_meta, [key]}, value} = first | rest] = current,

lib/elixir/lib/enum.ex

@@ -2573,7 +2573,12 @@ defmodule Enum do
       iex> Enum.slide([:a, :b, :c, :d, :e, :f, :g], -4..-2, 1)
       [:a, :d, :e, :f, :b, :c, :g]
 
+      # Insert at negative indices (counting from the end)
+      iex> Enum.slide([:a, :b, :c, :d, :e, :f, :g], 3, -1)
+      [:a, :b, :c, :e, :f, :g, :d]
+
   """
+  @doc since: "1.13.0"
   def slide(enumerable, range_or_single_index, insertion_index)
 
   def slide(enumerable, single_index, insertion_index) when is_integer(single_index) do
@@ -2587,14 +2592,19 @@ defmodule Enum do
   end
 
   # Normalize negative input ranges like Enum.slice/2
-  def slide(enumerable, first..last, insertion_index) when first < 0 or last < 0 do
+  def slide(enumerable, first..last, insertion_index)
+      when first < 0 or last < 0 or insertion_index < 0 do
     count = Enum.count(enumerable)
     normalized_first = if first >= 0, do: first, else: first + count
     normalized_last = if last >= 0, do: last, else: last + count
 
-    if normalized_first >= 0 and normalized_first < count and normalized_first != insertion_index do
+    normalized_insertion_index =
+      if insertion_index >= 0, do: insertion_index, else: insertion_index + count
+
+    if normalized_first >= 0 and normalized_first < count and
+         normalized_first != normalized_insertion_index do
       normalized_range = normalized_first..normalized_last//1
-      slide(enumerable, normalized_range, insertion_index)
+      slide(enumerable, normalized_range, normalized_insertion_index)
     else
       Enum.to_list(enumerable)
     end
@@ -2605,7 +2615,7 @@ defmodule Enum do
   end
 
   def slide(_, first..last, insertion_index)
-      when insertion_index > first and insertion_index < last do
+      when insertion_index > first and insertion_index <= last do
     raise "Insertion index for slide must be outside the range being moved " <>
             "(tried to insert #{first}..#{last} at #{insertion_index})"
   end

lib/elixir/lib/exception.ex

@@ -732,6 +732,10 @@ defmodule ArgumentError do
       ) do
     message =
       cond do
+        not proper_list?(args) ->
+          "you attempted to apply a function named #{inspect(function)} on module #{inspect(module)} " <>
+            "with arguments #{inspect(args)}. Arguments (the third argument of apply) must always be a proper list"
+
         # Note that args may be an empty list even if they were supplied
         not is_atom(module) and is_atom(function) and args == [] ->
           "you attempted to apply a function named #{inspect(function)} on #{inspect(module)}. " <>
@@ -747,10 +751,6 @@ defmodule ArgumentError do
           "you attempted to apply a function named #{inspect(function)} on module #{inspect(module)}. " <>
             "However #{inspect(function)} is not a valid function name. Function names (the second argument " <>
             "of apply) must always be an atom"
-
-        not is_list(args) ->
-          "you attempted to apply a function named #{inspect(function)} on module #{inspect(module)} " <>
-            "with arguments #{inspect(args)}. Arguments (the third argument of apply) must always be a list"
       end
 
     {%{exception | message: message}, stacktrace}
@@ -759,6 +759,9 @@ defmodule ArgumentError do
   def blame(exception, stacktrace) do
     {exception, stacktrace}
   end
+
+  defp proper_list?(list) when length(list) >= 0, do: true
+  defp proper_list?(_), do: false
 end
 
 defmodule ArithmeticError do

lib/elixir/lib/kernel/cli.ex

@@ -218,12 +218,16 @@ defmodule Kernel.CLI do
 
   # Parse shared options
 
-  defp parse_shared([opt | _], _config) when opt in @standalone_opts do
+  defp warn_standalone(opt) do
     IO.puts(:stderr, "#{opt} : Standalone options can't be combined with other options")
+  end
+
+  defp parse_shared([opt | _], _config) when opt in @standalone_opts do
+    warn_standalone(opt)
     System.halt(1)
   end
 
-  defp parse_shared([opt | t], config) when opt in ["-v", "--version"] do
+  defp parse_shared([opt | t], _config) when opt in ["-v", "--version"] do
     if function_exported?(IEx, :started?, 0) and IEx.started?() do
       IO.puts("IEx " <> System.build_info()[:build])
     else
@@ -231,7 +235,8 @@ defmodule Kernel.CLI do
       IO.puts("Elixir " <> System.build_info()[:build])
     end
 
-    parse_shared(t, config)
+    t != [] && warn_standalone(opt)
+    System.halt(0)
   end
 
   defp parse_shared(["-pa", h | t], config) do

lib/elixir/lib/keyword.ex

@@ -1374,27 +1374,9 @@ defmodule Keyword do
     end
   end
 
-  @doc """
-  Maps the function `fun` over all key-value pairs in `keywords`,
-  returning a keyword list with all the values replaced with
-  the result of the function.
-
-  ## Examples
-
-      iex> Keyword.map([one: 1, two: 2, three: 3], fn {_key, val} -> to_string(val) end)
-      [one: "1", two: "2", three: "3"]
-
-  """
-  @doc since: "1.13.0"
-  @spec map(t, ({key, value} -> value)) :: t
-  def map(keywords, fun) when is_list(keywords) and is_function(fun, 1) do
-    do_map(keywords, fun)
-  end
-
-  defp do_map([], _fun), do: []
-
-  defp do_map([{key, value} | rest], fun) do
-    new_value = fun.({key, value})
-    [{key, new_value} | do_map(rest, fun)]
+  @doc false
+  @deprecated "Use Keyword.new/2 instead"
+  def map(keywords, fun) when is_list(keywords) do
+    Enum.map(keywords, fn {k, v} -> {k, fun.({k, v})} end)
   end
 end

lib/elixir/lib/macro.ex

@@ -661,7 +661,9 @@ defmodule Macro do
   dependency on the given struct.
   """
   @doc since: "1.8.0"
-  @spec struct!(module, Macro.Env.t()) :: %{__struct__: module} when module: module()
+  @spec struct!(module, Macro.Env.t()) ::
+          %{required(:__struct__) => module, optional(atom) => any}
+        when module: module()
   def struct!(module, env) when is_atom(module) do
     if module == env.module do
       Module.get_attribute(module, :__struct__)
@@ -943,7 +945,7 @@ defmodule Macro do
   """
   @spec to_string(t()) :: String.t()
   def to_string(tree) do
-    doc = Inspect.Algebra.format(Code.quoted_to_algebra(tree), :infinity)
+    doc = Inspect.Algebra.format(Code.quoted_to_algebra(tree), 98)
     IO.iodata_to_binary(doc)
   end
 

lib/elixir/lib/map.ex

@@ -979,12 +979,14 @@ defmodule Map do
   `fun` receives the key and value of each of the
   elements in the map as a key-value pair.
 
-  `Map.filter/2` is faster than using `map |> Enum.filter(fun) |> Enum.into(%{})`,
-  as no intermediate list is being built.
-
   See also `reject/2` which discards all elements where the
   function returns a truthy value.
 
+  > Note: if you find yourself doing multiple calls to `Map.filter/2`
+  > and `Map.reject/2` in a pipeline, it is likely more efficient
+  > to use `Enum.map/2` and `Enum.filter/2` instead and convert to
+  > a map at the end using `Map.new/1`.
+
   ## Examples
 
       iex> Map.filter(%{one: 1, two: 2, three: 3}, fn {_key, val} -> rem(val, 2) == 1 end)
@@ -1010,9 +1012,8 @@ defmodule Map do
   end
 
   @doc """
-  Returns map excluding the pairs from `map` for which `fun` returns a truthy value.
-  `Map.reject/2` is faster than using `map |> Enum.reject(fun) |> Enum.into(%{})`,
-  as no intermediate list is being built.
+  Returns map excluding the pairs from `map` for which `fun` returns
+  a truthy value.
 
   See also `filter/2`.
 
@@ -1040,28 +1041,9 @@ defmodule Map do
     end
   end
 
-  @doc """
-  Maps the function `fun` over all key-value pairs in `map`, returning a map
-  with all the values replaced with the result of the function.
-
-  ## Examples
-
-      iex> Map.map(%{1 => "joe", 2 => "mike", 3 => "robert"}, fn {_key, val} -> String.capitalize(val) end)
-      %{1 => "Joe", 2 => "Mike", 3 => "Robert"}
-
-  """
-  @doc since: "1.13.0"
-  @spec map(map, ({key, value} -> value)) :: map
-  def map(map, fun) when is_map(map) and is_function(fun, 1) do
-    iter = :maps.iterator(map)
-    next = :maps.next(iter)
-    :maps.from_list(do_map(next, fun))
-  end
-
-  defp do_map(:none, _fun), do: []
-
-  defp do_map({key, value, iter}, fun) do
-    new_value = fun.({key, value})
-    [{key, new_value} | do_map(:maps.next(iter), fun)]
+  @doc false
+  @deprecated "Use Map.new/2 instead"
+  def map(map, fun) when is_map(map) do
+    new(map, fn {k, v} -> {k, fun.({k, v})} end)
   end
 end

lib/elixir/lib/module/types.ex

@@ -354,7 +354,7 @@ defmodule Module.Types do
   end
 
   defp simplify_type?(type, other) do
-    map_type?(type) and not map_type?(other)
+    map_like_type?(type) and not map_like_type?(other)
   end
 
   ## EXPRESSION FORMATTING
@@ -505,6 +505,10 @@ defmodule Module.Types do
   defp map_type?({:map, _}), do: true
   defp map_type?(_other), do: false
 
+  defp map_like_type?({:map, _}), do: true
+  defp map_like_type?({:union, union}), do: Enum.any?(union, &map_like_type?/1)
+  defp map_like_type?(_other), do: false
+
   defp atom_type?(:atom), do: true
   defp atom_type?({:atom, _}), do: false
   defp atom_type?({:union, union}), do: Enum.all?(union, &atom_type?/1)

lib/elixir/lib/module/types/expr.ex

@@ -469,12 +469,19 @@ defmodule Module.Types.Expr do
   end
 
   defp of_clauses(clauses, stack, context) do
-    reduce_ok(clauses, context, fn {:->, _meta, [head, body]}, context = acc ->
+    reduce_ok(clauses, context, fn {:->, meta, [head, body]}, context = acc ->
       {patterns, guards} = extract_head(head)
 
-      with {:ok, _, context} <- Pattern.of_head(patterns, guards, stack, context),
-           {:ok, _expr_type, context} <- of_expr(body, :dynamic, stack, context),
-           do: {:ok, keep_warnings(acc, context)}
+      case Pattern.of_head(patterns, guards, stack, context) do
+        {:ok, _, context} ->
+          with {:ok, _expr_type, context} <- of_expr(body, :dynamic, stack, context) do
+            {:ok, keep_warnings(acc, context)}
+          end
+
+        error ->
+          # Skip the clause if it the head has an error
+          if meta[:generated], do: {:ok, acc}, else: error
+      end
     end)
   end
 

lib/elixir/lib/process.ex

@@ -383,7 +383,7 @@ defmodule Process do
   @type spawn_opt ::
           :link
           | :monitor
-          | {:monitor, :erlang.monitor_option()}
+          | {:monitor, monitor_option()}
           | {:priority, :low | :normal | :high}
           | {:fullsweep_after, non_neg_integer}
           | {:min_heap_size, non_neg_integer}
@@ -392,6 +392,10 @@ defmodule Process do
           | {:message_queue_data, :off_heap | :on_heap}
   @type spawn_opts :: [spawn_opt]
 
+  # TODO: Use :erlang.monitor_option() on Erlang/OTP 24+
+  @typep monitor_option ::
+           [alias: :explicit_unalias | :demonitor | :reply_demonitor, tag: term()]
+
   @doc """
   Spawns the given function according to the given options.
 

lib/elixir/lib/protocol.ex

@@ -723,14 +723,14 @@ defmodule Protocol do
 
   defp callback_ast_to_fa({kind, {:"::", meta, [{name, _, args}, _return]}, _pos})
        when kind in [:callback, :macrocallback] do
-    [{{name, length(args)}, meta}]
+    [{{name, length(List.wrap(args))}, meta}]
   end
 
   defp callback_ast_to_fa(
          {kind, {:when, _, [{:"::", meta, [{name, _, args}, _return]}, _vars]}, _pos}
        )
        when kind in [:callback, :macrocallback] do
-    [{{name, length(args)}, meta}]
+    [{{name, length(List.wrap(args))}, meta}]
   end
 
   defp callback_ast_to_fa({kind, _, _pos}) when kind in [:callback, :macrocallback] do

lib/elixir/lib/string.ex

@@ -2736,7 +2736,7 @@ defmodule String do
     end
   end
 
-  defp codepoint_byte_size(cp) when cp <= 0x00FF, do: 1
+  defp codepoint_byte_size(cp) when cp <= 0x007F, do: 1
   defp codepoint_byte_size(cp) when cp <= 0x07FF, do: 2
   defp codepoint_byte_size(cp) when cp <= 0xFFFF, do: 3
   defp codepoint_byte_size(_), do: 4

lib/elixir/lib/task.ex

@@ -657,12 +657,14 @@ defmodule Task do
   end
 
   defp build_stream(enumerable, fun, options) do
-    owner = self()
+    fn acc, acc_fun ->
+      owner = get_owner(self())
 
-    &Task.Supervised.stream(enumerable, &1, &2, get_callers(owner), fun, options, fn ->
-      {:ok, pid} = Task.Supervised.start_link(get_owner(owner), :nomonitor)
-      {:ok, :link, pid}
-    end)
+      Task.Supervised.stream(enumerable, acc, acc_fun, get_callers(self()), fun, options, fn ->
+        {:ok, pid} = Task.Supervised.start_link(owner, :nomonitor)
+        {:ok, :link, pid}
+      end)
+    end
   end
 
   # Returns a tuple with the node where this is executed and either the
@@ -991,7 +993,7 @@ defmodule Task do
 
     * the task process exited with the reason `:normal`
     * the task isn't linked to the caller (the task was started
-      with `Task.async_nolink/2` or `Task.Supervisor.async_nolink/3`)
+      with `Task.Supervisor.async_nolink/2` or `Task.Supervisor.async_nolink/4`)
     * the caller is trapping exits
 
   A timeout, in milliseconds or `:infinity`, can be given with a default value
@@ -1080,7 +1082,7 @@ defmodule Task do
 
   `Task.yield_many/2` allows developers to spawn multiple tasks
   and retrieve the results received in a given timeframe.
-  If we combine it with `Task.shutdown/2` (or `Task.ignore/2`),
+  If we combine it with `Task.shutdown/2` (or `Task.ignore/1`),
   it allows us to gather those results and cancel (or ignore)
   the tasks that have not replied in time.
 

lib/elixir/lib/task/supervisor.ex

@@ -74,7 +74,7 @@ defmodule Task.Supervisor do
 
   This function could also receive `:restart` and `:shutdown` as options
   but those two options have been deprecated and it is now preferred to
-  give them directly to `start_child` and `async`.
+  give them directly to `start_child`.
   """
   @spec start_link([option]) :: Supervisor.on_start()
   def start_link(options \\ []) do
@@ -105,7 +105,7 @@ defmodule Task.Supervisor do
 
   The `supervisor` must be a reference as defined in `Supervisor`.
   The task will still be linked to the caller, see `Task.async/3` for
-  more information and `async_nolink/2` for a non-linked variant.
+  more information and `async_nolink/3` for a non-linked variant.
 
   Raises an error if `supervisor` has reached the maximum number of
   children.
@@ -126,7 +126,7 @@ defmodule Task.Supervisor do
 
   The `supervisor` must be a reference as defined in `Supervisor`.
   The task will still be linked to the caller, see `Task.async/3` for
-  more information and `async_nolink/2` for a non-linked variant.
+  more information and `async_nolink/3` for a non-linked variant.
 
   Raises an error if `supervisor` has reached the maximum number of
   children.
@@ -235,7 +235,7 @@ defmodule Task.Supervisor do
   children.
 
   Note this function requires the task supervisor to have `:temporary`
-  as the `:restart` option (the default), as `async_nolink/4` keeps a
+  as the `:restart` option (the default), as `async_nolink/5` keeps a
   direct reference to the task which is lost if the task is restarted.
   """
   @spec async_nolink(Supervisor.supervisor(), module, atom, [term], Keyword.t()) :: Task.t()
@@ -249,7 +249,7 @@ defmodule Task.Supervisor do
 
   Each element will be prepended to the given `args` and processed by its
   own task. The tasks will be spawned under the given `supervisor` and
-  linked to the caller process, similarly to `async/4`.
+  linked to the caller process, similarly to `async/5`.
 
   When streamed, each task will emit `{:ok, value}` upon successful
   completion or `{:exit, reason}` if the caller is trapping exits.
@@ -308,7 +308,7 @@ defmodule Task.Supervisor do
 
   Each element in `enumerable` is passed as argument to the given function `fun`
   and processed by its own task. The tasks will be spawned under the given
-  `supervisor` and linked to the caller process, similarly to `async/2`.
+  `supervisor` and linked to the caller process, similarly to `async/3`.
 
   See `async_stream/6` for discussion, options, and examples.
   """
@@ -325,7 +325,7 @@ defmodule Task.Supervisor do
 
   Each element in `enumerable` will be prepended to the given `args` and processed
   by its own task. The tasks will be spawned under the given `supervisor` and
-  will not be linked to the caller process, similarly to `async_nolink/4`.
+  will not be linked to the caller process, similarly to `async_nolink/5`.
 
   See `async_stream/6` for discussion, options, and examples.
   """
@@ -350,7 +350,7 @@ defmodule Task.Supervisor do
   Each element in `enumerable` is passed as argument to the given function `fun`
   and processed by its own task. The tasks will be spawned under the given
   `supervisor` and will not be linked to the caller process, similarly
-  to `async_nolink/2`.
+  to `async_nolink/3`.
 
   See `async_stream/6` for discussion and examples.
   """
@@ -387,7 +387,7 @@ defmodule Task.Supervisor do
   Note that the spawned process is not linked to the caller, but
   only to the supervisor. This command is useful in case the
   task needs to perform side-effects (like I/O) and you have no
-  interest on its results nor if it completes successfully.
+  interest in its results nor if it completes successfully.
 
   ## Options
 
@@ -395,9 +395,9 @@ defmodule Task.Supervisor do
       `:transient` or `:permanent`. `:temporary` means the task is never
       restarted, `:transient` means it is restarted if the exit is not
       `:normal`, `:shutdown` or `{:shutdown, reason}`. A `:permanent` restart
-      strategy means it is always restarted. It defaults to `:temporary`.
+      strategy means it is always restarted.
 
-    * `:shutdown` - `:brutal_kill` if the tasks must be killed directly on shutdown
+    * `:shutdown` - `:brutal_kill` if the task must be killed directly on shutdown
       or an integer indicating the timeout value, defaults to 5000 milliseconds.
 
   """
@@ -413,7 +413,7 @@ defmodule Task.Supervisor do
   @doc """
   Starts a task as a child of the given `supervisor`.
 
-  Similar to `start_child/2` except the task is specified
+  Similar to `start_child/3` except the task is specified
   by the given `module`, `fun` and `args`.
   """
   @spec start_child(Supervisor.supervisor(), module, atom, [term], keyword) ::
@@ -484,20 +484,22 @@ defmodule Task.Supervisor do
   end
 
   defp build_stream(supervisor, link_type, enumerable, fun, options) do
-    shutdown = options[:shutdown]
-    owner = self()
+    fn acc, acc_fun ->
+      shutdown = options[:shutdown]
+      owner = get_owner(self())
 
-    &Task.Supervised.stream(enumerable, &1, &2, get_callers(owner), fun, options, fn ->
-      args = [get_owner(owner), :monitor]
+      Task.Supervised.stream(enumerable, acc, acc_fun, get_callers(self()), fun, options, fn ->
+        args = [owner, :monitor]
 
-      case start_child_with_spec(supervisor, args, :temporary, shutdown) do
-        {:ok, pid} ->
-          if link_type == :link, do: Process.link(pid)
-          {:ok, link_type, pid}
+        case start_child_with_spec(supervisor, args, :temporary, shutdown) do
+          {:ok, pid} ->
+            if link_type == :link, do: Process.link(pid)
+            {:ok, link_type, pid}
 
-        {:error, :max_children} ->
-          {:error, :max_children}
-      end
-    end)
+          {:error, :max_children} ->
+            {:error, :max_children}
+        end
+      end)
+    end
   end
 end

lib/elixir/lib/uri.ex

@@ -12,7 +12,7 @@ defmodule URI do
   """
 
   defstruct scheme: nil,
-            path: "",
+            path: nil,
             query: nil,
             fragment: nil,
             authority: nil,
@@ -24,7 +24,7 @@ defmodule URI do
           authority: authority,
           fragment: nil | binary,
           host: nil | binary,
-          path: binary,
+          path: nil | binary,
           port: nil | :inet.port_number(),
           query: nil | binary,
           scheme: nil | binary,
@@ -479,12 +479,13 @@ defmodule URI do
   Creates a new URI struct from a URI or a string.
 
   If a `%URI{}` struct is given, it returns `{:ok, uri}`. If a string is
-  given, it will parse it and returns `{:ok, uri}`. If the string is
-  invalid, it returns `{:error, part}` instead, with the invalid part of the URI.
+  given, it will parse and validate it. If the string is valid, it returns
+  `{:ok, uri}`, otherwise it returns `{:error, part}` with the invalid part
+  of the URI. For parsing URIs without further validation, see `parse/1`.
 
   This function can parse both absolute and relative URLs. You can check
   if a URI is absolute or relative by checking if the `scheme` field is
-  `nil` or not. All fields may be `nil`, except for the `path`.
+  `nil` or not.
 
   When a URI is given without a port, the value returned by `URI.default_port/1`
   for the URI's scheme is used for the `:port` field. The scheme is also
@@ -551,7 +552,7 @@ defmodule URI do
       {:ok, %URI{
         fragment: nil,
         host: nil,
-        path: "",
+        path: nil,
         port: 443,
         query: "query",
         scheme: "https",
@@ -633,6 +634,8 @@ defmodule URI do
     end
   end
 
+  defp uri_from_map(%{path: ""} = map), do: uri_from_map(%{map | path: nil})
+
   defp uri_from_map(map) do
     uri = Map.merge(%URI{}, map)
 
@@ -657,21 +660,106 @@ defmodule URI do
   end
 
   @doc """
-  Parses a well-formed URI into its components.
+  Parses a URI into its components, without further validation.
 
-  This function is deprecated as it fails to raise in case of invalid URIs.
-  Use `URI.new!/1` or `URI.new/1` instead. In case you want to mimic the
-  behaviour of this function, you can do:
+  This function can parse both absolute and relative URLs. You can check
+  if a URI is absolute or relative by checking if the `scheme` field is
+  nil or not. Furthermore, this function expects both absolute and
+  relative URIs to be well-formed and does not perform any validation.
+  See the "Examples" section below. Use `new/1` if you want more strict
+  validation.
 
-      case URI.new(path) do
-        {:ok, uri} -> uri
-        {:error, _} -> %URI{path: path}
-      end
+  When a URI is given without a port, the value returned by `URI.default_port/1`
+  for the URI's scheme is used for the `:port` field. The scheme is also
+  normalized to lowercase.
+
+  If a `%URI{}` struct is given to this function, this function returns it
+  unmodified.
+
+  > Note: this function sets the field :authority for backwards
+  > compatibility reasons but it is deprecated.
+
+  ## Examples
+
+      iex> URI.parse("https://elixir-lang.org/")
+      %URI{
+        authority: "elixir-lang.org",
+        fragment: nil,
+        host: "elixir-lang.org",
+        path: "/",
+        port: 443,
+        query: nil,
+        scheme: "https",
+        userinfo: nil
+      }
+
+      iex> URI.parse("//elixir-lang.org/")
+      %URI{
+        authority: "elixir-lang.org",
+        fragment: nil,
+        host: "elixir-lang.org",
+        path: "/",
+        port: nil,
+        query: nil,
+        scheme: nil,
+        userinfo: nil
+      }
+
+      iex> URI.parse("/foo/bar")
+      %URI{
+        authority: nil,
+        fragment: nil,
+        host: nil,
+        path: "/foo/bar",
+        port: nil,
+        query: nil,
+        scheme: nil,
+        userinfo: nil
+      }
+
+      iex> URI.parse("foo/bar")
+      %URI{
+        authority: nil,
+        fragment: nil,
+        host: nil,
+        path: "foo/bar",
+        port: nil,
+        query: nil,
+        scheme: nil,
+        userinfo: nil
+      }
+
+  In contrast to `URI.new/1`, this function will parse poorly-formed
+  URIs, for example:
+
+      iex> URI.parse("/invalid_greater_than_in_path/>")
+      %URI{
+        authority: nil,
+        fragment: nil,
+        host: nil,
+        path: "/invalid_greater_than_in_path/>",
+        port: nil,
+        query: nil,
+        scheme: nil,
+        userinfo: nil
+      }
+
+  Another example is a URI with brackets in query strings. It is accepted
+  by `parse/1` but it will be refused by `new/1`:
+
+      iex> URI.parse("/?foo[bar]=baz")
+      %URI{
+        authority: nil,
+        fragment: nil,
+        host: nil,
+        path: "/",
+        port: nil,
+        query: "foo[bar]=baz",
+        scheme: nil,
+        userinfo: nil
+      }
 
-  Also note this function sets the authority field, but the field has been
-  deprecated and it is not set by `URI.new!/1` and `URI.new/1`.
   """
-  @doc deprecated: "Use URI.new/1 or URI.new!/1 instead"
   @spec parse(t | binary) :: t
   def parse(%URI{} = uri), do: uri
 
@@ -705,6 +793,7 @@ defmodule URI do
                 ],
                 parts
 
+    path = nillify(path)
     scheme = nillify(scheme)
     query = nillify_query(query_with_question_mark)
     {authority, userinfo, host, port} = split_authority(authority_with_slashes)
@@ -803,6 +892,7 @@ defmodule URI do
     %{rel | scheme: base.scheme, path: remove_dot_segments_from_path(rel.path)}
   end
 
+  # TODO: Check only for nils in future versions
   def merge(%URI{} = base, %URI{path: rel_path} = rel) when rel_path in ["", nil] do
     %{base | query: rel.query || base.query, fragment: rel.fragment}
   end
@@ -816,7 +906,7 @@ defmodule URI do
     merge(parse(base), parse(rel))
   end
 
-  defp merge_paths("", rel_path), do: merge_paths("/", rel_path)
+  defp merge_paths(nil, rel_path), do: merge_paths("/", rel_path)
   defp merge_paths(_, "/" <> _ = rel_path), do: remove_dot_segments_from_path(rel_path)
 
   defp merge_paths(base_path, rel_path) do
@@ -856,7 +946,8 @@ end
 
 defimpl String.Chars, for: URI do
   def to_string(%{host: host, path: path} = uri)
-      when host != nil and path != "" and binary_part(path, 0, 1) != "/" do
+      when host != nil and is_binary(path) and
+             path != "" and binary_part(path, 0, 1) != "/" do
     raise ArgumentError,
           ":path in URI must be empty or an absolute path if URL has a :host, got: #{inspect(uri)}"
   end

lib/elixir/lib/version.ex

@@ -440,7 +440,7 @@ defmodule Version do
   The internal representation is opaque.
   """
   @spec compile_requirement(Requirement.t()) :: Requirement.t()
-  def compile_requirement(requirement) do
+  def compile_requirement(%Requirement{} = requirement) do
     requirement
   end
 

lib/elixir/pages/compatibility-and-deprecations.md

@@ -8,12 +8,11 @@ Elixir applies bug fixes only to the latest minor branch. Security patches are a
 
 Elixir version | Support
 :------------- | :-----------------------------
-1.13           | Development
-1.12           | Bug fixes and security patches
+1.13           | Bug fixes and security patches
+1.12           | Security patches only
 1.11           | Security patches only
 1.10           | Security patches only
 1.9            | Security patches only
-1.8            | Security patches only
 
 New releases are announced in the read-only [announcements mailing list](https://groups.google.com/group/elixir-lang-ann). All security releases [will be tagged with `[security]`](https://groups.google.com/forum/#!searchin/elixir-lang-ann/%5Bsecurity%5D%7Csort:date).
 
@@ -188,3 +187,4 @@ Version | Deprecated feature                                  | Replaced by (ava
 [v1.10]: https://github.com/elixir-lang/elixir/blob/v1.10/CHANGELOG.md#4-hard-deprecations
 [v1.11]: https://github.com/elixir-lang/elixir/blob/v1.11/CHANGELOG.md#4-hard-deprecations
 [v1.12]: https://github.com/elixir-lang/elixir/blob/v1.12/CHANGELOG.md#4-hard-deprecations
+[v1.13]: https://github.com/elixir-lang/elixir/blob/v1.13/CHANGELOG.md#4-hard-deprecations

lib/elixir/src/elixir.erl

@@ -254,7 +254,7 @@ eval_quoted(Tree, Binding, #{line := Line} = E) ->
 eval_forms(Tree, Binding, Opts) when is_list(Opts) ->
   eval_forms(Tree, Binding, env_for_eval(Opts));
 eval_forms(Tree, RawBinding, OrigE) ->
-  {Vars, Binding} = normalize_binding(RawBinding, [], []),
+  {Vars, Binding} = normalize_binding(RawBinding, [], [], 0),
   E = elixir_env:with_vars(OrigE, Vars),
   {_, S} = elixir_env:env_to_erl(E),
   {Erl, NewErlS, NewExS, NewE} = quoted_to_erl(Tree, E, S),
@@ -275,12 +275,12 @@ eval_forms(Tree, RawBinding, OrigE) ->
       {Value, elixir_erl_var:dump_binding(NewBinding, NewExS, NewErlS), NewE}
   end.
 
-normalize_binding([{Key, Value} | Binding], Vars, Acc) when is_atom(Key) ->
-  normalize_binding(Binding, [{Key, nil} | Vars], [{{Key, nil}, Value} | Acc]);
-normalize_binding([{Pair, Value} | Binding], Vars, Acc) ->
-  normalize_binding(Binding, [Pair | Vars], [{Pair, Value} | Acc]);
-normalize_binding([], Vars, Acc) ->
-  {Vars, Acc}.
+normalize_binding([{Key, Value} | Binding], Vars, Acc, I) when is_atom(Key) ->
+  normalize_binding(Binding, [{{Key, nil}, I} | Vars], [{{Key, nil}, Value} | Acc], I + 1);
+normalize_binding([{Pair, Value} | Binding], Vars, Acc, I) ->
+  normalize_binding(Binding, [{Pair, I} | Vars], [{Pair, Value} | Acc], I + 1);
+normalize_binding([], Vars, Acc, _I) ->
+  {maps:from_list(Vars), Acc}.
 
 recur_eval([Expr | Exprs], Binding, Env) ->
   {value, Value, NewBinding} =
@@ -359,8 +359,8 @@ string_to_tokens(String, StartLine, StartColumn, File, Opts) when is_integer(Sta
       {error, {Location, to_binary(Error), to_binary(Token)}}
   end.
 
-tokens_to_quoted(Tokens, File, Opts) ->
-  handle_parsing_opts(File, Opts),
+tokens_to_quoted(Tokens, WarningFile, Opts) ->
+  handle_parsing_opts(WarningFile, Opts),
 
   try elixir_parser:parse(Tokens) of
     {ok, Forms} ->
@@ -370,7 +370,7 @@ tokens_to_quoted(Tokens, File, Opts) ->
     {error, {Line, _, [Error, Token]}} ->
       {error, {parser_location(Line), to_binary(Error), to_binary(Token)}}
   after
-    erase(elixir_parser_file),
+    erase(elixir_parser_warning_file),
     erase(elixir_parser_columns),
     erase(elixir_token_metadata),
     erase(elixir_literal_encoder)
@@ -406,7 +406,7 @@ parser_location(Meta) ->
 to_binary(List) when is_list(List) -> elixir_utils:characters_to_binary(List);
 to_binary(Atom) when is_atom(Atom) -> atom_to_binary(Atom, utf8).
 
-handle_parsing_opts(File, Opts) ->
+handle_parsing_opts(WarningFile, Opts) ->
   LiteralEncoder =
     case lists:keyfind(literal_encoder, 1, Opts) of
       {literal_encoder, Fun} -> Fun;
@@ -414,7 +414,7 @@ handle_parsing_opts(File, Opts) ->
     end,
   TokenMetadata = lists:keyfind(token_metadata, 1, Opts) == {token_metadata, true},
   Columns = lists:keyfind(columns, 1, Opts) == {columns, true},
-  put(elixir_parser_file, File),
+  put(elixir_parser_warning_file, WarningFile),
   put(elixir_parser_columns, Columns),
   put(elixir_token_metadata, TokenMetadata),
   put(elixir_literal_encoder, LiteralEncoder).

lib/elixir/src/elixir_compiler.erl

@@ -133,7 +133,8 @@ fast_compile({defmodule, Meta, [Mod, [{do, TailBlock}]]}, NoLineE) ->
       'Elixir.Macro':expand(Mod, E)
   end,
 
-  elixir_module:compile(Expanded, Block, [], E).
+  ContextModules = [Expanded | ?key(E, context_modules)],
+  elixir_module:compile(Expanded, Block, [], E#{context_modules := ContextModules}).
 
 %% Bootstrapper
 

lib/elixir/src/elixir_dispatch.erl

@@ -16,7 +16,7 @@ default_functions() ->
 default_macros() ->
   [{?kernel, elixir_imported_macros()}].
 default_requires() ->
-  ['Elixir.Kernel', 'Elixir.Kernel.Typespec'].
+  ['Elixir.Application', 'Elixir.Kernel', 'Elixir.Kernel.Typespec'].
 
 %% This is used by elixir_quote. Note we don't record the
 %% import locally because at that point there is no
@@ -68,9 +68,7 @@ import_function(Meta, Name, Arity, E) ->
   end.
 
 require_function(Meta, Receiver, Name, Arity, E) ->
-  Required =
-    is_element(Receiver, ?key(E, requires)) orelse
-      is_deprecated_require(Meta, Receiver, Name, E),
+  Required = is_element(Receiver, ?key(E, requires)),
 
   case is_macro({Name, Arity}, Receiver, Required) of
     true  -> false;
@@ -182,7 +180,7 @@ do_expand_import(Meta, {Name, Arity} = Tuple, Args, Module, S, E, Result) ->
 expand_require(Meta, Receiver, {Name, Arity} = Tuple, Args, S, E) ->
   Required =
     (Receiver == ?key(E, module)) orelse required(Meta) orelse
-      is_element(Receiver, ?key(E, requires)) orelse is_deprecated_require(Meta, Receiver, Name, E),
+      is_element(Receiver, ?key(E, requires)),
 
   case is_macro(Tuple, Receiver, Required) of
     true ->
@@ -194,13 +192,6 @@ expand_require(Meta, Receiver, {Name, Arity} = Tuple, Args, S, E) ->
       error
   end.
 
-is_deprecated_require(Meta, 'Elixir.Application', Name, E)
-    when Name == 'compile_env'; Name == 'compile_env!' ->
-  elixir_errors:form_warn(Meta, E, ?MODULE, {implicit_application_require, Name}),
-  true;
-is_deprecated_require(_, _, _, _) ->
-  false.
-
 %% Expansion helpers
 
 expand_macro_fun(Meta, Fun, Receiver, Name, Args, S, E) ->
@@ -296,8 +287,6 @@ prune_stacktrace([], _MFA, Info, _E) ->
 
 %% ERROR HANDLING
 
-format_error({implicit_application_require, Name}) ->
-  io_lib:format("you must require Application before calling Application.~ts",[Name]);
 format_error({macro_conflict, {Receiver, Name, Arity}}) ->
   io_lib:format("call to local macro ~ts/~B conflicts with imported ~ts.~ts/~B, "
     "please rename the local macro or remove the conflicting import",

lib/elixir/src/elixir_env.erl

@@ -37,9 +37,12 @@ to_caller({Line, #elixir_ex{vars={Read, _}}, Env}) ->
 to_caller(#{} = Env) ->
   Env.
 
-with_vars(Env, Vars) ->
-  {VarVersions, _} = lists:mapfoldl(fun(Var, I) -> {{Var, I}, I + 1} end, 0, Vars),
-  Env#{versioned_vars := maps:from_list(VarVersions)}.
+with_vars(Env, Vars) when is_list(Vars) ->
+  {ReversedVars, _} =
+    lists:foldl(fun(Var, {Acc, I}) -> {[{Var, I} | Acc], I + 1} end, {[], 0}, Vars),
+  Env#{versioned_vars := maps:from_list(ReversedVars)};
+with_vars(Env, #{} = Vars) ->
+  Env#{versioned_vars := Vars}.
 
 reset_vars(Env) ->
   Env#{versioned_vars := #{}}.

lib/elixir/src/elixir_erl_for.erl

@@ -21,7 +21,7 @@ translate_reduce(Meta, Cases, Expr, Reduce, S) ->
     ({'case', CaseAnn, _, CaseBlock}, InnerAcc) -> {'case', CaseAnn, InnerAcc, CaseBlock}
   end,
 
-  {build_reduce(Ann, TCases, InnerFun, TExpr, TReduce, false, SE), SE}.
+  build_reduce(Ann, TCases, InnerFun, TExpr, TReduce, false, SE).
 
 translate_into(Meta, Cases, Expr, Opts, Return, S) ->
   Ann = ?ann(Meta),
@@ -115,11 +115,11 @@ build_inline(Ann, Clauses, Expr, Into, Uniq, S) ->
 
 build_inline_each(Ann, Clauses, Expr, false, Uniq, S) ->
   InnerFun = fun(InnerExpr, _InnerAcc) -> InnerExpr end,
-  {build_reduce(Ann, Clauses, InnerFun, Expr, {nil, Ann}, Uniq, S), S};
+  build_reduce(Ann, Clauses, InnerFun, Expr, {nil, Ann}, Uniq, S);
 build_inline_each(Ann, Clauses, Expr, {nil, _} = Into, Uniq, S) ->
   InnerFun = fun(InnerExpr, InnerAcc) -> {cons, Ann, InnerExpr, InnerAcc} end,
-  ReduceExpr = build_reduce(Ann, Clauses, InnerFun, Expr, Into, Uniq, S),
-  {?remote(Ann, lists, reverse, [ReduceExpr]), S};
+  {ReduceExpr, SR} = build_reduce(Ann, Clauses, InnerFun, Expr, Into, Uniq, S),
+  {?remote(Ann, lists, reverse, [ReduceExpr]), SR};
 build_inline_each(Ann, Clauses, Expr, {bin, _, []}, Uniq, S) ->
   {InnerValue, SV} = build_var(Ann, S),
   Generated = erl_anno:set_generated(true, Ann),
@@ -138,8 +138,8 @@ build_inline_each(Ann, Clauses, Expr, {bin, _, []}, Uniq, S) ->
     ]}
   end,
 
-  ReduceExpr = build_reduce(Ann, Clauses, InnerFun, Expr, {nil, Ann}, Uniq, SV),
-  {?remote(Ann, erlang, list_to_bitstring, [ReduceExpr]), SV}.
+  {ReduceExpr, SR} = build_reduce(Ann, Clauses, InnerFun, Expr, {nil, Ann}, Uniq, SV),
+  {?remote(Ann, erlang, list_to_bitstring, [ReduceExpr]), SR}.
 
 build_into(Ann, Clauses, Expr, {map, _, []}, Uniq, S) ->
   {ReduceExpr, SR} = build_inline_each(Ann, Clauses, Expr, {nil, Ann}, Uniq, S),
@@ -161,7 +161,7 @@ build_into(Ann, Clauses, Expr, Into, Uniq, S) ->
     ?remote(Ann, 'Elixir.Collectable', into, [Into])
   },
 
-  IntoReduceExpr = build_reduce(Ann, Clauses, InnerFun, Expr, Acc, Uniq, SD),
+  {IntoReduceExpr, SN} = build_reduce(Ann, Clauses, InnerFun, Expr, Acc, Uniq, SD),
 
   TryExpr =
     {'try', Ann,
@@ -173,7 +173,7 @@ build_into(Ann, Clauses, Expr, Into, Uniq, S) ->
       [stacktrace_clause(Ann, Fun, Acc, Kind, Reason, Stack)],
       []},
 
-  {{block, Ann, [MatchExpr, TryExpr]}, SD}.
+  {{block, Ann, [MatchExpr, TryExpr]}, SN}.
 
 stacktrace_clause(Ann, Fun, Acc, Kind, Reason, Stack) ->
   {clause, Ann,
@@ -186,7 +186,7 @@ stacktrace_clause(Ann, Fun, Acc, Kind, Reason, Stack) ->
 
 build_reduce(Ann, Clauses, InnerFun, Expr, Into, false, S) ->
   {Acc, SA} = build_var(Ann, S),
-  build_reduce_each(Clauses, InnerFun(Expr, Acc), Into, Acc, SA);
+  {build_reduce_each(Clauses, InnerFun(Expr, Acc), Into, Acc, SA), SA};
 build_reduce(Ann, Clauses, InnerFun, Expr, Into, true, S) ->
   %% Those variables are used only inside the anonymous function
   %% so we don't need to worry about returning the scope.
@@ -209,7 +209,7 @@ build_reduce(Ann, Clauses, InnerFun, Expr, Into, true, S) ->
   ]},
 
   EnumReduceCall = build_reduce_each(Clauses, InnerExpr, NewInto, Acc, SU),
-  ?remote(Ann, erlang, element, [{integer, Ann, 1}, EnumReduceCall]).
+  {?remote(Ann, erlang, element, [{integer, Ann, 1}, EnumReduceCall]), SU}.
 
 build_reduce_each([{enum, Meta, Left, Right, Filters} | T], Expr, Arg, Acc, S) ->
   Ann = ?ann(Meta),

lib/elixir/src/elixir_errors.erl

@@ -87,9 +87,12 @@ snippet(InputString, Location, StartLine, StartColumn) ->
   case lists:keyfind(column, 1, Location) of
     {column, Column} ->
       Lines = string:split(InputString, "\n", all),
-      Snippet = elixir_utils:characters_to_binary(lists:nth(Line - StartLine + 1, Lines)),
+      Snippet = (lists:nth(Line - StartLine + 1, Lines)),
       Offset = if Line == StartLine -> Column - StartColumn; true -> Column - 1 end,
-      #{content => Snippet, offset => Offset};
+      case string:trim(Snippet, leading) of
+        [] -> nil;
+        _ -> #{content => elixir_utils:characters_to_binary(Snippet), offset => Offset}
+      end;
 
     false ->
       nil

lib/elixir/src/elixir_interpolation.erl

@@ -3,9 +3,7 @@
 -export([extract/6, unescape_string/1, unescape_string/2,
 unescape_tokens/1, unescape_map/1]).
 -include("elixir.hrl").
--define(is_hex(S), ((S >= $0 andalso S =< $9) orelse
-                    (S >= $A andalso S =< $F) orelse
-                    (S >= $a andalso S =< $f))).
+-include("elixir_tokenizer.hrl").
 
 %% Extract string interpolations
 
@@ -37,6 +35,9 @@ extract([$\n | Rest], Buffer, Output, Line, _Column, Scope, Interpol, Last) ->
 extract([$\\, Last | Rest], Buffer, Output, Line, Column, Scope, Interpol, Last) ->
   extract(Rest, [Last | Buffer], Output, Line, Column+2, Scope, Interpol, Last);
 
+extract([$\\, Last, Last, Last | Rest], Buffer, Output, Line, Column, Scope, Interpol, [Last, Last, Last] = All) ->
+  extract(Rest, [Last, Last, Last | Buffer], Output, Line, Column+4, Scope, Interpol, All);
+
 extract([$\\, $#, ${ | Rest], Buffer, Output, Line, Column, Scope, true, Last) ->
   extract(Rest, [${, $#, $\\ | Buffer], Output, Line, Column+1, Scope, true, Last);
 
@@ -57,8 +58,8 @@ extract([$#, ${ | Rest], Buffer, Output, Line, Column, Scope, true, Last) ->
       {error, {string, Line, Column, "missing interpolation terminator: \"}\"", []}}
   end;
 
-extract([$\\, Char | Rest], Buffer, Output, Line, Column, Scope, Interpol, Last) ->
-  extract(Rest, [Char, $\\ | Buffer], Output, Line, Column + 2, Scope, Interpol, Last);
+extract([$\\ | Rest], Buffer, Output, Line, Column, Scope, Interpol, Last) ->
+  extract_char(Rest, [$\\ | Buffer], Output, Line, Column + 1, Scope, Interpol, Last);
 
 %% Catch all clause
 
@@ -67,8 +68,21 @@ extract([Char1, Char2 | Rest], Buffer, Output, Line, Column, Scope, Interpol, La
   extract([Char2 | Rest], [Char1 | Buffer], Output, Line, Column + 1, Scope, Interpol, Last);
 
 extract(Rest, Buffer, Output, Line, Column, Scope, Interpol, Last) ->
+  extract_char(Rest, Buffer, Output, Line, Column, Scope, Interpol, Last).
+
+extract_char(Rest, Buffer, Output, Line, Column, Scope, Interpol, Last) ->
   [Char | NewRest] = unicode_util:gc(Rest),
-  extract(NewRest, [Char | Buffer], Output, Line, Column + 1, Scope, Interpol, Last).
+
+  if
+    ?bidi(Char) ->
+      Token = io_lib:format("\\u~4.16.0B", [Char]),
+      Pre = "invalid bidirectional formatting character in string: ",
+      Pos = io_lib:format(". If you want to use such character, use it in its escaped ~ts form instead", [Token]),
+      {error, {Line, Column, {Pre, Pos}, Token}};
+
+    true ->
+      extract(NewRest, [Char | Buffer], Output, Line, Column + 1, Scope, Interpol, Last)
+  end.
 
 %% Handle newlines. Heredocs require special attention
 

lib/elixir/src/elixir_parser.yrl

@@ -552,6 +552,7 @@ kw_base -> kw_eol container_expr : [{'$1', '$2'}].
 kw_base -> kw_base ',' kw_eol container_expr : [{'$3', '$4'} | '$1'].
 
 kw_call -> kw_base : reverse('$1').
+kw_call -> kw_base ',' : warn_trailing_comma('$2'), reverse('$1').
 kw_call -> kw_base ',' matched_expr : maybe_bad_keyword_call_follow_up('$2', '$1', '$3').
 
 kw_data -> kw_base : reverse('$1').
@@ -639,7 +640,6 @@ map -> struct_op struct_expr eol map_args : {'%', meta_from_token('$1'), ['$2',
 
 Erlang code.
 
--define(file(), get(elixir_parser_file)).
 -define(columns(), get(elixir_parser_columns)).
 -define(token_metadata(), get(elixir_token_metadata)).
 
@@ -728,10 +728,12 @@ build_op(AST, {_Kind, Location, '//'}, Right) ->
 
 build_op({UOp, _, [Left]}, {_Kind, {Line, Column, _} = Location, 'in'}, Right) when ?rearrange_uop(UOp) ->
   %% TODO: Remove "not left in right" rearrangement on v2.0
-  elixir_errors:erl_warn({Line, Column}, ?file(),
+  warn(
+    {Line, Column},
     "\"not expr1 in expr2\" is deprecated. "
     "Instead use \"expr1 not in expr2\" if you require Elixir v1.5+, "
-    "or \"not(expr1 in expr2)\" if you have to support earlier Elixir versions"),
+    "or \"not(expr1 in expr2)\" if you have to support earlier Elixir versions"
+  ),
   Meta = meta_from_location(Location),
   {UOp, Meta, [{'in', Meta, [Left, Right]}]};
 
@@ -1166,17 +1168,24 @@ error_invalid_kw_identifier({_, Location, do}) ->
 error_invalid_kw_identifier({_, Location, KW}) ->
   return_error(Location, "syntax error before: ", "'" ++ atom_to_list(KW) ++ ":'").
 
+%% TODO: Make this an error on v2.0
+warn_trailing_comma({',', {Line, Column, _}}) ->
+  warn({Line, Column}, "trailing commas are not allowed inside function/macro call arguments").
+
 %% TODO: Make this an error on v2.0
 warn_empty_paren({_, {Line, Column, _}}) ->
-  elixir_errors:erl_warn({Line, Column}, ?file(),
+  warn(
+    {Line, Column},
     "invalid expression (). "
     "If you want to invoke or define a function, make sure there are "
     "no spaces between the function name and its arguments. If you wanted "
-    "to pass an empty block or code, pass a value instead, such as a nil or an atom").
+    "to pass an empty block or code, pass a value instead, such as a nil or an atom"
+  ).
 
 %% TODO: Make this an error on v2.0
 warn_pipe({arrow_op, {Line, Column, _}, Op}, {_, [_ | _], [_ | _]}) ->
-  elixir_errors:erl_warn({Line, Column}, ?file(),
+  warn(
+    {Line, Column},
     io_lib:format(
       "parentheses are required when piping into a function call. For example:\n\n"
       "    foo 1 ~ts bar 2 ~ts baz 3\n\n"
@@ -1190,6 +1199,14 @@ warn_pipe(_Token, _) ->
   ok.
 
 warn_empty_stab_clause({stab_op, {Line, Column, _}, '->'}) ->
-  elixir_errors:erl_warn({Line, Column}, ?file(),
+  warn(
+    {Line, Column},
     "an expression is always required on the right side of ->. "
-    "Please provide a value after ->").
+    "Please provide a value after ->"
+  ).
+
+warn(LineColumn, Message) ->
+  case get(elixir_parser_warning_file) of
+    nil -> ok;
+    File -> elixir_errors:erl_warn(LineColumn, File, Message)
+  end.

lib/elixir/src/elixir_tokenizer.erl

@@ -1,28 +1,8 @@
 -module(elixir_tokenizer).
 -include("elixir.hrl").
+-include("elixir_tokenizer.hrl").
 -export([tokenize/1, tokenize/3, tokenize/4, invalid_do_error/1]).
 
-%% Numbers
--define(is_hex(S), (?is_digit(S) orelse (S >= $A andalso S =< $F) orelse (S >= $a andalso S =< $f))).
--define(is_bin(S), (S >= $0 andalso S =< $1)).
--define(is_octal(S), (S >= $0 andalso S =< $7)).
-
-%% Digits and letters
--define(is_digit(S), (S >= $0 andalso S =< $9)).
--define(is_upcase(S), (S >= $A andalso S =< $Z)).
--define(is_downcase(S), (S >= $a andalso S =< $z)).
-
-%% Others
--define(is_quote(S), (S =:= $" orelse S =:= $')).
--define(is_sigil(S), (S =:= $/ orelse S =:= $< orelse S =:= $" orelse S =:= $' orelse
-                      S =:= $[ orelse S =:= $( orelse S =:= ${ orelse S =:= $|)).
-
-%% Spaces
--define(is_horizontal_space(S), (S =:= $\s orelse S =:= $\t)).
--define(is_vertical_space(S), (S =:= $\r orelse S =:= $\n)).
--define(is_space(S), (?is_horizontal_space(S) orelse ?is_vertical_space(S))).
-
-%% Operators
 -define(at_op(T),
   T =:= $@).
 
@@ -198,9 +178,13 @@ tokenize([$0, $o, H | T], Line, Column, Scope, Tokens) when ?is_octal(H) ->
 % Comments
 
 tokenize([$# | String], Line, Column, Scope, Tokens) ->
-  {Rest, Comment} = tokenize_comment(String, [$#]),
-  preserve_comments(Line, Column, Tokens, Comment, Rest, Scope),
-  tokenize(Rest, Line, Column, Scope, reset_eol(Tokens));
+  case tokenize_comment(String, [$#]) of
+    {error, Char} ->
+      error_comment(Char, [$# | String], Line, Column, Scope, Tokens);
+    {Rest, Comment} ->
+      preserve_comments(Line, Column, Tokens, Comment, Rest, Scope),
+      tokenize(Rest, Line, Column, Scope, reset_eol(Tokens))
+  end;
 
 % Sigils
 
@@ -645,9 +629,7 @@ tokenize([$% | T], Line, Column, Scope, Tokens) ->
   tokenize(T, Line, Column + 1, Scope, [{'%', {Line, Column, nil}} | Tokens]);
 
 tokenize([$. | T], Line, Column, Scope, Tokens) ->
-  DotInfo = {Line, Column, nil},
-  {Rest, EndLine, EndColumn} = strip_dot_space(T, Line, Column + 1, [{'.', DotInfo}| Tokens], Scope),
-  handle_dot([$. | Rest], EndLine, EndColumn, DotInfo, Scope, Tokens);
+  tokenize_dot(T, Line, Column + 1, {Line, Column, nil}, Scope, Tokens);
 
 % Identifiers
 
@@ -719,18 +701,23 @@ strip_horizontal_space([H | T], Counter) when ?is_horizontal_space(H) ->
 strip_horizontal_space(T, Counter) ->
   {T, Counter}.
 
-strip_dot_space(T, Line, Column, Tokens, Scope) ->
+tokenize_dot(T, Line, Column, DotInfo, Scope, Tokens) ->
   case strip_horizontal_space(T, 0) of
-    {"#" ++ R, _} ->
-      {Rest, Comment} = tokenize_comment(R, [$#]),
-      preserve_comments(Line, Column, Tokens, Comment, Rest, Scope),
-      strip_dot_space(Rest, Line, 1, reset_eol(Tokens), Scope);
+    {[$# | R], _} ->
+      case tokenize_comment(R, [$#]) of
+        {error, Char} ->
+          error_comment(Char, [$# | R], Line, Column, Scope, Tokens);
+
+        {Rest, Comment} ->
+          preserve_comments(Line, Column, Tokens, Comment, Rest, Scope),
+          tokenize_dot(Rest, Line, 1, DotInfo, Scope, Tokens)
+      end;
     {"\r\n" ++ Rest, _} ->
-      strip_dot_space(Rest, Line + 1, 1, eol(Line, Column, Tokens), Scope);
+      tokenize_dot(Rest, Line + 1, 1, DotInfo, Scope, Tokens);
     {"\n" ++ Rest, _} ->
-      strip_dot_space(Rest, Line + 1, 1, eol(Line, Column, Tokens), Scope);
+      tokenize_dot(Rest, Line + 1, 1, DotInfo, Scope, Tokens);
     {Rest, Length} ->
-      {Rest, Line, Column + Length}
+      handle_dot([$. | Rest], Line, Column + Length, DotInfo, Scope, Tokens)
   end.
 
 handle_char(0)   -> {"\\0", "null byte"};
@@ -1171,11 +1158,18 @@ tokenize_comment("\r\n" ++ _ = Rest, Acc) ->
   {Rest, lists:reverse(Acc)};
 tokenize_comment("\n" ++ _ = Rest, Acc) ->
   {Rest, lists:reverse(Acc)};
+tokenize_comment([H | _Rest], _) when ?bidi(H) ->
+  {error, H};
 tokenize_comment([H | Rest], Acc) ->
   tokenize_comment(Rest, [H | Acc]);
 tokenize_comment([], Acc) ->
   {[], lists:reverse(Acc)}.
 
+error_comment(H, Comment, Line, Column, Scope, Tokens) ->
+  Token = io_lib:format("\\u~4.16.0B", [H]),
+  Reason = {Line, Column, "invalid bidirectional formatting character in comment: ", Token},
+  error(Reason, Comment, Scope, Tokens).
+
 preserve_comments(Line, Column, Tokens, Comment, Rest, Scope) ->
   case Scope#elixir_tokenizer.preserve_comments of
     Fun when is_function(Fun) ->

lib/elixir/src/elixir_tokenizer.hrl

@@ -0,0 +1,31 @@
+%% Numbers
+-define(is_hex(S), (?is_digit(S) orelse (S >= $A andalso S =< $F) orelse (S >= $a andalso S =< $f))).
+-define(is_bin(S), (S >= $0 andalso S =< $1)).
+-define(is_octal(S), (S >= $0 andalso S =< $7)).
+
+%% Digits and letters
+-define(is_digit(S), (S >= $0 andalso S =< $9)).
+-define(is_upcase(S), (S >= $A andalso S =< $Z)).
+-define(is_downcase(S), (S >= $a andalso S =< $z)).
+
+%% Others
+-define(is_quote(S), (S =:= $" orelse S =:= $')).
+-define(is_sigil(S), (S =:= $/ orelse S =:= $< orelse S =:= $" orelse S =:= $' orelse
+                      S =:= $[ orelse S =:= $( orelse S =:= ${ orelse S =:= $|)).
+
+%% Spaces
+-define(is_horizontal_space(S), (S =:= $\s orelse S =:= $\t)).
+-define(is_vertical_space(S), (S =:= $\r orelse S =:= $\n)).
+-define(is_space(S), (?is_horizontal_space(S) orelse ?is_vertical_space(S))).
+
+%% Bidirectional control
+%% Retrieved from https://trojansource.codes/trojan-source.pdf
+-define(bidi(C), C =:= 16#202A;
+                 C =:= 16#202B;
+                 C =:= 16#202D;
+                 C =:= 16#202E;
+                 C =:= 16#2066;
+                 C =:= 16#2067;
+                 C =:= 16#2068;
+                 C =:= 16#202C;
+                 C =:= 16#2069).

lib/elixir/test/elixir/application_test.exs

@@ -6,6 +6,8 @@ defmodule ApplicationTest do
   import PathHelpers
   import ExUnit.CaptureIO
 
+  @app :elixir
+
   test "application environment" do
     assert_raise ArgumentError, ~r/because the application was not loaded nor configured/, fn ->
       Application.fetch_env!(:unknown, :unknown)
@@ -75,12 +77,14 @@ defmodule ApplicationTest do
 
       assert Application.put_env(:elixir, :unknown, nested: [key: :value]) == :ok
 
+      assert compile_env(@app, :unknown, :default) == [nested: [key: :value]]
       assert compile_env(:elixir, :unknown, :default) == [nested: [key: :value]]
       assert_received {:compile_env, :elixir, [:unknown], {:ok, [nested: [key: :value]]}}
 
       assert compile_env(:elixir, :unknown) == [nested: [key: :value]]
       assert_received {:compile_env, :elixir, [:unknown], {:ok, [nested: [key: :value]]}}
 
+      assert compile_env!(@app, :unknown) == [nested: [key: :value]]
       assert compile_env!(:elixir, :unknown) == [nested: [key: :value]]
       assert_received {:compile_env, :elixir, [:unknown], {:ok, [nested: [key: :value]]}}
 

lib/elixir/test/elixir/base_test.exs

@@ -799,10 +799,21 @@ defmodule BaseTest do
         |> Enum.shuffle()
         |> IO.iodata_to_binary()
 
+      allowed_opts =
+        encode
+        |> Function.info()
+        |> Keyword.fetch!(:name)
+        |> case do
+          :encode16 -> [:case]
+          :encode64 -> [:padding]
+          :url_encode64 -> [:padding]
+          _ -> [:case, :padding]
+        end
+
       expected =
         data
-        |> encode.(case: encode_case, pad: pad?)
-        |> decode.(case: decode_case, pad: pad?)
+        |> encode.(Keyword.take([case: encode_case, padding: pad?], allowed_opts))
+        |> decode.(Keyword.take([case: decode_case, padding: pad?], allowed_opts))
 
       assert data == expected,
              "identity did not match for #{inspect(data)} when #{inspect(encode)} (#{encode_case})"

lib/elixir/test/elixir/code_fragment_test.exs

@@ -915,6 +915,11 @@ defmodule CodeFragmentTest do
       assert cc2q("foo(123, ~r/") == s2q("foo(123, __cursor__())")
     end
 
+    test "no warnings" do
+      assert cc2q(~s"?\\ ") == s2q("__cursor__()")
+      assert cc2q(~s"{fn -> end, ") == s2q("{fn -> nil end, __cursor__()}")
+    end
+
     test "options" do
       opts = [columns: true]
       assert cc2q("foo(", opts) == s2q("foo(__cursor__())", opts)

lib/elixir/test/elixir/code_normalizer/formatted_ast_test.exs

@@ -15,11 +15,14 @@ defmodule Code.Normalizer.FormatterASTTest do
     good = String.trim(good)
 
     to_quoted_opts =
-      [
-        literal_encoder: &{:ok, {:__block__, &2, [&1]}},
-        token_metadata: true,
-        unescape: false
-      ] ++ opts
+      Keyword.merge(
+        [
+          literal_encoder: &{:ok, {:__block__, &2, [&1]}},
+          token_metadata: true,
+          unescape: false
+        ],
+        opts
+      )
 
     {quoted, comments} = Code.string_to_quoted_with_comments!(good, to_quoted_opts)
 
@@ -161,6 +164,12 @@ defmodule Code.Normalizer.FormatterASTTest do
     end
   end
 
+  describe "lists" do
+    test "on module attribute" do
+      assert_same ~S"@foo [1]"
+    end
+  end
+
   describe "charlists" do
     test "without escapes" do
       assert_same ~S['']
@@ -349,6 +358,18 @@ defmodule Code.Normalizer.FormatterASTTest do
       assert_same ~S"{:wrapped, 1, [opt1: true, opt2: false]}"
       assert_same ~S"{:unwrapped, 1, opt1: true, opt2: false}"
     end
+
+    test "on module attribute" do
+      assert_same ~S"""
+      @foo a: b,
+           c: d
+      """
+
+      assert_same ~S"@foo [
+        a: b,
+        c: d
+      ]"
+    end
   end
 
   describe "preserves user choice on parenthesis" do
@@ -464,6 +485,29 @@ defmodule Code.Normalizer.FormatterASTTest do
       """
     end
 
+    test "handles comments with unescaped literal" do
+      assert_same """
+                  # before
+                  Mix.install([:foo])
+                  # after
+                  """,
+                  literal_encoder: fn literal, _ -> {:ok, literal} end
+
+      assert_same """
+                  # before
+                  Mix.install([1 + 2, :foo])
+                  # after
+                  """,
+                  literal_encoder: fn literal, _ -> {:ok, literal} end
+
+      assert_same """
+                  # before
+                  Mix.install([:foo, 1 + 2])
+                  # after
+                  """,
+                  literal_encoder: fn literal, _ -> {:ok, literal} end
+    end
+
     test "before and after expressions with newlines" do
       assert_same """
       # before comment
@@ -516,5 +560,12 @@ defmodule Code.Normalizer.FormatterASTTest do
       end
       """
     end
+
+    test "keyword literals with variable values" do
+      assert_same(~S"""
+      foo = foo()
+      [foo: foo]
+      """)
+    end
   end
 end

lib/elixir/test/elixir/code_normalizer/quoted_ast_test.exs

@@ -6,6 +6,7 @@ defmodule Code.Normalizer.QuotedASTTest do
   describe "quoted_to_algebra/2" do
     test "variable" do
       assert quoted_to_string(quote(do: foo)) == "foo"
+      assert quoted_to_string({:{}, [], nil}) == "{}"
     end
 
     test "local call" do
@@ -150,7 +151,7 @@ defmodule Code.Normalizer.QuotedASTTest do
       assert quoted_to_string(quote do: {1, 2}) == "{1, 2}"
       assert quoted_to_string(quote do: {1}) == "{1}"
       assert quoted_to_string(quote do: {1, 2, 3}) == "{1, 2, 3}"
-      assert quoted_to_string(quote do: {1, 2, 3, foo: :bar}) == "{1, 2, 3, [foo: :bar]}"
+      assert quoted_to_string(quote do: {1, 2, 3, foo: :bar}) == "{1, 2, 3, foo: :bar}"
     end
 
     test "tuple call" do
@@ -536,11 +537,31 @@ defmodule Code.Normalizer.QuotedASTTest do
 
       assert quoted_to_string(quote(do: {[]})) == "{[]}"
       assert quoted_to_string(quote(do: {[a: b]})) == "{[a: b]}"
-      assert quoted_to_string(quote(do: {x, a: b})) == "{x, [a: b]}"
+      assert quoted_to_string(quote(do: {x, a: b})) == "{x, a: b}"
       assert quoted_to_string(quote(do: foo(else: a))) == "foo(else: a)"
       assert quoted_to_string(quote(do: foo(catch: a))) == "foo(catch: a)"
       assert quoted_to_string(quote(do: foo |> [bar: :baz])) == "foo |> [bar: :baz]"
     end
+
+    test "list in module attribute" do
+      assert quoted_to_string(
+               quote do
+                 @foo [1]
+               end
+             ) == "@foo [1]"
+
+      assert quoted_to_string(
+               quote do
+                 @foo [foo: :bar]
+               end
+             ) == "@foo foo: :bar"
+
+      assert quoted_to_string(
+               quote do
+                 @foo [1, foo: :bar]
+               end
+             ) == "@foo [1, foo: :bar]"
+    end
   end
 
   describe "quoted_to_algebra/2 escapes" do

lib/elixir/test/elixir/code_test.exs

@@ -35,10 +35,6 @@ defmodule CodeTest do
       assert {3, _} = Code.eval_string("a + b", [a: 1, b: 2], __ENV__)
     end
 
-    test "returns bindings from a different context" do
-      assert Code.eval_string("var!(a, Sample) = 1") == {1, [{{:a, Sample}, 1}]}
-    end
-
     test "supports unnamed scopes" do
       assert {%RuntimeError{}, [a: %RuntimeError{}]} =
                Code.eval_string("a = (try do (raise \"hello\") rescue e -> e end)")
@@ -48,6 +44,16 @@ defmodule CodeTest do
       assert Code.eval_string("Kernel.if true, do: :ok", [], requires: [Z, Kernel]) == {:ok, []}
     end
 
+    test "returns bindings from a different context" do
+      assert Code.eval_string("var!(a, Sample) = 1") == {1, [{{:a, Sample}, 1}]}
+    end
+
+    test "does not raise on duplicate bindings" do
+      # The order of which values win is not guaranteed, but it should evaluate successfully.
+      assert Code.eval_string("b = String.Chars.to_string(a)", a: 0, a: 1) ==
+               {"1", [{:b, "1"}, {:a, 1}]}
+    end
+
     test "with many options" do
       options = [
         functions: [{Kernel, [is_atom: 1]}],

lib/elixir/test/elixir/enum_test.exs

@@ -823,6 +823,7 @@ defmodule EnumTest do
       end
 
       assert Enum.slide([:a, :b, :c, :d, :e, :f], 3..3, 2) == [:a, :b, :d, :c, :e, :f]
+      assert Enum.slide([:a, :b, :c, :d, :e, :f], 3, 3) == [:a, :b, :c, :d, :e, :f]
     end
 
     test "on a subsection of a list reorders the range correctly" do
@@ -946,6 +947,33 @@ defmodule EnumTest do
         assert slide.(list) == slide.(range)
       end
     end
+
+    test "inserts at negative indices" do
+      for zero_to_5 <- [0..5, Enum.to_list(0..5)] do
+        assert Enum.slide(zero_to_5, 0, -1) == [1, 2, 3, 4, 5, 0]
+        assert Enum.slide(zero_to_5, 1, -1) == [0, 2, 3, 4, 5, 1]
+        assert Enum.slide(zero_to_5, 1..2, -2) == [0, 3, 4, 1, 2, 5]
+        assert Enum.slide(zero_to_5, -5..-4//1, -2) == [0, 3, 4, 1, 2, 5]
+      end
+
+      assert Enum.slide([:a, :b, :c, :d, :e, :f], -5..-3//1, -2) ==
+               Enum.slide([:a, :b, :c, :d, :e, :f], 1..3, 4)
+    end
+
+    test "raises when insertion index would fall inside the range" do
+      for zero_to_5 <- [0..5, Enum.to_list(0..5)] do
+        assert_raise RuntimeError, fn ->
+          Enum.slide(zero_to_5, 2..3, -3)
+        end
+      end
+
+      for zero_to_10 <- [0..10, Enum.to_list(0..10)],
+          insertion_idx <- 3..5 do
+        assert_raise RuntimeError, fn ->
+          assert Enum.slide(zero_to_10, 2..5, insertion_idx)
+        end
+      end
+    end
   end
 
   test "scan/2" do

lib/elixir/test/elixir/exception_test.exs

@@ -423,7 +423,11 @@ defmodule ExceptionTest do
 
       assert blame_message(123, &apply(Kernel, :+, &1)) ==
                "you attempted to apply a function named :+ on module Kernel with arguments 123. " <>
-                 "Arguments (the third argument of apply) must always be a list"
+                 "Arguments (the third argument of apply) must always be a proper list"
+
+      assert blame_message(123, &apply(Kernel, :+, [&1 | 456])) ==
+               "you attempted to apply a function named :+ on module Kernel with arguments [123 | 456]. " <>
+                 "Arguments (the third argument of apply) must always be a proper list"
     end
 
     test "annotates function clause errors" do

lib/elixir/test/elixir/kernel/binary_test.exs

@@ -47,6 +47,15 @@ defmodule Kernel.BinaryTest do
              )
   end
 
+  test "heredoc with heredoc inside interpolation" do
+    assert """
+           1
+           #{"""
+           2
+           """}
+           """ == "1\n2\n\n"
+  end
+
   test "UTF-8" do
     assert byte_size(" ゆんゆん") == 13
   end

lib/elixir/test/elixir/kernel/cli_test.exs

@@ -71,10 +71,14 @@ defmodule Kernel.CLITest do
     assert output =~ "Erlang/OTP #{System.otp_release()}"
     assert output =~ "Elixir #{System.version()}"
 
+    output = iex('--version')
+    assert output =~ "Erlang/OTP #{System.otp_release()}"
+    assert output =~ "IEx #{System.version()}"
+
     output = elixir('--version -e "IO.puts(:test_output)"')
     assert output =~ "Erlang/OTP #{System.otp_release()}"
     assert output =~ "Elixir #{System.version()}"
-    assert output =~ "test_output"
+    assert output =~ "Standalone options can't be combined with other options"
   end
 
   test "--short-version smoke test" do

lib/elixir/test/elixir/kernel/comprehension_test.exs

@@ -98,6 +98,19 @@ defmodule Kernel.ComprehensionTest do
     assert Process.get(:into_halt)
   end
 
+  test "nested for comprehensions with unique values" do
+    assert for(x <- [1, 1, 2], uniq: true, do: for(y <- [3, 3], uniq: true, do: x * y)) == [
+             [3],
+             [6]
+           ]
+
+    assert for(<<x <- "abcabc">>,
+             uniq: true,
+             into: "",
+             do: for(<<y <- "zz">>, uniq: true, into: "", do: to_bin(x) <> to_bin(y))
+           ) == "azbzcz"
+  end
+
   test "for comprehensions with nilly filters" do
     assert for(x <- 1..3, nilly(), do: x * 2) == []
   end

lib/elixir/test/elixir/kernel/errors_test.exs

@@ -491,6 +491,21 @@ defmodule Kernel.ErrorsTest do
     assert_eval_raise TokenMissingError, ~r/nofile:1:3: invalid escape \\ at end of file/, '1 \\'
   end
 
+  test "show snippet on missing tokens" do
+    assert_eval_raise TokenMissingError,
+                      "nofile:1:25: missing terminator: end (for \"do\" starting at line 1)\n" <>
+                        "    |\n" <>
+                        "  1 | defmodule ShowSnippet do\n" <>
+                        "    |                         ^",
+                      'defmodule ShowSnippet do'
+  end
+
+  test "don't show snippet when error line is empty" do
+    assert_eval_raise TokenMissingError,
+                      "nofile:3:1: missing terminator: end (for \"do\" starting at line 1)",
+                      'defmodule ShowSnippet do\n\n'
+  end
+
   test "function local conflict" do
     assert_eval_raise CompileError,
                       "nofile:3: imported Kernel.&&/2 conflicts with local function",
@@ -816,6 +831,24 @@ defmodule Kernel.ErrorsTest do
                       'x = 8; <<a, b::size(^x)>> = <<?a, ?b>>'
   end
 
+  test "invalid bidi in source" do
+    assert_eval_raise SyntaxError,
+                      ~r"nofile:1:1: invalid bidirectional formatting character in comment: \\u202A",
+                      '# This is a \u202A'
+
+    assert_eval_raise SyntaxError,
+                      ~r"nofile:1:5: invalid bidirectional formatting character in comment: \\u202A",
+                      'foo. # This is a \u202A'
+
+    assert_eval_raise SyntaxError,
+                      ~r"nofile:1:12: invalid bidirectional formatting character in string: \\u202A. If you want to use such character, use it in its escaped \\u202A form instead",
+                      '"this is a \u202A"'
+
+    assert_eval_raise SyntaxError,
+                      ~r"nofile:1:13: invalid bidirectional formatting character in string: \\u202A. If you want to use such character, use it in its escaped \\u202A form instead",
+                      '"this is a \\\u202A"'
+  end
+
   test "function head with guard" do
     assert_eval_raise CompileError, "nofile:2: missing :do option in \"def\"", '''
     defmodule Kernel.ErrorsTest.BodyessFunctionWithGuard do

lib/elixir/test/elixir/kernel/sigils_test.exs

@@ -42,6 +42,14 @@ bar) in ["foo\\\nbar", "foo\\\r\nbar"]
            """
   end
 
+  test "sigil S with escaping" do
+    assert "\"" == ~S"\""
+
+    assert "\"\"\"\n" == ~S"""
+           \"""
+           """
+  end
+
   test "sigil s/S expand to binary when possible" do
     assert Macro.expand(quote(do: ~s(foo)), __ENV__) == "foo"
     assert Macro.expand(quote(do: ~S(foo)), __ENV__) == "foo"

lib/elixir/test/elixir/kernel/warning_test.exs

@@ -1163,9 +1163,11 @@ defmodule Kernel.WarningTest do
 
             def without_specs(term, options \\ [])
 
+            @callback foo :: {:ok, term}
             @callback foo(term) :: {:ok, term}
             @callback foo(term, keyword) :: {:ok, term, keyword}
 
+            @callback foo_when :: {:ok, x} when x: term
             @callback foo_when(x) :: {:ok, x} when x: term
             @callback foo_when(x, opts) :: {:ok, x, opts} when x: term, opts: keyword
 
@@ -1178,12 +1180,18 @@ defmodule Kernel.WarningTest do
         """)
       end)
 
+    assert message =~
+             "cannot define @callback foo/0 inside protocol, use def/1 to outline your protocol definition\n  nofile:1"
+
     assert message =~
              "cannot define @callback foo/1 inside protocol, use def/1 to outline your protocol definition\n  nofile:1"
 
     assert message =~
              "cannot define @callback foo/2 inside protocol, use def/1 to outline your protocol definition\n  nofile:1"
 
+    assert message =~
+             "cannot define @callback foo_when/0 inside protocol, use def/1 to outline your protocol definition\n  nofile:1"
+
     assert message =~
              "cannot define @callback foo_when/1 inside protocol, use def/1 to outline your protocol definition\n  nofile:1"
 
@@ -1901,6 +1909,11 @@ defmodule Kernel.WarningTest do
     end
   end
 
+  test "warnings on trailing comma on call" do
+    assert capture_err(fn -> Code.eval_string("Keyword.merge([], foo: 1,)") end) =~
+             "trailing commas are not allowed inside function/macro call arguments"
+  end
+
   test "defstruct warns with duplicate keys" do
     assert capture_err(fn ->
              Code.eval_string("""

lib/elixir/test/elixir/kernel_test.exs

@@ -723,8 +723,12 @@ defmodule KernelTest do
       result = expand_to_string(quote(do: rand() in 1..2))
       assert result =~ "var = rand()"
 
-      assert result =~
-               ":erlang.andalso(:erlang.is_integer(var), :erlang.andalso(:erlang.>=(var, 1), :erlang.\"=<\"(var, 2)))"
+      assert result =~ """
+             :erlang.andalso(
+               :erlang.is_integer(var),
+               :erlang.andalso(:erlang.>=(var, 1), :erlang.\"=<\"(var, 2))
+             )\
+             """
 
       # Empty list
       assert expand_to_string(quote(do: :x in [])) =~ "_ = :x\nfalse"
@@ -754,14 +758,26 @@ defmodule KernelTest do
       assert expand_to_string(quote(do: foo in [])) ==
                "_ = foo\nfalse"
 
-      assert expand_to_string(quote(do: foo in [1, 2, 3])) ==
-               ":erlang.orelse(:erlang.orelse(:erlang.\"=:=\"(foo, 1), :erlang.\"=:=\"(foo, 2)), :erlang.\"=:=\"(foo, 3))"
+      assert expand_to_string(quote(do: foo in [1, 2, 3])) == """
+             :erlang.orelse(
+               :erlang.orelse(:erlang.\"=:=\"(foo, 1), :erlang.\"=:=\"(foo, 2)),
+               :erlang.\"=:=\"(foo, 3)
+             )\
+             """
 
-      assert expand_to_string(quote(do: foo in 0..1)) ==
-               ":erlang.andalso(:erlang.is_integer(foo), :erlang.andalso(:erlang.>=(foo, 0), :erlang.\"=<\"(foo, 1)))"
+      assert expand_to_string(quote(do: foo in 0..1)) == """
+             :erlang.andalso(
+               :erlang.is_integer(foo),
+               :erlang.andalso(:erlang.>=(foo, 0), :erlang.\"=<\"(foo, 1))
+             )\
+             """
 
-      assert expand_to_string(quote(do: foo in -1..0)) ==
-               ":erlang.andalso(:erlang.is_integer(foo), :erlang.andalso(:erlang.>=(foo, -1), :erlang.\"=<\"(foo, 0)))"
+      assert expand_to_string(quote(do: foo in -1..0)) == """
+             :erlang.andalso(
+               :erlang.is_integer(foo),
+               :erlang.andalso(:erlang.>=(foo, -1), :erlang.\"=<\"(foo, 0))
+             )\
+             """
 
       assert expand_to_string(quote(do: foo in 1..1)) ==
                ":erlang.\"=:=\"(foo, 1)"

lib/elixir/test/elixir/macro_test.exs

@@ -810,6 +810,14 @@ defmodule MacroTest do
       defmodule Foo.Bar do
         assert __MODULE__ in __ENV__.context_modules
       end
+
+      assert Foo.Bar in __ENV__.context_modules
+
+      Code.compile_string("""
+      defmodule Foo.Bar.Compiled do
+        true = __MODULE__ in __ENV__.context_modules
+      end
+      """)
     end
 
     test "to_match/1" do

lib/elixir/test/elixir/module/types/types_test.exs

@@ -17,6 +17,10 @@ defmodule Module.Types.TypesTest do
     end
   end
 
+  defmacro generated(ast) do
+    Macro.prewalk(ast, fn node -> Macro.update_meta(node, &([generated: true] ++ &1)) end)
+  end
+
   def __expr__({patterns, guards, body}) do
     with {:ok, _types, context} <-
            Pattern.of_head(patterns, guards, TypeHelper.new_stack(), TypeHelper.new_context()),
@@ -213,6 +217,32 @@ defmodule Module.Types.TypesTest do
              """
     end
 
+    test "warns on guards from cases unless generated" do
+      string =
+        warning(
+          [var],
+          [is_integer(var)],
+          case var do
+            _ when is_binary(var) -> :ok
+          end
+        )
+
+      assert is_binary(string)
+
+      string =
+        generated(
+          warning(
+            [var],
+            [is_integer(var)],
+            case var do
+              _ when is_binary(var) -> :ok
+            end
+          )
+        )
+
+      assert string == :none
+    end
+
     test "only show relevant traces in warning" do
       string = warning([x = y, z], [is_integer(x) and is_binary(y) and is_boolean(z)], {x, y, z})
 
@@ -606,6 +636,41 @@ defmodule Module.Types.TypesTest do
                  %{foo: ^other_key} = event
              """
     end
+
+    test "expands map when maps are nested" do
+      string =
+        warning(
+          [map1, map2],
+          (
+            [_var1, _var2] = [map1, map2]
+            %{} = map1
+            %{} = map2.subkey
+          )
+        )
+
+      assert string == """
+             incompatible types:
+
+                 %{subkey: var1, optional(dynamic()) => dynamic()} !~ %{optional(dynamic()) => dynamic()} | %{optional(dynamic()) => dynamic()}
+
+             in expression:
+
+                 # types_test.ex:5
+                 map2.subkey
+
+             where "map2" was given the type %{optional(dynamic()) => dynamic()} | %{optional(dynamic()) => dynamic()} in:
+
+                 # types_test.ex:3
+                 [_var1, _var2] = [map1, map2]
+
+             where "map2" was given the type %{subkey: var1, optional(dynamic()) => dynamic()} (due to calling var.field) in:
+
+                 # types_test.ex:5
+                 map2.subkey
+
+             HINT: "var.field" (without parentheses) implies "var" is a map() while "var.fun()" (with parentheses) implies "var" is an atom()
+             """
+    end
   end
 
   describe "regressions" do

lib/elixir/test/elixir/string_test.exs

@@ -5,15 +5,6 @@ defmodule StringTest do
 
   doctest String
 
-  test "heredoc with heredoc inside interpolation" do
-    assert """
-           1
-           #{"""
-           2
-           """}
-           """ == "1\n2\n\n"
-  end
-
   test "next_codepoint/1" do
     assert String.next_codepoint("ésoj") == {"é", "soj"}
     assert String.next_codepoint(<<255>>) == {<<255>>, ""}
@@ -661,6 +652,7 @@ defmodule StringTest do
     assert String.slice("あいうえお", -10..-15) == ""
     assert String.slice("hello あいうえお Unicode", 8..-1) == "うえお Unicode"
     assert String.slice("abc", -1..14) == "c"
+    assert String.slice("a·̀ͯ‿.⁀:", 0..-2) == "a·̀ͯ‿.⁀"
 
     assert_raise FunctionClauseError, fn ->
       String.slice(nil, 0..1)

lib/elixir/test/elixir/task/supervisor_test.exs

@@ -450,6 +450,15 @@ defmodule Task.SupervisorTest do
 
       assert_receive :done
     end
+
+    test "consuming from another process", config do
+      parent = self()
+      stream = Task.Supervisor.async_stream(config[:supervisor], [1, 2, 3], &send(parent, &1))
+      Task.start(Stream, :run, [stream])
+      assert_receive 1
+      assert_receive 2
+      assert_receive 3
+    end
   end
 
   describe "async_stream_nolink" do

lib/elixir/test/elixir/task_test.exs

@@ -829,6 +829,15 @@ defmodule TaskTest do
 
       assert_receive :done
     end
+
+    test "consuming from another process" do
+      parent = self()
+      stream = Task.async_stream([1, 2, 3], &send(parent, &1))
+      Task.start(Stream, :run, [stream])
+      assert_receive 1
+      assert_receive 2
+      assert_receive 3
+    end
   end
 
   for {desc, concurrency} <- [==: 4, <: 2, >: 8] do

lib/elixir/test/elixir/test_helper.exs

@@ -39,6 +39,14 @@ defmodule PathHelpers do
     executable_path("elixirc")
   end
 
+  def iex(args) do
+    run_cmd(iex_executable(), args)
+  end
+
+  def iex_executable do
+    executable_path("iex")
+  end
+
   def write_beam({:module, name, bin, _} = res) do
     File.mkdir_p!(unquote(path))
     beam_path = Path.join(unquote(path), Atom.to_string(name) <> ".beam")

lib/elixir/test/elixir/uri_test.exs

@@ -94,7 +94,7 @@ defmodule URITest do
 
   describe "new/1" do
     test "empty" do
-      assert URI.new("") == {:ok, %URI{path: ""}}
+      assert URI.new("") == {:ok, %URI{}}
     end
 
     test "errors on bad URIs" do
@@ -131,7 +131,7 @@ defmodule URITest do
         query: nil,
         fragment: nil,
         port: 443,
-        path: "",
+        path: nil,
         userinfo: nil
       }
 
@@ -349,9 +349,9 @@ defmodule URITest do
     assert URI.merge(base, "") |> to_string == "http://example.com/foo/bar"
     assert URI.merge(base, "#fragment") |> to_string == "http://example.com/foo/bar#fragment"
     assert URI.merge(base, "?query") |> to_string == "http://example.com/foo/bar?query"
-    assert URI.merge(base, %URI{path: ""}) |> to_string == "http://example.com/foo/bar"
+    assert URI.merge(base, %URI{}) |> to_string == "http://example.com/foo/bar"
 
-    assert URI.merge(base, %URI{path: "", fragment: "fragment"})
+    assert URI.merge(base, %URI{fragment: "fragment"})
            |> to_string == "http://example.com/foo/bar#fragment"
 
     base = URI.new!("http://example.com")
@@ -449,7 +449,7 @@ defmodule URITest do
         query: nil,
         fragment: nil,
         port: 443,
-        path: "",
+        path: nil,
         userinfo: nil
       }
 
@@ -551,7 +551,7 @@ defmodule URITest do
       expected_uri = %URI{
         scheme: "http",
         host: "foo.com",
-        path: "",
+        path: nil,
         query: nil,
         fragment: nil,
         port: 4444,
@@ -564,7 +564,7 @@ defmodule URITest do
       expected_uri = %URI{
         scheme: "https",
         host: "foo.com",
-        path: "",
+        path: nil,
         query: nil,
         fragment: nil,
         port: 443,
@@ -577,7 +577,7 @@ defmodule URITest do
       expected_uri = %URI{
         scheme: "http",
         host: "foo.com",
-        path: "",
+        path: nil,
         query: nil,
         fragment: nil,
         port: 4444,
@@ -655,5 +655,11 @@ defmodule URITest do
     test "preserves an empty query" do
       assert URI.parse("http://foo.com/?").query == ""
     end
+
+    test "merges empty path" do
+      base = URI.parse("http://example.com")
+      assert URI.merge(base, "/foo") |> to_string == "http://example.com/foo"
+      assert URI.merge(base, "foo") |> to_string == "http://example.com/foo"
+    end
   end
 end

lib/elixir/test/elixir/version_test.exs

@@ -308,6 +308,15 @@ defmodule VersionTest do
     assert Version.match?("0.7.0", req)
   end
 
+  test "compile_requirement/1" do
+    {:ok, req} = Version.parse_requirement("1.2.3")
+    assert req == Version.compile_requirement(req)
+
+    assert_raise(FunctionClauseError, fn ->
+      Version.compile_requirement("~> 1.2.3")
+    end)
+  end
+
   test "compile requirement" do
     {:ok, req} = Version.parse_requirement("1.2.3")
     req = Version.compile_requirement(req)

lib/iex/lib/iex/server.ex

@@ -239,12 +239,13 @@ defmodule IEx.Server do
   defp handle_take_over(
          {:EXIT, pid, reason},
          state,
-         _evaluator,
-         _evaluator_ref,
+         evaluator,
+         evaluator_ref,
          _input,
          callback
        ) do
     if pid == Process.group_leader() do
+      stop_evaluator(evaluator, evaluator_ref)
       exit(reason)
     else
       callback.(state)

lib/mix/lib/mix.ex

@@ -663,9 +663,10 @@ defmodule Mix do
 
     config = Keyword.get(opts, :config, [])
     system_env = Keyword.get(opts, :system_env, [])
+    consolidate_protocols? = Keyword.get(opts, :consolidate_protocols, true)
 
     id =
-      {deps, config, system_env}
+      {deps, config, system_env, consolidate_protocols?}
       |> :erlang.term_to_binary()
       |> :erlang.md5()
       |> Base.encode16(case: :lower)
@@ -674,7 +675,70 @@ defmodule Mix do
 
     case Mix.State.get(:installed) do
       nil ->
-        :ok
+        Application.put_all_env(config, persistent: true)
+        System.put_env(system_env)
+
+        installs_root =
+          System.get_env("MIX_INSTALL_DIR") || Path.join(Mix.Utils.mix_cache(), "installs")
+
+        version = "elixir-#{System.version()}-erts-#{:erlang.system_info(:version)}"
+        dir = Path.join([installs_root, version, id])
+
+        if opts[:verbose] do
+          Mix.shell().info("Mix.install/2 using #{dir}")
+        end
+
+        if force? do
+          File.rm_rf!(dir)
+        end
+
+        config = [
+          version: "0.1.0",
+          build_embedded: false,
+          build_per_environment: true,
+          build_path: "_build",
+          lockfile: "mix.lock",
+          deps_path: "deps",
+          deps: deps,
+          app: :mix_install,
+          erlc_paths: ["src"],
+          elixirc_paths: ["lib"],
+          compilers: [],
+          consolidate_protocols: consolidate_protocols?
+        ]
+
+        started_apps = Application.started_applications()
+        :ok = Mix.Local.append_archives()
+        :ok = Mix.ProjectStack.push(@mix_install_project, config, "nofile")
+        build_dir = Path.join(dir, "_build")
+
+        try do
+          run_deps? = not File.dir?(build_dir)
+          File.mkdir_p!(dir)
+
+          File.cd!(dir, fn ->
+            if run_deps? do
+              Mix.Task.rerun("deps.get")
+            end
+
+            Mix.Task.rerun("deps.loadpaths")
+
+            # Hex and SSL can use a good amount of memory after the registry fetching,
+            # so we stop any app started during deps resolution.
+            stop_apps(Application.started_applications() -- started_apps)
+
+            Mix.Task.rerun("compile")
+          end)
+
+          for app <- Mix.Project.deps_apps() do
+            Application.ensure_all_started(app)
+          end
+
+          Mix.State.put(:installed, id)
+          :ok
+        after
+          Mix.ProjectStack.pop()
+        end
 
       ^id when not force? ->
         :ok
@@ -682,71 +746,14 @@ defmodule Mix do
       _ ->
         Mix.raise("Mix.install/2 can only be called with the same dependencies in the given VM")
     end
+  end
 
-    Application.put_all_env(config, persistent: true)
-    System.put_env(system_env)
-
-    installs_root =
-      System.get_env("MIX_INSTALL_DIR") || Path.join(Mix.Utils.mix_cache(), "installs")
-
-    version = "elixir-#{System.version()}-erts-#{:erlang.system_info(:version)}"
-    dir = Path.join([installs_root, version, id])
-
-    if opts[:verbose] do
-      Mix.shell().info("Mix.install/2 using #{dir}")
-    end
-
-    if force? do
-      File.rm_rf!(dir)
-    end
-
-    config = [
-      version: "0.1.0",
-      build_embedded: false,
-      build_per_environment: true,
-      build_path: "_build",
-      lockfile: "mix.lock",
-      deps_path: "deps",
-      deps: deps,
-      app: :mix_install,
-      erlc_paths: ["src"],
-      elixirc_paths: ["lib"],
-      compilers: [],
-      consolidate_protocols: Keyword.get(opts, :consolidate_protocols, true)
-    ]
-
-    started_apps = Application.started_applications()
-    :ok = Mix.Local.append_archives()
-    :ok = Mix.ProjectStack.push(@mix_install_project, config, "nofile")
-    build_dir = Path.join(dir, "_build")
-
-    try do
-      run_deps? = not File.dir?(build_dir)
-      File.mkdir_p!(dir)
-
-      File.cd!(dir, fn ->
-        if run_deps? do
-          Mix.Task.rerun("deps.get")
-        end
-
-        Mix.Task.rerun("deps.loadpaths")
-
-        # Hex and SSL can use a good amount of memory after the registry fetching,
-        # so we stop any app started during deps resolution.
-        stop_apps(Application.started_applications() -- started_apps)
-
-        Mix.Task.rerun("compile")
-      end)
-
-      for app <- Mix.Project.deps_apps() do
-        Application.ensure_all_started(app)
-      end
-
-      Mix.State.put(:installed, id)
-      :ok
-    after
-      Mix.ProjectStack.pop()
-    end
+  @doc """
+  Returns whether `Mix.install/2` was called in the current node.
+  """
+  @doc since: "1.13.0"
+  def installed? do
+    Mix.State.get(:installed) != nil
   end
 
   defp stop_apps([]), do: :ok

lib/mix/lib/mix/compilers/elixir.ex

@@ -725,37 +725,78 @@ defmodule Mix.Compilers.Elixir do
 
     for %{scm: scm, opts: opts} = dep <- Mix.Dep.cached(),
         not scm.fetchable?,
-        Mix.Utils.last_modified(Path.join([opts[:build], ".mix", base])) > modified,
-        path <- Mix.Dep.load_paths(dep),
-        beam <- Path.wildcard(Path.join(path, "*.beam")),
-        Mix.Utils.last_modified(beam) > modified,
+        manifest = Path.join([opts[:build], ".mix", base]),
+        Mix.Utils.last_modified(manifest) > modified,
         reduce: {stale_modules, %{}, old_exports} do
       {modules, exports, new_exports} ->
-        module = beam |> Path.basename() |> Path.rootname() |> String.to_atom()
-        export = exports_md5(module, false)
-        modules = Map.put(modules, module, [])
+        {_manifest_modules, dep_sources} = read_manifest(manifest)
 
-        # If the exports are the same, then the API did not change,
-        # so we do not mark the export as stale. Note this has to
-        # be very conservative. If the module is not loaded or if
-        # the exports were not there, we need to consider it a stale
-        # export.
-        exports =
-          if export && old_exports[module] == export,
-            do: exports,
-            else: Map.put(exports, module, [])
+        # TODO: Use :maps.from_keys/2 on Erlang/OTP 24+
+        dep_modules =
+          for path <- Mix.Dep.load_paths(dep),
+              beam <- Path.wildcard(Path.join(path, "*.beam")),
+              Mix.Utils.last_modified(beam) > modified,
+              do: {beam |> Path.basename() |> Path.rootname() |> String.to_atom(), []},
+              into: %{}
 
-        # In any case, we always store it as the most update export
-        # that we have, otherwise we delete it.
-        new_exports =
-          if export,
-            do: Map.put(new_exports, module, export),
-            else: Map.delete(new_exports, module)
+        # If any module has a compile time dependency on a changed module
+        # within the dependnecy, they will be recompiled. However, export
+        # and runtime dependencies won't have recompiled so we need to
+        # propagate them to the parent app.
+        dep_modules = fixpoint_dep_modules(dep_sources, dep_modules, false, [])
 
-        {modules, exports, new_exports}
+        # Update exports
+        {exports, new_exports} =
+          for {module, _} <- dep_modules, reduce: {exports, new_exports} do
+            {exports, new_exports} ->
+              export = exports_md5(module, false)
+
+              # If the exports are the same, then the API did not change,
+              # so we do not mark the export as stale. Note this has to
+              # be very conservative. If the module is not loaded or if
+              # the exports were not there, we need to consider it a stale
+              # export.
+              exports =
+                if export && old_exports[module] == export,
+                  do: exports,
+                  else: Map.put(exports, module, [])
+
+              # In any case, we always store it as the most update export
+              # that we have, otherwise we delete it.
+              new_exports =
+                if export,
+                  do: Map.put(new_exports, module, export),
+                  else: Map.delete(new_exports, module)
+
+              {exports, new_exports}
+          end
+
+        {Map.merge(modules, dep_modules), exports, new_exports}
     end
   end
 
+  defp fixpoint_dep_modules([source | sources], modules, new_modules?, acc_sources) do
+    source(export_references: export_refs, runtime_references: runtime_refs) = source
+
+    if has_any_key?(modules, export_refs) or has_any_key?(modules, runtime_refs) do
+      new_modules = Enum.reject(source(source, :modules), &Map.has_key?(modules, &1))
+      new_modules? = new_modules? or new_modules != []
+      modules = Enum.reduce(new_modules, modules, &Map.put(&2, &1, []))
+      fixpoint_dep_modules(sources, modules, new_modules?, acc_sources)
+    else
+      fixpoint_dep_modules(sources, modules, new_modules?, [source | acc_sources])
+    end
+  end
+
+  defp fixpoint_dep_modules([], modules, false, _),
+    do: modules
+
+  defp fixpoint_dep_modules([], modules, true, []),
+    do: modules
+
+  defp fixpoint_dep_modules([], modules, true, sources),
+    do: fixpoint_dep_modules(sources, modules, false, [])
+
   defp exports_md5(module, use_attributes?) do
     cond do
       function_exported?(module, :__info__, 1) ->

lib/mix/lib/mix/compilers/test.ex

@@ -25,52 +25,57 @@ defmodule Mix.Compilers.Test do
   def require_and_run(matched_test_files, test_paths, opts) do
     stale = opts[:stale]
 
-    {test_files, stale_manifest_pid, parallel_require_opts} =
+    {test_files, stale_manifest_pid, parallel_require_callbacks} =
       if stale do
         set_up_stale(matched_test_files, test_paths, opts)
       else
         {matched_test_files, nil, []}
       end
 
-    if test_files == [] do
-      :noop
-    else
-      task = ExUnit.async_run()
-      parallel_require_opts = profile_opts(parallel_require_opts, opts)
-      warnings_as_errors? = Keyword.get(opts, :warnings_as_errors, false)
+    cond do
+      test_files == [] ->
+        :noop
 
-      try do
-        failed? =
-          case Kernel.ParallelCompiler.require(test_files, parallel_require_opts) do
-            {:ok, _, [_ | _]} when warnings_as_errors? -> true
-            {:ok, _, _} -> false
-            {:error, _, _} -> exit({:shutdown, 1})
+      Keyword.get(opts, :profile_require) == "time" ->
+        Kernel.ParallelCompiler.require(test_files, profile: :time)
+        :noop
+
+      true ->
+        task = ExUnit.async_run()
+        warnings_as_errors? = Keyword.get(opts, :warnings_as_errors, false)
+
+        try do
+          failed? =
+            case Kernel.ParallelCompiler.require(test_files, parallel_require_callbacks) do
+              {:ok, _, [_ | _]} when warnings_as_errors? -> true
+              {:ok, _, _} -> false
+              {:error, _, _} -> exit({:shutdown, 1})
+            end
+
+          %{failures: failures} = results = ExUnit.await_run(task)
+
+          if failures == 0 do
+            if failed? do
+              message =
+                "\nERROR! Test suite aborted after successful execution due to warnings while using the --warnings-as-errors option"
+
+              IO.puts(:stderr, IO.ANSI.format([:red, message]))
+              exit({:shutdown, 1})
+            end
+
+            agent_write_manifest(stale_manifest_pid)
           end
 
-        %{failures: failures} = results = ExUnit.await_run(task)
-
-        if failures == 0 do
-          if failed? do
-            message =
-              "\nERROR! Test suite aborted after successful execution due to warnings while using the --warnings-as-errors option"
-
-            IO.puts(:stderr, IO.ANSI.format([:red, message]))
-            exit({:shutdown, 1})
-          end
-
-          agent_write_manifest(stale_manifest_pid)
+          {:ok, results}
+        catch
+          kind, reason ->
+            # In case there is an error, shut down the runner task
+            # before the error propagates up and trigger links.
+            Task.shutdown(task)
+            :erlang.raise(kind, reason, __STACKTRACE__)
+        after
+          agent_stop(stale_manifest_pid)
         end
-
-        {:ok, results}
-      catch
-        kind, reason ->
-          # In case there is an error, shut down the runner task
-          # before the error propagates up and trigger links.
-          Task.shutdown(task)
-          :erlang.raise(kind, reason, __STACKTRACE__)
-      after
-        agent_stop(stale_manifest_pid)
-      end
     end
   end
 
@@ -167,14 +172,6 @@ defmodule Mix.Compilers.Test do
     sources
   end
 
-  defp profile_opts(target, opts) do
-    if Keyword.get(opts, :profile_require) == "time" do
-      Keyword.put(target, :profile, :time)
-    else
-      target
-    end
-  end
-
   ## Manifest
 
   defp manifest, do: Path.join(Mix.Project.manifest_path(), @stale_manifest)

lib/mix/lib/mix/release.ex

@@ -457,13 +457,12 @@ defmodule Mix.Release do
     end
   end
 
-  defp valid_config?(n) when is_number(n), do: true
-  defp valid_config?(a) when is_atom(a), do: true
-  defp valid_config?(b) when is_binary(b), do: true
+  defp valid_config?(m) when is_map(m),
+    do: Enum.all?(Map.delete(m, :__struct__), &valid_config?/1)
+
   defp valid_config?(l) when is_list(l), do: Enum.all?(l, &valid_config?/1)
-  defp valid_config?(m) when is_map(m), do: Enum.all?(m, &valid_config?/1)
   defp valid_config?(t) when is_tuple(t), do: Enum.all?(Tuple.to_list(t), &valid_config?/1)
-  defp valid_config?(_), do: false
+  defp valid_config?(o), do: is_number(o) or is_atom(o) or is_binary(o)
 
   defp merge_provider_config(%{config_providers: []}, sys_config, _), do: {sys_config, false}
 
@@ -610,13 +609,24 @@ defmodule Mix.Release do
   end
 
   defp build_release_spec(release, modes) do
-    %{name: name, version: version, erts_version: erts_version, applications: apps} = release
+    %{
+      name: name,
+      version: version,
+      erts_version: erts_version,
+      applications: apps,
+      options: options
+    } = release
+
+    skip_mode_validation_for =
+      options
+      |> Keyword.get(:skip_mode_validation_for, [])
+      |> MapSet.new()
 
     rel_apps =
       for {app, mode} <- modes do
         properties = Map.get(apps, app) || throw({:error, "Unknown application #{inspect(app)}"})
         children = Keyword.get(properties, :applications, [])
-        validate_mode!(app, mode, modes, children)
+        app in skip_mode_validation_for || validate_mode!(app, mode, modes, children)
         build_app_for_release(app, mode, properties)
       end
 

lib/mix/lib/mix/shell.ex

@@ -18,6 +18,8 @@ defmodule Mix.Shell do
 
   @doc """
   Executes the given command and returns its exit status.
+
+  Shortcut for `cmd/2` with empty options.
   """
   @callback cmd(command :: String.t()) :: integer
 
@@ -46,6 +48,17 @@ defmodule Mix.Shell do
 
   @doc """
   Prompts the user for confirmation.
+
+  Shortcut for `yes?/2` with empty options.
+  """
+  @callback yes?(message :: binary) :: boolean
+
+  @doc """
+  Prompts the user for confirmation.
+
+  ## Options
+
+    * `:default` - `:yes` or `:no` (the default is `:yes`)
   """
   @callback yes?(message :: binary, options :: keyword) :: boolean
 

lib/mix/lib/mix/tasks/format.ex

@@ -213,6 +213,7 @@ defmodule Mix.Tasks.Format do
   to be formatted. The options are returned for reflection
   purposes.
   """
+  @doc since: "1.13.0"
   def formatter_for_file(file, opts \\ []) do
     {dot_formatter, formatter_opts} = eval_dot_formatter(opts)
 
@@ -494,12 +495,12 @@ defmodule Mix.Tasks.Format do
     ext = Path.extname(file)
 
     cond do
-      ext in ~w(.ex .exs) ->
-        &elixir_format(&1, [file: file] ++ formatter_opts)
-
       plugin = find_plugin_for_extension(formatter_opts, ext) ->
         &plugin.format(&1, [extension: ext] ++ formatter_opts)
 
+      ext in ~w(.ex .exs) ->
+        &elixir_format(&1, [file: file] ++ formatter_opts)
+
       true ->
         & &1
     end

lib/mix/lib/mix/tasks/release.ex

@@ -455,6 +455,14 @@ defmodule Mix.Tasks.Release do
     * `:steps` - a list of steps to execute when assembling the release. See
       the "Steps" section for more information.
 
+    * `:skip_mode_validation_for` - a list of application names
+      (atoms) specifying applications to skip strict validation of
+      "unsafe" modes. An "unsafe" case is when a parent application
+      mode is `:permanent` but one of the applications it depends on
+      is set to `:load`. Use this with care, as a release with
+      invalid modes may no longer boot without additional tweaks.
+      Defaults to `[]`.
+
   Note each release definition can be given as an anonymous function. This
   is useful if some release attributes are expensive to compute:
 

lib/mix/lib/mix/tasks/test.coverage.ex

@@ -299,6 +299,7 @@ defmodule Mix.Tasks.Test.Coverage do
     print_summary(module_results, totals, summary_opts)
 
     if totals < get_threshold(summary_opts) do
+      print_failed_threshold(totals, get_threshold(summary_opts))
       System.at_exit(fn _ -> exit({:shutdown, 3}) end)
     end
 
@@ -353,6 +354,13 @@ defmodule Mix.Tasks.Test.Coverage do
     Mix.shell().info("")
   end
 
+  defp print_failed_threshold(totals, threshold) do
+    Mix.shell().info("Coverage test failed, threshold not met:\n")
+    Mix.shell().info("    Coverage:  #{format_number(totals, 6)}%")
+    Mix.shell().info("    Threshold: #{format_number(threshold, 6)}%")
+    Mix.shell().info("")
+  end
+
   defp display({percentage, name}, threshold) do
     Mix.shell().info([
       color(percentage, threshold),
@@ -369,6 +377,9 @@ defmodule Mix.Tasks.Test.Coverage do
   defp color(percentage, threshold) when percentage >= threshold, do: :green
   defp color(_, _), do: :red
 
+  defp format_number(number, length) when is_integer(number),
+    do: format_number(number / 1, length)
+
   defp format_number(number, length), do: :io_lib.format("~#{length}.2f", [number])
 
   defp format_name(name) when is_binary(name), do: name

lib/mix/lib/mix/tasks/test.ex

@@ -160,7 +160,8 @@ defmodule Mix.Tasks.Test do
 
     * `--preload-modules` - preloads all modules defined in applications
 
-    * `--profile-require` - profiles the time spent to require test files
+    * `--profile-require` - profiles the time spent to require test files.
+      Used only for debugging. The test suite does not run.
 
     * `--raise` - raises if the test suite failed
 

lib/mix/test/mix/release_test.exs

@@ -479,6 +479,13 @@ defmodule Mix.ReleaseTest do
       assert message =~
                "Application :mix has mode :permanent but it depends on :elixir which is set to :none"
     end
+
+    test "does not raise on child unsafe mode if parent is in `skip_mode_validation_for`" do
+      # `:mix` is a parent that depends on `:elixir`
+      release = release(applications: [elixir: :load], skip_mode_validation_for: [:mix])
+      assert make_boot_script(release, @boot_script_path, release.boot_scripts.start) == :ok
+      assert make_boot_script(release, @boot_script_path, release.boot_scripts.start_clean) == :ok
+    end
   end
 
   describe "make_cookie/1" do
@@ -585,6 +592,11 @@ defmodule Mix.ReleaseTest do
     test "errors on bad config" do
       assert {:error, "Could not read configuration file." <> _} =
                make_sys_config(release([]), [foo: [bar: self()]], "unused/runtime/path")
+
+      env = %{__ENV__ | lexical_tracker: self()}
+
+      assert {:error, "Could not read configuration file." <> _} =
+               make_sys_config(release([]), [foo: [bar: env]], "unused/runtime/path")
     end
   end
 

lib/mix/test/mix/tasks/archive_test.exs

@@ -135,6 +135,7 @@ defmodule Mix.Tasks.ArchiveTest do
     end)
   end
 
+  @compile {:no_warn_undefined, GitRepo.Archive}
   test "archive.install from Git" do
     in_fixture("git_repo", fn ->
       File.mkdir_p!("config")
@@ -148,15 +149,20 @@ defmodule Mix.Tasks.ArchiveTest do
       require Application
       true = Application.compile_env!(:git_repo, :archive_config)
 
-      defmodule GitRepo do
+      defmodule GitRepo.Archive do
         def hello do
           "World"
         end
       end
       """)
 
+      System.cmd("git", ~w[init])
+      System.cmd("git", ~w[add .])
+      System.cmd("git", ~w[commit -m first-commit])
+
       send(self(), {:mix_shell_input, :yes?, true})
       Mix.Tasks.Archive.Install.run(["git", File.cwd!()])
+      assert GitRepo.Archive.hello() == "World"
 
       message = "Generated archive \"git_repo-0.1.0.ez\" with MIX_ENV=prod"
       assert_received {:mix_shell, :info, [^message]}
@@ -165,7 +171,7 @@ defmodule Mix.Tasks.ArchiveTest do
       assert File.dir?(tmp_path("userhome/.mix/archives/git_repo-0.1.0/git_repo-0.1.0/ebin"))
     end)
   after
-    purge([GitRepo, GitRepo.MixProject])
+    purge([GitRepo.Archive, GitRepo.MixProject])
   end
 
   test "archive install, update, and uninstall life-cycle" do

lib/mix/test/mix/tasks/test_test.exs

@@ -168,7 +168,14 @@ defmodule Mix.Tasks.TestTest do
 
         # We skip a test in bar to force coverage below the default threshold
         # which should result in an exit status of 1.
-        assert {_, code} = mix_code(["test", "--cover", "--exclude", "maybe_skip"])
+        assert {output, code} = mix_code(["test", "--cover", "--exclude", "maybe_skip"])
+
+        assert output =~ """
+               Coverage test failed, threshold not met:
+
+                   Coverage:    0.00%
+                   Threshold:  90.00%
+               """
 
         unless windows?() do
           assert code == 3

lib/mix/test/mix/umbrella_test.exs

@@ -439,12 +439,9 @@ defmodule Mix.UmbrellaTest do
       Mix.Project.in_project(:bar, "bar", fn _ ->
         File.write!("../foo/lib/foo.ex", "defmodule Foo, do: defstruct [:bar]")
 
-        Mix.Task.run("compile", ["--verbose"])
-
         # Add struct dependency
         File.write!("lib/bar.ex", "defmodule Bar, do: %Foo{bar: true}")
-
-        assert Mix.Tasks.Compile.Elixir.run(["--verbose"]) == {:ok, []}
+        Mix.Task.run("compile", ["--verbose"])
         assert_receive {:mix_shell, :info, ["Compiled lib/bar.ex"]}
 
         # Recompiles for struct dependencies
@@ -458,6 +455,37 @@ defmodule Mix.UmbrellaTest do
     end)
   end
 
+  test "recompiles after compile through runtime path dependency changes" do
+    in_fixture("umbrella_dep/deps/umbrella/apps", fn ->
+      Mix.Project.in_project(:bar, "bar", fn _ ->
+        File.write!("../foo/lib/foo.bar.ex", """
+        defmodule Foo.Bar do
+          def hello, do: Foo.Baz.hello()
+        end
+        """)
+
+        File.write!("../foo/lib/foo.baz.ex", """
+        defmodule Foo.Baz do
+          def hello, do: "from bar"
+        end
+        """)
+
+        # Add compile time to Foo.Bar
+        File.write!("lib/bar.ex", "defmodule Bar, do: Foo.Bar.hello()")
+        Mix.Task.run("compile", ["--verbose"])
+        assert_receive {:mix_shell, :info, ["Compiled lib/bar.ex"]}
+
+        # Recompiles for due to compile dependency via runtime dependencies
+        mtime = File.stat!("_build/dev/lib/bar/.mix/compile.elixir").mtime
+        ensure_touched("_build/dev/lib/foo/ebin/Elixir.Foo.Baz.beam", mtime)
+        ensure_touched("_build/dev/lib/foo/.mix/compile.elixir", mtime)
+
+        assert Mix.Tasks.Compile.Elixir.run(["--verbose"]) == {:ok, []}
+        assert_receive {:mix_shell, :info, ["Compiled lib/bar.ex"]}
+      end)
+    end)
+  end
+
   test "reloads app in app tracer if .app changes" do
     in_fixture("umbrella_dep/deps/umbrella/apps", fn ->
       deps = [{:foo, in_umbrella: true}]

lib/mix/test/mix_test.exs

@@ -137,6 +137,16 @@ defmodule MixTest do
       Application.delete_env(:unknown_app, :foo, persistent: true)
     end
 
+    test "install?", %{tmp_dir: tmp_dir} do
+      refute Mix.installed?()
+
+      Mix.install([
+        {:install_test, path: Path.join(tmp_dir, "install_test")}
+      ])
+
+      assert Mix.installed?()
+    end
+
     defp test_project(%{tmp_dir: tmp_dir}) do
       path = :code.get_path()