Fix typo in changelog (#8114)

beind => being

.appveyor.yml

@@ -11,8 +11,8 @@ test_script:
 environment:
   ELIXIR_ASSERT_TIMEOUT: 2000
 
-configuration: Test
-
 matrix:
   allow_failures:
-    - configuration: Test
+    - platform: x86
+    - platform: x64
+    - platform: Any CPU

CHANGELOG.md

@@ -16,7 +16,7 @@ The Elixir codebase itself has been already fully formatted and all further cont
 
 Supervisors in Elixir are responsible for starting, shutting down and restarting child process when things go wrong. Most of the interaction with supervisors happen with the Supervisor module and it contains three main strategies: `:one_for_one`, `:rest_for_one` and `:one_for_all`.
 
-However, sometimes the children of a supervisor are not known upfront and are rather started dynamically. For example, if you are building a web server, you have each request beind handled by a separate supervised process. Those cases were handled in the Supervisor module under a special strategy called `:simple_one_for_one`.
+However, sometimes the children of a supervisor are not known upfront and are rather started dynamically. For example, if you are building a web server, you have each request being handled by a separate supervised process. Those cases were handled in the Supervisor module under a special strategy called `:simple_one_for_one`.
 
 Unfortunately, this special strategy changed the semantics of the supervisor in regards to initialization and shutdown. Plus some APIs expected different inputs or would be completely unavailable depending on the supervision strategy.
 
@@ -123,10 +123,107 @@ Those improvements will help developers better understand the relationship betwe
 
 Other improvements in Mix include better compiler diagnostics for editor integration, support for the `--slowest N` flag in `mix test` that shows the slowest tests in your suite, and a new `mix profile.eprof` task that provides time based profiling, complementing the existing `mix profile.cprof` (count based) and `mix profile.fprof` (flame based).
 
+## v1.6.6 (2018-06-20)
+
+This release supports Erlang/OTP 21.0 by removing all warnings and by properly supporting the new Erlang logger module.
+
+### 1. Bug fixes
+
+#### Elixir
+
+  * [Base] Do not raise when finding bad digits in `Base.decode32!` with `case: :mixed`
+  * [Code] Preserve the user's choice when `fn` is followed by a newline and it has only a single clause
+  * [DynamicSupervisor] Properly account for restarting children in the `:max_children` configuration
+  * [String] Add performant impl for string upcase/downcase `:ascii` mode
+  * [Task.Supervisor] Fix type spec for `start_child/4`
+
+#### Logger
+
+  * [Logger] Do not crash truncation when truncate is set to infinity
+
+#### Mix
+
+  * [mix format] Match files starting with dot
+
+## v1.6.5 (2018-05-07)
+
+This release supports Erlang/OTP 21.0-rc by removing all warnings and by properly redirecting logger output. Note it is not guaranteed it will support Erlang/OTP 21.0 final.
+
+### 1. Bug fixes
+
+#### Elixir
+
+  * [Code] Preserve the user's choice in the formatter on parens call with next break fits
+  * [Code] Preserve the user's choice in the formatter on calls without parens when we have one argument per line
+  * [Code] Fix formatting when there is a tilde in the first element of a bitstring
+  * [Kernel] Support specsdiff flag on `__info__` spec clauses
+  * [Kernel] Do not exclude hygienic vars in `defguard`
+  * [Kernel.SpecialForms] Mark `for` comprehensions as generated to avoid dialyzer warnings
+  * [Macro] Make sure `Macro.to_string/2` emits valid quoted expressions
+  * [Task] Support `:infinity` timeout on `Task.yield_many/2`
+  * [Task.Supervisor] Do not crash spawning supervised tasks when the parent process is dead
+  * [URI] Fix parsing of URIs with trailing `?`
+
+## v1.6.4 (2018-03-16)
+
+### 1. Bug fixes
+
+#### Elixir
+
+  * [Code.Formatter] Do not double escape quoted keyword list identifiers
+  * [Kernel] Properly support `into: binary` in Erlang/OTP 20.3
+
+## v1.6.3 (2018-03-09)
+
+### 1. Enhancements
+
+#### Elixir
+
+  * [Code.Formatter] Support comments in the middle of pipelines, `when` and `|` expressions
+
+### 2. Bug fixes
+
+#### Elixir
+
+  * [Code.Formatter] Consider commas when breaking groups
+  * [Code.Formatter] Ensure proper precedence between `&` and operators
+  * [Code.Formatter] Consider `.formatter.exs` when formatting stdin
+
+#### Logger
+
+  * [Logger.Translator] Ensure logger doesn't crash when reporting named `DynamicSupervisor`
+
+## v1.6.2 (2018-02-28)
+
+### 1. Enhancements
+
+#### Mix
+
+  * [mix compile.erlang] Teach Mix erlang compiler alternative spelling for `-behavior` declaration
+  * [mix format] Support the `:subdirectories` configuration that points to other directories with their own `.formatter.exs` file. This is useful in umbrella applications. `mix new --umbrella` has also been changed to use this new configuration by default
+  * [mix format] Include the current environment for missing dependency errors
+
+### 2. Bug fixes
+
+#### Elixir
+
+  * [Code.Formatter] Ensure `->` does not exceed line length
+  * [DynamicSupervisor] Properly tag error reports generated by dynamic supervisors so they can be properly translated by `Logger`
+  * [DynamicSupervisor] Consider extra arguments during child restart
+  * [Kernel] Ensure arguments given to a guard defined with `defguard` are evaluated in the correct order
+  * [Module] Do not remove docs for previous function declaration when `@impl true` is used
+  * [Supervisor] Ensure `use Supervisor` properly adds the `@behaviour Supervisor` annotation
+
+#### Mix
+
+  * [Mix.Shell] Bring back `Mix.Shell.cmd/2` - this arity was defined via a default argument that was accidentally removed
+
 ## v1.6.1 (2018-01-29)
 
 ### 1. Enhancements
 
+#### Elixir
+
   * [DynamicSupervisor] Implement `child_spec/1` for DynamicSupervisor
   * [Kernel] Raise better error messages on invalid map syntax
 
@@ -135,13 +232,13 @@ Other improvements in Mix include better compiler diagnostics for editor integra
 #### Elixir
 
   * [Code.Formatter] Only rearrange `not in` operator if explicitly opted-in
-  * [Code.Formatter] Ensure do blocks do not exceed line length on calls with a single argument
+  * [Code.Formatter] Ensure `do` blocks do not exceed line length on calls with a single argument
   * [Collectable] Support bitstrings in Collectable and for-comprehensions (regression in v1.6.0)
   * [GenServer] Do not override user own `@opts` attribute
   * [Enum] Reintroduce zipping of any enumerable of enumerables in `Enum.zip/1` (regression in v1.6.0)
   * [Macro] Reorder kw blocks in `Macro.to_string/1` to avoid warnings
   * [Protocol] Fix protocol consolidation when some chunks may be missing
-  * [Stream] Reintroduce zipping of any enumerable of enumerables in `Enum.zip/1` (regression in v1.6.0)
+  * [Stream] Reintroduce zipping of any enumerable of enumerables in `Stream.zip/1` (regression in v1.6.0)
   * [Supervisor] Do not override user own `@opts` attribute
   * [Supervisor] Add `@spec` to second clause of `start_link/2`
 
@@ -235,6 +332,7 @@ Other improvements in Mix include better compiler diagnostics for editor integra
   * [Kernel] Solve a precedence issue between `&` and `|`, such as `[&Foo.bar/1 | &Baz.bat/2]`
   * [Kernel] Do not load dynamic Elixir modules as `:in_memory` as this value is not officially supported by the code server. Instead, use an empty list, which is the same value used by Erlang.
   * [Kernel] Validate variable struct name is atom when used in pattern matching
+  * [Kernel] No longer generate documentation for `defdelegate` functions automatically to avoid overriding previously specified `@doc`
   * [Macro] Fix `Macro.to_string/2` for tuple calls, such as `alias Foo.{Bar, Baz}`
   * [MapSet] Return valid MapSet when unioning a legacy MapSet
   * [Regex] Return a leading empty space when splitting on empty pattern. This makes the `split` operation consistent with the other operations in the `Regex` module
@@ -255,15 +353,23 @@ Other improvements in Mix include better compiler diagnostics for editor integra
 
 #### Elixir
 
+  * [GenServer] Warn if `init/1` is not defined in `GenServer`. This brings GenServer closer to the implementation in OTP and aligns all behaviours to require the `init/1` callback
   * [Inspect.Algebra] `surround/3` and `surround_many/6` are deprecated in favor of `container_doc/6`
+  * [Kernel] Specifying map types with variable keys without defining the type as required/optional is deprecated
   * [Kernel.ParallelCompiler] `files/2` and `files_to_path/3` are deprecated in favor of `compile/2` and `compile_to_path/3`
   * [Kernel.ParallelRequire] `files/2` is deprecated in favor of `Kernel.ParallelCompiler.require/2`
-  * [GenServer] Warn if `init/1` is not defined in `GenServer`. This brings GenServer closer to the implementation in OTP and aligns all behaviours to require the `init/1` callback
+  * [Supervisor] The `:simple_one_for_one` strategy is deprecated in favor of `DynamicSupervisor`
+  * [Supervisor] Passing a list of args to `Supervisor.start_child/2` is deprecated in favor of `DynamicSupervisor`
+  * [Task.Supervisor] Passing `:restart` and `:shutdown` to `Task.Supervisor.start_link/2` is deprecated (it should be passed on start child instead)
 
 #### ExUnit
 
   * [ExUnit.Formatter] `:case_started` and `:case_finished` events are deprecated in favor of `:module_started` and `:module_finished`
 
+#### Mix
+
+  * [Mix.Compilers.Erlang] Returning `{:ok, val} | :error` from custom Erlang compilers is deprecated in favor of `{:ok, val, warnings} | {:error, errors, warnings}`
+
 ### 4. Deprecations
 
 #### Elixir

README.md

@@ -2,8 +2,6 @@
 =========
 [![Travis build](https://secure.travis-ci.org/elixir-lang/elixir.svg?branch=master
 "Build Status")](https://travis-ci.org/elixir-lang/elixir)
-[![Windows build](https://ci.appveyor.com/api/projects/status/macwuxq7aiiv61g1?svg=true)](https://ci.appveyor.com/project/josevalim/elixir)
-
 
 Elixir is a dynamic, functional language designed for building scalable and maintainable applications.
 

VERSION

@@ -1 +1 @@
-1.6.1
+1.6.6

lib/elixir/lib/agent.ex

@@ -139,7 +139,12 @@ defmodule Agent do
   @typedoc "The agent state"
   @type state :: term
 
-  @doc false
+  @doc """
+  Returns a specification to start an agent under a supervisor.
+
+  See `Supervisor`.
+  """
+  @since "1.5.0"
   def child_spec(arg) do
     %{
       id: Agent,
@@ -150,7 +155,11 @@ defmodule Agent do
   @doc false
   defmacro __using__(opts) do
     quote location: :keep, bind_quoted: [opts: opts] do
-      @doc false
+      @doc """
+      Returns a specification to start this module under a supervisor.
+
+      See `Supervisor`.
+      """
       def child_spec(arg) do
         default = %{
           id: __MODULE__,

lib/elixir/lib/application.ex

@@ -79,10 +79,19 @@ defmodule Application do
     * [`:application` module](http://www.erlang.org/doc/man/application.html)
     * [Applications – OTP Design Principles](http://www.erlang.org/doc/design_principles/applications.html)
 
-  A developer may also implement the `stop/1` callback (automatically defined
-  by `use Application`) which does any application cleanup. It receives the
-  application state and can return any value. Note that shutting down the
-  supervisor is automatically handled by the VM.
+  When an application is shutting down, its `c:stop/1` callback is called after
+  the supervision tree has been stopped by the runtime. This callback allows the
+  application to do any final cleanup. The argument is the state returned by
+  `c:start/2`, if it did, or `[]` otherwise. The return value of `c:stop/1` is
+  ignored.
+
+  By using `Application`, modules get a default implementation of `c:stop/1`
+  that ignores its argument and returns `:ok`, but it can be overridden.
+
+  Application callback modules may also implement the optional callback
+  `c:prep_stop/1`. If present, `c:prep_stop/1` is invoked before the supervision
+  tree is terminated. Its argument is the state returned by `c:start/2`, if it did,
+  or `[]` otherwise, and its return value is passed to `c:stop/1`.
 
   An application without a supervision tree doesn't define an application
   module callback in the application definition in `mix.exs` file. Even though
@@ -185,16 +194,26 @@ defmodule Application do
               | {:ok, pid, state}
               | {:error, reason :: term}
 
+  @doc """
+  Called before stopping the application.
+
+  This function is called before the top-level supervisor is terminated. It
+  receives the state returned by `c:start/2`, if it did, or `[]` otherwise.
+  The return value is later passed to `c:stop/1`.
+  """
+  @callback prep_stop(state) :: state
+
   @doc """
   Called after an application has been stopped.
 
   This function is called after an application has been stopped, i.e., after its
   supervision tree has been stopped. It should do the opposite of what the
-  `start/2` callback did, and should perform any necessary cleanup. The return
+  `c:start/2` callback did, and should perform any necessary cleanup. The return
   value of this callback is ignored.
 
-  `state` is the return value of the `start/2` callback or the return value of
-  the `prep_stop/1` function if the application module defines such a function.
+  `state` is the state returned by `c:start/2`, if it did, or `[]` otherwise.
+  If the optional callback `c:prep_stop/1` is present, `state` is its return
+  value instead.
 
   `use Application` defines a default implementation of this function which does
   nothing and just returns `:ok`.
@@ -212,7 +231,7 @@ defmodule Application do
   @callback start_phase(phase :: term, start_type, phase_args :: term) ::
               :ok | {:error, reason :: term}
 
-  @optional_callbacks start_phase: 3
+  @optional_callbacks start_phase: 3, prep_stop: 1
 
   @doc false
   defmacro __using__(_) do

lib/elixir/lib/base.ex

@@ -184,16 +184,19 @@ defmodule Base do
   end
 
   defp decode_char_clauses(alphabet, :mixed) when length(alphabet) == 32 do
-    alphabet
-    |> Stream.with_index()
-    |> Enum.flat_map(fn {encoding, value} = pair ->
-      if encoding in ?A..?Z do
-        [pair, {encoding - ?A + ?a, value}]
-      else
-        [pair]
-      end
-    end)
-    |> decode_clauses()
+    clauses =
+      alphabet
+      |> Stream.with_index()
+      |> Enum.flat_map(fn {encoding, value} = pair ->
+        if encoding in ?A..?Z do
+          [pair, {encoding - ?A + ?a, value}]
+        else
+          [pair]
+        end
+      end)
+      |> decode_clauses()
+
+    clauses ++ bad_digit_clause()
   end
 
   defp decode_mixed_clauses(first, second) do

lib/elixir/lib/code.ex

@@ -343,14 +343,14 @@ defmodule Code do
 
       [foo, bar]
 
-  You can also force keywords to be rendered on multiple lines by
-  having each entry on its own line:
+  You can also force function calls and keywords to be rendered on multiple
+  lines by having each entry on its own line:
 
       defstruct name: nil,
                 age: 0
 
   The code above will be kept with one keyword entry per line by the
-  formatter. To avoid that, just keep everything on a single line.
+  formatter. To avoid that, just squash everything into a single line.
 
   ### Parens and no parens in function calls
 

lib/elixir/lib/code/formatter.ex

@@ -10,6 +10,7 @@ defmodule Code.Formatter do
   @min_line 0
   @max_line 9_999_999
   @empty empty()
+  @ampersand_prec Code.Identifier.unary_op(:&) |> elem(1)
 
   # Operators that do not have space between operands
   @no_space_binary_operators [:..]
@@ -17,8 +18,8 @@ defmodule Code.Formatter do
   # Operators that do not have newline between operands (as well as => and keywords)
   @no_newline_binary_operators [:\\, :in]
 
-  # Left associative operators that start on the next line in case of breaks
-  @left_new_line_before_binary_operators [:|>, :~>>, :<<~, :~>, :<~, :<~>, :<|>]
+  # Left associative operators that start on the next line in case of breaks (always pipes)
+  @pipeline_operators [:|>, :~>>, :<<~, :~>, :<~, :<~>, :<|>]
 
   # Right associative operators that start on the next line in case of breaks
   @right_new_line_before_binary_operators [:|, :when]
@@ -253,6 +254,7 @@ defmodule Code.Formatter do
       |> Keyword.get(:locals_without_parens, [])
       |> MapSet.new()
       |> MapSet.union(@locals_without_parens)
+      |> MapSet.to_list()
 
     %{
       locals_without_parens: locals_without_parens,
@@ -325,12 +327,12 @@ defmodule Code.Formatter do
 
   # Special AST nodes from compiler feedback.
 
-  defp quoted_to_algebra({:special, :clause_args, [args]}, _context, state) do
+  defp quoted_to_algebra({{:special, :clause_args}, _meta, [args]}, _context, state) do
     {doc, state} = clause_args_to_algebra(args, state)
     {group(doc), state}
   end
 
-  defp quoted_to_algebra({:special, :bitstring_segment, [arg, last]}, _context, state) do
+  defp quoted_to_algebra({{:special, :bitstring_segment}, _meta, [arg, last]}, _context, state) do
     bitstring_segment_to_algebra({arg, -1}, state, last)
   end
 
@@ -418,11 +420,11 @@ defmodule Code.Formatter do
   # {}
   # {1, 2}
   defp quoted_to_algebra({:{}, meta, args}, _context, state) do
-    tuple_to_algebra(meta, args, :flex_glue, state)
+    tuple_to_algebra(meta, args, :flex_break, state)
   end
 
   defp quoted_to_algebra({:__block__, meta, [{left, right}]}, _context, state) do
-    tuple_to_algebra(meta, [left, right], :flex_glue, state)
+    tuple_to_algebra(meta, [left, right], :flex_break, state)
   end
 
   defp quoted_to_algebra({:__block__, meta, [list]}, _context, state) when is_list(list) do
@@ -520,7 +522,7 @@ defmodule Code.Formatter do
   end
 
   defp quoted_to_algebra({:fn, meta, [_ | _] = clauses}, _context, state) do
-    anon_fun_to_algebra(clauses, line(meta), end_line(meta), state)
+    anon_fun_to_algebra(clauses, line(meta), end_line(meta), state, eol?(meta))
   end
 
   defp quoted_to_algebra({fun, meta, args}, context, state) when is_atom(fun) and is_list(args) do
@@ -553,7 +555,16 @@ defmodule Code.Formatter do
         {left, state} =
           case left_arg do
             {:__block__, _, [atom]} when is_atom(atom) ->
-              {atom |> Code.Identifier.inspect_as_key() |> string(), state}
+              key =
+                case Code.Identifier.classify(atom) do
+                  type when type in [:callable_local, :callable_operator, :not_callable] ->
+                    IO.iodata_to_binary([Atom.to_string(atom), ?:])
+
+                  _ ->
+                    IO.iodata_to_binary([?", Atom.to_string(atom), ?", ?:])
+                end
+
+              {string(key), state}
 
             {{:., _, [:erlang, :binary_to_atom]}, _, [{:<<>>, _, entries}, :utf8]} ->
               interpolation_to_algebra(entries, @double_quote, state, "\"", "\":")
@@ -670,23 +681,75 @@ defmodule Code.Formatter do
   # strict or flex mode around.
   defp binary_op_to_algebra(op, op_string, meta, left_arg, right_arg, context, state) do
     %{operand_nesting: nesting} = state
-    binary_op_to_algebra(op, op_string, meta, left_arg, right_arg, context, state, nil, nesting)
+    binary_op_to_algebra(op, op_string, meta, left_arg, right_arg, context, state, nesting)
   end
 
-  defp binary_op_to_algebra(
-         op,
-         op_string,
-         meta,
-         left_arg,
-         right_arg,
-         context,
-         state,
-         parent_info,
-         nesting
-       ) do
+  defp binary_op_to_algebra(op, op_string, meta, left_arg, right_arg, context, state, _nesting)
+       when op in @right_new_line_before_binary_operators do
     op_info = Code.Identifier.binary_op(op)
-    left_context = force_many_args_or_operand(context, :parens_arg)
-    right_context = force_many_args_or_operand(context, :operand)
+    op_string = op_string <> " "
+    left_context = left_op_context(context)
+    right_context = right_op_context(context)
+
+    min_line =
+      case left_arg do
+        {_, left_meta, _} -> line(left_meta)
+        _ -> line(meta)
+      end
+
+    {operands, max_line} =
+      unwrap_right(right_arg, op, meta, right_context, [{{:root, left_context}, left_arg}])
+
+    operand_to_algebra = fn
+      {{:root, context}, arg}, _args, newlines, state ->
+        {doc, state} = binary_operand_to_algebra(arg, context, state, op, op_info, :left, 2)
+        {doc, @empty, newlines, state}
+
+      {{kind, context}, arg}, _args, newlines, state ->
+        {doc, state} = binary_operand_to_algebra(arg, context, state, op, op_info, kind, 0)
+        doc = doc |> nest_by_length(op_string) |> force_keyword(arg)
+        {concat(op_string, doc), @empty, newlines, state}
+    end
+
+    operand_to_algebra_with_comments(
+      operands,
+      meta,
+      min_line,
+      max_line,
+      state,
+      operand_to_algebra
+    )
+  end
+
+  defp binary_op_to_algebra(op, _, meta, left_arg, right_arg, context, state, _nesting)
+       when op in @pipeline_operators do
+    op_info = Code.Identifier.binary_op(op)
+    left_context = left_op_context(context)
+    right_context = right_op_context(context)
+    max_line = line(meta)
+
+    {pipes, min_line} =
+      unwrap_pipes(left_arg, meta, left_context, [{{op, right_context}, right_arg}])
+
+    operand_to_algebra = fn
+      {{:root, context}, arg}, _args, newlines, state ->
+        {doc, state} = binary_operand_to_algebra(arg, context, state, op, op_info, :left, 2)
+        {doc, @empty, newlines, state}
+
+      {{op, context}, arg}, _args, newlines, state ->
+        op_info = Code.Identifier.binary_op(op)
+        op_string = Atom.to_string(op) <> " "
+        {doc, state} = binary_operand_to_algebra(arg, context, state, op, op_info, :right, 0)
+        {concat(op_string, doc), @empty, newlines, state}
+    end
+
+    operand_to_algebra_with_comments(pipes, meta, min_line, max_line, state, operand_to_algebra)
+  end
+
+  defp binary_op_to_algebra(op, op_string, meta, left_arg, right_arg, context, state, nesting) do
+    op_info = Code.Identifier.binary_op(op)
+    left_context = left_op_context(context)
+    right_context = right_op_context(context)
 
     {left, state} =
       binary_operand_to_algebra(left_arg, left_context, state, op, op_info, :left, 2)
@@ -703,39 +766,6 @@ defmodule Code.Formatter do
           op_string = " " <> op_string <> " "
           concat(concat(group(left), op_string), group(right))
 
-        op in @left_new_line_before_binary_operators ->
-          op_string = op_string <> " "
-
-          # If the parent is of the same type (computed via same precedence),
-          # we cannot group the left side yet.
-          left = if op_info == parent_info, do: left, else: group(left)
-
-          doc = glue(left, concat(op_string, group(right)))
-          if Keyword.get(meta, :eol, false), do: force_unfit(doc), else: doc
-
-        op in @right_new_line_before_binary_operators ->
-          op_string = op_string <> " "
-
-          # If the parent is of the same type (computed via same precedence),
-          # we need to nest the left side because of the associativity.
-          left =
-            if op_info == parent_info do
-              nest_by_length(left, op_string)
-            else
-              group(left)
-            end
-
-          # If the right side is of the same type, we will keep recursing
-          # and do the nesting on the left side later on (as written above).
-          right =
-            case right_arg do
-              {^op, _, [_, _]} -> right
-              _ -> right |> nest_by_length(op_string) |> force_keyword(right_arg) |> group()
-            end
-
-          doc = glue(left, concat(op_string, right))
-          if Keyword.get(meta, :eol, false), do: force_unfit(doc), else: doc
-
         true ->
           next_break_fits? =
             op in @next_break_fits_operators and next_break_fits?(right_arg, state) and
@@ -769,10 +799,11 @@ defmodule Code.Formatter do
   end
 
   defp binary_operand_to_algebra(operand, context, state, parent_op, parent_info, side, nesting) do
+    {parent_assoc, parent_prec} = parent_info
+
     with {op, meta, [left, right]} <- operand,
          op_info = Code.Identifier.binary_op(op),
          {_assoc, prec} <- op_info do
-      {parent_assoc, parent_prec} = parent_info
       op_string = Atom.to_string(op)
 
       cond do
@@ -780,17 +811,7 @@ defmodule Code.Formatter do
         # the correct side, we respect the nesting rule to avoid multiple
         # nestings. This only applies for left associativity or same operator.
         parent_prec == prec and parent_assoc == side and (side == :left or op == parent_op) ->
-          binary_op_to_algebra(
-            op,
-            op_string,
-            meta,
-            left,
-            right,
-            context,
-            state,
-            parent_info,
-            nesting
-          )
+          binary_op_to_algebra(op, op_string, meta, left, right, context, state, nesting)
 
         # If the parent requires parens or the precedence is inverted or
         # it is in the wrong side, then we *need* parenthesis.
@@ -799,16 +820,18 @@ defmodule Code.Formatter do
              parent_op in @required_parens_logical_binary_operands) or parent_prec > prec or
             (parent_prec == prec and parent_assoc != side) ->
           {operand, state} =
-            binary_op_to_algebra(op, op_string, meta, left, right, context, state, parent_info, 2)
+            binary_op_to_algebra(op, op_string, meta, left, right, context, state, 2)
 
           {wrap_in_parens(operand), state}
 
         # Otherwise, we rely on precedence but also nest.
         true ->
-          binary_op_to_algebra(op, op_string, meta, left, right, context, state, parent_info, 2)
+          binary_op_to_algebra(op, op_string, meta, left, right, context, state, 2)
       end
     else
-      {:&, _, [arg]} when not is_integer(arg) and side == :left ->
+      {:&, _, [arg]}
+      when not is_integer(arg) and side == :left
+      when not is_integer(arg) and parent_assoc == :left and parent_prec > @ampersand_prec ->
         {doc, state} = quoted_to_algebra(operand, context, state)
         {wrap_in_parens(doc), state}
 
@@ -817,6 +840,45 @@ defmodule Code.Formatter do
     end
   end
 
+  defp unwrap_pipes({op, meta, [left, right]}, _meta, context, acc)
+       when op in @pipeline_operators do
+    left_context = left_op_context(context)
+    right_context = right_op_context(context)
+    unwrap_pipes(left, meta, left_context, [{{op, right_context}, right} | acc])
+  end
+
+  defp unwrap_pipes(left, meta, context, acc) do
+    min_line =
+      case left do
+        {_, meta, _} -> line(meta)
+        _ -> line(meta)
+      end
+
+    {[{{:root, context}, left} | acc], min_line}
+  end
+
+  defp unwrap_right({op, meta, [left, right]}, op, _meta, context, acc) do
+    left_context = left_op_context(context)
+    right_context = right_op_context(context)
+    unwrap_right(right, op, meta, right_context, [{{:left, left_context}, left} | acc])
+  end
+
+  defp unwrap_right(right, _op, meta, context, acc) do
+    acc = [{{:right, context}, right} | acc]
+    {Enum.reverse(acc), line(meta)}
+  end
+
+  defp operand_to_algebra_with_comments(operands, meta, min_line, max_line, state, fun) do
+    {docs, comments?, state} =
+      quoted_to_algebra_with_comments(operands, [], min_line, max_line, 1, state, fun)
+
+    if comments? or Keyword.get(meta, :eol, false) do
+      {docs |> Enum.reduce(&line(&2, &1)) |> force_unfit(), state}
+    else
+      {docs |> Enum.reduce(&glue(&2, &1)), state}
+    end
+  end
+
   ## Module attributes
 
   # @Foo
@@ -898,7 +960,7 @@ defmodule Code.Formatter do
   # expression.{arguments}
   defp remote_to_algebra({{:., _, [target, :{}]}, meta, args}, _context, state) do
     {target_doc, state} = remote_target_to_algebra(target, state)
-    {call_doc, state} = tuple_to_algebra(meta, args, :glue, state)
+    {call_doc, state} = tuple_to_algebra(meta, args, :break, state)
     {concat(concat(target_doc, "."), call_doc), state}
   end
 
@@ -998,7 +1060,11 @@ defmodule Code.Formatter do
   # function(arguments)
   defp local_to_algebra(fun, meta, args, context, state) when is_atom(fun) do
     skip_parens =
-      if skip_parens?(fun, meta, args, state), do: :skip_unless_many_args, else: :skip_if_do_end
+      cond do
+        not Keyword.get(meta, :no_parens, false) -> :required
+        local_without_parens?(fun, args, state) -> :skip_unless_many_args
+        true -> :skip_if_do_end
+      end
 
     {{call_doc, state}, wrap_in_parens?} =
       call_args_to_algebra(args, meta, context, skip_parens, true, state)
@@ -1021,7 +1087,7 @@ defmodule Code.Formatter do
   #
   defp call_args_to_algebra([], meta, _context, _parens, _list_to_keyword?, state) do
     {args_doc, _join, state} =
-      args_to_algebra_with_comments([], meta, false, false, :glue, state, &{&1, &2})
+      args_to_algebra_with_comments([], meta, false, :none, :break, state, &{&1, &2})
 
     {{surround("(", args_doc, ")"), state}, false}
   end
@@ -1055,7 +1121,6 @@ defmodule Code.Formatter do
 
   defp call_args_to_algebra_no_blocks(meta, args, skip_parens?, list_to_keyword?, extra, state) do
     {left, right} = split_last(args)
-    generators_count = count_generators(args)
     {keyword?, right} = last_arg_to_keyword(right, list_to_keyword?)
 
     context =
@@ -1065,21 +1130,22 @@ defmodule Code.Formatter do
         if skip_parens?, do: :no_parens_arg, else: :parens_arg
       end
 
-    if left != [] and keyword? and skip_parens? and generators_count == 0 do
+    if left != [] and keyword? and skip_parens? and no_generators?(args) do
       call_args_to_algebra_with_no_parens_keywords(meta, left, right, context, extra, state)
     else
-      next_break_fits? = next_break_fits?(right, state)
-      force_keyword? = keyword? and force_keyword?(right)
-      non_empty_eol? = left != [] and not next_break_fits? and Keyword.get(meta, :eol, false)
-      join = if generators_count > 1 or force_keyword? or non_empty_eol?, do: :line, else: :glue
       args = if keyword?, do: left ++ right, else: left ++ [right]
+      many_eol? = match?([_, _ | _], args) and Keyword.get(meta, :eol, false)
+      join = if force_args?(args) or many_eol?, do: :line, else: :break
+
+      next_break_fits? = join == :break and next_break_fits?(right, state)
+      last_arg_mode = if next_break_fits?, do: :next_break_fits, else: :none
 
       {args_doc, _join, state} =
         args_to_algebra_with_comments(
           args,
           meta,
           skip_parens?,
-          next_break_fits?,
+          last_arg_mode,
           join,
           state,
           &quoted_to_algebra(&1, context, &2)
@@ -1113,14 +1179,17 @@ defmodule Code.Formatter do
 
   defp call_args_to_algebra_with_no_parens_keywords(meta, left, right, context, extra, state) do
     to_algebra_fun = &quoted_to_algebra(&1, context, &2)
+    join = if force_args?(left), do: :line, else: :break
 
     {left_doc, _join, state} =
-      args_to_algebra_with_comments(left, meta, true, false, :glue, state, to_algebra_fun)
+      args_to_algebra_with_comments(left, meta, true, :force_comma, join, state, to_algebra_fun)
+
+    join = if force_args?(right) or force_args?(left ++ right), do: :line, else: :break
 
     {right_doc, _join, state} =
-      args_to_algebra_with_comments(right, meta, false, false, :glue, state, to_algebra_fun)
+      args_to_algebra_with_comments(right, meta, false, :none, join, state, to_algebra_fun)
 
-    right_doc = "," |> glue(right_doc) |> force_keyword(right) |> group(:inherit)
+    right_doc = apply(Inspect.Algebra, join, []) |> concat(right_doc) |> group(:inherit)
 
     doc =
       with_next_break_fits(true, right_doc, fn right_doc ->
@@ -1136,17 +1205,17 @@ defmodule Code.Formatter do
     {doc, state}
   end
 
-  defp skip_parens?(fun, meta, args, %{locals_without_parens: locals_without_parens}) do
+  defp local_without_parens?(fun, args, %{locals_without_parens: locals_without_parens}) do
     length = length(args)
 
-    length > 0 and Keyword.get(meta, :no_parens, false) and
+    length > 0 and
       Enum.any?(locals_without_parens, fn {key, val} ->
         key == fun and (val == :* or val == length)
       end)
   end
 
-  defp count_generators(args) do
-    Enum.count(args, &match?({:<-, _, [_, _]}, &1))
+  defp no_generators?(args) do
+    not Enum.any?(args, &match?({:<-, _, [_, _]}, &1))
   end
 
   defp do_end_blocks([{{:__block__, meta, [:do]}, _} | _] = blocks) do
@@ -1258,15 +1327,15 @@ defmodule Code.Formatter do
 
   defp bitstring_to_algebra(meta, args, state) do
     last = length(args) - 1
-    join = if Keyword.get(meta, :eol, false), do: :line, else: :flex_glue
+    join = if Keyword.get(meta, :eol, false), do: :line, else: :flex_break
     to_algebra_fun = &bitstring_segment_to_algebra(&1, &2, last)
 
     {args_doc, join, state} =
       args
       |> Enum.with_index()
-      |> args_to_algebra_with_comments(meta, false, false, join, state, to_algebra_fun)
+      |> args_to_algebra_with_comments(meta, false, :none, join, state, to_algebra_fun)
 
-    if join == :flex_glue do
+    if join == :flex_break do
       {"<<" |> concat(args_doc) |> nest(2) |> concat(">>") |> group(), state}
     else
       {surround("<<", args_doc, ">>"), state}
@@ -1274,7 +1343,7 @@ defmodule Code.Formatter do
   end
 
   defp bitstring_segment_to_algebra({{:<-, meta, [left, right]}, i}, state, last) do
-    left = {:special, :bitstring_segment, [left, last]}
+    left = {{:special, :bitstring_segment}, meta, [left, last]}
     {doc, state} = quoted_to_algebra({:<-, meta, [left, right]}, :parens_arg, state)
     {bitstring_wrap_parens(doc, i, last), state}
   end
@@ -1282,8 +1351,17 @@ defmodule Code.Formatter do
   defp bitstring_segment_to_algebra({{:::, _, [segment, spec]}, i}, state, last) do
     {doc, state} = quoted_to_algebra(segment, :parens_arg, state)
     {spec, state} = bitstring_spec_to_algebra(spec, state)
-    doc = concat(concat(doc, "::"), wrap_in_parens_if_inspected_atom(spec))
-    {bitstring_wrap_parens(doc, i, last), state}
+
+    spec = wrap_in_parens_if_inspected_atom(spec)
+    spec = if i == last, do: bitstring_wrap_parens(spec, i, last), else: spec
+
+    doc =
+      doc
+      |> bitstring_wrap_parens(i, -1)
+      |> concat("::")
+      |> concat(spec)
+
+    {doc, state}
   end
 
   defp bitstring_segment_to_algebra({segment, i}, state, last) do
@@ -1301,40 +1379,38 @@ defmodule Code.Formatter do
     quoted_to_algebra_with_parens_if_operator(spec, :parens_arg, state)
   end
 
-  defp bitstring_wrap_parens(doc, i, last) do
-    if i == 0 or i == last do
-      string = format_to_string(doc)
+  defp bitstring_wrap_parens(doc, i, last) when i == 0 or i == last do
+    string = format_to_string(doc)
 
-      if (i == 0 and String.starts_with?(string, "<<")) or
-           (i == last and String.ends_with?(string, ">>")) do
-        wrap_in_parens(doc)
-      else
-        doc
-      end
+    if (i == 0 and String.starts_with?(string, ["~", "<<"])) or
+         (i == last and String.ends_with?(string, [">>"])) do
+      wrap_in_parens(doc)
     else
       doc
     end
   end
 
+  defp bitstring_wrap_parens(doc, _, _), do: doc
+
   ## Literals
 
   defp list_to_algebra(meta, args, state) do
-    join = if Keyword.get(meta, :eol, false), do: :line, else: :glue
+    join = if Keyword.get(meta, :eol, false), do: :line, else: :break
     fun = &quoted_to_algebra(&1, :parens_arg, &2)
 
     {args_doc, _join, state} =
-      args_to_algebra_with_comments(args, meta, false, false, join, state, fun)
+      args_to_algebra_with_comments(args, meta, false, :none, join, state, fun)
 
     {surround("[", args_doc, "]"), state}
   end
 
   defp map_to_algebra(meta, name_doc, [{:|, _, [left, right]}], state) do
-    join = if Keyword.get(meta, :eol, false), do: :line, else: :glue
+    join = if Keyword.get(meta, :eol, false), do: :line, else: :break
     fun = &quoted_to_algebra(&1, :parens_arg, &2)
     {left_doc, state} = fun.(left, state)
 
     {right_doc, _join, state} =
-      args_to_algebra_with_comments(right, meta, false, false, join, state, fun)
+      args_to_algebra_with_comments(right, meta, false, :none, join, state, fun)
 
     args_doc =
       left_doc
@@ -1346,11 +1422,11 @@ defmodule Code.Formatter do
   end
 
   defp map_to_algebra(meta, name_doc, args, state) do
-    join = if Keyword.get(meta, :eol, false), do: :line, else: :glue
+    join = if Keyword.get(meta, :eol, false), do: :line, else: :break
     fun = &quoted_to_algebra(&1, :parens_arg, &2)
 
     {args_doc, _join, state} =
-      args_to_algebra_with_comments(args, meta, false, false, join, state, fun)
+      args_to_algebra_with_comments(args, meta, false, :none, join, state, fun)
 
     name_doc = "%" |> concat(name_doc) |> concat("{")
     {surround(name_doc, args_doc, "}"), state}
@@ -1361,9 +1437,9 @@ defmodule Code.Formatter do
     fun = &quoted_to_algebra(&1, :parens_arg, &2)
 
     {args_doc, join, state} =
-      args_to_algebra_with_comments(args, meta, false, false, join, state, fun)
+      args_to_algebra_with_comments(args, meta, false, :none, join, state, fun)
 
-    if join == :flex_glue do
+    if join == :flex_break do
       {"{" |> concat(args_doc) |> nest(1) |> concat("}") |> group(), state}
     else
       {surround("{", args_doc, "}"), state}
@@ -1471,7 +1547,7 @@ defmodule Code.Formatter do
   defp heredoc_line(["", _ | _]), do: nest(line(), :reset)
   defp heredoc_line(_), do: line()
 
-  defp args_to_algebra_with_comments(args, meta, skip_parens?, next_break_fits?, join, state, fun) do
+  defp args_to_algebra_with_comments(args, meta, skip_parens?, last_arg_mode, join, state, fun) do
     min_line = line(meta)
     max_line = end_line(meta)
 
@@ -1479,10 +1555,11 @@ defmodule Code.Formatter do
       {doc, state} = fun.(arg, state)
 
       doc =
-        cond do
-          args != [] -> concat(doc, ",")
-          next_break_fits? -> next_break_fits(doc, :enabled)
-          true -> doc
+        case args do
+          [_ | _] -> concat_to_last_group(doc, ",")
+          [] when last_arg_mode == :force_comma -> concat_to_last_group(doc, ",")
+          [] when last_arg_mode == :next_break_fits -> next_break_fits(doc, :enabled)
+          [] when last_arg_mode == :none -> doc
         end
 
       {doc, @empty, newlines, state}
@@ -1510,18 +1587,24 @@ defmodule Code.Formatter do
       join == :line or comments? ->
         {args_docs |> Enum.reduce(&line(&2, &1)) |> force_unfit(), :line, state}
 
-      join == :glue ->
-        {args_docs |> Enum.reduce(&glue(&2, &1)), :glue, state}
+      join == :break ->
+        {args_docs |> Enum.reduce(&glue(&2, &1)), :break, state}
 
-      join == :flex_glue ->
-        {args_docs |> Enum.reduce(&flex_glue(&2, &1)), :flex_glue, state}
+      join == :flex_break ->
+        {args_docs |> Enum.reduce(&flex_glue(&2, &1)), :flex_break, state}
     end
   end
 
   ## Anonymous functions
 
   # fn -> block end
-  defp anon_fun_to_algebra([{:->, meta, [[], body]}] = clauses, _min_line, max_line, state) do
+  defp anon_fun_to_algebra(
+         [{:->, meta, [[], body]}] = clauses,
+         _min_line,
+         max_line,
+         state,
+         _multi_clauses_style
+       ) do
     min_line = line(meta)
     {body_doc, state} = block_to_algebra(body, min_line, max_line, state)
 
@@ -1540,15 +1623,27 @@ defmodule Code.Formatter do
   # fn x ->
   #   y
   # end
-  defp anon_fun_to_algebra([{:->, meta, [args, body]}] = clauses, _min_line, max_line, state) do
+  defp anon_fun_to_algebra(
+         [{:->, meta, [args, body]}] = clauses,
+         _min_line,
+         max_line,
+         state,
+         false = _multi_clauses_style
+       ) do
     min_line = line(meta)
     {args_doc, state} = clause_args_to_algebra(args, min_line, state)
     {body_doc, state} = block_to_algebra(body, min_line, max_line, state)
 
+    head =
+      args_doc
+      |> ungroup_if_group()
+      |> concat(" ->")
+      |> nest(:cursor)
+      |> group()
+
     doc =
       "fn "
-      |> concat(group(nest(args_doc, :cursor)))
-      |> concat(" ->")
+      |> concat(head)
       |> glue(body_doc)
       |> nest(2)
       |> glue("end")
@@ -1564,7 +1659,7 @@ defmodule Code.Formatter do
   #   args2 ->
   #     block2
   # end
-  defp anon_fun_to_algebra(clauses, min_line, max_line, state) do
+  defp anon_fun_to_algebra(clauses, min_line, max_line, state, _multi_clauses_style) do
     {clauses_doc, state} = clauses_to_algebra(clauses, min_line, max_line, state)
     {"fn" |> line(clauses_doc) |> nest(2) |> line("end") |> force_unfit(), state}
   end
@@ -1594,15 +1689,12 @@ defmodule Code.Formatter do
     {args_doc, state} = clause_args_to_algebra(args, min_line, state)
     {body_doc, state} = block_to_algebra(body, min_line, max_line, state)
 
-    clause_doc =
-      " ->"
-      |> glue(body_doc)
-      |> nest(2)
-
     doc =
       args_doc
+      |> ungroup_if_group()
+      |> concat(" ->")
       |> group()
-      |> concat(clause_doc)
+      |> concat(break() |> concat(body_doc) |> nest(2))
       |> wrap_in_parens()
       |> maybe_force_clauses(clauses)
       |> group()
@@ -1670,7 +1762,15 @@ defmodule Code.Formatter do
 
     state = %{state | operand_nesting: nesting}
     {body_doc, state} = block_to_algebra(body, min_line, end_line(meta), state)
-    {concat(args_doc, " ->" |> glue(body_doc) |> nest(2)), state}
+
+    doc =
+      args_doc
+      |> ungroup_if_group()
+      |> concat(" ->")
+      |> group()
+      |> concat(break() |> concat(body_doc) |> nest(2))
+
+    {doc, state}
   end
 
   defp add_max_line_to_last_clause([{op, meta, args}], max_line) do
@@ -1686,7 +1786,7 @@ defmodule Code.Formatter do
     fun = &clause_args_to_algebra/2
 
     {args_docs, _join, state} =
-      args_to_algebra_with_comments([args], meta, false, false, :glue, state, fun)
+      args_to_algebra_with_comments([args], meta, false, :none, :break, state, fun)
 
     {args_docs, state}
   end
@@ -1694,7 +1794,7 @@ defmodule Code.Formatter do
   # fn a, b, c when d -> e end
   defp clause_args_to_algebra([{:when, meta, args}], state) do
     {args, right} = split_last(args)
-    left = {:special, :clause_args, [args]}
+    left = {{:special, :clause_args}, meta, [args]}
     binary_op_to_algebra(:when, "when", meta, left, right, :no_parens_arg, state)
   end
 
@@ -1824,6 +1924,9 @@ defmodule Code.Formatter do
 
   ## Quoted helpers
 
+  defp left_op_context(context), do: force_many_args_or_operand(context, :parens_arg)
+  defp right_op_context(context), do: force_many_args_or_operand(context, :operand)
+
   defp force_many_args_or_operand(:no_parens_one_arg, _choice), do: :no_parens_arg
   defp force_many_args_or_operand(:parens_one_arg, _choice), do: :parens_arg
   defp force_many_args_or_operand(:no_parens_arg, _choice), do: :no_parens_arg
@@ -1985,38 +2088,47 @@ defmodule Code.Formatter do
       )
   end
 
+  # A literal list is a keyword or (... -> ...)
   defp last_arg_to_keyword([_ | _] = arg, _list_to_keyword?) do
     {keyword?(arg), arg}
   end
 
+  # This is a list of tuples, it can be converted to keywords.
   defp last_arg_to_keyword({:__block__, _, [[_ | _] = arg]} = block, true) do
     if keyword?(arg), do: {true, arg}, else: {false, block}
   end
 
+  # Otherwise we don't have a keyword.
   defp last_arg_to_keyword(arg, _list_to_keyword?) do
     {false, arg}
   end
 
-  defp force_keyword?(keyword) do
-    match?([_, _ | _], keyword) and force_keyword?(keyword, MapSet.new())
+  defp force_args?(args) do
+    match?([_, _ | _], args) and force_args?(args, MapSet.new())
   end
 
-  defp force_keyword?([{{_, meta, _}, _} | keyword], lines) do
-    line = line(meta)
+  defp force_args?([[arg | _] | args], lines) do
+    force_args?([arg | args], lines)
+  end
 
-    if line in lines do
+  defp force_args?([arg | args], lines) do
+    line =
+      case arg do
+        {{_, meta, _}, _} -> line(meta)
+        {_, meta, _} -> line(meta)
+      end
+
+    if MapSet.member?(lines, line) do
       false
     else
-      force_keyword?(keyword, MapSet.put(lines, line))
+      force_args?(args, MapSet.put(lines, line))
     end
   end
 
-  defp force_keyword?([], _lines) do
-    true
-  end
+  defp force_args?([], _lines), do: true
 
   defp force_keyword(doc, arg) do
-    if force_keyword?(arg), do: force_unfit(doc), else: doc
+    if force_args?(arg), do: force_unfit(doc), else: doc
   end
 
   defp keyword?([{key, _} | list]) do
@@ -2039,6 +2151,10 @@ defmodule Code.Formatter do
     false
   end
 
+  defp eol?(meta) do
+    Keyword.get(meta, :eol, false)
+  end
+
   defp line(meta) do
     Keyword.get(meta, :line, @max_line)
   end
@@ -2049,6 +2165,20 @@ defmodule Code.Formatter do
 
   ## Algebra helpers
 
+  # Relying on the inner document is brittle and error prone.
+  # It would be best if we had a mechanism to apply this.
+  defp concat_to_last_group({:doc_cons, left, right}, concat) do
+    {:doc_cons, left, concat_to_last_group(right, concat)}
+  end
+
+  defp concat_to_last_group({:doc_group, group, mode}, concat) do
+    {:doc_group, {:doc_cons, group, concat}, mode}
+  end
+
+  defp concat_to_last_group(other, concat) do
+    {:doc_cons, other, concat}
+  end
+
   defp ungroup_if_group({:doc_group, group, _mode}), do: group
   defp ungroup_if_group(other), do: other
 

lib/elixir/lib/collectable.ex

@@ -101,6 +101,9 @@ defimpl Collectable, for: BitString do
       acc, {:cont, x} when is_bitstring(x) ->
         <<IO.iodata_to_binary(acc)::bitstring, x::bitstring>>
 
+      acc, :done when is_bitstring(acc) ->
+        acc
+
       acc, :done ->
         IO.iodata_to_binary(acc)
 

lib/elixir/lib/dynamic_supervisor.ex

@@ -11,21 +11,27 @@ defmodule DynamicSupervisor do
 
   ## Examples
 
-  A dynamic supervisor is started with no children, only with the
-  supervision strategy (the only strategy currently supported is
-  `:one_for_one`):
+  A dynamic supervisor is started with no children, often under a
+  supervisor with the supervision strategy (the only strategy currently
+  supported is `:one_for_one`) and a name:
 
-      {:ok, sup} = DynamicSupervisor.start_link(strategy: :one_for_one)
+      children = [
+        {DynamicSupervisor, strategy: :one_for_one, name: MyApp.DynamicSupervisor}
+      ]
+
+      Supervisor.start_link(strategy: :one_for_one)
+
+  The options given in the child specification are documented in `start_link/1`.
 
   Once the dynamic supervisor is running, we can start children
   with `start_child/2`, which receives a child specification:
 
-      {:ok, agent1} = DynamicSupervisor.start_child(sup, {Agent, fn -> %{} end})
+      {:ok, agent1} = DynamicSupervisor.start_child(MyApp.DynamicSupervisor, {Agent, fn -> %{} end})
       Agent.update(agent1, &Map.put(&1, :key, "value"))
       Agent.get(agent1, & &1)
       #=> %{key: "value"}
 
-      {:ok, agent2} = DynamicSupervisor.start_child(sup, {Agent, fn -> %{} end})
+      {:ok, agent2} = DynamicSupervisor.start_child(MyApp.DynamicSupervisor, {Agent, fn -> %{} end})
       Agent.get(agent2, & &1)
       #=> %{}
 
@@ -45,6 +51,7 @@ defmodule DynamicSupervisor do
           DynamicSupervisor.start_link(__MODULE__, arg, name: __MODULE__)
         end
 
+        @impl true
         def init(_arg) do
           DynamicSupervisor.init(strategy: :one_for_one)
         end
@@ -74,16 +81,18 @@ defmodule DynamicSupervisor do
         end
 
         def start_child(foo, bar, baz) do
-          # This will start child by calling MyWorker.start_link(implicit_arg, foo, bar, baz)
+          # This will start child by calling MyWorker.start_link(initial_arg, foo, bar, baz)
           Supervisor.start_child(__MODULE__, [foo, bar, baz])
         end
 
-        def init(implicit_arg) do
+        @impl true
+        def init(initial_arg) do
           children = [
-            worker(MyWorker, [implicit_arg])
+            # Or the deprecated: worker(MyWorker, [initial_arg])
+            %{id: MyWorker, start: {MyWorker, :start_link, [initial_arg]})
           ]
 
-          supervise(children, strategy: :simple_one_for_one)
+          Supervisor.init(children, strategy: :simple_one_for_one)
         end
       end
 
@@ -97,22 +106,24 @@ defmodule DynamicSupervisor do
         end
 
         def start_child(foo, bar, baz) do
-          # This will start child by calling MyWorker.start_link(implicit_arg, foo, bar, baz)
-          spec = Supervisor.Spec.worker(MyWorker, [foo, bar, baz])
+          # If MyWorker is not using the new child specs, we need to pass a map:
+          # spec = %{id: MyWorker, start: {MyWorker, :start_link, [foo, bar, baz]}}
+          spec = {MyWorker, foo: foo, bar: bar, baz: baz}
           DynamicSupervisor.start_child(__MODULE__, spec)
         end
 
-        def init(implicit_arg) do
+        @impl true
+        def init(initial_arg) do
           DynamicSupervisor.init(
             strategy: :one_for_one,
-            extra_arguments: [implicit_arg]
+            extra_arguments: [initial_arg]
           )
         end
       end
 
   The difference is that the `DynamicSupervisor` expects the child specification
   at the moment `start_child/2` is called, and no longer on the init callback.
-  If there are any implicit arguments given on initialization, such as `[implicit_arg]`,
+  If there are any initial arguments given on initialization, such as `[initial_arg]`,
   it can be given in the `:extra_arguments` flag on `DynamicSupervisor.init/1`.
   """
 
@@ -126,13 +137,14 @@ defmodule DynamicSupervisor do
   """
   @callback init(args :: term) :: {:ok, sup_flags()} | :ignore
 
-  @opaque sup_flags() :: %{
-            strategy: strategy(),
-            intensity: non_neg_integer(),
-            period: pos_integer(),
-            max_children: non_neg_integer() | :infinity,
-            extra_arguments: [term()]
-          }
+  @typedoc "The supervisor flags returned on init"
+  @type sup_flags() :: %{
+          strategy: strategy(),
+          intensity: non_neg_integer(),
+          period: pos_integer(),
+          max_children: non_neg_integer() | :infinity,
+          extra_arguments: [term()]
+        }
 
   @typedoc "Option values used by the `start*` functions"
   @type option :: {:name, Supervisor.name()} | init_option()
@@ -151,6 +163,13 @@ defmodule DynamicSupervisor do
   @typedoc "Supported strategies"
   @type strategy :: :one_for_one
 
+  @typedoc "Return values of `start_child` functions"
+  @type on_start_child ::
+          {:ok, pid}
+          | {:ok, pid, info :: term}
+          | :ignore
+          | {:error, {:already_started, pid} | :max_children | term}
+
   defstruct [
     :args,
     :extra_arguments,
@@ -161,11 +180,15 @@ defmodule DynamicSupervisor do
     :max_restarts,
     :max_seconds,
     children: %{},
-    dynamic: 0,
     restarts: []
   ]
 
-  @doc false
+  @doc """
+  Returns a specification to start a dynamic supervisor under a supervisor.
+
+  See `Supervisor`.
+  """
+  @since "1.6.1"
   def child_spec(arg) do
     %{
       id: DynamicSupervisor,
@@ -179,7 +202,11 @@ defmodule DynamicSupervisor do
     quote location: :keep, bind_quoted: [opts: opts] do
       @behaviour DynamicSupervisor
 
-      @doc false
+      @doc """
+      Returns a specification to start this module under a supervisor.
+
+      See `Supervisor`.
+      """
       def child_spec(arg) do
         default = %{
           id: __MODULE__,
@@ -191,9 +218,6 @@ defmodule DynamicSupervisor do
       end
 
       defoverridable child_spec: 1
-
-      @doc false
-      def init(arg)
     end
   end
 
@@ -271,7 +295,7 @@ defmodule DynamicSupervisor do
   this function returns `{:error, :max_children}`.
   """
   @spec start_child(Supervisor.supervisor(), :supervisor.child_spec() | {module, term} | module) ::
-          Supervisor.on_start_child()
+          on_start_child()
   def start_child(supervisor, {_, _, _, _, _, _} = child_spec) do
     validate_and_start_child(supervisor, child_spec)
   end
@@ -385,7 +409,7 @@ defmodule DynamicSupervisor do
 
   The map contains the following keys:
 
-    * `:specs` - always 1 as dynamic supervisors have a single specification
+    * `:specs` - the number of children processes
 
     * `:active` - the count of all actively running child processes managed by
       this supervisor
@@ -478,7 +502,14 @@ defmodule DynamicSupervisor do
 
     case mod.init(args) do
       {:ok, flags} when is_map(flags) ->
-        state = %DynamicSupervisor{mod: mod, args: args, name: name || {self(), mod}}
+        name =
+          cond do
+            is_nil(name) -> {self(), mod}
+            is_atom(name) -> {:local, name}
+            is_tuple(name) -> name
+          end
+
+        state = %DynamicSupervisor{mod: mod, args: args, name: name}
 
         case init(state, flags) do
           {:ok, state} -> {:ok, state}
@@ -594,10 +625,10 @@ defmodule DynamicSupervisor do
   end
 
   def handle_call({:start_child, child}, _from, state) do
-    %{dynamic: dynamic, max_children: max_children} = state
+    %{children: children, max_children: max_children} = state
 
-    if dynamic < max_children do
-      handle_start_child(child, %{state | dynamic: dynamic + 1})
+    if map_size(children) < max_children do
+      handle_start_child(child, state)
     else
       {:reply, {:error, :max_children}, state}
     end
@@ -614,7 +645,7 @@ defmodule DynamicSupervisor do
         {:reply, reply, save_child(pid, mfa, restart, shutdown, type, modules, state)}
 
       _ ->
-        {:reply, reply, update_in(state.dynamic, &(&1 - 1))}
+        {:reply, reply, state}
     end
   end
 
@@ -633,14 +664,14 @@ defmodule DynamicSupervisor do
     end
   end
 
-  defp save_child(pid, {m, f, _}, :temporary, shutdown, type, modules, state) do
-    put_in(state.children[pid], {{m, f, :undefined}, :temporary, shutdown, type, modules})
-  end
-
   defp save_child(pid, mfa, restart, shutdown, type, modules, state) do
+    mfa = mfa_for_restart(mfa, restart)
     put_in(state.children[pid], {mfa, restart, shutdown, type, modules})
   end
 
+  defp mfa_for_restart({m, f, _}, :temporary), do: {m, f, :undefined}
+  defp mfa_for_restart(mfa, _), do: mfa
+
   defp exit_reason(:exit, reason, _), do: reason
   defp exit_reason(:error, reason, stack), do: {reason, stack}
   defp exit_reason(:throw, value, stack), do: {{:nocatch, value}, stack}
@@ -857,9 +888,8 @@ defmodule DynamicSupervisor do
     {:ok, delete_child(pid, state)}
   end
 
-  defp delete_child(pid, state) do
-    %{children: children, dynamic: dynamic} = state
-    %{state | children: Map.delete(children, pid), dynamic: dynamic - 1}
+  defp delete_child(pid, %{children: children} = state) do
+    %{state | children: Map.delete(children, pid)}
   end
 
   defp restart_child(pid, child, state) do
@@ -902,8 +932,9 @@ defmodule DynamicSupervisor do
 
   defp restart_child(:one_for_one, current_pid, child, state) do
     {{m, f, args} = mfa, restart, shutdown, type, modules} = child
+    %{extra_arguments: extra} = state
 
-    case start_child(m, f, args) do
+    case start_child(m, f, extra ++ args) do
       {:ok, pid, _} ->
         state = delete_child(current_pid, state)
         {:ok, save_child(pid, mfa, restart, shutdown, type, modules, state)}
@@ -922,21 +953,21 @@ defmodule DynamicSupervisor do
     end
   end
 
-  defp report_error(error, reason, pid, child, %{name: name}) do
+  defp report_error(error, reason, pid, child, %{name: name, extra_arguments: extra}) do
     :error_logger.error_report(
-      :supervision_report,
+      :supervisor_report,
       supervisor: name,
       errorContext: error,
       reason: reason,
-      offender: extract_child(pid, child)
+      offender: extract_child(pid, child, extra)
     )
   end
 
-  defp extract_child(pid, {mfa, restart, shutdown, type, _modules}) do
+  defp extract_child(pid, {{m, f, args}, restart, shutdown, type, _modules}, extra) do
     [
       pid: pid,
       id: :undefined,
-      mfargs: mfa,
+      mfargs: {m, f, extra ++ args},
       restart_type: restart,
       shutdown: shutdown,
       child_type: type

lib/elixir/lib/gen_event/stream.ex

@@ -161,7 +161,8 @@ defimpl Enumerable, for: GenEvent.Stream do
 
   defp flush_events(ref) do
     receive do
-      {_from, {_pid, ^ref}, {notify, _event}} when notify in [:notify, :ack_notify, :sync_notify] ->
+      {_from, {_pid, ^ref}, {notify, _event}}
+      when notify in [:notify, :ack_notify, :sync_notify] ->
         flush_events(ref)
     after
       0 -> :ok

lib/elixir/lib/gen_server.ex

@@ -23,10 +23,17 @@ defmodule GenServer do
 
         # Callbacks
 
+        @impl true
+        def init(stack) do
+          {:ok, stack}
+        end
+
+        @impl true
         def handle_call(:pop, _from, [h | t]) do
           {:reply, h, t}
         end
 
+        @impl true
         def handle_cast({:push, item}, state) do
           {:noreply, [item | state]}
         end
@@ -151,22 +158,15 @@ defmodule GenServer do
 
         # Server (callbacks)
 
+        @impl true
         def handle_call(:pop, _from, [h | t]) do
           {:reply, h, t}
         end
 
-        def handle_call(request, from, state) do
-          # Call the default implementation from GenServer
-          super(request, from, state)
-        end
-
+        @impl true
         def handle_cast({:push, item}, state) do
           {:noreply, [item | state]}
         end
-
-        def handle_cast(request, state) do
-          super(request, state)
-        end
       end
 
   In practice, it is common to have both server and client functions in
@@ -196,11 +196,13 @@ defmodule GenServer do
           GenServer.start_link(__MODULE__, %{})
         end
 
+        @impl true
         def init(state) do
           schedule_work() # Schedule work to be performed on start
           {:ok, state}
         end
 
+        @impl true
         def handle_info(:work, state) do
           # Do the desired work here
           schedule_work() # Reschedule once more
@@ -570,7 +572,11 @@ defmodule GenServer do
     quote location: :keep, bind_quoted: [opts: opts] do
       @behaviour GenServer
 
-      @doc false
+      @doc """
+      Returns a specification to start this module under a supervisor.
+
+      See `Supervisor`.
+      """
       def child_spec(arg) do
         default = %{
           id: __MODULE__,
@@ -660,8 +666,8 @@ defmodule GenServer do
             {:ok, args}
           end
 
-      But you want to define your own implementation that converts the \
-      arguments given to GenServer.start_link/3 to the server state
+      You can copy the implementation above or define your own that converts \
+      the arguments given to GenServer.start_link/3 to the server state.
       """
 
       :elixir_errors.warn(env.line, env.file, message)

lib/elixir/lib/kernel.ex

@@ -2709,7 +2709,10 @@ defmodule Kernel do
     case function? do
       true ->
         value =
-          with {_, doc} when doc_attr? <- Module.get_attribute(env.module, name, stack), do: doc
+          case Module.get_attribute(env.module, name, stack) do
+            {_, doc} when doc_attr? -> doc
+            other -> other
+          end
 
         try do
           :elixir_quote.escape(value, false)
@@ -2722,13 +2725,21 @@ defmodule Kernel do
           {val, _} -> val
         end
 
+      false when doc_attr? ->
+        {escaped, _} = :elixir_quote.escape(stack, false)
+
+        quote do
+          case Module.get_attribute(__MODULE__, unquote(name), unquote(escaped)) do
+            {_, doc} -> doc
+            other -> other
+          end
+        end
+
       false ->
         {escaped, _} = :elixir_quote.escape(stack, false)
 
         quote do
-          with {_, doc} when unquote(doc_attr?) <-
-                 Module.get_attribute(__MODULE__, unquote(name), unquote(escaped)),
-               do: doc
+          Module.get_attribute(__MODULE__, unquote(name), unquote(escaped))
         end
     end
   end
@@ -4560,7 +4571,7 @@ defmodule Kernel do
   allowed in guards, and otherwise creates a private macro that can be used
   both inside or outside guards in the current module.
 
-  Similar to `defmacrop/2`, `defguardp/2` must be defined before its use
+  Similar to `defmacrop/2`, `defguardp/1` must be defined before its use
   in the current module.
   """
   @spec defguardp(Macro.t()) :: Macro.t()
@@ -4570,25 +4581,33 @@ defmodule Kernel do
 
   defp define_guard(kind, guard, env) do
     case :elixir_utils.extract_guards(guard) do
-      {call, impl} when length(impl) < 2 ->
+      {call, [_, _ | _]} ->
+        raise ArgumentError,
+              "invalid syntax in defguard #{Macro.to_string(call)}, " <>
+                "only a single when clause is allowed"
+
+      {call, impls} ->
         case Macro.decompose_call(call) do
           {_name, args} ->
             validate_variable_only_args!(call, args)
 
-            quoted =
-              quote do
-                require Kernel.Utils
-                Kernel.Utils.defguard(unquote(args), unquote(impl))
-              end
+            case impls do
+              [] ->
+                define(kind, call, nil, env)
 
-            define(kind, call, [do: quoted], env)
+              [guard] ->
+                quoted =
+                  quote do
+                    require Kernel.Utils
+                    Kernel.Utils.defguard(unquote(args), unquote(guard))
+                  end
+
+                define(kind, call, [do: quoted], env)
+            end
 
           _invalid_definition ->
             raise ArgumentError, "invalid syntax in defguard #{Macro.to_string(call)}"
         end
-
-      {call, _multiple_impls} ->
-        raise ArgumentError, "invalid syntax in defguard #{Macro.to_string(call)}"
     end
   end
 

lib/elixir/lib/kernel/parallel_compiler.ex

@@ -356,6 +356,7 @@ defmodule Kernel.ParallelCompiler do
       {:file_done, child_pid, file, {kind, reason, stack}} ->
         discard_down(child_pid)
         print_error(file, kind, reason, stack)
+        cancel_waiting_timer(queued, child_pid)
         terminate(queued)
         {:error, [to_error(file, kind, reason, stack)], warnings}
 

lib/elixir/lib/kernel/utils.ex

@@ -200,19 +200,19 @@ defmodule Kernel.Utils do
 
   defp extract_refs_from_args(args) do
     Macro.postwalk(args, [], fn
-      {ref, _meta, context} = var, acc when is_atom(ref) and is_atom(context) ->
-        {var, [{ref, context} | acc]}
+      {ref, meta, context} = var, acc when is_atom(ref) and is_atom(context) ->
+        {var, [{ref, var_context(meta, context)} | acc]}
 
       node, acc ->
         {node, acc}
     end)
   end
 
-  # Finds every reference to `refs` in `expr` and wraps them in an unquote.
-  defp unquote_every_ref(expr, refs) do
-    Macro.postwalk(expr, fn
-      {ref, _meta, context} = var when is_atom(ref) and is_atom(context) ->
-        case {ref, context} in refs do
+  # Finds every reference to `refs` in `guard` and wraps them in an unquote.
+  defp unquote_every_ref(guard, refs) do
+    Macro.postwalk(guard, fn
+      {ref, meta, context} = var when is_atom(ref) and is_atom(context) ->
+        case {ref, var_context(meta, context)} in refs do
           true -> literal_unquote(var)
           false -> var
         end
@@ -222,13 +222,15 @@ defmodule Kernel.Utils do
     end)
   end
 
-  # Prefaces `expr` with unquoted versions of `refs`.
-  defp unquote_refs_once(expr, refs) do
-    {^expr, used_refs} =
-      Macro.postwalk(expr, [], fn
-        {ref, _meta, context} = var, acc when is_atom(ref) and is_atom(context) ->
-          case {ref, context} in refs and {ref, context} not in acc do
-            true -> {var, [{ref, context} | acc]}
+  # Prefaces `guard` with unquoted versions of `refs`.
+  defp unquote_refs_once(guard, refs) do
+    {_, used_refs} =
+      Macro.postwalk(guard, [], fn
+        {ref, meta, context} = var, acc when is_atom(ref) and is_atom(context) ->
+          pair = {ref, var_context(meta, context)}
+
+          case pair in refs and pair not in acc do
+            true -> {var, [pair | acc]}
             false -> {var, acc}
           end
 
@@ -236,17 +238,30 @@ defmodule Kernel.Utils do
           {node, acc}
       end)
 
-    for {ref, context} <- :lists.reverse(used_refs) do
-      var = {ref, [], context}
-      quote do: unquote(var) = unquote(literal_unquote(var))
-    end ++ List.wrap(expr)
+    vars = for {ref, context} <- :lists.reverse(used_refs), do: context_to_var(ref, context)
+    exprs = for var <- vars, do: literal_unquote(var)
+
+    quote do
+      {unquote_splicing(vars)} = {unquote_splicing(exprs)}
+      unquote(guard)
+    end
   end
 
   defp literal_quote(ast) do
-    {:quote, [], [[do: {:__block__, [], List.wrap(ast)}]]}
+    {:quote, [], [[do: ast]]}
   end
 
   defp literal_unquote(ast) do
     {:unquote, [], List.wrap(ast)}
   end
+
+  defp context_to_var(ref, ctx) when is_atom(ctx), do: {ref, [], ctx}
+  defp context_to_var(ref, ctx) when is_integer(ctx), do: {ref, [counter: ctx], nil}
+
+  defp var_context(meta, kind) do
+    case :lists.keyfind(:counter, 1, meta) do
+      {:counter, counter} -> counter
+      false -> kind
+    end
+  end
 end

lib/elixir/lib/macro.ex

@@ -719,7 +719,13 @@ defmodule Macro do
   end
 
   # All other structures
-  def to_string(other, fun), do: fun.(other, inspect(other, []))
+  def to_string(other, fun) do
+    fun.(other, inspect_no_limit(other))
+  end
+
+  defp inspect_no_limit(value) do
+    Kernel.inspect(value, limit: :infinity, printable_limit: :infinity)
+  end
 
   defp bitpart_to_string({:::, _, [left, right]} = ast, fun) do
     result =
@@ -773,7 +779,7 @@ defmodule Macro do
           "\#{" <> to_string(arg, fun) <> "}"
 
         binary when is_binary(binary) ->
-          binary = inspect(binary, [])
+          binary = inspect_no_limit(binary)
           :binary.part(binary, 1, byte_size(binary) - 2)
       end)
 
@@ -781,7 +787,7 @@ defmodule Macro do
   end
 
   defp module_to_string(atom, _fun) when is_atom(atom) do
-    inspect(atom, [])
+    inspect_no_limit(atom)
   end
 
   defp module_to_string({:&, _, [val]} = expr, fun) when not is_integer(val) do
@@ -839,11 +845,25 @@ defmodule Macro do
     :error
   end
 
-  defp sigil_call({sigil, _, [{:<<>>, _, _} = bin, args]} = ast, fun)
+  defp sigil_call({sigil, _, [{:<<>>, _, _} = parts, args]} = ast, fun)
        when is_atom(sigil) and is_list(args) do
     case Atom.to_string(sigil) do
-      <<"sigil_", name>> ->
-        {:ok, fun.(ast, "~" <> <<name>> <> interpolate(bin, fun) <> sigil_args(args, fun))}
+      <<"sigil_", name>> when name >= ?A and name <= ?Z ->
+        {:<<>>, _, [binary]} = parts
+
+        formatted =
+          if :binary.last(binary) == ?\n do
+            binary = String.replace(binary, ~s["""], ~s["\\""])
+            <<?~, name, ~s["""\n], binary::binary, ~s["""], sigil_args(args, fun)::binary>>
+          else
+            {left, right} = select_sigil_container(binary)
+            <<?~, name, left, binary::binary, right, sigil_args(args, fun)::binary>>
+          end
+
+        {:ok, fun.(ast, formatted)}
+
+      <<"sigil_", name>> when name >= ?a and name <= ?z ->
+        {:ok, fun.(ast, "~" <> <<name>> <> interpolate(parts, fun) <> sigil_args(args, fun))}
 
       _ ->
         :error
@@ -854,6 +874,18 @@ defmodule Macro do
     :error
   end
 
+  defp select_sigil_container(binary) do
+    cond do
+      :binary.match(binary, ["\""]) == :nomatch -> {?", ?"}
+      :binary.match(binary, ["\'"]) == :nomatch -> {?', ?'}
+      :binary.match(binary, ["(", ")"]) == :nomatch -> {?(, ?)}
+      :binary.match(binary, ["[", "]"]) == :nomatch -> {?[, ?]}
+      :binary.match(binary, ["{", "}"]) == :nomatch -> {?{, ?}}
+      :binary.match(binary, ["<", ">"]) == :nomatch -> {?<, ?>}
+      true -> {?/, ?/}
+    end
+  end
+
   defp sigil_args([], _fun), do: ""
   defp sigil_args(args, fun), do: fun.(args, List.to_string(args))
 
@@ -952,8 +984,9 @@ defmodule Macro do
   end
 
   defp map_list_to_string(list, fun) do
-    Enum.map_join(list, ", ", fn {key, value} ->
-      to_string(key, fun) <> " => " <> to_string(value, fun)
+    Enum.map_join(list, ", ", fn
+      {key, value} -> to_string(key, fun) <> " => " <> to_string(value, fun)
+      other -> to_string(other, fun)
     end)
   end
 

lib/elixir/lib/map.ex

@@ -505,6 +505,8 @@ defmodule Map do
   side into the struct, even if the key is not part of the struct. Instead,
   use `Kernel.struct/2`.
 
+  Inlined by the compiler.
+
   ## Examples
 
       iex> Map.merge(%{a: 1, b: 2}, %{a: 3, d: 4})

lib/elixir/lib/module.ex

@@ -297,6 +297,8 @@ defmodule Module do
   When just a module is provided, the function is assumed to be
   `__after_compile__/2`.
 
+  Callbacks registered first will run last.
+
   #### Example
 
       defmodule MyModule do
@@ -312,13 +314,18 @@ defmodule Module do
   A hook that will be invoked before the module is compiled.
 
   Accepts a module or a `{module, function_or_macro_name}` tuple. The
-  function/macro must take one argument: the module environment. If it's a
-  macro, its returned value will be injected at the end of the module definition
-  before the compilation starts.
+  function/macro must take one argument: the module environment. If
+  it's a macro, its returned value will be injected at the end of the
+  module definition before the compilation starts.
 
   When just a module is provided, the function/macro is assumed to be
   `__before_compile__/1`.
 
+  Callbacks registered first will run last. Any overridable definition
+  will be made concrete before the first callback runs. A definition may
+  be made overridable again in another before compile callback and it
+  will be made concrete one last time after after all callbacks run.
+
   *Note*: unlike `@after_compile`, the callback function/macro must
   be placed in a separate module (because when the callback is invoked,
   the current module does not yet exist).
@@ -1195,6 +1202,20 @@ defmodule Module do
     raise ArgumentError, "expected an Elixir module, got: #{inspect(original)}"
   end
 
+  @doc false
+  # TODO: Remove in 2.0 - deprecated.
+  def add_doc(module, line, kind, function_tuple, signature \\ [], doc) do
+    assert_not_compiled!(:add_doc, module)
+
+    if kind in [:defp, :defmacrop, :typep] do
+      if doc, do: {:error, :private_doc}, else: :ok
+    else
+      table = data_table_for(module)
+      compile_doc(table, line, kind, function_tuple, signature, doc, __ENV__, false)
+      :ok
+    end
+  end
+
   @doc false
   # Used internally to compile documentation.
   # This function is private and must be used only internally.
@@ -1205,22 +1226,17 @@ defmodule Module do
     pair = {name, arity}
 
     impl = compile_impl(table, name, env, kind, args)
-    compile_doc(table, pair, env, kind, args, impl)
-    compile_deprecated(table, pair)
+    _deprecated = compile_deprecated(table, pair)
+    _since = compile_since(table)
+
+    # TODO: Store @since and @deprecated alongside the docs
+    {line, doc} = get_doc_info(table, env)
+    compile_doc(table, line, kind, pair, args, doc, env, impl)
 
     :ok
   end
 
-  defp compile_doc(table, pair, env, kind, args, impl) do
-    {line, doc} = get_doc_info(table, env, impl)
-
-    # TODO: Store @since alongside the docs
-    _ = get_since_info(table)
-
-    add_doc(table, line, kind, pair, args, doc, env)
-  end
-
-  defp add_doc(_table, line, kind, {name, arity}, _args, doc, env)
+  defp compile_doc(_table, line, kind, {name, arity}, _args, doc, env, _impl)
        when kind in [:defp, :defmacrop] do
     if doc do
       error_message =
@@ -1231,23 +1247,37 @@ defmodule Module do
     end
   end
 
-  defp add_doc(table, line, kind, pair, args, doc, env) do
+  defp compile_doc(table, line, kind, pair, args, doc, env, impl) do
     signature = build_signature(args, env)
 
     case :ets.lookup(table, {:doc, pair}) do
       [] ->
+        doc = if is_nil(doc) && impl, do: false, else: doc
         :ets.insert(table, {{:doc, pair}, line, kind, signature, doc})
 
       [{doc_tuple, line, _current_kind, current_sign, current_doc}] ->
         signature = merge_signatures(current_sign, signature, 1)
         doc = if is_nil(doc), do: current_doc, else: doc
+        doc = if is_nil(doc) && impl, do: false, else: doc
         :ets.insert(table, {doc_tuple, line, kind, signature, doc})
     end
   end
 
+  defp compile_since(table) do
+    case :ets.take(table, :since) do
+      [{:since, since, _, _}] when is_binary(since) -> since
+      _ -> nil
+    end
+  end
+
   defp compile_deprecated(table, pair) do
-    if reason = get_deprecated_info(table) do
-      :ets.insert(table, {{:deprecated, pair}, reason})
+    case :ets.take(table, :deprecated) do
+      [{:deprecated, reason, _, _}] when is_binary(reason) ->
+        :ets.insert(table, {{:deprecated, pair}, reason})
+        reason
+
+      _ ->
+        nil
     end
   end
 
@@ -1723,27 +1753,13 @@ defmodule Module do
     value
   end
 
-  defp get_doc_info(table, env, impl) do
+  defp get_doc_info(table, env) do
     case :ets.take(table, :doc) do
       [{:doc, {_, _} = pair, _, _}] ->
         pair
 
-      [] when impl == false ->
-        {env.line, nil}
-
       [] ->
-        {env.line, false}
-    end
-  end
-
-  defp get_since_info(table) do
-    :ets.take(table, :since)
-  end
-
-  defp get_deprecated_info(table) do
-    case :ets.take(table, :deprecated) do
-      [{:deprecated, reason, _, _}] -> reason
-      [] -> nil
+        {env.line, nil}
     end
   end
 

lib/elixir/lib/port.ex

@@ -117,7 +117,7 @@ defmodule Port do
   reimplementing core part of the Runtime System, such as the `:user` and
   `:shell` processes.
 
-  ## Zombie processes
+  ## Zombie OS processes
 
   A port can be closed via the `close/1` function or by sending a `{pid, :close}`
   message. However, if the VM crashes, a long-running program started by the port
@@ -132,7 +132,7 @@ defmodule Port do
   script in bash:
 
       #!/bin/sh
-      "$@"
+      "$@" &
       pid=$!
       while read line ; do
         :
@@ -236,8 +236,9 @@ defmodule Port do
 
   def info(port, :registered_name) do
     case :erlang.port_info(port, :registered_name) do
+      :undefined -> nil
       [] -> {:registered_name, []}
-      other -> nillify(other)
+      other -> other
     end
   end
 

lib/elixir/lib/registry.ex

@@ -321,7 +321,12 @@ defmodule Registry do
     start_link([keys: keys, name: name] ++ options)
   end
 
-  @doc false
+  @doc """
+  Returns a specification to start a registry under a supervisor.
+
+  See `Supervisor`.
+  """
+  @since "1.5.0"
   def child_spec(opts) do
     %{
       id: Keyword.get(opts, :name, Registry),

lib/elixir/lib/stream.ex

@@ -318,11 +318,11 @@ defmodule Stream do
 
   """
   @spec drop(Enumerable.t(), non_neg_integer) :: Enumerable.t()
-  def drop(enum, n) when n >= 0 do
+  def drop(enum, n) when is_integer(n) and n >= 0 do
     lazy(enum, n, fn f1 -> R.drop(f1) end)
   end
 
-  def drop(enum, n) when n < 0 do
+  def drop(enum, n) when is_integer(n) and n < 0 do
     n = abs(n)
 
     lazy(enum, {0, [], []}, fn f1 ->

lib/elixir/lib/string.ex

@@ -644,15 +644,19 @@ defmodule String do
   end
 
   def upcase(string, :ascii) when is_binary(string) do
-    for <<x <- string>>,
-      do: if(x >= ?a and x <= ?z, do: <<x - 32>>, else: <<x>>),
-      into: ""
+    IO.iodata_to_binary(upcase_ascii(string))
   end
 
   def upcase(string, mode) when mode in @conditional_mappings do
     String.Casing.upcase(string, [], mode)
   end
 
+  defp upcase_ascii(<<char, rest::bits>>) when char >= ?a and char <= ?z,
+    do: [char - 32 | upcase_ascii(rest)]
+
+  defp upcase_ascii(<<char, rest::bits>>), do: [char | upcase_ascii(rest)]
+  defp upcase_ascii(<<>>), do: []
+
   @doc """
   Converts all characters in the given string to lowercase according to `mode`.
 
@@ -700,15 +704,19 @@ defmodule String do
   end
 
   def downcase(string, :ascii) when is_binary(string) do
-    for <<x <- string>>,
-      do: if(x >= ?A and x <= ?Z, do: <<x + 32>>, else: <<x>>),
-      into: ""
+    IO.iodata_to_binary(downcase_ascii(string))
   end
 
   def downcase(string, mode) when mode in @conditional_mappings do
     String.Casing.downcase(string, [], mode)
   end
 
+  defp downcase_ascii(<<char, rest::bits>>) when char >= ?A and char <= ?Z,
+    do: [char + 32 | downcase_ascii(rest)]
+
+  defp downcase_ascii(<<char, rest::bits>>), do: [char | downcase_ascii(rest)]
+  defp downcase_ascii(<<>>), do: []
+
   @doc """
   Converts the first character in the given string to
   uppercase and the remainder to lowercase according to `mode`.

lib/elixir/lib/supervisor.ex

@@ -28,14 +28,17 @@ defmodule Supervisor do
 
         ## Callbacks
 
+        @impl true
         def init(stack) do
           {:ok, stack}
         end
 
+        @impl true
         def handle_call(:pop, _from, [h | t]) do
           {:reply, h, t}
         end
 
+        @impl true
         def handle_cast({:push, h}, t) do
           {:noreply, [h | t]}
         end
@@ -101,13 +104,10 @@ defmodule Supervisor do
 
   ## Start and shutdown
 
-  When the supervisor starts, it traverses all children and retrieves
-  each child specification. It is at this moment `{Stack, [:hello]}`
-  becomes a child specification by calling `Stack.child_spec([:hello])`.
-
-  Then the supervisor starts each child in the order they are defined.
-  This is done by calling the function defined under the `:start` key
-  in the child specification and typically defaults to `start_link/1`.
+  When the supervisor starts, it traverses all child specifications and
+  then starts each child in the order they are defined. This is done by
+  calling the function defined under the `:start` key in the child
+  specification and typically defaults to `start_link/1`.
 
   The `start_link/1` (or a custom) is then called for each child process.
   The `start_link/1` function must return `{:ok, pid}` where `pid` is the
@@ -124,8 +124,8 @@ defmodule Supervisor do
   then awaiting for a time interval for the child process to terminate. This
   interval defaults to 5000 milliseconds. If the child process does not
   terminate in this interval, the supervisor abruptly terminates the child
-  with reason `:brutal_kill`. The shutdown time can be configured in the
-  child specification which is fully detailed in the next section.
+  with reason `:kill`. The shutdown time can be configured in the child
+  specification which is fully detailed in the next section.
 
   If the child process is not trapping exits, it will shutdown immediately
   when it receives the first exit signal. If the child process is trapping
@@ -212,7 +212,8 @@ defmodule Supervisor do
     * `:permanent` - the child process is always restarted.
 
     * `:temporary` - the child process is never restarted, regardless
-      of the supervision strategy.
+      of the supervision strategy: any termination (even abnormal) is
+      considered successful.
 
     * `:transient` - the child process is restarted only if it
       terminates abnormally, i.e., with an exit reason other than
@@ -301,7 +302,9 @@ defmodule Supervisor do
   function.
 
   You may also completely override the `child_spec/1` function in the Stack module
-  and return your own child specification.
+  and return your own child specification. Note there is no guarantee the `child_spec/1`
+  function will be called by the Supervisor process, as other processes may invoke
+  it to retrieve the child specification before reaching the supervisor.
 
   ## Exit reasons and restarts
 
@@ -343,6 +346,7 @@ defmodule Supervisor do
           Supervisor.start_link(__MODULE__, arg, name: __MODULE__)
         end
 
+        @impl true
         def init(_arg) do
           children = [
             {Stack, [:hello]}
@@ -441,6 +445,9 @@ defmodule Supervisor do
       the child processes, i.e., the child processes after the terminated
       one in start order, are terminated. Then the terminated child
       process and the rest of the child processes are restarted.
+      
+  In the above, process termination refers to unsuccessful termination, which
+  is determined by the `:restart` option.
 
   There is also a deprecated strategy called `:simple_one_for_one` which
   has been replaced by the `DynamicSupervisor`. The `:simple_one_for_one`
@@ -459,8 +466,13 @@ defmodule Supervisor do
   defmacro __using__(opts) do
     quote location: :keep, bind_quoted: [opts: opts] do
       import Supervisor.Spec
+      @behaviour Supervisor
 
-      @doc false
+      @doc """
+      Returns a specification to start this module under a supervisor.
+
+      See `Supervisor`.
+      """
       def child_spec(arg) do
         default = %{
           id: __MODULE__,
@@ -472,9 +484,6 @@ defmodule Supervisor do
       end
 
       defoverridable child_spec: 1
-
-      @doc false
-      def init(arg)
     end
   end
 

lib/elixir/lib/task.ex

@@ -173,7 +173,12 @@ defmodule Task do
 
   @type t :: %__MODULE__{}
 
-  @doc false
+  @doc """
+  Returns a specification to start a task under a supervisor.
+
+  See `Supervisor`.
+  """
+  @since "1.5.0"
   def child_spec(arg) do
     %{
       id: Task,
@@ -185,7 +190,12 @@ defmodule Task do
   @doc false
   defmacro __using__(opts) do
     quote location: :keep, bind_quoted: [opts: opts] do
-      @doc false
+      @doc """
+      Returns a specification to start this module under a supervisor.
+
+      See `Supervisor`.
+      """
+      @since "1.5.0"
       def child_spec(arg) do
         default = %{
           id: __MODULE__,
@@ -429,8 +439,8 @@ defmodule Task do
   defp get_info(pid) do
     self_or_name =
       case Process.info(pid, :registered_name) do
-        {:registered_name, []} -> self()
-        {:registered_name, name} -> name
+        {:registered_name, name} when is_atom(name) -> name
+        _ -> pid
       end
 
     {node(), self_or_name}
@@ -638,7 +648,11 @@ defmodule Task do
   @spec yield_many([t], timeout) :: [{t, {:ok, term} | {:exit, term} | nil}]
   def yield_many(tasks, timeout \\ 5000) do
     timeout_ref = make_ref()
-    timer_ref = Process.send_after(self(), timeout_ref, timeout)
+
+    timer_ref =
+      if timeout != :infinity do
+        Process.send_after(self(), timeout_ref, timeout)
+      end
 
     try do
       yield_many(tasks, timeout_ref, :infinity)
@@ -646,7 +660,7 @@ defmodule Task do
       {:noconnection, reason} ->
         exit({reason, {__MODULE__, :yield_many, [tasks, timeout]}})
     after
-      Process.cancel_timer(timer_ref)
+      timer_ref && Process.cancel_timer(timer_ref)
       receive do: (^timeout_ref -> :ok), after: (0 -> :ok)
     end
   end

lib/elixir/lib/task/supervisor.ex

@@ -5,8 +5,18 @@ defmodule Task.Supervisor do
   This module defines a supervisor which can be used to dynamically
   supervise tasks.
 
-  `start_link/1` can be used to start the supervisor. See the `Task`
-  module for more examples.
+  A task supervisor is started with no children, often under a
+  supervisor and a name:
+
+      children = [
+        {Task.Supervisor, name: MyApp.TaskSupervisor}
+      ]
+
+      Supervisor.start_link(strategy: :one_for_one)
+
+  The options given in the child specification are documented in `start_link/1`.
+
+  See the `Task` module for more examples.
 
   ## Name registration
 
@@ -32,17 +42,33 @@ defmodule Task.Supervisor do
   @doc """
   Starts a new supervisor.
 
-  The supported options are:
+  ## Examples
+
+  A task supervisor is typically started under a supervision tree using
+  the tuple format:
+
+      {Task.Supervisor, name: MyApp.TaskSupervisor}
+
+  You can also start it by calling `start_link/1` directly:
+
+      Task.Supervisor.start_link(name: MyApp.TaskSupervisor)
+
+  But this is recommended only for scripting and should be avoided in
+  production code. Generally speaking, processes should always be started
+  inside supervision trees.
+
+  ## Options
 
     * `:name` - used to register a supervisor name, the supported values are
       described under the `Name Registration` section in the `GenServer` module
       docs;
 
-    * `:max_restarts`, `:max_seconds` and `:max_children` - as specified in `DynamicSupervisor`;
+    * `:max_restarts`, `:max_seconds` and `:max_children` - as specified in
+      `DynamicSupervisor`;
 
   This function could also receive `:restart` and `:shutdown` as options
   but those two options have been deprecated and it is now preferred to
-  give them directly to `start_child` and `async` when supported.
+  give them directly to `start_child` and `async`.
   """
   @spec start_link([option]) :: Supervisor.on_start()
   # TODO: Deprecate passing restart and shutdown here on Elixir v1.8.
@@ -291,7 +317,7 @@ defmodule Task.Supervisor do
       or an integer indicating the timeout value, defaults to 5000 milliseconds.
 
   """
-  @spec start_child(Supervisor.supervisor(), (() -> any)) :: {:ok, pid}
+  @spec start_child(Supervisor.supervisor(), (() -> any)) :: DynamicSupervisor.on_start_child()
   def start_child(supervisor, fun, options \\ []) do
     restart = options[:restart]
     shutdown = options[:shutdown]
@@ -305,7 +331,8 @@ defmodule Task.Supervisor do
   Similar to `start_child/2` except the task is specified
   by the given `module`, `fun` and `args`.
   """
-  @spec start_child(Supervisor.supervisor(), module, atom, [term]) :: {:ok, pid}
+  @spec start_child(Supervisor.supervisor(), module, atom, [term]) ::
+          DynamicSupervisor.on_start_child()
   def start_child(supervisor, module, fun, args, options \\ [])
       when is_atom(fun) and is_list(args) do
     restart = options[:restart]
@@ -322,8 +349,8 @@ defmodule Task.Supervisor do
   defp get_info(self) do
     name =
       case Process.info(self, :registered_name) do
-        {:registered_name, []} -> self
-        {:registered_name, name} -> name
+        {:registered_name, name} when is_atom(name) -> name
+        _ -> self
       end
 
     {node(), name}

lib/elixir/lib/uri.ex

@@ -422,11 +422,13 @@ defmodule URI do
 
     parts = Regex.run(regex, string)
 
-    destructure [_, _, scheme, _, authority, path, _, query, _, fragment], parts
+    destructure [_, _, scheme, _, authority, path, query_with_question_mark, _, _, fragment],
+                parts
+
     scheme = nillify(scheme)
     authority = nillify(authority)
     path = nillify(path)
-    query = nillify(query)
+    query = nillify_query(query_with_question_mark)
     {userinfo, host, port} = split_authority(authority)
 
     scheme = scheme && String.downcase(scheme)
@@ -444,6 +446,10 @@ defmodule URI do
     }
   end
 
+  defp nillify_query("?"), do: ""
+  defp nillify_query("?" <> query), do: query
+  defp nillify_query(_other), do: nil
+
   # Split an authority into its userinfo, host and port parts.
   defp split_authority(string) do
     regex = Regex.recompile!(~r/(^(.*)@)?(\[[a-zA-Z0-9:.]*\]|[^:]*)(:(\d*))?/)

lib/elixir/pages/Compatibility and Deprecations.md

@@ -12,7 +12,7 @@ Elixir version | Supported Erlang/OTP versions
 1.3            | 18 - 19
 1.4            | 18 - 19 (and OTP 20 from v1.4.5)
 1.5            | 18 - 20
-1.6            | 19 - 20
+1.6            | 19 - 20 (and OTP 21 from v1.6.6)
 
 While Elixir often adds compatibility to new Erlang versions on released branches, such as support for OTP 20 in v1.4.5, those releases usually contain the minimum changes for Elixir to run without errors. Only the next minor release, in this case v1.5.0, does effectively leverage the new features provided by the latest Erlang release.
 
@@ -35,6 +35,7 @@ Deprecated feature                               | Deprecated in | Replaced by (
 `Enum.partition/2`                               | [v1.6]        | `Enum.split_with/2` (v1.4)
 `Keyword.replace/3`                              | [v1.6]        | Use `Keyword.fetch/2` + `Keyword.put/3` (v1.0)
 `Macro.unescape_tokens/1` and `Macro.unescape_tokens/2` | [v1.6] | Use `Enum.map/2` to traverse over the arguments (v1.0)
+`Module.add_doc/6`                               | [v1.6]        | Use `@doc` instead
 `Map.replace/3`                                  | [v1.6]        | Use `Map.fetch/2` + `Map.put/3` (v1.0)
 `Range.range?/1`                                 | [v1.6]        | Pattern match on `_.._` instead (v1.0)
 `Atom.to_char_list/1`                            | [v1.5]        | `Atom.to_charlist/1` (v1.3)
@@ -63,6 +64,7 @@ EEx: `<%=` in middle and end expressions         | [v1.5]        | Use `<%` (= i
 `Float.to_string/2`                              | [v1.4]        | `:erlang.float_to_binary/2` (OTP 17)
 `HashDict` module                                | [v1.4]        | `Map` (v1.2)
 `HashSet` module                                 | [v1.4]        | `MapSet` (v1.1)
+ Multi-letter aliases in `OptionParser`          | [v1.4]        | Use single-letter aliases (v1.0)
 `Set` module                                     | [v1.4]        | `MapSet` (v1.1)
 `Stream.uniq/2`                                  | [v1.4]        | `Stream.uniq_by/2` (v1.2)
 `IEx.Helpers.import_file/2`                      | [v1.4]        | `IEx.Helpers.import_file_if_available/1` (v1.3)
@@ -70,6 +72,7 @@ EEx: `<%=` in middle and end expressions         | [v1.5]        | Use `<%` (= i
 `Mix.Utils.underscore/1`                         | [v1.4]        | `Macro.underscore/1` (v1.2)
 Variable used as function call                   | [v1.4]        | Use parentheses (v1.0)
 Anonymous functions with no expression after `->` | [v1.4]       | Use an expression or explicitly return `nil` (v1.0)
+Support for making private functions overridable | [v1.4]        | Use public functions (v1.0)
 `Dict` module                                    | [v1.3]        | `Keyword` (v1.0) or `Map` (v1.2)
 `Keyword.size/1`                                 | [v1.3]        | `Kernel.length/1` (v1.0)
 `Map.size/1`                                     | [v1.3]        | `Kernel.map_size/1` (v1.0)

lib/elixir/pages/Guards.md

@@ -109,7 +109,8 @@ Other constructs are `for`, `with`, `try`/`rescue`/`catch`/`else`/, and the `mat
 
 ## Failing guards
 
-Errors in guards do not result in runtime errors, but in guards failing. For example, the `length/1` function only works with lists. If we use it with anything else, a runtime error is raised:
+In guards, when functions would normally raise exceptions, they cause the guard to fail instead.
+For example, the `length/1` function only works with lists. If we use it with anything else, a runtime error is raised:
 
 ```elixir
 iex> length("hello")

lib/elixir/pages/Operators.md

@@ -26,7 +26,7 @@ Operator
 `\|`                                                                                     | Right to left
 `::`                                                                                     | Right to left
 `when`                                                                                   | Right to left
-`<-`, `\\`                                                                               | Left to right
+`<-` `\\`                                                                                | Left to right
 
 ## Comparison operators
 

lib/elixir/src/elixir.app.src

@@ -1,6 +1,6 @@
 {application, elixir,
 [{description, "elixir"},
- {vsn, "1.6.1"},
+ {vsn, "1.6.6"},
  {modules, [
 	elixir
   ]},

lib/elixir/src/elixir_def.erl

@@ -190,12 +190,15 @@ run_with_location_change(File, E, Callback) ->
 def_to_clauses(_Kind, Meta, Args, [], nil, E) ->
   check_args_for_bodiless_clause(Meta, Args, E),
   [];
-def_to_clauses(Kind, Meta, _Args, _Guards, nil, E) ->
-  elixir_errors:form_error(Meta, ?key(E, file), elixir_expand, {missing_option, Kind, [do]});
 def_to_clauses(_Kind, Meta, Args, Guards, [{do, Body}], _E) ->
   [{Meta, Args, Guards, Body}];
-def_to_clauses(Kind, Meta, Args, Guards, Body, _E) ->
-  [{Meta, Args, Guards, {'try', [{origin,  Kind} | Meta], [Body]}}].
+def_to_clauses(Kind, Meta, Args, Guards, Body, E) ->
+  case is_list(Body) andalso lists:keyfind(do, 1, Body) of
+    {do, _} ->
+      [{Meta, Args, Guards, {'try', [{origin,  Kind} | Meta], [Body]}}];
+    false ->
+      elixir_errors:form_error(Meta, ?key(E, file), elixir_expand, {missing_option, Kind, [do]})
+  end.
 
 run_on_definition_callbacks(Kind, Module, Name, Args, Guards, Body, E) ->
   Callbacks = ets:lookup_element(elixir_module:data_table(Module), on_definition, 2),

lib/elixir/src/elixir_dispatch.erl

@@ -397,6 +397,10 @@ deprecation('Elixir.Macro', unescape_tokens, 1) ->
   "instead traverse over the arguments using Enum.map/2";
 deprecation('Elixir.Macro', unescape_tokens, 2) ->
   "instead traverse over the arguments using Enum.map/2";
+deprecation('Elixir.Module', add_doc, 5) ->
+  "use @doc instead";
+deprecation('Elixir.Module', add_doc, 6) ->
+  "use @doc instead";
 deprecation('Elixir.Range', 'range?', 1) ->
   "instead pattern match on left..right";
 deprecation('Elixir.Stream', filter_map, 3) ->

lib/elixir/src/elixir_erl.erl

@@ -293,29 +293,43 @@ add_info_function(Line, Module, Def, Defmacro, Deprecated) ->
   AllowedArgs = lists:map(fun(Atom) -> {atom, Line, Atom} end, AllowedAttrs),
 
   Spec =
-    {attribute, Line, spec, {{'__info__', 1},
-      [{type, Line, 'fun', [
-        {type, Line, product, [
-          {type, Line, union, AllowedArgs}
-        ]},
-        {type, Line, union, [
-          {type, Line, atom, []},
-          {type, Line, list, [
+    %% TODO: Remove this check once we depend only on 20
+    case erlang:system_info(otp_release) of
+      "19" ->
+        {attribute, Line, spec, {{'__info__', 1},
+          [{type, Line, 'fun', [
+            {type, Line, product, [
+              {type, Line, union, AllowedArgs}
+            ]},
             {type, Line, union, [
-              {type, Line, tuple, [
-                {type, Line, atom, []},
-                {type, Line, any, []}
-              ]},
-              {type, Line, tuple, [
-                {type, Line, atom, []},
-                {type, Line, byte, []},
-                {type, Line, integer, []}
+              {type, Line, atom, []},
+              {type, Line, list, [
+                {type, Line, union, [
+                  {type, Line, tuple, [
+                    {type, Line, atom, []},
+                    {type, Line, any, []}
+                  ]},
+                  {type, Line, tuple, [
+                    {type, Line, atom, []},
+                    {type, Line, byte, []},
+                    {type, Line, integer, []}
+                  ]}
+                ]}
               ]}
             ]}
-          ]}
-        ]}
-      ]}]
-    }},
+          ]}]
+        }};
+
+      _ ->
+        {attribute, Line, spec, {{'__info__', 1},
+          [{type, Line, 'fun', [
+            {type, Line, product, [
+              {type, Line, union, AllowedArgs}
+            ]},
+            {type, Line, any, []}
+          ]}]
+        }}
+    end,
 
   Info =
     {function, 0, '__info__', 1, [

lib/elixir/src/elixir_erl_compiler.erl

@@ -63,6 +63,7 @@ handle_file_warning(_, _File, {_Line, erl_lint, {undefined_behaviour_func, _, _}
 handle_file_warning(_, _File, {_Line, erl_lint, {undefined_behaviour, _}}) -> ok;
 handle_file_warning(_, _File, {_Line, erl_lint, {ill_defined_behaviour_callbacks, _}}) -> ok;
 handle_file_warning(_, _File, {_Line, erl_lint, {ill_defined_optional_callbacks, _}}) -> ok;
+handle_file_warning(_, _File, {_Line, erl_lint, {deprecated,{erlang,get_stacktrace,0},_}}) -> ok;
 
 handle_file_warning(_, File, {Line, Module, Desc}) ->
   Message = format_error(Module, Desc),
@@ -86,6 +87,10 @@ format_error(erl_lint, {undefined_function, {F, A}}) ->
 format_error(erl_lint, {spec_fun_undefined, {M, F, A}}) ->
   io_lib:format("spec for undefined function ~ts.~ts/~B", [elixir_aliases:inspect(M), F, A]);
 
+%% Mention the capture operator in make_fun
+format_error(sys_core_fold, {no_effect, {erlang, make_fun, 3}}) ->
+  "the result of the capture operator & (:erlang.make_fun/3) is never used";
+
 %% Make no_effect clauses pretty
 format_error(sys_core_fold, {no_effect, {erlang, F, A}}) ->
   {Fmt, Args} = case erl_internal:comp_op(F, A) of

lib/elixir/src/elixir_erl_for.erl

@@ -21,7 +21,7 @@ translate(Meta, Args, Return, S) ->
 
   case comprehension_expr(TInto, TExpr) of
     {inline, TIntoExpr} ->
-      {build_inline(Ann, TCases, TIntoExpr, TInto, TUniq, SF), SF};
+      build_inline(Ann, TCases, TIntoExpr, TInto, TUniq, SF);
     {into, TIntoExpr} ->
       build_into(Ann, TCases, TIntoExpr, TInto, TUniq, SF)
   end.
@@ -91,28 +91,29 @@ collect_filters([], Acc) ->
 
 build_inline(Ann, Clauses, Expr, Into, Uniq, S) ->
   case not Uniq and lists:all(fun(Clause) -> element(1, Clause) == bin end, Clauses) of
-    true  -> build_comprehension(Ann, Clauses, Expr, Into);
+    true  -> {build_comprehension(Ann, Clauses, Expr, Into), S};
     false -> build_inline_each(Ann, Clauses, Expr, Into, Uniq, S)
   end.
 
 build_inline_each(Ann, Clauses, Expr, false, Uniq, S) ->
   InnerFun = fun(InnerExpr, _InnerAcc) -> InnerExpr end,
-  build_reduce(Ann, Clauses, InnerFun, Expr, {nil, Ann}, Uniq, S);
+  {build_reduce(Ann, Clauses, InnerFun, Expr, {nil, Ann}, Uniq, S), S};
 build_inline_each(Ann, Clauses, Expr, {nil, _} = Into, Uniq, S) ->
   InnerFun = fun(InnerExpr, InnerAcc) -> {cons, Ann, InnerExpr, InnerAcc} end,
   ReduceExpr = build_reduce(Ann, Clauses, InnerFun, Expr, Into, Uniq, S),
-  elixir_erl:remote(Ann, lists, reverse, [ReduceExpr]);
+  {elixir_erl:remote(Ann, lists, reverse, [ReduceExpr]), S};
 build_inline_each(Ann, Clauses, Expr, {bin, _, []}, Uniq, S) ->
   {InnerValue, SV} = build_var(Ann, S),
+  Generated = erl_anno:set_generated(true, Ann),
 
   InnerFun = fun(InnerExpr, InnerAcc) ->
     {'case', Ann, InnerExpr, [
-      {clause, Ann,
+      {clause, Generated,
        [InnerValue],
        [[elixir_erl:remote(Ann, erlang, is_binary, [InnerValue]),
          elixir_erl:remote(Ann, erlang, is_list, [InnerAcc])]],
-       [{cons, Ann, InnerAcc, InnerValue}]},
-      {clause, Ann,
+       [{cons, Generated, InnerAcc, InnerValue}]},
+      {clause, Generated,
        [InnerValue],
        [[elixir_erl:remote(Ann, erlang, is_bitstring, [InnerValue]),
          elixir_erl:remote(Ann, erlang, is_bitstring, [InnerAcc])]],
@@ -120,14 +121,14 @@ build_inline_each(Ann, Clauses, Expr, {bin, _, []}, Uniq, S) ->
           {bin_element, Ann, InnerAcc, default, [bitstring]},
           {bin_element, Ann, InnerValue, default, [bitstring]}
        ]}]},
-      {clause, Ann,
+      {clause, Generated,
        [InnerValue],
        [[elixir_erl:remote(Ann, erlang, is_bitstring, [InnerValue])]],
        [{bin, Ann, [
           {bin_element, Ann, elixir_erl:remote(Ann, erlang, iolist_to_binary, [InnerAcc]), default, [bitstring]},
           {bin_element, Ann, InnerValue, default, [bitstring]}
        ]}]},
-      {clause, Ann,
+      {clause, Generated,
        [InnerValue],
        [],
        [elixir_erl:remote(Ann, erlang, error, [{tuple, Ann, [{atom, Ann, badarg}, InnerValue]}])]}
@@ -135,11 +136,21 @@ build_inline_each(Ann, Clauses, Expr, {bin, _, []}, Uniq, S) ->
   end,
 
   ReduceExpr = build_reduce(Ann, Clauses, InnerFun, Expr, {nil, Ann}, Uniq, SV),
-  elixir_erl:remote(Ann, erlang, iolist_to_binary, [ReduceExpr]).
+
+  {{'case', Ann, ReduceExpr, [
+    {clause, Ann,
+     [InnerValue],
+     [[elixir_erl:remote(Ann, erlang, is_bitstring, [InnerValue])]],
+     [InnerValue]},
+    {clause, Ann,
+     [InnerValue],
+     [],
+     [elixir_erl:remote(Ann, erlang, iolist_to_binary, [InnerValue])]}
+  ]}, SV}.
 
 build_into(Ann, Clauses, Expr, {map, _, []}, Uniq, S) ->
-  ReduceExpr = build_inline_each(Ann, Clauses, Expr, {nil, Ann}, Uniq, S),
-  {elixir_erl:remote(Ann, maps, from_list, [ReduceExpr]), S};
+  {ReduceExpr, SR} = build_inline_each(Ann, Clauses, Expr, {nil, Ann}, Uniq, S),
+  {elixir_erl:remote(Ann, maps, from_list, [ReduceExpr]), SR};
 build_into(Ann, Clauses, Expr, Into, Uniq, S) ->
   {Fun, SF}    = build_var(Ann, S),
   {Acc, SA}    = build_var(Ann, SF),

lib/elixir/src/elixir_parser.yrl

@@ -305,7 +305,7 @@ eoe -> ';' : '$1'.
 eoe -> eol ';' : '$1'.
 
 fn_eoe -> 'fn' : '$1'.
-fn_eoe -> 'fn' eoe : '$1'.
+fn_eoe -> 'fn' eoe : next_is_eol('$1').
 
 do_eoe -> 'do' : '$1'.
 do_eoe -> 'do' eoe : '$1'.
@@ -659,7 +659,7 @@ meta_from_token_with_end_line(Begin, End) ->
   case ?formatter_metadata() of
     true ->
       [{end_line, line_from_location(?location(End))}
-       | meta_from_token(Begin)];
+       | eol_op(?location(Begin)) ++ meta_from_token(Begin)];
     false ->
       meta_from_token(Begin)
   end.

lib/elixir/src/elixir_tokenizer.erl

@@ -182,7 +182,8 @@ tokenize([$~, S, H, H, H | T] = Original, Line, Column, Scope, Tokens) when ?is_
     {ok, NewLine, NewColumn, Parts, Rest} ->
       {Final, Modifiers} = collect_modifiers(Rest, []),
       Token = {sigil, {Line, Column, nil}, S, Parts, Modifiers, <<H, H, H>>},
-      tokenize(Final, NewLine, NewColumn, Scope, [Token | Tokens]);
+      NewColumn2 = NewColumn + length(Modifiers),
+      tokenize(Final, NewLine, NewColumn2, Scope, [Token | Tokens]);
     {error, Reason} ->
       {error, Reason, Original, Tokens}
   end;
@@ -192,7 +193,8 @@ tokenize([$~, S, H | T] = Original, Line, Column, Scope, Tokens) when ?is_sigil(
     {NewLine, NewColumn, Parts, Rest} ->
       {Final, Modifiers} = collect_modifiers(Rest, []),
       Token = {sigil, {Line, Column, nil}, S, Parts, Modifiers, <<H>>},
-      tokenize(Final, NewLine, NewColumn, Scope, [Token | Tokens]);
+      NewColumn2 = NewColumn + length(Modifiers),
+      tokenize(Final, NewLine, NewColumn2, Scope, [Token | Tokens]);
     {error, Reason} ->
       Sigil = [$~, S, H],
       interpolation_error(Reason, Original, Tokens, " (for sigil ~ts starting at line ~B)", [Sigil, Line])
@@ -735,7 +737,7 @@ extract_heredoc(Line0, Column0, Rest0, Marker, Scope) ->
       %% in the final heredoc body three lines below.
       case extract_heredoc_body(Line0, Column0, Marker, [$\n | Rest1], []) of
         {ok, Line1, Body, Rest2, Spaces} ->
-          {ok, Line1, 1, tl(remove_heredoc_spaces(Body, Spaces, Marker, Scope)), Rest2};
+          {ok, Line1, 4 + Spaces, tl(remove_heredoc_spaces(Body, Spaces, Marker, Scope)), Rest2};
         {error, Reason, ErrorLine} ->
           Terminator = [Marker, Marker, Marker],
           {Message, Token} = heredoc_error_message(Reason, Line0, Terminator),

lib/elixir/test/elixir/base_test.exs

@@ -473,7 +473,7 @@ defmodule BaseTest do
     assert :error == decode32("66FF", case: :lower)
   end
 
-  test "decode32!/1,2 error on non-alphabet digit" do
+  test "decode32!/1,2 argument error on non-alphabet digit" do
     assert_raise ArgumentError, "non-alphabet digit found: \")\" (byte 41)", fn ->
       decode32!("MZX)6YTB")
     end
@@ -485,6 +485,10 @@ defmodule BaseTest do
     assert_raise ArgumentError, "non-alphabet digit found: \"M\" (byte 77)", fn ->
       decode32!("MZXW6YTBOI======", case: :lower)
     end
+
+    assert_raise ArgumentError, "non-alphabet digit found: \"0\" (byte 48)", fn ->
+      decode32!("0ZXW6YTB0I======", case: :mixed)
+    end
   end
 
   test "decode32/1 errors on incorrect padding" do

lib/elixir/test/elixir/code_formatter/calls_test.exs

@@ -201,9 +201,23 @@ defmodule Code.Formatter.CallsTest do
       assert_format bad, good, @short_length
     end
 
+    test "with arguments on comma limit" do
+      bad = """
+      import(foo(abc, cde), :next)
+      """
+
+      good = """
+      import(
+        foo(abc, cde),
+        :next
+      )
+      """
+
+      assert_format bad, good, @medium_length
+    end
+
     test "with keyword lists" do
       assert_same "foo(foo: 1, bar: 2)"
-
       assert_same "foo(:hello, foo: 1, bar: 2)"
 
       bad = """
@@ -220,8 +234,14 @@ defmodule Code.Formatter.CallsTest do
 
       assert_format bad, good, @short_length
 
-      # Check it preserves multiline.
-      assert_same good
+      bad = """
+      foo(:hello, foo: 1,
+        bar: 2)
+      """
+
+      assert_format bad, """
+      foo(:hello, foo: 1, bar: 2)
+      """
     end
 
     test "with lists maybe rewritten as keyword lists" do
@@ -235,21 +255,19 @@ defmodule Code.Formatter.CallsTest do
       assert_same "bar = if foo, do: bar, else: baz"
 
       assert_same """
-                  for :one,
-                      :two,
-                      :three,
-                      fn ->
-                        :ok
-                      end
-                  """,
-                  @short_length
+      for :one,
+          :two,
+          :three,
+          fn ->
+            :ok
+          end
+      """
 
       assert_same """
-                  for :one, fn ->
-                    :ok
-                  end
-                  """,
-                  @medium_length
+      for :one, fn ->
+        :ok
+      end
+      """
     end
 
     test "without parens on line limit" do
@@ -263,17 +281,78 @@ defmodule Code.Formatter.CallsTest do
       assert_format bad, good, @short_length
     end
 
+    test "without parens on comma limit" do
+      bad = """
+      import foo(abc, cde), :next
+      """
+
+      good = """
+      import foo(
+               abc,
+               cde
+             ),
+             :next
+      """
+
+      assert_format bad, good, @medium_length
+    end
+
     test "without parens and with keyword lists preserves multiline" do
       assert_same """
       defstruct foo: 1,
                 bar: 2
       """
 
+      assert_same """
+      config :app,
+        foo: 1
+      """
+
       assert_same """
       config :app,
         foo: 1,
         bar: 2
       """
+
+      assert_same """
+      config :app, :key,
+        foo: 1,
+        bar: 2
+      """
+
+      assert_same """
+      config :app,
+             :key,
+             foo: 1,
+             bar: 2
+      """
+
+      bad = """
+      config :app, foo: 1,
+        bar: 2
+      """
+
+      assert_format bad, """
+      config :app,
+        foo: 1,
+        bar: 2
+      """
+    end
+
+    test "without parens and with keyword lists on comma limit" do
+      bad = """
+      import foo(abc, cde), opts: :next
+      """
+
+      good = """
+      import foo(
+               abc,
+               cde
+             ),
+             opts: :next
+      """
+
+      assert_format bad, good, @medium_length
     end
 
     test "without parens and with keyword lists on line limit" do
@@ -434,9 +513,32 @@ defmodule Code.Formatter.CallsTest do
       )
       """
 
+      assert_same """
+      call(
+        :hello,
+        :foo,
+        :bar
+      ) do
+        1 + 2
+      end
+      """
+
       # Doesn't preserve this because only the beginning has a newline
       assert_format "call(\nfoo, bar, baz)", "call(foo, bar, baz)"
 
+      # Doesn't preserve because there are no args
+      bad = """
+      call() do
+        1 + 2
+      end
+      """
+
+      assert_format bad, """
+      call do
+        1 + 2
+      end
+      """
+
       # Doesn't preserve because we have a single argument with next break fits
       bad = """
       call(
@@ -617,6 +719,14 @@ defmodule Code.Formatter.CallsTest do
 
       # Doesn't preserve this because only the beginning has a newline
       assert_format "Remote.call(\nfoo, bar, baz)", "Remote.call(foo, bar, baz)"
+
+      assert_same """
+      Remote.call(
+        :hello,
+        :foo,
+        fn -> :bar end
+      )
+      """
     end
   end
 

lib/elixir/test/elixir/code_formatter/comments_test.exs

@@ -56,36 +56,19 @@ defmodule Code.Formatter.CommentsTest do
 
       assert_format bad, good
 
-      bad = """
-      foo    # this is foo
-      |> bar # this is bar
-      |> baz # this is baz
-      """
-
-      good = """
-      # this is foo
-      # this is bar
-      # this is baz
-      foo
-      |> bar
-      |> baz
-      """
-
-      assert_format bad, good, @short_length
-
       bad = """
       foo   # this is foo
-      | bar # this is bar
-      | baz # this is baz
+      ++ bar # this is bar
+      ++ baz # this is baz
       """
 
       good = """
       # this is foo
       # this is bar
       # this is baz
-      foo
-      | bar
-      | baz
+      foo ++
+        bar ++
+        baz
       """
 
       assert_format bad, good, @short_length
@@ -466,14 +449,15 @@ defmodule Code.Formatter.CommentsTest do
 
       assert_format bad, ~S"""
       # fn
-      # before head
-      # middle head
-      # after head
-      fn hello ->
-        # before body
-        # middle body
-        world
-        # after body
+      fn
+        # before head
+        # middle head
+        hello ->
+          # after head
+          # before body
+          # middle body
+          world
+          # after body
       end
       """
     end
@@ -721,6 +705,118 @@ defmodule Code.Formatter.CommentsTest do
     end
   end
 
+  describe "operators" do
+    test "with comment before, during and after uniform pipelines" do
+      assert_same """
+      foo
+      # |> bar
+      # |> baz
+      |> bat
+      """
+
+      bad = """
+      # before
+      foo    # this is foo
+      |> bar # this is bar
+      |> baz # this is baz
+      # after
+      """
+
+      good = """
+      # before
+      # this is foo
+      foo
+      # this is bar
+      |> bar
+      # this is baz
+      |> baz
+
+      # after
+      """
+
+      assert_format bad, good, @short_length
+    end
+
+    test "with comment before, during and after mixed pipelines" do
+      assert_same """
+      foo
+      # |> bar
+      # |> baz
+      ~> bat
+      """
+
+      bad = """
+      # before
+      foo    # this is foo
+      ~> bar # this is bar
+      <|> baz # this is baz
+      # after
+      """
+
+      good = """
+      # before
+      # this is foo
+      foo
+      # this is bar
+      ~> bar
+      # this is baz
+      <|> baz
+
+      # after
+      """
+
+      assert_format bad, good, @short_length
+    end
+
+    test "with comment before, during and after uniform right" do
+      assert_same """
+      foo
+      # | bar
+      # | baz
+      | bat
+      """
+
+      bad = """
+      # before
+      foo    # this is foo
+      | bar # this is bar
+      | baz # this is baz
+      # after
+      """
+
+      good = """
+      # before
+      # this is foo
+      foo
+      # this is bar
+      | bar
+      # this is baz
+      | baz
+
+      # after
+      """
+
+      assert_format bad, good, @short_length
+    end
+
+    test "with comment before, during and after mixed right" do
+      assert_same """
+      one
+      # when two
+      # when three
+      when four
+           # | five
+           | six
+      """
+    end
+
+    test "handles nodes without meta info" do
+      assert_same "(a -> b) |> (c -> d)"
+      assert_same "(a -> b) when c: d"
+      assert_same "(a -> b) when (c -> d)"
+    end
+  end
+
   describe "containers" do
     test "with comment outside before, during and after" do
       assert_same ~S"""

lib/elixir/test/elixir/code_formatter/containers_test.exs

@@ -178,9 +178,30 @@ defmodule Code.Formatter.ContainersTest do
       assert_same keyword, @short_length
     end
 
+    test "with keyword lists on comma line limit" do
+      bad = """
+      [
+        foooo: 1,
+        barrr: 2
+      ]
+      """
+
+      good = """
+      [
+        foooo:
+          1,
+        barrr: 2
+      ]
+      """
+
+      assert_format bad, good, @short_length
+    end
+
     test "with quoted keyword lists" do
       assert_same ~S(["with spaces": 1])
       assert_same ~S(["one #{two} three": 1])
+      assert_same ~S(["\w": 1, "\\w": 2])
+      assert_same ~S(["Elixir.Foo": 1, "Elixir.Bar": 2])
       assert_format ~S(["Foo": 1, "Bar": 2]), ~S([Foo: 1, Bar: 2])
     end
 
@@ -226,13 +247,19 @@ defmodule Code.Formatter.ContainersTest do
       assert_format "<<1,2,3>>", "<<1, 2, 3>>"
     end
 
-    test "add parens on first and last in case of ambiguity" do
+    test "add parens on first and last in case of binary ambiguity" do
       assert_format "<< <<>> >>", "<<(<<>>)>>"
       assert_format "<< <<>> + <<>> >>", "<<(<<>> + <<>>)>>"
       assert_format "<< 1 + <<>> >>", "<<(1 + <<>>)>>"
       assert_format "<< <<>> + 1 >>", "<<(<<>> + 1)>>"
       assert_format "<< <<>>, <<>>, <<>> >>", "<<(<<>>), <<>>, (<<>>)>>"
-      assert_format "<< <<>>::1, <<>>::2, <<>>::3 >>", "<<(<<>>::1), <<>>::2, <<>>::3>>"
+      assert_format "<< <<>>::1, <<>>::2, <<>>::3 >>", "<<(<<>>)::1, <<>>::2, <<>>::3>>"
+      assert_format "<< <<>>::<<>> >>", "<<(<<>>)::(<<>>)>>"
+    end
+
+    test "add parens on first in case of operator ambiguity" do
+      assert_format "<< ~~~1::8 >>", "<<(~~~1)::8>>"
+      assert_format "<< ~s[foo]::binary >>", "<<(~s[foo])::binary>>"
     end
 
     test "with modifiers" do
@@ -455,7 +482,7 @@ defmodule Code.Formatter.ContainersTest do
       }
       """
 
-      assert_format bad, good, @short_length
+      assert_format bad, good, line_length: 11
     end
 
     test "removes trailing comma" do
@@ -559,7 +586,7 @@ defmodule Code.Formatter.ContainersTest do
       }
       """
 
-      assert_format bad, good, @short_length
+      assert_format bad, good, line_length: 11
     end
 
     test "removes trailing comma" do

lib/elixir/test/elixir/code_formatter/general_test.exs

@@ -187,6 +187,22 @@ defmodule Code.Formatter.GeneralTest do
       assert_same code, @short_length
     end
 
+    test "with a single clause, followed by a newline, and can fit in one line" do
+      assert_same """
+      fn
+        hello -> world
+      end
+      """
+    end
+
+    test "with a single clause, followed by a newline, and can not fit in one line" do
+      assert_same """
+      SomeModule.long_function_name_that_approaches_max_columns(argument, acc, fn
+        %SomeStruct{key: key}, acc -> more_code(key, acc)
+      end)
+      """
+    end
+
     test "with multiple clauses" do
       code = """
       fn
@@ -329,6 +345,46 @@ defmodule Code.Formatter.GeneralTest do
       end
       """
     end
+
+    test "with -> on line limit" do
+      bad = """
+      fn ab, cd ->
+        ab + cd
+      end
+      """
+
+      good = """
+      fn ab,
+         cd ->
+        ab + cd
+      end
+      """
+
+      assert_format bad, good, @short_length
+
+      bad = """
+      fn
+        ab, cd ->
+          1
+        xy, zw ->
+          2
+      end
+      """
+
+      good = """
+      fn
+        ab,
+        cd ->
+          1
+
+        xy,
+        zw ->
+          2
+      end
+      """
+
+      assert_format bad, good, @short_length
+    end
   end
 
   describe "anonymous functions types" do

lib/elixir/test/elixir/code_formatter/integration_test.exs

@@ -348,17 +348,17 @@ defmodule Code.Formatter.IntegrationTest do
   test "no parens keywords at the end of the line" do
     bad = """
     defmodule Mod do
-      defp token_list_downcase(<<char, rest::binary>>, acc) when is_whitespace(char) or is_comma(char), do: token_list_downcase(rest, acc)
-      defp token_list_downcase(some_really_long_arg11, some_really_long_arg22, some_really_long_arg33), do: token_list_downcase(rest, acc)
+      def token_list_downcase(<<char, rest::binary>>, acc) when is_whitespace(char) or is_comma(char), do: token_list_downcase(rest, acc)
+      def token_list_downcase(some_really_long_arg11, some_really_long_arg22, some_really_long_arg33), do: token_list_downcase(rest, acc)
     end
     """
 
     assert_format bad, """
     defmodule Mod do
-      defp token_list_downcase(<<char, rest::binary>>, acc) when is_whitespace(char) or is_comma(char),
+      def token_list_downcase(<<char, rest::binary>>, acc) when is_whitespace(char) or is_comma(char),
         do: token_list_downcase(rest, acc)
 
-      defp token_list_downcase(some_really_long_arg11, some_really_long_arg22, some_really_long_arg33),
+      def token_list_downcase(some_really_long_arg11, some_really_long_arg22, some_really_long_arg33),
         do: token_list_downcase(rest, acc)
     end
     """
@@ -436,8 +436,8 @@ defmodule Code.Formatter.IntegrationTest do
     """
   end
 
-  test "first argument in a call without parens" do
-    bad = """
+  test "first argument in a call without parens with comments" do
+    assert_same """
     with bar ::
            :ok
            | :invalid
@@ -445,28 +445,58 @@ defmodule Code.Formatter.IntegrationTest do
            | :other
     """
 
-    assert_format bad, """
-    # | :unknown
-    with bar ::
-           :ok
-           | :invalid
-           | :other
-    """
-
-    bad = """
+    assert_same """
     @spec bar ::
             :ok
             | :invalid
             # | :unknown
             | :other
     """
+  end
+
+  test "capture with operators" do
+    assert_same """
+    "this works" |> (&String.upcase/1) |> (&String.downcase/1)
+    """
+
+    assert_same """
+    "this works" || (&String.upcase/1) || (&String.downcase/1)
+    """
+
+    assert_same """
+    "this works" == (&String.upcase/1) == (&String.downcase/1)
+    """
+
+    bad = """
+    "this works" = (&String.upcase/1) = (&String.downcase/1)
+    """
 
     assert_format bad, """
-    # | :unknown
-    @spec bar ::
-            :ok
-            | :invalid
-            | :other
+    "this works" = (&String.upcase/1) = &String.downcase/1
+    """
+
+    bad = """
+    "this works" ++ (&String.upcase/1) ++ (&String.downcase/1)
+    """
+
+    assert_format bad, """
+    "this works" ++ (&String.upcase/1) ++ &String.downcase/1
+    """
+
+    bad = """
+    "this works" | (&String.upcase/1) | (&String.downcase/1)
+    """
+
+    assert_format bad, """
+    "this works" | (&String.upcase/1) | &String.downcase/1
+    """
+
+    bad = ~S"""
+    "this works" \\ (&String.upcase/1) \\ (&String.downcase/1)
+    """
+
+    assert_format bad, ~S"""
+    "this works" \\ &String.upcase/1 \\ &String.downcase/1
     """
   end
 end

lib/elixir/test/elixir/code_formatter/operators_test.exs

@@ -812,9 +812,9 @@ defmodule Code.Formatter.OperatorsTest do
     end
 
     test "with operators outside" do
-      assert_same "(& &1) == & &2"
-      assert_same "(& &1) and & &2"
-      assert_same "(&foo/1) and &bar/1"
+      assert_same "(& &1) == (& &2)"
+      assert_same "(& &1) and (& &2)"
+      assert_same "(&foo/1) and (&bar/1)"
       assert_same "[(&IO.puts/1) | &IO.puts/2]"
     end
 

lib/elixir/test/elixir/dynamic_supervisor_test.exs

@@ -340,7 +340,7 @@ defmodule DynamicSupervisorTest do
       assert %{workers: 4, active: 2} = DynamicSupervisor.count_children(pid)
     end
 
-    test "restarting children counted in max_children" do
+    test "restarting on init children counted in max_children" do
       child = current_module_worker([:restart, :error], restart: :permanent)
       opts = [strategy: :one_for_one, max_children: 1, max_restarts: 100_000]
       {:ok, pid} = DynamicSupervisor.start_link(opts)
@@ -353,6 +353,39 @@ defmodule DynamicSupervisorTest do
       assert {:error, :max_children} = DynamicSupervisor.start_child(pid, child)
     end
 
+    test "restarting on exit children counted in max_children" do
+      child = current_module_worker([:ok2], restart: :permanent)
+      opts = [strategy: :one_for_one, max_children: 1, max_restarts: 100_000]
+      {:ok, pid} = DynamicSupervisor.start_link(opts)
+
+      assert {:ok, child_pid} = DynamicSupervisor.start_child(pid, child)
+      assert_kill(child_pid, :shutdown)
+      assert %{workers: 1, active: 1} = DynamicSupervisor.count_children(pid)
+
+      child = current_module_worker([:ok2], restart: :permanent)
+      assert {:error, :max_children} = DynamicSupervisor.start_child(pid, child)
+    end
+
+    test "restarting a child with extra_arguments successfully restarts child" do
+      parent = self()
+
+      fun = fn ->
+        send(parent, :from_child)
+        :timer.sleep(:infinity)
+      end
+
+      {:ok, sup} = DynamicSupervisor.start_link(strategy: :one_for_one, extra_arguments: [fun])
+      child = %{id: Task, restart: :transient, start: {Task, :start_link, []}}
+
+      assert {:ok, child} = DynamicSupervisor.start_child(sup, child)
+      assert is_pid(child)
+      assert_receive :from_child
+      assert %{active: 1, workers: 1} = DynamicSupervisor.count_children(sup)
+      assert_kill(child, :oops)
+      assert_receive :from_child
+      assert %{workers: 1, active: 1} = DynamicSupervisor.count_children(sup)
+    end
+
     test "child is restarted when trying again" do
       child = current_module_worker([:try_again, self()], restart: :permanent)
       {:ok, pid} = DynamicSupervisor.start_link(strategy: :one_for_one, max_restarts: 2)

lib/elixir/test/elixir/fixtures/dialyzer/for_bitstring.ex

@@ -0,0 +1,5 @@
+defmodule Dialyzer.ForBitstring do
+  def foo() do
+    for a <- 1..3, into: "", do: <<a>>
+  end
+end

lib/elixir/test/elixir/kernel/dialyzer_test.exs

@@ -48,10 +48,25 @@ defmodule Kernel.DialyzerTest do
       |> String.to_charlist()
 
     File.cp!(context.base_plt, plt)
-    dialyzer = [analysis_type: :succ_typings, check_plt: false, files_rec: [dir], plts: [plt]]
+    warnings = Map.get(context, :warnings, [])
+
+    dialyzer = [
+      analysis_type: :succ_typings,
+      check_plt: false,
+      files_rec: [dir],
+      plts: [plt],
+      warnings: warnings
+    ]
+
     {:ok, [outdir: dir, dialyzer: dialyzer]}
   end
 
+  @tag warnings: [:specdiffs]
+  test "no warnings on specdiffs", context do
+    copy_beam!(context, Dialyzer.RemoteCall)
+    assert_dialyze_no_warnings!(context)
+  end
+
   test "no warnings on valid remote calls", context do
     copy_beam!(context, Dialyzer.RemoteCall)
     assert_dialyze_no_warnings!(context)
@@ -105,6 +120,11 @@ defmodule Kernel.DialyzerTest do
     assert_dialyze_no_warnings!(context)
   end
 
+  test "no warnings on for comprehensions with bitstrings", context do
+    copy_beam!(context, Dialyzer.ForBitstring)
+    assert_dialyze_no_warnings!(context)
+  end
+
   test "no warnings on for falsy check that always boolean", context do
     copy_beam!(context, Dialyzer.ForBooleanCheck)
     assert_dialyze_no_warnings!(context)

lib/elixir/test/elixir/kernel/docs_test.exs

@@ -178,6 +178,9 @@ defmodule Kernel.DocsTest do
 
           @doc false
           def qux(true), do: false
+
+          Module.add_doc(__MODULE__, __ENV__.line, :def, {:nullary, 0}, [], "add_doc")
+          def nullary, do: 0
         end
       )
 
@@ -191,6 +194,7 @@ defmodule Kernel.DocsTest do
                {{:bar, 1}, _, :def, [{:arg, _, nil}], "Multiple bodiless clause doc"},
                {{:baz, 1}, _, :def, [{:arg, _, nil}], "Multiple bodiless clause and docs"},
                {{:foo, 1}, _, :def, [{:arg, _, nil}], "Function doc"},
+               {{:nullary, 0}, _, :def, [], "add_doc"},
                {{:qux, 1}, _, :def, [{:bool, _, Elixir}], false}
              ] = docs[:docs]
 
@@ -207,4 +211,42 @@ defmodule Kernel.DocsTest do
              ] = docs[:callback_docs]
     end
   end
+
+  test "@impl true doesn't set @doc false if previous implementation has docs" do
+    write_beam(
+      defmodule Docs do
+        defmodule SampleBehaviour do
+          @callback foo(any()) :: any()
+          @callback bar() :: any()
+          @callback baz() :: any()
+        end
+
+        @behaviour SampleBehaviour
+
+        @doc "Foo docs"
+        def foo(nil), do: nil
+
+        @impl true
+        def foo(_), do: false
+
+        @impl true
+        def bar(), do: true
+
+        @doc "Baz docs"
+        @impl true
+        def baz(), do: true
+
+        def fuz(), do: true
+      end
+    )
+
+    docs = Code.get_docs(Docs, :all)
+
+    assert [
+             {{:bar, 0}, _, :def, [], false},
+             {{:baz, 0}, _, :def, [], "Baz docs"},
+             {{:foo, 1}, _, :def, [{:arg1, [], _}], "Foo docs"},
+             {{:fuz, 0}, _, :def, [], nil}
+           ] = docs[:docs]
+  end
 end

lib/elixir/test/elixir/kernel/errors_test.exs

@@ -701,6 +701,12 @@ defmodule Kernel.ErrorsTest do
       def foo(n) when is_number(n)
     end
     '''
+
+    assert_eval_raise CompileError, "nofile:2: missing :do option in \"def\"", '''
+    defmodule Kernel.ErrorsTest.BodyessFunctionWithGuard do
+      def foo(n) when is_number(n), true
+    end
+    '''
   end
 
   test "invalid args for bodyless clause" do

lib/elixir/test/elixir/kernel/expansion_test.exs

@@ -1945,7 +1945,7 @@ defmodule Kernel.ExpansionTest do
       message = ~r"literal <<>> in bitstring supports only type specifiers"
 
       assert_raise CompileError, message, fn ->
-        expand(quote(do: <<(<<"foo">>::32)>>))
+        expand(quote(do: <<(<<"foo">>)::32>>))
       end
     end
 

lib/elixir/test/elixir/kernel/fn_test.exs

@@ -58,7 +58,7 @@ defmodule Kernel.FnTest do
     assert (&List.flatten/1).([[0]]) == [0]
     assert (&List.flatten/1).([[0]]) == [0]
     assert (&List.flatten(&1)).([[0]]) == [0]
-    assert (&List.flatten(&1)) == &List.flatten/1
+    assert (&List.flatten(&1)) == (&List.flatten/1)
   end
 
   test "capture local" do
@@ -77,7 +77,7 @@ defmodule Kernel.FnTest do
     assert (&is_atom/1).(:a)
     assert (&is_atom/1).(:a)
     assert (&is_atom(&1)).(:a)
-    assert (&is_atom(&1)) == &is_atom/1
+    assert (&is_atom(&1)) == (&is_atom/1)
   end
 
   test "capture macro" do
@@ -102,7 +102,7 @@ defmodule Kernel.FnTest do
     mod = List
     assert (&mod.flatten(&1)).([1, [2], 3]) == [1, 2, 3]
     assert (&mod.flatten/1).([1, [2], 3]) == [1, 2, 3]
-    assert (&mod.flatten/1) == &List.flatten/1
+    assert (&mod.flatten/1) == (&List.flatten/1)
   end
 
   test "local partial application" do

lib/elixir/test/elixir/kernel/guard_test.exs

@@ -4,7 +4,7 @@ defmodule Kernel.GuardTest do
   use ExUnit.Case, async: true
 
   describe "Kernel.defguard(p) usage" do
-    defmodule Guards.In.Macros do
+    defmodule GuardsInMacros do
       defguard is_foo(atom) when atom == :foo
 
       defmacro is_compile_time_foo(atom) when is_foo(atom) do
@@ -13,14 +13,15 @@ defmodule Kernel.GuardTest do
     end
 
     test "guards can be used in other macros in the same module" do
-      require Guards.In.Macros
-      assert Guards.In.Macros.is_foo(:foo)
-      refute Guards.In.Macros.is_foo(:baz)
-      assert Guards.In.Macros.is_compile_time_foo(:foo)
+      require GuardsInMacros
+      assert GuardsInMacros.is_foo(:foo)
+      refute GuardsInMacros.is_foo(:baz)
+      assert GuardsInMacros.is_compile_time_foo(:foo)
     end
 
-    defmodule Guards.In.Funs do
+    defmodule GuardsInFuns do
       defguard is_foo(atom) when atom == :foo
+      defguard is_equal(foo, bar) when foo == bar
 
       def is_foobar(atom) when is_foo(atom) do
         is_foo(atom)
@@ -28,12 +29,19 @@ defmodule Kernel.GuardTest do
     end
 
     test "guards can be used in other funs in the same module" do
-      require Guards.In.Funs
-      assert Guards.In.Funs.is_foo(:foo)
-      refute Guards.In.Funs.is_foo(:bar)
+      require GuardsInFuns
+      assert GuardsInFuns.is_foo(:foo)
+      refute GuardsInFuns.is_foo(:bar)
     end
 
-    defmodule Macros.In.Guards do
+    test "guards do not change code evaluation semantics" do
+      require GuardsInFuns
+      x = 1
+      assert GuardsInFuns.is_equal(x = 2, x) == false
+      assert x == 2
+    end
+
+    defmodule MacrosInGuards do
       defmacro is_foo(atom) do
         quote do
           unquote(atom) == :foo
@@ -44,35 +52,35 @@ defmodule Kernel.GuardTest do
     end
 
     test "macros can be used in other guards in the same module" do
-      require Macros.In.Guards
-      assert Macros.In.Guards.is_foobar(:foo)
-      assert Macros.In.Guards.is_foobar(:bar)
-      refute Macros.In.Guards.is_foobar(:baz)
+      require MacrosInGuards
+      assert MacrosInGuards.is_foobar(:foo)
+      assert MacrosInGuards.is_foobar(:bar)
+      refute MacrosInGuards.is_foobar(:baz)
     end
 
-    defmodule Guards.In.Guards do
+    defmodule GuardsInGuards do
       defguard is_foo(atom) when atom == :foo
       defguard is_foobar(atom) when is_foo(atom) or atom == :bar
     end
 
     test "guards can be used in other guards in the same module" do
-      require Guards.In.Guards
-      assert Guards.In.Guards.is_foobar(:foo)
-      assert Guards.In.Guards.is_foobar(:bar)
-      refute Guards.In.Guards.is_foobar(:baz)
+      require GuardsInGuards
+      assert GuardsInGuards.is_foobar(:foo)
+      assert GuardsInGuards.is_foobar(:bar)
+      refute GuardsInGuards.is_foobar(:baz)
     end
 
-    defmodule Default.Args do
+    defmodule DefaultArgs do
       defguard is_divisible(value, remainder \\ 2)
                when is_integer(value) and rem(value, remainder) == 0
     end
 
     test "permits default values in args" do
-      require Default.Args
-      assert Default.Args.is_divisible(2)
-      refute Default.Args.is_divisible(1)
-      assert Default.Args.is_divisible(3, 3)
-      refute Default.Args.is_divisible(3, 4)
+      require DefaultArgs
+      assert DefaultArgs.is_divisible(2)
+      refute DefaultArgs.is_divisible(1)
+      assert DefaultArgs.is_divisible(3, 3)
+      refute DefaultArgs.is_divisible(3, 4)
     end
 
     test "doesn't allow matching in args" do
@@ -101,7 +109,28 @@ defmodule Kernel.GuardTest do
       end
     end
 
-    defmodule Integer.Private.Guards do
+    defmodule GuardFromMacro do
+      defmacro __using__(_) do
+        quote do
+          defguard is_even(value) when is_integer(value) and rem(value, 2) == 0
+        end
+      end
+    end
+
+    test "defguard defines a guard from inside another macro" do
+      defmodule UseGuardFromMacro do
+        use GuardFromMacro
+
+        def assert! do
+          assert is_even(0)
+          refute is_even(1)
+        end
+      end
+
+      UseGuardFromMacro.assert!()
+    end
+
+    defmodule IntegerPrivateGuards do
       defguardp is_even(value) when is_integer(value) and rem(value, 2) == 0
 
       def is_even_and_large?(value) when is_even(value) and value > 100, do: true
@@ -113,19 +142,19 @@ defmodule Kernel.GuardTest do
     end
 
     test "defguardp defines private guards that work inside and outside guard clauses" do
-      assert Integer.Private.Guards.is_even_and_large?(102)
-      refute Integer.Private.Guards.is_even_and_large?(98)
-      refute Integer.Private.Guards.is_even_and_large?(99)
-      refute Integer.Private.Guards.is_even_and_large?(103)
+      assert IntegerPrivateGuards.is_even_and_large?(102)
+      refute IntegerPrivateGuards.is_even_and_large?(98)
+      refute IntegerPrivateGuards.is_even_and_large?(99)
+      refute IntegerPrivateGuards.is_even_and_large?(103)
 
-      assert Integer.Private.Guards.is_even_and_small?(98)
-      refute Integer.Private.Guards.is_even_and_small?(99)
-      refute Integer.Private.Guards.is_even_and_small?(102)
-      refute Integer.Private.Guards.is_even_and_small?(103)
+      assert IntegerPrivateGuards.is_even_and_small?(98)
+      refute IntegerPrivateGuards.is_even_and_small?(99)
+      refute IntegerPrivateGuards.is_even_and_small?(102)
+      refute IntegerPrivateGuards.is_even_and_small?(103)
 
       assert_raise CompileError, ~r"cannot invoke local is_even/1 inside guard", fn ->
-        defmodule Integer.Private.Guard.Utils do
-          import Integer.Private.Guards
+        defmodule IntegerPrivateGuardUtils do
+          import IntegerPrivateGuards
 
           def is_even_and_large?(value) when is_even(value) and value > 100, do: true
           def is_even_and_large?(_), do: false
@@ -133,8 +162,8 @@ defmodule Kernel.GuardTest do
       end
 
       assert_raise CompileError, ~r"undefined function is_even/1", fn ->
-        defmodule Integer.Private.Function.Utils do
-          import Integer.Private.Guards
+        defmodule IntegerPrivateFunctionUtils do
+          import IntegerPrivateGuards
 
           def is_even_and_small?(value) do
             if is_even(value) and value <= 100, do: true, else: false
@@ -365,9 +394,7 @@ defmodule Kernel.GuardTest do
             :erlang.+(:erlang.+(unquote(foo), unquote(bar)), unquote(baz))
           end
           false -> quote do
-            foo = unquote(foo)
-            bar = unquote(bar)
-            baz = unquote(baz)
+            {foo, bar, baz} = {unquote(foo), unquote(bar), unquote(baz)}
             :erlang.+(:erlang.+(foo, bar), baz)
           end
         end
@@ -387,8 +414,7 @@ defmodule Kernel.GuardTest do
             :erlang.+(unquote(foo), unquote(bar))
           end
           false -> quote do
-            foo = unquote(foo)
-            bar = unquote(bar)
+            {foo, bar} = {unquote(foo), unquote(bar)}
             :erlang.+(foo, bar)
           end
         end
@@ -410,9 +436,7 @@ defmodule Kernel.GuardTest do
             end
           false ->
             quote() do
-              foo = unquote(foo)
-              bar = unquote(bar)
-              baz = unquote(baz)
+              {foo, bar, baz} = {unquote(foo), unquote(bar), unquote(baz)}
               :erlang.+(:erlang.+(:erlang.+(foo, foo), bar), baz)
             end
         end

lib/elixir/test/elixir/kernel/lexical_tracker_test.exs

@@ -234,21 +234,21 @@ defmodule Kernel.LexicalTrackerTest do
           NotAModule
           Remote.func()
           R.func()
-          &extract/2
-          &is_record/1
-          &R.func/0
-          &Remote.func/0
-          &Integer.is_even/1
+          _ = &extract/2
+          _ = &is_record/1
+          _ = &R.func/0
+          _ = &Remote.func/0
+          _ = &Integer.is_even/1
           %Macro.Env{}
         end
 
-        &extract/2
-        &is_record/1
-        &R.func/0
-        &Remote.func/0
-        &Integer.is_even/1
+        _ = &extract/2
+        _ = &is_record/1
+        _ = &R.func/0
+        _ = &Remote.func/0
+        _ = &Integer.is_even/1
 
-        &is_record/1; def b(a), do: is_record(a)
+        _ = &is_record/1; def b(a), do: is_record(a)
 
         %Macro.Env{}
 

lib/elixir/test/elixir/kernel/typespec_test.exs

@@ -835,8 +835,12 @@ defmodule Kernel.TypespecTest do
         quote(do: @type(literal_keyword_list_type_key() :: [{binary(), integer()}])),
         quote(do: @type(literal_empty_map() :: %{})),
         quote(do: @type(literal_map_with_key() :: %{key: integer()})),
-        quote(do: @type(literal_map_with_required_key() :: %{required(bitstring()) => integer()})),
-        quote(do: @type(literal_map_with_optional_key() :: %{optional(bitstring()) => integer()})),
+        quote(
+          do: @type(literal_map_with_required_key() :: %{required(bitstring()) => integer()})
+        ),
+        quote(
+          do: @type(literal_map_with_optional_key() :: %{optional(bitstring()) => integer()})
+        ),
         quote(do: @type(literal_struct_all_fields_any_type() :: %SomeStruct{})),
         quote(do: @type(literal_struct_all_fields_key_type() :: %SomeStruct{key: integer()})),
         quote(do: @type(literal_empty_tuple() :: {})),
@@ -863,7 +867,9 @@ defmodule Kernel.TypespecTest do
         quote(do: @type(builtin_list() :: list())),
         quote(do: @type(builtin_nonempty_list() :: nonempty_list())),
         quote(do: @type(builtin_maybe_improper_list() :: maybe_improper_list())),
-        quote(do: @type(builtin_nonempty_maybe_improper_list() :: nonempty_maybe_improper_list())),
+        quote(
+          do: @type(builtin_nonempty_maybe_improper_list() :: nonempty_maybe_improper_list())
+        ),
         quote(do: @type(builtin_mfa() :: mfa())),
         quote(do: @type(builtin_module() :: module())),
         quote(do: @type(builtin_no_return() :: no_return())),

lib/elixir/test/elixir/kernel/warning_test.exs

@@ -1092,13 +1092,13 @@ defmodule Kernel.WarningTest do
   end
 
   test "unused variable in defguard" do
-    capture_err(fn ->
-      Code.eval_string("""
-      defmodule Sample do
-        defguard foo(bar, baz) when bar
-      end
-      """)
-    end) =~ "variable \"baz\" is unused"
+    assert capture_err(fn ->
+             Code.eval_string("""
+             defmodule Sample do
+               defguard foo(bar, baz) when bar
+             end
+             """)
+           end) =~ "variable \"baz\" is unused"
   after
     purge(Sample)
   end
@@ -1129,65 +1129,39 @@ defmodule Kernel.WarningTest do
   end
 
   test "defguard overriding defmacro" do
-    capture_err(fn ->
-      Code.eval_string("""
-      defmodule Sample do
-        defmacro foo(bar), do: bar == :bar
-        defguard foo(baz) when baz == :baz
-      end
-      """)
-    end) =~ "this clause cannot match because a previous clause at line 2 always matches"
+    assert capture_err(fn ->
+             Code.eval_string("""
+             defmodule Sample do
+               defmacro foo(bar), do: bar == :bar
+               defguard foo(baz) when baz == :baz
+             end
+             """)
+           end) =~ "this clause cannot match because a previous clause at line 2 always matches"
   after
     purge(Sample)
   end
 
   test "defmacro overriding defguard" do
-    capture_err(fn ->
-      Code.eval_string("""
-      defmodule Sample do
-        defguard foo(baz) when baz == :baz
-        defmacro foo(bar), do: bar == :bar
-      end
-      """)
-    end) =~ "this clause cannot match because a previous clause at line 2 always matches"
-  after
-    purge(Sample)
-  end
-
-  test "defguardp overriding defmacrop" do
-    capture_err(fn ->
-      Code.eval_string("""
-      defmodule Sample do
-        defmacrop foo(bar), do: bar == :bar
-        defguardp foo(baz) when baz == :baz
-      end
-      """)
-    end) =~ "this clause cannot match because a previous clause at line 2 always matches"
-  after
-    purge(Sample)
-  end
-
-  test "defmacrop overriding defguardp" do
-    capture_err(fn ->
-      Code.eval_string("""
-      defmodule Sample do
-      defguardp foo(baz) when baz == :baz
-        defmacrop foo(bar), do: bar == :bar
-      end
-      """)
-    end) =~ "this clause cannot match because a previous clause at line 2 always matches"
+    assert capture_err(fn ->
+             Code.eval_string("""
+             defmodule Sample do
+               defguard foo(baz) when baz == :baz
+               defmacro foo(bar), do: bar == :bar
+             end
+             """)
+           end) =~ "this clause cannot match because a previous clause at line 2 always matches"
   after
     purge(Sample)
   end
 
   test "defguard needs an implementation" do
-    capture_err(fn ->
-      Code.eval_string("""
-      defmodule Sample do
-        defguard foo(bar)
-      end
-      """)
-    end) =~ "implementation not provided for predefined defguard"
+    assert capture_err(fn ->
+             Code.eval_string("""
+             defmodule Sample do
+               defguard foo(bar)
+             end
+             """)
+           end) =~ "implementation not provided for predefined defmacro foo/1"
   after
     purge(Sample)
   end

lib/elixir/test/elixir/macro_test.exs

@@ -318,16 +318,26 @@ defmodule MacroTest do
     end
 
     test "sigil call" do
-      assert Macro.to_string(quote(do: ~r"123")) == ~s/~r"123"/
-      assert Macro.to_string(quote(do: ~r"123"u)) == ~s/~r"123"u/
-      assert Macro.to_string(quote(do: ~r"\n123")) == ~s/~r"\\\\n123"/
+      assert Macro.to_string(quote(do: ~r"123")) == ~S/~r"123"/
+      assert Macro.to_string(quote(do: ~r"123"u)) == ~S/~r"123"u/
+      assert Macro.to_string(quote(do: ~r"\n123")) == ~S/~r"\\n123"/
 
       assert Macro.to_string(quote(do: ~r"1#{two}3")) == ~S/~r"1#{two}3"/
       assert Macro.to_string(quote(do: ~r"1#{two}3"u)) == ~S/~r"1#{two}3"u/
 
-      assert Macro.to_string(quote(do: ~R"123")) == ~s/~R"123"/
-      assert Macro.to_string(quote(do: ~R"123"u)) == ~s/~R"123"u/
-      assert Macro.to_string(quote(do: ~R"\n123")) == ~s/~R"\\\\n123"/
+      assert Macro.to_string(quote(do: ~R"123")) == ~S/~R"123"/
+      assert Macro.to_string(quote(do: ~R"123"u)) == ~S/~R"123"u/
+      assert Macro.to_string(quote(do: ~R"\n123")) == ~S/~R"\n123"/
+
+      assert Macro.to_string(quote(do: ~S["'(123)'"])) == ~S/~S["'(123)'"]/
+
+      assert Macro.to_string(
+               quote do
+                 ~S"""
+                 "123"
+                 """
+               end
+             ) == ~s[~S"""\n"123"\n"""]
     end
 
     test "tuple call" do
@@ -586,6 +596,7 @@ defmodule MacroTest do
       assert Macro.to_string(quote(do: %Test{foo: 1, bar: 1})) == "%Test{foo: 1, bar: 1}"
       assert Macro.to_string(quote(do: %Test{struct | foo: 2})) == "%Test{struct | foo: 2}"
       assert Macro.to_string(quote(do: %Test{} + 1)) == "%Test{} + 1"
+      assert Macro.to_string(quote(do: %Test{foo(1)} + 2)) == "%Test{foo(1)} + 2"
     end
 
     test "binary operators" do

lib/elixir/test/elixir/node_test.exs

@@ -5,6 +5,7 @@ defmodule NodeTest do
 
   doctest Node
 
+  @tag :unix
   test "start/3 and stop/0" do
     assert Node.stop() == {:error, :not_found}
     assert {:ok, _} = Node.start(:hello, :shortnames, 15000)

lib/elixir/test/elixir/path_test.exs

@@ -36,67 +36,69 @@ defmodule PathTest do
     assert_raise ArgumentError, ~r/null byte/, fn -> Path.wildcard("foo\0bar") end
   end
 
-  if windows?() do
-    describe "Windows" do
-      test "relative/1" do
-        assert Path.relative("C:/usr/local/bin") == "usr/local/bin"
-        assert Path.relative("C:\\usr\\local\\bin") == "usr\\local\\bin"
-        assert Path.relative("C:usr\\local\\bin") == "usr\\local\\bin"
+  describe "Windows" do
+    @describetag :windows
 
-        assert Path.relative("/usr/local/bin") == "usr/local/bin"
-        assert Path.relative("usr/local/bin") == "usr/local/bin"
-        assert Path.relative("../usr/local/bin") == "../usr/local/bin"
-      end
+    test "relative/1" do
+      assert Path.relative("C:/usr/local/bin") == "usr/local/bin"
+      assert Path.relative("C:\\usr\\local\\bin") == "usr\\local\\bin"
+      assert Path.relative("C:usr\\local\\bin") == "usr\\local\\bin"
 
-      test "relative_to/2" do
-        assert Path.relative_to("D:/usr/local/foo", "D:/usr/") == "local/foo"
-        assert Path.relative_to("D:/usr/local/foo", "d:/usr/") == "local/foo"
-        assert Path.relative_to("d:/usr/local/foo", "D:/usr/") == "local/foo"
-        assert Path.relative_to("D:/usr/local/foo", "d:/") == "usr/local/foo"
-        assert Path.relative_to("D:/usr/local/foo", "D:/") == "usr/local/foo"
-        assert Path.relative_to("D:/usr/local/foo", "d:") == "D:/usr/local/foo"
-        assert Path.relative_to("D:/usr/local/foo", "D:") == "D:/usr/local/foo"
-      end
-
-      test "type/1" do
-        assert Path.type("C:/usr/local/bin") == :absolute
-        assert Path.type('C:\\usr\\local\\bin') == :absolute
-        assert Path.type("C:usr\\local\\bin") == :volumerelative
-
-        assert Path.type("/usr/local/bin") == :volumerelative
-        assert Path.type('usr/local/bin') == :relative
-        assert Path.type("../usr/local/bin") == :relative
-      end
-
-      test "split/1" do
-        assert Path.split("C:\\foo\\bar") == ["c:/", "foo", "bar"]
-        assert Path.split("C:/foo/bar") == ["c:/", "foo", "bar"]
-      end
+      assert Path.relative("/usr/local/bin") == "usr/local/bin"
+      assert Path.relative("usr/local/bin") == "usr/local/bin"
+      assert Path.relative("../usr/local/bin") == "../usr/local/bin"
     end
-  else
-    describe "Unix" do
-      test "relative/1" do
-        assert Path.relative("/usr/local/bin") == "usr/local/bin"
-        assert Path.relative("usr/local/bin") == "usr/local/bin"
-        assert Path.relative("../usr/local/bin") == "../usr/local/bin"
-        assert Path.relative("/") == "."
-        assert Path.relative('/') == "."
-        assert Path.relative(['/usr', ?/, "local/bin"]) == "usr/local/bin"
-      end
 
-      test "type/1" do
-        assert Path.type("/usr/local/bin") == :absolute
-        assert Path.type("usr/local/bin") == :relative
-        assert Path.type("../usr/local/bin") == :relative
+    test "relative_to/2" do
+      assert Path.relative_to("D:/usr/local/foo", "D:/usr/") == "local/foo"
+      assert Path.relative_to("D:/usr/local/foo", "d:/usr/") == "local/foo"
+      assert Path.relative_to("d:/usr/local/foo", "D:/usr/") == "local/foo"
+      assert Path.relative_to("D:/usr/local/foo", "d:/") == "usr/local/foo"
+      assert Path.relative_to("D:/usr/local/foo", "D:/") == "usr/local/foo"
+      assert Path.relative_to("D:/usr/local/foo", "d:") == "D:/usr/local/foo"
+      assert Path.relative_to("D:/usr/local/foo", "D:") == "D:/usr/local/foo"
+    end
 
-        assert Path.type('/usr/local/bin') == :absolute
-        assert Path.type('usr/local/bin') == :relative
-        assert Path.type('../usr/local/bin') == :relative
+    test "type/1" do
+      assert Path.type("C:/usr/local/bin") == :absolute
+      assert Path.type('C:\\usr\\local\\bin') == :absolute
+      assert Path.type("C:usr\\local\\bin") == :volumerelative
 
-        assert Path.type(['/usr/', 'local/bin']) == :absolute
-        assert Path.type(['usr/', 'local/bin']) == :relative
-        assert Path.type(['../usr', '/local/bin']) == :relative
-      end
+      assert Path.type("/usr/local/bin") == :volumerelative
+      assert Path.type('usr/local/bin') == :relative
+      assert Path.type("../usr/local/bin") == :relative
+    end
+
+    test "split/1" do
+      assert Path.split("C:\\foo\\bar") == ["c:/", "foo", "bar"]
+      assert Path.split("C:/foo/bar") == ["c:/", "foo", "bar"]
+    end
+  end
+
+  describe "Unix" do
+    @describetag :unix
+
+    test "relative/1" do
+      assert Path.relative("/usr/local/bin") == "usr/local/bin"
+      assert Path.relative("usr/local/bin") == "usr/local/bin"
+      assert Path.relative("../usr/local/bin") == "../usr/local/bin"
+      assert Path.relative("/") == "."
+      assert Path.relative('/') == "."
+      assert Path.relative(['/usr', ?/, "local/bin"]) == "usr/local/bin"
+    end
+
+    test "type/1" do
+      assert Path.type("/usr/local/bin") == :absolute
+      assert Path.type("usr/local/bin") == :relative
+      assert Path.type("../usr/local/bin") == :relative
+
+      assert Path.type('/usr/local/bin') == :absolute
+      assert Path.type('usr/local/bin') == :relative
+      assert Path.type('../usr/local/bin') == :relative
+
+      assert Path.type(['/usr/', 'local/bin']) == :absolute
+      assert Path.type(['usr/', 'local/bin']) == :relative
+      assert Path.type(['../usr', '/local/bin']) == :relative
     end
   end
 

lib/elixir/test/elixir/system_test.exs

@@ -92,12 +92,14 @@ defmodule SystemTest do
     end
   end
 
-  if windows?() do
-    test "cmd/2 win" do
+  describe "Windows" do
+    @describetag :windows
+
+    test "cmd/2" do
       assert {"hello\r\n", 0} = System.cmd("cmd", ~w[/c echo hello])
     end
 
-    test "cmd/3 (with options) win" do
+    test "cmd/3 (with options)" do
       assert {["hello\r\n"], 0} =
                System.cmd(
                  "cmd",
@@ -113,7 +115,7 @@ defmodule SystemTest do
 
     @echo "echo-elixir-test"
 
-    test "cmd/2 with absolute and relative Windows paths" do
+    test "cmd/2 with absolute and relative paths" do
       echo = tmp_path(@echo)
       File.mkdir_p!(Path.dirname(echo))
       File.cp!(System.find_executable("cmd"), echo)
@@ -132,12 +134,16 @@ defmodule SystemTest do
     after
       File.rm_rf!(Path.dirname(tmp_path(@echo)))
     end
-  else
-    test "cmd/2 unix" do
+  end
+
+  describe "Unix" do
+    @describetag :unix
+
+    test "cmd/2" do
       assert {"hello\n", 0} = System.cmd("echo", ["hello"])
     end
 
-    test "cmd/3 (with options) unix" do
+    test "cmd/3 (with options)" do
       opts = [
         into: [],
         cd: System.cwd!(),
@@ -152,7 +158,7 @@ defmodule SystemTest do
 
     @echo "echo-elixir-test"
 
-    test "cmd/2 with absolute and relative Unix paths" do
+    test "cmd/2 with absolute and relative paths" do
       echo = tmp_path(@echo)
       File.mkdir_p!(Path.dirname(echo))
       File.cp!(System.find_executable("echo"), echo)

lib/elixir/test/elixir/task_test.exs

@@ -323,6 +323,19 @@ defmodule TaskTest do
       assert Task.yield_many([task1, task2, task3], 0) ==
                [{task1, {:ok, :result}}, {task2, nil}, {task3, {:exit, :normal}}]
     end
+
+    test "returns results on infinity timeout" do
+      task1 = %Task{ref: make_ref(), owner: self()}
+      task2 = %Task{ref: make_ref(), owner: self()}
+      task3 = %Task{ref: make_ref(), owner: self()}
+
+      send(self(), {task1.ref, :result})
+      send(self(), {task2.ref, :result})
+      send(self(), {:DOWN, task3.ref, :process, self(), :normal})
+
+      assert Task.yield_many([task1, task2, task3], :infinity) ==
+               [{task1, {:ok, :result}}, {task2, {:ok, :result}}, {task3, {:exit, :normal}}]
+    end
   end
 
   describe "shutdown/2" do

lib/elixir/test/elixir/test_helper.exs

@@ -1,6 +1,3 @@
-assert_timeout = String.to_integer(System.get_env("ELIXIR_ASSERT_TIMEOUT") || "500")
-ExUnit.start(trace: "--trace" in System.argv(), assert_receive_timeout: assert_timeout)
-
 # Beam files compiled on demand
 path = Path.expand("../../tmp/beams", __DIR__)
 File.rm_rf!(path)
@@ -87,3 +84,12 @@ defmodule CodeFormatterHelpers do
     end
   end
 end
+
+assert_timeout = String.to_integer(System.get_env("ELIXIR_ASSERT_TIMEOUT") || "500")
+exclude = if PathHelpers.windows?(), do: [unix: true], else: [windows: true]
+
+ExUnit.start(
+  trace: "--trace" in System.argv(),
+  assert_receive_timeout: assert_timeout,
+  exclude: exclude
+)

lib/elixir/test/elixir/uri_test.exs

@@ -306,6 +306,10 @@ defmodule URITest do
       assert URI.parse("http://example.com/#").fragment == ""
       assert URI.parse("http://example.com/test#").fragment == ""
     end
+
+    test "preserves an empty query" do
+      assert URI.parse("http://foo.com/?").query == ""
+    end
   end
 
   test "default_port/1,2" do

lib/elixir/test/erlang/tokenizer_test.erl

@@ -151,6 +151,10 @@ string_test() ->
   [{bin_string, {1, 1, nil}, [<<"f\"">>]}] = tokenize("\"f\\\"\""),
   [{list_string, {1, 1, nil}, [<<"foo">>]}] = tokenize("'foo'").
 
+heredoc_test() ->
+  [{bin_heredoc, {1, 1, nil}, [<<"heredoc\n">>]}] = tokenize("\"\"\"\nheredoc\n\"\"\""),
+  [{bin_heredoc, {1, 1, nil}, [<<"heredoc\n">>]}, {';', {3, 5, 0}}] = tokenize("\"\"\"\n heredoc\n \"\"\";").
+
 empty_string_test() ->
   [{bin_string, {1, 1, nil}, [<<>>]}] = tokenize("\"\""),
   [{list_string, {1, 1, nil}, [<<>>]}] = tokenize("''").
@@ -204,6 +208,12 @@ sigil_terminator_test() ->
   [{sigil, {1, 1, nil}, 114, [<<"foo">>], [], <<"/">>}] = tokenize("~r/foo/"),
   [{sigil, {1, 1, nil}, 114, [<<"foo">>], [], <<"[">>}] = tokenize("~r[foo]"),
   [{sigil, {1, 1, nil}, 114, [<<"foo">>], [], <<"\"">>}] = tokenize("~r\"foo\""),
+  [{sigil, {1, 1, nil}, 114, [<<"foo">>], [], <<"/">>},
+   {comp_op, {1, 9, nil}, '=='},
+   {identifier, {1, 12, nil}, bar}] = tokenize("~r/foo/ == bar"),
+  [{sigil, {1, 1, nil}, 114, [<<"foo">>], "iu", <<"/">>},
+   {comp_op, {1, 11, nil}, '=='},
+   {identifier, {1, 14, nil}, bar}] = tokenize("~r/foo/iu == bar"),
   [{sigil, {1, 1, nil}, 83, [<<"sigil heredoc\n">>], [], <<"\"\"\"">>}] = tokenize("~S\"\"\"\nsigil heredoc\n\"\"\""),
   [{sigil, {1, 1, nil}, 83, [<<"sigil heredoc\n">>], [], <<"'''">>}] = tokenize("~S'''\nsigil heredoc\n'''").
 

lib/ex_unit/lib/ex_unit/callbacks.ex

@@ -5,15 +5,25 @@ defmodule ExUnit.Callbacks do
   This module defines both `setup` and `setup_all` callbacks, as well as
   the `on_exit/2`, `start_supervised/2` and `stop_supervised/1` functions.
 
-  The setup callbacks are defined via macros and each one can optionally
-  receive a map with metadata, usually referred to as `context`. The context
-  to be used in the tests can be optionally extended by the callbacks by
-  returning a properly structured value (see below).
+  The setup callbacks are defined via macros and each one can
+  optionally receive a map with test state and metadata, usually
+  referred to as `context`. The context to be used in the tests can be
+  optionally extended by the setup callbacks by returning a properly
+  structured value (see below).
 
   The `setup_all` callbacks are invoked only once per module, before any
   test runs. All `setup` callbacks are run before each test. No callback
   runs if the test case has no tests or all tests have been filtered out.
 
+  Both `setup` and `setup_all` can be defined by a block, by passing
+  an atom naming a unary function, or by passing a list of such
+  atoms. Both can opt to receive the current context by specifying it
+  as parameter if defined by a block. Functions used to define a test
+  setup must accept the context as single argument.
+
+  A test module can define mutiple `setup` and `setup_all` callbacks,
+  and they are invoked in order of appearance.
+
   `start_supervised/2` is used to start processes under a supervisor. The
   supervisor is linked to the current test process. The supervisor as well
   as all child processes are guaranteed to terminate before any `on_exit/2`
@@ -43,13 +53,10 @@ defmodule ExUnit.Callbacks do
 
   ## Context
 
-  If `setup_all` returns a keyword list, a map, or `{:ok, keywords | map}`,
-  the keyword list/map will be merged into the current context and will be
-  available in all subsequent `setup_all`, `setup`, and the `test` itself.
-
-  Similarly, if `setup` returns a keyword list, map, or `{:ok, keywords | map}`,
-  the returned keyword list/map will be merged into the current context and will
-  be available in all subsequent `setup` and the `test` itself.
+  If `setup_all` or `setup` return a keyword list, a map, or `{:ok,
+  keywords | map}`, the keyword list or map will be merged into the
+  current context and will be available in all subsequent `setup_all`,
+  `setup`, and the `test` itself.
 
   Returning `:ok` leaves the context unchanged (both in `setup` and `setup_all`
   callbacks).
@@ -132,10 +139,25 @@ defmodule ExUnit.Callbacks do
   @doc """
   Defines a callback to be run before each test in a case.
 
+  Pass a block or name of a unary function as atom, or list of such
+  atoms.
+
+  Can return values to be merged into the context, to set up state for
+  tests. See section Context above for details.
+
   ## Examples
 
+      def clean_up_tmp_directory(context) do
+        # perform setup
+        :ok
+      end
+
       setup :clean_up_tmp_directory
 
+      setup do
+        [conn: Plug.Conn.build_conn()]
+      end
+
   """
   defmacro setup(block) do
     if Keyword.keyword?(block) do
@@ -151,6 +173,12 @@ defmodule ExUnit.Callbacks do
   @doc """
   Defines a callback to be run before each test in a case.
 
+  Pass a block or name of a unary function as atom, or list of such
+  atoms.
+
+  Can return values to be merged into the context, to set up state for
+  tests. See section Context above for details.
+
   ## Examples
 
       setup context do
@@ -173,10 +201,25 @@ defmodule ExUnit.Callbacks do
   @doc """
   Defines a callback to be run before all tests in a case.
 
+  Pass a block or name of a unary function as atom, or list of such
+  atoms.
+
+  Can return values to be merged into the context, to set up state for
+  tests. See section Context above for details.
+
   ## Examples
 
+      def clean_up_tmp_directory(context) do
+        # perform setup
+        :ok
+      end
+
       setup_all :clean_up_tmp_directory
 
+      setup_all do
+        [conn: Plug.Conn.build_conn()]
+      end
+
   """
   defmacro setup_all(block) do
     if Keyword.keyword?(block) do
@@ -196,6 +239,12 @@ defmodule ExUnit.Callbacks do
   @doc """
   Defines a callback to be run before all tests in a case.
 
+  Pass a block or name of a unary function as atom, or list of such
+  atoms.
+
+  Can return values to be merged into the context, to set up state for
+  tests. See section Context above for details.
+
   ## Examples
 
       setup_all context do

lib/ex_unit/lib/ex_unit/capture_log.ex

@@ -70,7 +70,7 @@ defmodule ExUnit.CaptureLog do
     {:ok, string_io} = StringIO.open("")
 
     try do
-      _ = :gen_event.which_handlers(:error_logger)
+      _ = Process.whereis(:error_logger) && :gen_event.which_handlers(:error_logger)
       :ok = add_capture(string_io, opts)
       ref = ExUnit.CaptureServer.log_capture_on(self())
 

lib/iex/lib/iex.ex

@@ -643,7 +643,7 @@ defmodule IEx do
   By default, the number of stops in a breakpoint is 1. Any follow-up
   call won't stop the code execution unless another breakpoint is set.
 
-  Alternatively, the number of be increased by passing the `stops`
+  Alternatively, the number of stops can be increased by passing the `stops`
   argument. `IEx.Helpers.reset_break/1` and `IEx.Helpers.reset_break/3`
   can be used to reset the number back to zero. Note the module remains
   "instrumented" even after all stops on all breakpoints are consumed.

lib/iex/lib/iex/autocomplete.ex

@@ -55,10 +55,10 @@ defmodule IEx.Autocomplete do
 
   @doc false
   def exports(mod) do
-    if function_exported?(mod, :__info__, 1) do
+    if Code.ensure_loaded?(mod) and function_exported?(mod, :__info__, 1) do
       mod.__info__(:macros) ++ (mod.__info__(:functions) -- [__info__: 1])
     else
-      mod.module_info(:exports)
+      mod.module_info(:exports) -- [module_info: 0, module_info: 1]
     end
   end
 

lib/iex/lib/iex/helpers.ex

@@ -240,10 +240,15 @@ defmodule IEx.Helpers do
 
       subl path/to/file:line
 
-  Custom editors are supported by using the __FILE__ and __LINE__
-  notations. For example, vi/vim users can set `ELIXIR_EDITOR` to:
+  It is important that you choose an editor command that does
+  not block nor that attempts to run an editor directly in the
+  terminal. Command-line based editors likely extra configuration
+  so they open up the given file and line in a separate window.
 
-      ELIXIR_EDITOR="vi +__LINE__ __FILE__"
+  Custom editors are supported by using the __FILE__ and __LINE__
+  notations, for example:
+
+      ELIXIR_EDITOR="my_editor +__LINE__ __FILE__"
 
   and Elixir will properly interpolate values.
 
@@ -251,8 +256,8 @@ defmodule IEx.Helpers do
   `ELIXIR_EDITOR` can be set "echo" if you prefer to display the
   location rather than opening it.
 
-  Keep in mind the location may not exist when opening
-  precompiled source code, such as Elixir itself.
+  Keep in mind the location may not exist when opening precompiled
+  source code.
 
   ## Examples
 

lib/iex/test/iex/helpers_test.exs

@@ -195,30 +195,38 @@ defmodule IEx.HelpersTest do
                ~r/#{@elixir_erl}:\d+$/
     end
 
-    test "opens OTP lists module" do
-      assert capture_iex("open(:lists)") |> maybe_trim_quotes() =~ ~r/#{@lists_erl}:\d+$/
+    # Some installations remove the source file once Erlang is compiled. See #7348.
+    if File.regular?(@lists_erl) do
+      test "opens OTP lists module" do
+        assert capture_iex("open(:lists)") |> maybe_trim_quotes() =~ ~r/#{@lists_erl}:\d+$/
+      end
+
+      test "opens OTP lists module.function" do
+        assert capture_iex("open(:lists.reverse)") |> maybe_trim_quotes() =~
+                 ~r/#{@lists_erl}:\d+$/
+      end
+
+      test "opens OTP lists module.function/arity" do
+        assert capture_iex("open(:lists.reverse/1)") |> maybe_trim_quotes() =~
+                 ~r/#{@lists_erl}:\d+$/
+      end
     end
 
-    test "opens OTP lists module.function" do
-      assert capture_iex("open(:lists.reverse)") |> maybe_trim_quotes() =~ ~r/#{@lists_erl}:\d+$/
-    end
+    # Some installations remove the source file once Erlang is compiled. See #7348.
+    if File.regular?(@httpc_erl) do
+      test "opens OTP httpc module" do
+        assert capture_iex("open(:httpc)") |> maybe_trim_quotes() =~ ~r/#{@httpc_erl}:\d+$/
+      end
 
-    test "opens OTP lists module.function/arity" do
-      assert capture_iex("open(:lists.reverse/1)") |> maybe_trim_quotes() =~
-               ~r/#{@lists_erl}:\d+$/
-    end
+      test "opens OTP httpc module.function" do
+        assert capture_iex("open(:httpc.request)") |> maybe_trim_quotes() =~
+                 ~r/#{@httpc_erl}:\d+$/
+      end
 
-    test "opens OTP httpc module" do
-      assert capture_iex("open(:httpc)") |> maybe_trim_quotes() =~ ~r/#{@httpc_erl}:\d+$/
-    end
-
-    test "opens OTP httpc module.function" do
-      assert capture_iex("open(:httpc.request)") |> maybe_trim_quotes() =~ ~r/#{@httpc_erl}:\d+$/
-    end
-
-    test "opens OTP httpc module.function/arity" do
-      assert capture_iex("open(:httpc.request/1)") |> maybe_trim_quotes() =~
-               ~r/#{@httpc_erl}:\d+$/
+      test "opens OTP httpc module.function/arity" do
+        assert capture_iex("open(:httpc.request/1)") |> maybe_trim_quotes() =~
+                 ~r/#{@httpc_erl}:\d+$/
+      end
     end
 
     test "errors OTP preloaded module" do

lib/logger/lib/logger.ex

@@ -16,8 +16,10 @@ defmodule Logger do
       performant when required but also apply backpressure
       when under stress.
 
-    * Wraps OTP's [`:error_logger`](http://erlang.org/doc/man/error_logger.html)
-      to prevent it from overflowing.
+    * Plugs into Erlang's [`:logger`](http://erlang.org/doc/man/logger.html)
+      (from Erlang/OTP 21) to convert terms to Elixir syntax or wraps
+      Erlang's [`:error_logger`](http://erlang.org/doc/man/error_logger.html)
+      in earlier Erlang/OTP versions to prevent it from overflowing.
 
   Logging is useful for tracking when an event of interest happens in your
   system. For example, it may be helpful to log whenever a user is deleted.
@@ -62,8 +64,8 @@ defmodule Logger do
     * Runtime configuration - can be set before the `:logger`
       application is started, but may be changed during runtime
 
-    * Error logger configuration - configuration for the
-      wrapper around OTP's [`:error_logger`](http://erlang.org/doc/man/error_logger.html)
+    * Erlang configuration - options that handle integration with
+      Erlang's logging facilities
 
   ### Application configuration
 
@@ -152,19 +154,24 @@ defmodule Logger do
   ### Error logger configuration
 
   The following configuration applies to `Logger`'s wrapper around
-  OTP's [`:error_logger`](http://erlang.org/doc/man/error_logger.html).
-  All the configurations below must be set before the `:logger` application starts.
+  Erlang's logging functionalities. All the configurations below must
+  be set before the `:logger` application starts.
 
     * `:handle_otp_reports` - redirects OTP reports to `Logger` so
-      they are formatted in Elixir terms. This uninstalls OTP's
-      logger that prints terms to terminal. Defaults to `true`.
+      they are formatted in Elixir terms. This effectively disables
+      Erlang standard logger. Defaults to `true`.
 
     * `:handle_sasl_reports` - redirects supervisor, crash and
       progress reports to `Logger` so they are formatted in Elixir
       terms. Your application must guarantee `:sasl` is started before
       `:logger`. This means you may see some initial reports written
-      in Erlang syntax until the Logger application kicks in and
-      uninstalls SASL's logger in favor of its own. Defaults to `false`.
+      in Erlang syntax until the Logger application kicks in.
+      Defaults to `false`.
+
+  From Erlang/OTP 21, `:handle_sasl_reports` only has an effect if
+  `:handle_otp_reports` is true.
+
+  The following configurations apply only for Erlang/OTP 20 and earlier:
 
     * `:discard_threshold_for_error_logger` - if `:error_logger` has more than
       `discard_threshold` messages in its inbox, messages will be dropped
@@ -176,19 +183,17 @@ defmodule Logger do
       350 (0.75 * threshold) entries and 50 (0.1 * theshold) messages will
       be processed before the threshold is checked once again.
 
-  For example, to configure `Logger` to redirect all
-  [`:error_logger`](http://erlang.org/doc/man/error_logger.html) messages
-  using a `config/config.exs` file:
+  For example, to configure `Logger` to redirect all Erlang messages using a
+  `config/config.exs` file:
 
       config :logger,
         handle_otp_reports: true,
         handle_sasl_reports: true
 
-  Furthermore, `Logger` allows messages sent by OTP's `:error_logger`
-  to be translated into an Elixir format via translators. Translators
-  can be dynamically added at any time with the `add_translator/1`
-  and `remove_translator/1` APIs. Check `Logger.Translator` for more
-  information.
+  Furthermore, `Logger` allows messages sent by Erlang to be translated
+  into an Elixir format via translators. Translators can be added at any
+  time with the `add_translator/1` and `remove_translator/1` APIs. Check
+  `Logger.Translator` for more information.
 
   ## Backends
 
@@ -506,7 +511,8 @@ defmodule Logger do
     :sync_threshold,
     :truncate,
     :level,
-    :utc_log
+    :utc_log,
+    :discard_threshold
   ]
   @spec configure(keyword) :: :ok
   def configure(options) do
@@ -522,7 +528,7 @@ defmodule Logger do
   """
   @spec flush :: :ok
   def flush do
-    _ = :gen_event.which_handlers(:error_logger)
+    _ = Process.whereis(:error_logger) && :gen_event.which_handlers(:error_logger)
     :gen_event.sync_notify(Logger, :flush)
   end
 
@@ -532,8 +538,7 @@ defmodule Logger do
   ## Options
 
     * `:flush` - when `true`, guarantees all messages currently sent
-      to both Logger and OTP's [`:error_logger`](http://erlang.org/doc/man/error_logger.html)
-      are processed before the backend is added
+      to `Logger` are processed before the backend is added
 
   """
   @spec add_backend(atom, keyword) :: Supervisor.on_start_child()
@@ -559,8 +564,7 @@ defmodule Logger do
   ## Options
 
     * `:flush` - when `true`, guarantees all messages currently sent
-      to both Logger and OTP's [`:error_logger`](http://erlang.org/doc/man/error_logger.html)
-      are processed before the backend is removed
+      to `Logger` are processed before the backend is removed
 
   """
   @spec remove_backend(atom, keyword) :: :ok | {:error, term}
@@ -609,14 +613,17 @@ defmodule Logger do
       when level in @levels and is_list(metadata) do
     case __metadata__() do
       {true, pdict} ->
-        %{mode: mode, truncate: truncate, level: min_level, utc_log: utc_log?} =
-          Logger.Config.__data__()
+        %{
+          mode: mode,
+          truncate: truncate,
+          level: min_level,
+          utc_log: utc_log?
+        } = Logger.Config.__data__()
 
-        if compare_levels(level, min_level) != :lt and mode != :discard do
-          metadata = [pid: self()] ++ Keyword.merge(pdict, metadata)
-          {message, metadata} = normalize_message(chardata_or_fun, metadata)
+        with true <- compare_levels(level, min_level) != :lt and mode != :discard,
+             metadata = [pid: self()] ++ Keyword.merge(pdict, metadata),
+             {message, metadata} <- normalize_message(chardata_or_fun, metadata) do
           truncated = truncate(message, truncate)
-
           tuple = {Logger, truncated, Logger.Utils.timestamp(utc_log?), metadata}
 
           try do
@@ -628,7 +635,7 @@ defmodule Logger do
             :exit, reason -> {:error, reason}
           end
         else
-          :ok
+          _ -> :ok
         end
 
       {false, _} ->
@@ -754,15 +761,17 @@ defmodule Logger do
   end
 
   defp normalize_message(fun, metadata) when is_function(fun, 0) do
-    normalize_message(fun.(), metadata)
+    case fun.() do
+      {message, fun_metadata} -> {message, Keyword.merge(metadata, fun_metadata)}
+      :skip -> :skip
+      message -> {message, metadata}
+    end
   end
 
-  defp normalize_message({message, fun_metadata}, metadata) when is_list(fun_metadata) do
-    {message, Keyword.merge(metadata, fun_metadata)}
+  defp normalize_message(message, metadata) do
+    {message, metadata}
   end
 
-  defp normalize_message(message, metadata), do: {message, metadata}
-
   defp truncate(data, n) when is_list(data) or is_binary(data), do: Logger.Utils.truncate(data, n)
   defp truncate(data, n), do: Logger.Utils.truncate(to_string(data), n)
 

lib/logger/lib/logger/app.ex

@@ -7,8 +7,15 @@ defmodule Logger.App do
   def start(_type, _args) do
     otp_reports? = Application.get_env(:logger, :handle_otp_reports)
     sasl_reports? = Application.get_env(:logger, :handle_sasl_reports)
-    threshold = Application.get_env(:logger, :discard_threshold_for_error_logger)
-    error_handler = {:error_logger, Logger.ErrorHandler, {otp_reports?, sasl_reports?, threshold}}
+
+    otp_children =
+      if otp_logger?() do
+        []
+      else
+        threshold = Application.get_env(:logger, :discard_threshold_for_error_logger)
+        arg = {:error_logger, Logger.ErrorHandler, {otp_reports?, sasl_reports?, threshold}}
+        [%{id: Logger.ErrorHandler, start: {Logger.Watcher, :start_link, [arg]}}]
+      end
 
     children = [
       %{
@@ -17,19 +24,23 @@ defmodule Logger.App do
         modules: :dynamic
       },
       {Logger.Watcher, {Logger, Logger.Config, []}},
-      {Logger.WatcherSupervisor, {Logger.Config, :handlers, []}},
-      %{
-        id: Logger.ErrorHandler,
-        start: {Logger.Watcher, :start_link, [error_handler]}
-      }
+      {Logger.WatcherSupervisor, {Logger.Config, :handlers, []}}
+      | otp_children
     ]
 
     config = Logger.Config.new()
 
     case Supervisor.start_link(children, strategy: :rest_for_one, name: Logger.Supervisor) do
       {:ok, sup} ->
-        handlers = [error_logger_tty_h: otp_reports?, sasl_report_tty_h: sasl_reports?]
-        delete_handlers(handlers)
+        if otp_logger?() do
+          if otp_reports? do
+            add_elixir_handler(sasl_reports?)
+            delete_erlang_handler()
+          end
+        else
+          delete_old_handlers(otp_reports?, sasl_reports?)
+        end
+
         {:ok, sup, config}
 
       {:error, _} = error ->
@@ -45,9 +56,11 @@ defmodule Logger.App do
 
   @doc false
   def stop(config) do
-    Logger.Config.deleted_handlers()
-    |> add_handlers()
+    if otp_logger?() do
+      _ = :logger.remove_handler(Logger)
+    end
 
+    add_handlers(Logger.Config.deleted_handlers())
     Logger.Config.delete(config)
   end
 
@@ -73,17 +86,54 @@ defmodule Logger.App do
     end
   end
 
-  defp delete_handlers(handlers) do
-    to_delete =
-      for {handler, delete?} <- handlers,
-          delete? && :error_logger.delete_report_handler(handler) != {:error, :module_not_found},
-          do: handler
+  # TODO: Remove conditional error_logger code once we require Erlang/OTP 21+.
+  defp otp_logger? do
+    is_pid(Process.whereis(:logger))
+  end
 
-    [] = Logger.Config.deleted_handlers(to_delete)
+  defp add_elixir_handler(sasl_reports?) do
+    config = %{level: :debug, sasl_reports?: sasl_reports?}
+    :logger.add_handler(Logger, Logger.ErlangHandler, config)
+  end
+
+  defp delete_erlang_handler() do
+    with {:ok, %{module: module} = config} <- :logger.get_handler_config(:default),
+         :ok <- :logger.remove_handler(:default) do
+      %{level: level} = :logger.get_primary_config()
+      :logger.update_primary_config(%{level: :debug})
+      primary_config = {:primary, %{level: level}}
+      handler_config = {:default, module, config}
+      [] = Logger.Config.deleted_handlers([primary_config, handler_config])
+      :ok
+    else
+      _ -> :ok
+    end
+  end
+
+  defp delete_old_handlers(otp_reports?, sasl_reports?) do
+    deleted =
+      for {tty, true} <- [error_logger_tty_h: otp_reports?, sasl_report_tty_h: sasl_reports?],
+          :error_logger.delete_report_handler(tty) != {:error, :module_not_found},
+          do: tty
+
+    [] = Logger.Config.deleted_handlers(deleted)
     :ok
   end
 
   defp add_handlers(handlers) do
-    Enum.each(handlers, &:error_logger.add_report_handler/1)
+    for handler <- handlers do
+      case handler do
+        {handler, module, config} ->
+          :logger.add_handler(handler, module, config)
+
+        {:primary, config} ->
+          :logger.update_primary_config(config)
+
+        handler ->
+          :error_logger.add_report_handler(handler)
+      end
+    end
+
+    :ok
   end
 end

lib/logger/lib/logger/erlang_handler.ex

@@ -0,0 +1,133 @@
+defmodule Logger.ErlangHandler do
+  @moduledoc false
+
+  @doc """
+  Hook required by `:logger`.
+  """
+  def log(%{meta: %{domain: [:otp, :sasl | _]}}, %{sasl_reports?: false}) do
+    :ok
+  end
+
+  def log(%{meta: %{domain: [:supervisor_report]}}, %{sasl_reports?: false}) do
+    :ok
+  end
+
+  def log(%{level: level, msg: msg, meta: erl_meta}, _config) do
+    level = erlang_level_to_elixir_level(level)
+
+    Logger.bare_log(level, fn ->
+      try do
+        meta = extract_metadata(erl_meta)
+
+        case msg do
+          {:string, string} ->
+            {string, meta}
+
+          {:report, %{label: label, report: report} = complete} when map_size(complete) == 2 ->
+            translate(level, :report, {label, report}, meta, erl_meta)
+
+          {:report, %{label: {:error_logger, _}, format: format, args: args}} ->
+            translate(level, :format, {format, args}, meta, erl_meta)
+
+          {:report, report} ->
+            translate(level, :report, {:logger, report}, meta, erl_meta)
+
+          {format, args} ->
+            translate(level, :format, {format, args}, meta, erl_meta)
+        end
+      rescue
+        e ->
+          [
+            "Failure while translating Erlang's logger event\n",
+            Exception.format(:error, e, System.stacktrace())
+          ]
+      end
+    end)
+  end
+
+  defp erlang_level_to_elixir_level(:emergency), do: :error
+  defp erlang_level_to_elixir_level(:alert), do: :error
+  defp erlang_level_to_elixir_level(:critical), do: :error
+  defp erlang_level_to_elixir_level(:error), do: :error
+  defp erlang_level_to_elixir_level(:warning), do: :warn
+  defp erlang_level_to_elixir_level(:notice), do: :info
+  defp erlang_level_to_elixir_level(:info), do: :info
+  defp erlang_level_to_elixir_level(:debug), do: :debug
+
+  defp extract_metadata(map) do
+    metadata = []
+
+    metadata =
+      case map do
+        %{mfa: {mod, fun, arity}} -> [module: mod, function: form_fa(fun, arity)] ++ metadata
+        _ -> metadata
+      end
+
+    metadata =
+      case map do
+        %{file: file, line: line} -> [file: List.to_string(file), line: line] ++ metadata
+        _ -> metadata
+      end
+
+    metadata =
+      case map do
+        %{pid: pid} -> [pid: pid] ++ metadata
+        _ -> metadata
+      end
+
+    metadata
+  rescue
+    _ -> []
+  end
+
+  defp form_fa(fun, arity) do
+    Atom.to_string(fun) <> "/" <> Integer.to_string(arity)
+  end
+
+  @doc """
+  Shared translation convenience.
+  """
+  def translate(level, kind, data, meta, erl_meta) do
+    %{
+      level: min_level,
+      truncate: truncate,
+      translators: translators
+    } = Logger.Config.__data__()
+
+    case translate(translators, min_level, level, kind, data, meta) do
+      :none -> {translate_fallback(kind, data, erl_meta, truncate), meta}
+      other -> other
+    end
+  end
+
+  defp translate([{mod, fun} | t], min_level, level, kind, data, meta) do
+    case apply(mod, fun, [min_level, level, kind, data]) do
+      {:ok, chardata, transdata} -> {chardata, Keyword.merge(meta, transdata)}
+      {:ok, chardata} -> {chardata, meta}
+      :skip -> :skip
+      :none -> translate(t, min_level, level, kind, data, meta)
+    end
+  end
+
+  defp translate([], _min_level, _level, _kind, _data, _meta) do
+    :none
+  end
+
+  defp translate_fallback(:report, {:logger, data}, %{report_cb: callback} = meta, truncate) do
+    translate_fallback(:format, callback.(data), meta, truncate)
+  end
+
+  defp translate_fallback(:format, {format, args}, _meta, truncate) do
+    format
+    |> Logger.Utils.scan_inspect(args, truncate)
+    |> :io_lib.build_text()
+  end
+
+  defp translate_fallback(:report, {_type, %{} = data}, _meta, _truncate) do
+    Kernel.inspect(Map.to_list(data))
+  end
+
+  defp translate_fallback(:report, {_type, data}, _meta, _truncate) do
+    Kernel.inspect(data)
+  end
+end

lib/logger/lib/logger/error_handler.ex

@@ -2,6 +2,8 @@ defmodule Logger.ErrorHandler do
   @moduledoc false
   @behaviour :gen_event
 
+  # TODO: Remove this when we require Erlang/OTP 21+.
+
   def init({otp?, sasl?, threshold}) do
     # We store the Logger PID in the state because when we are shutting
     # down the Logger application, the Logger process may be terminated
@@ -83,17 +85,13 @@ defmodule Logger.ErrorHandler do
     %{
       mode: mode,
       level: min_level,
-      truncate: truncate,
-      utc_log: utc_log?,
-      translators: translators
+      utc_log: utc_log?
     } = Logger.Config.__data__()
 
     with true <- Logger.compare_levels(level, min_level) != :lt and mode != :discard,
-         {:ok, message} <- translate(translators, min_level, level, kind, data, truncate) do
-      message = Logger.Utils.truncate(message, truncate)
-
+         meta = [pid: ensure_pid(pid), error_logger: ensure_type(type)],
+         {message, meta} <- Logger.ErlangHandler.translate(level, kind, data, meta, %{}) do
       # Mode is always async to avoid clogging the error_logger
-      meta = [pid: ensure_pid(pid), error_logger: ensure_type(type)]
       event = {Logger, message, Logger.Utils.timestamp(utc_log?), meta}
       :gen_event.notify(state.logger, {level, gl, event})
     end
@@ -155,25 +153,4 @@ defmodule Logger.ErrorHandler do
       0 -> :ok
     end
   end
-
-  defp translate([{mod, fun} | t], min_level, level, kind, data, truncate) do
-    case apply(mod, fun, [min_level, level, kind, data]) do
-      {:ok, chardata} -> {:ok, chardata}
-      :skip -> :skip
-      :none -> translate(t, min_level, level, kind, data, truncate)
-    end
-  end
-
-  defp translate([], _min_level, _level, :format, {format, args}, truncate) do
-    msg =
-      format
-      |> Logger.Utils.scan_inspect(args, truncate)
-      |> :io_lib.build_text()
-
-    {:ok, msg}
-  end
-
-  defp translate([], _min_level, _level, :report, {_type, data}, _truncate) do
-    {:ok, Kernel.inspect(data)}
-  end
 end

lib/logger/lib/logger/formatter.ex

@@ -20,7 +20,7 @@ defmodule Logger.Formatter do
     * `$message`  - the log message
     * `$level`    - the log level
     * `$node`     - the node that prints the message
-    * `$metadata` - user controlled data presented in `"key=val key2=val2"` format
+    * `$metadata` - user controlled data presented in `"key=val key2=val2 "` format
     * `$levelpad` - sets to a single space if level is 4 characters long,
       otherwise set to the empty space. Used to align the message after level.
 

lib/logger/lib/logger/translator.ex

@@ -3,7 +3,7 @@ defmodule Logger.Translator do
   Default translation for Erlang log messages.
 
   Logger allows developers to rewrite log messages provided by
-  Erlang applications into a format more compatible with Elixir
+  OTP applications into a format more compatible with Elixir
   log messages by providing a translator.
 
   A translator is simply a tuple containing a module and a function
@@ -13,14 +13,15 @@ defmodule Logger.Translator do
 
     * `min_level` - the current Logger level
     * `level` - the level of the message being translated
-    * `kind` - if the message is a report or a format
-    * `message` - the message to format. If it is a report, it is a tuple
-      with `{report_type, report_data}`, if it is a format, it is a
-      tuple with `{format_message, format_args}`
+    * `kind` - if the message is a `:report` or `:format`
+    * `message` - the message to format. If it is `:report`, it is a tuple
+      with `{report_type, report_data}`, if it is `:format`, it is a
+      tuple with `{format_message, format_args}`.
 
   The function must return:
 
-    * `{:ok, chardata}` - if the message was translated with its translation
+    * `{:ok, chardata, metadata}` - if the message translation with its metadata
+    * `{:ok, chardata}` - the translated message
     * `:skip` - if the message is not meant to be translated nor logged
     * `:none` - if there is no translation, which triggers the next translator
 
@@ -28,17 +29,67 @@ defmodule Logger.Translator do
   and the default messages translated by Logger.
   """
 
-  # The name_or_id checks are required to support old OTP projects.
-
+  @doc """
+  Built-in translation function.
+  """
   def translate(min_level, level, kind, message)
 
+  ## Erlang/OTP 21 and after
+
+  def translate(min_level, _level, :report, {:logger, %{label: label} = report}) do
+    case label do
+      {:gen_server, :terminate} ->
+        report_gen_server_terminate(min_level, report)
+
+      {:gen_event, :terminate} ->
+        report_gen_event_terminate(min_level, report)
+
+      _ ->
+        :skip
+    end
+  end
+
+  def translate(min_level, _level, :report, {{:proc_lib, :crash}, data}) do
+    report_crash(min_level, data)
+  end
+
+  def translate(min_level, _level, :report, {{:supervisor, :progress}, data}) do
+    report_supervisor_progress(min_level, data)
+  end
+
+  def translate(min_level, _level, :report, {{:supervisor, _}, data}) do
+    report_supervisor(min_level, data)
+  end
+
+  def translate(
+        _min_level,
+        _level,
+        :report,
+        {{:application_controller, :progress}, [application: app, started_at: node]}
+      ) do
+    {:ok, ["Application ", Atom.to_string(app), " started at " | inspect(node)]}
+  end
+
+  def translate(
+        _min_level,
+        _level,
+        :report,
+        {{:application_controller, :exit}, [application: app, exited: reason, type: _type]}
+      ) do
+    {:ok, ["Application ", Atom.to_string(app), " exited: " | Application.format_error(reason)]}
+  end
+
+  ## Erlang/OTP 20 and before
+
   def translate(min_level, :error, :format, message) do
     opts = Application.get_env(:logger, :translator_inspect_opts)
 
     case message do
       {'** Generic server ' ++ _, [name, last, state, reason | client]} ->
+        {formatted, _reason} = format_reason(reason)
+
         msg =
-          ["GenServer #{inspect(name)} terminating", format_stop(reason)] ++
+          ["GenServer #{inspect(name)} terminating", formatted] ++
             ["\nLast message#{format_from(client)}: #{inspect(last, opts)}"]
 
         if min_level == :debug do
@@ -48,9 +99,11 @@ defmodule Logger.Translator do
         end
 
       {'** gen_event handler ' ++ _, [name, manager, last, state, reason]} ->
+        {formatted, _reason} = format_reason(reason)
+
         msg =
           ["GenEvent handler #{inspect(name)} installed in #{inspect(manager)} terminating"] ++
-            [format_stop(reason), "\nLast message: #{inspect(last, opts)}"]
+            [formatted, "\nLast message: #{inspect(last, opts)}"]
 
         if min_level == :debug do
           {:ok, [msg | "\nState: #{inspect(state, opts)}"]}
@@ -59,9 +112,11 @@ defmodule Logger.Translator do
         end
 
       {'** Task ' ++ _, [name, starter, function, args, reason]} ->
+        {formatted, _reason} = format_reason(reason)
+
         msg =
           ["Task #{inspect(name)} started from #{inspect(starter)} terminating"] ++
-            [format_stop(reason), "\nFunction: #{inspect(function, opts)}"] ++
+            [formatted, "\nFunction: #{inspect(function, opts)}"] ++
             ["\n    Args: #{inspect(args, opts)}"]
 
         {:ok, msg}
@@ -80,109 +135,102 @@ defmodule Logger.Translator do
         :std_info,
         [application: app, exited: reason, type: _type]
       }) do
-    {:ok, "Application #{app} exited: #{Application.format_error(reason)}"}
+    {:ok, ["Application ", Atom.to_string(app), " exited: " | Application.format_error(reason)]}
+  end
+
+  def translate(min_level, :error, :report, {{:error_logger, :error_report}, data}) do
+    report_supervisor(min_level, data)
   end
 
   def translate(min_level, :error, :report, {:supervisor_report, data}) do
-    translate_supervisor(min_level, data)
+    report_supervisor(min_level, data)
   end
 
   def translate(min_level, :error, :report, {:crash_report, data}) do
-    translate_crash(min_level, data)
+    report_crash(min_level, data)
   end
 
-  def translate(min_level, :info, :report, {:progress, data}) do
-    translate_progress(min_level, data)
+  def translate(min_level, :info, :report, {:progress, [{:supervisor, _} | _] = data}) do
+    report_supervisor_progress(min_level, data)
   end
 
+  def translate(_min_level, :info, :report, {:progress, [application: app, started_at: node]}) do
+    {:ok, ["Application ", Atom.to_string(app), " started at " | inspect(node)]}
+  end
+
+  ## Helpers
+
   def translate(_min_level, _level, _kind, _message) do
     :none
   end
 
-  defp translate_supervisor(
-         min_level,
-         supervisor: sup,
-         errorContext: context,
-         reason: reason,
-         offender: [{:pid, pid}, {name_or_id, name} | offender]
-       )
-       when is_pid(pid) and context !== :shutdown and name_or_id in [:name, :id] do
-    msg =
-      ["Child ", inspect(name), " of Supervisor ", sup_name(sup)] ++
-        [?\s, sup_context(context), "\n** (exit) "] ++
-        [offender_reason(reason, context), "\nPid: ", inspect(pid)] ++
-        child_info(min_level, offender)
+  defp report_gen_server_terminate(min_level, report) do
+    inspect_opts = Application.get_env(:logger, :translator_inspect_opts)
 
-    {:ok, msg}
+    %{
+      client_info: client,
+      last_message: last,
+      name: name,
+      reason: reason,
+      state: state
+    } = report
+
+    {formatted, _reason} = format_reason(reason)
+
+    msg =
+      ["GenServer ", inspect(name), " terminating", formatted] ++
+        ["\nLast message", format_last_message_from(client), ": ", inspect(last, inspect_opts)]
+
+    if min_level == :debug do
+      {:ok, [msg, "\nState: ", inspect(state, inspect_opts) | format_client_info(client)]}
+    else
+      {:ok, msg}
+    end
   end
 
-  defp translate_supervisor(
-         min_level,
-         supervisor: sup,
-         errorContext: context,
-         reason: reason,
-         offender: [{:pid, _pid}, {name_or_id, name} | offender]
-       )
-       when name_or_id in [:name, :id] do
-    msg =
-      ["Child ", inspect(name), " of Supervisor ", sup_name(sup)] ++
-        [?\s, sup_context(context), "\n** (exit) ", offender_reason(reason, context)] ++
-        child_info(min_level, offender)
+  defp report_gen_event_terminate(min_level, report) do
+    inspect_opts = Application.get_env(:logger, :translator_inspect_opts)
 
-    {:ok, msg}
+    %{
+      handler: handler,
+      last_message: last,
+      name: name,
+      reason: reason,
+      state: state
+    } = report
+
+    reason =
+      case reason do
+        {:EXIT, why} -> why
+        _ -> reason
+      end
+
+    {formatted, _reason} = format_reason(reason)
+
+    msg =
+      ["GenEvent handler ", inspect(handler), " installed in ", inspect(name), " terminating"] ++
+        [formatted, "\nLast message: ", inspect(last, inspect_opts)]
+
+    if min_level == :debug do
+      {:ok, [msg, "\nState: ", inspect(state, inspect_opts)]}
+    else
+      {:ok, msg}
+    end
   end
 
-  defp translate_supervisor(
+  defp report_supervisor_progress(
          min_level,
          supervisor: sup,
-         errorContext: context,
-         reason: reason,
-         offender: [{:pid, pid} | offender]
+         started: [{:pid, pid}, {:id, id} | started]
        ) do
     msg =
-      ["Child of Supervisor ", sup_name(sup), ?\s, sup_context(context), "\n** (exit) "] ++
-        [offender_reason(reason, context), "\nPid: ", inspect(pid)] ++
-        child_info(min_level, offender)
+      ["Child ", inspect(id), " of Supervisor ", sup_name(sup), " started"] ++
+        ["\nPid: ", inspect(pid)] ++ child_info(min_level, started)
 
     {:ok, msg}
   end
 
-  defp translate_supervisor(
-         min_level,
-         supervisor: sup,
-         errorContext: context,
-         reason: reason,
-         offender: [{:nb_children, n}, {name_or_id, name} | offender]
-       )
-       when name_or_id in [:name, :id] do
-    msg =
-      ["Children ", inspect(name), " of Supervisor ", sup_name(sup), ?\s, sup_context(context)] ++
-        ["\n** (exit) ", offender_reason(reason, context), "\nNumber: ", inspect(n)] ++
-        child_info(min_level, offender)
-
-    {:ok, msg}
-  end
-
-  defp translate_supervisor(_min_level, _other), do: :none
-
-  defp translate_progress(_min_level, application: app, started_at: node_name) do
-    {:ok, ["Application ", to_string(app), " started at " | inspect(node_name)]}
-  end
-
-  defp translate_progress(
-         min_level,
-         supervisor: sup,
-         started: [{:pid, pid}, {name_or_id, name} | started]
-       )
-       when name_or_id in [:name, :id] do
-    msg =
-      ["Child ", inspect(name), " of Supervisor ", sup_name(sup)] ++
-        [" started", "\nPid: ", inspect(pid)] ++ child_info(min_level, started)
-
-    {:ok, msg}
-  end
-
-  defp translate_progress(
+  defp report_supervisor_progress(
          min_level,
          supervisor: sup,
          started: [{:pid, pid} | started]
@@ -194,7 +242,70 @@ defmodule Logger.Translator do
     {:ok, msg}
   end
 
-  defp translate_progress(_min_level, _other), do: :none
+  defp report_supervisor_progress(_min_level, _other), do: :none
+
+  defp report_supervisor(
+         min_level,
+         supervisor: sup,
+         errorContext: context,
+         reason: reason,
+         offender: [{:pid, pid}, {:id, id} | offender]
+       ) do
+    pid_info =
+      if is_pid(pid) and context != :shutdown do
+        ["\nPid: ", inspect(pid)]
+      else
+        []
+      end
+
+    msg =
+      ["Child ", inspect(id), " of Supervisor ", sup_name(sup)] ++
+        [?\s, sup_context(context), "\n** (exit) ", offender_reason(reason, context)] ++
+        pid_info ++ child_info(min_level, offender)
+
+    {:ok, msg}
+  end
+
+  defp report_supervisor(
+         min_level,
+         supervisor: sup,
+         errorContext: context,
+         reason: reason,
+         offender: [{:nb_children, n}, {:id, id} | offender]
+       ) do
+    msg =
+      ["Children ", inspect(id), " of Supervisor ", sup_name(sup), ?\s, sup_context(context)] ++
+        ["\n** (exit) ", offender_reason(reason, context), "\nNumber: ", Integer.to_string(n)] ++
+        child_info(min_level, offender)
+
+    {:ok, msg}
+  end
+
+  defp report_supervisor(
+         min_level,
+         supervisor: sup,
+         errorContext: context,
+         reason: reason,
+         offender: [{:pid, pid} | offender]
+       ) do
+    msg =
+      ["Child of Supervisor ", sup_name(sup), ?\s, sup_context(context)] ++
+        ["\n** (exit) ", offender_reason(reason, context), "\nPid: ", inspect(pid)] ++
+        child_info(min_level, offender)
+
+    {:ok, msg}
+  end
+
+  defp report_supervisor(_min_level, _other), do: :none
+
+  # If start call raises reason will be of form {:EXIT, reason}
+  defp offender_reason({:EXIT, reason}, :start_error) do
+    Exception.format_exit(reason)
+  end
+
+  defp offender_reason(reason, _context) do
+    Exception.format_exit(reason)
+  end
 
   defp sup_name({:local, name}), do: inspect(name)
   defp sup_name({:global, name}), do: inspect(name)
@@ -210,14 +321,20 @@ defmodule Logger.Translator do
     ["\nStart Call: ", format_mfa(mod, fun, args) | child_debug(min_level, debug)]
   end
 
+  # Comes from bridge with MFA
   defp child_info(min_level, [{:mfa, {mod, fun, args}} | debug]) do
     ["\nStart Call: ", format_mfa(mod, fun, args) | child_debug(min_level, debug)]
   end
 
+  # Comes from bridge with Mod
   defp child_info(min_level, [{:mod, mod} | debug]) do
     ["\nStart Module: ", inspect(mod) | child_debug(min_level, debug)]
   end
 
+  defp child_info(_min_level, _child) do
+    []
+  end
+
   defp child_debug(:debug, restart_type: restart, shutdown: shutdown, child_type: type) do
     ["\nRestart: ", inspect(restart), "\nShutdown: ", inspect(shutdown)] ++
       ["\nType: ", inspect(type)]
@@ -227,16 +344,7 @@ defmodule Logger.Translator do
     []
   end
 
-  # If start call raises reason will be of form {:EXIT, reason}
-  defp offender_reason({:EXIT, reason}, :start_error) do
-    Exception.format_exit(reason)
-  end
-
-  defp offender_reason(reason, _context) do
-    Exception.format_exit(reason)
-  end
-
-  defp translate_crash(min_level, [
+  defp report_crash(min_level, [
          [
            {:initial_call, _} = initial_call,
            {:pid, pid},
@@ -245,14 +353,16 @@ defmodule Logger.Translator do
          ],
          linked
        ]) do
+    reason = Exception.normalize(kind, exception, stack)
+
     msg =
-      ["Process ", crash_name(pid, name), " terminating", format(kind, exception, stack)] ++
+      ["Process ", crash_name(pid, name), " terminating", format(kind, reason, stack)] ++
         [crash_info(min_level, [initial_call | crashed])] ++ crash_linked(min_level, linked)
 
     {:ok, msg}
   end
 
-  defp translate_crash(min_level, [
+  defp report_crash(min_level, [
          [
            {:pid, pid},
            {:registered_name, name},
@@ -260,8 +370,10 @@ defmodule Logger.Translator do
          ],
          linked
        ]) do
+    reason = Exception.normalize(kind, exception, stack)
+
     msg =
-      ["Process ", crash_name(pid, name), " terminating", format(kind, exception, stack)] ++
+      ["Process ", crash_name(pid, name), " terminating", format(kind, reason, stack)] ++
         [crash_info(min_level, crashed), crash_linked(min_level, linked)]
 
     {:ok, msg}
@@ -309,7 +421,6 @@ defmodule Logger.Translator do
   defp crash_debug(:current_stacktrace, stack, prefix) do
     stack_prefix = [prefix | "    "]
     stacktrace = Enum.map(stack, &[stack_prefix | Exception.format_stacktrace_entry(&1)])
-
     [prefix, "Current Stacktrace:" | stacktrace]
   end
 
@@ -339,43 +450,59 @@ defmodule Logger.Translator do
     end)
   end
 
-  defp crash_neighbour(min_level, [{:pid, pid}, {:registered_name, []} | info]) do
-    indent = "    "
+  @indent "    "
 
-    [?\n, indent, inspect(pid) | crash_info(min_level, info, [?\n, indent | indent])]
+  defp crash_neighbour(min_level, [{:pid, pid}, {:registered_name, []} | info]) do
+    [?\n, @indent, inspect(pid) | crash_info(min_level, info, [?\n, @indent | @indent])]
   end
 
   defp crash_neighbour(min_level, [{:pid, pid}, {:registered_name, name} | info]) do
-    indent = "    "
-
-    [?\n, indent, inspect(name), " (", inspect(pid), ")"] ++
-      crash_info(min_level, info, [?\n, indent | indent])
+    [?\n, @indent, inspect(name), " (", inspect(pid), ")"] ++
+      crash_info(min_level, info, [?\n, @indent | @indent])
   end
 
-  defp format_stop({maybe_exception, [_ | _] = maybe_stacktrace} = reason) do
+  defp format_last_message_from({_, {name, _}}), do: [" (from ", inspect(name), ")"]
+  defp format_last_message_from({from, _}), do: [" (from ", inspect(from), ")"]
+  defp format_last_message_from(_), do: []
+
+  defp format_client_info({from, :dead}),
+    do: ["\nClient ", inspect(from), " is dead"]
+
+  defp format_client_info({from, :remote}),
+    do: ["\nClient ", inspect(from), " is remote on node ", inspect(node(from))]
+
+  defp format_client_info({_, {name, stacktrace}}),
+    do: ["\nClient ", inspect(name), " is alive\n" | format_stacktrace(stacktrace)]
+
+  defp format_client_info(_),
+    do: []
+
+  defp format_reason({maybe_exception, [_ | _] = maybe_stacktrace} = reason) do
     try do
       format_stacktrace(maybe_stacktrace)
     catch
       :error, _ ->
-        format_stop_banner(reason)
+        {format_stop(reason), {reason, []}}
     else
       formatted_stacktrace ->
-        [format_stop_banner(maybe_exception, maybe_stacktrace) | formatted_stacktrace]
+        {formatted, reason} = maybe_normalize(maybe_exception, maybe_stacktrace)
+        {[formatted | formatted_stacktrace], {reason, maybe_stacktrace}}
     end
   end
 
-  defp format_stop(reason) do
-    format_stop_banner(reason)
+  defp format_reason(reason) do
+    {format_stop(reason), {reason, []}}
   end
 
-  defp format_stop_banner(reason) do
+  defp format_stop(reason) do
     ["\n** (stop) " | Exception.format_exit(reason)]
   end
 
-  # OTP processes rewrite the :undef error to these reasons when logging
+  # Erlang processes rewrite the :undef error to these reasons when logging
+  # TODO: This clause can be removed when we support only Erlang/OTP 21+.
   @gen_undef [:"module could not be loaded", :"function not exported"]
 
-  defp format_stop_banner(undef, [{mod, fun, args, _info} | _] = stacktrace)
+  defp maybe_normalize(undef, [{mod, fun, args, _info} | _] = stacktrace)
        when undef in @gen_undef and is_atom(mod) and is_atom(fun) do
     cond do
       is_list(args) ->
@@ -385,47 +512,44 @@ defmodule Logger.Translator do
         format_undef(mod, fun, args, undef, stacktrace)
 
       true ->
-        format_stop_banner(undef)
+        {format_stop(undef), undef}
     end
   end
 
-  defp format_stop_banner(reason, stacktrace) do
+  defp maybe_normalize(reason, stacktrace) do
     # If this is already an exception (even an ErlangError), we format it as an
     # exception. Otherwise, we try to normalize it, and if it's normalized as an
     # ErlangError we instead format it as an exit.
     if Exception.exception?(reason) do
-      [?\n | Exception.format_banner(:error, reason, stacktrace)]
+      {[?\n | Exception.format_banner(:error, reason, stacktrace)], reason}
     else
       case Exception.normalize(:error, reason, stacktrace) do
         %ErlangError{} ->
-          format_stop_banner(reason)
+          {format_stop(reason), reason}
 
         exception ->
-          [?\n | Exception.format_banner(:error, exception, stacktrace)]
+          {[?\n | Exception.format_banner(:error, exception, stacktrace)], exception}
       end
     end
   end
 
-  defp format_undef(mod, fun, arity, undef, stacktrace) do
-    opts = [module: mod, function: fun, arity: arity, reason: undef]
-    exception = UndefinedFunctionError.exception(opts)
-    [?\n | Exception.format_banner(:error, exception, stacktrace)]
-  end
-
   defp format(kind, payload, stacktrace) do
     [?\n, Exception.format_banner(kind, payload, stacktrace) | format_stacktrace(stacktrace)]
   end
 
   defp format_stacktrace(stacktrace) do
     for entry <- stacktrace do
-      [<<"\n    ">> | Exception.format_stacktrace_entry(entry)]
+      ["\n    " | Exception.format_stacktrace_entry(entry)]
     end
   end
 
   defp format_mfa(mod, fun, :undefined),
     do: [inspect(mod), ?., Code.Identifier.inspect_as_function(fun) | "/?"]
 
-  defp format_mfa(mod, fun, args), do: Exception.format_mfa(mod, fun, args)
+  defp format_mfa(mod, fun, args),
+    do: Exception.format_mfa(mod, fun, args)
+
+  ## Deprecated helpers
 
   defp format_from([]), do: ""
   defp format_from([from]), do: " (from #{inspect(from)})"
@@ -439,10 +563,16 @@ defmodule Logger.Translator do
   end
 
   defp format_client([from, stacktrace]) when is_list(stacktrace) do
-    ["\nClient #{inspect(from)} is alive\n" | Exception.format_stacktrace(stacktrace)]
+    ["\nClient #{inspect(from)} is alive\n" | format_stacktrace(stacktrace)]
   end
 
   defp format_client(_) do
     []
   end
+
+  defp format_undef(mod, fun, arity, undef, stacktrace) do
+    opts = [module: mod, function: fun, arity: arity, reason: undef]
+    exception = UndefinedFunctionError.exception(opts)
+    {[?\n | Exception.format_banner(:error, exception, stacktrace)], exception}
+  end
 end

lib/logger/lib/logger/utils.ex

@@ -139,7 +139,7 @@ defmodule Logger.Utils do
     # arguments according to the truncate limit.
     {args, _} =
       Enum.map_reduce(args, truncate, fn arg, acc ->
-        if is_binary(arg) do
+        if is_binary(arg) and acc != :infinity do
           truncate_n(arg, acc)
         else
           {arg, acc}

lib/logger/test/logger/config_test.exs

@@ -24,7 +24,6 @@ defmodule Logger.ConfigTest do
     Process.whereis(Logger) |> Process.exit(:kill)
     wait_for_logger()
     wait_for_handler(Logger, Logger.Config)
-    wait_for_handler(:error_logger, Logger.ErrorHandler)
   end
 
   test "Logger.Config updates config on config_change/3" do

lib/logger/test/logger/erlang_handler_test.exs

@@ -0,0 +1,109 @@
+defmodule Logger.ErlangHandlerTest do
+  use Logger.Case
+  @moduletag :logger
+
+  defmodule CustomTranslator do
+    def t(:debug, :info, :format, {'hello: ~p', [:ok]}) do
+      :skip
+    end
+
+    def t(:debug, :info, :format, {'world: ~p', [:ok]}) do
+      {:ok, "rewritten"}
+    end
+
+    def t(:debug, :info, :report, {:logger, %{hello: :ok}}) do
+      :skip
+    end
+
+    def t(:debug, :info, :report, {:logger, %{world: :ok}}) do
+      {:ok, "rewritten"}
+    end
+
+    def t(:debug, :info, :report, {:logger, %{error: error}}) do
+      raise(error)
+    end
+
+    def t(_, _, _, _) do
+      :none
+    end
+  end
+
+  test "add_translator/1 and remove_translator/1 for error_logger" do
+    assert Logger.add_translator({CustomTranslator, :t})
+
+    assert capture_log(fn ->
+             :error_logger.info_msg('hello: ~p', [:ok])
+           end) == ""
+
+    assert capture_log(fn ->
+             :error_logger.info_msg('world: ~p', [:ok])
+           end) =~ "[info]  rewritten"
+  after
+    assert Logger.remove_translator({CustomTranslator, :t})
+  end
+
+  test "add_translator/1 and remove_translator/1 for logger formats" do
+    assert Logger.add_translator({CustomTranslator, :t})
+
+    assert capture_log(fn ->
+             :logger.info('hello: ~p', [:ok])
+           end) == ""
+
+    assert capture_log(fn ->
+             :logger.info('world: ~p', [:ok])
+           end) =~ "[info]  rewritten"
+
+    assert capture_log(fn ->
+             :logger.info(%{hello: :ok})
+           end) == ""
+
+    assert capture_log(fn ->
+             :logger.info(%{world: :ok})
+           end) =~ "[info]  rewritten"
+  after
+    assert Logger.remove_translator({CustomTranslator, :t})
+  end
+
+  test "handles translation error" do
+    assert Logger.add_translator({CustomTranslator, :t})
+
+    message = capture_log(fn -> :logger.info(%{error: "oops"}) end)
+    assert message =~ "[info]  Failure while translating Erlang's logger event\n"
+    assert message =~ "** (RuntimeError) oops\n"
+  after
+    assert Logger.remove_translator({CustomTranslator, :t})
+  end
+
+  test "converts Erlang metadata" do
+    Logger.configure_backend(:console, metadata: [:file, :line, :module, :function])
+
+    message =
+      capture_log(fn ->
+        :logger.info("ok", %{file: 'file.erl', line: 13, mfa: {Foo, :bar, 3}})
+      end)
+
+    assert message =~ "module=Foo"
+    assert message =~ "function=bar/3"
+    assert message =~ "file=file.erl"
+    assert message =~ "line=13"
+  after
+    Logger.configure_backend(:console, metadata: [])
+  end
+
+  test "uses reporting callback with Elixir inspection" do
+    assert capture_log(fn ->
+             callback = fn %{hello: :world} -> {"~p~n", [:formatted]} end
+             :logger.info(%{hello: :world}, %{report_cb: callback})
+           end) =~ "[info]  :formatted"
+  end
+
+  test "converts log levels" do
+    assert capture_log(fn -> :logger.emergency('ok') end) =~ "[error] ok"
+    assert capture_log(fn -> :logger.alert('ok') end) =~ "[error] ok"
+    assert capture_log(fn -> :logger.critical('ok') end) =~ "[error] ok"
+    assert capture_log(fn -> :logger.error('ok') end) =~ "[error] ok"
+    assert capture_log(fn -> :logger.warning('ok') end) =~ "[warn]  ok"
+    assert capture_log(fn -> :logger.info('ok') end) =~ "[info]  ok"
+    assert capture_log(fn -> :logger.debug('ok') end) =~ "[debug] ok"
+  end
+end

lib/logger/test/logger/error_handler_test.exs

@@ -1,5 +1,34 @@
 defmodule Logger.ErrorHandlerTest do
   use Logger.Case
+  @moduletag :error_logger
+
+  test "add_translator/1 and remove_translator/1" do
+    defmodule CustomTranslator do
+      def t(:debug, :info, :format, {'hello: ~p', [:ok]}) do
+        :skip
+      end
+
+      def t(:debug, :info, :format, {'world: ~p', [:ok]}) do
+        {:ok, "rewritten"}
+      end
+
+      def t(_, _, _, _) do
+        :none
+      end
+    end
+
+    assert Logger.add_translator({CustomTranslator, :t})
+
+    assert capture_log(fn ->
+             :error_logger.info_msg('hello: ~p', [:ok])
+           end) == ""
+
+    assert capture_log(fn ->
+             :error_logger.info_msg('world: ~p', [:ok])
+           end) =~ "\[info\]  rewritten"
+  after
+    assert Logger.remove_translator({CustomTranslator, :t})
+  end
 
   test "survives after crashes" do
     expected_result =

lib/logger/test/logger/translator_test.exs

@@ -339,15 +339,6 @@ defmodule Logger.TranslatorTest do
            """s
   end
 
-  test "translates application stop" do
-    assert capture_log(fn ->
-             :ok = Application.start(:eex)
-             Application.stop(:eex)
-           end) =~ """
-           Application eex exited: :stopped
-           """
-  end
-
   test "translates application start" do
     assert capture_log(fn ->
              Application.start(:eex)
@@ -357,6 +348,15 @@ defmodule Logger.TranslatorTest do
            """
   end
 
+  test "translates application stop" do
+    assert capture_log(fn ->
+             :ok = Application.start(:eex)
+             Application.stop(:eex)
+           end) =~ """
+           Application eex exited: :stopped
+           """
+  end
+
   test "translates Process crashes" do
     assert capture_log(:info, fn ->
              {_, ref} = spawn_monitor(fn -> raise "oops" end)
@@ -484,7 +484,7 @@ defmodule Logger.TranslatorTest do
              receive do: ({:DOWN, ^ref, _, _, _} -> :ok)
            end) =~ ~r"""
            Ancestors: \[#PID<\d+\.\d+\.\d+>\](?:
-           Message Queue Length: 1(?#TODO: Require once depend on 20)|)
+           Message Queue Length: 1(?#TODO: Require once depend on Erlang/OTP 20)|)
            Messages: \[:message\]
            Links: \[\]
            Dictionary: \[\]
@@ -516,7 +516,7 @@ defmodule Logger.TranslatorTest do
                    Stack Size: \d+
                    Reductions: \d+(?:
                    Current Stacktrace:
-                       test/logger/translator_test.exs:\d+: Logger.TranslatorTest.sleep/1(?#TODO: Require once depend on 20)|)
+                       test/logger/translator_test.exs:\d+: Logger.TranslatorTest.sleep/1(?#TODO: Require once depend on Erlang/OTP 20)|)
            """
   end
 
@@ -694,6 +694,60 @@ defmodule Logger.TranslatorTest do
            """
   end
 
+  test "translates DynamicSupervisor reports abnormal shutdown" do
+    assert capture_log(:info, fn ->
+             trap = Process.flag(:trap_exit, true)
+             child = %{id: __MODULE__, start: {__MODULE__, :abnormal, []}}
+             {:ok, pid} = DynamicSupervisor.start_link(strategy: :one_for_one)
+             {:ok, _pid2} = DynamicSupervisor.start_child(pid, child)
+             Process.exit(pid, :normal)
+             receive do: ({:EXIT, ^pid, _} -> :ok)
+             Process.flag(:trap_exit, trap)
+           end) =~ ~r"""
+           \[error\] Child :undefined of Supervisor #PID<\d+\.\d+\.\d+> \(Supervisor\.Default\) shutdown abnormally
+           \*\* \(exit\) :stop
+           Pid: #PID<\d+\.\d+\.\d+>
+           Start Call: Logger.TranslatorTest.abnormal\(\)
+           """
+  end
+
+  test "translates DynamicSupervisor reports extra_arguments in in abnormal shutdown" do
+    assert capture_log(:info, fn ->
+             trap = Process.flag(:trap_exit, true)
+
+             {:ok, pid} =
+               DynamicSupervisor.start_link(strategy: :one_for_one, extra_arguments: [:extra])
+
+             child = %{id: __MODULE__, start: {__MODULE__, :abnormal2, [:args]}}
+             {:ok, _pid2} = DynamicSupervisor.start_child(pid, child)
+             Process.exit(pid, :normal)
+             receive do: ({:EXIT, ^pid, _} -> :ok)
+             Process.flag(:trap_exit, trap)
+           end) =~ ~r"""
+           \[error\] Child :undefined of Supervisor #PID<\d+\.\d+\.\d+> \(Supervisor\.Default\) shutdown abnormally
+           \*\* \(exit\) :stop
+           Pid: #PID<\d+\.\d+\.\d+>
+           Start Call: Logger.TranslatorTest.abnormal2\(:extra, :args\)
+           """
+  end
+
+  test "translates named DynamicSupervisor reports abnormal shutdown" do
+    assert capture_log(:info, fn ->
+             trap = Process.flag(:trap_exit, true)
+             child = %{id: __MODULE__, start: {__MODULE__, :abnormal, []}}
+             {:ok, pid} = DynamicSupervisor.start_link(strategy: :one_for_one, name: __MODULE__)
+             {:ok, _pid2} = DynamicSupervisor.start_child(pid, child)
+             Process.exit(pid, :normal)
+             receive do: ({:EXIT, ^pid, _} -> :ok)
+             Process.flag(:trap_exit, trap)
+           end) =~ ~r"""
+           \[error\] Child :undefined of Supervisor Logger.TranslatorTest shutdown abnormally
+           \*\* \(exit\) :stop
+           Pid: #PID<\d+\.\d+\.\d+>
+           Start Call: Logger.TranslatorTest.abnormal\(\)
+           """
+  end
+
   test "translates :supervisor_bridge progress" do
     assert capture_log(:info, fn ->
              trap = Process.flag(:trap_exit, true)
@@ -721,7 +775,7 @@ defmodule Logger.TranslatorTest do
            """
   end
 
-  test "handles :undefined MFA properly" do
+  test "reports :undefined MFA properly" do
     defmodule WeirdFunctionNamesGenServer do
       use GenServer
 
@@ -787,6 +841,10 @@ defmodule Logger.TranslatorTest do
     :proc_lib.start_link(__MODULE__, :abnormal_init, [])
   end
 
+  def abnormal2(:extra, :args) do
+    :proc_lib.start_link(__MODULE__, :abnormal_init, [])
+  end
+
   def abnormal_init() do
     Process.flag(:trap_exit, true)
     :proc_lib.init_ack({:ok, self()})

lib/logger/test/logger/utils_test.exs

@@ -5,9 +5,9 @@ defmodule Logger.UtilsTest do
 
   import Kernel, except: [inspect: 2]
 
-  defp inspect(format, args) do
+  defp inspect(format, args, truncate \\ 10) do
     format
-    |> Logger.Utils.scan_inspect(args, 10)
+    |> Logger.Utils.scan_inspect(args, truncate)
     |> :io_lib.unscan_format()
   end
 
@@ -81,13 +81,13 @@ defmodule Logger.UtilsTest do
     assert truncate(long_string, :infinity) == long_string
   end
 
-  test "inspect/2 formats" do
+  test "scan_inspect/3 formats" do
     assert inspect('~p', [1]) == {'~ts', [["1"]]}
     assert inspect("~p", [1]) == {'~ts', [["1"]]}
     assert inspect(:"~p", [1]) == {'~ts', [["1"]]}
   end
 
-  test "inspect/2 sigils" do
+  test "scan_inspect/3 sigils" do
     assert inspect('~10.10tp', [1]) == {'~ts', [["1"]]}
     assert inspect('~-10.10tp', [1]) == {'~ts', [["1"]]}
 
@@ -95,24 +95,24 @@ defmodule Logger.UtilsTest do
     assert inspect('~10.10x~p~n', [1, 2, 3]) == {'~10.10x~ts~n', [1, 2, ["3"]]}
   end
 
-  test "inspect/2 with modifier t has no effect (as it is the default)" do
+  test "scan_inspect/3 with modifier t has no effect (as it is the default)" do
     assert inspect('~tp', [1]) == {'~ts', [["1"]]}
     assert inspect('~tw', [1]) == {'~ts', [["1"]]}
   end
 
-  test "inspect/2 with modifier l always prints lists" do
+  test "scan_inspect/3 with modifier l always prints lists" do
     assert inspect('~lp', ['abc']) == {'~ts', [["[", "97", ",", " ", "98", ",", " ", "99", "]"]]}
     assert inspect('~lw', ['abc']) == {'~ts', [["[", "97", ",", " ", "98", ",", " ", "99", "]"]]}
   end
 
-  test "inspect/2 with modifier for width" do
+  test "scan_inspect/3 with modifier for width" do
     assert inspect('~5lp', ['abc']) ==
              {'~ts', [["[", "97", ",", "\n ", "98", ",", "\n ", "99", "]"]]}
 
     assert inspect('~5lw', ['abc']) == {'~ts', [["[", "97", ",", " ", "98", ",", " ", "99", "]"]]}
   end
 
-  test "inspect/2 with modifier for limit" do
+  test "scan_inspect/3 with modifier for limit" do
     assert inspect('~5lP', ['abc', 2]) ==
              {'~ts', [["[", "97", ",", "\n ", "98", ",", "\n ", "...", "]"]]}
 
@@ -120,13 +120,18 @@ defmodule Logger.UtilsTest do
              {'~ts', [["[", "97", ",", " ", "98", ",", " ", "...", "]"]]}
   end
 
-  test "inspect/2 truncates binaries" do
+  test "scan_inspect/3 truncates binaries" do
     assert inspect('~ts', ["abcdeabcdeabcdeabcde"]) == {'~ts', ["abcdeabcde"]}
 
     assert inspect('~ts~ts~ts', ["abcdeabcde", "abcde", "abcde"]) ==
              {'~ts~ts~ts', ["abcdeabcde", "", ""]}
   end
 
+  test "scan_inspect/3 with :infinity truncate" do
+    long_string = String.duplicate("foo", 10000)
+    assert inspect('~ts', [long_string], :infinity) == {'~ts', [long_string]}
+  end
+
   test "timestamp/1" do
     assert {{_, _, _}, {_, _, _, _}} = timestamp(true)
   end

lib/logger/test/logger_test.exs

@@ -4,44 +4,13 @@ defmodule LoggerTest do
 
   setup_all do
     Logger.configure_backend(:console, metadata: [:application, :module])
-
-    on_exit(fn ->
-      Logger.configure_backend(:console, metadata: [])
-    end)
+    on_exit(fn -> Logger.configure_backend(:console, metadata: []) end)
   end
 
   defp msg_with_meta(text) do
     msg("module=LoggerTest #{text}")
   end
 
-  test "add_translator/1 and remove_translator/1" do
-    defmodule CustomTranslator do
-      def t(:debug, :info, :format, {'hello: ~p', [:ok]}) do
-        :skip
-      end
-
-      def t(:debug, :info, :format, {'world: ~p', [:ok]}) do
-        {:ok, "rewritten"}
-      end
-
-      def t(_, _, _, _) do
-        :none
-      end
-    end
-
-    assert Logger.add_translator({CustomTranslator, :t})
-
-    assert capture_log(fn ->
-             :error_logger.info_msg('hello: ~p', [:ok])
-           end) == ""
-
-    assert capture_log(fn ->
-             :error_logger.info_msg('world: ~p', [:ok])
-           end) =~ "\[info\]  rewritten"
-  after
-    assert Logger.remove_translator({CustomTranslator, :t})
-  end
-
   test "add_backend/1 and remove_backend/1" do
     assert :ok = Logger.remove_backend(:console)
     assert Application.get_env(:logger, :backends) == []
@@ -348,4 +317,21 @@ defmodule LoggerTest do
     Logger.App.stop()
     Application.start(:logger)
   end
+
+  test "configure/1 sets options" do
+    Logger.configure(sync_threshold: 10)
+    Logger.configure(truncate: 4048)
+    Logger.configure(utc_log: true)
+    Logger.configure(discard_threshold: 10_000)
+    logger_data = Logger.Config.__data__()
+    assert Map.get(logger_data, :sync_threshold) == 10
+    assert Map.get(logger_data, :truncate) == 4048
+    assert Map.get(logger_data, :utc_log) == true
+    assert Map.get(logger_data, :discard_threshold) == 10_000
+  after
+    Logger.configure(sync_threshold: 20)
+    Logger.configure(sync_threshold: 8096)
+    Logger.configure(utc_log: false)
+    Logger.configure(discard_threshold: 500)
+  end
 end

lib/logger/test/test_helper.exs

@@ -1,5 +1,7 @@
 Logger.configure_backend(:console, colors: [enabled: false])
-ExUnit.start()
+
+exclude = if Process.whereis(:logger), do: [:error_logger], else: [:logger]
+ExUnit.start(exclude: exclude)
 
 defmodule Logger.Case do
   use ExUnit.CaseTemplate

lib/mix/lib/mix/compilers/elixir.ex

@@ -84,7 +84,13 @@ defmodule Mix.Compilers.Elixir do
     stale_local_deps = stale_local_deps(manifest, modified)
 
     {modules, structs, changed} =
-      update_stale_entries(all_modules, all_sources, removed ++ changed, stale_local_deps, %{})
+      update_stale_entries(
+        all_modules,
+        all_sources,
+        removed ++ changed,
+        stale_local_deps,
+        stale_local_deps
+      )
 
     stale = changed -- removed
 

lib/mix/lib/mix/shell.ex

@@ -81,7 +81,7 @@ defmodule Mix.Shell do
     * `:quiet` - overrides the callback to no-op
 
   """
-  def cmd(command, options, callback) when is_function(callback, 1) do
+  def cmd(command, options \\ [], callback) when is_function(callback, 1) do
     callback =
       if Keyword.get(options, :quiet, false) do
         fn x -> x end

lib/mix/lib/mix/tasks/cmd.ex

@@ -18,6 +18,18 @@ defmodule Mix.Tasks.Cmd do
       mix cmd --app app1 --app app2 echo pwd
 
   Aborts when a command exits with a non-zero status.
+
+  ## Zombie OS processes
+
+  Beware that the Erlang VM does not terminate child processes
+  when it shuts down. Therefore, if you use `mix cmd` to start
+  long running processes and then shutdown the VM, it is likely
+  that those child processes won't be terminated with the VM.
+
+  A solution is to make sure the child processes listen to the
+  stdndard input and terminate when standard input is closed.
+  We discuss this topic at length in the "Zombie OS processes"
+  of the `Port` module documentation.
   """
 
   def run(args) do

lib/mix/lib/mix/tasks/compile.erlang.ex

@@ -165,6 +165,9 @@ defmodule Mix.Tasks.Compile.Erlang do
       {:attribute, _, :behaviour, behaviour} ->
         %{erl | behaviours: [behaviour | erl.behaviours]}
 
+      {:attribute, _, :behavior, behaviour} ->
+        %{erl | behaviours: [behaviour | erl.behaviours]}
+
       {:attribute, _, :compile, value} when is_list(value) ->
         %{erl | compile: value ++ erl.compile}
 

lib/mix/lib/mix/tasks/format.ex

@@ -13,58 +13,62 @@ defmodule Mix.Tasks.Format do
 
   ## Formatting options
 
-  Formatting is done with the `Code.format_string!/2` function.
-  For complete list of formatting options please refer to its
-  description.
-  A `.formatter.exs` file can also be defined for customizing input
-  files and the formatter itself.
-
-  ## Task-specific options
-
-    * `--check-formatted` - check that the file is already formatted.
-      This is useful in pre-commit hooks and CI scripts if you want to
-      reject contributions with unformatted code. However, keep in mind,
-      that the formatting output may differ between Elixir versions as
-      improvements and fixes are applied to the formatter.
-
-    * `--check-equivalent` - check if the file after formatting has the
-      same AST. If the ASTs are not equivalent, it is a bug in the code
-      formatter. This option is recommended if you are automatically
-      formatting files.
-
-    * `--dry-run` - do not save files after formatting.
-
-    * `--dot-formatter` - the file with formatter configuration.
-      Defaults to `.formatter.exs` if one is available, see next section.
-
-  If any of the `--check-*` flags are given and a check fails, the formatted
-  contents won't be written to disk nor printed to stdout.
-
-  ## `.formatter.exs`
-
   The formatter will read a `.formatter.exs` in the current directory for
-  formatter configuration. It should return a keyword list with any of the
-  options supported by `Code.format_string!/2`.
+  formatter configuration. Evaluating this file should return a keyword list
+  with any of the options supported by `Code.format_string!/2`.
 
   The `.formatter.exs` also supports other options:
 
     * `:inputs` (a list of paths and patterns) - specifies the default inputs
       to be used by this task. For example, `["mix.exs", "{config,lib,test}/**/*.{ex,exs}"]`.
 
+    * `:subdirectories` (a list of paths and patterns) - specifies subdirectories
+      that have their own formatting rules. Each subdirectory should have a
+      `.formatter.exs` that configures how entries in that subdirectory should be
+      formatted as. Configuration between `.formatter.exs` are not shared nor
+      inherited. If a `.formatter.exs` lists "lib/app" as a subdirectory, the rules
+      in `.formatter.exs` won't be available in `lib/app/.formatter.exs`.
+      Note that the parent `.formatter.exs` must not specify files inside the "lib/app"
+      subdirectory in its `:inputs` configuration. If this happens, the behaviour of
+      which formatter configuration will be picked is unspecified.
+
     * `:import_deps` (a list of dependencies as atoms) - specifies a list
        of dependencies whose formatter configuration will be imported.
-       See the "Importing dependencies configuration" section below for more
-       information.
+       When specified, the formatter should run in the same directory as
+       the `mix.exs` file that defines those depednencies. See the "Importing
+       dependencies configuration" section below for more information.
 
-    * `:export` (a keyword list) - specifies formatter configuration to be exported. See the
-      "Importing dependencies configuration" section below.
+    * `:export` (a keyword list) - specifies formatter configuration to be exported.
+      See the "Importing dependencies configuration" section below.
+
+  ## Task-specific options
+
+    * `--check-formatted` - check that the file is already formatted.
+      This is useful in pre-commit hooks and CI scripts if you want to
+      reject contributions with unformatted code. However keep in mind
+      that the formatted output may differ between Elixir versions as
+      improvements and fixes are applied to the formatter.
+
+    * `--check-equivalent` - check if the files after formatting have the
+      same AST as before formatting. If the ASTs are not equivalent,
+      it is a bug in the code formatter. This option is recommended if you
+      are automatically formatting files.
+
+    * `--dry-run` - do not save files after formatting.
+
+    * `--dot-formatter` - path to the file with formatter configuration.
+      Defaults to `.formatter.exs` if one is available. See the "`.formatter.exs`"
+      section for more information.
+
+  If any of the `--check-*` flags are given and a check fails, the formatted
+  contents won't be written to disk nor printed to standard output.
 
   ## When to format code
 
-  We recommend developers to format code directly in their editors. Either
-  automatically on save or via an explicit command/key binding. If such option
-  is not yet available in your editor of choice, adding the required integration
-  is relatively simple as it is a matter of invoking
+  We recommend developers to format code directly in their editors, either
+  automatically when saving a file or via an explicit command or key binding. If
+  such option is not yet available in your editor of choice, adding the required
+  integration is usually a matter of invoking:
 
       cd $project && mix format $file
 
@@ -72,7 +76,7 @@ defmodule Mix.Tasks.Format do
   project.
 
   It is also possible to format code across the whole project by passing a list
-  of patterns and files to `mix format`, as showed at the top of this task
+  of patterns and files to `mix format`, as shown at the top of this task
   documentation. This list can also be set in the `.formatter.exs` under the
   `:inputs` key.
 
@@ -80,15 +84,15 @@ defmodule Mix.Tasks.Format do
 
   This task supports importing formatter configuration from dependencies.
 
-  A dependency that wants to export formatter configuration needs to have a `.formatter.exs` file
-  at the root of the project. In this file, the dependency can export a `:export` option with
-  configuration to export. For now, only one option is supported under `:export`:
-  `:locals_without_parens` (whose value has the same shape as the value of the
-  `:locals_without_parens` in `Code.format_string!/2`).
+  A dependency that wants to export formatter configuration needs to have a
+  `.formatter.exs` file at the root of the project. In this file, the dependency
+  can export a `:export` option with configuration to export. For now, only one
+  option is supported under `:export`: `:locals_without_parens` (whose value has
+  the same shape as the value of the `:locals_without_parens` in `Code.format_string!/2`).
 
-  The functions listed under `:locals_without_parens` in the `:export` option of a dependency
-  can be imported in a project by listing that dependency in the `:import_deps`
-  option of the formatter configuration file of the project.
+  The functions listed under `:locals_without_parens` in the `:export` option of
+  a dependency can be imported in a project by listing that dependency in the
+  `:import_deps` option of the formatter configuration file of the project.
 
   For example, consider I have a project `my_app` that depends on `my_dep`.
   `my_dep` wants to export some configuration, so `my_dep/.formatter.exs`
@@ -121,94 +125,153 @@ defmodule Mix.Tasks.Format do
     dry_run: :boolean
   ]
 
-  @deps_manifest "cached_formatter_deps"
+  @manifest "cached_dot_formatter"
+  @manifest_vsn 1
 
   def run(args) do
     {opts, args} = OptionParser.parse!(args, strict: @switches)
-    formatter_opts = eval_dot_formatter(opts)
-    formatter_opts = fetch_deps_opts(formatter_opts)
+    {dot_formatter, formatter_opts} = eval_dot_formatter(opts)
+
+    {formatter_opts_and_subs, _sources} =
+      eval_deps_and_subdirectories(dot_formatter, [], formatter_opts, [dot_formatter])
 
     args
-    |> expand_args(formatter_opts)
-    |> Task.async_stream(&format_file(&1, opts, formatter_opts), ordered: false, timeout: 30000)
+    |> expand_args(dot_formatter, formatter_opts_and_subs)
+    |> Task.async_stream(&format_file(&1, opts), ordered: false, timeout: 30000)
     |> Enum.reduce({[], [], []}, &collect_status/2)
     |> check!()
   end
 
+  @doc """
+  Returns formatter options to be used for the given file.
+  """
+  def formatter_opts_for_file(file, opts \\ []) do
+    {dot_formatter, formatter_opts} = eval_dot_formatter(opts)
+
+    {formatter_opts_and_subs, _sources} =
+      eval_deps_and_subdirectories(dot_formatter, [], formatter_opts, [dot_formatter])
+
+    split = file |> Path.relative_to_cwd() |> Path.split()
+    find_formatter_opts_for_file(split, formatter_opts_and_subs)
+  end
+
   defp eval_dot_formatter(opts) do
-    case dot_formatter(opts) do
-      {:ok, dot_formatter} -> eval_file_with_keyword_list(dot_formatter)
-      :error -> []
-    end
-  end
-
-  defp dot_formatter(opts) do
     cond do
-      dot_formatter = opts[:dot_formatter] -> {:ok, dot_formatter}
-      File.regular?(".formatter.exs") -> {:ok, ".formatter.exs"}
-      true -> :error
-    end
-  end
+      dot_formatter = opts[:dot_formatter] ->
+        {dot_formatter, eval_file_with_keyword_list(dot_formatter)}
 
-  # This function reads exported configuration from the imported dependencies and deals with
-  # caching the result of reading such configuration in a manifest file.
-  defp fetch_deps_opts(formatter_opts) do
-    deps = Keyword.get(formatter_opts, :import_deps, [])
-
-    cond do
-      deps == [] ->
-        formatter_opts
-
-      is_list(deps) ->
-        # Since we have dependencies listed, we write the manifest even if those dependencies
-        # don't export anything so that we avoid lookups everytime.
-        deps_manifest = Path.join(Mix.Project.manifest_path(), @deps_manifest)
-
-        dep_parenless_calls =
-          if deps_dot_formatters_stale?(deps_manifest) do
-            dep_parenless_calls = eval_deps_opts(deps)
-            write_deps_manifest(deps_manifest, dep_parenless_calls)
-            dep_parenless_calls
-          else
-            read_deps_manifest(deps_manifest)
-          end
-
-        Keyword.update(
-          formatter_opts,
-          :locals_without_parens,
-          dep_parenless_calls,
-          &(&1 ++ dep_parenless_calls)
-        )
+      File.regular?(".formatter.exs") ->
+        {".formatter.exs", eval_file_with_keyword_list(".formatter.exs")}
 
       true ->
-        Mix.raise("Expected :import_deps to return a list of dependencies, got: #{inspect(deps)}")
+        {".formatter.exs", []}
     end
   end
 
-  defp deps_dot_formatters_stale?(deps_manifest) do
-    Mix.Utils.stale?([".formatter.exs" | Mix.Project.config_files()], [deps_manifest])
+  # This function reads exported configuration from the imported
+  # dependencies and subdirectories and deals with caching the result
+  # of reading such configuration in a manifest file.
+  defp eval_deps_and_subdirectories(dot_formatter, prefix, formatter_opts, sources) do
+    deps = Keyword.get(formatter_opts, :import_deps, [])
+    subs = Keyword.get(formatter_opts, :subdirectories, [])
+
+    if not is_list(deps) do
+      Mix.raise("Expected :import_deps to return a list of dependencies, got: #{inspect(deps)}")
+    end
+
+    if not is_list(subs) do
+      Mix.raise("Expected :subdirectories to return a list of directories, got: #{inspect(subs)}")
+    end
+
+    if deps == [] and subs == [] do
+      {{formatter_opts, []}, sources}
+    else
+      manifest = Path.join(Mix.Project.manifest_path(), @manifest)
+
+      maybe_cache_in_manifest(dot_formatter, manifest, fn ->
+        {subdirectories, sources} = eval_subs_opts(subs, prefix, sources)
+        {{eval_deps_opts(formatter_opts, deps), subdirectories}, sources}
+      end)
+    end
   end
 
-  defp read_deps_manifest(deps_manifest) do
-    deps_manifest |> File.read!() |> :erlang.binary_to_term()
+  defp maybe_cache_in_manifest(dot_formatter, manifest, fun) do
+    cond do
+      is_nil(Mix.Project.get()) or dot_formatter != ".formatter.exs" -> fun.()
+      entry = read_manifest(manifest) -> entry
+      true -> write_manifest!(manifest, fun.())
+    end
   end
 
-  defp write_deps_manifest(deps_manifest, parenless_calls) do
-    File.mkdir_p!(Path.dirname(deps_manifest))
-    File.write!(deps_manifest, :erlang.term_to_binary(parenless_calls))
+  def read_manifest(manifest) do
+    with {:ok, binary} <- File.read(manifest),
+         {:ok, {@manifest_vsn, entry, sources}} <- safe_binary_to_term(binary),
+         expanded_sources = Enum.flat_map(sources, &Path.wildcard(&1, match_dot: true)),
+         false <- Mix.Utils.stale?(Mix.Project.config_files() ++ expanded_sources, [manifest]) do
+      {entry, sources}
+    else
+      _ -> nil
+    end
   end
 
-  defp eval_deps_opts(deps) do
+  defp safe_binary_to_term(binary) do
+    {:ok, :erlang.binary_to_term(binary)}
+  rescue
+    _ -> :error
+  end
+
+  defp write_manifest!(manifest, {entry, sources}) do
+    File.mkdir_p!(Path.dirname(manifest))
+    File.write!(manifest, :erlang.term_to_binary({@manifest_vsn, entry, sources}))
+    {entry, sources}
+  end
+
+  defp eval_deps_opts(formatter_opts, []) do
+    formatter_opts
+  end
+
+  defp eval_deps_opts(formatter_opts, deps) do
     deps_paths = Mix.Project.deps_paths()
 
-    for dep <- deps,
-        dep_path = assert_valid_dep_and_fetch_path(dep, deps_paths),
-        dep_dot_formatter = Path.join(dep_path, ".formatter.exs"),
-        File.regular?(dep_dot_formatter),
-        dep_opts = eval_file_with_keyword_list(dep_dot_formatter),
-        parenless_call <- dep_opts[:export][:locals_without_parens] || [],
-        uniq: true,
-        do: parenless_call
+    parenless_calls =
+      for dep <- deps,
+          dep_path = assert_valid_dep_and_fetch_path(dep, deps_paths),
+          dep_dot_formatter = Path.join(dep_path, ".formatter.exs"),
+          File.regular?(dep_dot_formatter),
+          dep_opts = eval_file_with_keyword_list(dep_dot_formatter),
+          parenless_call <- dep_opts[:export][:locals_without_parens] || [],
+          uniq: true,
+          do: parenless_call
+
+    Keyword.update(
+      formatter_opts,
+      :locals_without_parens,
+      parenless_calls,
+      &(&1 ++ parenless_calls)
+    )
+  end
+
+  defp eval_subs_opts(subs, prefix, sources) do
+    {subs, sources} =
+      Enum.flat_map_reduce(subs, sources, fn sub, sources ->
+        prefix = Path.join(prefix ++ [sub])
+        {Path.wildcard(prefix), [Path.join(prefix, ".formatter.exs") | sources]}
+      end)
+
+    Enum.flat_map_reduce(subs, sources, fn sub, sources ->
+      sub_formatter = Path.join(sub, ".formatter.exs")
+
+      if File.exists?(sub_formatter) do
+        formatter_opts = eval_file_with_keyword_list(sub_formatter)
+
+        {formatter_opts_and_subs, sources} =
+          eval_deps_and_subdirectories(:in_memory, [sub], formatter_opts, sources)
+
+        {[{sub, formatter_opts_and_subs}], sources}
+      else
+        {[], sources}
+      end
+    end)
   end
 
   defp assert_valid_dep_and_fetch_path(dep, deps_paths) when is_atom(dep) do
@@ -226,7 +289,7 @@ defmodule Mix.Tasks.Format do
       :error ->
         Mix.raise(
           "Unknown dependency #{inspect(dep)} given to :import_deps in the formatter configuration. " <>
-            "The dependency is not listed in your mix.exs file"
+            "The dependency is not listed in your mix.exs for environment #{inspect(Mix.env())}"
         )
     end
   end
@@ -245,22 +308,20 @@ defmodule Mix.Tasks.Format do
     opts
   end
 
-  defp expand_args([], formatter_opts) do
-    if inputs = formatter_opts[:inputs] do
-      expand_files_and_patterns(List.wrap(inputs), ".formatter.exs")
-    else
+  defp expand_args([], dot_formatter, formatter_opts_and_subs) do
+    if no_entries_in_formatter_opts?(formatter_opts_and_subs) do
       Mix.raise(
         "Expected one or more files/patterns to be given to mix format " <>
-          "or for a .formatter.exs to exist with an :inputs key"
+          "or for a .formatter.exs to exist with an :inputs or :subdirectories key"
       )
     end
+
+    dot_formatter
+    |> expand_dot_inputs([], formatter_opts_and_subs, %{})
+    |> Enum.uniq()
   end
 
-  defp expand_args(files_and_patterns, _formatter_opts) do
-    expand_files_and_patterns(files_and_patterns, "command line")
-  end
-
-  defp expand_files_and_patterns(files_and_patterns, context) do
+  defp expand_args(files_and_patterns, _dot_formatter, {formatter_opts, subs}) do
     files =
       for file_or_pattern <- files_and_patterns,
           file <- stdin_or_wildcard(file_or_pattern),
@@ -269,16 +330,52 @@ defmodule Mix.Tasks.Format do
 
     if files == [] do
       Mix.raise(
-        "Could not find a file to format. The files/patterns from #{context} " <>
+        "Could not find a file to format. The files/patterns given to command line " <>
           "did not point to any existing file. Got: #{inspect(files_and_patterns)}"
       )
     end
 
-    files
+    for file <- files do
+      if file == :stdin do
+        {file, formatter_opts}
+      else
+        split = file |> Path.relative_to_cwd() |> Path.split()
+        {file, find_formatter_opts_for_file(split, {formatter_opts, subs})}
+      end
+    end
+  end
+
+  defp expand_dot_inputs(dot_formatter, prefix, {formatter_opts, subs}, acc) do
+    if no_entries_in_formatter_opts?({formatter_opts, subs}) do
+      Mix.raise("Expected :inputs or :subdirectories key in #{dot_formatter}")
+    end
+
+    map =
+      for input <- List.wrap(formatter_opts[:inputs]),
+          file <- Path.wildcard(Path.join(prefix ++ [input]), match_dot: true),
+          do: {file, formatter_opts},
+          into: %{}
+
+    Enum.reduce(subs, Map.merge(acc, map), fn {sub, formatter_opts_and_subs}, acc ->
+      sub_formatter = Path.join(sub, ".formatter.exs")
+      expand_dot_inputs(sub_formatter, [sub], formatter_opts_and_subs, acc)
+    end)
+  end
+
+  defp find_formatter_opts_for_file(split, {formatter_opts, subs}) do
+    Enum.find_value(subs, formatter_opts, fn {sub, formatter_opts_and_subs} ->
+      if List.starts_with?(split, Path.split(sub)) do
+        find_formatter_opts_for_file(split, formatter_opts_and_subs)
+      end
+    end)
+  end
+
+  defp no_entries_in_formatter_opts?({formatter_opts, subs}) do
+    is_nil(formatter_opts[:inputs]) and subs == []
   end
 
   defp stdin_or_wildcard("-"), do: [:stdin]
-  defp stdin_or_wildcard(path), do: Path.wildcard(path)
+  defp stdin_or_wildcard(path), do: Path.wildcard(path, match_dot: true)
 
   defp read_file(:stdin) do
     {IO.stream(:stdio, :line) |> Enum.to_list() |> IO.iodata_to_binary(), file: "stdin"}
@@ -288,7 +385,7 @@ defmodule Mix.Tasks.Format do
     {File.read!(file), file: file}
   end
 
-  defp format_file(file, task_opts, formatter_opts) do
+  defp format_file({file, formatter_opts}, task_opts) do
     {input, extra_opts} = read_file(file)
     output = IO.iodata_to_binary([Code.format_string!(input, extra_opts ++ formatter_opts), ?\n])
 

lib/mix/lib/mix/tasks/new.ex

@@ -255,7 +255,8 @@ defmodule Mix.Tasks.New do
   embed_template(:formatter_umbrella, """
   # Used by "mix format"
   [
-    inputs: ["mix.exs", "apps/*/mix.exs", "apps/*/{config,lib,test}/**/*.{ex,exs}"]
+    inputs: ["mix.exs", "config/*.exs"],
+    subdirectories: ["apps/*"]
   ]
   """)
 

lib/mix/lib/mix/tasks/profile.eprof.ex

@@ -152,7 +152,7 @@ defmodule Mix.Tasks.Profile.Eprof do
   def profile(fun, opts) do
     fun
     |> profile_and_analyse(opts)
-    |> print_output
+    |> print_output()
   end
 
   defp profile_and_analyse(fun, opts) do
@@ -165,11 +165,15 @@ defmodule Mix.Tasks.Profile.Eprof do
     :eprof.profile([], fun, matching_pattern(opts))
 
     results =
-      :eprof.dump()
-      |> extract_results
-      |> filter_results(opts)
-      |> sort_results(opts)
-      |> add_totals
+      Enum.map(:eprof.dump(), fn {pid, call_results} ->
+        parsed_calls =
+          call_results
+          |> filter_results(opts)
+          |> sort_results(opts)
+          |> add_totals()
+
+        {pid, parsed_calls}
+      end)
 
     :eprof.stop()
 
@@ -191,9 +195,6 @@ defmodule Mix.Tasks.Profile.Eprof do
     end
   end
 
-  defp extract_results([]), do: []
-  defp extract_results([{_pid, call_results}]), do: call_results
-
   defp filter_results(call_results, opts) do
     calls_opt = Keyword.get(opts, :calls, 0)
     time_opt = Keyword.get(opts, :time, 0)
@@ -227,14 +228,23 @@ defmodule Mix.Tasks.Profile.Eprof do
 
   @header ["#", "CALLS", "%", "TIME", "µS/CALL"]
 
-  defp print_output({0, _, _, _}), do: print_function_count(0)
+  defp print_output([]) do
+    print_function_count(0)
+  end
 
-  defp print_output({function_count, call_results, call_count, total_time}) do
+  defp print_output(results) do
+    Enum.each(results, &print_result/1)
+  end
+
+  defp print_result({pid, {function_count, call_results, call_count, total_time}}) do
     formatted_rows = Enum.map(call_results, &format_row(&1, total_time))
     formatted_total = format_total(total_time, call_count)
 
     column_lengths = column_lengths(@header, formatted_rows)
 
+    IO.puts("")
+
+    print_pid_row(pid)
     print_row(@header, column_lengths)
     print_row(formatted_total, column_lengths)
     Enum.each(formatted_rows, &print_row(&1, column_lengths))
@@ -244,6 +254,10 @@ defmodule Mix.Tasks.Profile.Eprof do
     print_function_count(function_count)
   end
 
+  defp print_pid_row(pid) do
+    IO.puts("Profile results of #{inspect(pid)}")
+  end
+
   defp format_row({{module, function, arity}, {count, time}}, total_time) do
     mfa = Exception.format_mfa(module, function, arity)
     time_percentage = :erlang.float_to_binary(100 * divide(time, total_time), [{:decimals, 2}])

lib/mix/lib/mix/tasks/run.ex

@@ -7,7 +7,7 @@ defmodule Mix.Tasks.Run do
   Starts and runs the current application.
 
   `mix run` can be used to start the current application dependencies
-  ant the application itself. For long running systems, this is typically
+  and the application itself. For long running systems, this is typically
   done with the `--no-halt` option:
 
       mix run --no-halt

lib/mix/lib/mix/tasks/xref.ex

@@ -63,7 +63,7 @@ defmodule Mix.Tasks.Xref do
   Prints a file dependency graph where an edge from `A` to `B` indicates
   that `A` depends on `B`.
 
-      mix xref graph --format dot
+      mix xref graph --format stats
 
   The following options are accepted:
 

lib/mix/test/mix/rebar_test.exs

@@ -249,6 +249,9 @@ defmodule Mix.RebarTest do
       end
     end
 
+    # We run only on Unix because Windows has a hard time
+    # removing the rebar executable after executed.
+    @tag [unix: true]
     test "applies variables from :system_env option when compiling dependencies for Rebar" do
       Mix.Project.push(RebarAsDepWithEnv)
 
@@ -294,6 +297,9 @@ defmodule Mix.RebarTest do
       end
     end
 
+    # We run only on Unix because Windows has a hard time
+    # removing the rebar executable after executed.
+    @tag [unix: true]
     test "applies variables from :system_env option when compiling dependencies for rebar3" do
       Mix.Project.push(Rebar3AsDep)
 

lib/mix/test/mix/tasks/format_test.exs

@@ -16,7 +16,7 @@ defmodule Mix.Tasks.FormatTest do
   end
 
   test "formats the given files", context do
-    in_tmp context.test, fn ->
+    in_tmp(context.test, fn ->
       File.write!("a.ex", """
       foo bar
       """)
@@ -26,11 +26,11 @@ defmodule Mix.Tasks.FormatTest do
       assert File.read!("a.ex") == """
              foo(bar)
              """
-    end
+    end)
   end
 
   test "formats the given pattern", context do
-    in_tmp context.test, fn ->
+    in_tmp(context.test, fn ->
       File.write!("a.ex", """
       foo bar
       """)
@@ -40,11 +40,11 @@ defmodule Mix.Tasks.FormatTest do
       assert File.read!("a.ex") == """
              foo(bar)
              """
-    end
+    end)
   end
 
   test "is a no-op if the file is already formatted", context do
-    in_tmp context.test, fn ->
+    in_tmp(context.test, fn ->
       File.write!("a.ex", """
       foo(bar)
       """)
@@ -52,11 +52,11 @@ defmodule Mix.Tasks.FormatTest do
       File.touch!("a.ex", {{2000, 1, 1}, {0, 0, 0}})
       Mix.Tasks.Format.run(["a.ex"])
       assert File.stat!("a.ex").mtime == {{2000, 1, 1}, {0, 0, 0}}
-    end
+    end)
   end
 
   test "does not write file to disk on dry-run", context do
-    in_tmp context.test, fn ->
+    in_tmp(context.test, fn ->
       File.write!("a.ex", """
       foo bar
       """)
@@ -66,11 +66,11 @@ defmodule Mix.Tasks.FormatTest do
       assert File.read!("a.ex") == """
              foo bar
              """
-    end
+    end)
   end
 
   test "reads file from stdin and prints to stdout", context do
-    in_tmp context.test, fn ->
+    in_tmp(context.test, fn ->
       File.write!("a.ex", """
       foo bar
       """)
@@ -87,11 +87,28 @@ defmodule Mix.Tasks.FormatTest do
       assert File.read!("a.ex") == """
              foo(bar)
              """
-    end
+    end)
+  end
+
+  test "reads file from stdin and prints to stdout with formatter", context do
+    in_tmp(context.test, fn ->
+      File.write!(".formatter.exs", """
+      [locals_without_parens: [foo: 1]]
+      """)
+
+      output =
+        capture_io("foo :bar", fn ->
+          Mix.Tasks.Format.run(["-"])
+        end)
+
+      assert output == """
+             foo :bar
+             """
+    end)
   end
 
   test "checks if file is formatted with --check-formatted", context do
-    in_tmp context.test, fn ->
+    in_tmp(context.test, fn ->
       File.write!("a.ex", """
       foo bar
       """)
@@ -110,7 +127,7 @@ defmodule Mix.Tasks.FormatTest do
       assert File.read!("a.ex") == """
              foo(bar)
              """
-    end
+    end)
   end
 
   test "checks if stdin is formatted with --check-formatted" do
@@ -129,7 +146,7 @@ defmodule Mix.Tasks.FormatTest do
   end
 
   test "checks if file is equivalent with --check-equivalent", context do
-    in_tmp context.test, fn ->
+    in_tmp(context.test, fn ->
       File.write!("a.ex", """
       foo bar
       """)
@@ -139,11 +156,11 @@ defmodule Mix.Tasks.FormatTest do
       assert File.read!("a.ex") == """
              foo(bar)
              """
-    end
+    end)
   end
 
   test "uses inputs and configuration from .formatter.exs", context do
-    in_tmp context.test, fn ->
+    in_tmp(context.test, fn ->
       File.write!(".formatter.exs", """
       [
         inputs: ["a.ex"],
@@ -160,11 +177,39 @@ defmodule Mix.Tasks.FormatTest do
       assert File.read!("a.ex") == """
              foo bar(baz)
              """
-    end
+    end)
+  end
+
+  test "expands patterns in inputs from .formatter.exs", context do
+    in_tmp(context.test, fn ->
+      File.write!(".formatter.exs", """
+      [
+        inputs: ["{a,.b}.ex"]
+      ]
+      """)
+
+      File.write!("a.ex", """
+      foo bar
+      """)
+
+      File.write!(".b.ex", """
+      foo bar
+      """)
+
+      Mix.Tasks.Format.run([])
+
+      assert File.read!("a.ex") == """
+             foo(bar)
+             """
+
+      assert File.read!(".b.ex") == """
+             foo(bar)
+             """
+    end)
   end
 
   test "uses inputs and configuration from --dot-formatter", context do
-    in_tmp context.test, fn ->
+    in_tmp(context.test, fn ->
       File.write!("custom_formatter.exs", """
       [
         inputs: ["a.ex"],
@@ -181,13 +226,65 @@ defmodule Mix.Tasks.FormatTest do
       assert File.read!("a.ex") == """
              foo bar(baz)
              """
-    end
+    end)
+  end
+
+  test "can read exported configuration from subdirectories", context do
+    in_tmp(context.test, fn ->
+      File.write!(".formatter.exs", """
+      [subdirectories: ["lib"]]
+      """)
+
+      File.mkdir_p!("lib")
+
+      File.write!("lib/.formatter.exs", """
+      [inputs: "a.ex", locals_without_parens: [my_fun: 2]]
+      """)
+
+      formatter_opts = Mix.Tasks.Format.formatter_opts_for_file("lib/extra/a.ex")
+      assert [my_fun: 2] = Keyword.get(formatter_opts, :locals_without_parens)
+
+      File.write!("lib/a.ex", """
+      my_fun :foo, :bar
+      other_fun :baz
+      """)
+
+      Mix.Tasks.Format.run([])
+
+      assert File.read!("lib/a.ex") == """
+             my_fun :foo, :bar
+             other_fun(:baz)
+             """
+
+      Mix.Tasks.Format.run(["lib/a.ex"])
+
+      assert File.read!("lib/a.ex") == """
+             my_fun :foo, :bar
+             other_fun(:baz)
+             """
+
+      # No caching without a project
+      manifest_path = Path.join(Mix.Project.manifest_path(), "cached_dot_formatter")
+      refute File.regular?(manifest_path)
+
+      # Caching with a project
+      Mix.Project.push(__MODULE__.FormatWithDepsApp)
+      Mix.Tasks.Format.run(["lib/a.ex"])
+      manifest_path = Path.join(Mix.Project.manifest_path(), "cached_dot_formatter")
+      assert File.regular?(manifest_path)
+
+      # Let's check that the manifest gets updated if it's stale.
+      File.touch!(manifest_path, {{1970, 1, 1}, {0, 0, 0}})
+
+      Mix.Tasks.Format.run(["lib/a.ex"])
+      assert File.stat!(manifest_path).mtime > {{1970, 1, 1}, {0, 0, 0}}
+    end)
   end
 
   test "can read exported configuration from dependencies", context do
     Mix.Project.push(__MODULE__.FormatWithDepsApp)
 
-    in_tmp context.test, fn ->
+    in_tmp(context.test, fn ->
       File.write!(".formatter.exs", """
       [import_deps: [:my_dep]]
       """)
@@ -208,21 +305,130 @@ defmodule Mix.Tasks.FormatTest do
              my_fun :foo, :bar
              """
 
-      manifest_path = Path.join(Mix.Project.manifest_path(), "cached_formatter_deps")
+      manifest_path = Path.join(Mix.Project.manifest_path(), "cached_dot_formatter")
       assert File.regular?(manifest_path)
 
       # Let's check that the manifest gets updated if it's stale.
       File.touch!(manifest_path, {{1970, 1, 1}, {0, 0, 0}})
 
+      formatter_opts = Mix.Tasks.Format.formatter_opts_for_file("a.ex")
+      assert [my_fun: 2] = Keyword.get(formatter_opts, :locals_without_parens)
+
       Mix.Tasks.Format.run(["a.ex"])
       assert File.stat!(manifest_path).mtime > {{1970, 1, 1}, {0, 0, 0}}
-    end
+    end)
+  end
+
+  test "can read exported configuration from dependencies and subdirectories", context do
+    Mix.Project.push(__MODULE__.FormatWithDepsApp)
+
+    in_tmp(context.test, fn ->
+      File.mkdir_p!("deps/my_dep/")
+
+      File.write!("deps/my_dep/.formatter.exs", """
+      [export: [locals_without_parens: [my_fun: 2]]]
+      """)
+
+      File.mkdir_p!("lib/sub")
+      File.mkdir_p!("lib/not_used_and_wont_raise")
+
+      File.write!(".formatter.exs", """
+      [subdirectories: ["lib"]]
+      """)
+
+      File.write!("lib/.formatter.exs", """
+      [subdirectories: ["*"]]
+      """)
+
+      File.write!("lib/sub/.formatter.exs", """
+      [inputs: "a.ex", import_deps: [:my_dep]]
+      """)
+
+      File.write!("lib/sub/a.ex", """
+      my_fun :foo, :bar
+      other_fun :baz
+      """)
+
+      Mix.Tasks.Format.run([])
+
+      assert File.read!("lib/sub/a.ex") == """
+             my_fun :foo, :bar
+             other_fun(:baz)
+             """
+
+      Mix.Tasks.Format.run(["lib/sub/a.ex"])
+
+      assert File.read!("lib/sub/a.ex") == """
+             my_fun :foo, :bar
+             other_fun(:baz)
+             """
+
+      # Update .formatter.exs, check that file is updated
+      File.write!("lib/sub/.formatter.exs", """
+      [inputs: "a.ex"]
+      """)
+
+      File.touch!("lib/sub/.formatter.exs", {{2030, 1, 1}, {0, 0, 0}})
+      Mix.Tasks.Format.run([])
+
+      assert File.read!("lib/sub/a.ex") == """
+             my_fun(:foo, :bar)
+             other_fun(:baz)
+             """
+
+      # Add a new entry to "lib" and it also gets picked.
+      File.mkdir_p!("lib/extra")
+
+      File.write!("lib/extra/.formatter.exs", """
+      [inputs: "a.ex", locals_without_parens: [other_fun: 1]]
+      """)
+
+      File.write!("lib/extra/a.ex", """
+      my_fun :foo, :bar
+      other_fun :baz
+      """)
+
+      File.touch!("lib/extra/.formatter.exs", {{2030, 1, 1}, {0, 0, 0}})
+      Mix.Tasks.Format.run([])
+
+      formatter_opts = Mix.Tasks.Format.formatter_opts_for_file("lib/extra/a.ex")
+      assert [other_fun: 1] = Keyword.get(formatter_opts, :locals_without_parens)
+
+      assert File.read!("lib/extra/a.ex") == """
+             my_fun(:foo, :bar)
+             other_fun :baz
+             """
+    end)
+  end
+
+  test "validates subdirectories in :subdirectories", context do
+    in_tmp(context.test, fn ->
+      File.write!(".formatter.exs", """
+      [subdirectories: "oops"]
+      """)
+
+      message = "Expected :subdirectories to return a list of directories, got: \"oops\""
+      assert_raise Mix.Error, message, fn -> Mix.Tasks.Format.run([]) end
+
+      File.write!(".formatter.exs", """
+      [subdirectories: ["lib"]]
+      """)
+
+      File.mkdir_p!("lib")
+
+      File.write!("lib/.formatter.exs", """
+      []
+      """)
+
+      message = "Expected :inputs or :subdirectories key in lib/.formatter.exs"
+      assert_raise Mix.Error, message, fn -> Mix.Tasks.Format.run([]) end
+    end)
   end
 
   test "validates dependencies in :import_deps", context do
     Mix.Project.push(__MODULE__.FormatWithDepsApp)
 
-    in_tmp context.test, fn ->
+    in_tmp(context.test, fn ->
       File.write!(".formatter.exs", """
       [import_deps: [:my_dep]]
       """)
@@ -239,14 +445,14 @@ defmodule Mix.Tasks.FormatTest do
 
       message =
         "Unknown dependency :nonexistent_dep given to :import_deps in the formatter configuration. " <>
-          "The dependency is not listed in your mix.exs file"
+          "The dependency is not listed in your mix.exs for environment :dev"
 
       assert_raise Mix.Error, message, fn -> Mix.Tasks.Format.run([]) end
-    end
+    end)
   end
 
   test "raises on invalid arguments", context do
-    in_tmp context.test, fn ->
+    in_tmp(context.test, fn ->
       assert_raise Mix.Error, ~r"Expected one or more files\/patterns to be given", fn ->
         Mix.Tasks.Format.run([])
       end
@@ -254,11 +460,11 @@ defmodule Mix.Tasks.FormatTest do
       assert_raise Mix.Error, ~r"Could not find a file to format", fn ->
         Mix.Tasks.Format.run(["unknown.whatever"])
       end
-    end
+    end)
   end
 
   test "raises SyntaxError when parsing invalid source file", context do
-    in_tmp context.test, fn ->
+    in_tmp(context.test, fn ->
       File.write!("a.ex", """
       defmodule <%= module %>.Bar do end
       """)
@@ -268,6 +474,6 @@ defmodule Mix.Tasks.FormatTest do
       end
 
       assert_received {:mix_shell, :error, ["mix format failed for file: a.ex"]}
-    end
+    end)
   end
 end

lib/mix/test/mix/tasks/profile.eprof_test.exs

@@ -18,6 +18,14 @@ defmodule Mix.Tasks.Profile.EprofTest do
     end
   end
 
+  test "profiles evaluated expression in multiple processes", context do
+    in_tmp context.test, fn ->
+      assert capture_io(fn ->
+               Eprof.run(["-e", "spawn(fn -> #{@expr} end)"])
+             end) =~ ~r(String\.Chars\.Integer\.to_string\/1\s+\d)
+    end
+  end
+
   test "profiles the script", context do
     in_tmp context.test, fn ->
       profile_script_name = "profile_script.ex"

lib/mix/test/mix/umbrella_test.exs

@@ -326,7 +326,7 @@ defmodule Mix.UmbrellaTest do
     end
   end
 
-  test "recompiles after path dependency changes" do
+  test "recompiles after runtime path dependency changes" do
     in_fixture("umbrella_dep/deps/umbrella/apps", fn ->
       Mix.Project.in_project(:bar, "bar", fn _ ->
         Mix.Task.run("compile", ["--verbose"])
@@ -360,22 +360,51 @@ defmodule Mix.UmbrellaTest do
         # Noop for runtime dependencies
         mtime = File.stat!("_build/dev/lib/bar/.mix/compile.elixir").mtime
         ensure_touched("_build/dev/lib/foo/ebin/Elixir.Foo.beam", mtime)
-
-        mtime = File.stat!("_build/dev/lib/bar/.mix/compile.elixir").mtime
         ensure_touched("_build/dev/lib/foo/.mix/compile.elixir", mtime)
 
         assert Mix.Tasks.Compile.Elixir.run(["--verbose"]) == {:noop, []}
+      end)
+    end)
+  end
+
+  test "recompiles after compile time path dependency changes" do
+    in_fixture("umbrella_dep/deps/umbrella/apps", fn ->
+      Mix.Project.in_project(:bar, "bar", fn _ ->
+        Mix.Task.run("compile", ["--verbose"])
 
         # Add compile time dependency
         File.write!("lib/bar.ex", "defmodule Bar, do: Foo.foo")
+
         assert Mix.Tasks.Compile.Elixir.run(["--verbose"]) == {:ok, []}
         assert_receive {:mix_shell, :info, ["Compiled lib/bar.ex"]}
 
         # Recompiles for compile time dependencies
         mtime = File.stat!("_build/dev/lib/bar/.mix/compile.elixir").mtime
         ensure_touched("_build/dev/lib/foo/ebin/Elixir.Foo.beam", mtime)
+        ensure_touched("_build/dev/lib/foo/.mix/compile.elixir", mtime)
 
+        assert Mix.Tasks.Compile.Elixir.run(["--verbose"]) == {:ok, []}
+        assert_receive {:mix_shell, :info, ["Compiled lib/bar.ex"]}
+      end)
+    end)
+  end
+
+  test "recompiles after struct path dependency changes" do
+    in_fixture("umbrella_dep/deps/umbrella/apps", fn ->
+      Mix.Project.in_project(:bar, "bar", fn _ ->
+        File.write!("../foo/lib/foo.ex", "defmodule Foo, do: defstruct [:bar]")
+
+        Mix.Task.run("compile", ["--verbose"])
+
+        # Add struct dependency
+        File.write!("lib/bar.ex", "defmodule Bar, do: %Foo{bar: true}")
+
+        assert Mix.Tasks.Compile.Elixir.run(["--verbose"]) == {:ok, []}
+        assert_receive {:mix_shell, :info, ["Compiled lib/bar.ex"]}
+
+        # Recompiles for struct dependencies
         mtime = File.stat!("_build/dev/lib/bar/.mix/compile.elixir").mtime
+        ensure_touched("_build/dev/lib/foo/ebin/Elixir.Foo.beam", mtime)
         ensure_touched("_build/dev/lib/foo/.mix/compile.elixir", mtime)
 
         assert Mix.Tasks.Compile.Elixir.run(["--verbose"]) == {:ok, []}

lib/mix/test/mix/utils_test.exs

@@ -48,34 +48,31 @@ defmodule Mix.UtilsTest do
   end
 
   test "symlink or copy" do
-    in_fixture "archive", fn ->
+    in_fixture("archive", fn ->
       File.mkdir_p!("_build/archive")
       result = Mix.Utils.symlink_or_copy(Path.expand("ebin"), Path.expand("_build/archive/ebin"))
       assert_ebin_symlinked_or_copied(result)
-    end
+    end)
   end
 
   test "symlink or copy removes previous directories" do
-    in_fixture "archive", fn ->
+    in_fixture("archive", fn ->
       File.mkdir_p!("_build/archive/ebin")
       result = Mix.Utils.symlink_or_copy(Path.expand("ebin"), Path.expand("_build/archive/ebin"))
       assert_ebin_symlinked_or_copied(result)
-    end
+    end)
   end
 
-  @windows? match?({:win32, _}, :os.type())
-  unless @windows? do
-    test "symlink or copy erases wrong symlinks" do
-      in_fixture "archive", fn ->
-        File.mkdir_p!("_build/archive")
-        Mix.Utils.symlink_or_copy(Path.expand("priv"), Path.expand("_build/archive/ebin"))
+  @tag unix: true
+  test "symlink or copy erases wrong symlinks" do
+    in_fixture("archive", fn ->
+      File.mkdir_p!("_build/archive")
+      build_ebin = Path.expand("_build/archive/ebin")
+      Mix.Utils.symlink_or_copy(Path.expand("priv"), build_ebin)
 
-        result =
-          Mix.Utils.symlink_or_copy(Path.expand("ebin"), Path.expand("_build/archive/ebin"))
-
-        assert_ebin_symlinked_or_copied(result)
-      end
-    end
+      result = Mix.Utils.symlink_or_copy(Path.expand("ebin"), build_ebin)
+      assert_ebin_symlinked_or_copied(result)
+    end)
   end
 
   test "proxy_config reads from env and returns credentials" do

lib/mix/test/test_helper.exs

@@ -1,7 +1,9 @@
 Mix.start()
 Mix.shell(Mix.Shell.Process)
 Application.put_env(:mix, :colors, enabled: false)
-ExUnit.start(trace: "--trace" in System.argv())
+
+exclude = if match?({:win32, _}, :os.type()), do: [unix: true], else: [windows: true]
+ExUnit.start(trace: "--trace" in System.argv(), exclude: exclude)
 
 unless {1, 7, 4} <= Mix.SCM.Git.git_version() do
   IO.puts(:stderr, "Skipping tests with git sparse checkouts...")