Release v1.6.6

Erlang/OTP 21.0 only forwards certain events to the error_logger,
which means we need to bring the full logger implementation from
master.

.appveyor.yml

@@ -11,8 +11,8 @@ test_script:
 environment:
   ELIXIR_ASSERT_TIMEOUT: 2000
 
-configuration: Test
-
 matrix:
   allow_failures:
-    - configuration: Test
+    - platform: x86
+    - platform: x64
+    - platform: Any CPU

CHANGELOG.md

@@ -1,16 +1,69 @@
 # Changelog for Elixir v1.6
 
-## Compiler diagnostics
-
-TODO.
-
 ## Code formatter
 
-TODO.
+The big feature in Elixir v1.6 is the addition of a code formatter and an accompanying `mix format` task that adds automatic formatting to your projects.
 
-## Stream data and property testing
+The goal of the formatter is to automate the styling of codebases into a unique and consistent layout used across teams and the whole community. Code is now easier to write, as you no longer need to concern yourself with formatting rules. Code is also easier to read, as you no longer need to convert the styles of other developers in your mind.
 
-TODO.
+The formatter also helps new developers to learn the language, by giving immediate feedback on code structure, and eases code reviews by allowing teams to focus on business rules and code quality, rather than code style.
+
+To automatically format your codebase, you can run the new `mix format` task. A `.formatter.exs` file may be added to your project root for rudimentary formatter configuration. The mix task also supports flags for CI integration. For instance, you can make your build or a Pull Request fail if the code is not formatted accordingly. We also recommend developers to check their favorite editor and see if they already provide key bindings for `mix format`, allowing a file or a code snippet to be formatted without ceremony.
+
+The Elixir codebase itself has been already fully formatted and all further contributions are expected to contain formatted code. We recommend existing codebases to be formatted in steps. While the formatter will correctly handle long lines and complex expressions, refactoring the code by breaking those into variables or smaller functions as you format them will lead to overall cleaner and more readable codebases.
+
+## Dynamic Supervisor
+
+Supervisors in Elixir are responsible for starting, shutting down and restarting child process when things go wrong. Most of the interaction with supervisors happen with the Supervisor module and it contains three main strategies: `:one_for_one`, `:rest_for_one` and `:one_for_all`.
+
+However, sometimes the children of a supervisor are not known upfront and are rather started dynamically. For example, if you are building a web server, you have each request beind handled by a separate supervised process. Those cases were handled in the Supervisor module under a special strategy called `:simple_one_for_one`.
+
+Unfortunately, this special strategy changed the semantics of the supervisor in regards to initialization and shutdown. Plus some APIs expected different inputs or would be completely unavailable depending on the supervision strategy.
+
+Elixir v1.6 addresses this issue by introducing a new `DynamicSupervisor` module, which encapsulates the old `:simple_one_for_one` strategy and APIs in a proper module while allowing the documentation and API of the `Supervisor` module to focus on its main use cases. Having a separate `DynamicSupervisor` module also makes it simpler to add new features to the dynamic supervisor, such as the new `:max_children` option that limits the maximum number of children supervised dynamically.
+
+## `@deprecated` and `@since` attributes
+
+This release also introduces two new attributes associated to function definitions: `@deprecated` and `@since`. The former marks if a function or macro is deprecated, the latter annotates the version the API was introduced:
+
+    @doc "Breaks a collection into chunks"
+    @since "1.0.0"
+    @deprecated "Use chunk_every/2 instead"
+    def chunk(collection, chunk_size) do
+      chunk_every(collection, chunk_size)
+    end
+
+The `mix xref` task was also updated to warn if your project calls deprecated code. So if a definition is marked as `@deprecated` and a module invokes it, a warning will be emitted during compilation. This effectively provides libraries and frameworks a mechanism to deprecate code without causing multiple warnings to be printed in runtime and without impacting performance.
+
+Note those attributes are not yet available to tools that generate documentation. Such functionality will be added in future releases as it requires changes to how Elixir stores documentation in BEAM files. We still recommend developers to properly annotate their APIs, as the information will then be already available when the tooling is updated.
+
+## defguard and defguardp
+
+Elixir provides the concepts of guards: expressions used alongside pattern matching to select a matching clause. Let's see an example straight from Elixir's home page:
+
+    def serve_drinks(%User{age: age}) when age >= 21 do
+      # Code that serves drinks!
+    end
+
+`%User{age: age}` is matching on a `User` struct with an age field and `when age >= 21` is the guard.
+
+Since only a handful of constructs are [allowed in guards](https://hexdocs.pm/elixir/guards.html#content), if you were in a situation where you had to check the age to be more than or equal to 21 in multiple times, extracting the guard to a separate function would be [less than obvious and error prone](https://github.com/elixir-lang/elixir/issues/2469). To address those issues, this release introduces `defguard/1` and `defguardp/1`:
+
+    defguard is_drinking_age(age) when age >= 21
+
+    def serve_drinks(%User{age: age}) when is_drinking_age(age) do
+      # Code that serves drinks!
+    end
+
+## IEx improvements
+
+IEx also got its share of improvements. The new code formatter allows us to pretty print code snippets, types and specifications, improving the overall experience when exploring code through the terminal.
+
+The autocomplete mechanism also got smarter, being able to provide context autocompletion. For example, typing `t Enum.` and hitting TAB will autocomplete only the types in Enum (in contrast to all functions). Typing `b GenServer.` and hitting TAB will autocomplete only the behaviour callbacks.
+
+Finally, the breakpoint functionality added in Elixir v1.5 has been improved to support pattern matching and guards. For example, to pattern match on a function call when the first argument is the atom `:foo`, you may do:
+
+    break! SomeFunction.call(:foo, _, _)
 
 ## mix xref
 
@@ -20,7 +73,7 @@ One of such additions is the `--include-siblings` option that can be given to al
 
     $ mix xref callers SomeModule --include-siblings
 
-The `graph` command in `mix xref` can also output general statistics about the graph. In the hexpm project, you would get:
+The `graph` command in `mix xref` now can also output general statistics about the graph. In [the hexpm project](https://github.com/hexpm/hexpm), you would get:
 
     $ mix xref graph --format stats
     Tracked files: 129 (nodes)
@@ -52,7 +105,7 @@ The `graph` command in `mix xref` can also output general statistics about the g
       * lib/hexpm/accounts/user.ex (19)
       * lib/hexpm/repo.ex (19)
 
-`mix xref graph` also get the `--only-nodes` and `--label` options. The former asks Mix to only output file names (nodes) without the edges. The latter allows you to focus on certain relationships:
+`mix xref graph` also got the `--only-nodes` and `--label` options. The former asks Mix to only output file names (nodes) without the edges. The latter allows you to focus on certain relationships:
 
       # To get all files that depend on lib/foo.ex
       mix xref graph --sink lib/foo.ex --only-nodes
@@ -68,7 +121,132 @@ The `graph` command in `mix xref` can also output general statistics about the g
 
 Those improvements will help developers better understand the relationship between files and reveal potentially complex parts of their systems.
 
-## v1.6.0-dev
+Other improvements in Mix include better compiler diagnostics for editor integration, support for the `--slowest N` flag in `mix test` that shows the slowest tests in your suite, and a new `mix profile.eprof` task that provides time based profiling, complementing the existing `mix profile.cprof` (count based) and `mix profile.fprof` (flame based).
+
+## v1.6.6 (2018-06-20)
+
+This release supports Erlang/OTP 21.0 by removing all warnings and by properly supporting the new Erlang logger module.
+
+### 1. Bug fixes
+
+#### Elixir
+
+  * [Base] Do not raise when finding bad digits in `Base.decode32!` with `case: :mixed`
+  * [Code] Preserve the user's choice when `fn` is followed by a newline and it has only a single clause
+  * [DynamicSupervisor] Properly account for restarting children in the `:max_children` configuration
+  * [String] Add performant impl for string upcase/downcase `:ascii` mode
+  * [Task.Supervisor] Fix type spec for `start_child/4`
+
+#### Logger
+
+  * [Logger] Do not crash truncation when truncate is set to infinity
+
+#### Mix
+
+  * [mix format] Match files starting with dot
+
+## v1.6.5 (2018-05-07)
+
+This release supports Erlang/OTP 21.0-rc by removing all warnings and by properly redirecting logger output. Note it is not guaranteed it will support Erlang/OTP 21.0 final.
+
+### 1. Bug fixes
+
+#### Elixir
+
+  * [Code] Preserve the user's choice in the formatter on parens call with next break fits
+  * [Code] Preserve the user's choice in the formatter on calls without parens when we have one argument per line
+  * [Code] Fix formatting when there is a tilde in the first element of a bitstring
+  * [Kernel] Support specsdiff flag on `__info__` spec clauses
+  * [Kernel] Do not exclude hygienic vars in `defguard`
+  * [Kernel.SpecialForms] Mark `for` comprehensions as generated to avoid dialyzer warnings
+  * [Macro] Make sure `Macro.to_string/2` emits valid quoted expressions
+  * [Task] Support `:infinity` timeout on `Task.yield_many/2`
+  * [Task.Supervisor] Do not crash spawning supervised tasks when the parent process is dead
+  * [URI] Fix parsing of URIs with trailing `?`
+
+## v1.6.4 (2018-03-16)
+
+### 1. Bug fixes
+
+#### Elixir
+
+  * [Code.Formatter] Do not double escape quoted keyword list identifiers
+  * [Kernel] Properly support `into: binary` in Erlang/OTP 20.3
+
+## v1.6.3 (2018-03-09)
+
+### 1. Enhancements
+
+#### Elixir
+
+  * [Code.Formatter] Support comments in the middle of pipelines, `when` and `|` expressions
+
+### 2. Bug fixes
+
+#### Elixir
+
+  * [Code.Formatter] Consider commas when breaking groups
+  * [Code.Formatter] Ensure proper precedence between `&` and operators
+  * [Code.Formatter] Consider `.formatter.exs` when formatting stdin
+
+#### Logger
+
+  * [Logger.Translator] Ensure logger doesn't crash when reporting named `DynamicSupervisor`
+
+## v1.6.2 (2018-02-28)
+
+### 1. Enhancements
+
+#### Mix
+
+  * [mix compile.erlang] Teach Mix erlang compiler alternative spelling for `-behavior` declaration
+  * [mix format] Support the `:subdirectories` configuration that points to other directories with their own `.formatter.exs` file. This is useful in umbrella applications. `mix new --umbrella` has also been changed to use this new configuration by default
+  * [mix format] Include the current environment for missing dependency errors
+
+### 2. Bug fixes
+
+#### Elixir
+
+  * [Code.Formatter] Ensure `->` does not exceed line length
+  * [DynamicSupervisor] Properly tag error reports generated by dynamic supervisors so they can be properly translated by `Logger`
+  * [DynamicSupervisor] Consider extra arguments during child restart
+  * [Kernel] Ensure arguments given to a guard defined with `defguard` are evaluated in the correct order
+  * [Module] Do not remove docs for previous function declaration when `@impl true` is used
+  * [Supervisor] Ensure `use Supervisor` properly adds the `@behaviour Supervisor` annotation
+
+#### Mix
+
+  * [Mix.Shell] Bring back `Mix.Shell.cmd/2` - this arity was defined via a default argument that was accidentally removed
+
+## v1.6.1 (2018-01-29)
+
+### 1. Enhancements
+
+#### Elixir
+
+  * [DynamicSupervisor] Implement `child_spec/1` for DynamicSupervisor
+  * [Kernel] Raise better error messages on invalid map syntax
+
+### 2. Bug fixes
+
+#### Elixir
+
+  * [Code.Formatter] Only rearrange `not in` operator if explicitly opted-in
+  * [Code.Formatter] Ensure `do` blocks do not exceed line length on calls with a single argument
+  * [Collectable] Support bitstrings in Collectable and for-comprehensions (regression in v1.6.0)
+  * [GenServer] Do not override user own `@opts` attribute
+  * [Enum] Reintroduce zipping of any enumerable of enumerables in `Enum.zip/1` (regression in v1.6.0)
+  * [Macro] Reorder kw blocks in `Macro.to_string/1` to avoid warnings
+  * [Protocol] Fix protocol consolidation when some chunks may be missing
+  * [Stream] Reintroduce zipping of any enumerable of enumerables in `Stream.zip/1` (regression in v1.6.0)
+  * [Supervisor] Do not override user own `@opts` attribute
+  * [Supervisor] Add `@spec` to second clause of `start_link/2`
+
+#### ExUnit
+
+  * [ExUnit.Case] Reintroduce `:case` in ExUnit setup/setup_all/test context
+
+## v1.6.0 (2018-01-17)
 
 ### 1. Enhancements
 
@@ -78,10 +256,15 @@ Those improvements will help developers better understand the relationship betwe
 
 #### Elixir
 
+  * [Calendar] Add truncate to `Time`, `DateTime` and `NaiveDateTime` to facilitate microsecond precision pruning
   * [Code] Add `format_string!/2` and `format_file!/2` for automatic code formatting
   * [Code] Support column annotations in quoted expressions with `columns: true` in `Code.string_to_quoted/2`
+  * [DynamicSupervisor] Add `DynamicSupervisor` designed to manage children that are added and removed dynamically
+  * [Exception] Make `Exception.blame/3` extensible by adding an optional `blame/2` callback to exceptions
+  * [Exception] Improve the printing of guards on blamed exceptions
   * [Enumerable] Add `Enumerable.slice/1` and optimize many `Enum` operations with the new protocol. This allows data-structures with index-based random access to provide a non-linear implementation
-  * [Inspect.Algebra] Add `:strict` and `:flex` breaks
+  * [Inspect] Show UTF-8 BOM on inspected strings
+  * [Inspect.Algebra] Add `:strict` and `:flex` breaks - this gives more control over the document fitting
   * [Inspect.Algebra] Allow a group to inherit the parent group break
   * [Inspect.Algebra] Add `force_unfit/1` and `next_break_fits/2` which give more control over document fitting
   * [Inspect.Algebra] Add `collapse_lines/1` for collapsing multiple lines to a maximum value
@@ -89,25 +272,42 @@ Those improvements will help developers better understand the relationship betwe
   * [Kernel] Prefix variables with V when emitting Erlang code. This improves the integration with tools such as Erlang code formatters and the GUI debugger
   * [Kernel] Warn on the use of `length(x) == 0` in guards
   * [Kernel] Warn if `catch` comes before `rescue` in try
+  * [Kernel] Warn if heredoc is outdented compared to its closing quotes
+  * [Kernel] Add `defguard/1` and `defguardp/1` to make it easier to build guard-safe macros
   * [Kernel.ParallelCompiler] Add `compile/2`, `compile_to_path/3` and `require/2` which provide detailed information about warnings and errors
+  * [Kernel.SpecialForms] Support the `uniq: true` flag in `for` comprehensions
+  * [Module] Introduce `@deprecated` and `@since` attributes
+  * [Module] Emit conflicting behaviour warnings if the same behaviour is given more than once
+  * [List] Rearrange equals and inserts for shorter diff scripts in `List.myers_difference/2`
+  * [Record] Allow `:macros` and `:includes` to be given to `Record.extract/2`
   * [Stream] Add `Stream.intersperse/2`
   * [String] Update to Unicode 10
   * [String] Allow passing empty string `match` to `String.replace/4`
-  * [Task] Allow a custom supervisor to be given to `Task.Supervisor.async_stream/3`
+  * [String] Support context and language sensitive operations in `String.upcase/2` and `String.downcase/2`. Currently only the `:greek` context is supported
+  * [String] Support `:ascii` conversion in `String.upcase/2` and `String.downcase/2`
   * [Time] Add `Time.add/3`
 
 #### ExUnit
 
+  * [ExUnit.Assertions] Perform inclusive checks in `assert_in_delta`
   * [ExUnit.Callbacks] Add `ExUnit.Callbacks.start_supervised!/2`
   * [ExUnit.Case] Generate a random seed per test based on the test suite seed
 
 #### IEx
 
+  * [IEx.Autocomplete] Provide contextual autocompletion: `t Enum.` will autocomplete types, `b Enum` will autocomplete callbacks
+  * [IEx.CLI] Provide hints for developers when a bad host name is given to `--remsh`
   * [IEx.Helpers] Automatically include specs when showing documentation for functions/macros
   * [IEx.Helpers] Improve formatting of behaviours and typespecs by using the formatter
+  * [IEx.Helpers] Allow pattern matching and guard expressions when on `IEx.break!`
+
+#### Logger
+
+  * [Logger] Add `:discard_threshold` to Logger to help with message queue overflow
 
 #### Mix
 
+  * [mix app.start] Add `--preload-modules` to `mix app.start`
   * [mix archive.build] Allow `mix archive.build` to bundle dot files via an option
   * [mix compile] Define a behavior for Mix compiler tasks and return diagnostics from compiler tasks
   * [mix compile] Track struct dependencies between files and recompile them only if the struct changes
@@ -116,26 +316,36 @@ Those improvements will help developers better understand the relationship betwe
   * [mix profile.eprof] Add a new task for time-based profiling with eprof
   * [mix test] Run all functions in a describe block by giving the `file:line` the describe block starts
   * [mix test] Report the top N slowest tests with the `--slowest N` flag
+  * [mix test] Report the number of doctests and tests separately
   * [mix xref] Support `--include-siblings` in reports for umbrella support
   * [mix xref] Add `mix xref graph --format stats`
   * [mix xref] Add `--only-nodes` and `--label` filters to mix xref graph
+  * [mix xref] Add `mix xref deprecated` that shows the callsite of deprecated functions
 
 ### 2. Bug fixes
 
 #### Elixir
 
   * [CLI] Support path with spaces as argument to elixir.bat
+  * [Inspect] Properly handle minus signal for non-decimal negative integers
+  * [Integer] Do not raise on non-integer values in `is_odd`/`is_even`
   * [Kernel] Solve a precedence issue between `&` and `|`, such as `[&Foo.bar/1 | &Baz.bat/2]`
   * [Kernel] Do not load dynamic Elixir modules as `:in_memory` as this value is not officially supported by the code server. Instead, use an empty list, which is the same value used by Erlang.
   * [Kernel] Validate variable struct name is atom when used in pattern matching
+  * [Kernel] No longer generate documentation for `defdelegate` functions automatically to avoid overriding previously specified `@doc`
   * [Macro] Fix `Macro.to_string/2` for tuple calls, such as `alias Foo.{Bar, Baz}`
   * [MapSet] Return valid MapSet when unioning a legacy MapSet
-  * [String] Properly downcase the greek sigma letter in `String.downcase/1`
+  * [Regex] Return a leading empty space when splitting on empty pattern. This makes the `split` operation consistent with the other operations in the `Regex` module
+  * [Stream] Ensure `Stream.chunk_while/4` does not emit more elements than necessary when halted
+  * [String] Return a leading empty space when splitting on empty string. This makes the `split` operation consistent with the other operations in the `String` module
   * [URI] Preserve empty fragments in `URI.parse/1`
 
 #### Mix
 
+  * [mix app.start] Improve the quality of reports if app fails to boot
+  * [mix cmd] Allow `mix cmd` to be invoked multiple times without marking it as executed
   * [mix deps] Ensure optional dependencies in umbrella applications are loaded
+  * [mix deps.update] Ensure transitive new non-Hex dependencies are also fetched when a repo is updated
   * [mix xref] Take compile dependencies with higher priority than runtime ones when building a graph
   * [mix xref] Handle external files for xref callers and warnings
 
@@ -143,14 +353,23 @@ Those improvements will help developers better understand the relationship betwe
 
 #### Elixir
 
+  * [GenServer] Warn if `init/1` is not defined in `GenServer`. This brings GenServer closer to the implementation in OTP and aligns all behaviours to require the `init/1` callback
   * [Inspect.Algebra] `surround/3` and `surround_many/6` are deprecated in favor of `container_doc/6`
+  * [Kernel] Specifying map types with variable keys without defining the type as required/optional is deprecated
   * [Kernel.ParallelCompiler] `files/2` and `files_to_path/3` are deprecated in favor of `compile/2` and `compile_to_path/3`
   * [Kernel.ParallelRequire] `files/2` is deprecated in favor of `Kernel.ParallelCompiler.require/2`
+  * [Supervisor] The `:simple_one_for_one` strategy is deprecated in favor of `DynamicSupervisor`
+  * [Supervisor] Passing a list of args to `Supervisor.start_child/2` is deprecated in favor of `DynamicSupervisor`
+  * [Task.Supervisor] Passing `:restart` and `:shutdown` to `Task.Supervisor.start_link/2` is deprecated (it should be passed on start child instead)
 
 #### ExUnit
 
   * [ExUnit.Formatter] `:case_started` and `:case_finished` events are deprecated in favor of `:module_started` and `:module_finished`
 
+#### Mix
+
+  * [Mix.Compilers.Erlang] Returning `{:ok, val} | :error` from custom Erlang compilers is deprecated in favor of `{:ok, val, warnings} | {:error, errors, warnings}`
+
 ### 4. Deprecations
 
 #### Elixir
@@ -158,6 +377,7 @@ Those improvements will help developers better understand the relationship betwe
   * [Enum] `Enum.partition/2` is deprecated in favor of `Enum.split_with/2`
   * [Keyword] `Keyword.replace/3` is deprecated in favor of `Keyword.fetch/2` and `Keyword.put/3`
   * [Map] `Map.replace/3` is deprecated in favor of `Map.fetch/2` and `Map.put/3`
+  * [Macro] `Macro.unescape_tokens/1` and `Macro.unescape_tokens/2` are deprecated in favor of `Enum.map/2`
   * [Range] Deprecate `Range.range?/1` in favor of pattern matching on `_ .. _`
 
 ## v1.5

Makefile

@@ -1,7 +1,7 @@
 REBAR ?= "$(CURDIR)/rebar"
 PREFIX ?= /usr/local
 SHARE_PREFIX ?= $(PREFIX)/share
-CANONICAL := master/
+CANONICAL := v1.6/
 ELIXIRC := bin/elixirc --verbose --ignore-module-conflict
 ERLC := erlc -I lib/elixir/include
 ERL := erl -I lib/elixir/include -noshell -pa lib/elixir/ebin
@@ -243,13 +243,13 @@ build_man: man/iex.1 man/elixir.1
 
 man/iex.1:
 	$(Q) cp man/iex.1.in man/iex.1
-	$(Q) sed -i.bak "/{COMMON}/r common" man/iex.1
+	$(Q) sed -i.bak "/{COMMON}/r man/common" man/iex.1
 	$(Q) sed -i.bak "/{COMMON}/d" man/iex.1
 	$(Q) rm -f man/iex.1.bak
 
 man/elixir.1:
 	$(Q) cp man/elixir.1.in man/elixir.1
-	$(Q) sed -i.bak "/{COMMON}/r common" man/elixir.1
+	$(Q) sed -i.bak "/{COMMON}/r man/common" man/elixir.1
 	$(Q) sed -i.bak "/{COMMON}/d" man/elixir.1
 	$(Q) rm -f man/elixir.1.bak
 

README.md

@@ -2,8 +2,6 @@
 =========
 [![Travis build](https://secure.travis-ci.org/elixir-lang/elixir.svg?branch=master
 "Build Status")](https://travis-ci.org/elixir-lang/elixir)
-[![Windows build](https://ci.appveyor.com/api/projects/status/macwuxq7aiiv61g1?svg=true)](https://ci.appveyor.com/project/josevalim/elixir)
-
 
 Elixir is a dynamic, functional language designed for building scalable and maintainable applications.
 
@@ -32,10 +30,10 @@ If tests pass, you are ready to move on to the [Getting Started guide][1]
 or to try Interactive Elixir by running `bin/iex` in your terminal.
 
 However, if tests fail, it is likely you have an outdated Erlang version
-(Elixir requires Erlang 18.0 or later). You can check your Erlang version
+(Elixir requires Erlang 19.0 or later). You can check your Erlang version
 by calling `erl` in the command line. You will see some information as follows:
 
-    Erlang/OTP 18 [erts-7.0] [source] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false]
+    Erlang/OTP 19 [erts-8.0] [smp:2:2] [async-threads:10] [kernel-poll:false]
 
 If you have properly set up your dependencies and tests still fail,
 you may want to open up a bug report, as explained next.

RELEASE.md

@@ -10,7 +10,7 @@ This document simply outlines the release process:
 
 3. Ensure CHANGELOG is updated and add current date
 
-4. If a new `vMAJOR.MINOR`, replace "master" with "vVERSION" in the "Compatibility and Deprecations" page and commit
+4. Update "Compatibility and Deprecations" if a new OTP version is supported. If a new `vMAJOR.MINOR`, replace "master" with "vVERSION" in the "Deprecations" section
 
 5. If a new `vMAJOR.MINOR`, create a new branch "vMAJOR.MINOR" and set `CANONICAL=` in Makefile
 

VERSION

@@ -1 +1 @@
-1.6.0-dev
+1.6.6

lib/elixir/docs.exs

@@ -4,29 +4,20 @@
   groups_for_modules: [
     # [Kernel, Kernel.SpecialForms],
 
-    "Data & Behaviours": [
-      Access,
+    "Basic Types": [
       Atom,
       Base,
       Bitwise,
       Calendar,
       Calendar.ISO,
       Date,
-      Date.Range,
       DateTime,
-      Enum,
       Exception,
       Float,
       Integer,
-      Keyword,
-      List,
-      Map,
-      MapSet,
       NaiveDateTime,
-      Range,
       Record,
       Regex,
-      Stream,
       String,
       Time,
       Tuple,
@@ -34,6 +25,18 @@
       Version,
     ],
 
+    "Collections & Enumerables": [
+      Access,
+      Date.Range,
+      Enum,
+      Keyword,
+      List,
+      Map,
+      MapSet,
+      Range,
+      Stream,
+    ],
+
     "IO & System": [
       File,
       File.Stat,

lib/elixir/lib/agent.ex

@@ -139,7 +139,12 @@ defmodule Agent do
   @typedoc "The agent state"
   @type state :: term
 
-  @doc false
+  @doc """
+  Returns a specification to start an agent under a supervisor.
+
+  See `Supervisor`.
+  """
+  @since "1.5.0"
   def child_spec(arg) do
     %{
       id: Agent,
@@ -149,17 +154,19 @@ defmodule Agent do
 
   @doc false
   defmacro __using__(opts) do
-    quote location: :keep do
-      @opts unquote(opts)
+    quote location: :keep, bind_quoted: [opts: opts] do
+      @doc """
+      Returns a specification to start this module under a supervisor.
 
-      @doc false
+      See `Supervisor`.
+      """
       def child_spec(arg) do
         default = %{
           id: __MODULE__,
           start: {__MODULE__, :start_link, [arg]}
         }
 
-        Supervisor.child_spec(default, @opts)
+        Supervisor.child_spec(default, unquote(Macro.escape(opts)))
       end
 
       defoverridable child_spec: 1

lib/elixir/lib/application.ex

@@ -79,10 +79,19 @@ defmodule Application do
     * [`:application` module](http://www.erlang.org/doc/man/application.html)
     * [Applications – OTP Design Principles](http://www.erlang.org/doc/design_principles/applications.html)
 
-  A developer may also implement the `stop/1` callback (automatically defined
-  by `use Application`) which does any application cleanup. It receives the
-  application state and can return any value. Note that shutting down the
-  supervisor is automatically handled by the VM.
+  When an application is shutting down, its `c:stop/1` callback is called after
+  the supervision tree has been stopped by the runtime. This callback allows the
+  application to do any final cleanup. The argument is the state returned by
+  `c:start/2`, if it did, or `[]` otherwise. The return value of `c:stop/1` is
+  ignored.
+
+  By using `Application`, modules get a default implementation of `c:stop/1`
+  that ignores its argument and returns `:ok`, but it can be overridden.
+
+  Application callback modules may also implement the optional callback
+  `c:prep_stop/1`. If present, `c:prep_stop/1` is invoked before the supervision
+  tree is terminated. Its argument is the state returned by `c:start/2`, if it did,
+  or `[]` otherwise, and its return value is passed to `c:stop/1`.
 
   An application without a supervision tree doesn't define an application
   module callback in the application definition in `mix.exs` file. Even though
@@ -186,15 +195,25 @@ defmodule Application do
               | {:error, reason :: term}
 
   @doc """
-  Called when an application is stopped.
+  Called before stopping the application.
 
-  This function is called when an application has stopped, i.e., when its
+  This function is called before the top-level supervisor is terminated. It
+  receives the state returned by `c:start/2`, if it did, or `[]` otherwise.
+  The return value is later passed to `c:stop/1`.
+  """
+  @callback prep_stop(state) :: state
+
+  @doc """
+  Called after an application has been stopped.
+
+  This function is called after an application has been stopped, i.e., after its
   supervision tree has been stopped. It should do the opposite of what the
-  `start/2` callback did, and should perform any necessary cleanup. The return
+  `c:start/2` callback did, and should perform any necessary cleanup. The return
   value of this callback is ignored.
 
-  `state` is the return value of the `start/2` callback or the return value of
-  the `prep_stop/1` function if the application module defines such a function.
+  `state` is the state returned by `c:start/2`, if it did, or `[]` otherwise.
+  If the optional callback `c:prep_stop/1` is present, `state` is its return
+  value instead.
 
   `use Application` defines a default implementation of this function which does
   nothing and just returns `:ok`.
@@ -212,7 +231,7 @@ defmodule Application do
   @callback start_phase(phase :: term, start_type, phase_args :: term) ::
               :ok | {:error, reason :: term}
 
-  @optional_callbacks start_phase: 3
+  @optional_callbacks start_phase: 3, prep_stop: 1
 
   @doc false
   defmacro __using__(_) do

lib/elixir/lib/base.ex

@@ -184,16 +184,19 @@ defmodule Base do
   end
 
   defp decode_char_clauses(alphabet, :mixed) when length(alphabet) == 32 do
-    alphabet
-    |> Stream.with_index()
-    |> Enum.flat_map(fn {encoding, value} = pair ->
-      if encoding in ?A..?Z do
-        [pair, {encoding - ?A + ?a, value}]
-      else
-        [pair]
-      end
-    end)
-    |> decode_clauses()
+    clauses =
+      alphabet
+      |> Stream.with_index()
+      |> Enum.flat_map(fn {encoding, value} = pair ->
+        if encoding in ?A..?Z do
+          [pair, {encoding - ?A + ?a, value}]
+        else
+          [pair]
+        end
+      end)
+      |> decode_clauses()
+
+    clauses ++ bad_digit_clause()
   end
 
   defp decode_mixed_clauses(first, second) do

lib/elixir/lib/calendar/iso.ex

@@ -74,16 +74,15 @@ defmodule Calendar.ISO do
       {2000, 1, 1, 12, 0, 0, {0, 6}}
 
   """
-  @spec naive_datetime_from_iso_days(Calendar.iso_days()) ::
-          {
-            Calendar.year(),
-            Calendar.month(),
-            Calendar.day(),
-            Calendar.hour(),
-            Calendar.minute(),
-            Calendar.second(),
-            Calendar.microsecond()
-          }
+  @spec naive_datetime_from_iso_days(Calendar.iso_days()) :: {
+          Calendar.year(),
+          Calendar.month(),
+          Calendar.day(),
+          Calendar.hour(),
+          Calendar.minute(),
+          Calendar.second(),
+          Calendar.microsecond()
+        }
   @impl true
   def naive_datetime_from_iso_days({days, day_fraction}) do
     {year, month, day} = date_from_iso_days(days)

lib/elixir/lib/code.ex

@@ -303,7 +303,7 @@ defmodule Code do
   The string concatenation makes the code fit on a single line and also
   gives more options to the formatter.
 
-  A similar example is when the formatter breaks a fuction definition
+  A similar example is when the formatter breaks a function definition
   over multiple clauses:
 
       def my_function(
@@ -343,14 +343,14 @@ defmodule Code do
 
       [foo, bar]
 
-  You can also force keywords to be rendered on multiple lines by
-  having each entry on its own line:
+  You can also force function calls and keywords to be rendered on multiple
+  lines by having each entry on its own line:
 
       defstruct name: nil,
                 age: 0
 
   The code above will be kept with one keyword entry per line by the
-  formatter. To avoid that, just keep everything on a single line.
+  formatter. To avoid that, just squash everything into a single line.
 
   ### Parens and no parens in function calls
 
@@ -379,10 +379,9 @@ defmodule Code do
                 arg2,
                 arg3
 
-  If the last argument is a data structure of variable length, such as
-  maps and lists, and the beginning of the data structure fits on the
-  same line as the function call, then no indentation happens, this
-  allows code like this:
+  If the last argument is a data structure, such as maps and lists, and
+  the beginning of the data structure fits on the same line as the function
+  call, then no indentation happens, this allows code like this:
 
       Enum.reduce(some_collection, initial_value, fn element, acc ->
         # code

lib/elixir/lib/code/formatter.ex

@@ -10,6 +10,7 @@ defmodule Code.Formatter do
   @min_line 0
   @max_line 9_999_999
   @empty empty()
+  @ampersand_prec Code.Identifier.unary_op(:&) |> elem(1)
 
   # Operators that do not have space between operands
   @no_space_binary_operators [:..]
@@ -17,8 +18,8 @@ defmodule Code.Formatter do
   # Operators that do not have newline between operands (as well as => and keywords)
   @no_newline_binary_operators [:\\, :in]
 
-  # Left associative operators that start on the next line in case of breaks
-  @left_new_line_before_binary_operators [:|>, :~>>, :<<~, :~>, :<~, :<~>, :<|>]
+  # Left associative operators that start on the next line in case of breaks (always pipes)
+  @pipeline_operators [:|>, :~>>, :<<~, :~>, :<~, :<~>, :<|>]
 
   # Right associative operators that start on the next line in case of breaks
   @right_new_line_before_binary_operators [:|, :when]
@@ -92,7 +93,9 @@ defmodule Code.Formatter do
 
     # Stdlib,
     defrecord: 2,
+    defrecord: 3,
     defrecordp: 2,
+    defrecordp: 3,
 
     # Testing
     all: :*,
@@ -251,6 +254,7 @@ defmodule Code.Formatter do
       |> Keyword.get(:locals_without_parens, [])
       |> MapSet.new()
       |> MapSet.union(@locals_without_parens)
+      |> MapSet.to_list()
 
     %{
       locals_without_parens: locals_without_parens,
@@ -274,7 +278,8 @@ defmodule Code.Formatter do
   defp next_eol('\r\n' ++ rest, count), do: next_eol(rest, count + 1)
   defp next_eol(_, count), do: count
 
-  defp previous_eol([{token, {_, _, count}} | _]) when token in [:eol, :",", :";"] and count > 0 do
+  defp previous_eol([{token, {_, _, count}} | _])
+       when token in [:eol, :",", :";"] and count > 0 do
     count
   end
 
@@ -322,12 +327,12 @@ defmodule Code.Formatter do
 
   # Special AST nodes from compiler feedback.
 
-  defp quoted_to_algebra({:special, :clause_args, [args]}, _context, state) do
+  defp quoted_to_algebra({{:special, :clause_args}, _meta, [args]}, _context, state) do
     {doc, state} = clause_args_to_algebra(args, state)
     {group(doc), state}
   end
 
-  defp quoted_to_algebra({:special, :bitstring_segment, [arg, last]}, _context, state) do
+  defp quoted_to_algebra({{:special, :bitstring_segment}, _meta, [arg, last]}, _context, state) do
     bitstring_segment_to_algebra({arg, -1}, state, last)
   end
 
@@ -415,11 +420,11 @@ defmodule Code.Formatter do
   # {}
   # {1, 2}
   defp quoted_to_algebra({:{}, meta, args}, _context, state) do
-    tuple_to_algebra(meta, args, state)
+    tuple_to_algebra(meta, args, :flex_break, state)
   end
 
   defp quoted_to_algebra({:__block__, meta, [{left, right}]}, _context, state) do
-    tuple_to_algebra(meta, [left, right], state)
+    tuple_to_algebra(meta, [left, right], :flex_break, state)
   end
 
   defp quoted_to_algebra({:__block__, meta, [list]}, _context, state) when is_list(list) do
@@ -451,7 +456,8 @@ defmodule Code.Formatter do
     {atom_to_algebra(atom), state}
   end
 
-  defp quoted_to_algebra({:__block__, meta, [integer]}, _context, state) when is_integer(integer) do
+  defp quoted_to_algebra({:__block__, meta, [integer]}, _context, state)
+       when is_integer(integer) do
     {integer_to_algebra(Keyword.fetch!(meta, :original)), state}
   end
 
@@ -504,12 +510,19 @@ defmodule Code.Formatter do
 
   # not(left in right)
   # left not in right
-  defp quoted_to_algebra({:not, meta, [{:in, _, [left, right]}]}, context, state) do
-    binary_op_to_algebra(:in, "not in", meta, left, right, context, state)
+  defp quoted_to_algebra({:not, meta, [{:in, _, [left, right]} = arg]}, context, state) do
+    %{rename_deprecated_at: since} = state
+
+    # TODO: Remove since check on Elixir v2.0 and the OP arrengement is removed.
+    if meta[:operator] == :"not in" || (since && Version.match?(since, "~> 1.5")) do
+      binary_op_to_algebra(:in, "not in", meta, left, right, context, state)
+    else
+      unary_op_to_algebra(:not, meta, arg, context, state)
+    end
   end
 
   defp quoted_to_algebra({:fn, meta, [_ | _] = clauses}, _context, state) do
-    anon_fun_to_algebra(clauses, line(meta), end_line(meta), state)
+    anon_fun_to_algebra(clauses, line(meta), end_line(meta), state, eol?(meta))
   end
 
   defp quoted_to_algebra({fun, meta, args}, context, state) when is_atom(fun) and is_list(args) do
@@ -542,7 +555,16 @@ defmodule Code.Formatter do
         {left, state} =
           case left_arg do
             {:__block__, _, [atom]} when is_atom(atom) ->
-              {atom |> Code.Identifier.inspect_as_key() |> string(), state}
+              key =
+                case Code.Identifier.classify(atom) do
+                  type when type in [:callable_local, :callable_operator, :not_callable] ->
+                    IO.iodata_to_binary([Atom.to_string(atom), ?:])
+
+                  _ ->
+                    IO.iodata_to_binary([?", Atom.to_string(atom), ?", ?:])
+                end
+
+              {string(key), state}
 
             {{:., _, [:erlang, :binary_to_atom]}, _, [{:<<>>, _, entries}, :utf8]} ->
               interpolation_to_algebra(entries, @double_quote, state, "\"", "\":")
@@ -558,7 +580,7 @@ defmodule Code.Formatter do
       end
 
     doc =
-      with_next_break_fits(next_break_fits?(right_arg), right, fn right ->
+      with_next_break_fits(next_break_fits?(right_arg, state), right, fn right ->
         concat(group(left), group(nest(glue(op, group(right)), 2, :break)))
       end)
 
@@ -659,23 +681,75 @@ defmodule Code.Formatter do
   # strict or flex mode around.
   defp binary_op_to_algebra(op, op_string, meta, left_arg, right_arg, context, state) do
     %{operand_nesting: nesting} = state
-    binary_op_to_algebra(op, op_string, meta, left_arg, right_arg, context, state, nil, nesting)
+    binary_op_to_algebra(op, op_string, meta, left_arg, right_arg, context, state, nesting)
   end
 
-  defp binary_op_to_algebra(
-         op,
-         op_string,
-         meta,
-         left_arg,
-         right_arg,
-         context,
-         state,
-         parent_info,
-         nesting
-       ) do
+  defp binary_op_to_algebra(op, op_string, meta, left_arg, right_arg, context, state, _nesting)
+       when op in @right_new_line_before_binary_operators do
     op_info = Code.Identifier.binary_op(op)
-    left_context = force_many_args_or_operand(context, :parens_arg)
-    right_context = force_many_args_or_operand(context, :operand)
+    op_string = op_string <> " "
+    left_context = left_op_context(context)
+    right_context = right_op_context(context)
+
+    min_line =
+      case left_arg do
+        {_, left_meta, _} -> line(left_meta)
+        _ -> line(meta)
+      end
+
+    {operands, max_line} =
+      unwrap_right(right_arg, op, meta, right_context, [{{:root, left_context}, left_arg}])
+
+    operand_to_algebra = fn
+      {{:root, context}, arg}, _args, newlines, state ->
+        {doc, state} = binary_operand_to_algebra(arg, context, state, op, op_info, :left, 2)
+        {doc, @empty, newlines, state}
+
+      {{kind, context}, arg}, _args, newlines, state ->
+        {doc, state} = binary_operand_to_algebra(arg, context, state, op, op_info, kind, 0)
+        doc = doc |> nest_by_length(op_string) |> force_keyword(arg)
+        {concat(op_string, doc), @empty, newlines, state}
+    end
+
+    operand_to_algebra_with_comments(
+      operands,
+      meta,
+      min_line,
+      max_line,
+      state,
+      operand_to_algebra
+    )
+  end
+
+  defp binary_op_to_algebra(op, _, meta, left_arg, right_arg, context, state, _nesting)
+       when op in @pipeline_operators do
+    op_info = Code.Identifier.binary_op(op)
+    left_context = left_op_context(context)
+    right_context = right_op_context(context)
+    max_line = line(meta)
+
+    {pipes, min_line} =
+      unwrap_pipes(left_arg, meta, left_context, [{{op, right_context}, right_arg}])
+
+    operand_to_algebra = fn
+      {{:root, context}, arg}, _args, newlines, state ->
+        {doc, state} = binary_operand_to_algebra(arg, context, state, op, op_info, :left, 2)
+        {doc, @empty, newlines, state}
+
+      {{op, context}, arg}, _args, newlines, state ->
+        op_info = Code.Identifier.binary_op(op)
+        op_string = Atom.to_string(op) <> " "
+        {doc, state} = binary_operand_to_algebra(arg, context, state, op, op_info, :right, 0)
+        {concat(op_string, doc), @empty, newlines, state}
+    end
+
+    operand_to_algebra_with_comments(pipes, meta, min_line, max_line, state, operand_to_algebra)
+  end
+
+  defp binary_op_to_algebra(op, op_string, meta, left_arg, right_arg, context, state, nesting) do
+    op_info = Code.Identifier.binary_op(op)
+    left_context = left_op_context(context)
+    right_context = right_op_context(context)
 
     {left, state} =
       binary_operand_to_algebra(left_arg, left_context, state, op, op_info, :left, 2)
@@ -692,42 +766,9 @@ defmodule Code.Formatter do
           op_string = " " <> op_string <> " "
           concat(concat(group(left), op_string), group(right))
 
-        op in @left_new_line_before_binary_operators ->
-          op_string = op_string <> " "
-
-          # If the parent is of the same type (computed via same precedence),
-          # we cannot group the left side yet.
-          left = if op_info == parent_info, do: left, else: group(left)
-
-          doc = glue(left, concat(op_string, group(right)))
-          if Keyword.get(meta, :eol, false), do: force_unfit(doc), else: doc
-
-        op in @right_new_line_before_binary_operators ->
-          op_string = op_string <> " "
-
-          # If the parent is of the same type (computed via same precedence),
-          # we need to nest the left side because of the associativity.
-          left =
-            if op_info == parent_info do
-              nest_by_length(left, op_string)
-            else
-              group(left)
-            end
-
-          # If the right side is of the same type, we will keep recursing
-          # and do the nesting on the left side later on (as written above).
-          right =
-            case right_arg do
-              {^op, _, [_, _]} -> right
-              _ -> right |> nest_by_length(op_string) |> force_keyword(right_arg) |> group()
-            end
-
-          doc = glue(left, concat(op_string, right))
-          if Keyword.get(meta, :eol, false), do: force_unfit(doc), else: doc
-
         true ->
           next_break_fits? =
-            op in @next_break_fits_operators and next_break_fits?(right_arg) and
+            op in @next_break_fits_operators and next_break_fits?(right_arg, state) and
               not Keyword.get(meta, :eol, false)
 
           with_next_break_fits(next_break_fits?, right, fn right ->
@@ -758,10 +799,11 @@ defmodule Code.Formatter do
   end
 
   defp binary_operand_to_algebra(operand, context, state, parent_op, parent_info, side, nesting) do
+    {parent_assoc, parent_prec} = parent_info
+
     with {op, meta, [left, right]} <- operand,
          op_info = Code.Identifier.binary_op(op),
          {_assoc, prec} <- op_info do
-      {parent_assoc, parent_prec} = parent_info
       op_string = Atom.to_string(op)
 
       cond do
@@ -769,17 +811,7 @@ defmodule Code.Formatter do
         # the correct side, we respect the nesting rule to avoid multiple
         # nestings. This only applies for left associativity or same operator.
         parent_prec == prec and parent_assoc == side and (side == :left or op == parent_op) ->
-          binary_op_to_algebra(
-            op,
-            op_string,
-            meta,
-            left,
-            right,
-            context,
-            state,
-            parent_info,
-            nesting
-          )
+          binary_op_to_algebra(op, op_string, meta, left, right, context, state, nesting)
 
         # If the parent requires parens or the precedence is inverted or
         # it is in the wrong side, then we *need* parenthesis.
@@ -788,16 +820,18 @@ defmodule Code.Formatter do
              parent_op in @required_parens_logical_binary_operands) or parent_prec > prec or
             (parent_prec == prec and parent_assoc != side) ->
           {operand, state} =
-            binary_op_to_algebra(op, op_string, meta, left, right, context, state, parent_info, 2)
+            binary_op_to_algebra(op, op_string, meta, left, right, context, state, 2)
 
           {wrap_in_parens(operand), state}
 
         # Otherwise, we rely on precedence but also nest.
         true ->
-          binary_op_to_algebra(op, op_string, meta, left, right, context, state, parent_info, 2)
+          binary_op_to_algebra(op, op_string, meta, left, right, context, state, 2)
       end
     else
-      {:&, _, [arg]} when not is_integer(arg) and side == :left ->
+      {:&, _, [arg]}
+      when not is_integer(arg) and side == :left
+      when not is_integer(arg) and parent_assoc == :left and parent_prec > @ampersand_prec ->
         {doc, state} = quoted_to_algebra(operand, context, state)
         {wrap_in_parens(doc), state}
 
@@ -806,6 +840,45 @@ defmodule Code.Formatter do
     end
   end
 
+  defp unwrap_pipes({op, meta, [left, right]}, _meta, context, acc)
+       when op in @pipeline_operators do
+    left_context = left_op_context(context)
+    right_context = right_op_context(context)
+    unwrap_pipes(left, meta, left_context, [{{op, right_context}, right} | acc])
+  end
+
+  defp unwrap_pipes(left, meta, context, acc) do
+    min_line =
+      case left do
+        {_, meta, _} -> line(meta)
+        _ -> line(meta)
+      end
+
+    {[{{:root, context}, left} | acc], min_line}
+  end
+
+  defp unwrap_right({op, meta, [left, right]}, op, _meta, context, acc) do
+    left_context = left_op_context(context)
+    right_context = right_op_context(context)
+    unwrap_right(right, op, meta, right_context, [{{:left, left_context}, left} | acc])
+  end
+
+  defp unwrap_right(right, _op, meta, context, acc) do
+    acc = [{{:right, context}, right} | acc]
+    {Enum.reverse(acc), line(meta)}
+  end
+
+  defp operand_to_algebra_with_comments(operands, meta, min_line, max_line, state, fun) do
+    {docs, comments?, state} =
+      quoted_to_algebra_with_comments(operands, [], min_line, max_line, 1, state, fun)
+
+    if comments? or Keyword.get(meta, :eol, false) do
+      {docs |> Enum.reduce(&line(&2, &1)) |> force_unfit(), state}
+    else
+      {docs |> Enum.reduce(&glue(&2, &1)), state}
+    end
+  end
+
   ## Module attributes
 
   # @Foo
@@ -887,7 +960,7 @@ defmodule Code.Formatter do
   # expression.{arguments}
   defp remote_to_algebra({{:., _, [target, :{}]}, meta, args}, _context, state) do
     {target_doc, state} = remote_target_to_algebra(target, state)
-    {call_doc, state} = tuple_to_algebra(meta, args, state)
+    {call_doc, state} = tuple_to_algebra(meta, args, :break, state)
     {concat(concat(target_doc, "."), call_doc), state}
   end
 
@@ -906,7 +979,8 @@ defmodule Code.Formatter do
   # Mod.function()
   # var.function
   # expression.function(arguments)
-  defp remote_to_algebra({{:., _, [target, fun]}, meta, args}, context, state) when is_atom(fun) do
+  defp remote_to_algebra({{:., _, [target, fun]}, meta, args}, context, state)
+       when is_atom(fun) do
     {target_doc, state} = remote_target_to_algebra(target, state)
     fun = remote_fun_to_algebra(target, fun, length(args), state)
     remote_doc = target_doc |> concat(".") |> concat(string(fun))
@@ -986,7 +1060,11 @@ defmodule Code.Formatter do
   # function(arguments)
   defp local_to_algebra(fun, meta, args, context, state) when is_atom(fun) do
     skip_parens =
-      if skip_parens?(fun, meta, args, state), do: :skip_unless_many_args, else: :skip_if_do_end
+      cond do
+        not Keyword.get(meta, :no_parens, false) -> :required
+        local_without_parens?(fun, args, state) -> :skip_unless_many_args
+        true -> :skip_if_do_end
+      end
 
     {{call_doc, state}, wrap_in_parens?} =
       call_args_to_algebra(args, meta, context, skip_parens, true, state)
@@ -1008,8 +1086,8 @@ defmodule Code.Formatter do
   #   * :required - never skip parens
   #
   defp call_args_to_algebra([], meta, _context, _parens, _list_to_keyword?, state) do
-    {args_doc, state} =
-      args_to_algebra_with_comments([], meta, false, false, false, state, &{&1, &2})
+    {args_doc, _join, state} =
+      args_to_algebra_with_comments([], meta, false, :none, :break, state, &{&1, &2})
 
     {{surround("(", args_doc, ")"), state}, false}
   end
@@ -1043,7 +1121,6 @@ defmodule Code.Formatter do
 
   defp call_args_to_algebra_no_blocks(meta, args, skip_parens?, list_to_keyword?, extra, state) do
     {left, right} = split_last(args)
-    generators_count = count_generators(args)
     {keyword?, right} = last_arg_to_keyword(right, list_to_keyword?)
 
     context =
@@ -1053,26 +1130,31 @@ defmodule Code.Formatter do
         if skip_parens?, do: :no_parens_arg, else: :parens_arg
       end
 
-    if left != [] and keyword? and skip_parens? and generators_count == 0 do
+    if left != [] and keyword? and skip_parens? and no_generators?(args) do
       call_args_to_algebra_with_no_parens_keywords(meta, left, right, context, extra, state)
     else
-      next_break_fits? = next_break_fits?(right)
-      force_keyword? = keyword? and force_keyword?(right)
-      non_empty_eol? = left != [] and not next_break_fits? and Keyword.get(meta, :eol, false)
-      force_unfit? = generators_count > 1 or force_keyword? or non_empty_eol?
       args = if keyword?, do: left ++ right, else: left ++ [right]
+      many_eol? = match?([_, _ | _], args) and Keyword.get(meta, :eol, false)
+      join = if force_args?(args) or many_eol?, do: :line, else: :break
 
-      {args_doc, state} =
+      next_break_fits? = join == :break and next_break_fits?(right, state)
+      last_arg_mode = if next_break_fits?, do: :next_break_fits, else: :none
+
+      {args_doc, _join, state} =
         args_to_algebra_with_comments(
           args,
           meta,
           skip_parens?,
-          next_break_fits?,
-          force_unfit?,
+          last_arg_mode,
+          join,
           state,
           &quoted_to_algebra(&1, context, &2)
         )
 
+      # If we have a single argument, then we won't have an option to break
+      # before the "extra" part, so we ungroup it and build it later.
+      args_doc = ungroup_if_group(args_doc)
+
       doc =
         if skip_parens? do
           " "
@@ -1097,14 +1179,17 @@ defmodule Code.Formatter do
 
   defp call_args_to_algebra_with_no_parens_keywords(meta, left, right, context, extra, state) do
     to_algebra_fun = &quoted_to_algebra(&1, context, &2)
+    join = if force_args?(left), do: :line, else: :break
 
-    {left_doc, state} =
-      args_to_algebra_with_comments(left, meta, true, false, false, state, to_algebra_fun)
+    {left_doc, _join, state} =
+      args_to_algebra_with_comments(left, meta, true, :force_comma, join, state, to_algebra_fun)
 
-    {right_doc, state} =
-      args_to_algebra_with_comments(right, meta, false, false, false, state, to_algebra_fun)
+    join = if force_args?(right) or force_args?(left ++ right), do: :line, else: :break
 
-    right_doc = "," |> glue(right_doc) |> force_keyword(right) |> group(:inherit)
+    {right_doc, _join, state} =
+      args_to_algebra_with_comments(right, meta, false, :none, join, state, to_algebra_fun)
+
+    right_doc = apply(Inspect.Algebra, join, []) |> concat(right_doc) |> group(:inherit)
 
     doc =
       with_next_break_fits(true, right_doc, fn right_doc ->
@@ -1120,17 +1205,17 @@ defmodule Code.Formatter do
     {doc, state}
   end
 
-  defp skip_parens?(fun, meta, args, %{locals_without_parens: locals_without_parens}) do
+  defp local_without_parens?(fun, args, %{locals_without_parens: locals_without_parens}) do
     length = length(args)
 
-    length > 0 and Keyword.get(meta, :no_parens, false) and
+    length > 0 and
       Enum.any?(locals_without_parens, fn {key, val} ->
         key == fun and (val == :* or val == length)
       end)
   end
 
-  defp count_generators(args) do
-    Enum.count(args, &match?({:<-, _, [_, _]}, &1))
+  defp no_generators?(args) do
+    not Enum.any?(args, &match?({:<-, _, [_, _]}, &1))
   end
 
   defp do_end_blocks([{{:__block__, meta, [:do]}, _} | _] = blocks) do
@@ -1187,7 +1272,8 @@ defmodule Code.Formatter do
     ["\n" | entries]
   end
 
-  defp interpolation_to_algebra([entry | entries], escape, state, acc, last) when is_binary(entry) do
+  defp interpolation_to_algebra([entry | entries], escape, state, acc, last)
+       when is_binary(entry) do
     acc = concat(acc, escape_string(entry, escape))
     interpolation_to_algebra(entries, escape, state, acc, last)
   end
@@ -1241,19 +1327,23 @@ defmodule Code.Formatter do
 
   defp bitstring_to_algebra(meta, args, state) do
     last = length(args) - 1
+    join = if Keyword.get(meta, :eol, false), do: :line, else: :flex_break
     to_algebra_fun = &bitstring_segment_to_algebra(&1, &2, last)
-    force_unfit? = Keyword.get(meta, :eol, false)
 
-    {args_doc, state} =
+    {args_doc, join, state} =
       args
       |> Enum.with_index()
-      |> args_to_algebra_with_comments(meta, false, false, force_unfit?, state, to_algebra_fun)
+      |> args_to_algebra_with_comments(meta, false, :none, join, state, to_algebra_fun)
 
-    {surround("<<", args_doc, ">>"), state}
+    if join == :flex_break do
+      {"<<" |> concat(args_doc) |> nest(2) |> concat(">>") |> group(), state}
+    else
+      {surround("<<", args_doc, ">>"), state}
+    end
   end
 
   defp bitstring_segment_to_algebra({{:<-, meta, [left, right]}, i}, state, last) do
-    left = {:special, :bitstring_segment, [left, last]}
+    left = {{:special, :bitstring_segment}, meta, [left, last]}
     {doc, state} = quoted_to_algebra({:<-, meta, [left, right]}, :parens_arg, state)
     {bitstring_wrap_parens(doc, i, last), state}
   end
@@ -1261,8 +1351,17 @@ defmodule Code.Formatter do
   defp bitstring_segment_to_algebra({{:::, _, [segment, spec]}, i}, state, last) do
     {doc, state} = quoted_to_algebra(segment, :parens_arg, state)
     {spec, state} = bitstring_spec_to_algebra(spec, state)
-    doc = concat(concat(doc, "::"), wrap_in_parens_if_inspected_atom(spec))
-    {bitstring_wrap_parens(doc, i, last), state}
+
+    spec = wrap_in_parens_if_inspected_atom(spec)
+    spec = if i == last, do: bitstring_wrap_parens(spec, i, last), else: spec
+
+    doc =
+      doc
+      |> bitstring_wrap_parens(i, -1)
+      |> concat("::")
+      |> concat(spec)
+
+    {doc, state}
   end
 
   defp bitstring_segment_to_algebra({segment, i}, state, last) do
@@ -1280,40 +1379,38 @@ defmodule Code.Formatter do
     quoted_to_algebra_with_parens_if_operator(spec, :parens_arg, state)
   end
 
-  defp bitstring_wrap_parens(doc, i, last) do
-    if i == 0 or i == last do
-      string = format_to_string(doc)
+  defp bitstring_wrap_parens(doc, i, last) when i == 0 or i == last do
+    string = format_to_string(doc)
 
-      if (i == 0 and String.starts_with?(string, "<<")) or
-           (i == last and String.ends_with?(string, ">>")) do
-        wrap_in_parens(doc)
-      else
-        doc
-      end
+    if (i == 0 and String.starts_with?(string, ["~", "<<"])) or
+         (i == last and String.ends_with?(string, [">>"])) do
+      wrap_in_parens(doc)
     else
       doc
     end
   end
 
+  defp bitstring_wrap_parens(doc, _, _), do: doc
+
   ## Literals
 
   defp list_to_algebra(meta, args, state) do
-    to_algebra_fun = &quoted_to_algebra(&1, :parens_arg, &2)
-    force_unfit? = Keyword.get(meta, :eol, false)
+    join = if Keyword.get(meta, :eol, false), do: :line, else: :break
+    fun = &quoted_to_algebra(&1, :parens_arg, &2)
 
-    {args_doc, state} =
-      args_to_algebra_with_comments(args, meta, false, false, force_unfit?, state, to_algebra_fun)
+    {args_doc, _join, state} =
+      args_to_algebra_with_comments(args, meta, false, :none, join, state, fun)
 
     {surround("[", args_doc, "]"), state}
   end
 
   defp map_to_algebra(meta, name_doc, [{:|, _, [left, right]}], state) do
+    join = if Keyword.get(meta, :eol, false), do: :line, else: :break
     fun = &quoted_to_algebra(&1, :parens_arg, &2)
-    force_unfit? = Keyword.get(meta, :eol, false)
     {left_doc, state} = fun.(left, state)
 
-    {right_doc, state} =
-      args_to_algebra_with_comments(right, meta, false, false, force_unfit?, state, fun)
+    {right_doc, _join, state} =
+      args_to_algebra_with_comments(right, meta, false, :none, join, state, fun)
 
     args_doc =
       left_doc
@@ -1325,33 +1422,27 @@ defmodule Code.Formatter do
   end
 
   defp map_to_algebra(meta, name_doc, args, state) do
-    force_unfit? = Keyword.get(meta, :eol, false)
+    join = if Keyword.get(meta, :eol, false), do: :line, else: :break
     fun = &quoted_to_algebra(&1, :parens_arg, &2)
 
-    {args_doc, state} =
-      args_to_algebra_with_comments(args, meta, false, false, force_unfit?, state, fun)
+    {args_doc, _join, state} =
+      args_to_algebra_with_comments(args, meta, false, :none, join, state, fun)
 
     name_doc = "%" |> concat(name_doc) |> concat("{")
     {surround(name_doc, args_doc, "}"), state}
   end
 
-  defp tuple_to_algebra(meta, args, state) do
-    force_unfit? = Keyword.get(meta, :eol, false)
+  defp tuple_to_algebra(meta, args, join, state) do
+    join = if Keyword.get(meta, :eol, false), do: :line, else: join
     fun = &quoted_to_algebra(&1, :parens_arg, &2)
 
-    next_break_fits? =
-      args != [] and next_break_fits?(Enum.fetch!(args, -1)) and
-        not Keyword.get(meta, :eol, false)
+    {args_doc, join, state} =
+      args_to_algebra_with_comments(args, meta, false, :none, join, state, fun)
 
-    {args_doc, state} =
-      args_to_algebra_with_comments(args, meta, false, next_break_fits?, force_unfit?, state, fun)
-
-    doc = surround("{", args_doc, "}")
-
-    if next_break_fits? do
-      {next_break_fits(doc, :disabled), state}
+    if join == :flex_break do
+      {"{" |> concat(args_doc) |> nest(1) |> concat("}") |> group(), state}
     else
-      {doc, state}
+      {surround("{", args_doc, "}"), state}
     end
   end
 
@@ -1456,15 +1547,7 @@ defmodule Code.Formatter do
   defp heredoc_line(["", _ | _]), do: nest(line(), :reset)
   defp heredoc_line(_), do: line()
 
-  defp args_to_algebra_with_comments(
-         args,
-         meta,
-         skip_parens?,
-         next_break_fits?,
-         force_unfit?,
-         state,
-         fun
-       ) do
+  defp args_to_algebra_with_comments(args, meta, skip_parens?, last_arg_mode, join, state, fun) do
     min_line = line(meta)
     max_line = end_line(meta)
 
@@ -1472,10 +1555,11 @@ defmodule Code.Formatter do
       {doc, state} = fun.(arg, state)
 
       doc =
-        cond do
-          args != [] -> concat(doc, ",")
-          next_break_fits? -> next_break_fits(doc, :enabled)
-          true -> doc
+        case args do
+          [_ | _] -> concat_to_last_group(doc, ",")
+          [] when last_arg_mode == :force_comma -> concat_to_last_group(doc, ",")
+          [] when last_arg_mode == :next_break_fits -> next_break_fits(doc, :enabled)
+          [] when last_arg_mode == :none -> doc
         end
 
       {doc, @empty, newlines, state}
@@ -1498,20 +1582,29 @@ defmodule Code.Formatter do
 
     cond do
       args_docs == [] ->
-        {@empty, state}
+        {@empty, :empty, state}
 
-      force_unfit? or comments? ->
-        {args_docs |> Enum.reduce(&line(&2, &1)) |> force_unfit(), state}
+      join == :line or comments? ->
+        {args_docs |> Enum.reduce(&line(&2, &1)) |> force_unfit(), :line, state}
 
-      true ->
-        {args_docs |> Enum.reduce(&glue(&2, &1)), state}
+      join == :break ->
+        {args_docs |> Enum.reduce(&glue(&2, &1)), :break, state}
+
+      join == :flex_break ->
+        {args_docs |> Enum.reduce(&flex_glue(&2, &1)), :flex_break, state}
     end
   end
 
   ## Anonymous functions
 
   # fn -> block end
-  defp anon_fun_to_algebra([{:->, meta, [[], body]}] = clauses, _min_line, max_line, state) do
+  defp anon_fun_to_algebra(
+         [{:->, meta, [[], body]}] = clauses,
+         _min_line,
+         max_line,
+         state,
+         _multi_clauses_style
+       ) do
     min_line = line(meta)
     {body_doc, state} = block_to_algebra(body, min_line, max_line, state)
 
@@ -1530,15 +1623,27 @@ defmodule Code.Formatter do
   # fn x ->
   #   y
   # end
-  defp anon_fun_to_algebra([{:->, meta, [args, body]}] = clauses, _min_line, max_line, state) do
+  defp anon_fun_to_algebra(
+         [{:->, meta, [args, body]}] = clauses,
+         _min_line,
+         max_line,
+         state,
+         false = _multi_clauses_style
+       ) do
     min_line = line(meta)
     {args_doc, state} = clause_args_to_algebra(args, min_line, state)
     {body_doc, state} = block_to_algebra(body, min_line, max_line, state)
 
+    head =
+      args_doc
+      |> ungroup_if_group()
+      |> concat(" ->")
+      |> nest(:cursor)
+      |> group()
+
     doc =
       "fn "
-      |> concat(group(nest(args_doc, :cursor)))
-      |> concat(" ->")
+      |> concat(head)
       |> glue(body_doc)
       |> nest(2)
       |> glue("end")
@@ -1554,7 +1659,7 @@ defmodule Code.Formatter do
   #   args2 ->
   #     block2
   # end
-  defp anon_fun_to_algebra(clauses, min_line, max_line, state) do
+  defp anon_fun_to_algebra(clauses, min_line, max_line, state, _multi_clauses_style) do
     {clauses_doc, state} = clauses_to_algebra(clauses, min_line, max_line, state)
     {"fn" |> line(clauses_doc) |> nest(2) |> line("end") |> force_unfit(), state}
   end
@@ -1584,15 +1689,12 @@ defmodule Code.Formatter do
     {args_doc, state} = clause_args_to_algebra(args, min_line, state)
     {body_doc, state} = block_to_algebra(body, min_line, max_line, state)
 
-    clause_doc =
-      " ->"
-      |> glue(body_doc)
-      |> nest(2)
-
     doc =
       args_doc
+      |> ungroup_if_group()
+      |> concat(" ->")
       |> group()
-      |> concat(clause_doc)
+      |> concat(break() |> concat(body_doc) |> nest(2))
       |> wrap_in_parens()
       |> maybe_force_clauses(clauses)
       |> group()
@@ -1660,7 +1762,15 @@ defmodule Code.Formatter do
 
     state = %{state | operand_nesting: nesting}
     {body_doc, state} = block_to_algebra(body, min_line, end_line(meta), state)
-    {concat(args_doc, " ->" |> glue(body_doc) |> nest(2)), state}
+
+    doc =
+      args_doc
+      |> ungroup_if_group()
+      |> concat(" ->")
+      |> group()
+      |> concat(break() |> concat(body_doc) |> nest(2))
+
+    {doc, state}
   end
 
   defp add_max_line_to_last_clause([{op, meta, args}], max_line) do
@@ -1674,13 +1784,17 @@ defmodule Code.Formatter do
   defp clause_args_to_algebra(args, min_line, state) do
     meta = [line: min_line]
     fun = &clause_args_to_algebra/2
-    args_to_algebra_with_comments([args], meta, false, false, false, state, fun)
+
+    {args_docs, _join, state} =
+      args_to_algebra_with_comments([args], meta, false, :none, :break, state, fun)
+
+    {args_docs, state}
   end
 
   # fn a, b, c when d -> e end
   defp clause_args_to_algebra([{:when, meta, args}], state) do
     {args, right} = split_last(args)
-    left = {:special, :clause_args, [args]}
+    left = {{:special, :clause_args}, meta, [args]}
     binary_op_to_algebra(:when, "when", meta, left, right, :no_parens_arg, state)
   end
 
@@ -1810,6 +1924,9 @@ defmodule Code.Formatter do
 
   ## Quoted helpers
 
+  defp left_op_context(context), do: force_many_args_or_operand(context, :parens_arg)
+  defp right_op_context(context), do: force_many_args_or_operand(context, :operand)
+
   defp force_many_args_or_operand(:no_parens_one_arg, _choice), do: :no_parens_arg
   defp force_many_args_or_operand(:parens_one_arg, _choice), do: :parens_arg
   defp force_many_args_or_operand(:no_parens_arg, _choice), do: :no_parens_arg
@@ -1916,75 +2033,102 @@ defmodule Code.Formatter do
     end
   end
 
-  defp next_break_fits?({:<<>>, meta, [_ | _] = entries}) do
-    meta[:format] == :bin_heredoc or not interpolated?(entries)
+  defp next_break_fits?({:{}, meta, _args}, state) do
+    eol_or_comments?(meta, state)
   end
 
-  defp next_break_fits?({{:., _, [String, :to_charlist]}, _, [{:<<>>, meta, [_ | _]}]}) do
+  defp next_break_fits?({:__block__, meta, [{_, _}]}, state) do
+    eol_or_comments?(meta, state)
+  end
+
+  defp next_break_fits?({:<<>>, meta, [_ | _] = entries}, state) do
+    meta[:format] == :bin_heredoc or
+      (not interpolated?(entries) and eol_or_comments?(meta, state))
+  end
+
+  defp next_break_fits?({{:., _, [String, :to_charlist]}, _, [{:<<>>, meta, [_ | _]}]}, _state) do
     meta[:format] == :list_heredoc
   end
 
-  defp next_break_fits?({{:., _, [_left, :{}]}, _, _}) do
+  defp next_break_fits?({{:., _, [_left, :{}]}, _, _}, _state) do
     true
   end
 
-  defp next_break_fits?({:__block__, meta, [string]}) when is_binary(string) do
+  defp next_break_fits?({:__block__, meta, [string]}, _state) when is_binary(string) do
     meta[:format] == :bin_heredoc
   end
 
-  defp next_break_fits?({:__block__, meta, [list]}) when is_list(list) do
+  defp next_break_fits?({:__block__, meta, [list]}, _state) when is_list(list) do
     meta[:format] != :charlist
   end
 
-  defp next_break_fits?({form, _, [_ | _]}) when form in [:fn, :%{}, :%] do
+  defp next_break_fits?({form, _, [_ | _]}, _state) when form in [:fn, :%{}, :%] do
     true
   end
 
-  defp next_break_fits?({fun, meta, args}) when is_atom(fun) and is_list(args) do
+  defp next_break_fits?({fun, meta, args}, _state) when is_atom(fun) and is_list(args) do
     meta[:terminator] in [@double_heredoc, @single_heredoc] and
       fun |> Atom.to_string() |> String.starts_with?("sigil_")
   end
 
-  defp next_break_fits?({{:__block__, _, [atom]}, expr}) when is_atom(atom) do
-    next_break_fits?(expr)
+  defp next_break_fits?({{:__block__, _, [atom]}, expr}, state) when is_atom(atom) do
+    next_break_fits?(expr, state)
   end
 
-  defp next_break_fits?(_) do
+  defp next_break_fits?(_, _state) do
     false
   end
 
+  defp eol_or_comments?(meta, %{comments: comments}) do
+    Keyword.get(meta, :eol, false) or
+      (
+        min_line = line(meta)
+        max_line = end_line(meta)
+        Enum.any?(comments, fn {line, _, _} -> line > min_line and line < max_line end)
+      )
+  end
+
+  # A literal list is a keyword or (... -> ...)
   defp last_arg_to_keyword([_ | _] = arg, _list_to_keyword?) do
     {keyword?(arg), arg}
   end
 
+  # This is a list of tuples, it can be converted to keywords.
   defp last_arg_to_keyword({:__block__, _, [[_ | _] = arg]} = block, true) do
     if keyword?(arg), do: {true, arg}, else: {false, block}
   end
 
+  # Otherwise we don't have a keyword.
   defp last_arg_to_keyword(arg, _list_to_keyword?) do
     {false, arg}
   end
 
-  defp force_keyword?(keyword) do
-    match?([_, _ | _], keyword) and force_keyword?(keyword, MapSet.new())
+  defp force_args?(args) do
+    match?([_, _ | _], args) and force_args?(args, MapSet.new())
   end
 
-  defp force_keyword?([{{_, meta, _}, _} | keyword], lines) do
-    line = line(meta)
+  defp force_args?([[arg | _] | args], lines) do
+    force_args?([arg | args], lines)
+  end
 
-    if line in lines do
+  defp force_args?([arg | args], lines) do
+    line =
+      case arg do
+        {{_, meta, _}, _} -> line(meta)
+        {_, meta, _} -> line(meta)
+      end
+
+    if MapSet.member?(lines, line) do
       false
     else
-      force_keyword?(keyword, MapSet.put(lines, line))
+      force_args?(args, MapSet.put(lines, line))
     end
   end
 
-  defp force_keyword?([], _lines) do
-    true
-  end
+  defp force_args?([], _lines), do: true
 
   defp force_keyword(doc, arg) do
-    if force_keyword?(arg), do: force_unfit(doc), else: doc
+    if force_args?(arg), do: force_unfit(doc), else: doc
   end
 
   defp keyword?([{key, _} | list]) do
@@ -2007,6 +2151,10 @@ defmodule Code.Formatter do
     false
   end
 
+  defp eol?(meta) do
+    Keyword.get(meta, :eol, false)
+  end
+
   defp line(meta) do
     Keyword.get(meta, :line, @max_line)
   end
@@ -2017,6 +2165,23 @@ defmodule Code.Formatter do
 
   ## Algebra helpers
 
+  # Relying on the inner document is brittle and error prone.
+  # It would be best if we had a mechanism to apply this.
+  defp concat_to_last_group({:doc_cons, left, right}, concat) do
+    {:doc_cons, left, concat_to_last_group(right, concat)}
+  end
+
+  defp concat_to_last_group({:doc_group, group, mode}, concat) do
+    {:doc_group, {:doc_cons, group, concat}, mode}
+  end
+
+  defp concat_to_last_group(other, concat) do
+    {:doc_cons, other, concat}
+  end
+
+  defp ungroup_if_group({:doc_group, group, _mode}), do: group
+  defp ungroup_if_group(other), do: other
+
   defp format_to_string(doc) do
     doc |> Inspect.Algebra.format(:infinity) |> IO.iodata_to_binary()
   end

lib/elixir/lib/code/identifier.ex

@@ -13,7 +13,7 @@ defmodule Code.Identifier do
   @spec unary_op(atom) :: {:non_associative, precedence :: pos_integer} | :error
   def unary_op(op) do
     cond do
-      op in [:&] -> {:non_associative, 100}
+      op in [:&] -> {:non_associative, 90}
       op in [:!, :^, :not, :+, :-, :~~~] -> {:non_associative, 300}
       op in [:@] -> {:non_associative, 320}
       true -> :error
@@ -36,7 +36,7 @@ defmodule Code.Identifier do
       op in [:when] -> {:right, 50}
       op in [:::] -> {:right, 60}
       op in [:|] -> {:right, 70}
-      op in [:=] -> {:right, 90}
+      op in [:=] -> {:right, 100}
       op in [:||, :|||, :or] -> {:left, 130}
       op in [:&&, :&&&, :and] -> {:left, 140}
       op in [:==, :!=, :=~, :===, :!==] -> {:left, 150}

lib/elixir/lib/collectable.ex

@@ -90,11 +90,40 @@ defimpl Collectable, for: List do
 end
 
 defimpl Collectable, for: BitString do
-  def into(original) do
+  def into(original) when is_binary(original) do
     fun = fn
-      acc, {:cont, x} when is_bitstring(x) -> [acc | x]
-      acc, :done -> IO.iodata_to_binary(acc)
-      _, :halt -> :ok
+      acc, {:cont, x} when is_binary(x) and is_list(acc) ->
+        [acc | x]
+
+      acc, {:cont, x} when is_bitstring(x) and is_bitstring(acc) ->
+        <<acc::bitstring, x::bitstring>>
+
+      acc, {:cont, x} when is_bitstring(x) ->
+        <<IO.iodata_to_binary(acc)::bitstring, x::bitstring>>
+
+      acc, :done when is_bitstring(acc) ->
+        acc
+
+      acc, :done ->
+        IO.iodata_to_binary(acc)
+
+      _, :halt ->
+        :ok
+    end
+
+    {[original], fun}
+  end
+
+  def into(original) when is_bitstring(original) do
+    fun = fn
+      acc, {:cont, x} when is_bitstring(x) ->
+        <<acc::bitstring, x::bitstring>>
+
+      acc, :done ->
+        acc
+
+      _, :halt ->
+        :ok
     end
 
     {original, fun}

lib/elixir/lib/dynamic_supervisor.ex

@@ -11,21 +11,27 @@ defmodule DynamicSupervisor do
 
   ## Examples
 
-  A dynamic supervisor is started with no children, only with the
-  supervision strategy (the only strategy currently supported is
-  `:one_for_one`):
+  A dynamic supervisor is started with no children, often under a
+  supervisor with the supervision strategy (the only strategy currently
+  supported is `:one_for_one`) and a name:
 
-      {:ok, sup} = DynamicSupervisor.start_link(strategy: :one_for_one)
+      children = [
+        {DynamicSupervisor, strategy: :one_for_one, name: MyApp.DynamicSupervisor}
+      ]
+
+      Supervisor.start_link(strategy: :one_for_one)
+
+  The options given in the child specification are documented in `start_link/1`.
 
   Once the dynamic supervisor is running, we can start children
   with `start_child/2`, which receives a child specification:
 
-      {:ok, agent1} = DynamicSupervisor.start_child(sup, {Agent, fn -> %{} end})
+      {:ok, agent1} = DynamicSupervisor.start_child(MyApp.DynamicSupervisor, {Agent, fn -> %{} end})
       Agent.update(agent1, &Map.put(&1, :key, "value"))
       Agent.get(agent1, & &1)
       #=> %{key: "value"}
 
-      {:ok, agent2} = DynamicSupervisor.start_child(sup, {Agent, fn -> %{} end})
+      {:ok, agent2} = DynamicSupervisor.start_child(MyApp.DynamicSupervisor, {Agent, fn -> %{} end})
       Agent.get(agent2, & &1)
       #=> %{}
 
@@ -45,6 +51,7 @@ defmodule DynamicSupervisor do
           DynamicSupervisor.start_link(__MODULE__, arg, name: __MODULE__)
         end
 
+        @impl true
         def init(_arg) do
           DynamicSupervisor.init(strategy: :one_for_one)
         end
@@ -57,6 +64,67 @@ defmodule DynamicSupervisor do
 
   A supervisor is bound to the same name registration rules as a `GenServer`.
   Read more about these rules in the documentation for `GenServer`.
+
+  ## Migrating from Supervisor's :simple_one_for_one
+
+  In case you were using the deprecated `:simple_one_for_one` strategy from
+  the `Supervisor` module, you can migrate to the `DynamicSupervisor` in
+  few steps.
+
+  Imagine the given "old" code:
+
+      defmodule MySupervisor do
+        use Supervisor
+
+        def start_link(arg) do
+          Supervisor.start_link(__MODULE__, arg, name: __MODULE__)
+        end
+
+        def start_child(foo, bar, baz) do
+          # This will start child by calling MyWorker.start_link(initial_arg, foo, bar, baz)
+          Supervisor.start_child(__MODULE__, [foo, bar, baz])
+        end
+
+        @impl true
+        def init(initial_arg) do
+          children = [
+            # Or the deprecated: worker(MyWorker, [initial_arg])
+            %{id: MyWorker, start: {MyWorker, :start_link, [initial_arg]})
+          ]
+
+          Supervisor.init(children, strategy: :simple_one_for_one)
+        end
+      end
+
+  It can be upgraded to the DynamicSupervisor like this:
+
+      defmodule MySupervisor do
+        use DynamicSupervisor
+
+        def start_link(arg) do
+          DynamicSupervisor.start_link(__MODULE__, arg, name: __MODULE__)
+        end
+
+        def start_child(foo, bar, baz) do
+          # If MyWorker is not using the new child specs, we need to pass a map:
+          # spec = %{id: MyWorker, start: {MyWorker, :start_link, [foo, bar, baz]}}
+          spec = {MyWorker, foo: foo, bar: bar, baz: baz}
+          DynamicSupervisor.start_child(__MODULE__, spec)
+        end
+
+        @impl true
+        def init(initial_arg) do
+          DynamicSupervisor.init(
+            strategy: :one_for_one,
+            extra_arguments: [initial_arg]
+          )
+        end
+      end
+
+  The difference is that the `DynamicSupervisor` expects the child specification
+  at the moment `start_child/2` is called, and no longer on the init callback.
+  If there are any initial arguments given on initialization, such as `[initial_arg]`,
+  it can be given in the `:extra_arguments` flag on `DynamicSupervisor.init/1`.
   """
 
   @behaviour GenServer
@@ -69,13 +137,14 @@ defmodule DynamicSupervisor do
   """
   @callback init(args :: term) :: {:ok, sup_flags()} | :ignore
 
-  @opaque sup_flags() :: %{
-            strategy: strategy(),
-            intensity: non_neg_integer(),
-            period: pos_integer(),
-            max_children: non_neg_integer() | :infinity,
-            extra_arguments: [term()]
-          }
+  @typedoc "The supervisor flags returned on init"
+  @type sup_flags() :: %{
+          strategy: strategy(),
+          intensity: non_neg_integer(),
+          period: pos_integer(),
+          max_children: non_neg_integer() | :infinity,
+          extra_arguments: [term()]
+        }
 
   @typedoc "Option values used by the `start*` functions"
   @type option :: {:name, Supervisor.name()} | init_option()
@@ -94,6 +163,13 @@ defmodule DynamicSupervisor do
   @typedoc "Supported strategies"
   @type strategy :: :one_for_one
 
+  @typedoc "Return values of `start_child` functions"
+  @type on_start_child ::
+          {:ok, pid}
+          | {:ok, pid, info :: term}
+          | :ignore
+          | {:error, {:already_started, pid} | :max_children | term}
+
   defstruct [
     :args,
     :extra_arguments,
@@ -104,17 +180,33 @@ defmodule DynamicSupervisor do
     :max_restarts,
     :max_seconds,
     children: %{},
-    dynamic: 0,
     restarts: []
   ]
 
+  @doc """
+  Returns a specification to start a dynamic supervisor under a supervisor.
+
+  See `Supervisor`.
+  """
+  @since "1.6.1"
+  def child_spec(arg) do
+    %{
+      id: DynamicSupervisor,
+      start: {DynamicSupervisor, :start_link, [arg]},
+      type: :supervisor
+    }
+  end
+
   @doc false
   defmacro __using__(opts) do
-    quote location: :keep do
+    quote location: :keep, bind_quoted: [opts: opts] do
       @behaviour DynamicSupervisor
-      @opts unquote(opts)
 
-      @doc false
+      @doc """
+      Returns a specification to start this module under a supervisor.
+
+      See `Supervisor`.
+      """
       def child_spec(arg) do
         default = %{
           id: __MODULE__,
@@ -122,13 +214,10 @@ defmodule DynamicSupervisor do
           type: :supervisor
         }
 
-        Supervisor.child_spec(default, @opts)
+        Supervisor.child_spec(default, unquote(Macro.escape(opts)))
       end
 
       defoverridable child_spec: 1
-
-      @doc false
-      def init(arg)
     end
   end
 
@@ -206,7 +295,7 @@ defmodule DynamicSupervisor do
   this function returns `{:error, :max_children}`.
   """
   @spec start_child(Supervisor.supervisor(), :supervisor.child_spec() | {module, term} | module) ::
-          Supervisor.on_start_child()
+          on_start_child()
   def start_child(supervisor, {_, _, _, _, _, _} = child_spec) do
     validate_and_start_child(supervisor, child_spec)
   end
@@ -320,7 +409,7 @@ defmodule DynamicSupervisor do
 
   The map contains the following keys:
 
-    * `:specs` - always 1 as dynamic supervisors have a single specification
+    * `:specs` - the number of children processes
 
     * `:active` - the count of all actively running child processes managed by
       this supervisor
@@ -413,7 +502,14 @@ defmodule DynamicSupervisor do
 
     case mod.init(args) do
       {:ok, flags} when is_map(flags) ->
-        state = %DynamicSupervisor{mod: mod, args: args, name: name || {self(), mod}}
+        name =
+          cond do
+            is_nil(name) -> {self(), mod}
+            is_atom(name) -> {:local, name}
+            is_tuple(name) -> name
+          end
+
+        state = %DynamicSupervisor{mod: mod, args: args, name: name}
 
         case init(state, flags) do
           {:ok, state} -> {:ok, state}
@@ -440,14 +536,15 @@ defmodule DynamicSupervisor do
          :ok <- validate_seconds(max_seconds),
          :ok <- validate_dynamic(max_children),
          :ok <- validate_extra_arguments(extra_arguments) do
-      {:ok, %{
-        state
-        | extra_arguments: extra_arguments,
-          max_children: max_children,
-          max_restarts: max_restarts,
-          max_seconds: max_seconds,
-          strategy: strategy
-      }}
+      {:ok,
+       %{
+         state
+         | extra_arguments: extra_arguments,
+           max_children: max_children,
+           max_restarts: max_restarts,
+           max_seconds: max_seconds,
+           strategy: strategy
+       }}
     end
   end
 
@@ -528,10 +625,10 @@ defmodule DynamicSupervisor do
   end
 
   def handle_call({:start_child, child}, _from, state) do
-    %{dynamic: dynamic, max_children: max_children} = state
+    %{children: children, max_children: max_children} = state
 
-    if dynamic < max_children do
-      handle_start_child(child, %{state | dynamic: dynamic + 1})
+    if map_size(children) < max_children do
+      handle_start_child(child, state)
     else
       {:reply, {:error, :max_children}, state}
     end
@@ -548,7 +645,7 @@ defmodule DynamicSupervisor do
         {:reply, reply, save_child(pid, mfa, restart, shutdown, type, modules, state)}
 
       _ ->
-        {:reply, reply, update_in(state.dynamic, &(&1 - 1))}
+        {:reply, reply, state}
     end
   end
 
@@ -567,14 +664,14 @@ defmodule DynamicSupervisor do
     end
   end
 
-  defp save_child(pid, {m, f, _}, :temporary, shutdown, type, modules, state) do
-    put_in(state.children[pid], {{m, f, :undefined}, :temporary, shutdown, type, modules})
-  end
-
   defp save_child(pid, mfa, restart, shutdown, type, modules, state) do
+    mfa = mfa_for_restart(mfa, restart)
     put_in(state.children[pid], {mfa, restart, shutdown, type, modules})
   end
 
+  defp mfa_for_restart({m, f, _}, :temporary), do: {m, f, :undefined}
+  defp mfa_for_restart(mfa, _), do: mfa
+
   defp exit_reason(:exit, reason, _), do: reason
   defp exit_reason(:error, reason, stack), do: {reason, stack}
   defp exit_reason(:throw, value, stack), do: {{:nocatch, value}, stack}
@@ -791,9 +888,8 @@ defmodule DynamicSupervisor do
     {:ok, delete_child(pid, state)}
   end
 
-  defp delete_child(pid, state) do
-    %{children: children, dynamic: dynamic} = state
-    %{state | children: Map.delete(children, pid), dynamic: dynamic - 1}
+  defp delete_child(pid, %{children: children} = state) do
+    %{state | children: Map.delete(children, pid)}
   end
 
   defp restart_child(pid, child, state) do
@@ -836,8 +932,9 @@ defmodule DynamicSupervisor do
 
   defp restart_child(:one_for_one, current_pid, child, state) do
     {{m, f, args} = mfa, restart, shutdown, type, modules} = child
+    %{extra_arguments: extra} = state
 
-    case start_child(m, f, args) do
+    case start_child(m, f, extra ++ args) do
       {:ok, pid, _} ->
         state = delete_child(current_pid, state)
         {:ok, save_child(pid, mfa, restart, shutdown, type, modules, state)}
@@ -856,21 +953,21 @@ defmodule DynamicSupervisor do
     end
   end
 
-  defp report_error(error, reason, pid, child, %{name: name}) do
+  defp report_error(error, reason, pid, child, %{name: name, extra_arguments: extra}) do
     :error_logger.error_report(
-      :supervision_report,
+      :supervisor_report,
       supervisor: name,
       errorContext: error,
       reason: reason,
-      offender: extract_child(pid, child)
+      offender: extract_child(pid, child, extra)
     )
   end
 
-  defp extract_child(pid, {mfa, restart, shutdown, type, _modules}) do
+  defp extract_child(pid, {{m, f, args}, restart, shutdown, type, _modules}, extra) do
     [
       pid: pid,
       id: :undefined,
-      mfargs: mfa,
+      mfargs: {m, f, extra ++ args},
       restart_type: restart,
       shutdown: shutdown,
       child_type: type

lib/elixir/lib/enum.ex

@@ -2718,10 +2718,10 @@ defmodule Enum do
   end
 
   @doc """
-  Zips corresponding elements from a list of enumerables
+  Zips corresponding elements from a finite collection of enumerables
   into one list of tuples.
 
-  The zipping finishes as soon as any enumerable in the given list completes.
+  The zipping finishes as soon as any enumerable in the given collection completes.
 
   ## Examples
 
@@ -2733,10 +2733,11 @@ defmodule Enum do
 
   """
   @spec zip([t]) :: t
+  @spec zip(t) :: t
 
   def zip([]), do: []
 
-  def zip(enumerables) when is_list(enumerables) do
+  def zip(enumerables) do
     Stream.zip(enumerables).({:cont, []}, &{:cont, [&1 | &2]})
     |> elem(1)
     |> :lists.reverse()

lib/elixir/lib/exception.ex

@@ -301,8 +301,17 @@ defmodule Exception do
 
   defp rewrite_guard(guard) do
     Macro.prewalk(guard, fn
-      {:., _, [:erlang, call]} -> rewrite_guard_call(call)
-      other -> other
+      {{:., _, [:erlang, :element]}, _, [{{:., _, [:erlang, :+]}, _, [int, 1]}, arg]} ->
+        {:elem, [], [arg, int]}
+
+      {{:., _, [:erlang, :element]}, _, [int, arg]} when is_integer(int) ->
+        {:elem, [], [arg, int - 1]}
+
+      {:., _, [:erlang, call]} ->
+        rewrite_guard_call(call)
+
+      other ->
+        other
     end)
   end
 

lib/elixir/lib/float.ex

@@ -3,6 +3,33 @@ import Kernel, except: [round: 1]
 defmodule Float do
   @moduledoc """
   Functions for working with floating-point numbers.
+
+  ## Kernel functions
+
+  There are functions related to floating-point numbers on the `Kernel` module
+  too. Here is a list of them:
+
+    * `Kernel.round/1`: rounds a number to the nearest integer.
+    * `Kernel.trunc/1`: returns the integer part of a number.
+
+  ## Known issues
+
+  There are some very well known problems with floating-point numbers
+  and arithmetics due to the fact most decimal fractions cannot be
+  represented by a floating-point binary.
+
+  For example, the numbers 0.1 and 0.01 are two of them, what means the result
+  of squaring 0.1 does not give 0.01 neither the closest representable. Here is
+  what happens in this case:
+
+    * The closest representable number to 0.1 is 0.1000000014
+    * The closest representable number to 0.01 is 0.0099999997
+    * Doing 0.1 * 0.1 should return 0.01, but because 0.1 is actually 0.1000000014,
+      the result is 0.010000000000000002, and because this is not the closest
+      representable number to 0.01, you'll get the wrong result for this operation
+
+  There are also other known problems like flooring or rounding numbers. See
+  `round/2` and `floor/2` for more details about them.
   """
 
   import Bitwise
@@ -79,13 +106,18 @@ defmodule Float do
   defp add_dot(acc, false), do: acc <> ".0"
 
   @doc """
-  Rounds a float to the largest integer less than or equal to `num`.
+  Rounds a float to the largest number less than or equal to `num`.
 
   `floor/2` also accepts a precision to round a floating-point value down
   to an arbitrary number of fractional digits (between 0 and 15).
   The operation is performed on the binary floating point, without a
   conversion to decimal.
 
+  This function always returns a float. `Kernel.trunc/1` may be used instead to
+  truncate the result to an integer afterwards.
+
+  ## Known issues
+
   The behaviour of `floor/2` for floats can be surprising. For example:
 
       iex> Float.floor(12.52, 2)
@@ -96,9 +128,6 @@ defmodule Float do
   and therefore the number above is internally represented as 12.51999999,
   which explains the behaviour above.
 
-  This function always returns a float. `Kernel.trunc/1` may be used instead to
-  truncate the result to an integer afterwards.
-
   ## Examples
 
       iex> Float.floor(34.25)
@@ -174,6 +203,8 @@ defmodule Float do
   `Kernel.round/1` if you want a function that accepts both floats
   and integers and always returns an integer.
 
+  ## Known issues
+
   The behaviour of `round/2` for floats can be surprising. For example:
 
       iex> Float.round(5.5675, 3)

lib/elixir/lib/gen_event/stream.ex

@@ -161,7 +161,8 @@ defimpl Enumerable, for: GenEvent.Stream do
 
   defp flush_events(ref) do
     receive do
-      {_from, {_pid, ^ref}, {notify, _event}} when notify in [:notify, :ack_notify, :sync_notify] ->
+      {_from, {_pid, ^ref}, {notify, _event}}
+      when notify in [:notify, :ack_notify, :sync_notify] ->
         flush_events(ref)
     after
       0 -> :ok

lib/elixir/lib/gen_server.ex

@@ -23,10 +23,17 @@ defmodule GenServer do
 
         # Callbacks
 
+        @impl true
+        def init(stack) do
+          {:ok, stack}
+        end
+
+        @impl true
         def handle_call(:pop, _from, [h | t]) do
           {:reply, h, t}
         end
 
+        @impl true
         def handle_cast({:push, item}, state) do
           {:noreply, [item | state]}
         end
@@ -151,22 +158,15 @@ defmodule GenServer do
 
         # Server (callbacks)
 
+        @impl true
         def handle_call(:pop, _from, [h | t]) do
           {:reply, h, t}
         end
 
-        def handle_call(request, from, state) do
-          # Call the default implementation from GenServer
-          super(request, from, state)
-        end
-
+        @impl true
         def handle_cast({:push, item}, state) do
           {:noreply, [item | state]}
         end
-
-        def handle_cast(request, state) do
-          super(request, state)
-        end
       end
 
   In practice, it is common to have both server and client functions in
@@ -196,11 +196,13 @@ defmodule GenServer do
           GenServer.start_link(__MODULE__, %{})
         end
 
+        @impl true
         def init(state) do
           schedule_work() # Schedule work to be performed on start
           {:ok, state}
         end
 
+        @impl true
         def handle_info(:work, state) do
           # Do the desired work here
           schedule_work() # Reschedule once more
@@ -567,18 +569,21 @@ defmodule GenServer do
 
   @doc false
   defmacro __using__(opts) do
-    quote location: :keep do
+    quote location: :keep, bind_quoted: [opts: opts] do
       @behaviour GenServer
-      @opts unquote(opts)
 
-      @doc false
+      @doc """
+      Returns a specification to start this module under a supervisor.
+
+      See `Supervisor`.
+      """
       def child_spec(arg) do
         default = %{
           id: __MODULE__,
           start: {__MODULE__, :start_link, [arg]}
         }
 
-        Supervisor.child_spec(default, @opts)
+        Supervisor.child_spec(default, unquote(Macro.escape(opts)))
       end
 
       defoverridable child_spec: 1
@@ -661,8 +666,8 @@ defmodule GenServer do
             {:ok, args}
           end
 
-      But you want to define your own implementation that converts the \
-      arguments given to GenServer.start_link/3 to the server state
+      You can copy the implementation above or define your own that converts \
+      the arguments given to GenServer.start_link/3 to the server state.
       """
 
       :elixir_errors.warn(env.line, env.file, message)
@@ -886,7 +891,8 @@ defmodule GenServer do
   See `multi_call/4` for more information.
   """
   @spec abcast([node], name :: atom, term) :: :abcast
-  def abcast(nodes \\ [node() | Node.list()], name, request) when is_list(nodes) and is_atom(name) do
+  def abcast(nodes \\ [node() | Node.list()], name, request)
+      when is_list(nodes) and is_atom(name) do
     msg = cast_msg(request)
     _ = for node <- nodes, do: do_send({name, node}, msg)
     :abcast

lib/elixir/lib/inspect.ex

@@ -262,6 +262,10 @@ defimpl Inspect, for: Integer do
 
   defp prepend_prefix(value, :decimal), do: value
 
+  defp prepend_prefix(<<?-, value::binary>>, base) do
+    "-" <> prepend_prefix(value, base)
+  end
+
   defp prepend_prefix(value, base) do
     prefix =
       case base do

lib/elixir/lib/inspect/algebra.ex

@@ -934,7 +934,7 @@ defmodule Inspect.Algebra do
     do: fits?(w, k, b?, [{apply_nesting(i, k, j), m, x} | t])
 
   defp fits?(w, k, b?, [{i, m, doc_cons(x, y)} | t]),
-    do: fits?(w, k, b?, [{i, m, x} | [{i, m, y} | t]])
+    do: fits?(w, k, b?, [{i, m, x}, {i, m, y} | t])
 
   defp fits?(w, k, b?, [{i, m, doc_group(x, _)} | t]),
     do: fits?(w, k, b?, [{i, m, x} | {:tail, b?, t}])
@@ -943,7 +943,7 @@ defmodule Inspect.Algebra do
   defp format(_, _, []), do: []
   defp format(w, k, [{_, _, :doc_nil} | t]), do: format(w, k, t)
   defp format(w, _, [{i, _, :doc_line} | t]), do: [indent(i) | format(w, i, t)]
-  defp format(w, k, [{i, m, doc_cons(x, y)} | t]), do: format(w, k, [{i, m, x} | [{i, m, y} | t]])
+  defp format(w, k, [{i, m, doc_cons(x, y)} | t]), do: format(w, k, [{i, m, x}, {i, m, y} | t])
   defp format(w, k, [{i, m, doc_color(x, c)} | t]), do: [ansi(c) | format(w, k, [{i, m, x} | t])]
   defp format(w, k, [{_, _, doc_string(s, l)} | t]), do: [s | format(w, k + l, t)]
   defp format(w, k, [{_, _, s} | t]) when is_binary(s), do: [s | format(w, k + byte_size(s), t)]

lib/elixir/lib/kernel.ex

@@ -1555,7 +1555,7 @@ defmodule Kernel do
     optimize_boolean(
       quote do
         case unquote(value) do
-          x when x in [false, nil] -> false
+          x when :"Elixir.Kernel".in(x, [false, nil]) -> false
           _ -> true
         end
       end
@@ -1566,7 +1566,7 @@ defmodule Kernel do
     optimize_boolean(
       quote do
         case unquote(value) do
-          x when x in [false, nil] -> true
+          x when :"Elixir.Kernel".in(x, [false, nil]) -> true
           _ -> false
         end
       end
@@ -2709,7 +2709,10 @@ defmodule Kernel do
     case function? do
       true ->
         value =
-          with {_, doc} when doc_attr? <- Module.get_attribute(env.module, name, stack), do: doc
+          case Module.get_attribute(env.module, name, stack) do
+            {_, doc} when doc_attr? -> doc
+            other -> other
+          end
 
         try do
           :elixir_quote.escape(value, false)
@@ -2722,13 +2725,21 @@ defmodule Kernel do
           {val, _} -> val
         end
 
+      false when doc_attr? ->
+        {escaped, _} = :elixir_quote.escape(stack, false)
+
+        quote do
+          case Module.get_attribute(__MODULE__, unquote(name), unquote(escaped)) do
+            {_, doc} -> doc
+            other -> other
+          end
+        end
+
       false ->
         {escaped, _} = :elixir_quote.escape(stack, false)
 
         quote do
-          with {_, doc} when unquote(doc_attr?) <-
-                 Module.get_attribute(__MODULE__, unquote(name), unquote(escaped)),
-               do: doc
+          Module.get_attribute(__MODULE__, unquote(name), unquote(escaped))
         end
     end
   end
@@ -2838,7 +2849,7 @@ defmodule Kernel do
     optimize_boolean(
       quote do
         case unquote(condition) do
-          x when x in [false, nil] -> unquote(else_clause)
+          x when :"Elixir.Kernel".in(x, [false, nil]) -> unquote(else_clause)
           _ -> unquote(do_clause)
         end
       end
@@ -3006,7 +3017,7 @@ defmodule Kernel do
   defmacro left && right do
     quote do
       case unquote(left) do
-        x when x in [false, nil] ->
+        x when :"Elixir.Kernel".in(x, [false, nil]) ->
           x
 
         _ ->
@@ -3042,7 +3053,7 @@ defmodule Kernel do
   defmacro left || right do
     quote do
       case unquote(left) do
-        x when x in [false, nil] ->
+        x when :"Elixir.Kernel".in(x, [false, nil]) ->
           unquote(right)
 
         x ->
@@ -3134,10 +3145,11 @@ defmodule Kernel do
     fun = fn {x, pos}, acc ->
       case x do
         {op, _, [_]} when op == :+ or op == :- ->
-          :elixir_errors.warn(__CALLER__.line, __CALLER__.file, <<
-            "piping into a unary operator is deprecated, please use the ",
-            "qualified name. For example, Kernel.+(5), instead of +5"
-          >>)
+          message =
+            <<"piping into a unary operator is deprecated, please use the ",
+              "qualified name. For example, Kernel.+(5), instead of +5">>
+
+          :elixir_errors.warn(__CALLER__.line, __CALLER__.file, message)
 
         _ ->
           :ok
@@ -3329,7 +3341,8 @@ defmodule Kernel do
     :lists.mapfoldl(fun, acc, list)
   end
 
-  defp ensure_evaled_element(elem, acc) when is_number(elem) or is_atom(elem) or is_binary(elem) do
+  defp ensure_evaled_element(elem, acc)
+       when is_number(elem) or is_atom(elem) or is_binary(elem) do
     {elem, acc}
   end
 
@@ -4558,7 +4571,7 @@ defmodule Kernel do
   allowed in guards, and otherwise creates a private macro that can be used
   both inside or outside guards in the current module.
 
-  Similar to `defmacrop/2`, `defguardp/2` must be defined before its use
+  Similar to `defmacrop/2`, `defguardp/1` must be defined before its use
   in the current module.
   """
   @spec defguardp(Macro.t()) :: Macro.t()
@@ -4568,25 +4581,33 @@ defmodule Kernel do
 
   defp define_guard(kind, guard, env) do
     case :elixir_utils.extract_guards(guard) do
-      {call, impl} when length(impl) < 2 ->
+      {call, [_, _ | _]} ->
+        raise ArgumentError,
+              "invalid syntax in defguard #{Macro.to_string(call)}, " <>
+                "only a single when clause is allowed"
+
+      {call, impls} ->
         case Macro.decompose_call(call) do
           {_name, args} ->
             validate_variable_only_args!(call, args)
 
-            quoted =
-              quote do
-                require Kernel.Utils
-                Kernel.Utils.defguard(unquote(args), unquote(impl))
-              end
+            case impls do
+              [] ->
+                define(kind, call, nil, env)
 
-            define(kind, call, [do: quoted], env)
+              [guard] ->
+                quoted =
+                  quote do
+                    require Kernel.Utils
+                    Kernel.Utils.defguard(unquote(args), unquote(guard))
+                  end
+
+                define(kind, call, [do: quoted], env)
+            end
 
           _invalid_definition ->
             raise ArgumentError, "invalid syntax in defguard #{Macro.to_string(call)}"
         end
-
-      {call, _multiple_impls} ->
-        raise ArgumentError, "invalid syntax in defguard #{Macro.to_string(call)}"
     end
   end
 
@@ -4795,10 +4816,6 @@ defmodule Kernel do
       for fun <- List.wrap(funs) do
         {name, args, as, as_args} = Kernel.Utils.defdelegate(fun, opts)
 
-        unless Module.get_attribute(__MODULE__, :doc) do
-          @doc "See `#{inspect(target)}.#{as}/#{:erlang.length(args)}`."
-        end
-
         def unquote(name)(unquote_splicing(args)) do
           unquote(target).unquote(as)(unquote_splicing(as_args))
         end

lib/elixir/lib/kernel/parallel_compiler.ex

@@ -356,6 +356,7 @@ defmodule Kernel.ParallelCompiler do
       {:file_done, child_pid, file, {kind, reason, stack}} ->
         discard_down(child_pid)
         print_error(file, kind, reason, stack)
+        cancel_waiting_timer(queued, child_pid)
         terminate(queued)
         {:error, [to_error(file, kind, reason, stack)], warnings}
 

lib/elixir/lib/kernel/special_forms.ex

@@ -1776,19 +1776,19 @@ defmodule Kernel.SpecialForms do
   allows matching on both the *kind* of the caught value as well as the value
   itself:
 
-    try do
-      exit(:shutdown)
-    catch
-      :exit, value
-        IO.puts "Exited with value #{inspect(value)}"
-    end
+      try do
+        exit(:shutdown)
+      catch
+        :exit, value
+          IO.puts "Exited with value #{inspect(value)}"
+      end
 
-    try do
-      exit(:shutdown)
-    catch
-      kind, value when kind in [:exit, :throw] ->
-        IO.puts "Caught exit or throw with value #{inspect(value)}"
-    end
+      try do
+        exit(:shutdown)
+      catch
+        kind, value when kind in [:exit, :throw] ->
+          IO.puts "Caught exit or throw with value #{inspect(value)}"
+      end
 
   The `catch` clause also supports `:error` alongside `:exit` and `:throw` as
   in Erlang, although this is commonly avoided in favor of `raise`/`rescue` control

lib/elixir/lib/kernel/typespec.ex

@@ -751,13 +751,11 @@ defmodule Kernel.Typespec do
     typespec_to_ast({:type, line, :charlist, []})
   end
 
-  defp typespec_to_ast(
-         {
-           :remote_type,
-           line,
-           [{:atom, _, :elixir}, {:atom, _, :nonempty_charlist}, []]
-         }
-       ) do
+  defp typespec_to_ast({
+         :remote_type,
+         line,
+         [{:atom, _, :elixir}, {:atom, _, :nonempty_charlist}, []]
+       }) do
     typespec_to_ast({:type, line, :nonempty_charlist, []})
   end
 

lib/elixir/lib/kernel/utils.ex

@@ -179,6 +179,7 @@ defmodule Kernel.Utils do
           is_integer(value) and rem(value, 2) == 0
         end
       end
+
   """
   defmacro defguard(args, expr) do
     defguard(args, expr, __CALLER__)
@@ -187,7 +188,7 @@ defmodule Kernel.Utils do
   @spec defguard([Macro.t()], Macro.t(), Macro.Env.t()) :: Macro.t()
   def defguard(args, expr, env) do
     {^args, vars} = extract_refs_from_args(args)
-    _valid? = :elixir_expand.expand(expr, %{env | context: :guard, vars: vars})
+    {expr, _scope} = :elixir_expand.expand(expr, %{env | context: :guard, vars: vars})
 
     quote do
       case Macro.Env.in_guard?(__CALLER__) do
@@ -199,19 +200,19 @@ defmodule Kernel.Utils do
 
   defp extract_refs_from_args(args) do
     Macro.postwalk(args, [], fn
-      {ref, _meta, context} = var, acc when is_atom(ref) and is_atom(context) ->
-        {var, [{ref, context} | acc]}
+      {ref, meta, context} = var, acc when is_atom(ref) and is_atom(context) ->
+        {var, [{ref, var_context(meta, context)} | acc]}
 
       node, acc ->
         {node, acc}
     end)
   end
 
-  # Finds every reference to `refs` in `expr` and wraps them in an unquote.
-  defp unquote_every_ref(expr, refs) do
-    Macro.postwalk(expr, fn
-      {ref, _meta, context} = var when is_atom(ref) and is_atom(context) ->
-        case {ref, context} in refs do
+  # Finds every reference to `refs` in `guard` and wraps them in an unquote.
+  defp unquote_every_ref(guard, refs) do
+    Macro.postwalk(guard, fn
+      {ref, meta, context} = var when is_atom(ref) and is_atom(context) ->
+        case {ref, var_context(meta, context)} in refs do
           true -> literal_unquote(var)
           false -> var
         end
@@ -221,13 +222,15 @@ defmodule Kernel.Utils do
     end)
   end
 
-  # Prefaces `expr` with unquoted versions of `refs`.
-  defp unquote_refs_once(expr, refs) do
-    {^expr, used_refs} =
-      Macro.postwalk(expr, [], fn
-        {ref, _meta, context} = var, acc when is_atom(ref) and is_atom(context) ->
-          case {ref, context} in refs and {ref, context} not in acc do
-            true -> {var, [{ref, context} | acc]}
+  # Prefaces `guard` with unquoted versions of `refs`.
+  defp unquote_refs_once(guard, refs) do
+    {_, used_refs} =
+      Macro.postwalk(guard, [], fn
+        {ref, meta, context} = var, acc when is_atom(ref) and is_atom(context) ->
+          pair = {ref, var_context(meta, context)}
+
+          case pair in refs and pair not in acc do
+            true -> {var, [pair | acc]}
             false -> {var, acc}
           end
 
@@ -235,17 +238,30 @@ defmodule Kernel.Utils do
           {node, acc}
       end)
 
-    for {ref, context} <- :lists.reverse(used_refs) do
-      var = {ref, [], context}
-      quote do: unquote(var) = unquote(literal_unquote(var))
-    end ++ List.wrap(expr)
+    vars = for {ref, context} <- :lists.reverse(used_refs), do: context_to_var(ref, context)
+    exprs = for var <- vars, do: literal_unquote(var)
+
+    quote do
+      {unquote_splicing(vars)} = {unquote_splicing(exprs)}
+      unquote(guard)
+    end
   end
 
   defp literal_quote(ast) do
-    {:quote, [], [[do: {:__block__, [], List.wrap(ast)}]]}
+    {:quote, [], [[do: ast]]}
   end
 
   defp literal_unquote(ast) do
     {:unquote, [], List.wrap(ast)}
   end
+
+  defp context_to_var(ref, ctx) when is_atom(ctx), do: {ref, [], ctx}
+  defp context_to_var(ref, ctx) when is_integer(ctx), do: {ref, [counter: ctx], nil}
+
+  defp var_context(meta, kind) do
+    case :lists.keyfind(:counter, 1, meta) do
+      {:counter, counter} -> counter
+      false -> kind
+    end
+  end
 end

lib/elixir/lib/keyword.ex

@@ -606,8 +606,10 @@ defmodule Keyword do
   end
 
   @doc """
-  Similar to `replace/3`, but will raise a `KeyError`
-  if the entry `key` does not exist.
+  Alters the value stored under `key` to `value`, but only
+  if the entry `key` already exists in `keywords`.
+
+  If `key` is not present in `keywords`, a `KeyError` exception is raised.
 
   ## Examples
 

lib/elixir/lib/list.ex

@@ -885,19 +885,24 @@ defmodule List do
     compact_reverse(rest, [{kind, [elem | result]} | acc])
   end
 
+  defp compact_reverse(rest, [{:eq, elem}, {:ins, elem}, {:eq, other} | acc]) do
+    compact_reverse(rest, [{:ins, elem}, {:eq, elem ++ other} | acc])
+  end
+
   defp compact_reverse([{kind, elem} | rest], acc) do
     compact_reverse(rest, [{kind, [elem]} | acc])
   end
 
   defp each_diagonal(diag, limit, _paths, next_paths) when diag > limit do
-    {:next, Enum.reverse(next_paths)}
+    {:next, :lists.reverse(next_paths)}
   end
 
   defp each_diagonal(diag, limit, paths, next_paths) do
     {path, rest} = proceed_path(diag, limit, paths)
 
-    with {:cont, path} <- follow_snake(path) do
-      each_diagonal(diag + 2, limit, rest, [path | next_paths])
+    case follow_snake(path) do
+      {:cont, path} -> each_diagonal(diag + 2, limit, rest, [path | next_paths])
+      {:done, edits} -> {:done, edits}
     end
   end
 

lib/elixir/lib/macro.ex

@@ -719,7 +719,13 @@ defmodule Macro do
   end
 
   # All other structures
-  def to_string(other, fun), do: fun.(other, inspect(other, []))
+  def to_string(other, fun) do
+    fun.(other, inspect_no_limit(other))
+  end
+
+  defp inspect_no_limit(value) do
+    Kernel.inspect(value, limit: :infinity, printable_limit: :infinity)
+  end
 
   defp bitpart_to_string({:::, _, [left, right]} = ast, fun) do
     result =
@@ -745,7 +751,7 @@ defmodule Macro do
   end
 
   # Block keywords
-  kw_keywords = [:do, :catch, :rescue, :after, :else]
+  kw_keywords = [:do, :rescue, :catch, :else, :after]
 
   defp kw_blocks?([{:do, _} | _] = kw) do
     Enum.all?(kw, &match?({x, _} when x in unquote(kw_keywords), &1))
@@ -773,7 +779,7 @@ defmodule Macro do
           "\#{" <> to_string(arg, fun) <> "}"
 
         binary when is_binary(binary) ->
-          binary = inspect(binary, [])
+          binary = inspect_no_limit(binary)
           :binary.part(binary, 1, byte_size(binary) - 2)
       end)
 
@@ -781,7 +787,7 @@ defmodule Macro do
   end
 
   defp module_to_string(atom, _fun) when is_atom(atom) do
-    inspect(atom, [])
+    inspect_no_limit(atom)
   end
 
   defp module_to_string({:&, _, [val]} = expr, fun) when not is_integer(val) do
@@ -839,11 +845,25 @@ defmodule Macro do
     :error
   end
 
-  defp sigil_call({sigil, _, [{:<<>>, _, _} = bin, args]} = ast, fun)
+  defp sigil_call({sigil, _, [{:<<>>, _, _} = parts, args]} = ast, fun)
        when is_atom(sigil) and is_list(args) do
     case Atom.to_string(sigil) do
-      <<"sigil_", name>> ->
-        {:ok, fun.(ast, "~" <> <<name>> <> interpolate(bin, fun) <> sigil_args(args, fun))}
+      <<"sigil_", name>> when name >= ?A and name <= ?Z ->
+        {:<<>>, _, [binary]} = parts
+
+        formatted =
+          if :binary.last(binary) == ?\n do
+            binary = String.replace(binary, ~s["""], ~s["\\""])
+            <<?~, name, ~s["""\n], binary::binary, ~s["""], sigil_args(args, fun)::binary>>
+          else
+            {left, right} = select_sigil_container(binary)
+            <<?~, name, left, binary::binary, right, sigil_args(args, fun)::binary>>
+          end
+
+        {:ok, fun.(ast, formatted)}
+
+      <<"sigil_", name>> when name >= ?a and name <= ?z ->
+        {:ok, fun.(ast, "~" <> <<name>> <> interpolate(parts, fun) <> sigil_args(args, fun))}
 
       _ ->
         :error
@@ -854,6 +874,18 @@ defmodule Macro do
     :error
   end
 
+  defp select_sigil_container(binary) do
+    cond do
+      :binary.match(binary, ["\""]) == :nomatch -> {?", ?"}
+      :binary.match(binary, ["\'"]) == :nomatch -> {?', ?'}
+      :binary.match(binary, ["(", ")"]) == :nomatch -> {?(, ?)}
+      :binary.match(binary, ["[", "]"]) == :nomatch -> {?[, ?]}
+      :binary.match(binary, ["{", "}"]) == :nomatch -> {?{, ?}}
+      :binary.match(binary, ["<", ">"]) == :nomatch -> {?<, ?>}
+      true -> {?/, ?/}
+    end
+  end
+
   defp sigil_args([], _fun), do: ""
   defp sigil_args(args, fun), do: fun.(args, List.to_string(args))
 
@@ -952,8 +984,9 @@ defmodule Macro do
   end
 
   defp map_list_to_string(list, fun) do
-    Enum.map_join(list, ", ", fn {key, value} ->
-      to_string(key, fun) <> " => " <> to_string(value, fun)
+    Enum.map_join(list, ", ", fn
+      {key, value} -> to_string(key, fun) <> " => " <> to_string(value, fun)
+      other -> to_string(other, fun)
     end)
   end
 

lib/elixir/lib/map.ex

@@ -297,8 +297,10 @@ defmodule Map do
   end
 
   @doc """
-  Similar to `replace/3`, but will raise a `KeyError`
-  if the key does not exist in the map.
+  Alters the value stored under `key` to `value`, but only
+  if the entry `key` already exists in `map`.
+
+  If `key` is not present in `map`, a `KeyError` exception is raised.
 
   ## Examples
 
@@ -503,6 +505,8 @@ defmodule Map do
   side into the struct, even if the key is not part of the struct. Instead,
   use `Kernel.struct/2`.
 
+  Inlined by the compiler.
+
   ## Examples
 
       iex> Map.merge(%{a: 1, b: 2}, %{a: 3, d: 4})

lib/elixir/lib/map_set.ex

@@ -150,7 +150,8 @@ defmodule MapSet do
   # If the first set is less than twice the size of the second map,
   # it is fastest to re-accumulate items in the first set that are not
   # present in the second set.
-  def difference(%MapSet{map: map1}, %MapSet{map: map2}) when map_size(map1) < map_size(map2) * 2 do
+  def difference(%MapSet{map: map1}, %MapSet{map: map2})
+      when map_size(map1) < map_size(map2) * 2 do
     map =
       map1
       |> Map.keys()

lib/elixir/lib/module.ex

@@ -118,6 +118,13 @@ defmodule Module do
   The mix compiler automatically looks for calls to deprecated modules
   and emit warnings during compilation, computed via `mix xref warnings`.
 
+  We recommend using this feature with care, especially library authors.
+  Deprecating code always pushes the burden towards library users. We
+  also recommend for deprecated functionality to be maintained for long
+  periods of time, even after deprecation, giving developers plenty of
+  time to update (except for cases where keeping the deprecated API is
+  undesired, such as in the presence of security issues).
+
   ### `@doc` (and `@since`)
 
   Provides documentation for the function or macro that follows the
@@ -290,6 +297,8 @@ defmodule Module do
   When just a module is provided, the function is assumed to be
   `__after_compile__/2`.
 
+  Callbacks registered first will run last.
+
   #### Example
 
       defmodule MyModule do
@@ -305,13 +314,18 @@ defmodule Module do
   A hook that will be invoked before the module is compiled.
 
   Accepts a module or a `{module, function_or_macro_name}` tuple. The
-  function/macro must take one argument: the module environment. If it's a
-  macro, its returned value will be injected at the end of the module definition
-  before the compilation starts.
+  function/macro must take one argument: the module environment. If
+  it's a macro, its returned value will be injected at the end of the
+  module definition before the compilation starts.
 
   When just a module is provided, the function/macro is assumed to be
   `__before_compile__/1`.
 
+  Callbacks registered first will run last. Any overridable definition
+  will be made concrete before the first callback runs. A definition may
+  be made overridable again in another before compile callback and it
+  will be made concrete one last time after after all callbacks run.
+
   *Note*: unlike `@after_compile`, the callback function/macro must
   be placed in a separate module (because when the callback is invoked,
   the current module does not yet exist).
@@ -1188,6 +1202,20 @@ defmodule Module do
     raise ArgumentError, "expected an Elixir module, got: #{inspect(original)}"
   end
 
+  @doc false
+  # TODO: Remove in 2.0 - deprecated.
+  def add_doc(module, line, kind, function_tuple, signature \\ [], doc) do
+    assert_not_compiled!(:add_doc, module)
+
+    if kind in [:defp, :defmacrop, :typep] do
+      if doc, do: {:error, :private_doc}, else: :ok
+    else
+      table = data_table_for(module)
+      compile_doc(table, line, kind, function_tuple, signature, doc, __ENV__, false)
+      :ok
+    end
+  end
+
   @doc false
   # Used internally to compile documentation.
   # This function is private and must be used only internally.
@@ -1198,22 +1226,17 @@ defmodule Module do
     pair = {name, arity}
 
     impl = compile_impl(table, name, env, kind, args)
-    compile_doc(table, pair, env, kind, args, impl)
-    compile_deprecated(table, pair)
+    _deprecated = compile_deprecated(table, pair)
+    _since = compile_since(table)
+
+    # TODO: Store @since and @deprecated alongside the docs
+    {line, doc} = get_doc_info(table, env)
+    compile_doc(table, line, kind, pair, args, doc, env, impl)
 
     :ok
   end
 
-  defp compile_doc(table, pair, env, kind, args, impl) do
-    {line, doc} = get_doc_info(table, env, impl)
-
-    # TODO: Store @since alongside the docs
-    _ = get_since_info(table)
-
-    add_doc(table, line, kind, pair, args, doc, env)
-  end
-
-  defp add_doc(_table, line, kind, {name, arity}, _args, doc, env)
+  defp compile_doc(_table, line, kind, {name, arity}, _args, doc, env, _impl)
        when kind in [:defp, :defmacrop] do
     if doc do
       error_message =
@@ -1224,23 +1247,37 @@ defmodule Module do
     end
   end
 
-  defp add_doc(table, line, kind, pair, args, doc, env) do
+  defp compile_doc(table, line, kind, pair, args, doc, env, impl) do
     signature = build_signature(args, env)
 
     case :ets.lookup(table, {:doc, pair}) do
       [] ->
+        doc = if is_nil(doc) && impl, do: false, else: doc
         :ets.insert(table, {{:doc, pair}, line, kind, signature, doc})
 
       [{doc_tuple, line, _current_kind, current_sign, current_doc}] ->
         signature = merge_signatures(current_sign, signature, 1)
         doc = if is_nil(doc), do: current_doc, else: doc
+        doc = if is_nil(doc) && impl, do: false, else: doc
         :ets.insert(table, {doc_tuple, line, kind, signature, doc})
     end
   end
 
+  defp compile_since(table) do
+    case :ets.take(table, :since) do
+      [{:since, since, _, _}] when is_binary(since) -> since
+      _ -> nil
+    end
+  end
+
   defp compile_deprecated(table, pair) do
-    if reason = get_deprecated_info(table) do
-      :ets.insert(table, {{:deprecated, pair}, reason})
+    case :ets.take(table, :deprecated) do
+      [{:deprecated, reason, _, _}] when is_binary(reason) ->
+        :ets.insert(table, {{:deprecated, pair}, reason})
+        reason
+
+      _ ->
+        nil
     end
   end
 
@@ -1301,7 +1338,7 @@ defmodule Module do
 
         not Code.ensure_compiled?(behaviour) ->
           message =
-            "module #{inspect(behaviour)} does not exist (in module #{inspect(env.module)})"
+            "@behaviour #{inspect(behaviour)} does not exist (in module #{inspect(env.module)})"
 
           unless standard_behaviour?(behaviour) do
             :elixir_errors.warn(env.line, env.file, message)
@@ -1716,27 +1753,13 @@ defmodule Module do
     value
   end
 
-  defp get_doc_info(table, env, impl) do
+  defp get_doc_info(table, env) do
     case :ets.take(table, :doc) do
       [{:doc, {_, _} = pair, _, _}] ->
         pair
 
-      [] when impl == false ->
-        {env.line, nil}
-
       [] ->
-        {env.line, false}
-    end
-  end
-
-  defp get_since_info(table) do
-    :ets.take(table, :since)
-  end
-
-  defp get_deprecated_info(table) do
-    case :ets.take(table, :deprecated) do
-      [{:deprecated, reason, _, _}] -> reason
-      [] -> nil
+        {env.line, nil}
     end
   end
 

lib/elixir/lib/module/locals_tracker.ex

@@ -61,7 +61,8 @@ defmodule Module.LocalsTracker do
   @doc """
   Adds an import dispatch to the given target.
   """
-  def add_import(d, function, module, {name, arity}) when is_tuple(function) and is_atom(module) do
+  def add_import(d, function, module, {name, arity})
+      when is_tuple(function) and is_atom(module) do
     tuple = {:import, name, arity}
     put_edge(d, tuple, module)
     put_edge(d, function, tuple)

lib/elixir/lib/port.ex

@@ -132,7 +132,7 @@ defmodule Port do
   script in bash:
 
       #!/bin/sh
-      "$@"
+      "$@" &
       pid=$!
       while read line ; do
         :
@@ -236,8 +236,9 @@ defmodule Port do
 
   def info(port, :registered_name) do
     case :erlang.port_info(port, :registered_name) do
+      :undefined -> nil
       [] -> {:registered_name, []}
-      other -> nillify(other)
+      other -> other
     end
   end
 

lib/elixir/lib/protocol.ex

@@ -317,14 +317,16 @@ defmodule Protocol do
         [
           {:abstract_code, {_raw, abstract_code}},
           {:attributes, attributes},
-          {:compile_info, compile_info},
-          {'ExDc', docs},
-          {'ExDp', deprecated}
+          {:compile_info, compile_info} | extra_chunks
         ] = entries
 
+        extra_chunks =
+          for {name, contents} when is_binary(contents) <- extra_chunks,
+              do: {List.to_string(name), contents}
+
         case attributes[:protocol] do
           [fallback_to_any: any] ->
-            {:ok, {protocol, any, abstract_code}, {compile_info, docs, deprecated}}
+            {:ok, {protocol, any, abstract_code}, {compile_info, extra_chunks}}
 
           _ ->
             {:error, :not_a_protocol}
@@ -493,15 +495,11 @@ defmodule Protocol do
   end
 
   # Finally compile the module and emit its bytecode.
-  defp compile(protocol, code, {compile_info, docs, deprecated}) do
+  defp compile(protocol, code, {compile_info, extra_chunks}) do
     opts = Keyword.take(compile_info, [:source])
     opts = if Code.compiler_options()[:debug_info], do: [:debug_info | opts], else: opts
     {:ok, ^protocol, binary, _warnings} = :compile.forms(code, [:return | opts])
-
-    case docs do
-      :missing_chunk -> {:ok, binary}
-      _ -> {:ok, :elixir_erl.add_beam_chunks(binary, [{"ExDc", docs}, {"ExDp", deprecated}])}
-    end
+    {:ok, :elixir_erl.add_beam_chunks(binary, extra_chunks)}
   end
 
   ## Definition callbacks

lib/elixir/lib/record.ex

@@ -54,11 +54,20 @@ defmodule Record do
       that contains the record definition to extract; with this option, this
       function uses the same path lookup used by the `-include` attribute used in
       Erlang modules.
+
     * `:from_lib` - (binary representing a path to a file) path to the Erlang
       file that contains the record definition to extract; with this option,
       this function uses the same path lookup used by the `-include_lib`
       attribute used in Erlang modules.
 
+    * `:includes` - (a list of directories as binaries) if the record being
+      extracted depends on relative includes, this option allows developers
+      to specify the directory those relative includes exist
+
+    * `:macros` - (keyword list of macro names and values) if the record
+      being extract depends on the values of macros, this option allows
+      the value of those macros to be set
+
   These options are expected to be literals (including the binary values) at
   compile time.
 

lib/elixir/lib/record/extractor.ex

@@ -1,28 +1,25 @@
 defmodule Record.Extractor do
   @moduledoc false
 
-  # Retrieve a record definition from an Erlang file using
-  # the same lookup as the *include* attribute from Erlang modules.
-  def extract(name, from: file) when is_binary(file) do
-    extract_record(name, from_file(file))
+  def extract(name, opts) do
+    extract_record(name, from_or_from_lib_file(opts))
   end
 
-  # Retrieve a record definition from an Erlang file using
-  # the same lookup as the *include_lib* attribute from Erlang modules.
-  def extract(name, from_lib: file) when is_binary(file) do
-    extract_record(name, from_lib_file(file))
+  def extract_all(opts) do
+    extract_all_records(from_or_from_lib_file(opts))
   end
 
-  # Retrieve all records definitions from an Erlang file using
-  # the same lookup as the *include* attribute from Erlang modules.
-  def extract_all(from: file) when is_binary(file) do
-    extract_all_records(from_file(file))
-  end
+  defp from_or_from_lib_file(opts) do
+    cond do
+      file = opts[:from] ->
+        {from_file(file), Keyword.delete(opts, :from)}
 
-  # Retrieve all records definitions from an Erlang file using
-  # the same lookup as the *include_lib* attribute from Erlang modules.
-  def extract_all(from_lib: file) when is_binary(file) do
-    extract_all_records(from_lib_file(file))
+      file = opts[:from_lib] ->
+        {from_lib_file(file), Keyword.delete(opts, :from_lib)}
+
+      true ->
+        raise ArgumentError, "expected :from or :from_lib to be given as option"
+    end
   end
 
   # Find file using the same lookup as the *include* attribute from Erlang modules.
@@ -49,20 +46,22 @@ defmodule Record.Extractor do
   end
 
   # Retrieve the record with the given name from the given file
-  defp extract_record(name, file) do
-    form = read_file(file)
+  defp extract_record(name, {file, opts}) do
+    form = read_file(file, opts)
     records = extract_records(form)
 
     if record = List.keyfind(records, name, 0) do
       parse_record(record, form)
     else
-      raise ArgumentError, "no record #{name} found at #{file}"
+      raise ArgumentError,
+            "no record #{name} found at #{file}. Or the record does not exist or " <>
+              "its entry is malformed or depends on other include files"
     end
   end
 
   # Retrieve all records from the given file
-  defp extract_all_records(file) do
-    form = read_file(file)
+  defp extract_all_records({file, opts}) do
+    form = read_file(file, opts)
     records = extract_records(form)
     for rec = {name, _fields} <- records, do: {name, parse_record(rec, form)}
   end
@@ -76,8 +75,8 @@ defmodule Record.Extractor do
   # includes record but with macros and other attributes expanded,
   # such as "-include(...)" and "-include_lib(...)". This is done
   # by using Erlang's epp.
-  defp read_file(file) do
-    case :epp.parse_file(file, []) do
+  defp read_file(file, opts) do
+    case :epp.parse_file(file, opts) do
       {:ok, form} ->
         form
 

lib/elixir/lib/regex.ex

@@ -42,9 +42,9 @@ defmodule Regex do
 
   For such reasons, we always recommend precompiling Elixir projects using
   the OTP version meant to run in production. In case cross-compilation is
-  really necessary, you can manually invoke `Regex.recompile/1` or `Regex.
-  recompile!/1` to perform a runtime version check and recompile the regex
-  if necessary.
+  really necessary, you can manually invoke `Regex.recompile/1` or
+  `Regex.recompile!/1` to perform a runtime version check and recompile the
+  regex if necessary.
 
   ## Modifiers
 
@@ -465,7 +465,8 @@ defmodule Regex do
     end
   end
 
-  def split(%Regex{re_pattern: compiled}, string, opts) when is_binary(string) and is_list(opts) do
+  def split(%Regex{re_pattern: compiled}, string, opts)
+      when is_binary(string) and is_list(opts) do
     on = Keyword.get(opts, :on, :first)
 
     case :re.run(string, compiled, [:global, capture: on]) do
@@ -653,7 +654,8 @@ defmodule Regex do
     string
   end
 
-  defp apply_list(whole, string, pos, replacement, [[{mpos, _} | _] | _] = list) when mpos > pos do
+  defp apply_list(whole, string, pos, replacement, [[{mpos, _} | _] | _] = list)
+       when mpos > pos do
     length = mpos - pos
     <<untouched::binary-size(length), rest::binary>> = string
     [untouched | apply_list(whole, rest, mpos, replacement, list)]

lib/elixir/lib/registry.ex

@@ -321,7 +321,12 @@ defmodule Registry do
     start_link([keys: keys, name: name] ++ options)
   end
 
-  @doc false
+  @doc """
+  Returns a specification to start a registry under a supervisor.
+
+  See `Supervisor`.
+  """
+  @since "1.5.0"
   def child_spec(opts) do
     %{
       id: Keyword.get(opts, :name, Registry),

lib/elixir/lib/stream.ex

@@ -318,11 +318,11 @@ defmodule Stream do
 
   """
   @spec drop(Enumerable.t(), non_neg_integer) :: Enumerable.t()
-  def drop(enum, n) when n >= 0 do
+  def drop(enum, n) when is_integer(n) and n >= 0 do
     lazy(enum, n, fn f1 -> R.drop(f1) end)
   end
 
-  def drop(enum, n) when n < 0 do
+  def drop(enum, n) when is_integer(n) and n < 0 do
     n = abs(n)
 
     lazy(enum, {0, [], []}, fn f1 ->
@@ -1091,10 +1091,10 @@ defmodule Stream do
   def zip(left, right), do: zip([left, right])
 
   @doc """
-  Zips corresponding elements from a list of enumerables
+  Zips corresponding elements from a finite collection of enumerables
   into one stream of tuples.
 
-  The zipping finishes as soon as any enumerable in the given list completes.
+  The zipping finishes as soon as any enumerable in the given collection completes.
 
   ## Examples
 
@@ -1105,7 +1105,12 @@ defmodule Stream do
 
   """
   @spec zip([Enumerable.t()]) :: Enumerable.t()
-  def zip(enumerables) when is_list(enumerables) do
+  @spec zip(Enumerable.t()) :: Enumerable.t()
+  def zip(enumerables) do
+    &prepare_zip(enumerables, &1, &2)
+  end
+
+  defp prepare_zip(enumerables, acc, fun) do
     step = &do_zip_step(&1, &2)
 
     enum_funs =
@@ -1113,7 +1118,7 @@ defmodule Stream do
         {&Enumerable.reduce(enum, &1, step), :cont}
       end)
 
-    &do_zip(enum_funs, &1, &2)
+    do_zip(enum_funs, acc, fun)
   end
 
   # This implementation of do_zip/3 works for any number of
@@ -1128,6 +1133,10 @@ defmodule Stream do
     {:suspended, acc, &do_zip(zips, &1, fun)}
   end
 
+  defp do_zip([], {:cont, acc}, _callback) do
+    {:done, acc}
+  end
+
   defp do_zip(zips, {:cont, acc}, callback) do
     try do
       do_zip_next_tuple(zips, acc, callback, [], [])

lib/elixir/lib/string.ex

@@ -635,20 +635,28 @@ defmodule String do
   @spec upcase(t, :default | :ascii | :greek) :: t
   def upcase(string, mode \\ :default)
 
+  def upcase("", _mode) do
+    ""
+  end
+
   def upcase(string, :default) when is_binary(string) do
-    String.Casing.upcase(string, "", :default)
+    String.Casing.upcase(string, [], :default)
   end
 
   def upcase(string, :ascii) when is_binary(string) do
-    for <<x <- string>>,
-      do: if(x >= ?a and x <= ?z, do: <<x - 32>>, else: <<x>>),
-      into: ""
+    IO.iodata_to_binary(upcase_ascii(string))
   end
 
   def upcase(string, mode) when mode in @conditional_mappings do
-    String.Casing.upcase(string, "", mode)
+    String.Casing.upcase(string, [], mode)
   end
 
+  defp upcase_ascii(<<char, rest::bits>>) when char >= ?a and char <= ?z,
+    do: [char - 32 | upcase_ascii(rest)]
+
+  defp upcase_ascii(<<char, rest::bits>>), do: [char | upcase_ascii(rest)]
+  defp upcase_ascii(<<>>), do: []
+
   @doc """
   Converts all characters in the given string to lowercase according to `mode`.
 
@@ -678,7 +686,7 @@ defmodule String do
   And `:greek` properly handles the context sensitive sigma in Greek:
 
       iex> String.downcase("ΣΣ")
-      "ςς"
+      "σσ"
 
       iex> String.downcase("ΣΣ", :greek)
       "σς"
@@ -687,20 +695,28 @@ defmodule String do
   @spec downcase(t, :default | :ascii | :greek) :: t
   def downcase(string, mode \\ :default)
 
+  def downcase("", _mode) do
+    ""
+  end
+
   def downcase(string, :default) when is_binary(string) do
-    String.Casing.downcase(string, "", :default)
+    String.Casing.downcase(string, [], :default)
   end
 
   def downcase(string, :ascii) when is_binary(string) do
-    for <<x <- string>>,
-      do: if(x >= ?A and x <= ?Z, do: <<x + 32>>, else: <<x>>),
-      into: ""
+    IO.iodata_to_binary(downcase_ascii(string))
   end
 
   def downcase(string, mode) when mode in @conditional_mappings do
-    String.Casing.downcase(string, "", mode)
+    String.Casing.downcase(string, [], mode)
   end
 
+  defp downcase_ascii(<<char, rest::bits>>) when char >= ?A and char <= ?Z,
+    do: [char + 32 | downcase_ascii(rest)]
+
+  defp downcase_ascii(<<char, rest::bits>>), do: [char | downcase_ascii(rest)]
+  defp downcase_ascii(<<>>), do: []
+
   @doc """
   Converts the first character in the given string to
   uppercase and the remainder to lowercase according to `mode`.

lib/elixir/lib/string_io.ex

@@ -99,11 +99,13 @@ defmodule StringIO do
 
   ## callbacks
 
+  @impl true
   def init({string, options}) do
     capture_prompt = options[:capture_prompt] || false
     {:ok, %{input: string, output: "", capture_prompt: capture_prompt}}
   end
 
+  @impl true
   def handle_info({:io_request, from, reply_as, req}, state) do
     state = io_request(from, reply_as, req, state)
     {:noreply, state}
@@ -113,6 +115,7 @@ defmodule StringIO do
     super(message, state)
   end
 
+  @impl true
   def handle_call(:contents, _from, %{input: input, output: output} = state) do
     {:reply, {input, output}, state}
   end

lib/elixir/lib/supervisor.ex

@@ -28,14 +28,17 @@ defmodule Supervisor do
 
         ## Callbacks
 
+        @impl true
         def init(stack) do
           {:ok, stack}
         end
 
+        @impl true
         def handle_call(:pop, _from, [h | t]) do
           {:reply, h, t}
         end
 
+        @impl true
         def handle_cast({:push, h}, t) do
           {:noreply, [h | t]}
         end
@@ -101,13 +104,10 @@ defmodule Supervisor do
 
   ## Start and shutdown
 
-  When the supervisor starts, it traverses all children and retrieves
-  each child specification. It is at this moment `{Stack, [:hello]}`
-  becomes a child specification by calling `Stack.child_spec([:hello])`.
-
-  Then the supervisor starts each child in the order they are defined.
-  This is done by calling the function defined under the `:start` key
-  in the child specification and typically defaults to `start_link/1`.
+  When the supervisor starts, it traverses all child specifications and
+  then starts each child in the order they are defined. This is done by
+  calling the function defined under the `:start` key in the child
+  specification and typically defaults to `start_link/1`.
 
   The `start_link/1` (or a custom) is then called for each child process.
   The `start_link/1` function must return `{:ok, pid}` where `pid` is the
@@ -124,8 +124,8 @@ defmodule Supervisor do
   then awaiting for a time interval for the child process to terminate. This
   interval defaults to 5000 milliseconds. If the child process does not
   terminate in this interval, the supervisor abruptly terminates the child
-  with reason `:brutal_kill`. The shutdown time can be configured in the
-  child specification which is fully detailed in the next section.
+  with reason `:kill`. The shutdown time can be configured in the child
+  specification which is fully detailed in the next section.
 
   If the child process is not trapping exits, it will shutdown immediately
   when it receives the first exit signal. If the child process is trapping
@@ -212,7 +212,8 @@ defmodule Supervisor do
     * `:permanent` - the child process is always restarted.
 
     * `:temporary` - the child process is never restarted, regardless
-      of the supervision strategy.
+      of the supervision strategy: any termination (even abnormal) is
+      considered successful.
 
     * `:transient` - the child process is restarted only if it
       terminates abnormally, i.e., with an exit reason other than
@@ -301,7 +302,9 @@ defmodule Supervisor do
   function.
 
   You may also completely override the `child_spec/1` function in the Stack module
-  and return your own child specification.
+  and return your own child specification. Note there is no guarantee the `child_spec/1`
+  function will be called by the Supervisor process, as other processes may invoke
+  it to retrieve the child specification before reaching the supervisor.
 
   ## Exit reasons and restarts
 
@@ -343,6 +346,7 @@ defmodule Supervisor do
           Supervisor.start_link(__MODULE__, arg, name: __MODULE__)
         end
 
+        @impl true
         def init(_arg) do
           children = [
             {Stack, [:hello]}
@@ -441,12 +445,16 @@ defmodule Supervisor do
       the child processes, i.e., the child processes after the terminated
       one in start order, are terminated. Then the terminated child
       process and the rest of the child processes are restarted.
+      
+  In the above, process termination refers to unsuccessful termination, which
+  is determined by the `:restart` option.
 
   There is also a deprecated strategy called `:simple_one_for_one` which
   has been replaced by the `DynamicSupervisor`. The `:simple_one_for_one`
   supervisor was similar to `:one_for_one` but suits better when dynamically
   attaching children. Many functions in this module behaved slightly
-  differently when this strategy is used.
+  differently when this strategy is used. See the `DynamicSupervisor` module
+  for more information and migration strategies.
 
   ## Name registration
 
@@ -456,12 +464,15 @@ defmodule Supervisor do
 
   @doc false
   defmacro __using__(opts) do
-    quote location: :keep do
+    quote location: :keep, bind_quoted: [opts: opts] do
       import Supervisor.Spec
       @behaviour Supervisor
-      @opts unquote(opts)
 
-      @doc false
+      @doc """
+      Returns a specification to start this module under a supervisor.
+
+      See `Supervisor`.
+      """
       def child_spec(arg) do
         default = %{
           id: __MODULE__,
@@ -469,13 +480,10 @@ defmodule Supervisor do
           type: :supervisor
         }
 
-        Supervisor.child_spec(default, @opts)
+        Supervisor.child_spec(default, unquote(Macro.escape(opts)))
       end
 
       defoverridable child_spec: 1
-
-      @doc false
-      def init(arg)
     end
   end
 
@@ -528,7 +536,7 @@ defmodule Supervisor do
   @typedoc "The supervisor specification"
   @type child_spec :: %{
           required(:id) => term(),
-          required(:start) => {module(), function(), [term()]},
+          required(:start) => {module(), atom(), [term()]},
           optional(:restart) => :permanent | :transient | :temporary,
           optional(:shutdown) => :brutal_kill | non_neg_integer() | :infinity,
           optional(:type) => :worker | :supervisor,
@@ -592,7 +600,7 @@ defmodule Supervisor do
   ## Options
 
     * `:strategy` - the restart strategy option. It can be either
-      `:one_for_one`, `:rest_for_one`, `:one_for_all`, or
+      `:one_for_one`, `:rest_for_one`, `:one_for_all`, or the deprecated
       `:simple_one_for_one`.
 
     * `:max_restarts` - the maximum number of restarts allowed in
@@ -722,14 +730,6 @@ defmodule Supervisor do
       #=> %{id: {Agent, 1},
       #=>   start: {Agent, :start_link, [fn -> :ok end]}}
 
-  It may also be used when there is a need to change the number
-  of arguments when starting a module under a `:simple_one_for_one`
-  strategy, since most args may be given dynamically:
-
-      Supervisor.child_spec(Agent, start: {Agent, :start_link, []})
-      #=> %{id: Agent,
-      #=>   start: {Agent, :start_link, []}}
-
   """
   @spec child_spec(child_spec() | {module, arg :: term} | module, keyword) :: child_spec()
   def child_spec(module_or_map, overrides)
@@ -768,6 +768,10 @@ defmodule Supervisor do
   name, the supported values are described in the "Name registration"
   section in the `GenServer` module docs.
   """
+
+  # It is important to keep the 2-arity spec because it is a catch
+  # all to start_link(children, options).
+  @spec start_link(module, term) :: on_start
   @spec start_link(module, term, GenServer.options()) :: on_start
   def start_link(module, arg, options \\ []) when is_list(options) do
     case Keyword.get(options, :name) do
@@ -821,13 +825,13 @@ defmodule Supervisor do
   returns `{:error, error}` where `error` is a term containing information about
   the error and child specification.
   """
-  @spec start_child(supervisor, :supervisor.child_spec() | {module, term} | module) ::
+  @spec start_child(supervisor, :supervisor.child_spec() | {module, term} | module | [term]) ::
           on_start_child
   def start_child(supervisor, {_, _, _, _, _, _} = child_spec) do
     call(supervisor, {:start_child, child_spec})
   end
 
-  # TODO: Deprecate this on Elixir v1.8
+  # TODO: Deprecate this on Elixir v1.8. Remove and update typespec on v2.0.
   def start_child(supervisor, args) when is_list(args) do
     call(supervisor, {:start_child, args})
   end

lib/elixir/lib/task.ex

@@ -173,7 +173,12 @@ defmodule Task do
 
   @type t :: %__MODULE__{}
 
-  @doc false
+  @doc """
+  Returns a specification to start a task under a supervisor.
+
+  See `Supervisor`.
+  """
+  @since "1.5.0"
   def child_spec(arg) do
     %{
       id: Task,
@@ -184,10 +189,13 @@ defmodule Task do
 
   @doc false
   defmacro __using__(opts) do
-    quote location: :keep do
-      @opts unquote(opts)
+    quote location: :keep, bind_quoted: [opts: opts] do
+      @doc """
+      Returns a specification to start this module under a supervisor.
 
-      @doc false
+      See `Supervisor`.
+      """
+      @since "1.5.0"
       def child_spec(arg) do
         default = %{
           id: __MODULE__,
@@ -195,7 +203,7 @@ defmodule Task do
           restart: :temporary
         }
 
-        Supervisor.child_spec(default, @opts)
+        Supervisor.child_spec(default, unquote(Macro.escape(opts)))
       end
 
       defoverridable child_spec: 1
@@ -431,8 +439,8 @@ defmodule Task do
   defp get_info(pid) do
     self_or_name =
       case Process.info(pid, :registered_name) do
-        {:registered_name, []} -> self()
-        {:registered_name, name} -> name
+        {:registered_name, name} when is_atom(name) -> name
+        _ -> pid
       end
 
     {node(), self_or_name}
@@ -640,7 +648,11 @@ defmodule Task do
   @spec yield_many([t], timeout) :: [{t, {:ok, term} | {:exit, term} | nil}]
   def yield_many(tasks, timeout \\ 5000) do
     timeout_ref = make_ref()
-    timer_ref = Process.send_after(self(), timeout_ref, timeout)
+
+    timer_ref =
+      if timeout != :infinity do
+        Process.send_after(self(), timeout_ref, timeout)
+      end
 
     try do
       yield_many(tasks, timeout_ref, :infinity)
@@ -648,7 +660,7 @@ defmodule Task do
       {:noconnection, reason} ->
         exit({reason, {__MODULE__, :yield_many, [tasks, timeout]}})
     after
-      Process.cancel_timer(timer_ref)
+      timer_ref && Process.cancel_timer(timer_ref)
       receive do: (^timeout_ref -> :ok), after: (0 -> :ok)
     end
   end

lib/elixir/lib/task/supervisor.ex

@@ -5,8 +5,18 @@ defmodule Task.Supervisor do
   This module defines a supervisor which can be used to dynamically
   supervise tasks.
 
-  `start_link/1` can be used to start the supervisor. See the `Task`
-  module for more examples.
+  A task supervisor is started with no children, often under a
+  supervisor and a name:
+
+      children = [
+        {Task.Supervisor, name: MyApp.TaskSupervisor}
+      ]
+
+      Supervisor.start_link(strategy: :one_for_one)
+
+  The options given in the child specification are documented in `start_link/1`.
+
+  See the `Task` module for more examples.
 
   ## Name registration
 
@@ -20,11 +30,6 @@ defmodule Task.Supervisor do
           | {:restart, :supervisor.restart()}
           | {:shutdown, :supervisor.shutdown()}
 
-  @typedoc "Supervisor spec used by `async_stream`"
-  @type async_stream_supervisor ::
-          Supervisor.supervisor()
-          | (term -> Supervisor.supervisor())
-
   @doc false
   def child_spec(arg) do
     %{
@@ -37,17 +42,33 @@ defmodule Task.Supervisor do
   @doc """
   Starts a new supervisor.
 
-  The supported options are:
+  ## Examples
+
+  A task supervisor is typically started under a supervision tree using
+  the tuple format:
+
+      {Task.Supervisor, name: MyApp.TaskSupervisor}
+
+  You can also start it by calling `start_link/1` directly:
+
+      Task.Supervisor.start_link(name: MyApp.TaskSupervisor)
+
+  But this is recommended only for scripting and should be avoided in
+  production code. Generally speaking, processes should always be started
+  inside supervision trees.
+
+  ## Options
 
     * `:name` - used to register a supervisor name, the supported values are
       described under the `Name Registration` section in the `GenServer` module
       docs;
 
-    * `:max_restarts`, `:max_seconds` and `:max_children` - as specified in `DynamicSupervisor`;
+    * `:max_restarts`, `:max_seconds` and `:max_children` - as specified in
+      `DynamicSupervisor`;
 
   This function could also receive `:restart` and `:shutdown` as options
   but those two options have been deprecated and it is now preferred to
-  give them directly to `start_child` and `async` when supported.
+  give them directly to `start_child` and `async`.
   """
   @spec start_link([option]) :: Supervisor.on_start()
   # TODO: Deprecate passing restart and shutdown here on Elixir v1.8.
@@ -158,15 +179,6 @@ defmodule Task.Supervisor do
   own task. The tasks will be spawned under the given `supervisor` and
   linked to the current process, similarly to `async/4`.
 
-  You may also provide a function as the `supervisor`. Before each task is
-  started, the function will be invoked (in a new process which is linked to
-  the current process) with the stream entry that the to-be-spawned task will
-  process as its argument. The function should return a supervisor pid or name,
-  which will be used to spawn the task. This allows one to dynamically start
-  tasks in different locations in the supervision tree(s) on the local (or
-  another) node. Notably, this enables the distribution of concurrent stream
-  tasks over multiple nodes.
-
   When streamed, each task will emit `{:ok, value}` upon successful
   completion or `{:exit, reason}` if the caller is trapping exits.
   Results are emitted in the same order as the original `enumerable`.
@@ -207,7 +219,7 @@ defmodule Task.Supervisor do
       Enum.to_list(stream)
 
   """
-  @spec async_stream(async_stream_supervisor, Enumerable.t(), module, atom, [term], keyword) ::
+  @spec async_stream(Supervisor.supervisor(), Enumerable.t(), module, atom, [term], keyword) ::
           Enumerable.t()
   def async_stream(supervisor, enumerable, module, function, args, options \\ [])
       when is_atom(module) and is_atom(function) and is_list(args) do
@@ -224,7 +236,7 @@ defmodule Task.Supervisor do
 
   See `async_stream/6` for discussion, options, and examples.
   """
-  @spec async_stream(async_stream_supervisor, Enumerable.t(), (term -> term), keyword) ::
+  @spec async_stream(Supervisor.supervisor(), Enumerable.t(), (term -> term), keyword) ::
           Enumerable.t()
   def async_stream(supervisor, enumerable, fun, options \\ []) when is_function(fun, 1) do
     build_stream(supervisor, :link, enumerable, fun, options)
@@ -241,7 +253,7 @@ defmodule Task.Supervisor do
   See `async_stream/6` for discussion, options, and examples.
   """
   @spec async_stream_nolink(
-          async_stream_supervisor,
+          Supervisor.supervisor(),
           Enumerable.t(),
           module,
           atom,
@@ -263,7 +275,7 @@ defmodule Task.Supervisor do
 
   See `async_stream/6` for discussion and examples.
   """
-  @spec async_stream_nolink(async_stream_supervisor, Enumerable.t(), (term -> term), keyword) ::
+  @spec async_stream_nolink(Supervisor.supervisor(), Enumerable.t(), (term -> term), keyword) ::
           Enumerable.t()
   def async_stream_nolink(supervisor, enumerable, fun, options \\ []) when is_function(fun, 1) do
     build_stream(supervisor, :nolink, enumerable, fun, options)
@@ -305,7 +317,7 @@ defmodule Task.Supervisor do
       or an integer indicating the timeout value, defaults to 5000 milliseconds.
 
   """
-  @spec start_child(Supervisor.supervisor(), (() -> any)) :: {:ok, pid}
+  @spec start_child(Supervisor.supervisor(), (() -> any)) :: DynamicSupervisor.on_start_child()
   def start_child(supervisor, fun, options \\ []) do
     restart = options[:restart]
     shutdown = options[:shutdown]
@@ -319,7 +331,8 @@ defmodule Task.Supervisor do
   Similar to `start_child/2` except the task is specified
   by the given `module`, `fun` and `args`.
   """
-  @spec start_child(Supervisor.supervisor(), module, atom, [term]) :: {:ok, pid}
+  @spec start_child(Supervisor.supervisor(), module, atom, [term]) ::
+          DynamicSupervisor.on_start_child()
   def start_child(supervisor, module, fun, args, options \\ [])
       when is_atom(fun) and is_list(args) do
     restart = options[:restart]
@@ -336,8 +349,8 @@ defmodule Task.Supervisor do
   defp get_info(self) do
     name =
       case Process.info(self, :registered_name) do
-        {:registered_name, []} -> self
-        {:registered_name, name} -> name
+        {:registered_name, name} when is_atom(name) -> name
+        _ -> self
       end
 
     {node(), name}
@@ -354,27 +367,12 @@ defmodule Task.Supervisor do
     %Task{pid: pid, ref: ref, owner: owner}
   end
 
-  defp supervisor_fun(supervisor, {_module, _fun, _args})
-       when is_function(supervisor, 1) do
-    fn {_module, _fun, [entry | _rest_args]} -> supervisor.(entry) end
-  end
-
-  defp supervisor_fun(supervisor, fun)
-       when is_function(supervisor, 1) and is_function(fun, 1) do
-    fn {_erlang, _apply, [_fun, [entry]]} -> supervisor.(entry) end
-  end
-
-  defp supervisor_fun(supervisor, _fun) do
-    fn _mfa -> supervisor end
-  end
-
   defp build_stream(supervisor, link_type, enumerable, fun, options) do
-    supervisor_fun = supervisor_fun(supervisor, fun)
     shutdown = options[:shutdown]
 
     &Task.Supervised.stream(enumerable, &1, &2, fun, options, fn owner, mfa ->
       args = [owner, :monitor, get_info(owner), mfa]
-      {:ok, pid} = start_child_with_spec(supervisor_fun.(mfa), args, :temporary, shutdown)
+      {:ok, pid} = start_child_with_spec(supervisor, args, :temporary, shutdown)
       if link_type == :link, do: Process.link(pid)
       {link_type, pid}
     end)

lib/elixir/lib/tuple.ex

@@ -2,9 +2,9 @@ defmodule Tuple do
   @moduledoc """
   Functions for working with tuples.
 
-  Tuples are ordered collections of elements. Tuples can contain elements
-  of any type, and a tuple can contain elements of different types. Curly
-  braces can be used to create tuples:
+  Tuples are composite data types with a fixed number of elements. Tuples
+  can contain elements of any type, and a tuple can contain elements of
+  different types. Curly braces can be used to create tuples:
 
       iex> {}
       {}

lib/elixir/lib/uri.ex

@@ -422,11 +422,13 @@ defmodule URI do
 
     parts = Regex.run(regex, string)
 
-    destructure [_, _, scheme, _, authority, path, _, query, _, fragment], parts
+    destructure [_, _, scheme, _, authority, path, query_with_question_mark, _, _, fragment],
+                parts
+
     scheme = nillify(scheme)
     authority = nillify(authority)
     path = nillify(path)
-    query = nillify(query)
+    query = nillify_query(query_with_question_mark)
     {userinfo, host, port} = split_authority(authority)
 
     scheme = scheme && String.downcase(scheme)
@@ -444,6 +446,10 @@ defmodule URI do
     }
   end
 
+  defp nillify_query("?"), do: ""
+  defp nillify_query("?" <> query), do: query
+  defp nillify_query(_other), do: nil
+
   # Split an authority into its userinfo, host and port parts.
   defp split_authority(string) do
     regex = Regex.recompile!(~r/(^(.*)@)?(\[[a-zA-Z0-9:.]*\]|[^:]*)(:(\d*))?/)

lib/elixir/pages/Compatibility and Deprecations.md

@@ -6,35 +6,15 @@ Erlang/OTP versioning is independent from the versioning of Elixir. Each version
 
 Elixir version | Supported Erlang/OTP versions
 :------------- | :----------------------------
-v1.0.0         | 17
-v1.0.1         | 17
-v1.0.2         | 17
-v1.0.3         | 17
-v1.0.4         | 17
-v1.0.5         | 17-18
-v1.1.0         | 17-18
-v1.1.1         | 17-18
-v1.2.0         | 18
-v1.2.1         | 18
-v1.2.2         | 18
-v1.2.3         | 18
-v1.2.4         | 18
-v1.2.5         | 18
-v1.2.6         | 18-19
-v1.3.0         | 18-19
-v1.3.1         | 18-19
-v1.3.2         | 18-19
-v1.3.3         | 18-19
-v1.3.4         | 18-19
-v1.4.0         | 18-19
-v1.4.1         | 18-19
-v1.4.2         | 18-19
-v1.4.3         | 18-19
-v1.4.4         | 18-19
-v1.4.5         | 18-20
-v1.5.0         | 18-20
-v1.5.1         | 18-20
-v1.5.2         | 18-20
+1.0            | 17 - 17 (and OTP 18 from v1.0.5)
+1.1            | 17 - 18
+1.2            | 18 - 18 (and OTP 19 from v1.2.6)
+1.3            | 18 - 19
+1.4            | 18 - 19 (and OTP 20 from v1.4.5)
+1.5            | 18 - 20
+1.6            | 19 - 20 (and OTP 21 from v1.6.6)
+
+While Elixir often adds compatibility to new Erlang versions on released branches, such as support for OTP 20 in v1.4.5, those releases usually contain the minimum changes for Elixir to run without errors. Only the next minor release, in this case v1.5.0, does effectively leverage the new features provided by the latest Erlang release.
 
 ## Deprecations
 
@@ -54,9 +34,10 @@ Deprecated feature                               | Deprecated in | Replaced by (
 :----------------------------------------------- | :------------ | :----------------------------
 `Enum.partition/2`                               | [v1.6]        | `Enum.split_with/2` (v1.4)
 `Keyword.replace/3`                              | [v1.6]        | Use `Keyword.fetch/2` + `Keyword.put/3` (v1.0)
-`Map.replace/3`                                  | [v1.6]        | Use `Map.fetch/2` + `Map.put/3` (v1.0)
 `Macro.unescape_tokens/1` and `Macro.unescape_tokens/2` | [v1.6] | Use `Enum.map/2` to traverse over the arguments (v1.0)
-`Range.range?/1`                                 | [v1.6]        | Pattern match on `left..right` instead (v1.0)
+`Module.add_doc/6`                               | [v1.6]        | Use `@doc` instead
+`Map.replace/3`                                  | [v1.6]        | Use `Map.fetch/2` + `Map.put/3` (v1.0)
+`Range.range?/1`                                 | [v1.6]        | Pattern match on `_.._` instead (v1.0)
 `Atom.to_char_list/1`                            | [v1.5]        | `Atom.to_charlist/1` (v1.3)
 `Enum.filter_map/3`                              | [v1.5]        | `Enum.filter/2` + `Enum.map/2` or for comprehensions (v1.0)
 `Float.to_char_list/1`                           | [v1.5]        | `Float.to_charlist/1` (v1.3)
@@ -83,14 +64,16 @@ EEx: `<%=` in middle and end expressions         | [v1.5]        | Use `<%` (= i
 `Float.to_string/2`                              | [v1.4]        | `:erlang.float_to_binary/2` (OTP 17)
 `HashDict` module                                | [v1.4]        | `Map` (v1.2)
 `HashSet` module                                 | [v1.4]        | `MapSet` (v1.1)
+ Multi-letter aliases in `OptionParser`          | [v1.4]        | Use single-letter aliases (v1.0)
 `Set` module                                     | [v1.4]        | `MapSet` (v1.1)
 `Stream.uniq/2`                                  | [v1.4]        | `Stream.uniq_by/2` (v1.2)
-`IEx.Helpers.import_file/2`                      | [v1.4]        | [`IEx.Helpers.import_file_if_available/1`](https://hexdocs.pm/iex/IEx.Helpers.html#import_file_if_available/1) (v1.3)
+`IEx.Helpers.import_file/2`                      | [v1.4]        | `IEx.Helpers.import_file_if_available/1` (v1.3)
 `Mix.Utils.camelize/1`                           | [v1.4]        | `Macro.camelize/1` (v1.2)
 `Mix.Utils.underscore/1`                         | [v1.4]        | `Macro.underscore/1` (v1.2)
 Variable used as function call                   | [v1.4]        | Use parentheses (v1.0)
 Anonymous functions with no expression after `->` | [v1.4]       | Use an expression or explicitly return `nil` (v1.0)
-`Dict` module                                    | [v1.3]        | `Keyword` (v1.0);<br/>`Map` (v1.2)
+Support for making private functions overridable | [v1.4]        | Use public functions (v1.0)
+`Dict` module                                    | [v1.3]        | `Keyword` (v1.0) or `Map` (v1.2)
 `Keyword.size/1`                                 | [v1.3]        | `Kernel.length/1` (v1.0)
 `Map.size/1`                                     | [v1.3]        | `Kernel.map_size/1` (v1.0)
 `Set` behaviour                                  | [v1.3]        | `MapSet` data structure (v1.1)
@@ -103,13 +86,12 @@ Map or dictionary as second argument in `Enum.group_by/3` | [v1.3] | Use `Enum.r
 Non-map as second argument in `URI.decode_query/2` | [v1.3]      | Use a map (v1.0)
 `Dict` behaviour                                 | [v1.2]        | `MapSet` data structure (v1.1)
 `Access` protocol                                | [v1.1]        | `Access` behaviour (v1.1)
-`as: true \| false` in `alias/2` and `require/2` | [v1.1]     | *None*
+`as: true \| false` in `alias/2` and `require/2` | [v1.1]        | *None*
 `?\xHEX`                                         | [v1.1]        | `0xHEX` (v1.0)
-Empty string in `String.starts_with?/2`, `String.ends_with?/2`, `String.contains?/2`.<br/>*__NOTE__: Feature made back available in v1.3* | [v1.1] to [v1.2] | Explicitly check for `""` beforehand (v1.0)
 
 [v1.1]: https://github.com/elixir-lang/elixir/blob/v1.1/CHANGELOG.md#4-deprecations
 [v1.2]: https://github.com/elixir-lang/elixir/blob/v1.2/CHANGELOG.md#changelog-for-elixir-v12
 [v1.3]: https://github.com/elixir-lang/elixir/blob/v1.3/CHANGELOG.md#4-deprecations
 [v1.4]: https://github.com/elixir-lang/elixir/blob/v1.4/CHANGELOG.md#4-deprecations
 [v1.5]: https://github.com/elixir-lang/elixir/blob/v1.5/CHANGELOG.md#4-deprecations
-[v1.6]: https://github.com/elixir-lang/elixir/blob/master/CHANGELOG.md#4-deprecations
+[v1.6]: https://github.com/elixir-lang/elixir/blob/v1.6/CHANGELOG.md#4-deprecations

lib/elixir/pages/Guards.md

@@ -1,6 +1,8 @@
 # Guards
 
-Guards are a way to augment pattern matching with more complex checks; they are allowed in a predefined set of constructs where pattern matching is allowed.
+Guards are a way to augment pattern matching with more complex checks. They are allowed in a predefined set of constructs where pattern matching is allowed.
+
+Not all expressions are allowed in guard clauses, but only a handful of them. This is a deliberate choice. This way, Elixir (and Erlang) can make sure that nothing bad happens while executing guards and no mutations happen anywhere. It also allows the compiler to optimize the code related to guards efficiently.
 
 ## List of allowed expressions
 
@@ -100,11 +102,15 @@ In the example above, we show how guards can be used in function clauses. There
   end
   ```
 
+  * custom guards can also be defined with `Kernel.defguard/1` and `Kernel.defguardp/1`.
+    A custom guard is always defined based on existing guards.
+
 Other constructs are `for`, `with`, `try`/`rescue`/`catch`/`else`/, and the `match?/2` macro in the `Kernel` module.
 
 ## Failing guards
 
-Errors in guards do not result in runtime errors, but in guards failing. For example, the `length/1` function only works with lists. If we use it with anything else, a runtime error is raised:
+In guards, when functions would normally raise exceptions, they cause the guard to fail instead.
+For example, the `length/1` function only works with lists. If we use it with anything else, a runtime error is raised:
 
 ```elixir
 iex> length("hello")
@@ -125,10 +131,6 @@ iex> case "hello" do
 
 In many cases, we can take advantage of this. In the code above, we used `length/1` to both check that the given thing is a list *and* check some properties of its length (instead of using `is_list(something) and length(something) > 0`).
 
-## Expressions in guard clauses
-
-Not all expressions are allowed in guard clauses, but only a handful of them. This is a deliberate choice: only a predefined set of side-effect-free functions are allowed. This way, Elixir (and Erlang) can make sure that nothing bad happens while executing guards and no mutations happen anywhere. This behaviour is also coherent with pattern match, which is a naturally a side-effect-free operation. Finally, keeping expressions allowed in clauses to a close set of predefined ones allows the compiler to optimize the code related to choosing the right clause.
-
 ## Defining custom guard expressions
 
 As mentioned before, only the expressions listed in this page are allowed in guards. However, we can take advantage of macros to write custom guards that can simplify our programs or make them more domain-specific. At the end of the day, what matters is that the *output* of the macros (which is what will be compiled) boils down to a combinations of the allowed expressions.
@@ -165,6 +167,14 @@ def my_function(number) when is_even(number) do
 end
 ```
 
+While it's possible to create custom guards with macros, it's recommended to define them using `defguard` and `defguardp` which perform additional compile-time checks. Here's an example:
+
+```elixir
+defmodule MyInteger do
+  defguard is_even(value) when is_integer(value) and rem(value, 2) == 0
+end
+```
+
 ## Multiple guards in the same clause
 
 There exists an additional way to simplify a chain of `or`s in guards: Elixir supports writing "multiple guards" in the same clause. This:

lib/elixir/pages/Operators.md

@@ -26,7 +26,7 @@ Operator
 `\|`                                                                                     | Right to left
 `::`                                                                                     | Right to left
 `when`                                                                                   | Right to left
-`<-`, `\\`                                                                               | Left to right
+`<-` `\\`                                                                                | Left to right
 
 ## Comparison operators
 

lib/elixir/src/elixir.app.src

@@ -1,6 +1,6 @@
 {application, elixir,
 [{description, "elixir"},
- {vsn, "1.5.2"},
+ {vsn, "1.6.6"},
  {modules, [
 	elixir
   ]},

lib/elixir/src/elixir_compiler.erl

@@ -76,14 +76,19 @@ compile(Forms, Vars, #{line := Line, file := File} = E) ->
 
   {Module, Binary} = elixir_erl_compiler:noenv_forms(Form, File, [nowarn_nomatch]),
   code:load_binary(Module, "", Binary),
-  dispatch(Module, Fun, Args, I, EE).
 
-dispatch(Module, Fun, Args, I, E) ->
+  Purgeable = beam_lib:chunks(Binary, [labeled_locals]) ==
+              {ok, {Module, [{labeled_locals, []}]}},
+  dispatch(Module, Fun, Args, Purgeable, I, EE).
+
+dispatch(Module, Fun, Args, Purgeable, I, E) ->
   Res = Module:Fun(Args),
   code:delete(Module),
-  case code:soft_purge(Module) of
-    true -> return_compiler_module(I);
-    false -> ok
+  if Purgeable ->
+      code:purge(Module),
+      return_compiler_module(I);
+     true ->
+       ok
   end,
   {Res, E}.
 

lib/elixir/src/elixir_def.erl

@@ -190,12 +190,15 @@ run_with_location_change(File, E, Callback) ->
 def_to_clauses(_Kind, Meta, Args, [], nil, E) ->
   check_args_for_bodiless_clause(Meta, Args, E),
   [];
-def_to_clauses(Kind, Meta, _Args, _Guards, nil, E) ->
-  elixir_errors:form_error(Meta, ?key(E, file), elixir_expand, {missing_option, Kind, [do]});
 def_to_clauses(_Kind, Meta, Args, Guards, [{do, Body}], _E) ->
   [{Meta, Args, Guards, Body}];
-def_to_clauses(Kind, Meta, Args, Guards, Body, _E) ->
-  [{Meta, Args, Guards, {'try', [{origin,  Kind} | Meta], [Body]}}].
+def_to_clauses(Kind, Meta, Args, Guards, Body, E) ->
+  case is_list(Body) andalso lists:keyfind(do, 1, Body) of
+    {do, _} ->
+      [{Meta, Args, Guards, {'try', [{origin,  Kind} | Meta], [Body]}}];
+    false ->
+      elixir_errors:form_error(Meta, ?key(E, file), elixir_expand, {missing_option, Kind, [do]})
+  end.
 
 run_on_definition_callbacks(Kind, Module, Name, Args, Guards, Body, E) ->
   Callbacks = ets:lookup_element(elixir_module:data_table(Module), on_definition, 2),

lib/elixir/src/elixir_dispatch.erl

@@ -397,6 +397,10 @@ deprecation('Elixir.Macro', unescape_tokens, 1) ->
   "instead traverse over the arguments using Enum.map/2";
 deprecation('Elixir.Macro', unescape_tokens, 2) ->
   "instead traverse over the arguments using Enum.map/2";
+deprecation('Elixir.Module', add_doc, 5) ->
+  "use @doc instead";
+deprecation('Elixir.Module', add_doc, 6) ->
+  "use @doc instead";
 deprecation('Elixir.Range', 'range?', 1) ->
   "instead pattern match on left..right";
 deprecation('Elixir.Stream', filter_map, 3) ->

lib/elixir/src/elixir_erl.erl

@@ -293,29 +293,43 @@ add_info_function(Line, Module, Def, Defmacro, Deprecated) ->
   AllowedArgs = lists:map(fun(Atom) -> {atom, Line, Atom} end, AllowedAttrs),
 
   Spec =
-    {attribute, Line, spec, {{'__info__', 1},
-      [{type, Line, 'fun', [
-        {type, Line, product, [
-          {type, Line, union, AllowedArgs}
-        ]},
-        {type, Line, union, [
-          {type, Line, atom, []},
-          {type, Line, list, [
+    %% TODO: Remove this check once we depend only on 20
+    case erlang:system_info(otp_release) of
+      "19" ->
+        {attribute, Line, spec, {{'__info__', 1},
+          [{type, Line, 'fun', [
+            {type, Line, product, [
+              {type, Line, union, AllowedArgs}
+            ]},
             {type, Line, union, [
-              {type, Line, tuple, [
-                {type, Line, atom, []},
-                {type, Line, any, []}
-              ]},
-              {type, Line, tuple, [
-                {type, Line, atom, []},
-                {type, Line, byte, []},
-                {type, Line, integer, []}
+              {type, Line, atom, []},
+              {type, Line, list, [
+                {type, Line, union, [
+                  {type, Line, tuple, [
+                    {type, Line, atom, []},
+                    {type, Line, any, []}
+                  ]},
+                  {type, Line, tuple, [
+                    {type, Line, atom, []},
+                    {type, Line, byte, []},
+                    {type, Line, integer, []}
+                  ]}
+                ]}
               ]}
             ]}
-          ]}
-        ]}
-      ]}]
-    }},
+          ]}]
+        }};
+
+      _ ->
+        {attribute, Line, spec, {{'__info__', 1},
+          [{type, Line, 'fun', [
+            {type, Line, product, [
+              {type, Line, union, AllowedArgs}
+            ]},
+            {type, Line, any, []}
+          ]}]
+        }}
+    end,
 
   Info =
     {function, 0, '__info__', 1, [

lib/elixir/src/elixir_erl_compiler.erl

@@ -63,6 +63,7 @@ handle_file_warning(_, _File, {_Line, erl_lint, {undefined_behaviour_func, _, _}
 handle_file_warning(_, _File, {_Line, erl_lint, {undefined_behaviour, _}}) -> ok;
 handle_file_warning(_, _File, {_Line, erl_lint, {ill_defined_behaviour_callbacks, _}}) -> ok;
 handle_file_warning(_, _File, {_Line, erl_lint, {ill_defined_optional_callbacks, _}}) -> ok;
+handle_file_warning(_, _File, {_Line, erl_lint, {deprecated,{erlang,get_stacktrace,0},_}}) -> ok;
 
 handle_file_warning(_, File, {Line, Module, Desc}) ->
   Message = format_error(Module, Desc),
@@ -86,6 +87,10 @@ format_error(erl_lint, {undefined_function, {F, A}}) ->
 format_error(erl_lint, {spec_fun_undefined, {M, F, A}}) ->
   io_lib:format("spec for undefined function ~ts.~ts/~B", [elixir_aliases:inspect(M), F, A]);
 
+%% Mention the capture operator in make_fun
+format_error(sys_core_fold, {no_effect, {erlang, make_fun, 3}}) ->
+  "the result of the capture operator & (:erlang.make_fun/3) is never used";
+
 %% Make no_effect clauses pretty
 format_error(sys_core_fold, {no_effect, {erlang, F, A}}) ->
   {Fmt, Args} = case erl_internal:comp_op(F, A) of

lib/elixir/src/elixir_erl_for.erl

@@ -3,20 +3,14 @@
 -include("elixir.hrl").
 
 translate(Meta, Args, Return, S) ->
-  {AccName, _, SA} = elixir_erl_var:build('_', S),
-  {VarName, _, SV} = elixir_erl_var:build('_', SA),
-
   Ann = ?ann(Meta),
-  Acc = {var, Ann, AccName},
-  Var = {var, Ann, VarName},
-
   {Cases, [{do, Expr} | Opts]} = elixir_utils:split_last(Args),
 
   {TInto, SI} =
     case lists:keyfind(into, 1, Opts) of
-      {into, Into} -> elixir_erl_pass:translate(Into, SV);
-      false when Return -> {{nil, Ann}, SV};
-      false -> {false, SV}
+      {into, Into} -> elixir_erl_pass:translate(Into, S);
+      false when Return -> {{nil, Ann}, S};
+      false -> {false, S}
     end,
 
   TUniq = lists:keyfind(uniq, 1, Opts) == {uniq, true},
@@ -27,9 +21,9 @@ translate(Meta, Args, Return, S) ->
 
   case comprehension_expr(TInto, TExpr) of
     {inline, TIntoExpr} ->
-      {build_inline(Ann, TCases, TIntoExpr, TInto, TUniq, Var, Acc, SF), SF};
+      build_inline(Ann, TCases, TIntoExpr, TInto, TUniq, SF);
     {into, TIntoExpr} ->
-      build_into(Ann, TCases, TIntoExpr, TInto, TUniq, Var, Acc, SF)
+      build_into(Ann, TCases, TIntoExpr, TInto, TUniq, SF)
   end.
 
 %% In case we have no return, we wrap the expression
@@ -95,30 +89,72 @@ collect_filters([], Acc) ->
   {Acc, []}.
 
 
-build_inline(Ann, Clauses, Expr, Into, Uniq, _Var, Acc, S) ->
+build_inline(Ann, Clauses, Expr, Into, Uniq, S) ->
   case not Uniq and lists:all(fun(Clause) -> element(1, Clause) == bin end, Clauses) of
-    true  -> build_comprehension(Ann, Clauses, Expr, Into);
-    false -> build_inline(Ann, Clauses, Expr, Into, Uniq, Acc, S)
+    true  -> {build_comprehension(Ann, Clauses, Expr, Into), S};
+    false -> build_inline_each(Ann, Clauses, Expr, Into, Uniq, S)
   end.
 
-build_inline(Ann, Clauses, Expr, false, Uniq, Acc, S) ->
+build_inline_each(Ann, Clauses, Expr, false, Uniq, S) ->
   InnerFun = fun(InnerExpr, _InnerAcc) -> InnerExpr end,
-  build_reduce(Ann, Clauses, InnerFun, Expr, {nil, Ann}, Uniq, Acc, S);
-build_inline(Ann, Clauses, Expr, {nil, _} = Into, Uniq, Acc, S) ->
+  {build_reduce(Ann, Clauses, InnerFun, Expr, {nil, Ann}, Uniq, S), S};
+build_inline_each(Ann, Clauses, Expr, {nil, _} = Into, Uniq, S) ->
   InnerFun = fun(InnerExpr, InnerAcc) -> {cons, Ann, InnerExpr, InnerAcc} end,
-  ReduceExpr = build_reduce(Ann, Clauses, InnerFun, Expr, Into, Uniq, Acc, S),
-  elixir_erl:remote(Ann, lists, reverse, [ReduceExpr]);
-build_inline(Ann, Clauses, Expr, {bin, _, _} = Into, Uniq, Acc, S) ->
-  InnerFun = fun(InnerExpr, InnerAcc) -> {cons, Ann, InnerAcc, InnerExpr} end,
-  ReduceExpr = build_reduce(Ann, Clauses, InnerFun, Expr, Into, Uniq, Acc, S),
-  elixir_erl:remote(Ann, erlang, iolist_to_binary, [ReduceExpr]).
+  ReduceExpr = build_reduce(Ann, Clauses, InnerFun, Expr, Into, Uniq, S),
+  {elixir_erl:remote(Ann, lists, reverse, [ReduceExpr]), S};
+build_inline_each(Ann, Clauses, Expr, {bin, _, []}, Uniq, S) ->
+  {InnerValue, SV} = build_var(Ann, S),
+  Generated = erl_anno:set_generated(true, Ann),
 
-build_into(Ann, Clauses, Expr, {map, _, []}, Uniq, _Var, Acc, S) ->
-  ReduceExpr = build_inline(Ann, Clauses, Expr, {nil, Ann}, Uniq, Acc, S),
-  {elixir_erl:remote(Ann, maps, from_list, [ReduceExpr]), S};
+  InnerFun = fun(InnerExpr, InnerAcc) ->
+    {'case', Ann, InnerExpr, [
+      {clause, Generated,
+       [InnerValue],
+       [[elixir_erl:remote(Ann, erlang, is_binary, [InnerValue]),
+         elixir_erl:remote(Ann, erlang, is_list, [InnerAcc])]],
+       [{cons, Generated, InnerAcc, InnerValue}]},
+      {clause, Generated,
+       [InnerValue],
+       [[elixir_erl:remote(Ann, erlang, is_bitstring, [InnerValue]),
+         elixir_erl:remote(Ann, erlang, is_bitstring, [InnerAcc])]],
+       [{bin, Ann, [
+          {bin_element, Ann, InnerAcc, default, [bitstring]},
+          {bin_element, Ann, InnerValue, default, [bitstring]}
+       ]}]},
+      {clause, Generated,
+       [InnerValue],
+       [[elixir_erl:remote(Ann, erlang, is_bitstring, [InnerValue])]],
+       [{bin, Ann, [
+          {bin_element, Ann, elixir_erl:remote(Ann, erlang, iolist_to_binary, [InnerAcc]), default, [bitstring]},
+          {bin_element, Ann, InnerValue, default, [bitstring]}
+       ]}]},
+      {clause, Generated,
+       [InnerValue],
+       [],
+       [elixir_erl:remote(Ann, erlang, error, [{tuple, Ann, [{atom, Ann, badarg}, InnerValue]}])]}
+    ]}
+  end,
 
-build_into(Ann, Clauses, Expr, Into, Uniq, Fun, Acc, S) ->
-  {Kind, SK}   = build_var(Ann, S),
+  ReduceExpr = build_reduce(Ann, Clauses, InnerFun, Expr, {nil, Ann}, Uniq, SV),
+
+  {{'case', Ann, ReduceExpr, [
+    {clause, Ann,
+     [InnerValue],
+     [[elixir_erl:remote(Ann, erlang, is_bitstring, [InnerValue])]],
+     [InnerValue]},
+    {clause, Ann,
+     [InnerValue],
+     [],
+     [elixir_erl:remote(Ann, erlang, iolist_to_binary, [InnerValue])]}
+  ]}, SV}.
+
+build_into(Ann, Clauses, Expr, {map, _, []}, Uniq, S) ->
+  {ReduceExpr, SR} = build_inline_each(Ann, Clauses, Expr, {nil, Ann}, Uniq, S),
+  {elixir_erl:remote(Ann, maps, from_list, [ReduceExpr]), SR};
+build_into(Ann, Clauses, Expr, Into, Uniq, S) ->
+  {Fun, SF}    = build_var(Ann, S),
+  {Acc, SA}    = build_var(Ann, SF),
+  {Kind, SK}   = build_var(Ann, SA),
   {Reason, SR} = build_var(Ann, SK),
   {Stack, ST}  = build_var(Ann, SR),
   {Done, SD}   = build_var(Ann, ST),
@@ -132,7 +168,7 @@ build_into(Ann, Clauses, Expr, Into, Uniq, Fun, Acc, S) ->
     elixir_erl:remote(Ann, 'Elixir.Collectable', into, [Into])
   },
 
-  IntoReduceExpr = build_reduce(Ann, Clauses, InnerFun, Expr, Acc, Uniq, Acc, SD),
+  IntoReduceExpr = build_reduce(Ann, Clauses, InnerFun, Expr, Acc, Uniq, SD),
 
   TryExpr =
     {'try', Ann,
@@ -153,12 +189,14 @@ build_into(Ann, Clauses, Expr, Into, Uniq, Fun, Acc, S) ->
 
 %% Helpers
 
-build_reduce(_Ann, Clauses, InnerFun, Expr, Into, false, Acc, S) ->
-  build_reduce_each(Clauses, InnerFun(Expr, Acc), Into, Acc, S);
-build_reduce(Ann, Clauses, InnerFun, Expr, Into, true, Acc, S) ->
+build_reduce(Ann, Clauses, InnerFun, Expr, Into, false, S) ->
+  {Acc, SA} = build_var(Ann, S),
+  build_reduce_each(Clauses, InnerFun(Expr, Acc), Into, Acc, SA);
+build_reduce(Ann, Clauses, InnerFun, Expr, Into, true, S) ->
   %% Those variables are used only inside the anonymous function
   %% so we don't need to worry about returning the scope.
-  {Value, SV} = build_var(Ann, S),
+  {Acc, SA} = build_var(Ann, S),
+  {Value, SV} = build_var(Ann, SA),
   {IntoAcc, SI} = build_var(Ann, SV),
   {UniqAcc, SU} = build_var(Ann, SI),
 
@@ -302,12 +340,12 @@ join_filter(Ann, {nil, Filter}, True, False) ->
     {clause, Ann, [{atom, Ann, false}], [], [False]}
   ]};
 join_filter(Ann, {Var, Filter}, True, False) ->
-  Guard =
-    {op, Ann, 'orelse',
-      {op, Ann, '==', Var, {atom, Ann, false}},
-      {op, Ann, '==', Var, {atom, Ann, nil}}},
+  Guards = [
+    [{op, Ann, '==', Var, {atom, Ann, false}}],
+    [{op, Ann, '==', Var, {atom, Ann, nil}}]
+  ],
 
   {'case', Ann, Filter, [
-    {clause, Ann, [Var], [[Guard]], [False]},
+    {clause, Ann, [Var], Guards, [False]},
     {clause, Ann, [{var, Ann, '_'}], [], [True]}
   ]}.

lib/elixir/src/elixir_erl_pass.erl

@@ -25,7 +25,7 @@ translate({'=', Meta, [Left, Right]}, S) ->
       Reason = {tuple, Generated, [{atom, Generated, badmatch}, ResultVar]},
       RaiseExpr = elixir_erl:remote(Generated, erlang, error, [Reason]),
       GuardsExp = {'if', Generated, [
-        {clause, Generated, [], [ExtraGuards], [True]},
+        {clause, Generated, [], [ExtraGuards], [ResultVar]},
         {clause, Generated, [], [[True]], [RaiseExpr]}
       ]},
       {{block, Generated, [ResultMatch, GuardsExp]}, SL2};

lib/elixir/src/elixir_expand.erl

@@ -560,31 +560,28 @@ var_context(Meta, Kind) ->
 
 expand_case(true, Meta, Expr, Opts, E) ->
   {EExpr, EE} = expand(Expr, E),
-  {EOpts, EO} = elixir_clauses:'case'(Meta, Opts, EE),
 
   ROpts =
     case proplists:get_value(optimize_boolean, Meta, false) of
       true ->
         case elixir_utils:returns_boolean(EExpr) of
-          true -> rewrite_case_clauses(EOpts);
-          false -> generated_case_clauses(EOpts)
+          true -> rewrite_case_clauses(Opts);
+          false -> generated_case_clauses(Opts)
         end;
 
       false ->
-        EOpts
+        Opts
     end,
 
-  {{'case', Meta, [EExpr, ROpts]}, EO};
+  {EOpts, EO} = elixir_clauses:'case'(Meta, ROpts, EE),
+  {{'case', Meta, [EExpr, EOpts]}, EO};
 expand_case(false, Meta, Expr, Opts, E) ->
   {Case, _} = expand_case(true, Meta, Expr, Opts, E),
   {Case, E}.
 
 rewrite_case_clauses([{do, [
   {'->', FalseMeta, [
-    [{'when', _, [Var, {{'.', _, [erlang, 'orelse']}, _, [
-      {{'.', _, [erlang, '=:=']}, _, [Var, nil]},
-      {{'.', _, [erlang, '=:=']}, _, [Var, false]}
-    ]}]}],
+    [{'when', _, [Var, {{'.', _, ['Elixir.Kernel', 'in']}, _, [Var, [false, nil]]}]}],
     FalseExpr
   ]},
   {'->', TrueMeta, [

lib/elixir/src/elixir_parser.yrl

@@ -59,8 +59,8 @@ Right     50 when_op_eol.     %% when
 Right     60 type_op_eol.     %% ::
 Right     70 pipe_op_eol.     %% |
 Right     80 assoc_op_eol.    %% =>
-Right     90 match_op_eol.    %% =
-Nonassoc 100 capture_op_eol.  %% &
+Nonassoc  90 capture_op_eol.  %% &
+Right    100 match_op_eol.    %% =
 Left     130 or_op_eol.       %% ||, |||, or
 Left     140 and_op_eol.      %% &&, &&&, and
 Left     150 comp_op_eol.     %% ==, !=, =~, ===, !==
@@ -305,7 +305,7 @@ eoe -> ';' : '$1'.
 eoe -> eol ';' : '$1'.
 
 fn_eoe -> 'fn' : '$1'.
-fn_eoe -> 'fn' eoe : '$1'.
+fn_eoe -> 'fn' eoe : next_is_eol('$1').
 
 do_eoe -> 'do' : '$1'.
 do_eoe -> 'do' eoe : '$1'.
@@ -659,7 +659,7 @@ meta_from_token_with_end_line(Begin, End) ->
   case ?formatter_metadata() of
     true ->
       [{end_line, line_from_location(?location(End))}
-       | meta_from_token(Begin)];
+       | eol_op(?location(Begin)) ++ meta_from_token(Begin)];
     false ->
       meta_from_token(Begin)
   end.
@@ -688,7 +688,13 @@ build_op({_Kind, Location, 'in'}, {UOp, _, [Left]}, Right) when ?rearrange_uop(U
   {UOp, meta_from_location(Location), [{'in', meta_from_location(Location), [Left, Right]}]};
 
 build_op({_Kind, Location, 'not in'}, Left, Right) ->
-  {'not', meta_from_location(Location), [{'in', meta_from_location(Location), [Left, Right]}]};
+  InMeta = meta_from_location(Location),
+  NotMeta =
+    case ?formatter_metadata() of
+      true -> [{operator, 'not in'} | InMeta];
+      false -> InMeta
+    end,
+  {'not', NotMeta, [{'in', InMeta, [Left, Right]}]};
 build_op({_Kind, Location, Op}, Left, Right) ->
   {Op, eol_op(Location) ++ meta_from_location(Location), [Left, Right]}.
 

lib/elixir/src/elixir_tokenizer.erl

@@ -182,7 +182,8 @@ tokenize([$~, S, H, H, H | T] = Original, Line, Column, Scope, Tokens) when ?is_
     {ok, NewLine, NewColumn, Parts, Rest} ->
       {Final, Modifiers} = collect_modifiers(Rest, []),
       Token = {sigil, {Line, Column, nil}, S, Parts, Modifiers, <<H, H, H>>},
-      tokenize(Final, NewLine, NewColumn, Scope, [Token | Tokens]);
+      NewColumn2 = NewColumn + length(Modifiers),
+      tokenize(Final, NewLine, NewColumn2, Scope, [Token | Tokens]);
     {error, Reason} ->
       {error, Reason, Original, Tokens}
   end;
@@ -192,7 +193,8 @@ tokenize([$~, S, H | T] = Original, Line, Column, Scope, Tokens) when ?is_sigil(
     {NewLine, NewColumn, Parts, Rest} ->
       {Final, Modifiers} = collect_modifiers(Rest, []),
       Token = {sigil, {Line, Column, nil}, S, Parts, Modifiers, <<H>>},
-      tokenize(Final, NewLine, NewColumn, Scope, [Token | Tokens]);
+      NewColumn2 = NewColumn + length(Modifiers),
+      tokenize(Final, NewLine, NewColumn2, Scope, [Token | Tokens]);
     {error, Reason} ->
       Sigil = [$~, S, H],
       interpolation_error(Reason, Original, Tokens, " (for sigil ~ts starting at line ~B)", [Sigil, Line])
@@ -473,6 +475,10 @@ tokenize("\r\n" ++ Rest, Line, Column, Scope, Tokens) ->
 
 % Others
 
+tokenize([$%, $[ | Rest], Line, _Column, _Scope, Tokens) ->
+  Reason = {Line, "expected %{ to define a map, got: ", [$%, $[]},
+  {error, Reason, Rest, Tokens};
+
 tokenize([$%, ${ | T], Line, Column, Scope, Tokens) ->
   tokenize([${ | T], Line, Column + 1, Scope, [{'%{}', {Line, Column, nil}} | Tokens]);
 
@@ -711,7 +717,7 @@ collect_modifiers(Rest, Buffer) ->
 %% Heredocs
 
 extract_heredoc_with_interpolation(Line, Column, Scope, Interpol, T, H) ->
-  case extract_heredoc(Line, Column, T, H) of
+  case extract_heredoc(Line, Column, T, H, Scope) of
     {ok, NewLine, NewColumn, Body, Rest} ->
       case elixir_interpolation:extract(Line + 1, 1, Scope, Interpol, Body, 0) of
         {error, Reason} ->
@@ -723,7 +729,7 @@ extract_heredoc_with_interpolation(Line, Column, Scope, Interpol, T, H) ->
       Error
   end.
 
-extract_heredoc(Line0, Column0, Rest0, Marker) ->
+extract_heredoc(Line0, Column0, Rest0, Marker, Scope) ->
   case extract_heredoc_header(Rest0) of
     {ok, Rest1} ->
       %% We prepend a new line so we can transparently remove
@@ -731,7 +737,7 @@ extract_heredoc(Line0, Column0, Rest0, Marker) ->
       %% in the final heredoc body three lines below.
       case extract_heredoc_body(Line0, Column0, Marker, [$\n | Rest1], []) of
         {ok, Line1, Body, Rest2, Spaces} ->
-          {ok, Line1, 1, tl(remove_heredoc_spaces(Body, Spaces)), Rest2};
+          {ok, Line1, 4 + Spaces, tl(remove_heredoc_spaces(Body, Spaces, Marker, Scope)), Rest2};
         {error, Reason, ErrorLine} ->
           Terminator = [Marker, Marker, Marker],
           {Message, Token} = heredoc_error_message(Reason, Line0, Terminator),
@@ -746,23 +752,49 @@ heredoc_error_message(eof, Line, Terminator) ->
   {io_lib:format("missing terminator: ~ts (for heredoc starting at line ~B)",
                  [Terminator, Line]),
    []};
-heredoc_error_message(misplacedterminator, _Line, Terminator) ->
+heredoc_error_message(badterminator, _Line, Terminator) ->
   {"invalid location for heredoc terminator, please escape token or move it to its own line: ",
    Terminator}.
+
 %% Remove spaces from heredoc based on the position of the final quotes.
 
-remove_heredoc_spaces(Body, 0) ->
-  lists:reverse([0 | Body]);
-remove_heredoc_spaces(Body, Spaces) ->
-  remove_heredoc_spaces([0 | Body], [], Spaces, Spaces).
-remove_heredoc_spaces([H, $\n | T], [Backtrack | Buffer], Spaces, Original) when Spaces > 0, ?is_horizontal_space(H) ->
-  remove_heredoc_spaces([Backtrack, $\n | T], Buffer, Spaces - 1, Original);
-remove_heredoc_spaces([$\n=H | T], Buffer, _Spaces, Original) ->
-  remove_heredoc_spaces(T, [H | Buffer], Original, Original);
-remove_heredoc_spaces([H | T], Buffer, Spaces, Original) ->
-  remove_heredoc_spaces(T, [H | Buffer], Spaces, Original);
-remove_heredoc_spaces([], Buffer, _Spaces, _Original) ->
-  Buffer.
+remove_heredoc_spaces(Body, Spaces, Marker, Scope) ->
+  case trim_spaces(Body, [0], Spaces, false) of
+    {Acc, false} ->
+      Acc;
+    {Acc, Line} ->
+      Msg = io_lib:format("outdented heredoc line. The contents inside the heredoc should be indented "
+                          "at the same level as the closing ~ts. The following is forbidden:~n~n"
+                          "    def text do~n"
+                          "      \"\"\"~n"
+                          "    contents~n"
+                          "      \"\"\"~n"
+                          "    end~n~n"
+                          "Instead make sure the contents are indented as much as the heredoc closing:~n~n"
+                          "    def text do~n"
+                          "      \"\"\"~n"
+                          "      contents~n"
+                          "      \"\"\"~n"
+                          "    end~n~n"
+                          "The current heredoc line is indented too little", [[Marker, Marker, Marker]]),
+      elixir_errors:warn(Line, Scope#elixir_tokenizer.file, Msg),
+      Acc
+  end.
+
+trim_spaces([{Line, Entry} | Rest], Acc, Spaces, Warned) ->
+  case trim_space(lists:reverse(Entry), Spaces) of
+    {Trimmed, true} when Warned == false ->
+      trim_spaces(Rest, Trimmed ++ Acc, Spaces, Line);
+    {Trimmed, _} ->
+      trim_spaces(Rest, Trimmed ++ Acc, Spaces, Warned)
+  end;
+trim_spaces([], Acc, _Spaces, Warned) ->
+  {Acc, Warned}.
+
+trim_space(Rest, 0) -> {Rest, false};
+trim_space([$\n], _) -> {[$\n], false};
+trim_space([H | T], Spaces) when ?is_horizontal_space(H) -> trim_space(T, Spaces - 1);
+trim_space(Rest, _Spaces) -> {Rest, true}.
 
 %% Extract the heredoc header.
 
@@ -780,11 +812,11 @@ extract_heredoc_header(_) ->
 %% is aligned.
 
 extract_heredoc_body(Line, _Column, Marker, Rest, Buffer) ->
-  case extract_heredoc_line(Marker, Rest, Buffer, 0) of
-    {ok, NewBuffer, NewRest} ->
-      extract_heredoc_body(Line + 1, 1, Marker, NewRest, NewBuffer);
-    {ok, NewBuffer, NewRest, Spaces} ->
-      {ok, Line, NewBuffer, NewRest, Spaces};
+  case extract_heredoc_line(Marker, Rest, [], 0) of
+    {ok, Entry, NewRest} ->
+      extract_heredoc_body(Line + 1, 1, Marker, NewRest, [{Line, Entry} | Buffer]);
+    {done, Entry, NewRest, Spaces} ->
+      {ok, Line, [{Line, Entry} | Buffer], NewRest, Spaces};
     {error, Reason} ->
       {error, Reason, Line}
   end.
@@ -797,7 +829,7 @@ extract_heredoc_line(Marker, [$\\, $\\ | T], Buffer) ->
 extract_heredoc_line(Marker, [$\\, Marker | T], Buffer) ->
   extract_heredoc_line(Marker, T, [Marker, $\\ | Buffer]);
 extract_heredoc_line(Marker, [Marker, Marker, Marker | _], _) ->
-  {error, misplacedterminator};
+  {error, badterminator};
 extract_heredoc_line(_, "\r\n" ++ Rest, Buffer) ->
   {ok, [$\n | Buffer], Rest};
 extract_heredoc_line(_, "\n" ++ Rest, Buffer) ->
@@ -812,7 +844,7 @@ extract_heredoc_line(_, _, _) ->
 extract_heredoc_line(Marker, [H | T], Buffer, Counter) when ?is_horizontal_space(H) ->
   extract_heredoc_line(Marker, T, [H | Buffer], Counter + 1);
 extract_heredoc_line(Marker, [Marker, Marker, Marker | T], Buffer, Counter) ->
-  {ok, Buffer, T, Counter};
+  {done, Buffer, T, Counter};
 extract_heredoc_line(Marker, Rest, Buffer, _Counter) ->
   extract_heredoc_line(Marker, Rest, Buffer).
 

lib/elixir/test/elixir/base_test.exs

@@ -473,7 +473,7 @@ defmodule BaseTest do
     assert :error == decode32("66FF", case: :lower)
   end
 
-  test "decode32!/1,2 error on non-alphabet digit" do
+  test "decode32!/1,2 argument error on non-alphabet digit" do
     assert_raise ArgumentError, "non-alphabet digit found: \")\" (byte 41)", fn ->
       decode32!("MZX)6YTB")
     end
@@ -485,6 +485,10 @@ defmodule BaseTest do
     assert_raise ArgumentError, "non-alphabet digit found: \"M\" (byte 77)", fn ->
       decode32!("MZXW6YTBOI======", case: :lower)
     end
+
+    assert_raise ArgumentError, "non-alphabet digit found: \"0\" (byte 48)", fn ->
+      decode32!("0ZXW6YTB0I======", case: :mixed)
+    end
   end
 
   test "decode32/1 errors on incorrect padding" do

lib/elixir/test/elixir/code_formatter/calls_test.exs

@@ -100,21 +100,6 @@ defmodule Code.Formatter.CallsTest do
                   @short_length
     end
 
-    test "for binaries" do
-      bad = "foo(<<1, 2, 3, 4>>)"
-
-      good = """
-      foo(<<
-        1,
-        2,
-        3,
-        4
-      >>)
-      """
-
-      assert_format bad, good, @short_length
-    end
-
     test "for lists" do
       bad = "foo([1, 2, 3, 4])"
 
@@ -146,6 +131,40 @@ defmodule Code.Formatter.CallsTest do
 
       assert_format bad, good, @medium_length
     end
+
+    test "for binaries only on eol" do
+      bad = "foo(<<1, 2, 3, 4>>)"
+
+      good = """
+      foo(
+        <<1, 2,
+          3, 4>>
+      )
+      """
+
+      assert_format bad, good, @short_length
+
+      bad = """
+      foo(<<
+        # foo
+        1,
+        2,
+        3,
+        4>>)
+      """
+
+      good = """
+      foo(<<
+        # foo
+        1,
+        2,
+        3,
+        4
+      >>)
+      """
+
+      assert_format bad, good, @short_length
+    end
   end
 
   describe "local calls" do
@@ -182,9 +201,23 @@ defmodule Code.Formatter.CallsTest do
       assert_format bad, good, @short_length
     end
 
+    test "with arguments on comma limit" do
+      bad = """
+      import(foo(abc, cde), :next)
+      """
+
+      good = """
+      import(
+        foo(abc, cde),
+        :next
+      )
+      """
+
+      assert_format bad, good, @medium_length
+    end
+
     test "with keyword lists" do
       assert_same "foo(foo: 1, bar: 2)"
-
       assert_same "foo(:hello, foo: 1, bar: 2)"
 
       bad = """
@@ -201,8 +234,14 @@ defmodule Code.Formatter.CallsTest do
 
       assert_format bad, good, @short_length
 
-      # Check it preserves multiline.
-      assert_same good
+      bad = """
+      foo(:hello, foo: 1,
+        bar: 2)
+      """
+
+      assert_format bad, """
+      foo(:hello, foo: 1, bar: 2)
+      """
     end
 
     test "with lists maybe rewritten as keyword lists" do
@@ -216,21 +255,19 @@ defmodule Code.Formatter.CallsTest do
       assert_same "bar = if foo, do: bar, else: baz"
 
       assert_same """
-                  for :one,
-                      :two,
-                      :three,
-                      fn ->
-                        :ok
-                      end
-                  """,
-                  @short_length
+      for :one,
+          :two,
+          :three,
+          fn ->
+            :ok
+          end
+      """
 
       assert_same """
-                  for :one, fn ->
-                    :ok
-                  end
-                  """,
-                  @medium_length
+      for :one, fn ->
+        :ok
+      end
+      """
     end
 
     test "without parens on line limit" do
@@ -244,17 +281,78 @@ defmodule Code.Formatter.CallsTest do
       assert_format bad, good, @short_length
     end
 
+    test "without parens on comma limit" do
+      bad = """
+      import foo(abc, cde), :next
+      """
+
+      good = """
+      import foo(
+               abc,
+               cde
+             ),
+             :next
+      """
+
+      assert_format bad, good, @medium_length
+    end
+
     test "without parens and with keyword lists preserves multiline" do
       assert_same """
       defstruct foo: 1,
                 bar: 2
       """
 
+      assert_same """
+      config :app,
+        foo: 1
+      """
+
       assert_same """
       config :app,
         foo: 1,
         bar: 2
       """
+
+      assert_same """
+      config :app, :key,
+        foo: 1,
+        bar: 2
+      """
+
+      assert_same """
+      config :app,
+             :key,
+             foo: 1,
+             bar: 2
+      """
+
+      bad = """
+      config :app, foo: 1,
+        bar: 2
+      """
+
+      assert_format bad, """
+      config :app,
+        foo: 1,
+        bar: 2
+      """
+    end
+
+    test "without parens and with keyword lists on comma limit" do
+      bad = """
+      import foo(abc, cde), opts: :next
+      """
+
+      good = """
+      import foo(
+               abc,
+               cde
+             ),
+             opts: :next
+      """
+
+      assert_format bad, good, @medium_length
     end
 
     test "without parens and with keyword lists on line limit" do
@@ -415,9 +513,32 @@ defmodule Code.Formatter.CallsTest do
       )
       """
 
+      assert_same """
+      call(
+        :hello,
+        :foo,
+        :bar
+      ) do
+        1 + 2
+      end
+      """
+
       # Doesn't preserve this because only the beginning has a newline
       assert_format "call(\nfoo, bar, baz)", "call(foo, bar, baz)"
 
+      # Doesn't preserve because there are no args
+      bad = """
+      call() do
+        1 + 2
+      end
+      """
+
+      assert_format bad, """
+      call do
+        1 + 2
+      end
+      """
+
       # Doesn't preserve because we have a single argument with next break fits
       bad = """
       call(
@@ -598,6 +719,14 @@ defmodule Code.Formatter.CallsTest do
 
       # Doesn't preserve this because only the beginning has a newline
       assert_format "Remote.call(\nfoo, bar, baz)", "Remote.call(foo, bar, baz)"
+
+      assert_same """
+      Remote.call(
+        :hello,
+        :foo,
+        fn -> :bar end
+      )
+      """
     end
   end
 

lib/elixir/test/elixir/code_formatter/comments_test.exs

@@ -56,36 +56,19 @@ defmodule Code.Formatter.CommentsTest do
 
       assert_format bad, good
 
-      bad = """
-      foo    # this is foo
-      |> bar # this is bar
-      |> baz # this is baz
-      """
-
-      good = """
-      # this is foo
-      # this is bar
-      # this is baz
-      foo
-      |> bar
-      |> baz
-      """
-
-      assert_format bad, good, @short_length
-
       bad = """
       foo   # this is foo
-      | bar # this is bar
-      | baz # this is baz
+      ++ bar # this is bar
+      ++ baz # this is baz
       """
 
       good = """
       # this is foo
       # this is bar
       # this is baz
-      foo
-      | bar
-      | baz
+      foo ++
+        bar ++
+        baz
       """
 
       assert_format bad, good, @short_length
@@ -466,14 +449,15 @@ defmodule Code.Formatter.CommentsTest do
 
       assert_format bad, ~S"""
       # fn
-      # before head
-      # middle head
-      # after head
-      fn hello ->
-        # before body
-        # middle body
-        world
-        # after body
+      fn
+        # before head
+        # middle head
+        hello ->
+          # after head
+          # before body
+          # middle body
+          world
+          # after body
       end
       """
     end
@@ -721,6 +705,118 @@ defmodule Code.Formatter.CommentsTest do
     end
   end
 
+  describe "operators" do
+    test "with comment before, during and after uniform pipelines" do
+      assert_same """
+      foo
+      # |> bar
+      # |> baz
+      |> bat
+      """
+
+      bad = """
+      # before
+      foo    # this is foo
+      |> bar # this is bar
+      |> baz # this is baz
+      # after
+      """
+
+      good = """
+      # before
+      # this is foo
+      foo
+      # this is bar
+      |> bar
+      # this is baz
+      |> baz
+
+      # after
+      """
+
+      assert_format bad, good, @short_length
+    end
+
+    test "with comment before, during and after mixed pipelines" do
+      assert_same """
+      foo
+      # |> bar
+      # |> baz
+      ~> bat
+      """
+
+      bad = """
+      # before
+      foo    # this is foo
+      ~> bar # this is bar
+      <|> baz # this is baz
+      # after
+      """
+
+      good = """
+      # before
+      # this is foo
+      foo
+      # this is bar
+      ~> bar
+      # this is baz
+      <|> baz
+
+      # after
+      """
+
+      assert_format bad, good, @short_length
+    end
+
+    test "with comment before, during and after uniform right" do
+      assert_same """
+      foo
+      # | bar
+      # | baz
+      | bat
+      """
+
+      bad = """
+      # before
+      foo    # this is foo
+      | bar # this is bar
+      | baz # this is baz
+      # after
+      """
+
+      good = """
+      # before
+      # this is foo
+      foo
+      # this is bar
+      | bar
+      # this is baz
+      | baz
+
+      # after
+      """
+
+      assert_format bad, good, @short_length
+    end
+
+    test "with comment before, during and after mixed right" do
+      assert_same """
+      one
+      # when two
+      # when three
+      when four
+           # | five
+           | six
+      """
+    end
+
+    test "handles nodes without meta info" do
+      assert_same "(a -> b) |> (c -> d)"
+      assert_same "(a -> b) when c: d"
+      assert_same "(a -> b) when (c -> d)"
+    end
+  end
+
   describe "containers" do
     test "with comment outside before, during and after" do
       assert_same ~S"""

lib/elixir/test/elixir/code_formatter/containers_test.exs

@@ -18,16 +18,12 @@ defmodule Code.Formatter.ContainersTest do
       assert_format "{1,2,3}", "{1, 2, 3}"
     end
 
-    test "is strict on line limits" do
+    test "is flex on line limits" do
       bad = "{1, 2, 3, 4}"
 
       good = """
-      {
-        1,
-        2,
-        3,
-        4
-      }
+      {1, 2, 3,
+       4}
       """
 
       assert_format bad, good, @short_length
@@ -182,9 +178,30 @@ defmodule Code.Formatter.ContainersTest do
       assert_same keyword, @short_length
     end
 
+    test "with keyword lists on comma line limit" do
+      bad = """
+      [
+        foooo: 1,
+        barrr: 2
+      ]
+      """
+
+      good = """
+      [
+        foooo:
+          1,
+        barrr: 2
+      ]
+      """
+
+      assert_format bad, good, @short_length
+    end
+
     test "with quoted keyword lists" do
       assert_same ~S(["with spaces": 1])
       assert_same ~S(["one #{two} three": 1])
+      assert_same ~S(["\w": 1, "\\w": 2])
+      assert_same ~S(["Elixir.Foo": 1, "Elixir.Bar": 2])
       assert_format ~S(["Foo": 1, "Bar": 2]), ~S([Foo: 1, Bar: 2])
     end
 
@@ -230,13 +247,19 @@ defmodule Code.Formatter.ContainersTest do
       assert_format "<<1,2,3>>", "<<1, 2, 3>>"
     end
 
-    test "add parens on first and last in case of ambiguity" do
+    test "add parens on first and last in case of binary ambiguity" do
       assert_format "<< <<>> >>", "<<(<<>>)>>"
       assert_format "<< <<>> + <<>> >>", "<<(<<>> + <<>>)>>"
       assert_format "<< 1 + <<>> >>", "<<(1 + <<>>)>>"
       assert_format "<< <<>> + 1 >>", "<<(<<>> + 1)>>"
       assert_format "<< <<>>, <<>>, <<>> >>", "<<(<<>>), <<>>, (<<>>)>>"
-      assert_format "<< <<>>::1, <<>>::2, <<>>::3 >>", "<<(<<>>::1), <<>>::2, <<>>::3>>"
+      assert_format "<< <<>>::1, <<>>::2, <<>>::3 >>", "<<(<<>>)::1, <<>>::2, <<>>::3>>"
+      assert_format "<< <<>>::<<>> >>", "<<(<<>>)::(<<>>)>>"
+    end
+
+    test "add parens on first in case of operator ambiguity" do
+      assert_format "<< ~~~1::8 >>", "<<(~~~1)::8>>"
+      assert_format "<< ~s[foo]::binary >>", "<<(~s[foo])::binary>>"
     end
 
     test "with modifiers" do
@@ -261,16 +284,12 @@ defmodule Code.Formatter.ContainersTest do
       assert_same "<<(<<y>> <- x)>>"
     end
 
-    test "is strict on line limits" do
+    test "is flex on line limits" do
       bad = "<<1, 2, 3, 4>>"
 
       good = """
-      <<
-        1,
-        2,
-        3,
-        4
-      >>
+      <<1, 2, 3,
+        4>>
       """
 
       assert_format bad, good, @short_length
@@ -463,7 +482,7 @@ defmodule Code.Formatter.ContainersTest do
       }
       """
 
-      assert_format bad, good, @short_length
+      assert_format bad, good, line_length: 11
     end
 
     test "removes trailing comma" do
@@ -567,7 +586,7 @@ defmodule Code.Formatter.ContainersTest do
       }
       """
 
-      assert_format bad, good, @short_length
+      assert_format bad, good, line_length: 11
     end
 
     test "removes trailing comma" do

lib/elixir/test/elixir/code_formatter/general_test.exs

@@ -187,6 +187,22 @@ defmodule Code.Formatter.GeneralTest do
       assert_same code, @short_length
     end
 
+    test "with a single clause, followed by a newline, and can fit in one line" do
+      assert_same """
+      fn
+        hello -> world
+      end
+      """
+    end
+
+    test "with a single clause, followed by a newline, and can not fit in one line" do
+      assert_same """
+      SomeModule.long_function_name_that_approaches_max_columns(argument, acc, fn
+        %SomeStruct{key: key}, acc -> more_code(key, acc)
+      end)
+      """
+    end
+
     test "with multiple clauses" do
       code = """
       fn
@@ -329,6 +345,46 @@ defmodule Code.Formatter.GeneralTest do
       end
       """
     end
+
+    test "with -> on line limit" do
+      bad = """
+      fn ab, cd ->
+        ab + cd
+      end
+      """
+
+      good = """
+      fn ab,
+         cd ->
+        ab + cd
+      end
+      """
+
+      assert_format bad, good, @short_length
+
+      bad = """
+      fn
+        ab, cd ->
+          1
+        xy, zw ->
+          2
+      end
+      """
+
+      good = """
+      fn
+        ab,
+        cd ->
+          1
+
+        xy,
+        zw ->
+          2
+      end
+      """
+
+      assert_format bad, good, @short_length
+    end
   end
 
   describe "anonymous functions types" do

lib/elixir/test/elixir/code_formatter/integration_test.exs

@@ -345,20 +345,20 @@ defmodule Code.Formatter.IntegrationTest do
     """
   end
 
-  test "no parens keywords right on line limit" do
+  test "no parens keywords at the end of the line" do
     bad = """
     defmodule Mod do
-      defp token_list_downcase(<<char, rest::binary>>, acc) when is_whitespace(char) or is_comma(char), do: token_list_downcase(rest, acc)
-      defp token_list_downcase(some_really_long_arg11, some_really_long_arg22, some_really_long_arg33), do: token_list_downcase(rest, acc)
+      def token_list_downcase(<<char, rest::binary>>, acc) when is_whitespace(char) or is_comma(char), do: token_list_downcase(rest, acc)
+      def token_list_downcase(some_really_long_arg11, some_really_long_arg22, some_really_long_arg33), do: token_list_downcase(rest, acc)
     end
     """
 
     assert_format bad, """
     defmodule Mod do
-      defp token_list_downcase(<<char, rest::binary>>, acc) when is_whitespace(char) or is_comma(char),
+      def token_list_downcase(<<char, rest::binary>>, acc) when is_whitespace(char) or is_comma(char),
         do: token_list_downcase(rest, acc)
 
-      defp token_list_downcase(some_really_long_arg11, some_really_long_arg22, some_really_long_arg33),
+      def token_list_downcase(some_really_long_arg11, some_really_long_arg22, some_really_long_arg33),
         do: token_list_downcase(rest, acc)
     end
     """
@@ -382,6 +382,25 @@ defmodule Code.Formatter.IntegrationTest do
     assert_format bad, good, line_length: 18
   end
 
+  test "do at the end of the line with single argument" do
+    bad = """
+    defmodule Location do
+      def new(line, column) when is_integer(line) and line >= 0 and is_integer(column) and column >= 0 do
+        %{column: column, line: line}
+      end
+    end
+    """
+
+    assert_format bad, """
+    defmodule Location do
+      def new(line, column)
+          when is_integer(line) and line >= 0 and is_integer(column) and column >= 0 do
+        %{column: column, line: line}
+      end
+    end
+    """
+  end
+
   test "tuples as trees" do
     bad = """
     @document Parser.parse(
@@ -399,23 +418,26 @@ defmodule Code.Formatter.IntegrationTest do
 
     assert_format bad, """
     @document Parser.parse(
-                {"html", [], [
-                  {"head", [], []},
-                  {"body", [], [
-                    {"div", [], [
-                      {"p", [], ["1"]},
-                      {"p", [], ["2"]},
-                      {"div", [], [{"p", [], ["3"]}, {"p", [], ["4"]}]},
-                      {"p", [], ["5"]}
+                {"html", [],
+                 [
+                   {"head", [], []},
+                   {"body", [],
+                    [
+                      {"div", [],
+                       [
+                         {"p", [], ["1"]},
+                         {"p", [], ["2"]},
+                         {"div", [], [{"p", [], ["3"]}, {"p", [], ["4"]}]},
+                         {"p", [], ["5"]}
+                       ]}
                     ]}
-                  ]}
-                ]}
+                 ]}
               )
     """
   end
 
-  test "first argument in a call without parens" do
-    bad = """
+  test "first argument in a call without parens with comments" do
+    assert_same """
     with bar ::
            :ok
            | :invalid
@@ -423,28 +445,58 @@ defmodule Code.Formatter.IntegrationTest do
            | :other
     """
 
-    assert_format bad, """
-    # | :unknown
-    with bar ::
-           :ok
-           | :invalid
-           | :other
-    """
-
-    bad = """
+    assert_same """
     @spec bar ::
             :ok
             | :invalid
             # | :unknown
             | :other
     """
+  end
+
+  test "capture with operators" do
+    assert_same """
+    "this works" |> (&String.upcase/1) |> (&String.downcase/1)
+    """
+
+    assert_same """
+    "this works" || (&String.upcase/1) || (&String.downcase/1)
+    """
+
+    assert_same """
+    "this works" == (&String.upcase/1) == (&String.downcase/1)
+    """
+
+    bad = """
+    "this works" = (&String.upcase/1) = (&String.downcase/1)
+    """
 
     assert_format bad, """
-    # | :unknown
-    @spec bar ::
-            :ok
-            | :invalid
-            | :other
+    "this works" = (&String.upcase/1) = &String.downcase/1
+    """
+
+    bad = """
+    "this works" ++ (&String.upcase/1) ++ (&String.downcase/1)
+    """
+
+    assert_format bad, """
+    "this works" ++ (&String.upcase/1) ++ &String.downcase/1
+    """
+
+    bad = """
+    "this works" | (&String.upcase/1) | (&String.downcase/1)
+    """
+
+    assert_format bad, """
+    "this works" | (&String.upcase/1) | &String.downcase/1
+    """
+
+    bad = ~S"""
+    "this works" \\ (&String.upcase/1) \\ (&String.downcase/1)
+    """
+
+    assert_format bad, ~S"""
+    "this works" \\ &String.upcase/1 \\ &String.downcase/1
     """
   end
 end

lib/elixir/test/elixir/code_formatter/operators_test.exs

@@ -121,7 +121,13 @@ defmodule Code.Formatter.OperatorsTest do
     end
 
     test "not in" do
-      assert_format "not(foo in bar)", "foo not in bar"
+      assert_format "not foo in bar", "not (foo in bar)"
+      assert_format "not foo in bar", "foo not in bar", rename_deprecated_at: "1.5.0"
+
+      assert_format "not(foo in bar)", "not (foo in bar)"
+      assert_format "not(foo in bar)", "foo not in bar", rename_deprecated_at: "1.5.0"
+
+      assert_same "foo not in bar"
       assert_same "(not foo) in bar"
       assert_same "(!foo) in bar"
     end
@@ -806,9 +812,9 @@ defmodule Code.Formatter.OperatorsTest do
     end
 
     test "with operators outside" do
-      assert_same "(& &1) == & &2"
-      assert_same "(& &1) and & &2"
-      assert_same "(&foo/1) and &bar/1"
+      assert_same "(& &1) == (& &2)"
+      assert_same "(& &1) and (& &2)"
+      assert_same "(&foo/1) and (&bar/1)"
       assert_same "[(&IO.puts/1) | &IO.puts/2]"
     end
 

lib/elixir/test/elixir/dynamic_supervisor_test.exs

@@ -9,6 +9,12 @@ defmodule DynamicSupervisorTest do
     def init(args), do: args
   end
 
+  test "can be supervised directly" do
+    children = [{DynamicSupervisor, strategy: :one_for_one, name: :dyn_sup_spec_test}]
+    assert {:ok, _} = Supervisor.start_link(children, strategy: :one_for_one)
+    assert DynamicSupervisor.which_children(:dyn_sup_spec_test) == []
+  end
+
   describe "use/2" do
     test "generates child_spec/1" do
       assert Simple.child_spec([:hello]) == %{
@@ -40,13 +46,14 @@ defmodule DynamicSupervisorTest do
   describe "init/1" do
     test "set default options" do
       assert DynamicSupervisor.init(strategy: :one_for_one) ==
-               {:ok, %{
-                 strategy: :one_for_one,
-                 intensity: 3,
-                 period: 5,
-                 max_children: :infinity,
-                 extra_arguments: []
-               }}
+               {:ok,
+                %{
+                  strategy: :one_for_one,
+                  intensity: 3,
+                  period: 5,
+                  max_children: :infinity,
+                  extra_arguments: []
+                }}
     end
   end
 
@@ -333,7 +340,7 @@ defmodule DynamicSupervisorTest do
       assert %{workers: 4, active: 2} = DynamicSupervisor.count_children(pid)
     end
 
-    test "restarting children counted in max_children" do
+    test "restarting on init children counted in max_children" do
       child = current_module_worker([:restart, :error], restart: :permanent)
       opts = [strategy: :one_for_one, max_children: 1, max_restarts: 100_000]
       {:ok, pid} = DynamicSupervisor.start_link(opts)
@@ -346,6 +353,39 @@ defmodule DynamicSupervisorTest do
       assert {:error, :max_children} = DynamicSupervisor.start_child(pid, child)
     end
 
+    test "restarting on exit children counted in max_children" do
+      child = current_module_worker([:ok2], restart: :permanent)
+      opts = [strategy: :one_for_one, max_children: 1, max_restarts: 100_000]
+      {:ok, pid} = DynamicSupervisor.start_link(opts)
+
+      assert {:ok, child_pid} = DynamicSupervisor.start_child(pid, child)
+      assert_kill(child_pid, :shutdown)
+      assert %{workers: 1, active: 1} = DynamicSupervisor.count_children(pid)
+
+      child = current_module_worker([:ok2], restart: :permanent)
+      assert {:error, :max_children} = DynamicSupervisor.start_child(pid, child)
+    end
+
+    test "restarting a child with extra_arguments successfully restarts child" do
+      parent = self()
+
+      fun = fn ->
+        send(parent, :from_child)
+        :timer.sleep(:infinity)
+      end
+
+      {:ok, sup} = DynamicSupervisor.start_link(strategy: :one_for_one, extra_arguments: [fun])
+      child = %{id: Task, restart: :transient, start: {Task, :start_link, []}}
+
+      assert {:ok, child} = DynamicSupervisor.start_child(sup, child)
+      assert is_pid(child)
+      assert_receive :from_child
+      assert %{active: 1, workers: 1} = DynamicSupervisor.count_children(sup)
+      assert_kill(child, :oops)
+      assert_receive :from_child
+      assert %{workers: 1, active: 1} = DynamicSupervisor.count_children(sup)
+    end
+
     test "child is restarted when trying again" do
       child = current_module_worker([:try_again, self()], restart: :permanent)
       {:ok, pid} = DynamicSupervisor.start_link(strategy: :one_for_one, max_restarts: 2)

lib/elixir/test/elixir/enum_test.exs

@@ -878,10 +878,7 @@ defmodule EnumTest do
     assert Enum.zip([[1]]) == [{1}]
 
     assert Enum.zip([[], [], [], []]) == []
-
-    assert_raise FunctionClauseError, fn ->
-      Enum.zip(%{})
-    end
+    assert Enum.zip(%{}) == []
   end
 end
 

lib/elixir/test/elixir/exception_test.exs

@@ -275,11 +275,10 @@ defmodule ExceptionTest do
     {:error, reason} = __MODULE__.Sup.start_link(fn -> return end)
     assert Exception.format_exit(reason) =~ "bad child specification, invalid restart type: :foo"
 
-    return =
-      {
-        :ok,
-        {{:one_for_one, 1, 1}, [{:child, {:m, :f, []}, :temporary, :foo, :worker, []}]}
-      }
+    return = {
+      :ok,
+      {{:one_for_one, 1, 1}, [{:child, {:m, :f, []}, :temporary, :foo, :worker, []}]}
+    }
 
     {:error, reason} = __MODULE__.Sup.start_link(fn -> return end)
     assert Exception.format_exit(reason) =~ "bad child specification, invalid shutdown: :foo"
@@ -292,51 +291,47 @@ defmodule ExceptionTest do
     {:error, reason} = __MODULE__.Sup.start_link(fn -> return end)
     assert Exception.format_exit(reason) =~ "bad child specification, invalid modules: :foo"
 
-    return =
-      {
-        :ok,
-        {{:one_for_one, 1, 1}, [{:child, {:m, :f, []}, :temporary, 1, :worker, [{:foo}]}]}
-      }
+    return = {
+      :ok,
+      {{:one_for_one, 1, 1}, [{:child, {:m, :f, []}, :temporary, 1, :worker, [{:foo}]}]}
+    }
 
     {:error, reason} = __MODULE__.Sup.start_link(fn -> return end)
     assert Exception.format_exit(reason) =~ "bad child specification, invalid module: {:foo}"
 
-    return =
+    return = {
+      :ok,
       {
-        :ok,
-        {
-          {:one_for_one, 1, 1},
-          [
-            {:child, {:m, :f, []}, :permanent, 1, :worker, []},
-            {:child, {:m, :f, []}, :permanent, 1, :worker, []}
-          ]
-        }
+        {:one_for_one, 1, 1},
+        [
+          {:child, {:m, :f, []}, :permanent, 1, :worker, []},
+          {:child, {:m, :f, []}, :permanent, 1, :worker, []}
+        ]
       }
+    }
 
     {:error, reason} = __MODULE__.Sup.start_link(fn -> return end)
 
     assert Exception.format_exit(reason) =~
              "bad child specification, more than one child specification has the id: :child"
 
-    return =
-      {
-        :ok,
-        {{:one_for_one, 1, 1}, [{:child, {Kernel, :exit, [:foo]}, :temporary, 1, :worker, []}]}
-      }
+    return = {
+      :ok,
+      {{:one_for_one, 1, 1}, [{:child, {Kernel, :exit, [:foo]}, :temporary, 1, :worker, []}]}
+    }
 
     {:error, reason} = __MODULE__.Sup.start_link(fn -> return end)
 
     assert Exception.format_exit(reason) ==
              "shutdown: failed to start child: :child\n    ** (EXIT) :foo"
 
-    return =
+    return = {
+      :ok,
       {
-        :ok,
-        {
-          {:one_for_one, 1, 1},
-          [{:child, {Kernel, :apply, [fn -> {:error, :foo} end, []]}, :temporary, 1, :worker, []}]
-        }
+        {:one_for_one, 1, 1},
+        [{:child, {Kernel, :apply, [fn -> {:error, :foo} end, []]}, :temporary, 1, :worker, []}]
       }
+    }
 
     {:error, reason} = __MODULE__.Sup.start_link(fn -> return end)
 
@@ -439,34 +434,48 @@ defmodule ExceptionTest do
       end
 
       test "annotates args and clauses from mfa" do
-        {:ok, :def, clauses} = Exception.blame_mfa(Keyword, :pop, [%{}, :key, nil])
+        import PathHelpers
+
+        write_beam(
+          defmodule Blaming do
+            def with_elem(x, y) when elem(x, 1) == 0 and elem(x, y) == 1 do
+              {x, y}
+            end
+
+            def fetch(%module{} = container, key), do: {module, container, key}
+            def fetch(map, key) when is_map(map), do: {map, key}
+            def fetch(list, key) when is_list(list) and is_atom(key), do: {list, key}
+            def fetch(nil, _key), do: nil
+
+            require Integer
+            def even_and_odd(foo, bar) when Integer.is_even(foo) and Integer.is_odd(bar), do: :ok
+          end
+        )
+
+        :code.delete(Blaming)
+        :code.purge(Blaming)
+
+        {:ok, :def, clauses} = Exception.blame_mfa(Blaming, :with_elem, [1, 2])
 
         assert annotated_clauses_to_string(clauses) == [
-                 "{[+keywords+, +key+, +default+], [-is_list(keywords)-]}"
+                 "{[+x+, +y+], [-elem(x, 1) == 0- and -elem(x, y) == 1-]}"
                ]
 
-        {:ok, :def, clauses} = Exception.blame_mfa(Keyword, :fetch, [[], "oops"])
-
-        assert annotated_clauses_to_string(clauses) == [
-                 "{[+keywords+, +key+], [+is_list(keywords)+ and -is_atom(key)-]}"
-               ]
-
-        {:ok, :def, clauses} = Exception.blame_mfa(Path, :type, [self()])
-
-        assert annotated_clauses_to_string(clauses) == [
-                 "{[+name+], [-is_list(name)-, -is_binary(name)-]}"
-               ]
-
-        {:ok, :def, clauses} = Exception.blame_mfa(Access, :fetch, [self(), "oops"])
+        {:ok, :def, clauses} = Exception.blame_mfa(Blaming, :fetch, [self(), "oops"])
 
         assert annotated_clauses_to_string(clauses) == [
                  "{[-%module{} = container-, +key+], []}",
                  "{[+map+, +key+], [-is_map(map)-]}",
                  "{[+list+, +key+], [-is_list(list)- and -is_atom(key)-]}",
-                 "{[+list+, +key+], [-is_list(list)-]}",
                  "{[-nil-, +_key+], []}"
                ]
 
+        {:ok, :def, clauses} = Exception.blame_mfa(Blaming, :even_and_odd, [1, 1])
+
+        assert annotated_clauses_to_string(clauses) == [
+                 "{[+foo+, +bar+], [+is_integer(foo)+ and -Bitwise.band(foo, 1) == 0- and (+is_integer(bar)+ and +Bitwise.band(bar, 1) == 1+)]}"
+               ]
+
         {:ok, :defmacro, clauses} = Exception.blame_mfa(Kernel, :!, [true])
 
         assert annotated_clauses_to_string(clauses) == [

lib/elixir/test/elixir/fixtures/dialyzer/for_bitstring.ex

@@ -0,0 +1,5 @@
+defmodule Dialyzer.ForBitstring do
+  def foo() do
+    for a <- 1..3, into: "", do: <<a>>
+  end
+end

lib/elixir/test/elixir/inspect_test.exs

@@ -194,14 +194,17 @@ defmodule Inspect.NumberTest do
 
   test "hex" do
     assert inspect(100, base: :hex) == "0x64"
+    assert inspect(-100, base: :hex) == "-0x64"
   end
 
   test "octal" do
     assert inspect(100, base: :octal) == "0o144"
+    assert inspect(-100, base: :octal) == "-0o144"
   end
 
   test "binary" do
     assert inspect(86, base: :binary) == "0b1010110"
+    assert inspect(-86, base: :binary) == "-0b1010110"
   end
 
   test "float" do

lib/elixir/test/elixir/kernel/comprehension_test.exs

@@ -143,8 +143,38 @@ defmodule Kernel.ComprehensionTest do
   end
 
   test "for comprehensions into binary" do
-    enum = 1..3
-    assert for(x <- enum, into: "", do: to_bin(x * 2)) == <<2, 4, 6>>
+    enum = 0..3
+
+    assert (for x <- enum, into: "" do
+              to_bin(x * 2)
+            end) == <<0, 2, 4, 6>>
+
+    assert (for x <- enum, into: "" do
+              if Integer.is_even(x), do: <<x::size(2)>>, else: <<x::size(1)>>
+            end) == <<0::size(2), 1::size(1), 2::size(2), 3::size(1)>>
+  end
+
+  test "for comprehensions into dynamic binary" do
+    enum = 0..3
+    into = ""
+
+    assert (for x <- enum, into: into do
+              to_bin(x * 2)
+            end) == <<0, 2, 4, 6>>
+
+    assert (for x <- enum, into: into do
+              if Integer.is_even(x), do: <<x::size(2)>>, else: <<x::size(1)>>
+            end) == <<0::size(2), 1::size(1), 2::size(2), 3::size(1)>>
+
+    into = <<7::size(1)>>
+
+    assert (for x <- enum, into: into do
+              to_bin(x * 2)
+            end) == <<7::size(1), 0, 2, 4, 6>>
+
+    assert (for x <- enum, into: into do
+              if Integer.is_even(x), do: <<x::size(2)>>, else: <<x::size(1)>>
+            end) == <<7::size(1), 0::size(2), 1::size(1), 2::size(2), 3::size(1)>>
   end
 
   test "for comprehensions where value is not used" do
@@ -238,9 +268,39 @@ defmodule Kernel.ComprehensionTest do
     assert for(x <- enum, into: [], do: x * 2) == [2, 4, 6]
   end
 
-  test "list for comprehensions into binaries" do
-    enum = [1, 2, 3]
-    assert for(x <- enum, into: "", do: to_bin(x * 2)) == <<2, 4, 6>>
+  test "list for comprehensions into binary" do
+    enum = [0, 1, 2, 3]
+
+    assert (for x <- enum, into: "" do
+              to_bin(x * 2)
+            end) == <<0, 2, 4, 6>>
+
+    assert (for x <- enum, into: "" do
+              if Integer.is_even(x), do: <<x::size(2)>>, else: <<x::size(1)>>
+            end) == <<0::size(2), 1::size(1), 2::size(2), 3::size(1)>>
+  end
+
+  test "list for comprehensions into dynamic binary" do
+    enum = [0, 1, 2, 3]
+    into = ""
+
+    assert (for x <- enum, into: into do
+              to_bin(x * 2)
+            end) == <<0, 2, 4, 6>>
+
+    assert (for x <- enum, into: into do
+              if Integer.is_even(x), do: <<x::size(2)>>, else: <<x::size(1)>>
+            end) == <<0::size(2), 1::size(1), 2::size(2), 3::size(1)>>
+
+    into = <<7::size(1)>>
+
+    assert (for x <- enum, into: into do
+              to_bin(x * 2)
+            end) == <<7::size(1), 0, 2, 4, 6>>
+
+    assert (for x <- enum, into: into do
+              if Integer.is_even(x), do: <<x::size(2)>>, else: <<x::size(1)>>
+            end) == <<7::size(1), 0::size(2), 1::size(1), 2::size(2), 3::size(1)>>
   end
 
   test "map for comprehensions into map" do
@@ -278,9 +338,39 @@ defmodule Kernel.ComprehensionTest do
     assert for(<<x <- bin>>, into: [], do: x * 2) == [2, 4, 6]
   end
 
-  test "binary for comprehensions into binaries" do
-    bin = <<1, 2, 3>>
-    assert for(<<x <- bin>>, into: "", do: to_bin(x * 2)) == <<2, 4, 6>>
+  test "binary for comprehensions into binary" do
+    bin = <<0, 1, 2, 3>>
+
+    assert (for <<x <- bin>>, into: "" do
+              to_bin(x * 2)
+            end) == <<0, 2, 4, 6>>
+
+    assert (for <<x <- bin>>, into: "" do
+              if Integer.is_even(x), do: <<x::size(2)>>, else: <<x::size(1)>>
+            end) == <<0::size(2), 1::size(1), 2::size(2), 3::size(1)>>
+  end
+
+  test "binary for comprehensions into dynamic binary" do
+    bin = <<0, 1, 2, 3>>
+    into = ""
+
+    assert (for <<x <- bin>>, into: into do
+              to_bin(x * 2)
+            end) == <<0, 2, 4, 6>>
+
+    assert (for <<x <- bin>>, into: into do
+              if Integer.is_even(x), do: <<x::size(2)>>, else: <<x::size(1)>>
+            end) == <<0::size(2), 1::size(1), 2::size(2), 3::size(1)>>
+
+    into = <<7::size(1)>>
+
+    assert (for <<x <- bin>>, into: into do
+              to_bin(x * 2)
+            end) == <<7::size(1), 0, 2, 4, 6>>
+
+    assert (for <<x <- bin>>, into: into do
+              if Integer.is_even(x), do: <<x::size(2)>>, else: <<x::size(1)>>
+            end) == <<7::size(1), 0::size(2), 1::size(1), 2::size(2), 3::size(1)>>
   end
 
   test "binary for comprehensions with variable size" do

lib/elixir/test/elixir/kernel/dialyzer_test.exs

@@ -48,10 +48,25 @@ defmodule Kernel.DialyzerTest do
       |> String.to_charlist()
 
     File.cp!(context.base_plt, plt)
-    dialyzer = [analysis_type: :succ_typings, check_plt: false, files_rec: [dir], plts: [plt]]
+    warnings = Map.get(context, :warnings, [])
+
+    dialyzer = [
+      analysis_type: :succ_typings,
+      check_plt: false,
+      files_rec: [dir],
+      plts: [plt],
+      warnings: warnings
+    ]
+
     {:ok, [outdir: dir, dialyzer: dialyzer]}
   end
 
+  @tag warnings: [:specdiffs]
+  test "no warnings on specdiffs", context do
+    copy_beam!(context, Dialyzer.RemoteCall)
+    assert_dialyze_no_warnings!(context)
+  end
+
   test "no warnings on valid remote calls", context do
     copy_beam!(context, Dialyzer.RemoteCall)
     assert_dialyze_no_warnings!(context)
@@ -105,6 +120,11 @@ defmodule Kernel.DialyzerTest do
     assert_dialyze_no_warnings!(context)
   end
 
+  test "no warnings on for comprehensions with bitstrings", context do
+    copy_beam!(context, Dialyzer.ForBitstring)
+    assert_dialyze_no_warnings!(context)
+  end
+
   test "no warnings on for falsy check that always boolean", context do
     copy_beam!(context, Dialyzer.ForBooleanCheck)
     assert_dialyze_no_warnings!(context)

lib/elixir/test/elixir/kernel/docs_test.exs

@@ -178,6 +178,9 @@ defmodule Kernel.DocsTest do
 
           @doc false
           def qux(true), do: false
+
+          Module.add_doc(__MODULE__, __ENV__.line, :def, {:nullary, 0}, [], "add_doc")
+          def nullary, do: 0
         end
       )
 
@@ -191,6 +194,7 @@ defmodule Kernel.DocsTest do
                {{:bar, 1}, _, :def, [{:arg, _, nil}], "Multiple bodiless clause doc"},
                {{:baz, 1}, _, :def, [{:arg, _, nil}], "Multiple bodiless clause and docs"},
                {{:foo, 1}, _, :def, [{:arg, _, nil}], "Function doc"},
+               {{:nullary, 0}, _, :def, [], "add_doc"},
                {{:qux, 1}, _, :def, [{:bool, _, Elixir}], false}
              ] = docs[:docs]
 
@@ -207,4 +211,42 @@ defmodule Kernel.DocsTest do
              ] = docs[:callback_docs]
     end
   end
+
+  test "@impl true doesn't set @doc false if previous implementation has docs" do
+    write_beam(
+      defmodule Docs do
+        defmodule SampleBehaviour do
+          @callback foo(any()) :: any()
+          @callback bar() :: any()
+          @callback baz() :: any()
+        end
+
+        @behaviour SampleBehaviour
+
+        @doc "Foo docs"
+        def foo(nil), do: nil
+
+        @impl true
+        def foo(_), do: false
+
+        @impl true
+        def bar(), do: true
+
+        @doc "Baz docs"
+        @impl true
+        def baz(), do: true
+
+        def fuz(), do: true
+      end
+    )
+
+    docs = Code.get_docs(Docs, :all)
+
+    assert [
+             {{:bar, 0}, _, :def, [], false},
+             {{:baz, 0}, _, :def, [], "Baz docs"},
+             {{:foo, 1}, _, :def, [{:arg1, [], _}], "Foo docs"},
+             {{:fuz, 0}, _, :def, [], nil}
+           ] = docs[:docs]
+  end
 end

lib/elixir/test/elixir/kernel/errors_test.exs

@@ -108,6 +108,12 @@ defmodule Kernel.ErrorsTest do
     assert_eval_raise SyntaxError, msg, "foo:+1"
   end
 
+  test "invalid map start" do
+    assert_eval_raise SyntaxError,
+                      "nofile:1: expected %{ to define a map, got: %[",
+                      "{:ok, %[], %{}}"
+  end
+
   test "sigil terminator" do
     assert_eval_raise TokenMissingError,
                       "nofile:3: missing terminator: \" (for sigil ~r\" starting at line 1)",
@@ -695,6 +701,12 @@ defmodule Kernel.ErrorsTest do
       def foo(n) when is_number(n)
     end
     '''
+
+    assert_eval_raise CompileError, "nofile:2: missing :do option in \"def\"", '''
+    defmodule Kernel.ErrorsTest.BodyessFunctionWithGuard do
+      def foo(n) when is_number(n), true
+    end
+    '''
   end
 
   test "invalid args for bodyless clause" do

lib/elixir/test/elixir/kernel/expansion_test.exs

@@ -1945,7 +1945,7 @@ defmodule Kernel.ExpansionTest do
       message = ~r"literal <<>> in bitstring supports only type specifiers"
 
       assert_raise CompileError, message, fn ->
-        expand(quote(do: <<(<<"foo">>::32)>>))
+        expand(quote(do: <<(<<"foo">>)::32>>))
       end
     end
 

lib/elixir/test/elixir/kernel/fn_test.exs

@@ -58,7 +58,7 @@ defmodule Kernel.FnTest do
     assert (&List.flatten/1).([[0]]) == [0]
     assert (&List.flatten/1).([[0]]) == [0]
     assert (&List.flatten(&1)).([[0]]) == [0]
-    assert (&List.flatten(&1)) == &List.flatten/1
+    assert (&List.flatten(&1)) == (&List.flatten/1)
   end
 
   test "capture local" do
@@ -77,7 +77,7 @@ defmodule Kernel.FnTest do
     assert (&is_atom/1).(:a)
     assert (&is_atom/1).(:a)
     assert (&is_atom(&1)).(:a)
-    assert (&is_atom(&1)) == &is_atom/1
+    assert (&is_atom(&1)) == (&is_atom/1)
   end
 
   test "capture macro" do
@@ -102,7 +102,7 @@ defmodule Kernel.FnTest do
     mod = List
     assert (&mod.flatten(&1)).([1, [2], 3]) == [1, 2, 3]
     assert (&mod.flatten/1).([1, [2], 3]) == [1, 2, 3]
-    assert (&mod.flatten/1) == &List.flatten/1
+    assert (&mod.flatten/1) == (&List.flatten/1)
   end
 
   test "local partial application" do

lib/elixir/test/elixir/kernel/guard_test.exs

@@ -4,98 +4,83 @@ defmodule Kernel.GuardTest do
   use ExUnit.Case, async: true
 
   describe "Kernel.defguard(p) usage" do
-    test "successfully defines guard" do
-      defmodule(Success, do: defguard(foo(bar, baz) when bar + baz))
-    end
+    defmodule GuardsInMacros do
+      defguard is_foo(atom) when atom == :foo
 
-    test "successfully defines private guard" do
-      defmodule PrivateSuccess do
-        defguardp foo(bar, baz) when bar + baz
-        def fizz(a, b) when foo(a, b), do: :buzz
+      defmacro is_compile_time_foo(atom) when is_foo(atom) do
+        quote do: unquote(__MODULE__).is_foo(unquote(atom))
       end
     end
 
     test "guards can be used in other macros in the same module" do
-      defmodule Guards.In.Macros do
-        defguard is_foo(atom) when atom == :foo
+      require GuardsInMacros
+      assert GuardsInMacros.is_foo(:foo)
+      refute GuardsInMacros.is_foo(:baz)
+      assert GuardsInMacros.is_compile_time_foo(:foo)
+    end
 
-        defmacro is_foobar(atom) when is_foo(atom) do
-          quote bind_quoted: [atom: atom], do: is_foo(atom)
-        end
+    defmodule GuardsInFuns do
+      defguard is_foo(atom) when atom == :foo
+      defguard is_equal(foo, bar) when foo == bar
+
+      def is_foobar(atom) when is_foo(atom) do
+        is_foo(atom)
       end
     end
 
     test "guards can be used in other funs in the same module" do
-      defmodule Guards.In.Funs do
-        defguard is_foo(atom) when atom == :foo
+      require GuardsInFuns
+      assert GuardsInFuns.is_foo(:foo)
+      refute GuardsInFuns.is_foo(:bar)
+    end
 
-        def is_foobar(atom) when is_foo(atom) do
-          is_foo(atom)
+    test "guards do not change code evaluation semantics" do
+      require GuardsInFuns
+      x = 1
+      assert GuardsInFuns.is_equal(x = 2, x) == false
+      assert x == 2
+    end
+
+    defmodule MacrosInGuards do
+      defmacro is_foo(atom) do
+        quote do
+          unquote(atom) == :foo
         end
       end
+
+      defguard is_foobar(atom) when is_foo(atom) or atom == :bar
     end
 
     test "macros can be used in other guards in the same module" do
-      defmodule Macros.In.Guards do
-        defmacro is_foo(atom) do
-          quote do
-            unquote(atom) == :foo
-          end
-        end
+      require MacrosInGuards
+      assert MacrosInGuards.is_foobar(:foo)
+      assert MacrosInGuards.is_foobar(:bar)
+      refute MacrosInGuards.is_foobar(:baz)
+    end
 
-        defguard is_foobar(atom) when is_foo(atom) or atom == :bar
-      end
+    defmodule GuardsInGuards do
+      defguard is_foo(atom) when atom == :foo
+      defguard is_foobar(atom) when is_foo(atom) or atom == :bar
     end
 
     test "guards can be used in other guards in the same module" do
-      defmodule Guards.In.Guards do
-        defguard is_foo(atom) when atom == :foo
-        defguard is_foobar(atom) when is_foo(atom) or atom == :bar
-      end
+      require GuardsInGuards
+      assert GuardsInGuards.is_foobar(:foo)
+      assert GuardsInGuards.is_foobar(:bar)
+      refute GuardsInGuards.is_foobar(:baz)
     end
 
-    # The below works––and is supposed to be equivalent to the above.
-    # However it doesn't look like any defguard is getting enough context about
-    # the caller env to recognize that the above 3 cases should be valid,
-    # raising CompileError undefined function is_foo/1.
-    # I suspect that inaccurate information is being provided to :elixir_expand.expand/2,
-    # making it think these are remote, rather than local, calls, based on the backtrace
-    # through :elixir_expand.expand_remote/7.
-    test "macros can be used in other macros" do
-      defmodule Macros.In.Macros do
-        defmacro is_foo(atom) do
-          quote do
-            case Macro.Env.in_guard?(__CALLER__) do
-              true ->
-                unquote(atom) == :foo
-
-              false ->
-                atom = unquote(atom)
-                atom == :foo
-            end
-          end
-        end
-
-        defmacro is_foobar(atom) do
-          quote do
-            case Macro.Env.in_guard?(__CALLER__) do
-              true ->
-                is_foo(unquote(atom)) or unquote(atom) == :bar
-
-              false ->
-                atom = unquote(atom)
-                is_foo(atom) or atom == :bar
-            end
-          end
-        end
-      end
+    defmodule DefaultArgs do
+      defguard is_divisible(value, remainder \\ 2)
+               when is_integer(value) and rem(value, remainder) == 0
     end
 
     test "permits default values in args" do
-      defmodule Default.Args do
-        defguard is_divisible(value, remainder \\ 2)
-                 when is_integer(value) and rem(value, remainder) == 0
-      end
+      require DefaultArgs
+      assert DefaultArgs.is_divisible(2)
+      refute DefaultArgs.is_divisible(1)
+      assert DefaultArgs.is_divisible(3, 3)
+      refute DefaultArgs.is_divisible(3, 4)
     end
 
     test "doesn't allow matching in args" do
@@ -124,58 +109,52 @@ defmodule Kernel.GuardTest do
       end
     end
 
-    test "defguard defines guards that work inside and outside guard clauses" do
-      defmodule Integer.Guards do
-        defguard is_even(value) when is_integer(value) and rem(value, 2) == 0
-
-        def is_even_and_large?(value) when is_even(value) and value > 100, do: true
-        def is_even_and_large?(_), do: false
+    defmodule GuardFromMacro do
+      defmacro __using__(_) do
+        quote do
+          defguard is_even(value) when is_integer(value) and rem(value, 2) == 0
+        end
       end
+    end
 
-      defmodule Integer.Utils do
-        import Integer.Guards
+    test "defguard defines a guard from inside another macro" do
+      defmodule UseGuardFromMacro do
+        use GuardFromMacro
 
-        def is_even_and_small?(value) do
-          if is_even(value) and value <= 100, do: true, else: false
+        def assert! do
+          assert is_even(0)
+          refute is_even(1)
         end
       end
 
-      assert Integer.Guards.is_even_and_large?(102)
-      refute Integer.Guards.is_even_and_large?(98)
-      refute Integer.Guards.is_even_and_large?(99)
-      refute Integer.Guards.is_even_and_large?(103)
+      UseGuardFromMacro.assert!()
+    end
 
-      assert Integer.Utils.is_even_and_small?(98)
-      refute Integer.Utils.is_even_and_small?(99)
-      refute Integer.Utils.is_even_and_small?(102)
-      refute Integer.Utils.is_even_and_small?(103)
+    defmodule IntegerPrivateGuards do
+      defguardp is_even(value) when is_integer(value) and rem(value, 2) == 0
+
+      def is_even_and_large?(value) when is_even(value) and value > 100, do: true
+      def is_even_and_large?(_), do: false
+
+      def is_even_and_small?(value) do
+        if is_even(value) and value <= 100, do: true, else: false
+      end
     end
 
     test "defguardp defines private guards that work inside and outside guard clauses" do
-      defmodule Integer.Private.Guards do
-        defguardp is_even(value) when is_integer(value) and rem(value, 2) == 0
+      assert IntegerPrivateGuards.is_even_and_large?(102)
+      refute IntegerPrivateGuards.is_even_and_large?(98)
+      refute IntegerPrivateGuards.is_even_and_large?(99)
+      refute IntegerPrivateGuards.is_even_and_large?(103)
 
-        def is_even_and_large?(value) when is_even(value) and value > 100, do: true
-        def is_even_and_large?(_), do: false
-
-        def is_even_and_small?(value) do
-          if is_even(value) and value <= 100, do: true, else: false
-        end
-      end
-
-      assert Integer.Private.Guards.is_even_and_large?(102)
-      refute Integer.Private.Guards.is_even_and_large?(98)
-      refute Integer.Private.Guards.is_even_and_large?(99)
-      refute Integer.Private.Guards.is_even_and_large?(103)
-
-      assert Integer.Private.Guards.is_even_and_small?(98)
-      refute Integer.Private.Guards.is_even_and_small?(99)
-      refute Integer.Private.Guards.is_even_and_small?(102)
-      refute Integer.Private.Guards.is_even_and_small?(103)
+      assert IntegerPrivateGuards.is_even_and_small?(98)
+      refute IntegerPrivateGuards.is_even_and_small?(99)
+      refute IntegerPrivateGuards.is_even_and_small?(102)
+      refute IntegerPrivateGuards.is_even_and_small?(103)
 
       assert_raise CompileError, ~r"cannot invoke local is_even/1 inside guard", fn ->
-        defmodule Integer.Private.Guard.Utils do
-          import Integer.Private.Guards
+        defmodule IntegerPrivateGuardUtils do
+          import IntegerPrivateGuards
 
           def is_even_and_large?(value) when is_even(value) and value > 100, do: true
           def is_even_and_large?(_), do: false
@@ -183,8 +162,8 @@ defmodule Kernel.GuardTest do
       end
 
       assert_raise CompileError, ~r"undefined function is_even/1", fn ->
-        defmodule Integer.Private.Function.Utils do
-          import Integer.Private.Guards
+        defmodule IntegerPrivateFunctionUtils do
+          import IntegerPrivateGuards
 
           def is_even_and_small?(value) do
             if is_even(value) and value <= 100, do: true, else: false
@@ -412,13 +391,11 @@ defmodule Kernel.GuardTest do
         Code.string_to_quoted("""
         case Macro.Env.in_guard? __CALLER__ do
           true -> quote do
-            unquote(foo) + unquote(bar) + unquote(baz)
+            :erlang.+(:erlang.+(unquote(foo), unquote(bar)), unquote(baz))
           end
           false -> quote do
-            foo = unquote(foo)
-            bar = unquote(bar)
-            baz = unquote(baz)
-            foo + bar + baz
+            {foo, bar, baz} = {unquote(foo), unquote(bar), unquote(baz)}
+            :erlang.+(:erlang.+(foo, bar), baz)
           end
         end
         """)
@@ -434,12 +411,11 @@ defmodule Kernel.GuardTest do
         Code.string_to_quoted("""
         case Macro.Env.in_guard? __CALLER__ do
           true -> quote do
-            unquote(foo) + unquote(bar)
+            :erlang.+(unquote(foo), unquote(bar))
           end
           false -> quote do
-            foo = unquote(foo)
-            bar = unquote(bar)
-            foo + bar
+            {foo, bar} = {unquote(foo), unquote(bar)}
+            :erlang.+(foo, bar)
           end
         end
         """)
@@ -453,16 +429,16 @@ defmodule Kernel.GuardTest do
 
       {:ok, goal} =
         Code.string_to_quoted("""
-        case Macro.Env.in_guard? __CALLER__ do
-          true -> quote do
-            unquote(foo) + unquote(foo) + unquote(bar) + unquote(baz)
-          end
-          false -> quote do
-            foo = unquote(foo)
-            bar = unquote(bar)
-            baz = unquote(baz)
-            foo + foo + bar + baz
-          end
+        case(Macro.Env.in_guard?(__CALLER__)) do
+          true ->
+            quote() do
+              :erlang.+(:erlang.+(:erlang.+(unquote(foo), unquote(foo)), unquote(bar)), unquote(baz))
+            end
+          false ->
+            quote() do
+              {foo, bar, baz} = {unquote(foo), unquote(bar), unquote(baz)}
+              :erlang.+(:erlang.+(:erlang.+(foo, foo), bar), baz)
+            end
         end
         """)
 

lib/elixir/test/elixir/kernel/lexical_tracker_test.exs

@@ -234,21 +234,21 @@ defmodule Kernel.LexicalTrackerTest do
           NotAModule
           Remote.func()
           R.func()
-          &extract/2
-          &is_record/1
-          &R.func/0
-          &Remote.func/0
-          &Integer.is_even/1
+          _ = &extract/2
+          _ = &is_record/1
+          _ = &R.func/0
+          _ = &Remote.func/0
+          _ = &Integer.is_even/1
           %Macro.Env{}
         end
 
-        &extract/2
-        &is_record/1
-        &R.func/0
-        &Remote.func/0
-        &Integer.is_even/1
+        _ = &extract/2
+        _ = &is_record/1
+        _ = &R.func/0
+        _ = &Remote.func/0
+        _ = &Integer.is_even/1
 
-        &is_record/1; def b(a), do: is_record(a)
+        _ = &is_record/1; def b(a), do: is_record(a)
 
         %Macro.Env{}
 

lib/elixir/test/elixir/kernel/typespec_test.exs

@@ -835,8 +835,12 @@ defmodule Kernel.TypespecTest do
         quote(do: @type(literal_keyword_list_type_key() :: [{binary(), integer()}])),
         quote(do: @type(literal_empty_map() :: %{})),
         quote(do: @type(literal_map_with_key() :: %{key: integer()})),
-        quote(do: @type(literal_map_with_required_key() :: %{required(bitstring()) => integer()})),
-        quote(do: @type(literal_map_with_optional_key() :: %{optional(bitstring()) => integer()})),
+        quote(
+          do: @type(literal_map_with_required_key() :: %{required(bitstring()) => integer()})
+        ),
+        quote(
+          do: @type(literal_map_with_optional_key() :: %{optional(bitstring()) => integer()})
+        ),
         quote(do: @type(literal_struct_all_fields_any_type() :: %SomeStruct{})),
         quote(do: @type(literal_struct_all_fields_key_type() :: %SomeStruct{key: integer()})),
         quote(do: @type(literal_empty_tuple() :: {})),
@@ -863,7 +867,9 @@ defmodule Kernel.TypespecTest do
         quote(do: @type(builtin_list() :: list())),
         quote(do: @type(builtin_nonempty_list() :: nonempty_list())),
         quote(do: @type(builtin_maybe_improper_list() :: maybe_improper_list())),
-        quote(do: @type(builtin_nonempty_maybe_improper_list() :: nonempty_maybe_improper_list())),
+        quote(
+          do: @type(builtin_nonempty_maybe_improper_list() :: nonempty_maybe_improper_list())
+        ),
         quote(do: @type(builtin_mfa() :: mfa())),
         quote(do: @type(builtin_module() :: module())),
         quote(do: @type(builtin_no_return() :: no_return())),

lib/elixir/test/elixir/kernel/warning_test.exs

@@ -8,6 +8,20 @@ defmodule Kernel.WarningTest do
     capture_io(:stderr, fun)
   end
 
+  test "outdented heredoc" do
+    output =
+      capture_err(fn ->
+        Code.eval_string("""
+          '''
+        outdented
+          '''
+        """)
+      end)
+
+    assert output =~ "outdented heredoc line"
+    assert output =~ "nofile:2"
+  end
+
   test "unused variable" do
     output =
       capture_err(fn ->
@@ -819,7 +833,7 @@ defmodule Kernel.WarningTest do
                @behaviour UndefinedBehaviour
              end
              """)
-           end) =~ "module UndefinedBehaviour does not exist (in module Sample)"
+           end) =~ "@behaviour UndefinedBehaviour does not exist (in module Sample)"
   after
     purge(Sample)
   end
@@ -1078,13 +1092,13 @@ defmodule Kernel.WarningTest do
   end
 
   test "unused variable in defguard" do
-    capture_err(fn ->
-      Code.eval_string("""
-      defmodule Sample do
-        defguard foo(bar, baz) when bar
-      end
-      """)
-    end) =~ "variable \"baz\" is unused"
+    assert capture_err(fn ->
+             Code.eval_string("""
+             defmodule Sample do
+               defguard foo(bar, baz) when bar
+             end
+             """)
+           end) =~ "variable \"baz\" is unused"
   after
     purge(Sample)
   end
@@ -1115,65 +1129,39 @@ defmodule Kernel.WarningTest do
   end
 
   test "defguard overriding defmacro" do
-    capture_err(fn ->
-      Code.eval_string("""
-      defmodule Sample do
-        defmacro foo(bar), do: bar == :bar
-        defguard foo(baz) when baz == :baz
-      end
-      """)
-    end) =~ "this clause cannot match because a previous clause at line 2 always matches"
+    assert capture_err(fn ->
+             Code.eval_string("""
+             defmodule Sample do
+               defmacro foo(bar), do: bar == :bar
+               defguard foo(baz) when baz == :baz
+             end
+             """)
+           end) =~ "this clause cannot match because a previous clause at line 2 always matches"
   after
     purge(Sample)
   end
 
   test "defmacro overriding defguard" do
-    capture_err(fn ->
-      Code.eval_string("""
-      defmodule Sample do
-        defguard foo(baz) when baz == :baz
-        defmacro foo(bar), do: bar == :bar
-      end
-      """)
-    end) =~ "this clause cannot match because a previous clause at line 2 always matches"
-  after
-    purge(Sample)
-  end
-
-  test "defguardp overriding defmacrop" do
-    capture_err(fn ->
-      Code.eval_string("""
-      defmodule Sample do
-        defmacrop foo(bar), do: bar == :bar
-        defguardp foo(baz) when baz == :baz
-      end
-      """)
-    end) =~ "this clause cannot match because a previous clause at line 2 always matches"
-  after
-    purge(Sample)
-  end
-
-  test "defmacrop overriding defguardp" do
-    capture_err(fn ->
-      Code.eval_string("""
-      defmodule Sample do
-      defguardp foo(baz) when baz == :baz
-        defmacrop foo(bar), do: bar == :bar
-      end
-      """)
-    end) =~ "this clause cannot match because a previous clause at line 2 always matches"
+    assert capture_err(fn ->
+             Code.eval_string("""
+             defmodule Sample do
+               defguard foo(baz) when baz == :baz
+               defmacro foo(bar), do: bar == :bar
+             end
+             """)
+           end) =~ "this clause cannot match because a previous clause at line 2 always matches"
   after
     purge(Sample)
   end
 
   test "defguard needs an implementation" do
-    capture_err(fn ->
-      Code.eval_string("""
-      defmodule Sample do
-        defguard foo(bar)
-      end
-      """)
-    end) =~ "implementation not provided for predefined defguard"
+    assert capture_err(fn ->
+             Code.eval_string("""
+             defmodule Sample do
+               defguard foo(bar)
+             end
+             """)
+           end) =~ "implementation not provided for predefined defmacro foo/1"
   after
     purge(Sample)
   end

lib/elixir/test/elixir/kernel_test.exs

@@ -243,6 +243,14 @@ defmodule KernelTest do
     assert_raise BadBooleanError, fn -> 0 or 1 end
   end
 
+  test "if/2 boolean optimization does not leak variables during expansion" do
+    if false do
+      :ok
+    else
+      assert __ENV__.vars == []
+    end
+  end
+
   describe "in/2" do
     test "with literals on right side" do
       assert 2 in [1, 2, 3]

lib/elixir/test/elixir/list_test.exs

@@ -217,14 +217,30 @@ defmodule ListTest do
     end
   end
 
-  test "myers_difference/2" do
-    assert List.myers_difference([], []) == []
-    assert List.myers_difference([], [1, 2, 3]) == [ins: [1, 2, 3]]
-    assert List.myers_difference([1, 2, 3], []) == [del: [1, 2, 3]]
-    assert List.myers_difference([1, 2, 3], [1, 2, 3]) == [eq: [1, 2, 3]]
-    assert List.myers_difference([1, 2, 3], [1, 4, 2, 3]) == [eq: [1], ins: [4], eq: [2, 3]]
-    assert List.myers_difference([1, 4, 2, 3], [1, 2, 3]) == [eq: [1], del: [4], eq: [2, 3]]
-    assert List.myers_difference([1], [[1]]) == [del: [1], ins: [[1]]]
-    assert List.myers_difference([[1]], [1]) == [del: [[1]], ins: [1]]
+  describe "myers_difference/2" do
+    test "follows paper implementation" do
+      assert List.myers_difference([], []) == []
+      assert List.myers_difference([], [1, 2, 3]) == [ins: [1, 2, 3]]
+      assert List.myers_difference([1, 2, 3], []) == [del: [1, 2, 3]]
+      assert List.myers_difference([1, 2, 3], [1, 2, 3]) == [eq: [1, 2, 3]]
+      assert List.myers_difference([1, 2, 3], [1, 4, 2, 3]) == [eq: [1], ins: [4], eq: [2, 3]]
+      assert List.myers_difference([1, 4, 2, 3], [1, 2, 3]) == [eq: [1], del: [4], eq: [2, 3]]
+      assert List.myers_difference([1], [[1]]) == [del: [1], ins: [[1]]]
+      assert List.myers_difference([[1]], [1]) == [del: [[1]], ins: [1]]
+    end
+
+    test "rearranges inserts and equals for smaller diffs" do
+      assert List.myers_difference([3, 2, 0, 2], [2, 2, 0, 2]) ==
+               [del: [3], ins: [2], eq: [2, 0, 2]]
+
+      assert List.myers_difference([3, 2, 1, 0, 2], [2, 1, 2, 1, 0, 2]) ==
+               [del: [3], ins: [2, 1], eq: [2, 1, 0, 2]]
+
+      assert List.myers_difference([3, 2, 2, 1, 0, 2], [2, 2, 1, 2, 1, 0, 2]) ==
+               [del: [3], eq: [2, 2, 1], ins: [2, 1], eq: [0, 2]]
+
+      assert List.myers_difference([3, 2, 0, 2], [2, 2, 1, 0, 2]) ==
+               [del: [3], eq: [2], ins: [2, 1], eq: [0, 2]]
+    end
   end
 end

lib/elixir/test/elixir/macro_test.exs

@@ -196,7 +196,7 @@ defmodule MacroTest do
       quoted =
         quote context: Kernel do
           case 1 do
-            unquote(temp_var) when unquote(temp_var) in [false, nil] -> false
+            unquote(temp_var) when :"Elixir.Kernel".in(unquote(temp_var), [false, nil]) -> false
             unquote(temp_var) -> unquote(temp_var)
           end
         end
@@ -210,7 +210,7 @@ defmodule MacroTest do
       quoted =
         quote context: Kernel do
           case 1 do
-            unquote(temp_var) when unquote(temp_var) in [false, nil] -> false
+            unquote(temp_var) when :"Elixir.Kernel".in(unquote(temp_var), [false, nil]) -> false
             unquote(temp_var) -> unquote(temp_var)
           end
         end
@@ -248,7 +248,7 @@ defmodule MacroTest do
     quoted =
       quote context: Kernel do
         case 1 do
-          unquote(temp_var) when unquote(temp_var) in [false, nil] -> false
+          unquote(temp_var) when :"Elixir.Kernel".in(unquote(temp_var), [false, nil]) -> false
           unquote(temp_var) -> unquote(temp_var)
         end
       end
@@ -318,16 +318,26 @@ defmodule MacroTest do
     end
 
     test "sigil call" do
-      assert Macro.to_string(quote(do: ~r"123")) == ~s/~r"123"/
-      assert Macro.to_string(quote(do: ~r"123"u)) == ~s/~r"123"u/
-      assert Macro.to_string(quote(do: ~r"\n123")) == ~s/~r"\\\\n123"/
+      assert Macro.to_string(quote(do: ~r"123")) == ~S/~r"123"/
+      assert Macro.to_string(quote(do: ~r"123"u)) == ~S/~r"123"u/
+      assert Macro.to_string(quote(do: ~r"\n123")) == ~S/~r"\\n123"/
 
       assert Macro.to_string(quote(do: ~r"1#{two}3")) == ~S/~r"1#{two}3"/
       assert Macro.to_string(quote(do: ~r"1#{two}3"u)) == ~S/~r"1#{two}3"u/
 
-      assert Macro.to_string(quote(do: ~R"123")) == ~s/~R"123"/
-      assert Macro.to_string(quote(do: ~R"123"u)) == ~s/~R"123"u/
-      assert Macro.to_string(quote(do: ~R"\n123")) == ~s/~R"\\\\n123"/
+      assert Macro.to_string(quote(do: ~R"123")) == ~S/~R"123"/
+      assert Macro.to_string(quote(do: ~R"123"u)) == ~S/~R"123"u/
+      assert Macro.to_string(quote(do: ~R"\n123")) == ~S/~R"\n123"/
+
+      assert Macro.to_string(quote(do: ~S["'(123)'"])) == ~S/~S["'(123)'"]/
+
+      assert Macro.to_string(
+               quote do
+                 ~S"""
+                 "123"
+                 """
+               end
+             ) == ~s[~S"""\n"123"\n"""]
     end
 
     test "tuple call" do
@@ -412,6 +422,45 @@ defmodule MacroTest do
       assert Macro.to_string(quoted) <> "\n" == expected
     end
 
+    test "try" do
+      quoted =
+        quote do
+          try do
+            foo
+          catch
+            _, _ ->
+              2
+          rescue
+            ArgumentError ->
+              1
+          after
+            4
+          else
+            _ ->
+              3
+          end
+        end
+
+      expected = """
+      try() do
+        foo
+      rescue
+        ArgumentError ->
+          1
+      catch
+        _, _ ->
+          2
+      else
+        _ ->
+          3
+      after
+        4
+      end
+      """
+
+      assert Macro.to_string(quoted) <> "\n" == expected
+    end
+
     test "fn" do
       assert Macro.to_string(quote(do: fn -> 1 + 2 end)) == "fn -> 1 + 2 end"
       assert Macro.to_string(quote(do: fn x -> x + 1 end)) == "fn x -> x + 1 end"
@@ -547,6 +596,7 @@ defmodule MacroTest do
       assert Macro.to_string(quote(do: %Test{foo: 1, bar: 1})) == "%Test{foo: 1, bar: 1}"
       assert Macro.to_string(quote(do: %Test{struct | foo: 2})) == "%Test{struct | foo: 2}"
       assert Macro.to_string(quote(do: %Test{} + 1)) == "%Test{} + 1"
+      assert Macro.to_string(quote(do: %Test{foo(1)} + 2)) == "%Test{foo(1)} + 2"
     end
 
     test "binary operators" do

lib/elixir/test/elixir/map_test.exs

@@ -200,6 +200,9 @@ defmodule MapTest do
     assert_raise BadStructError, "expected a struct named MapTest.ExternalUser, got: %{}", fn ->
       %ExternalUser{map | name: "meg"}
     end
+
+    assert Code.eval_string("%struct{} = %ExternalUser{}", [], __ENV__) ==
+             {%ExternalUser{}, [struct: ExternalUser]}
   end
 
   test "structs with variable name" do

lib/elixir/test/elixir/node_test.exs

@@ -5,6 +5,7 @@ defmodule NodeTest do
 
   doctest Node
 
+  @tag :unix
   test "start/3 and stop/0" do
     assert Node.stop() == {:error, :not_found}
     assert {:ok, _} = Node.start(:hello, :shortnames, 15000)

lib/elixir/test/elixir/path_test.exs

@@ -36,67 +36,69 @@ defmodule PathTest do
     assert_raise ArgumentError, ~r/null byte/, fn -> Path.wildcard("foo\0bar") end
   end
 
-  if windows?() do
-    describe "Windows" do
-      test "relative/1" do
-        assert Path.relative("C:/usr/local/bin") == "usr/local/bin"
-        assert Path.relative("C:\\usr\\local\\bin") == "usr\\local\\bin"
-        assert Path.relative("C:usr\\local\\bin") == "usr\\local\\bin"
+  describe "Windows" do
+    @describetag :windows
 
-        assert Path.relative("/usr/local/bin") == "usr/local/bin"
-        assert Path.relative("usr/local/bin") == "usr/local/bin"
-        assert Path.relative("../usr/local/bin") == "../usr/local/bin"
-      end
+    test "relative/1" do
+      assert Path.relative("C:/usr/local/bin") == "usr/local/bin"
+      assert Path.relative("C:\\usr\\local\\bin") == "usr\\local\\bin"
+      assert Path.relative("C:usr\\local\\bin") == "usr\\local\\bin"
 
-      test "relative_to/2" do
-        assert Path.relative_to("D:/usr/local/foo", "D:/usr/") == "local/foo"
-        assert Path.relative_to("D:/usr/local/foo", "d:/usr/") == "local/foo"
-        assert Path.relative_to("d:/usr/local/foo", "D:/usr/") == "local/foo"
-        assert Path.relative_to("D:/usr/local/foo", "d:/") == "usr/local/foo"
-        assert Path.relative_to("D:/usr/local/foo", "D:/") == "usr/local/foo"
-        assert Path.relative_to("D:/usr/local/foo", "d:") == "D:/usr/local/foo"
-        assert Path.relative_to("D:/usr/local/foo", "D:") == "D:/usr/local/foo"
-      end
-
-      test "type/1" do
-        assert Path.type("C:/usr/local/bin") == :absolute
-        assert Path.type('C:\\usr\\local\\bin') == :absolute
-        assert Path.type("C:usr\\local\\bin") == :volumerelative
-
-        assert Path.type("/usr/local/bin") == :volumerelative
-        assert Path.type('usr/local/bin') == :relative
-        assert Path.type("../usr/local/bin") == :relative
-      end
-
-      test "split/1" do
-        assert Path.split("C:\\foo\\bar") == ["c:/", "foo", "bar"]
-        assert Path.split("C:/foo/bar") == ["c:/", "foo", "bar"]
-      end
+      assert Path.relative("/usr/local/bin") == "usr/local/bin"
+      assert Path.relative("usr/local/bin") == "usr/local/bin"
+      assert Path.relative("../usr/local/bin") == "../usr/local/bin"
     end
-  else
-    describe "Unix" do
-      test "relative/1" do
-        assert Path.relative("/usr/local/bin") == "usr/local/bin"
-        assert Path.relative("usr/local/bin") == "usr/local/bin"
-        assert Path.relative("../usr/local/bin") == "../usr/local/bin"
-        assert Path.relative("/") == "."
-        assert Path.relative('/') == "."
-        assert Path.relative(['/usr', ?/, "local/bin"]) == "usr/local/bin"
-      end
 
-      test "type/1" do
-        assert Path.type("/usr/local/bin") == :absolute
-        assert Path.type("usr/local/bin") == :relative
-        assert Path.type("../usr/local/bin") == :relative
+    test "relative_to/2" do
+      assert Path.relative_to("D:/usr/local/foo", "D:/usr/") == "local/foo"
+      assert Path.relative_to("D:/usr/local/foo", "d:/usr/") == "local/foo"
+      assert Path.relative_to("d:/usr/local/foo", "D:/usr/") == "local/foo"
+      assert Path.relative_to("D:/usr/local/foo", "d:/") == "usr/local/foo"
+      assert Path.relative_to("D:/usr/local/foo", "D:/") == "usr/local/foo"
+      assert Path.relative_to("D:/usr/local/foo", "d:") == "D:/usr/local/foo"
+      assert Path.relative_to("D:/usr/local/foo", "D:") == "D:/usr/local/foo"
+    end
 
-        assert Path.type('/usr/local/bin') == :absolute
-        assert Path.type('usr/local/bin') == :relative
-        assert Path.type('../usr/local/bin') == :relative
+    test "type/1" do
+      assert Path.type("C:/usr/local/bin") == :absolute
+      assert Path.type('C:\\usr\\local\\bin') == :absolute
+      assert Path.type("C:usr\\local\\bin") == :volumerelative
 
-        assert Path.type(['/usr/', 'local/bin']) == :absolute
-        assert Path.type(['usr/', 'local/bin']) == :relative
-        assert Path.type(['../usr', '/local/bin']) == :relative
-      end
+      assert Path.type("/usr/local/bin") == :volumerelative
+      assert Path.type('usr/local/bin') == :relative
+      assert Path.type("../usr/local/bin") == :relative
+    end
+
+    test "split/1" do
+      assert Path.split("C:\\foo\\bar") == ["c:/", "foo", "bar"]
+      assert Path.split("C:/foo/bar") == ["c:/", "foo", "bar"]
+    end
+  end
+
+  describe "Unix" do
+    @describetag :unix
+
+    test "relative/1" do
+      assert Path.relative("/usr/local/bin") == "usr/local/bin"
+      assert Path.relative("usr/local/bin") == "usr/local/bin"
+      assert Path.relative("../usr/local/bin") == "../usr/local/bin"
+      assert Path.relative("/") == "."
+      assert Path.relative('/') == "."
+      assert Path.relative(['/usr', ?/, "local/bin"]) == "usr/local/bin"
+    end
+
+    test "type/1" do
+      assert Path.type("/usr/local/bin") == :absolute
+      assert Path.type("usr/local/bin") == :relative
+      assert Path.type("../usr/local/bin") == :relative
+
+      assert Path.type('/usr/local/bin') == :absolute
+      assert Path.type('usr/local/bin') == :relative
+      assert Path.type('../usr/local/bin') == :relative
+
+      assert Path.type(['/usr/', 'local/bin']) == :absolute
+      assert Path.type(['usr/', 'local/bin']) == :relative
+      assert Path.type(['../usr', '/local/bin']) == :relative
     end
   end
 

lib/elixir/test/elixir/stream_test.exs

@@ -1099,10 +1099,8 @@ defmodule StreamTest do
     assert Stream.zip([concat, cycle]) |> Enum.to_list() ==
              [{1, :a}, {2, :b}, {3, :c}, {4, :a}, {5, :b}, {6, :c}]
 
-    assert_raise FunctionClauseError, fn ->
-      enum_of_enums = Stream.cycle([[1, 2], [:a, :b]])
-      Stream.zip(enum_of_enums)
-    end
+    assert Stream.chunk_every([0, 1, 2, 3], 2) |> Stream.zip() |> Enum.to_list() ==
+             [{0, 2}, {1, 3}]
   end
 
   test "zip/1 does not leave streams suspended" do

lib/elixir/test/elixir/string_test.exs

@@ -168,11 +168,17 @@ defmodule StringTest do
   end
 
   test "downcase/1 with greek final sigma" do
-    assert String.downcase("ΣΣ") == "ςς"
-    assert String.downcase("ΣΣ ΣΣ") == "ςς ςς"
+    assert String.downcase("Σ") == "σ"
+    assert String.downcase("ΣΣ") == "σσ"
+    assert String.downcase("Σ ΣΣ") == "σ σσ"
+    assert String.downcase("ΜΕΣ'ΑΠΟ") == "μεσ'απο"
+    assert String.downcase("ΑΣ'ΤΟΥΣ") == "ασ'τουσ"
 
-    assert String.downcase("ΣΣ", :greek) == "σς"
-    assert String.downcase("ΣΣ ΣΣ", :greek) == "σς σς"
+    assert String.downcase("Σ", :greek) == "σ"
+    assert String.downcase("Σ ΣΣ", :greek) == "σ σς"
+    assert String.downcase("Σ ΣΑΣ Σ", :greek) == "σ σας σ"
+    assert String.downcase("ΜΕΣ'ΑΠΟ", :greek) == "μεσ'απο"
+    assert String.downcase("ΑΣ'ΤΟΥΣ", :greek) == "ασ'τους"
   end
 
   test "downcase/1 with ascii" do

lib/elixir/test/elixir/system_test.exs

@@ -92,12 +92,14 @@ defmodule SystemTest do
     end
   end
 
-  if windows?() do
-    test "cmd/2 win" do
+  describe "Windows" do
+    @describetag :windows
+
+    test "cmd/2" do
       assert {"hello\r\n", 0} = System.cmd("cmd", ~w[/c echo hello])
     end
 
-    test "cmd/3 (with options) win" do
+    test "cmd/3 (with options)" do
       assert {["hello\r\n"], 0} =
                System.cmd(
                  "cmd",
@@ -113,7 +115,7 @@ defmodule SystemTest do
 
     @echo "echo-elixir-test"
 
-    test "cmd/2 with absolute and relative Windows paths" do
+    test "cmd/2 with absolute and relative paths" do
       echo = tmp_path(@echo)
       File.mkdir_p!(Path.dirname(echo))
       File.cp!(System.find_executable("cmd"), echo)
@@ -132,12 +134,16 @@ defmodule SystemTest do
     after
       File.rm_rf!(Path.dirname(tmp_path(@echo)))
     end
-  else
-    test "cmd/2 unix" do
+  end
+
+  describe "Unix" do
+    @describetag :unix
+
+    test "cmd/2" do
       assert {"hello\n", 0} = System.cmd("echo", ["hello"])
     end
 
-    test "cmd/3 (with options) unix" do
+    test "cmd/3 (with options)" do
       opts = [
         into: [],
         cd: System.cwd!(),
@@ -152,7 +158,7 @@ defmodule SystemTest do
 
     @echo "echo-elixir-test"
 
-    test "cmd/2 with absolute and relative Unix paths" do
+    test "cmd/2 with absolute and relative paths" do
       echo = tmp_path(@echo)
       File.mkdir_p!(Path.dirname(echo))
       File.cp!(System.find_executable("echo"), echo)

lib/elixir/test/elixir/task/supervisor_test.exs

@@ -22,19 +22,6 @@ defmodule Task.SupervisorTest do
     number
   end
 
-  def sleep_and_return_ancestor(number, :another_arg) do
-    sleep_and_return_ancestor(number)
-  end
-
-  def sleep_and_return_ancestor(number) do
-    Process.sleep(number)
-    {:dictionary, dictionary} = Process.info(self(), :dictionary)
-
-    dictionary
-    |> Keyword.get(:"$ancestors")
-    |> List.first()
-  end
-
   test "can be supervised directly", config do
     modules = [{Task.Supervisor, name: config.test}]
     assert {:ok, _} = Supervisor.start_link(modules, strategy: :one_for_one)
@@ -319,80 +306,6 @@ defmodule Task.SupervisorTest do
 
       refute_received _
     end
-
-    test "streams an enumerable with fun and supervisor fun", %{supervisor: supervisor} do
-      {:ok, other_supervisor} = Task.Supervisor.start_link()
-
-      assert fn i -> if rem(i, 2) == 0, do: supervisor, else: other_supervisor end
-             |> Task.Supervisor.async_stream(1..4, &sleep_and_return_ancestor/1, @opts)
-             |> Enum.to_list() ==
-               [ok: other_supervisor, ok: supervisor, ok: other_supervisor, ok: supervisor]
-    end
-
-    test "streams an enumerable with mfa and supervisor fun", %{supervisor: supervisor} do
-      {:ok, other_supervisor} = Task.Supervisor.start_link()
-      fun = :sleep_and_return_ancestor
-
-      assert fn i -> if rem(i, 2) == 0, do: supervisor, else: other_supervisor end
-             |> Task.Supervisor.async_stream(1..4, __MODULE__, fun, [], @opts)
-             |> Enum.to_list() ==
-               [ok: other_supervisor, ok: supervisor, ok: other_supervisor, ok: supervisor]
-    end
-
-    test "streams an enumerable with mfa with args and supervisor fun", %{supervisor: supervisor} do
-      {:ok, other_supervisor} = Task.Supervisor.start_link()
-      fun = :sleep_and_return_ancestor
-
-      assert fn i -> if rem(i, 2) == 0, do: supervisor, else: other_supervisor end
-             |> Task.Supervisor.async_stream(1..4, __MODULE__, fun, [:another_arg], @opts)
-             |> Enum.to_list() ==
-               [ok: other_supervisor, ok: supervisor, ok: other_supervisor, ok: supervisor]
-    end
-
-    test "streams an enumerable with fun and executes supervisor fun in monitor process",
-         context do
-      %{supervisor: supervisor} = context
-      parent = self()
-
-      supervisor_fun = fn _i ->
-        {:links, links} = Process.info(self(), :links)
-        assert parent in links
-        send(parent, {parent, self()})
-        supervisor
-      end
-
-      assert supervisor_fun
-             |> Task.Supervisor.async_stream(1..4, &sleep_and_return_ancestor/1, @opts)
-             |> Enum.to_list() == [ok: supervisor, ok: supervisor, ok: supervisor, ok: supervisor]
-
-      receive do
-        {^parent, linked} ->
-          for _ <- 1..3, do: assert_received({^parent, ^linked})
-      after
-        0 ->
-          flunk("Did not receive any message from monitor process.")
-      end
-    end
-
-    test "streams an enumerable with fun and bad supervisor fun" do
-      Process.flag(:trap_exit, true)
-
-      stream =
-        fn _i -> raise "bad" end
-        |> Task.Supervisor.async_stream(1..4, &sleep_and_return_ancestor/1, @opts)
-
-      assert {{%RuntimeError{message: "bad"}, _stacktrace}, _mfa} = catch_exit(Stream.run(stream))
-
-      refute_received _
-
-      stream =
-        fn _i -> :not_a_supervisor end
-        |> Task.Supervisor.async_stream(1..4, &sleep_and_return_ancestor/1, @opts)
-
-      assert {{:noproc, _stacktrace}, _mfa} = catch_exit(Stream.run(stream))
-
-      refute_received _
-    end
   end
 
   describe "async_stream_nolink" do

lib/elixir/test/elixir/task_test.exs

@@ -323,6 +323,19 @@ defmodule TaskTest do
       assert Task.yield_many([task1, task2, task3], 0) ==
                [{task1, {:ok, :result}}, {task2, nil}, {task3, {:exit, :normal}}]
     end
+
+    test "returns results on infinity timeout" do
+      task1 = %Task{ref: make_ref(), owner: self()}
+      task2 = %Task{ref: make_ref(), owner: self()}
+      task3 = %Task{ref: make_ref(), owner: self()}
+
+      send(self(), {task1.ref, :result})
+      send(self(), {task2.ref, :result})
+      send(self(), {:DOWN, task3.ref, :process, self(), :normal})
+
+      assert Task.yield_many([task1, task2, task3], :infinity) ==
+               [{task1, {:ok, :result}}, {task2, {:ok, :result}}, {task3, {:exit, :normal}}]
+    end
   end
 
   describe "shutdown/2" do

lib/elixir/test/elixir/test_helper.exs

@@ -1,6 +1,3 @@
-assert_timeout = String.to_integer(System.get_env("ELIXIR_ASSERT_TIMEOUT") || "500")
-ExUnit.start(trace: "--trace" in System.argv(), assert_receive_timeout: assert_timeout)
-
 # Beam files compiled on demand
 path = Path.expand("../../tmp/beams", __DIR__)
 File.rm_rf!(path)
@@ -87,3 +84,12 @@ defmodule CodeFormatterHelpers do
     end
   end
 end
+
+assert_timeout = String.to_integer(System.get_env("ELIXIR_ASSERT_TIMEOUT") || "500")
+exclude = if PathHelpers.windows?(), do: [unix: true], else: [windows: true]
+
+ExUnit.start(
+  trace: "--trace" in System.argv(),
+  assert_receive_timeout: assert_timeout,
+  exclude: exclude
+)

lib/elixir/test/elixir/uri_test.exs

@@ -306,6 +306,10 @@ defmodule URITest do
       assert URI.parse("http://example.com/#").fragment == ""
       assert URI.parse("http://example.com/test#").fragment == ""
     end
+
+    test "preserves an empty query" do
+      assert URI.parse("http://foo.com/?").query == ""
+    end
   end
 
   test "default_port/1,2" do

lib/elixir/test/erlang/tokenizer_test.erl

@@ -151,6 +151,10 @@ string_test() ->
   [{bin_string, {1, 1, nil}, [<<"f\"">>]}] = tokenize("\"f\\\"\""),
   [{list_string, {1, 1, nil}, [<<"foo">>]}] = tokenize("'foo'").
 
+heredoc_test() ->
+  [{bin_heredoc, {1, 1, nil}, [<<"heredoc\n">>]}] = tokenize("\"\"\"\nheredoc\n\"\"\""),
+  [{bin_heredoc, {1, 1, nil}, [<<"heredoc\n">>]}, {';', {3, 5, 0}}] = tokenize("\"\"\"\n heredoc\n \"\"\";").
+
 empty_string_test() ->
   [{bin_string, {1, 1, nil}, [<<>>]}] = tokenize("\"\""),
   [{list_string, {1, 1, nil}, [<<>>]}] = tokenize("''").
@@ -204,6 +208,12 @@ sigil_terminator_test() ->
   [{sigil, {1, 1, nil}, 114, [<<"foo">>], [], <<"/">>}] = tokenize("~r/foo/"),
   [{sigil, {1, 1, nil}, 114, [<<"foo">>], [], <<"[">>}] = tokenize("~r[foo]"),
   [{sigil, {1, 1, nil}, 114, [<<"foo">>], [], <<"\"">>}] = tokenize("~r\"foo\""),
+  [{sigil, {1, 1, nil}, 114, [<<"foo">>], [], <<"/">>},
+   {comp_op, {1, 9, nil}, '=='},
+   {identifier, {1, 12, nil}, bar}] = tokenize("~r/foo/ == bar"),
+  [{sigil, {1, 1, nil}, 114, [<<"foo">>], "iu", <<"/">>},
+   {comp_op, {1, 11, nil}, '=='},
+   {identifier, {1, 14, nil}, bar}] = tokenize("~r/foo/iu == bar"),
   [{sigil, {1, 1, nil}, 83, [<<"sigil heredoc\n">>], [], <<"\"\"\"">>}] = tokenize("~S\"\"\"\nsigil heredoc\n\"\"\""),
   [{sigil, {1, 1, nil}, 83, [<<"sigil heredoc\n">>], [], <<"'''">>}] = tokenize("~S'''\nsigil heredoc\n'''").
 

lib/elixir/unicode/properties.ex

@@ -24,10 +24,39 @@ rangify = fn [head | tail] ->
   [{first, last} | acc]
 end
 
-acc = {[], [], [], %{}, %{}}
+# A character is case ignorable if:
+#
+#    Word_Break(C) = MidLetter or MidNumLet or Single_Quote, or
+#    General_Category(C) = Nonspacing_Mark (Mn), Enclosing_Mark (Me), Format (Cf),
+#                          Modifier_Letter (Lm), or Modifier_Symbol (Sk).
+#
+# Word breaks are defined below based on TR29 (http://unicode.org/reports/tr29/).
+# The categories are computed later.
+case_ignorable = [
+  0x0027,
+  0x002E,
+  0x2018,
+  0x2019,
+  0x2024,
+  0xFE52,
+  0xFF07,
+  0xFF0E,
+  0x00B7,
+  0x0387,
+  0x05F4,
+  0x2027,
+  0x003A,
+  0xFE13,
+  0xFE55,
+  0xFF1A
+]
 
-{codes, letters, non_breakable, decompositions, combining_classes} =
-  Enum.reduce(File.stream!(data_path), acc, fn line, {cacc, lacc, wacc, dacc, kacc} ->
+acc = {[], [], case_ignorable, [], %{}, %{}}
+cased_letter_categories = :binary.compile_pattern(["Ll", "Lt", "Lu"])
+case_ignorable_categories = :binary.compile_pattern(["Mn", "Me", "Cf", "Lm", "Sk"])
+
+{codes, cased_letters, case_ignorable, non_breakable, decompositions, combining_classes} =
+  Enum.reduce(File.stream!(data_path), acc, fn line, {cacc, lacc, iacc, wacc, dacc, kacc} ->
     [
       codepoint,
       _name,
@@ -55,10 +84,19 @@ acc = {[], [], [], %{}, %{}}
         cacc
       end
 
-    lacc =
-      case category do
-        "L" <> _ -> [String.to_integer(codepoint, 16) | lacc]
-        _ -> lacc
+    cased_letter_categories = :binary.compile_pattern(["Ll", "Lt", "Lu"])
+    case_ignorable_categories = :binary.compile_pattern(["Mn", "Me", "Cf", "Lm", "Sk"])
+
+    {lacc, iacc} =
+      cond do
+        match?({0, _}, :binary.match(category, cased_letter_categories)) ->
+          {[String.to_integer(codepoint, 16) | lacc], iacc}
+
+        match?({0, _}, :binary.match(category, case_ignorable_categories)) ->
+          {lacc, [String.to_integer(codepoint, 16) | iacc]}
+
+        true ->
+          {lacc, iacc}
       end
 
     wacc =
@@ -88,7 +126,7 @@ acc = {[], [], [], %{}, %{}}
         {n, ""} -> Map.put(kacc, String.to_integer(codepoint, 16), n)
       end
 
-    {cacc, lacc, wacc, dacc, kacc}
+    {cacc, lacc, iacc, wacc, dacc, kacc}
   end)
 
 defmodule String.Casing do
@@ -111,63 +149,88 @@ defmodule String.Casing do
 
   def downcase(<<unquote(sigma), rest::bits>>, acc, mode) do
     downcased =
-      case mode == :greek and starts_with_letter?(rest) do
-        true -> 0x03C3
-        false -> 0x03C2
+      if mode == :greek and cased_letter_list?(acc) and not cased_letter_binary?(rest) do
+        <<0x03C2::utf8>>
+      else
+        <<0x03C3::utf8>>
       end
 
-    downcase(rest, <<acc::binary, downcased::utf8>>, mode)
+    downcase(rest, [downcased | acc], mode)
   end
 
   for {codepoint, _upper, lower, _title} <- codes,
       lower && lower != codepoint,
       codepoint not in @conditional_downcase do
     def downcase(<<unquote(codepoint), rest::bits>>, acc, mode) do
-      downcase(rest, acc <> unquote(lower), mode)
+      downcase(rest, [unquote(lower) | acc], mode)
     end
   end
 
   def downcase(<<char, rest::bits>>, acc, mode) do
-    downcase(rest, <<acc::binary, char>>, mode)
+    downcase(rest, [<<char>> | acc], mode)
   end
 
-  def downcase("", acc, _mode), do: acc
+  def downcase("", acc, _mode), do: IO.iodata_to_binary(:lists.reverse(acc))
 
   # Sigma handling
 
-  defp starts_with_letter?(<<codepoint::utf8, _::bits>>) do
-    letter?(codepoint)
-  end
-
-  defp starts_with_letter?(_) do
-    false
-  end
-
-  for {first, last} <- rangify.(letters) do
-    if first == last do
-      defp letter?(unquote(first)), do: true
+  defp cased_letter_binary?(<<codepoint::utf8, rest::bits>>) do
+    if case_ignorable?(codepoint) do
+      cased_letter_binary?(rest)
     else
-      defp letter?(codepoint)
+      cased_letter?(codepoint)
+    end
+  end
+
+  defp cased_letter_binary?(_), do: false
+
+  defp cased_letter_list?([<<codepoint::utf8>> | rest]) do
+    if case_ignorable?(codepoint) do
+      cased_letter_list?(rest)
+    else
+      cased_letter?(codepoint)
+    end
+  end
+
+  defp cased_letter_list?(_), do: false
+
+  for {first, last} <- rangify.(cased_letters) do
+    if first == last do
+      defp cased_letter?(unquote(first)), do: true
+    else
+      defp cased_letter?(codepoint)
            when codepoint >= unquote(first) and codepoint <= unquote(last),
            do: true
     end
   end
 
-  defp letter?(_), do: false
+  defp cased_letter?(_), do: false
+
+  for {first, last} <- rangify.(case_ignorable) do
+    if first == last do
+      defp case_ignorable?(unquote(first)), do: true
+    else
+      defp case_ignorable?(codepoint)
+           when codepoint >= unquote(first) and codepoint <= unquote(last),
+           do: true
+    end
+  end
+
+  defp case_ignorable?(_), do: false
 
   # Upcase
 
   for {codepoint, upper, _lower, _title} <- codes, upper && upper != codepoint do
     def upcase(<<unquote(codepoint), rest::bits>>, acc, mode) do
-      upcase(rest, acc <> unquote(upper), mode)
+      upcase(rest, [unquote(upper) | acc], mode)
     end
   end
 
   def upcase(<<char, rest::bits>>, acc, mode) do
-    upcase(rest, <<acc::binary, char>>, mode)
+    upcase(rest, [char | acc], mode)
   end
 
-  def upcase("", acc, _mode), do: acc
+  def upcase("", acc, _mode), do: IO.iodata_to_binary(:lists.reverse(acc))
 
   # Titlecase once