Release v1.4.4

Signed-off-by: José Valim <jose.valim@plataformatec.com.br>

CHANGELOG.md

@@ -1,6 +1,6 @@
 # Changelog for Elixir v1.4
 
-Elixir v1.4 brings new features, enhancements and bug fixes into Elixir. The most notable changes are the addition of the `Registry` module and the `Task.async_stream/3` and `Task.async_stream/5` which aids developers in writing concurrent software. Those two features and a couple other improvements are described in detail below followed by the complete list of changes.
+Elixir v1.4 brings new features, enhancements and bug fixes into Elixir. The most notable changes are the addition of the `Registry` module and the `Task.async_stream/3` and `Task.async_stream/5` which aid developers in writing concurrent software. Those two features and a couple other improvements are described in detail below followed by the complete list of changes.
 
 ## Registry
 
@@ -55,9 +55,9 @@ While the snippet above works fine in many occasions, for large collections it w
 
 The `Task.async_stream` functions are also lazy, allowing developers to partially consume the stream until a condition is reached. Furthermore, `Task.Supervisor.async_stream/4` and `Task.Supervisor.async_stream/6` can be used to ensure the concurrent tasks are spawned under a given supervisor.
 
-## Application inflection
+## Application inference
 
-Mix v1.4 now automatically inflects the list of applications that are required on runtime from your dependencies list.
+Mix v1.4 now automatically infers the list of applications that are required on runtime from your dependencies list.
 
 In previous Mix versions, most of your dependencies had to be added both to your dependencies list and applications list. Here is how a `mix.exs` would look like:
 
@@ -72,7 +72,7 @@ In previous Mix versions, most of your dependencies had to be added both to your
 
 This was error prone as many developers would not list their dependencies in their applications list.
 
-Mix v1.4 now automatically inflects your applications list as long as you leave the `:applications` key empty. The `mix.exs` above can be rewritten to:
+Mix v1.4 now automatically infers your applications list as long as you leave the `:applications` key empty. The `mix.exs` above can be rewritten to:
 
     def application do
       [extra_applications: [:logger]]
@@ -91,7 +91,121 @@ Finally, if there is a dependency you don't want to include in the application r
 
 We hope this feature provides a more streamlined workflow for developers who are building releases for their Elixir projects.
 
-## v1.4.0-rc.0 (2016-11-28)
+## Mix install from SCM
+
+Mix v1.4 can now install escripts and archives from both Git and Hex, providing you with even more options for distributing Elixir code.
+
+This makes it possible to distribute CLI applications written in Elixir by publishing a package which builds an escript to Hex. [`ex_doc`](https://hex.pm/packages/ex_doc) has been updated to serve as an example of how to use this new functionality.
+
+Simply running:
+
+    mix escript.install hex ex_doc
+
+will fetch `ex_doc` and its dependencies, build them, and then install `ex_doc` to `~/.mix/escripts` (by default). After adding `~/.mix/escripts` to your `PATH`, running `ex_doc` is as simple as:
+
+    ex_doc
+
+You can now also install archives from Hex in this way. Since they are fetched and built on the user's machine, they do not have the same limitations as pre-built archives. However, keep in mind archives run alongside every Mix project, which may lead to conflicts. For this reason, escripts is the preferred format.
+
+It is also possible to install escripts and archives by providing a Git/GitHub repo. See `mix help escript.install` and `mix help archive.install` for more details.
+
+## v1.4.4 (2017-05-15)
+
+This version includes changes that make Elixir fully compatible with Erlang OTP 20-rc.1.
+
+### 1. Bug fixes
+
+#### Elixir
+
+  * [Map] Fix regression on struct update syntax
+
+## v1.4.3 (2017-05-15)
+
+This version includes changes that make Elixir fully compatible with Erlang OTP 20-rc.1.
+
+### 1. Enhancements
+
+#### Elixir
+
+  * [Kernel] Improve compilation time for modules with many clauses
+  * [Map] Warn when attempting to override `__struct__` key
+  * [Regex] Add `recompile/1` and `recompile!/1` to ease transition to OTP 20 for archives and stored regexes
+
+#### Logger
+
+  * [Logger.Translator] Handle OTP 20 GenServer log messages
+
+#### Mix
+
+  * [mix compile] Recompile projects if OTP version changes
+
+### 2. Bug fixes
+
+#### Elixir
+
+  * [Kernel] Fix code generation when non-binary bitstrings are in AST
+  * [Record] Properly escape fields passed to `defrecord`
+
+#### ExUnit
+
+  * [ExUnit.Diff] Do not fail when comparing maps with nil or boolean keys
+
+#### IEx
+
+  * [IEx.Helpers] Do not log exits on `IEx.Helpers.c/2` failures
+
+#### Mix
+
+  * [mix archive.install] Detect proper path on URLs with query strings
+  * [mix loadpaths] Do not assume all paths in loadpaths exist
+
+
+## v1.4.2 (2017-02-16)
+
+### 1. Bug fixes
+
+#### EEx
+
+  * [EEx] Support middle expressions on trim mode
+
+#### Elixir
+
+  * [Calendar] Correct typo on Calendar types
+  * [Kernel] Ensure redefined functions point to the proper source
+  * [OptionParser] Add `:allow_nonexistent_atoms` to support unsafe behaviour prior to v1.4
+  * [Stream] Allow consuming multiple items from suspended enumerable in `Stream.transform/3`
+  * [String] Incorporate new grapheme rules in Unicode 9
+
+#### IEx
+
+  * [IEx.Autocomplete] Do not crash on aliases which are not known at compile time
+
+#### Mix
+
+  * [Mix.Umbrella] Ensure umbrella projects can depend on other umbrella projects
+  * [Mix.Archive] Ensure previous archives with `.ez` extension are deleted
+
+## v1.4.1 (2017-01-26)
+
+### 1. Bug fixes
+
+#### Elixir
+
+  * [Kernel] Remove warning when making private functions overridable
+  * [Path] Ensure `Path.join/1` returns strings for lists of one element
+  * [Regex] Ensure `Regex.escape/1` also escapes `-`
+
+#### IEx
+
+  * [IEx] Disable ANSI detection for powershell to avoid false positives
+
+#### Mix
+
+  * [Mix.Make] Run `make clean` for `erlang.mk`
+  * [Mix.Rebar] Support all of rebar3 dependency package declaration
+  * [Mix.Rebar] Only pass overrides from parent to child in Rebar dep
+
+## v1.4.0 (2017-01-05)
 
 ### 1. Enhancements
 
@@ -101,14 +215,16 @@ We hope this feature provides a more streamlined workflow for developers who are
   * [Calendar] Support `NaiveDateTime.add/3` and `NaiveDateTime.diff/3` for adding seconds (up to microseconds) as well as the difference between two NaiveDateTimes in seconds (up to microseconds)
   * [Calendar] Add `Date.leap_year?/1` and `Date.day_of_week/1`
   * [Calendar] Ensure `Date`, `Time` and `NaiveDateTime` APIs work with any struct that provides the same set of fields as their respective struct. For example, a `NaiveDateTime` can be given to `Date` since it contains a superset of the fields in the `Date` struct
+  * [Calendar] Add `Time.utc_now/0` and `NaiveDateTime.utc_now/0`
   * [Enum] Add `Enum.map_every/2` that invokes the given function with every nth item
   * [Enum] Add `min/2`, `max/2`, `min_max/2`, `min_by/3`, `max_by/3`, and `min_max_by/3` that allow a function specifying the default value when the enumerable is empty
   * [Enum] Introduce `Enum.zip/1` to zip multiple entries at once
   * [Float] Introduce `Float.ratio/1` that returns a tuple with the numerator and denominator as integers to retrieve the given float
-  * [GenServer] Log error on default `handle_info/2` implementation
+  * [GenServer] Log warn on default `handle_info/2` implementation
   * [Inspect] Support syntax coloring via the `:syntax_color` option
   * [Integer] `Integer.digits/2` now accepts negative integers
   * [Integer] Add `Integer.mod/2` and `Integer.floor_div/2`
+  * [IO] Add `:label` option to `IO.inspect/2` to help distinguish multiple `IO.inspect/2` calls.
   * [Kernel] Recognize merge conflict markers in source and provide a readable error message
   * [Kernel] Warn on unused module attributes
   * [Kernel] Improve compiler message on unexpected end of line
@@ -135,6 +251,7 @@ We hope this feature provides a more streamlined workflow for developers who are
 
   * [IEx.Autocomplete] Stop appending a trailing dot when autocompleting modules in IEx
   * [IEx.Autocomplete] Support autocompletion for structs
+  * [IEx.Autocomplete] Improve IEx autocomplete to support navigating map atom keys
   * [IEx.Helpers] `c/1` now compiles in memory by default to avoid common issue where `.beam` files remain at projects root directory
   * [IEx.Helpers] Add info about protocols in `i/1`
   * [IEx.Server] Support interrupting IEx evaluation through the Ctrl+G prompt
@@ -142,12 +259,14 @@ We hope this feature provides a more streamlined workflow for developers who are
 #### Mix
 
   * [mix archive] Compress archive files built by `mix archive` as they are now unzipped during installation
-  * [mix compile] Automatically inflect the list of applications for Mix projects
+  * [mix archive] Install from SCM
+  * [mix compile] Automatically infer the list of applications for Mix projects
   * [mix cmd] Add the ability to specify one or more apps in `mix cmd`
   * [mix deps] Warn if there are non-applications in the `apps` directory for umbrella projects
   * [mix deps] Add warning for invalid paths on `mix deps.clean`
   * [mix deps] Add `Mix.Project.apps_paths` that returns the paths to children applications in umbrella projects
   * [mix deps] Add `MIX_REBAR` environment variable for overriding local rebar
+  * [mix escript] Install from SCM
   * [mix new] Check directory existence in `mix new` and ask how to proceed if one exists
   * [mix new] Applications built with the `--sup` flag now have an individual module to work as application callback
   * [mix test] Add `--formatter` option to `mix test`
@@ -157,9 +276,12 @@ We hope this feature provides a more streamlined workflow for developers who are
 
 #### Elixir
 
+  * [Access] Do not accept nils in `Access.key/1` and `Access.key/2` in favor of explicit default values (or Access.key!/1 if you expect the key to always be available)
   * [Float] Avoid multiple roundings in `Float.ceil/2`, `Float.floor/2` and `Float.round/2`
   * [Kernel] Don't crash in `macro_exported?/3` when dealing with Erlang modules
   * [Kernel] Ensure locals calls are rewritten when calling a local function or macro from inside a module
+  * [Kernel] Annotate the context for variables as zero-arity funs in quotes
+  * [Kernel.SpecialForms] Ensure comprehensions with guards and filters keep proper ordering,
   * [Kernel.SpecialForms] Produce meaningful warning when with's else clauses have no effect
   * [Macro] Wrap fn calls in parens in `Macro.to_string/2`
   * [Macro] Do not print aliases as keys inside keyword lists in `Macro.to_string/2`
@@ -167,6 +289,7 @@ We hope this feature provides a more streamlined workflow for developers who are
   * [Stream] Ensure `Stream.take/2` does not consume next element on `:suspend`
   * [String] Fix infinite recursion in `String.replace_leading/3` and `String.replace_trailing/3` when given an empty string
   * [Task] Fix `Task.shutdown/1,2` infinite block when task has no monitor
+  * [Task] Ensure task cannot link after parents unlinks
 
 #### ExUnit
 
@@ -197,12 +320,14 @@ We hope this feature provides a more streamlined workflow for developers who are
 
 #### Elixir
 
+  * [Access] `Access.key/1` is deprecated due to erratic behaviour for missing keys, please use `Access.key/2` instead with proper default values
   * [Behaviour] The `Behaviour` module is deprecated. Callbacks may now be defined directly via the `@callback` attribute
   * [Enum] Deprecate `Enum.uniq/2` in favor of `Enum.uniq_by/2`
-  * [Float] `Float.to_char_list/2` and `Float.to_string/2` are deprecated (use the :erlang functions if such conversions are desired)
+  * [Float] `Float.to_char_list/2` and `Float.to_string/2` are deprecated (use the `:erlang.float_to_list/2` and `:erlang.float_to_binary/2` functions if such conversions are desired)
   * [Kernel] Deprecate support for making private functions overridable. Overridable functions must always be public as they must be contracts
   * [Kernel] Warn if variable is used as a function call
   * [OptionParser] Deprecate aliases with multiple letters, such as `-abc`
+  * [Set] Deprecate the `Set` module
   * [Stream] Deprecate `Stream.uniq/2` in favor of `Stream.uniq_by/2`
 
 #### IEx
@@ -211,7 +336,7 @@ We hope this feature provides a more streamlined workflow for developers who are
 
 #### Mix
 
-  * [Mix.Utils] `underscore/1` and `camelize/1` are deprecated
+  * [Mix.Utils] `underscore/1` and `camelize/1` are deprecated in favor of `Macro.underscore/1` and `Macro.camelize/1`
 
 ## v1.3
 

Makefile

@@ -1,7 +1,7 @@
 REBAR ?= "$(CURDIR)/rebar"
 PREFIX ?= /usr/local
 SHARE_PREFIX ?= $(PREFIX)/share
-CANONICAL := master/
+CANONICAL :=
 ELIXIRC := bin/elixirc --verbose --ignore-module-conflict
 ERLC := erlc -I lib/elixir/include
 ERL := erl -I lib/elixir/include -noshell -pa lib/elixir/ebin

RELEASE.md

@@ -22,7 +22,7 @@ This document simply outlines the release process:
 
 9. Publish new zips with `make zips`, upload `Precompiled.zip` and `Docs.zip` to GitHub Releases
 
-10. Add the release to `elixir.csv` file in `elixir-lang/elixir-lang.github.com`
+10. Add the release to `elixir.csv` and `_data/elixir-versions.yml` files in `elixir-lang/elixir-lang.github.com`
 
 11. After a new `vMAJOR.MINOR`, move back to master, bump versions, start new CHANGELOG, add `-dev` back and commit "Start vMAJOR.MINOR+1"
 

VERSION

@@ -1 +1 @@
-1.4.0-rc.0
+1.4.4

bin/elixir.bat

@@ -100,27 +100,6 @@ for  /d %%d in ("%originPath%..\lib\*.") do (
 )
 setlocal disabledelayedexpansion
 
-rem ******* detect ANSI terminal support ********************
-timeout 0 2>nul >nul || goto run
-where /Q powershell || goto run
-
-set ASSERT_ANSI= ^
-  $err = 1; ^
-  $Kernel32 = Add-Type -Name 'Kernel32' -PassThru -MemberDefinition ' ^
-    [DllImport(\"Kernel32.dll\", SetLastError = true)] ^
-    public static extern IntPtr GetStdHandle(int nStdHandle); ^
-    [DllImport(\"Kernel32.dll\", SetLastError = true)] ^
-    public static extern bool GetConsoleMode(IntPtr hWnd, ref UInt32 lpMode); ^
-  '; ^
-  $StdoutHandle = $Kernel32::GetStdHandle(-11); ^
-  $ConsoleMode = New-Object -TypeName UInt32; ^
-  $null = $Kernel32::GetConsoleMode($StdoutHandle, [ref]$ConsoleMode); ^
-  if ($ConsoleMode -band 0x4) { $err = 0 } ^
-  exit $err
-
-powershell -NoProfile -NonInteractive -Command %ASSERT_ANSI% || goto run
-set parsErlang=%parsErlang% -elixir ansi_enabled true
-
 :run
 if not %runMode% == "iex" (
   set beforeExtra=-noshell -s elixir start_cli %beforeExtra%

lib/eex/lib/eex/compiler.ex

@@ -90,7 +90,9 @@ defmodule EEx.Compiler do
       {contents, start, list}
     end
   end
-
+  defp look_ahead_text([{:middle_expr, line, _, chars} | t], _start, contents) do
+    {contents ++ chars, line, t}
+  end
   defp look_ahead_text(t, start, contents) do
     {contents, start, t}
   end

lib/eex/lib/eex/tokenizer.ex

@@ -181,7 +181,6 @@ defmodule EEx.Tokenizer do
   # If trim mode is enabled and the token is on a line with
   # only itself and whitespace, trim the whitespace around it,
   # including the line break following it if there is one.
-
   defp trim_if_needed(rest, line, opts, buffer, acc) do
     original = {rest, line, buffer}
     if opts[:trim] do

lib/eex/test/eex_test.exs

@@ -331,6 +331,19 @@ foo
     assert_eval expected, string, [], trim: true
   end
 
+  test "trim mode with middle expression" do
+    string = """
+    <%= cond do %>
+    <% false -> %>
+      this
+    <% true -> %>
+      that
+    <% end %>
+    """
+    expected = "  that\n"
+    assert_eval expected, string, [], trim: true
+  end
+
   test "evaluates the source from a given file" do
     filename = Path.join(__DIR__, "fixtures/eex_template.eex")
     result = EEx.eval_file(filename)

lib/elixir/lib/access.ex

@@ -358,6 +358,28 @@ defmodule Access do
 
   ## Accessors
 
+  @doc false
+  def key(key) do
+    IO.warn "Access.key/1 is deprecated due to erratic behaviour for missing keys, " <>
+            "please use Access.key/2 instead with proper default values " <>
+            "(or Access.key!/1 if you expect the key to always be available)"
+
+    fn
+      :get, data, next ->
+        next.(Map.get(to_map(data), key))
+      :get_and_update, data, next ->
+        value = Map.get(to_map(data), key)
+        case next.(value) do
+          {get, update} -> {get, Map.put(data, key, update)}
+          :pop -> {value, Map.delete(data, key)}
+        end
+    end
+  end
+
+  defp to_map(nil), do: %{}
+  defp to_map(%{} = map), do: map
+  defp to_map(data), do: raise "Access.key/1 expected a map/struct or nil, got: #{inspect data}"
+
   @doc """
   Returns a function that accesses the given key in a map/struct.
 
@@ -367,13 +389,13 @@ defmodule Access do
   The returned function uses the default value if the key does not exist.
   This can be used to specify defaults and safely traverse missing keys:
 
-      iex> get_in(%{}, [Access.key(:user, %{}), Access.key(:name)])
+      iex> get_in(%{}, [Access.key(:user, %{}), Access.key(:name, nil)])
       nil
 
   Such is also useful when using update functions, allowing us to introduce
   values as we traverse the data-structure for updates:
 
-      iex> put_in(%{}, [Access.key(:user, %{}), Access.key(:name)], "Mary")
+      iex> put_in(%{}, [Access.key(:user, %{}), Access.key(:name, nil)], "Mary")
       %{user: %{name: "Mary"}}
 
   ## Examples
@@ -381,23 +403,23 @@ defmodule Access do
       iex> map = %{user: %{name: "john"}}
       iex> get_in(map, [Access.key(:unknown, %{}), Access.key(:name, "john")])
       "john"
-      iex> get_and_update_in(map, [Access.key(:user), Access.key(:name)], fn
+      iex> get_and_update_in(map, [Access.key!(:user), Access.key!(:name)], fn
       ...>   prev -> {prev, String.upcase(prev)}
       ...> end)
       {"john", %{user: %{name: "JOHN"}}}
-      iex> pop_in(map, [Access.key(:user), Access.key(:name)])
+      iex> pop_in(map, [Access.key!(:user), Access.key!(:name)])
       {"john", %{user: %{}}}
 
   An error is raised if the accessed structure is not a map or a struct:
 
-      iex> get_in(nil, [Access.key(:foo)])
+      iex> get_in(nil, [Access.key(:foo, nil)])
       ** (BadMapError) expected a map, got: nil
 
-      iex> get_in([], [Access.key(:foo)])
+      iex> get_in([], [Access.key(:foo, nil)])
       ** (BadMapError) expected a map, got: []
 
   """
-  def key(key, default \\ nil) do
+  def key(key, default) do
     fn
       :get, data, next ->
         next.(Map.get(data, key, default))

lib/elixir/lib/calendar.ex

@@ -51,24 +51,24 @@ defmodule Calendar do
   @type std_offset  :: integer
 
   @typedoc "Any map/struct that contains the date fields"
-  @type date :: %{calendar: calendar, year: year, month: month, day: day}
+  @type date :: %{optional(any) => any, calendar: calendar, year: year, month: month, day: day}
 
   @typedoc "Any map/struct that contains the time fields"
-  @type time :: %{hour: hour, minute: minute, second: second, microsecond: microsecond}
+  @type time :: %{optional(any) => any, hour: hour, minute: minute, second: second, microsecond: microsecond}
 
   @typedoc "Any map/struct that contains the naive_datetime fields"
-  @type naive_date_time :: %{calendar: calendar, year: year, month: month, day: day,
-                             hour: hour, minute: minute, second: second, microsecond: microsecond}
+  @type naive_datetime :: %{optional(any) => any, calendar: calendar, year: year, month: month, day: day,
+                            hour: hour, minute: minute, second: second, microsecond: microsecond}
 
   @typedoc "Any map/struct that contains the datetime fields"
-  @type date_time :: %{calendar: calendar, year: year, month: month, day: day,
-                       hour: hour, minute: minute, second: second, microsecond: microsecond,
-                       time_zone: time_zone, zone_abbr: zone_abbr, utc_offset: utc_offset, std_offset: std_offset}
+  @type datetime :: %{optional(any) => any, calendar: calendar, year: year, month: month, day: day,
+                      hour: hour, minute: minute, second: second, microsecond: microsecond,
+                      time_zone: time_zone, zone_abbr: zone_abbr, utc_offset: utc_offset, std_offset: std_offset}
 
   @doc """
-  Returns the last day of the month for the given year-month pair.
+  Returns how many days there are in the given year-month.
   """
-  @callback last_day_of_month(year, month) :: day
+  @callback days_in_month(year, month) :: day
 
   @doc """
   Returns true if the given year is a leap year.
@@ -90,7 +90,7 @@ defmodule Calendar do
   @callback date_to_string(year, month, day) :: String.t
 
   @doc """
-  Coverts the date time (without time zone) into a string according to the calendar.
+  Converts the date time (without time zone) into a string according to the calendar.
   """
   @callback naive_datetime_to_string(year, month, day, hour, minute, second, microsecond) :: String.t
 
@@ -175,6 +175,24 @@ defmodule Date do
     calendar.leap_year?(year)
   end
 
+  @doc """
+  Returns the number of days in the given date month.
+
+  ## Examples
+
+      iex> Date.days_in_month(~D[1900-01-13])
+      31
+      iex> Date.days_in_month(~D[1900-02-09])
+      28
+      iex> Date.days_in_month(~N[2000-02-20 01:23:45])
+      29
+
+  """
+  @spec days_in_month(Calendar.date) :: Calendar.day
+  def days_in_month(%{calendar: calendar, year: year, month: month}) do
+    calendar.days_in_month(year, month)
+  end
+
   @doc """
   Builds a new ISO date.
 
@@ -1598,9 +1616,7 @@ defmodule DateTime do
 
   WARNING: the ISO 8601 datetime format does not contain the time zone nor
   its abbreviation, which means information is lost when converting to such
-  format. This is also why this module does not provide a `from_iso8601/1`
-  function, as it is impossible to build a proper `DateTime` from only the
-  information in the ISO 8601 string.
+  format.
 
   ### Examples
 
@@ -1630,6 +1646,86 @@ defmodule DateTime do
                                      time_zone, zone_abbr, utc_offset, std_offset)
   end
 
+  @doc """
+  Parses the extended "Date and time of day" format described by
+  [ISO 8601:2004](https://en.wikipedia.org/wiki/ISO_8601).
+
+  Since ISO8601 does not include the proper time zone, the given
+  string will be converted to UTC and its offset in seconds will be
+  returned as part of this function. Therefore offset information
+  must be present in the string.
+
+  As specified in the standard, the separator "T" may be omitted if
+  desired as there is no ambiguity within this function.
+
+  Time representations with reduced accuracy are not supported.
+
+  ## Examples
+
+      iex> DateTime.from_iso8601("2015-01-23T23:50:07Z")
+      {:ok, %DateTime{calendar: Calendar.ISO, day: 23, hour: 23, microsecond: {0, 0}, minute: 50, month: 1, second: 7, std_offset: 0,
+                      time_zone: "Etc/UTC", utc_offset: 0, year: 2015, zone_abbr: "UTC"}, 0}
+      iex> DateTime.from_iso8601("2015-01-23T23:50:07.123+02:30")
+      {:ok, %DateTime{calendar: Calendar.ISO, day: 23, hour: 21, microsecond: {123000, 3}, minute: 20, month: 1, second: 7, std_offset: 0,
+                      time_zone: "Etc/UTC", utc_offset: 0, year: 2015, zone_abbr: "UTC"}, 9000}
+
+      iex> DateTime.from_iso8601("2015-01-23P23:50:07")
+      {:error, :invalid_format}
+      iex> DateTime.from_iso8601("2015-01-23 23:50:07A")
+      {:error, :invalid_format}
+      iex> DateTime.from_iso8601("2015-01-23T23:50:07")
+      {:error, :missing_offset}
+      iex> DateTime.from_iso8601("2015-01-23 23:50:61")
+      {:error, :invalid_time}
+      iex> DateTime.from_iso8601("2015-01-32 23:50:07")
+      {:error, :invalid_date}
+
+      iex> DateTime.from_iso8601("2015-01-23T23:50:07.123-00:00")
+      {:error, :invalid_format}
+      iex> DateTime.from_iso8601("2015-01-23T23:50:07.123-00:60")
+      {:error, :invalid_format}
+
+  """
+  @spec from_iso8601(String.t) :: {:ok, t, Calendar.utc_offset} | {:error, atom}
+  def from_iso8601(<<year::4-bytes, ?-, month::2-bytes, ?-, day::2-bytes, sep,
+                     hour::2-bytes, ?:, min::2-bytes, ?:, sec::2-bytes, rest::binary>>) when sep in [?\s, ?T] do
+    with {year, ""}       <- Integer.parse(year),
+         {month, ""}      <- Integer.parse(month),
+         {day, ""}        <- Integer.parse(day),
+         {hour, ""}       <- Integer.parse(hour),
+         {min, ""}        <- Integer.parse(min),
+         {sec, ""}        <- Integer.parse(sec),
+         {microsec, rest} <- Calendar.ISO.parse_microsecond(rest),
+         {:ok, date}      <- Calendar.ISO.date(year, month, day),
+         {:ok, time}      <- Time.new(hour, min, sec, microsec),
+         {:ok, offset}    <- parse_offset(rest) do
+      %{year: year, month: month, day: day} = date
+      %{hour: hour, minute: minute, second: second, microsecond: microsecond} = time
+
+      erl = {{year, month, day}, {hour, minute, second}}
+      seconds = :calendar.datetime_to_gregorian_seconds(erl)
+      {{year, month, day}, {hour, minute, second}} =
+        :calendar.gregorian_seconds_to_datetime(seconds - offset)
+
+      {:ok, %DateTime{year: year, month: month, day: day,
+                      hour: hour, minute: minute, second: second, microsecond: microsecond,
+                      std_offset: 0, utc_offset: 0, zone_abbr: "UTC", time_zone: "Etc/UTC"}, offset}
+    else
+      {:error, reason} -> {:error, reason}
+      _ -> {:error, :invalid_format}
+    end
+  end
+  def from_iso8601(_) do
+    {:error, :invalid_format}
+  end
+  defp parse_offset(rest) do
+    case Calendar.ISO.parse_offset(rest) do
+      {offset, ""} when is_integer(offset) -> {:ok, offset}
+      {nil, ""} -> {:error, :missing_offset}
+      _ -> {:error, :invalid_format}
+    end
+  end
+
   @doc """
   Converts the given datetime to a string according to its calendar.
 

lib/elixir/lib/calendar/iso.ex

@@ -17,32 +17,32 @@ defmodule Calendar.ISO do
   @type day   :: 1..31
 
   @doc """
-  Returns the last day of the month for the given year.
+  Returns how many days there are in the given year-month.
 
   ## Examples
 
-      iex> Calendar.ISO.last_day_of_month(1900, 1)
+      iex> Calendar.ISO.days_in_month(1900, 1)
       31
-      iex> Calendar.ISO.last_day_of_month(1900, 2)
+      iex> Calendar.ISO.days_in_month(1900, 2)
       28
-      iex> Calendar.ISO.last_day_of_month(2000, 2)
+      iex> Calendar.ISO.days_in_month(2000, 2)
       29
-      iex> Calendar.ISO.last_day_of_month(2001, 2)
+      iex> Calendar.ISO.days_in_month(2001, 2)
       28
-      iex> Calendar.ISO.last_day_of_month(2004, 2)
+      iex> Calendar.ISO.days_in_month(2004, 2)
       29
-      iex> Calendar.ISO.last_day_of_month(2004, 4)
+      iex> Calendar.ISO.days_in_month(2004, 4)
       30
 
   """
-  @spec last_day_of_month(year, month) :: 28..31
-  def last_day_of_month(year, month)
+  @spec days_in_month(year, month) :: 28..31
+  def days_in_month(year, month)
 
-  def last_day_of_month(year, 2) do
+  def days_in_month(year, 2) do
     if leap_year?(year), do: 29, else: 28
   end
-  def last_day_of_month(_, month) when month in [4, 6, 9, 11], do: 30
-  def last_day_of_month(_, month) when month in 1..12, do: 31
+  def days_in_month(_, month) when month in [4, 6, 9, 11], do: 30
+  def days_in_month(_, month) when month in 1..12, do: 31
 
   @doc """
   Returns if the given year is a leap year.

lib/elixir/lib/exception.ex

@@ -647,7 +647,7 @@ defmodule UndefinedFunctionError do
   def message(%{reason: :"function not exported",  module: module, function: function, arity: arity, exports: exports}) do
     suffix =
       if macro_exported?(module, function, arity) do
-        " but #{inspect(module)} defines a macro with the same name and arity." <>
+        ". However there is a macro with the same name and arity." <>
           " Be sure to require #{inspect(module)} if you intend to invoke this macro"
       else
         did_you_mean(module, function, arity, exports)

lib/elixir/lib/float.ex

@@ -203,7 +203,8 @@ defmodule Float do
     count = count - exp + 1023
 
     cond do
-      count <= 0 -> # There is no decimal precision
+      count <= 0 or  # There is no decimal precision
+      (0 == exp and <<0::52>> == significant) -> #zero or minus zero
         float
 
       count >= 104 -> # Precision beyond 15 digits

lib/elixir/lib/gen_server.ex

@@ -568,8 +568,8 @@ defmodule GenServer do
             {_, []}   -> self()
             {_, name} -> name
           end
-        :error_logger.error_msg('~p ~p received unexpected message in handle_info/2: ~p~n',
-                                [__MODULE__, proc, msg])
+        :error_logger.warning_msg('~p ~p received unexpected message in handle_info/2: ~p~n',
+                                  [__MODULE__, proc, msg])
         {:noreply, state}
       end
 

lib/elixir/lib/io/ansi/docs.ex

@@ -2,6 +2,7 @@ defmodule IO.ANSI.Docs do
   @moduledoc false
 
   @bullets [?*, ?-, ?+]
+  @spaces [" ", "\n", "\t"]
 
   @doc """
   The default options used by this module.
@@ -204,7 +205,7 @@ defmodule IO.ANSI.Docs do
     |> Enum.join(" ")
     |> handle_links
     |> handle_inline(options)
-    |> String.split(~r{\s})
+    |> String.split(@spaces)
     |> write_with_wrap(options[:width] - byte_size(indent), indent, no_wrap)
 
     unless no_wrap, do: newline_after_block()
@@ -266,8 +267,10 @@ defmodule IO.ANSI.Docs do
     count = Enum.map(lines, &length/1) |> Enum.max
     lines = Enum.map(lines, &pad_to_number_of_columns(&1, count))
 
-    widths = for line <- lines, do:
-              (for {_col, length} <- line, do: length)
+    widths =
+      for line <- lines do
+        for {_col, length} <- line, do: length
+      end
 
     col_widths = Enum.reduce(widths,
                              List.duplicate(0, count),
@@ -280,15 +283,16 @@ defmodule IO.ANSI.Docs do
     line
     |> String.trim("|")
     |> String.trim()
-    |> String.split(~r/\s\|\s/)
+    |> String.split(" | ")
     |> Enum.map(&render_column(&1, options))
   end
 
   defp render_column(col, options) do
-    col = col
-          |> String.replace(~r/\\ \|/x, "|")
-          |> handle_links
-          |> handle_inline(options)
+    col =
+      col
+      |> String.replace("\\\|", "|")
+      |> handle_links
+      |> handle_inline(options)
     {col, length_without_escape(col, 0)}
   end
 
@@ -319,8 +323,17 @@ defmodule IO.ANSI.Docs do
   defp render_table([], _, _),
     do: nil
 
-  defp table_header?(row), do:
-    Enum.all?(row, fn {col, _} -> col =~ ~r/^:?-+:?$/ end)
+  defp table_header?(row) do
+    Enum.all?(row, fn {col, _} -> table_header_column?(col) end)
+  end
+
+  defp table_header_column?(":" <> row), do: table_header_contents?(row)
+  defp table_header_column?(row), do: table_header_contents?(row)
+
+  defp table_header_contents?("-" <> row), do: table_header_contents?(row)
+  defp table_header_contents?(":"), do: true
+  defp table_header_contents?(""), do: true
+  defp table_header_contents?(_), do: false
 
   defp draw_table_row(cols_and_widths, options, heading \\ false) do
     columns =
@@ -336,11 +349,7 @@ defmodule IO.ANSI.Docs do
   end
 
   defp table_line?(line) do
-    Regex.match?(~r'''
-      ( ^ \s{0,3} \| (?: [^|]+ \|)+ \s* $ )
-    |
-      (\s \| \s)
-    '''x, line)
+    line =~ " | "
   end
 
   ## Helpers
@@ -416,14 +425,17 @@ defmodule IO.ANSI.Docs do
   end
 
   defp escape_underlines_in_link(text) do
-    Regex.replace(~r{https?\S*}, text, &String.replace(&1, "_", "\\_"))
+    ~r{https?\S*}
+    |> Regex.recompile!
+    |> Regex.replace(text, &String.replace(&1, "_", "\\_"))
   end
 
   defp remove_square_brackets_in_link(text) do
-    Regex.replace(~r{\[(.*?)\]\((.*?)\)}, text, "\\1 (\\2)")
+    ~r{\[(.*?)\]\((.*?)\)}
+    |> Regex.recompile!
+    |> Regex.replace(text, "\\1 (\\2)")
   end
 
-
   # We have four entries: **, *, _ and `.
   #
   # The first three behave the same while the last one is simpler

lib/elixir/lib/kernel.ex

@@ -2399,7 +2399,11 @@ defmodule Kernel do
 
   # @attribute(value)
   defp do_at([arg], meta, name, function?, env) do
-    read? = :lists.keymember(:context, 1, meta)
+    line =
+      case :lists.keymember(:context, 1, meta) do
+        true -> nil
+        false -> env.line
+      end
 
     cond do
       function? ->
@@ -2413,11 +2417,11 @@ defmodule Kernel do
         {stack, _} = :elixir_quote.escape(env_stacktrace(env), false)
         arg = {env.line, arg}
         quote do: Module.put_attribute(__MODULE__, unquote(name), unquote(arg),
-                                       unquote(stack), unquote(read?))
+                                       unquote(stack), unquote(line))
 
       true ->
         quote do: Module.put_attribute(__MODULE__, unquote(name), unquote(arg),
-                                       nil, unquote(read?))
+                                       nil, unquote(line))
     end
   end
 
@@ -3704,7 +3708,7 @@ defmodule Kernel do
     end
   end
 
-  @doc """
+  @doc ~S"""
   Defines a protocol.
 
   A protocol specifies an API that should be defined by its
@@ -3712,43 +3716,46 @@ defmodule Kernel do
 
   ## Examples
 
-  In Elixir, only `false` and `nil` are considered falsy values.
-  Everything else evaluates to `true` in `if/2` clauses. Depending
-  on the application, it may be important to specify a `blank?`
-  protocol that returns a boolean for other data types that should
-  be considered "blank". For instance, an empty list or an empty
-  binary could be considered blank.
+  In Elixir, we have two verbs for checking how many items there
+  are in a data structure: `length` and `size`.  `length` means the
+  information must be computed. For example, `length(list)` needs to
+  traverse the whole list to calculate its length. On the other hand,
+  `tuple_size(tuple)` and `byte_size(binary)` do not depend on the
+  tuple and binary size as the size information is precomputed in
+  the data structure.
 
-  Such protocol could be implemented as follows:
+  Although Elixir includes specific functions such as `tuple_size`,
+  `binary_size` and `map_size`, sometimes we want to be able to
+  retrieve the size of a data structure regardless of its type.
+  In Elixir we can write polymorphic code, i.e. code that works
+  with different shapes/types, by using protocols. A size protocol
+  could be implemented as follows:
 
-      defprotocol Blank do
-        @doc "Returns `true` if `data` is considered blank/empty"
-        def blank?(data)
+      defprotocol Size do
+        @doc "Calculates the size (and not the length!) of a data structure"
+        def size(data)
       end
 
-  Now that the protocol is defined it can be implemented. It needs to be
-  implemented for each Elixir type; for example:
+  Now that the protocol can be implemented for every data structure
+  the protocol may have a compliant implementation for:
 
-      # Integers are never blank
-      defimpl Blank, for: Integer do
-        def blank?(number), do: false
+      defimpl Size, for: Binary do
+        def size(binary), do: byte_size(binary)
       end
 
-      # The only blank list is the empty one
-      defimpl Blank, for: List do
-        def blank?([]), do: true
-        def blank?(_),  do: false
+      defimpl Size, for: Map do
+        def size(map), do: map_size(map)
       end
 
-      # The only blank atoms are "false" and "nil"
-      defimpl Blank, for: Atom do
-        def blank?(false), do: true
-        def blank?(nil),   do: true
-        def blank?(_),     do: false
+      defimpl Size, for: Tuple do
+        def size(tuple), do: tuple_size(tuple)
       end
 
-  The implementation of the `Blank` protocol would need to be defined for all
-  Elixir types. The available types are:
+  Notice we didn't implement it for lists as we don't have the
+  `size` information on lists, rather its value needs to be
+  computed with `length`.
+
+  It is possible to implement protocols for all Elixir types:
 
     * Structs (see below)
     * `Tuple`
@@ -3768,11 +3775,11 @@ defmodule Kernel do
 
   The real benefit of protocols comes when mixed with structs.
   For instance, Elixir ships with many data types implemented as
-  structs, like `MapSet`. We can implement the `Blank` protocol
+  structs, like `MapSet`. We can implement the `Size` protocol
   for those types as well:
 
-      defimpl Blank, for: MapSet do
-        def blank?(enum_like), do: Enum.empty?(enum_like)
+      defimpl Size, for: MapSet do
+        def size(map_set), do: MapSet.size(map_set)
       end
 
   When implementing a protocol for a struct, the `:for` option can
@@ -3782,54 +3789,54 @@ defmodule Kernel do
       defmodule User do
         defstruct [:email, :name]
 
-        defimpl Blank do
-          def blank?(%User{}), do: false
+        defimpl Size do
+          def size(%User{}), do: 2 # two fields
         end
       end
 
-  If a protocol is not found for a given type, it will fallback to
-  `Any`. Protocols that are implemented for maps don't work by default
-  on structs; look at `defstruct/1` for more information about deriving
+  If a protocol implementation is not found for a given type,
+  invoking the protocol will raise unless it is configured to
+  fallback to `Any`. Conveniences for building implementations
+  on top of existing ones are also available, look at `defstruct/1`
+  for more information about deriving
   protocols.
 
   ## Fallback to any
 
   In some cases, it may be convenient to provide a default
-  implementation for all types. This can be achieved by
-  setting the `@fallback_to_any` attribute to `true` in the protocol
+  implementation for all types. This can be achieved by setting
+  the `@fallback_to_any` attribute to `true` in the protocol
   definition:
 
-      defprotocol Blank do
+      defprotocol Size do
         @fallback_to_any true
-        def blank?(data)
+        def size(data)
       end
 
-  The `Blank` protocol can now be implemented for `Any`:
+  The `Size` protocol can now be implemented for `Any`:
 
-      defimpl Blank, for: Any do
-        def blank?(_), do: true
+      defimpl Size, for: Any do
+        def size(_), do: 0
       end
 
-  One may wonder why such behaviour (fallback to any) is not the default one.
-
-  It is two-fold: first, the majority of protocols cannot
-  implement an action in a generic way for all types; in fact,
-  providing a default implementation may be harmful, because users
-  may rely on the default implementation instead of providing a
-  specialized one.
-
-  Second, falling back to `Any` adds an extra lookup to all types,
-  which is unnecessary overhead unless an implementation for `Any` is
-  required.
+  Although the implementation above is arguably not a reasonable
+  one. For example, it makes no sense to say a PID or an Integer
+  have a size of 0. That's one of the reasons why `@fallback_to_any`
+  is an opt-in behaviour. For the majority of protocols, raising
+  an error when a protocol is not implemented is the proper behaviour.
 
   ## Types
 
   Defining a protocol automatically defines a type named `t`, which
   can be used as follows:
 
-      @spec present?(Blank.t) :: boolean
-      def present?(blank) do
-        not Blank.blank?(blank)
+      @spec print_size(Size.t) :: :ok
+      def print_size(data) do
+        IO.puts(case Size.size(data) do
+          0 -> "data has no items"
+          1 -> "data has one item"
+          n -> "data has #{n} items"
+        end)
       end
 
   The `@spec` above expresses that all types allowed to implement the
@@ -3868,12 +3875,12 @@ defmodule Kernel do
   all implementations are known up-front, Elixir provides a feature
   called protocol consolidation. For this reason, all protocols are
   compiled with `debug_info` set to `true`, regardless of the option
-  set by `elixirc` compiler. The debug info though may be removed
-  after consolidation.
+  set by `elixirc` compiler. The debug info though may be removed after
+  consolidation.
 
-  For more information on how to apply protocol consolidation to
-  a given project, please check the functions in the `Protocol`
-  module or the `mix compile.protocols` task.
+  Protocol consolidation is applied by default to all Mix projects.
+  For applying consolidation manually, please check the functions in
+  the `Protocol` module or the `mix compile.protocols` task.
   """
   defmacro defprotocol(name, do_block)
 

lib/elixir/lib/kernel/special_forms.ex

@@ -222,7 +222,7 @@ defmodule Kernel.SpecialForms do
       iex> x = 1
       iex> <<x::8>> == <<x::size(8)>>
       true
-      iex> <<x::8 * 4>> == <<x::size(8)-unit(4)>>
+      iex> <<x::8*4>> == <<x::size(8)-unit(4)>>
       true
 
   This syntax reflects the fact the effective size is given by
@@ -293,7 +293,7 @@ defmodule Kernel.SpecialForms do
 
       defmodule ImageTyper
         @png_signature <<137::size(8), 80::size(8), 78::size(8), 71::size(8),
-                      13::size(8), 10::size(8), 26::size(8), 10::size(8)>>
+                         13::size(8), 10::size(8), 26::size(8), 10::size(8)>>
         @jpg_signature <<255::size(8), 216::size(8)>>
 
         def type(<<@png_signature, rest::binary>>), do: :png
@@ -1241,7 +1241,7 @@ defmodule Kernel.SpecialForms do
       [2, 4, 6, 8]
 
       # A comprehension with two generators
-      iex> for x <- [1, 2], y <- [2, 3], do: x*y
+      iex> for x <- [1, 2], y <- [2, 3], do: x * y
       [2, 3, 4, 6]
 
   Filters can also be given:
@@ -1263,7 +1263,7 @@ defmodule Kernel.SpecialForms do
   need to organize bitstring streams:
 
       iex> pixels = <<213, 45, 132, 64, 76, 32, 76, 0, 0, 234, 32, 15>>
-      iex> for <<r::8, g::8, b::8 <- pixels >>, do: {r, g, b}
+      iex> for <<r::8, g::8, b::8 <- pixels>>, do: {r, g, b}
       [{213, 45, 132}, {64, 76, 32}, {76, 0, 0}, {234, 32, 15}]
 
   Variable assignments inside the comprehension, be it in generators,
@@ -1301,7 +1301,7 @@ defmodule Kernel.SpecialForms do
       iex> opts = %{width: 10, height: 15}
       iex> with {:ok, width} <- Map.fetch(opts, :width),
       ...>      {:ok, height} <- Map.fetch(opts, :height),
-      ...>   do: {:ok, width * height}
+      ...>      do: {:ok, width * height}
       {:ok, 150}
 
   If all clauses match, the `do` block is executed, returning its result.
@@ -1310,14 +1310,14 @@ defmodule Kernel.SpecialForms do
       iex> opts = %{width: 10}
       iex> with {:ok, width} <- Map.fetch(opts, :width),
       ...>      {:ok, height} <- Map.fetch(opts, :height),
-      ...>   do: {:ok, width * height}
+      ...>      do: {:ok, width * height}
       :error
 
   Guards can be used in patterns as well:
 
       iex> users = %{"melany" => "guest", "bob" => :admin}
       iex> with {:ok, role} when not is_binary(role) <- Map.fetch(users, "bob"),
-      ...>   do: {:ok, to_string(role)}
+      ...>      do: {:ok, to_string(role)}
       {:ok, "admin"}
 
   As in `for/1`, variables bound inside `with/1` won't leak;
@@ -1328,7 +1328,7 @@ defmodule Kernel.SpecialForms do
       iex> with {:ok, width} <- Map.fetch(opts, :width),
       ...>      double_width = width * 2,
       ...>      {:ok, height} <- Map.fetch(opts, :height),
-      ...>   do: {:ok, double_width * height}
+      ...>      do: {:ok, double_width * height}
       {:ok, 300}
       iex> width
       nil

lib/elixir/lib/map_set.ex

@@ -93,18 +93,14 @@ defmodule MapSet do
   end
 
   defp do_new([], acc) do
-    acc
-    |> :lists.reverse
-    |> :maps.from_list
+    :maps.from_list(acc)
   end
   defp do_new([item | rest], acc) do
     do_new(rest, [{item, true} | acc])
   end
 
   defp do_new_transform([], _fun, acc) do
-    acc
-    |> :lists.reverse
-    |> :maps.from_list
+    :maps.from_list(acc)
   end
   defp do_new_transform([item | rest], fun, acc) do
     do_new_transform(rest, fun, [{fun.(item), true} | acc])

lib/elixir/lib/module.ex

@@ -808,13 +808,6 @@ defmodule Module do
             "cannot make private macro #{name}/#{arity} overridable, overriding " <>
             "private macros is not supported"
         clause ->
-          {{_def, kind, _line, _file, _check, _location, _defaults}, _clauses} = clause
-
-          # TODO: Remove on v2.0
-          if kind == :defp do
-            IO.warn "making private functions (#{name}/#{arity} in this case) overridable is deprecated"
-          end
-
           neighbours =
             if :elixir_compiler.get_opt(:internal) do
               []
@@ -843,8 +836,6 @@ defmodule Module do
   @doc """
   Puts a module attribute with key and value in the given module.
 
-  It returns the value of the attribute after normalization.
-
   ## Examples
 
       defmodule MyModule do
@@ -852,9 +843,9 @@ defmodule Module do
       end
 
   """
-  @spec put_attribute(module, key :: atom, value :: term) :: term
+  @spec put_attribute(module, key :: atom, value :: term) :: :ok
   def put_attribute(module, key, value) do
-    put_attribute(module, key, value, nil, false)
+    put_attribute(module, key, value, nil, nil)
   end
 
   @doc """
@@ -910,7 +901,7 @@ defmodule Module do
     table = data_table_for(module)
     case :ets.take(table, key) do
       [{_, value, _accumulated? = true, _}] ->
-        :ets.insert(table, {key, [], true, true})
+        :ets.insert(table, {key, [], true, nil})
         value
       [{_, value, _, _}] ->
         value
@@ -920,9 +911,10 @@ defmodule Module do
   end
 
   @doc """
-  Registers an attribute. By registering an attribute, a developer
-  is able to customize how Elixir will store and accumulate the
-  attribute values.
+  Registers an attribute.
+
+  By registering an attribute, a developer is able to customize
+  how Elixir will store and accumulate the attribute values.
 
   ## Options
 
@@ -960,7 +952,7 @@ defmodule Module do
     end
 
     if Keyword.get(opts, :accumulate) do
-      :ets.insert_new(table, {new, [], _accumulated? = true, _read? = true}) ||
+      :ets.insert_new(table, {new, [], _accumulated? = true, _unread_line = nil}) ||
         :ets.update_element(table, new, {3, true})
     end
 
@@ -1031,7 +1023,7 @@ defmodule Module do
         [] -> [value]
       end
 
-    :ets.insert(table, {key, new, true, true})
+    :ets.insert(table, {key, new, true, nil})
   end
 
   @doc false
@@ -1043,7 +1035,7 @@ defmodule Module do
 
     case :ets.lookup(table, key) do
       [{^key, val, _, _}] ->
-        :ets.update_element(table, key, {4, true})
+        :ets.update_element(table, key, {4, nil})
         val
       [] when is_list(stack) ->
         # TODO: Consider raising instead of warning on v2.0 as it usually cascades
@@ -1058,25 +1050,25 @@ defmodule Module do
   @doc false
   # Used internally by Kernel's @.
   # This function is private and must be used only internally.
-  def put_attribute(module, key, value, stack, read?) when is_atom(key) do
+  def put_attribute(module, key, value, stack, unread_line) when is_atom(key) do
     assert_not_compiled!(:put_attribute, module)
     table = data_table_for(module)
     value = preprocess_attribute(key, value)
 
     case :ets.lookup(table, key) do
-      [{^key, {line, <<_::binary>>}, accumulated?, _read?}]
+      [{^key, {line, <<_::binary>>}, accumulated?, _unread_line}]
           when key in [:doc, :typedoc, :moduledoc] and is_list(stack) ->
         IO.warn "redefining @#{key} attribute previously set at line #{line}", stack
-        :ets.insert(table, {key, value, accumulated?, read?})
+        :ets.insert(table, {key, value, accumulated?, unread_line})
 
       [{^key, current, _accumulated? = true, _read?}] ->
-        :ets.insert(table, {key, [value | current], true, read?})
+        :ets.insert(table, {key, [value | current], true, unread_line})
 
       _ ->
-        :ets.insert(table, {key, value, false, read?})
+        :ets.insert(table, {key, value, false, unread_line})
     end
 
-    value
+    :ok
   end
 
   ## Helpers

lib/elixir/lib/option_parser.ex

@@ -30,35 +30,33 @@ defmodule OptionParser do
   Elixir converts switches to underscored atoms, so `--source-path` becomes
   `:source_path`. This is done to better suit Elixir conventions. However, this
   means that switches can't contain underscores and switches that do contain
-  underscores are always returned in the list of invalid options.
+  underscores are always returned in the list of invalid switches.
 
-  Without any options, this function will try to parse all switches in the `argv`.
+  When parsing, it is common to list switches and their expected types:
 
-      iex> OptionParser.parse(["--debug"])
+      iex> OptionParser.parse(["--debug"], switches: [debug: :boolean])
       {[debug: true], [], []}
 
-      iex> OptionParser.parse(["--source", "lib"])
+      iex> OptionParser.parse(["--source", "lib"], switches: [source: :string])
       {[source: "lib"], [], []}
 
-      iex> OptionParser.parse(["--source-path", "lib", "test/enum_test.exs", "--verbose"])
+      iex> OptionParser.parse(["--source-path", "lib", "test/enum_test.exs", "--verbose"],
+      ...>                    switches: [source_path: :string, verbose: :boolean])
       {[source_path: "lib", verbose: true], ["test/enum_test.exs"], []}
 
-  Switches followed by a value will be assigned the value, as a string.
-  Switches without an argument, like `--debug` in the examples above, will
-  automatically be set to `true`.
+  We will explore the valid switches and operation modes of option parser below.
 
   ## Options
 
   The following options are supported:
 
     * `:switches` or `:strict` - see the "Switch definitions" section below
+    * `:allow_nonexistent_atoms` - see the "Parsing dynamic switches" section below
     * `:aliases` - see the "Aliases" section below
 
   ## Switch definitions
 
-  Often it is better to explicitly list the known
-  switches and their formats. The switches can be specified via one of two
-  options:
+  Switches can be specified via one of two options:
 
     * `:switches` - defines some switches and their types. This function
       still attempts to parse switches that are not in this list.
@@ -89,20 +87,20 @@ defmodule OptionParser do
     * `:float` - parses the value as a float
     * `:string` - parses the value as a string
 
-  If a switch can't be parsed according to the given type, it is returned
-  in the invalid options list.
+  If a switch can't be parsed according to the given type, it is
+  returned in the invalid options list.
 
   ### Modifiers
 
   Switches can be specified with modifiers, which change how
   they behave. The following modifiers are supported:
 
-    * `:keep` - keeps duplicated items instead of overriding them; works with
-      all types except `:count`. Specifying `switch_name: :keep` assumes the
-      type of `:switch_name` will be `:string`.
+    * `:keep` - keeps duplicated items instead of overriding them;
+      works with all types except `:count`. Specifying `switch_name: :keep`
+      assumes the type of `:switch_name` will be `:string`.
 
-  Note that if you want to use `:keep` with a type other than `:string`, use a list
-  as the type for the switch. For example: `[foo: [:integer, :keep]]`.
+  To use `:keep` with a type other than `:string`, use a list as the type
+  for the switch. For example: `[foo: [:integer, :keep]]`.
 
   ### Negation switches
 
@@ -112,6 +110,46 @@ defmodule OptionParser do
       iex> OptionParser.parse(["--no-op", "path/to/file"], switches: [op: :boolean])
       {[op: false], ["path/to/file"], []}
 
+  ### Parsing dynamic switches
+
+  `OptionParser` also includes a dynamic mode where it will attempt to parse
+  switches dynamically. Such can be done by not specifying the `:switches` or
+  `:strict` option.
+
+      iex> OptionParser.parse(["--debug"])
+      {[debug: true], [], []}
+
+
+  Switches followed by a value will be assigned the value, as a string. Switches
+  without an argument, like `--debug` in the examples above, will automatically be
+  set to `true`.
+
+  Since Elixir converts switches to atoms, the dynamic mode will only parse
+  switches that translates to atoms used by the runtime. Therefore, the code below
+  likely won't parse the given option since the `:option_parser_example` atom is
+  never used anywhere:
+
+      OptionParser.parse(["--option-parser-example"])
+      # Does nothing more...
+
+  However, the code below does since the `:option_parser_example` atom is used
+  at some point later (or earlier) on:
+
+      {opts, _, _} = OptionParser.parse(["--option-parser-example"])
+      opts[:option_parser_example]
+
+  In other words, when using dynamic mode, Elixir will do the correct thing and
+  only parse options that are used by the runtime, ignoring all others. If you
+  would like to parse all switches, regardless if they exist or not, you can
+  force creation of atoms by passing `allow_nonexistent_atoms: true` as option.
+  Such option is useful when you are building command-line applications that
+  receive dynamically-named arguments but must be used with care on long-running
+  systems.
+
+  Switches followed by a value will be assigned the value, as a string.
+  Switches without an argument, like `--debug` in the examples above, will
+  automatically be set to `true`.
+
   ## Aliases
 
   A set of aliases can be specified in the `:aliases` option:
@@ -203,10 +241,12 @@ defmodule OptionParser do
 
   ## Example
 
-      iex> OptionParser.parse_head(["--source", "lib", "test/enum_test.exs", "--verbose"])
+      iex> OptionParser.parse_head(["--source", "lib", "test/enum_test.exs", "--verbose"],
+      ...>                         switches: [source: :string, verbose: :boolean])
       {[source: "lib"], ["test/enum_test.exs", "--verbose"], []}
 
-      iex> OptionParser.parse_head(["--verbose", "--source", "lib", "test/enum_test.exs", "--unlock"])
+      iex> OptionParser.parse_head(["--verbose", "--source", "lib", "test/enum_test.exs", "--unlock"],
+      ...>                         switches: [source: :string, verbose: :boolean, unlock: :boolean])
       {[verbose: true, source: "lib"], ["test/enum_test.exs", "--unlock"], []}
 
   """
@@ -226,10 +266,12 @@ defmodule OptionParser do
 
   ## Examples
 
-      iex> OptionParser.parse_head!(["--source", "lib", "path/to/file", "--verbose"])
+      iex> OptionParser.parse_head!(["--source", "lib", "path/to/file", "--verbose"],
+      ...>                         switches: [source: :string, verbose: :boolean])
       {[source: "lib"], ["path/to/file", "--verbose"]}
 
-      iex> OptionParser.parse_head!(["--number", "lib", "test/enum_test.exs", "--verbose"], strict: [number: :integer])
+      iex> OptionParser.parse_head!(["--number", "lib", "test/enum_test.exs", "--verbose"],
+      ...>                          strict: [number: :integer])
       ** (OptionParser.ParseError) 1 error found!
       --number : Expected type integer, got "lib"
 
@@ -251,8 +293,8 @@ defmodule OptionParser do
     {Enum.reverse(opts), Enum.reverse(args), Enum.reverse(invalid)}
   end
 
-  defp do_parse(argv, {aliases, switches, strict?} = config, opts, args, invalid, all?) do
-    case next(argv, aliases, switches, strict?) do
+  defp do_parse(argv, {aliases, switches, strict?, allow_nonexistent_atoms?} = config, opts, args, invalid, all?) do
+    case next(argv, aliases, switches, strict?, allow_nonexistent_atoms?) do
       {:ok, option, value, rest} ->
         # the option exists and it was successfully parsed
         kinds = List.wrap Keyword.get(switches, option)
@@ -299,7 +341,6 @@ defmodule OptionParser do
     * `{:error, rest}` - there are no switches at the head of the given `argv`
 
   """
-
   @spec next(argv, options) ::
         {:ok, key :: atom, value :: term, argv} |
         {:invalid, String.t, String.t | nil, argv} |
@@ -307,35 +348,35 @@ defmodule OptionParser do
         {:error, argv}
 
   def next(argv, opts \\ []) when is_list(argv) and is_list(opts) do
-    {aliases, switches, strict?} = compile_config(opts)
-    next(argv, aliases, switches, strict?)
+    {aliases, switches, strict?, allow_nonexistent_atoms?} = compile_config(opts)
+    next(argv, aliases, switches, strict?, allow_nonexistent_atoms?)
   end
 
-  defp next([], _aliases, _switches, _strict?) do
+  defp next([], _aliases, _switches, _strict?, _allow_nonexistent_atoms?) do
     {:error, []}
   end
 
-  defp next(["--" | _] = argv, _aliases, _switches, _strict?) do
+  defp next(["--" | _] = argv, _aliases, _switches, _strict?, _allow_nonexistent_atoms?) do
     {:error, argv}
   end
 
-  defp next(["-" | _] = argv, _aliases, _switches, _strict?) do
+  defp next(["-" | _] = argv, _aliases, _switches, _strict?, _allow_nonexistent_atoms?) do
     {:error, argv}
   end
 
-  defp next(["- " <> _ | _] = argv, _aliases, _switches, _strict?) do
+  defp next(["- " <> _ | _] = argv, _aliases, _switches, _strict?, _allow_nonexistent_atoms?) do
     {:error, argv}
   end
 
   # Handles --foo or --foo=bar
-  defp next(["--" <> option | rest], _aliases, switches, strict?) do
+  defp next(["--" <> option | rest], _aliases, switches, strict?, allow_nonexistent_atoms?) do
     {option, value} = split_option(option)
-    tagged = tag_option(option, switches)
-    do_next(tagged, value, "--" <> option, rest, switches, strict?)
+    tagged = tag_option(option, switches, allow_nonexistent_atoms?)
+    do_next(tagged, value, "--" <> option, rest, switches, strict?, allow_nonexistent_atoms?)
   end
 
   # Handles -a, -abc, -abc=something
-  defp next(["-" <> option | rest] = argv, aliases, switches, strict?) do
+  defp next(["-" <> option | rest] = argv, aliases, switches, strict?, allow_nonexistent_atoms?) do
     {option, value} = split_option(option)
     original = "-" <> option
 
@@ -345,26 +386,26 @@ defmodule OptionParser do
       String.contains?(option, ["-", "_"]) ->
         {:undefined, original, value, rest}
       String.length(option) > 1 ->
-        key = get_option_key(option)
+        key = get_option_key(option, allow_nonexistent_atoms?)
         option_key = aliases[key]
         if key && option_key do
           IO.warn "multi-letter aliases are deprecated, got: #{inspect(key)}"
-          do_next({:default, option_key}, value, original, rest, switches, strict?)
+          do_next({:default, option_key}, value, original, rest, switches, strict?, allow_nonexistent_atoms?)
         else
-          next(expand_multiletter_alias(option, value) ++ rest, aliases, switches, strict?)
+          next(expand_multiletter_alias(option, value) ++ rest, aliases, switches, strict?, allow_nonexistent_atoms?)
         end
       true ->
         # We have a regular one-letter alias here
-        tagged = tag_oneletter_alias(option, aliases)
-        do_next(tagged, value, original, rest, switches, strict?)
+        tagged = tag_oneletter_alias(option, aliases, allow_nonexistent_atoms?)
+        do_next(tagged, value, original, rest, switches, strict?, allow_nonexistent_atoms?)
     end
   end
 
-  defp next(argv, _aliases, _switches, _strict?) do
+  defp next(argv, _aliases, _switches, _strict?, _allow_nonexistent_atoms?) do
     {:error, argv}
   end
 
-  defp do_next(tagged, value, original, rest, switches, strict?) do
+  defp do_next(tagged, value, original, rest, switches, strict?, allow_nonexistent_atoms?) do
     if strict? and not option_defined?(tagged, switches) do
       {:undefined, original, value, rest}
     else
@@ -489,6 +530,7 @@ defmodule OptionParser do
 
   defp compile_config(opts) do
     aliases = opts[:aliases] || []
+    allow_nonexistent_atoms? = opts[:allow_nonexistent_atoms] || false
 
     {switches, strict?} = cond do
       opts[:switches] && opts[:strict] ->
@@ -501,7 +543,7 @@ defmodule OptionParser do
         {[], false}
     end
 
-    {aliases, switches, strict?}
+    {aliases, switches, strict?, allow_nonexistent_atoms?}
   end
 
   defp validate_option(value, kinds) do
@@ -552,27 +594,27 @@ defmodule OptionParser do
     end
   end
 
-  defp tag_option("no-" <> option = original, switches) do
+  defp tag_option("no-" <> option = original, switches, allow_nonexistent_atoms?) do
     cond do
-      (negated = get_option_key(option)) && :boolean in List.wrap(switches[negated]) ->
+      (negated = get_option_key(option, allow_nonexistent_atoms?)) && :boolean in List.wrap(switches[negated]) ->
         {:negated, negated}
-      option_key = get_option_key(original) ->
+      option_key = get_option_key(original, allow_nonexistent_atoms?) ->
         {:default, option_key}
       true ->
         :unknown
     end
   end
 
-  defp tag_option(option, _switches) do
-    if option_key = get_option_key(option) do
+  defp tag_option(option, _switches, allow_nonexistent_atoms?) do
+    if option_key = get_option_key(option, allow_nonexistent_atoms?) do
       {:default, option_key}
     else
       :unknown
     end
   end
 
-  defp tag_oneletter_alias(alias, aliases) when is_binary(alias) do
-    if option_key = aliases[to_existing_key(alias)] do
+  defp tag_oneletter_alias(alias, aliases, allow_nonexistent_atoms?) when is_binary(alias) do
+    if option_key = aliases[to_existing_key(alias, allow_nonexistent_atoms?)] do
       {:default, option_key}
     else
       :unknown
@@ -662,13 +704,15 @@ defmodule OptionParser do
   defp to_underscore(<<>>, acc),
     do: acc
 
-  def get_option_key(option) do
+  def get_option_key(option, allow_nonexistent_atoms?) do
     if string = to_underscore(option) do
-      to_existing_key(string)
+      to_existing_key(string, allow_nonexistent_atoms?)
     end
   end
 
-  defp to_existing_key(option) do
+  defp to_existing_key(option, true),
+    do: String.to_atom(option)
+  defp to_existing_key(option, false) do
     try do
       String.to_existing_atom(option)
     rescue
@@ -702,7 +746,8 @@ defmodule OptionParser do
   end
 
   defp get_type(option, opts, types) do
-    key = option |> String.trim_leading("-") |> get_option_key()
+    allow_nonexistent_atoms? = opts[:allow_nonexistent_atoms] || false
+    key = option |> String.trim_leading("-") |> get_option_key(allow_nonexistent_atoms?)
 
     if option_key = opts[:aliases][key] do
       types[option_key]

lib/elixir/lib/path.ex

@@ -428,10 +428,10 @@ defmodule Path do
   end
 
   @doc """
-  Joins a list of strings.
+  Joins a list of paths.
 
-  This function should be used to convert a list of strings to a path.
-  Note that any trailing slash is removed on join.
+  This function should be used to convert a list of paths to a path.
+  Note that any trailing slash is removed when joining.
 
   ## Examples
 
@@ -445,11 +445,11 @@ defmodule Path do
       "/foo/bar"
 
   """
-  @spec join([t]) :: binary
+  @spec join(nonempty_list(t)) :: binary
   def join([name1, name2 | rest]), do:
     join([join(name1, name2) | rest])
   def join([name]), do:
-    name
+    IO.chardata_to_string(name)
 
   @doc """
   Joins two paths.

lib/elixir/lib/record.ex

@@ -316,6 +316,7 @@ defmodule Record do
       Keyword.keyword?(args) ->
         create(atom, fields, args, caller)
       true ->
+        fields = Macro.escape(fields)
         case Macro.expand(args, caller) do
           {:{}, _, [^atom | list]} when length(list) == length(fields) ->
             record = List.to_tuple([atom | list])

lib/elixir/lib/regex.ex

@@ -1,14 +1,13 @@
 defmodule Regex do
   @moduledoc ~S"""
-  Provides regular expressions for Elixir. Built on top of Erlang's `:re`
-  module.
+  Provides regular expressions for Elixir.
 
-  As the `:re` module, Regex is based on PCRE
-  (Perl Compatible Regular Expressions). More information can be
-  found in the [`:re` module documentation](http://www.erlang.org/doc/man/re.html).
+  Regex is based on PCRE (Perl Compatible Regular Expressions) and
+  built on top of Erlang's `:re` module. More information can be found
+  in the [`:re` module documentation](http://www.erlang.org/doc/man/re.html).
 
-  Regular expressions in Elixir can be created using `Regex.compile!/2`
-  or using the special form with [`~r`](Kernel.html#sigil_r/2) or [`~R`](Kernel.html#sigil_R/2):
+  Regular expressions in Elixir can be created using the sigils
+  [`~r`](Kernel.html#sigil_r/2) or [`~R`](Kernel.html#sigil_R/2):
 
       # A simple regular expressions that matches foo anywhere in the string
       ~r/foo/
@@ -16,9 +15,38 @@ defmodule Regex do
       # A regular expression with case insensitive and Unicode options
       ~r/foo/iu
 
+  Regular expressions created via sigils are pre-compiled and stored
+  in the `.beam` file. Notice this may be a problem if you are precompiling
+  Elixir, see the "Precompilation" section for more information.
+
   A Regex is represented internally as the `Regex` struct. Therefore,
   `%Regex{}` can be used whenever there is a need to match on them.
 
+  Keep in mind it is not guaranteed two regular expressions from the
+  same source are equal, for example:
+
+      ~r/(?<foo>.)(?<bar>.)/ == ~r/(?<foo>.)(?<bar>.)/
+
+  may return `true` or `false` depending on your machine, endianess,
+  available optimizations and others. You can, however, retrieve the source
+  of a compiled regular expression by accessing the `source` field, and then
+  compare those directly:
+
+      ~r/(?<foo>.)(?<bar>.)/.source == ~r/(?<foo>.)(?<bar>.)/.source
+
+  ## Precompilation
+
+  Regular expressions built with sigil are precompiled and stored in `.beam`
+  files. This may be a problem if you are precompiling Elixir to run in
+  different OTP releases, as OTP releases may update the underlying regular
+  expression engine at any time.
+
+  For such reasons, we always recomend precompiling Elixir projects using
+  the OTP version meant to run in production. In case cross-compilation is
+  really necessary, you can manually invoke `Regex.recompile/1` or `Regex.
+  recompile!/1` to perform a runtime version check and recompile the regex
+  if necessary.
+
   ## Modifiers
 
   The modifiers available when creating a Regex are:
@@ -78,7 +106,7 @@ defmodule Regex do
 
   """
 
-  defstruct re_pattern: nil, source: "", opts: ""
+  defstruct re_pattern: nil, source: "", opts: "", re_version: ""
 
   @type t :: %__MODULE__{re_pattern: term, source: binary, opts: binary}
 
@@ -91,7 +119,7 @@ defmodule Regex do
 
   The given options can either be a binary with the characters
   representing the same regex options given to the `~r` sigil,
-  or a list of options, as expected by the Erlang's [`:re` module](http://www.erlang.org/doc/man/re.html).
+  or a list of options, as expected by the Erlang's `:re` module.
 
   It returns `{:ok, regex}` in case of success,
   `{:error, reason}` otherwise.
@@ -106,40 +134,84 @@ defmodule Regex do
 
   """
   @spec compile(binary, binary | [term]) :: {:ok, t} | {:error, any}
-  def compile(source, options \\ "")
+  def compile(source, options \\ "") do
+    compile(source, options, version())
+  end
 
-  def compile(source, options) when is_binary(options) do
+  defp compile(source, options, version) when is_binary(options) do
     case translate_options(options, []) do
       {:error, rest} ->
         {:error, {:invalid_option, rest}}
 
       translated_options ->
-        compile(source, translated_options, options)
+        compile(source, translated_options, options, version)
     end
   end
 
-  def compile(source, options) when is_list(options) do
-    compile(source, options, "")
+  defp compile(source, options, version) when is_list(options) do
+    compile(source, options, "", version)
   end
 
-  defp compile(source, opts, doc_opts) when is_binary(source) do
+  defp compile(source, opts, doc_opts, version) when is_binary(source) do
     case :re.compile(source, opts) do
       {:ok, re_pattern} ->
-        {:ok, %Regex{re_pattern: re_pattern, source: source, opts: doc_opts}}
+        {:ok, %Regex{re_pattern: re_pattern, re_version: version, source: source, opts: doc_opts}}
       error ->
         error
     end
   end
 
   @doc """
-  Compiles the regular expression according to the given options.
-  Fails with `Regex.CompileError` if the regex cannot be compiled.
+  Compiles the regular expression and raises `Regex.CompileError` in case of errors.
   """
   @spec compile!(binary, binary | [term]) :: t
   def compile!(source, options \\ "") do
     case compile(source, options) do
       {:ok, regex} -> regex
-      {:error, {reason, at}} -> raise Regex.CompileError, message: "#{reason} at position #{at}"
+      {:error, {reason, at}} -> raise Regex.CompileError, "#{reason} at position #{at}"
+    end
+  end
+
+  @doc """
+  Recompiles the existing regular expression if necessary.
+
+  This checks the version stored in the regular expression
+  and recompiles the regex in case of version mismatch.
+  """
+  @spec recompile(t) :: t
+  def recompile(%Regex{} = regex) do
+    version = version()
+
+    # We use Map.get/3 by choice to support old regexes versions.
+    case Map.get(regex, :re_version, :error) do
+      ^version ->
+        {:ok, regex}
+      _ ->
+        %{source: source, opts: opts} = regex
+        compile(source, opts, version)
+    end
+  end
+
+  @doc """
+  Recompiles the existing regular expression and raises `Regex.CompileError` in case of errors.
+  """
+  @spec recompile!(t) :: t
+  def recompile!(regex) do
+    case recompile(regex) do
+      {:ok, regex} -> regex
+      {:error, {reason, at}} -> raise Regex.CompileError, "#{reason} at position #{at}"
+    end
+  end
+
+  @doc """
+  Returns the version of the underlying Regex engine.
+  """
+  # TODO: No longer check for function_exported? on OTP 20+.
+  def version do
+    if function_exported?(:re, :version, 0) do
+      :re.version()
+    else
+      "8.33 2013-05-29"
     end
   end
 
@@ -629,9 +701,6 @@ defmodule Regex do
     [get_index(string, h) | get_indexes(string, t, arity - 1)]
   end
 
-  {:ok, pattern} = :re.compile(~S"[.^$*+?()\[\]{}\\\|\s#]", [:unicode])
-  @escape_pattern pattern
-
   @doc ~S"""
   Escapes a string to be literally matched in a regex.
 
@@ -646,9 +715,15 @@ defmodule Regex do
   """
   @spec escape(String.t) :: String.t
   def escape(string) when is_binary(string) do
-    :re.replace(string, @escape_pattern, "\\\\&", [:global, {:return, :binary}])
+    escape(string, [])
   end
 
+  for char <- '.^$*+?()[]{}|#-\\\t\n\v\f\r\s' do
+    defp escape(<<unquote(char), rest::binary>>, acc), do: escape(rest, [acc, ?\\, unquote(char)])
+  end
+  defp escape(<<char, rest::binary>>, acc), do: escape(rest, [acc, char])
+  defp escape(<<>>, acc), do: IO.iodata_to_binary(acc)
+
   # Helpers
 
   @doc false

lib/elixir/lib/set.ex

@@ -9,7 +9,9 @@ defmodule Set do
   @type values :: [ value ]
   @type t :: map
 
-  # TODO: Deprecate every function by 1.4
+  # TODO: Remove by 2.0
+  # (hard-deprecated in elixir_dispatch)
+
   defmacrop target(set) do
     quote do
       case unquote(set) do

lib/elixir/lib/stream.ex

@@ -789,16 +789,18 @@ defmodule Stream do
         do_after(after_fun, user_acc)
         :erlang.raise(kind, reason, stacktrace)
     else
-      {:suspended, [val], next} ->
-        do_transform_user(val, user_acc, user, fun, :cont, next, inner_acc, inner, after_fun)
-      {_, [val]} ->
-        do_transform_user(val, user_acc, user, fun, :halt, next, inner_acc, inner, after_fun)
-      {_, []} ->
-        do_transform(user_acc, user, fun, :halt, next, inner_acc, inner, after_fun)
+      {:suspended, vals, next} ->
+        do_transform_user(:lists.reverse(vals), user_acc, user, fun, :cont, next, inner_acc, inner, after_fun)
+      {_, vals} ->
+        do_transform_user(:lists.reverse(vals), user_acc, user, fun, :halt, next, inner_acc, inner, after_fun)
     end
   end
 
-  defp do_transform_user(val, user_acc, user, fun, next_op, next, inner_acc, inner, after_fun) do
+  defp do_transform_user([], user_acc, user, fun, next_op, next, inner_acc, inner, after_fun) do
+    do_transform(user_acc, user, fun, next_op, next, inner_acc, inner, after_fun)
+  end
+
+  defp do_transform_user([val | vals], user_acc, user, fun, next_op, next, inner_acc, inner, after_fun) do
     user.(val, user_acc)
   catch
     kind, reason ->
@@ -808,20 +810,20 @@ defmodule Stream do
       :erlang.raise(kind, reason, stacktrace)
   else
     {[], user_acc} ->
-      do_transform(user_acc, user, fun, next_op, next, inner_acc, inner, after_fun)
+      do_transform_user(vals, user_acc, user, fun, next_op, next, inner_acc, inner, after_fun)
     {list, user_acc} when is_list(list) ->
-      do_list_transform(user_acc, user, fun, next_op, next, inner_acc, inner,
+      do_list_transform(vals, user_acc, user, fun, next_op, next, inner_acc, inner,
                         &Enumerable.List.reduce(list, &1, fun), after_fun)
     {:halt, user_acc} ->
       next.({:halt, []})
       do_after(after_fun, user_acc)
       {:halted, elem(inner_acc, 1)}
     {other, user_acc} ->
-      do_enum_transform(user_acc, user, fun, next_op, next, inner_acc, inner,
+      do_enum_transform(vals, user_acc, user, fun, next_op, next, inner_acc, inner,
                         &Enumerable.reduce(other, &1, inner), after_fun)
   end
 
-  defp do_list_transform(user_acc, user, fun, next_op, next, inner_acc, inner, reduce, after_fun) do
+  defp do_list_transform(vals, user_acc, user, fun, next_op, next, inner_acc, inner, reduce, after_fun) do
     try do
       reduce.(inner_acc)
     catch
@@ -832,17 +834,17 @@ defmodule Stream do
         :erlang.raise(kind, reason, stacktrace)
     else
       {:done, acc} ->
-        do_transform(user_acc, user, fun, next_op, next, {:cont, acc}, inner, after_fun)
+        do_transform_user(vals, user_acc, user, fun, next_op, next, {:cont, acc}, inner, after_fun)
       {:halted, acc} ->
         next.({:halt, []})
         do_after(after_fun, user_acc)
         {:halted, acc}
       {:suspended, acc, c} ->
-        {:suspended, acc, &do_list_transform(user_acc, user, fun, next_op, next, &1, inner, c, after_fun)}
+        {:suspended, acc, &do_list_transform(vals, user_acc, user, fun, next_op, next, &1, inner, c, after_fun)}
     end
   end
 
-  defp do_enum_transform(user_acc, user, fun, next_op, next, {op, inner_acc}, inner, reduce, after_fun) do
+  defp do_enum_transform(vals, user_acc, user, fun, next_op, next, {op, inner_acc}, inner, reduce, after_fun) do
     try do
       reduce.({op, [:outer | inner_acc]})
     catch
@@ -855,15 +857,15 @@ defmodule Stream do
       # Only take into account outer halts when the op is not halt itself.
       # Otherwise, we were the ones wishing to halt, so we should just stop.
       {:halted, [:outer | acc]} when op != :halt ->
-        do_transform(user_acc, user, fun, next_op, next, {:cont, acc}, inner, after_fun)
+        do_transform_user(vals, user_acc, user, fun, next_op, next, {:cont, acc}, inner, after_fun)
       {:halted, [_ | acc]} ->
         next.({:halt, []})
         do_after(after_fun, user_acc)
         {:halted, acc}
       {:done, [_ | acc]} ->
-        do_transform(user_acc, user, fun, next_op, next, {:cont, acc}, inner, after_fun)
+        do_transform_user(vals, user_acc, user, fun, next_op, next, {:cont, acc}, inner, after_fun)
       {:suspended, [_ | acc], c} ->
-        {:suspended, acc, &do_enum_transform(user_acc, user, fun, next_op, next, &1, inner, c, after_fun)}
+        {:suspended, acc, &do_enum_transform(vals, user_acc, user, fun, next_op, next, &1, inner, c, after_fun)}
     end
   end
 
@@ -902,7 +904,6 @@ defmodule Stream do
 
   @doc false
   def uniq(enum, fun) do
-    IO.warn "Stream.uniq/2 is deprecated, use Stream.uniq_by/2 instead"
     uniq_by(enum, fun)
   end
 

lib/elixir/lib/task.ex

@@ -56,28 +56,27 @@ defmodule Task do
 
   ## Supervised tasks
 
-  It is also possible to spawn a task under a supervisor
-  with `start_link/1` and `start_link/3`:
-
-      Task.start_link(fn -> IO.puts "ok" end)
-
-  Such tasks can be mounted in your supervision tree as:
+  It is also possible to spawn a task under a supervisor:
 
       import Supervisor.Spec
 
       children = [
+        #
         worker(Task, [fn -> IO.puts "ok" end])
       ]
 
+  Internally the supervisor will invoke `Task.start_link/1`.
+
   Since these tasks are supervised and not directly linked to
   the caller, they cannot be awaited on. Note `start_link/1`,
   unlike `async/1`, returns `{:ok, pid}` (which is
   the result expected by supervision trees).
 
   By default, most supervision strategies will try to restart
-  a worker after it exits regardless of the reason. If you design the
-  task to terminate normally (as in the example with `IO.puts/2` above),
-  consider passing `restart: :transient` in the options to `Supervisor.Spec.worker/3`.
+  a worker after it exits regardless of the reason. If you design
+  the task to terminate normally (as in the example with `IO.puts/2`
+  above), consider passing `restart: :transient` in the options
+  to `Supervisor.Spec.worker/3`.
 
   ## Dynamically supervised tasks
 
@@ -286,16 +285,18 @@ defmodule Task do
   Returns a stream that runs the given `module`, `function` and `args`
   concurrently on each item in `enumerable`.
 
-  Each item will be appended to the given `args` and processed by its
-  own task. The tasks will be linked to the current process similar to
-  `async/3`.
+  Each item will be prepended to the given `args` and processed by its
+  own task. The tasks will be linked to an intermediate process that is
+  then linked to the current process. This means a failure in a task
+  terminates the current process and a failure in the current process
+  terminates all tasks.
 
   When streamed, each task will emit `{:ok, val}` upon successful
   completion or `{:exit, val}` if the caller is trapping exits. Results
   are emitted in the same order as the original `enumerable`.
 
   The level of concurrency can be controlled via the `:max_concurrency`
-  option and defaults to `System.schedulers_online/1`. The timeout
+  option and defaults to `System.schedulers_online/0`. The timeout
   can also be given as option and defaults to 5000 and it defaults to
   the maximum amount of time to wait without a task reply.
 
@@ -307,9 +308,10 @@ defmodule Task do
   ## Options
 
     * `:max_concurrency` - sets the maximum number of tasks to run
-      at the same time. Defaults to `System.schedulers_online/1`.
+      at the same time. Defaults to `System.schedulers_online/0`.
     * `:timeout` - the maximum amount of time to wait without
       receiving a task reply (across all running tasks).
+      Defaults to `5000`.
 
   ## Example
 

lib/elixir/lib/task/supervised.ex

@@ -10,29 +10,21 @@ defmodule Task.Supervised do
     {:ok, :proc_lib.spawn_link(__MODULE__, :noreply, [info, fun])}
   end
 
-  def start_link(caller, link, info, fun) do
-    {:ok, spawn_link(caller, link, info, fun)}
+  def start_link(caller, monitor, info, fun) do
+    {:ok, spawn_link(caller, monitor, info, fun)}
   end
 
-  def spawn_link(caller, link \\ :nolink, info, fun) do
-    :proc_lib.spawn_link(__MODULE__, :reply, [caller, link, info, fun])
+  def spawn_link(caller, monitor \\ :nomonitor, info, fun) do
+    :proc_lib.spawn_link(__MODULE__, :reply, [caller, monitor, info, fun])
   end
 
-  def reply(caller, link, info, mfa) do
+  def reply(caller, monitor, info, mfa) do
     initial_call(mfa)
-    case link do
-      :link ->
-        try do
-          Process.link(caller)
-        catch
-          :error, :noproc ->
-            exit({:shutdown, :noproc})
-        end
-        reply(caller, nil, @ref_timeout, info, mfa)
+    case monitor do
       :monitor ->
         mref = Process.monitor(caller)
         reply(caller, mref, @ref_timeout, info, mfa)
-      :nolink ->
+      :nomonitor ->
         reply(caller, nil, :infinity, info, mfa)
     end
   end
@@ -157,37 +149,41 @@ defmodule Task.Supervised do
     parent = self()
 
     # Start a process responsible for translating down messages.
-    {monitor_pid, monitor_ref} = spawn_monitor(fn -> stream_monitor(parent) end)
+    {:trap_exit, trap_exit} =
+      Process.info(self(), :trap_exit)
+    {monitor_pid, monitor_ref} =
+      Process.spawn(fn -> stream_monitor(parent, mfa, spawn, trap_exit) end, [:link, :monitor])
+
     send(monitor_pid, {parent, monitor_ref})
 
     stream_reduce(acc, max_concurrency, 0, 0, %{}, next,
-                  reducer, mfa, spawn, monitor_pid, monitor_ref, timeout)
+                  reducer, monitor_pid, monitor_ref, timeout)
   end
 
-  defp stream_reduce({:halt, acc}, _max, _spawned, _delivered, waiting, next,
-                     _reducer, _mfa, _spawn, monitor_pid, monitor_ref, timeout) do
+  defp stream_reduce({:halt, acc}, _max, _spawned, _delivered, _waiting, next,
+                     _reducer, monitor_pid, monitor_ref, timeout) do
+    stream_close(monitor_pid, monitor_ref, timeout)
     is_function(next) && next.({:halt, []})
-    stream_close(waiting, monitor_pid, monitor_ref, timeout)
     {:halted, acc}
   end
 
   defp stream_reduce({:suspend, acc}, max, spawned, delivered, waiting, next,
-                     reducer, mfa, spawn, monitor_pid, monitor_ref, timeout) do
+                     reducer, monitor_pid, monitor_ref, timeout) do
     {:suspended, acc, &stream_reduce(&1, max, spawned, delivered, waiting, next,
-                                     reducer, mfa, spawn, monitor_pid, monitor_ref, timeout)}
+                                     reducer, monitor_pid, monitor_ref, timeout)}
   end
 
   # All spawned, all delivered, next is done.
-  defp stream_reduce({:cont, acc}, _max, spawned, delivered, waiting, next,
-                     _reducer, _mfa, _spawn, monitor_pid, monitor_ref, timeout)
+  defp stream_reduce({:cont, acc}, _max, spawned, delivered, _waiting, next,
+                     _reducer, monitor_pid, monitor_ref, timeout)
        when spawned == delivered and next == :done do
-    stream_close(waiting, monitor_pid, monitor_ref, timeout)
+    stream_close(monitor_pid, monitor_ref, timeout)
     {:done, acc}
   end
 
-  # No more tasks to spawn because max == 0 or next is done.
+  # No more tasks to spawned because max == 0 or next is done.
   defp stream_reduce({:cont, acc}, max, spawned, delivered, waiting, next,
-                     reducer, mfa, spawn, monitor_pid, monitor_ref, timeout)
+                     reducer, monitor_pid, monitor_ref, timeout)
        when max == 0
        when next == :done do
     receive do
@@ -195,7 +191,7 @@ defmodule Task.Supervised do
         %{^position => {pid, :running}} = waiting
         waiting = Map.put(waiting, position, {pid, {:ok, value}})
         stream_reduce({:cont, acc}, max, spawned, delivered, waiting, next,
-                      reducer, mfa, spawn, monitor_pid, monitor_ref, timeout)
+                      reducer, monitor_pid, monitor_ref, timeout)
       {:down, {^monitor_ref, position}, reason} ->
         waiting =
           case waiting do
@@ -203,53 +199,53 @@ defmodule Task.Supervised do
             %{^position => {_, :running}} -> Map.put(waiting, position, {nil, {:exit, reason}})
           end
         stream_deliver({:cont, acc}, max + 1, spawned, delivered, waiting, next,
-                       reducer, mfa, spawn, monitor_pid, monitor_ref, timeout)
+                       reducer, monitor_pid, monitor_ref, timeout)
       {:DOWN, ^monitor_ref, _, ^monitor_pid, reason} ->
-        stream_cleanup_inbox(monitor_ref)
+        stream_cleanup_inbox(monitor_pid, monitor_ref)
         exit({reason, {__MODULE__, :stream, [timeout]}})
     after
       timeout ->
-        stream_close(waiting, monitor_pid, monitor_ref, timeout)
+        stream_close(monitor_pid, monitor_ref, timeout)
         exit({:timeout, {__MODULE__, :stream, [timeout]}})
     end
   end
 
   defp stream_reduce({:cont, acc}, max, spawned, delivered, waiting, next,
-                     reducer, mfa, spawn, monitor_pid, monitor_ref, timeout) do
+                     reducer, monitor_pid, monitor_ref, timeout) do
     try do
       next.({:cont, []})
     catch
       kind, reason ->
         stacktrace = System.stacktrace
-        stream_close(waiting, monitor_pid, monitor_ref, timeout)
+        stream_close(monitor_pid, monitor_ref, timeout)
         :erlang.raise(kind, reason, stacktrace)
     else
       {:suspended, [value], next} ->
-        waiting = stream_spawn(value, spawned, waiting, mfa, spawn, monitor_pid, monitor_ref)
+        waiting = stream_spawn(value, spawned, waiting, monitor_pid, monitor_ref, timeout)
         stream_reduce({:cont, acc}, max - 1, spawned + 1, delivered, waiting, next,
-                      reducer, mfa, spawn, monitor_pid, monitor_ref, timeout)
+                      reducer, monitor_pid, monitor_ref, timeout)
       {_, [value]} ->
-        waiting = stream_spawn(value, spawned, waiting, mfa, spawn, monitor_pid, monitor_ref)
+        waiting = stream_spawn(value, spawned, waiting, monitor_pid, monitor_ref, timeout)
         stream_reduce({:cont, acc}, max - 1, spawned + 1, delivered, waiting, :done,
-                      reducer, mfa, spawn, monitor_pid, monitor_ref, timeout)
+                      reducer, monitor_pid, monitor_ref, timeout)
       {_, []} ->
         stream_reduce({:cont, acc}, max, spawned, delivered, waiting, :done,
-                      reducer, mfa, spawn, monitor_pid, monitor_ref, timeout)
+                      reducer, monitor_pid, monitor_ref, timeout)
     end
   end
 
   defp stream_deliver({:suspend, acc}, max, spawned, delivered, waiting, next,
-                      reducer, mfa, spawn, monitor_pid, monitor_ref, timeout) do
+                      reducer, monitor_pid, monitor_ref, timeout) do
     {:suspended, acc, &stream_deliver(&1, max, spawned, delivered, waiting, next,
-                                      reducer, mfa, spawn, monitor_pid, monitor_ref, timeout)}
+                                      reducer, monitor_pid, monitor_ref, timeout)}
   end
   defp stream_deliver({:halt, acc}, max, spawned, delivered, waiting, next,
-                      reducer, mfa, spawn, monitor_pid, monitor_ref, timeout) do
+                      reducer, monitor_pid, monitor_ref, timeout) do
     stream_reduce({:halt, acc}, max, spawned, delivered, waiting, next,
-                  reducer, mfa, spawn, monitor_pid, monitor_ref, timeout)
+                  reducer, monitor_pid, monitor_ref, timeout)
   end
   defp stream_deliver({:cont, acc}, max, spawned, delivered, waiting, next,
-                      reducer, mfa, spawn, monitor_pid, monitor_ref, timeout) do
+                      reducer, monitor_pid, monitor_ref, timeout) do
     case waiting do
       %{^delivered => {nil, reply}} ->
         try do
@@ -258,38 +254,44 @@ defmodule Task.Supervised do
           kind, reason ->
             stacktrace = System.stacktrace
             is_function(next) && next.({:halt, []})
-            stream_close(waiting, monitor_pid, monitor_ref, timeout)
+            stream_close(monitor_pid, monitor_ref, timeout)
             :erlang.raise(kind, reason, stacktrace)
         else
           pair ->
             stream_deliver(pair, max, spawned, delivered + 1, Map.delete(waiting, delivered), next,
-                           reducer, mfa, spawn, monitor_pid, monitor_ref, timeout)
+                           reducer, monitor_pid, monitor_ref, timeout)
         end
       %{} ->
         stream_reduce({:cont, acc}, max, spawned, delivered, waiting, next,
-                      reducer, mfa, spawn, monitor_pid, monitor_ref, timeout)
+                      reducer, monitor_pid, monitor_ref, timeout)
     end
   end
 
-  defp stream_close(waiting, monitor_pid, monitor_ref, timeout) do
-    for {_, {pid, _}} <- waiting do
-      pid && Process.unlink(pid)
-    end
-    send(monitor_pid, {:down, monitor_ref})
+  defp stream_close(monitor_pid, monitor_ref, timeout) do
+    send(monitor_pid, {:stop, monitor_ref})
     receive do
-      {:DOWN, ^monitor_ref, _, _, {:shutdown, ^monitor_ref}} ->
+      {:DOWN, ^monitor_ref, _, _, :normal} ->
+        stream_cleanup_inbox(monitor_pid, monitor_ref)
         :ok
       {:DOWN, ^monitor_ref, _, _, reason} ->
+        stream_cleanup_inbox(monitor_pid, monitor_ref)
         exit({reason, {__MODULE__, :stream, [timeout]}})
     end
-    stream_cleanup_inbox(monitor_ref)
+  end
+
+  defp stream_cleanup_inbox(monitor_pid, monitor_ref) do
+    receive do
+      {:EXIT, ^monitor_pid, _} -> stream_cleanup_inbox(monitor_ref)
+    after
+      0 -> stream_cleanup_inbox(monitor_ref)
+    end
   end
 
   defp stream_cleanup_inbox(monitor_ref) do
     receive do
       {{^monitor_ref, _}, _} ->
         stream_cleanup_inbox(monitor_ref)
-      {:DOWN, {^monitor_ref, _}, _} ->
+      {:down, {^monitor_ref, _}, _} ->
         stream_cleanup_inbox(monitor_ref)
     after
       0 ->
@@ -300,38 +302,47 @@ defmodule Task.Supervised do
   defp stream_mfa({mod, fun, args}, arg), do: {mod, fun, [arg | args]}
   defp stream_mfa(fun, arg), do: {:erlang, :apply, [fun, [arg]]}
 
-  defp stream_spawn(value, spawned, waiting, mfa, spawn, monitor_pid, monitor_ref) do
-    owner = self()
-    {type, pid} = spawn.(owner, stream_mfa(mfa, value))
-    send(monitor_pid, {:up, owner, monitor_ref, spawned, type, pid})
-    Map.put(waiting, spawned, {pid, :running})
+  defp stream_spawn(value, spawned, waiting, monitor_pid, monitor_ref, timeout) do
+    send(monitor_pid, {:spawn, spawned, value})
+
+    receive do
+      {:spawned, {^monitor_ref, ^spawned}, pid} ->
+        send(pid, {self(), {monitor_ref, spawned}})
+        Map.put(waiting, spawned, {pid, :running})
+      {:DOWN, ^monitor_ref, _, ^monitor_pid, reason} ->
+        stream_cleanup_inbox(monitor_pid, monitor_ref)
+        exit({reason, {__MODULE__, :stream, [timeout]}})
+    end
   end
 
-  defp stream_monitor(parent_pid) do
+  defp stream_monitor(parent_pid, mfa, spawn, trap_exit) do
+    Process.flag(:trap_exit, trap_exit)
     parent_ref = Process.monitor(parent_pid)
     receive do
       {^parent_pid, monitor_ref} ->
-        stream_monitor(parent_pid, parent_ref, monitor_ref, %{})
+        stream_monitor(parent_pid, parent_ref, mfa, spawn, monitor_ref, %{})
       {:DOWN, ^parent_ref, _, _, reason} ->
         exit(reason)
     end
   end
 
-  defp stream_monitor(parent_pid, parent_ref, monitor_ref, counters) do
+  defp stream_monitor(parent_pid, parent_ref, mfa, spawn, monitor_ref, counters) do
     receive do
-      {:up, owner, ^monitor_ref, counter, type, pid} ->
+      {:spawn, counter, value} ->
+        {type, pid} = spawn.(parent_pid, stream_mfa(mfa, value))
         ref = Process.monitor(pid)
-        send(pid, {owner, {monitor_ref, counter}})
+        send(parent_pid, {:spawned, {monitor_ref, counter}, pid})
         counters = Map.put(counters, ref, {counter, type, pid})
-        stream_monitor(parent_pid, parent_ref, monitor_ref, counters)
-      {:down, ^monitor_ref} ->
+        stream_monitor(parent_pid, parent_ref, mfa, spawn, monitor_ref, counters)
+      {:stop, ^monitor_ref} ->
+        Process.flag(:trap_exit, true)
         for {ref, {_counter, _, pid}} <- counters do
           Process.exit(pid, :kill)
           receive do
             {:DOWN, ^ref, _, _, _} -> :ok
           end
         end
-        exit({:shutdown, monitor_ref})
+        exit(:normal)
       {:DOWN, ^parent_ref, _, _, reason} ->
         for {_ref, {_counter, :link, pid}} <- counters do
           Process.exit(pid, reason)
@@ -340,7 +351,9 @@ defmodule Task.Supervised do
       {:DOWN, ref, _, _, reason} ->
         {{counter, _, _}, counters} = Map.pop(counters, ref)
         send(parent_pid, {:down, {monitor_ref, counter}, reason})
-        stream_monitor(parent_pid, parent_ref, monitor_ref, counters)
+        stream_monitor(parent_pid, parent_ref, mfa, spawn, monitor_ref, counters)
+      {:EXIT, _, _} ->
+        stream_monitor(parent_pid, parent_ref, mfa, spawn, monitor_ref, counters)
     end
   end
 end

lib/elixir/lib/task/supervisor.ex

@@ -106,7 +106,7 @@ defmodule Task.Supervisor do
   """
   @spec async_nolink(Supervisor.supervisor, module, atom, [term]) :: Task.t
   def async_nolink(supervisor, module, fun, args) do
-    do_async(supervisor, :monitor, module, fun, args)
+    do_async(supervisor, :nolink, module, fun, args)
   end
 
   @doc """
@@ -122,7 +122,7 @@ defmodule Task.Supervisor do
   are emitted in the same order as the original `enumerable`.
 
   The level of concurrency can be controlled via the `:max_concurrency`
-  option and defaults to `System.schedulers_online/1`. The timeout
+  option and defaults to `System.schedulers_online/0`. The timeout
   can also be given as option and defaults to 5000 and it defaults to
   the maximum amount of time to wait without a task reply.
 
@@ -133,9 +133,10 @@ defmodule Task.Supervisor do
   ## Options
 
     * `:max_concurrency` - sets the maximum number of tasks to run
-      at the same time. Defaults to `System.schedulers_online/1`.
+      at the same time. Defaults to `System.schedulers_online/0`.
     * `:timeout` - the maximum amount of time to wait without
       receiving a task reply (across all running tasks).
+      Defaults to `5000`.
 
   ## Examples
 
@@ -181,7 +182,7 @@ defmodule Task.Supervisor do
         Enumerable.t
   def async_stream_nolink(supervisor, enumerable, module, function, args, options \\ [])
       when is_atom(module) and is_atom(function) and is_list(args) do
-    build_stream(supervisor, :monitor, enumerable, {module, function, args}, options)
+    build_stream(supervisor, :nolink, enumerable, {module, function, args}, options)
   end
 
   @doc """
@@ -197,7 +198,7 @@ defmodule Task.Supervisor do
   @spec async_stream_nolink(Supervisor.supervisor, Enumerable.t, (term -> term), Keyword.t) ::
         Enumerable.t
   def async_stream_nolink(supervisor, enumerable, fun, options \\ []) when is_function(fun, 1) do
-    build_stream(supervisor, :monitor, enumerable, fun, options)
+    build_stream(supervisor, :nolink, enumerable, fun, options)
   end
 
   @doc """
@@ -250,7 +251,7 @@ defmodule Task.Supervisor do
 
   defp do_async(supervisor, link_type, module, fun, args) do
     owner = self()
-    args = [owner, link_type, get_info(owner), {module, fun, args}]
+    args = [owner, :monitor, get_info(owner), {module, fun, args}]
     {:ok, pid} = Supervisor.start_child(supervisor, args)
     if link_type == :link, do: Process.link(pid)
     ref = Process.monitor(pid)
@@ -258,12 +259,12 @@ defmodule Task.Supervisor do
     %Task{pid: pid, ref: ref, owner: owner}
   end
 
-  defp build_stream(supervisor, type, enumerable, fun, options) do
+  defp build_stream(supervisor, link_type, enumerable, fun, options) do
     &Task.Supervised.stream(enumerable, &1, &2, fun, options, fn owner, mfa ->
-      args = [owner, type, get_info(owner), mfa]
+      args = [owner, :monitor, get_info(owner), mfa]
       {:ok, pid} = Supervisor.start_child(supervisor, args)
-      if type == :link, do: Process.link(pid)
-      {type, pid}
+      if link_type == :link, do: Process.link(pid)
+      {link_type, pid}
     end)
   end
 end

lib/elixir/lib/uri.ex

@@ -400,8 +400,8 @@ defmodule URI do
   def parse(%URI{} = uri), do: uri
 
   def parse(string) when is_binary(string) do
-    # From http://tools.ietf.org/html/rfc3986#appendix-B
-    regex = ~r/^(([a-z][a-z0-9\+\-\.]*):)?(\/\/([^\/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))?/i
+    # From https://tools.ietf.org/html/rfc3986#appendix-B
+    regex = Regex.recompile!(~r/^(([a-z][a-z0-9\+\-\.]*):)?(\/\/([^\/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))?/i)
     parts = nillify(Regex.run(regex, string))
 
     destructure [_, _, scheme, _, authority, path, _, query, _, fragment], parts
@@ -419,7 +419,8 @@ defmodule URI do
 
   # Split an authority into its userinfo, host and port parts.
   defp split_authority(string) do
-    components = Regex.run(~r/(^(.*)@)?(\[[a-zA-Z0-9:.]*\]|[^:]*)(:(\d*))?/, string || "")
+    regex = Regex.recompile!(~r/(^(.*)@)?(\[[a-zA-Z0-9:.]*\]|[^:]*)(:(\d*))?/)
+    components = Regex.run(regex, string || "")
 
     destructure [_, _, userinfo, host, _, port], nillify(components)
     host = if host, do: host |> String.trim_leading("[") |> String.trim_trailing("]")
@@ -473,6 +474,9 @@ defmodule URI do
   def merge(_base, %URI{scheme: rel_scheme} = rel) when rel_scheme != nil do
     rel
   end
+  def merge(base, %URI{authority: authority} = rel) when authority != nil do
+    %{rel | scheme: base.scheme}
+  end
   def merge(%URI{} = base, %URI{path: rel_path} = rel) when rel_path in ["", nil] do
     %{base | query: rel.query || base.query, fragment: rel.fragment}
   end

lib/elixir/lib/version.ex

@@ -238,9 +238,10 @@ defmodule Version do
   @spec parse(String.t) :: {:ok, t} | :error
   def parse(string) when is_binary(string) do
     case Version.Parser.parse_version(string) do
-      {:ok, {major, minor, patch, pre}} ->
+      {:ok, {major, minor, patch, pre, build_parts}} ->
+        build = if build_parts == [], do: nil, else: Enum.join(build_parts, "")
         version = %Version{major: major, minor: minor, patch: patch,
-                       pre: pre, build: get_build(string)}
+                           pre: pre, build: build}
         {:ok, version}
      :error ->
        :error
@@ -312,23 +313,13 @@ defmodule Version do
 
   defp to_matchable(string, allow_pre?) do
     case Version.Parser.parse_version(string) do
-      {:ok, {major, minor, patch, pre}} ->
+      {:ok, {major, minor, patch, pre, _build_parts}} ->
         {major, minor, patch, pre, allow_pre?}
       :error ->
         raise InvalidVersionError, message: string
     end
   end
 
-  defp get_build(string) do
-    case Regex.run(~r/\+([^\s]+)$/, string) do
-      nil ->
-        nil
-
-      [_, build] ->
-        build
-    end
-  end
-
   defmodule Parser.DSL do
     @moduledoc false
 
@@ -396,67 +387,92 @@ defmodule Version do
       Enum.filter(Enum.reverse(acc), &(&1 != :' '))
     end
 
-    @version_regex ~r/^
-      (\d+)                 # major
-      (?:\.(\d+))?          # minor
-      (?:\.(\d+))?          # patch
-      (?:\-([\d\w\.\-]+))?  # pre
-      (?:\+([\d\w\.\-]+))?  # build
-      $/x
-
     @spec parse_requirement(String.t) :: {:ok, term} | :error
     def parse_requirement(source) do
       lexed = lexer(source, [])
       to_matchspec(lexed)
     end
 
-    defp nillify(""), do: nil
-    defp nillify(o),  do: o
-
     @spec parse_version(String.t) :: {:ok, Version.matchable} | :error
     def parse_version(string, approximate? \\ false) when is_binary(string) do
-      if parsed = Regex.run(@version_regex, string) do
-        destructure [_, major, minor, patch, pre], parsed
-        patch = nillify(patch)
-        pre   = nillify(pre)
+      destructure [version_with_pre, build], String.split(string, "+", parts: 2)
+      destructure [version, pre], String.split(version_with_pre, "-", parts: 2)
+      destructure [major, minor, patch], String.split(version, ".")
 
-        if is_nil(minor) or (is_nil(patch) and not approximate?) do
-          :error
-        else
-          major = String.to_integer(major)
-          minor = String.to_integer(minor)
-          patch = patch && String.to_integer(patch)
+      with {:ok, major} <- require_digits(major),
+           {:ok, minor} <- require_digits(minor),
+           {:ok, patch} <- maybe_patch(patch, approximate?),
+           {:ok, pre_parts} <- optional_dot_separated(pre),
+           {:ok, pre_parts} <- convert_parts_to_integer(pre_parts, []),
+           {:ok, build_parts} <- optional_dot_separated(build) do
+        {:ok, {major, minor, patch, pre_parts, build_parts}}
+      else
+        _other -> :error
+      end
+    end
 
-          case parse_pre(pre) do
-            {:ok, pre} ->
-              {:ok, {major, minor, patch, pre}}
-            :error ->
-              :error
-          end
-        end
+    defp require_digits(nil), do: :error
+    defp require_digits(string) do
+      if leading_zero?(string), do: :error, else: parse_digits(string, "")
+    end
+
+    defp leading_zero?(<<?0, _, _::binary>>), do: true
+    defp leading_zero?(_), do: false
+
+    defp parse_digits(<<char, rest::binary>>, acc) when char in ?0..?9,
+      do: parse_digits(rest, <<acc::binary, char>>)
+    defp parse_digits(<<>>, acc) when byte_size(acc) > 0,
+      do: {:ok, String.to_integer(acc)}
+    defp parse_digits(_, _acc),
+      do: :error
+
+    defp maybe_patch(patch, approximate?)
+    defp maybe_patch(nil, true), do: {:ok, nil}
+    defp maybe_patch(patch, _), do: require_digits(patch)
+
+    defp optional_dot_separated(nil), do: {:ok, []}
+    defp optional_dot_separated(string) do
+      parts = String.split(string, ".")
+      if Enum.all?(parts, &(&1 != "" and valid_identifier?(&1))) do
+        {:ok, parts}
       else
         :error
       end
     end
 
-    defp parse_pre(nil), do: {:ok, []}
-    defp parse_pre(pre), do: parse_pre(String.split(pre, "."), [])
-
-    defp parse_pre([piece | t], acc) do
-      cond do
-        piece =~ ~r/^(0|[1-9][0-9]*)$/ ->
-          parse_pre(t, [String.to_integer(piece) | acc])
-        piece =~ ~r/^[0-9]*$/ ->
-          :error
-        true ->
-          parse_pre(t, [piece | acc])
+    defp convert_parts_to_integer([part | rest], acc) do
+      case parse_digits(part, "") do
+        {:ok, integer} ->
+          if leading_zero?(part) do
+            :error
+          else
+            convert_parts_to_integer(rest, [integer | acc])
+          end
+        :error ->
+          convert_parts_to_integer(rest, [part | acc])
       end
     end
 
-    defp parse_pre([], acc) do
+    defp convert_parts_to_integer([], acc) do
       {:ok, Enum.reverse(acc)}
     end
 
+    defp valid_identifier?(<<char, rest::binary>>)
+         when char in ?0..?9
+         when char in ?a..?z
+         when char in ?A..?Z
+         when char == ?- do
+      valid_identifier?(rest)
+    end
+
+    defp valid_identifier?(<<>>) do
+      true
+    end
+
+    defp valid_identifier?(_other) do
+      false
+    end
+
     defp valid_requirement?([]), do: false
     defp valid_requirement?([a | next]), do: valid_requirement?(a, next)
 
@@ -574,7 +590,7 @@ defmodule Version do
 
     defp parse_condition(version, approximate? \\ false) do
       case parse_version(version, approximate?) do
-        {:ok, version} -> version
+        {:ok, {major, minor, patch, pre, _build}} -> {major, minor, patch, pre}
         :error -> throw :invalid_matchspec
       end
     end

lib/elixir/pages/Writing Documentation.md

@@ -4,10 +4,10 @@ Elixir treats documentation as a first-class citizen. This means documentation s
 
 ## Markdown
 
-Elixir documentation is written using Markdown. There are plenty of guides on Markdown online, we recommend the ones available at GitHub as a getting started point:
+Elixir documentation is written using Markdown. There are plenty of guides on Markdown online, we recommend the ones available on GitHub as a getting started point:
 
-  * https://help.github.com/articles/markdown-basics/
-  * https://help.github.com/articles/github-flavored-markdown/
+  * [https://help.github.com/articles/markdown-basics/](https://help.github.com/articles/markdown-basics/)
+  * [https://help.github.com/articles/markdown-basics/](https://help.github.com/articles/github-flavored-markdown/)
 
 ## Module Attributes
 
@@ -34,7 +34,7 @@ Documentation in Elixir is usually attached to module attributes. Let's see an e
       end
     end
 
-The `@moduledoc` attribute is used to add documentation to the module. `@doc` is used before a function to provide documentation for it. Besides the attributes above, `@typedoc` can also be used to attach documentation to types defined as part of typespecs.
+The `@moduledoc` attribute is used to add documentation to the module. `@doc` is used before a function to provide documentation for the function that follows. Besides the attributes above, `@typedoc` can also be used to attach documentation to types defined as part of typespecs.
 
 ## Function Arguments
 
@@ -55,11 +55,11 @@ The compiler will infer this argument as `map`. Sometimes the inference will be
 
 When writing documentation:
 
-  * Keep the first paragraph of the documentation concise and simple, typically one-line. Tools like [ExDoc](https://github.com/elixir-lang/ex_doc/) use the first line to generate a summary.
+  * Keep the first paragraph of the documentation concise and simple, typically one line. Tools like [ExDoc](https://github.com/elixir-lang/ex_doc/) use the first line to generate a summary.
 
   * Reference modules by their full name.
 
-  Markdown uses backticks (`` ` ``) to quote code. Elixir builds on top of that to automatically generate links when module or function names are referenced. For this reason, always use full module names. If you have a module called `MyApp.Hello`, always reference it as `` `MyApp.Hello` `` and never as `` `Hello` ``.
+Markdown uses backticks (`` ` ``) to quote code. Elixir builds on top of that to automatically generate links when module or function names are referenced. For this reason, always use full module names. If you have a module called `MyApp.Hello`, always reference it as `` `MyApp.Hello` `` and never as `` `Hello` ``.
 
   * Reference functions by name and arity if they are local, as in `` `world/1` ``, or by module, name and arity if pointing to an external module: `` `MyApp.Hello.world/1` ``.
 
@@ -73,21 +73,21 @@ When writing documentation:
 
 ## Doctests
 
-We recommend that developers include examples in their documentation, often under their own `## Examples` heading. To ensure examples do not get out of date, Elixir's test framework (ExUnit) provides a feature called doctests that allows developers to test the examples in their documentation. Doctests work by parsing out code samples starting with `iex>` from the documentation. You can read more about it at `ExUnit.DocTest`.
+We recommend that developers include examples in their documentation, often under their own `## Examples` heading. To ensure examples do not get out of date, Elixir's test framework (ExUnit) provides a feature called doctests that allows developers to test the examples in their documentation. Doctests work by parsing out code samples starting with `iex>` from the documentation. You can read more about it in the documentation for `ExUnit.DocTest`.
 
-Notice doctests have limitations. When you cannot doctest a function, because it relies on state or side-effects, we recommend developers include examples directly without the `iex>` prompt.
+Note that doctests have limitations. When you cannot doctest a function, because it relies on state or side-effects, we recommend developers include examples directly without the `iex>` prompt.
 
-## Documentation != Comments
+## Documentation is not Comments
 
 Elixir treats documentation and code comments as different concepts. Documentation is for users of your Application Programming Interface (API), be it your co-worker or your future self. Modules and functions must always be documented if they are part of your API.
 
-Code comments are for developers reading the code. They are useful to mark improvements, leave notes for developers reading the code (for example, you decided not to call a function due to a bug in a library) and so forth.
+Code comments are for developers reading the code. They are useful to mark improvements, leave notes for developers reading the code (for example, you decided not to call a function due to a bug in a library), and so on.
 
 In other words: documentation is required, code comments are optional.
 
 ## Hiding Internal Modules and Functions
 
-Besides the modules and functions libraries provide as part of their public interface, libraries may also implement important functionality that is not part of their API. While these modules and functions can be accessed, they are meant to be internal to the library and thus should not have documentation for end users.
+Besides the modules and functions that libraries provide as part of their public interface, libraries may also implement important functionality that is not part of their API. While these modules and functions can be accessed, they are meant to be internal to the library and thus should not have documentation for end users.
 
 Luckily, Elixir allows developers to hide modules and functions from the documentation. For example, one common practice for documenting internal behaviour is to set the `@moduledoc` attribute to `false` while documenting each function:
 
@@ -102,7 +102,7 @@ Luckily, Elixir allows developers to hide modules and functions from the documen
       end
     end
 
-Similarly, developers can add `@doc false` to functions they do not want to be publicly exposed:
+Similarly, developers can add `@doc false` to functions they do not want to show up in the documentation:
 
     defmodule MyApp.Sample do
       @doc false
@@ -113,7 +113,7 @@ However, keep in mind that adding `@doc false` does not make the function privat
 
   * Move the undocumented function to a module with `@moduledoc false`, like `MyApp.Hidden`, ensuring the function won't be accidentally exposed or imported. Remember you can use `@moduledoc false` to hide a whole module and still document each function with `@doc`. Tools will still ignore the module.
 
-  * Start the function name with one or two underscores, for example, `__add__/2`, and add `@doc false`. The compiler does not import functions with leading underscores and they hint to anyone reading the code of their intended private usage.
+  * Start the function name with one or two underscores, for example, `__add__/2`, and add `@doc false`. The compiler does not import functions with leading underscores and they suggest to anyone reading the code that they're meant to be used privately.
 
 ## Documenting Private Functions
 
@@ -121,6 +121,6 @@ Elixir warns if a private function has a `@doc` attribute and discards its conte
 
 Private functions may still need internal documentation for maintainers, though. That can be accomplished with code comments.
 
-## Code.get_docs/2
+## Retrieving Documentation
 
 Elixir stores documentation inside pre-defined chunks in the bytecode. It can be accessed from Elixir by using the `Code.get_docs/2` function. This also means documentation is only accessed when required and not when modules are loaded by the Virtual Machine. The only downside is that modules defined in-memory, like the ones defined in IEx, cannot have their documentation accessed as they do not have their bytecode written to disk.

lib/elixir/src/elixir.erl

@@ -47,6 +47,11 @@ start(_Type, _Args) ->
       erlang:halt(1)
   end,
 
+  %% We need to make sure the re module is preloaded
+  %% to make function_exported checks on it fast.
+  %% TODO: Remove this once we support OTP 20+.
+  _ = code:ensure_loaded(re),
+
   case code:ensure_loaded(?system) of
     {module, ?system} ->
       Endianness = ?system:endianness(),

lib/elixir/src/elixir_def.erl

@@ -271,11 +271,11 @@ export(Kind, {Name, Arity}) when Kind == def; Kind == defp ->
 function_for_stored_definition(Ann, {Name, Arity}, Clauses) ->
   {function, Ann, Name, Arity, Clauses}.
 
-add_definition(_Line, nil, Body, {Head, Tail}) ->
+add_definition(_Ann, nil, Body, {Head, Tail}) ->
   {[Body | Head], Tail};
-add_definition(Line, Location, Body, {Head, Tail}) ->
-  {Head,
-   [{attribute, Line, file, Location}, Body | Tail]}.
+add_definition(Ann, Location, Body, {Head, Tail}) ->
+  FileAnn = erl_anno:set_generated(false, Ann),
+  {Head, [{attribute, FileAnn, file, Location}, Body | Tail]}.
 
 default_function_for(Kind, Name, {clause, Ann, Args, _Guards, _Exprs} = Clause)
     when Kind == defmacro; Kind == defmacrop ->
@@ -303,12 +303,12 @@ store_each(Check, Kind, File, Location, Module, Defaults, {function, Ann, Name,
 
   {FinalAnn, FinalLocation, FinalDefaults} = case ets:take(Defs, {def, Tuple}) of
     [{_, StoredKind, StoredAnn, StoredFile, StoredCheck,
-        StoredLocation, {StoredDefaults, LastHasBody, LastDefaults}}] ->
+        _StoredLocation, {StoredDefaults, LastHasBody, LastDefaults}}] ->
       check_valid_kind(Ann, File, Name, Arity, Kind, StoredKind),
       (Check and StoredCheck) andalso
         check_valid_clause(Ann, File, Name, Arity, Kind, Data, StoredAnn, StoredFile),
       check_valid_defaults(Ann, File, Name, Arity, Kind, Defaults, StoredDefaults, LastDefaults, LastHasBody),
-      {StoredAnn, StoredLocation, {max(Defaults, StoredDefaults), HasBody, Defaults}};
+      {StoredAnn, Location, {max(Defaults, StoredDefaults), HasBody, Defaults}};
     [] ->
       {Ann, Location, {Defaults, HasBody, Defaults}}
   end,

lib/elixir/src/elixir_dispatch.erl

@@ -353,5 +353,8 @@ deprecation('Elixir.HashSet', _, _) ->
   "use the MapSet module instead";
 deprecation('Elixir.Dict', _, _) ->
   "use the Map module for working with maps or the Keyword module for working with keyword lists";
+deprecation('Elixir.Set', _, _) ->
+  "use the MapSet module for working with sets";
+
 deprecation(_, _, _) ->
   false.

lib/elixir/src/elixir_errors.erl

@@ -140,6 +140,11 @@ handle_file_warning(_, _File, {_Line, v3_kernel, bad_call}) -> ok;
 %% We handle unused local warnings ourselves
 handle_file_warning(_, _File, {_Line, erl_lint, {unused_function, _}}) -> ok;
 
+%% Properly format keys using inspect.
+handle_file_warning(_, File, {Line, v3_core, {map_key_repeated, Key}}) ->
+  Message = io_lib:format("key ~ts will be overridden in map", ['Elixir.Kernel':inspect(Key)]),
+  warn(Line, File, Message);
+
 %% Make no_effect clauses pretty
 handle_file_warning(_, File, {Line, sys_core_fold, {no_effect, {erlang, F, A}}}) ->
   {Fmt, Args} = case erl_internal:comp_op(F, A) of

lib/elixir/src/elixir_for.erl

@@ -114,7 +114,9 @@ translate_gen(_Meta, Left, Right, T, S) ->
   SF = SL#elixir_scope{extra=S#elixir_scope.extra, extra_guards=nil},
 
   {TT, {TFilters, TS}} = translate_filters(T, SF),
-  Guards = ExtraGuards ++ translate_guards(TLeftGuards) ++ TFilters,
+
+  %% The list of guards is kept in reverse order
+  Guards = TFilters ++ translate_guards(TLeftGuards) ++ ExtraGuards,
   {TLeft, TRight, Guards, TT, TS}.
 
 translate_guards([]) ->

lib/elixir/src/elixir_map.erl

@@ -12,8 +12,9 @@ expand_map(Meta, Args, E) ->
   validate_kv(Meta, EArgs, Args, E),
   {{'%{}', Meta, EArgs}, EA}.
 
-expand_struct(Meta, Left, Right, #{context := Context} = E) ->
-  {[ELeft, ERight], EE} = elixir_exp:expand_args([Left, Right], E),
+expand_struct(Meta, Left, {'%{}', MapMeta, MapArgs}, #{context := Context} = E) ->
+  CleanArgs = clean_struct_key_from_map_args(Meta, MapArgs, E),
+  {[ELeft, ERight], EE} = elixir_exp:expand_args([Left, {'%{}', MapMeta, CleanArgs}], E),
 
   case validate_struct(ELeft, Context) of
     true when is_atom(ELeft) ->
@@ -38,14 +39,24 @@ expand_struct(Meta, Left, Right, #{context := Context} = E) ->
       false -> Meta
     end,
 
-  case ERight of
-    {'%{}', _, _} -> ok;
-    _ -> compile_error(Meta, ?m(E, file),
-           "expected struct to be followed by a map, got: ~ts",
-           ['Elixir.Macro':to_string(ERight)])
-  end,
+  {{'%', EMeta, [ELeft, ERight]}, EE};
+expand_struct(Meta, _Left, Right, E) ->
+  compile_error(Meta, ?m(E, file), "expected struct to be followed by a map, got: ~ts",
+                ['Elixir.Macro':to_string(Right)]).
 
-  {{'%', EMeta, [ELeft, ERight]}, EE}.
+clean_struct_key_from_map_args(Meta, [{'|', PipeMeta, [Left, MapAssocs]}], E) ->
+  [{'|', PipeMeta, [Left, clean_struct_key_from_map_assocs(Meta, MapAssocs, E)]}];
+clean_struct_key_from_map_args(Meta, MapAssocs, E) ->
+  clean_struct_key_from_map_assocs(Meta, MapAssocs, E).
+
+clean_struct_key_from_map_assocs(Meta, Assocs, E) ->
+  case lists:keytake('__struct__', 1, Assocs) of
+    {value, _, CleanAssocs} ->
+      elixir_errors:warn(?line(Meta), ?m(E, file), "key :__struct__ is ignored when using structs"),
+      CleanAssocs;
+    false ->
+      Assocs
+  end.
 
 validate_struct({'^', _, [{Var, _, Ctx}]}, match) when is_atom(Var), is_atom(Ctx) -> true;
 validate_struct({Var, _Meta, Ctx}, match) when is_atom(Var), is_atom(Ctx) -> true;
@@ -66,7 +77,7 @@ translate_map(Meta, Args, S) ->
   translate_map(Meta, Assocs, TUpdate, US).
 
 translate_struct(_Meta, Name, {'%{}', MapMeta, Assocs}, S) when is_tuple(Name) ->
-  translate_map(MapMeta, Assocs ++ [{'__struct__', Name}], nil, S);
+  translate_map(MapMeta, [{'__struct__', Name} | Assocs], nil, S);
 
 translate_struct(Meta, Name, {'%{}', MapMeta, Args}, S) ->
   {Assocs, TUpdate, US} = extract_assoc_update(Args, S),
@@ -98,11 +109,11 @@ translate_struct(Meta, Name, {'%{}', MapMeta, Args}, S) ->
         {clause, Generated, [Var], [], [elixir_utils:erl_call(Ann, erlang, error, [Error])]}
       ]}, TS};
     match ->
-      translate_map(MapMeta, Assocs ++ [{'__struct__', Name}], nil, US);
+      translate_map(MapMeta, [{'__struct__', Name} | Assocs], nil, US);
     expand ->
-      Keys = [K || {K, _} <- Assocs],
+      Keys = ['__struct__'] ++ [K || {K, _} <- Assocs],
       {StructAssocs, _} = elixir_quote:escape(maps:to_list(maps:without(Keys, Struct)), false),
-      translate_map(MapMeta, StructAssocs ++ Assocs ++ [{'__struct__', Name}], nil, US)
+      translate_map(MapMeta, [{'__struct__', Name}] ++ StructAssocs ++ Assocs, nil, US)
   end.
 
 %% Helpers

lib/elixir/src/elixir_module.erl

@@ -85,7 +85,7 @@ do_compile(Line, Module, Block, Vars, E) ->
     Forms3 = attributes_form(Line, File, Data, PersistedAttrs, Forms2),
 
     elixir_locals:ensure_no_import_conflict(Line, File, Module, All),
-    warn_unused_attributes(Line, File, Data, PersistedAttrs),
+    warn_unused_attributes(File, Data, PersistedAttrs),
 
     Location = {elixir_utils:characters_to_list(elixir_utils:relative_to_cwd(File)), Line},
 
@@ -141,7 +141,7 @@ build(Line, File, Module, Docs, Lexical) ->
   end,
 
   Data = ets:new(Module, [set, public]),
-  Defs = ets:new(Module, [bag, public]),
+  Defs = ets:new(Module, [duplicate_bag, public]),
   Ref  = elixir_code_server:call({defmodule, self(),
                                  {Module, Data, Defs, Line, File}}),
 
@@ -152,26 +152,26 @@ build(Line, File, Module, Docs, Lexical) ->
     end,
 
   ets:insert(Data, [
-    % {Key, Value, Accumulate?, Read?}
-    {after_compile, [], true, true},
-    {before_compile, [], true, true},
-    {behaviour, [], true, true},
-    {compile, [], true, true},
-    {derive, [], true, true},
-    {dialyzer, [], true, true},
-    {external_resource, [], true, true},
-    {moduledoc, nil, false, true},
-    {on_definition, OnDefinition, true, true},
-    {on_load, [], true, true},
+    % {Key, Value, Accumulate?, UnreadLine}
+    {after_compile, [], true, nil},
+    {before_compile, [], true, nil},
+    {behaviour, [], true, nil},
+    {compile, [], true, nil},
+    {derive, [], true, nil},
+    {dialyzer, [], true, nil},
+    {external_resource, [], true, nil},
+    {moduledoc, nil, false, nil},
+    {on_definition, OnDefinition, true, nil},
+    {on_load, [], true, nil},
 
     % Types
-    {callback, [], true, true},
-    {opaque, [], true, true},
-    {optional_callbacks, [], true, true},
-    {macrocallback, [], true, true},
-    {spec, [], true, true},
-    {type, [], true, true},
-    {typep, [], true, true}
+    {callback, [], true, nil},
+    {opaque, [], true, nil},
+    {optional_callbacks, [], true, nil},
+    {macrocallback, [], true, nil},
+    {spec, [], true, nil},
+    {type, [], true, nil},
+    {typep, [], true, nil}
   ]),
 
   Persisted = [behaviour, on_load, compile, external_resource, dialyzer, vsn],
@@ -425,11 +425,11 @@ check_module_availability(Line, File, Module) ->
       ok
   end.
 
-warn_unused_attributes(Line, File, Data, PersistedAttrs) ->
-  ReservedAttrs = [after_compile, before_compile, moduledoc, on_definition | PersistedAttrs],
-  Keys = ets:select(Data, [{{'$1', '_', '_', false}, [{is_atom, '$1'}], ['$1']}]),
+warn_unused_attributes(File, Data, PersistedAttrs) ->
+  ReservedAttrs = [after_compile, before_compile, moduledoc, on_definition, impl | PersistedAttrs],
+  Keys = ets:select(Data, [{{'$1', '_', '_', '$2'}, [{is_atom, '$1'}, {is_integer, '$2'}], [['$1', '$2']]}]),
   [elixir_errors:form_warn([{line, Line}], File, ?MODULE, {unused_attribute, Key}) ||
-   Key <- Keys, not lists:member(Key, ReservedAttrs)].
+   [Key, Line] <- Keys, not lists:member(Key, ReservedAttrs)].
 
 % __INFO__
 

lib/elixir/src/elixir_quote.erl

@@ -189,12 +189,15 @@ do_quote({'__aliases__', Meta, [H | T]} = Alias, #elixir_quote{aliases_hygiene=t
       Aliases when is_list(Aliases) -> false
     end,
   AliasMeta = keystore(alias, keydelete(counter, Meta), Annotation),
-  do_quote_tuple({'__aliases__', AliasMeta, [H | T]}, Q, E);
+  do_quote_tuple('__aliases__', AliasMeta, [H | T], Q, E);
 
 %% Vars
 
+do_quote({Left, Meta, nil}, #elixir_quote{vars_hygiene=true, imports_hygiene=true} = Q, E) when is_atom(Left) ->
+  do_quote_import(Left, Meta, Q#elixir_quote.context, Q, E);
+
 do_quote({Left, Meta, nil}, #elixir_quote{vars_hygiene=true} = Q, E) when is_atom(Left) ->
-  do_quote_tuple({Left, Meta, Q#elixir_quote.context}, Q, E);
+  do_quote_tuple(Left, Meta, Q#elixir_quote.context, Q, E);
 
 %% Unquote
 
@@ -207,28 +210,11 @@ do_quote({{'.', Meta, [Left, unquote]}, _, [Expr]}, #elixir_quote{unquote=true}
 %% Imports
 
 do_quote({'&', Meta, [{'/', _, [{F, _, C}, A]}] = Args},
-    #elixir_quote{imports_hygiene=true} = Q, E) when is_atom(F), is_integer(A), is_atom(C) ->
+         #elixir_quote{imports_hygiene=true} = Q, E) when is_atom(F), is_integer(A), is_atom(C) ->
   do_quote_fa('&', Meta, Args, F, A, Q, E);
 
 do_quote({Name, Meta, ArgsOrAtom}, #elixir_quote{imports_hygiene=true} = Q, E) when is_atom(Name) ->
-  Arity = case is_atom(ArgsOrAtom) of
-    true  -> 0;
-    false -> length(ArgsOrAtom)
-  end,
-
-  NewMeta = case (keyfind(import, Meta) == false) andalso
-      elixir_dispatch:find_import(Meta, Name, Arity, E) of
-    false ->
-      case (Arity == 1) andalso keyfind(ambiguous_op, Meta) of
-        {ambiguous_op, nil} -> keystore(ambiguous_op, Meta, Q#elixir_quote.context);
-        _ -> Meta
-      end;
-    Receiver ->
-      keystore(import, keystore(context, Meta, Q#elixir_quote.context), Receiver)
-  end,
-
-  Annotated = annotate({Name, NewMeta, ArgsOrAtom}, Q#elixir_quote.context),
-  do_quote_tuple(Annotated, Q, E);
+  do_quote_import(Name, Meta, ArgsOrAtom, Q, E);
 
 do_quote({_, _, _} = Tuple, #elixir_quote{escape=false} = Q, E) ->
   Annotated = annotate(Tuple, Q#elixir_quote.context),
@@ -252,7 +238,7 @@ do_quote(BitString, #elixir_quote{escape=true} = Q, _) when is_bitstring(BitStri
       {BitString, Q};
     Size ->
       <<Bits:Size, Bytes/binary>> = BitString,
-      {{'<<>>', [], [{'::', [], [Bits, Size]}, Bytes]}, Q}
+      {{'<<>>', [], [{'::', [], [Bits, {size, [], [Size]}]}, {'::', [], [Bytes, {binary, [], []}]}]}, Q}
   end;
 
 do_quote(Map, #elixir_quote{escape=true} = Q, E) when is_map(Map) ->
@@ -300,6 +286,26 @@ bad_escape(Arg) ->
                    "The supported values are: lists, tuples, maps, atoms, numbers, bitstrings, ",
                    "PIDs and remote functions in the format &Mod.fun/arity">>).
 
+do_quote_import(Name, Meta, ArgsOrAtom, #elixir_quote{imports_hygiene=true} = Q, E) ->
+  Arity = case is_atom(ArgsOrAtom) of
+    true  -> 0;
+    false -> length(ArgsOrAtom)
+  end,
+
+  NewMeta = case (keyfind(import, Meta) == false) andalso
+      elixir_dispatch:find_import(Meta, Name, Arity, E) of
+    false ->
+      case (Arity == 1) andalso keyfind(ambiguous_op, Meta) of
+        {ambiguous_op, nil} -> keystore(ambiguous_op, Meta, Q#elixir_quote.context);
+        _ -> Meta
+      end;
+    Receiver ->
+      keystore(import, keystore(context, Meta, Q#elixir_quote.context), Receiver)
+  end,
+
+  Annotated = annotate({Name, NewMeta, ArgsOrAtom}, Q#elixir_quote.context),
+  do_quote_tuple(Annotated, Q, E).
+
 do_quote_call(Left, Meta, Expr, Args, Q, E) ->
   All  = [meta(Meta, Q), Left, {unquote, Meta, [Expr]}, Args,
           Q#elixir_quote.context],
@@ -313,13 +319,16 @@ do_quote_fa(Target, Meta, Args, F, A, Q, E) ->
       false    -> Meta;
       Receiver -> keystore(import_fa, Meta, {Receiver, Q#elixir_quote.context})
     end,
-  do_quote_tuple({Target, NewMeta, Args}, Q, E).
+  do_quote_tuple(Target, NewMeta, Args, Q, E).
+
+do_quote_tuple({Left, Meta, Right}, Q, E) ->
+  do_quote_tuple(Left, Meta, Right, Q, E).
 
 % In a def unquote(name)(args) expression name will be an atom literal,
 % thus location: :keep will not have enough information to generate the proper file/line annotation.
 % This alters metadata to force Elixir to show the file to which the definition is added
 % instead of the file where definition is quoted (i.e. we behave the opposite to location: :keep).
-do_quote_tuple({Left, Meta, [{{unquote, _, _}, _, _}, _] = Right}, Q, E) when ?defs(Left) ->
+do_quote_tuple(Left, Meta, [{{unquote, _, _}, _, _}, _] = Right, Q, E) when ?defs(Left) ->
   {TLeft, LQ}  = do_quote(Left, Q, E),
   {[Head, Body], RQ} = do_quote(Right, LQ, E),
   {'{}', [], [HLeft, HMeta, HRight]} = Head,
@@ -327,7 +336,7 @@ do_quote_tuple({Left, Meta, [{{unquote, _, _}, _, _}, _] = Right}, Q, E) when ?d
   NewHead = {'{}', [], [HLeft, NewMeta, HRight]},
   {{'{}', [], [TLeft, meta(Meta, Q), [NewHead, Body]]}, RQ};
 
-do_quote_tuple({Left, Meta, Right}, Q, E) ->
+do_quote_tuple(Left, Meta, Right, Q, E) ->
   {TLeft, LQ}  = do_quote(Left, Q, E),
   {TRight, RQ} = do_quote(Right, LQ, E),
   {{'{}', [], [TLeft, meta(Meta, Q), TRight]}, RQ}.

lib/elixir/test/elixir/exception_test.exs

@@ -263,34 +263,17 @@ defmodule ExceptionTest do
     assert Exception.format_exit(reason) |> String.starts_with?(expected_to_start_with)
   end
 
-  test "format_exit with call with exception" do
-    # Fake reason to prevent error_logger printing to stdout
-    fsm_reason = {%ArgumentError{}, [{:not_a_real_module, :function, 0, []}]}
-    reason = try do
-      :gen_fsm.sync_send_event(spawn(fn() ->
-        Process.sleep(200)
-        exit(fsm_reason)
-      end), :hello)
-    catch
-      :exit, reason -> reason
-    end
-
-    formatted = Exception.format_exit(reason)
-    assert formatted =~ ~r"exited in: :gen_fsm\.sync_send_event\(#PID<\d+\.\d+\.\d+>, :hello\)"
-    assert formatted =~ ~r"\s{4}\*\* \(EXIT\) an exception was raised:\n"
-    assert formatted =~ ~r"\s{8}\*\* \(ArgumentError\) argument error\n"
-    assert formatted =~ ~r"\s{12}:not_a_real_module\.function/0"
-  end
-
   test "format_exit with nested calls" do
+    Process.flag(:trap_exit, true)
     # Fake reason to prevent error_logger printing to stdout
-    event_fun = fn() ->
-      Process.sleep(200)
-      exit(:normal)
-    end
-    server_pid = spawn(fn()-> :gen_event.call(spawn(event_fun), :handler, :hello) end)
+    exit_fun = fn() -> receive do: (_ -> exit(:normal)) end
+    outer_pid =
+      spawn_link(fn()->
+        Process.flag(:trap_exit, true)
+        :gen_event.call(spawn_link(exit_fun), :handler, :hello)
+      end)
     reason = try do
-      :gen_server.call(server_pid, :hi)
+      :gen_server.call(outer_pid, :hi)
     catch
       :exit, reason -> reason
     end
@@ -302,15 +285,17 @@ defmodule ExceptionTest do
   end
 
   test "format_exit with nested calls and exception" do
+    Process.flag(:trap_exit, true)
     # Fake reason to prevent error_logger printing to stdout
-    event_reason = {%ArgumentError{}, [{:not_a_real_module, :function, 0, []}]}
-    event_fun = fn() ->
-      Process.sleep(200)
-      exit(event_reason)
-    end
-    server_pid = spawn(fn()-> :gen_event.call(spawn(event_fun), :handler, :hello) end)
+    exit_reason = {%ArgumentError{}, [{:not_a_real_module, :function, 0, []}]}
+    exit_fun = fn() -> receive do: (_ -> exit(exit_reason)) end
+    outer_pid =
+      spawn_link(fn()->
+        Process.flag(:trap_exit, true)
+        :gen_event.call(spawn_link(exit_fun), :handler, :hello)
+      end)
     reason = try do
-      :gen_server.call(server_pid, :hi)
+      :gen_server.call(outer_pid, :hi)
     catch
       :exit, reason -> reason
     end
@@ -386,8 +371,9 @@ defmodule ExceptionTest do
   end
 
   test "UndefinedFunctionError when the mfa is a macro but require wasn't called" do
+    _ = Code.ensure_loaded(Integer)
     assert %UndefinedFunctionError{module: Integer, function: :is_odd, arity: 1} |> message ==
-           "function Integer.is_odd/1 is undefined or private but Integer defines " <>
+           "function Integer.is_odd/1 is undefined or private. However there is " <>
            "a macro with the same name and arity. Be sure to require Integer if " <>
            "you intend to invoke this macro"
   end

lib/elixir/test/elixir/float_test.exs

@@ -80,6 +80,7 @@ defmodule FloatTest do
     assert Float.ceil(0.32453e-10) === 1.0
     assert Float.ceil(-0.32453e-10) === 0.0
     assert Float.ceil(1.32453e-10) === 1.0
+    assert Float.ceil(0.0) === 0.0
   end
 
   test "ceil with precision" do
@@ -94,6 +95,8 @@ defmodule FloatTest do
 
     assert Float.ceil(12.32453e-20, 2) === 0.01
     assert Float.ceil(-12.32453e-20, 2) === 0.0
+
+    assert Float.ceil(0.0, 2) === 0.0
   end
 
   test "round" do

lib/elixir/test/elixir/gen_server_test.exs

@@ -3,6 +3,8 @@ Code.require_file "test_helper.exs", __DIR__
 defmodule GenServerTest do
   use ExUnit.Case, async: true
 
+  import ExUnit.CaptureLog
+
   defmodule Stack do
     use GenServer
 
@@ -161,4 +163,13 @@ defmodule GenServerTest do
     {:ok, _} = GenServer.start(Stack, [], name: :stack_for_stop)
     assert GenServer.stop(:stack_for_stop, :normal) == :ok
   end
+
+  test "handle_info/2 warning" do
+    {:ok, pid} = GenServer.start(Stack, [])
+    log = capture_log(fn ->
+      send(pid, :foo)
+      GenServer.stop(pid)
+    end)
+    assert log =~ "[warn]  GenServerTest.Stack #{inspect pid} received unexpected message in handle_info/2: :foo"
+  end
 end

lib/elixir/test/elixir/kernel/comprehension_test.exs

@@ -51,6 +51,12 @@ defmodule Kernel.ComprehensionTest do
     assert for(x when x == 3 when x == 7 <- 1..10, do: x) == [3, 7]
   end
 
+  test "for comprehensions with guards and filters" do
+    assert for({var, _} when is_atom(var) <- [{:foo, 1}, {2, :bar}],
+               var = Atom.to_string(var),
+               do: var) == ["foo"]
+  end
+
   test "for comprehensions with map key matching" do
     maps = [%{x: 1}, %{y: 2}, %{x: 3}]
     assert for(%{x: v} <- maps, do: v * 2) == [2, 6]

lib/elixir/test/elixir/kernel/expansion_test.exs

@@ -153,7 +153,7 @@ defmodule Kernel.ExpansionTest do
   end
 
   test "locals: in guards" do
-    assert expand(quote(do: fn pid when :erlang.==(pid, self) -> pid end)) ==
+    assert expand_and_clean(quote(do: fn pid when :erlang.==(pid, self) -> pid end), [:import, :context]) ==
            quote(do: fn pid when :erlang.==(pid, :erlang.self()) -> pid end)
   end
 
@@ -337,22 +337,22 @@ defmodule Kernel.ExpansionTest do
   ## Cond
 
   test "cond: expands each clause" do
-    assert expand_and_clean(quote do: (cond do x = 1 -> x; _ -> x end)) ==
+    assert expand(quote do: (cond do x = 1 -> x; _ -> x end)) ==
            quote do: (cond do x = 1 -> x; _ -> x() end)
   end
 
   test "cond: does not share lexical scope between clauses" do
-    assert expand_and_clean(quote do: (cond do 1 -> import List; 2 -> flatten([1, 2, 3]) end)) ==
+    assert expand(quote do: (cond do 1 -> import List; 2 -> flatten([1, 2, 3]) end)) ==
            quote do: (cond do 1 -> import :"Elixir.List", []; 2 -> flatten([1, 2, 3]) end)
   end
 
   test "cond: does not leaks vars on head" do
-    assert expand_and_clean(quote do: (cond do x = 1 -> x; y = 2 -> y end; :erlang.+(x, y))) ==
+    assert expand(quote do: (cond do x = 1 -> x; y = 2 -> y end; :erlang.+(x, y))) ==
            quote do: (cond do x = 1 -> x; y = 2 -> y end; :erlang.+(x(), y()))
   end
 
   test "cond: leaks vars" do
-    assert expand_and_clean(quote do: (cond do 1 -> x = 1; 2 -> y = 2 end; :erlang.+(x, y))) ==
+    assert expand(quote do: (cond do 1 -> x = 1; 2 -> y = 2 end; :erlang.+(x, y))) ==
            quote do: (cond do 1 -> x = 1; 2 -> y = 2 end; :erlang.+(x, y))
   end
 
@@ -365,27 +365,27 @@ defmodule Kernel.ExpansionTest do
   ## Case
 
   test "case: expands each clause" do
-    assert expand_and_clean(quote do: (case w do x -> x; _ -> x end)) ==
+    assert expand(quote do: (case w do x -> x; _ -> x end)) ==
            quote do: (case w() do x -> x; _ -> x() end)
   end
 
   test "case: does not share lexical scope between clauses" do
-    assert expand_and_clean(quote do: (case w do 1 -> import List; 2 -> flatten([1, 2, 3]) end)) ==
+    assert expand(quote do: (case w do 1 -> import List; 2 -> flatten([1, 2, 3]) end)) ==
            quote do: (case w() do 1 -> import :"Elixir.List", []; 2 -> flatten([1, 2, 3]) end)
   end
 
   test "case: expands guards" do
-    assert expand_and_clean(quote do: (case w do x when x when __ENV__.context -> true end)) ==
+    assert expand(quote do: (case w do x when x when __ENV__.context -> true end)) ==
            quote do: (case w() do x when x when :guard -> true end)
   end
 
   test "case: does not leaks vars on head" do
-    assert expand_and_clean(quote do: (case w do x -> x; y -> y end; :erlang.+(x, y))) ==
+    assert expand(quote do: (case w do x -> x; y -> y end; :erlang.+(x, y))) ==
            quote do: (case w() do x -> x; y -> y end; :erlang.+(x(), y()))
   end
 
   test "case: leaks vars" do
-    assert expand_and_clean(quote do: (case w do x -> x = x; y -> y = y end; :erlang.+(x, y))) ==
+    assert expand(quote do: (case w do x -> x = x; y -> y = y end; :erlang.+(x, y))) ==
            quote do: (case w() do x -> x = x; y -> y = y end; :erlang.+(x, y))
   end
 
@@ -398,32 +398,32 @@ defmodule Kernel.ExpansionTest do
   ## Receive
 
   test "receive: expands each clause" do
-    assert expand_and_clean(quote do: (receive do x -> x; _ -> x end)) ==
+    assert expand(quote do: (receive do x -> x; _ -> x end)) ==
            quote do: (receive do x -> x; _ -> x() end)
   end
 
   test "receive: does not share lexical scope between clauses" do
-    assert expand_and_clean(quote do: (receive do 1 -> import List; 2 -> flatten([1, 2, 3]) end)) ==
+    assert expand(quote do: (receive do 1 -> import List; 2 -> flatten([1, 2, 3]) end)) ==
            quote do: (receive do 1 -> import :"Elixir.List", []; 2 -> flatten([1, 2, 3]) end)
   end
 
   test "receive: expands guards" do
-    assert expand_and_clean(quote do: (receive do x when x when __ENV__.context -> true end)) ==
+    assert expand(quote do: (receive do x when x when __ENV__.context -> true end)) ==
            quote do: (receive do x when x when :guard -> true end)
   end
 
   test "receive: does not leaks clause vars" do
-    assert expand_and_clean(quote do: (receive do x -> x; y -> y end; :erlang.+(x, y))) ==
+    assert expand(quote do: (receive do x -> x; y -> y end; :erlang.+(x, y))) ==
            quote do: (receive do x -> x; y -> y end; :erlang.+(x(), y()))
   end
 
   test "receive: leaks vars" do
-    assert expand_and_clean(quote do: (receive do x -> x = x; y -> y = y end; :erlang.+(x, y))) ==
+    assert expand(quote do: (receive do x -> x = x; y -> y = y end; :erlang.+(x, y))) ==
            quote do: (receive do x -> x = x; y -> y = y end; :erlang.+(x, y))
   end
 
   test "receive: leaks vars on after" do
-    assert expand_and_clean(quote do: (receive do x -> x = x after y -> y; w = y end; :erlang.+(x, w))) ==
+    assert expand(quote do: (receive do x -> x = x after y -> y; w = y end; :erlang.+(x, w))) ==
            quote do: (receive do x -> x = x after y() -> y(); w = y() end; :erlang.+(x, w))
   end
 
@@ -540,8 +540,8 @@ defmodule Kernel.ExpansionTest do
     13
   end
 
-  defp expand_and_clean(expr) do
-    cleaner = &Keyword.drop(&1, [:export])
+  defp expand_and_clean(expr, vars) do
+    cleaner = &Keyword.drop(&1, vars)
     expr
     |> expand_env(__ENV__)
     |> elem(0)

lib/elixir/test/elixir/kernel/quote_test.exs

@@ -434,4 +434,18 @@ defmodule Kernel.QuoteTest.ImportsHygieneTest do
   test "explicitly overridden imports" do
     assert with_length() == 5
   end
+
+  defmodule BinaryUtils do
+    defmacro int32 do
+      quote do
+        integer-size(32)
+      end
+    end
+  end
+
+  test "checks the context also for variables to zero-arity functions" do
+    import BinaryUtils
+    {:int32, meta, __MODULE__} =  quote do: int32
+    assert meta[:import] == BinaryUtils
+  end
 end

lib/elixir/test/elixir/kernel/warning_test.exs

@@ -28,6 +28,31 @@ defmodule Kernel.WarningTest do
     purge Sample
   end
 
+  test "unused variable in redefined function in different file" do
+    output = capture_err(fn ->
+      Code.eval_string """
+      defmodule Sample do
+        defmacro __using__(_) do
+          quote location: :keep do
+            def function(arg)
+          end
+        end
+      end
+      """
+      Code.eval_string("""
+      defmodule RedefineSample do
+        use Sample
+        def function(var123), do: nil
+      end
+      """, [], file: "redefine_sample.ex")
+    end)
+    assert output =~ "redefine_sample.ex:3"
+    assert output =~ "variable \"var123\" is unused"
+  after
+    purge Sample
+    purge RedefineSample
+  end
+
   test "useless literal" do
     message = "code block contains unused literal \"oops\""
 
@@ -348,6 +373,19 @@ defmodule Kernel.WarningTest do
     purge Sample
   end
 
+  # TODO: Remove this check once we depend only on 19
+  if :erlang.system_info(:otp_release) >= '20' do
+    test "duplicate map keys" do
+      assert capture_err(fn ->
+        defmodule DuplicateKeys do
+          assert %{a: :b, a: :c} == %{a: :c}
+          assert %{1 => 2, 1 => 3} == %{1 => 3}
+          assert %{:a => :b, a: :c} == %{a: :c}
+        end
+      end) =~ "key :a will be overridden in map"
+    end
+  end
+
   test "unused guard" do
     assert capture_err(fn ->
       Code.eval_string """
@@ -685,12 +723,11 @@ defmodule Kernel.WarningTest do
       Code.eval_string """
       defmodule Sample do
         @at "Something"
-        Module.put_attribute(__MODULE__, :put_attribute, "Something")
       end
       """
     end)
     assert content =~ "module attribute @at was set but never used"
-    assert content =~ "module attribute @put_attribute was set but never used"
+    assert content =~ "nofile:2"
   after
     purge Sample
   end
@@ -744,6 +781,27 @@ defmodule Kernel.WarningTest do
     purge Sample
   end
 
+  defmodule User do
+    defstruct [:name]
+  end
+
+  test ":__struct__ is ignored when using structs" do
+    assert capture_err(fn ->
+      Code.eval_string """
+      assert %Kernel.WarningTest.User{__struct__: Ignored, name: "joe"} ==
+             %Kernel.WarningTest.User{name: "joe"}
+      """, [], __ENV__
+    end) =~ "key :__struct__ is ignored when using structs"
+
+    assert capture_err(fn ->
+      Code.eval_string """
+      user = %Kernel.WarningTest.User{name: "meg"}
+      assert %Kernel.WarningTest.User{user | __struct__: Ignored, name: "joe"} ==
+             %Kernel.WarningTest.User{__struct__: Kernel.WarningTest.User, name: "joe"}
+      """, [], __ENV__
+    end) =~ "key :__struct__ is ignored when using structs"
+  end
+
   defp purge(list) when is_list(list) do
     Enum.each list, &purge/1
   end

lib/elixir/test/elixir/macro_test.exs

@@ -43,7 +43,7 @@ defmodule MacroTest do
   end
 
   test "escape handles bitstring" do
-    assert {:<<>>, [], [{:::, [], [1, 4]}, ","]} == Macro.escape(<<300::12>>)
+    assert {:<<>>, [], [{:::, [], [1, {:size, [], [4]}]}, {:::, [], [",", {:binary, [], []}]}]} == Macro.escape(<<300::12>>)
   end
 
   test "escape works recursively" do

lib/elixir/test/elixir/map_test.exs

@@ -86,12 +86,6 @@ defmodule MapTest do
     assert %{1 => 2, a: :b,} == %{1 => 2, a: :b}
   end
 
-  test "maps with duplicate keys" do
-    assert %{a: :b, a: :c} == %{a: :c}
-    assert %{1 => 2, 1 => 3} == %{1 => 3}
-    assert %{:a => :b, a: :c} == %{a: :c}
-  end
-
   test "update maps" do
     assert %{@sample | a: 3} == %{a: 3, b: 2}
 

lib/elixir/test/elixir/module_test.exs

@@ -60,7 +60,7 @@ defmodule ModuleTest do
 
   test "module attributes returns value" do
     in_module do
-      assert (@return [:foo, :bar]) == [:foo, :bar]
+      assert (@return [:foo, :bar]) == :ok
       _ = @return
     end
   end

lib/elixir/test/elixir/option_parser_test.exs

@@ -256,6 +256,11 @@ defmodule OptionParserTest do
            == {[foo: true, boo: "-"], ["-"], []}
   end
 
+  test "allow nonexistent atoms" do
+    assert OptionParser.parse(["--option-key-creates-atom"], allow_nonexistent_atoms: true) ==
+           {[{String.to_atom("option_key_creates_atom"), true}], [], []}
+  end
+
   test "correctly handles negative integers" do
     assert OptionParser.parse(["arg1", "-43"])
       == {[], ["arg1", "-43"], []}

lib/elixir/test/elixir/path_test.exs

@@ -211,6 +211,8 @@ defmodule PathTest do
     assert Path.join(['/foo/', "/bar/"]) == "/foo/bar"
     assert Path.join(["/", ""]) == "/"
     assert Path.join(["/", "", "bar"]) == "/bar"
+    assert Path.join(['foo', [?b, "a", ?r]]) == "foo/bar"
+    assert Path.join([[?f, 'o', "o"]]) == "foo"
   end
 
   test "join two" do

lib/elixir/test/elixir/record_test.exs

@@ -136,7 +136,7 @@ defmodule RecordTest do
     binary: <<1, 2, 3>>,
     charlist: 'abc'
 
-  test "records with literal defaults" do
+  test "records with literal defaults and on-the-fly record" do
     assert defaults(defaults()) == [
       struct: ~D[2016-01-01],
       map: %{},
@@ -163,6 +163,35 @@ defmodule RecordTest do
     assert defaults(defaults(), :charlist) == 'abc'
   end
 
+  test "records with literal defaults and record in a variable" do
+    defaults = defaults()
+
+    assert defaults(defaults) == [
+      struct: ~D[2016-01-01],
+      map: %{},
+      tuple_zero: {},
+      tuple_one: {1},
+      tuple_two: {1, 2},
+      tuple_three: {1, 2, 3},
+      list: [1, 2, 3],
+      call: MapSet.new,
+      string: "abc",
+      binary: <<1, 2, 3>>,
+      charlist: 'abc'
+    ]
+    assert defaults(defaults, :struct) == ~D[2016-01-01]
+    assert defaults(defaults, :map) == %{}
+    assert defaults(defaults, :tuple_zero) == {}
+    assert defaults(defaults, :tuple_one) == {1}
+    assert defaults(defaults, :tuple_two) == {1, 2}
+    assert defaults(defaults, :tuple_three) == {1, 2, 3}
+    assert defaults(defaults, :list) == [1, 2, 3]
+    assert defaults(defaults, :call) == MapSet.new
+    assert defaults(defaults, :string) == "abc"
+    assert defaults(defaults, :binary) == <<1, 2, 3>>
+    assert defaults(defaults, :charlist) == 'abc'
+  end
+
   test "records with dynamic arguments" do
     record = file_info()
     assert file_info(record, :size) == :undefined

lib/elixir/test/elixir/regex_test.exs

@@ -91,7 +91,55 @@ defmodule RegexTest do
     refute <<?<, 255, ?>>> =~ ~r/<.>/u
   end
 
-  test "names" do
+  test "ungreedy" do
+    assert Regex.run(~r/[\d ]+/, "1 2 3 4 5"), ["1 2 3 4 5"]
+    assert Regex.run(~r/[\d ]?+/, "1 2 3 4 5"), ["1"]
+    assert Regex.run(~r/[\d ]+/U, "1 2 3 4 5"), ["1"]
+  end
+
+  test "regex?/1" do
+    assert Regex.regex?(~r/foo/)
+    refute Regex.regex?(0)
+  end
+
+  test "compile/1" do
+    {:ok, regex} = Regex.compile("foo")
+    assert Regex.regex?(regex)
+    assert {:error, _} = Regex.compile("*foo")
+    assert {:error, _} = Regex.compile("foo", "y")
+    assert {:error, _} = Regex.compile("foo", "uy")
+  end
+
+  test "compile/1 with Erlang options" do
+    {:ok, regex} = Regex.compile("foo\\sbar", [:dotall, {:newline, :anycrlf}])
+    assert "foo\nbar" =~ regex
+  end
+
+  test "compile!/1" do
+    assert Regex.regex?(Regex.compile!("foo"))
+
+    assert_raise Regex.CompileError, ~r/position 0$/, fn ->
+      Regex.compile!("*foo")
+    end
+  end
+
+  test "recompile/1" do
+    new_regex = ~r/foo/
+    {:ok, regex} = Regex.recompile(new_regex)
+    assert Regex.regex?(regex)
+    assert Regex.regex?(Regex.recompile!(new_regex))
+
+    old_regex = Map.delete(~r/foo/, :re_version)
+    {:ok, regex} = Regex.recompile(old_regex)
+    assert Regex.regex?(regex)
+    assert Regex.regex?(Regex.recompile!(old_regex))
+  end
+
+  test "opts/1" do
+    assert Regex.opts(Regex.compile!("foo", "i")) == "i"
+  end
+
+  test "names/1" do
     assert Regex.names(~r/(?<FOO>foo)/) == ["FOO"]
   end
 
@@ -219,12 +267,6 @@ defmodule RegexTest do
     assert Regex.replace(~r[a(b)c], "abcabc", fn "abc", "b" -> "ac" end, global: false) == "acabc"
   end
 
-  test "ungreedy" do
-    assert Regex.run(~r/[\d ]+/, "1 2 3 4 5"), ["1 2 3 4 5"]
-    assert Regex.run(~r/[\d ]?+/, "1 2 3 4 5"), ["1"]
-    assert Regex.run(~r/[\d ]+/U, "1 2 3 4 5"), ["1"]
-  end
-
   test "escape" do
     assert matches_escaped?(".")
     refute matches_escaped?(".", "x")
@@ -247,6 +289,7 @@ defmodule RegexTest do
     assert matches_escaped?("# lol")
 
     assert matches_escaped?("\\A.^$*+?()[{\\| \t\n\x20\\z #hello\u202F\u205F")
+    assert Regex.match? Regex.compile!("[" <> Regex.escape("!-#") <> "]"), "-"
 
     assert Regex.escape("{}") == "\\{\\}"
     assert Regex.escape("[]") == "\\[\\]"

lib/elixir/test/elixir/stream_test.exs

@@ -481,6 +481,14 @@ defmodule StreamTest do
     assert Process.get(:stream_transform)
   end
 
+  test "transform/3 (via flat_map) handles multiple returns from suspension" do
+    assert [false]
+           |> Stream.take(1)
+           |> Stream.concat([true])
+           |> Stream.flat_map(&[&1])
+           |> Enum.to_list() == [false, true]
+  end
+
   test "iterate/2" do
     stream = Stream.iterate(0, &(&1+2))
     assert Enum.take(stream, 5) == [0, 2, 4, 6, 8]

lib/elixir/test/elixir/string_test.exs

@@ -449,7 +449,8 @@ defmodule StringTest do
     # CLRF
     assert String.graphemes("\r\n\f") == ["\r\n", "\f"]
     # Regional indicator
-    assert String.graphemes("\u{1F1E6}\u{1F1E7}\u{1F1E8}") == ["\u{1F1E6}\u{1F1E7}\u{1F1E8}"]
+    assert String.graphemes("\u{1F1E6}\u{1F1E7}") == ["\u{1F1E6}\u{1F1E7}"]
+    assert String.graphemes("\u{1F1E6}\u{1F1E7}\u{1F1E8}") == ["\u{1F1E6}\u{1F1E7}", "\u{1F1E8}"]
     # Hangul
     assert String.graphemes("\u1100\u115D\uB4A4") == ["ᄀᅝ뒤"]
     # Special Marking with Extended

lib/elixir/test/elixir/system_test.exs

@@ -13,7 +13,7 @@ defmodule SystemTest do
     assert is_binary build_info[:version]
 
     if build_info[:revision] != "" do
-      assert String.length(build_info[:revision]) == 7
+      assert String.length(build_info[:revision]) >= 7
     end
 
     version_file = Path.join([__DIR__, "../../../..", "VERSION"]) |> Path.expand

lib/elixir/test/elixir/task_test.exs

@@ -492,6 +492,7 @@ defmodule TaskTest do
         Process.flag(:trap_exit, true)
         assert 1..4 |> Task.async_stream(&exit/1, @opts) |> Enum.to_list ==
                [exit: 1, exit: 2, exit: 3, exit: 4]
+        refute_received {:EXIT, _, _}
       end
 
       test "shuts down unused tasks" do

lib/elixir/test/elixir/uri_test.exs

@@ -238,6 +238,7 @@ defmodule URITest do
     end
 
     assert URI.merge("http://google.com/foo", "http://example.com/baz") |> to_string == "http://example.com/baz"
+    assert URI.merge("http://google.com/foo", "//example.com/baz") |> to_string == "http://example.com/baz"
 
     assert URI.merge("http://example.com", URI.parse("/foo")) |> to_string == "http://example.com/foo"
 

lib/elixir/test/elixir/version_test.exs

@@ -66,9 +66,17 @@ defmodule VersionTest do
     assert {:ok, %V{major: 1, minor: 4, patch: 5, pre: ["6-g3318bd5"]}} = V.parse("1.4.5-6-g3318bd5+ignore")
 
     assert :error = V.parse("foobar")
-    assert :error = V.parse("2.3")
     assert :error = V.parse("2")
+    assert :error = V.parse("2.")
+    assert :error = V.parse("2.3")
+    assert :error = V.parse("2.3.")
+    assert :error = V.parse("2.3.0-")
+    assert :error = V.parse("2.3.0+")
     assert :error = V.parse("2.3.0-01")
+    assert :error = V.parse("2.3.00-1")
+    assert :error = V.parse("2.3.00")
+    assert :error = V.parse("2.03.0")
+    assert :error = V.parse("02.3.0")
   end
 
   test "to_string" do

lib/elixir/unicode/unicode.ex

@@ -52,20 +52,6 @@ defmodule String.Unicode do
     end
   end
 
-  # Handle Hangul L
-  for codepoint <- cluster["L"] do
-    def next_grapheme_size(<<unquote(codepoint), rest::binary>>) do
-      next_hangul_l_size(rest, unquote(byte_size(codepoint)))
-    end
-  end
-
-  # Handle Hangul T
-  for codepoint <- cluster["T"] do
-    def next_grapheme_size(<<unquote(codepoint), rest::binary>>) do
-      next_hangul_t_size(rest, unquote(byte_size(codepoint)))
-    end
-  end
-
   # Handle Regional
   for codepoint <- cluster["Regional_Indicator"] do
     def next_grapheme_size(<<unquote(codepoint), rest::binary>>) do
@@ -73,14 +59,48 @@ defmodule String.Unicode do
     end
   end
 
-  # Handle extended entries
+  # Handle Hangul L
+  for codepoint <- cluster["L"] do
+    def next_grapheme_size(<<unquote(codepoint), rest::binary>>) do
+      next_hangul_l_size(rest, unquote(byte_size(codepoint)))
+    end
+  end
 
+  # Handle Hangul V
+  for codepoint <- cluster["LV"] ++ cluster["V"]  do
+    def next_grapheme_size(<<unquote(codepoint), rest::binary>>) do
+      next_hangul_v_size(rest, unquote(byte_size(codepoint)))
+    end
+  end
+
+  # Handle Hangul T
+  for codepoint <- cluster["LVT"] ++ cluster["T"] do
+    def next_grapheme_size(<<unquote(codepoint), rest::binary>>) do
+      next_hangul_t_size(rest, unquote(byte_size(codepoint)))
+    end
+  end
+
+  # Handle E_Base
+  for codepoint <- cluster["E_Base"] ++ cluster["E_Base_GAZ"] do
+    def next_grapheme_size(<<unquote(codepoint), rest::binary>>) do
+      next_extend_size(rest, unquote(byte_size(codepoint)), :e_base)
+    end
+  end
+
+  # Handle ZWJ
+  for codepoint <- cluster["ZWJ"] do
+    def next_grapheme_size(<<unquote(codepoint), rest::binary>>) do
+      next_extend_size(rest, unquote(byte_size(codepoint)), :zwj)
+    end
+  end
+
+  # Handle extended entries
   def next_grapheme_size(<<cp::utf8, rest::binary>>) do
     case cp do
-      x when x <= 0x007F -> next_extend_size(rest, 1)
-      x when x <= 0x07FF -> next_extend_size(rest, 2)
-      x when x <= 0xFFFF -> next_extend_size(rest, 3)
-      _                  -> next_extend_size(rest, 4)
+      x when x <= 0x007F -> next_extend_size(rest, 1, :other)
+      x when x <= 0x07FF -> next_extend_size(rest, 2, :other)
+      x when x <= 0xFFFF -> next_extend_size(rest, 3, :other)
+      _                  -> next_extend_size(rest, 4, :other)
     end
   end
 
@@ -92,82 +112,139 @@ defmodule String.Unicode do
     nil
   end
 
-  # Handle Hangul L
-  for codepoint <- cluster["L"] do
-    defp next_hangul_l_size(<<unquote(codepoint), rest::binary>>, size) do
-      next_hangul_l_size(rest, size + unquote(byte_size(codepoint)))
-    end
-  end
-
-  for codepoint <- cluster["LV"] do
-    defp next_hangul_l_size(<<unquote(codepoint), rest::binary>>, size) do
-      next_hangul_v_size(rest, size + unquote(byte_size(codepoint)))
-    end
-  end
-
-  for codepoint <- cluster["LVT"] do
-    defp next_hangul_l_size(<<unquote(codepoint), rest::binary>>, size) do
-      next_hangul_t_size(rest, size + unquote(byte_size(codepoint)))
-    end
-  end
-
+  # Handle hanguls
   defp next_hangul_l_size(rest, size) do
-    next_hangul_v_size(rest, size)
-  end
-
-  # Handle Hangul V
-  for codepoint <- cluster["V"] do
-    defp next_hangul_v_size(<<unquote(codepoint), rest::binary>>, size) do
-      next_hangul_v_size(rest, size + unquote(byte_size(codepoint)))
+    case next_hangul(rest, size) do
+      {:l, rest, size} -> next_hangul_l_size(rest, size)
+      {:v, rest, size} -> next_hangul_v_size(rest, size)
+      {:lv, rest, size} -> next_hangul_v_size(rest, size)
+      {:lvt, rest, size} -> next_hangul_t_size(rest, size)
+      _ -> next_extend_size(rest, size, :other)
     end
   end
 
   defp next_hangul_v_size(rest, size) do
-    next_hangul_t_size(rest, size)
-  end
-
-  # Handle Hangul T
-  for codepoint <- cluster["T"] do
-    defp next_hangul_t_size(<<unquote(codepoint), rest::binary>>, size) do
-      next_hangul_t_size(rest, size + unquote(byte_size(codepoint)))
+    case next_hangul(rest, size) do
+      {:v, rest, size} -> next_hangul_v_size(rest, size)
+      {:t, rest, size} -> next_hangul_t_size(rest, size)
+      _ -> next_extend_size(rest, size, :other)
     end
   end
 
   defp next_hangul_t_size(rest, size) do
-    next_extend_size(rest, size)
+    case next_hangul(rest, size) do
+      {:t, rest, size} -> next_hangul_t_size(rest, size)
+      _ -> next_extend_size(rest, size, :other)
+    end
+  end
+
+  for codepoint <- cluster["L"] do
+    defp next_hangul(<<unquote(codepoint), rest::binary>>, size) do
+      {:l, rest, size + unquote(byte_size(codepoint))}
+    end
+  end
+
+  for codepoint <- cluster["V"] do
+    defp next_hangul(<<unquote(codepoint), rest::binary>>, size) do
+      {:v, rest, size + unquote(byte_size(codepoint))}
+    end
+  end
+
+  for codepoint <- cluster["T"] do
+    defp next_hangul(<<unquote(codepoint), rest::binary>>, size) do
+      {:t, rest, size + unquote(byte_size(codepoint))}
+    end
+  end
+
+  for codepoint <- cluster["LV"] do
+    defp next_hangul(<<unquote(codepoint), rest::binary>>, size) do
+      {:lv, rest, size + unquote(byte_size(codepoint))}
+    end
+  end
+
+  for codepoint <- cluster["LVT"] do
+    defp next_hangul(<<unquote(codepoint), rest::binary>>, size) do
+      {:lvt, rest, size + unquote(byte_size(codepoint))}
+    end
+  end
+
+  defp next_hangul(_, _) do
+    false
   end
 
   # Handle regional
   for codepoint <- cluster["Regional_Indicator"] do
     defp next_regional_size(<<unquote(codepoint), rest::binary>>, size) do
-      next_regional_size(rest, size + unquote(byte_size(codepoint)))
+      next_extend_size(rest, size + unquote(byte_size(codepoint)), :other)
     end
   end
-
   defp next_regional_size(rest, size) do
-    next_extend_size(rest, size)
+    next_extend_size(rest, size, :other)
   end
 
-  # Handle Extend+SpacingMark
-  for codepoint <- cluster["Extend"] ++ cluster["SpacingMark"]  do
-    defp next_extend_size(<<unquote(codepoint), rest::binary>>, size) do
-      next_extend_size(rest, size + unquote(byte_size(codepoint)))
+  # Handle Extend+SpacingMark+ZWJ
+  for codepoint <- cluster["Extend"] do
+    defp next_extend_size(<<unquote(codepoint), rest::binary>>, size, marker) do
+      next_extend_size(rest, size + unquote(byte_size(codepoint)), keep_ebase(marker))
     end
   end
 
-  defp next_extend_size(rest, size) do
+  for codepoint <- cluster["SpacingMark"] do
+    defp next_extend_size(<<unquote(codepoint), rest::binary>>, size, _marker) do
+      next_extend_size(rest, size + unquote(byte_size(codepoint)), :other)
+    end
+  end
+
+  for codepoint <- cluster["ZWJ"] do
+    defp next_extend_size(<<unquote(codepoint), rest::binary>>, size, _marker) do
+      next_extend_size(rest, size + unquote(byte_size(codepoint)), :zwj)
+    end
+  end
+
+  for codepoint <- cluster["E_Modifier"] do
+    defp next_extend_size(<<unquote(codepoint), rest::binary>>, size, :e_base) do
+      next_extend_size(rest, size + unquote(byte_size(codepoint)), :other)
+    end
+  end
+
+  for codepoint <- cluster["Glue_After_Zwj"] do
+    defp next_extend_size(<<unquote(codepoint), rest::binary>>, size, :zwj) do
+      next_extend_size(rest, size + unquote(byte_size(codepoint)), :other)
+    end
+  end
+
+  for codepoint <- cluster["E_Base_GAZ"] do
+    defp next_extend_size(<<unquote(codepoint), rest::binary>>, size, :zwj) do
+      next_extend_size(rest, size + unquote(byte_size(codepoint)), :e_base)
+    end
+  end
+
+  defp next_extend_size(rest, size, _) do
     {size, rest}
   end
 
+  defp keep_ebase(:e_base), do: :e_base
+  defp keep_ebase(_), do: :other
+
   # Handle Prepend
   for codepoint <- cluster["Prepend"] do
     defp next_prepend_size(<<unquote(codepoint), rest::binary>>, size) do
       next_prepend_size(rest, size + unquote(byte_size(codepoint)))
     end
   end
-  
+
+  # However, if we see a control character, we have to break it
+  for codepoint <- cluster["CR"] ++ cluster["LF"] ++ cluster["Control"] do
+    defp next_prepend_size(<<unquote(codepoint), _::binary>> = rest, size) do
+      {size, rest}
+    end
+  end
+
   defp next_prepend_size(rest, size) do
-    {size, rest}
+    case next_grapheme_size(rest) do
+      {more, rest} ->  {more + size, rest}
+      nil -> {size, rest}
+    end
   end
 
   # Graphemes

lib/ex_unit/lib/ex_unit/diff.ex

@@ -436,6 +436,10 @@ defmodule ExUnit.Diff do
     inspect(key) <> " => "
   end
 
+  defp format_key(key, true) when is_nil(key) or is_boolean(key) do
+    inspect(key) <> ": "
+  end
+
   defp format_key(key, true) do
     ":" <> result = inspect(key)
     result <> ": "

lib/ex_unit/lib/ex_unit/doc_test.ex

@@ -426,7 +426,7 @@ defmodule ExUnit.DocTest do
   end
 
   defp extract_tests(line_no, doc, module) do
-    all_lines = String.split(doc, ~r/\n/, trim: false)
+    all_lines = String.split(doc, "\n", trim: false)
     lines = adjust_indent(all_lines, line_no + 1, module)
     extract_tests(lines, "", "", [], true, module)
   end
@@ -626,11 +626,29 @@ defmodule ExUnit.DocTest do
         [mod, message] = :binary.split(error, ")")
         {:error, Module.concat([mod]), String.trim_leading(message)}
       _ ->
-        if string =~ ~r/\A#[A-Z][\w\.]*</mu do
+        if is_inspected?(string) do
           {:inspect, inspect(string)}
         else
           {:test, string}
         end
     end
   end
+
+  defp is_inspected?(<<?#, char, rest::binary>>) when char in ?A..?Z,
+    do: is_inspected_end?(rest)
+  defp is_inspected?(_),
+    do: false
+
+  defp is_inspected_end?(<<?., char, rest::binary>>) when char in ?A..?Z,
+    do: is_inspected_end?(rest)
+  defp is_inspected_end?(<<char, rest::binary>>)
+       when char in ?A..?Z
+       when char in ?a..?z
+       when char in ?0..?9
+       when char == ?_,
+    do: is_inspected_end?(rest)
+  defp is_inspected_end?(<<?<, _::binary>>),
+    do: true
+  defp is_inspected_end?(_),
+    do: false
 end

lib/ex_unit/lib/ex_unit/filters.ex

@@ -14,10 +14,11 @@ defmodule ExUnit.Filters do
   """
   @spec parse_path(String.t) :: {String.t, any}
   def parse_path(file) do
-    case Regex.run(~r/^(.+):(\d+)$/, file, capture: :all_but_first) do
-      [file, line_number] ->
-        {file, exclude: [:test], include: [line: line_number]}
-      nil ->
+    {paths, [line]} = file |> String.split(":") |> Enum.split(-1)
+    case Integer.parse(line) do
+      {_, ""} ->
+        {Enum.join(paths, ":"), exclude: [:test], include: [line: line]}
+      _ ->
         {file, []}
     end
   end

lib/ex_unit/test/ex_unit/diff_test.exs

@@ -289,6 +289,10 @@ defmodule ExUnit.DiffTest do
 
     assert script(%{"foo-bar": 1}, %{}) == [{:eq, "%{"}, [[del: "\"foo-bar\": 1"]], {:eq, "}"}]
     assert script(%{}, %{}) == [eq: "%{}"]
+
+    assert script(%{nil: 42}, %{}) == [{:eq, "%{"}, [[del: "nil: 42"]], {:eq, "}"}]
+    assert script(%{true: 42}, %{}) == [{:eq, "%{"}, [[del: "true: 42"]], {:eq, "}"}]
+    assert script(%{false: 42}, %{}) == [{:eq, "%{"}, [[del: "false: 42"]], {:eq, "}"}]
   end
 
   test "structs" do

lib/iex/lib/iex/autocomplete.ex

@@ -1,20 +1,22 @@
 defmodule IEx.Autocomplete do
   @moduledoc false
 
-  def expand('') do
+  def expand(expr, server \\ IEx.Server)
+
+  def expand('', _server) do
     expand_import("")
   end
 
-  def expand([h | t]=expr) do
+  def expand([h | t]=expr, server) do
     cond do
       h === ?. and t != [] ->
-        expand_dot(reduce(t))
+        expand_dot(reduce(t), server)
       h === ?: and t == [] ->
         expand_erlang_modules()
       identifier?(h) ->
-        expand_expr(reduce(expr))
+        expand_expr(reduce(expr), server)
       (h == ?/) and t != [] and identifier?(hd(t)) ->
-        expand_expr(reduce(t))
+        expand_expr(reduce(t), server)
       h in '([{' ->
         expand('')
       true ->
@@ -26,31 +28,33 @@ defmodule IEx.Autocomplete do
     (h in ?a..?z) or (h in ?A..?Z) or (h in ?0..?9) or h in [?_, ??, ?!]
   end
 
-  defp expand_dot(expr) do
+  defp expand_dot(expr, server) do
     case Code.string_to_quoted expr do
       {:ok, atom} when is_atom(atom) ->
-        expand_call(atom, "")
+        expand_call(atom, "", server)
       {:ok, {:__aliases__, _, list}} ->
-        expand_elixir_modules(list, "")
+        expand_elixir_modules(list, "", server)
+      {:ok, {_, _, _} = ast_node} ->
+        expand_call(ast_node, "", server)
       _ ->
         no()
     end
   end
 
-  defp expand_expr(expr) do
+  defp expand_expr(expr, server) do
     case Code.string_to_quoted expr do
       {:ok, atom} when is_atom(atom) ->
         expand_erlang_modules(Atom.to_string(atom))
       {:ok, {atom, _, nil}} when is_atom(atom) ->
         expand_import(Atom.to_string(atom))
       {:ok, {:__aliases__, _, [root]}} ->
-        expand_elixir_modules([], Atom.to_string(root))
+        expand_elixir_modules([], Atom.to_string(root), server)
       {:ok, {:__aliases__, _, [h | _] = list}} when is_atom(h) ->
         hint = Atom.to_string(List.last(list))
         list = Enum.take(list, length(list) - 1)
-        expand_elixir_modules(list, hint)
-      {:ok, {{:., _, [mod, fun]}, _, []}} when is_atom(fun) ->
-        expand_call(mod, Atom.to_string(fun))
+        expand_elixir_modules(list, hint, server)
+      {:ok, {{:., _, [ast_node, fun]}, _, []}} when is_atom(fun) ->
+        expand_call(ast_node, Atom.to_string(fun), server)
       _ ->
         no()
     end
@@ -105,21 +109,38 @@ defmodule IEx.Autocomplete do
   ## Expand calls
 
   # :atom.fun
-  defp expand_call(mod, hint) when is_atom(mod) do
+  defp expand_call(mod, hint, _server) when is_atom(mod) do
     expand_require(mod, hint)
   end
 
   # Elixir.fun
-  defp expand_call({:__aliases__, _, list}, hint) do
-    expand_alias(list)
-    |> normalize_module
-    |> expand_require(hint)
+  defp expand_call({:__aliases__, _, list}, hint, server) do
+    case expand_alias(list, server) do
+      {:ok, alias} -> expand_require(alias, hint)
+      :error -> no()
+    end
   end
 
-  defp expand_call(_, _) do
+  # variable.fun_or_key
+  defp expand_call({_, _, _} = ast_node, hint, server) do
+    case value_from_binding(ast_node, server) do
+      {:ok, mod} when is_atom(mod) -> expand_call(mod, hint, server)
+      {:ok, map} when is_map(map) -> expand_map_field_access(map, hint)
+      _otherwise -> no()
+    end
+  end
+
+  defp expand_call(_, _, _) do
     no()
   end
 
+  defp expand_map_field_access(map, hint) do
+    case match_map_fields(map, hint) do
+      [%{kind: :map_key, name: name, value_is_map: false}] when name == hint -> no()
+      map_fields when is_list(map_fields) -> format_expansion(map_fields, hint)
+    end
+  end
+
   defp expand_require(mod, hint) do
     format_expansion match_module_funs(mod, hint), hint
   end
@@ -145,43 +166,38 @@ defmodule IEx.Autocomplete do
 
   ## Elixir modules
 
-  defp expand_elixir_modules([], hint) do
-    expand_elixir_modules(Elixir, hint, match_aliases(hint))
+  defp expand_elixir_modules([], hint, server) do
+    aliases = match_aliases(hint, server)
+    expand_elixir_modules_from_aliases(Elixir, hint, aliases)
   end
 
-  defp expand_elixir_modules(list, hint) do
-    expand_alias(list)
-    |> normalize_module
-    |> expand_elixir_modules(hint, [])
+  defp expand_elixir_modules(list, hint, server) do
+    case expand_alias(list, server) do
+      {:ok, alias} -> expand_elixir_modules_from_aliases(alias, hint, [])
+      :error -> no()
+    end
   end
 
-  defp expand_elixir_modules(mod, hint, aliases) do
+  defp expand_elixir_modules_from_aliases(mod, hint, aliases) do
     aliases
     |> Kernel.++(match_elixir_modules(mod, hint))
     |> Kernel.++(match_module_funs(mod, hint))
     |> format_expansion(hint)
   end
 
-  defp expand_alias([name | rest] = list) do
-    module = Module.concat(Elixir, name)
-    Enum.find_value env_aliases(), list, fn {alias, mod} ->
-      if alias === module do
-        case Atom.to_string(mod) do
-          "Elixir." <> mod ->
-            Module.concat [mod | rest]
-          _ ->
-            mod
-        end
-      end
+  defp expand_alias([name | rest], server) when is_atom(name) do
+    case Keyword.fetch(aliases_from_env(server), Module.concat(Elixir, name)) do
+      {:ok, name} when rest == [] -> {:ok, name}
+      {:ok, name} -> {:ok, Module.concat([name | rest])}
+      :error -> {:ok, Module.concat([name | rest])}
     end
   end
-
-  defp env_aliases do
-    Application.get_env(:iex, :autocomplete_server).current_env.aliases
+  defp expand_alias([_ | _], _) do
+    :error
   end
 
-  defp match_aliases(hint) do
-    for {alias, _mod} <- env_aliases(),
+  defp match_aliases(hint, server) do
+    for {alias, _mod} <- aliases_from_env(server),
         [name] = Module.split(alias),
         starts_with?(name, hint) do
       %{kind: :module, type: :alias, name: name}
@@ -203,14 +219,6 @@ defmodule IEx.Autocomplete do
 
   ## Helpers
 
-  defp normalize_module(mod) do
-    if is_list(mod) do
-      Module.concat(mod)
-    else
-      mod
-    end
-  end
-
   defp match_modules(hint, root) do
     get_modules(root)
     |> :lists.usort()
@@ -269,6 +277,14 @@ defmodule IEx.Autocomplete do
     end
   end
 
+  defp match_map_fields(map, hint) do
+    for {key, value} <- Map.to_list(map),
+        is_atom(key),
+        key = Atom.to_string(key),
+        String.starts_with?(key, hint),
+        do: %{kind: :map_key, name: key, value_is_map: is_map(value)}
+  end
+
   defp get_module_funs(mod) do
     docs = Code.get_docs(mod, :docs) || []
     module_info_funs(mod) |> Enum.reject(&hidden_fun?(&1, docs))
@@ -316,6 +332,10 @@ defmodule IEx.Autocomplete do
     for a <- :lists.sort(arities), do: "#{name}/#{a}"
   end
 
+  defp to_entries(%{kind: :map_key, name: name}) do
+    [name]
+  end
+
   defp to_uniq_entries(%{kind: :module}) do
     []
   end
@@ -324,6 +344,10 @@ defmodule IEx.Autocomplete do
     to_entries(fun)
   end
 
+  defp to_uniq_entries(%{kind: :map_key}) do
+    []
+  end
+
   defp to_hint(%{kind: :module, name: name}, hint) when name == hint do
     format_hint(name, name) <> "."
   end
@@ -336,8 +360,50 @@ defmodule IEx.Autocomplete do
     format_hint(name, hint)
   end
 
+  defp to_hint(%{kind: :map_key, name: name, value_is_map: true}, hint) when name == hint do
+    format_hint(name, hint) <> "."
+  end
+
+  defp to_hint(%{kind: :map_key, name: name}, hint) do
+    format_hint(name, hint)
+  end
+
   defp format_hint(name, hint) do
     hint_size = byte_size(hint)
     :binary.part(name, hint_size, byte_size(name) - hint_size)
   end
+
+  defp aliases_from_env(server) do
+    with evaluator when is_pid(evaluator) <- server.evaluator,
+         {:ok, aliases} <- IEx.Evaluator.value_from_env(evaluator, :aliases) do
+      aliases
+    else
+      _ -> []
+    end
+  end
+
+  defp value_from_binding(ast_node, server) do
+    with evaluator when is_pid(evaluator) <- server.evaluator(),
+         {var, map_key_path} <- extract_from_ast(ast_node, []) do
+      IEx.Evaluator.value_from_binding(evaluator, var, map_key_path)
+    else
+      _ -> :error
+    end
+  end
+
+  defp extract_from_ast(var_name, acc) when is_atom(var_name) do
+    {var_name, acc}
+  end
+
+  defp extract_from_ast({var_name, _, nil}, acc) when is_atom(var_name) do
+    {var_name, acc}
+  end
+
+  defp extract_from_ast({{:., _, [ast_node, fun]}, _, []}, acc) when is_atom(fun) do
+    extract_from_ast(ast_node, [fun | acc])
+  end
+
+  defp extract_from_ast(_ast_node, _acc) do
+    :error
+  end
 end

lib/iex/lib/iex/evaluator.ex

@@ -24,20 +24,64 @@ defmodule IEx.Evaluator do
     end
   end
 
+  @doc """
+  Gets a value out of the binding, using the provided
+  variable name and map key path.
+  """
+  @spec value_from_binding(pid, atom, [atom]) :: {:ok, any} | :error
+  def value_from_binding(evaluator, var_name, map_key_path) do
+    ref = make_ref()
+    send evaluator, {:value_from_binding, ref, self(), var_name, map_key_path}
+
+    receive do
+      {^ref, result} -> result
+    after
+      5000 -> :error
+    end
+  end
+
+  @doc """
+  Returns the current session environment if a session exists.
+  """
+  @spec value_from_env(pid, atom) :: {:ok, term} | :error
+  def value_from_env(evaluator, key) do
+    ref = make_ref()
+    send evaluator, {:value_from_env, ref, self(), key}
+
+    receive do
+      {^ref, result} -> result
+    after
+      5000 -> :error
+    end
+  end
+
   defp loop(server, history, state) do
     receive do
       {:eval, ^server, code, iex_state} ->
         {result, history, state} = eval(code, iex_state, history, state)
         send server, {:evaled, self(), result}
         loop(server, history, state)
-      {:peek_env, receiver} ->
-        send receiver, {:peek_env, state.env}
+      {:value_from_env, ref, receiver, key} ->
+        send receiver, {ref, Map.fetch(state.env, key)}
+        loop(server, history, state)
+      {:value_from_binding, ref, receiver, var_name, map_key_path} ->
+        value = traverse_binding(state.binding, var_name, map_key_path)
+        send receiver, {ref, value}
         loop(server, history, state)
       {:done, ^server} ->
         :ok
     end
   end
 
+  defp traverse_binding(binding, var_name, map_key_path) do
+    accumulator = Keyword.fetch(binding, var_name)
+
+    Enum.reduce map_key_path, accumulator, fn
+      key, {:ok, map} when is_map(map) -> Map.fetch(map, key)
+      _key, _acc -> :error
+    end
+  end
+
   defp loop_state(opts) do
     env =
       if env = opts[:env] do

lib/iex/lib/iex/helpers.ex

@@ -158,7 +158,7 @@ defmodule IEx.Helpers do
 
     {erls, exs} = Enum.split_with(found, &String.ends_with?(&1, ".erl"))
 
-    modules = Enum.map(erls, fn(source) ->
+    erl_modules = Enum.map(erls, fn(source) ->
       {module, binary} = compile_erlang(source)
       unless path == :in_memory do
         base = source |> Path.basename |> Path.rootname
@@ -167,7 +167,13 @@ defmodule IEx.Helpers do
       module
     end)
 
-    modules ++ compile_elixir(exs, path)
+    ex_modules = try do
+      compile_elixir(exs, path)
+    catch
+      _, _ -> raise CompileError
+    end
+
+    erl_modules ++ ex_modules
   end
 
   @doc """

lib/iex/lib/iex/server.ex

@@ -43,19 +43,15 @@ defmodule IEx.Server do
   end
 
   @doc """
-  Returns the current session environment if a session exists.
+  Returns the PID of the IEx evaluator process if it exists.
   """
-  @spec current_env :: Macro.Env.t
-  def current_env() do
-    case IEx.Server.whereis() do
-      nil -> %Macro.Env{}
-      server ->
-        send(server, {:peek_env, self()})
-        receive do
-          {:peek_env, %Macro.Env{} = env} -> env
-        after
-          5000 -> %Macro.Env{}
-        end
+  @spec evaluator :: pid | nil
+  def evaluator() do
+    case IEx.Server.local do
+      nil -> nil
+      pid ->
+        {:dictionary, dictionary} = Process.info(pid, :dictionary)
+        dictionary[:evaluator]
     end
   end
 
@@ -144,7 +140,11 @@ defmodule IEx.Server do
     loop(run_state(opts), evaluator, Process.monitor(evaluator))
   end
 
-  defp start_evaluator(opts) do
+  @doc """
+  Starst an evaluator using the provided options.
+  """
+  @spec start_evaluator(Keyword.t) :: pid
+  def start_evaluator(opts) do
     self_pid = self()
     self_leader = Process.group_leader
     evaluator = opts[:evaluator] ||
@@ -192,9 +192,6 @@ defmodule IEx.Server do
         exit_loop(evaluator, evaluator_ref)
       {:input, ^input, {:error, :terminated}} ->
         exit_loop(evaluator, evaluator_ref)
-      {:peek_env, receiver} ->
-        send evaluator, {:peek_env, receiver}
-        wait_input(state, evaluator, evaluator_ref, input)
       msg ->
         handle_take_over(msg, evaluator, evaluator_ref, input, fn ->
           wait_input(state, evaluator, evaluator_ref, input)
@@ -272,7 +269,11 @@ defmodule IEx.Server do
 
   defp allow_take?(identifier) do
     message = IEx.color(:eval_interrupt, "#{identifier}\nAllow? [Yn] ")
-    IO.gets(:stdio, message) =~ ~r/^(Y(es)?)?$/i
+    yes?(IO.gets(:stdio, message))
+  end
+
+  defp yes?(string) do
+    is_binary(string) and String.trim(string) in ["", "y", "Y", "yes", "YES", "Yes"]
   end
 
   ## State

lib/iex/mix.exs

@@ -11,7 +11,6 @@ defmodule IEx.Mixfile do
     [registered: [IEx.Supervisor, IEx.Config],
      mod: {IEx.App, []},
      env: [
-      autocomplete_server: IEx.Server,
       colors: [],
       inspect: [pretty: true],
       history_size: 20,

lib/iex/test/iex/autocomplete_test.exs

@@ -3,8 +3,29 @@ Code.require_file "../test_helper.exs", __DIR__
 defmodule IEx.AutocompleteTest do
   use ExUnit.Case, async: true
 
+  setup context do
+    previous_line = context[:previous_line]
+
+    if previous_line do
+      ExUnit.CaptureIO.capture_io(fn ->
+        evaluator = IEx.Server.start_evaluator([])
+        Process.put(:evaluator, evaluator)
+        send evaluator, {:eval, self(), previous_line <> "\n", %IEx.State{}}
+        assert_receive {:evaled, _, _}
+      end)
+    end
+
+    :ok
+  end
+
+  defmodule MyServer do
+    def evaluator do
+      Process.get(:evaluator)
+    end
+  end
+
   def expand(expr) do
-    IEx.Autocomplete.expand(Enum.reverse expr)
+    IEx.Autocomplete.expand(Enum.reverse(expr), MyServer)
   end
 
   test "Erlang module completion" do
@@ -69,6 +90,7 @@ defmodule IEx.AutocompleteTest do
     assert expand('.')   == {:no, '', []}
     assert expand('Xyz') == {:no, '', []}
     assert expand('x.Foo') == {:no, '', []}
+    assert expand('x.Foo.get_by') == {:no, '', []}
   end
 
   test "Elixir root submodule completion" do
@@ -93,6 +115,62 @@ defmodule IEx.AutocompleteTest do
     assert expand('String.printable?/') == {:yes, '', ['printable?/1']}
   end
 
+  @tag previous_line: "mod = String"
+  test "function completion using a variable bound to a module" do
+    assert expand('mod.print') == {:yes, 'able?', []}
+  end
+
+  @tag previous_line: "map = %{foo: 1, bar_1: 23, bar_2: 14}"
+  test "map atom key completion is supported" do
+    assert expand('map.f') == {:yes, 'oo', []}
+    assert expand('map.b') == {:yes, 'ar_', []}
+    assert expand('map.bar_') == {:yes, '', ['bar_1', 'bar_2']}
+    assert expand('map.c') == {:no, '', []}
+    assert expand('map.') == {:yes, '', ['bar_1', 'bar_2', 'foo']}
+    assert expand('map.foo') == {:no, '', []}
+  end
+
+  @tag previous_line: "map = %{nested: %{deeply: %{foo: 1, bar_1: 23, bar_2: 14, mod: String, num: 1}}}"
+  test "nested map atom key completion is supported" do
+    assert expand('map.nested.deeply.f') == {:yes, 'oo', []}
+    assert expand('map.nested.deeply.b') == {:yes, 'ar_', []}
+    assert expand('map.nested.deeply.bar_') == {:yes, '', ['bar_1', 'bar_2']}
+    assert expand('map.nested.deeply.') == {:yes, '', ['bar_1', 'bar_2', 'foo', 'mod', 'num']}
+    assert expand('map.nested.deeply.mod.print') == {:yes, 'able?', []}
+
+    assert expand('map.nested') == {:yes, '.', []}
+    assert expand('map.nested.deeply') == {:yes, '.', []}
+    assert expand('map.nested.deeply.foo') == {:no, '', []}
+
+    assert expand('map.nested.deeply.c') == {:no, '', []}
+    assert expand('map.a.b.c.f') == {:no, '', []}
+  end
+
+  @tag previous_line: ~s(map = %{"foo" => 1})
+  test "map string key completion is not supported" do
+    assert expand('map.f') == {:no, '', []}
+  end
+
+  @tag previous_line: "num = 5; map = %{nested: %{num: 23}}"
+  test "autocompletion off a bound variable only works for modules and maps" do
+    assert expand('num.print') == {:no, '', []}
+    assert expand('map.nested.num.f') == {:no, '', []}
+    assert expand('map.nested.num.key.f') == {:no, '', []}
+  end
+
+  @tag previous_line: "map = %{nested: %{deeply: %{num: 23}}}"
+  test "autocompletion using access syntax does is not supported" do
+    assert expand('map[:nested][:deeply].n') == {:no, '', []}
+    assert expand('map[:nested].deeply.n') == {:no, '', []}
+    assert expand('map.nested.[:deeply].n') == {:no, '', []}
+  end
+
+  @tag previous_line: "num = 5"
+  test "autocompletion off of unbound variables is not supported" do
+    assert expand('other_var.f') == {:no, '', []}
+    assert expand('a.b.c.d') == {:no, '', []}
+  end
+
   test "macro completion" do
     {:yes, '', list} = expand('Kernel.is_')
     assert is_list(list)
@@ -136,23 +214,15 @@ defmodule IEx.AutocompleteTest do
     assert expand('IEx.AutocompleteTest.SublevelTest.') == {:yes, 'LevelA', []}
   end
 
-  defmodule MyServer do
-    def current_env do
-      %Macro.Env{aliases: [{MyList, List}, {EList, :lists}]}
-    end
-  end
-
+  @tag previous_line: "alias List, as: MyList"
   test "complete aliases of Elixir modules" do
-    Application.put_env(:iex, :autocomplete_server, MyServer)
-
     assert expand('MyL') == {:yes, 'ist', []}
     assert expand('MyList') == {:yes, '.', []}
     assert expand('MyList.to_integer') == {:yes, [], ['to_integer/1', 'to_integer/2']}
   end
 
+  @tag previous_line: "alias :lists, as: EList"
   test "complete aliases of Erlang modules" do
-    Application.put_env(:iex, :autocomplete_server, MyServer)
-
     assert expand('EL') == {:yes, 'ist', []}
     assert expand('EList') == {:yes, '.', []}
     assert expand('EList.map') == {:yes, [], ['map/2', 'mapfoldl/3', 'mapfoldr/3']}
@@ -160,21 +230,21 @@ defmodule IEx.AutocompleteTest do
 
   test "completion for functions added when compiled module is reloaded" do
     {:module, _, bytecode, _} =
-      defmodule Elixir.Sample do
+      defmodule Sample do
         def foo(), do: 0
       end
-    File.write!("Elixir.Sample.beam", bytecode)
+    File.write!("Elixir.IEx.AutocompleteTest.Sample.beam", bytecode)
     assert Code.get_docs(Sample, :docs)
-    assert expand('Sample.foo') == {:yes, '', ['foo/0']}
+    assert expand('IEx.AutocompleteTest.Sample.foo') == {:yes, '', ['foo/0']}
 
     Code.compiler_options(ignore_module_conflict: true)
-    defmodule Elixir.Sample do
+    defmodule Sample do
       def foo(), do: 0
       def foobar(), do: 0
     end
-    assert expand('Sample.foo') == {:yes, '', ['foo/0', 'foobar/0']}
+    assert expand('IEx.AutocompleteTest.Sample.foo') == {:yes, '', ['foo/0', 'foobar/0']}
   after
-    File.rm("Elixir.Sample.beam")
+    File.rm("Elixir.IEx.AutocompleteTest.Sample.beam")
     Code.compiler_options(ignore_module_conflict: false)
     :code.purge(Sample)
     :code.delete(Sample)
@@ -184,7 +254,12 @@ defmodule IEx.AutocompleteTest do
     defstruct [:my_val]
   end
 
-  test "completion for structs" do
+  test "completion for struct names" do
     assert expand('%IEx.AutocompleteTest.MyStr') == {:yes, 'uct', []}
   end
+
+  @tag previous_line: "struct = %IEx.AutocompleteTest.MyStruct{}"
+  test "completion for struct keys" do
+    assert expand('struct.my') == {:yes, '_val', []}
+  end
 end

lib/iex/test/iex/helpers_test.exs

@@ -296,19 +296,12 @@ defmodule IEx.HelpersTest do
     cleanup_modules([Sample])
   end
 
-  test "c/2 helper" do
-    assert_raise UndefinedFunctionError, ~r"function Sample\.run/0 is undefined", fn ->
-      Sample.run
+  test "c helper with error" do
+    ExUnit.CaptureIO.capture_io fn ->
+      with_file "sample.ex", "raise \"oops\"", fn ->
+        assert_raise CompileError, fn -> c("sample.ex") end
+      end
     end
-
-    filename = "sample.ex"
-    with_file filename, test_module_code(), fn ->
-      assert c(filename, ".") == [Sample]
-      assert File.exists?("Elixir.Sample.beam")
-      assert Sample.run == :run
-    end
-  after
-    cleanup_modules([Sample])
   end
 
   test "c helper with full path" do
@@ -366,21 +359,6 @@ defmodule IEx.HelpersTest do
     cleanup_modules([:sample])
   end
 
-  test "c/2 helper erlang" do
-    assert_raise UndefinedFunctionError, ~r"function :sample.hello/0 is undefined", fn ->
-      :sample.hello
-    end
-
-    filename = "sample.erl"
-    with_file filename, erlang_module_code(), fn ->
-      assert c(filename, ".") == [:sample]
-      assert :sample.hello == :world
-      assert File.exists?("sample.beam")
-    end
-  after
-    cleanup_modules([:sample])
-  end
-
   test "c helper skips unknown files" do
     assert_raise UndefinedFunctionError, ~r"function :sample.hello/0 is undefined", fn ->
       :sample.hello
@@ -396,6 +374,20 @@ defmodule IEx.HelpersTest do
     cleanup_modules([:sample, Sample2])
   end
 
+  test "c helper with path" do
+    assert_raise UndefinedFunctionError, ~r"function Sample\.run/0 is undefined", fn ->
+      Sample.run
+    end
+
+    filename = "sample.ex"
+    with_file filename, test_module_code(), fn ->
+      assert c(filename, ".") == [Sample]
+      assert File.exists?("Elixir.Sample.beam")
+      assert Sample.run == :run
+    end
+  after
+    cleanup_modules([Sample])
+  end
 
   test "l helper" do
     assert_raise UndefinedFunctionError, ~r"function Sample.run/0 is undefined", fn ->

lib/logger/lib/logger/formatter.ex

@@ -84,7 +84,9 @@ defmodule Logger.Formatter do
   def compile({mod, fun}) when is_atom(mod) and is_atom(fun), do: {mod, fun}
 
   def compile(str) do
-    for part <- Regex.split(~r/(?<head>)\$[a-z]+(?<tail>)/, str, on: [:head, :tail], trim: true) do
+    regex = Regex.recompile!(~r/(?<head>)\$[a-z]+(?<tail>)/)
+
+    for part <- Regex.split(regex, str, on: [:head, :tail], trim: true) do
       case part do
         "$" <> code -> compile_code(String.to_atom(code))
         _           -> part

lib/logger/lib/logger/translator.ex

@@ -36,11 +36,12 @@ defmodule Logger.Translator do
     opts = Application.get_env(:logger, :translator_inspect_opts)
 
     case message do
-      {'** Generic server ' ++ _, [name, last, state, reason]} ->
+      {'** Generic server ' ++ _, [name, last, state, reason | client]} ->
         msg = ["GenServer #{inspect name} terminating", format_stop(reason),
-               "\nLast message: #{inspect last, opts}"]
+               "\nLast message#{format_from(client)}: #{inspect last, opts}"]
         if min_level == :debug do
-          {:ok, [msg | "\nState: #{inspect state, opts}"]}
+          {:ok, [msg, "\nState: #{inspect state, opts}" |
+                 format_client(client)]}
         else
           {:ok, msg}
         end
@@ -375,4 +376,24 @@ defmodule Logger.Translator do
 
   defp format_mfa(mod, fun, :undefined), do: [inspect(mod), ?., to_string(fun) | "/?"]
   defp format_mfa(mod, fun, args), do: Exception.format_mfa(mod, fun, args)
+
+  defp format_from([]),
+    do: ""
+  defp format_from([from]),
+    do: " (from #{inspect(from)})"
+  defp format_from([from, stacktrace]) when is_list(stacktrace),
+    do: " (from #{inspect(from)})"
+  defp format_from([from, node_name]) when is_atom(node_name),
+    do: " (from #{inspect(from)} on #{inspect(node_name)})"
+
+  defp format_client([from]) do
+    "\nClient #{inspect(from)} is dead"
+  end
+  defp format_client([from, stacktrace]) when is_list(stacktrace) do
+    ["\nClient #{inspect(from)} is alive\n" |
+      Exception.format_stacktrace(stacktrace)]
+  end
+  defp format_client(_) do
+    []
+  end
 end

lib/logger/test/logger/translator_test.exs

@@ -5,9 +5,18 @@ defmodule Logger.TranslatorTest do
   defmodule MyGenServer do
     use GenServer
 
+    def handle_cast(:error, _) do
+      raise "oops"
+    end
+
     def handle_call(:error, _, _) do
       raise "oops"
     end
+    def handle_call(:error_on_down, {pid, _}, _) do
+      mon = Process.monitor(pid)
+      assert_receive {:DOWN, ^mon, _, _, _}
+      raise "oops"
+    end
   end
 
   defmodule MyGenEvent do
@@ -93,20 +102,101 @@ defmodule Logger.TranslatorTest do
     Application.put_env(:logger, :translator_inspect_opts, [])
   end
 
-  test "translates GenServer crashes on debug" do
+  # TODO: Remove this check once we depend only on 20
+  if :erlang.system_info(:otp_release) >= '20' do
+    test "translates GenServer crashes on debug" do
+      {:ok, pid} = GenServer.start(MyGenServer, :ok)
+
+      assert capture_log(:debug, fn ->
+        catch_exit(GenServer.call(pid, :error))
+      end) =~ ~r"""
+      \[error\] GenServer #PID<\d+\.\d+\.\d+> terminating
+      \*\* \(RuntimeError\) oops
+      .*
+      Last message \(from #PID<\d+\.\d+\.\d+>\): :error
+      State: :ok
+      Client #PID<\d+\.\d+\.\d+> is alive
+      .*
+      """s
+    end
+
+    test "translates GenServer crashes with named client on debug" do
+      {:ok, pid} = GenServer.start(MyGenServer, :ok)
+
+      assert capture_log(:debug, fn ->
+        Process.register(self(), :named_client)
+        catch_exit(GenServer.call(pid, :error))
+      end) =~ ~r"""
+      \[error\] GenServer #PID<\d+\.\d+\.\d+> terminating
+      \*\* \(RuntimeError\) oops
+      .*
+      Last message \(from :named_client\): :error
+      State: :ok
+      Client :named_client is alive
+      .*
+      """s
+    end
+
+    test "translates GenServer crashes with dead client on debug" do
+      {:ok, pid} = GenServer.start(MyGenServer, :ok)
+
+      assert capture_log(:debug, fn ->
+        mon = Process.monitor(pid)
+        spawn_link(fn() ->
+          catch_exit(GenServer.call(pid, :error_on_down, 0))
+        end)
+        assert_receive {:DOWN, ^mon, _, _, _}
+      end) =~ ~r"""
+      \[error\] GenServer #PID<\d+\.\d+\.\d+> terminating
+      \*\* \(RuntimeError\) oops
+      .*
+      Last message \(from #PID<\d+\.\d+\.\d+>\): :error_on_down
+      State: :ok
+      Client #PID<\d+\.\d+\.\d+> is dead
+      """s
+    end
+  else
+    test "translates GenServer crashes on debug" do
+      {:ok, pid} = GenServer.start(MyGenServer, :ok)
+
+      assert capture_log(:debug, fn ->
+        catch_exit(GenServer.call(pid, :error))
+      end) =~ ~r"""
+      \[error\] GenServer #PID<\d+\.\d+\.\d+> terminating
+      \*\* \(RuntimeError\) oops
+      .*
+      Last message: :error
+      State: :ok
+      """s
+    end
+  end
+
+  test "translates GenServer crashes with no client" do
     {:ok, pid} = GenServer.start(MyGenServer, :ok)
 
     assert capture_log(:debug, fn ->
-      catch_exit(GenServer.call(pid, :error))
+      mon = Process.monitor(pid)
+      GenServer.cast(pid, :error)
+      assert_receive {:DOWN, ^mon, _, _, _}
     end) =~ ~r"""
     \[error\] GenServer #PID<\d+\.\d+\.\d+> terminating
     \*\* \(RuntimeError\) oops
     .*
-    Last message: :error
+    Last message: {:"\$gen_cast", :error}
     State: :ok
     """s
   end
 
+  test "translates GenServer crashes with no client on debug" do
+    {:ok, pid} = GenServer.start(MyGenServer, :ok)
+
+    refute capture_log(:debug, fn ->
+      mon = Process.monitor(pid)
+      GenServer.cast(pid, :error)
+      assert_receive {:DOWN, ^mon, _, _, _}
+    end) =~ "Client"
+  end
+
   test "translates :gen_event crashes" do
     {:ok, pid} = :gen_event.start()
     :ok = :gen_event.add_handler(pid, MyGenEvent, :ok)
@@ -150,6 +240,21 @@ defmodule Logger.TranslatorTest do
     """s
   end
 
+  test "translates Task async_stream crashes with neighbour" do
+    fun = fn -> Task.async_stream([:oops], :erlang, :error, []) |> Enum.to_list() end
+    {:ok, pid} = Task.start(__MODULE__, :task, [self(), fun])
+
+    assert capture_log(:debug, fn ->
+      ref = Process.monitor(pid)
+      send(pid, :go)
+      receive do: ({:DOWN, ^ref, _, _, _} -> :ok)
+    end) =~ ~r"""
+    Neighbours:
+        #{inspect pid}
+            Initial Call: Logger\.TranslatorTest\.task/2
+    """
+  end
+
   test "translates Task undef module crash" do
     assert capture_log(fn ->
       {:ok, pid} = Task.start(:module_does_not_exist, :undef, [])
@@ -244,24 +349,6 @@ defmodule Logger.TranslatorTest do
     """
   end
 
-  test "translates :proc_lib crashes" do
-    {:ok, pid} = Task.start_link(__MODULE__, :task, [self()])
-
-    assert capture_log(:info, fn ->
-      ref = Process.monitor(pid)
-      send(pid, :message)
-      send(pid, :go)
-      receive do: ({:DOWN, ^ref, _, _, _} -> :ok)
-    end) =~ ~r"""
-    \[error\] Process #PID<\d+\.\d+\.\d+> terminating
-    \*\* \(exit\) an exception was raised:
-        \*\* \(RuntimeError\) oops
-    .*
-    Initial Call: Logger.TranslatorTest.task/1
-    Ancestors: \[#PID<\d+\.\d+\.\d+>\]
-    """s
-  end
-
   test "translates Process crashes" do
     assert capture_log(:info, fn ->
       {_, ref} = spawn_monitor(fn() -> raise "oops" end)
@@ -275,6 +362,24 @@ defmodule Logger.TranslatorTest do
     """
   end
 
+  test "translates :proc_lib crashes" do
+    {:ok, pid} = Task.start_link(__MODULE__, :task, [self()])
+
+    assert capture_log(:info, fn ->
+      ref = Process.monitor(pid)
+
+      send(pid, :go)
+      receive do: ({:DOWN, ^ref, _, _, _} -> :ok)
+    end) =~ ~r"""
+    \[error\] Process #PID<\d+\.\d+\.\d+> terminating
+    \*\* \(exit\) an exception was raised:
+        \*\* \(RuntimeError\) oops
+    .*
+    Initial Call: Logger.TranslatorTest.task/1
+    Ancestors: \[#PID<\d+\.\d+\.\d+>\]
+    """s
+  end
+
   test "translates :proc_lib crashes with name" do
     {:ok, pid} = Task.start_link(__MODULE__, :task,
                                  [self(), fn() ->
@@ -648,5 +753,4 @@ defmodule Logger.TranslatorTest do
     :proc_lib.init_ack({:ok, self()})
     receive do: ({:EXIT, _, _} -> exit(:stop))
   end
-
 end

lib/mix/lib/mix/compilers/test.ex

@@ -36,6 +36,10 @@ defmodule Mix.Compilers.Test do
         Mix.shell.info "No stale tests."
         :noop
 
+      [] when test_patterns == [] ->
+        Mix.shell.info "There are no tests to run"
+        :noop
+
       [] ->
         Mix.shell.error "Test patterns did not match any file: " <> Enum.join(test_patterns, ", ")
         :noop

lib/mix/lib/mix/dep.ex

@@ -297,7 +297,7 @@ defmodule Mix.Dep do
   end
 
   defp check_manifest(%{scm: scm} = dep, build_path) do
-    vsn = System.version
+    vsn = {System.version, :erlang.system_info(:otp_release)}
 
     case Mix.Dep.ElixirSCM.read(build_path) do
       {:ok, old_vsn, _} when old_vsn != vsn ->

lib/mix/lib/mix/dep/elixir_scm.ex

@@ -2,7 +2,7 @@
 defmodule Mix.Dep.ElixirSCM do
   @moduledoc false
   @manifest ".compile.elixir_scm"
-  @manifest_vsn :v1
+  @manifest_vsn :v2
 
   def manifest(manifest_path \\ Mix.Project.manifest_path) do
     Path.join(manifest_path, @manifest)
@@ -13,7 +13,7 @@ defmodule Mix.Dep.ElixirSCM do
     File.mkdir_p!(manifest_path)
 
     manifest_data =
-      {@manifest_vsn, System.version, config[:build_scm]}
+      {@manifest_vsn, {System.version, :erlang.system_info(:otp_release)}, config[:build_scm]}
       |> :erlang.term_to_binary(compressed: 9)
 
     File.write!(manifest(manifest_path), manifest_data)
@@ -23,15 +23,10 @@ defmodule Mix.Dep.ElixirSCM do
     case File.read(manifest(manifest_path)) do
       {:ok, contents} ->
         try do
-          :erlang.binary_to_term(contents)
-        else
-          {@manifest_vsn, vsn, scm} ->
-            {:ok, vsn, scm}
-          _ ->
-            {:ok, "1.0.0", nil} # Force old version if file exists but old format
+          {@manifest_vsn, vsn, scm} = :erlang.binary_to_term(contents)
+          {:ok, vsn, scm}
         rescue
-          _ ->
-            {:ok, "1.0.0", nil} # Force old version if file exists but old format
+          _ -> {:ok, {"1.0.0", '17'}, nil}
         end
       _ ->
         :error

lib/mix/lib/mix/dep/loader.ex

@@ -264,11 +264,13 @@ defmodule Mix.Dep.Loader do
     {dep, deps}
   end
 
-  defp rebar_dep(%Mix.Dep{app: app, opts: opts} = dep, children, manager) do
-    config = File.cd!(opts[:dest], fn ->
-      Mix.Rebar.load_config(".")
-    end)
-    extra = Mix.Rebar.merge_config(dep.extra, config)
+  defp rebar_dep(%Mix.Dep{app: app, opts: opts, extra: overrides} = dep, children, manager) do
+    config =
+      File.cd!(opts[:dest], fn -> Mix.Rebar.load_config(".") end)
+
+    config =
+      Mix.Rebar.apply_overrides(app, config, overrides)
+
     deps =
       if children do
         from = Path.join(opts[:dest], "rebar.config")
@@ -277,9 +279,10 @@ defmodule Mix.Dep.Loader do
         # files in the dependency
         Enum.map(children, &to_dep(&1, from, manager))
       else
-        rebar_children(app, config, extra, manager)
+        rebar_children(config, manager, opts[:dest])
       end
-    {%{dep | extra: extra}, deps}
+
+    {%{dep | extra: config}, deps}
   end
 
   defp make_dep(dep) do
@@ -301,17 +304,18 @@ defmodule Mix.Dep.Loader do
     |> elem(0)
   end
 
-  defp rebar_children(app, root_config, extra, manager) do
-    from = Path.absname("rebar.config")
+  defp rebar_children(root_config, manager, dest) do
+    from = Path.absname(Path.join(dest, "rebar.config"))
     Mix.Rebar.recur(root_config, fn config ->
-      app
-      |> Mix.Rebar.deps(config, overrides(manager, extra))
-      |> Enum.map(fn dep -> %{to_dep(dep, from, manager) | extra: extra} end)
+      overrides = overrides(manager, config)
+      config
+      |> Mix.Rebar.deps()
+      |> Enum.map(fn dep -> %{to_dep(dep, from, manager) | extra: overrides} end)
     end) |> Enum.concat
   end
 
-  defp overrides(:rebar3, extra), do: extra[:overrides] || []
-  defp overrides(_, _extra), do: []
+  defp overrides(:rebar3, config), do: config[:overrides] || []
+  defp overrides(_, _config), do: []
 
   defp validate_app(%Mix.Dep{opts: opts, requirement: req, app: app} = dep) do
     opts_app = opts[:app]

lib/mix/lib/mix/local/installer.ex

@@ -99,7 +99,7 @@ defmodule Mix.Local.Installer do
   defp do_install({module, name}, src, opts) do
     src_basename = Path.basename(URI.parse(src).path)
     dst = Path.join(Mix.Local.path_for(name), src_basename)
-    previous_files = module.find_previous_versions(src, dst)
+    previous_files = module.find_previous_versions(src_basename, dst)
 
     if opts[:force] || should_install?(name, src, previous_files) do
       case Mix.Utils.read_path(src, opts) do

lib/mix/lib/mix/project.ex

@@ -233,9 +233,10 @@ defmodule Mix.Project do
   @spec apps_paths() :: %{atom => Path.t} | nil
   def apps_paths(config \\ config()) do
     if apps_path = config[:apps_path] do
-      Mix.ProjectStack.read_cache(:apps_path) ||
-        Mix.ProjectStack.write_cache(:apps_path,
-          config[:apps] |> umbrella_apps(apps_path) |> to_apps_path(apps_path))
+      key = {:apps_paths, Mix.Project.get!}
+      Mix.ProjectStack.read_cache(key) ||
+        Mix.ProjectStack.write_cache(key,
+          config[:apps] |> umbrella_apps(apps_path) |> to_apps_paths(apps_path))
     end
   end
 
@@ -249,7 +250,7 @@ defmodule Mix.Project do
     apps
   end
 
-  defp to_apps_path(apps, apps_path) do
+  defp to_apps_paths(apps, apps_path) do
     for app <- apps,
         path = path_with_mix_exs_otherwise_warn(app, apps_path),
         do: {app, path},

lib/mix/lib/mix/project_stack.ex

@@ -18,7 +18,7 @@ defmodule Mix.ProjectStack do
     get_and_update fn %{stack: stack} = state ->
       # Consider the first children to always have io_done
       # because we don't need to print anything unless another
-      # project talks ahold of the shell.
+      # project takes ahold of the shell.
       io_done? = stack == []
 
       config  = Keyword.merge(config, state.post_config)

lib/mix/lib/mix/rebar.ex

@@ -61,108 +61,6 @@ defmodule Mix.Rebar do
     end
   end
 
-  @doc """
-  Merges a rebar3 parent config with a child config.
-  """
-  # From https://github.com/rebar/rebar3/blob/b1da2ec0674df89599564252734bd4d794436425/src/rebar_opts.erl#L103
-  def merge_config(old, new) do
-    Keyword.merge(old, new, fn
-      :deps, old, _new               -> old
-      {:deps, _}, _old, new          -> new
-      :plugins, _old, new            -> new
-      {:plugins, _}, _old, new       -> new
-      :profiles, old, new            -> merge_config(old, new)
-      :mib_first_files, value, value -> value
-      :mib_first_files, old, new     -> old ++ new
-      :relx, old, new                -> tuple_merge(new, old)
-      _key, old, new when is_list(new) ->
-        case :io_lib.printable_list(new) do
-          true when new == [] ->
-            if :io_lib.printable_list(old), do: new, else: old
-          true ->
-            new
-          false ->
-            tuple_merge(old, new)
-        end
-      _key, _old, new -> new
-    end)
-  end
-
-  # From https://github.com/rebar/rebar3/blob/b1da2ec0674df89599564252734bd4d794436425/src/rebar_utils.erl#L282
-  defp tuple_merge(old, new),
-    do: do_tuple_merge(tuple_sort(old), tuple_sort(new))
-
-  defp do_tuple_merge(old, []),
-    do: old
-  defp do_tuple_merge(olds, [new | news]),
-    do: do_tuple_umerge_dedup(umerge(:new, olds, [], news, new), [])
-
-  defp umerge(_, [], [], acc, current),
-    do: [current | acc]
-  defp umerge(:new, [], news, acc, current),
-    do: Enum.reverse(news, [current | acc])
-  defp umerge(:old, olds, [], acc, current),
-    do: Enum.reverse(olds, [current | acc])
-  defp umerge(:new, [old | olds], news, acc, current) do
-    {dir, merged, new_current} = compare({:new, current}, {:old, old})
-    umerge(dir, olds, news, [merged | acc], new_current)
-  end
-  defp umerge(:old, olds, [new | news], acc, current) do
-    {dir, merged, new_current} = compare({:new, new}, {:old, current})
-    umerge(dir, olds, news, [merged | acc], new_current)
-  end
-
-  defp compare({priority, a}, {secondary, b}) when is_tuple(a) and is_tuple(b) do
-    ka = elem(a, 0)
-    kb = elem(b, 0)
-    cond do
-      ka == kb -> {secondary, a, b}
-      ka  < kb -> {secondary, a, b}
-      ka  > kb -> {priority, b, a}
-    end
-  end
-  defp compare({priority, a}, {secondary, b}) when not is_tuple(a) and not is_tuple(b) do
-    cond do
-      a == b -> {secondary, a, b}
-      a  < b -> {secondary, a, b}
-      a  > b -> {priority, b, a}
-    end
-  end
-  defp compare({priority, a}, {secondary, b}) when is_tuple(a) and not is_tuple(b) do
-    ka = elem(a, 0)
-    cond do
-      ka == b -> {secondary, a, b}
-      ka  < b -> {secondary, a, b}
-      ka  > b -> {priority, b, a}
-    end
-  end
-  defp compare({priority, a}, {secondary, b}) when not is_tuple(a) and is_tuple(b) do
-    kb = elem(b, 0)
-    cond do
-      a == kb -> {secondary, a, b}
-      a  < kb -> {secondary, a, b}
-      a  > kb -> {priority, b, a}
-    end
-  end
-
-  defp do_tuple_umerge_dedup([], acc), do: acc
-  defp do_tuple_umerge_dedup([h | t], acc) do
-    if h in t do
-      do_tuple_umerge_dedup(t, acc)
-    else
-      do_tuple_umerge_dedup(t, [h | acc])
-    end
-  end
-
-  defp tuple_sort(list) do
-    Enum.sort(list, fn
-      a, b when is_tuple(a) and is_tuple(b) -> elem(a, 0) <= elem(b, 0)
-      a, b when is_tuple(a) -> elem(a, 0) <= b
-      a, b when is_tuple(b) -> a <= elem(b, 0)
-      a, b -> a <= b
-    end)
-  end
-
   @doc """
   Serializes a Rebar config to a term file.
   """
@@ -178,7 +76,7 @@ defmodule Mix.Rebar do
   def dependency_config(config) do
     Enum.map(config, fn
       {:erl_opts, opts} ->
-        {:erl_opts, Enum.reject(opts, &(&1 == :warnings_as_errors))}
+        {:erl_opts, List.delete(opts, :warnings_as_errors)}
       other ->
         other
     end)
@@ -187,10 +85,9 @@ defmodule Mix.Rebar do
   @doc """
   Parses the dependencies in given `rebar.config` to Mix's dependency format.
   """
-  def deps(app, config, overrides) do
+  def deps(config) do
     # We don't have to handle rebar3 profiles because dependencies
     # are always in the default profile which cannot be customized
-    config = apply_overrides(app, config, overrides)
     if deps = config[:deps] do
       Enum.map(deps, &parse_dep/1)
     else
@@ -201,23 +98,23 @@ defmodule Mix.Rebar do
   @doc """
   Runs `fun` for the given config and for each `sub_dirs` in the
   given Rebar config.
-  """
-  def recur(config, fun) when is_binary(config) do
-    recur(load_config(config), fun)
-  end
 
+  `sub_dirs` is only supported in Rebar 2. In Rebar 3, the equivalent
+  to umbrella apps cannot be used as dependencies, so we don't need
+  to worry about such cases in Mix.
+  """
   def recur(config, fun) do
     subs =
       (config[:sub_dirs] || [])
-      |> Enum.map(&Path.wildcard(&1))
-      |> Enum.concat
+      |> Enum.flat_map(&Path.wildcard(&1))
       |> Enum.filter(&File.dir?(&1))
-      |> Enum.map(&recur(&1, fun))
-      |> Enum.concat
+      |> Enum.flat_map(&recur(load_config(&1), fun))
 
     [fun.(config) | subs]
   end
 
+  # Translate a rebar dependency declaration to a mix declaration
+  # From http://www.rebar3.org/docs/dependencies#section-declaring-dependencies
   defp parse_dep(app) when is_atom(app) do
     {app, ">= 0.0.0"}
   end
@@ -226,10 +123,6 @@ defmodule Mix.Rebar do
     {app, List.to_string(req)}
   end
 
-  defp parse_dep({app, req, {:pkg, package}}) when is_list(req) do
-    {app, List.to_string(req), hex: package}
-  end
-
   defp parse_dep({app, source}) when is_tuple(source) do
     parse_dep({app, nil, source, []})
   end
@@ -239,25 +132,33 @@ defmodule Mix.Rebar do
   end
 
   defp parse_dep({app, req, source, opts}) do
-    [scm, url | source] = Tuple.to_list(source)
-
-    ref =
-      case source do
-        ["" | _]                -> [branch: "HEAD"]
-        [{:branch, branch} | _] -> [branch: to_string(branch)]
-        [{:tag, tag} | _]       -> [tag: to_string(tag)]
-        [{:ref, ref} | _]       -> [ref: to_string(ref)]
-        [ref | _]               -> [ref: to_string(ref)]
-        _                       -> []
-      end
+    source = parse_source(source)
 
     compile =
       if :proplists.get_value(:raw, opts, false),
         do: [compile: false],
         else: []
 
-    mix_opts = [{scm, to_string(url)}] ++ ref ++ compile
-    {app, compile_req(req), mix_opts}
+    {app, compile_req(req), source ++ compile}
+  end
+
+  defp parse_source({:pkg, pkg}) do
+    [hex: pkg]
+  end
+  defp parse_source(source) do
+    [scm, url | source] = Tuple.to_list(source)
+
+    ref =
+      case source do
+        ["" | _] -> [branch: "HEAD"]
+        [{:branch, branch} | _] -> [branch: to_string(branch)]
+        [{:tag, tag} | _] -> [tag: to_string(tag)]
+        [{:ref, ref} | _] -> [ref: to_string(ref)]
+        [ref | _] -> [ref: to_string(ref)]
+        _ -> []
+      end
+
+    [{scm, to_string(url)}] ++ ref
   end
 
   defp compile_req(nil) do
@@ -265,11 +166,18 @@ defmodule Mix.Rebar do
   end
 
   defp compile_req(req) do
-    case Regex.compile(List.to_string(req)) do
-      {:ok, re} ->
-        re
-      {:error, reason} ->
-        Mix.raise "Unable to compile version regex: #{inspect req}, #{reason}"
+    req = List.to_string(req)
+
+    case Version.parse_requirement(req) do
+      {:ok, _} ->
+        req
+      :error ->
+        case Regex.compile(req) do
+          {:ok, re} ->
+            re
+          {:error, reason} ->
+            Mix.raise "Unable to compile version regex: #{inspect req}, #{reason}"
+        end
     end
   end
 
@@ -312,9 +220,11 @@ defmodule Mix.Rebar do
     end
   end
 
-  defp apply_overrides(app, config, overrides) do
+  @doc """
+  Applies the given overrides for app config.
+  """
+  def apply_overrides(app, config, overrides) do
     # Inefficient. We want the order we get here though.
-
     config =
       Enum.reduce(overrides, config, fn
         {:override, overrides}, config ->
@@ -327,7 +237,7 @@ defmodule Mix.Rebar do
 
     config =
       Enum.reduce(overrides, config, fn
-        {:override, oapp, overrides}, config when oapp == app ->
+        {:override, ^app, overrides}, config ->
           Enum.reduce(overrides, config, fn {key, value}, config ->
             Keyword.put(config, key, value)
           end)
@@ -335,14 +245,17 @@ defmodule Mix.Rebar do
           config
       end)
 
-    Enum.reduce(overrides, config, fn
-      {:add, oapp, overrides}, config when oapp == app ->
-        Enum.reduce(overrides, config, fn {key, value}, config ->
-          old_value = Keyword.get(config, key, [])
-          Keyword.put(config, key, value ++ old_value)
+    config =
+      Enum.reduce(overrides, config, fn
+        {:add, ^app, overrides}, config ->
+          Enum.reduce(overrides, config, fn {key, value}, config ->
+            old_value = Keyword.get(config, key, [])
+            Keyword.put(config, key, value ++ old_value)
+          end)
+        _, config ->
+          config
       end)
-      _, config ->
-        config
-    end)
+
+    Keyword.update(config, :overrides, overrides, &(overrides ++ &1))
   end
 end

lib/mix/lib/mix/shell/io.ex

@@ -46,31 +46,27 @@ defmodule Mix.Shell.IO do
   end
 
   @doc """
-  Prints a message and prompts the user for
-  input. Input will be consumed until Enter is pressed.
+  Prints a message and prompts the user for input.
+
+  Input will be consumed until Enter is pressed.
   """
   def prompt(message) do
     print_app()
-    IO.gets message <> " "
+    IO.gets(message <> " ")
   end
 
   @doc """
   Prints a message and asks the user if they want to proceed.
-  The user must press Enter or type anything that matches the "yes"
-  regex `~r/^Y(es)?$/i`.
+
+  The user must press Enter or type one of "y", "yes", "Y", "YES" or
+  "Yes".
   """
   def yes?(message) do
     print_app()
-    got_yes? IO.gets(message <> " [Yn] ")
+    answer = IO.gets(message <> " [Yn] ")
+    is_binary(answer) and String.trim(answer) in ["", "y", "Y", "yes", "YES", "Yes"]
   end
 
-  defp got_yes?(answer) when is_binary(answer) do
-    answer =~ ~r/^(Y(es)?)?$/i
-  end
-
-  # The IO server may return :eof or :error
-  defp got_yes?(_), do: false
-
   defp red(message) do
     [:red, :bright, message]
   end

lib/mix/lib/mix/shell/quiet.ex

@@ -32,15 +32,17 @@ defmodule Mix.Shell.Quiet do
   defdelegate error(message), to: Mix.Shell.IO
 
   @doc """
-  Prints a message and prompts the user for
-  input. Input will be consumed until Enter is pressed.
+  Prints a message and prompts the user for input.
+
+  Input will be consumed until Enter is pressed.
   """
   defdelegate prompt(message), to: Mix.Shell.IO
 
   @doc """
-  Receives a message and asks the user if they want to proceed.
-  The user must press enter or type anything that matches the "yes"
-  regex `~r/^Y(es)?$/i`.
+  Prints a message and asks the user if they want to proceed.
+
+  The user must press Enter or type one of "y", "yes", "Y", "YES" or
+  "Yes".
   """
   defdelegate yes?(message), to: Mix.Shell.IO
 end

lib/mix/lib/mix/task.ex

@@ -68,13 +68,19 @@ defmodule Mix.Task do
     # entire load path so make sure we only return unique modules.
 
     for(dir <- dirs,
-        {:ok, files} = :erl_prim_loader.list_dir(to_charlist(dir)),
-        file <- files,
+        file <- safe_list_dir(to_charlist(dir)),
         mod = task_from_path(file),
         do: mod)
     |> Enum.uniq
   end
 
+  defp safe_list_dir(path) do
+    case :erl_prim_loader.list_dir(path) do
+      {:ok, paths} -> paths
+      {:error, _} -> []
+    end
+  end
+
   @prefix_size byte_size("Elixir.Mix.Tasks.")
   @suffix_size byte_size(".beam")
 

lib/mix/lib/mix/tasks/archive.build.ex

@@ -7,14 +7,24 @@ defmodule Mix.Tasks.Archive.Build do
   Builds an archive according to the specification of the
   [Erlang Archive Format](http://www.erlang.org/doc/man/code.html).
 
+  Archives are meant to contain small projects, usually installed
+  locally. Archives may be installed into a Mix environment by
+  running `mix archive.install`. Once installed, the archive is
+  available to all Mix projects. For this reason, the functionality
+  behind archives is limited. For instance, archives do not include
+  dependencies, as those would conflict with any dependency in a
+  Mix project after the archive is installed. In general, we recommend
+  the usage of archives to be limited for extensions of Mix, such
+  as custom SCMs, package managers, etc. For general scripts to be
+  installed into machines, please see `mix escript.build`.
+
   The archive will be created in the current directory (which is
   expected to be the project root), unless an argument `-o` is
   provided with the file name.
 
-  Archives are meant to contain small projects, usually installed
-  locally.  By default, this command archives the current project
-  but the `-i` and `-o` options can be used to archive any directory.
-  For example, `mix archive.build` with no options translates to:
+  By default, this command archives the current project but the `-i`
+  option can be used to archive any directory. For example,
+  `mix archive.build` with no options translates to:
 
       mix archive.build -i _build/ENV/lib/APP -o APP-VERSION.ez
 

lib/mix/lib/mix/tasks/archive.install.ex

@@ -6,14 +6,15 @@ defmodule Mix.Tasks.Archive.Install do
   @moduledoc """
   Installs an archive locally.
 
-  If no argument is supplied but there is an archive in the project's root directory
-  (created with `mix archive.build`), then the archive will be installed
-  locally. For example:
+  If no argument is supplied but there is an archive in the project's
+  root directory (created with `mix archive.build`), then the archive
+  will be installed locally. For example:
 
       mix do archive.build, archive.install
 
-  If an argument is provided, it should be a local path or a URL to a prebuilt archive,
-  a git repository, a github repository, or a hex package.
+  If an argument is provided, it should be a local path or a URL to a
+  prebuilt archive, a git repository, a github repository, or a hex
+  package.
 
       mix archive.install archive.ez
       mix archive.install path/to/archive.ez
@@ -105,7 +106,7 @@ defmodule Mix.Tasks.Archive.Install do
 
   defp archives(name) do
     Mix.Local.path_for(:archive)
-    |> Path.join(name)
+    |> Path.join(name <> "{,*.ez}")
     |> Path.wildcard
   end
 

lib/mix/lib/mix/tasks/compile.app.ex

@@ -18,7 +18,7 @@ defmodule Mix.Tasks.Compile.App do
   function in your `mix.exs` with the following options:
 
     * `:applications` - all applications your application depends
-      on at runtime. By default, this list is automatically inflected
+      on at runtime. By default, this list is automatically inferred
       from your dependencies. Any extra Erlang/Elixir dependency
       must be specified in `:extra_applications`. Mix and other tools
       use the application list in order to start your dependencies

lib/mix/lib/mix/tasks/deps.compile.ex

@@ -191,19 +191,29 @@ defmodule Mix.Tasks.Deps.Compile do
       Mix.raise "\"#{manager}\" installation failed"
   end
 
-  defp do_make(%{opts: opts} = dep, config) do
-    os_type = :os.type
+  defp do_make(dep, config) do
+    command = make_command(dep)
+    do_command(dep, config, command, true, [{"IS_DEP", "1"}])
+  end
+
+  defp make_command(dep) do
+    makefile_win? = makefile_win?(dep)
 
     command =
-      cond do
-        match?({:win32, _}, os_type) and File.regular?(Path.join(opts[:dest], "Makefile.win")) ->
+      case :os.type do
+        {:win32, _} when makefile_win? ->
           "nmake /F Makefile.win"
-        match?({:unix, type} when type in [:freebsd, :openbsd], os_type) ->
+        {:unix, type} when type in [:freebsd, :openbsd] ->
           "gmake"
-        true ->
+        _ ->
           "make"
       end
-    do_command(dep, config, command, true, [{"IS_DEP", "1"}])
+
+    if erlang_mk?(dep) do
+      "#{command} clean && #{command}"
+    else
+      command
+    end
   end
 
   defp do_compile(%Mix.Dep{opts: opts} = dep, config) do
@@ -248,4 +258,12 @@ defmodule Mix.Tasks.Deps.Compile do
       req
     end
   end
+
+  defp erlang_mk?(%Mix.Dep{opts: opts}) do
+    File.regular?(Path.join(opts[:dest], "erlang.mk"))
+  end
+
+  defp makefile_win?(%Mix.Dep{opts: opts}) do
+    File.regular?(Path.join(opts[:dest], "Makefile.win"))
+  end
 end

lib/mix/lib/mix/tasks/escript.build.ex

@@ -197,10 +197,13 @@ defmodule Mix.Tasks.Escript.Build do
 
     extra_apps =
       if function_exported?(mod, :application, 0) do
-        mod.application[:applications]
+        app = mod.application()
+        Keyword.get(app, :applications, []) ++ Keyword.get(app, :extra_applications, [])
+      else
+        []
       end
 
-    Enum.filter(extra_apps || [], &(&1 in [:eex, :ex_unit, :mix, :iex, :logger]))
+    Enum.filter(extra_apps, &(&1 in [:eex, :ex_unit, :mix, :iex, :logger]))
   end
 
   defp app_files(app) do

lib/mix/lib/mix/tasks/loadpaths.ex

@@ -58,7 +58,7 @@ defmodule Mix.Tasks.Loadpaths do
   end
 
   defp load_project(config, _args) do
-    vsn = System.version
+    vsn = {System.version, :erlang.system_info(:otp_release)}
     scm = config[:build_scm]
 
     # Erase the app build if we have lock mismatch.

lib/mix/lib/mix/tasks/new.ex

@@ -158,7 +158,7 @@ defmodule Mix.Tasks.New do
   end
 
   defp check_application_name!(name, inferred?) do
-    unless name =~ ~r/^[a-z][\w_]*$/ do
+    unless name =~ Regex.recompile!(~r/^[a-z][a-z0-9_]*$/) do
       Mix.raise "Application name must start with a letter and have only lowercase " <>
                 "letters, numbers and underscore, got: #{inspect name}" <>
                 (if inferred? do
@@ -171,7 +171,7 @@ defmodule Mix.Tasks.New do
   end
 
   defp check_mod_name_validity!(name) do
-    unless name =~ ~r/^[A-Z]\w*(\.[A-Z]\w*)*$/ do
+    unless name =~ Regex.recompile!(~r/^[A-Z]\w*(\.[A-Z]\w*)*$/) do
       Mix.raise "Module name must be a valid Elixir alias (for example: Foo.Bar), got: #{inspect name}"
     end
   end

lib/mix/lib/mix/tasks/test.ex

@@ -122,8 +122,8 @@ defmodule Mix.Tasks.Test do
   ## Configuration
 
     * `:test_paths` - list of paths containing test files, defaults to
-      `["test"]`. It is expected all test paths to contain a `test_helper.exs`
-      file.
+      `["test"]` if the `test` directory exists, otherwise it defaults to `[]`.
+      It is expected all test paths to contain a `test_helper.exs` file.
 
     * `:test_pattern` - a pattern to load test files, defaults to `*_test.exs`.
 
@@ -225,7 +225,7 @@ defmodule Mix.Tasks.Test do
     ex_unit_opts = ex_unit_opts(opts)
     ExUnit.configure(ex_unit_opts)
 
-    test_paths = project[:test_paths] || ["test"]
+    test_paths = project[:test_paths] || default_test_paths()
     Enum.each(test_paths, &require_test_helper(&1))
     ExUnit.configure(merge_helper_opts(ex_unit_opts))
 
@@ -365,4 +365,12 @@ defmodule Mix.Tasks.Test do
       Mix.raise "Cannot run tests because test helper file #{inspect file} does not exist"
     end
   end
+
+  defp default_test_paths do
+    if File.dir?("test") do
+      ["test"]
+    else
+      []
+    end
+  end
 end

lib/mix/lib/mix/utils.ex

@@ -138,7 +138,11 @@ defmodule Mix.Utils do
 
   def extract_files(paths, pattern) do
     Enum.flat_map(paths, fn path ->
-      if File.regular?(path), do: [path], else: Path.wildcard("#{path}/**/#{pattern}")
+      case :elixir_utils.read_file_type(path) do
+        {:ok, :directory} -> Path.wildcard("#{path}/**/#{pattern}")
+        {:ok, :regular} -> [path]
+        _ -> []
+      end
     end) |> Enum.uniq
   end
 

lib/mix/test/mix/rebar_test.exs

@@ -33,212 +33,231 @@ defmodule Mix.RebarTest do
     end
   end
 
-  test "load Rebar config" do
-    path = MixTest.Case.fixture_path("rebar_dep")
-    config = Mix.Rebar.load_config(path)
-    assert config[:sub_dirs] == ['apps/*']
-    assert config[:SCRIPT] == 'rebar.config.script'
-  end
-
-  test "execute rebar.config.script on dependency directory" do
-    path = MixTest.Case.fixture_path("rebar_dep_script")
-    config = Mix.Rebar.load_config(path)
-    assert config[:dir] == {:ok, String.to_charlist(path)}
-  end
-
-  test "parse Rebar dependencies" do
-    config = [deps: [{:git_rebar, '~> 1.0'}]]
-    assert [{:git_rebar, "~> 1.0"}] ==
-           Mix.Rebar.deps(:foo, config, [])
-
-    config = [deps: [{:git_rebar, '~> 1.0', {:pkg, :rebar_fork}}]]
-    assert [{:git_rebar, "~> 1.0", hex: :rebar_fork}] ==
-           Mix.Rebar.deps(:foo, config, [])
-
-    config = [deps: [{:git_rebar, '0.1..*', {:git, '../../test/fixtures/git_rebar', :master}}]]
-    assert [{:git_rebar, ~r"0.1..*", [git: "../../test/fixtures/git_rebar", ref: "master"]}] ==
-           Mix.Rebar.deps(:foo, config, [])
-
-    config = [deps: [{:git_rebar, {:git, '../../test/fixtures/git_rebar', :master}}]]
-    assert [{:git_rebar, ">= 0.0.0", [git: "../../test/fixtures/git_rebar", ref: "master"]}] ==
-           Mix.Rebar.deps(:foo, config, [])
-
-    config = [deps: [{:git_rebar, '0.1..*', {:git, '../../test/fixtures/git_rebar'}, [:raw]}]]
-    assert [{:git_rebar, ~r"0.1..*", [git: "../../test/fixtures/git_rebar", compile: false]}] ==
-           Mix.Rebar.deps(:foo, config, [])
-
-    config = [deps: [{:git_rebar, '', {:git, '../../test/fixtures/git_rebar', {:ref, '64691eb'}}}]]
-    assert [{:git_rebar, ~r"", [git: "../../test/fixtures/git_rebar", ref: "64691eb"]}] ==
-           Mix.Rebar.deps(:foo, config, [])
-
-    config = [deps: {:git_rebar, '~> 2.0'}]
-    overrides = [{:override, [deps: [{:git_rebar, '~> 1.0'}]]}]
-    assert [{:git_rebar, "~> 1.0"}] ==
-           Mix.Rebar.deps(:foo, config, overrides)
-
-    config = [deps: [{:git_rebar, '~> 2.0'}]]
-    overrides = [{:override, [deps: [{:git_rebar, '~> 1.0'}]]}]
-    assert [{:git_rebar, "~> 1.0"}] ==
-           Mix.Rebar.deps(:foo, config, overrides)
-
-    config = [deps: [{:git_rebar, '~> 2.0'}]]
-    overrides = [{:override, :bar, [deps: [{:git_rebar, '~> 1.0'}]]}]
-    assert [{:git_rebar, "~> 2.0"}] ==
-           Mix.Rebar.deps(:foo, config, overrides)
-
-    config = [deps: [{:git_rebar, '~> 2.0'}]]
-    overrides = [{:override, :foo, [deps: [{:git_rebar, '~> 1.0'}]]}]
-    assert [{:git_rebar, "~> 1.0"}] ==
-           Mix.Rebar.deps(:foo, config, overrides)
-
-    config = [deps: [{:git_rebar, '~> 1.0'}]]
-    overrides = [{:add, :foo, [deps: [{:git_rebar2, '~> 2.0'}]]}]
-    assert [{:git_rebar2, "~> 2.0"}, {:git_rebar, "~> 1.0"}] ==
-           Mix.Rebar.deps(:foo, config, overrides)
-
-  end
-
-  test "convert rebar config to dependency config" do
-    config = Mix.Rebar.load_config(fixture_path("rebar_dep"))
-    dep_config = Mix.Rebar.dependency_config(config)
-
-    assert config[:erl_opts] == [:warnings_as_errors]
-    assert dep_config[:erl_opts] == []
-  end
-
-  test "parse Rebar dependencies from rebar.config" do
-    Mix.Project.push(RebarAsDep)
-
-    deps = Mix.Dep.loaded([])
-    assert Enum.find(deps, &(&1.app == :rebar_dep))
-
-    assert Enum.find(deps, fn %Mix.Dep{app: app, opts: opts} ->
-      if app == :git_rebar do
-        assert Enum.find(opts, &match?({:git, _}, &1))
-        assert Enum.find(opts, &match?({:ref, "master"}, &1))
-        true
-      end
-    end)
-  end
-
-  test "inherit Rebar manager" do
-    Mix.Project.push(Rebar3AsDep)
-
-    deps = Mix.Dep.loaded([])
-    assert Enum.all?(deps, &(&1.manager == :rebar3))
-  end
-
-  test "Rebar overrides" do
-    Mix.Project.push(RebarOverrideAsDep)
-
-    in_tmp "Rebar overrides", fn ->
-      Mix.Tasks.Deps.Get.run []
-      assert Mix.Dep.loaded([]) |> Enum.map(& &1.app) ==
-             [:git_repo, :git_rebar, :rebar_override]
+  describe "load_config/1" do
+    test "loads rebar.config" do
+      path = MixTest.Case.fixture_path("rebar_dep")
+      config = Mix.Rebar.load_config(path)
+      assert config[:sub_dirs] == ['apps/*']
+      assert config[:SCRIPT] == 'rebar.config.script'
+    end
+
+    test "loads rebar.config.script on dependency directory" do
+      path = MixTest.Case.fixture_path("rebar_dep_script")
+      config = Mix.Rebar.load_config(path)
+      assert config[:dir] == {:ok, String.to_charlist(path)}
     end
-  after
-    purge [GitRepo.Mixfile]
   end
 
-  test "recurs over sub dirs" do
-    path = MixTest.Case.fixture_path("rebar_dep")
+  describe "deps/1" do
+    test "parses Rebar dependencies" do
+      config = [deps: [{:git_rebar, '~> 1.0'}]]
+      assert [{:git_rebar, "~> 1.0"}] ==
+             Mix.Rebar.deps(config)
 
-    File.cd! path, fn ->
-     config = Mix.Rebar.load_config(path)
+      config = [deps: [{:git_rebar, '~> 1.0', {:pkg, :rebar_fork}}]]
+      assert [{:git_rebar, "~> 1.0", hex: :rebar_fork}] ==
+             Mix.Rebar.deps(config)
 
-      Mix.Rebar.recur(config, fn config ->
-        if config[:sub_dirs] == ['from_apps_another'] do
-          Process.put(:inside_apps_another, true)
+      config = [deps: [{:git_rebar, {:pkg, :rebar_fork}}]]
+      assert [{:git_rebar, ">= 0.0.0", hex: :rebar_fork}] ==
+             Mix.Rebar.deps(config)
+
+      config = [deps: [{:git_rebar, '0.1..*', {:git, '../../test/fixtures/git_rebar', :master}}]]
+      assert [{:git_rebar, ~r"0.1..*", [git: "../../test/fixtures/git_rebar", ref: "master"]}] ==
+             Mix.Rebar.deps(config)
+
+      config = [deps: [{:git_rebar, {:git, '../../test/fixtures/git_rebar', :master}}]]
+      assert [{:git_rebar, ">= 0.0.0", [git: "../../test/fixtures/git_rebar", ref: "master"]}] ==
+             Mix.Rebar.deps(config)
+
+      config = [deps: [{:git_rebar, '0.1..*', {:git, '../../test/fixtures/git_rebar'}, [:raw]}]]
+      assert [{:git_rebar, ~r"0.1..*", [git: "../../test/fixtures/git_rebar", compile: false]}] ==
+             Mix.Rebar.deps(config)
+
+      config = [deps: [{:git_rebar, '', {:git, '../../test/fixtures/git_rebar', {:ref, '64691eb'}}}]]
+      assert [{:git_rebar, ~r"", [git: "../../test/fixtures/git_rebar", ref: "64691eb"]}] ==
+             Mix.Rebar.deps(config)
+    end
+  end
+
+  describe "apply_overrides/3" do
+    test "applies overrides" do
+      config = [deps: {:git_rebar, '~> 2.0'}]
+      overrides = [{:override, [deps: [{:git_rebar, '~> 1.0'}]]}]
+      assert Mix.Rebar.apply_overrides(:foo, config, overrides) ==
+             [deps: [{:git_rebar, '~> 1.0'}], overrides: overrides]
+
+      config = [deps: [{:git_rebar, '~> 2.0'}]]
+      overrides = [{:override, :bar, [deps: [{:git_rebar, '~> 1.0'}]]}]
+      assert Mix.Rebar.apply_overrides(:foo, config, overrides) ==
+             [deps: [{:git_rebar, '~> 2.0'}], overrides: overrides]
+
+      config = [deps: [{:git_rebar, '~> 2.0'}]]
+      overrides = [{:override, :foo, [deps: [{:git_rebar, '~> 1.0'}]]}]
+      assert Mix.Rebar.apply_overrides(:foo, config, overrides) ==
+             [deps: [{:git_rebar, '~> 1.0'}], overrides: overrides]
+
+      config = [deps: [{:git_rebar, '~> 1.0'}]]
+      overrides = [{:add, :foo, [deps: [{:git_rebar2, '~> 2.0'}]]}]
+      assert Mix.Rebar.apply_overrides(:foo, config, overrides) ==
+             [deps: [{:git_rebar2, '~> 2.0'}, {:git_rebar, '~> 1.0'}], overrides: overrides]
+    end
+
+    test "concatenates overrides" do
+      config = [deps: {:git_rebar, '~> 2.0'}, overrides: [{:add, :bar, []}]]
+      overrides = [{:override, [deps: [{:git_rebar, '~> 1.0'}]]}]
+      assert Mix.Rebar.apply_overrides(:foo, config, overrides) ==
+             [deps: [{:git_rebar, '~> 1.0'}], overrides: overrides ++ [{:add, :bar, []}]]
+    end
+  end
+
+  describe "dependency_config/1" do
+    test "converts Rebar config to dependency config" do
+      config = Mix.Rebar.load_config(fixture_path("rebar_dep"))
+      dep_config = Mix.Rebar.dependency_config(config)
+
+      assert config[:erl_opts] == [:warnings_as_errors]
+      assert dep_config[:erl_opts] == []
+    end
+  end
+
+  describe "recur/1" do
+    test "recurs over sub dirs" do
+      path = MixTest.Case.fixture_path("rebar_dep")
+
+      File.cd! path, fn ->
+       config = Mix.Rebar.load_config(path)
+
+        Mix.Rebar.recur(config, fn config ->
+          if config[:sub_dirs] == ['from_apps_another'] do
+            Process.put(:inside_apps_another, true)
+          end
+        end)
+      end
+
+      unless Process.get(:inside_apps_another) do
+        flunk "Expected inside_apps_another to return true"
+      end
+    end
+  end
+
+  describe "integration with Mix" do
+    test "inherits Rebar manager" do
+      Mix.Project.push(Rebar3AsDep)
+      deps = Mix.Dep.loaded([])
+      assert Enum.all?(deps, &(&1.manager == :rebar3))
+    end
+
+
+    test "parses Rebar dependencies from rebar.config" do
+      Mix.Project.push(RebarAsDep)
+
+      deps = Mix.Dep.loaded([])
+      assert Enum.find(deps, &(&1.app == :rebar_dep))
+
+      assert Enum.find(deps, fn %Mix.Dep{app: app, opts: opts} ->
+        if app == :git_rebar do
+          assert Enum.find(opts, &match?({:git, _}, &1))
+          assert Enum.find(opts, &match?({:ref, "master"}, &1))
+          true
         end
       end)
     end
 
-    unless Process.get(:inside_apps_another) do
-      flunk "Expected inside_apps_another to return true"
-    end
-  end
+    test "handles Rebar overrides" do
+      Mix.Project.push(RebarOverrideAsDep)
 
-  test "get and compile dependencies for Rebar" do
-    Mix.Project.push(RebarAsDep)
-
-    in_tmp "get and compile dependencies for Rebar", fn ->
-      Mix.Tasks.Deps.Get.run []
-      assert_received {:mix_shell, :info, ["* Getting git_rebar" <> _]}
-
-      Mix.Tasks.Deps.Compile.run []
-      assert_received {:mix_shell, :run, ["===> Compiling git_rebar\n"]}
-      assert_received {:mix_shell, :run, ["===> Compiling rebar_dep\n"]}
-      assert :git_rebar.any_function == :ok
-      assert :rebar_dep.any_function == :ok
-
-      load_paths = Mix.Dep.loaded([])
-        |> Enum.map(&Mix.Dep.load_paths(&1))
-        |> Enum.concat
-
-      assert File.exists?("_build/dev/lib/rebar_dep/ebin/rebar_dep.beam")
-      assert File.exists?("_build/dev/lib/git_rebar/ebin/git_rebar.beam")
-
-      # Assert we have no .compile.lock as a .compile.lock
-      # means we check for the Elixir version on every command.
-      refute File.exists?("_build/dev/lib/rebar_dep/.compile.lock")
-      refute File.exists?("_build/dev/lib/git_rebar/.compile.lock")
-
-      assert Enum.any?(load_paths, &String.ends_with?(&1, "git_rebar/ebin"))
-      assert Enum.any?(load_paths, &String.ends_with?(&1, "rebar_dep/ebin"))
-    end
-  end
-
-  test "get and compile dependencies for rebar3" do
-    Mix.Project.push(Rebar3AsDep)
-
-    in_tmp "get and compile dependencies for rebar3", fn ->
-      Mix.Tasks.Deps.Get.run []
-      assert_received {:mix_shell, :info, ["* Getting git_rebar " <> _]}
-
-      Mix.Tasks.Deps.Compile.run []
-      assert_received {:mix_shell, :run, ["===> Compiling git_rebar\n"]}
-      assert_received {:mix_shell, :run, ["===> Compiling rebar_dep\n"]}
-      assert :git_rebar.any_function == :ok
-      assert :rebar_dep.any_function == :ok
-
-      load_paths = Mix.Dep.loaded([])
-        |> Enum.map(&Mix.Dep.load_paths(&1))
-        |> Enum.concat
-
-      assert File.exists?("_build/dev/lib/rebar_dep/ebin/rebar_dep.beam")
-      assert File.exists?("_build/dev/lib/git_rebar/ebin/git_rebar.beam")
-
-      # Assert we have no .compile.lock as a .compile.lock
-      # means we check for the Elixir version on every command.
-      refute File.exists?("_build/dev/lib/rebar_dep/.compile.lock")
-      refute File.exists?("_build/dev/lib/git_rebar/.compile.lock")
-
-      assert Enum.any?(load_paths, &String.ends_with?(&1, "git_rebar/ebin"))
-      assert Enum.any?(load_paths, &String.ends_with?(&1, "rebar_dep/ebin"))
-    end
-  end
-
-  test "get and compile dependencies for Rebar with Mix" do
-    Mix.Project.push(RebarAsDep)
-
-    in_tmp "get and compile dependencies for Rebar with Mix", fn ->
-      File.write! MixTest.Case.tmp_path("rebar_dep/mix.exs"), """
-      defmodule RebarDep.Mixfile do
-        use Mix.Project
-
-        def project do
-          [app: :rebar_dep,
-           version: "0.0.1"]
-        end
+      in_tmp "Rebar overrides", fn ->
+        Mix.Tasks.Deps.Get.run []
+        assert Mix.Dep.loaded([]) |> Enum.map(& &1.app) ==
+               [:git_repo, :git_rebar, :rebar_override]
       end
-      """
+    after
+      purge [GitRepo.Mixfile]
+    end
 
-      Mix.Tasks.Deps.Compile.run []
-      assert_received {:mix_shell, :info, ["==> rebar_dep"]}
-      assert_received {:mix_shell, :info, ["Generated rebar_dep app"]}
-      assert File.regular?("_build/dev/lib/rebar_dep/ebin/rebar_dep.app")
+    test "gets and compiles dependencies for Rebar" do
+      Mix.Project.push(RebarAsDep)
+
+      in_tmp "get and compile dependencies for Rebar", fn ->
+        Mix.Tasks.Deps.Get.run []
+        assert_received {:mix_shell, :info, ["* Getting git_rebar" <> _]}
+
+        Mix.Tasks.Deps.Compile.run []
+        assert_received {:mix_shell, :run, ["===> Compiling git_rebar\n"]}
+        assert_received {:mix_shell, :run, ["===> Compiling rebar_dep\n"]}
+        assert :git_rebar.any_function == :ok
+        assert :rebar_dep.any_function == :ok
+
+        load_paths = Mix.Dep.loaded([])
+          |> Enum.map(&Mix.Dep.load_paths(&1))
+          |> Enum.concat
+
+        assert File.exists?("_build/dev/lib/rebar_dep/ebin/rebar_dep.beam")
+        assert File.exists?("_build/dev/lib/git_rebar/ebin/git_rebar.beam")
+
+        # Assert we have no .compile.lock as a .compile.lock
+        # means we check for the Elixir version on every command.
+        refute File.exists?("_build/dev/lib/rebar_dep/.compile.lock")
+        refute File.exists?("_build/dev/lib/git_rebar/.compile.lock")
+
+        assert Enum.any?(load_paths, &String.ends_with?(&1, "git_rebar/ebin"))
+        assert Enum.any?(load_paths, &String.ends_with?(&1, "rebar_dep/ebin"))
+      end
+    end
+
+    test "gets and compiles dependencies for rebar3" do
+      Mix.Project.push(Rebar3AsDep)
+
+      in_tmp "get and compile dependencies for rebar3", fn ->
+        Mix.Tasks.Deps.Get.run []
+        assert_received {:mix_shell, :info, ["* Getting git_rebar " <> _]}
+
+        Mix.Tasks.Deps.Compile.run []
+        assert_received {:mix_shell, :run, ["===> Compiling git_rebar\n"]}
+        assert_received {:mix_shell, :run, ["===> Compiling rebar_dep\n"]}
+        assert :git_rebar.any_function == :ok
+        assert :rebar_dep.any_function == :ok
+
+        load_paths = Mix.Dep.loaded([])
+          |> Enum.map(&Mix.Dep.load_paths(&1))
+          |> Enum.concat
+
+        assert File.exists?("_build/dev/lib/rebar_dep/ebin/rebar_dep.beam")
+        assert File.exists?("_build/dev/lib/git_rebar/ebin/git_rebar.beam")
+
+        # Assert we have no .compile.lock as a .compile.lock
+        # means we check for the Elixir version on every command.
+        refute File.exists?("_build/dev/lib/rebar_dep/.compile.lock")
+        refute File.exists?("_build/dev/lib/git_rebar/.compile.lock")
+
+        assert Enum.any?(load_paths, &String.ends_with?(&1, "git_rebar/ebin"))
+        assert Enum.any?(load_paths, &String.ends_with?(&1, "rebar_dep/ebin"))
+      end
+    end
+
+    test "gets and compiles dependencies for Rebar with Mix" do
+      Mix.Project.push(RebarAsDep)
+
+      in_tmp "get and compile dependencies for Rebar with Mix", fn ->
+        File.write! MixTest.Case.tmp_path("rebar_dep/mix.exs"), """
+        defmodule RebarDep.Mixfile do
+          use Mix.Project
+
+          def project do
+            [app: :rebar_dep,
+             version: "0.0.1"]
+          end
+        end
+        """
+
+        Mix.Tasks.Deps.Compile.run []
+        assert_received {:mix_shell, :info, ["==> rebar_dep"]}
+        assert_received {:mix_shell, :info, ["Generated rebar_dep app"]}
+        assert File.regular?("_build/dev/lib/rebar_dep/ebin/rebar_dep.app")
+      end
+    after
+      File.rm MixTest.Case.tmp_path("rebar_dep/mix.exs")
     end
-  after
-    File.rm MixTest.Case.tmp_path("rebar_dep/mix.exs")
   end
 end

lib/mix/test/mix/tasks/archive_test.exs

@@ -52,6 +52,11 @@ defmodule Mix.Tasks.ArchiveTest do
       archive = tmp_path("userhome/.mix/archives/archive-0.1.0/archive-0.1.0/ebin")
       assert to_charlist(archive) in :code.get_path
 
+      # Try to override it with URL
+      send self(), {:mix_shell_input, :yes?, false}
+      Mix.Tasks.Archive.Install.run ["https://example.com/archive-0.1.0?hello.ez"]
+      assert_received {:mix_shell, :yes?, ["Found existing archive: " <> _]}
+
       # Loading the archive should emit warning again
       Mix.Local.append_archives
       assert_received {:mix_shell, :error, [^version_error]}

lib/mix/test/mix/tasks/compile.app_test.exs

@@ -85,7 +85,7 @@ defmodule Mix.Tasks.Compile.AppTest do
     end
   end
 
-  test "automatically inflects applications" do
+  test "automatically infers applications" do
     Mix.Project.push CustomDeps
 
     in_fixture "no_mixfile", fn ->

lib/mix/test/mix/tasks/compile.elixir_test.exs

@@ -8,6 +8,8 @@ defmodule Mix.Tasks.Compile.ElixirTest do
     :ok
   end
 
+  @elixir_otp_version {System.version, :erlang.system_info(:otp_release)}
+
   test "compiles a project without per environment build" do
     Mix.Project.pop
     Mix.ProjectStack.post_config [build_per_environment: false]
@@ -43,7 +45,7 @@ defmodule Mix.Tasks.Compile.ElixirTest do
 
       assert File.exists?("_build/dev/lib/sample")
       assert File.exists?("_build/dev/consolidated")
-      assert Mix.Dep.ElixirSCM.read == {:ok, System.version, Mix.SCM.Path}
+      assert Mix.Dep.ElixirSCM.read == {:ok, @elixir_otp_version, Mix.SCM.Path}
 
       Mix.Task.clear
       File.write!("_build/dev/consolidated/.to_be_removed", "")
@@ -52,7 +54,7 @@ defmodule Mix.Tasks.Compile.ElixirTest do
       File.touch!("_build/dev/lib/sample/.compile.elixir_scm", {{2010, 1, 1}, {0, 0, 0}})
 
       Mix.Tasks.Compile.run []
-      assert Mix.Dep.ElixirSCM.read == {:ok, System.version, Mix.SCM.Path}
+      assert Mix.Dep.ElixirSCM.read == {:ok, @elixir_otp_version, Mix.SCM.Path}
       assert File.stat!("_build/dev/lib/sample/.compile.elixir_scm").mtime > {{2010, 1, 1}, {0, 0, 0}}
       refute File.exists?("_build/dev/consolidated/.to_be_removed")
     end
@@ -64,15 +66,15 @@ defmodule Mix.Tasks.Compile.ElixirTest do
       purge [A, B]
 
       assert_received {:mix_shell, :info, ["Compiled lib/a.ex"]}
-      assert Mix.Dep.ElixirSCM.read == {:ok, System.version, Mix.SCM.Path}
+      assert Mix.Dep.ElixirSCM.read == {:ok, @elixir_otp_version, Mix.SCM.Path}
 
       Mix.Task.clear
-      manifest_data = :erlang.term_to_binary({:v1, System.version, :another})
+      manifest_data = :erlang.term_to_binary({:v2, @elixir_otp_version, :another})
       File.write!("_build/dev/lib/sample/.compile.elixir_scm", manifest_data)
       File.touch!("_build/dev/lib/sample/.compile.elixir_scm", {{2010, 1, 1}, {0, 0, 0}})
 
       Mix.Tasks.Compile.run []
-      assert Mix.Dep.ElixirSCM.read == {:ok, System.version, Mix.SCM.Path}
+      assert Mix.Dep.ElixirSCM.read == {:ok, @elixir_otp_version, Mix.SCM.Path}
       assert File.stat!("_build/dev/lib/sample/.compile.elixir_scm").mtime > {{2010, 1, 1}, {0, 0, 0}}
     end
   end

lib/mix/test/mix/tasks/deps_test.exs

@@ -571,7 +571,7 @@ defmodule Mix.Tasks.DepsTest do
       Mix.Tasks.Deps.Loadpaths.run []
 
       File.mkdir_p!("_build/dev/lib/ok/ebin")
-      manifest_data = :erlang.term_to_binary({:v1, System.version, :scm})
+      manifest_data = :erlang.term_to_binary({:v2, {System.version, :erlang.system_info(:otp_release)}, :scm})
       File.write!("_build/dev/lib/ok/.compile.elixir_scm", manifest_data)
       Mix.Task.clear
 

lib/mix/test/mix/tasks/escript_test.exs

@@ -47,7 +47,7 @@ defmodule Mix.Tasks.EscriptTest do
     end
 
     def application do
-      [applications: []]
+      [applications: [], extra_applications: [:crypto]]
     end
   end