Release v1.4.2

Allow consuming multiple items from suspended enumerable in Stream.transform/3
Closes #5763. Closes #5772. Signed-off-by: José Valim <jose.valim@plataformatec.com.br>
2017-02-16 15:15:31 +01:00 · 2017-02-16 13:50:16 +01:00 · 2017-02-13 15:01:34 +01:00 · 2017-02-09 20:41:50 +01:00 · 2017-02-05 11:06:37 +01:00 · 2017-02-01 23:19:33 +01:00
35 changed files with 762 additions and 612 deletions
@@ -109,6 +109,51 @@ You can now also install archives from Hex in this way. Since they are fetched a

 It is also possible to install escripts and archives by providing a Git/GitHub repo. See `mix help escript.install` and `mix help archive.install` for more details.

+## v1.4.2 (2017-02-16)
+
+### 1. Bug fixes
+
+#### EEx
+
+  * [EEx] Support middle expressions on trim mode
+
+#### Elixir
+
+  * [Calendar] Correct typo on Calendar types
+  * [Kernel] Ensure redefined functions point to the proper source
+  * [OptionParser] Add `:allow_inexistent_atoms` to support unsafe behaviour prior to v1.4
+  * [Stream] Allow consuming multiple items from suspended enumerable in `Stream.transform/3`
+  * [String] Incorporate new grapheme rules in Unicode 9
+
+#### IEx
+
+  * [IEx.Autocomplete] Do not crash on aliases which are not known at compile time
+
+#### Mix
+
+  * [Mix.Umbrella] Ensure umbrella projects can depend on other umbrella projects
+  * [Mix.Archive] Ensure previous archives with `.ez` extension are deleted
+
+## v1.4.1 (2017-01-26)
+
+### 1. Bug fixes
+
+#### Elixir
+
+  * [Kernel] Remove warning when making private functions overridable
+  * [Path] Ensure `Path.join/1` returns strings for lists of one element
+  * [Regex] Ensure `Regex.escape/1` also escapes `-`
+
+#### IEx
+
+  * [IEx] Disable ANSI detection for powershell to avoid false positives
+
+#### Mix
+
+  * [Mix.Make] Run `make clean` for `erlang.mk`
+  * [Mix.Rebar] Support all of rebar3 dependency package declaration
+  * [Mix.Rebar] Only pass overrides from parent to child in Rebar dep
+
 ## v1.4.0 (2017-01-05)

 ### 1. Enhancements
@@ -119,6 +164,7 @@ It is also possible to install escripts and archives by providing a Git/GitHub r
  * [Calendar] Support `NaiveDateTime.add/3` and `NaiveDateTime.diff/3` for adding seconds (up to microseconds) as well as the difference between two NaiveDateTimes in seconds (up to microseconds)
  * [Calendar] Add `Date.leap_year?/1` and `Date.day_of_week/1`
  * [Calendar] Ensure `Date`, `Time` and `NaiveDateTime` APIs work with any struct that provides the same set of fields as their respective struct. For example, a `NaiveDateTime` can be given to `Date` since it contains a superset of the fields in the `Date` struct
+  * [Calendar] Add `Time.utc_now/0` and `NaiveDateTime.utc_now/0`
  * [Enum] Add `Enum.map_every/2` that invokes the given function with every nth item
  * [Enum] Add `min/2`, `max/2`, `min_max/2`, `min_by/3`, `max_by/3`, and `min_max_by/3` that allow a function specifying the default value when the enumerable is empty
  * [Enum] Introduce `Enum.zip/1` to zip multiple entries at once
@@ -22,7 +22,7 @@ This document simply outlines the release process:

 9. Publish new zips with `make zips`, upload `Precompiled.zip` and `Docs.zip` to GitHub Releases

-10. Add the release to `elixir.csv` file in `elixir-lang/elixir-lang.github.com`
+10. Add the release to `elixir.csv` and `_data/elixir-versions.yml` files in `elixir-lang/elixir-lang.github.com`

 11. After a new `vMAJOR.MINOR`, move back to master, bump versions, start new CHANGELOG, add `-dev` back and commit "Start vMAJOR.MINOR+1"

@@ -1 +1 @@
-1.4.0
+1.4.2
@@ -100,27 +100,6 @@ for  /d %%d in ("%originPath%..\lib\*.") do (
 )
 setlocal disabledelayedexpansion

-rem ******* detect ANSI terminal support ********************
-timeout 0 2>nul >nul || goto run
-where /Q powershell || goto run
-
-set ASSERT_ANSI= ^
-  $err = 1; ^
-  $Kernel32 = Add-Type -Name 'Kernel32' -PassThru -MemberDefinition ' ^
-    [DllImport(\"Kernel32.dll\", SetLastError = true)] ^
-    public static extern IntPtr GetStdHandle(int nStdHandle); ^
-    [DllImport(\"Kernel32.dll\", SetLastError = true)] ^
-    public static extern bool GetConsoleMode(IntPtr hWnd, ref UInt32 lpMode); ^
-  '; ^
-  $StdoutHandle = $Kernel32::GetStdHandle(-11); ^
-  $ConsoleMode = New-Object -TypeName UInt32; ^
-  $null = $Kernel32::GetConsoleMode($StdoutHandle, [ref]$ConsoleMode); ^
-  if ($ConsoleMode -band 0x4) { $err = 0 } ^
-  exit $err
-
-powershell -NoProfile -NonInteractive -Command %ASSERT_ANSI% || goto run
-set parsErlang=%parsErlang% -elixir ansi_enabled true
-
 :run
 if not %runMode% == "iex" (
  set beforeExtra=-noshell -s elixir start_cli %beforeExtra%
@@ -90,7 +90,9 @@ defmodule EEx.Compiler do
      {contents, start, list}
    end
  end
-
+  defp look_ahead_text([{:middle_expr, line, _, chars} | t], _start, contents) do
+    {contents ++ chars, line, t}
+  end
  defp look_ahead_text(t, start, contents) do
    {contents, start, t}
  end
@@ -181,7 +181,6 @@ defmodule EEx.Tokenizer do
  # If trim mode is enabled and the token is on a line with
  # only itself and whitespace, trim the whitespace around it,
  # including the line break following it if there is one.
-
  defp trim_if_needed(rest, line, opts, buffer, acc) do
    original = {rest, line, buffer}
    if opts[:trim] do
@@ -331,6 +331,19 @@ foo
    assert_eval expected, string, [], trim: true
  end

+  test "trim mode with middle expression" do
+    string = """
+    <%= cond do %>
+    <% false -> %>
+      this
+    <% true -> %>
+      that
+    <% end %>
+    """
+    expected = "  that\n"
+    assert_eval expected, string, [], trim: true
+  end
+
  test "evaluates the source from a given file" do
    filename = Path.join(__DIR__, "fixtures/eex_template.eex")
    result = EEx.eval_file(filename)
@@ -51,19 +51,19 @@ defmodule Calendar do
  @type std_offset  :: integer

  @typedoc "Any map/struct that contains the date fields"
-  @type date :: %{calendar: calendar, year: year, month: month, day: day}
+  @type date :: %{optional(any) => any, calendar: calendar, year: year, month: month, day: day}

  @typedoc "Any map/struct that contains the time fields"
-  @type time :: %{hour: hour, minute: minute, second: second, microsecond: microsecond}
+  @type time :: %{optional(any) => any, hour: hour, minute: minute, second: second, microsecond: microsecond}

  @typedoc "Any map/struct that contains the naive_datetime fields"
-  @type naive_date_time :: %{calendar: calendar, year: year, month: month, day: day,
-                             hour: hour, minute: minute, second: second, microsecond: microsecond}
+  @type naive_datetime :: %{optional(any) => any, calendar: calendar, year: year, month: month, day: day,
+                            hour: hour, minute: minute, second: second, microsecond: microsecond}

  @typedoc "Any map/struct that contains the datetime fields"
-  @type date_time :: %{calendar: calendar, year: year, month: month, day: day,
-                       hour: hour, minute: minute, second: second, microsecond: microsecond,
-                       time_zone: time_zone, zone_abbr: zone_abbr, utc_offset: utc_offset, std_offset: std_offset}
+  @type datetime :: %{optional(any) => any, calendar: calendar, year: year, month: month, day: day,
+                      hour: hour, minute: minute, second: second, microsecond: microsecond,
+                      time_zone: time_zone, zone_abbr: zone_abbr, utc_offset: utc_offset, std_offset: std_offset}

  @doc """
  Returns how many days there are in the given year-month.
@@ -808,13 +808,6 @@ defmodule Module do
            "cannot make private macro #{name}/#{arity} overridable, overriding " <>
            "private macros is not supported"
        clause ->
-          {{_def, kind, _line, _file, _check, _location, _defaults}, _clauses} = clause
-
-          # TODO: Remove on v2.0
-          if kind == :defp do
-            IO.warn "making private functions (#{name}/#{arity} in this case) overridable is deprecated"
-          end
-
          neighbours =
            if :elixir_compiler.get_opt(:internal) do
              []
@@ -30,35 +30,33 @@ defmodule OptionParser do
  Elixir converts switches to underscored atoms, so `--source-path` becomes
  `:source_path`. This is done to better suit Elixir conventions. However, this
  means that switches can't contain underscores and switches that do contain
-  underscores are always returned in the list of invalid options.
+  underscores are always returned in the list of invalid switches.

-  Without any options, this function will try to parse all switches in the `argv`.
+  When parsing, it is common to list switches and their expected types:

-      iex> OptionParser.parse(["--debug"])
+      iex> OptionParser.parse(["--debug"], switches: [debug: :boolean])
      {[debug: true], [], []}

-      iex> OptionParser.parse(["--source", "lib"])
+      iex> OptionParser.parse(["--source", "lib"], switches: [source: :string])
      {[source: "lib"], [], []}

-      iex> OptionParser.parse(["--source-path", "lib", "test/enum_test.exs", "--verbose"])
+      iex> OptionParser.parse(["--source-path", "lib", "test/enum_test.exs", "--verbose"],
+      ...>                    switches: [source_path: :string, verbose: :boolean])
      {[source_path: "lib", verbose: true], ["test/enum_test.exs"], []}

-  Switches followed by a value will be assigned the value, as a string.
-  Switches without an argument, like `--debug` in the examples above, will
-  automatically be set to `true`.
+  We will explore the valid switches and operation modes of option parser below.

  ## Options

  The following options are supported:

    * `:switches` or `:strict` - see the "Switch definitions" section below
+    * `:allow_nonexistent_atoms` - see the "Parsing dynamic switches" section below
    * `:aliases` - see the "Aliases" section below

  ## Switch definitions

-  Often it is better to explicitly list the known
-  switches and their formats. The switches can be specified via one of two
-  options:
+  Switches can be specified via one of two options:

    * `:switches` - defines some switches and their types. This function
      still attempts to parse switches that are not in this list.
@@ -89,20 +87,20 @@ defmodule OptionParser do
    * `:float` - parses the value as a float
    * `:string` - parses the value as a string

-  If a switch can't be parsed according to the given type, it is returned
-  in the invalid options list.
+  If a switch can't be parsed according to the given type, it is
+  returned in the invalid options list.

  ### Modifiers

  Switches can be specified with modifiers, which change how
  they behave. The following modifiers are supported:

-    * `:keep` - keeps duplicated items instead of overriding them; works with
-      all types except `:count`. Specifying `switch_name: :keep` assumes the
-      type of `:switch_name` will be `:string`.
+    * `:keep` - keeps duplicated items instead of overriding them;
+      works with all types except `:count`. Specifying `switch_name: :keep`
+      assumes the type of `:switch_name` will be `:string`.

-  Note that if you want to use `:keep` with a type other than `:string`, use a list
-  as the type for the switch. For example: `[foo: [:integer, :keep]]`.
+  To use `:keep` with a type other than `:string`, use a list as the type
+  for the switch. For example: `[foo: [:integer, :keep]]`.

  ### Negation switches

@@ -112,6 +110,46 @@ defmodule OptionParser do
      iex> OptionParser.parse(["--no-op", "path/to/file"], switches: [op: :boolean])
      {[op: false], ["path/to/file"], []}

+  ### Parsing dynamic switches
+
+  `OptionParser` also includes a dynamic mode where it will attempt to parse
+  switches dynamically. Such can be done by not specifying the `:switches` or
+  `:strict` option.
+
+      iex> OptionParser.parse(["--debug"])
+      {[debug: true], [], []}
+
+
+  Switches followed by a value will be assigned the value, as a string. Switches
+  without an argument, like `--debug` in the examples above, will automatically be
+  set to `true`.
+
+  Since Elixir converts switches to atoms, the dynamic mode will only parse
+  switches that translates to atoms used by the runtime. Therefore, the code below
+  likely won't parse the given option since the `:option_parser_example` atom is
+  never used anywhere:
+
+      OptionParser.parse(["--option-parser-example"])
+      # Does nothing more...
+
+  However, the code below does since the `:option_parser_example` atom is used
+  at some point later (or earlier) on:
+
+      {opts, _, _} = OptionParser.parse(["--option-parser-example"])
+      opts[:option_parser_example]
+
+  In other words, when using dynamic mode, Elixir will do the correct thing and
+  only parse options that are used by the runtime, ignoring all others. If you
+  would like to parse all switches, regardless if they exist or not, you can
+  force creation of atoms by passing `allow_nonexistent_atoms: true` as option.
+  Such option is useful when you are building command-line applications that
+  receive dynamically-named arguments but must be used with care on long-running
+  systems.
+
+  Switches followed by a value will be assigned the value, as a string.
+  Switches without an argument, like `--debug` in the examples above, will
+  automatically be set to `true`.
+
  ## Aliases

  A set of aliases can be specified in the `:aliases` option:
@@ -203,10 +241,12 @@ defmodule OptionParser do

  ## Example

-      iex> OptionParser.parse_head(["--source", "lib", "test/enum_test.exs", "--verbose"])
+      iex> OptionParser.parse_head(["--source", "lib", "test/enum_test.exs", "--verbose"],
+      ...>                         switches: [source: :string, verbose: :boolean])
      {[source: "lib"], ["test/enum_test.exs", "--verbose"], []}

-      iex> OptionParser.parse_head(["--verbose", "--source", "lib", "test/enum_test.exs", "--unlock"])
+      iex> OptionParser.parse_head(["--verbose", "--source", "lib", "test/enum_test.exs", "--unlock"],
+      ...>                         switches: [source: :string, verbose: :boolean, unlock: :boolean])
      {[verbose: true, source: "lib"], ["test/enum_test.exs", "--unlock"], []}

  """
@@ -226,10 +266,12 @@ defmodule OptionParser do

  ## Examples

-      iex> OptionParser.parse_head!(["--source", "lib", "path/to/file", "--verbose"])
+      iex> OptionParser.parse_head!(["--source", "lib", "path/to/file", "--verbose"],
+      ...>                         switches: [source: :string, verbose: :boolean])
      {[source: "lib"], ["path/to/file", "--verbose"]}

-      iex> OptionParser.parse_head!(["--number", "lib", "test/enum_test.exs", "--verbose"], strict: [number: :integer])
+      iex> OptionParser.parse_head!(["--number", "lib", "test/enum_test.exs", "--verbose"],
+      ...>                          strict: [number: :integer])
      ** (OptionParser.ParseError) 1 error found!
      --number : Expected type integer, got "lib"

@@ -251,8 +293,8 @@ defmodule OptionParser do
    {Enum.reverse(opts), Enum.reverse(args), Enum.reverse(invalid)}
  end

-  defp do_parse(argv, {aliases, switches, strict?} = config, opts, args, invalid, all?) do
-    case next(argv, aliases, switches, strict?) do
+  defp do_parse(argv, {aliases, switches, strict?, allow_nonexistent_atoms?} = config, opts, args, invalid, all?) do
+    case next(argv, aliases, switches, strict?, allow_nonexistent_atoms?) do
      {:ok, option, value, rest} ->
        # the option exists and it was successfully parsed
        kinds = List.wrap Keyword.get(switches, option)
@@ -299,7 +341,6 @@ defmodule OptionParser do
    * `{:error, rest}` - there are no switches at the head of the given `argv`

  """
-
  @spec next(argv, options) ::
        {:ok, key :: atom, value :: term, argv} |
        {:invalid, String.t, String.t | nil, argv} |
@@ -307,35 +348,35 @@ defmodule OptionParser do
        {:error, argv}

  def next(argv, opts \\ []) when is_list(argv) and is_list(opts) do
-    {aliases, switches, strict?} = compile_config(opts)
-    next(argv, aliases, switches, strict?)
+    {aliases, switches, strict?, allow_nonexistent_atoms?} = compile_config(opts)
+    next(argv, aliases, switches, strict?, allow_nonexistent_atoms?)
  end

-  defp next([], _aliases, _switches, _strict?) do
+  defp next([], _aliases, _switches, _strict?, _allow_nonexistent_atoms?) do
    {:error, []}
  end

-  defp next(["--" | _] = argv, _aliases, _switches, _strict?) do
+  defp next(["--" | _] = argv, _aliases, _switches, _strict?, _allow_nonexistent_atoms?) do
    {:error, argv}
  end

-  defp next(["-" | _] = argv, _aliases, _switches, _strict?) do
+  defp next(["-" | _] = argv, _aliases, _switches, _strict?, _allow_nonexistent_atoms?) do
    {:error, argv}
  end

-  defp next(["- " <> _ | _] = argv, _aliases, _switches, _strict?) do
+  defp next(["- " <> _ | _] = argv, _aliases, _switches, _strict?, _allow_nonexistent_atoms?) do
    {:error, argv}
  end

  # Handles --foo or --foo=bar
-  defp next(["--" <> option | rest], _aliases, switches, strict?) do
+  defp next(["--" <> option | rest], _aliases, switches, strict?, allow_nonexistent_atoms?) do
    {option, value} = split_option(option)
-    tagged = tag_option(option, switches)
-    do_next(tagged, value, "--" <> option, rest, switches, strict?)
+    tagged = tag_option(option, switches, allow_nonexistent_atoms?)
+    do_next(tagged, value, "--" <> option, rest, switches, strict?, allow_nonexistent_atoms?)
  end

  # Handles -a, -abc, -abc=something
-  defp next(["-" <> option | rest] = argv, aliases, switches, strict?) do
+  defp next(["-" <> option | rest] = argv, aliases, switches, strict?, allow_nonexistent_atoms?) do
    {option, value} = split_option(option)
    original = "-" <> option

@@ -345,26 +386,26 @@ defmodule OptionParser do
      String.contains?(option, ["-", "_"]) ->
        {:undefined, original, value, rest}
      String.length(option) > 1 ->
-        key = get_option_key(option)
+        key = get_option_key(option, allow_nonexistent_atoms?)
        option_key = aliases[key]
        if key && option_key do
          IO.warn "multi-letter aliases are deprecated, got: #{inspect(key)}"
-          do_next({:default, option_key}, value, original, rest, switches, strict?)
+          do_next({:default, option_key}, value, original, rest, switches, strict?, allow_nonexistent_atoms?)
        else
-          next(expand_multiletter_alias(option, value) ++ rest, aliases, switches, strict?)
+          next(expand_multiletter_alias(option, value) ++ rest, aliases, switches, strict?, allow_nonexistent_atoms?)
        end
      true ->
        # We have a regular one-letter alias here
-        tagged = tag_oneletter_alias(option, aliases)
-        do_next(tagged, value, original, rest, switches, strict?)
+        tagged = tag_oneletter_alias(option, aliases, allow_nonexistent_atoms?)
+        do_next(tagged, value, original, rest, switches, strict?, allow_nonexistent_atoms?)
    end
  end

-  defp next(argv, _aliases, _switches, _strict?) do
+  defp next(argv, _aliases, _switches, _strict?, _allow_nonexistent_atoms?) do
    {:error, argv}
  end

-  defp do_next(tagged, value, original, rest, switches, strict?) do
+  defp do_next(tagged, value, original, rest, switches, strict?, allow_nonexistent_atoms?) do
    if strict? and not option_defined?(tagged, switches) do
      {:undefined, original, value, rest}
    else
@@ -489,6 +530,7 @@ defmodule OptionParser do

  defp compile_config(opts) do
    aliases = opts[:aliases] || []
+    allow_nonexistent_atoms? = opts[:allow_nonexistent_atoms] || false

    {switches, strict?} = cond do
      opts[:switches] && opts[:strict] ->
@@ -501,7 +543,7 @@ defmodule OptionParser do
        {[], false}
    end

-    {aliases, switches, strict?}
+    {aliases, switches, strict?, allow_nonexistent_atoms?}
  end

  defp validate_option(value, kinds) do
@@ -552,27 +594,27 @@ defmodule OptionParser do
    end
  end

-  defp tag_option("no-" <> option = original, switches) do
+  defp tag_option("no-" <> option = original, switches, allow_nonexistent_atoms?) do
    cond do
-      (negated = get_option_key(option)) && :boolean in List.wrap(switches[negated]) ->
+      (negated = get_option_key(option, allow_nonexistent_atoms?)) && :boolean in List.wrap(switches[negated]) ->
        {:negated, negated}
-      option_key = get_option_key(original) ->
+      option_key = get_option_key(original, allow_nonexistent_atoms?) ->
        {:default, option_key}
      true ->
        :unknown
    end
  end

-  defp tag_option(option, _switches) do
-    if option_key = get_option_key(option) do
+  defp tag_option(option, _switches, allow_nonexistent_atoms?) do
+    if option_key = get_option_key(option, allow_nonexistent_atoms?) do
      {:default, option_key}
    else
      :unknown
    end
  end

-  defp tag_oneletter_alias(alias, aliases) when is_binary(alias) do
-    if option_key = aliases[to_existing_key(alias)] do
+  defp tag_oneletter_alias(alias, aliases, allow_nonexistent_atoms?) when is_binary(alias) do
+    if option_key = aliases[to_existing_key(alias, allow_nonexistent_atoms?)] do
      {:default, option_key}
    else
      :unknown
@@ -662,13 +704,15 @@ defmodule OptionParser do
  defp to_underscore(<<>>, acc),
    do: acc

-  def get_option_key(option) do
+  def get_option_key(option, allow_nonexistent_atoms?) do
    if string = to_underscore(option) do
-      to_existing_key(string)
+      to_existing_key(string, allow_nonexistent_atoms?)
    end
  end

-  defp to_existing_key(option) do
+  defp to_existing_key(option, true),
+    do: String.to_atom(option)
+  defp to_existing_key(option, false) do
    try do
      String.to_existing_atom(option)
    rescue
@@ -702,7 +746,8 @@ defmodule OptionParser do
  end

  defp get_type(option, opts, types) do
-    key = option |> String.trim_leading("-") |> get_option_key()
+    allow_nonexistent_atoms? = opts[:allow_nonexistent_atoms] || false
+    key = option |> String.trim_leading("-") |> get_option_key(allow_nonexistent_atoms?)

    if option_key = opts[:aliases][key] do
      types[option_key]
@@ -428,10 +428,10 @@ defmodule Path do
  end

  @doc """
-  Joins a list of strings.
+  Joins a list of paths.

-  This function should be used to convert a list of strings to a path.
-  Note that any trailing slash is removed on join.
+  This function should be used to convert a list of paths to a path.
+  Note that any trailing slash is removed when joining.

  ## Examples

@@ -445,11 +445,11 @@ defmodule Path do
      "/foo/bar"

  """
-  @spec join([t]) :: binary
+  @spec join(nonempty_list(t)) :: binary
  def join([name1, name2 | rest]), do:
    join([join(name1, name2) | rest])
  def join([name]), do:
-    name
+    IO.chardata_to_string(name)

  @doc """
  Joins two paths.
@@ -629,7 +629,7 @@ defmodule Regex do
    [get_index(string, h) | get_indexes(string, t, arity - 1)]
  end

-  {:ok, pattern} = :re.compile(~S"[.^$*+?()\[\]{}\\\|\s#]", [:unicode])
+  {:ok, pattern} = :re.compile(~S"[.^$*+?()\[\]{}\\\|\s#-]", [:unicode])
  @escape_pattern pattern

  @doc ~S"""
@@ -789,16 +789,18 @@ defmodule Stream do
        do_after(after_fun, user_acc)
        :erlang.raise(kind, reason, stacktrace)
    else
-      {:suspended, [val], next} ->
-        do_transform_user(val, user_acc, user, fun, :cont, next, inner_acc, inner, after_fun)
-      {_, [val]} ->
-        do_transform_user(val, user_acc, user, fun, :halt, next, inner_acc, inner, after_fun)
-      {_, []} ->
-        do_transform(user_acc, user, fun, :halt, next, inner_acc, inner, after_fun)
+      {:suspended, vals, next} ->
+        do_transform_user(:lists.reverse(vals), user_acc, user, fun, :cont, next, inner_acc, inner, after_fun)
+      {_, vals} ->
+        do_transform_user(:lists.reverse(vals), user_acc, user, fun, :halt, next, inner_acc, inner, after_fun)
    end
  end

-  defp do_transform_user(val, user_acc, user, fun, next_op, next, inner_acc, inner, after_fun) do
+  defp do_transform_user([], user_acc, user, fun, next_op, next, inner_acc, inner, after_fun) do
+    do_transform(user_acc, user, fun, next_op, next, inner_acc, inner, after_fun)
+  end
+
+  defp do_transform_user([val | vals], user_acc, user, fun, next_op, next, inner_acc, inner, after_fun) do
    user.(val, user_acc)
  catch
    kind, reason ->
@@ -808,20 +810,20 @@ defmodule Stream do
      :erlang.raise(kind, reason, stacktrace)
  else
    {[], user_acc} ->
-      do_transform(user_acc, user, fun, next_op, next, inner_acc, inner, after_fun)
+      do_transform_user(vals, user_acc, user, fun, next_op, next, inner_acc, inner, after_fun)
    {list, user_acc} when is_list(list) ->
-      do_list_transform(user_acc, user, fun, next_op, next, inner_acc, inner,
+      do_list_transform(vals, user_acc, user, fun, next_op, next, inner_acc, inner,
                        &Enumerable.List.reduce(list, &1, fun), after_fun)
    {:halt, user_acc} ->
      next.({:halt, []})
      do_after(after_fun, user_acc)
      {:halted, elem(inner_acc, 1)}
    {other, user_acc} ->
-      do_enum_transform(user_acc, user, fun, next_op, next, inner_acc, inner,
+      do_enum_transform(vals, user_acc, user, fun, next_op, next, inner_acc, inner,
                        &Enumerable.reduce(other, &1, inner), after_fun)
  end

-  defp do_list_transform(user_acc, user, fun, next_op, next, inner_acc, inner, reduce, after_fun) do
+  defp do_list_transform(vals, user_acc, user, fun, next_op, next, inner_acc, inner, reduce, after_fun) do
    try do
      reduce.(inner_acc)
    catch
@@ -832,17 +834,17 @@ defmodule Stream do
        :erlang.raise(kind, reason, stacktrace)
    else
      {:done, acc} ->
-        do_transform(user_acc, user, fun, next_op, next, {:cont, acc}, inner, after_fun)
+        do_transform_user(vals, user_acc, user, fun, next_op, next, {:cont, acc}, inner, after_fun)
      {:halted, acc} ->
        next.({:halt, []})
        do_after(after_fun, user_acc)
        {:halted, acc}
      {:suspended, acc, c} ->
-        {:suspended, acc, &do_list_transform(user_acc, user, fun, next_op, next, &1, inner, c, after_fun)}
+        {:suspended, acc, &do_list_transform(vals, user_acc, user, fun, next_op, next, &1, inner, c, after_fun)}
    end
  end

-  defp do_enum_transform(user_acc, user, fun, next_op, next, {op, inner_acc}, inner, reduce, after_fun) do
+  defp do_enum_transform(vals, user_acc, user, fun, next_op, next, {op, inner_acc}, inner, reduce, after_fun) do
    try do
      reduce.({op, [:outer | inner_acc]})
    catch
@@ -855,15 +857,15 @@ defmodule Stream do
      # Only take into account outer halts when the op is not halt itself.
      # Otherwise, we were the ones wishing to halt, so we should just stop.
      {:halted, [:outer | acc]} when op != :halt ->
-        do_transform(user_acc, user, fun, next_op, next, {:cont, acc}, inner, after_fun)
+        do_transform_user(vals, user_acc, user, fun, next_op, next, {:cont, acc}, inner, after_fun)
      {:halted, [_ | acc]} ->
        next.({:halt, []})
        do_after(after_fun, user_acc)
        {:halted, acc}
      {:done, [_ | acc]} ->
-        do_transform(user_acc, user, fun, next_op, next, {:cont, acc}, inner, after_fun)
+        do_transform_user(vals, user_acc, user, fun, next_op, next, {:cont, acc}, inner, after_fun)
      {:suspended, [_ | acc], c} ->
-        {:suspended, acc, &do_enum_transform(user_acc, user, fun, next_op, next, &1, inner, c, after_fun)}
+        {:suspended, acc, &do_enum_transform(vals, user_acc, user, fun, next_op, next, &1, inner, c, after_fun)}
    end
  end

@@ -56,28 +56,27 @@ defmodule Task do

  ## Supervised tasks

-  It is also possible to spawn a task under a supervisor
-  with `start_link/1` and `start_link/3`:
-
-      Task.start_link(fn -> IO.puts "ok" end)
-
-  Such tasks can be mounted in your supervision tree as:
+  It is also possible to spawn a task under a supervisor:

      import Supervisor.Spec

      children = [
+        #
        worker(Task, [fn -> IO.puts "ok" end])
      ]

+  Internally the supervisor will invoke `Task.start_link/1`.
+
  Since these tasks are supervised and not directly linked to
  the caller, they cannot be awaited on. Note `start_link/1`,
  unlike `async/1`, returns `{:ok, pid}` (which is
  the result expected by supervision trees).

  By default, most supervision strategies will try to restart
-  a worker after it exits regardless of the reason. If you design the
-  task to terminate normally (as in the example with `IO.puts/2` above),
-  consider passing `restart: :transient` in the options to `Supervisor.Spec.worker/3`.
+  a worker after it exits regardless of the reason. If you design
+  the task to terminate normally (as in the example with `IO.puts/2`
+  above), consider passing `restart: :transient` in the options
+  to `Supervisor.Spec.worker/3`.

  ## Dynamically supervised tasks

@@ -286,7 +285,7 @@ defmodule Task do
  Returns a stream that runs the given `module`, `function` and `args`
  concurrently on each item in `enumerable`.

-  Each item will be appended to the given `args` and processed by its
+  Each item will be prepended to the given `args` and processed by its
  own task. The tasks will be linked to an intermediate process that is
  then linked to the current process. This means a failure in a task
  terminates the current process and a failure in the current process
@@ -297,7 +296,7 @@ defmodule Task do
  are emitted in the same order as the original `enumerable`.

  The level of concurrency can be controlled via the `:max_concurrency`
-  option and defaults to `System.schedulers_online/1`. The timeout
+  option and defaults to `System.schedulers_online/0`. The timeout
  can also be given as option and defaults to 5000 and it defaults to
  the maximum amount of time to wait without a task reply.

@@ -309,9 +308,10 @@ defmodule Task do
  ## Options

    * `:max_concurrency` - sets the maximum number of tasks to run
-      at the same time. Defaults to `System.schedulers_online/1`.
+      at the same time. Defaults to `System.schedulers_online/0`.
    * `:timeout` - the maximum amount of time to wait without
      receiving a task reply (across all running tasks).
+      Defaults to `5000`.

  ## Example

@@ -122,7 +122,7 @@ defmodule Task.Supervisor do
  are emitted in the same order as the original `enumerable`.

  The level of concurrency can be controlled via the `:max_concurrency`
-  option and defaults to `System.schedulers_online/1`. The timeout
+  option and defaults to `System.schedulers_online/0`. The timeout
  can also be given as option and defaults to 5000 and it defaults to
  the maximum amount of time to wait without a task reply.

@@ -133,9 +133,10 @@ defmodule Task.Supervisor do
  ## Options

    * `:max_concurrency` - sets the maximum number of tasks to run
-      at the same time. Defaults to `System.schedulers_online/1`.
+      at the same time. Defaults to `System.schedulers_online/0`.
    * `:timeout` - the maximum amount of time to wait without
      receiving a task reply (across all running tasks).
+      Defaults to `5000`.

  ## Examples

@@ -303,12 +303,12 @@ store_each(Check, Kind, File, Location, Module, Defaults, {function, Ann, Name,

  {FinalAnn, FinalLocation, FinalDefaults} = case ets:take(Defs, {def, Tuple}) of
    [{_, StoredKind, StoredAnn, StoredFile, StoredCheck,
-        StoredLocation, {StoredDefaults, LastHasBody, LastDefaults}}] ->
+        _StoredLocation, {StoredDefaults, LastHasBody, LastDefaults}}] ->
      check_valid_kind(Ann, File, Name, Arity, Kind, StoredKind),
      (Check and StoredCheck) andalso
        check_valid_clause(Ann, File, Name, Arity, Kind, Data, StoredAnn, StoredFile),
      check_valid_defaults(Ann, File, Name, Arity, Kind, Defaults, StoredDefaults, LastDefaults, LastHasBody),
-      {StoredAnn, StoredLocation, {max(Defaults, StoredDefaults), HasBody, Defaults}};
+      {StoredAnn, Location, {max(Defaults, StoredDefaults), HasBody, Defaults}};
    [] ->
      {Ann, Location, {Defaults, HasBody, Defaults}}
  end,
@@ -426,7 +426,7 @@ check_module_availability(Line, File, Module) ->
  end.

 warn_unused_attributes(File, Data, PersistedAttrs) ->
-  ReservedAttrs = [after_compile, before_compile, moduledoc, on_definition | PersistedAttrs],
+  ReservedAttrs = [after_compile, before_compile, moduledoc, on_definition, impl | PersistedAttrs],
  Keys = ets:select(Data, [{{'$1', '_', '_', '$2'}, [{is_atom, '$1'}, {is_integer, '$2'}], [['$1', '$2']]}]),
  [elixir_errors:form_warn([{line, Line}], File, ?MODULE, {unused_attribute, Key}) ||
   [Key, Line] <- Keys, not lists:member(Key, ReservedAttrs)].
@@ -264,16 +264,16 @@ defmodule ExceptionTest do
  end

  test "format_exit with call with exception" do
+    Process.flag(:trap_exit, true)
    # Fake reason to prevent error_logger printing to stdout
-    fsm_reason = {%ArgumentError{}, [{:not_a_real_module, :function, 0, []}]}
-    reason = try do
-      :gen_fsm.sync_send_event(spawn(fn() ->
-        Process.sleep(200)
-        exit(fsm_reason)
-      end), :hello)
-    catch
-      :exit, reason -> reason
-    end
+    exit_reason = {%ArgumentError{}, [{:not_a_real_module, :function, 0, []}]}
+    exit_fun = fn() -> receive do: (_ -> exit(exit_reason)) end
+    reason =
+      try do
+        :gen_fsm.sync_send_event(spawn_link(exit_fun), :hello)
+      catch
+        :exit, reason -> reason
+      end

    formatted = Exception.format_exit(reason)
    assert formatted =~ ~r"exited in: :gen_fsm\.sync_send_event\(#PID<\d+\.\d+\.\d+>, :hello\)"
@@ -283,14 +283,16 @@ defmodule ExceptionTest do
  end

  test "format_exit with nested calls" do
+    Process.flag(:trap_exit, true)
    # Fake reason to prevent error_logger printing to stdout
-    event_fun = fn() ->
-      Process.sleep(200)
-      exit(:normal)
-    end
-    server_pid = spawn(fn()-> :gen_event.call(spawn(event_fun), :handler, :hello) end)
+    exit_fun = fn() -> receive do: (_ -> exit(:normal)) end
+    outer_pid =
+      spawn_link(fn()->
+        Process.flag(:trap_exit, true)
+        :gen_event.call(spawn_link(exit_fun), :handler, :hello)
+      end)
    reason = try do
-      :gen_server.call(server_pid, :hi)
+      :gen_server.call(outer_pid, :hi)
    catch
      :exit, reason -> reason
    end
@@ -302,15 +304,17 @@ defmodule ExceptionTest do
  end

  test "format_exit with nested calls and exception" do
+    Process.flag(:trap_exit, true)
    # Fake reason to prevent error_logger printing to stdout
-    event_reason = {%ArgumentError{}, [{:not_a_real_module, :function, 0, []}]}
-    event_fun = fn() ->
-      Process.sleep(200)
-      exit(event_reason)
-    end
-    server_pid = spawn(fn()-> :gen_event.call(spawn(event_fun), :handler, :hello) end)
+    exit_reason = {%ArgumentError{}, [{:not_a_real_module, :function, 0, []}]}
+    exit_fun = fn() -> receive do: (_ -> exit(exit_reason)) end
+    outer_pid =
+      spawn_link(fn()->
+        Process.flag(:trap_exit, true)
+        :gen_event.call(spawn_link(exit_fun), :handler, :hello)
+      end)
    reason = try do
-      :gen_server.call(server_pid, :hi)
+      :gen_server.call(outer_pid, :hi)
    catch
      :exit, reason -> reason
    end
@@ -28,6 +28,31 @@ defmodule Kernel.WarningTest do
    purge Sample
  end

+  test "unused variable in redefined function in different file" do
+    output = capture_err(fn ->
+      Code.eval_string """
+      defmodule Sample do
+        defmacro __using__(_) do
+          quote location: :keep do
+            def function(arg)
+          end
+        end
+      end
+      """
+      Code.eval_string("""
+      defmodule RedefineSample do
+        use Sample
+        def function(var123), do: nil
+      end
+      """, [], file: "redefine_sample.ex")
+    end)
+    assert output =~ "redefine_sample.ex:3"
+    assert output =~ "variable \"var123\" is unused"
+  after
+    purge Sample
+    purge RedefineSample
+  end
+
  test "useless literal" do
    message = "code block contains unused literal \"oops\""

@@ -256,6 +256,11 @@ defmodule OptionParserTest do
           == {[foo: true, boo: "-"], ["-"], []}
  end

+  test "allow nonexistent atoms" do
+    assert OptionParser.parse(["--option-key-creates-atom"], allow_nonexistent_atoms: true) ==
+           {[{String.to_atom("option_key_creates_atom"), true}], [], []}
+  end
+
  test "correctly handles negative integers" do
    assert OptionParser.parse(["arg1", "-43"])
      == {[], ["arg1", "-43"], []}
@@ -211,6 +211,8 @@ defmodule PathTest do
    assert Path.join(['/foo/', "/bar/"]) == "/foo/bar"
    assert Path.join(["/", ""]) == "/"
    assert Path.join(["/", "", "bar"]) == "/bar"
+    assert Path.join(['foo', [?b, "a", ?r]]) == "foo/bar"
+    assert Path.join([[?f, 'o', "o"]]) == "foo"
  end

  test "join two" do
@@ -247,6 +247,7 @@ defmodule RegexTest do
    assert matches_escaped?("# lol")

    assert matches_escaped?("\\A.^$*+?()[{\\| \t\n\x20\\z #hello\u202F\u205F")
+    assert Regex.match? Regex.compile!("[" <> Regex.escape("!-#") <> "]"), "-"

    assert Regex.escape("{}") == "\\{\\}"
    assert Regex.escape("[]") == "\\[\\]"
@@ -481,6 +481,14 @@ defmodule StreamTest do
    assert Process.get(:stream_transform)
  end

+  test "transform/3 (via flat_map) handles multiple returns from suspension" do
+    assert [false]
+           |> Stream.take(1)
+           |> Stream.concat([true])
+           |> Stream.flat_map(&[&1])
+           |> Enum.to_list() == [false, true]
+  end
+
  test "iterate/2" do
    stream = Stream.iterate(0, &(&1+2))
    assert Enum.take(stream, 5) == [0, 2, 4, 6, 8]
@@ -449,7 +449,8 @@ defmodule StringTest do
    # CLRF
    assert String.graphemes("\r\n\f") == ["\r\n", "\f"]
    # Regional indicator
-    assert String.graphemes("\u{1F1E6}\u{1F1E7}\u{1F1E8}") == ["\u{1F1E6}\u{1F1E7}\u{1F1E8}"]
+    assert String.graphemes("\u{1F1E6}\u{1F1E7}") == ["\u{1F1E6}\u{1F1E7}"]
+    assert String.graphemes("\u{1F1E6}\u{1F1E7}\u{1F1E8}") == ["\u{1F1E6}\u{1F1E7}", "\u{1F1E8}"]
    # Hangul
    assert String.graphemes("\u1100\u115D\uB4A4") == ["ᄀᅝ뒤"]
    # Special Marking with Extended
@@ -52,20 +52,6 @@ defmodule String.Unicode do
    end
  end

-  # Handle Hangul L
-  for codepoint <- cluster["L"] do
-    def next_grapheme_size(<<unquote(codepoint), rest::binary>>) do
-      next_hangul_l_size(rest, unquote(byte_size(codepoint)))
-    end
-  end
-
-  # Handle Hangul T
-  for codepoint <- cluster["T"] do
-    def next_grapheme_size(<<unquote(codepoint), rest::binary>>) do
-      next_hangul_t_size(rest, unquote(byte_size(codepoint)))
-    end
-  end
-
  # Handle Regional
  for codepoint <- cluster["Regional_Indicator"] do
    def next_grapheme_size(<<unquote(codepoint), rest::binary>>) do
@@ -73,14 +59,48 @@ defmodule String.Unicode do
    end
  end

-  # Handle extended entries
+  # Handle Hangul L
+  for codepoint <- cluster["L"] do
+    def next_grapheme_size(<<unquote(codepoint), rest::binary>>) do
+      next_hangul_l_size(rest, unquote(byte_size(codepoint)))
+    end
+  end

+  # Handle Hangul V
+  for codepoint <- cluster["LV"] ++ cluster["V"]  do
+    def next_grapheme_size(<<unquote(codepoint), rest::binary>>) do
+      next_hangul_v_size(rest, unquote(byte_size(codepoint)))
+    end
+  end
+
+  # Handle Hangul T
+  for codepoint <- cluster["LVT"] ++ cluster["T"] do
+    def next_grapheme_size(<<unquote(codepoint), rest::binary>>) do
+      next_hangul_t_size(rest, unquote(byte_size(codepoint)))
+    end
+  end
+
+  # Handle E_Base
+  for codepoint <- cluster["E_Base"] ++ cluster["E_Base_GAZ"] do
+    def next_grapheme_size(<<unquote(codepoint), rest::binary>>) do
+      next_extend_size(rest, unquote(byte_size(codepoint)), :e_base)
+    end
+  end
+
+  # Handle ZWJ
+  for codepoint <- cluster["ZWJ"] do
+    def next_grapheme_size(<<unquote(codepoint), rest::binary>>) do
+      next_extend_size(rest, unquote(byte_size(codepoint)), :zwj)
+    end
+  end
+
+  # Handle extended entries
  def next_grapheme_size(<<cp::utf8, rest::binary>>) do
    case cp do
-      x when x <= 0x007F -> next_extend_size(rest, 1)
-      x when x <= 0x07FF -> next_extend_size(rest, 2)
-      x when x <= 0xFFFF -> next_extend_size(rest, 3)
-      _                  -> next_extend_size(rest, 4)
+      x when x <= 0x007F -> next_extend_size(rest, 1, :other)
+      x when x <= 0x07FF -> next_extend_size(rest, 2, :other)
+      x when x <= 0xFFFF -> next_extend_size(rest, 3, :other)
+      _                  -> next_extend_size(rest, 4, :other)
    end
  end

@@ -92,82 +112,139 @@ defmodule String.Unicode do
    nil
  end

-  # Handle Hangul L
-  for codepoint <- cluster["L"] do
-    defp next_hangul_l_size(<<unquote(codepoint), rest::binary>>, size) do
-      next_hangul_l_size(rest, size + unquote(byte_size(codepoint)))
-    end
-  end
-
-  for codepoint <- cluster["LV"] do
-    defp next_hangul_l_size(<<unquote(codepoint), rest::binary>>, size) do
-      next_hangul_v_size(rest, size + unquote(byte_size(codepoint)))
-    end
-  end
-
-  for codepoint <- cluster["LVT"] do
-    defp next_hangul_l_size(<<unquote(codepoint), rest::binary>>, size) do
-      next_hangul_t_size(rest, size + unquote(byte_size(codepoint)))
-    end
-  end
-
+  # Handle hanguls
  defp next_hangul_l_size(rest, size) do
-    next_hangul_v_size(rest, size)
-  end
-
-  # Handle Hangul V
-  for codepoint <- cluster["V"] do
-    defp next_hangul_v_size(<<unquote(codepoint), rest::binary>>, size) do
-      next_hangul_v_size(rest, size + unquote(byte_size(codepoint)))
+    case next_hangul(rest, size) do
+      {:l, rest, size} -> next_hangul_l_size(rest, size)
+      {:v, rest, size} -> next_hangul_v_size(rest, size)
+      {:lv, rest, size} -> next_hangul_v_size(rest, size)
+      {:lvt, rest, size} -> next_hangul_t_size(rest, size)
+      _ -> next_extend_size(rest, size, :other)
    end
  end

  defp next_hangul_v_size(rest, size) do
-    next_hangul_t_size(rest, size)
-  end
-
-  # Handle Hangul T
-  for codepoint <- cluster["T"] do
-    defp next_hangul_t_size(<<unquote(codepoint), rest::binary>>, size) do
-      next_hangul_t_size(rest, size + unquote(byte_size(codepoint)))
+    case next_hangul(rest, size) do
+      {:v, rest, size} -> next_hangul_v_size(rest, size)
+      {:t, rest, size} -> next_hangul_t_size(rest, size)
+      _ -> next_extend_size(rest, size, :other)
    end
  end

  defp next_hangul_t_size(rest, size) do
-    next_extend_size(rest, size)
+    case next_hangul(rest, size) do
+      {:t, rest, size} -> next_hangul_t_size(rest, size)
+      _ -> next_extend_size(rest, size, :other)
+    end
+  end
+
+  for codepoint <- cluster["L"] do
+    defp next_hangul(<<unquote(codepoint), rest::binary>>, size) do
+      {:l, rest, size + unquote(byte_size(codepoint))}
+    end
+  end
+
+  for codepoint <- cluster["V"] do
+    defp next_hangul(<<unquote(codepoint), rest::binary>>, size) do
+      {:v, rest, size + unquote(byte_size(codepoint))}
+    end
+  end
+
+  for codepoint <- cluster["T"] do
+    defp next_hangul(<<unquote(codepoint), rest::binary>>, size) do
+      {:t, rest, size + unquote(byte_size(codepoint))}
+    end
+  end
+
+  for codepoint <- cluster["LV"] do
+    defp next_hangul(<<unquote(codepoint), rest::binary>>, size) do
+      {:lv, rest, size + unquote(byte_size(codepoint))}
+    end
+  end
+
+  for codepoint <- cluster["LVT"] do
+    defp next_hangul(<<unquote(codepoint), rest::binary>>, size) do
+      {:lvt, rest, size + unquote(byte_size(codepoint))}
+    end
+  end
+
+  defp next_hangul(_, _) do
+    false
  end

  # Handle regional
  for codepoint <- cluster["Regional_Indicator"] do
    defp next_regional_size(<<unquote(codepoint), rest::binary>>, size) do
-      next_regional_size(rest, size + unquote(byte_size(codepoint)))
+      next_extend_size(rest, size + unquote(byte_size(codepoint)), :other)
    end
  end
-
  defp next_regional_size(rest, size) do
-    next_extend_size(rest, size)
+    next_extend_size(rest, size, :other)
  end

-  # Handle Extend+SpacingMark
-  for codepoint <- cluster["Extend"] ++ cluster["SpacingMark"]  do
-    defp next_extend_size(<<unquote(codepoint), rest::binary>>, size) do
-      next_extend_size(rest, size + unquote(byte_size(codepoint)))
+  # Handle Extend+SpacingMark+ZWJ
+  for codepoint <- cluster["Extend"] do
+    defp next_extend_size(<<unquote(codepoint), rest::binary>>, size, marker) do
+      next_extend_size(rest, size + unquote(byte_size(codepoint)), keep_ebase(marker))
    end
  end

-  defp next_extend_size(rest, size) do
+  for codepoint <- cluster["SpacingMark"] do
+    defp next_extend_size(<<unquote(codepoint), rest::binary>>, size, _marker) do
+      next_extend_size(rest, size + unquote(byte_size(codepoint)), :other)
+    end
+  end
+
+  for codepoint <- cluster["ZWJ"] do
+    defp next_extend_size(<<unquote(codepoint), rest::binary>>, size, _marker) do
+      next_extend_size(rest, size + unquote(byte_size(codepoint)), :zwj)
+    end
+  end
+
+  for codepoint <- cluster["E_Modifier"] do
+    defp next_extend_size(<<unquote(codepoint), rest::binary>>, size, :e_base) do
+      next_extend_size(rest, size + unquote(byte_size(codepoint)), :other)
+    end
+  end
+
+  for codepoint <- cluster["Glue_After_Zwj"] do
+    defp next_extend_size(<<unquote(codepoint), rest::binary>>, size, :zwj) do
+      next_extend_size(rest, size + unquote(byte_size(codepoint)), :other)
+    end
+  end
+
+  for codepoint <- cluster["E_Base_GAZ"] do
+    defp next_extend_size(<<unquote(codepoint), rest::binary>>, size, :zwj) do
+      next_extend_size(rest, size + unquote(byte_size(codepoint)), :e_base)
+    end
+  end
+
+  defp next_extend_size(rest, size, _) do
    {size, rest}
  end

+  defp keep_ebase(:e_base), do: :e_base
+  defp keep_ebase(_), do: :other
+
  # Handle Prepend
  for codepoint <- cluster["Prepend"] do
    defp next_prepend_size(<<unquote(codepoint), rest::binary>>, size) do
      next_prepend_size(rest, size + unquote(byte_size(codepoint)))
    end
  end
-  
+
+  # However, if we see a control character, we have to break it
+  for codepoint <- cluster["CR"] ++ cluster["LF"] ++ cluster["Control"] do
+    defp next_prepend_size(<<unquote(codepoint), _::binary>> = rest, size) do
+      {size, rest}
+    end
+  end
+
  defp next_prepend_size(rest, size) do
-    {size, rest}
+    case next_grapheme_size(rest) do
+      {more, rest} ->  {more + size, rest}
+      nil -> {size, rest}
+    end
  end

  # Graphemes
@@ -115,9 +115,10 @@ defmodule IEx.Autocomplete do

  # Elixir.fun
  defp expand_call({:__aliases__, _, list}, hint, server) do
-    expand_alias(list, server)
-    |> normalize_module
-    |> expand_require(hint)
+    case expand_alias(list, server) do
+      {:ok, alias} -> expand_require(alias, hint)
+      :error -> no()
+    end
  end

  # variable.fun_or_key
@@ -171,9 +172,10 @@ defmodule IEx.Autocomplete do
  end

  defp expand_elixir_modules(list, hint, server) do
-    expand_alias(list, server)
-    |> normalize_module
-    |> expand_elixir_modules_from_aliases(hint, [])
+    case expand_alias(list, server) do
+      {:ok, alias} -> expand_elixir_modules_from_aliases(alias, hint, [])
+      :error -> no()
+    end
  end

  defp expand_elixir_modules_from_aliases(mod, hint, aliases) do
@@ -183,19 +185,16 @@ defmodule IEx.Autocomplete do
    |> format_expansion(hint)
  end

-  defp expand_alias([name | rest] = list, server) do
-    module = Module.concat(Elixir, name)
-    Enum.find_value aliases_from_env(server), list, fn {alias, mod} ->
-      if alias === module do
-        case Atom.to_string(mod) do
-          "Elixir." <> mod ->
-            Module.concat [mod | rest]
-          _ ->
-            mod
-        end
-      end
+  defp expand_alias([name | rest], server) when is_atom(name) do
+    case Keyword.fetch(aliases_from_env(server), Module.concat(Elixir, name)) do
+      {:ok, name} when rest == [] -> {:ok, name}
+      {:ok, name} -> {:ok, Module.concat([name | rest])}
+      :error -> {:ok, Module.concat([name | rest])}
    end
  end
+  defp expand_alias([_ | _], _) do
+    :error
+  end

  defp match_aliases(hint, server) do
    for {alias, _mod} <- aliases_from_env(server),
@@ -220,14 +219,6 @@ defmodule IEx.Autocomplete do

  ## Helpers

-  defp normalize_module(mod) do
-    if is_list(mod) do
-      Module.concat(mod)
-    else
-      mod
-    end
-  end
-
  defp match_modules(hint, root) do
    get_modules(root)
    |> :lists.usort()
@@ -90,6 +90,7 @@ defmodule IEx.AutocompleteTest do
    assert expand('.')   == {:no, '', []}
    assert expand('Xyz') == {:no, '', []}
    assert expand('x.Foo') == {:no, '', []}
+    assert expand('x.Foo.get_by') == {:no, '', []}
  end

  test "Elixir root submodule completion" do
@@ -264,11 +264,13 @@ defmodule Mix.Dep.Loader do
    {dep, deps}
  end

-  defp rebar_dep(%Mix.Dep{app: app, opts: opts} = dep, children, manager) do
-    config = File.cd!(opts[:dest], fn ->
-      Mix.Rebar.load_config(".")
-    end)
-    extra = Mix.Rebar.merge_config(dep.extra, config)
+  defp rebar_dep(%Mix.Dep{app: app, opts: opts, extra: overrides} = dep, children, manager) do
+    config =
+      File.cd!(opts[:dest], fn -> Mix.Rebar.load_config(".") end)
+
+    config =
+      Mix.Rebar.apply_overrides(app, config, overrides)
+
    deps =
      if children do
        from = Path.join(opts[:dest], "rebar.config")
@@ -277,9 +279,10 @@ defmodule Mix.Dep.Loader do
        # files in the dependency
        Enum.map(children, &to_dep(&1, from, manager))
      else
-        rebar_children(app, config, extra, manager)
+        rebar_children(config, manager, opts[:dest])
      end
-    {%{dep | extra: extra}, deps}
+
+    {%{dep | extra: config}, deps}
  end

  defp make_dep(dep) do
@@ -301,17 +304,18 @@ defmodule Mix.Dep.Loader do
    |> elem(0)
  end

-  defp rebar_children(app, root_config, extra, manager) do
-    from = Path.absname("rebar.config")
+  defp rebar_children(root_config, manager, dest) do
+    from = Path.absname(Path.join(dest, "rebar.config"))
    Mix.Rebar.recur(root_config, fn config ->
-      app
-      |> Mix.Rebar.deps(config, overrides(manager, extra))
-      |> Enum.map(fn dep -> %{to_dep(dep, from, manager) | extra: extra} end)
+      overrides = overrides(manager, config)
+      config
+      |> Mix.Rebar.deps()
+      |> Enum.map(fn dep -> %{to_dep(dep, from, manager) | extra: overrides} end)
    end) |> Enum.concat
  end

-  defp overrides(:rebar3, extra), do: extra[:overrides] || []
-  defp overrides(_, _extra), do: []
+  defp overrides(:rebar3, config), do: config[:overrides] || []
+  defp overrides(_, _config), do: []

  defp validate_app(%Mix.Dep{opts: opts, requirement: req, app: app} = dep) do
    opts_app = opts[:app]
@@ -233,9 +233,10 @@ defmodule Mix.Project do
  @spec apps_paths() :: %{atom => Path.t} | nil
  def apps_paths(config \\ config()) do
    if apps_path = config[:apps_path] do
-      Mix.ProjectStack.read_cache(:apps_path) ||
-        Mix.ProjectStack.write_cache(:apps_path,
-          config[:apps] |> umbrella_apps(apps_path) |> to_apps_path(apps_path))
+      key = {:apps_paths, Mix.Project.get!}
+      Mix.ProjectStack.read_cache(key) ||
+        Mix.ProjectStack.write_cache(key,
+          config[:apps] |> umbrella_apps(apps_path) |> to_apps_paths(apps_path))
    end
  end

@@ -249,7 +250,7 @@ defmodule Mix.Project do
    apps
  end

-  defp to_apps_path(apps, apps_path) do
+  defp to_apps_paths(apps, apps_path) do
    for app <- apps,
        path = path_with_mix_exs_otherwise_warn(app, apps_path),
        do: {app, path},
@@ -18,7 +18,7 @@ defmodule Mix.ProjectStack do
    get_and_update fn %{stack: stack} = state ->
      # Consider the first children to always have io_done
      # because we don't need to print anything unless another
-      # project talks ahold of the shell.
+      # project takes ahold of the shell.
      io_done? = stack == []

      config  = Keyword.merge(config, state.post_config)
@@ -61,108 +61,6 @@ defmodule Mix.Rebar do
    end
  end

-  @doc """
-  Merges a rebar3 parent config with a child config.
-  """
-  # From https://github.com/rebar/rebar3/blob/b1da2ec0674df89599564252734bd4d794436425/src/rebar_opts.erl#L103
-  def merge_config(old, new) do
-    Keyword.merge(old, new, fn
-      :deps, old, _new               -> old
-      {:deps, _}, _old, new          -> new
-      :plugins, _old, new            -> new
-      {:plugins, _}, _old, new       -> new
-      :profiles, old, new            -> merge_config(old, new)
-      :mib_first_files, value, value -> value
-      :mib_first_files, old, new     -> old ++ new
-      :relx, old, new                -> tuple_merge(new, old)
-      _key, old, new when is_list(new) ->
-        case :io_lib.printable_list(new) do
-          true when new == [] ->
-            if :io_lib.printable_list(old), do: new, else: old
-          true ->
-            new
-          false ->
-            tuple_merge(old, new)
-        end
-      _key, _old, new -> new
-    end)
-  end
-
-  # From https://github.com/rebar/rebar3/blob/b1da2ec0674df89599564252734bd4d794436425/src/rebar_utils.erl#L282
-  defp tuple_merge(old, new),
-    do: do_tuple_merge(tuple_sort(old), tuple_sort(new))
-
-  defp do_tuple_merge(old, []),
-    do: old
-  defp do_tuple_merge(olds, [new | news]),
-    do: do_tuple_umerge_dedup(umerge(:new, olds, [], news, new), [])
-
-  defp umerge(_, [], [], acc, current),
-    do: [current | acc]
-  defp umerge(:new, [], news, acc, current),
-    do: Enum.reverse(news, [current | acc])
-  defp umerge(:old, olds, [], acc, current),
-    do: Enum.reverse(olds, [current | acc])
-  defp umerge(:new, [old | olds], news, acc, current) do
-    {dir, merged, new_current} = compare({:new, current}, {:old, old})
-    umerge(dir, olds, news, [merged | acc], new_current)
-  end
-  defp umerge(:old, olds, [new | news], acc, current) do
-    {dir, merged, new_current} = compare({:new, new}, {:old, current})
-    umerge(dir, olds, news, [merged | acc], new_current)
-  end
-
-  defp compare({priority, a}, {secondary, b}) when is_tuple(a) and is_tuple(b) do
-    ka = elem(a, 0)
-    kb = elem(b, 0)
-    cond do
-      ka == kb -> {secondary, a, b}
-      ka  < kb -> {secondary, a, b}
-      ka  > kb -> {priority, b, a}
-    end
-  end
-  defp compare({priority, a}, {secondary, b}) when not is_tuple(a) and not is_tuple(b) do
-    cond do
-      a == b -> {secondary, a, b}
-      a  < b -> {secondary, a, b}
-      a  > b -> {priority, b, a}
-    end
-  end
-  defp compare({priority, a}, {secondary, b}) when is_tuple(a) and not is_tuple(b) do
-    ka = elem(a, 0)
-    cond do
-      ka == b -> {secondary, a, b}
-      ka  < b -> {secondary, a, b}
-      ka  > b -> {priority, b, a}
-    end
-  end
-  defp compare({priority, a}, {secondary, b}) when not is_tuple(a) and is_tuple(b) do
-    kb = elem(b, 0)
-    cond do
-      a == kb -> {secondary, a, b}
-      a  < kb -> {secondary, a, b}
-      a  > kb -> {priority, b, a}
-    end
-  end
-
-  defp do_tuple_umerge_dedup([], acc), do: acc
-  defp do_tuple_umerge_dedup([h | t], acc) do
-    if h in t do
-      do_tuple_umerge_dedup(t, acc)
-    else
-      do_tuple_umerge_dedup(t, [h | acc])
-    end
-  end
-
-  defp tuple_sort(list) do
-    Enum.sort(list, fn
-      a, b when is_tuple(a) and is_tuple(b) -> elem(a, 0) <= elem(b, 0)
-      a, b when is_tuple(a) -> elem(a, 0) <= b
-      a, b when is_tuple(b) -> a <= elem(b, 0)
-      a, b -> a <= b
-    end)
-  end
-
  @doc """
  Serializes a Rebar config to a term file.
  """
@@ -178,7 +76,7 @@ defmodule Mix.Rebar do
  def dependency_config(config) do
    Enum.map(config, fn
      {:erl_opts, opts} ->
-        {:erl_opts, Enum.reject(opts, &(&1 == :warnings_as_errors))}
+        {:erl_opts, List.delete(opts, :warnings_as_errors)}
      other ->
        other
    end)
@@ -187,10 +85,9 @@ defmodule Mix.Rebar do
  @doc """
  Parses the dependencies in given `rebar.config` to Mix's dependency format.
  """
-  def deps(app, config, overrides) do
+  def deps(config) do
    # We don't have to handle rebar3 profiles because dependencies
    # are always in the default profile which cannot be customized
-    config = apply_overrides(app, config, overrides)
    if deps = config[:deps] do
      Enum.map(deps, &parse_dep/1)
    else
@@ -201,23 +98,23 @@ defmodule Mix.Rebar do
  @doc """
  Runs `fun` for the given config and for each `sub_dirs` in the
  given Rebar config.
-  """
-  def recur(config, fun) when is_binary(config) do
-    recur(load_config(config), fun)
-  end

+  `sub_dirs` is only supported in Rebar 2. In Rebar 3, the equivalent
+  to umbrella apps cannot be used as dependencies, so we don't need
+  to worry about such cases in Mix.
+  """
  def recur(config, fun) do
    subs =
      (config[:sub_dirs] || [])
-      |> Enum.map(&Path.wildcard(&1))
-      |> Enum.concat
+      |> Enum.flat_map(&Path.wildcard(&1))
      |> Enum.filter(&File.dir?(&1))
-      |> Enum.map(&recur(&1, fun))
-      |> Enum.concat
+      |> Enum.flat_map(&recur(load_config(&1), fun))

    [fun.(config) | subs]
  end

+  # Translate a rebar dependency declaration to a mix declaration
+  # From http://www.rebar3.org/docs/dependencies#section-declaring-dependencies
  defp parse_dep(app) when is_atom(app) do
    {app, ">= 0.0.0"}
  end
@@ -226,10 +123,6 @@ defmodule Mix.Rebar do
    {app, List.to_string(req)}
  end

-  defp parse_dep({app, req, {:pkg, package}}) when is_list(req) do
-    {app, List.to_string(req), hex: package}
-  end
-
  defp parse_dep({app, source}) when is_tuple(source) do
    parse_dep({app, nil, source, []})
  end
@@ -239,25 +132,33 @@ defmodule Mix.Rebar do
  end

  defp parse_dep({app, req, source, opts}) do
-    [scm, url | source] = Tuple.to_list(source)
-
-    ref =
-      case source do
-        ["" | _]                -> [branch: "HEAD"]
-        [{:branch, branch} | _] -> [branch: to_string(branch)]
-        [{:tag, tag} | _]       -> [tag: to_string(tag)]
-        [{:ref, ref} | _]       -> [ref: to_string(ref)]
-        [ref | _]               -> [ref: to_string(ref)]
-        _                       -> []
-      end
+    source = parse_source(source)

    compile =
      if :proplists.get_value(:raw, opts, false),
        do: [compile: false],
        else: []

-    mix_opts = [{scm, to_string(url)}] ++ ref ++ compile
-    {app, compile_req(req), mix_opts}
+    {app, compile_req(req), source ++ compile}
+  end
+
+  defp parse_source({:pkg, pkg}) do
+    [hex: pkg]
+  end
+  defp parse_source(source) do
+    [scm, url | source] = Tuple.to_list(source)
+
+    ref =
+      case source do
+        ["" | _] -> [branch: "HEAD"]
+        [{:branch, branch} | _] -> [branch: to_string(branch)]
+        [{:tag, tag} | _] -> [tag: to_string(tag)]
+        [{:ref, ref} | _] -> [ref: to_string(ref)]
+        [ref | _] -> [ref: to_string(ref)]
+        _ -> []
+      end
+
+    [{scm, to_string(url)}] ++ ref
  end

  defp compile_req(nil) do
@@ -265,11 +166,18 @@ defmodule Mix.Rebar do
  end

  defp compile_req(req) do
-    case Regex.compile(List.to_string(req)) do
-      {:ok, re} ->
-        re
-      {:error, reason} ->
-        Mix.raise "Unable to compile version regex: #{inspect req}, #{reason}"
+    req = List.to_string(req)
+
+    case Version.parse_requirement(req) do
+      {:ok, _} ->
+        req
+      :error ->
+        case Regex.compile(req) do
+          {:ok, re} ->
+            re
+          {:error, reason} ->
+            Mix.raise "Unable to compile version regex: #{inspect req}, #{reason}"
+        end
    end
  end

@@ -312,9 +220,11 @@ defmodule Mix.Rebar do
    end
  end

-  defp apply_overrides(app, config, overrides) do
+  @doc """
+  Applies the given overrides for app config.
+  """
+  def apply_overrides(app, config, overrides) do
    # Inefficient. We want the order we get here though.
-
    config =
      Enum.reduce(overrides, config, fn
        {:override, overrides}, config ->
@@ -327,7 +237,7 @@ defmodule Mix.Rebar do

    config =
      Enum.reduce(overrides, config, fn
-        {:override, oapp, overrides}, config when oapp == app ->
+        {:override, ^app, overrides}, config ->
          Enum.reduce(overrides, config, fn {key, value}, config ->
            Keyword.put(config, key, value)
          end)
@@ -335,14 +245,17 @@ defmodule Mix.Rebar do
          config
      end)

-    Enum.reduce(overrides, config, fn
-      {:add, oapp, overrides}, config when oapp == app ->
-        Enum.reduce(overrides, config, fn {key, value}, config ->
-          old_value = Keyword.get(config, key, [])
-          Keyword.put(config, key, value ++ old_value)
+    config =
+      Enum.reduce(overrides, config, fn
+        {:add, ^app, overrides}, config ->
+          Enum.reduce(overrides, config, fn {key, value}, config ->
+            old_value = Keyword.get(config, key, [])
+            Keyword.put(config, key, value ++ old_value)
+          end)
+        _, config ->
+          config
      end)
-      _, config ->
-        config
-    end)
+
+    Keyword.update(config, :overrides, overrides, &(overrides ++ &1))
  end
 end
@@ -106,7 +106,7 @@ defmodule Mix.Tasks.Archive.Install do

  defp archives(name) do
    Mix.Local.path_for(:archive)
-    |> Path.join(name)
+    |> Path.join(name <> "{,*.ez}")
    |> Path.wildcard
  end

@@ -191,19 +191,29 @@ defmodule Mix.Tasks.Deps.Compile do
      Mix.raise "\"#{manager}\" installation failed"
  end

-  defp do_make(%{opts: opts} = dep, config) do
-    os_type = :os.type
+  defp do_make(dep, config) do
+    command = make_command(dep)
+    do_command(dep, config, command, true, [{"IS_DEP", "1"}])
+  end
+
+  defp make_command(dep) do
+    makefile_win? = makefile_win?(dep)

    command =
-      cond do
-        match?({:win32, _}, os_type) and File.regular?(Path.join(opts[:dest], "Makefile.win")) ->
+      case :os.type do
+        {:win32, _} when makefile_win? ->
          "nmake /F Makefile.win"
-        match?({:unix, type} when type in [:freebsd, :openbsd], os_type) ->
+        {:unix, type} when type in [:freebsd, :openbsd] ->
          "gmake"
-        true ->
+        _ ->
          "make"
      end
-    do_command(dep, config, command, true, [{"IS_DEP", "1"}])
+
+    if erlang_mk?(dep) do
+      "#{command} clean && #{command}"
+    else
+      command
+    end
  end

  defp do_compile(%Mix.Dep{opts: opts} = dep, config) do
@@ -248,4 +258,12 @@ defmodule Mix.Tasks.Deps.Compile do
      req
    end
  end
+
+  defp erlang_mk?(%Mix.Dep{opts: opts}) do
+    File.regular?(Path.join(opts[:dest], "erlang.mk"))
+  end
+
+  defp makefile_win?(%Mix.Dep{opts: opts}) do
+    File.regular?(Path.join(opts[:dest], "Makefile.win"))
+  end
 end
@@ -33,212 +33,231 @@ defmodule Mix.RebarTest do
    end
  end

-  test "load Rebar config" do
-    path = MixTest.Case.fixture_path("rebar_dep")
-    config = Mix.Rebar.load_config(path)
-    assert config[:sub_dirs] == ['apps/*']
-    assert config[:SCRIPT] == 'rebar.config.script'
-  end
-
-  test "execute rebar.config.script on dependency directory" do
-    path = MixTest.Case.fixture_path("rebar_dep_script")
-    config = Mix.Rebar.load_config(path)
-    assert config[:dir] == {:ok, String.to_charlist(path)}
-  end
-
-  test "parse Rebar dependencies" do
-    config = [deps: [{:git_rebar, '~> 1.0'}]]
-    assert [{:git_rebar, "~> 1.0"}] ==
-           Mix.Rebar.deps(:foo, config, [])
-
-    config = [deps: [{:git_rebar, '~> 1.0', {:pkg, :rebar_fork}}]]
-    assert [{:git_rebar, "~> 1.0", hex: :rebar_fork}] ==
-           Mix.Rebar.deps(:foo, config, [])
-
-    config = [deps: [{:git_rebar, '0.1..*', {:git, '../../test/fixtures/git_rebar', :master}}]]
-    assert [{:git_rebar, ~r"0.1..*", [git: "../../test/fixtures/git_rebar", ref: "master"]}] ==
-           Mix.Rebar.deps(:foo, config, [])
-
-    config = [deps: [{:git_rebar, {:git, '../../test/fixtures/git_rebar', :master}}]]
-    assert [{:git_rebar, ">= 0.0.0", [git: "../../test/fixtures/git_rebar", ref: "master"]}] ==
-           Mix.Rebar.deps(:foo, config, [])
-
-    config = [deps: [{:git_rebar, '0.1..*', {:git, '../../test/fixtures/git_rebar'}, [:raw]}]]
-    assert [{:git_rebar, ~r"0.1..*", [git: "../../test/fixtures/git_rebar", compile: false]}] ==
-           Mix.Rebar.deps(:foo, config, [])
-
-    config = [deps: [{:git_rebar, '', {:git, '../../test/fixtures/git_rebar', {:ref, '64691eb'}}}]]
-    assert [{:git_rebar, ~r"", [git: "../../test/fixtures/git_rebar", ref: "64691eb"]}] ==
-           Mix.Rebar.deps(:foo, config, [])
-
-    config = [deps: {:git_rebar, '~> 2.0'}]
-    overrides = [{:override, [deps: [{:git_rebar, '~> 1.0'}]]}]
-    assert [{:git_rebar, "~> 1.0"}] ==
-           Mix.Rebar.deps(:foo, config, overrides)
-
-    config = [deps: [{:git_rebar, '~> 2.0'}]]
-    overrides = [{:override, [deps: [{:git_rebar, '~> 1.0'}]]}]
-    assert [{:git_rebar, "~> 1.0"}] ==
-           Mix.Rebar.deps(:foo, config, overrides)
-
-    config = [deps: [{:git_rebar, '~> 2.0'}]]
-    overrides = [{:override, :bar, [deps: [{:git_rebar, '~> 1.0'}]]}]
-    assert [{:git_rebar, "~> 2.0"}] ==
-           Mix.Rebar.deps(:foo, config, overrides)
-
-    config = [deps: [{:git_rebar, '~> 2.0'}]]
-    overrides = [{:override, :foo, [deps: [{:git_rebar, '~> 1.0'}]]}]
-    assert [{:git_rebar, "~> 1.0"}] ==
-           Mix.Rebar.deps(:foo, config, overrides)
-
-    config = [deps: [{:git_rebar, '~> 1.0'}]]
-    overrides = [{:add, :foo, [deps: [{:git_rebar2, '~> 2.0'}]]}]
-    assert [{:git_rebar2, "~> 2.0"}, {:git_rebar, "~> 1.0"}] ==
-           Mix.Rebar.deps(:foo, config, overrides)
-
-  end
-
-  test "convert rebar config to dependency config" do
-    config = Mix.Rebar.load_config(fixture_path("rebar_dep"))
-    dep_config = Mix.Rebar.dependency_config(config)
-
-    assert config[:erl_opts] == [:warnings_as_errors]
-    assert dep_config[:erl_opts] == []
-  end
-
-  test "parse Rebar dependencies from rebar.config" do
-    Mix.Project.push(RebarAsDep)
-
-    deps = Mix.Dep.loaded([])
-    assert Enum.find(deps, &(&1.app == :rebar_dep))
-
-    assert Enum.find(deps, fn %Mix.Dep{app: app, opts: opts} ->
-      if app == :git_rebar do
-        assert Enum.find(opts, &match?({:git, _}, &1))
-        assert Enum.find(opts, &match?({:ref, "master"}, &1))
-        true
-      end
-    end)
-  end
-
-  test "inherit Rebar manager" do
-    Mix.Project.push(Rebar3AsDep)
-
-    deps = Mix.Dep.loaded([])
-    assert Enum.all?(deps, &(&1.manager == :rebar3))
-  end
-
-  test "Rebar overrides" do
-    Mix.Project.push(RebarOverrideAsDep)
-
-    in_tmp "Rebar overrides", fn ->
-      Mix.Tasks.Deps.Get.run []
-      assert Mix.Dep.loaded([]) |> Enum.map(& &1.app) ==
-             [:git_repo, :git_rebar, :rebar_override]
+  describe "load_config/1" do
+    test "loads rebar.config" do
+      path = MixTest.Case.fixture_path("rebar_dep")
+      config = Mix.Rebar.load_config(path)
+      assert config[:sub_dirs] == ['apps/*']
+      assert config[:SCRIPT] == 'rebar.config.script'
+    end
+
+    test "loads rebar.config.script on dependency directory" do
+      path = MixTest.Case.fixture_path("rebar_dep_script")
+      config = Mix.Rebar.load_config(path)
+      assert config[:dir] == {:ok, String.to_charlist(path)}
    end
-  after
-    purge [GitRepo.Mixfile]
  end

-  test "recurs over sub dirs" do
-    path = MixTest.Case.fixture_path("rebar_dep")
+  describe "deps/1" do
+    test "parses Rebar dependencies" do
+      config = [deps: [{:git_rebar, '~> 1.0'}]]
+      assert [{:git_rebar, "~> 1.0"}] ==
+             Mix.Rebar.deps(config)

-    File.cd! path, fn ->
-     config = Mix.Rebar.load_config(path)
+      config = [deps: [{:git_rebar, '~> 1.0', {:pkg, :rebar_fork}}]]
+      assert [{:git_rebar, "~> 1.0", hex: :rebar_fork}] ==
+             Mix.Rebar.deps(config)

-      Mix.Rebar.recur(config, fn config ->
-        if config[:sub_dirs] == ['from_apps_another'] do
-          Process.put(:inside_apps_another, true)
+      config = [deps: [{:git_rebar, {:pkg, :rebar_fork}}]]
+      assert [{:git_rebar, ">= 0.0.0", hex: :rebar_fork}] ==
+             Mix.Rebar.deps(config)
+
+      config = [deps: [{:git_rebar, '0.1..*', {:git, '../../test/fixtures/git_rebar', :master}}]]
+      assert [{:git_rebar, ~r"0.1..*", [git: "../../test/fixtures/git_rebar", ref: "master"]}] ==
+             Mix.Rebar.deps(config)
+
+      config = [deps: [{:git_rebar, {:git, '../../test/fixtures/git_rebar', :master}}]]
+      assert [{:git_rebar, ">= 0.0.0", [git: "../../test/fixtures/git_rebar", ref: "master"]}] ==
+             Mix.Rebar.deps(config)
+
+      config = [deps: [{:git_rebar, '0.1..*', {:git, '../../test/fixtures/git_rebar'}, [:raw]}]]
+      assert [{:git_rebar, ~r"0.1..*", [git: "../../test/fixtures/git_rebar", compile: false]}] ==
+             Mix.Rebar.deps(config)
+
+      config = [deps: [{:git_rebar, '', {:git, '../../test/fixtures/git_rebar', {:ref, '64691eb'}}}]]
+      assert [{:git_rebar, ~r"", [git: "../../test/fixtures/git_rebar", ref: "64691eb"]}] ==
+             Mix.Rebar.deps(config)
+    end
+  end
+
+  describe "apply_overrides/3" do
+    test "applies overrides" do
+      config = [deps: {:git_rebar, '~> 2.0'}]
+      overrides = [{:override, [deps: [{:git_rebar, '~> 1.0'}]]}]
+      assert Mix.Rebar.apply_overrides(:foo, config, overrides) ==
+             [deps: [{:git_rebar, '~> 1.0'}], overrides: overrides]
+
+      config = [deps: [{:git_rebar, '~> 2.0'}]]
+      overrides = [{:override, :bar, [deps: [{:git_rebar, '~> 1.0'}]]}]
+      assert Mix.Rebar.apply_overrides(:foo, config, overrides) ==
+             [deps: [{:git_rebar, '~> 2.0'}], overrides: overrides]
+
+      config = [deps: [{:git_rebar, '~> 2.0'}]]
+      overrides = [{:override, :foo, [deps: [{:git_rebar, '~> 1.0'}]]}]
+      assert Mix.Rebar.apply_overrides(:foo, config, overrides) ==
+             [deps: [{:git_rebar, '~> 1.0'}], overrides: overrides]
+
+      config = [deps: [{:git_rebar, '~> 1.0'}]]
+      overrides = [{:add, :foo, [deps: [{:git_rebar2, '~> 2.0'}]]}]
+      assert Mix.Rebar.apply_overrides(:foo, config, overrides) ==
+             [deps: [{:git_rebar2, '~> 2.0'}, {:git_rebar, '~> 1.0'}], overrides: overrides]
+    end
+
+    test "concatenates overrides" do
+      config = [deps: {:git_rebar, '~> 2.0'}, overrides: [{:add, :bar, []}]]
+      overrides = [{:override, [deps: [{:git_rebar, '~> 1.0'}]]}]
+      assert Mix.Rebar.apply_overrides(:foo, config, overrides) ==
+             [deps: [{:git_rebar, '~> 1.0'}], overrides: overrides ++ [{:add, :bar, []}]]
+    end
+  end
+
+  describe "dependency_config/1" do
+    test "converts Rebar config to dependency config" do
+      config = Mix.Rebar.load_config(fixture_path("rebar_dep"))
+      dep_config = Mix.Rebar.dependency_config(config)
+
+      assert config[:erl_opts] == [:warnings_as_errors]
+      assert dep_config[:erl_opts] == []
+    end
+  end
+
+  describe "recur/1" do
+    test "recurs over sub dirs" do
+      path = MixTest.Case.fixture_path("rebar_dep")
+
+      File.cd! path, fn ->
+       config = Mix.Rebar.load_config(path)
+
+        Mix.Rebar.recur(config, fn config ->
+          if config[:sub_dirs] == ['from_apps_another'] do
+            Process.put(:inside_apps_another, true)
+          end
+        end)
+      end
+
+      unless Process.get(:inside_apps_another) do
+        flunk "Expected inside_apps_another to return true"
+      end
+    end
+  end
+
+  describe "integration with Mix" do
+    test "inherits Rebar manager" do
+      Mix.Project.push(Rebar3AsDep)
+      deps = Mix.Dep.loaded([])
+      assert Enum.all?(deps, &(&1.manager == :rebar3))
+    end
+
+
+    test "parses Rebar dependencies from rebar.config" do
+      Mix.Project.push(RebarAsDep)
+
+      deps = Mix.Dep.loaded([])
+      assert Enum.find(deps, &(&1.app == :rebar_dep))
+
+      assert Enum.find(deps, fn %Mix.Dep{app: app, opts: opts} ->
+        if app == :git_rebar do
+          assert Enum.find(opts, &match?({:git, _}, &1))
+          assert Enum.find(opts, &match?({:ref, "master"}, &1))
+          true
        end
      end)
    end

-    unless Process.get(:inside_apps_another) do
-      flunk "Expected inside_apps_another to return true"
-    end
-  end
+    test "handles Rebar overrides" do
+      Mix.Project.push(RebarOverrideAsDep)

-  test "get and compile dependencies for Rebar" do
-    Mix.Project.push(RebarAsDep)
-
-    in_tmp "get and compile dependencies for Rebar", fn ->
-      Mix.Tasks.Deps.Get.run []
-      assert_received {:mix_shell, :info, ["* Getting git_rebar" <> _]}
-
-      Mix.Tasks.Deps.Compile.run []
-      assert_received {:mix_shell, :run, ["===> Compiling git_rebar\n"]}
-      assert_received {:mix_shell, :run, ["===> Compiling rebar_dep\n"]}
-      assert :git_rebar.any_function == :ok
-      assert :rebar_dep.any_function == :ok
-
-      load_paths = Mix.Dep.loaded([])
-        |> Enum.map(&Mix.Dep.load_paths(&1))
-        |> Enum.concat
-
-      assert File.exists?("_build/dev/lib/rebar_dep/ebin/rebar_dep.beam")
-      assert File.exists?("_build/dev/lib/git_rebar/ebin/git_rebar.beam")
-
-      # Assert we have no .compile.lock as a .compile.lock
-      # means we check for the Elixir version on every command.
-      refute File.exists?("_build/dev/lib/rebar_dep/.compile.lock")
-      refute File.exists?("_build/dev/lib/git_rebar/.compile.lock")
-
-      assert Enum.any?(load_paths, &String.ends_with?(&1, "git_rebar/ebin"))
-      assert Enum.any?(load_paths, &String.ends_with?(&1, "rebar_dep/ebin"))
-    end
-  end
-
-  test "get and compile dependencies for rebar3" do
-    Mix.Project.push(Rebar3AsDep)
-
-    in_tmp "get and compile dependencies for rebar3", fn ->
-      Mix.Tasks.Deps.Get.run []
-      assert_received {:mix_shell, :info, ["* Getting git_rebar " <> _]}
-
-      Mix.Tasks.Deps.Compile.run []
-      assert_received {:mix_shell, :run, ["===> Compiling git_rebar\n"]}
-      assert_received {:mix_shell, :run, ["===> Compiling rebar_dep\n"]}
-      assert :git_rebar.any_function == :ok
-      assert :rebar_dep.any_function == :ok
-
-      load_paths = Mix.Dep.loaded([])
-        |> Enum.map(&Mix.Dep.load_paths(&1))
-        |> Enum.concat
-
-      assert File.exists?("_build/dev/lib/rebar_dep/ebin/rebar_dep.beam")
-      assert File.exists?("_build/dev/lib/git_rebar/ebin/git_rebar.beam")
-
-      # Assert we have no .compile.lock as a .compile.lock
-      # means we check for the Elixir version on every command.
-      refute File.exists?("_build/dev/lib/rebar_dep/.compile.lock")
-      refute File.exists?("_build/dev/lib/git_rebar/.compile.lock")
-
-      assert Enum.any?(load_paths, &String.ends_with?(&1, "git_rebar/ebin"))
-      assert Enum.any?(load_paths, &String.ends_with?(&1, "rebar_dep/ebin"))
-    end
-  end
-
-  test "get and compile dependencies for Rebar with Mix" do
-    Mix.Project.push(RebarAsDep)
-
-    in_tmp "get and compile dependencies for Rebar with Mix", fn ->
-      File.write! MixTest.Case.tmp_path("rebar_dep/mix.exs"), """
-      defmodule RebarDep.Mixfile do
-        use Mix.Project
-
-        def project do
-          [app: :rebar_dep,
-           version: "0.0.1"]
-        end
+      in_tmp "Rebar overrides", fn ->
+        Mix.Tasks.Deps.Get.run []
+        assert Mix.Dep.loaded([]) |> Enum.map(& &1.app) ==
+               [:git_repo, :git_rebar, :rebar_override]
      end
-      """
+    after
+      purge [GitRepo.Mixfile]
+    end

-      Mix.Tasks.Deps.Compile.run []
-      assert_received {:mix_shell, :info, ["==> rebar_dep"]}
-      assert_received {:mix_shell, :info, ["Generated rebar_dep app"]}
-      assert File.regular?("_build/dev/lib/rebar_dep/ebin/rebar_dep.app")
+    test "gets and compiles dependencies for Rebar" do
+      Mix.Project.push(RebarAsDep)
+
+      in_tmp "get and compile dependencies for Rebar", fn ->
+        Mix.Tasks.Deps.Get.run []
+        assert_received {:mix_shell, :info, ["* Getting git_rebar" <> _]}
+
+        Mix.Tasks.Deps.Compile.run []
+        assert_received {:mix_shell, :run, ["===> Compiling git_rebar\n"]}
+        assert_received {:mix_shell, :run, ["===> Compiling rebar_dep\n"]}
+        assert :git_rebar.any_function == :ok
+        assert :rebar_dep.any_function == :ok
+
+        load_paths = Mix.Dep.loaded([])
+          |> Enum.map(&Mix.Dep.load_paths(&1))
+          |> Enum.concat
+
+        assert File.exists?("_build/dev/lib/rebar_dep/ebin/rebar_dep.beam")
+        assert File.exists?("_build/dev/lib/git_rebar/ebin/git_rebar.beam")
+
+        # Assert we have no .compile.lock as a .compile.lock
+        # means we check for the Elixir version on every command.
+        refute File.exists?("_build/dev/lib/rebar_dep/.compile.lock")
+        refute File.exists?("_build/dev/lib/git_rebar/.compile.lock")
+
+        assert Enum.any?(load_paths, &String.ends_with?(&1, "git_rebar/ebin"))
+        assert Enum.any?(load_paths, &String.ends_with?(&1, "rebar_dep/ebin"))
+      end
+    end
+
+    test "gets and compiles dependencies for rebar3" do
+      Mix.Project.push(Rebar3AsDep)
+
+      in_tmp "get and compile dependencies for rebar3", fn ->
+        Mix.Tasks.Deps.Get.run []
+        assert_received {:mix_shell, :info, ["* Getting git_rebar " <> _]}
+
+        Mix.Tasks.Deps.Compile.run []
+        assert_received {:mix_shell, :run, ["===> Compiling git_rebar\n"]}
+        assert_received {:mix_shell, :run, ["===> Compiling rebar_dep\n"]}
+        assert :git_rebar.any_function == :ok
+        assert :rebar_dep.any_function == :ok
+
+        load_paths = Mix.Dep.loaded([])
+          |> Enum.map(&Mix.Dep.load_paths(&1))
+          |> Enum.concat
+
+        assert File.exists?("_build/dev/lib/rebar_dep/ebin/rebar_dep.beam")
+        assert File.exists?("_build/dev/lib/git_rebar/ebin/git_rebar.beam")
+
+        # Assert we have no .compile.lock as a .compile.lock
+        # means we check for the Elixir version on every command.
+        refute File.exists?("_build/dev/lib/rebar_dep/.compile.lock")
+        refute File.exists?("_build/dev/lib/git_rebar/.compile.lock")
+
+        assert Enum.any?(load_paths, &String.ends_with?(&1, "git_rebar/ebin"))
+        assert Enum.any?(load_paths, &String.ends_with?(&1, "rebar_dep/ebin"))
+      end
+    end
+
+    test "gets and compiles dependencies for Rebar with Mix" do
+      Mix.Project.push(RebarAsDep)
+
+      in_tmp "get and compile dependencies for Rebar with Mix", fn ->
+        File.write! MixTest.Case.tmp_path("rebar_dep/mix.exs"), """
+        defmodule RebarDep.Mixfile do
+          use Mix.Project
+
+          def project do
+            [app: :rebar_dep,
+             version: "0.0.1"]
+          end
+        end
+        """
+
+        Mix.Tasks.Deps.Compile.run []
+        assert_received {:mix_shell, :info, ["==> rebar_dep"]}
+        assert_received {:mix_shell, :info, ["Generated rebar_dep app"]}
+        assert File.regular?("_build/dev/lib/rebar_dep/ebin/rebar_dep.app")
+      end
+    after
+      File.rm MixTest.Case.tmp_path("rebar_dep/mix.exs")
    end
-  after
-    File.rm MixTest.Case.tmp_path("rebar_dep/mix.exs")
  end
 end
@@ -1,6 +1,6 @@
 {application, elixir,
 [{description, "elixir"},
- {vsn, "1.4.0"},
+ {vsn, "1.4.2"},
 {modules, [
 	elixir
  ]},
Author	SHA1	Message	Date
José Valim	27fdc68ff4	Release v1.4.2	2017-02-16 15:15:31 +01:00
José Valim	2d4722a12f	Allow consuming multiple items from suspended enumerable in Stream.transform/3 Closes #5763. Closes #5772. Signed-off-by: José Valim <jose.valim@plataformatec.com.br>	2017-02-16 13:50:16 +01:00
José Valim	e002ac55b2	Incorporate new grapheme rules in Unicode 9 Signed-off-by: José Valim <jose.valim@plataformatec.com.br>	2017-02-13 15:01:34 +01:00
José Valim	fb89d5548d	Support middle expressions on trim mode, closes #5752 Signed-off-by: José Valim <jose.valim@plataformatec.com.br>	2017-02-09 20:41:50 +01:00
José Valim	8618d8effa	Properly cache apps_paths configuration, closes #5622 Signed-off-by: José Valim <jose.valim@plataformatec.com.br>	2017-02-05 11:06:37 +01:00
Eric Meadows-Jönsson	bd1c3ab8d7	Delete previous .ez archives Signed-off-by: Eric Meadows-Jönsson <eric.meadows.jonsson@gmail.com>	2017-02-01 23:19:33 +01:00
José Valim	2110342661	Improve docs for OptionParser Signed-off-by: José Valim <jose.valim@plataformatec.com.br>	2017-01-31 11:06:01 +01:00
Gal Tsubery	8533df25a9	Fix redefined function source location (#5720 ) When functions are redefined in a different file, the translation uses the first definition source location for all subsequent definitions. This manifests itself as warnings that are ascribed to the wrong file. Fixes #5719 Signed-off-by: José Valim <jose.valim@plataformatec.com.br>	2017-01-31 11:05:55 +01:00
José Valim	9e8ac51fb3	Only expand aliases known at compile time, closes #5721 Signed-off-by: José Valim <jose.valim@plataformatec.com.br>	2017-01-30 21:21:47 +01:00
Andrew Dryga	451b8eb9a3	Implement allow_inexistent_atoms for OptionParser (#5709 ) Signed-off-by: José Valim <jose.valim@plataformatec.com.br>	2017-01-28 22:45:59 +01:00
Aleksei Magusev	23742f1237	Correct datetime types spelling in Calendar	2017-01-26 13:34:20 +01:00
José Valim	3f9ffbeb82	Also mention new versions file	2017-01-26 13:24:15 +01:00
José Valim	0397a46bb7	Release v1.4.1	2017-01-26 13:02:51 +01:00
Eric Meadows-Jönsson	b87e58e7e3	Run `make clean` for erlang.mk (#5698 ) Signed-off-by: José Valim <jose.valim@plataformatec.com.br>	2017-01-26 12:59:35 +01:00
José Valim	6cb09a77e6	Update CHANGELOG	2017-01-24 14:13:11 +01:00
José Valim	12edab720c	Remove warning when making private functions overridable	2017-01-23 17:01:49 +01:00
José Valim	74f9a34015	Only pass overrides from parent to child rebar dep (#5687 ) Signed-off-by: José Valim <jose.valim@plataformatec.com.br>	2017-01-23 15:52:00 +01:00
Eric Meadows-Jönsson	49af97a800	Support rebar3 dependency package declaration (#5678 ) Currently only {app, “~> 1.0.0”, {pkg, hex_package}} is supported. This change will also support {app, {pkg, hex_package}}. Signed-off-by: José Valim <jose.valim@plataformatec.com.br>	2017-01-23 12:44:08 +01:00
José Valim	b806ee8559	override will likely be impl	2017-01-20 23:02:54 +01:00
James Fish	7d09c2b857	Wait for message before crash in exception tests Signed-off-by: José Valim <jose.valim@plataformatec.com.br>	2017-01-17 16:41:08 -07:00
José Valim	f38646781b	Do not warn on unused override attribute	2017-01-17 16:17:32 -07:00
José Valim	b5777de582	appended -> prepended, closes #5656 Signed-off-by: José Valim <jose.valim@plataformatec.com.br>	2017-01-14 11:11:45 +01:00
Andrea Leopardi	67f77ec1cd	Fix Path.join/1 for lists of one element (#5639 ) Also fix the spec for Path.join/1 to mention that the input list of paths has to be non-empty, and fix the documentation to mention that this function takes a list of paths (Path.t), not strings like it said before. Signed-off-by: José Valim <jose.valim@plataformatec.com.br>	2017-01-10 11:02:40 +01:00
Saša Jurić	8a91f648a8	Fix typespecs in Calendar (#5635 ) Signed-off-by: José Valim <jose.valim@plataformatec.com.br>	2017-01-09 20:23:16 +01:00
Devon Estes	7f61b0ff05	Add `-` to Regex.escape/1 (#5626 ) We were not escaping the `-` character, commonly found in character classes, in cases where it is being used on its own. I've added a test for this new escape behavior, and also implemented the change. Signed-off-by: José Valim <jose.valim@plataformatec.com.br>	2017-01-08 18:16:06 +01:00
José Valim	c830fde31e	Improve task docs, closes #5618	2017-01-05 11:02:53 +01:00
Matt Widman	019e6b85e6	Correct documentation referencing System.schedulers_online/1 to System.schedulers_online/0 (#5568 ) Signed-off-by: José Valim <jose.valim@plataformatec.com.br>	2017-01-05 10:56:17 +01:00
José Valim	097fe87eb7	Disable ansi detection for powershell to avoid false positives Closes #5615	2017-01-05 10:49:00 +01:00
Wojtek Mach	6ec1e7f7b4	Add new functions Time.utc_now/0, NaiveDateTime.utc_now/0 to CHANGELOG (#5617 )	2017-01-05 10:42:23 +01:00
@@ -1 +1 @@
 .4.0
 .4.2