Release v1.3.1

Signed-off-by: José Valim <jose.valim@plataformatec.com.br>

CHANGELOG.md

@@ -102,11 +102,11 @@ In any case, the previous behaviour can be brought back with the `--verbose` fla
 
 ### mix xref
 
-Speaking about warnings, Mix v1.3 includes a new task called `xref` that performs cross reference checks in your code. One of such checks is the ability to find calls to modules and functions that do not exist. For example, if in your library code you call `ThisModuleDoesNotExist.foo(1, 2, 3)`, `mix xref --unreachable` will be able to find such code and let you know about it.
+Speaking about warnings, Mix v1.3 includes a new task called `xref` that performs cross reference checks in your code. One of such checks is the ability to find calls to modules and functions that do not exist. For example, if in your library code you call `ThisModuleDoesNotExist.foo(1, 2, 3)`, `mix xref unreachable` will be able to find such code and let you know about it.
 
 Since such checks can discover possible bugs in your codebase, a new compiler called `xref` has been added to `Mix.compilers/0`, so they run by default every time you compile your code.
 
-We have included other modes in `xref`, such as `mix xref --callers Foo`, to find all places in your code that a function from the module `Foo` is called. We hope other tools and text editors can leverage such features to provide useful functionality for their users.
+We have included other modes in `xref`, such as `mix xref callers Foo`, to find all places in your code that a function from the module `Foo` is called. We hope other tools and text editors can leverage such features to provide useful functionality for their users.
 
 ### Better dependency tracking
 
@@ -268,7 +268,38 @@ end
 
 By restricting hierarchies in favor of named setups, it is straight-forward for the developer to glance at each describe block and know exactly the setup steps involved.
 
-## v1.3.0-rc.1 (2016-08-06)
+## v1.3.1 (2016-06-28)
+
+### 1. Enhancements
+
+#### IEx
+
+  * [IEx.Helpers] `Add import_file_if_available` for importing files only if they are available
+  * [IEx.Helpers] `Add import_if_available` for importing modules only if they are available
+
+### 2. Bug fixes
+
+#### Elixir
+
+  * [Kernel] Ensure structs can be expanded in dynamic module names
+  * [Kernel] Ensure aliases warnings are not accidentally discarded when the same module is imported
+  * [Kernel.ParallelCompiler] Ensure two modules with cyclic struct dependencies cannot run into a deadlock when compiling
+  * [Kernel.Typespec] Support module attributes in remote types
+  * [Module] Do not expect stacktraces to be always present when dispatching to locals during the module compilation
+
+#### IEx
+
+  * [IEx.Helpers] Fix `h` helper for operators
+
+#### Mix
+
+  * [Mix] Do not load modules for xref purposes, instead use BEAM info
+  * [Mix] Ensure `deps.check` does not check archives (that's done in loadpaths)
+  * [Mix] Validate application properties before traversing them
+  * [Mix] Check for proper Makefile when compiling on Windows
+  * [Mix] Enforce space after comma in `mix do`
+
+## v1.3.0 (2016-06-21)
 
 ### 1. Enhancements
 
@@ -283,6 +314,7 @@ By restricting hierarchies in favor of named setups, it is straight-forward for
   * [CLI] Add `--logger-otp-reports BOOL` and `--logger-sasl-reports BOOL` switches
   * [Compiler] Emit a summary of compilation errors when modules are missing
   * [Enum] Add `Enum.group_by/3` that allows developers to map on the value being grouped
+  * [Enum] Make list values in maps returned by `Enum.group_by/2` and `Enum.group_by/3` preserve the order of the input enumerable instead of reversing it.
   * [Enum] Add `Enum.drop_every/2` that drops every `nth`, including the first one
   * [Exception] Suggest possible functions on `UndefinedFunctionError` for existing modules
   * [Exception] Warn if unknown fields are given to `raise/2`

Makefile

@@ -1,7 +1,7 @@
 REBAR ?= "$(CURDIR)/rebar"
 PREFIX ?= /usr/local
-DOCS := master
-CANONICAL := master
+DOCS := v1.3
+CANONICAL := stable
 ELIXIRC := bin/elixirc --verbose --ignore-module-conflict
 ERLC := erlc -I lib/elixir/include
 ERL := erl -I lib/elixir/include -noshell -pa lib/elixir/ebin

RELEASE.md

@@ -28,7 +28,7 @@ This document simply outlines the release process:
 
 11. Create a new branch "vMAJOR.MINOR"
 
-12. Move docs generation to `docs/vMAJOR.MINOR` in Makefile, set CANONICAL reference and copy them to `docs/stable` (change index.html accordingly)
+12. Move docs generation to `docs/vMAJOR.MINOR` in Makefile, set CANONICAL to stable and copy them to `docs/stable` (change index.html accordingly)
 
 13. In master, bump versions, start new CHANGELOG, add `-dev` back and commit "Start vVERSION+1"
 

VERSION

@@ -1 +1 @@
-1.3.0-rc.1
+1.3.1

lib/elixir/lib/enum.ex

@@ -997,7 +997,8 @@ defmodule Enum do
 
   # TODO: Remove on 2.0
   def group_by(enumerable, dict, fun) when is_function(fun, 1) do
-    IO.warn "Enum.group_by/3 with a map/dictionary as second element is deprecated, please use a map instead"
+    IO.warn "Enum.group_by/3 with a map/dictionary as second element is deprecated. " <>
+      "A map is used by default and it is no longer required to pass one to this function"
     reduce(reverse(enumerable), dict, fn(entry, categories) ->
       Dict.update(categories, fun.(entry), [entry], &[entry | &1])
     end)

lib/elixir/lib/gen_server.ex

@@ -746,7 +746,7 @@ defmodule GenServer do
   @spec reply(from, term) :: :ok
   def reply(client, reply)
 
-  def reply({to, tag}, reply) do
+  def reply({to, tag}, reply) when is_pid(to) do
     try do
       send(to, {tag, reply})
       :ok

lib/elixir/lib/kernel.ex

@@ -3473,7 +3473,7 @@ defmodule Kernel do
       end
 
       unquote(builder)
-
+      Kernel.Utils.announce_struct(__MODULE__)
       fields
     end
   end

lib/elixir/lib/kernel/lexical_tracker.ex

@@ -148,17 +148,15 @@ defmodule Kernel.LexicalTracker do
 
   def handle_cast({:add_import, module, fas, line, warn}, state) when is_atom(module) do
     directives =
-      for {{:import, {import_module, _, _}}, _} = directive <- state.directives,
-          module != import_module,
-          do: directive,
-          into: %{}
-
-    directives = add_directive(directives, module, line, warn, :import)
+      state.directives
+      |> Enum.reject(&match?({{:import, {^module, _, _}}, _}, &1))
+      |> :maps.from_list
+      |> add_directive(module, line, warn, :import)
 
     directives =
-      Enum.reduce fas, directives, fn {function, arity}, directives ->
+      Enum.reduce(fas, directives, fn {function, arity}, directives ->
         add_directive(directives, {module, function, arity}, line, warn, :import)
-      end
+      end)
 
     {:noreply, %{state | directives: directives}}
   end

lib/elixir/lib/kernel/typespec.ex

@@ -864,6 +864,19 @@ defmodule Kernel.Typespec do
     {:op, line(meta), op, {:integer, line(meta), integer}}
   end
 
+  # Handle remote calls in the form of @module_attribute.type.
+  # These are not handled by the general remote type clause as calling
+  # Macro.expand/2 on the remote does not expand module attributes (but expands
+  # things like __MODULE__).
+  defp typespec({{:., meta, [{:@, _, [{attr, _, _}]}, name]}, _, args} = orig, vars, caller) do
+    remote = Module.get_attribute(caller.module, attr)
+    unless is_atom(remote) and remote != nil do
+      message = "invalid remote in typespec: #{Macro.to_string(orig)} (@#{attr} is #{inspect remote})"
+      compile_error(caller, message)
+    end
+    remote_type({typespec(remote, vars, caller), meta, typespec(name, vars, caller), args}, vars, caller)
+  end
+
   # Handle remote calls
   defp typespec({{:., meta, [remote, name]}, _, args} = orig, vars, caller) do
     remote = Macro.expand remote, caller

lib/elixir/lib/kernel/utils.ex

@@ -84,4 +84,11 @@ defmodule Kernel.Utils do
      List.wrap(Module.get_attribute(module, :enforce_keys)),
      Module.get_attribute(module, :derive)}
   end
+
+  def announce_struct(module) do
+    case :erlang.get(:elixir_compiler_pid) do
+      :undefined -> :ok
+      pid -> send(pid, {:struct_available, module})
+    end
+  end
 end

lib/elixir/lib/macro.ex

@@ -1098,7 +1098,7 @@ defmodule Macro do
 
       case expand do
         {:ok, receiver, quoted} ->
-          next = :elixir_counter.next
+          next = :erlang.unique_integer()
           {:elixir_quote.linify_with_context_counter(0, {receiver, next}, quoted), true}
         {:ok, _receiver, _name, _args} ->
           {original, false}
@@ -1119,7 +1119,7 @@ defmodule Macro do
 
         case expand do
           {:ok, receiver, quoted} ->
-            next = :elixir_counter.next
+            next = :erlang.unique_integer()
             {:elixir_quote.linify_with_context_counter(0, {receiver, next}, quoted), true}
           :error ->
             {original, false}

lib/elixir/lib/module.ex

@@ -369,7 +369,7 @@ defmodule Module do
     * `@compile :debug_info` - includes `:debug_info` regardless of the
       setting in `Code.compiler_options`
 
-    * `@compile {:debug_info, false} - disables `:debug_info` regardless
+    * `@compile {:debug_info, false}` - disables `:debug_info` regardless
       of the setting in `Code.compiler_options`
 
     * `@compile {:inline, some_fun: 2, other_fun: 3}` - inlines the given
@@ -499,7 +499,7 @@ defmodule Module do
       raise ArgumentError, "expected :file to be given as option"
     end
 
-    next = :elixir_counter.next
+    next = :erlang.unique_integer()
     line = Keyword.get(opts, :line, 0)
     quoted = :elixir_quote.linify_with_context_counter(line, {module, next}, quoted)
     :elixir_module.compile(module, quoted, [], :elixir.env_for_eval(opts))

lib/elixir/lib/system.ex

@@ -34,7 +34,6 @@ defmodule System do
       next = System.monotonic_time()
       diff = next - prev
 
-
   Generally speaking, the VM provides three time measurements:
 
     * `os_time/0` - the time reported by the OS. This time may be
@@ -157,8 +156,7 @@ defmodule System do
     %{build:    build,
       date:     get_date,
       revision: revision,
-      version:  version,
-      }
+      version:  version}
   end
 
   # Returns a string of the build info
@@ -730,6 +728,30 @@ defmodule System do
     :os.system_time(normalize_time_unit(unit))
   end
 
+  @doc """
+  Returns the OTP release number.
+  """
+  @spec otp_release :: String.t
+  def otp_release do
+    :erlang.list_to_binary :erlang.system_info(:otp_release)
+  end
+
+  @doc """
+  Returns the number of schedulers in the VM.
+  """
+  @spec schedulers :: pos_integer
+  def schedulers do
+    :erlang.system_info(:schedulers)
+  end
+
+  @doc """
+  Returns the number of schedulers online in the VM.
+  """
+  @spec schedulers_online :: pos_integer
+  def schedulers_online do
+    :erlang.system_info(:schedulers_online)
+  end
+
   @doc """
   Generates and returns an integer that is unique in the current runtime
   instance.

lib/elixir/src/elixir_counter.erl

@@ -1,38 +0,0 @@
--module(elixir_counter).
--export([start_link/0, init/1, handle_call/3, handle_cast/2,
-  handle_info/2, terminate/2, code_change/3, next/0]).
--behaviour(gen_server).
-
--define(timeout, 30000).    %% 30 seconds
--define(limit, 4294967295). %% 2^32 - 1
-
-next() ->
-  gen_server:call(?MODULE, next, ?timeout).
-
-start_link() ->
-  gen_server:start_link({local, ?MODULE}, ?MODULE, 0, []).
-
-init(Counter) ->
-  {ok, Counter}.
-
-handle_call(next, _From, Counter) ->
-  {reply, Counter, bump(Counter)};
-handle_call(Request, _From, Counter) ->
-  {stop, {badcall, Request}, Counter}.
-
-handle_cast(Request, Counter) ->
-  {stop, {badcast, Request}, Counter}.
-
-handle_info(_Request, Counter) ->
-  {noreply, Counter}.
-
-terminate(_Reason, _Counter) ->
-  ok.
-
-code_change(_Old, Counter, _Extra) ->
-  {ok, Counter}.
-
-bump(Counter) when Counter < ?limit ->
-  Counter + 1;
-bump(_Counter) ->
-  0.

lib/elixir/src/elixir_def.erl

@@ -105,8 +105,6 @@ store_definition(Meta, Line, Kind, CheckClauses, Name, Args, Guards, Body, KeepL
   store_each(CheckClauses, Kind, File, Location, Module, DefaultsLength, Function),
   [store_each(false, Kind, File, Location, Module, 0,
     default_function_for(Kind, Name, Default)) || Default <- Defaults],
-
-  make_struct_available(Kind, Module, Name, Args),
   {Name, Arity}.
 
 %% @on_definition
@@ -117,16 +115,6 @@ run_on_definition_callbacks(Kind, Line, Module, Name, Args, Guards, Expr, E) ->
   _ = [Mod:Fun(Env, Kind, Name, Args, Guards, Expr) || {Mod, Fun} <- Callbacks],
   ok.
 
-make_struct_available(def, Module, '__struct__', []) ->
-  case erlang:get(elixir_compiler_pid) of
-    undefined -> ok;
-    Pid ->
-      Pid ! {struct_available, Module},
-      ok
-  end;
-make_struct_available(_, _, _, _) ->
-  ok.
-
 %% Retrieve location from meta file (if Key == keep)
 %% or @file, otherwise nil
 
@@ -218,7 +206,7 @@ is_macro(_)         -> false.
 unwrap_definitions(File, Module) ->
   Table = elixir_module:defs_table(Module),
   Entries = ets:match(Table, {{def, '$1'}, '_', '_', '_', '_', '_', '_'}),
-  {All, Private} = unwrap_definition(Entries, File, Module, Table, [], []),
+  {All, Private} = unwrap_definition(lists:sort(Entries), File, Module, Table, [], []),
   Unreachable = elixir_locals:warn_unused_local(File, Module, Private),
   split_definition(All, Unreachable, [], [], [], [], [], {[], []}).
 

lib/elixir/src/elixir_dispatch.erl

@@ -202,7 +202,7 @@ expand_macro_named(Meta, Receiver, Name, Arity, Args, E) ->
 
 expand_quoted(Meta, Receiver, Name, Arity, Quoted, E) ->
   Line = ?line(Meta),
-  Next = elixir_counter:next(),
+  Next = erlang:unique_integer(),
 
   try
     elixir_exp:expand(

lib/elixir/src/elixir_fn.erl

@@ -8,7 +8,7 @@ translate(Meta, Clauses, S) ->
     {Args, Guards} = elixir_clauses:extract_splat_guards(ArgsWithGuards),
     {TClause, TS } = elixir_clauses:clause(CMeta, fun translate_fn_match/2,
                                             Args, Expr, Guards, Acc),
-    {TClause, elixir_scope:mergef(S, TS)}
+    {TClause, elixir_scope:mergec(S, TS)}
   end,
 
   {TClauses, NS} = lists:mapfoldl(Transformer, S, Clauses),
@@ -111,7 +111,7 @@ handle_capture(false, Meta, Expr, E, Sequential) ->
   do_capture(Meta, Expr, E, Sequential).
 
 do_capture(Meta, Expr, E, Sequential) ->
-  case do_escape(Expr, elixir_counter:next(), E, []) of
+  case do_escape(Expr, erlang:unique_integer(), E, []) of
     {_, []} when not Sequential ->
       invalid_capture(Meta, Expr, E);
     {EExpr, EDict} ->

lib/elixir/src/elixir_locals.erl

@@ -31,7 +31,7 @@ local_for(Module, Name, Arity, Given) ->
     {Kind, Ann, [_ | _] = Clauses} when Given == nil; Kind == Given ->
       get_function(Ann, Module, Clauses);
     _ ->
-      [_ | T] = erlang:get_stacktrace(),
+      {current_stacktrace, [_ | T]} = erlang:process_info(self(), current_stacktrace),
       erlang:raise(error, undef, [{Module, Name, Arity, []} | T])
   end.
 

lib/elixir/src/elixir_map.erl

@@ -19,7 +19,7 @@ expand_struct(Meta, Left, Right, #{context := Context} = E) ->
     true when is_atom(ELeft) ->
       %% We always record structs when they are expanded
       %% as they expect the reference at compile time.
-      elixir_lexical:record_remote(ELeft, nil, ?m(E, lexical_tracker));
+      elixir_lexical:record_remote(ELeft, '__struct__', 1, nil, ?line(Meta), ?m(E, lexical_tracker));
     true ->
       ok;
     false when Context == match ->
@@ -31,7 +31,9 @@ expand_struct(Meta, Left, Right, #{context := Context} = E) ->
   end,
 
   EMeta =
-    case lists:member(ELeft, ?m(E, context_modules)) of
+    %% We also include the current module because it won't be present
+    %% in context module in case the module name is defined dynamically.
+    case lists:member(ELeft, [?m(E, module) | ?m(E, context_modules)]) of
       true  -> [{struct, context} | Meta];
       false -> Meta
     end,

lib/elixir/src/elixir_scope.erl

@@ -2,8 +2,7 @@
 -module(elixir_scope).
 -export([translate_var/4, build_var/2, context_info/1,
   load_binding/2, dump_binding/2,
-  mergev/2, mergec/2, mergef/2,
-  merge_vars/2, merge_opt_vars/2,
+  mergev/2, mergec/2, merge_vars/2, merge_opt_vars/2,
   warn_unsafe_var/4, warn_underscored_var_access/3, format_error/1
 ]).
 -include("elixir.hrl").
@@ -128,14 +127,6 @@ mergec(S1, S2) ->
     caller=S2#elixir_scope.caller
  }.
 
-%% Similar to mergec but does not merge the user vars counter.
-
-mergef(S1, S2) ->
-  S1#elixir_scope{
-    super=S2#elixir_scope.super,
-    caller=S2#elixir_scope.caller
- }.
-
 %% Mergers.
 
 merge_vars(V, V) -> V;
@@ -212,8 +203,8 @@ format_error({unused_match, Name, Kind}) ->
 
 format_error({unsafe_var, Name}) ->
   io_lib:format("the variable \"~ts\" is unsafe as it has been set inside "
-                "a case/cond/receive/if/&&/||. Please explicitly return the "
-                "variable value instead. For example:\n\n"
+                "a case/cond/receive/if/&&/||. Please explicitly return "
+                "the variable value instead. For example:\n\n"
                 "    case int do\n"
                 "      1 -> atom = :one\n"
                 "      2 -> atom = :two\n"

lib/elixir/src/elixir_sup.erl

@@ -15,7 +15,7 @@ init(ok) ->
       2000,                         % Shutdown = brutal_kill | int() >= 0 | infinity
       worker,                       % Type     = worker | supervisor
       [elixir_config]               % Modules  = [Module] | dynamic
-   },
+    },
 
     {
       elixir_code_server,
@@ -25,17 +25,7 @@ init(ok) ->
       2000,                         % Shutdown = brutal_kill | int() >= 0 | infinity
       worker,                       % Type     = worker | supervisor
       [elixir_code_server]          % Modules  = [Module] | dynamic
-   },
-
-    {
-      elixir_counter,
-      {elixir_counter, start_link, []},
-
-      permanent,                    % Restart  = permanent | transient | temporary
-      2000,                         % Shutdown = brutal_kill | int() >= 0 | infinity
-      worker,                       % Type     = worker | supervisor
-      [elixir_counter]              % Modules  = [Module] | dynamic
-   }
+    }
   ],
 
   {ok, {{one_for_one, 3, 10}, Workers}}.

lib/elixir/src/elixir_with.erl

@@ -45,7 +45,7 @@ expand(Meta, Args, E) ->
 
   {ECases, EC} = lists:mapfoldl(fun expand/2, E, Cases),
   {EDoExpr, _} = elixir_exp:expand(DoExpr, EC),
-  {EElseExpr, _} = expand_else(ElseExpr, E),
+  {EElseExpr, _} = expand_else(Meta, ElseExpr, E),
   {{with, Meta, ECases ++ [[{do, EDoExpr} | EElseExpr]]}, E}.
 
 expand({'<-', Meta, [Left, Right]}, E) ->
@@ -55,25 +55,36 @@ expand({'<-', Meta, [Left, Right]}, E) ->
 expand(X, E) ->
   elixir_exp:expand(X, E).
 
-expand_else(KV, E) when is_list(KV) ->
+expand_else(_Meta, KV, E) when is_list(KV) ->
   {[{do, EClauses}], EC} = elixir_exp_clauses:'case'([], [{do, KV}], E),
   {[{else, EClauses}], EC};
-expand_else(nil, E) ->
-  {[], E}.
+expand_else(_Meta, nil, E) ->
+  {[], E};
+expand_else(Meta, _KV, E) ->
+  Message = "expected -> clauses for else in with",
+  elixir_errors:compile_error(Meta, ?m(E, file), Message, []).
 
 %% Translation
 
 translate(Meta, Args, S) ->
   {Parts, [{do, Expr} | ExprList]} = elixir_utils:split_last(Args),
-  CaseExpr =
-    case ExprList of
-      [{else, ElseExpr}] ->
-        build_else(Meta, build_case(Parts, {ok, Expr}, fun(X) -> {error, X} end), ElseExpr);
-      [] ->
-        build_case(Parts, Expr, fun(X) -> X end)
-    end,
-  {TC, TS} = elixir_translator:translate(CaseExpr, S#elixir_scope{extra=nil}),
-  {TC, elixir_scope:mergec(S, TS)}.
+  case ExprList of
+    [{else, ElseExpr}] ->
+      {TCases, TS} = translate_case(Parts, {ok, Expr}, fun(X) -> {error, X} end, S),
+      translate_else(Meta, TCases, ElseExpr, TS);
+    [] ->
+      translate_case(Parts, Expr, fun(X) -> X end, S)
+  end.
+
+translate_case(Parts, DoExpr, Wrapper, S) ->
+  Cases = build_case(Parts, DoExpr, Wrapper),
+  {TCases, TS} = elixir_translator:translate(Cases, S#elixir_scope{extra=nil}),
+  {TCases, elixir_scope:mergec(S, TS)}.
+
+translate_else(Meta, WithCases, ElseExpr, S) ->
+  ElseClauses = build_else(Meta, ElseExpr),
+  {TClauses, TS} = elixir_clauses:clauses(Meta, ElseClauses, S#elixir_scope{extra=nil}),
+  {{'case', ?ann(Meta), WithCases, TClauses}, elixir_scope:mergec(S, TS)}.
 
 build_case([{'<-', Meta, [{Name, _, Ctx}, _] = Args} | Rest], DoExpr, Wrapper)
     when is_atom(Name) andalso is_atom(Ctx) ->
@@ -90,18 +101,15 @@ build_case([Expr | Rest], DoExpr, Wrapper) ->
 build_case([], DoExpr, _Wrapper) ->
   DoExpr.
 
-build_else(Meta, WithCases, ElseClauses) ->
+build_else(Meta, ElseClauses) ->
   Result = {result, Meta, ?MODULE},
-  Clauses = [
-    {'->', Meta, [[{ok, Result}], Result]}
-    | else_to_error_clause(ElseClauses)
-  ] ++ [build_raise(Meta)],
-  {'case', Meta, [WithCases, [{do, Clauses}]]}.
+  [{match, Meta, [{ok, Result}], Result} |
+    each_clause_to_error_match(ElseClauses)] ++ [build_raise(Meta)].
 
-else_to_error_clause(Clauses) ->
-  [{'->', Meta, [[{error, Match}], Expr]} ||
+each_clause_to_error_match(Clauses) ->
+  [{match, Meta, [{error, Match}], Expr} ||
     {'->', Meta, [[Match], Expr]} <- Clauses].
 
 build_raise(Meta) ->
-  Other = {raise, Meta, ?MODULE},
-  {'->', ?generated, [[{error, Other}], {{'.', Meta, [erlang, error]}, Meta, [{with_clause, Other}]}]}.
+  Other = {other, Meta, ?MODULE},
+  {match, ?generated, [{error, Other}], {{'.', Meta, [erlang, error]}, Meta, [{with_clause, Other}]}}.

lib/elixir/test/elixir/behaviour_test.exs

@@ -3,30 +3,33 @@ Code.require_file "test_helper.exs", __DIR__
 defmodule BehaviourTest do
   use ExUnit.Case, async: true
 
-  defmodule Sample do
-    use Behaviour
+  {_, _, sample_binary, _} =
+    defmodule Sample do
+      use Behaviour
 
-    defcallback first(integer) :: integer
+      defcallback first(integer) :: integer
 
-    defcallback foo(atom(), binary) :: binary
+      defcallback foo(atom(), binary) :: binary
 
-    defcallback bar(External.hello, my_var :: binary) :: binary
+      defcallback bar(External.hello, my_var :: binary) :: binary
 
-    defcallback guarded(my_var) :: my_var when my_var: binary
+      defcallback guarded(my_var) :: my_var when my_var: binary
 
-    defcallback orr(atom | integer) :: atom
+      defcallback orr(atom | integer) :: atom
 
-    defcallback literal(123, {atom}, :atom, [integer], true) :: atom
+      defcallback literal(123, {atom}, :atom, [integer], true) :: atom
 
-    defmacrocallback last(integer) :: Macro.t
-  end
+      defmacrocallback last(integer) :: Macro.t
+    end
+
+  @sample_binary sample_binary
 
   test "callbacks" do
     assert Sample.__behaviour__(:callbacks) == [first: 1, guarded: 1, "MACRO-last": 2, literal: 5, orr: 1, foo: 2, bar: 2]
   end
 
   test "specs" do
-    assert length(Keyword.get_values(Sample.module_info[:attributes], :callback)) == 7
+    assert length(Kernel.Typespec.beam_callbacks(@sample_binary)) == 7
   end
 
   test "default is not supported" do

lib/elixir/test/elixir/fixtures/parallel_struct/bar.ex

@@ -1,4 +1,3 @@
 defmodule Bar do
-  defstruct name: ""
-  def foo?(%Foo{}), do: true
+  defstruct name: "", foo: %Foo{}
 end

lib/elixir/test/elixir/kernel/dialyzer_test.exs

@@ -48,6 +48,7 @@ defmodule Kernel.DialyzerTest do
     {:ok, [outdir: dir, dialyzer: dialyzer]}
   end
 
+  @tag otp19: false
   test "no warnings on valid remote calls", context do
     copy_beam! context, Dialyzer.RemoteCall
     assert_dialyze_no_warnings! context
@@ -58,6 +59,7 @@ defmodule Kernel.DialyzerTest do
     assert_dialyze_no_warnings! context
   end
 
+  @tag otp19: false
   test "no warnings on raise", context do
     copy_beam! context, Dialyzer.Raise
     assert_dialyze_no_warnings! context

lib/elixir/test/elixir/kernel/fn_test.exs

@@ -18,6 +18,18 @@ defmodule Kernel.FnTest do
     refute (fn ^x -> true; _ -> false end).(1.0)
   end
 
+  test "case function hoisting does not affect anonymous fns" do
+    assert :undefined =
+             (if is_a?(:atom, 0) do
+                user = :defined
+              else
+                (fn() ->
+                  user = :undefined
+                  user
+                end).()
+              end)
+  end
+
   test "capture with access" do
     assert (&(&1[:hello])).([hello: :world]) == :world
   end

lib/elixir/test/elixir/kernel/lexical_tracker_test.exs

@@ -161,6 +161,7 @@ defmodule Kernel.LexicalTrackerTest do
           &R.func/0
           &Remote.func/0
           &Integer.is_even/1
+          %Macro.Env{}
         end
 
         &extract/2
@@ -171,6 +172,8 @@ defmodule Kernel.LexicalTrackerTest do
 
         &is_record/1; def b(a), do: is_record(a)
 
+        %Macro.Env{}
+
         Kernel.LexicalTracker.remote_dispatches(__ENV__.module)
       end |> elem(3)
       """)
@@ -181,14 +184,16 @@ defmodule Kernel.LexicalTrackerTest do
     assert {9, Integer, :is_even, 1} in compile_remote_calls
     assert {15, Record, :is_record, 1} in compile_remote_calls
     assert {18, Integer, :is_even, 1} in compile_remote_calls
-    assert {21, Record, :extract, 2} in compile_remote_calls
-    assert {22, Record, :is_record, 1} in compile_remote_calls
-    assert {23, Remote, :func, 0} in compile_remote_calls
+    assert {19, Macro.Env, :__struct__, 1} in compile_remote_calls
+    assert {22, Record, :extract, 2} in compile_remote_calls
+    assert {23, Record, :is_record, 1} in compile_remote_calls
     assert {24, Remote, :func, 0} in compile_remote_calls
-    assert {25, Integer, :is_even, 1} in compile_remote_calls
-    assert {27, Kernel, :def, 2} in compile_remote_calls
-    assert {27, Record, :is_record, 1} in compile_remote_calls
-    assert {29, Kernel.LexicalTracker, :remote_dispatches, 1} in compile_remote_calls
+    assert {25, Remote, :func, 0} in compile_remote_calls
+    assert {26, Integer, :is_even, 1} in compile_remote_calls
+    assert {28, Kernel, :def, 2} in compile_remote_calls
+    assert {28, Record, :is_record, 1} in compile_remote_calls
+    assert {30, Macro.Env, :__struct__, 1} in compile_remote_calls
+    assert {32, Kernel.LexicalTracker, :remote_dispatches, 1} in compile_remote_calls
 
     runtime_remote_calls = unroll_dispatches(runtime_remote_calls)
     assert {7, Record, :extract, 2} in runtime_remote_calls
@@ -200,7 +205,7 @@ defmodule Kernel.LexicalTrackerTest do
     assert {16, Remote, :func, 0} in runtime_remote_calls
     assert {17, Remote, :func, 0} in runtime_remote_calls
     assert {18, :erlang, :==, 2} in runtime_remote_calls
-    assert {27, :erlang, :is_tuple, 1} in runtime_remote_calls
+    assert {28, :erlang, :is_tuple, 1} in runtime_remote_calls
   end
 
   defp unroll_dispatches(dispatches) do

lib/elixir/test/elixir/kernel/macros_test.exs

@@ -48,7 +48,7 @@ defmodule Kernel.MacrosTest do
     assert_raise UndefinedFunctionError, fn -> x.value end
   end
 
-  test "bang do block" do
+  test "macros with bang and do block have proper precedence" do
     import Kernel.MacrosTest.Nested
     assert (do_identity! do 1 end) == 1
     assert (Kernel.MacrosTest.Nested.do_identity! do 1 end) == 1

lib/elixir/test/elixir/kernel/raise_test.exs

@@ -283,9 +283,10 @@ defmodule Kernel.RaiseTest do
   end
 
   test "badfun error" do
-    x = :example
+    # Avoid "invalid function call" warning in >= OTP 19
+    x = fn -> :example end
     result = try do
-      x.(2)
+      x.().(2)
     rescue
       x in [BadFunctionError] -> Exception.message(x)
     end

lib/elixir/test/elixir/kernel/typespec_test.exs

@@ -455,6 +455,27 @@ defmodule Kernel.TypespecTest do
            types(module)
   end
 
+  test "@type with module attributes" do
+    module = test_module do
+      @keyword Keyword
+      @type kw :: @keyword.t
+      @type kw(value) :: @keyword.t(value)
+    end
+
+    assert [type: {:kw, {:remote_type, _, [{:atom, _, Keyword}, {:atom, _, :t}, []]}, _},
+            type: {:kw, {:remote_type, _, [{:atom, _, Keyword}, {:atom, _, :t}, [{:var, _, :value}]]}, [{:var, _, :value}]}] =
+      types(module)
+  end
+
+  test "invalid remote @type with module attribute that does not evaluate to a module" do
+    assert_raise CompileError, ~r/\(@foo is "bar"\)/, fn ->
+      test_module do
+        @foo "bar"
+        @type t :: @foo.t
+      end
+    end
+  end
+
   test "defines_type?" do
     test_module do
       @type mytype :: tuple

lib/elixir/test/elixir/kernel/warning_test.exs

@@ -310,6 +310,20 @@ defmodule Kernel.WarningTest do
     purge Sample
   end
 
+  test "unused alias when also import" do
+    assert capture_err(fn ->
+      Code.compile_string """
+      defmodule Sample do
+        alias :lists, as: List
+        import MapSet
+        new()
+      end
+      """
+    end) =~ "unused alias List"
+  after
+    purge Sample
+  end
+
   test "unused inside dynamic module" do
     import List, only: [flatten: 1], warn: false
 
@@ -421,8 +435,8 @@ defmodule Kernel.WarningTest do
     assert capture_err(fn ->
       Code.eval_string ~S"""
       defmodule Sample do
-      def hello(arg \\ 0), do: nil
-      def hello(arg), do: nil
+        def hello(arg \\ 0), do: nil
+        def hello(arg), do: nil
       end
       """
     end) =~ "definitions with multiple clauses and default values require a function head"
@@ -551,7 +565,7 @@ defmodule Kernel.WarningTest do
       """
     end) =~ "undefined behaviour function foo/0 (for behaviour Sample1)"
   after
-    purge [Sample1, Sample2, Sample3]
+    purge [Sample1, Sample2]
   end
 
   test "undefined macro for behaviour" do
@@ -567,7 +581,7 @@ defmodule Kernel.WarningTest do
       """
     end) =~ "undefined behaviour macro foo/0 (for behaviour Sample1)"
   after
-    purge [Sample1, Sample2, Sample3]
+    purge [Sample1, Sample2]
   end
 
   test "undefined behavior" do

lib/elixir/test/elixir/kernel/with_test.exs

@@ -47,6 +47,13 @@ defmodule Kernel.WithTest do
     assert result == :error
   end
 
+  test "does not leak variables to else" do
+    state = 1
+    result = with 1 <- state, state = 2, :ok <- error(), do: state, else: (_ -> state)
+    assert result == 1
+    assert state == 1
+  end
+
   test "errors in with" do
     assert_raise RuntimeError, fn ->
       with({:ok, res} <- oops(), do: res)
@@ -68,6 +75,12 @@ defmodule Kernel.WithTest do
     end
   end
 
+  test "invalid else form" do
+    assert_raise CompileError, "nofile:1: expected -> clauses for else in with",  fn ->
+      Code.eval_quoted(quote do: with(_ <- true, do: :ok, else: :error))
+    end
+  end
+
   defp four() do
     4
   end

lib/elixir/test/elixir/map_test.exs

@@ -147,6 +147,16 @@ defmodule MapTest do
     %^struct{name: "john"} = user
   end
 
+  test "structs when using dynamic modules" do
+    defmodule Module.concat(MapTest, DynamicUser) do
+      defstruct [:name, :age]
+
+      def sample do
+        %__MODULE__{}
+      end
+    end
+  end
+
   test "structs when quoted" do
     assert (quote do
       %User{foo: 1}

lib/elixir/test/elixir/module_test.exs

@@ -245,6 +245,17 @@ defmodule ModuleTest do
     assert ModuleHygiene.test == [1, 2, 3]
   end
 
+  test "ensure function clauses are ordered" do
+    {_, _, binary, _} =
+      defmodule Ordered do
+        def foo(:foo), do: :bar
+        def baz(:baz), do: :bat
+      end
+    atoms = :beam_lib.chunks(binary, [:atoms])
+    assert :erlang.phash2(atoms) == 53987778
+  end
+
+  # TODO: Remove this check once we depend only on 19
   if :erlang.system_info(:otp_release) >= '19' do
     test "create with generated true does not emit warnings" do
       contents =

lib/elixir/test/elixir/protocol_test.exs

@@ -5,18 +5,24 @@ defmodule ProtocolTest do
 
   doctest Protocol
 
-  defprotocol Sample do
-    @type t :: any
-    @doc "Ok"
-    @spec ok(t) :: boolean
-    def ok(term)
-  end
+  {_, _, sample_binary, _} =
+    defprotocol Sample do
+      @type t :: any
+      @doc "Ok"
+      @spec ok(t) :: boolean
+      def ok(term)
+    end
 
-  defprotocol WithAny do
-    @fallback_to_any true
-    @doc "Ok"
-    def ok(term)
-  end
+  @sample_binary sample_binary
+
+  {_, _, with_any_binary, _} =
+    defprotocol WithAny do
+      @fallback_to_any true
+      @doc "Ok"
+      def ok(term)
+    end
+
+  @with_any_binary with_any_binary
 
   defprotocol Derivable do
     def ok(a)
@@ -123,11 +129,11 @@ defmodule ProtocolTest do
   end
 
   test "protocol defines callbacks" do
-    assert get_callbacks(Sample, :ok, 1) ==
-      [{:type, [11], :fun, [{:type, [11], :product, [{:user_type, [11], :t, []}]}, {:type, [11], :boolean, []}]}]
+    assert get_callbacks(@sample_binary, :ok, 1) ==
+      [{:type, 12, :fun, [{:type, 12, :product, [{:user_type, 12, :t, []}]}, {:type, 12, :boolean, []}]}]
 
-    assert get_callbacks(WithAny, :ok, 1) ==
-      [{:type, [18], :fun, [{:type, [18], :product, [{:user_type, [18], :t, []}]}, {:type, [18], :term, []}]}]
+    assert get_callbacks(@with_any_binary, :ok, 1) ==
+      [{:type, 22, :fun, [{:type, 22, :product, [{:user_type, 22, :t, []}]}, {:type, 22, :term, []}]}]
   end
 
   test "protocol defines functions and attributes" do
@@ -182,8 +188,8 @@ defmodule ProtocolTest do
     assert Multi.test(:a) == :a
   end
 
-  defp get_callbacks(module, name, arity) do
-    callbacks = for {:callback, info} <- module.__info__(:attributes), do: hd(info)
+  defp get_callbacks(beam, name, arity) do
+    callbacks = Kernel.Typespec.beam_callbacks(beam)
     List.keyfind(callbacks, {name, arity}, 0) |> elem(1)
   end
 
@@ -308,6 +314,8 @@ defmodule Protocol.ConsolidationTest do
   {:ok, binary} = Protocol.consolidate(Sample, [Any, ImplStruct])
   :code.load_binary(Sample, 'protocol_test.exs', binary)
 
+  @sample_binary binary
+
   # Any should be moved to the end
   :code.purge(WithAny)
   :code.delete(WithAny)
@@ -367,7 +375,7 @@ defmodule Protocol.ConsolidationTest do
   end
 
   test "consolidated keeps callbacks" do
-    callbacks = for {:callback, info} <- Sample.__info__(:attributes), do: hd(info)
+    callbacks = Kernel.Typespec.beam_callbacks(@sample_binary)
     assert callbacks != []
   end
 

lib/elixir/test/elixir/system_test.exs

@@ -187,4 +187,16 @@ defmodule SystemTest do
     time = System.monotonic_time(:nanoseconds)
     assert abs(System.convert_time_unit(time, :nanoseconds, :microseconds)) < abs(time)
   end
+
+  test "schedulers/0" do
+    assert System.schedulers >= 1
+  end
+
+  test "schedulers_online/0" do
+    assert System.schedulers_online >= 1
+  end
+
+  test "otp_release/0" do
+    assert is_binary System.otp_release
+  end
 end

lib/elixir/test/elixir/test_helper.exs

@@ -1,4 +1,10 @@
-ExUnit.start [trace: "--trace" in System.argv]
+exclude =
+  case :erlang.system_info(:otp_release) do
+    '19' -> [otp19: false]
+    _    -> []
+  end
+
+ExUnit.start [exclude: exclude, trace: "--trace" in System.argv]
 
 # Beam files compiled on demand
 path = Path.expand("../../tmp/beams", __DIR__)

lib/elixir/test/erlang/string_test.erl

@@ -15,7 +15,6 @@ extract_interpolations(String) ->
       Z
    end.
 
-
 % Interpolations
 
 extract_interpolations_without_interpolation_test() ->

lib/ex_unit/lib/ex_unit/assertions.ex

@@ -379,7 +379,6 @@ defmodule ExUnit.Assertions do
                           <> ExUnit.Assertions.__mailbox__(self()))
         end
 
-      _ = unquote(vars) # Silence warnings
       received
     end
   end
@@ -428,6 +427,14 @@ defmodule ExUnit.Assertions do
     Enum.uniq_by(pins, &elem(&1, 0))
   end
 
+  defp collect_vars_from_pattern({:when, _, [left, right]}) do
+    pattern = collect_vars_from_pattern(left)
+    for {name, _, context} = var <- collect_vars_from_pattern(right),
+      Enum.any?(pattern, &match?({^name, _, ^context}, &1)),
+      into: pattern,
+      do: var
+  end
+
   defp collect_vars_from_pattern(expr) do
     Macro.prewalk(expr, [], fn
       {:::, _, [left, _]}, acc ->

lib/ex_unit/test/ex_unit/assertions_test.exs

@@ -266,6 +266,13 @@ defmodule ExUnit.AssertionsTest do
     :world = world
   end
 
+  test "assert received does not leak external variables used in guards" do
+    send self(), {:hello, :world}
+    guard_world = :world
+    assert_received {:hello, world} when world == guard_world
+    :world = world
+  end
+
   test "refute received does not wait" do
     false = refute_received :hello
   end

lib/iex/lib/iex/helpers.ex

@@ -23,8 +23,8 @@ defmodule IEx.Helpers do
   There are many other helpers available:
 
     * `b/1`           - prints callbacks info and docs for a given module
-    * `c/1`           - compiles a file at the current directory
-    * `c/2`           - compiles a file at the given path
+    * `c/1`           - compiles a file into the current directory
+    * `c/2`           - compiles a file to the given path
     * `cd/1`          - changes the current directory
     * `clear/0`       - clears the screen
     * `flush/0`       - flushes all messages sent to the shell
@@ -115,6 +115,7 @@ defmodule IEx.Helpers do
 
       c "baz.ex"
       #=> [Baz]
+
   """
   def c(files, path \\ ".") when is_binary(path) do
     files = List.wrap(files)
@@ -123,10 +124,7 @@ defmodule IEx.Helpers do
       raise ArgumentError, "expected a binary or a list of binaries as argument"
     end
 
-    {found, not_found} =
-      files
-      |> Enum.map(&Path.expand(&1, path))
-      |> Enum.partition(&File.exists?/1)
+    {found, not_found} = Enum.partition(files, &File.exists?/1)
 
     unless Enum.empty?(not_found) do
       raise ArgumentError, "could not find files #{Enum.join(not_found, ", ")}"
@@ -535,6 +533,15 @@ defmodule IEx.Helpers do
     end
   end
 
+  @doc """
+  Similar to `import_file` but only imports it if it is available.
+  """
+  defmacro import_file_if_available(path) do
+    quote do
+      import_file(unquote(path), optional: true)
+    end
+  end
+
   @doc """
   Evaluates the contents of the file at `path` as if it were directly typed into
   the shell.
@@ -587,6 +594,25 @@ defmodule IEx.Helpers do
     raise ArgumentError, "import_file/1 expects a literal binary as its argument"
   end
 
+  @doc """
+  Calls `import/2` with the given arguments, but only if the module is available.
+
+  This lets you put imports in `.iex.exs` files (including `~/.iex.exs`) without
+  getting compile errors if you open a console where the module is not available.
+
+  ## Example
+
+      # In ~/.iex.exs
+      import_if_available Ecto.Query
+  """
+  defmacro import_if_available(quoted_module, opts \\ []) do
+    module = Macro.expand(quoted_module, __CALLER__)
+
+    if Code.ensure_loaded?(module) do
+      quote do: import unquote(quoted_module), unquote(opts)
+    end
+  end
+
   # Compiles and loads an Erlang source file, returns {module, binary}
   defp compile_erlang(source) do
     source = Path.relative_to_cwd(source) |> String.to_charlist

lib/iex/lib/iex/introspection.ex

@@ -108,7 +108,7 @@ defmodule IEx.Introspection do
     if docs = Code.get_docs(mod, :docs) do
       if doc = find_doc(docs, fun, arity) do
         if callback_module = is_nil(elem(doc, 4)) and callback_module(mod, fun, arity) do
-          filter = &match?({^fun, _}, elem(&1, 0))
+          filter = &match?({^fun, ^arity}, elem(&1, 0))
           print_callback_docs(callback_module, filter, &print_doc/2)
         else
           print_doc(doc)
@@ -139,6 +139,8 @@ defmodule IEx.Introspection do
     end)
   end
 
+  defp has_content?(nil),
+    do: false
   defp has_content?({_, _, _, _, false}),
     do: false
   defp has_content?({{name, _}, _, _, _, nil}),
@@ -147,14 +149,11 @@ defmodule IEx.Introspection do
     do: true
 
   defp callback_module(mod, fun, arity) do
+    filter = &match?({{^fun, ^arity}, _}, &1)
     mod.module_info(:attributes)
     |> Keyword.get_values(:behaviour)
     |> Stream.concat()
-    |> Enum.find(fn module ->
-      module.module_info(:attributes)
-      |> Enum.filter(&match?({:callback, _}, &1))
-      |> Enum.any?(&match?({_, [{{^fun, ^arity}, _} | _]}, &1))
-    end)
+    |> Enum.find(&Enum.any?(Typespec.beam_callbacks(&1), filter))
   end
 
   defp print_doc({{fun, _}, _line, kind, args, doc}) do

lib/iex/test/iex/helpers_test.exs

@@ -34,9 +34,11 @@ defmodule IEx.HelpersTest do
   test "h helper function" do
     pwd_h = "* def pwd()\n\nPrints the current working directory.\n\n"
     c_h   = "* def c(files, path \\\\ \".\")\n\nCompiles the given files."
+    eq_h  = "* def ==(left, right)\n\nReturns `true` if the two items are equal.\n\n"
 
     assert capture_io(fn -> h IEx.Helpers.pwd/0 end) =~ pwd_h
     assert capture_io(fn -> h IEx.Helpers.c/2 end) =~ c_h
+    assert capture_io(fn -> h ==/2 end) =~ eq_h
 
     assert capture_io(fn -> h IEx.Helpers.c/1 end) =~ c_h
     assert capture_io(fn -> h pwd end) =~ pwd_h
@@ -77,14 +79,17 @@ defmodule IEx.HelpersTest do
       @doc "Docs for MyBehaviour.first"
       @callback first(integer) :: integer
       @callback second(integer) :: integer
+      @callback second(integer, integer) :: integer
     end
     """
     impl = """
     defmodule Impl do
       @behaviour MyBehaviour
       def first(0), do: 0
-      @doc "Docs for Impl.second"
+      @doc "Docs for Impl.second/1"
       def second(0), do: 0
+      @doc "Docs for Impl.second/2"
+      def second(0, 0), do: 0
     end
     """
     files = ["my_behaviour.ex", "impl.ex"]
@@ -92,10 +97,11 @@ defmodule IEx.HelpersTest do
       assert c(files) |> Enum.sort == [Impl, MyBehaviour]
 
       assert capture_io(fn -> h Impl.first/1 end) == "* @callback first(integer()) :: integer()\n\nDocs for MyBehaviour.first\n"
-      assert capture_io(fn -> h Impl.second/1 end) == "* def second(int)\n\nDocs for Impl.second\n"
+      assert capture_io(fn -> h Impl.second/1 end) == "* def second(int)\n\nDocs for Impl.second/1\n"
+      assert capture_io(fn -> h Impl.second/2 end) == "* def second(int1, int2)\n\nDocs for Impl.second/2\n"
 
       assert capture_io(fn -> h Impl.first end) == "* @callback first(integer()) :: integer()\n\nDocs for MyBehaviour.first\n"
-      assert capture_io(fn -> h Impl.second end) == "* def second(int)\n\nDocs for Impl.second\n"
+      assert capture_io(fn -> h Impl.second end) == "* def second(int)\n\nDocs for Impl.second/1\n* def second(int1, int2)\n\nDocs for Impl.second/2\n"
     end
   after
     cleanup_modules([Impl, MyBehaviour])
@@ -255,11 +261,17 @@ defmodule IEx.HelpersTest do
   end
 
   test "import_file when the file is missing" do
-    assert "nil" == capture_iex("import_file \"nonexistent\", optional: true")
-
     failing = capture_iex("import_file \"nonexistent\"")
     assert "** (File.Error) could not read file" <> _ = failing
     assert failing =~ "no such file or directory"
+
+    assert "nil" == capture_iex("import_file_if_available \"nonexistent\"")
+  end
+
+  test "import_if_available helper" do
+    assert "nil" == capture_iex("import_if_available NoSuchModule")
+    assert "[1, 2, 3]" == capture_iex("import_if_available Integer; digits 123")
+    assert "[1, 2, 3]" == capture_iex("import_if_available Integer, only: [digits: 1]; digits 123")
   end
 
   test "c helper" do

lib/iex/test/test_helper.exs

@@ -70,4 +70,3 @@ defmodule IEx.Case do
     |> String.trim
   end
 end
-

lib/mix/lib/mix/compilers/elixir.ex

@@ -177,8 +177,6 @@ defmodule Mix.Compilers.Elixir do
   end
 
   defp each_module(pid, cwd, source, module, binary) do
-    beam = Atom.to_string(module) <> ".beam"
-
     {compile_references, runtime_references} = Kernel.LexicalTracker.remote_references(module)
 
     compile_references =
@@ -214,7 +212,7 @@ defmodule Mix.Compilers.Elixir do
         module: module,
         kind: kind,
         source: source,
-        beam: beam,
+        beam: nil, # They are calculated when writing the manifest
         binary: binary
       )
 
@@ -265,8 +263,10 @@ defmodule Mix.Compilers.Elixir do
   ## Resolution
 
   defp update_stale_sources(sources, removed, changed) do
+    # Remove delete sources
     sources =
-      Enum.reject(sources, fn source(source: source) -> source in removed end)
+      Enum.reduce(removed, sources, &List.keydelete(&2, &1, source(:source)))
+    # Store empty sources for the changed ones as the compiler appends data
     sources =
       Enum.reduce(changed, sources, &List.keystore(&2, &1, source(:source), source(source: &1)))
     sources
@@ -394,13 +394,14 @@ defmodule Mix.Compilers.Elixir do
     File.mkdir_p!(Path.dirname(manifest))
 
     modules =
-      for module(beam: beam, binary: binary) = module <- modules do
+      for module(binary: binary, module: module) = entry <- modules do
+        beam = Atom.to_string(module) <> ".beam"
         if binary do
           beam_path = Path.join(compile_path, beam)
           File.write!(beam_path, binary)
           File.touch!(beam_path, timestamp)
         end
-        module(module, binary: nil)
+        module(entry, binary: nil, beam: beam)
       end
 
     manifest_data =

lib/mix/lib/mix/tasks/app.tree.ex

@@ -56,11 +56,18 @@ defmodule Mix.Tasks.App.Tree do
 
     if opts[:format] == "dot" do
       Mix.Utils.write_dot_graph!("app_tree.dot", "application tree",
-                                 {:normal, app}, callback, opts)
-      Mix.shell.info "Generated \"app_tree.dot\" in current directory.\n" <>
-                     "You can use http://www.graphviz.org/ to open it."
+                                 [{:normal, app}], callback, opts)
+      """
+      Generated "app_tree.dot" in the current directory. To generate a PNG:
+
+         dot -Tpng app_tree.dot -o app_tree.png
+
+      For more options see http://www.graphviz.org/.
+      """
+      |> String.trim_trailing
+      |> Mix.shell.info
     else
-      Mix.Utils.print_tree({:normal, app}, callback, opts)
+      Mix.Utils.print_tree([{:normal, app}], callback, opts)
     end
   end
 

lib/mix/lib/mix/tasks/clean.ex

@@ -47,7 +47,8 @@ defmodule Mix.Tasks.Clean do
 
   # Loadpaths without checks because compilers may be defined in deps.
   defp loadpaths! do
-    Mix.Task.run "loadpaths", ["--no-elixir-version-check", "--no-deps-check"]
+    Mix.Task.run "loadpaths", ["--no-elixir-version-check", "--no-deps-check", "--no-archives-check"]
     Mix.Task.reenable "loadpaths"
+    Mix.Task.reenable "deps.check"
   end
 end

lib/mix/lib/mix/tasks/compile.app.ex

@@ -90,14 +90,14 @@ defmodule Mix.Tasks.Compile.App do
         best_guess
       end
 
+      properties = ensure_correct_properties(app, config, properties)
+
       # Ensure we always prepend the standard application dependencies
-      core_apps = [:kernel, :stdlib] ++ language_app(config)
       properties = Keyword.update!(properties, :applications, fn apps ->
-        core_apps ++ apps
+        [:kernel, :stdlib] ++ language_app(config) ++ apps
       end)
 
-      properties = ensure_correct_properties(app, config, properties)
-      contents   = {:application, app, properties}
+      contents = {:application, app, properties}
 
       Mix.Project.ensure_structure()
       File.write!(target, :io_lib.format("~p.", [contents]), [:utf8])

lib/mix/lib/mix/tasks/compile.ex

@@ -11,7 +11,7 @@ defmodule Mix.Tasks.Compile do
   ## Configuration
 
     * `:compilers` - compilers to run, defaults to:
-      `[:yeec, :leex, :erlang, :elixir, :xref, :app]`
+      `[:yecc, :leex, :erlang, :elixir, :xref, :app]`
 
     * `:consolidate_protocols` - when `true`, runs protocol
       consolidation via the `compile.protocols` task
@@ -93,8 +93,9 @@ defmodule Mix.Tasks.Compile do
 
   # Loadpaths without checks because compilers may be defined in deps.
   defp loadpaths! do
-    Mix.Task.run "loadpaths", ["--no-elixir-version-check", "--no-deps-check"]
+    Mix.Task.run "loadpaths", ["--no-elixir-version-check", "--no-deps-check", "--no-archives-check"]
     Mix.Task.reenable "loadpaths"
+    Mix.Task.reenable "deps.check"
   end
 
   defp consolidate_protocols? do

lib/mix/lib/mix/tasks/compile.xref.ex

@@ -36,7 +36,7 @@ defmodule Mix.Tasks.Compile.Xref do
   end
 
   defp run_xref do
-    Mix.Task.run("xref", ["--warnings"])
+    Mix.Task.run("xref", ["warnings"])
   end
 
   defp needs_xref?(opts) do

lib/mix/lib/mix/tasks/deps.check.ex

@@ -5,28 +5,20 @@ defmodule Mix.Tasks.Deps.Check do
                          format_status: 1, check_lock: 1]
 
   @moduledoc """
-  Checks if all dependencies are valid, loading them along
-  the way.
+  Checks if all dependencies are valid,
+  loading them along the way.
 
   If there is an invalid dependency, its status is printed
   before aborting.
 
-  This task is not shown in `mix help` but it is part
-  of the `mix` public API and can be depended on.
-
   ## Command line options
 
-    * `--no-archives-check` - do not check archives
-    * `--no-deps-check` - do not check deps
+    * `--no-deps-check` - do not check or compile deps, only load available ones
     * `--no-compile` - do not compile dependencies
 
   """
   @spec run(OptionParser.argv) :: :ok
   def run(args) do
-    unless "--no-archives-check" in args do
-      Mix.Task.run "archive.check", args
-    end
-
     all = Enum.map(Mix.Dep.cached(), &check_lock/1)
 
     unless "--no-deps-check" in args do
@@ -95,11 +87,16 @@ defmodule Mix.Tasks.Deps.Check do
 
   defp partition([dep | deps], not_ok, compile) do
     cond do
-      from_umbrella?(dep)      -> partition(deps, not_ok, compile)
-      compilable?(dep)         -> partition(deps, not_ok, [dep | compile])
-      ok?(dep) and local?(dep) -> partition(deps, not_ok, [dep | compile])
-      ok?(dep)                 -> partition(deps, not_ok, compile)
-      true                     -> partition(deps, [dep | not_ok], compile)
+      compilable?(dep) ->
+        if from_umbrella?(dep) do
+          partition(deps, not_ok, compile)
+        else
+          partition(deps, not_ok, [dep | compile])
+        end
+      ok?(dep) ->
+        partition(deps, not_ok, compile)
+      true ->
+        partition(deps, [dep | not_ok], compile)
     end
   end
 
@@ -123,7 +120,7 @@ defmodule Mix.Tasks.Deps.Check do
   defp compilable?(%Mix.Dep{status: {:noappfile, _}}), do: true
   defp compilable?(%Mix.Dep{status: {:scmlock, _}}), do: true
   defp compilable?(%Mix.Dep{status: :compile}), do: true
-  defp compilable?(%Mix.Dep{}), do: false
+  defp compilable?(%Mix.Dep{} = dep), do: ok?(dep) and local?(dep)
 
   defp show_not_ok!([]) do
     :ok

lib/mix/lib/mix/tasks/deps.compile.ex

@@ -33,7 +33,7 @@ defmodule Mix.Tasks.Deps.Compile do
   import Mix.Dep, only: [loaded: 1, available?: 1, loaded_by_name: 2,
                          make?: 1, mix?: 1]
 
-  @switches [include_children: :boolean]
+  @switches [include_children: :boolean, force: :boolean]
 
   @spec run(OptionParser.argv) :: :ok
   def run(args) do
@@ -44,18 +44,18 @@ defmodule Mix.Tasks.Deps.Compile do
     Mix.Project.get!
 
     case OptionParser.parse(args, switches: @switches) do
-      {_, [], _} ->
+      {opts, [], _} ->
         # Because this command may be invoked explicitly with
         # deps.compile, we simply try to compile any available
         # dependency.
-        compile(Enum.filter(loaded(env: Mix.env), &available?/1))
+        compile(Enum.filter(loaded(env: Mix.env), &available?/1), opts)
       {opts, tail, _} ->
-        compile(loaded_by_name(tail, [env: Mix.env] ++ opts))
+        compile(loaded_by_name(tail, [env: Mix.env] ++ opts), opts)
     end
   end
 
   @doc false
-  def compile(deps) do
+  def compile(deps, options \\ []) do
     shell  = Mix.shell
     config = Mix.Project.deps_config
 
@@ -65,6 +65,8 @@ defmodule Mix.Tasks.Deps.Compile do
       Enum.map(deps, fn %Mix.Dep{app: app, status: status, opts: opts, scm: scm} = dep ->
         check_unavailable!(app, status)
 
+        clean(app, options)
+
         compiled? = cond do
           not is_nil(opts[:compile]) ->
             do_compile dep, config
@@ -93,6 +95,12 @@ defmodule Mix.Tasks.Deps.Compile do
     if true in compiled, do: Mix.Dep.Lock.touch_manifest, else: :ok
   end
 
+  defp clean(app, opts) do
+    if Keyword.get(opts, :force, false) do
+      File.rm_rf! Path.join [Mix.Project.build_path, "lib", Atom.to_string(app)]
+    end
+  end
+
   defp touch_fetchable(scm, path) do
     if scm.fetchable? do
       File.mkdir_p!(path)
@@ -125,7 +133,7 @@ defmodule Mix.Tasks.Deps.Compile do
       end
 
       try do
-        res = Mix.Task.run("compile", ["--no-deps", "--no-elixir-version-check"])
+        res = Mix.Task.run("compile", ["--no-deps", "--no-archives-check", "--no-elixir-version-check"])
         :ok in List.wrap(res)
       catch
         kind, reason ->
@@ -176,12 +184,13 @@ defmodule Mix.Tasks.Deps.Compile do
       Mix.raise "\"#{manager}\" installation failed"
   end
 
-  defp do_make(dep, config) do
-    command = if match?({:win32, _}, :os.type) and File.regular?("Makefile.win") do
-      "nmake /F Makefile.win"
-    else
-      "make"
-    end
+  defp do_make(%{opts: opts} = dep, config) do
+    command =
+      if match?({:win32, _}, :os.type) and File.regular?(Path.join(opts[:dest], "Makefile.win")) do
+        "nmake /F Makefile.win"
+      else
+        "make"
+      end
     do_command(dep, config, command, true)
   end
 

lib/mix/lib/mix/tasks/deps.tree.ex

@@ -51,12 +51,19 @@ defmodule Mix.Tasks.Deps.Tree do
 
     if opts[:format] == "dot" do
       callback = callback(&format_dot/1, deps, opts)
-      Mix.Utils.write_dot_graph!("deps_tree.dot", "dependency tree", root, callback, opts)
-      Mix.shell.info "Generated \"deps_tree.dot\" in current directory.\n" <>
-                     "You can use http://www.graphviz.org/ to open it."
+      Mix.Utils.write_dot_graph!("deps_tree.dot", "dependency tree", [root], callback, opts)
+      """
+      Generated "deps_tree.dot" in the current directory. To generate a PNG:
+
+          dot -Tpng deps_tree.dot -o deps_tree.png
+
+      For more options see http://www.graphviz.org/.
+      """
+      |> String.trim_trailing
+      |> Mix.shell.info
     else
       callback = callback(&format_tree/1, deps, opts)
-      Mix.Utils.print_tree(root, callback, opts)
+      Mix.Utils.print_tree([root], callback, opts)
     end
   end
 

lib/mix/lib/mix/tasks/do.ex

@@ -6,6 +6,8 @@ defmodule Mix.Tasks.Do do
   @moduledoc """
   Executes the tasks separated by comma.
 
+  The comma should be followed by a space.
+
   ## Examples
 
   The example below prints the available compilers and
@@ -24,27 +26,21 @@ defmodule Mix.Tasks.Do do
 
   @doc false
   def gather_commands(args) do
-    gather_commands(args, [], [])
+    gather_commands args, [], []
   end
 
-  defp gather_commands([], current, commands) do
-    [current | commands]
-    |> Enum.reject(&(&1 == []))
-    |> Enum.map(&Enum.reverse(&1))
-    |> Enum.reverse
+  def gather_commands([head | rest], current, acc)
+      when binary_part(head, byte_size(head), -1) == "," do
+    part    = binary_part(head, 0, byte_size(head) - 1)
+    current = Enum.reverse([part | current])
+    gather_commands rest, [], [current | acc]
   end
 
-  defp gather_commands([arg | rest], current, commands) do
-    case String.split(arg, ",", parts: 2) do
-      [arg] ->
-        gather_commands(rest, [arg | current], commands)
-      [left, right] ->
-        rest    = append_unless_empty(right, rest)
-        current = append_unless_empty(left, current)
-        gather_commands(rest, [], [current | commands])
-    end
+  def gather_commands([head | rest], current, acc) do
+    gather_commands rest, [head | current], acc
   end
 
-  defp append_unless_empty("", list), do: list
-  defp append_unless_empty(h, list),  do: [h | list]
+  def gather_commands([], current, acc) do
+    Enum.reverse [Enum.reverse(current) | acc]
+  end
 end

lib/mix/lib/mix/tasks/help.ex

@@ -105,8 +105,9 @@ defmodule Mix.Tasks.Help do
 
   # Loadpaths without checks because tasks may be defined in deps.
   defp loadpaths! do
-    Mix.Task.run "loadpaths", ["--no-elixir-version-check", "--no-deps-check"]
+    Mix.Task.run "loadpaths", ["--no-elixir-version-check", "--no-deps-check", "--no-archives-check"]
     Mix.Task.reenable "loadpaths"
+    Mix.Task.reenable "deps.check"
   end
 
   defp load_tasks() do

lib/mix/lib/mix/tasks/loadpaths.ex

@@ -12,12 +12,13 @@ defmodule Mix.Tasks.Loadpaths do
   ## Command line options
 
     * `--no-archives-check` - do not check archive
-    * `--no-deps-check` - do not check dependencies (also implies --no-archives-check)
+    * `--no-deps-check` - do not check dependencies
     * `--no-elixir-version-check` - do not check Elixir version
 
   """
 
   @spec run(OptionParser.argv) :: :ok
+
   def run(args) do
     config = Mix.Project.config
 
@@ -25,7 +26,11 @@ defmodule Mix.Tasks.Loadpaths do
       check_elixir_version(config, args)
     end
 
-    # --no-deps is used only internally. It has not purpose
+    unless "--no-archives-check" in args do
+      Mix.Task.run "archive.check", args
+    end
+
+    # --no-deps is used only internally. It has no purpose
     # from Mix.CLI because the CLI itself already loads deps.
     unless "--no-deps" in args do
       Mix.Task.run "deps.check", args

lib/mix/lib/mix/tasks/new.ex

@@ -251,7 +251,7 @@ defmodule Mix.Tasks.New do
        elixir: "~> <%= @version %>",
        build_embedded: Mix.env == :prod,
        start_permanent: Mix.env == :prod,
-       deps: deps]
+       deps: deps()]
     end
 
     # Configuration for the OTP application

lib/mix/lib/mix/tasks/test.ex

@@ -3,7 +3,7 @@ defmodule Mix.Tasks.Test do
     @moduledoc false
 
     def start(compile_path, opts) do
-      Mix.shell.info "Cover compiling modules ... "
+      Mix.shell.info "Cover compiling modules ..."
       _ = :cover.start
 
       case :cover.compile_beam_directory(compile_path |> to_charlist) do
@@ -16,7 +16,7 @@ defmodule Mix.Tasks.Test do
       output = opts[:output]
 
       fn() ->
-        Mix.shell.info "\nGenerating cover results ... "
+        Mix.shell.info "\nGenerating cover results ..."
         File.mkdir_p!(output)
         Enum.each :cover.modules, fn(mod) ->
           {:ok, _} = :cover.analyse_to_file(mod, '#{output}/#{mod}.html', [:html])

lib/mix/lib/mix/tasks/xref.ex

@@ -2,7 +2,7 @@ defmodule Mix.Tasks.Xref do
   use Mix.Task
 
   alias Mix.Tasks.Compile.Elixir, as: E
-  import Mix.Compilers.Elixir, only: [read_manifest: 2, source: 1, source: 2]
+  import Mix.Compilers.Elixir, only: [read_manifest: 2, source: 1, source: 2, module: 1]
 
   @shortdoc "Performs cross reference checks"
   @recursive true
@@ -12,23 +12,82 @@ defmodule Mix.Tasks.Xref do
 
   ## Xref modes
 
-  The following options control the information xref can emit.
+  The `xref` task expects a mode as first argument:
+  
+      mix xref MODE
+  
+  All available modes are discussed below.
+  
+  ### warnings
 
-    * `--warnings` - prints warnings for violated cross reference checks
-    * `--unreachable` - prints all unreachable "file:line: module.function/arity" entries
-    * `--callers` - prints all references of given `Module`, `Module.function`, or
-      `Module.function/arity`
+  Prints warnings for violated cross reference checks:
+  
+      mix xref warnings
+  
+  This is the mode used by Mix during compilation.
 
-  ## Command line options
+  ### unreachable
+
+  Prints all unreachable "file:line: module.function/arity" entries:
+  
+      mix xref unreachable
+  
+  The "file:line" represents the file and line a call to an unknown
+  "module.function/arity" is made.
+  
+  ### callers CALLEE
+
+  Prints all callers of the given `CALLEE`, which can be one of: `Module`,
+  `Module.function`, or `Module.function/arity`. Examples:
+
+      mix xref callers MyMod
+      mix xref callers MyMod.fun
+      mix xref callers MyMod.fun/3
+
+  ### graph
+
+  Prints a file dependency graph where an edge from `A` to `B` indicates
+  that `A` depends on `B`.
+ 
+      mix xref graph --format dot
+
+  The following options are accepted:
+
+    * `--exclude` - paths to exclude
+
+    * `--source` - display all files that the given source file references (directly or indirectly)
+
+    * `--sink` - display all files that reference the given file (directly or indirectly)
+
+    * `--format` - can be set to one of:
+
+      * `pretty` - use Unicode codepoints for formatting the graph.
+        This is the default except on Windows
+
+      * `plain` - do not use Unicode codepoints for formatting the graph.
+        This is the default on Windows
+
+      * `dot` - produces a DOT graph description in `xref_graph.dot` in the
+        current directory. Warning: this will override any previously generated file
+
+  The `--source` and `--sink` options are particularly useful when trying to understand how
+  the modules in a particular file interact with the whole system.
+
+  ## Shared options
+
+  Those options are shared across all modes:
 
     * `--no-compile` - do not compile even if files require compilation
+
     * `--no-deps-check` - do not check dependencies
+
     * `--no-archives-check` - do not check archives
+
     * `--no-elixir-version-check` - do not check the Elixir version from mix.exs
 
   ## Configuration
 
-    All configuration for Xref should be placed under the key `:xref`.
+  All configuration for Xref should be placed under the key `:xref`.
 
     * `:exclude` - a list of modules and `{module, function, arity}` tuples to ignore when checking
       cross references. For example: `[MissingModule, {MissingModule2, :missing_func, 2}]`
@@ -36,31 +95,32 @@ defmodule Mix.Tasks.Xref do
   """
 
   @switches [compile: :boolean, deps_check: :boolean, archives_check: :boolean,
-             warnings: :boolean, unreachable: :boolean, elixir_version_check: :boolean,
-             callers: :string]
+             elixir_version_check: :boolean, exclude: :keep, format: :string,
+             source: :string, sink: :string]
 
   @doc """
   Runs this task.
   """
   @spec run(OptionParser.argv) :: :ok | :error
   def run(args) do
-    {opts, _} =
+    {opts, args} =
       OptionParser.parse!(args, strict: @switches)
 
     if Keyword.get(opts, :compile, true) do
       Mix.Task.run("compile")
     end
 
-    modes = [:warnings, :unreachable, :callers]
-    case Keyword.take(opts, modes) do
-      [warnings: true] ->
+    case args do
+      ["warnings"] ->
         warnings()
-      [unreachable: true] ->
+      ["unreachable"] ->
         unreachable()
-      [callers: callee] ->
+      ["callers", callee] ->
         callers(callee)
+      ["graph"] ->
+        graph(opts)
       _ ->
-        Mix.raise "xref expects exactly one of the following modes: --warnings, --unreachable, --callers"
+        Mix.raise "xref doesn't support this command, see mix help xref for more information"
     end
   end
 
@@ -90,11 +150,16 @@ defmodule Mix.Tasks.Xref do
     :ok
   end
 
+  defp graph(opts) do
+    write_graph(file_references(), excluded(opts), opts)
+
+    :ok
+  end
+
   ## Unreachable
 
   defp unreachable(pair_fun) do
     excludes = excludes()
-
     each_source_entries(&source_warnings(&1, excludes), pair_fun)
   end
 
@@ -102,20 +167,38 @@ defmodule Mix.Tasks.Xref do
     source(runtime_dispatches: runtime_dispatches) = source
 
     for {module, func_arity_lines} <- runtime_dispatches,
+        exports = load_exports(module),
         {{func, arity}, lines} <- func_arity_lines,
-        warning = unreachable_mfa(module, func, arity, lines, excludes),
+        warning = unreachable_mfa(exports, module, func, arity, lines, excludes),
         do: warning
   end
 
-  defp unreachable_mfa(module, func, arity, lines, excludes) do
+  defp load_exports(module) do
+    if :code.is_loaded(module) do
+      # If the module is loaded, we will use the faster function_exported?/3 check
+      module
+    else
+      # Otherwise we get all exports from :beam_lib to avoid loading modules
+      with file when is_list(file) <- :code.which(module),
+           {:ok, {^module, [exports: exports]}} <- :beam_lib.chunks(file, [:exports]) do
+        exports
+      else
+        _ -> :unknown_module
+      end
+    end
+  end
+
+  defp unreachable_mfa(exports, module, func, arity, lines, excludes) do
     cond do
       excluded?(module, func, arity, excludes) ->
         nil
       skip?(module, func, arity) ->
         nil
-      not Code.ensure_loaded?(module) ->
+      exports == :unknown_module ->
         {Enum.sort(lines), :unknown_module, module, func, arity}
-      not function_exported?(module, func, arity) ->
+      is_atom(exports) and not function_exported?(module, func, arity) ->
+        {Enum.sort(lines), :unknown_function, module, func, arity}
+      is_list(exports) and not {func, arity} in exports ->
         {Enum.sort(lines), :unknown_function, module, func, arity}
       true ->
         nil
@@ -210,7 +293,7 @@ defmodule Mix.Tasks.Xref do
   defp source_calls_for_filter(source, filter) do
     runtime_dispatches = source(source, :runtime_dispatches)
     compile_dispatches = source(source, :compile_dispatches)
-    dispatches = Stream.concat(runtime_dispatches, compile_dispatches)
+    dispatches = runtime_dispatches ++ compile_dispatches
 
     calls =
       for {module, func_arity_lines} <- dispatches,
@@ -279,12 +362,130 @@ defmodule Mix.Tasks.Xref do
 
   defp raise_invalid_callee(callee) do
     message =
-      "xref --callers expects Module, Module.function, or Module.function/arity, got: " <>
+      "xref callers CALLEE expects Module, Module.function, or Module.function/arity, got: " <>
       callee
 
     Mix.raise message
   end
 
+  ## Graph helpers
+
+  defp excluded(opts) do
+    Keyword.get_values(opts, :exclude)
+    |> Enum.flat_map(&[{&1, nil}, {&1, "(compile)"}, {&1, "(runtime)"}])
+  end
+
+  defp file_references() do
+    module_sources =
+      for manifest <- E.manifests(),
+          manifest_data = read_manifest(manifest, ""),
+          module(module: module, source: source) <- manifest_data,
+          source = Enum.find(manifest_data, &match?(source(source: ^source), &1)),
+          do: {module, source},
+          into: %{}
+
+    all_modules = MapSet.new(module_sources, &elem(&1, 0))
+
+    Map.new module_sources, fn {module, source} ->
+      source(runtime_references: runtime, compile_references: compile, source: file) = source
+      compile_references =
+        compile
+        |> MapSet.new()
+        |> MapSet.delete(module)
+        |> MapSet.intersection(all_modules)
+        |> Enum.filter(&module_sources[&1] != source)
+        |> Enum.map(&{source(module_sources[&1], :source), "(compile)"})
+
+      runtime_references =
+        runtime
+        |> MapSet.new()
+        |> MapSet.delete(module)
+        |> MapSet.intersection(all_modules)
+        |> Enum.filter(&module_sources[&1] != source)
+        |> Enum.map(&{source(module_sources[&1], :source), nil})
+
+      {file, compile_references ++ runtime_references}
+    end
+  end
+
+  defp write_graph(file_references, excluded, opts) do
+    {root, file_references} =
+      case {opts[:source], opts[:sink]} do
+        {nil, nil} ->
+          {Enum.map(file_references, &{elem(&1, 0), nil}) -- excluded, file_references}
+
+        {source, nil} ->
+          if file_references[source] do
+            {[{source, nil}], file_references}
+          else
+            Mix.raise "Source could not be found: #{source}"
+          end
+
+        {nil, sink} ->
+          if file_references[sink] do
+            file_references = filter_for_sink(file_references, sink)
+            roots =
+              file_references
+              |> Map.delete(sink)
+              |> Enum.map(&{elem(&1, 0), nil})
+            {roots -- excluded, file_references}
+          else
+            Mix.raise "Sink could not be found: #{sink}"
+          end
+
+        {_, _} ->
+          Mix.raise "mix xref graph expects only one of --source and --sink"
+      end
+
+    callback =
+      fn {file, type} ->
+        children = Map.get(file_references, file, [])
+        {{file, type}, children -- excluded}
+      end
+
+    if opts[:format] == "dot" do
+      Mix.Utils.write_dot_graph!("xref_graph.dot", "xref graph",
+                                 root, callback, opts)
+      """
+      Generated "xref_graph.dot" in the current directory. To generate a PNG:
+
+         dot -Tpng xref_graph.dot -o xref_graph.png
+
+      For more options see http://www.graphviz.org/.
+      """
+      |> String.trim_trailing()
+      |> Mix.shell.info()
+    else
+      Mix.Utils.print_tree(root, callback, opts)
+    end
+  end
+
+  defp filter_for_sink(file_references, sink) do
+    file_references
+    |> invert_references()
+    |> do_filter_for_sink([{sink, nil}], %{})
+    |> invert_references()
+  end
+
+  defp do_filter_for_sink(file_references, new_nodes, acc) do
+    Enum.reduce new_nodes, acc, fn {new_node_name, _type}, acc ->
+      new_nodes = file_references[new_node_name]
+      if acc[new_node_name] || !new_nodes do
+        acc
+      else
+        do_filter_for_sink(file_references, new_nodes, Map.put(acc, new_node_name, new_nodes))
+      end
+    end
+  end
+
+  defp invert_references(file_references) do
+    Enum.reduce file_references, %{}, fn {file, references}, acc ->
+      Enum.reduce references, acc, fn {reference, type}, acc ->
+        Map.update(acc, reference, [{file, type}], &[{file, type} | &1])
+      end
+    end
+  end
+
   ## Helpers
 
   defp each_source_entries(entries_fun, pair_fun) do

lib/mix/lib/mix/utils.ex

@@ -142,31 +142,40 @@ defmodule Mix.Utils do
     end) |> Enum.uniq
   end
 
+  @type tree_node :: {name :: String.Chars.t, edge_info :: String.Chars.t}
+
   @doc """
   Prints the given tree according to the callback.
 
   The callback will be invoked for each node and it
-  must either return `{printed, children}` tuple or
-  `false` if the given node must not be printed.
+  must return a `{printed, children}` tuple.
   """
-  @spec print_tree(term, (term -> {String.t, [term]}), Keyword.t) :: :ok
-  def print_tree(root, callback, opts \\ []) do
+  @spec print_tree([tree_node], (tree_node -> {tree_node, [tree_node]}), Keyword.t) :: :ok
+  def print_tree(nodes, callback, opts \\ []) do
     pretty =
       case Keyword.get(opts, :format) do
         "pretty" -> true
         "plain" -> false
         _ -> elem(:os.type, 0) != :win32
       end
-    print_tree([root], [], pretty, callback)
+    print_tree(nodes, [], nil, MapSet.new(), pretty, callback)
+
+    :ok
   end
 
-  defp print_tree([], _depth, _pretty, _callback), do: :ok
-  defp print_tree([node | nodes], depth, pretty, callback) do
-    {{name, info}, children} =  callback.(node)
-    space = if info, do: " ", else: ""
-    Mix.shell.info("#{depth(pretty, depth)}#{prefix(pretty, depth, nodes)}#{name}#{space}#{info}")
-    print_tree(children, [(nodes != []) | depth], pretty, callback)
-    print_tree(nodes, depth, pretty, callback)
+  defp print_tree([], _depth, _parent, seen, _pretty, _callback), do: seen
+  defp print_tree([node | nodes], depth, parent, seen, pretty, callback) do
+    {{name, info}, children} = callback.(node)
+    key = {parent, name}
+
+    if MapSet.member?(seen, key) do
+      seen
+    else
+      space = if info, do: " ", else: ""
+      Mix.shell.info("#{depth(pretty, depth)}#{prefix(pretty, depth, nodes)}#{name}#{space}#{info}")
+      seen = print_tree(children, [(nodes != []) | depth], name, MapSet.put(seen, key), pretty, callback)
+      print_tree(nodes, depth, parent, seen, pretty, callback)
+    end
   end
 
   defp depth(_pretty, []),    do: ""
@@ -188,37 +197,42 @@ defmodule Mix.Utils do
   Outputs the given tree according to the callback as a DOT graph.
 
   The callback will be invoked for each node and it
-  must either return `{printed, children}` tuple or
-  `false` if the given node must not be printed.
+  must return a `{printed, children}` tuple.
   """
-  @spec write_dot_graph!(Path.t, String.t, term, (term -> {String.t, [term]}), Keyword.t) :: :ok
-  def write_dot_graph!(path, title, root, callback, _opts \\ []) do
-    {{parent, _}, children} = callback.(root)
-    {dot, _} = build_dot_graph(parent, children, %{}, callback)
+  @spec write_dot_graph!(Path.t, String.t, [tree_node], (tree_node -> {tree_node, [tree_node]}), Keyword.t) :: :ok
+  def write_dot_graph!(path, title, nodes, callback, _opts \\ []) do
+    {dot, _} = build_dot_graph(make_ref(), nodes, MapSet.new(), callback)
     File.write! path, "digraph \"#{title}\" {\n#{dot}}\n"
   end
 
   defp build_dot_graph(_parent, [], seen, _callback), do: {"", seen}
   defp build_dot_graph(parent, [node | nodes], seen, callback) do
     {{name, edge_info}, children} = callback.(node)
-    {current, seen}  = build_dot_current(parent, name, edge_info, seen)
-    {children, seen} = build_dot_graph(name, children, seen, callback)
-    {siblings, seen} = build_dot_graph(parent, nodes, seen, callback)
-    {current <> children <> siblings, seen}
+    key = {parent, name}
+
+    if MapSet.member?(seen, key) do
+      {"", seen}
+    else
+      seen = MapSet.put(seen, key)
+      current = build_dot_current(parent, name, edge_info)
+      {children, seen} = build_dot_graph(name, children, seen, callback)
+      {siblings, seen} = build_dot_graph(parent, nodes, seen, callback)
+      {current <> children <> siblings, seen}
+    end
   end
 
-  defp build_dot_current(parent, name, edge_info, seen) do
-    key = {parent, name}
-    case seen do
-      %{^key => _} ->
-        {"", seen}
-      %{} when is_nil(edge_info) ->
-        {~s(  "#{parent}" -> "#{name}"\n),
-         Map.put(seen, key, true),}
-      %{} ->
-        {~s(  "#{parent}" -> "#{name}" [label=\"#{edge_info}\"]\n),
-         Map.put(seen, key, true)}
-    end
+  defp build_dot_current(parent, name, edge_info) do
+    edge_info =
+      if edge_info do
+         ~s( [label="#{edge_info}"])
+      end
+
+    parent =
+      unless is_reference(parent) do
+        ~s("#{parent}" -> )
+      end
+
+    ~s(  #{parent}"#{name}"#{edge_info}\n)
   end
 
   @doc false

lib/mix/test/mix/tasks/app.tree_test.exs

@@ -87,6 +87,7 @@ defmodule Mix.Tasks.App.TreeTest do
 
       assert File.read!("app_tree.dot") == """
         digraph "application tree" {
+          "test"
           "test" -> "elixir"
           "test" -> "logger"
           "logger" -> "elixir"

lib/mix/test/mix/tasks/deps.tree_test.exs

@@ -117,6 +117,7 @@ defmodule Mix.Tasks.Deps.TreeTest do
 
       assert File.read!("deps_tree.dot") == """
         digraph "dependency tree" {
+          "sample"
           "sample" -> "git_repo" [label=">= 0.1.0"]
           "sample" -> "deps_on_git_repo" [label="0.2.0"]
         }
@@ -127,6 +128,7 @@ defmodule Mix.Tasks.Deps.TreeTest do
 
       assert File.read!("deps_tree.dot") == """
         digraph "dependency tree" {
+          "sample"
           "sample" -> "git_repo" [label=">= 0.1.0"]
           "sample" -> "deps_on_git_repo" [label="0.2.0"]
           "deps_on_git_repo" -> "git_repo" [label=""]

lib/mix/test/mix/tasks/deps_test.exs

@@ -193,6 +193,20 @@ defmodule Mix.Tasks.DepsTest do
 
   ## deps.unlock
 
+  test "cleans and recompiles artifacts if --force given" do
+    Mix.Project.push SuccessfulDepsApp
+
+    in_fixture "deps_status", fn ->
+      Mix.Tasks.Deps.Compile.run []
+      File.touch! "_build/dev/lib/ok/clean-me"
+
+      Mix.Tasks.Deps.Compile.run ["--force"]
+      refute File.exists? "_build/dev/lib/ok/clean-me"
+    end
+  end
+
+  ## deps.unlock
+
   test "unlocks all deps", context do
     Mix.Project.push DepsApp
     in_tmp context.test, fn ->

lib/mix/test/mix/tasks/do_test.exs

@@ -11,15 +11,11 @@ defmodule Mix.Tasks.DoTest do
     end
   end
 
-  test "gather_command ignore spaces and trailing commas" do
+  test "gather_command returns a list of commands" do
     import Mix.Tasks.Do, only: [gather_commands: 1]
     assert gather_commands(["compile", "--list,", "help"]) == [["compile", "--list"], ["help"]]
-    assert gather_commands(["compile", "--list,help"]) == [["compile", "--list"], ["help"]]
-    assert gather_commands(["help", ",compile", "--list"]) == [["help"], ["compile", "--list"]]
-    assert gather_commands(["compile", "--list", ",", "help"]) == [["compile", "--list"], ["help"]]
+    assert gather_commands(["help,", "compile", "--list"]) == [["help"], ["compile", "--list"]]
     assert gather_commands(["compile,", "run", "-e", "IO.puts :hello"]) == [["compile"], ["run", "-e", "IO.puts :hello"]]
-    assert gather_commands(
-      [",", "compile,", "run", "-e", "IO.puts :hello",",foo", "--bar", "--baz", ",", "baz,qux,abc", ","]) ==
-      [["compile"], ["run", "-e", "IO.puts :hello"], ["foo", "--bar", "--baz"], ["baz"], ["qux"], ["abc"]]
+    assert gather_commands(["compile,", "run", "-e", "[1, 2]"]) == [["compile"], ["run", "-e", "[1, 2]"]]
   end
 end

lib/mix/test/mix/tasks/xref_test.exs

@@ -129,25 +129,46 @@ defmodule Mix.Tasks.XrefTest do
 
   test "warnings: handles multiple modules in one file" do
     assert_warnings """
-    defmodule A do
+    defmodule A1 do
       def a, do: A2.no_func
       def b, do: A2.a
     end
 
     defmodule A2 do
-      def a, do: A.no_func
-      def b, do: A.b
+      def a, do: A1.no_func
+      def b, do: A1.b
     end
     """, """
     warning: function A2.no_func/0 is undefined or private
       lib/a.ex:2
 
-    warning: function A.no_func/0 is undefined or private
+    warning: function A1.no_func/0 is undefined or private
       lib/a.ex:7
 
     """
   end
 
+  test "warnings: doesn't load unloaded modules" do
+    assert_warnings """
+    defmodule A1 do
+      @compile {:autoload, false}
+      @on_load :init
+      def init do
+        raise "oops"
+      end
+    end
+
+    defmodule A2 do
+      def a, do: A1.no_func
+      def b, do: A1.init
+    end
+    """, """
+    warning: function A1.no_func/0 is undefined or private
+      lib/a.ex:10
+
+    """
+  end
+
   test "warnings: groups multiple warnings in one file" do
     assert_warnings """
     defmodule A do
@@ -241,7 +262,7 @@ defmodule Mix.Tasks.XrefTest do
       File.write!("lib/a.ex", contents)
 
       assert capture_io(:stderr, fn ->
-        assert Mix.Task.run("xref", ["--warnings"]) == :error
+        assert Mix.Task.run("xref", ["warnings"]) == :error
       end) == expected
     end
   end
@@ -251,7 +272,7 @@ defmodule Mix.Tasks.XrefTest do
       File.write!("lib/a.ex", contents)
 
       assert capture_io(:stderr, fn ->
-        assert Mix.Task.run("xref", ["--warnings"]) == :ok
+        assert Mix.Task.run("xref", ["warnings"]) == :ok
       end) == ""
     end
   end
@@ -274,7 +295,7 @@ defmodule Mix.Tasks.XrefTest do
       File.write!("lib/a.ex", contents)
 
       assert capture_io(fn ->
-        assert Mix.Task.run("xref", ["--unreachable"]) == :error
+        assert Mix.Task.run("xref", ["unreachable"]) == :error
       end) == expected
     end
   end
@@ -420,12 +441,10 @@ defmodule Mix.Tasks.XrefTest do
 
   test "callers: no argument gives error" do
     in_fixture "no_mixfile", fn ->
-      message =
-        "Could not invoke task \"xref\": 1 error found!\n" <>
-        "--callers : Missing argument of type string"
+      message = "xref doesn't support this command, see mix help xref for more information"
 
       assert_raise Mix.Error, message, fn ->
-        assert Mix.Task.run("xref", ["--callers"]) == :error
+        assert Mix.Task.run("xref", ["callers"]) == :error
       end
     end
   end
@@ -433,10 +452,10 @@ defmodule Mix.Tasks.XrefTest do
   test "callers: gives nice error for quotable but invalid callers spec" do
     in_fixture "no_mixfile", fn ->
       message =
-        "xref --callers expects Module, Module.function, or Module.function/arity, got: Module.func(arg)"
+        "xref callers CALLEE expects Module, Module.function, or Module.function/arity, got: Module.func(arg)"
 
       assert_raise Mix.Error, message, fn ->
-        Mix.Task.run("xref", ["--callers", "Module.func(arg)"])
+        Mix.Task.run("xref", ["callers", "Module.func(arg)"])
       end
     end
   end
@@ -444,10 +463,10 @@ defmodule Mix.Tasks.XrefTest do
   test "callers: gives nice error for unquotable callers spec" do
     in_fixture "no_mixfile", fn ->
       message =
-        "xref --callers expects Module, Module.function, or Module.function/arity, got: %"
+        "xref callers CALLEE expects Module, Module.function, or Module.function/arity, got: %"
 
       assert_raise Mix.Error, message, fn ->
-        Mix.Task.run("xref", ["--callers", "%"])
+        Mix.Task.run("xref", ["callers", "%"])
       end
     end
   end
@@ -458,8 +477,146 @@ defmodule Mix.Tasks.XrefTest do
       File.write!("lib/b.ex", contents_b)
 
       assert capture_io(fn ->
-        assert Mix.Task.run("xref", ["--callers", callee]) == :ok
+        assert Mix.Task.run("xref", ["callers", callee]) == :ok
       end) == expected
     end
   end
+
+  ## Graph
+
+  test "graph: basic usage" do
+    assert_graph """
+    lib/a.ex
+    └── lib/b.ex
+        └── lib/a.ex
+    lib/b.ex
+    lib/c.ex
+    lib/d.ex
+    └── lib/a.ex (compile)
+    """
+  end
+
+  test "graph: exclude" do
+    assert_graph ~w[--exclude lib/c.ex --exclude lib/b.ex], """
+    lib/a.ex
+    lib/d.ex
+    └── lib/a.ex (compile)
+    """
+  end
+
+  test "graph: exclude 1" do
+    assert_graph ~w[--exclude lib/d.ex], """
+    lib/a.ex
+    └── lib/b.ex
+        └── lib/a.ex
+    lib/b.ex
+    lib/c.ex
+    """
+  end
+
+  test "graph: dot format" do
+    assert_graph ~w[--format dot], true, """
+    digraph "xref graph" {
+      "lib/a.ex"
+      "lib/a.ex" -> "lib/b.ex"
+      "lib/b.ex" -> "lib/a.ex"
+      "lib/b.ex"
+      "lib/c.ex"
+      "lib/d.ex"
+      "lib/d.ex" -> "lib/a.ex" [label="(compile)"]
+    }
+    """
+  end
+
+  test "graph: source" do
+    assert_graph ~w[--source lib/a.ex], """
+    lib/a.ex
+    └── lib/b.ex
+        └── lib/a.ex
+    """
+  end
+
+  test "graph: invalid source" do
+    assert_raise Mix.Error, "Source could not be found: lib/a2.ex", fn ->
+      assert_graph ~w[--source lib/a2.ex], ""
+    end
+  end
+
+  test "graph: sink" do
+    assert_graph ~w[--sink lib/b.ex], """
+    lib/a.ex
+    └── lib/b.ex
+        └── lib/a.ex
+    lib/d.ex
+    └── lib/a.ex (compile)
+    """
+  end
+
+  test "graph: invalid sink" do
+    assert_raise Mix.Error, "Sink could not be found: lib/b2.ex", fn ->
+      assert_graph ~w[--sink lib/b2.ex], ""
+    end
+  end
+
+  test "graph: sink and source is error" do
+    assert_raise Mix.Error, "mix xref graph expects only one of --source and --sink", fn ->
+      assert_graph ~w[--source lib/a.ex --sink lib/b.ex], ""
+    end
+  end
+
+  defp assert_graph(opts \\ [], dot \\ false, expected) do
+    in_fixture "no_mixfile", fn ->
+      File.write! "lib/a.ex", """
+      defmodule A do
+        def a do
+          B.a
+        end
+
+        def b, do: :ok
+      end
+      """
+
+      File.write! "lib/b.ex", """
+      defmodule B do
+        def a do
+          A.a
+          B.a
+        end
+      end
+      """
+
+      File.write! "lib/c.ex", """
+      defmodule C do
+      end
+      """
+
+      File.write! "lib/d.ex", """
+      defmodule :d do
+        A.b
+      end
+      """
+
+      assert Mix.Task.run("xref", opts ++ ["graph"]) == :ok
+
+      result =
+        if dot do
+          File.read!("xref_graph.dot")
+        else
+          assert "Compiling 4 files (.ex)\nGenerated sample app\n" <> result =
+            receive_until_no_messages([])
+
+          result
+        end
+
+      assert result == expected
+    end
+  end
+
+  defp receive_until_no_messages(acc) do
+    receive do
+      {:mix_shell, :info, [line]} -> receive_until_no_messages([acc, line | "\n"])
+    after
+      0 -> IO.iodata_to_binary(acc)
+    end
+  end
 end

lib/mix/test/mix/umbrella_test.exs

@@ -135,7 +135,7 @@ defmodule Mix.UmbrellaTest do
     Mix.env(:test)
   end
 
-  test "loads umbrella child dependencies in umbrellas" do
+  test "loads umbrella sibling dependencies with :in_umbrella" do
     in_fixture "umbrella_dep/deps/umbrella", fn ->
       Mix.Project.in_project :umbrella, ".", fn _ ->
         File.write! "apps/bar/mix.exs", """
@@ -157,6 +157,31 @@ defmodule Mix.UmbrellaTest do
     end
   end
 
+  test "finds umbrella sibling dependencies conflicts with :in_umbrella" do
+    in_fixture "umbrella_dep/deps/umbrella", fn ->
+      Mix.Project.in_project :umbrella, ".", fn _ ->
+        File.write! "apps/bar/mix.exs", """
+        defmodule Bar.Mixfile do
+          use Mix.Project
+
+          def project do
+            [app: :bar,
+             version: "0.1.0",
+             deps: [{:foo, in_umbrella: true, env: :unknown}]]
+          end
+        end
+        """
+
+        assert_raise Mix.Error, fn ->
+          Mix.Tasks.Deps.Get.run []
+        end
+
+        assert_received {:mix_shell, :error, ["Dependencies have diverged:"]}
+        assert_received {:mix_shell, :error, ["  the dependency foo in mix.exs is overriding a child dependency" <> _]}
+      end
+    end
+  end
+
   ## Umbrellas as a dependency
 
   test "list deps for umbrella as dependency" do

src/elixir.app.src

@@ -1,6 +1,6 @@
 {application, elixir,
 [{description, "elixir"},
- {vsn, "1.3.0-rc.1"},
+ {vsn, "1.3.1"},
  {modules, [
 	elixir
   ]},