Release v1.5.3

Signed-off-by: José Valim <jose.valim@plataformatec.com.br>

.appveyor.yml

@@ -0,0 +1,9 @@
+version: 1-{branch}+{build}
+
+build_script:
+  - cmd: C:\MinGW\msys\1.0\bin\make
+  - cmd: rmdir /s /q .git
+before_test:
+  - cmd: set PATH=%PATH%;C:\Program Files\erl8.3\erts-8.3\bin
+test_script:
+  - cmd: C:\MinGW\msys\1.0\bin\make --keep-going test_windows

.gitignore

@@ -1,17 +1,14 @@
-/doc
-/ebin
-/lib/*/ebin/*
-/lib/*/tmp
-/lib/elixir/src/*_lexer.erl
+/doc/
+/lib/*/ebin/
+/lib/*/_build/
+/lib/*/tmp/
 /lib/elixir/src/*_parser.erl
 /lib/elixir/src/elixir.app.src
-/lib/elixir/test/ebin
+/lib/elixir/test/ebin/
 /man/elixir.1
 /man/iex.1
-/rel/elixir
 /Docs-v*.zip
 /Precompiled-v*.zip
 /.eunit
-/.release
 .elixir.plt
 erl_crash.dump

.travis.yml

@@ -1,19 +1,38 @@
 language: erlang
-
-otp_release:
-  - 18.0
-  - 18.1
-  - 18.2
-  - 18.3
-  - 19.0
-  - 19.1
-
 sudo: false
 
-script: "make compile && rm -rf .git && make test"
+os: linux
+otp_release: 18.0
+
+matrix:
+  include:
+    - os: linux
+      otp_release: 18.1
+    - os: linux
+      otp_release: 18.2
+    - os: linux
+      otp_release: 18.3
+    - os: linux
+      otp_release: 19.0
+    - os: linux
+      otp_release: 19.1
+    - os: linux
+      otp_release: 19.2
+    - os: linux
+      otp_release: 19.3
+    - os: linux
+      otp_release: 20.0
+
+env:
+  - ELIXIR_ASSERT_TIMEOUT=2000
+
+script:
+  - make compile
+  - rm -rf .git
+  - make test
+  - dialyzer -pa lib/elixir/ebin --build_plt --output_plt elixir.plt --apps lib/elixir/ebin/elixir.beam lib/elixir/ebin/Elixir.Kernel.beam
 
 notifications:
-  irc: "irc.freenode.org#elixir-lang"
   recipients:
     - jose.valim@plataformatec.com.br
     - eric.meadows.jonsson@gmail.com

CHANGELOG.md

@@ -1,364 +1,402 @@
-# Changelog for Elixir v1.4
+# Changelog for Elixir v1.5
 
-Elixir v1.4 brings new features, enhancements and bug fixes into Elixir. The most notable changes are the addition of the `Registry` module and the `Task.async_stream/3` and `Task.async_stream/5` which aid developers in writing concurrent software. Those two features and a couple other improvements are described in detail below followed by the complete list of changes.
+Elixir v1.5 brings new features, enhancements and bug fixes to Elixir. It is the first release to leverage features added as part of Erlang/OTP 20. It is also the last release that supports Erlang/OTP 18.
 
-## Registry
+## UTF-8 atoms, function names and variables
 
-The registry is a local, decentralized and scalable key-value process storage:
+Elixir v1.5 supports non-quoted atoms and variables to be in UTF-8 when using Erlang/OTP 20+. For example:
 
-  * Local because keys and values are only accessible to the current node (opposite to distributed)
-  * Decentralized because there is no single entity responsible for managing the registry
-  * Scalable because performance scales linearly with the addition of more cores upon partitioning
-
-A registry is chosen upon start to have unique or duplicate keys. Every key-value pair is associated to the process registering the key. Keys are automatically removed once the owner process terminates.
-
-    iex> Registry.start_link(:unique, MyRegistry)
-    iex> {:ok, _} = Registry.register(MyRegistry, "hello", 1)
-    iex> Registry.lookup(MyRegistry, "hello")
-    [{self(), 1}]
-
-With the registry, developers can provide dynamic process names, module-function dispatch or even a local pubsub system. See the `Registry` documentation for more information.
-
-## Syntax coloring
-
-Elixir v1.4 introduces the ability to syntax color inspected data structures:
-
-    iex> IO.puts inspect([hello: 1, world: "!"], syntax_colors: [atom: :cyan])
-    [hello: 1, world: "!"]
-
-Coloring is done with ANSI colors as specified in the `IO.ANSI` module.
-
-IEx automatically relies on this feature to provide syntax coloring for evaluated shell results. This behaviour can be configured via the `:syntax_colors` coloring option:
-
-    IEx.configure [colors: [syntax_colors: [atom: :cyan, string: :green]]]
-
-To disable coloring altogether, pass an empty list to `:syntax_colors`.
-
-## Calendar
-
-Elixir v1.3 introduced new calendar types. This release continues evolving the Calendar APIs by adding functions for comparing, adding and calculating the difference between types, retrieve the `day_of_week/1`, check if the current date is a `leap_year?/1` and more.
-
-## Task.async_stream
-
-When there is a need to traverse a collection of items concurrently, Elixir developers often resort to tasks:
-
-    collection
-    |> Enum.map(&Task.async(SomeMod, :function, [&1]))
-    |> Enum.map(&Task.await/1)
-
-While the snippet above works fine in many occasions, for large collections it will spawn and run concurrently as many tasks as there are items in the collection.
-
-`Task.async_stream/3` and `Task.async_stream/5` allows developers to process collections concurrently while controlling the maximum amount of concurrent tasks:
-
-    collection
-    |> Task.async_stream(SomeMod, :function, [], max_concurrency: System.schedulers_online)
-
-The `Task.async_stream` functions are also lazy, allowing developers to partially consume the stream until a condition is reached. Furthermore, `Task.Supervisor.async_stream/4` and `Task.Supervisor.async_stream/6` can be used to ensure the concurrent tasks are spawned under a given supervisor.
-
-## Application inference
-
-Mix v1.4 now automatically infers the list of applications that are required on runtime from your dependencies list.
-
-In previous Mix versions, most of your dependencies had to be added both to your dependencies list and applications list. Here is how a `mix.exs` would look like:
-
-    def application do
-      [applications: [:logger, :plug, :postgrex]]
+    test "こんにちは世界" do
+      assert :こんにちは世界
     end
 
-    def deps do
-      [{:plug, "~> 1.2"},
-       {:postgrex, "~> 1.0"}]
+Or:
+
+    saudação = "Bom dia!"
+
+Elixir follows the recommendations in [Unicode Annex #31](http://unicode.org/reports/tr31/) to make the language more accessible to other languages and communities. Identifiers must start with a letter or underscore, optionally followed by letters, digits, and underscores. Here letter means any UTF-8 letter-character (optionally with a combining mark) and digit means a UTF-8 decimal-digit-character. If you're using ASCII, this does what you'd expect.
+
+Examples of valid variables are:
+
+    name    josé   _age   まつもと  _42  адрес
+    
+Examples of invalid variables include:
+
+    name•   a±2  42
+
+Symbols, such as mathematical notations and emoji, are not allowed identifiers.
+
+For a complete reference on Elixir syntax, see the [Syntax Reference](https://hexdocs.pm/elixir/syntax-reference.html). For technical details on Unicode support, see [Unicode Syntax](https://hexdocs.pm/elixir/unicode-syntax.html).
+
+## IEx improvements
+
+IEx got many improvements. The autocompletion system is now capable of autocompleting variables and user imports. New helpers have also been added:
+
+  * `exports/1` lists all exports (functions and macros) in a given module
+  * `open/1` opens up the source of a module or function directly in your editor. For example, `open MyApp.Module`
+  * `runtime_info/0` prints general information about the running system, such as number of cores, runtime version, allocation of memory in the VM and more
+
+IEx also features a breakpoint system for code debugging. The following functions have been added to aid debugging:
+
+  * `break!/2` - sets up a breakpoint for a given `Mod.fun/arity`
+  * `break!/4` - sets up a breakpoint for the given module, function, arity
+  * `breaks/0` - prints all breakpoints and their ids
+  * `continue/0` - continues until the next breakpoint in the same process
+  * `open/0` - opens editor on the current breakpoint
+  * `remove_breaks/0` - removes all breakpoints in all modules
+  * `remove_breaks/1` - removes all breakpoints in a given module
+  * `reset_break/1` - sets the number of stops on the given id to zero
+  * `reset_break/3` - sets the number of stops on the given module, function, arity to zer
+  * `respawn/0` - starts a new shell (breakpoints will ask for permission once more)
+  * `whereami/1` - shows the current location
+
+## Exception.blame
+
+`Exception.blame/3` is a new function in Elixir that is capable of attaching debug information to certain exceptions. Currently this is used to augment `FunctionClauseError`s with a summary of all clauses and which parts of clause match and which ones didn't. For example:
+
+    iex> Access.fetch(:foo, :bar)
+    ** (FunctionClauseError) no function clause matching in Access.fetch/2
+
+    The following arguments were given to Access.fetch/2:
+
+        # 1
+        :foo
+
+        # 2
+        :bar
+
+    Attempted function clauses (showing 5 out of 5):
+
+        def fetch(-%struct{} = container-, key)
+        def fetch(map, key) when -is_map(map)-
+        def fetch(list, key) when -is_list(list)- and is_atom(key)
+        def fetch(list, key) when -is_list(list)-
+        def fetch(-nil-, _key)
+
+    (elixir) lib/access.ex:261: Access.fetch/2
+
+In the example above, an argument that did not match or guard that did not evaluate to true are shown between `-`. If the terminal supports ANSI coloring, they are wrapped in red instead of the `-` character.
+
+Since blaming an exception can be expensive, `Exception.blame/3` must be used exclusively in debugging situations. It is not advised to apply it to production components such as a Logger. This feature has been integrated into the compiler, the command line, ExUnit and IEx.
+
+This feature also requires Erlang/OTP 20+.
+
+## Streamlined child specs
+
+Elixir v1.5 streamlines how supervisors are defined and used in Elixir. Elixir now allows child specifications, which specify how a child process is supervised, to be defined in modules. In previous versions, a project using Phoenix would write:
+
+    import Supervisor.Spec
+
+    children = [
+      supervisor(MyApp.Repo, []),
+      supervisor(MyApp.Endpoint, [])
+    ]
+
+    Supervisor.start_link(children, strategy: :one_for_one)
+
+In Elixir v1.5, one might do:
+
+    children = [
+      MyApp.Repo,
+      MyApp.Endpoint
+    ]
+
+    Supervisor.start_link(children, strategy: :one_for_one)
+
+The above works by calling the `child_spec/1` function on the given modules.
+
+This new approach allows `MyApp.Repo` and `MyApp.Endpoint` to control how they run under a supervisor. This reduces the chances of mistakes being made, such as starting an Ecto repository as a worker or forgetting to declare that tasks are temporary in a supervision tree.
+
+If it is necessary to configure any of the children, such can be done by passing a tuple instead of an atom:
+
+    children = [
+      {MyApp.Repo, url: "ecto://localhost:4567/my_dev"},
+      MyApp.Endpoint
+    ]
+
+The modules `Agent`, `Registry`, `Task`, and `Task.Supervisor` have been updated to include a `child_spec/1` function, allowing them to be used directly in a supervision tree similar to the examples above. `use Agent`, `use GenServer`, `use Supervisor`, and `use Task` have also been updated to automatically define an overridable `child_spec/1` function.
+
+Finally, child specifications are now provided as maps (data-structures) instead of the previous `Supervisor.Spec.worker/3` and `Supervisor.Spec.supervisor/3` APIs. This behaviour also aligns with how supervisors are configured in Erlang/OTP 18+. See the updated `Supervisor` docs for more information, as well as the new `Supervisor.init/2` and `Supervisor.child_spec/2` functions.
+
+## @impl
+
+This release also allows developers to mark which functions in a given module are an implementation of a callback. For example, when using the [Plug](https://github.com/elixir-lang/plug) project, one needs to implement both `init/1` and `call/2` when writing a Plug:
+
+    defmodule MyApp do
+      @behaviour Plug
+
+      def init(_opts) do
+        opts
+      end
+
+      def call(conn, _opts) do
+        Plug.Conn.send_resp(conn, 200, "hello world")
+      end
     end
 
-This was error prone as many developers would not list their dependencies in their applications list.
+The problem with the approach above is that, once more and more functions are added to the `MyApp` module, it becomes increasingly harder to know the purposes of the `init/1` and `call/2` functions. For example, for a developer unfamiliar with Plug, are those functions part of the `MyApp` API or are they implementations of a given callback?
 
-Mix v1.4 now automatically infers your applications list as long as you leave the `:applications` key empty. The `mix.exs` above can be rewritten to:
+Elixir v1.5 introduces the `@impl` attribute, which allows us to mark that certain functions are implementation of callbacks:
 
-    def application do
-      [extra_applications: [:logger]]
+    defmodule MyApp do
+      @behaviour Plug
+
+      @impl true
+      def init(_opts) do
+        opts
+      end
+
+      @impl true
+      def call(conn, _opts) do
+        Plug.Conn.send_resp(conn, 200, "hello world")
+      end
     end
 
-    def deps do
-      [{:plug, "~> 1.2"},
-       {:postgrex, "~> 1.0"}]
-    end
+You may even use `@impl Plug` if you want to explicitly document which behaviour defines the callback you are implementing.
 
-With the above, Mix will automatically build your application list based on your dependencies. Applications that are part of Erlang or Elixir that are required at runtime, such as `:logger`, must be added to the `:extra_applications` list. All extra applications will be included in the application list.
+Overall, using `@impl` has the following advantages:
 
-Finally, if there is a dependency you don't want to include in the application runtime list, you can do so by specifying the `runtime: false` option:
+  * Readability of the code is increased, as it is now clear which functions are part of your API and which ones are callback implementations. To reinforce this idea, `@impl true` automatically marks the function as `@doc false`, disabling documentation unless `@doc` is explicitly set
 
-    {:distillery, "> 0.0.0", runtime: false}
+  * If you define `@impl` before a function that is not a callback, Elixir will error. This is useful in case of typos or in case the behaviour definition changes (such as a new major version of a library you depend on is released)
 
-We hope this feature provides a more streamlined workflow for developers who are building releases for their Elixir projects.
+  * If you use `@impl` in one implementation, Elixir will force you to declare `@impl` for all other implementations in the same module, keeping your modules consistent
 
-## Mix install from SCM
+## Calendar improvements
 
-Mix v1.4 can now install escripts and archives from both Git and Hex, providing you with even more options for distributing Elixir code.
+This release brings further improvements to Calendar types. It adds arithmetic and others functions to `Time`, `Date`, `NaiveDateTime` and `Datetime` as well as conversion between different calendars.
 
-This makes it possible to distribute CLI applications written in Elixir by publishing a package which builds an escript to Hex. [`ex_doc`](https://hex.pm/packages/ex_doc) has been updated to serve as an example of how to use this new functionality.
-
-Simply running:
-
-    mix escript.install hex ex_doc
-
-will fetch `ex_doc` and its dependencies, build them, and then install `ex_doc` to `~/.mix/escripts` (by default). After adding `~/.mix/escripts` to your `PATH`, running `ex_doc` is as simple as:
-
-    ex_doc
-
-You can now also install archives from Hex in this way. Since they are fetched and built on the user's machine, they do not have the same limitations as pre-built archives. However, keep in mind archives run alongside every Mix project, which may lead to conflicts. For this reason, escripts is the preferred format.
-
-It is also possible to install escripts and archives by providing a Git/GitHub repo. See `mix help escript.install` and `mix help archive.install` for more details.
-
-## v1.4.5 (2017-06-22)
-
-This version includes changes that make Elixir fully compatible with Erlang/OTP 20.
-
-### 1. Enhancements
-
-#### Logger
-
-  * [Logger] Handle changes to crash reports in OTP 20
-
-### 2. Bug fixes
-
-#### Elixir
-
-  * [DateTime] Fix `DateTime.from_iso8601/2` when offset has no colon
-  * [Registry] Do not leak EXIT messages on `Registry.dispatch/3`
-
-## v1.4.4 (2017-05-15)
-
-This version includes changes that make Elixir fully compatible with Erlang/OTP 20-rc.1.
+## v1.5.3 (2017-12-19)
 
 ### 1. Bug fixes
 
 #### Elixir
 
-  * [Map] Fix regression on struct update syntax
-
-## v1.4.3 (2017-05-15)
-
-This version includes changes that make Elixir fully compatible with Erlang/OTP 20-rc.1.
-
-### 1. Enhancements
-
-#### Elixir
-
-  * [Kernel] Improve compilation time for modules with many clauses
-  * [Map] Warn when attempting to override `__struct__` key
-  * [Regex] Add `recompile/1` and `recompile!/1` to ease transition to OTP 20 for archives and stored regexes
-
-#### Logger
-
-  * [Logger.Translator] Handle OTP 20 GenServer log messages
-
-#### Mix
-
-  * [mix compile] Recompile projects if OTP version changes
-
-### 2. Bug fixes
-
-#### Elixir
-
-  * [Kernel] Fix code generation when non-binary bitstrings are in AST
-  * [Record] Properly escape fields passed to `defrecord`
-  * [Version] Reject leading zeros according to the SemVer spec
+  * [Calendar] Consider microseconds in maximum possible Calendar.ISO datetime
+  * [Enum] Fix `chunk_every/4` when `step > count`
+  * [Kernel] Warn duplicate definitions in macros
+  * [Kernel] Remove dialyzer warnings from else in with clauses
+  * [Kernel] Do not warning on upcoming `@deprecated` and `@since` attributes for v1.6
+  * [MapSet] Return valid MapSet when union-ing a legacy MapSet
 
 #### ExUnit
 
-  * [ExUnit.Diff] Do not fail when comparing maps with nil or boolean keys
+  * [ExUnit] Fix compiler warnings in `assert_receive/3`
+
+## v1.5.2 (2017-09-29)
+
+### 1. Enhacements
+
+#### Elixir
+
+  * [Kernel] Optimize function definition with multiple clauses by not traversing the internal clauses table
+  * [Kernel] Warn if unary operators are followed by new lines
+  * [Registry] Use the name of the Registry as its `:id` in the `child_spec/1` function
+
+### 2. Bug fixes
+
+#### Elixir
+
+  * [DateTime] Fix negative microsecond result when passing negative Unix epochs to `from_unix/2`
+  * [Kernel] Improve error message for oversized atoms
+  * [Kernel] Ensure `@impl` attribute also propagates to clauses from default arguments
+  * [Kernel] Emit proper error for unknown vars inside binary pattern in match
 
 #### IEx
 
-  * [IEx.Helpers] Do not log exits on `IEx.Helpers.c/2` failures
+  * [IEx] Do not crash IEx unexpectedly on `System.stop/0`
+  * [IEx.Helpers] Ensure exiting a breakpoint set inside a breakpoint does not terminate the shell unexpectedly
 
 #### Mix
 
-  * [mix archive.install] Detect proper path on URLs with query strings
-  * [mix loadpaths] Do not assume all paths in loadpaths exist
+  * [mix local.hex] Ensure `--if-missing` flag works as advertised
+  * [mix test] Do not trigger additional error reports when there is a failure when loading test files
+  * [Mix.SCM.Git] Ensure errors when invoking `git` propagate correctly
 
+## v1.5.1 (2017-08-01)
 
-## v1.4.2 (2017-02-16)
-
-### 1. Bug fixes
+### 1. Enhancements
 
 #### EEx
 
-  * [EEx] Support middle expressions on trim mode
+  * [EEx.Engine] Add `handle_begin` and `handle_end` to EEx
 
 #### Elixir
 
-  * [Calendar] Correct typo on Calendar types
-  * [Kernel] Ensure redefined functions point to the proper source
-  * [OptionParser] Add `:allow_nonexistent_atoms` to support unsafe behaviour prior to v1.4
-  * [Stream] Allow consuming multiple items from suspended enumerable in `Stream.transform/3`
-  * [String] Incorporate new grapheme rules in Unicode 9
-
-#### IEx
-
-  * [IEx.Autocomplete] Do not crash on aliases which are not known at compile time
+  * [Kernel] Do not use references on function/macro definitions - this provides large improvements in compilation times in some rare corner cases
+  * [Supervisor] Support mixing old and new typespecs in `Supervisor.init/2` and `Supevisor.start_link/2`
 
 #### Mix
 
-  * [Mix.Umbrella] Ensure umbrella projects can depend on other umbrella projects
-  * [Mix.Archive] Ensure previous archives with `.ez` extension are deleted
+  * [mix profile] Allow profile tasks to run without a project
 
-## v1.4.1 (2017-01-26)
+### 2. Bug fixes
 
-### 1. Bug fixes
+#### EEx
+
+  * [EEx.Engine] Do not re-use the value of the `init/1` callback throughout the compilation stack
 
 #### Elixir
 
-  * [Kernel] Remove warning when making private functions overridable
-  * [Path] Ensure `Path.join/1` returns strings for lists of one element
-  * [Regex] Ensure `Regex.escape/1` also escapes `-`
+  * [Kernel] Ensure dialyzer does not emit warnings in some uses of `with`
+  * [Kernel] Fix dialyzer warnings when `defmacrop` is used in modules
+  * [Kernel] Ensure Elixir modules can be dialyzed without starting the Elixir application
+  * [Kernel] Do not serialize references in quoted expressions
+  * [Kernel] Make sure structs expansion use the latest definition available when struct modules are recompiled
+  * [Task] Support `:infinity` timeout on Task streams
+  * [Typespec] Ensure typespecs allow `tuple` to be used as variable names
 
-#### IEx
-
-  * [IEx] Disable ANSI detection for powershell to avoid false positives
-
-#### Mix
-
-  * [Mix.Make] Run `make clean` for `erlang.mk`
-  * [Mix.Rebar] Support all of rebar3 dependency package declaration
-  * [Mix.Rebar] Only pass overrides from parent to child in Rebar dep
-
-## v1.4.0 (2017-01-05)
+## v1.5.0 (2017-07-25)
 
 ### 1. Enhancements
 
 #### Elixir
 
-  * [Calendar] Add `Date.compare/2`, `Time.compare/2`, `NaiveDateTime.compare/2` and `DateTime.compare/2`
-  * [Calendar] Support `NaiveDateTime.add/3` and `NaiveDateTime.diff/3` for adding seconds (up to microseconds) as well as the difference between two NaiveDateTimes in seconds (up to microseconds)
-  * [Calendar] Add `Date.leap_year?/1` and `Date.day_of_week/1`
-  * [Calendar] Ensure `Date`, `Time` and `NaiveDateTime` APIs work with any struct that provides the same set of fields as their respective struct. For example, a `NaiveDateTime` can be given to `Date` since it contains a superset of the fields in the `Date` struct
-  * [Calendar] Add `Time.utc_now/0` and `NaiveDateTime.utc_now/0`
-  * [Enum] Add `Enum.map_every/2` that invokes the given function with every nth item
-  * [Enum] Add `min/2`, `max/2`, `min_max/2`, `min_by/3`, `max_by/3`, and `min_max_by/3` that allow a function specifying the default value when the enumerable is empty
-  * [Enum] Introduce `Enum.zip/1` to zip multiple entries at once
-  * [Float] Introduce `Float.ratio/1` that returns a tuple with the numerator and denominator as integers to retrieve the given float
-  * [GenServer] Log warn on default `handle_info/2` implementation
-  * [Inspect] Support syntax coloring via the `:syntax_color` option
-  * [Integer] `Integer.digits/2` now accepts negative integers
-  * [Integer] Add `Integer.mod/2` and `Integer.floor_div/2`
-  * [IO] Add `:label` option to `IO.inspect/2` to help distinguish multiple `IO.inspect/2` calls.
-  * [Kernel] Recognize merge conflict markers in source and provide a readable error message
-  * [Kernel] Warn on unused module attributes
-  * [Kernel] Improve compiler message on unexpected end of line
-  * [Kernel] Raise `BadBooleanError` when a non-boolean is given on the left-hand side of `and`/`or`
-  * [List] Add `List.pop_at/3`
-  * [List] Add `List.myers_difference/2`
-  * [OptionParser] Expand multi-letter aliases in `OptionParser`
-  * [Process] Add `Process.send_after/4`
-  * [Process] Improve error messages on `Process.register/2` errors
-  * [Registry] Add a local, decentralized and scalable key-value process storage
-  * [Stream] Add `Stream.map_every/2` that invokes the given function with every nth item
-  * [Stream] Introduce `Stream.zip/1` to lazily zip multiple entries at once
-  * [String] Update to Unicode 9.0.0
-  * [Task] Add `Task.async_stream/3` and `Task.async_stream/5` as well as the supervised versions `Task.Supervisor.async_stream/4` and `Task.Supervisor.async_stream/6`
-  * [URI] Allow 0 as URI scheme default port
+  * [Access] Optimize `Access.get/2`
+  * [Base] Optimise Base encode/decode
+  * [Calendar] Implement Inspect for DateTime with Calendar.ISO
+  * [Calendar] Add "ISO days" format for conversions between Calendars and `Date.convert/2`, `Time.convert/2`, `NaiveDateTime.convert/2` and `DateTime.convert/2` (as well as bang variants)
+  * [Calendar] Add `:calendar` field to `Time` struct
+  * [Calendar] Add `Time.diff/3`, `Date.add/2`, `Date.diff/2`, `DateTime.diff/3`
+  * [Calendar] Add `Date.range/2`
+  * [Calendar] Add `Date.new/4`, `DateTime.utc_now/1`, `NaiveDateTime.new/8` and `Time.new/5` that allow specifing calendar
+  * [Enum] Add `Enum.chunk_by/4` and `Stream.chunk_by/4`
+  * [Enum] Add `Enum.chunk_every/2` and `Enum.chunk_every/4` with a more explicit API than `Enum.chunk/2` and `Enum.chunk/4`
+  * [Exception] Add `Exception.blame/3` that adds metadata to exceptions
+  * [File] Add `File.read_link/1` and `File.read_link!/1`
+  * [File] Introduce `:trim_bom` option for `File.stream!/2`
+  * [Inspect] Add `:printable_limit` to control the limit of printable structures
+  * [Integer] Add `Integer.gcd/2`
+  * [Kernel] Add `left not in right` to check that the left side is not in the enumerable on the right
+  * [Kernel] Use the new `debug_info` chunk in OTP 20. This provides a mechanism for tools to retrieve the Elixir AST from beam files
+  * [Kernel] `defoverridable/1` accepts a module name as argument and marks all callbacks as overridable
+  * [Kernel] Allow non-quoted Unicode atoms and variables according to Unicode Annex #31 (see Unicode Syntax document)
+  * [Kernel] Warn when a `:__struct__` key is used when building/updating structs
+  * [Kernel] Cache the AST on definitions. This speeds up the compilation time from 10% to 15% measured across different projects
+  * [Kernel] Improve compiler error message on invalid patterns and guards
+  * [Keyword] Add `replace/3` and `replace!/3` for replacing an existing key
+  * [List] `List.starts_with?/2`
+  * [Macro] Introduce `Macro.generate_arguments/2`
+  * [Map] Optimize `Map.merge/3` by choosing merging direction
+  * [Map] Add `replace/3` and `replace!/3` for replacing an existing key
+  * [Map] Raise `BadMapError` in `Map.equal?/2` when either of the two arguments is not a map
+  * [MapSet] Reduce `MapSet` size when serialized to approximately half
+  * [Process] Add `Process.cancel_timer/2`
+  * [Protocol] Show available implementations on `Protocol.UndefinedError` if the protocol has been consolidated
+  * [Registry] Support ETS guard conditions in `Registry.match/3`
+  * [Registry] Support `parallel: true` in `Registry.dispatch/3`
+  * [Registry] Introduce `Registry.unregister_match/4`
+  * [Stream] Add `Stream.chunk_every/2` and `Stream.chunk_every/4` with a more explicit API than `Stream.chunk/2` and `Stream.chunk/4`
+  * [String] Optimise binary pattern matching in `String.split/1` and `String.trim_*/1`
+  * [Supervisor] Add `Supervisor.init/2` and `Supervisor.child_spec/2`
+  * [Supervisor] Allow `module` and `{module, arg}` to be given to `Supervisor.start_link/2` and invoke `module.child_spec(arg)` on each argument
+  * [Task] Support `:on_timeout` in `Task.async_stream` to control how tasks are terminated
+  * [Task] Add `ordered: false` support to `Task.async_stream`
 
 #### ExUnit
 
-  * [ExUnit.Diff] Use red or green background for whitespace-only diffs
-  * [ExUnit.Doctest] Allow inspected structures with multiples lines and unicode characters in the doctest result
-  * [ExUnit.Formatter] Replace lhs/rhs with left/right in the formatter for clarity
+  * [ExUnit] Show code snippet from test source file in case of test errors
+  * [ExUnit] Use `Exception.blame/3` when formatting test errors
+  * [ExUnit] Make `assert_raise/2` fail if the underlying exception has a broken `message/1` implementation
+  * [ExUnit] Add `start_supervised/2` and `stop_supervised/1` to ExUnit. Processes started by this function are automatically shut down when the test exits
 
 #### IEx
 
-  * [IEx.Autocomplete] Stop appending a trailing dot when autocompleting modules in IEx
-  * [IEx.Autocomplete] Support autocompletion for structs
-  * [IEx.Autocomplete] Improve IEx autocomplete to support navigating map atom keys
-  * [IEx.Helpers] `c/1` now compiles in memory by default to avoid common issue where `.beam` files remain at projects root directory
-  * [IEx.Helpers] Add info about protocols in `i/1`
-  * [IEx.Server] Support interrupting IEx evaluation through the Ctrl+G prompt
+  * [IEx.Autocomplete] Support autocompletion of variable names
+  * [IEx.Autocomplete] Support autocompletion of functions imported using `import Mod, only: [...]`
+  * [IEx.Evaluator] Use `Exception.blame/3` when showing errors in the terminal
+  * [IEx.Helpers] Add `exports/1` IEx helper to list all exports in a module
+  * [IEx.Helpers] Add `break!/2`, `break!/4`, `breaks/0`, `continue/0`, `open/0`, `remove_breaks/0`, `remove_breaks/1`, `reset_break/1`, `reset_break/3` and `whereami/1` for code debugging
+  * [IEx.Helpers] No longer emit warnings for IEx commands without parentheses
+  * [IEx.Helpers] Add `runtime_info/0` for printing runtime system information
+  * [IEx.Helpers] Add `open/1` to open the source of a given module/function in your editor
+  * [IEx.Info] Implement `IEx.Info` protocol for calendar types
+
+#### Logger
+
+  * [Logger] Add `metadata: :all` configuration to log all metadata
 
 #### Mix
 
-  * [mix archive] Compress archive files built by `mix archive` as they are now unzipped during installation
-  * [mix archive] Install from SCM
-  * [mix compile] Automatically infer the list of applications for Mix projects
-  * [mix cmd] Add the ability to specify one or more apps in `mix cmd`
-  * [mix deps] Warn if there are non-applications in the `apps` directory for umbrella projects
-  * [mix deps] Add warning for invalid paths on `mix deps.clean`
-  * [mix deps] Add `Mix.Project.apps_paths` that returns the paths to children applications in umbrella projects
-  * [mix deps] Add `MIX_REBAR` environment variable for overriding local rebar
-  * [mix escript] Install from SCM
-  * [mix new] Check directory existence in `mix new` and ask how to proceed if one exists
-  * [mix new] Applications built with the `--sup` flag now have an individual module to work as application callback
-  * [mix test] Add `--formatter` option to `mix test`
-  * [mix xref] Provide "did you mean?" suggestions for `mix xref`
+  * [mix compile.elixir] Add `--all-warnings` option to Elixir compiler that shows all warnings from the previous compilation (instead of just of the files being compiled)
+  * [mix escript.build] Strip debug information from escripts by default and add option `:strip_beam` which defaults to true
+  * [mix loadpaths] Ensure `--no-deps-check` do not trigger SCM callbacks (such as `git`)
+  * [mix local.hex] Add `--if-missing` flag to `local.hex` mix task
+  * [mix profile.cprof] Add `Mix.Tasks.Profile.Cprof` for count-based profiling
+  * [mix new] New styling for generated applications
 
 ### 2. Bug fixes
 
 #### Elixir
 
-  * [Access] Do not accept nils in `Access.key/1` and `Access.key/2` in favor of explicit default values (or Access.key!/1 if you expect the key to always be available)
-  * [Float] Avoid multiple roundings in `Float.ceil/2`, `Float.floor/2` and `Float.round/2`
-  * [Kernel] Don't crash in `macro_exported?/3` when dealing with Erlang modules
-  * [Kernel] Ensure locals calls are rewritten when calling a local function or macro from inside a module
-  * [Kernel] Annotate the context for variables as zero-arity funs in quotes
-  * [Kernel.SpecialForms] Ensure comprehensions with guards and filters keep proper ordering,
-  * [Kernel.SpecialForms] Produce meaningful warning when with's else clauses have no effect
-  * [Macro] Wrap fn calls in parens in `Macro.to_string/2`
-  * [Macro] Do not print aliases as keys inside keyword lists in `Macro.to_string/2`
-  * [OptionParser] Support options in `OptionParser.to_argv/2` to ensure `:count` switches are correctly encoded
-  * [Stream] Ensure `Stream.take/2` does not consume next element on `:suspend`
-  * [String] Fix infinite recursion in `String.replace_leading/3` and `String.replace_trailing/3` when given an empty string
-  * [Task] Fix `Task.shutdown/1,2` infinite block when task has no monitor
-  * [Task] Ensure task cannot link after parents unlinks
+  * [Calendar] Ensure `Calendar.ISO` raises a readable error when reaching up the year 10000 restriction
+  * [Calendar] Return `{:error, :invalid_time}` for wrong precision instead of crashing when parsing ISO dates
+  * [Enumerable] Raise `Protocol.UndefinedError` on bad functions in Enumerable implementation
+  * [File] Ensure recursive file operations raise on paths with null bytes (*security issue reported by Griffin Byatt*)
+  * [File] Support `:ram`/`:raw` files in `File.copy/2`
+  * [Inspect] Do not use colors when inspecting error messages
+  * [Kernel] Support guards on anonymous functions of zero arity
+  * [Kernel] Fix compilation of maps used as maps keys inside matches
+  * [Kernel] Ensure `do` clause in `with` is tail call optimizable
+  * [Module] `on_definition/6` callback receives body wrapped in a keyword list, such as `[do: body]`. This solves a bug where it was impossible to distinguish between a bodyless clause and a function that returns `nil`.
+  * [Path] Ensure recursive path operations raise on paths with null bytes (*security issue reported by Griffin Byatt*)
+  * [Protocol] Do not lose source compile info on protocol consolidation
+  * [Record] Properly escape quoted expressions passed to `defrecord`
+  * [Regex] Fix `inspect/2` for regexes with `/` terminator in them
+  * [Registry] Ensure `Registry.match/4` works with `:_` as key
+  * [Stream] Fix stream cycle over empty enumerable
+  * [String] Consider Unicode non-characters valid according to the specification in `String.valid?/1`
+  * [StringIO] Fix encoding and performance issues in `StringIO.get_until`
+  * [System] Raise on paths with null bytes in `System.cmd/2` and in `System.find_executable/1` (*security issue reported by Griffin Byatt*)
+  * [System] Raise on ill-formed environment variables (*security issue reported by Griffin Byatt*)
 
 #### ExUnit
 
-  * [ExUnit] Fix a race condition in `assert_receive` where we would assert a message was not received but show it in the list of messages when the message is delivered right after the timeout value
+  * [ExUnit] Properly account failed tests when `setup_all` fails
+  * [ExUnit] Having two or more `describe` blocks with the same name will now raise an error.
 
-### IEx
+#### IEx
 
-  * [IEx.Helpers] Purge consolidated protocols before and after `recompile/0`
+  * [IEx] Skip autocompletion of module names that are invalid without being quoted
+  * [IEx] Skip autocompletion of functions with default arguments with `@doc false`
+  * [IEx] Do not start oldshell alongside IEx
 
-### Mix
+#### Mix
 
-  * [Mix.Dep] Use `gmake` on FreeBSD instead of `make` when compiling make dependencies
-  * [Mix.Project] Only copy files from source when they're newer than destination (for Windows machines)
-  * [Mix.Task] Ensure non-recursive tasks inside umbrella are reenabled
+  * [mix compile.elixir] Store multiple sources in case of module conflicts. This solves an issue where `_build` would get corrupted when compiling Elixir projects with module conflicts
+  * [mix compile.erlang] Do not silently discard Erlang compile errors
+  * [mix compile.erlang] Properly track `-compile` module attribute when specified as a list
+  * [mix compile.protocols] Ensure protocol implementations do not "disappear" when switching between applications in umbrella projects by having separate consolidation paths per project
+  * [mix compile.protocols] Do not raise when consolidating a protocol that was converted into a module
 
 ### 3. Soft deprecations (no warnings emitted)
 
 #### Elixir
 
-  * [Enum] `Enum.partition/2` has been deprecated in favor of `Enum.split_with/2`
-  * [System] Deprecate plural time units in favor of singular ones to align with future Erlang releases
-
-#### ExUnit
-
-  * [ExUnit] Using GenEvent to implement ExUnit formatters is deprecated. Please use the new `GenServer` based formatters instead
+  * [Kernel] `not left in right` is soft-deprecated in favor of `left not in right`
 
 ### 4. Deprecations
 
 #### Elixir
 
-  * [Access] `Access.key/1` is deprecated due to erratic behaviour for missing keys, please use `Access.key/2` instead with proper default values
-  * [Behaviour] The `Behaviour` module is deprecated. Callbacks may now be defined directly via the `@callback` attribute
-  * [Enum] Deprecate `Enum.uniq/2` in favor of `Enum.uniq_by/2`
-  * [Float] `Float.to_char_list/2` and `Float.to_string/2` are deprecated (use the `:erlang.float_to_list/2` and `:erlang.float_to_binary/2` functions if such conversions are desired)
-  * [HashDict] The `HashDict` module is deprecated, in favour of the `Map`
-  * [HashSet] The `HashDict` module is deprecated, in favour of the `MapSet`  
-  * [Kernel] Deprecate support for making private functions overridable. Overridable functions must always be public as they must be contracts
-  * [Kernel] Anonymous functions with no expression after `->`, in favor of using an expression or returning `nil`
-  * [Kernel] Warn if variable is used as a function call
-  * [OptionParser] Deprecate aliases with multiple letters, such as `-abc`
-  * [Set] Deprecation of the `Set` module in favor of `MapSet`
-  * [Stream] Deprecate `Stream.uniq/2` in favor of `Stream.uniq_by/2`
+  * `Atom.to_char_list/1`, `Float.to_char_list/1`, `Integer.to_char_list/1`, `Integer.to_char_list/2`, `Kernel.to_char_list/1`, `List.Chars.to_char_list/1`, `String.to_char_list/1` have been deprecated in favor of their `to_charlist` version. This aligns with the naming conventions in both Erlang and Elixir
+  * [Enum] Deprecate `Enum.filter_map/3` in favor of `Enum.filter/2` + `Enum.map/2` or for-comprehensions
+  * [GenEvent] Deprecate `GenEvent` and provide alternatives in its docs
+  * [Kernel] Using `()` to mean `nil` is deprecated
+  * [Kernel] `:as_char_lists value` in `Inspect.Opts.t/0` type, in favor of `:as_charlists`
+  * [Kernel] `:char_lists` key in `Inspect.Opts.t/0` type, in favor of `:charlists`
+  * [Module] Using Erlang parse transforms via `@compile {:parse_transform, _}` is deprecated
+  * [Stream] Deprecate `Stream.filter_map/3` in favor of `Stream.filter/2` + `Stream.map/2`
+  * [String] `String.ljust/3` and `String.rjust/3` are deprecated in favor of `String.pad_leading/3` and `String.pad_trailing/3` with a binary padding
+  * [String] `String.strip/1` and `String.strip/2` are deprecated in favor of `String.trim/1` and `String.trim/2`
+  * [String] `String.lstrip/1` and `String.rstrip/1` are deprecated in favor of `String.trim_leading/1` and `String.trim_trailing/1`
+  * [String] `String.lstrip/2` and `String.rstrip/2` are deprecated in favor of `String.trim_leading/2` and `String.trim_trailing/2` with a binary as second argument
+  * [Typespec] `char_list/0` type is deprecated in favor of `charlist/0`
 
-#### IEx
+#### EEx
 
-  * [IEx.Helpers] `import_file/2` is deprecated in favor of `import_file_if_available/1`
+  * [EEx] Deprecate `<%= ` in "middle" and "end" expressions, e.g.: `<%= else %>` and `<%= end %>`
 
-#### Mix
+## v1.4
 
-  * [Mix.Utils] `underscore/1` and `camelize/1` are deprecated in favor of `Macro.underscore/1` and `Macro.camelize/1`
-
-## v1.3
-
-The CHANGELOG for v1.3 releases can be found [in the v1.3 branch](https://github.com/elixir-lang/elixir/blob/v1.3/CHANGELOG.md).
+The CHANGELOG for v1.4 releases can be found [in the v1.4 branch](https://github.com/elixir-lang/elixir/blob/v1.4/CHANGELOG.md).

ISSUE_TEMPLATE.md

@@ -7,8 +7,8 @@
 
 ### Environment
 
-* Elixir version (elixir -v):
-* Operating system:
+* Elixir & Erlang versions (elixir --version): 
+* Operating system: 
 
 ### Current behavior
 

Makefile

@@ -1,7 +1,7 @@
 REBAR ?= "$(CURDIR)/rebar"
 PREFIX ?= /usr/local
 SHARE_PREFIX ?= $(PREFIX)/share
-CANONICAL :=
+CANONICAL := v1.5/
 ELIXIRC := bin/elixirc --verbose --ignore-module-conflict
 ERLC := erlc -I lib/elixir/include
 ERL := erl -I lib/elixir/include -noshell -pa lib/elixir/ebin
@@ -16,7 +16,7 @@ INSTALL_PROGRAM = $(INSTALL) -m755
 GIT_REVISION = $(strip $(shell git rev-parse HEAD 2> /dev/null ))
 GIT_TAG = $(strip $(shell head="$(call GIT_REVISION)"; git tag --points-at $$head 2> /dev/null | tail -1) )
 
-.PHONY: install compile erlang elixir build_plt clean_plt dialyze test clean install_man clean_man docs Docs.zip Precompiled.zip zips
+.PHONY: install compile erlang elixir build_plt clean_plt dialyze test clean clean_residual_files install_man clean_man docs Docs.zip Precompiled.zip zips
 .NOTPARALLEL: compile
 
 #==> Functions
@@ -24,8 +24,8 @@ GIT_TAG = $(strip $(shell head="$(call GIT_REVISION)"; git tag --points-at $$hea
 define CHECK_ERLANG_RELEASE
 	$(Q) erl -noshell -eval '{V,_} = string:to_integer(erlang:system_info(otp_release)), io:fwrite("~s", [is_integer(V) and (V >= 18)])' -s erlang halt | grep -q '^true'; \
 		if [ $$? != 0 ]; then \
-		   echo "At least Erlang 18.0 is required to build Elixir"; \
-		   exit 1; \
+		  echo "At least Erlang 18.0 is required to build Elixir"; \
+		  exit 1; \
 		fi;
 endef
 
@@ -60,36 +60,38 @@ compile: lib/elixir/src/elixir.app.src erlang elixir
 
 lib/elixir/src/elixir.app.src: src/elixir.app.src
 	$(Q) $(call CHECK_ERLANG_RELEASE)
-	$(Q) rm -rf lib/elixir/src/elixir.app.src
+	$(Q) rm -f lib/elixir/src/elixir.app.src
 	$(Q) echo "%% This file is automatically generated from <project_root>/src/elixir.app.src" \
-	                             >lib/elixir/src/elixir.app.src
+	          >lib/elixir/src/elixir.app.src
 	$(Q) cat src/elixir.app.src >>lib/elixir/src/elixir.app.src
 
 erlang:
 	$(Q) cd lib/elixir && $(REBAR) compile
 
-# Since Mix depends on EEx and EEx depends on
-# Mix, we first compile EEx without the .app
-# file, then mix and then compile EEx fully
+# Since Mix depends on EEx and EEx depends on Mix,
+# we first compile EEx without the .app file,
+# then mix and then compile EEx fully
 elixir: stdlib lib/eex/ebin/Elixir.EEx.beam mix ex_unit logger eex iex
 
 stdlib: $(KERNEL) VERSION
 $(KERNEL): lib/elixir/lib/*.ex lib/elixir/lib/*/*.ex lib/elixir/lib/*/*/*.ex
-	$(Q) if [ ! -f $(KERNEL) ]; then                  \
-		echo "==> bootstrap (compile)";                 \
-		$(ERL) -s elixir_compiler core -s erlang halt;  \
+	$(Q) if [ ! -f $(KERNEL) ]; then \
+		echo "==> bootstrap (compile)"; \
+		$(ERL) -s elixir_compiler bootstrap -s erlang halt; \
 	fi
 	@ echo "==> elixir (compile)";
 	$(Q) cd lib/elixir && ../../$(ELIXIRC) "lib/kernel.ex" -o ebin;
 	$(Q) cd lib/elixir && ../../$(ELIXIRC) "lib/**/*.ex" -o ebin;
 	$(Q) $(MAKE) unicode
-	$(Q) rm -rf lib/elixir/ebin/elixir.app
+	$(Q) rm -f lib/elixir/ebin/elixir.app
 	$(Q) cd lib/elixir && $(REBAR) compile
 
 unicode: $(UNICODE)
 $(UNICODE): lib/elixir/unicode/*
 	@ echo "==> unicode (compile)";
-	$(Q) cd lib/elixir && ../../$(ELIXIRC) unicode/unicode.ex -o ebin;
+	$(Q) $(ELIXIRC) lib/elixir/unicode/unicode.ex -o lib/elixir/ebin;
+	$(Q) $(ELIXIRC) lib/elixir/unicode/properties.ex -o lib/elixir/ebin;
+	$(Q) $(ELIXIRC) lib/elixir/unicode/tokenizer.ex -o lib/elixir/ebin;
 
 $(eval $(call APP_TEMPLATE,ex_unit,ExUnit))
 $(eval $(call APP_TEMPLATE,logger,Logger))
@@ -100,15 +102,15 @@ $(eval $(call APP_TEMPLATE,iex,IEx))
 install: compile
 	@ echo "==> elixir (install)"
 	$(Q) for dir in lib/*; do \
-		rm -Rf $(DESTDIR)$(PREFIX)/$(LIBDIR)/elixir/$$dir/ebin; \
+		rm -rf $(DESTDIR)$(PREFIX)/$(LIBDIR)/elixir/$$dir/ebin; \
 		$(INSTALL_DIR) "$(DESTDIR)$(PREFIX)/$(LIBDIR)/elixir/$$dir/ebin"; \
 		$(INSTALL_DATA) $$dir/ebin/* "$(DESTDIR)$(PREFIX)/$(LIBDIR)/elixir/$$dir/ebin"; \
 	done
 	$(Q) $(INSTALL_DIR) "$(DESTDIR)$(PREFIX)/$(LIBDIR)/elixir/bin"
 	$(Q) $(INSTALL_PROGRAM) $(filter-out %.ps1, $(filter-out %.bat, $(wildcard bin/*))) "$(DESTDIR)$(PREFIX)/$(LIBDIR)/elixir/bin"
 	$(Q) $(INSTALL_DIR) "$(DESTDIR)$(PREFIX)/$(BINDIR)"
-	$(Q) for file in "$(DESTDIR)$(PREFIX)"/$(LIBDIR)/elixir/bin/* ; do \
-		ln -sf "../$(LIBDIR)/elixir/bin/$${file##*/}" "$(DESTDIR)$(PREFIX)/$(BINDIR)/" ; \
+	$(Q) for file in "$(DESTDIR)$(PREFIX)"/$(LIBDIR)/elixir/bin/*; do \
+		ln -sf "../$(LIBDIR)/elixir/bin/$${file##*/}" "$(DESTDIR)$(PREFIX)/$(BINDIR)/"; \
 	done
 	$(MAKE) install_man
 
@@ -116,18 +118,24 @@ clean:
 	cd lib/elixir && $(REBAR) clean
 	rm -rf ebin
 	rm -rf lib/*/ebin
-	rm -rf lib/elixir/test/ebin
-	rm -rf lib/*/tmp
-	rm -rf lib/mix/test/fixtures/git_repo
-	rm -rf lib/mix/test/fixtures/deps_on_git_repo
-	rm -rf lib/mix/test/fixtures/git_rebar
-	rm -rf lib/elixir/src/elixir.app.src
-	$(MAKE) clean_man
+	rm -f lib/elixir/src/elixir.app.src
+	$(Q) $(MAKE) clean_residual_files
 
-clean_exbeam:
+clean_elixir:
 	$(Q) rm -f lib/*/ebin/Elixir.*.beam
 
-#==> Create Documentation
+clean_residual_files:
+	rm -rf lib/*/_build/
+	rm -rf lib/*/tmp/
+	rm -rf lib/elixir/test/ebin/
+	rm -rf lib/mix/test/fixtures/deps_on_git_repo/
+	rm -rf lib/mix/test/fixtures/git_rebar/
+	rm -rf lib/mix/test/fixtures/git_repo/
+	rm -rf lib/mix/test/fixtures/git_sparse_repo/
+	rm -f erl_crash.dump
+	$(Q) $(MAKE) clean_man
+
+#==> Documentation tasks
 
 LOGO_PATH = $(shell test -f ../docs/logo.png && echo "--logo ../docs/logo.png")
 SOURCE_REF = $(shell tag="$(call GIT_TAG)" revision="$(call GIT_REVISION)"; echo "$${tag:-$$revision}\c")
@@ -139,7 +147,7 @@ docs: compile ../ex_doc/bin/ex_doc docs_elixir docs_eex docs_mix docs_iex docs_e
 docs_elixir: compile ../ex_doc/bin/ex_doc
 	@ echo "==> ex_doc (elixir)"
 	$(Q) rm -rf doc/elixir
-	$(call COMPILE_DOCS,Elixir,elixir,Kernel,-e "lib/elixir/pages/Behaviours.md" -e "lib/elixir/pages/Guards.md" -e "lib/elixir/pages/Naming Conventions.md" -e "lib/elixir/pages/Operators.md" -e "lib/elixir/pages/Typespecs.md" -e "lib/elixir/pages/Writing Documentation.md")
+	$(call COMPILE_DOCS,Elixir,elixir,Kernel,-e "lib/elixir/pages/Behaviours.md" -e "lib/elixir/pages/Deprecations.md" -e "lib/elixir/pages/Guards.md" -e "lib/elixir/pages/Naming Conventions.md" -e "lib/elixir/pages/Operators.md" -e "lib/elixir/pages/Syntax Reference.md" -e "lib/elixir/pages/Typespecs.md" -e "lib/elixir/pages/Unicode Syntax.md" -e "lib/elixir/pages/Writing Documentation.md")
 
 docs_eex: compile ../ex_doc/bin/ex_doc
 	@ echo "==> ex_doc (eex)"
@@ -170,24 +178,30 @@ docs_logger: compile ../ex_doc/bin/ex_doc
 	@ echo "ex_doc is not found in ../ex_doc as expected. See README for more information."
 	@ false
 
-#==> Zips
+#==> Zip tasks
 
 Docs.zip: docs
-	rm -rf Docs-v$(VERSION).zip
+	rm -f Docs-v$(VERSION).zip
 	zip -9 -r Docs-v$(VERSION).zip CHANGELOG.md doc NOTICE LICENSE README.md
 	@ echo "Docs file created $(CURDIR)/Docs-v$(VERSION).zip"
 
 Precompiled.zip: build_man compile
-	rm -rf Precompiled-v$(VERSION).zip
+	rm -f Precompiled-v$(VERSION).zip
 	zip -9 -r Precompiled-v$(VERSION).zip bin CHANGELOG.md lib/*/ebin LICENSE man NOTICE README.md VERSION
 	@ echo "Precompiled file created $(CURDIR)/Precompiled-v$(VERSION).zip"
 
 zips: Precompiled.zip Docs.zip
 
-#==> Tests tasks
+#==> Test tasks
 
 test: test_erlang test_elixir
 
+test_windows: test test_taskkill
+
+test_taskkill:
+	taskkill //IM erl.exe //F //T //FI "MEMUSAGE gt 0"
+	taskkill //IM epmd.exe //F //T //FI "MEMUSAGE gt 0"
+
 TEST_ERL = lib/elixir/test/erlang
 TEST_EBIN = lib/elixir/test/ebin
 TEST_ERLS = $(addprefix $(TEST_EBIN)/, $(addsuffix .beam, $(basename $(notdir $(wildcard $(TEST_ERL)/*.erl)))))
@@ -238,17 +252,19 @@ man/iex.1:
 	$(Q) cp man/iex.1.in man/iex.1
 	$(Q) sed -i.bak "/{COMMON}/r common" man/iex.1
 	$(Q) sed -i.bak "/{COMMON}/d" man/iex.1
-	$(Q) rm man/iex.1.bak
+	$(Q) rm -f man/iex.1.bak
 
 man/elixir.1:
 	$(Q) cp man/elixir.1.in man/elixir.1
 	$(Q) sed -i.bak "/{COMMON}/r common" man/elixir.1
 	$(Q) sed -i.bak "/{COMMON}/d" man/elixir.1
-	$(Q) rm man/elixir.1.bak
+	$(Q) rm -f man/elixir.1.bak
 
 clean_man:
 	rm -f man/elixir.1
+	rm -f man/elixir.1.bak
 	rm -f man/iex.1
+	rm -f man/iex.1.bak
 
 install_man: build_man
 	$(Q) mkdir -p $(DESTDIR)$(SHARE_PREFIX)/man/man1

README.md

@@ -1,7 +1,11 @@
 ![Elixir](https://github.com/elixir-lang/elixir-lang.github.com/raw/master/images/logo/logo.png)
 =========
-[![Build Status](https://secure.travis-ci.org/elixir-lang/elixir.svg?branch=master
+[![Travis build](https://secure.travis-ci.org/elixir-lang/elixir.svg?branch=master
 "Build Status")](https://travis-ci.org/elixir-lang/elixir)
+[![Windows build](https://ci.appveyor.com/api/projects/status/macwuxq7aiiv61g1?svg=true)](https://ci.appveyor.com/project/josevalim/elixir)
+
+
+Elixir is a dynamic, functional language designed for building scalable and maintainable applications.
 
 For more about Elixir, installation and documentation,
 [check Elixir's website](http://elixir-lang.org/).
@@ -62,7 +66,9 @@ code is divided in applications inside the `lib` folder:
 
 You can run all tests in the root directory with `make test` and you can
 also run tests for a specific framework `make test_#{NAME}`, for example,
-`make test_ex_unit`.
+`make test_ex_unit`. If you just changed something in the Elixir's standard
+library, you can run only that portion through `make test_stdlib`, as
+`test_elixir` also runs tests for the other projects (EEx, ExUnit, etc.).
 
 In case you are changing a single file, you can compile and run tests only
 for that particular file for fast development cycles. For example, if you
@@ -73,6 +79,28 @@ bin/elixirc lib/elixir/lib/string.ex -o lib/elixir/ebin
 bin/elixir lib/elixir/test/elixir/string_test.exs
 ```
 
+To recompile (including Erlang modules):
+
+```sh
+make compile
+```
+
+If your contribution fails the build during the bootstrapping of the language,
+you can reproduce it locally by deleting all of Elixir beam files and compiling
+again:
+
+```sh
+make clean_elixir compile
+```
+
+Or to rebuild everything from scratch without running tests:
+
+```sh
+make clean compile
+```
+
+More tasks can be found by reading the [Makefile](./Makefile).
+
 After your changes are done, please remember to run the full suite with
 `make test`.
 
@@ -126,7 +154,7 @@ cd ../elixir && make docs
 
 This will produce documentation sets for `elixir`, `mix`, etc., under
 the `doc` directory. If you are planning to contribute documentation,
-[please check our best practices for writing documentation](http://elixir-lang.org/docs/stable/elixir/writing-documentation.html).
+[please check our best practices for writing documentation](https://hexdocs.pm/elixir/writing-documentation.html).
 
 ## Development links
 

RELEASE.md

@@ -10,34 +10,26 @@ This document simply outlines the release process:
 
 3. Ensure CHANGELOG is updated and add current date
 
-4. Commit changes above with title "Release vVERSION" and generate new tag
+4. If a new `vMAJOR.MINOR`, replace "master" with "vVERSION" in the "Deprecations" page and commit
 
-5. Run `make clean test` to ensure all tests pass from scratch and the CI is green
+5. If a new `vMAJOR.MINOR`, create a new branch "vMAJOR.MINOR" and set `CANONICAL=` in Makefile
 
-6. Recompile an existing project (for example, Ecto) to ensure manifests can be upgraded
+6. Commit changes above with title "Release vVERSION" and generate new tag
 
-7. Push branch and the new tag
+7. Run `make clean test` to ensure all tests pass from scratch and the CI is green
 
-8. If a new `vMAJOR.MINOR`, create a new branch "vMAJOR.MINOR" and set `CANONICAL=` in Makefile before building docs
+8. Recompile an existing project (for example, Ecto) to ensure manifests can be upgraded
 
-9. Publish new zips with `make zips`, upload `Precompiled.zip` and `Docs.zip` to GitHub Releases
+9. Push branch and the new tag
 
-10. Add the release to `elixir.csv` and `_data/elixir-versions.yml` files in `elixir-lang/elixir-lang.github.com`
+10. Publish new zips with `make zips`, upload `Precompiled.zip` and `Docs.zip` to GitHub Releases
 
-11. After a new `vMAJOR.MINOR`, move back to master, bump versions, start new CHANGELOG, add `-dev` back and commit "Start vMAJOR.MINOR+1"
+11. Add the release to `elixir.csv` and `_data/elixir-versions.yml` files in `elixir-lang/elixir-lang.github.com`
+
+12. After a new `vMAJOR.MINOR`, move back to master, bump versions, start new CHANGELOG, add `-dev` back and commit "Start vMAJOR.MINOR+1"
 
 ## Places where version is mentioned
 
 * VERSION
 * CHANGELOG.md
 * src/elixir.app.src (not lib/elixir/src/elixir.app.src)
-
-## Deprecation policy
-
-Elixir deprecations happens in 3 steps:
-
-  1. The feature is soft-deprecated. It means both CHANGELOG and documentation must list the feature as deprecated but no warning is effectively emitted by running the code. There is no requirement to soft-deprecate a feature.
-
-  2. The feature is effectively deprecated by emitting warnings on usage. In order to deprecate a feature, the proposed alternative MUST exist for AT LEAST two versions. For example, `Enum.uniq/2` was soft-deprecated in favor of `Enum.uniq_by/2` in Elixir v1.1. This means a deprecation warning may only be emitted by Elixir v1.3 or later.
-
-  3. The feature is removed. This can only happen on major releases. This means deprecated features in Elixir v1.x shall only be removed by Elixir v2.x.

VERSION

@@ -1 +1 @@
-1.4.5
+1.5.3

bin/elixir

@@ -2,7 +2,6 @@
 if [ $# -eq 0 ] || [ "$1" = "--help" ] || [ "$1" = "-h" ]; then
   echo "Usage: `basename $0` [options] [.exs file] [data]
 
-  -v                          Prints version and exits
   -e COMMAND                  Evaluates the given command (*)
   -r FILE                     Requires the given files/patterns (*)
   -S SCRIPT                   Finds and executes the given script in PATH
@@ -14,12 +13,14 @@ if [ $# -eq 0 ] || [ "$1" = "--help" ] || [ "$1" = "-h" ]; then
   --cookie COOKIE             Sets a cookie for this distributed node
   --detached                  Starts the Erlang VM detached from console
   --erl SWITCHES              Switches to be passed down to Erlang (*)
+  --help, -h                  Prints this message and exits
   --hidden                    Makes a hidden node
   --logger-otp-reports BOOL   Enables or disables OTP reporting
   --logger-sasl-reports BOOL  Enables or disables SASL reporting
   --name NAME                 Makes and assigns a name to the distributed node
   --no-halt                   Does not halt the Erlang VM after execution
   --sname NAME                Makes and assigns a short name to the distributed node
+  --version, -v               Prints Elixir version and exits
   --werl                      Uses Erlang's Windows shell GUI (Windows only)
 
 ** Options marked with (*) can be given more than once
@@ -74,14 +75,14 @@ while [ $I -le $# ]; do
     --logger-otp-reports)
         I=$(expr $I + 1)
         eval "VAL=\${$I}"
-        if [ "$VAL" == 'true' ] || [ "$VAL" == 'false' ]; then
+        if [ "$VAL" = 'true' ] || [ "$VAL" = 'false' ]; then
             ERL="$ERL -logger handle_otp_reports "$VAL""
         fi
         ;;
     --logger-sasl-reports)
         I=$(expr $I + 1)
         eval "VAL=\${$I}"
-        if [ "$VAL" == 'true' ] || [ "$VAL" == 'false' ]; then
+        if [ "$VAL" = 'true' ] || [ "$VAL" = 'false' ]; then
             ERL="$ERL -logger handle_sasl_reports "$VAL""
         fi
         ;;

bin/elixir.bat

@@ -10,7 +10,6 @@ goto parseopts
 :documentation
 echo Usage: %~nx0 [options] [.exs file] [data]
 echo.
-echo   -v                          Prints version and exits
 echo   -e COMMAND                  Evaluates the given command (*)
 echo   -r FILE                     Requires the given files/patterns (*)
 echo   -S SCRIPT                   Finds and executes the given script in PATH
@@ -22,12 +21,14 @@ echo   --app APP                   Starts the given app and its dependencies (*)
 echo   --cookie COOKIE             Sets a cookie for this distributed node
 echo   --detached                  Starts the Erlang VM detached from console
 echo   --erl SWITCHES              Switches to be passed down to Erlang (*)
+echo   --help, -h                  Prints this message and exits
 echo   --hidden                    Makes a hidden node
 echo   --logger-otp-reports BOOL   Enables or disables OTP reporting
 echo   --logger-sasl-reports BOOL  Enables or disables SASL reporting
 echo   --name NAME                 Makes and assigns a name to the distributed node
 echo   --no-halt                   Does not halt the Erlang VM after execution
 echo   --sname NAME                Makes and assigns a short name to the distributed node
+echo   --version, -v               Prints Elixir version and exits
 echo   --werl                      Uses Erlang's Windows shell GUI
 echo.
 echo ** Options marked with (*) can be given more than once

bin/elixirc

@@ -2,12 +2,15 @@
 if [ $# -eq 0 ] || [ "$1" = "--help" ] || [ "$1" = "-h" ]; then
   echo "Usage: `basename $0` [elixir switches] [compiler switches] [.ex files]
 
-  -o                   The directory to output compiled files
-  --no-docs            Do not attach documentation to compiled modules
-  --no-debug-info      Do not attach debug info to compiled modules
-  --verbose            Print compilation status
-  --warnings-as-errors Treat warnings as errors and return non-zero exit code
-  --ignore-module-conflict
+  -o                        The directory to output compiled files
+
+  --help, -h                Prints this message and exits
+  --ignore-module-conflict  Does not emit warnings if a module was previously defined
+  --no-debug-info           Does not attach debug info to compiled modules
+  --no-docs                 Does not attach documentation to compiled modules
+  --verbose                 Prints compilation status
+  --version, -v             Prints Elixir version and exits
+  --warnings-as-errors      Treats warnings as errors and return non-zero exit code
 
 ** Options given after -- are passed down to the executed code
 ** Options can be passed to the Erlang runtime using ELIXIR_ERL_OPTIONS

bin/elixirc.bat

@@ -14,12 +14,15 @@ goto run
 :documentation
 echo Usage: %~nx0 [elixir switches] [compiler switches] [.ex files]
 echo.
-echo  -o                   The directory to output compiled files
-echo  --no-docs            Do not attach documentation to compiled modules
-echo  --no-debug-info      Do not attach debug info to compiled modules
-echo  --verbose            Print compilation status
-echo  --warnings-as-errors Treat warnings as errors and return non-zero exit code
-echo  --ignore-module-conflict
+echo   -o                        The directory to output compiled files
+echo.
+echo   --help, -h                Prints this message and exits
+echo   --ignore-module-conflict  Does not emit warnings if a module was previously defined
+echo   --no-debug-info           Does not attach debug info to compiled modules
+echo   --no-docs                 Does not attach documentation to compiled modules
+echo   --verbose                 Prints compilation status
+echo   --version, -v             Prints Elixir version and exits
+echo   --warnings-as-errors      Treats warnings as errors and returns non-zero exit code
 echo.
 echo ** Options given after -- are passed down to the executed code
 echo ** Options can be passed to the Erlang runtime using ELIXIR_ERL_OPTIONS

bin/iex

@@ -2,7 +2,6 @@
 if [ $# -gt 0 ] && ([ "$1" = "--help" ] || [ "$1" = "-h" ]); then
   echo "Usage: `basename $0` [options] [.exs file] [data]
 
-  -v                          Prints version and exits
   -e COMMAND                  Evaluates the given command (*)
   -r FILE                     Requires the given files/patterns (*)
   -S SCRIPT                   Finds and executes the given script in PATH
@@ -14,12 +13,14 @@ if [ $# -gt 0 ] && ([ "$1" = "--help" ] || [ "$1" = "-h" ]); then
   --cookie COOKIE             Sets a cookie for this distributed node
   --detached                  Starts the Erlang VM detached from console
   --erl SWITCHES              Switches to be passed down to Erlang (*)
+  --help, -h                  Prints this message and exits
   --hidden                    Makes a hidden node
   --logger-otp-reports BOOL   Enables or disables OTP reporting
   --logger-sasl-reports BOOL  Enables or disables SASL reporting
   --name NAME                 Makes and assigns a name to the distributed node
   --no-halt                   Does not halt the Erlang VM after execution
   --sname NAME                Makes and assigns a short name to the distributed node
+  --version, -v               Prints IEx version and exits
   --werl                      Uses Erlang's Windows shell GUI (Windows only)
 
   --dot-iex PATH              Overrides default .iex.exs file and uses path instead;
@@ -44,4 +45,4 @@ readlink_f () {
 
 SELF=$(readlink_f "$0")
 SCRIPT_PATH=$(dirname "$SELF")
-exec "$SCRIPT_PATH"/elixir --no-halt --erl "-user Elixir.IEx.CLI" +iex "$@"
+exec "$SCRIPT_PATH"/elixir --no-halt --erl "-noshell -user Elixir.IEx.CLI" +iex "$@"

bin/iex.bat

@@ -9,29 +9,30 @@ goto run
 :documentation
 echo Usage: %~nx0 [options] [.exs file] [data]
 echo.
-echo  -v                          Prints version and exits
-echo  -e COMMAND                  Evaluates the given command (*)
-echo  -r FILE                     Requires the given files/patterns (*)
-echo  -S SCRIPT                   Finds and executes the given script in PATH
-echo  -pr FILE                    Requires the given files/patterns in parallel (*)
-echo  -pa PATH                    Prepends the given path to Erlang code path (*)
-echo  -pz PATH                    Appends the given path to Erlang code path (*)
+echo   -e COMMAND                  Evaluates the given command (*)
+echo   -r FILE                     Requires the given files/patterns (*)
+echo   -S SCRIPT                   Finds and executes the given script in PATH
+echo   -pr FILE                    Requires the given files/patterns in parallel (*)
+echo   -pa PATH                    Prepends the given path to Erlang code path (*)
+echo   -pz PATH                    Appends the given path to Erlang code path (*)
 echo.
-echo  --app APP                   Starts the given app and its dependencies (*)
-echo  --cookie COOKIE             Sets a cookie for this distributed node
-echo  --detached                  Starts the Erlang VM detached from console
-echo  --erl SWITCHES              Switches to be passed down to Erlang (*)
-echo  --hidden                    Makes a hidden node
-echo  --logger-otp-reports BOOL   Enables or disables OTP reporting
-echo  --logger-sasl-reports BOOL  Enables or disables SASL reporting
-echo  --name NAME                 Makes and assigns a name to the distributed node
-echo  --no-halt                   Does not halt the Erlang VM after execution
-echo  --sname NAME                Makes and assigns a short name to the distributed node
-echo  --werl                      Uses Erlang's Windows shell GUI (Windows only)
+echo   --app APP                   Starts the given app and its dependencies (*)
+echo   --cookie COOKIE             Sets a cookie for this distributed node
+echo   --detached                  Starts the Erlang VM detached from console
+echo   --erl SWITCHES              Switches to be passed down to Erlang (*)
+echo   --help, -h                  Prints this message and exits
+echo   --hidden                    Makes a hidden node
+echo   --logger-otp-reports BOOL   Enables or disables OTP reporting
+echo   --logger-sasl-reports BOOL  Enables or disables SASL reporting
+echo   --name NAME                 Makes and assigns a name to the distributed node
+echo   --no-halt                   Does not halt the Erlang VM after execution
+echo   --sname NAME                Makes and assigns a short name to the distributed node
+echo   --version, -v               Prints IEx version and exits
+echo   --werl                      Uses Erlang's Windows shell GUI (Windows only)
 echo.
-echo  --dot-iex PATH              Overrides default .iex.exs file and uses path instead;
-echo                              path can be empty, then no file will be loaded
-echo  --remsh NAME                Connects to a node using a remote shell
+echo   --dot-iex PATH              Overrides default .iex.exs file and uses path instead;
+echo                               path can be empty, then no file will be loaded
+echo   --remsh NAME                Connects to a node using a remote shell
 echo.
 echo ** Options marked with (*) can be given more than once
 echo ** Options given after the .exs file or -- are passed down to the executed code
@@ -40,6 +41,6 @@ goto end
 
 :run
 @if defined IEX_WITH_WERL (@set __ELIXIR_IEX_FLAGS=--werl) else (set __ELIXIR_IEX_FLAGS=)
-call "%~dp0\elixir.bat" +iex --erl "-user Elixir.IEx.CLI" --no-halt %__ELIXIR_IEX_FLAGS% %*
+call "%~dp0\elixir.bat" --no-halt --erl "-noshell -user Elixir.IEx.CLI" +iex %__ELIXIR_IEX_FLAGS% %*
 :end
 endlocal

lib/eex/lib/eex.ex

@@ -166,7 +166,7 @@ defmodule EEx do
   Gets a string `source` and generate a quoted expression
   that can be evaluated by Elixir or compiled to a function.
   """
-  @spec compile_string(String.t, Keyword.t) :: Macro.t | no_return
+  @spec compile_string(String.t, keyword) :: Macro.t | no_return
   def compile_string(source, options \\ []) when is_binary(source) and is_list(options) do
     EEx.Compiler.compile(source, options)
   end
@@ -175,7 +175,7 @@ defmodule EEx do
   Gets a `filename` and generate a quoted expression
   that can be evaluated by Elixir or compiled to a function.
   """
-  @spec compile_file(String.t, Keyword.t) :: Macro.t | no_return
+  @spec compile_file(String.t, keyword) :: Macro.t | no_return
   def compile_file(filename, options \\ []) when is_binary(filename) and is_list(options) do
     options = Keyword.merge options, [file: filename, line: 1]
     compile_string(File.read!(filename), options)
@@ -190,7 +190,7 @@ defmodule EEx do
       "foo baz"
 
   """
-  @spec eval_string(String.t, Keyword.t, Keyword.t) :: any
+  @spec eval_string(String.t, keyword, keyword) :: any
   def eval_string(source, bindings \\ [], options \\ [])
       when is_binary(source) and is_list(bindings) and is_list(options) do
     compiled = compile_string(source, options)
@@ -209,7 +209,7 @@ defmodule EEx do
       EEx.eval_file "sample.eex", [bar: "baz"] #=> "foo baz"
 
   """
-  @spec eval_file(String.t, Keyword.t, Keyword.t) :: any
+  @spec eval_file(String.t, keyword, keyword) :: any
   def eval_file(filename, bindings \\ [], options \\ [])
       when is_binary(filename) and is_list(bindings) and is_list(options) do
     options  = Keyword.put options, :file, filename

lib/eex/lib/eex/compiler.ex

@@ -9,16 +9,17 @@ defmodule EEx.Compiler do
   and the engine together by handling the tokens and invoking
   the engine every time a full expression or text is received.
   """
-  @spec compile(String.t, Keyword.t) :: Macro.t | no_return
+  @spec compile(String.t, keyword) :: Macro.t | no_return
   def compile(source, opts) when is_binary(source) and is_list(opts) do
-    file   = opts[:file] || "nofile"
-    line   = opts[:line] || 1
-    trim   = opts[:trim] || false
+    file = opts[:file] || "nofile"
+    line = opts[:line] || 1
+    trim = opts[:trim] || false
     case EEx.Tokenizer.tokenize(source, line, trim: trim) do
       {:ok, tokens} ->
         state = %{engine: opts[:engine] || @default_engine,
                   file: file, line: line, quoted: [], start_line: nil}
-        generate_buffer(tokens, state.engine.init(opts), [], state)
+        init = state.engine.init(opts)
+        generate_buffer(tokens, init, [], state)
       {:error, line, message} ->
         raise EEx.SyntaxError, line: line, file: file, message: message
     end
@@ -27,39 +28,59 @@ defmodule EEx.Compiler do
   # Generates the buffers by handling each expression from the tokenizer.
   # It returns Macro.t/0 or it raises.
 
-  defp generate_buffer([{:text, chars} | t], buffer, scope, state) do
+  defp generate_buffer([{:text, chars} | rest], buffer, scope, state) do
     buffer = state.engine.handle_text(buffer, IO.chardata_to_string(chars))
-    generate_buffer(t, buffer, scope, state)
+    generate_buffer(rest, buffer, scope, state)
   end
 
-  defp generate_buffer([{:expr, line, mark, chars} | t], buffer, scope, state) do
+  defp generate_buffer([{:expr, line, mark, chars} | rest], buffer, scope, state) do
     expr = Code.string_to_quoted!(chars, [line: line, file: state.file])
     buffer = state.engine.handle_expr(buffer, IO.chardata_to_string(mark), expr)
-    generate_buffer(t, buffer, scope, state)
+    generate_buffer(rest, buffer, scope, state)
   end
 
-  defp generate_buffer([{:start_expr, start_line, mark, chars} | t], buffer, scope, state) do
-    {contents, line, t} = look_ahead_text(t, start_line, chars)
-    {contents, t} = generate_buffer(t, "", [contents | scope],
-                                    %{state | quoted: [], line: line, start_line: start_line})
+  defp generate_buffer([{:start_expr, start_line, mark, chars} | rest], buffer, scope, state) do
+    {contents, line, rest} = look_ahead_text(rest, start_line, chars)
+    {contents, rest} =
+      generate_buffer(rest, state.engine.handle_begin(buffer), [contents | scope],
+                      %{state | quoted: [], line: line, start_line: start_line})
     buffer = state.engine.handle_expr(buffer, IO.chardata_to_string(mark), contents)
-    generate_buffer(t, buffer, scope, state)
+    generate_buffer(rest, buffer, scope, state)
   end
 
-  defp generate_buffer([{:middle_expr, line, _, chars} | t], buffer, [current | scope], state) do
+  defp generate_buffer([{:middle_expr, line, '', chars} | rest], buffer, [current | scope], state) do
     {wrapped, state} = wrap_expr(current, line, buffer, chars, state)
-    generate_buffer(t, "", [wrapped | scope], %{state | line: line})
+    generate_buffer(rest, state.engine.handle_begin(buffer), [wrapped | scope], %{state | line: line})
   end
 
-  defp generate_buffer([{:end_expr, line, _, chars} | t], buffer, [current | _], state) do
+  defp generate_buffer([{:middle_expr, line, modifier, chars} | t], buffer, scope, state) do
+    message = "unexpected beginning of EEx tag \"<%#{modifier}\" on \"<%#{modifier}#{chars}%>\", " <>
+              "please remove \"#{modifier}\" accordingly"
+    :elixir_errors.warn line, state.file, message
+    generate_buffer([{:middle_expr, line, '', chars} | t], buffer, scope, state)
+    # TODO: Make this an error on Elixir v2.0 since it accidentally worked previously.
+    # raise EEx.SyntaxError, message: message, file: state.file, line: line
+  end
+
+  defp generate_buffer([{:end_expr, line, '', chars} | rest], buffer, [current | _], state) do
     {wrapped, state} = wrap_expr(current, line, buffer, chars, state)
     tuples = Code.string_to_quoted!(wrapped, [line: state.start_line, file: state.file])
     buffer = insert_quoted(tuples, state.quoted)
-    {buffer, t}
+    {buffer, rest}
+  end
+
+  defp generate_buffer([{:end_expr, line, modifier, chars} | t], buffer, [_ | _] = scope, state) do
+    message = "unexpected beginning of EEx tag \"<%#{modifier}\" on end of expression \"<%#{modifier}#{chars}%>\", " <>
+              "please remove \"#{modifier}\" accordingly"
+    :elixir_errors.warn line, state.file, message
+    generate_buffer([{:end_expr, line, '', chars} | t], buffer, scope, state)
+    # TODO: Make this an error on Elixir v2.0 since it accidentally worked previously.
+    # raise EEx.SyntaxError, message: message, file: state.file, line: line
   end
 
   defp generate_buffer([{:end_expr, line, _, chars} | _], _buffer, [], state) do
-    raise EEx.SyntaxError, message: "unexpected token #{inspect chars}", file: state.file, line: line
+    raise EEx.SyntaxError, message: "unexpected end of expression <%#{chars}%>",
+                           file: state.file, line: line
   end
 
   defp generate_buffer([], buffer, [], state) do
@@ -78,23 +99,23 @@ defmodule EEx.Compiler do
     key = length(state.quoted)
     placeholder = '__EEX__(' ++ Integer.to_charlist(key) ++ ');'
     {current ++ placeholder ++ new_lines ++ chars,
-     %{state | quoted: [{key, buffer} | state.quoted]}}
+     %{state | quoted: [{key, state.engine.handle_end(buffer)} | state.quoted]}}
   end
 
   # Look text ahead on expressions
 
-  defp look_ahead_text([{:text, text}, {:middle_expr, line, _, chars} | t]=list, start, contents) do
+  defp look_ahead_text([{:text, text}, {:middle_expr, line, _, chars} | rest] = tokens, start, contents) do
     if only_spaces?(text) do
-      {contents ++ text ++ chars, line, t}
+      {contents ++ text ++ chars, line, rest}
     else
-      {contents, start, list}
+      {contents, start, tokens}
     end
   end
-  defp look_ahead_text([{:middle_expr, line, _, chars} | t], _start, contents) do
-    {contents ++ chars, line, t}
+  defp look_ahead_text([{:middle_expr, line, _, chars} | rest], _start, contents) do
+    {contents ++ chars, line, rest}
   end
-  defp look_ahead_text(t, start, contents) do
-    {contents, start, t}
+  defp look_ahead_text(tokens, start, contents) do
+    {contents, start, tokens}
   end
 
   defp only_spaces?(chars) do

lib/eex/lib/eex/engine.ex

@@ -2,19 +2,26 @@ defmodule EEx.Engine do
   @moduledoc ~S"""
   Basic EEx engine that ships with Elixir.
 
-  An engine needs to implement four functions:
+  An engine needs to implement six functions:
 
-    * `init(opts)` - returns the initial buffer
+    * `init(opts)` - called at the beginning of every text
+      and it must return the initial state.
 
-    * `handle_body(quoted)` - receives the final built quoted
-      expression, should do final post-processing and return a
-      quoted expression.
+    * `handle_body(state)` - receives the state of the document
+      and it must return a quoted expression.
 
-    * `handle_text(buffer, text)` - it receives the buffer,
+    * `handle_text(state, text)` - it receives the state,
       the text and must return a new quoted expression.
 
-    * `handle_expr(buffer, marker, expr)` - it receives the buffer,
-      the marker, the expr and must return a new quoted expression.
+    * `handle_expr(state, marker, expr)` - it receives the state,
+      the marker, the expr and must return a new state.
+
+    * `handle_begin(state)` - called every time there a new state
+      is needed with an empty buffer. Typically called for do/end
+      blocks, case expressions, anonymous functions, etc
+
+    * `handle_end(state)` - opposite of `handle_begin(state)` and
+      it must return quoted expression
 
       The marker is what follows exactly after `<%`. For example,
       `<% foo %>` has an empty marker, but `<%= foo %>` has `"="`
@@ -27,10 +34,14 @@ defmodule EEx.Engine do
   default implementations for the functions above.
   """
 
-  @callback init(opts :: Keyword.t) :: Macro.t
-  @callback handle_body(quoted :: Macro.t) :: Macro.t
-  @callback handle_text(buffer :: Macro.t, text :: String.t) :: Macro.t
-  @callback handle_expr(buffer :: Macro.t, marker :: String.t, expr :: Macro.t) :: Macro.t
+  @type state :: term
+
+  @callback init(opts :: keyword) :: state
+  @callback handle_body(state) :: Macro.t
+  @callback handle_text(state, text :: String.t) :: state
+  @callback handle_expr(state, marker :: String.t, expr :: Macro.t) :: state
+  @callback handle_begin(state) :: state
+  @callback handle_end(state) :: Macro.t
 
   @doc false
   defmacro __using__(_) do
@@ -45,6 +56,14 @@ defmodule EEx.Engine do
         EEx.Engine.handle_body(quoted)
       end
 
+      def handle_begin(quoted) do
+        EEx.Engine.handle_begin(quoted)
+      end
+
+      def handle_end(quoted) do
+        EEx.Engine.handle_end(quoted)
+      end
+
       def handle_text(buffer, text) do
         EEx.Engine.handle_text(buffer, text)
       end
@@ -53,7 +72,7 @@ defmodule EEx.Engine do
         EEx.Engine.handle_expr(buffer, marker, expr)
       end
 
-      defoverridable [handle_body: 1, handle_expr: 3, handle_text: 2, init: 1]
+      defoverridable EEx.Engine
     end
   end
 
@@ -83,7 +102,7 @@ defmodule EEx.Engine do
 
   @doc false
   # TODO: Raise on 2.0
-  @spec fetch_assign!(map, Map.key) :: term | nil
+  @spec fetch_assign!(Access.t, Access.key) :: term | nil
   def fetch_assign!(assigns, key) do
     case Access.fetch(assigns, key) do
       {:ok, val} ->
@@ -104,6 +123,20 @@ defmodule EEx.Engine do
     ""
   end
 
+  @doc """
+  Returns an empty string as the new buffer.
+  """
+  def handle_begin(_previous) do
+    ""
+  end
+
+  @doc """
+  End of the new buffer.
+  """
+  def handle_end(quoted) do
+    quoted
+  end
+
   @doc """
   The default implementation simply returns the given expression.
   """

lib/eex/lib/eex/tokenizer.ex

@@ -19,7 +19,7 @@ defmodule EEx.Tokenizer do
 
   Or `{:error, line, error}` in case of errors.
   """
-  @spec tokenize(binary | charlist, line, Keyword.t) :: {:ok, [token]} | {:error, line, String.t}
+  @spec tokenize(binary | charlist, line, keyword) :: {:ok, [token]} | {:error, line, String.t}
   def tokenize(bin, line, opts \\ [])
 
   def tokenize(bin, line, opts)

lib/eex/mix.exs

@@ -2,8 +2,10 @@ defmodule EEx.Mixfile do
   use Mix.Project
 
   def project do
-    [app: :eex,
-     version: System.version,
-     build_per_environment: false]
+    [
+      app: :eex,
+      version: System.version,
+      build_per_environment: false
+    ]
   end
 end

lib/eex/test/eex/smart_engine_test.exs

@@ -1,9 +1,8 @@
 Code.require_file "../test_helper.exs", __DIR__
 
 defmodule EEx.SmartEngineTest do
-  use ExUnit.Case, async: true
-
-  import ExUnit.CaptureIO
+  # TODO: Make this async: true once capture_io is removed
+  use ExUnit.Case
 
   test "evaluates simple string" do
     assert_eval "foo bar", "foo bar"
@@ -18,7 +17,7 @@ defmodule EEx.SmartEngineTest do
   end
 
   test "error with missing assigns" do
-    stderr = capture_io(:stderr, fn ->
+    stderr = ExUnit.CaptureIO.capture_io(:stderr, fn ->
       assert_eval "", "<%= @foo %>", assigns: %{}
     end)
     assert stderr =~ "assign @foo not available in EEx template"
@@ -28,7 +27,7 @@ defmodule EEx.SmartEngineTest do
     assert_eval "1\n2\n3\n", "<%= for x <- [1, 2, 3] do %><%= x %>\n<% end %>"
   end
 
-  test "compiled preserved line numbers" do
+  test "preserves line numbers" do
     result = EEx.compile_string("<%= @hello %>", engine: EEx.SmartEngine)
     Macro.prewalk(result, fn
       {_left, meta, _right} ->
@@ -39,7 +38,7 @@ defmodule EEx.SmartEngineTest do
   end
 
   defp assert_eval(expected, actual, binding \\ []) do
-    result = EEx.eval_string(actual, binding, file: __ENV__.file)
+    result = EEx.eval_string(actual, binding, file: __ENV__.file, engine: EEx.SmartEngine)
     assert result == expected
   end
 end

lib/eex/test/eex_test.exs

@@ -53,381 +53,422 @@ defmodule EExTest do
   doctest EEx.Engine
   doctest EEx.SmartEngine
 
-  test "evaluates simple string" do
-    assert_eval "foo bar", "foo bar"
-  end
+  describe "evaluates" do
+    test "simple string" do
+      assert_eval "foo bar", "foo bar"
+    end
 
-  test "evaluates with embedded" do
-    assert_eval "foo bar", "foo <%= :bar %>"
-  end
+    test "Unicode" do
+      template = """
+        • <%= "•" %> •
+        <%= "Jößé Vâlìm" %> Jößé Vâlìm
+      """
+      assert_eval "  • • •\n  Jößé Vâlìm Jößé Vâlìm\n", template
+    end
 
-  test "evaluates with embedded and the binding" do
-    assert EEx.eval_string("foo <%= bar %>", [bar: 1]) == "foo 1"
-  end
+    test "trim mode" do
+      string = "<%= 123 %> \n456\n  <%= 789 %>"
+      expected = "123456\n789"
+      assert_eval expected, string, [], trim: true
+    end
 
-  test "evaluates with embedded do end" do
-    assert_eval "foo bar", "foo <%= if true do %>bar<% end %>"
-  end
+    test "trim mode with middle expression" do
+      string = """
+      <%= cond do %>
+      <% false -> %>
+        this
+      <% true -> %>
+        that
+      <% end %>
+      """
+      expected = "  that\n"
+      assert_eval expected, string, [], trim: true
+    end
 
-  test "evaluates with embedded do end and eval the expression" do
-    assert_eval "foo ", "foo <%= if false do %>bar<% end %>"
-  end
+    test "embedded code" do
+      assert_eval "foo bar", "foo <%= :bar %>"
+    end
 
-  test "evaluates with embedded do end and nested print expression" do
-    assert_eval "foo bar", "foo <%= if true do %><%= :bar %><% end %>"
-  end
+    test "embedded code with binding" do
+      assert EEx.eval_string("foo <%= bar %>", [bar: 1]) == "foo 1"
+    end
 
-  test "evaluates with embedded do end and nested expressions" do
-    assert_eval "foo bar baz", "foo <%= if true do %>bar <% Process.put(:eex_text, 1) %><%= :baz %><% end %>"
-    assert Process.get(:eex_text) == 1
-  end
+    test "embedded code with do end when true" do
+      assert_eval "foo bar", "foo <%= if true do %>bar<% end %>"
+    end
 
-  test "evaluates with embedded middle expression" do
-    assert_eval "foo bar", "foo <%= if true do %>bar<% else %>baz<% end %>"
-  end
+    test "embedded code with do end when false" do
+      assert_eval "foo ", "foo <%= if false do %>bar<% end %>"
+    end
 
-  test "evaluates with embedded middle expression and eval the expression" do
-    assert_eval "foo baz", "foo <%= if false do %>bar<% else %>baz<% end %>"
-  end
+    test "embedded code with do end and expression" do
+      assert_eval "foo bar", "foo <%= if true do %><%= :bar %><% end %>"
+    end
 
-  test "evaluates with nested start expression" do
-    assert_eval "foo bar", "foo <%= if true do %><%= if true do %>bar<% end %><% end %>"
-  end
+    test "embedded code with do end and multiple expressions" do
+      assert_eval "foo bar baz", "foo <%= if true do %>bar <% Process.put(:eex_text, 1) %><%= :baz %><% end %>"
+      assert Process.get(:eex_text) == 1
+    end
 
-  test "evaluates with nested middle expression" do
-    assert_eval "foo baz", "foo <%= if true do %><%= if false do %>bar<% else %>baz<% end %><% end %>"
-  end
+    test "embedded code with middle expression" do
+      assert_eval "foo bar", "foo <%= if true do %>bar<% else %>baz<% end %>"
+    end
 
-  test "evaluates with parentheses after end in end token" do
-    assert_eval " 101  102  103 ", "<%= Enum.map([1, 2, 3], (fn x -> %> <%= 100 + x %> <% end) ) %>"
-  end
+    test "embedded code with evaluated middle expression" do
+      assert_eval "foo baz", "foo <%= if false do %>bar<% else %>baz<% end %>"
+    end
 
-  test "evaluates with defined variable" do
-    assert_eval "foo 1", "foo <% bar = 1 %><%= bar %>"
-  end
+    test "embedded code with nested do end" do
+      assert_eval "foo bar", "foo <%= if true do %><%= if true do %>bar<% end %><% end %>"
+    end
 
-  test "evaluates with require code" do
-    assert_eval "foo 1,2,3", "foo <% require Enum, as: E %><%= E.join [1, 2, 3], \",\" %>"
-  end
+    test "embedded code with nested do end with middle expression" do
+      assert_eval "foo baz", "foo <%= if true do %><%= if false do %>bar<% else %>baz<% end %><% end %>"
+    end
 
-  test "evaluates with end of token" do
-    assert_eval "foo bar %>", "foo bar %>"
-  end
+    test "embedded code with parentheses after end in end token" do
+      assert_eval " 101  102  103 ", "<%= Enum.map([1, 2, 3], (fn x -> %> <%= 100 + x %> <% end) ) %>"
+    end
 
-  test "raises a syntax error when the token is invalid" do
-    assert_raise EEx.SyntaxError, "nofile:1: missing token '%>'", fn ->
-      EEx.compile_string "foo <%= bar"
+    test "embedded code with variable definition" do
+      assert_eval "foo 1", "foo <% bar = 1 %><%= bar %>"
+    end
+
+    test "embedded code with require" do
+      assert_eval "foo 1,2,3", "foo <% require Enum, as: E %><%= E.join [1, 2, 3], \",\" %>"
+    end
+
+    test "with end of token" do
+      assert_eval "foo bar %>", "foo bar %>"
     end
   end
 
-  test "raises a syntax error when end expression is found without a start expression" do
-    assert_raise EEx.SyntaxError, "nofile:1: unexpected token ' end '",  fn ->
-      EEx.compile_string "foo <% end %>"
+  describe "raises syntax errors" do
+    test "when the token is invalid" do
+      assert_raise EEx.SyntaxError, "nofile:1: missing token '%>'", fn ->
+        EEx.compile_string "foo <%= bar"
+      end
+    end
+
+    test "when end expression is found without a start expression" do
+      assert_raise EEx.SyntaxError, "nofile:1: unexpected end of expression <% end %>",  fn ->
+        EEx.compile_string "foo <% end %>"
+      end
+    end
+
+    test "when start expression is found without an end expression" do
+      assert_raise EEx.SyntaxError, "nofile:2: unexpected end of string, expected a closing '<% end %>'", fn ->
+        EEx.compile_string "foo\n<% if true do %>"
+      end
+    end
+
+    test "when nested end expression is found without a start expression" do
+      assert_raise EEx.SyntaxError, "nofile:1: unexpected end of expression <% end %>", fn ->
+        EEx.compile_string "foo <% if true do %><% end %><% end %>"
+      end
+    end
+
+    test "when middle expression has a modifier" do
+      assert ExUnit.CaptureIO.capture_io(:stderr, fn ->
+        EEx.compile_string "foo <%= if true do %>true<%= else %>false<% end %>"
+      end) =~ ~s[unexpected beginning of EEx tag \"<%=\" on \"<%= else %>\"]
+    end
+
+    test "when end expression has a modifier" do
+      assert ExUnit.CaptureIO.capture_io(:stderr, fn ->
+        EEx.compile_string "foo <%= if true do %>true<% else %>false<%= end %>"
+      end) =~ ~s[unexpected beginning of EEx tag \"<%=\" on end of expression \"<%= end %>\"]
     end
   end
 
-  test "raises a syntax error when start expression is found without an end expression" do
-    assert_raise EEx.SyntaxError, "nofile:2: unexpected end of string, expected a closing '<% end %>'", fn ->
-      EEx.compile_string "foo\n<% if true do %>"
+  describe "environment" do
+    test "respects line numbers" do
+      expected = """
+      foo
+      2
+      """
+
+      string = """
+      foo
+      <%= __ENV__.line %>
+      """
+
+      assert_eval expected, string
+    end
+
+    test "respects line numbers inside nested expressions" do
+      expected = """
+      foo
+
+      3
+
+      5
+      """
+
+      string = """
+      foo
+      <%= if true do %>
+      <%= __ENV__.line %>
+      <% end %>
+      <%= __ENV__.line %>
+      """
+
+      assert_eval expected, string
+    end
+
+    test "respects line numbers inside start expression" do
+      expected = """
+      foo
+
+      true
+
+      5
+      """
+
+      string = """
+      foo
+      <%= if __ENV__.line == 2 do %>
+      <%= true %>
+      <% end %>
+      <%= __ENV__.line %>
+      """
+
+      assert_eval expected, string
+    end
+
+    test "respects line numbers inside middle expression with ->" do
+      expected = """
+      foo
+
+      true
+
+      7
+      """
+
+      string = """
+      foo
+      <%= cond do %>
+      <% false -> %> false
+      <% __ENV__.line == 4 -> %>
+      <%= true %>
+      <% end %>
+      <%= __ENV__.line %>
+      """
+
+      assert_eval expected, string
+    end
+
+    test "respects line number inside middle expressions with keywords" do
+      expected = """
+      foo
+
+      5
+
+      7
+      """
+
+      string = """
+      foo
+      <%= if false do %>
+      <%= __ENV__.line %>
+      <% else %>
+      <%= __ENV__.line %>
+      <% end %>
+      <%= __ENV__.line %>
+      """
+
+      assert_eval expected, string
+    end
+
+    test "respects files" do
+      assert_eval "sample.ex", "<%= __ENV__.file %>", [], file: "sample.ex"
     end
   end
 
-  test "raises a syntax error when nested end expression is found without a start expression" do
-    assert_raise EEx.SyntaxError, "nofile:1: unexpected token ' end '", fn ->
-      EEx.compile_string "foo <% if true do %><% end %><% end %>"
+  describe "clauses" do
+    test "inside functions" do
+      expected = """
+
+      Number 1
+
+      Number 2
+
+      Number 3
+
+      """
+
+      string = """
+      <%= Enum.map [1, 2, 3], fn x -> %>
+      Number <%= x %>
+      <% end %>
+      """
+
+      assert_eval expected, string
+    end
+
+    test "inside cond" do
+      expected = """
+      foo
+
+      true
+
+      """
+
+      string = """
+      foo
+      <%= cond do %>
+      <% false -> %> false
+      <% fn -> 1 end -> %>
+      <%= true %>
+      <% end %>
+      """
+
+      assert_eval expected, string
+    end
+
+    test "inside cond with do end" do
+      string = """
+      <% y = ["a", "b", "c"] %>
+      <%= cond do %>
+       <% "a" in y -> %>
+        Good
+       <% true -> %>
+        <% if true do %>true<% else %>false<% end %>
+        Bad
+      <% end %>
+      """
+
+      assert_eval "\n\n  Good\n \n", string
     end
   end
 
-  test "respects line numbers" do
-    expected = """
-foo
-2
-"""
+  describe "buffers" do
+    test "unused buffers are kept out" do
+      string = """
+      <%= 123 %>
+      <% if true do %>
+        <%= 456 %>
+      <% end %>
+      <%= 789 %>
+      """
 
-    string = """
-foo
-<%= __ENV__.line %>
-"""
+      assert_eval "123\n\n789\n", string
+    end
 
-    assert_eval expected, string
-  end
-
-  test "respects line numbers inside nested expressions" do
-    expected = """
-foo
-
-3
-
-5
-"""
-
-    string = """
-foo
-<%= if true do %>
-<%= __ENV__.line %>
-<% end %>
-<%= __ENV__.line %>
-"""
-
-    assert_eval expected, string
-  end
-
-  test "respects line numbers inside start expression" do
-    expected = """
-foo
-
-true
-
-5
-"""
-
-    string = """
-foo
-<%= if __ENV__.line == 2 do %>
-<%= true %>
-<% end %>
-<%= __ENV__.line %>
-"""
-
-    assert_eval expected, string
-  end
-
-  test "respects line numbers inside middle expression with ->" do
-    expected = """
-foo
-
-true
-
-7
-"""
-
-    string = """
-foo
-<%= cond do %>
-<% false -> %> false
-<% __ENV__.line == 4 -> %>
-<%= true %>
-<% end %>
-<%= __ENV__.line %>
-"""
-
-    assert_eval expected, string
-  end
-
-  test "respects line number inside middle expressions with keywords" do
-    expected = """
-foo
-
-5
-
-7
-"""
-
-    string = """
-foo
-<%= if false do %>
-<%= __ENV__.line %>
-<% else %>
-<%= __ENV__.line %>
-<% end %>
-<%= __ENV__.line %>
-"""
-
-    assert_eval expected, string
-  end
-
-  test "respects files" do
-    assert_eval "sample.ex", "<%= __ENV__.file %>", [], file: "sample.ex"
-  end
-
-  test "properly handle functions" do
-    expected = """
-
-Number 1
-
-Number 2
-
-Number 3
-
-"""
-
-    string = """
-<%= Enum.map [1, 2, 3], fn x -> %>
-Number <%= x %>
-<% end %>
-"""
-
-    assert_eval expected, string
-  end
-
-  test "properly handle functions on the left side of clauses" do
-    expected = """
-foo
-
-true
-
-"""
-
-    string = """
-foo
-<%= cond do %>
-<% false -> %> false
-<% fn -> 1 end -> %>
-<%= true %>
-<% end %>
-"""
-
-    assert_eval expected, string
-  end
-
-  test "evaluates nested do expressions" do
-    string = """
-    <% y = ["a", "b", "c"] %>
-    <%= cond do %>
-     <% "a" in y -> %>
-      Good
-     <% true -> %>
-      <% if true do %>true<% else %>false<% end %>
-      Bad
-    <% end %>
-    """
-
-    assert_eval "\n\n  Good\n \n", string
-  end
-
-  test "evaluates expressions with buffers" do
-    string = """
-    <%= 123 %>
-    <% if true do %>
-      <%= 456 %>
-    <% end %>
-    <%= 789 %>
-    """
-
-    assert_eval "123\n\n789\n", string
-  end
-
-  test "for comprehensions" do
-    string = """
-    <%= for _name <- packages || [] do %>
-    <% end %>
-    <%= all || :done %>
-    """
-    assert_eval "\ndone\n", string, packages: nil, all: nil
-  end
-
-  test "Unicode" do
-    template = """
-      • <%= "•" %> •
-      <%= "Jößé Vâlìm" %> Jößé Vâlìm
-    """
-    result = EEx.eval_string(template)
-    assert result == "  • • •\n  Jößé Vâlìm Jößé Vâlìm\n"
-  end
-
-  test "trim mode" do
-    string = "<%= 123 %> \n456\n  <%= 789 %>"
-    expected = "123456\n789"
-    assert_eval expected, string, [], trim: true
-  end
-
-  test "trim mode with middle expression" do
-    string = """
-    <%= cond do %>
-    <% false -> %>
-      this
-    <% true -> %>
-      that
-    <% end %>
-    """
-    expected = "  that\n"
-    assert_eval expected, string, [], trim: true
-  end
-
-  test "evaluates the source from a given file" do
-    filename = Path.join(__DIR__, "fixtures/eex_template.eex")
-    result = EEx.eval_file(filename)
-    assert result == "foo bar.\n"
-  end
-
-  test "evaluates the source from a given file with bindings" do
-    filename = Path.join(__DIR__, "fixtures/eex_template_with_bindings.eex")
-    result = EEx.eval_file(filename, [bar: 1])
-    assert result == "foo 1\n"
-  end
-
-  test "raises an Exception when there's an error with the given file" do
-    assert_raise File.Error, "could not read file \"non-existent.eex\": no such file or directory", fn ->
-      filename = "non-existent.eex"
-      EEx.compile_file(filename)
+    test "inside comprehensions" do
+      string = """
+      <%= for _name <- packages || [] do %>
+      <% end %>
+      <%= all || :done %>
+      """
+      assert_eval "\ndone\n", string, packages: nil, all: nil
     end
   end
 
-  test "sets external resource attribute" do
-    assert EExTest.Compiled.__info__(:attributes)[:external_resource] ==
-           [Path.join(__DIR__, "fixtures/eex_template_with_bindings.eex")]
+  describe "from file" do
+    test "evaluates the source" do
+      filename = Path.join(__DIR__, "fixtures/eex_template.eex")
+      result = EEx.eval_file(filename)
+      assert_normalized_newline_equal "foo bar.\n", result
+    end
+
+    test "evaluates the source with bindings" do
+      filename = Path.join(__DIR__, "fixtures/eex_template_with_bindings.eex")
+      result = EEx.eval_file(filename, [bar: 1])
+      assert_normalized_newline_equal "foo 1\n", result
+    end
+
+    test "raises an Exception when file is missing" do
+      assert_raise File.Error, "could not read file \"non-existent.eex\": no such file or directory", fn ->
+        filename = "non-existent.eex"
+        EEx.compile_file(filename)
+      end
+    end
+
+    test "sets external resource attribute" do
+      assert EExTest.Compiled.__info__(:attributes)[:external_resource] ==
+             [Path.join(__DIR__, "fixtures/eex_template_with_bindings.eex")]
+    end
   end
 
-  test "defined from string" do
-    assert EExTest.Compiled.string_sample(1, 2) == "3"
-  end
+  describe "precompiled" do
 
-  test "defined from file" do
-    assert EExTest.Compiled.file_sample(1) == "foo 1\n"
-    assert EExTest.Compiled.public_file_sample(1) == "foo 1\n"
-  end
+    test "from string" do
+      assert EExTest.Compiled.string_sample(1, 2) == "3"
+    end
 
-  test "defined from file do not affect backtrace" do
-    assert EExTest.Compiled.before_compile ==
-      {8,
-        {EExTest.Compiled,
-          :before_compile,
-          0,
-          [file: to_charlist(Path.relative_to_cwd(__ENV__.file)), line: 7]
+    test "from file" do
+      assert_normalized_newline_equal "foo 1\n", EExTest.Compiled.file_sample(1)
+      assert_normalized_newline_equal "foo 1\n", EExTest.Compiled.public_file_sample(1)
+    end
+
+    test "from file does not affect backtrace" do
+      assert EExTest.Compiled.before_compile ==
+        {8,
+          {EExTest.Compiled,
+            :before_compile,
+            0,
+            [file: to_charlist(Path.relative_to_cwd(__ENV__.file)), line: 7]
+         }
        }
-     }
 
-    assert EExTest.Compiled.after_compile ==
-      {23,
-        {EExTest.Compiled,
-          :after_compile,
-          0,
-          [file: to_charlist(Path.relative_to_cwd(__ENV__.file)), line: 22]
+      assert EExTest.Compiled.after_compile ==
+        {23,
+          {EExTest.Compiled,
+            :after_compile,
+            0,
+            [file: to_charlist(Path.relative_to_cwd(__ENV__.file)), line: 22]
+         }
        }
-     }
 
-    assert EExTest.Compiled.unknown ==
-      {29,
-        {EExTest.Compiled,
-          :unknown,
-          0,
-          [file: 'unknown', line: 28]
+      assert EExTest.Compiled.unknown ==
+        {29,
+          {EExTest.Compiled,
+            :unknown,
+            0,
+            [file: 'unknown', line: 28]
+         }
        }
-     }
+    end
   end
 
   defmodule TestEngine do
     @behaviour EEx.Engine
 
     def init(_opts) do
-      ""
+      "INIT"
     end
 
     def handle_body(body) do
-      {:wrapped, body}
+      "BODY(#{body})"
+    end
+
+    def handle_begin(_) do
+      "BEGIN"
+    end
+
+    def handle_end(buffer) do
+      buffer <> ":END"
     end
 
     def handle_text(buffer, text) do
-      EEx.Engine.handle_text(buffer, text)
+      buffer <> ":TEXT(#{String.trim(text)})"
     end
 
-    def handle_expr(buffer, mark, expr) do
-      EEx.Engine.handle_expr(buffer, mark, expr)
+    def handle_expr(buffer, "=", expr) do
+      buffer <> ":EQUAL(#{Macro.to_string(expr)})"
     end
   end
 
-  test "calls handle_body" do
-    assert {:wrapped, "foo"} = EEx.eval_string("foo", [], engine: TestEngine)
+  describe "custom engines" do
+    test "text" do
+      assert_eval "BODY(INIT:TEXT(foo))", "foo", [], engine: TestEngine
+    end
+
+    test "begin/end" do
+      assert_eval ~s[BODY(INIT:TEXT(foo):EQUAL(if() do\n  "BEGIN:TEXT(this):END"\nelse\n  "BEGIN:TEXT(that):END"\nend))],
+                  "foo <%= if do %>this<% else %>that<% end %>", [], engine: TestEngine
+    end
   end
 
   defp assert_eval(expected, actual, binding \\ [], opts \\ []) do
@@ -435,4 +476,8 @@ foo
     result = EEx.eval_string(actual, binding, opts)
     assert result == expected
   end
+
+  defp assert_normalized_newline_equal(expected, actual) do
+    assert String.replace(expected, "\r\n", "\n") == String.replace(actual, "\r\n", "\n")
+  end
 end

lib/elixir/lib/access.ex

@@ -60,7 +60,7 @@ defmodule Access do
 
   ## Static lookups
 
-  The `Access` syntax (`foo[bar]`) cannot be used to access fields in
+  The `Access` syntax (`data[key]`) cannot be used to access fields in
   structs, since structs do not implement the `Access` behaviour by
   default. It is also a design decision: the dynamic access lookup
   is meant to be used for dynamic key-value structures, like maps
@@ -73,8 +73,7 @@ defmodule Access do
 
       user = %User{name: "John"}
       user[:name]
-      # ** (UndefinedFunctionError) undefined function User.fetch/2
-      #    (User does not implement the Access behaviour)
+      # ** (UndefinedFunctionError) undefined function User.fetch/2 (User does not implement the Access behaviour)
 
   Structs instead use the `user.name` syntax to access fields:
 
@@ -82,7 +81,7 @@ defmodule Access do
       #=> "John"
 
   The same `user.name` syntax can also be used by `Kernel.put_in/2`
-  to for updating structs fields:
+  for updating structs fields:
 
       put_in user.name, "Mary"
       #=> %User{name: "Mary"}
@@ -107,12 +106,14 @@ defmodule Access do
   functions for traversing other structures, like tuples and lists,
   to be used alongside `Kernel.put_in/2` in others.
 
-  For instance, given a user with a list of languages, here is how to
-  deeply traverse the map and convert all language names to uppercase:
+  For instance, given a user map with `:name` and `:languages` keys, here is how
+  to deeply traverse the map and convert all language names to uppercase:
 
-      iex> user = %{name: "john",
-      ...>          languages: [%{name: "elixir", type: :functional},
-      ...>                      %{name: "c", type: :procedural}]}
+      iex> languages = [
+      ...>   %{name: "elixir", type: :functional},
+      ...>   %{name: "c", type: :procedural},
+      ...> ]
+      iex> user = %{name: "john", languages: languages}
       iex> update_in user, [:languages, Access.all(), :name], &String.upcase/1
       %{name: "john",
         languages: [%{name: "ELIXIR", type: :functional},
@@ -123,7 +124,7 @@ defmodule Access do
 
   ## Implementing the Access behaviour for custom data structures
 
-  In order to be able to use the `Access` protocol with custom data structures
+  In order to be able to use the `Access` behaviour with custom data structures
   (which have to be structs), such structures have to implement the `Access`
   behaviour. For example, for a `User` struct, this would have to be done:
 
@@ -136,15 +137,30 @@ defmodule Access do
 
   """
 
-  @type t :: list | map | nil | any
+  @type container :: keyword | struct | map
+  @type nil_container :: nil
+  @type any_container :: any
+  @type t :: container | nil_container | any_container
   @type key :: any
   @type value :: any
 
+  @type get_fun(data, get_value) ::
+        (:get, data, (term -> term) ->
+        {get_value, new_data :: container})
+
+  @type get_and_update_fun(data, get_value) ::
+        (:get_and_update, data, (term -> term) ->
+        {get_value, new_data :: container} | :pop)
+
+  @type access_fun(data, get_value) ::
+        get_fun(data, get_value) | get_and_update_fun(data, get_value)
+
   @doc """
   Invoked in order to access the value stored under `key` in the given term `term`.
 
   This function should return `{:ok, value}` where `value` is the value under
-  `key` if it succeeded, or `:error` if the key does not exist in the structure.
+  `key` if the key exists in the term, or `:error` if the key does not exist in
+  the term.
 
   Many of the functions defined in the `Access` module internally call this
   function. This function is also used when the square-brackets access syntax
@@ -153,7 +169,6 @@ defmodule Access do
   value}` then `value` is returned, or if it returns `:error` then `nil` is
   returned.
 
-
   See the `Map.fetch/2` and `Keyword.fetch/2` implementations for examples of
   how to implement this callback.
   """
@@ -163,7 +178,7 @@ defmodule Access do
   Invoked in order to access the value stored under `key` in the given term `term`,
   defaulting to `default` if not present.
 
-  This function should return the value under the key `key` in `term` if there's
+  This function should return the value under `key` in `term` if there's
   such key, otherwise `default`.
 
   For most data structures, this can be implemented using `fetch/2` internally;
@@ -176,44 +191,47 @@ defmodule Access do
         end
       end
 
-  See the `Map.get/3` and `Keyword.get/3` implementations for more examples.
+  See the `Map.get/3` and `Keyword.get/3` implementations for examples of
+  how to implement this callback.
   """
   @callback get(term :: t, key, default :: value) :: value
 
   @doc """
   Invoked in order to access the value under `key` and update it at the same time.
 
-  The implementation of this callback should invoke the passed function with the
-  value under key `key` in the passed structure, or `nil` if the key is not
-  present. This function should return either `{value_to_return, new_value}` or
-  `:pop`.
+  The implementation of this callback should invoke `fun` with the value under
+  `key` in the passed structure `data`, or with `nil` if `key` is not present in it.
+  This function must return either `{get_value, update_value}` or `:pop`.
 
-  If it returns `{value_to_return, new_value}`, the return value of this
-  callback should be `{value_to_return, new_term}` where `new_term` is `term`
-  after updating the value of `key` with `new_value`.
+  If the passed function returns `{get_value, update_value}`,
+  the return value of this callback should be `{get_value, new_data}`, where:
+  - `get_value` is the retrieved value (which can be operated on before being returned)
+  - `update_value` is the new value to be stored under `key`
+  - `new_data` is `data` after updating the value of `key` with `update_value`.
 
-  If it returns `:pop`, the return value of this callback should be `{value,
-  new_term}` where `value` is the value under `key` or `nil` if not present, and
-  `new_term` is `term` without the key `key`.
+  If the passed function returns `:pop`, the return value of this callback
+  must be `{value, new_data}` where `value` is the value under `key`
+  (or `nil` if not present) and `new_data` is `data` without `key`.
 
   See the implementations of `Map.get_and_update/3` or `Keyword.get_and_update/3`
   for more examples.
   """
-  @callback get_and_update(term :: t, key, (value -> {value, value} | :pop)) :: {value, t}
+  @callback get_and_update(data, key, (value -> {get_value, value} | :pop)) ::
+            {get_value, data} when get_value: var, data: container | any_container
 
   @doc """
-  Invoked to "pop" the value under `key` out of the given term.
+  Invoked to "pop" the value under `key` out of the given data structure.
 
-  When the key `key` exists in the given `term`, the implementation should
-  return a `{value, new_term}` tuple where `value` is the value that was under
-  `key` and `new_term` is `term` without `key`.
+  When `key` exists in the given structure `data`, the implementation should
+  return a `{value, new_data}` tuple where `value` is the value that was under
+  `key` and `new_data` is `term` without `key`.
 
-  When the key `key` is not present in the given `term`, a tuple `{value, term}`
+  When `key` is not present in the given structure, a tuple `{value, data}`
   should be returned, where `value` is implementation-defined.
 
   See the implementations for `Map.pop/3` or `Keyword.pop/3` for more examples.
   """
-  @callback pop(term :: t, key) :: {value, t}
+  @callback pop(data, key) :: {value, data} when data: container | any_container
 
   defmacrop raise_undefined_behaviour(e, struct, top) do
     quote do
@@ -232,11 +250,15 @@ defmodule Access do
   @doc """
   Fetches the value for the given key in a container (a map, keyword
   list, or struct that implements the `Access` behaviour).
+
+  Returns `{:ok, value}` where `value` is the value under `key` if there is such
+  a key, or `:error` if `key` is not found.
   """
-  @spec fetch(t, term) :: {:ok, term} | :error
+  @spec fetch(container, term) :: {:ok, term} | :error
+  @spec fetch(nil_container, any) :: :error
   def fetch(container, key)
 
-  def fetch(%{__struct__: struct} = container, key) do
+  def fetch(%struct{} = container, key) do
     struct.fetch(container, key)
   rescue
     e in UndefinedFunctionError ->
@@ -244,11 +266,17 @@ defmodule Access do
   end
 
   def fetch(map, key) when is_map(map) do
-    Map.fetch(map, key)
+    case map do
+      %{^key => value} -> {:ok, value}
+      _ -> :error
+    end
   end
 
   def fetch(list, key) when is_list(list) and is_atom(key) do
-    Keyword.fetch(list, key)
+    case :lists.keyfind(key, 1, list) do
+      {_, value} -> {:ok, value}
+      false -> :error
+    end
   end
 
   def fetch(list, key) when is_list(list) do
@@ -263,31 +291,65 @@ defmodule Access do
   @doc """
   Gets the value for the given key in a container (a map, keyword
   list, or struct that implements the `Access` behaviour).
+
+  Returns the value under `key` if there is such a key, or `default` if `key` is
+  not found.
   """
-  @spec get(t, term, term) :: term
-  def get(container, key, default \\ nil) do
-    case fetch(container, key) do
+  @spec get(container, term, term) :: term
+  @spec get(nil_container, any, default) :: default when default: var
+  def get(container, key, default \\ nil)
+
+  def get(%{__struct__: struct} = container, key, default) do
+    try do
+      struct.fetch(container, key)
+    rescue
+      e in UndefinedFunctionError ->
+        raise_undefined_behaviour e, struct, {^struct, :fetch, [^container, ^key], _}
+    else
       {:ok, value} -> value
       :error -> default
     end
   end
 
-  @doc """
-  Gets and updates the given key in a container (a map, keyword
-  list, or struct that implements the `Access` behaviour).
+  def get(map, key, default) when is_map(map) do
+    case map do
+      %{^key => value} -> value
+      _ -> default
+    end
+  end
 
-  This `fun` argument receives the value of `key` (or `nil` if `key`
-  is not present) and must return a two-element tuple: the "get" value
-  (the retrieved value, which can be operated on before being returned)
-  and the new value to be stored under `key`. The `fun` may also
-  return `:pop`, implying the current value shall be removed
-  from the container and returned.
+  def get(list, key, default) when is_list(list) and is_atom(key) do
+    case :lists.keyfind(key, 1, list) do
+      {_, value} -> value
+      false -> default
+    end
+  end
+
+  def get(list, key, _default) when is_list(list) do
+    raise ArgumentError,
+      "the Access calls for keywords expect the key to be an atom, got: " <> inspect(key)
+  end
+
+  def get(nil, _key, default) do
+    default
+  end
+
+  @doc """
+  Gets and updates the given key in a `container` (a map, a keyword list,
+  a struct that implements the `Access` behaviour).
+
+  The `fun` argument receives the value of `key` (or `nil` if `key` is not
+  present in `container`) and must return a two-element tuple `{get_value, update_value}`:
+  the "get" value `get_value` (the retrieved value, which can be operated on before
+  being returned) and the new value to be stored under `key` (`update_value`).
+  `fun` may also return `:pop`, which means the current value
+  should be removed from the container and returned.
 
   The returned value is a two-element tuple with the "get" value returned by
   `fun` and a new container with the updated value under `key`.
   """
-  @spec get_and_update(container :: t, key, (value -> {get_value, update_value} | :pop)) ::
-        {get_value, container :: t} when get_value: var, update_value: value
+  @spec get_and_update(data, key, (value -> {get_value, value} | :pop)) ::
+        {get_value, data} when get_value: var, data: container
   def get_and_update(container, key, fun)
 
   def get_and_update(%{__struct__: struct} = container, key, fun) do
@@ -336,6 +398,7 @@ defmodule Access do
       {nil, %{creator: "Valim", name: "Elixir"}}
 
   """
+  @spec pop(data, key) :: {value, data} when data: container
   def pop(%{__struct__: struct} = container, key) do
     struct.pop(container, key)
   rescue
@@ -358,28 +421,6 @@ defmodule Access do
 
   ## Accessors
 
-  @doc false
-  def key(key) do
-    IO.warn "Access.key/1 is deprecated due to erratic behaviour for missing keys, " <>
-            "please use Access.key/2 instead with proper default values " <>
-            "(or Access.key!/1 if you expect the key to always be available)"
-
-    fn
-      :get, data, next ->
-        next.(Map.get(to_map(data), key))
-      :get_and_update, data, next ->
-        value = Map.get(to_map(data), key)
-        case next.(value) do
-          {get, update} -> {get, Map.put(data, key, update)}
-          :pop -> {value, Map.delete(data, key)}
-        end
-    end
-  end
-
-  defp to_map(nil), do: %{}
-  defp to_map(%{} = map), do: map
-  defp to_map(data), do: raise "Access.key/1 expected a map/struct or nil, got: #{inspect data}"
-
   @doc """
   Returns a function that accesses the given key in a map/struct.
 
@@ -389,13 +430,13 @@ defmodule Access do
   The returned function uses the default value if the key does not exist.
   This can be used to specify defaults and safely traverse missing keys:
 
-      iex> get_in(%{}, [Access.key(:user, %{}), Access.key(:name, nil)])
+      iex> get_in(%{}, [Access.key(:user, %{}), Access.key(:name)])
       nil
 
   Such is also useful when using update functions, allowing us to introduce
-  values as we traverse the data-structure for updates:
+  values as we traverse the data structure for updates:
 
-      iex> put_in(%{}, [Access.key(:user, %{}), Access.key(:name, nil)], "Mary")
+      iex> put_in(%{}, [Access.key(:user, %{}), Access.key(:name)], "Mary")
       %{user: %{name: "Mary"}}
 
   ## Examples
@@ -403,23 +444,24 @@ defmodule Access do
       iex> map = %{user: %{name: "john"}}
       iex> get_in(map, [Access.key(:unknown, %{}), Access.key(:name, "john")])
       "john"
-      iex> get_and_update_in(map, [Access.key!(:user), Access.key!(:name)], fn
+      iex> get_and_update_in(map, [Access.key(:user), Access.key(:name)], fn
       ...>   prev -> {prev, String.upcase(prev)}
       ...> end)
       {"john", %{user: %{name: "JOHN"}}}
-      iex> pop_in(map, [Access.key!(:user), Access.key!(:name)])
+      iex> pop_in(map, [Access.key(:user), Access.key(:name)])
       {"john", %{user: %{}}}
 
   An error is raised if the accessed structure is not a map or a struct:
 
-      iex> get_in(nil, [Access.key(:foo, nil)])
+      iex> get_in(nil, [Access.key(:foo)])
       ** (BadMapError) expected a map, got: nil
 
-      iex> get_in([], [Access.key(:foo, nil)])
+      iex> get_in([], [Access.key(:foo)])
       ** (BadMapError) expected a map, got: []
 
   """
-  def key(key, default) do
+  @spec key(key, term) :: access_fun(data :: struct | map, get_value :: term)
+  def key(key, default \\ nil) do
     fn
       :get, data, next ->
         next.(Map.get(data, key, default))
@@ -438,7 +480,7 @@ defmodule Access do
   The returned function is typically passed as an accessor to `Kernel.get_in/2`,
   `Kernel.get_and_update_in/3`, and friends.
 
-  Raises if the key does not exist.
+  The returned function raises if the key does not exist.
 
   ## Examples
 
@@ -460,6 +502,7 @@ defmodule Access do
       ** (RuntimeError) Access.key!/1 expected a map/struct, got: []
 
   """
+  @spec key!(key) :: access_fun(data :: struct | map, get_value :: term)
   def key!(key) do
     fn
       :get, %{} = data, next ->
@@ -481,7 +524,7 @@ defmodule Access do
   The returned function is typically passed as an accessor to `Kernel.get_in/2`,
   `Kernel.get_and_update_in/3`, and friends.
 
-  Raises if the index is out of bounds.
+  The returned function raises if `index` is out of bounds.
 
   ## Examples
 
@@ -501,6 +544,7 @@ defmodule Access do
       ** (RuntimeError) Access.elem/1 expected a tuple, got: %{}
 
   """
+  @spec elem(non_neg_integer) :: access_fun(data :: tuple, get_value :: term)
   def elem(index) when is_integer(index) do
     pos = index + 1
 
@@ -537,7 +581,7 @@ defmodule Access do
       {["john", "mary"], [%{}, %{}]}
 
   Here is an example that traverses the list dropping even
-  numbers and multipling odd numbers by 2:
+  numbers and multiplying odd numbers by 2:
 
       iex> require Integer
       iex> get_and_update_in([1, 2, 3, 4, 5], [Access.all], fn
@@ -551,6 +595,7 @@ defmodule Access do
       ** (RuntimeError) Access.all/0 expected a list, got: %{}
 
   """
+  @spec all() :: access_fun(data :: list, get_value :: list)
   def all() do
     &all/3
   end
@@ -560,7 +605,7 @@ defmodule Access do
   end
 
   defp all(:get_and_update, data, next) when is_list(data) do
-    all(data, next, [], [])
+    all(data, next, _gets = [], _updates = [])
   end
 
   defp all(_op, data, _next) do
@@ -622,8 +667,10 @@ defmodule Access do
 
       iex> get_in(%{}, [Access.at(1)])
       ** (RuntimeError) Access.at/1 expected a list, got: %{}
+
   """
-  def at(index) when index >= 0 do
+  @spec at(non_neg_integer) :: access_fun(data :: list, get_value :: term)
+  def at(index) when is_integer(index) and index >= 0 do
     fn(op, data, next) -> at(op, data, index, next) end
   end
 

lib/elixir/lib/agent.ex

@@ -6,16 +6,18 @@ defmodule Agent do
   must be accessed from different processes or by the same process
   at different points in time.
 
-  The Agent module provides a basic server implementation that
+  The `Agent` module provides a basic server implementation that
   allows state to be retrieved and updated via a simple API.
 
   ## Examples
 
   For example, in the Mix tool that ships with Elixir, we need
   to keep a set of all tasks executed by a given project. Since
-  this set is shared, we can implement it with an Agent:
+  this set is shared, we can implement it with an agent:
 
       defmodule Mix.TasksServer do
+        use Agent
+
         def start_link do
           Agent.start_link(fn -> MapSet.new end, name: __MODULE__)
         end
@@ -42,12 +44,11 @@ defmodule Agent do
         end
       end
 
-  Note that agents still provide a segregation between the
-  client and server APIs, as seen in GenServers. In particular,
-  all code inside the function passed to the agent is executed
-  by the agent. This distinction is important because you may
-  want to avoid expensive operations inside the agent, as it will
-  effectively block the agent until the request is fulfilled.
+  Agents provide a segregation between the client and server APIs (similar
+  to GenServers). In particular, the anonymous functions given to the `Agent`
+  are executed inside the agent (the server). This distinction is important
+  because you may want to avoid expensive operations inside the agent,
+  as they will effectively block the agent until the request is fulfilled.
 
   Consider these two examples:
 
@@ -61,16 +62,36 @@ defmodule Agent do
         Agent.get(agent, &(&1)) |> do_something_expensive()
       end
 
-  The first function blocks the agent. The second function copies
-  all the state to the client and then executes the operation in the
-  client. The difference is whether the data is large enough to require
-  processing in the server, at least initially, or small enough to be
-  sent to the client cheaply.
+  The first function blocks the agent. The second function copies all the state
+  to the client and then executes the operation in the client. One aspect to
+  consider is whether the data is large enough to require processing in the server,
+  at least initially, or small enough to be sent to the client cheaply. Another
+  factor is whether the data needs to be processed atomically: getting the
+  state and calling `do_something_expensive(state)` outside of the agent means
+  that the agent's state can be updated in the meantime. This is specially
+  important in case of updates as computing the new state in the client rather
+  than in the server can lead to race conditions if multiple clients are trying
+  to update the same state to different values.
 
-  ## Name Registration
+  Finally note `use Agent` defines a `child_spec/1` function, allowing the
+  defined module to be put under a supervision tree. The generated
+  `child_spec/1` can be customized with the following options:
 
-  An Agent is bound to the same name registration rules as GenServers.
-  Read more about it in the `GenServer` docs.
+    * `:id` - the child specification id, defauts to the current module
+    * `:start` - how to start the child process (defaults to calling `__MODULE__.start_link/1`)
+    * `:restart` - when the child should be restarted, defaults to `:permanent`
+    * `:shutdown` - how to shut down the child
+
+  For example:
+
+      use Agent, restart: :transient, shutdown: 10_000
+
+  See the `Supervisor` docs for more information.
+
+  ## Name registration
+
+  An agent is bound to the same name registration rules as GenServers.
+  Read more about it in the `GenServer` documentation.
 
   ## A word on distributed agents
 
@@ -95,14 +116,15 @@ defmodule Agent do
   ## Hot code swapping
 
   An agent can have its code hot swapped live by simply passing a module,
-  function, and args tuple to the update instruction. For example, imagine
+  function, and arguments tuple to the update instruction. For example, imagine
   you have an agent named `:sample` and you want to convert its inner state
-  from some dict structure to a map. It can be done with the following
+  from a keyword list to a map. It can be done with the following
   instruction:
 
       {:update, :sample, {:advanced, {Enum, :into, [%{}]}}}
 
-  The agent's state will be added to the given list as the first argument.
+  The agent's state will be added to the given list of arguments (`[%{}]`) as
+  the first argument.
   """
 
   @typedoc "Return values of `start*` functions"
@@ -117,13 +139,41 @@ defmodule Agent do
   @typedoc "The agent state"
   @type state :: term
 
+  @doc false
+  def child_spec(arg) do
+    %{
+      id: Agent,
+      start: {Agent, :start_link, [arg]}
+    }
+  end
+
+  @doc false
+  defmacro __using__(opts) do
+    quote location: :keep, bind_quoted: [opts: opts] do
+      spec = [
+        id: opts[:id] || __MODULE__,
+        start: Macro.escape(opts[:start]) || quote(do: {__MODULE__, :start_link, [arg]}),
+        restart: opts[:restart] || :permanent,
+        shutdown: opts[:shutdown] || 5000,
+        type: :worker
+      ]
+
+      @doc false
+      def child_spec(arg) do
+        %{unquote_splicing(spec)}
+      end
+
+      defoverridable child_spec: 1
+    end
+  end
+
   @doc """
   Starts an agent linked to the current process with the given function.
 
   This is often used to start the agent as part of a supervision tree.
 
-  Once the agent is spawned, the given function is invoked and its return
-  value is used as the agent state. Note that `start_link` does not return
+  Once the agent is spawned, the given function `fun` is invoked and its return
+  value is used as the agent state. Note that `start_link/2` does not return
   until the given function has returned.
 
   ## Options
@@ -148,8 +198,18 @@ defmodule Agent do
   specified name already exists, the function returns
   `{:error, {:already_started, pid}}` with the PID of that process.
 
-  If the given function callback fails with `reason`, the function returns
-  `{:error, reason}`.
+  If the given function callback fails, the function returns `{:error, reason}`.
+
+  ## Examples
+
+      iex> {:ok, pid} = Agent.start_link(fn -> 42 end)
+      iex> Agent.get(pid, fn state -> state end)
+      42
+
+      iex> {:error, {exception, _stacktrace}} = Agent.start(fn -> raise "oops" end)
+      iex> exception
+      %RuntimeError{message: "oops"}
+
   """
   @spec start_link((() -> term), GenServer.options) :: on_start
   def start_link(fun, options \\ []) when is_function(fun, 0) do
@@ -157,11 +217,11 @@ defmodule Agent do
   end
 
   @doc """
-  Starts an agent linked to the current process with the given module
-  function and arguments.
+  Starts an agent linked to the current process.
 
-  Same as `start_link/2` but a module, function and args are expected
-  instead of an anonymous function.
+  Same as `start_link/2` but a module, function, and arguments are expected
+  instead of an anonymous function; `fun` in `module` will be called with the
+  given arguments `args` to initialize the state.
   """
   @spec start_link(module, atom, [any], GenServer.options) :: on_start
   def start_link(module, fun, args, options \\ []) do
@@ -172,6 +232,13 @@ defmodule Agent do
   Starts an agent process without links (outside of a supervision tree).
 
   See `start_link/2` for more information.
+
+  ## Examples
+
+      iex> {:ok, pid} = Agent.start(fn -> 42 end)
+      iex> Agent.get(pid, fn(state) -> state end)
+      42
+
   """
   @spec start((() -> term), GenServer.options) :: on_start
   def start(fun, options \\ []) when is_function(fun, 0) do
@@ -179,10 +246,9 @@ defmodule Agent do
   end
 
   @doc """
-  Starts an agent with the given module function and arguments.
+  Starts an agent without links with the given module, function, and arguments.
 
-  Similar to `start/2` but a module, function and args are expected
-  instead of an anonymous function.
+  See `start_link/4` for more information.
   """
   @spec start(module, atom, [any], GenServer.options) :: on_start
   def start(module, fun, args, options \\ []) do
@@ -190,13 +256,24 @@ defmodule Agent do
   end
 
   @doc """
-  Gets an agent value via the given function.
+  Gets an agent value via the given anonymous function.
 
   The function `fun` is sent to the `agent` which invokes the function
   passing the agent state. The result of the function invocation is
-  returned.
+  returned from this function.
+
+  `timeout` is an integer greater than zero which specifies how many
+  milliseconds are allowed before the agent executes the function and returns
+  the result value, or the atom `:infinity` to wait indefinitely. If no result
+  is received within the specified time, the function call fails and the caller
+  exits.
+
+  ## Examples
+
+      iex> {:ok, pid} = Agent.start_link(fn -> 42 end)
+      iex> Agent.get(pid, fn state -> state end)
+      42
 
-  A timeout can also be specified (it has a default value of 5000).
   """
   @spec get(agent, (state -> a), timeout) :: a when a: var
   def get(agent, fun, timeout \\ 5000) when is_function(fun, 1) do
@@ -206,9 +283,9 @@ defmodule Agent do
   @doc """
   Gets an agent value via the given function.
 
-  Same as `get/3` but a module, function and args are expected
+  Same as `get/3` but a module, function, and arguments are expected
   instead of an anonymous function. The state is added as first
-  argument to the given list of args.
+  argument to the given list of arguments.
   """
   @spec get(agent, module, atom, [term], timeout) :: any
   def get(agent, module, fun, args, timeout \\ 5000) do
@@ -216,14 +293,28 @@ defmodule Agent do
   end
 
   @doc """
-  Gets and updates the agent state in one operation.
+  Gets and updates the agent state in one operation via the given anonymous
+  function.
 
   The function `fun` is sent to the `agent` which invokes the function
   passing the agent state. The function must return a tuple with two
-  elements, the first being the value to return (i.e. the `get` value)
-  and the second one is the new state.
+  elements, the first being the value to return (that is, the "get" value)
+  and the second one being the new state of the agent.
+
+  `timeout` is an integer greater than zero which specifies how many
+  milliseconds are allowed before the agent executes the function and returns
+  the result value, or the atom `:infinity` to wait indefinitely. If no result
+  is received within the specified time, the function call fails and the caller
+  exits.
+
+  ## Examples
+
+      iex> {:ok, pid} = Agent.start_link(fn -> 42 end)
+      iex> Agent.get_and_update(pid, fn state -> {state, state + 1} end)
+      42
+      iex> Agent.get(pid, fn state -> state end)
+      43
 
-  A timeout can also be specified (it has a default value of 5000).
   """
   @spec get_and_update(agent, (state -> {a, state}), timeout) :: a when a: var
   def get_and_update(agent, fun, timeout \\ 5000) when is_function(fun, 1) do
@@ -231,11 +322,11 @@ defmodule Agent do
   end
 
   @doc """
-  Gets and updates the agent state in one operation.
+  Gets and updates the agent state in one operation via the given function.
 
-  Same as `get_and_update/3` but a module, function and args are expected
+  Same as `get_and_update/3` but a module, function, and arguments are expected
   instead of an anonymous function. The state is added as first
-  argument to the given list of args.
+  argument to the given list of arguments.
   """
   @spec get_and_update(agent, module, atom, [term], timeout) :: any
   def get_and_update(agent, module, fun, args, timeout \\ 5000) do
@@ -243,13 +334,28 @@ defmodule Agent do
   end
 
   @doc """
-  Updates the agent state.
+  Updates the agent state via the given anonymous function.
 
   The function `fun` is sent to the `agent` which invokes the function
-  passing the agent state. The function must return the new state.
+  passing the agent state. The return value of `fun` becomes the new
+  state of the agent.
 
-  A timeout can also be specified (it has a default value of 5000).
   This function always returns `:ok`.
+
+  `timeout` is an integer greater than zero which specifies how many
+  milliseconds are allowed before the agent executes the function and returns
+  the result value, or the atom `:infinity` to wait indefinitely. If no result
+  is received within the specified time, the function call fails and the caller
+  exits.
+
+  ## Examples
+
+      iex> {:ok, pid} = Agent.start_link(fn -> 42 end)
+      iex> Agent.update(pid, fn state -> state + 1 end)
+      :ok
+      iex> Agent.get(pid, fn state -> state end)
+      43
+
   """
   @spec update(agent, (state -> state), timeout) :: :ok
   def update(agent, fun, timeout \\ 5000) when is_function(fun, 1) do
@@ -257,11 +363,11 @@ defmodule Agent do
   end
 
   @doc """
-  Updates the agent state.
+  Updates the agent state via the given function.
 
-  Same as `update/3` but a module, function and args are expected
+  Same as `update/3` but a module, function, and arguments are expected
   instead of an anonymous function. The state is added as first
-  argument to the given list of args.
+  argument to the given list of arguments.
   """
   @spec update(agent, module, atom, [term], timeout) :: :ok
   def update(agent, module, fun, args, timeout \\ 5000) do
@@ -269,13 +375,14 @@ defmodule Agent do
   end
 
   @doc """
-  Performs a cast (fire and forget) operation on the agent state.
+  Performs a cast (*fire and forget*) operation on the agent state.
 
   The function `fun` is sent to the `agent` which invokes the function
-  passing the agent state. The function must return the new state.
+  passing the agent state. The return value of `fun` becomes the new
+  state of the agent.
 
-  Note that `cast` returns `:ok` immediately, regardless of whether the
-  destination node or agent exists.
+  Note that `cast` returns `:ok` immediately, regardless of whether `agent` (or
+  the node it should live on) exists.
   """
   @spec cast(agent, (state -> state)) :: :ok
   def cast(agent, fun) when is_function(fun, 1) do
@@ -283,11 +390,11 @@ defmodule Agent do
   end
 
   @doc """
-  Performs a cast (fire and forget) operation on the agent state.
+  Performs a cast (*fire and forget*) operation on the agent state.
 
-  Same as `cast/2` but a module, function and args are expected
+  Same as `cast/2` but a module, function, and arguments are expected
   instead of an anonymous function. The state is added as first
-  argument to the given list of args.
+  argument to the given list of arguments.
   """
   @spec cast(agent, module, atom, [term]) :: :ok
   def cast(agent, module, fun, args) do
@@ -295,18 +402,25 @@ defmodule Agent do
   end
 
   @doc """
-  Stops the agent with the given `reason`.
+  Synchronously stops the agent with the given `reason`.
 
-  It returns `:ok` if the server terminates with the given
-  reason, if it terminates with another reason, the call will
+  It returns `:ok` if the agent terminates with the given
+  reason. If the agent terminates with another reason, the call will
   exit.
 
   This function keeps OTP semantics regarding error reporting.
   If the reason is any other than `:normal`, `:shutdown` or
   `{:shutdown, _}`, an error report will be logged.
+
+  ## Examples
+
+      iex> {:ok, pid} = Agent.start_link(fn -> 42 end)
+      iex> Agent.stop(pid)
+      :ok
+
   """
   @spec stop(agent, reason :: term, timeout) :: :ok
   def stop(agent, reason \\ :normal, timeout \\ :infinity) do
-    :gen.stop(agent, reason, timeout)
+    GenServer.stop(agent, reason, timeout)
   end
 end

lib/elixir/lib/application.ex

@@ -9,17 +9,57 @@ defmodule Application do
   Applications are defined with an application file named `APP.app` where
   `APP` is the application name, usually in `underscore_case`. The application
   file must reside in the same `ebin` directory as the compiled modules of the
-  application.
+  application. In Elixir, the Mix build tool is responsible for compiling your
+  source code and generating your application `.app` file. You can learn more
+  about the generation of `.app` files by typing `mix help compile.app`.
 
-  In Elixir, Mix is responsible for compiling your source code and
-  generating your application `.app` file. Furthermore, Mix is also
-  responsible for configuring, starting and stopping your application
-  and its dependencies. For this reason, this documentation will focus
-  on the remaining aspects of your application: the application environment
-  and the application callback module.
+  Once your application is compiled, running your system is a matter of starting
+  your current application and its dependencies. Differently from other languages,
+  Elixir does not have a `main` procedure that is responsible for starting your
+  system. Instead, you start one or more applications, each with their own
+  initialization and termination logic.
 
-  You can learn more about Mix generation of `.app` files by typing
-  `mix help compile.app`.
+  Starting an application is done via the "application module callback", which
+  is a module that defines the `start/2` function. The `start/2` function should
+  then start a supervisor, which is often called as the top-level supervisor, since
+  it sits at the root of a potentially long supervision tree. When the system is
+  shutting down, all applications shut down their top-level supervisor, which
+  terminates children in the opposite order they are started.
+
+  We have mentioned the Mix build tool is responsible for compiling applications,
+  but it is also capable of running applications. For example, `mix test`
+  automatically starts your application dependencies and your application itself
+  before your test runs. `mix run --no-halt` also boots your current project and
+  can be used to start a long running system. See `mix help run`.
+
+  Developers can also use tools like [Distillery](https://github.com/bitwalker/distillery)
+  that build **releases**. Releases are able to package all of your source code
+  as well as the Erlang VM into a single directory. Releases also give you explicit
+  control over how each application is started and in which order. They also provide
+  a more streamlined mechanism for starting and stopping systems, debugging, logging,
+  as well as system monitoring.
+
+  Finally, Elixir provides tools such as escripts and archives, which are
+  different mechanisms for packaging your application. Those are typically used
+  when tools must be shared between developers and not as deployment options.
+  See `mix help archive.build` and `mix help escript.build` for more detail.
+
+  Shutting down a live system cleanly can be done by calling `System.stop/1`.
+  It will shut down all applications in the opposite order they are started.
+  Each application will then shutdown its top-level supervisor, if one is
+  available, which then shuts down its children.
+
+  From Erlang/OTP 19.1, a SIGTERM from the operating system will automatically
+  translate to `System.stop/0`. Erlang/OTP 20 gives user more explicit control
+  over OS signals via the `:os.set_signal/2` function.
+
+  Applications also provide an "application environment", which is how
+  applications are configured. The application environment can either be set
+  statically, via a configuration file, or dynamically via `put_env/3` and
+  friends.
+
+  Over the next sections, we will cover the "application environment" and
+  the "application module callback" in more detail.
 
   ## Application environment
 
@@ -40,9 +80,13 @@ defmodule Application do
       Application.get_env(:APP_NAME, :hello)
       #=> :world
 
-  It is also possible to put and delete values from the application value,
-  including new values that are not defined in the environment file (although
-  this should be avoided).
+  Applications and dependencies in Mix projects are typically configured
+  via the `config/config.exs` file. For example, someone using your
+  application can configure the `:hello` key as follows:
+
+      config :APP_NAME, hello: :brand_new_world
+
+  It is also possible to configure applications dynamically via `put_env/3`.
 
   Keep in mind that each application is responsible for its environment.
   Do not use the functions in this module for directly accessing or modifying
@@ -78,8 +122,8 @@ defmodule Application do
 
   The `type` argument passed to `start/2` is usually `:normal` unless in a
   distributed setup where application takeovers and failovers are configured.
-  This particular aspect of applications is explained in more detail in the
-  OTP documentation:
+  Distributed applications is beyond the scope of this documentation. For those
+  interested on the topic, please access the OTP documentation:
 
     * [`:application` module](http://www.erlang.org/doc/man/application.html)
     * [Applications – OTP Design Principles](http://www.erlang.org/doc/design_principles/applications.html)
@@ -88,12 +132,18 @@ defmodule Application do
   by `use Application`) which does any application cleanup. It receives the
   application state and can return any value. Note that shutting down the
   supervisor is automatically handled by the VM.
+
+  An application without a supervision tree doesn't define an application
+  module callback in the application definition in `mix.exs` file. Even though
+  there is no module with application callbacks such as `start/2` and
+  `stop/1`, the application can be started and stopped the same way as an
+  application with a supervision tree.
   """
 
   @doc """
   Called when an application is started.
 
-  This function is called when an the application is started using
+  This function is called when an application is started using
   `Application.start/2` (and functions on top of that, such as
   `Application.ensure_started/2`). This function should start the top-level
   process of the application (which should be the top supervisor of the
@@ -104,7 +154,7 @@ defmodule Application do
 
     * `:normal` - used if the startup is a normal startup or if the application
       is distributed and is started on the current node because of a failover
-      from another mode and the application specification key `:start_phases`
+      from another node and the application specification key `:start_phases`
       is `:undefined`.
     * `{:takeover, node}` - used if the application is distributed and is
       started on the current node because of a failover on the node `node`.
@@ -145,6 +195,20 @@ defmodule Application do
   """
   @callback stop(state) :: term
 
+  @doc """
+  Start an application in synchronous phases.
+
+  This function is called after `start/2` finishes but before
+  `Application.start/2` returns. It will be called once for every start phase
+  defined in the application's (and any included applications') specification,
+  in the order they are listed in.
+  """
+  @callback start_phase(phase :: term, start_type, phase_args :: term) ::
+    :ok |
+    {:error, reason :: term}
+
+  @optional_callbacks start_phase: 3
+
   @doc false
   defmacro __using__(_) do
     quote location: :keep do
@@ -155,7 +219,7 @@ defmodule Application do
         :ok
       end
 
-      defoverridable [stop: 1]
+      defoverridable Application
     end
   end
 
@@ -269,7 +333,7 @@ defmodule Application do
 
   ## Options
 
-    * `:timeout`    - the timeout for the change (defaults to 5000ms)
+    * `:timeout` - the timeout for the change (defaults to `5_000` milliseconds)
     * `:persistent` - persists the given value on application load and reloads
 
   If `put_env/4` is called before the application is loaded, the application
@@ -334,7 +398,7 @@ defmodule Application do
   started before this application is. If not, `{:error, {:not_started, app}}` is
   returned, where `app` is the name of the missing application.
 
-  In case you want to automatically  load **and start** all of `app`'s dependencies,
+  In case you want to automatically load **and start** all of `app`'s dependencies,
   see `ensure_all_started/2`.
 
   The `type` argument specifies the type of the application:
@@ -419,7 +483,7 @@ defmodule Application do
       #=> "bar-123"
 
   For more information on code paths, check the `Code` module in
-  Elixir and also Erlang's `:code` module.
+  Elixir and also Erlang's [`:code` module](http://www.erlang.org/doc/man/code.html).
   """
   @spec app_dir(app) :: String.t
   def app_dir(app) when is_atom(app) do

lib/elixir/lib/atom.ex

@@ -37,7 +37,8 @@ defmodule Atom do
     :erlang.atom_to_list(atom)
   end
 
-  # TODO: Deprecate by v1.5
+  # TODO: Remove by 2.0
+  # (hard-deprecated in elixir_dispatch)
   @doc false
   @spec to_char_list(atom) :: charlist
   def to_char_list(atom), do: Atom.to_charlist(atom)

lib/elixir/lib/base.ex

@@ -3,7 +3,7 @@ defmodule Base do
 
   @moduledoc """
   This module provides data encoding and decoding functions
-  according to [RFC 4648](http://tools.ietf.org/html/rfc4648).
+  according to [RFC 4648](https://tools.ietf.org/html/rfc4648).
 
   This document defines the commonly used base 16, base 32, and base
   64 encoding schemes.
@@ -92,58 +92,145 @@ defmodule Base do
 
   """
 
-  b16_alphabet    = Enum.with_index '0123456789ABCDEF'
-  b64_alphabet    = Enum.with_index 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/'
-  b64url_alphabet = Enum.with_index 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_'
-  b32_alphabet    = Enum.with_index 'ABCDEFGHIJKLMNOPQRSTUVWXYZ234567'
-  b32hex_alphabet = Enum.with_index '0123456789ABCDEFGHIJKLMNOPQRSTUV'
+  b16_alphabet    = '0123456789ABCDEF'
+  b64_alphabet    = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/'
+  b64url_alphabet = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_'
+  b32_alphabet    = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ234567'
+  b32hex_alphabet = '0123456789ABCDEFGHIJKLMNOPQRSTUV'
 
-  Enum.each [{:enc16,    :dec16,    b16_alphabet},
-             {:enc32,    :dec32,    b32_alphabet},
-             {:enc64,    :dec64,    b64_alphabet},
-             {:enc64url, :dec64url, b64url_alphabet},
-             {:enc32hex, :dec32hex, b32hex_alphabet}], fn({enc, dec, alphabet}) ->
-    for {encoding, value} <- alphabet do
-      defp unquote(enc)(unquote(value)), do: unquote(encoding)
-      defp unquote(dec)(unquote(encoding)), do: unquote(value)
-    end
-    defp unquote(dec)(c) do
-      raise ArgumentError, "non-alphabet digit found: #{inspect <<c>>, binaries: :as_strings} (byte #{c})"
+  defmacrop encode_pair(alphabet, case, value) do
+    quote do
+      case unquote(value) do
+        unquote(encode_pair_clauses(alphabet, case))
+      end
     end
   end
 
-  @compile {:inline, from_upper: 1, from_lower: 1, from_mixed: 1,
-                     to_lower: 1, to_upper: 1, enc16: 1, dec16: 1,
-                     enc32: 1, dec32: 1, enc32hex: 1, dec32hex: 1,
-                     enc64: 1, dec64: 1, enc64url: 1, dec64url: 1}
+  defp encode_pair_clauses(alphabet, case) when case in [:sensitive, :upper] do
+    shift = shift(alphabet)
+    alphabet
+    |> Enum.with_index()
+    |> encode_clauses(shift)
+  end
 
-  defp to_lower(char) when char in ?A..?Z,
-    do: char + (?a - ?A)
-  defp to_lower(char),
-    do: char
+  defp encode_pair_clauses(alphabet, :lower) do
+    shift = shift(alphabet)
+    alphabet
+    |> Stream.map(fn c -> (if c in ?A..?Z, do: c - ?A + ?a, else: c) end)
+    |> Enum.with_index()
+    |> encode_clauses(shift)
+  end
 
-  defp to_upper(char), do: char
+  defp shift(alphabet) do
+    alphabet
+    |> length()
+    |> :math.log2()
+    |> round()
+  end
 
-  defp from_upper(char), do: char
+  defp encode_clauses(alphabet, shift) do
+    for {encoding1, value1} <- alphabet, {encoding2, value2} <- alphabet do
+      encoding = bsl(encoding1, 8) + encoding2
+      value = bsl(value1, shift) + value2
+      [clause] = quote do: (unquote(value) -> unquote(encoding))
+      clause
+    end
+  end
 
-  defp from_lower(char) when char in ?a..?z,
-    do: char - (?a - ?A)
-  defp from_lower(char) when not char in ?A..?Z,
-    do: char
-  defp from_lower(char),
-    do: raise(ArgumentError, "non-alphabet digit found: \"#{<<char>>}\" (byte #{char})")
+  defmacrop decode_char(alphabet, case, encoding) do
+    quote do
+      case unquote(encoding) do
+        unquote(decode_char_clauses(alphabet, case))
+      end
+    end
+  end
 
-  defp from_mixed(char) when char in ?a..?z,
-    do: char - (?a - ?A)
-  defp from_mixed(char),
-    do: char
+  defp decode_char_clauses(alphabet, case) when case in [:sensitive, :upper] do
+    clauses =
+      alphabet
+      |> Enum.with_index()
+      |> decode_clauses()
+    clauses ++ bad_digit_clause()
+  end
 
-  defp maybe_pad(subject, false, _, _),
-    do: subject
-  defp maybe_pad(subject, _, group_size, pad) do
-    case rem(byte_size(subject), group_size) do
-      0 -> subject
-      x -> subject <> String.duplicate(pad, group_size - x)
+  defp decode_char_clauses(alphabet, :lower) do
+    {uppers, rest} =
+      alphabet
+      |> Stream.with_index()
+      |> Enum.split_with(fn {encoding, _} -> encoding in ?A..?Z end)
+    lowers =
+      Enum.map(uppers, fn {encoding, value} -> {encoding - ?A + ?a, value} end)
+
+    if length(uppers) > length(rest) do
+      decode_mixed_clauses(lowers, rest)
+    else
+      decode_mixed_clauses(rest, lowers)
+    end
+  end
+
+  defp decode_char_clauses(alphabet, :mixed) when length(alphabet) == 16 do
+    alphabet = Enum.with_index(alphabet)
+    lowers =
+      alphabet
+      |> Stream.filter(fn {encoding, _} -> encoding in ?A..?Z end)
+      |> Enum.map(fn {encoding, value} -> {encoding - ?A + ?a, value} end)
+    decode_mixed_clauses(alphabet, lowers)
+  end
+
+  defp decode_char_clauses(alphabet, :mixed) when length(alphabet) == 32 do
+    alphabet
+    |> Stream.with_index()
+    |> Enum.flat_map(fn {encoding, value} = pair ->
+         if encoding in ?A..?Z do
+           [pair, {encoding - ?A + ?a, value}]
+         else
+           [pair]
+         end
+       end)
+    |> decode_clauses()
+  end
+
+  defp decode_mixed_clauses(first, second) do
+    first_clauses = decode_clauses(first)
+    second_clauses = decode_clauses(second) ++ bad_digit_clause()
+    join_clause =
+      quote do
+        encoding ->
+          case encoding do
+            unquote(second_clauses)
+          end
+      end
+    first_clauses ++ join_clause
+  end
+
+  defp decode_clauses(alphabet) do
+    for {encoding, value} <- alphabet do
+      [clause] = quote do: (unquote(encoding) -> unquote(value))
+      clause
+    end
+  end
+
+  defp bad_digit_clause() do
+    quote do
+      c ->
+        raise ArgumentError,
+              "non-alphabet digit found: #{inspect <<c>>, binaries: :as_strings} (byte #{c})"
+    end
+  end
+
+  defp maybe_pad(body, "", _, _),
+    do: body
+  defp maybe_pad(body, tail, false, _),
+    do: body <> tail
+  defp maybe_pad(body, tail, _, group_size) do
+    case group_size - rem(byte_size(tail), group_size) do
+      ^group_size -> body <> tail
+      6 -> body <> tail <> "======"
+      5 -> body <> tail <> "====="
+      4 -> body <> tail <> "===="
+      3 -> body <> tail <> "==="
+      2 -> body <> tail <> "=="
+      1 -> body <> tail <> "="
     end
   end
 
@@ -170,8 +257,7 @@ defmodule Base do
       "666f6f626172"
 
   """
-  @spec encode16(binary) :: binary
-  @spec encode16(binary, Keyword.t) :: binary
+  @spec encode16(binary, keyword) :: binary
   def encode16(data, opts \\ []) when is_binary(data) do
     case = Keyword.get(opts, :case, :upper)
     do_encode16(case, data)
@@ -204,8 +290,7 @@ defmodule Base do
       {:ok, "foobar"}
 
   """
-  @spec decode16(binary) :: {:ok, binary} | :error
-  @spec decode16(binary, Keyword.t) :: {:ok, binary} | :error
+  @spec decode16(binary, keyword) :: {:ok, binary} | :error
   def decode16(string, opts \\ []) do
     {:ok, decode16!(string, opts)}
   rescue
@@ -242,8 +327,7 @@ defmodule Base do
       "foobar"
 
   """
-  @spec decode16!(binary) :: binary
-  @spec decode16!(binary, Keyword.t) :: binary
+  @spec decode16!(binary, keyword) :: binary
   def decode16!(string, opts \\ [])
 
   def decode16!(string, opts) when is_binary(string) and rem(byte_size(string), 2) == 0 do
@@ -273,8 +357,7 @@ defmodule Base do
       "Zm9vYg"
 
   """
-  @spec encode64(binary) :: binary
-  @spec encode64(binary, Keyword.t) :: binary
+  @spec encode64(binary, keyword) :: binary
   def encode64(data, opts \\ []) when is_binary(data) do
     pad? = Keyword.get(opts, :padding, true)
     do_encode64(data, pad?)
@@ -304,8 +387,7 @@ defmodule Base do
       {:ok, "foob"}
 
   """
-  @spec decode64(binary) :: {:ok, binary} | :error
-  @spec decode64(binary, Keyword.t) :: {:ok, binary} | :error
+  @spec decode64(binary, keyword) :: {:ok, binary} | :error
   def decode64(string, opts \\ []) when is_binary(string) do
     {:ok, decode64!(string, opts)}
   rescue
@@ -339,8 +421,7 @@ defmodule Base do
       "foob"
 
   """
-  @spec decode64!(binary) :: binary
-  @spec decode64!(binary, Keyword.t) :: binary
+  @spec decode64!(binary, keyword) :: binary
   def decode64!(string, opts \\ []) when is_binary(string) do
     pad? = Keyword.get(opts, :padding, true)
     string |> remove_ignored(opts[:ignore]) |> do_decode64(pad?)
@@ -362,8 +443,7 @@ defmodule Base do
       "_3_-_A"
 
   """
-  @spec url_encode64(binary) :: binary
-  @spec url_encode64(binary, Keyword.t) :: binary
+  @spec url_encode64(binary, keyword) :: binary
   def url_encode64(data, opts \\ []) when is_binary(data) do
     pad? = Keyword.get(opts, :padding, true)
     do_encode64url(data, pad?)
@@ -391,8 +471,7 @@ defmodule Base do
       {:ok, <<255, 127, 254, 252>>}
 
   """
-  @spec url_decode64(binary) :: {:ok, binary} | :error
-  @spec url_decode64(binary, Keyword.t) :: {:ok, binary} | :error
+  @spec url_decode64(binary, keyword) :: {:ok, binary} | :error
   def url_decode64(string, opts \\ []) when is_binary(string) do
     {:ok, url_decode64!(string, opts)}
   rescue
@@ -424,8 +503,7 @@ defmodule Base do
       <<255, 127, 254, 252>>
 
   """
-  @spec url_decode64!(binary) :: binary
-  @spec url_decode64!(binary, Keyword.t) :: binary
+  @spec url_decode64!(binary, keyword) :: binary
   def url_decode64!(string, opts \\ []) when is_binary(string)  do
     pad? = Keyword.get(opts, :padding, true)
     string |> remove_ignored(opts[:ignore]) |> do_decode64url(pad?)
@@ -463,8 +541,7 @@ defmodule Base do
       "MZXW6YTBOI"
 
   """
-  @spec encode32(binary) :: binary
-  @spec encode32(binary, Keyword.t) :: binary
+  @spec encode32(binary, keyword) :: binary
   def encode32(data, opts \\ []) when is_binary(data) do
     case = Keyword.get(opts, :case, :upper)
     pad? = Keyword.get(opts, :padding, true)
@@ -507,8 +584,7 @@ defmodule Base do
       {:ok, "foobar"}
 
   """
-  @spec decode32(binary) :: {:ok, binary} | :error
-  @spec decode32(binary, Keyword.t) :: {:ok, binary} | :error
+  @spec decode32(binary, keyword) :: {:ok, binary} | :error
   def decode32(string, opts \\ []) do
     {:ok, decode32!(string, opts)}
   rescue
@@ -554,8 +630,7 @@ defmodule Base do
       "foobar"
 
   """
-  @spec decode32!(binary) :: binary
-  @spec decode32!(binary, Keyword.t) :: binary
+  @spec decode32!(binary, keyword) :: binary
   def decode32!(string, opts \\ []) when is_binary(string) do
     case = Keyword.get(opts, :case, :upper)
     pad? = Keyword.get(opts, :padding, true)
@@ -595,12 +670,11 @@ defmodule Base do
       "CPNMUOJ1E8"
 
   """
-  @spec hex_encode32(binary) :: binary
-  @spec hex_encode32(binary, Keyword.t) :: binary
+  @spec hex_encode32(binary, keyword) :: binary
   def hex_encode32(data, opts \\ []) when is_binary(data) do
     case = Keyword.get(opts, :case, :upper)
     pad? = Keyword.get(opts, :padding, true)
-    do_hex_encode32(case, data, pad?)
+    do_encode32hex(case, data, pad?)
   end
 
   @doc """
@@ -640,8 +714,7 @@ defmodule Base do
       {:ok, "foobar"}
 
   """
-  @spec hex_decode32(binary) :: {:ok, binary} | :error
-  @spec hex_decode32(binary, Keyword.t) :: {:ok, binary} | :error
+  @spec hex_decode32(binary, keyword) :: {:ok, binary} | :error
   def hex_decode32(string, opts \\ []) do
     {:ok, hex_decode32!(string, opts)}
   rescue
@@ -688,257 +761,310 @@ defmodule Base do
       "foobar"
 
   """
-  @spec hex_decode32!(binary) :: binary
-  @spec hex_decode32!(binary, Keyword.t) :: binary
+  @spec hex_decode32!(binary, keyword) :: binary
   def hex_decode32!(string, opts \\ []) when is_binary(string) do
     case = Keyword.get(opts, :case, :upper)
     pad? = Keyword.get(opts, :padding, true)
-    do_hex_decode32(case, string, pad?)
+    do_decode32hex(case, string, pad?)
   end
 
   defp remove_ignored(string, nil), do: string
   defp remove_ignored(string, :whitespace) do
-    for <<c::8 <- string>>, not c in '\s\t\r\n', into: <<>>, do: <<c::8>>
+    for <<char::8 <- string>>, char not in '\s\t\r\n', into: <<>>, do: <<char::8>>
+  end
+
+  enc16 = [upper: :enc16_upper, lower: :enc16_lower]
+
+  for {case, fun} <- enc16 do
+    defp unquote(fun)(char) do
+      encode_pair(unquote(b16_alphabet), unquote(case), char)
+    end
   end
 
   defp do_encode16(_, <<>>), do: <<>>
-  defp do_encode16(:upper, data) do
-    for <<c::4 <- data>>, into: <<>>, do: <<enc16(c)::8>>
+
+  for {case, fun} <- enc16 do
+    defp do_encode16(unquote(case), data) do
+      split =  8 * div(byte_size(data), 8)
+      <<main::size(split)-binary, rest::binary>> = data
+      main =
+        for <<c1::8, c2::8, c3::8, c4::8, c5::8, c6::8, c7::8, c8::8 <- main>>, into: <<>> do
+          <<unquote(fun)(c1)::16, unquote(fun)(c2)::16,
+            unquote(fun)(c3)::16, unquote(fun)(c4)::16,
+            unquote(fun)(c5)::16, unquote(fun)(c6)::16,
+            unquote(fun)(c7)::16, unquote(fun)(c8)::16>>
+        end
+      case rest do
+        <<c1::8, c2::8, c3::8, c4::8, c5::8, c6::8, c7::8>> ->
+          <<main::binary, unquote(fun)(c1)::16, unquote(fun)(c2)::16,
+            unquote(fun)(c3)::16, unquote(fun)(c4)::16,
+            unquote(fun)(c5)::16, unquote(fun)(c6)::16,
+            unquote(fun)(c7)::16>>
+        <<c1::8, c2::8, c3::8, c4::8, c5::8, c6::8>> ->
+          <<main::binary, unquote(fun)(c1)::16, unquote(fun)(c2)::16,
+            unquote(fun)(c3)::16, unquote(fun)(c4)::16,
+            unquote(fun)(c5)::16, unquote(fun)(c6)::16>>
+        <<c1::8, c2::8, c3::8, c4::8, c5::8>> ->
+          <<main::binary, unquote(fun)(c1)::16, unquote(fun)(c2)::16,
+            unquote(fun)(c3)::16, unquote(fun)(c4)::16,
+            unquote(fun)(c5)::16>>
+        <<c1::8, c2::8, c3::8, c4::8>> ->
+          <<main::binary, unquote(fun)(c1)::16, unquote(fun)(c2)::16,
+            unquote(fun)(c3)::16, unquote(fun)(c4)::16>>
+        <<c1::8, c2::8, c3::8>> ->
+          <<main::binary, unquote(fun)(c1)::16, unquote(fun)(c2)::16,
+            unquote(fun)(c3)::16>>
+        <<c1::8, c2::8>> ->
+          <<main::binary, unquote(fun)(c1)::16, unquote(fun)(c2)::16>>
+        <<c1::8>> ->
+          <<main::binary, unquote(fun)(c1)::16>>
+        <<>> ->
+          main
+      end
+    end
   end
-  defp do_encode16(:lower, data) do
-    for <<c::4 <- data>>, into: <<>>, do: <<to_lower(enc16(c))::8>>
+
+  dec16 = [upper: :dec16_upper, lower: :dec16_lower, mixed: :dec16_mixed]
+
+  for {case, fun} <- dec16 do
+    defp unquote(fun)(encoding) do
+      decode_char(unquote(b16_alphabet), unquote(case), encoding)
+    end
   end
 
   defp do_decode16(_, <<>>), do: <<>>
-  defp do_decode16(:upper, string) when rem(byte_size(string), 2) == 0 do
-    for <<c1::8, c2::8 <- string>>, into: <<>> do
-      <<dec16(c1)::4, dec16(c2)::4>>
-    end
-  end
-  defp do_decode16(:lower, string) when rem(byte_size(string), 2) == 0 do
-    for <<c1::8, c2::8 <- string>>, into: <<>> do
-      <<dec16(from_lower(c1))::4, dec16(from_lower(c2))::4>>
-    end
-  end
-  defp do_decode16(:mixed, string) when rem(byte_size(string), 2) == 0 do
-    for <<c1::8, c2::8 <- string>>, into: <<>> do
-      <<dec16(from_mixed(c1))::4, dec16(from_mixed(c2))::4>>
+
+  for {case, fun} <- dec16 do
+    defp do_decode16(unquote(case), string) do
+      split =  8 * div(byte_size(string), 8)
+      <<main::size(split)-binary, rest::binary>> = string
+      main =
+        for <<c1::8, c2::8, c3::8, c4::8, c5::8, c6::8, c7::8, c8::8 <- main>>, into: <<>> do
+          <<unquote(fun)(c1)::4, unquote(fun)(c2)::4,
+            unquote(fun)(c3)::4, unquote(fun)(c4)::4,
+            unquote(fun)(c5)::4, unquote(fun)(c6)::4,
+            unquote(fun)(c7)::4, unquote(fun)(c8)::4>>
+        end
+      case rest do
+        <<c1::8, c2::8, c3::8, c4::8, c5::8, c6::8>> ->
+          <<main::bits, unquote(fun)(c1)::4, unquote(fun)(c2)::4,
+            unquote(fun)(c3)::4, unquote(fun)(c4)::4,
+            unquote(fun)(c5)::4, unquote(fun)(c6)::4>>
+        <<c1::8, c2::8, c3::8, c4::8>> ->
+          <<main::bits, unquote(fun)(c1)::4, unquote(fun)(c2)::4,
+            unquote(fun)(c3)::4, unquote(fun)(c4)::4>>
+        <<c1::8, c2::8>> ->
+          <<main::bits, unquote(fun)(c1)::4, unquote(fun)(c2)::4>>
+        <<_::8>> ->
+          raise ArgumentError, "odd-length string"
+        <<>> ->
+          main
+      end
     end
   end
 
-  defp do_encode64(<<>>, _), do: <<>>
-  defp do_encode64(data, pad?) do
-    split =  3 * div(byte_size(data), 3)
-    <<main::size(split)-binary, rest::binary>> = data
-    main = for <<c::6 <- main>>, into: <<>>, do: <<enc64(c)::8>>
-    tail = case rest do
-      <<c1::6, c2::6, c3::4>> ->
-        <<enc64(c1)::8, enc64(c2)::8, enc64(bsl(c3, 2))::8>>
-      <<c1::6, c2::2>> ->
-        <<enc64(c1)::8, enc64(bsl(c2, 4))::8>>
-      <<>> ->
-        <<>>
+  for {base, alphabet} <- ["64": b64_alphabet, "64url": b64url_alphabet] do
+    pair = :"enc#{base}_pair"
+    char = :"enc#{base}_char"
+    do_encode = :"do_encode#{base}"
+
+    defp unquote(pair)(value) do
+      encode_pair(unquote(alphabet), :sensitive, value)
     end
-    main <> maybe_pad(tail, pad?, 4, "=")
-  end
 
-  defp do_decode64(<<>>, _), do: <<>>
-  defp do_decode64(string, false) do
-    maybe_pad(string, true, 4, "=") |> do_decode64(true)
-  end
-  defp do_decode64(string, _pad?) when rem(byte_size(string), 4) == 0 do
-    split = byte_size(string) - 4
-    <<main::size(split)-binary, rest::binary>> = string
-    main = for <<c::8 <- main>>, into: <<>>, do: <<dec64(c)::6>>
-    tail = case rest do
-      <<c1::8, c2::8, ?=, ?=>> ->
-        <<dec64(c1)::6, bsr(dec64(c2), 4)::2>>
-      <<c1::8, c2::8, c3::8, ?=>> ->
-        <<dec64(c1)::6, dec64(c2)::6, bsr(dec64(c3), 2)::4>>
-      <<c1::8, c2::8, c3::8, c4::8>> ->
-        <<dec64(c1)::6, dec64(c2)::6, dec64(c3)::6, dec64(c4)::6>>
-      <<>> ->
-        <<>>
+    defp unquote(char)(value) do
+      value
+      |> unquote(pair)()
+      |> band(0x00FF)
     end
-    main <> tail
-  end
-  defp do_decode64(_, _) do
-    raise ArgumentError, "incorrect padding"
-  end
 
-  defp do_encode64url(<<>>, _), do: <<>>
-  defp do_encode64url(data, pad?) do
-    split =  3 * div(byte_size(data), 3)
-    <<main::size(split)-binary, rest::binary>> = data
-    main = for <<c::6 <- main>>, into: <<>>, do: <<enc64url(c)::8>>
-    tail = case rest do
-      <<c1::6, c2::6, c3::4>> ->
-        <<enc64url(c1)::8, enc64url(c2)::8, enc64url(bsl(c3, 2))::8>>
-      <<c1::6, c2::2>> ->
-        <<enc64url(c1)::8, enc64url(bsl(c2, 4))::8>>
-      <<>> ->
-        <<>>
+    defp unquote(do_encode)(<<>>, _), do: <<>>
+    defp unquote(do_encode)(data, pad?) do
+      split =  6 * div(byte_size(data), 6)
+      <<main::size(split)-binary, rest::binary>> = data
+      main = for <<c1::12, c2::12, c3::12, c4::12 <- main>>, into: <<>> do
+        <<unquote(pair)(c1)::16, unquote(pair)(c2)::16,
+          unquote(pair)(c3)::16, unquote(pair)(c4)::16>>
+      end
+      tail = case rest do
+        <<c1::12, c2::12, c3::12, c::4>> ->
+          <<unquote(pair)(c1)::16, unquote(pair)(c2)::16, unquote(pair)(c3)::16,
+            unquote(char)(bsl(c, 2))::8>>
+        <<c1::12, c2::12, c3::8>> ->
+          <<unquote(pair)(c1)::16, unquote(pair)(c2)::16,
+            unquote(pair)(bsl(c3, 4))::16>>
+        <<c1::12, c2::12>> ->
+          <<unquote(pair)(c1)::16, unquote(pair)(c2)::16>>
+        <<c1::12, c2::4>> ->
+          <<unquote(pair)(c1)::16, unquote(char)(bsl(c2, 2))::8>>
+        <<c1::8>> ->
+          <<unquote(pair)(bsl(c1, 4))::16>>
+        <<>> ->
+          <<>>
+      end
+      maybe_pad(main, tail, pad?, 4)
     end
-    main <> maybe_pad(tail, pad?, 4, "=")
   end
 
-  defp do_decode64url(<<>>, _), do: <<>>
-  defp do_decode64url(string, false) do
-    maybe_pad(string, true, 4, "=") |> do_decode64url(true)
-  end
-  defp do_decode64url(string, _pad?) when rem(byte_size(string), 4) == 0 do
-    split = byte_size(string) - 4
-    <<main::size(split)-binary, rest::binary>> = string
-    main = for <<c::8 <- main>>, into: <<>>, do: <<dec64url(c)::6>>
-    tail = case rest do
-      <<c1::8, c2::8, ?=, ?=>> ->
-        <<dec64url(c1)::6, bsr(dec64url(c2), 4)::2>>
-      <<c1::8, c2::8, c3::8, ?=>> ->
-        <<dec64url(c1)::6, dec64url(c2)::6, bsr(dec64url(c3), 2)::4>>
-      <<c1::8, c2::8, c3::8, c4::8>> ->
-        <<dec64url(c1)::6, dec64url(c2)::6, dec64url(c3)::6, dec64url(c4)::6>>
-      <<>> ->
-        <<>>
+  for {base, alphabet} <- ["64": b64_alphabet, "64url": b64url_alphabet] do
+    fun = :"dec#{base}"
+    do_decode = :"do_decode#{base}"
+
+    defp unquote(fun)(encoding) do
+      decode_char(unquote(alphabet), :sensitive, encoding)
+    end
+
+    defp unquote(do_decode)(<<>>, _), do: <<>>
+    defp unquote(do_decode)(string, pad?) do
+      segs = div(byte_size(string) + 7, 8) - 1
+      <<main::size(segs)-binary-unit(64), rest::binary>> = string
+      main =
+        for <<c1::8, c2::8, c3::8, c4::8, c5::8, c6::8, c7::8, c8::8 <- main>>, into: <<>> do
+          <<unquote(fun)(c1)::6, unquote(fun)(c2)::6, unquote(fun)(c3)::6,
+            unquote(fun)(c4)::6, unquote(fun)(c5)::6, unquote(fun)(c6)::6,
+            unquote(fun)(c7)::6, unquote(fun)(c8)::6>>
+        end
+      case rest do
+        <<c1::8, c2::8, ?=, ?=>> ->
+          <<main::bits, unquote(fun)(c1)::6, bsr(unquote(fun)(c2), 4)::2>>
+        <<c1::8, c2::8, c3::8, ?=>> ->
+          <<main::bits, unquote(fun)(c1)::6, unquote(fun)(c2)::6,
+            bsr(unquote(fun)(c3), 2)::4>>
+        <<c1::8, c2::8, c3::8, c4::8>> ->
+          <<main::bits, unquote(fun)(c1)::6, unquote(fun)(c2)::6,
+            unquote(fun)(c3)::6, unquote(fun)(c4)::6>>
+        <<c1::8, c2::8, c3::8, c4::8, c5::8, c6::8, ?=, ?=>> ->
+          <<main::bits, unquote(fun)(c1)::6, unquote(fun)(c2)::6,
+            unquote(fun)(c3)::6, unquote(fun)(c4)::6, unquote(fun)(c5)::6,
+            bsr(unquote(fun)(c6), 4)::2>>
+        <<c1::8, c2::8, c3::8, c4::8, c5::8, c6::8, c7::8, ?=>> ->
+          <<main::bits, unquote(fun)(c1)::6, unquote(fun)(c2)::6,
+            unquote(fun)(c3)::6, unquote(fun)(c4)::6, unquote(fun)(c5)::6,
+            unquote(fun)(c6)::6, bsr(unquote(fun)(c7), 2)::4>>
+        <<c1::8, c2::8, c3::8, c4::8, c5::8, c6::8, c7::8, c8::8>> ->
+          <<main::bits, unquote(fun)(c1)::6, unquote(fun)(c2)::6,
+            unquote(fun)(c3)::6, unquote(fun)(c4)::6, unquote(fun)(c5)::6,
+            unquote(fun)(c6)::6, unquote(fun)(c7)::6, unquote(fun)(c8)::6>>
+        <<c1::8, c2::8>> when not pad? ->
+          <<main::bits, unquote(fun)(c1)::6, bsr(unquote(fun)(c2), 4)::2>>
+        <<c1::8, c2::8, c3::8>> when not pad? ->
+          <<main::bits, unquote(fun)(c1)::6, unquote(fun)(c2)::6,
+            bsr(unquote(fun)(c3), 2)::4>>
+        <<c1::8, c2::8, c3::8, c4::8, c5::8, c6::8>> when not pad? ->
+          <<main::bits, unquote(fun)(c1)::6, unquote(fun)(c2)::6,
+            unquote(fun)(c3)::6, unquote(fun)(c4)::6, unquote(fun)(c5)::6,
+            bsr(unquote(fun)(c6), 4)::2>>
+        <<c1::8, c2::8, c3::8, c4::8, c5::8, c6::8, c7::8>> when not pad? ->
+          <<main::bits, unquote(fun)(c1)::6, unquote(fun)(c2)::6,
+            unquote(fun)(c3)::6, unquote(fun)(c4)::6, unquote(fun)(c5)::6,
+            unquote(fun)(c6)::6, bsr(unquote(fun)(c7), 2)::4>>
+        _ ->
+          raise ArgumentError, "incorrect padding"
+      end
     end
-    main <> tail
-  end
-  defp do_decode64url(_, _) do
-    raise ArgumentError, "incorrect padding"
   end
 
-  defp do_encode32(_, <<>>, _), do: <<>>
+  for {base, alphabet} <- ["32": b32_alphabet, "32hex": b32hex_alphabet],
+      case <- [:upper, :lower] do
+    pair = :"enc#{base}_#{case}_pair"
+    char = :"enc#{base}_#{case}_char"
+    do_encode = :"do_encode#{base}"
 
-  for {case, fun} <- [upper: :to_upper, lower: :to_lower] do
-    defp do_encode32(unquote(case), data, pad?) do
+    defp unquote(pair)(value) do
+      encode_pair(unquote(alphabet), unquote(case), value)
+    end
+
+    defp unquote(char)(value) do
+      value
+      |> unquote(pair)()
+      |> band(0x00FF)
+    end
+
+    defp unquote(do_encode)(_, <<>>, _), do: <<>>
+    defp unquote(do_encode)(unquote(case), data, pad?) do
       split =  5 * div(byte_size(data), 5)
       <<main::size(split)-binary, rest::binary>> = data
-      main = for <<c::5 <- main>>, into: <<>>, do: <<unquote(fun)(enc32(c))::8>>
+      main =
+        for <<c1::10, c2::10, c3::10, c4::10 <- main>>, into: <<>> do
+          <<unquote(pair)(c1)::16, unquote(pair)(c2)::16,
+            unquote(pair)(c3)::16, unquote(pair)(c4)::16>>
+        end
       tail = case rest do
-        <<c1::5, c2::5, c3::5, c4::5, c5::5, c6::5, c7::2>> ->
-          <<unquote(fun)(enc32(c1))::8, unquote(fun)(enc32(c2))::8,
-            unquote(fun)(enc32(c3))::8, unquote(fun)(enc32(c4))::8,
-            unquote(fun)(enc32(c5))::8, unquote(fun)(enc32(c6))::8,
-            unquote(fun)(enc32(bsl(c7, 3)))::8>>
-        <<c1::5, c2::5, c3::5, c4::5, c5::4>> ->
-          <<unquote(fun)(enc32(c1))::8, unquote(fun)(enc32(c2))::8,
-            unquote(fun)(enc32(c3))::8, unquote(fun)(enc32(c4))::8,
-            unquote(fun)(enc32(bsl(c5, 1)))::8>>
-        <<c1::5, c2::5, c3::5, c4::1>> ->
-          <<unquote(fun)(enc32(c1))::8, unquote(fun)(enc32(c2))::8,
-            unquote(fun)(enc32(c3))::8, unquote(fun)(enc32(bsl(c4, 4)))::8>>
-        <<c1::5, c2::3>> ->
-          <<unquote(fun)(enc32(c1))::8, unquote(fun)(enc32(bsl(c2, 2)))::8>>
+        <<c1::10, c2::10, c3::10, c4::2>> ->
+          <<unquote(pair)(c1)::16, unquote(pair)(c2)::16,
+            unquote(pair)(c3)::16, unquote(char)(bsl(c4, 3))::8>>
+        <<c1::10, c2::10, c3::4>> ->
+          <<unquote(pair)(c1)::16, unquote(pair)(c2)::16,
+            unquote(char)(bsl(c3, 1))::8>>
+        <<c1::10, c2::6>> ->
+          <<unquote(pair)(c1)::16, unquote(pair)(bsl(c2, 4))::16>>
+        <<c1::8>> ->
+          <<unquote(pair)(bsl(c1, 2))::16>>
         <<>> ->
           <<>>
       end
-      main <> maybe_pad(tail, pad?, 8, "=")
+      maybe_pad(main, tail, pad?, 8)
     end
   end
 
-  defp do_decode32(_, <<>>, _), do: <<>>
-  defp do_decode32(case, string, false),
-    do: do_decode32(case, maybe_pad(string, true, 8, "="), true)
+  for {base, alphabet} <- ["32": b32_alphabet, "32hex": b32hex_alphabet],
+      case <- [:upper, :lower, :mixed] do
+    fun = :"dec#{base}_#{case}"
+    do_decode = :"do_decode#{base}"
 
-  for {case, fun} <- [upper: :from_upper, lower: :from_lower, mixed: :from_mixed] do
-    defp do_decode32(unquote(case), string, _pad?) when rem(byte_size(string), 8) == 0 do
-      split = byte_size(string) - 8
-      <<main::size(split)-binary, rest::binary>> = string
-      main = for <<c::8 <- main>>, into: <<>>, do: <<dec32(unquote(fun)(c))::5>>
-      tail = case rest do
+    defp unquote(fun)(encoding) do
+      decode_char(unquote(alphabet), unquote(case), encoding)
+    end
+
+    defp unquote(do_decode)(_, <<>>, _), do: <<>>
+
+    defp unquote(do_decode)(unquote(case), string, pad?) do
+      segs = div(byte_size(string) + 7, 8) - 1
+      <<main::size(segs)-binary-unit(64), rest::binary>> = string
+      main =
+        for <<c1::8, c2::8, c3::8, c4::8, c5::8, c6::8, c7::8, c8::8 <- main>>, into: <<>> do
+          <<unquote(fun)(c1)::5, unquote(fun)(c2)::5,
+            unquote(fun)(c3)::5, unquote(fun)(c4)::5,
+            unquote(fun)(c5)::5, unquote(fun)(c6)::5,
+            unquote(fun)(c7)::5, unquote(fun)(c8)::5>>
+        end
+      case rest do
         <<c1::8, c2::8, ?=, ?=, ?=, ?=, ?=, ?=>> ->
-          <<dec32(unquote(fun)(c1))::5, bsr(dec32(unquote(fun)(c2)), 2)::3>>
+          <<main::bits, unquote(fun)(c1)::5, bsr(unquote(fun)(c2), 2)::3>>
         <<c1::8, c2::8, c3::8, c4::8, ?=, ?=, ?=, ?=>> ->
-          <<dec32(unquote(fun)(c1))::5, dec32(unquote(fun)(c2))::5,
-            dec32(unquote(fun)(c3))::5, bsr(dec32(unquote(fun)(c4)), 4)::1>>
+          <<main::bits, unquote(fun)(c1)::5, unquote(fun)(c2)::5,
+            unquote(fun)(c3)::5, bsr(unquote(fun)(c4), 4)::1>>
         <<c1::8, c2::8, c3::8, c4::8, c5::8, ?=, ?=, ?=>> ->
-          <<dec32(unquote(fun)(c1))::5, dec32(unquote(fun)(c2))::5,
-            dec32(unquote(fun)(c3))::5, dec32(unquote(fun)(c4))::5,
-            bsr(dec32(unquote(fun)(c5)), 1)::4>>
+          <<main::bits, unquote(fun)(c1)::5, unquote(fun)(c2)::5,
+            unquote(fun)(c3)::5, unquote(fun)(c4)::5,
+            bsr(unquote(fun)(c5), 1)::4>>
         <<c1::8, c2::8, c3::8, c4::8, c5::8, c6::8, c7::8, ?=>> ->
-          <<dec32(unquote(fun)(c1))::5, dec32(unquote(fun)(c2))::5,
-            dec32(unquote(fun)(c3))::5, dec32(unquote(fun)(c4))::5,
-            dec32(unquote(fun)(c5))::5, dec32(unquote(fun)(c6))::5,
-            bsr(dec32(unquote(fun)(c7)), 3)::2>>
+          <<main::bits, unquote(fun)(c1)::5, unquote(fun)(c2)::5,
+            unquote(fun)(c3)::5, unquote(fun)(c4)::5,
+            unquote(fun)(c5)::5, unquote(fun)(c6)::5,
+            bsr(unquote(fun)(c7), 3)::2>>
         <<c1::8, c2::8, c3::8, c4::8, c5::8, c6::8, c7::8, c8::8>> ->
-          <<dec32(unquote(fun)(c1))::5, dec32(unquote(fun)(c2))::5,
-            dec32(unquote(fun)(c3))::5, dec32(unquote(fun)(c4))::5,
-            dec32(unquote(fun)(c5))::5, dec32(unquote(fun)(c6))::5,
-            dec32(unquote(fun)(c7))::5, dec32(unquote(fun)(c8))::5>>
-        <<>> ->
-          <<>>
+          <<main::bits, unquote(fun)(c1)::5, unquote(fun)(c2)::5,
+            unquote(fun)(c3)::5, unquote(fun)(c4)::5,
+            unquote(fun)(c5)::5, unquote(fun)(c6)::5,
+            unquote(fun)(c7)::5, unquote(fun)(c8)::5>>
+        <<c1::8, c2::8>> when not pad? ->
+          <<main::bits, unquote(fun)(c1)::5, bsr(unquote(fun)(c2), 2)::3>>
+        <<c1::8, c2::8, c3::8, c4::8>> when not pad? ->
+          <<main::bits, unquote(fun)(c1)::5, unquote(fun)(c2)::5,
+            unquote(fun)(c3)::5, bsr(unquote(fun)(c4), 4)::1>>
+        <<c1::8, c2::8, c3::8, c4::8, c5::8>> when not pad? ->
+          <<main::bits, unquote(fun)(c1)::5, unquote(fun)(c2)::5,
+            unquote(fun)(c3)::5, unquote(fun)(c4)::5,
+            bsr(unquote(fun)(c5), 1)::4>>
+        <<c1::8, c2::8, c3::8, c4::8, c5::8, c6::8, c7::8>> when not pad? ->
+          <<main::bits, unquote(fun)(c1)::5, unquote(fun)(c2)::5,
+            unquote(fun)(c3)::5, unquote(fun)(c4)::5,
+            unquote(fun)(c5)::5, unquote(fun)(c6)::5,
+            bsr(unquote(fun)(c7), 3)::2>>
+        _ ->
+          raise ArgumentError, "incorrect padding"
       end
-      main <> tail
     end
   end
-
-  defp do_decode32(_, _, _),
-    do: raise ArgumentError, "incorrect padding"
-
-  defp do_hex_encode32(_, <<>>, _), do: <<>>
-
-  for {case, fun} <- [upper: :to_upper, lower: :to_lower] do
-    defp do_hex_encode32(unquote(case), data, pad?) do
-      split =  5 * div(byte_size(data), 5)
-      <<main::size(split)-binary, rest::binary>> = data
-      main = for <<c::5 <- main>>, into: <<>>, do: <<unquote(fun)(enc32hex(c))::8>>
-      tail = case rest do
-        <<c1::5, c2::5, c3::5, c4::5, c5::5, c6::5, c7::2>> ->
-          <<unquote(fun)(enc32hex(c1))::8, unquote(fun)(enc32hex(c2))::8,
-            unquote(fun)(enc32hex(c3))::8, unquote(fun)(enc32hex(c4))::8,
-            unquote(fun)(enc32hex(c5))::8, unquote(fun)(enc32hex(c6))::8,
-            unquote(fun)(enc32hex(bsl(c7, 3)))::8>>
-        <<c1::5, c2::5, c3::5, c4::5, c5::4>> ->
-          <<unquote(fun)(enc32hex(c1))::8, unquote(fun)(enc32hex(c2))::8,
-            unquote(fun)(enc32hex(c3))::8, unquote(fun)(enc32hex(c4))::8,
-            unquote(fun)(enc32hex(bsl(c5, 1)))::8>>
-        <<c1::5, c2::5, c3::5, c4::1>> ->
-          <<unquote(fun)(enc32hex(c1))::8, unquote(fun)(enc32hex(c2))::8,
-            unquote(fun)(enc32hex(c3))::8, unquote(fun)(enc32hex(bsl(c4, 4)))::8>>
-        <<c1::5, c2::3>> ->
-          <<unquote(fun)(enc32hex(c1))::8, unquote(fun)(enc32hex(bsl(c2, 2)))::8>>
-        <<>> ->
-          <<>>
-      end
-      main <> maybe_pad(tail, pad?, 8, "=")
-    end
-  end
-
-  defp do_hex_decode32(_, <<>>, _), do: <<>>
-  defp do_hex_decode32(case, string, false),
-    do: do_hex_decode32(case, maybe_pad(string, true, 8, "="), true)
-
-  for {case, fun} <- [upper: :from_upper, lower: :from_lower, mixed: :from_mixed] do
-    defp do_hex_decode32(unquote(case), string, _pad?) when rem(byte_size(string), 8) == 0 do
-      split = byte_size(string) - 8
-      <<main::size(split)-binary, rest::binary>> = string
-      main = for <<c::8 <- main>>, into: <<>>, do: <<dec32hex(unquote(fun)(c))::5>>
-      tail = case rest do
-        <<c1::8, c2::8, ?=, ?=, ?=, ?=, ?=, ?=>> ->
-          <<dec32hex(unquote(fun)(c1))::5, bsr(dec32hex(unquote(fun)(c2)), 2)::3>>
-        <<c1::8, c2::8, c3::8, c4::8, ?=, ?=, ?=, ?=>> ->
-          <<dec32hex(unquote(fun)(c1))::5, dec32hex(unquote(fun)(c2))::5,
-            dec32hex(unquote(fun)(c3))::5, bsr(dec32hex(unquote(fun)(c4)), 4)::1>>
-        <<c1::8, c2::8, c3::8, c4::8, c5::8, ?=, ?=, ?=>> ->
-          <<dec32hex(unquote(fun)(c1))::5, dec32hex(unquote(fun)(c2))::5,
-            dec32hex(unquote(fun)(c3))::5, dec32hex(unquote(fun)(c4))::5,
-            bsr(dec32hex(unquote(fun)(c5)), 1)::4>>
-        <<c1::8, c2::8, c3::8, c4::8, c5::8, c6::8, c7::8, ?=>> ->
-          <<dec32hex(unquote(fun)(c1))::5, dec32hex(unquote(fun)(c2))::5,
-            dec32hex(unquote(fun)(c3))::5, dec32hex(unquote(fun)(c4))::5,
-            dec32hex(unquote(fun)(c5))::5, dec32hex(unquote(fun)(c6))::5,
-            bsr(dec32hex(unquote(fun)(c7)), 3)::2>>
-        <<c1::8, c2::8, c3::8, c4::8, c5::8, c6::8, c7::8, c8::8>> ->
-          <<dec32hex(unquote(fun)(c1))::5, dec32hex(unquote(fun)(c2))::5,
-            dec32hex(unquote(fun)(c3))::5, dec32hex(unquote(fun)(c4))::5,
-            dec32hex(unquote(fun)(c5))::5, dec32hex(unquote(fun)(c6))::5,
-            dec32hex(unquote(fun)(c7))::5, dec32hex(unquote(fun)(c8))::5>>
-        <<>> ->
-          <<>>
-      end
-      main <> tail
-    end
-  end
-
-  defp do_hex_decode32(_, _, _),
-    do: raise ArgumentError, "incorrect padding"
 end

lib/elixir/lib/calendar/date.ex

@@ -0,0 +1,597 @@
+defmodule Date do
+  @moduledoc """
+  A Date struct and functions.
+
+  The Date struct contains the fields year, month, day and calendar.
+  New dates can be built with the `new/3` function or using the `~D`
+  sigil:
+
+      iex> ~D[2000-01-01]
+      ~D[2000-01-01]
+
+  Both `new/3` and sigil return a struct where the date fields can
+  be accessed directly:
+
+      iex> date = ~D[2000-01-01]
+      iex> date.year
+      2000
+      iex> date.month
+      1
+
+  The functions on this module work with the `Date` struct as well
+  as any struct that contains the same fields as the `Date` struct,
+  such as `NaiveDateTime` and `DateTime`. Such functions expect
+  `t:Calendar.date/0` in their typespecs (instead of `t:t/0`).
+
+  Developers should avoid creating the Date structs directly
+  and instead rely on the functions provided by this module as well
+  as the ones in 3rd party calendar libraries.
+
+  ## Comparing dates
+
+  Comparisons in Elixir using `==`, `>`, `<` and similar are structural
+  and based on the `Date` struct fields. For proper comparison between
+  dates, use the `compare/2` function.
+
+  ## Using epochs
+
+  The `add/2` and `diff/2` functions can be used for computing dates
+  or retrieving the amount of days betweens instants. For example, if there
+  is an interest in computing the amount of days from the Unix epoch
+  (1970-01-01):
+
+      iex> Date.diff(~D[2010-04-17], ~D[1970-01-01])
+      14716
+
+      iex> Date.add(~D[1970-01-01], 14716)
+      ~D[2010-04-17]
+
+  Those functions are optimized to deal with common epochs, such
+  as the Unix Epoch above or the Gregorian Epoch (0000-01-01).
+  """
+
+  @enforce_keys [:year, :month, :day]
+  defstruct [:year, :month, :day, calendar: Calendar.ISO]
+
+  @type t :: %Date{year: Calendar.year, month: Calendar.month,
+                   day: Calendar.day, calendar: Calendar.calendar}
+
+  @doc """
+  Returns a range of dates.
+
+  A range of dates represents a discrete number of dates where
+  the first and last values are dates with matching calendars.
+
+  Ranges of dates can be either increasing (`first <= last`) or
+  decreasing (`first > last`). They are also always inclusive.
+
+  ## Examples
+
+      iex> Date.range(~D[1999-01-01], ~D[2000-01-01])
+      #DateRange<~D[1999-01-01], ~D[2000-01-01]>
+
+      iex> Date.range(~N[2000-01-01 09:00:00], ~D[1999-01-01])
+      #DateRange<~N[2000-01-01 09:00:00], ~D[1999-01-01]>
+
+  A range of dates implements the `Enumerable` protocol, which means
+  functions in the `Enum` module can be used to work with
+  ranges:
+
+      iex> range = Date.range(~D[2001-01-01], ~D[2002-01-01])
+      iex> Enum.count(range)
+      366
+      iex> Enum.member?(range, ~D[2001-02-01])
+      true
+      iex> Enum.reduce(range, 0, fn _date, acc -> acc - 1 end)
+      -366
+  """
+
+  @spec range(Calendar.date, Calendar.date) :: Date.Range.t
+  def range(%{calendar: calendar} = first, %{calendar: calendar} = last) do
+    {first_days, _} = to_iso_days(first)
+    {last_days, _} = to_iso_days(last)
+
+    %Date.Range{
+      first: first,
+      last: last,
+      first_in_iso_days: first_days,
+      last_in_iso_days: last_days,
+    }
+  end
+
+  def range(%{calendar: _, year: _, month: _, day: _},
+            %{calendar: _, year: _, month: _, day: _}) do
+    raise ArgumentError, "both dates must have matching calendars"
+  end
+
+  @doc """
+  Returns the current date in UTC.
+
+  ## Examples
+
+      iex> date = Date.utc_today()
+      iex> date.year >= 2016
+      true
+
+  """
+  @spec utc_today(Calendar.calendar) :: t
+  def utc_today(calendar \\ Calendar.ISO)
+
+  def utc_today(Calendar.ISO) do
+    {:ok, {year, month, day}, _, _} = Calendar.ISO.from_unix(System.os_time, :native)
+    %Date{year: year, month: month, day: day}
+  end
+
+  def utc_today(calendar) do
+    calendar
+    |> DateTime.utc_now
+    |> DateTime.to_date
+  end
+
+  @doc """
+  Returns true if the year in the given `date` is a leap year.
+
+  ## Examples
+
+      iex> Date.leap_year?(~D[2000-01-01])
+      true
+      iex> Date.leap_year?(~D[2001-01-01])
+      false
+      iex> Date.leap_year?(~D[2004-01-01])
+      true
+      iex> Date.leap_year?(~D[1900-01-01])
+      false
+      iex> Date.leap_year?(~N[2004-01-01 01:23:45])
+      true
+
+  """
+  @spec leap_year?(Calendar.date) :: boolean()
+  def leap_year?(date)
+
+  def leap_year?(%{calendar: calendar, year: year}) do
+    calendar.leap_year?(year)
+  end
+
+  @doc """
+  Returns the number of days in the given `date` month.
+
+  ## Examples
+
+      iex> Date.days_in_month(~D[1900-01-13])
+      31
+      iex> Date.days_in_month(~D[1900-02-09])
+      28
+      iex> Date.days_in_month(~N[2000-02-20 01:23:45])
+      29
+
+  """
+  @spec days_in_month(Calendar.date) :: Calendar.day
+  def days_in_month(date)
+
+  def days_in_month(%{calendar: calendar, year: year, month: month}) do
+    calendar.days_in_month(year, month)
+  end
+
+  @doc """
+  Builds a new ISO date.
+
+  Expects all values to be integers. Returns `{:ok, date}` if each
+  entry fits its appropriate range, returns `{:error, reason}` otherwise.
+
+  ## Examples
+
+      iex> Date.new(2000, 1, 1)
+      {:ok, ~D[2000-01-01]}
+      iex> Date.new(2000, 13, 1)
+      {:error, :invalid_date}
+      iex> Date.new(2000, 2, 29)
+      {:ok, ~D[2000-02-29]}
+
+      iex> Date.new(2000, 2, 30)
+      {:error, :invalid_date}
+      iex> Date.new(2001, 2, 29)
+      {:error, :invalid_date}
+
+  """
+  @spec new(Calendar.year, Calendar.month, Calendar.day) :: {:ok, t} | {:error, atom}
+  def new(year, month, day, calendar \\ Calendar.ISO) do
+    if calendar.valid_date?(year, month, day) do
+      {:ok, %Date{year: year, month: month, day: day, calendar: calendar}}
+    else
+      {:error, :invalid_date}
+    end
+  end
+
+  @doc """
+  Converts the given date to a string according to its calendar.
+
+  ### Examples
+
+      iex> Date.to_string(~D[2000-02-28])
+      "2000-02-28"
+      iex> Date.to_string(~N[2000-02-28 01:23:45])
+      "2000-02-28"
+
+  """
+  @spec to_string(Calendar.date) :: String.t
+  def to_string(date)
+
+  def to_string(%{calendar: calendar, year: year, month: month, day: day}) do
+    calendar.date_to_string(year, month, day)
+  end
+
+  @doc """
+  Parses the extended "Dates" format described by
+  [ISO 8601:2004](https://en.wikipedia.org/wiki/ISO_8601).
+
+  ## Examples
+
+      iex> Date.from_iso8601("2015-01-23")
+      {:ok, ~D[2015-01-23]}
+
+      iex> Date.from_iso8601("2015:01:23")
+      {:error, :invalid_format}
+
+      iex> Date.from_iso8601("2015-01-32")
+      {:error, :invalid_date}
+
+  """
+  @spec from_iso8601(String.t) :: {:ok, t} | {:error, atom}
+  def from_iso8601(string, calendar \\ Calendar.ISO)
+
+  def from_iso8601(<<year::4-bytes, ?-, month::2-bytes, ?-, day::2-bytes>>, calendar) do
+    with {year, ""} <- Integer.parse(year),
+         {month, ""} <- Integer.parse(month),
+         {day, ""} <- Integer.parse(day) do
+      with {:ok, date} <- new(year, month, day, Calendar.ISO),
+           do: convert(date, calendar)
+    else
+      _ -> {:error, :invalid_format}
+    end
+  end
+
+  def from_iso8601(<<_::binary>>, _calendar) do
+    {:error, :invalid_format}
+  end
+
+  @doc """
+  Parses the extended "Dates" format described by
+  [ISO 8601:2004](https://en.wikipedia.org/wiki/ISO_8601).
+
+  Raises if the format is invalid.
+
+  ## Examples
+
+      iex> Date.from_iso8601!("2015-01-23")
+      ~D[2015-01-23]
+      iex> Date.from_iso8601!("2015:01:23")
+      ** (ArgumentError) cannot parse "2015:01:23" as date, reason: :invalid_format
+  """
+  @spec from_iso8601!(String.t) :: t
+  def from_iso8601!(string, calendar \\ Calendar.ISO) do
+    case from_iso8601(string, calendar) do
+      {:ok, value} ->
+        value
+      {:error, reason} ->
+        raise ArgumentError, "cannot parse #{inspect string} as date, reason: #{inspect reason}"
+    end
+  end
+
+  @doc """
+  Converts the given `date` to
+  [ISO 8601:2004](https://en.wikipedia.org/wiki/ISO_8601).
+
+  By default, `Date.to_iso8601/2` returns dates formatted in the "extended"
+  format, for human readability. It also supports the "basic" format through passing the `:basic` option.
+
+  Only supports converting dates which are in the ISO calendar,
+  or other calendars in which the days also start at midnight.
+  Attempting to convert dates from other calendars will raise an `ArgumentError`.
+
+  ### Examples
+
+      iex> Date.to_iso8601(~D[2000-02-28])
+      "2000-02-28"
+
+      iex> Date.to_iso8601(~D[2000-02-28], :basic)
+      "20000228"
+
+      iex> Date.to_iso8601(~N[2000-02-28 00:00:00])
+      "2000-02-28"
+
+  """
+  @spec to_iso8601(Calendar.date, :extended | :basic) :: String.t
+  def to_iso8601(date, format \\ :extended) when format in [:basic, :extended] do
+    %{year: year, month: month, day: day} = convert!(date, Calendar.ISO)
+    Calendar.ISO.date_to_iso8601(year, month, day, format)
+  end
+
+  @doc """
+  Converts the given `date` to an Erlang date tuple.
+
+  Only supports converting dates which are in the ISO calendar,
+  or other calendars in which the days also start at midnight.
+  Attempting to convert dates from other calendars will raise.
+
+  ## Examples
+
+      iex> Date.to_erl(~D[2000-01-01])
+      {2000, 1, 1}
+
+      iex> Date.to_erl(~N[2000-01-01 00:00:00])
+      {2000, 1, 1}
+
+  """
+  @spec to_erl(Calendar.date) :: :calendar.date
+  def to_erl(date) do
+    %{year: year, month: month, day: day} = convert!(date, Calendar.ISO)
+    {year, month, day}
+  end
+
+  @doc """
+  Converts an Erlang date tuple to a `Date` struct.
+
+  Only supports converting dates which are in the ISO calendar,
+  or other calendars in which the days also start at midnight.
+  Attempting to convert dates from other calendars will return an error tuple.
+
+  ## Examples
+
+      iex> Date.from_erl({2000, 1, 1})
+      {:ok, ~D[2000-01-01]}
+      iex> Date.from_erl({2000, 13, 1})
+      {:error, :invalid_date}
+
+  """
+  @spec from_erl(:calendar.date) :: {:ok, t} | {:error, atom}
+  def from_erl(tuple, calendar \\ Calendar.ISO)
+
+  def from_erl({year, month, day}, calendar) do
+    with {:ok, date} <- new(year, month, day, Calendar.ISO),
+         do: convert(date, calendar)
+  end
+
+  @doc """
+  Converts an Erlang date tuple but raises for invalid dates.
+
+  ## Examples
+
+      iex> Date.from_erl!({2000, 1, 1})
+      ~D[2000-01-01]
+      iex> Date.from_erl!({2000, 13, 1})
+      ** (ArgumentError) cannot convert {2000, 13, 1} to date, reason: :invalid_date
+
+  """
+  @spec from_erl!(:calendar.date) :: t
+  def from_erl!(tuple) do
+    case from_erl(tuple) do
+      {:ok, value} ->
+        value
+      {:error, reason} ->
+        raise ArgumentError, "cannot convert #{inspect tuple} to date, reason: #{inspect reason}"
+    end
+  end
+
+  @doc """
+  Compares two date structs.
+
+  Returns `:gt` if first date is later than the second
+  and `:lt` for vice versa. If the two dates are equal
+  `:eq` is returned.
+
+  ## Examples
+
+      iex> Date.compare(~D[2016-04-16], ~D[2016-04-28])
+      :lt
+
+  This function can also be used to compare across more
+  complex calendar types by considering only the date fields:
+
+      iex> Date.compare(~D[2016-04-16], ~N[2016-04-28 01:23:45])
+      :lt
+      iex> Date.compare(~D[2016-04-16], ~N[2016-04-16 01:23:45])
+      :eq
+      iex> Date.compare(~N[2016-04-16 12:34:56], ~N[2016-04-16 01:23:45])
+      :eq
+
+  """
+  @spec compare(Calendar.date, Calendar.date) :: :lt | :eq | :gt
+  def compare(%{calendar: calendar, year: year1, month: month1, day: day1},
+              %{calendar: calendar, year: year2, month: month2, day: day2}) do
+    case {{year1, month1, day1}, {year2, month2, day2}} do
+      {first, second} when first > second -> :gt
+      {first, second} when first < second -> :lt
+      _ -> :eq
+    end
+  end
+  def compare(date1, date2) do
+    if Calendar.compatible_calendars?(date1.calendar, date2.calendar) do
+      case {to_iso_days(date1), to_iso_days(date2)} do
+        {first, second} when first > second -> :gt
+        {first, second} when first < second -> :lt
+        _ -> :eq
+      end
+    else
+      raise ArgumentError, """
+      cannot compare #{inspect date1} with #{inspect date2}.
+
+      This comparison would be ambiguous as their calendars have incompatible day rollover moments.
+      Specify an exact time of day (using `DateTime`s) to resolve this ambiguity
+      """
+    end
+  end
+
+  @doc """
+  Converts the given `date` from it's calendar to the given `calendar`.
+
+  Returns `{:ok, date}` if the calendars are compatible,
+  or `{:error, :incompatible_calendars}` if they are not.
+
+  See also `Calendar.compatible_calendars?/2`.
+
+  ## Examples
+
+  Imagine someone implements `Calendar.Holocene`, a calendar based on the
+  Gregorian calendar that adds exactly 10,000 years to the current Gregorian
+  year:
+
+      iex> Date.convert(~D[2000-01-01], Calendar.Holocene)
+      {:ok, %Date{calendar: Calendar.Holocene, year: 12000, month: 1, day: 1}}
+
+  """
+  @spec convert(Calendar.date, Calendar.calendar) :: {:ok, t} | {:error, :incompatible_calendars}
+  def convert(%{calendar: calendar, year: year, month: month, day: day}, calendar) do
+    {:ok, %Date{calendar: calendar, year: year, month: month, day: day}}
+  end
+  def convert(%{calendar: calendar} = date, target_calendar) do
+    if Calendar.compatible_calendars?(calendar, target_calendar) do
+      result_date =
+        date
+        |> to_iso_days()
+        |> from_iso_days(target_calendar)
+      {:ok, result_date}
+    else
+      {:error, :incompatible_calendars}
+    end
+  end
+
+  @doc """
+  Similar to `Date.convert/2`, but raises an `ArgumentError`
+  if the conversion between the two calendars is not possible.
+
+  ## Examples
+
+  Imagine someone implements `Calendar.Holocene`, a calendar based on the
+  Gregorian calendar that adds exactly 10,000 years to the current Gregorian
+  year:
+
+      iex> Date.convert!(~D[2000-01-01], Calendar.Holocene)
+      %Date{calendar: Calendar.Holocene, year: 12000, month: 1, day: 1}
+
+  """
+  @spec convert!(Calendar.date, Calendar.calendar) :: t
+  def convert!(date, calendar) do
+    case convert(date, calendar) do
+      {:ok, value} ->
+        value
+      {:error, reason} ->
+        raise ArgumentError, "cannot convert #{inspect date} to target calendar #{inspect calendar}, reason: #{inspect reason}"
+    end
+  end
+
+  @doc """
+  Adds the number of days to the given `date`.
+
+  The days are counted as gregorian days. The date is returned in the same
+  calendar as it was given in.
+
+  ## Examples
+
+      iex> Date.add(~D[2000-01-03], -2)
+      ~D[2000-01-01]
+      iex> Date.add(~D[2000-01-01], 2)
+      ~D[2000-01-03]
+
+      iex> Date.add(~N[2000-01-01 09:00:00], 2)
+      ~D[2000-01-03]
+
+  """
+  @spec add(Calendar.date, integer()) :: t
+  def add(%{calendar: calendar} = date, days) do
+    {iso_days_days, fraction} = to_iso_days(date)
+    from_iso_days({iso_days_days + days, fraction}, calendar)
+  end
+
+  @doc """
+  Calculates the difference between two dates, in a full number of days.
+
+  It returns the number of gregorian days between the dates. Only `Date`
+  structs that follow the same or compatible calendars can be compared
+  this way. If two calendars are not compatible, it will raise.
+
+  ## Examples
+
+      iex> Date.diff(~D[2000-01-03], ~D[2000-01-01])
+      2
+      iex> Date.diff(~D[2000-01-01], ~D[2000-01-03])
+      -2
+
+      iex> Date.diff(~D[2000-01-01], ~N[2000-01-03 09:00:00])
+      -2
+
+  """
+  @spec diff(Calendar.date, Calendar.date) :: integer
+  def diff(%{calendar: Calendar.ISO, year: year1, month: month1, day: day1},
+           %{calendar: Calendar.ISO, year: year2, month: month2, day: day2}) do
+    Calendar.ISO.date_to_iso_days_days(year1, month1, day1) -
+      Calendar.ISO.date_to_iso_days_days(year2, month2, day2)
+  end
+
+  def diff(%{calendar: calendar1} = date1, %{calendar: calendar2} = date2) do
+    if Calendar.compatible_calendars?(calendar1, calendar2) do
+      {days1, _} = to_iso_days(date1)
+      {days2, _} = to_iso_days(date2)
+      days1 - days2
+    else
+      raise ArgumentError, "cannot calculate the difference between #{inspect date1} and #{inspect date2} because their calendars are not compatible and thus the result would be ambiguous"
+    end
+  end
+
+  defp to_iso_days(%{calendar: Calendar.ISO, year: year, month: month, day: day}) do
+    {Calendar.ISO.date_to_iso_days_days(year, month, day), {0, 86400000000}}
+  end
+  defp to_iso_days(%{calendar: calendar, year: year, month: month, day: day}) do
+    calendar.naive_datetime_to_iso_days(year, month, day, 0, 0, 0, {0, 0})
+  end
+
+  defp from_iso_days({days, _}, Calendar.ISO) do
+    {year, month, day} = Calendar.ISO.date_from_iso_days_days(days)
+    %Date{year: year, month: month, day: day, calendar: Calendar.ISO}
+  end
+  defp from_iso_days(iso_days, target_calendar) do
+    {year, month, day, _, _, _, _} = target_calendar.naive_datetime_from_iso_days(iso_days)
+    %Date{year: year, month: month, day: day, calendar: target_calendar}
+  end
+
+  @doc """
+  Calculates the day of the week of a given `date`.
+
+  Returns the day of the week as an integer. For the ISO 8601
+  calendar (the default), it is an integer from 1 to 7, where
+  1 is Monday and 7 is Sunday.
+
+  ## Examples
+
+      iex> Date.day_of_week(~D[2016-10-31])
+      1
+      iex> Date.day_of_week(~D[2016-11-01])
+      2
+      iex> Date.day_of_week(~N[2016-11-01 01:23:45])
+      2
+
+  """
+  @spec day_of_week(Calendar.date) :: non_neg_integer()
+  def day_of_week(date)
+
+  def day_of_week(%{calendar: calendar, year: year, month: month, day: day}) do
+    calendar.day_of_week(year, month, day)
+  end
+
+  ## Helpers
+
+  defimpl String.Chars do
+    def to_string(%{calendar: calendar, year: year, month: month, day: day}) do
+      calendar.date_to_string(year, month, day)
+    end
+  end
+
+  defimpl Inspect do
+    def inspect(%{calendar: Calendar.ISO, year: year, month: month, day: day}, _) do
+      "~D[" <> Calendar.ISO.date_to_string(year, month, day) <> "]"
+    end
+
+    def inspect(date, opts) do
+      Inspect.Any.inspect(date, opts)
+    end
+  end
+end

lib/elixir/lib/calendar/date_range.ex

@@ -0,0 +1,86 @@
+defmodule Date.Range do
+  @moduledoc """
+  Returns an inclusive range between dates.
+
+  Ranges must be created with the `Date.range/2` function.
+
+  The following fields are public:
+
+    * `:first` - the initial date on the range
+    * `:last` - the last date on the range
+
+  The remaining fields are private and should not be accessed.
+  """
+
+  @type t :: %__MODULE__{first: Date.t, last: Date.t,
+                         first_in_iso_days: Calendar.days,
+                         last_in_iso_days: Calendar.days}
+
+  defstruct [:first, :last, :first_in_iso_days, :last_in_iso_days]
+
+  defimpl Enumerable do
+    def member?(%{first: %{calendar: calendar, year: first_year, month: first_month, day: first_day},
+                  last: %{calendar: calendar, year: last_year, month: last_month, day: last_day},
+                  first_in_iso_days: first_in_iso_days, last_in_iso_days: last_in_iso_days},
+                %Date{calendar: calendar, year: year, month: month, day: day}) do
+      first = {first_year, first_month, first_day}
+      last = {last_year, last_month, last_day}
+      date = {year, month, day}
+
+      if first_in_iso_days <= last_in_iso_days do
+        {:ok, date >= first and date <= last}
+      else
+        {:ok, date >= last and date <= first}
+      end
+    end
+
+    def member?(_, _) do
+      {:ok, false}
+    end
+
+    def count(%Date.Range{first_in_iso_days: first_in_iso_days, last_in_iso_days: last_in_iso_days}) do
+      {:ok, abs(first_in_iso_days - last_in_iso_days) + 1}
+    end
+
+    def reduce(%Date.Range{first_in_iso_days: first_in_iso_days, last_in_iso_days: last_in_iso_days,
+                           first: %{calendar: calendar}}, acc, fun) do
+      reduce(first_in_iso_days, last_in_iso_days, acc, fun, calendar, first_in_iso_days <= last_in_iso_days)
+    end
+
+    defp reduce(_x, _y, {:halt, acc}, _fun, _calendar, _up?) do
+      {:halted, acc}
+    end
+
+    defp reduce(x, y, {:suspend, acc}, fun, calendar, up?) do
+      {:suspended, acc, &reduce(x, y, &1, fun, calendar, up?)}
+    end
+
+    defp reduce(x, y, {:cont, acc}, fun, calendar, up? = true) when x <= y do
+      reduce(x + 1, y, fun.(date_from_iso_days_days(x, calendar), acc), fun, calendar, up?)
+    end
+
+    defp reduce(x, y, {:cont, acc}, fun, calendar, up? = false) when x >= y do
+      reduce(x - 1, y, fun.(date_from_iso_days_days(x, calendar), acc), fun, calendar, up?)
+    end
+
+    defp reduce(_, _, {:cont, acc}, _fun, _calendar, _up) do
+      {:done, acc}
+    end
+
+    defp date_from_iso_days_days(days, Calendar.ISO) do
+      {year, month, day} = Calendar.ISO.date_from_iso_days_days(days)
+      %Date{year: year, month: month, day: day, calendar: Calendar.ISO}
+    end
+
+    defp date_from_iso_days_days(days, calendar) do
+      {year, month, day, _, _, _, _} = calendar.naive_datetime_from_iso_days({days, {0, 86400000000}})
+      %Date{year: year, month: month, day: day, calendar: calendar}
+    end
+  end
+
+  defimpl Inspect do
+    def inspect(%Date.Range{first: first, last: last}, _) do
+      "#DateRange<" <> inspect(first) <> ", " <> inspect(last) <> ">"
+    end
+  end
+end

lib/elixir/lib/calendar/datetime.ex

@@ -0,0 +1,670 @@
+defmodule DateTime do
+  @moduledoc """
+  A datetime implementation with a time zone.
+
+  This datetime can be seen as an ephemeral snapshot
+  of a datetime at a given time zone. For such purposes,
+  it also includes both UTC and Standard offsets, as
+  well as the zone abbreviation field used exclusively
+  for formatting purposes.
+
+  Remember, comparisons in Elixir using `==`, `>`, `<` and friends
+  are structural and based on the DateTime struct fields. For proper
+  comparison between datetimes, use the `compare/2` function.
+
+  The functions on this module work with the `DateTime` struct as well
+  as any struct that contains the same fields as the `DateTime` struct.
+  Such functions expect `t:Calendar.datetime/0` in their typespecs
+  (instead of `t:t/0`).
+
+  Developers should avoid creating the DateTime struct directly
+  and instead rely on the functions provided by this module as
+  well as the ones in 3rd party calendar libraries.
+
+  ## Where are my functions?
+
+  You will notice this module only contains conversion
+  functions as well as functions that work on UTC. This
+  is because a proper DateTime implementation requires a
+  TimeZone database which currently is not provided as part
+  of Elixir.
+
+  Such may be addressed in upcoming versions, meanwhile,
+  use 3rd party packages to provide DateTime building and
+  similar functionality with time zone backing.
+  """
+
+  @enforce_keys [:year, :month, :day, :hour, :minute, :second,
+                 :time_zone, :zone_abbr, :utc_offset, :std_offset]
+  defstruct [:year, :month, :day, :hour, :minute, :second, :time_zone,
+             :zone_abbr, :utc_offset, :std_offset, microsecond: {0, 0}, calendar: Calendar.ISO]
+
+  @type t :: %__MODULE__{year: Calendar.year, month: Calendar.month, day: Calendar.day,
+                         calendar: Calendar.calendar, hour: Calendar.hour, minute: Calendar.minute,
+                         second: Calendar.second, microsecond: Calendar.microsecond,
+                         time_zone: Calendar.time_zone, zone_abbr: Calendar.zone_abbr,
+                         utc_offset: Calendar.utc_offset, std_offset: Calendar.std_offset}
+
+  @unix_days :calendar.date_to_gregorian_days({1970, 1, 1})
+
+  @doc """
+  Returns the current datetime in UTC.
+
+  ## Examples
+
+      iex> datetime = DateTime.utc_now()
+      iex> datetime.time_zone
+      "Etc/UTC"
+
+  """
+  @spec utc_now(Calendar.calendar) :: t
+  def utc_now(calendar \\ Calendar.ISO) do
+    System.os_time |> from_unix!(:native, calendar)
+  end
+
+  @doc """
+  Converts the given Unix time to `DateTime`.
+
+  The integer can be given in different unit
+  according to `System.convert_time_unit/3` and it will
+  be converted to microseconds internally.
+
+  Unix times are always in UTC and therefore the DateTime
+  will be returned in UTC.
+
+  ## Examples
+
+      iex> {:ok, datetime} = DateTime.from_unix(1464096368)
+      iex> datetime
+      #DateTime<2016-05-24 13:26:08Z>
+
+      iex> {:ok, datetime} = DateTime.from_unix(1432560368868569, :microsecond)
+      iex> datetime
+      #DateTime<2015-05-25 13:26:08.868569Z>
+
+  The unit can also be an integer as in `t:System.time_unit/0`:
+
+      iex> {:ok, datetime} = DateTime.from_unix(143256036886856, 1024)
+      iex> datetime
+      #DateTime<6403-03-17 07:05:22.320Z>
+
+  Negative Unix times are supported, up to -62167219200 seconds,
+  which is equivalent to "0000-01-01T00:00:00Z" or 0 Gregorian seconds.
+  """
+  @spec from_unix(integer, :native | System.time_unit, Calendar.calendar) :: {:ok, t} | {:error, atom}
+  def from_unix(integer, unit \\ :second, calendar \\ Calendar.ISO) when is_integer(integer) do
+    case Calendar.ISO.from_unix(integer, unit) do
+      {:ok, {year, month, day}, {hour, minute, second}, microsecond} ->
+        iso_datetime = %DateTime{year: year, month: month, day: day,
+                                 hour: hour, minute: minute, second: second, microsecond: microsecond,
+                                 std_offset: 0, utc_offset: 0, zone_abbr: "UTC", time_zone: "Etc/UTC"}
+        convert(iso_datetime, calendar)
+      {:error, _} = error ->
+        error
+    end
+  end
+
+  @doc """
+  Converts the given Unix time to `DateTime`.
+
+  The integer can be given in different unit
+  according to `System.convert_time_unit/3` and it will
+  be converted to microseconds internally.
+
+  Unix times are always in UTC and therefore the DateTime
+  will be returned in UTC.
+
+  ## Examples
+
+      # An easy way to get the Unix epoch is passing 0 to this function
+      iex> DateTime.from_unix!(0)
+      #DateTime<1970-01-01 00:00:00Z>
+
+      iex> DateTime.from_unix!(1464096368)
+      #DateTime<2016-05-24 13:26:08Z>
+
+      iex> DateTime.from_unix!(1432560368868569, :microsecond)
+      #DateTime<2015-05-25 13:26:08.868569Z>
+
+  """
+  @spec from_unix!(integer, :native | System.time_unit, Calendar.calendar) :: t
+  def from_unix!(integer, unit \\ :second, calendar \\ Calendar.ISO) when is_atom(unit) do
+    case from_unix(integer, unit, calendar) do
+      {:ok, datetime} ->
+        datetime
+      {:error, :invalid_unix_time} ->
+        raise ArgumentError, "invalid Unix time #{integer}"
+    end
+  end
+
+  @doc """
+  Converts the given `NaiveDateTime` to `DateTime`.
+
+  It expects a time zone to put the NaiveDateTime in.
+  Currently it only supports "Etc/UTC" as time zone.
+
+  ## Examples
+
+      iex> {:ok, datetime} = DateTime.from_naive(~N[2016-05-24 13:26:08.003], "Etc/UTC")
+      iex> datetime
+      #DateTime<2016-05-24 13:26:08.003Z>
+
+  """
+  @spec from_naive(NaiveDateTime.t, Calendar.time_zone) :: {:ok, t}
+  def from_naive(naive_datetime, time_zone)
+
+  def from_naive(%NaiveDateTime{calendar: calendar,
+                                hour: hour, minute: minute, second: second, microsecond: microsecond,
+                                year: year, month: month, day: day}, "Etc/UTC") do
+    {:ok, %DateTime{calendar: calendar, year: year, month: month, day: day,
+                    hour: hour, minute: minute, second: second, microsecond: microsecond,
+                    std_offset: 0, utc_offset: 0, zone_abbr: "UTC", time_zone: "Etc/UTC"}}
+  end
+
+  @doc """
+  Converts the given `NaiveDateTime` to `DateTime`.
+
+  It expects a time zone to put the NaiveDateTime in.
+  Currently it only supports "Etc/UTC" as time zone.
+
+  ## Examples
+
+      iex> DateTime.from_naive!(~N[2016-05-24 13:26:08.003], "Etc/UTC")
+      #DateTime<2016-05-24 13:26:08.003Z>
+
+  """
+  @spec from_naive!(NaiveDateTime.t, Calendar.time_zone) :: t
+  def from_naive!(naive_datetime, time_zone) do
+    case from_naive(naive_datetime, time_zone) do
+      {:ok, datetime} ->
+        datetime
+      {:error, reason} ->
+        raise ArgumentError, "cannot parse #{inspect naive_datetime} to datetime, reason: #{inspect reason}"
+    end
+  end
+
+  @doc """
+  Converts the given `datetime` to Unix time.
+
+  The `datetime` is expected to be using the ISO calendar
+  with a year greater than or equal to 0.
+
+  It will return the integer with the given unit,
+  according to `System.convert_time_unit/3`.
+
+  ## Examples
+
+      iex> 1464096368 |> DateTime.from_unix!() |> DateTime.to_unix()
+      1464096368
+
+      iex> dt = %DateTime{calendar: Calendar.ISO, day: 20, hour: 18, microsecond: {273806, 6},
+      ...>                minute: 58, month: 11, second: 19, time_zone: "America/Montevideo",
+      ...>                utc_offset: -10800, std_offset: 3600, year: 2014, zone_abbr: "UYST"}
+      iex> DateTime.to_unix(dt)
+      1416517099
+
+      iex> flamel = %DateTime{calendar: Calendar.ISO, day: 22, hour: 8, microsecond: {527771, 6},
+      ...>                minute: 2, month: 3, second: 25, std_offset: 0, time_zone: "Etc/UTC",
+      ...>                utc_offset: 0, year: 1418, zone_abbr: "UTC"}
+      iex> DateTime.to_unix(flamel)
+      -17412508655
+
+  """
+  @spec to_unix(Calendar.datetime, System.time_unit) :: integer
+  def to_unix(datetime, unit \\ :second)
+
+  def to_unix(%{utc_offset: utc_offset, std_offset: std_offset} = datetime, unit) do
+    {days, fraction} = to_iso_days(datetime)
+    unix_units = Calendar.ISO.iso_days_to_unit({days - @unix_days, fraction}, unit)
+    offset_units = System.convert_time_unit(utc_offset + std_offset, :second, unit)
+    unix_units - offset_units
+  end
+
+  @doc """
+  Converts the given `datetime` into a `NaiveDateTime`.
+
+  Because `NaiveDateTime` does not hold time zone information,
+  any time zone related data will be lost during the conversion.
+
+  ## Examples
+
+      iex> dt = %DateTime{year: 2000, month: 2, day: 29, zone_abbr: "CET",
+      ...>                hour: 23, minute: 0, second: 7, microsecond: {0, 1},
+      ...>                utc_offset: 3600, std_offset: 0, time_zone: "Europe/Warsaw"}
+      iex> DateTime.to_naive(dt)
+      ~N[2000-02-29 23:00:07.0]
+
+  """
+  @spec to_naive(t) :: NaiveDateTime.t
+  def to_naive(%DateTime{calendar: calendar, year: year, month: month, day: day,
+                         hour: hour, minute: minute, second: second, microsecond: microsecond}) do
+    %NaiveDateTime{year: year, month: month, day: day, calendar: calendar,
+                   hour: hour, minute: minute, second: second, microsecond: microsecond}
+  end
+
+  @doc """
+  Converts a `DateTime` into a `Date`.
+
+  Because `Date` does not hold time nor time zone information,
+  data will be lost during the conversion.
+
+  ## Examples
+
+      iex> dt = %DateTime{year: 2000, month: 2, day: 29, zone_abbr: "CET",
+      ...>                hour: 23, minute: 0, second: 7, microsecond: {0, 0},
+      ...>                utc_offset: 3600, std_offset: 0, time_zone: "Europe/Warsaw"}
+      iex> DateTime.to_date(dt)
+      ~D[2000-02-29]
+
+  """
+  @spec to_date(t) :: Date.t
+  def to_date(%DateTime{year: year, month: month, day: day, calendar: calendar}) do
+    %Date{year: year, month: month, day: day, calendar: calendar}
+  end
+
+  @doc """
+  Converts a `DateTime` into `Time`.
+
+  Because `Time` does not hold date nor time zone information,
+  data will be lost during the conversion.
+
+  ## Examples
+
+      iex> dt = %DateTime{year: 2000, month: 2, day: 29, zone_abbr: "CET",
+      ...>                hour: 23, minute: 0, second: 7, microsecond: {0, 1},
+      ...>                utc_offset: 3600, std_offset: 0, time_zone: "Europe/Warsaw"}
+      iex> DateTime.to_time(dt)
+      ~T[23:00:07.0]
+
+  """
+  @spec to_time(t) :: Time.t
+  def to_time(%DateTime{hour: hour, minute: minute, second: second, microsecond: microsecond, calendar: calendar}) do
+    %Time{hour: hour, minute: minute, second: second, microsecond: microsecond, calendar: calendar}
+  end
+
+  @doc """
+  Converts the given datetime to
+  [ISO 8601:2004](https://en.wikipedia.org/wiki/ISO_8601) format.
+
+  By default, `DateTime.to_iso8601/2` returns datetimes formatted in the "extended"
+  format, for human readability. It also supports the "basic" format through passing the `:basic` option.
+
+  Only supports converting datetimes which are in the ISO calendar,
+  attempting to convert datetimes from other calendars will raise.
+
+  WARNING: the ISO 8601 datetime format does not contain the time zone nor
+  its abbreviation, which means information is lost when converting to such
+  format.
+
+  ### Examples
+
+      iex> dt = %DateTime{year: 2000, month: 2, day: 29, zone_abbr: "CET",
+      ...>                hour: 23, minute: 0, second: 7, microsecond: {0, 0},
+      ...>                utc_offset: 3600, std_offset: 0, time_zone: "Europe/Warsaw"}
+      iex> DateTime.to_iso8601(dt)
+      "2000-02-29T23:00:07+01:00"
+
+      iex> dt = %DateTime{year: 2000, month: 2, day: 29, zone_abbr: "UTC",
+      ...>                hour: 23, minute: 0, second: 7, microsecond: {0, 0},
+      ...>                utc_offset: 0, std_offset: 0, time_zone: "Etc/UTC"}
+      iex> DateTime.to_iso8601(dt)
+      "2000-02-29T23:00:07Z"
+
+      iex> dt = %DateTime{year: 2000, month: 2, day: 29, zone_abbr: "AMT",
+      ...>                hour: 23, minute: 0, second: 7, microsecond: {0, 0},
+      ...>                utc_offset: -14400, std_offset: 0, time_zone: "America/Manaus"}
+      iex> DateTime.to_iso8601(dt, :extended)
+      "2000-02-29T23:00:07-04:00"
+
+      iex> dt = %DateTime{year: 2000, month: 2, day: 29, zone_abbr: "AMT",
+      ...>                hour: 23, minute: 0, second: 7, microsecond: {0, 0},
+      ...>                utc_offset: -14400, std_offset: 0, time_zone: "America/Manaus"}
+      iex> DateTime.to_iso8601(dt, :basic)
+      "20000229T230007-0400"
+
+  """
+  @spec to_iso8601(Calendar.datetime, :extended | :basic ) :: String.t
+  def to_iso8601(datetime, format \\ :extended)
+
+  def to_iso8601(_, format) when format not in [:extended, :basic] do
+    raise ArgumentError, "DateTime.to_iso8601/2 expects format to be :extended or :basic, got: #{inspect format}"
+  end
+
+  def to_iso8601(%{calendar: Calendar.ISO, year: year, month: month, day: day,
+                  hour: hour, minute: minute, second: second, microsecond: microsecond,
+                  time_zone: time_zone, zone_abbr: zone_abbr, utc_offset: utc_offset, std_offset: std_offset}, format) do
+    Calendar.ISO.datetime_to_iso8601(year, month, day, hour, minute, second, microsecond,
+                                     time_zone, zone_abbr, utc_offset, std_offset, format)
+  end
+
+  def to_iso8601(%{calendar: _, year: _, month: _, day: _,
+                   hour: _, minute: _, second: _, microsecond: _,
+                   time_zone: _, zone_abbr: _, utc_offset: _, std_offset: _} = datetime, format) do
+    datetime
+    |> convert!(Calendar.ISO)
+    |> to_iso8601(format)
+  end
+
+  @doc """
+  Parses the extended "Date and time of day" format described by
+  [ISO 8601:2004](https://en.wikipedia.org/wiki/ISO_8601).
+
+  Since ISO8601 does not include the proper time zone, the given
+  string will be converted to UTC and its offset in seconds will be
+  returned as part of this function. Therefore offset information
+  must be present in the string.
+
+  As specified in the standard, the separator "T" may be omitted if
+  desired as there is no ambiguity within this function.
+
+  Time representations with reduced accuracy are not supported.
+
+  Note that while ISO8601 allows datetimes to specify 24:00:00 as the
+  zero hour of the next day, this notation is not supported by Elixir.
+
+  ## Examples
+
+      iex> {:ok, datetime, 0} = DateTime.from_iso8601("2015-01-23T23:50:07Z")
+      iex> datetime
+      #DateTime<2015-01-23 23:50:07Z>
+
+      iex> {:ok, datetime, 9000} = DateTime.from_iso8601("2015-01-23T23:50:07.123+02:30")
+      iex> datetime
+      #DateTime<2015-01-23 21:20:07.123Z>
+
+      iex> {:ok, datetime, 9000} = DateTime.from_iso8601("2015-01-23T23:50:07,123+02:30")
+      iex> datetime
+      #DateTime<2015-01-23 21:20:07.123Z>
+
+      iex> DateTime.from_iso8601("2015-01-23P23:50:07")
+      {:error, :invalid_format}
+      iex> DateTime.from_iso8601("2015-01-23 23:50:07A")
+      {:error, :invalid_format}
+      iex> DateTime.from_iso8601("2015-01-23T23:50:07")
+      {:error, :missing_offset}
+      iex> DateTime.from_iso8601("2015-01-23 23:50:61")
+      {:error, :invalid_time}
+      iex> DateTime.from_iso8601("2015-01-32 23:50:07")
+      {:error, :invalid_date}
+
+      iex> DateTime.from_iso8601("2015-01-23T23:50:07.123-00:00")
+      {:error, :invalid_format}
+      iex> DateTime.from_iso8601("2015-01-23T23:50:07.123-00:60")
+      {:error, :invalid_format}
+
+  """
+  @spec from_iso8601(String.t, Calendar.calendar) :: {:ok, t, Calendar.utc_offset} | {:error, atom}
+  def from_iso8601(string, calendar \\ Calendar.ISO)
+
+  def from_iso8601(<<year::4-bytes, ?-, month::2-bytes, ?-, day::2-bytes, sep,
+                     hour::2-bytes, ?:, min::2-bytes, ?:, sec::2-bytes, rest::binary>>, calendar) when sep in [?\s, ?T] do
+    with {year, ""} <- Integer.parse(year),
+         {month, ""} <- Integer.parse(month),
+         {day, ""} <- Integer.parse(day),
+         {hour, ""} <- Integer.parse(hour),
+         {minute, ""} <- Integer.parse(min),
+         {second, ""} <- Integer.parse(sec),
+         {microsecond, rest} <- Calendar.ISO.parse_microsecond(rest),
+         {:ok, date} <- Date.new(year, month, day),
+         {:ok, time} <- Time.new(hour, minute, second, microsecond),
+         {:ok, offset} <- parse_offset(rest) do
+      %{year: year, month: month, day: day} = date
+      %{hour: hour, minute: minute, second: second, microsecond: microsecond} = time
+      {_, precision} = microsecond
+
+      datetime =
+        Calendar.ISO.naive_datetime_to_iso_days(year, month, day, hour, minute, second, microsecond)
+        |> apply_tz_offset(offset)
+        |> from_iso_days("Etc/UTC", "UTC", 0, 0, calendar, precision)
+
+      {:ok, %{datetime | microsecond: microsecond}, offset}
+    else
+      {:error, reason} -> {:error, reason}
+      _ -> {:error, :invalid_format}
+    end
+  end
+
+  def from_iso8601(_, _) do
+    {:error, :invalid_format}
+  end
+
+  defp parse_offset(rest) do
+    case Calendar.ISO.parse_offset(rest) do
+      {offset, ""} when is_integer(offset) -> {:ok, offset}
+      {nil, ""} -> {:error, :missing_offset}
+      _ -> {:error, :invalid_format}
+    end
+  end
+
+  @doc """
+  Converts the given `datetime` to a string according to its calendar.
+
+  ### Examples
+
+      iex> dt = %DateTime{year: 2000, month: 2, day: 29, zone_abbr: "CET",
+      ...>                hour: 23, minute: 0, second: 7, microsecond: {0, 0},
+      ...>                utc_offset: 3600, std_offset: 0, time_zone: "Europe/Warsaw"}
+      iex> DateTime.to_string(dt)
+      "2000-02-29 23:00:07+01:00 CET Europe/Warsaw"
+
+      iex> dt = %DateTime{year: 2000, month: 2, day: 29, zone_abbr: "UTC",
+      ...>                hour: 23, minute: 0, second: 7, microsecond: {0, 0},
+      ...>                utc_offset: 0, std_offset: 0, time_zone: "Etc/UTC"}
+      iex> DateTime.to_string(dt)
+      "2000-02-29 23:00:07Z"
+
+      iex> dt = %DateTime{year: 2000, month: 2, day: 29, zone_abbr: "AMT",
+      ...>                hour: 23, minute: 0, second: 7, microsecond: {0, 0},
+      ...>                utc_offset: -14400, std_offset: 0, time_zone: "America/Manaus"}
+      iex> DateTime.to_string(dt)
+      "2000-02-29 23:00:07-04:00 AMT America/Manaus"
+
+  """
+  @spec to_string(Calendar.datetime) :: String.t
+  def to_string(datetime)
+
+  def to_string(%{calendar: calendar, year: year, month: month, day: day,
+                  hour: hour, minute: minute, second: second, microsecond: microsecond,
+                  time_zone: time_zone, zone_abbr: zone_abbr, utc_offset: utc_offset, std_offset: std_offset}) do
+    calendar.datetime_to_string(year, month, day, hour, minute, second, microsecond,
+                                time_zone, zone_abbr, utc_offset, std_offset)
+  end
+
+  defimpl String.Chars do
+    def to_string(%{calendar: calendar, year: year, month: month, day: day,
+                    hour: hour, minute: minute, second: second, microsecond: microsecond,
+                    time_zone: time_zone, zone_abbr: zone_abbr, utc_offset: utc_offset, std_offset: std_offset}) do
+      calendar.datetime_to_string(year, month, day, hour, minute, second, microsecond,
+                                  time_zone, zone_abbr, utc_offset, std_offset)
+    end
+  end
+
+  defimpl Inspect do
+    def inspect(%{calendar: Calendar.ISO, year: year, month: month, day: day,
+                  hour: hour, minute: minute, second: second, microsecond: microsecond,
+                  time_zone: time_zone, zone_abbr: zone_abbr, utc_offset: utc_offset, std_offset: std_offset}, _) do
+      "#DateTime<" <> Calendar.ISO.datetime_to_string(year, month, day, hour, minute, second, microsecond,
+                                                      time_zone, zone_abbr, utc_offset, std_offset) <> ">"
+    end
+
+    def inspect(datetime, opts) do
+      Inspect.Any.inspect(datetime, opts)
+    end
+  end
+
+  @doc """
+  Compares two datetime structs.
+
+  Returns `:gt` if first datetime is later than the second
+  and `:lt` for vice versa. If the two datetimes are equal
+  `:eq` is returned.
+
+  Note that both utc and stc offsets will be taken into
+  account when comparison is done.
+
+  ## Examples
+
+      iex> dt1 = %DateTime{year: 2000, month: 2, day: 29, zone_abbr: "AMT",
+      ...>                 hour: 23, minute: 0, second: 7, microsecond: {0, 0},
+      ...>                 utc_offset: -14400, std_offset: 0, time_zone: "America/Manaus"}
+      iex> dt2 = %DateTime{year: 2000, month: 2, day: 29, zone_abbr: "CET",
+      ...>                 hour: 23, minute: 0, second: 7, microsecond: {0, 0},
+      ...>                 utc_offset: 3600, std_offset: 0, time_zone: "Europe/Warsaw"}
+      iex> DateTime.compare(dt1, dt2)
+      :gt
+
+  """
+  @spec compare(Calendar.datetime, Calendar.datetime) :: :lt | :eq | :gt
+  def compare(%DateTime{utc_offset: utc_offset1, std_offset: std_offset1} = datetime1,
+              %DateTime{utc_offset: utc_offset2, std_offset: std_offset2} = datetime2) do
+    {days1, {parts1, ppd1}} =
+      datetime1
+      |> to_iso_days()
+      |> apply_tz_offset(utc_offset1 + std_offset1)
+
+    {days2, {parts2, ppd2}} =
+      datetime2
+      |> to_iso_days()
+      |> apply_tz_offset(utc_offset2 + std_offset2)
+
+    # Ensure fraction tuples have same denominator.
+    iso_days1 = {days1, parts1 * ppd2}
+    iso_days2 = {days2, parts2 * ppd1}
+
+    case {iso_days1, iso_days2}  do
+      {first, second} when first > second -> :gt
+      {first, second} when first < second -> :lt
+      _ -> :eq
+    end
+  end
+
+  @doc """
+  Subtracts `datetime2` from `datetime1`.
+
+  The answer can be returned in any `unit` available from `t:System.time_unit/0`.
+
+  This function returns the difference in seconds where seconds are measured
+  according to `Calendar.ISO`.
+
+  ## Examples
+
+      iex> dt1 = %DateTime{year: 2000, month: 2, day: 29, zone_abbr: "AMT",
+      ...>                 hour: 23, minute: 0, second: 7, microsecond: {0, 0},
+      ...>                 utc_offset: -14400, std_offset: 0, time_zone: "America/Manaus"}
+      iex> dt2 = %DateTime{year: 2000, month: 2, day: 29, zone_abbr: "CET",
+      ...>                 hour: 23, minute: 0, second: 7, microsecond: {0, 0},
+      ...>                 utc_offset: 3600, std_offset: 0, time_zone: "Europe/Warsaw"}
+      iex> DateTime.diff(dt1, dt2)
+      18000
+      iex> DateTime.diff(dt2, dt1)
+      -18000
+
+  """
+  @spec diff(Calendar.datetime, Calendar.datetime) :: integer()
+  def diff(%{utc_offset: utc_offset1, std_offset: std_offset1} = datetime1,
+           %{utc_offset: utc_offset2, std_offset: std_offset2} = datetime2, unit \\ :second) do
+    naive_diff =
+      (datetime1 |> to_iso_days() |> Calendar.ISO.iso_days_to_unit(unit)) -
+      (datetime2 |> to_iso_days() |> Calendar.ISO.iso_days_to_unit(unit))
+    offset_diff =
+      (utc_offset2 + std_offset2) - (utc_offset1 + std_offset1)
+    naive_diff + System.convert_time_unit(offset_diff, :second, unit)
+  end
+
+  @doc """
+  Converts a given `datetime` from one calendar to another.
+
+  If it is not possible to convert unambiguously between the calendars
+  (see `Calendar.compatible_calendars?/2`), an `{:error, :incompatible_calendars}` tuple
+  is returned.
+
+  ## Examples
+
+  Imagine someone implements `Calendar.Holocene`, a calendar based on the
+  Gregorian calendar that adds exactly 10,000 years to the current Gregorian
+  year:
+
+      iex> dt1 = %DateTime{year: 2000, month: 2, day: 29, zone_abbr: "AMT",
+      ...>                 hour: 23, minute: 0, second: 7, microsecond: {0, 0},
+      ...>                 utc_offset: -14400, std_offset: 0, time_zone: "America/Manaus"}
+      iex> DateTime.convert(dt1, Calendar.Holocene)
+      {:ok, %DateTime{calendar: Calendar.Holocene, day: 29, hour: 23,
+                      microsecond: {0, 0}, minute: 0, month: 2, second: 7, std_offset: 0,
+                      time_zone: "America/Manaus", utc_offset: -14400, year: 12000,
+                      zone_abbr: "AMT"}}
+
+  """
+  @spec convert(Calendar.datetime, Calendar.calendar) :: {:ok, t} | {:error, :incompatible_calendars}
+  def convert(%{calendar: calendar, year: year, month: month, day: day,
+                hour: hour, minute: minute, second: second, microsecond: microsecond,
+                time_zone: time_zone, zone_abbr: zone_abbr, utc_offset: utc_offset, std_offset: std_offset}, calendar) do
+    {:ok, %DateTime{calendar: calendar, year: year, month: month, day: day,
+                    hour: hour, minute: minute, second: second, microsecond: microsecond,
+                    time_zone: time_zone, zone_abbr: zone_abbr, utc_offset: utc_offset, std_offset: std_offset}}
+  end
+
+  def convert(%{calendar: dt_calendar, microsecond: {_, precision}} = datetime, calendar) do
+    if Calendar.compatible_calendars?(dt_calendar, calendar) do
+      result_datetime =
+        datetime
+        |> to_iso_days
+        |> from_iso_days(datetime, calendar, precision)
+      {:ok, result_datetime}
+    else
+      {:error, :incompatible_calendars}
+    end
+  end
+
+  @doc """
+  Converts a given `datetime` from one calendar to another.
+
+  If it is not possible to convert unambiguously between the calendars
+  (see `Calendar.compatible_calendars?/2`), an ArgumentError is raised.
+
+  ## Examples
+
+  Imagine someone implements `Calendar.Holocene`, a calendar based on the
+  Gregorian calendar that adds exactly 10,000 years to the current Gregorian
+  year:
+
+      iex> dt1 = %DateTime{year: 2000, month: 2, day: 29, zone_abbr: "AMT",
+      ...>                 hour: 23, minute: 0, second: 7, microsecond: {0, 0},
+      ...>                 utc_offset: -14400, std_offset: 0, time_zone: "America/Manaus"}
+      iex> DateTime.convert!(dt1, Calendar.Holocene)
+      %DateTime{calendar: Calendar.Holocene, day: 29, hour: 23,
+                microsecond: {0, 0}, minute: 0, month: 2, second: 7, std_offset: 0,
+                time_zone: "America/Manaus", utc_offset: -14400, year: 12000,
+                zone_abbr: "AMT"}
+
+  """
+  @spec convert!(Calendar.datetime, Calendar.calendar) :: t | no_return
+  def convert!(datetime, calendar) do
+    case convert(datetime, calendar) do
+      {:ok, value} ->
+        value
+      {:error, :incompatible_calendars} ->
+        raise ArgumentError, "cannot convert #{inspect datetime} to target calendar #{inspect calendar}, reason: #{inspect datetime.calendar} and #{inspect calendar} have different day rollover moments, making this conversion ambiguous"
+    end
+  end
+
+  defp to_iso_days(%{calendar: calendar,year: year, month: month, day: day,
+                     hour: hour, minute: minute, second: second, microsecond: microsecond}) do
+    calendar.naive_datetime_to_iso_days(year, month, day, hour, minute, second, microsecond)
+  end
+
+  defp from_iso_days(iso_days, datetime, calendar, precision) do
+    %{time_zone: time_zone, zone_abbr: zone_abbr, utc_offset: utc_offset, std_offset: std_offset} = datetime
+    from_iso_days(iso_days, time_zone, zone_abbr, utc_offset, std_offset, calendar, precision)
+  end
+
+  defp from_iso_days(iso_days, time_zone, zone_abbr, utc_offset, std_offset, calendar, precision) do
+    {year, month, day, hour, minute, second, {microsecond, _}} = calendar.naive_datetime_from_iso_days(iso_days)
+    %DateTime{calendar: calendar, year: year, month: month, day: day,
+              hour: hour, minute: minute, second: second, microsecond: {microsecond, precision},
+              time_zone: time_zone, zone_abbr: zone_abbr, utc_offset: utc_offset, std_offset: std_offset}
+  end
+
+  defp apply_tz_offset(iso_days, offset) do
+    Calendar.ISO.add_day_fraction_to_iso_days(iso_days, -offset, 86400)
+  end
+end

lib/elixir/lib/calendar/iso.ex

@@ -7,14 +7,136 @@ defmodule Calendar.ISO do
   today. The proleptic means the Gregorian rules for leap years are
   applied for all time, consequently the dates give different results
   before the year 1583 from when the Gregorian calendar was adopted.
+
+  Note that while ISO8601 allows times and datetimes to specify
+  24:00:00 as the zero hour of the next day, this notation is not
+  supported by Elixir.
   """
 
   @behaviour Calendar
-  @unix_epoch :calendar.datetime_to_gregorian_seconds {{1970, 1, 1}, {0, 0, 0}}
 
-  @type year  :: 0..9999
+  @unix_epoch 62167219200
+  @unix_start 1_000_000 * -@unix_epoch
+  @unix_end 315569519999999999 - @unix_epoch * 1_000_000
+  @unix_range_microseconds @unix_start..@unix_end
+
+  @type year :: 0..9999
   @type month :: 1..12
-  @type day   :: 1..31
+  @type day :: 1..31
+
+  @seconds_per_minute 60
+  @seconds_per_hour 60 * 60
+  @seconds_per_day 24 * 60 * 60 # Note that this does _not_ handle leap seconds.
+  @microseconds_per_second 1_000_000
+
+  @doc """
+  Returns the `t:Calendar.iso_days` format of the specified date.
+
+  ## Examples
+
+      iex> Calendar.ISO.naive_datetime_to_iso_days(0, 1, 1, 0, 0, 0, {0, 6})
+      {0, {0, 86400000000}}
+      iex> Calendar.ISO.naive_datetime_to_iso_days(2000, 1, 1, 12, 0, 0, {0, 6})
+      {730485, {43200000000, 86400000000}}
+      iex> Calendar.ISO.naive_datetime_to_iso_days(2000, 1, 1, 13, 0, 0, {0, 6})
+      {730485, {46800000000, 86400000000}}
+
+  """
+  @spec naive_datetime_to_iso_days(Calendar.year, Calendar.month, Calendar.day,
+                                   Calendar.hour, Calendar.minute, Calendar.second,
+                                   Calendar.microsecond) :: Calendar.iso_days
+  def naive_datetime_to_iso_days(year, month, day, hour, minute, second, microsecond) do
+    {date_to_iso_days_days(year, month, day),
+     time_to_day_fraction(hour, minute, second, microsecond)}
+  end
+
+  @doc """
+  Converts the `t:Calendar.iso_days` format to the datetime format specified by this calendar.
+
+  ## Examples
+
+      iex> Calendar.ISO.naive_datetime_from_iso_days({0, {0, 86400}})
+      {0, 1, 1, 0, 0, 0, {0, 6}}
+      iex> Calendar.ISO.naive_datetime_from_iso_days({730485, {0, 86400}})
+      {2000, 1, 1, 0, 0, 0, {0, 6}}
+      iex> Calendar.ISO.naive_datetime_from_iso_days({730485, {43200, 86400}})
+      {2000, 1, 1, 12, 0, 0, {0, 6}}
+
+  """
+  @spec naive_datetime_from_iso_days(Calendar.iso_days) ::
+        {Calendar.year, Calendar.month, Calendar.day,
+         Calendar.hour, Calendar.minute, Calendar.second, Calendar.microsecond}
+  def naive_datetime_from_iso_days({days, day_fraction}) do
+    {year, month, day} = date_from_iso_days_days(days)
+    {hour, minute, second, microsecond} = time_from_day_fraction(day_fraction)
+    {year, month, day, hour, minute, second, microsecond}
+  end
+
+  @doc """
+  Returns the normalized day fraction of the specified time.
+
+  ## Examples
+
+      iex> Calendar.ISO.time_to_day_fraction(0, 0, 0, {0, 6})
+      {0, 86400000000}
+      iex> Calendar.ISO.time_to_day_fraction(12, 34, 56, {123, 6})
+      {45296000123, 86400000000}
+
+  """
+  @spec time_to_day_fraction(Calendar.hour, Calendar.minute,
+                             Calendar.second, Calendar.microsecond) :: Calendar.day_fraction
+  def time_to_day_fraction(0, 0, 0, {0, _}) do
+    {0, 86400000000}
+  end
+  def time_to_day_fraction(hour, minute, second, {microsecond, _}) do
+    combined_seconds = hour * @seconds_per_hour + minute * @seconds_per_minute + second
+    {combined_seconds * @microseconds_per_second + microsecond, @seconds_per_day * @microseconds_per_second}
+  end
+
+  @doc """
+  Converts a day fraction to this Calendar's representation of time.
+
+  ## Examples
+
+      iex> Calendar.ISO.time_from_day_fraction({1,2})
+      {12, 0, 0, {0, 6}}
+      iex> Calendar.ISO.time_from_day_fraction({13,24})
+      {13, 0, 0, {0, 6}}
+
+  """
+  @spec time_from_day_fraction(Calendar.day_fraction) ::
+        {Calendar.hour, Calendar.minute, Calendar.second, Calendar.microsecond}
+  def time_from_day_fraction({parts_in_day, parts_per_day}) do
+    total_microseconds = div(parts_in_day * @seconds_per_day * @microseconds_per_second, parts_per_day)
+    {hours, rest_microseconds1} = div_mod(total_microseconds, @seconds_per_hour * @microseconds_per_second)
+    {minutes, rest_microseconds2} = div_mod(rest_microseconds1, @seconds_per_minute * @microseconds_per_second)
+    {seconds, microseconds} = div_mod(rest_microseconds2, @microseconds_per_second)
+    {hours, minutes, seconds, {microseconds, 6}}
+  end
+
+  # Converts year, month, day to count of days since 0000-01-01.
+  @doc false
+  def date_to_iso_days_days(0, 1, 1) do
+    0
+  end
+  def date_to_iso_days_days(1970, 1, 1) do
+    719528
+  end
+  def date_to_iso_days_days(year, month, day) when year <= 9999 do
+    :calendar.date_to_gregorian_days(year, month, day)
+  end
+
+  # Converts count of days since 0000-01-01 to {year, month, day} tuple.
+  @doc false
+  def date_from_iso_days_days(days) when days <= 3652424 do
+    :calendar.gregorian_days_to_date(days)
+  end
+
+  defp div_mod(int1, int2) do
+    div = div(int1, int2)
+    mod = int1 - (div * int2)
+    {div, mod}
+  end
 
   @doc """
   Returns how many days there are in the given year-month.
@@ -92,6 +214,28 @@ defmodule Calendar.ISO do
     :calendar.day_of_the_week(year, month, day)
   end
 
+  @doc """
+  Converts the given time into a string.
+  """
+  def time_to_string(hour, minute, second, microsecond, format \\ :extended)
+
+  def time_to_string(hour, minute, second, {_, 0}, format) do
+    time_to_string_format(hour, minute, second, format)
+  end
+
+  def time_to_string(hour, minute, second, {microsecond, precision}, format) do
+    time_to_string_format(hour, minute, second, format) <>
+      "." <> (microsecond |> zero_pad(6) |> binary_part(0, precision))
+  end
+
+  defp time_to_string_format(hour, minute, second, :extended) do
+    zero_pad(hour, 2) <> ":" <> zero_pad(minute, 2) <> ":" <> zero_pad(second, 2)
+  end
+
+  defp time_to_string_format(hour, minute, second, :basic) do
+    zero_pad(hour, 2) <> zero_pad(minute, 2) <> zero_pad(second, 2)
+  end
+
   @doc """
   Converts the given date into a string.
   """
@@ -99,6 +243,11 @@ defmodule Calendar.ISO do
     zero_pad(year, 4) <> "-" <> zero_pad(month, 2) <> "-" <> zero_pad(day, 2)
   end
 
+  defp date_to_string(year, month, day, :extended), do: date_to_string(year, month, day)
+  defp date_to_string(year, month, day, :basic) do
+    zero_pad(year, 4) <> zero_pad(month, 2) <> zero_pad(day, 2)
+  end
+
   @doc """
   Converts the datetime (without time zone) into a string.
   """
@@ -117,15 +266,37 @@ defmodule Calendar.ISO do
       zone_to_string(utc_offset, std_offset, zone_abbr, time_zone)
   end
 
-  defp offset_to_string(0, 0, "Etc/UTC"), do: "Z"
-  defp offset_to_string(utc, std, _zone) do
-    total  = utc + std
+  def valid_date?(year, month, day) do
+    year <= 9999 and :calendar.valid_date(year, month, day)
+  end
+
+  def valid_time?(hour, minute, second, {microsecond, precision}) do
+    hour in 0..23 and minute in 0..59 and second in 0..60 and
+      microsecond in 0..999_999 and precision in 0..6
+  end
+
+  def day_rollover_relative_to_midnight_utc() do
+    {0, 1}
+  end
+
+  defp offset_to_string(utc, std, zone, format \\ :extended)
+  defp offset_to_string(0, 0, "Etc/UTC", _format), do: "Z"
+  defp offset_to_string(utc, std, _zone, format) do
+    total = utc + std
     second = abs(total)
     minute = second |> rem(3600) |> div(60)
-    hour   = second |> div(3600)
+    hour = div(second, 3600)
+    format_offset(total, hour, minute, format)
+  end
+
+  defp format_offset(total, hour, minute, :extended) do
     sign(total) <> zero_pad(hour, 2) <> ":" <> zero_pad(minute, 2)
   end
 
+  defp format_offset(total, hour, minute, :basic) do
+    sign(total) <> zero_pad(hour, 2) <> zero_pad(minute, 2)
+  end
+
   defp zone_to_string(0, 0, _abbr, "Etc/UTC"), do: ""
   defp zone_to_string(_, _, abbr, zone), do: " " <> abbr <> " " <> zone
 
@@ -139,37 +310,17 @@ defmodule Calendar.ISO do
 
   ## Helpers
 
-  @doc false
-  def time_to_string(hour, minute, second, {_, 0}) do
-    time_to_string(hour, minute, second)
-  end
-  def time_to_string(hour, minute, second, {microsecond, precision}) do
-    time_to_string(hour, minute, second) <> "." <>
-      (microsecond |> zero_pad(6) |> binary_part(0, precision))
-  end
-  defp time_to_string(hour, minute, second) do
-    zero_pad(hour, 2) <> ":" <> zero_pad(minute, 2) <> ":" <> zero_pad(second, 2)
-  end
-
-  @doc false
-  def date(year, month, day) when is_integer(year) and is_integer(month) and is_integer(day) do
-    if :calendar.valid_date(year, month, day) and year <= 9999 do
-      {:ok, %Date{year: year, month: month, day: day}}
-    else
-      {:error, :invalid_date}
-    end
-  end
-
   @doc false
   def from_unix(integer, unit) when is_integer(integer) do
     total = System.convert_time_unit(integer, unit, :microsecond)
-    if total < -@unix_epoch * 1_000_000 do
-      {:error, :invalid_unix_time}
-    else
-      microsecond = rem(total, 1_000_000)
+    if total in @unix_range_microseconds do
+      microseconds = Integer.mod(total, @microseconds_per_second)
+      seconds = @unix_epoch + Integer.floor_div(total, @microseconds_per_second)
       precision = precision_for_unit(unit)
-      {date, time} = :calendar.gregorian_seconds_to_datetime(@unix_epoch + div(total, 1_000_000))
-      {:ok, date, time, {microsecond, precision}}
+      {date, time} = :calendar.gregorian_seconds_to_datetime(seconds)
+      {:ok, date, time, {microseconds, precision}}
+    else
+      {:error, :invalid_unix_time}
     end
   end
 
@@ -186,26 +337,26 @@ defmodule Calendar.ISO do
     do: precision_for_unit(div(number, 10), precision + 1)
 
   @doc false
-  def date_to_iso8601(year, month, day) do
-    date_to_string(year, month, day)
+  def date_to_iso8601(year, month, day, format \\ :extended) do
+    date_to_string(year, month, day, format)
   end
 
   @doc false
-  def time_to_iso8601(hour, minute, second, microsecond) do
-    time_to_string(hour, minute, second, microsecond)
+  def time_to_iso8601(hour, minute, second, microsecond, format \\ :extended) do
+    time_to_string(hour, minute, second, microsecond, format)
   end
 
   @doc false
-  def naive_datetime_to_iso8601(year, month, day, hour, minute, second, microsecond) do
-    date_to_string(year, month, day) <> "T" <> time_to_string(hour, minute, second, microsecond)
+  def naive_datetime_to_iso8601(year, month, day, hour, minute, second, microsecond, format \\ :extended) do
+    date_to_string(year, month, day, format) <> "T" <> time_to_string(hour, minute, second, microsecond, format)
   end
 
   @doc false
   def datetime_to_iso8601(year, month, day, hour, minute, second, microsecond,
-                          time_zone, _zone_abbr, utc_offset, std_offset) do
-    date_to_string(year, month, day) <> "T" <>
-      time_to_string(hour, minute, second, microsecond) <>
-      offset_to_string(utc_offset, std_offset, time_zone)
+                          time_zone, _zone_abbr, utc_offset, std_offset, format \\ :extended) do
+    date_to_string(year, month, day, format) <> "T" <>
+      time_to_string(hour, minute, second, microsecond, format) <>
+      offset_to_string(utc_offset, std_offset, time_zone, format)
   end
 
   @doc false
@@ -220,12 +371,17 @@ defmodule Calendar.ISO do
         {{String.to_integer(binary_part(microsecond, 0, 6)), 6}, rest}
     end
   end
+
+  def parse_microsecond("," <> rest) do
+    parse_microsecond("." <> rest)
+  end
+
   def parse_microsecond(rest) do
     {{0, 0}, rest}
   end
 
-  defp parse_microsecond(<<h, t::binary>>, precision, acc) when h in ?0..?9,
-    do: parse_microsecond(t, precision + 1, <<acc::binary, h>>)
+  defp parse_microsecond(<<head, tail::binary>>, precision, acc) when head in ?0..?9,
+    do: parse_microsecond(tail, precision + 1, <<acc::binary, head>>)
   defp parse_microsecond(rest, precision, acc),
     do: {acc, precision, rest}
 
@@ -259,4 +415,34 @@ defmodule Calendar.ISO do
       _ -> :error
     end
   end
+
+  @doc false
+  def iso_days_to_unit({days, {parts, ppd}}, unit) do
+    day_microseconds = days * @seconds_per_day * @microseconds_per_second
+    microseconds = div(parts * @seconds_per_day * @microseconds_per_second, ppd)
+    System.convert_time_unit(day_microseconds + microseconds, :microsecond, unit)
+  end
+
+  @doc false
+  def add_day_fraction_to_iso_days({days, {parts, ppd}}, add, ppd) do
+    normalize_iso_days(days, parts + add, ppd)
+  end
+  def add_day_fraction_to_iso_days({days, {parts, ppd}}, add, add_ppd) do
+    parts = parts * add_ppd
+    add = add * ppd
+    gcd = Integer.gcd(ppd, add_ppd)
+    result_parts = div(parts + add, gcd)
+    result_ppd = div(ppd * add_ppd, gcd)
+    normalize_iso_days(days, result_parts, result_ppd)
+  end
+
+  defp normalize_iso_days(days, parts, ppd) do
+    days_offset = div(parts, ppd)
+    parts = rem(parts, ppd)
+    if parts < 0 do
+      {days + days_offset - 1, {parts + ppd, ppd}}
+    else
+      {days + days_offset, {parts, ppd}}
+    end
+  end
 end

lib/elixir/lib/calendar/naive_datetime.ex

@@ -0,0 +1,695 @@
+defmodule NaiveDateTime do
+  @moduledoc """
+  A NaiveDateTime struct (without a time zone) and functions.
+
+  The NaiveDateTime struct contains the fields year, month, day, hour,
+  minute, second, microsecond and calendar. New naive datetimes can be
+  built with the `new/2` and `new/7` functions or using the `~N` sigil:
+
+      iex> ~N[2000-01-01 23:00:07]
+      ~N[2000-01-01 23:00:07]
+
+  The date and time fields in the struct can be accessed directly:
+
+      iex> naive = ~N[2000-01-01 23:00:07]
+      iex> naive.year
+      2000
+      iex> naive.second
+      7
+
+  We call them "naive" because this datetime representation does not
+  have a time zone. This means the datetime may not actually exist in
+  certain areas in the world even though it is valid.
+
+  For example, when daylight saving changes are applied by a region,
+  the clock typically moves forward or backward by one hour. This means
+  certain datetimes never occur or may occur more than once. Since
+  `NaiveDateTime` is not validated against a time zone, such errors
+  would go unnoticed.
+
+  The functions on this module work with the `NaiveDateTime` struct as well
+  as any struct that contains the same fields as the `NaiveDateTime` struct,
+  such as `DateTime`. Such functions expect
+  `t:Calendar.naive_datetime/0` in their typespecs (instead of `t:t/0`).
+
+  Developers should avoid creating the NaiveDateTime structs directly
+  and instead rely on the functions provided by this module as well
+  as the ones in 3rd party calendar libraries.
+
+  ## Comparing naive date times
+
+  Comparisons in Elixir using `==`, `>`, `<` and similar are structural
+  and based on the `NaiveDateTime` struct fields. For proper comparison
+  between naive datetimes, use the `compare/2` function.
+
+  ## Using epochs
+
+  The `add/3` and `diff/3` functions can be used for computing with
+  date times or retrieving the amount of seconds betweens instants.
+  For example, if there is an interest in computing the amount of
+  seconds from the Unix epoch (1970-01-01 00:00:00):
+
+      iex> NaiveDateTime.diff(~N[2010-04-17 14:00:00], ~N[1970-01-01 00:00:00])
+      1271512800
+
+      iex> NaiveDateTime.add(~N[1970-01-01 00:00:00], 1271512800)
+      ~N[2010-04-17 14:00:00]
+
+  Those functions are optimized to deal with common epochs, such
+  as the Unix Epoch above or the Gregorian Epoch (0000-01-01 00:00:00).
+  """
+
+  @enforce_keys [:year, :month, :day, :hour, :minute, :second]
+  defstruct [:year, :month, :day, :hour, :minute, :second, microsecond: {0, 0}, calendar: Calendar.ISO]
+
+  @type t :: %NaiveDateTime{year: Calendar.year, month: Calendar.month, day: Calendar.day,
+                            calendar: Calendar.calendar, hour: Calendar.hour, minute: Calendar.minute,
+                            second: Calendar.second, microsecond: Calendar.microsecond}
+
+  @doc """
+  Returns the current naive datetime in UTC.
+
+  Prefer using `DateTime.utc_now/0` when possible as, opposite
+  to `NaiveDateTime`, it will keep the time zone information.
+
+  ## Examples
+
+      iex> naive_datetime = NaiveDateTime.utc_now()
+      iex> naive_datetime.year >= 2016
+      true
+
+  """
+  @spec utc_now(Calendar.calendar) :: t
+  def utc_now(calendar \\ Calendar.ISO)
+
+  def utc_now(Calendar.ISO) do
+    {:ok, {year, month, day}, {hour, minute, second}, microsecond} =
+      Calendar.ISO.from_unix(:os.system_time, :native)
+    %NaiveDateTime{year: year, month: month, day: day,
+                   hour: hour, minute: minute, second: second,
+                   microsecond: microsecond, calendar: Calendar.ISO}
+  end
+
+  def utc_now(calendar) do
+    calendar
+    |> DateTime.utc_now
+    |> DateTime.to_naive
+  end
+
+  @doc """
+  Builds a new ISO naive datetime.
+
+  Expects all values to be integers. Returns `{:ok, naive_datetime}`
+  if each entry fits its appropriate range, returns `{:error, reason}`
+  otherwise.
+
+  ## Examples
+
+      iex> NaiveDateTime.new(2000, 1, 1, 0, 0, 0)
+      {:ok, ~N[2000-01-01 00:00:00]}
+      iex> NaiveDateTime.new(2000, 13, 1, 0, 0, 0)
+      {:error, :invalid_date}
+      iex> NaiveDateTime.new(2000, 2, 29, 0, 0, 0)
+      {:ok, ~N[2000-02-29 00:00:00]}
+      iex> NaiveDateTime.new(2000, 2, 30, 0, 0, 0)
+      {:error, :invalid_date}
+      iex> NaiveDateTime.new(2001, 2, 29, 0, 0, 0)
+      {:error, :invalid_date}
+
+      iex> NaiveDateTime.new(2000, 1, 1, 23, 59, 59, {0, 1})
+      {:ok, ~N[2000-01-01 23:59:59.0]}
+      iex> NaiveDateTime.new(2000, 1, 1, 23, 59, 59, 999_999)
+      {:ok, ~N[2000-01-01 23:59:59.999999]}
+      iex> NaiveDateTime.new(2000, 1, 1, 23, 59, 60, 999_999)
+      {:ok, ~N[2000-01-01 23:59:60.999999]}
+      iex> NaiveDateTime.new(2000, 1, 1, 24, 59, 59, 999_999)
+      {:error, :invalid_time}
+      iex> NaiveDateTime.new(2000, 1, 1, 23, 60, 59, 999_999)
+      {:error, :invalid_time}
+      iex> NaiveDateTime.new(2000, 1, 1, 23, 59, 61, 999_999)
+      {:error, :invalid_time}
+      iex> NaiveDateTime.new(2000, 1, 1, 23, 59, 59, 1_000_000)
+      {:error, :invalid_time}
+
+  """
+  @spec new(Calendar.year, Calendar.month, Calendar.day,
+            Calendar.hour, Calendar.minute, Calendar.second, Calendar.microsecond, Calendar.calendar) ::
+        {:ok, t} | {:error, atom}
+  def new(year, month, day, hour, minute, second, microsecond \\ {0, 0}, calendar \\ Calendar.ISO) do
+    with {:ok, date} <- Date.new(year, month, day, calendar),
+         {:ok, time} <- Time.new(hour, minute, second, microsecond, calendar),
+         do: new(date, time)
+  end
+
+  @doc """
+  Builds a naive datetime from date and time structs.
+
+  ## Examples
+
+      iex> NaiveDateTime.new(~D[2010-01-13], ~T[23:00:07.005])
+      {:ok, ~N[2010-01-13 23:00:07.005]}
+
+  """
+  @spec new(Date.t, Time.t) :: {:ok, t}
+  def new(date, time)
+
+  def new(%Date{calendar: calendar, year: year, month: month, day: day},
+          %Time{calendar: calendar, hour: hour, minute: minute, second: second, microsecond: microsecond}) do
+    {:ok, %NaiveDateTime{calendar: calendar, year: year, month: month, day: day,
+                         hour: hour, minute: minute, second: second, microsecond: microsecond}}
+  end
+
+  @doc """
+  Adds a specified amount of time to a `NaiveDateTime`.
+
+  Accepts an `integer` in any `unit` available from `t:System.time_unit/0`.
+  Negative values will be move backwards in time.
+
+  This operation is only possible if both calendars are convertible to `Calendar.ISO`.
+
+  ## Examples
+
+      # adds seconds by default
+      iex> NaiveDateTime.add(~N[2014-10-02 00:29:10], 2)
+      ~N[2014-10-02 00:29:12]
+
+      # accepts negative offsets
+      iex> NaiveDateTime.add(~N[2014-10-02 00:29:10], -2)
+      ~N[2014-10-02 00:29:08]
+
+      # can work with other units
+      iex> NaiveDateTime.add(~N[2014-10-02 00:29:10], 2_000, :millisecond)
+      ~N[2014-10-02 00:29:12]
+
+      # keeps the same precision
+      iex> NaiveDateTime.add(~N[2014-10-02 00:29:10.021], 21, :second)
+      ~N[2014-10-02 00:29:31.021]
+
+      # changes below the precision will not be visible
+      iex> hidden = NaiveDateTime.add(~N[2014-10-02 00:29:10], 21, :millisecond)
+      iex> hidden.microsecond  # ~N[2014-10-02 00:29:10]
+      {21000, 0}
+
+      # from Gregorian seconds
+      iex> NaiveDateTime.add(~N[0000-01-01 00:00:00], 63579428950)
+      ~N[2014-10-02 00:29:10]
+
+  """
+  @spec add(t, integer, System.time_unit) :: t
+  def add(%NaiveDateTime{microsecond: {_, precision}, calendar: calendar} = naive_datetime,
+          integer, unit \\ :second) when is_integer(integer) do
+    ppd = System.convert_time_unit(86400, :second, unit)
+    naive_datetime
+    |> to_iso_days()
+    |> Calendar.ISO.add_day_fraction_to_iso_days(integer, ppd)
+    |> from_iso_days(calendar, precision)
+  end
+
+  @doc """
+  Subtracts `naive_datetime2` from `naive_datetime1`.
+
+  The answer can be returned in any `unit` available from `t:System.time_unit/0`.
+
+  This function returns the difference in seconds where seconds are measured
+  according to `Calendar.ISO`.
+
+  ## Examples
+
+      iex> NaiveDateTime.diff(~N[2014-10-02 00:29:12], ~N[2014-10-02 00:29:10])
+      2
+      iex> NaiveDateTime.diff(~N[2014-10-02 00:29:12], ~N[2014-10-02 00:29:10], :microsecond)
+      2_000_000
+      iex> NaiveDateTime.diff(~N[2014-10-02 00:29:10.042], ~N[2014-10-02 00:29:10.021], :millisecond)
+      21
+      iex> NaiveDateTime.diff(~N[2014-10-02 00:29:10], ~N[2014-10-02 00:29:12])
+      -2
+
+      # to Gregorian seconds
+      iex> NaiveDateTime.diff(~N[2014-10-02 00:29:10], ~N[0000-01-01 00:00:00])
+      63579428950
+
+  """
+  @spec diff(t, t, System.time_unit) :: integer
+  def diff(%NaiveDateTime{} = naive_datetime1,
+           %NaiveDateTime{} = naive_datetime2,
+           unit \\ :second) do
+    if not Calendar.compatible_calendars?(naive_datetime1.calendar, naive_datetime2.calendar) do
+      raise ArgumentError, "cannot calculate the difference between #{inspect naive_datetime1} and #{inspect naive_datetime2} because their calendars are not compatible and thus the result would be ambiguous"
+    end
+
+    units1 = naive_datetime1 |> to_iso_days() |> Calendar.ISO.iso_days_to_unit(unit)
+    units2 = naive_datetime2 |> to_iso_days() |> Calendar.ISO.iso_days_to_unit(unit)
+    units1 - units2
+  end
+
+  @doc """
+  Converts a `NaiveDateTime` into a `Date`.
+
+  Because `Date` does not hold time information,
+  data will be lost during the conversion.
+
+  ## Examples
+
+      iex> NaiveDateTime.to_date(~N[2002-01-13 23:00:07])
+      ~D[2002-01-13]
+
+  """
+  @spec to_date(t) :: Date.t
+  def to_date(%NaiveDateTime{year: year, month: month, day: day, calendar: calendar}) do
+    %Date{year: year, month: month, day: day, calendar: calendar}
+  end
+
+  @doc """
+  Converts a `NaiveDateTime` into `Time`.
+
+  Because `Time` does not hold date information,
+  data will be lost during the conversion.
+
+  ## Examples
+
+      iex> NaiveDateTime.to_time(~N[2002-01-13 23:00:07])
+      ~T[23:00:07]
+
+  """
+  @spec to_time(t) :: Time.t
+  def to_time(%NaiveDateTime{hour: hour, minute: minute, second: second, microsecond: microsecond, calendar: calendar}) do
+    %Time{hour: hour, minute: minute, second: second, microsecond: microsecond, calendar: calendar}
+  end
+
+  @doc """
+  Converts the given naive datetime to a string according to its calendar.
+
+  ### Examples
+
+      iex> NaiveDateTime.to_string(~N[2000-02-28 23:00:13])
+      "2000-02-28 23:00:13"
+      iex> NaiveDateTime.to_string(~N[2000-02-28 23:00:13.001])
+      "2000-02-28 23:00:13.001"
+
+  This function can also be used to convert a DateTime to a string without
+  the time zone information:
+
+      iex> dt = %DateTime{year: 2000, month: 2, day: 29, zone_abbr: "CET",
+      ...>                hour: 23, minute: 0, second: 7, microsecond: {0, 0},
+      ...>                utc_offset: 3600, std_offset: 0, time_zone: "Europe/Warsaw"}
+      iex> NaiveDateTime.to_string(dt)
+      "2000-02-29 23:00:07"
+
+  """
+  @spec to_string(Calendar.naive_datetime) :: String.t
+  def to_string(naive_datetime)
+
+  def to_string(%{calendar: calendar, year: year, month: month, day: day,
+                  hour: hour, minute: minute, second: second, microsecond: microsecond}) do
+    calendar.naive_datetime_to_string(year, month, day, hour, minute, second, microsecond)
+  end
+
+  @doc """
+  Parses the extended "Date and time of day" format described by
+  [ISO 8601:2004](https://en.wikipedia.org/wiki/ISO_8601).
+
+  Timezone offset may be included in the string but they will be
+  simply discarded as such information is not included in naive date
+  times.
+
+  As specified in the standard, the separator "T" may be omitted if
+  desired as there is no ambiguity within this function.
+
+  Time representations with reduced accuracy are not supported.
+
+  Note that while ISO8601 allows datetimes to specify 24:00:00 as the
+  zero hour of the next day, this notation is not supported by Elixir.
+
+  ## Examples
+
+      iex> NaiveDateTime.from_iso8601("2015-01-23 23:50:07")
+      {:ok, ~N[2015-01-23 23:50:07]}
+      iex> NaiveDateTime.from_iso8601("2015-01-23T23:50:07")
+      {:ok, ~N[2015-01-23 23:50:07]}
+      iex> NaiveDateTime.from_iso8601("2015-01-23T23:50:07Z")
+      {:ok, ~N[2015-01-23 23:50:07]}
+
+      iex> NaiveDateTime.from_iso8601("2015-01-23 23:50:07.0")
+      {:ok, ~N[2015-01-23 23:50:07.0]}
+      iex> NaiveDateTime.from_iso8601("2015-01-23 23:50:07,0123456")
+      {:ok, ~N[2015-01-23 23:50:07.012345]}
+      iex> NaiveDateTime.from_iso8601("2015-01-23 23:50:07.0123456")
+      {:ok, ~N[2015-01-23 23:50:07.012345]}
+      iex> NaiveDateTime.from_iso8601("2015-01-23T23:50:07.123Z")
+      {:ok, ~N[2015-01-23 23:50:07.123]}
+
+      iex> NaiveDateTime.from_iso8601("2015-01-23P23:50:07")
+      {:error, :invalid_format}
+      iex> NaiveDateTime.from_iso8601("2015:01:23 23-50-07")
+      {:error, :invalid_format}
+      iex> NaiveDateTime.from_iso8601("2015-01-23 23:50:07A")
+      {:error, :invalid_format}
+      iex> NaiveDateTime.from_iso8601("2015-01-23 23:50:61")
+      {:error, :invalid_time}
+      iex> NaiveDateTime.from_iso8601("2015-01-32 23:50:07")
+      {:error, :invalid_date}
+
+      iex> NaiveDateTime.from_iso8601("2015-01-23T23:50:07.123+02:30")
+      {:ok, ~N[2015-01-23 23:50:07.123]}
+      iex> NaiveDateTime.from_iso8601("2015-01-23T23:50:07.123+00:00")
+      {:ok, ~N[2015-01-23 23:50:07.123]}
+      iex> NaiveDateTime.from_iso8601("2015-01-23T23:50:07.123-02:30")
+      {:ok, ~N[2015-01-23 23:50:07.123]}
+      iex> NaiveDateTime.from_iso8601("2015-01-23T23:50:07.123-00:00")
+      {:error, :invalid_format}
+      iex> NaiveDateTime.from_iso8601("2015-01-23T23:50:07.123-00:60")
+      {:error, :invalid_format}
+      iex> NaiveDateTime.from_iso8601("2015-01-23T23:50:07.123-24:00")
+      {:error, :invalid_format}
+
+  """
+  @spec from_iso8601(String.t, Calendar.calendar) :: {:ok, t} | {:error, atom}
+  def from_iso8601(string, calendar \\ Calendar.ISO)
+
+  def from_iso8601(<<year::4-bytes, ?-, month::2-bytes, ?-, day::2-bytes, sep,
+                     hour::2-bytes, ?:, min::2-bytes, ?:, sec::2-bytes, rest::binary>>, calendar) when sep in [?\s, ?T] do
+    with {year, ""} <- Integer.parse(year),
+         {month, ""} <- Integer.parse(month),
+         {day, ""} <- Integer.parse(day),
+         {hour, ""} <- Integer.parse(hour),
+         {min, ""} <- Integer.parse(min),
+         {sec, ""} <- Integer.parse(sec),
+         {microsec, rest} <- Calendar.ISO.parse_microsecond(rest),
+         {_offset, ""} <- Calendar.ISO.parse_offset(rest) do
+      with {:ok, utc_date} <- new(year, month, day, hour, min, sec, microsec, Calendar.ISO),
+           do: convert(utc_date, calendar)
+    else
+      _ -> {:error, :invalid_format}
+    end
+  end
+
+  def from_iso8601(<<_::binary>>, _calendar) do
+    {:error, :invalid_format}
+  end
+
+  @doc """
+  Parses the extended "Date and time of day" format described by
+  [ISO 8601:2004](https://en.wikipedia.org/wiki/ISO_8601).
+
+  Raises if the format is invalid.
+
+  ## Examples
+
+      iex> NaiveDateTime.from_iso8601!("2015-01-23T23:50:07.123Z")
+      ~N[2015-01-23 23:50:07.123]
+      iex> NaiveDateTime.from_iso8601!("2015-01-23T23:50:07,123Z")
+      ~N[2015-01-23 23:50:07.123]
+      iex> NaiveDateTime.from_iso8601!("2015-01-23P23:50:07")
+      ** (ArgumentError) cannot parse "2015-01-23P23:50:07" as naive datetime, reason: :invalid_format
+
+  """
+  @spec from_iso8601!(String.t, Calendar.calendar) :: t | no_return
+  def from_iso8601!(string, calendar \\ Calendar.ISO) do
+    case from_iso8601(string, calendar) do
+      {:ok, value} ->
+        value
+      {:error, reason} ->
+        raise ArgumentError, "cannot parse #{inspect string} as naive datetime, reason: #{inspect reason}"
+    end
+  end
+
+  @doc """
+  Converts the given naive datetime to
+  [ISO 8601:2004](https://en.wikipedia.org/wiki/ISO_8601).
+
+  By default, `NaiveDateTime.to_iso8601/2` returns naive datetimes formatted in the "extended"
+  format, for human readability. It also supports the "basic" format through passing the `:basic` option.
+
+  Only supports converting naive datetimes which are in the ISO calendar,
+  attempting to convert naive datetimes from other calendars will raise.
+
+  ### Examples
+
+      iex> NaiveDateTime.to_iso8601(~N[2000-02-28 23:00:13])
+      "2000-02-28T23:00:13"
+
+      iex> NaiveDateTime.to_iso8601(~N[2000-02-28 23:00:13.001])
+      "2000-02-28T23:00:13.001"
+
+      iex> NaiveDateTime.to_iso8601(~N[2000-02-28 23:00:13.001], :basic)
+      "20000228T230013.001"
+
+  This function can also be used to convert a DateTime to ISO8601 without
+  the time zone information:
+
+      iex> dt = %DateTime{year: 2000, month: 2, day: 29, zone_abbr: "CET",
+      ...>                hour: 23, minute: 0, second: 7, microsecond: {0, 0},
+      ...>                utc_offset: 3600, std_offset: 0, time_zone: "Europe/Warsaw"}
+      iex> NaiveDateTime.to_iso8601(dt)
+      "2000-02-29T23:00:07"
+
+  """
+  @spec to_iso8601(Calendar.naive_datetime, :basic | :extended) :: String.t
+  def to_iso8601(naive_datetime, format \\ :extended)
+
+  def to_iso8601(%{year: year, month: month, day: day,
+                   hour: hour, minute: minute, second: second, microsecond: microsecond, calendar:
+    Calendar.ISO}, format) when format in [:basic, :extended] do
+    Calendar.ISO.naive_datetime_to_iso8601(year, month, day, hour, minute, second, microsecond, format)
+  end
+
+  def to_iso8601(%{year: _, month: _, day: _,
+                   hour: _, minute: _, second: _, microsecond: _, calendar: _} = naive_datetime, format) when format in [:basic, :extended] do
+    naive_datetime
+    |> convert!(Calendar.ISO)
+    |> to_iso8601(format)
+  end
+
+  def to_iso8601(_date, format) do
+    raise ArgumentError, "NaiveDateTime.to_iso8601/2 expects format to be :extended or :basic, got: #{inspect format}"
+  end
+
+  @doc """
+  Converts a `NaiveDateTime` struct to an Erlang datetime tuple.
+
+  Only supports converting naive datetimes which are in the ISO calendar,
+  attempting to convert naive datetimes from other calendars will raise.
+
+  WARNING: Loss of precision may occur, as Erlang time tuples only store
+  hour/minute/second.
+
+  ## Examples
+
+      iex> NaiveDateTime.to_erl(~N[2000-01-01 13:30:15])
+      {{2000, 1, 1}, {13, 30, 15}}
+
+  This function can also be used to convert a DateTime to a erl format
+  without the time zone information:
+
+      iex> dt = %DateTime{year: 2000, month: 2, day: 29, zone_abbr: "CET",
+      ...>                hour: 23, minute: 0, second: 7, microsecond: {0, 0},
+      ...>                utc_offset: 3600, std_offset: 0, time_zone: "Europe/Warsaw"}
+      iex> NaiveDateTime.to_erl(dt)
+      {{2000, 2, 29}, {23, 00, 07}}
+
+  """
+  @spec to_erl(t) :: :calendar.datetime
+  def to_erl(naive_datetime)
+
+  @spec to_erl(Calendar.time) :: :calendar.time
+  def to_erl(%{calendar: _, year: _, month: _, day: _,
+               hour: _, minute: _, second: _} = naive_datetime) do
+    %{year: year, month: month, day: day,
+      hour: hour, minute: minute, second: second} = convert!(naive_datetime, Calendar.ISO)
+    {{year, month, day}, {hour, minute, second}}
+  end
+
+  @doc """
+  Converts an Erlang datetime tuple to a `NaiveDateTime` struct.
+
+  Attempting to convert an invalid ISO calendar date will produce an error tuple.
+
+  ## Examples
+
+      iex> NaiveDateTime.from_erl({{2000, 1, 1}, {13, 30, 15}})
+      {:ok, ~N[2000-01-01 13:30:15]}
+      iex> NaiveDateTime.from_erl({{2000, 1, 1}, {13, 30, 15}}, {5000, 3})
+      {:ok, ~N[2000-01-01 13:30:15.005]}
+      iex> NaiveDateTime.from_erl({{2000, 13, 1}, {13, 30, 15}})
+      {:error, :invalid_date}
+      iex> NaiveDateTime.from_erl({{2000, 13, 1},{13, 30, 15}})
+      {:error, :invalid_date}
+
+  """
+  @spec from_erl(:calendar.datetime, Calendar.microsecond) :: {:ok, t} | {:error, atom}
+  def from_erl(tuple, microsecond \\ {0, 0}, calendar \\ Calendar.ISO)
+
+  def from_erl({{year, month, day}, {hour, minute, second}}, microsecond, calendar) do
+    with {:ok, utc_date} <- new(year, month, day, hour, minute, second, microsecond),
+         do: convert(utc_date, calendar)
+  end
+
+   @doc """
+  Converts an Erlang datetime tuple to a `NaiveDateTime` struct.
+
+  Raises if the datetime is invalid.
+  Attempting to convert an invalid ISO calendar date will produce an error tuple.
+
+  ## Examples
+
+      iex> NaiveDateTime.from_erl!({{2000, 1, 1}, {13, 30, 15}})
+      ~N[2000-01-01 13:30:15]
+      iex> NaiveDateTime.from_erl!({{2000, 1, 1}, {13, 30, 15}}, {5000, 3})
+      ~N[2000-01-01 13:30:15.005]
+      iex> NaiveDateTime.from_erl!({{2000, 13, 1}, {13, 30, 15}})
+      ** (ArgumentError) cannot convert {{2000, 13, 1}, {13, 30, 15}} to naive datetime, reason: :invalid_date
+
+  """
+  @spec from_erl!(:calendar.datetime, Calendar.microsecond) :: t | no_return
+  def from_erl!(tuple, microsecond \\ {0, 0}) do
+    case from_erl(tuple, microsecond) do
+      {:ok, value} ->
+        value
+      {:error, reason} ->
+        raise ArgumentError, "cannot convert #{inspect tuple} to naive datetime, reason: #{inspect reason}"
+    end
+  end
+
+  @doc """
+  Compares two `NaiveDateTime` structs.
+
+  Returns `:gt` if first is later than the second
+  and `:lt` for vice versa. If the two NaiveDateTime
+  are equal `:eq` is returned.
+
+  ## Examples
+
+      iex> NaiveDateTime.compare(~N[2016-04-16 13:30:15], ~N[2016-04-28 16:19:25])
+      :lt
+      iex> NaiveDateTime.compare(~N[2016-04-16 13:30:15.1], ~N[2016-04-16 13:30:15.01])
+      :gt
+
+  This function can also be used to compare a DateTime without
+  the time zone information:
+
+      iex> dt = %DateTime{year: 2000, month: 2, day: 29, zone_abbr: "CET",
+      ...>                hour: 23, minute: 0, second: 7, microsecond: {0, 0},
+      ...>                utc_offset: 3600, std_offset: 0, time_zone: "Europe/Warsaw"}
+      iex> NaiveDateTime.compare(dt, ~N[2000-02-29 23:00:07])
+      :eq
+      iex> NaiveDateTime.compare(dt, ~N[2000-01-29 23:00:07])
+      :gt
+      iex> NaiveDateTime.compare(dt, ~N[2000-03-29 23:00:07])
+      :lt
+
+  """
+  @spec compare(Calendar.naive_datetime, Calendar.naive_datetime) :: :lt | :eq | :gt
+  def compare(%{calendar: calendar1} = naive_datetime1, %{calendar: calendar2} = naive_datetime2) do
+    if Calendar.compatible_calendars?(calendar1, calendar2) do
+      case {to_iso_days(naive_datetime1), to_iso_days(naive_datetime2)} do
+        {first, second} when first > second -> :gt
+        {first, second} when first < second -> :lt
+        _ -> :eq
+      end
+    else
+      raise ArgumentError, """
+      cannot compare #{inspect naive_datetime1} with #{inspect naive_datetime2}.
+
+      This comparison would be ambiguous as their calendars have incompatible day rollover moments.
+      Specify an exact time of day (using `DateTime`s) to resolve this ambiguity
+      """
+    end
+  end
+
+  @doc """
+  Converts the given `naive_datetime` from one calendar to another.
+
+  If it is not possible to convert unambiguously between the calendars
+  (see `Calendar.compatible_calendars?/2`), an `{:error, :incompatible_calendars}` tuple
+  is returned.
+
+  ## Examples
+
+  Imagine someone implements `Calendar.Holocene`, a calendar based on the
+  Gregorian calendar that adds exactly 10,000 years to the current Gregorian
+  year:
+
+      iex> NaiveDateTime.convert(~N[2000-01-01 13:30:15], Calendar.Holocene)
+      {:ok, %NaiveDateTime{calendar: Calendar.Holocene, year: 12000, month: 1, day: 1,
+                           hour: 13, minute: 30, second: 15, microsecond: {0, 0}}}
+
+  """
+  @spec convert(Calendar.naive_datetime, Calendar.calendar) :: {:ok, t} | {:error, :incompatible_calendars}
+  def convert(%{calendar: calendar, year: year, month: month, day: day,
+                hour: hour, minute: minute, second: second, microsecond: microsecond}, calendar) do
+    {:ok, %NaiveDateTime{calendar: calendar, year: year, month: month, day: day,
+                         hour: hour, minute: minute, second: second, microsecond: microsecond}}
+  end
+
+  def convert(%{calendar: ndt_calendar, microsecond: {_, precision}} = naive_datetime, calendar) do
+    if Calendar.compatible_calendars?(ndt_calendar, calendar) do
+      result_naive_datetime =
+        naive_datetime
+        |> to_iso_days
+        |> from_iso_days(calendar, precision)
+      {:ok, result_naive_datetime}
+    else
+      {:error, :incompatible_calendars}
+    end
+  end
+
+  @doc """
+  Converts the given `naive_datetime` from one calendar to another.
+
+  If it is not possible to convert unambiguously between the calendars
+  (see `Calendar.compatible_calendars?/2`), an ArgumentError is raised.
+
+  ## Examples
+
+  Imagine someone implements `Calendar.Holocene`, a calendar based on the
+  Gregorian calendar that adds exactly 10,000 years to the current Gregorian
+  year:
+
+      iex> NaiveDateTime.convert!(~N[2000-01-01 13:30:15], Calendar.Holocene)
+      %NaiveDateTime{calendar: Calendar.Holocene, year: 12000, month: 1, day: 1,
+                     hour: 13, minute: 30, second: 15, microsecond: {0, 0}}
+
+  """
+  @spec convert!(Calendar.naive_datetime, Calendar.calendar) :: t
+  def convert!(naive_datetime, calendar) do
+    case convert(naive_datetime, calendar) do
+      {:ok, value} ->
+        value
+      {:error, :incompatible_calendars} ->
+        raise ArgumentError, "cannot convert #{inspect naive_datetime} to target calendar #{inspect calendar}, reason: #{inspect naive_datetime.calendar} and #{inspect calendar} have different day rollover moments, making this conversion ambiguous"
+    end
+  end
+
+  ## Helpers
+
+  defp to_iso_days(%{calendar: calendar, year: year, month: month, day: day,
+                     hour: hour, minute: minute, second: second, microsecond: microsecond}) do
+    calendar.naive_datetime_to_iso_days(year, month, day, hour, minute, second, microsecond)
+  end
+
+  defp from_iso_days(iso_days, calendar, precision) do
+    {year, month, day, hour, minute, second, {microsecond, _}} =
+      calendar.naive_datetime_from_iso_days(iso_days)
+    %NaiveDateTime{calendar: calendar, year: year, month: month, day: day,
+                   hour: hour, minute: minute, second: second, microsecond: {microsecond, precision}}
+  end
+
+  defimpl String.Chars do
+    def to_string(%{calendar: calendar, year: year, month: month, day: day,
+                    hour: hour, minute: minute, second: second, microsecond: microsecond}) do
+      calendar.naive_datetime_to_string(year, month, day, hour, minute, second, microsecond)
+    end
+  end
+
+  defimpl Inspect do
+    def inspect(%{calendar: Calendar.ISO, year: year, month: month, day: day,
+                  hour: hour, minute: minute, second: second, microsecond: microsecond}, _) do
+      formatted = Calendar.ISO.naive_datetime_to_string(year, month, day, hour, minute, second, microsecond)
+      "~N[" <> formatted <> "]"
+    end
+
+    def inspect(naive, opts) do
+      Inspect.Any.inspect(naive, opts)
+    end
+  end
+end

lib/elixir/lib/calendar/time.ex

@@ -0,0 +1,491 @@
+defmodule Time do
+  @moduledoc """
+  A Time struct and functions.
+
+  The Time struct contains the fields hour, minute, second and microseconds.
+  New times can be built with the `new/4` function or using the `~T`
+  sigil:
+
+      iex> ~T[23:00:07.001]
+      ~T[23:00:07.001]
+
+  Both `new/4` and sigil return a struct where the time fields can
+  be accessed directly:
+
+      iex> time = ~T[23:00:07.001]
+      iex> time.hour
+      23
+      iex> time.microsecond
+      {1000, 3}
+
+  The functions on this module work with the `Time` struct as well
+  as any struct that contains the same fields as the `Time` struct,
+  such as `NaiveDateTime` and `DateTime`. Such functions expect
+  `t:Calendar.time/0` in their typespecs (instead of `t:t/0`).
+
+  Developers should avoid creating the Time structs directly
+  and instead rely on the functions provided by this module as well
+  as the ones in 3rd party calendar libraries.
+
+  ## Comparing times
+
+  Comparisons in Elixir using `==`, `>`, `<` and similar are structural
+  and based on the `Time` struct fields. For proper comparison between
+  times, use the `compare/2` function.
+  """
+
+  @enforce_keys [:hour, :minute, :second]
+  defstruct [:hour, :minute, :second, microsecond: {0, 0}, calendar: Calendar.ISO]
+
+  @type t :: %Time{hour: Calendar.hour, minute: Calendar.minute,
+                   second: Calendar.second, microsecond: Calendar.microsecond, calendar: Calendar.calendar}
+
+  @doc """
+  Returns the current time in UTC.
+
+  ## Examples
+
+      iex> time = Time.utc_now()
+      iex> time.hour >= 0
+      true
+
+  """
+  @spec utc_now(Calendar.calendar) :: t
+  def utc_now(calendar \\ Calendar.ISO) do
+    {:ok, _, {hour, minute, second}, microsecond} = Calendar.ISO.from_unix(:os.system_time, :native)
+    iso_time = %Time{hour: hour, minute: minute, second: second, microsecond: microsecond, calendar: Calendar.ISO}
+    convert!(iso_time, calendar)
+  end
+
+  @doc """
+  Builds a new time.
+
+  Expects all values to be integers. Returns `{:ok, time}` if each
+  entry fits its appropriate range, returns `{:error, reason}` otherwise.
+
+  Note a time may have 60 seconds in case of leap seconds. Microseconds
+  can also be given with a precision, which must be an integer between
+  0 and 6.
+
+  ## Examples
+
+      iex> Time.new(0, 0, 0, 0)
+      {:ok, ~T[00:00:00.000000]}
+      iex> Time.new(23, 59, 59, 999_999)
+      {:ok, ~T[23:59:59.999999]}
+      iex> Time.new(23, 59, 60, 999_999)
+      {:ok, ~T[23:59:60.999999]}
+
+      # Time with microseconds and their precision
+      iex> Time.new(23, 59, 60, {10_000, 2})
+      {:ok, ~T[23:59:60.01]}
+
+      iex> Time.new(24, 59, 59, 999_999)
+      {:error, :invalid_time}
+      iex> Time.new(23, 60, 59, 999_999)
+      {:error, :invalid_time}
+      iex> Time.new(23, 59, 61, 999_999)
+      {:error, :invalid_time}
+      iex> Time.new(23, 59, 59, 1_000_000)
+      {:error, :invalid_time}
+
+      # Invalid precision
+      Time.new(23, 59, 59, {999_999, 10})
+      {:error, :invalid_time}
+
+  """
+  @spec new(Calendar.hour, Calendar.minute, Calendar.second, Calendar.microsecond, Calendar.calendar) ::
+        {:ok, t} | {:error, atom}
+  def new(hour, minute, second, microsecond \\ {0, 0}, calendar \\ Calendar.ISO)
+
+  def new(hour, minute, second, microsecond, calendar) when is_integer(microsecond) do
+    new(hour, minute, second, {microsecond, 6}, calendar)
+  end
+
+  def new(hour, minute, second, {microsecond, precision}, calendar)
+      when is_integer(hour) and is_integer(minute) and is_integer(second) and
+           is_integer(microsecond) and is_integer(precision) do
+    case calendar.valid_time?(hour, minute, second, {microsecond, precision}) do
+      true ->
+        {:ok, %Time{hour: hour, minute: minute, second: second, microsecond: {microsecond, precision}, calendar: calendar}}
+      false ->
+        {:error, :invalid_time}
+    end
+  end
+
+  @doc """
+  Converts the given `time` to a string.
+
+  ### Examples
+
+      iex> Time.to_string(~T[23:00:00])
+      "23:00:00"
+      iex> Time.to_string(~T[23:00:00.001])
+      "23:00:00.001"
+      iex> Time.to_string(~T[23:00:00.123456])
+      "23:00:00.123456"
+
+      iex> Time.to_string(~N[2015-01-01 23:00:00.001])
+      "23:00:00.001"
+      iex> Time.to_string(~N[2015-01-01 23:00:00.123456])
+      "23:00:00.123456"
+
+  """
+  @spec to_string(Calendar.time) :: String.t
+  def to_string(time)
+
+  def to_string(%{hour: hour, minute: minute, second: second, microsecond: microsecond, calendar: calendar}) do
+    calendar.time_to_string(hour, minute, second, microsecond)
+  end
+
+  @doc """
+  Parses the extended "Local time" format described by
+  [ISO 8601:2004](https://en.wikipedia.org/wiki/ISO_8601).
+
+  Timezone offset may be included in the string but they will be
+  simply discarded as such information is not included in times.
+
+  As specified in the standard, the separator "T" may be omitted if
+  desired as there is no ambiguity within this function.
+
+  Time representations with reduced accuracy are not supported.
+
+  Note that while ISO8601 allows times to specify 24:00:00 as the
+  zero hour of the next day, this notation is not supported by Elixir.
+
+  ## Examples
+
+      iex> Time.from_iso8601("23:50:07")
+      {:ok, ~T[23:50:07]}
+      iex> Time.from_iso8601("23:50:07Z")
+      {:ok, ~T[23:50:07]}
+      iex> Time.from_iso8601("T23:50:07Z")
+      {:ok, ~T[23:50:07]}
+
+      iex> Time.from_iso8601("23:50:07,0123456")
+      {:ok, ~T[23:50:07.012345]}
+      iex> Time.from_iso8601("23:50:07.0123456")
+      {:ok, ~T[23:50:07.012345]}
+      iex> Time.from_iso8601("23:50:07.123Z")
+      {:ok, ~T[23:50:07.123]}
+
+      iex> Time.from_iso8601("2015:01:23 23-50-07")
+      {:error, :invalid_format}
+      iex> Time.from_iso8601("23:50:07A")
+      {:error, :invalid_format}
+      iex> Time.from_iso8601("23:50:07.")
+      {:error, :invalid_format}
+      iex> Time.from_iso8601("23:50:61")
+      {:error, :invalid_time}
+
+  """
+  @spec from_iso8601(String.t) :: {:ok, t} | {:error, atom}
+  def from_iso8601(string, calendar \\ Calendar.ISO)
+
+  def from_iso8601(<<?T, h, rest::binary>>, calendar) when h in ?0..?9 do
+    from_iso8601(<<h, rest::binary>>, calendar)
+  end
+
+  def from_iso8601(<<hour::2-bytes, ?:, min::2-bytes, ?:, sec::2-bytes, rest::binary>>, calendar) do
+    with {hour, ""} <- Integer.parse(hour),
+         {min, ""} <- Integer.parse(min),
+         {sec, ""} <- Integer.parse(sec),
+         {microsec, rest} <- Calendar.ISO.parse_microsecond(rest),
+         {_offset, ""} <- Calendar.ISO.parse_offset(rest) do
+      with {:ok, utc_time} <- new(hour, min, sec, microsec, Calendar.ISO),
+           do: convert(utc_time, calendar)
+    else
+      _ -> {:error, :invalid_format}
+    end
+  end
+
+  def from_iso8601(<<_::binary>>, _calendar) do
+    {:error, :invalid_format}
+  end
+
+  @doc """
+  Parses the extended "Local time" format described by
+  [ISO 8601:2004](https://en.wikipedia.org/wiki/ISO_8601).
+
+  Raises if the format is invalid.
+
+  ## Examples
+
+      iex> Time.from_iso8601!("23:50:07,123Z")
+      ~T[23:50:07.123]
+      iex> Time.from_iso8601!("23:50:07.123Z")
+      ~T[23:50:07.123]
+      iex> Time.from_iso8601!("2015:01:23 23-50-07")
+      ** (ArgumentError) cannot parse "2015:01:23 23-50-07" as time, reason: :invalid_format
+  """
+  @spec from_iso8601!(String.t) :: t
+  def from_iso8601!(string) do
+    case from_iso8601(string) do
+      {:ok, value} ->
+        value
+      {:error, reason} ->
+        raise ArgumentError, "cannot parse #{inspect string} as time, reason: #{inspect reason}"
+    end
+  end
+
+  @doc """
+  Converts the given time to
+  [ISO 8601:2004](https://en.wikipedia.org/wiki/ISO_8601).
+
+  By default, `Time.to_iso8601/2` returns times formatted in the "extended"
+  format, for human readability. It also supports the "basic" format through
+  passing the `:basic` option.
+
+  ### Examples
+
+      iex> Time.to_iso8601(~T[23:00:13])
+      "23:00:13"
+
+      iex> Time.to_iso8601(~T[23:00:13.001])
+      "23:00:13.001"
+
+      iex> Time.to_iso8601(~T[23:00:13.001], :basic)
+      "230013.001"
+
+      iex> Time.to_iso8601(~N[2010-04-17 23:00:13])
+      "23:00:13"
+
+  """
+  @spec to_iso8601(Calendar.time, :extended | :basic) :: String.t
+  def to_iso8601(time, format \\ :extended) when format in [:extended, :basic] do
+    %{hour: hour, minute: minute, second: second, microsecond: microsecond} = convert!(time, Calendar.ISO)
+    Calendar.ISO.time_to_iso8601(hour, minute, second, microsecond, format)
+  end
+
+  @doc """
+  Converts given `time` to an Erlang time tuple.
+
+  WARNING: Loss of precision may occur, as Erlang time tuples
+  only contain hours/minutes/seconds.
+
+  ## Examples
+
+      iex> Time.to_erl(~T[23:30:15.999])
+      {23, 30, 15}
+
+      iex> Time.to_erl(~N[2010-04-17 23:30:15.999])
+      {23, 30, 15}
+
+  """
+  @spec to_erl(Calendar.time) :: :calendar.time
+  def to_erl(time) do
+    %{hour: hour, minute: minute, second: second} = convert!(time, Calendar.ISO)
+    {hour, minute, second}
+  end
+
+  @doc """
+  Converts an Erlang time tuple to a `Time` struct.
+
+  ## Examples
+
+      iex> Time.from_erl({23, 30, 15}, {5000, 3})
+      {:ok, ~T[23:30:15.005]}
+      iex> Time.from_erl({24, 30, 15})
+      {:error, :invalid_time}
+
+  """
+  @spec from_erl(:calendar.time, Calendar.microsecond, Calendar.calendar) :: {:ok, t} | {:error, atom}
+  def from_erl(tuple, microsecond \\ {0, 0}, calendar \\ Calendar.ISO)
+
+  def from_erl({hour, minute, second}, microsecond, calendar) do
+    with {:ok, time} <- new(hour, minute, second, microsecond, Calendar.ISO),
+         do: convert(time, calendar)
+  end
+
+  @doc """
+  Converts an Erlang time tuple to a `Time` struct.
+
+  ## Examples
+
+      iex> Time.from_erl!({23, 30, 15})
+      ~T[23:30:15]
+      iex> Time.from_erl!({23, 30, 15}, {5000, 3})
+      ~T[23:30:15.005]
+      iex> Time.from_erl!({24, 30, 15})
+      ** (ArgumentError) cannot convert {24, 30, 15} to time, reason: :invalid_time
+
+  """
+  @spec from_erl!(:calendar.time, Calendar.microsecond, Calendar.calendar) :: t
+  def from_erl!(tuple, microsecond \\ {0, 0}, calendar \\ Calendar.ISO) do
+    case from_erl(tuple, microsecond, calendar) do
+      {:ok, value} ->
+        value
+      {:error, reason} ->
+        raise ArgumentError, "cannot convert #{inspect tuple} to time, reason: #{inspect reason}"
+    end
+  end
+
+  @doc """
+  Compares two time structs.
+
+  Returns `:gt` if first time is later than the second
+  and `:lt` for vice versa. If the two times are equal
+  `:eq` is returned.
+
+  ## Examples
+
+      iex> Time.compare(~T[16:04:16], ~T[16:04:28])
+      :lt
+      iex> Time.compare(~T[16:04:16], ~T[16:04:16])
+      :eq
+      iex> Time.compare(~T[16:04:16.01], ~T[16:04:16.001])
+      :gt
+
+  This function can also be used to compare across more
+  complex calendar types by considering only the time fields:
+
+      iex> Time.compare(~N[1900-01-01 16:04:16], ~N[2015-01-01 16:04:16])
+      :eq
+      iex> Time.compare(~N[2015-01-01 16:04:16], ~N[2015-01-01 16:04:28])
+      :lt
+      iex> Time.compare(~N[2015-01-01 16:04:16.01], ~N[2000-01-01 16:04:16.001])
+      :gt
+
+  """
+  @spec compare(Calendar.time, Calendar.time) :: :lt | :eq | :gt
+  def compare(%{calendar: calendar, hour: hour1, minute: minute1, second: second1, microsecond: {microsecond1, _}},
+              %{calendar: calendar, hour: hour2, minute: minute2, second: second2, microsecond: {microsecond2, _}}) do
+    case {{hour1, minute1, second1, microsecond1}, {hour2, minute2, second2, microsecond2}} do
+      {first, second} when first > second -> :gt
+      {first, second} when first < second -> :lt
+      _ -> :eq
+    end
+  end
+
+  def compare(time1, time2) do
+    {parts1, ppd1} = to_day_fraction(time1)
+    {parts2, ppd2} = to_day_fraction(time2)
+
+    case {parts1 * ppd2, parts2 * ppd1} do
+      {first, second} when first > second -> :gt
+      {first, second} when first < second -> :lt
+      _ -> :eq
+    end
+  end
+
+  @doc """
+  Converts given `time` to a different calendar.
+
+  Returns `{:ok, time}` if the conversion was successful,
+  or `{:error, reason}` if it was not, for some reason.
+
+  ## Examples
+
+  Imagine someone implements `Calendar.Holocene`, a calendar based on the
+  Gregorian calendar that adds exactly 10,000 years to the current Gregorian
+  year:
+
+      iex> Time.convert(~T[13:30:15], Calendar.Holocene)
+      {:ok, %Time{calendar: Calendar.Holocene, hour: 13, minute: 30, second: 15, microsecond: {0, 0}}}
+
+  """
+  @spec convert(Calendar.time, Calendar.calendar) :: {:ok, t} | {:error, atom}
+  def convert(%{calendar: calendar, hour: hour, minute: minute, second: second, microsecond: microsecond}, calendar) do
+    {:ok, %Time{calendar: calendar, hour: hour, minute: minute, second: second, microsecond: microsecond}}
+  end
+
+  def convert(%{microsecond: {_, precision}} = time, calendar) do
+    {hour, minute, second, {microsecond, _}} =
+      time
+      |> to_day_fraction()
+      |> calendar.time_from_day_fraction
+    {:ok, %Time{calendar: calendar, hour: hour, minute: minute, second: second,
+                microsecond: {microsecond, precision}}}
+  end
+
+  @doc """
+  Similar to `Time.convert/2`, but raises an `ArgumentError`
+  if the conversion between the two calendars is not possible.
+
+  ## Examples
+
+  Imagine someone implements `Calendar.Holocene`, a calendar based on the
+  Gregorian calendar that adds exactly 10,000 years to the current Gregorian
+  year:
+
+      iex> Time.convert!(~T[13:30:15], Calendar.Holocene)
+      %Time{calendar: Calendar.Holocene, hour: 13, minute: 30, second: 15, microsecond: {0, 0}}
+
+  """
+  @spec convert!(Calendar.time, Calendar.calendar) :: t
+  def convert!(time, calendar) do
+    case convert(time, calendar) do
+      {:ok, value} ->
+        value
+      {:error, reason} ->
+        raise ArgumentError, "cannot convert #{inspect time} to target calendar #{inspect calendar}, reason: #{inspect reason}"
+    end
+  end
+
+  @doc """
+  Returns the difference between two times, considering only the hour, minute
+  second and microsecond.
+
+  As with the `compare/2` function both `Time` structs and other structures
+  containing time can be used. If for instance a `NaiveDateTime` or `DateTime`
+  is passed, only the hour, month, second, and microsecond is considered. Any
+  additional information about a date or time zone is ignored when calculating
+  the difference.
+
+  The answer can be returned in any `unit` available from
+  `t:System.time_unit/0`. If the first unit is smaller than
+  the second, a negative number is returned.
+
+  This function returns the difference in seconds where seconds
+  are measured according to `Calendar.ISO`.
+
+  ## Examples
+
+      iex> Time.diff(~T[00:29:12], ~T[00:29:10])
+      2
+
+      # When passing a `NaiveDateTime` the date part is ignored.
+      iex> Time.diff(~N[2017-01-01 00:29:12], ~T[00:29:10])
+      2
+
+      # Two `NaiveDateTime` structs could have big differences in the date
+      # but only the time part is considered.
+      iex> Time.diff(~N[2017-01-01 00:29:12], (~N[1900-02-03 00:29:10]))
+      2
+
+      iex> Time.diff(~T[00:29:12], ~T[00:29:10], :microsecond)
+      2_000_000
+      iex> Time.diff(~T[00:29:10], ~T[00:29:12], :microsecond)
+      -2_000_000
+
+  """
+  @spec diff(Calendar.time, Calendar.time, System.time_unit) :: integer
+  def diff(time1, time2, unit \\ :second) do
+    fraction1 = to_day_fraction(time1)
+    fraction2 = to_day_fraction(time2)
+    Calendar.ISO.iso_days_to_unit({0, fraction1}, unit) -
+      Calendar.ISO.iso_days_to_unit({0, fraction2}, unit)
+  end
+
+  ## Helpers
+
+  defp to_day_fraction(%{hour: hour, minute: minute, second: second, microsecond: {_, _} = microsecond, calendar: calendar}) do
+    calendar.time_to_day_fraction(hour, minute, second, microsecond)
+  end
+
+  defimpl String.Chars do
+    def to_string(%{hour: hour, minute: minute, second: second, microsecond: microsecond, calendar: calendar}) do
+      calendar.time_to_string(hour, minute, second, microsecond)
+    end
+  end
+
+  defimpl Inspect do
+    def inspect(%{hour: hour, minute: minute, second: second, microsecond: microsecond, calendar: Calendar.ISO}, _) do
+      "~T[" <> Calendar.ISO.time_to_string(hour, minute, second, microsecond) <> "]"
+    end
+
+    def inspect(time, opts) do
+      Inspect.Any.inspect(time, opts)
+    end
+  end
+end

lib/elixir/lib/code.ex

@@ -104,6 +104,12 @@ defmodule Code do
   The `binding` argument is a keyword list of variable bindings.
   The `opts` argument is a keyword list of environment options.
 
+  **Warning**: `string` can be any Elixir code and will be executed with
+  the same privileges as the Erlang VM: this means that such code could
+  compromise the machine (for example by executing system commands).
+  Don't use `eval_string/3` with untrusted input (such as strings coming
+  from the network).
+
   ## Options
 
   Options can be:
@@ -335,11 +341,11 @@ defmodule Code do
 
   Accepts `relative_to` as an argument to tell where the file is located.
   The return value is the same as that of `load_file/2`. If the file was already
-  required/loaded, doesn't do anything and returns `nil`.
+  required/loaded, `require_file` doesn't do anything and returns `nil`.
 
   Notice that if `require_file` is invoked by different processes concurrently,
   the first process to invoke `require_file` acquires a lock and the remaining
-  ones will block until the file is available. I.e. if `require_file` is called
+  ones will block until the file is available. I.e., if `require_file` is called
   N times with a given file, it will be loaded only once. The first process to
   call `require_file` will get the list of loaded modules, others will get `nil`.
 
@@ -442,7 +448,7 @@ defmodule Code do
 
     Enum.each(opts, fn({key, value}) ->
       cond do
-        not key in available ->
+        key not in available ->
           raise "unknown compiler option: #{inspect(key)}"
         not is_boolean(value) ->
           raise "compiler option #{inspect(key)} should be a boolean, got: #{inspect(value)}"
@@ -498,7 +504,7 @@ defmodule Code do
   module uses this function to check if a specific parser exists for a given
   URI scheme.
 
-  ## Code.ensure_compiled/1
+  ## `ensure_compiled/1`
 
   Elixir also contains an `ensure_compiled/1` function that is a
   superset of `ensure_loaded/1`.
@@ -507,8 +513,13 @@ defmodule Code do
   you may need to use a module that was not yet compiled, therefore
   it can't even be loaded.
 
-  `ensure_compiled/1` halts the current process until the
-  module we are depending on is available.
+  When invoked, `ensure_compiled/1` halts the compilation of the caller
+  until the module given to `ensure_compiled/1` becomes available or
+  all files for the current project have been compiled. If compilation
+  finishes and the module is not available, an error tuple is returned.
+
+  `ensure_compiled/1` does not apply to dependencies, as dependencies
+  must be compiled upfront.
 
   In most cases, `ensure_loaded/1` is enough. `ensure_compiled/1`
   must be used in rare cases, usually involving macros that need to

lib/elixir/lib/collectable.ex

@@ -24,23 +24,54 @@ defprotocol Collectable do
   `Enumerable` protocol. `into/1` can be seen as the opposite of
   `Enumerable.reduce/3`. If `Enumerable` is about taking values out,
   `Collectable.into/1` is about collecting those values into a structure.
+
+  ## Examples
+
+  To show how to manually use the `Collectable` protocol, let's play with its
+  implementation for `MapSet`.
+
+      iex> {initial_acc, collector_fun} = Collectable.into(MapSet.new())
+      iex> updated_acc = Enum.reduce([1, 2, 3], initial_acc, fn elem, acc ->
+      ...>   collector_fun.(acc, {:cont, elem})
+      ...> end)
+      iex> collector_fun.(updated_acc, :done)
+      #MapSet<[1, 2, 3]>
+
+  To show how the protocol can be implemented, we can take again a look at the
+  implementation for `MapSet`. In this implementation "collecting" elements
+  simply means inserting them in the set through `MapSet.put/2`.
+
+      defimpl Collectable do
+        def into(original) do
+          collector_fun = fn
+            set, {:cont, elem} -> MapSet.put(set, elem)
+            set, :done -> set
+            _set, :halt -> :ok
+          end
+
+          {original, collector_fun}
+        end
+      end
+
   """
 
   @type command :: {:cont, term} | :done | :halt
 
   @doc """
-  Returns a function that collects values alongside
-  the initial accumulation value.
+  Returns an initial accumulator and a "collector" function.
 
-  The returned function receives a collectable and injects a given
-  value into it for every `{:cont, term}` instruction.
+  The returned function receives a term and a command and injects the term into
+  the collectable on every `{:cont, term}` command.
 
-  `:done` is passed when no further values will be injected, useful
-  for closing resources and normalizing values. A collectable must
-  be returned on `:done`.
+  `:done` is passed as a command when no further values will be injected. This
+  is useful when there's a need to close resources or normalizing values. A
+  collectable must be returned when the command is `:done`.
 
-  If injection is suddenly interrupted, `:halt` is passed and it can
-  return any value, as it won't be used.
+  If injection is suddenly interrupted, `:halt` is passed and the function
+  can return any value as it won't be used.
+
+  For examples on how to use the `Collectable` protocol and `into/1` see the
+  module documentation.
   """
   @spec into(t) :: {term, (term, command -> t | term)}
   def into(collectable)

lib/elixir/lib/dict.ex

@@ -13,14 +13,13 @@ defmodule Dict do
   @type value :: any
   @type t :: list | map
 
-  # TODO: Deprecate every function by 1.4
+  # TODO: Remove by 2.0
+  # (hard-deprecated in elixir_dispatch)
+
   defmacro __using__(_) do
     # Use this import to guarantee proper code expansion
     import Kernel, except: [size: 1]
 
-    %{file: file, line: line} = __CALLER__
-    :elixir_errors.warn(line, file, "the Dict module is deprecated")
-
     quote do
       def get(dict, key, default \\ nil) do
         case fetch(dict, key) do

lib/elixir/lib/exception.ex

@@ -22,7 +22,8 @@ defmodule Exception do
   }
 
   @typedoc "The kind handled by formatting functions"
-  @type kind :: :error | :exit | :throw | {:EXIT, pid}
+  @type kind :: :error | non_error_kind
+  @typep non_error_kind :: :exit | :throw | {:EXIT, pid}
 
   @type stacktrace :: [stacktrace_entry]
   @type stacktrace_entry ::
@@ -30,7 +31,7 @@ defmodule Exception do
         {(... -> any), arity_or_args, location}
 
   @typep arity_or_args :: non_neg_integer | list
-  @typep location :: Keyword.t
+  @typep location :: keyword
 
   @callback exception(term) :: t
   @callback message(t) :: String.t
@@ -80,7 +81,7 @@ defmodule Exception do
   an empty stacktrace, `[]`, must be used.
   """
   @spec normalize(:error, any, stacktrace) :: t
-  @spec normalize(kind, payload, stacktrace) :: payload when payload: var
+  @spec normalize(non_error_kind, payload, stacktrace) :: payload when payload: var
 
   # Generating a stacktrace is expensive, default to nil
   # to only fetch it when needed.
@@ -140,9 +141,7 @@ defmodule Exception do
   Note that `{:EXIT, pid}` do not generate a stacktrace though
   (as they are retrieved as messages without stacktraces).
   """
-
   @spec format(kind, any, stacktrace | nil) :: String.t
-
   def format(kind, payload, stacktrace \\ nil)
 
   def format({:EXIT, _} = kind, any, _) do
@@ -158,6 +157,162 @@ defmodule Exception do
     end
   end
 
+  @doc """
+  Attaches information to exceptions for extra debugging.
+
+  This operation is potentially expensive, as it reads data
+  from the filesystem, parse beam files, evaluates code and
+  so on. Currently the following exceptions may be annotated:
+
+    * `FunctionClauseError` - annotated with the arguments
+      used on the call and available clauses
+
+  """
+  @spec blame(:error, any, stacktrace) :: {t, stacktrace}
+  @spec blame(non_error_kind, payload, stacktrace) :: {payload, stacktrace} when payload: var
+  def blame(kind, error, stacktrace)
+
+  def blame(:error, error, stacktrace) do
+    case normalize(:error, error, stacktrace) do
+      %{__struct__: FunctionClauseError} = struct ->
+        blame_function_clause_error(struct, stacktrace)
+      _ ->
+        {error, stacktrace}
+    end
+  end
+
+  def blame(_kind, reason, stacktrace) do
+    {reason, stacktrace}
+  end
+
+  defp blame_function_clause_error(%{module: module, function: function, arity: arity} = exception,
+                                   [{module, function, args, meta} | rest])
+       when length(args) == arity do
+    exception =
+      case blame_mfa(module, function, args) do
+        {:ok, kind, clauses} -> %{exception | args: args, kind: kind, clauses: clauses}
+        :error -> %{exception | args: args}
+      end
+    {exception, [{module, function, arity, meta} | rest]}
+  end
+  defp blame_function_clause_error(exception, stacktrace) do
+    {exception, stacktrace}
+  end
+
+  @doc """
+  Blames the invocation of the given module, function and arguments.
+
+  This function will retrieve the available clauses from bytecode
+  and evaluate them against the given arguments. The clauses are
+  returned as a list of `{args, guards}` pairs where each argument
+  and each top-level condition in a guard separated by `and`/`or`
+  is wrapped in a tuple with blame metadata.
+
+  This function returns either `{:ok, definition, clauses}` or `:error`.
+  Where `definition` is `:def`, `:defp`, `:defmacro` or `:defmacrop`.
+  Note this functionality requires Erlang/OTP 20, otherwise `:error`
+  is always returned.
+  """
+  @spec blame_mfa(module, function, args :: [term]) ::
+        {:ok, :def | :defp | :defmacro | :defmacrop, [{args :: [term], guards :: [term]}]} | :error
+  def blame_mfa(module, function, args) when is_atom(module) and is_atom(function) and is_list(args) do
+    try do
+      blame_mfa(module, function, length(args), args)
+    rescue
+      _ -> :error
+    end
+  end
+
+  defp blame_mfa(module, function, arity, call_args) do
+    with path when is_list(path) <- :code.which(module),
+         {:ok, {_, [debug_info: {:debug_info_v1, backend, data}]}} <- :beam_lib.chunks(path, [:debug_info]),
+         {:ok, %{definitions: defs}} <- backend.debug_info(:elixir_v1, module, data, []),
+         {_, kind, _, clauses} <- List.keyfind(defs, {function, arity}, 0) do
+      clauses =
+        for {meta, ex_args, guards, _block} <- clauses do
+          scope = :elixir_erl.definition_scope(meta, "nofile")
+          {erl_args, scope} =
+            :elixir_erl_clauses.match(&:elixir_erl_pass.translate_args/2, ex_args, scope)
+          {args, binding} =
+            [call_args, ex_args, erl_args]
+            |> Enum.zip()
+            |> Enum.map_reduce([], &blame_arg/2)
+          guards = Enum.map(guards, &blame_guard(&1, scope, binding))
+          {args, guards}
+        end
+      {:ok, kind, clauses}
+    else
+      _ -> :error
+    end
+  end
+
+  defp blame_arg({call_arg, ex_arg, erl_arg}, binding) do
+    {match?, binding} = blame_arg(erl_arg, call_arg, binding)
+    {blame_wrap(match?, rewrite_arg(ex_arg)), binding}
+  end
+
+  defp blame_arg(erl_arg, call_arg, binding) do
+    binding = :orddict.store(:VAR, call_arg, binding)
+    try do
+      {:value, _, binding} = :erl_eval.expr({:match, 0, erl_arg, {:var, 0, :VAR}}, binding, :none)
+      {true, binding}
+    rescue
+      _ -> {false, binding}
+    end
+  end
+
+  defp rewrite_arg(arg) do
+    Macro.prewalk(arg, fn
+      {:%{}, meta, [__struct__: Range, first: first, last: last]} ->
+        {:.., meta, [first, last]}
+      other ->
+        other
+    end)
+  end
+
+  defp blame_guard({{:., _, [:erlang, op]}, meta, [left, right]}, scope, binding)
+       when op == :andalso or op == :orelse do
+    {rewrite_guard_call(op), meta, [
+      blame_guard(left, scope, binding),
+      blame_guard(right, scope, binding)
+    ]}
+  end
+
+  defp blame_guard(ex_guard, scope, binding) do
+    {erl_guard, _} = :elixir_erl_pass.translate(ex_guard, scope)
+    match? =
+      try do
+        {:value, true, _} = :erl_eval.expr(erl_guard, binding, :none)
+        true
+      rescue
+        _ -> false
+      end
+    blame_wrap(match?, rewrite_guard(ex_guard))
+  end
+
+  defp rewrite_guard(guard) do
+    Macro.prewalk(guard, fn
+      {:., _, [:erlang, call]} -> rewrite_guard_call(call)
+      other -> other
+    end)
+  end
+
+  defp rewrite_guard_call(:"orelse"), do: :or
+  defp rewrite_guard_call(:"andalso"), do: :and
+  defp rewrite_guard_call(:"=<"), do: :<=
+  defp rewrite_guard_call(:"/="), do: :!=
+  defp rewrite_guard_call(:"=:="), do: :===
+  defp rewrite_guard_call(:"=/="), do: :!==
+
+  defp rewrite_guard_call(op) when op in [:band, :bor, :bnot, :bsl, :bsr, :bxor],
+    do: {:., [], [Bitwise, op]}
+  defp rewrite_guard_call(op) when op in [:xor, :element, :size],
+    do: {:., [], [:erlang, op]}
+  defp rewrite_guard_call(op),
+    do: op
+
+  defp blame_wrap(match?, ast), do: %{match?: match?, node: ast}
+
   @doc """
   Formats an exit. It returns a string.
 
@@ -274,15 +429,15 @@ defmodule Exception do
   end
 
   defp format_exit_reason({:bad_start_spec, start_spec}) do
-    "bad start spec: invalid children: " <> inspect(start_spec)
+    "bad child specification, invalid children: " <> inspect(start_spec)
   end
 
   defp format_exit_reason({:start_spec, start_spec}) do
-    "bad start spec: " <> format_sup_spec(start_spec)
+    "bad child specification, " <> format_sup_spec(start_spec)
   end
 
   defp format_exit_reason({:supervisor_data, data}) do
-    "bad supervisor data: " <> format_sup_data(data)
+    "bad supervisor configuration, " <> format_sup_data(data)
   end
 
   defp format_exit_reason(reason), do: inspect(reason)
@@ -301,44 +456,57 @@ defmodule Exception do
   end
 
   defp format_sup_data({:invalid_intensity, intensity}) do
-    "invalid intensity: " <> inspect(intensity)
+    "invalid max_restarts (intensity): " <> inspect(intensity)
   end
 
   defp format_sup_data({:invalid_period, period}) do
-    "invalid period: " <> inspect(period)
+    "invalid max_seconds (period): " <> inspect(period)
   end
 
-  defp format_sup_data(other), do: inspect(other)
+  defp format_sup_data(other), do: "got: #{inspect other}"
+
+  defp format_sup_spec({:duplicate_child_name, id}) do
+    """
+    more than one child specification has the id: #{inspect id}.
+    If using maps as child specifications, make sure the :id keys are unique.
+    If using a module or {module, arg} as child, use Supervisor.child_spec/2 to change the :id, for example:
+
+        children = [
+          Supervisor.child_spec({MyWorker, arg}, id: :my_worker_1),
+          Supervisor.child_spec({MyWorker, arg}, id: :my_worker_2)
+        ]
+    """
+  end
 
   defp format_sup_spec({:invalid_child_spec, child_spec}) do
-   "invalid child spec: " <> inspect(child_spec)
+   "invalid child specification: #{inspect child_spec}"
   end
 
   defp format_sup_spec({:invalid_child_type, type}) do
-    "invalid child type: " <> inspect(type)
+    "invalid child type: #{inspect type}. Must be :worker or :supervisor."
   end
 
   defp format_sup_spec({:invalid_mfa, mfa}) do
-    "invalid mfa: " <> inspect(mfa)
+    "invalid mfa: #{inspect mfa}"
   end
 
   defp format_sup_spec({:invalid_restart_type, restart}) do
-    "invalid restart type: " <> inspect(restart)
+    "invalid restart type: #{inspect restart}. Must be :permanent, :transient or :temporary."
   end
 
   defp format_sup_spec({:invalid_shutdown, shutdown}) do
-    "invalid shutdown: " <> inspect(shutdown)
+    "invalid shutdown: #{inspect shutdown}. Must be an integer >= 0, :infinity or :brutal_kill."
   end
 
   defp format_sup_spec({:invalid_module, mod}) do
-    "invalid module: " <> inspect(mod)
+    "invalid module: #{inspect mod}. Must be an atom."
   end
 
   defp format_sup_spec({:invalid_modules, modules}) do
-    "invalid modules: " <> inspect(modules)
+    "invalid modules: #{inspect modules}. Must be a list of atoms or :dynamic."
   end
 
-  defp format_sup_spec(other), do: inspect(other)
+  defp format_sup_spec(other), do: "got: #{inspect other}"
 
   @doc """
   Receives a stacktrace entry and formats it into a string.
@@ -429,17 +597,12 @@ defmodule Exception do
   "anonymous fn in func/arity"
   """
   def format_mfa(module, fun, arity) when is_atom(module) and is_atom(fun) do
-    fun =
-      case inspect(fun) do
-        ":" <> fun -> fun
-        fun -> fun
-      end
-
-    case match?("\"-" <> _, fun) and String.split(fun, "-") do
-      [ "\"", outer_fun, "fun", _count, "\"" ] ->
-        "anonymous fn#{format_arity(arity)} in #{inspect module}.#{outer_fun}"
-      _ ->
-        "#{inspect module}.#{fun}#{format_arity(arity)}"
+    case Inspect.Function.extract_anonymous_fun_parent(Atom.to_string(fun)) do
+      {outer_name, outer_arity} ->
+        "anonymous fn#{format_arity(arity)} in " <>
+          "#{inspect(module)}.#{Inspect.Function.escape_name(outer_name)}/#{outer_arity}"
+      :error ->
+        "#{inspect(module)}.#{Inspect.Function.escape_name(fun)}#{format_arity(arity)}"
     end
   end
 
@@ -644,19 +807,8 @@ defmodule UndefinedFunctionError do
       " is undefined (module #{inspect module} is not available)"
   end
 
-  def message(%{reason: :"function not exported",  module: module, function: function, arity: arity, exports: exports}) do
-    suffix =
-      if macro_exported?(module, function, arity) do
-        ". However there is a macro with the same name and arity." <>
-          " Be sure to require #{inspect(module)} if you intend to invoke this macro"
-      else
-        did_you_mean(module, function, arity, exports)
-      end
-
-    "function " <>
-      Exception.format_mfa(module, function, arity) <>
-      " is undefined or private" <>
-      suffix
+  def message(%{reason: :"function not exported",  module: module, function: function, arity: arity}) do
+    IO.iodata_to_binary(function_not_exported(module, function, arity, nil))
   end
 
   def message(%{reason: :"function not available", module: module, function: function, arity: arity}) do
@@ -669,10 +821,23 @@ defmodule UndefinedFunctionError do
     "function " <> Exception.format_mfa(module, function, arity) <> " is undefined (#{reason})"
   end
 
+  @doc false
+  def function_not_exported(module, function, arity, exports) do
+    suffix =
+      if macro_exported?(module, function, arity) do
+        ". However there is a macro with the same name and arity. " <>
+          "Be sure to require #{inspect(module)} if you intend to invoke this macro"
+      else
+        did_you_mean(module, function, exports)
+      end
+
+    ["function ", Exception.format_mfa(module, function, arity), " is undefined or private", suffix]
+  end
+
   @function_threshold 0.77
   @max_suggestions 5
 
-  defp did_you_mean(module, function, _arity, exports) do
+  defp did_you_mean(module, function, exports) do
     exports = exports || exports_for(module)
 
     result =
@@ -691,14 +856,13 @@ defmodule UndefinedFunctionError do
       |> Enum.sort(&elem(&1, 1) <= elem(&2, 1))
 
     case result do
-      []          -> ""
-      suggestions -> ". Did you mean one of:\n\n#{Enum.map(suggestions, &format_fa/1)}"
+      []          -> []
+      suggestions -> [". Did you mean one of:\n\n" | Enum.map(suggestions, &format_fa/1)]
     end
   end
 
   defp format_fa({_dist, fun, arity}) do
-    fun = with ":" <> fun <- inspect(fun), do: fun
-    "      * " <> fun <> "/" <> Integer.to_string(arity) <> "\n"
+    ["      * ", Inspect.Function.escape_name(fun), ?/, Integer.to_string(arity), ?\n]
   end
 
   defp exports_for(module) do
@@ -714,16 +878,55 @@ defmodule UndefinedFunctionError do
 end
 
 defmodule FunctionClauseError do
-  defexception [:module, :function, :arity]
+  defexception [:module, :function, :arity, :kind, :args, :clauses]
 
   def message(exception) do
-    if exception.function do
-      formatted = Exception.format_mfa exception.module, exception.function, exception.arity
-      "no function clause matching in #{formatted}"
-    else
-      "no function clause matches"
+    case exception do
+      %{function: nil} ->
+        "no function clause matches"
+      %{module: module, function: function, arity: arity} ->
+        formatted = Exception.format_mfa module, function, arity
+        "no function clause matching in #{formatted}" <> blame(exception, &inspect/1, &blame_match/2)
     end
   end
+
+  defp blame_match(%{match?: true, node: node}, _),
+    do: Macro.to_string(node)
+  defp blame_match(%{match?: false, node: node}, _),
+    do: "-" <> Macro.to_string(node) <> "-"
+  defp blame_match(_, string),
+    do: string
+
+  @doc false
+  def blame(%{args: nil}, _, _) do
+    ""
+  end
+  def blame(%{module: module, function: function, arity: arity,
+              kind: kind, args: args, clauses: clauses}, inspect_fun, ast_fun) do
+    mfa = Exception.format_mfa(module, function, arity)
+
+    formatted_args =
+      args
+      |> Enum.with_index(1)
+      |> Enum.map(fn {arg, i} -> "\n    # #{i}\n    #{inspect_fun.(arg)}\n" end)
+
+    formatted_clauses =
+      if clauses do
+        top_10 =
+          clauses
+          |> Enum.take(10)
+          |> Enum.map(fn {args, guards} ->
+            code = Enum.reduce(guards, {function, [], args}, &{:when, [], [&2, &1]})
+            "    #{kind} " <> Macro.to_string(code, ast_fun) <> "\n"
+          end)
+
+        "\nAttempted function clauses (showing #{length(top_10)} out of #{length(clauses)}):\n\n#{top_10}"
+      else
+        ""
+      end
+
+    "\n\nThe following arguments were given to #{mfa}:\n#{formatted_args}#{formatted_clauses}"
+  end
 end
 
 defmodule Code.LoadError do
@@ -738,11 +941,22 @@ end
 defmodule Protocol.UndefinedError do
   defexception [:protocol, :value, description: ""]
 
-  def message(exception) do
-    msg = "protocol #{inspect exception.protocol} not implemented for #{inspect exception.value}"
-    case exception.description do
-      "" -> msg
-      descr -> msg <> ", " <> descr
+  def message(%{protocol: protocol, value: value, description: description}) do
+    "protocol #{inspect protocol} not implemented for #{inspect value}" <>
+      maybe_description(description) <> maybe_available(protocol)
+  end
+
+  defp maybe_description(""), do: ""
+  defp maybe_description(description), do: ", " <> description
+
+  defp maybe_available(protocol) do
+    case protocol.__protocol__(:impls) do
+      {:consolidated, []} ->
+        ". There are no implementations for this protocol."
+      {:consolidated, types} ->
+        ". This protocol is implemented for: #{Enum.map_join(types, ", ", &inspect/1)}"
+      :not_consolidated ->
+        ""
     end
   end
 end
@@ -825,11 +1039,22 @@ defmodule File.CopyError do
   end
 end
 
+defmodule File.LinkError do
+  defexception [:reason, :existing, :new, action: ""]
+
+  def message(exception) do
+    formatted =
+      IO.iodata_to_binary(:file.format_error(exception.reason))
+    "could not #{exception.action} from #{inspect(exception.existing)} to " <>
+      "#{inspect(exception.new)}: #{formatted}"
+  end
+end
+
 defmodule ErlangError do
   defexception [:original]
 
   def message(exception) do
-    "erlang error: #{inspect(exception.original)}"
+    "Erlang error: #{inspect(exception.original)}"
   end
 
   @doc false
@@ -875,7 +1100,7 @@ defmodule ErlangError do
 
   def normalize({:badkey, key}, stacktrace) do
     term =
-      case stacktrace || :erlang.get_stacktrace do
+      case ensure_stacktrace(stacktrace) do
         [{Map, :get_and_update!, [map, _, _], _} | _] -> map
         [{Map, :update!, [map, _, _], _} | _] -> map
         [{:maps, :update, [_, _, map], _} | _] -> map
@@ -902,13 +1127,13 @@ defmodule ErlangError do
   end
 
   def normalize(:undef, stacktrace) do
-    stacktrace = stacktrace || :erlang.get_stacktrace
+    stacktrace = ensure_stacktrace(stacktrace)
     {mod, fun, arity} = from_stacktrace(stacktrace)
     %UndefinedFunctionError{module: mod, function: fun, arity: arity}
   end
 
   def normalize(:function_clause, stacktrace) do
-    {mod, fun, arity} = from_stacktrace(stacktrace || :erlang.get_stacktrace)
+    {mod, fun, arity} = from_stacktrace(ensure_stacktrace(stacktrace))
     %FunctionClauseError{module: mod, function: fun, arity: arity}
   end
 
@@ -920,6 +1145,18 @@ defmodule ErlangError do
     %ErlangError{original: other}
   end
 
+  defp ensure_stacktrace(nil) do
+    try do
+      :erlang.get_stacktrace()
+    rescue
+      _ -> []
+    end
+  end
+
+  defp ensure_stacktrace(stacktrace) do
+    stacktrace
+  end
+
   defp from_stacktrace([{module, function, args, _} | _]) when is_list(args) do
     {module, function, length(args)}
   end

lib/elixir/lib/file.ex

@@ -10,6 +10,12 @@ defmodule File do
   via `cp/3` and remove files and directories recursively
   via `rm_rf/1`.
 
+  Paths given to functions in this module can be either relative to the
+  current working directory (as returned by `File.cwd/0`), or absolute
+  paths. Shell conventions like `~` are not expanded automatically.
+  To use paths like `~/Downloads`, you can use `Path.expand/1` or
+  `Path.expand/2` to expand your path to an absolute path.
+
   ## Encoding
 
   In order to write and read files, one must use the functions
@@ -70,8 +76,6 @@ defmodule File do
   about such options and other performance considerations.
   """
 
-  alias :file, as: F
-
   @type posix :: :file.posix()
   @type io_device :: :file.io_device()
   @type stat_options :: [time: :local | :universal | :posix]
@@ -85,6 +89,9 @@ defmodule File do
   @doc """
   Returns `true` if the path is a regular file.
 
+  This function follows symbolic links, so if a symbolic link points to a
+  regular file, `true` is returned.
+
   ## Examples
 
       File.regular? __ENV__.file #=> true
@@ -96,7 +103,28 @@ defmodule File do
   end
 
   @doc """
-  Returns `true` if the path is a directory.
+  Returns `true` if the given path is a directory.
+
+  This function follows symbolic links, so if a symbolic link points to a
+  directory, `true` is returned.
+
+  ## Examples
+
+      File.dir?("./test")
+      #=> true
+
+      File.dir?("test")
+      #=> true
+
+      File.dir?("/usr/bin")
+      #=> true
+
+      File.dir?("~/Downloads")
+      #=> false
+
+      "~/Downloads" |> Path.expand |> File.dir?
+      #=> true
+
   """
   @spec dir?(Path.t) :: boolean
   def dir?(path) do
@@ -122,7 +150,7 @@ defmodule File do
   """
   @spec exists?(Path.t) :: boolean
   def exists?(path) do
-    match?({:ok, _}, F.read_file_info(IO.chardata_to_string(path)))
+    match?({:ok, _}, :file.read_file_info(IO.chardata_to_string(path)))
   end
 
   @doc """
@@ -141,7 +169,7 @@ defmodule File do
   """
   @spec mkdir(Path.t) :: :ok | {:error, posix}
   def mkdir(path) do
-    F.make_dir(IO.chardata_to_string(path))
+    :file.make_dir(IO.chardata_to_string(path))
   end
 
   @doc """
@@ -187,7 +215,7 @@ defmodule File do
         {:error, :einval}
       else
         _ = do_mkdir_p(parent)
-        case F.make_dir(path) do
+        case :file.make_dir(path) do
           {:error, :eexist} = error ->
             if dir?(path), do: :ok, else: error
           other ->
@@ -228,7 +256,7 @@ defmodule File do
   """
   @spec read(Path.t) :: {:ok, binary} | {:error, posix}
   def read(path) do
-    F.read_file(IO.chardata_to_string(path))
+    :file.read_file(IO.chardata_to_string(path))
   end
 
   @doc """
@@ -269,7 +297,7 @@ defmodule File do
   @spec stat(Path.t, stat_options) :: {:ok, File.Stat.t} | {:error, posix}
   def stat(path, opts \\ []) do
     opts = Keyword.put_new(opts, :time, :universal)
-    case F.read_file_info(IO.chardata_to_string(path), opts) do
+    case :file.read_file_info(IO.chardata_to_string(path), opts) do
       {:ok, fileinfo} ->
         {:ok, File.Stat.from_record(fileinfo)}
       error ->
@@ -278,7 +306,7 @@ defmodule File do
   end
 
   @doc """
-  Same as `stat/2` but returns the `File.Stat` directly and
+  Same as `stat/2` but returns the `File.Stat` directly, or
   throws `File.Error` if an error is returned.
   """
   @spec stat!(Path.t, stat_options) :: File.Stat.t | no_return
@@ -314,7 +342,7 @@ defmodule File do
   @spec lstat(Path.t, stat_options) :: {:ok, File.Stat.t} | {:error, posix}
   def lstat(path, opts \\ []) do
     opts = Keyword.put_new(opts, :time, :universal)
-    case F.read_link_info(IO.chardata_to_string(path), opts) do
+    case :file.read_link_info(IO.chardata_to_string(path), opts) do
       {:ok, fileinfo} ->
         {:ok, File.Stat.from_record(fileinfo)}
       error ->
@@ -323,7 +351,7 @@ defmodule File do
   end
 
   @doc """
-  Same as `lstat/2` but returns the `File.Stat` struct directly and
+  Same as `lstat/2` but returns the `File.Stat` struct directly, or
   throws `File.Error` if an error is returned.
   """
   @spec lstat!(Path.t, stat_options) :: File.Stat.t | no_return
@@ -336,6 +364,44 @@ defmodule File do
     end
   end
 
+  @doc """
+  Reads the symbolic link at `path`.
+
+  If `path` exists and is a symlink, returns `{:ok, target}`, otherwise returns
+  `{:error, reason}`.
+
+  For more details, see
+  [`:file.read_link/1`](http://erlang.org/doc/man/file.html#read_link-1).
+
+  Typical error reasons are:
+
+    * `:einval` - path is not a symbolic link
+    * `:enoent` - path does not exist
+    * `:enotsup` - symbolic links are not supported on the current platform
+
+  """
+  @spec read_link(Path.t) :: {:ok, binary} | {:error, posix}
+  def read_link(path) do
+    case path |> IO.chardata_to_string |> :file.read_link do
+      {:ok, target} -> {:ok, IO.chardata_to_string(target)}
+      error -> error
+    end
+  end
+
+  @doc """
+  Same as `read_link/1` but returns the target directly or throws `File.Error` if an error is
+  returned.
+  """
+  @spec read_link!(Path.t) :: binary | no_return
+  def read_link!(path) do
+    case read_link(path) do
+      {:ok, resolved} ->
+        resolved
+      {:error, reason} ->
+        raise File.Error, reason: reason, action: "read link", path: IO.chardata_to_string(path)
+    end
+  end
+
   @doc """
   Writes the given `File.Stat` back to the filesystem at the given
   path. Returns `:ok` or `{:error, reason}`.
@@ -343,7 +409,7 @@ defmodule File do
   @spec write_stat(Path.t, File.Stat.t, stat_options) :: :ok | {:error, posix}
   def write_stat(path, stat, opts \\ []) do
     opts = Keyword.put_new(opts, :time, :universal)
-    F.write_file_info(IO.chardata_to_string(path), File.Stat.to_record(stat), opts)
+    :file.write_file_info(IO.chardata_to_string(path), File.Stat.to_record(stat), opts)
   end
 
   @doc """
@@ -397,6 +463,32 @@ defmodule File do
     end
   end
 
+  @doc """
+  Creates a hard link `new` to the file `existing`.
+
+  Returns `:ok` if successful, `{:error, reason}` otherwise.
+  If the operating system does not support hard links, returns
+  `{:error, :enotsup}`.
+  """
+  def ln(existing, new) do
+    :file.make_link(IO.chardata_to_string(existing), IO.chardata_to_string(new))
+  end
+
+  @doc """
+  Same as `ln/2` but raises an exception if it fails.
+
+  Returns `:ok` otherwise
+  """
+  def ln!(existing, new) do
+    case ln(existing, new) do
+      :ok -> :ok
+      {:error, reason} ->
+        raise File.LinkError, reason: reason, action: "create hard link",
+          existing: IO.chardata_to_string(existing),
+          new: IO.chardata_to_string(new)
+    end
+  end
+
   @doc """
   Creates a symbolic link `new` to the file or directory `existing`.
 
@@ -405,7 +497,22 @@ defmodule File do
   `{:error, :enotsup}`.
   """
   def ln_s(existing, new) do
-    F.make_symlink(existing, new)
+    :file.make_symlink(IO.chardata_to_string(existing), IO.chardata_to_string(new))
+  end
+
+  @doc """
+  Same as `ln_s/2` but raises an exception if it fails.
+
+  Returns `:ok` otherwise
+  """
+  def ln_s!(existing, new) do
+    case ln_s(existing, new) do
+      :ok -> :ok
+      {:error, reason} ->
+        raise File.LinkError, reason: reason, action: "create symlink",
+          existing: IO.chardata_to_string(existing),
+          new: IO.chardata_to_string(new)
+    end
   end
 
   @doc """
@@ -432,7 +539,7 @@ defmodule File do
   """
   @spec copy(Path.t | io_device, Path.t | io_device, pos_integer | :infinity) :: {:ok, non_neg_integer} | {:error, posix}
   def copy(source, destination, bytes_count \\ :infinity) do
-    F.copy(maybe_to_string(source), maybe_to_string(destination), bytes_count)
+    :file.copy(maybe_to_string(source), maybe_to_string(destination), bytes_count)
   end
 
   @doc """
@@ -472,7 +579,7 @@ defmodule File do
   """
   @spec rename(Path.t, Path.t) :: :ok | {:error, posix}
   def rename(source, destination) do
-    F.rename(source, destination)
+    :file.rename(source, destination)
   end
 
   @doc """
@@ -533,10 +640,9 @@ defmodule File do
   `destination`. If the source is a directory, it copies
   the contents inside source into the destination.
 
-  If a file already exists in the destination,
-  it invokes a callback which should return
-  `true` if the existing file should be overwritten,
-  `false` otherwise. The callback defaults to return `true`.
+  If a file already exists in the destination, it invokes `callback`.
+  `callback` must be a function that takes two arguments: `source` and `destination`.
+  The callback should return `true` if the existing file should be overwritten and `false` otherwise.
 
   If a directory already exists in the destination
   where a file is meant to be (or vice versa), this
@@ -564,15 +670,21 @@ defmodule File do
       File.cp_r "samples", "tmp"
 
       # Same as before, but asks the user how to proceed in case of conflicts
-      File.cp_r "samples", "tmp", fn(source, destination) ->
+      File.cp_r "samples", "tmp", fn source, destination ->
         IO.gets("Overwriting #{destination} by #{source}. Type y to confirm. ") == "y\n"
       end
 
   """
   @spec cp_r(Path.t, Path.t, (Path.t, Path.t -> boolean)) :: {:ok, [binary]} | {:error, posix, binary}
-  def cp_r(source, destination, callback \\ fn(_, _) -> true end) when is_function(callback) do
-    source = IO.chardata_to_string(source)
-    destination = IO.chardata_to_string(destination)
+  def cp_r(source, destination, callback \\ fn _, _ -> true end) when is_function(callback, 2) do
+    source =
+      source
+      |> IO.chardata_to_string()
+      |> assert_no_null_byte!("File.cp_r/3")
+    destination =
+      destination
+      |> IO.chardata_to_string()
+      |> assert_no_null_byte!("File.cp_r/3")
 
     case do_cp_r(source, destination, callback, []) do
       {:error, _, _} = error -> error
@@ -601,12 +713,12 @@ defmodule File do
       {:ok, :regular} ->
         do_cp_file(src, dest, callback, acc)
       {:ok, :symlink} ->
-        case F.read_link(src) do
+        case :file.read_link(src) do
           {:ok, link} -> do_cp_link(link, src, dest, callback, acc)
           {:error, reason} -> {:error, reason, src}
         end
       {:ok, :directory} ->
-        case F.list_dir(src) do
+        case :file.list_dir(src) do
           {:ok, files} ->
             case mkdir(dest) do
               success when success in [:ok, {:error, :eexist}] ->
@@ -634,7 +746,7 @@ defmodule File do
 
   # Both src and dest are files.
   defp do_cp_file(src, dest, callback, acc) do
-    case F.copy(src, {dest, [:exclusive]}) do
+    case :file.copy(src, {dest, [:exclusive]}) do
       {:ok, _} ->
         copy_file_mode!(src, dest)
         [dest | acc]
@@ -655,14 +767,14 @@ defmodule File do
 
   # Both src and dest are files.
   defp do_cp_link(link, src, dest, callback, acc) do
-    case F.make_symlink(link, dest) do
+    case :file.make_symlink(link, dest) do
       :ok ->
         [dest | acc]
       {:error, :eexist} ->
         if path_differs?(src, dest) and callback.(src, dest) do
-          # If rm/1 fails, F.make_symlink/2 will fail
+          # If rm/1 fails, :file.make_symlink/2 will fail
           _ = rm(dest)
-          case F.make_symlink(link, dest) do
+          case :file.make_symlink(link, dest) do
             :ok -> [dest | acc]
             {:error, reason} -> {:error, reason, src}
           end
@@ -704,7 +816,7 @@ defmodule File do
   @spec write(Path.t, iodata, [mode]) :: :ok | {:error, posix}
   def write(path, content, modes \\ []) do
     modes = normalize_modes(modes, false)
-    F.write_file(IO.chardata_to_string(path), content, modes)
+    :file.write_file(IO.chardata_to_string(path), content, modes)
   end
 
   @doc """
@@ -713,7 +825,7 @@ defmodule File do
   @spec write!(Path.t, iodata, [mode]) :: :ok | no_return
   def write!(path, content, modes \\ []) do
     modes = normalize_modes(modes, false)
-    case F.write_file(path, content, modes) do
+    case :file.write_file(path, content, modes) do
       :ok -> :ok
       {:error, reason} ->
         raise File.Error, reason: reason, action: "write to file",
@@ -749,7 +861,7 @@ defmodule File do
   @spec rm(Path.t) :: :ok | {:error, posix}
   def rm(path) do
     path = IO.chardata_to_string(path)
-    case F.delete(path) do
+    case :file.delete(path) do
       :ok ->
         :ok
       {:error, :eacces} = e ->
@@ -761,7 +873,7 @@ defmodule File do
 
   defp change_mode_windows(path) do
     if match? {:win32, _}, :os.type do
-      case F.read_file_info(path) do
+      case :file.read_file_info(path) do
         {:ok, file_info} when elem(file_info, 3) in [:read, :none] ->
           change_mode_windows(path, file_info)
         _ ->
@@ -772,7 +884,7 @@ defmodule File do
 
   defp change_mode_windows(path, file_info) do
     case chmod(path, (elem(file_info, 7) + 0o200)) do
-      :ok -> F.delete(path)
+      :ok -> :file.delete(path)
       {:error, _reason} = error -> error
     end
   end
@@ -805,7 +917,7 @@ defmodule File do
   """
   @spec rmdir(Path.t) :: :ok | {:error, posix}
   def rmdir(path) do
-    F.del_dir(IO.chardata_to_string(path))
+    :file.del_dir(IO.chardata_to_string(path))
   end
 
   @doc """
@@ -841,7 +953,10 @@ defmodule File do
   """
   @spec rm_rf(Path.t) :: {:ok, [binary]} | {:error, posix, binary}
   def rm_rf(path) do
-    do_rm_rf(IO.chardata_to_string(path), {:ok, []})
+    path
+    |> IO.chardata_to_string()
+    |> assert_no_null_byte!("File.rm_rf/1")
+    |> do_rm_rf({:ok, []})
   end
 
   defp do_rm_rf(path, {:ok, _} = entry) do
@@ -902,7 +1017,7 @@ defmodule File do
           _ -> {:ok, :regular}
         end
       {:ok, :directory} ->
-        F.list_dir(path)
+        :file.list_dir(path)
       {:ok, _} ->
         {:ok, :regular}
       {:error, reason} ->
@@ -1008,7 +1123,7 @@ defmodule File do
   def open(path, modes_or_function \\ [])
 
   def open(path, modes) when is_list(modes) do
-    F.open(IO.chardata_to_string(path), normalize_modes(modes, true))
+    :file.open(IO.chardata_to_string(path), normalize_modes(modes, true))
   end
 
   def open(path, function) when is_function(function, 1) do
@@ -1096,7 +1211,7 @@ defmodule File do
   """
   @spec cwd() :: {:ok, binary} | {:error, posix}
   def cwd() do
-    case F.get_cwd do
+    case :file.get_cwd do
       {:ok, base} -> {:ok, IO.chardata_to_string(fix_drive_letter(base))}
       {:error, _} = error -> error
     end
@@ -1104,7 +1219,7 @@ defmodule File do
 
   defp fix_drive_letter([l, ?:, ?/ | rest] = original) when l in ?A..?Z do
     case :os.type() do
-      {:win32, _} -> [l+?a-?A, ?:, ?/ | rest]
+      {:win32, _} -> [l + ?a - ?A, ?:, ?/ | rest]
       _ -> original
     end
   end
@@ -1130,7 +1245,7 @@ defmodule File do
   """
   @spec cd(Path.t) :: :ok | {:error, posix}
   def cd(path) do
-    F.set_cwd(IO.chardata_to_string(path))
+    :file.set_cwd(IO.chardata_to_string(path))
   end
 
   @doc """
@@ -1154,7 +1269,7 @@ defmodule File do
   Raises an error if retrieving or changing the current
   directory fails.
   """
-  @spec cd!(Path.t, (() -> res)) :: res | no_return when res: var
+  @spec cd!(Path.t, (() -> res)) :: res when res: var
   def cd!(path, function) do
     old = cwd!()
     cd!(path)
@@ -1173,7 +1288,7 @@ defmodule File do
   """
   @spec ls(Path.t) :: {:ok, [binary]} | {:error, posix}
   def ls(path \\ ".") do
-    case F.list_dir(IO.chardata_to_string(path)) do
+    case :file.list_dir(IO.chardata_to_string(path)) do
       {:ok, file_list} -> {:ok, Enum.map(file_list, &IO.chardata_to_string/1)}
       {:error, _} = error -> error
     end
@@ -1203,7 +1318,7 @@ defmodule File do
   """
   @spec close(io_device) :: :ok | {:error, posix | :badarg | :terminated}
   def close(io_device) do
-    F.close(io_device)
+    :file.close(io_device)
   end
 
   @doc """
@@ -1234,6 +1349,11 @@ defmodule File do
   One may also consider passing the `:delayed_write` option if the stream
   is meant to be written to under a tight loop.
 
+  ## Byte order marks
+
+  If you pass `:trim_bom` in the modes parameter, the stream will
+  trim UTF-8, UTF-16 and UTF-32 byte order marks when reading from file.
+
   ## Examples
 
       # Read in 2048 byte chunks rather than lines
@@ -1256,26 +1376,28 @@ defmodule File do
 
   ## Permissions
 
-    * 0o400 - read permission: owner
-    * 0o200 - write permission: owner
-    * 0o100 - execute permission: owner
+  File permissions are specified by adding together the following octal flags:
 
-    * 0o040 - read permission: group
-    * 0o020 - write permission: group
-    * 0o010 - execute permission: group
+    * `0o400` - read permission: owner
+    * `0o200` - write permission: owner
+    * `0o100` - execute permission: owner
 
-    * 0o004 - read permission: other
-    * 0o002 - write permission: other
-    * 0o001 - execute permission: other
+    * `0o040` - read permission: group
+    * `0o020` - write permission: group
+    * `0o010` - execute permission: group
 
-  For example, setting the mode 0o755 gives it
+    * `0o004` - read permission: other
+    * `0o002` - write permission: other
+    * `0o001` - execute permission: other
+
+  For example, setting the mode `0o755` gives it
   write, read and execute permission to the owner
   and both read and execute permission to group
   and others.
   """
   @spec chmod(Path.t, non_neg_integer) :: :ok | {:error, posix}
   def chmod(path, mode) do
-    F.change_mode(IO.chardata_to_string(path), mode)
+    :file.change_mode(IO.chardata_to_string(path), mode)
   end
 
   @doc """
@@ -1298,7 +1420,7 @@ defmodule File do
   """
   @spec chgrp(Path.t, non_neg_integer) :: :ok | {:error, posix}
   def chgrp(path, gid) do
-    F.change_group(IO.chardata_to_string(path), gid)
+    :file.change_group(IO.chardata_to_string(path), gid)
   end
 
   @doc """
@@ -1321,7 +1443,7 @@ defmodule File do
   """
   @spec chown(Path.t, non_neg_integer) :: :ok | {:error, posix}
   def chown(path, uid) do
-    F.change_owner(IO.chardata_to_string(path), uid)
+    :file.change_owner(IO.chardata_to_string(path), uid)
   end
 
   @doc """
@@ -1341,14 +1463,26 @@ defmodule File do
 
   @read_ahead_size 64 * 1024
 
+  defp assert_no_null_byte!(binary, operation) do
+    case :binary.match(binary, "\0") do
+      {_, _} ->
+        raise ArgumentError, "cannot execute #{operation} for path with null byte, got: #{inspect binary}"
+      :nomatch ->
+        binary
+    end
+  end
+
   defp normalize_modes([:utf8 | rest], binary?) do
     [encoding: :utf8] ++ normalize_modes(rest, binary?)
   end
   defp normalize_modes([:read_ahead | rest], binary?) do
     [read_ahead: @read_ahead_size] ++ normalize_modes(rest, binary?)
   end
-  # TODO: Deprecate :char_list mode by v1.5
+  # TODO: Remove :char_list mode by 2.0
   defp normalize_modes([mode | rest], _binary?) when mode in [:charlist, :char_list] do
+    if mode == :char_list do
+      IO.warn "the :char_list mode is deprecated, use :charlist"
+    end
     normalize_modes(rest, false)
   end
   defp normalize_modes([mode | rest], binary?) do
@@ -1357,8 +1491,10 @@ defmodule File do
   defp normalize_modes([], true), do: [:binary]
   defp normalize_modes([], false), do: []
 
-  defp maybe_to_string(path) when is_pid(path),
+  defp maybe_to_string(path) when is_list(path),
+    do: IO.chardata_to_string(path)
+  defp maybe_to_string(path) when is_binary(path),
     do: path
   defp maybe_to_string(path),
-    do: IO.chardata_to_string(path)
+    do: path
 end

lib/elixir/lib/file/stream.ex

@@ -36,7 +36,7 @@ defmodule File.Stream do
 
   defimpl Collectable do
     def into(%{path: path, modes: modes, raw: raw} = stream) do
-      modes = for mode <- modes, not mode in [:read], do: mode
+      modes = for mode <- modes, mode not in [:read], do: mode
 
       case :file.open(path, [:write | modes]) do
         {:ok, device} ->
@@ -67,13 +67,14 @@ defmodule File.Stream do
   end
 
   defimpl Enumerable do
-    def reduce(%{path: path, modes: modes, line_or_bytes: line_or_bytes, raw: raw}, acc, fun) do
-      modes = for mode <- modes, not mode in [:write, :append], do: mode
+    @read_ahead_size 64 * 1024
 
+    def reduce(%{path: path, modes: modes, line_or_bytes: line_or_bytes, raw: raw}, acc, fun) do
       start_fun =
         fn ->
-          case :file.open(path, modes) do
-            {:ok, device}    -> device
+          case :file.open(path, read_modes(modes)) do
+            {:ok, device} ->
+              if :trim_bom in modes, do: trim_bom(device), else: device
             {:error, reason} ->
               raise File.Error, reason: reason, action: "stream", path: path
           end
@@ -88,12 +89,70 @@ defmodule File.Stream do
       Stream.resource(start_fun, next_fun, &:file.close/1).(acc, fun)
     end
 
-    def count(_stream) do
-      {:error, __MODULE__}
+    def count(%{path: path, modes: modes, line_or_bytes: :line} = stream) do
+      pattern = :binary.compile_pattern("\n")
+      counter = &count_lines(&1, path, pattern, read_function(stream), 0)
+
+      case File.open(path, read_modes(modes), counter) do
+        {:ok, count} ->
+          {:ok, count}
+        {:error, reason} ->
+          raise File.Error, reason: reason, action: "stream", path: path
+      end
+    end
+
+    def count(%{path: path, line_or_bytes: bytes}) do
+      case File.stat(path) do
+        {:ok, %{size: 0}} ->
+          {:error, __MODULE__}
+        {:ok, %{size: size}} ->
+          {:ok, div(size, bytes) + if(rem(size, bytes) == 0, do: 0, else: 1)}
+        {:error, reason} ->
+          raise File.Error, reason: reason, action: "stream", path: path
+      end
     end
 
     def member?(_stream, _term) do
       {:error, __MODULE__}
     end
+
+    defp trim_bom(device) do
+      header = IO.binread(device, 4)
+      {:ok, _new_pos} = :file.position(device, bom_length(header))
+      device
+    end
+
+    defp bom_length(<<239, 187, 191, _rest::binary>>),
+      do: 3
+    defp bom_length(<<254, 255, _rest::binary>>),
+      do: 2
+    defp bom_length(<<255, 254, _rest::binary>>),
+      do: 2
+    defp bom_length(<<0, 0, 254, 255, _rest::binary>>),
+      do: 4
+    defp bom_length(<<254, 255, 0, 0, _rest::binary>>),
+      do: 4
+    defp bom_length(_binary),
+      do: 0
+
+    defp read_modes(modes) do
+      for mode <- modes, mode not in [:write, :append, :trim_bom], do: mode
+    end
+
+    defp count_lines(device, path, pattern, read, count) do
+      case read.(device) do
+        data when is_binary(data) ->
+          count_lines(device, path, pattern, read, count + count_lines(data, pattern))
+        :eof ->
+          count
+        {:error, reason} ->
+          raise File.Error, reason: reason, action: "stream", path: path
+      end
+    end
+
+    defp count_lines(data, pattern), do: length(:binary.matches(data, pattern))
+
+    defp read_function(%{raw: true}), do: &IO.binread(&1, @read_ahead_size)
+    defp read_function(%{raw: false}), do: &IO.read(&1, @read_ahead_size)
   end
 end

lib/elixir/lib/float.ex

@@ -2,12 +2,14 @@ import Kernel, except: [round: 1]
 
 defmodule Float do
   @moduledoc """
-  Functions for working with floating point numbers.
+  Functions for working with floating-point numbers.
   """
 
   import Bitwise
 
   @power_of_2_to_52 4503599627370496
+  @precision_range 0..15
+  @type precision_range :: 0..15
 
   @doc """
   Parses a binary into a float.
@@ -78,7 +80,7 @@ defmodule Float do
   @doc """
   Rounds a float to the largest integer less than or equal to `num`.
 
-  `floor/2` also accepts a precision to round a floating point value down
+  `floor/2` also accepts a precision to round a floating-point value down
   to an arbitrary number of fractional digits (between 0 and 15).
   The operation is performed on the binary floating point, without a
   conversion to decimal.
@@ -106,15 +108,21 @@ defmodule Float do
       34.25
 
   """
-  @spec floor(float, 0..15) :: float
-  def floor(number, precision \\ 0) when is_float(number) and precision in 0..15 do
+  @spec floor(float, precision_range) :: float
+  def floor(number, precision \\ 0)
+
+  def floor(number, precision) when is_float(number) and precision in @precision_range do
     round(number, precision, :floor)
   end
 
+  def floor(number, precision) when is_float(number) do
+    raise ArgumentError, invalid_precision_message(precision)
+  end
+
   @doc """
   Rounds a float to the smallest integer greater than or equal to `num`.
 
-  `ceil/2` also accepts a precision to round a floating point value down
+  `ceil/2` also accepts a precision to round a floating-point value down
   to an arbitrary number of fractional digits (between 0 and 15).
 
   The operation is performed on the binary floating point, without a
@@ -143,13 +151,19 @@ defmodule Float do
       34.26
 
   """
-  @spec ceil(float, 0..15) :: float
-  def ceil(number, precision \\ 0) when is_float(number) and precision in 0..15 do
+  @spec ceil(float, precision_range) :: float
+  def ceil(number, precision \\ 0)
+
+  def ceil(number, precision) when is_float(number) and precision in @precision_range do
     round(number, precision, :ceil)
   end
 
+  def ceil(number, precision) when is_float(number) do
+    raise ArgumentError, invalid_precision_message(precision)
+  end
+
   @doc """
-  Rounds a floating point value to an arbitrary number of fractional
+  Rounds a floating-point value to an arbitrary number of fractional
   digits (between 0 and 15).
 
   The rounding direction always ties to half up. The operation is
@@ -188,15 +202,20 @@ defmodule Float do
       12.341444444444441
 
   """
-  @spec round(float, 0..15) :: float
-
+  @spec round(float, precision_range) :: float
   # This implementation is slow since it relies on big integers.
   # Faster implementations are available on more recent papers
   # and could be implemented in the future.
-  def round(float, precision \\ 0) when is_float(float) and precision in 0..15 do
+  def round(float, precision \\ 0)
+
+  def round(float, precision) when is_float(float) and precision in @precision_range do
     round(float, precision, :half_up)
   end
 
+  def round(number, precision) when is_float(number) do
+    raise ArgumentError, invalid_precision_message(precision)
+  end
+
   defp round(float, precision, rounding) do
     <<sign::1, exp::11, significant::52-bitstring>> = <<float::float>>
     {num, count, _} = decompose(significant)
@@ -399,22 +418,29 @@ defmodule Float do
     IO.iodata_to_binary(:io_lib_format.fwrite_g(float))
   end
 
-  # TODO: Deprecate by v1.5
+  # TODO: Remove by 2.0
+  # (hard-deprecated in elixir_dispatch)
   @doc false
   def to_char_list(float), do: Float.to_charlist(float)
 
   @doc false
+  # TODO: Remove by 2.0
+  # (hard-deprecated in elixir_dispatch)
   def to_char_list(float, options) do
-    IO.warn "Float.to_char_list/2 is deprecated, use :erlang.float_to_list/2 instead"
     :erlang.float_to_list(float, expand_compact(options))
   end
 
   @doc false
+  # TODO: Remove by 2.0
+  # (hard-deprecated in elixir_dispatch)
   def to_string(float, options) do
-    IO.warn "Float.to_string/2 is deprecated, use :erlang.float_to_binary/2 instead"
     :erlang.float_to_binary(float, expand_compact(options))
   end
 
+  defp invalid_precision_message(precision) do
+    "precision #{precision} is out of valid range of #{inspect @precision_range}"
+  end
+
   defp expand_compact([{:compact, false} | t]), do: expand_compact(t)
   defp expand_compact([{:compact, true} | t]),  do: [:compact | expand_compact(t)]
   defp expand_compact([h | t]),                 do: [h | expand_compact(t)]

lib/elixir/lib/gen_event.ex

@@ -1,304 +1,90 @@
 defmodule GenEvent do
-  # TODO: Deprecate by 1.5
+  # TODO: Remove by 2.0
+
+  # Functions from this module are deprecated in elixir_dispatch.
 
   @moduledoc """
-  A behaviour module for implementing event handling functionality.
+  WARNING: this module is deprecated.
 
-  The event handling model consists of a generic event manager
-  process with an arbitrary number of event handlers which are
-  added and deleted dynamically.
+  If you are interested in implementing an event manager, please read the
+  "Alternatives" section below. If you have to implement an event handler to
+  integrate with an existing system, such as Elixir's Logger, please use
+  `:gen_event` instead.
 
-  An event manager implemented using this module will have a standard
-  set of interface functions and include functionality for tracing and
-  error reporting. It will also fit into a supervision tree.
+  ## Alternatives
 
-  ## Example
+  There are a few suitable alternatives to replace GenEvent. Each of them can be
+  the most beneficial based on the use case.
 
-  There are many use cases for event handlers. For example, a logging
-  system can be built using event handlers where each log message is
-  an event and different event handlers can be attached to handle the
-  log messages. One handler may print error messages on the terminal,
-  another can write it to a file, while a third one can keep the
-  messages in memory (like a buffer) until they are read.
+  ### Supervisor and GenServers
 
-  As an example, let's have a GenEvent that accumulates messages until
-  they are collected by an explicit call.
+  One alternative to GenEvent is a very minimal solution consisting of using a
+  supervisor and multiple GenServers started under it. The supervisor acts as
+  the "event manager" and the children GenServers act as the "event handlers".
+  This approach has some shortcomings (it provides no backpressure for example)
+  but can still replace GenEvent for low-profile usages of it. [This blog post
+  by José
+  Valim](http://blog.plataformatec.com.br/2016/11/replacing-genevent-by-a-supervisor-genserver/)
+  has more detailed information on this approach.
 
-      # Define an Event Handler
-      defmodule LoggerHandler do
-        use GenEvent
+  ### GenStage
 
-        # Callbacks
+  If the use case where you were using GenEvent requires more complex logic,
+  [GenStage](https://github.com/elixir-lang/gen_stage) provides a great
+  alternative. GenStage is an external Elixir library maintained by the Elixir
+  team; it provides tool to implement systems that exchange events in a
+  demand-driven way with built-in support for backpressure. See the [GenStage
+  documentation](https://hexdocs.pm/gen_stage) for more information.
 
-        def handle_event({:log, x}, messages) do
-          {:ok, [x | messages]}
-        end
+  ### `:gen_event`
 
-        def handle_call(:messages, messages) do
-          {:ok, Enum.reverse(messages), []}
-        end
-      end
-
-      # Start a new event manager.
-      {:ok, pid} = GenEvent.start_link([])
-
-      # Attach an event handler to the event manager.
-      GenEvent.add_handler(pid, LoggerHandler, [])
-      #=> :ok
-
-      # Send some events to the event manager.
-      GenEvent.notify(pid, {:log, 1})
-      #=> :ok
-
-      GenEvent.notify(pid, {:log, 2})
-      #=> :ok
-
-      # Call functions on specific handlers in the manager.
-      GenEvent.call(pid, LoggerHandler, :messages)
-      #=> [1, 2]
-
-      GenEvent.call(pid, LoggerHandler, :messages)
-      #=> []
-
-  We start a new event manager by calling `GenEvent.start_link/1`.
-  Notifications can be sent to the event manager which will then
-  invoke `c:handle_event/2` for each registered handler.
-
-  We can add new handlers with `add_handler/3` and `add_mon_handler/3`.
-  Calls can also be made to specific handlers by using `call/3`.
-
-  ## Callbacks
-
-  There are 6 callbacks required to be implemented in a `GenEvent`. By
-  adding `use GenEvent` to your module, Elixir will automatically define
-  all 6 callbacks for you, leaving it up to you to implement the ones
-  you want to customize.
-
-  ## Name Registration
-
-  A GenEvent is bound to the same name registration rules as a `GenServer`.
-  Read more about it in the `GenServer` docs.
-
-  ## Modes
-
-  GenEvent supports three different notifications.
-
-  On `GenEvent.ack_notify/2`, the manager acknowledges each event,
-  providing backpressure, but processing of the message happens
-  asynchronously.
-
-  On `GenEvent.sync_notify/2`, the manager acknowledges an event
-  just after it is processed by all event handlers.
-
-  On `GenEvent.notify/2`, all events are processed asynchronously and
-  there is no ack (which means there is no backpressure).
-
-  ## Streaming
-
-  `GenEvent` messages can be streamed with the help of `stream/2`.
-  You will need to start another process to consume the stream:
-
-      Task.start_link fn ->
-        stream = GenEvent.stream(pid)
-
-        # Discard the next 3 events
-        _ = Enum.take(stream, 3)
-
-        # Print all remaining events
-        for event <- stream do
-          IO.inspect event
-        end
-      end
-
-  Now call `GenEvent.notify/2` multiple times. You will see the
-  first three events will be skipped while the rest will be
-  continuously printed.
-
-  ## Learn more and compatibility
-
-  If you wish to find out more about GenEvent, the documentation and links
-  in Erlang can provide extra insight.
-
-    * [`:gen_event` module documentation](http://www.erlang.org/doc/man/gen_event.html)
-    * [Event Handlers – Learn You Some Erlang for Great Good!](http://learnyousomeerlang.com/event-handlers)
-
-  Keep in mind though Elixir and Erlang gen events are not 100% compatible.
-  The `:gen_event.add_sup_handler/3` is not supported by Elixir's GenEvent,
-  which in turn supports `GenEvent.add_mon_handler/3`.
-
-  The benefits of the monitoring approach are described in the "Don't drink
-  too much kool aid" section of the "Learn you some Erlang" link above. Due
-  to those changes, Elixir's GenEvent does not trap exits by default.
-
-  Furthermore, Elixir also normalizes the `{:error, _}` tuples returned
-  by many functions, in order to be more consistent with themselves and
-  the `GenServer` module.
+  If your use case requires exactly what GenEvent provided, or you have to
+  integrate with an existing `:gen_event`-based system, you can still use the
+  [`:gen_event`](http://erlang.org/doc/man/gen_event.html) Erlang module.
   """
 
-  @doc """
-  Invoked when the handler is added to the `GenEvent` process. `add_handler/3`
-  (and `add_mon_handler/3`) will block until it returns.
-
-  `args` is the argument term (third argument) passed to `add_handler/3`.
-
-  Returning `{:ok, state}` will cause `add_handler/3` to return `:ok` and the
-  handler to become part of the `GenEvent` loop with state `state`.
-
-  Returning `{:ok, state, :hibernate}` is similar to
-  `{:ok, state}` except the `GenEvent` process is hibernated before continuing
-  its loop. See `c:handle_event/2` for more information on hibernation.
-
-  Returning `{:error, reason}` will cause `add_handler/3` to return
-  `{:error, reason}` and the handler is not added to `GenEvent` loop.
-  """
   @callback init(args :: term) ::
     {:ok, state} |
     {:ok, state, :hibernate} |
     {:error, reason :: any} when state: any
 
-  @doc """
-  Invoked to handle `notify/2`, `ack_notify/2` or `sync_notify/2` messages.
-
-  `event` is the event message and `state` is the current state of the handler.
-
-  Returning `{:ok, new_state}` sets the handler's state to `new_state` and the
-  `GenEvent` loop continues.
-
-  Returning `{:ok, new_state, :hibernate}` is similar to
-  `{:ok, new_state}` except the process is hibernated once all handlers have
-  handled the events. The `GenEvent` process will continue the loop once a
-  message is its message queue. If a message is already in the message queue
-  this will be immediately. Hibernating a `GenEvent` causes garbage collection
-  and leaves a continuous heap that minimises the memory used by the process.
-
-  Hibernating should not be used aggressively as too much time could be spent
-  garbage collecting. Normally it should only be used when a message is not
-  expected soon and minimising the memory of the process is shown to be
-  beneficial.
-
-  Returning `:remove_handler` removes the handler from the `GenEvent` loop and
-  calls `c:terminate/2` with reason `:remove_handler` and state `state`.
-  """
   @callback handle_event(event :: term, state :: term) ::
     {:ok, new_state} |
     {:ok, new_state, :hibernate} |
     :remove_handler when new_state: term
 
-  @doc """
-  Invoked to handle synchronous `call/4` messages to a specific handler.
-
-  `request` is the request message sent by a `call/4` and `state` is the current
-  state of the handler.
-
-  Returning `{:ok, reply, new_state}` sends `reply` as a response to the call
-  and sets the handler's state to `new_state`.
-
-  Returning `{:ok, reply, new_state, :hibernate}` is similar to
-  `{:ok, reply, new_state}` except the process is hibernated. See
-  `c:handle_event/2` for more information on hibernation.
-
-  Returning `{:remove_handler, reply}` sends `reply` as a response to the call,
-  removes the handler from the `GenEvent` loop and calls `c:terminate/2` with
-  reason `:remove_handler` and state `state`.
-  """
-
   @callback handle_call(request :: term, state :: term) ::
     {:ok, reply, new_state} |
     {:ok, reply, new_state, :hibernate} |
     {:remove_handler, reply} when reply: term, new_state: term
 
-  @doc """
-  Invoked to handle all other messages. All handlers are run in the `GenEvent`
-  process so messages intended for other handlers should be ignored with a catch
-  all clause.
-
-  `msg` is the message and `state` is the current state of the handler.
-
-  Return values are the same as `c:handle_event/2`.
-  """
   @callback handle_info(msg :: term, state :: term) ::
     {:ok, new_state} |
     {:ok, new_state, :hibernate} |
     :remove_handler when new_state: term
 
-  @doc """
-  Invoked when the server is about to exit. It should do any cleanup required.
-
-  `reason` is removal reason and `state` is the current state of the handler.
-  The return value is returned to `GenEvent.remove_handler/3` or ignored if
-  removing for another reason.
-
-  `reason` is one of:
-
-  -  `:stop` - manager is terminating
-  -  `{:stop, term}` - monitored process terminated (for monitored handlers)
-  -  `:remove_handler` - handler is being removed
-  -  `{:error, term}` - handler crashed or returned a bad value and an error is
-  logged
-  -  `term` - any term passed to functions like `GenEvent.remove_handler/3`
-
-  If part of a supervision tree, a `GenEvent`'s `Supervisor` will send an exit
-  signal when shutting it down. The exit signal is based on the shutdown
-  strategy in the child's specification. If it is `:brutal_kill` the `GenEvent`
-  is killed and so `c:terminate/2` is not called for its handlers. However if it is
-  a timeout the `Supervisor` will send the exit signal `:shutdown` and the
-  `GenEvent` will have the duration of the timeout to call `c:terminate/2` on all
-  of its handlers - if the process is still alive after the timeout it is
-  killed.
-
-  If the `GenEvent` receives an exit signal (that is not `:normal`) from any
-  process when it is not trapping exits it will exit abruptly with the same
-  reason and so not call the handlers' `c:terminate/2`. Note that a process does
-  *NOT* trap exits by default and an exit signal is sent when a linked process
-  exits or its node is disconnected. Therefore it is not guaranteed that
-  `c:terminate/2` is called when a `GenEvent` exits.
-
-  Care should be taken to cleanup because the `GenEvent` can continue to loop
-  after removing the handler. This is different to most other OTP behaviours.
-  For example if the handler controls a `port` (e.g. `:gen_tcp.socket`) or
-  `t:File.io_device/0`, it will be need to be closed in `c:terminate/2` as the
-  process is not exiting so will not be automatically cleaned up.
-  """
   @callback terminate(reason, state :: term) ::
     term when reason: :stop | {:stop, term} | :remove_handler | {:error, term} | term
 
-  @doc """
-  Invoked to change the state of the handler when a different version of the
-  handler's module is loaded (hot code swapping) and the state's term
-  structure should be changed.
-
-  `old_vsn` is the previous version of the module (defined by the `@vsn`
-  attribute) when upgrading. When downgrading the previous version is wrapped in
-  a 2-tuple with first element `:down`. `state` is the current state of the
-  handler and `extra` is any extra data required to change the state.
-
-  Returning `{:ok, new_state}` changes the state to `new_state` and the code
-  change is successful.
-
-  If `c:code_change/3` raises, the code change fails and the handler will continue
-  with its previous state. Therefore this callback does not usually contain side
-  effects.
-  """
   @callback code_change(old_vsn, state :: term, extra :: term) ::
     {:ok, new_state :: term} when old_vsn: term | {:down, term}
 
-  @typedoc "Return values of `start*` functions"
   @type on_start :: {:ok, pid} | {:error, {:already_started, pid}}
 
-  @typedoc "The GenEvent manager name"
   @type name :: atom | {:global, term} | {:via, module, term}
 
-  @typedoc "Options used by the `start*` functions"
   @type options :: [name: name]
 
-  @typedoc "The event manager reference"
   @type manager :: pid | name | {atom, node}
 
-  @typedoc "Supported values for new handlers"
   @type handler :: atom | {atom, term}
 
   @doc false
   defmacro __using__(_) do
+    %{file: file, line: line} = __CALLER__
+    deprecation_message = "the GenEvent module is deprecated, see its documentation for alternatives"
+    :elixir_errors.warn(line, file, deprecation_message)
+
     quote location: :keep do
       @behaviour :gen_event
 
@@ -347,33 +133,13 @@ defmodule GenEvent do
     end
   end
 
-  @doc """
-  Starts an event manager linked to the current process.
-
-  This is often used to start the `GenEvent` as part of a supervision tree.
-
-  It accepts the `:name` option which is described under the `Name Registration`
-  section in the `GenServer` module docs.
-
-  If the event manager is successfully created and initialized, the function
-  returns `{:ok, pid}`, where `pid` is the PID of the server. If a process with
-  the specified server name already exists, the function returns
-  `{:error, {:already_started, pid}}` with the PID of that process.
-
-  Note that a `GenEvent` started with `start_link/1` is linked to the
-  parent process and will exit not only on crashes but also if the parent
-  process exits with `:normal` reason.
-  """
+  @doc false
   @spec start_link(options) :: on_start
   def start_link(options \\ []) when is_list(options) do
     do_start(:link, options)
   end
 
-  @doc """
-  Starts an event manager process without links (outside of a supervision tree).
-
-  See `start_link/1` for more information.
-  """
+  @doc false
   @spec start(options) :: on_start
   def start(options \\ []) when is_list(options) do
     do_start(:nolink, options)
@@ -405,106 +171,27 @@ defmodule GenEvent do
     end
   end
 
-  @doc """
-  Returns a stream that consumes events from the `manager`.
-
-  The stream is a `GenEvent` struct that implements the `Enumerable`
-  protocol. Consumption of events only begins when enumeration starts.
-
-  Note streaming is specific to Elixir's GenEvent and does not work
-  with Erlang ones.
-
-  ## Options
-
-    * `:timeout` - raises if no event arrives in X milliseconds
-      (defaults to `:infinity`)
-
-  """
-  @spec stream(manager, Keyword.t) :: GenEvent.Stream.t
+  @doc false
+  @spec stream(manager, keyword) :: GenEvent.Stream.t
   def stream(manager, options \\ []) do
     %GenEvent.Stream{
       manager: manager,
       timeout: Keyword.get(options, :timeout, :infinity)}
   end
 
-  @doc """
-  Adds a new event handler to the event `manager`.
-
-  The event manager will call the `c:init/1` callback with `args` to
-  initiate the event handler and its internal state.
-
-  If `c:init/1` returns a correct value indicating successful completion,
-  the event manager adds the event handler and this function returns
-  `:ok`. If the callback fails with `reason` or returns `{:error, reason}`,
-  the event handler is ignored and this function returns `{:error, reason}`.
-
-  If the given handler was previously installed at the manager, this
-  function returns `{:error, :already_present}`.
-
-  For installing multiple instances of the same handler, `{Module, id}` instead
-  of `Module` must be used. The handler could be then referenced with
-  `{Module, id}` instead of just `Module`.
-  """
+  @doc false
   @spec add_handler(manager, handler, term) :: :ok | {:error, term}
   def add_handler(manager, handler, args) do
     rpc(manager, {:add_handler, handler, args})
   end
 
-  @doc """
-  Adds a monitored event handler to the event `manager`.
-
-  Expects the same input and returns the same values as `add_handler/3`.
-
-  ## Monitored handlers
-
-  A monitored handler implies the calling process will now be monitored
-  by the GenEvent manager.
-
-  If the calling process later terminates with `reason`, the event manager
-  will delete the event handler by calling the `c:terminate/2` callback with
-  `{:stop, reason}` as argument. If the event handler later is deleted,
-  the event manager sends a message `{:gen_event_EXIT, handler, reason}`
-  to the calling process. Reason is one of the following:
-
-    * `:normal` - if the event handler has been removed due to a call to
-      `remove_handler/3`, or `:remove_handler` has been returned by a callback
-      function
-
-    * `:shutdown` - if the event handler has been removed because the event
-      manager is terminating
-
-    * `{:swapped, new_handler, pid}` - if the process PID has replaced the
-      event handler by another
-
-    * `term` - if the event handler is removed due to an error. Which term
-      depends on the error
-
-  Keep in mind that the `{:gen_event_EXIT, handler, reason}` message is not
-  guaranteed to be delivered in case the manager crashes. If you want to
-  guarantee the message is delivered, you have two options:
-
-    * monitor the event manager
-    * link to the event manager and then set `Process.flag(:trap_exit, true)`
-      in your handler callback
-
-  Finally, this functionality only works with GenEvent started via this
-  module (it is not backwards compatible with Erlang's `:gen_event`).
-  """
+  @doc false
   @spec add_mon_handler(manager, handler, term) :: :ok | {:error, term}
   def add_mon_handler(manager, handler, args) do
     rpc(manager, {:add_mon_handler, handler, args, self()})
   end
 
-  @doc """
-  Sends an event notification to the event `manager`.
-
-  The event manager will call `c:handle_event/2` for each
-  installed event handler.
-
-  `notify` is asynchronous and will return immediately after the
-  notification is sent. `notify` will not fail even if the specified
-  event manager does not exist, unless it is specified as an atom.
-  """
+  @doc false
   @spec notify(manager, term) :: :ok
   def notify(manager, event)
 
@@ -535,45 +222,19 @@ defmodule GenEvent do
     :ok
   end
 
-  @doc """
-  Sends a sync event notification to the event `manager`.
-
-  In other words, this function only returns `:ok` after the event manager
-  invokes the `c:handle_event/2` callback on each installed event handler.
-
-  See `notify/2` for more info.
-  """
+  @doc false
   @spec sync_notify(manager, term) :: :ok
   def sync_notify(manager, event) do
     rpc(manager, {:sync_notify, event})
   end
 
-  @doc """
-  Sends an ack event notification to the event `manager`.
-
-  In other words, this function only returns `:ok` as soon as the
-  event manager starts processing this event, but it does not wait
-  for event handlers to process the sent event.
-
-  See `notify/2` for more info. Note this function is specific
-  to Elixir's GenEvent and does not work with Erlang ones.
-  """
+  @doc false
   @spec ack_notify(manager, term) :: :ok
   def ack_notify(manager, event) do
     rpc(manager, {:ack_notify, event})
   end
 
-  @doc """
-  Makes a synchronous call to the event `handler` installed in `manager`.
-
-  The given `request` is sent and the caller waits until a reply arrives or
-  a timeout occurs. The event manager will call `c:handle_call/2` to handle
-  the request.
-
-  The return value `reply` is defined in the return value of `c:handle_call/2`.
-  If the specified event handler is not installed, the function returns
-  `{:error, :not_found}`.
-  """
+  @doc false
   @spec call(manager, handler, term, timeout) ::  term | {:error, term}
   def call(manager, handler, request, timeout \\ 5000) do
     try do
@@ -586,70 +247,31 @@ defmodule GenEvent do
     end
   end
 
-  @doc """
-  Removes an event handler from the event `manager`.
-
-  The event manager will call `c:terminate/2` to terminate the event handler
-  and return the callback value. If the specified event handler is not
-  installed, the function returns `{:error, :not_found}`.
-  """
+  @doc false
   @spec remove_handler(manager, handler, term) :: term | {:error, term}
   def remove_handler(manager, handler, args) do
     rpc(manager, {:delete_handler, handler, args})
   end
 
-  @doc """
-  Replaces an old event handler with a new one in the event `manager`.
-
-  First, the old event handler is deleted by calling `c:terminate/2` with
-  the given `args1` and collects the return value. Then the new event handler
-  is added and initiated by calling `init({args2, term})`, where `term` is the
-  return value of calling `c:terminate/2` in the old handler. This makes it
-  possible to transfer information from one handler to another.
-
-  The new handler will be added even if the specified old event handler
-  is not installed or if the handler fails to terminate with a given reason
-  in which case `state = {:error, term}`.
-
-  If `c:init/1` in the second handler returns a correct value, this
-  function returns `:ok`.
-  """
+  @doc false
   @spec swap_handler(manager, handler, term, handler, term) :: :ok | {:error, term}
   def swap_handler(manager, handler1, args1, handler2, args2) do
     rpc(manager, {:swap_handler, handler1, args1, handler2, args2})
   end
 
-  @doc """
-  Replaces an old event handler with a new monitored one in the event `manager`.
-
-  Read the docs for `add_mon_handler/3` and `swap_handler/5` for more information.
-  """
+  @doc false
   @spec swap_mon_handler(manager, handler, term, handler, term) :: :ok | {:error, term}
   def swap_mon_handler(manager, handler1, args1, handler2, args2) do
     rpc(manager, {:swap_mon_handler, handler1, args1, handler2, args2, self()})
   end
 
-  @doc """
-  Returns a list of all event handlers installed in the `manager`.
-  """
+  @doc false
   @spec which_handlers(manager) :: [handler]
   def which_handlers(manager) do
     rpc(manager, :which_handlers)
   end
 
-  @doc """
-  Stops the manager with the given `reason`.
-
-  Before terminating, the event manager will call
-  `terminate(:stop, ...)` for each installed event handler.
-  It returns `:ok` if the manager terminates with the given
-  reason, if it terminates with another reason, the call will
-  exit.
-
-  This function keeps OTP semantics regarding error reporting.
-  If the reason is any other than `:normal`, `:shutdown` or
-  `{:shutdown, _}`, an error report will be logged.
-  """
+  @doc false
   @spec stop(manager, reason :: term, timeout) :: :ok
   def stop(manager, reason \\ :normal, timeout \\ :infinity) do
     :gen.stop(manager, reason, timeout)
@@ -1151,7 +773,7 @@ defmodule GenEvent do
   defp report_error(handler, reason, state, last_in, name) do
     reason =
       case reason do
-        {:undef, [{m, f, a, _} | _]=mfas} ->
+        {:undef, [{m, f, a, _} | _] = mfas} ->
           cond do
             :code.is_loaded(m) === false ->
               {:"module could not be loaded", mfas}

lib/elixir/lib/gen_event/stream.ex

@@ -137,7 +137,7 @@ defimpl Enumerable, for: GenEvent.Stream do
   # If we reach this branch, the handler was not removed yet,
   # so we trigger a request for doing so.
   defp stop(stream, {pid, ref, _} = acc) do
-    _ = GenEvent.remove_handler(pid, {pid, ref}, :shutdown)
+    _ = :gen_event.delete_handler(pid, {pid, ref}, :shutdown)
     stop(stream, {:removed, acc})
   end
 

lib/elixir/lib/gen_server.ex

@@ -11,8 +11,8 @@ defmodule GenServer do
   ## Example
 
   The GenServer behaviour abstracts the common client-server interaction.
-  Developers are only required to implement the callbacks and functionality they are
-  interested in.
+  Developers are only required to implement the callbacks and functionality
+  they are interested in.
 
   Let's start with a code example and then explore the available callbacks.
   Imagine we want a GenServer that works like a stack, allowing us to push
@@ -56,13 +56,28 @@ defmodule GenServer do
   that must be handled by the `c:handle_call/3` callback in the GenServer.
   A `cast/2` message must be handled by `c:handle_cast/2`.
 
-  ## Callbacks
+  ## use GenServer and callbacks
 
   There are 6 callbacks required to be implemented in a `GenServer`. By
   adding `use GenServer` to your module, Elixir will automatically define
   all 6 callbacks for you, leaving it up to you to implement the ones
   you want to customize.
 
+  `use GenServer` also defines a `child_spec/1` function, allowing the
+  defined module to be put under a supervision tree. The generated
+  `child_spec/1` can be customized with the following options:
+
+    * `:id` - the child specification id, defauts to the current module
+    * `:start` - how to start the child process (defaults to calling `__MODULE__.start_link/1`)
+    * `:restart` - when the child should be restarted, defaults to `:permanent`
+    * `:shutdown` - how to shut down the child
+
+  For example:
+
+      use GenServer, restart: :transient, shutdown: 10_000
+
+  See the `Supervisor` docs for more information.
+
   ## Name Registration
 
   Both `start_link/3` and `start/3` support the `GenServer` to register
@@ -73,12 +88,12 @@ defmodule GenServer do
       using `Process.register/2`.
 
     * `{:global, term}`- the GenServer is registered globally with the given
-      term using the functions in the `:global` module.
+      term using the functions in the [`:global` module](http://www.erlang.org/doc/man/global.html).
 
     * `{:via, module, term}` - the GenServer is registered with the given
       mechanism and name. The `:via` option expects a module that exports
       `register_name/2`, `unregister_name/1`, `whereis_name/1` and `send/2`.
-      One such example is the `:global` module which uses these functions
+      One such example is the [`:global` module](http://www.erlang.org/doc/man/global.html) which uses these functions
       for keeping the list of names of processes and their associated PIDs
       that are available globally for a network of Elixir nodes. Elixir also
       ships with a local, decentralized and scalable registry called `Registry`
@@ -200,12 +215,12 @@ defmodule GenServer do
   ## Debugging with the :sys module
 
   GenServers, as [special processes](http://erlang.org/doc/design_principles/spec_proc.html),
-  can be debugged using the `:sys` module. Through various hooks, this module
+  can be debugged using the [`:sys` module](http://www.erlang.org/doc/man/sys.html). Through various hooks, this module
   allows developers to introspect the state of the process and trace
   system events that happen during its execution, such as received messages,
   sent replies and state changes.
 
-  Let's explore the basic functions from the `:sys` module used for debugging:
+  Let's explore the basic functions from the [`:sys` module](http://www.erlang.org/doc/man/sys.html) used for debugging:
 
     * [`:sys.get_state/2`](http://erlang.org/doc/man/sys.html#get_state-2) -
       allows retrieval of the state of the process. In the case of
@@ -227,6 +242,10 @@ defmodule GenServer do
       to switch off debugging once we're done. Excessive debug handlers or
       those that should be turned off, but weren't, can seriously damage
       the performance of the system.
+    * [`:sys.suspend/2`](http://erlang.org/doc/man/sys.html#suspend-2) - allows
+      to suspend a process so that it only replies to system messages but no
+      other messages. A suspended process can be reactivated via
+      [`:sys.resume/2`](http://erlang.org/doc/man/sys.html#resume-2).
 
   Let's see how we could use those functions for debugging the stack server
   we defined earlier.
@@ -430,12 +449,17 @@ defmodule GenServer do
   `reason` is exit reason and `state` is the current state of the `GenServer`.
   The return value is ignored.
 
-  `c:terminate/2` is called if a callback (except `c:init/1`) returns a `:stop`
-  tuple, raises, calls `Kernel.exit/1` or returns an invalid value. It may also
-  be called if the `GenServer` traps exits using `Process.flag/2` *and* the
-  parent process sends an exit signal.
+  `c:terminate/2` is called if a callback (except `c:init/1`) does one of the
+  following:
 
-  If part of a supervision tree a `GenServer`'s `Supervisor` will send an exit
+    * returns a `:stop` tuple
+    * raises
+    * calls `Kernel.exit/1`
+    * returns an invalid value
+    * the `GenServer` traps exits (using `Process.flag/2`) *and* the parent
+      process sends an exit signal
+
+  If part of a supervision tree, a `GenServer`'s `Supervisor` will send an exit
   signal when shutting it down. The exit signal is based on the shutdown
   strategy in the child's specification. If it is `:brutal_kill` the `GenServer`
   is killed and so `c:terminate/2` is not called. However if it is a timeout the
@@ -456,7 +480,7 @@ defmodule GenServer do
   `:gen_tcp.socket`) or `t:File.io_device/0`, they will be closed on receiving a
   `GenServer`'s exit signal and do not need to be closed in `c:terminate/2`.
 
-  If `reason` is not `:normal`, `:shutdown` nor `{:shutdown, term}` an error is
+  If `reason` is not `:normal`, `:shutdown`, nor `{:shutdown, term}` an error is
   logged.
   """
   @callback terminate(reason, state :: term) ::
@@ -537,10 +561,25 @@ defmodule GenServer do
   @type from :: {pid, tag :: term}
 
   @doc false
-  defmacro __using__(_) do
-    quote location: :keep do
+  defmacro __using__(opts) do
+    quote location: :keep, bind_quoted: [opts: opts] do
       @behaviour GenServer
 
+      spec = [
+        id: opts[:id] || __MODULE__,
+        start: Macro.escape(opts[:start]) || quote(do: {__MODULE__, :start_link, [arg]}),
+        restart: opts[:restart] || :permanent,
+        shutdown: opts[:shutdown] || 5000,
+        type: :worker
+      ]
+
+      @doc false
+      def child_spec(arg) do
+        %{unquote_splicing(spec)}
+      end
+
+      defoverridable child_spec: 1
+
       @doc false
       def init(args) do
         {:ok, args}
@@ -568,8 +607,8 @@ defmodule GenServer do
             {_, []}   -> self()
             {_, name} -> name
           end
-        :error_logger.warning_msg('~p ~p received unexpected message in handle_info/2: ~p~n',
-                                  [__MODULE__, proc, msg])
+        :error_logger.error_msg('~p ~p received unexpected message in handle_info/2: ~p~n',
+                                [__MODULE__, proc, msg])
         {:noreply, state}
       end
 
@@ -598,8 +637,7 @@ defmodule GenServer do
         {:ok, state}
       end
 
-      defoverridable [init: 1, handle_call: 3, handle_info: 2,
-                      handle_cast: 2, terminate: 2, code_change: 3]
+      defoverridable GenServer
     end
   end
 
@@ -627,8 +665,7 @@ defmodule GenServer do
       milliseconds initializing or it will be terminated and the start function
       will return `{:error, :timeout}`
 
-    * `:debug` - if present, the corresponding function in the [`:sys`
-      module](http://www.erlang.org/doc/man/sys.html) is invoked
+    * `:debug` - if present, the corresponding function in the [`:sys` module](http://www.erlang.org/doc/man/sys.html) is invoked
 
     * `:spawn_opt` - if present, its value is passed as options to the
       underlying process as in `Process.spawn/4`
@@ -670,7 +707,7 @@ defmodule GenServer do
         :gen.start(:gen_server, link, tuple, module, args, opts)
       {{:via, via_module, _term} = tuple, opts} when is_atom(via_module) ->
         :gen.start(:gen_server, link, tuple, module, args, opts)
-      other ->
+      {other, _} ->
         raise ArgumentError, """
         expected :name option to be one of:
 
@@ -685,7 +722,7 @@ defmodule GenServer do
   end
 
   @doc """
-  Stops the server with the given `reason`.
+  Synchronously stops the server with the given `reason`.
 
   The `c:terminate/2` callback of the given `server` will be invoked before
   exiting. This function returns `:ok` if the server terminates with the
@@ -891,7 +928,7 @@ defmodule GenServer do
 
   @doc """
   Returns the `pid` or `{name, node}` of a GenServer process, or `nil` if
-  no process is associated with the given name.
+  no process is associated with the given `server`.
 
   ## Examples
 
@@ -903,25 +940,32 @@ defmodule GenServer do
 
   """
   @spec whereis(server) :: pid | {atom, node} | nil
+  def whereis(server)
+
   def whereis(pid) when is_pid(pid), do: pid
+
   def whereis(name) when is_atom(name) do
     Process.whereis(name)
   end
+
   def whereis({:global, name}) do
     case :global.whereis_name(name) do
       pid when is_pid(pid) -> pid
       :undefined           -> nil
     end
   end
+
   def whereis({:via, mod, name}) do
     case apply(mod, :whereis_name, [name]) do
       pid when is_pid(pid) -> pid
       :undefined           -> nil
     end
   end
+
   def whereis({name, local}) when is_atom(name) and local == node() do
     Process.whereis(name)
   end
+
   def whereis({name, node} = server) when is_atom(name) and is_atom(node) do
     server
   end

lib/elixir/lib/hash_dict.ex

@@ -5,7 +5,8 @@ defmodule HashDict do
   Use the `Map` module instead.
   """
 
-  # TODO: Deprecate every function by 1.4
+  # TODO: Remove by 2.0
+  # (hard-deprecated in elixir_dispatch)
 
   use Dict
 
@@ -220,16 +221,35 @@ defmodule HashDict do
 end
 
 defimpl Enumerable, for: HashDict do
-  def reduce(dict, acc, fun),  do: HashDict.reduce(dict, acc, fun)
-  def member?(dict, {k, v}), do: {:ok, match?({:ok, ^v}, HashDict.fetch(dict, k))}
-  def member?(_dict, _),       do: {:ok, false}
-  def count(dict),             do: {:ok, HashDict.size(dict)}
+  def reduce(dict, acc, fun) do
+    # Avoid warnings about HashDict being deprecated.
+    module = HashDict
+    module.reduce(dict, acc, fun)
+  end
+
+  def member?(dict, {key, value}) do
+    # Avoid warnings about HashDict being deprecated.
+    module = HashDict
+    {:ok, match?({:ok, ^value}, module.fetch(dict, key))}
+  end
+
+  def member?(_dict, _) do
+    {:ok, false}
+  end
+
+  def count(dict) do
+    # Avoid warnings about HashDict being deprecated.
+    module = HashDict
+    {:ok, module.size(dict)}
+  end
 end
 
 defimpl Collectable, for: HashDict do
   def into(original) do
+    # Avoid warnings about HashDict being deprecated.
+    module = HashDict
     {original, fn
-      dict, {:cont, {k, v}} -> HashDict.put(dict, k, v)
+      dict, {:cont, {key, value}} -> module.put(dict, key, value)
       dict, :done -> dict
       _, :halt -> :ok
     end}
@@ -240,6 +260,8 @@ defimpl Inspect, for: HashDict do
   import Inspect.Algebra
 
   def inspect(dict, opts) do
-    concat ["#HashDict<", Inspect.List.inspect(HashDict.to_list(dict), opts), ">"]
+    # Avoid warnings about HashDict being deprecated.
+    module = HashDict
+    concat ["#HashDict<", Inspect.List.inspect(module.to_list(dict), opts), ">"]
   end
 end

lib/elixir/lib/hash_set.ex

@@ -5,7 +5,8 @@ defmodule HashSet do
   Use the `MapSet` module instead.
   """
 
-  # TODO: Deprecate every function by 1.4
+  # TODO: Remove by 2.0
+  # (hard-deprecated in elixir_dispatch)
 
   @node_bitmap 0b111
   @node_shift 3
@@ -235,15 +236,31 @@ defmodule HashSet do
 end
 
 defimpl Enumerable, for: HashSet do
-  def reduce(set, acc, fun), do: HashSet.reduce(set, acc, fun)
-  def member?(set, v),       do: {:ok, HashSet.member?(set, v)}
-  def count(set),            do: {:ok, HashSet.size(set)}
+  def reduce(set, acc, fun) do
+    # Avoid warnings about HashSet being deprecated.
+    module = HashSet
+    module.reduce(set, acc, fun)
+  end
+
+  def member?(set, term) do
+    # Avoid warnings about HashSet being deprecated.
+    module = HashSet
+    {:ok, module.member?(set, term)}
+  end
+
+  def count(set) do
+    # Avoid warnings about HashSet being deprecated.
+    module = HashSet
+    {:ok, module.size(set)}
+  end
 end
 
 defimpl Collectable, for: HashSet do
   def into(original) do
+    # Avoid warnings about HashSet being deprecated.
+    module = HashSet
     {original, fn
-      set, {:cont, x} -> HashSet.put(set, x)
+      set, {:cont, term} -> module.put(set, term)
       set, :done -> set
       _, :halt -> :ok
     end}
@@ -254,6 +271,8 @@ defimpl Inspect, for: HashSet do
   import Inspect.Algebra
 
   def inspect(set, opts) do
-    concat ["#HashSet<", Inspect.List.inspect(HashSet.to_list(set), opts), ">"]
+    # Avoid warnings about HashSet being deprecated.
+    module = HashSet
+    concat ["#HashSet<", Inspect.List.inspect(module.to_list(set), opts), ">"]
   end
 end

lib/elixir/lib/inspect.ex

@@ -69,86 +69,44 @@ defimpl Inspect, for: Atom do
   defp color_key(nil), do: :nil
   defp color_key(_), do: :atom
 
-  def inspect(false),  do: "false"
-  def inspect(true),   do: "true"
-  def inspect(nil),    do: "nil"
-  def inspect(:""),    do: ":\"\""
+  def inspect(atom) when is_nil(atom) or is_boolean(atom) do
+    Atom.to_string(atom)
+  end
 
-  def inspect(atom) do
+  def inspect(atom) when is_atom(atom) do
     binary = Atom.to_string(atom)
 
-    cond do
-      valid_ref_identifier?(binary) ->
-        if only_elixir?(binary) do
-          binary
-        else
-          "Elixir." <> rest = binary
-          rest
+    case Macro.classify_identifier(atom) do
+      :alias ->
+        case binary do
+          binary when binary in ["Elixir", "Elixir.Elixir"] ->
+            binary
+          "Elixir.Elixir." <> _rest ->
+            binary
+          "Elixir." <> rest ->
+            rest
         end
-      valid_atom_identifier?(binary) ->
+      type when type in [:callable, :not_callable] ->
         ":" <> binary
-      atom in [:%{}, :{}, :<<>>, :..., :%] ->
-        ":" <> binary
-      atom in Macro.binary_ops or atom in Macro.unary_ops ->
-        ":" <> binary
-      true ->
-        IO.iodata_to_binary [?:, ?", Inspect.BitString.escape(binary, ?"), ?"]
+      :other ->
+        {escaped, _} = Inspect.BitString.escape(binary, ?")
+        IO.iodata_to_binary [?:, ?", escaped, ?"]
     end
   end
-
-  defp only_elixir?("Elixir." <> rest), do: only_elixir?(rest)
-  defp only_elixir?("Elixir"), do: true
-  defp only_elixir?(_), do: false
-
-  # Detect if atom is an atom alias (Elixir.Foo.Bar.Baz)
-
-  defp valid_ref_identifier?("Elixir" <> rest) do
-    valid_ref_piece?(rest)
-  end
-
-  defp valid_ref_identifier?(_), do: false
-
-  defp valid_ref_piece?(<<?., h, t::binary>>) when h in ?A..?Z do
-    valid_ref_piece? valid_identifier?(t)
-  end
-
-  defp valid_ref_piece?(<<>>), do: true
-  defp valid_ref_piece?(_),    do: false
-
-  # Detect if atom
-
-  defp valid_atom_identifier?(<<h, t::binary>>) when h in ?a..?z or h in ?A..?Z or h == ?_ do
-    valid_atom_piece?(t)
-  end
-
-  defp valid_atom_identifier?(_), do: false
-
-  defp valid_atom_piece?(t) do
-    case valid_identifier?(t) do
-      <<>>              -> true
-      <<??>>            -> true
-      <<?!>>            -> true
-      <<?@, t::binary>> -> valid_atom_piece?(t)
-      _                 -> false
-    end
-  end
-
-  defp valid_identifier?(<<h, t::binary>>)
-      when h in ?a..?z
-      when h in ?A..?Z
-      when h in ?0..?9
-      when h == ?_ do
-    valid_identifier? t
-  end
-
-  defp valid_identifier?(other), do: other
 end
 
 defimpl Inspect, for: BitString do
-  def inspect(term, %Inspect.Opts{binaries: bins, base: base} = opts) when is_binary(term) do
-    if base == :decimal and (bins == :as_strings or (bins == :infer and String.printable?(term))) do
-      inspected = IO.iodata_to_binary([?", escape(term, ?"), ?"])
-      color(inspected, :string, opts)
+  def inspect(term, opts) when is_binary(term) do
+    %Inspect.Opts{binaries: bins, base: base, printable_limit: printable_limit} = opts
+
+    if base == :decimal and
+       (bins == :as_strings or (bins == :infer and String.printable?(term, printable_limit))) do
+      inspected =
+        case escape(term, ?", printable_limit) do
+          {escaped, ""} -> [?", escaped, ?"]
+          {escaped, _} -> [?", escaped, ?", " <> ..."]
+        end
+      color(IO.iodata_to_binary(inspected), :string, opts)
     else
       inspect_bitstring(term, opts)
     end
@@ -162,56 +120,66 @@ defimpl Inspect, for: BitString do
 
   @doc false
   def escape(other, char) do
-    escape(other, char, [])
+    escape(other, char, :infinity, [])
   end
 
-  defp escape(<<char, t::binary>>, char, acc) do
-    escape(t, char, [acc | [?\\, char]])
+  @doc false
+  def escape(other, char, count) do
+    escape(other, char, count, [])
   end
-  defp escape(<<?#, ?{, t::binary>>, char, acc) do
-    escape(t, char, [acc | '\\\#{'])
+
+  defp escape(<<_, _::binary>> = binary, _char, 0, acc) do
+    {acc, binary}
   end
-  defp escape(<<?\a, t::binary>>, char, acc) do
-    escape(t, char, [acc | '\\a'])
+  defp escape(<<char, t::binary>>, char, count, acc) do
+    escape(t, char, decrement(count), [acc | [?\\, char]])
   end
-  defp escape(<<?\b, t::binary>>, char, acc) do
-    escape(t, char, [acc | '\\b'])
+  defp escape(<<?#, ?{, t::binary>>, char, count, acc) do
+    escape(t, char, decrement(count), [acc | '\\\#{'])
   end
-  defp escape(<<?\d, t::binary>>, char, acc) do
-    escape(t, char, [acc | '\\d'])
+  defp escape(<<?\a, t::binary>>, char, count, acc) do
+    escape(t, char, decrement(count), [acc | '\\a'])
   end
-  defp escape(<<?\e, t::binary>>, char, acc) do
-    escape(t, char, [acc | '\\e'])
+  defp escape(<<?\b, t::binary>>, char, count, acc) do
+    escape(t, char, decrement(count), [acc | '\\b'])
   end
-  defp escape(<<?\f, t::binary>>, char, acc) do
-    escape(t, char, [acc | '\\f'])
+  defp escape(<<?\d, t::binary>>, char, count, acc) do
+    escape(t, char, decrement(count), [acc | '\\d'])
   end
-  defp escape(<<?\n, t::binary>>, char, acc) do
-    escape(t, char, [acc | '\\n'])
+  defp escape(<<?\e, t::binary>>, char, count, acc) do
+    escape(t, char, decrement(count), [acc | '\\e'])
   end
-  defp escape(<<?\r, t::binary>>, char, acc) do
-    escape(t, char, [acc | '\\r'])
+  defp escape(<<?\f, t::binary>>, char, count, acc) do
+    escape(t, char, decrement(count), [acc | '\\f'])
   end
-  defp escape(<<?\\, t::binary>>, char, acc) do
-    escape(t, char, [acc | '\\\\'])
+  defp escape(<<?\n, t::binary>>, char, count, acc) do
+    escape(t, char, decrement(count), [acc | '\\n'])
   end
-  defp escape(<<?\t, t::binary>>, char, acc) do
-    escape(t, char, [acc | '\\t'])
+  defp escape(<<?\r, t::binary>>, char, count, acc) do
+    escape(t, char, decrement(count), [acc | '\\r'])
   end
-  defp escape(<<?\v, t::binary>>, char, acc) do
-    escape(t, char, [acc | '\\v'])
+  defp escape(<<?\\, t::binary>>, char, count, acc) do
+    escape(t, char, decrement(count), [acc | '\\\\'])
   end
-  defp escape(<<h::utf8, t::binary>>, char, acc)
+  defp escape(<<?\t, t::binary>>, char, count, acc) do
+    escape(t, char, decrement(count), [acc | '\\t'])
+  end
+  defp escape(<<?\v, t::binary>>, char, count, acc) do
+    escape(t, char, decrement(count), [acc | '\\v'])
+  end
+  defp escape(<<h::utf8, t::binary>>, char, count, acc)
        when h in 0x20..0x7E
        when h in 0xA0..0xD7FF
        when h in 0xE000..0xFFFD
        when h in 0x10000..0x10FFFF do
-    escape(t, char, [acc | <<h::utf8>>])
+    escape(t, char, decrement(count), [acc | <<h::utf8>>])
   end
-  defp escape(<<h, t::binary>>, char, acc) do
-    escape(t, char, [acc | escape_char(h)])
+  defp escape(<<h, t::binary>>, char, count, acc) do
+    escape(t, char, decrement(count), [acc | escape_char(h)])
+  end
+  defp escape(<<>>, _char, _count, acc) do
+    {acc, <<>>}
   end
-  defp escape(<<>>, _char, acc), do: acc
 
   @doc false
   # Also used by Regex
@@ -235,8 +203,8 @@ defimpl Inspect, for: BitString do
              to_hex(d), to_hex(e), to_hex(f), ?}]
   end
 
-  defp to_hex(c) when c in 0..9, do: ?0+c
-  defp to_hex(c) when c in 10..15, do: ?A+c-10
+  defp to_hex(c) when c in 0..9, do: ?0 + c
+  defp to_hex(c) when c in 10..15, do: ?A + c - 10
 
   ## Bitstrings
 
@@ -273,6 +241,7 @@ defimpl Inspect, for: BitString do
     Inspect.Integer.inspect(h, opts) <> "::size(" <> Integer.to_string(size) <> ")"
   end
 
+  @compile {:inline, decrement: 1}
   defp decrement(:infinity), do: :infinity
   defp decrement(counter),   do: counter - 1
 end
@@ -282,14 +251,19 @@ defimpl Inspect, for: List do
     color("[]", :list, opts)
   end
 
-  # TODO: Deprecate :char_lists and :as_char_lists keys in v1.5
-  def inspect(term, %Inspect.Opts{charlists: lists, char_lists: lists_deprecated} = opts) do
+  # TODO: Remove :char_list and :as_char_lists handling in 2.0
+  def inspect(term, opts) do
+    %Inspect.Opts{charlists: lists, char_lists: lists_deprecated, printable_limit: printable_limit} = opts
     lists =
       if lists == :infer and lists_deprecated != :infer do
         case lists_deprecated do
           :as_char_lists ->
+            IO.warn "the :char_lists inspect option and its :as_char_lists " <>
+              "value are deprecated, use the :charlists option and its " <>
+              ":as_charlists value instead"
             :as_charlists
           _ ->
+            IO.warn "the :char_lists inspect option is deprecated, use :charlists instead"
             lists_deprecated
         end
       else
@@ -301,8 +275,13 @@ defimpl Inspect, for: List do
     close = color("]", :list, opts)
 
     cond do
-      lists == :as_charlists or (lists == :infer and printable?(term)) ->
-        IO.iodata_to_binary [?', Inspect.BitString.escape(IO.chardata_to_string(term), ?'), ?']
+      lists == :as_charlists or (lists == :infer and printable?(term, printable_limit)) ->
+        inspected =
+          case Inspect.BitString.escape(IO.chardata_to_string(term), ?', printable_limit) do
+            {escaped, ""} -> [?', escaped, ?']
+            {escaped, _} -> [?', escaped, ?', " ++ ..."]
+          end
+        IO.iodata_to_binary inspected
       keyword?(term) ->
         surround_many(open, term, close, opts, &keyword/2, sep)
       true ->
@@ -328,17 +307,25 @@ defimpl Inspect, for: List do
   def keyword?(_other), do: false
 
   @doc false
-  def printable?([char | rest]) when char in 32..126, do: printable?(rest)
-  def printable?([?\n | rest]), do: printable?(rest)
-  def printable?([?\r | rest]), do: printable?(rest)
-  def printable?([?\t | rest]), do: printable?(rest)
-  def printable?([?\v | rest]), do: printable?(rest)
-  def printable?([?\b | rest]), do: printable?(rest)
-  def printable?([?\f | rest]), do: printable?(rest)
-  def printable?([?\e | rest]), do: printable?(rest)
-  def printable?([?\a | rest]), do: printable?(rest)
-  def printable?([]), do: true
-  def printable?(_), do: false
+  def printable?(list), do: printable?(list, :infinity)
+
+  @doc false
+  def printable?(_, 0), do: true
+  def printable?([char | rest], counter) when char in 32..126, do: printable?(rest, decrement(counter))
+  def printable?([?\n | rest], counter), do: printable?(rest, decrement(counter))
+  def printable?([?\r | rest], counter), do: printable?(rest, decrement(counter))
+  def printable?([?\t | rest], counter), do: printable?(rest, decrement(counter))
+  def printable?([?\v | rest], counter), do: printable?(rest, decrement(counter))
+  def printable?([?\b | rest], counter), do: printable?(rest, decrement(counter))
+  def printable?([?\f | rest], counter), do: printable?(rest, decrement(counter))
+  def printable?([?\e | rest], counter), do: printable?(rest, decrement(counter))
+  def printable?([?\a | rest], counter), do: printable?(rest, decrement(counter))
+  def printable?([], _counter), do: true
+  def printable?(_, _counter), do: false
+
+  @compile {:inline, decrement: 1}
+  defp decrement(:infinity), do: :infinity
+  defp decrement(counter),   do: counter - 1
 
   ## Private
 
@@ -369,20 +356,21 @@ defimpl Inspect, for: Map do
     open = color("%" <> name <> "{", :map, opts)
     sep = color(",", :map, opts)
     close = color("}", :map, opts)
-    surround_many(open, map, close, opts, traverse_fun(map), sep)
+    surround_many(open, map, close, opts, traverse_fun(map, opts), sep)
   end
 
-  defp traverse_fun(list) do
+  defp traverse_fun(list, opts) do
     if Inspect.List.keyword?(list) do
       &Inspect.List.keyword/2
     else
-      &to_map/2
+      sep = color(" => ", :map, opts)
+      &to_map(&1, &2, sep)
     end
   end
 
-  defp to_map({key, value}, opts) do
+  defp to_map({key, value}, opts, sep) do
     concat(
-      concat(to_doc(key, opts), " => "),
+      concat(to_doc(key, opts), sep),
       to_doc(value, opts)
     )
   end
@@ -430,31 +418,28 @@ defimpl Inspect, for: Regex do
   defp escape(bin, term),
     do: escape(bin, [], term)
 
-  defp escape(<<?\\, term>> <> rest, buf, term),
-    do: escape(rest, [buf | [?\\, term]], term)
-
-  defp escape(<<term>> <> rest, buf, term),
+  defp escape(<<term, rest::binary>>, buf, term),
     do: escape(rest, [buf | [?\\, term]], term)
 
   # The list of characters is from 'String.printable?' implementation
   # minus characters treated specially by regex: \s, \d, \b, \e
 
-  defp escape(<<?\n>> <> rest, buf, term),
+  defp escape(<<?\n, rest::binary>>, buf, term),
     do: escape(rest, [buf | '\\n'], term)
 
-  defp escape(<<?\r>> <> rest, buf, term),
+  defp escape(<<?\r, rest::binary>>, buf, term),
     do: escape(rest, [buf | '\\r'], term)
 
-  defp escape(<<?\t>> <> rest, buf, term),
+  defp escape(<<?\t, rest::binary>>, buf, term),
     do: escape(rest, [buf | '\\t'], term)
 
-  defp escape(<<?\v>> <> rest, buf, term),
+  defp escape(<<?\v, rest::binary>>, buf, term),
     do: escape(rest, [buf | '\\v'], term)
 
-  defp escape(<<?\f>> <> rest, buf, term),
+  defp escape(<<?\f, rest::binary>>, buf, term),
     do: escape(rest, [buf | '\\f'], term)
 
-  defp escape(<<?\a>> <> rest, buf, term),
+  defp escape(<<?\a, rest::binary>>, buf, term),
     do: escape(rest, [buf | '\\a'], term)
 
   defp escape(<<char::utf8, rest::binary>>, buf, term)
@@ -474,9 +459,10 @@ defimpl Inspect, for: Function do
   def inspect(function, _opts) do
     fun_info = :erlang.fun_info(function)
     mod = fun_info[:module]
+    name = fun_info[:name]
 
     if fun_info[:type] == :external and fun_info[:env] == [] do
-      "&#{Inspect.Atom.inspect(mod)}.#{fun_info[:name]}/#{fun_info[:arity]}"
+      "&#{Inspect.Atom.inspect(mod)}.#{escape_name(name)}/#{fun_info[:arity]}"
     else
       case Atom.to_charlist(mod) do
         'elixir_compiler_' ++ _ ->
@@ -491,6 +477,41 @@ defimpl Inspect, for: Function do
     end
   end
 
+  def escape_name(atom) when is_atom(atom) do
+    string = Atom.to_string(atom)
+
+    case Macro.classify_identifier(atom) do
+      :callable ->
+        string
+      type when type in [:not_callable, :alias] ->
+        "\"" <> string <> "\""
+      :other ->
+        {escaped, _} = Inspect.BitString.escape(string, ?")
+        IO.iodata_to_binary [?", escaped, ?"]
+    end
+  end
+
+  # Example of this format: -NAME/ARITY-fun-COUNT-
+  def extract_anonymous_fun_parent(atom) when is_atom(atom) do
+    extract_anonymous_fun_parent(Atom.to_string(atom))
+  end
+
+  def extract_anonymous_fun_parent("-" <> rest) do
+    [trailing | reversed] =
+      rest
+      |> String.split("/")
+      |> Enum.reverse()
+
+    case String.split(trailing, "-") do
+      [arity, _inner, _count, ""] ->
+        {reversed |> Enum.reverse |> Enum.join("/") |> String.to_atom(), arity}
+      _other ->
+        :error
+    end
+  end
+
+  def extract_anonymous_fun_parent(other) when is_binary(other), do: :error
+
   defp default_inspect(mod, fun_info) do
     "#Function<#{uniq(fun_info)}/#{fun_info[:arity]} in " <>
       "#{Inspect.Atom.inspect(mod)}#{extract_name(fun_info[:name])}>"
@@ -501,10 +522,11 @@ defimpl Inspect, for: Function do
   end
 
   defp extract_name(name) do
-    name = Atom.to_string(name)
-    case :binary.split(name, "-", [:global]) do
-      ["", name | _] -> "." <> name
-      _ -> "." <> name
+    case extract_anonymous_fun_parent(name) do
+      {name, arity} ->
+        "." <> escape_name(name) <> "/" <> arity
+      :error ->
+        "." <> escape_name(name)
     end
   end
 

lib/elixir/lib/inspect/algebra.ex

@@ -24,8 +24,11 @@ defmodule Inspect.Opts do
       is printable, otherwise as list.
 
     * `:limit` - limits the number of items that are printed for tuples,
-      bitstrings, and lists, does not apply to strings nor charlists, defaults
-      to 50.
+      bitstrings, maps, lists and any other collection of items. It does not
+      apply to strings nor charlists and defaults to 50.
+
+    * `:printable_limit` - limits the number of bytes that are printed for strings
+      and char lists. Defaults to 4096.
 
     * `:pretty` - if set to `true` enables pretty printing, defaults to `false`.
 
@@ -33,9 +36,9 @@ defmodule Inspect.Opts do
       printing to IO devices. Set to 0 to force each item to be printed on its
       own line.
 
-    * `:base` - prints integers as `:binary`, `:octal`, `:decimal`, or `:hex`, defaults
-      to `:decimal`. When inspecting binaries any `:base` other than `:decimal`
-      implies `binaries: :as_binaries`.
+    * `:base` - prints integers as `:binary`, `:octal`, `:decimal`, or `:hex`,
+      defaults to `:decimal`. When inspecting binaries any `:base` other than
+      `:decimal` implies `binaries: :as_binaries`.
 
     * `:safe` - when `false`, failures while inspecting structs will be raised
       as errors instead of being wrapped in the `Inspect.Error` exception. This
@@ -49,12 +52,13 @@ defmodule Inspect.Opts do
       Colors can be any `t:IO.ANSI.ansidata/0` as accepted by `IO.ANSI.format/1`.
   """
 
-  # TODO: Deprecate char_lists key by v1.5
+  # TODO: Remove :char_lists key by 2.0
   defstruct structs: true,
             binaries: :infer,
             charlists: :infer,
             char_lists: :infer,
             limit: 50,
+            printable_limit: 4096,
             width: 80,
             base: :decimal,
             pretty: false,
@@ -63,13 +67,14 @@ defmodule Inspect.Opts do
 
   @type color_key :: atom
 
-  # TODO: Deprecate char_lists key and :as_char_lists value by v1.5
+  # TODO: Remove :char_lists key and :as_char_lists value by 2.0
   @type t :: %__MODULE__{
                structs: boolean,
                binaries: :infer | :as_binaries | :as_strings,
                charlists: :infer | :as_lists | :as_charlists,
                char_lists: :infer | :as_lists | :as_char_lists,
                limit: pos_integer | :infinity,
+               printable_limit: pos_integer | :infinity,
                width: pos_integer | :infinity,
                base: :decimal | :binary | :hex | :octal,
                pretty: boolean,
@@ -159,7 +164,7 @@ defmodule Inspect.Algebra do
   @tail_separator " |"
   @newline "\n"
   @nesting 1
-  @break " "
+  @space " "
 
   # Functional interface to "doc" records
 
@@ -172,7 +177,7 @@ defmodule Inspect.Algebra do
 
   @typep doc_nest :: {:doc_nest, t, non_neg_integer}
   defmacrop doc_nest(doc, indent) do
-    quote do: {:doc_nest, unquote(doc), unquote(indent) }
+    quote do: {:doc_nest, unquote(doc), unquote(indent)}
   end
 
   @typep doc_break :: {:doc_break, binary}
@@ -238,7 +243,7 @@ defmodule Inspect.Algebra do
             try do
               Process.put(:inspect_trap, true)
 
-              res = Inspect.Map.inspect(map, opts)
+              res = Inspect.Map.inspect(map, %{opts | syntax_colors: []})
               res = IO.iodata_to_binary(format(res, :infinity))
 
               exception = Inspect.Error.exception(
@@ -277,7 +282,7 @@ defmodule Inspect.Algebra do
   @spec empty() :: :doc_nil
   def empty, do: :doc_nil
 
-  @doc """
+  @doc ~S"""
   Concatenates two document entities returning a new document.
 
   ## Examples
@@ -292,7 +297,7 @@ defmodule Inspect.Algebra do
     doc_cons(doc1, doc2)
   end
 
-  @doc """
+  @doc ~S"""
   Concatenates a list of documents returning a new document.
 
   ## Examples
@@ -307,7 +312,7 @@ defmodule Inspect.Algebra do
     fold_doc(docs, &concat(&1, &2))
   end
 
-  @doc """
+  @doc ~S"""
   Colors a document if the `color_key` has a color in the options.
   """
   @spec color(t, Inspect.Opts.color_key, Inspect.Opts.t) :: doc_color
@@ -372,19 +377,19 @@ defmodule Inspect.Algebra do
   @spec break(binary) :: doc_break
   def break(string) when is_binary(string), do: doc_break(string)
 
-  @doc """
-  Returns a document entity representing the default break.
+  @doc ~S"""
+  Returns a document entity with the `" "` string as break.
 
-  Same as calling `break/1` with the default break.
+  See `break/1` for more information.
   """
   @spec break() :: doc_break
-  def break(), do: doc_break(@break)
+  def break(), do: doc_break(@space)
 
-  @doc """
-  Glues two documents together inserting the default break between them.
+  @doc ~S"""
+  Glues two documents together inserting `" "` as a break between them.
 
-  The break that is inserted between `left` and `right` is the one returned by
-  `break/0`.
+  This means the two documents will be separated by `" "` in case they
+  fit in the same line. Otherwise a line break is used.
 
   ## Examples
 
@@ -396,7 +401,7 @@ defmodule Inspect.Algebra do
   @spec glue(t, t) :: t
   def glue(doc1, doc2), do: concat(doc1, concat(break(), doc2))
 
-  @doc """
+  @doc ~S"""
   Glues two documents (`doc1` and `doc2`) together inserting the given
   break `break_string` between them.
 
@@ -416,6 +421,9 @@ defmodule Inspect.Algebra do
   @doc ~S"""
   Returns a group containing the specified document `doc`.
 
+  Documents in a group are attempted to be rendered together
+  to the best of the renderer ability.
+
   ## Examples
 
       iex> doc = Inspect.Algebra.group(
@@ -445,7 +453,7 @@ defmodule Inspect.Algebra do
     doc_group(doc)
   end
 
-  @doc """
+  @doc ~S"""
   Inserts a mandatory single space between two documents.
 
   ## Examples
@@ -471,7 +479,7 @@ defmodule Inspect.Algebra do
   @spec line(t, t) :: t
   def line(doc1, doc2), do: concat(doc1, concat(:doc_line, doc2))
 
-  @doc """
+  @doc ~S"""
   Folds a list of documents into a document using the given folder function.
 
   The list of documents is folded "from the right"; in that, this function is

lib/elixir/lib/integer.ex

@@ -232,74 +232,46 @@ defmodule Integer do
       ** (ArgumentError) invalid base 38
 
   """
-  @spec parse(binary, 2..36) :: {integer, binary} | :error | no_return
+  @spec parse(binary, 2..36) :: {integer, binary} | :error
   def parse(binary, base \\ 10)
 
-  def parse("", base) when base in 2..36,
-    do: :error
-
-  def parse(binary, base) when is_binary(binary) and base in 2..36 do
-    parse_in_base(binary, base)
+  def parse(_binary, base) when not base in 2..36 do
+    raise ArgumentError, "invalid base #{inspect base}"
   end
 
-  def parse(binary, base) when is_binary(binary) do
-    raise ArgumentError, "invalid base #{base}"
-  end
-
-  defp parse_in_base("-" <> bin, base) do
-    case do_parse(bin, base) do
-      {number, remainder} -> {-number, remainder}
-      :error -> :error
+  def parse(binary, base) do
+    case count_digits(binary, base) do
+      0 ->
+        :error
+      count ->
+        {digits, rem} = :erlang.split_binary(binary, count)
+        {:erlang.binary_to_integer(digits, base), rem}
     end
   end
 
-  defp parse_in_base("+" <> bin, base) do
-    do_parse(bin, base)
-  end
-
-  defp parse_in_base(binary, base) when is_binary(binary) do
-    do_parse(binary, base)
-  end
-
-  defp do_parse(<<char, rest::binary>>, base) do
-    if valid_digit_in_base?(char, base) do
-      do_parse(rest, base, parse_digit(char))
-    else
-      :error
+  defp count_digits(<<sign, rest::binary>>, base) when sign in '+-' do
+    case count_digits_nosign(rest, base, 1) do
+      1 -> 0
+      count -> count
     end
   end
 
-  defp do_parse(_, _) do
-    :error
+  defp count_digits(<<rest::binary>>, base) do
+    count_digits_nosign(rest, base, 0)
   end
 
-  defp do_parse(<<char, rest::binary>> = bin, base, acc) do
-    if valid_digit_in_base?(char, base) do
-      do_parse(rest, base, base * acc + parse_digit(char))
-    else
-      {acc, bin}
+  digits = [{?0..?9, -?0}, {?A..?Z, 10 - ?A}, {?a..?z, 10 - ?a}]
+
+  for {chars, diff} <- digits, char <- chars do
+    digit = char + diff
+
+    defp count_digits_nosign(<<unquote(char), rest::binary>>, base, count)
+        when base > unquote(digit) do
+      count_digits_nosign(rest, base, count + 1)
     end
   end
 
-  defp do_parse(bitstring, _, acc) do
-    {acc, bitstring}
-  end
-
-  defp parse_digit(char) do
-    cond do
-      char in ?0..?9 -> char - ?0
-      char in ?A..?Z -> char - ?A + 10
-      true           -> char - ?a + 10
-    end
-  end
-
-  defp valid_digit_in_base?(char, base) do
-    if base <= 10 do
-      char in ?0..(?0 + base - 1)
-    else
-      char in ?0..?9 or char in ?A..(?A + base - 11) or char in ?a..(?a + base - 11)
-    end
-  end
+  defp count_digits_nosign(<<_::binary>>, _, count), do: count
 
   @doc """
   Returns a binary which corresponds to the text representation
@@ -401,8 +373,54 @@ defmodule Integer do
     :erlang.integer_to_list(integer, base)
   end
 
-  # TODO: Deprecate by v1.5
+  @doc """
+  Returns the greatest common divisor of the two given integers.
+
+  The greatest common divisor (GCD) of `integer1` and `integer2` is the largest positive
+  integer that divides both `integer1` and `integer2` without leaving a remainder.
+
+  By convention, `gcd(0, 0)` returns `0`.
+
+  ## Examples
+
+      iex> Integer.gcd(2, 3)
+      1
+
+      iex> Integer.gcd(8, 12)
+      4
+
+      iex> Integer.gcd(8, -12)
+      4
+
+      iex> Integer.gcd(10, 0)
+      10
+
+      iex> Integer.gcd(7, 7)
+      7
+
+      iex> Integer.gcd(0, 0)
+      0
+
+  """
+  @spec gcd(0, 0) :: 0
+  @spec gcd(integer, integer) :: pos_integer
+  def gcd(integer1, integer2) when is_integer(integer1) and is_integer(integer2) do
+    gcd_positive(abs(integer1), abs(integer2))
+  end
+
+  defp gcd_positive(0, integer2), do: integer2
+  defp gcd_positive(integer1, 0), do: integer1
+  defp gcd_positive(integer1, integer2), do: gcd_positive(integer2, rem(integer1, integer2))
+
+  # TODO: Remove by 2.0
+  # (hard-deprecated in elixir_dispatch)
   @doc false
   @spec to_char_list(integer) :: charlist
   def to_char_list(integer), do: Integer.to_charlist(integer)
+
+  # TODO: Remove by 2.0
+  # (hard-deprecated in elixir_dispatch)
+  @doc false
+  @spec to_char_list(integer, 2..36) :: charlist
+  def to_char_list(integer, base), do: Integer.to_charlist(integer, base)
 end

lib/elixir/lib/io.ex

@@ -276,7 +276,7 @@ defmodule IO do
       after: [2, 4, 6]
 
   """
-  @spec inspect(item, Keyword.t) :: item when item: var
+  @spec inspect(item, keyword) :: item when item: var
   def inspect(item, opts \\ []) do
     inspect :stdio, item, opts
   end
@@ -286,7 +286,7 @@ defmodule IO do
 
   See `inspect/2` for a full list of options.
   """
-  @spec inspect(device, item, Keyword.t) :: item when item: var
+  @spec inspect(device, item, keyword) :: item when item: var
   def inspect(device, item, opts) when is_list(opts) do
     label    = if (label = opts[:label]), do: [to_chardata(label), ": "], else: []
     opts     = struct(Inspect.Opts, opts)

lib/elixir/lib/io/ansi.ex

@@ -25,9 +25,9 @@ defmodule IO.ANSI do
 
   import IO.ANSI.Sequence
 
-  @typep ansicode :: atom
-  @typep ansilist :: maybe_improper_list(char | ansicode | binary | ansilist, binary | ansicode | [])
-  @type  ansidata :: ansilist | ansicode | binary
+  @type ansicode :: atom
+  @type ansilist :: maybe_improper_list(char | ansicode | binary | ansilist, binary | ansicode | [])
+  @type ansidata :: ansilist | ansicode | binary
 
   @doc """
   Checks if ANSI coloring is supported and enabled on this machine.

lib/elixir/lib/io/ansi/docs.ex

@@ -104,7 +104,7 @@ defmodule IO.ANSI.Docs do
     process_fenced_code_block(rest, text, indent, options, _delimiter = "~~~")
   end
 
-  defp process(all=[line | rest], text, indent, options) do
+  defp process(all = [line | rest], text, indent, options) do
     {stripped, count} = strip_spaces(line, 0, :infinity)
     cond do
       link_label?(stripped, count) ->
@@ -269,7 +269,11 @@ defmodule IO.ANSI.Docs do
 
     widths =
       for line <- lines do
-        for {_col, length} <- line, do: length
+        if table_header?(line) do
+          for _ <- line, do: 0
+        else
+          for {_col, length} <- line, do: length
+        end
       end
 
     col_widths = Enum.reduce(widths,
@@ -291,6 +295,7 @@ defmodule IO.ANSI.Docs do
     col =
       col
       |> String.replace("\\\|", "|")
+      |> String.trim()
       |> handle_links
       |> handle_inline(options)
     {col, length_without_escape(col, 0)}
@@ -306,6 +311,8 @@ defmodule IO.ANSI.Docs do
   defp render_table([first, second | rest], widths, options) do
     combined = Enum.zip(first, widths)
     if table_header?(second) do
+      alignments = Enum.map(second, &column_alignment/1)
+      options = Keyword.put_new(options, :alignments, alignments)
       draw_table_row(combined, options, :heading)
       render_table(rest, widths, options)
     else
@@ -323,6 +330,14 @@ defmodule IO.ANSI.Docs do
   defp render_table([], _, _),
     do: nil
 
+  defp column_alignment({line, _}) do
+    cond do
+      String.starts_with?(line, ":") and String.ends_with?(line, ":") -> :center
+      String.ends_with?(line, ":") -> :right
+      true -> :left
+    end
+  end
+
   defp table_header?(row) do
     Enum.all?(row, fn {col, _} -> table_header_column?(col) end)
   end
@@ -336,10 +351,13 @@ defmodule IO.ANSI.Docs do
   defp table_header_contents?(_), do: false
 
   defp draw_table_row(cols_and_widths, options, heading \\ false) do
+    default_alignments = List.duplicate(:left, length(cols_and_widths))
+    alignments = Keyword.get(options, :alignments, default_alignments)
+
     columns =
-      Enum.map_join(cols_and_widths, " | ", fn {{col, length}, width} ->
-        col <> String.duplicate(" ", width - length)
-      end)
+      cols_and_widths
+      |> Enum.zip(alignments)
+      |> Enum.map_join(" | ", &generate_table_cell/1)
 
     if heading do
       write(:doc_table_heading, columns, options)
@@ -348,6 +366,20 @@ defmodule IO.ANSI.Docs do
     end
   end
 
+  defp generate_table_cell({{{col, length}, width}, :center}) do
+    pad = if rem(length, 2) == 0, do: 1, else: rem(width, 2)
+    spaces = div(width, 2) - div(length, 2)
+    String.duplicate(" ", spaces) <> col <> String.duplicate(" ", spaces + pad)
+  end
+
+  defp generate_table_cell({{{col, length}, width}, :right}) do
+    String.duplicate(" ", width - length) <> col
+  end
+
+  defp generate_table_cell({{{col, length}, width}, _}) do
+    col <> String.duplicate(" ", width - length)
+  end
+
   defp table_line?(line) do
     line =~ " | "
   end

lib/elixir/lib/kernel.ex

@@ -20,8 +20,9 @@ defmodule Kernel do
   cannot be skipped. These are described in `Kernel.SpecialForms`.
 
   Some of the functions described in this module are inlined by
-  the Elixir compiler into their Erlang counterparts in the `:erlang`
-  module. Those functions are called BIFs (built-in internal functions)
+  the Elixir compiler into their Erlang counterparts in the
+  [`:erlang` module](http://www.erlang.org/doc/man/erlang.html).
+  Those functions are called BIFs (built-in internal functions)
   in Erlang-land and they exhibit interesting properties, as some of
   them are allowed in guards and others are used for compiler
   optimizations.
@@ -110,7 +111,7 @@ defmodule Kernel do
       "llo"
 
   """
-  @spec binary_part(binary, pos_integer, integer) :: binary
+  @spec binary_part(binary, non_neg_integer, integer) :: binary
   def binary_part(binary, start, length) do
     :erlang.binary_part(binary, start, length)
   end
@@ -260,6 +261,8 @@ defmodule Kernel do
   @doc """
   Returns the head of a list. Raises `ArgumentError` if the list is empty.
 
+  It works with improper lists.
+
   Allowed in guard tests. Inlined by the compiler.
 
   ## Examples
@@ -270,6 +273,9 @@ defmodule Kernel do
       hd([])
       #=> ** (ArgumentError) argument error
 
+      hd([1 | 2])
+      #=> 1
+
   """
   @spec hd(nonempty_maybe_improper_list(elem, any)) :: elem when elem: term
   def hd(list) do
@@ -336,7 +342,7 @@ defmodule Kernel do
   end
 
   @doc """
-  Returns `true` if `term` is a floating point number; otherwise returns `false`.
+  Returns `true` if `term` is a floating-point number; otherwise returns `false`.
 
   Allowed in guard tests. Inlined by the compiler.
   """
@@ -395,7 +401,7 @@ defmodule Kernel do
   end
 
   @doc """
-  Returns `true` if `term` is either an integer or a floating point number;
+  Returns `true` if `term` is either an integer or a floating-point number;
   otherwise returns `false`.
 
   Allowed in guard tests. Inlined by the compiler.
@@ -511,8 +517,9 @@ defmodule Kernel do
 
   @doc """
   Returns the biggest of the two given terms according to
-  Erlang's term ordering. If the terms compare equal, the
-  first one is returned.
+  Erlang's term ordering.
+
+  If the terms compare equal, the first one is returned.
 
   Inlined by the compiler.
 
@@ -531,8 +538,9 @@ defmodule Kernel do
 
   @doc """
   Returns the smallest of the two given terms according to
-  Erlang's term ordering. If the terms compare equal, the
-  first one is returned.
+  Erlang's term ordering.
+
+  If the terms compare equal, the first one is returned.
 
   Inlined by the compiler.
 
@@ -637,9 +645,9 @@ defmodule Kernel do
       :hello
 
   """
-  @spec send(dest :: pid | port | atom | {atom, node}, msg) :: msg when msg: any
-  def send(dest, msg) do
-    :erlang.send(dest, msg)
+  @spec send(dest :: pid | port | atom | {atom, node}, message) :: message when message: any
+  def send(dest, message) do
+    :erlang.send(dest, message)
   end
 
   @doc """
@@ -655,8 +663,12 @@ defmodule Kernel do
   @doc """
   Spawns the given function and returns its PID.
 
-  Check the `Process` and `Node` modules for other functions
-  to handle processes, including spawning functions in nodes.
+  Typically developers do not use the `spawn` functions, instead they use
+  abstractions such as `Task`, `GenServer` and `Agent`, built on top of
+  `spawn`, that spawns processes with more conveniences in terms of
+  introspection and debugging.
+
+  Check the `Process` module for more process-related functions.
 
   The anonymous function receives 0 arguments, and may return any value.
 
@@ -665,7 +677,7 @@ defmodule Kernel do
   ## Examples
 
       current = self()
-      child   = spawn(fn -> send current, {self(), 1 + 2} end)
+      child = spawn(fn -> send current, {self(), 1 + 2} end)
 
       receive do
         {^child, 3} -> IO.puts "Received 3 back"
@@ -678,11 +690,15 @@ defmodule Kernel do
   end
 
   @doc """
-  Spawns the given module and function passing the given args
-  and returns its PID.
+  Spawns the given function `fun` from the given `module` passing it the given
+  `args` and returns its PID.
 
-  Check the `Process` and `Node` modules for other functions
-  to handle processes, including spawning functions in nodes.
+  Typically developers do not use the `spawn` functions, instead they use
+  abstractions such as `Task`, `GenServer` and `Agent`, built on top of
+  `spawn`, that spawns processes with more conveniences in terms of
+  introspection and debugging.
+
+  Check the `Process` module for more process-related functions.
 
   Inlined by the compiler.
 
@@ -697,10 +713,15 @@ defmodule Kernel do
   end
 
   @doc """
-  Spawns the given function, links it to the current process and returns its PID.
+  Spawns the given function, links it to the current process, and returns its PID.
 
-  Check the `Process` and `Node` modules for other functions
-  to handle processes, including spawning functions in nodes.
+  Typically developers do not use the `spawn` functions, instead they use
+  abstractions such as `Task`, `GenServer` and `Agent`, built on top of
+  `spawn`, that spawns processes with more conveniences in terms of
+  introspection and debugging.
+
+  Check the `Process` module for more process-related functions. For more
+  information on linking, check `Process.link/1`.
 
   The anonymous function receives 0 arguments, and may return any value.
 
@@ -709,7 +730,7 @@ defmodule Kernel do
   ## Examples
 
       current = self()
-      child   = spawn_link(fn -> send current, {self(), 1 + 2} end)
+      child = spawn_link(fn -> send(current, {self(), 1 + 2}) end)
 
       receive do
         {^child, 3} -> IO.puts "Received 3 back"
@@ -722,11 +743,16 @@ defmodule Kernel do
   end
 
   @doc """
-  Spawns the given module and function passing the given args,
-  links it to the current process and returns its PID.
+  Spawns the given function `fun` from the given `module` passing it the given
+  `args`, links it to the current process, and returns its PID.
 
-  Check the `Process` and `Node` modules for other functions
-  to handle processes, including spawning functions in nodes.
+  Typically developers do not use the `spawn` functions, instead they use
+  abstractions such as `Task`, `GenServer` and `Agent`, built on top of
+  `spawn`, that spawns processes with more conveniences in terms of
+  introspection and debugging.
+
+  Check the `Process` module for more process-related functions. For more
+  information on linking, check `Process.link/1`.
 
   Inlined by the compiler.
 
@@ -744,8 +770,12 @@ defmodule Kernel do
   Spawns the given function, monitors it and returns its PID
   and monitoring reference.
 
-  Check the `Process` and `Node` modules for other functions
-  to handle processes, including spawning functions in nodes.
+  Typically developers do not use the `spawn` functions, instead they use
+  abstractions such as `Task`, `GenServer` and `Agent`, built on top of
+  `spawn`, that spawns processes with more conveniences in terms of
+  introspection and debugging.
+
+  Check the `Process` module for more process-related functions.
 
   The anonymous function receives 0 arguments, and may return any value.
 
@@ -766,8 +796,12 @@ defmodule Kernel do
   Spawns the given module and function passing the given args,
   monitors it and returns its PID and monitoring reference.
 
-  Check the `Process` and `Node` modules for other functions
-  to handle processes, including spawning functions in nodes.
+  Typically developers do not use the `spawn` functions, instead they use
+  abstractions such as `Task`, `GenServer` and `Agent`, built on top of
+  `spawn`, that spawns processes with more conveniences in terms of
+  introspection and debugging.
+
+  Check the `Process` module for more process-related functions.
 
   Inlined by the compiler.
 
@@ -796,6 +830,8 @@ defmodule Kernel do
   @doc """
   Returns the tail of a list. Raises `ArgumentError` if the list is empty.
 
+  It works with improper lists.
+
   Allowed in guard tests. Inlined by the compiler.
 
   ## Examples
@@ -806,6 +842,15 @@ defmodule Kernel do
       tl([])
       #=> ** (ArgumentError) argument error
 
+      tl([:one])
+      #=> []
+
+      tl([:a, :b | :c])
+      #=> [:b | :c]
+
+      tl([:a | %{b: 1}])
+      #=> %{b: 1}
+
   """
   @spec tl(nonempty_maybe_improper_list(elem, tail)) ::
         maybe_improper_list(elem, tail) | tail when elem: term, tail: term
@@ -1054,8 +1099,8 @@ defmodule Kernel do
   """
   @spec not(true) :: false
   @spec not(false) :: true
-  def not(arg) do
-    :erlang.not(arg)
+  def not(value) do
+    :erlang.not(value)
   end
 
   @doc """
@@ -1333,20 +1378,20 @@ defmodule Kernel do
       true
 
   """
-  defmacro !(arg)
+  defmacro !(value)
 
-  defmacro !({:!, _, [arg]}) do
+  defmacro !({:!, _, [value]}) do
     optimize_boolean(quote do
-      case unquote(arg) do
+      case unquote(value) do
         x when x in [false, nil] -> false
         _ -> true
       end
     end)
   end
 
-  defmacro !(arg) do
+  defmacro !(value) do
     optimize_boolean(quote do
-      case unquote(arg) do
+      case unquote(value) do
         x when x in [false, nil] -> true
         _ -> false
       end
@@ -1403,7 +1448,10 @@ defmodule Kernel do
   If `msg` is an atom, it just calls `raise/2` with the atom as the first
   argument and `[]` as the second argument.
 
-  If `msg` is anything else, raises an `ArgumentError` exception.
+  If `msg` is an exception struct, it is raised as is.
+
+  If `msg` is anything else, `raise` will fail with an `ArgumentError`
+  exception.
 
   ## Examples
 
@@ -1419,22 +1467,22 @@ defmodule Kernel do
       end
 
   """
-  defmacro raise(msg) do
+  defmacro raise(message) do
     # Try to figure out the type at compilation time
     # to avoid dead code and make Dialyzer happy.
-    msg = case not is_binary(msg) and bootstrapped?(Macro) do
-      true  -> Macro.expand(msg, __CALLER__)
-      false -> msg
+    message = case not is_binary(message) and bootstrapped?(Macro) do
+      true  -> Macro.expand(message, __CALLER__)
+      false -> message
     end
 
-    case msg do
-      msg when is_binary(msg) ->
+    case message do
+      message when is_binary(message) ->
         quote do
-          :erlang.error RuntimeError.exception(unquote(msg))
+          :erlang.error RuntimeError.exception(unquote(message))
         end
-      {:<<>>, _, _} = msg ->
+      {:<<>>, _, _} = message ->
         quote do
-          :erlang.error RuntimeError.exception(unquote(msg))
+          :erlang.error RuntimeError.exception(unquote(message))
         end
       alias when is_atom(alias) ->
         quote do
@@ -1442,7 +1490,7 @@ defmodule Kernel do
         end
       _ ->
         quote do
-          :erlang.error Kernel.Utils.raise(unquote(msg))
+          :erlang.error Kernel.Utils.raise(unquote(message))
         end
     end
   end
@@ -1464,9 +1512,9 @@ defmodule Kernel do
       ** (ArgumentError) Sample
 
   """
-  defmacro raise(exception, attrs) do
+  defmacro raise(exception, attributes) do
     quote do
-      :erlang.error unquote(exception).exception(unquote(attrs))
+      :erlang.error unquote(exception).exception(unquote(attributes))
     end
   end
 
@@ -1494,35 +1542,35 @@ defmodule Kernel do
           end
       end
   """
-  defmacro reraise(msg, stacktrace) do
+  defmacro reraise(message, stacktrace) do
     # Try to figure out the type at compilation time
     # to avoid dead code and make Dialyzer happy.
 
-    case Macro.expand(msg, __CALLER__) do
-      msg when is_binary(msg) ->
+    case Macro.expand(message, __CALLER__) do
+      message when is_binary(message) ->
         quote do
-          :erlang.raise :error, RuntimeError.exception(unquote(msg)), unquote(stacktrace)
+          :erlang.raise :error, RuntimeError.exception(unquote(message)), unquote(stacktrace)
         end
-      {:<<>>, _, _} = msg ->
+      {:<<>>, _, _} = message ->
         quote do
-          :erlang.raise :error, RuntimeError.exception(unquote(msg)), unquote(stacktrace)
+          :erlang.raise :error, RuntimeError.exception(unquote(message)), unquote(stacktrace)
         end
       alias when is_atom(alias) ->
         quote do
           :erlang.raise :error, unquote(alias).exception([]), unquote(stacktrace)
         end
-      msg ->
+      message ->
         quote do
           stacktrace = unquote(stacktrace)
-          case unquote(msg) do
-            msg when is_binary(msg) ->
-              :erlang.raise :error, RuntimeError.exception(msg), stacktrace
+          case unquote(message) do
+            message when is_binary(message) ->
+              :erlang.raise :error, RuntimeError.exception(message), stacktrace
             atom when is_atom(atom) ->
               :erlang.raise :error, atom.exception([]), stacktrace
             %{__struct__: struct, __exception__: true} = other when is_atom(struct) ->
               :erlang.raise :error, other, stacktrace
             other ->
-              message = "reraise/2 expects an alias, string or exception as the first argument, got: #{inspect other}"
+              message = "reraise/2 expects a module name, string or exception as the first argument, got: #{inspect other}"
               :erlang.error ArgumentError.exception(message)
           end
         end
@@ -1544,10 +1592,11 @@ defmodule Kernel do
           stacktrace = System.stacktrace
           reraise WrapperError, [exception: exception], stacktrace
       end
+
   """
-  defmacro reraise(exception, attrs, stacktrace) do
+  defmacro reraise(exception, attributes, stacktrace) do
     quote do
-      :erlang.raise :error, unquote(exception).exception(unquote(attrs)), unquote(stacktrace)
+      :erlang.raise :error, unquote(exception).exception(unquote(attributes)), unquote(stacktrace)
     end
   end
 
@@ -1638,15 +1687,15 @@ defmodule Kernel do
       #=> #Function<...>
 
   """
-  @spec inspect(Inspect.t, Keyword.t) :: String.t
-  def inspect(arg, opts \\ []) when is_list(opts) do
-    opts  = struct(Inspect.Opts, opts)
+  @spec inspect(Inspect.t, keyword) :: String.t
+  def inspect(term, opts \\ []) when is_list(opts) do
+    opts = struct(Inspect.Opts, opts)
     limit = case opts.pretty do
       true  -> opts.width
       false -> :infinity
     end
     IO.iodata_to_binary(
-      Inspect.Algebra.format(Inspect.Algebra.to_doc(arg, opts), limit)
+      Inspect.Algebra.format(Inspect.Algebra.to_doc(term, opts), limit)
     )
   end
 
@@ -1691,10 +1740,10 @@ defmodule Kernel do
 
   """
   @spec struct(module | struct, Enum.t) :: struct
-  def struct(struct, kv \\ []) do
-    struct(struct, kv, fn({key, val}, acc) ->
-      case :maps.is_key(key, acc) and key != :__struct__ do
-        true  -> :maps.put(key, val, acc)
+  def struct(struct, fields \\ []) do
+    struct(struct, fields, fn {key, val}, acc ->
+      case Map.has_key?(acc, key) and key != :__struct__ do
+        true  -> Map.put(acc, key, val)
         false -> acc
       end
     end)
@@ -1720,17 +1769,17 @@ defmodule Kernel do
 
   """
   @spec struct!(module | struct, Enum.t) :: struct | no_return
-  def struct!(struct, kv \\ [])
+  def struct!(struct, fields \\ [])
 
-  def struct!(struct, kv) when is_atom(struct) do
-    struct.__struct__(kv)
+  def struct!(struct, fields) when is_atom(struct) do
+    struct.__struct__(fields)
   end
 
-  def struct!(struct, kv) when is_map(struct) do
-    struct(struct, kv, fn
+  def struct!(struct, fields) when is_map(struct) do
+    struct(struct, fields, fn
       {:__struct__, _}, acc -> acc
       {key, val}, acc ->
-        :maps.update(key, val, acc)
+        Map.replace!(acc, key, val)
     end)
   end
 
@@ -1738,16 +1787,16 @@ defmodule Kernel do
     struct.__struct__()
   end
 
-  defp struct(struct, kv, fun) when is_atom(struct) do
-    struct(struct.__struct__(), kv, fun)
+  defp struct(struct, fields, fun) when is_atom(struct) do
+    struct(struct.__struct__(), fields, fun)
   end
 
   defp struct(%{__struct__: _} = struct, [], _fun) do
     struct
   end
 
-  defp struct(%{__struct__: _} = struct, kv, fun) do
-    Enum.reduce(kv, struct, fun)
+  defp struct(%{__struct__: _} = struct, fields, fun) do
+    Enum.reduce(fields, struct, fun)
   end
 
   @doc """
@@ -1788,7 +1837,7 @@ defmodule Kernel do
       [27, 23]
 
   If the previous value before invoking the function is `nil`,
-  the function *will* receive nil as a value and must handle it
+  the function *will* receive `nil` as a value and must handle it
   accordingly.
   """
   @spec get_in(Access.t, nonempty_list(term)) :: term
@@ -1941,21 +1990,28 @@ defmodule Kernel do
   In case any entry returns `nil`, its key will be removed
   and the deletion will be considered a success.
   """
-  @spec pop_in(Access.t, nonempty_list(term)) :: {term, Access.t}
+  @spec pop_in(data, nonempty_list(Access.get_and_update_fun(term, data) | term)) ::
+        {term, data} when data: Access.container
   def pop_in(data, keys)
-  def pop_in(nil, [h | _]), do: Access.pop(nil, h)
-  def pop_in(data, keys), do: do_pop_in(data, keys)
 
-  defp do_pop_in(nil, [_ | _]),
+  def pop_in(nil, [key | _]) do
+    raise ArgumentError, "could not pop key #{inspect key} on a nil value"
+  end
+
+  def pop_in(data, keys) when is_list(keys) do
+    pop_in_data(data, keys)
+  end
+
+  defp pop_in_data(nil, [_ | _]),
     do: :pop
-  defp do_pop_in(data, [h]) when is_function(h),
-    do: h.(:get_and_update, data, fn _ -> :pop end)
-  defp do_pop_in(data, [h | t]) when is_function(h),
-    do: h.(:get_and_update, data, &do_pop_in(&1, t))
-  defp do_pop_in(data, [h]),
-    do: Access.pop(data, h)
-  defp do_pop_in(data, [h | t]),
-    do: Access.get_and_update(data, h, &do_pop_in(&1, t))
+  defp pop_in_data(data, [fun]) when is_function(fun),
+    do: fun.(:get_and_update, data, fn _ -> :pop end)
+  defp pop_in_data(data, [fun | tail]) when is_function(fun),
+    do: fun.(:get_and_update, data, &pop_in_data(&1, tail))
+  defp pop_in_data(data, [key]),
+    do: Access.pop(data, key)
+  defp pop_in_data(data, [key | tail]),
+    do: Access.get_and_update(data, key, &pop_in_data(&1, tail))
 
   @doc """
   Puts a value in a nested structure via the given `path`.
@@ -2239,12 +2295,12 @@ defmodule Kernel do
       "foo"
 
   """
-  defmacro to_string(arg) do
-    quote do: String.Chars.to_string(unquote(arg))
+  defmacro to_string(term) do
+    quote do: String.Chars.to_string(unquote(term))
   end
 
   @doc """
-  Converts the argument to a charlist according to the `List.Chars` protocol.
+  Converts the given term to a charlist according to the `List.Chars` protocol.
 
   ## Examples
 
@@ -2252,8 +2308,8 @@ defmodule Kernel do
       'foo'
 
   """
-  defmacro to_charlist(arg) do
-    quote do: List.Chars.to_charlist(unquote(arg))
+  defmacro to_charlist(term) do
+    quote do: List.Chars.to_charlist(unquote(term))
   end
 
   @doc """
@@ -2299,15 +2355,15 @@ defmodule Kernel do
 
   `match?/2` is very useful when filtering of finding a value in an enumerable:
 
-      list = [{:a, 1}, {:b, 2}, {:a, 3}]
-      Enum.filter list, &match?({:a, _}, &1)
-      #=> [{:a, 1}, {:a, 3}]
+      iex> list = [a: 1, b: 2, a: 3]
+      iex> Enum.filter(list, &match?({:a, _}, &1))
+      [a: 1, a: 3]
 
   Guard clauses can also be given to the match:
 
-      list = [{:a, 1}, {:b, 2}, {:a, 3}]
-      Enum.filter list, &match?({:a, x} when x < 2, &1)
-      #=> [{:a, 1}]
+      iex> list = [a: 1, b: 2, a: 3]
+      iex> Enum.filter(list, &match?({:a, x} when x < 2, &1))
+      [a: 1]
 
   However, variables assigned in the match will not be available
   outside of the function call (unlike regular pattern matching with the `=`
@@ -2334,10 +2390,10 @@ defmodule Kernel do
   Reads and writes attributes of the current module.
 
   The canonical example for attributes is annotating that a module
-  implements the OTP behaviour called `gen_server`:
+  implements an OTP behaviour, such as `GenServer`:
 
       defmodule MyServer do
-        @behaviour :gen_server
+        @behaviour GenServer
         # ... callbacks ...
       end
 
@@ -2379,7 +2435,7 @@ defmodule Kernel do
 
     cond do
       # Check for Module as it is compiled later than Kernel
-      not bootstrapped?(Module) ->
+      not bootstrapped?(Macro) ->
         nil
 
       not function? and __CALLER__.context == :match ->
@@ -2413,6 +2469,13 @@ defmodule Kernel do
         :elixir_errors.warn env.line, env.file,
                             "@behavior attribute is not supported, please use @behaviour instead"
 
+      # TODO: Remove :compile check once on 2.0 as we no longer
+      # need to warn on parse transforms in Module.put_attribute.
+      name == :compile ->
+        {stack, _} = :elixir_quote.escape(env_stacktrace(env), false)
+        quote do: Module.put_attribute(__MODULE__, unquote(name), unquote(arg),
+                                       unquote(stack), unquote(line))
+
       :lists.member(name, [:moduledoc, :typedoc, :doc]) ->
         {stack, _} = :elixir_quote.escape(env_stacktrace(env), false)
         arg = {env.line, arg}
@@ -2460,14 +2523,13 @@ defmodule Kernel do
     raise ArgumentError, "expected 0 or 1 argument for @#{name}, got: #{length(args)}"
   end
 
-  defp typespec(:type),               do: :deftype
-  defp typespec(:typep),              do: :deftypep
-  defp typespec(:opaque),             do: :defopaque
-  defp typespec(:spec),               do: :defspec
-  defp typespec(:callback),           do: :defcallback
-  defp typespec(:macrocallback),      do: :defmacrocallback
-  defp typespec(:optional_callbacks), do: :defoptional_callbacks
-  defp typespec(_),                   do: false
+  defp typespec(:type),          do: :deftype
+  defp typespec(:typep),         do: :deftypep
+  defp typespec(:opaque),        do: :defopaque
+  defp typespec(:spec),          do: :defspec
+  defp typespec(:callback),      do: :defcallback
+  defp typespec(:macrocallback), do: :defmacrocallback
+  defp typespec(_),              do: false
 
   @doc """
   Returns the binding for the given context as a keyword list.
@@ -2654,9 +2716,12 @@ defmodule Kernel do
   end
 
   @doc """
-  Returns a range with the specified start and end.
+  Returns a range with the specified `first` and `last` integers.
 
-  Both ends are included.
+  If last is larger than first, the range will be increasing from
+  first to last. If first is larger than last, the range will be
+  decreasing from first to last. If first is equal to last, the range
+  will contain one element, which is the number itself.
 
   ## Examples
 
@@ -2848,7 +2913,7 @@ defmodule Kernel do
   defmacro left |> right do
     [{h, _} | t] = Macro.unpipe({:|>, [], [left, right]})
     :lists.foldl fn {x, pos}, acc ->
-      # TODO: raise an error in `Macro.pipe/3` by 1.5
+      # TODO: raise an error in `Macro.pipe/3` by 2.0
       case Macro.pipe_warning(x) do
         nil -> :ok
         message ->
@@ -2920,11 +2985,18 @@ defmodule Kernel do
 
       Enum.member?([1, 2, 3], x)
 
+  Elixir also supports `left not in right`, which evaluates to
+  `not(left in right)`:
+
+      iex> x = 1
+      iex> x not in [1, 2, 3]
+      false
+
   ## Guards
 
-  The `in/2` operator can be used in guard clauses as long as the
-  right-hand side is a range or a list. In such cases, Elixir will expand the
-  operator to a valid guard expression. For example:
+  The `in/2` operator (as well as `not in`) can be used in guard clauses as
+  long as the right-hand side is a range or a list. In such cases, Elixir will
+  expand the operator to a valid guard expression. For example:
 
       when x in [1, 2, 3]
 
@@ -2938,8 +3010,20 @@ defmodule Kernel do
 
   translates to:
 
-      when x >= 1 and x <= 3
+      when is_integer(x) and x >= 1 and x <= 3
 
+  Note that only integers can be considered inside a range by `in`.
+
+  ### AST considerations
+
+  `left not in right` is parsed by the compiler into the AST:
+
+      {:not, _, [{:in, _, [left, right]}]}
+
+  This is the same AST as `not(left in right)`.
+
+  Additionally, `Macro.to_string/2` will translate all occurrences of
+  this AST to `left not in right`.
   """
   defmacro left in right do
     in_module? = (__CALLER__.context == nil)
@@ -3205,8 +3289,17 @@ defmodule Kernel do
       result
     end
 
-    {escaped, _} = :elixir_quote.escape(block, false)
-    module_vars  = module_vars(env.vars, 0)
+    escaped =
+      case env do
+        %{function: nil, lexical_tracker: pid} when is_pid(pid) ->
+          integer = Kernel.LexicalTracker.write_cache(pid, block)
+          quote do: Kernel.LexicalTracker.read_cache(unquote(pid), unquote(integer))
+        %{} ->
+          {escaped, _} = :elixir_quote.escape(block, false)
+          escaped
+      end
+
+    module_vars = module_vars(env.vars, 0)
 
     quote do
       unquote(with_alias)
@@ -3248,7 +3341,7 @@ defmodule Kernel do
 
     under = String.to_atom(<<"_@", :erlang.integer_to_binary(counter)::binary>>)
     args  = [key, kind, under, var]
-    [{:{}, [], args} | module_vars(vars, counter+1)]
+    [{:{}, [], args} | module_vars(vars, counter + 1)]
   end
 
   defp module_vars([], _counter) do
@@ -3295,7 +3388,7 @@ defmodule Kernel do
     end
   end
 
-  @doc """
+  @doc ~S"""
   Defines a function with the given name and body.
 
   ## Examples
@@ -3317,6 +3410,43 @@ defmodule Kernel do
   In the example above, a `sum/2` function is defined; this function receives
   two arguments and returns their sum.
 
+  ## Default arguments
+
+  `\\` is used to specify a default value for a parameter of a function. For
+  example:
+
+      defmodule MyMath do
+        def multiply_by(number, factor \\ 2) do
+          number * factor
+        end
+      end
+
+      MyMath.multiply_by(4, 3) #=> 12
+      MyMath.multiply_by(4) #=> 8
+
+  The compiler translates this into multiple functions with different arities,
+  here `Foo.multiply_by/1` and `Foo.multiply_by/2`, that represent cases when
+  arguments for parameters with default values are passed or not passed.
+
+  When defining a function with default arguments as well as multiple
+  explicitly declared clauses, you must write a function head that declares the
+  defaults. For example:
+
+      defmodule MyString do
+        def join(string1, string2 \\ nil, separator \\ " ")
+
+        def join(string1, nil, _separator) do
+          string1
+        end
+
+        def join(string1, string2, separator) do
+          string1 <> separator <> string2
+        end
+      end
+
+  Note that `\\` can't be used with anonymous functions because they
+  can only have a single arity.
+
   ## Function and variable names
 
   Function and variable names have the following syntax:
@@ -3396,6 +3526,8 @@ defmodule Kernel do
   @doc """
   Defines a macro with the given name and body.
 
+  Check `def/2` for rules on naming and default arguments.
+
   ## Examples
 
       defmodule MyLogic do
@@ -3422,7 +3554,8 @@ defmodule Kernel do
   Private macros are only accessible from the same module in which they are
   defined.
 
-  Check `defmacro/2` for more information.
+  Check `defmacro/2` for more information, and check `def/2` for rules on
+  naming and default arguments.
 
   """
   defmacro defmacrop(call, expr \\ nil) do
@@ -3430,20 +3563,29 @@ defmodule Kernel do
   end
 
   defp define(kind, call, expr, env) do
-    assert_module_scope(env, kind, 2)
+    module = assert_module_scope(env, kind, 2)
     assert_no_function_scope(env, kind, 2)
-    line = env.line
 
-    {call, unquoted_call} = :elixir_quote.escape(call, true)
-    {expr, unquoted_expr} = :elixir_quote.escape(expr, true)
+    {escaped_call, unquoted_call} = :elixir_quote.escape(call, true)
+    {escaped_expr, unquoted_expr} = :elixir_quote.escape(expr, true)
+
+    escaped_expr =
+      case unquoted_expr do
+        true ->
+          escaped_expr
+        false ->
+          key = :erlang.unique_integer()
+          :elixir_module.write_cache(module, key, expr)
+          quote do: :elixir_module.read_cache(unquote(module), unquote(key))
+      end
 
     # Do not check clauses if any expression was unquoted
     check_clauses = not(unquoted_expr or unquoted_call)
     pos = :elixir_locals.cache_env(env)
 
     quote do
-      :elixir_def.store_definition(unquote(line), unquote(kind), unquote(check_clauses),
-                                   unquote(call), unquote(expr), unquote(pos))
+      :elixir_def.store_definition(unquote(kind), unquote(check_clauses),
+                                   unquote(escaped_call), unquote(escaped_expr), unquote(pos))
     end
   end
 
@@ -3568,7 +3710,7 @@ defmodule Kernel do
             def __struct__(kv) do
               {map, keys} =
                 Enum.reduce(kv, {__struct__(), @enforce_keys}, fn {key, val}, {map, keys} ->
-                  {:maps.update(key, val, map), :lists.delete(key, keys)}
+                  {Map.replace!(map, key, val), List.delete(keys, key)}
                 end)
               case keys do
                 [] -> map
@@ -3582,7 +3724,7 @@ defmodule Kernel do
             _ = @enforce_keys
             def __struct__(kv) do
               :lists.foldl(fn {key, val}, acc ->
-                :maps.update(key, val, acc)
+                Map.replace!(acc, key, val)
               end, __struct__(), kv)
             end
           end
@@ -3683,7 +3825,7 @@ defmodule Kernel do
         end
       end
 
-      @spec exception(Keyword.t) :: Exception.t
+      @spec exception(keyword) :: Exception.t
       # TODO: Only call Kernel.struct! by Elixir v1.5
       def exception(args) when is_list(args) do
         struct = __struct__()
@@ -3739,7 +3881,7 @@ defmodule Kernel do
   Now that the protocol can be implemented for every data structure
   the protocol may have a compliant implementation for:
 
-      defimpl Size, for: Binary do
+      defimpl Size, for: BitString do
         def size(binary), do: byte_size(binary)
       end
 
@@ -3846,9 +3988,10 @@ defmodule Kernel do
 
   Any protocol module contains three extra functions:
 
-    * `__protocol__/1` - returns the protocol name when `:name` is given, and a
+    * `__protocol__/1` - returns the protocol name when `:name` is given, a
       keyword list with the protocol functions and their arities when
-      `:functions` is given
+      `:functions` is given, and a list of the implementations when `:impls` is
+      given
 
     * `impl_for/1` - receives a structure and returns the module that
       implements the protocol for the structure, `nil` otherwise
@@ -3933,10 +4076,42 @@ defmodule Kernel do
   As seen as in the example above, `super` can be used to call the default
   implementation.
 
+  If `@behaviour` has been defined, `defoverridable` can also be called with a
+  module as an argument. All implemented callbacks from the behaviour above the
+  call to `defoverridable` will be marked as overridable.
+
+  ## Example
+
+      defmodule Behaviour do
+        @callback foo :: any
+      end
+
+      defmodule DefaultMod do
+        defmacro __using__(_opts) do
+          quote do
+            @behaviour Behaviour
+
+            def foo do
+              "Override me"
+            end
+
+            defoverridable Behaviour
+          end
+        end
+      end
+
+      defmodule InheritMod do
+        use DefaultMod
+
+        def foo do
+          "Overriden"
+        end
+      end
+
   """
-  defmacro defoverridable(keywords) do
+  defmacro defoverridable(keywords_or_behaviour) do
     quote do
-      Module.make_overridable(__MODULE__, unquote(keywords))
+      Module.make_overridable(__MODULE__, unquote(keywords_or_behaviour))
     end
   end
 
@@ -3998,7 +4173,7 @@ defmodule Kernel do
   from different modules:
 
       defmodule MyModule do
-        defmacro __using__(opts) do
+        defmacro __using__(_opts) do
           quote do
             import MyModule.Foo
             import MyModule.Bar
@@ -4028,7 +4203,7 @@ defmodule Kernel do
   the `__using__/1` callback, unless those functions are the default
   implementation of a previously defined `@callback` or are functions
   meant to be overridden (see `defoverridable/1`). Even in these cases,
-  defining functions should be seen as a "last resource".
+  defining functions should be seen as a "last resort".
 
   In case you want to provide some existing functionality to the user module,
   please define it in a module which will be imported accordingly; for example,
@@ -4074,6 +4249,8 @@ defmodule Kernel do
 
   Delegation only works with functions; delegating macros is not supported.
 
+  Check `def/2` for rules on naming and default arguments.
+
   ## Options
 
     * `:to` - the module to dispatch to.
@@ -4116,14 +4293,16 @@ defmodule Kernel do
           "Kernel.defdelegate/2 :append_first option is deprecated")
       end
 
-      for fun <- List.wrap(funs),
-        {name, args, as, as_args} <- Kernel.Utils.defdelegate(fun, opts) do
-          unless Module.get_attribute(__MODULE__, :doc) do
-            @doc "See `#{inspect target}.#{as}/#{:erlang.length args}`."
-          end
-          def unquote(name)(unquote_splicing(args)) do
-            unquote(target).unquote(as)(unquote_splicing(as_args))
-          end
+      for fun <- List.wrap(funs) do
+        {name, args, as, as_args} = Kernel.Utils.defdelegate(fun, opts)
+
+        unless Module.get_attribute(__MODULE__, :doc) do
+          @doc "See `#{inspect target}.#{as}/#{:erlang.length args}`."
+        end
+
+        def unquote(name)(unquote_splicing(args)) do
+          unquote(target).unquote(as)(unquote_splicing(as_args))
+        end
       end
     end
   end
@@ -4434,7 +4613,7 @@ defmodule Kernel do
   defp assert_module_scope(env, fun, arity) do
     case env.module do
       nil -> raise ArgumentError, "cannot invoke #{fun}/#{arity} outside module"
-      _   -> :ok
+      mod -> mod
     end
   end
 
@@ -4452,8 +4631,9 @@ defmodule Kernel do
     end
   end
 
-  # TODO: Deprecate by v1.5
   @doc false
+  # TODO: Remove by 2.0
+  # (hard-deprecated in elixir_dispatch)
   defmacro to_char_list(arg) do
     quote do: Kernel.to_charlist(unquote(arg))
   end

lib/elixir/lib/kernel/cli.ex

@@ -59,6 +59,20 @@ defmodule Kernel.CLI do
     Enum.reverse(config.errors, errors)
   end
 
+  @doc false
+  def format_error(kind, reason, stacktrace) do
+    {blamed, stacktrace} = Exception.blame(kind, reason, stacktrace)
+    iodata =
+      case blamed do
+        %FunctionClauseError{} ->
+          [Exception.format_banner(kind, reason, stacktrace),
+           pad(FunctionClauseError.blame(blamed, &inspect/1, &blame_match/2))]
+        _ ->
+          Exception.format_banner(kind, blamed, stacktrace)
+      end
+    [iodata, ?\n, Exception.format_stacktrace(prune_stacktrace(stacktrace))]
+  end
+
   ## Helpers
 
   defp at_exit(res) do
@@ -119,13 +133,36 @@ defmodule Kernel.CLI do
     end
   end
 
-  defp print_error(kind, reason, trace) do
-    IO.puts :stderr, Exception.format(kind, reason, prune_stacktrace(trace))
+  ## Error handling
+
+  defp print_error(kind, reason, stacktrace) do
+    IO.write :stderr, format_error(kind, reason, stacktrace)
   end
 
-  @elixir_internals [:elixir, :elixir_exp, :elixir_compiler, :elixir_module, :elixir_clauses,
-                     :elixir_translator, :elixir_expand, :elixir_lexical, :elixir_exp_clauses,
-                     :elixir_def, :elixir_map]
+  defp blame_match(%{match?: true, node: node}, _),
+    do: blame_ansi(:normal, "+", node)
+  defp blame_match(%{match?: false, node: node}, _),
+    do: blame_ansi(:red, "-", node)
+  defp blame_match(_, string),
+    do: string
+
+  defp blame_ansi(color, no_ansi, node) do
+    if IO.ANSI.enabled? do
+      [color | Macro.to_string(node)]
+      |> IO.ANSI.format(true)
+      |> IO.iodata_to_binary()
+    else
+      no_ansi <> Macro.to_string(node) <> no_ansi
+    end
+  end
+
+  defp pad(string) do
+    "    " <> String.replace(string, "\n", "\n    ")
+  end
+
+  @elixir_internals [:elixir, :elixir_expand, :elixir_compiler, :elixir_module,
+                     :elixir_clauses, :elixir_lexical, :elixir_def, :elixir_map,
+                     :elixir_erl, :elixir_erl_clauses, :elixir_erl_pass, Kernel.ErrorHandler]
 
   defp prune_stacktrace([{mod, _, _, _} | t]) when mod in @elixir_internals do
     prune_stacktrace(t)
@@ -411,25 +448,17 @@ defmodule Kernel.CLI do
   end
 
   defp filter_multiple_patterns(patterns) do
-    matched_files = Enum.map patterns, fn(pattern) ->
-      case filter_patterns(pattern) do
-        []    -> {:missing, pattern}
-        files -> {:ok, files}
+    {files, missing} =
+      Enum.reduce patterns, {[], []}, fn pattern, {files, missing} ->
+        case filter_patterns(pattern) do
+          [] -> {files, [pattern | missing]}
+          match -> {match ++ files, missing}
+        end
       end
-    end
 
-    files = Enum.filter_map matched_files,
-       fn(match) -> elem(match, 0) == :ok end,
-       &elem(&1, 1)
-
-    missing_patterns = Enum.filter_map matched_files,
-       fn(match) -> elem(match, 0) == :missing end,
-       &elem(&1, 1)
-
-    if missing_patterns == [] do
-      {:ok, :lists.usort(Enum.concat(files))}
-    else
-      {:missing, :lists.usort(missing_patterns)}
+    case missing do
+      [] -> {:ok, :lists.usort(files)}
+      _  -> {:missing, :lists.usort(missing)}
     end
   end
 

lib/elixir/lib/kernel/lexical_tracker.ex

@@ -81,6 +81,18 @@ defmodule Kernel.LexicalTracker do
     :gen_server.cast(pid, {:alias_dispatch, module})
   end
 
+  @doc false
+  def write_cache(pid, value) do
+    key = :erlang.unique_integer()
+    :gen_server.cast(pid, {:write_cache, key, value})
+    key
+  end
+
+  @doc false
+  def read_cache(pid, key) do
+    :gen_server.call(pid, {:read_cache, key}, @timeout)
+  end
+
   @doc false
   def collect_unused_imports(pid) do
     unused(pid, :import)
@@ -99,7 +111,7 @@ defmodule Kernel.LexicalTracker do
 
   def init(dest) do
     {:ok, %{directives: %{}, references: %{}, compile: %{},
-            runtime: %{}, dest: dest}}
+            runtime: %{}, dest: dest, cache: %{}}}
   end
 
   @doc false
@@ -124,6 +136,14 @@ defmodule Kernel.LexicalTracker do
     {:reply, state.dest, state}
   end
 
+  def handle_call({:read_cache, key}, _from, %{cache: cache} = state) do
+    {:reply, Map.fetch!(cache, key), state}
+  end
+
+  def handle_cast({:write_cache, key, value}, %{cache: cache} = state) do
+    {:noreply, Map.put(state, :cache, Map.put(cache, key, value))}
+  end
+
   def handle_cast({:remote_reference, module, mode}, state) do
     {:noreply, %{state | references: add_reference(state.references, module, mode)}}
   end

lib/elixir/lib/kernel/parallel_compiler.ex

@@ -30,6 +30,9 @@ defmodule Kernel.ParallelCompiler do
     * `:each_module` - for each module compiled, invokes the callback passing
       the file, module and the module bytecode
 
+    * `:each_warning` - for each warning, invokes the callback passing
+      the file, line number, and warning message
+
     * `:dest` - the destination directory for the BEAM files. When using `files/2`,
       this information is only used to properly annotate the BEAM files before
       they are loaded into memory. If you want a file to actually be written to
@@ -110,17 +113,21 @@ defmodule Kernel.ParallelCompiler do
         :erlang.put(:elixir_compiler_file, file)
         :erlang.process_flag(:error_handler, Kernel.ErrorHandler)
 
-        exit(try do
-          _ = if output do
-            :elixir_compiler.file_to_path(file, output)
-          else
-            :elixir_compiler.file(file, Keyword.get(options, :dest))
+        result =
+          try do
+            _ = if output do
+              :elixir_compiler.file_to_path(file, output)
+            else
+              :elixir_compiler.file(file, Keyword.get(options, :dest))
+            end
+            :ok
+          catch
+            kind, reason ->
+              {kind, reason, System.stacktrace}
           end
-          {:shutdown, file}
-        catch
-          kind, reason ->
-            {:failure, kind, reason, System.stacktrace}
-        end)
+
+        send(parent, {:file_compiled, self(), file, result})
+        exit(:shutdown)
       end
 
     timeout = Keyword.get(options, :long_compilation_threshold, 10) * 1_000
@@ -151,15 +158,12 @@ defmodule Kernel.ParallelCompiler do
     # are waiting on the same module required by the faulty code. However,
     # since we need to pick something to be first, the one with fewer edges
     # sounds like a sane choice.
-    entries =
-      entries
-      |> Enum.group_by(&elem(&1, 0), &elem(&1, 1))
-      |> Enum.sort_by(&length(elem(&1, 1)))
-      |> Enum.find_value([], &elem(&1, 1))
-
-    case entries do
+    entries
+    |> Enum.group_by(&elem(&1, 0), &elem(&1, 1))
+    |> Enum.sort_by(&length(elem(&1, 1)))
+    |> case do
+      [{_on, refs} | _] -> spawn_compilers(%{state | entries: refs})
       [] -> handle_deadlock(waiting, queued)
-      _  -> spawn_compilers(%{state | entries: entries})
     end
   end
 
@@ -229,26 +233,58 @@ defmodule Kernel.ParallelCompiler do
         end
         spawn_compilers(state)
 
-      {:DOWN, _down_ref, :process, down_pid, {:shutdown, file}} ->
+      {:warning, file, line, message} ->
+        if callback = Keyword.get(options, :each_warning) do
+          callback.(file, line, message)
+        end
+        wait_for_messages(state)
+
+      {:file_compiled, child_pid, file, :ok} ->
+        discard_down(child_pid)
+
         if callback = Keyword.get(options, :each_file) do
           callback.(file)
         end
 
-        cancel_waiting_timer(queued, down_pid)
+        cancel_waiting_timer(queued, child_pid)
 
         # Sometimes we may have spurious entries in the waiting
         # list because someone invoked try/rescue UndefinedFunctionError
-        new_entries = List.delete(entries, down_pid)
-        new_queued  = List.keydelete(queued, down_pid, 0)
-        new_waiting = List.keydelete(waiting, down_pid, 1)
+        new_entries = List.delete(entries, child_pid)
+        new_queued  = List.keydelete(queued, child_pid, 0)
+        new_waiting = List.keydelete(waiting, child_pid, 1)
         spawn_compilers(%{state | entries: new_entries, waiting: new_waiting, queued: new_queued})
 
-      {:DOWN, down_ref, :process, _down_pid, reason} ->
-        handle_failure(down_ref, reason, queued)
+      {:file_compiled, child_pid, file, {kind, reason, stack}} ->
+        discard_down(child_pid)
+        print_error(file, kind, reason, stack)
+        terminate(queued)
+
+      {:DOWN, ref, :process, _pid, reason} ->
+        handle_down(queued, ref, reason)
         wait_for_messages(state)
     end
   end
 
+  defp discard_down(pid) do
+    receive do
+      {:DOWN, _, :process, ^pid, _} -> :ok
+    end
+  end
+
+  defp handle_down(_queued, _ref, :normal) do
+    :ok
+  end
+  defp handle_down(queued, ref, reason) do
+    case List.keyfind(queued, ref, 1) do
+      {_child, ^ref, file, _timer_ref} ->
+        print_error(file, :exit, reason, [])
+        terminate(queued)
+      _ ->
+        :ok
+    end
+  end
+
   defp handle_deadlock(waiting, queued) do
     deadlock =
       for {pid, _, file, _} <- queued do
@@ -258,7 +294,7 @@ defmodule Kernel.ParallelCompiler do
         {_kind, ^pid, _, on, _} = List.keyfind(waiting, pid, 1)
         error = CompileError.exception(description: "deadlocked waiting on module #{inspect on}",
                                        file: nil, line: nil)
-        print_failure(file, {:failure, :error, error, stacktrace})
+        print_error(file, :error, error, stacktrace)
 
         {file, on}
       end
@@ -282,51 +318,16 @@ defmodule Kernel.ParallelCompiler do
     exit({:shutdown, 1})
   end
 
-  defp handle_failure(ref, reason, queued) do
-    if file = find_failure(ref, queued) do
-      print_failure(file, reason)
-      for {pid, _, _, _} <- queued do
-        Process.exit(pid, :kill)
-      end
-      exit({:shutdown, 1})
+  defp terminate(queued) do
+    for {pid, _, _, _} <- queued do
+      Process.exit(pid, :kill)
     end
+    exit({:shutdown, 1})
   end
 
-  defp find_failure(ref, queued) do
-    case List.keyfind(queued, ref, 1) do
-      {_child, ^ref, file, _timer_ref} -> file
-      _ -> nil
-    end
-  end
-
-  defp print_failure(_file, {:shutdown, _}) do
-    :ok
-  end
-
-  defp print_failure(file, {:failure, kind, reason, stacktrace}) do
-    IO.puts "\n== Compilation error on file #{Path.relative_to_cwd(file)} =="
-    IO.puts Exception.format(kind, reason, prune_stacktrace(stacktrace))
-  end
-
-  defp print_failure(file, reason) do
-    IO.puts "\n== Compilation error on file #{Path.relative_to_cwd(file)} =="
-    IO.puts Exception.format(:exit, reason, [])
-  end
-
-  @elixir_internals [:elixir, :elixir_exp, :elixir_compiler, :elixir_module, :elixir_clauses,
-                     :elixir_translator, :elixir_expand, :elixir_lexical, :elixir_exp_clauses,
-                     :elixir_def, :elixir_map, Kernel.ErrorHandler]
-
-  defp prune_stacktrace([{mod, _, _, _} | t]) when mod in @elixir_internals do
-    prune_stacktrace(t)
-  end
-
-  defp prune_stacktrace([h | t]) do
-    [h | prune_stacktrace(t)]
-  end
-
-  defp prune_stacktrace([]) do
-    []
+  defp print_error(file, kind, reason, stack) do
+    IO.write ["\n== Compilation error in file #{Path.relative_to_cwd(file)} ==\n",
+             Kernel.CLI.format_error(kind, reason, stack)]
   end
 
   defp cancel_waiting_timer(queued, child_pid) do

lib/elixir/lib/kernel/parallel_require.ex

@@ -49,17 +49,22 @@ defmodule Kernel.ParallelRequire do
 
   defp spawn_requires([file | files], waiting, callbacks, schedulers, result) do
     parent = self()
+
     {pid, ref} = :erlang.spawn_monitor fn ->
       :erlang.put(:elixir_compiler_pid, parent)
       :erlang.put(:elixir_compiler_file, file)
 
-      exit(try do
-        new = Code.require_file(file) || []
-        {:required, Enum.map(new, &elem(&1, 0)), file}
-      catch
-        kind, reason ->
-          {:failure, kind, reason, System.stacktrace}
-      end)
+      result =
+        try do
+          new = Code.require_file(file) || []
+          {:required, Enum.map(new, &elem(&1, 0))}
+        catch
+          kind, reason ->
+            {kind, reason, System.stacktrace}
+        end
+
+      send(parent, {:file_required, self(), file, result})
+      exit(:shutdown)
     end
 
     spawn_requires(files, [{pid, ref} | waiting], callbacks, schedulers, result)
@@ -67,28 +72,21 @@ defmodule Kernel.ParallelRequire do
 
   defp wait_for_messages(files, waiting, callbacks, schedulers, result) do
     receive do
-      {:DOWN, ref, :process, pid, status} ->
-        tuple = {pid, ref}
-        if tuple in waiting do
-          waiting = List.delete(waiting, tuple)
-
-          case status do
-            {:required, mods, file} ->
-              if each_file_callback = callbacks[:each_file] do
-                each_file_callback.(file)
-              end
-
-              spawn_requires(files, waiting, callbacks, schedulers, mods ++ result)
-
-            {:failure, kind, reason, stacktrace} ->
-              :erlang.raise(kind, reason, stacktrace)
-
-            other ->
-              :erlang.raise(:exit, other, [])
-          end
-        else
-          spawn_requires(files, waiting, callbacks, schedulers, result)
+      {:file_required, pid, file, {:required, mods}} ->
+        discard_down(pid)
+        if each_file_callback = callbacks[:each_file] do
+          each_file_callback.(file)
         end
+        waiting = List.keydelete(waiting, pid, 0)
+        spawn_requires(files, waiting, callbacks, schedulers, mods ++ result)
+
+      {:file_required, pid, _file, {kind, reason, stacktrace}} ->
+        discard_down(pid)
+        :erlang.raise(kind, reason, stacktrace)
+
+      {:DOWN, ref, :process, pid, reason} ->
+        handle_down(waiting, pid, ref, reason)
+        spawn_requires(files, waiting, callbacks, schedulers, result)
 
       {:module_available, child, ref, file, module, binary} ->
         if each_module_callback = callbacks[:each_module] do
@@ -106,4 +104,17 @@ defmodule Kernel.ParallelRequire do
         spawn_requires(files, waiting, callbacks, schedulers, result)
     end
   end
+
+  defp discard_down(pid) do
+    receive do
+      {:DOWN, _, :process, ^pid, _} -> :ok
+    end
+  end
+
+  defp handle_down(waiting, pid, ref, reason) do
+    if reason != :normal and {pid, ref} in waiting do
+      :erlang.raise(:exit, reason, [])
+    end
+    :ok
+  end
 end

lib/elixir/lib/kernel/special_forms.ex

@@ -316,7 +316,7 @@ defmodule Kernel.SpecialForms do
   defmacro unquote(:<<>>)(args), do: error!([args])
 
   @doc """
-  Defines a remote call or an alias.
+  Defines a remote call, a call to an anonymous function, or an alias.
 
   The dot (`.`) in Elixir can be used for remote calls:
 
@@ -324,8 +324,16 @@ defmodule Kernel.SpecialForms do
       "foo"
 
   In this example above, we have used `.` to invoke `downcase` in the
-  `String` alias, passing "FOO" as argument. We can also use the dot
-  for creating aliases:
+  `String` module, passing `"FOO"` as argument.
+
+  The dot may be used to invoke anonymous functions too:
+
+      iex> (fn(n) -> n end).(7)
+      7
+
+  in which case there is a function on the left hand side.
+
+  We can also use the dot for creating aliases:
 
       iex> Hello.World
       Hello.World
@@ -360,6 +368,13 @@ defmodule Kernel.SpecialForms do
   a remote call regardless of the quote contents. This decision is also reflected
   in the quoted expressions discussed below.
 
+  When the dot is used to invoke an anonymous function there is only one
+  operand, but it is still written using a postfix notation:
+
+      iex> negate = fn(n) -> -n end
+      iex> negate.(7)
+      -7
+
   ## Quoted expression
 
   When `.` is used, the quoted expression may take two distinct
@@ -378,9 +393,9 @@ defmodule Kernel.SpecialForms do
 
   This tuple follows the general quoted expression structure in Elixir,
   with the name as first argument, some keyword list as metadata as second,
-  and the number of arguments as third. In this case, the arguments is the
-  alias `String` and the atom `:downcase`. The second argument is **always**
-  an atom:
+  and the list of arguments as third. In this case, the arguments are the
+  alias `String` and the atom `:downcase`. The second argument in a remote call
+  is **always** an atom regardless of the literal used in the call:
 
       iex> quote do
       ...>   String."downcase"("FOO")
@@ -390,6 +405,15 @@ defmodule Kernel.SpecialForms do
   The tuple containing `:.` is wrapped in another tuple, which actually
   represents the function call, and has `"FOO"` as argument.
 
+  In the case of calls to anonymous functions, the inner tuple with the dot
+  special form has only one argument, reflecting the fact that the operator is
+  unary:
+
+      iex> quote do
+      ...>   negate.(0)
+      ...> end
+      {{:., [], [{:negate, [], __MODULE__}]}, [], [0]}
+
   When the right side is an alias (i.e. starts with uppercase), we get instead:
 
       iex> quote do
@@ -485,13 +509,12 @@ defmodule Kernel.SpecialForms do
   defmacro alias(module, opts), do: error!([module, opts])
 
   @doc """
-  Requires a given module to be compiled and loaded.
+  Requires a module in order to use its macros.
 
   ## Examples
 
-  Notice that usually modules should not be required before usage,
-  the only exception is if you want to use the macros from a module.
-  In such cases, you need to explicitly require them.
+  Public functions in modules are globally available, but in order to use
+  macros, you need to opt-in by requiring the module they are defined in.
 
   Let's suppose you created your own `if/2` implementation in the module
   `MyMacros`. If you want to invoke it, you need to first explicitly
@@ -1593,7 +1616,7 @@ defmodule Kernel.SpecialForms do
   defmacro cond(clauses), do: error!([clauses])
 
   @doc ~S"""
-  Evaluates the given expressions and handles any error, exit
+  Evaluates the given expressions and handles any error, exit,
   or throw that may have happened.
 
   ## Examples
@@ -1605,28 +1628,29 @@ defmodule Kernel.SpecialForms do
           IO.puts "Invalid argument given"
       catch
         value ->
-          IO.puts "caught #{value}"
+          IO.puts "Caught #{inspect(value)}"
       else
         value ->
-          IO.puts "Success! The result was #{value}"
+          IO.puts "Success! The result was #{inspect(value)}"
       after
         IO.puts "This is printed regardless if it failed or succeed"
       end
 
-  The rescue clause is used to handle exceptions, while the catch
-  clause can be used to catch thrown values. The else clause can
-  be used to control flow based on the result of the expression.
-  Catch, rescue and else clauses work based on pattern matching.
+  The `rescue` clause is used to handle exceptions, while the `catch`
+  clause can be used to catch thrown values and exits.
+  The `else` clause can be used to control flow based on the result of
+  the expression. `catch`, `rescue`, and `else` clauses work based on
+  pattern matching (similar to the `case` special form).
 
   Note that calls inside `try/1` are not tail recursive since the VM
   needs to keep the stacktrace in case an exception happens.
 
-  ## Rescue clauses
+  ## `rescue` clauses
 
-  Besides relying on pattern matching, rescue clauses provides some
-  conveniences around exceptions that allows one to rescue an
-  exception by its name. All the following formats are valid rescue
-  expressions:
+  Besides relying on pattern matching, `rescue` clauses provide some
+  conveniences around exceptions that allow one to rescue an
+  exception by its name. All the following formats are valid patterns
+  in `rescue` clauses:
 
       try do
         UndefinedModule.undefined_function
@@ -1656,7 +1680,7 @@ defmodule Kernel.SpecialForms do
 
   ## Erlang errors
 
-  Erlang errors are transformed into Elixir ones during rescue:
+  Erlang errors are transformed into Elixir ones when rescuing:
 
       try do
         :erlang.error(:badarg)
@@ -1675,7 +1699,7 @@ defmodule Kernel.SpecialForms do
       end
 
   In fact, `ErlangError` can be used to rescue any error that is
-  not an Elixir error proper. For example, it can be used to rescue
+  not a proper Elixir error. For example, it can be used to rescue
   the earlier `:badarg` error too, prior to transformation:
 
       try do
@@ -1686,25 +1710,45 @@ defmodule Kernel.SpecialForms do
 
   ## Catching throws and exits
 
-  The catch clause can be used to catch throws values and exits.
+  The `catch` clause can be used to catch thrown values and exits.
 
       try do
         exit(:shutdown)
       catch
-        :exit, :shutdown -> IO.puts "Exited with shutdown reason"
+        :exit, :shutdown ->
+          IO.puts "Exited with shutdown reason"
       end
 
       try do
         throw(:sample)
       catch
         :throw, :sample ->
-          IO.puts "sample thrown"
+          IO.puts ":sample was thrown"
       end
 
-  catch values also support `:error`, as in Erlang, although it is
-  commonly avoided in favor of raise/rescue control mechanisms.
+  The `catch` clause also supports `:error` alongside `:exit` and `:throw`, as
+  in Erlang, although it is commonly avoided in favor of `raise`/`rescue` control
+  mechanisms. One reason for this is that when catching `:error`, the error is
+  not automatically transformed into an Elixir error:
 
-  ## After clauses
+      try do
+        :erlang.error(:badarg)
+      catch
+        :error, :badarg ->
+          :ok
+      end
+
+  Note that it is possible to match both on the caught value as well as the *kind*
+  of such value:
+
+      try do
+        exit(:shutdown)
+      catch
+        kind, value when kind in [:exit, :throw] ->
+          IO.puts "Exited with or thrown value #{inspect(value)}"
+      end
+
+  ## `after` clauses
 
   An `after` clause allows you to define cleanup logic that will be invoked both
   when the tried block of code succeeds and also when an error is raised. Note
@@ -1721,9 +1765,9 @@ defmodule Kernel.SpecialForms do
         File.rm("tmp/story.txt")
       end
 
-  ## Else clauses
+  ## `else` clauses
 
-  Else clauses allow the result of the expression to be pattern
+  `else` clauses allow the result of the tried expression to be pattern
   matched on:
 
       x = 2
@@ -1739,7 +1783,7 @@ defmodule Kernel.SpecialForms do
           :large
       end
 
-  If an else clause is not present and no exceptions are raised,
+  If an `else` clause is not present and no exceptions are raised,
   the result of the expression will be returned:
 
       x = 1
@@ -1751,9 +1795,9 @@ defmodule Kernel.SpecialForms do
             :infinity
         end
 
-  However, when an else clause is present but the result of the expression
-  does not match any of the patterns an exception will be raised. This
-  exception will not be caught by a catch or rescue in the same try:
+  However, when an `else` clause is present but the result of the expression
+  does not match any of the patterns then an exception will be raised. This
+  exception will not be caught by a `catch` or `rescue` in the same `try`:
 
       x = 1
       try do
@@ -1773,8 +1817,8 @@ defmodule Kernel.SpecialForms do
           :error_b
       end
 
-  Similarly, an exception inside an else clause is not caught or rescued
-  inside the same try:
+  Similarly, an exception inside an `else` clause is not caught or rescued
+  inside the same `try`:
 
       try do
         try do
@@ -1794,10 +1838,25 @@ defmodule Kernel.SpecialForms do
       end
 
   This means the VM no longer needs to keep the stacktrace once inside
-  an else clause and so tail recursion is possible when using a `try`
-  with a tail call as the final call inside an else clause. The same
+  an `else` clause and so tail recursion is possible when using a `try`
+  with a tail call as the final call inside an `else` clause. The same
   is true for `rescue` and `catch` clauses.
 
+  Only the result of the tried expression falls down to the `else` clause.
+  If the `try` ends up in the `rescue` or `catch` clauses, their result
+  will not fall down to `else`:
+
+      try do
+        throw(:catch_this)
+      catch
+        :throw, :catch_this ->
+          :it_was_caught
+      else
+        # :it_was_caught will not fall down to this "else" clause.
+        other ->
+          {:else, other}
+      end
+
   ## Variable handling
 
   Since an expression inside `try` may not have been evaluated
@@ -1864,8 +1923,8 @@ defmodule Kernel.SpecialForms do
       end
 
   The `after` clause can be specified even if there are no match clauses.
-  The timeout value given to `after` can be a variable; two special
-  values are allowed:
+  The timeout value given to `after` can be any expression evaluating to
+  one of the allowed values:
 
     * `:infinity` - the process should wait indefinitely for a matching
       message, this is the same as not using a timeout
@@ -1873,6 +1932,10 @@ defmodule Kernel.SpecialForms do
     * `0` - if there is no matching message in the mailbox, the timeout
       will occur immediately
 
+    * positive integer smaller than `4_294_967_295` (`0xFFFFFFFF`
+      in hex notation) - it should be possible to represent the timeout
+      value as an unsigned 32-bit integer.
+
   ## Variables handling
 
   The `receive/1` special form handles variables exactly as the `case/2`

lib/elixir/lib/kernel/typespec.ex

@@ -11,8 +11,11 @@ defmodule Kernel.Typespec do
 
   """
   defmacro deftype(type) do
+    pos = :elixir_locals.cache_env(__CALLER__)
+    %{line: line, file: file, module: module} = __CALLER__
     quote do
-      Kernel.Typespec.deftype(:type, unquote(Macro.escape(type, unquote: true)), __ENV__)
+      Kernel.Typespec.deftype(:type, unquote(Macro.escape(type, unquote: true)),
+                              unquote(line), unquote(file), unquote(module), unquote(pos))
     end
   end
 
@@ -26,8 +29,11 @@ defmodule Kernel.Typespec do
 
   """
   defmacro defopaque(type) do
+    pos = :elixir_locals.cache_env(__CALLER__)
+    %{line: line, file: file, module: module} = __CALLER__
     quote do
-      Kernel.Typespec.deftype(:opaque, unquote(Macro.escape(type, unquote: true)), __ENV__)
+      Kernel.Typespec.deftype(:opaque, unquote(Macro.escape(type, unquote: true)),
+                              unquote(line), unquote(file), unquote(module), unquote(pos))
     end
   end
 
@@ -41,8 +47,11 @@ defmodule Kernel.Typespec do
 
   """
   defmacro deftypep(type) do
+    pos = :elixir_locals.cache_env(__CALLER__)
+    %{line: line, file: file, module: module} = __CALLER__
     quote do
-      Kernel.Typespec.deftype(:typep, unquote(Macro.escape(type, unquote: true)), __ENV__)
+      Kernel.Typespec.deftype(:typep, unquote(Macro.escape(type, unquote: true)),
+                              unquote(line), unquote(file), unquote(module), unquote(pos))
     end
   end
 
@@ -56,8 +65,11 @@ defmodule Kernel.Typespec do
 
   """
   defmacro defspec(spec) do
+    pos = :elixir_locals.cache_env(__CALLER__)
+    %{line: line, file: file, module: module} = __CALLER__
     quote do
-      Kernel.Typespec.defspec(:spec, unquote(Macro.escape(spec, unquote: true)), __ENV__)
+      Kernel.Typespec.defspec(:spec, unquote(Macro.escape(spec, unquote: true)),
+                              unquote(line), unquote(file), unquote(module), unquote(pos))
     end
   end
 
@@ -71,12 +83,14 @@ defmodule Kernel.Typespec do
 
   """
   defmacro defcallback(spec) do
+    pos = :elixir_locals.cache_env(__CALLER__)
+    %{line: line, file: file, module: module} = __CALLER__
     quote do
-      Kernel.Typespec.defspec(:callback, unquote(Macro.escape(spec, unquote: true)), __ENV__)
+      Kernel.Typespec.defspec(:callback, unquote(Macro.escape(spec, unquote: true)),
+                              unquote(line), unquote(file), unquote(module), unquote(pos))
     end
   end
 
-
   @doc """
   Defines a macro callback.
   This macro is responsible for handling the attribute `@macrocallback`.
@@ -87,32 +101,14 @@ defmodule Kernel.Typespec do
 
   """
   defmacro defmacrocallback(spec) do
+    pos = :elixir_locals.cache_env(__CALLER__)
+    %{line: line, file: file, module: module} = __CALLER__
     quote do
-      Kernel.Typespec.defspec(:macrocallback, unquote(Macro.escape(spec, unquote: true)), __ENV__)
+      Kernel.Typespec.defspec(:macrocallback, unquote(Macro.escape(spec, unquote: true)),
+                              unquote(line), unquote(file), unquote(module), unquote(pos))
     end
   end
 
-  defmacro defoptional_callbacks(callbacks) do
-    quote do
-      Module.store_typespec(__ENV__.module, :optional_callbacks, {__ENV__.line, unquote(callbacks)})
-    end
-  end
-
-  @doc """
-  Defines a `type`, `typep` or `opaque` by receiving a typespec expression.
-  """
-  def define_type(kind, expr, doc \\ nil, env) do
-    Module.store_typespec(env.module, kind, {kind, expr, doc, env})
-  end
-
-  @doc """
-  Defines a `spec` by receiving a typespec expression.
-  """
-  @spec define_spec(atom, Macro.t, Macro.Env.t) :: Keyword.t
-  def define_spec(kind, expr, env) do
-    defspec(kind, expr, env)
-  end
-
   @doc """
   Returns `true` if the current module defines a given type
   (private, opaque or not). This function is only available
@@ -157,7 +153,7 @@ defmodule Kernel.Typespec do
   @doc """
   Converts a spec clause back to Elixir AST.
   """
-  @spec spec_to_ast(atom, tuple) :: {atom, Keyword.t, [Macro.t]}
+  @spec spec_to_ast(atom, tuple) :: {atom, keyword, [Macro.t]}
   def spec_to_ast(name, spec)
   def spec_to_ast(name, {:type, line, :fun, [{:type, _, :product, args}, result]})
       when is_atom(name) do
@@ -335,50 +331,49 @@ defmodule Kernel.Typespec do
   ## Macro callbacks
 
   @doc false
-  def defspec(kind, expr, caller) when kind in [:callback, :macrocallback] do
+  def defspec(kind, expr, line, file, module, pos) when kind in [:callback, :macrocallback] do
     case spec_to_signature(expr) do
       {name, arity} ->
-        store_callbackdoc(caller, caller.module, kind, name, arity)
+        store_callbackdoc(line, file, module, kind, name, arity)
       :error ->
         :error
     end
-    Module.store_typespec(caller.module, kind, {kind, expr, caller})
+    Module.store_typespec(module, kind, {kind, expr, pos})
   end
 
   @doc false
-  def defspec(kind, expr, caller) do
-    Module.store_typespec(caller.module, kind, {kind, expr, caller})
+  def defspec(kind, expr, _line, _file, module, pos) do
+    Module.store_typespec(module, kind, {kind, expr, pos})
   end
 
-  defp store_callbackdoc(caller, module, kind, name, arity) do
+  defp store_callbackdoc(line, _file, module, kind, name, arity) do
     table = :elixir_module.data_table(module)
-    {line, doc} = get_doc_info(table, :doc, caller)
+    {line, doc} = get_doc_info(table, :doc, line)
     :ets.insert(table, {{:callbackdoc, {name, arity}}, line, kind, doc})
   end
 
-  defp get_doc_info(table, attr, caller) do
+  defp get_doc_info(table, attr, line) do
     case :ets.take(table, attr) do
       [{^attr, {line, doc}, _, _}] -> {line, doc}
-      [] -> {caller.line, nil}
+      [] -> {line, nil}
     end
   end
 
   @doc false
-  def deftype(kind, expr, caller) do
-    module = caller.module
+  def deftype(kind, expr, line, file, module, pos) do
     case type_to_signature(expr) do
-      {name, arity} -> store_typedoc(caller, caller.module, kind, name, arity)
+      {name, arity} -> store_typedoc(line, file, module, kind, name, arity)
       :error -> :error
     end
-    Module.store_typespec(module, kind, {kind, expr, caller})
+    Module.store_typespec(module, kind, {kind, expr, pos})
   end
 
-  defp store_typedoc(caller, module, kind, name, arity) do
+  defp store_typedoc(line, file, module, kind, name, arity) do
     table = :elixir_module.data_table(module)
-    {line, doc} = get_doc_info(table, :typedoc, caller)
+    {line, doc} = get_doc_info(table, :typedoc, line)
 
     if kind == :typep && doc do
-      :elixir_errors.warn(caller.line, caller.file, "type #{name}/#{arity} is private, " <>
+      :elixir_errors.warn(line, file, "type #{name}/#{arity} is private, " <>
                           "@typedoc's are always discarded for private types")
     end
 
@@ -388,7 +383,9 @@ defmodule Kernel.Typespec do
   ## Translation from Elixir AST to typespec AST
 
   @doc false
-  def translate_type(kind, {:::, _, [{name, _, args}, definition]}, caller) when is_atom(name) and name != ::: do
+  def translate_type(kind, {:::, _, [{name, _, args}, definition]}, pos) when is_atom(name) and name != ::: do
+    caller = :elixir_locals.get_cached_env(pos)
+
     args =
       if is_atom(args) do
         []
@@ -409,34 +406,37 @@ defmodule Kernel.Typespec do
         :opaque -> {:opaque, true}
       end
 
-    if elixir_builtin_type?(name, arity) do
-      :elixir_errors.handle_file_error(caller.file,
-        {caller.line, :erl_lint, {:builtin_type, {name, arity}}})
+    if builtin_type?(name, arity) do
+      compile_error caller, "type #{name}/#{arity} is a builtin type and it cannot be redefined"
     end
 
     {{kind, {name, arity}, type}, caller.line, export}
   end
 
-  def translate_type(_kind, other, caller) do
+  def translate_type(_kind, other, pos) do
+    caller = :elixir_locals.get_cached_env(pos)
     type_spec = Macro.to_string(other)
     compile_error caller, "invalid type specification: #{type_spec}"
   end
 
-  defp elixir_builtin_type?(:as_boolean, 1), do: true
-  defp elixir_builtin_type?(:struct, 0), do: true
-  defp elixir_builtin_type?(:charlist, 0), do: true
-  # TODO: Deprecate char_list type by v1.5
-  defp elixir_builtin_type?(:char_list, 0), do: true
-  defp elixir_builtin_type?(:keyword, 0), do: true
-  defp elixir_builtin_type?(:keyword, 1), do: true
-  defp elixir_builtin_type?(_, _), do: false
+  defp builtin_type?(:as_boolean, 1), do: true
+  defp builtin_type?(:struct, 0), do: true
+  defp builtin_type?(:charlist, 0), do: true
+  # TODO: Remove char_list type by 2.0
+  defp builtin_type?(:char_list, 0), do: true
+  defp builtin_type?(:nonempty_charlist, 0), do: true
+  defp builtin_type?(:keyword, 0), do: true
+  defp builtin_type?(:keyword, 1), do: true
+  defp builtin_type?(name, arity), do: :erl_internal.is_type(name, arity)
 
   @doc false
-  def translate_spec(kind, {:when, _meta, [spec, guard]}, caller) do
+  def translate_spec(kind, {:when, _meta, [spec, guard]}, pos) do
+    caller = :elixir_locals.get_cached_env(pos)
     translate_spec(kind, spec, guard, caller)
   end
 
-  def translate_spec(kind, spec, caller) do
+  def translate_spec(kind, spec, pos) do
+    caller = :elixir_locals.get_cached_env(pos)
     translate_spec(kind, spec, [], caller)
   end
 
@@ -457,8 +457,6 @@ defmodule Kernel.Typespec do
 
   defp translate_spec(kind, meta, name, args, return, guard, caller) when is_atom(args),
     do: translate_spec(kind, meta, name, [], return, guard, caller)
-  defp translate_spec(:macrocallback, meta, name, args, return, guard, caller),
-    do: translate_spec(:callback, meta, :"MACRO-#{name}", macro_args(args), return, guard, caller)
   defp translate_spec(kind, meta, name, args, return, guard, caller) do
     ensure_no_defaults!(args)
 
@@ -480,10 +478,6 @@ defmodule Kernel.Typespec do
     {{kind, {name, arity}, spec}, caller.line}
   end
 
-  defp macro_args(args) do
-    [quote(do: {line :: Macro.Env.line, env :: Macro.Env.t}) | args]
-  end
-
   defp ensure_no_defaults!(args) do
     :lists.foreach fn
       {:::, _, [left, right]} ->
@@ -634,7 +628,7 @@ defmodule Kernel.Typespec do
   end
 
   defp typespec_to_ast({:type, line, :fun, []}) do
-    typespec_to_ast({:type, line, :fun, [{:type, line, :any}, {:type, line, :any, []} ]})
+    typespec_to_ast({:type, line, :fun, [{:type, line, :any}, {:type, line, :any, []}]})
   end
 
   defp typespec_to_ast({:type, line, :range, [left, right]}) do
@@ -659,12 +653,16 @@ defmodule Kernel.Typespec do
   end
 
   # Special shortcut(s)
-  # TODO: Deprecate char_list type by v1.5
+  # TODO: Remove char_list type by 2.0
   defp typespec_to_ast({:remote_type, line, [{:atom, _, :elixir}, {:atom, _, type}, []]})
       when type in [:charlist, :char_list] do
     typespec_to_ast({:type, line, :charlist, []})
   end
 
+  defp typespec_to_ast({:remote_type, line, [{:atom, _, :elixir}, {:atom, _, :nonempty_charlist}, []]}) do
+    typespec_to_ast({:type, line, :nonempty_charlist, []})
+  end
+
   defp typespec_to_ast({:remote_type, line, [{:atom, _, :elixir}, {:atom, _, :struct}, []]}) do
     typespec_to_ast({:type, line, :struct, []})
   end
@@ -767,7 +765,7 @@ defmodule Kernel.Typespec do
         {{:optional, meta2, [k]}, v} ->
           {:type, line(meta2), :map_field_assoc, [typespec(k, vars, caller), typespec(v, vars, caller)]}
         {k, v} ->
-          # TODO: Emit warnings on v1.5
+          # TODO: Emit warnings on v1.6 (when we drop OTP 18 support)
           # :elixir_errors.warn(caller.line, caller.file,
           #   "invalid map specification. %{foo => bar} is deprecated in favor of " <>
           #   "%{required(foo) => bar} and %{optional(foo) => bar}. required/1 is an " <>
@@ -905,7 +903,7 @@ defmodule Kernel.Typespec do
   end
 
   # Handle tuples
-  defp typespec({:tuple, meta, args}, _vars, _caller) when args == [] or is_atom(args) do
+  defp typespec({:tuple, meta, []}, _vars, _caller) do
     {:type, line(meta), :tuple, :any}
   end
 
@@ -933,18 +931,38 @@ defmodule Kernel.Typespec do
   end
 
   # Handle local calls
-  defp typespec({type, meta, arguments}, vars, caller) when type in [:string, :nonempty_string] do
-    :elixir_errors.warn caller.line, caller.file, "#{type}() type use is discouraged. For character lists, use " <>
-      "charlist() type, for strings, String.t()\n#{Exception.format_stacktrace(Macro.Env.stacktrace(caller))}"
+  defp typespec({:string, meta, arguments}, vars, caller) do
+    :elixir_errors.warn caller.line, caller.file,
+      "string() type use is discouraged. " <>
+      "For character lists, use charlist() type, for strings, String.t()\n" <>
+      Exception.format_stacktrace(Macro.Env.stacktrace(caller))
+
     arguments = for arg <- arguments, do: typespec(arg, vars, caller)
-    {:type, line(meta), type, arguments}
+    {:type, line(meta), :string, arguments}
   end
 
-  # TODO: Deprecate char_list type by v1.5
+  defp typespec({:nonempty_string, meta, arguments}, vars, caller) do
+    :elixir_errors.warn caller.line, caller.file,
+      "nonempty_string() type use is discouraged. " <>
+      "For non-empty character lists, use nonempty_charlist() type, for strings, String.t()\n" <>
+      Exception.format_stacktrace(Macro.Env.stacktrace(caller))
+
+    arguments = for arg <- arguments, do: typespec(arg, vars, caller)
+    {:type, line(meta), :nonempty_string, arguments}
+  end
+
+  # TODO: Remove char_list type by 2.0
   defp typespec({type, _meta, []}, vars, caller) when type in [:charlist, :char_list] do
+    if type == :char_list do
+      :elixir_errors.warn caller.line, caller.file, "the char_list() type is deprecated, use charlist()"
+    end
     typespec((quote do: :elixir.charlist()), vars, caller)
   end
 
+  defp typespec({:nonempty_charlist, _meta, []}, vars, caller) do
+    typespec((quote do: :elixir.nonempty_charlist()), vars, caller)
+  end
+
   defp typespec({:struct, _meta, []}, vars, caller) do
     typespec((quote do: :elixir.struct()), vars, caller)
   end
@@ -1014,7 +1032,7 @@ defmodule Kernel.Typespec do
 
   defp remote_type({remote, meta, name, arguments}, vars, caller) do
     arguments = for arg <- arguments, do: typespec(arg, vars, caller)
-    {:remote_type, line(meta), [ remote, name, arguments ]}
+    {:remote_type, line(meta), [remote, name, arguments]}
   end
 
   defp collect_union({:|, _, [a, b]}), do: [a | collect_union(b)]

lib/elixir/lib/kernel/utils.ex

@@ -23,7 +23,7 @@ defmodule Kernel.Utils do
   """
   def defdelegate(fun, opts) when is_list(opts) do
     # TODO: Remove by 2.0
-    append_first = Keyword.get(opts, :append_first, false)
+    append_first? = Keyword.get(opts, :append_first, false)
 
     {name, args} =
       case Macro.decompose_call(fun) do
@@ -31,38 +31,30 @@ defmodule Kernel.Utils do
         _ -> raise ArgumentError, "invalid syntax in defdelegate #{Macro.to_string(fun)}"
       end
 
-    as_args_list = normalize_args(args)
     as = Keyword.get(opts, :as, name)
-    :lists.map(fn as_args ->
-      formal_args = make_formal_args(as_args)
-      as_args = case append_first do
-        true  -> tl(as_args) ++ [hd(as_args)]
-        false -> as_args
-      end
-      {name, formal_args, as, as_args}
-    end, as_args_list)
+    as_args = build_as_args(args, append_first?)
+
+    {name, args, as, as_args}
   end
 
-  defp make_formal_args(args) do
-    fun = &match?({name, _, mod} when is_atom(name) and is_atom(mod), &1)
-    :lists.filter(fun, args)
+  defp build_as_args(args, append_first?) do
+    as_args = :lists.map(&build_as_arg/1, args)
+
+    case append_first? do
+      true -> tl(as_args) ++ [hd(as_args)]
+      false -> as_args
+    end
   end
 
-  defp normalize_args(raw_args) do
-    :lists.foldr(fn
-      ({:\\, _, [arg, default_arg]}, [as_args | _] = as_args_list) ->
-        new_as_args = [default_arg | as_args]
-        [new_as_args | add_arg(as_args_list, arg)]
-      (arg, as_args_list) ->
-        add_arg(as_args_list, arg)
-    end, [[]], raw_args)
+  defp build_as_arg({:\\, _, [arg, _default_arg]}), do: validate_arg(arg)
+  defp build_as_arg(arg), do: validate_arg(arg)
+
+  defp validate_arg({name, _, mod} = arg) when is_atom(name) and is_atom(mod) do
+    arg
   end
 
-  defp add_arg(as_args_list, {name, _, mod} = arg) when is_atom(name) and is_atom(mod),
-    do: :lists.map(&([arg | &1]), as_args_list)
-  defp add_arg(_, code) do
-    raise ArgumentError,
-      "defdelegate/2 only accepts function parameters, got: #{Macro.to_string(code)}"
+  defp validate_arg(ast) do
+    raise ArgumentError, "defdelegate/2 only accepts function parameters, got: #{Macro.to_string(ast)}"
   end
 
   @doc """
@@ -92,8 +84,15 @@ defmodule Kernel.Utils do
         raise ArgumentError, "struct field names must be atoms, got: #{inspect other}"
     end, fields)
 
+    enforce_keys = List.wrap(Module.get_attribute(module, :enforce_keys))
+
+    :lists.foreach(fn
+      key when is_atom(key) -> :ok
+      key -> raise ArgumentError, "keys given to @enforce_keys must be atoms, got: #{inspect key}"
+    end, enforce_keys)
+
     {:maps.put(:__struct__, module, :maps.from_list(fields)),
-     List.wrap(Module.get_attribute(module, :enforce_keys)),
+     enforce_keys,
      Module.get_attribute(module, :derive)}
   end
 
@@ -120,7 +119,7 @@ defmodule Kernel.Utils do
     exception
   end
   def raise(other) do
-    ArgumentError.exception("raise/1 expects an alias, string or exception as " <>
+    ArgumentError.exception("raise/1 expects a module name, string or exception as " <>
                             "the first argument, got: #{inspect other}")
   end
 end

lib/elixir/lib/keyword.ex

@@ -6,6 +6,31 @@ defmodule Keyword do
   element of the tuple is an atom and the second element
   can be any value.
 
+  For example, the following is a keyword list:
+
+      [{:exit_on_close, true}, {:active, :once}, {:packet_size, 1024}]
+
+  Elixir provides a special and more concise syntax for keyword lists
+  that looks like this:
+
+      [exit_on_close: true, active: :once, packet_size: 1024]
+
+  This is also the syntax that Elixir uses to inspect keyword lists:
+
+      iex> [{:active, :once}]
+      [active: :once]
+
+  The two syntaxes are completely equivalent. Note that when keyword
+  lists are passed as the last argument to a function, if the short-hand
+  syntax is used then the square brackets around the keyword list can
+  be omitted as well. For example, the following:
+
+      String.split("1-0", "-", trim: true, parts: 2)
+
+  is equivalent to:
+
+      String.split("1-0", "-", [trim: true, parts: 2])
+
   A keyword may have duplicated keys so it is not strictly
   a key-value store. However most of the functions in this module
   behave exactly as a dictionary so they work similarly to
@@ -15,6 +40,10 @@ defmodule Keyword do
   the given key, regardless if duplicated entries exist.
   Similarly, `Keyword.put/3` and `Keyword.delete/3` ensure all
   duplicated entries for a given key are removed when invoked.
+  Note that operations that require keys to be found in the keyword
+  list (like `Keyword.get/3`) need to traverse the list in order
+  to find keys, so these operations may be slower than their map
+  counterparts.
 
   A handful of functions exist to handle duplicated keys, in
   particular, `Enum.into/2` allows creating new keywords without
@@ -22,7 +51,7 @@ defmodule Keyword do
   a given key and `delete_first/2` deletes just one of the existing
   entries.
 
-  The functions in Keyword do not guarantee any property when
+  The functions in `Keyword` do not guarantee any property when
   it comes to ordering. However, since a keyword list is simply a
   list, all the operations defined in `Enum` and `List` can be
   applied too, especially when ordering is required.
@@ -145,7 +174,6 @@ defmodule Keyword do
       3
 
   """
-  @spec get(t, key) :: value
   @spec get(t, key, value) :: value
   def get(keywords, key, default \\ nil) when is_list(keywords) and is_atom(key) do
     case :lists.keyfind(key, 1, keywords) do
@@ -534,6 +562,49 @@ defmodule Keyword do
     end
   end
 
+  @doc """
+  Alters the value stored under `key` to `value`, but only
+  if the entry `key` already exists in the keyword list.
+
+  In the case a value is stored multiple times in the keyword list,
+  later occurrences are removed.
+
+  ## Examples
+
+      iex> Keyword.replace([a: 1], :b, 2)
+      [a: 1]
+      iex> Keyword.replace([a: 1, b: 2, a: 4], :a, 3)
+      [a: 3, b: 2]
+
+  """
+  @spec replace(t, key, value) :: t
+  def replace(keywords, key, value) when is_list(keywords) and is_atom(key) do
+    case :lists.keyfind(key, 1, keywords) do
+      {^key, _} -> [{key, value} | delete(keywords, key)]
+      false -> keywords
+    end
+  end
+
+  @doc """
+  Similar to `replace/3`, but will raise a `KeyError`
+  if the entry `key` does not exist.
+
+  ## Examples
+
+      iex> Keyword.replace!([a: 1, b: 2, a: 4], :a, 3)
+      [a: 3, b: 2]
+      iex> Keyword.replace!([a: 1], :b, 2)
+      ** (KeyError) key :b not found in: [a: 1]
+
+  """
+  @spec replace!(t, key, value) :: t
+  def replace!(keywords, key, value) when is_list(keywords) and is_atom(key) do
+    case :lists.keyfind(key, 1, keywords) do
+      {^key, _} -> [{key, value} | delete(keywords, key)]
+      false -> raise KeyError, key: key, term: keywords
+    end
+  end
+
   @doc """
   Checks if two keywords are equal.
 
@@ -803,7 +874,7 @@ defmodule Keyword do
   """
   @spec drop(t, [key]) :: t
   def drop(keywords, keys) when is_list(keywords) do
-    :lists.filter(fn {k, _} -> not k in keys end, keywords)
+    :lists.filter(fn {key, _} -> key not in keys end, keywords)
   end
 
   @doc """
@@ -908,8 +979,8 @@ defmodule Keyword do
 
   @doc false
   # TODO: Remove on 2.0
+  # (hard-deprecated in elixir_dispatch)
   def size(keyword) do
-    IO.warn "Keyword.size/1 is deprecated, please use Kernel.length/1"
     length(keyword)
   end
 end

lib/elixir/lib/list.ex

@@ -116,9 +116,10 @@ defmodule List do
 
   """
   @spec delete(list, any) :: list
-  def delete(list, item) do
-    :lists.delete(item, list)
-  end
+  def delete(list, item)
+  def delete([item | list], item), do: list
+  def delete([other | list], item), do: [other | delete(list, item)]
+  def delete([], _item), do: []
 
   @doc """
   Duplicates the given element `n` times in a list.
@@ -576,6 +577,38 @@ defmodule List do
     end
   end
 
+  @doc """
+  Returns `true` if `list` starts with the given `prefix` list; otherwise returns `false`.
+
+  If `prefix` is an empty list, it returns `true`.
+
+  ### Examples
+
+      iex> List.starts_with?([1, 2, 3], [1, 2])
+      true
+
+      iex> List.starts_with?([1, 2], [1, 2, 3])
+      false
+
+      iex> List.starts_with?([:alpha], [])
+      true
+
+      iex> List.starts_with?([], [:alpha])
+      false
+
+  """
+  @spec starts_with?(list, list) :: boolean
+  @spec starts_with?(list, []) :: true
+  @spec starts_with?([], nonempty_list) :: false
+  def starts_with?(list, prefix)
+
+  def starts_with?([head | tail], [head | prefix_tail]),
+    do: starts_with?(tail, prefix_tail);
+  def starts_with?(list, []) when is_list(list),
+    do: true
+  def starts_with?(list, [_ | _]) when is_list(list),
+    do: false
+
   @doc """
   Converts a charlist to an atom.
 

lib/elixir/lib/list/chars.ex

@@ -1,18 +1,24 @@
 defprotocol List.Chars do
   @moduledoc ~S"""
-  The List.Chars protocol is responsible for
-  converting a structure to a list (only if applicable).
+  The `List.Chars` protocol is responsible for
+  converting a structure to a charlist (only if applicable).
+
   The only function required to be implemented is
-  `to_charlist` which does the conversion.
+  `to_charlist/1` which does the conversion.
 
   The `to_charlist/1` function automatically imported
   by `Kernel` invokes this protocol.
   """
 
+  @doc """
+  Converts `term` to a charlist.
+  """
+  @spec to_charlist(t) :: charlist
   def to_charlist(term)
 
-  # TODO: Deprecate by v1.5
   @doc false
+  # TODO: Remove by 2.0
+  # (hard-deprecated in elixir_dispatch)
   Kernel.def to_char_list(term) do
     __MODULE__.to_charlist(term)
   end

lib/elixir/lib/macro.ex

@@ -56,25 +56,30 @@ defmodule Macro do
   """
 
   @typedoc "Abstract Syntax Tree (AST)"
-  @type t :: expr | {t, t} | atom | number | binary | pid | fun | [t]
-  @type expr :: {expr | atom, Keyword.t, atom | [t]}
+  @type t :: expr | literal
 
-  @binary_ops [:===, :!==,
-    :==, :!=, :<=, :>=,
-    :&&, :||, :<>, :++, :--, :\\, :::, :<-, :.., :|>, :=~,
-    :<, :>, :->,
-    :+, :-, :*, :/, :=, :|, :.,
-    :and, :or, :when, :in,
-    :~>>, :<<~, :~>, :<~, :<~>, :<|>,
-    :<<<, :>>>, :|||, :&&&, :^^^, :~~~]
+  @typedoc "Represents expressions in the AST"
+  @type expr :: {expr | atom, keyword, atom | [t]}
+
+  @typedoc "Represents literals in the AST"
+  @type literal :: atom | number | binary | fun | {t, t} | [t]
+
+  binary_ops =
+    [:===, :!==, :==, :!=, :<=, :>=,
+     :&&, :||, :<>, :++, :--, :\\, :::, :<-, :.., :|>, :=~,
+     :<, :>, :->,
+     :+, :-, :*, :/, :=, :|, :.,
+     :and, :or, :when, :in,
+     :~>>, :<<~, :~>, :<~, :<~>, :<|>,
+     :<<<, :>>>, :|||, :&&&, :^^^, :~~~]
 
   @doc false
-  defmacro binary_ops, do: @binary_ops
+  defmacro binary_ops, do: unquote(binary_ops)
 
-  @unary_ops [:!, :@, :^, :not, :+, :-, :~~~, :&]
+  unary_ops = [:!, :@, :^, :not, :+, :-, :~~~, :&]
 
   @doc false
-  defmacro unary_ops, do: @unary_ops
+  defmacro unary_ops, do: unquote(unary_ops)
 
   @spec binary_op_props(atom) :: {:left | :right, precedence :: integer}
   defp binary_op_props(o) do
@@ -98,6 +103,69 @@ defmodule Macro do
     end
   end
 
+  # Classifies the given atom into one of the following categories:
+  #
+  #   * :alias - a valid Elixir alias, like Foo, Foo.Bar and so on
+  #
+  #   * :callable - an atom that can be used as a function call after the
+  #     . operator (for example, :<> is callable because Foo.<>(1, 2, 3) is valid
+  #     syntax); this category includes identifiers like :foo
+  #
+  #   * :not_callable - an atom that cannot be used as a function call after the
+  #     . operator (for example, :<<>> is not callable because Foo.<<>> is a
+  #     syntax error); this category includes atoms like :Foo, since they are
+  #     valid identifiers but they need quotes to be used in function calls
+  #     (Foo."Bar")
+  #
+  #   * :other - any other atom (these are usually escaped when inspected, like
+  #     :"foo and bar")
+  @doc false
+  def classify_identifier(atom) when is_atom(atom) do
+    charlist = Atom.to_charlist(atom)
+
+    cond do
+      atom in [:"%", :"%{}", :"{}", :"<<>>", :"...", :"..", :"."] ->
+        :not_callable
+      atom in unquote(unary_ops) or atom in unquote(binary_ops) ->
+        :callable
+      valid_alias?(charlist) ->
+        :alias
+      true ->
+        case :elixir_config.safe_get(:identifier_tokenizer, String.Tokenizer).tokenize(charlist) do
+          {kind, _acc, [], _, _, special} ->
+            if kind == :identifier and not :lists.member(?@, special) do
+              :callable
+            else
+              :not_callable
+            end
+          _ ->
+            :other
+        end
+    end
+  end
+
+  defp valid_alias?('Elixir' ++ rest), do: valid_alias_piece?(rest)
+  defp valid_alias?(_other), do: false
+
+  defp valid_alias_piece?([?., char | rest]) when char >= ?A and char <= ?Z,
+    do: valid_alias_piece?(trim_leading_while_valid_identifier(rest))
+  defp valid_alias_piece?([]),
+    do: true
+  defp valid_alias_piece?(_other),
+    do: false
+
+  defp trim_leading_while_valid_identifier([char | rest])
+       when char >= ?a and char <= ?z
+       when char >= ?A and char <= ?Z
+       when char >= ?0 and char <= ?9
+       when char == ?_ do
+    trim_leading_while_valid_identifier(rest)
+  end
+
+  defp trim_leading_while_valid_identifier(other) do
+    other
+  end
+
   @doc """
   Breaks a pipeline expression into a list.
 
@@ -159,7 +227,7 @@ defmodule Macro do
   end
 
   def pipe(expr, {call, _, [_, _]} = call_args, _integer)
-      when call in unquote(@binary_ops) do
+      when call in unquote(binary_ops) do
     raise ArgumentError, "cannot pipe #{to_string expr} into #{to_string call_args}, " <>
                          "the #{to_string call} operator can only take two arguments"
   end
@@ -192,7 +260,7 @@ defmodule Macro do
   end
 
   @doc false
-  def pipe_warning({call, _, _}) when call in unquote(@unary_ops) do
+  def pipe_warning({call, _, _}) when call in unquote(unary_ops) do
     "piping into a unary operator is deprecated. You could use e.g. Kernel.+(5) instead of +5"
   end
   def pipe_warning(_), do: nil
@@ -212,7 +280,7 @@ defmodule Macro do
       {:sample, [], []}
 
   """
-  @spec update_meta(t, (Keyword.t -> Keyword.t)) :: t
+  @spec update_meta(t, (keyword -> keyword)) :: t
   def update_meta(quoted, fun)
 
   def update_meta({left, meta, right}, fun) when is_list(meta) do
@@ -223,6 +291,21 @@ defmodule Macro do
     other
   end
 
+  @doc """
+  Generates AST nodes for a given number of required argument variables using
+  `Macro.var/2`.
+
+  ## Examples
+
+      iex> Macro.generate_arguments(2, __MODULE__)
+      [{:var1, [], __MODULE__}, {:var2, [], __MODULE__}]
+
+  """
+  def generate_arguments(0, _), do: []
+  def generate_arguments(amount, context) when is_integer(amount) and amount > 0 and is_atom(context) do
+    for id <- 1..amount, do: Macro.var(String.to_atom("var" <> Integer.to_string(id)), context)
+  end
+
   @doc """
   Generates an AST node representing the variable given
   by the atoms `var` and `context`.
@@ -391,8 +474,7 @@ defmodule Macro do
       1
 
   """
-  @spec escape(term) :: Macro.t
-  @spec escape(term, Keyword.t) :: Macro.t
+  @spec escape(term, keyword) :: Macro.t
   def escape(expr, opts \\ []) do
     elem(:elixir_quote.escape(expr, Keyword.get(opts, :unquote, false)), 0)
   end
@@ -503,7 +585,7 @@ defmodule Macro do
       def unescape_map(?u), do: true
       def unescape_map(e),  do: e
 
-  If the `unescape_map/1` function returns `false`. The char is
+  If the `unescape_map/1` function returns `false`, the char is
   not escaped and the backslash is kept in the string.
 
   Hexadecimals and Unicode codepoints will be escaped if the map
@@ -569,7 +651,6 @@ defmodule Macro do
       "one + two"
 
   """
-  @spec to_string(Macro.t) :: String.t
   @spec to_string(Macro.t, (Macro.t, String.t -> String.t)) :: String.t
   def to_string(tree, fun \\ fn(_ast, string) -> string end)
 
@@ -667,7 +748,7 @@ defmodule Macro do
   end
 
   # Binary ops
-  def to_string({op, _, [left, right]} = ast, fun) when op in unquote(@binary_ops) do
+  def to_string({op, _, [left, right]} = ast, fun) when op in unquote(binary_ops) do
     fun.(ast, op_to_string(left, fun, op, :left) <> " #{op} " <> op_to_string(right, fun, op, :right))
   end
 
@@ -692,9 +773,14 @@ defmodule Macro do
     fun.(ast, "&(" <> to_string(arg, fun) <> ")")
   end
 
+  # left not in right
+  def to_string({:not, _, [{:in, _, [left, right]}]} = ast, fun)  do
+    fun.(ast, to_string(left, fun) <> " not in " <> to_string(right, fun))
+  end
+
   # Unary ops
   def to_string({unary, _, [{binary, _, [_, _]} = arg]} = ast, fun)
-      when unary in unquote(@unary_ops) and binary in unquote(@binary_ops) do
+      when unary in unquote(unary_ops) and binary in unquote(binary_ops) do
     fun.(ast, Atom.to_string(unary) <> "(" <> to_string(arg, fun) <> ")")
   end
 
@@ -702,13 +788,13 @@ defmodule Macro do
     fun.(ast, "not " <> to_string(arg, fun))
   end
 
-  def to_string({op, _, [arg]} = ast, fun) when op in unquote(@unary_ops) do
+  def to_string({op, _, [arg]} = ast, fun) when op in unquote(unary_ops) do
     fun.(ast, Atom.to_string(op) <> to_string(arg, fun))
   end
 
   # Access
   def to_string({{:., _, [Access, :get]}, _, [{op, _, _} = left, right]} = ast, fun)
-      when op in unquote(@binary_ops) do
+      when op in unquote(binary_ops) do
     fun.(ast, "(" <> to_string(left, fun) <> ")" <> to_string([right], fun))
   end
 
@@ -740,7 +826,8 @@ defmodule Macro do
       list == [] ->
         "[]"
       :io_lib.printable_list(list) ->
-        IO.iodata_to_binary [?', Inspect.BitString.escape(IO.chardata_to_string(list), ?'), ?']
+        {escaped, _} = Inspect.BitString.escape(IO.chardata_to_string(list), ?')
+        IO.iodata_to_binary [?', escaped, ?']
       Inspect.List.keyword?(list) ->
         "[" <> kw_list_to_string(list, fun) <> "]"
       true ->
@@ -776,10 +863,10 @@ defmodule Macro do
   end
 
   # Block keywords
-  @kw_keywords [:do, :catch, :rescue, :after, :else]
+  kw_keywords = [:do, :catch, :rescue, :after, :else]
 
   defp kw_blocks?([{:do, _} | _] = kw) do
-    Enum.all?(kw, &match?({x, _} when x in unquote(@kw_keywords), &1))
+    Enum.all?(kw, &match?({x, _} when x in unquote(kw_keywords), &1))
   end
   defp kw_blocks?(_), do: false
 
@@ -840,6 +927,8 @@ defmodule Macro do
     do: "(" <> module_to_string(arg, fun) <> ")."
   defp call_to_string({:., _, [arg]}, fun),
     do: module_to_string(arg, fun) <> "."
+  defp call_to_string({:., _, [left, right]}, fun) when is_atom(right),
+    do: module_to_string(left, fun) <> "." <> call_to_string_for_atom(right)
   defp call_to_string({:., _, [left, right]}, fun),
     do: module_to_string(left, fun) <> "." <> call_to_string(right, fun)
   defp call_to_string(other, fun),
@@ -851,6 +940,10 @@ defmodule Macro do
     target <> "(" <> args <> ")"
   end
 
+  defp call_to_string_for_atom(atom) do
+    Inspect.Function.escape_name(atom)
+  end
+
   defp args_to_string(args, fun) do
     {list, last} = :elixir_utils.split_last(args)
 
@@ -867,7 +960,7 @@ defmodule Macro do
   end
 
   defp kw_blocks_to_string(kw, fun) do
-    Enum.reduce(@kw_keywords, " ", fn(x, acc) ->
+    Enum.reduce(unquote(kw_keywords), " ", fn(x, acc) ->
       case Keyword.has_key?(kw, x) do
         true  -> acc <> kw_block_to_string(x, Keyword.get(kw, x), fun)
         false -> acc
@@ -920,22 +1013,22 @@ defmodule Macro do
     end)
   end
 
-  defp parenthise(expr, fun) do
+  defp wrap_in_parenthesis(expr, fun) do
     "(" <> to_string(expr, fun) <> ")"
   end
 
-  defp op_to_string({op, _, [_, _]} = expr, fun, parent_op, side) when op in unquote(@binary_ops) do
+  defp op_to_string({op, _, [_, _]} = expr, fun, parent_op, side) when op in unquote(binary_ops) do
     {parent_assoc, parent_prec} = binary_op_props(parent_op)
     {_, prec}                   = binary_op_props(op)
     cond do
       parent_prec < prec -> to_string(expr, fun)
-      parent_prec > prec -> parenthise(expr, fun)
+      parent_prec > prec -> wrap_in_parenthesis(expr, fun)
       true ->
         # parent_prec == prec, so look at associativity.
         if parent_assoc == side do
           to_string(expr, fun)
         else
-          parenthise(expr, fun)
+          wrap_in_parenthesis(expr, fun)
         end
     end
   end
@@ -1013,7 +1106,7 @@ defmodule Macro do
 
   That said, we need to expand the aliases node above to an
   atom, so we can retrieve its length. Expanding the node is
-  not straight-forward because we also need to expand the
+  not straightforward because we also need to expand the
   caller aliases. For example:
 
       alias MyHelpers, as: My
@@ -1202,31 +1295,28 @@ defmodule Macro do
     "Elixir." <> rest = Atom.to_string(atom)
     underscore(rest)
   end
-
-  def underscore(""), do: ""
-
   def underscore(<<h, t::binary>>) do
     <<to_lower_char(h)>> <> do_underscore(t, h)
   end
+  def underscore("") do
+    ""
+  end
+
 
   defp do_underscore(<<h, t, rest::binary>>, _)
-      when (h >= ?A and h <= ?Z) and not (t >= ?A and t <= ?Z) and t != ?. do
+      when (h >= ?A and h <= ?Z) and not (t >= ?A and t <= ?Z) and t != ?. and t != ?_ do
     <<?_, to_lower_char(h), t>> <> do_underscore(rest, t)
   end
-
   defp do_underscore(<<h, t::binary>>, prev)
-      when (h >= ?A and h <= ?Z) and not (prev >= ?A and prev <= ?Z) do
+      when (h >= ?A and h <= ?Z) and not (prev >= ?A and prev <= ?Z) and prev != ?_ do
     <<?_, to_lower_char(h)>> <> do_underscore(t, h)
   end
-
   defp do_underscore(<<?., t::binary>>, _) do
     <<?/>> <> underscore(t)
   end
-
   defp do_underscore(<<h, t::binary>>, _) do
     <<to_lower_char(h)>> <> do_underscore(t, h)
   end
-
   defp do_underscore(<<>>, _) do
     <<>>
   end
@@ -1244,37 +1334,37 @@ defmodule Macro do
       iex> Macro.camelize "foo_bar"
       "FooBar"
 
+  If uppercase characters are present, they are not modified in anyway
+  as a mechanism to preserve acronyms:
+
+      iex> Macro.camelize "API.V1"
+      "API.V1"
+      iex> Macro.camelize "API_SPEC"
+      "API_SPEC"
+
   """
   @spec camelize(String.t) :: String.t
   def camelize(string)
 
   def camelize(""),
     do: ""
-
   def camelize(<<?_, t::binary>>),
     do: camelize(t)
-
   def camelize(<<h, t::binary>>),
     do: <<to_upper_char(h)>> <> do_camelize(t)
 
   defp do_camelize(<<?_, ?_, t::binary>>),
     do: do_camelize(<<?_, t::binary >>)
-
   defp do_camelize(<<?_, h, t::binary>>) when h >= ?a and h <= ?z,
     do: <<to_upper_char(h)>> <> do_camelize(t)
-
   defp do_camelize(<<?_, h, t::binary>>) when h >= ?0 and h <= ?9,
     do: <<h>> <> do_camelize(t)
-
   defp do_camelize(<<?_>>),
     do: <<>>
-
   defp do_camelize(<<?/, t::binary>>),
     do: <<?.>> <> camelize(t)
-
   defp do_camelize(<<h, t::binary>>),
     do: <<h>> <> do_camelize(t)
-
   defp do_camelize(<<>>),
     do: <<>>
 

lib/elixir/lib/macro/env.ex

@@ -36,11 +36,20 @@ defmodule Macro.Env do
     * `macros` - a list of macros imported from each module
     * `macro_aliases` - a list of aliases defined inside the current macro
     * `context_modules` - a list of modules defined in the current context
-    * `vars` - a list keeping all defined variables as `{var, context}`
-    * `export_vars` - a list keeping all variables to be exported in a
-      construct (may be `nil`)
     * `lexical_tracker` - PID of the lexical tracker which is responsible for
       keeping user info
+    * `vars` - a list keeping all defined variables as `{var, context}`
+
+  The following fields are private and must not be accessed or relied on:
+
+    * `export_vars` - a list keeping all variables to be exported in a
+      construct (may be `nil`)
+    * `match_vars` - controls how "new" variables are handled. Inside a
+      match it is a list with all variables in a match. Outside of a match
+      is either `:warn` or `:apply`
+    * `prematch_vars` - a list of variables defined before a match (is
+      `nil` when not inside a match)
+
   """
 
   @type name_arity :: {atom, arity}
@@ -54,10 +63,13 @@ defmodule Macro.Env do
   @type macros :: [{module, [name_arity]}]
   @type context_modules :: [module]
   @type vars :: [{atom, atom | non_neg_integer}]
-  @type export_vars :: vars | nil
-  @type lexical_tracker :: pid
+  @type lexical_tracker :: pid | nil
   @type local :: atom | nil
 
+  @opaque export_vars :: vars | nil
+  @opaque match_vars :: vars | :warn | :apply
+  @opaque prematch_vars :: vars | nil
+
   @type t :: %{__struct__: __MODULE__,
                module: atom,
                file: file,
@@ -72,6 +84,8 @@ defmodule Macro.Env do
                context_modules: context_modules,
                vars: vars,
                export_vars: export_vars,
+               match_vars: match_vars,
+               prematch_vars: prematch_vars,
                lexical_tracker: lexical_tracker}
 
   def __struct__ do
@@ -88,8 +102,10 @@ defmodule Macro.Env do
       macro_aliases: [],
       context_modules: [],
       vars: [],
+      lexical_tracker: nil,
       export_vars: nil,
-      lexical_tracker: nil}
+      match_vars: :warn,
+      prematch_vars: nil}
   end
 
   def __struct__(kv) do
@@ -100,7 +116,7 @@ defmodule Macro.Env do
   Returns a keyword list containing the file and line
   information as keys.
   """
-  @spec location(t) :: Keyword.t
+  @spec location(t) :: keyword
   def location(env)
   def location(%{__struct__: Macro.Env, file: file, line: line}) do
     [file: file, line: line]

lib/elixir/lib/map.ex

@@ -14,7 +14,7 @@ defmodule Map do
   in the example above has a different order than the map that was created).
 
   Maps do not impose any restriction on the key type: anything can be a key in a
-  map. As a key-value structure, maps do not allow duplicated keys; keys are
+  map. As a key-value structure, maps do not allow duplicated keys. Keys are
   compared using the exact-equality operator (`===`). If colliding keys are defined
   in a map literal, the last one prevails.
 
@@ -39,7 +39,7 @@ defmodule Map do
 
   The alternative access syntax `map.key` is provided alongside `[]` when the
   map has a `:key` key; note that while `map[key]` will return `nil` if `map`
-  doesn't contain the key `key`, `map.key` will raise if `map` doesn't contain
+  doesn't contain `key`, `map.key` will raise if `map` doesn't contain
   the key `:key`.
 
       iex> map = %{foo: "bar", baz: "bong"}
@@ -94,7 +94,7 @@ defmodule Map do
 
   @type key :: any
   @type value :: any
-  @compile {:inline, fetch: 2, put: 3, delete: 2, has_key?: 2}
+  @compile {:inline, fetch: 2, fetch!: 2, get: 2, put: 3, delete: 2, has_key?: 2, replace!: 3}
 
   @doc """
   Returns all keys from `map`.
@@ -164,6 +164,7 @@ defmodule Map do
   """
   @spec new(Enumerable.t) :: map
   def new(enumerable)
+  def new(list) when is_list(list), do: :maps.from_list(list)
   def new(%{__struct__: _} = struct), do: new_from_enum(struct)
   def new(%{} = map), do: map
   def new(enum), do: new_from_enum(enum)
@@ -212,6 +213,7 @@ defmodule Map do
       iex> Map.has_key?(%{a: 1}, :b)
       false
 
+  Inlined by the compiler.
   """
   @spec has_key?(map, key) :: boolean
   def has_key?(map, key), do: :maps.is_key(key, map)
@@ -229,6 +231,7 @@ defmodule Map do
       iex> Map.fetch(%{a: 1}, :b)
       :error
 
+  Inlined by the compiler.
   """
   @spec fetch(map, key) :: {:ok, value} | :error
   def fetch(map, key), do: :maps.find(key, map)
@@ -250,10 +253,7 @@ defmodule Map do
   """
   @spec fetch!(map, key) :: value | no_return
   def fetch!(map, key) do
-    case fetch(map, key) do
-      {:ok, value} -> value
-      :error -> raise KeyError, key: key, term: map
-    end
+    :maps.get(key, map)
   end
 
   @doc """
@@ -263,19 +263,65 @@ defmodule Map do
   ## Examples
 
       iex> Map.put_new(%{a: 1}, :b, 2)
-      %{b: 2, a: 1}
+      %{a: 1, b: 2}
       iex> Map.put_new(%{a: 1, b: 2}, :a, 3)
       %{a: 1, b: 2}
 
   """
   @spec put_new(map, key, value) :: map
   def put_new(map, key, value) do
-    case has_key?(map, key) do
-      true  -> map
-      false -> put(map, key, value)
+    case map do
+      %{^key => _value} ->
+        map
+      %{} ->
+        put(map, key, value)
+      other ->
+        :erlang.error({:badmap, other})
     end
   end
 
+  @doc """
+  Alters the value stored under `key` to `value`, but only
+  if the entry `key` already exists in `map`.
+
+  ## Examples
+
+      iex> Map.replace(%{a: 1}, :b, 2)
+      %{a: 1}
+      iex> Map.replace(%{a: 1, b: 2}, :a, 3)
+      %{a: 3, b: 2}
+
+  """
+  @spec replace(map, key, value) :: map
+  def replace(map, key, value) do
+    case map do
+      %{^key => _value} ->
+        put(map, key, value)
+      %{} ->
+        map
+      other ->
+        :erlang.error({:badmap, other})
+    end
+  end
+
+  @doc """
+  Similar to `replace/3`, but will raise a `KeyError`
+  if the key does not exist in the map.
+
+  ## Examples
+
+      iex> Map.replace!(%{a: 1, b: 2}, :a, 3)
+      %{a: 3, b: 2}
+      iex> Map.replace!(%{a: 1}, :b, 2)
+      ** (KeyError) key :b not found in: %{a: 1}
+
+  Inlined by the compiler.
+  """
+  @spec replace!(map, key, value) :: map
+  def replace!(map, key, value) do
+    :maps.update(key, value, map)
+  end
+
   @doc """
   Evaluates `fun` and puts the result under `key`
   in `map` unless `key` is already present.
@@ -299,9 +345,13 @@ defmodule Map do
   """
   @spec put_new_lazy(map, key, (() -> value)) :: map
   def put_new_lazy(map, key, fun) when is_function(fun, 0) do
-    case has_key?(map, key) do
-      true  -> map
-      false -> put(map, key, fun.())
+    case map do
+      %{^key => _value} ->
+        map
+      %{} ->
+        put(map, key, fun.())
+      other ->
+        :erlang.error({:badmap, other})
     end
   end
 
@@ -323,20 +373,25 @@ defmodule Map do
   def take(map, keys) when is_map(map) do
     keys
     |> Enum.to_list
-    |> do_take(map, [])
+    |> take(map, [])
   end
 
   def take(non_map, _keys) do
     :erlang.error({:badmap, non_map})
   end
 
-  defp do_take([], _map, acc), do: :maps.from_list(acc)
-  defp do_take([key | rest], map, acc) do
-    acc = case fetch(map, key) do
-      {:ok, value} -> [{key, value} | acc]
-      :error -> acc
-    end
-    do_take(rest, map, acc)
+  defp take([], _map, acc) do
+    :maps.from_list(acc)
+  end
+
+  defp take([key | rest], map, acc) do
+    acc =
+      case map do
+        %{^key => value} -> [{key, value} | acc]
+        %{} -> acc
+      end
+
+    take(rest, map, acc)
   end
 
   @doc """
@@ -358,12 +413,15 @@ defmodule Map do
       3
 
   """
-  @spec get(map, key) :: value
   @spec get(map, key, value) :: value
   def get(map, key, default \\ nil) do
-    case fetch(map, key) do
-      {:ok, value} -> value
-      :error -> default
+    case map do
+      %{^key => value} ->
+        value
+      %{} ->
+        default
+      other ->
+        :erlang.error({:badmap, other}, [map, key, default])
     end
   end
 
@@ -391,9 +449,13 @@ defmodule Map do
   """
   @spec get_lazy(map, key, (() -> value)) :: value
   def get_lazy(map, key, fun) when is_function(fun, 0) do
-    case fetch(map, key) do
-      {:ok, value} -> value
-      :error -> fun.()
+    case map do
+      %{^key => value} ->
+        value
+      %{} ->
+        fun.()
+      other ->
+        :erlang.error({:badmap, other}, [map, key, fun])
     end
   end
 
@@ -407,6 +469,7 @@ defmodule Map do
       iex> Map.put(%{a: 1, b: 2}, :a, 3)
       %{a: 3, b: 2}
 
+  Inlined by the compiler.
   """
   @spec put(map, key, value) :: map
   def put(map, key, value) do
@@ -425,6 +488,7 @@ defmodule Map do
       iex> Map.delete(%{b: 2}, :a)
       %{b: 2}
 
+  Inlined by the compiler.
   """
   @spec delete(map, key) :: map
   def delete(map, key), do: :maps.remove(key, map)
@@ -468,9 +532,15 @@ defmodule Map do
   """
   @spec merge(map, map, (key, value, value -> value)) :: map
   def merge(map1, map2, callback) when is_function(callback, 3) do
-    :maps.fold fn k, v2, acc ->
-      update(acc, k, v2, fn(v1) -> callback.(k, v1, v2) end)
-    end, map1, map2
+    if map_size(map1) > map_size(map2) do
+      :maps.fold fn key, val2, acc ->
+        update(acc, key, val2, fn val1 -> callback.(key, val1, val2) end)
+      end, map1, map2
+    else
+      :maps.fold fn key, val2, acc ->
+        update(acc, key, val2, fn val1 -> callback.(key, val2, val1) end)
+      end, map2, map1
+    end
   end
 
   @doc """
@@ -478,7 +548,8 @@ defmodule Map do
 
   If `key` is present in `map` with value `value`, `fun` is invoked with
   argument `value` and its result is used as the new value of `key`. If `key` is
-  not present in `map`, `initial` is inserted as the value of `key`.
+  not present in `map`, `initial` is inserted as the value of `key`. The initial
+  value will not be passed through the update function.
 
   ## Examples
 
@@ -490,11 +561,13 @@ defmodule Map do
   """
   @spec update(map, key, value, (value -> value)) :: map
   def update(map, key, initial, fun) when is_function(fun, 1) do
-    case fetch(map, key) do
-      {:ok, value} ->
+    case map do
+      %{^key => value} ->
         put(map, key, fun.(value))
-      :error ->
+      %{} ->
         put(map, key, initial)
+      other ->
+        :erlang.error({:badmap, other}, [map, key, initial, fun])
     end
   end
 
@@ -518,8 +591,12 @@ defmodule Map do
   @spec pop(map, key, value) :: {value, map}
   def pop(map, key, default \\ nil) do
     case map do
-      %{^key => value} -> {value, delete(map, key)}
-      %{} -> {default, map}
+      %{^key => value} ->
+        {value, delete(map, key)}
+      %{} ->
+        {default, map}
+      other ->
+        :erlang.error({:badmap, other}, [map, key, default])
     end
   end
 
@@ -549,9 +626,13 @@ defmodule Map do
   """
   @spec pop_lazy(map, key, (() -> value)) :: {value, map}
   def pop_lazy(map, key, fun) when is_function(fun, 0) do
-    case fetch(map, key) do
-      {:ok, value} -> {value, delete(map, key)}
-      :error -> {fun.(), map}
+    case map do
+      %{^key => value} ->
+        {value, delete(map, key)}
+      %{} ->
+        {fun.(), map}
+      other ->
+        :erlang.error({:badmap, other}, [map, key, fun])
     end
   end
 
@@ -575,8 +656,8 @@ defmodule Map do
     |> drop_list(map)
   end
 
-  def drop(non_map, _keys) do
-    :erlang.error({:badmap, non_map})
+  def drop(non_map, keys) do
+    :erlang.error({:badmap, non_map}, [non_map, keys])
   end
 
   defp drop_list([], acc), do: acc
@@ -585,7 +666,7 @@ defmodule Map do
   end
 
   @doc """
-  Takes all entries corresponding to the given `keys` in `maps` and extracts
+  Takes all entries corresponding to the given `keys` in `map` and extracts
   them into a separate map.
 
   Returns a tuple with the new map and the old map with removed keys.
@@ -604,22 +685,23 @@ defmodule Map do
   def split(map, keys) when is_map(map) do
     keys
     |> Enum.to_list
-    |> do_split([], map)
+    |> split([], map)
   end
 
-  def split(non_map, _keys) do
-    :erlang.error({:badmap, non_map})
+  def split(non_map, keys) do
+    :erlang.error({:badmap, non_map}, [non_map, keys])
   end
 
-  defp do_split([], inc, exc) do
-    {:maps.from_list(inc), exc}
+  defp split([], included, excluded) do
+    {:maps.from_list(included), excluded}
   end
-  defp do_split([key | rest], inc, exc) do
-    case fetch(exc, key) do
-      {:ok, value} ->
-        do_split(rest, [{key, value} | inc], delete(exc, key))
-      :error ->
-        do_split(rest, inc, exc)
+
+  defp split([key | rest], included, excluded) do
+    case excluded do
+      %{^key => value} ->
+        split(rest, [{key, value} | included], delete(excluded, key))
+      _other ->
+        split(rest, included, excluded)
     end
   end
 
@@ -639,18 +721,12 @@ defmodule Map do
       ** (KeyError) key :b not found in: %{a: 1}
 
   """
-  @spec update!(map, key, (value -> value)) :: map | no_return
-  def update!(%{} = map, key, fun) when is_function(fun, 1) do
-    case fetch(map, key) do
-      {:ok, value} ->
-        put(map, key, fun.(value))
-      :error ->
-        raise KeyError, term: map, key: key
-    end
+  @spec update!(map, key, (value -> value)) :: map
+  def update!(map, key, fun) when is_function(fun, 1) do
+    value = fetch!(map, key)
+    put(map, key, fun.(value))
   end
 
-  def update!(map, _key, _fun), do: :erlang.error({:badmap, map})
-
   @doc """
   Gets the value from `key` and updates it, all in one pass.
 
@@ -684,25 +760,19 @@ defmodule Map do
 
   """
   @spec get_and_update(map, key, (value -> {get, value} | :pop)) :: {get, map} when get: term
-  def get_and_update(%{} = map, key, fun) when is_function(fun, 1) do
-    current =
-      case :maps.find(key, map) do
-        {:ok, value} -> value
-        :error -> nil
-      end
+  def get_and_update(map, key, fun) when is_function(fun, 1) do
+    current = get(map, key)
 
     case fun.(current) do
       {get, update} ->
-        {get, :maps.put(key, update, map)}
+        {get, put(map, key, update)}
       :pop ->
-        {current, :maps.remove(key, map)}
+        {current, delete(map, key)}
       other ->
         raise "the given function must return a two-element tuple or :pop, got: #{inspect(other)}"
     end
   end
 
-  def get_and_update(map, _key, _fun), do: :erlang.error({:badmap, map})
-
   @doc """
   Gets the value from `key` and updates it. Raises if there is no `key`.
 
@@ -728,24 +798,19 @@ defmodule Map do
 
   """
   @spec get_and_update!(map, key, (value -> {get, value})) :: {get, map} | no_return when get: term
-  def get_and_update!(%{} = map, key, fun) when is_function(fun, 1) do
-    case :maps.find(key, map) do
-      {:ok, value} ->
-        case fun.(value) do
-          {get, update} ->
-            {get, :maps.put(key, update, map)}
-          :pop ->
-            {value, :maps.remove(key, map)}
-          other ->
-            raise "the given function must return a two-element tuple or :pop, got: #{inspect(other)}"
-        end
-      :error ->
-        raise KeyError, term: map, key: key
+  def get_and_update!(map, key, fun) when is_function(fun, 1) do
+    value = fetch!(map, key)
+
+    case fun.(value) do
+      {get, update} ->
+        {get, put(map, key, update)}
+      :pop ->
+        {value, delete(map, key)}
+      other ->
+        raise "the given function must return a two-element tuple or :pop, got: #{inspect(other)}"
     end
   end
 
-  def get_and_update!(map, _key, _fun), do: :erlang.error({:badmap, map})
-
   @doc """
   Converts a `struct` to map.
 
@@ -768,11 +833,11 @@ defmodule Map do
   """
   @spec from_struct(atom | struct) :: map
   def from_struct(struct) when is_atom(struct) do
-    :maps.remove(:__struct__, struct.__struct__)
+    delete(struct.__struct__(), :__struct__)
   end
 
-  def from_struct(%{__struct__: _} = struct) do
-    :maps.remove(:__struct__, struct)
+  def from_struct(%_{} = struct) do
+    delete(struct, :__struct__)
   end
 
   @doc """
@@ -790,12 +855,16 @@ defmodule Map do
 
   """
   @spec equal?(map, map) :: boolean
+  def equal?(map1, map2)
+
   def equal?(%{} = map1, %{} = map2), do: map1 === map2
+  def equal?(%{} = map1, map2), do: :erlang.error({:badmap, map2}, [map1, map2])
+  def equal?(term, other), do: :erlang.error({:badmap, term}, [term, other])
 
   @doc false
   # TODO: Remove on 2.0
+  # (hard-deprecated in elixir_dispatch)
   def size(map) do
-    IO.warn "Map.size/1 is deprecated, please use Kernel.map_size/1"
     map_size(map)
   end
 end

lib/elixir/lib/map_set.ex

@@ -8,15 +8,15 @@ defmodule MapSet do
       iex> MapSet.new
       #MapSet<[]>
 
-  A set can contain any kind of elements and elements in a set don't have to be
+  A set can contain any kind of elements, and elements in a set don't have to be
   of the same type. By definition, sets can't contain duplicate elements: when
   inserting an element in a set where it's already present, the insertion is
   simply a no-op.
 
-      iex> set = MapSet.new
-      iex> MapSet.put(set, "foo")
+      iex> map_set = MapSet.new
+      iex> MapSet.put(map_set, "foo")
       #MapSet<["foo"]>
-      iex> set |> MapSet.put("foo") |> MapSet.put("foo")
+      iex> map_set |> MapSet.put("foo") |> MapSet.put("foo")
       #MapSet<["foo"]>
 
   A `MapSet` is represented internally using the `%MapSet{}` struct. This struct
@@ -28,16 +28,16 @@ defmodule MapSet do
   Note that, however, the struct fields are private and must not be accessed
   directly; use the functions in this module to perform operations on sets.
 
-  Sets can also be constructed starting from other collection-type data
+  `MapSet`s can also be constructed starting from other collection-type data
   structures: for example, see `MapSet.new/1` or `Enum.into/2`.
   """
 
   @type value :: term
 
-  @opaque t(value) :: %__MODULE__{map: %{optional(value) => true}}
+  @opaque t(value) :: %__MODULE__{map: %{optional(value) => []}}
   @type t :: t(term)
 
-  defstruct map: %{}
+  defstruct map: %{}, version: 2
 
   @doc """
   Returns a new set.
@@ -63,18 +63,20 @@ defmodule MapSet do
 
   """
   @spec new(Enum.t) :: t
-  def new(%__MODULE__{} = mapset), do: mapset
+  def new(enumerable)
+
+  def new(%__MODULE__{} = map_set), do: map_set
   def new(enumerable) do
     map =
       enumerable
       |> Enum.to_list
-      |> do_new([])
+      |> new_from_list([])
 
     %MapSet{map: map}
   end
 
   @doc """
-  Creates a mapset from an enumerable via the transformation function.
+  Creates a set from an enumerable via the transformation function.
 
   ## Examples
 
@@ -87,46 +89,47 @@ defmodule MapSet do
     map =
       enumerable
       |> Enum.to_list
-      |> do_new_transform(transform, [])
+      |> new_from_list_transform(transform, [])
 
     %MapSet{map: map}
   end
 
-  defp do_new([], acc) do
+  defp new_from_list([], acc) do
     :maps.from_list(acc)
   end
-  defp do_new([item | rest], acc) do
-    do_new(rest, [{item, true} | acc])
-  end
 
-  defp do_new_transform([], _fun, acc) do
+  defp new_from_list([item | rest], acc) do
+    new_from_list(rest, [{item, []} | acc])
+  end
+
+  defp new_from_list_transform([], _fun, acc) do
     :maps.from_list(acc)
   end
-  defp do_new_transform([item | rest], fun, acc) do
-    do_new_transform(rest, fun, [{fun.(item), true} | acc])
+  defp new_from_list_transform([item | rest], fun, acc) do
+    new_from_list_transform(rest, fun, [{fun.(item), []} | acc])
   end
 
   @doc """
-  Deletes `value` from `set`.
+  Deletes `value` from `map_set`.
 
-  Returns a new set which is a copy of `set` but without `value`.
+  Returns a new set which is a copy of `map_set` but without `value`.
 
   ## Examples
 
-      iex> set = MapSet.new([1, 2, 3])
-      iex> MapSet.delete(set, 4)
+      iex> map_set = MapSet.new([1, 2, 3])
+      iex> MapSet.delete(map_set, 4)
       #MapSet<[1, 2, 3]>
-      iex> MapSet.delete(set, 2)
+      iex> MapSet.delete(map_set, 2)
       #MapSet<[1, 3]>
 
   """
   @spec delete(t(val1), val2) :: t(val1) when val1: value, val2: value
-  def delete(%MapSet{map: map} = set, value) do
-    %{set | map: Map.delete(map, value)}
+  def delete(%MapSet{map: map} = map_set, value) do
+    %{map_set | map: Map.delete(map, value)}
   end
 
   @doc """
-  Returns a set that is `set1` without the members of `set2`.
+  Returns a set that is `map_set1` without the members of `map_set2`.
 
   ## Examples
 
@@ -135,16 +138,17 @@ defmodule MapSet do
 
   """
   @spec difference(t(val1), t(val2)) :: t(val1) when val1: value, val2: value
-  def difference(mapset1, mapset2)
+  def difference(map_set1, map_set2)
 
   # If the first set is less than twice the size of the second map,
   # it is fastest to re-accumulate items in the first set that are not
   # present in the second set.
   def difference(%MapSet{map: map1}, %MapSet{map: map2})
       when map_size(map1) < map_size(map2) * 2 do
-    map = map1
-    |> Map.keys
-    |> filter_not_in(map2)
+    map =
+      map1
+      |> Map.keys
+      |> filter_not_in(map2)
 
     %MapSet{map: map}
   end
@@ -152,23 +156,24 @@ defmodule MapSet do
   # If the second set is less than half the size of the first set, it's fastest
   # to simply iterate through each item in the second set, deleting them from
   # the first set.
-  def difference(%MapSet{map: map1}, %MapSet{map: map2}) do
-    %MapSet{map: Map.drop(map1, Map.keys(map2))}
+  def difference(%MapSet{map: map1} = map_set, %MapSet{map: map2}) do
+    %{map_set | map: Map.drop(map1, Map.keys(map2))}
   end
 
   defp filter_not_in(keys, map2, acc \\ [])
   defp filter_not_in([], _map2, acc), do: :maps.from_list(acc)
   defp filter_not_in([key | rest], map2, acc) do
-    acc = if Map.has_key?(map2, key) do
-      acc
-    else
-      [{key, true} | acc]
-    end
+    acc =
+      if Map.has_key?(map2, key) do
+        acc
+      else
+        [{key, []} | acc]
+      end
     filter_not_in(rest, map2, acc)
   end
 
   @doc """
-  Checks if `set1` and `set2` have no members in common.
+  Checks if `map_set1` and `map_set2` have no members in common.
 
   ## Examples
 
@@ -211,12 +216,18 @@ defmodule MapSet do
 
   """
   @spec equal?(t, t) :: boolean
-  def equal?(%MapSet{map: map1}, %MapSet{map: map2}) do
+  def equal?(%MapSet{map: map1, version: version}, %MapSet{map: map2, version: version}) do
     Map.equal?(map1, map2)
   end
 
+  # Elixir v1.5 change the map representation, so on
+  # version mismatch we need to compare the keys directly.
+  def equal?(%MapSet{map: map1}, %MapSet{map: map2}) do
+    map_size(map1) == map_size(map2) and map_subset?(Map.keys(map1), map2)
+  end
+
   @doc """
-  Returns a set containing only members that `set1` and `set2` have in common.
+  Returns a set containing only members that `map_set1` and `map_set2` have in common.
 
   ## Examples
 
@@ -228,14 +239,13 @@ defmodule MapSet do
 
   """
   @spec intersection(t(val), t(val)) :: t(val) when val: value
-  def intersection(%MapSet{map: map1}, %MapSet{map: map2}) do
+  def intersection(%MapSet{map: map1} = map_set, %MapSet{map: map2}) do
     {map1, map2} = order_by_size(map1, map2)
-
-    %MapSet{map: Map.take(map2, Map.keys(map1))}
+    %{map_set | map: Map.take(map2, Map.keys(map1))}
   end
 
   @doc """
-  Checks if `set` contains `value`.
+  Checks if `map_set` contains `value`.
 
   ## Examples
 
@@ -251,7 +261,7 @@ defmodule MapSet do
   end
 
   @doc """
-  Inserts `value` into `set` if `set` doesn't already contain it.
+  Inserts `value` into `map_set` if `map_set` doesn't already contain it.
 
   ## Examples
 
@@ -262,12 +272,12 @@ defmodule MapSet do
 
   """
   @spec put(t(val), new_val) :: t(val | new_val) when val: value, new_val: value
-  def put(%MapSet{map: map} = set, value) do
-    %{set | map: Map.put(map, value, true)}
+  def put(%MapSet{map: map} = map_set, value) do
+    %{map_set | map: Map.put(map, value, [])}
   end
 
   @doc """
-  Returns the number of elements in `set`.
+  Returns the number of elements in `map_set`.
 
   ## Examples
 
@@ -281,9 +291,9 @@ defmodule MapSet do
   end
 
   @doc """
-  Checks if `set1`'s members are all contained in `set2`.
+  Checks if `map_set1`'s members are all contained in `map_set2`.
 
-  This function checks if `set1` is a subset of `set2`.
+  This function checks if `map_set1` is a subset of `map_set2`.
 
   ## Examples
 
@@ -298,23 +308,23 @@ defmodule MapSet do
     if map_size(map1) <= map_size(map2) do
       map1
       |> Map.keys
-      |> do_subset?(map2)
+      |> map_subset?(map2)
     else
       false
     end
   end
 
-  defp do_subset?([], _), do: true
-  defp do_subset?([key | rest], map2) do
+  defp map_subset?([], _), do: true
+  defp map_subset?([key | rest], map2) do
     if Map.has_key?(map2, key) do
-      do_subset?(rest, map2)
+      map_subset?(rest, map2)
     else
       false
     end
   end
 
   @doc """
-  Converts `set` to a list.
+  Converts `map_set` to a list.
 
   ## Examples
 
@@ -328,7 +338,7 @@ defmodule MapSet do
   end
 
   @doc """
-  Returns a set containing all members of `set1` and `set2`.
+  Returns a set containing all members of `map_set1` and `map_set2`.
 
   ## Examples
 
@@ -337,24 +347,30 @@ defmodule MapSet do
 
   """
   @spec union(t(val1), t(val2)) :: t(val1 | val2) when val1: value, val2: value
+  def union(map_set1, map_set2)
+
+  def union(%MapSet{map: map1, version: version} = map_set, %MapSet{map: map2, version: version}) do
+    %{map_set | map: Map.merge(map1, map2)}
+  end
   def union(%MapSet{map: map1}, %MapSet{map: map2}) do
-    %MapSet{map: Map.merge(map1, map2)}
+    map = new_from_list(Map.keys(map1) ++ Map.keys(map2), [])
+    %MapSet{map: map}
   end
 
   defp order_by_size(map1, map2) when map_size(map1) > map_size(map2), do: {map2, map1}
   defp order_by_size(map1, map2), do: {map1, map2}
 
   defimpl Enumerable do
-    def reduce(set, acc, fun), do: Enumerable.List.reduce(MapSet.to_list(set), acc, fun)
-    def member?(set, val),     do: {:ok, MapSet.member?(set, val)}
-    def count(set),            do: {:ok, MapSet.size(set)}
+    def reduce(map_set, acc, fun), do: Enumerable.List.reduce(MapSet.to_list(map_set), acc, fun)
+    def member?(map_set, val), do: {:ok, MapSet.member?(map_set, val)}
+    def count(map_set), do: {:ok, MapSet.size(map_set)}
   end
 
   defimpl Collectable do
     def into(original) do
       {original, fn
-        set, {:cont, x} -> MapSet.put(set, x)
-        set, :done -> set
+        map_set, {:cont, x} -> MapSet.put(map_set, x)
+        map_set, :done -> map_set
         _, :halt -> :ok
       end}
     end
@@ -363,8 +379,8 @@ defmodule MapSet do
   defimpl Inspect do
     import Inspect.Algebra
 
-    def inspect(set, opts) do
-      concat ["#MapSet<", Inspect.List.inspect(MapSet.to_list(set), opts), ">"]
+    def inspect(map_set, opts) do
+      concat ["#MapSet<", Inspect.List.inspect(MapSet.to_list(map_set), opts), ">"]
     end
   end
 end

lib/elixir/lib/module/locals_tracker.ex

@@ -65,11 +65,15 @@ defmodule Module.LocalsTracker do
   """
   @spec reachable(ref) :: [local]
   def reachable(ref) do
-    reachable_from(:gen_server.call(to_pid(ref), :digraph, @timeout), :local)
+    ref
+    |> to_pid()
+    |> :gen_server.call(:digraph, @timeout)
+    |> reachable_from(:local)
+    |> :sets.to_list()
   end
 
   defp reachable_from(d, starting) do
-    :sets.to_list(reduce_reachable(d, starting, :sets.new))
+    reduce_reachable(d, starting, :sets.new)
   end
 
   defp reduce_reachable(d, vertex, vertices) do
@@ -134,8 +138,8 @@ defmodule Module.LocalsTracker do
 
   # Reattach a previously yanked node
   @doc false
-  def reattach(pid, kind, tuple, neighbours) do
-    :gen_server.cast(to_pid(pid), {:reattach, kind, tuple, neighbours})
+  def reattach(pid, tuple, kind, function, neighbours) do
+    :gen_server.cast(to_pid(pid), {:reattach, tuple, kind, function, neighbours})
   end
 
   # Collecting all conflicting imports with the given functions
@@ -143,11 +147,11 @@ defmodule Module.LocalsTracker do
   def collect_imports_conflicts(pid, all_defined) do
     d = :gen_server.call(pid, :digraph, @timeout)
 
-    for {name, arity} <- all_defined,
+    for {{name, arity}, _, meta, _} <- all_defined,
         :digraph.in_neighbours(d, {:import, name, arity}) != [],
         n = :digraph.out_neighbours(d, {:import, name, arity}),
         n != [] do
-      {n, name, arity}
+      {meta, {n, name, arity}}
     end
   end
 
@@ -157,73 +161,63 @@ defmodule Module.LocalsTracker do
   @doc false
   def collect_unused_locals(ref, private) do
     d = :gen_server.call(to_pid(ref), :digraph, @timeout)
-    {unreachable(d, private), collect_warnings(d, private)}
-  end
-
-  defp unreachable(d, private) do
-    unreachable = for {tuple, _, _} <- private, do: tuple
-
-    private =
-      for {tuple, :defp, _} <- private do
-        neighbours = :digraph.in_neighbours(d, tuple)
-        neighbours = for {_, _} = t <- neighbours, do: t
-        {tuple, :sets.from_list(neighbours)}
-      end
-
-    reduce_unreachable(private, [], :sets.from_list(unreachable))
-  end
-
-  defp reduce_unreachable([{vertex, callers} | t], acc, unreachable) do
-    if :sets.is_subset(callers, unreachable) do
-      reduce_unreachable(t, [{vertex, callers} | acc], unreachable)
-    else
-      reduce_unreachable(acc ++ t, [], :sets.del_element(vertex, unreachable))
-    end
-  end
-
-  defp reduce_unreachable([], _acc, unreachable) do
-    :sets.to_list(unreachable)
-  end
-
-  defp collect_warnings(d, private) do
     reachable = reachable_from(d, :local)
+    reattached = :digraph.out_neighbours(d, :reattach)
+    {unreachable(reachable, reattached, private), collect_warnings(reachable, private)}
+  end
+
+  defp unreachable(reachable, reattached, private) do
+    for {tuple, kind, _, _} <- private,
+        not reachable?(tuple, kind, reachable, reattached),
+        do: tuple
+  end
+
+  defp reachable?(tuple, :defmacrop, reachable, reattached) do
+    # All private micros are unreachable unless they have been
+    # reattached and they are reachable.
+    :lists.member(tuple, reattached) and :sets.is_element(tuple, reachable)
+  end
+  defp reachable?(tuple, :defp, reachable, _reattached) do
+    :sets.is_element(tuple, reachable)
+  end
+
+  defp collect_warnings(reachable, private) do
     :lists.foldl(&collect_warnings(&1, &2, reachable), [], private)
   end
 
-  defp collect_warnings({tuple, kind, 0}, acc, reachable) do
-    if :lists.member(tuple, reachable) do
+  defp collect_warnings({_, _, false, _}, acc, _reachable) do
+    acc
+  end
+
+  defp collect_warnings({tuple, kind, meta, 0}, acc, reachable) do
+    if :sets.is_element(tuple, reachable) do
       acc
     else
-      [{:unused_def, tuple, kind} | acc]
+      [{meta, {:unused_def, tuple, kind}} | acc]
     end
   end
 
-  defp collect_warnings({tuple, kind, default}, acc, reachable) when default > 0 do
+  defp collect_warnings({tuple, kind, meta, default}, acc, reachable) when default > 0 do
     {name, arity} = tuple
     min = arity - default
     max = arity
 
-    invoked = for {n, a} <- reachable, n == name, a in min..max, do: a
-
-    if invoked == [] do
-      [{:unused_def, tuple, kind} | acc]
-    else
-      case :lists.min(invoked) - min do
-        0 -> acc
-        ^default -> [{:unused_args, tuple} | acc]
-        unused_args -> [{:unused_args, tuple, unused_args} | acc]
-      end
+    case min_reachable_default(max, min, :none, name, reachable) do
+      :none -> [{meta, {:unused_def, tuple, kind}} | acc]
+      ^min -> acc
+      ^max -> [{meta, {:unused_args, tuple}} | acc]
+      diff -> [{meta, {:unused_args, tuple, diff}} | acc]
     end
   end
 
-  @doc false
-  def cache_env(pid, env) do
-    :gen_server.call(pid, {:cache_env, env}, @timeout)
+  defp min_reachable_default(max, min, last, name, reachable) when max >= min do
+    case :sets.is_element({name, max}, reachable) do
+      true -> min_reachable_default(max - 1, min, max, name, reachable)
+      false -> min_reachable_default(max - 1, min, last, name, reachable)
+    end
   end
-
-  @doc false
-  def get_cached_env(pid, ref) do
-    :gen_server.call(pid, {:get_cached_env, ref}, @timeout)
+  defp min_reachable_default(_max, _min, last, _name, _reachable) do
+    last
   end
 
   # Stops the gen server
@@ -237,79 +231,76 @@ defmodule Module.LocalsTracker do
   def init([]) do
     d = :digraph.new([:protected])
     :digraph.add_vertex(d, :local)
-    {:ok, {d, []}}
+    :digraph.add_vertex(d, :reattach)
+    {:ok, d}
   end
 
-  @doc false
-  def handle_call({:cache_env, env}, _from, {d, cache}) do
-    case cache do
-      [{i, ^env} | _] ->
-        {:reply, i, {d, cache}}
-      t ->
-        i = length(t)
-        {:reply, i, {d, [{i, env} | t]}}
-    end
-  end
-
-  def handle_call({:get_cached_env, ref}, _from, {_, cache} = state) do
-    {^ref, env} = :lists.keyfind(ref, 1, cache)
-    {:reply, env, state}
-  end
-
-  def handle_call({:yank, local}, _from, {d, _} = state) do
+  def handle_call({:yank, local}, _from, d) do
     out_vertices = :digraph.out_neighbours(d, local)
     :digraph.del_edges(d, :digraph.out_edges(d, local))
-    {:reply, {[], out_vertices}, state}
+    {:reply, {[], out_vertices}, d}
   end
 
-  def handle_call(:digraph, _from, {d, _} = state) do
-    {:reply, d, state}
+  def handle_call(:digraph, _from, d) do
+    {:reply, d, d}
   end
 
   @doc false
-  def handle_info(_msg, state) do
-    {:noreply, state}
+  def handle_info(_msg, d) do
+    {:noreply, d}
   end
 
-  def handle_cast({:add_local, from, to}, {d, _} = state) do
+  def handle_cast({:add_local, from, to}, d) do
     handle_add_local(d, from, to)
-    {:noreply, state}
+    {:noreply, d}
   end
 
-  def handle_cast({:add_import, function, module, {name, arity}}, {d, _} = state) do
+  def handle_cast({:add_import, function, module, {name, arity}}, d) do
     handle_import(d, function, module, name, arity)
-    {:noreply, state}
+    {:noreply, d}
   end
 
-  def handle_cast({:add_definition, kind, tuple}, {d, _} = state) do
+  def handle_cast({:add_definition, kind, tuple}, d) do
     handle_add_definition(d, kind, tuple)
-    {:noreply, state}
+    {:noreply, d}
   end
 
-  def handle_cast({:add_defaults, kind, {name, arity}, defaults}, {d, _} = state) do
+  def handle_cast({:add_defaults, kind, {name, arity}, defaults}, d) do
     for i <- :lists.seq(arity - defaults, arity - 1) do
       handle_add_definition(d, kind, {name, i})
-      handle_add_local(d, {name, i}, {name, i + 1})
+      handle_add_local(d, {name, i}, {name, arity})
     end
-    {:noreply, state}
+    {:noreply, d}
   end
 
-  def handle_cast({:reattach, _kind, tuple, {in_neigh, out_neigh}}, {d, _} = state) do
+  def handle_cast({:reattach, tuple, kind, function, {in_neigh, out_neigh}}, d) do
+    # Reattach the old function
     for from <- in_neigh do
       :digraph.add_vertex(d, from)
-      replace_edge!(d, from, tuple)
+      replace_edge!(d, from, function)
     end
 
     for to <- out_neigh do
       :digraph.add_vertex(d, to)
-      replace_edge!(d, tuple, to)
+      replace_edge!(d, function, to)
     end
 
-    {:noreply, state}
+    # Add the new definition
+    handle_add_definition(d, kind, tuple)
+
+    # Make a call from the old function to the new one
+    if function != tuple do
+      handle_add_local(d, function, tuple)
+    end
+
+    # Finally marked the new one as reattached
+    replace_edge!(d, :reattach, tuple)
+
+    {:noreply, d}
   end
 
-  def handle_cast(:stop, state) do
-    {:stop, :normal, state}
+  def handle_cast(:stop, d) do
+    {:stop, :normal, d}
   end
 
   @doc false

lib/elixir/lib/node.ex

@@ -89,6 +89,9 @@ defmodule Node do
 
   For more information, see
   [`:erlang.monitor_node/2`](http://www.erlang.org/doc/man/erlang.html#monitor_node-2).
+  
+  For monitoring status changes of all nodes, see
+  [`:net_kernel.monitor_nodes/3`](http://www.erlang.org/doc/man/net_kernel.html#monitor_nodes-2).
   """
   @spec monitor(t, boolean) :: true
   def monitor(node, flag) do
@@ -101,6 +104,9 @@ defmodule Node do
 
   For more information, see
   [`:erlang.monitor_node/3`](http://www.erlang.org/doc/man/erlang.html#monitor_node-3).
+  
+  For monitoring status changes of all nodes, see
+  [`:net_kernel.monitor_nodes/3`](http://www.erlang.org/doc/man/net_kernel.html#monitor_nodes-2).
   """
   @spec monitor(t, boolean, [:allow_passive_connect]) :: true
   def monitor(node, flag, options) do

lib/elixir/lib/option_parser.ex

@@ -4,9 +4,9 @@ defmodule OptionParser do
   """
 
   @type argv    :: [String.t]
-  @type parsed  :: Keyword.t
+  @type parsed  :: keyword
   @type errors  :: [{String.t, String.t | nil}]
-  @type options :: [switches: Keyword.t, strict: Keyword.t, aliases: Keyword.t]
+  @type options :: [switches: keyword, strict: keyword, aliases: keyword]
 
   defmodule ParseError do
     defexception [:message]
@@ -125,12 +125,12 @@ defmodule OptionParser do
   set to `true`.
 
   Since Elixir converts switches to atoms, the dynamic mode will only parse
-  switches that translates to atoms used by the runtime. Therefore, the code below
+  switches that translate to atoms used by the runtime. Therefore, the code below
   likely won't parse the given option since the `:option_parser_example` atom is
   never used anywhere:
 
       OptionParser.parse(["--option-parser-example"])
-      # Does nothing more...
+      # The :option_parser_example atom is not used anywhere below
 
   However, the code below does since the `:option_parser_example` atom is used
   at some point later (or earlier) on:
@@ -193,7 +193,7 @@ defmodule OptionParser do
   """
   @spec parse(argv, options) :: {parsed, argv, errors}
   def parse(argv, opts \\ []) when is_list(argv) and is_list(opts) do
-    do_parse(argv, compile_config(opts), [], [], [], true)
+    do_parse(argv, build_config(opts), [], [], [], true)
   end
 
   @doc """
@@ -252,7 +252,7 @@ defmodule OptionParser do
   """
   @spec parse_head(argv, options) :: {parsed, argv, errors}
   def parse_head(argv, opts \\ []) when is_list(argv) and is_list(opts) do
-    do_parse(argv, compile_config(opts), [], [], [], false)
+    do_parse(argv, build_config(opts), [], [], [], false)
   end
 
   @doc """
@@ -293,8 +293,8 @@ defmodule OptionParser do
     {Enum.reverse(opts), Enum.reverse(args), Enum.reverse(invalid)}
   end
 
-  defp do_parse(argv, {aliases, switches, strict?, allow_nonexistent_atoms?} = config, opts, args, invalid, all?) do
-    case next(argv, aliases, switches, strict?, allow_nonexistent_atoms?) do
+  defp do_parse(argv, %{switches: switches} = config, opts, args, invalid, all?) do
+    case next_with_config(argv, config) do
       {:ok, option, value, rest} ->
         # the option exists and it was successfully parsed
         kinds = List.wrap Keyword.get(switches, option)
@@ -348,35 +348,35 @@ defmodule OptionParser do
         {:error, argv}
 
   def next(argv, opts \\ []) when is_list(argv) and is_list(opts) do
-    {aliases, switches, strict?, allow_nonexistent_atoms?} = compile_config(opts)
-    next(argv, aliases, switches, strict?, allow_nonexistent_atoms?)
+    next_with_config(argv, build_config(opts))
   end
 
-  defp next([], _aliases, _switches, _strict?, _allow_nonexistent_atoms?) do
+  defp next_with_config([], _config) do
     {:error, []}
   end
 
-  defp next(["--" | _] = argv, _aliases, _switches, _strict?, _allow_nonexistent_atoms?) do
+  defp next_with_config(["--" | _] = argv, _config) do
     {:error, argv}
   end
 
-  defp next(["-" | _] = argv, _aliases, _switches, _strict?, _allow_nonexistent_atoms?) do
+  defp next_with_config(["-" | _] = argv, _config) do
     {:error, argv}
   end
 
-  defp next(["- " <> _ | _] = argv, _aliases, _switches, _strict?, _allow_nonexistent_atoms?) do
+  defp next_with_config(["- " <> _ | _] = argv, _config) do
     {:error, argv}
   end
 
   # Handles --foo or --foo=bar
-  defp next(["--" <> option | rest], _aliases, switches, strict?, allow_nonexistent_atoms?) do
+  defp next_with_config(["--" <> option | rest], config) do
     {option, value} = split_option(option)
-    tagged = tag_option(option, switches, allow_nonexistent_atoms?)
-    do_next(tagged, value, "--" <> option, rest, switches, strict?, allow_nonexistent_atoms?)
+    tagged = tag_option(option, config)
+    next_tagged(tagged, value, "--" <> option, rest, config)
   end
 
   # Handles -a, -abc, -abc=something
-  defp next(["-" <> option | rest] = argv, aliases, switches, strict?, allow_nonexistent_atoms?) do
+  defp next_with_config(["-" <> option | rest] = argv, config) do
+    %{aliases: aliases, allow_nonexistent_atoms?: allow_nonexistent_atoms?} = config
     {option, value} = split_option(option)
     original = "-" <> option
 
@@ -390,22 +390,22 @@ defmodule OptionParser do
         option_key = aliases[key]
         if key && option_key do
           IO.warn "multi-letter aliases are deprecated, got: #{inspect(key)}"
-          do_next({:default, option_key}, value, original, rest, switches, strict?, allow_nonexistent_atoms?)
+          next_tagged({:default, option_key}, value, original, rest, config)
         else
-          next(expand_multiletter_alias(option, value) ++ rest, aliases, switches, strict?, allow_nonexistent_atoms?)
+          next_with_config(expand_multiletter_alias(option, value) ++ rest, config)
         end
       true ->
         # We have a regular one-letter alias here
-        tagged = tag_oneletter_alias(option, aliases, allow_nonexistent_atoms?)
-        do_next(tagged, value, original, rest, switches, strict?, allow_nonexistent_atoms?)
+        tagged = tag_oneletter_alias(option, config)
+        next_tagged(tagged, value, original, rest, config)
     end
   end
 
-  defp next(argv, _aliases, _switches, _strict?, _allow_nonexistent_atoms?) do
+  defp next_with_config(argv, _config) do
     {:error, argv}
   end
 
-  defp do_next(tagged, value, original, rest, switches, strict?, allow_nonexistent_atoms?) do
+  defp next_tagged(tagged, value, original, rest, %{switches: switches, strict?: strict?}) do
     if strict? and not option_defined?(tagged, switches) do
       {:undefined, original, value, rest}
     else
@@ -485,7 +485,7 @@ defmodule OptionParser do
 
   """
   @spec split(String.t) :: argv
-  def split(string) do
+  def split(string) when is_binary(string) do
     do_split(String.trim_leading(string, " "), "", [], nil)
   end
 
@@ -528,10 +528,7 @@ defmodule OptionParser do
 
   ## Helpers
 
-  defp compile_config(opts) do
-    aliases = opts[:aliases] || []
-    allow_nonexistent_atoms? = opts[:allow_nonexistent_atoms] || false
-
+  defp build_config(opts) do
     {switches, strict?} = cond do
       opts[:switches] && opts[:strict] ->
         raise ArgumentError, ":switches and :strict cannot be given together"
@@ -543,7 +540,12 @@ defmodule OptionParser do
         {[], false}
     end
 
-    {aliases, switches, strict?, allow_nonexistent_atoms?}
+    %{
+      aliases: opts[:aliases] || [],
+      allow_nonexistent_atoms?: opts[:allow_nonexistent_atoms] || false,
+      strict?: strict?,
+      switches: switches
+    }
   end
 
   defp validate_option(value, kinds) do
@@ -594,7 +596,7 @@ defmodule OptionParser do
     end
   end
 
-  defp tag_option("no-" <> option = original, switches, allow_nonexistent_atoms?) do
+  defp tag_option("no-" <> option = original, %{switches: switches, allow_nonexistent_atoms?: allow_nonexistent_atoms?}) do
     cond do
       (negated = get_option_key(option, allow_nonexistent_atoms?)) && :boolean in List.wrap(switches[negated]) ->
         {:negated, negated}
@@ -605,7 +607,7 @@ defmodule OptionParser do
     end
   end
 
-  defp tag_option(option, _switches, allow_nonexistent_atoms?) do
+  defp tag_option(option, %{allow_nonexistent_atoms?: allow_nonexistent_atoms?}) do
     if option_key = get_option_key(option, allow_nonexistent_atoms?) do
       {:default, option_key}
     else
@@ -613,7 +615,7 @@ defmodule OptionParser do
     end
   end
 
-  defp tag_oneletter_alias(alias, aliases, allow_nonexistent_atoms?) when is_binary(alias) do
+  defp tag_oneletter_alias(alias, %{aliases: aliases, allow_nonexistent_atoms?: allow_nonexistent_atoms?}) when is_binary(alias) do
     if option_key = aliases[to_existing_key(alias, allow_nonexistent_atoms?)] do
       {:default, option_key}
     else
@@ -704,7 +706,7 @@ defmodule OptionParser do
   defp to_underscore(<<>>, acc),
     do: acc
 
-  def get_option_key(option, allow_nonexistent_atoms?) do
+  defp get_option_key(option, allow_nonexistent_atoms?) do
     if string = to_underscore(option) do
       to_existing_key(string, allow_nonexistent_atoms?)
     end

lib/elixir/lib/path.ex

@@ -12,14 +12,15 @@ defmodule Path do
   that require it (like `wildcard/2` and `expand/1`).
   """
 
-  alias :filename, as: FN
   @type t :: :unicode.chardata()
 
   @doc """
   Converts the given path to an absolute one. Unlike
   `expand/1`, no attempt is made to resolve `..`, `.` or `~`.
 
-  ## Unix examples
+  ## Examples
+
+  ### Unix
 
       Path.absname("foo")
       #=> "/usr/local/foo"
@@ -27,12 +28,12 @@ defmodule Path do
       Path.absname("../x")
       #=> "/usr/local/../x"
 
-  ## Windows
+  ### Windows
 
       Path.absname("foo").
-      "D:/usr/local/foo"
+      #=> "D:/usr/local/foo"
       Path.absname("../x").
-      "D:/usr/local/../x"
+      #=> "D:/usr/local/../x"
 
   """
   @spec absname(t) :: binary
@@ -41,8 +42,10 @@ defmodule Path do
   end
 
   @doc """
-  Builds a path from `relative_to` to `path`. If `path` is already
-  an absolute path, `relative_to` is ignored. See also `relative_to/2`.
+  Builds a path from `relative_to` to `path`.
+
+  If `path` is already an absolute path, `relative_to` is ignored. See also
+  `relative_to/2`.
 
   Unlike `expand/2`, no attempt is made to
   resolve `..`, `.` or `~`.
@@ -97,7 +100,7 @@ defmodule Path do
     do: do_absname_join(IO.chardata_to_string(left), relative(right), [], major_os_type())
 
   defp do_absname_join(<<uc_letter, ?:, rest::binary>>, relativename, [], :win32) when uc_letter in ?A..?Z, do:
-    do_absname_join(rest, relativename, [?:, uc_letter+?a-?A], :win32)
+    do_absname_join(rest, relativename, [?:, uc_letter + ?a - ?A], :win32)
   defp do_absname_join(<<?\\, rest::binary>>, relativename, result, :win32), do:
     do_absname_join(<<?/, rest::binary>>, relativename, result, :win32)
   defp do_absname_join(<<?/, rest::binary>>, relativename, [?., ?/ | result], os_type), do:
@@ -141,10 +144,11 @@ defmodule Path do
 
   @doc """
   Expands the path relative to the path given as the second argument
-  expanding any `.` and `..` characters. If the path is already an
-  absolute path, `relative_to` is ignored.
+  expanding any `.` and `..` characters.
 
-  Note, that this function treats `path` with a leading `~` as
+  If the path is already an absolute path, `relative_to` is ignored.
+
+  Note that this function treats a `path` with a leading `~` as
   an absolute one.
 
   The second argument is first expanded to an absolute path.
@@ -169,7 +173,9 @@ defmodule Path do
   @doc """
   Returns the path type.
 
-  ## Unix examples
+  ## Examples
+
+  ### Unix
 
       Path.type("/")                #=> :absolute
       Path.type("/usr/local/bin")   #=> :absolute
@@ -177,7 +183,7 @@ defmodule Path do
       Path.type("../usr/local/bin") #=> :relative
       Path.type("~/file")           #=> :relative
 
-  ## Windows examples
+  ### Windows
 
       Path.type("D:/usr/local/bin") #=> :absolute
       Path.type("usr/local/bin")    #=> :relative
@@ -186,20 +192,24 @@ defmodule Path do
 
   """
   @spec type(t) :: :absolute | :relative | :volumerelative
-  def type(name) when is_list(name) or is_binary(name) do
+  def type(name)
+      when is_list(name)
+      when is_binary(name) do
     pathtype(name, major_os_type()) |> elem(0)
   end
 
   @doc """
   Forces the path to be a relative path.
 
-  ## Unix examples
+  ## Examples
+
+  ### Unix
 
       Path.relative("/usr/local/bin")   #=> "usr/local/bin"
       Path.relative("usr/local/bin")    #=> "usr/local/bin"
       Path.relative("../usr/local/bin") #=> "../usr/local/bin"
 
-  ## Windows examples
+  ### Windows
 
       Path.relative("D:/usr/local/bin") #=> "usr/local/bin"
       Path.relative("usr/local/bin")    #=> "usr/local/bin"
@@ -225,46 +235,49 @@ defmodule Path do
     end
   end
 
-  defp unix_pathtype(<<?/, relative::binary>>), do:
-    {:absolute, relative}
-  defp unix_pathtype([?/ | relative]), do:
-    {:absolute, relative}
-  defp unix_pathtype([list | rest]) when is_list(list), do:
-    unix_pathtype(list ++ rest)
-  defp unix_pathtype(relative), do:
-    {:relative, relative}
+  defp unix_pathtype(path) when path in ["/", '/'],
+    do: {:absolute, "."}
+  defp unix_pathtype(<<?/, relative::binary>>),
+    do: {:absolute, relative}
+  defp unix_pathtype([?/ | relative]),
+    do: {:absolute, relative}
+  defp unix_pathtype([list | rest]) when is_list(list),
+    do: unix_pathtype(list ++ rest)
+  defp unix_pathtype(relative),
+    do: {:relative, relative}
 
   @slash [?/, ?\\]
 
-  defp win32_pathtype([list | rest]) when is_list(list), do:
-    win32_pathtype(list++rest)
-  defp win32_pathtype([char, list | rest]) when is_list(list), do:
-    win32_pathtype([char | list++rest])
-  defp win32_pathtype(<<c1, c2, relative::binary>>) when c1 in @slash and c2 in @slash, do:
-    {:absolute, relative}
-  defp win32_pathtype(<<c, relative::binary>>) when c in @slash, do:
-    {:volumerelative, relative}
-  defp win32_pathtype(<<_letter, ?:, c, relative::binary>>) when c in @slash, do:
-    {:absolute, relative}
-  defp win32_pathtype(<<_letter, ?:, relative::binary>>), do:
-    {:volumerelative, relative}
+  defp win32_pathtype([list | rest]) when is_list(list),
+    do: win32_pathtype(list ++ rest)
+  defp win32_pathtype([char, list | rest]) when is_list(list),
+    do: win32_pathtype([char | list ++ rest])
+  defp win32_pathtype(<<c1, c2, relative::binary>>) when c1 in @slash and c2 in @slash,
+    do: {:absolute, relative}
+  defp win32_pathtype(<<char, relative::binary>>) when char in @slash,
+    do: {:volumerelative, relative}
+  defp win32_pathtype(<<_letter, ?:, char, relative::binary>>) when char in @slash,
+    do: {:absolute, relative}
+  defp win32_pathtype(<<_letter, ?:, relative::binary>>),
+    do: {:volumerelative, relative}
 
-  defp win32_pathtype([c1, c2 | relative]) when c1 in @slash and c2 in @slash, do:
-    {:absolute, relative}
-  defp win32_pathtype([c | relative]) when c in @slash, do:
-    {:volumerelative, relative}
-  defp win32_pathtype([c1, c2, list | rest]) when is_list(list), do:
-    win32_pathtype([c1, c2 | list++rest])
-  defp win32_pathtype([_letter, ?:, c | relative]) when c in @slash, do:
-    {:absolute, relative}
-  defp win32_pathtype([_letter, ?: | relative]), do:
-    {:volumerelative, relative}
-  defp win32_pathtype(relative), do:
-    {:relative, relative}
+  defp win32_pathtype([c1, c2 | relative]) when c1 in @slash and c2 in @slash,
+    do: {:absolute, relative}
+  defp win32_pathtype([char | relative]) when char in @slash,
+    do: {:volumerelative, relative}
+  defp win32_pathtype([c1, c2, list | rest]) when is_list(list),
+    do: win32_pathtype([c1, c2 | list ++ rest])
+  defp win32_pathtype([_letter, ?:, char | relative]) when char in @slash,
+    do: {:absolute, relative}
+  defp win32_pathtype([_letter, ?: | relative]),
+    do: {:volumerelative, relative}
+  defp win32_pathtype(relative),
+    do: {:relative, relative}
 
   @doc """
   Returns the given `path` relative to the given `from` path.
-  In other words, it tries to strip the `from` prefix from `path`.
+
+  In other words, this function tries to strip the `from` prefix from `path`.
 
   This function does not query the file system, so it assumes
   no symlinks between the paths.
@@ -304,8 +317,10 @@ defmodule Path do
 
   @doc """
   Convenience to get the path relative to the current working
-  directory. If, for some reason, the current working directory
-  cannot be retrieved, returns the full path.
+  directory.
+
+  If, for some reason, the current working directory
+  cannot be retrieved, this function returns the given `path`.
   """
   @spec relative_to_cwd(t) :: binary
   def relative_to_cwd(path) do
@@ -333,13 +348,15 @@ defmodule Path do
   """
   @spec basename(t) :: binary
   def basename(path) do
-    FN.basename(IO.chardata_to_string(path))
+    :filename.basename(IO.chardata_to_string(path))
   end
 
   @doc """
   Returns the last component of `path` with the `extension`
-  stripped. This function should be used to remove a specific
-  extension which may, or may not, be there.
+  stripped.
+
+  This function should be used to remove a specific
+  extension which may or may not be there.
 
   ## Examples
 
@@ -355,7 +372,7 @@ defmodule Path do
   """
   @spec basename(t, t) :: binary
   def basename(path, extension) do
-    FN.basename(IO.chardata_to_string(path), IO.chardata_to_string(extension))
+    :filename.basename(IO.chardata_to_string(path), IO.chardata_to_string(extension))
   end
 
   @doc """
@@ -369,10 +386,13 @@ defmodule Path do
       iex> Path.dirname("/foo/bar/baz.ex")
       "/foo/bar"
 
+      iex> Path.dirname("/foo/bar/")
+      "/foo/bar"
+
   """
   @spec dirname(t) :: binary
   def dirname(path) do
-    FN.dirname(IO.chardata_to_string(path))
+    :filename.dirname(IO.chardata_to_string(path))
   end
 
   @doc """
@@ -389,7 +409,7 @@ defmodule Path do
   """
   @spec extname(t) :: binary
   def extname(path) do
-    FN.extension(IO.chardata_to_string(path))
+    :filename.extension(IO.chardata_to_string(path))
   end
 
   @doc """
@@ -406,12 +426,14 @@ defmodule Path do
   """
   @spec rootname(t) :: binary
   def rootname(path) do
-    FN.rootname(IO.chardata_to_string(path))
+    :filename.rootname(IO.chardata_to_string(path))
   end
 
   @doc """
-  Returns the `path` with the `extension` stripped. This function should be used to
-  remove a specific extension which might, or might not, be there.
+  Returns the `path` with the `extension` stripped.
+
+  This function should be used to remove a specific extension which may
+  or may not be there.
 
   ## Examples
 
@@ -424,7 +446,7 @@ defmodule Path do
   """
   @spec rootname(t, t) :: binary
   def rootname(path, extension) do
-    FN.rootname(IO.chardata_to_string(path), IO.chardata_to_string(extension))
+    :filename.rootname(IO.chardata_to_string(path), IO.chardata_to_string(extension))
   end
 
   @doc """
@@ -455,13 +477,16 @@ defmodule Path do
   Joins two paths.
 
   The right path will always be expanded to its relative format
-  and any trailing slash is removed on join.
+  and any trailing slash will be removed when joining.
 
   ## Examples
 
       iex> Path.join("foo", "bar")
       "foo/bar"
 
+      iex> Path.join("/foo", "/bar/")
+      "/foo/bar"
+
   """
   @spec join(t, t) :: binary
   def join(left, right) do
@@ -512,7 +537,7 @@ defmodule Path do
   def split(""), do: []
 
   def split(path) do
-    FN.split(IO.chardata_to_string(path))
+    :filename.split(IO.chardata_to_string(path))
   end
 
   defmodule Wildcard do
@@ -542,7 +567,7 @@ defmodule Path do
   end
 
   @doc """
-  Traverses paths according to the given `glob` expression, and returns a
+  Traverses paths according to the given `glob` expression and returns a
   list of matches.
 
   The wildcard looks like an ordinary path, except that certain
@@ -568,7 +593,7 @@ defmodule Path do
 
   Other characters represent themselves. Only paths that have
   exactly the same character in the same position will match. Note
-  that matching is case-sensitive; i.e. "a" will not match "A".
+  that matching is case-sensitive: `"a"` will not match `"A"`.
 
   By default, the patterns `*` and `?` do not match files starting
   with a dot `.` unless `match_dot: true` is given in `opts`.
@@ -576,7 +601,7 @@ defmodule Path do
   ## Examples
 
   Imagine you have a directory called `projects` with three Elixir projects
-  inside of it: `elixir`, `ex_doc` and `dynamo`. You can find all `.beam` files
+  inside of it: `elixir`, `ex_doc`, and `plug`. You can find all `.beam` files
   inside the `ebin` directory of each project as follows:
 
       Path.wildcard("projects/*/ebin/**/*.beam")
@@ -586,21 +611,23 @@ defmodule Path do
       Path.wildcard("projects/*/ebin/**/*.{beam,app}")
 
   """
-  @spec wildcard(t, Keyword.t) :: [binary]
+  @spec wildcard(t, keyword) :: [binary]
   def wildcard(glob, opts \\ []) do
     mod = if Keyword.get(opts, :match_dot), do: :file, else: Path.Wildcard
     glob
-    |> chardata_to_list()
+    |> chardata_to_list!()
     |> :filelib.wildcard(mod)
     |> Enum.map(&IO.chardata_to_string/1)
   end
 
-  # expand_dot the given path by expanding "..", "." and "~".
-
-  defp chardata_to_list(chardata) do
+  defp chardata_to_list!(chardata) do
     case :unicode.characters_to_list(chardata) do
       result when is_list(result) ->
-        result
+        if 0 in result do
+          raise ArgumentError, "cannot execute Path.wildcard/2 for path with null byte, got: #{inspect chardata}"
+        else
+          result
+        end
 
       {:error, encoded, rest} ->
         raise UnicodeConversionError, encoded: encoded, rest: rest, kind: :invalid
@@ -629,6 +656,8 @@ defmodule Path do
     end
   end
 
+  # expand_dot the given path by expanding "..", "." and "~".
+
   defp expand_dot(<<"/", rest::binary>>),
     do: "/" <> do_expand_dot(rest)
   defp expand_dot(<<letter, ":/", rest::binary>>) when letter in ?a..?z,

lib/elixir/lib/port.ex

@@ -63,11 +63,12 @@ defmodule Port do
 
   On its turn, the port will send the connected process the following messages:
 
-    * `{port, {:port, data}}` - data sent by the port
+    * `{port, {:data, data}}` - data sent by the port
     * `{port, :closed}` - reply to the `{pid, :close}` message
     * `{port, :connected}` - reply to the `{pid, {:connect, new_pid}}` message
-    * `{:EXIT, port, reason}` - exit signals in case the port crashes and the
-      owner process is trapping exits
+    * `{:EXIT, port, reason}` - exit signals in case the port crashes. If reason
+      is not `:normal`, this message will only be received if the owner process
+      is trapping exits
 
   ## Open mechanisms
 

lib/elixir/lib/process.ex

@@ -4,7 +4,7 @@ defmodule Process do
 
   Besides the functions available in this module, the `Kernel` module
   exposes and auto-imports some basic functionality related to processes
-  available through the functions:
+  available through the following functions:
 
     * `Kernel.spawn/1` and `Kernel.spawn/3`
     * `Kernel.spawn_link/1` and `Kernel.spawn_link/3`
@@ -15,32 +15,31 @@ defmodule Process do
   """
 
   @doc """
-  Returns `true` if the process exists and is alive (i.e. it is not exiting
-  and has not exited yet). Otherwise, returns `false`.
+  Tells whether the given process is alive.
 
-  `pid` must refer to a process at the local node.
+  If the process identified by `pid` is alive (that is, it's not exiting and has
+  not exited yet) than this function returns `true`. Otherwise, it returns
+  `false`.
+
+  `pid` must refer to a process running on the local node.
 
   Inlined by the compiler.
   """
   @spec alive?(pid) :: boolean
-  def alive?(pid) do
-    :erlang.is_process_alive(pid)
-  end
+  defdelegate alive?(pid), to: :erlang, as: :is_process_alive
 
   @doc """
   Returns all key-value pairs in the process dictionary.
 
   Inlined by the compiler.
   """
-  @spec get :: [{term, term}]
-  def get do
-    :erlang.get()
-  end
+  @spec get() :: [{term, term}]
+  defdelegate get(), to: :erlang
 
   @doc """
-  Returns the value for the given `key` or `default` if `key` is not set.
+  Returns the value for the given `key` in the process dictionary,
+  or `default` if `key` is not set.
   """
-  @spec get(term) :: term
   @spec get(term, default :: term) :: term
   def get(key, default \\ nil) do
     case :erlang.get(key) do
@@ -57,25 +56,30 @@ defmodule Process do
   Inlined by the compiler.
   """
   @spec get_keys() :: [term]
-  def get_keys() do
-    :erlang.get_keys()
-  end
+  defdelegate get_keys(), to: :erlang
 
   @doc """
-  Returns all keys that have the given `value`.
+  Returns all keys in the process dictionary that have the given `value`.
 
   Inlined by the compiler.
   """
   @spec get_keys(term) :: [term]
-  def get_keys(value) do
-    :erlang.get_keys(value)
-  end
+  defdelegate get_keys(value), to: :erlang
 
   @doc """
   Stores the given `key`-`value` pair in the process dictionary.
 
-  The return value is the value that was previously stored under the key `key`
-  (or `nil` in case no value was stored under `key`).
+  The return value of this function is the value that was previously stored
+  under `key`, or `nil` in case no value was stored under `key`.
+
+  ## Examples
+
+      # Assuming :locale was not set
+      Process.put(:locale, "en")
+      #=> nil
+      Process.put(:locale, "fr")
+      #=> "en"
+
   """
   @spec put(term, term) :: term | nil
   def put(key, value) do
@@ -84,6 +88,18 @@ defmodule Process do
 
   @doc """
   Deletes the given `key` from the process dictionary.
+
+  Returns the value that was under `key` in the process dictionary,
+  or `nil` if `key` was not stored in the process dictionary.
+
+  ## Examples
+
+      Process.put(:comments, ["comment", "other comment"])
+      Process.delete(:comments)
+      #=> ["comment", "other comment"]
+      Process.delete(:comments)
+      #=> nil
+
   """
   @spec delete(term) :: term | nil
   def delete(key) do
@@ -91,7 +107,7 @@ defmodule Process do
   end
 
   @doc """
-  Sends an exit signal with the given `reason` to the `pid`.
+  Sends an exit signal with the given `reason` to `pid`.
 
   The following behaviour applies if `reason` is any term except `:normal`
   or `:kill`:
@@ -108,7 +124,7 @@ defmodule Process do
   If it is trapping exits, the exit signal is transformed into a message
   `{:EXIT, from, :normal}` and delivered to its message queue.
 
-  If `reason` is the atom `:kill`, that is if `exit(pid, :kill)` is called,
+  If `reason` is the atom `:kill`, that is if `Process.exit(pid, :kill)` is called,
   an untrappable exit signal is sent to `pid` which will unconditionally exit
   with reason `:killed`.
 
@@ -117,27 +133,28 @@ defmodule Process do
   ## Examples
 
       Process.exit(pid, :kill)
+      #=> true
 
   """
   @spec exit(pid, term) :: true
-  def exit(pid, reason) do
-    :erlang.exit(pid, reason)
-  end
+  defdelegate exit(pid, reason), to: :erlang
 
   @doc """
-  Sleeps the current process by `timeout`.
+  Sleeps the current process for the given `timeout`.
 
   `timeout` is either the number of milliseconds to sleep as an
   integer or the atom `:infinity`. When `:infinity` is given,
-  the current process will suspend forever.
+  the current process will sleep forever, and not
+  consume or reply to messages.
 
   **Use this function with extreme care**. For almost all situations
   where you would use `sleep/1` in Elixir, there is likely a
-  more correct, faster and precise way of achieving it with
+  more correct, faster and precise way of achieving the same with
   message passing.
 
-  For example, if you are waiting a process to perform some
-  action, it is better to communicate.
+  For example, if you are waiting for a process to perform some
+  action, it is better to communicate the progress of such action
+  with messages.
 
   In other words, **do not**:
 
@@ -164,11 +181,11 @@ defmodule Process do
         30_000 -> :timeout # Optional timeout
       end
 
-  Or even use `Task.async/1` and `Task.await/2` in the example
-  above.
+  For cases like the one above, `Task.async/1` and `Task.await/2` are
+  preferred.
 
   Similarly, if you are waiting for a process to terminate,
-  use monitor instead of sleep. **Do not**:
+  monitor that process instead of sleeping. **Do not**:
 
       Task.start_link fn ->
         ...
@@ -192,6 +209,7 @@ defmodule Process do
       end
 
   """
+  @spec sleep(timeout) :: :ok
   def sleep(timeout)
       when is_integer(timeout) and timeout >= 0
       when timeout == :infinity do
@@ -201,45 +219,48 @@ defmodule Process do
   @doc """
   Sends a message to the given process.
 
-  If the option `:noconnect` is used and sending the message would require an
-  auto-connection to another node the message is not sent and `:noconnect` is
-  returned.
+  ## Options
 
-  If the option `:nosuspend` is used and sending the message would cause the
-  sender to be suspended the message is not sent and `:nosuspend` is returned.
+    * `:noconnect` - when used, if sending the message would require an
+      auto-connection to another node the message is not sent and `:noconnect` is
+      returned.
+
+    * `:nosuspend` - when used, if sending the message would cause the sender to
+      be suspended the message is not sent and `:nosuspend` is returned.
 
   Otherwise the message is sent and `:ok` is returned.
 
   ## Examples
 
-      iex> Process.send({:name, :node_does_not_exist}, :hi, [:noconnect])
+      iex> Process.send({:name, :node_that_does_not_exist}, :hi, [:noconnect])
       :noconnect
 
+  Inlined by the compiler.
   """
   @spec send(dest, msg, [option]) :: :ok | :noconnect | :nosuspend
         when dest: pid | port | atom | {atom, node},
              msg: any,
              option: :noconnect | :nosuspend
-  def send(dest, msg, options) do
-    :erlang.send(dest, msg, options)
-  end
+  defdelegate send(dest, msg, options), to: :erlang
 
   @doc """
   Sends `msg` to `dest` after `time` milliseconds.
 
   If `dest` is a PID, it must be the PID of a local process, dead or alive.
   If `dest` is an atom, it must be the name of a registered process
-  which is looked up at the time of delivery. No error is given if the name does
+  which is looked up at the time of delivery. No error is produced if the name does
   not refer to a process.
 
-  This function returns a timer reference, which can be read or canceled with
-  `read_timer/1` and `cancel_timer/1`.
+  This function returns a timer reference, which can be read with `read_timer/1`
+  or canceled with `cancel_timer/1`.
 
-  Finally, the timer will be automatically canceled if the given `dest` is a PID
+  The timer will be automatically canceled if the given `dest` is a PID
   which is not alive or when the given PID exits. Note that timers will not be
   automatically canceled when `dest` is an atom (as the atom resolution is done
   on delivery).
 
+  Inlined by the compiler.
+
   ## Options
 
     * `:abs` - (boolean) when `false`, `time` is treated as relative to the
@@ -248,6 +269,10 @@ defmodule Process do
     To read more about Erlang monotonic time and other time-related concepts,
     look at the documentation for the `System` module. Defaults to `false`.
 
+  ## Examples
+
+      timer_ref = Process.send_after(pid, :hi, 1000)
+
   """
   @spec send_after(pid | atom, term, non_neg_integer, [option]) :: reference
         when option: {:abs, boolean}
@@ -256,24 +281,40 @@ defmodule Process do
   end
 
   @doc """
-  Cancels a timer created by `send_after/3`.
+  Cancels a timer returned by `send_after/3`.
 
   When the result is an integer, it represents the time in milliseconds
   left until the timer would have expired.
 
-  When the result is `false`, a timer corresponding to `timer_ref` could
-  not be found. This can be either because the timer expired, already has
-  been canceled, or because `timer_ref` never corresponded to a timer.
+  When the result is `false`, a timer corresponding to `timer_ref` could not be
+  found. This can happen either because the timer expired, because it has
+  already been canceled, or because `timer_ref` never corresponded to a timer.
 
-  If the timer has expired, the timeout message has been sent, but it does
-  not tell you whether or not it has arrived at its destination yet.
+  Even if the timer had expired and the message was sent, this function does not
+  tell you if the timeout message has arrived at its destination yet.
+
+  ## Options
+
+    * `:async` - (boolean) when `false`, the request for cancellation is
+      synchronous. When `true`, the request for cancellation is asynchronous,
+      meaning that the request to cancel the timer is issued and `:ok` is
+      returned right away. Defaults to `false`.
+
+    * `:info` - (boolean) whether to return information about the timer being
+      cancelled. When the `:async` option is `false` and `:info` is `true`, then
+      either an integer or `false` (like described above) is returned. If
+      `:async` is `false` and `:info` is `false`, `:ok` is returned. If `:async`
+      is `true` and `:info` is `true`, a message in the form `{:cancel_timer,
+      timer_ref, result}` (where `result` is an integer or `false` like
+      described above) is sent to the caller of this function when the
+      cancellation has been performed. If `:async` is `true` and `:info` is
+      `false`, no message is sent. Defaults to `true`.
 
   Inlined by the compiler.
   """
-  @spec cancel_timer(reference) :: non_neg_integer | false
-  def cancel_timer(timer_ref) do
-    :erlang.cancel_timer(timer_ref)
-  end
+  @spec cancel_timer(reference, options) :: non_neg_integer | false | :ok
+        when options: [async: boolean, info: boolean]
+  defdelegate cancel_timer(timer_ref, options \\ []), to: :erlang
 
   @doc """
   Reads a timer created by `send_after/3`.
@@ -281,19 +322,17 @@ defmodule Process do
   When the result is an integer, it represents the time in milliseconds
   left until the timer will expire.
 
-  When the result is `false`, a timer corresponding to `timer_ref` could
-  not be found. This can be either because the timer expired, already has
+  When the result is `false`, a timer corresponding to `timer_ref` could not be
+  found. This can be either because the timer expired, because it has already
   been canceled, or because `timer_ref` never corresponded to a timer.
 
-  If the timer has expired, the timeout message has been sent, but it does
-  not tell you whether or not it has arrived at its destination yet.
+  Even if the timer had expired and the message was sent, this function does not
+  tell you if the timeout message has arrived at its destination yet.
 
   Inlined by the compiler.
   """
   @spec read_timer(reference) :: non_neg_integer | false
-  def read_timer(timer_ref) do
-    :erlang.read_timer(timer_ref)
-  end
+  defdelegate read_timer(timer_ref), to: :erlang
 
   @type spawn_opt  :: :link | :monitor | {:priority, :low | :normal | :high} |
                       {:fullsweep_after, non_neg_integer} |
@@ -309,18 +348,16 @@ defmodule Process do
   containing the PID and the monitoring reference, otherwise
   just the spawned process PID.
 
-  It also accepts extra options, for the list of available options
+  More options are available; for the comprehensive list of available options
   check [`:erlang.spawn_opt/4`](http://www.erlang.org/doc/man/erlang.html#spawn_opt-4).
 
   Inlined by the compiler.
   """
   @spec spawn((() -> any), spawn_opts) :: pid | {pid, reference}
-  def spawn(fun, opts) do
-    :erlang.spawn_opt(fun, opts)
-  end
+  defdelegate spawn(fun, opts), to: :erlang, as: :spawn_opt
 
   @doc """
-  Spawns the given function from module `mod`, passing the given `args`
+  Spawns the given function `fun` from module `mod`, passing the given `args`
   according to the given options.
 
   The result depends on the given options. In particular,
@@ -334,13 +371,22 @@ defmodule Process do
   Inlined by the compiler.
   """
   @spec spawn(module, atom, list, spawn_opts) :: pid | {pid, reference}
-  def spawn(mod, fun, args, opts) do
-    :erlang.spawn_opt(mod, fun, args, opts)
-  end
+  defdelegate spawn(mod, fun, args, opts), to: :erlang, as: :spawn_opt
 
   @doc """
-  The calling process starts monitoring the given `item`.
-  It returns the monitor reference.
+  Starts monitoring the given `item` from the calling process.
+
+  Once the monitored process dies, a message is delivered to the
+  monitoring process in the shape of:
+
+      {:DOWN, ref, :process, object, reason}
+
+  where:
+
+    * `ref` is a monitor reference returned by this function;
+    * `object` is either a `pid` of the monitored process (if monitoring
+      a PID) or `{name, node}` (if monitoring a remote or local name);
+    * `reason` is the exit reason.
 
   See [the need for monitoring](http://elixir-lang.org/getting-started/mix-otp/genserver.html#the-need-for-monitoring)
   for an example.
@@ -354,61 +400,75 @@ defmodule Process do
   end
 
   @doc """
+  Demonitors the monitor identifies by the given `reference`.
+
   If `monitor_ref` is a reference which the calling process
-  obtained by calling `monitor/1`, this monitoring is turned off.
+  obtained by calling `monitor/1`, that monitoring is turned off.
   If the monitoring is already turned off, nothing happens.
 
   See [`:erlang.demonitor/2`](http://www.erlang.org/doc/man/erlang.html#demonitor-2) for more info.
 
   Inlined by the compiler.
   """
-  @spec demonitor(reference) :: true
   @spec demonitor(reference, options :: [:flush | :info]) :: boolean
-  def demonitor(monitor_ref, options \\ []) do
-    :erlang.demonitor(monitor_ref, options)
-  end
+  defdelegate demonitor(monitor_ref, options \\ []), to: :erlang
 
   @doc """
-  Returns a list of process identifiers corresponding to all the
+  Returns a list of PIDs corresponding to all the
   processes currently existing on the local node.
 
-  Note that a process that is exiting, exists but is not alive, i.e.,
-  `alive?/1` will return `false` for a process that is exiting,
-  but its process identifier will be part of the result returned.
+  Note that if a process is exiting, it is considered to exist but not be
+  alive. This means that for such process, `alive?/1` will return `false` but
+  its PID will be part of the list of PIDs returned by this function.
 
   See [`:erlang.processes/0`](http://www.erlang.org/doc/man/erlang.html#processes-0) for more info.
+
+  Inlined by the compiler.
   """
-  @spec list :: [pid]
-  def list do
-    :erlang.processes()
-  end
+  @spec list() :: [pid]
+  defdelegate list(), to: :erlang, as: :processes
 
   @doc """
-  Creates a link between the calling process and another process
-  (or port) `pid`, if there is not such a link already.
+  Creates a link between the calling process and the given item (process or
+  port).
+
+  Links are bidirectional. Linked processes can be unlinked by using `unlink/1`.
+
+  If such a link exists already, this function does nothing since there can only
+  be one link between two given processes. If a process tries to create a link
+  to itself, nothing will happen.
+
+  When two processes are linked, each one receives exit signals from the other
+  (see also `exit/2`). Let's assume `pid1` and `pid2` are linked. If `pid2`
+  exits with a reason other than `:normal` (which is also the exit reason used
+  when a process finishes its job) and `pid1` is not trapping exits (see
+  `flag/2`), then `pid1` will exit with the same reason as `pid2` and in turn
+  emit an exit signal to all its other linked processes. The behaviour when
+  `pid1` is trapping exits is described in `exit/2`.
 
   See [`:erlang.link/1`](http://www.erlang.org/doc/man/erlang.html#link-1) for more info.
 
   Inlined by the compiler.
   """
   @spec link(pid | port) :: true
-  def link(pid) do
-    :erlang.link(pid)
-  end
+  defdelegate link(pid_or_port), to: :erlang
 
   @doc """
-  Removes the link, if there is one, between the calling process and
-  the process or port referred to by `pid`. Returns `true` and does not
-  fail, even if there is no link or `id` does not exist
+
+  Removes the link between the calling process and the given item (process or
+  port).
+
+  If there is no such link, this function does nothing. If `pid_or_port` does
+  not exist, this function does not produce any errors and simply does nothing.
+
+  The return value of this function is always `true`.
 
   See [`:erlang.unlink/1`](http://www.erlang.org/doc/man/erlang.html#unlink-1) for more info.
 
   Inlined by the compiler.
   """
   @spec unlink(pid | port) :: true
-  def unlink(pid) do
-    :erlang.unlink(pid)
-  end
+  defdelegate unlink(pid_or_port), to: :erlang
 
   @doc """
   Registers the given `pid_or_port` under the given `name`.
@@ -416,15 +476,23 @@ defmodule Process do
   `name` must be an atom and can then be used instead of the
   PID/port identifier when sending messages with `Kernel.send/2`.
 
-  `register/2` will fail with `ArgumentError` if the PID/Port is
-  not existing locally and alive, if the name is already registered
-  or if the `pid_or_port` is already registered to a different `name`.
+  `register/2` will fail with `ArgumentError` in any of the following cases:
+
+    * the PID/Port is not existing locally and alive
+    * the name is already registered
+    * the `pid_or_port` is already registered under a different `name`
 
   The following names are reserved and cannot be assigned to
-  processes nor ports: `nil`, `false`, `true` or `:undefined`.
+  processes nor ports:
+
+    * `nil`
+    * `false`
+    * `true`
+    * `:undefined`
+
   """
   @spec register(pid | port, atom) :: true
-  def register(pid_or_port, name) when is_atom(name) and not name in [nil, false, true, :undefined] do
+  def register(pid_or_port, name) when is_atom(name) and name not in [nil, false, true, :undefined] do
     :erlang.register(name, pid_or_port)
   catch
     :error, :badarg when node(pid_or_port) != node()  ->
@@ -446,15 +514,15 @@ defmodule Process do
 
   Fails with `ArgumentError` if the name is not registered
   to any PID or port.
+
+  Inlined by the compiler.
   """
   @spec unregister(atom) :: true
-  def unregister(name) do
-    :erlang.unregister(name)
-  end
+  defdelegate unregister(name), to: :erlang
 
   @doc """
-  Returns the PID or port identifier with the registered `name`.
-  Returns `nil` if the name is not registered.
+  Returns the PID or port identifier registered under `name` or `nil` if the
+  name is not registered.
 
   See [`:erlang.whereis/1`](http://www.erlang.org/doc/man/erlang.html#whereis-1) for more info.
   """
@@ -464,16 +532,20 @@ defmodule Process do
   end
 
   @doc """
-  Returns the PID of the group leader for the process which evaluates the function.
+  Returns the PID of the group leader for the calling process.
+
+  Inlined by the compiler.
   """
-  @spec group_leader :: pid
-  def group_leader do
-    :erlang.group_leader
-  end
+  @spec group_leader() :: pid
+  defdelegate group_leader(), to: :erlang
 
   @doc """
-  Sets the group leader of `pid` to `leader`. Typically, this is used when a processes
-  started from a certain shell should have a group leader other than `:init`.
+  Sets the group leader of the given `pid` to `leader`.
+
+  Typically, this is used when a process started from a certain shell should
+  have a group leader other than `:init`.
+
+  Inlined by the compiler.
   """
   @spec group_leader(pid, leader :: pid) :: true
   def group_leader(pid, leader) do
@@ -482,46 +554,67 @@ defmodule Process do
 
   @doc """
   Returns a list of names which have been registered using `register/2`.
-  """
-  @spec registered :: [atom]
-  def registered do
-    :erlang.registered()
-  end
 
-  @typep process_flag :: :trap_exit | :error_handler | :min_heap_size |
-                         :min_bin_vheap_size | :priority | :save_calls |
-                         :sensitive
+  Inlined by the compiler.
+  """
+  @spec registered() :: [atom]
+  defdelegate registered(), to: :erlang
+
+  @typep heap_size :: non_neg_integer |
+                      %{size: non_neg_integer, kill: boolean, error_logger: boolean}
+
+  @typep priority_level :: :low | :normal | :high | :max
+
   @doc """
-  Sets certain flags for the process which calls this function.
-  Returns the old value of the `flag`.
+  Sets the given `flag` to `value` for the calling process.
+
+  Returns the old value of `flag`.
 
   See [`:erlang.process_flag/2`](http://www.erlang.org/doc/man/erlang.html#process_flag-2) for more info.
+
+  Note that `flag` values `:max_heap_size` and `:message_queue_data` are only available since OTP 19.
+
+  Inlined by the compiler.
   """
-  @spec flag(process_flag, term) :: term
-  def flag(flag, value) do
-    :erlang.process_flag(flag, value)
-  end
+  @spec flag(:error_handler, module) :: module
+  @spec flag(:max_heap_size, heap_size) :: heap_size
+  @spec flag(:message_queue_data, :erlang.message_queue_data) :: :erlang.message_queue_data
+  @spec flag(:min_bin_vheap_size, non_neg_integer) :: non_neg_integer
+  @spec flag(:min_heap_size, non_neg_integer) :: non_neg_integer
+  @spec flag(:monitor_nodes, term) :: term
+  @spec flag({:monitor_nodes, term()}, term) :: term
+  @spec flag(:priority, priority_level) :: priority_level
+  @spec flag(:save_calls, 0..10_000) :: 0..10_000
+  @spec flag(:sensitive, boolean) :: boolean
+  @spec flag(:trap_exit, boolean) :: boolean
+  defdelegate flag(flag, value), to: :erlang, as: :process_flag
 
   @doc """
-  Sets certain flags for the process `pid`, in the same manner as `flag/2`.
-  Returns the old value of the `flag`. The allowed values for `flag` are
-  only a subset of those allowed in `flag/2`, namely `:save_calls`.
+  Sets the given `flag` to `value` for the given process `pid`.
+
+  Returns the old value of `flag`.
+
+  It raises `ArgumentError` if `pid` is not a local process.
+
+  The allowed values for `flag` are only a subset of those allowed in `flag/2`,
+  namely `:save_calls`.
 
   See [`:erlang.process_flag/3`](http://www.erlang.org/doc/man/erlang.html#process_flag-3) for more info.
+
+  Inlined by the compiler.
   """
-  @spec flag(pid, :save_calls, non_neg_integer) :: non_neg_integer
-  def flag(pid, flag, value) do
-    :erlang.process_flag(pid, flag, value)
-  end
+  @spec flag(pid, :save_calls, 0..10_000) :: 0..10_000
+  defdelegate flag(pid, flag, value), to: :erlang, as: :process_flag
 
   @doc """
   Returns information about the process identified by `pid`, or returns `nil` if the process
   is not alive.
+
   Use this only for debugging information.
 
   See [`:erlang.process_info/1`](http://www.erlang.org/doc/man/erlang.html#process_info-1) for more info.
   """
-  @spec info(pid) :: Keyword.t
+  @spec info(pid) :: keyword
   def info(pid) do
     nillify :erlang.process_info(pid)
   end
@@ -548,7 +641,9 @@ defmodule Process do
   end
 
   @doc """
-  Puts the calling process into a wait state
+  Puts the calling process into a "hibernation" state.
+
+  The calling process is put into a waiting state
   where its memory allocation has been reduced as much as possible,
   which is useful if the process does not expect to receive any messages
   in the near future.
@@ -558,9 +653,7 @@ defmodule Process do
   Inlined by the compiler.
   """
   @spec hibernate(module, atom, list) :: no_return
-  def hibernate(mod, fun, args) do
-    :erlang.hibernate(mod, fun, args)
-  end
+  defdelegate hibernate(mod, fun_name, args), to: :erlang
 
   @compile {:inline, nillify: 1}
   defp nillify(:undefined), do: nil

lib/elixir/lib/protocol.ex

@@ -23,9 +23,9 @@ defmodule Protocol do
                            :lists.seq(2, arity))
     type_args = [quote(do: t) | type_args]
 
-    call_args = :lists.map(fn i -> {String.to_atom(<<?x, i + 64>>), [], __MODULE__} end,
+    call_args = :lists.map(fn pos -> Macro.var(String.to_atom("var" <> Integer.to_string(pos)), __MODULE__) end,
                            :lists.seq(2, arity))
-    call_args = [quote(do: t) | call_args]
+    call_args = [quote(do: term) | call_args]
 
     quote do
       name  = unquote(name)
@@ -39,7 +39,7 @@ defmodule Protocol do
 
       # Generate the actual implementation
       Kernel.def unquote(name)(unquote_splicing(call_args)) do
-        impl_for!(t).unquote(name)(unquote_splicing(call_args))
+        impl_for!(term).unquote(name)(unquote_splicing(call_args))
       end
 
       # Convert the spec to callback if possible,
@@ -50,7 +50,7 @@ defmodule Protocol do
   end
 
   defmacro def(_) do
-    raise ArgumentError, "invalid args for def inside defprotocol"
+    raise ArgumentError, "invalid arguments for def inside defprotocol"
   end
 
   @doc """
@@ -179,7 +179,7 @@ defmodule Protocol do
     prefix = Atom.to_charlist(protocol) ++ '.'
     extract_matching_by_attribute paths, prefix, fn
       _mod, attributes ->
-        case attributes[:impl] do
+        case attributes[:protocol_impl] do
           [protocol: ^protocol, for: for] -> for
           _ -> nil
         end
@@ -254,23 +254,22 @@ defmodule Protocol do
     {:error, :not_a_protocol} |
     {:error, :no_beam_info}
   def consolidate(protocol, types) when is_atom(protocol) do
-    with {:ok, info} <- beam_protocol(protocol),
-         {:ok, code, docs} <- change_debug_info(info, types),
-         do: compile(code, docs)
+    with {:ok, ast_info, chunks_info} <- beam_protocol(protocol),
+         {:ok, code} <- change_debug_info(ast_info, types),
+         do: compile(protocol, code, chunks_info)
   end
 
-  @docs_chunk 'ExDc'
-
   defp beam_protocol(protocol) do
-    chunk_ids = [:abstract_code, :attributes, @docs_chunk]
+    chunk_ids = [:abstract_code, :attributes, :compile_info, 'ExDc']
     opts = [:allow_missing_chunks]
     case :beam_lib.chunks(beam_file(protocol), chunk_ids, opts) do
       {:ok, {^protocol, [{:abstract_code, {_raw, abstract_code}},
                          {:attributes, attributes},
-                         {@docs_chunk, docs}]}} ->
+                         {:compile_info, compile_info},
+                         {'ExDc', docs}]}} ->
         case attributes[:protocol] do
           [fallback_to_any: any] ->
-            {:ok, {protocol, any, abstract_code, docs}}
+            {:ok, {protocol, any, abstract_code}, {compile_info, docs}}
           _ ->
             {:error, :not_a_protocol}
         end
@@ -288,29 +287,33 @@ defmodule Protocol do
 
   # Change the debug information to the optimized
   # impl_for/1 dispatch version.
-  defp change_debug_info({protocol, any, code, docs}, types) do
+  defp change_debug_info({protocol, any, code}, types) do
     types   = if any, do: types, else: List.delete(types, Any)
     all     = [Any] ++ for {_guard, mod} <- __builtin__(), do: mod
     structs = types -- all
     case change_impl_for(code, protocol, types, structs, false, []) do
-      {:ok, ret} -> {:ok, ret, docs}
+      {:ok, ret} -> {:ok, ret}
       other      -> other
     end
   end
 
-  defp change_impl_for([{:function, line, :__protocol__, 1, clauses} | t], protocol, types, structs, _, acc) do
+  defp change_impl_for([{:function, line, :__protocol__, 1, clauses} | tail], protocol, types, structs, _, acc) do
+    abstract_types = :erl_parse.abstract(:lists.usort(types))
+
     clauses = :lists.map(fn
       {:clause, l, [{:atom, _, :consolidated?}], [], [{:atom, _, _}]} ->
         {:clause, l, [{:atom, 0, :consolidated?}], [], [{:atom, 0, true}]}
+      {:clause, l, [{:atom, _, :impls}], [], [{:atom, _, _}]} ->
+        {:clause, l, [{:atom, 0, :impls}], [], [{:tuple, 0, [{:atom, 0, :consolidated}, abstract_types]}]}
       {:clause, _, _, _, _} = c ->
         c
     end, clauses)
 
-    change_impl_for(t, protocol, types, structs, true,
+    change_impl_for(tail, protocol, types, structs, true,
                     [{:function, line, :__protocol__, 1, clauses} | acc])
   end
 
-  defp change_impl_for([{:function, line, :impl_for, 1, _} | t], protocol, types, structs, is_protocol, acc) do
+  defp change_impl_for([{:function, line, :impl_for, 1, _} | tail], protocol, types, structs, protocol?, acc) do
     fallback = if Any in types, do: load_impl(protocol, Any)
 
     clauses = for {guard, mod} <- __builtin__(),
@@ -320,26 +323,45 @@ defmodule Protocol do
     clauses = [struct_clause_for(line) | clauses] ++
               [fallback_clause_for(fallback, protocol, line)]
 
-    change_impl_for(t, protocol, types, structs, is_protocol,
+    change_impl_for(tail, protocol, types, structs, protocol?,
                     [{:function, line, :impl_for, 1, clauses} | acc])
   end
 
-  defp change_impl_for([{:function, line, :struct_impl_for, 1, _} | t], protocol, types, structs, is_protocol, acc) do
+  defp change_impl_for([{:function, line, :struct_impl_for, 1, _} | tail], protocol, types, structs, protocol?, acc) do
     fallback = if Any in types, do: load_impl(protocol, Any)
     clauses = for struct <- structs, do: each_struct_clause_for(struct, protocol, line)
     clauses = clauses ++ [fallback_clause_for(fallback, protocol, line)]
 
-    change_impl_for(t, protocol, types, structs, is_protocol,
+    change_impl_for(tail, protocol, types, structs, protocol?,
                     [{:function, line, :struct_impl_for, 1, clauses} | acc])
   end
 
-  defp change_impl_for([h | t], protocol, info, types, is_protocol, acc) do
-    change_impl_for(t, protocol, info, types, is_protocol, [h | acc])
+  defp change_impl_for([{:attribute, line, :spec, {{:__protocol__, 1}, funspecs}} | tail], protocol, types, structs, protocol?, acc) do
+    new_specs = for spec <- funspecs do
+      case spec do
+        {:type, line, :fun, [{:type, _, :product, [{:atom, _, :consolidated?}]}, _]} ->
+          {:type, line, :fun,
+           [{:type, line, :product, [{:atom, 0, :consolidated?}]},
+            {:atom, 0, true}]}
+        {:type, line, :fun, [{:type, _, :product, [{:atom, _, :impls}]}, _]} ->
+          {:type, line, :fun,
+           [{:type, line, :product, [{:atom, 0, :impls}]},
+            {:type, 0, :tuple,
+             [{:atom, 0, :consolidated},
+              {:type, 0, :list, [{:type, 0, :module, []}]}]}]}
+        other -> other
+      end
+    end
+    change_impl_for(tail, protocol, types, structs, protocol?, [{:attribute, line, :spec, {{:__protocol__, 1}, new_specs}} | acc])
   end
 
-  defp change_impl_for([], protocol, _info, _types, is_protocol, acc) do
-    if is_protocol do
-      {:ok, {protocol, Enum.reverse(acc)}}
+  defp change_impl_for([head | tail], protocol, info, types, protocol?, acc) do
+    change_impl_for(tail, protocol, info, types, protocol?, [head | acc])
+  end
+
+  defp change_impl_for([], _protocol, _info, _types, protocol?, acc) do
+    if protocol? do
+      {:ok, Enum.reverse(acc)}
     else
       {:error, :not_a_protocol}
     end
@@ -369,9 +391,9 @@ defmodule Protocol do
           [{:var, line, :x}]}]}
   end
 
-  defp each_struct_clause_for(other, protocol, line) do
-    {:clause, line, [{:atom, line, other}], [],
-      [{:atom, line, load_impl(protocol, other)}]}
+  defp each_struct_clause_for(struct, protocol, line) do
+    {:clause, line, [{:atom, line, struct}], [],
+      [{:atom, line, load_impl(protocol, struct)}]}
   end
 
   defp fallback_clause_for(value, _protocol, line) do
@@ -384,13 +406,14 @@ defmodule Protocol do
   end
 
   # Finally compile the module and emit its bytecode.
-  defp compile({protocol, code}, docs) do
-    opts = if Code.compiler_options[:debug_info], do: [:debug_info], else: []
+  defp compile(protocol, code, {compile_info, docs}) do
+    opts = Keyword.take(compile_info, [:source])
+    opts = if Code.compiler_options[:debug_info], do: [:debug_info | opts], else: opts
     {:ok, ^protocol, binary, _warnings} = :compile.forms(code, [:return | opts])
     {:ok,
       case docs do
         :missing_chunk -> binary
-        _ -> :elixir_module.add_beam_chunk(binary, @docs_chunk, docs)
+        _ -> :elixir_erl.add_beam_chunks(binary, [{"ExDc", docs}])
       end}
   end
 
@@ -504,18 +527,20 @@ defmodule Protocol do
       @doc false
       @spec __protocol__(:module) :: __MODULE__
       @spec __protocol__(:functions) :: unquote(Protocol.__functions_spec__(@functions))
-      @spec __protocol__(:consolidated?) :: boolean
+      @spec __protocol__(:consolidated?) :: false
+      @spec __protocol__(:impls) :: :not_consolidated
       Kernel.def __protocol__(:module), do: __MODULE__
       Kernel.def __protocol__(:functions), do: unquote(:lists.sort(@functions))
       Kernel.def __protocol__(:consolidated?), do: false
+      Kernel.def __protocol__(:impls), do: :not_consolidated
     end
   end
 
   @doc false
   def __functions_spec__([]),
     do: []
-  def __functions_spec__([h | t]),
-    do: [:lists.foldl(&{:|, [], [&1, &2]}, h, t), quote(do: ...)]
+  def __functions_spec__([head | tail]),
+    do: [:lists.foldl(&{:|, [], [&1, &2]}, head, tail), quote(do: ...)]
 
   @doc false
   def __impl__(protocol, opts) do
@@ -556,8 +581,8 @@ defmodule Protocol do
 
         unquote(block)
 
-        Module.register_attribute(__MODULE__, :impl, persist: true)
-        @impl [protocol: @protocol, for: @for]
+        Module.register_attribute(__MODULE__, :protocol_impl, persist: true)
+        @protocol_impl [protocol: @protocol, for: @for]
 
         unquote(impl)
       end
@@ -601,8 +626,8 @@ defmodule Protocol do
           apply(mod, fun, args)
         else
           Module.create(Module.concat(protocol, for), quote do
-            Module.register_attribute(__MODULE__, :impl, persist: true)
-            @impl [protocol: unquote(protocol), for: unquote(for)]
+            Module.register_attribute(__MODULE__, :protocol_impl, persist: true)
+            @protocol_impl [protocol: unquote(protocol), for: unquote(for)]
 
             @doc false
             @spec __impl__(:target) :: unquote(impl)
@@ -633,9 +658,9 @@ defmodule Protocol do
 
     specs = Module.get_attribute(module, :spec)
     found =
-      :lists.map(fn {:spec, expr, caller} ->
+      :lists.map(fn {:spec, expr, pos} ->
         if Kernel.Typespec.spec_to_signature(expr) == signature do
-          Kernel.Typespec.define_spec(:callback, expr, caller)
+          Module.store_typespec(module, :callback, {:callback, expr, pos})
           true
         end
       end, specs)

lib/elixir/lib/range.ex

@@ -5,11 +5,11 @@ defmodule Range do
   A range represents a discrete number of values where
   the first and last values are integers.
 
-  Ranges can be either increasing (first <= last) or
-  decreasing (first > last). Ranges are also always
+  Ranges can be either increasing (`first <= last`) or
+  decreasing (`first > last`). Ranges are also always
   inclusive.
 
-  A Range is represented internally as a struct. However,
+  A range is represented internally as a struct. However,
   the most common form of creating and matching on ranges
   is via the `../2` macro, auto-imported from `Kernel`:
 
@@ -21,8 +21,9 @@ defmodule Range do
       iex> last
       3
 
-  A Range implements the Enumerable protocol, which means
-  all of the functions in the Enum module is available:
+  A range implements the `Enumerable` protocol, which means
+  functions in the `Enum` module can be used to work with
+  ranges:
 
       iex> range = 1..10
       1..10
@@ -77,23 +78,23 @@ end
 
 defimpl Enumerable, for: Range do
   def reduce(first..last, acc, fun) do
-    reduce(first, last, acc, fun, last >= first)
+    reduce(first, last, acc, fun, _up? = last >= first)
   end
 
-  defp reduce(_x, _y, {:halt, acc}, _fun, _up) do
+  defp reduce(_x, _y, {:halt, acc}, _fun, _up?) do
     {:halted, acc}
   end
 
-  defp reduce(x, y, {:suspend, acc}, fun, up) do
-    {:suspended, acc, &reduce(x, y, &1, fun, up)}
+  defp reduce(x, y, {:suspend, acc}, fun, up?) do
+    {:suspended, acc, &reduce(x, y, &1, fun, up?)}
   end
 
-  defp reduce(x, y, {:cont, acc}, fun, true) when x <= y do
-    reduce(x + 1, y, fun.(x, acc), fun, true)
+  defp reduce(x, y, {:cont, acc}, fun, _up? = true) when x <= y do
+    reduce(x + 1, y, fun.(x, acc), fun, _up? = true)
   end
 
-  defp reduce(x, y, {:cont, acc}, fun, false) when x >= y do
-    reduce(x - 1, y, fun.(x, acc), fun, false)
+  defp reduce(x, y, {:cont, acc}, fun, _up? = false) when x >= y do
+    reduce(x - 1, y, fun.(x, acc), fun, _up? = false)
   end
 
   defp reduce(_, _, {:cont, acc}, _fun, _up) do

lib/elixir/lib/record.ex

@@ -71,6 +71,7 @@ defmodule Record do
        uid: :undefined, gid: :undefined]
 
   """
+  @spec extract(name :: atom, keyword) :: keyword
   def extract(name, opts) when is_atom(name) and is_list(opts) do
     Record.Extractor.extract(name, opts)
   end
@@ -99,6 +100,7 @@ defmodule Record do
   These options are expected to be literals (including the binary values) at
   compile time.
   """
+  @spec extract_all(keyword) :: [{name :: atom, keyword}]
   def extract_all(opts) when is_list(opts) do
     Record.Extractor.extract_all(opts)
   end
@@ -222,16 +224,17 @@ defmodule Record do
       user(name: name) = record
       name #=> "meg"
 
-  By default, Elixir uses the record name as the first element of the tuple (the
-  "tag"). However, a different tag can be specified when defining a record:
+  By default, Elixir uses the record name as the first element of the tuple (the "tag").
+  However, a different tag can be specified when defining a record,
+  as in the following example, in which we use `Customer` as the second argument of `defrecord/3`:
 
       defmodule User do
         require Record
-        Record.defrecord :user, User, name: nil
+        Record.defrecord :user, Customer, name: nil
       end
 
       require User
-      User.user() #=> {User, nil}
+      User.user() #=> {Customer, nil}
 
   ## Defining extracted records with anonymous functions in the values
 
@@ -291,14 +294,14 @@ defmodule Record do
   @doc false
   def __fields__(type, fields) do
     :lists.map(fn
-      {key, val} when is_atom(key) ->
+      {key, value} when is_atom(key) ->
         try do
-          Macro.escape(val)
+          Macro.escape(value)
         rescue
           e in [ArgumentError] ->
             raise ArgumentError, "invalid value for record field #{key}, " <> Exception.message(e)
         else
-          val -> {key, val}
+          value -> {key, value}
         end
       key when is_atom(key) ->
         {key, nil}
@@ -309,51 +312,51 @@ defmodule Record do
 
   # Callback invoked from record/0 and record/1 macros.
   @doc false
-  def __access__(atom, fields, args, caller) do
+  def __access__(tag, fields, args, caller) do
     cond do
       is_atom(args) ->
-        index(atom, fields, args)
+        index(tag, fields, args)
       Keyword.keyword?(args) ->
-        create(atom, fields, args, caller)
+        create(tag, fields, args, caller)
       true ->
         fields = Macro.escape(fields)
         case Macro.expand(args, caller) do
-          {:{}, _, [^atom | list]} when length(list) == length(fields) ->
-            record = List.to_tuple([atom | list])
-            Record.__keyword__(atom, fields, record)
-          {^atom, arg} when length(fields) == 1 ->
-            Record.__keyword__(atom, fields, {atom, arg})
+          {:{}, _, [^tag | list]} when length(list) == length(fields) ->
+            record = List.to_tuple([tag | list])
+            Record.__keyword__(tag, fields, record)
+          {^tag, arg} when length(fields) == 1 ->
+            Record.__keyword__(tag, fields, {tag, arg})
           _ ->
-            quote do: Record.__keyword__(unquote(atom), unquote(fields), unquote(args))
+            quote do: Record.__keyword__(unquote(tag), unquote(fields), unquote(args))
         end
     end
   end
 
   # Callback invoked from the record/2 macro.
   @doc false
-  def __access__(atom, fields, record, args, caller) do
+  def __access__(tag, fields, record, args, caller) do
     cond do
       is_atom(args) ->
-        get(atom, fields, record, args)
+        get(tag, fields, record, args)
       Keyword.keyword?(args) ->
-        update(atom, fields, record, args, caller)
+        update(tag, fields, record, args, caller)
       true ->
-        msg = "expected arguments to be a compile time atom or keywords, got: #{Macro.to_string args}"
+        msg = "expected arguments to be a compile time atom or a keyword list, got: #{Macro.to_string args}"
         raise ArgumentError, msg
     end
   end
 
   # Gets the index of field.
-  defp index(atom, fields, field) do
+  defp index(tag, fields, field) do
     if index = find_index(fields, field, 0) do
       index - 1 # Convert to Elixir index
     else
-      raise ArgumentError, "record #{inspect atom} does not have the key: #{inspect field}"
+      raise ArgumentError, "record #{inspect tag} does not have the key: #{inspect field}"
     end
   end
 
   # Creates a new record with the given default fields and keyword values.
-  defp create(atom, fields, keyword, caller) do
+  defp create(tag, fields, keyword, caller) do
     in_match = Macro.Env.in_match?(caller)
     keyword = apply_underscore(fields, keyword)
 
@@ -371,15 +374,15 @@ defmodule Record do
 
     case remaining do
       [] ->
-        {:{}, [], [atom | match]}
+        {:{}, [], [tag | match]}
       _  ->
         keys = for {key, _} <- remaining, do: key
-        raise ArgumentError, "record #{inspect atom} does not have the key: #{inspect hd(keys)}"
+        raise ArgumentError, "record #{inspect tag} does not have the key: #{inspect hd(keys)}"
     end
   end
 
   # Updates a record given by var with the given keyword.
-  defp update(atom, fields, var, keyword, caller) do
+  defp update(tag, fields, var, keyword, caller) do
     if Macro.Env.in_match?(caller) do
       raise ArgumentError, "cannot invoke update style macro inside match"
     end
@@ -393,20 +396,20 @@ defmodule Record do
           :erlang.setelement(unquote(index), unquote(acc), unquote(value))
         end
       else
-        raise ArgumentError, "record #{inspect atom} does not have the key: #{inspect key}"
+        raise ArgumentError, "record #{inspect tag} does not have the key: #{inspect key}"
       end
     end
   end
 
   # Gets a record key from the given var.
-  defp get(atom, fields, var, key) do
+  defp get(tag, fields, var, key) do
     index = find_index(fields, key, 0)
     if index do
       quote do
         :erlang.element(unquote(index), unquote(var))
       end
     else
-      raise ArgumentError, "record #{inspect atom} does not have the key: #{inspect key}"
+      raise ArgumentError, "record #{inspect tag} does not have the key: #{inspect key}"
     end
   end
 
@@ -416,18 +419,18 @@ defmodule Record do
 
   # Returns a keyword list of the record
   @doc false
-  def __keyword__(atom, fields, record) do
-    if is_record(record, atom) do
+  def __keyword__(tag, fields, record) do
+    if is_record(record, tag) do
       [_tag | values] = Tuple.to_list(record)
       case join_keyword(fields, values, []) do
         kv when is_list(kv) ->
           kv
         expected_fields ->
-          msg = "expected argument to be a #{inspect atom} record with #{expected_fields} fields, got: #{inspect record}"
+          msg = "expected argument to be a #{inspect tag} record with #{expected_fields} fields, got: #{inspect record}"
           raise ArgumentError, msg
       end
     else
-      msg = "expected argument to be a literal atom, literal keyword or a #{inspect atom} record, got runtime: #{inspect record}"
+      msg = "expected argument to be a literal atom, literal keyword or a #{inspect tag} record, got runtime: #{inspect record}"
       raise ArgumentError, msg
     end
   end

lib/elixir/lib/regex.ex

@@ -21,7 +21,6 @@ defmodule Regex do
 
   A Regex is represented internally as the `Regex` struct. Therefore,
   `%Regex{}` can be used whenever there is a need to match on them.
-
   Keep in mind it is not guaranteed two regular expressions from the
   same source are equal, for example:
 
@@ -715,14 +714,32 @@ defmodule Regex do
   """
   @spec escape(String.t) :: String.t
   def escape(string) when is_binary(string) do
-    escape(string, [])
+    string
+    |> escape(_length = 0, string)
+    |> IO.iodata_to_binary
   end
 
-  for char <- '.^$*+?()[]{}|#-\\\t\n\v\f\r\s' do
-    defp escape(<<unquote(char), rest::binary>>, acc), do: escape(rest, [acc, ?\\, unquote(char)])
+  @escapable '.^$*+?()[]{}|#-\\\t\n\v\f\r\s'
+
+  defp escape(<<char, rest::binary>>, length, original) when char in @escapable do
+    escape_char(rest, length, original, char)
+  end
+
+  defp escape(<<_, rest::binary>>, length, original) do
+    escape(rest, length + 1, original)
+  end
+
+  defp escape(<<>>, _length, original) do
+    original
+  end
+
+  defp escape_char(<<rest::binary>>, 0, _original, char) do
+    [?\\, char | escape(rest, 0, rest)]
+  end
+
+  defp escape_char(<<rest::binary>>, length, original, char) do
+    [binary_part(original, 0, length), ?\\, char | escape(rest, 0, rest)]
   end
-  defp escape(<<char, rest::binary>>, acc), do: escape(rest, [acc, char])
-  defp escape(<<>>, acc), do: IO.iodata_to_binary(acc)
 
   # Helpers
 

lib/elixir/lib/registry.ex

@@ -23,11 +23,11 @@ defmodule Registry do
 
   ## Using in `:via`
 
-  Once the registry is started with a given name (using
-  `Registry.start_link/2`), it can be used to register and access named
+  Once the registry is started with a given name using
+  `Registry.start_link/1`, it can be used to register and access named
   processes using the `{:via, Registry, {registry, key}}` tuple:
 
-      {:ok, _} = Registry.start_link(:unique, Registry.ViaTest)
+      {:ok, _} = Registry.start_link(keys: :unique, name: Registry.ViaTest)
       name = {:via, Registry, {Registry.ViaTest, "agent"}}
       {:ok, _} = Agent.start_link(fn -> 0 end, name: name)
       Agent.get(name, & &1)
@@ -38,7 +38,7 @@ defmodule Registry do
 
   Typically the registry is started as part of a supervision tree though:
 
-      supervisor(Registry, [:unique, Registry.ViaTest])
+      {Registry, keys: :unique, name: Registry.ViaTest}
 
   Only registries with unique keys can be used in `:via`. If the name is
   already taken, the case-specific `start_link` function (`Agent.start_link/2`
@@ -50,7 +50,7 @@ defmodule Registry do
   dispatch logic triggered from the caller. For example, let's say we have a
   duplicate registry started as so:
 
-      {:ok, _} = Registry.start_link(:duplicate, Registry.DispatcherTest)
+      {:ok, _} = Registry.start_link(keys: :duplicate, name: Registry.DispatcherTest)
 
   By calling `register/3`, different processes can register under a given key
   and associate any value under that key. In this case, let's register the
@@ -109,10 +109,9 @@ defmodule Registry do
 
   In this example, we will also set the number of partitions to the number of
   schedulers online, which will make the registry more performant on highly
-  concurrent environments as each partition will spawn a new process, allowing
-  dispatching to happen in parallel:
+  concurrent environments:
 
-      {:ok, _} = Registry.start_link(:duplicate, Registry.PubSubTest,
+      {:ok, _} = Registry.start_link(keys: :duplicate, name: Registry.PubSubTest,
                                      partitions: System.schedulers_online)
       {:ok, _} = Registry.register(Registry.PubSubTest, "hello", [])
       Registry.dispatch(Registry.PubSubTest, "hello", fn entries ->
@@ -150,9 +149,7 @@ defmodule Registry do
   Note that the registry uses one ETS table plus two ETS tables per partition.
   """
 
-  # TODO: Decide if it should be started as part of Elixir's supervision tree.
-
-  @kind [:unique, :duplicate]
+  @keys [:unique, :duplicate]
   @all_info -1
   @key_info -2
 
@@ -160,7 +157,7 @@ defmodule Registry do
   @type registry :: atom
 
   @typedoc "The type of the registry"
-  @type kind :: :unique | :duplicate
+  @type keys :: :unique | :duplicate
 
   @typedoc "The type of keys allowed on registration"
   @type key :: term
@@ -220,25 +217,36 @@ defmodule Registry do
 
   Manually it can be started as:
 
-      Registry.start_link(:unique, MyApp.Registry)
+      Registry.start_link(keys: :unique, name: MyApp.Registry)
 
   In your supervisor tree, you would write:
 
-      supervisor(Registry, [:unique, MyApp.Registry])
+      Supervisor.start_link([
+        {Registry, keys: :unique, name: MyApp.Registry}
+      ])
 
   For intensive workloads, the registry may also be partitioned (by specifying
   the `:partitions` option). If partitioning is required then a good default is to
   set the number of partitions to the number of schedulers available:
 
-      Registry.start_link(:unique, MyApp.Registry, partitions: System.schedulers_online())
+      Registry.start_link(keys: :unique, name: MyApp.Registry,
+                          partitions: System.schedulers_online())
 
   or:
 
-      supervisor(Registry, [:unique, MyApp.Registry, [partitions: System.schedulers_online()]])
+      Supervisor.start_link([
+        {Registry, keys: :unique, name: MyApp.Registry,
+                   partitions: System.schedulers_online()}
+      ])
 
   ## Options
 
-  The registry supports the following options:
+  The registry requires the following keys:
+
+    * `:keys` - choose if keys are `:unique` or `:duplicate`
+    * `:name` - the name of the registry and its tables
+
+  The following keys are optional:
 
     * `:partitions` - the number of partitions in the registry. Defaults to `1`.
     * `:listeners` - a list of named processes which are notified of `:register`
@@ -248,9 +256,19 @@ defmodule Registry do
     * `:meta` - a keyword list of metadata to be attached to the registry.
 
   """
-  @spec start_link(kind, registry, options) :: {:ok, pid} | {:error, term}
-        when options: [partitions: pos_integer, listeners: [atom], meta: [{meta_key, meta_value}]]
-  def start_link(kind, registry, options \\ []) when kind in @kind and is_atom(registry) do
+  @spec start_link(keys: keys, name: registry, partitions: pos_integer, listeners: [atom],
+                   meta: [{meta_key, meta_value}]) :: {:ok, pid} | {:error, term}
+  def start_link(options) do
+    keys = Keyword.get(options, :keys)
+    unless keys in @keys do
+      raise ArgumentError, "expected :keys to be given and be one of :unique or :duplicate, got: #{inspect keys}"
+    end
+
+    name = Keyword.get(options, :name)
+    unless is_atom(name) do
+      raise ArgumentError, "expected :name to be given and to be an atom, got: #{inspect name}"
+    end
+
     meta = Keyword.get(options, :meta, [])
     unless Keyword.keyword?(meta) do
       raise ArgumentError, "expected :meta to be a keyword list, got: #{inspect meta}"
@@ -267,9 +285,29 @@ defmodule Registry do
     end
 
     # The @info format must be kept in sync with Registry.Partition optimization.
-    entries = [{@all_info, {kind, partitions, nil, nil, listeners}},
-               {@key_info, {kind, partitions, nil}} | meta]
-    Registry.Supervisor.start_link(kind, registry, partitions, listeners, entries)
+    entries = [{@all_info, {keys, partitions, nil, nil, listeners}},
+               {@key_info, {keys, partitions, nil}} | meta]
+    Registry.Supervisor.start_link(keys, name, partitions, listeners, entries)
+  end
+
+  @doc """
+  Starts the registry as a supervisor process.
+
+  Similar to `start_link/1` except the required options,
+  `keys` and `name` are given as arguments.
+  """
+  @spec start_link(keys, registry, keyword) :: {:ok, pid} | {:error, term}
+  def start_link(keys, name, options \\ []) when keys in @keys and is_atom(name) do
+    start_link([keys: keys, name: name] ++ options)
+  end
+
+  @doc false
+  def child_spec(opts) do
+    %{
+      id: Keyword.get(opts, :name, Registry),
+      start: {Registry, :start_link, [opts]},
+      type: :supervisor
+    }
   end
 
   @doc """
@@ -323,17 +361,16 @@ defmodule Registry do
   associated to the pid. If there are no entries for the given key,
   the callback is never invoked.
 
-  If the registry is not partitioned, the callback is invoked in the process
-  that calls `dispatch/3`. If the registry is partitioned, the callback is
-  invoked concurrently per partition by starting a task linked to the
-  caller. The callback, however, is only invoked if there are entries for that
-  partition.
+  If the registry is partitioned, the callback is invoked multiple times
+  per partition. If the registry is partitioned and `parallel: true` is
+  given as an option, the dispatching happens in parallel. In both cases,
+  the callback is only invoked if there are entries for that partition.
 
   See the module documentation for examples of using the `dispatch/3`
   function for building custom dispatching or a pubsub system.
   """
-  @spec dispatch(registry, key, (entries :: [{pid, value}] -> term)) :: :ok
-  def dispatch(registry, key, mfa_or_fun)
+  @spec dispatch(registry, key, (entries :: [{pid, value}] -> term), keyword) :: :ok
+  def dispatch(registry, key, mfa_or_fun, opts \\ [])
       when is_atom(registry) and is_function(mfa_or_fun, 1)
       when is_atom(registry) and tuple_size(mfa_or_fun) == 3 do
     case key_info!(registry) do
@@ -347,17 +384,33 @@ defmodule Registry do
         |> safe_lookup_second(key)
         |> apply_non_empty_to_mfa_or_fun(mfa_or_fun)
       {:duplicate, partitions, _} ->
-        registry
-        |> dispatch_task(key, mfa_or_fun, partitions)
-        |> Enum.each(&Task.await(&1, :infinity))
+        if Keyword.get(opts, :parallel, false) do
+          registry
+          |> dispatch_parallel(key, mfa_or_fun, partitions)
+          |> Enum.each(&Task.await(&1, :infinity))
+        else
+          dispatch_serial(registry, key, mfa_or_fun, partitions)
+        end
     end
     :ok
   end
 
-  defp dispatch_task(_registry, _key, _mfa_or_fun, 0) do
+  defp dispatch_serial(_registry, _key, _mfa_or_fun, 0) do
+    :ok
+  end
+  defp dispatch_serial(registry, key, mfa_or_fun, partition) do
+    partition = partition - 1
+    registry
+    |> key_ets!(partition)
+    |> safe_lookup_second(key)
+    |> apply_non_empty_to_mfa_or_fun(mfa_or_fun)
+    dispatch_serial(registry, key, mfa_or_fun, partition)
+  end
+
+  defp dispatch_parallel(_registry, _key, _mfa_or_fun, 0) do
     []
   end
-  defp dispatch_task(registry, key, mfa_or_fun, partition) do
+  defp dispatch_parallel(registry, key, mfa_or_fun, partition) do
     partition = partition - 1
     parent = self()
     task = Task.async(fn ->
@@ -368,7 +421,7 @@ defmodule Registry do
       Process.unlink(parent)
       :ok
     end)
-    [task | dispatch_task(registry, key, mfa_or_fun, partition)]
+    [task | dispatch_parallel(registry, key, mfa_or_fun, partition)]
   end
 
   defp apply_non_empty_to_mfa_or_fun([], _mfa_or_fun) do
@@ -446,6 +499,12 @@ defmodule Registry do
   value or tuple element, while :"$1" can be used to temporarily assign part
   of pattern to a variable for a subsequent comparison.
 
+  It is possible to pass list of guard conditions for more precise matching.
+  Each guard is a tuple, which describes check that should be passed by assigned part of pattern.
+  For example :"$1" > 1 guard condition would be expressed as {:>, :"$1", 1} tuple.
+  Please note that guard conditions will work only for assigned variables like :"$1", :"$2", etc.
+  Avoid usage of special match variables :"$_" and :"$$", because it might not work as expected.
+
   An empty list will be returned if there is no match.
 
   For unique registries, a single partition lookup is necessary. For
@@ -467,11 +526,16 @@ defmodule Registry do
       [{self(), {1, :atom, 1}}, {self(), {2, :atom, 2}}]
       iex> Registry.match(Registry.MatchTest, "hello", {:"$1", :_, :"$1"}) |> Enum.sort()
       [{self(), {1, :atom, 1}}, {self(), {2, :atom, 2}}]
+      iex> Registry.match(Registry.MatchTest, "hello", {:_, :_, :"$1"}, [{:>, :"$1", 1}])
+      [{self(), {2, :atom, 2}}]
+      iex> Registry.match(Registry.MatchTest, "hello", {:_, :"$1", :_}, [{:is_atom, :"$1"}]) |> Enum.sort()
+      [{self(), {1, :atom, 1}}, {self(), {2, :atom, 2}}]
 
   """
-  @spec match(registry, key, match_pattern :: atom() | tuple()) :: [{pid, term}]
-  def match(registry, key, pattern) when is_atom(registry) do
-    spec = [{{key, {:_, pattern}}, [], [{:element, 2, :"$_"}]}]
+  @spec match(registry, key, match_pattern :: atom() | tuple(), guards :: list()) :: [{pid, term}]
+  def match(registry, key, pattern, guards \\ []) when is_atom(registry) and is_list(guards) do
+    guards = [{:"=:=", {:element, 1, :"$_"}, {:const, key}} | guards]
+    spec = [{{:_, {:_, pattern}}, guards, [{:element, 2, :"$_"}]}]
 
     case key_info!(registry) do
       {:unique, partitions, key_ets} ->
@@ -524,7 +588,16 @@ defmodule Registry do
   def keys(registry, pid) when is_atom(registry) and is_pid(pid) do
     {kind, partitions, _, pid_ets, _} = info!(registry)
     {_, pid_ets} = pid_ets || pid_ets!(registry, pid, partitions)
-    keys = safe_lookup_second(pid_ets, pid)
+
+    keys = try do
+      spec = [{{pid, :'$1', :'$2'}, [], [{{:'$1', :'$2'}}]}]
+      :ets.select(pid_ets, spec)
+    catch
+      :error, :badarg -> []
+    end
+
+    # Handle the possibility of fake keys
+    keys = gather_keys(keys, [], false)
 
     cond do
       kind == :unique -> Enum.uniq(keys)
@@ -532,6 +605,19 @@ defmodule Registry do
     end
   end
 
+  defp gather_keys([{key, {_, remaining}} | rest], acc, _fake) do
+    gather_keys(rest, [key | acc], {key, remaining})
+  end
+  defp gather_keys([{key, _} | rest], acc, fake) do
+    gather_keys(rest, [key | acc], fake)
+  end
+  defp gather_keys([], acc, {key, remaining}) do
+    List.duplicate(key, remaining) ++ Enum.reject(acc, & &1 === key)
+  end
+  defp gather_keys([], acc, false) do
+    acc
+  end
+
   @doc """
   Unregisters all entries for the given `key` associated to the current
   process in `registry`.
@@ -542,7 +628,7 @@ defmodule Registry do
 
   ## Examples
 
-  It unregister all entries for `key` for unique registries:
+  For unique registries:
 
       iex> Registry.start_link(:unique, Registry.UniqueUnregisterTest)
       iex> Registry.register(Registry.UniqueUnregisterTest, "hello", :world)
@@ -553,7 +639,7 @@ defmodule Registry do
       iex> Registry.keys(Registry.UniqueUnregisterTest, self())
       []
 
-  As well as duplicate registries:
+  For duplicate registries:
 
       iex> Registry.start_link(:duplicate, Registry.DuplicateUnregisterTest)
       iex> Registry.register(Registry.DuplicateUnregisterTest, "hello", :world)
@@ -589,6 +675,95 @@ defmodule Registry do
     :ok
   end
 
+  @doc """
+  Unregister entries for a given key matching a pattern.
+
+  ## Examples
+
+  For unique registries it can be used to conditionally unregister a key on
+  the basis of whether or not it matches a particular value.
+
+      iex> Registry.start_link(:unique, Registry.UniqueUnregisterMatchTest)
+      iex> Registry.register(Registry.UniqueUnregisterMatchTest, "hello", :world)
+      iex> Registry.keys(Registry.UniqueUnregisterMatchTest, self())
+      ["hello"]
+      iex> Registry.unregister_match(Registry.UniqueUnregisterMatchTest, "hello", :foo)
+      :ok
+      iex> Registry.keys(Registry.UniqueUnregisterMatchTest, self())
+      ["hello"]
+      iex> Registry.unregister_match(Registry.UniqueUnregisterMatchTest, "hello", :world)
+      :ok
+      iex> Registry.keys(Registry.UniqueUnregisterMatchTest, self())
+      []
+
+  For duplicate registries:
+
+      iex> Registry.start_link(:duplicate, Registry.DuplicateUnregisterMatchTest)
+      iex> Registry.register(Registry.DuplicateUnregisterMatchTest, "hello", :world_a)
+      iex> Registry.register(Registry.DuplicateUnregisterMatchTest, "hello", :world_b)
+      iex> Registry.register(Registry.DuplicateUnregisterMatchTest, "hello", :world_c)
+      iex> Registry.keys(Registry.DuplicateUnregisterMatchTest, self())
+      ["hello", "hello", "hello"]
+      iex> Registry.unregister_match(Registry.DuplicateUnregisterMatchTest, "hello", :world_a)
+      :ok
+      iex> Registry.keys(Registry.DuplicateUnregisterMatchTest, self())
+      ["hello", "hello"]
+      iex> Registry.lookup(Registry.DuplicateUnregisterMatchTest, "hello")
+      [{self(), :world_b}, {self(), :world_c}]
+  """
+  def unregister_match(registry, key, pattern, guards \\ []) when is_list(guards) do
+    self = self()
+
+    {kind, partitions, key_ets, pid_ets, listeners} = info!(registry)
+    {key_partition, pid_partition} = partitions(kind, key, self, partitions)
+    key_ets = key_ets || key_ets!(registry, key_partition)
+    {pid_server, pid_ets} = pid_ets || pid_ets!(registry, pid_partition)
+
+    # Remove first from the key_ets because in case of crashes
+    # the pid_ets will still be able to clean up. The last step is
+    # to clean if we have no more entries.
+
+    # Here we want to count all entries for this pid under this key, regardless
+    # of pattern.
+    underscore_guard = {:"=:=", {:element, 1, :"$_"}, {:const, key}}
+    total_spec = [{{:_, {self, :_}}, [underscore_guard], [true]}]
+    total = :ets.select_count(key_ets, total_spec)
+
+    # We only want to delete things that match the pattern
+    delete_spec = [{{:_, {self, pattern}}, [underscore_guard | guards] ,[true]}]
+    case :ets.select_delete(key_ets, delete_spec) do
+      # We deleted everything, we can just delete the object
+      ^total ->
+        true = :ets.delete_object(pid_ets, {self, key, key_ets})
+
+        unlink_if_unregistered(pid_server, pid_ets, self)
+
+        for listener <- listeners do
+          Kernel.send(listener, {:unregister, registry, key, self})
+        end
+
+      0 ->
+        :ok
+
+      deleted ->
+        # There are still entries remaining for this pid. delete_object/2 with
+        # duplicate_bag tables will remove every entry, but we only want to
+        # remove those we have deleted. The solution is to introduce a temp_entry
+        # that indicates how many keys WILL be remaining after the delete operation.
+        remaining = total - deleted
+        temp_entry = {self, key, {key_ets, remaining}}
+        true = :ets.insert(pid_ets, temp_entry)
+        true = :ets.delete_object(pid_ets, {self, key, key_ets})
+        real_keys = List.duplicate({self, key, key_ets}, remaining)
+        true = :ets.insert(pid_ets, real_keys)
+        # We've recreated the real remaining key entries, so we can now delete
+        # our temporary entry.
+        true = :ets.delete_object(pid_ets, temp_entry)
+    end
+
+    :ok
+  end
+
   @doc """
   Registers the current process under the given `key` in `registry`.
 
@@ -597,7 +772,7 @@ defmodule Registry do
   lookup.
 
   This function returns `{:ok, owner}` or `{:error, reason}`.
-  The `owner` is the pid is the registry partition responsible for
+  The `owner` is the pid in the registry partition responsible for
   the pid. The owner is automatically linked to the caller.
 
   If the registry has unique keys, it will return `{:ok, owner}` unless
@@ -813,7 +988,7 @@ defmodule Registry.Supervisor do
     true = :ets.insert(registry, entries)
 
     children =
-      for i <- 0..partitions-1 do
+      for i <- 0..partitions - 1 do
         key_partition = Registry.Partition.key_name(registry, i)
         pid_partition = Registry.Partition.pid_name(registry, i)
         arg = {kind, registry, i, partitions, key_partition, pid_partition, listeners}
@@ -839,7 +1014,7 @@ defmodule Registry.Partition do
 
   # This process owns the equivalent key and pid ets tables
   # and is responsible for monitoring processes that map to
-  # the its own pid table.
+  # its own pid table.
   use GenServer
   @all_info -1
   @key_info -2
@@ -913,6 +1088,12 @@ defmodule Registry.Partition do
   def handle_info({:EXIT, pid, _reason}, ets) do
     entries = :ets.take(ets, pid)
     for {_pid, key, key_ets} <- entries do
+      key_ets = case key_ets do
+        # In case the fake key ets is being used. See unregister_match/2.
+        {key_ets, _} -> key_ets
+        _ -> key_ets
+      end
+
       try do
         :ets.match_delete(key_ets, {key, {pid, :_}})
       catch

lib/elixir/lib/set.ex

@@ -6,7 +6,7 @@ defmodule Set do
   """
 
   @type value :: any
-  @type values :: [ value ]
+  @type values :: [value]
   @type t :: map
 
   # TODO: Remove by 2.0

lib/elixir/lib/stream.ex

@@ -85,7 +85,7 @@ defmodule Stream do
   Note the functions in this module are guaranteed to return enumerables.
   Since enumerables can have different shapes (structs, anonymous functions,
   and so on), the functions in this module may return any of those shapes
-  and that this may change at any time. For example, a function that today
+  and this may change at any time. For example, a function that today
   returns an anonymous function may return a struct in future releases.
   """
 
@@ -121,67 +121,63 @@ defmodule Stream do
 
   ## Transformers
 
-  @doc """
-  Shortcut to `chunk(enum, n, n)`.
-  """
-  @spec chunk(Enumerable.t, pos_integer) :: Enumerable.t
+  # Deprecate on v1.7
+  @doc false
   def chunk(enum, n), do: chunk(enum, n, n, nil)
 
-  @doc """
-  Streams the enumerable in chunks, containing `n` items each, where
-  each new chunk starts `step` elements into the enumerable.
+  # Deprecate on v1.7
+  @doc false
+  def chunk(enum, n, step, leftover \\ nil)
+      when is_integer(n) and n > 0 and is_integer(step) and step > 0 do
+    chunk_every(enum, n, step, leftover || :discard)
+  end
 
-  `step` is optional and, if not passed, defaults to `n`, i.e.
-  chunks do not overlap. If the final chunk does not have `n`
-  elements to fill the chunk, elements are taken as necessary
-  from `leftover` if it was passed. If `leftover` is passed and
-  does not have enough elements to fill the chunk, then the chunk is
-  returned anyway with less than `n` elements. If `leftover` is not
-  passed at all or is `nil`, then the partial chunk is discarded
-  from the result.
+  @doc """
+  Shortcut to `chunk_every(enum, count, count)`.
+  """
+  @spec chunk_every(Enumerable.t, pos_integer) :: Enumerable.t
+  def chunk_every(enum, count), do: chunk_every(enum, count, count, [])
+
+  @doc """
+  Streams the enumerable in chunks, containing `count` items each,
+  where each new chunk starts `step` elements into the enumerable.
+
+  `step` is optional and, if not passed, defaults to `count`, i.e.
+  chunks do not overlap.
+
+  If the last chunk does not have `count` elements to fill the chunk,
+  elements are taken from `leftover` to fill in the chunk. If `leftover`
+  does not have enough elements to fill the chunk, then a partial chunk
+  is returned with less than `count` elements.
+
+  If `:discard` is given in `leftover`, the last chunk is discarded
+  unless it has exactly `count` elements.
 
   ## Examples
 
-      iex> Stream.chunk([1, 2, 3, 4, 5, 6], 2) |> Enum.to_list
+      iex> Stream.chunk_every([1, 2, 3, 4, 5, 6], 2) |> Enum.to_list
       [[1, 2], [3, 4], [5, 6]]
 
-      iex> Stream.chunk([1, 2, 3, 4, 5, 6], 3, 2) |> Enum.to_list
+      iex> Stream.chunk_every([1, 2, 3, 4, 5, 6], 3, 2, :discard) |> Enum.to_list
       [[1, 2, 3], [3, 4, 5]]
 
-      iex> Stream.chunk([1, 2, 3, 4, 5, 6], 3, 2, [7]) |> Enum.to_list
+      iex> Stream.chunk_every([1, 2, 3, 4, 5, 6], 3, 2, [7]) |> Enum.to_list
       [[1, 2, 3], [3, 4, 5], [5, 6, 7]]
 
-      iex> Stream.chunk([1, 2, 3, 4, 5, 6], 3, 3, []) |> Enum.to_list
+      iex> Stream.chunk_every([1, 2, 3, 4, 5, 6], 3, 3, []) |> Enum.to_list
       [[1, 2, 3], [4, 5, 6]]
 
   """
-  @spec chunk(Enumerable.t, pos_integer, pos_integer) :: Enumerable.t
-  @spec chunk(Enumerable.t, pos_integer, pos_integer, Enumerable.t | nil) :: Enumerable.t
-  def chunk(enum, n, step, leftover \\ nil)
-      when is_integer(n) and n > 0 and is_integer(step) and step > 0 do
-    limit = :erlang.max(n, step)
-    if is_nil(leftover) do
-      lazy enum, {[], 0}, fn(f1) -> R.chunk(n, step, limit, f1) end
-    else
-      lazy enum, {[], 0},
-           fn(f1) -> R.chunk(n, step, limit, f1) end,
-           &do_chunk(&1, n, leftover, &2)
-     end
-  end
-
-  defp do_chunk(acc(_, {_, 0}, _) = acc, _, _, _) do
-    {:cont, acc}
-  end
-
-  defp do_chunk(acc(h, {buffer, count} = old, t), n, leftover, f1) do
-    buffer = :lists.reverse(buffer, Enum.take(leftover, n - count))
-    next_with_acc(f1, buffer, h, old, t)
+  @spec chunk_every(Enumerable.t, pos_integer, pos_integer, Enumerable.t | :discard) :: Enumerable.t
+  def chunk_every(enum, count, step, leftover \\ [])
+      when is_integer(count) and count > 0 and is_integer(step) and step > 0 do
+    R.chunk_every(&chunk_while/4, enum, count, step, leftover)
   end
 
   @doc """
-  Chunks the `enum` by buffering elements for which `fun` returns
-  the same value and only emit them when `fun` returns a new value
-  or the `enum` finishes.
+  Chunks the `enum` by buffering elements for which `fun` returns the same value.
+
+  Elements are only emitted when `fun` returns a new value or the `enum` finishes.
 
   ## Examples
 
@@ -192,20 +188,54 @@ defmodule Stream do
   """
   @spec chunk_by(Enumerable.t, (element -> any)) :: Enumerable.t
   def chunk_by(enum, fun) do
-    lazy enum, nil,
-         fn(f1) -> R.chunk_by(fun, f1) end,
-         &do_chunk_by(&1, &2)
+    R.chunk_by(&chunk_while/4, enum, fun)
   end
 
-  defp do_chunk_by(acc(_, nil, _) = acc, _f1) do
-    {:cont, acc}
+  @doc """
+  Chunks the `enum` with fine grained control when every chunk is emitted.
+
+  `chunk_fun` receives the current element and the accumulator and
+  must return `{:cont, element, acc}` to emit the given chunk and
+  continue with accumulator or `{:cont, acc}` to not emit any chunk
+  and continue with the return accumulator.
+
+  `after_fun` is invoked when iteration is done and must also return
+  `{:cont, element, acc}` or `{:cont, acc}`.
+
+  ## Examples
+
+      iex> chunk_fun = fn i, acc ->
+      ...>   if rem(i, 2) == 0 do
+      ...>     {:cont, Enum.reverse([i | acc]), []}
+      ...>   else
+      ...>     {:cont, [i | acc]}
+      ...>   end
+      ...> end
+      iex> after_fun = fn
+      ...>   [] -> {:cont, []}
+      ...>   acc -> {:cont, Enum.reverse(acc), []}
+      ...> end
+      iex> stream = Stream.chunk_while(1..10, [], chunk_fun, after_fun)
+      iex> Enum.to_list(stream)
+      [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10]]
+
+  """
+  @spec chunk_while(Enumerable.t, acc,
+                    (element, acc -> {:cont, chunk, acc} | {:cont, acc} | {:halt, acc}),
+                    (acc -> {:cont, chunk, acc} | {:cont, acc})) :: Enumerable.t when chunk: any
+  def chunk_while(enum, acc, chunk_fun, after_fun) do
+    lazy enum, acc,
+         fn(f1) -> R.chunk_while(chunk_fun, f1) end,
+         &after_chunk_while(&1, &2, after_fun)
   end
 
-  defp do_chunk_by(acc(h, {buffer, _}, t), f1) do
-    next_with_acc(f1, :lists.reverse(buffer), h, nil, t)
+  defp after_chunk_while(acc(h, acc, t), f1, after_fun) do
+    case after_fun.(acc) do
+      {:cont, emit, acc} -> next_with_acc(f1, emit, h, acc, t)
+      {:cont, acc} -> {:cont, acc(h, acc, t)}
+    end
   end
 
-
   @doc """
   Creates a stream that only emits elements if they are different from the last emitted element.
 
@@ -235,7 +265,7 @@ defmodule Stream do
 
   """
   @spec dedup_by(Enumerable.t, (element -> term)) :: Enumerable.t
-  def dedup_by(enum, fun) when is_function(fun, 1) do
+  def dedup_by(enum, fun) do
     lazy enum, nil, fn f1 -> R.dedup(fun, f1) end
   end
 
@@ -352,7 +382,7 @@ defmodule Stream do
 
   """
   @spec each(Enumerable.t, (element -> term)) :: Enumerable.t
-  def each(enum, fun) when is_function(fun, 1) do
+  def each(enum, fun) do
     lazy enum, fn(f1) ->
       fn(x, acc) ->
         fun.(x)
@@ -362,8 +392,10 @@ defmodule Stream do
   end
 
   @doc """
-  Creates a stream that will apply the given function on enumeration and
-  flatten the result, but only one level deep.
+  Maps the given `fun` over `enumerable` and flattens the result.
+
+  This function returns a new stream built by appending the result of invoking `fun`
+  on each element of `enumerable` together.
 
   ## Examples
 
@@ -397,20 +429,9 @@ defmodule Stream do
     lazy enum, fn(f1) -> R.filter(fun, f1) end
   end
 
-  @doc """
-  Creates a stream that filters and then maps elements according
-  to given functions.
-
-  Exists for symmetry with `Enum.filter_map/3`.
-
-  ## Examples
-
-      iex> stream = Stream.filter_map(1..6, fn(x) -> rem(x, 2) == 0 end, &(&1 * 2))
-      iex> Enum.to_list(stream)
-      [4, 8, 12]
-
-  """
-  @spec filter_map(Enumerable.t, (element -> as_boolean(term)), (element -> any)) :: Enumerable.t
+  @doc false
+  # TODO: Remove on 2.0
+  # (hard-deprecated in elixir_dispatch)
   def filter_map(enum, filter, mapper) do
     lazy enum, fn(f1) -> R.filter_map(filter, mapper, f1) end
   end
@@ -448,7 +469,7 @@ defmodule Stream do
   is delayed until the stream is executed. See `run/1` for an example.
   """
   @spec into(Enumerable.t, Collectable.t, (term -> term)) :: Enumerable.t
-  def into(enum, collectable, transform \\ fn x -> x end) when is_function(transform, 1) do
+  def into(enum, collectable, transform \\ fn x -> x end) do
     &do_into(enum, collectable, transform, &1, &2)
   end
 
@@ -574,7 +595,8 @@ defmodule Stream do
   @doc """
   Creates a stream that applies the given function to each
   element, emits the result and uses the same result as the accumulator
-  for the next computation.
+  for the next computation. Uses the first element in the enumerable
+  as the starting value.
 
   ## Examples
 
@@ -736,7 +758,7 @@ defmodule Stream do
   @spec transform(Enumerable.t, acc, fun) :: Enumerable.t
         when fun: (element, acc -> {Enumerable.t, acc} | {:halt, acc}),
              acc: any
-  def transform(enum, acc, reducer) when is_function(reducer, 2) do
+  def transform(enum, acc, reducer) do
     &do_transform(enum, fn -> acc end, reducer, &1, &2, nil)
   end
 
@@ -753,8 +775,7 @@ defmodule Stream do
   @spec transform(Enumerable.t, (() -> acc), fun, (acc -> term)) :: Enumerable.t
         when fun: (element, acc -> {Enumerable.t, acc} | {:halt, acc}),
              acc: any
-  def transform(enum, start_fun, reducer, after_fun)
-      when is_function(start_fun, 0) and is_function(reducer, 2) and is_function(after_fun, 1) do
+  def transform(enum, start_fun, reducer, after_fun) do
     &do_transform(enum, start_fun, reducer, &1, &2, after_fun)
   end
 
@@ -903,6 +924,8 @@ defmodule Stream do
   end
 
   @doc false
+  # TODO: Remove on 2.0
+  # (hard-deprecated in elixir_dispatch)
   def uniq(enum, fun) do
     uniq_by(enum, fun)
   end
@@ -950,7 +973,6 @@ defmodule Stream do
       [{1, 3}, {2, 4}, {3, 5}]
 
   """
-  @spec with_index(Enumerable.t) :: Enumerable.t
   @spec with_index(Enumerable.t, integer) :: Enumerable.t
   def with_index(enum, offset \\ 0) do
     lazy enum, offset, fn(f1) -> R.with_index(f1) end
@@ -1116,6 +1138,10 @@ defmodule Stream do
   @spec cycle(Enumerable.t) :: Enumerable.t
   def cycle(enumerable)
 
+  def cycle([]) do
+    raise ArgumentError, "cannot cycle over empty enumerable"
+  end
+
   def cycle(enumerable) when is_list(enumerable) do
     unfold {enumerable, enumerable}, fn
       {source, [h | t]}      -> {h, {source, t}}
@@ -1146,6 +1172,8 @@ defmodule Stream do
       {:stream_cycle, acc} ->
         {:halted, acc}
     else
+      {state, []} when state in [:done, :halted] ->
+        raise ArgumentError, "cannot cycle over empty enumerable"
       {state, acc} when state in [:done, :halted] ->
         do_cycle(cycle, cycle, {:cont, acc})
       {:suspended, acc, continuation} ->
@@ -1193,7 +1221,7 @@ defmodule Stream do
 
   """
   @spec repeatedly((() -> element)) :: Enumerable.t
-  def repeatedly(generator_fun) when is_function(generator_fun, 0) do
+  def repeatedly(generator_fun) do
     &do_repeatedly(generator_fun, &1, &2)
   end
 
@@ -1362,12 +1390,12 @@ defmodule Stream do
   @compile {:inline, lazy: 2, lazy: 3, lazy: 4}
 
   defp lazy(%Stream{done: nil, funs: funs} = lazy, fun),
-    do: %{lazy | funs: [fun | funs] }
+    do: %{lazy | funs: [fun | funs]}
   defp lazy(enum, fun),
     do: %Stream{enum: enum, funs: [fun]}
 
   defp lazy(%Stream{done: nil, funs: funs, accs: accs} = lazy, acc, fun),
-    do: %{lazy | funs: [fun | funs], accs: [acc | accs] }
+    do: %{lazy | funs: [fun | funs], accs: [acc | accs]}
   defp lazy(enum, acc, fun),
     do: %Stream{enum: enum, funs: [fun], accs: [acc]}
 

lib/elixir/lib/stream/reducers.ex

@@ -1,49 +1,65 @@
 defmodule Stream.Reducers do
-  # Collection of reducers shared by Enum and Stream.
+  # Collection of reducers and utilities shared by Enum and Stream.
   @moduledoc false
 
-  defmacro chunk(amount, step, limit, fun \\ nil) do
-    quote do
-      fn entry, acc(head, {buffer, count}, tail) ->
-        buffer = [entry | buffer]
-        count = count + 1
+  def chunk_every(chunk_by, enumerable, count, step, leftover) do
+    limit = :erlang.max(count, step)
+    chunk_by.(enumerable, {[], 0}, fn entry, {acc_buffer, acc_count} ->
+      acc_buffer = [entry | acc_buffer]
+      acc_count = acc_count + 1
 
-        new_state =
-          if count >= unquote(limit) do
-            left = count - unquote(step)
-            {Enum.take(buffer, left), left}
-          else
-            {buffer, count}
-          end
-
-        if count == unquote(amount) do
-          next_with_acc(unquote(fun), :lists.reverse(buffer), head, new_state, tail)
+      new_state =
+        if acc_count >= limit do
+          remaining = acc_count - step
+          {Enum.take(acc_buffer, remaining), remaining}
         else
-          skip(acc(head, new_state, tail))
+          {acc_buffer, acc_count}
         end
+
+      if acc_count == count do
+        {:cont, :lists.reverse(acc_buffer), new_state}
+      else
+        {:cont, new_state}
       end
-    end
+    end, fn {acc_buffer, acc_count} ->
+      if leftover == :discard or acc_count == 0 or (step > count and acc_count >= count) do
+        {:cont, []}
+      else
+        {:cont, :lists.reverse(acc_buffer, Enum.take(leftover, count - acc_count)), []}
+      end
+    end)
   end
 
-  defmacro chunk_by(callback, fun \\ nil) do
+  def chunk_by(chunk_by, enumerable, fun) do
+    chunk_by.(enumerable, nil, fn
+      entry, nil ->
+        {:cont, {[entry], fun.(entry)}}
+      entry, {acc, value} ->
+        case fun.(entry) do
+          ^value -> {:cont, {[entry | acc], value}}
+          new_value -> {:cont, :lists.reverse(acc), {[entry], new_value}}
+        end
+    end, fn
+      nil -> {:cont, :done}
+      {acc, _value} -> {:cont, :lists.reverse(acc), :done}
+    end)
+  end
+
+  defmacro chunk_while(callback, fun \\ nil) do
     quote do
-      fn
-        entry, acc(head, {buffer, value}, tail) ->
-          new_value = unquote(callback).(entry)
-          if new_value == value do
-            skip(acc(head, {[entry | buffer], value}, tail))
-          else
-            next_with_acc(unquote(fun), :lists.reverse(buffer), head, {[entry], new_value}, tail)
-          end
-        entry, acc(head, nil, tail) ->
-          skip(acc(head, {[entry], unquote(callback).(entry)}, tail))
+      fn entry, acc(head, acc, tail) ->
+        case unquote(callback).(entry, acc) do
+          {:cont, emit, acc} -> next_with_acc(unquote(fun), emit, head, acc, tail)
+          {:cont, acc} -> skip(acc(head, acc, tail))
+          {:halt, acc} -> {:halt, acc(head, acc, tail)}
+        end
       end
     end
   end
 
   defmacro dedup(callback, fun \\ nil) do
     quote do
-      fn(entry, acc(head, prev, tail) = acc) ->
+      fn entry, acc(head, prev, tail) = acc ->
         value = unquote(callback).(entry)
         case prev do
           {:value, ^value} -> skip(acc)
@@ -89,7 +105,7 @@ defmodule Stream.Reducers do
 
   defmacro filter(callback, fun \\ nil) do
     quote do
-      fn(entry, acc) ->
+      fn entry, acc ->
         if unquote(callback).(entry) do
           next(unquote(fun), entry, acc)
         else
@@ -101,7 +117,7 @@ defmodule Stream.Reducers do
 
   defmacro filter_map(filter, mapper, fun \\ nil) do
     quote do
-      fn(entry, acc) ->
+      fn entry, acc ->
         if unquote(filter).(entry) do
           next(unquote(fun), unquote(mapper).(entry), acc)
         else
@@ -113,7 +129,7 @@ defmodule Stream.Reducers do
 
   defmacro map(callback, fun \\ nil) do
     quote do
-      fn(entry, acc) ->
+      fn entry, acc ->
         next(unquote(fun), unquote(callback).(entry), acc)
       end
     end
@@ -132,7 +148,7 @@ defmodule Stream.Reducers do
 
   defmacro reject(callback, fun \\ nil) do
     quote do
-      fn(entry, acc) ->
+      fn entry, acc ->
         unless unquote(callback).(entry) do
           next(unquote(fun), entry, acc)
         else
@@ -156,7 +172,7 @@ defmodule Stream.Reducers do
 
   defmacro scan3(callback, fun \\ nil) do
     quote do
-      fn(entry, acc(head, acc, tail)) ->
+      fn entry, acc(head, acc, tail) ->
         value = unquote(callback).(entry, acc)
         next_with_acc(unquote(fun), value, head, value, tail)
       end
@@ -165,7 +181,7 @@ defmodule Stream.Reducers do
 
   defmacro take(fun \\ nil) do
     quote do
-      fn(entry, acc(head, curr, tail) = original) ->
+      fn entry, acc(head, curr, tail) = original ->
         case curr do
           0 ->
             {:halt, original}
@@ -192,7 +208,7 @@ defmodule Stream.Reducers do
 
   defmacro take_while(callback, fun \\ nil) do
     quote do
-      fn(entry, acc) ->
+      fn entry, acc ->
         if unquote(callback).(entry) do
           next(unquote(fun), entry, acc)
         else
@@ -204,7 +220,7 @@ defmodule Stream.Reducers do
 
   defmacro uniq_by(callback, fun \\ nil) do
     quote do
-      fn(entry, acc(head, prev, tail) = original) ->
+      fn entry, acc(head, prev, tail) = original ->
         value = unquote(callback).(entry)
         if Map.has_key?(prev, value) do
           skip(original)
@@ -217,7 +233,7 @@ defmodule Stream.Reducers do
 
   defmacro with_index(fun \\ nil) do
     quote do
-      fn(entry, acc(head, counter, tail)) ->
+      fn entry, acc(head, counter, tail) ->
         next_with_acc(unquote(fun), {entry, counter}, head, counter + 1, tail)
       end
     end

lib/elixir/lib/string.ex

@@ -55,6 +55,9 @@ defmodule String do
   The current Elixir version implements Extended Grapheme Cluster
   algorithm.
 
+  For converting a binary to a different encoding and for Unicode
+  normalization mechanisms, see Erlang's `:unicode` module.
+
   ## String and binary operations
 
   To act according to the Unicode Standard, many functions
@@ -208,39 +211,49 @@ defmodule String do
   @doc """
   Checks if a string contains only printable characters.
 
+  Takes an optional `limit` as a second argument. `printable?/2` only
+  checks the printability of the string up to the `limit`.
+
   ## Examples
 
       iex> String.printable?("abc")
       true
 
+      iex> String.printable?("abc" <> <<0>>)
+      false
+
+      iex> String.printable?("abc" <> <<0>>, 2)
+      true
+
   """
   @spec printable?(t) :: boolean
-  def printable?(string)
+  @spec printable?(t, non_neg_integer | :infinity) :: boolean
+  def printable?(string, counter \\ :infinity)
+
+  def printable?(<<>>, _), do: true
+  def printable?(_, 0), do: true
 
   for char <- 0x20..0x7E do
-    def printable?(<<unquote(char), rest::binary>>) do
-      printable?(rest)
+    def printable?(<<unquote(char), rest::binary>>, counter) do
+      printable?(rest, decrement(counter))
     end
   end
-  def printable?(<<?\n, rest::binary>>), do: printable?(rest)
-  def printable?(<<?\r, rest::binary>>), do: printable?(rest)
-  def printable?(<<?\t, rest::binary>>), do: printable?(rest)
-  def printable?(<<?\v, rest::binary>>), do: printable?(rest)
-  def printable?(<<?\b, rest::binary>>), do: printable?(rest)
-  def printable?(<<?\f, rest::binary>>), do: printable?(rest)
-  def printable?(<<?\e, rest::binary>>), do: printable?(rest)
-  def printable?(<<?\d, rest::binary>>), do: printable?(rest)
-  def printable?(<<?\a, rest::binary>>), do: printable?(rest)
-
-  def printable?(<<char::utf8, rest::binary>>)
+  for char <- '\n\r\t\v\b\f\e\d\a' do
+    def printable?(<<unquote(char), rest::binary>>, counter) do
+      printable?(rest, decrement(counter))
+    end
+  end
+  def printable?(<<char::utf8, rest::binary>>, counter)
       when char in 0xA0..0xD7FF
       when char in 0xE000..0xFFFD
       when char in 0x10000..0x10FFFF do
-    printable?(rest)
+    printable?(rest, decrement(counter))
   end
 
-  def printable?(<<>>), do: true
-  def printable?(binary) when is_binary(binary), do: false
+  def printable?(binary, _) when is_binary(binary), do: false
+
+  defp decrement(:infinity), do: :infinity
+  defp decrement(counter),   do: counter - 1
 
   @doc ~S"""
   Divides a string into substrings at each Unicode whitespace
@@ -270,19 +283,29 @@ defmodule String do
   Divides a string into substrings based on a pattern.
 
   Returns a list of these substrings. The pattern can
-  be a string, a list of strings or a regular expression.
+  be a string, a list of strings, or a regular expression.
 
   The string is split into as many parts as possible by
-  default, but can be controlled via the `parts: pos_integer` option.
-  If you pass `parts: :infinity`, it will return all possible parts
-  (`:infinity` is the default).
+  default, but can be controlled via the `:parts` option.
 
   Empty strings are only removed from the result if the
-  `trim` option is set to `true` (default is `false`).
+  `:trim` option is set to `true`.
 
   When the pattern used is a regular expression, the string is
-  split using `Regex.split/3`. In that case this function accepts
-  additional options which are documented in `Regex.split/3`.
+  split using `Regex.split/3`.
+
+  ## Options
+
+    * `:parts` (positive integer or `:infinity`) - the string
+      is split into at most as many parts as this options specifies.
+      If `:infinity`, the string will be split into all possible
+      parts. Defaults to `:infinity`.
+
+    * `:trim` (boolean) - if `true`, empty strings are removed from
+      the resulting list.
+
+  This function also accepts all options accepted by `Regex.split/3`
+  if `pattern` is a regular expression.
 
   ## Examples
 
@@ -337,8 +360,7 @@ defmodule String do
       ["1", "2", "3", "4"]
 
   """
-  @spec split(t, pattern | Regex.t) :: [t]
-  @spec split(t, pattern | Regex.t, Keyword.t) :: [t]
+  @spec split(t, pattern | Regex.t, keyword) :: [t]
   def split(string, pattern, options \\ [])
 
   def split(string, %Regex{} = pattern, options) when is_binary(string) do
@@ -395,7 +417,7 @@ defmodule String do
       ["a", "b", "c", "d"]
 
   """
-  @spec splitter(t, pattern, Keyword.t) :: Enumerable.t
+  @spec splitter(t, pattern, keyword) :: Enumerable.t
   def splitter(string, pattern, options \\ []) do
     pattern = maybe_compile_pattern(pattern)
     trim    = Keyword.get(options, :trim, false)
@@ -411,15 +433,10 @@ defmodule String do
   end
 
   defp do_splitter(bin, pattern, trim) do
-    case :binary.match(bin, pattern) do
-      {0, length} when trim ->
-        do_splitter(:binary.part(bin, length, byte_size(bin) - length), pattern, trim)
-      {pos, length} ->
-        final = pos + length
-        {:binary.part(bin, 0, pos),
-         :binary.part(bin, final, byte_size(bin) - final)}
-      :nomatch ->
-        {bin, :nomatch}
+    case :binary.split(bin, pattern) do
+      ["", second] when trim -> do_splitter(second, pattern, trim)
+      [first, second] -> {first, second}
+      [first] -> {first, :nomatch}
     end
   end
 
@@ -465,7 +482,7 @@ defmodule String do
   end
 
   def split_at(string, position) when is_integer(position) and position < 0 do
-    position = length(string) - abs(position)
+    position = length(string) + position
     case position >= 0 do
       true  -> do_split_at(string, position)
       false -> {"", string}
@@ -601,11 +618,13 @@ defmodule String do
   end
 
   @doc false
-  # TODO: Deprecate by 1.5
+  # TODO: Remove by 2.0
+  # (hard-deprecated in elixir_dispatch)
   defdelegate rstrip(binary), to: String.Break, as: :trim_trailing
 
   @doc false
-  # TODO: Deprecate by 1.5
+  # TODO: Remove by 2.0
+  # (hard-deprecated in elixir_dispatch)
   def rstrip(string, char) when is_integer(char) do
     replace_trailing(string, <<char::utf8>>, "")
   end
@@ -647,15 +666,15 @@ defmodule String do
 
   defp replace_leading(string, match, replacement, prefix_size, suffix_size, acc) when suffix_size >= 0 do
     case string do
-      <<prefix::size(prefix_size)-binary, suffix::size(suffix_size)-binary>> when prefix == match ->
+      <<prefix::size(prefix_size)-binary, suffix::binary>> when prefix == match ->
         replace_leading(suffix, match, replacement, prefix_size, suffix_size - prefix_size, acc + 1)
       _ ->
-        duplicate(replacement, acc) <> string
+        prepend_unless_empty(duplicate(replacement, acc), string)
     end
   end
 
   defp replace_leading(string, _match, replacement, _prefix_size, _suffix_size, acc) do
-    duplicate(replacement, acc) <> string
+    prepend_unless_empty(duplicate(replacement, acc), string)
   end
 
   @doc """
@@ -695,15 +714,15 @@ defmodule String do
 
   defp replace_trailing(string, match, replacement, prefix_size, suffix_size, acc) when prefix_size >= 0 do
     case string do
-      <<prefix::size(prefix_size)-binary, suffix::size(suffix_size)-binary>> when suffix == match ->
+      <<prefix::size(prefix_size)-binary, suffix::binary>> when suffix == match ->
         replace_trailing(prefix, match, replacement, prefix_size - suffix_size, suffix_size, acc + 1)
       _ ->
-        string <> duplicate(replacement, acc)
+        append_unless_empty(string, duplicate(replacement, acc))
     end
   end
 
   defp replace_trailing(string, _match, replacement, _prefix_size, _suffix_size, acc) do
-    string <> duplicate(replacement, acc)
+    append_unless_empty(string, duplicate(replacement, acc))
   end
 
   @doc """
@@ -736,11 +755,10 @@ defmodule String do
   def replace_prefix(string, match, replacement)
       when is_binary(string) and is_binary(match) and is_binary(replacement) do
     prefix_size = byte_size(match)
-    suffix_size = byte_size(string) - prefix_size
 
     case string do
-      <<prefix::size(prefix_size)-binary, suffix::size(suffix_size)-binary>> when prefix == match ->
-        replacement <> suffix
+      <<prefix::size(prefix_size)-binary, suffix::binary>> when prefix == match ->
+        prepend_unless_empty(replacement, suffix)
       _ ->
         string
     end
@@ -779,31 +797,43 @@ defmodule String do
     prefix_size = byte_size(string) - suffix_size
 
     case string do
-      <<prefix::size(prefix_size)-binary, suffix::size(suffix_size)-binary>> when suffix == match ->
-        prefix <> replacement
+      <<prefix::size(prefix_size)-binary, suffix::binary>> when suffix == match ->
+        append_unless_empty(prefix, replacement)
       _ ->
         string
     end
   end
 
+  @compile {:inline, prepend_unless_empty: 2, append_unless_empty: 2}
+
+  defp prepend_unless_empty("", suffix), do: suffix
+  defp prepend_unless_empty(prefix, suffix), do: prefix <> suffix
+
+  defp append_unless_empty(prefix, ""), do: prefix
+  defp append_unless_empty(prefix, suffix), do: prefix <> suffix
+
   @doc false
-  # TODO: Deprecate by 1.5
+  # TODO: Remove by 2.0
+  # (hard-deprecated in elixir_dispatch)
   defdelegate lstrip(binary), to: String.Break, as: :trim_leading
 
   @doc false
-  # TODO: Deprecate by 1.5
+  # TODO: Remove by 2.0
+  # (hard-deprecated in elixir_dispatch)
   def lstrip(string, char) when is_integer(char) do
     replace_leading(string, <<char::utf8>>, "")
   end
 
   @doc false
-  # TODO: Deprecate by 1.5
+  # TODO: Remove by 2.0
+  # (hard-deprecated in elixir_dispatch)
   def strip(string) do
     trim(string)
   end
 
   @doc false
-  # TODO: Deprecate by 1.5
+  # TODO: Remove by 2.0
+  # (hard-deprecated in elixir_dispatch)
   def strip(string, char) do
     trim(string, <<char::utf8>>)
   end
@@ -1023,13 +1053,15 @@ defmodule String do
   end
 
   @doc false
-  # TODO: Deprecate by 1.5
+  # TODO: Remove by 2.0
+  # (hard-deprecated in elixir_dispatch)
   def rjust(subject, len, pad \\ ?\s) when is_integer(pad) and is_integer(len) and len >= 0 do
     pad(:leading, subject, len, [<<pad::utf8>>])
   end
 
   @doc false
-  # TODO: Deprecate by 1.5
+  # TODO: Remove by 2.0
+  # (hard-deprecated in elixir_dispatch)
   def ljust(subject, len, pad \\ ?\s) when is_integer(pad) and is_integer(len) and len >= 0 do
     pad(:trailing, subject, len, [<<pad::utf8>>])
   end
@@ -1038,11 +1070,23 @@ defmodule String do
   Returns a new string created by replacing occurrences of `pattern` in
   `subject` with `replacement`.
 
-  By default, it replaces all occurrences, unless the `global` option is
-  set to `false`, where it will only replace the first one
-
   The `pattern` may be a string or a regular expression.
 
+  By default it replaces all occurrences but this behaviour can be controlled
+  through the `:global` option; see the "Options" section below.
+
+  ## Options
+
+    * `:global` - (boolean) if `true`, all occurrences of `pattern` are replaced
+      with `replacement`, otherwise only the first occurrence is
+      replaced. Defaults to `true`
+
+    * `:insert_replaced` - (integer or list of integers) specifies the position
+      where to insert the replaced part inside the `replacement`. If any
+      position given in the `:insert_replaced` option is larger than the
+      replacement string, or is negative, an `ArgumentError` is raised. See the
+      examples below
+
   ## Examples
 
       iex> String.replace("a,b,c", ",", "-")
@@ -1077,10 +1121,8 @@ defmodule String do
       iex> String.replace("a,b,c", ",", "[]", insert_replaced: [1, 1])
       "a[,,]b[,,]c"
 
-  If any position given in the `:insert_replaced` option is larger than the
-  replacement string, or is negative, an `ArgumentError` is raised.
   """
-  @spec replace(t, pattern | Regex.t, t, Keyword.t) :: t
+  @spec replace(t, pattern | Regex.t, t, keyword) :: t
   def replace(subject, pattern, replacement, options \\ []) when is_binary(replacement) do
     if Regex.regex?(pattern) do
       Regex.replace(pattern, subject, replacement, global: options[:global])
@@ -1145,9 +1187,13 @@ defmodule String do
 
   defp do_reverse(nil, acc), do: IO.iodata_to_binary(acc)
 
+  @compile {:inline, duplicate: 2}
+
   @doc """
   Returns a string `subject` duplicated `n` times.
 
+  Inlined by the compiler.
+
   ## Examples
 
       iex> String.duplicate("abc", 0)
@@ -1161,7 +1207,7 @@ defmodule String do
 
   """
   @spec duplicate(t, non_neg_integer) :: t
-  def duplicate(subject, n) when is_integer(n) and n >= 0 do
+  def duplicate(subject, n) do
     :binary.copy(subject, n)
   end
 
@@ -1227,6 +1273,9 @@ defmodule String do
       iex> String.valid?(<<0xFFFF :: 16>>)
       false
 
+      iex> String.valid?(<<0xEF, 0xB7, 0x90>>)
+      true
+
       iex> String.valid?("asd" <> <<0xFFFF :: 16>>)
       false
 
@@ -1234,24 +1283,14 @@ defmodule String do
   @spec valid?(t) :: boolean
   def valid?(string)
 
-  noncharacters = Enum.to_list(0xFDD0..0xFDEF) ++
-    [0x0FFFE, 0x0FFFF, 0x1FFFE, 0x1FFFF, 0x2FFFE, 0x2FFFF,
-     0x3FFFE, 0x3FFFF, 0x4FFFE, 0x4FFFF, 0x5FFFE, 0x5FFFF,
-     0x6FFFE, 0x6FFFF, 0x7FFFE, 0x7FFFF, 0x8FFFE, 0x8FFFF,
-     0x9FFFE, 0x9FFFF, 0x10FFFE, 0x10FFFF]
-
-  for noncharacter <- noncharacters do
-    def valid?(<<unquote(noncharacter)::utf8, _::binary>>), do: false
-  end
-
   def valid?(<<_::utf8, t::binary>>), do: valid?(t)
   def valid?(<<>>), do: true
   def valid?(_), do: false
 
   @doc false
   # TODO: Remove on 2.0
+  # (hard-deprecated in elixir_dispatch)
   def valid_character?(string) do
-    IO.warn "String.valid_character?/1 is deprecated, please use valid?/1 instead"
     case string do
       <<_::utf8>> -> valid?(string)
       _ -> false
@@ -1279,8 +1318,8 @@ defmodule String do
       iex> String.chunk(<<?a, ?b, ?c, 0>>, :valid)
       ["abc\0"]
 
-      iex> String.chunk(<<?a, ?b, ?c, 0, 0x0FFFF::utf8>>, :valid)
-      ["abc\0", <<0x0FFFF::utf8>>]
+      iex> String.chunk(<<?a, ?b, ?c, 0, 0xFFFF::utf16>>, :valid)
+      ["abc\0", <<0xFFFF::utf16>>]
 
       iex> String.chunk(<<?a, ?b, ?c, 0, 0x0FFFF::utf8>>, :printable)
       ["abc", <<0, 0x0FFFF::utf8>>]
@@ -1468,7 +1507,7 @@ defmodule String do
   end
 
   def at(string, position) when is_integer(position) and position < 0 do
-    position = length(string) - abs(position)
+    position = length(string) + position
     case position >= 0 do
       true  -> do_at(string, position)
       false -> nil
@@ -1808,8 +1847,16 @@ defmodule String do
   @doc """
   Converts a string to an atom.
 
-  Currently Elixir does not support the conversion of strings
-  that contain Unicode codepoints greater than 0xFF.
+  Warning: this function creates atoms dynamically and atoms are
+  not garbage collected. Therefore, `string` should not be an
+  untrusted value, such as input received from a socket or during
+  a web request. Consider using `to_existing_atom/1` instead.
+
+  By default, the maximum number of atoms is `1_048_576`. This limit
+  can be raised or lowered using the VM option `+t`.
+
+  The maximum atom size is of 255 characters. Prior to OTP 20,
+  only latin1 characters are allowed.
 
   Inlined by the compiler.
 
@@ -1827,8 +1874,8 @@ defmodule String do
   @doc """
   Converts a string to an existing atom.
 
-  Currently Elixir does not support the conversion of strings
-  that contain Unicode codepoints greater than 0xFF.
+  The maximum atom size is of 255 characters. Prior to OTP 20,
+  only latin1 characters are allowed.
 
   Inlined by the compiler.
 
@@ -1882,10 +1929,9 @@ defmodule String do
   @doc """
   Returns a float whose text representation is `string`.
 
-  `string` must be the string representation of a float.
-  If a string representation of an integer wants to be used,
-  then `Float.parse/1` should be used instead,
-  otherwise an argument error will be raised.
+  `string` must be the string representation of a float including a decimal point.
+  In order to parse a string without decimal point as a float then `Float.parse/1`
+  should be used. Otherwise, an `ArgumentError` will be raised.
 
   Inlined by the compiler.
 
@@ -1897,6 +1943,9 @@ defmodule String do
       iex> String.to_float("3.0")
       3.0
 
+      String.to_float("3")
+      #=> ** (ArgumentError) argument error
+
   """
   @spec to_float(String.t) :: float
   def to_float(string) do
@@ -2017,7 +2066,8 @@ defmodule String do
     end)
   end
 
-  # TODO: Deprecate by v1.5
+  # TODO: Remove by 2.0
+  # (hard-deprecated in elixir_dispatch)
   @doc false
   @spec to_char_list(t) :: charlist
   def to_char_list(string), do: String.to_charlist(string)

lib/elixir/lib/string/chars.ex

@@ -4,16 +4,21 @@ defprotocol String.Chars do
   @moduledoc ~S"""
   The `String.Chars` protocol is responsible for
   converting a structure to a binary (only if applicable).
+
   The only function required to be implemented is
-  `to_string` which does the conversion.
+  `to_string/1`, which does the conversion.
 
   The `to_string/1` function automatically imported
   by `Kernel` invokes this protocol. String
-  interpolation also invokes `to_string` in its
+  interpolation also invokes `to_string/1` in its
   arguments. For example, `"foo#{bar}"` is the same
   as `"foo" <> to_string(bar)`.
   """
 
+  @doc """
+  Converts `term` to a string.
+  """
+  @spec to_string(t) :: String.t
   def to_string(term)
 end
 

lib/elixir/lib/string_io.ex

@@ -40,7 +40,7 @@ defmodule StringIO do
       {"", ">"}
 
   """
-  @spec open(binary, Keyword.t) :: {:ok, pid}
+  @spec open(binary, keyword) :: {:ok, pid}
   def open(string, options \\ []) when is_binary(string) do
     GenServer.start_link(__MODULE__, {string, options}, [])
   end
@@ -216,20 +216,12 @@ defmodule StringIO do
 
   ## get_chars
 
-  defp get_chars(encoding, prompt, n,
-                 %{input: input, output: output, capture_prompt: capture_prompt} = s) do
+  defp get_chars(encoding, prompt, n, %{input: input} = s) do
     case do_get_chars(input, encoding, n) do
       {:error, _} = error ->
         {error, s}
       {result, input} ->
-        s =
-          if capture_prompt do
-            %{s | output: <<output::binary, IO.chardata_to_string(prompt)::binary>>}
-          else
-            s
-          end
-
-        {result, %{s | input: input}}
+        {result, state_after_read(s, input, prompt)}
     end
   end
 
@@ -263,90 +255,83 @@ defmodule StringIO do
 
   ## get_line
 
-  defp get_line(encoding, prompt,
-                %{input: input, output: output, capture_prompt: capture_prompt} = s) do
-    case :unicode.characters_to_list(input, encoding) do
-      {:error, _, _} ->
-        {{:error, :collect_line}, s}
-      {:incomplete, _, _} ->
-        {{:error, :collect_line}, s}
-      chars ->
-        {result, input} = do_get_line(chars, encoding)
+  defp get_line(encoding, prompt, %{input: input} = s) do
+    case bytes_until_eol(input, encoding, 0) do
+      {:split, 0} ->
+        {:eof, state_after_read(s, "", prompt)}
+      {:split, count} ->
+        {result, remainder} = :erlang.split_binary(input, count)
 
-        s =
-          if capture_prompt do
-            %{s | output: <<output::binary, IO.chardata_to_string(prompt)::binary>>}
-          else
-            s
-          end
+        {result, state_after_read(s, remainder, prompt)}
+      {:replace_split, count} ->
+        {result, remainder} = :erlang.split_binary(input, count)
 
-        {result, %{s | input: input}}
+        {binary_part(result, 0, byte_size(result) - 2) <> "\n", state_after_read(s, remainder, prompt)}
+      :error
+        -> {{:error, :collect_line}, s}
     end
   end
 
-  defp do_get_line('', _encoding) do
-    {:eof, ""}
-  end
-
-  defp do_get_line(chars, encoding) do
-    {line, rest} = collect_line(chars)
-    {:unicode.characters_to_binary(line, encoding),
-      :unicode.characters_to_binary(rest, encoding)}
-  end
-
   ## get_until
 
-  defp get_until(encoding, prompt, mod, fun, args,
-                 %{input: input, output: output, capture_prompt: capture_prompt} = s) do
-    case :unicode.characters_to_list(input, encoding) do
-      {:error, _, _} ->
-        {:error, s}
-      {:incomplete, _, _} ->
-        {:error, s}
-      chars ->
-        {result, input, count} = do_get_until(chars, encoding, mod, fun, args)
-
+  defp get_until(encoding, prompt, mod, fun, args, %{input: input} = s) do
+    case do_get_until(input, encoding, mod, fun, args) do
+      {result, input, count} ->
         input =
-          case input do
-            :eof -> ""
-            _ -> :unicode.characters_to_binary(input, encoding)
-          end
+         case input do
+           :eof -> ""
+           _ -> list_to_binary(input, encoding)
+         end
 
-        s =
-          if capture_prompt do
-            %{s | output: <<output::binary, :binary.copy(IO.chardata_to_string(prompt), count)::binary>>}
-          else
-            s
-          end
+        {get_until_result(result, encoding), state_after_read(s, input, prompt, count)}
 
-        {result, %{s | input: input}}
+      :error ->
+        {:error, s}
     end
   end
 
   defp do_get_until(chars, encoding, mod, fun, args, continuation \\ [], count \\ 0)
 
-  defp do_get_until('', encoding, mod, fun, args, continuation, count) do
+  defp do_get_until("", encoding, mod, fun, args, continuation, count) do
     case apply(mod, fun, [continuation, :eof | args]) do
       {:done, result, rest} ->
         {result, rest, count + 1}
       {:more, next_continuation} ->
-        do_get_until('', encoding, mod, fun, args, next_continuation, count + 1)
+        do_get_until("", encoding, mod, fun, args, next_continuation, count + 1)
     end
   end
 
   defp do_get_until(chars, encoding, mod, fun, args, continuation, count) do
-    {line, rest} = collect_line(chars)
+    case bytes_until_eol(chars, encoding, 0) do
+      {r, c} when r in [:split, :replace_split] ->
+        {line, rest} = :erlang.split_binary(chars, c)
 
-    case apply(mod, fun, [continuation, line | args]) do
-      {:done, result, :eof} ->
-        {result, rest, count + 1}
-      {:done, result, extra} ->
-        {result, extra ++ rest, count + 1}
-      {:more, next_continuation} ->
-        do_get_until(rest, encoding, mod, fun, args, next_continuation, count + 1)
+        case apply(mod, fun, [continuation, binary_to_list(line, encoding) | args]) do
+          {:done, result, :eof} ->
+            {result, rest, count + 1}
+          {:done, result, extra} ->
+            {result, extra ++ binary_to_list(rest, encoding), count + 1}
+          {:more, next_continuation} ->
+            do_get_until(rest, encoding, mod, fun, args, next_continuation, count + 1)
+        end
+
+      :error ->
+        :error
     end
   end
 
+  defp binary_to_list(l, _)        when is_list(l),   do: l
+  defp binary_to_list(b, :unicode) when is_binary(b), do: to_charlist(b)
+  defp binary_to_list(b, :latin1)  when is_binary(b), do: :binary.bin_to_list(b)
+  defp list_to_binary(b, _)        when is_binary(b), do: b
+  defp list_to_binary(l, :unicode) when is_list(l),   do: to_string(l)
+  defp list_to_binary(l, :latin1)  when is_list(l),   do: :binary.list_to_bin(l)
+
+  # From http://erlang.org/doc/apps/stdlib/io_protocol.html: Result can be any
+  # Erlang term, but if it is a list(), the I/O server can convert it to a binary().
+  defp get_until_result(l, encoding) when is_list(l), do: list_to_binary(l, encoding)
+  defp get_until_result(other, _), do: other
+
   ## io_requests
 
   defp io_requests([r | rs], {:ok, s}) do
@@ -359,25 +344,29 @@ defmodule StringIO do
 
   ## helpers
 
-  defp collect_line(chars) do
-    collect_line(chars, [])
+  defp state_after_read(state, remainder, prompt, count \\ 1)
+
+  defp state_after_read(%{capture_prompt: false} = s, remainder, _prompt, _count) do
+    %{s | input: remainder}
   end
 
-  defp collect_line([], stack) do
-    {:lists.reverse(stack), []}
+  defp state_after_read(%{capture_prompt: true, output: output} = s, remainder, prompt, count) do
+    %{s | input: remainder, output: <<output::binary, :binary.copy(IO.chardata_to_string(prompt), count)::binary>>}
   end
 
-  defp collect_line([?\r, ?\n | rest], stack) do
-    {:lists.reverse([?\n | stack]), rest}
+  defp bytes_until_eol("", _, count), do: {:split, count}
+  defp bytes_until_eol(<<"\r\n"::binary, _::binary>>, _, count), do: {:replace_split, count + 2}
+  defp bytes_until_eol(<<"\n"::binary, _::binary>>, _, count), do: {:split, count + 1}
+
+  defp bytes_until_eol(<<head::utf8, tail::binary>>, :unicode, count) do
+    bytes_until_eol(tail, :unicode, count + byte_size(<<head::utf8>>))
   end
 
-  defp collect_line([?\n | rest], stack) do
-    {:lists.reverse([?\n | stack]), rest}
+  defp bytes_until_eol(<<_, tail::binary>>, :latin1, count) do
+    bytes_until_eol(tail, :latin1, count + 1)
   end
 
-  defp collect_line([h | t], stack) do
-    collect_line(t, [h | stack])
-  end
+  defp bytes_until_eol(<<_::binary>>, _, _), do: :error
 
   defp io_reply(from, reply_as, reply) do
     send from, {:io_reply, reply_as, reply}

lib/elixir/lib/supervisor.ex

@@ -1,27 +1,33 @@
 defmodule Supervisor do
   @moduledoc ~S"""
-  A behaviour module for implementing supervision functionality.
+  A behaviour module for implementing supervisors.
 
-  A supervisor is a process which supervises other processes, which we refer
-  to as *child processes*. Supervisors are used to build a hierarchical process
-  structure called a *supervision tree*. Supervision trees are a nice way to
-  structure fault-tolerant applications.
+  A supervisor is a process which supervises other processes, which we
+  refer to as *child processes*. Supervisors are used to build a hierarchical
+  process structure called a *supervision tree*. Supervision trees provide
+  fault-tolerance and encapsulate how our applications start and shutdown.
 
   A supervisor implemented using this module has a standard set
   of interface functions and includes functionality for tracing and error
-  reporting. It also fits into a supervision tree.
+  reporting.
 
   ## Examples
 
   In order to define a supervisor, we need to first define a child process
-  that is going to be supervised. In order to do so, we will define a GenServer
-  that represents a stack:
+  that will be supervised. As an example, we will define a GenServer that
+  represents a stack:
 
       defmodule Stack do
         use GenServer
 
-        def start_link(state, opts \\ []) do
-          GenServer.start_link(__MODULE__, state, opts)
+        def start_link(state) do
+          GenServer.start_link(__MODULE__, state, name: __MODULE__)
+        end
+
+        ## Callbacks
+
+        def init(stack) do
+          {:ok, stack}
         end
 
         def handle_call(:pop, _from, [h | t]) do
@@ -33,50 +39,50 @@ defmodule Supervisor do
         end
       end
 
-  We can now define our supervisor and start it as follows:
+  The stack is a small wrapper around lists. It allows us to put
+  an element on the top of the stack, by prepending to the list,
+  and to get the top of the stack by pattern matching.
 
-      # Import helpers for defining supervisors
-      import Supervisor.Spec
+  We can now start a supervisor that will start and supervise our
+  stack process as follows:
 
-      # Supervise the Stack server which will be started with
-      # two arguments. The initial stack, [:hello], and a
-      # keyword list containing the GenServer options that
-      # set the registered name of the server to MyStack.
-      children = [
-        worker(Stack, [[:hello], [name: MyStack]])
-      ]
+      # Start the supervisor with the stack as a single child.
+      #
+      # The first element of the tuple is the module containing
+      # the child implementation, the second is the argument
+      # given to start_link, in this case a stack with `:hello`.
+      {:ok, pid} = Supervisor.start_link([
+        {Stack, [:hello]}
+      ], strategy: :one_for_one)
 
-      # Start the supervisor with our child
-      {:ok, pid} = Supervisor.start_link(children, strategy: :one_for_one)
-
-      # There is one child worker started
+      # After started, we can query the supervisor for information
       Supervisor.count_children(pid)
       #=> %{active: 1, specs: 1, supervisors: 0, workers: 1}
 
   Notice that when starting the GenServer, we are registering it
-  with name `MyStack`, which allows us to call it directly and
-  get what is on the stack:
+  with name `Stack`, which allows us to call it directly and get
+  what is on the stack:
 
-      GenServer.call(MyStack, :pop)
+      GenServer.call(Stack, :pop)
       #=> :hello
 
-      GenServer.cast(MyStack, {:push, :world})
+      GenServer.cast(Stack, {:push, :world})
       #=> :ok
 
-      GenServer.call(MyStack, :pop)
+      GenServer.call(Stack, :pop)
       #=> :world
 
   However, there is a bug in our stack server. If we call `:pop` and
   the stack is empty, it is going to crash because no clause matches:
 
-      GenServer.call(MyStack, :pop)
-      ** (exit) exited in: GenServer.call(MyStack, :pop, 5000)
+      GenServer.call(Stack, :pop)
+      ** (exit) exited in: GenServer.call(Stack, :pop, 5000)
 
   Luckily, since the server is being supervised by a supervisor, the
   supervisor will automatically start a new one, with the initial stack
   of `[:hello]`:
 
-      GenServer.call(MyStack, :pop)
+      GenServer.call(Stack, :pop)
       #=> :hello
 
   Supervisors support different strategies; in the example above, we
@@ -84,9 +90,178 @@ defmodule Supervisor do
   workers and supervisors as children, each of them with their specific
   configuration, shutdown values, and restart strategies.
 
-  The rest of this documentation will cover supervision strategies; also read
-  the documentation for the `Supervisor.Spec` module to learn about the
-  specification for workers and supervisors.
+  The rest of this document will cover how child processes are started,
+  how they can be specified, different supervision strategies and more.
+
+  ## The initialization process
+
+  In the previous section, we have started a supervisor with one child:
+
+      Supervisor.start_link([
+        {Stack, [:hello]}
+      ], strategy: :one_for_one)
+
+  The first argument given to `start_link` is a list of children.
+  In the example above, we have passed a tuple, where the child is
+  implemented by the `Stack` module and receives an initial argument
+  of `[:hello]` on `Stack.start_link/1`.
+
+  Generally speaking, starting the child process happens in three steps:
+
+    1. First the supervisor calls `Stack.child_spec([:hello])`. This
+       function must return a **child specification** which describes
+       how the `Stack` process is supervised. When you `use GenServer`,
+       a `child_spec/1` is automatically defined for you but we will see
+       when and how it can be configured. This function is called once
+       when the supervisor starts (or in case of hot code upgrades).
+
+    2. The **child specification** tells the supervisor which function
+       to invoke to start the child process. By default, it is the
+       `start_link/1` function, receiving the same argument and defined
+       in the same module as the `child_spec/1` function. This function
+       is called every time a new child process is necessary. For example,
+       when we crashed a `Stack` in the previous session,
+       `Stack.start_link([:hello])` was called once more to start a new stack.
+
+    3. Finally, `Stack.start_link/1` starts a new process that runs
+      `Stack.init/1`, which is responsible for setting a process that
+       will react to messages.
+
+  In summary, when the `Supervisor.start_link(children, opts)` is called,
+  it traverses the list of children and retrieves their `child_spec/1`.
+  Then each child specification describes how each child is started,
+  typically via the `start_link/1` function. The supervisor invokes the
+  `start_link/1` when it initializes and whenever the child process needs
+  to be restarted. The new process started by `start_link/1` often
+  executes the `init` callback as its first step. The `init` callback is
+  where we initialize and configure the child process.
+
+  ## Child specification
+
+  The child specification describes how the supervisor should start and
+  supervise a child process. We have learned that, when we invoked
+  `use GenServer`, a `Stack.child_spec/1` was automatically defined for
+  us. Let's invoke it and see what it returns:
+
+      Stack.child_spec([:hello])
+      #=> %{
+        id: Stack,
+        start: {Stack, :start_link, [[:hello]]},
+        restart: :permanent,
+        shutdown: 5000,
+        type: :worker
+      }
+
+  The child specification contains 5 keys. The first two are required
+  and the remaining ones are optional:
+
+    * `:id` - a value used to identify the child specification
+      internally by the supervisor; defaults to the given module.
+      In case of conflicting `:id`, the supervisor will refuse
+      to initialize and require explicit IDs. This key is required.
+
+    * `:start` - a tuple with the module-function-args to be invoked
+      to start the child process. This key is required.
+
+    * `:restart` - an atom that defines when a terminated child process
+       should be restarted (see the "Restart values" section below).
+       This key is optional and defaults to `:permanent`.
+
+    * `:shutdown` - an atom that defines how a child process should be
+      terminated (see the "Shutdown values" section below). This key
+      is optional and defaults to `5000` if the type is `:worker` or
+      `:infinity` if the type is `:supervisor`.
+
+    * `:type` - if the child process is a `:worker` or a `:supervisor`.
+      This key is optional and defaults to `:worker`.
+
+  There is a sixth key, called `:modules`, which is rarely changed and
+  it is set automatically based on the value in `:start`.
+
+  Most times, the behaviour module you are implementing will take care
+  of setting up a proper `child_spec/1` for you. For example, `use Supervisor`
+  will define a `child_spec/1` where the `:type` is set to `:supervisor`
+  and the `:shutdown` is `:infinity`. Still, if you need to customize
+  a certain behaviour, you can do so by defining your own `child_spec/1`
+  function or by passing options on `use`. For example, to specify a
+  `GenServer` with a shutdown limit of 10 seconds (10_000 miliseconds),
+  one might do:
+
+      use GenServer, shutdown: 10_000
+
+  Let's understand what the `:shutdown` and `:restart` options control.
+
+  ### Shutdown values (:shutdown)
+
+  The following shutdown values are supported in the `:shutdown` option:
+
+    * `:brutal_kill` - the child process is unconditionally terminated
+      using `Process.exit(child, :kill)`.
+
+    * any integer >= 0 - the amount of time in miliseconds that the
+      supervisor will wait for children to terminate after emitting a
+      `Process.exit(child, :shutdown)` signal.  If the child process is
+      not trapping exits, the initial `:shutdown` signal will terminate
+      the child process immediately. If the child process is trapping
+      exits, it has the given amount of time in miliseconds to terminate.
+      If it doesn't terminate within the specified time, the child process
+      is unconditionally terminated by the supervisor via
+      `Process.exit(child, :kill)`.
+
+    * `:infinity` - works as an integer except the supervisor will wait
+      indefinitely for the child to terminate. If the child process is a
+      supervisor, the recommend value is `:infinity` to give the supervisor
+      and its children enough time to shutdown. This option can be used with
+      regular workers but doing so is discouraged and requires extreme care.
+      If not used carefully and the child process does not terminate, it means
+      your application will never terminate as well.
+
+  ### Restart values (:restart)
+
+  The `:restart` option controls what the supervisor should consider to
+  be a successful termination or not. If the termination is successful,
+  the supervisor won't restart the child. If the child process crashed,
+  the supervisor will start a new one.
+
+  The following restart values are supported in the `:restart` option:
+
+    * `:permanent` - the child process is always restarted.
+
+    * `:temporary` - the child process is never restarted, regardless
+      of the supervision strategy.
+
+    * `:transient` - the child process is restarted only if it
+      terminates abnormally, i.e., with an exit reason other than
+      `:normal`, `:shutdown` or `{:shutdown, term}`.
+
+  For a more complete understanding of the exit reasons and their
+  impact, see the "Exit reasons" section next.
+
+  ## Exit reasons
+
+  A supervisor restarts a child process depending on its `:restart`
+  configuration. For example, when `:restart` is set `:transient`, the
+  supervisr does not restart the child in case it exits with reason `:normal`,
+  `:shutdown` or `{:shutdown, term}`.
+
+  So one may ask: which exit reason should I choose when exiting? There are
+  three options:
+
+    * `:normal` - in such cases, the exit won't be logged, there is no restart
+      in transient mode, and linked processes do not exit
+
+    * `:shutdown` or `{:shutdown, term}` - in such cases, the exit won't be
+      logged, there is no restart in transient mode, and linked processes exit
+      with the same reason unless they're trapping exits
+
+    * any other term - in such cases, the exit will be logged, there are
+      restarts in transient mode, and linked processes exit with the same
+      reason unless they're trapping exits
+
+  Notice that supervisor that reached maximum restart intensity will exit with
+  `:shutdown` reason. In this case the supervisor will only be restarted if its
+  child specification was defined with the `:restart` option is set to `:permanent`
+  (the default).
 
   ## Module-based supervisors
 
@@ -95,23 +270,28 @@ defmodule Supervisor do
   explicitly defining a supervision module:
 
       defmodule MyApp.Supervisor do
-        # Automatically imports Supervisor.Spec
+        # Automatically defines child_spec/1
         use Supervisor
 
-        def start_link do
-          Supervisor.start_link(__MODULE__, [])
+        def start_link(arg) do
+          Supervisor.start_link(__MODULE__, arg, name: __MODULE__)
         end
 
-        def init([]) do
-          children = [
-            worker(Stack, [[:hello]])
-          ]
-
-          # supervise/2 is imported from Supervisor.Spec
-          supervise(children, strategy: :one_for_one)
+        def init(_arg) do
+          Supervisor.init([
+            {Stack, [:hello]}
+          ], strategy: :one_for_one)
         end
       end
 
+  The difference between the two approaches is that a module-based
+  supervisor gives you more direct control over how the supervisor
+  is initialized. Instead of calling `Supervisor.start_link/2` with
+  a list of children that are automatically initialized, we have
+  defined a supervisor alongside its `c:init/1` callback and manually
+  initialized the children by calling `Supervisor.init/2`, passing
+  the same arguments we would have given to `start_link/2`.
+
   You may want to use a module-based supervisor if:
 
     * You need to perform some particular action on supervisor
@@ -124,10 +304,63 @@ defmodule Supervisor do
       requires the whole tree to be restarted in order to
       perform such swaps.
 
-  ## Strategies
+  Note `use Supervisor` defines a `child_spec/1` function, allowing
+  the defined module to be put under a supervision tree. The generated
+  `child_spec/1` can be customized with the following options:
 
-  Supervisors support different supervision strategies (through the `:strategy`
-  option, as seen above):
+    * `:id` - the child specification id, defauts to the current module
+    * `:start` - how to start the child process (defaults to calling `__MODULE__.start_link/1`)
+    * `:restart` - when the supervisor should be restarted, defaults to `:permanent`
+
+  ## start_link/2, init/2 and strategies
+
+  So far we have started the supervisor passing a single child as a tuple
+  as well as a strategy called `:one_for_one`:
+
+      Supervisor.start_link([
+        {Stack, [:hello]}
+      ], strategy: :one_for_one)
+
+  Or:
+
+      Supervisor.init([
+        {Stack, [:hello]}
+      ], strategy: :one_for_one)
+
+  However, children can be specified in three different formats and
+  supervisors support different options. Let's formally define those.
+
+  The first argument given to `start_link/2` is a list of children which may
+  be either:
+
+    * a module - such as `Stack`. In this case, it is equivalent to passing
+      `{Stack, []}` (which means `Stack.child_spec/1` is invoked with an empty
+      keywords list)
+    * a tuple with a module as first element and the start argument as second -
+      such as `{Stack, [:hello]}`. When such format is used, the supervisor
+      will retrieve the child specification from the given module.
+    * a map representing the child specification itself - such as the child
+      specification map outlined in the previous section.
+
+  The second argument is a keyword list of options:
+
+    * `:strategy` - the restart strategy option. It can be either
+      `:one_for_one`, `:rest_for_one`, `:one_for_all`, or
+      `:simple_one_for_one`. See the "Strategies" section.
+
+    * `:max_restarts` - the maximum amount of restarts allowed in
+      a time frame. Defaults to `3`.
+
+    * `:max_seconds` - the time frame in which `:max_restarts` applies.
+      Defaults to `5`.
+
+  The `:strategy` option is required and by default a maximum of 3 restarts
+  is allowed within 5 seconds.
+
+  ### Strategies
+
+  Supervisors support different supervision strategies (through the
+  `:strategy` option, as seen above):
 
     * `:one_for_one` - if a child process terminates, only that
       process is restarted.
@@ -149,86 +382,78 @@ defmodule Supervisor do
 
   ## Simple one for one
 
-  The `:simple_one_for_one` supervisor is useful when you want to dynamically
-  start and stop supervised children. For example, imagine you want to
-  dynamically create multiple stacks. We can do so by defining a `:simple_one_for_one`
-  supervisor:
+  The `:simple_one_for_one` supervisor is useful when you want to
+  dynamically start and stop supervised children. As an example,
+  let's start multiple agents dynamically to keep state.
 
-      # Import helpers for defining supervisors
-      import Supervisor.Spec
+  One important aspect in `:simple_one_for_one` supervisors is
+  that we often want to pass the `:start` arguments later on,
+  when starting the children dynamically, rather than when the
+  child specification is defined. In such cases, we should not do
 
-      # This time, we don't pass any argument because
-      # the argument will be given when we start the child
-      children = [
-        worker(Stack, [], restart: :transient)
+      Supervisor.start_link [
+        {Agent, fn -> 0 end}
       ]
 
-      # Start the supervisor with our one child as a template
-      {:ok, sup_pid} = Supervisor.start_link(children, strategy: :simple_one_for_one)
+  as the example above would force all agents to have the same state.
+  In such cases, we can use the `child_spec/2` function to build
+  and override the fields in a child specification:
 
-      # No child worker is active yet until start_child is called
+      # Override the :start field to have no args.
+      # The second argument has no effect thanks to it.
+      agent_spec =
+        Supervisor.child_spec(Agent, start: {Agent, :start_link, []})
+
+      # We start a supervisor with a simple one for one strategy.
+      # The agent won't be started now but later on.
+      {:ok, sup_pid} =
+        Supervisor.start_link([agent_spec], strategy: :simple_one_for_one)
+
+      # No child worker is active until start_child is called
       Supervisor.count_children(sup_pid)
       #=> %{active: 0, specs: 1, supervisors: 0, workers: 0}
 
-  There are a couple differences here:
+  The simple one for one strategy can define only one child which works
+  as a template for when we call `start_child/2`.
 
-    * the simple one for one specification can define only one child which
-      works as a template for when we call `start_child/2`
+  With the supervisor started, let's dynamically start agents:
 
-    * we have defined the child to have a restart strategy of `:transient`. This
-      means that, if the child process exits due to a `:normal`, `:shutdown`,
-      or `{:shutdown, term}` reason, it won't be restarted. This is useful
-      as it allows our workers to politely shutdown and be removed from the
-      `:simple_one_for_one` supervisor, without being restarted. You can find
-      more information about restart strategies in the documentation for the
-      `Supervisor.Spec` module
+      {:ok, agent1} = Supervisor.start_child(sup_pid, [fn -> 0 end])
+      Agent.update(agent1, & &1 + 1)
+      Agent.get(agent1, & &1) #=> 1
 
-  With the supervisor defined, let's dynamically start stacks:
-
-      {:ok, pid} = Supervisor.start_child(sup_pid, [[:hello, :world], []])
-      GenServer.call(pid, :pop) #=> :hello
-      GenServer.call(pid, :pop) #=> :world
-
-      {:ok, pid} = Supervisor.start_child(sup_pid, [[:something, :else], []])
-      GenServer.call(pid, :pop) #=> :something
-      GenServer.call(pid, :pop) #=> :else
+      {:ok, agent2} = Supervisor.start_child(sup_pid, [fn -> %{} end])
+      Agent.get(agent2, & &1) #=> %{}
 
       Supervisor.count_children(sup_pid)
       #=> %{active: 2, specs: 1, supervisors: 0, workers: 2}
 
-  ## Exit reasons
-
-  From the example above, you may have noticed that the `:transient` restart
-  strategy for the worker does not restart the child in case it exits with
-  reason `:normal`, `:shutdown` or `{:shutdown, term}`.
-
-  So one may ask: which exit reason should I choose when exiting my worker?
-  There are three options:
-
-    * `:normal` - in such cases, the exit won't be logged, there is no restart
-      in transient mode, and linked processes do not exit
-
-    * `:shutdown` or `{:shutdown, term}` - in such cases, the exit won't be
-      logged, there is no restart in transient mode, and linked processes exit
-      with the same reason unless they're trapping exits
-
-    * any other term - in such cases, the exit will be logged, there are
-      restarts in transient mode, and linked processes exit with the same reason
-      unless they're trapping exits
-
   ## Name registration
 
   A supervisor is bound to the same name registration rules as a `GenServer`.
   Read more about these rules in the documentation for `GenServer`.
-
   """
 
   @doc false
-  defmacro __using__(_) do
-    quote location: :keep do
+  defmacro __using__(opts) do
+    quote location: :keep, bind_quoted: [opts: opts] do
       @behaviour Supervisor
       import Supervisor.Spec
 
+      spec = [
+        id: opts[:id] || __MODULE__,
+        start: Macro.escape(opts[:start]) || quote(do: {__MODULE__, :start_link, [arg]}),
+        restart: opts[:restart] || :permanent,
+        type: :supervisor
+      ]
+
+      @doc false
+      def child_spec(arg) do
+        %{unquote_splicing(spec)}
+      end
+
+      defoverridable child_spec: 1
+
       @doc false
       def init(arg)
     end
@@ -237,10 +462,8 @@ defmodule Supervisor do
   @doc """
   Callback invoked to start the supervisor and during hot code upgrades.
   """
-  # TODO: Support {:ok, [child_spec], Keyword.t}
-  # TODO: Document options here and update Supervisor.Spec
   @callback init(args :: term) ::
-    {:ok, {:supervisor.sup_flags, [Supervisor.Spec.spec]}} |
+    {:ok, {:supervisor.sup_flags, [:supervisor.child_spec]}} |
     :ignore
 
   @typedoc "Return values of `start_link` functions"
@@ -256,32 +479,45 @@ defmodule Supervisor do
   @typedoc "The Supervisor name"
   @type name :: atom | {:global, term} | {:via, module, term}
 
+  @typedoc "Option values used by the `start*` functions"
+  @type option :: {:name, name} | flag()
+
   @typedoc "Options used by the `start*` functions"
-  @type options :: [name: name,
-                    strategy: Supervisor.Spec.strategy,
-                    max_restarts: non_neg_integer,
-                    max_seconds: non_neg_integer]
+  @type options :: [option, ...]
 
   @typedoc "The supervisor reference"
   @type supervisor :: pid | name | {atom, node}
 
+  @typedoc "Options given to `start_link/2` and `init/2`"
+  @type flag :: {:strategy, strategy} |
+                {:max_restarts, non_neg_integer} |
+                {:max_seconds, pos_integer}
+
+  @typedoc "Supported strategies"
+  @type strategy :: :simple_one_for_one | :one_for_one | :one_for_all | :rest_for_one
+
+  @typedoc "The supervisor specification"
+  @type child_spec :: :supervisor.child_spec()
+
   @doc """
   Starts a supervisor with the given children.
 
-  A strategy is required to be provided through the `:strategy` option.
-  Furthermore, the `:max_restarts` and `:max_seconds` options can be
-  configured as described in the documentation for `Supervisor.Spec.supervise/2`.
+  The children is a list of modules, 2-element tuples with module and
+  arguments or a map with the child specification. A strategy is required
+  to be provided through the `:strategy` option. See
+  "start_link/2, init/2 and strategies" for examples and other options.
 
   The options can also be used to register a supervisor name.
   The supported values are described under the "Name registration"
   section in the `GenServer` module docs.
 
-  If the supervisor and its child processes are successfully created
-  (i.e., if the start function of each child process returns `{:ok, child}`,
+  If the supervisor and its child processes are successfully spawned
+  (if the start function of each child process returns `{:ok, child}`,
   `{:ok, child, info}`, or `:ignore`) this function returns
-  `{:ok, pid}`, where `pid` is the PID of the supervisor. If a process with the
-   specified name already exists, the function returns `{:error,
-   {:already_started, pid}}`, where `pid` is the PID of that process.
+  `{:ok, pid}`, where `pid` is the PID of the supervisor. If the supervisor
+  is given a name and a process with the specified name already exists,
+  the function returns `{:error,  {:already_started, pid}}`, where `pid`
+  is the PID of that process.
 
   If the start function of any of the child processes fails or returns an error
   tuple or an erroneous value, the supervisor first terminates with reason
@@ -292,10 +528,182 @@ defmodule Supervisor do
   process and exits not only on crashes but also if the parent process exits
   with `:normal` reason.
   """
-  @spec start_link([Supervisor.Spec.spec], options) :: on_start
+  @spec start_link([:supervisor.child_spec | {module, term} | module], options) :: on_start
   def start_link(children, options) when is_list(children) do
-    spec = Supervisor.Spec.supervise(children, options)
-    start_link(Supervisor.Default, spec, options)
+    {sup_opts, start_opts} = Keyword.split(options, [:strategy, :max_seconds, :max_restarts])
+    start_link(Supervisor.Default, {children, sup_opts}, start_opts)
+  end
+
+  @doc """
+  Receives a list of children to initialize and a set of options.
+
+  This is typically invoked at the end of the `c:init/1` callback of
+  module-based supervisors. See the sections "Module-based supervisors"
+  and "start_link/2, init/2 and strategies" in the module
+  documentation for more information.
+
+  This function returns a tuple containing the supervisor
+  flags and child specifications.
+
+  ## Examples
+
+      def init(_arg) do
+        Supervisor.init([
+          {Stack, [:hello]}
+        ], strategy: :one_for_one)
+      end
+
+  ## Options
+
+    * `:strategy` - the restart strategy option. It can be either
+      `:one_for_one`, `:rest_for_one`, `:one_for_all`, or
+      `:simple_one_for_one`. You can learn more about strategies
+      in the `Supervisor` module docs.
+
+    * `:max_restarts` - the maximum amount of restarts allowed in
+      a time frame. Defaults to `3`.
+
+    * `:max_seconds` - the time frame in which `:max_restarts` applies.
+      Defaults to `5`.
+
+  The `:strategy` option is required and by default a maximum of 3 restarts
+  is allowed within 5 seconds. Check the `Supervisor` module for a detailed
+  description of the available strategies.
+  """
+  @spec init([:supervisor.child_spec | {module, term} | module], flag) :: {:ok, tuple}
+  def init(children, options) when is_list(children) and is_list(options) do
+    unless strategy = options[:strategy] do
+      raise ArgumentError, "expected :strategy option to be given"
+    end
+    intensity = Keyword.get(options, :max_restarts, 3)
+    period = Keyword.get(options, :max_seconds, 5)
+    flags = %{strategy: strategy, intensity: intensity, period: period}
+    {:ok, {flags, Enum.map(children, &init_child/1)}}
+  end
+
+  defp init_child(module) when is_atom(module) do
+    init_child({module, []})
+  end
+  defp init_child({module, arg}) when is_atom(module) do
+    try do
+      module.child_spec(arg)
+    rescue
+      e in UndefinedFunctionError ->
+        case System.stacktrace do
+          [{^module, :child_spec, [^arg], _} | _] ->
+            raise ArgumentError, child_spec_error(module)
+          stack ->
+            reraise e, stack
+        end
+    end
+  end
+  defp init_child(map) when is_map(map) do
+    map
+  end
+  defp init_child({_, _, _, _, _, _} = tuple) do
+    tuple
+  end
+  defp init_child(other) do
+    raise ArgumentError, """
+    supervisors expect each child to be one of:
+
+      * a module
+      * a {module, arg} tuple
+      * a child specification as a map with at least the :id and :start fields
+      * or a tuple with 6 elements generated by Supervisor.Spec (deprecated)
+
+    Got: #{inspect other}
+    """
+  end
+
+  defp child_spec_error(module) do
+    if Code.ensure_loaded?(module) do
+      """
+      The module #{inspect module} was given as a child to a supervisor
+      but it does not implement child_spec/1.
+
+      If you own the given module, please define a child_spec/1 function
+      that receives an argument and returns a child specification as a map.
+      For example:
+
+          def child_spec(opts) do
+            %{
+              id: __MODULE__,
+              start: {__MODULE__, :start_link, [opts]},
+              type: :worker,
+              restart: :permanent,
+              shutdown: 500
+            }
+          end
+
+      Note that "use Agent", "use GenServer" and so on automatically define
+      this function for you.
+
+      However, if you don't own the given module and it doesn't implement
+      child_spec/1, instead of passing the module name directly as a supervisor
+      child, you will have to pass a child specification as a map:
+
+          %{
+            id: #{inspect module},
+            start: {#{inspect module}, :start_link, [arg1, arg2]}
+          }
+
+      See the Supervisor documentation for more information.
+      """
+    else
+      "The module #{inspect module} was given as a child to a supervisor but it does not exist."
+    end
+  end
+
+  @doc """
+  Builds and overrides a child specification.
+
+  Similar to `start_link/2` and `init/2`, it expects a
+  `module`, `{module, arg}` or a map as the child specification.
+  If a module is given, the specification is retrieved by calling
+  `module.child_spec(arg)`.
+
+  After the child specification is retrieved, the fields on `config`
+  are directly applied on the child spec. If `config` has keys that
+  do not map to any child specification field, an error is raised.
+
+  See the "Child specification" section in the module documentation
+  for all of the available keys for overriding.
+
+  ## Examples
+
+  This function is often used to set an `:id` option when
+  the same module needs to be started multiple times in the
+  supervision tree:
+
+      Supervisor.child_spec({Agent, fn -> :ok end}, id: {Agent, 1})
+      #=> %{id: {Agent, 1},
+            start: {Agent, :start_link, [fn -> :ok end]}}
+
+  It may also be used when there is a need to change the number
+  of arguments when starting a module under a `:simple_one_for_one`
+  strategy, since most args may be given dynamically:
+
+      Supervisor.child_spec(Agent, start: {Agent, :start_link, []})
+      #=> %{id: Agent,
+            start: {Agent, :start_link, []}}
+
+  """
+  @spec child_spec(child_spec() | {module, arg :: term} | module, keyword) :: child_spec()
+  def child_spec(module_or_map, overrides)
+
+  def child_spec({_, _, _, _, _, _} = tuple, _overrides) do
+    raise ArgumentError, "old tuple-based child specification #{inspect tuple} " <>
+                         "is not supported in Supervisor.child_spec/2"
+  end
+
+  def child_spec(module_or_map, overrides) do
+    Enum.reduce overrides, init_child(module_or_map), fn
+      {key, value}, acc when key in [:id, :start, :restart, :shutdown, :type, :modules] ->
+        Map.put(acc, key, value)
+      {key, _value}, _acc ->
+        raise ArgumentError, "unknown key #{inspect key} in child specification override"
+    end
   end
 
   @doc """
@@ -303,8 +711,8 @@ defmodule Supervisor do
 
   To start the supervisor, the `c:init/1` callback will be invoked in the given
   `module`, with `arg` as its argument. The `c:init/1` callback must return a
-  supervisor specification which can be created with the help of the functions
-  in the `Supervisor.Spec` module (especially `Supervisor.Spec.supervise/2`).
+  supervisor specification which can be created with the help of the `init/2`
+  function.
 
   If the `c:init/1` callback returns `:ignore`, this function returns
   `:ignore` as well and the supervisor terminates with reason `:normal`.
@@ -317,7 +725,7 @@ defmodule Supervisor do
   section in the `GenServer` module docs.
   """
   @spec start_link(module, term) :: on_start
-  @spec start_link(module, term, options) :: on_start
+  @spec start_link(module, term, GenServer.options) :: on_start
   def start_link(module, arg, options \\ []) when is_list(options) do
     case Keyword.get(options, :name) do
       nil ->
@@ -372,7 +780,9 @@ defmodule Supervisor do
   returns `{:error, error}` where `error` is a term containing information about
   the error and child specification.
   """
-  @spec start_child(supervisor, Supervisor.Spec.spec | [term]) :: on_start_child
+  # TODO: Once we add DynamicSupervisor, we need to enforce receiving
+  # a map here and deprecate the list and tuple formats.
+  @spec start_child(supervisor, :supervisor.child_spec | [term]) :: on_start_child
   def start_child(supervisor, child_spec_or_args) do
     call(supervisor, {:start_child, child_spec_or_args})
   end
@@ -396,7 +806,7 @@ defmodule Supervisor do
   for the given child id or there is no process with the given PID, this
   function returns `{:error, :not_found}`.
   """
-  @spec terminate_child(supervisor, pid | Supervisor.Spec.child_id) :: :ok | {:error, error}
+  @spec terminate_child(supervisor, pid | term()) :: :ok | {:error, error}
         when error: :not_found | :simple_one_for_one
   def terminate_child(supervisor, pid_or_child_id) do
     call(supervisor, {:terminate_child, pid_or_child_id})
@@ -414,7 +824,7 @@ defmodule Supervisor do
 
   This operation is not supported by `:simple_one_for_one` supervisors.
   """
-  @spec delete_child(supervisor, Supervisor.Spec.child_id) :: :ok | {:error, error}
+  @spec delete_child(supervisor, term()) :: :ok | {:error, error}
         when error: :not_found | :simple_one_for_one | :running | :restarting
   def delete_child(supervisor, child_id) do
     call(supervisor, {:delete_child, child_id})
@@ -444,7 +854,7 @@ defmodule Supervisor do
 
   This operation is not supported by `:simple_one_for_one` supervisors.
   """
-  @spec restart_child(supervisor, Supervisor.Spec.child_id) ::
+  @spec restart_child(supervisor, term()) ::
         {:ok, child} | {:ok, child, term} | {:error, error}
         when error: :not_found | :simple_one_for_one | :running | :restarting | term
   def restart_child(supervisor, child_id) do
@@ -472,10 +882,10 @@ defmodule Supervisor do
 
   """
   @spec which_children(supervisor) ::
-        [{Supervisor.Spec.child_id | :undefined,
-           child | :restarting,
-           Supervisor.Spec.worker,
-           Supervisor.Spec.modules}]
+        [{term() | :undefined,
+          child | :restarting,
+          :worker | :supervisor,
+          :supervisor.modules}]
   def which_children(supervisor) do
     call(supervisor, :which_children)
   end
@@ -505,7 +915,7 @@ defmodule Supervisor do
   end
 
   @doc """
-  Stops the given supervisor with the given `reason`.
+  Synchronously stops the given supervisor with the given `reason`.
 
   It returns `:ok` if the supervisor terminates with the given
   reason. If it terminates with another reason, the call exits.

lib/elixir/lib/supervisor/default.ex

@@ -1,13 +1,7 @@
 defmodule Supervisor.Default do
   @moduledoc false
 
-  @doc """
-  Supervisor callback that simply returns the given args.
-
-  This is the supervisor used by `Supervisor.start_link/2`
-  and others.
-  """
-  def init(args) do
-    args
+  def init({children, opts}) do
+    Supervisor.init(children, opts)
   end
 end

lib/elixir/lib/supervisor/spec.ex

@@ -1,5 +1,9 @@
 defmodule Supervisor.Spec do
   @moduledoc """
+  NOTE: The functions in this module are deprecated and they do not
+  work with the module-based child specs introduced in Elixir v1.5.
+  Please see the `Supervisor` documentation instead.
+
   Convenience functions for defining supervisor specifications.
 
   ## Example
@@ -76,6 +80,10 @@ defmodule Supervisor.Spec do
       terminates abnormally, i.e., with an exit reason other than
       `:normal`, `:shutdown` or `{:shutdown, term}`
 
+  Notice that supervisor that reached maximum restart intensity will exit with `:shutdown` reason.
+  In this case the supervisor will only be restarted if its child specification was defined with
+  the `:restart` option is set to `:permanent` (the default).
+
   ### Shutdown values (:shutdown)
 
   The following shutdown values are supported in the `:shutdown` option:
@@ -96,7 +104,8 @@ defmodule Supervisor.Spec do
 
   """
 
-  # TODO: Update and provide a digest of strategies once we include DynamicSupervisor.
+  # TODO: Deprecate all functions in this module on Elixir v1.8.
+  # Also deprecate entry in Supervisor.Default.
 
   @typedoc "Supported strategies"
   @type strategy :: :simple_one_for_one | :one_for_one | :one_for_all | :rest_for_one
@@ -105,7 +114,7 @@ defmodule Supervisor.Spec do
   @type restart :: :permanent | :transient | :temporary
 
   @typedoc "Supported shutdown values"
-  @type shutdown :: :brutal_kill | :infinity | non_neg_integer
+  @type shutdown :: timeout | :brutal_kill
 
   @typedoc "Supported worker values"
   @type worker :: :worker | :supervisor
@@ -155,9 +164,7 @@ defmodule Supervisor.Spec do
   """
   @spec supervise([spec], strategy: strategy,
                           max_restarts: non_neg_integer,
-                          max_seconds: non_neg_integer) :: {:ok, tuple}
-  # TODO: Make it return a tuple of format {:ok, children, opts}
-  # TODO: Deprecate once the new tuple format has been established
+                          max_seconds: pos_integer) :: {:ok, tuple}
   def supervise(children, options) do
     unless strategy = options[:strategy] do
       raise ArgumentError, "expected :strategy option to be given"
@@ -166,10 +173,21 @@ defmodule Supervisor.Spec do
     maxR = Keyword.get(options, :max_restarts, 3)
     maxS = Keyword.get(options, :max_seconds, 5)
 
-    assert_unique_ids(Enum.map(children, &elem(&1, 0)))
+    assert_unique_ids(Enum.map(children, &get_id/1))
     {:ok, {{strategy, maxR, maxS}, children}}
   end
 
+  defp get_id({id, _, _, _, _, _}) do
+    id
+  end
+  defp get_id(other) do
+    raise ArgumentError,
+      "invalid tuple specification given to supervise/2. If you are trying to use " <>
+      "the map child specification that is part of the Elixir v1.5, use Supervisor.init/2 " <>
+      "instead of Supervisor.Spec.supervise/2. See the Supervisor module for more information. " <>
+      "Got: #{inspect other}"
+  end
+
   defp assert_unique_ids([id | rest]) do
     if id in rest do
       raise ArgumentError,
@@ -233,7 +251,6 @@ defmodule Supervisor.Spec do
     child(:supervisor, module, args, options)
   end
 
-  # TODO: Do and expose proper child validation
   defp child(type, module, args, options) do
     id       = Keyword.get(options, :id, module)
     modules  = Keyword.get(options, :modules, modules(module))
@@ -245,7 +262,6 @@ defmodule Supervisor.Spec do
       restart, shutdown, type, modules}
   end
 
-  # TODO: Remove GenEvent when there is no more GenEvent v2.0
   defp modules(GenEvent), do: :dynamic
   defp modules(module),   do: [module]
 end

lib/elixir/lib/system.ex

@@ -220,7 +220,7 @@ defmodule System do
 
   defp fix_drive_letter([l, ?:, ?/ | rest] = original) when l in ?A..?Z do
     case :os.type() do
-      {:win32, _} -> [l+?a-?A, ?:, ?/ | rest]
+      {:win32, _} -> [l + ?a - ?A, ?:, ?/ | rest]
       _ -> original
     end
   end
@@ -339,6 +339,8 @@ defmodule System do
   """
   @spec find_executable(binary) :: binary | nil
   def find_executable(program) when is_binary(program) do
+    assert_no_null_byte!(program, "System.find_executable/1")
+
     case :os.find_executable(String.to_charlist(program)) do
       false -> nil
       other -> List.to_string(other)
@@ -346,10 +348,10 @@ defmodule System do
   end
 
   @doc """
-  System environment variables.
+  Returns all system environment variables.
 
-  Returns a list of all environment variables. Each variable is given as a
-  `{name, value}` tuple where both `name` and `value` are strings.
+  The returned value is a map containing name-value pairs.
+  Variable names and their values are strings.
   """
   @spec get_env() :: %{optional(String.t) => String.t}
   def get_env do
@@ -361,13 +363,13 @@ defmodule System do
   end
 
   @doc """
-  Environment variable value.
+  Returns the value of the given environment variable.
 
-  Returns the value of the environment variable
-  `varname` as a binary, or `nil` if the environment
+  The returned value of the environment variable
+  `varname` is a string, or `nil` if the environment
   variable is undefined.
   """
-  @spec get_env(binary) :: binary | nil
+  @spec get_env(String.t) :: String.t | nil
   def get_env(varname) when is_binary(varname) do
     case :os.getenv(String.to_charlist(varname)) do
       false -> nil
@@ -393,8 +395,13 @@ defmodule System do
   """
   @spec put_env(binary, binary) :: :ok
   def put_env(varname, value) when is_binary(varname) and is_binary(value) do
-   :os.putenv String.to_charlist(varname), String.to_charlist(value)
-   :ok
+    case :binary.match(varname, "=") do
+      {_, _} ->
+        raise ArgumentError, "cannot execute System.put_env/2 for key with \"=\", got: #{inspect varname}"
+      :nomatch ->
+        :os.putenv String.to_charlist(varname), String.to_charlist(value)
+        :ok
+    end
   end
 
   @doc """
@@ -433,10 +440,13 @@ defmodule System do
   end
 
   @doc """
-  Halts the Erlang runtime system.
+  Immediately halts the Erlang runtime system.
 
-  Halts the Erlang runtime system where the argument `status` must be a
-  non-negative integer, the atom `:abort` or a binary.
+  Terminates the Erlang runtime system without properly shutting down
+  applications and ports. Please see `stop/1` for a careful shutdown of the
+  system.
+
+  `status` must be a non-negative integer, the atom `:abort` or a binary.
 
     * If an integer, the runtime system exits with the integer value which
       is returned to the operating system.
@@ -459,7 +469,6 @@ defmodule System do
       System.halt(:abort)
 
   """
-  @spec halt() :: no_return
   @spec halt(non_neg_integer | binary | :abort) :: no_return
   def halt(status \\ 0)
 
@@ -471,6 +480,37 @@ defmodule System do
     :erlang.halt(String.to_charlist(status))
   end
 
+  @doc """
+  Carefully stops the Erlang runtime system.
+
+  All applications are taken down smoothly, all code is unloaded, and all ports
+  are closed before the system terminates by calling `halt/1`.
+
+  `status` must be a non-negative integer value which is returned by the
+  runtime system to the operating system.
+
+  Note that on many platforms, only the status codes 0-255 are supported
+  by the operating system.
+
+  For more information, see [`:init.stop/1`](http://erlang.org/doc/man/init.html#stop-1).
+
+  ## Examples
+
+      System.stop(0)
+      System.stop(1)
+
+  """
+  @spec stop(non_neg_integer | binary) :: no_return
+  def stop(status \\ 0)
+
+  def stop(status) when is_integer(status) do
+    :init.stop(status)
+  end
+
+  def stop(status) when is_binary(status) do
+    :init.stop(String.to_charlist(status))
+  end
+
   @doc ~S"""
   Executes the given `command` with `args`.
 
@@ -491,7 +531,7 @@ defmodule System do
   Internally, this function uses a `Port` for interacting with the
   outside world. However, if you plan to run a long-running program,
   ports guarantee stdin/stdout devices will be closed but it does not
-  automatically terminate the problem. The documentation for the
+  automatically terminate the program. The documentation for the
   `Port` module describes this problem and possible solutions under
   the "Zombie processes" section.
 
@@ -552,9 +592,10 @@ defmodule System do
   redirecting and so on, please check
   [`:os.cmd/1`](http://www.erlang.org/doc/man/os.html#cmd-1).
   """
-  @spec cmd(binary, [binary], Keyword.t) ::
+  @spec cmd(binary, [binary], keyword) ::
         {Collectable.t, exit_status :: non_neg_integer}
   def cmd(command, args, opts \\ []) when is_binary(command) and is_list(args) do
+    assert_no_null_byte!(command, "System.cmd/3")
     cmd = String.to_charlist(command)
 
     cmd =
@@ -800,6 +841,15 @@ defmodule System do
     :erlang.unique_integer(modifiers)
   end
 
+  defp assert_no_null_byte!(binary, operation) do
+    case :binary.match(binary, "\0") do
+      {_, _} ->
+        raise ArgumentError, "cannot execute #{operation} for program with null byte, got: #{inspect binary}"
+      :nomatch ->
+        binary
+    end
+  end
+
   defp normalize_time_unit(:native),
     do: :native
 

lib/elixir/lib/task.ex

@@ -56,27 +56,51 @@ defmodule Task do
 
   ## Supervised tasks
 
-  It is also possible to spawn a task under a supervisor:
+  It is also possible to spawn a task under a supervisor.
+  It is often done by defining the task in its own module:
 
-      import Supervisor.Spec
+      defmodule MyTask do
+        use Task
 
-      children = [
-        #
-        worker(Task, [fn -> IO.puts "ok" end])
-      ]
+        def start_link(arg) do
+          Task.start_link(__MODULE__, :run, [arg])
+        end
 
-  Internally the supervisor will invoke `Task.start_link/1`.
+        def run(arg) do
+          # ...
+        end
+      end
+
+  And then passing it to the supervisor:
+
+      Supervisor.start_link([MyTask])
 
   Since these tasks are supervised and not directly linked to
   the caller, they cannot be awaited on. Note `start_link/1`,
-  unlike `async/1`, returns `{:ok, pid}` (which is
-  the result expected by supervision trees).
+  unlike `async/1`, returns `{:ok, pid}` (which is the result
+  expected by supervisors).
 
-  By default, most supervision strategies will try to restart
-  a worker after it exits regardless of the reason. If you design
-  the task to terminate normally (as in the example with `IO.puts/2`
-  above), consider passing `restart: :transient` in the options
-  to `Supervisor.Spec.worker/3`.
+  Note `use Task` defines a `child_spec/1` function, allowing the
+  defined module to be put under a supervision tree. The generated
+  `child_spec/1` can be customized with the following options:
+
+    * `:id` - the child specification id, defauts to the current module
+    * `:start` - how to start the child process (defaults to calling `__MODULE__.start_link/1`)
+    * `:restart` - when the child should be restarted, defaults to `:temporary`
+    * `:shutdown` - how to shut down the child
+
+  Opposite to `GenServer`, `Agent` and `Supervisor`, a Task has
+  a default `:restart` of `:temporary`. This means the task will
+  not be restarted even if it crashes. If you desire the task to
+  be restarted for non-successful exists, do:
+
+      use Task, restart: :transient
+
+  If you want the task to always be restarted:
+
+      use Task, restart: :permanent
+
+  See the `Supervisor` docs for more information.
 
   ## Dynamically supervised tasks
 
@@ -94,11 +118,9 @@ defmodule Task do
   However, in the majority of cases, you want to add the task supervisor
   to your supervision tree:
 
-      import Supervisor.Spec
-
-      children = [
-        supervisor(Task.Supervisor, [[name: MyApp.TaskSupervisor]])
-      ]
+      Supervisor.start_link([
+        {Task.Supervisor, name: MyApp.TaskSupervisor}
+      ])
 
   Now you can dynamically start supervised tasks:
 
@@ -151,10 +173,41 @@ defmodule Task do
 
   @type t :: %__MODULE__{}
 
+  @doc false
+  def child_spec(arg) do
+    %{
+      id: Task,
+      start: {Task, :start_link, [arg]},
+      restart: :temporary
+    }
+  end
+
+  @doc false
+  defmacro __using__(opts) do
+    quote location: :keep, bind_quoted: [opts: opts] do
+      spec = [
+        id: opts[:id] || __MODULE__,
+        start: Macro.escape(opts[:start]) || quote(do: {__MODULE__, :start_link, [arg]}),
+        restart: opts[:restart] || :temporary,
+        shutdown: opts[:shutdown] || 5000,
+        type: :worker
+      ]
+
+      @doc false
+      def child_spec(arg) do
+        %{unquote_splicing(spec)}
+      end
+
+      defoverridable child_spec: 1
+    end
+  end
+
   @doc """
-  Starts a task as part of a supervision tree.
+  Starts a process linked to the current process.
+
+  This is often used to start the process as part of a supervision tree.
   """
-  @spec start_link(fun) :: {:ok, pid}
+  @spec start_link((() -> any)) :: {:ok, pid}
   def start_link(fun) do
     start_link(:erlang, :apply, [fun, []])
   end
@@ -174,7 +227,7 @@ defmodule Task do
   (i.e. no interest in the returned result) and it should not
   be linked to the current process.
   """
-  @spec start(fun) :: {:ok, pid}
+  @spec start((() -> any)) :: {:ok, pid}
   def start(fun) do
     start(:erlang, :apply, [fun, []])
   end
@@ -203,7 +256,7 @@ defmodule Task do
 
   See also `async/3`.
   """
-  @spec async(fun) :: t
+  @spec async((() -> any)) :: t
   def async(fun) do
     async(:erlang, :apply, [fun, []])
   end
@@ -282,7 +335,7 @@ defmodule Task do
   end
 
   @doc """
-  Returns a stream that runs the given `module`, `function` and `args`
+  Returns a stream that runs the given `module`, `function`, and `args`
   concurrently on each item in `enumerable`.
 
   Each item will be prepended to the given `args` and processed by its
@@ -291,14 +344,14 @@ defmodule Task do
   terminates the current process and a failure in the current process
   terminates all tasks.
 
-  When streamed, each task will emit `{:ok, val}` upon successful
-  completion or `{:exit, val}` if the caller is trapping exits. Results
-  are emitted in the same order as the original `enumerable`.
+  When streamed, each task will emit `{:ok, value}` upon successful
+  completion or `{:exit, reason}` if the caller is trapping exits.
+  Results are emitted in the same order as the original `enumerable`.
 
   The level of concurrency can be controlled via the `:max_concurrency`
-  option and defaults to `System.schedulers_online/0`. The timeout
-  can also be given as option and defaults to 5000 and it defaults to
-  the maximum amount of time to wait without a task reply.
+  option and defaults to `System.schedulers_online/0`. A timeout
+  can also be given as an option representing the maximum amount of
+  time to wait without a task reply.
 
   Finally, consider using `Task.Supervisor.async_stream/6` to start tasks
   under a supervisor. If you find yourself trapping exits to handle exits
@@ -309,9 +362,17 @@ defmodule Task do
 
     * `:max_concurrency` - sets the maximum number of tasks to run
       at the same time. Defaults to `System.schedulers_online/0`.
-    * `:timeout` - the maximum amount of time to wait without
-      receiving a task reply (across all running tasks).
-      Defaults to `5000`.
+    * `:ordered` - whether the results should be returned in the same order
+      as the input stream. This option is useful when you have large
+      streams and don't want to buffer results before they are delivered.
+      Defaults to `true`.
+    * `:timeout` - the maximum amount of time (in milliseconds) each
+      task is allowed to execute for. Defaults to `5000`.
+    * `:on_timeout` - what do to when a task times out. The possible
+      values are:
+      * `:exit` (default) - the process that spawned the tasks exits.
+      * `:kill_task` - the task that timed out is killed. The value
+        emitted for that task is `{:exit, :timeout}`.
 
   ## Example
 
@@ -328,23 +389,32 @@ defmodule Task do
       Enum.to_list(stream)
 
   """
-  @spec async_stream(Enumerable.t, module, atom, [term], Keyword.t) :: Enumerable.t
+  @spec async_stream(Enumerable.t, module, atom, [term], keyword) :: Enumerable.t
   def async_stream(enumerable, module, function, args, options \\ [])
       when is_atom(module) and is_atom(function) and is_list(args) do
     build_stream(enumerable, {module, function, args}, options)
   end
 
   @doc """
-  Returns a stream that runs the given `function` concurrently on each
-  item in `enumerable`.
+  Returns a stream that runs the given function `fun` concurrently
+  on each item in `enumerable`.
 
-  Each `enumerable` item is passed as argument to the `function` and
-  processed by its own task. The tasks will be linked to the current
-  process, similar to `async/1`.
+  Each `enumerable` item is passed as argument to the given function `fun` and
+  processed by its own task. The tasks will be linked to the current process,
+  similarly to `async/1`.
 
-  See `async_stream/5` for discussion and examples.
+  ## Example
+
+  Count the codepoints in each string asynchronously, then add the counts together using reduce.
+
+      iex> strings = ["long string", "longer string", "there are many of these"]
+      iex> stream = Task.async_stream(strings, fn text -> text |> String.codepoints |> Enum.count end)
+      iex> Enum.reduce(stream, 0, fn {:ok, num}, acc -> num + acc end)
+      47
+
+  See `async_stream/5` for discussion, options, and more examples.
   """
-  @spec async_stream(Enumerable.t, (term -> term), Keyword.t) :: Enumerable.t
+  @spec async_stream(Enumerable.t, (term -> term), keyword) :: Enumerable.t
   def async_stream(enumerable, fun, options \\ []) when is_function(fun, 1) do
     build_stream(enumerable, fun, options)
   end
@@ -355,12 +425,17 @@ defmodule Task do
     end)
   end
 
-  defp get_info(self) do
-    {node(),
-     case Process.info(self, :registered_name) do
-       {:registered_name, []} -> self()
-       {:registered_name, name} -> name
-     end}
+  # Returns a tuple with the node where this is executed and either the
+  # registered name of the given pid or the pid of where this is executed. Used
+  # when exiting from tasks to print out from where the task was started.
+  defp get_info(pid) do
+    self_or_name =
+      case Process.info(pid, :registered_name) do
+        {:registered_name, []} -> self()
+        {:registered_name, name} -> name
+      end
+
+    {node(), self_or_name}
   end
 
   @doc """
@@ -419,12 +494,8 @@ defmodule Task do
 
   @doc false
   # TODO: Remove on 2.0
-  def find(tasks, msg) do
-    IO.warn "Task.find/2 is deprecated, please match on the message directly"
-    do_find(tasks, msg)
-  end
-
-  defp do_find(tasks, {ref, reply}) when is_reference(ref) do
+  # (hard-deprecated in elixir_dispatch)
+  def find(tasks, {ref, reply}) when is_reference(ref) do
     Enum.find_value tasks, fn
       %Task{ref: ^ref} = task ->
         Process.demonitor(ref, [:flush])
@@ -434,14 +505,14 @@ defmodule Task do
     end
   end
 
-  defp do_find(tasks, {:DOWN, ref, _, proc, reason} = msg) when is_reference(ref) do
+  def find(tasks, {:DOWN, ref, _, proc, reason} = msg) when is_reference(ref) do
     find = fn %Task{ref: task_ref} -> task_ref == ref end
     if Enum.find(tasks, find) do
       exit({reason(reason, proc), {__MODULE__, :find, [tasks, msg]}})
     end
   end
 
-  defp do_find(_tasks, _msg) do
+  def find(_tasks, _msg) do
     nil
   end
 
@@ -575,7 +646,7 @@ defmodule Task do
     end
   end
 
-  defp yield_many([%Task{ref: ref, owner: owner}=task | rest], timeout_ref, timeout) do
+  defp yield_many([%Task{ref: ref, owner: owner} = task | rest], timeout_ref, timeout) do
     if owner != self() do
       raise ArgumentError, invalid_owner_error(task)
     end

lib/elixir/lib/task/supervised.ex

@@ -145,73 +145,120 @@ defmodule Task.Supervised do
   def stream(enumerable, acc, reducer, mfa, options, spawn) do
     next = &Enumerable.reduce(enumerable, &1, fn x, acc -> {:suspend, [x | acc]} end)
     max_concurrency = Keyword.get(options, :max_concurrency, System.schedulers_online)
+    ordered? = Keyword.get(options, :ordered, true)
     timeout = Keyword.get(options, :timeout, 5000)
+    on_timeout = Keyword.get(options, :on_timeout, :exit)
     parent = self()
 
-    # Start a process responsible for translating down messages.
-    {:trap_exit, trap_exit} =
-      Process.info(self(), :trap_exit)
-    {monitor_pid, monitor_ref} =
-      Process.spawn(fn -> stream_monitor(parent, mfa, spawn, trap_exit) end, [:link, :monitor])
+    {:trap_exit, trap_exit?} = Process.info(self(), :trap_exit)
 
+    # Start a process responsible for spawning processes and translating "down"
+    # messages. This process will trap exits if the current process is trapping
+    # exit, or it won't trap exits otherwise.
+    {monitor_pid, monitor_ref} =
+      Process.spawn(fn ->
+        stream_monitor(parent, mfa, spawn, trap_exit?, timeout)
+      end, [:link, :monitor])
+
+    # Now that we have the pid of the "monitor" process and the reference of the
+    # monitor we use to monitor such process, we can inform the monitor process
+    # about our reference to it.
     send(monitor_pid, {parent, monitor_ref})
 
-    stream_reduce(acc, max_concurrency, 0, 0, %{}, next,
-                  reducer, monitor_pid, monitor_ref, timeout)
+    config = %{
+      reducer: reducer,
+      monitor_pid: monitor_pid,
+      monitor_ref: monitor_ref,
+      ordered: ordered?,
+      timeout: timeout,
+      on_timeout: on_timeout,
+    }
+    stream_reduce(acc, max_concurrency, _spawned = 0, _delivered = 0, _waiting = %{}, next, config)
   end
 
-  defp stream_reduce({:halt, acc}, _max, _spawned, _delivered, _waiting, next,
-                     _reducer, monitor_pid, monitor_ref, timeout) do
+  defp stream_reduce({:halt, acc}, _max, _spawned, _delivered, _waiting, next, config) do
+    %{monitor_pid: monitor_pid, monitor_ref: monitor_ref, timeout: timeout} = config
     stream_close(monitor_pid, monitor_ref, timeout)
     is_function(next) && next.({:halt, []})
     {:halted, acc}
   end
 
-  defp stream_reduce({:suspend, acc}, max, spawned, delivered, waiting, next,
-                     reducer, monitor_pid, monitor_ref, timeout) do
-    {:suspended, acc, &stream_reduce(&1, max, spawned, delivered, waiting, next,
-                                     reducer, monitor_pid, monitor_ref, timeout)}
+  defp stream_reduce({:suspend, acc}, max, spawned, delivered, waiting, next, config) do
+    continuation = &stream_reduce(&1, max, spawned, delivered, waiting, next, config)
+    {:suspended, acc, continuation}
   end
 
-  # All spawned, all delivered, next is done.
+  # All spawned, all delivered, next is :done.
   defp stream_reduce({:cont, acc}, _max, spawned, delivered, _waiting, next,
-                     _reducer, monitor_pid, monitor_ref, timeout)
+                     %{monitor_pid: monitor_pid, monitor_ref: monitor_ref, timeout: timeout})
        when spawned == delivered and next == :done do
     stream_close(monitor_pid, monitor_ref, timeout)
     {:done, acc}
   end
 
-  # No more tasks to spawned because max == 0 or next is done.
-  defp stream_reduce({:cont, acc}, max, spawned, delivered, waiting, next,
-                     reducer, monitor_pid, monitor_ref, timeout)
+  # No more tasks to spawn because max == 0 or next is :done. We wait for task
+  # responses or tasks going down.
+  defp stream_reduce({:cont, acc}, max, spawned, delivered, waiting, next, config)
        when max == 0
        when next == :done do
+    %{monitor_pid: monitor_pid, monitor_ref: monitor_ref,
+      timeout: timeout, on_timeout: on_timeout, ordered: ordered?} = config
+
     receive do
-      {{^monitor_ref, position}, value} ->
+      # The task at position "position" replied with "value". We put the
+      # response in the "waiting" map and do nothing, since we'll only act on
+      # this response when the replying task dies (we'll notice in the :down
+      # message).
+      {{^monitor_ref, position}, reply} ->
         %{^position => {pid, :running}} = waiting
-        waiting = Map.put(waiting, position, {pid, {:ok, value}})
-        stream_reduce({:cont, acc}, max, spawned, delivered, waiting, next,
-                      reducer, monitor_pid, monitor_ref, timeout)
-      {:down, {^monitor_ref, position}, reason} ->
-        waiting =
+        waiting = Map.put(waiting, position, {pid, {:ok, reply}})
+        stream_reduce({:cont, acc}, max, spawned, delivered, waiting, next, config)
+
+      # The task at position "position" died for some reason. We check if it
+      # replied already (then the death is peaceful) or if it's still running
+      # (then the reply from this task will be {:exit, reason}). This message is
+      # sent to us by the monitor process, not by the dying task directly.
+      {kind, {^monitor_ref, position}, reason} when kind in [:down , :timed_out] ->
+        result =
           case waiting do
-            %{^position => {_, {:ok, _} = ok}} -> Map.put(waiting, position, {nil, ok})
-            %{^position => {_, :running}} -> Map.put(waiting, position, {nil, {:exit, reason}})
+            # If the task replied, we don't care whether it went down for timeout
+            # or for normal reasons.
+            %{^position => {_, {:ok, _} = ok}} ->
+              ok
+            # If the task exited by itself before replying, we emit {:exit, reason}.
+            %{^position => {_, :running}} when kind == :down ->
+              {:exit, reason}
+            # If the task timed out before replying, we either exit (on_timeout: :exit)
+            # or emit {:exit, :timeout} (on_timeout: :kill_task) (note the task is already
+            # dead at this point).
+            %{^position => {_, :running}} when kind == :timed_out ->
+              if on_timeout == :exit do
+                stream_cleanup_inbox(monitor_pid, monitor_ref)
+                exit({:timeout, {__MODULE__, :stream, [timeout]}})
+              else
+                {:exit, :timeout}
+              end
           end
-        stream_deliver({:cont, acc}, max + 1, spawned, delivered, waiting, next,
-                       reducer, monitor_pid, monitor_ref, timeout)
+
+        if ordered? do
+          waiting = Map.put(waiting, position, {:done, result})
+          stream_deliver({:cont, acc}, max + 1, spawned, delivered, waiting, next, config)
+        else
+          pair = deliver_now(result, acc, next, config)
+          stream_reduce(pair, max + 1, spawned, delivered + 1, waiting, next, config)
+        end
+
+      # The monitor process died. We just cleanup the messages from the monitor
+      # process and exit.
       {:DOWN, ^monitor_ref, _, ^monitor_pid, reason} ->
         stream_cleanup_inbox(monitor_pid, monitor_ref)
         exit({reason, {__MODULE__, :stream, [timeout]}})
-    after
-      timeout ->
-        stream_close(monitor_pid, monitor_ref, timeout)
-        exit({:timeout, {__MODULE__, :stream, [timeout]}})
     end
   end
 
-  defp stream_reduce({:cont, acc}, max, spawned, delivered, waiting, next,
-                     reducer, monitor_pid, monitor_ref, timeout) do
+  defp stream_reduce({:cont, acc}, max, spawned, delivered, waiting, next, config) do
+    %{monitor_pid: monitor_pid, monitor_ref: monitor_ref,
+      timeout: timeout} = config
     try do
       next.({:cont, []})
     catch
@@ -222,32 +269,44 @@ defmodule Task.Supervised do
     else
       {:suspended, [value], next} ->
         waiting = stream_spawn(value, spawned, waiting, monitor_pid, monitor_ref, timeout)
-        stream_reduce({:cont, acc}, max - 1, spawned + 1, delivered, waiting, next,
-                      reducer, monitor_pid, monitor_ref, timeout)
+        stream_reduce({:cont, acc}, max - 1, spawned + 1, delivered, waiting, next, config)
       {_, [value]} ->
         waiting = stream_spawn(value, spawned, waiting, monitor_pid, monitor_ref, timeout)
-        stream_reduce({:cont, acc}, max - 1, spawned + 1, delivered, waiting, :done,
-                      reducer, monitor_pid, monitor_ref, timeout)
+        stream_reduce({:cont, acc}, max - 1, spawned + 1, delivered, waiting, :done, config)
       {_, []} ->
-        stream_reduce({:cont, acc}, max, spawned, delivered, waiting, :done,
-                      reducer, monitor_pid, monitor_ref, timeout)
+        stream_reduce({:cont, acc}, max, spawned, delivered, waiting, :done, config)
     end
   end
 
-  defp stream_deliver({:suspend, acc}, max, spawned, delivered, waiting, next,
-                      reducer, monitor_pid, monitor_ref, timeout) do
-    {:suspended, acc, &stream_deliver(&1, max, spawned, delivered, waiting, next,
-                                      reducer, monitor_pid, monitor_ref, timeout)}
+  defp deliver_now(reply, acc, next, config) do
+    %{reducer: reducer, monitor_pid: monitor_pid,
+      monitor_ref: monitor_ref, timeout: timeout} = config
+    try do
+      reducer.(reply, acc)
+    catch
+      kind, reason ->
+        stacktrace = System.stacktrace()
+        is_function(next) && next.({:halt, []})
+        stream_close(monitor_pid, monitor_ref, timeout)
+        :erlang.raise(kind, reason, stacktrace)
+    end
   end
-  defp stream_deliver({:halt, acc}, max, spawned, delivered, waiting, next,
-                      reducer, monitor_pid, monitor_ref, timeout) do
-    stream_reduce({:halt, acc}, max, spawned, delivered, waiting, next,
-                  reducer, monitor_pid, monitor_ref, timeout)
+
+  defp stream_deliver({:suspend, acc}, max, spawned, delivered, waiting, next, config) do
+    continuation = &stream_deliver(&1, max, spawned, delivered, waiting, next, config)
+    {:suspended, acc, continuation}
   end
-  defp stream_deliver({:cont, acc}, max, spawned, delivered, waiting, next,
-                      reducer, monitor_pid, monitor_ref, timeout) do
+
+  defp stream_deliver({:halt, acc}, max, spawned, delivered, waiting, next, config) do
+    stream_reduce({:halt, acc}, max, spawned, delivered, waiting, next, config)
+  end
+
+  defp stream_deliver({:cont, acc}, max, spawned, delivered, waiting, next, config) do
+    %{reducer: reducer, monitor_pid: monitor_pid,
+      monitor_ref: monitor_ref, timeout: timeout} = config
+
     case waiting do
-      %{^delivered => {nil, reply}} ->
+      %{^delivered => {:done, reply}} ->
         try do
           reducer.(reply, acc)
         catch
@@ -258,12 +317,10 @@ defmodule Task.Supervised do
             :erlang.raise(kind, reason, stacktrace)
         else
           pair ->
-            stream_deliver(pair, max, spawned, delivered + 1, Map.delete(waiting, delivered), next,
-                           reducer, monitor_pid, monitor_ref, timeout)
+            stream_deliver(pair, max, spawned, delivered + 1, Map.delete(waiting, delivered), next, config)
         end
       %{} ->
-        stream_reduce({:cont, acc}, max, spawned, delivered, waiting, next,
-                      reducer, monitor_pid, monitor_ref, timeout)
+        stream_reduce({:cont, acc}, max, spawned, delivered, waiting, next, config)
     end
   end
 
@@ -291,7 +348,7 @@ defmodule Task.Supervised do
     receive do
       {{^monitor_ref, _}, _} ->
         stream_cleanup_inbox(monitor_ref)
-      {:down, {^monitor_ref, _}, _} ->
+      {kind, {^monitor_ref, _}, _} when kind in [:down, :timed_out] ->
         stream_cleanup_inbox(monitor_ref)
     after
       0 ->
@@ -299,9 +356,8 @@ defmodule Task.Supervised do
     end
   end
 
-  defp stream_mfa({mod, fun, args}, arg), do: {mod, fun, [arg | args]}
-  defp stream_mfa(fun, arg), do: {:erlang, :apply, [fun, [arg]]}
-
+  # This function spawns a task for the given "value", and puts the pid of this
+  # new task in the map of "waiting" tasks, which is returned.
   defp stream_spawn(value, spawned, waiting, monitor_pid, monitor_ref, timeout) do
     send(monitor_pid, {:spawn, spawned, value})
 
@@ -315,45 +371,126 @@ defmodule Task.Supervised do
     end
   end
 
-  defp stream_monitor(parent_pid, mfa, spawn, trap_exit) do
-    Process.flag(:trap_exit, trap_exit)
+  defp stream_monitor(parent_pid, mfa, spawn, trap_exit?, timeout) do
+    Process.flag(:trap_exit, trap_exit?)
+
     parent_ref = Process.monitor(parent_pid)
+
+    # Let's wait for the parent process to tell this process the monitor ref
+    # it's using to monitor this process. If the parent process dies while this
+    # process waits, this process dies with the same reason.
     receive do
       {^parent_pid, monitor_ref} ->
-        stream_monitor(parent_pid, parent_ref, mfa, spawn, monitor_ref, %{})
+        config = %{
+          parent_pid: parent_pid,
+          parent_ref: parent_ref,
+          mfa: mfa,
+          spawn: spawn,
+          monitor_ref: monitor_ref,
+          timeout: timeout,
+        }
+        stream_monitor_loop(_running_tasks = %{}, config)
       {:DOWN, ^parent_ref, _, _, reason} ->
         exit(reason)
     end
   end
 
-  defp stream_monitor(parent_pid, parent_ref, mfa, spawn, monitor_ref, counters) do
+  defp stream_monitor_loop(running_tasks, config) do
+    %{parent_pid: parent_pid, parent_ref: parent_ref, mfa: mfa,
+      spawn: spawn, monitor_ref: monitor_ref, timeout: timeout} = config
+
     receive do
-      {:spawn, counter, value} ->
-        {type, pid} = spawn.(parent_pid, stream_mfa(mfa, value))
+      # The parent process is telling us to spawn a new task to process
+      # "value". We spawn it and notify the parent about its pid.
+      {:spawn, position, value} ->
+        {type, pid} = spawn.(parent_pid, normalize_mfa_with_arg(mfa, value))
         ref = Process.monitor(pid)
-        send(parent_pid, {:spawned, {monitor_ref, counter}, pid})
-        counters = Map.put(counters, ref, {counter, type, pid})
-        stream_monitor(parent_pid, parent_ref, mfa, spawn, monitor_ref, counters)
+
+        # Schedule a timeout message to ourselves, unless the timeout was set to :infinity
+        timer_ref = case timeout do
+          :infinity -> nil
+          timeout -> Process.send_after(self(), {:timeout, {monitor_ref, ref}}, timeout)
+        end
+
+        send(parent_pid, {:spawned, {monitor_ref, position}, pid})
+        task_info = %{
+          position: position,
+          type: type,
+          pid: pid,
+          timer_ref: timer_ref,
+          timed_out?: false,
+        }
+        running_tasks = Map.put(running_tasks, ref, task_info)
+        stream_monitor_loop(running_tasks, config)
+
+      # The parent process is telling us to stop because the stream is being
+      # closed. In this case, we forcely kill all spawned processes and then
+      # exit gracefully ourselves.
       {:stop, ^monitor_ref} ->
         Process.flag(:trap_exit, true)
-        for {ref, {_counter, _, pid}} <- counters do
+        for {ref, %{pid: pid}} <- running_tasks do
           Process.exit(pid, :kill)
           receive do
             {:DOWN, ^ref, _, _, _} -> :ok
           end
         end
         exit(:normal)
+
+      # The parent process went down with a given reason. We kill all the
+      # spawned processes (that are also linked) with the same reason, and then
+      # exit ourself with the same reason.
       {:DOWN, ^parent_ref, _, _, reason} ->
-        for {_ref, {_counter, :link, pid}} <- counters do
+        for {_ref, %{type: :link, pid: pid}} <- running_tasks do
           Process.exit(pid, reason)
         end
         exit(reason)
+
+      # One of the spawned processes went down. We inform the parent process of
+      # this and keep going.
       {:DOWN, ref, _, _, reason} ->
-        {{counter, _, _}, counters} = Map.pop(counters, ref)
-        send(parent_pid, {:down, {monitor_ref, counter}, reason})
-        stream_monitor(parent_pid, parent_ref, mfa, spawn, monitor_ref, counters)
+        {%{position: position, timer_ref: timer_ref, timed_out?: timed_out?}, running_tasks} = Map.pop(running_tasks, ref)
+        if timer_ref != nil do
+          :ok = Process.cancel_timer(timer_ref, async: true, info: false)
+        end
+        message_kind = if(timed_out?, do: :timed_out, else: :down)
+        send(parent_pid, {message_kind, {monitor_ref, position}, reason})
+        stream_monitor_loop(running_tasks, config)
+
+      # One of the spawned processes timed out. We kill that process here
+      # regardless of the value of :on_timeout. We then send a message to the
+      # parent process informing it that a task timed out, and the parent
+      # process decides what to do.
+      {:timeout, {^monitor_ref, ref}} ->
+        running_tasks =
+          case running_tasks do
+            %{^ref => %{pid: pid, timed_out?: false} = task_info} ->
+              unlink_and_kill(pid)
+              Map.put(running_tasks, ref, %{task_info | timed_out?: true})
+            _other ->
+              running_tasks
+          end
+        stream_monitor_loop(running_tasks, config)
+
       {:EXIT, _, _} ->
-        stream_monitor(parent_pid, parent_ref, mfa, spawn, monitor_ref, counters)
+        stream_monitor_loop(running_tasks, config)
     end
   end
+
+  defp unlink_and_kill(pid) do
+    caller = self()
+    ref = make_ref()
+    enforcer = spawn(fn ->
+      mon = Process.monitor(caller)
+      receive do
+        {:done, ^ref} -> :ok
+        {:DOWN, ^mon, _, _, _} -> Process.exit(pid, :kill)
+      end
+    end)
+    Process.unlink(pid)
+    Process.exit(pid, :kill)
+    send(enforcer, {:done, ref})
+  end
+
+  defp normalize_mfa_with_arg({mod, fun, args}, arg), do: {mod, fun, [arg | args]}
+  defp normalize_mfa_with_arg(fun, arg), do: {:erlang, :apply, [fun, [arg]]}
 end

lib/elixir/lib/task/supervisor.ex

@@ -3,13 +3,10 @@ defmodule Task.Supervisor do
   A task supervisor.
 
   This module defines a supervisor which can be used to dynamically
-  supervise tasks. Behind the scenes, this module is implemented as a
-  `:simple_one_for_one` supervisor where the workers are temporary by
-  default (that is, they are not restarted after they die; read the docs
-  for `start_link/1` for more information on choosing the restart
-  strategy).
+  supervise tasks.
 
-  See the `Task` module for more information.
+  `start_link/1` can be used to start the supervisor. See the `Task`
+  module for more examples.
 
   ## Name registration
 
@@ -17,33 +14,55 @@ defmodule Task.Supervisor do
   `GenServer`. Read more about them in the `GenServer` docs.
   """
 
+  @typedoc "Option values used by `start_link`"
+  @type option :: Supervisor.option |
+                  {:restart, :supervisor.restart} |
+                  {:shutdown, :supervisor.shutdown}
+
+  @doc false
+  def child_spec(arg) do
+    %{
+      id: Task.Supervivsor,
+      start: {Task.Supervisor, :start_link, [arg]},
+      type: :supervisor
+    }
+  end
+
   @doc """
   Starts a new supervisor.
 
   The supported options are:
 
-  * `:name` - used to register a supervisor name, the supported values are
-    described under the `Name Registration` section in the `GenServer` module
-    docs;
+    * `:name` - used to register a supervisor name, the supported values are
+      described under the `Name Registration` section in the `GenServer` module
+      docs;
 
-  * `:restart` - the restart strategy, may be `:temporary` (the default),
-    `:transient` or `:permanent`. Check `Supervisor.Spec` for more info.
-    Defaults to `:temporary` so tasks aren't automatically restarted when
-    they complete nor in case of crashes;
+    * `:restart` - the restart strategy, may be `:temporary` (the default),
+      `:transient` or `:permanent`. `:temporary` means the task is never
+      restarted, `:transient` means it is restarted if the exit is not
+      `:normal`, `:shutdown` or `{:shutdown, reason}`. A `:permanent` restart
+      strategy means it is always restarted. It defaults to `:temporary` so
+      tasks aren't automatically restarted when they complete nor in case of
+      crashes. Note the `:async` functions in this module support only `:temporary`
+      restarts;
 
-  * `:shutdown` - `:brutal_kill` if the tasks must be killed directly on shutdown
-    or an integer indicating the timeout value, defaults to 5000 milliseconds;
+    * `:shutdown` - `:brutal_kill` if the tasks must be killed directly on shutdown
+      or an integer indicating the timeout value, defaults to 5000 milliseconds;
 
-  * `:max_restarts` and `:max_seconds` - as specified in `Supervisor.Spec.supervise/2`;
+    * `:max_restarts` and `:max_seconds` - as specified in `Supervisor`;
 
   """
-  @spec start_link(Supervisor.options) :: Supervisor.on_start
+  @spec start_link([option]) :: Supervisor.on_start
   def start_link(opts \\ []) do
-    import Supervisor.Spec
     {restart, opts}  = Keyword.pop(opts, :restart, :temporary)
     {shutdown, opts} = Keyword.pop(opts, :shutdown, 5000)
-    children = [worker(Task.Supervised, [], restart: restart, shutdown: shutdown)]
-    Supervisor.start_link(children, [strategy: :simple_one_for_one] ++ opts)
+    child = %{
+      id: Task.Supervised,
+      start: {Task.Supervised, :start_link, []},
+      restart: restart,
+      shutdown: shutdown
+    }
+    Supervisor.start_link([child], [strategy: :simple_one_for_one] ++ opts)
   end
 
   @doc """
@@ -52,8 +71,12 @@ defmodule Task.Supervisor do
   The `supervisor` must be a reference as defined in `Task.Supervisor`.
   The task will still be linked to the caller, see `Task.async/3` for
   more information and `async_nolink/2` for a non-linked variant.
+
+  Note this function requires the task supervisor to have `:temporary`
+  as the `:restart` option (the default), as `async/2` keeps a direct
+  reference to the task which is lost if the task is restarted.
   """
-  @spec async(Supervisor.supervisor, fun) :: Task.t
+  @spec async(Supervisor.supervisor, (() -> any)) :: Task.t
   def async(supervisor, fun) do
     async(supervisor, :erlang, :apply, [fun, []])
   end
@@ -64,6 +87,10 @@ defmodule Task.Supervisor do
   The `supervisor` must be a reference as defined in `Task.Supervisor`.
   The task will still be linked to the caller, see `Task.async/3` for
   more information and `async_nolink/2` for a non-linked variant.
+
+  Note this function requires the task supervisor to have `:temporary`
+  as the `:restart` option (the default), as `async/4` keeps a direct
+  reference to the task which is lost if the task is restarted.
   """
   @spec async(Supervisor.supervisor, module, atom, [term]) :: Task.t
   def async(supervisor, module, fun, args) do
@@ -77,11 +104,15 @@ defmodule Task.Supervisor do
   The task won't be linked to the caller, see `Task.async/3` for
   more information.
 
+  Note this function requires the task supervisor to have `:temporary`
+  as the `:restart` option (the default), as `async_nolink/2` keeps a
+  direct reference to the task which is lost if the task is restarted.
+
   ## Compatibility with OTP behaviours
 
   If you create a task using `async_nolink` inside an OTP behaviour
   like `GenServer`, you should match on the message coming from the
-  task inside your `GenServer.handle_info/2` callback.
+  task inside your `c:GenServer.handle_info/2` callback.
 
   The reply sent by the task will be in the format `{ref, result}`,
   where `ref` is the monitor reference held by the task struct
@@ -92,7 +123,7 @@ defmodule Task.Supervisor do
   with the same `ref` value that is held by the task struct. If the task
   terminates normally, the reason in the `:DOWN` message will be `:normal`.
   """
-  @spec async_nolink(Supervisor.supervisor, fun) :: Task.t
+  @spec async_nolink(Supervisor.supervisor, (() -> any)) :: Task.t
   def async_nolink(supervisor, fun) do
     async_nolink(supervisor, :erlang, :apply, [fun, []])
   end
@@ -103,6 +134,10 @@ defmodule Task.Supervisor do
   The `supervisor` must be a reference as defined in `Task.Supervisor`.
   The task won't be linked to the caller, see `Task.async/3` for
   more information.
+
+  Note this function requires the task supervisor to have `:temporary`
+  as the `:restart` option (the default), as `async_nolink/4` keeps a
+  direct reference to the task which is lost if the task is restarted.
   """
   @spec async_nolink(Supervisor.supervisor, module, atom, [term]) :: Task.t
   def async_nolink(supervisor, module, fun, args) do
@@ -110,21 +145,25 @@ defmodule Task.Supervisor do
   end
 
   @doc """
-  Returns a stream that runs the given `module`, `function` and `args`
+  Returns a stream that runs the given `module`, `function`, and `args`
   concurrently on each item in `enumerable`.
 
-  Each item will be appended to the given `args` and processed by its
+  Each item will be prepended to the given `args` and processed by its
   own task. The tasks will be spawned under the given `supervisor` and
-  linked to the current process, similar to `async/4`.
+  linked to the current process, similarly to `async/4`.
 
-  When streamed, each task will emit `{:ok, val}` upon successful
-  completion or `{:exit, val}` if the caller is trapping exits. Results
-  are emitted in the same order as the original `enumerable`.
+  When streamed, each task will emit `{:ok, value}` upon successful
+  completion or `{:exit, reason}` if the caller is trapping exits.
+  Results are emitted in the same order as the original `enumerable`.
 
   The level of concurrency can be controlled via the `:max_concurrency`
-  option and defaults to `System.schedulers_online/0`. The timeout
-  can also be given as option and defaults to 5000 and it defaults to
-  the maximum amount of time to wait without a task reply.
+  option and defaults to `System.schedulers_online/0`. A timeout
+  can also be given as an option representing the maximum amount of
+  time to wait without a task reply.
+
+  Note this function requires the task supervisor to have `:temporary`
+  as the `:restart` option (the default), as `async_stream/6` keeps a
+  direct reference to the task which is lost if the task is restarted.
 
   Finally, if you find yourself trapping exits to handle exits inside
   the async stream, consider using `async_stream_nolink/6` to start tasks
@@ -134,9 +173,18 @@ defmodule Task.Supervisor do
 
     * `:max_concurrency` - sets the maximum number of tasks to run
       at the same time. Defaults to `System.schedulers_online/0`.
-    * `:timeout` - the maximum amount of time to wait without
-      receiving a task reply (across all running tasks).
+    * `:ordered` - whether the results should be returned in the same order
+      as the input stream. This option is useful when you have large
+      streams and don't want to buffer results before they are delivered.
+      Defaults to `true`.
+    * `:timeout` - the maximum amount of time to wait (in milliseconds)
+      without receiving a task reply (across all running tasks).
       Defaults to `5000`.
+    * `:on_timeout` - what do to when a task times out. The possible
+      values are:
+      * `:exit` (default) - the process that spawned the tasks exits.
+      * `:kill_task` - the task that timed out is killed. The value
+        emitted for that task is `{:exit, :timeout}`.
 
   ## Examples
 
@@ -144,8 +192,9 @@ defmodule Task.Supervisor do
 
       stream = Task.Supervisor.async_stream(MySupervisor, collection, Mod, :expensive_fun, [])
       Enum.to_list(stream)
+
   """
-  @spec async_stream(Supervisor.supervisor, Enumerable.t, module, atom, [term], Keyword.t) ::
+  @spec async_stream(Supervisor.supervisor, Enumerable.t, module, atom, [term], keyword) ::
         Enumerable.t
   def async_stream(supervisor, enumerable, module, function, args, options \\ [])
       when is_atom(module) and is_atom(function) and is_list(args) do
@@ -153,32 +202,32 @@ defmodule Task.Supervisor do
   end
 
   @doc """
-  Returns a stream that runs the given `function` concurrently on each
-  item in `enumerable`.
+  Returns a stream that runs the given function `fun` concurrently
+  on each item in `enumerable`.
 
-  Each item will be appended to the given `args` and processed by its
-  own task. The tasks will be spawned under the given `supervisor` and
-  are linked to the current process, similar to `async/2`.
+  Each item in `enumerable` is passed as argument to the given function `fun`
+  and processed by its own task. The tasks will be spawned under the given
+  `supervisor` and linked to the current process, similarly to `async/2`.
 
-  See `async_stream/6` for discussion and examples.
+  See `async_stream/6` for discussion, options, and examples.
   """
-  @spec async_stream(Supervisor.supervisor, Enumerable.t, (term -> term), Keyword.t) ::
+  @spec async_stream(Supervisor.supervisor, Enumerable.t, (term -> term), keyword) ::
         Enumerable.t
   def async_stream(supervisor, enumerable, fun, options \\ []) when is_function(fun, 1) do
     build_stream(supervisor, :link, enumerable, fun, options)
   end
 
   @doc """
-  Returns a stream that runs the given `module`, `function` and `args`
+  Returns a stream that runs the given `module`, `function`, and `args`
   concurrently on each item in `enumerable`.
 
-  Each item will be appended to the given `args` and processed by its
-  own task. The tasks will be spawned under the given `supervisor` and
-  are not linked to the current process, similar to `async_nolink/4`.
+  Each item in `enumerable` will be prepended to the given `args` and processed
+  by its own task. The tasks will be spawned under the given `supervisor` and
+  will not be linked to the current process, similarly to `async_nolink/4`.
 
-  See `async_stream/6` for discussion and examples.
+  See `async_stream/6` for discussion, options, and examples.
   """
-  @spec async_stream_nolink(Supervisor.supervisor, Enumerable.t, module, atom, [term], Keyword.t) ::
+  @spec async_stream_nolink(Supervisor.supervisor, Enumerable.t, module, atom, [term], keyword) ::
         Enumerable.t
   def async_stream_nolink(supervisor, enumerable, module, function, args, options \\ [])
       when is_atom(module) and is_atom(function) and is_list(args) do
@@ -189,13 +238,13 @@ defmodule Task.Supervisor do
   Returns a stream that runs the given `function` concurrently on each
   item in `enumerable`.
 
-  Each item will be appended to the given `args` and processed by its
-  own task. The tasks will be spawned under the given `supervisor` and
-  are not linked to the current process, similar to `async_nolink/2`.
+  Each item in `enumerable` is passed as argument to the given function `fun`
+  and processed by its own task. The tasks will be spawned under the given
+  `supervisor` and linked to the current process, similarly to `async_nolink/2`.
 
   See `async_stream/6` for discussion and examples.
   """
-  @spec async_stream_nolink(Supervisor.supervisor, Enumerable.t, (term -> term), Keyword.t) ::
+  @spec async_stream_nolink(Supervisor.supervisor, Enumerable.t, (term -> term), keyword) ::
         Enumerable.t
   def async_stream_nolink(supervisor, enumerable, fun, options \\ []) when is_function(fun, 1) do
     build_stream(supervisor, :nolink, enumerable, fun, options)
@@ -225,7 +274,7 @@ defmodule Task.Supervisor do
   task needs to perform side-effects (like I/O) and does not need
   to report back to the caller.
   """
-  @spec start_child(Supervisor.supervisor, fun) :: {:ok, pid}
+  @spec start_child(Supervisor.supervisor, (() -> any)) :: {:ok, pid}
   def start_child(supervisor, fun) do
     start_child(supervisor, :erlang, :apply, [fun, []])
   end
@@ -237,14 +286,14 @@ defmodule Task.Supervisor do
   by the given `module`, `fun` and `args`.
   """
   @spec start_child(Supervisor.supervisor, module, atom, [term]) :: {:ok, pid}
-  def start_child(supervisor, module, fun, args) do
+  def start_child(supervisor, module, fun, args) when is_atom(fun) and is_list(args) do
     Supervisor.start_child(supervisor, [get_info(self()), {module, fun, args}])
   end
 
   defp get_info(self) do
     {node(),
      case Process.info(self, :registered_name) do
-       {:registered_name, []} -> self()
+       {:registered_name, []} -> self
        {:registered_name, name} -> name
      end}
   end

lib/elixir/lib/tuple.ex

@@ -2,7 +2,7 @@ defmodule Tuple do
   @moduledoc """
   Functions for working with tuples.
 
-  Tuples are ordered collection of elements; tuples can contain elements of any
+  Tuples are ordered collections of elements; tuples can contain elements of any
   type, and a tuple can contain elements of different types. Curly braces can be
   used to create tuples:
 

lib/elixir/lib/uri.ex

@@ -3,8 +3,8 @@ defmodule URI do
   Utilities for working with URIs.
 
   This module provides functions for working with URIs (for example, parsing
-  URIs or encoding query strings). For reference, most of the functions in this
-  module refer to [RFC 3986](https://tools.ietf.org/html/rfc3986).
+  URIs or encoding query strings). The functions in this module are implemented
+  according to [RFC 3986](https://tools.ietf.org/html/rfc3986).
   """
 
   defstruct scheme: nil, path: nil, query: nil,
@@ -42,11 +42,11 @@ defmodule URI do
   """
   @spec default_port(binary) :: nil | non_neg_integer
   def default_port(scheme) when is_binary(scheme) do
-    :elixir_config.get({:uri, scheme})
+    :elixir_config.safe_get({:uri, scheme}, nil)
   end
 
   @doc """
-  Registers the default port `port` for the given `scheme`.
+  Registers the default `port` for the given `scheme`.
 
   After this function is called, `port` will be returned by
   `default_port/1` for the given scheme `scheme`. Note that this function
@@ -157,7 +157,9 @@ defmodule URI do
       nil ->
         dict
       {{key, value}, rest} ->
-        decode_query_into_dict(rest, Dict.put(dict, key, value))
+        # Avoid warnings about Dict being deprecated
+        dict_module = Dict
+        decode_query_into_dict(rest, dict_module.put(dict, key, value))
     end
   end
 
@@ -202,7 +204,7 @@ defmodule URI do
   Checks if the character is a "reserved" character in a URI.
 
   Reserved characters are specified in
-  [RFC 3986, section 2.2](http://tools.ietf.org/html/rfc3986#section-2.2).
+  [RFC 3986, section 2.2](https://tools.ietf.org/html/rfc3986#section-2.2).
 
   ## Examples
 
@@ -219,7 +221,7 @@ defmodule URI do
   Checks if the character is a "unreserved" character in a URI.
 
   Unreserved characters are specified in
-  [RFC 3986, section 2.3](http://tools.ietf.org/html/rfc3986#section-2.3).
+  [RFC 3986, section 2.3](https://tools.ietf.org/html/rfc3986#section-2.3).
 
   ## Examples
 
@@ -253,12 +255,19 @@ defmodule URI do
   end
 
   @doc """
-  Percent-escapes the given string.
+  Percent-escapes all characters that require escaped in a string.
 
-  This function accepts a `predicate` function as an optional argument; if
-  passed, this function will be called with each character (byte) in `string` as
-  its argument and should return `true` if that character should not be escaped
-  and left as is.
+  This means reserved characters, such as `:` and `/`, and the so-
+  called unreserved characters, which have the same meaning both
+  escaped and unescaped, won't be escaped by default.
+
+  See `encode_www_form` if you are interested in escaping reserved
+  characters too.
+
+  This function also accepts a `predicate` function as an optional
+  argument. If passed, this function will be called with each byte
+  in `string` as its argument and should return `true` if the given
+  byte should be left as is.
 
   ## Examples
 
@@ -366,7 +375,7 @@ defmodule URI do
   `URI.parse/1` can be used to parse a wide range of URIs.
 
   This function uses the parsing regular expression as defined
-  in [RFC 3986, Appendix B](http://tools.ietf.org/html/rfc3986#appendix-B).
+  in [RFC 3986, Appendix B](https://tools.ietf.org/html/rfc3986#appendix-B).
 
   When a URI is given without a port, the value returned by
   `URI.default_port/1` for the URI's scheme is used for the `:port` field.
@@ -454,7 +463,7 @@ defmodule URI do
   Merges two URIs.
 
   This function merges two URIs as per
-  [RFC 3986, section 5.2](http://tools.ietf.org/html/rfc3986#section-5.2).
+  [RFC 3986, section 5.2](https://tools.ietf.org/html/rfc3986#section-5.2).
 
   ## Examples
 
@@ -472,10 +481,10 @@ defmodule URI do
     raise ArgumentError, "you must merge onto an absolute URI"
   end
   def merge(_base, %URI{scheme: rel_scheme} = rel) when rel_scheme != nil do
-    rel
+    %{rel | path: remove_dot_segments_from_path(rel.path)}
   end
   def merge(base, %URI{authority: authority} = rel) when authority != nil do
-    %{rel | scheme: base.scheme}
+    %{rel | scheme: base.scheme, path: remove_dot_segments_from_path(rel.path)}
   end
   def merge(%URI{} = base, %URI{path: rel_path} = rel) when rel_path in ["", nil] do
     %{base | query: rel.query || base.query, fragment: rel.fragment}
@@ -491,7 +500,7 @@ defmodule URI do
   defp merge_paths(nil, rel_path),
     do: merge_paths("/", rel_path)
   defp merge_paths(_, "/" <> _ = rel_path),
-    do: rel_path
+    do: remove_dot_segments_from_path(rel_path)
   defp merge_paths(base_path, rel_path) do
     [_ | base_segments] = path_to_segments(base_path)
     path_to_segments(rel_path)
@@ -500,6 +509,17 @@ defmodule URI do
     |> Enum.join("/")
   end
 
+  defp remove_dot_segments_from_path(nil) do
+    nil
+  end
+
+  defp remove_dot_segments_from_path(path) do
+    path
+    |> path_to_segments()
+    |> remove_dot_segments([])
+    |> Enum.join("/")
+  end
+
   defp remove_dot_segments([], [head, ".." | acc]),
     do: remove_dot_segments([], [head | acc])
   defp remove_dot_segments([], acc),
@@ -513,7 +533,7 @@ defmodule URI do
   defp remove_dot_segments([head | tail], acc),
     do: remove_dot_segments(tail, [head | acc])
 
-  def path_to_segments(path) do
+  defp path_to_segments(path) do
     [head | tail] = String.split(path, "/")
     reverse_and_discard_empty(tail, [head])
   end
@@ -537,7 +557,7 @@ defimpl String.Chars, for: URI do
         _     -> uri
       end
 
-    # Based on http://tools.ietf.org/html/rfc3986#section-5.3
+    # Based on https://tools.ietf.org/html/rfc3986#section-5.3
     authority = extract_authority(uri)
 
     if(scheme, do: scheme <> ":", else: "") <>
@@ -553,7 +573,7 @@ defimpl String.Chars, for: URI do
   defp extract_authority(%{host: host, userinfo: userinfo, port: port}) do
     # According to the grammar at
     # https://tools.ietf.org/html/rfc3986#appendix-A, a "host" can have a colon
-    # in it only if it's an IPv6 or "IPvFuture" address), so if there's a colon
+    # in it only if it's an IPv6 or "IPvFuture" address, so if there's a colon
     # in the host we can safely surround it with [].
     if(userinfo, do: userinfo <> "@", else: "") <>
       if(String.contains?(host, ":"), do: "[" <> host <> "]", else: host) <>

lib/elixir/lib/version.ex

@@ -24,15 +24,15 @@ defmodule Version do
 
   ## Struct
 
-  The version is represented by the Version struct and fields
-  are named according to Semver: `:major`, `:minor`, `:patch`,
-  `:pre` and `:build`.
+  The version is represented by the `Version` struct and fields
+  are named according to SemVer: `:major`, `:minor`, `:patch`,
+  `:pre`, and `:build`.
 
   ## Requirements
 
   Requirements allow you to specify which versions of a given
-  dependency you are willing to work against. It supports common
-  operators like `>=`, `<=`, `>`, `==` and friends that
+  dependency you are willing to work against. Requirements support common
+  operators like `>=`, `<=`, `>`, `==`, and friends that
   work as one would expect:
 
       # Only version 2.0.0
@@ -64,10 +64,10 @@ defmodule Version do
   `~> 2.0`       | `>= 2.0.0 and < 3.0.0`
   `~> 2.1`       | `>= 2.1.0 and < 3.0.0`
 
-  When `allow_pre: false` is set the requirement will not match a
+  When `allow_pre: false` is set, the requirement will not match a
   pre-release version unless the operand is a pre-release version.
-  The default is to allow always allow pre-releases but note that in
-  Hex `:allow_pre` is set to `false.` See the table below for examples.
+  The default is to always allow pre-releases but note that in
+  Hex `:allow_pre` is set to `false`. See the table below for examples.
 
   Requirement    | Version     | `:allow_pre` | Matches
   :------------- | :---------- | :----------- | :------
@@ -111,11 +111,27 @@ defmodule Version do
   end
 
   defmodule InvalidRequirementError do
-    defexception [:message]
+    defexception [:requirement]
+
+    def exception(requirement) when is_binary(requirement) do
+      %__MODULE__{requirement: requirement}
+    end
+
+    def message(%{requirement: requirement}) do
+      "invalid requirement: #{inspect requirement}"
+    end
   end
 
   defmodule InvalidVersionError do
-    defexception [:message]
+    defexception [:version]
+
+    def exception(version) when is_binary(version) do
+      %__MODULE__{version: version}
+    end
+
+    def message(%{version: version}) do
+      "invalid version: #{inspect version}"
+    end
   end
 
   @doc """
@@ -123,32 +139,32 @@ defmodule Version do
 
   Returns `true` if `version` satisfies `requirement`, `false` otherwise.
   Raises a `Version.InvalidRequirementError` exception if `requirement` is not
-  parsable, or `Version.InvalidVersionError` if `version` is not parsable.
+  parsable, or a `Version.InvalidVersionError` exception if `version` is not parsable.
   If given an already parsed version and requirement this function won't
   raise.
 
   ## Options
 
-    * `:allow_pre` - when `false` pre-release versions will not match
-      unless the operand is a pre-release version, see the table above
-      for examples  (default: `true`);
+    * `:allow_pre` (boolean) - when `false`, pre-release versions will not match
+      unless the operand is a pre-release version. See the table above
+      for examples. Defaults to `true`.
 
   ## Examples
 
-      iex> Version.match?("2.0.0", ">1.0.0")
+      iex> Version.match?("2.0.0", "> 1.0.0")
       true
 
-      iex> Version.match?("2.0.0", "==1.0.0")
+      iex> Version.match?("2.0.0", "== 1.0.0")
       false
 
-      iex> Version.match?("foo", "==1.0.0")
-      ** (Version.InvalidVersionError) foo
+      iex> Version.match?("foo", "== 1.0.0")
+      ** (Version.InvalidVersionError) invalid version: "foo"
 
-      iex> Version.match?("2.0.0", "== ==1.0.0")
-      ** (Version.InvalidRequirementError) == ==1.0.0
+      iex> Version.match?("2.0.0", "== == 1.0.0")
+      ** (Version.InvalidRequirementError) invalid requirement: "== == 1.0.0"
 
   """
-  @spec match?(version, requirement, Keyword.t) :: boolean
+  @spec match?(version, requirement, keyword) :: boolean
   def match?(version, requirement, opts \\ [])
 
   def match?(version, requirement, opts) when is_binary(requirement) do
@@ -156,7 +172,7 @@ defmodule Version do
       {:ok, requirement} ->
         match?(version, requirement, opts)
       :error ->
-        raise InvalidRequirementError, message: requirement
+        raise InvalidRequirementError, requirement
     end
   end
 
@@ -172,20 +188,22 @@ defmodule Version do
   end
 
   @doc """
-  Compares two versions. Returns `:gt` if the first version is greater than
-  the second one, and `:lt` for vice versa. If the two versions are equal `:eq`
-  is returned.
+  Compares two versions.
+
+  Returns `:gt` if the first version is greater than the second one, and `:lt`
+  for vice versa. If the two versions are equal, `:eq` is returned.
 
   Pre-releases are strictly less than their corresponding release versions.
 
   Patch segments are compared lexicographically if they are alphanumeric, and
   numerically otherwise.
 
-  Build segments are ignored, if two versions differ only in their build segment
+  Build segments are ignored: if two versions differ only in their build segment
   they are considered to be equal.
 
-  Raises a `Version.InvalidVersionError` exception if any of the two are not
-  parsable. If given an already parsed version this function won't raise.
+  Raises a `Version.InvalidVersionError` exception if any of the two given
+  versions are not parsable. If given an already parsed version this function
+  won't raise.
 
   ## Examples
 
@@ -202,7 +220,7 @@ defmodule Version do
       :eq
 
       iex> Version.compare("invalid", "2.0.1")
-      ** (Version.InvalidVersionError) invalid
+      ** (Version.InvalidVersionError) invalid version: "invalid"
 
   """
   @spec compare(version, version) :: :gt | :eq | :lt
@@ -223,7 +241,7 @@ defmodule Version do
   end
 
   @doc """
-  Parses a version string into a `Version`.
+  Parses a version string into a `Version` struct.
 
   ## Examples
 
@@ -259,24 +277,24 @@ defmodule Version do
       #Version<2.0.1-alpha1>
 
       iex> Version.parse!("2.0-alpha1")
-      ** (Version.InvalidVersionError) 2.0-alpha1
+      ** (Version.InvalidVersionError) invalid version: "2.0-alpha1"
 
   """
   @spec parse!(String.t) :: t | no_return
   def parse!(string) when is_binary(string) do
     case parse(string) do
       {:ok, version} -> version
-      :error -> raise InvalidVersionError, message: string
+      :error -> raise InvalidVersionError, string
     end
   end
 
   @doc """
-  Parses a version requirement string into a `Version.Requirement`.
+  Parses a version requirement string into a `Version.Requirement` struct.
 
   ## Examples
 
-      iex> {:ok, req} = Version.parse_requirement("== 2.0.1")
-      iex> req
+      iex> {:ok, requirement} = Version.parse_requirement("== 2.0.1")
+      iex> requirement
       #Version.Requirement<== 2.0.1>
 
       iex> Version.parse_requirement("== == 2.0.1")
@@ -297,7 +315,7 @@ defmodule Version do
   Compiles a requirement to its internal representation with
   `:ets.match_spec_compile/1` for faster matching.
 
-  The internal representation is opaque and can not be converted to external
+  The internal representation is opaque and cannot be converted to external
   term format and then back again without losing its properties (meaning it
   can not be sent to a process on another node and still remain a valid
   compiled match_spec, nor can it be stored on disk).
@@ -316,75 +334,55 @@ defmodule Version do
       {:ok, {major, minor, patch, pre, _build_parts}} ->
         {major, minor, patch, pre, allow_pre?}
       :error ->
-        raise InvalidVersionError, message: string
-    end
-  end
-
-  defmodule Parser.DSL do
-    @moduledoc false
-
-    defmacro deflexer(match, do: body) when is_binary(match) do
-      quote do
-        def lexer(unquote(match) <> rest, acc) do
-          lexer(rest, [unquote(body) | acc])
-        end
-      end
-    end
-
-    defmacro deflexer(acc, do: body) do
-      quote do
-        def lexer("", unquote(acc)) do
-          unquote(body)
-        end
-      end
-    end
-
-    defmacro deflexer(char, acc, do: body) do
-      quote do
-        def lexer(<<unquote(char)::utf8, rest::binary>>, unquote(acc)) do
-          unquote(char) = <<unquote(char)::utf8>>
-
-          lexer(rest, unquote(body))
-        end
-      end
+        raise InvalidVersionError, string
     end
   end
 
   defmodule Parser do
     @moduledoc false
-    import Parser.DSL
 
-    deflexer ">=",    do: :>=
-    deflexer "<=",    do: :<=
-    deflexer "~>",    do: :~>
-    deflexer ">",     do: :>
-    deflexer "<",     do: :<
-    deflexer "==",    do: :==
-    deflexer "!=",    do: :!=
-    deflexer "!",     do: :!=
-    deflexer " or ",  do: :||
-    deflexer " and ", do: :&&
-    deflexer " ",     do: :' '
-
-    deflexer x, [] do
-      [x, :'==']
-    end
-
-    deflexer x, [h | acc] do
-      cond do
-        is_binary h ->
-          [h <> x | acc]
-
-        h in [:||, :&&] ->
-          [x, :==, h | acc]
-
-        true ->
-          [x, h | acc]
+    operators = [
+      {">=", :>=},
+      {"<=", :<=},
+      {"~>", :~>},
+      {">", :>},
+      {"<", :<},
+      {"==", :==},
+      {"!=", :!=},
+      {"!", :!=},
+      {" or ", :||},
+      {" and ", :&&},
+    ]
+    for {string_op, atom_op} <- operators do
+      def lexer(unquote(string_op) <> rest, acc) do
+        lexer(rest, [unquote(atom_op) | acc])
       end
     end
 
-    deflexer acc do
-      Enum.filter(Enum.reverse(acc), &(&1 != :' '))
+    def lexer(" " <> rest, acc) do
+      lexer(rest, acc)
+    end
+
+    def lexer(<<char::utf8, rest::binary>>, []) do
+      lexer(rest, [<<char::utf8>>, :==])
+    end
+
+    def lexer(<<char::utf8, body::binary>>, [head | acc]) do
+      acc =
+        case head do
+          head when is_binary(head) ->
+            [<<head::binary, char::utf8>> | acc]
+          head when head in [:||, :&&] ->
+            [<<char::utf8>>, :==, head | acc]
+          _other ->
+            [<<char::utf8>>, head | acc]
+        end
+
+      lexer(body, acc)
+    end
+
+    def lexer("", acc) do
+      Enum.reverse(acc)
     end
 
     @spec parse_requirement(String.t) :: {:ok, term} | :error
@@ -397,9 +395,10 @@ defmodule Version do
     def parse_version(string, approximate? \\ false) when is_binary(string) do
       destructure [version_with_pre, build], String.split(string, "+", parts: 2)
       destructure [version, pre], String.split(version_with_pre, "-", parts: 2)
-      destructure [major, minor, patch], String.split(version, ".")
+      destructure [major, minor, patch, next], String.split(version, ".")
 
-      with {:ok, major} <- require_digits(major),
+      with nil <- next,
+           {:ok, major} <- require_digits(major),
            {:ok, minor} <- require_digits(minor),
            {:ok, patch} <- maybe_patch(patch, approximate?),
            {:ok, pre_parts} <- optional_dot_separated(pre),

lib/elixir/mix.exs

@@ -2,8 +2,10 @@ defmodule Elixir.Mixfile do
   use Mix.Project
 
   def project do
-    [app: :elixir,
-     version: System.version,
-     build_per_environment: false]
+    [
+      app: :elixir,
+      version: System.version,
+      build_per_environment: false
+    ]
   end
 end

lib/elixir/pages/Deprecations.md

@@ -0,0 +1,72 @@
+# Deprecations
+
+## Policy
+
+Elixir deprecations happen in 3 steps:
+
+  1. The feature is soft-deprecated. It means both CHANGELOG and documentation must list the feature as deprecated but no warning is effectively emitted by running the code. There is no requirement to soft-deprecate a feature.
+
+  2. The feature is effectively deprecated by emitting warnings on usage. In order to deprecate a feature, the proposed alternative MUST exist for AT LEAST two versions. For example, `Enum.uniq/2` was soft-deprecated in favor of `Enum.uniq_by/2` in Elixir v1.1. This means a deprecation warning may only be emitted by Elixir v1.3 or later.
+
+  3. The feature is removed. This can only happen on major releases. This means deprecated features in Elixir v1.x shall only be removed by Elixir v2.x.
+
+
+## Table of deprecations
+
+Deprecated feature                               | Deprecated in | Replaced by (available since)
+:----------------------------------------------- | :------------ | :----------------------------
+`Atom.to_char_list/1`                            | [v1.5]        | `Atom.to_charlist/1` (v1.3)
+`Enum.filter_map/3`                              | [v1.5]        | `Enum.filter/2` + `Enum.map/2` or for comprehensions (v1.0)
+`Float.to_char_list/1`                           | [v1.5]        | `Float.to_charlist/1` (v1.3)
+`GenEvent` module                                | [v1.5]        | `Supervisor` and `GenServer` (v1.0);<br/>[`GenStage`](https://hex.pm/packages/gen_stage) (v1.3);<br/>[`:gen_event`](http://www.erlang.org/doc/man/gen_event.html) (OTP 17)
+`Integer.to_char_list/1` and `Integer.to_char_list/2` | [v1.5]   | `Integer.to_charlist/1` and `Integer.to_charlist/2` (v1.3)
+`Kernel.to_char_list/1`                          | [v1.5]        | `Kernel.to_charlist/1` (v1.3)
+`List.Chars.to_char_list/1`                      | [v1.5]        | `List.Chars.to_charlist/1` (v1.3)
+`Stream.filter_map/3`                            | [v1.5]        | `Stream.filter/2` + `Stream.map/2` (v1.0)
+`String.ljust/3` and `String.rjust/3`            | [v1.5]        | `String.pad_leading/3` and `String.pad_trailing/3` with a binary padding (v1.3)
+`String.strip/1` and `String.strip/2`            | [v1.5]        | `String.trim/1` and `String.trim/2` (v1.3)
+`String.lstrip/1` and `String.rstrip/1`          | [v1.5]        | `String.trim_leading/1` and `String.trim_trailing/1` (v1.3)
+`String.lstrip/2` and `String.rstrip/2`          | [v1.5]        | `String.trim_leading/2` and `String.trim_trailing/2` with a binary as second argument (v1.3)
+`String.to_char_list/1`                          | [v1.5]        | `String.to_charlist/1` (v1.3)
+`()` to mean `nil`                               | [v1.5]        | `nil` (v1.0)
+`:as_char_lists` value in `t:Inspect.Opts.t/0` type | [v1.5]     | `:as_charlists` (v1.3)
+`:char_lists` key in `t:Inspect.Opts.t/0` type   | [v1.5]        | `:charlists` (v1.3)
+`char_list/0` type                               | [v1.5]        | `charlist/0` type (v1.3)
+`@compile {:parse_transform, _}` in `Module`     | [v1.5]        | *None*
+EEx: `<%=` in middle and end expressions         | [v1.5]        | Use `<%` (= is allowed only on start expressions) (v1.0)
+`Access.key/1`                                   | [v1.4]        | `Access.key/2` (v1.3)
+`Behaviour` module                               | [v1.4]        | `@callback` (v1.0)
+`Enum.uniq/2`                                    | [v1.4]        | `Enum.uniq_by/2` (v1.2)
+`Float.to_char_list/2`                           | [v1.4]        | `:erlang.float_to_list/2` (OTP 17)
+`Float.to_string/2`                              | [v1.4]        | `:erlang.float_to_binary/2` (OTP 17)
+`HashDict` module                                | [v1.4]        | `Map` (v1.2)
+`HashSet` module                                 | [v1.4]        | `MapSet` (v1.1)
+`Set` module                                     | [v1.4]        | `MapSet` (v1.1)
+`Stream.uniq/2`                                  | [v1.4]        | `Stream.uniq_by/2` (v1.2)
+`IEx.Helpers.import_file/2`                      | [v1.4]        | [`IEx.Helpers.import_file_if_available/1`](https://hexdocs.pm/iex/IEx.Helpers.html#import_file_if_available/1) (v1.3)
+`Mix.Utils.camelize/1`                           | [v1.4]        | `Macro.camelize/1` (v1.2)
+`Mix.Utils.underscore/1`                         | [v1.4]        | `Macro.underscore/1` (v1.2)
+Variable used as function call                   | [v1.4]        | Use parentheses (v1.0)
+Anonymous functions with no expression after `->` | [v1.4]       | Use an expression or explicitly return `nil` (v1.0)
+`Dict` module                                    | [v1.3]        | `Keyword` (v1.0);<br/>`Map` (v1.2)
+`Keyword.size/1`                                 | [v1.3]        | `Kernel.length/1` (v1.0)
+`Map.size/1`                                     | [v1.3]        | `Kernel.map_size/1` (v1.0)
+`Set` behaviour                                  | [v1.3]        | `MapSet` data structure (v1.1)
+`String.valid_character?/1`                      | [v1.3]        | `String.valid?/1` (v1.0)
+`Task.find/2`                                    | [v1.3]        | Use direct message matching (v1.0)
+`:append_first` option in `Kernel.defdelegate/2` | [v1.3]        | Define the function explicitly (v1.0)
+`/r` option in `Regex`                           | [v1.3]        | `/U` (v1.1)
+`\x{X*}` inside strings/sigils/charlists         | [v1.3]        | `\uXXXX` or `\u{X*}` (v1.1)
+Map or dictionary as second argument in `Enum.group_by/3` | [v1.3] | Use `Enum.reduce/3` (v1.0)
+Non-map as second argument in `URI.decode_query/2` | [v1.3]      | Use a map (v1.0)
+`Dict` behaviour                                 | [v1.2]        | `MapSet` data structure (v1.1)
+`Access` protocol                                | [v1.1]        | `Access` behaviour (v1.1)
+`as: true \| false` in `alias/2` and `require/2` | [v1.1]     | *None*
+`?\xHEX`                                         | [v1.1]        | `0xHEX` (v1.0)
+Empty string in `String.starts_with?/2`, `String.ends_with?/2`, `String.contains?/2`.<br/>*__NOTE__: Feature made back available in v1.3* | [v1.1] to [v1.2] | Explicitly check for `""` beforehand (v1.0)
+
+[v1.1]: https://github.com/elixir-lang/elixir/blob/v1.1/CHANGELOG.md#4-deprecations
+[v1.2]: https://github.com/elixir-lang/elixir/blob/v1.2/CHANGELOG.md#changelog-for-elixir-v12
+[v1.3]: https://github.com/elixir-lang/elixir/blob/v1.3/CHANGELOG.md#4-deprecations
+[v1.4]: https://github.com/elixir-lang/elixir/blob/v1.4/CHANGELOG.md#4-deprecations
+[v1.5]: https://github.com/elixir-lang/elixir/blob/v1.5/CHANGELOG.md#4-deprecations

lib/elixir/pages/Guards.md

@@ -11,7 +11,7 @@ For reference, the following is a comprehensive list of all expressions allowed
   * arithmetic binary operators (`+`, `-`, `*`, `/`)
   * arithmetic unary operators (`+`, `-`)
   * binary concatenation operator (`<>`)
-  * `in` operator (as long as the right-hand side is a list or a range)
+  * `in` and `not in` operators (as long as the right-hand side is a list or a range)
   * the following "type-check" functions (all documented in the `Kernel` module):
     * `is_atom/1`
     * `is_binary/1`
@@ -55,6 +55,8 @@ For reference, the following is a comprehensive list of all expressions allowed
     * `bsr/1` or the `>>>` operator
     * `bxor/2` or the `^^^` operator
 
+Macros constructed out of any combination of the above guards are also valid guards - for example, `Integer.is_even/1`. See the section "Defining custom guard expressions" below.
+
 ## Why guards
 
 Let's see an example of a guard used in a function clause:
@@ -102,14 +104,14 @@ Other constructs are `for`, `with`, `try`/`rescue`/`catch`/`else`/, and the `mat
 
 ## Failing guards
 
-Errors in guards do not result in a runtime error, but in the erroring guard fail. For example, the `length/1` function only works with lists, and if we use it on anything else it fails:
+Errors in guards do not result in runtime errors, but in guards failing. For example, the `length/1` function only works with lists. If we use it with anything else, a runtime error is raised:
 
 ```elixir
 iex> length("hello")
 ** (ArgumentError) argument error
 ```
 
-However, when used in guards, it simply makes the corresponding clause fail (i.e., not match):
+However, when used in guards, the corresponding clause simply fails to match:
 
 ```elixir
 iex> case "hello" do
@@ -121,7 +123,7 @@ iex> case "hello" do
 :length_failed
 ```
 
-In many cases, we can take advantage of this: in the code above, for example, we can use `length/1` to both check that the given thing is a list *and* check some properties of its length (instead of using `is_list(something) and length(something) > 0`).
+In many cases, we can take advantage of this. In the code above, we used `length/1` to both check that the given thing is a list *and* check some properties of its length (instead of using `is_list(something) and length(something) > 0`).
 
 ## Expressions in guard clauses
 
@@ -141,7 +143,7 @@ def my_function(number) when is_integer(number) and rem(number, 2) == 0 do
 end
 ```
 
-This would be repetitive to write everytime we need this check, so, as mentioned at the beginning of this section, we can abstract this away using a macro. Remember that defining a function that performs this check wouldn't work because we can't use custom functions in guards. Our macro would look like this:
+This would be repetitive to write every time we need this check, so, as mentioned at the beginning of this section, we can abstract this away using a macro. Remember that defining a function that performs this check wouldn't work because we can't use custom functions in guards. Our macro would look like this:
 
 ```elixir
 defmodule MyInteger do
@@ -189,4 +191,32 @@ def foo(_other) do
 end
 ```
 
-The two forms are exactly the same semantically but there are cases where the latter one may be more aesthetically pleasing.
+For most cases, the two forms are exactly the same. However, there exists a subtle difference in the case of failing guards, as discussed in the section above.
+In case of a boolean expression guard, a failed element means the whole guard fails. In case of multiple guards it means the next one will be evaluated.
+The difference can be highlighted with an example:
+
+```elixir
+def multiguard(value)
+    when map_size(value) < 1
+    when tuple_size(value) < 1 do
+  :guard_passed
+end
+def multiguard(_value) do
+  :guard_failed
+end
+
+def boolean(value) when map_size(value) < 1 or tuple_size(value) < 1 do
+  :guard_passed
+end
+def boolean(value) do
+  :guard_failed
+end
+
+multiguard(%{}) #=> :guard_passed
+multiguard({})  #=> :guard_passed
+
+boolean(%{}) #=> :guard_passed
+boolean({})  #=> :guard_failed
+```
+
+For cases where guards do not rely on the failing guard behavior the two forms are exactly the same semantically but there are cases where multiple guard clauses may be more aesthetically pleasing.

lib/elixir/pages/Naming Conventions.md

@@ -40,7 +40,7 @@ Due to this property, Elixir relies on functions starting with underscore to att
     iex> String.__info__(:functions)
     [at: 2, capitalize: 1, chunk: 2, ...]
 
-Elixir also includes 4 special variables that follow the double underscore format. These forms retrieve compile-time information about the current environment: `__MODULE__/0`, `__DIR__/0`, `__ENV__/0` and `__CALLER__/0`.
+Elixir also includes four special forms that follow the double underscore format. These forms retrieve compile-time information about the current environment: `__MODULE__/0`, `__DIR__/0`, `__ENV__/0` and `__CALLER__/0`.
 
 ## Trailing bang (foo!)