Missing deprecation notices in CHANGELOG v1.4 (#7614)

* :message_queue_len and :current_stacktrace added
* :messages and :dictionary not always present

Signed-off-by: José Valim <jose.valim@plataformatec.com.br>

.travis.yml

@@ -2,6 +2,11 @@ language: erlang
 
 otp_release:
   - 18.0
+  - 18.1
+  - 18.2
+  - 18.3
+  - 19.0
+  - 19.1
 
 sudo: false
 

CHANGELOG.md

@@ -1,404 +1,364 @@
-# Changelog for Elixir v1.3
+# Changelog for Elixir v1.4
 
-Elixir v1.3 brings many improvements to the language, the compiler and its tooling, specially Mix (Elixir's build tool) and ExUnit (Elixir's test framework).
+Elixir v1.4 brings new features, enhancements and bug fixes into Elixir. The most notable changes are the addition of the `Registry` module and the `Task.async_stream/3` and `Task.async_stream/5` which aid developers in writing concurrent software. Those two features and a couple other improvements are described in detail below followed by the complete list of changes.
 
-## Language improvements
+## Registry
 
-The language has been improved semantically and includes new types and APIs. Let's see the three major features.
+The registry is a local, decentralized and scalable key-value process storage:
 
-### Deprecation of imperative assignment
+  * Local because keys and values are only accessible to the current node (opposite to distributed)
+  * Decentralized because there is no single entity responsible for managing the registry
+  * Scalable because performance scales linearly with the addition of more cores upon partitioning
 
-Elixir will now warn if constructs like `if`, `case` and friends assign to a variable that is accessed in an outer scope. As an example, imagine a function called `format` that receives a message and some options and it must return a path alongside the message:
+A registry is chosen upon start to have unique or duplicate keys. Every key-value pair is associated to the process registering the key. Keys are automatically removed once the owner process terminates.
 
-```elixir
-def format(message, opts) do
-  path =
-    if (file = opts[:file]) && (line = opts[:line]) do
-      relative = Path.relative_to_cwd(file)
-      message  = Exception.format_file_line(relative, line) <> " " <> message
-      relative
+    iex> Registry.start_link(:unique, MyRegistry)
+    iex> {:ok, _} = Registry.register(MyRegistry, "hello", 1)
+    iex> Registry.lookup(MyRegistry, "hello")
+    [{self(), 1}]
+
+With the registry, developers can provide dynamic process names, module-function dispatch or even a local pubsub system. See the `Registry` documentation for more information.
+
+## Syntax coloring
+
+Elixir v1.4 introduces the ability to syntax color inspected data structures:
+
+    iex> IO.puts inspect([hello: 1, world: "!"], syntax_colors: [atom: :cyan])
+    [hello: 1, world: "!"]
+
+Coloring is done with ANSI colors as specified in the `IO.ANSI` module.
+
+IEx automatically relies on this feature to provide syntax coloring for evaluated shell results. This behaviour can be configured via the `:syntax_colors` coloring option:
+
+    IEx.configure [colors: [syntax_colors: [atom: :cyan, string: :green]]]
+
+To disable coloring altogether, pass an empty list to `:syntax_colors`.
+
+## Calendar
+
+Elixir v1.3 introduced new calendar types. This release continues evolving the Calendar APIs by adding functions for comparing, adding and calculating the difference between types, retrieve the `day_of_week/1`, check if the current date is a `leap_year?/1` and more.
+
+## Task.async_stream
+
+When there is a need to traverse a collection of items concurrently, Elixir developers often resort to tasks:
+
+    collection
+    |> Enum.map(&Task.async(SomeMod, :function, [&1]))
+    |> Enum.map(&Task.await/1)
+
+While the snippet above works fine in many occasions, for large collections it will spawn and run concurrently as many tasks as there are items in the collection.
+
+`Task.async_stream/3` and `Task.async_stream/5` allows developers to process collections concurrently while controlling the maximum amount of concurrent tasks:
+
+    collection
+    |> Task.async_stream(SomeMod, :function, [], max_concurrency: System.schedulers_online)
+
+The `Task.async_stream` functions are also lazy, allowing developers to partially consume the stream until a condition is reached. Furthermore, `Task.Supervisor.async_stream/4` and `Task.Supervisor.async_stream/6` can be used to ensure the concurrent tasks are spawned under a given supervisor.
+
+## Application inference
+
+Mix v1.4 now automatically infers the list of applications that are required on runtime from your dependencies list.
+
+In previous Mix versions, most of your dependencies had to be added both to your dependencies list and applications list. Here is how a `mix.exs` would look like:
+
+    def application do
+      [applications: [:logger, :plug, :postgrex]]
     end
 
-  {path, message}
-end
-```
-
-The `if` block above is implicitly changing the value in `message`. Now imagine we want to move the `if` block to its own function to clean up the implementation:
-
-```elixir
-def format(message, opts) do
-  path = with_file_and_line(message, opts)
-  {path, message}
-end
-
-defp with_file_and_line(message, opts) do
-  if (file = opts[:file]) && (line = opts[:line]) do
-    relative = Path.relative_to_cwd(file)
-    message  = Exception.format_file_line(relative, line) <> " " <> message
-    relative
-  end
-end
-```
-
-The refactored version is broken because the `if` block was actually returning two values, the relative path *and* the new message. Elixir v1.3 will warn on such cases, forcing both variables to be explicitly returned from `if`, `case` and other constructs. Furthermore, this change gives us the opportunity to unify the language scoping rules in future releases.
-
-### Calendar types and sigils
-
-Elixir v1.3 introduces the `Calendar` module as well as 4 new calendar types:
-
-  * `Date` - used to store dates (year, month, day) in a given calendar
-  * `Time` - used to store time (hour, minute, second, microseconds)
-  * `NaiveDateTime` - used to store datetimes without a timezone (year, month, day, hour, minute, second, microseconds) in a given calendar. It is called naïve because without a timezone, the datetime may not actually exist. For example, when there are daylight savings changes, a whole hour may not exist (when the clock moves forward) or a particular instant may happen twice (when the clock moves backwards)
-  * `DateTime` - used to store datetimes with timezone (year, month, day, hour, minute, second, microsecond and time zone, with abbreviation, UTC and standard offset)
-
-The current Calendar modules and its types is to provide a base for interoperatibility in the ecosystem instead of full-featured datetime API. This release includes basic functionality for building new types and converting them from and back strings.
-
-Elixir v1.3 also introduces 3 new sigils related to the types above:
-
-  * `~D[2016-05-29]` - builds a new date
-  * `~T[08:00:00]` and `~T[08:00:00.285]` - builds a new time (with different precisions)
-  * `~N[2016-05-29 08:00:00]` - builds a naive date time
-
-### Access selectors
-
-This release introduces new accessors to make it simpler for developers to traverse nested data structures, traversing and updating data in different ways.  For instance, given a user with a list of languages, here is how to deeply traverse the map and convert all language names to uppercase:
-
-```iex
-iex> user = %{name: "john",
-...>          languages: [%{name: "elixir", type: :functional},
-...>                      %{name: "c", type: :procedural}]}
-iex> update_in user, [:languages, Access.all(), :name], &String.upcase/1
-%{name: "john",
-  languages: [%{name: "ELIXIR", type: :functional},
-              %{name: "C", type: :procedural}]}
-```
-
-You can see the new accessors in the `Access` module.
-
-## Mix
-
-Mix includes new tasks to improve your everyday workflow. Some of those tasks relies on many compiler improvements to know more about your code, providing static analysis to find possible bugs in your code and faster compilation cycles.
-
-### Compiling n files
-
-Mix no longer announces every file it compiles. Instead it outputs how many files there is to compile per compilers. Here is the output for a project like [`gettext`](https://github.com/elixir-lang/gettext):
-
-```
-Compiling 1 file (.yrl)
-Compiling 1 file (.erl)
-Compiling 19 files (.ex)
-Generated gettext app
-```
-
-In case a file is taking too long to compile, Mix will announce such, for example:
-
-```
-Compiling lib/gettext.ex (it's taking more than 5s)
-```
-
-The goal of these changes is to put an increased focus on the "warnings" emitted by the compiler.
-
-In any case, the previous behaviour can be brought back with the `--verbose` flag and the compilation threshold for files that are taking long can be set via the `--long-compilation-threshold` option.
-
-### mix xref
-
-Speaking about warnings, Mix v1.3 includes a new task called `xref` that performs cross reference checks in your code. One of such checks is the ability to find calls to modules and functions that do not exist. For example, if in your library code you call `ThisModuleDoesNotExist.foo(1, 2, 3)`, `mix xref --unreachable` will be able to find such code and let you know about it.
-
-Since such checks can discover possible bugs in your codebase, a new compiler called `xref` has been added to `Mix.compilers/0`, so they run by default every time you compile your code.
-
-We have included other modes in `xref`, such as `mix xref --callers Foo`, to find all places in your code that a function from the module `Foo` is called. We hope other tools and text editors can leverage such features to provide useful functionality for their users.
-
-### Better dependency tracking
-
-Besides `xref`, Elixir v1.3 provides better module tracking generally. For example, in previous versions, if you changed a `:path` dependency, Elixir would always fully recompile the current project. In this release, we have improved the tracking algorithms such that, if you change a `:path` dependency, only the files that depend on such dependency are recompiled.
-
-Such improvements do not only make compilation faster but they also make working with umbrella applications much more productive. Previously, changing a sibling application triggered a full project recompilation, now Elixir can track between sibling applications and recompile only what is needed.
-
-### mix app.tree and deps.tree
-
-Mix also includes both `mix app.tree` and `mix deps.tree`. The first will list all applications your current project needs to start in order to boot (i.e. the ones listed in `application/0` in your `mix.exs`) while the second will lists all of your dependencies and so on recursively.
-
-Here is a quick example from [Plug](https://github.com/elixir-lang/plug):
-
-```elixir
-$ mix app.tree
-plug
-├── elixir
-├── crypto
-├── logger
-│   └── elixir
-└── mime
-    └── elixir
-```
-
-### mix escript.install
-
-Mix also includes `mix escript.install` and `mix escript.uninstall` tasks for managing escripts. The tasks was designed in a way to mimic the existing `mix archive` functionality except that:
-
-  * Archives must be used sparingly because every new archive installed affects Mix performance, as every new archive is loaded when Mix boots. Escripts solve this by being managed apart from your Elixir/Mix installed
-  * Archives depends on the current Elixir version. Therefore, updating your Elixir version may break an archive. Fortunately, escripts include Elixir inside themselves, and therefore do not depend on your Elixir system version
-
-Escripts will be installed at `~/.mix/escripts` which must be added to your [`PATH` environment variable](https://en.wikipedia.org/wiki/PATH_(variable)).
-
-### Option parser integration
-
-Elixir v1.3 includes improvements to the option parser, including `OptionParser.parse!/2` and `OptionParser.parse_head!/2` functions that will raise in case of invalid or unknown switches. Mix builds on top of this functionality to provide automatic error reporting solving a common complaint where invalid options were not reported by Mix tasks.
-
-For example, invoking `mix test --unknown` in earlier Elixir versions would silently discard the `--unknown` option. Now `mix test` correctly reports such errors:
-
-```
-$ mix test --unknown
-** (Mix) Could not invoke task "test": 1 error found!
---unknown : Unknown option
-```
-
-Note not all tasks have been updated to use strict option parsing. Some tasks, like `mix compile`, are actually a front-end to many other tasks, and as such, it cannot effectively assert which options are valid.
-
-## ExUnit
-
-ExUnit packs many improvements on the tooling side, better integration with external tools, as well as mechanisms to improve the readability of your tests.
-
-### mix test --stale
-
-ExUnit builds on top of `mix xref` to provide the `mix test --stale` functionality. When the `--stale` flag is given, `mix` will only run the tests that may have changed since the last time you ran `mix test --stale`. For example:
-
-  * If you saved a test file on disk, Mix will run that file and ignore the ones that have not changed
-  * If you changed a library file, for example, `lib/foo.ex` that defines `Foo`, any test that invokes a function in `Foo` directly or indirectly will also run
-  * If you modify your `mix.exs` or your `test/test_helper.exs`, Mix will run the whole test suite
-
-This feature provides a great workflow for developers, allowing them to effortlessly focus on parts of the codebase when developing new features.
-
-### Diffing
-
-ExUnit will now include diff-ing output every time a developer asserts `assert left == right` in their tests. For example, the assertion:
-
-```elixir
-assert "fox jumps over the lazy dog" ==
-       "brown fox jumps over the dog"
-```
-
-will fail with
-
-```elixir
-  1) test compare (Difference)
-     lib/ex_unit/examples/difference.exs:10
-     Assertion with == failed
-     lhs:  "fox jumps over the lazy dog"
-     rhs:  "brown fox jumps over the dog"
-     stacktrace:
-       lib/ex_unit/examples/difference.exs:11: (test)
-```
-
-in a way that "lazy" in "lhs" will be shown in red to denote it has been removed from "rhs" while "brown" in "rhs" will be shown in green to denote it has been added to the "rhs".
-
-When working with large or nested data structures, the diffing algorithm makes it fast and convenient to spot the actual differences in the asserted values.
-
-### Test types
-
-ExUnit v1.3 includes the ability to register different test types. This means libraries like QuickCheck can now provide functionality such as:
-
-```elixir
-defmodule StringTest do
-  use ExUnit.Case, async: true
-  use PropertyTestingLibrary
-
-  property "starts_with?" do
-    forall({s1, s2} <- {utf8, utf8}) do
-      String.starts_with?(s1 <> s2, s1)
-    end
-  end
-end
-```
-
-At the end of the run, ExUnit will also report it as a property, including both the amount of tests and properties:
-
-```
-1 property, 10 tests, 0 failures
-```
-
-### Named setups and describes
-
-Finally, ExUnit v1.3 includes the ability to organize tests together in describe blocks:
-
-```elixir
-defmodule StringTest do
-  use ExUnit.Case, async: true
-
-  describe "String.capitalize/2" do
-    test "uppercases the first grapheme" do
-      assert "T" <> _ = String.capitalize("test")
+    def deps do
+      [{:plug, "~> 1.2"},
+       {:postgrex, "~> 1.0"}]
     end
 
-    test "lowercases the remaining graphemes" do
-      assert "Test" = String.capitalize("TEST")
+This was error prone as many developers would not list their dependencies in their applications list.
+
+Mix v1.4 now automatically infers your applications list as long as you leave the `:applications` key empty. The `mix.exs` above can be rewritten to:
+
+    def application do
+      [extra_applications: [:logger]]
     end
-  end
-end
-```
 
-Every test inside a describe block will be tagged with the describe block name. This allows developers to run tests that belong to particular blocks, be them in the same file or across many files:
+    def deps do
+      [{:plug, "~> 1.2"},
+       {:postgrex, "~> 1.0"}]
+    end
 
-```
-$ mix test --only describe:"String.capitalize/2"
-```
+With the above, Mix will automatically build your application list based on your dependencies. Applications that are part of Erlang or Elixir that are required at runtime, such as `:logger`, must be added to the `:extra_applications` list. All extra applications will be included in the application list.
 
-Note describe blocks cannot be nested. Instead of relying on hierarchy for composition, we want developers to build on top of named setups. For example:
+Finally, if there is a dependency you don't want to include in the application runtime list, you can do so by specifying the `runtime: false` option:
 
-```elixir
-defmodule UserManagementTest do
-  use ExUnit.Case, async: true
+    {:distillery, "> 0.0.0", runtime: false}
 
-  describe "when user is logged in and is an admin" do
-    setup [:log_user_in, :set_type_to_admin]
+We hope this feature provides a more streamlined workflow for developers who are building releases for their Elixir projects.
 
-    test ...
-  end
+## Mix install from SCM
 
-  describe "when user is logged in and is a manager" do
-    setup [:log_user_in, :set_type_to_manager]
+Mix v1.4 can now install escripts and archives from both Git and Hex, providing you with even more options for distributing Elixir code.
 
-    test ...
-  end
+This makes it possible to distribute CLI applications written in Elixir by publishing a package which builds an escript to Hex. [`ex_doc`](https://hex.pm/packages/ex_doc) has been updated to serve as an example of how to use this new functionality.
 
-  defp log_user_in(context) do
-    # ...
-  end
-end
-```
+Simply running:
 
-By restricting hierarchies in favor of named setups, it is straight-forward for the developer to glance at each describe block and know exactly the setup steps involved.
+    mix escript.install hex ex_doc
 
-## v1.3.0-rc.1 (2016-08-06)
+will fetch `ex_doc` and its dependencies, build them, and then install `ex_doc` to `~/.mix/escripts` (by default). After adding `~/.mix/escripts` to your `PATH`, running `ex_doc` is as simple as:
+
+    ex_doc
+
+You can now also install archives from Hex in this way. Since they are fetched and built on the user's machine, they do not have the same limitations as pre-built archives. However, keep in mind archives run alongside every Mix project, which may lead to conflicts. For this reason, escripts is the preferred format.
+
+It is also possible to install escripts and archives by providing a Git/GitHub repo. See `mix help escript.install` and `mix help archive.install` for more details.
+
+## v1.4.5 (2017-06-22)
+
+This version includes changes that make Elixir fully compatible with Erlang/OTP 20.
 
 ### 1. Enhancements
 
-#### EEx
-
-  * [EEx.Engine] Support an `init/1` function in engines that will return the initial buffer (defaults to an empty string)
-
-#### Elixir
-
-  * [Access] Add support for `Access.all/0`, `Access.elem/1`, `Access.key/2` and `Access.key!/1` for traversing nested data structures
-  * [Calendar] Add `Calendar` and `Date`, `Time`, `NaiveDateTime` and `DateTime` types
-  * [CLI] Add `--logger-otp-reports BOOL` and `--logger-sasl-reports BOOL` switches
-  * [Compiler] Emit a summary of compilation errors when modules are missing
-  * [Enum] Add `Enum.group_by/3` that allows developers to map on the value being grouped
-  * [Enum] Add `Enum.drop_every/2` that drops every `nth`, including the first one
-  * [Exception] Suggest possible functions on `UndefinedFunctionError` for existing modules
-  * [Exception] Warn if unknown fields are given to `raise/2`
-  * [File] Support IO devices in `File.copy/3`
-  * [GenServer] Raise a more meaningful exit if you try to `GenServer.call/3` yourself
-  * [Inspect] Support `:base` option when inspecting binaries
-  * [IO] Add `IO.warn/2` that will print a warning message with stacktrace and notify the compiler a warning was printed (in case --warnings-as-errors was enabled)
-  * [Kernel] Support `generated: true` in quote
-  * [Kernel] Support `Kernel.pop_in/1` and `Kernel.pop_in/2` for yanking a value from a nested data structure
-  * [Kernel] Allow variable struct names when matching, for example, `%module{key: "value"} = struct`
-  * [Kernel] Allow guards on the left side of `<-` in `for` and `with` special forms
-  * [Kernel] Support `else` chunks in `with`
-  * [Kernel] Track `{module, function, arity}` imports and warn on unused ones when such are specified in `:only`
-  * [Kernel] Add `keyword/0` and `keyword/1` built-in types to typespecs
-  * [Kernel] Add sigils for date (`~D[2015-04-17]`), time (`~T[08:00:00]`) and naive date times `~N[2015-04-17 08:00:00]`
-  * [Kernel] Support `@enforce_keys` on `defstruct/1` to guarantee some keys are explicitly given when building structs
-  * [OptionParser] Add support for `:count` switch type
-  * [OptionParser] Add `parse!/2` and `parse_head!/2` that raise `OptionParser.ParseError` in case of errors
-  * [Process] Add `Process.sleep/1`
-  * [Range] `Range.range?/1` now checks the validity of a range.
-  * [Regex] Support `:include_captures` in `Regex.split/3`
-  * [String] Add `String.myers_difference/2` for calculating the difference between two strings
-  * [System] Add `System.os_time/0` and `System.os_time/1`
-  * [Typespec] Add support for `%{required(foo) => bar}` and `%{optional(foo) => bar}` forms (Erlang 19 only)
-  * [Typespec] Add support for `@optional_callbacks` to mark certain that certain callbacks may be optionally implemented
-  * [Typespec] Introduce `%{...}` to mean any map (Erlang 19 only)
-  * [URI] Add `URI.merge/2`
-  * [Version] Add `Version.parse!/1`
-
-#### ExUnit
-
-  * [ExUnit] Show pinned variables on failed `assert ^left = right` and `assert match?(^left, right)` assertions
-  * [ExUnit] Add `ExUnit.Case.register_attribute` which allow attributes to be cleaned up whenever a test is defined
-  * [ExUnit] Add `ExUnit.Case.register_test` and support the ability to tag "tests" by type. This will allow projects like QuickCheck to change the wording in formatters to say "10 properties" instead of "10 tests"
-  * [ExUnit] Support diffing of values when using `==` in `assert`
-  * [ExUnit] Start running tests as soon as cases are loaded. This feature is enabled by default when running tests through Mix
-  * [ExUnit] Raise a straight-forward error message in case a duplicate test name is defined
-  * [ExUnit] Bump the default number of max cases to double of schedulers to support both IO and CPU bound tests
-  * [ExUnit] Support for named setups in `setup` and `setup_all`
-  * [ExUnit] Support for bundling tests together with `describe/2`
-
-#### IEx
-
-  * [IEx] Add `nl/2` that loads a given module on a list of nodes
-  * [IEx.Helpers] No longer restart applications on `recompile/1`
-  * [IEx.Autocomplete] Improve IEx expand to handle functions after `&`
-
 #### Logger
 
-  * [Logger] Introduce `Logger.reset_metadata/0,1`
-
-#### Mix
-
-  * [Mix] Add `mix xref` and `mix compile.xref` that runs cross-reference checks, with the latter running after compilation by default
-  * [Mix] Add `mix app.tree` and `mix deps.tree`
-  * [Mix] Add `Mix.Task.rerun/2` that reenables and re-runs a task
-  * [Mix] Integrate `OptionParser.ParseError` into Mix, automatically converting such exceptions into `Mix.Error` and embedding the task information
-  * [Mix] Support `@preferred_cli_env` attribute when defining tasks
-  * [Mix] Support `mix test --raise` that will raise when a test suite fails (instead of setting the exit code to 1)
-  * [Mix] Enable rebar3 manager by default for Hex dependencies
-  * [Mix] Add `mix escript.install` to install escripts
-  * [Mix] Print stacktraces for `Mix.Error` when `MIX_DEBUG=1` is set
-  * [Mix] Add a user friendly error for merge conflicts on `mix.lock`
-  * [Mix] Track files between path dependencies. This means umbrella applications will no longer trigger full recompilation when a sibling changes. Instead it will only recompile the files affected by the sibling changes
-  * [Mix] No longer print every file being compiled. Instead a generic "Compiling N files (.ext)" will be printed and files will only be logged in case they take more than 5 seconds to compile. This threshold can be customized by passing the `--long-compilation-threshold` flag and the previous behaviour can be reenabled by giving `--verbose` to `mix compile`
-  * [Mix] Add `mix test --stale` that uses static analysis on source files to know which tests should run when source files changes. If any test file changes, it will also re-run. Changing a configuration file or the test helper will trigger a full recompilation
+  * [Logger] Handle changes to crash reports in OTP 20
 
 ### 2. Bug fixes
 
 #### Elixir
 
-  * [Application] Ensure `Application.spec/2` returns nil for unknown applications
-  * [GenServer] Ensures `cast/2` returns `:ok` if locally registered process is not found
-  * [Inspect] Ensure binaries break into new lines when inspected
-  * [Kernel] Do not choke on capture operator with argument above `&191`
-  * [Kernel] Raise if `defstruct` is called multiple times
-  * [Kernel] Ensure `Module.create/3` respects var/alias hygiene
-  * [Kernel] Support non-literal ranges on the right side of `in/2`
-  * [Macro] Fix `Macro.to_string/1` on a call of a capture argument, for example `&(&1).(:x)`
-  * [OptionParser] Allow `OptionParser` to parse negative numbers
-  * [Record] Fix `Record.is_record/2` when dealing with non-record tuples
-  * [String] Ensure `strip` also removes non-breaking whitespaces (and ensure `split` still does not split on them)
-  * [URI] Use square brackets for IPv6 in `URI.to_string/1`
+  * [DateTime] Fix `DateTime.from_iso8601/2` when offset has no colon
+  * [Registry] Do not leak EXIT messages on `Registry.dispatch/3`
 
-#### Mix
+## v1.4.4 (2017-05-15)
 
-  * [Mix] Improve task not found message when Mix would include the not found task as a suggestion due to different casing
-  * [Mix] Ignore lock revision when the lock is out of date when updating Mix dependencies. Before this fix, Git tags and branches in the lock file would erroneously take higher precedence than the one in `mix.exs`
-  * [Mix] Only recompile empty Elixir files if they change instead of recompiling them on every run
-  * [Mix] Ensure .app file is written in UTF-8 (this allows app descriptions to contain UTF-8 characters)
-  * [Mix.Dep] Always specify the `:env` option internally for dependencies to avoid false positives in the dependency resolution
-  * [Mix.Dep] Correctly detect conflict from cousin optional dependencies in the dependency resolution algorithm
+This version includes changes that make Elixir fully compatible with Erlang/OTP 20-rc.1.
 
-### 3. Soft deprecations (no warnings emitted)
-
-  * [Float] `Float.to_string/2` and `Float.to_char_list/2` has been soft-deprecated as Elixir will now attempt to print the shortest and most accurate representation by default. Developers can always fallback to `:erlang.float_to_binary/2` and `:erlang.float_to_list/2` if they need the previous functionality
-  * [Kernel] `to_char_list` functions have been soft-deprecated in favor of `to_charlist`. This aligns with the naming conventions in both Erlang and Elixir
-  * [String] The confusing `String.strip/2`, `String.lstrip/2` and `String.rstrip/2` API has been soft deprecated in favor of `String.trim/2`, `String.trim_leading/2` and `String.trim_trailing/2`
-  * [String] The confusing `String.ljust/3` and `String.rjust/3` API has been soft deprecated in favor of `String.pad_leading/3` and `String.pad_trailing/3`
-  * [Typespec] `char_list` is soft-deprecated in favor of `charlist`
-
-### 4. Deprecations
-
-This release deprecates many APIs that have been soft-deprecated in previous Elixir versions.
+### 1. Bug fixes
 
 #### Elixir
 
-  * [Dict] `Dict` is no longer a behaviour and its functions will be deprecated in upcoming releases
-  * [Enum] Passing a dictionary to `Enum.group_by/3` is deprecated
-  * [Kernel] `\x{H*}` in strings/sigils/charlists is deprecated
-  * [Kernel] Add deprecation for `defdelegate` list arguments and `:append_first` option. The previously undocumented and deprecated support for matching has been removed
-  * [Kernel] Warn if a variable is assigned inside `case`/`if`/etc and used outside the block
-  * [Keyword] `Keyword.size/1` is deprecated in favor of `Kernel.length/1`
-  * [Map] `Map.size/1` is deprecated in favor of `Kernel.map_size/1`
-  * [Regex] The option `/r` (for ungreedy) has been deprecated in favor of `/U`
-  * [Set] `Set` is no longer a behaviour and its functions will be deprecated in upcoming releases
-  * [String] `String.valid_character?/1` is deprecated in favor of `String.valid?/1` with pattern matching
-  * [Task] `Task.find/2` is deprecated in favor of explicit message matching
-  * [URI] Passing a non-map to `URI.decode_query/2` is deprecated
+  * [Map] Fix regression on struct update syntax
+
+## v1.4.3 (2017-05-15)
+
+This version includes changes that make Elixir fully compatible with Erlang/OTP 20-rc.1.
+
+### 1. Enhancements
+
+#### Elixir
+
+  * [Kernel] Improve compilation time for modules with many clauses
+  * [Map] Warn when attempting to override `__struct__` key
+  * [Regex] Add `recompile/1` and `recompile!/1` to ease transition to OTP 20 for archives and stored regexes
+
+#### Logger
+
+  * [Logger.Translator] Handle OTP 20 GenServer log messages
+
+#### Mix
+
+  * [mix compile] Recompile projects if OTP version changes
+
+### 2. Bug fixes
+
+#### Elixir
+
+  * [Kernel] Fix code generation when non-binary bitstrings are in AST
+  * [Record] Properly escape fields passed to `defrecord`
+  * [Version] Reject leading zeros according to the SemVer spec
+
+#### ExUnit
+
+  * [ExUnit.Diff] Do not fail when comparing maps with nil or boolean keys
+
+#### IEx
+
+  * [IEx.Helpers] Do not log exits on `IEx.Helpers.c/2` failures
+
+#### Mix
+
+  * [mix archive.install] Detect proper path on URLs with query strings
+  * [mix loadpaths] Do not assume all paths in loadpaths exist
+
+
+## v1.4.2 (2017-02-16)
+
+### 1. Bug fixes
+
+#### EEx
+
+  * [EEx] Support middle expressions on trim mode
+
+#### Elixir
+
+  * [Calendar] Correct typo on Calendar types
+  * [Kernel] Ensure redefined functions point to the proper source
+  * [OptionParser] Add `:allow_nonexistent_atoms` to support unsafe behaviour prior to v1.4
+  * [Stream] Allow consuming multiple items from suspended enumerable in `Stream.transform/3`
+  * [String] Incorporate new grapheme rules in Unicode 9
+
+#### IEx
+
+  * [IEx.Autocomplete] Do not crash on aliases which are not known at compile time
+
+#### Mix
+
+  * [Mix.Umbrella] Ensure umbrella projects can depend on other umbrella projects
+  * [Mix.Archive] Ensure previous archives with `.ez` extension are deleted
+
+## v1.4.1 (2017-01-26)
+
+### 1. Bug fixes
+
+#### Elixir
+
+  * [Kernel] Remove warning when making private functions overridable
+  * [Path] Ensure `Path.join/1` returns strings for lists of one element
+  * [Regex] Ensure `Regex.escape/1` also escapes `-`
+
+#### IEx
+
+  * [IEx] Disable ANSI detection for powershell to avoid false positives
+
+#### Mix
+
+  * [Mix.Make] Run `make clean` for `erlang.mk`
+  * [Mix.Rebar] Support all of rebar3 dependency package declaration
+  * [Mix.Rebar] Only pass overrides from parent to child in Rebar dep
+
+## v1.4.0 (2017-01-05)
+
+### 1. Enhancements
+
+#### Elixir
+
+  * [Calendar] Add `Date.compare/2`, `Time.compare/2`, `NaiveDateTime.compare/2` and `DateTime.compare/2`
+  * [Calendar] Support `NaiveDateTime.add/3` and `NaiveDateTime.diff/3` for adding seconds (up to microseconds) as well as the difference between two NaiveDateTimes in seconds (up to microseconds)
+  * [Calendar] Add `Date.leap_year?/1` and `Date.day_of_week/1`
+  * [Calendar] Ensure `Date`, `Time` and `NaiveDateTime` APIs work with any struct that provides the same set of fields as their respective struct. For example, a `NaiveDateTime` can be given to `Date` since it contains a superset of the fields in the `Date` struct
+  * [Calendar] Add `Time.utc_now/0` and `NaiveDateTime.utc_now/0`
+  * [Enum] Add `Enum.map_every/2` that invokes the given function with every nth item
+  * [Enum] Add `min/2`, `max/2`, `min_max/2`, `min_by/3`, `max_by/3`, and `min_max_by/3` that allow a function specifying the default value when the enumerable is empty
+  * [Enum] Introduce `Enum.zip/1` to zip multiple entries at once
+  * [Float] Introduce `Float.ratio/1` that returns a tuple with the numerator and denominator as integers to retrieve the given float
+  * [GenServer] Log warn on default `handle_info/2` implementation
+  * [Inspect] Support syntax coloring via the `:syntax_color` option
+  * [Integer] `Integer.digits/2` now accepts negative integers
+  * [Integer] Add `Integer.mod/2` and `Integer.floor_div/2`
+  * [IO] Add `:label` option to `IO.inspect/2` to help distinguish multiple `IO.inspect/2` calls.
+  * [Kernel] Recognize merge conflict markers in source and provide a readable error message
+  * [Kernel] Warn on unused module attributes
+  * [Kernel] Improve compiler message on unexpected end of line
+  * [Kernel] Raise `BadBooleanError` when a non-boolean is given on the left-hand side of `and`/`or`
+  * [List] Add `List.pop_at/3`
+  * [List] Add `List.myers_difference/2`
+  * [OptionParser] Expand multi-letter aliases in `OptionParser`
+  * [Process] Add `Process.send_after/4`
+  * [Process] Improve error messages on `Process.register/2` errors
+  * [Registry] Add a local, decentralized and scalable key-value process storage
+  * [Stream] Add `Stream.map_every/2` that invokes the given function with every nth item
+  * [Stream] Introduce `Stream.zip/1` to lazily zip multiple entries at once
+  * [String] Update to Unicode 9.0.0
+  * [Task] Add `Task.async_stream/3` and `Task.async_stream/5` as well as the supervised versions `Task.Supervisor.async_stream/4` and `Task.Supervisor.async_stream/6`
+  * [URI] Allow 0 as URI scheme default port
+
+#### ExUnit
+
+  * [ExUnit.Diff] Use red or green background for whitespace-only diffs
+  * [ExUnit.Doctest] Allow inspected structures with multiples lines and unicode characters in the doctest result
+  * [ExUnit.Formatter] Replace lhs/rhs with left/right in the formatter for clarity
+
+#### IEx
+
+  * [IEx.Autocomplete] Stop appending a trailing dot when autocompleting modules in IEx
+  * [IEx.Autocomplete] Support autocompletion for structs
+  * [IEx.Autocomplete] Improve IEx autocomplete to support navigating map atom keys
+  * [IEx.Helpers] `c/1` now compiles in memory by default to avoid common issue where `.beam` files remain at projects root directory
+  * [IEx.Helpers] Add info about protocols in `i/1`
+  * [IEx.Server] Support interrupting IEx evaluation through the Ctrl+G prompt
+
+#### Mix
+
+  * [mix archive] Compress archive files built by `mix archive` as they are now unzipped during installation
+  * [mix archive] Install from SCM
+  * [mix compile] Automatically infer the list of applications for Mix projects
+  * [mix cmd] Add the ability to specify one or more apps in `mix cmd`
+  * [mix deps] Warn if there are non-applications in the `apps` directory for umbrella projects
+  * [mix deps] Add warning for invalid paths on `mix deps.clean`
+  * [mix deps] Add `Mix.Project.apps_paths` that returns the paths to children applications in umbrella projects
+  * [mix deps] Add `MIX_REBAR` environment variable for overriding local rebar
+  * [mix escript] Install from SCM
+  * [mix new] Check directory existence in `mix new` and ask how to proceed if one exists
+  * [mix new] Applications built with the `--sup` flag now have an individual module to work as application callback
+  * [mix test] Add `--formatter` option to `mix test`
+  * [mix xref] Provide "did you mean?" suggestions for `mix xref`
+
+### 2. Bug fixes
+
+#### Elixir
+
+  * [Access] Do not accept nils in `Access.key/1` and `Access.key/2` in favor of explicit default values (or Access.key!/1 if you expect the key to always be available)
+  * [Float] Avoid multiple roundings in `Float.ceil/2`, `Float.floor/2` and `Float.round/2`
+  * [Kernel] Don't crash in `macro_exported?/3` when dealing with Erlang modules
+  * [Kernel] Ensure locals calls are rewritten when calling a local function or macro from inside a module
+  * [Kernel] Annotate the context for variables as zero-arity funs in quotes
+  * [Kernel.SpecialForms] Ensure comprehensions with guards and filters keep proper ordering,
+  * [Kernel.SpecialForms] Produce meaningful warning when with's else clauses have no effect
+  * [Macro] Wrap fn calls in parens in `Macro.to_string/2`
+  * [Macro] Do not print aliases as keys inside keyword lists in `Macro.to_string/2`
+  * [OptionParser] Support options in `OptionParser.to_argv/2` to ensure `:count` switches are correctly encoded
+  * [Stream] Ensure `Stream.take/2` does not consume next element on `:suspend`
+  * [String] Fix infinite recursion in `String.replace_leading/3` and `String.replace_trailing/3` when given an empty string
+  * [Task] Fix `Task.shutdown/1,2` infinite block when task has no monitor
+  * [Task] Ensure task cannot link after parents unlinks
+
+#### ExUnit
+
+  * [ExUnit] Fix a race condition in `assert_receive` where we would assert a message was not received but show it in the list of messages when the message is delivered right after the timeout value
+
+### IEx
+
+  * [IEx.Helpers] Purge consolidated protocols before and after `recompile/0`
+
+### Mix
+
+  * [Mix.Dep] Use `gmake` on FreeBSD instead of `make` when compiling make dependencies
+  * [Mix.Project] Only copy files from source when they're newer than destination (for Windows machines)
+  * [Mix.Task] Ensure non-recursive tasks inside umbrella are reenabled
+
+### 3. Soft deprecations (no warnings emitted)
+
+#### Elixir
+
+  * [Enum] `Enum.partition/2` has been deprecated in favor of `Enum.split_with/2`
+  * [System] Deprecate plural time units in favor of singular ones to align with future Erlang releases
+
+#### ExUnit
+
+  * [ExUnit] Using GenEvent to implement ExUnit formatters is deprecated. Please use the new `GenServer` based formatters instead
+
+### 4. Deprecations
+
+#### Elixir
+
+  * [Access] `Access.key/1` is deprecated due to erratic behaviour for missing keys, please use `Access.key/2` instead with proper default values
+  * [Behaviour] The `Behaviour` module is deprecated. Callbacks may now be defined directly via the `@callback` attribute
+  * [Enum] Deprecate `Enum.uniq/2` in favor of `Enum.uniq_by/2`
+  * [Float] `Float.to_char_list/2` and `Float.to_string/2` are deprecated (use the `:erlang.float_to_list/2` and `:erlang.float_to_binary/2` functions if such conversions are desired)
+  * [HashDict] The `HashDict` module is deprecated, in favour of the `Map`
+  * [HashSet] The `HashDict` module is deprecated, in favour of the `MapSet`  
+  * [Kernel] Deprecate support for making private functions overridable. Overridable functions must always be public as they must be contracts
+  * [Kernel] Anonymous functions with no expression after `->`, in favor of using an expression or returning `nil`
+  * [Kernel] Warn if variable is used as a function call
+  * [OptionParser] Deprecate aliases with multiple letters, such as `-abc`
+  * [Set] Deprecation of the `Set` module in favor of `MapSet`
+  * [Stream] Deprecate `Stream.uniq/2` in favor of `Stream.uniq_by/2`
+
+#### IEx
+
+  * [IEx.Helpers] `import_file/2` is deprecated in favor of `import_file_if_available/1`
+
+#### Mix
+
+  * [Mix.Utils] `underscore/1` and `camelize/1` are deprecated in favor of `Macro.underscore/1` and `Macro.camelize/1`
+
+## v1.3
+
+The CHANGELOG for v1.3 releases can be found [in the v1.3 branch](https://github.com/elixir-lang/elixir/blob/v1.3/CHANGELOG.md).

ISSUE_TEMPLATE.md

@@ -1,6 +1,6 @@
 ### Precheck
 
-* Do not use the issues tracker for help or support (try Stack Overflow, IRC, mailing list, etc)
+* Do not use the issues tracker for help or support (try Elixir Forum, Stack Overflow, IRC, etc.)
 * For proposing a new feature, please start a discussion on the Elixir Core mailing list
 * For bugs, do a quick search and make sure the bug has not yet been reported
 * Finally, be nice and have fun!

Makefile

@@ -1,13 +1,14 @@
 REBAR ?= "$(CURDIR)/rebar"
 PREFIX ?= /usr/local
-DOCS := master
-CANONICAL := master
+SHARE_PREFIX ?= $(PREFIX)/share
+CANONICAL :=
 ELIXIRC := bin/elixirc --verbose --ignore-module-conflict
 ERLC := erlc -I lib/elixir/include
 ERL := erl -I lib/elixir/include -noshell -pa lib/elixir/ebin
 VERSION := $(strip $(shell cat VERSION))
 Q := @
 LIBDIR := lib
+BINDIR := bin
 INSTALL = install
 INSTALL_DIR = $(INSTALL) -m755 -d
 INSTALL_DATA = $(INSTALL) -m644
@@ -15,13 +16,13 @@ INSTALL_PROGRAM = $(INSTALL) -m755
 GIT_REVISION = $(strip $(shell git rev-parse HEAD 2> /dev/null ))
 GIT_TAG = $(strip $(shell head="$(call GIT_REVISION)"; git tag --points-at $$head 2> /dev/null | tail -1) )
 
-.PHONY: install compile erlang elixir build_plt clean_plt dialyze test clean install_man clean_man docs Docs.zip Precompiled.zip publish_zips publish_docs publish_mix
+.PHONY: install compile erlang elixir build_plt clean_plt dialyze test clean install_man clean_man docs Docs.zip Precompiled.zip zips
 .NOTPARALLEL: compile
 
 #==> Functions
 
 define CHECK_ERLANG_RELEASE
-	$(Q) erl -noshell -eval 'io:fwrite("~s", [erlang:system_info(otp_release) >= "18"])' -s erlang halt | grep -q '^true'; \
+	$(Q) erl -noshell -eval '{V,_} = string:to_integer(erlang:system_info(otp_release)), io:fwrite("~s", [is_integer(V) and (V >= 18)])' -s erlang halt | grep -q '^true'; \
 		if [ $$? != 0 ]; then \
 		   echo "At least Erlang 18.0 is required to build Elixir"; \
 		   exit 1; \
@@ -105,9 +106,9 @@ install: compile
 	done
 	$(Q) $(INSTALL_DIR) "$(DESTDIR)$(PREFIX)/$(LIBDIR)/elixir/bin"
 	$(Q) $(INSTALL_PROGRAM) $(filter-out %.ps1, $(filter-out %.bat, $(wildcard bin/*))) "$(DESTDIR)$(PREFIX)/$(LIBDIR)/elixir/bin"
-	$(Q) $(INSTALL_DIR) "$(DESTDIR)$(PREFIX)/bin"
+	$(Q) $(INSTALL_DIR) "$(DESTDIR)$(PREFIX)/$(BINDIR)"
 	$(Q) for file in "$(DESTDIR)$(PREFIX)"/$(LIBDIR)/elixir/bin/* ; do \
-		ln -sf "../$(LIBDIR)/elixir/bin/$${file##*/}" "$(DESTDIR)$(PREFIX)/bin/" ; \
+		ln -sf "../$(LIBDIR)/elixir/bin/$${file##*/}" "$(DESTDIR)$(PREFIX)/$(BINDIR)/" ; \
 	done
 	$(MAKE) install_man
 
@@ -130,14 +131,15 @@ clean_exbeam:
 
 LOGO_PATH = $(shell test -f ../docs/logo.png && echo "--logo ../docs/logo.png")
 SOURCE_REF = $(shell tag="$(call GIT_TAG)" revision="$(call GIT_REVISION)"; echo "$${tag:-$$revision}\c")
-COMPILE_DOCS = bin/elixir ../ex_doc/bin/ex_doc "$(1)" "$(VERSION)" "lib/$(2)/ebin" -m "$(3)" -u "https://github.com/elixir-lang/elixir" --source-ref "$(call SOURCE_REF)" $(call LOGO_PATH) -o doc/$(2) -a http://elixir-lang.org/docs/$(CANONICAL)/$(2)/ -p http://elixir-lang.org/docs.html $(4)
+DOCS_FORMAT = html
+COMPILE_DOCS = bin/elixir ../ex_doc/bin/ex_doc "$(1)" "$(VERSION)" "lib/$(2)/ebin" -m "$(3)" -u "https://github.com/elixir-lang/elixir" --source-ref "$(call SOURCE_REF)" $(call LOGO_PATH) -o doc/$(2) -n https://hexdocs.pm/$(2)/$(CANONICAL) -p http://elixir-lang.org/docs.html -f "$(DOCS_FORMAT)" $(4)
 
 docs: compile ../ex_doc/bin/ex_doc docs_elixir docs_eex docs_mix docs_iex docs_ex_unit docs_logger
 
 docs_elixir: compile ../ex_doc/bin/ex_doc
 	@ echo "==> ex_doc (elixir)"
 	$(Q) rm -rf doc/elixir
-	$(call COMPILE_DOCS,Elixir,elixir,Kernel,-e "lib/elixir/pages/Behaviours.md" -e "lib/elixir/pages/Naming Conventions.md" -e "lib/elixir/pages/Typespecs.md" -e "lib/elixir/pages/Writing Documentation.md")
+	$(call COMPILE_DOCS,Elixir,elixir,Kernel,-e "lib/elixir/pages/Behaviours.md" -e "lib/elixir/pages/Guards.md" -e "lib/elixir/pages/Naming Conventions.md" -e "lib/elixir/pages/Operators.md" -e "lib/elixir/pages/Typespecs.md" -e "lib/elixir/pages/Writing Documentation.md")
 
 docs_eex: compile ../ex_doc/bin/ex_doc
 	@ echo "==> ex_doc (eex)"
@@ -180,13 +182,7 @@ Precompiled.zip: build_man compile
 	zip -9 -r Precompiled-v$(VERSION).zip bin CHANGELOG.md lib/*/ebin LICENSE man NOTICE README.md VERSION
 	@ echo "Precompiled file created $(CURDIR)/Precompiled-v$(VERSION).zip"
 
-#==> Publish
-
-publish_zips: Precompiled.zip Docs.zip
-
-publish_docs: docs
-	rm -rf ../docs/$(DOCS)/*/
-	cp -R doc/* ../docs/$(DOCS)
+zips: Precompiled.zip Docs.zip
 
 #==> Tests tasks
 
@@ -255,9 +251,9 @@ clean_man:
 	rm -f man/iex.1
 
 install_man: build_man
-	$(Q) mkdir -p $(DESTDIR)$(PREFIX)/share/man/man1
-	$(Q) $(INSTALL_DATA) man/elixir.1  $(DESTDIR)$(PREFIX)/share/man/man1
-	$(Q) $(INSTALL_DATA) man/elixirc.1 $(DESTDIR)$(PREFIX)/share/man/man1
-	$(Q) $(INSTALL_DATA) man/iex.1     $(DESTDIR)$(PREFIX)/share/man/man1
-	$(Q) $(INSTALL_DATA) man/mix.1     $(DESTDIR)$(PREFIX)/share/man/man1
+	$(Q) mkdir -p $(DESTDIR)$(SHARE_PREFIX)/man/man1
+	$(Q) $(INSTALL_DATA) man/elixir.1  $(DESTDIR)$(SHARE_PREFIX)/man/man1
+	$(Q) $(INSTALL_DATA) man/elixirc.1 $(DESTDIR)$(SHARE_PREFIX)/man/man1
+	$(Q) $(INSTALL_DATA) man/iex.1     $(DESTDIR)$(SHARE_PREFIX)/man/man1
+	$(Q) $(INSTALL_DATA) man/mix.1     $(DESTDIR)$(SHARE_PREFIX)/man/man1
 	$(MAKE) clean_man

README.md

@@ -6,7 +6,7 @@
 For more about Elixir, installation and documentation,
 [check Elixir's website](http://elixir-lang.org/).
 
-## Usage
+## Compiling from source
 
 To run Elixir from source, clone this repository to your machine, compile and test it:
 
@@ -24,9 +24,8 @@ If Elixir fails to build (specifically when pulling in a new version via
 `git`), be sure to remove any previous build artifacts by running
 `make clean`, then `make test`.
 
-If tests pass, you are ready to move on to the
-[Getting Started guide][1] or to try Interactive Elixir by running:
-`bin/iex` in your terminal.
+If tests pass, you are ready to move on to the [Getting Started guide][1]
+or to try Interactive Elixir by running `bin/iex` in your terminal.
 
 However, if tests fail, it is likely you have an outdated Erlang version
 (Elixir requires Erlang 18.0 or later). You can check your Erlang version
@@ -34,15 +33,20 @@ by calling `erl` in the command line. You will see some information as follows:
 
     Erlang/OTP 18 [erts-7.0] [source] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false]
 
-If you have the correct version and tests still fail, please
-[open an issue][2].
+If you have properly set up your dependencies and tests still fail,
+you may want to open up a bug report, as explained next.
+
+## Bug reports
+
+For reporting bugs, [visit our issues tracker][2] and follow the steps
+for reporting a new issue. Please disclose security vulnerabilities
+privately at elixir-security@googlegroups.com.
 
 ## Contributing
 
-We welcome everyone to contribute to Elixir and help us tackle
-existing issues! To do so, there are a few things you need to know
-about the code. First, Elixir code is divided in applications inside
-the `lib` folder:
+We welcome everyone to contribute to Elixir and help us tackle existing issues!
+To do so, there are a few things you need to know about the code. First, Elixir
+code is divided in applications inside the `lib` folder:
 
 * `elixir` - Contains Elixir's kernel and stdlib
 
@@ -86,12 +90,28 @@ case you are looking for some examples:
 * [Implement capture_io for ExUnit – Pull Request](https://github.com/elixir-lang/elixir/pull/1059)
 
 We usually keep a list of enhancements and bugs [in the issue tracker][2].
-For proposing a new feature, please start a discussion in the
+For proposing new features, please start a discussion in the
 [Elixir Core mailing list][3]. Keep in mind that it is your responsibility
 to argue and explain why a feature is useful and how it will impact the
 codebase and the community. Finally, remember all interactions in our official
 spaces follow our [Code of Conduct][7].
 
+### Reviewing changes
+
+Once a pull request is sent, the Elixir team will review your changes.
+We outline our process below to clarify the roles of everyone involved.
+
+All pull requests must be approved by two committers before being merged into
+the repository. In case any changes are necessary, the team will leave
+appropriate comments requesting changes to the code.
+
+The Elixir team may optionally assign someone to review a pull request.
+In case someone is assigned, they must explicitly approve the code before
+another team member can merge it.
+
+When review is completed, your pull request will be squashed and merged
+into the repository.
+
 ## Building documentation
 
 Building the documentation requires [ExDoc](https://github.com/elixir-lang/ex_doc)

RELEASE.md

@@ -18,19 +18,13 @@ This document simply outlines the release process:
 
 7. Push branch and the new tag
 
-8. Publish new docs with `make publish_docs`, copy docs to `docs/stable` if appropriate, and push to GitHub Pages
+8. If a new `vMAJOR.MINOR`, create a new branch "vMAJOR.MINOR" and set `CANONICAL=` in Makefile before building docs
 
-9. Publish new zips with `make publish_zips`, upload `Precompiled.zip` and `Docs.zip` to GitHub Releases
+9. Publish new zips with `make zips`, upload `Precompiled.zip` and `Docs.zip` to GitHub Releases
 
-10. Add the release to `elixir.csv` file in `elixir-lang/elixir-lang.github.com`
+10. Add the release to `elixir.csv` and `_data/elixir-versions.yml` files in `elixir-lang/elixir-lang.github.com`
 
-## New vMAJOR.MINOR releases
-
-11. Create a new branch "vMAJOR.MINOR"
-
-12. Move docs generation to `docs/vMAJOR.MINOR` in Makefile, set CANONICAL reference and copy them to `docs/stable` (change index.html accordingly)
-
-13. In master, bump versions, start new CHANGELOG, add `-dev` back and commit "Start vVERSION+1"
+11. After a new `vMAJOR.MINOR`, move back to master, bump versions, start new CHANGELOG, add `-dev` back and commit "Start vMAJOR.MINOR+1"
 
 ## Places where version is mentioned
 

VERSION

@@ -1 +1 @@
-1.3.0-rc.1
+1.4.5

bin/elixir.bat

@@ -1,10 +1,10 @@
-@if defined ELIXIR_CLI_ECHO (@echo on) else  (@echo off)
+@if defined ELIXIR_CLI_ECHO (@echo on) else (@echo off)
 setlocal
-if    ""%1""==""""       goto :documentation
-if /I ""%1""==""--help"" goto :documentation
-if /I ""%1""==""-h""     goto :documentation
-if /I ""%1""==""/h""     goto :documentation
-if    ""%1""==""/?""     goto :documentation
+if    ""%1""==""""       goto documentation
+if /I ""%1""==""--help"" goto documentation
+if /I ""%1""==""-h""     goto documentation
+if /I ""%1""==""/h""     goto documentation
+if    ""%1""==""/?""     goto documentation
 goto parseopts
 
 :documentation
@@ -61,51 +61,52 @@ set par="%1"
 shift
 if "%par%"=="" (
   rem if no parameters defined
-  goto :expand_erl_libs
+  goto expand_erl_libs
 )
 if "%par%"=="""" (
   rem if no parameters defined - special case for parameter that is already quoted
-  goto :expand_erl_libs
+  goto expand_erl_libs
 )
 rem ******* EXECUTION OPTIONS **********************
-IF "%par%"==""--werl"" (Set useWerl=1)
-IF "%par%"==""+iex"" (Set runMode="iex")
+if "%par%"==""--werl"" (set useWerl=1)
+if "%par%"==""+iex"" (set runMode="iex")
 rem ******* ELIXIR PARAMETERS **********************
 rem Note: we don't have to do anything with options that don't take an argument
-IF """"=="%par:-e=%"      (shift) 
-IF """"=="%par:-r=%"      (shift) 
-IF """"=="%par:-pr=%"     (shift) 
-IF """"=="%par:-pa=%"     (shift) 
-IF """"=="%par:-pz=%"     (shift) 
-IF """"=="%par:--app=%"   (shift) 
-IF """"=="%par:--remsh=%" (shift) 
+if """"=="%par:-e=%"      (shift)
+if """"=="%par:-r=%"      (shift)
+if """"=="%par:-pr=%"     (shift)
+if """"=="%par:-pa=%"     (shift)
+if """"=="%par:-pz=%"     (shift)
+if """"=="%par:--app=%"   (shift)
+if """"=="%par:--remsh=%" (shift)
 rem ******* ERLANG PARAMETERS **********************
-IF """"=="%par:--detached=%"            (Set parsErlang=%parsErlang% -detached) 
-IF """"=="%par:--hidden=%"              (Set parsErlang=%parsErlang% -hidden)
-IF """"=="%par:--cookie=%"              (Set parsErlang=%parsErlang% -setcookie %1 && shift)
-IF """"=="%par:--sname=%"               (Set parsErlang=%parsErlang% -sname %1 && shift) 
-IF """"=="%par:--name=%"                (Set parsErlang=%parsErlang% -name %1 && shift) 
-IF """"=="%par:--logger-otp-reports=%"  (Set parsErlang=%parsErlang% -logger handle_otp_reports %1 && shift) 
-IF """"=="%par:--logger-sasl-reports=%" (Set parsErlang=%parsErlang% -logger handle_sasl_reports %1 && shift) 
-IF """"=="%par:--erl=%"                 (Set beforeExtra=%beforeExtra% %~1 && shift) 
+if """"=="%par:--detached=%"            (set parsErlang=%parsErlang% -detached)
+if """"=="%par:--hidden=%"              (set parsErlang=%parsErlang% -hidden)
+if """"=="%par:--cookie=%"              (set parsErlang=%parsErlang% -setcookie %1 && shift)
+if """"=="%par:--sname=%"               (set parsErlang=%parsErlang% -sname %1 && shift)
+if """"=="%par:--name=%"                (set parsErlang=%parsErlang% -name %1 && shift)
+if """"=="%par:--logger-otp-reports=%"  (set parsErlang=%parsErlang% -logger handle_otp_reports %1 && shift)
+if """"=="%par:--logger-sasl-reports=%" (set parsErlang=%parsErlang% -logger handle_sasl_reports %1 && shift)
+if """"=="%par:--erl=%"                 (set beforeExtra=%beforeExtra% %~1 && shift)
 goto:startloop
 
 rem ******* assume all pre-params are parsed ********************
 :expand_erl_libs
 rem ******* expand all ebin paths as Windows does not support the ..\*\ebin wildcard ********************
-SETLOCAL enabledelayedexpansion
+setlocal enabledelayedexpansion
 set ext_libs=
 for  /d %%d in ("%originPath%..\lib\*.") do (
   set ext_libs=!ext_libs! -pa "%%~fd\ebin"
 )
-SETLOCAL disabledelayedexpansion
+setlocal disabledelayedexpansion
+
 :run
-IF NOT %runMode% == "iex" (
+if not %runMode% == "iex" (
   set beforeExtra=-noshell -s elixir start_cli %beforeExtra%
 )
-IF %useWerl% EQU 1 (
+if %useWerl% equ 1 (
   start werl.exe %ext_libs% %ELIXIR_ERL_OPTIONS% %parsErlang% %beforeExtra% -extra %*
-) ELSE (
+) else (
   erl.exe %ext_libs% %ELIXIR_ERL_OPTIONS% %parsErlang% %beforeExtra% -extra %*
 )
 :end

bin/elixirc.bat

@@ -1,12 +1,12 @@
-@if defined ELIXIR_CLI_ECHO (@echo on) else  (@echo off)
+@if defined ELIXIR_CLI_ECHO (@echo on) else (@echo off)
 setlocal
 set argc=0
 for %%A in (%*) do (
-    if /I "%%A"=="--help" goto documentation
-    if /I "%%A"=="-h"     goto documentation
-    if /I "%%A"=="/h"     goto documentation
-    if    "%%A"=="/?"     goto documentation
-    set /A argc+=1
+  if /I "%%A"=="--help" goto documentation
+  if /I "%%A"=="-h"     goto documentation
+  if /I "%%A"=="/h"     goto documentation
+  if    "%%A"=="/?"     goto documentation
+  set /A argc+=1
 )
 if %argc%==0 goto documentation
 goto run

bin/iex.bat

@@ -1,5 +1,5 @@
 @if defined ELIXIR_CLI_ECHO (@echo on) else  (@echo off)
-SETLOCAL
+setlocal
 if /I ""%1""==""--help"" goto documentation
 if /I ""%1""==""-h""     goto documentation
 if /I ""%1""==""/h""     goto documentation
@@ -42,4 +42,4 @@ goto end
 @if defined IEX_WITH_WERL (@set __ELIXIR_IEX_FLAGS=--werl) else (set __ELIXIR_IEX_FLAGS=)
 call "%~dp0\elixir.bat" +iex --erl "-user Elixir.IEx.CLI" --no-halt %__ELIXIR_IEX_FLAGS% %*
 :end
-ENDLOCAL
+endlocal

bin/mix.bat

@@ -1,2 +1,2 @@
-@if defined ELIXIR_CLI_ECHO (@echo on) else  (@echo off)
-call "%~dp0\elixir.bat" "%~dp0\mix" %*
+@if defined ELIXIR_CLI_ECHO (@echo on) else (@echo off)
+call "%~dp0\elixir.bat" "%~dp0\mix" %*

lib/eex/lib/eex.ex

@@ -111,7 +111,7 @@ defmodule EEx do
 
   """
   defmacro function_from_string(kind, name, source, args \\ [], options \\ []) do
-    quote bind_quoted: binding do
+    quote bind_quoted: binding() do
       info = Keyword.merge [file: __ENV__.file, line: __ENV__.line], options
       args = Enum.map args, fn arg -> {arg, [line: info[:line]], nil} end
       compiled = EEx.compile_string(source, info)
@@ -148,7 +148,7 @@ defmodule EEx do
 
   """
   defmacro function_from_file(kind, name, file, args \\ [], options \\ []) do
-    quote bind_quoted: binding do
+    quote bind_quoted: binding() do
       info = Keyword.merge options, [file: file, line: 1]
       args = Enum.map args, fn arg -> {arg, [line: 1], nil} end
       compiled = EEx.compile_file(file, info)
@@ -166,7 +166,8 @@ defmodule EEx do
   Gets a string `source` and generate a quoted expression
   that can be evaluated by Elixir or compiled to a function.
   """
-  def compile_string(source, options \\ []) do
+  @spec compile_string(String.t, Keyword.t) :: Macro.t | no_return
+  def compile_string(source, options \\ []) when is_binary(source) and is_list(options) do
     EEx.Compiler.compile(source, options)
   end
 
@@ -174,7 +175,8 @@ defmodule EEx do
   Gets a `filename` and generate a quoted expression
   that can be evaluated by Elixir or compiled to a function.
   """
-  def compile_file(filename, options \\ []) do
+  @spec compile_file(String.t, Keyword.t) :: Macro.t | no_return
+  def compile_file(filename, options \\ []) when is_binary(filename) and is_list(options) do
     options = Keyword.merge options, [file: filename, line: 1]
     compile_string(File.read!(filename), options)
   end
@@ -188,7 +190,9 @@ defmodule EEx do
       "foo baz"
 
   """
-  def eval_string(source, bindings \\ [], options \\ []) do
+  @spec eval_string(String.t, Keyword.t, Keyword.t) :: any
+  def eval_string(source, bindings \\ [], options \\ [])
+      when is_binary(source) and is_list(bindings) and is_list(options) do
     compiled = compile_string(source, options)
     do_eval(compiled, bindings, options)
   end
@@ -205,7 +209,9 @@ defmodule EEx do
       EEx.eval_file "sample.eex", [bar: "baz"] #=> "foo baz"
 
   """
-  def eval_file(filename, bindings \\ [], options \\ []) do
+  @spec eval_file(String.t, Keyword.t, Keyword.t) :: any
+  def eval_file(filename, bindings \\ [], options \\ [])
+      when is_binary(filename) and is_list(bindings) and is_list(options) do
     options  = Keyword.put options, :file, filename
     compiled = compile_file(filename, options)
     do_eval(compiled, bindings, options)

lib/eex/lib/eex/compiler.ex

@@ -9,7 +9,8 @@ defmodule EEx.Compiler do
   and the engine together by handling the tokens and invoking
   the engine every time a full expression or text is received.
   """
-  def compile(source, opts) do
+  @spec compile(String.t, Keyword.t) :: Macro.t | no_return
+  def compile(source, opts) when is_binary(source) and is_list(opts) do
     file   = opts[:file] || "nofile"
     line   = opts[:line] || 1
     trim   = opts[:trim] || false
@@ -23,7 +24,8 @@ defmodule EEx.Compiler do
     end
   end
 
-  # Generates the buffers by handling each expression from the tokenizer
+  # Generates the buffers by handling each expression from the tokenizer.
+  # It returns Macro.t/0 or it raises.
 
   defp generate_buffer([{:text, chars} | t], buffer, scope, state) do
     buffer = state.engine.handle_text(buffer, IO.chardata_to_string(chars))
@@ -88,7 +90,9 @@ defmodule EEx.Compiler do
       {contents, start, list}
     end
   end
-
+  defp look_ahead_text([{:middle_expr, line, _, chars} | t], _start, contents) do
+    {contents ++ chars, line, t}
+  end
   defp look_ahead_text(t, start, contents) do
     {contents, start, t}
   end

lib/eex/lib/eex/engine.ex

@@ -2,7 +2,7 @@ defmodule EEx.Engine do
   @moduledoc ~S"""
   Basic EEx engine that ships with Elixir.
 
-  An engine needs to implement three functions:
+  An engine needs to implement four functions:
 
     * `init(opts)` - returns the initial buffer
 
@@ -27,10 +27,10 @@ defmodule EEx.Engine do
   default implementations for the functions above.
   """
 
-  @callback init(Keyword.t) :: Macro.t
-  @callback handle_body(Macro.t) :: Macro.t
-  @callback handle_text(Macro.t, String.t) :: Macro.t
-  @callback handle_expr(Macro.t, String.t, Macro.t) :: Macro.t
+  @callback init(opts :: Keyword.t) :: Macro.t
+  @callback handle_body(quoted :: Macro.t) :: Macro.t
+  @callback handle_text(buffer :: Macro.t, text :: String.t) :: Macro.t
+  @callback handle_expr(buffer :: Macro.t, marker :: String.t, expr :: Macro.t) :: Macro.t
 
   @doc false
   defmacro __using__(_) do
@@ -41,8 +41,8 @@ defmodule EEx.Engine do
         EEx.Engine.init(opts)
       end
 
-      def handle_body(body) do
-        EEx.Engine.handle_body(body)
+      def handle_body(quoted) do
+        EEx.Engine.handle_body(quoted)
       end
 
       def handle_text(buffer, text) do
@@ -72,6 +72,7 @@ defmodule EEx.Engine do
       end
 
   """
+  @spec handle_assign(Macro.t) :: Macro.t
   def handle_assign({:@, meta, [{name, _, atom}]}) when is_atom(name) and is_atom(atom) do
     line = meta[:line] || 0
     quote line: line, do: EEx.Engine.fetch_assign!(var!(assigns), unquote(name))
@@ -82,6 +83,7 @@ defmodule EEx.Engine do
 
   @doc false
   # TODO: Raise on 2.0
+  @spec fetch_assign!(map, Map.key) :: term | nil
   def fetch_assign!(assigns, key) do
     case Access.fetch(assigns, key) do
       {:ok, val} ->
@@ -124,7 +126,6 @@ defmodule EEx.Engine do
 
   All other markers are not implemented by this engine.
   """
-  @spec handle_expr(Macro.t, String.t, Macro.t) :: Macro.t
   def handle_expr(buffer, "=", expr) do
     quote do
       tmp1 = unquote(buffer)

lib/eex/lib/eex/tokenizer.ex

@@ -1,26 +1,34 @@
 defmodule EEx.Tokenizer do
   @moduledoc false
 
+  @type content :: IO.chardata
+  @type line :: non_neg_integer
+  @type token :: {:text, content} |
+                 {:expr | :start_expr | :middle_expr | :end_expr, line, '=' | '', content}
+
   @doc """
   Tokenizes the given charlist or binary.
 
   It returns {:ok, list} with the following tokens:
 
-    * `{:text, contents}`
-    * `{:expr, line, marker, contents}`
-    * `{:start_expr, line, marker, contents}`
-    * `{:middle_expr, line, marker, contents}`
-    * `{:end_expr, line, marker, contents}`
+    * `{:text, content}`
+    * `{:expr, line, marker, content}`
+    * `{:start_expr, line, marker, content}`
+    * `{:middle_expr, line, marker, content}`
+    * `{:end_expr, line, marker, content}`
 
   Or `{:error, line, error}` in case of errors.
   """
+  @spec tokenize(binary | charlist, line, Keyword.t) :: {:ok, [token]} | {:error, line, String.t}
   def tokenize(bin, line, opts \\ [])
 
-  def tokenize(bin, line, opts) when is_binary(bin) do
+  def tokenize(bin, line, opts)
+      when is_binary(bin) and is_integer(line) and line >= 0 and is_list(opts) do
     tokenize(String.to_charlist(bin), line, opts)
   end
 
-  def tokenize(list, line, opts) do
+  def tokenize(list, line, opts)
+      when is_list(list) and is_integer(line) and line >= 0 and is_list(opts) do
     tokenize(list, line, opts, [], [])
   end
 
@@ -173,7 +181,6 @@ defmodule EEx.Tokenizer do
   # If trim mode is enabled and the token is on a line with
   # only itself and whitespace, trim the whitespace around it,
   # including the line break following it if there is one.
-
   defp trim_if_needed(rest, line, opts, buffer, acc) do
     original = {rest, line, buffer}
     if opts[:trim] do

lib/eex/test/eex_test.exs

@@ -4,7 +4,7 @@ require EEx
 
 defmodule EExTest.Compiled do
   def before_compile do
-    fill_in_stacktrace
+    fill_in_stacktrace()
     {__ENV__.line, hd(tl(System.stacktrace))}
   end
 
@@ -19,13 +19,13 @@ defmodule EExTest.Compiled do
   def file_sample(arg), do: private_file_sample(arg)
 
   def after_compile do
-    fill_in_stacktrace
+    fill_in_stacktrace()
     {__ENV__.line, hd(tl(System.stacktrace))}
   end
 
   @file "unknown"
   def unknown do
-    fill_in_stacktrace
+    fill_in_stacktrace()
     {__ENV__.line, hd(tl(System.stacktrace))}
   end
 
@@ -331,6 +331,19 @@ foo
     assert_eval expected, string, [], trim: true
   end
 
+  test "trim mode with middle expression" do
+    string = """
+    <%= cond do %>
+    <% false -> %>
+      this
+    <% true -> %>
+      that
+    <% end %>
+    """
+    expected = "  that\n"
+    assert_eval expected, string, [], trim: true
+  end
+
   test "evaluates the source from a given file" do
     filename = Path.join(__DIR__, "fixtures/eex_template.eex")
     result = EEx.eval_file(filename)

lib/elixir/lib/access.ex

@@ -17,7 +17,7 @@ defmodule Access do
 
   ## Dynamic lookups
 
-  Out of the box, Access works with `Keyword` and `Map`:
+  Out of the box, `Access` works with `Keyword` and `Map`:
 
       iex> keywords = [a: 1, b: 2]
       iex> keywords[:a]
@@ -31,7 +31,10 @@ defmodule Access do
       iex> star_ratings[1.5]
       "★☆"
 
-  Access can be combined with `Kernel.put_in/3` to put a value
+  Note that the dynamic lookup syntax (`term[key]`) roughly translates to
+  `Access.get(term, key, nil)`.
+
+  `Access` can be combined with `Kernel.put_in/3` to put a value
   in a given key:
 
       iex> map = %{a: 1, b: 2}
@@ -44,47 +47,51 @@ defmodule Access do
       iex> put_in users["john"][:age], 28
       %{"john" => %{age: 28}, "meg" => %{age: 23}}
 
-  Furthermore, Access transparently ignores `nil` values:
+  Furthermore, `Access` transparently ignores `nil` values:
 
       iex> keywords = [a: 1, b: 2]
       iex> keywords[:c][:unknown]
       nil
 
-  Since Access is a behaviour, it can be implemented to key-value
+  Since `Access` is a behaviour, it can be implemented for key-value
   data structures. The implementation should be added to the
-  module that defines the struct being access. Access requires the
+  module that defines the struct being accessed. `Access` requires the
   key comparison to be implemented using the `===` operator.
 
   ## Static lookups
 
-  The Access syntax (`foo[bar]`) cannot be used to access fields in
-  structs, since structs do not implement the Access behaviour by
-  default. It is also design decision: the dynamic access lookup
+  The `Access` syntax (`foo[bar]`) cannot be used to access fields in
+  structs, since structs do not implement the `Access` behaviour by
+  default. It is also a design decision: the dynamic access lookup
   is meant to be used for dynamic key-value structures, like maps
-  and keywords, and not by static ones like structs.
+  and keywords, and not by static ones like structs (where fields are
+  known and not dynamic).
 
-  Therefore Elixir provides a static lookup for map and structs
-  fields. Imagine a struct named `User` with name and age fields.
+  Therefore Elixir provides a static lookup for struct fields and for atom
+  fields in maps. Imagine a struct named `User` with a `:name` field.
   The following would raise:
 
-      user = %User{name: "john"}
+      user = %User{name: "John"}
       user[:name]
-      ** (UndefinedFunctionError) undefined function User.fetch/2
-         (User does not implement the Access behaviour)
+      # ** (UndefinedFunctionError) undefined function User.fetch/2
+      #    (User does not implement the Access behaviour)
 
-  Structs instead use the `user.name` syntax:
+  Structs instead use the `user.name` syntax to access fields:
 
       user.name
-      #=> "john"
+      #=> "John"
 
   The same `user.name` syntax can also be used by `Kernel.put_in/2`
   to for updating structs fields:
 
-      put_in user.name, "mary"
-      %User{name: "mary"}
+      put_in user.name, "Mary"
+      #=> %User{name: "Mary"}
 
   Differently from `user[:name]`, `user.name` is not extensible via
-  a behaviour and is restricted to only maps and structs.
+  a behaviour and is restricted only to structs and atom keys in maps.
+
+  As mentioned above, this works for atom keys in maps as well. Refer to the
+  `Map` module for more information on this.
 
   Summing up:
 
@@ -111,18 +118,102 @@ defmodule Access do
         languages: [%{name: "ELIXIR", type: :functional},
                     %{name: "C", type: :procedural}]}
 
-  See the functions `key/1`, `key!/1`, `elem/1` and `all/0` for the current
-  accessors.
+  See the functions `key/1`, `key!/1`, `elem/1`, and `all/0` for some of the
+  available accessors.
+
+  ## Implementing the Access behaviour for custom data structures
+
+  In order to be able to use the `Access` protocol with custom data structures
+  (which have to be structs), such structures have to implement the `Access`
+  behaviour. For example, for a `User` struct, this would have to be done:
+
+      defmodule User do
+        defstruct [:name, :email]
+
+        @behaviour Access
+        # Implementation of the Access callbacks...
+      end
+
   """
 
-  @type t :: list | map | nil
+  @type t :: list | map | nil | any
   @type key :: any
   @type value :: any
 
-  @callback fetch(t, key) :: {:ok, value} | :error
-  @callback get(t, key, value) :: value
-  @callback get_and_update(t, key, (value -> {value, value} | :pop)) :: {value, t}
-  @callback pop(t, key) :: {value, t}
+  @doc """
+  Invoked in order to access the value stored under `key` in the given term `term`.
+
+  This function should return `{:ok, value}` where `value` is the value under
+  `key` if it succeeded, or `:error` if the key does not exist in the structure.
+
+  Many of the functions defined in the `Access` module internally call this
+  function. This function is also used when the square-brackets access syntax
+  (`structure[key]`) is used: the `fetch/2` callback implemented by the module
+  that defines the `structure` struct is invoked and if it returns `{:ok,
+  value}` then `value` is returned, or if it returns `:error` then `nil` is
+  returned.
+
+
+  See the `Map.fetch/2` and `Keyword.fetch/2` implementations for examples of
+  how to implement this callback.
+  """
+  @callback fetch(term :: t, key) :: {:ok, value} | :error
+
+  @doc """
+  Invoked in order to access the value stored under `key` in the given term `term`,
+  defaulting to `default` if not present.
+
+  This function should return the value under the key `key` in `term` if there's
+  such key, otherwise `default`.
+
+  For most data structures, this can be implemented using `fetch/2` internally;
+  for example:
+
+      def get(structure, key, default) do
+        case fetch(structure, key) do
+          {:ok, value} -> value
+          :error       -> default
+        end
+      end
+
+  See the `Map.get/3` and `Keyword.get/3` implementations for more examples.
+  """
+  @callback get(term :: t, key, default :: value) :: value
+
+  @doc """
+  Invoked in order to access the value under `key` and update it at the same time.
+
+  The implementation of this callback should invoke the passed function with the
+  value under key `key` in the passed structure, or `nil` if the key is not
+  present. This function should return either `{value_to_return, new_value}` or
+  `:pop`.
+
+  If it returns `{value_to_return, new_value}`, the return value of this
+  callback should be `{value_to_return, new_term}` where `new_term` is `term`
+  after updating the value of `key` with `new_value`.
+
+  If it returns `:pop`, the return value of this callback should be `{value,
+  new_term}` where `value` is the value under `key` or `nil` if not present, and
+  `new_term` is `term` without the key `key`.
+
+  See the implementations of `Map.get_and_update/3` or `Keyword.get_and_update/3`
+  for more examples.
+  """
+  @callback get_and_update(term :: t, key, (value -> {value, value} | :pop)) :: {value, t}
+
+  @doc """
+  Invoked to "pop" the value under `key` out of the given term.
+
+  When the key `key` exists in the given `term`, the implementation should
+  return a `{value, new_term}` tuple where `value` is the value that was under
+  `key` and `new_term` is `term` without `key`.
+
+  When the key `key` is not present in the given `term`, a tuple `{value, term}`
+  should be returned, where `value` is implementation-defined.
+
+  See the implementations for `Map.pop/3` or `Keyword.pop/3` for more examples.
+  """
+  @callback pop(term :: t, key) :: {value, t}
 
   defmacrop raise_undefined_behaviour(e, struct, top) do
     quote do
@@ -139,7 +230,8 @@ defmodule Access do
   end
 
   @doc """
-  Fetches the container's value for the given key.
+  Fetches the value for the given key in a container (a map, keyword
+  list, or struct that implements the `Access` behaviour).
   """
   @spec fetch(t, term) :: {:ok, term} | :error
   def fetch(container, key)
@@ -151,15 +243,12 @@ defmodule Access do
       raise_undefined_behaviour e, struct, {^struct, :fetch, [^container, ^key], _}
   end
 
-  def fetch(%{} = map, key) do
-    :maps.find(key, map)
+  def fetch(map, key) when is_map(map) do
+    Map.fetch(map, key)
   end
 
   def fetch(list, key) when is_list(list) and is_atom(key) do
-    case :lists.keyfind(key, 1, list) do
-      {^key, value} -> {:ok, value}
-      false -> :error
-    end
+    Keyword.fetch(list, key)
   end
 
   def fetch(list, key) when is_list(list) do
@@ -172,7 +261,8 @@ defmodule Access do
   end
 
   @doc """
-  Gets the container's value for the given key.
+  Gets the value for the given key in a container (a map, keyword
+  list, or struct that implements the `Access` behaviour).
   """
   @spec get(t, term, term) :: term
   def get(container, key, default \\ nil) do
@@ -183,19 +273,21 @@ defmodule Access do
   end
 
   @doc """
-  Gets and updates the container's value for the given key, in a single pass.
+  Gets and updates the given key in a container (a map, keyword
+  list, or struct that implements the `Access` behaviour).
 
   This `fun` argument receives the value of `key` (or `nil` if `key`
   is not present) and must return a two-element tuple: the "get" value
   (the retrieved value, which can be operated on before being returned)
   and the new value to be stored under `key`. The `fun` may also
   return `:pop`, implying the current value shall be removed
-  from the map and returned.
+  from the container and returned.
 
-  The returned value is a tuple with the "get" value returned by
-  `fun` and a new map with the updated value under `key`.
+  The returned value is a two-element tuple with the "get" value returned by
+  `fun` and a new container with the updated value under `key`.
   """
-  @spec get_and_update(t, key, (value -> {get, value})) :: {get, t} when get: var
+  @spec get_and_update(container :: t, key, (value -> {get_value, update_value} | :pop)) ::
+        {get_value, container :: t} when get_value: var, update_value: value
   def get_and_update(container, key, fun)
 
   def get_and_update(%{__struct__: struct} = container, key, fun) do
@@ -205,7 +297,7 @@ defmodule Access do
       raise_undefined_behaviour e, struct, {^struct, :get_and_update, [^container, ^key, ^fun], _}
   end
 
-  def get_and_update(%{} = map, key, fun) do
+  def get_and_update(map, key, fun) when is_map(map) do
     Map.get_and_update(map, key, fun)
   end
 
@@ -218,19 +310,47 @@ defmodule Access do
       "could not put/update key #{inspect key} on a nil value"
   end
 
+  @doc """
+  Removes the entry with a given key from a container (a map, keyword
+  list, or struct that implements the `Access` behaviour).
+
+  Returns a tuple containing the value associated with the key and the
+  updated container. `nil` is returned for the value if the key isn't
+  in the container.
+
+  ## Examples
+
+  With a map:
+
+      iex> Access.pop(%{name: "Elixir", creator: "Valim"}, :name)
+      {"Elixir", %{creator: "Valim"}}
+
+  A keyword list:
+
+      iex> Access.pop([name: "Elixir", creator: "Valim"], :name)
+      {"Elixir", [creator: "Valim"]}
+
+  An unknown key:
+
+      iex> Access.pop(%{name: "Elixir", creator: "Valim"}, :year)
+      {nil, %{creator: "Valim", name: "Elixir"}}
+
+  """
   def pop(%{__struct__: struct} = container, key) do
     struct.pop(container, key)
   rescue
     e in UndefinedFunctionError ->
       raise_undefined_behaviour e, struct, {^struct, :pop, [^container, ^key], _}
   end
-  def pop(list, key) when is_list(list), do: Keyword.pop(list, key)
+
   def pop(map, key) when is_map(map) do
-    case map do
-      %{^key => value} -> {value, :maps.remove(key, map)}
-      %{} -> {nil, map}
-    end
+    Map.pop(map, key)
   end
+
+  def pop(list, key) when is_list(list) do
+    Keyword.pop(list, key)
+  end
+
   def pop(nil, key) do
     raise ArgumentError,
       "could not pop key #{inspect key} on a nil value"
@@ -238,43 +358,17 @@ defmodule Access do
 
   ## Accessors
 
-  @doc """
-  Accesses the given key in a map/struct.
+  @doc false
+  def key(key) do
+    IO.warn "Access.key/1 is deprecated due to erratic behaviour for missing keys, " <>
+            "please use Access.key/2 instead with proper default values " <>
+            "(or Access.key!/1 if you expect the key to always be available)"
 
-  Uses the default value if the key does not exist
-  or if the value being accessed is `nil`.
-
-  ## Examples
-
-      iex> get_in(%{}, [Access.key(:unknown), Access.key(:name)])
-      nil
-      iex> get_in(%{}, [Access.key(:unknown, %{name: "john"}), Access.key(:name)])
-      "john"
-      iex> get_in(%{}, [Access.key(:unknown), Access.key(:name, "john")])
-      "john"
-
-      iex> map = %{user: %{name: "john"}}
-      iex> get_in(map, [Access.key(:unknown), Access.key(:name, "john")])
-      "john"
-      iex> get_and_update_in(map, [Access.key(:user), Access.key(:name)], fn
-      ...>   prev -> {prev, String.upcase(prev)}
-      ...> end)
-      {"john", %{user: %{name: "JOHN"}}}
-      iex> pop_in(map, [Access.key(:user), Access.key(:name)])
-      {"john", %{user: %{}}}
-
-  An error is raised if the accessed structure is not a map/struct/nil:
-
-      iex> get_in([], [Access.key(:foo)])
-      ** (RuntimeError) Access.key/1 expected a map/struct or nil, got: []
-
-  """
-  def key(key, default \\ nil) do
     fn
       :get, data, next ->
-        next.(Map.get(to_map(data), key, default))
+        next.(Map.get(to_map(data), key))
       :get_and_update, data, next ->
-        value = Map.get(to_map(data), key, default)
+        value = Map.get(to_map(data), key)
         case next.(value) do
           {get, update} -> {get, Map.put(data, key, update)}
           :pop -> {value, Map.delete(data, key)}
@@ -287,7 +381,62 @@ defmodule Access do
   defp to_map(data), do: raise "Access.key/1 expected a map/struct or nil, got: #{inspect data}"
 
   @doc """
-  Accesses the given key in a map/struct.
+  Returns a function that accesses the given key in a map/struct.
+
+  The returned function is typically passed as an accessor to `Kernel.get_in/2`,
+  `Kernel.get_and_update_in/3`, and friends.
+
+  The returned function uses the default value if the key does not exist.
+  This can be used to specify defaults and safely traverse missing keys:
+
+      iex> get_in(%{}, [Access.key(:user, %{}), Access.key(:name, nil)])
+      nil
+
+  Such is also useful when using update functions, allowing us to introduce
+  values as we traverse the data-structure for updates:
+
+      iex> put_in(%{}, [Access.key(:user, %{}), Access.key(:name, nil)], "Mary")
+      %{user: %{name: "Mary"}}
+
+  ## Examples
+
+      iex> map = %{user: %{name: "john"}}
+      iex> get_in(map, [Access.key(:unknown, %{}), Access.key(:name, "john")])
+      "john"
+      iex> get_and_update_in(map, [Access.key!(:user), Access.key!(:name)], fn
+      ...>   prev -> {prev, String.upcase(prev)}
+      ...> end)
+      {"john", %{user: %{name: "JOHN"}}}
+      iex> pop_in(map, [Access.key!(:user), Access.key!(:name)])
+      {"john", %{user: %{}}}
+
+  An error is raised if the accessed structure is not a map or a struct:
+
+      iex> get_in(nil, [Access.key(:foo, nil)])
+      ** (BadMapError) expected a map, got: nil
+
+      iex> get_in([], [Access.key(:foo, nil)])
+      ** (BadMapError) expected a map, got: []
+
+  """
+  def key(key, default) do
+    fn
+      :get, data, next ->
+        next.(Map.get(data, key, default))
+      :get_and_update, data, next ->
+        value = Map.get(data, key, default)
+        case next.(value) do
+          {get, update} -> {get, Map.put(data, key, update)}
+          :pop -> {value, Map.delete(data, key)}
+        end
+    end
+  end
+
+  @doc """
+  Returns a function that accesses the given key in a map/struct.
+
+  The returned function is typically passed as an accessor to `Kernel.get_in/2`,
+  `Kernel.get_and_update_in/3`, and friends.
 
   Raises if the key does not exist.
 
@@ -327,7 +476,10 @@ defmodule Access do
   end
 
   @doc ~S"""
-  Accesses the element at the given index in a tuple.
+  Returns a function that accesses the element at the given index in a tuple.
+
+  The returned function is typically passed as an accessor to `Kernel.get_in/2`,
+  `Kernel.get_and_update_in/3`, and friends.
 
   Raises if the index is out of bounds.
 
@@ -367,7 +519,10 @@ defmodule Access do
   end
 
   @doc ~S"""
-  Accesses all the elements in a list.
+  Returns a function that accesses all the elements in a list.
+
+  The returned function is typically passed as an accessor to `Kernel.get_in/2`,
+  `Kernel.get_and_update_in/3`, and friends.
 
   ## Examples
 
@@ -424,7 +579,10 @@ defmodule Access do
   end
 
   @doc ~S"""
-  Accesses the element at `index` (zero based) of a list.
+  Returns a function that accesses the element at `index` (zero based) of a list.
+
+  The returned function is typically passed as an accessor to `Kernel.get_in/2`,
+  `Kernel.get_and_update_in/3`, and friends.
 
   ## Examples
 
@@ -435,6 +593,13 @@ defmodule Access do
       ...>   prev -> {prev, String.upcase(prev)}
       ...> end)
       {"john", [%{name: "JOHN"}, %{name: "mary"}]}
+
+  `at/1` can also be used to pop elements out of a list or
+  a key inside of a list:
+
+      iex> list = [%{name: "john"}, %{name: "mary"}]
+      iex> pop_in(list, [Access.at(0)])
+      {%{name: "john"}, [%{name: "mary"}]}
       iex> pop_in(list, [Access.at(0), :name])
       {"john", [%{}, %{name: "mary"}]}
 
@@ -477,7 +642,7 @@ defmodule Access do
   defp get_and_update_at([head | rest], 0, next, updates) do
     case next.(head) do
       {get, update} -> {get, :lists.reverse([update | updates], rest)}
-      :pop -> {head, :lists.reverse([head | updates], rest)}
+      :pop -> {head, :lists.reverse(updates, rest)}
     end
   end
 

lib/elixir/lib/agent.ex

@@ -144,9 +144,9 @@ defmodule Agent do
   ## Return values
 
   If the server is successfully created and initialized, the function returns
-  `{:ok, pid}`, where `pid` is the pid of the server. If an agent with the
+  `{:ok, pid}`, where `pid` is the PID of the server. If an agent with the
   specified name already exists, the function returns
-  `{:error, {:already_started, pid}}` with the pid of that process.
+  `{:error, {:already_started, pid}}` with the PID of that process.
 
   If the given function callback fails with `reason`, the function returns
   `{:error, reason}`.

lib/elixir/lib/application.ex

@@ -26,7 +26,7 @@ defmodule Application do
   Once an application is started, OTP provides an application environment
   that can be used to configure the application.
 
-  Assuming you are inside a Mix project, you can edit the `application`
+  Assuming you are inside a Mix project, you can edit the `application/0`
   function in the `mix.exs` file to the following:
 
       def application do
@@ -116,10 +116,10 @@ defmodule Application do
   specification key (e.g., `mod: {MyApp, [:my_args]}`).
 
   This function should either return `{:ok, pid}` or `{:ok, pid, state}` if
-  startup is successful. `pid` should be the pid of the top supervisor. `state`
+  startup is successful. `pid` should be the PID of the top supervisor. `state`
   can be an arbitrary term, and if omitted will default to `[]`; if the
   application is later stopped, `state` is passed to the `stop/1` callback (see
-  the documentation for the `stop/2` callback for more information).
+  the documentation for the `c:stop/1` callback for more information).
 
   `use Application` provides no default implementation for the `start/2`
   callback.
@@ -466,7 +466,7 @@ defmodule Application do
     try do
       do_format_error(reason)
     catch
-      # A user could create an error that looks like a builtin one
+      # A user could create an error that looks like a built-in one
       # causing an error.
       :error, _ ->
         inspect(reason)

lib/elixir/lib/base.ex

@@ -158,8 +158,8 @@ defmodule Base do
 
   The values for `:case` can be:
 
-    * `:upper` - use upper case characters (default)
-    * `:lower` - use lower case characters
+    * `:upper` - uses upper case characters (default)
+    * `:lower` - uses lower case characters
 
   ## Examples
 
@@ -188,9 +188,9 @@ defmodule Base do
 
   The values for `:case` can be:
 
-    * `:upper` - only allow upper case characters (default)
-    * `:lower` - only allow lower case characters
-    * `:mixed` - allow mixed case characters
+    * `:upper` - only allows upper case characters (default)
+    * `:lower` - only allows lower case characters
+    * `:mixed` - allows mixed case characters
 
   ## Examples
 
@@ -223,9 +223,9 @@ defmodule Base do
 
   The values for `:case` can be:
 
-    * `:upper` - only allow upper case characters (default)
-    * `:lower` - only allow lower case characters
-    * `:mixed` - allow mixed case characters
+    * `:upper` - only allows upper case characters (default)
+    * `:lower` - only allows lower case characters
+    * `:mixed` - allows mixed case characters
 
   An `ArgumentError` exception is raised if the padding is incorrect or
   a non-alphabet character is present in the string.
@@ -443,8 +443,8 @@ defmodule Base do
 
   The values for `:case` can be:
 
-    * `:upper` - use upper case characters (default)
-    * `:lower` - use lower case characters
+    * `:upper` - uses upper case characters (default)
+    * `:lower` - uses lower case characters
 
   The values for `:padding` can be:
 
@@ -483,14 +483,14 @@ defmodule Base do
 
   The values for `:case` can be:
 
-    * `:upper` - only allow  upper case characters (default)
-    * `:lower` - only allow lower case characters
-    * `:mixed` - allow mixed case characters
+    * `:upper` - only allows  upper case characters (default)
+    * `:lower` - only allows lower case characters
+    * `:mixed` - allows mixed case characters
 
   The values for `:padding` can be:
 
-    * `true` - require the input string to be padded to the nearest multiple of 8 (default)
-    * `false` - ignore padding from the input string
+    * `true` - requires the input string to be padded to the nearest multiple of 8 (default)
+    * `false` - ignores padding from the input string
 
   ## Examples
 
@@ -530,14 +530,14 @@ defmodule Base do
 
   The values for `:case` can be:
 
-    * `:upper` - only allow upper case characters (default)
-    * `:lower` - only allow lower case characters
-    * `:mixed` - allow mixed case characters
+    * `:upper` - only allows upper case characters (default)
+    * `:lower` - only allows lower case characters
+    * `:mixed` - allows mixed case characters
 
   The values for `:padding` can be:
 
-    * `true` - require the input string to be padded to the nearest multiple of 8 (default)
-    * `false` - ignore padding from the input string
+    * `true` - requires the input string to be padded to the nearest multiple of 8 (default)
+    * `false` - ignores padding from the input string
 
   ## Examples
 
@@ -575,8 +575,8 @@ defmodule Base do
 
   The values for `:case` can be:
 
-    * `:upper` - use upper case characters (default)
-    * `:lower` - use lower case characters
+    * `:upper` - uses upper case characters (default)
+    * `:lower` - uses lower case characters
 
   The values for `:padding` can be:
 
@@ -616,14 +616,14 @@ defmodule Base do
 
   The values for `:case` can be:
 
-    * `:upper` - only allow upper case characters (default)
-    * `:lower` - only allow lower case characters
-    * `:mixed` - allow mixed case characters
+    * `:upper` - only allows upper case characters (default)
+    * `:lower` - only allows lower case characters
+    * `:mixed` - allows mixed case characters
 
   The values for `:padding` can be:
 
-    * `true` - require the input string to be padded to the nearest multiple of 8 (default)
-    * `false` - ignore padding from the input string
+    * `true` - requires the input string to be padded to the nearest multiple of 8 (default)
+    * `false` - ignores padding from the input string
 
   ## Examples
 
@@ -664,14 +664,14 @@ defmodule Base do
 
   The values for `:case` can be:
 
-    * `:upper` - only allow upper case characters (default)
-    * `:lower` - only allow lower case characters
-    * `:mixed` - allow mixed case characters
+    * `:upper` - only allows upper case characters (default)
+    * `:lower` - only allows lower case characters
+    * `:mixed` - allows mixed case characters
 
   The values for `:padding` can be:
 
-    * `true` - require the input string to be padded to the nearest multiple of 8 (default)
-    * `false` - ignore padding from the input string
+    * `true` - requires the input string to be padded to the nearest multiple of 8 (default)
+    * `false` - ignores padding from the input string
 
   ## Examples
 

lib/elixir/lib/behaviour.ex

@@ -2,12 +2,13 @@ defmodule Behaviour do
   @moduledoc """
   This module has been deprecated.
 
-  Instead of `defcallback`, one can simply use `@callback`.
-  Instead of `defmacrocallback`, one can simply use `@macrocallback`.
-  Instead of `__behaviour__(:callbacks)`, one can simply use `behaviour_info(:callbacks)`.
-  """
+  Instead of `defcallback/1` and `defmacrocallback/1`, the `@callback` and
+  `@macrocallback` module attributes can be used (respectively). See the
+  documentation for `Module` for more information on these attributes.
 
-  # TODO: Deprecate by 1.4
+  Instead of `MyModule.__behaviour__(:callbacks)`,
+  `MyModule.behaviour_info(:callbacks)` can be used.
+  """
 
   @doc """
   Defines a function callback according to the given type specification.
@@ -79,6 +80,12 @@ defmodule Behaviour do
   @doc false
   defmacro __using__(_) do
     quote do
+      warning =
+        "the Behaviour module is deprecated. Instead of using this module, " <>
+        "use the @callback and @macrocallback module attributes. See the " <>
+        "documentation for Module for more information on these attributes"
+      IO.warn(warning)
+
       @doc false
       def __behaviour__(:callbacks) do
         __MODULE__.behaviour_info(:callbacks)

lib/elixir/lib/bitwise.ex

@@ -14,8 +14,8 @@ defmodule Bitwise do
   If you prefer to use only operators or skip them, you can
   pass the following options:
 
-    * `:only_operators` - include only operators
-    * `:skip_operators` - skip operators
+    * `:only_operators` - includes only operators
+    * `:skip_operators` - skips operators
 
   For example:
 

lib/elixir/lib/calendar/iso.ex

@@ -10,32 +10,39 @@ defmodule Calendar.ISO do
   """
 
   @behaviour Calendar
+  @unix_epoch :calendar.datetime_to_gregorian_seconds {{1970, 1, 1}, {0, 0, 0}}
+
+  @type year  :: 0..9999
+  @type month :: 1..12
+  @type day   :: 1..31
 
   @doc """
-  Builds and validates an ISO date.
+  Returns how many days there are in the given year-month.
 
   ## Examples
 
-      iex> Calendar.ISO.date(2000, 1, 1)
-      {:ok, ~D[2000-01-01]}
-      iex> Calendar.ISO.date(2000, 13, 1)
-      {:error, :invalid_date}
-      iex> Calendar.ISO.date(2000, 2, 29)
-      {:ok, ~D[2000-02-29]}
-
-      iex> Calendar.ISO.date(2000, 2, 30)
-      {:error, :invalid_date}
-      iex> Calendar.ISO.date(2001, 2, 29)
-      {:error, :invalid_date}
+      iex> Calendar.ISO.days_in_month(1900, 1)
+      31
+      iex> Calendar.ISO.days_in_month(1900, 2)
+      28
+      iex> Calendar.ISO.days_in_month(2000, 2)
+      29
+      iex> Calendar.ISO.days_in_month(2001, 2)
+      28
+      iex> Calendar.ISO.days_in_month(2004, 2)
+      29
+      iex> Calendar.ISO.days_in_month(2004, 4)
+      30
 
   """
-  def date(year, month, day) when is_integer(year) and is_integer(month) and is_integer(day) do
-    if :calendar.valid_date(year, month, day) and year <= 9999 do
-      {:ok, %Date{year: year, month: month, day: day}}
-    else
-      {:error, :invalid_date}
-    end
+  @spec days_in_month(year, month) :: 28..31
+  def days_in_month(year, month)
+
+  def days_in_month(year, 2) do
+    if leap_year?(year), do: 29, else: 28
   end
+  def days_in_month(_, month) when month in [4, 6, 9, 11], do: 30
+  def days_in_month(_, month) when month in 1..12, do: 31
 
   @doc """
   Returns if the given year is a leap year.
@@ -52,53 +59,64 @@ defmodule Calendar.ISO do
       false
 
   """
+  @spec leap_year?(year) :: boolean()
   def leap_year?(year) when is_integer(year) and year >= 0 do
     rem(year, 4) === 0 and (rem(year, 100) > 0 or rem(year, 400) === 0)
   end
 
   @doc """
-  Converts the given structure into a string.
+  Calculates the day of the week from the given `year`, `month`, and `day`.
 
-  It uses the ISO8601 standard except for DateTime where the
-  timezone information is added between brackets.
+  It is an integer from 1 to 7, where 1 is Monday and 7 is Sunday.
+
+  ## Examples
+
+      iex> Calendar.ISO.day_of_week(2016, 10, 31)
+      1
+      iex> Calendar.ISO.day_of_week(2016, 11, 01)
+      2
+      iex> Calendar.ISO.day_of_week(2016, 11, 02)
+      3
+      iex> Calendar.ISO.day_of_week(2016, 11, 03)
+      4
+      iex> Calendar.ISO.day_of_week(2016, 11, 04)
+      5
+      iex> Calendar.ISO.day_of_week(2016, 11, 05)
+      6
+      iex> Calendar.ISO.day_of_week(2016, 11, 06)
+      7
   """
-  def to_string(%Date{year: year, month: month, day: day}) do
-    date_to_string(year, month, day)
+  @spec day_of_week(year, month, day) :: 1..7
+  def day_of_week(year, month, day)
+      when is_integer(year) and is_integer(month) and is_integer(day) do
+    :calendar.day_of_the_week(year, month, day)
   end
 
-  def to_string(%Time{hour: hour, minute: minute, second: second, microsecond: microsecond}) do
-    time_to_string(hour, minute, second, microsecond)
+  @doc """
+  Converts the given date into a string.
+  """
+  def date_to_string(year, month, day) do
+    zero_pad(year, 4) <> "-" <> zero_pad(month, 2) <> "-" <> zero_pad(day, 2)
   end
 
-  def to_string(%NaiveDateTime{year: year, month: month, day: day,
-                               hour: hour, minute: minute, second: second, microsecond: microsecond}) do
+  @doc """
+  Converts the datetime (without time zone) into a string.
+  """
+  def naive_datetime_to_string(year, month, day, hour, minute, second, microsecond) do
     date_to_string(year, month, day) <> " " <> time_to_string(hour, minute, second, microsecond)
   end
 
-  def to_string(%DateTime{year: year, month: month, day: day, zone_abbr: zone_abbr,
-                          hour: hour, minute: minute, second: second, microsecond: microsecond,
-                          utc_offset: utc_offset, std_offset: std_offset, time_zone: time_zone}) do
+  @doc """
+  Convers the datetime (with time zone) into a string.
+  """
+  def datetime_to_string(year, month, day, hour, minute, second, microsecond,
+                         time_zone, zone_abbr, utc_offset, std_offset) do
     date_to_string(year, month, day) <> " " <>
       time_to_string(hour, minute, second, microsecond) <>
       offset_to_string(utc_offset, std_offset, time_zone) <>
       zone_to_string(utc_offset, std_offset, zone_abbr, time_zone)
   end
 
-  defp date_to_string(year, month, day) do
-    zero_pad(year, 4) <> "-" <> zero_pad(month, 2) <> "-" <> zero_pad(day, 2)
-  end
-
-  defp time_to_string(hour, minute, second, 0) do
-    zero_pad(hour, 2) <> ":" <> zero_pad(minute, 2) <> ":" <> zero_pad(second, 2)
-  end
-  defp time_to_string(hour, minute, second, {_, 0}) do
-    time_to_string(hour, minute, second, 0)
-  end
-  defp time_to_string(hour, minute, second, {microsecond, precision}) do
-    time_to_string(hour, minute, second, 0) <> "." <>
-      (microsecond |> zero_pad(6) |> binary_part(0, precision))
-  end
-
   defp offset_to_string(0, 0, "Etc/UTC"), do: "Z"
   defp offset_to_string(utc, std, _zone) do
     total  = utc + std
@@ -122,22 +140,69 @@ defmodule Calendar.ISO do
   ## Helpers
 
   @doc false
-  def to_iso8601(%Date{year: year, month: month, day: day}) do
+  def time_to_string(hour, minute, second, {_, 0}) do
+    time_to_string(hour, minute, second)
+  end
+  def time_to_string(hour, minute, second, {microsecond, precision}) do
+    time_to_string(hour, minute, second) <> "." <>
+      (microsecond |> zero_pad(6) |> binary_part(0, precision))
+  end
+  defp time_to_string(hour, minute, second) do
+    zero_pad(hour, 2) <> ":" <> zero_pad(minute, 2) <> ":" <> zero_pad(second, 2)
+  end
+
+  @doc false
+  def date(year, month, day) when is_integer(year) and is_integer(month) and is_integer(day) do
+    if :calendar.valid_date(year, month, day) and year <= 9999 do
+      {:ok, %Date{year: year, month: month, day: day}}
+    else
+      {:error, :invalid_date}
+    end
+  end
+
+  @doc false
+  def from_unix(integer, unit) when is_integer(integer) do
+    total = System.convert_time_unit(integer, unit, :microsecond)
+    if total < -@unix_epoch * 1_000_000 do
+      {:error, :invalid_unix_time}
+    else
+      microsecond = rem(total, 1_000_000)
+      precision = precision_for_unit(unit)
+      {date, time} = :calendar.gregorian_seconds_to_datetime(@unix_epoch + div(total, 1_000_000))
+      {:ok, date, time, {microsecond, precision}}
+    end
+  end
+
+  defp precision_for_unit(unit) do
+    subsecond = div System.convert_time_unit(1, :second, unit), 10
+    precision_for_unit(subsecond, 0)
+  end
+
+  defp precision_for_unit(0, precision),
+    do: precision
+  defp precision_for_unit(_, 6),
+    do: 6
+  defp precision_for_unit(number, precision),
+    do: precision_for_unit(div(number, 10), precision + 1)
+
+  @doc false
+  def date_to_iso8601(year, month, day) do
     date_to_string(year, month, day)
   end
 
-  def to_iso8601(%Time{hour: hour, minute: minute, second: second, microsecond: microsecond}) do
+  @doc false
+  def time_to_iso8601(hour, minute, second, microsecond) do
     time_to_string(hour, minute, second, microsecond)
   end
 
-  def to_iso8601(%NaiveDateTime{year: year, month: month, day: day,
-                                hour: hour, minute: minute, second: second, microsecond: microsecond}) do
+  @doc false
+  def naive_datetime_to_iso8601(year, month, day, hour, minute, second, microsecond) do
     date_to_string(year, month, day) <> "T" <> time_to_string(hour, minute, second, microsecond)
   end
 
-  def to_iso8601(%DateTime{year: year, month: month, day: day,
-                           hour: hour, minute: minute, second: second, microsecond: microsecond,
-                           utc_offset: utc_offset, std_offset: std_offset, time_zone: time_zone}) do
+  @doc false
+  def datetime_to_iso8601(year, month, day, hour, minute, second, microsecond,
+                          time_zone, _zone_abbr, utc_offset, std_offset) do
     date_to_string(year, month, day) <> "T" <>
       time_to_string(hour, minute, second, microsecond) <>
       offset_to_string(utc_offset, std_offset, time_zone)
@@ -175,6 +240,14 @@ defmodule Calendar.ISO do
     do: parse_offset(1, hour, min, rest)
   def parse_offset(<<?-, hour::2-bytes, ?:, min::2-bytes, rest::binary>>),
     do: parse_offset(-1, hour, min, rest)
+  def parse_offset(<<?+, hour::2-bytes, min::2-bytes, rest::binary>>),
+    do: parse_offset(1, hour, min, rest)
+  def parse_offset(<<?-, hour::2-bytes, min::2-bytes, rest::binary>>),
+    do: parse_offset(-1, hour, min, rest)
+  def parse_offset(<<?+, hour::2-bytes, rest::binary>>),
+    do: parse_offset(1, hour, "00", rest)
+  def parse_offset(<<?-, hour::2-bytes, rest::binary>>),
+    do: parse_offset(-1, hour, "00", rest)
   def parse_offset(_),
     do: :error
 

lib/elixir/lib/code.ex

@@ -104,7 +104,9 @@ defmodule Code do
   The `binding` argument is a keyword list of variable bindings.
   The `opts` argument is a keyword list of environment options.
 
-  Those options can be:
+  ## Options
+
+  Options can be:
 
     * `:file` - the file to be considered in the evaluation
     * `:line` - the line on which the script starts
@@ -148,7 +150,7 @@ defmodule Code do
       iex> Code.eval_string("a = a + b", [a: 1, b: 2])
       {3, [a: 3, b: 2]}
 
-  For convenience, you can pass `__ENV__` as the `opts` argument and
+  For convenience, you can pass `__ENV__/0` as the `opts` argument and
   all imports, requires and aliases defined in the current environment
   will be automatically carried over:
 
@@ -172,7 +174,12 @@ defmodule Code do
   @doc """
   Evaluates the quoted contents.
 
-  See `eval_string/3` for a description of arguments and return values.
+  **Warning**: Calling this function inside a macro is considered bad
+  practice as it will attempt to evaluate runtime values at compile time.
+  Macro arguments are typically transformed by unquoting them into the
+  returned quoted expressions (instead of evaluated).
+
+  See `eval_string/3` for a description of bindings and options.
 
   ## Examples
 
@@ -180,7 +187,7 @@ defmodule Code do
       iex> Code.eval_quoted(contents, [a: 1, b: 2], file: __ENV__.file, line: __ENV__.line)
       {3, [a: 1, b: 2]}
 
-  For convenience, you can pass `__ENV__` as the `opts` argument and
+  For convenience, you can pass `__ENV__/0` as the `opts` argument and
   all options will be automatically extracted from the current environment:
 
       iex> contents = quote(do: var!(a) + var!(b))
@@ -247,9 +254,9 @@ defmodule Code do
   ## Options
 
     * `:file` - the filename to be used in stacktraces
-      and the file reported in the `__ENV__` variable
+      and the file reported in the `__ENV__/0` macro
 
-    * `:line` - the line reported in the `__ENV__` variable
+    * `:line` - the line reported in the `__ENV__/0` macro
 
     * `:existing_atoms_only` - when `true`, raises an error
       when non-existing atoms are found by the tokenizer
@@ -390,11 +397,11 @@ defmodule Code do
   ## Examples
 
       iex> Code.available_compiler_options
-      [:docs, :debug_info, :ignore_module_conflict, :warnings_as_errors]
+      [:docs, :debug_info, :ignore_module_conflict, :relative_paths, :warnings_as_errors]
 
   """
   def available_compiler_options do
-    [:docs, :debug_info, :ignore_module_conflict, :warnings_as_errors]
+    [:docs, :debug_info, :ignore_module_conflict, :relative_paths, :warnings_as_errors]
   end
 
   @doc """
@@ -414,7 +421,11 @@ defmodule Code do
     * `:ignore_module_conflict` - when `true`, override modules that were
       already defined without raising errors, `false` by default
 
-    * `:warnings_as_errors` - cause compilation to fail when warnings are
+    * `:relative_paths` - when `true`, use relative paths in quoted nodes,
+      warnings and errors generated by the compiler, `true` by default.
+      Note disabling this option won't affect runtime warnings and errors.
+
+    * `:warnings_as_errors` - causes compilation to fail when warnings are
       generated
 
   It returns the new list of compiler options.
@@ -429,9 +440,16 @@ defmodule Code do
   def compiler_options(opts) do
     available = available_compiler_options()
 
-    for {k, _} <- opts,
-        not k in available,
-        do: raise "unknown compiler options: #{k}"
+    Enum.each(opts, fn({key, value}) ->
+      cond do
+        not key in available ->
+          raise "unknown compiler option: #{inspect(key)}"
+        not is_boolean(value) ->
+          raise "compiler option #{inspect(key)} should be a boolean, got: #{inspect(value)}"
+        true ->
+          :ok
+      end
+    end)
 
     :elixir_config.update :compiler_options, &Enum.into(opts, &1)
   end
@@ -505,6 +523,8 @@ defmodule Code do
       {:error, :nofile}
 
   """
+  @spec ensure_loaded(module) ::
+        {:module, module} | {:error, :embedded | :badfile | :nofile | :on_load_failure}
   def ensure_loaded(module) when is_atom(module) do
     :code.ensure_loaded(module)
   end
@@ -522,7 +542,7 @@ defmodule Code do
       true
 
   """
-  def ensure_loaded?(module) do
+  def ensure_loaded?(module) when is_atom(module) do
     match?({:module, ^module}, ensure_loaded(module))
   end
 
@@ -539,6 +559,8 @@ defmodule Code do
   Check `ensure_loaded/1` for more information on module loading
   and when to use `ensure_loaded/1` or `ensure_compiled/1`.
   """
+  @spec ensure_compiled(module) ::
+        {:module, module} | {:error, :embedded | :badfile | :nofile | :on_load_failure}
   def ensure_compiled(module) when is_atom(module) do
     case :code.ensure_loaded(module) do
       {:error, :nofile} = error ->
@@ -559,7 +581,8 @@ defmodule Code do
   is already loaded or was successfully loaded and compiled.
   Returns `false` otherwise.
   """
-  def ensure_compiled?(module) do
+  @spec ensure_compiled?(module) :: boolean
+  def ensure_compiled?(module) when is_atom(module) do
     match?({:module, ^module}, ensure_compiled(module))
   end
 

lib/elixir/lib/dict.ex

@@ -72,7 +72,7 @@ defmodule Dict do
       end
 
       def take(dict, keys) do
-        Enum.reduce(keys, new, fn key, acc ->
+        Enum.reduce(keys, new(), fn key, acc ->
           case fetch(dict, key) do
             {:ok, value} -> put(acc, key, value)
             :error -> acc
@@ -166,7 +166,7 @@ defmodule Dict do
       end
 
       def split(dict, keys) do
-        Enum.reduce(keys, {new, dict}, fn key, {inc, exc} = acc ->
+        Enum.reduce(keys, {new(), dict}, fn key, {inc, exc} = acc ->
           case fetch(exc, key) do
             {:ok, value} ->
               {put(inc, key, value), delete(exc, key)}

lib/elixir/lib/exception.ex

@@ -7,7 +7,7 @@ defmodule Exception do
   `System.stacktrace/0` will return the stacktrace for the
   last throw/error/exit that occurred in the current process.
 
-  Do not rely on the particular format returned by the `format`
+  Do not rely on the particular format returned by the `format*`
   functions in this module. They may be changed in future releases
   in order to better suit Elixir's tool chain. In other words,
   by using the functions in this module it is guaranteed you will
@@ -15,7 +15,11 @@ defmodule Exception do
   """
 
   @typedoc "The exception type"
-  @type t :: %{__struct__: module, __exception__: true}
+  @type t :: %{
+    required(:__struct__) => module,
+    required(:__exception__) => true,
+    atom => any
+  }
 
   @typedoc "The kind handled by formatting functions"
   @type kind :: :error | :exit | :throw | {:EXIT, pid}
@@ -236,7 +240,7 @@ defmodule Exception do
   defp format_exit_reason(:noconnection), do: "no connection"
 
   defp format_exit_reason(:noproc) do
-    "no process"
+    "no process: the process is not alive or there's no process currently associated with the given name, possibly because its application isn't started"
   end
 
   defp format_exit_reason({:nodedown, node_name}) when is_atom(node_name) do
@@ -380,7 +384,7 @@ defmodule Exception do
   is retrieved from `Process.info/2`.
   """
   def format_stacktrace(trace \\ nil) do
-    trace = trace || case Process.info(self, :current_stacktrace) do
+    trace = trace || case Process.info(self(), :current_stacktrace) do
       {:current_stacktrace, t} -> Enum.drop(t, 3)
     end
 
@@ -507,7 +511,7 @@ defmodule SystemLimitError do
 end
 
 defmodule SyntaxError do
-  defexception [file: nil, line: nil, description: "syntax error"]
+  defexception [:file, :line, description: "syntax error"]
 
   def message(exception) do
     Exception.format_file_line(Path.relative_to_cwd(exception.file), exception.line) <>
@@ -516,7 +520,7 @@ defmodule SyntaxError do
 end
 
 defmodule TokenMissingError do
-  defexception [file: nil, line: nil, description: "expression is incomplete"]
+  defexception [:file, :line, description: "expression is incomplete"]
 
   def message(%{file: file, line: line, description: description}) do
     Exception.format_file_line(file && Path.relative_to_cwd(file), line) <>
@@ -525,7 +529,7 @@ defmodule TokenMissingError do
 end
 
 defmodule CompileError do
-  defexception [file: nil, line: nil, description: "compile error"]
+  defexception [:file, :line, description: "compile error"]
 
   def message(%{file: file, line: line, description: description}) do
     Exception.format_file_line(file && Path.relative_to_cwd(file), line) <>
@@ -534,7 +538,7 @@ defmodule CompileError do
 end
 
 defmodule BadFunctionError do
-  defexception [term: nil]
+  defexception [:term]
 
   def message(exception) do
     "expected a function, got: #{inspect(exception.term)}"
@@ -542,7 +546,7 @@ defmodule BadFunctionError do
 end
 
 defmodule BadStructError do
-  defexception [struct: nil, term: nil]
+  defexception [:struct, :term]
 
   def message(exception) do
     "expected a struct named #{inspect(exception.struct)}, got: #{inspect(exception.term)}"
@@ -550,15 +554,23 @@ defmodule BadStructError do
 end
 
 defmodule BadMapError do
-  defexception [term: nil]
+  defexception [:term]
 
   def message(exception) do
     "expected a map, got: #{inspect(exception.term)}"
   end
 end
 
+defmodule BadBooleanError do
+  defexception [:term, :operator]
+
+  def message(exception) do
+    "expected a boolean on left-side of \"#{exception.operator}\", got: #{inspect(exception.term)}"
+  end
+end
+
 defmodule MatchError do
-  defexception [term: nil]
+  defexception [:term]
 
   def message(exception) do
     "no match of right hand side value: #{inspect(exception.term)}"
@@ -566,7 +578,7 @@ defmodule MatchError do
 end
 
 defmodule CaseClauseError do
-  defexception [term: nil]
+  defexception [:term]
 
   def message(exception) do
     "no case clause matching: #{inspect(exception.term)}"
@@ -574,7 +586,7 @@ defmodule CaseClauseError do
 end
 
 defmodule WithClauseError do
-  defexception [term: nil]
+  defexception [:term]
 
   def message(exception) do
     "no with clause matching: #{inspect(exception.term)}"
@@ -590,7 +602,7 @@ defmodule CondClauseError do
 end
 
 defmodule TryClauseError do
-  defexception [term: nil]
+  defexception [:term]
 
   def message(exception) do
     "no try clause matching: #{inspect(exception.term)}"
@@ -598,7 +610,7 @@ defmodule TryClauseError do
 end
 
 defmodule BadArityError do
-  defexception [function: nil, args: nil]
+  defexception [:function, :args]
 
   def message(exception) do
     fun  = exception.function
@@ -614,7 +626,7 @@ defmodule BadArityError do
 end
 
 defmodule UndefinedFunctionError do
-  defexception [module: nil, function: nil, arity: nil, reason: nil]
+  defexception [:module, :function, :arity, :reason, :exports]
 
   def message(%{reason: nil, module: module, function: function, arity: arity} = e) do
     cond do
@@ -632,9 +644,19 @@ defmodule UndefinedFunctionError do
       " is undefined (module #{inspect module} is not available)"
   end
 
-  def message(%{reason: :"function not exported",  module: module, function: function, arity: arity}) do
-    "function " <> Exception.format_mfa(module, function, arity) <>
-    " is undefined or private" <> did_you_mean(module, function, arity)
+  def message(%{reason: :"function not exported",  module: module, function: function, arity: arity, exports: exports}) do
+    suffix =
+      if macro_exported?(module, function, arity) do
+        ". However there is a macro with the same name and arity." <>
+          " Be sure to require #{inspect(module)} if you intend to invoke this macro"
+      else
+        did_you_mean(module, function, arity, exports)
+      end
+
+    "function " <>
+      Exception.format_mfa(module, function, arity) <>
+      " is undefined or private" <>
+      suffix
   end
 
   def message(%{reason: :"function not available", module: module, function: function, arity: arity}) do
@@ -650,8 +672,8 @@ defmodule UndefinedFunctionError do
   @function_threshold 0.77
   @max_suggestions 5
 
-  defp did_you_mean(module, function, _arity) do
-    exports = exports_for(module)
+  defp did_you_mean(module, function, _arity, exports) do
+    exports = exports || exports_for(module)
 
     result =
       case Keyword.take(exports, [function]) do
@@ -692,7 +714,7 @@ defmodule UndefinedFunctionError do
 end
 
 defmodule FunctionClauseError do
-  defexception [module: nil, function: nil, arity: nil]
+  defexception [:module, :function, :arity]
 
   def message(exception) do
     if exception.function do
@@ -714,7 +736,7 @@ defmodule Code.LoadError do
 end
 
 defmodule Protocol.UndefinedError do
-  defexception [protocol: nil, value: nil, description: ""]
+  defexception [:protocol, :value, description: ""]
 
   def message(exception) do
     msg = "protocol #{inspect exception.protocol} not implemented for #{inspect exception.value}"
@@ -726,7 +748,7 @@ defmodule Protocol.UndefinedError do
 end
 
 defmodule KeyError do
-  defexception key: nil, term: nil
+  defexception [:key, :term]
 
   def message(exception) do
     msg = "key #{inspect exception.key} not found"
@@ -770,16 +792,23 @@ defmodule Enum.EmptyError do
 end
 
 defmodule File.Error do
-  defexception [reason: nil, action: "", path: nil]
+  defexception [:reason, :path, action: ""]
 
-  def message(exception) do
-    formatted = IO.iodata_to_binary(:file.format_error(exception.reason))
-    "could not #{exception.action} #{inspect(exception.path)}: #{formatted}"
+  def message(%{action: action, reason: reason, path: path}) do
+    formatted =
+      case {action, reason} do
+        {"remove directory", :eexist} ->
+          "directory is not empty"
+        _ ->
+          IO.iodata_to_binary(:file.format_error(reason))
+      end
+
+    "could not #{action} #{inspect(path)}: #{formatted}"
   end
 end
 
 defmodule File.CopyError do
-  defexception [reason: nil, source: nil, destination: nil, on: "", action: ""]
+  defexception [:reason, :source, :destination, on: "", action: ""]
 
   def message(exception) do
     formatted =
@@ -797,7 +826,7 @@ defmodule File.CopyError do
 end
 
 defmodule ErlangError do
-  defexception [original: nil]
+  defexception [:original]
 
   def message(exception) do
     "erlang error: #{inspect(exception.original)}"
@@ -840,6 +869,10 @@ defmodule ErlangError do
     %BadMapError{term: term}
   end
 
+  def normalize({:badbool, op, term}, _stacktrace) do
+    %BadBooleanError{operator: op, term: term}
+  end
+
   def normalize({:badkey, key}, stacktrace) do
     term =
       case stacktrace || :erlang.get_stacktrace do

lib/elixir/lib/file.ex

@@ -135,7 +135,7 @@ defmodule File do
       directories of `path`
     * `:eexist`  - there is already a file or directory named `path`
     * `:enoent`  - a component of `path` does not exist
-    * `:enospc`  - there is a no space left on the device
+    * `:enospc`  - there is no space left on the device
     * `:enotdir` - a component of `path` is not a directory;
       on some platforms, `:enoent` is returned instead
   """
@@ -165,7 +165,7 @@ defmodule File do
 
     * `:eacces`  - missing search or write permissions for the parent
       directories of `path`
-    * `:enospc`  - there is a no space left on the device
+    * `:enospc`  - there is no space left on the device
     * `:enotdir` - a component of `path` is not a directory
   """
   @spec mkdir_p(Path.t) :: :ok | {:error, posix}
@@ -456,7 +456,7 @@ defmodule File do
   specify the `destination` filename, it is not sufficient to simply specify
   its directory.
 
-  It returns `:ok` in case of success, returns `{:error, reason}` otherwise.
+  Returns `:ok` in case of success, `{:error, reason}` otherwise.
 
   Note: The command `mv` in Unix systems behaves differently depending
   if `source` is a file and the `destination` is an existing directory.
@@ -680,6 +680,9 @@ defmodule File do
   contents are overwritten. Returns `:ok` if successful, or `{:error, reason}`
   if an error occurs.
 
+  `content` must be `iodata` (a list of bytes or a binary). Setting the
+  encoding for this function has no effect.
+
   **Warning:** Every time this function is invoked, a file descriptor is opened
   and a new process is spawned to write to the file. For this reason, if you are
   doing multiple writes in a loop, opening the file via `File.open/2` and using
@@ -691,7 +694,7 @@ defmodule File do
     * `:enoent`  - a component of the file name does not exist
     * `:enotdir` - a component of the file name is not a directory;
       on some platforms, `:enoent` is returned instead
-    * `:enospc`  - there is a no space left on the device
+    * `:enospc`  - there is no space left on the device
     * `:eacces`  - missing permission for writing the file or searching one of
       the parent directories
     * `:eisdir`  - the named file is a directory
@@ -922,7 +925,7 @@ defmodule File do
   end
 
   @doc ~S"""
-  Opens the given `path` according to the given list of `modes`.
+  Opens the given `path`.
 
   In order to write and read files, one must use the functions
   in the `IO` module. By default, a file is opened in `:binary` mode,
@@ -931,6 +934,12 @@ defmodule File do
   option when opening the file and then all other functions from
   `IO` are available, since they work directly with Unicode data.
 
+  `modes_or_function` can either be a list of modes or a function. If it's a
+  list, it's considered to be a list of modes (that are documented below). If
+  it's a function, then it's equivalent to calling `open(path, [],
+  modes_or_function)`. See the documentation for `open/3` for more information
+  on this function.
+
   The allowed modes:
 
     * `:binary` - opens the file in binary mode, disabling special handling of unicode sequences
@@ -977,7 +986,7 @@ defmodule File do
 
     * `{:ok, io_device}` - the file has been opened in the requested mode.
 
-      `io_device` is actually the pid of the process which handles the file.
+      `io_device` is actually the PID of the process which handles the file.
       This process is linked to the process which originally opened the file.
       If any process to which the `io_device` is linked terminates, the file
       will be closed and the process itself will be terminated.
@@ -996,13 +1005,13 @@ defmodule File do
   """
   @spec open(Path.t, [mode | :ram]) :: {:ok, io_device} | {:error, posix}
   @spec open(Path.t, (io_device -> res)) :: {:ok, res} | {:error, posix} when res: var
-  def open(path, modes \\ [])
+  def open(path, modes_or_function \\ [])
 
   def open(path, modes) when is_list(modes) do
     F.open(IO.chardata_to_string(path), normalize_modes(modes, true))
   end
 
-  def open(path, function) when is_function(function) do
+  def open(path, function) when is_function(function, 1) do
     open(path, [], function)
   end
 
@@ -1013,7 +1022,7 @@ defmodule File do
   automatically closed after the function returns, regardless
   if there was an error when executing the function.
 
-  It returns `{:ok, function_result}` in case of success,
+  Returns `{:ok, function_result}` in case of success,
   `{:error, reason}` otherwise.
 
   This function expects the file to be closed with success,
@@ -1027,43 +1036,51 @@ defmodule File do
         IO.read(file, :line)
       end)
 
+  See `open/2` for the list of available `modes`.
   """
   @spec open(Path.t, [mode | :ram], (io_device -> res)) :: {:ok, res} | {:error, posix} when res: var
-  def open(path, modes, function) do
+  def open(path, modes, function) when is_list(modes) and is_function(function, 1) do
     case open(path, modes) do
-      {:ok, device} ->
+      {:ok, io_device} ->
         try do
-          {:ok, function.(device)}
+          {:ok, function.(io_device)}
         after
-          :ok = close(device)
+          :ok = close(io_device)
         end
       other -> other
     end
   end
 
   @doc """
-  Same as `open/2` but raises an error if file could not be opened.
+  Similar to `open/2` but raises an error if file could not be opened.
 
-  Returns the `io_device` otherwise.
+  Returns the IO device otherwise.
+
+  See `open/2` for the list of available modes.
   """
-  @spec open!(Path.t, [mode]) :: io_device | no_return
-  def open!(path, modes \\ []) do
-    case open(path, modes) do
-      {:ok, device}    -> device
+  @spec open!(Path.t, [mode | :ram]) :: io_device | no_return
+  @spec open!(Path.t, (io_device -> res)) :: res | no_return when res: var
+  def open!(path, modes_or_function \\ []) do
+    case open(path, modes_or_function) do
+      {:ok, io_device_or_function_result} ->
+        io_device_or_function_result
       {:error, reason} ->
         raise File.Error, reason: reason, action: "open", path: IO.chardata_to_string(path)
     end
   end
 
   @doc """
-  Same as `open/3` but raises an error if file could not be opened.
+  Similar to `open/3` but raises an error if file could not be opened.
 
-  Returns the function result otherwise.
+  If it succeeds opening the file, it returns the `function` result on the IO device.
+
+  See `open/2` for the list of available `modes`.
   """
   @spec open!(Path.t, [mode | :ram], (io_device -> res)) :: res | no_return when res: var
   def open!(path, modes, function) do
     case open(path, modes, function) do
-      {:ok, device}    -> device
+      {:ok, function_result} ->
+        function_result
       {:error, reason} ->
         raise File.Error, reason: reason, action: "open", path: IO.chardata_to_string(path)
     end
@@ -1139,7 +1156,7 @@ defmodule File do
   """
   @spec cd!(Path.t, (() -> res)) :: res | no_return when res: var
   def cd!(path, function) do
-    old = cwd!
+    old = cwd!()
     cd!(path)
     try do
       function.()
@@ -1151,7 +1168,7 @@ defmodule File do
   @doc """
   Returns the list of files in the given directory.
 
-  It returns `{:ok, [files]}` in case of success,
+  Returns `{:ok, [files]}` in case of success,
   `{:error, reason}` otherwise.
   """
   @spec ls(Path.t) :: {:ok, [binary]} | {:error, posix}
@@ -1182,7 +1199,7 @@ defmodule File do
 
   Note that if the option `:delayed_write` was used when opening the file,
   `close/1` might return an old write error and not even try to close the file.
-  See `open/2`.
+  See `open/2` for more information.
   """
   @spec close(io_device) :: :ok | {:error, posix | :badarg | :terminated}
   def close(io_device) do
@@ -1195,7 +1212,7 @@ defmodule File do
   The stream implements both `Enumerable` and `Collectable` protocols,
   which means it can be used both for read and write.
 
-  The `line_or_byte` argument configures how the file is read when
+  The `line_or_bytes` argument configures how the file is read when
   streaming, by `:line` (default) or by a given number of bytes.
 
   Operating the stream can fail on open for the same reasons as
@@ -1210,7 +1227,7 @@ defmodule File do
   in raw mode for performance reasons. Therefore, Elixir **will** open
   streams in `:raw` mode with the `:read_ahead` option unless an encoding
   is specified. This means any data streamed into the file must be
-  converted to `iodata` type. If you pass `[:utf8]` in the modes parameter,
+  converted to `t:iodata/0` type. If you pass `[:utf8]` in the modes parameter,
   the underlying stream will use `IO.write/2` and the `String.Chars` protocol
   to convert the data. See `IO.binwrite/2` and `IO.write/2` .
 
@@ -1221,7 +1238,7 @@ defmodule File do
 
       # Read in 2048 byte chunks rather than lines
       File.stream!("./test/test.data", [], 2048)
-      #=>  %File.Stream{line_or_bytes: 2048, modes: [:raw, :read_ahead, :binary],
+      #=> %File.Stream{line_or_bytes: 2048, modes: [:raw, :read_ahead, :binary],
       #=> path: "./test/test.data", raw: true}
 
   See `Stream.run/1` for an example of streaming into a file.

lib/elixir/lib/float.ex

@@ -5,6 +5,10 @@ defmodule Float do
   Functions for working with floating point numbers.
   """
 
+  import Bitwise
+
+  @power_of_2_to_52 4503599627370496
+
   @doc """
   Parses a binary into a float.
 
@@ -22,10 +26,8 @@ defmodule Float do
 
       iex> Float.parse("34")
       {34.0, ""}
-
       iex> Float.parse("34.25")
       {34.25, ""}
-
       iex> Float.parse("56.5xyz")
       {56.5, "xyz"}
 
@@ -78,6 +80,18 @@ defmodule Float do
 
   `floor/2` also accepts a precision to round a floating point value down
   to an arbitrary number of fractional digits (between 0 and 15).
+  The operation is performed on the binary floating point, without a
+  conversion to decimal.
+
+  The behaviour of `floor/2` for floats can be surprising. For example:
+
+      iex> Float.floor(12.52, 2)
+      12.51
+
+  One may have expected it to floor to 12.52. This is not a bug.
+  Most decimal fractions cannot be represented as a binary floating point
+  and therefore the number above is internally represented as 12.51999999,
+  which explains the behaviour above.
 
   This function always returns a float. `Kernel.trunc/1` may be used instead to
   truncate the result to an integer afterwards.
@@ -86,21 +100,15 @@ defmodule Float do
 
       iex> Float.floor(34.25)
       34.0
-
       iex> Float.floor(-56.5)
       -57.0
-
       iex> Float.floor(34.259, 2)
       34.25
 
   """
   @spec floor(float, 0..15) :: float
   def floor(number, precision \\ 0) when is_float(number) and precision in 0..15 do
-    power     = power_of_10(precision)
-    number    = number * power
-    truncated = trunc(number)
-    variance  = if number - truncated < 0, do: -1.0, else: 0.0
-    (truncated + variance) / power
+    round(number, precision, :floor)
   end
 
   @doc """
@@ -109,6 +117,19 @@ defmodule Float do
   `ceil/2` also accepts a precision to round a floating point value down
   to an arbitrary number of fractional digits (between 0 and 15).
 
+  The operation is performed on the binary floating point, without a
+  conversion to decimal.
+
+  The behaviour of `ceil/2` for floats can be surprising. For example:
+
+      iex> Float.ceil(-12.52, 2)
+      -12.51
+
+  One may have expected it to ceil to -12.52. This is not a bug.
+  Most decimal fractions cannot be represented as a binary floating point
+  and therefore the number above is internally represented as -12.51999999,
+  which explains the behaviour above.
+
   This function always returns floats. `Kernel.trunc/1` may be used instead to
   truncate the result to an integer afterwards.
 
@@ -116,60 +137,228 @@ defmodule Float do
 
       iex> Float.ceil(34.25)
       35.0
-
       iex> Float.ceil(-56.5)
       -56.0
-
       iex> Float.ceil(34.251, 2)
       34.26
 
   """
   @spec ceil(float, 0..15) :: float
   def ceil(number, precision \\ 0) when is_float(number) and precision in 0..15 do
-    power     = power_of_10(precision)
-    number    = number * power
-    truncated = trunc(number)
-    variance  = if number - truncated > 0, do: 1.0, else: 0.0
-    (truncated + variance) / power
+    round(number, precision, :ceil)
   end
 
   @doc """
-  Rounds a floating point value to an arbitrary number of fractional digits
-  (between 0 and 15).
+  Rounds a floating point value to an arbitrary number of fractional
+  digits (between 0 and 15).
+
+  The rounding direction always ties to half up. The operation is
+  performed on the binary floating point, without a conversion to decimal.
 
   This function only accepts floats and always returns a float. Use
-  `Kernel.round/1` if you want a function that accepts both floats and integers
-  and always returns an integer.
+  `Kernel.round/1` if you want a function that accepts both floats
+  and integers and always returns an integer.
+
+  The behaviour of `round/2` for floats can be surprising. For example:
+
+      iex> Float.round(5.5675, 3)
+      5.567
+
+  One may have expected it to round to the half up 5.568. This is not a bug.
+  Most decimal fractions cannot be represented as a binary floating point
+  and therefore the number above is internally represented as 5.567499999,
+  which explains the behaviour above. If you want exact rounding for decimals,
+  you must use a decimal library. The behaviour above is also in accordance
+  to reference implementations, such as "Correctly Rounded Binary-Decimal and
+  Decimal-Binary Conversions" by David M. Gay.
 
   ## Examples
 
+      iex> Float.round(12.5)
+      13.0
       iex> Float.round(5.5674, 3)
       5.567
-
       iex> Float.round(5.5675, 3)
-      5.568
-
+      5.567
       iex> Float.round(-5.5674, 3)
       -5.567
-
-      iex> Float.round(-5.5675, 3)
-      -5.568
-
       iex> Float.round(-5.5675)
       -6.0
+      iex> Float.round(12.341444444444441, 15)
+      12.341444444444441
 
   """
   @spec round(float, 0..15) :: float
-  def round(number, precision \\ 0) when is_float(number) and precision in 0..15 do
-    power = power_of_10(precision)
-    Kernel.round(number * power) / power
+
+  # This implementation is slow since it relies on big integers.
+  # Faster implementations are available on more recent papers
+  # and could be implemented in the future.
+  def round(float, precision \\ 0) when is_float(float) and precision in 0..15 do
+    round(float, precision, :half_up)
   end
 
-  Enum.reduce 0..15, 1, fn x, acc ->
+  defp round(float, precision, rounding) do
+    <<sign::1, exp::11, significant::52-bitstring>> = <<float::float>>
+    {num, count, _} = decompose(significant)
+    count = count - exp + 1023
+
+    cond do
+      count <= 0 or  # There is no decimal precision
+      (0 == exp and <<0::52>> == significant) -> #zero or minus zero
+        float
+
+      count >= 104 -> # Precision beyond 15 digits
+        case rounding do
+          :ceil when sign === 0 -> 1 / power_of_10(precision)
+          :floor when sign === 1 -> -1 / power_of_10(precision)
+          _ -> 0.0
+        end
+
+      count <= precision -> # We are asking more precision than we have
+        float
+
+      true ->
+        # Difference in precision between float and asked precision
+        # We subtract 1 because we need to calculate the remainder too
+        diff = count - precision - 1
+
+        # Get up to latest so we calculate the remainder
+        power_of_10 = power_of_10(diff)
+
+        # Convert the numerand to decimal base
+        num = num * power_of_5(count)
+
+        # Move to the given precision - 1
+        num = div(num, power_of_10)
+        div = div(num, 10)
+        num = rounding(rounding, sign, num, div)
+
+        # Convert back to float without loss
+        # http://www.exploringbinary.com/correct-decimal-to-floating-point-using-big-integers/
+        den = power_of_10(precision)
+        boundary = den <<< 52
+
+        cond do
+          num == 0 ->
+            0.0
+          num >= boundary ->
+            {den, exp} = scale_down(num, boundary, 52)
+            decimal_to_float(sign, num, den, exp)
+          true ->
+            {num, exp} = scale_up(num, boundary, 52)
+            decimal_to_float(sign, num, den, exp)
+        end
+    end
+  end
+
+  defp scale_up(num, boundary, exp) when num >= boundary, do: {num, exp}
+  defp scale_up(num, boundary, exp), do: scale_up(num <<< 1, boundary, exp - 1)
+
+  defp scale_down(num, den, exp) do
+    new_den = den <<< 1
+    if num < new_den do
+      {den >>> 52, exp}
+    else
+      scale_down(num, new_den, exp + 1)
+    end
+  end
+
+  defp decimal_to_float(sign, num, den, exp) do
+    quo = div(num, den)
+    rem = num - quo * den
+
+    tmp =
+      case den >>> 1 do
+        den when rem > den -> quo + 1
+        den when rem < den -> quo
+        _ when (quo &&& 1) === 1 -> quo + 1
+        _ -> quo
+      end
+
+    tmp = tmp - @power_of_2_to_52
+    <<tmp::float>> = <<sign::1, (exp + 1023)::11, tmp::52>>
+    tmp
+  end
+
+  defp rounding(:floor, 1, _num, div), do: div + 1
+  defp rounding(:ceil, 0, _num, div), do: div + 1
+  defp rounding(:half_up, _sign, num, div) do
+    case rem(num, 10) do
+      rem when rem < 5 -> div
+      rem when rem >= 5 -> div + 1
+    end
+  end
+  defp rounding(_, _, _, div), do: div
+
+  Enum.reduce 0..104, 1, fn x, acc ->
     defp power_of_10(unquote(x)), do: unquote(acc)
     acc * 10
   end
 
+  Enum.reduce 0..104, 1, fn x, acc ->
+    defp power_of_5(unquote(x)), do: unquote(acc)
+    acc * 5
+  end
+
+  @doc """
+  Returns a pair of integers whose ratio is exactly equal
+  to the original float and with a positive denominator.
+
+  ## Examples
+
+      iex> Float.ratio(3.14)
+      {7070651414971679, 2251799813685248}
+      iex> Float.ratio(-3.14)
+      {-7070651414971679, 2251799813685248}
+      iex> Float.ratio(1.5)
+      {3, 2}
+      iex> Float.ratio(-1.5)
+      {-3, 2}
+      iex> Float.ratio(16.0)
+      {16, 1}
+      iex> Float.ratio(-16.0)
+      {-16, 1}
+
+  """
+  def ratio(float) when is_float(float) do
+    <<sign::1, exp::11, significant::52-bitstring>> = <<float::float>>
+    {num, _, den} = decompose(significant)
+    num = sign(sign, num)
+    case exp - 1023 do
+      exp when exp > 0 ->
+        {den, exp} = shift_right(den, exp)
+        {shift_left(num, exp), den}
+      exp when exp < 0 ->
+        {num, shift_left(den, -exp)}
+      0 ->
+        {num, den}
+    end
+  end
+
+  defp decompose(significant) do
+    decompose(significant, 1, 0, 2, 1, 1)
+  end
+
+  defp decompose(<<1::1, bits::bitstring>>, count, last_count, power, _last_power, acc) do
+    decompose(bits, count + 1, count, power <<< 1, power, shift_left(acc, count - last_count) + 1)
+  end
+  defp decompose(<<0::1, bits::bitstring>>, count, last_count, power, last_power, acc) do
+    decompose(bits, count + 1, last_count, power <<< 1, last_power, acc)
+  end
+  defp decompose(<<>>, _count, last_count, _power, last_power, acc) do
+    {acc, last_count, last_power}
+  end
+
+  defp sign(0, num), do: num
+  defp sign(1, num), do: -num
+
+  defp shift_left(num, 0), do: num
+  defp shift_left(num, times), do: shift_left(num <<< 1, times - 1)
+
+  defp shift_right(num, 0), do: {num, 0}
+  defp shift_right(1, times), do: {1, times}
+  defp shift_right(num, times), do: shift_right(num >>> 1, times - 1)
+
   @doc """
   Returns a charlist which corresponds to the text representation
   of the given float.
@@ -214,15 +403,15 @@ defmodule Float do
   @doc false
   def to_char_list(float), do: Float.to_charlist(float)
 
-  # TODO: Deprecate by v1.4
   @doc false
   def to_char_list(float, options) do
+    IO.warn "Float.to_char_list/2 is deprecated, use :erlang.float_to_list/2 instead"
     :erlang.float_to_list(float, expand_compact(options))
   end
 
-  # TODO: Deprecate by v1.4
   @doc false
   def to_string(float, options) do
+    IO.warn "Float.to_string/2 is deprecated, use :erlang.float_to_binary/2 instead"
     :erlang.float_to_binary(float, expand_compact(options))
   end
 

lib/elixir/lib/gen_event.ex

@@ -1,4 +1,6 @@
 defmodule GenEvent do
+  # TODO: Deprecate by 1.5
+
   @moduledoc """
   A behaviour module for implementing event handling functionality.
 
@@ -60,7 +62,7 @@ defmodule GenEvent do
 
   We start a new event manager by calling `GenEvent.start_link/1`.
   Notifications can be sent to the event manager which will then
-  invoke `handle_event/2` for each registered handler.
+  invoke `c:handle_event/2` for each registered handler.
 
   We can add new handlers with `add_handler/3` and `add_mon_handler/3`.
   Calls can also be made to specific handlers by using `call/3`.
@@ -144,7 +146,7 @@ defmodule GenEvent do
 
   Returning `{:ok, state, :hibernate}` is similar to
   `{:ok, state}` except the `GenEvent` process is hibernated before continuing
-  its loop. See `handle_event/2` for more information on hibernation.
+  its loop. See `c:handle_event/2` for more information on hibernation.
 
   Returning `{:error, reason}` will cause `add_handler/3` to return
   `{:error, reason}` and the handler is not added to `GenEvent` loop.
@@ -175,7 +177,7 @@ defmodule GenEvent do
   beneficial.
 
   Returning `:remove_handler` removes the handler from the `GenEvent` loop and
-  calls `terminate/2` with reason `:remove_handler` and state `state`.
+  calls `c:terminate/2` with reason `:remove_handler` and state `state`.
   """
   @callback handle_event(event :: term, state :: term) ::
     {:ok, new_state} |
@@ -193,10 +195,10 @@ defmodule GenEvent do
 
   Returning `{:ok, reply, new_state, :hibernate}` is similar to
   `{:ok, reply, new_state}` except the process is hibernated. See
-  `handle_event/2` for more information on hibernation.
+  `c:handle_event/2` for more information on hibernation.
 
   Returning `{:remove_handler, reply}` sends `reply` as a response to the call,
-  removes the handler from the `GenEvent` loop and calls `terminate/2` with
+  removes the handler from the `GenEvent` loop and calls `c:terminate/2` with
   reason `:remove_handler` and state `state`.
   """
 
@@ -212,7 +214,7 @@ defmodule GenEvent do
 
   `msg` is the message and `state` is the current state of the handler.
 
-  Return values are the same as `handle_event/2`.
+  Return values are the same as `c:handle_event/2`.
   """
   @callback handle_info(msg :: term, state :: term) ::
     {:ok, new_state} |
@@ -238,23 +240,23 @@ defmodule GenEvent do
   If part of a supervision tree, a `GenEvent`'s `Supervisor` will send an exit
   signal when shutting it down. The exit signal is based on the shutdown
   strategy in the child's specification. If it is `:brutal_kill` the `GenEvent`
-  is killed and so `terminate/2` is not called for its handlers. However if it is
+  is killed and so `c:terminate/2` is not called for its handlers. However if it is
   a timeout the `Supervisor` will send the exit signal `:shutdown` and the
-  `GenEvent` will have the duration of the timeout to call `terminate/2` on all
+  `GenEvent` will have the duration of the timeout to call `c:terminate/2` on all
   of its handlers - if the process is still alive after the timeout it is
   killed.
 
   If the `GenEvent` receives an exit signal (that is not `:normal`) from any
   process when it is not trapping exits it will exit abruptly with the same
-  reason and so not call the handlers' `terminate/2`. Note that a process does
+  reason and so not call the handlers' `c:terminate/2`. Note that a process does
   *NOT* trap exits by default and an exit signal is sent when a linked process
   exits or its node is disconnected. Therefore it is not guaranteed that
-  `terminate/2` is called when a `GenEvent` exits.
+  `c:terminate/2` is called when a `GenEvent` exits.
 
   Care should be taken to cleanup because the `GenEvent` can continue to loop
   after removing the handler. This is different to most other OTP behaviours.
   For example if the handler controls a `port` (e.g. `:gen_tcp.socket`) or
-  `File.io_device`, it will be need to be closed in `terminate/2` as the
+  `t:File.io_device/0`, it will be need to be closed in `c:terminate/2` as the
   process is not exiting so will not be automatically cleaned up.
   """
   @callback terminate(reason, state :: term) ::
@@ -273,7 +275,7 @@ defmodule GenEvent do
   Returning `{:ok, new_state}` changes the state to `new_state` and the code
   change is successful.
 
-  If `code_change/3` raises, the code change fails and the handler will continue
+  If `c:code_change/3` raises, the code change fails and the handler will continue
   with its previous state. Therefore this callback does not usually contain side
   effects.
   """
@@ -312,11 +314,16 @@ defmodule GenEvent do
 
       @doc false
       def handle_call(msg, state) do
+        proc =
+          case Process.info(self(), :registered_name) do
+            {_, []}   -> self()
+            {_, name} -> name
+          end
+
         # We do this to trick Dialyzer to not complain about non-local returns.
-        reason = {:bad_call, msg}
         case :erlang.phash2(1, 1) do
-          0 -> exit(reason)
-          1 -> {:remove_handler, reason}
+          0 -> raise "attempted to call GenEvent #{inspect proc} but no handle_call/2 clause was provided"
+          1 -> {:remove_handler, {:bad_call, msg}}
         end
       end
 
@@ -349,9 +356,9 @@ defmodule GenEvent do
   section in the `GenServer` module docs.
 
   If the event manager is successfully created and initialized, the function
-  returns `{:ok, pid}`, where pid is the pid of the server. If a process with
+  returns `{:ok, pid}`, where `pid` is the PID of the server. If a process with
   the specified server name already exists, the function returns
-  `{:error, {:already_started, pid}}` with the pid of that process.
+  `{:error, {:already_started, pid}}` with the PID of that process.
 
   Note that a `GenEvent` started with `start_link/1` is linked to the
   parent process and will exit not only on crashes but also if the parent
@@ -380,8 +387,21 @@ defmodule GenEvent do
         :gen.start(GenEvent, mode, @no_callback, [], [])
       atom when is_atom(atom) ->
         :gen.start(GenEvent, mode, {:local, atom}, @no_callback, [], [])
-      other when is_tuple(other) ->
-        :gen.start(GenEvent, mode, other, @no_callback, [], [])
+      {:global, _term} = tuple ->
+        :gen.start(GenEvent, mode, tuple, @no_callback, [], [])
+      {:via, via_module, _term} = tuple when is_atom(via_module) ->
+        :gen.start(GenEvent, mode, tuple, @no_callback, [], [])
+      other ->
+        raise ArgumentError, """
+        expected :name option to be one of:
+
+          * nil
+          * atom
+          * {:global, term}
+          * {:via, module, term}
+
+        Got: #{inspect(other)}
+        """
     end
   end
 
@@ -410,10 +430,10 @@ defmodule GenEvent do
   @doc """
   Adds a new event handler to the event `manager`.
 
-  The event manager will call the `init/1` callback with `args` to
+  The event manager will call the `c:init/1` callback with `args` to
   initiate the event handler and its internal state.
 
-  If `init/1` returns a correct value indicating successful completion,
+  If `c:init/1` returns a correct value indicating successful completion,
   the event manager adds the event handler and this function returns
   `:ok`. If the callback fails with `reason` or returns `{:error, reason}`,
   the event handler is ignored and this function returns `{:error, reason}`.
@@ -441,7 +461,7 @@ defmodule GenEvent do
   by the GenEvent manager.
 
   If the calling process later terminates with `reason`, the event manager
-  will delete the event handler by calling the `terminate/2` callback with
+  will delete the event handler by calling the `c:terminate/2` callback with
   `{:stop, reason}` as argument. If the event handler later is deleted,
   the event manager sends a message `{:gen_event_EXIT, handler, reason}`
   to the calling process. Reason is one of the following:
@@ -453,7 +473,7 @@ defmodule GenEvent do
     * `:shutdown` - if the event handler has been removed because the event
       manager is terminating
 
-    * `{:swapped, new_handler, pid}` - if the process pid has replaced the
+    * `{:swapped, new_handler, pid}` - if the process PID has replaced the
       event handler by another
 
     * `term` - if the event handler is removed due to an error. Which term
@@ -478,7 +498,7 @@ defmodule GenEvent do
   @doc """
   Sends an event notification to the event `manager`.
 
-  The event manager will call `handle_event/2` for each
+  The event manager will call `c:handle_event/2` for each
   installed event handler.
 
   `notify` is asynchronous and will return immediately after the
@@ -497,7 +517,7 @@ defmodule GenEvent do
     end
   end
 
-  def notify({:via, mod, name}, msg) do
+  def notify({:via, mod, name}, msg) when is_atom(mod) do
     try do
       mod.send(name, {:notify, msg})
       :ok
@@ -506,8 +526,12 @@ defmodule GenEvent do
     end
   end
 
-  def notify(other, msg) do
-    send(other, {:notify, msg})
+  def notify(manager, msg)
+      when is_pid(manager)
+      when is_atom(manager)
+      when tuple_size(manager) == 2 and
+        is_atom(elem(manager, 0)) and is_atom(elem(manager, 1)) do
+    send(manager, {:notify, msg})
     :ok
   end
 
@@ -515,7 +539,7 @@ defmodule GenEvent do
   Sends a sync event notification to the event `manager`.
 
   In other words, this function only returns `:ok` after the event manager
-  invokes the `handle_event/2` callback on each installed event handler.
+  invokes the `c:handle_event/2` callback on each installed event handler.
 
   See `notify/2` for more info.
   """
@@ -543,10 +567,10 @@ defmodule GenEvent do
   Makes a synchronous call to the event `handler` installed in `manager`.
 
   The given `request` is sent and the caller waits until a reply arrives or
-  a timeout occurs. The event manager will call `handle_call/2` to handle
+  a timeout occurs. The event manager will call `c:handle_call/2` to handle
   the request.
 
-  The return value `reply` is defined in the return value of `handle_call/2`.
+  The return value `reply` is defined in the return value of `c:handle_call/2`.
   If the specified event handler is not installed, the function returns
   `{:error, :not_found}`.
   """
@@ -565,7 +589,7 @@ defmodule GenEvent do
   @doc """
   Removes an event handler from the event `manager`.
 
-  The event manager will call `terminate/2` to terminate the event handler
+  The event manager will call `c:terminate/2` to terminate the event handler
   and return the callback value. If the specified event handler is not
   installed, the function returns `{:error, :not_found}`.
   """
@@ -577,17 +601,17 @@ defmodule GenEvent do
   @doc """
   Replaces an old event handler with a new one in the event `manager`.
 
-  First, the old event handler is deleted by calling `terminate/2` with
+  First, the old event handler is deleted by calling `c:terminate/2` with
   the given `args1` and collects the return value. Then the new event handler
   is added and initiated by calling `init({args2, term})`, where `term` is the
-  return value of calling `terminate/2` in the old handler. This makes it
+  return value of calling `c:terminate/2` in the old handler. This makes it
   possible to transfer information from one handler to another.
 
   The new handler will be added even if the specified old event handler
   is not installed or if the handler fails to terminate with a given reason
   in which case `state = {:error, term}`.
 
-  If `init/1` in the second handler returns a correct value, this
+  If `c:init/1` in the second handler returns a correct value, this
   function returns `:ok`.
   """
   @spec swap_handler(manager, handler, term, handler, term) :: :ok | {:error, term}
@@ -646,7 +670,7 @@ defmodule GenEvent do
     init_it(starter, self(), name, mod, args, options)
   end
 
-  def init_it(starter, parent, name, _, _, options) do
+  def init_it(starter, parent, name, _mod, _args, options) do
     Process.put(:"$initial_call", {__MODULE__, :init_it, 6})
     debug =
       if function_exported?(:gen, :debug_options, 2) do

lib/elixir/lib/gen_event/stream.ex

@@ -1,14 +1,5 @@
 defmodule GenEvent.Stream do
-  @moduledoc """
-  Defines a `GenEvent` stream.
-
-  This is a struct returned by `GenEvent.stream/2`. The struct is public and
-  contains the following fields:
-
-    * `:manager`  - the manager reference given to `GenEvent.stream/2`
-    * `:timeout`  - the timeout between events, defaults to `:infinity`
-
-  """
+  @moduledoc false
   defstruct manager: nil, timeout: :infinity
 
   @type t :: %__MODULE__{

lib/elixir/lib/gen_server.ex

@@ -53,8 +53,8 @@ defmodule GenServer do
   while **cast** messages do not.
 
   Every time you do a `GenServer.call/3`, the client will send a message
-  that must be handled by the `handle_call/3` callback in the GenServer.
-  A `cast/2` message must be handled by `handle_cast/2`.
+  that must be handled by the `c:handle_call/3` callback in the GenServer.
+  A `cast/2` message must be handled by `c:handle_cast/2`.
 
   ## Callbacks
 
@@ -79,10 +79,12 @@ defmodule GenServer do
       mechanism and name. The `:via` option expects a module that exports
       `register_name/2`, `unregister_name/1`, `whereis_name/1` and `send/2`.
       One such example is the `:global` module which uses these functions
-      for keeping the list of names of processes and their  associated pid's
-      that are available globally for a network of Erlang nodes.
+      for keeping the list of names of processes and their associated PIDs
+      that are available globally for a network of Elixir nodes. Elixir also
+      ships with a local, decentralized and scalable registry called `Registry`
+      for locally storing names that are generated dynamically.
 
-  For example, we could start and register our Stack server locally as follows:
+  For example, we could start and register our `Stack` server locally as follows:
 
       # Start the server and register it locally with name MyStack
       {:ok, _} = GenServer.start_link(Stack, [:hello], name: MyStack)
@@ -101,6 +103,11 @@ defmodule GenServer do
     * `{:via, module, name}` if the server is registered through an alternative
       registry
 
+  If there is an interest to register dynamic names locally, do not use
+  atoms, as atoms are never garbage collected and therefore dynamically
+  generated atoms won't be garbage collected. For such cases, you can
+  set up your own local registry by using the `Registry` module.
+
   ## Client / Server APIs
 
   Although in the example above we have used `GenServer.start_link/3` and
@@ -151,13 +158,109 @@ defmodule GenServer do
   the same module. If the server and/or client implementations are growing
   complex, you may want to have them in different modules.
 
-  ## Receiving custom messages
+  ## Receiving "regular" messages
 
   The goal of a `GenServer` is to abstract the "receive" loop for developers,
   automatically handling system messages, support code change, synchronous
   calls and more. Therefore, you should never call your own "receive" inside
   the GenServer callbacks as doing so will cause the GenServer to misbehave.
-  If you want to receive custom messages, always receive them in `handle_info/2`.
+
+  Besides the synchronous and asynchronous communication provided by `call/3`
+  and `cast/2`, "regular" messages sent by functions such `Kernel.send/2`,
+  `Process.send_after/4` and similar, can be handled inside the `c:handle_info/2`
+  callback.
+
+  `c:handle_info/2` can be used in many situations, such as handling monitor
+  DOWN messages sent by `Process.monitor/1`. Another use case for `c:handle_info/2`
+  is to perform periodic work, with the help of `Process.send_after/4`:
+
+      defmodule MyApp.Periodically do
+        use GenServer
+
+        def start_link do
+          GenServer.start_link(__MODULE__, %{})
+        end
+
+        def init(state) do
+          schedule_work() # Schedule work to be performed on start
+          {:ok, state}
+        end
+
+        def handle_info(:work, state) do
+          # Do the desired work here
+          schedule_work() # Reschedule once more
+          {:noreply, state}
+        end
+
+        defp schedule_work() do
+          Process.send_after(self(), :work, 2 * 60 * 60 * 1000) # In 2 hours
+        end
+      end
+
+  ## Debugging with the :sys module
+
+  GenServers, as [special processes](http://erlang.org/doc/design_principles/spec_proc.html),
+  can be debugged using the `:sys` module. Through various hooks, this module
+  allows developers to introspect the state of the process and trace
+  system events that happen during its execution, such as received messages,
+  sent replies and state changes.
+
+  Let's explore the basic functions from the `:sys` module used for debugging:
+
+    * [`:sys.get_state/2`](http://erlang.org/doc/man/sys.html#get_state-2) -
+      allows retrieval of the state of the process. In the case of
+      a GenServer process, it will be the callback module state, as
+      passed into the callback functions as last argument.
+    * [`:sys.get_status/2`](http://erlang.org/doc/man/sys.html#get_status-2) -
+      allows retrieval of the status of the process. This status includes
+      the process dictionary, if the process is running or is suspended,
+      the parent PID, the debugger state, and the state of the behaviour module,
+      which includes the callback module state (as returned by `:sys.get_state/2`).
+      It's possible to change how this status is represented by defining
+      the optional `c:GenServer.format_status/2` callback.
+    * [`:sys.trace/3`](http://erlang.org/doc/man/sys.html#trace-3) -
+      prints all the system events to `:stdio`.
+    * [`:sys.statistics/3`](http://erlang.org/doc/man/sys.html#statistics-3) -
+      manages collection of process statistics.
+    * [`:sys.no_debug/2`](http://erlang.org/doc/man/sys.html#no_debug-2) -
+      turns off all debug handlers for the given process. It is very important
+      to switch off debugging once we're done. Excessive debug handlers or
+      those that should be turned off, but weren't, can seriously damage
+      the performance of the system.
+
+  Let's see how we could use those functions for debugging the stack server
+  we defined earlier.
+
+      iex> {:ok, pid} = Stack.start_link([])
+      iex> :sys.statistics(pid, true) # turn on collecting process statistics
+      iex> :sys.trace(pid, true) # turn on event printing
+      iex> Stack.push(pid, 1)
+      *DBG* <0.122.0> got cast {push,1}
+      *DBG* <0.122.0> new state [1]
+      :ok
+      iex> :sys.get_state(pid)
+      [1]
+      iex> Stack.pop(pid)
+      *DBG* <0.122.0> got call pop from <0.80.0>
+      *DBG* <0.122.0> sent 1 to <0.80.0>, new state []
+      1
+      iex> :sys.statistics(pid, :get)
+      {:ok,
+       [start_time: {{2016, 7, 16}, {12, 29, 41}},
+        current_time: {{2016, 7, 16}, {12, 29, 50}},
+        reductions: 117, messages_in: 2, messages_out: 0]}
+      iex> :sys.no_debug(pid) # turn off all debug handlers
+      :ok
+      iex> :sys.get_status(pid)
+      {:status, #PID<0.122.0>, {:module, :gen_server},
+       [["$initial_call": {Stack, :init, 1},            # pdict
+         "$ancestors": [#PID<0.80.0>, #PID<0.51.0>]],
+        :running,                                       # :running | :suspended
+        #PID<0.80.0>,                                   # parent
+        [],                                             # debugger state
+        [header: 'Status for generic server <0.122.0>', # module status
+         data: [{'Status', :running}, {'Parent', #PID<0.80.0>},
+           {'Logged events', []}], data: [{'State', [1]}]]]}
 
   ## Learn more
 
@@ -172,7 +275,7 @@ defmodule GenServer do
   """
 
   @doc """
-  Invoked when the server is started. `start_link/3` (or `start/3`) will
+  Invoked when the server is started. `start_link/3` or `start/3` will
   block until it returns.
 
   `args` is the argument term (second argument) passed to `start_link/3`.
@@ -186,10 +289,10 @@ defmodule GenServer do
 
   Returning `{:ok, state, :hibernate}` is similar to
   `{:ok, state}` except the process is hibernated before entering the loop. See
-  `handle_call/3` for more information on hibernation.
+  `c:handle_call/3` for more information on hibernation.
 
   Returning `:ignore` will cause `start_link/3` to return `:ignore` and the
-  process will exit normally without entering the loop or calling `terminate/2`.
+  process will exit normally without entering the loop or calling `c:terminate/2`.
   If used when part of a supervision tree the parent supervisor will not fail
   to start nor immediately try to restart the `GenServer`. The remainder of the
   supervision tree will be (re)started and so the `GenServer` should not be
@@ -204,7 +307,7 @@ defmodule GenServer do
 
   Returning `{:stop, reason}` will cause `start_link/3` to return
   `{:error, reason}` and the process to exit with reason `reason` without
-  entering the loop or calling `terminate/2`.
+  entering the loop or calling `c:terminate/2`.
   """
   @callback init(args :: term) ::
     {:ok, state} |
@@ -217,7 +320,7 @@ defmodule GenServer do
   reply is received (unless the call times out or nodes are disconnected).
 
   `request` is the request message sent by a `call/3`, `from` is a 2-tuple
-  containing the caller's pid and a term that uniquely identifies the call, and
+  containing the caller's PID and a term that uniquely identifies the call, and
   `state` is the current state of the `GenServer`.
 
   Returning `{:reply, reply, new_state}` sends the response `reply` to the
@@ -259,7 +362,7 @@ defmodule GenServer do
   `{:noreply, new_state}` except a timeout or hibernation occurs as with a
   `:reply` tuple.
 
-  Returning `{:stop, reason, reply, new_state}` stops the loop and `terminate/2`
+  Returning `{:stop, reason, reply, new_state}` stops the loop and `c:terminate/2`
   is called with reason `reason` and state `new_state`. Then the `reply` is sent
   as the response to call and the process exits with reason `reason`.
 
@@ -291,9 +394,9 @@ defmodule GenServer do
 
   Returning `{:noreply, new_state, :hibernate}` is similar to
   `{:noreply, new_state}` except the process is hibernated before continuing the
-  loop. See `handle_call/3` for more information.
+  loop. See `c:handle_call/3` for more information.
 
-  Returning `{:stop, reason, new_state}` stops the loop and `terminate/2` is
+  Returning `{:stop, reason, new_state}` stops the loop and `c:terminate/2` is
   called with the reason `reason` and state `new_state`. The process exits with
   reason `reason`.
 
@@ -311,7 +414,7 @@ defmodule GenServer do
   `msg` is the message and `state` is the current state of the `GenServer`. When
   a timeout occurs the message is `:timeout`.
 
-  Return values are the same as `handle_cast/2`.
+  Return values are the same as `c:handle_cast/2`.
 
   If this callback is not implemented, the default implementation by
   `use GenServer` will return `{:noreply, state}`.
@@ -327,7 +430,7 @@ defmodule GenServer do
   `reason` is exit reason and `state` is the current state of the `GenServer`.
   The return value is ignored.
 
-  `terminate/2` is called if a callback (except `init/1`) returns a `:stop`
+  `c:terminate/2` is called if a callback (except `c:init/1`) returns a `:stop`
   tuple, raises, calls `Kernel.exit/1` or returns an invalid value. It may also
   be called if the `GenServer` traps exits using `Process.flag/2` *and* the
   parent process sends an exit signal.
@@ -335,23 +438,23 @@ defmodule GenServer do
   If part of a supervision tree a `GenServer`'s `Supervisor` will send an exit
   signal when shutting it down. The exit signal is based on the shutdown
   strategy in the child's specification. If it is `:brutal_kill` the `GenServer`
-  is killed and so `terminate/2` is not called. However if it is a timeout the
+  is killed and so `c:terminate/2` is not called. However if it is a timeout the
   `Supervisor` will send the exit signal `:shutdown` and the `GenServer` will
-  have the duration of the timeout to call `terminate/2` - if the process is
+  have the duration of the timeout to call `c:terminate/2` - if the process is
   still alive after the timeout it is killed.
 
   If the `GenServer` receives an exit signal (that is not `:normal`) from any
   process when it is not trapping exits it will exit abruptly with the same
-  reason and so not call `terminate/2`. Note that a process does *NOT* trap
+  reason and so not call `c:terminate/2`. Note that a process does *NOT* trap
   exits by default and an exit signal is sent when a linked process exits or its
   node is disconnected.
 
-  Therefore it is not guaranteed that `terminate/2` is called when a `GenServer`
+  Therefore it is not guaranteed that `c:terminate/2` is called when a `GenServer`
   exits. For such reasons, we usually recommend important clean-up rules to
   happen in separated processes either by use of monitoring or by links
   themselves. For example if the `GenServer` controls a `port` (e.g.
-  `:gen_tcp.socket`) or `File.io_device`, they will be closed on receiving a
-  `GenServer`'s exit signal and do not need to be closed in `terminate/2`.
+  `:gen_tcp.socket`) or `t:File.io_device/0`, they will be closed on receiving a
+  `GenServer`'s exit signal and do not need to be closed in `c:terminate/2`.
 
   If `reason` is not `:normal`, `:shutdown` nor `{:shutdown, term}` an error is
   logged.
@@ -375,7 +478,7 @@ defmodule GenServer do
   Returning `{:error, reason}` fails the code change with reason `reason` and
   the state remains as the previous state.
 
-  If `code_change/3` raises the code change fails and the loop will continue
+  If `c:code_change/3` raises the code change fails and the loop will continue
   with its previous state. Therefore this callback does not usually contain side effects.
   """
   @callback code_change(old_vsn, state :: term, extra :: term) ::
@@ -387,7 +490,7 @@ defmodule GenServer do
 
   This callback can be useful to control the *appearance* of the status of the
   `GenServer`. For example, it can be used to return a compact representation of
-  the `GenServers`'s state to avoid having large state terms printed.
+  the `GenServer`'s state to avoid having large state terms printed.
 
     * one of `:sys.get_status/1` or `:sys.get_status/2` is invoked to get the
       status of the `GenServer`; in such cases, `reason` is `:normal`
@@ -428,7 +531,7 @@ defmodule GenServer do
   @typedoc """
   Tuple describing the client of a call request.
 
-  `pid` is the pid of the caller and `tag` is a unique term used to identify the
+  `pid` is the PID of the caller and `tag` is a unique term used to identify the
   call.
   """
   @type from :: {pid, tag :: term}
@@ -445,26 +548,43 @@ defmodule GenServer do
 
       @doc false
       def handle_call(msg, _from, state) do
+        proc =
+          case Process.info(self(), :registered_name) do
+            {_, []}   -> self()
+            {_, name} -> name
+          end
+
         # We do this to trick Dialyzer to not complain about non-local returns.
-        reason = {:bad_call, msg}
         case :erlang.phash2(1, 1) do
-          0 -> exit(reason)
-          1 -> {:stop, reason, state}
+          0 -> raise "attempted to call GenServer #{inspect proc} but no handle_call/3 clause was provided"
+          1 -> {:stop, {:bad_call, msg}, state}
         end
       end
 
       @doc false
-      def handle_info(_msg, state) do
+      def handle_info(msg, state) do
+        proc =
+          case Process.info(self(), :registered_name) do
+            {_, []}   -> self()
+            {_, name} -> name
+          end
+        :error_logger.warning_msg('~p ~p received unexpected message in handle_info/2: ~p~n',
+                                  [__MODULE__, proc, msg])
         {:noreply, state}
       end
 
       @doc false
       def handle_cast(msg, state) do
+        proc =
+          case Process.info(self(), :registered_name) do
+            {_, []}   -> self()
+            {_, name} -> name
+          end
+
         # We do this to trick Dialyzer to not complain about non-local returns.
-        reason = {:bad_cast, msg}
         case :erlang.phash2(1, 1) do
-          0 -> exit(reason)
-          1 -> {:stop, reason, state}
+          0 -> raise "attempted to cast GenServer #{inspect proc} but no handle_cast/2 clause was provided"
+          1 -> {:stop, {:bad_cast, msg}, state}
         end
       end
 
@@ -488,15 +608,15 @@ defmodule GenServer do
 
   This is often used to start the `GenServer` as part of a supervision tree.
 
-  Once the server is started, the `init/1` function of the given `module` is
+  Once the server is started, the `c:init/1` function of the given `module` is
   called with `args` as its arguments to initialize the server. To ensure a
-  synchronized start-up procedure, this function does not return until `init/1`
+  synchronized start-up procedure, this function does not return until `c:init/1`
   has returned.
 
   Note that a `GenServer` started with `start_link/3` is linked to the
   parent process and will exit in case of crashes from the parent. The GenServer
   will also exit due to the `:normal` reasons in case it is configured to trap
-  exits in the `init/1` callback.
+  exits in the `c:init/1` callback.
 
   ## Options
 
@@ -516,11 +636,11 @@ defmodule GenServer do
   ## Return values
 
   If the server is successfully created and initialized, this function returns
-  `{:ok, pid}`, where `pid` is the pid of the server. If a process with the
+  `{:ok, pid}`, where `pid` is the PID of the server. If a process with the
   specified server name already exists, this function returns
-  `{:error, {:already_started, pid}}` with the pid of that process.
+  `{:error, {:already_started, pid}}` with the PID of that process.
 
-  If the `init/1` callback fails with `reason`, this function returns
+  If the `c:init/1` callback fails with `reason`, this function returns
   `{:error, reason}`. Otherwise, if it returns `{:stop, reason}`
   or `:ignore`, the process is terminated and this function returns
   `{:error, reason}` or `:ignore`, respectively.
@@ -546,15 +666,28 @@ defmodule GenServer do
         :gen.start(:gen_server, link, module, args, opts)
       {atom, opts} when is_atom(atom) ->
         :gen.start(:gen_server, link, {:local, atom}, module, args, opts)
-      {other, opts} when is_tuple(other) ->
-        :gen.start(:gen_server, link, other, module, args, opts)
+      {{:global, _term} = tuple, opts} ->
+        :gen.start(:gen_server, link, tuple, module, args, opts)
+      {{:via, via_module, _term} = tuple, opts} when is_atom(via_module) ->
+        :gen.start(:gen_server, link, tuple, module, args, opts)
+      other ->
+        raise ArgumentError, """
+        expected :name option to be one of:
+
+          * nil
+          * atom
+          * {:global, term}
+          * {:via, module, term}
+
+        Got: #{inspect(other)}
+        """
     end
   end
 
   @doc """
   Stops the server with the given `reason`.
 
-  The `terminate/2` callback of the given `server` will be invoked before
+  The `c:terminate/2` callback of the given `server` will be invoked before
   exiting. This function returns `:ok` if the server terminates with the
   given reason; if it terminates with another reason, the call exits.
 
@@ -571,7 +704,7 @@ defmodule GenServer do
   Makes a synchronous call to the `server` and waits for its reply.
 
   The client sends the given `request` to the server and waits until a reply
-  arrives or a timeout occurs. `handle_call/3` will be called on the server
+  arrives or a timeout occurs. `c:handle_call/3` will be called on the server
   to handle the request.
 
   `server` can be any of the values described in the "Name registration"
@@ -616,7 +749,7 @@ defmodule GenServer do
   is unknown whether the destination `server` successfully
   handled the message.
 
-  `handle_cast/2` will be called on the server to handle
+  `c:handle_cast/2` will be called on the server to handle
   the request. In case the `server` is on a node which is
   not yet connected to the caller one, the call is going to
   block until a connection happens. This is different than
@@ -660,7 +793,7 @@ defmodule GenServer do
   See `multi_call/4` for more information.
   """
   @spec abcast([node], name :: atom, term) :: :abcast
-  def abcast(nodes \\ nodes(), name, request) when is_list(nodes) and is_atom(name) do
+  def abcast(nodes \\ [node() | Node.list()], name, request) when is_list(nodes) and is_atom(name) do
     msg = cast_msg(request)
     _   = for node <- nodes, do: do_send({name, node}, msg)
     :abcast
@@ -710,19 +843,19 @@ defmodule GenServer do
   """
   @spec multi_call([node], name :: atom, term, timeout) ::
                   {replies :: [{node, term}], bad_nodes :: [node]}
-  def multi_call(nodes \\ nodes(), name, request, timeout \\ :infinity) do
+  def multi_call(nodes \\ [node() | Node.list()], name, request, timeout \\ :infinity) do
     :gen_server.multi_call(nodes, name, request, timeout)
   end
 
   @doc """
   Replies to a client.
 
-  This function can be used to explicitely send a reply to a client that called
+  This function can be used to explicitly send a reply to a client that called
   `call/3` or `multi_call/4` when the reply cannot be specified in the return
-  value of `handle_call/3`.
+  value of `c:handle_call/3`.
 
   `client` must be the `from` argument (the second argument) accepted by
-  `handle_call/3` callbacks. `reply` is an arbitrary term which will be given
+  `c:handle_call/3` callbacks. `reply` is an arbitrary term which will be given
   back to the client as the return value of the call.
 
   Note that `reply/2` can be called from any process, not just the GenServer
@@ -740,13 +873,14 @@ defmodule GenServer do
 
       def handle_info({:reply, from}, state) do
         GenServer.reply(from, :one_second_has_passed)
+        {:noreply, state}
       end
 
   """
   @spec reply(from, term) :: :ok
   def reply(client, reply)
 
-  def reply({to, tag}, reply) do
+  def reply({to, tag}, reply) when is_pid(to) do
     try do
       send(to, {tag, reply})
       :ok
@@ -791,10 +925,4 @@ defmodule GenServer do
   def whereis({name, node} = server) when is_atom(name) and is_atom(node) do
     server
   end
-
-  @compile {:inline, [nodes: 0]}
-
-  defp nodes do
-    [node() | :erlang.nodes()]
-  end
 end

lib/elixir/lib/inspect.ex

@@ -27,7 +27,7 @@ defprotocol Inspect do
         end
       end
 
-  The `concat` function comes from `Inspect.Algebra` and it
+  The `concat/1` function comes from `Inspect.Algebra` and it
   concatenates algebra documents together. In the example above,
   it is concatenating the string `"MapSet<"` (all strings are
   valid algebra documents that keep their formatting when pretty
@@ -61,10 +61,14 @@ end
 defimpl Inspect, for: Atom do
   require Macro
 
-  def inspect(atom, _opts) do
-    inspect(atom)
+  def inspect(atom, opts) do
+    color(inspect(atom), color_key(atom), opts)
   end
 
+  defp color_key(atom) when is_boolean(atom), do: :boolean
+  defp color_key(nil), do: :nil
+  defp color_key(_), do: :atom
+
   def inspect(false),  do: "false"
   def inspect(true),   do: "true"
   def inspect(nil),    do: "nil"
@@ -88,7 +92,7 @@ defimpl Inspect, for: Atom do
       atom in Macro.binary_ops or atom in Macro.unary_ops ->
         ":" <> binary
       true ->
-        <<?:, ?", Inspect.BitString.escape(binary, ?")::binary, ?">>
+        IO.iodata_to_binary [?:, ?", Inspect.BitString.escape(binary, ?"), ?"]
     end
   end
 
@@ -143,7 +147,8 @@ end
 defimpl Inspect, for: BitString do
   def inspect(term, %Inspect.Opts{binaries: bins, base: base} = opts) when is_binary(term) do
     if base == :decimal and (bins == :as_strings or (bins == :infer and String.printable?(term))) do
-      <<?", escape(term, ?")::binary, ?">>
+      inspected = IO.iodata_to_binary([?", escape(term, ?"), ?"])
+      color(inspected, :string, opts)
     else
       inspect_bitstring(term, opts)
     end
@@ -157,96 +162,92 @@ defimpl Inspect, for: BitString do
 
   @doc false
   def escape(other, char) do
-    escape(other, char, <<>>)
+    escape(other, char, [])
   end
 
-  defp escape(<<char, t::binary>>, char, binary) do
-    escape(t, char, <<binary::binary, ?\\, char>>)
+  defp escape(<<char, t::binary>>, char, acc) do
+    escape(t, char, [acc | [?\\, char]])
   end
-  defp escape(<<?#, ?{, t::binary>>, char, binary) do
-    escape(t, char, <<binary::binary, ?\\, ?#, ?{>>)
+  defp escape(<<?#, ?{, t::binary>>, char, acc) do
+    escape(t, char, [acc | '\\\#{'])
   end
-  defp escape(<<?\a, t::binary>>, char, binary) do
-    escape(t, char, <<binary::binary, ?\\, ?a>>)
+  defp escape(<<?\a, t::binary>>, char, acc) do
+    escape(t, char, [acc | '\\a'])
   end
-  defp escape(<<?\b, t::binary>>, char, binary) do
-    escape(t, char, <<binary::binary, ?\\, ?b>>)
+  defp escape(<<?\b, t::binary>>, char, acc) do
+    escape(t, char, [acc | '\\b'])
   end
-  defp escape(<<?\d, t::binary>>, char, binary) do
-    escape(t, char, <<binary::binary, ?\\, ?d>>)
+  defp escape(<<?\d, t::binary>>, char, acc) do
+    escape(t, char, [acc | '\\d'])
   end
-  defp escape(<<?\e, t::binary>>, char, binary) do
-    escape(t, char, <<binary::binary, ?\\, ?e>>)
+  defp escape(<<?\e, t::binary>>, char, acc) do
+    escape(t, char, [acc | '\\e'])
   end
-  defp escape(<<?\f, t::binary>>, char, binary) do
-    escape(t, char, <<binary::binary, ?\\, ?f>>)
+  defp escape(<<?\f, t::binary>>, char, acc) do
+    escape(t, char, [acc | '\\f'])
   end
-  defp escape(<<?\n, t::binary>>, char, binary) do
-    escape(t, char, <<binary::binary, ?\\, ?n>>)
+  defp escape(<<?\n, t::binary>>, char, acc) do
+    escape(t, char, [acc | '\\n'])
   end
-  defp escape(<<?\r, t::binary>>, char, binary) do
-    escape(t, char, <<binary::binary, ?\\, ?r>>)
+  defp escape(<<?\r, t::binary>>, char, acc) do
+    escape(t, char, [acc | '\\r'])
   end
-  defp escape(<<?\\, t::binary>>, char, binary) do
-    escape(t, char, <<binary::binary, ?\\, ?\\>>)
+  defp escape(<<?\\, t::binary>>, char, acc) do
+    escape(t, char, [acc | '\\\\'])
   end
-  defp escape(<<?\t, t::binary>>, char, binary) do
-    escape(t, char, <<binary::binary, ?\\, ?t>>)
+  defp escape(<<?\t, t::binary>>, char, acc) do
+    escape(t, char, [acc | '\\t'])
   end
-  defp escape(<<?\v, t::binary>>, char, binary) do
-    escape(t, char, <<binary::binary, ?\\, ?v>>)
+  defp escape(<<?\v, t::binary>>, char, acc) do
+    escape(t, char, [acc | '\\v'])
   end
-  defp escape(<<h::utf8, t::binary>>, char, binary) do
-    head = <<h::utf8>>
-    if String.printable?(head) do
-      escape(t, char, append(head, binary))
-    else
-      <<byte::8, h::binary>> = head
-      t = <<h::binary, t::binary>>
-      escape(t, char, <<binary::binary, escape_char(byte)::binary>>)
-    end
+  defp escape(<<h::utf8, t::binary>>, char, acc)
+       when h in 0x20..0x7E
+       when h in 0xA0..0xD7FF
+       when h in 0xE000..0xFFFD
+       when h in 0x10000..0x10FFFF do
+    escape(t, char, [acc | <<h::utf8>>])
   end
-  defp escape(<<h, t::binary>>, char, binary) do
-    escape(t, char, <<binary::binary, escape_char(h)::binary>>)
+  defp escape(<<h, t::binary>>, char, acc) do
+    escape(t, char, [acc | escape_char(h)])
   end
-  defp escape(<<>>, _char, binary), do: binary
+  defp escape(<<>>, _char, acc), do: acc
 
   @doc false
   # Also used by Regex
   def escape_char(0) do
-    <<?\\, ?0>>
+    '\\0'
   end
 
   def escape_char(char) when char < 0x100 do
     <<a::4, b::4>> = <<char::8>>
-    <<?\\, ?x, to_hex(a), to_hex(b)>>
+    ['\\x', to_hex(a), to_hex(b)]
   end
 
   def escape_char(char) when char < 0x10000 do
     <<a::4, b::4, c::4, d::4>> = <<char::16>>
-    <<?\\, ?x, ?{, to_hex(a), to_hex(b), to_hex(c), to_hex(d), ?}>>
+    ['\\x{', to_hex(a), to_hex(b), to_hex(c), to_hex(d), ?}]
   end
 
   def escape_char(char) when char < 0x1000000 do
     <<a::4, b::4, c::4, d::4, e::4, f::4>> = <<char::24>>
-    <<?\\, ?x, ?{, to_hex(a), to_hex(b), to_hex(c),
-                   to_hex(d), to_hex(e), to_hex(f), ?}>>
+    ['\\x{', to_hex(a), to_hex(b), to_hex(c),
+             to_hex(d), to_hex(e), to_hex(f), ?}]
   end
 
   defp to_hex(c) when c in 0..9, do: ?0+c
   defp to_hex(c) when c in 10..15, do: ?A+c-10
 
-  defp append(<<h, t::binary>>, binary), do: append(t, <<binary::binary, h>>)
-  defp append(<<>>, binary), do: binary
-
   ## Bitstrings
 
-  defp inspect_bitstring("", _opts) do
-    "<<>>"
+  defp inspect_bitstring("", opts) do
+    color("<<>>", :binary, opts)
   end
 
   defp inspect_bitstring(bitstring, opts) do
-    nest surround("<<", each_bit(bitstring, opts.limit, opts), ">>"), 1
+    left = color("<<", :binary, opts)
+    right = color(">>", :binary, opts)
+    nest surround(left, each_bit(bitstring, opts.limit, opts), right), 1
   end
 
   defp each_bit(_, 0, _) do
@@ -277,7 +278,9 @@ defimpl Inspect, for: BitString do
 end
 
 defimpl Inspect, for: List do
-  def inspect([], _opts), do: "[]"
+  def inspect([], opts) do
+    color("[]", :list, opts)
+  end
 
   # TODO: Deprecate :char_lists and :as_char_lists keys in v1.5
   def inspect(term, %Inspect.Opts{charlists: lists, char_lists: lists_deprecated} = opts) do
@@ -293,22 +296,24 @@ defimpl Inspect, for: List do
         lists
       end
 
+    open = color("[", :list, opts)
+    sep = color(",", :list, opts)
+    close = color("]", :list, opts)
+
     cond do
       lists == :as_charlists or (lists == :infer and printable?(term)) ->
-        <<?', Inspect.BitString.escape(IO.chardata_to_string(term), ?')::binary, ?'>>
+        IO.iodata_to_binary [?', Inspect.BitString.escape(IO.chardata_to_string(term), ?'), ?']
       keyword?(term) ->
-        surround_many("[", term, "]", opts, &keyword/2)
+        surround_many(open, term, close, opts, &keyword/2, sep)
       true ->
-        surround_many("[", term, "]", opts, &to_doc/2)
+        surround_many(open, term, close, opts, &to_doc/2, sep)
     end
   end
 
   @doc false
   def keyword({key, value}, opts) do
-    concat(
-      key_to_binary(key) <> ": ",
-      to_doc(value, opts)
-    )
+    key = color(key_to_binary(key) <> ": ", :atom, opts)
+    concat(key, to_doc(value, opts))
   end
 
   @doc false
@@ -323,15 +328,15 @@ defimpl Inspect, for: List do
   def keyword?(_other), do: false
 
   @doc false
-  def printable?([c | cs]) when c in 32..126, do: printable?(cs)
-  def printable?([?\n | cs]), do: printable?(cs)
-  def printable?([?\r | cs]), do: printable?(cs)
-  def printable?([?\t | cs]), do: printable?(cs)
-  def printable?([?\v | cs]), do: printable?(cs)
-  def printable?([?\b | cs]), do: printable?(cs)
-  def printable?([?\f | cs]), do: printable?(cs)
-  def printable?([?\e | cs]), do: printable?(cs)
-  def printable?([?\a | cs]), do: printable?(cs)
+  def printable?([char | rest]) when char in 32..126, do: printable?(rest)
+  def printable?([?\n | rest]), do: printable?(rest)
+  def printable?([?\r | rest]), do: printable?(rest)
+  def printable?([?\t | rest]), do: printable?(rest)
+  def printable?([?\v | rest]), do: printable?(rest)
+  def printable?([?\b | rest]), do: printable?(rest)
+  def printable?([?\f | rest]), do: printable?(rest)
+  def printable?([?\e | rest]), do: printable?(rest)
+  def printable?([?\a | rest]), do: printable?(rest)
   def printable?([]), do: true
   def printable?(_), do: false
 
@@ -346,10 +351,11 @@ defimpl Inspect, for: List do
 end
 
 defimpl Inspect, for: Tuple do
-  def inspect({}, _opts), do: "{}"
-
   def inspect(tuple, opts) do
-    surround_many("{", Tuple.to_list(tuple), "}", opts, &to_doc/2)
+    open = color("{", :tuple, opts)
+    sep = color(",", :tuple, opts)
+    close = color("}", :tuple, opts)
+    surround_many(open, Tuple.to_list(tuple), close, opts, &to_doc/2, sep)
   end
 end
 
@@ -360,7 +366,10 @@ defimpl Inspect, for: Map do
 
   def inspect(map, name, opts) do
     map = :maps.to_list(map)
-    surround_many("%" <> name <> "{", map, "}", opts, traverse_fun(map))
+    open = color("%" <> name <> "{", :map, opts)
+    sep = color(",", :map, opts)
+    close = color("}", :map, opts)
+    surround_many(open, map, close, opts, traverse_fun(map), sep)
   end
 
   defp traverse_fun(list) do
@@ -380,9 +389,9 @@ defimpl Inspect, for: Map do
 end
 
 defimpl Inspect, for: Integer do
-  def inspect(term, %Inspect.Opts{base: base}) do
-    Integer.to_string(term, base_to_value(base))
-    |> prepend_prefix(base)
+  def inspect(term, %Inspect.Opts{base: base} = opts) do
+    inspected = Integer.to_string(term, base_to_value(base)) |> prepend_prefix(base)
+    color(inspected, :number, opts)
   end
 
   defp base_to_value(base) do
@@ -406,60 +415,57 @@ defimpl Inspect, for: Integer do
 end
 
 defimpl Inspect, for: Float do
-  def inspect(term, _opts) do
-    IO.iodata_to_binary(:io_lib_format.fwrite_g(term))
+  def inspect(term, opts) do
+    inspected = IO.iodata_to_binary(:io_lib_format.fwrite_g(term))
+    color(inspected, :number, opts)
   end
 end
 
 defimpl Inspect, for: Regex do
-  def inspect(regex, _opts) do
-    delim = ?/
-    concat ["~r",
-            <<delim, escape(regex.source, delim)::binary, delim>>,
-            regex.opts]
+  def inspect(regex, opts) do
+    source = IO.iodata_to_binary(['~r/', escape(regex.source, ?/), ?/, regex.opts])
+    color(source, :regex, opts)
   end
 
   defp escape(bin, term),
-    do: escape(bin, <<>>, term)
+    do: escape(bin, [], term)
 
   defp escape(<<?\\, term>> <> rest, buf, term),
-    do: escape(rest, buf <> <<?\\, term>>, term)
+    do: escape(rest, [buf | [?\\, term]], term)
 
   defp escape(<<term>> <> rest, buf, term),
-    do: escape(rest, buf <> <<?\\, term>>, term)
+    do: escape(rest, [buf | [?\\, term]], term)
 
-  # the list of characters is from "String.printable?" impl
+  # The list of characters is from 'String.printable?' implementation
   # minus characters treated specially by regex: \s, \d, \b, \e
 
   defp escape(<<?\n>> <> rest, buf, term),
-    do: escape(rest, <<buf::binary, ?\\, ?n>>, term)
+    do: escape(rest, [buf | '\\n'], term)
 
   defp escape(<<?\r>> <> rest, buf, term),
-    do: escape(rest, <<buf::binary, ?\\, ?r>>, term)
+    do: escape(rest, [buf | '\\r'], term)
 
   defp escape(<<?\t>> <> rest, buf, term),
-    do: escape(rest, <<buf::binary, ?\\, ?t>>, term)
+    do: escape(rest, [buf | '\\t'], term)
 
   defp escape(<<?\v>> <> rest, buf, term),
-    do: escape(rest, <<buf::binary, ?\\, ?v>>, term)
+    do: escape(rest, [buf | '\\v'], term)
 
   defp escape(<<?\f>> <> rest, buf, term),
-    do: escape(rest, <<buf::binary, ?\\, ?f>>, term)
+    do: escape(rest, [buf | '\\f'], term)
 
   defp escape(<<?\a>> <> rest, buf, term),
-    do: escape(rest, <<buf::binary, ?\\, ?a>>, term)
+    do: escape(rest, [buf | '\\a'], term)
 
-  defp escape(<<c::utf8>> <> rest, buf, term) do
-    charstr = <<c::utf8>>
-    if String.printable?(charstr) and not c in [?\d, ?\b, ?\e] do
-      escape(rest, buf <> charstr, term)
-    else
-      escape(rest, buf <> Inspect.BitString.escape_char(c), term)
-    end
-  end
+  defp escape(<<char::utf8, rest::binary>>, buf, term)
+       when char in 0x20..0x7E
+       when char in 0xA0..0xD7FF
+       when char in 0xE000..0xFFFD
+       when char in 0x10000..0x10FFFF,
+    do: escape(rest, [buf | <<char::utf8>>], term)
 
-  defp escape(<<c>> <> rest, buf, term),
-    do: escape(rest, <<buf::binary, Inspect.BitString.escape_char(c)>>, term)
+  defp escape(<<char, rest::binary>>, buf, term),
+    do: escape(rest, [buf | Inspect.BitString.escape_char(char)], term)
 
   defp escape(<<>>, buf, _), do: buf
 end
@@ -516,7 +522,7 @@ end
 
 defimpl Inspect, for: Port do
   def inspect(port, _opts) do
-    IO.iodata_to_binary :erlang.port_to_list(port)
+    IO.iodata_to_binary(:erlang.port_to_list(port))
   end
 end
 
@@ -537,7 +543,8 @@ defimpl Inspect, for: Any do
       dunder ->
         if :maps.keys(dunder) == :maps.keys(map) do
           pruned = :maps.remove(:__exception__, :maps.remove(:__struct__, map))
-          Inspect.Map.inspect(pruned, Inspect.Atom.inspect(struct, opts), opts)
+          colorless_opts = %{opts | syntax_colors: []}
+          Inspect.Map.inspect(pruned, Inspect.Atom.inspect(struct, colorless_opts), opts)
         else
           Inspect.Map.inspect(map, opts)
         end

lib/elixir/lib/inspect/algebra.ex

@@ -33,8 +33,8 @@ defmodule Inspect.Opts do
       printing to IO devices. Set to 0 to force each item to be printed on its
       own line.
 
-    * `:base` - print integers as :binary, :octal, :decimal, or :hex, defaults
-      to :decimal. When inspecting binaries any `:base` other than `:decimal`
+    * `:base` - prints integers as `:binary`, `:octal`, `:decimal`, or `:hex`, defaults
+      to `:decimal`. When inspecting binaries any `:base` other than `:decimal`
       implies `binaries: :as_binaries`.
 
     * `:safe` - when `false`, failures while inspecting structs will be raised
@@ -42,6 +42,11 @@ defmodule Inspect.Opts do
       is useful when debugging failures and crashes for custom inspect
       implementations
 
+    * `:syntax_colors` - when set to a keyword list of colors the output will
+      be colorized. The keys are types and the values are the colors to use for
+      each type. e.g. `[number: :red, atom: :blue]`. Types can include
+      `:number`, `:atom`, `regex`, `:tuple`, `:map`, `:list`, and `:reset`.
+      Colors can be any `t:IO.ANSI.ansidata/0` as accepted by `IO.ANSI.format/1`.
   """
 
   # TODO: Deprecate char_lists key by v1.5
@@ -53,7 +58,10 @@ defmodule Inspect.Opts do
             width: 80,
             base: :decimal,
             pretty: false,
-            safe: true
+            safe: true,
+            syntax_colors: []
+
+  @type color_key :: atom
 
   # TODO: Deprecate char_lists key and :as_char_lists value by v1.5
   @type t :: %__MODULE__{
@@ -65,7 +73,9 @@ defmodule Inspect.Opts do
                width: pos_integer | :infinity,
                base: :decimal | :binary | :hex | :octal,
                pretty: boolean,
-               safe: boolean}
+               safe: boolean,
+               syntax_colors: [{color_key, IO.ANSI.ansidata}]
+             }
 end
 
 defmodule Inspect.Error do
@@ -100,7 +110,7 @@ defmodule Inspect.Algebra do
 
   The functions `nest/2`, `space/2` and `line/2` help you put the
   document together into a rigid structure. However, the document
-  algebra gets interesting when using functions like `break/2`, which
+  algebra gets interesting when using functions like `break/1`, which
   converts the given string into a line break depending on how much space
   there is to print. Let's glue two docs together with a break and then
   render it:
@@ -153,7 +163,7 @@ defmodule Inspect.Algebra do
 
   # Functional interface to "doc" records
 
-  @type t :: :doc_nil | :doc_line | doc_cons | doc_nest | doc_break | doc_group | binary
+  @type t :: :doc_nil | :doc_line | doc_cons | doc_nest | doc_break | doc_group | doc_color | binary
 
   @typep doc_cons :: {:doc_cons, t, t}
   defmacrop doc_cons(left, right) do
@@ -175,6 +185,11 @@ defmodule Inspect.Algebra do
     quote do: {:doc_group, unquote(group)}
   end
 
+  @typep doc_color :: {:doc_color, t, IO.ANSI.ansidata}
+  defmacrop doc_color(doc, color) do
+    quote do: {:doc_color, unquote(doc), unquote(color)}
+  end
+
   defmacrop is_doc(doc) do
     if Macro.Env.in_guard?(__CALLER__) do
       do_is_doc(doc)
@@ -192,15 +207,17 @@ defmodule Inspect.Algebra do
       is_binary(unquote(doc)) or
       unquote(doc) in [:doc_nil, :doc_line] or
       (is_tuple(unquote(doc)) and
-       elem(unquote(doc), 0) in [:doc_cons, :doc_nest, :doc_break, :doc_group])
+       elem(unquote(doc), 0) in [:doc_cons, :doc_nest, :doc_break, :doc_group, :doc_color])
     end
   end
 
   @doc """
-  Converts an Elixir structure to an algebra document
-  according to the inspect protocol.
+  Converts an Elixir term to an algebra document
+  according to the `Inspect` protocol.
   """
   @spec to_doc(any, Inspect.Opts.t) :: t
+  def to_doc(term, opts)
+
   def to_doc(%{__struct__: struct} = map, %Inspect.Opts{} = opts) when is_atom(struct) do
     if opts.structs do
       try do
@@ -261,30 +278,50 @@ defmodule Inspect.Algebra do
   def empty, do: :doc_nil
 
   @doc """
-  Concatenates two document entities.
+  Concatenates two document entities returning a new document.
 
   ## Examples
 
-      iex> doc = Inspect.Algebra.concat "hello", "world"
+      iex> doc = Inspect.Algebra.concat("hello", "world")
       iex> Inspect.Algebra.format(doc, 80)
       ["hello", "world"]
 
   """
-  @spec concat(t, t) :: doc_cons
-  def concat(x, y) when is_doc(x) and is_doc(y) do
-    doc_cons(x, y)
+  @spec concat(t, t) :: t
+  def concat(doc1, doc2) when is_doc(doc1) and is_doc(doc2) do
+    doc_cons(doc1, doc2)
   end
 
   @doc """
-  Concatenates a list of documents.
+  Concatenates a list of documents returning a new document.
+
+  ## Examples
+
+      iex> doc = Inspect.Algebra.concat(["a", "b", "c"])
+      iex> Inspect.Algebra.format(doc, 80)
+      ["a", "b", "c"]
+
   """
-  @spec concat([t]) :: doc_cons
-  def concat(docs) do
+  @spec concat([t]) :: t
+  def concat(docs) when is_list(docs) do
     fold_doc(docs, &concat(&1, &2))
   end
 
+  @doc """
+  Colors a document if the `color_key` has a color in the options.
+  """
+  @spec color(t, Inspect.Opts.color_key, Inspect.Opts.t) :: doc_color
+  def color(doc, color_key, %Inspect.Opts{syntax_colors: syntax_colors}) when is_doc(doc) do
+    if precolor = Keyword.get(syntax_colors, color_key) do
+      postcolor = Keyword.get(syntax_colors, :reset, :reset)
+      concat(doc_color(doc, precolor), doc_color(empty(), postcolor))
+    else
+      doc
+    end
+  end
+
   @doc ~S"""
-  Nests document entity `x` positions deep.
+  Nests the given document at the given `level`.
 
   Nesting will be appended to the line breaks.
 
@@ -296,58 +333,88 @@ defmodule Inspect.Algebra do
 
   """
   @spec nest(t, non_neg_integer) :: doc_nest
-  def nest(x, 0) when is_doc(x) do
-    x
+  def nest(doc, level)
+
+  def nest(doc, 0) when is_doc(doc) do
+    doc
   end
 
-  def nest(x, i) when is_doc(x) and is_integer(i) do
-    doc_nest(x, i)
+  def nest(doc, level) when is_doc(doc) and is_integer(level) and level > 0 do
+    doc_nest(doc, level)
   end
 
   @doc ~S"""
-  Document entity representing a break.
+  Returns a document entity representing a break based on the given
+  `string`.
 
-  This break can be rendered as a linebreak or as spaces,
+  This break can be rendered as a linebreak or as the given `string`,
   depending on the `mode` of the chosen layout or the provided
   separator.
 
   ## Examples
 
-  Let's glue two docs together with a break and then render it:
+  Let's create a document by concatenating two strings with a break between
+  them:
 
-      iex> doc = Inspect.Algebra.glue("a", " ", "b")
+      iex> doc = Inspect.Algebra.concat(["a", Inspect.Algebra.break("\t"), "b"])
       iex> Inspect.Algebra.format(doc, 80)
-      ["a", " ", "b"]
+      ["a", "\t", "b"]
 
-  Notice the break was represented as is, because we haven't reached
-  a line limit. Once we do, it is replaced by a newline:
+  Notice the break was represented with the given string, because we didn't
+  reach a line limit. Once we do, it is replaced by a newline:
 
-      iex> doc = Inspect.Algebra.glue(String.duplicate("a", 20), " ", "b")
+      iex> break = Inspect.Algebra.break("\t")
+      iex> doc = Inspect.Algebra.concat([String.duplicate("a", 20), break, "b"])
       iex> Inspect.Algebra.format(doc, 10)
       ["aaaaaaaaaaaaaaaaaaaa", "\n", "b"]
 
   """
   @spec break(binary) :: doc_break
-  def break(s) when is_binary(s), do: doc_break(s)
+  def break(string) when is_binary(string), do: doc_break(string)
 
+  @doc """
+  Returns a document entity representing the default break.
+
+  Same as calling `break/1` with the default break.
+  """
   @spec break() :: doc_break
   def break(), do: doc_break(@break)
 
   @doc """
-  Inserts a break between two docs. See `break/1` for more info.
+  Glues two documents together inserting the default break between them.
+
+  The break that is inserted between `left` and `right` is the one returned by
+  `break/0`.
+
+  ## Examples
+
+      iex> doc = Inspect.Algebra.glue("hello", "world")
+      iex> Inspect.Algebra.format(doc, 80)
+      ["hello", " ", "world"]
+
   """
-  @spec glue(t, t) :: doc_cons
-  def glue(x, y), do: concat(x, concat(break, y))
+  @spec glue(t, t) :: t
+  def glue(doc1, doc2), do: concat(doc1, concat(break(), doc2))
 
   @doc """
-  Inserts a break, passed as the second argument, between two docs,
-  the first and the third arguments.
+  Glues two documents (`doc1` and `doc2`) together inserting the given
+  break `break_string` between them.
+
+  For more information on how the break is inserted, see `break/1`.
+
+  ## Examples
+
+      iex> doc = Inspect.Algebra.glue("hello", "\t", "world")
+      iex> Inspect.Algebra.format(doc, 80)
+      ["hello", "\t", "world"]
+
   """
-  @spec glue(t, binary, t) :: doc_cons
-  def glue(x, g, y) when is_binary(g), do: concat(x, concat(break(g), y))
+  @spec glue(t, binary, t) :: t
+  def glue(doc1, break_string, doc2) when is_binary(break_string),
+    do: concat(doc1, concat(break(break_string), doc2))
 
   @doc ~S"""
-  Returns a group containing the specified document.
+  Returns a group containing the specified document `doc`.
 
   ## Examples
 
@@ -374,83 +441,91 @@ defmodule Inspect.Algebra do
 
   """
   @spec group(t) :: doc_group
-  def group(d) when is_doc(d) do
-    doc_group(d)
+  def group(doc) when is_doc(doc) do
+    doc_group(doc)
   end
 
   @doc """
-  Inserts a mandatory single space between two document entities.
+  Inserts a mandatory single space between two documents.
 
   ## Examples
 
-      iex> doc = Inspect.Algebra.space "Hughes", "Wadler"
-      iex> Inspect.Algebra.format(doc, 80)
+      iex> doc = Inspect.Algebra.space("Hughes", "Wadler")
+      iex> Inspect.Algebra.format(doc, 5)
       ["Hughes", " ", "Wadler"]
 
   """
-  @spec space(t, t) :: doc_cons
-  def space(x, y), do: concat(x, concat(" ", y))
+  @spec space(t, t) :: t
+  def space(doc1, doc2), do: concat(doc1, concat(" ", doc2))
 
   @doc ~S"""
-  Inserts a mandatory linebreak between two document entities.
+  Inserts a mandatory linebreak between two documents.
 
   ## Examples
 
-      iex> doc = Inspect.Algebra.line "Hughes", "Wadler"
+      iex> doc = Inspect.Algebra.line("Hughes", "Wadler")
       iex> Inspect.Algebra.format(doc, 80)
       ["Hughes", "\n", "Wadler"]
 
   """
-  @spec line(t, t) :: doc_cons
-  def line(x, y), do: concat(x, concat(:doc_line, y))
+  @spec line(t, t) :: t
+  def line(doc1, doc2), do: concat(doc1, concat(:doc_line, doc2))
 
   @doc """
-  Folds a list of document entities into a document entity
-  using a function that is passed as the first argument.
+  Folds a list of documents into a document using the given folder function.
+
+  The list of documents is folded "from the right"; in that, this function is
+  similar to `List.foldr/3`, except that it doesn't expect an initial
+  accumulator and uses the last element of `docs` as the initial accumulator.
 
   ## Examples
 
-      iex> doc = ["A", "B"]
-      iex> doc = Inspect.Algebra.fold_doc(doc, fn(x, y) ->
-      ...>   Inspect.Algebra.concat [x, "!", y]
+      iex> docs = ["A", "B", "C"]
+      iex> docs = Inspect.Algebra.fold_doc(docs, fn(doc, acc) ->
+      ...>   Inspect.Algebra.concat([doc, "!", acc])
       ...> end)
-      iex> Inspect.Algebra.format(doc, 80)
-      ["A", "!", "B"]
+      iex> Inspect.Algebra.format(docs, 80)
+      ["A", "!", "B", "!", "C"]
 
   """
   @spec fold_doc([t], ((t, t) -> t)) :: t
-  def fold_doc(list, fun)
-  def fold_doc([], _), do: empty
-  def fold_doc([doc], _), do: doc
-  def fold_doc([d | ds], fun), do: fun.(d, fold_doc(ds, fun))
+  def fold_doc(docs, folder_fun)
+
+  def fold_doc([], _folder_fun),
+    do: empty()
+  def fold_doc([doc], _folder_fun),
+    do: doc
+  def fold_doc([doc | docs], folder_fun) when is_function(folder_fun, 2),
+    do: folder_fun.(doc, fold_doc(docs, folder_fun))
 
   # Elixir conveniences
 
   @doc ~S"""
   Surrounds a document with characters.
 
-  Puts the document between left and right enclosing and nesting it.
-  The document is marked as a group, to show the maximum as possible
-  concisely together.
+  Puts the given document `doc` between the `left` and `right` documents enclosing
+  and nesting it. The document is marked as a group, to show the maximum as
+  possible concisely together.
 
   ## Examples
 
-      iex> doc = Inspect.Algebra.surround "[", Inspect.Algebra.glue("a", "b"), "]"
+      iex> doc = Inspect.Algebra.surround("[", Inspect.Algebra.glue("a", "b"), "]")
       iex> Inspect.Algebra.format(doc, 3)
       ["[", "a", "\n ", "b", "]"]
 
   """
-  @spec surround(binary, t, binary) :: t
-  def surround(left, doc, right) do
-    group concat left, concat(nest(doc, @nesting), right)
+  @spec surround(t, t, t) :: t
+  def surround(left, doc, right) when is_doc(left) and is_doc(doc) and is_doc(right) do
+    group(concat(left, concat(nest(doc, @nesting), right)))
   end
 
   @doc ~S"""
   Maps and glues a collection of items.
 
-  It uses the given left and right as surrounding and a separator for
-  each item. A limit can be passed which, once reached, stops gluing
-  and outputs "..." instead.
+  It uses the given `left` and `right` documents as surrounding and the
+  separator document `separator` to separate items in `docs`. A limit can be
+  passed: when this limit is reached, this function stops gluing and outputs
+  `"..."` instead.
 
   ## Examples
 
@@ -468,9 +543,11 @@ defmodule Inspect.Algebra do
       ...>         %Inspect.Opts{limit: 3}, fn i, _opts -> to_string(i) end, "!")
       iex> Inspect.Algebra.format(doc, 20) |> IO.iodata_to_binary
       "[1! 2! 3! ...]"
+
   """
-  @spec surround_many(binary, [any], binary, Inspect.Opts.t, (term, Inspect.Opts.t -> t), binary) :: t
-  def surround_many(left, docs, right, opts, fun, separator \\ @surround_separator) do
+  @spec surround_many(t, [any], t, Inspect.Opts.t, (term, Inspect.Opts.t -> t), t) :: t
+  def surround_many(left, docs, right, %Inspect.Opts{} = opts, fun, separator \\ @surround_separator)
+      when is_doc(left) and is_list(docs) and is_doc(right) and is_function(fun, 2) and is_doc(separator) do
     do_surround_many(left, docs, right, opts.limit, opts, fun, separator)
   end
 
@@ -516,16 +593,25 @@ defmodule Inspect.Algebra do
   defp decrement(:infinity), do: :infinity
   defp decrement(counter),   do: counter - 1
 
-  @doc """
-  The formatting function.
+  @doc ~S"""
+  Formats a given document for a given width.
 
   Takes the maximum width and a document to print as its arguments
   and returns an IO data representation of the best layout for the
   document to fit in the given width.
+
+  ## Examples
+
+      iex> doc = Inspect.Algebra.glue("hello", " ", "world")
+      iex> Inspect.Algebra.format(doc, 30) |> IO.iodata_to_binary()
+      "hello world"
+      iex> Inspect.Algebra.format(doc, 10) |> IO.iodata_to_binary()
+      "hello\nworld"
+
   """
   @spec format(t, non_neg_integer | :infinity) :: iodata
-  def format(d, w) when w == :infinity or w >= 0 do
-    format(w, 0, [{0, default_mode(w), doc_group(d)}])
+  def format(doc, width) when is_doc(doc) and (width == :infinity or width >= 0) do
+    format(width, 0, [{0, default_mode(width), doc_group(doc)}])
   end
 
   defp default_mode(:infinity), do: :flat
@@ -540,10 +626,11 @@ defmodule Inspect.Algebra do
   defp fits?(_, [{_, _, :doc_line} | _]),           do: true
   defp fits?(w, [{_, _, :doc_nil} | t]),            do: fits?(w, t)
   defp fits?(w, [{i, m, doc_cons(x, y)} | t]),      do: fits?(w, [{i, m, x} | [{i, m, y} | t]])
+  defp fits?(w, [{i, m, doc_color(x, _)} | t]),    do: fits?(w, [{i, m, x} | t])
   defp fits?(w, [{i, m, doc_nest(x, j)} | t]),      do: fits?(w, [{i + j, m, x} | t])
   defp fits?(w, [{i, _, doc_group(x)} | t]),        do: fits?(w, [{i, :flat, x} | t])
-  defp fits?(w, [{_, _, s} | t]) when is_binary(s), do: fits?((w - byte_size s), t)
-  defp fits?(w, [{_, :flat, doc_break(s)} | t]),    do: fits?((w - byte_size s), t)
+  defp fits?(w, [{_, _, s} | t]) when is_binary(s), do: fits?((w - byte_size(s)), t)
+  defp fits?(w, [{_, :flat, doc_break(s)} | t]),    do: fits?((w - byte_size(s)), t)
   defp fits?(_, [{_, :break, doc_break(_)} | _]),   do: true
 
   @spec format(integer | :infinity, integer, [{integer, mode, t}]) :: [binary]
@@ -553,8 +640,9 @@ defmodule Inspect.Algebra do
   defp format(w, k, [{i, m, doc_cons(x, y)} | t]),      do: format(w, k, [{i, m, x} | [{i, m, y} | t]])
   defp format(w, k, [{i, m, doc_nest(x, j)} | t]),      do: format(w, k, [{i + j, m, x} | t])
   defp format(w, k, [{i, m, doc_group(x)} | t]),        do: format(w, k, [{i, m, x} | t])
-  defp format(w, k, [{_, _, s} | t]) when is_binary(s), do: [s | format(w, (k + byte_size s), t)]
-  defp format(w, k, [{_, :flat, doc_break(s)} | t]),    do: [s | format(w, (k + byte_size s), t)]
+  defp format(w, k, [{i, m, doc_color(x, c)} | t]),     do: [ansi(c) | format(w, k, [{i, m, x} | t])]
+  defp format(w, k, [{_, _, s} | t]) when is_binary(s), do: [s | format(w, (k + byte_size(s)), t)]
+  defp format(w, k, [{_, :flat, doc_break(s)} | t]),    do: [s | format(w, (k + byte_size(s)), t)]
   defp format(w, k, [{i, :break, doc_break(s)} | t]) do
     k = k + byte_size(s)
 
@@ -565,6 +653,10 @@ defmodule Inspect.Algebra do
     end
   end
 
+  defp ansi(color) do
+    IO.ANSI.format_fragment(color, true)
+  end
+
   defp indent(0), do: @newline
   defp indent(i), do: @newline <> :binary.copy(" ", i)
 end

lib/elixir/lib/integer.ex

@@ -60,33 +60,97 @@ defmodule Integer do
   end
 
   @doc """
-  Returns the ordered digits for the given non-negative `integer`.
+  Computes the modulo remainder of an integer division.
 
-  An optional `base` value may be provided representing the radix for the returned
-  digits. This one can be an integer >= 2.
+  `Integer.mod/2` uses floored division, which means that
+  the result will always have the sign of the `divisor`.
+
+  Raises an `ArithmeticError` exception if one of the arguments is not an
+  integer, or when the `divisor` is `0`.
 
   ## Examples
 
-      iex> Integer.digits(101)
-      [1, 0, 1]
+      iex> Integer.mod(5, 2)
+      1
+      iex> Integer.mod(6, -4)
+      -2
+
+  """
+  @spec mod(integer, neg_integer | pos_integer) :: integer
+  def mod(dividend, divisor) do
+    remainder = rem(dividend, divisor)
+    if remainder * divisor < 0 do
+      remainder + divisor
+    else
+      remainder
+    end
+  end
+
+  @doc """
+  Performs a floored integer division.
+
+  Raises an `ArithmeticError` exception if one of the arguments is not an
+  integer, or when the `divisor` is `0`.
+
+  `Integer.floor_div/2` performs *floored* integer division. This means that
+  the result is always rounded towards negative infinity.
+
+  If you want to perform truncated integer division (rounding towards zero),
+  use `Kernel.div/2` instead.
+
+  ## Examples
+
+      iex> Integer.floor_div(5, 2)
+      2
+      iex> Integer.floor_div(6, -4)
+      -2
+      iex> Integer.floor_div(-99, 2)
+      -50
+
+  """
+  @spec floor_div(integer, neg_integer | pos_integer) :: integer
+  def floor_div(dividend, divisor) do
+    if (dividend * divisor < 0) and rem(dividend, divisor) != 0 do
+      div(dividend, divisor) - 1
+    else
+      div(dividend, divisor)
+    end
+  end
+
+  @doc """
+  Returns the ordered digits for the given `integer`.
+
+  An optional `base` value may be provided representing the radix for the returned
+  digits. This one must be an integer >= 2.
+
+  ## Examples
+
+      iex> Integer.digits(123)
+      [1, 2, 3]
 
       iex> Integer.digits(170, 2)
       [1, 0, 1, 0, 1, 0, 1, 0]
 
+      iex> Integer.digits(-170, 2)
+      [-1, 0, -1, 0, -1, 0, -1, 0]
+
   """
-  @spec digits(non_neg_integer, pos_integer) :: [non_neg_integer, ...]
+  @spec digits(integer, pos_integer) :: [integer, ...]
   def digits(integer, base \\ 10)
-      when is_integer(integer) and integer >= 0 and is_integer(base) and base >= 2 do
+      when is_integer(integer) and is_integer(base) and base >= 2 do
     do_digits(integer, base, [])
   end
 
-  defp do_digits(0, _base, []), do: [0]
-  defp do_digits(1, _base, []), do: [1]
-  defp do_digits(base, base, []), do: [1, 0]
-  defp do_digits(0, _base, acc), do: acc
-  defp do_digits(integer, base, acc) do
-    do_digits div(integer, base), base, [rem(integer, base) | acc]
-  end
+  defp do_digits(digit, base, []) when abs(digit) < base,
+    do: [digit]
+  defp do_digits(digit, base, []) when digit == -base,
+    do: [-1, 0]
+  defp do_digits(base, base, []),
+    do: [1, 0]
+  defp do_digits(0, _base, acc),
+    do: acc
+  defp do_digits(integer, base, acc),
+    do: do_digits(div(integer, base), base, [rem(integer, base) | acc])
 
   @doc """
   Returns the integer represented by the ordered `digits`.
@@ -111,14 +175,21 @@ defmodule Integer do
     do_undigits(digits, base, 0)
   end
 
-  defp do_undigits([], _base, []), do: 0
-  defp do_undigits([0], _base, []), do: 0
-  defp do_undigits([1], _base, []), do: 1
-  defp do_undigits([1, 0], base, []), do: base
-  defp do_undigits([], _base, acc), do: acc
-  defp do_undigits([digit | tail], base, acc) do
-    do_undigits(tail, base, acc * base + digit)
-  end
+  defp do_undigits([], _base, 0),
+    do: 0
+  defp do_undigits([digit], base, 0) when is_integer(digit) and digit < base,
+    do: digit
+  defp do_undigits([1, 0], base, 0),
+    do: base
+  defp do_undigits([0 | tail], base, 0),
+    do: do_undigits(tail, base, 0)
+
+  defp do_undigits([], _base, acc),
+    do: acc
+  defp do_undigits([digit | _], base, _) when is_integer(digit) and digit >= base,
+    do: raise ArgumentError, "invalid digit #{digit} in base #{base}"
+  defp do_undigits([digit | tail], base, acc) when is_integer(digit),
+    do: do_undigits(tail, base, acc * base + digit)
 
   @doc """
   Parses a text representation of an integer.
@@ -192,7 +263,7 @@ defmodule Integer do
 
   defp do_parse(<<char, rest::binary>>, base) do
     if valid_digit_in_base?(char, base) do
-      do_parse(rest, base, parse_digit(char, base))
+      do_parse(rest, base, parse_digit(char))
     else
       :error
     end
@@ -204,7 +275,7 @@ defmodule Integer do
 
   defp do_parse(<<char, rest::binary>> = bin, base, acc) do
     if valid_digit_in_base?(char, base) do
-      do_parse(rest, base, base * acc + parse_digit(char, base))
+      do_parse(rest, base, base * acc + parse_digit(char))
     else
       {acc, bin}
     end
@@ -214,7 +285,7 @@ defmodule Integer do
     {acc, bitstring}
   end
 
-  defp parse_digit(char, _) do
+  defp parse_digit(char) do
     cond do
       char in ?0..?9 -> char - ?0
       char in ?A..?Z -> char - ?A + 10

lib/elixir/lib/io.ex

@@ -3,7 +3,7 @@ defmodule IO do
   Functions handling input/output (IO).
 
   Many functions in this module expect an IO device as an argument.
-  An IO device must be a pid or an atom representing a process.
+  An IO device must be a PID or an atom representing a process.
   For convenience, Elixir provides `:stdio` and `:stderr` as
   shortcuts to Erlang's `:standard_io` and `:standard_error`.
 
@@ -17,7 +17,7 @@ defmodule IO do
 
   ## IO devices
 
-  An IO device may be an atom or a pid. In case it is an atom,
+  An IO device may be an atom or a PID. In case it is an atom,
   the atom must be the name of a registered process. In addition,
   Elixir provides two shortcuts:
 
@@ -38,8 +38,6 @@ defmodule IO do
   @type nodata :: {:error, term} | :eof
   @type chardata() :: :unicode.chardata()
 
-  import :erlang, only: [group_leader: 0]
-
   defmacrop is_iodata(data) do
     quote do
       is_list(unquote(data)) or is_binary(unquote(data))
@@ -55,7 +53,7 @@ defmodule IO do
 
   It returns:
 
-    * `data` - the input characters
+    * `data` - the output characters
 
     * `:eof` - end of file was encountered
 
@@ -67,7 +65,7 @@ defmodule IO do
   empty string in case the device has reached EOF.
   """
   @spec read(device, :all | :line | non_neg_integer) :: chardata | nodata
-  def read(device \\ group_leader(), line_or_chars)
+  def read(device \\ :stdio, line_or_chars)
 
   def read(device, :all) do
     do_read_all(map_dev(device), "")
@@ -92,13 +90,13 @@ defmodule IO do
   @doc """
   Reads from the IO `device`. The operation is Unicode unsafe.
 
-  The `device` is iterated by the given number of characters or line by line if
+  The `device` is iterated by the given number of bytes or line by line if
   `:line` is given.
   Alternatively, if `:all` is given, then whole `device` is returned.
 
   It returns:
 
-    * `data` - the input characters
+    * `data` - the output bytes
 
     * `:eof` - end of file was encountered
 
@@ -113,7 +111,7 @@ defmodule IO do
   as it will return the wrong result.
   """
   @spec binread(device, :all | :line | non_neg_integer) :: iodata | nodata
-  def binread(device \\ group_leader(), line_or_chars)
+  def binread(device \\ :stdio, line_or_chars)
 
   def binread(device, :all) do
     do_binread_all(map_dev(device), "")
@@ -158,7 +156,7 @@ defmodule IO do
 
   """
   @spec write(device, chardata | String.Chars.t) :: :ok
-  def write(device \\ group_leader(), item) do
+  def write(device \\ :stdio, item) do
     :io.put_chars map_dev(device), to_chardata(item)
   end
 
@@ -173,7 +171,7 @@ defmodule IO do
   as it will return the wrong result.
   """
   @spec binwrite(device, iodata) :: :ok | {:error, term}
-  def binwrite(device \\ group_leader(), item) when is_iodata(item) do
+  def binwrite(device \\ :stdio, item) when is_iodata(item) do
     :file.write map_dev(device), item
   end
 
@@ -182,7 +180,7 @@ defmodule IO do
   but adds a newline at the end.
   """
   @spec puts(device, chardata | String.Chars.t) :: :ok
-  def puts(device \\ group_leader(), item) do
+  def puts(device \\ :stdio, item) do
     :io.put_chars map_dev(device), [to_chardata(item), ?\n]
   end
 
@@ -233,32 +231,67 @@ defmodule IO do
   @doc """
   Inspects and writes the given `item` to the device.
 
+  It's important to note that it returns the given `item` unchanged.
+  This makes it possible to "spy" on values by inserting an
+  `IO.inspect/2` call almost anywhere in your code, for example,
+  in the middle of a pipeline.
+
   It enables pretty printing by default with width of
   80 characters. The width can be changed by explicitly
   passing the `:width` option.
 
-  See `Inspect.Opts` for a full list of options.
+  The output can be decorated with a label, by providing the `:label`
+  option to easily distinguish it from other `IO.inspect/2` calls.
+  The label will be printed before the inspected `item`.
+
+  See `Inspect.Opts` for a full list of remaining formatting options.
 
   ## Examples
 
-      IO.inspect Process.list, width: 40
+      IO.inspect <<0, 1, 2>>, width: 40
+
+  Prints:
+
+      <<0, 1, 2>>
+
+  We can use the `:label` option to decorate the output:
+
+      IO.inspect 1..100, label: "a wonderful range"
+
+  Prints:
+
+      a wonderful range: 1..100
+
+  The `:label` option is especially useful with pipelines:
+
+      [1, 2, 3]
+      |> IO.inspect(label: "before")
+      |> Enum.map(&(&1 * 2))
+      |> IO.inspect(label: "after")
+      |> Enum.sum
+
+  Prints:
+
+      before: [1, 2, 3]
+      after: [2, 4, 6]
 
   """
   @spec inspect(item, Keyword.t) :: item when item: var
   def inspect(item, opts \\ []) do
-    inspect group_leader(), item, opts
+    inspect :stdio, item, opts
   end
 
   @doc """
   Inspects `item` according to the given options using the IO `device`.
 
-  See `Inspect.Opts` for a full list of options.
+  See `inspect/2` for a full list of options.
   """
   @spec inspect(device, item, Keyword.t) :: item when item: var
   def inspect(device, item, opts) when is_list(opts) do
-    opts   = struct(Inspect.Opts, opts)
-    iodata = Inspect.Algebra.format(Inspect.Algebra.to_doc(item, opts), opts.width)
-    puts device, iodata
+    label    = if (label = opts[:label]), do: [to_chardata(label), ": "], else: []
+    opts     = struct(Inspect.Opts, opts)
+    chardata = Inspect.Algebra.format(Inspect.Algebra.to_doc(item, opts), opts.width)
+    puts device, [label, chardata]
     item
   end
 
@@ -277,7 +310,7 @@ defmodule IO do
   def getn(prompt, count \\ 1)
 
   def getn(prompt, count) when is_integer(count) and count > 0 do
-    getn(group_leader, prompt, count)
+    getn(:stdio, prompt, count)
   end
 
   def getn(device, prompt) when not is_integer(prompt) do
@@ -307,7 +340,7 @@ defmodule IO do
     :io.get_chars(map_dev(device), to_chardata(prompt), count)
   end
 
-  @doc """
+  @doc ~S"""
   Reads a line from the IO `device`.
 
   It returns:
@@ -329,7 +362,7 @@ defmodule IO do
 
   """
   @spec gets(device, chardata | String.Chars.t) :: chardata | nodata
-  def gets(device \\ group_leader(), prompt) do
+  def gets(device \\ :stdio, prompt) do
     :io.get_line(map_dev(device), to_chardata(prompt))
   end
 
@@ -343,7 +376,7 @@ defmodule IO do
   The `device` is iterated by the given number of characters or line by line if
   `:line` is given.
 
-  This reads from the IO as utf-8. Check out
+  This reads from the IO as UTF-8. Check out
   `IO.binstream/2` to handle the IO as a raw binary.
 
   Note that an IO stream has side effects and every time

lib/elixir/lib/io/ansi.ex

@@ -42,7 +42,7 @@ defmodule IO.ANSI do
     Application.get_env(:elixir, :ansi_enabled, false)
   end
 
-  @doc "Sets foreground color"
+  @doc "Sets foreground color."
   @spec color(0..255) :: String.t
   def color(code) when code in 0..255, do: "\e[38;5;#{code}m"
 
@@ -56,7 +56,7 @@ defmodule IO.ANSI do
     color(16 + (36 * r) + (6 * g) + b)
   end
 
-  @doc "Sets background color"
+  @doc "Sets background color."
   @spec color_background(0..255) :: String.t
   def color_background(code) when code in 0..255, do: "\e[48;5;#{code}m"
 
@@ -70,97 +70,109 @@ defmodule IO.ANSI do
     color_background(16 + (36 * r) + (6 * g) + b)
   end
 
-  @doc "Resets all attributes"
+  @doc "Resets all attributes."
   defsequence :reset, 0
 
-  @doc "Bright (increased intensity) or Bold"
+  @doc "Bright (increased intensity) or bold."
   defsequence :bright, 1
 
-  @doc "Faint (decreased intensity), not widely supported"
+  @doc "Faint (decreased intensity). Not widely supported."
   defsequence :faint, 2
 
-  @doc "Italic: on. Not widely supported. Sometimes treated as inverse"
+  @doc "Italic: on. Not widely supported. Sometimes treated as inverse."
   defsequence :italic, 3
 
-  @doc "Underline: Single"
+  @doc "Underline: single."
   defsequence :underline, 4
 
-  @doc "Blink: Slow. Less than 150 per minute"
+  @doc "Blink: slow. Less than 150 per minute."
   defsequence :blink_slow, 5
 
-  @doc "Blink: Rapid. MS-DOS ANSI.SYS; 150 per minute or more; not widely supported"
+  @doc "Blink: rapid. MS-DOS ANSI.SYS; 150 per minute or more; not widely supported."
   defsequence :blink_rapid, 6
 
-  @doc "Image: Negative. Swap foreground and background"
+  @doc "Image: negative. Swap foreground and background."
   defsequence :inverse, 7
 
-  @doc "Image: Negative. Swap foreground and background"
+  @doc "Image: negative. Swap foreground and background."
   defsequence :reverse, 7
 
-  @doc "Conceal. Not widely supported"
+  @doc "Conceal. Not widely supported."
   defsequence :conceal, 8
 
-  @doc "Crossed-out. Characters legible, but marked for deletion. Not widely supported"
+  @doc "Crossed-out. Characters legible, but marked for deletion. Not widely supported."
   defsequence :crossed_out, 9
 
-  @doc "Sets primary (default) font"
+  @doc "Sets primary (default) font."
   defsequence :primary_font, 10
 
   for font_n <- [1, 2, 3, 4, 5, 6, 7, 8, 9] do
-    @doc "Sets alternative font #{font_n}"
+    @doc "Sets alternative font #{font_n}."
     defsequence :"font_#{font_n}", font_n + 10
   end
 
-  @doc "Normal color or intensity"
+  @doc "Normal color or intensity."
   defsequence :normal, 22
 
-  @doc "Not italic"
+  @doc "Not italic."
   defsequence :not_italic, 23
 
-  @doc "Underline: None"
+  @doc "Underline: none."
   defsequence :no_underline, 24
 
-  @doc "Blink: off"
+  @doc "Blink: off."
   defsequence :blink_off, 25
 
+  @doc "Image: positive. Normal foreground and background."
+  defsequence :inverse_off, 27
+
+  @doc "Image: positive. Normal foreground and background."
+  defsequence :reverse_off, 27
+
   colors = [:black, :red, :green, :yellow, :blue, :magenta, :cyan, :white]
 
   for {color, code} <- Enum.with_index(colors) do
-    @doc "Sets foreground color to #{color}"
+    @doc "Sets foreground color to #{color}."
     defsequence color, code + 30
 
-    @doc "Sets background color to #{color}"
+    @doc "Sets foreground color to light #{color}."
+    defsequence :"light_#{color}", code + 90
+
+    @doc "Sets background color to #{color}."
     defsequence :"#{color}_background", code + 40
+
+    @doc "Sets background color to light #{color}."
+    defsequence :"light_#{color}_background", code + 100
   end
 
-  @doc "Default text color"
+  @doc "Default text color."
   defsequence :default_color, 39
 
-  @doc "Default background color"
+  @doc "Default background color."
   defsequence :default_background, 49
 
-  @doc "Framed"
+  @doc "Framed."
   defsequence :framed, 51
 
-  @doc "Encircled"
+  @doc "Encircled."
   defsequence :encircled, 52
 
-  @doc "Overlined"
+  @doc "Overlined."
   defsequence :overlined, 53
 
-  @doc "Not framed or encircled"
+  @doc "Not framed or encircled."
   defsequence :not_framed_encircled, 54
 
-  @doc "Not overlined"
+  @doc "Not overlined."
   defsequence :not_overlined, 55
 
-  @doc "Sends cursor home"
+  @doc "Sends cursor home."
   defsequence :home, "", "H"
 
-  @doc "Clears screen"
+  @doc "Clears screen."
   defsequence :clear, "2", "J"
 
-  @doc "Clears line"
+  @doc "Clears line."
   defsequence :clear_line, "2", "K"
 
   defp format_sequence(other) do
@@ -186,7 +198,7 @@ defmodule IO.ANSI do
       [[[[[[], "Hello, "] | "\e[31m"] | "\e[1m"], "world!"] | "\e[0m"]
 
   """
-  def format(chardata, emit? \\ enabled?) when is_boolean(emit?) do
+  def format(chardata, emit? \\ enabled?()) when is_boolean(emit?) do
     do_format(chardata, [], [], emit?, :maybe)
   end
 
@@ -206,7 +218,7 @@ defmodule IO.ANSI do
       [[[[[[] | "\e[1m"], 87], 111], 114], 100]
 
   """
-  def format_fragment(chardata, emit? \\ enabled?) when is_boolean(emit?) do
+  def format_fragment(chardata, emit? \\ enabled?()) when is_boolean(emit?) do
     do_format(chardata, [], [], emit?, false)
   end
 

lib/elixir/lib/io/ansi/docs.ex

@@ -2,6 +2,7 @@ defmodule IO.ANSI.Docs do
   @moduledoc false
 
   @bullets [?*, ?-, ?+]
+  @spaces [" ", "\n", "\t"]
 
   @doc """
   The default options used by this module.
@@ -10,11 +11,11 @@ defmodule IO.ANSI.Docs do
 
     * `:enabled`           - toggles coloring on and off (true)
     * `:doc_bold`          - bold text (bright)
-    * `:doc_code`          - code blocks (cyan, bright)
-    * `:doc_headings`      - h1 and h2 headings (yellow, bright)
+    * `:doc_code`          - code blocks (cyan)
+    * `:doc_headings`      - h1, h2, h3, h4, h5, h6 headings (yellow)
     * `:doc_inline_code`   - inline code (cyan)
     * `:doc_table_heading` - style for table headings
-    * `:doc_title`         - top level heading (reverse, yellow, bright)
+    * `:doc_title`         - top level heading (reverse, yellow)
     * `:doc_underline`     - underlined text (underline)
     * `:width`             - the width to format the text (80)
 
@@ -24,7 +25,7 @@ defmodule IO.ANSI.Docs do
   def default_options do
     [enabled:           true,
      doc_bold:          [:bright],
-     doc_code:          [:cyan, :bright],
+     doc_code:          [:cyan],
      doc_headings:      [:yellow],
      doc_inline_code:   [:cyan],
      doc_table_heading: [:reverse],
@@ -40,12 +41,12 @@ defmodule IO.ANSI.Docs do
   """
   def print_heading(heading, options \\ []) do
     IO.puts IO.ANSI.reset
-    options = Keyword.merge(default_options, options)
+    options = Keyword.merge(default_options(), options)
     width   = options[:width]
     padding = div(width + String.length(heading), 2)
     heading = heading |> String.pad_leading(padding) |> String.pad_trailing(width)
     write(:doc_title, heading, options)
-    newline_after_block
+    newline_after_block()
   end
 
   @doc """
@@ -55,7 +56,7 @@ defmodule IO.ANSI.Docs do
   defined in `default_options/1`.
   """
   def print(doc, options \\ []) do
-    options = Keyword.merge(default_options, options)
+    options = Keyword.merge(default_options(), options)
     doc
     |> String.split(["\r\n", "\n"], trim: false)
     |> Enum.map(&String.trim_trailing/1)
@@ -66,22 +67,23 @@ defmodule IO.ANSI.Docs do
     write_text(text, indent, options)
   end
 
-  defp process(["# " <> heading | rest], text, indent, options) do
-    write_text(text, indent, options)
-    write_h1(String.trim(heading), options)
-    process(rest, [], "", options)
+  defp process(["# " <> _ = heading | rest], text, indent, options) do
+    write_heading(heading, rest, text, indent, options)
   end
-
-  defp process(["## " <> heading | rest], text, indent, options) do
-    write_text(text, indent, options)
-    write_h2(String.trim(heading), options)
-    process(rest, [], "", options)
+  defp process(["## " <> _ = heading | rest], text, indent, options) do
+    write_heading(heading, rest, text, indent, options)
   end
-
-  defp process(["### " <> heading | rest], text, indent, options) do
-    write_text(text, indent, options)
-    write_h3(String.trim(heading), indent, options)
-    process(rest, [], "", options)
+  defp process(["### " <> _ = heading | rest], text, indent, options) do
+    write_heading(heading, rest, text, indent, options)
+  end
+  defp process(["#### " <> _ = heading | rest], text, indent, options) do
+    write_heading(heading, rest, text, indent, options)
+  end
+  defp process(["##### " <> _ = heading | rest], text, indent, options) do
+    write_heading(heading, rest, text, indent, options)
+  end
+  defp process(["###### " <> _ = heading | rest], text, indent, options) do
+    write_heading(heading, rest, text, indent, options)
   end
 
   defp process(["" | rest], text, indent, options) do
@@ -118,19 +120,11 @@ defmodule IO.ANSI.Docs do
 
   ## Headings
 
-  defp write_h1(heading, options) do
-    write_h2(String.upcase(heading), options)
-  end
-
-  defp write_h2(heading, options) do
+  defp write_heading(heading, rest, text, indent, options) do
+    write_text(text, indent, options)
     write(:doc_headings, heading, options)
-    newline_after_block
-  end
-
-  defp write_h3(heading, indent, options) do
-    IO.write(indent)
-    write(:doc_headings, heading, options)
-    newline_after_block
+    newline_after_block()
+    process(rest, [], "", options)
   end
 
   ## Lists
@@ -211,7 +205,7 @@ defmodule IO.ANSI.Docs do
     |> Enum.join(" ")
     |> handle_links
     |> handle_inline(options)
-    |> String.split(~r{\s})
+    |> String.split(@spaces)
     |> write_with_wrap(options[:width] - byte_size(indent), indent, no_wrap)
 
     unless no_wrap, do: newline_after_block()
@@ -255,8 +249,8 @@ defmodule IO.ANSI.Docs do
   end
 
   defp write_code(code, indent, options) do
-    write(:doc_code, "#{indent}┃ #{Enum.join(Enum.reverse(code), "\n#{indent}┃ ")}", options)
-    newline_after_block
+    write(:doc_code, "#{indent}    #{Enum.join(Enum.reverse(code), "\n#{indent}    ")}", options)
+    newline_after_block()
   end
 
   ## Tables
@@ -264,7 +258,7 @@ defmodule IO.ANSI.Docs do
   defp process_table(lines, indent, options) do
     {table, rest} = Enum.split_while(lines, &table_line?/1)
     table_lines(table, options)
-    newline_after_block
+    newline_after_block()
     process(rest, [], indent, options)
   end
 
@@ -273,8 +267,10 @@ defmodule IO.ANSI.Docs do
     count = Enum.map(lines, &length/1) |> Enum.max
     lines = Enum.map(lines, &pad_to_number_of_columns(&1, count))
 
-    widths = for line <- lines, do:
-              (for {_col, length} <- line, do: length)
+    widths =
+      for line <- lines do
+        for {_col, length} <- line, do: length
+      end
 
     col_widths = Enum.reduce(widths,
                              List.duplicate(0, count),
@@ -287,15 +283,16 @@ defmodule IO.ANSI.Docs do
     line
     |> String.trim("|")
     |> String.trim()
-    |> String.split(~r/\s\|\s/)
+    |> String.split(" | ")
     |> Enum.map(&render_column(&1, options))
   end
 
   defp render_column(col, options) do
-    col = col
-          |> String.replace(~r/\\ \|/x, "|")
-          |> handle_links
-          |> handle_inline(options)
+    col =
+      col
+      |> String.replace("\\\|", "|")
+      |> handle_links
+      |> handle_inline(options)
     {col, length_without_escape(col, 0)}
   end
 
@@ -326,8 +323,17 @@ defmodule IO.ANSI.Docs do
   defp render_table([], _, _),
     do: nil
 
-  defp table_header?(row), do:
-    Enum.all?(row, fn {col, _} -> col =~ ~r/^:?-+:?$/ end)
+  defp table_header?(row) do
+    Enum.all?(row, fn {col, _} -> table_header_column?(col) end)
+  end
+
+  defp table_header_column?(":" <> row), do: table_header_contents?(row)
+  defp table_header_column?(row), do: table_header_contents?(row)
+
+  defp table_header_contents?("-" <> row), do: table_header_contents?(row)
+  defp table_header_contents?(":"), do: true
+  defp table_header_contents?(""), do: true
+  defp table_header_contents?(_), do: false
 
   defp draw_table_row(cols_and_widths, options, heading \\ false) do
     columns =
@@ -343,11 +349,7 @@ defmodule IO.ANSI.Docs do
   end
 
   defp table_line?(line) do
-    Regex.match?(~r'''
-      ( ^ \s{0,3} \| (?: [^|]+ \|)+ \s* $ )
-    |
-      (\s \| \s)
-    '''x, line)
+    line =~ " | "
   end
 
   ## Helpers
@@ -423,14 +425,17 @@ defmodule IO.ANSI.Docs do
   end
 
   defp escape_underlines_in_link(text) do
-    Regex.replace(~r{https?\S*}, text, &String.replace(&1, "_", "\\_"))
+    ~r{https?\S*}
+    |> Regex.recompile!
+    |> Regex.replace(text, &String.replace(&1, "_", "\\_"))
   end
 
   defp remove_square_brackets_in_link(text) do
-    Regex.replace(~r{\[(.*?)\]\((.*?)\)}, text, "\\1 (\\2)")
+    ~r{\[(.*?)\]\((.*?)\)}
+    |> Regex.recompile!
+    |> Regex.replace(text, "\\1 (\\2)")
   end
 
-
   # We have four entries: **, *, _ and `.
   #
   # The first three behave the same while the last one is simpler

lib/elixir/lib/kernel/cli.ex

@@ -76,22 +76,22 @@ defmodule Kernel.CLI do
           fun.(elem(res, 1))
         catch
           :exit, {:shutdown, int} when is_integer(int) ->
-            send parent, {self, {:shutdown, int}}
+            send parent, {self(), {:shutdown, int}}
             exit({:shutdown, int})
           :exit, reason
               when reason == :normal
               when reason == :shutdown
               when tuple_size(reason) == 2 and elem(reason, 0) == :shutdown ->
-            send parent, {self, {:shutdown, 0}}
+            send parent, {self(), {:shutdown, 0}}
             exit(reason)
           kind, reason ->
             stack = System.stacktrace
             print_error(kind, reason, stack)
-            send parent, {self, {:shutdown, 1}}
+            send parent, {self(), {:shutdown, 1}}
             exit(to_exit(kind, reason, stack))
         else
           _ ->
-            send parent, {self, res}
+            send parent, {self(), res}
         end
       end)
 

lib/elixir/lib/kernel/error_handler.ex

@@ -3,16 +3,19 @@
 defmodule Kernel.ErrorHandler do
   @moduledoc false
 
+  @spec undefined_function(module, atom, list) :: term
   def undefined_function(module, fun, args) do
     ensure_loaded(module) or ensure_compiled(module, :module)
     :error_handler.undefined_function(module, fun, args)
   end
 
+  @spec undefined_lambda(module, fun, list) :: term
   def undefined_lambda(module, fun, args) do
     ensure_loaded(module) or ensure_compiled(module, :module)
     :error_handler.undefined_lambda(module, fun, args)
   end
 
+  @spec ensure_loaded(module) :: boolean
   def ensure_loaded(module) do
     case :code.ensure_loaded(module) do
       {:module, _} -> true
@@ -20,15 +23,17 @@ defmodule Kernel.ErrorHandler do
     end
   end
 
+  @spec ensure_compiled(module, atom) :: boolean
   # Never wait on nil because it should never be defined.
   def ensure_compiled(nil, _kind) do
     false
   end
+
   def ensure_compiled(module, kind) do
     parent = :erlang.get(:elixir_compiler_pid)
     ref    = :erlang.make_ref
     send parent, {:waiting, kind, self(), ref, module, :elixir_module.compiler_modules()}
-    :erlang.garbage_collect(self)
+    :erlang.garbage_collect(self())
     receive do
       {^ref, :found}     -> true
       {^ref, :not_found} -> false

lib/elixir/lib/kernel/lexical_tracker.ex

@@ -1,5 +1,6 @@
-# This is an Elixir module responsible for tracking
-# the usage of aliases, imports and requires in the Elixir scope.
+# This is an Elixir module responsible for tracking references
+# to modules, remote dispatches, and the usage of
+# aliases/imports/requires in the Elixir scope.
 #
 # Note that since this is required for bootstrap, we can't use
 # any of the `GenServer.Behaviour` conveniences.
@@ -39,7 +40,7 @@ defmodule Kernel.LexicalTracker do
 
   # Internal API
 
-  # Starts the tracker and returns its pid.
+  # Starts the tracker and returns its PID.
   @doc false
   def start_link(dest) do
     :gen_server.start_link(__MODULE__, dest, [])
@@ -51,32 +52,32 @@ defmodule Kernel.LexicalTracker do
   end
 
   @doc false
-  def add_import(pid, module, fas, line, warn) do
+  def add_import(pid, module, fas, line, warn) when is_atom(module) do
     :gen_server.cast(pid, {:add_import, module, fas, line, warn})
   end
 
   @doc false
-  def add_alias(pid, module, line, warn) do
+  def add_alias(pid, module, line, warn) when is_atom(module) do
     :gen_server.cast(pid, {:add_alias, module, line, warn})
   end
 
   @doc false
-  def remote_reference(pid, module, mode) do
+  def remote_reference(pid, module, mode) when is_atom(module) do
     :gen_server.cast(pid, {:remote_reference, module, mode})
   end
 
   @doc false
-  def remote_dispatch(pid, module, fa, line, mode) do
+  def remote_dispatch(pid, module, fa, line, mode) when is_atom(module) do
     :gen_server.cast(pid, {:remote_dispatch, module, fa, line, mode})
   end
 
   @doc false
-  def import_dispatch(pid, module, fa, line, mode) do
+  def import_dispatch(pid, module, fa, line, mode) when is_atom(module) do
     :gen_server.cast(pid, {:import_dispatch, module, fa, line, mode})
   end
 
   @doc false
-  def alias_dispatch(pid, module) do
+  def alias_dispatch(pid, module) when is_atom(module) do
     :gen_server.cast(pid, {:alias_dispatch, module})
   end
 
@@ -146,19 +147,17 @@ defmodule Kernel.LexicalTracker do
     {:noreply, %{state | directives: add_dispatch(state.directives, module, :alias)}}
   end
 
-  def handle_cast({:add_import, module, fas, line, warn}, state) when is_atom(module) do
+  def handle_cast({:add_import, module, fas, line, warn}, state) do
     directives =
-      for {{:import, {import_module, _, _}}, _} = directive <- state.directives,
-          module != import_module,
-          do: directive,
-          into: %{}
-
-    directives = add_directive(directives, module, line, warn, :import)
+      state.directives
+      |> Enum.reject(&match?({{:import, {^module, _, _}}, _}, &1))
+      |> :maps.from_list
+      |> add_directive(module, line, warn, :import)
 
     directives =
-      Enum.reduce fas, directives, fn {function, arity}, directives ->
+      Enum.reduce(fas, directives, fn {function, arity}, directives ->
         add_directive(directives, {module, function, arity}, line, warn, :import)
-      end
+      end)
 
     {:noreply, %{state | directives: directives}}
   end

lib/elixir/lib/kernel/parallel_compiler.ex

@@ -30,8 +30,8 @@ defmodule Kernel.ParallelCompiler do
     * `:each_module` - for each module compiled, invokes the callback passing
       the file, module and the module bytecode
 
-    * `:dest` - the destination directory for the beam files. When using `files/2`,
-      this information is only used to properly annotate the beam files before
+    * `:dest` - the destination directory for the BEAM files. When using `files/2`,
+      this information is only used to properly annotate the BEAM files before
       they are loaded into memory. If you want a file to actually be written to
       `dest`, use `files_to_path/3` instead.
 
@@ -107,6 +107,7 @@ defmodule Kernel.ParallelCompiler do
       :erlang.spawn_monitor fn ->
         # Set the elixir_compiler_pid used by our custom Kernel.ErrorHandler.
         :erlang.put(:elixir_compiler_pid, parent)
+        :erlang.put(:elixir_compiler_file, file)
         :erlang.process_flag(:error_handler, Kernel.ErrorHandler)
 
         exit(try do
@@ -144,7 +145,7 @@ defmodule Kernel.ParallelCompiler do
     # Instead of releasing all files at once, we release them in groups
     # based on the module they are waiting on. We pick the module being
     # depended on with less edges, as it is the mostly likely source of
-    # error (for example, someone made a type). This may not always be
+    # error (for example, someone made a typo). This may not always be
     # true though: for example, if there is a macro injecting code into
     # multiple modules and such code becomes faulty, now multiple modules
     # are waiting on the same module required by the faulty code. However,
@@ -206,8 +207,10 @@ defmodule Kernel.ParallelCompiler do
 
       {:waiting, kind, child, ref, on, defining} ->
         # Oops, we already got it, do not put it on waiting.
+        # Alternatively, we're waiting on ourselves,
+        # send :found so that we can crash with a better error.
         waiting =
-          if :lists.any(&match?({^kind, ^on}, &1), result) do
+          if :lists.any(&match?({^kind, ^on}, &1), result) or on in defining do
             send child, {ref, :found}
             waiting
           else

lib/elixir/lib/kernel/parallel_require.ex

@@ -47,21 +47,22 @@ defmodule Kernel.ParallelRequire do
     wait_for_messages(files, waiting, callbacks, schedulers, result)
   end
 
-  defp spawn_requires([h | t], waiting, callbacks, schedulers, result) do
+  defp spawn_requires([file | files], waiting, callbacks, schedulers, result) do
     parent = self()
     {pid, ref} = :erlang.spawn_monitor fn ->
       :erlang.put(:elixir_compiler_pid, parent)
+      :erlang.put(:elixir_compiler_file, file)
 
       exit(try do
-        new = Code.require_file(h) || []
-        {:required, Enum.map(new, &elem(&1, 0)), h}
+        new = Code.require_file(file) || []
+        {:required, Enum.map(new, &elem(&1, 0)), file}
       catch
         kind, reason ->
           {:failure, kind, reason, System.stacktrace}
       end)
     end
 
-    spawn_requires(t, [{pid, ref} | waiting], callbacks, schedulers, result)
+    spawn_requires(files, [{pid, ref} | waiting], callbacks, schedulers, result)
   end
 
   defp wait_for_messages(files, waiting, callbacks, schedulers, result) do

lib/elixir/lib/kernel/special_forms.ex

@@ -7,13 +7,11 @@ defmodule Kernel.SpecialForms do
   `alias/2`, `case/2`, etc). The macros `{}` and `<<>>` are also special
   forms used to define tuple and binary data structures respectively.
 
-  This module also documents Elixir's pseudo variables (`__ENV__`,
-  `__MODULE__`, `__DIR__` and `__CALLER__`). Pseudo variables return
-  information about Elixir's compilation environment and can only
-  be read, never assigned to.
+  This module also documents macros that return information about Elixir's
+  compilation environment, such as (`__ENV__/0`, `__MODULE__/0`, `__DIR__/0` and `__CALLER__/0`).
 
-  Finally, it also documents two special forms, `__block__` and
-  `__aliases__`, which are not intended to be called directly by the
+  Finally, it also documents two special forms, `__block__/1` and
+  `__aliases__/1`, which are not intended to be called directly by the
   developer but they appear in quoted contents since they are essential
   in Elixir's constructs.
   """
@@ -29,19 +27,21 @@ defmodule Kernel.SpecialForms do
   @doc """
   Creates a tuple.
 
-  Only two item tuples are considered literals in Elixir.
-  Therefore all other tuples are represented in the AST
-  as a call to the special form `:{}`.
+  More information about the tuple data type and about functions to manipulate
+  tuples can be found in the `Tuple` module; some functions for working with
+  tuples are also available in `Kernel` (such as `Kernel.elem/2` or
+  `Kernel.tuple_size/1`).
 
-  Conveniences for manipulating tuples can be found in the
-  `Tuple` module. Some functions for working with tuples are
-  also available in `Kernel`, namely `Kernel.elem/2`,
-  `Kernel.put_elem/3` and `Kernel.tuple_size/1`.
+  ## AST representation
 
-  ## Examples
+  Only two-item tuples are considered literals in Elixir and return themselves
+  when quoted. Therefore, all other tuples are represented in the AST as calls to
+  the `:{}` special form.
 
-      iex> {1, 2, 3}
-      {1, 2, 3}
+      iex> quote do
+      ...>   {1, 2}
+      ...> end
+      {1, 2}
 
       iex> quote do
       ...>   {1, 2, 3}
@@ -54,56 +54,14 @@ defmodule Kernel.SpecialForms do
   @doc """
   Creates a map.
 
-  Maps are key-value stores where keys are compared
-  using the match operator (`===`). Maps can be created with
-  the `%{}` special form where keys are associated via `=>`:
-
-      %{1 => 2}
-
-  Maps also support the keyword notation, as other special forms,
-  as long as they are at the end of the argument list:
-
-      %{hello: :world, with: :keywords}
-      %{:hello => :world, with: :keywords}
-
-  If a map has duplicated keys, the last key will always have
-  higher precedence:
-
-      iex> %{a: :b, a: :c}
-      %{a: :c}
-
-  Conveniences for manipulating maps can be found in the
-  `Map` module.
-
-  ## Access syntax
-
-  Besides the access functions available in the `Map` module,
-  like `Map.get/3` and `Map.fetch/2`, a map can be accessed using the
-  `.` operator:
-
-      iex> map = %{a: :b}
-      iex> map.a
-      :b
-
-  Note that the `.` operator expects the key `:a` to exist in the map.
-  If not, an `ArgumentError` is raised.
-
-  ## Update syntax
-
-  Maps also support an update syntax:
-
-      iex> map = %{:a => :b}
-      iex> %{map | :a => :c}
-      %{:a => :c}
-
-  Notice the update syntax requires the given keys to exist.
-  Trying to update a key that does not exist will raise an `KeyError`.
+  See the `Map` module for more information about maps, their syntax, and ways to
+  access and manipulate them.
 
   ## AST representation
 
-  Regardless if `=>` or the keywords syntax is used, Maps are
-  always represented internally as a list of two-element tuples
-  for simplicity:
+  Regardless of whether `=>` or the keyword syntax is used, key-value pairs in
+  maps are always represented internally as a list of two-element tuples for
+  simplicity:
 
       iex> quote do
       ...>   %{"a" => :b, c: :d}
@@ -173,10 +131,10 @@ defmodule Kernel.SpecialForms do
 
   - `integer`
   - `float`
-  - `bits` (alias for bitstring)
+  - `bits` (alias for `bitstring`)
   - `bitstring`
   - `binary`
-  - `bytes` (alias for binary)
+  - `bytes` (alias for `binary`)
   - `utf8`
   - `utf16`
   - `utf32`
@@ -198,13 +156,13 @@ defmodule Kernel.SpecialForms do
       iex> <<102, rest>>
       ** (ArgumentError) argument error
 
-  We can solve this by explicitly tagging it as a binary:
+  We can solve this by explicitly tagging it as `binary`:
 
       iex> rest = "oo"
       iex> <<102, rest::binary>>
       "foo"
 
-  The utf8, utf16, and utf32 types are for Unicode codepoints. They
+  The `utf8`, `utf16`, and `utf32` types are for Unicode codepoints. They
   can also be applied to literal strings and charlists:
 
       iex> <<"foo"::utf16>>
@@ -264,7 +222,7 @@ defmodule Kernel.SpecialForms do
       iex> x = 1
       iex> <<x::8>> == <<x::size(8)>>
       true
-      iex> <<x::8 * 4>> == <<x::size(8)-unit(4)>>
+      iex> <<x::8*4>> == <<x::size(8)-unit(4)>>
       true
 
   This syntax reflects the fact the effective size is given by
@@ -279,8 +237,8 @@ defmodule Kernel.SpecialForms do
   -------------------- | ----------------
   `signed`             | `integer`
   `unsigned` (default) | `integer`
-  `little`             | `integer`, `utf16`, `utf32`
-  `big` (default)      | `integer`, `utf16`, `utf32`
+  `little`             | `integer`, `float`, `utf16`, `utf32`
+  `big` (default)      | `integer`, `float`, `utf16`, `utf32`
   `native`             | `integer`, `utf16`, `utf32`
 
   ### Sign
@@ -335,7 +293,7 @@ defmodule Kernel.SpecialForms do
 
       defmodule ImageTyper
         @png_signature <<137::size(8), 80::size(8), 78::size(8), 71::size(8),
-                      13::size(8), 10::size(8), 26::size(8), 10::size(8)>>
+                         13::size(8), 10::size(8), 26::size(8), 10::size(8)>>
         @jpg_signature <<255::size(8), 216::size(8)>>
 
         def type(<<@png_signature, rest::binary>>), do: :png
@@ -397,7 +355,7 @@ defmodule Kernel.SpecialForms do
       iex> Kernel.'+'(1, 2)
       3
 
-  Note that `Kernel."HELLO"` will be treated as a remote call and not an alias.
+  Note that `Kernel."FUNCTION_NAME"` will be treated as a remote call and not an alias.
   This choice was done so every time single- or double-quotes are used, we have
   a remote call regardless of the quote contents. This decision is also reflected
   in the quoted expressions discussed below.
@@ -439,7 +397,7 @@ defmodule Kernel.SpecialForms do
       ...> end
       {:__aliases__, [alias: false], [:Hello, :World]}
 
-  We go into more details about aliases in the `__aliases__` special form
+  We go into more details about aliases in the `__aliases__/1` special form
   documentation.
 
   ## Unquoting
@@ -452,7 +410,7 @@ defmodule Kernel.SpecialForms do
       ...> end
       {{:., [], [{:__aliases__, [alias: false], [:String]}, :downcase]}, [], ["FOO"]}
 
-  Similar to `Kernel."HELLO"`, `unquote(x)` will always generate a remote call,
+  Similar to `Kernel."FUNCTION_NAME"`, `unquote(x)` will always generate a remote call,
   independent of the value of `x`. To generate an alias via the quoted expression,
   one needs to rely on `Module.concat/2`:
 
@@ -496,6 +454,16 @@ defmodule Kernel.SpecialForms do
 
       alias Foo.Bar.Baz, as: Baz
 
+  We can also alias multiple modules in one line:
+
+      alias Foo.{Bar, Baz, Biz}
+
+  Is the same as:
+
+      alias Foo.Bar
+      alias Foo.Baz
+      alias Foo.Biz
+
   ## Lexical scope
 
   `import/2`, `require/2` and `alias/2` are called directives and all
@@ -654,7 +622,7 @@ defmodule Kernel.SpecialForms do
   @doc """
   Returns the current module name as an atom or `nil` otherwise.
 
-  Although the module can be accessed in the `__ENV__`, this macro
+  Although the module can be accessed in the `__ENV__/0`, this macro
   is a convenient shortcut.
   """
   defmacro __MODULE__, do: error!([])
@@ -1069,8 +1037,8 @@ defmodule Kernel.SpecialForms do
 
       Hygiene.return_length #=> 3
 
-  Notice how `return_length` returns 3 even though the `length/1`
-  function is not imported. In fact, even if `return_length`
+  Notice how `Hygiene.return_length/0` returns `3` even though the `Kernel.length/1`
+  function is not imported. In fact, even if `return_length/0`
   imported a function with the same name and arity from another
   module, it wouldn't affect the function result:
 
@@ -1079,10 +1047,10 @@ defmodule Kernel.SpecialForms do
         get_length
       end
 
-  Calling this new `return_length` will still return 3 as result.
+  Calling this new `return_length/0` will still return `3` as result.
 
   Elixir is smart enough to delay the resolution to the latest
-  moment possible. So, if you call `length([1, 2, 3])` inside quote,
+  possible moment. So, if you call `length([1, 2, 3])` inside quote,
   but no `length/1` function is available, it is then expanded in
   the caller:
 
@@ -1273,7 +1241,7 @@ defmodule Kernel.SpecialForms do
       [2, 4, 6, 8]
 
       # A comprehension with two generators
-      iex> for x <- [1, 2], y <- [2, 3], do: x*y
+      iex> for x <- [1, 2], y <- [2, 3], do: x * y
       [2, 3, 4, 6]
 
   Filters can also be given:
@@ -1295,7 +1263,7 @@ defmodule Kernel.SpecialForms do
   need to organize bitstring streams:
 
       iex> pixels = <<213, 45, 132, 64, 76, 32, 76, 0, 0, 234, 32, 15>>
-      iex> for <<r::8, g::8, b::8 <- pixels >>, do: {r, g, b}
+      iex> for <<r::8, g::8, b::8 <- pixels>>, do: {r, g, b}
       [{213, 45, 132}, {64, 76, 32}, {76, 0, 0}, {234, 32, 15}]
 
   Variable assignments inside the comprehension, be it in generators,
@@ -1333,7 +1301,7 @@ defmodule Kernel.SpecialForms do
       iex> opts = %{width: 10, height: 15}
       iex> with {:ok, width} <- Map.fetch(opts, :width),
       ...>      {:ok, height} <- Map.fetch(opts, :height),
-      ...>   do: {:ok, width * height}
+      ...>      do: {:ok, width * height}
       {:ok, 150}
 
   If all clauses match, the `do` block is executed, returning its result.
@@ -1342,14 +1310,14 @@ defmodule Kernel.SpecialForms do
       iex> opts = %{width: 10}
       iex> with {:ok, width} <- Map.fetch(opts, :width),
       ...>      {:ok, height} <- Map.fetch(opts, :height),
-      ...>   do: {:ok, width * height}
+      ...>      do: {:ok, width * height}
       :error
 
   Guards can be used in patterns as well:
 
       iex> users = %{"melany" => "guest", "bob" => :admin}
       iex> with {:ok, role} when not is_binary(role) <- Map.fetch(users, "bob"),
-      ...>   do: {:ok, to_string(role)}
+      ...>      do: {:ok, to_string(role)}
       {:ok, "admin"}
 
   As in `for/1`, variables bound inside `with/1` won't leak;
@@ -1360,7 +1328,7 @@ defmodule Kernel.SpecialForms do
       iex> with {:ok, width} <- Map.fetch(opts, :width),
       ...>      double_width = width * 2,
       ...>      {:ok, height} <- Map.fetch(opts, :height),
-      ...>   do: {:ok, double_width * height}
+      ...>      do: {:ok, double_width * height}
       {:ok, 300}
       iex> width
       nil
@@ -1783,7 +1751,7 @@ defmodule Kernel.SpecialForms do
             :infinity
         end
 
-  However when an else clause is present but the result of the expression
+  However, when an else clause is present but the result of the expression
   does not match any of the patterns an exception will be raised. This
   exception will not be caught by a catch or rescue in the same try:
 
@@ -1805,7 +1773,7 @@ defmodule Kernel.SpecialForms do
           :error_b
       end
 
-  Similarly an exception inside an else clause is not caught or rescued
+  Similarly, an exception inside an else clause is not caught or rescued
   inside the same try:
 
       try do
@@ -1881,7 +1849,7 @@ defmodule Kernel.SpecialForms do
       end
 
   An optional `after` clause can be given in case the message was not
-  received after the specified timeout period:
+  received after the given timeout period, specified in milliseconds:
 
       receive do
         {:selector, i, value} when is_integer(i) ->

lib/elixir/lib/kernel/typespec.ex

@@ -108,6 +108,7 @@ defmodule Kernel.Typespec do
   @doc """
   Defines a `spec` by receiving a typespec expression.
   """
+  @spec define_spec(atom, Macro.t, Macro.Env.t) :: Keyword.t
   def define_spec(kind, expr, env) do
     defspec(kind, expr, env)
   end
@@ -117,7 +118,9 @@ defmodule Kernel.Typespec do
   (private, opaque or not). This function is only available
   for modules being compiled.
   """
-  def defines_type?(module, name, arity) do
+  @spec defines_type?(module, atom, arity) :: boolean
+  def defines_type?(module, name, arity)
+      when is_atom(module) and is_atom(name) and arity in 0..255 do
     finder = fn {_kind, expr, _caller} ->
       type_to_signature(expr) == {name, arity}
     end
@@ -129,7 +132,9 @@ defmodule Kernel.Typespec do
   Returns `true` if the current module defines a given spec.
   This function is only available for modules being compiled.
   """
-  def defines_spec?(module, name, arity) do
+  @spec defines_spec?(module, atom, arity) :: boolean
+  def defines_spec?(module, name, arity)
+      when is_atom(module) and is_atom(name) and arity in 0..255 do
     finder = fn {_kind, expr, _caller} ->
       spec_to_signature(expr) == {name, arity}
     end
@@ -140,7 +145,9 @@ defmodule Kernel.Typespec do
   Returns `true` if the current module defines a callback.
   This function is only available for modules being compiled.
   """
-  def defines_callback?(module, name, arity) do
+  @spec defines_callback?(module, atom, arity) :: boolean
+  def defines_callback?(module, name, arity)
+      when is_atom(module) and is_atom(name) and arity in 0..255 do
     finder = fn {_kind, expr, _caller} ->
       spec_to_signature(expr) == {name, arity}
     end
@@ -150,8 +157,10 @@ defmodule Kernel.Typespec do
   @doc """
   Converts a spec clause back to Elixir AST.
   """
+  @spec spec_to_ast(atom, tuple) :: {atom, Keyword.t, [Macro.t]}
   def spec_to_ast(name, spec)
-  def spec_to_ast(name, {:type, line, :fun, [{:type, _, :product, args}, result]}) do
+  def spec_to_ast(name, {:type, line, :fun, [{:type, _, :product, args}, result]})
+      when is_atom(name) do
     meta = [line: line]
     body = {name, meta, Enum.map(args, &typespec_to_ast/1)}
 
@@ -169,11 +178,12 @@ defmodule Kernel.Typespec do
     end
   end
 
-  def spec_to_ast(name, {:type, line, :fun, []}) do
+  def spec_to_ast(name, {:type, line, :fun, []}) when is_atom(name) do
     {:::, [line: line], [{name, [line: line], []}, quote(do: term)]}
   end
 
-  def spec_to_ast(name, {:type, line, :bounded_fun, [{:type, _, :fun, [{:type, _, :product, args}, result]}, constraints]}) do
+  def spec_to_ast(name, {:type, line, :bounded_fun, [{:type, _, :fun, [{:type, _, :product, args}, result]}, constraints]})
+      when is_atom(name) do
     guards =
       for {:type, _, :constraint, [{:atom, _, :is_subtype}, [{:var, _, var}, type]]} <- constraints do
         {var, typespec_to_ast(type)}
@@ -206,7 +216,7 @@ defmodule Kernel.Typespec do
     quote do: unquote(record)(unquote_splicing(args)) :: unquote(type)
   end
 
-  def type_to_ast({name, type, args}) do
+  def type_to_ast({name, type, args}) when is_atom(name) do
     args = for arg <- args, do: typespec_to_ast(arg)
     quote do: unquote(name)(unquote_splicing(args)) :: unquote(typespec_to_ast(type))
   end
@@ -222,12 +232,12 @@ defmodule Kernel.Typespec do
   end
 
   @doc """
-  Returns all types available from the module's beam code.
+  Returns all types available from the module's BEAM code.
 
   The result is returned as a list of tuples where the first
   element is the type (`:typep`, `:type` and `:opaque`).
 
-  The module must have a corresponding beam file which can be
+  The module must have a corresponding BEAM file which can be
   located by the runtime system.
   """
   @spec beam_types(module | binary) :: [tuple] | nil
@@ -250,12 +260,12 @@ defmodule Kernel.Typespec do
   end
 
   @doc """
-  Returns all specs available from the module's beam code.
+  Returns all specs available from the module's BEAM code.
 
   The result is returned as a list of tuples where the first
   element is spec name and arity and the second is the spec.
 
-  The module must have a corresponding beam file which can be
+  The module must have a corresponding BEAM file which can be
   located by the runtime system.
   """
   @spec beam_specs(module | binary) :: [tuple] | nil
@@ -264,12 +274,12 @@ defmodule Kernel.Typespec do
   end
 
   @doc """
-  Returns all callbacks available from the module's beam code.
+  Returns all callbacks available from the module's BEAM code.
 
   The result is returned as a list of tuples where the first
   element is spec name and arity and the second is the spec.
 
-  The module must have a corresponding beam file
+  The module must have a corresponding BEAM file
   which can be located by the runtime system.
   """
   @spec beam_callbacks(module | binary) :: [tuple] | nil
@@ -348,7 +358,7 @@ defmodule Kernel.Typespec do
 
   defp get_doc_info(table, attr, caller) do
     case :ets.take(table, attr) do
-      [{^attr, {line, doc}}] -> {line, doc}
+      [{^attr, {line, doc}, _, _}] -> {line, doc}
       [] -> {caller.line, nil}
     end
   end
@@ -448,7 +458,7 @@ defmodule Kernel.Typespec do
   defp translate_spec(kind, meta, name, args, return, guard, caller) when is_atom(args),
     do: translate_spec(kind, meta, name, [], return, guard, caller)
   defp translate_spec(:macrocallback, meta, name, args, return, guard, caller),
-    do: translate_spec(:callback, meta, :"MACRO-#{name}", [quote(do: env :: Macro.Env.t) | args], return, guard, caller)
+    do: translate_spec(:callback, meta, :"MACRO-#{name}", macro_args(args), return, guard, caller)
   defp translate_spec(kind, meta, name, args, return, guard, caller) do
     ensure_no_defaults!(args)
 
@@ -470,6 +480,10 @@ defmodule Kernel.Typespec do
     {{kind, {name, arity}, spec}, caller.line}
   end
 
+  defp macro_args(args) do
+    [quote(do: {line :: Macro.Env.line, env :: Macro.Env.t}) | args]
+  end
+
   defp ensure_no_defaults!(args) do
     :lists.foreach fn
       {:::, _, [left, right]} ->
@@ -574,7 +588,7 @@ defmodule Kernel.Typespec do
   defp typespec_to_ast({:type, line, :map, fields}) do
     fields = Enum.map fields, fn
       {:type, _, :map_field_assoc, :any} ->
-        {:..., [line: line], nil}
+        {{:optional, [], [{:any, [], []}]}, {:any, [], []}}
       {:type, _, :map_field_exact, [{:atom, _, k}, v]} ->
         {k, typespec_to_ast(v)}
       {:type, _, :map_field_exact, [k, v]} ->
@@ -746,8 +760,6 @@ defmodule Kernel.Typespec do
   defp typespec({:%{}, meta, fields} = map, vars, caller) do
     fields =
       :lists.map(fn
-        :... ->
-          {:type, line(meta), :map_field_assoc, :any}
         {k, v} when is_atom(k) ->
           {:type, line(meta), :map_field_exact, [typespec(k, vars, caller), typespec(v, vars, caller)]}
         {{:required, meta2, [k]}, v} ->
@@ -773,6 +785,8 @@ defmodule Kernel.Typespec do
   end
 
   defp typespec({:%, _, [name, {:%{}, meta, fields}]}, vars, caller) do
+    # We cannot set a function name to avoid tracking
+    # as a compile time dependency, because for structs it actually is one.
     module = Macro.expand(name, caller)
 
     struct =
@@ -809,6 +823,8 @@ defmodule Kernel.Typespec do
   end
 
   defp typespec({:record, meta, [atom, fields]}, vars, caller) do
+    # We cannot set a function name to avoid tracking
+    # as a compile time dependency because for records it actually is one.
     case Macro.expand({atom, [], [{atom, [], []}]}, caller) do
       keyword when is_list(keyword) ->
         types =
@@ -864,9 +880,24 @@ defmodule Kernel.Typespec do
     {:op, line(meta), op, {:integer, line(meta), integer}}
   end
 
+  # Handle remote calls in the form of @module_attribute.type.
+  # These are not handled by the general remote type clause as calling
+  # Macro.expand/2 on the remote does not expand module attributes (but expands
+  # things like __MODULE__).
+  defp typespec({{:., meta, [{:@, _, [{attr, _, _}]}, name]}, _, args} = orig, vars, caller) do
+    remote = Module.get_attribute(caller.module, attr)
+    unless is_atom(remote) and remote != nil do
+      message = "invalid remote in typespec: #{Macro.to_string(orig)} (@#{attr} is #{inspect remote})"
+      compile_error(caller, message)
+    end
+    remote_type({typespec(remote, vars, caller), meta, typespec(name, vars, caller), args}, vars, caller)
+  end
+
   # Handle remote calls
   defp typespec({{:., meta, [remote, name]}, _, args} = orig, vars, caller) do
-    remote = Macro.expand remote, caller
+    # We set a function name to avoid tracking
+    # aliases in typespecs as compile time dependencies.
+    remote = Macro.expand(remote, %{caller | function: {:typespec, 0}})
     unless is_atom(remote) do
       compile_error(caller, "invalid remote in typespec: #{Macro.to_string(orig)}")
     end

lib/elixir/lib/kernel/utils.ex

@@ -3,8 +3,13 @@ import Kernel, except: [destructure: 2, defdelegate: 2, defstruct: 2]
 defmodule Kernel.Utils do
   @moduledoc false
 
-  def destructure(list, count) when is_list(list), do: destructure_list(list, count)
-  def destructure(nil, count), do: destructure_nil(count)
+  @doc """
+  Callback for destructure.
+  """
+  def destructure(list, count) when is_list(list) and is_integer(count) and count >= 0,
+    do: destructure_list(list, count)
+  def destructure(nil, count) when is_integer(count) and count >= 0,
+    do: destructure_nil(count)
 
   defp destructure_list(_, 0), do: []
   defp destructure_list([], count), do: destructure_nil(count)
@@ -13,7 +18,11 @@ defmodule Kernel.Utils do
   defp destructure_nil(0), do: []
   defp destructure_nil(count), do: [nil | destructure_nil(count - 1)]
 
-  def defdelegate(fun, opts) do
+  @doc """
+  Callback for defdelegate.
+  """
+  def defdelegate(fun, opts) when is_list(opts) do
+    # TODO: Remove by 2.0
     append_first = Keyword.get(opts, :append_first, false)
 
     {name, args} =
@@ -56,6 +65,9 @@ defmodule Kernel.Utils do
       "defdelegate/2 only accepts function parameters, got: #{Macro.to_string(code)}"
   end
 
+  @doc """
+  Callback for defstruct.
+  """
   def defstruct(module, fields) do
     case fields do
       fs when is_list(fs) ->
@@ -84,4 +96,31 @@ defmodule Kernel.Utils do
      List.wrap(Module.get_attribute(module, :enforce_keys)),
      Module.get_attribute(module, :derive)}
   end
+
+  @doc """
+  Announcing callback for defstruct.
+  """
+  def announce_struct(module) do
+    case :erlang.get(:elixir_compiler_pid) do
+      :undefined -> :ok
+      pid -> send(pid, {:struct_available, module})
+    end
+  end
+
+  @doc """
+  Callback for raise.
+  """
+  def raise(msg) when is_binary(msg) do
+    RuntimeError.exception(msg)
+  end
+  def raise(atom) when is_atom(atom) do
+    atom.exception([])
+  end
+  def raise(%{__struct__: struct, __exception__: true} = exception) when is_atom(struct) do
+    exception
+  end
+  def raise(other) do
+    ArgumentError.exception("raise/1 expects an alias, string or exception as " <>
+                            "the first argument, got: #{inspect other}")
+  end
 end

lib/elixir/lib/keyword.ex

@@ -25,7 +25,7 @@ defmodule Keyword do
   The functions in Keyword do not guarantee any property when
   it comes to ordering. However, since a keyword list is simply a
   list, all the operations defined in `Enum` and `List` can be
-  applied too, specially when ordering is required.
+  applied too, especially when ordering is required.
   """
 
   @compile :inline_list_funcs
@@ -226,18 +226,26 @@ defmodule Keyword do
 
   defp get_and_update([{key, current} | t], acc, key, fun) do
     case fun.(current) do
-      {get, value} -> {get, :lists.reverse(acc, [{key, value} | t])}
-      :pop         -> {current, :lists.reverse(acc, t)}
+      {get, value} ->
+        {get, :lists.reverse(acc, [{key, value} | t])}
+      :pop ->
+        {current, :lists.reverse(acc, t)}
+      other ->
+        raise "the given function must return a two-element tuple or :pop, got: #{inspect(other)}"
     end
   end
 
-  defp get_and_update([h | t], acc, key, fun),
+  defp get_and_update([{_, _} = h | t], acc, key, fun),
     do: get_and_update(t, [h | acc], key, fun)
 
   defp get_and_update([], acc, key, fun) do
     case fun.(nil) do
-      {get, update} -> {get, [{key, update} | :lists.reverse(acc)]}
-      :pop -> {nil, :lists.reverse(acc)}
+      {get, update} ->
+        {get, [{key, update} | :lists.reverse(acc)]}
+      :pop ->
+        {nil, :lists.reverse(acc)}
+      other ->
+        raise "the given function must return a two-element tuple or :pop, got: #{inspect(other)}"
     end
   end
 
@@ -281,6 +289,8 @@ defmodule Keyword do
         {get, :lists.reverse(acc, [{key, value} | delete(keywords, key)])}
       :pop ->
         {value, :lists.reverse(acc, keywords)}
+      other ->
+        raise "the given function must return a two-element tuple or :pop, got: #{inspect(other)}"
     end
   end
 
@@ -561,11 +571,23 @@ defmodule Keyword do
       iex> Keyword.merge([a: 1, b: 2], [a: 3, d: 4, a: 5])
       [b: 2, a: 3, d: 4, a: 5]
 
+      iex> Keyword.merge([a: 1], [2, 3])
+      ** (ArgumentError) expected a keyword list as the second argument, got: [2, 3]
+
   """
   @spec merge(t, t) :: t
   def merge(keywords1, keywords2) when is_list(keywords1) and is_list(keywords2) do
-    fun = fn {k, _v} -> not has_key?(keywords2, k) end
-    :lists.filter(fun, keywords1) ++ keywords2
+    if keyword?(keywords2) do
+      fun = fn
+        {key, _value} when is_atom(key) ->
+          not has_key?(keywords2, key)
+        _ ->
+          raise ArgumentError, message: "expected a keyword list as the first argument, got: #{inspect keywords1}"
+      end
+      :lists.filter(fun, keywords1) ++ keywords2
+    else
+      raise ArgumentError, message: "expected a keyword list as the second argument, got: #{inspect keywords2}"
+    end
   end
 
   @doc """
@@ -596,26 +618,40 @@ defmodule Keyword do
       ...> end)
       [b: 2, a: 4, d: 4, a: 8]
 
+      iex> Keyword.merge([a: 1, b: 2], [:a, :b], fn :a, v1, v2 ->
+      ...>  v1 + v2
+      ...> end)
+      ** (ArgumentError) expected a keyword list as the second argument, got: [:a, :b]
+
   """
   @spec merge(t, t, (key, value, value -> value)) :: t
-  def merge(keywords1, keywords2, fun) when is_list(keywords1) and is_list(keywords2) do
-    do_merge(keywords2, [], keywords1, keywords1, fun)
-  end
-
-  defp do_merge([{k, v2} | t], acc, rest, original, fun) do
-    case :lists.keyfind(k, 1, original) do
-      {^k, v1} ->
-        do_merge(t, [{k, fun.(k, v1, v2)} | acc],
-                 delete(rest, k), :lists.keydelete(k, 1, original), fun)
-      false ->
-        do_merge(t, [{k, v2} | acc], rest, original, fun)
+  def merge(keywords1, keywords2, fun) when is_list(keywords1) and is_list(keywords2) and is_function(fun, 3) do
+    if keyword?(keywords1) do
+      do_merge(keywords2, [], keywords1, keywords1, fun, keywords2)
+    else
+      raise ArgumentError, message: "expected a keyword list as the first argument, got: #{inspect keywords1}"
     end
   end
 
-  defp do_merge([], acc, rest, _original, _fun) do
+  defp do_merge([{key, value2} | tail], acc, rest, original, fun, keywords2) when is_atom(key) do
+    case :lists.keyfind(key, 1, original) do
+      {^key, value1} ->
+        do_merge(tail, [{key, fun.(key, value1, value2)} | acc],
+                 delete(rest, key), :lists.keydelete(key, 1, original), fun, keywords2)
+
+      false ->
+        do_merge(tail, [{key, value2} | acc], rest, original, fun, keywords2)
+    end
+  end
+
+  defp do_merge([], acc, rest, _original, _fun, _keywords2) do
     rest ++ :lists.reverse(acc)
   end
 
+  defp do_merge(_other, _acc, _rest, _original, _fun, keywords2) do
+    raise ArgumentError, message: "expected a keyword list as the second argument, got: #{inspect keywords2}"
+  end
+
   @doc """
   Returns whether a given `key` exists in the given `keywords`.
 
@@ -719,6 +755,7 @@ defmodule Keyword do
       {[a: 1, c: 3, a: 4], [b: 2]}
 
   """
+  @spec split(t, [key]) :: {t, t}
   def split(keywords, keys) when is_list(keywords) do
     fun = fn {k, v}, {take, drop} ->
       case k in keys do
@@ -746,6 +783,7 @@ defmodule Keyword do
       [a: 1, c: 3, a: 5]
 
   """
+  @spec take(t, [key]) :: t
   def take(keywords, keys) when is_list(keywords) do
     :lists.filter(fn {k, _} -> k in keys end, keywords)
   end
@@ -763,6 +801,7 @@ defmodule Keyword do
       [a: 1, c: 3, a: 5]
 
   """
+  @spec drop(t, [key]) :: t
   def drop(keywords, keys) when is_list(keywords) do
     :lists.filter(fn {k, _} -> not k in keys end, keywords)
   end
@@ -835,13 +874,13 @@ defmodule Keyword do
 
   ## Examples
 
-      iex> Keyword.pop_first [a: 1], :a
+      iex> Keyword.pop_first([a: 1], :a)
       {1, []}
-      iex> Keyword.pop_first [a: 1], :b
+      iex> Keyword.pop_first([a: 1], :b)
       {nil, [a: 1]}
-      iex> Keyword.pop_first [a: 1], :b, 3
+      iex> Keyword.pop_first([a: 1], :b, 3)
       {3, [a: 1]}
-      iex> Keyword.pop_first [a: 1, a: 2], :a
+      iex> Keyword.pop_first([a: 1, a: 2], :a)
       {1, [a: 2]}
 
   """
@@ -862,6 +901,7 @@ defmodule Keyword do
       [a: 1]
 
   """
+  @spec to_list(t) :: t
   def to_list(keyword) when is_list(keyword) do
     keyword
   end

lib/elixir/lib/list.ex

@@ -1,24 +1,66 @@
 defmodule List do
   @moduledoc """
-  Specialized functions that only work on lists.
+  Functions that work on (linked) lists.
 
-  In general, favor using the `Enum` API instead of `List`.
+  Lists in Elixir are specified between square brackets:
 
-  Index access for list is linear. Negative indexes are also
-  supported but they imply the list will be iterated twice,
-  one to calculate the proper index and another to perform the
-  operation.
+      iex> [1, "two", 3, :four]
+      [1, "two", 3, :four]
 
-  A decision was taken to delegate most functions to
-  Erlang's standard library but follow Elixir's convention
-  of receiving the subject (in this case, a list) as the
-  first argument.
+  Two lists can be concatenated and subtracted using the
+  `Kernel.++/2` and `Kernel.--/2` operators:
+
+      iex> [1, 2, 3] ++ [4, 5, 6]
+      [1, 2, 3, 4, 5, 6]
+      iex> [1, true, 2, false, 3, true] -- [true, false]
+      [1, 2, 3, true]
+
+  Lists in Elixir are effectively linked lists, which means
+  they are internally represented in pairs containing the
+  head and the tail of a list:
+
+      iex> [head | tail] = [1, 2, 3]
+      iex> head
+      1
+      iex> tail
+      [2, 3]
+
+  Similarly, we could write the list `[1, 2, 3]` using only
+  such pairs (called cons cells):
+
+      iex> [1 | [2 | [3 | []]]]
+      [1, 2, 3]
+
+  Some lists, called improper lists, do not have an empty list as
+  the second element in the last cons cell:
+
+      iex> [1 | [2 | [3 | 4]]]
+      [1, 2, 3 | 4]
+
+  Although improper lists are generally avoided, they are used in some
+  special circumstances like iodata and chardata entities (see the `IO` module).
+
+  Due to their cons cell based representation, prepending an element
+  to a list is always fast (constant time), while appending becomes
+  slower as the list grows in size (linear time):
+
+      iex> list = [1, 2, 3]
+      iex> [0 | list]   # fast
+      [0, 1, 2, 3]
+      iex> list ++ [4]  # slow
+      [1, 2, 3, 4]
+
+  The `Kernel` module contains many functions to manipulate lists
+  and that are allowed in guards. For example, `Kernel.hd/1` to
+  retrieve the head, `Kernel.tl/1` to fetch the tail and
+  `Kernel.length/1` for calculating the length. Keep in mind that,
+  similar to appending to a list, calculating the length needs to
+  traverse the whole list.
 
   ## Charlists
 
-  If a list is made of non-negative integers, it can also
-  be called as a charlist. Elixir uses single quotes to
-  define charlists:
+  If a list is made of non-negative integers, it can also be called
+  a charlist. Elixir uses single quotes to define charlists:
 
       iex> 'héllo'
       [104, 233, 108, 108, 111]
@@ -32,7 +74,7 @@ defmodule List do
   The rationale behind this behaviour is to better support
   Erlang libraries which may return text as charlists
   instead of Elixir strings. One example of such functions
-  is `Application.loaded_applications`:
+  is `Application.loaded_applications/0`:
 
       Application.loaded_applications
       #=>  [{:stdlib, 'ERTS  CXC 138 10', '2.6'},
@@ -40,22 +82,37 @@ defmodule List do
             {:elixir, 'elixir', '1.0.0'},
             {:kernel, 'ERTS  CXC 138 10', '4.1'},
             {:logger, 'logger', '1.0.0'}]
+
+  ## List and Enum modules
+
+  This module aims to provide operations that are specific
+  to lists, like conversion between data types, updates,
+  deletions and key lookups (for lists of tuples). For traversing
+  lists in general, developers should use the functions in the
+  `Enum` module that work across a variety of data types.
+
+  In both `Enum` and `List` modules, any kind of index access
+  on a list is linear. Negative indexes are also supported but
+  they imply the list will be iterated twice, one to calculate
+  the proper index and another to perform the operation.
   """
 
   @compile :inline_list_funcs
 
   @doc """
-  Deletes the given item from the list. Returns a list without
-  the item. If the item occurs more than once in the list, just
+  Deletes the given `item` from the `list`. Returns a new list without
+  the item.
+
+  If the `item` occurs more than once in the `list`, just
   the first occurrence is removed.
 
   ## Examples
 
-      iex> List.delete([1, 2, 3], 1)
-      [2, 3]
+      iex> List.delete([:a, :b, :c], :a)
+      [:b, :c]
 
-      iex> List.delete([1, 2, 2, 3], 2)
-      [1, 2, 3]
+      iex> List.delete([:a, :b, :b, :c], :b)
+      [:a, :b, :c]
 
   """
   @spec delete(list, any) :: list
@@ -74,7 +131,6 @@ defmodule List do
       iex> List.duplicate([1, 2], 2)
       [[1, 2], [1, 2]]
 
-
   """
   @spec duplicate(elem, non_neg_integer) :: [elem] when elem: var
   def duplicate(elem, n) do
@@ -161,7 +217,7 @@ defmodule List do
   """
   @spec first([elem]) :: nil | elem when elem: var
   def first([]),      do: nil
-  def first([h | _]), do: h
+  def first([head | _]), do: head
 
   @doc """
   Returns the last element in `list` or `nil` if `list` is empty.
@@ -179,9 +235,9 @@ defmodule List do
 
   """
   @spec last([elem]) :: nil | elem when elem: var
-  def last([]),      do: nil
-  def last([h]),     do: h
-  def last([_ | t]), do: last(t)
+  def last([]), do: nil
+  def last([head]), do: head
+  def last([_ | tail]), do: last(tail)
 
   @doc """
   Receives a list of tuples and returns the first tuple
@@ -222,7 +278,7 @@ defmodule List do
       false
 
   """
-  @spec keymember?([tuple], any, non_neg_integer) :: any
+  @spec keymember?([tuple], any, non_neg_integer) :: boolean
   def keymember?(list, key, position) do
     :lists.keymember(key, position + 1, list)
   end
@@ -261,9 +317,10 @@ defmodule List do
   end
 
   @doc """
-  Receives a list of tuples and replaces the item
-  identified by `key` at `position`. If the item
-  does not exist, it is added to the end of the list.
+  Receives a `list` of tuples and replaces the item
+  identified by `key` at `position`.
+
+  If the item does not exist, it is added to the end of the `list`.
 
   ## Examples
 
@@ -280,7 +337,7 @@ defmodule List do
   end
 
   @doc """
-  Receives a list of tuples and deletes the first tuple
+  Receives a `list` of tuples and deletes the first tuple
   where the item at `position` matches the
   given `key`. Returns the new list.
 
@@ -330,6 +387,7 @@ defmodule List do
 
   @doc """
   Wraps the argument in a list.
+
   If the argument is already a list, returns the list.
   If the argument is `nil`, returns an empty list.
 
@@ -380,8 +438,9 @@ defmodule List do
 
   @doc """
   Returns a list with `value` inserted at the specified `index`.
+
   Note that `index` is capped at the list length. Negative indices
-  indicate an offset from the end of the list.
+  indicate an offset from the end of the `list`.
 
   ## Examples
 
@@ -399,7 +458,7 @@ defmodule List do
 
   """
   @spec insert_at(list, integer, any) :: list
-  def insert_at(list, index, value) do
+  def insert_at(list, index, value) when is_integer(index) do
     if index < 0 do
       do_insert_at(list, length(list) + index + 1, value)
     else
@@ -409,7 +468,8 @@ defmodule List do
 
   @doc """
   Returns a list with a replaced value at the specified `index`.
-  Negative indices indicate an offset from the end of the list.
+
+  Negative indices indicate an offset from the end of the `list`.
   If `index` is out of bounds, the original `list` is returned.
 
   ## Examples
@@ -428,7 +488,7 @@ defmodule List do
 
   """
   @spec replace_at(list, integer, any) :: list
-  def replace_at(list, index, value) do
+  def replace_at(list, index, value) when is_integer(index) do
     if index < 0 do
       do_replace_at(list, length(list) + index, value)
     else
@@ -438,7 +498,8 @@ defmodule List do
 
   @doc """
   Returns a list with an updated value at the specified `index`.
-  Negative indices indicate an offset from the end of the list.
+
+  Negative indices indicate an offset from the end of the `list`.
   If `index` is out of bounds, the original `list` is returned.
 
   ## Examples
@@ -457,7 +518,7 @@ defmodule List do
 
   """
   @spec update_at([elem], integer, (elem -> any)) :: list when elem: var
-  def update_at(list, index, fun) do
+  def update_at(list, index, fun) when is_function(fun, 1) and is_integer(index) do
     if index < 0 do
       do_update_at(list, length(list) + index, fun)
     else
@@ -467,7 +528,8 @@ defmodule List do
 
   @doc """
   Produces a new list by removing the value at the specified `index`.
-  Negative indices indicate an offset from the end of the list.
+
+  Negative indices indicate an offset from the end of the `list`.
   If `index` is out of bounds, the original `list` is returned.
 
   ## Examples
@@ -483,11 +545,34 @@ defmodule List do
 
   """
   @spec delete_at(list, integer) :: list
-  def delete_at(list, index) do
+  def delete_at(list, index) when is_integer(index) do
+    elem(pop_at(list, index), 1)
+  end
+
+  @doc """
+  Returns and removes the value at the specified `index` in the `list`.
+
+  Negative indices indicate an offset from the end of the `list`.
+  If `index` is out of bounds, the original `list` is returned.
+
+  ## Examples
+
+      iex> List.pop_at([1, 2, 3], 0)
+      {1, [2, 3]}
+      iex> List.pop_at([1, 2, 3], 5)
+      {nil, [1, 2, 3]}
+      iex> List.pop_at([1, 2, 3], 5, 10)
+      {10, [1, 2, 3]}
+      iex> List.pop_at([1, 2, 3], -1)
+      {3, [1, 2]}
+
+  """
+  @spec pop_at(list, integer, any) :: {any, list}
+  def pop_at(list, index, default \\ nil) when is_integer(index) do
     if index < 0 do
-      do_delete_at(list, length(list) + index)
+      do_pop_at(list, length(list) + index, default, [])
     else
-      do_delete_at(list, index)
+      do_pop_at(list, index, default, [])
     end
   end
 
@@ -646,6 +731,109 @@ defmodule List do
     end
   end
 
+  @doc """
+  Returns a keyword list that represents an *edit script*.
+
+  The algorithm is outlined in the
+  "An O(ND) Difference Algorithm and Its Variations" paper by E. Myers.
+
+  An *edit script* is a keyword list. Each key describes the "editing action" to
+  take in order to bring `list1` closer to being equal to `list2`; a key can be
+  `:eq`, `:ins`, or `:del`. Each value is a sublist of either `list1` or `list2`
+  that should be inserted (if the corresponding key `:ins`), deleted (if the
+  corresponding key is `:del`), or left alone (if the corresponding key is
+  `:eq`) in `list1` in order to be closer to `list2`.
+
+  ## Examples
+
+      iex> List.myers_difference([1, 4, 2, 3], [1, 2, 3, 4])
+      [eq: [1], del: [4], eq: [2, 3], ins: [4]]
+
+  """
+  @spec myers_difference(list, list) :: [{:eq | :ins | :del, list}] | nil
+  def myers_difference(list1, list2) when is_list(list1) and is_list(list2) do
+    path = {0, 0, list1, list2, []}
+    find_script(0, length(list1) + length(list2), [path])
+  end
+
+  defp find_script(envelope, max, _paths) when envelope > max do
+    nil
+  end
+
+  defp find_script(envelope, max, paths) do
+    case each_diagonal(-envelope, envelope, paths, []) do
+      {:done, edits} -> compact_reverse(edits, [])
+      {:next, paths} -> find_script(envelope + 1, max, paths)
+    end
+  end
+
+  defp compact_reverse([], acc), do: acc
+
+  defp compact_reverse([{kind, elem} | rest], [{kind, result} | acc]) do
+    compact_reverse(rest, [{kind, [elem | result]} | acc])
+  end
+
+  defp compact_reverse([{kind, elem} | rest], acc) do
+    compact_reverse(rest, [{kind, [elem]} | acc])
+  end
+
+  defp each_diagonal(diag, limit, _paths, next_paths) when diag > limit do
+    {:next, Enum.reverse(next_paths)}
+  end
+
+  defp each_diagonal(diag, limit, paths, next_paths) do
+    {path, rest} = proceed_path(diag, limit, paths)
+    with {:cont, path} <- follow_snake(path) do
+      each_diagonal(diag + 2, limit, rest, [path | next_paths])
+    end
+  end
+
+  defp proceed_path(0, 0, [path]), do: {path, []}
+
+  defp proceed_path(diag, limit, [path | _] = paths) when diag == -limit do
+    {move_down(path), paths}
+  end
+
+  defp proceed_path(diag, limit, [path]) when diag == limit do
+    {move_right(path), []}
+  end
+
+  defp proceed_path(_diag, _limit, [path1, path2 | rest]) do
+    if elem(path1, 1) > elem(path2, 1) do
+      {move_right(path1), [path2 | rest]}
+    else
+      {move_down(path2), [path2 | rest]}
+    end
+  end
+
+  defp move_right({x, y, list1, [elem | rest], edits}) do
+    {x + 1, y, list1, rest, [{:ins, elem} | edits]}
+  end
+
+  defp move_right({x, y, list1, [], edits}) do
+    {x + 1, y, list1, [], edits}
+  end
+
+  defp move_down({x, y, [elem | rest], list2, edits}) do
+    {x, y + 1, rest, list2, [{:del, elem} | edits]}
+  end
+
+  defp move_down({x, y, [], list2, edits}) do
+    {x, y + 1, [], list2, edits}
+  end
+
+  defp follow_snake({x, y, [elem | rest1], [elem | rest2], edits}) do
+    follow_snake({x + 1, y + 1, rest1, rest2, [{:eq, elem} | edits]})
+  end
+
+  defp follow_snake({_x, _y, [], [], edits}) do
+    {:done, edits}
+  end
+
+  defp follow_snake(path) do
+    {:cont, path}
+  end
+
   ## Helpers
 
   # replace_at
@@ -662,8 +850,8 @@ defmodule List do
     [value | rest]
   end
 
-  defp do_replace_at([h | t], index, value) do
-    [h | do_replace_at(t, index - 1, value)]
+  defp do_replace_at([head | tail], index, value) do
+    [head | do_replace_at(tail, index - 1, value)]
   end
 
   # insert_at
@@ -676,8 +864,8 @@ defmodule List do
     [value | list]
   end
 
-  defp do_insert_at([h | t], index, value) do
-    [h | do_insert_at(t, index - 1, value)]
+  defp do_insert_at([head | tail], index, value) do
+    [head | do_insert_at(tail, index - 1, value)]
   end
 
   # update_at
@@ -690,30 +878,30 @@ defmodule List do
     list
   end
 
-  defp do_update_at([h | t], index, fun) do
-    [h | do_update_at(t, index - 1, fun)]
+  defp do_update_at([head | tail], index, fun) do
+    [head | do_update_at(tail, index - 1, fun)]
   end
 
   defp do_update_at([], _index, _fun) do
     []
   end
 
-  # delete_at
+  # pop_at
 
-  defp do_delete_at([], _index) do
-    []
+  defp do_pop_at([], _index, default, acc) do
+    {default, :lists.reverse(acc)}
   end
 
-  defp do_delete_at([_ | t], 0) do
-    t
+  defp do_pop_at(list, index, default, []) when index < 0 do
+    {default, list}
   end
 
-  defp do_delete_at(list, index) when index < 0 do
-    list
+  defp do_pop_at([head | tail], 0, _default, acc) do
+    {head, :lists.reverse(acc, tail)}
   end
 
-  defp do_delete_at([h | t], index) do
-    [h | do_delete_at(t, index - 1)]
+  defp do_pop_at([head | tail], index, default, acc) do
+    do_pop_at(tail, index - 1, default, [head | acc])
   end
 
   # zip
@@ -731,8 +919,8 @@ defmodule List do
     {nil, nil}
   end
 
-  defp do_zip_each([h | t], acc) do
-    {t, [h | acc]}
+  defp do_zip_each([head | tail], acc) do
+    {tail, [head | acc]}
   end
 
   defp do_zip_each([], _) do

lib/elixir/lib/list/chars.ex

@@ -5,8 +5,8 @@ defprotocol List.Chars do
   The only function required to be implemented is
   `to_charlist` which does the conversion.
 
-  The `to_charlist` function automatically imported
-  by Kernel invokes this protocol.
+  The `to_charlist/1` function automatically imported
+  by `Kernel` invokes this protocol.
   """
 
   def to_charlist(term)

lib/elixir/lib/macro.ex

@@ -231,7 +231,7 @@ defmodule Macro do
 
   In order to build a variable, a context is expected.
   Most of the times, in order to preserve hygiene, the
-  context must be `__MODULE__`:
+  context must be `__MODULE__/0`:
 
       iex> Macro.var(:foo, __MODULE__)
       {:foo, [], __MODULE__}
@@ -400,7 +400,7 @@ defmodule Macro do
   @doc """
   Validates the given expressions are valid quoted expressions.
 
-  Checks the `type:Macro.t` for the specification of a valid
+  Checks the `t:Macro.t/0` for the specification of a valid
   quoted expression.
 
   It returns `:ok` if the expression is valid. Otherwise it returns a tuple in the form of
@@ -503,8 +503,8 @@ defmodule Macro do
       def unescape_map(?u), do: true
       def unescape_map(e),  do: e
 
-  If the `unescape_map` function returns `false`. The char is
-  not escaped and `\` is kept in the charlist.
+  If the `unescape_map/1` function returns `false`. The char is
+  not escaped and the backslash is kept in the string.
 
   Hexadecimals and Unicode codepoints will be escaped if the map
   function returns `true` for `?x`. Unicode codepoints if the map
@@ -512,7 +512,7 @@ defmodule Macro do
 
   ## Examples
 
-  Using the `unescape_map` function defined above is easy:
+  Using the `unescape_map/1` function defined above is easy:
 
       Macro.unescape_string "example\\n", &unescape_map(&1)
 
@@ -551,11 +551,23 @@ defmodule Macro do
   @doc """
   Converts the given expression to a binary.
 
+  The given `fun` is called for every node in the AST with two arguments: the
+  AST of the node being printed and the string representation of that same
+  node. The return value of this function is used as the final string
+  representation for that AST node.
+
   ## Examples
 
       iex> Macro.to_string(quote(do: foo.bar(1, 2, 3)))
       "foo.bar(1, 2, 3)"
 
+      iex> Macro.to_string(quote(do: 1 + 2), fn
+      ...>   1, _string -> "one"
+      ...>   2, _string -> "two"
+      ...>   _ast, string -> string
+      ...> end)
+      "one + two"
+
   """
   @spec to_string(Macro.t) :: String.t
   @spec to_string(Macro.t, (Macro.t, String.t -> String.t)) :: String.t
@@ -728,8 +740,8 @@ defmodule Macro do
       list == [] ->
         "[]"
       :io_lib.printable_list(list) ->
-        "'" <> Inspect.BitString.escape(IO.chardata_to_string(list), ?') <> "'"
-      Keyword.keyword?(list) ->
+        IO.iodata_to_binary [?', Inspect.BitString.escape(IO.chardata_to_string(list), ?'), ?']
+      Inspect.List.keyword?(list) ->
         "[" <> kw_list_to_string(list, fun) <> "]"
       true ->
         "[" <> Enum.map_join(list, ", ", &to_string(&1, fun)) <> "]"
@@ -751,11 +763,11 @@ defmodule Macro do
     to_string(ast, fun)
   end
 
-  defp bitmods_to_string({:-, _, [left, right]} = ast, fun, _, _) do
+  defp bitmods_to_string({op, _, [left, right]} = ast, fun, _, _) when op in [:*, :-] do
     result =
-      bitmods_to_string(left, fun, :-, :left) <>
-      "-" <>
-      bitmods_to_string(right, fun, :-, :right)
+      bitmods_to_string(left, fun, op, :left) <>
+      Atom.to_string(op) <>
+      bitmods_to_string(right, fun, op, :right)
     fun.(ast, result)
   end
 
@@ -824,6 +836,8 @@ defmodule Macro do
     do: Atom.to_string(atom)
   defp call_to_string({:., _, [{:&, _, [val]} = arg]}, fun) when not is_integer(val),
     do: "(" <> module_to_string(arg, fun) <> ")."
+  defp call_to_string({:., _, [{:fn, _, _} = arg]}, fun),
+    do: "(" <> module_to_string(arg, fun) <> ")."
   defp call_to_string({:., _, [arg]}, fun),
     do: module_to_string(arg, fun) <> "."
   defp call_to_string({:., _, [left, right]}, fun),
@@ -840,7 +854,7 @@ defmodule Macro do
   defp args_to_string(args, fun) do
     {list, last} = :elixir_utils.split_last(args)
 
-    if last != [] and Keyword.keyword?(last) do
+    if last != [] and Inspect.List.keyword?(last) do
       prefix =
         case list do
           [] -> ""
@@ -885,7 +899,7 @@ defmodule Macro do
 
   defp map_to_string(list, fun) do
     cond do
-      Keyword.keyword?(list) -> kw_list_to_string(list, fun)
+      Inspect.List.keyword?(list) -> kw_list_to_string(list, fun)
       true -> map_list_to_string(list, fun)
     end
   end
@@ -958,7 +972,7 @@ defmodule Macro do
 
     * Macros (local or remote)
     * Aliases are expanded (if possible) and return atoms
-    * Pseudo-variables (`__ENV__`, `__MODULE__` and `__DIR__`)
+    * Compilation environment macros (`__ENV__/0`, `__MODULE__/0` and `__DIR__/0`)
     * Module attributes reader (`@foo`)
 
   If the expression cannot be expanded, it returns the expression
@@ -1050,7 +1064,7 @@ defmodule Macro do
     end
   end
 
-  # Expand pseudo-variables
+  # Expand compilation environment macros
   defp do_expand_once({:__MODULE__, _, atom}, env) when is_atom(atom),
     do: {env.module, true}
   defp do_expand_once({:__DIR__, _, atom}, env) when is_atom(atom),
@@ -1098,7 +1112,7 @@ defmodule Macro do
 
       case expand do
         {:ok, receiver, quoted} ->
-          next = :elixir_counter.next
+          next = :erlang.unique_integer()
           {:elixir_quote.linify_with_context_counter(0, {receiver, next}, quoted), true}
         {:ok, _receiver, _name, _args} ->
           {original, false}
@@ -1119,7 +1133,7 @@ defmodule Macro do
 
         case expand do
           {:ok, receiver, quoted} ->
-            next = :elixir_counter.next
+            next = :erlang.unique_integer()
             {:elixir_quote.linify_with_context_counter(0, {receiver, next}, quoted), true}
           :error ->
             {original, false}

lib/elixir/lib/macro/env.ex

@@ -3,8 +3,8 @@ defmodule Macro.Env do
   A struct that holds compile time environment information.
 
   The current environment can be accessed at any time as
-  `__ENV__`. Inside macros, the caller environment can be
-  accessed as `__CALLER__`.
+  `__ENV__/0`. Inside macros, the caller environment can be
+  accessed as `__CALLER__/0`.
 
   An instance of `Macro.Env` must not be modified by hand. If you need to
   create a custom environment to pass to `Code.eval_quoted/3`, use the
@@ -41,7 +41,6 @@ defmodule Macro.Env do
       construct (may be `nil`)
     * `lexical_tracker` - PID of the lexical tracker which is responsible for
       keeping user info
-    * `local` - the module to expand local functions to
   """
 
   @type name_arity :: {atom, arity}
@@ -73,8 +72,7 @@ defmodule Macro.Env do
                context_modules: context_modules,
                vars: vars,
                export_vars: export_vars,
-               lexical_tracker: lexical_tracker,
-               local: local}
+               lexical_tracker: lexical_tracker}
 
   def __struct__ do
     %{__struct__: __MODULE__,

lib/elixir/lib/map.ex

@@ -2,10 +2,94 @@ defmodule Map do
   @moduledoc """
   A set of functions for working with maps.
 
-  Maps are key-value stores where keys can be any value and
-  are compared using the match operator (`===`). Maps can be
-  created with the `%{}` special form defined in the
-  `Kernel.SpecialForms` module.
+  Maps are the "go to" key-value data structure in Elixir. Maps can be created
+  with the `%{}` syntax, and key-value pairs can be expressed as `key => value`:
+
+      iex> %{}
+      %{}
+      iex> %{"one" => :two, 3 => "four"}
+      %{3 => "four", "one" => :two}
+
+  Key-value pairs in a map do not follow any order (that's why the printed map
+  in the example above has a different order than the map that was created).
+
+  Maps do not impose any restriction on the key type: anything can be a key in a
+  map. As a key-value structure, maps do not allow duplicated keys; keys are
+  compared using the exact-equality operator (`===`). If colliding keys are defined
+  in a map literal, the last one prevails.
+
+  When the key in a key-value pair is an atom, the `key: value` shorthand syntax
+  can be used (as in many other special forms), provided key-value pairs are put at
+  the end:
+
+      iex> %{"hello" => "world", a: 1, b: 2}
+      %{:a => 1, :b => 2, "hello" => "world"}
+
+  Keys in maps can be accessed through some of the functions in this module
+  (such as `Map.get/3` or `Map.fetch/2`) or through the `[]` syntax provided by
+  the `Access` module:
+
+      iex> map = %{a: 1, b: 2}
+      iex> Map.fetch(map, :a)
+      {:ok, 1}
+      iex> map[:b]
+      2
+      iex> map["non_existing_key"]
+      nil
+
+  The alternative access syntax `map.key` is provided alongside `[]` when the
+  map has a `:key` key; note that while `map[key]` will return `nil` if `map`
+  doesn't contain the key `key`, `map.key` will raise if `map` doesn't contain
+  the key `:key`.
+
+      iex> map = %{foo: "bar", baz: "bong"}
+      iex> map.foo
+      "bar"
+      iex> map.non_existing_key
+      ** (KeyError) key :non_existing_key not found in: %{baz: "bong", foo: "bar"}
+
+  Maps can be pattern matched on; when a map is on the left-hand side of a
+  pattern match, it will match if the map on the right-hand side contains the
+  keys on the left-hand side and their values match the ones on the left-hand
+  side. This means that an empty map matches every map.
+
+      iex> %{} = %{foo: "bar"}
+      %{foo: "bar"}
+      iex> %{a: a} = %{:a => 1, "b" => 2, [:c, :e, :e] => 3}
+      iex> a
+      1
+      iex> %{:c => 3} = %{:a => 1, 2 => :b}
+      ** (MatchError) no match of right hand side value: %{2 => :b, :a => 1}
+
+  Variables can be used as map keys both when writing map literals as well as
+  when matching:
+
+      iex> n = 1
+      1
+      iex> %{n => :one}
+      %{1 => :one}
+      iex> %{^n => :one} = %{1 => :one, 2 => :two, 3 => :three}
+      %{1 => :one, 2 => :two, 3 => :three}
+
+  Maps also support a specific update syntax to update the value stored under
+  *existing* atom keys:
+
+      iex> map = %{one: 1, two: 2}
+      iex> %{map | one: "one"}
+      %{one: "one", two: 2}
+      iex> %{map | three: 3}
+      ** (KeyError) key :three not found
+
+  ## Modules to work with maps
+
+  This module aims to provide functions that perform operations specific to maps
+  (like accessing keys, updating values, and so on). For traversing maps as
+  collections, developers should use the `Enum` module that works across a
+  variety of data types.
+
+  The `Kernel` module also provides a few functions to work with maps: for
+  example, `Kernel.map_size/1` to know the number of key-value pairs in a map or
+  `Kernel.is_map/1` to know if a term is a map.
   """
 
   @type key :: any
@@ -39,6 +123,9 @@ defmodule Map do
   @doc """
   Converts `map` to a list.
 
+  Each key-value pair in the map is converted to a two-element tuple `{key,
+  value}` in the resulting list.
+
   ## Examples
 
       iex> Map.to_list(%{a: 1})
@@ -75,7 +162,7 @@ defmodule Map do
       %{a: 3}
 
   """
-  @spec new(Enum.t) :: map
+  @spec new(Enumerable.t) :: map
   def new(enumerable)
   def new(%{__struct__: _} = struct), do: new_from_enum(struct)
   def new(%{} = map), do: map
@@ -88,7 +175,7 @@ defmodule Map do
   end
 
   @doc """
-  Creates a map from an `enumerable` via the transformation function.
+  Creates a map from an `enumerable` via the given transformation function.
 
   Duplicated keys are removed; the latest one prevails.
 
@@ -98,24 +185,25 @@ defmodule Map do
       %{a: :a, b: :b}
 
   """
-  @spec new(Enum.t, (term -> {key, value})) :: map
-  def new(enumerable, transform) do
+  @spec new(Enumerable.t, (term -> {key, value})) :: map
+  def new(enumerable, transform) when is_function(transform, 1) do
     enumerable
     |> Enum.to_list
-    |> do_new_transform(transform, [])
+    |> new_transform(transform, [])
   end
 
-  defp do_new_transform([], _fun, acc) do
+  defp new_transform([], _fun, acc) do
     acc
     |> :lists.reverse
     |> :maps.from_list
   end
-  defp do_new_transform([item | rest], fun, acc) do
-    do_new_transform(rest, fun, [fun.(item) | acc])
+
+  defp new_transform([item | rest], fun, acc) do
+    new_transform(rest, fun, [fun.(item) | acc])
   end
 
   @doc """
-  Returns whether a given `key` exists in the given `map`.
+  Returns whether the given `key` exists in the given `map`.
 
   ## Examples
 
@@ -129,9 +217,10 @@ defmodule Map do
   def has_key?(map, key), do: :maps.is_key(key, map)
 
   @doc """
-  Fetches the value for a specific `key` and returns it in a tuple.
+  Fetches the value for a specific `key` in the given `map`.
 
-  If the `key` does not exist, returns `:error`.
+  If `map` contains the given `key` with value `value`, then `{:ok, value}` is
+  returned. If `map` doesn't contain `key`, `:error` is returned.
 
   ## Examples
 
@@ -145,9 +234,11 @@ defmodule Map do
   def fetch(map, key), do: :maps.find(key, map)
 
   @doc """
-  Fetches the value for specific `key`.
+  Fetches the value for a specific `key` in the given `map`, erroring out if
+  `map` doesn't contain `key`.
 
-  If `key` does not exist, a `KeyError` is raised.
+  If `map` contains the given `key`, the corresponding value is returned. If
+  `map` doesn't contain `key`, a `KeyError` exception is raised.
 
   ## Examples
 
@@ -167,7 +258,7 @@ defmodule Map do
 
   @doc """
   Puts the given `value` under `key` unless the entry `key`
-  already exists.
+  already exists in `map`.
 
   ## Examples
 
@@ -187,10 +278,11 @@ defmodule Map do
 
   @doc """
   Evaluates `fun` and puts the result under `key`
-  in map unless `key` is already present.
+  in `map` unless `key` is already present.
 
-  This is useful if the value is very expensive to calculate or
-  generally difficult to setup and teardown again.
+  This function is useful in case you want to compute the value to put under
+  `key` only if `key` is not already present (e.g., the value is expensive to
+  calculate or generally difficult to setup and teardown again).
 
   ## Examples
 
@@ -214,8 +306,10 @@ defmodule Map do
   end
 
   @doc """
-  Takes all entries corresponding to the given keys and
-  returns them in a new map.
+  Returns a new map with all the key-value pairs in `map` where the key
+  is in `keys`.
+
+  If `keys` contains keys that are not in `map`, they're simply ignored.
 
   ## Examples
 
@@ -224,12 +318,18 @@ defmodule Map do
 
   """
   @spec take(map, Enumerable.t) :: map
-  def take(map, keys) do
+  def take(map, keys)
+
+  def take(map, keys) when is_map(map) do
     keys
     |> Enum.to_list
     |> do_take(map, [])
   end
 
+  def take(non_map, _keys) do
+    :erlang.error({:badmap, non_map})
+  end
+
   defp do_take([], _map, acc), do: :maps.from_list(acc)
   defp do_take([key | rest], map, acc) do
     acc = case fetch(map, key) do
@@ -240,10 +340,11 @@ defmodule Map do
   end
 
   @doc """
-  Gets the value for a specific `key`.
+  Gets the value for a specific `key` in `map`.
 
-  If `key` does not exist, return the default value
-  (`nil` if no default value).
+  If `key` is present in `map` with value `value`, then `value` is
+  returned. Otherwise, `default` is returned (which is `nil` unless
+  specified otherwise).
 
   ## Examples
 
@@ -267,9 +368,10 @@ defmodule Map do
   end
 
   @doc """
-  Gets the value for a specific `key`.
+  Gets the value for a specific `key` in `map`.
 
-  If `key` does not exist, lazily evaluates `fun` and returns its result.
+  If `key` is present in `map` with value `value`, then `value` is
+  returned. Otherwise, `fun` is evaluated and its result is returned.
 
   This is useful if the default value is very expensive to calculate or
   generally difficult to setup and teardown again.
@@ -296,7 +398,7 @@ defmodule Map do
   end
 
   @doc """
-  Puts the given `value` under `key`.
+  Puts the given `value` under `key` in `map`.
 
   ## Examples
 
@@ -307,12 +409,12 @@ defmodule Map do
 
   """
   @spec put(map, key, value) :: map
-  def put(map, key, val) do
-    :maps.put(key, val, map)
+  def put(map, key, value) do
+    :maps.put(key, value, map)
   end
 
   @doc """
-  Deletes the entries in `map` for a specific `key`.
+  Deletes the entry in `map` for a specific `key`.
 
   If the `key` does not exist, returns `map` unchanged.
 
@@ -330,7 +432,8 @@ defmodule Map do
   @doc """
   Merges two maps into one.
 
-  All keys in `map2` will be added to `map1`, overriding any existing one.
+  All keys in `map2` will be added to `map1`, overriding any existing one
+  (i.e., the keys in `map2` "have precedence" over the ones in `map1`).
 
   If you have a struct and you would like to merge a set of keys into the
   struct, do not use this function, as it would merge all keys on the right
@@ -347,10 +450,13 @@ defmodule Map do
   defdelegate merge(map1, map2), to: :maps
 
   @doc """
-  Merges two maps into one.
+  Merges two maps into one, resolving conflicts through the given `callback`.
 
-  All keys in `map2` will be added to `map1`. The given function will
-  be invoked with the key, value1 and value2 to solve conflicts.
+  All keys in `map2` will be added to `map1`. The given function will be invoked
+  when there are duplicate keys; its arguments are `key` (the duplicate key),
+  `value1` (the value of `key` in `map1`), and `value2` (the value of `key` in
+  `map2`). The value returned by `callback` is used as the value under `key` in
+  the resulting map.
 
   ## Examples
 
@@ -361,7 +467,7 @@ defmodule Map do
 
   """
   @spec merge(map, map, (key, value, value -> value)) :: map
-  def merge(map1, map2, callback) do
+  def merge(map1, map2, callback) when is_function(callback, 3) do
     :maps.fold fn k, v2, acc ->
       update(acc, k, v2, fn(v1) -> callback.(k, v1, v2) end)
     end, map1, map2
@@ -370,7 +476,9 @@ defmodule Map do
   @doc """
   Updates the `key` in `map` with the given function.
 
-  If the `key` does not exist, inserts the given `initial` value.
+  If `key` is present in `map` with value `value`, `fun` is invoked with
+  argument `value` and its result is used as the new value of `key`. If `key` is
+  not present in `map`, `initial` is inserted as the value of `key`.
 
   ## Examples
 
@@ -381,7 +489,7 @@ defmodule Map do
 
   """
   @spec update(map, key, value, (value -> value)) :: map
-  def update(map, key, initial, fun) do
+  def update(map, key, initial, fun) when is_function(fun, 1) do
     case fetch(map, key) do
       {:ok, value} ->
         put(map, key, fun.(value))
@@ -393,6 +501,10 @@ defmodule Map do
   @doc """
   Returns and removes the value associated with `key` in `map`.
 
+  If `key` is present in `map` with value `value`, `{value, new_map}` is
+  returned where `new_map` is the result of removing `key` from `map`. If `key`
+  is not present in `map`, `{default, map}` is returned.
+
   ## Examples
 
       iex> Map.pop(%{a: 1}, :a)
@@ -414,6 +526,11 @@ defmodule Map do
   @doc """
   Lazily returns and removes the value associated with `key` in `map`.
 
+  If `key` is present in `map` with value `value`, `{value, new_map}` is
+  returned where `new_map` is the result of removing `key` from `map`. If `key`
+  is not present in `map`, `{fun_result, map}` is returned, where `fun_result`
+  is the result of applying `fun`.
+
   This is useful if the default value is very expensive to calculate or
   generally difficult to setup and teardown again.
 
@@ -441,6 +558,8 @@ defmodule Map do
   @doc """
   Drops the given `keys` from `map`.
 
+  If `keys` contains keys that are not in `map`, they're simply ignored.
+
   ## Examples
 
       iex> Map.drop(%{a: 1, b: 2, c: 3}, [:b, :d])
@@ -448,20 +567,26 @@ defmodule Map do
 
   """
   @spec drop(map, Enumerable.t) :: map
-  def drop(map, keys) do
+  def drop(map, keys)
+
+  def drop(map, keys) when is_map(map) do
     keys
     |> Enum.to_list
     |> drop_list(map)
   end
 
+  def drop(non_map, _keys) do
+    :erlang.error({:badmap, non_map})
+  end
+
   defp drop_list([], acc), do: acc
   defp drop_list([key | rest], acc) do
-    drop_list(rest, Map.delete(acc, key))
+    drop_list(rest, delete(acc, key))
   end
 
   @doc """
-  Takes all entries corresponding to the given `keys` and extracts them into a
-  separate `map`.
+  Takes all entries corresponding to the given `keys` in `maps` and extracts
+  them into a separate map.
 
   Returns a tuple with the new map and the old map with removed keys.
 
@@ -474,12 +599,18 @@ defmodule Map do
 
   """
   @spec split(map, Enumerable.t) :: {map, map}
-  def split(map, keys) do
+  def split(map, keys)
+
+  def split(map, keys) when is_map(map) do
     keys
     |> Enum.to_list
     |> do_split([], map)
   end
 
+  def split(non_map, _keys) do
+    :erlang.error({:badmap, non_map})
+  end
+
   defp do_split([], inc, exc) do
     {:maps.from_list(inc), exc}
   end
@@ -493,9 +624,11 @@ defmodule Map do
   end
 
   @doc """
-  Updates the `key` with the given function.
+  Updates `key` with the given function.
 
-  If the `key` does not exist, raises `KeyError`.
+  If `key` is present in `map` with value `value`, `fun` is invoked with
+  argument `value` and its result is used as the new value of `key`. If `key` is
+  not present in `map`, a `KeyError` exception is raised.
 
   ## Examples
 
@@ -503,16 +636,16 @@ defmodule Map do
       %{a: 2}
 
       iex> Map.update!(%{a: 1}, :b, &(&1 * 2))
-      ** (KeyError) key :b not found
+      ** (KeyError) key :b not found in: %{a: 1}
 
   """
   @spec update!(map, key, (value -> value)) :: map | no_return
-  def update!(%{} = map, key, fun) do
+  def update!(%{} = map, key, fun) when is_function(fun, 1) do
     case fetch(map, key) do
       {:ok, value} ->
         put(map, key, fun.(value))
       :error ->
-        :erlang.error({:badkey, key})
+        raise KeyError, term: map, key: key
     end
   end
 
@@ -521,12 +654,12 @@ defmodule Map do
   @doc """
   Gets the value from `key` and updates it, all in one pass.
 
-  This `fun` argument receives the value of `key` (or `nil` if `key`
-  is not present) and must return a two-element tuple: the "get" value
-  (the retrieved value, which can be operated on before being returned)
-  and the new value to be stored under `key`. The `fun` may also
-  return `:pop`, implying the current value shall be removed
-  from `map` and returned.
+  `fun` is called with the current value under `key` in `map` (or `nil` if `key`
+  is not present in `map`) and must return a two-element tuple: the "get" value
+  (the retrieved value, which can be operated on before being returned) and the
+  new value to be stored under `key` in the resulting new map. `fun` may also
+  return `:pop`, which means the current value shall be removed from `map` and
+  returned (making this function behave like `Map.pop(map, key)`.
 
   The returned value is a tuple with the "get" value returned by
   `fun` and a new map with the updated value under `key`.
@@ -551,7 +684,7 @@ defmodule Map do
 
   """
   @spec get_and_update(map, key, (value -> {get, value} | :pop)) :: {get, map} when get: term
-  def get_and_update(%{} = map, key, fun) do
+  def get_and_update(%{} = map, key, fun) when is_function(fun, 1) do
     current =
       case :maps.find(key, map) do
         {:ok, value} -> value
@@ -559,8 +692,12 @@ defmodule Map do
       end
 
     case fun.(current) do
-      {get, update} -> {get, :maps.put(key, update, map)}
-      :pop          -> {current, :maps.remove(key, map)}
+      {get, update} ->
+        {get, :maps.put(key, update, map)}
+      :pop ->
+        {current, :maps.remove(key, map)}
+      other ->
+        raise "the given function must return a two-element tuple or :pop, got: #{inspect(other)}"
     end
   end
 
@@ -569,13 +706,8 @@ defmodule Map do
   @doc """
   Gets the value from `key` and updates it. Raises if there is no `key`.
 
-  This `fun` argument receives the value of `key` and must return a
-  two-element tuple: the "get" value (the retrieved value, which can be
-  operated on before being returned) and the new value to be stored under
-  `key`.
-
-  The returned value is a tuple with the "get" value returned by `fun` and a
-  new map with the updated value under `key`.
+  Behaves exactly like `get_and_update/3`, but raises a `KeyError` exception if
+  `key` is not present in `map`.
 
   ## Examples
 
@@ -587,7 +719,7 @@ defmodule Map do
       iex> Map.get_and_update!(%{a: 1}, :b, fn current_value ->
       ...>   {current_value, "new value!"}
       ...> end)
-      ** (KeyError) key :b not found
+      ** (KeyError) key :b not found in: %{a: 1}
 
       iex> Map.get_and_update!(%{a: 1}, :a, fn _ ->
       ...>   :pop
@@ -596,15 +728,19 @@ defmodule Map do
 
   """
   @spec get_and_update!(map, key, (value -> {get, value})) :: {get, map} | no_return when get: term
-  def get_and_update!(%{} = map, key, fun) do
+  def get_and_update!(%{} = map, key, fun) when is_function(fun, 1) do
     case :maps.find(key, map) do
       {:ok, value} ->
         case fun.(value) do
-          {get, update} -> {get, :maps.put(key, update, map)}
-          :pop          -> {value, :maps.remove(key, map)}
+          {get, update} ->
+            {get, :maps.put(key, update, map)}
+          :pop ->
+            {value, :maps.remove(key, map)}
+          other ->
+            raise "the given function must return a two-element tuple or :pop, got: #{inspect(other)}"
         end
       :error ->
-        :erlang.error({:badkey, key})
+        raise KeyError, term: map, key: key
     end
   end
 
@@ -614,7 +750,8 @@ defmodule Map do
   Converts a `struct` to map.
 
   It accepts the struct module or a struct itself and
-  simply removes the `__struct__` field from the struct.
+  simply removes the `__struct__` field from the given struct
+  or from a new struct generated from the given module.
 
   ## Example
 

lib/elixir/lib/map_set.ex

@@ -1,16 +1,42 @@
 defmodule MapSet do
   @moduledoc """
-  A set of functions for working with sets.
+  Functions that work on sets.
 
-  The `MapSet` is represented internally as a struct,
-  therefore `%MapSet{}` can be used whenever there is a
-  need to match on any `MapSet`. Note though the struct
-  fields are private and must not be accessed directly.
-  Instead, use the functions in this module.
+  `MapSet` is the "go to" set data structure in Elixir. A set can be constructed
+  using `MapSet.new/0`:
+
+      iex> MapSet.new
+      #MapSet<[]>
+
+  A set can contain any kind of elements and elements in a set don't have to be
+  of the same type. By definition, sets can't contain duplicate elements: when
+  inserting an element in a set where it's already present, the insertion is
+  simply a no-op.
+
+      iex> set = MapSet.new
+      iex> MapSet.put(set, "foo")
+      #MapSet<["foo"]>
+      iex> set |> MapSet.put("foo") |> MapSet.put("foo")
+      #MapSet<["foo"]>
+
+  A `MapSet` is represented internally using the `%MapSet{}` struct. This struct
+  can be used whenever there's a need to pattern match on something being a `MapSet`:
+
+      iex> match?(%MapSet{}, MapSet.new())
+      true
+
+  Note that, however, the struct fields are private and must not be accessed
+  directly; use the functions in this module to perform operations on sets.
+
+  Sets can also be constructed starting from other collection-type data
+  structures: for example, see `MapSet.new/1` or `Enum.into/2`.
   """
 
-  @opaque t :: %__MODULE__{map: map}
   @type value :: term
+
+  @opaque t(value) :: %__MODULE__{map: %{optional(value) => true}}
+  @type t :: t(term)
+
   defstruct map: %{}
 
   @doc """
@@ -56,8 +82,8 @@ defmodule MapSet do
       #MapSet<[2, 4]>
 
   """
-  @spec new(Enum.t, (term -> term)) :: t
-  def new(enumerable, transform) do
+  @spec new(Enum.t, (term -> val)) :: t(val) when val: value
+  def new(enumerable, transform) when is_function(transform, 1) do
     map =
       enumerable
       |> Enum.to_list
@@ -67,18 +93,14 @@ defmodule MapSet do
   end
 
   defp do_new([], acc) do
-    acc
-    |> :lists.reverse
-    |> :maps.from_list
+    :maps.from_list(acc)
   end
   defp do_new([item | rest], acc) do
     do_new(rest, [{item, true} | acc])
   end
 
   defp do_new_transform([], _fun, acc) do
-    acc
-    |> :lists.reverse
-    |> :maps.from_list
+    :maps.from_list(acc)
   end
   defp do_new_transform([item | rest], fun, acc) do
     do_new_transform(rest, fun, [{fun.(item), true} | acc])
@@ -98,7 +120,7 @@ defmodule MapSet do
       #MapSet<[1, 3]>
 
   """
-  @spec delete(t, value) :: t
+  @spec delete(t(val1), val2) :: t(val1) when val1: value, val2: value
   def delete(%MapSet{map: map} = set, value) do
     %{set | map: Map.delete(map, value)}
   end
@@ -112,12 +134,14 @@ defmodule MapSet do
       #MapSet<[1]>
 
   """
-  @spec difference(t, t) :: t
+  @spec difference(t(val1), t(val2)) :: t(val1) when val1: value, val2: value
+  def difference(mapset1, mapset2)
+
   # If the first set is less than twice the size of the second map,
   # it is fastest to re-accumulate items in the first set that are not
   # present in the second set.
   def difference(%MapSet{map: map1}, %MapSet{map: map2})
-  when map_size(map1) < map_size(map2) * 2 do
+      when map_size(map1) < map_size(map2) * 2 do
     map = map1
     |> Map.keys
     |> filter_not_in(map2)
@@ -203,7 +227,7 @@ defmodule MapSet do
       #MapSet<[]>
 
   """
-  @spec intersection(t, t) :: t
+  @spec intersection(t(val), t(val)) :: t(val) when val: value
   def intersection(%MapSet{map: map1}, %MapSet{map: map2}) do
     {map1, map2} = order_by_size(map1, map2)
 
@@ -237,7 +261,7 @@ defmodule MapSet do
       #MapSet<[1, 2, 3, 4]>
 
   """
-  @spec put(t, value) :: t
+  @spec put(t(val), new_val) :: t(val | new_val) when val: value, new_val: value
   def put(%MapSet{map: map} = set, value) do
     %{set | map: Map.put(map, value, true)}
   end
@@ -298,7 +322,7 @@ defmodule MapSet do
       [1, 2, 3]
 
   """
-  @spec to_list(t) :: list
+  @spec to_list(t(val)) :: [val] when val: value
   def to_list(%MapSet{map: map}) do
     Map.keys(map)
   end
@@ -312,7 +336,7 @@ defmodule MapSet do
       #MapSet<[1, 2, 3, 4]>
 
   """
-  @spec union(t, t) :: t
+  @spec union(t(val1), t(val2)) :: t(val1 | val2) when val1: value, val2: value
   def union(%MapSet{map: map1}, %MapSet{map: map2}) do
     %MapSet{map: Map.merge(map1, map2)}
   end

lib/elixir/lib/module.ex

@@ -8,225 +8,191 @@ defmodule Module do
   After a module is compiled, using many of the functions in
   this module will raise errors, since it is out of their scope
   to inspect runtime data. Most of the runtime data can be inspected
-  via the `__info__(attr)` function attached to each compiled module.
+  via the `__info__/1` function attached to each compiled module.
 
   ## Module attributes
 
   Each module can be decorated with one or more attributes. The following ones
   are currently defined by Elixir:
 
-    * `@after_compile`
+  ### @after_compile
 
-      A hook that will be invoked right after the current module is compiled.
-      Accepts a module or a tuple `{<module>, <function atom>}`.
-      See the "Compile callbacks" section below.
+  A hook that will be invoked right after the current module is compiled.
+  Accepts a module or a tuple `{<module>, <function atom>}`.
+  See the "Compile callbacks" section below.
 
-    * `@before_compile`
+  ### @before_compile
 
-      A hook that will be invoked before the module is compiled.
-      Accepts a module or a tuple `{<module>, <function/macro atom>}`.
-      See the "Compile callbacks" section below.
+  A hook that will be invoked before the module is compiled.
+  Accepts a module or a tuple `{<module>, <function/macro atom>}`.
+  See the "Compile callbacks" section below.
 
-    * `@behaviour` (notice the British spelling)
+  ### @behaviour (notice the British spelling)
 
-      Behaviours can be referenced by modules to ensure they implement
-      required specific function signatures defined by `@callback`.
+  Behaviours can be referenced by modules to ensure they implement
+  required specific function signatures defined by `@callback`.
 
-      For example, you can specify the URI.Parser behaviour as follows:
+  For example, you can specify the `URI.Parser` behaviour as follows:
 
-          defmodule URI.Parser do
-            @doc "Parses the given URL"
-            @callback parse(uri_info :: URI.t) :: URI.t
+      defmodule URI.Parser do
+        @doc "Defines a default port"
+        @callback default_port() :: integer
 
-            @doc "Defines a default port"
-            @callback default_port() :: integer
+        @doc "Parses the given URL"
+        @callback parse(uri_info :: URI.t) :: URI.t
+      end
+
+  And then a module may use it as:
+
+      defmodule URI.HTTP do
+        @behaviour URI.Parser
+        def default_port(), do: 80
+        def parse(info), do: info
+      end
+
+  If the behaviour changes or `URI.HTTP` does not implement
+  one of the callbacks, a warning will be raised.
+
+  ### @compile
+
+  Defines options for module compilation. This is used to configure
+  both Elixir and Erlang compilers, as any other compilation pass
+  added by external tools. For example:
+
+      defmodule M do
+        @compile {:inline, my_fun: 1}
+
+        def my_fun(arg) do
+          to_string(arg)
+        end
+      end
+
+  Multiple uses of `@compile` will accumulate instead of overriding
+  previous ones. See the "Compile options" section below.
+
+  ### @doc
+
+  Provides documentation for the function or macro that follows the
+  attribute.
+
+  Accepts a string (often a heredoc) or `false` where `@doc false` will
+  make the function/macro invisible to the documentation extraction tools
+  like ExDoc. For example:
+
+      defmodule M do
+        @doc "Hello world"
+        def hello do
+          "world"
+        end
+
+        @doc """
+        Sums `a` to `b`.
+        """
+        def sum(a, b) do
+          a + b
+        end
+      end
+
+  ### @dialyzer
+
+  Defines warnings to request or suppress when using a version of
+  `:dialyzer` that supports module attributes.
+
+  Accepts an atom, a tuple, or a list of atoms and tuples. For example:
+
+      defmodule M do
+        @dialyzer {:nowarn_function, my_fun: 1}
+
+        def my_fun(arg) do
+          M.not_a_function(arg)
+        end
+      end
+
+  For the list of supported warnings, see
+  [`:dialyzer` module](http://www.erlang.org/doc/man/dialyzer.html).
+
+  Multiple uses of `@dialyzer` will accumulate instead of overriding
+  previous ones.
+
+  ### @external_resource
+
+  Specifies an external resource to the current module.
+
+  Many times a module embeds information from an external file. This
+  attribute allows the module to annotate which external resources
+  have been used.
+
+  Tools like Mix may use this information to ensure the module is
+  recompiled in case any of the external resources change.
+
+  ### @file
+
+  Changes the filename used in stacktraces for the function or macro that
+  follows the attribute, such as:
+
+      defmodule M do
+        @doc "Hello world"
+        @file "hello.ex"
+        def hello do
+          "world"
+        end
+      end
+
+  ### @moduledoc
+
+  Provides documentation for the current module, such as:
+
+      defmodule M do
+        @moduledoc """
+        A very useful module
+        """
+      end
+
+  Accepts a string (which is often a heredoc) or `false` where
+  `@moduledoc false` will make the module invisible to the
+  documentation extraction tools like ExDoc.
+
+  ### @on_definition
+
+  A hook that will be invoked when each function or macro in the current
+  module is defined. Useful when annotating functions.
+
+  Accepts a module or a tuple `{<module>, <function atom>}`. See the
+  "Compile callbacks" section below.
+
+  ### @on_load
+
+  A hook that will be invoked whenever the module is loaded.
+
+  Accepts a function atom of a function in the current module. The function
+  must have arity 0 (no arguments) and has to return `:ok`, otherwise the
+  loading of the module will be aborted. For example:
+
+      defmodule M do
+        @on_load :load_check
+
+        def load_check do
+          if some_condition() do
+            :ok
+          else
+            :abort
           end
+        end
 
-          And then a module may use it as:
+        def some_condition do
+          false
+        end
+      end
 
-          defmodule URI.HTTP do
-            @behaviour URI.Parser
-            def default_port(), do: 80
-            def parse(info), do: info
-          end
+  ### @vsn
 
-      If the behaviour changes or URI.HTTP does not implement one of the
-      callbacks, a warning will be raised.
+  Specify the module version. Accepts any valid Elixir value, for example:
 
-      Specifies an OTP or user-defined behaviour.
+      defmodule M do
+        @vsn "1.0"
+      end
 
-      ### Example
-
-          defmodule M do
-            @behaviour :gen_event
-
-            # ...
-          end
-
-    * `@callback`, `@macrocallback`, and `@optional_callbacks`
-
-      These attributes are used to define a behaviour (as shown in the
-      documentation for `@behaviour` above). `@callback` defines a function
-      callback, `@macrocallback` defines a macro callback, and
-      `@optional_callbacks` specifies which callbacks and macrocallbacks are
-      optional.
-
-    * `@compile`
-
-      Defines options for module compilation. This is used to configure
-      both Elixir and Erlang compilers, as any other compilation pass
-      added by external tools.
-
-      Multiple uses of `@compile` will accumulate instead of overriding
-      previous ones. See the "Compile options" section below.
-
-      ### Example
-
-          defmodule M do
-            @compile {:inline, myfun: 1}
-
-            def myfun(arg) do
-              to_string(arg)
-            end
-          end
-
-    * `@doc`
-
-      Provides documentation for the function or macro that follows the
-      attribute.
-
-      Accepts a string (often a heredoc) or `false` where `@doc false` will
-      make the function/macro invisible to the documentation extraction tools
-      like ExDoc.
-
-      Can be invoked more than once.
-
-      ### Example
-
-          defmodule M do
-            @doc "Hello world"
-            def hello do
-              "world"
-            end
-
-            @doc """
-            Sums `a` to `b`.
-            """
-            def sum(a, b) do
-              a + b
-            end
-          end
-
-    * `@dialyzer`
-
-      Defines warnings to request or suppress when using a version of
-      `:dialyzer` that supports module attributes.
-
-      Accepts an atom, a tuple, or a list of atoms and tuples.
-
-      For the list of supported warnings, see
-      [`:dialyzer` module](http://www.erlang.org/doc/man/dialyzer.html).
-
-      Multiple uses of `@dialyzer` will accumulate instead of overriding
-      previous ones.
-
-      ### Example
-
-          defmodule M do
-            @dialyzer {:nowarn_function, myfun: 1}
-
-            def myfun(arg) do
-              M.not_a_function(arg)
-            end
-          end
-
-    * `@external_resource`
-
-      Specifies an external resource to the current module.
-
-      Many times a module embeds information from an external file. This
-      attribute allows the module to annotate which external resources
-      have been used.
-
-      Tools like Mix may use this information to ensure the module is
-      recompiled in case any of the external resources change.
-
-    * `@file`
-
-      Changes the filename used in stacktraces for the function or macro that
-      follows the attribute.
-
-      Accepts a string. Can be used more than once.
-
-      ### Example
-
-          defmodule M do
-            @doc "Hello world"
-            @file "hello.ex"
-            def hello do
-              "world"
-            end
-          end
-
-    * `@moduledoc`
-
-      Provides documentation for the current module.
-
-      Accepts a string (which is often a heredoc) or `false` where
-      `@moduledoc false` will make the module invisible to the
-      documentation extraction tools like ExDoc.
-
-      ### Example
-
-          defmodule M do
-            @moduledoc """
-            A very useful module
-            """
-          end
-
-    * `@on_definition`
-
-      A hook that will be invoked when each function or macro in the current
-      module is defined. Useful when annotating functions.
-
-      Accepts a module or a tuple `{<module>, <function atom>}`. See the
-      "Compile callbacks" section below.
-
-    * `@on_load`
-
-      A hook that will be invoked whenever the module is loaded.
-
-      Accepts a function atom of a function in the current module. The function
-      must have arity 0 (no arguments) and has to return `:ok`, otherwise the
-      loading of the module will be aborted.
-
-      ### Example
-
-          defmodule M do
-            @on_load :load_check
-
-            def load_check do
-              if some_condition() do
-                :ok
-              else
-                nil
-              end
-            end
-
-            def some_condition do
-              false
-            end
-          end
-
-    * `@vsn`
-
-      Specify the module version. Accepts any valid Elixir value.
-
-      ### Example
-
-          defmodule M do
-            @vsn "1.0"
-          end
+  ### Typespec attributes
 
   The following attributes are part of typespecs and are also reserved by
   Elixir:
@@ -240,6 +206,8 @@ defmodule Module do
     * `@optional_callbacks` - specifies which behaviour callbacks and macro
       behaviour callbacks are optional
 
+  ### Custom attributes
+
   In addition to the built-in attributes outlined above, custom attributes may
   also be added. A custom attribute is any valid identifier prefixed with an
   `@` and followed by a valid Elixir value:
@@ -256,7 +224,7 @@ defmodule Module do
   There are three callbacks that are invoked when functions are defined,
   as well as before and immediately after the module bytecode is generated.
 
-  ### `@after_compile`
+  ### @after_compile
 
   A hook that will be invoked right after the current module is compiled.
 
@@ -275,7 +243,7 @@ defmodule Module do
         end
       end
 
-  ### `@before_compile`
+  ### @before_compile
 
   A hook that will be invoked before the module is compiled.
 
@@ -305,7 +273,7 @@ defmodule Module do
         @before_compile A
       end
 
-  ### `@on_definition`
+  ### @on_definition
 
   A hook that will be invoked when each function or macro in the current
   module is defined. Useful when annotating functions.
@@ -367,10 +335,10 @@ defmodule Module do
   below:
 
     * `@compile :debug_info` - includes `:debug_info` regardless of the
-      setting in `Code.compiler_options`
+      setting in `Code.compiler_options/1`
 
-    * `@compile {:debug_info, false} - disables `:debug_info` regardless
-      of the setting in `Code.compiler_options`
+    * `@compile {:debug_info, false}` - disables `:debug_info` regardless
+      of the setting in `Code.compiler_options/1`
 
     * `@compile {:inline, some_fun: 2, other_fun: 3}` - inlines the given
       name/arity pairs
@@ -428,18 +396,23 @@ defmodule Module do
 
       Foo.sum(1, 2) #=> 3
 
-  For convenience, you can pass `__ENV__` as an argument and
-  all options will be automatically extracted from the environment:
+  For convenience, you can pass any `Macro.Env` struct, such
+  as  `__ENV__/0`, as the first argument or as options. Both
+  the module and all options will be automatically extracted
+  from the environment:
 
       defmodule Foo do
         contents = quote do: (def sum(a, b), do: a + b)
-        Module.eval_quoted __MODULE__, contents, [], __ENV__
+        Module.eval_quoted __ENV__, contents
       end
 
       Foo.sum(1, 2) #=> 3
 
+  Note that if you pass a `Macro.Env` struct as first argument
+  while also passing `opts`, they will be merged with `opts`
+  having precedence.
   """
-  def eval_quoted(module, quoted, binding \\ [], opts \\ [])
+  def eval_quoted(module_or_env, quoted, binding \\ [], opts \\ [])
 
   def eval_quoted(%Macro.Env{} = env, quoted, binding, opts) do
     eval_quoted(env.module, quoted, binding, Keyword.merge(Map.to_list(env), opts))
@@ -499,7 +472,7 @@ defmodule Module do
       raise ArgumentError, "expected :file to be given as option"
     end
 
-    next = :elixir_counter.next
+    next = :erlang.unique_integer()
     line = Keyword.get(opts, :line, 0)
     quoted = :elixir_quote.linify_with_context_counter(line, {module, next}, quoted)
     :elixir_module.compile(module, quoted, [], :elixir.env_for_eval(opts))
@@ -622,7 +595,7 @@ defmodule Module do
           kind,
           merge_signatures(old_sign, signature, 1),
           if(is_nil(doc), do: old_doc, else: doc)
-       })
+        })
         :ok
     end
   end
@@ -825,11 +798,15 @@ defmodule Module do
   def make_overridable(module, tuples) do
     assert_not_compiled!(:make_overridable, module)
 
-    :lists.foreach(fn tuple ->
+    :lists.foreach(fn {name, arity} = tuple ->
       case :elixir_def.take_definition(module, tuple) do
         false ->
-          {name, arity} = tuple
-          raise "cannot make function #{name}/#{arity} overridable because it was not defined"
+          raise ArgumentError,
+            "cannot make function #{name}/#{arity} overridable because it was not defined"
+        {{_def, :defmacrop, _line, _file, _check, _location, _defaults}, _clauses} ->
+          raise ArgumentError,
+            "cannot make private macro #{name}/#{arity} overridable, overriding " <>
+            "private macros is not supported"
         clause ->
           neighbours =
             if :elixir_compiler.get_opt(:internal) do
@@ -857,11 +834,7 @@ defmodule Module do
   end
 
   @doc """
-  Puts an Erlang attribute to the given module with the given
-  key and value.
-
-  The semantics of putting the attribute depends
-  if the attribute was registered or not via `register_attribute/3`.
+  Puts a module attribute with key and value in the given module.
 
   ## Examples
 
@@ -870,45 +843,25 @@ defmodule Module do
       end
 
   """
+  @spec put_attribute(module, key :: atom, value :: term) :: :ok
   def put_attribute(module, key, value) do
-    put_attribute(module, key, value, nil)
-  end
-
-  @doc false
-  def put_attribute(module, key, value, stack) when is_atom(key) do
-    assert_not_compiled!(:put_attribute, module)
-    table = data_table_for(module)
-    value = preprocess_attribute(key, value)
-    acc   = :ets.lookup_element(table, {:elixir, :acc_attributes}, 2)
-    warn_if_redefining_doc_attribute(stack, table, key)
-
-    new =
-      if :lists.member(key, acc) do
-        case :ets.lookup(table, key) do
-          [{^key, old}] -> [value | old]
-          [] -> [value]
-        end
-      else
-        value
-      end
-
-    :ets.insert(table, {key, new})
+    put_attribute(module, key, value, nil, nil)
   end
 
   @doc """
   Gets the given attribute from a module.
 
   If the attribute was marked with `accumulate` with
-  `Module.register_attribute/3`, a list is always returned. `nil` is returned
-  if the attribute has not been marked with `accumulate` and has not been set
-  to any value.
+  `Module.register_attribute/3`, a list is always returned.
+  `nil` is returned if the attribute has not been marked with
+  `accumulate` and has not been set to any value.
 
   The `@` macro compiles to a call to this function. For example,
   the following code:
 
       @foo
 
-  Expands close to:
+  Expands to something akin to:
 
       Module.get_attribute(__MODULE__, :foo)
 
@@ -924,13 +877,15 @@ defmodule Module do
       end
 
   """
-  @spec get_attribute(atom, atom) :: term
+  @spec get_attribute(module, atom) :: term
   def get_attribute(module, key) do
     get_attribute(module, key, nil)
   end
 
   @doc """
-  Deletes all attributes that match the given key.
+  Deletes the module attribute that matches the given key.
+
+  It returns the deleted attribute value (or `nil` if nothing was set).
 
   ## Examples
 
@@ -940,18 +895,26 @@ defmodule Module do
       end
 
   """
-  @spec delete_attribute(atom, atom) :: :ok
+  @spec delete_attribute(module, key :: atom) :: (value :: term)
   def delete_attribute(module, key) when is_atom(key) do
     assert_not_compiled!(:delete_attribute, module)
     table = data_table_for(module)
-    :ets.delete(table, key)
-    :ok
+    case :ets.take(table, key) do
+      [{_, value, _accumulated? = true, _}] ->
+        :ets.insert(table, {key, [], true, nil})
+        value
+      [{_, value, _, _}] ->
+        value
+      [] ->
+        nil
+    end
   end
 
   @doc """
-  Registers an attribute. By registering an attribute, a developer
-  is able to customize how Elixir will store and accumulate the
-  attribute values.
+  Registers an attribute.
+
+  By registering an attribute, a developer is able to customize
+  how Elixir will store and accumulate the attribute values.
 
   ## Options
 
@@ -989,9 +952,11 @@ defmodule Module do
     end
 
     if Keyword.get(opts, :accumulate) do
-      old = :ets.lookup_element(table, {:elixir, :acc_attributes}, 2)
-      :ets.insert(table, {{:elixir, :acc_attributes}, [new | old]})
+      :ets.insert_new(table, {new, [], _accumulated? = true, _unread_line = nil}) ||
+        :ets.update_element(table, new, {3, true})
     end
+
+    :ok
   end
 
   @doc """
@@ -1012,8 +977,8 @@ defmodule Module do
   end
 
   @doc false
-  # Used internally to compile documentation. This function
-  # is private and must be used only internally.
+  # Used internally to compile documentation.
+  # This function is private and must be used only internally.
   def compile_doc(env, kind, name, args, _guards, _body) do
     module = env.module
     table  = data_table_for(module)
@@ -1046,52 +1011,80 @@ defmodule Module do
   end
 
   @doc false
-  # Used internally to compile types. This function
-  # is private and must be used only internally.
+  # Used internally to compile types.
+  # This function is private and must be used only internally.
   def store_typespec(module, key, value) when is_atom(key) do
     assert_not_compiled!(:put_attribute, module)
     table = data_table_for(module)
 
     new =
       case :ets.lookup(table, key) do
-        [{^key, old}] -> [value | old]
+        [{^key, old, _, _}] -> [value | old]
         [] -> [value]
       end
 
-    :ets.insert(table, {key, new})
+    :ets.insert(table, {key, new, true, nil})
   end
 
   @doc false
-  def get_attribute(module, key, stack) when is_atom(key) and (is_list(stack) or is_nil(stack)) do
+  # Used internally by Kernel's @.
+  # This function is private and must be used only internally.
+  def get_attribute(module, key, stack) when is_atom(key) do
     assert_not_compiled!(:get_attribute, module)
     table = data_table_for(module)
 
     case :ets.lookup(table, key) do
-      [{^key, val}] ->
+      [{^key, val, _, _}] ->
+        :ets.update_element(table, key, {4, nil})
         val
+      [] when is_list(stack) ->
+        # TODO: Consider raising instead of warning on v2.0 as it usually cascades
+        IO.warn "undefined module attribute @#{key}, " <>
+                "please remove access to @#{key} or explicitly set it before access", stack
+        nil
       [] ->
-        acc = :ets.lookup_element(table, {:elixir, :acc_attributes}, 2)
-
-        cond do
-          :lists.member(key, acc) ->
-            []
-          is_list(stack) ->
-            IO.warn "undefined module attribute @#{key}, " <>
-                    "please remove access to @#{key} or explicitly set it before access", stack
-            nil
-          true ->
-            nil
-        end
+        nil
     end
   end
 
+  @doc false
+  # Used internally by Kernel's @.
+  # This function is private and must be used only internally.
+  def put_attribute(module, key, value, stack, unread_line) when is_atom(key) do
+    assert_not_compiled!(:put_attribute, module)
+    table = data_table_for(module)
+    value = preprocess_attribute(key, value)
+
+    case :ets.lookup(table, key) do
+      [{^key, {line, <<_::binary>>}, accumulated?, _unread_line}]
+          when key in [:doc, :typedoc, :moduledoc] and is_list(stack) ->
+        IO.warn "redefining @#{key} attribute previously set at line #{line}", stack
+        :ets.insert(table, {key, value, accumulated?, unread_line})
+
+      [{^key, current, _accumulated? = true, _read?}] ->
+        :ets.insert(table, {key, [value | current], true, unread_line})
+
+      _ ->
+        :ets.insert(table, {key, value, false, unread_line})
+    end
+
+    :ok
+  end
+
   ## Helpers
 
   defp preprocess_attribute(key, value) when key in [:moduledoc, :typedoc, :doc] do
-    unless match?({line, _} when is_integer(line), value) do
-      raise ArgumentError, "expected #{key} attribute given in the {line, doc} format, got: #{inspect(value)}"
+    case value do
+      {line, doc} when is_integer(line) and (is_binary(doc) or is_boolean(doc) or is_nil(doc)) ->
+        value
+      {line, doc} when is_integer(line) ->
+        raise ArgumentError,
+          "expected the #{key} attribute to contain a binary, a boolean, or nil, got: #{inspect(doc)}"
+      _other ->
+        raise ArgumentError,
+          "expected the #{key} attribute to be {line, doc} (where \"doc\" is " <>
+          "a binary, a boolean, or nil), got: #{inspect(value)}"
     end
-    value
   end
 
   defp preprocess_attribute(:on_load, atom) when is_atom(atom) do
@@ -1126,7 +1119,7 @@ defmodule Module do
 
   defp get_doc_info(table, env) do
     case :ets.take(table, :doc) do
-      [doc: {_, _} = pair] -> pair
+      [{:doc, {_, _} = pair, _, _}] -> pair
       [] -> {env.line, nil}
     end
   end
@@ -1144,16 +1137,4 @@ defmodule Module do
       raise ArgumentError,
         "could not call #{fun} on module #{inspect module} because it was already compiled"
   end
-
-  defp warn_if_redefining_doc_attribute(stack, table, key)
-      when is_list(stack) and key in [:doc, :typedoc, :moduledoc] do
-    case :ets.lookup(table, key) do
-      [{_, {line, val}}] when val != false ->
-        IO.warn "redefining @#{key} attribute previously set at line #{line}", stack
-      _ ->
-        false
-    end
-  end
-
-  defp warn_if_redefining_doc_attribute(nil, _table, _key), do: false
 end

lib/elixir/lib/module/locals_tracker.ex

@@ -83,13 +83,12 @@ defmodule Module.LocalsTracker do
   defp to_pid(pid) when is_pid(pid),  do: pid
   defp to_pid(mod) when is_atom(mod) do
     table = :elixir_module.data_table(mod)
-    [{_, val}] = :ets.lookup(table, {:elixir, :locals_tracker})
-    val
+    :ets.lookup_element(table, {:elixir, :locals_tracker}, 2)
   end
 
   # Internal API
 
-  # Starts the tracker and returns its pid.
+  # Starts the tracker and returns its PID.
   @doc false
   def start_link do
     :gen_server.start_link(__MODULE__, [], [])

lib/elixir/lib/node.ex

@@ -146,7 +146,7 @@ defmodule Node do
   `:ignored` if the local node is not alive.
 
   For more information, see
-  [`:erlang.connect_node/1`](http://www.erlang.org/doc/man/net_kernel.html#connect_node-1).
+  [`:net_kernel.connect_node/1`](http://www.erlang.org/doc/man/net_kernel.html#connect_node-1).
   """
   @spec connect(t) :: boolean | :ignored
   def connect(node) do
@@ -154,8 +154,8 @@ defmodule Node do
   end
 
   @doc """
-  Returns the pid of a new process started by the application of `fun`
-  on `node`. If `node` does not exist, a useless pid is returned.
+  Returns the PID of a new process started by the application of `fun`
+  on `node`. If `node` does not exist, a useless PID is returned.
 
   For the list of available options, see
   [`:erlang.spawn/2`](http://www.erlang.org/doc/man/erlang.html#spawn-2).
@@ -168,10 +168,10 @@ defmodule Node do
   end
 
   @doc """
-  Returns the pid of a new process started by the application of `fun`
+  Returns the PID of a new process started by the application of `fun`
   on `node`.
 
-  If `node` does not exist, a useless pid is returned.
+  If `node` does not exist, a useless PID is returned.
 
   For the list of available options, see
   [`:erlang.spawn_opt/3`](http://www.erlang.org/doc/man/erlang.html#spawn_opt-3).
@@ -184,10 +184,10 @@ defmodule Node do
   end
 
   @doc """
-  Returns the pid of a new process started by the application of
+  Returns the PID of a new process started by the application of
   `module.function(args)` on `node`.
 
-  If `node` does not exist, a useless pid is returned.
+  If `node` does not exist, a useless PID is returned.
 
   For the list of available options, see
   [`:erlang.spawn/4`](http://www.erlang.org/doc/man/erlang.html#spawn-4).
@@ -200,10 +200,10 @@ defmodule Node do
   end
 
   @doc """
-  Returns the pid of a new process started by the application of
+  Returns the PID of a new process started by the application of
   `module.function(args)` on `node`.
 
-  If `node` does not exist, a useless pid is returned.
+  If `node` does not exist, a useless PID is returned.
 
   For the list of available options, see
   [`:erlang.spawn/5`](http://www.erlang.org/doc/man/erlang.html#spawn_opt-5).
@@ -216,10 +216,10 @@ defmodule Node do
   end
 
   @doc """
-  Returns the pid of a new linked process started by the application of `fun` on `node`.
+  Returns the PID of a new linked process started by the application of `fun` on `node`.
 
   A link is created between the calling process and the new process, atomically.
-  If `node` does not exist, a useless pid is returned (and due to the link, an exit
+  If `node` does not exist, a useless PID is returned (and due to the link, an exit
   signal with exit reason `:noconnection` will be received).
 
   Inlined by the compiler.
@@ -230,11 +230,11 @@ defmodule Node do
   end
 
   @doc """
-  Returns the pid of a new linked process started by the application of
+  Returns the PID of a new linked process started by the application of
   `module.function(args)` on `node`.
 
   A link is created between the calling process and the new process, atomically.
-  If `node` does not exist, a useless pid is returned (and due to the link, an exit
+  If `node` does not exist, a useless PID is returned (and due to the link, an exit
   signal with exit reason `:noconnection` will be received).
 
   Inlined by the compiler.

lib/elixir/lib/option_parser.ex

@@ -13,84 +13,153 @@ defmodule OptionParser do
   end
 
   @doc """
-  Parses `argv` into a keywords list.
+  Parses `argv` into a keyword list.
 
-  It returns a three-element tuple as follows:
+  It returns a three-element tuple with the form `{parsed, args, invalid}`, where:
 
-     1. parsed switches,
-     2. remaining arguments,
-     3. invalid options.
+    * `parsed` is a keyword list of parsed switches with `{switch_name, value}`
+      tuples in it; `switch_name` is the atom representing the switch name while
+      `value` is the value for that switch parsed according to `opts` (see the
+      "Examples" section for more information)
+    * `args` is a list of the remaining arguments in `argv` as strings
+    * `invalid` is a list of invalid options as `{option_name, value}` where
+      `option_name` is the raw option and `value` is `nil` if the option wasn't
+      expected or the string value if the value didn't have the expected type for
+      the corresponding option
 
-  ## Examples
+  Elixir converts switches to underscored atoms, so `--source-path` becomes
+  `:source_path`. This is done to better suit Elixir conventions. However, this
+  means that switches can't contain underscores and switches that do contain
+  underscores are always returned in the list of invalid switches.
 
-      iex> OptionParser.parse(["--debug"])
+  When parsing, it is common to list switches and their expected types:
+
+      iex> OptionParser.parse(["--debug"], switches: [debug: :boolean])
       {[debug: true], [], []}
 
-      iex> OptionParser.parse(["--source", "lib"])
+      iex> OptionParser.parse(["--source", "lib"], switches: [source: :string])
       {[source: "lib"], [], []}
 
-      iex> OptionParser.parse(["--source-path", "lib", "test/enum_test.exs", "--verbose"])
+      iex> OptionParser.parse(["--source-path", "lib", "test/enum_test.exs", "--verbose"],
+      ...>                    switches: [source_path: :string, verbose: :boolean])
       {[source_path: "lib", verbose: true], ["test/enum_test.exs"], []}
 
-  By default, Elixir will try to automatically parse all switches.
-  Switches followed by a value will be assigned the value, as a string.
-  Switches without an argument, like `--debug` will automatically
-  be set to `true`.
+  We will explore the valid switches and operation modes of option parser below.
 
-  Note: Elixir also converts the switches to underscore atoms, so
-  `--source-path` becomes `:source_path`, to better suit Elixir
-  conventions. This means that option names on the command line cannot
-  contain underscores; such options will be put in the invalid options
-  list.
+  ## Options
 
-  ## Switch Definitions
+  The following options are supported:
 
-  Often it is better to explicitly list the known
-  switches and their formats. The switches can be specified via two
-  alternative options:
+    * `:switches` or `:strict` - see the "Switch definitions" section below
+    * `:allow_nonexistent_atoms` - see the "Parsing dynamic switches" section below
+    * `:aliases` - see the "Aliases" section below
 
-    * `:switches` - defines some switches. An attempt is still made to parse
-      switches that do not appear in the list.
+  ## Switch definitions
 
-    * `:strict` - the switches are strict. Any switch that is not specified
-      in the list is returned in the invalid options list.
+  Switches can be specified via one of two options:
 
-  Note that you should only supply the `:switches` or `:strict` option.
-  If you supply both, an error will be raised.
+    * `:switches` - defines some switches and their types. This function
+      still attempts to parse switches that are not in this list.
+    * `:strict` - defines strict switches. Any switch in `argv` that is not
+      specified in the list is returned in the invalid options list.
+
+  Both these options accept a keyword list of `{name, type}` tuples where `name`
+  is an atom defining the name of the switch and `type` is an atom that
+  specifies the type for the value of this switch (see the "Types" section below
+  for the possible types and more information about type casting).
+
+  Note that you should only supply the `:switches` or the`:strict` option.
+  If you supply both, an `ArgumentError` exception will be raised.
 
   ### Types
 
-  Option parser switches may take 0 or 1 argument.
+  Switches parsed by `OptionParser` may take zero or one arguments.
 
-  The following switches take no argument:
+  The following switches types take no arguments:
 
-    * `:boolean` - sets the value to true when given
-    * `:count`   - counts the number of times the switch is given
+    * `:boolean` - sets the value to `true` when given (see also the
+      "Negation switches" section below)
+    * `:count` - counts the number of times the switch is given
 
-  The following switches take 1 argument:
+  The following switches take one argument:
 
-    * `:integer` - parses the upcoming value as an integer.
-    * `:float`   - parses the upcoming value as a float.
-    * `:string`  - parses the upcoming value as a string.
+    * `:integer` - parses the value as an integer
+    * `:float` - parses the value as a float
+    * `:string` - parses the value as a string
 
-  If a switch can't be parsed, it is returned in the invalid
-  options list.
+  If a switch can't be parsed according to the given type, it is
+  returned in the invalid options list.
 
   ### Modifiers
 
   Switches can be specified with modifiers, which change how
   they behave. The following modifiers are supported:
 
-    * `:keep` - keeps duplicated items instead of overriding them.
-      Works with all types except `:count`.
+    * `:keep` - keeps duplicated items instead of overriding them;
+      works with all types except `:count`. Specifying `switch_name: :keep`
+      assumes the type of `:switch_name` will be `:string`.
 
-  Note: if you want to use `:keep` with a non-string type, use a list, e.g.
-  `[foo: [:integer, :keep]]`.
+  To use `:keep` with a type other than `:string`, use a list as the type
+  for the switch. For example: `[foo: [:integer, :keep]]`.
 
-  ### Examples
+  ### Negation switches
 
-  Here are some examples of option parser working with different types
-  and modifiers:
+  In case a switch `SWITCH` is specified to have type `:boolean`, it may be
+  passed as `--no-SWITCH` as well which will set the option to `false`:
+
+      iex> OptionParser.parse(["--no-op", "path/to/file"], switches: [op: :boolean])
+      {[op: false], ["path/to/file"], []}
+
+  ### Parsing dynamic switches
+
+  `OptionParser` also includes a dynamic mode where it will attempt to parse
+  switches dynamically. Such can be done by not specifying the `:switches` or
+  `:strict` option.
+
+      iex> OptionParser.parse(["--debug"])
+      {[debug: true], [], []}
+
+
+  Switches followed by a value will be assigned the value, as a string. Switches
+  without an argument, like `--debug` in the examples above, will automatically be
+  set to `true`.
+
+  Since Elixir converts switches to atoms, the dynamic mode will only parse
+  switches that translates to atoms used by the runtime. Therefore, the code below
+  likely won't parse the given option since the `:option_parser_example` atom is
+  never used anywhere:
+
+      OptionParser.parse(["--option-parser-example"])
+      # Does nothing more...
+
+  However, the code below does since the `:option_parser_example` atom is used
+  at some point later (or earlier) on:
+
+      {opts, _, _} = OptionParser.parse(["--option-parser-example"])
+      opts[:option_parser_example]
+
+  In other words, when using dynamic mode, Elixir will do the correct thing and
+  only parse options that are used by the runtime, ignoring all others. If you
+  would like to parse all switches, regardless if they exist or not, you can
+  force creation of atoms by passing `allow_nonexistent_atoms: true` as option.
+  Such option is useful when you are building command-line applications that
+  receive dynamically-named arguments but must be used with care on long-running
+  systems.
+
+  Switches followed by a value will be assigned the value, as a string.
+  Switches without an argument, like `--debug` in the examples above, will
+  automatically be set to `true`.
+
+  ## Aliases
+
+  A set of aliases can be specified in the `:aliases` option:
+
+      iex> OptionParser.parse(["-d"], aliases: [d: :debug])
+      {[debug: true], [], []}
+
+  ## Examples
+
+  Here are some examples of working with different types and modifiers:
 
       iex> OptionParser.parse(["--unlock", "path/to/file"], strict: [unlock: :boolean])
       {[unlock: true], ["path/to/file"], []}
@@ -121,21 +190,6 @@ defmodule OptionParser do
       iex> OptionParser.parse(["--unlock", "path/to/file", "--unlock", "path/to/another/file"], strict: [unlock: :keep])
       {[unlock: "path/to/file", unlock: "path/to/another/file"], [], []}
 
-  ### Negation switches
-
-  In case a switch is declared as boolean, it may be passed as `--no-SWITCH`
-  which will set the option to `false`:
-
-      iex> OptionParser.parse(["--no-op", "path/to/file"], switches: [op: :boolean])
-      {[op: false], ["path/to/file"], []}
-
-  ## Aliases
-
-  A set of aliases can be given as options too:
-
-      iex> OptionParser.parse(["-d"], aliases: [d: :debug])
-      {[debug: true], [], []}
-
   """
   @spec parse(argv, options) :: {parsed, argv, errors}
   def parse(argv, opts \\ []) when is_list(argv) and is_list(opts) do
@@ -146,14 +200,16 @@ defmodule OptionParser do
   The same as `parse/2` but raises an `OptionParser.ParseError`
   exception if any invalid options are given.
 
-  If there weren't any errors, returns a three-element tuple as follows:
+  If there are no errors, returns a `{parsed, rest}` tuple where:
 
-      1. parsed options,
-      2. remaining arguments,
-      3. empty list.
+    * `parsed` is the list of parsed switches (same as in `parse/2`)
+    * `rest` is the list of arguments (same as in `parse/2`)
 
   ## Examples
 
+      iex> OptionParser.parse!(["--debug", "path/to/file"], strict: [debug: :boolean])
+      {[debug: true], ["path/to/file"]}
+
       iex> OptionParser.parse!(["--limit", "xyz"], strict: [limit: :integer])
       ** (OptionParser.ParseError) 1 error found!
       --limit : Expected type integer, got "xyz"
@@ -172,7 +228,7 @@ defmodule OptionParser do
   @spec parse!(argv, options) :: {parsed, argv} | no_return
   def parse!(argv, opts \\ []) when is_list(argv) and is_list(opts) do
     case parse(argv, opts) do
-      {parsed, argv, []} -> {parsed, argv}
+      {parsed, args, []} -> {parsed, args}
       {_, _, errors} -> raise ParseError, format_errors(errors, opts)
     end
   end
@@ -185,10 +241,12 @@ defmodule OptionParser do
 
   ## Example
 
-      iex> OptionParser.parse_head(["--source", "lib", "test/enum_test.exs", "--verbose"])
+      iex> OptionParser.parse_head(["--source", "lib", "test/enum_test.exs", "--verbose"],
+      ...>                         switches: [source: :string, verbose: :boolean])
       {[source: "lib"], ["test/enum_test.exs", "--verbose"], []}
 
-      iex> OptionParser.parse_head(["--verbose", "--source", "lib", "test/enum_test.exs", "--unlock"])
+      iex> OptionParser.parse_head(["--verbose", "--source", "lib", "test/enum_test.exs", "--unlock"],
+      ...>                         switches: [source: :string, verbose: :boolean, unlock: :boolean])
       {[verbose: true, source: "lib"], ["test/enum_test.exs", "--unlock"], []}
 
   """
@@ -201,15 +259,19 @@ defmodule OptionParser do
   The same as `parse_head/2` but raises an `OptionParser.ParseError`
   exception if any invalid options are given.
 
-  If there weren't any errors, returns a three-element tuple as follows:
+  If there are no errors, returns a `{parsed, rest}` tuple where:
 
-      1. parsed options,
-      2. remaining arguments,
-      3. empty list.
+    * `parsed` is the list of parsed switches (same as in `parse_head/2`)
+    * `rest` is the list of arguments (same as in `parse_head/2`)
 
   ## Examples
 
-      iex> OptionParser.parse_head!(["--number", "lib", "test/enum_test.exs", "--verbose"], strict: [number: :integer])
+      iex> OptionParser.parse_head!(["--source", "lib", "path/to/file", "--verbose"],
+      ...>                         switches: [source: :string, verbose: :boolean])
+      {[source: "lib"], ["path/to/file", "--verbose"]}
+
+      iex> OptionParser.parse_head!(["--number", "lib", "test/enum_test.exs", "--verbose"],
+      ...>                          strict: [number: :integer])
       ** (OptionParser.ParseError) 1 error found!
       --number : Expected type integer, got "lib"
 
@@ -219,10 +281,10 @@ defmodule OptionParser do
       --verbose : Missing argument of type integer
       --source : Expected type integer, got "lib"
   """
-  @spec parse_head!(argv, options) :: {parsed, argv, errors} | no_return
+  @spec parse_head!(argv, options) :: {parsed, argv} | no_return
   def parse_head!(argv, opts \\ []) when is_list(argv) and is_list(opts) do
     case parse_head(argv, opts) do
-      {parsed, argv, []} -> {parsed, argv}
+      {parsed, args, []} -> {parsed, args}
       {_, _, errors} -> raise ParseError, format_errors(errors, opts)
     end
   end
@@ -231,12 +293,12 @@ defmodule OptionParser do
     {Enum.reverse(opts), Enum.reverse(args), Enum.reverse(invalid)}
   end
 
-  defp do_parse(argv, {aliases, switches, strict}=config, opts, args, invalid, all?) do
-    case next(argv, aliases, switches, strict) do
+  defp do_parse(argv, {aliases, switches, strict?, allow_nonexistent_atoms?} = config, opts, args, invalid, all?) do
+    case next(argv, aliases, switches, strict?, allow_nonexistent_atoms?) do
       {:ok, option, value, rest} ->
         # the option exists and it was successfully parsed
         kinds = List.wrap Keyword.get(switches, option)
-        new_opts = do_store_option(opts, option, value, kinds)
+        new_opts = store_option(opts, option, value, kinds)
         do_parse(rest, config, new_opts, args, invalid, all?)
 
       {:invalid, option, value, rest} ->
@@ -250,7 +312,7 @@ defmodule OptionParser do
       {:error, ["--" | rest]} ->
         {Enum.reverse(opts), Enum.reverse(args, rest), Enum.reverse(invalid)}
 
-      {:error, [arg | rest]=remaining_args} ->
+      {:error, [arg | rest] = remaining_args} ->
         # there is no option
         if all? do
           do_parse(rest, config, opts, [arg | args], invalid, all?)
@@ -264,22 +326,21 @@ defmodule OptionParser do
   Low-level function that parses one option.
 
   It accepts the same options as `parse/2` and `parse_head/2`
-  as both functions are built on top of next. This function
+  as both functions are built on top of this function. This function
   may return:
 
     * `{:ok, key, value, rest}` - the option `key` with `value` was
       successfully parsed
 
     * `{:invalid, key, value, rest}` - the option `key` is invalid with `value`
-      (returned when the switch type does not match the one given via the
-      command line)
+      (returned when the value cannot be parsed according to the switch type)
 
     * `{:undefined, key, value, rest}` - the option `key` is undefined
       (returned in strict mode when the switch is unknown)
 
-    * `{:error, rest}` - there are no switches at the top of the given argv
-  """
+    * `{:error, rest}` - there are no switches at the head of the given `argv`
 
+  """
   @spec next(argv, options) ::
         {:ok, key :: atom, value :: term, argv} |
         {:invalid, String.t, String.t | nil, argv} |
@@ -287,82 +348,132 @@ defmodule OptionParser do
         {:error, argv}
 
   def next(argv, opts \\ []) when is_list(argv) and is_list(opts) do
-    {aliases, switches, strict} = compile_config(opts)
-    next(argv, aliases, switches, strict)
+    {aliases, switches, strict?, allow_nonexistent_atoms?} = compile_config(opts)
+    next(argv, aliases, switches, strict?, allow_nonexistent_atoms?)
   end
 
-  defp next([], _aliases, _switches, _strict) do
+  defp next([], _aliases, _switches, _strict?, _allow_nonexistent_atoms?) do
     {:error, []}
   end
 
-  defp next(["--" | _]=argv, _aliases, _switches, _strict) do
+  defp next(["--" | _] = argv, _aliases, _switches, _strict?, _allow_nonexistent_atoms?) do
     {:error, argv}
   end
 
-  defp next(["-" | _]=argv, _aliases, _switches, _strict) do
+  defp next(["-" | _] = argv, _aliases, _switches, _strict?, _allow_nonexistent_atoms?) do
     {:error, argv}
   end
 
-  defp next(["- " <> _ | _]=argv, _aliases, _switches, _strict) do
+  defp next(["- " <> _ | _] = argv, _aliases, _switches, _strict?, _allow_nonexistent_atoms?) do
     {:error, argv}
   end
 
-  defp next(["-" <> option | rest] = argv, aliases, switches, strict) do
+  # Handles --foo or --foo=bar
+  defp next(["--" <> option | rest], _aliases, switches, strict?, allow_nonexistent_atoms?) do
+    {option, value} = split_option(option)
+    tagged = tag_option(option, switches, allow_nonexistent_atoms?)
+    do_next(tagged, value, "--" <> option, rest, switches, strict?, allow_nonexistent_atoms?)
+  end
+
+  # Handles -a, -abc, -abc=something
+  defp next(["-" <> option | rest] = argv, aliases, switches, strict?, allow_nonexistent_atoms?) do
     {option, value} = split_option(option)
     original = "-" <> option
-    tagged = tag_option(option, switches, aliases)
 
     cond do
-      negative_number?(original) ->
+      is_nil(value) and negative_number?(original) ->
         {:error, argv}
-      strict and not option_defined?(tagged, switches) ->
+      String.contains?(option, ["-", "_"]) ->
         {:undefined, original, value, rest}
-      true ->
-        {option, kinds, value} = normalize_option(tagged, value, switches)
-        {value, kinds, rest} = normalize_value(value, kinds, rest, strict)
-        case validate_option(value, kinds) do
-          {:ok, new_value} -> {:ok, option, new_value, rest}
-          :invalid         -> {:invalid, original, value, rest}
+      String.length(option) > 1 ->
+        key = get_option_key(option, allow_nonexistent_atoms?)
+        option_key = aliases[key]
+        if key && option_key do
+          IO.warn "multi-letter aliases are deprecated, got: #{inspect(key)}"
+          do_next({:default, option_key}, value, original, rest, switches, strict?, allow_nonexistent_atoms?)
+        else
+          next(expand_multiletter_alias(option, value) ++ rest, aliases, switches, strict?, allow_nonexistent_atoms?)
         end
+      true ->
+        # We have a regular one-letter alias here
+        tagged = tag_oneletter_alias(option, aliases, allow_nonexistent_atoms?)
+        do_next(tagged, value, original, rest, switches, strict?, allow_nonexistent_atoms?)
     end
   end
 
-  defp next(argv, _aliases, _switches, _strict) do
+  defp next(argv, _aliases, _switches, _strict?, _allow_nonexistent_atoms?) do
     {:error, argv}
   end
 
-  @doc """
-  Receives a key-value enumerable and converts it to argv.
+  defp do_next(tagged, value, original, rest, switches, strict?, allow_nonexistent_atoms?) do
+    if strict? and not option_defined?(tagged, switches) do
+      {:undefined, original, value, rest}
+    else
+      {option, kinds, value} = normalize_option(tagged, value, switches)
+      {value, kinds, rest} = normalize_value(value, kinds, rest, strict?)
+      case validate_option(value, kinds) do
+        {:ok, new_value} -> {:ok, option, new_value, rest}
+        :invalid         -> {:invalid, original, value, rest}
+      end
+    end
+  end
 
-  Keys must be atoms. Keys with nil value are discarded,
+  @doc """
+  Receives a key-value enumerable and converts it to `t:argv/0`.
+
+  Keys must be atoms. Keys with `nil` value are discarded,
   boolean values are converted to `--key` or `--no-key`
-  and all other values are converted using `to_string/1`.
+  (if the value is `true` or `false`, respectively),
+  and all other values are converted using `Kernel.to_string/1`.
+
+  It is advised to pass to `to_argv/2` the same set of `options`
+  given to `parse/2`. Some switches can only be reconstructed
+  correctly with the `switches` information in hand.
 
   ## Examples
 
       iex>  OptionParser.to_argv([foo_bar: "baz"])
       ["--foo-bar", "baz"]
-
       iex>  OptionParser.to_argv([bool: true, bool: false, discarded: nil])
       ["--bool", "--no-bool"]
 
+  Some switches will output different values based on the switches
+  flag:
+
+      iex> OptionParser.to_argv([number: 2], switches: [])
+      ["--number", "2"]
+      iex> OptionParser.to_argv([number: 2], switches: [number: :count])
+      ["--number", "--number"]
+
   """
-  @spec to_argv(Enumerable.t) :: argv
-  def to_argv(enum) do
+  @spec to_argv(Enumerable.t, options) :: argv
+  def to_argv(enum, opts \\ []) do
+    switches = Keyword.get(opts, :switches, [])
     Enum.flat_map(enum, fn
       {_key, nil}  -> []
       {key, true}  -> [to_switch(key)]
       {key, false} -> [to_switch(key, "--no-")]
-      {key, value} -> [to_switch(key), to_string(value)]
+      {key, value} -> to_argv(key, value, switches)
     end)
   end
 
+  defp to_argv(key, value, switches) do
+    if switches[key] == :count do
+      List.duplicate(to_switch(key), value)
+    else
+      [to_switch(key), to_string(value)]
+    end
+  end
+
   defp to_switch(key, prefix \\ "--") when is_atom(key) do
     prefix <> String.replace(Atom.to_string(key), "_", "-")
   end
 
   @doc ~S"""
-  Splits a string into argv chunks.
+  Splits a string into `t:argv/0` chunks.
+
+  This function splits the given `string` into a list of strings in a similar
+  way to many shells.
 
   ## Examples
 
@@ -371,6 +482,7 @@ defmodule OptionParser do
 
       iex> OptionParser.split("foo \"bar baz\"")
       ["foo", "bar baz"]
+
   """
   @spec split(String.t) :: argv
   def split(string) do
@@ -418,19 +530,20 @@ defmodule OptionParser do
 
   defp compile_config(opts) do
     aliases = opts[:aliases] || []
+    allow_nonexistent_atoms? = opts[:allow_nonexistent_atoms] || false
 
-    {switches, strict} = cond do
+    {switches, strict?} = cond do
       opts[:switches] && opts[:strict] ->
         raise ArgumentError, ":switches and :strict cannot be given together"
-      s = opts[:switches] ->
-        {s, false}
-      s = opts[:strict] ->
-        {s, true}
+      switches = opts[:switches] ->
+        {switches, false}
+      strict = opts[:strict] ->
+        {strict, true}
       true ->
         {[], false}
     end
 
-    {aliases, switches, strict}
+    {aliases, switches, strict?, allow_nonexistent_atoms?}
   end
 
   defp validate_option(value, kinds) do
@@ -470,7 +583,7 @@ defmodule OptionParser do
     end
   end
 
-  defp do_store_option(dict, option, value, kinds) do
+  defp store_option(dict, option, value, kinds) do
     cond do
       :count in kinds ->
         Keyword.update(dict, option, value, & &1 + 1)
@@ -481,34 +594,42 @@ defmodule OptionParser do
     end
   end
 
-  defp tag_option("-no-" <> option, switches, _aliases) do
+  defp tag_option("no-" <> option = original, switches, allow_nonexistent_atoms?) do
     cond do
-      (negated = get_option(option)) && :boolean in List.wrap(switches[negated]) ->
+      (negated = get_option_key(option, allow_nonexistent_atoms?)) && :boolean in List.wrap(switches[negated]) ->
         {:negated, negated}
-      option = get_option("no-" <> option) ->
-        {:default, option}
+      option_key = get_option_key(original, allow_nonexistent_atoms?) ->
+        {:default, option_key}
       true ->
         :unknown
     end
   end
 
-  defp tag_option("-" <> option, _switches, _aliases) do
-    if option = get_option(option) do
-      {:default, option}
+  defp tag_option(option, _switches, allow_nonexistent_atoms?) do
+    if option_key = get_option_key(option, allow_nonexistent_atoms?) do
+      {:default, option_key}
     else
       :unknown
     end
   end
 
-  defp tag_option(option, _switches, aliases) when is_binary(option) do
-    opt = get_option(option)
-    if alias = aliases[opt] do
-      {:default, alias}
+  defp tag_oneletter_alias(alias, aliases, allow_nonexistent_atoms?) when is_binary(alias) do
+    if option_key = aliases[to_existing_key(alias, allow_nonexistent_atoms?)] do
+      {:default, option_key}
     else
       :unknown
     end
   end
 
+  defp expand_multiletter_alias(letters, value) when is_binary(letters) do
+    {last, expanded} =
+      letters
+      |> String.codepoints()
+      |> Enum.map(&("-" <> &1))
+      |> List.pop_at(-1)
+    expanded ++ [last <> if(value, do: "=" <> value, else: "")]
+  end
+
   defp option_defined?(:unknown, _switches) do
     false
   end
@@ -537,7 +658,7 @@ defmodule OptionParser do
     {option, List.wrap(switches[option]), value}
   end
 
-  defp normalize_value(nil, kinds, t, strict) do
+  defp normalize_value(nil, kinds, t, strict?) do
     cond do
       :boolean in kinds ->
         {true, kinds, t}
@@ -546,7 +667,7 @@ defmodule OptionParser do
       value_in_tail?(t) ->
         [h | t] = t
         {h, kinds, t}
-      kinds == [] and strict ->
+      kinds == [] and strict? ->
         {nil, kinds, t}
       kinds == [] ->
         {true, kinds, t}
@@ -555,7 +676,7 @@ defmodule OptionParser do
     end
   end
 
-  defp normalize_value(value, kinds, t, _) do
+  defp normalize_value(value, kinds, t, _strict?) do
     {value, kinds, t}
   end
 
@@ -583,9 +704,19 @@ defmodule OptionParser do
   defp to_underscore(<<>>, acc),
     do: acc
 
-  defp get_option(option) do
-    if str = to_underscore(option) do
-      String.to_atom(str)
+  def get_option_key(option, allow_nonexistent_atoms?) do
+    if string = to_underscore(option) do
+      to_existing_key(string, allow_nonexistent_atoms?)
+    end
+  end
+
+  defp to_existing_key(option, true),
+    do: String.to_atom(option)
+  defp to_existing_key(option, false) do
+    try do
+      String.to_existing_atom(option)
+    rescue
+      ArgumentError -> nil
     end
   end
 
@@ -593,34 +724,35 @@ defmodule OptionParser do
     match?({_, ""}, Float.parse(arg))
   end
 
-  defp format_errors(errors, opts) do
+  defp format_errors([_ | _] = errors, opts) do
     types = opts[:switches] || opts[:strict]
-    info  = Enum.map(errors, &format_error(&1, opts, types))
-    total = length(errors)
-    error = if total == 1, do: "error", else: "errors"
-    "#{total} #{error} found!#{info}"
+    error_count = length(errors)
+    error = if error_count == 1, do: "error", else: "errors"
+    "#{error_count} #{error} found!\n" <>
+      Enum.map_join(errors, "\n", &format_error(&1, opts, types))
   end
 
   defp format_error({option, nil}, opts, types) do
     if type = get_type(option, opts, types) do
-      "\n#{option} : Missing argument of type #{type}"
+      "#{option} : Missing argument of type #{type}"
     else
-      "\n#{option} : Unknown option"
+      "#{option} : Unknown option"
     end
   end
 
   defp format_error({option, value}, opts, types) do
     type = get_type(option, opts, types)
-    "\n#{option} : Expected type #{type}, got #{inspect value}"
+    "#{option} : Expected type #{type}, got #{inspect value}"
   end
 
   defp get_type(option, opts, types) do
-    option_key = option |> String.trim_leading("-") |> get_option()
+    allow_nonexistent_atoms? = opts[:allow_nonexistent_atoms] || false
+    key = option |> String.trim_leading("-") |> get_option_key(allow_nonexistent_atoms?)
 
-    if option_alias = opts[:aliases][option_key] do
-      types[option_alias]
-    else
+    if option_key = opts[:aliases][key] do
       types[option_key]
+    else
+      types[key]
     end
   end
 end

lib/elixir/lib/path.ex

@@ -105,7 +105,7 @@ defmodule Path do
   defp do_absname_join(<<?/, rest::binary>>, relativename, [?/ | result], os_type), do:
     do_absname_join(rest, relativename, [?/ | result], os_type)
   defp do_absname_join(<<>>, <<>>, result, os_type), do:
-    IO.iodata_to_binary(reverse_maybe_remove_dirsep(result, os_type))
+    IO.iodata_to_binary(reverse_maybe_remove_dir_sep(result, os_type))
   defp do_absname_join(<<>>, relativename, [?: | rest], :win32), do:
     do_absname_join(relativename, <<>>, [?: | rest], :win32)
   defp do_absname_join(<<>>, relativename, [?/ | result], os_type), do:
@@ -115,13 +115,13 @@ defmodule Path do
   defp do_absname_join(<<char, rest::binary>>, relativename, result, os_type), do:
     do_absname_join(rest, relativename, [char | result], os_type)
 
-  defp reverse_maybe_remove_dirsep([?/, ?:, letter], :win32), do:
+  defp reverse_maybe_remove_dir_sep([?/, ?:, letter], :win32), do:
     [letter, ?:, ?/]
-  defp reverse_maybe_remove_dirsep([?/], _), do:
+  defp reverse_maybe_remove_dir_sep([?/], _), do:
     [?/]
-  defp reverse_maybe_remove_dirsep([?/ | name], _), do:
+  defp reverse_maybe_remove_dir_sep([?/ | name], _), do:
     :lists.reverse(name)
-  defp reverse_maybe_remove_dirsep(name, _), do:
+  defp reverse_maybe_remove_dir_sep(name, _), do:
     :lists.reverse(name)
 
   @doc """
@@ -187,7 +187,7 @@ defmodule Path do
   """
   @spec type(t) :: :absolute | :relative | :volumerelative
   def type(name) when is_list(name) or is_binary(name) do
-    pathtype(name, major_os_type) |> elem(0)
+    pathtype(name, major_os_type()) |> elem(0)
   end
 
   @doc """
@@ -428,10 +428,10 @@ defmodule Path do
   end
 
   @doc """
-  Joins a list of strings.
+  Joins a list of paths.
 
-  This function should be used to convert a list of strings to a path.
-  Note that any trailing slash is removed on join.
+  This function should be used to convert a list of paths to a path.
+  Note that any trailing slash is removed when joining.
 
   ## Examples
 
@@ -445,11 +445,11 @@ defmodule Path do
       "/foo/bar"
 
   """
-  @spec join([t]) :: binary
+  @spec join(nonempty_list(t)) :: binary
   def join([name1, name2 | rest]), do:
     join([join(name1, name2) | rest])
   def join([name]), do:
-    name
+    IO.chardata_to_string(name)
 
   @doc """
   Joins two paths.
@@ -467,16 +467,16 @@ defmodule Path do
   def join(left, right) do
     left    = IO.chardata_to_string(left)
     os_type = major_os_type()
-    do_join(left, right, os_type) |> remove_dirsep(os_type)
+    do_join(left, right, os_type) |> remove_dir_sep(os_type)
   end
 
   defp do_join("", right, os_type),   do: relative(right, os_type)
   defp do_join("/", right, os_type),  do: "/" <> relative(right, os_type)
-  defp do_join(left, right, os_type), do: remove_dirsep(left, os_type) <> "/" <> relative(right, os_type)
+  defp do_join(left, right, os_type), do: remove_dir_sep(left, os_type) <> "/" <> relative(right, os_type)
 
-  defp remove_dirsep("", _os_type), do: ""
-  defp remove_dirsep("/", _os_type), do: "/"
-  defp remove_dirsep(bin, os_type) do
+  defp remove_dir_sep("", _os_type), do: ""
+  defp remove_dir_sep("/", _os_type), do: "/"
+  defp remove_dir_sep(bin, os_type) do
     last = :binary.last(bin)
     if last == ?/ or (last == ?\\ and os_type == :win32) do
       binary_part(bin, 0, byte_size(bin) - 1)
@@ -522,13 +522,8 @@ defmodule Path do
       call({:read_link_info, file})
     end
 
-    # For compatibility with buggy Erlang 17.1.
-    def read_file_info(file) do
-      call({:read_link_info, file})
-    end
-
     def list_dir(dir) do
-      case call({:list_dir, dir})  do
+      case call({:list_dir, dir}) do
         {:ok, files} ->
           {:ok, for(file <- files, hd(file) != ?., do: file)}
         other ->
@@ -625,7 +620,7 @@ defmodule Path do
   defp resolve_home(""), do: System.user_home!
 
   defp resolve_home(rest) do
-    case {rest, major_os_type} do
+    case {rest, major_os_type()} do
       {"\\" <> _, :win32} ->
         System.user_home! <> rest
       {"/" <> _, _} ->

lib/elixir/lib/port.ex

@@ -1,6 +1,162 @@
 defmodule Port do
-  @moduledoc """
-  Functions related to Erlang ports.
+  @moduledoc ~S"""
+  Functions for interacting with the external world through ports.
+
+  Ports provide a mechanism to start operating system processes external
+  to the Erlang VM and communicate with them via message passing.
+
+  ## Example
+
+      iex> port = Port.open({:spawn, "cat"}, [:binary])
+      iex> send port, {self(), {:command, "hello"}}
+      iex> send port, {self(), {:command, "world"}}
+      iex> flush()
+      {#Port<0.1444>, {:data, "hello"}}
+      {#Port<0.1444>, {:data, "world"}}
+      iex> send port, {self(), :close}
+      :ok
+      iex> flush()
+      {#Port<0.1464>, :closed}
+      :ok
+
+  In the example above, we have created a new port that executes the
+  program `cat`. `cat` is a program available on UNIX systems that
+  receives data from multiple inputs and concatenates them in the output.
+
+  After the port was created, we sent it two commands in the form of
+  messages using `Kernel.send/2`. The first command has the binary payload
+  of "hello" and the second has "world".
+
+  After sending those two messages, we invoked the IEx helper `flush()`,
+  which printed all messages received from the port, in this case we got
+  "hello" and "world" back. Notice the messages are in binary because we
+  passed the `:binary` option when opening the port in `Port.open/2`. Without
+  such option, it would have yielded a list of bytes.
+
+  Once everything was done, we closed the port.
+
+  Elixir provides many conveniences for working with ports and some drawbacks.
+  We will explore those below.
+
+  ## Message and function APIs
+
+  There are two APIs for working with ports. It can be either asynchronous via
+  message passing, as in the example above, or by calling the functions on this
+  module.
+
+  The messages supported by ports and their counterpart function APIs are
+  listed below:
+
+    * `{pid, {:command, binary}}` - sends the given data to the port.
+      See `command/3`.
+
+    * `{pid, :close}` - closes the port. Unless the port is already closed,
+      the port will reply with `{port, :closed}` message once it has flushed
+      its buffers and effectively closed. See `close/1`.
+
+    * `{pid, {:connect, new_pid}}` - sets the `new_pid` as the new owner of
+      the port. Once a port is opened, the port is linked and connected to the
+      caller process and communication to the port only happens through the
+      connected process. This message makes `new_pid` the new connected processes.
+      Unless the port is dead, the port will reply to the old owner with
+      `{port, :connected}`. See `connect/2`.
+
+  On its turn, the port will send the connected process the following messages:
+
+    * `{port, {:port, data}}` - data sent by the port
+    * `{port, :closed}` - reply to the `{pid, :close}` message
+    * `{port, :connected}` - reply to the `{pid, {:connect, new_pid}}` message
+    * `{:EXIT, port, reason}` - exit signals in case the port crashes and the
+      owner process is trapping exits
+
+  ## Open mechanisms
+
+  The port can be opened through four main mechanisms.
+
+  As a short summary, prefer to using the `:spawn` and `:spawn_executable`
+  options mentioned below. The other two options, `:spawn_driver` and `:fd`
+  are for advanced usage within the VM. Also consider using `System.cmd/3`
+  if all you want is to execute a program and retrieve its return value.
+
+  ### spawn
+
+  The `:spawn` tuple receives a binary that is going to be executed as a
+  full invocation. For example, we can use it to invoke "echo hello" directly:
+
+      iex> port = Port.open({:spawn, "echo oops"}, [:binary])
+      iex> flush()
+      {#Port<0.1444>, {:data, "oops\n"}}
+
+  `:spawn` will retrieve the program name from the argument and traverse your
+  OS `$PATH` environment variable looking for a matching program.
+
+  Although the above is handy, it means it is impossible to invoke an executable
+  that has whitespaces on its name or in any of its arguments. For those reasons,
+  most times it is preferrable to execute `:spawn_executable`.
+
+  ### spawn_executable
+
+  Spawn executable is a more restricted and explicit version of spawn. It expects
+  full file paths to the executable you want to execute. If they are in your `$PATH`,
+  they can be retrieved by calling `System.find_executable/1`:
+
+      iex> path = System.find_executable("echo")
+      iex> port = Port.open({:spawn_executable, path}, [:binary, args: ["hello world"]])
+      iex> flush()
+      {#Port<0.1380>, {:data, "hello world\n"}}
+
+  When using `:spawn_executable`, the list of arguments can be passed via
+  the `:args` option as done above. For the full list of options, see the
+  documentation for the Erlang function `:erlang.open_port/2`.
+
+  ### spawn_driver
+
+  Spawn driver is used to start Port Drivers, which are programs written in
+  C that implements a specific communication protocols and are dynamically
+  linked to the Erlang VM. Port drivers are an advanced topic and one of the
+  mechanisms for integrating C code, alongside NIFs. For more information,
+  [please check the Erlang docs](http://erlang.org/doc/reference_manual/ports.html).
+
+  ### fd
+
+  The `:fd` name option allows developers to access `in` and `out` file
+  descriptors used by the Erlang VM. You would use those only if you are
+  reimplementing core part of the Runtime System, such as the `:user` and
+  `:shell` processes.
+
+  ## Zombie processes
+
+  A port can be closed via the `close/1` function or by sending a `{pid, :close}`
+  message. However, if the VM crashes, a long-running program started by the port
+  will have its stdin and stdout channels closed but **it won't be automatically
+  terminated**.
+
+  While most UNIX command line tools will exit once its communication channels
+  are closed, not all command line applications will do so. While we encourage
+  graceful termination by detecting if stdin/stdout has been closed, we do not
+  always have control over how 3rd party software terminates. In those cases,
+  you can wrap the application in a script that checks for stdin. Here is such
+  script in bash:
+
+      #!/bin/sh
+      "$@"
+      pid=$!
+      while read line ; do
+        :
+      done
+      kill -KILL $pid
+
+
+  Now instead of:
+
+      Port.open({:spawn_executable, "/path/to/program"},
+                [args: ["a", "b", "c"]])
+
+  You may invoke:
+
+      Port.open({:spawn_executable, "/path/to/wrapper"},
+                [args: ["/path/to/program", "a", "b", "c"]])
+
   """
 
   @type name :: {:spawn, charlist | binary} |
@@ -9,24 +165,22 @@ defmodule Port do
                 {:fd, non_neg_integer, non_neg_integer}
 
   @doc """
-  Opens an Erlang port given a tuple `name` and a list of `settings`.
+  Opens a port given a tuple `name` and a list of `options`.
 
-  ## Name
+  The module documentation above contains documentation and examples
+  for the supported `name` values, summarized below:
 
-  The supported values for `name` are:
+    * `{:spawn, command}` - runs an external program. `command` must contain
+      the program name and optionally a list of arguments separated by space.
+      If passing programs or arguments with space in their name, use the next option.
+    * `{:spawn_executable, filename}` - runs the executable given by the absolute
+      file name `filename`. Arguments can be passed via the `:args` option.
+    * `{:spawn_driver, command}` - spawns so-called port drivers.
+    * `{:fd, fd_in, fd_out}` - accesses file descriptors, `fd_in` and `fd_out`
+      opened by the VM.
 
-    * `{:spawn, command}` - to run an external program. The first space separated
-      word of `command` will be considered as the name of the program to run, so
-      use `{:spawn_executable, command}` to run a program having spaces in its name.
-    * `{:spawn_driver, command}` - similar to `{:spawn, command}`, but to run a
-      loaded driver.
-    * `{:spawn_executable, filename}` - similar to `{:spawn, filename}`, but to run
-      an external executable. With this option, `filename` in its whole is considered
-      the name of the program to execute.
-    * `{:fd, fd_in, fd_out}` - to access file descriptors used by Erlang, `fd_in`
-      being used for standard input, `fd_out` for standard output.
-
-  For more information, see [`:erlang.open_port/2`](http://www.erlang.org/doc/man/erlang.html#open_port-2).
+  For more information and the list of options, see
+  [`:erlang.open_port/2`](http://www.erlang.org/doc/man/erlang.html#open_port-2).
 
   Inlined by the compiler.
   """
@@ -72,36 +226,7 @@ defmodule Port do
   end
 
   @doc """
-  Sends a synchronous control command to the `port` and returns its reply as a binary.
-
-  Not all port drivers support this feature.
-
-  For more information, see [`:erlang.port_control/3`](http://www.erlang.org/doc/man/erlang.html#port_control-3).
-
-  Inlined by the compiler.
-  """
-  @spec control(port, integer, iodata) :: iodata | binary
-  def control(port, operation, data) do
-    :erlang.port_control(port, operation, data)
-  end
-
-  @doc """
-  Makes a synchronous call to the `port` and returns its reply as a term.
-
-  Not all port drivers support this control feature.
-
-  For more information, see [`:erlang.port_call/3`](http://www.erlang.org/doc/man/erlang.html#port_call-3).
-
-  Inlined by the compiler.
-  """
-  @spec call(port, integer, term) :: term
-  def call(port, operation, data) do
-    :erlang.port_call(port, operation, data)
-  end
-
-  @doc """
-  Returns information about the `port`
-  or `nil` if the port is closed.
+  Returns information about the `port` or `nil` if the port is closed.
 
   For more information, see [`:erlang.port_info/1`](http://www.erlang.org/doc/man/erlang.html#port_info-1).
   """
@@ -110,8 +235,7 @@ defmodule Port do
   end
 
   @doc """
-  Returns information about the `port`
-  or `nil` if the port is closed.
+  Returns information about the `port` or `nil` if the port is closed.
 
   For more information, see [`:erlang.port_info/2`](http://www.erlang.org/doc/man/erlang.html#port_info-2).
   """
@@ -130,9 +254,7 @@ defmodule Port do
   end
 
   @doc """
-  Returns a list of the ports for the current node.
-
-  For more information, see [`:erlang.ports/0`](http://www.erlang.org/doc/man/erlang.html#ports-0).
+  Returns a list of all ports in the current node.
 
   Inlined by the compiler.
   """

lib/elixir/lib/process.ex

@@ -103,15 +103,14 @@ defmodule Process do
        message `{:EXIT, from, reason}` and delivered to the message queue
        of `pid`.
 
-    3. If `reason` is the atom `:normal`, `pid` will not exit (unless it
-       is the calling process's pid, in which case it will exit with the
-       reason `:normal`). If it is trapping exits, the exit signal is
-       transformed into a message `{:EXIT, from, :normal}` and delivered
-       to its message queue.
+  If `reason` is the atom `:normal`, `pid` will not exit (unless `pid` is
+  the calling process, in which case it will exit with the reason `:normal`).
+  If it is trapping exits, the exit signal is transformed into a message
+  `{:EXIT, from, :normal}` and delivered to its message queue.
 
-    4. If `reason` is the atom `:kill`, that is if `exit(pid, :kill)` is
-       called, an untrappable exit signal is sent to `pid` which will
-       unconditionally exit with exit reason `:killed`.
+  If `reason` is the atom `:kill`, that is if `exit(pid, :kill)` is called,
+  an untrappable exit signal is sent to `pid` which will unconditionally exit
+  with reason `:killed`.
 
   Inlined by the compiler.
 
@@ -217,10 +216,10 @@ defmodule Process do
       :noconnect
 
   """
-  @spec send(dest, msg, [option]) :: :ok | :noconnect | :nosuspend when
-        dest: pid | port | atom | {atom, node},
-        msg: any,
-        option: :noconnect | :nosuspend
+  @spec send(dest, msg, [option]) :: :ok | :noconnect | :nosuspend
+        when dest: pid | port | atom | {atom, node},
+             msg: any,
+             option: :noconnect | :nosuspend
   def send(dest, msg, options) do
     :erlang.send(dest, msg, options)
   end
@@ -228,7 +227,7 @@ defmodule Process do
   @doc """
   Sends `msg` to `dest` after `time` milliseconds.
 
-  If `dest` is a pid, it must be the pid of a local process, dead or alive.
+  If `dest` is a PID, it must be the PID of a local process, dead or alive.
   If `dest` is an atom, it must be the name of a registered process
   which is looked up at the time of delivery. No error is given if the name does
   not refer to a process.
@@ -236,14 +235,24 @@ defmodule Process do
   This function returns a timer reference, which can be read or canceled with
   `read_timer/1` and `cancel_timer/1`.
 
-  Finally, the timer will be automatically canceled if the given `dest` is a pid
-  which is not alive or when the given pid exits. Note that timers will not be
+  Finally, the timer will be automatically canceled if the given `dest` is a PID
+  which is not alive or when the given PID exits. Note that timers will not be
   automatically canceled when `dest` is an atom (as the atom resolution is done
   on delivery).
+
+  ## Options
+
+    * `:abs` - (boolean) when `false`, `time` is treated as relative to the
+    current monotonic time. When `true`, `time` is the absolute value of the
+    Erlang monotonic time at which `msg` should be delivered to `dest`.
+    To read more about Erlang monotonic time and other time-related concepts,
+    look at the documentation for the `System` module. Defaults to `false`.
+
   """
-  @spec send_after(pid | atom, term, non_neg_integer) :: reference
-  def send_after(dest, msg, time) do
-    :erlang.send_after(time, dest, msg)
+  @spec send_after(pid | atom, term, non_neg_integer, [option]) :: reference
+        when option: {:abs, boolean}
+  def send_after(dest, msg, time, opts \\ []) do
+    :erlang.send_after(time, dest, msg, opts)
   end
 
   @doc """
@@ -297,8 +306,8 @@ defmodule Process do
 
   The result depends on the given options. In particular,
   if `:monitor` is given as an option, it will return a tuple
-  containing the pid and the monitoring reference, otherwise
-  just the spawned process pid.
+  containing the PID and the monitoring reference, otherwise
+  just the spawned process PID.
 
   It also accepts extra options, for the list of available options
   check [`:erlang.spawn_opt/4`](http://www.erlang.org/doc/man/erlang.html#spawn_opt-4).
@@ -316,8 +325,8 @@ defmodule Process do
 
   The result depends on the given options. In particular,
   if `:monitor` is given as an option, it will return a tuple
-  containing the pid and the monitoring reference, otherwise
-  just the spawned process pid.
+  containing the PID and the monitoring reference, otherwise
+  just the spawned process PID.
 
   It also accepts extra options, for the list of available options
   check [`:erlang.spawn_opt/4`](http://www.erlang.org/doc/man/erlang.html#spawn_opt-4).
@@ -402,24 +411,41 @@ defmodule Process do
   end
 
   @doc """
-  Associates the atom `name` with a `pid` or a port identifier.
+  Registers the given `pid_or_port` under the given `name`.
 
-  `name`, can then be used instead of the `pid` / port identifier with the `Kernel.send/2`
-  function. `Process.register/2` will fail with `ArgumentError` if the pid supplied
-  is no longer alive, (check with `alive?/1`) or if the name is already registered
-  (check with `whereis/1`) or if the `pid` is already registered to a different `name`.
+  `name` must be an atom and can then be used instead of the
+  PID/port identifier when sending messages with `Kernel.send/2`.
+
+  `register/2` will fail with `ArgumentError` if the PID/Port is
+  not existing locally and alive, if the name is already registered
+  or if the `pid_or_port` is already registered to a different `name`.
+
+  The following names are reserved and cannot be assigned to
+  processes nor ports: `nil`, `false`, `true` or `:undefined`.
   """
   @spec register(pid | port, atom) :: true
-  def register(pid, name) when not name in [nil, false, true] and is_atom(name) do
-    :erlang.register(name, pid)
+  def register(pid_or_port, name) when is_atom(name) and not name in [nil, false, true, :undefined] do
+    :erlang.register(name, pid_or_port)
+  catch
+    :error, :badarg when node(pid_or_port) != node()  ->
+      message = "could not register the #{pid_or_port pid_or_port} because it belongs to another node"
+      :erlang.error ArgumentError.exception(message), [pid_or_port, name]
+    :error, :badarg ->
+      message = "could not register the #{pid_or_port pid_or_port} with " <>
+                "name #{inspect name}. Or it is not alive, or the name is already " <>
+                "taken, or it has already been given another name"
+      :erlang.error ArgumentError.exception(message), [pid_or_port, name]
   end
 
+  defp pid_or_port(pid) when is_pid(pid), do: "pid #{inspect pid}"
+  defp pid_or_port(port) when is_port(port), do: "port #{inspect port}"
+
   @doc """
-  Removes the registered `name`, associated with a pid or a port identifier.
+  Removes the registered `name`, associated with a PID
+  or a port identifier.
 
-  Fails with `ArgumentError` if the name is not registered to any pid or port.
-
-  See [`:erlang.unregister/1`](http://www.erlang.org/doc/man/erlang.html#unregister-1) for more info.
+  Fails with `ArgumentError` if the name is not registered
+  to any PID or port.
   """
   @spec unregister(atom) :: true
   def unregister(name) do
@@ -427,7 +453,7 @@ defmodule Process do
   end
 
   @doc """
-  Returns the pid or port identifier with the registered `name`.
+  Returns the PID or port identifier with the registered `name`.
   Returns `nil` if the name is not registered.
 
   See [`:erlang.whereis/1`](http://www.erlang.org/doc/man/erlang.html#whereis-1) for more info.
@@ -438,7 +464,7 @@ defmodule Process do
   end
 
   @doc """
-  Returns the pid of the group leader for the process which evaluates the function.
+  Returns the PID of the group leader for the process which evaluates the function.
   """
   @spec group_leader :: pid
   def group_leader do

lib/elixir/lib/protocol.ex

@@ -420,7 +420,7 @@ defmodule Protocol do
         _ = unquote(block)
 
         # Finalize expansion
-        unquote(after_defprotocol)
+        unquote(after_defprotocol())
       end
     end
   end
@@ -439,18 +439,28 @@ defmodule Protocol do
         struct_impl_for(struct)
       end
 
-      # Define the implementation for builtins.
+      # Define the implementation for built-ins
       :lists.foreach(fn {guard, mod} ->
         target = Module.concat(__MODULE__, mod)
 
         Kernel.def impl_for(data) when :erlang.unquote(guard)(data) do
           case impl_for?(unquote(target)) do
             true  -> unquote(target).__impl__(:target)
-            false -> any_impl_for
+            false -> any_impl_for()
           end
         end
       end, builtin)
 
+      # Define a catch-all impl_for/1 clause to pacify Dialyzer (since
+      # destructuring opaque types is illegal, Dialyzer will think none of the
+      # previous clauses matches opaque types, and without this clause, will
+      # conclude that impl_for can't handle an opaque argument). This is a hack
+      # since it relies on Dialyzer not being smart enough to conclude that all
+      # opaque types will get the any_impl_for/0 implementation.
+      Kernel.def impl_for(_) do
+        any_impl_for()
+      end
+
       @doc false
       @spec impl_for!(term) :: atom | no_return
       Kernel.def impl_for!(data) do
@@ -459,9 +469,9 @@ defmodule Protocol do
 
       # Internal handler for Any
       if @fallback_to_any do
-        Kernel.defp any_impl_for, do: __MODULE__.Any.__impl__(:target)
+        Kernel.defp any_impl_for(), do: __MODULE__.Any.__impl__(:target)
       else
-        Kernel.defp any_impl_for, do: nil
+        Kernel.defp any_impl_for(), do: nil
       end
 
       # Internal handler for Structs
@@ -469,7 +479,7 @@ defmodule Protocol do
         target = Module.concat(__MODULE__, struct)
         case impl_for?(target) do
           true  -> target.__impl__(:target)
-          false -> any_impl_for
+          false -> any_impl_for()
         end
       end
 

lib/elixir/lib/range.ex

@@ -11,7 +11,7 @@ defmodule Range do
 
   A Range is represented internally as a struct. However,
   the most common form of creating and matching on ranges
-  is via the `../2` macro, auto-imported from Kernel:
+  is via the `../2` macro, auto-imported from `Kernel`:
 
       iex> range = 1..3
       1..3

lib/elixir/lib/record.ex

@@ -1,6 +1,6 @@
 defmodule Record do
   @moduledoc """
-  Module to work with, define and import records.
+  Module to work with, define, and import records.
 
   Records are simply tuples where the first element is an atom:
 
@@ -17,15 +17,14 @@ defmodule Record do
     1. to work with short, internal data
     2. to interface with Erlang records
 
-  The macros `defrecord/3` and `defrecordp/3` can be used to create
-  records while `extract/2` can be used to extract records from Erlang
-  files.
+  The macros `defrecord/3` and `defrecordp/3` can be used to create records
+  while `extract/2` and `extract_all/1` can be used to extract records from
+  Erlang files.
 
   ## Types
 
-  Types can be defined for tuples with the `record/2` macro (only available
-  in typespecs). Like with the generated record macros it will expand to
-  a tuple.
+  Types can be defined for tuples with the `record/2` macro (only available in
+  typespecs). This macro will expand to a tuple as seen in the example below:
 
       defmodule MyModule do
         require Record
@@ -34,14 +33,34 @@ defmodule Record do
         @type user :: record(:user, name: String.t, age: integer)
         # expands to: "@type user :: {:user, String.t, integer}"
       end
+
   """
 
   @doc """
   Extracts record information from an Erlang file.
 
   Returns a quoted expression containing the fields as a list
-  of tuples. It expects the record name to be an atom and the
-  library path to be a string at expansion time.
+  of tuples.
+
+  `name`, which is the name of the extracted record, is expected to be an atom
+  *at compile time*.
+
+  ## Options
+
+  This function accepts the following options, which are exclusive to each other
+  (i.e., only one of them can be used in the same call):
+
+    * `:from` - (binary representing a path to a file) path to the Erlang file
+      that contains the record definition to extract; with this option, this
+      function uses the same path lookup used by the `-include` attribute used in
+      Erlang modules.
+    * `:from_lib` - (binary representing a path to a file) path to the Erlang
+      file that contains the record definition to extract; with this option,
+      this function uses the same path lookup used by the `-include_lib`
+      attribute used in Erlang modules.
+
+  These options are expected to be literals (including the binary values) at
+  compile time.
 
   ## Examples
 
@@ -59,18 +78,33 @@ defmodule Record do
   @doc """
   Extracts all records information from an Erlang file.
 
-  Returns a keyword list containing extracted record names as keys, and
-  lists of tuples describing the fields as values. It expects a named
-  argument :from or :from_lib, which correspond to *include* or
-  *include_lib* attribute from Erlang modules, respectively.
+  Returns a keyword list of `{record_name, fields}` tuples where `record_name`
+  is the name of an extracted record and `fields` is a list of `{field, value}`
+  tuples representing the fields for that record.
 
+  ## Options
+
+  This function accepts the following options, which are exclusive to each other
+  (i.e., only one of them can be used in the same call):
+
+    * `:from` - (binary representing a path to a file) path to the Erlang file
+      that contains the record definitions to extract; with this option, this
+      function uses the same path lookup used by the `-include` attribute used in
+      Erlang modules.
+    * `:from_lib` - (binary representing a path to a file) path to the Erlang
+      file that contains the record definitions to extract; with this option,
+      this function uses the same path lookup used by the `-include_lib`
+      attribute used in Erlang modules.
+
+  These options are expected to be literals (including the binary values) at
+  compile time.
   """
   def extract_all(opts) when is_list(opts) do
     Record.Extractor.extract_all(opts)
   end
 
   @doc """
-  Checks if the given `data` is a record of `kind`.
+  Checks if the given `data` is a record of kind `kind`.
 
   This is implemented as a macro so it can be used in guard clauses.
 
@@ -128,11 +162,27 @@ defmodule Record do
   end
 
   @doc """
-  Defines a set of macros to create and access a record.
+  Defines a set of macros to create, access, and pattern match
+  on a record.
 
-  The macros are going to have `name`, a tag (which defaults)
-  to the name if none is given, and a set of fields given by
-  `kv`.
+  The name of the generated macros will be `name` (which has to be an
+  atom). `tag` is also an atom and is used as the "tag" for the record (i.e.,
+  the first element of the record tuple); by default (if `nil`), it's the same
+  as `name`. `kv` is a keyword list of `name: default_value` fields for the
+  new record.
+
+  The following macros are generated:
+
+    * `name/0` to create a new record with default values for all fields
+    * `name/1` to create a new record with the given fields and values,
+      to get the zero-based index of the given field in a record or to
+      convert the given record to a keyword list
+    * `name/2` to update an existing record with the given fields and values
+      or to access a given field in a given record
+
+  All these macros are public macros (as defined by `defmacro`).
+
+  See the "Examples" section for examples on how to use these macros.
 
   ## Examples
 
@@ -142,7 +192,10 @@ defmodule Record do
       end
 
   In the example above, a set of macros named `user` but with different
-  arities will be defined to manipulate the underlying record:
+  arities will be defined to manipulate the underlying record.
+
+      # Import the module to make the user macros locally available
+      import User
 
       # To create records
       record = user()        #=> {:user, "meg", 25}
@@ -154,6 +207,10 @@ defmodule Record do
       # To update the record
       user(record, age: 26) #=> {:user, "meg", 26}
 
+      # To get the zero-based index of the field in record tuple
+      # (index 0 is occupied by the record "tag")
+      user(:name) #=> 1
+
       # Convert a record to a keyword list
       user(record) #=> [name: "meg", age: 26]
 
@@ -165,8 +222,8 @@ defmodule Record do
       user(name: name) = record
       name #=> "meg"
 
-  By default, Elixir uses the record name as the first element of
-  the tuple (the tag). But it can be changed to something else:
+  By default, Elixir uses the record name as the first element of the tuple (the
+  "tag"). However, a different tag can be specified when defining a record:
 
       defmodule User do
         require Record
@@ -176,12 +233,12 @@ defmodule Record do
       require User
       User.user() #=> {User, nil}
 
-  ## Defining extracted records with anonymous functions
+  ## Defining extracted records with anonymous functions in the values
 
-  If a record defines an anonymous function, an `ArgumentError`
-  will occur if you attempt to create a record with it.
-  This can occur unintentionally when defining a record after extracting
-  it from an Erlang library that uses anonymous functions for defaults.
+  If a record defines an anonymous function in the default values, an
+  `ArgumentError` will be raised. This can happen unintentionally when defining
+  a record after extracting it from an Erlang library that uses anonymous
+  functions for defaults.
 
       Record.defrecord :my_rec, Record.extract(...)
       #=> ** (ArgumentError) invalid value for record field fun_field,
@@ -195,6 +252,7 @@ defmodule Record do
         Record.defrecord :my_rec, Record.extract(...) |> Keyword.merge(fun_field: &__MODULE__.foo/2)
         def foo(bar, baz), do: IO.inspect({bar, baz})
       end
+
   """
   defmacro defrecord(name, tag \\ nil, kv) do
     quote bind_quoted: [name: name, tag: tag, kv: kv] do
@@ -258,12 +316,13 @@ defmodule Record do
       Keyword.keyword?(args) ->
         create(atom, fields, args, caller)
       true ->
+        fields = Macro.escape(fields)
         case Macro.expand(args, caller) do
           {:{}, _, [^atom | list]} when length(list) == length(fields) ->
             record = List.to_tuple([atom | list])
-            Macro.escape(Record.__keyword__(atom, fields, record))
+            Record.__keyword__(atom, fields, record)
           {^atom, arg} when length(fields) == 1 ->
-            Macro.escape(Record.__keyword__(atom, fields, {atom, arg}))
+            Record.__keyword__(atom, fields, {atom, arg})
           _ ->
             quote do: Record.__keyword__(unquote(atom), unquote(fields), unquote(args))
         end
@@ -360,17 +419,26 @@ defmodule Record do
   def __keyword__(atom, fields, record) do
     if is_record(record, atom) do
       [_tag | values] = Tuple.to_list(record)
-      join_keyword(fields, values, [])
+      case join_keyword(fields, values, []) do
+        kv when is_list(kv) ->
+          kv
+        expected_fields ->
+          msg = "expected argument to be a #{inspect atom} record with #{expected_fields} fields, got: #{inspect record}"
+          raise ArgumentError, msg
+      end
     else
       msg = "expected argument to be a literal atom, literal keyword or a #{inspect atom} record, got runtime: #{inspect record}"
       raise ArgumentError, msg
     end
   end
 
+  # Returns a keyword list, or expected number of fields on size mismatch
   defp join_keyword([{field, _default} | fields], [value | values], acc),
     do: join_keyword(fields, values, [{field, value} | acc])
   defp join_keyword([], [], acc),
     do: :lists.reverse(acc)
+  defp join_keyword(rest_fields, _rest_values, acc),
+    do: length(acc) + length(rest_fields) # expected fields
 
   defp apply_underscore(fields, keyword) do
     case Keyword.fetch(keyword, :_) do

lib/elixir/lib/regex.ex

@@ -1,14 +1,13 @@
 defmodule Regex do
   @moduledoc ~S"""
-  Provides regular expressions for Elixir. Built on top of Erlang's `:re`
-  module.
+  Provides regular expressions for Elixir.
 
-  As the `:re` module, Regex is based on PCRE
-  (Perl Compatible Regular Expressions). More information can be
-  found in the [`:re` module documentation](http://www.erlang.org/doc/man/re.html).
+  Regex is based on PCRE (Perl Compatible Regular Expressions) and
+  built on top of Erlang's `:re` module. More information can be found
+  in the [`:re` module documentation](http://www.erlang.org/doc/man/re.html).
 
-  Regular expressions in Elixir can be created using `Regex.compile!/2`
-  or using the special form with [`~r`](Kernel.html#sigil_r/2) or [`~R`](Kernel.html#sigil_R/2):
+  Regular expressions in Elixir can be created using the sigils
+  [`~r`](Kernel.html#sigil_r/2) or [`~R`](Kernel.html#sigil_R/2):
 
       # A simple regular expressions that matches foo anywhere in the string
       ~r/foo/
@@ -16,9 +15,38 @@ defmodule Regex do
       # A regular expression with case insensitive and Unicode options
       ~r/foo/iu
 
+  Regular expressions created via sigils are pre-compiled and stored
+  in the `.beam` file. Notice this may be a problem if you are precompiling
+  Elixir, see the "Precompilation" section for more information.
+
   A Regex is represented internally as the `Regex` struct. Therefore,
   `%Regex{}` can be used whenever there is a need to match on them.
 
+  Keep in mind it is not guaranteed two regular expressions from the
+  same source are equal, for example:
+
+      ~r/(?<foo>.)(?<bar>.)/ == ~r/(?<foo>.)(?<bar>.)/
+
+  may return `true` or `false` depending on your machine, endianess,
+  available optimizations and others. You can, however, retrieve the source
+  of a compiled regular expression by accessing the `source` field, and then
+  compare those directly:
+
+      ~r/(?<foo>.)(?<bar>.)/.source == ~r/(?<foo>.)(?<bar>.)/.source
+
+  ## Precompilation
+
+  Regular expressions built with sigil are precompiled and stored in `.beam`
+  files. This may be a problem if you are precompiling Elixir to run in
+  different OTP releases, as OTP releases may update the underlying regular
+  expression engine at any time.
+
+  For such reasons, we always recomend precompiling Elixir projects using
+  the OTP version meant to run in production. In case cross-compilation is
+  really necessary, you can manually invoke `Regex.recompile/1` or `Regex.
+  recompile!/1` to perform a runtime version check and recompile the regex
+  if necessary.
+
   ## Modifiers
 
   The modifiers available when creating a Regex are:
@@ -27,7 +55,7 @@ defmodule Regex do
       modifiers like `\w`, `\W`, `\s` and friends to also match on Unicode.
       It expects valid Unicode strings to be given on match
 
-    * `caseless` (i) - add case insensitivity
+    * `caseless` (i) - adds case insensitivity
 
     * `dotall` (s) - causes dot to match newlines and also set newline to
       anycrlf; the new line setting can be overridden by setting `(*CR)` or
@@ -70,7 +98,7 @@ defmodule Regex do
       explicitly captured subpatterns, but not the complete matching part of
       the string
 
-    * `:none` - do not return matching subpatterns at all
+    * `:none` - does not return matching subpatterns at all
 
     * `:all_names` - captures all names in the Regex
 
@@ -78,7 +106,7 @@ defmodule Regex do
 
   """
 
-  defstruct re_pattern: nil, source: "", opts: ""
+  defstruct re_pattern: nil, source: "", opts: "", re_version: ""
 
   @type t :: %__MODULE__{re_pattern: term, source: binary, opts: binary}
 
@@ -91,7 +119,7 @@ defmodule Regex do
 
   The given options can either be a binary with the characters
   representing the same regex options given to the `~r` sigil,
-  or a list of options, as expected by the Erlang's [`:re` module](http://www.erlang.org/doc/man/re.html).
+  or a list of options, as expected by the Erlang's `:re` module.
 
   It returns `{:ok, regex}` in case of success,
   `{:error, reason}` otherwise.
@@ -106,40 +134,84 @@ defmodule Regex do
 
   """
   @spec compile(binary, binary | [term]) :: {:ok, t} | {:error, any}
-  def compile(source, options \\ "")
+  def compile(source, options \\ "") do
+    compile(source, options, version())
+  end
 
-  def compile(source, options) when is_binary(options) do
+  defp compile(source, options, version) when is_binary(options) do
     case translate_options(options, []) do
       {:error, rest} ->
         {:error, {:invalid_option, rest}}
 
       translated_options ->
-        compile(source, translated_options, options)
+        compile(source, translated_options, options, version)
     end
   end
 
-  def compile(source, options) when is_list(options) do
-    compile(source, options, "")
+  defp compile(source, options, version) when is_list(options) do
+    compile(source, options, "", version)
   end
 
-  defp compile(source, opts, doc_opts) when is_binary(source) do
+  defp compile(source, opts, doc_opts, version) when is_binary(source) do
     case :re.compile(source, opts) do
       {:ok, re_pattern} ->
-        {:ok, %Regex{re_pattern: re_pattern, source: source, opts: doc_opts}}
+        {:ok, %Regex{re_pattern: re_pattern, re_version: version, source: source, opts: doc_opts}}
       error ->
         error
     end
   end
 
   @doc """
-  Compiles the regular expression according to the given options.
-  Fails with `Regex.CompileError` if the regex cannot be compiled.
+  Compiles the regular expression and raises `Regex.CompileError` in case of errors.
   """
   @spec compile!(binary, binary | [term]) :: t
   def compile!(source, options \\ "") do
     case compile(source, options) do
       {:ok, regex} -> regex
-      {:error, {reason, at}} -> raise Regex.CompileError, message: "#{reason} at position #{at}"
+      {:error, {reason, at}} -> raise Regex.CompileError, "#{reason} at position #{at}"
+    end
+  end
+
+  @doc """
+  Recompiles the existing regular expression if necessary.
+
+  This checks the version stored in the regular expression
+  and recompiles the regex in case of version mismatch.
+  """
+  @spec recompile(t) :: t
+  def recompile(%Regex{} = regex) do
+    version = version()
+
+    # We use Map.get/3 by choice to support old regexes versions.
+    case Map.get(regex, :re_version, :error) do
+      ^version ->
+        {:ok, regex}
+      _ ->
+        %{source: source, opts: opts} = regex
+        compile(source, opts, version)
+    end
+  end
+
+  @doc """
+  Recompiles the existing regular expression and raises `Regex.CompileError` in case of errors.
+  """
+  @spec recompile!(t) :: t
+  def recompile!(regex) do
+    case recompile(regex) do
+      {:ok, regex} -> regex
+      {:error, {reason, at}} -> raise Regex.CompileError, "#{reason} at position #{at}"
+    end
+  end
+
+  @doc """
+  Returns the version of the underlying Regex engine.
+  """
+  # TODO: No longer check for function_exported? on OTP 20+.
+  def version do
+    if function_exported?(:re, :version, 0) do
+      :re.version()
+    else
+      "8.33 2013-05-29"
     end
   end
 
@@ -184,7 +256,7 @@ defmodule Regex do
 
   ## Options
 
-    * `:return`  - set to `:index` to return indexes. Defaults to `:binary`.
+    * `:return`  - sets to `:index` to return indexes. Defaults to `:binary`.
     * `:capture` - what to capture in the result. Check the moduledoc for `Regex`
       to see the possible capture values.
 
@@ -290,7 +362,7 @@ defmodule Regex do
     names
   end
 
-  @doc """
+  @doc ~S"""
   Same as `run/3`, but scans the target several times collecting all
   matches of the regular expression.
 
@@ -299,7 +371,7 @@ defmodule Regex do
 
   ## Options
 
-    * `:return`  - set to `:index` to return indexes. Defaults to `:binary`.
+    * `:return`  - sets to `:index` to return indexes. Defaults to `:binary`.
     * `:capture` - what to capture in the result. Check the moduledoc for `Regex`
       to see the possible capture values.
 
@@ -314,6 +386,9 @@ defmodule Regex do
       iex> Regex.scan(~r/e/, "abcd")
       []
 
+      iex> Regex.scan(~r/\p{Sc}/u, "$, £, and €")
+      [["$"], ["£"], ["€"]]
+
   """
   @spec scan(t, String.t, [term]) :: [[String.t]]
   def scan(regex, string, options \\ [])
@@ -352,28 +427,28 @@ defmodule Regex do
 
   ## Examples
 
-      iex> Regex.split(~r/-/, "a-b-c")
+      iex> Regex.split(~r{-}, "a-b-c")
       ["a", "b", "c"]
 
-      iex> Regex.split(~r/-/, "a-b-c", [parts: 2])
+      iex> Regex.split(~r{-}, "a-b-c", [parts: 2])
       ["a", "b-c"]
 
-      iex> Regex.split(~r/-/, "abc")
+      iex> Regex.split(~r{-}, "abc")
       ["abc"]
 
-      iex> Regex.split(~r//, "abc")
+      iex> Regex.split(~r{}, "abc")
       ["a", "b", "c", ""]
 
-      iex> Regex.split(~r/a(?<second>b)c/, "abc")
+      iex> Regex.split(~r{a(?<second>b)c}, "abc")
       ["", ""]
 
-      iex> Regex.split(~r/a(?<second>b)c/, "abc", on: [:second])
+      iex> Regex.split(~r{a(?<second>b)c}, "abc", on: [:second])
       ["a", "c"]
 
-      iex> Regex.split(~r/(x)/, "Elixir", include_captures: true)
+      iex> Regex.split(~r{(x)}, "Elixir", include_captures: true)
       ["Eli", "x", "ir"]
 
-      iex> Regex.split(~r/a(?<second>b)c/, "abc", on: [:second], include_captures: true)
+      iex> Regex.split(~r{a(?<second>b)c}, "abc", on: [:second], include_captures: true)
       ["a", "b", "c"]
 
   """
@@ -456,8 +531,10 @@ defmodule Regex do
 
   The replacement can be either a string or a function. The string
   is used as a replacement for every match and it allows specific
-  captures to be accessed via `\\N` or `\g{N}`, where `N` is the
-  capture. In case `\\0` is used, the whole match is inserted.
+  captures to be accessed via `\N` or `\g{N}`, where `N` is the
+  capture. In case `\0` is used, the whole match is inserted. Note
+  that in regexes the backslash needs to be escaped, hence in practice
+  you'll need to use `\\N` and `\\g{N}`.
 
   When the replacement is a function, the function may have arity
   N where each argument maps to a capture, with the first argument
@@ -624,9 +701,6 @@ defmodule Regex do
     [get_index(string, h) | get_indexes(string, t, arity - 1)]
   end
 
-  {:ok, pattern} = :re.compile(~S"[.^$*+?()\[\]{}\\\|\s#]", [:unicode])
-  @escape_pattern pattern
-
   @doc ~S"""
   Escapes a string to be literally matched in a regex.
 
@@ -641,9 +715,15 @@ defmodule Regex do
   """
   @spec escape(String.t) :: String.t
   def escape(string) when is_binary(string) do
-    :re.replace(string, @escape_pattern, "\\\\&", [:global, {:return, :binary}])
+    escape(string, [])
   end
 
+  for char <- '.^$*+?()[]{}|#-\\\t\n\v\f\r\s' do
+    defp escape(<<unquote(char), rest::binary>>, acc), do: escape(rest, [acc, ?\\, unquote(char)])
+  end
+  defp escape(<<char, rest::binary>>, acc), do: escape(rest, [acc, char])
+  defp escape(<<>>, acc), do: IO.iodata_to_binary(acc)
+
   # Helpers
 
   @doc false

lib/elixir/lib/registry.ex

@@ -0,0 +1,927 @@
+defmodule Registry do
+  @moduledoc ~S"""
+  A local, decentralized and scalable key-value process storage.
+
+  It allows developers to lookup one or more processes with a given key.
+  If the registry has `:unique` keys, a key points to 0 or 1 processes.
+  If the registry allows `:duplicate` keys, a single key may point to any
+  number of processes. In both cases, different keys could identify the
+  same process.
+
+  Each entry in the registry is associated to the process that has
+  registered the key. If the process crashes, the keys associated to that
+  process are automatically removed. All key comparisons in the registry
+  are done using the match operation (`===`).
+
+  The registry can be used for different purposes, such as name lookups (using
+  the `:via` option), storing properties, custom dispatching rules, or a pubsub
+  implementation. We explore some of those use cases below.
+
+  The registry may also be transparently partitioned, which provides
+  more scalable behaviour for running registries on highly concurrent
+  environments with thousands or millions of entries.
+
+  ## Using in `:via`
+
+  Once the registry is started with a given name (using
+  `Registry.start_link/2`), it can be used to register and access named
+  processes using the `{:via, Registry, {registry, key}}` tuple:
+
+      {:ok, _} = Registry.start_link(:unique, Registry.ViaTest)
+      name = {:via, Registry, {Registry.ViaTest, "agent"}}
+      {:ok, _} = Agent.start_link(fn -> 0 end, name: name)
+      Agent.get(name, & &1)
+      #=> 0
+      Agent.update(name, & &1 + 1)
+      Agent.get(name, & &1)
+      #=> 1
+
+  Typically the registry is started as part of a supervision tree though:
+
+      supervisor(Registry, [:unique, Registry.ViaTest])
+
+  Only registries with unique keys can be used in `:via`. If the name is
+  already taken, the case-specific `start_link` function (`Agent.start_link/2`
+  in the example above) will return `{:error, {:already_started, current_pid}}`.
+
+  ## Using as a dispatcher
+
+  `Registry` has a dispatch mechanism that allows developers to implement custom
+  dispatch logic triggered from the caller. For example, let's say we have a
+  duplicate registry started as so:
+
+      {:ok, _} = Registry.start_link(:duplicate, Registry.DispatcherTest)
+
+  By calling `register/3`, different processes can register under a given key
+  and associate any value under that key. In this case, let's register the
+  current process under the key `"hello"` and attach the `{IO, :inspect}` tuple
+  to it:
+
+      {:ok, _} = Registry.register(Registry.DispatcherTest, "hello", {IO, :inspect})
+
+  Now, an entity interested in dispatching events for a given key may call
+  `dispatch/3` passing in the key and a callback. This callback will be invoked
+  with a list of all the values registered under the requested key, alongside
+  the pid of the process that registered each value, in the form of `{pid,
+  value}` tuples. In our example, `value` will be the `{module, function}` tuple
+  in the code above:
+
+      Registry.dispatch(Registry.DispatcherTest, "hello", fn entries ->
+        for {pid, {module, function}} <- entries, do: apply(module, function, [pid])
+      end)
+      # Prints #PID<...> where the pid is for the process that called register/3 above
+      #=> :ok
+
+  Dispatching happens in the process that calls `dispatch/3` either serially or
+  concurrently in case of multiple partitions (via spawned tasks). The
+  registered processes are not involved in dispatching unless involving them is
+  done explicitly (for example, by sending them a message in the callback).
+
+  Furthermore, if there is a failure when dispatching, due to a bad
+  registration, dispatching will always fail and the registered process will not
+  be notified. Therefore let's make sure we at least wrap and report those
+  errors:
+
+      require Logger
+      Registry.dispatch(Registry.DispatcherTest, "hello", fn entries ->
+        for {pid, {module, function}} <- entries do
+          try do
+            apply(module, function, [pid])
+          catch
+            kind, reason ->
+              formatted = Exception.format(kind, reason, System.stacktrace)
+              Logger.error "Registry.dispatch/3 failed with #{formatted}"
+          end
+        end
+      end)
+      # Prints #PID<...>
+      #=> :ok
+
+  You could also replace the whole `apply` system by explicitly sending
+  messages. That's the example we will see next.
+
+  ## Using as a PubSub
+
+  Registries can also be used to implement a local, non-distributed, scalable
+  PubSub by relying on the `dispatch/3` function, similarly to the previous
+  section: in this case, however, we will send messages to each associated
+  process, instead of invoking a given module-function.
+
+  In this example, we will also set the number of partitions to the number of
+  schedulers online, which will make the registry more performant on highly
+  concurrent environments as each partition will spawn a new process, allowing
+  dispatching to happen in parallel:
+
+      {:ok, _} = Registry.start_link(:duplicate, Registry.PubSubTest,
+                                     partitions: System.schedulers_online)
+      {:ok, _} = Registry.register(Registry.PubSubTest, "hello", [])
+      Registry.dispatch(Registry.PubSubTest, "hello", fn entries ->
+        for {pid, _} <- entries, do: send(pid, {:broadcast, "world"})
+      end)
+      #=> :ok
+
+  The example above broadcasted the message `{:broadcast, "world"}` to all
+  processes registered under the "topic" (or "key" as we called it until now)
+  `"hello"`.
+
+  The third argument given to `register/3` is a value associated to the
+  current process. While in the previous section we used it when dispatching,
+  in this particular example we are not interested in it, so we have set it
+  to an empty list. You could store a more meaningful value if necessary.
+
+  ## Registrations
+
+  Looking up, dispatching and registering are efficient and immediate at
+  the cost of delayed unsubscription. For example, if a process crashes,
+  its keys are automatically removed from the registry but the change may
+  not propagate immediately. This means certain operations may return processes
+  that are already dead. When such may happen, it will be explicitly stated
+  in the function documentation.
+
+  However, keep in mind those cases are typically not an issue. After all, a
+  process referenced by a pid may crash at any time, including between getting
+  the value from the registry and sending it a message. Many parts of the standard
+  library are designed to cope with that, such as `Process.monitor/1` which will
+  deliver the `:DOWN` message immediately if the monitored process is already dead
+  and `Kernel.send/2` which acts as a no-op for dead processes.
+
+  ## ETS
+
+  Note that the registry uses one ETS table plus two ETS tables per partition.
+  """
+
+  # TODO: Decide if it should be started as part of Elixir's supervision tree.
+
+  @kind [:unique, :duplicate]
+  @all_info -1
+  @key_info -2
+
+  @typedoc "The registry identifier"
+  @type registry :: atom
+
+  @typedoc "The type of the registry"
+  @type kind :: :unique | :duplicate
+
+  @typedoc "The type of keys allowed on registration"
+  @type key :: term
+
+  @typedoc "The type of values allowed on registration"
+  @type value :: term
+
+  @typedoc "The type of registry metadata keys"
+  @type meta_key :: atom | tuple
+
+  @typedoc "The type of registry metadata values"
+  @type meta_value :: term
+
+  ## Via callbacks
+
+  @doc false
+  def whereis_name({registry, key}) do
+    case key_info!(registry) do
+      {:unique, partitions, key_ets} ->
+        key_ets = key_ets || key_ets!(registry, key, partitions)
+        case safe_lookup_second(key_ets, key) do
+          {pid, _} ->
+            if Process.alive?(pid), do: pid, else: :undefined
+          _ ->
+            :undefined
+        end
+      {kind, _, _} ->
+        raise ArgumentError, ":via is not supported for #{kind} registries"
+    end
+  end
+
+  @doc false
+  def register_name({registry, key}, pid) when pid == self() do
+    case register(registry, key, nil) do
+      {:ok, _} -> :yes
+      {:error, _} -> :no
+    end
+  end
+
+  @doc false
+  def send({registry, key}, msg) do
+    case lookup(registry, key) do
+      [{pid, _}] -> Kernel.send(pid, msg)
+      [] -> :erlang.error(:badarg, [{registry, key}, msg])
+    end
+  end
+
+  @doc false
+  def unregister_name({registry, key}) do
+    unregister(registry, key)
+  end
+
+  ## Registry API
+
+  @doc """
+  Starts the registry as a supervisor process.
+
+  Manually it can be started as:
+
+      Registry.start_link(:unique, MyApp.Registry)
+
+  In your supervisor tree, you would write:
+
+      supervisor(Registry, [:unique, MyApp.Registry])
+
+  For intensive workloads, the registry may also be partitioned (by specifying
+  the `:partitions` option). If partitioning is required then a good default is to
+  set the number of partitions to the number of schedulers available:
+
+      Registry.start_link(:unique, MyApp.Registry, partitions: System.schedulers_online())
+
+  or:
+
+      supervisor(Registry, [:unique, MyApp.Registry, [partitions: System.schedulers_online()]])
+
+  ## Options
+
+  The registry supports the following options:
+
+    * `:partitions` - the number of partitions in the registry. Defaults to `1`.
+    * `:listeners` - a list of named processes which are notified of `:register`
+      and `:unregister` events. The registered process must be monitored by the
+      listener if the listener wants to be notified if the registered process
+      crashes.
+    * `:meta` - a keyword list of metadata to be attached to the registry.
+
+  """
+  @spec start_link(kind, registry, options) :: {:ok, pid} | {:error, term}
+        when options: [partitions: pos_integer, listeners: [atom], meta: [{meta_key, meta_value}]]
+  def start_link(kind, registry, options \\ []) when kind in @kind and is_atom(registry) do
+    meta = Keyword.get(options, :meta, [])
+    unless Keyword.keyword?(meta) do
+      raise ArgumentError, "expected :meta to be a keyword list, got: #{inspect meta}"
+    end
+
+    partitions = Keyword.get(options, :partitions, 1)
+    unless is_integer(partitions) and partitions >= 1 do
+      raise ArgumentError, "expected :partitions to be a positive integer, got: #{inspect partitions}"
+    end
+
+    listeners = Keyword.get(options, :listeners, [])
+    unless is_list(listeners) and Enum.all?(listeners, &is_atom/1) do
+      raise ArgumentError, "expected :listeners to be a list of named processes, got: #{inspect listeners}"
+    end
+
+    # The @info format must be kept in sync with Registry.Partition optimization.
+    entries = [{@all_info, {kind, partitions, nil, nil, listeners}},
+               {@key_info, {kind, partitions, nil}} | meta]
+    Registry.Supervisor.start_link(kind, registry, partitions, listeners, entries)
+  end
+
+  @doc """
+  Updates the value for `key` for the current process in the unique `registry`.
+
+  Returns a `{new_value, old_value}` tuple or `:error` if there
+  is no such key assigned to the current process.
+
+  If a non-unique registry is given, an error is raised.
+
+  ## Examples
+
+      iex> Registry.start_link(:unique, Registry.UpdateTest)
+      iex> {:ok, _} = Registry.register(Registry.UpdateTest, "hello", 1)
+      iex> Registry.lookup(Registry.UpdateTest, "hello")
+      [{self(), 1}]
+      iex> Registry.update_value(Registry.UpdateTest, "hello", & &1 + 1)
+      {2, 1}
+      iex> Registry.lookup(Registry.UpdateTest, "hello")
+      [{self(), 2}]
+
+  """
+  @spec update_value(registry, key, (value -> value)) :: {new_value :: term, old_value :: term} | :error
+  def update_value(registry, key, callback) when is_atom(registry) and is_function(callback, 1) do
+    case key_info!(registry) do
+      {:unique, partitions, key_ets} ->
+        key_ets = key_ets || key_ets!(registry, key, partitions)
+        try do
+          :ets.lookup_element(key_ets, key, 2)
+        catch
+          :error, :badarg -> :error
+        else
+          {pid, old_value} when pid == self() ->
+            new_value = callback.(old_value)
+            :ets.insert(key_ets, {key, {pid, new_value}})
+            {new_value, old_value}
+          {_, _} ->
+            :error
+        end
+      {kind, _, _} ->
+        raise ArgumentError, "Registry.update_value/3 is not supported for #{kind} registries"
+    end
+  end
+
+  @doc """
+  Invokes the callback with all entries under `key` in each partition
+  for the given `registry`.
+
+  The list of `entries` is a non-empty list of two-element tuples where
+  the first element is the pid and the second element is the value
+  associated to the pid. If there are no entries for the given key,
+  the callback is never invoked.
+
+  If the registry is not partitioned, the callback is invoked in the process
+  that calls `dispatch/3`. If the registry is partitioned, the callback is
+  invoked concurrently per partition by starting a task linked to the
+  caller. The callback, however, is only invoked if there are entries for that
+  partition.
+
+  See the module documentation for examples of using the `dispatch/3`
+  function for building custom dispatching or a pubsub system.
+  """
+  @spec dispatch(registry, key, (entries :: [{pid, value}] -> term)) :: :ok
+  def dispatch(registry, key, mfa_or_fun)
+      when is_atom(registry) and is_function(mfa_or_fun, 1)
+      when is_atom(registry) and tuple_size(mfa_or_fun) == 3 do
+    case key_info!(registry) do
+      {:unique, partitions, key_ets} ->
+        (key_ets || key_ets!(registry, key, partitions))
+        |> safe_lookup_second(key)
+        |> List.wrap()
+        |> apply_non_empty_to_mfa_or_fun(mfa_or_fun)
+      {:duplicate, 1, key_ets} ->
+        key_ets
+        |> safe_lookup_second(key)
+        |> apply_non_empty_to_mfa_or_fun(mfa_or_fun)
+      {:duplicate, partitions, _} ->
+        registry
+        |> dispatch_task(key, mfa_or_fun, partitions)
+        |> Enum.each(&Task.await(&1, :infinity))
+    end
+    :ok
+  end
+
+  defp dispatch_task(_registry, _key, _mfa_or_fun, 0) do
+    []
+  end
+  defp dispatch_task(registry, key, mfa_or_fun, partition) do
+    partition = partition - 1
+    parent = self()
+    task = Task.async(fn ->
+      registry
+      |> key_ets!(partition)
+      |> safe_lookup_second(key)
+      |> apply_non_empty_to_mfa_or_fun(mfa_or_fun)
+      Process.unlink(parent)
+      :ok
+    end)
+    [task | dispatch_task(registry, key, mfa_or_fun, partition)]
+  end
+
+  defp apply_non_empty_to_mfa_or_fun([], _mfa_or_fun) do
+    :ok
+  end
+  defp apply_non_empty_to_mfa_or_fun(entries, {module, function, args}) do
+    apply(module, function, [entries | args])
+  end
+  defp apply_non_empty_to_mfa_or_fun(entries, fun) do
+    fun.(entries)
+  end
+
+  @doc """
+  Finds the `{pid, value}` pair for the given `key` in `registry` in no particular order.
+
+  An empty list if there is no match.
+
+  For unique registries, a single partition lookup is necessary. For
+  duplicate registries, all partitions must be looked up.
+
+  ## Examples
+
+  In the example below we register the current process and look it up
+  both from itself and other processes:
+
+      iex> Registry.start_link(:unique, Registry.UniqueLookupTest)
+      iex> Registry.lookup(Registry.UniqueLookupTest, "hello")
+      []
+      iex> {:ok, _} = Registry.register(Registry.UniqueLookupTest, "hello", :world)
+      iex> Registry.lookup(Registry.UniqueLookupTest, "hello")
+      [{self(), :world}]
+      iex> Task.async(fn -> Registry.lookup(Registry.UniqueLookupTest, "hello") end) |> Task.await
+      [{self(), :world}]
+
+  The same applies to duplicate registries:
+
+      iex> Registry.start_link(:duplicate, Registry.DuplicateLookupTest)
+      iex> Registry.lookup(Registry.DuplicateLookupTest, "hello")
+      []
+      iex> {:ok, _} = Registry.register(Registry.DuplicateLookupTest, "hello", :world)
+      iex> Registry.lookup(Registry.DuplicateLookupTest, "hello")
+      [{self(), :world}]
+      iex> {:ok, _} = Registry.register(Registry.DuplicateLookupTest, "hello", :another)
+      iex> Enum.sort(Registry.lookup(Registry.DuplicateLookupTest, "hello"))
+      [{self(), :another}, {self(), :world}]
+
+  """
+  @spec lookup(registry, key) :: [{pid, value}]
+  def lookup(registry, key) when is_atom(registry) do
+    case key_info!(registry) do
+      {:unique, partitions, key_ets} ->
+        key_ets = key_ets || key_ets!(registry, key, partitions)
+        case safe_lookup_second(key_ets, key) do
+          {_, _} = pair ->
+            [pair]
+          _ ->
+            []
+        end
+
+      {:duplicate, 1, key_ets} ->
+        safe_lookup_second(key_ets, key)
+
+      {:duplicate, partitions, _key_ets} ->
+        for partition <- 0..(partitions - 1),
+            pair <- safe_lookup_second(key_ets!(registry, partition), key),
+            do: pair
+    end
+  end
+
+  @doc """
+  Returns `{pid, value}` pairs under the given `key` in `registry` that match `pattern`.
+
+  Pattern must be an atom or a tuple that will match the structure of the
+  value stored in the registry. The atom `:_` can be used to ignore a given
+  value or tuple element, while :"$1" can be used to temporarily assign part
+  of pattern to a variable for a subsequent comparison.
+
+  An empty list will be returned if there is no match.
+
+  For unique registries, a single partition lookup is necessary. For
+  duplicate registries, all partitions must be looked up.
+
+  ## Examples
+
+  In the example below we register the current process under the same
+  key in a duplicate registry but with different values:
+
+      iex> Registry.start_link(:duplicate, Registry.MatchTest)
+      iex> {:ok, _} = Registry.register(Registry.MatchTest, "hello", {1, :atom, 1})
+      iex> {:ok, _} = Registry.register(Registry.MatchTest, "hello", {2, :atom, 2})
+      iex> Registry.match(Registry.MatchTest, "hello", {1, :_, :_})
+      [{self(), {1, :atom, 1}}]
+      iex> Registry.match(Registry.MatchTest, "hello", {2, :_, :_})
+      [{self(), {2, :atom, 2}}]
+      iex> Registry.match(Registry.MatchTest, "hello", {:_, :atom, :_}) |> Enum.sort()
+      [{self(), {1, :atom, 1}}, {self(), {2, :atom, 2}}]
+      iex> Registry.match(Registry.MatchTest, "hello", {:"$1", :_, :"$1"}) |> Enum.sort()
+      [{self(), {1, :atom, 1}}, {self(), {2, :atom, 2}}]
+
+  """
+  @spec match(registry, key, match_pattern :: atom() | tuple()) :: [{pid, term}]
+  def match(registry, key, pattern) when is_atom(registry) do
+    spec = [{{key, {:_, pattern}}, [], [{:element, 2, :"$_"}]}]
+
+    case key_info!(registry) do
+      {:unique, partitions, key_ets} ->
+        key_ets = key_ets || key_ets!(registry, key, partitions)
+        :ets.select(key_ets, spec)
+
+      {:duplicate, 1, key_ets} ->
+        :ets.select(key_ets, spec)
+
+      {:duplicate, partitions, _key_ets} ->
+        for partition <- 0..(partitions - 1),
+            pair <- :ets.select(key_ets!(registry, partition), spec),
+            do: pair
+    end
+  end
+
+  @doc """
+  Returns the known keys for the given `pid` in `registry` in no particular order.
+
+  If the registry is unique, the keys are unique. Otherwise
+  they may contain duplicates if the process was registered
+  under the same key multiple times. The list will be empty
+  if the process is dead or it has no keys in this registry.
+
+  ## Examples
+
+  Registering under a unique registry does not allow multiple entries:
+
+      iex> Registry.start_link(:unique, Registry.UniqueKeysTest)
+      iex> Registry.keys(Registry.UniqueKeysTest, self())
+      []
+      iex> {:ok, _} = Registry.register(Registry.UniqueKeysTest, "hello", :world)
+      iex> Registry.register(Registry.UniqueKeysTest, "hello", :later) # registry is :unique
+      {:error, {:already_registered, self()}}
+      iex> Registry.keys(Registry.UniqueKeysTest, self())
+      ["hello"]
+
+  Such is possible for duplicate registries though:
+
+      iex> Registry.start_link(:duplicate, Registry.DuplicateKeysTest)
+      iex> Registry.keys(Registry.DuplicateKeysTest, self())
+      []
+      iex> {:ok, _} = Registry.register(Registry.DuplicateKeysTest, "hello", :world)
+      iex> {:ok, _} = Registry.register(Registry.DuplicateKeysTest, "hello", :world)
+      iex> Registry.keys(Registry.DuplicateKeysTest, self())
+      ["hello", "hello"]
+
+  """
+  @spec keys(registry, pid) :: [key]
+  def keys(registry, pid) when is_atom(registry) and is_pid(pid) do
+    {kind, partitions, _, pid_ets, _} = info!(registry)
+    {_, pid_ets} = pid_ets || pid_ets!(registry, pid, partitions)
+    keys = safe_lookup_second(pid_ets, pid)
+
+    cond do
+      kind == :unique -> Enum.uniq(keys)
+      true -> keys
+    end
+  end
+
+  @doc """
+  Unregisters all entries for the given `key` associated to the current
+  process in `registry`.
+
+  Always returns `:ok` and automatically unlinks the current process from
+  the owner if there are no more keys associated to the current process. See
+  also `register/3` to read more about the "owner".
+
+  ## Examples
+
+  It unregister all entries for `key` for unique registries:
+
+      iex> Registry.start_link(:unique, Registry.UniqueUnregisterTest)
+      iex> Registry.register(Registry.UniqueUnregisterTest, "hello", :world)
+      iex> Registry.keys(Registry.UniqueUnregisterTest, self())
+      ["hello"]
+      iex> Registry.unregister(Registry.UniqueUnregisterTest, "hello")
+      :ok
+      iex> Registry.keys(Registry.UniqueUnregisterTest, self())
+      []
+
+  As well as duplicate registries:
+
+      iex> Registry.start_link(:duplicate, Registry.DuplicateUnregisterTest)
+      iex> Registry.register(Registry.DuplicateUnregisterTest, "hello", :world)
+      iex> Registry.register(Registry.DuplicateUnregisterTest, "hello", :world)
+      iex> Registry.keys(Registry.DuplicateUnregisterTest, self())
+      ["hello", "hello"]
+      iex> Registry.unregister(Registry.DuplicateUnregisterTest, "hello")
+      :ok
+      iex> Registry.keys(Registry.DuplicateUnregisterTest, self())
+      []
+
+  """
+  @spec unregister(registry, key) :: :ok
+  def unregister(registry, key) when is_atom(registry) do
+    self = self()
+    {kind, partitions, key_ets, pid_ets, listeners} = info!(registry)
+    {key_partition, pid_partition} = partitions(kind, key, self, partitions)
+    key_ets = key_ets || key_ets!(registry, key_partition)
+    {pid_server, pid_ets} = pid_ets || pid_ets!(registry, pid_partition)
+
+    # Remove first from the key_ets because in case of crashes
+    # the pid_ets will still be able to clean up. The last step is
+    # to clean if we have no more entries.
+    true = :ets.match_delete(key_ets, {key, {self, :_}})
+    true = :ets.delete_object(pid_ets, {self, key, key_ets})
+
+    unlink_if_unregistered(pid_server, pid_ets, self)
+
+    for listener <- listeners do
+      Kernel.send(listener, {:unregister, registry, key, self})
+    end
+
+    :ok
+  end
+
+  @doc """
+  Registers the current process under the given `key` in `registry`.
+
+  A value to be associated with this registration must also be given.
+  This value will be retrieved whenever dispatching or doing a key
+  lookup.
+
+  This function returns `{:ok, owner}` or `{:error, reason}`.
+  The `owner` is the pid is the registry partition responsible for
+  the pid. The owner is automatically linked to the caller.
+
+  If the registry has unique keys, it will return `{:ok, owner}` unless
+  the key is already associated to a pid, in which case it returns
+  `{:error, {:already_registered, pid}}`.
+
+  If the registry has duplicate keys, multiple registrations from the
+  current process under the same key are allowed.
+
+  ## Examples
+
+  Registering under a unique registry does not allow multiple entries:
+
+      iex> Registry.start_link(:unique, Registry.UniqueRegisterTest)
+      iex> {:ok, _} = Registry.register(Registry.UniqueRegisterTest, "hello", :world)
+      iex> Registry.register(Registry.UniqueRegisterTest, "hello", :later)
+      {:error, {:already_registered, self()}}
+      iex> Registry.keys(Registry.UniqueRegisterTest, self())
+      ["hello"]
+
+  Such is possible for duplicate registries though:
+
+      iex> Registry.start_link(:duplicate, Registry.DuplicateRegisterTest)
+      iex> {:ok, _} = Registry.register(Registry.DuplicateRegisterTest, "hello", :world)
+      iex> {:ok, _} = Registry.register(Registry.DuplicateRegisterTest, "hello", :world)
+      iex> Registry.keys(Registry.DuplicateRegisterTest, self())
+      ["hello", "hello"]
+
+  """
+  @spec register(registry, key, value) :: {:ok, pid} | {:error, {:already_registered, pid}}
+  def register(registry, key, value) when is_atom(registry) do
+    self = self()
+    {kind, partitions, key_ets, pid_ets, listeners} = info!(registry)
+    {key_partition, pid_partition} = partitions(kind, key, self, partitions)
+    key_ets = key_ets || key_ets!(registry, key_partition)
+    {pid_server, pid_ets} = pid_ets || pid_ets!(registry, pid_partition)
+
+    # Notice we write first to the pid ets table because it will
+    # always be able to do the clean up. If we register first to the
+    # key one and the process crashes, the key will stay there forever.
+    Process.link(pid_server)
+    true = :ets.insert(pid_ets, {self, key, key_ets})
+    case register_key(kind, pid_server, key_ets, key, {key, {self, value}}) do
+      {:ok, _} = ok ->
+        for listener <- listeners do
+          Kernel.send(listener, {:register, registry, key, self, value})
+        end
+        ok
+      {:error, {:already_registered, ^self}} = error ->
+        error
+      {:error, _} = error ->
+        true = :ets.delete_object(pid_ets, {self, key, key_ets})
+        unlink_if_unregistered(pid_server, pid_ets, self)
+        error
+    end
+  end
+
+  defp register_key(:duplicate, pid_server, key_ets, _key, entry) do
+    true = :ets.insert(key_ets, entry)
+    {:ok, pid_server}
+  end
+  defp register_key(:unique, pid_server, key_ets, key, entry) do
+    if :ets.insert_new(key_ets, entry) do
+      {:ok, pid_server}
+    else
+      # Notice we have to call register_key recursively
+      # because we are always at odds of a race condition.
+      case :ets.lookup(key_ets, key) do
+        [{^key, {pid, _}} = current] ->
+          if Process.alive?(pid) do
+            {:error, {:already_registered, pid}}
+          else
+            :ets.delete_object(key_ets, current)
+            register_key(:unique, pid_server, key_ets, key, entry)
+          end
+        [] ->
+          register_key(:unique, pid_server, key_ets, key, entry)
+      end
+    end
+  end
+
+  @doc """
+  Reads registry metadata given on `start_link/3`.
+
+  Atoms and tuples are allowed as keys.
+
+  ## Examples
+
+      iex> Registry.start_link(:unique, Registry.MetaTest, meta: [custom_key: "custom_value"])
+      iex> Registry.meta(Registry.MetaTest, :custom_key)
+      {:ok, "custom_value"}
+      iex> Registry.meta(Registry.MetaTest, :unknown_key)
+      :error
+
+  """
+  @spec meta(registry, meta_key) :: {:ok, meta_value} | :error
+  def meta(registry, key) when is_atom(registry) and (is_atom(key) or is_tuple(key)) do
+    try do
+      :ets.lookup(registry, key)
+    catch
+      :error, :badarg ->
+        raise ArgumentError, "unknown registry: #{inspect registry}"
+    else
+      [{^key, value}] -> {:ok, value}
+      _ -> :error
+    end
+  end
+
+  @doc """
+  Stores registry metadata.
+
+  Atoms and tuples are allowed as keys.
+
+  ## Examples
+
+      iex> Registry.start_link(:unique, Registry.PutMetaTest)
+      iex> Registry.put_meta(Registry.PutMetaTest, :custom_key, "custom_value")
+      :ok
+      iex> Registry.meta(Registry.PutMetaTest, :custom_key)
+      {:ok, "custom_value"}
+      iex> Registry.put_meta(Registry.PutMetaTest, {:tuple, :key}, "tuple_value")
+      :ok
+      iex> Registry.meta(Registry.PutMetaTest, {:tuple, :key})
+      {:ok, "tuple_value"}
+
+  """
+  @spec put_meta(registry, meta_key, meta_value) :: :ok
+  def put_meta(registry, key, value) when is_atom(registry) and (is_atom(key) or is_tuple(key)) do
+    try do
+      :ets.insert(registry, {key, value})
+      :ok
+    catch
+      :error, :badarg ->
+        raise ArgumentError, "unknown registry: #{inspect registry}"
+    end
+  end
+
+  ## Helpers
+
+  @compile {:inline, hash: 2}
+
+  defp hash(term, limit) do
+    :erlang.phash2(term, limit)
+  end
+
+  defp info!(registry) do
+    try do
+      :ets.lookup_element(registry, @all_info, 2)
+    catch
+      :error, :badarg ->
+        raise ArgumentError, "unknown registry: #{inspect registry}"
+    end
+  end
+
+  defp key_info!(registry) do
+    try do
+      :ets.lookup_element(registry, @key_info, 2)
+    catch
+      :error, :badarg ->
+        raise ArgumentError, "unknown registry: #{inspect registry}"
+    end
+  end
+
+  defp key_ets!(registry, key, partitions) do
+    :ets.lookup_element(registry, hash(key, partitions), 2)
+  end
+
+  defp key_ets!(registry, partition) do
+    :ets.lookup_element(registry, partition, 2)
+  end
+
+  defp pid_ets!(registry, key, partitions) do
+    :ets.lookup_element(registry, hash(key, partitions), 3)
+  end
+
+  defp pid_ets!(registry, partition) do
+    :ets.lookup_element(registry, partition, 3)
+  end
+
+  defp safe_lookup_second(ets, key) do
+    try do
+      :ets.lookup_element(ets, key, 2)
+    catch
+      :error, :badarg -> []
+    end
+  end
+
+  defp partitions(:unique, key, pid, partitions) do
+    {hash(key, partitions), hash(pid, partitions)}
+  end
+  defp partitions(:duplicate, _key, pid, partitions) do
+    partition = hash(pid, partitions)
+    {partition, partition}
+  end
+
+  defp unlink_if_unregistered(pid_server, pid_ets, self) do
+    unless :ets.member(pid_ets, self) do
+      Process.unlink(pid_server)
+    end
+  end
+end
+
+defmodule Registry.Supervisor do
+  @moduledoc false
+  use Supervisor
+
+  def start_link(kind, registry, partitions, listeners, entries) do
+    Supervisor.start_link(__MODULE__, {kind, registry, partitions, listeners, entries}, name: registry)
+  end
+
+  def init({kind, registry, partitions, listeners, entries}) do
+    ^registry = :ets.new(registry, [:set, :public, :named_table, read_concurrency: true])
+    true = :ets.insert(registry, entries)
+
+    children =
+      for i <- 0..partitions-1 do
+        key_partition = Registry.Partition.key_name(registry, i)
+        pid_partition = Registry.Partition.pid_name(registry, i)
+        arg = {kind, registry, i, partitions, key_partition, pid_partition, listeners}
+        worker(Registry.Partition, [pid_partition, arg], id: pid_partition)
+      end
+
+    supervise(children, strategy: strategy_for_kind(kind))
+  end
+
+  # Unique registries have their key partition hashed by key.
+  # This means that, if a pid partition crashes, it may have
+  # entries from all key partitions, so we need to crash all.
+  defp strategy_for_kind(:unique), do: :one_for_all
+
+  # Duplicate registries have both key and pid partitions hashed
+  # by pid. This means that, if a pid partition crashes, all of
+  # its associated entries are in its sibling table, so we crash one.
+  defp strategy_for_kind(:duplicate), do: :one_for_one
+end
+
+defmodule Registry.Partition do
+  @moduledoc false
+
+  # This process owns the equivalent key and pid ets tables
+  # and is responsible for monitoring processes that map to
+  # the its own pid table.
+  use GenServer
+  @all_info -1
+  @key_info -2
+
+  @doc """
+  Returns the name of key partition table.
+  """
+  @spec key_name(atom, non_neg_integer) :: atom
+  def key_name(registry, partition) do
+    Module.concat(registry, "KeyPartition" <> Integer.to_string(partition))
+  end
+
+  @doc """
+  Returns the name of pid partition table.
+  """
+  @spec pid_name(atom, non_neg_integer) :: atom
+  def pid_name(name, partition) do
+    Module.concat(name, "PIDPartition" <> Integer.to_string(partition))
+  end
+
+  @doc """
+  Starts the registry partition.
+
+  The process is only responsible for monitoring, demonitoring
+  and cleaning up when monitored processes crash.
+  """
+  def start_link(registry, arg) do
+    GenServer.start_link(__MODULE__, arg, name: registry)
+  end
+
+  ## Callbacks
+
+  def init({kind, registry, i, partitions, key_partition, pid_partition, listeners}) do
+    Process.flag(:trap_exit, true)
+    key_ets = init_key_ets(kind, key_partition)
+    pid_ets = init_pid_ets(kind, pid_partition)
+
+    # If we have only one partition, we do an optimization which
+    # is to write the table information alongside the registry info.
+    if partitions == 1 do
+      entries =
+        [{@key_info, {kind, partitions, key_ets}},
+         {@all_info, {kind, partitions, key_ets, {self(), pid_ets}, listeners}}]
+      true = :ets.insert(registry, entries)
+    else
+      true = :ets.insert(registry, {i, key_ets, {self(), pid_ets}})
+    end
+
+    {:ok, pid_ets}
+  end
+
+  # The key partition is a set for unique keys,
+  # duplicate bag for duplicate ones.
+  defp init_key_ets(:unique, key_partition) do
+    :ets.new(key_partition, [:set, :public, read_concurrency: true, write_concurrency: true])
+  end
+  defp init_key_ets(:duplicate, key_partition) do
+    :ets.new(key_partition, [:duplicate_bag, :public, read_concurrency: true, write_concurrency: true])
+  end
+
+  # A process can always have multiple keys, so the
+  # pid partition is always a duplicate bag.
+  defp init_pid_ets(_, pid_partition) do
+    :ets.new(pid_partition, [:duplicate_bag, :public, read_concurrency: true, write_concurrency: true])
+  end
+
+  def handle_call(:sync, _, state) do
+    {:reply, :ok, state}
+  end
+
+  def handle_info({:EXIT, pid, _reason}, ets) do
+    entries = :ets.take(ets, pid)
+    for {_pid, key, key_ets} <- entries do
+      try do
+        :ets.match_delete(key_ets, {key, {pid, :_}})
+      catch
+        :error, :badarg -> :badarg
+      end
+    end
+    {:noreply, ets}
+  end
+  def handle_info(msg, state) do
+    super(msg, state)
+  end
+end

lib/elixir/lib/set.ex

@@ -9,7 +9,9 @@ defmodule Set do
   @type values :: [ value ]
   @type t :: map
 
-  # TODO: Deprecate every function by 1.4
+  # TODO: Remove by 2.0
+  # (hard-deprecated in elixir_dispatch)
+
   defmacrop target(set) do
     quote do
       case unquote(set) do

lib/elixir/lib/stream.ex

@@ -104,18 +104,18 @@ defmodule Stream do
     {:cont, acc}
   end
 
-  defmacrop next(f, entry, acc) do
-    quote do: unquote(f).(unquote(entry), unquote(acc))
+  defmacrop next(fun, entry, acc) do
+    quote do: unquote(fun).(unquote(entry), unquote(acc))
   end
 
-  defmacrop acc(h, n, t) do
-    quote do: [unquote(h), unquote(n) | unquote(t)]
+  defmacrop acc(head, state, tail) do
+    quote do: [unquote(head), unquote(state) | unquote(tail)]
   end
 
-  defmacrop next_with_acc(f, entry, h, n, t) do
+  defmacrop next_with_acc(fun, entry, head, state, tail) do
     quote do
-      {reason, [h | t]} = unquote(f).(unquote(entry), [unquote(h) | unquote(t)])
-      {reason, [h, unquote(n) | t]}
+      {reason, [head | tail]} = unquote(fun).(unquote(entry), [unquote(head) | unquote(tail)])
+      {reason, [head, unquote(state) | tail]}
     end
   end
 
@@ -124,7 +124,7 @@ defmodule Stream do
   @doc """
   Shortcut to `chunk(enum, n, n)`.
   """
-  @spec chunk(Enumerable.t, non_neg_integer) :: Enumerable.t
+  @spec chunk(Enumerable.t, pos_integer) :: Enumerable.t
   def chunk(enum, n), do: chunk(enum, n, n, nil)
 
   @doc """
@@ -292,7 +292,7 @@ defmodule Stream do
 
   The first item is always dropped, unless `nth` is 0.
 
-  `nth` must be a non-negative integer, or `FunctionClauseError` will be thrown.
+  `nth` must be a non-negative integer.
 
   ## Examples
 
@@ -310,6 +310,7 @@ defmodule Stream do
 
   """
   @spec drop_every(Enumerable.t, non_neg_integer) :: Enumerable.t
+  def drop_every(enum, nth)
   def drop_every(enum, 0), do: %Stream{enum: enum}
   def drop_every([], _nth), do: %Stream{enum: []}
 
@@ -340,7 +341,7 @@ defmodule Stream do
 
   ## Examples
 
-      iex> stream = Stream.each([1, 2, 3], fn(x) -> send self, x end)
+      iex> stream = Stream.each([1, 2, 3], fn(x) -> send self(), x end)
       iex> Enum.to_list(stream)
       iex> receive do: (x when is_integer(x) -> x)
       1
@@ -351,7 +352,7 @@ defmodule Stream do
 
   """
   @spec each(Enumerable.t, (element -> term)) :: Enumerable.t
-  def each(enum, fun) do
+  def each(enum, fun) when is_function(fun, 1) do
     lazy enum, fn(f1) ->
       fn(x, acc) ->
         fun.(x)
@@ -422,6 +423,10 @@ defmodule Stream do
   This operation will block the caller by the given interval
   every time a new item is streamed.
 
+  Do not use this function to generate a sequence of numbers.
+  If blocking the caller process is not necessary, use
+  `Stream.iterate(0, & &1 + 1)` instead.
+
   ## Examples
 
       iex> Stream.interval(10) |> Enum.take(10)
@@ -431,7 +436,7 @@ defmodule Stream do
   @spec interval(non_neg_integer) :: Enumerable.t
   def interval(n) do
     unfold 0, fn(count) ->
-      :timer.sleep(n)
+      Process.sleep(n)
       {count, count + 1}
     end
   end
@@ -442,8 +447,8 @@ defmodule Stream do
   This function is often used with `run/1` since any evaluation
   is delayed until the stream is executed. See `run/1` for an example.
   """
-  @spec into(Enumerable.t, Collectable.t) :: Enumerable.t
-  def into(enum, collectable, transform \\ fn x -> x end) do
+  @spec into(Enumerable.t, Collectable.t, (term -> term)) :: Enumerable.t
+  def into(enum, collectable, transform \\ fn x -> x end) when is_function(transform, 1) do
     &do_into(enum, collectable, transform, &1, &2)
   end
 
@@ -490,6 +495,41 @@ defmodule Stream do
     lazy enum, fn(f1) -> R.map(fun, f1) end
   end
 
+  @doc """
+  Creates a stream that will apply the given function on
+  every `nth` item from the enumerable.
+
+  The first item is always passed to the given function.
+
+  `nth` must be a non-negative integer.
+
+  ## Examples
+
+      iex> stream = Stream.map_every(1..10, 2, fn(x) -> x * 2 end)
+      iex> Enum.to_list(stream)
+      [2, 2, 6, 4, 10, 6, 14, 8, 18, 10]
+
+      iex> stream = Stream.map_every([1, 2, 3, 4, 5], 1, fn(x) -> x * 2 end)
+      iex> Enum.to_list(stream)
+      [2, 4, 6, 8, 10]
+
+      iex> stream = Stream.map_every(1..5, 0, fn(x) -> x * 2 end)
+      iex> Enum.to_list(stream)
+      [1, 2, 3, 4, 5]
+
+  """
+  @spec map_every(Enumerable.t, non_neg_integer, (element -> any)) :: Enumerable.t
+  def map_every(enum, nth, fun)
+
+  def map_every(enum, 1, fun), do: map(enum, fun)
+  def map_every(enum, 0, _fun), do: %Stream{enum: enum}
+  def map_every([], _nth, _fun), do: %Stream{enum: []}
+
+  def map_every(enum, nth, fun) when is_integer(nth) and nth > 0 do
+    lazy enum, nth, fn(f1) -> R.map_every(nth, fun, f1) end
+  end
+
+
   @doc """
   Creates a stream that will reject elements according to
   the given function on enumeration.
@@ -545,7 +585,7 @@ defmodule Stream do
   """
   @spec scan(Enumerable.t, (element, acc -> any)) :: Enumerable.t
   def scan(enum, fun) do
-    lazy enum, :first, fn(f1) -> R.scan_2(fun, f1) end
+    lazy enum, :first, fn(f1) -> R.scan2(fun, f1) end
   end
 
   @doc """
@@ -562,7 +602,7 @@ defmodule Stream do
   """
   @spec scan(Enumerable.t, acc, (element, acc -> any)) :: Enumerable.t
   def scan(enum, acc, fun) do
-    lazy enum, acc, fn(f1) -> R.scan_3(fun, f1) end
+    lazy enum, acc, fn(f1) -> R.scan3(fun, f1) end
   end
 
   @doc """
@@ -607,7 +647,7 @@ defmodule Stream do
 
   The first item is always included, unless `nth` is 0.
 
-  `nth` must be a non-negative integer, or `FunctionClauseError` will be thrown.
+  `nth` must be a non-negative integer.
 
   ## Examples
 
@@ -625,6 +665,7 @@ defmodule Stream do
 
   """
   @spec take_every(Enumerable.t, non_neg_integer) :: Enumerable.t
+  def take_every(enum, nth)
   def take_every(_enum, 0), do: %Stream{enum: []}
   def take_every([], _nth), do: %Stream{enum: []}
 
@@ -692,9 +733,9 @@ defmodule Stream do
       [1, 2, 3]
 
   """
-  @spec transform(Enumerable.t, acc, fun) :: Enumerable.t when
-        fun: (element, acc -> {Enumerable.t, acc} | {:halt, acc}),
-        acc: any
+  @spec transform(Enumerable.t, acc, fun) :: Enumerable.t
+        when fun: (element, acc -> {Enumerable.t, acc} | {:halt, acc}),
+             acc: any
   def transform(enum, acc, reducer) when is_function(reducer, 2) do
     &do_transform(enum, fn -> acc end, reducer, &1, &2, nil)
   end
@@ -709,9 +750,9 @@ defmodule Stream do
   This function can be seen as a combination of `Stream.resource/3` with
   `Stream.transform/3`.
   """
-  @spec transform(Enumerable.t, (() -> acc), fun, (acc -> term)) :: Enumerable.t when
-        fun: (element, acc -> {Enumerable.t, acc} | {:halt, acc}),
-        acc: any
+  @spec transform(Enumerable.t, (() -> acc), fun, (acc -> term)) :: Enumerable.t
+        when fun: (element, acc -> {Enumerable.t, acc} | {:halt, acc}),
+             acc: any
   def transform(enum, start_fun, reducer, after_fun)
       when is_function(start_fun, 0) and is_function(reducer, 2) and is_function(after_fun, 1) do
     &do_transform(enum, start_fun, reducer, &1, &2, after_fun)
@@ -721,92 +762,110 @@ defmodule Stream do
     inner = &do_transform_each(&1, &2, fun)
     step  = &do_transform_step(&1, &2)
     next  = &Enumerable.reduce(enumerables, &1, step)
-    do_transform(user_acc.(), user, fun, [], next, inner_acc, inner, after_fun)
+    do_transform(user_acc.(), user, fun, :cont, next, inner_acc, inner, after_fun)
   end
 
-  defp do_transform(user_acc, _user, _fun, _next_acc, _next, {:halt, inner_acc}, _inner, after_fun) do
+  defp do_transform(user_acc, _user, _fun, _next_op, next, {:halt, inner_acc}, _inner, after_fun) do
+    next.({:halt, []})
     do_after(after_fun, user_acc)
     {:halted, inner_acc}
   end
 
-  defp do_transform(user_acc, user, fun, next_acc, next, {:suspend, inner_acc}, inner, after_fun) do
-    {:suspended, inner_acc, &do_transform(user_acc, user, fun, next_acc, next, &1, inner, after_fun)}
+  defp do_transform(user_acc, user, fun, next_op, next, {:suspend, inner_acc}, inner, after_fun) do
+    {:suspended, inner_acc, &do_transform(user_acc, user, fun, next_op, next, &1, inner, after_fun)}
   end
 
-  defp do_transform(user_acc, user, fun, next_acc, next, inner_acc, inner, after_fun) do
-    case next.({:cont, next_acc}) do
-      {:suspended, [val | next_acc], next} ->
-        try do
-          user.(val, user_acc)
-        catch
-          kind, reason ->
-            stacktrace = System.stacktrace
-            next.({:halt, next_acc})
-            do_after(after_fun, user_acc)
-            :erlang.raise(kind, reason, stacktrace)
-        else
-          {[], user_acc} ->
-            do_transform(user_acc, user, fun, next_acc, next, inner_acc, inner, after_fun)
-          {list, user_acc} when is_list(list) ->
-            do_list_transform(user_acc, user, fun, next_acc, next, inner_acc, inner,
-                              &Enumerable.List.reduce(list, &1, fun), after_fun)
-          {:halt, user_acc} ->
-            next.({:halt, next_acc})
-            do_after(after_fun, user_acc)
-            {:halted, elem(inner_acc, 1)}
-          {other, user_acc} ->
-            do_enum_transform(user_acc, user, fun, next_acc, next, inner_acc, inner,
-                              &Enumerable.reduce(other, &1, inner), after_fun)
-        end
-      {reason, _} ->
+  defp do_transform(user_acc, _user, _fun, :halt, _next, {_, inner_acc}, _inner, after_fun) do
+    do_after(after_fun, user_acc)
+    {:halted, inner_acc}
+  end
+
+  defp do_transform(user_acc, user, fun, :cont, next, inner_acc, inner, after_fun) do
+    try do
+      next.({:cont, []})
+    catch
+      kind, reason ->
+        stacktrace = System.stacktrace
         do_after(after_fun, user_acc)
-        {reason, elem(inner_acc, 1)}
+        :erlang.raise(kind, reason, stacktrace)
+    else
+      {:suspended, vals, next} ->
+        do_transform_user(:lists.reverse(vals), user_acc, user, fun, :cont, next, inner_acc, inner, after_fun)
+      {_, vals} ->
+        do_transform_user(:lists.reverse(vals), user_acc, user, fun, :halt, next, inner_acc, inner, after_fun)
     end
   end
 
-  defp do_list_transform(user_acc, user, fun, next_acc, next, inner_acc, inner, reduce, after_fun) do
+  defp do_transform_user([], user_acc, user, fun, next_op, next, inner_acc, inner, after_fun) do
+    do_transform(user_acc, user, fun, next_op, next, inner_acc, inner, after_fun)
+  end
+
+  defp do_transform_user([val | vals], user_acc, user, fun, next_op, next, inner_acc, inner, after_fun) do
+    user.(val, user_acc)
+  catch
+    kind, reason ->
+      stacktrace = System.stacktrace
+      next.({:halt, []})
+      do_after(after_fun, user_acc)
+      :erlang.raise(kind, reason, stacktrace)
+  else
+    {[], user_acc} ->
+      do_transform_user(vals, user_acc, user, fun, next_op, next, inner_acc, inner, after_fun)
+    {list, user_acc} when is_list(list) ->
+      do_list_transform(vals, user_acc, user, fun, next_op, next, inner_acc, inner,
+                        &Enumerable.List.reduce(list, &1, fun), after_fun)
+    {:halt, user_acc} ->
+      next.({:halt, []})
+      do_after(after_fun, user_acc)
+      {:halted, elem(inner_acc, 1)}
+    {other, user_acc} ->
+      do_enum_transform(vals, user_acc, user, fun, next_op, next, inner_acc, inner,
+                        &Enumerable.reduce(other, &1, inner), after_fun)
+  end
+
+  defp do_list_transform(vals, user_acc, user, fun, next_op, next, inner_acc, inner, reduce, after_fun) do
     try do
       reduce.(inner_acc)
     catch
       kind, reason ->
         stacktrace = System.stacktrace
-        next.({:halt, next_acc})
+        next.({:halt, []})
         do_after(after_fun, user_acc)
         :erlang.raise(kind, reason, stacktrace)
     else
       {:done, acc} ->
-        do_transform(user_acc, user, fun, next_acc, next, {:cont, acc}, inner, after_fun)
+        do_transform_user(vals, user_acc, user, fun, next_op, next, {:cont, acc}, inner, after_fun)
       {:halted, acc} ->
-        next.({:halt, next_acc})
+        next.({:halt, []})
         do_after(after_fun, user_acc)
         {:halted, acc}
       {:suspended, acc, c} ->
-        {:suspended, acc, &do_list_transform(user_acc, user, fun, next_acc, next, &1, inner, c, after_fun)}
+        {:suspended, acc, &do_list_transform(vals, user_acc, user, fun, next_op, next, &1, inner, c, after_fun)}
     end
   end
 
-  defp do_enum_transform(user_acc, user, fun, next_acc, next, {op, inner_acc}, inner, reduce, after_fun) do
+  defp do_enum_transform(vals, user_acc, user, fun, next_op, next, {op, inner_acc}, inner, reduce, after_fun) do
     try do
       reduce.({op, [:outer | inner_acc]})
     catch
       kind, reason ->
         stacktrace = System.stacktrace
-        next.({:halt, next_acc})
+        next.({:halt, []})
         do_after(after_fun, user_acc)
         :erlang.raise(kind, reason, stacktrace)
     else
       # Only take into account outer halts when the op is not halt itself.
       # Otherwise, we were the ones wishing to halt, so we should just stop.
       {:halted, [:outer | acc]} when op != :halt ->
-        do_transform(user_acc, user, fun, next_acc, next, {:cont, acc}, inner, after_fun)
+        do_transform_user(vals, user_acc, user, fun, next_op, next, {:cont, acc}, inner, after_fun)
       {:halted, [_ | acc]} ->
-        next.({:halt, next_acc})
+        next.({:halt, []})
         do_after(after_fun, user_acc)
         {:halted, acc}
       {:done, [_ | acc]} ->
-        do_transform(user_acc, user, fun, next_acc, next, {:cont, acc}, inner, after_fun)
+        do_transform_user(vals, user_acc, user, fun, next_op, next, {:cont, acc}, inner, after_fun)
       {:suspended, [_ | acc], c} ->
-        {:suspended, acc, &do_enum_transform(user_acc, user, fun, next_acc, next, &1, inner, c, after_fun)}
+        {:suspended, acc, &do_enum_transform(vals, user_acc, user, fun, next_op, next, &1, inner, c, after_fun)}
     end
   end
 
@@ -837,20 +896,49 @@ defmodule Stream do
       iex> Stream.uniq([1, 2, 3, 3, 2, 1]) |> Enum.to_list
       [1, 2, 3]
 
-      iex> Stream.uniq([{1, :x}, {2, :y}, {2, :z}, {1, :x}], fn {x, _} -> x end) |> Enum.to_list
-      [{1, :x}, {2, :y}]
-
   """
   @spec uniq(Enumerable.t) :: Enumerable.t
-  @spec uniq(Enumerable.t, (element -> term)) :: Enumerable.t
-  def uniq(enum, fun \\ fn x -> x end) do
-    lazy enum, %{}, fn f1 -> R.uniq(fun, f1) end
+  def uniq(enum) do
+    uniq_by(enum, fn x -> x end)
+  end
+
+  @doc false
+  def uniq(enum, fun) do
+    uniq_by(enum, fun)
+  end
+
+  @doc """
+  Creates a stream that only emits elements if they are unique, by removing the
+  elements for which function `fun` returned duplicate items.
+
+  The function `fun` maps every element to a term which is used to
+  determine if two elements are duplicates.
+
+  Keep in mind that, in order to know if an element is unique
+  or not, this function needs to store all unique values emitted
+  by the stream. Therefore, if the stream is infinite, the number
+  of items stored will grow infinitely, never being garbage collected.
+
+  ## Example
+
+      iex> Stream.uniq_by([{1, :x}, {2, :y}, {1, :z}], fn {x, _} -> x end) |> Enum.to_list
+      [{1, :x}, {2, :y}]
+
+      iex> Stream.uniq_by([a: {:tea, 2}, b: {:tea, 2}, c: {:coffee, 1}], fn {_, y} -> y end) |> Enum.to_list
+      [a: {:tea, 2}, c: {:coffee, 1}]
+
+  """
+  @spec uniq_by(Enumerable.t, (element -> term)) :: Enumerable.t
+  def uniq_by(enum, fun) do
+    lazy enum, %{}, fn f1 -> R.uniq_by(fun, f1) end
   end
 
   @doc """
   Creates a stream where each item in the enumerable will
   be wrapped in a tuple alongside its index.
 
+  If an `offset` is given, we will index from the given offset instead of from zero.
+
   ## Examples
 
       iex> stream = Stream.with_index([1, 2, 3])
@@ -920,15 +1008,35 @@ defmodule Stream do
 
   """
   @spec zip(Enumerable.t, Enumerable.t) :: Enumerable.t
-  def zip(left, right) do
-    step      = &do_zip_step(&1, &2)
-    left_fun  = &Enumerable.reduce(left, &1, step)
-    right_fun = &Enumerable.reduce(right, &1, step)
+  def zip(left, right), do: zip([left, right])
 
-    # Return a function as a lazy enumerator.
-    &do_zip([{left_fun, []}, {right_fun, []}], &1, &2)
+  @doc """
+  Zips corresponding elements from a collection of enumerables
+  into one stream of tuples.
+
+  The zipping finishes as soon as any enumerable completes.
+
+  ## Examples
+
+      iex> concat = Stream.concat(1..3, 4..6)
+      iex> cycle = Stream.cycle(["foo", "bar", "baz"])
+      iex> Stream.zip([concat, [:a, :b, :c], cycle]) |> Enum.to_list
+      [{1, :a, "foo"}, {2, :b, "bar"}, {3, :c, "baz"}]
+
+  """
+  @spec zip([Enumerable.t]) :: Enumerable.t
+  def zip(enumerables) do
+    step      = &do_zip_step(&1, &2)
+    enum_funs = Enum.map(enumerables, fn enum ->
+      {&Enumerable.reduce(enum, &1, step), :cont}
+    end)
+
+    &do_zip(enum_funs, &1, &2)
   end
 
+  # This implementation of do_zip/3 works for any number of
+  # streams to zip, even if right now zip/2 only zips two streams.
+
   defp do_zip(zips, {:halt, acc}, _fun) do
     do_zip_close(zips)
     {:halted, acc}
@@ -940,7 +1048,7 @@ defmodule Stream do
 
   defp do_zip(zips, {:cont, acc}, callback) do
     try do
-      do_zip(zips, acc, callback, [], [])
+      do_zip_next_tuple(zips, acc, callback, [], [])
     catch
       kind, reason ->
         stacktrace = System.stacktrace
@@ -949,34 +1057,47 @@ defmodule Stream do
     else
       {:next, buffer, acc} ->
         do_zip(buffer, acc, callback)
-      {:done, _} = o ->
-        o
+      {:done, _acc} = other ->
+        other
     end
   end
 
-  defp do_zip([{fun, fun_acc} | t], acc, callback, list, buffer) do
-    case fun.({:cont, fun_acc}) do
-      {:suspended, [i | fun_acc], fun} ->
-        do_zip(t, acc, callback, [i | list], [{fun, fun_acc} | buffer])
-      {_, _} ->
-        do_zip_close(:lists.reverse(buffer, t))
+  # do_zip_next_tuple/5 computes the next tuple formed by
+  # the next element of each zipped stream.
+
+  defp do_zip_next_tuple([{_, :halt} | zips], acc, _callback, _yielded_elems, buffer) do
+    do_zip_close(:lists.reverse(buffer, zips))
+    {:done, acc}
+  end
+
+  defp do_zip_next_tuple([{fun, :cont} | zips], acc, callback, yielded_elems, buffer) do
+    case fun.({:cont, []}) do
+      {:suspended, [elem], fun} ->
+        do_zip_next_tuple(zips, acc, callback, [elem | yielded_elems], [{fun, :cont} | buffer])
+      {_, [elem]} ->
+        do_zip_next_tuple(zips, acc, callback, [elem | yielded_elems], [{fun, :halt} | buffer])
+      {_, []} ->
+        # The current zipped stream terminated, so we close all the streams
+        # and return {:halted, acc} (which is returned as is by do_zip/3).
+        do_zip_close(:lists.reverse(buffer, zips))
         {:done, acc}
     end
   end
 
-  defp do_zip([], acc, callback, list, buffer) do
-    zipped = List.to_tuple(:lists.reverse(list))
+  defp do_zip_next_tuple([] = _zips, acc, callback, yielded_elems, buffer) do
+    # "yielded_elems" is a reversed list of results for the current iteration of
+    # zipping: it needs to be reversed and converted to a tuple to have the next
+    # tuple in the list resulting from zipping.
+    zipped = List.to_tuple(:lists.reverse(yielded_elems))
     {:next, :lists.reverse(buffer), callback.(zipped, acc)}
   end
 
-  defp do_zip_close([]), do: :ok
-  defp do_zip_close([{fun, acc} | t]) do
-    fun.({:halt, acc})
-    do_zip_close(t)
+  defp do_zip_close(zips) do
+    :lists.foreach(fn {fun, _} -> fun.({:halt, []}) end, zips)
   end
 
-  defp do_zip_step(x, acc) do
-    {:suspend, [x | acc]}
+  defp do_zip_step(x, []) do
+    {:suspend, [x]}
   end
 
   ## Sources

lib/elixir/lib/stream/reducers.ex

@@ -2,13 +2,13 @@ defmodule Stream.Reducers do
   # Collection of reducers shared by Enum and Stream.
   @moduledoc false
 
-  defmacro chunk(n, step, limit, f \\ nil) do
+  defmacro chunk(amount, step, limit, fun \\ nil) do
     quote do
-      fn entry, acc(h, {buffer, count}, t) ->
+      fn entry, acc(head, {buffer, count}, tail) ->
         buffer = [entry | buffer]
-        count  = count + 1
+        count = count + 1
 
-        new =
+        new_state =
           if count >= unquote(limit) do
             left = count - unquote(step)
             {Enum.take(buffer, left), left}
@@ -16,83 +16,82 @@ defmodule Stream.Reducers do
             {buffer, count}
           end
 
-        if count == unquote(n) do
-          next_with_acc(unquote(f), :lists.reverse(buffer), h, new, t)
+        if count == unquote(amount) do
+          next_with_acc(unquote(fun), :lists.reverse(buffer), head, new_state, tail)
         else
-          skip(acc(h, new, t))
+          skip(acc(head, new_state, tail))
         end
       end
     end
   end
 
-  defmacro chunk_by(callback, f \\ nil) do
+  defmacro chunk_by(callback, fun \\ nil) do
     quote do
       fn
-        entry, acc(h, {buffer, value}, t) ->
+        entry, acc(head, {buffer, value}, tail) ->
           new_value = unquote(callback).(entry)
           if new_value == value do
-            skip(acc(h, {[entry | buffer], value}, t))
+            skip(acc(head, {[entry | buffer], value}, tail))
           else
-            next_with_acc(unquote(f), :lists.reverse(buffer), h, {[entry], new_value}, t)
+            next_with_acc(unquote(fun), :lists.reverse(buffer), head, {[entry], new_value}, tail)
           end
-        entry, acc(h, nil, t) ->
-          skip(acc(h, {[entry], unquote(callback).(entry)}, t))
+        entry, acc(head, nil, tail) ->
+          skip(acc(head, {[entry], unquote(callback).(entry)}, tail))
       end
     end
   end
 
-  defmacro dedup(callback, f \\ nil) do
+  defmacro dedup(callback, fun \\ nil) do
     quote do
-      fn(entry, acc(h, prev, t) = acc) ->
+      fn(entry, acc(head, prev, tail) = acc) ->
         value = unquote(callback).(entry)
         case prev do
-          {:value, ^value}  -> skip(acc)
-          _ -> next_with_acc(unquote(f), entry, h, {:value, value}, t)
+          {:value, ^value} -> skip(acc)
+          _ -> next_with_acc(unquote(fun), entry, head, {:value, value}, tail)
         end
       end
     end
   end
 
-  defmacro drop(f \\ nil) do
+  defmacro drop(fun \\ nil) do
     quote do
       fn
-        _entry, acc(h, n, t) when n > 0 ->
-          skip(acc(h, n-1, t))
-        entry, acc(h, n, t) ->
-          next_with_acc(unquote(f), entry, h, n, t)
+        _entry, acc(head, amount, tail) when amount > 0 ->
+          skip(acc(head, amount - 1, tail))
+        entry, acc(head, amount, tail) ->
+          next_with_acc(unquote(fun), entry, head, amount, tail)
       end
     end
   end
 
-  defmacro drop_every(nth, f \\ nil) do
+  defmacro drop_every(nth, fun \\ nil) do
     quote do
       fn
-        entry, acc(h, n, t) when n === :first
-                            when n === unquote(nth) ->
-          skip(acc(h, 1, t))
-        entry, acc(h, n, t) ->
-          next_with_acc(unquote(f), entry, h, n+1, t)
+        entry, acc(head, curr, tail) when curr in [unquote(nth), :first] ->
+          skip(acc(head, 1, tail))
+        entry, acc(head, curr, tail) ->
+          next_with_acc(unquote(fun), entry, head, curr + 1, tail)
       end
     end
   end
 
-  defmacro drop_while(callback, f \\ nil) do
+  defmacro drop_while(callback, fun \\ nil) do
     quote do
-      fn entry, acc(h, bool, t) = orig ->
+      fn entry, acc(head, bool, tail) = original ->
         if bool and unquote(callback).(entry) do
-          skip(orig)
+          skip(original)
         else
-          next_with_acc(unquote(f), entry, h, false, t)
+          next_with_acc(unquote(fun), entry, head, false, tail)
         end
       end
     end
   end
 
-  defmacro filter(callback, f \\ nil) do
+  defmacro filter(callback, fun \\ nil) do
     quote do
       fn(entry, acc) ->
         if unquote(callback).(entry) do
-          next(unquote(f), entry, acc)
+          next(unquote(fun), entry, acc)
         else
           skip(acc)
         end
@@ -100,11 +99,11 @@ defmodule Stream.Reducers do
     end
   end
 
-  defmacro filter_map(filter, mapper, f \\ nil) do
+  defmacro filter_map(filter, mapper, fun \\ nil) do
     quote do
       fn(entry, acc) ->
         if unquote(filter).(entry) do
-          next(unquote(f), unquote(mapper).(entry), acc)
+          next(unquote(fun), unquote(mapper).(entry), acc)
         else
           skip(acc)
         end
@@ -112,19 +111,30 @@ defmodule Stream.Reducers do
     end
   end
 
-  defmacro map(callback, f \\ nil) do
+  defmacro map(callback, fun \\ nil) do
     quote do
       fn(entry, acc) ->
-        next(unquote(f), unquote(callback).(entry), acc)
+        next(unquote(fun), unquote(callback).(entry), acc)
       end
     end
   end
 
-  defmacro reject(callback, f \\ nil) do
+  defmacro map_every(nth, mapper, fun \\ nil) do
+    quote do
+      fn
+        entry, acc(head, curr, tail) when curr in [unquote(nth), :first] ->
+          next_with_acc(unquote(fun), unquote(mapper).(entry), head, 1, tail)
+        entry, acc(head, curr, tail) ->
+          next_with_acc(unquote(fun), entry, head, curr + 1, tail)
+      end
+    end
+  end
+
+  defmacro reject(callback, fun \\ nil) do
     quote do
       fn(entry, acc) ->
         unless unquote(callback).(entry) do
-          next(unquote(f), entry, acc)
+          next(unquote(fun), entry, acc)
         else
           skip(acc)
         end
@@ -132,62 +142,59 @@ defmodule Stream.Reducers do
     end
   end
 
-  defmacro scan_2(callback, f \\ nil) do
+  defmacro scan2(callback, fun \\ nil) do
     quote do
       fn
-        entry, acc(h, :first, t) ->
-          next_with_acc(unquote(f), entry, h, {:ok, entry}, t)
-        entry, acc(h, {:ok, acc}, t) ->
+        entry, acc(head, :first, tail) ->
+          next_with_acc(unquote(fun), entry, head, {:ok, entry}, tail)
+        entry, acc(head, {:ok, acc}, tail) ->
           value = unquote(callback).(entry, acc)
-          next_with_acc(unquote(f), value, h, {:ok, value}, t)
+          next_with_acc(unquote(fun), value, head, {:ok, value}, tail)
       end
     end
   end
 
-  defmacro scan_3(callback, f \\ nil) do
+  defmacro scan3(callback, fun \\ nil) do
     quote do
-      fn(entry, acc(h, acc, t)) ->
+      fn(entry, acc(head, acc, tail)) ->
         value = unquote(callback).(entry, acc)
-        next_with_acc(unquote(f), value, h, value, t)
+        next_with_acc(unquote(fun), value, head, value, tail)
       end
     end
   end
 
-  defmacro take(f \\ nil) do
+  defmacro take(fun \\ nil) do
     quote do
-      fn(entry, acc(h, n, t) = orig) ->
-        case n do
+      fn(entry, acc(head, curr, tail) = original) ->
+        case curr do
           0 ->
-            {:halt, orig}
+            {:halt, original}
           1 ->
-            case next_with_acc(unquote(f), entry, h, n-1, t) do
-              {:cont, acc} -> {:halt, acc}
-              reason -> reason
-            end
+            {_, acc} = next_with_acc(unquote(fun), entry, head, 0, tail)
+            {:halt, acc}
           _ ->
-            next_with_acc(unquote(f), entry, h, n-1, t)
+            next_with_acc(unquote(fun), entry, head, curr - 1, tail)
         end
       end
     end
   end
 
-  defmacro take_every(nth, f \\ nil) do
+  defmacro take_every(nth, fun \\ nil) do
     quote do
       fn
-        entry, acc(h, n, t) when n === :first
-                            when n === unquote(nth) ->
-          next_with_acc(unquote(f), entry, h, 1, t)
-        entry, acc(h, n, t) ->
-          skip(acc(h, n+1, t))
+        entry, acc(head, curr, tail) when curr in [unquote(nth), :first] ->
+          next_with_acc(unquote(fun), entry, head, 1, tail)
+        entry, acc(head, curr, tail) ->
+          skip(acc(head, curr + 1, tail))
       end
     end
   end
 
-  defmacro take_while(callback, f \\ nil) do
+  defmacro take_while(callback, fun \\ nil) do
     quote do
       fn(entry, acc) ->
         if unquote(callback).(entry) do
-          next(unquote(f), entry, acc)
+          next(unquote(fun), entry, acc)
         else
           {:halt, acc}
         end
@@ -195,23 +202,23 @@ defmodule Stream.Reducers do
     end
   end
 
-  defmacro uniq(callback, f \\ nil) do
+  defmacro uniq_by(callback, fun \\ nil) do
     quote do
-      fn(entry, acc(h, prev, t) = acc) ->
+      fn(entry, acc(head, prev, tail) = original) ->
         value = unquote(callback).(entry)
         if Map.has_key?(prev, value) do
-          skip(acc)
+          skip(original)
         else
-          next_with_acc(unquote(f), entry, h, Map.put(prev, value, true), t)
+          next_with_acc(unquote(fun), entry, head, Map.put(prev, value, true), tail)
         end
       end
     end
   end
 
-  defmacro with_index(f \\ nil) do
+  defmacro with_index(fun \\ nil) do
     quote do
-      fn(entry, acc(h, counter, t)) ->
-        next_with_acc(unquote(f), {entry, counter}, h, counter + 1, t)
+      fn(entry, acc(head, counter, tail)) ->
+        next_with_acc(unquote(fun), {entry, counter}, head, counter + 1, tail)
       end
     end
   end

lib/elixir/lib/string.ex

@@ -4,10 +4,10 @@ defmodule String do
   @moduledoc ~S"""
   A String in Elixir is a UTF-8 encoded binary.
 
-  ## Codepoints and graphemes
+  ## Codepoints and grapheme cluster
 
   The functions in this module act according to the Unicode
-  Standard, version 6.3.0.
+  Standard, version 9.0.0.
 
   As per the standard, a codepoint is a single Unicode Character,
   which may be represented by one or more bytes.
@@ -22,12 +22,11 @@ defmodule String do
       iex> String.length("é")
       1
 
-  Furthermore, this module also presents the concept of
-  graphemes. A single grapheme can consist of multiple codepoints
-  that may be perceived as a single character by readers. For example,
-  the "é" grapheme can be represented either as a single "e with acute"
-  codepoint (like above), or as the letter "e" followed by a
-  "combining acute accent" (two codepoints):
+  Furthermore, this module also presents the concept of grapheme cluster
+  (from now on referenced as graphemes). Graphemes can consist of multiple
+  codepoints that may be perceived as a single character by readers. For
+  example, "é" can be represented either as a single "e with acute" codepoint
+  or as the letter "e" followed by a "combining acute accent" (two codepoints):
 
       iex> string = "\u0065\u0301"
       iex> byte_size(string)
@@ -44,9 +43,9 @@ defmodule String do
 
   Graphemes can also be two characters that are interpreted
   as one by some languages. For example, some languages may
-  consider "ch" as a grapheme. However, since this information
-  depends on the locale, it is not taken into account by this
-  module.
+  consider "ch" as a single character. However, since this
+  information depends on the locale, it is not taken into account
+  by this module.
 
   In general, the functions in this module rely on the Unicode
   Standard, but do not contain any of the locale specific behaviour.
@@ -106,8 +105,8 @@ defmodule String do
 
   While this is much better (we don't traverse `full` twice),
   it could still be improved. In this case, since we want to
-  extract a substring from a string, we can use `byte_size/1`
-  and `binary_part/3` as there is no chance we will slice in
+  extract a substring from a string, we can use `Kernel.byte_size/1`
+  and `Kernel.binary_part/3` as there is no chance we will slice in
   the middle of a codepoint made of more than one byte:
 
       iex> take_prefix = fn full, prefix ->
@@ -173,14 +172,15 @@ defmodule String do
   a correct result even if an invalid codepoint is fed into it.
 
   In other words, this module expects invalid data to be detected
-  when retrieving data from the external source. For example, a
-  driver that reads strings from a database will be
-  responsible to check the validity of the encoding.
+  elsewhere, usually when retrieving data from the external source.
+  For example, a driver that reads strings from a database will be
+  responsible to check the validity of the encoding. `String.chunk/2`
+  can be used for breaking a string into valid and invalid parts.
 
   ## Patterns
 
   Many functions in this module work with patterns. For example,
-  String.split/2 can split a string into multiple patterns given
+  `String.split/2` can split a string into multiple patterns given
   a pattern. This pattern can be a string, a list of strings or
   a compiled pattern:
 
@@ -217,23 +217,27 @@ defmodule String do
   @spec printable?(t) :: boolean
   def printable?(string)
 
-  def printable?(<<h::utf8, t::binary>>)
-      when h in 0x20..0x7E
-      when h in 0xA0..0xD7FF
-      when h in 0xE000..0xFFFD
-      when h in 0x10000..0x10FFFF do
-    printable?(t)
+  for char <- 0x20..0x7E do
+    def printable?(<<unquote(char), rest::binary>>) do
+      printable?(rest)
+    end
   end
+  def printable?(<<?\n, rest::binary>>), do: printable?(rest)
+  def printable?(<<?\r, rest::binary>>), do: printable?(rest)
+  def printable?(<<?\t, rest::binary>>), do: printable?(rest)
+  def printable?(<<?\v, rest::binary>>), do: printable?(rest)
+  def printable?(<<?\b, rest::binary>>), do: printable?(rest)
+  def printable?(<<?\f, rest::binary>>), do: printable?(rest)
+  def printable?(<<?\e, rest::binary>>), do: printable?(rest)
+  def printable?(<<?\d, rest::binary>>), do: printable?(rest)
+  def printable?(<<?\a, rest::binary>>), do: printable?(rest)
 
-  def printable?(<<?\n, t::binary>>), do: printable?(t)
-  def printable?(<<?\r, t::binary>>), do: printable?(t)
-  def printable?(<<?\t, t::binary>>), do: printable?(t)
-  def printable?(<<?\v, t::binary>>), do: printable?(t)
-  def printable?(<<?\b, t::binary>>), do: printable?(t)
-  def printable?(<<?\f, t::binary>>), do: printable?(t)
-  def printable?(<<?\e, t::binary>>), do: printable?(t)
-  def printable?(<<?\d, t::binary>>), do: printable?(t)
-  def printable?(<<?\a, t::binary>>), do: printable?(t)
+  def printable?(<<char::utf8, rest::binary>>)
+      when char in 0xA0..0xD7FF
+      when char in 0xE000..0xFFFD
+      when char in 0x10000..0x10FFFF do
+    printable?(rest)
+  end
 
   def printable?(<<>>), do: true
   def printable?(binary) when is_binary(binary), do: false
@@ -276,6 +280,10 @@ defmodule String do
   Empty strings are only removed from the result if the
   `trim` option is set to `true` (default is `false`).
 
+  When the pattern used is a regular expression, the string is
+  split using `Regex.split/3`. In that case this function accepts
+  additional options which are documented in `Regex.split/3`.
+
   ## Examples
 
   Splitting with a string pattern:
@@ -305,6 +313,9 @@ defmodule String do
       iex> String.split(" a b c ", ~r{\s}, trim: true)
       ["a", "b", "c"]
 
+      iex> String.split("abc", ~r{b}, include_captures: true)
+      ["a", "b", "c"]
+
   Splitting on empty patterns returns graphemes:
 
       iex> String.split("abc", ~r{})
@@ -338,8 +349,6 @@ defmodule String do
     :binary.split(string, pattern, [:global])
   end
 
-  # TODO: Use :trim_all with :binary.split/3 whenever parts is
-  # infinity and pattern != "" and at least Erlang 18.2 is required.
   def split(string, pattern, options) when is_binary(string) do
     parts   = Keyword.get(options, :parts, :infinity)
     trim    = Keyword.get(options, :trim, false)
@@ -350,6 +359,7 @@ defmodule String do
   defp parts_to_index(:infinity),                      do: 0
   defp parts_to_index(n) when is_integer(n) and n > 0, do: n
 
+  defp split_each("", _pattern, true, 1), do: []
   defp split_each(string, _pattern, _trim, 1) when is_binary(string), do: [string]
   defp split_each(string, pattern, trim, count) do
     case do_splitter(string, pattern, trim) do
@@ -372,6 +382,18 @@ defmodule String do
   ## Options
 
     * :trim - when `true`, does not emit empty patterns
+
+  ## Examples
+
+      iex> String.splitter("1,2 3,4 5,6 7,8,...,99999", [" ", ","]) |> Enum.take(4)
+      ["1", "2", "3", "4"]
+
+      iex> String.splitter("abcd", "") |> Enum.take(10)
+      ["a", "b", "c", "d", ""]
+
+      iex> String.splitter("abcd", "", trim: true) |> Enum.take(10)
+      ["a", "b", "c", "d"]
+
   """
   @spec splitter(t, pattern, Keyword.t) :: Enumerable.t
   def splitter(string, pattern, options \\ []) do
@@ -402,6 +424,7 @@ defmodule String do
   end
 
   defp maybe_compile_pattern(""), do: ""
+  defp maybe_compile_pattern(pattern) when is_tuple(pattern), do: pattern
   defp maybe_compile_pattern(pattern), do: :binary.compile_pattern(pattern)
 
   @doc """
@@ -592,6 +615,11 @@ defmodule String do
 
   Returns the string untouched if there are no occurrences.
 
+  If `match` is `""`, this function raises an `ArgumentError` exception: this
+  happens because this function replaces **all** the occurrences of `match` at
+  the beginning of `string`, and it's impossible to replace "multiple"
+  occurrences of `""`.
+
   ## Examples
 
       iex> String.replace_leading("hello world", "hello ", "")
@@ -605,8 +633,13 @@ defmodule String do
       "ola ola world"
 
   """
+  @spec replace_leading(t, t, t) :: t | no_return
   def replace_leading(string, match, replacement)
       when is_binary(string) and is_binary(match) and is_binary(replacement) do
+    if match == "" do
+      raise ArgumentError, "cannot use an empty string as the match to replace"
+    end
+
     prefix_size = byte_size(match)
     suffix_size = byte_size(string) - prefix_size
     replace_leading(string, match, replacement, prefix_size, suffix_size, 0)
@@ -630,6 +663,11 @@ defmodule String do
 
   Returns the string untouched if there are no occurrences.
 
+  If `match` is `""`, this function raises an `ArgumentError` exception: this
+  happens because this function replaces **all** the occurrences of `match` at
+  the end of `string`, and it's impossible to replace "multiple" occurrences of
+  `""`.
+
   ## Examples
 
       iex> String.replace_trailing("hello world", " world", "")
@@ -643,8 +681,13 @@ defmodule String do
       "hello mundo mundo"
 
   """
+  @spec replace_trailing(t, t, t) :: t | no_return
   def replace_trailing(string, match, replacement)
       when is_binary(string) and is_binary(match) and is_binary(replacement) do
+    if match == "" do
+      raise ArgumentError, "cannot use an empty string as the match to replace"
+    end
+
     suffix_size = byte_size(match)
     prefix_size = byte_size(string) - suffix_size
     replace_trailing(string, match, replacement, prefix_size, suffix_size, 0)
@@ -666,7 +709,8 @@ defmodule String do
   @doc """
   Replaces prefix in `string` by `replacement` if it matches `match`.
 
-  Returns the string untouched if there is no match.
+  Returns the string untouched if there is no match. If `match` is an empty
+  string (`""`), `replacement` is just prepended to `string`.
 
   ## Examples
 
@@ -684,7 +728,11 @@ defmodule String do
       iex> String.replace_prefix("hello hello world", "hello ", "ola ")
       "ola hello world"
 
+      iex> String.replace_prefix("world", "", "hello ")
+      "hello world"
+
   """
+  @spec replace_prefix(t, t, t) :: t
   def replace_prefix(string, match, replacement)
       when is_binary(string) and is_binary(match) and is_binary(replacement) do
     prefix_size = byte_size(match)
@@ -701,7 +749,8 @@ defmodule String do
   @doc """
   Replaces suffix in `string` by `replacement` if it matches `match`.
 
-  Returns the string untouched if there is no match.
+  Returns the string untouched if there is no match. If `match` is an empty
+  string (`""`), `replacement` is just appended to `string`.
 
   ## Examples
 
@@ -719,7 +768,11 @@ defmodule String do
       iex> String.replace_suffix("hello world world", " world", " mundo")
       "hello world mundo"
 
+      iex> String.replace_suffix("hello", "", " world")
+      "hello world"
+
   """
+  @spec replace_suffix(t, t, t) :: t
   def replace_suffix(string, match, replacement)
       when is_binary(string) and is_binary(match) and is_binary(replacement) do
     suffix_size = byte_size(match)
@@ -1005,11 +1058,13 @@ defmodule String do
       iex> String.replace("a,b,c", ~r/,(.)/, ",\\1\\g{1}")
       "a,bb,cc"
 
-  Notice we had to escape the escape character `\`. By giving `\0`,
-  one can inject the whole matched pattern in the replacement string.
+  Notice we had to escape the backslash escape character (i.e., we used `\\N`
+  instead of just `\N` to escape the backslash; same thing for `\\g{N}`). By
+  giving `\0`, one can inject the whole matched pattern in the replacement
+  string.
 
   When the pattern is a string, a developer can use the replaced part inside
-  the `replacement` by using the `:insert_replace` option and specifying the
+  the `replacement` by using the `:insert_replaced` option and specifying the
   position(s) inside the `replacement` where the string pattern will be
   inserted:
 
@@ -1022,7 +1077,7 @@ defmodule String do
       iex> String.replace("a,b,c", ",", "[]", insert_replaced: [1, 1])
       "a[,,]b[,,]c"
 
-  If any position given in the `:insert_replace` option is larger than the
+  If any position given in the `:insert_replaced` option is larger than the
   replacement string, or is negative, an `ArgumentError` is raised.
   """
   @spec replace(t, pattern | Regex.t, t, Keyword.t) :: t
@@ -1208,8 +1263,8 @@ defmodule String do
 
   The trait can be one of two options:
 
-    * `:valid`     - the string is split into chunks of valid and invalid character
-      sequences
+    * `:valid` - the string is split into chunks of valid and invalid
+      character sequences
 
     * `:printable` - the string is split into chunks of printable and
       non-printable character sequences
@@ -1325,7 +1380,7 @@ defmodule String do
   defdelegate next_grapheme_size(string), to: String.Unicode
 
   @doc """
-  Returns the first grapheme from a utf8 string,
+  Returns the first grapheme from a UTF-8 string,
   `nil` if the string is empty.
 
   ## Examples
@@ -1346,7 +1401,7 @@ defmodule String do
   end
 
   @doc """
-  Returns the last grapheme from a utf8 string,
+  Returns the last grapheme from a UTF-8 string,
   `nil` if the string is empty.
 
   ## Examples
@@ -1370,7 +1425,7 @@ defmodule String do
   defp do_last(nil, last_char), do: last_char
 
   @doc """
-  Returns the number of Unicode graphemes in a utf8 string.
+  Returns the number of Unicode graphemes in a UTF-8 string.
 
   ## Examples
 
@@ -1385,7 +1440,7 @@ defmodule String do
   defdelegate length(string), to: String.Unicode
 
   @doc """
-  Returns the grapheme at the `position` of the given utf8 `string`.
+  Returns the grapheme at the `position` of the given UTF-8 `string`.
   If `position` is greater than `string` length, then it returns `nil`.
 
   ## Examples
@@ -1635,7 +1690,7 @@ defmodule String do
       iex> String.ends_with? "language", ["youth", "elixir"]
       false
 
-  An empty string will always match:
+  An empty suffix will always match:
 
       iex> String.ends_with? "language", ""
       true
@@ -1944,8 +1999,7 @@ defmodule String do
   @doc """
   Returns a keyword list that represents an edit script.
 
-  The algorithm is outlined in the
-  "An O(ND) Difference Algorithm and Its Variations" paper by E. Myers.
+  Check `List.myers_difference/2` for more information.
 
   ## Examples
 
@@ -1956,90 +2010,11 @@ defmodule String do
 
   """
   @spec myers_difference(t, t) :: [{:eq | :ins | :del, t}] | nil
-  def myers_difference(str1, str2) do
-    {chars1, len1} = chars_and_length(str1)
-    {chars2, len2} = chars_and_length(str2)
-
-    path = {0, 0, chars1, chars2, []}
-    find_script(0, len1 + len2, [path])
-  end
-
-  defp find_script(envelope, max, _paths) when envelope > max do
-    nil
-  end
-
-  defp find_script(envelope, max, paths) do
-    case each_diagonal(-envelope, envelope, paths, []) do
-      {:done, edits} -> compact_reverse(edits, [])
-      {:next, paths} -> find_script(envelope + 1, max, paths)
-    end
-  end
-
-  defp compact_reverse([], acc), do: acc
-
-  defp compact_reverse([{kind, char} | rest], [{kind, chars} | acc]) do
-    compact_reverse(rest, [{kind, char <> chars} | acc])
-  end
-
-  defp compact_reverse([elem | rest], acc) do
-    compact_reverse(rest, [elem | acc])
-  end
-
-  defp each_diagonal(diag, limit, _paths, next_paths) when diag > limit do
-    {:next, Enum.reverse(next_paths)}
-  end
-
-  defp each_diagonal(diag, limit, paths, next_paths) do
-    {path, rest} = proceed_path(diag, limit, paths)
-    with {:cont, path} <- follow_snake(path) do
-      each_diagonal(diag + 2, limit, rest, [path | next_paths])
-    end
-  end
-
-  defp proceed_path(0, 0, [path]), do: {path, []}
-
-  defp proceed_path(diag, limit, [path | _] = paths) when diag == -limit do
-    {move_down(path), paths}
-  end
-
-  defp proceed_path(diag, limit, [path]) when diag == limit do
-    {move_right(path), []}
-  end
-
-  defp proceed_path(_diag, _limit, [path1, path2 | rest]) do
-    if elem(path1, 1) > elem(path2, 1) do
-      {move_right(path1), [path2 | rest]}
-    else
-      {move_down(path2), [path2 | rest]}
-    end
-  end
-
-  defp move_right({x, y, chars1, [char | rest], edits}) do
-    {x + 1, y, chars1, rest, [{:ins, char} | edits]}
-  end
-
-  defp move_right({x, y, chars1, [], edits}) do
-    {x + 1, y, chars1, [], edits}
-  end
-
-  defp move_down({x, y, [char | rest], chars2, edits}) do
-    {x, y + 1, rest, chars2, [{:del, char} | edits]}
-  end
-
-  defp move_down({x, y, [], chars2, edits}) do
-    {x, y + 1, [], chars2, edits}
-  end
-
-  defp follow_snake({x, y, [char | rest1], [char | rest2], edits}) do
-    follow_snake({x + 1, y + 1, rest1, rest2, [{:eq, char} | edits]})
-  end
-
-  defp follow_snake({_x, _y, [], [], edits}) do
-    {:done, edits}
-  end
-
-  defp follow_snake(path) do
-    {:cont, path}
+  def myers_difference(string1, string2) do
+    List.myers_difference(graphemes(string1), graphemes(string2))
+    |> Enum.map(fn {kind, chars} ->
+      {kind, IO.iodata_to_binary(chars)}
+    end)
   end
 
   # TODO: Deprecate by v1.5

lib/elixir/lib/string/chars.ex

@@ -3,12 +3,12 @@ import Kernel, except: [to_string: 1]
 defprotocol String.Chars do
   @moduledoc ~S"""
   The `String.Chars` protocol is responsible for
-  converting a structure to a Binary (only if applicable).
+  converting a structure to a binary (only if applicable).
   The only function required to be implemented is
   `to_string` which does the conversion.
 
-  The `to_string` function automatically imported
-  by Kernel invokes this protocol. String
+  The `to_string/1` function automatically imported
+  by `Kernel` invokes this protocol. String
   interpolation also invokes `to_string` in its
   arguments. For example, `"foo#{bar}"` is the same
   as `"foo" <> to_string(bar)`.

lib/elixir/lib/supervisor.ex

@@ -39,8 +39,9 @@ defmodule Supervisor do
       import Supervisor.Spec
 
       # Supervise the Stack server which will be started with
-      # a single argument [:hello] and the default registered
-      # name of MyStack.
+      # two arguments. The initial stack, [:hello], and a
+      # keyword list containing the GenServer options that
+      # set the registered name of the server to MyStack.
       children = [
         worker(Stack, [[:hello], [name: MyStack]])
       ]
@@ -48,6 +49,10 @@ defmodule Supervisor do
       # Start the supervisor with our child
       {:ok, pid} = Supervisor.start_link(children, strategy: :one_for_one)
 
+      # There is one child worker started
+      Supervisor.count_children(pid)
+      #=> %{active: 1, specs: 1, supervisors: 0, workers: 1}
+
   Notice that when starting the GenServer, we are registering it
   with name `MyStack`, which allows us to call it directly and
   get what is on the stack:
@@ -64,7 +69,7 @@ defmodule Supervisor do
   However, there is a bug in our stack server. If we call `:pop` and
   the stack is empty, it is going to crash because no clause matches:
 
-      GenServer.call(:sup_stack, :pop)
+      GenServer.call(MyStack, :pop)
       ** (exit) exited in: GenServer.call(MyStack, :pop, 5000)
 
   Luckily, since the server is being supervised by a supervisor, the
@@ -90,6 +95,7 @@ defmodule Supervisor do
   explicitly defining a supervision module:
 
       defmodule MyApp.Supervisor do
+        # Automatically imports Supervisor.Spec
         use Supervisor
 
         def start_link do
@@ -157,9 +163,13 @@ defmodule Supervisor do
         worker(Stack, [], restart: :transient)
       ]
 
-      # Start the supervisor with our one child
+      # Start the supervisor with our one child as a template
       {:ok, sup_pid} = Supervisor.start_link(children, strategy: :simple_one_for_one)
 
+      # No child worker is active yet until start_child is called
+      Supervisor.count_children(sup_pid)
+      #=> %{active: 0, specs: 1, supervisors: 0, workers: 0}
+
   There are a couple differences here:
 
     * the simple one for one specification can define only one child which
@@ -218,6 +228,9 @@ defmodule Supervisor do
     quote location: :keep do
       @behaviour Supervisor
       import Supervisor.Spec
+
+      @doc false
+      def init(arg)
     end
   end
 
@@ -266,9 +279,9 @@ defmodule Supervisor do
   If the supervisor and its child processes are successfully created
   (i.e., if the start function of each child process returns `{:ok, child}`,
   `{:ok, child, info}`, or `:ignore`) this function returns
-  `{:ok, pid}`, where `pid` is the pid of the supervisor. If a process with the
+  `{:ok, pid}`, where `pid` is the PID of the supervisor. If a process with the
    specified name already exists, the function returns `{:error,
-   {:already_started, pid}}`, where `pid` is the pid of that process.
+   {:already_started, pid}}`, where `pid` is the PID of that process.
 
   If the start function of any of the child processes fails or returns an error
   tuple or an erroneous value, the supervisor first terminates with reason
@@ -286,14 +299,14 @@ defmodule Supervisor do
   end
 
   @doc """
-  Starts a supervisor module with the given `arg`.
+  Starts a supervisor process with the given `module` and `arg`.
 
-  To start the supervisor, the `init/1` callback will be invoked in the given
-  `module`, with `arg` as its argument. The `init/1` callback must return a
+  To start the supervisor, the `c:init/1` callback will be invoked in the given
+  `module`, with `arg` as its argument. The `c:init/1` callback must return a
   supervisor specification which can be created with the help of the functions
   in the `Supervisor.Spec` module (especially `Supervisor.Spec.supervise/2`).
 
-  If the `init/1` callback returns `:ignore`, this function returns
+  If the `c:init/1` callback returns `:ignore`, this function returns
   `:ignore` as well and the supervisor terminates with reason `:normal`.
   If it fails or returns an incorrect value, this function returns
   `{:error, term}` where `term` is a term with information about the
@@ -311,8 +324,21 @@ defmodule Supervisor do
         :supervisor.start_link(module, arg)
       atom when is_atom(atom) ->
         :supervisor.start_link({:local, atom}, module, arg)
-      other when is_tuple(other) ->
-        :supervisor.start_link(other, module, arg)
+      {:global, _term} = tuple ->
+        :supervisor.start_link(tuple, module, arg)
+      {:via, via_module, _term} = tuple when is_atom(via_module) ->
+        :supervisor.start_link(tuple, module, arg)
+      other ->
+        raise ArgumentError, """
+        expected :name option to be one of:
+
+          * nil
+          * atom
+          * {:global, term}
+          * {:via, module, term}
+
+        Got: #{inspect(other)}
+        """
     end
   end
 
@@ -334,11 +360,11 @@ defmodule Supervisor do
   respectively.
 
   If the child process start function returns `{:ok, child}` or `{:ok, child,
-  info}`, then child specification and pid are added to the supervisor and
+  info}`, then child specification and PID are added to the supervisor and
   this function returns the same value.
 
   If the child process start function returns `:ignore`, the child specification
-  is added to the supervisor, the pid is set to `:undefined` and this function
+  is added to the supervisor, the PID is set to `:undefined` and this function
   returns `{:ok, :undefined}`.
 
   If the child process start function returns an error tuple or an erroneous
@@ -352,13 +378,13 @@ defmodule Supervisor do
   end
 
   @doc """
-  Terminates the given children, identified by pid or child id.
+  Terminates the given children, identified by PID or child id.
 
   If the supervisor is not a `:simple_one_for_one`, the child id is expected
   and the process, if there's one, is terminated; the child specification is
   kept unless the child is temporary.
 
-  In case of a `:simple_one_for_one` supervisor, a pid is expected. If the child
+  In case of a `:simple_one_for_one` supervisor, a PID is expected. If the child
   specification identifier is given instead of a `pid`, this function returns
   `{:error, :simple_one_for_one}`.
 
@@ -367,7 +393,7 @@ defmodule Supervisor do
   `delete_child/2` to remove the child specification.
 
   If successful, this function returns `:ok`. If there is no child specification
-  for the given child id or there is no process with the given pid, this
+  for the given child id or there is no process with the given PID, this
   function returns `{:error, :not_found}`.
   """
   @spec terminate_child(supervisor, pid | Supervisor.Spec.child_id) :: :ok | {:error, error}
@@ -404,9 +430,9 @@ defmodule Supervisor do
   when the child terminates, and thus it is not possible to restart such children.
 
   If the child process start function returns `{:ok, child}` or `{:ok, child, info}`,
-  the pid is added to the supervisor and this function returns the same value.
+  the PID is added to the supervisor and this function returns the same value.
 
-  If the child process start function returns `:ignore`, the pid remains set to
+  If the child process start function returns `:ignore`, the PID remains set to
   `:undefined` and this function returns `{:ok, :undefined}`.
 
   This function may return an error with an appropriate error tuple if the
@@ -436,7 +462,7 @@ defmodule Supervisor do
     * `id` - as defined in the child specification or `:undefined` in the case
       of a `simple_one_for_one` supervisor
 
-    * `child` - the pid of the corresponding child process, `:restarting` if the
+    * `child` - the PID of the corresponding child process, `:restarting` if the
       process is about to be restarted, or `:undefined` if there is no such
       process
 

lib/elixir/lib/supervisor/spec.ex

@@ -38,7 +38,7 @@ defmodule Supervisor.Spec do
   Notice in this case we don't have to explicitly import
   `Supervisor.Spec` as `use Supervisor` automatically does so.
   Defining a module-based supervisor can be useful, for example,
-  to perform initialization tasks in the `init/1` callback.
+  to perform initialization tasks in the `c:init/1` callback.
 
   ## Supervisor and worker options
 
@@ -81,7 +81,7 @@ defmodule Supervisor.Spec do
   The following shutdown values are supported in the `:shutdown` option:
 
     * `:brutal_kill` - the child process is unconditionally terminated
-      using `exit(child, :kill)`
+      using `Process.exit(child, :kill)`
 
     * `:infinity` - if the child process is a supervisor, this is a mechanism
       to give the subtree enough time to shutdown; it can also be used with
@@ -129,7 +129,7 @@ defmodule Supervisor.Spec do
   supervise and a set of options.
 
   Returns a tuple containing the supervisor specification. This tuple can be
-  used as the return value of the `init/1` callback when implementing a
+  used as the return value of the `c:init/1` callback when implementing a
   module-based supervisor.
 
   ## Examples

lib/elixir/lib/system.ex

@@ -34,7 +34,6 @@ defmodule System do
       next = System.monotonic_time()
       diff = next - prev
 
-
   Generally speaking, the VM provides three time measurements:
 
     * `os_time/0` - the time reported by the OS. This time may be
@@ -54,7 +53,7 @@ defmodule System do
   the time, all calculations are done in the `:native` unit, to
   avoid loss of precision, with `convert_time_unit/3` being
   invoked at the end to convert to a specific time unit like
-  milliseconds or microseconds. See the `t:time_unit/0` type for
+  `:millisecond` or `:microsecond`. See the `t:time_unit/0` type for
   more information.
 
   For a more complete rundown on the VM support for different
@@ -66,25 +65,30 @@ defmodule System do
   @typedoc """
   The time unit to be passed to functions like `monotonic_time/1` and others.
 
-  The `:seconds`, `:milliseconds`, `:microseconds` and `:nanoseconds` time
+  The `:second`, `:millisecond`, `:microsecond` and `:nanosecond` time
   units controls the return value of the functions that accept a time unit.
 
   A time unit can also be a strictly positive integer. In this case, it
   represents the "parts per second": the time will be returned in `1 /
-  parts_per_second` seconds. For example, using the `:milliseconds` time unit
+  parts_per_second` seconds. For example, using the `:millisecond` time unit
   is equivalent to using `1000` as the time unit (as the time will be returned
   in 1/1000 seconds - milliseconds).
 
-  Keep in mind the Erlang API will use `:milli_seconds`, `:micro_seconds`
-  and `:nano_seconds` as time units although Elixir normalizes their spelling
-  to match the SI convention.
+  Keep in mind the Erlang API prior to version 19.1 will use `:milli_seconds`,
+  `:micro_seconds` and `:nano_seconds` as time units although Elixir normalizes
+  their spelling to match the SI convention.
   """
   @type time_unit ::
-    :seconds
+    :second
+    | :millisecond
+    | :microsecond
+    | :nanosecond
+    | pos_integer
+    # TODO: Deprecate these in Elixir 2.0
+    | :seconds
     | :milliseconds
     | :microseconds
     | :nanoseconds
-    | pos_integer
 
   @base_dir     :filename.join(__DIR__, "../../..")
   @version_file :filename.join(@base_dir, "VERSION")
@@ -113,11 +117,19 @@ defmodule System do
   # Tries to run "git rev-parse --short HEAD". In the case of success returns
   # the short revision hash. If that fails, returns an empty string.
   defmacrop get_revision do
-    :os.cmd('git rev-parse --short HEAD 2> /dev/null')
+    null =
+      case :os.type do
+        {:win32, _} -> 'NUL'
+        _ -> '/dev/null'
+      end
+
+    'git rev-parse --short HEAD 2> '
+    |> Kernel.++(null)
+    |> :os.cmd()
     |> strip
   end
 
-  defp revision, do: get_revision
+  defp revision, do: get_revision()
 
   # Get the date at compilation time.
   defmacrop get_date do
@@ -145,7 +157,7 @@ defmodule System do
   Returns Elixir's version as binary.
   """
   @spec version() :: String.t
-  def version, do: get_version
+  def version, do: get_version()
 
   @doc """
   Elixir build information.
@@ -154,22 +166,21 @@ defmodule System do
   """
   @spec build_info() :: map
   def build_info do
-    %{build:    build,
-      date:     get_date,
-      revision: revision,
-      version:  version,
-      }
+    %{build:    build(),
+      date:     get_date(),
+      revision: revision(),
+      version:  version()}
   end
 
   # Returns a string of the build info
   defp build do
-    {:ok, v} = Version.parse(version)
+    {:ok, v} = Version.parse(version())
 
     cond do
-      ([] == v.pre) or ("" == revision) ->
-        version
+      ([] == v.pre) or ("" == revision()) ->
+        version()
       true ->
-        "#{version} (#{revision})"
+        "#{version()} (#{revision()})"
     end
   end
 
@@ -222,7 +233,7 @@ defmodule System do
   Returns the current working directory or raises `RuntimeError`.
   """
   def cwd! do
-    cwd ||
+    cwd() ||
       raise RuntimeError, message: "could not get a current working directory, the current location is not accessible"
   end
 
@@ -242,7 +253,7 @@ defmodule System do
   instead of returning `nil` if no user home is set.
   """
   def user_home! do
-    user_home ||
+    user_home() ||
       raise RuntimeError, message: "could not find the user home, please set the HOME environment variable"
   end
 
@@ -275,7 +286,7 @@ defmodule System do
   instead of returning `nil` if no temp dir is set.
   """
   def tmp_dir! do
-    tmp_dir ||
+    tmp_dir() ||
       raise RuntimeError, message: "could not get a writable temporary directory, " <>
                                    "please set the TMPDIR environment variable"
   end
@@ -477,6 +488,13 @@ defmodule System do
   This function returns a tuple containing the collected result
   and the command exit status.
 
+  Internally, this function uses a `Port` for interacting with the
+  outside world. However, if you plan to run a long-running program,
+  ports guarantee stdin/stdout devices will be closed but it does not
+  automatically terminate the problem. The documentation for the
+  `Port` module describes this problem and possible solutions under
+  the "Zombie processes" section.
+
   ## Examples
 
       iex> System.cmd "echo", ["hello"]
@@ -494,7 +512,7 @@ defmodule System do
     * `:into` - injects the result into the given collectable, defaults to `""`
     * `:cd` - the directory to run the command in
     * `:env` - an enumerable of tuples containing environment key-value as binary
-    * `:arg0` - set the command arg0
+    * `:arg0` - sets the command arg0
     * `:stderr_to_stdout` - redirects stderr to stdout when `true`
     * `:parallelism` - when `true`, the VM will schedule port tasks to improve
       parallelism in the system. If set to `false`, the VM will try to perform
@@ -663,12 +681,12 @@ defmodule System do
   The result is rounded via the floor function.
 
   `convert_time_unit/3` accepts an additional time unit (other than the
-  ones in the `time_unit` type) called `:native`. `:native` is the time
+  ones in the `t:time_unit/0` type) called `:native`. `:native` is the time
   unit used by the Erlang runtime system. It's determined when the runtime
   starts and stays the same until the runtime is stopped. To determine what
   the `:native` unit amounts to in a system, you can call this function to
   convert 1 second to the `:native` time unit (i.e.,
-  `System.convert_time_unit(1, :seconds, :native)`).
+  `System.convert_time_unit(1, :second, :native)`).
   """
   @spec convert_time_unit(integer, time_unit | :native, time_unit | :native) :: integer
   def convert_time_unit(time, from_unit, to_unit) do
@@ -730,6 +748,30 @@ defmodule System do
     :os.system_time(normalize_time_unit(unit))
   end
 
+  @doc """
+  Returns the OTP release number.
+  """
+  @spec otp_release :: String.t
+  def otp_release do
+    :erlang.list_to_binary :erlang.system_info(:otp_release)
+  end
+
+  @doc """
+  Returns the number of schedulers in the VM.
+  """
+  @spec schedulers :: pos_integer
+  def schedulers do
+    :erlang.system_info(:schedulers)
+  end
+
+  @doc """
+  Returns the number of schedulers online in the VM.
+  """
+  @spec schedulers_online :: pos_integer
+  def schedulers_online do
+    :erlang.system_info(:schedulers_online)
+  end
+
   @doc """
   Generates and returns an integer that is unique in the current runtime
   instance.
@@ -760,6 +802,18 @@ defmodule System do
 
   defp normalize_time_unit(:native),
     do: :native
+
+  # TODO: Remove these mappings once Elixir requires Erlang/OTP 19.1
+  defp normalize_time_unit(:second),
+    do: :seconds
+  defp normalize_time_unit(:millisecond),
+    do: :milli_seconds
+  defp normalize_time_unit(:microsecond),
+    do: :micro_seconds
+  defp normalize_time_unit(:nanosecond),
+    do: :nano_seconds
+
+  # TODO: Warn on Elixir 1.5
   defp normalize_time_unit(:seconds),
     do: :seconds
   defp normalize_time_unit(:milliseconds),
@@ -768,6 +822,7 @@ defmodule System do
     do: :micro_seconds
   defp normalize_time_unit(:nanoseconds),
     do: :nano_seconds
+
   defp normalize_time_unit(unit) when is_integer(unit) and unit > 0,
     do: unit
 
@@ -779,8 +834,8 @@ defmodule System do
 
   defp normalize_time_unit(other) do
     raise ArgumentError,
-      "unsupported time unit. Expected :seconds, :milliseconds, " <>
-      ":microseconds, :nanoseconds, or a positive integer, " <>
+      "unsupported time unit. Expected :second, :millisecond, " <>
+      ":microsecond, :nanosecond, or a positive integer, " <>
       "got #{inspect other}"
   end
 end

lib/elixir/lib/task.ex

@@ -33,7 +33,7 @@ defmodule Task do
 
   There are two important things to consider when using `async`:
 
-    1. If you are using async tasks, you must await a reply
+    1. If you are using async tasks, you **must await** a reply
        as they are *always* sent. If you are not expecting a reply,
        consider using `Task.start_link/1` detailed below.
 
@@ -56,28 +56,27 @@ defmodule Task do
 
   ## Supervised tasks
 
-  It is also possible to spawn a task under a supervisor
-  with `start_link/1` and `start_link/3`:
-
-      Task.start_link(fn -> IO.puts "ok" end)
-
-  Such tasks can be mounted in your supervision tree as:
+  It is also possible to spawn a task under a supervisor:
 
       import Supervisor.Spec
 
       children = [
+        #
         worker(Task, [fn -> IO.puts "ok" end])
       ]
 
+  Internally the supervisor will invoke `Task.start_link/1`.
+
   Since these tasks are supervised and not directly linked to
   the caller, they cannot be awaited on. Note `start_link/1`,
   unlike `async/1`, returns `{:ok, pid}` (which is
   the result expected by supervision trees).
 
   By default, most supervision strategies will try to restart
-  a worker after it exits regardless of the reason. If you design the
-  task to terminate normally (as in the example with `IO.puts/2` above),
-  consider passing `restart: :transient` in the options to `worker/3`.
+  a worker after it exits regardless of the reason. If you design
+  the task to terminate normally (as in the example with `IO.puts/2`
+  above), consider passing `restart: :transient` in the options
+  to `Supervisor.Spec.worker/3`.
 
   ## Dynamically supervised tasks
 
@@ -127,8 +126,8 @@ defmodule Task do
       Task.Supervisor.async({MyApp.DistSupervisor, :remote@local},
                             MyMod, :my_fun, [arg1, arg2, arg3])
 
-  Note that, when working with distributed tasks, one should use the `async/4` function
-  that expects explicit module, function and arguments, instead of `async/2` that
+  Note that, when working with distributed tasks, one should use the `Task.Supervisor.async/4` function
+  that expects explicit module, function and arguments, instead of `Task.Supervisor.async/2` that
   works with anonymous functions. That's because anonymous functions expect
   the same module version to exist on all involved nodes. Check the `Agent` module
   documentation for more information on distributed processes as the limitations
@@ -165,7 +164,7 @@ defmodule Task do
   """
   @spec start_link(module, atom, [term]) :: {:ok, pid}
   def start_link(mod, fun, args) do
-    Task.Supervised.start_link(get_info(self), {mod, fun, args})
+    Task.Supervised.start_link(get_info(self()), {mod, fun, args})
   end
 
   @doc """
@@ -189,7 +188,7 @@ defmodule Task do
   """
   @spec start(module, atom, [term]) :: {:ok, pid}
   def start(mod, fun, args) do
-    Task.Supervised.start(get_info(self), {mod, fun, args})
+    Task.Supervised.start(get_info(self()), {mod, fun, args})
   end
 
   @doc """
@@ -257,7 +256,7 @@ defmodule Task do
       process immune to not only exits from the task but from
       any other processes.
 
-      Moreover, even when trapping exists, calling `await` will
+      Moreover, even when trapping exits, calling `await` will
       still exit if the task has terminated without sending its
       result back.
 
@@ -282,6 +281,80 @@ defmodule Task do
     %Task{pid: pid, ref: ref, owner: owner}
   end
 
+  @doc """
+  Returns a stream that runs the given `module`, `function` and `args`
+  concurrently on each item in `enumerable`.
+
+  Each item will be prepended to the given `args` and processed by its
+  own task. The tasks will be linked to an intermediate process that is
+  then linked to the current process. This means a failure in a task
+  terminates the current process and a failure in the current process
+  terminates all tasks.
+
+  When streamed, each task will emit `{:ok, val}` upon successful
+  completion or `{:exit, val}` if the caller is trapping exits. Results
+  are emitted in the same order as the original `enumerable`.
+
+  The level of concurrency can be controlled via the `:max_concurrency`
+  option and defaults to `System.schedulers_online/0`. The timeout
+  can also be given as option and defaults to 5000 and it defaults to
+  the maximum amount of time to wait without a task reply.
+
+  Finally, consider using `Task.Supervisor.async_stream/6` to start tasks
+  under a supervisor. If you find yourself trapping exits to handle exits
+  inside the async stream, consider using `Task.Supervisor.async_stream_nolink/6`
+  to start tasks that are not linked to the current process.
+
+  ## Options
+
+    * `:max_concurrency` - sets the maximum number of tasks to run
+      at the same time. Defaults to `System.schedulers_online/0`.
+    * `:timeout` - the maximum amount of time to wait without
+      receiving a task reply (across all running tasks).
+      Defaults to `5000`.
+
+  ## Example
+
+  Let's build a stream and then enumerate it:
+
+      stream = Task.async_stream(collection, Mod, :expensive_fun, [])
+      Enum.to_list(stream)
+
+  The concurrency can be increased or decreased using the `:max_concurrency`
+  option. For example, if the tasks are IO heavy, the value can be increased:
+
+      max_concurrency = System.schedulers_online * 2
+      stream = Task.async_stream(collection, Mod, :expensive_fun, [], max_concurrency: max_concurrency)
+      Enum.to_list(stream)
+
+  """
+  @spec async_stream(Enumerable.t, module, atom, [term], Keyword.t) :: Enumerable.t
+  def async_stream(enumerable, module, function, args, options \\ [])
+      when is_atom(module) and is_atom(function) and is_list(args) do
+    build_stream(enumerable, {module, function, args}, options)
+  end
+
+  @doc """
+  Returns a stream that runs the given `function` concurrently on each
+  item in `enumerable`.
+
+  Each `enumerable` item is passed as argument to the `function` and
+  processed by its own task. The tasks will be linked to the current
+  process, similar to `async/1`.
+
+  See `async_stream/5` for discussion and examples.
+  """
+  @spec async_stream(Enumerable.t, (term -> term), Keyword.t) :: Enumerable.t
+  def async_stream(enumerable, fun, options \\ []) when is_function(fun, 1) do
+    build_stream(enumerable, fun, options)
+  end
+
+  defp build_stream(enumerable, fun, options) do
+    &Task.Supervised.stream(enumerable, &1, &2, fun, options, fn owner, mfa ->
+      {:link, Task.Supervised.spawn_link(owner, get_info(owner), mfa)}
+    end)
+  end
+
   defp get_info(self) do
     {node(),
      case Process.info(self, :registered_name) do
@@ -291,7 +364,7 @@ defmodule Task do
   end
 
   @doc """
-  Awaits a task reply.
+  Awaits a task reply and returns it.
 
   A timeout, in milliseconds, can be given with default value
   of `5000`. In case the task process dies, this function will
@@ -314,7 +387,14 @@ defmodule Task do
 
   It is not recommended to `await` a long-running task inside an OTP
   behaviour such as `GenServer`. Instead, you should match on the message
-  coming from a task inside your `handle_info` callback.
+  coming from a task inside your `GenServer.handle_info/2` callback.
+
+  ## Examples
+
+      iex> task = Task.async(fn -> 1 + 1 end)
+      iex> Task.await(task)
+      2
+
   """
   @spec await(t, timeout) :: term | no_return
   def await(task, timeout \\ 5000)
@@ -365,7 +445,7 @@ defmodule Task do
     nil
   end
 
-  @doc """
+  @doc ~S"""
   Temporarily blocks the current process waiting for a task reply.
 
   Returns `{:ok, reply}` if the reply is received, `nil` if
@@ -388,6 +468,21 @@ defmodule Task do
   monitor's `:DOWN` message is in the message queue. If it has been
   demonitored or the message already received, this function will wait
   for the duration of the timeout awaiting the message.
+
+  If you intend to shut the task down if it has not responded within `timeout`
+  milliseconds, you should chain this together with `shutdown/1`, like so:
+
+      case Task.yield(task, timeout) || Task.shutdown(task) do
+        {:ok, result} ->
+          result
+        nil ->
+          Logger.warn "Failed to get a result in #{timeout}ms"
+          nil
+      end
+
+  That ensures that if the task completes after the `timeout` but before `shutdown/1`
+  has been called, you will still get the result, since `shutdown/1` is designed to
+  handle this case and return the result.
   """
   @spec yield(t, timeout) :: {:ok, term} | {:exit, term} | nil
   def yield(task, timeout \\ 5_000)
@@ -440,7 +535,7 @@ defmodule Task do
       tasks =
         for i <- 1..10 do
           Task.async(fn ->
-            :timer.sleep(i * 1000)
+            Process.sleep(i * 1000)
             i
           end)
         end
@@ -529,9 +624,9 @@ defmodule Task do
   `:shutdown` to shutdown all of its linked processes, including tasks, that
   are not trapping exits without generating any log messages.
 
-  This function assumes the task's monitor is still active or the monitor's
-  `:DOWN` message is in the message queue. If it has been demonitored, or the
-  message already received, this function will block forever awaiting the message.
+  If a task's monitor has already been demonitored or received  and there is not
+  a response waiting in the message queue this function will return
+  `{:exit, :noproc}` as the result or exit reason can not be determined.
   """
   @spec shutdown(t, timeout | :brutal_kill) :: {:ok, term} | {:exit, term} | nil
   def shutdown(task, shutdown \\ 5_000)
@@ -545,9 +640,10 @@ defmodule Task do
   end
 
   def shutdown(%Task{pid: pid} = task, :brutal_kill) do
+    mon = Process.monitor(pid)
     exit(pid, :kill)
 
-    case shutdown_receive(task, :brutal_kill, :infinity) do
+    case shutdown_receive(task, mon, :brutal_kill, :infinity) do
       {:down, proc, :noconnection} ->
         exit({reason(:noconnection, proc), {__MODULE__, :shutdown, [task, :brutal_kill]}})
       {:down, _, reason} ->
@@ -558,8 +654,9 @@ defmodule Task do
   end
 
   def shutdown(%Task{pid: pid} = task, timeout) do
+    mon = Process.monitor(pid)
     exit(pid, :shutdown)
-    case shutdown_receive(task, :shutdown, timeout) do
+    case shutdown_receive(task, mon, :shutdown, timeout) do
       {:down, proc, :noconnection} ->
         exit({reason(:noconnection, proc), {__MODULE__, :shutdown, [task, timeout]}})
       {:down, _, reason} ->
@@ -597,18 +694,24 @@ defmodule Task do
     end
   end
 
-  defp shutdown_receive(%{ref: ref} = task, type, timeout) do
+  defp shutdown_receive(%{ref: ref} = task, mon, type, timeout) do
     receive do
-      {:DOWN, ^ref, _, _, :shutdown} when type in [:shutdown, :timeout_kill] ->
+      {:DOWN, ^mon, _, _, :shutdown} when type in [:shutdown, :timeout_kill] ->
+        Process.demonitor(ref, [:flush])
         flush_reply(ref)
-      {:DOWN, ^ref, _, _, :killed} when type == :brutal_kill ->
+      {:DOWN, ^mon, _, _, :killed} when type == :brutal_kill ->
+        Process.demonitor(ref, [:flush])
         flush_reply(ref)
-      {:DOWN, ^ref, _, proc, reason} ->
+      {:DOWN, ^mon, _, proc, :noproc} ->
+        reason = flush_noproc(ref, proc, type)
+        flush_reply(ref) || reason
+      {:DOWN, ^mon, _, proc, reason} ->
+        Process.demonitor(ref, [:flush])
         flush_reply(ref) || {:down, proc, reason}
     after
       timeout ->
         Process.exit(task.pid, :kill)
-        shutdown_receive(task, :timeout_kill, :infinity)
+        shutdown_receive(task, mon, :timeout_kill, :infinity)
     end
   end
 
@@ -620,6 +723,21 @@ defmodule Task do
     end
   end
 
+  defp flush_noproc(ref, proc, type) do
+    receive do
+      {:DOWN, ^ref, _, _, :shutdown} when type in [:shutdown, :timeout_kill] ->
+        nil
+      {:DOWN, ^ref, _, _, :killed} when type == :brutal_kill ->
+        nil
+      {:DOWN, ^ref, _, _, reason} ->
+        {:down, proc, reason}
+    after
+      0 ->
+        Process.demonitor(ref, [:flush])
+        {:down, proc, :noproc}
+    end
+  end
+
   defp invalid_owner_error(task) do
     "task #{inspect task} must be queried from the owner but was queried from #{inspect self()}"
   end

lib/elixir/lib/task/supervised.ex

@@ -1,6 +1,5 @@
 defmodule Task.Supervised do
   @moduledoc false
-
   @ref_timeout 5_000
 
   def start(info, fun) do
@@ -11,29 +10,21 @@ defmodule Task.Supervised do
     {:ok, :proc_lib.spawn_link(__MODULE__, :noreply, [info, fun])}
   end
 
-  def start_link(caller, link, info, fun) do
-    {:ok, spawn_link(caller, link, info, fun)}
+  def start_link(caller, monitor, info, fun) do
+    {:ok, spawn_link(caller, monitor, info, fun)}
   end
 
-  def spawn_link(caller, link \\ :nolink, info, fun) do
-    :proc_lib.spawn_link(__MODULE__, :reply, [caller, link, info, fun])
+  def spawn_link(caller, monitor \\ :nomonitor, info, fun) do
+    :proc_lib.spawn_link(__MODULE__, :reply, [caller, monitor, info, fun])
   end
 
-  def reply(caller, link, info, mfa) do
+  def reply(caller, monitor, info, mfa) do
     initial_call(mfa)
-    case link do
-      :link ->
-        try do
-          Process.link(caller)
-        catch
-          :error, :noproc ->
-            exit({:shutdown, :noproc})
-        end
-        reply(caller, nil, @ref_timeout, info, mfa)
+    case monitor do
       :monitor ->
         mref = Process.monitor(caller)
         reply(caller, mref, @ref_timeout, info, mfa)
-      :nolink ->
+      :nomonitor ->
         reply(caller, nil, :infinity, info, mfa)
     end
   end
@@ -105,9 +96,9 @@ defmodule Task.Supervised do
   end
 
   defp exit(_info, _mfa, _log_reason, reason)
-      when reason == :normal
-      when reason == :shutdown
-      when tuple_size(reason) == 2 and elem(reason, 0) == :shutdown do
+       when reason == :normal
+       when reason == :shutdown
+       when tuple_size(reason) == 2 and elem(reason, 0) == :shutdown do
     exit(reason)
   end
 
@@ -120,7 +111,7 @@ defmodule Task.Supervised do
       '** When function  == ~p~n' ++
       '**      arguments == ~p~n' ++
       '** Reason for termination == ~n' ++
-      '** ~p~n', [self, get_from(info), fun, args, get_reason(log_reason)])
+      '** ~p~n', [self(), get_from(info), fun, args, get_reason(log_reason)])
 
     exit(reason)
   end
@@ -132,7 +123,7 @@ defmodule Task.Supervised do
   defp get_running({mod, fun, args}), do: {:erlang.make_fun(mod, fun, length(args)), args}
 
   defp get_reason({:undef, [{mod, fun, args, _info} | _] = stacktrace} = reason)
-  when is_atom(mod) and is_atom(fun) do
+       when is_atom(mod) and is_atom(fun) do
     cond do
       :code.is_loaded(mod) === false ->
         {:"module could not be loaded", stacktrace}
@@ -148,4 +139,221 @@ defmodule Task.Supervised do
   defp get_reason(reason) do
     reason
   end
+
+  ## Stream
+
+  def stream(enumerable, acc, reducer, mfa, options, spawn) do
+    next = &Enumerable.reduce(enumerable, &1, fn x, acc -> {:suspend, [x | acc]} end)
+    max_concurrency = Keyword.get(options, :max_concurrency, System.schedulers_online)
+    timeout = Keyword.get(options, :timeout, 5000)
+    parent = self()
+
+    # Start a process responsible for translating down messages.
+    {:trap_exit, trap_exit} =
+      Process.info(self(), :trap_exit)
+    {monitor_pid, monitor_ref} =
+      Process.spawn(fn -> stream_monitor(parent, mfa, spawn, trap_exit) end, [:link, :monitor])
+
+    send(monitor_pid, {parent, monitor_ref})
+
+    stream_reduce(acc, max_concurrency, 0, 0, %{}, next,
+                  reducer, monitor_pid, monitor_ref, timeout)
+  end
+
+  defp stream_reduce({:halt, acc}, _max, _spawned, _delivered, _waiting, next,
+                     _reducer, monitor_pid, monitor_ref, timeout) do
+    stream_close(monitor_pid, monitor_ref, timeout)
+    is_function(next) && next.({:halt, []})
+    {:halted, acc}
+  end
+
+  defp stream_reduce({:suspend, acc}, max, spawned, delivered, waiting, next,
+                     reducer, monitor_pid, monitor_ref, timeout) do
+    {:suspended, acc, &stream_reduce(&1, max, spawned, delivered, waiting, next,
+                                     reducer, monitor_pid, monitor_ref, timeout)}
+  end
+
+  # All spawned, all delivered, next is done.
+  defp stream_reduce({:cont, acc}, _max, spawned, delivered, _waiting, next,
+                     _reducer, monitor_pid, monitor_ref, timeout)
+       when spawned == delivered and next == :done do
+    stream_close(monitor_pid, monitor_ref, timeout)
+    {:done, acc}
+  end
+
+  # No more tasks to spawned because max == 0 or next is done.
+  defp stream_reduce({:cont, acc}, max, spawned, delivered, waiting, next,
+                     reducer, monitor_pid, monitor_ref, timeout)
+       when max == 0
+       when next == :done do
+    receive do
+      {{^monitor_ref, position}, value} ->
+        %{^position => {pid, :running}} = waiting
+        waiting = Map.put(waiting, position, {pid, {:ok, value}})
+        stream_reduce({:cont, acc}, max, spawned, delivered, waiting, next,
+                      reducer, monitor_pid, monitor_ref, timeout)
+      {:down, {^monitor_ref, position}, reason} ->
+        waiting =
+          case waiting do
+            %{^position => {_, {:ok, _} = ok}} -> Map.put(waiting, position, {nil, ok})
+            %{^position => {_, :running}} -> Map.put(waiting, position, {nil, {:exit, reason}})
+          end
+        stream_deliver({:cont, acc}, max + 1, spawned, delivered, waiting, next,
+                       reducer, monitor_pid, monitor_ref, timeout)
+      {:DOWN, ^monitor_ref, _, ^monitor_pid, reason} ->
+        stream_cleanup_inbox(monitor_pid, monitor_ref)
+        exit({reason, {__MODULE__, :stream, [timeout]}})
+    after
+      timeout ->
+        stream_close(monitor_pid, monitor_ref, timeout)
+        exit({:timeout, {__MODULE__, :stream, [timeout]}})
+    end
+  end
+
+  defp stream_reduce({:cont, acc}, max, spawned, delivered, waiting, next,
+                     reducer, monitor_pid, monitor_ref, timeout) do
+    try do
+      next.({:cont, []})
+    catch
+      kind, reason ->
+        stacktrace = System.stacktrace
+        stream_close(monitor_pid, monitor_ref, timeout)
+        :erlang.raise(kind, reason, stacktrace)
+    else
+      {:suspended, [value], next} ->
+        waiting = stream_spawn(value, spawned, waiting, monitor_pid, monitor_ref, timeout)
+        stream_reduce({:cont, acc}, max - 1, spawned + 1, delivered, waiting, next,
+                      reducer, monitor_pid, monitor_ref, timeout)
+      {_, [value]} ->
+        waiting = stream_spawn(value, spawned, waiting, monitor_pid, monitor_ref, timeout)
+        stream_reduce({:cont, acc}, max - 1, spawned + 1, delivered, waiting, :done,
+                      reducer, monitor_pid, monitor_ref, timeout)
+      {_, []} ->
+        stream_reduce({:cont, acc}, max, spawned, delivered, waiting, :done,
+                      reducer, monitor_pid, monitor_ref, timeout)
+    end
+  end
+
+  defp stream_deliver({:suspend, acc}, max, spawned, delivered, waiting, next,
+                      reducer, monitor_pid, monitor_ref, timeout) do
+    {:suspended, acc, &stream_deliver(&1, max, spawned, delivered, waiting, next,
+                                      reducer, monitor_pid, monitor_ref, timeout)}
+  end
+  defp stream_deliver({:halt, acc}, max, spawned, delivered, waiting, next,
+                      reducer, monitor_pid, monitor_ref, timeout) do
+    stream_reduce({:halt, acc}, max, spawned, delivered, waiting, next,
+                  reducer, monitor_pid, monitor_ref, timeout)
+  end
+  defp stream_deliver({:cont, acc}, max, spawned, delivered, waiting, next,
+                      reducer, monitor_pid, monitor_ref, timeout) do
+    case waiting do
+      %{^delivered => {nil, reply}} ->
+        try do
+          reducer.(reply, acc)
+        catch
+          kind, reason ->
+            stacktrace = System.stacktrace
+            is_function(next) && next.({:halt, []})
+            stream_close(monitor_pid, monitor_ref, timeout)
+            :erlang.raise(kind, reason, stacktrace)
+        else
+          pair ->
+            stream_deliver(pair, max, spawned, delivered + 1, Map.delete(waiting, delivered), next,
+                           reducer, monitor_pid, monitor_ref, timeout)
+        end
+      %{} ->
+        stream_reduce({:cont, acc}, max, spawned, delivered, waiting, next,
+                      reducer, monitor_pid, monitor_ref, timeout)
+    end
+  end
+
+  defp stream_close(monitor_pid, monitor_ref, timeout) do
+    send(monitor_pid, {:stop, monitor_ref})
+    receive do
+      {:DOWN, ^monitor_ref, _, _, :normal} ->
+        stream_cleanup_inbox(monitor_pid, monitor_ref)
+        :ok
+      {:DOWN, ^monitor_ref, _, _, reason} ->
+        stream_cleanup_inbox(monitor_pid, monitor_ref)
+        exit({reason, {__MODULE__, :stream, [timeout]}})
+    end
+  end
+
+  defp stream_cleanup_inbox(monitor_pid, monitor_ref) do
+    receive do
+      {:EXIT, ^monitor_pid, _} -> stream_cleanup_inbox(monitor_ref)
+    after
+      0 -> stream_cleanup_inbox(monitor_ref)
+    end
+  end
+
+  defp stream_cleanup_inbox(monitor_ref) do
+    receive do
+      {{^monitor_ref, _}, _} ->
+        stream_cleanup_inbox(monitor_ref)
+      {:down, {^monitor_ref, _}, _} ->
+        stream_cleanup_inbox(monitor_ref)
+    after
+      0 ->
+        :ok
+    end
+  end
+
+  defp stream_mfa({mod, fun, args}, arg), do: {mod, fun, [arg | args]}
+  defp stream_mfa(fun, arg), do: {:erlang, :apply, [fun, [arg]]}
+
+  defp stream_spawn(value, spawned, waiting, monitor_pid, monitor_ref, timeout) do
+    send(monitor_pid, {:spawn, spawned, value})
+
+    receive do
+      {:spawned, {^monitor_ref, ^spawned}, pid} ->
+        send(pid, {self(), {monitor_ref, spawned}})
+        Map.put(waiting, spawned, {pid, :running})
+      {:DOWN, ^monitor_ref, _, ^monitor_pid, reason} ->
+        stream_cleanup_inbox(monitor_pid, monitor_ref)
+        exit({reason, {__MODULE__, :stream, [timeout]}})
+    end
+  end
+
+  defp stream_monitor(parent_pid, mfa, spawn, trap_exit) do
+    Process.flag(:trap_exit, trap_exit)
+    parent_ref = Process.monitor(parent_pid)
+    receive do
+      {^parent_pid, monitor_ref} ->
+        stream_monitor(parent_pid, parent_ref, mfa, spawn, monitor_ref, %{})
+      {:DOWN, ^parent_ref, _, _, reason} ->
+        exit(reason)
+    end
+  end
+
+  defp stream_monitor(parent_pid, parent_ref, mfa, spawn, monitor_ref, counters) do
+    receive do
+      {:spawn, counter, value} ->
+        {type, pid} = spawn.(parent_pid, stream_mfa(mfa, value))
+        ref = Process.monitor(pid)
+        send(parent_pid, {:spawned, {monitor_ref, counter}, pid})
+        counters = Map.put(counters, ref, {counter, type, pid})
+        stream_monitor(parent_pid, parent_ref, mfa, spawn, monitor_ref, counters)
+      {:stop, ^monitor_ref} ->
+        Process.flag(:trap_exit, true)
+        for {ref, {_counter, _, pid}} <- counters do
+          Process.exit(pid, :kill)
+          receive do
+            {:DOWN, ^ref, _, _, _} -> :ok
+          end
+        end
+        exit(:normal)
+      {:DOWN, ^parent_ref, _, _, reason} ->
+        for {_ref, {_counter, :link, pid}} <- counters do
+          Process.exit(pid, reason)
+        end
+        exit(reason)
+      {:DOWN, ref, _, _, reason} ->
+        {{counter, _, _}, counters} = Map.pop(counters, ref)
+        send(parent_pid, {:down, {monitor_ref, counter}, reason})
+        stream_monitor(parent_pid, parent_ref, mfa, spawn, monitor_ref, counters)
+      {:EXIT, _, _} ->
+        stream_monitor(parent_pid, parent_ref, mfa, spawn, monitor_ref, counters)
+    end
+  end
 end

lib/elixir/lib/task/supervisor.ex

@@ -4,12 +4,14 @@ defmodule Task.Supervisor do
 
   This module defines a supervisor which can be used to dynamically
   supervise tasks. Behind the scenes, this module is implemented as a
-  `:simple_one_for_one` supervisor where the workers are temporary
-  (i.e. they are not restarted after they die).
+  `:simple_one_for_one` supervisor where the workers are temporary by
+  default (that is, they are not restarted after they die; read the docs
+  for `start_link/1` for more information on choosing the restart
+  strategy).
 
   See the `Task` module for more information.
 
-  ## Name Registration
+  ## Name registration
 
   A `Task.Supervisor` is bound to the same name registration rules as a
   `GenServer`. Read more about them in the `GenServer` docs.
@@ -65,7 +67,7 @@ defmodule Task.Supervisor do
   """
   @spec async(Supervisor.supervisor, module, atom, [term]) :: Task.t
   def async(supervisor, module, fun, args) do
-    do_async(supervisor, module, fun, args, :link)
+    do_async(supervisor, :link, module, fun, args)
   end
 
   @doc """
@@ -79,7 +81,7 @@ defmodule Task.Supervisor do
 
   If you create a task using `async_nolink` inside an OTP behaviour
   like `GenServer`, you should match on the message coming from the
-  task inside your `handle_info` callback.
+  task inside your `GenServer.handle_info/2` callback.
 
   The reply sent by the task will be in the format `{ref, result}`,
   where `ref` is the monitor reference held by the task struct
@@ -104,7 +106,99 @@ defmodule Task.Supervisor do
   """
   @spec async_nolink(Supervisor.supervisor, module, atom, [term]) :: Task.t
   def async_nolink(supervisor, module, fun, args) do
-    do_async(supervisor, module, fun, args, :monitor)
+    do_async(supervisor, :nolink, module, fun, args)
+  end
+
+  @doc """
+  Returns a stream that runs the given `module`, `function` and `args`
+  concurrently on each item in `enumerable`.
+
+  Each item will be appended to the given `args` and processed by its
+  own task. The tasks will be spawned under the given `supervisor` and
+  linked to the current process, similar to `async/4`.
+
+  When streamed, each task will emit `{:ok, val}` upon successful
+  completion or `{:exit, val}` if the caller is trapping exits. Results
+  are emitted in the same order as the original `enumerable`.
+
+  The level of concurrency can be controlled via the `:max_concurrency`
+  option and defaults to `System.schedulers_online/0`. The timeout
+  can also be given as option and defaults to 5000 and it defaults to
+  the maximum amount of time to wait without a task reply.
+
+  Finally, if you find yourself trapping exits to handle exits inside
+  the async stream, consider using `async_stream_nolink/6` to start tasks
+  that are not linked to the current process.
+
+  ## Options
+
+    * `:max_concurrency` - sets the maximum number of tasks to run
+      at the same time. Defaults to `System.schedulers_online/0`.
+    * `:timeout` - the maximum amount of time to wait without
+      receiving a task reply (across all running tasks).
+      Defaults to `5000`.
+
+  ## Examples
+
+  Let's build a stream and then enumerate it:
+
+      stream = Task.Supervisor.async_stream(MySupervisor, collection, Mod, :expensive_fun, [])
+      Enum.to_list(stream)
+  """
+  @spec async_stream(Supervisor.supervisor, Enumerable.t, module, atom, [term], Keyword.t) ::
+        Enumerable.t
+  def async_stream(supervisor, enumerable, module, function, args, options \\ [])
+      when is_atom(module) and is_atom(function) and is_list(args) do
+    build_stream(supervisor, :link, enumerable, {module, function, args}, options)
+  end
+
+  @doc """
+  Returns a stream that runs the given `function` concurrently on each
+  item in `enumerable`.
+
+  Each item will be appended to the given `args` and processed by its
+  own task. The tasks will be spawned under the given `supervisor` and
+  are linked to the current process, similar to `async/2`.
+
+  See `async_stream/6` for discussion and examples.
+  """
+  @spec async_stream(Supervisor.supervisor, Enumerable.t, (term -> term), Keyword.t) ::
+        Enumerable.t
+  def async_stream(supervisor, enumerable, fun, options \\ []) when is_function(fun, 1) do
+    build_stream(supervisor, :link, enumerable, fun, options)
+  end
+
+  @doc """
+  Returns a stream that runs the given `module`, `function` and `args`
+  concurrently on each item in `enumerable`.
+
+  Each item will be appended to the given `args` and processed by its
+  own task. The tasks will be spawned under the given `supervisor` and
+  are not linked to the current process, similar to `async_nolink/4`.
+
+  See `async_stream/6` for discussion and examples.
+  """
+  @spec async_stream_nolink(Supervisor.supervisor, Enumerable.t, module, atom, [term], Keyword.t) ::
+        Enumerable.t
+  def async_stream_nolink(supervisor, enumerable, module, function, args, options \\ [])
+      when is_atom(module) and is_atom(function) and is_list(args) do
+    build_stream(supervisor, :nolink, enumerable, {module, function, args}, options)
+  end
+
+  @doc """
+  Returns a stream that runs the given `function` concurrently on each
+  item in `enumerable`.
+
+  Each item will be appended to the given `args` and processed by its
+  own task. The tasks will be spawned under the given `supervisor` and
+  are not linked to the current process, similar to `async_nolink/2`.
+
+  See `async_stream/6` for discussion and examples.
+  """
+  @spec async_stream_nolink(Supervisor.supervisor, Enumerable.t, (term -> term), Keyword.t) ::
+        Enumerable.t
+  def async_stream_nolink(supervisor, enumerable, fun, options \\ []) when is_function(fun, 1) do
+    build_stream(supervisor, :nolink, enumerable, fun, options)
   end
 
   @doc """
@@ -116,7 +210,7 @@ defmodule Task.Supervisor do
   end
 
   @doc """
-  Returns all children pids.
+  Returns all children PIDs.
   """
   @spec children(Supervisor.supervisor) :: [pid]
   def children(supervisor) do
@@ -144,7 +238,7 @@ defmodule Task.Supervisor do
   """
   @spec start_child(Supervisor.supervisor, module, atom, [term]) :: {:ok, pid}
   def start_child(supervisor, module, fun, args) do
-    Supervisor.start_child(supervisor, [get_info(self), {module, fun, args}])
+    Supervisor.start_child(supervisor, [get_info(self()), {module, fun, args}])
   end
 
   defp get_info(self) do
@@ -155,13 +249,22 @@ defmodule Task.Supervisor do
      end}
   end
 
-  defp do_async(supervisor, module, fun, args, link_type) do
+  defp do_async(supervisor, link_type, module, fun, args) do
     owner = self()
-    args = [owner, link_type, get_info(owner), {module, fun, args}]
+    args = [owner, :monitor, get_info(owner), {module, fun, args}]
     {:ok, pid} = Supervisor.start_child(supervisor, args)
     if link_type == :link, do: Process.link(pid)
     ref = Process.monitor(pid)
     send pid, {owner, ref}
     %Task{pid: pid, ref: ref, owner: owner}
   end
+
+  defp build_stream(supervisor, link_type, enumerable, fun, options) do
+    &Task.Supervised.stream(enumerable, &1, &2, fun, options, fn owner, mfa ->
+      args = [owner, :monitor, get_info(owner), mfa]
+      {:ok, pid} = Supervisor.start_child(supervisor, args)
+      if link_type == :link, do: Process.link(pid)
+      {link_type, pid}
+    end)
+  end
 end

lib/elixir/lib/tuple.ex

@@ -2,8 +2,49 @@ defmodule Tuple do
   @moduledoc """
   Functions for working with tuples.
 
-  See also `Kernel.elem/2`, `Kernel.is_tuple/1`,
-  `Kernel.put_elem/3`, and `Kernel.tuple_size/1`.
+  Tuples are ordered collection of elements; tuples can contain elements of any
+  type, and a tuple can contain elements of different types. Curly braces can be
+  used to create tuples:
+
+      iex> {}
+      {}
+      iex> {1, :two, "three"}
+      {1, :two, "three"}
+
+  Tuples store elements contiguously in memory; this means that accessing a
+  tuple element by index (which can be done through the `Kernel.elem/2`
+  function) is a constant-time operation:
+
+      iex> tuple = {1, :two, "three"}
+      iex> elem(tuple, 0)
+      1
+      iex> elem(tuple, 2)
+      "three"
+
+  Same goes for getting the tuple size (via `Kernel.tuple_size/1`):
+
+      iex> tuple_size({})
+      0
+      iex> tuple_size({1, 2, 3})
+      3
+
+  Tuples being stored contiguously in memory also means that updating a tuple
+  (for example replacing an element with `Kernel.put_elem/3`) will make a copy
+  of the whole tuple.
+
+  Tuples are not meant to be used as a "collection" type (which is also
+  suggested by the absence of an implementation of the `Enumerable` protocol for
+  tuples): they're mostly meant to be used as a fixed-size container for
+  multiple elements. For example, tuples are often used to have functions return
+  "enriched" values: a common pattern is for functions to return `{:ok, value}`
+  for successful cases and `{:error, reason}` for unsuccessful cases. For
+  example, this is exactly what `File.read/1` does: it returns `{:ok, contents}`
+  if reading the given file is successful, or `{:error, reason}` otherwise
+  (e.g., `{:error, :enoent}` if the file doesn't exist).
+
+  This module provides functions to work with tuples; some more functions to
+  work with tuples can be found in `Kernel` (`Kernel.tuple_size/1`,
+  `Kernel.elem/2`, `Kernel.put_elem/3`, and others).
   """
 
   @doc """

lib/elixir/lib/uri.ex

@@ -40,7 +40,7 @@ defmodule URI do
       nil
 
   """
-  @spec default_port(binary) :: nil | pos_integer
+  @spec default_port(binary) :: nil | non_neg_integer
   def default_port(scheme) when is_binary(scheme) do
     :elixir_config.get({:uri, scheme})
   end
@@ -57,8 +57,8 @@ defmodule URI do
   application's start callback in case you want to register
   new URIs.
   """
-  @spec default_port(binary, pos_integer) :: :ok
-  def default_port(scheme, port) when is_binary(scheme) and is_integer(port) and port > 0 do
+  @spec default_port(binary, non_neg_integer) :: :ok
+  def default_port(scheme, port) when is_binary(scheme) and is_integer(port) and port >= 0 do
     :elixir_config.put({:uri, scheme}, port)
   end
 
@@ -256,11 +256,11 @@ defmodule URI do
   Percent-escapes the given string.
 
   This function accepts a `predicate` function as an optional argument; if
-  passed, this function will be called with each character (byte) in `str` as
+  passed, this function will be called with each character (byte) in `string` as
   its argument and should return `true` if that character should not be escaped
   and left as is.
 
-  ## Example
+  ## Examples
 
       iex> URI.encode("ftp://s-ite.tld/?value=put it+й")
       "ftp://s-ite.tld/?value=put%20it+%D0%B9"
@@ -343,8 +343,8 @@ defmodule URI do
     unpercent(tail, <<acc::binary, ?\s>>, spaces)
   end
 
-  defp unpercent(<<?%, hex_1, hex_2, tail::binary>>, acc, spaces) do
-    unpercent(tail, <<acc::binary, bsl(hex_to_dec(hex_1), 4) + hex_to_dec(hex_2)>>, spaces)
+  defp unpercent(<<?%, hex1, hex2, tail::binary>>, acc, spaces) do
+    unpercent(tail, <<acc::binary, bsl(hex_to_dec(hex1), 4) + hex_to_dec(hex2)>>, spaces)
   end
   defp unpercent(<<?%, _::binary>>, _acc, _spaces), do: throw(:malformed_uri)
 
@@ -400,8 +400,8 @@ defmodule URI do
   def parse(%URI{} = uri), do: uri
 
   def parse(string) when is_binary(string) do
-    # From http://tools.ietf.org/html/rfc3986#appendix-B
-    regex = ~r/^(([a-z][a-z0-9\+\-\.]*):)?(\/\/([^\/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))?/i
+    # From https://tools.ietf.org/html/rfc3986#appendix-B
+    regex = Regex.recompile!(~r/^(([a-z][a-z0-9\+\-\.]*):)?(\/\/([^\/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))?/i)
     parts = nillify(Regex.run(regex, string))
 
     destructure [_, _, scheme, _, authority, path, _, query, _, fragment], parts
@@ -419,7 +419,8 @@ defmodule URI do
 
   # Split an authority into its userinfo, host and port parts.
   defp split_authority(string) do
-    components = Regex.run(~r/(^(.*)@)?(\[[a-zA-Z0-9:.]*\]|[^:]*)(:(\d*))?/, string || "")
+    regex = Regex.recompile!(~r/(^(.*)@)?(\[[a-zA-Z0-9:.]*\]|[^:]*)(:(\d*))?/)
+    components = Regex.run(regex, string || "")
 
     destructure [_, _, userinfo, host, _, port], nillify(components)
     host = if host, do: host |> String.trim_leading("[") |> String.trim_trailing("]")
@@ -473,6 +474,9 @@ defmodule URI do
   def merge(_base, %URI{scheme: rel_scheme} = rel) when rel_scheme != nil do
     rel
   end
+  def merge(base, %URI{authority: authority} = rel) when authority != nil do
+    %{rel | scheme: base.scheme}
+  end
   def merge(%URI{} = base, %URI{path: rel_path} = rel) when rel_path in ["", nil] do
     %{base | query: rel.query || base.query, fragment: rel.fragment}
   end

lib/elixir/lib/version.ex

@@ -172,18 +172,32 @@ defmodule Version do
   end
 
   @doc """
-  Compares two versions. Returns `:gt` if first version is greater than
-  the second and `:lt` for vice versa. If the two versions are equal `:eq`
-  is returned
+  Compares two versions. Returns `:gt` if the first version is greater than
+  the second one, and `:lt` for vice versa. If the two versions are equal `:eq`
+  is returned.
 
-  Raises a `Version.InvalidVersionError` exception if `version` is not parsable.
-  If given an already parsed version this function won't raise.
+  Pre-releases are strictly less than their corresponding release versions.
+
+  Patch segments are compared lexicographically if they are alphanumeric, and
+  numerically otherwise.
+
+  Build segments are ignored, if two versions differ only in their build segment
+  they are considered to be equal.
+
+  Raises a `Version.InvalidVersionError` exception if any of the two are not
+  parsable. If given an already parsed version this function won't raise.
 
   ## Examples
 
       iex> Version.compare("2.0.1-alpha1", "2.0.0")
       :gt
 
+      iex> Version.compare("1.0.0-beta", "1.0.0-rc1")
+      :lt
+
+      iex> Version.compare("1.0.0-10", "1.0.0-2")
+      :gt
+
       iex> Version.compare("2.0.1+build0", "2.0.1")
       :eq
 
@@ -224,9 +238,10 @@ defmodule Version do
   @spec parse(String.t) :: {:ok, t} | :error
   def parse(string) when is_binary(string) do
     case Version.Parser.parse_version(string) do
-      {:ok, {major, minor, patch, pre}} ->
+      {:ok, {major, minor, patch, pre, build_parts}} ->
+        build = if build_parts == [], do: nil, else: Enum.join(build_parts, "")
         version = %Version{major: major, minor: minor, patch: patch,
-                       pre: pre, build: get_build(string)}
+                           pre: pre, build: build}
         {:ok, version}
      :error ->
        :error
@@ -298,23 +313,13 @@ defmodule Version do
 
   defp to_matchable(string, allow_pre?) do
     case Version.Parser.parse_version(string) do
-      {:ok, {major, minor, patch, pre}} ->
+      {:ok, {major, minor, patch, pre, _build_parts}} ->
         {major, minor, patch, pre, allow_pre?}
       :error ->
         raise InvalidVersionError, message: string
     end
   end
 
-  defp get_build(string) do
-    case Regex.run(~r/\+([^\s]+)$/, string) do
-      nil ->
-        nil
-
-      [_, build] ->
-        build
-    end
-  end
-
   defmodule Parser.DSL do
     @moduledoc false
 
@@ -382,67 +387,92 @@ defmodule Version do
       Enum.filter(Enum.reverse(acc), &(&1 != :' '))
     end
 
-    @version_regex ~r/^
-      (\d+)                 # major
-      (?:\.(\d+))?          # minor
-      (?:\.(\d+))?          # patch
-      (?:\-([\d\w\.\-]+))?  # pre
-      (?:\+([\d\w\.\-]+))?  # build
-      $/x
-
     @spec parse_requirement(String.t) :: {:ok, term} | :error
     def parse_requirement(source) do
       lexed = lexer(source, [])
       to_matchspec(lexed)
     end
 
-    defp nillify(""), do: nil
-    defp nillify(o),  do: o
-
     @spec parse_version(String.t) :: {:ok, Version.matchable} | :error
     def parse_version(string, approximate? \\ false) when is_binary(string) do
-      if parsed = Regex.run(@version_regex, string) do
-        destructure [_, major, minor, patch, pre], parsed
-        patch = nillify(patch)
-        pre   = nillify(pre)
+      destructure [version_with_pre, build], String.split(string, "+", parts: 2)
+      destructure [version, pre], String.split(version_with_pre, "-", parts: 2)
+      destructure [major, minor, patch], String.split(version, ".")
 
-        if is_nil(minor) or (is_nil(patch) and not approximate?) do
-          :error
-        else
-          major = String.to_integer(major)
-          minor = String.to_integer(minor)
-          patch = patch && String.to_integer(patch)
+      with {:ok, major} <- require_digits(major),
+           {:ok, minor} <- require_digits(minor),
+           {:ok, patch} <- maybe_patch(patch, approximate?),
+           {:ok, pre_parts} <- optional_dot_separated(pre),
+           {:ok, pre_parts} <- convert_parts_to_integer(pre_parts, []),
+           {:ok, build_parts} <- optional_dot_separated(build) do
+        {:ok, {major, minor, patch, pre_parts, build_parts}}
+      else
+        _other -> :error
+      end
+    end
 
-          case parse_pre(pre) do
-            {:ok, pre} ->
-              {:ok, {major, minor, patch, pre}}
-            :error ->
-              :error
-          end
-        end
+    defp require_digits(nil), do: :error
+    defp require_digits(string) do
+      if leading_zero?(string), do: :error, else: parse_digits(string, "")
+    end
+
+    defp leading_zero?(<<?0, _, _::binary>>), do: true
+    defp leading_zero?(_), do: false
+
+    defp parse_digits(<<char, rest::binary>>, acc) when char in ?0..?9,
+      do: parse_digits(rest, <<acc::binary, char>>)
+    defp parse_digits(<<>>, acc) when byte_size(acc) > 0,
+      do: {:ok, String.to_integer(acc)}
+    defp parse_digits(_, _acc),
+      do: :error
+
+    defp maybe_patch(patch, approximate?)
+    defp maybe_patch(nil, true), do: {:ok, nil}
+    defp maybe_patch(patch, _), do: require_digits(patch)
+
+    defp optional_dot_separated(nil), do: {:ok, []}
+    defp optional_dot_separated(string) do
+      parts = String.split(string, ".")
+      if Enum.all?(parts, &(&1 != "" and valid_identifier?(&1))) do
+        {:ok, parts}
       else
         :error
       end
     end
 
-    defp parse_pre(nil), do: {:ok, []}
-    defp parse_pre(pre), do: parse_pre(String.split(pre, "."), [])
-
-    defp parse_pre([piece | t], acc) do
-      cond do
-        piece =~ ~r/^(0|[1-9][0-9]*)$/ ->
-          parse_pre(t, [String.to_integer(piece) | acc])
-        piece =~ ~r/^[0-9]*$/ ->
-          :error
-        true ->
-          parse_pre(t, [piece | acc])
+    defp convert_parts_to_integer([part | rest], acc) do
+      case parse_digits(part, "") do
+        {:ok, integer} ->
+          if leading_zero?(part) do
+            :error
+          else
+            convert_parts_to_integer(rest, [integer | acc])
+          end
+        :error ->
+          convert_parts_to_integer(rest, [part | acc])
       end
     end
 
-    defp parse_pre([], acc) do
+    defp convert_parts_to_integer([], acc) do
       {:ok, Enum.reverse(acc)}
     end
 
+    defp valid_identifier?(<<char, rest::binary>>)
+         when char in ?0..?9
+         when char in ?a..?z
+         when char in ?A..?Z
+         when char == ?- do
+      valid_identifier?(rest)
+    end
+
+    defp valid_identifier?(<<>>) do
+      true
+    end
+
+    defp valid_identifier?(_other) do
+      false
+    end
+
     defp valid_requirement?([]), do: false
     defp valid_requirement?([a | next]), do: valid_requirement?(a, next)
 
@@ -560,7 +590,7 @@ defmodule Version do
 
     defp parse_condition(version, approximate? \\ false) do
       case parse_version(version, approximate?) do
-        {:ok, version} -> version
+        {:ok, {major, minor, patch, pre, _build}} -> {major, minor, patch, pre}
         :error -> throw :invalid_matchspec
       end
     end

lib/elixir/pages/Behaviours.md

@@ -27,7 +27,9 @@ For more information on typespecs, consult the ["Typespecs"](typespecs.html) pag
 
 ### Optional callbacks
 
-Optional callbacks are callbacks that callback modules may implement if they want to, but are not required to. Usually, behaviour modules provide a default implementation for such callbacks in case the callback module does not implement them. Optional callbacks can be defined through the `@optional_callbacks` module attribute, which has to be a list of `{function_or_macro_name, arity}` tuples. For example:
+Optional callbacks are callbacks that callback modules may implement if they want to, but are not required to.
+Usually, behaviour modules know if they should call those callbacks based on configuration, or they check if the callbacks are defined with `function_exported?/3` or `macro_exported?/3`.
+Optional callbacks can be defined through the `@optional_callbacks` module attribute, which has to be a keyword list with function or macro name as key and arity as value. For example:
 
     defmodule MyBehaviour do
       @callback vital_fun() :: any

lib/elixir/pages/Guards.md

@@ -0,0 +1,192 @@
+# Guards
+
+Guards are a way to augment pattern matching with more complex checks; they are allowed in a predefined set of constructs where pattern matching is allowed.
+
+## List of allowed expressions
+
+For reference, the following is a comprehensive list of all expressions allowed in guards:
+
+  * comparison operators (`==`, `!=`, `===`, `!==`, `>`, `>=`, `<`, `<=`)
+  * strictly boolean operators (`and`, `or`, `not`) (the `&&`, `||`, and `!` sibling operators are not allowed as they're not *strictly* boolean - meaning they don't require both sides to be booleans)
+  * arithmetic binary operators (`+`, `-`, `*`, `/`)
+  * arithmetic unary operators (`+`, `-`)
+  * binary concatenation operator (`<>`)
+  * `in` operator (as long as the right-hand side is a list or a range)
+  * the following "type-check" functions (all documented in the `Kernel` module):
+    * `is_atom/1`
+    * `is_binary/1`
+    * `is_bitstring/1`
+    * `is_boolean/1`
+    * `is_float/1`
+    * `is_function/1`
+    * `is_function/2`
+    * `is_integer/1`
+    * `is_list/1`
+    * `is_map/1`
+    * `is_nil/1`
+    * `is_number/1`
+    * `is_pid/1`
+    * `is_port/1`
+    * `is_reference/1`
+    * `is_tuple/1`
+  * the following guard-friendly functions (all documented in the `Kernel` module):
+    * `abs/1`
+    * `binary_part/3`
+    * `bit_size/1`
+    * `byte_size/1`
+    * `div/2`
+    * `elem/2`
+    * `hd/1`
+    * `length/1`
+    * `map_size/1`
+    * `node/0`
+    * `node/1`
+    * `rem/2`
+    * `round/1`
+    * `self/0`
+    * `tl/1`
+    * `trunc/1`
+    * `tuple_size/1`
+  * the following handful of Erlang bitwise operations, if imported from the `Bitwise` module:
+    * `band/2` or the `&&&` operator
+    * `bor/2` or the `|||` operator
+    * `bnot/1` or the `~~~` operator
+    * `bsl/1` or the `<<<` operator
+    * `bsr/1` or the `>>>` operator
+    * `bxor/2` or the `^^^` operator
+
+## Why guards
+
+Let's see an example of a guard used in a function clause:
+
+```elixir
+def empty_map?(map) when map_size(map) == 0, do: true
+def empty_map?(map) when is_map(map), do: false
+```
+
+Guards start with the `when` keyword, which is followed by a boolean expression (we will define the grammar of guards more formally later on).
+
+Writing the `empty_map?/1` function by only using pattern matching would not be possible (as pattern matching on `%{}` would match *every* map, not empty maps).
+
+## Where guards can be used
+
+In the example above, we show how guards can be used in function clauses. There are several constructs that allow guards; for example:
+
+  * function clauses:
+
+  ```elixir
+  def foo(term) when is_integer(term), do: term
+  def foo(term) when is_float(term), do: round(term)
+  ```
+
+  * `case` expressions:
+
+  ```elixir
+  case x do
+    1 -> :one
+    2 -> :two
+    n when is_integer(n) and n > 2 -> :larger_than_two
+  end
+  ```
+
+  * anonymous functions (`fn`s):
+
+  ```elixir
+  larger_than_two? = fn
+    n when is_integer(n) and n > 2 -> true
+    n when is_integer(n) -> false
+  end
+  ```
+
+Other constructs are `for`, `with`, `try`/`rescue`/`catch`/`else`/, and the `match?/2` macro in the `Kernel` module.
+
+## Failing guards
+
+Errors in guards do not result in a runtime error, but in the erroring guard fail. For example, the `length/1` function only works with lists, and if we use it on anything else it fails:
+
+```elixir
+iex> length("hello")
+** (ArgumentError) argument error
+```
+
+However, when used in guards, it simply makes the corresponding clause fail (i.e., not match):
+
+```elixir
+iex> case "hello" do
+...>   something when length(something) > 0 ->
+...>     :length_worked
+...>   _anything_else ->
+...>     :length_failed
+...> end
+:length_failed
+```
+
+In many cases, we can take advantage of this: in the code above, for example, we can use `length/1` to both check that the given thing is a list *and* check some properties of its length (instead of using `is_list(something) and length(something) > 0`).
+
+## Expressions in guard clauses
+
+Not all expressions are allowed in guard clauses, but only a handful of them. This is a deliberate choice: only a predefined set of side-effect-free functions are allowed. This way, Elixir (and Erlang) can make sure that nothing bad happens while executing guards and no mutations happen anywhere. This behaviour is also coherent with pattern match, which is a naturally a side-effect-free operation. Finally, keeping expressions allowed in clauses to a close set of predefined ones allows the compiler to optimize the code related to choosing the right clause.
+
+## Defining custom guard expressions
+
+As mentioned before, only the expressions listed in this page are allowed in guards. However, we can take advantage of macros to write custom guards that can simplify our programs or make them more domain-specific. At the end of the day, what matters is that the *output* of the macros (which is what will be compiled) boils down to a combinations of the allowed expressions.
+
+Let's look at a quick case study: we want to check that a function argument is an even or odd integer. With pattern matching, this is impossible to do since there are infinite integers, and thus we can't pattern match on the single even/odd numbers. Let's focus on checking for even numbers since checking for odd ones is almost identical.
+
+Such a guard would look like this:
+
+```elixir
+def my_function(number) when is_integer(number) and rem(number, 2) == 0 do
+  # do stuff
+end
+```
+
+This would be repetitive to write everytime we need this check, so, as mentioned at the beginning of this section, we can abstract this away using a macro. Remember that defining a function that performs this check wouldn't work because we can't use custom functions in guards. Our macro would look like this:
+
+```elixir
+defmodule MyInteger do
+  defmacro is_even(number) do
+    quote do
+      is_integer(unquote(number)) and rem(unquote(number), 2) == 0
+    end
+  end
+end
+```
+
+and then:
+
+```elixir
+import MyInteger, only: [is_even: 1]
+
+def my_function(number) when is_even(number) do
+  # do stuff
+end
+```
+
+## Multiple guards in the same clause
+
+There exists an additional way to simplify a chain of `or`s in guards: Elixir supports writing "multiple guards" in the same clause. This:
+
+```elixir
+def foo(term) when is_integer(term) or is_float(term) or is_nil(term),
+  do: :maybe_number
+def foo(_other),
+  do: :something_else
+```
+
+can be alternatively written as:
+
+```elixir
+def foo(term)
+    when is_integer(term)
+    when is_float(term)
+    when is_nil(term) do
+  :maybe_number
+end
+
+def foo(_other) do
+  :something_else
+end
+```
+
+The two forms are exactly the same semantically but there are cases where the latter one may be more aesthetically pleasing.

lib/elixir/pages/Naming Conventions.md

@@ -35,12 +35,12 @@ Function names may also start with an underscore. Such functions are never impor
     iex> _wont_be_imported()
     ** (CompileError) iex:1: undefined function _wont_be_imported/0
 
-Due to this property, Elixir relies on functions starting with underscore to attach compile-time metadata to modules. Such functions are most often in the `__foo__` format. For example, every module in Elixir has an `__info__` function:
+Due to this property, Elixir relies on functions starting with underscore to attach compile-time metadata to modules. Such functions are most often in the `__foo__` format. For example, every module in Elixir has an `__info__/1` function:
 
     iex> String.__info__(:functions)
     [at: 2, capitalize: 1, chunk: 2, ...]
 
-Elixir also includes 4 special variables that follow the double underscore format. These forms retrieve compile-time information about the current environment: `__MODULE__`, `__DIR__`, `__ENV__` and `__CALLER__`.
+Elixir also includes 4 special variables that follow the double underscore format. These forms retrieve compile-time information about the current environment: `__MODULE__/0`, `__DIR__/0`, `__ENV__/0` and `__CALLER__/0`.
 
 ## Trailing bang (foo!)
 
@@ -105,4 +105,4 @@ When you see `length`, the operation runs in linear time ("O(n) time") because t
 
 Examples: `Kernel.length/1`, `String.length/1`
 
-In other words, `size` functions will take the same amount of time whether the data structure is tiny or huge. `length` functions will take more time as the data structure grows in size.
+In other words, functions using the word "size" in its name will take the same amount of time whether the data structure is tiny or huge. Conversely, functions having "length" in its name will take more time as the data structure grows in size.

lib/elixir/pages/Operators.md

@@ -0,0 +1,154 @@
+# Operators
+
+This document covers operators in Elixir, how they are parsed, how they can be defined, and how they can be overridden.
+
+## Operator precedence and associativity
+
+The following is a list of all operators that Elixir is capable of parsing, ordered from higher to lower precedence, alongside their associativity:
+
+Operator                                                                                 | Associativity
+---------------------------------------------------------------------------------------- | -------------
+`@`                                                                                      | Unary
+`.`                                                                                      | Left to right
+`+` `-` `!` `^` `not` `~~~`                                                              | Unary
+`*` `/`                                                                                  | Left to right
+`+` `-`                                                                                  | Left to right
+`++` `--` `..` `<>`                                                                      | Right to left
+`in`                                                                                     | Left to right
+`\|>` `<<<` `>>>` `~>>` `<<~` `~>` `<~` `<~>` `<\|>`                                     | Left to right
+`<` `>` `<=` `>=`                                                                        | Left to right
+`==` `!=` `=~` `===` `!==`                                                               | Left to right
+`&&` `&&&` `and`                                                                         | Left to right
+`\|\|` `\|\|\|` `or`                                                                     | Left to right
+`=`                                                                                      | Right to left
+`=>`                                                                                     | Right to left
+`\|`                                                                                     | Right to left
+`::`                                                                                     | Right to left
+`when`                                                                                   | Right to left
+`<-`, `\\`                                                                               | Left to right
+`&`                                                                                      | Unary
+
+## Comparison operators
+
+Elixir provides the following built-in comparison operators:
+
+  * `==` - equality
+  * `===` - strict equality
+  * `!=` - inequality
+  * `!==` - strict inequality
+  * `>` - greater than
+  * `<` - less than
+  * `>=` - greater than or equal
+  * `<=` - less than or equal
+
+The only difference between `==` and `===` is that `===` is stricter when it comes to comparing integers and floats:
+
+```elixir
+iex> 1 == 1.0
+true
+iex> 1 === 1.0
+false
+```
+
+`!=` and `!==` act as the negation of `==` and `===`, respectively.
+
+### Term ordering
+
+In Elixir, different data types can be compared using comparison operators:
+
+```elixir
+iex> 1 < :an_atom
+true
+```
+
+The reason we can compare different data types is pragmatism. Sorting algorithms don’t need to worry about different data types in order to sort. For reference, the overall sorting order is defined below:
+
+```
+number < atom < reference < function < port < pid < tuple < map < list < bitstring
+```
+
+When comparing two numbers of different types (a number is either an integer or a float), a conversion to the type with greater precision will always occur, unless the comparison operator used is either `===` or `!==`. A float will be considered more precise than an integer, unless the float is greater/less than +/-9007199254740992.0, at which point all the significant figures of the float are to the left of the decimal point. This behavior exists so that the comparison of large numbers remains transitive.
+
+The collection types are compared using the following rules:
+
+* Tuples are compared by size then element by element.
+* Maps are compared by size then by keys in ascending term order then by values in key order. In the specific case of maps' key ordering, integers are always considered to be less than floats.
+* Lists are compared element by element.
+
+## Custom and overridden operators
+
+### Defining custom operators
+
+Elixir is capable of parsing a predefined set of operators; this means that it's not possible to define new operators (like one could do in Haskell, for example). However, not all operators that Elixir can parse are *used* by Elixir: for example, `+` and `||` are used by Elixir for addition and boolean *or*, but `<~>` is not used (but valid).
+
+To define an operator, you can use the usual `def*` constructs (`def`, `defp`, `defmacro`, and so on) but with a syntax similar to how the operator is used:
+
+```elixir
+defmodule MyOperators do
+  # We define ~> to return the maximum of the given two numbers,
+  # and <~ to return the minimum.
+
+  def a ~> b, do: max(a, b)
+  def a <~ b, do: min(a, b)
+end
+```
+
+To use the newly defined operators, we **have to** import the module that defines them:
+
+```elixir
+iex> import MyOperators
+iex> 1 ~> 2
+2
+iex> 1 <~ 2
+1
+```
+
+The following is a table of all the operators that Elixir is capable of parsing, but that are not used by default:
+
+  * `|`
+  * `|||`
+  * `&&&`
+  * `<<<`
+  * `>>>`
+  * `~>>`
+  * `<<~`
+  * `~>`
+  * `<~`
+  * `<~>`
+  * `<|>`
+  * `^^^`
+  * `~~~`
+
+The following operators are used by the `Bitwise` module when imported: `&&&`, `^^^`, `<<<`, `>>>`, `|||`, `~~~`. See the documentation for `Bitwise` for more information.
+
+### Redefining existing operators
+
+The operators that Elixir uses (for example, `+`) can be defined by any module and used in place of the ones defined by Elixir, provided they're specifically not imported from `Kernel` (which is imported everywhere by default). For example:
+
+```elixir
+defmodule WrongMath do
+  # Let's make math wrong by changing the meaning of +:
+  def a + b, do: a - b
+end
+```
+
+Now, we will get an error if we try to use this operator "out of the box":
+
+```elixir
+iex> import WrongMath
+iex> 1 + 2
+** (CompileError) iex:11: function +/2 imported from both WrongMath and Kernel, call is ambiguous
+```
+
+So, as mentioned above, we need to explicitly *not* import `+/2` from `Kernel`:
+
+```elixir
+iex> import WrongMath
+iex> import Kernel, except: [+: 2]
+iex> 1 + 2
+-1
+```
+
+### Final note
+
+While it's possible to defined unused operators (such as `<~>`) and to "override" predefined operators (such as `+`), the Elixir community generally discourages this. Custom-defined operators can be really hard to read and even more to understand, as they don't have a descriptive name like functions do. That said, some specific cases or custom domain specific languages (DSLs) may justify these practices.

lib/elixir/pages/Typespecs.md

@@ -3,7 +3,7 @@
 Elixir comes with a notation for declaring types and specifications. Elixir is a dynamically typed language, and as such, type specifications are never used by the compiler to optimize or modify code. Still, using type specifications is useful because
 
   * they provide documentation (for example, tools such as [ExDoc](https://github.com/elixir-lang/ex_doc) show type specifications in the documentation)
-  * they're used by tools sych as [Dialyzer](http://www.erlang.org/doc/man/dialyzer.html), that can analyze code with typespec to find type inconsistencies and possible bugs
+  * they're used by tools such as [Dialyzer](http://www.erlang.org/doc/man/dialyzer.html), that can analyze code with typespec to find type inconsistencies and possible bugs
 
 Type specifications (sometimes referred to as *typespecs*) are defined in different contexts using the following attributes:
 
@@ -24,22 +24,28 @@ The syntax Elixir provides for type specifications is similar to [the one in Erl
 
     type :: any()                   # the top type, the set of all terms
           | none()                  # the bottom type, contains no terms
+          | atom()
+          | map()                   # any map
           | pid()
           | port()
           | reference()
-          | tuple()
-          | atom()
-          | integer()
+          | struct()                # any struct
+          | tuple()                 # tuple of any size
+
+                                    ## Numbers
+          | float()                 # float
+          | integer()               # integer
+          | neg_integer()           # ..., -3, -2, -1
           | non_neg_integer()       # 0, 1, 2, 3, ...
           | pos_integer()           # 1, 2, 3, ...
-          | neg_integer()           # ..., -3, -2, -1
-          | float()
-          | map()                   # any map
-          | struct()                # any struct
-          | list(type)
-          | nonempty_list(type)
-          | improper_list(type1, type2)
-          | maybe_improper_list(type1, type2)
+
+                                                        ## Lists
+          | list(type)                                  # proper list ([]-terminated)
+          | nonempty_list(type)                         # non-empty proper list
+          | maybe_improper_list(type1, type2)           # proper or improper list
+          | nonempty_improper_list(type1, type2)        # improper list
+          | nonempty_maybe_improper_list(type1, type2)  # non-empty proper or improper list
+
           | Literals                # Described in section "Literals"
           | Builtin                 # Described in section "Built-in types"
           | Remotes                 # Described in section "Remote types"
@@ -49,40 +55,41 @@ The syntax Elixir provides for type specifications is similar to [the one in Erl
 The following literals are also supported in typespecs:
 
     type :: :atom                         ## Atoms
-          | 1                             ## Integers
-          | 1..10                         ## Integers from 1 to 10
-          | 1.0                           ## Floats
+          | true | false | nil            # Special atom literals
 
                                           ## Bitstrings
           | <<>>                          # empty bitstring
           | <<_::size>>                   # size is 0 or a positive integer
-          | <<_::_ * unit>>               # unit is an integer from 1 to 256
+          | <<_::_*unit>>                 # unit is an integer from 1 to 256
           | <<_::size, _::_*unit>>
 
-                                          ## Lists
-          | [type]                        # list with any number of type elements
-          | []                            # empty list
-          | [...]                         # shorthand for nonempty_list(any())
-          | [type, ...]                   # shorthand for nonempty_list(type)
-          | [key: type]                   # keyword lists
-
                                           ## Functions
           | (... -> type)                 # any arity, returns type
           | (() -> type)                  # 0-arity, returns type
           | (type1, type2 -> type)        # 2-arity, returns type
 
-                                          ## Maps
-          | %{}                           # empty map
-          | %{...}                        # any map
-          | %{key: type}                  # map with required key :key with value of type
-          | %{required(type1) => type2}   # map with required keys of type1 with values of type2
-          | %{optional(type1) => type2}   # map with optional keys of type1 with values of type2
-          | %SomeStruct{}                 # struct with all fields of any type
-          | %SomeStruct{key: type}        # struct with :key field of type
+                                          ## Integers
+          | 1                             # integer
+          | 1..10                         # integer from 1 to 10
+
+                                          ## Lists
+          | [type]                        # list with any number of type elements
+          | []                            # empty list
+          | [...]                         # shorthand for nonempty_list(any())
+          | [type, ...]                   # shorthand for nonempty_list(type)
+          | [key: value_type]             # keyword list with key :key of value_type
+
+                                                  ## Maps
+          | %{}                                   # empty map
+          | %{key: value_type}                    # map with required key :key of value_type
+          | %{required(key_type) => value_type}   # map with required pairs of key_type and value_type
+          | %{optional(key_type) => value_type}   # map with optional pairs of key_type and value_type
+          | %SomeStruct{}                         # struct with all fields of any type
+          | %SomeStruct{key: value_type}          # struct with required key :key of value_type
 
                                           ## Tuples
           | {}                            # empty tuple
-          | {:ok, type}                   # two element tuple with an atom and any type
+          | {:ok, type}                   # two-element tuple with an atom and any type
 
 ### Built-in types
 
@@ -91,42 +98,45 @@ The following types are also provided by Elixir as shortcuts on top of the basic
 Built-in type           | Defined as
 :---------------------- | :---------
 `term()`                | `any()`
-`binary()`              | `<<_::_ * 8>>`
-`bitstring()`           | `<<_::_ * 1>>`
+`arity()`               | `0..255`
+`as_boolean(t)`         | `t`
+`binary()`              | `<<_::_*8>>`
+`bitstring()`           | `<<_::_*1>>`
 `boolean()`             | `false` \| `true`
 `byte()`                | `0..255`
 `char()`                | `0..0x10FFFF`
-`number()`              | `integer()` \| `float()`
 `charlist()`            | `[char()]`
-`list()`                | `[any()]`
-`maybe_improper_list()` | `maybe_improper_list(any(), any())`
-`nonempty_list()`       | `nonempty_list(any())`
-`iolist()`              | `maybe_improper_list(byte() \| binary() \| iolist(), binary() \| [])`
-`iodata()`              | `iolist()` \| `binary()`
-`module()`              | `atom()` \| `tuple()`
-`arity()`               | `0..255`
-`mfa()`                 | `{atom(), atom(), arity()}`
-`identifier()`          | `pid()` \| `port()` \| `reference()`
-`node()`                | `atom()`
-`timeout()`             | `:infinity` \| `non_neg_integer()`
-`no_return()`           | `none()`
 `fun()`                 | `(... -> any)`
-`struct()`              | `%{__struct__: atom()}`
-`as_boolean(t)`         | `t`
+`identifier()`          | `pid()` \| `port()` \| `reference()`
+`iodata()`              | `iolist()` \| `binary()`
+`iolist()`              | `maybe_improper_list(byte() \| binary() \| iolist(), binary() \| [])`
 `keyword()`             | `[{atom(), any()}]`
 `keyword(t)`            | `[{atom(), t}]`
+`list()`                | `[any()]`
+`nonempty_list()`       | `nonempty_list(any())`
+`maybe_improper_list()` | `maybe_improper_list(any(), any())`
+`nonempty_maybe_improper_list()` | `nonempty_maybe_improper_list(any(), any())`
+`mfa()`                 | `{module(), atom(), arity()}`
+`module()`              | `atom()`
+`no_return()`           | `none()`
+`node()`                | `atom()`
+`number()`              | `integer()` \| `float()`
+`struct()`              | `%{:__struct__ => atom(), optional(atom()) => any()}`
+`timeout()`             | `:infinity` \| `non_neg_integer()`
 
 ### Remote types
 
-Any module is also able to define its own types and the modules in Elixir are no exception. For example, the `Range` module defines a `t` type that represents a range: this type can be referred to as `Range.t`. In a similar fashion, a string is `String.t`, any enumerable can be `Enum.t`, and so on.
+Any module is also able to define its own types and the modules in Elixir are no exception. For example, the `Range` module defines a `t/0` type that represents a range: this type can be referred to as `t:Range.t/0`. In a similar fashion, a string is `t:String.t/0`, any enumerable can be `t:Enum.t/0`, and so on.
 
 ### Maps
 
-The key types in maps are allowed to overlap, and if they do, the leftmost key takes precedence. A map value does not belong to this type if it contains a key that is not in the maps allowed keys.
+The key types in maps are allowed to overlap, and if they do, the leftmost key takes precedence.
+A map value does not belong to this type if it contains a key that is not in the allowed map keys.
 
-Because it is common to end a map type with `optional(any) => any` to denote that keys that do not belong to any other key in the map type are allowed, and may map to any value, the shorthand notation `...` is allowed as the last element of a map type.
+If you want to denote that keys that were not previously defined in the map are allowed,
+it is common to end a map type with `optional(any) => any`.
 
-Notice that the syntactic representation of `map()` is `%{...}` (or `%{optional(any) => any}`), not `%{}`. The notation `%{}` specifies the singleton type for the empty map.
+Notice that the syntactic representation of `map()` is `%{optional(any) => any}`, not `%{}`. The notation `%{}` specifies the singleton type for the empty map.
 
 ## Defining a type
 
@@ -154,6 +164,10 @@ Guards can be used to restrict type variables given as arguments to the function
 
     @spec function(arg) :: [arg] when arg: atom
 
+If you want to specify more than one variable, you separate them by a comma.
+
+    @spec function(arg1, arg2) :: [arg1, arg2] when arg1: atom, arg2: integer
+
 Type variables with no restriction can also be defined.
 
     @spec function(arg) :: [arg] when arg: var
@@ -170,6 +184,6 @@ Specifications can be overloaded just like ordinary functions.
 
 ## Notes
 
-Elixir discourages the use of type `string` as it might be confused with binaries which are referred to as "strings" in Elixir (as opposed to character lists). In order to use the type that is called `string` in Erlang, one has to use the `charlist` type which is a synonym for `string`. If you use `string`, you'll get a warning from the compiler.
+Elixir discourages the use of type `t:string/0` as it might be confused with binaries which are referred to as "strings" in Elixir (as opposed to character lists). In order to use the type that is called `t:string/0` in Erlang, one has to use the `t:charlist/0` type which is a synonym for `string`. If you use `string`, you'll get a warning from the compiler.
 
-If you want to refer to the "string" type (the one operated on by functions in the `String` module), use `String.t` type instead.
+If you want to refer to the "string" type (the one operated on by functions in the `String` module), use `t:String.t/0` type instead.

lib/elixir/pages/Writing Documentation.md

@@ -4,10 +4,10 @@ Elixir treats documentation as a first-class citizen. This means documentation s
 
 ## Markdown
 
-Elixir documentation is written using Markdown. There are plenty of guides on Markdown online, we recommend the ones available at GitHub as a getting started point:
+Elixir documentation is written using Markdown. There are plenty of guides on Markdown online, we recommend the ones available on GitHub as a getting started point:
 
-  * https://help.github.com/articles/markdown-basics/
-  * https://help.github.com/articles/github-flavored-markdown/
+  * [Basic writing and formatting syntax](https://help.github.com/articles/basic-writing-and-formatting-syntax/)
+  * [Mastering Markdown](https://guides.github.com/features/mastering-markdown/)
 
 ## Module Attributes
 
@@ -34,7 +34,7 @@ Documentation in Elixir is usually attached to module attributes. Let's see an e
       end
     end
 
-The `@moduledoc` attribute is used to add documentation to the module. `@doc` is used before a function to provide documentation for it. Besides the attributes above, `@typedoc` can also be used to attach documentation to types defined as part of typespecs.
+The `@moduledoc` attribute is used to add documentation to the module. `@doc` is used before a function to provide documentation for the function that follows. Besides the attributes above, `@typedoc` can also be used to attach documentation to types defined as part of typespecs.
 
 ## Function Arguments
 
@@ -44,7 +44,7 @@ When documenting a function, argument names are inferred by the compiler. For ex
       size
     end
 
-The compiler will infer this argument as `map`. Sometimes the inference will be suboptimal, specially if the function contains multiple clauses with the argument matching on different values each time. You can specify the proper names for documentation by declaring before a bodyless clause:
+The compiler will infer this argument as `map`. Sometimes the inference will be suboptimal, especially if the function contains multiple clauses with the argument matching on different values each time. You can specify the proper names for documentation by declaring only the function head at any moment before the implementation:
 
     def size(map)
     def size(%{size: size}) do
@@ -53,33 +53,41 @@ The compiler will infer this argument as `map`. Sometimes the inference will be
 
 ## Recommendations
 
-There are a couple tips we recommend developers to follow when writing documentation:
+When writing documentation:
 
-  * Keep the first paragraph of the documentation concise and simple, typically one-line. Tools like [ExDoc](https://github.com/elixir-lang/ex_doc/) use the first line to generate a summary.
+  * Keep the first paragraph of the documentation concise and simple, typically one line. Tools like [ExDoc](https://github.com/elixir-lang/ex_doc/) use the first line to generate a summary.
 
-  * Markdown uses backticks (`` ` ``) to quote code. Elixir builds on top of that to automatically generate links when module or function names are referenced. For this reason, always use full module names. If you have a module called `MyApp.Hello`, always reference it as `` `MyApp.Hello` `` and never as `` `Hello` ``. Function names must be referenced by name and arity if they are local, as in `` `world/1` ``, or by module, name and arity if pointing to an external module: `` `MyApp.Hello.world/1` ``. Referencing a `@callback` can be done by prepending `c:`, as in `` `c:world/1` ``; and a `@type` by using `t:`, as in `t:values/0`.
+  * Reference modules by their full name.
 
-  * When the documentation has multiple sections, always start the section heading by using `##`. The first heading is reserved for modules and function names.
+Markdown uses backticks (`` ` ``) to quote code. Elixir builds on top of that to automatically generate links when module or function names are referenced. For this reason, always use full module names. If you have a module called `MyApp.Hello`, always reference it as `` `MyApp.Hello` `` and never as `` `Hello` ``.
 
-  * When documenting a function with multiple clauses, the documentation must be placed before the first clause. Documentation is always per function and arity and not per clause.
+  * Reference functions by name and arity if they are local, as in `` `world/1` ``, or by module, name and arity if pointing to an external module: `` `MyApp.Hello.world/1` ``.
+
+  * Reference a `@callback` by prepending `c:`, as in `` `c:world/1` ``.
+
+  * Reference a `@type` by prepending `t:`, as in `` `t:values/0` ``.
+
+  * Start new sections with second level Markdown headers `##`. First level headers are reserved for module and function names.
+
+  * Place documentation before the first clause of multi-clause functions. Documentation is always per function and arity and not per clause.
 
 ## Doctests
 
-We recommend developers to include examples in their documentation, often under its own `## Examples` heading. To ensure examples do not get out of date, Elixir's test framework (ExUnit) provides a feature called doctests that allows developers to test the examples in their documentation. Doctests work by parsing out code samples starting with `iex>` from the documentation. You can read more about it at `ExUnit.DocTest`.
+We recommend that developers include examples in their documentation, often under their own `## Examples` heading. To ensure examples do not get out of date, Elixir's test framework (ExUnit) provides a feature called doctests that allows developers to test the examples in their documentation. Doctests work by parsing out code samples starting with `iex>` from the documentation. You can read more about it in the documentation for `ExUnit.DocTest`.
 
-Notice doctests have limitations. When you cannot doctest a function, because it relies on state or side-effects, we recommend developers to include examples directly without the `iex>` prompt.
+Note that doctests have limitations. When you cannot doctest a function, because it relies on state or side-effects, we recommend developers include examples directly without the `iex>` prompt.
 
-## Documentation != Comments
+## Documentation is not Comments
 
-Elixir treats documentation and code comments as different concepts. Documentation are for users of your Application Programming Interface (API), be it your co-worker or your future self. Modules and functions must always be documented if they are part of your API.
+Elixir treats documentation and code comments as different concepts. Documentation is for users of your Application Programming Interface (API), be it your co-worker or your future self. Modules and functions must always be documented if they are part of your API.
 
-Code comments are for developers reading the code. They are useful to mark improvements, leave notes for developers reading the code (for example, you decided not to call a function due to a bug in a library) and so forth.
+Code comments are for developers reading the code. They are useful to mark improvements, leave notes for developers reading the code (for example, you decided not to call a function due to a bug in a library), and so on.
 
 In other words: documentation is required, code comments are optional.
 
 ## Hiding Internal Modules and Functions
 
-Besides the modules and functions libraries provided as part of their public interface, libraries may also implement important functionality that are not part of their API. While these modules and functions can be accessed, they are meant to be internal to the library and thus should not have documentation for end users.
+Besides the modules and functions that libraries provide as part of their public interface, libraries may also implement important functionality that is not part of their API. While these modules and functions can be accessed, they are meant to be internal to the library and thus should not have documentation for end users.
 
 Luckily, Elixir allows developers to hide modules and functions from the documentation. For example, one common practice for documenting internal behaviour is to set the `@moduledoc` attribute to `false` while documenting each function:
 
@@ -94,25 +102,25 @@ Luckily, Elixir allows developers to hide modules and functions from the documen
       end
     end
 
-Similarly, developers can add `@doc false` to functions they do not want to be publicly exposed:
+Similarly, developers can add `@doc false` to functions they do not want to show up in the documentation:
 
     defmodule MyApp.Sample do
       @doc false
       def add(a, b), do: a + b
     end
 
-However keep in mind adding `@doc false` does not make the function private. The function above can still be invoked as `MyApp.Sample.add(1, 2)`. Not only that, if `MyApp.Sample` is imported, the `add/2` function will also be imported into the caller. For those reasons, be cautious when adding `@doc false` to functions, instead prefer one of:
+However, keep in mind that adding `@doc false` does not make the function private. The function above can still be invoked as `MyApp.Sample.add(1, 2)`. Not only that, if `MyApp.Sample` is imported, the `add/2` function will also be imported into the caller. For those reasons, be cautious when adding `@doc false` to functions, instead use one of these two options:
 
   * Move the undocumented function to a module with `@moduledoc false`, like `MyApp.Hidden`, ensuring the function won't be accidentally exposed or imported. Remember you can use `@moduledoc false` to hide a whole module and still document each function with `@doc`. Tools will still ignore the module.
 
-  * Start the function name with one or two underscores, for example, `__add__/2`, and add `@doc false`. The compiler does not import functions with leading underscores and they hint to anyone reading the code of their intended private usage.
+  * Start the function name with one or two underscores, for example, `__add__/2`, and add `@doc false`. The compiler does not import functions with leading underscores and they suggest to anyone reading the code that they're meant to be used privately.
 
 ## Documenting Private Functions
 
 Elixir warns if a private function has a `@doc` attribute and discards its content, because `@doc` is intended to be used only for your public interface.
 
-Private functions may still need internal documentation for maintainers, though. That can be accomplished with code commments.
+Private functions may still need internal documentation for maintainers, though. That can be accomplished with code comments.
 
-## Code.get_docs/2
+## Retrieving Documentation
 
 Elixir stores documentation inside pre-defined chunks in the bytecode. It can be accessed from Elixir by using the `Code.get_docs/2` function. This also means documentation is only accessed when required and not when modules are loaded by the Virtual Machine. The only downside is that modules defined in-memory, like the ones defined in IEx, cannot have their documentation accessed as they do not have their bytecode written to disk.

lib/elixir/rebar.config

@@ -1,23 +1,24 @@
 {erl_opts, [
-            warn_unused_vars,
-            warn_export_all,
-            warn_shadow_vars,
-            warn_unused_import,
-            warn_unused_function,
-            warn_bif_clash,
-            warn_unused_record,
-            warn_deprecated_function,
-            warn_obsolete_guard,
-            strict_validation,
-            warn_exported_vars,
-            %% warn_export_vars,
-            %% warn_missing_spec,
-            %% warn_untyped_record,
-            %% warnings_as_errors,
-            debug_info
-           ]}.
+  warn_unused_vars,
+  warn_export_all,
+  warn_shadow_vars,
+  warn_unused_import,
+  warn_unused_function,
+  warn_bif_clash,
+  warn_unused_record,
+  warn_deprecated_function,
+  warn_obsolete_guard,
+  strict_validation,
+  warn_exported_vars,
+  %% warn_export_vars,
+  %% warn_missing_spec,
+  %% warn_untyped_record,
+  %% warnings_as_errors,
+  debug_info,
+  {platform_define, "^18.*", old_map_specs}
+]}.
 
 {yrl_opts, [
-            {report, true},
-            {verbose, false}
-           ]}.
+  {report, true},
+  {verbose, false}
+]}.

lib/elixir/src/elixir.erl

@@ -14,11 +14,16 @@
 -export_type([charlist/0, char_list/0, struct/0, as_boolean/1, keyword/0, keyword/1]).
 -type charlist() :: string().
 -type char_list() :: string().
--type struct() :: #{'__struct__' => atom()}.
 -type as_boolean(T) :: T.
 -type keyword() :: [{atom(), any()}].
 -type keyword(T) :: [{atom(), T}].
 
+-ifdef(old_map_specs).
+-type struct() :: #{'__struct__' => atom(), atom() => any()}.
+-else.
+-type struct() :: #{'__struct__' := atom(), atom() => any()}.
+-endif.
+
 %% OTP Application API
 
 -export([start/2, stop/1, config_change/3]).
@@ -42,6 +47,11 @@ start(_Type, _Args) ->
       erlang:halt(1)
   end,
 
+  %% We need to make sure the re module is preloaded
+  %% to make function_exported checks on it fast.
+  %% TODO: Remove this once we support OTP 20+.
+  _ = code:ensure_loaded(re),
+
   case code:ensure_loaded(?system) of
     {module, ?system} ->
       Endianness = ?system:endianness(),
@@ -78,7 +88,8 @@ start(_Type, _Args) ->
                {{uri, <<"https">>}, 443},
                {{uri, <<"ldap">>}, 389}],
   CompilerOpts = #{docs => true, ignore_module_conflict => false,
-                   debug_info => true, warnings_as_errors => false},
+                   debug_info => true, warnings_as_errors => false,
+                   relative_paths => true},
   {ok, [[Home] | _]} = init:get_argument(home),
   Config = [{at_exit, []},
             {home, unicode:characters_to_binary(Home, Encoding, Encoding)},

lib/elixir/src/elixir.hrl

@@ -1,6 +1,8 @@
 -define(m(M, K), maps:get(K, M)).
 -define(ann(Opts), elixir_utils:get_ann(Opts)).
 -define(line(Opts), elixir_utils:get_line(Opts)).
+-define(generated(Opts), [{generated, true}, {location, ?line(Opts)}]).
+%% TODO: Remove once we drop Erlang 18 support
 -define(generated, [{generated, true}, {location, 0}]).
 
 -record(elixir_scope, {

lib/elixir/src/elixir_bitstring.erl

@@ -21,9 +21,9 @@ expand_bitstr(Fun, [{'::', Meta, [Left, Right]} | T], Acc, E) ->
 
   %% Variables defined outside the binary can be accounted
   %% on subparts, however we can't assign new variables.
-  case E of
-    {ER, _} -> ok;               %% expand_arg,  no assigns
-    _ -> ER = E#{context := nil} %% expand_each, revert assigns
+  ER = case E of
+    {EExtracted, _} -> EExtracted;        %% expand_arg,  no assigns
+    _               -> E#{context := nil} %% expand_each, revert assigns
   end,
 
   ERight = expand_bit_info(Meta, Right, ER),
@@ -161,7 +161,7 @@ build_bitstr_each(Fun, [H | T], Meta, S, Acc) ->
 
 build_bitstr_each(Fun, T, Meta, S, Acc, H, default, Types) when is_binary(H) ->
   Element =
-    case types_allow_splice(Types, []) of
+    case types_allow_splice(Types) of
       true ->
         %% See explanation in elixir_utils:elixir_to_erl/1 to know
         %% why we can simply convert the binary to a list.
@@ -187,13 +187,13 @@ build_bitstr_each(_Fun, _T, Meta, S, _Acc, H, _Size, _Types) when is_list(H); is
 
 build_bitstr_each(Fun, T, Meta, S, Acc, H, Size, Types) ->
   {Expr, NS} = Fun(H, S),
+  Splice = types_allow_splice(Types),
 
   case Expr of
-    {bin, _, Elements} ->
-      case (Size == default) andalso types_allow_splice(Types, Elements) of
-        true  -> build_bitstr_each(Fun, T, Meta, NS, lists:reverse(Elements, Acc));
-        false -> build_bitstr_each(Fun, T, Meta, NS, [{bin_element, ?ann(Meta), Expr, Size, Types} | Acc])
-      end;
+    {bin, _, Elements} when Splice, Size == default, S#elixir_scope.context == match ->
+      build_bitstr_each(Fun, T, Meta, NS, lists:reverse(Elements, Acc));
+    {bin, _, _} when Types == default ->
+      build_bitstr_each(Fun, T, Meta, NS, [{bin_element, ?ann(Meta), Expr, Size, [bitstring]} | Acc]);
     _ ->
       build_bitstr_each(Fun, T, Meta, NS, [{bin_element, ?ann(Meta), Expr, Size, Types} | Acc])
   end.
@@ -203,31 +203,12 @@ types_require_conversion([UTF | T]) when UTF == utf8; UTF == utf16; UTF == utf32
 types_require_conversion([]) -> true;
 types_require_conversion(_) -> false.
 
-types_allow_splice([bytes], Elements)  -> is_byte_size(Elements, 0);
-types_allow_splice([binary], Elements) -> is_byte_size(Elements, 0);
-types_allow_splice([bits], _)          -> true;
-types_allow_splice([bitstring], _)     -> true;
-types_allow_splice(default, _)         -> true;
-types_allow_splice(_, _)               -> false.
-
-is_byte_size([Element | T], Acc) ->
-  case elem_size(Element) of
-    {unknown, Unit} when Unit rem 8 == 0 -> is_byte_size(T, Acc);
-    {unknown, _Unit} -> false;
-    {Size, Unit} -> is_byte_size(T, Size*Unit + Acc)
-  end;
-is_byte_size([], Size) ->
-  Size rem 8 == 0.
-
-elem_size({bin_element, _, _, default, _})                -> {0, 0};
-elem_size({bin_element, _, _, {integer, _, Size}, Types}) -> {Size, unit_size(Types, 1)};
-elem_size({bin_element, _, _, _Size, Types})              -> {unknown, unit_size(Types, 1)}.
-
-unit_size([binary | T], _)       -> unit_size(T, 8);
-unit_size([bytes | T], _)        -> unit_size(T, 8);
-unit_size([{unit, Size} | _], _) -> Size;
-unit_size([_ | T], Guess)        -> unit_size(T, Guess);
-unit_size([], Guess)             -> Guess.
+types_allow_splice([bytes])     -> true;
+types_allow_splice([binary])    -> true;
+types_allow_splice([bits])      -> true;
+types_allow_splice([bitstring]) -> true;
+types_allow_splice(default)     -> true;
+types_allow_splice(_)           -> false.
 
 %% Extra bitstring specifiers
 

lib/elixir/src/elixir_clauses.erl

@@ -183,9 +183,10 @@ normalize_vars(Key, {Ref, Counter, _Safe},
         {atom, 0, nil}
     end,
 
-  %% TODO: Unsafe vars should raise in future versions.
-  %% When we do so, we can unify the export vars mechanism.
-  %% For v2.0, we will never export them (or always raise in case/cond/try).
+  %% TODO: For v2.0, we will never export unsafe vars but
+  %% we need to consider if we want to raise or a emit a warning.
+  %% Such a warning should be applied consistently to the language
+  %% (for example, case/try/receive/fn/etc).
   Value = {Ref, Counter, false},
 
   VS = S#elixir_scope{
@@ -198,17 +199,17 @@ normalize_vars(Key, {Ref, Counter, _Safe},
 % Generate match vars by checking if they were updated
 % or not and assigning the previous value.
 
-generate_match_vars([{Key, Value, Expr} | T], ClauseVars, Left, Right) ->
+generate_match_vars([{Key, {Value, _, _}, Expr} | T], ClauseVars, Left, Right) ->
   case maps:find(Key, ClauseVars) of
-    {ok, Value} ->
+    {ok, {Value, _, _}} ->
       generate_match_vars(T, ClauseVars, Left, Right);
-    {ok, Clause} ->
+    {ok, {Clause, _, _}} ->
       generate_match_vars(T, ClauseVars,
-        [{var, 0, element(1, Value)} | Left],
-        [{var, 0, element(1, Clause)} | Right]);
+        [{var, 0, Value} | Left],
+        [{var, 0, Clause} | Right]);
     error ->
       generate_match_vars(T, ClauseVars,
-        [{var, 0, element(1, Value)} | Left], [Expr | Right])
+        [{var, 0, Value} | Left], [Expr | Right])
   end;
 
 generate_match_vars([], _ClauseVars, Left, Right) ->

lib/elixir/src/elixir_code_server.erl

@@ -37,6 +37,9 @@ handle_call({defmodule, Pid, Tuple}, _From, Config) ->
   {Ref, New} = defmodule(Pid, Tuple, Config),
   {reply, Ref, New};
 
+handle_call({lookup, Module}, _From, Config) ->
+  {reply, ets:lookup(elixir_modules, Module), Config};
+
 handle_call({undefmodule, Ref}, _From, Config) ->
   {reply, ok, undefmodule(Ref, Config)};
 
@@ -63,7 +66,7 @@ handle_call({compilation_status, CompilerPid}, _From, Config) ->
   {reply, CompilationStatus,
    Config#elixir_code_server{compilation_status=CompilationStatusListNew}};
 
-handle_call(retrieve_module_name, _From, Config) ->
+handle_call(retrieve_compiler_module, _From, Config) ->
   case Config#elixir_code_server.mod_pool of
     {[H | T], Counter} ->
       {reply, module_tuple(H), Config#elixir_code_server{mod_pool={T, Counter}}};
@@ -107,7 +110,7 @@ handle_cast({unload_files, Files}, Config) ->
   Unloaded = maps:without(Files, Current),
   {noreply, Config#elixir_code_server{loaded=Unloaded}};
 
-handle_cast({return_module_name, H}, #elixir_code_server{mod_pool={T, Counter}} = Config) ->
+handle_cast({return_compiler_module, H}, #elixir_code_server{mod_pool={T, Counter}} = Config) ->
   {noreply, Config#elixir_code_server{mod_pool={[H | T], Counter}}};
 
 handle_cast(Request, Config) ->

lib/elixir/src/elixir_compiler.erl

@@ -71,7 +71,7 @@ code_loading_compilation(Forms, Vars, #{line := Line} = E) ->
   S = elixir_env:env_to_scope_with_vars(E, Dict),
   {Expr, EE, _S} = elixir:quoted_to_erl(Forms, E, S),
 
-  {Module, I} = retrieve_module_name(),
+  {Module, I} = retrieve_compiler_module(),
   Fun  = code_fun(?m(E, module)),
   Form = code_mod(Fun, Expr, Line, ?m(E, file), Module, Vars),
   Args = list_to_tuple([V || {_, _, _, V} <- Vars]),
@@ -126,7 +126,7 @@ dispatch_loaded(Module, Fun, Args, Purgeable, I, E) ->
   code:delete(Module),
   if Purgeable ->
       code:purge(Module),
-      return_module_name(I);
+      return_compiler_module(I);
      true ->
        ok
   end,
@@ -151,11 +151,11 @@ code_mod(Fun, Expr, Line, File, Module, Vars) when is_binary(File), is_integer(L
     ]}
   ].
 
-retrieve_module_name() ->
-  elixir_code_server:call(retrieve_module_name).
+retrieve_compiler_module() ->
+  elixir_code_server:call(retrieve_compiler_module).
 
-return_module_name(I) ->
-  elixir_code_server:cast({return_module_name, I}).
+return_compiler_module(I) ->
+  elixir_code_server:cast({return_compiler_module, I}).
 
 allows_fast_compilation({'__block__', _, Exprs}) ->
   lists:all(fun allows_fast_compilation/1, Exprs);
@@ -216,7 +216,8 @@ no_auto_import() ->
 
 core() ->
   {ok, _} = application:ensure_all_started(elixir),
-  Update = fun(Old) -> maps:merge(Old, #{docs => false, internal => true}) end,
+  Update = fun(Old) -> maps:merge(Old, #{docs => false, internal => true,
+                                         relative_paths => false}) end,
   _ = elixir_config:update(compiler_options, Update),
   [core_file(File) || File <- core_main()].
 

lib/elixir/src/elixir_counter.erl

@@ -1,38 +0,0 @@
--module(elixir_counter).
--export([start_link/0, init/1, handle_call/3, handle_cast/2,
-  handle_info/2, terminate/2, code_change/3, next/0]).
--behaviour(gen_server).
-
--define(timeout, 30000).    %% 30 seconds
--define(limit, 4294967295). %% 2^32 - 1
-
-next() ->
-  gen_server:call(?MODULE, next, ?timeout).
-
-start_link() ->
-  gen_server:start_link({local, ?MODULE}, ?MODULE, 0, []).
-
-init(Counter) ->
-  {ok, Counter}.
-
-handle_call(next, _From, Counter) ->
-  {reply, Counter, bump(Counter)};
-handle_call(Request, _From, Counter) ->
-  {stop, {badcall, Request}, Counter}.
-
-handle_cast(Request, Counter) ->
-  {stop, {badcast, Request}, Counter}.
-
-handle_info(_Request, Counter) ->
-  {noreply, Counter}.
-
-terminate(_Reason, _Counter) ->
-  ok.
-
-code_change(_Old, Counter, _Extra) ->
-  {ok, Counter}.
-
-bump(Counter) when Counter < ?limit ->
-  Counter + 1;
-bump(_Counter) ->
-  0.

lib/elixir/src/elixir_def.erl

@@ -4,10 +4,7 @@
   take_definition/2, store_definition/6, unwrap_definitions/2,
   store_each/7, format_error/1]).
 -include("elixir.hrl").
-
 -define(last_def, {elixir, last_def}).
--define(attr, {elixir, def_table}).
--define(clauses_attr, {elixir, clauses_table}).
 
 %% Table management functions. Called internally.
 
@@ -105,49 +102,34 @@ store_definition(Meta, Line, Kind, CheckClauses, Name, Args, Guards, Body, KeepL
   store_each(CheckClauses, Kind, File, Location, Module, DefaultsLength, Function),
   [store_each(false, Kind, File, Location, Module, 0,
     default_function_for(Kind, Name, Default)) || Default <- Defaults],
-
-  make_struct_available(Kind, Module, Name, Args),
   {Name, Arity}.
 
 %% @on_definition
 
 run_on_definition_callbacks(Kind, Line, Module, Name, Args, Guards, Expr, E) ->
   Env = elixir_env:linify({Line, E}),
-  Callbacks = elixir_module:get_attribute(Module, on_definition),
+  Callbacks = ets:lookup_element(elixir_module:data_table(Module), on_definition, 2),
   _ = [Mod:Fun(Env, Kind, Name, Args, Guards, Expr) || {Mod, Fun} <- Callbacks],
   ok.
 
-make_struct_available(def, Module, '__struct__', []) ->
-  case erlang:get(elixir_compiler_pid) of
-    undefined -> ok;
-    Pid ->
-      Pid ! {struct_available, Module},
-      ok
-  end;
-make_struct_available(_, _, _, _) ->
-  ok.
-
 %% Retrieve location from meta file (if Key == keep)
 %% or @file, otherwise nil
 
 retrieve_location(Location, Module) ->
-  case get_location_attribute(Module) of
-    nil when is_tuple(Location) ->
+  case ets:take(elixir_module:data_table(Module), file) of
+    [] when is_tuple(Location) ->
       {File, Line} = Location,
       {normalize_location(File), Line};
-    nil ->
+    [] ->
       nil;
-    File when is_binary(File) ->
+    [{file, File, _, _}] when is_binary(File) ->
       'Elixir.Module':delete_attribute(Module, file),
       {normalize_location(File), 0};
-    {File, Line} when is_binary(File) andalso is_integer(Line) ->
+    [{file, {File, Line}, _, _}] when is_binary(File) andalso is_integer(Line) ->
       'Elixir.Module':delete_attribute(Module, file),
       {normalize_location(File), Line}
   end.
 
-get_location_attribute(Module) ->
-  elixir_module:get_attribute(Module, file).
-
 normalize_location(File) ->
   elixir_utils:characters_to_list(elixir_utils:relative_to_cwd(File)).
 
@@ -218,7 +200,7 @@ is_macro(_)         -> false.
 unwrap_definitions(File, Module) ->
   Table = elixir_module:defs_table(Module),
   Entries = ets:match(Table, {{def, '$1'}, '_', '_', '_', '_', '_', '_'}),
-  {All, Private} = unwrap_definition(Entries, File, Module, Table, [], []),
+  {All, Private} = unwrap_definition(lists:sort(Entries), File, Module, Table, [], []),
   Unreachable = elixir_locals:warn_unused_local(File, Module, Private),
   split_definition(All, Unreachable, [], [], [], [], [], {[], []}).
 
@@ -228,7 +210,7 @@ unwrap_definition([[Tuple] | T], File, Module, Table, All, Private) ->
 
   case [Clause || {_, Clause} <- ets:lookup(Table, {clauses, Tuple})] of
     [] ->
-      warn_bodyless_function(Ann, File, Module, Kind, Tuple),
+      warn_bodyless_function(Check, Ann, File, Module, Kind, Tuple),
       unwrap_definition(T, File, Module, Table, All, Private);
     Clauses ->
       Unwrapped = {Tuple, Kind, Ann, Location,
@@ -289,11 +271,11 @@ export(Kind, {Name, Arity}) when Kind == def; Kind == defp ->
 function_for_stored_definition(Ann, {Name, Arity}, Clauses) ->
   {function, Ann, Name, Arity, Clauses}.
 
-add_definition(_Line, nil, Body, {Head, Tail}) ->
+add_definition(_Ann, nil, Body, {Head, Tail}) ->
   {[Body | Head], Tail};
-add_definition(Line, Location, Body, {Head, Tail}) ->
-  {Head,
-   [{attribute, Line, file, Location}, Body | Tail]}.
+add_definition(Ann, Location, Body, {Head, Tail}) ->
+  FileAnn = erl_anno:set_generated(false, Ann),
+  {Head, [{attribute, FileAnn, file, Location}, Body | Tail]}.
 
 default_function_for(Kind, Name, {clause, Ann, Args, _Guards, _Exprs} = Clause)
     when Kind == defmacro; Kind == defmacrop ->
@@ -301,9 +283,10 @@ default_function_for(Kind, Name, {clause, Ann, Args, _Guards, _Exprs} = Clause)
 default_function_for(_, Name, {clause, Ann, Args, _Guards, _Exprs} = Clause) ->
   {function, Ann, Name, length(Args), [Clause]}.
 
-warn_bodyless_function(_Ann, _File, 'Elixir.Module', _Kind, _Tuple) ->
-  ok; %% Documentation for __info__
-warn_bodyless_function(Ann, File, _Module, Kind, Tuple) ->
+warn_bodyless_function(Check, _Ann, _File, Module, _Kind, _Tuple)
+    when Check == false; Module == 'Elixir.Module' ->
+  ok;
+warn_bodyless_function(_Check, Ann, File, _Module, Kind, Tuple) ->
   elixir_errors:form_warn([{line, erl_anno:line(Ann)}], File, ?MODULE, {bodyless_clause, Kind, Tuple}),
   ok.
 
@@ -318,20 +301,16 @@ store_each(Check, Kind, File, Location, Module, Defaults, {function, Ann, Name,
   Tuple   = {Name, Arity},
   HasBody = Clauses =/= [],
 
-  case ets:take(Defs, {def, Tuple}) of
+  {FinalAnn, FinalLocation, FinalDefaults} = case ets:take(Defs, {def, Tuple}) of
     [{_, StoredKind, StoredAnn, StoredFile, StoredCheck,
-        StoredLocation, {StoredDefaults, LastHasBody, LastDefaults}}] ->
-      FinalAnn = StoredAnn,
-      FinalLocation = StoredLocation,
-      FinalDefaults = {max(Defaults, StoredDefaults), HasBody, Defaults},
+        _StoredLocation, {StoredDefaults, LastHasBody, LastDefaults}}] ->
       check_valid_kind(Ann, File, Name, Arity, Kind, StoredKind),
       (Check and StoredCheck) andalso
         check_valid_clause(Ann, File, Name, Arity, Kind, Data, StoredAnn, StoredFile),
-      check_valid_defaults(Ann, File, Name, Arity, Kind, Defaults, StoredDefaults, LastDefaults, LastHasBody);
+      check_valid_defaults(Ann, File, Name, Arity, Kind, Defaults, StoredDefaults, LastDefaults, LastHasBody),
+      {StoredAnn, Location, {max(Defaults, StoredDefaults), HasBody, Defaults}};
     [] ->
-      FinalAnn = Ann,
-      FinalLocation = Location,
-      FinalDefaults = {Defaults, HasBody, Defaults}
+      {Ann, Location, {Defaults, HasBody, Defaults}}
   end,
 
   Check andalso ets:insert(Data, {?last_def, {Name, Arity}}),
@@ -350,7 +329,7 @@ check_valid_clause(Ann, File, Name, Arity, Kind, Data, StoredAnn, StoredFile) ->
     {Name, Arity} -> [];
     [] -> [];
     _ ->
-      Relative = elixir_utils:relative_to_cwd(elixir_utils:relative_to_cwd(StoredFile)),
+      Relative = elixir_utils:relative_to_cwd(StoredFile),
       elixir_errors:form_warn([{line, erl_anno:line(Ann)}], File, ?MODULE,
         {ungrouped_clause, {Kind, Name, Arity, erl_anno:line(StoredAnn), Relative}})
   end.
@@ -407,7 +386,7 @@ assert_valid_name(_Meta, _Kind, _Name, _Args, _S) ->
 %% Format errors
 
 format_error({bodyless_clause, Kind, {Name, Arity}}) ->
-  io_lib:format("bodyless clause provided for nonexistent ~ts ~ts/~B", [Kind, Name, Arity]);
+  io_lib:format("implementation not provided for predefined ~ts ~ts/~B", [Kind, Name, Arity]);
 
 format_error({no_module, {Kind, Name, Arity}}) ->
   io_lib:format("cannot define function outside module, invalid scope for ~ts ~ts/~B", [Kind, Name, Arity]);
@@ -422,7 +401,7 @@ format_error({defs_with_defaults, Name, {Kind, Arity}, {K, A}}) when Arity < A -
 
 format_error({clauses_with_defaults, {Kind, Name, Arity}}) ->
   io_lib:format(""
-    "definitions with multiple clauses and default values require a function head. Instead of:\n"
+    "definitions with multiple clauses and default values require a header. Instead of:\n"
     "\n"
     "    def foo(:first_clause, b \\\\ :default) do ... end\n"
     "    def foo(:second_clause, b) do ... end\n"
@@ -433,7 +412,7 @@ format_error({clauses_with_defaults, {Kind, Name, Arity}}) ->
     "    def foo(:first_clause, b) do ... end\n"
     "    def foo(:second_clause, b) do ... end\n"
     "\n"
-   "~ts ~ts/~B has multiple clauses and defines defaults in a clause with a body", [Kind, Name, Arity]);
+   "~ts ~ts/~B has multiple clauses and defines defaults in one or more clauses", [Kind, Name, Arity]);
 
 format_error({ungrouped_clause, {Kind, Name, Arity, OrigLine, OrigFile}}) ->
   io_lib:format("clauses for the same ~ts should be grouped together, ~ts ~ts/~B was previously defined (~ts:~B)",
@@ -449,7 +428,7 @@ format_error({invalid_def, Kind, NameAndArgs}) ->
   io_lib:format("invalid syntax in ~ts ~ts", [Kind, 'Elixir.Macro':to_string(NameAndArgs)]);
 
 format_error(invalid_args_for_bodyless_clause) ->
-  "can use only variables and \\\\ as arguments in function heads";
+  "can use only variables and \\\\ as arguments in definition header";
 
 format_error({is_record, Kind}) ->
   io_lib:format("cannot define function named ~ts is_record/2 due to compability "

lib/elixir/src/elixir_def_overridable.erl

@@ -1,6 +1,6 @@
 % Holds the logic responsible for defining overridable functions and handling super.
 -module(elixir_def_overridable).
--export([setup/1, overridable/1, overridable/2, name/2, super/2, store_pending/1,
+-export([setup/1, overridable/1, overridable/2, kind_and_name/2, super/2, store_pending/1,
          ensure_defined/4, format_error/1]).
 -include("elixir.hrl").
 -define(attr, {elixir, overridable}).
@@ -23,15 +23,21 @@ ensure_defined(Meta, Module, Tuple, S) ->
     _ -> elixir_errors:form_error(Meta, S#elixir_scope.file, ?MODULE, {no_super, Module, Tuple})
   end.
 
-%% Gets the name based on the function and stored overridables
+%% Builds a function name based on how many times the function has been
+%% overridden.
 
-name(Module, Function) ->
-  name(Module, Function, overridable(Module)).
-
-name(_Module, {Name, _} = Function, Overridable) ->
-  {Count, _, _, _} = maps:get(Function, Overridable),
+name({Name, _} = _Function, Count) when is_integer(Count) ->
   elixir_utils:atom_concat([Name, " (overridable ", Count, ")"]).
 
+%% Returns the kind (def, defp, and so on) of the given overridable function and
+%% its overridable name.
+
+kind_and_name(Module, FunctionOrMacro) ->
+  Overridable = overridable(Module),
+  {Count, Clause, _, _} = maps:get(FunctionOrMacro, Overridable),
+  {{{def, _Function}, Kind, _Line, _File, _Check, _Location, _Defaults}, _Clauses} = Clause,
+  {Kind, name(FunctionOrMacro, Count)}.
+
 %% Store
 
 store(Module, Function, GenerateName) ->
@@ -44,10 +50,20 @@ store(Module, Function, GenerateName) ->
       {{{def, {Name, Arity}}, Kind, Line, File, _Check,
        Location, {Defaults, _HasBody, _LastDefaults}}, Clauses} = Clause,
 
-      {FinalKind, FinalName} = case GenerateName of
-        true  -> {defp, name(Module, Function, Overridable)};
-        false -> {Kind, Name}
-      end,
+      %% We don't support private overridable macros, and this is checked before
+      %% getting here (in Module.make_overridable/2); we don't need to check
+      %% here. We still need to support defps because we only deprecated them,
+      %% but we should drop support for them in 2.0 and remove defp from here.
+      %% TODO: Remove on v2.0
+      {FinalKind, FinalName, FinalArity} =
+        case GenerateName of
+          false ->
+            {Kind, Name, Arity};
+          true when Kind == defmacro ->
+            {defp, elixir_utils:macro_name(name(Function, Count)), Arity + 1};
+          true when Kind == def; Kind == defp ->
+            {defp, name(Function, Count), Arity}
+        end,
 
       case elixir_compiler:get_opt(internal) of
         false ->
@@ -56,7 +72,7 @@ store(Module, Function, GenerateName) ->
           ok
       end,
 
-      Def = {function, Line, FinalName, Arity, Clauses},
+      Def = {function, Line, FinalName, FinalArity, Clauses},
       elixir_def:store_each(false, FinalKind, File, Location, Module, Defaults, Def)
   end.
 

lib/elixir/src/elixir_dispatch.erl

@@ -60,7 +60,8 @@ import_function(Meta, Name, Arity, E) ->
   end.
 
 require_function(Meta, Receiver, Name, Arity, E) ->
-  case is_element({Name, Arity}, get_optional_macros(Receiver)) of
+  Required = is_element(Receiver, ?m(E, requires)),
+  case is_element({Name, Arity}, get_macros(Receiver, Required)) of
     true  -> false;
     false ->
       elixir_lexical:record_remote(Receiver, ?m(E, function), ?m(E, lexical_tracker)),
@@ -146,7 +147,7 @@ do_expand_import(Meta, {Name, Arity} = Tuple, Args, Module, E, Result) ->
       elixir_locals:record_import(Tuple, Receiver, Module, ?m(E, function)),
       {ok, Receiver, expand_macro_named(Meta, Receiver, Name, Arity, Args, E)};
     {import, Receiver} ->
-      case expand_require([{require, false} | Meta], Receiver, Tuple, Args, E) of
+      case expand_require([{required, true} | Meta], Receiver, Tuple, Args, E) of
         {ok, _, _} = Response -> Response;
         error -> {ok, Receiver, Name, Args}
       end;
@@ -161,19 +162,15 @@ do_expand_import(Meta, {Name, Arity} = Tuple, Args, Module, E, Result) ->
 
 expand_require(Meta, Receiver, {Name, Arity} = Tuple, Args, E) ->
   check_deprecation(Meta, Receiver, Name, Arity, E),
-  Module = ?m(E, module),
+  Required = (Receiver == ?m(E, module)) orelse is_element(Receiver, ?m(E, requires)) orelse required(Meta),
 
-  case is_element(Tuple, get_optional_macros(Receiver)) of
+  case is_element(Tuple, get_macros(Receiver, Required)) of
+    true when Required ->
+      elixir_lexical:record_remote(Receiver, Name, Arity, nil, ?line(Meta), ?m(E, lexical_tracker)),
+      {ok, Receiver, expand_macro_named(Meta, Receiver, Name, Arity, Args, E)};
     true ->
-      Requires = ?m(E, requires),
-      case (Receiver == Module) orelse is_element(Receiver, Requires) orelse skip_require(Meta) of
-        true  ->
-          elixir_lexical:record_remote(Receiver, Name, Arity, nil, ?line(Meta), ?m(E, lexical_tracker)),
-          {ok, Receiver, expand_macro_named(Meta, Receiver, Name, Arity, Args, E)};
-        false ->
-          Info = {unrequired_module, {Receiver, Name, length(Args), Requires}},
-          elixir_errors:form_error(Meta, ?m(E, file), ?MODULE, Info)
-      end;
+      Info = {unrequired_module, {Receiver, Name, length(Args), ?m(E, requires)}},
+      elixir_errors:form_error(Meta, ?m(E, file), ?MODULE, Info);
     false ->
       error
   end.
@@ -202,7 +199,7 @@ expand_macro_named(Meta, Receiver, Name, Arity, Args, E) ->
 
 expand_quoted(Meta, Receiver, Name, Arity, Quoted, E) ->
   Line = ?line(Meta),
-  Next = elixir_counter:next(),
+  Next = erlang:unique_integer(),
 
   try
     elixir_exp:expand(
@@ -220,8 +217,8 @@ caller(Line, E) ->
 
 %% Helpers
 
-skip_require(Meta) ->
-  lists:keyfind(require, 1, Meta) == {require, false}.
+required(Meta) ->
+  lists:keyfind(required, 1, Meta) == {required, true}.
 
 find_dispatch(Meta, Tuple, Extra, E) ->
   case is_import(Meta) of
@@ -289,9 +286,20 @@ format_error({ambiguous_call, {Mod1, Mod2, Name, Arity}}) ->
 %% INTROSPECTION
 
 %% Do not try to get macros from Erlang. Speeds up compilation a bit.
-get_optional_macros(erlang) -> [];
+get_macros(erlang, _) -> [];
 
-get_optional_macros(Receiver) ->
+get_macros(Receiver, false) ->
+  case code:is_loaded(Receiver) of
+    {file, _} ->
+      try
+        Receiver:'__info__'(macros)
+      catch
+        error:undef -> []
+      end;
+    false -> []
+  end;
+
+get_macros(Receiver, true) ->
   case code:ensure_loaded(Receiver) of
     {module, Receiver} ->
       try
@@ -339,5 +347,14 @@ deprecation_message(Warning, Message) ->
     Message -> Warning ++ ", " ++ Message
   end.
 
+deprecation('Elixir.HashDict', _, _) ->
+  "use maps and the Map module instead";
+deprecation('Elixir.HashSet', _, _) ->
+  "use the MapSet module instead";
+deprecation('Elixir.Dict', _, _) ->
+  "use the Map module for working with maps or the Keyword module for working with keyword lists";
+deprecation('Elixir.Set', _, _) ->
+  "use the MapSet module for working with sets";
+
 deprecation(_, _, _) ->
   false.

lib/elixir/src/elixir_env.erl

@@ -18,7 +18,7 @@ new() ->
     context_modules => [],                 %% modules defined in the current context
     vars => [],                            %% a set of defined variables
     export_vars => nil,                    %% a set of variables to be exported in some constructs
-    lexical_tracker => nil}.               %% holds the lexical tracker pid
+    lexical_tracker => nil}.               %% holds the lexical tracker PID
 
 linify({Line, Env}) ->
   Env#{line := Line}.

lib/elixir/src/elixir_errors.erl

@@ -67,6 +67,11 @@ parse_error(Line, File, Error, <<>>) ->
   end,
   do_raise(Line, File, 'Elixir.TokenMissingError', Message);
 
+%% Show a nicer message for end of line
+parse_error(Line, File, <<"syntax error before: ">>, <<"eol">>) ->
+  do_raise(Line, File, 'Elixir.SyntaxError',
+           <<"unexpectedly reached end of line. The current expression is invalid or incomplete">>);
+
 %% Show a nicer message for missing end tokens
 parse_error(Line, File, <<"syntax error before: ">>, <<"'end'">>) ->
   do_raise(Line, File, 'Elixir.SyntaxError', <<"unexpected token: end">>);
@@ -78,7 +83,7 @@ parse_error(Line, File, <<"syntax error before: ">>, <<"{sigil,", _Rest/binary>>
     true -> Content;
     false -> <<>>
   end,
-  Message = <<"syntax error before: sigil ~", Sigil, " starting with content '", Content2/binary, "'">>,
+  Message = <<"syntax error before: sigil \~", Sigil, " starting with content '", Content2/binary, "'">>,
   do_raise(Line, File, 'Elixir.SyntaxError', Message);
 
 %% Aliases are wrapped in ['']
@@ -135,6 +140,11 @@ handle_file_warning(_, _File, {_Line, v3_kernel, bad_call}) -> ok;
 %% We handle unused local warnings ourselves
 handle_file_warning(_, _File, {_Line, erl_lint, {unused_function, _}}) -> ok;
 
+%% Properly format keys using inspect.
+handle_file_warning(_, File, {Line, v3_core, {map_key_repeated, Key}}) ->
+  Message = io_lib:format("key ~ts will be overridden in map", ['Elixir.Kernel':inspect(Key)]),
+  warn(Line, File, Message);
+
 %% Make no_effect clauses pretty
 handle_file_warning(_, File, {Line, sys_core_fold, {no_effect, {erlang, F, A}}}) ->
   {Fmt, Args} = case erl_internal:comp_op(F, A) of
@@ -165,8 +175,7 @@ handle_file_warning(_, File, {Line, erl_lint, {undefined_behaviour, Module}}) ->
   case elixir_compiler:get_opt(internal) of
     true  -> ok;
     false ->
-      Message = io_lib:format("behaviour ~ts undefined", [elixir_aliases:inspect(Module)]),
-      warn(Line, File, Message)
+      warn(Line, File, ["behaviour ", elixir_aliases:inspect(Module), " is undefined"])
   end;
 
 %% Ignore unused vars at "weird" lines (<= 0)
@@ -186,8 +195,7 @@ handle_file_warning(_, File, {Line, sys_core_fold, nomatch_guard}) ->
 
 %% Properly format other unused vars
 handle_file_warning(_, File, {Line, erl_lint, {unused_var, Var}}) ->
-  Message = format_error(erl_lint, {unused_var, format_var(Var)}),
-  warn(Line, File, Message);
+  warn(Line, File, ["variable \"", format_var(Var), "\" is unused"]);
 
 %% Handle literal eval failures
 handle_file_warning(_, File, {Line, sys_core_fold, {eval_failure, Error}}) ->
@@ -210,7 +218,7 @@ handle_file_error(File, {Line, erl_lint, {unsafe_var, Var, {In, _Where}}}) ->
     'andalso' -> 'and';
     _ -> In
   end,
-  Message = io_lib:format("cannot define variable ~ts inside ~ts", [format_var(Var), Translated]),
+  Message = io_lib:format("cannot define variable \"~ts\" inside ~ts", [format_var(Var), Translated]),
   do_raise(Line, File, 'Elixir.CompileError', elixir_utils:characters_to_binary(Message));
 
 handle_file_error(File, {Line, erl_lint, {undefined_function, {F, A}}}) ->
@@ -242,11 +250,11 @@ file_format(Line, File) ->
   io_lib:format("~ts:~w", [elixir_utils:relative_to_cwd(File), Line]).
 
 format_var(Var) ->
-  list_to_atom(lists:takewhile(fun(X) -> X /= $@ end, atom_to_list(Var))).
+  lists:takewhile(fun(X) -> X /= $@ end, atom_to_list(Var)).
 
 %% TODO: Remove this clause when we depend only on Erlang 19.
 format_error(erl_lint, {bittype_mismatch, Val1, Val2, Kind}) ->
-  Desc = "conflict in ~s specification for bit field: '~p' and '~p'",
+  Desc = "conflict in ~s specification for bit field: \"~p\" and \"~p\"",
   io_lib:format(Desc, [Kind, Val1, Val2]);
 
 format_error([], Desc) ->