Release v0.12.2

The main reason for this change is because `Enum.first/1` does
not communicate that enumeration will happen, in contrast to
functions like Enum.at/2 and Enum.fetch/2 which are meant to
be used with non-indexed collections and the need for enumeration
is made a bit more obvious.

Closes #1983

.gitignore

@@ -1,5 +1,7 @@
-/.eunit/*
+/.eunit
 /.release
+/docs
+/ebin
 /lib/*/ebin/*
 /lib/*/tmp
 /lib/*/test/tmp
@@ -7,11 +9,7 @@
 /lib/elixir/src/*_lexer.erl
 /lib/elixir/src/*_parser.erl
 /lib/elixir/test/ebin
-/deps/*
-/ebin
 /rel/elixir
 erl_crash.dump
 .dialyzer_plt
 .dialyzer.base_plt
-.*.swp
-docs

.travis.yml

@@ -1,9 +1,11 @@
 language: erlang
-script: "make compile && make test"
+script: "make compile && rm -rf .git && make test"
 notifications:
   irc: "irc.freenode.org#elixir-lang"
   recipients:
     - jose.valim@plataformatec.com.br
-    - yrashk@gmail.com
+    - eric.meadows.jonsson@gmail.com
 otp_release:
+  - R16B02
+  - R16B01
   - R16B

CHANGELOG.md

@@ -1,9 +1,208 @@
+# v0.12.2 (2014-01-15)
+
+* Enhancements
+  * [EEx] Allow `EEx.AssignsEngine` to accept any Dict
+  * [Enum] Add `Enum.flat_map_reduce/3`
+  * [ExUnit] Support @moduletag in ExUnit cases
+  * [Kernel] Improve stacktraces to be relative to the compilation path and include the related application
+  * [Stream] Add `Stream.transform/3`
+
+* Bug fixes
+  * [ExUnit] `:include` in ExUnit only has effect if a test was previously excluded with `:exclude`
+  * [ExUnit] Only run `setup_all` and `teardown_all` if there are tests in the case
+  * [Kernel] Ensure bitstring modifier arguments are expanded
+  * [Kernel] Ensure compiler does not block on missing modules
+  * [Kernel] Ensure `<>/2` works only with binaries
+  * [Kernel] Fix usage of string literals inside `<<>>` when `utf8`/`utf16`/`utf32` is used as specifier
+  * [Mix] Ensure mix properly copies _build dependencies on Windows
+
+* Deprecations
+  * [Enum] Deprecate `Enum.first/1` in favor of `Enum.at/2` and `List.first/1`
+  * [Kernel] Deprecate continuable heredocs. In previous versions, Elixir would continue parsing on the same line the heredoc started, this behavior has been deprecated
+  * [Kernel] `is_alive/0` is deprecated in favor of `Node.alive?`
+  * [Kernel] `Kernel.inspect/2` with `Inspect.Opts[]` is deprecated in favor of `Inspect.Algebra.to_doc/2`
+  * [Kernel] `Kernel.inspect/2` with `:raw` option is deprecated, use `:records` option instead
+  * [Kernel] Deprecate `<-/2` in favor of `send/2`
+
+* Backwards incompatible changes
+  * [String] Change `String.next_grapheme/1` and `String.next_codepoint/1` to return `nil` on string end
+
+# v0.12.1 (2014-01-04)
+
+* Enhancements
+  * [ExUnit] Support `:include` and `:exclude` configuration options to filter which tests should run based on their tags. Those options are also supported via `mix test` as `--include` and `--exclude`
+  * [ExUnit] Allow doctests to match against `#MyModule<>`
+
+* Bug fixes
+  * [CLI] Abort when a pattern given to elixirc does not match any file
+  * [Float] Fix `Float.parse/1` to handle numbers of the form "-0.x"
+  * [IEx] Improve error message for `IEx.Helpers.r` when module does not exist
+  * [Mix] Ensure `deps.get` updates origin if lock origin and dep origin do not match
+  * [Mix] Use relative symlinks in _build
+  * [Typespec] Fix conversion of unary ops from typespec format to ast
+  * [Typespec] Fix handling of `tuple()` and `{}`
+
+* Deprecations
+  * [Kernel] Do not leak clause heads. Previously, a variable defined in a case/receive head clauses would leak to the outer scope. This behaviour is deprecated and will be removed in the next release.
+  * [Kernel] Deprecate `__FILE__` in favor of `__DIR__` or `__ENV__.file`
+
+* Backwards incompatible changes
+  * [GenFSM] GenServer now stops on unknown event/sync_event requests
+  * [GenServer] GenServer now stops on unknown call/cast requests
+  * [Kernel] Change how `->` is represented in AST. Now each clause is represented by its own AST node which makes composition easier. See commit 51aef55 for more information.
+
+# v0.12.0 (2013-12-15)
+
+* Enhancements
+  * [Exception] Allow `exception/1` to be overridden and promote it as the main mechanism to customize exceptions
+  * [File] Add `File.stream_to!/3`
+  * [Float] Add `Float.floor/1`, `Float.ceil/1` and `Float.round/3`
+  * [Kernel] Add `List.delete_at/2` and `List.updated_at/3`
+  * [Kernel] Add `Enum.reverse/2`
+  * [Kernel] Implement `defmodule/2`, `@/1`, `def/2` and friends in Elixir itself. `case/2`, `try/2` and `receive/1` have been made special forms. `var!/1`, `var!/2` and `alias!/1` have also been implemented in Elixir and demoted from special forms
+  * [Record] Support dynamic fields in `defrecordp`
+  * [Stream] Add `Stream.resource/3`
+  * [Stream] Add `Stream.zip/2`, `Stream.filter_map/3`, `Stream.each/2`, `Stream.take_every/2`, `Stream.chunk/2`, `Stream.chunk/3`, `Stream.chunk/4`, `Stream.chunk_by/2`, `Stream.scan/2`, `Stream.scan/3`, `Stream.uniq/2`, `Stream.after/2` and `Stream.run/1`
+  * [Stream] Support `Stream.take/2` and `Stream.drop/2` with negative counts
+
+* Bug fixes
+  * [HashDict] Ensure a `HashDict` stored in an attribute can be accessed via the attribute
+  * [Enum] Fix bug in `Enum.chunk/4` where you'd get an extra element when the enumerable was a multiple of the counter and a pad was given
+  * [IEx] Ensure `c/2` helper works with full paths
+  * [Kernel] `quote location: :keep` now only affects definitions in order to keep the proper trace in definition exceptions
+  * [Mix] Also symlink `include` directories in _build dependencies
+  * [Version] Fix `Version.match?/2` with `~>` and versions with alphanumeric build info (like `-dev`)
+
+* Deprecations
+  * [Enum] `Enumerable.count/1` and `Enumerable.member?/2` should now return tagged tuples. Please see `Enumerable` docs for more info
+  * [Enum] Deprecate `Enum.chunks/2`, `Enum.chunks/4` and `Enum.chunks_by/2` in favor of `Enum.chunk/2`, `Enum.chunk/4` and `Enum.chunk_by/2`
+  * [File] `File.binstream!/3` is deprecated. Simply use `File.stream!/3` which is able to figure out if `stream` or `binstream` operations should be used
+  * [Macro] `Macro.extract_args/1` is deprecated in favor of `Macro.decompose_call/1`
+
+* Backwards incompatible changes
+  * [Enum] Behaviour of `Enum.drop/2` and `Enum.take/2` has been switched when given negative counts
+  * [Enum] Behaviour of `Enum.zip/2` has been changed to stop as soon as the first enumerable finishes
+  * [Enum] `Enumerable.reduce/3` protocol has changed to support suspension. Please see `Enumerable` docs for more info
+  * [Mix] Require `:escript_main_module` to be set before generating escripts
+  * [Range] `Range.Iterator` protocol has changed in order to work with the new `Enumerable.reduce/3`. Please see `Range.Iterator` docs for more info
+  * [Stream] The `Stream.Lazy` structure has changed to accumulate functions and accumulators as we go (its inspected representation has also changed)
+  * [Typespec] `when` clauses were moved to the outer part of the spec and should be in the keywords format. So `add(a, b) when is_subtype(a, integer) and is_subtype(b, integer) :: integer` should now be written as `add(a, b) :: integer when a: integer, b: integer`
+
+# v0.11.2 (2013-11-14)
+
+* Enhancements
+  * [Mix] Add `mix iex` that redirects users to the proper `iex -S mix` command
+  * [Mix] Support `build_per_environment: true` in project configuration that manages a separete build per environment, useful when you have per-environment behaviour/compilation
+
+* Backwards incompatible changes
+  * [Mix] Mix now compiles files to `_build`. Projects should update just fine, however documentation and books may want to update to the latest information
+
+# v0.11.1 (2013-11-07)
+
+* Enhancements
+  * [Mix] Improve dependency convergence by explicitly checking each requirement instead of expecting all requirements to be equal
+  * [Mix] Support optional dependencies with `optional: true`. Optional dependencies are downloaded for the current project but they are automatically skipped when such project is used as a dependency
+
+* Bug fixes
+  * [Kernel] Set compilation status per ParallelCompiler and not globally
+  * [Mix] Ensure Mix does not load previous dependencies versions before `deps.get`/`deps.update`
+  * [Mix] Ensure umbrella apps are sorted before running recursive commands
+  * [Mix] Ensure umbrella apps run in the same environment as the parent project
+  * [Mix] Ensure dependency tree is topsorted before compiling
+  * [Mix] Raise error when duplicated projects are pushed into the stack
+  * [URI] Allow lowercase escapes in URI
+
+* Backwards incompatible changes
+  * [Mix] Setting `:load_paths` in your project configuration is deprecated
+
+# v0.11.0 (2013-11-02)
+
+* Enhancements
+  * [Code] Eval now returns variables from other contexts
+  * [Dict] Document and enforce all dicts use the match operator (`===`) when checking for keys
+  * [Enum] Add `Enum.slice/2` with a range
+  * [Enum] Document and enforce `Enum.member?/2` to use the match operator (`===`)
+  * [IEx] Split `IEx.Evaluator` from `IEx.Server` to allow custom evaluators
+  * [IEx] Add support for `IEx.pry` which halts a given process for inspection
+  * [IO] Add specs and allow some IO APIs to receive any data that implements `String.Chars`
+  * [Kernel] Improve stacktraces on command line interfaces
+  * [Kernel] Sigils can now handle balanced tokens as in `%s(f(o)o)`
+  * [Kernel] Emit warnings when an alias is not used
+  * [Macro] Add `Macro.pipe/3` and `Macro.unpipe/1` for building pipelines
+  * [Mix] Allow umbrella children to share dependencies between them
+  * [Mix] Allow mix to be escriptize'd
+  * [Mix] Speed mix projects compilation by relying on more manifests information
+  * [Protocol] Protocols now provide `impl_for/1` and `impl_for!/1` functions which receive a structure and returns its respective implementation, otherwise returns nil or an error
+  * [Set] Document and enforce all sets use the match operator (`===`) when checking for keys
+  * [String] Update to Unicode 6.3.0
+  * [String] Add `String.slice/2` with a range
+
+* Bug fixes
+  * [Exception] Ensure `defexception` fields can be set dynamically
+  * [Kernel] Guarantee aliases hygiene is respected when the current module name is not known upfront
+  * [Kernel] `Kernel.access/2` no longer flattens lists
+  * [Mix] Ensure cyclic dependencies are properly handled
+  * [String] Implement the extended grapheme cluster algorithm for `String` operations
+
+* Deprecations
+  * [Kernel] `pid_to_list/1`, `list_to_pid/1`, `binary_to_atom/2`, `binary_to_existing_atom/2` and `atom_to_binary/2` are deprecated in favor of their counterparts in the `:erlang` module
+  * [Kernel] `insert_elem/3` and `delete_elem/2` are deprecated in favor of `Tuple.insert_at/3` and `Tuple.delete_at/2`
+  * [Kernel] Use of `in` inside matches (as in `x in [1,2,3] -> x`) is deprecated in favor of the guard syntax (`x when x in [1,2,3]`)
+  * [Macro] `Macro.expand_all/2` is deprecated
+  * [Protocol] `@only` and `@except` in protocols are now deprecated
+  * [Protocol] Protocols no longer fallback to `Any` out of the box (this functionality needs to be explicitly enabled by setting `@fallback_to_any` to true)
+  * [String] `String.to_integer/1` and `String.to_float/1` are deprecated in favor of `Integer.parse/1` and `Float.parse/1`
+
+* Backwards incompatible changes
+  * [CLI] Reading `.elixirrc` has been dropped in favor of setting env vars
+  * [Kernel] `Kernel.access/2` now expects the second argument to be a compile time list
+  * [Kernel] `fn -> end` quoted expression is no longer wrapped in a `do` keyword
+  * [Kernel] Quoted variables from the same module must be explicitly shared. Previously, if a function returned `quote do: a = 1`, another function from the same module could access it as `quote do: a`. This has been fixed and the variables must be explicitly shared with `var!(a, __MODULE__)`
+  * [Mix] Umbrella apps now treat children apps as dependencies. This means all dependencies will be checked out in the umbrela `deps` directory. On upgrade, child apps need to point to the umbrella project by setting `deps_path: "../../deps_path", lockfile: "../../mix.lock"` in their project config
+  * [Process] `Process.group_leader/2` args have been reversed so the "subject" comes first
+  * [Protocol] Protocol no longer dispatches to `Number`, but to `Integer` and `Float`
+
+# v0.10.3 (2013-10-02)
+
+* Enhancements
+  * [Enum] Add `Enum.take_every/2`
+  * [IEx] IEx now respects signals sent from the Ctrl+G menu
+  * [Kernel] Allow documentation for types with `@typedoc`
+  * [Mix] Allow apps to be selected in umbrella projects
+  * [Record] Generated record functions `new` and `update` also take options with strings as keys
+  * [Stream] Add `Stream.unfold/1`
+
+* Bug fixes
+  * [Dict] Fix a bug when a HashDict was marked as equal when one was actually a subset of the other
+  * [EEx] Solve issue where `do` blocks inside templates were not properly aligned
+  * [ExUnit] Improve checks and have better error reports on poorly aligned doctests
+  * [Kernel] Fix handling of multiple heredocs on the same line
+  * [Kernel] Provide better error messages for match, guard and quoting errors
+  * [Kernel] Make `Kernel.raise/2` a macro to avoid messing up stacktraces
+  * [Kernel] Ensure `&()` works on quoted blocks with only one expression
+  * [Mix] Address an issue where a dependency was not compiled in the proper order when specified in different projects
+  * [Mix] Ensure `compile: false` is a valid mechanism for disabling the compilation of dependencies
+  * [Regex] Fix bug on `Regex.scan/3` when capturing groups and the regex has no groups
+  * [String] Fix a bug with `String.split/2` when given an empty pattern
+  * [Typespec] Guarantee typespecs error reports point to the proper line
+
+* Deprecations
+  * [Kernel] The previous partial application syntax (without the `&` operator) has now been deprecated
+  * [Regex] `Regex.captures/3` is deprecated in favor of `Regex.named_captures/3`
+  * [String] `String.valid_codepoint?/1` is deprecated in favor of pattern matching with `<<_ :: utf8 >>`
+
+* Backwards incompatible changes
+  * [IEx] The `r/0` helper has been removed as it caused surprising behaviour when many modules with dependencies were accumulated
+  * [Mix] `Mix.Version` was renamed to `Version`
+  * [Mix] `File.IteratorError` was renamed to `IO.StreamError`
+  * [Mix] `mix new` now defaults to the `--sup` option, use `--bare` to get the previous behaviour
+
 # v0.10.2 (2013-09-03)
 
-* enhancements
+* Enhancements
   * [CLI] Add `--verbose` to elixirc, which now is non-verbose by default
   * [Dict] Add `Dict.Behaviour` as a convenience to create your own dictionaries
-  * [Enum] Add `Enum.split/2`, `Enum.reduce/2`, `Enum.flat_map/2`, `Enum.chunks/2`, `Enum.chunks/4`, `Enum.chunks_by/2`, `Enum.concat/1` and `Enum.concat/2`
+  * [Enum] Add `Enum.split/2`, `Enum.reduce/2`, `Enum.flat_map/2`, `Enum.chunk/2`, `Enum.chunk/4`, `Enum.chunk_by/2`, `Enum.concat/1` and `Enum.concat/2`
   * [Enum] Support negative indices in `Enum.at/fetch/fetch!`
   * [ExUnit] Show failures on CLIFormatter as soon as they pop up
   * [IEx] Allow for strings in `h` helper
@@ -24,7 +223,7 @@
   * [String] Add `String.ljust/2`, `String.rjust/2`, `String.ljust/3` and `String.rjust/3`
   * [URI] `URI.parse/1` supports IPv6 addresses
 
-* bug fix
+* Bug fixes
   * [Behaviour] Do not compile behaviour docs if docs are disabled on compilation
   * [ExUnit] Doctests no longer eat too much space and provides detailed reports for poorly indented lines
   * [File] Fix a bug where `File.touch(file, datetime)` was not setting the proper datetime when the file did not exist
@@ -42,7 +241,7 @@
   * [String] `String.slice/3` still returns the proper result when there is no length to be extracted
   * [System] `System.get_env/0` now returns a list of tuples as previously advertised
 
-* deprecations
+* Deprecations
   * [Dict] `Dict.update/3` is deprecated in favor of `Dict.update!/3`
   * [Enum] `Enum.min/2` and `Enum.max/2` are deprecated in favor of `Enum.min_by/2` and `Enum.max_by/2`
   * [Enum] `Enum.join/2` and `Enum.map_join/3` with a char list are deprecated
@@ -55,7 +254,7 @@
   * [Macro] `Macro.unescape_binary/1` and `Macro.unescape_binary/2` are deprecated in favor of `Macro.unescape_string/1` and `Macro.unescape_string/2`
   * [Mix] `:umbrella` option for umbrella paths has been deprecated in favor of `:in_umbrella`
 
-* backwards incompatible changes
+* Backwards incompatible changes
   * [IO] IO functions now only accept iolists as arguments
   * [Kernel] `Binary.Chars` was renamed to `String.Chars`
   * [Kernel] The previous ambiguous import syntax `import :functions, Foo` was removed in favor of `import Foo, only: :functions`
@@ -63,7 +262,7 @@
 
 # v0.10.1 (2013-08-03)
 
-* enhancements
+* Enhancements
   * [Behaviour] Add support for `defmacrocallback/1`
   * [Enum] Add `Enum.shuffle/1`
   * [ExUnit] The `:trace` option now also reports run time for each test
@@ -86,7 +285,7 @@
   * [String] Add `Regex.scan/3` now supports capturing groups
   * [String] Add `String.reverse/1`
 
-* bug fix
+* Bug fixes
   * [Behaviour] Ensure callbacks are stored in the definition order
   * [CLI] Speed up boot time on Elixir .bat files
   * [IEx] Reduce cases where IEx parser can get stuck
@@ -101,19 +300,19 @@
   * [OptionParser] Ensure `:keep` and `:integer`|`:float` can be given together as options
   * [OptionParser] Ensure `--no-flag` sets `:flag` to false when `:flag` is a registered boolean switch
 
-* deprecations
+* Deprecations
   * [Kernel] `function(Mod.fun/arity)` and `function(fun/arity)` are deprecated in favor of `&Mod.fun/arity` and `&fun/arity`
   * [Kernel] `function/3` is deprecated in favor of `Module.function/3`
   * [Kernel] `Kernel.ParallelCompiler` now receives a set of callbacks instead of a single one
   * [Mix] `:test_coverage` option now expect keywords arguments and the `--cover` flag is now treated as a boolean
 
-* backwards incompatible changes
+* Backwards incompatible changes
   * [Regex] `Regex.scan/3` now always returns a list of lists, normalizing the result, instead of list with mixed lists and binaries
   * [System] `System.halt/2` was removed since the current Erlang implementation of such function is bugged
 
 # v0.10.0 (2013-07-15)
 
-* enhancements
+* Enhancements
   * [ExUnit] Support `trace: true` option which gives detailed reporting on test runs
   * [HashDict] Optimize `HashDict` to store pairs in a cons cell reducing storage per key by half
   * [Kernel] Add pretty printing support for inspect
@@ -126,7 +325,7 @@
   * [Mix] Projects can now define an `:elixir` key to outline supported Elixir versions
   * [Typespec] Improve error messages to contain file, line and the typespec itself
 
-* bug fix
+* Bug fixes
   * [CLI] Elixir can now run on Unix directories with `:` in its path
   * [Kernel] `match?/2` does not leak variables to outer scope
   * [Kernel] Keep `head|tail` format when splicing at the tail
@@ -135,7 +334,7 @@
   * [Mix] Empty projects no longer leave empty ebin files on `mix compile`
   * [Module] Calling `Module.register_attribute/3` no longer automatically changes it to persisted or accumulated
 
-* deprecations
+* Deprecations
   * [Enum] Receiving the index of iteration in `Enum.map/2` and `Enum.each/2` is deprecated in favor of `Stream.with_index/1`
   * [File] `File.iterator/1` and `File.biniterator/1` are deprecated in favor of `IO.stream/1` and `IO.binstream/1`
   * [File] `File.iterator!/2` and `File.biniterator!/2` are deprecated in favor of `File.stream!/2` and `File.binstream!/2`
@@ -144,7 +343,7 @@
   * [Mix] Deprecate `mix run EXPR` in favor of `mix run -e EXPR`
   * [Record] `Record.__index__/2` deprecated in favor of `Record.__record__(:index, key)`
 
-* backwards incompatible changes
+* Backwards incompatible changes
   * [Kernel] The `Binary.Inspect` protocol has been renamed to `Inspect`
   * [Kernel] Tighten up the grammar rules regarding parentheses omission, previously the examples below would compile but now they raise an error message:
 
@@ -157,7 +356,7 @@
 
 # v0.9.3 (2013-06-23)
 
-* enhancements
+* Enhancements
   * [File] Add `File.chgrp`, `File.chmod` and `File.chown`
   * [Kernel] Add `--warnings-as-errors` to Elixir's compiler options
   * [Kernel] Print warnings to stderr
@@ -171,7 +370,7 @@
   * [Mix] `github: ...` shortcut now uses the faster `git` schema instead of `https`
   * [Record] Allow types to be given to `defrecordp`
 
-* bug fix
+* Bug fixes
   * [Kernel] The elixir executable on Windows now supports the same options as the UNIX one
   * [Kernel] Improve error messages on default clauses clash
   * [Kernel] `__MODULE__.Foo` now returns `Foo` when outside of a Module
@@ -180,22 +379,22 @@
   * [Kernel] Allow `(x, y) when z` in function clauses and try expressions
   * [Mix] Mix now properly evaluates rebar scripts
 
-* deprecations
+* Deprecations
   * [Code] `Code.string_to_ast/1` has been deprecated in favor of `Code.string_to_quoted/1`
   * [Macro] `Macro.to_binary/1` has been deprecated in favor of `Macro.to_string/1`
   * [Typespec] Deprecate `(fun(...) -> ...)` in favor of `(... -> ...)`
 
-* backwards incompatible changes
+* Backwards incompatible changes
   * [Bitwise] Precedence of operators used by the Bitwise module were changed, check `elixir_parser.yrl` for more information
   * [File] `rm_rf` and `cp_r` now returns a tuple with three elements on failures
   * [Kernel] The quoted representation for `->` clauses changed from a tuple with two elements to a tuple with three elements to support metadata
-  * [Kernel] Sigils now dispatch to `sigil_$` instead of `__$__` where `$` is the sigil caracter
+  * [Kernel] Sigils now dispatch to `sigil_$` instead of `__$__` where `$` is the sigil character
   * [Macro] `Macro.expand/2` now expands until final form. Although this is backwards incompatible, it is very likely you do not need to change your code, since expansion until its final form is recommended, particularly if you are expecting an atom out of it
   * [Mix] No longer support beam files on `mix local`
 
 # v0.9.2 (2013-06-13)
 
-* enhancements
+* Enhancements
   * [ExUnit] `capture_io` now captures prompt by default
   * [Mix] Automatically import git dependencies from Rebar
   * [Mix] Support for dependencies directly from the umbrella application
@@ -203,7 +402,7 @@
   * [String] Add `String.contains?`
   * [URI] Implement `Binary.Chars` (aka `to_binary`) for `URI.Info`
 
-* bug fix
+* Bug fixes
   * [HashDict] Ensure HashDict uses exact match throughout its implementation
   * [IEx] Do not interpret ANSI codes in IEx results
   * [IEx] Ensure `--cookie` is set before accessing remote shell
@@ -211,21 +410,21 @@
   * [Mix] Fix usage of shell expressions in `Mix.Shell.cmd`
   * [Mix] Start the application by default on escripts
 
-* deprecations
+* Deprecations
   * [Regex] `Regex.index/2` is deprecated in favor `Regex.run/3`
   * [Kernel] `super` no longer supports implicit arguments
 
-* backwards incompatible changes
+* Backwards incompatible changes
   * [Kernel] The `=~` operator now returns true or false instead of an index
 
 # v0.9.1 (2013-05-30)
 
-* enhancements
+* Enhancements
   * [IEx] Limit the number of entries kept in history and allow it to be configured
   * [Kernel] Add `String.start_with?` and `String.end_with?`
   * [Typespec] Allow keywords, e.g. `[foo: integer, bar: boolean | module]`, in typespecs
 
-* bug fix
+* Bug fixes
   * [Dict] `Enum.to_list` and `Dict.to_list` now return the same results for dicts
   * [IEx] Enable shell customization via the `IEx.Options` module
   * [Kernel] Fix a bug where `unquote_splicing` did not work on the left side of a stab op
@@ -236,12 +435,12 @@
 
 # v0.9.0 (2013-05-23)
 
-* enhancements
+* Enhancements
   * [ExUnit] `ExUnit.CaptureIO` now accepts an input to be used during capture
   * [IEx] Add support for .iex files that are loaded during shell's boot process
   * [IEx] Add `import_file/1` helper
 
-* backwards incompatible changes
+* Backwards incompatible changes
   * [Enum] `Enum.Iterator` was replaced by the more composable and functional `Enumerable` protocol which supports reductions
   * [File] `File.iterator/1` and `File.biniterator/1` have been removed in favor of the safe `File.iterator!/1` and `File.biniterator!/1` ones
   * [Kernel] Erlang R15 is no longer supported
@@ -249,7 +448,7 @@
 
 # v0.8.3 (2013-05-22)
 
-* enhancements
+* Enhancements
   * [CLI] Flags `-p` and `-pr` fails if pattern match no files
   * [CLI] Support `--hidden` and `--cookie` flags for distributed Erlang
   * [Enum] Add `Enum.to_list/1`, `Enum.member?/2`, `Enum.uniq/2`, `Enum.max/1`, `Enum.max/2`, `Enum.min/1` and `Enum.min/2`
@@ -272,7 +471,7 @@
   * [Mix] Load beam files available at `MIX_PATH` on CLI usage
   * [String] Add `String.valid?` and `String.valid_character?`
 
-* bug fix
+* Bug fixes
   * [ExUnit] Handle exit messages from in ExUnit
   * [ExUnit] Failures on ExUnit's setup_all now invalidates all tests
   * [Kernel] Ensure we don't splice keyword args unecessarily
@@ -284,7 +483,7 @@
   * [Regex] Fix badmatch with `Regex.captures(%r/(.)/g, "cat")`
   * [URI] Downcase host and scheme and URIs
 
-* deprecations
+* Deprecations
   * [Code] `Code.eval` is deprecated in favor of `Code.eval_string`
   * [Exception] `Exception.format_entry` is deprecated in favor of `Exception.format_stacktrace_entry`
   * [ExUnit] `assert left inlist right` is deprecated in favor of `assert left in right`
@@ -293,7 +492,7 @@
   * [Kernel] `var_context` in quote was deprecated in favor of `context`
   * [Kernel] `Enum.at!` and `Dict.get!` is deprecated in favor of `Enum.fetch!` and `Dict.fetch!`
 
-* backwards incompatible changes
+* Backwards incompatible changes
   * [Dict] `List.Dict` was moved to `ListDict`
   * [IO] `IO.gets`, `IO.getn` and friends now return binaries when reading from stdio
   * [Kernel] Precedence of `|>` has changed to lower to support constructs like `1..5 |> Enum.to_list`
@@ -301,7 +500,7 @@
 
 # v0.8.2 (2013-04-20)
 
-* enhancements
+* Enhancements
   * [ExUnit] Use ANSI escape codes in CLI output
   * [ExUnit] Include suite run time on CLI results
   * [ExUnit] Add support to doctests, allowing test cases to be generated from code samples
@@ -314,7 +513,7 @@
   * [Mix] Support `--cover` on mix test and `test_coverage` on Mixfiles
   * [Record] Each record now provides `Record.options` with the options supported by its `new` and `update` functions
 
-* bug fix
+* Bug fixes
   * [Binary] inspect no longer escapes standalone hash `#`
   * [IEx] The `h` helper can now retrieve docs for special forms
   * [Kernel] Record optimizations were not being triggered in functions inside the record module
@@ -322,11 +521,11 @@
   * [Kernel] Fix a bug where nested records could not use the Record[] syntax
   * [Path] Fix a bug on `Path.expand` when expanding paths starting with `~`
 
-* deprecations
+* Deprecations
   * [Kernel] `setelem/3` is deprecated in favor of `set_elem/3`
   * [Kernel] `function(:is_atom, 1)` is deprecated in favor of `function(is_atom/1)`
 
-* backwards incompatible changes
+* Backwards incompatible changes
   * [Kernel] `unquote` now only applies to the closest quote. If your code contains a quote that contains another quote that calls unquote, it will no longer work. Use `Macro.escape` instead and pass your quoted contents up in steps, for example:
 
             quote do
@@ -341,7 +540,7 @@
 
 # v0.8.1 (2013-02-17)
 
-* enhancements
+* Enhancements
   * [ExUnit] Tests can now receive metadata set on setup/teardown callbacks
   * [ExUnit] Add support to ExUnit.CaseTemplate to share callbacks in between test cases
   * [IO] Add `IO.ANSI` to make it easy to write ANSI escape codes
@@ -354,13 +553,13 @@
   * [OptionParser] Switches are now overridden by default but can be kept in order if chosen
   * [Typespec] Better error reporting for invalid typespecs
 
-* bug fix
+* Bug fixes
   * [Mix] Allow Mix projects to be generated with just one letter
 
-* backwards incompatible changes
+* Backwards incompatible changes
   * [Kernel] `before_compile` and `after_compile` callbacks now receive the environment as first argument instead of the module
 
-* deprecations
+* Deprecations
   * [ExUnit] Explicitly defined test/setup/teardown functions are deprecated
   * [Kernel] Tidy up and clean `quote` API
   * [Kernel] Old `:local.(args)` syntax is deprecated
@@ -368,7 +567,7 @@
 
 # v0.8.0 (2013-01-28)
 
-* enhancements
+* Enhancements
   * [Binary] Support `<< "string" :: utf8 >>` as in Erlang
   * [Binary] Support `\a` escape character in binaries
   * [Binary] Support syntax shortcut for specifying size in bit syntax
@@ -391,16 +590,16 @@
   * [String] Add `String.capitalize` and `String.slice`
   * [System] Add `System.tmp_dir`, `System.cwd` and `System.user_home`
 
-* bug fix
+* Bug fixes
   * [Kernel] `import` with `only` accepts functions starting with underscore
   * [String] `String.first` and `String.last` return nil for empty binaries
   * [String] `String.rstrip` and `String.lstrip` now verify if argument is a binary
   * [Typespec] Support `...` inside typespec's lists
 
-* backwards incompatible changes
+* Backwards incompatible changes
   * [Kernel] The AST now allows metadata to be attached to each node. This means the second item in the AST is no longer an integer (representing the line), but a keywords list. Code that relies on the line information from AST or that manually generate AST nodes need to be properly updated
 
-* deprecations
+* Deprecations
   * [Dict] Deprecate `Binary.Dict` and `OrdDict` in favor of `HashDict` and `ListDict`
   * [File] Deprecate path related functions in favor of the module `Path`
   * [Kernel] The `/>` operator has been deprecated in favor of `|>`
@@ -410,7 +609,7 @@
 
 # v0.7.2 (2012-12-04)
 
-* enhancements
+* Enhancements
   * [CLI] `--debug-info` is now true by default
   * [ExUnit] Make ExUnit exit happen in two steps allowing developers to add custom `at_exit` hooks
   * [IEx] Many improvements to helpers functions `h/1`, `s/1` and others
@@ -423,12 +622,12 @@
   * [Record] Record accessors are now optimized and can be up to 6x faster in some cases
   * [String] Support `\xXX` and `\x{HEX}` escape sequences in strings, char lists and regexes
 
-* bug fix
+* Bug fixes
   * [Bootstrap] Compiling Elixir source no longer fails if environment variables contain utf-8 entries
   * [IEx] IEx will now wait for all command line options to be processed before starting
   * [Kernel] Ensure proper stacktraces when showing deprecations
 
-* deprecations
+* Deprecations
   * [Enum] `Enum.qsort` is deprecated in favor of `Enum.sort`
   * [List] `List.sort` and `List.uniq` have been deprecated in favor of their `Enum` counterparts
   * [Record] Default-based generated functions are deprecated
@@ -436,7 +635,7 @@
 
 # v0.7.1 (2012-11-18)
 
-* enhancements
+* Enhancements
   * [IEx] Only show documented functions and also show docs for default generated functions
   * [IO] Add `IO.binread`, `IO.binwrite` and `IO.binreadline` to handle raw binary file operations
   * [ExUnit] Add support for user configuration at `HOME/.ex_unit.exs`
@@ -454,20 +653,20 @@
   * [Typespec] Define a default type `t` for protocols and records
   * [Typespec] Add support for the access protocol in typespecs
 
-* bug fix
+* Bug fixes
   * [Kernel] Fix an issue where variables inside clauses remained unassigned
   * [Kernel] Ensure `defoverridable` functions can be referred in many clauses
   * [Kernel] Allow keywords as function names when following a dot (useful when integrating with erlang libraries)
   * [File] File is opened by default on binary mode instead of utf-8
 
-* deprecations
+* Deprecations
   * [Behaviour] `defcallback/1` is deprecated in favor of `defcallback/2` which matches erlang `@callbacks`
   * [Enum] `Enum.times` is deprecated in favor of using ranges
   * [System] `halt` moved to `System` module
 
 # v0.7.0 (2012-10-20)
 
-* enhancements
+* Enhancements
   * [Behaviour] Add Behaviour with a simple callback DSL to define callbacks
   * [Binary] Add a Dict binary that converts its keys to binaries on insertion
   * [Binary] Optimize `Binary.Inspect` and improve inspect for floats
@@ -493,7 +692,7 @@
   * [Regex] Add `:return` option to `Regex.run` and `Regex.scan`
   * [String] Add a String module responsible for handling UTf-8 binaries
 
-* bug fix
+* Bug fixes
   * [File] `File.cp` and `File.cp_r` now preserves the file's mode
   * [IEx] Fix a bug where printing to `:stdio` on `IEx` was causing it to hang
   * [Macro] Fix a bug where quoted expressions were not behaving the same as their non-quoted counterparts
@@ -501,12 +700,12 @@
   * [Mix] Mix now exits with status 1 in case of failures
   * [Protocol] Avoid false positives on protocol dispatch (a bug caused the dispatch to be triggered to an invalid protocol)
 
-* backwards incompatible changes
+* Backwards incompatible changes
   * [ExUnit] `setup` and `teardown` callbacks now receives the test name as second argument
   * [Kernel] Raw function definition with `def/4`, `defp/4`, `defmacro/4`, `defmacrop/4` now evaluates all arguments. The previous behaviour was accidental and did not properly evaluate all arguments
   * [Kernel] Change tuple-related (`elem` and `setelem`), Enum functions (`find_index`, `nth!` and `times`) and List functions (List.key*) to zero-index
 
-* deprecations
+* Deprecations
   * [Code] `Code.require_file` and `Code.load_file` now expect the full name as argument
   * [Enum] `List.reverse/1` and `List.zip/2` were moved to `Enum`
   * [GenServer] Rename `GenServer.Behavior` to `GenServer.Behaviour`
@@ -516,14 +715,14 @@
 
 # v0.6.0 (2012-08-01)
 
-* incompatible changes
+* Backwards incompatible changes
   * [Kernel] Compile files now follow `Elixir-ModuleName` convention to solve issues with Erlang embedded mode. This removes the `__MAIN__` pseudo-variable as modules are now located inside `Elixir` namespace
   * [Kernel] `__using__` callback triggered by `use` now receives just one argument. Caller information can be accessed via macros using `__CALLER__`
   * [Kernel] Comprehensions syntax changed to be more compatible with Erlang behavior
   * [Kernel] loop and recur are removed in favor of recursion with named functions
   * [Module] Removed data functions in favor of unifying the attributes API
 
-* deprecations
+* Deprecations
   * [Access] The semantics of the access protocol were reduced from a broad query API to simple data structure key-based access
   * [ExUnit] Some assertions are deprecated in favor of simply using `assert()`
   * [File] `File.read_info` is deprecated in favor of `File.stat`
@@ -535,7 +734,7 @@
   * [Module] `Module.function_defined?` is deprecated in favor of `Module.defines?`
   * [Module] `Module.defined_functions` is deprecated in favor of `Module.definitions_in`
 
-* enhancements
+* Enhancements
   * [Enum] Enhance Enum protocol to support `Enum.count`
   * [Enum] Optimize functions when a list is given as collection
   * [Enum] Add `find_index`, `nth!` and others

Makefile

@@ -1,4 +1,4 @@
-REBAR := "$(shell echo `pwd`/rebar)"
+REBAR := rebar
 ELIXIRC := bin/elixirc --verbose --ignore-module-conflict $(ELIXIRC_OPTS)
 ERLC := erlc -I lib/elixir/include
 ERL := erl -I lib/elixir/include -noshell -pa lib/elixir/ebin
@@ -19,12 +19,17 @@ INSTALL_PROGRAM = $(INSTALL) -m755
 define APP_TEMPLATE
 $(1): lib/$(1)/ebin/Elixir.$(2).beam lib/$(1)/ebin/$(1).app
 
-lib/$(1)/ebin/$(1).app:
-	$(Q) cd lib/$(1) && ../../bin/elixir -e "Mix.Server.start_link(:dev)" -r mix.exs -e "Mix.Task.run('compile.app')"
+lib/$(1)/ebin/$(1).app: lib/$(1)/mix.exs
+	$(Q) mkdir -p lib/$(1)/_build/shared/lib/$(1)
+	$(Q) cp -R lib/$(1)/ebin lib/$(1)/_build/shared/lib/$(1)/
+	$(Q) cd lib/$(1) && ../../bin/elixir -e "Mix.Sup.start_link()" -r mix.exs -e "Mix.Task.run('compile.app')"
+	$(Q) cp lib/$(1)/_build/shared/lib/$(1)/ebin/$(1).app lib/$(1)/ebin/$(1).app
+	$(Q) rm -rf lib/$(1)/_build
 
 lib/$(1)/ebin/Elixir.$(2).beam: $(wildcard lib/$(1)/lib/*.ex) $(wildcard lib/$(1)/lib/*/*.ex) $(wildcard lib/$(1)/lib/*/*/*.ex)
 	@ echo "==> $(1) (compile)"
-	$(Q) $$(ELIXIRC) "lib/$(1)/lib/**/*.ex" -o lib/$(1)/ebin
+	@ rm -rf lib/$(1)/ebin
+	$(Q) cd lib/$(1) && ../../$$(ELIXIRC) "lib/**/*.ex" -o ebin
 
 test_$(1): $(1)
 	@ echo "==> $(1) (exunit)"
@@ -42,10 +47,12 @@ compile: lib/elixir/src/elixir.app.src erlang elixir
 
 lib/elixir/src/elixir.app.src: src/elixir.app.src
 	$(Q) rm -rf lib/elixir/src/elixir.app.src
-	$(Q) cp src/elixir.app.src lib/elixir/src/elixir.app.src
+	$(Q) echo "%% This file is automatically generated from <project_root>/src/elixir.app.src" \
+	                             >lib/elixir/src/elixir.app.src
+	$(Q) cat src/elixir.app.src >>lib/elixir/src/elixir.app.src
 
 erlang:
-	$(Q) cd lib/elixir && $(REBAR) compile
+	$(Q) cd lib/elixir && ../../$(REBAR) compile
 
 # Since Mix depends on EEx and EEx depends on
 # Mix, we first compile EEx without the .app
@@ -58,17 +65,18 @@ $(KERNEL): lib/elixir/lib/*.ex lib/elixir/lib/*/*.ex
 		echo "==> bootstrap (compile)";                 \
 		$(ERL) -s elixir_compiler core -s erlang halt;  \
 	fi
-	@ echo "==> kernel (compile)";
-	$(Q) $(ELIXIRC) "lib/elixir/lib/**/*.ex" -o lib/elixir/ebin;
+	@ echo "==> elixir (compile)";
+	$(Q) cd lib/elixir && ../../$(ELIXIRC) "lib/kernel.ex" -o ebin;
+	$(Q) cd lib/elixir && ../../$(ELIXIRC) "lib/**/*.ex" -o ebin;
 	$(Q) $(MAKE) unicode
 	$(Q) rm -rf lib/elixir/ebin/elixir.app
-	$(Q) cd lib/elixir && $(REBAR) compile
+	$(Q) cd lib/elixir && ../../$(REBAR) compile
 
 unicode: $(UNICODE)
-$(UNICODE): lib/elixir/priv/unicode.ex lib/elixir/priv/UnicodeData.txt lib/elixir/priv/NamedSequences.txt
+$(UNICODE): lib/elixir/unicode/*
 	@ echo "==> unicode (compile)";
 	@ echo "This step can take up to a minute to compile in order to embed the Unicode database"
-	$(Q) $(ELIXIRC) lib/elixir/priv/unicode.ex -o lib/elixir/ebin;
+	$(Q) cd lib/elixir && ../../$(ELIXIRC) unicode/graphemes.ex unicode/unicode.ex -o ebin;
 
 $(eval $(call APP_TEMPLATE,ex_unit,ExUnit))
 $(eval $(call APP_TEMPLATE,eex,EEx))
@@ -89,9 +97,10 @@ install: compile
 	done
 
 clean:
-	cd lib/elixir && $(REBAR) clean
+	cd lib/elixir && ../../$(REBAR) clean
 	rm -rf ebin
 	rm -rf lib/*/ebin
+	rm -rf lib/elixir/test/ebin
 	rm -rf lib/*/tmp
 	rm -rf lib/mix/test/fixtures/git_repo
 	rm -rf lib/mix/test/fixtures/deps_on_git_repo
@@ -109,7 +118,7 @@ docs: compile ../ex_doc/bin/ex_doc
 	mkdir -p ebin
 	rm -rf docs
 	cp -R -f lib/*/ebin/*.beam ./ebin
-	bin/elixir ../ex_doc/bin/ex_doc "Elixir" "$(VERSION)" -m Kernel -u "https://github.com/elixir-lang/elixir" --source-ref "$(call SOURCE_REF)"
+	bin/elixir ../ex_doc/bin/ex_doc "Elixir" "$(VERSION)" "./ebin" -m Kernel -u "https://github.com/elixir-lang/elixir" --source-ref "$(call SOURCE_REF)"
 	rm -rf ebin
 
 ../ex_doc/bin/ex_doc:
@@ -121,9 +130,9 @@ release_zip: compile
 	zip -9 -r v$(VERSION).zip bin CHANGELOG.md LEGAL lib/*/ebin LICENSE README.md rel VERSION
 
 release_docs: docs
-	cd ../elixir-lang.github.com
-	rm -rf ../elixir-lang.github.com/docs/master
-	mv docs ../elixir-lang.github.com/docs/master
+	cd ../docs
+	rm -rf ../docs/master
+	mv docs ../docs/master
 
 release_erl: compile
 	$(Q) rm -rf rel/elixir
@@ -147,7 +156,7 @@ test_doc_test: compile
 	$(Q) cd lib/elixir && ../../bin/elixir -r "test/doc_test.exs";
 
 test_kernel: compile
-	@ echo "==> kernel (exunit)"
+	@ echo "==> elixir (exunit)"
 	$(Q) cd lib/elixir && ../../bin/elixir -r "test/elixir/test_helper.exs" -pr "test/elixir/**/*_test.exs";
 
 .dialyzer.base_plt:

README.md

@@ -10,11 +10,13 @@ If you want to contribute to Elixir or run it from source, clone this repository
 
     $ git clone https://github.com/elixir-lang/elixir.git
     $ cd elixir
-    $ make test
+    $ make clean test
+
+If Elixir fails to build (specifically when pulling in a new version via git), be sure to remove any previous build artifacts by running `make clean`, then `make test`.
 
 If tests pass, you are ready to move on to the [Getting Started guide][1] or to try Interactive Elixir by running: `bin/iex` in your terminal.
 
-However, if tests fail, it is likely you have an outdated Erlang version (Elixir requires Erlang R16B or later). You can check your Erlang version by calling `erl` in the command line. You will see some information as follow:
+However, if tests fail, it is likely you have an outdated Erlang version (Elixir requires Erlang R16B or later). You can check your Erlang version by calling `erl` in the command line. You will see some information as follows:
 
     Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false]
 

RELEASE.md

@@ -10,17 +10,17 @@ This document simply outlines the release process:
 
 4) Commit changes above with title "Release vVERSION"
 
-5) Push master and create tag from master branch
+5) Push master and create tag vVERSION from master branch
 
 6) Release new docs with `make release_docs`, move docs to `docs/stable`
 
-7) Release new zip with `make release_zip`, push new zip to Github Releases
+7) Release new zip with `make release_zip`, push new zip to GitHub Releases
 
 8) Push package to expm with `expm publish package.exs`
 
 9) Merge master into stable branch and push it
 
-10) After release, bump versions and add `-dev` back
+10) After release, bump versions, add `-dev` back and commit
 
 11) `make release_docs` once again and push `elixir-lang.github.com`
 

VERSION

@@ -1 +1 @@
-0.10.2
+0.12.2

bin/elixir

@@ -5,7 +5,7 @@ if [ $# -eq 0 ] || [ "$1" = "--help" ] || [ "$1" = "-h" ]; then
   -v                Prints version and exit
   -e \"command\"      Evaluates the given command (*)
   -r \"file\"         Requires the given files/patterns (*)
-  -S \"script\"       Finds and executes the given script (*)
+  -S \"script\"       Finds and executes the given script
   -pr \"file\"        Requires the given files/patterns in parallel (*)
   -pa \"path\"        Prepends the given path to Erlang code path (*)
   -pz \"path\"        Appends the given path to Erlang code path (*)
@@ -73,8 +73,6 @@ done
 
 SELF=$(readlink_f "$0")
 SCRIPT_PATH=$(dirname "$SELF")
-
-if [ -f "$HOME/.elixirrc" ]; then . "$HOME/.elixirrc"; fi
 if [ "$ELIXIR_NO_CLI" != "1" ]; then ERL="$ERL -s elixir start_cli"; fi
 
 if [ -f "$SCRIPT_PATH/../releases/RELEASES" ] && [ -f "$SCRIPT_PATH/erl" ]

bin/elixir.bat

@@ -12,7 +12,7 @@ echo.
 echo   -v                Prints version and exit
 echo   -e command        Evaluates the given command (*)
 echo   -r file           Requires the given files/patterns (*)
-echo   -S script         Finds and executes the given script (*)
+echo   -S script         Finds and executes the given script
 echo   -pr file          Requires the given files/patterns in parallel (*)
 echo   -pa path          Prepends the given path to Erlang code path (*)
 echo   -pz path          Appends the given path to Erlang code path (*)

bin/iex

@@ -5,7 +5,7 @@ if [ $# -gt 0 ] && ([ "$1" = "--help" ] || [ "$1" = "-h" ]); then
   -v                Prints version
   -e \"command\"      Evaluates the given command (*)
   -r \"file\"         Requires the given files/patterns (*)
-  -S \"script\"       Finds and executes the given script (*)
+  -S \"script\"       Finds and executes the given script
   -pr \"file\"        Requires the given files/patterns in parallel (*)
   -pa \"path\"        Prepends the given path to Erlang code path (*)
   -pz \"path\"        Appends the given path to Erlang code path (*)
@@ -38,4 +38,4 @@ readlink_f () {
 
 SELF=$(readlink_f "$0")
 SCRIPT_PATH=$(dirname "$SELF")
-ELIXIR_NO_CLI=1 exec "$SCRIPT_PATH"/elixir --no-halt --erl "-nouser -s Elixir.IEx.CLI" +iex "$@"
+ELIXIR_NO_CLI=1 exec "$SCRIPT_PATH"/elixir --no-halt --erl "-user Elixir.IEx.CLI" +iex "$@"

bin/mix

@@ -1,3 +1,3 @@
 #!/usr/bin/env elixir
 Mix.start
-Mix.CLI.run
+Mix.CLI.main

lib/eex/lib/eex.ex

@@ -27,6 +27,19 @@ defmodule EEx do
      above and is available to you if you want to provide your own
      ways of handling the compiled template.
 
+  ## Options
+
+  All functions in this module accepts EEx-related options.
+  They are:
+  
+  * `:line` - the line to be used as the template start.
+              Defaults to 1;
+  * `:file` - the file to be used in the template.
+              Defaults to the given file the template is read from
+              or to "nofile" when compiling from a string;
+  * `:engine` - the EEx engine to be used for compilation.
+                Defaults to `EEx.Engine`;
+
   ## Engine
 
   EEx has the concept of engines which allows you to modify or

lib/eex/lib/eex/compiler.ex

@@ -1,4 +1,4 @@
-defrecord EEx.State, engine: EEx.SmartEngine, dict: [], file: 'nofile', line: 1, start_line: 1
+defrecord EEx.State, engine: EEx.SmartEngine, dict: [], file: "nofile", line: 1, start_line: 1
 
 defmodule EEx.Compiler do
   @moduledoc false
@@ -23,7 +23,7 @@ defmodule EEx.Compiler do
   end
 
   defp generate_buffer([{ :expr, line, mark, chars }|t], buffer, scope, state) do
-    expr = maybe_block :elixir_translator.forms!(chars, line, state.file, [])
+    expr = Code.string_to_quoted!(chars, [line: line, file: state.file])
     buffer = state.engine.handle_expr(buffer, mark, expr)
     generate_buffer(t, buffer, scope, state)
   end
@@ -31,7 +31,7 @@ defmodule EEx.Compiler do
   defp generate_buffer([{ :start_expr, line, mark, chars }|t], buffer, scope, state) do
     { contents, t } = generate_buffer(t, "", [chars|scope], state.dict([]).line(line).start_line(line))
     buffer = state.engine.handle_expr(buffer, mark, contents)
-    generate_buffer(t, buffer, scope, state.dict([]))
+    generate_buffer(t, buffer, scope, state)
   end
 
   defp generate_buffer([{ :middle_expr, line, _, chars }|t], buffer, [current|scope], state) do
@@ -41,7 +41,7 @@ defmodule EEx.Compiler do
 
   defp generate_buffer([{ :end_expr, line, _, chars }|t], buffer, [current|_], state) do
     { wrapped, state } = wrap_expr(current, line, buffer, chars, state)
-    tuples = maybe_block :elixir_translator.forms!(wrapped, state.start_line, state.file, [])
+    tuples = Code.string_to_quoted!(wrapped, [line: state.start_line, file: state.file])
     buffer = insert_quotes(tuples, state.dict)
     { buffer, t }
   end
@@ -63,35 +63,29 @@ defmodule EEx.Compiler do
   defp wrap_expr(current, line, buffer, chars, state) do
     new_lines = List.duplicate(?\n, line - state.line)
 
-    if state.dict == [] and is_empty?(buffer) do
+    if state.dict == [] and empty?(buffer) do
       { current ++ new_lines ++ chars, state }
     else
       key = length(state.dict)
       placeholder = '__EEX__(' ++ integer_to_list(key) ++ ');'
-      { current ++ placeholder ++ new_lines ++ chars, state.update_dict([{key, buffer}|&1]) }
+      { current ++ placeholder ++ new_lines ++ chars, state.update_dict(&[{key, buffer}|&1]) }
     end
   end
 
   # Check if the syntax node represents an empty string
 
-  defp is_empty?(bin) when is_binary(bin) do
+  defp empty?(bin) when is_binary(bin) do
     bc(<<c>> inbits bin, not c in [?\s, ?\t, ?\r, ?\n], do: <<c>>) == ""
   end
 
-  defp is_empty?({ :<>, _, [left, right] }) do
-    is_empty?(left) and is_empty?(right)
+  defp empty?({ :<>, _, [left, right] }) do
+    empty?(left) and empty?(right)
   end
 
-  defp is_empty?(_) do
+  defp empty?(_) do
     false
   end
 
-  # Block wrapping
-
-  defp maybe_block([]),    do: nil
-  defp maybe_block([h]),   do: h
-  defp maybe_block(other), do: { :__block__, [], other }
-
   # Changes placeholder to real expression
 
   defp insert_quotes({ :__EEX__, _, [key] }, dict) do
@@ -108,7 +102,7 @@ defmodule EEx.Compiler do
   end
 
   defp insert_quotes(list, dict) when is_list(list) do
-    Enum.map list, insert_quotes(&1, dict)
+    Enum.map list, &insert_quotes(&1, dict)
   end
 
   defp insert_quotes(other, _dict) do

lib/eex/lib/eex/smart_engine.ex

@@ -88,7 +88,7 @@ defmodule EEx.AssignsEngine do
   defmacro __using__(_) do
     quote unquote: false do
       defp transform({ :@, line, [{ name, _, atom }] }) when is_atom(name) and is_atom(atom) do
-        quote do: Keyword.get(var!(assigns), unquote(name))
+        quote do: Dict.get(var!(assigns), unquote(name))
       end
 
       defp transform(arg) do

lib/eex/lib/eex/tokenizer.ex

@@ -103,7 +103,7 @@ defmodule EEx.Tokenizer do
     # token and, if so, it is not followed by an "end"
     # token. If this is the case, we are on a start expr.
     case :elixir_tokenizer.tokenize(rest, 1, file: "eex", check_terminators: false) do
-      { :ok, tokens } ->
+      { :ok, _line, tokens } ->
         tokens   = Enum.reverse(tokens)
         fn_index = fn_index(tokens)
 
@@ -136,7 +136,7 @@ defmodule EEx.Tokenizer do
   end
 
   defp end_index(tokens) do
-    Enum.find_index(tokens, match?({ :end, _ }, &1)) || :infinity
+    Enum.find_index(tokens, &match?({ :end, _ }, &1)) || :infinity
   end
 
   defp check_spaces(string, token) do

lib/eex/test/eex/smart_engine_test.exs

@@ -7,16 +7,21 @@ defmodule EEx.SmartEngineTest do
     assert_eval "foo bar", "foo bar"
   end
 
-  test "evaluates with assigns" do
+  test "evaluates with assigns as a ListDict" do
     assert_eval "1", "<%= @foo %>", assigns: [foo: 1]
   end
 
+  test "evaluates with assigns as a HashDict" do
+    assigns = HashDict.new [foo: 1]
+    assert_eval "1", "<%= @foo %>", assigns: assigns
+  end
+
   test "evaluates with loops" do
     assert_eval "1\n2\n3\n", "<%= lc x inlist [1, 2, 3] do %><%= x %>\n<% end %>"
   end
 
   defp assert_eval(expected, actual, binding // []) do
-    result = EEx.eval_string(actual, binding, file: __FILE__)
+    result = EEx.eval_string(actual, binding, file: __ENV__.file)
     assert result == expected
   end
 end

lib/eex/test/eex_test.exs

@@ -264,6 +264,21 @@ foo
     assert_eval expected, string
   end
 
+  test "evaluates nested do expressions" do
+    string = """
+    <% y = ["a", "b", "c"] %>
+    <%= cond do %>
+     <% "a" in y -> %>
+      Good
+     <% true -> %>
+      <% if true do %>true<% else %>false<% end %>
+      Bad
+    <% end %>
+    """
+
+    assert_eval "\n\n  Good\n \n", string
+  end
+
   test "unicode" do
     template = """
       • <%= "•" %> •
@@ -306,7 +321,7 @@ foo
         { EExText.Compiled,
           :before_compile,
           0,
-          [file: to_char_list(__FILE__), line: 7]
+          [file: to_char_list(Path.relative_to_cwd(__ENV__.file)), line: 7]
         }
       }
 
@@ -315,7 +330,7 @@ foo
         { EExText.Compiled,
           :after_compile,
           0,
-          [file: to_char_list(__FILE__), line: 18]
+          [file: to_char_list(Path.relative_to_cwd(__ENV__.file)), line: 18]
         }
       }
 
@@ -330,7 +345,7 @@ foo
   end
 
   defp assert_eval(expected, actual) do
-    result = EEx.eval_string(actual, [], file: __FILE__, engine: EEx.Engine)
+    result = EEx.eval_string(actual, [], file: __ENV__.file, engine: EEx.Engine)
     assert result == expected
   end
 end

lib/elixir/include/elixir.hrl

@@ -9,34 +9,43 @@
 -define(line(Opts), elixir_utils:get_line(Opts)).
 
 -record(elixir_scope, {
-  context=nil,             %% can be assign, guards or nil
-  extra=nil,               %% extra information about the context, like fn_match for fns
+  context=nil,             %% can be match, guards or nil
+  extra=nil,               %% extra information about the context, like fn_match and do_match
   noname=false,            %% when true, don't add new names (used by try)
   super=false,             %% when true, it means super was invoked
   caller=false,            %% when true, it means caller was invoked
   module=nil,              %% the current module
   function=nil,            %% the current function
   vars=[],                 %% a dict of defined variables and their alias
-  list_vars=nil,           %% a list of vars passed down to Macro.Env
   backup_vars=nil,         %% a copy of vars to be used on ^var
-  temp_vars=nil,           %% a set of all variables defined in a particular assign
-  clause_vars=nil,         %% a dict of all variables defined in a particular clause
+  match_vars=nil,          %% a set of all variables defined in a particular match
+  export_vars=nil,         %% a dict of all variables defined in a particular clause
   extra_guards=nil,        %% extra guards from args expansion
-  counter=[],              %% a counter for the variables defined
-  local=nil,               %% the scope to evaluate local functions against
-  context_modules=[],      %% modules defined in the current context
+  counter=[],              %% a dict counting the variables defined
+  file=(<<"nofile">>)      %% the current scope filename
+}).
+
+-record(elixir_env, {
+  module=nil,              %% the current module
+  file=(<<"nofile">>),     %% the current filename
+  line=1,                  %% the current line
+  function=nil,            %% the current function
+  context=nil,             %% can be match_vars, guards or nil
+  requires=[],             %% a set with modules required
+  aliases=[],              %% an orddict with aliases by new -> old names
+  functions=[],            %% a list with functions imported from module
+  macros=[],               %% a list with macros imported from module
   macro_aliases=[],        %% keep aliases defined inside a macro
-  aliases,                 %% an orddict with aliases by new -> old names
-  file,                    %% the current scope filename
-  requires,                %% a set with modules required
-  macro_macros=[],         %% a list with macros imported from module inside a macro
-  macros,                  %% a list with macros imported from module
-  macro_functions=[],      %% a list with functions imported from module inside a macro
-  functions                %% a list with functions imported from module
+  context_modules=[],      %% modules defined in the current context
+  vars=[],                 %% a set of defined variables
+  export_vars=nil,         %% a set of variables to be exported in certain constructs, may be nil
+  lexical_tracker=nil,     %% holds the lexical tracker pid
+  local=nil                %% the module to delegate local functions to
 }).
 
 -record(elixir_quote, {
-  line=0,
+  line=false,
+  keep=false,
   context=nil,
   vars_hygiene=true,
   aliases_hygiene=true,
@@ -46,8 +55,12 @@
   escape=false
 }).
 
-%% Introspection
--define(defs(Kind), Kind == def; Kind == defp; Kind == defmacro; Kind == defmacrop).
+-record(elixir_tokenizer, {
+  file,
+  terminators=[],
+  check_terminators=true,
+  existing_atoms_only=false
+}).
 
 %% Used in tokenization and interpolation
 

lib/elixir/lib/access.ex

@@ -11,8 +11,6 @@ defprotocol Access do
   following built-in types: keywords, records and functions.
   """
 
-  @only [List, Record, Atom]
-
   @doc """
   Receives the element being accessed and the access item.
   """
@@ -22,6 +20,7 @@ end
 defimpl Access, for: List do
   @doc """
   Access the given key in a tuple list.
+  The key is found via the `===` operator.
 
   ## Examples
 
@@ -34,14 +33,10 @@ defimpl Access, for: List do
       "★☆"
 
   """
+  def access(dict, key)
+  def access([{ key, value }|_], key), do: value
+  def access([{ _, _ }|t], key), do: access(t, key)
   def access([], _key), do: nil
-
-  def access(list, key) do
-    case :lists.keyfind(key, 1, list) do
-      { ^key, value } -> value
-      false -> nil
-    end
-  end
 end
 
 defimpl Access, for: Atom do

lib/elixir/lib/application/behaviour.ex

@@ -1,40 +1,40 @@
 defmodule Application.Behaviour do
   @moduledoc """
-  This module is a convenience to define application module callbacks.
+  Default callbacks for applications.
 
   In Erlang/OTP, an application is a component that can be started
-  and stopped as a unit, and which can be re-used in other systems
-  as well.
+  and stopped as a unit, and which can be reused in other systems.
 
-  The first step to achieve this is to define an application specification.
+  The first step in creating an application is to define an application specification.
   For example, if your application is named `:my_app`, an app specification
   should exist at `ebin/my_app.app`. This file is usually defined by
   build tools like Mix.
 
-  Then, with the app specification in hands, we must also define an
-  application module callback that controls how to start and stop
-  such applications. This module is about defining such callbacks.
+  With the app specification in hand, we must define
+  application module callbacks that control how to start and stop
+  instances of the application. This module is about defining such callbacks.
 
-  There are two callbacks required to be implemented:
+  There are two callbacks which must be implemented:
 
-  1. `start(type, args)` - It must return `{ :ok, pid }` or
+  1. `start(type, args)` - must return `{ :ok, pid }` or
      `{ :ok, pid, state }`, where `pid` is the process identifier
-     of the supervisor tree root;
+     of the supervisor tree root and `state` is application defined 
+     state information;
 
-  2. `stop(state)` receives the state returned by `start` and should
-     do any necessary cleaning up. Notice that shutting down the supervisor
+  2. `stop(state)` - receives the `state` returned by `start` and should
+     do any necessary clean up. Notice that shutting down the supervisor
      is automatically handled by the VM;
 
-  When using this module, it simply tags the module behaviour as
-  `:application` and defines a default `stop/1` callback. The `start/2`
-  still needs to be defined by the user.
+  When using this module, it tags the module behaviour as
+  `:application` and provides a default `stop/1` callback. The `start/2` callback
+  still needs to be implemented by the user.
 
   You can learn more about the `:application` module, the application
-  specification and the application module callbacks below:
+  specification and the application module callbacks from these sources:
 
-  http://www.erlang.org/doc/man/application.html
-  http://www.erlang.org/doc/design_principles/applications.html
-  http://learnyousomeerlang.com/building-otp-applications
+  * http://www.erlang.org/doc/man/application.html
+  * http://www.erlang.org/doc/design_principles/applications.html
+  * http://learnyousomeerlang.com/building-otp-applications
 
   ## Example
 
@@ -102,4 +102,4 @@ defmodule Application.Behaviour do
       defoverridable [stop: 1]
     end
   end
-end
+end

lib/elixir/lib/behaviour.ex

@@ -1,12 +1,12 @@
 defmodule Behaviour do
   @moduledoc """
-  A convenience module for defining behaviours.
+  Utilities for defining behaviour intefaces.
+
   Behaviours can be referenced by other modules
-  in order to ensure they implement the proper
-  callbacks.
+  to ensure they implement required callbacks.
 
   For example, you can specify the `URI.Parser`
-  behaviour as follow:
+  behaviour as follows:
 
       defmodule URI.Parser do
         use Behaviour
@@ -18,7 +18,7 @@ defmodule Behaviour do
         defcallback default_port() :: integer
       end
 
-  And then a specific module may use it as:
+  And then a module may use it as:
 
       defmodule URI.HTTP do
         @behaviour URI.Parser
@@ -26,63 +26,80 @@ defmodule Behaviour do
         def parse(info), do: info
       end
 
-  In case the behaviour changes or URI.HTTP does
+  If the behaviour changes or `URI.HTTP` does
   not implement one of the callbacks, a warning
   will be raised.
 
   ## Implementation
 
-  Behaviours since Erlang R15 must be defined via
+  Since Erlang R15, behaviours must be defined via
   `@callback` attributes. `defcallback` is a simple
   mechanism that defines the `@callback` attribute
-  according to the type specification and also allows
-  docs and defines a custom function signature.
+  according to the given type specification. `defcallback` allows
+  documentation to be created for the callback and defines
+  a custom function signature.
 
   The callbacks and their documentation can be retrieved
   via the `__behaviour__` callback function.
   """
 
   @doc """
-  Defines a callback according to the given type specification.
+  Define a function callback according to the given type specification.
   """
-  defmacro defcallback({ :::, _, [fun, return] }) do
-    do_defcallback(fun, return, __CALLER__)
+  defmacro defcallback({ :::, _, [fun, return_and_guard] }) do
+    { return, guard } = split_return_and_guard(return_and_guard)
+    do_defcallback(fun, return, guard, __CALLER__)
   end
 
   defmacro defcallback(fun) do
-    do_defcallback(fun, quote(do: term), __CALLER__)
+    do_defcallback(fun, quote(do: term), [], __CALLER__)
   end
 
   @doc """
-  Defines a macro callback according to the given type specification.
+  Define a macro callback according to the given type specification.
   """
-  defmacro defmacrocallback({ :::, _, [fun, return] }) do
-    do_defmacrocallback(fun, return, __CALLER__)
+  defmacro defmacrocallback({ :::, _, [fun, return_and_guard] }) do
+    { return, guard } = split_return_and_guard(return_and_guard)
+    do_defmacrocallback(fun, return, guard, __CALLER__)
   end
 
   defmacro defmacrocallback(fun) do
-    do_defmacrocallback(fun, quote(do: Macro.t), __CALLER__)
+    do_defmacrocallback(fun, quote(do: Macro.t), [], __CALLER__)
   end
 
-  defp do_defcallback(fun, spec, caller) do
-    case Macro.extract_args(fun) do
+  defp split_return_and_guard({ :when, _, [return, guard] }) do
+    { return, guard }
+  end
+
+  defp split_return_and_guard({ :|, meta, [left, right] }) do
+    { return, guard } = split_return_and_guard(right)
+    { { :|, meta, [left, return] }, guard }
+  end
+
+  defp split_return_and_guard(other) do
+    { other, [] }
+  end
+
+  defp do_defcallback(fun, return, guards, caller) do
+    case Macro.decompose_call(fun) do
       { name, args } ->
-        do_callback(:def, name, args, name, length(args), spec, caller)
-      :error ->
+        do_callback(:def, name, args, name, length(args), args, return, guards, caller)
+      _ ->
         raise ArgumentError, message: "invalid syntax in defcallback #{Macro.to_string(fun)}"
     end
   end
 
-  defp do_defmacrocallback(fun, spec, caller) do
-    case Macro.extract_args(fun) do
+  defp do_defmacrocallback(fun, return, guards, caller) do
+    case Macro.decompose_call(fun) do
       { name, args } ->
-        do_callback(:defmacro, :"MACRO-#{name}", [quote(do: env :: Macro.Env.t)|args], name, length(args), spec, caller)
-      :error ->
+        do_callback(:defmacro, :"MACRO-#{name}", [quote(do: env :: Macro.Env.t)|args],
+                    name, length(args), args, return, guards, caller)
+      _ ->
         raise ArgumentError, message: "invalid syntax in defmacrocallback #{Macro.to_string(fun)}"
     end
   end
 
-  defp do_callback(kind, name, args, docs_name, docs_arity, return, caller) do
+  defp do_callback(kind, name, args, docs_name, docs_arity, docs_args, return, guards, caller) do
     Enum.each args, fn
       { :::, _, [left, right] } ->
         ensure_not_default(left)
@@ -94,24 +111,75 @@ defmodule Behaviour do
     end
 
     quote do
-      @callback unquote(name)(unquote_splicing(args)) :: unquote(return)
-      Behaviour.store_docs __MODULE__, unquote(caller.line), unquote(kind), unquote(docs_name), unquote(docs_arity)
+      docs_args = unquote(Macro.escape(docs_args))
+      @callback unquote(name)(unquote_splicing(args)) :: unquote(return) when unquote(guards)
+      Behaviour.store_docs(__MODULE__, unquote(caller.line), unquote(kind),
+                           unquote(docs_name), docs_args, unquote(docs_arity))
     end
   end
 
   defp ensure_not_default({ ://, _, [_, _] }) do
-    raise ArgumentError, message: "default arguments // not supported in defcallback"
+    raise ArgumentError, message: "default arguments // not supported in defcallback/defmacrocallback"
   end
 
   defp ensure_not_default(_), do: :ok
 
   @doc false
-  def store_docs(module, line, kind, name, arity) do
+  def store_docs(module, line, kind, name, args, arity) do
     doc = Module.get_attribute module, :doc
     Module.delete_attribute module, :doc
-    Module.put_attribute module, :behaviour_docs, { { name, arity }, line, kind, doc }
+
+    signature = Enum.map(Stream.with_index(args), fn { arg, ix } ->
+      { simplify_signature(arg, ix + 1), [line: line], nil }
+    end)
+
+    Module.put_attribute module, :behaviour_docs, { { name, arity }, line, kind, signature, doc }
   end
 
+  defp simplify_signature({ :::, _, [var, _] }, i) do
+    simplify_signature(var, i)
+  end
+
+  defp simplify_signature({ :|, _, [first, _] }, i) do
+    simplify_signature(first, i)
+  end
+
+  defp simplify_signature({ :., _, [_, name] }, _i) do
+    name
+  end
+
+  defp simplify_signature({ :->, _, list }, i) when is_list(list) do
+    :"fun#{i}"
+  end
+
+  defp simplify_signature({ :.., _, list }, i) when is_list(list) do
+    :"range#{i}"
+  end
+
+  defp simplify_signature({ :<<>>, _, list }, i) when is_list(list) do
+    :"bitstring#{i}"
+  end
+
+  defp simplify_signature({ :{}, _, list }, i) when is_list(list) do
+    :"tuple#{i}"
+  end
+
+  defp simplify_signature({ name, _, args }, _i)
+      when is_atom(name) and (is_atom(args) or is_list(args)) do
+    name
+  end
+
+  defp simplify_signature({ ast, _, list }, i) when is_list(list) do
+    simplify_signature(ast, i)
+  end
+
+  defp simplify_signature(other, i) when is_integer(other), do: :"int#{i}"
+  defp simplify_signature(other, i) when is_boolean(other), do: :"bool#{i}"
+  defp simplify_signature(other, i) when is_atom(other),    do: :"atom#{i}"
+  defp simplify_signature(other, i) when is_list(other),    do: :"list#{i}"
+  defp simplify_signature(other, i) when is_tuple(other),   do: :"tuple#{i}"
+  defp simplify_signature(_, i), do: :"arg#{i}"
+
   @doc false
   defmacro __using__(_) do
     quote do

lib/elixir/lib/bitwise.ex

@@ -46,83 +46,83 @@ defmodule Bitwise do
   Bitwise not.
   """
   defmacro bnot(expr) do
-    quote do: __op__(:bnot, unquote(expr))
+    quote do: :erlang.bnot(unquote(expr))
   end
 
   @doc """
   Bitwise not as operator.
   """
   defmacro ~~~expr do
-    quote do: __op__(:bnot, unquote(expr))
+    quote do: :erlang.bnot(unquote(expr))
   end
 
   @doc """
   Bitwise and.
   """
   defmacro band(left, right) do
-    quote do: __op__(:band, unquote(left), unquote(right))
+    quote do: :erlang.band(unquote(left), unquote(right))
   end
 
   @doc """
   Bitwise and as operator.
   """
   defmacro left &&& right do
-    quote do: __op__(:band, unquote(left), unquote(right))
+    quote do: :erlang.band(unquote(left), unquote(right))
   end
 
   @doc """
   Bitwise or.
   """
   defmacro bor(left, right) do
-    quote do: __op__(:bor, unquote(left), unquote(right))
+    quote do: :erlang.bor(unquote(left), unquote(right))
   end
 
   @doc """
   Bitwise or as operator.
   """
   defmacro left ||| right do
-    quote do: __op__(:bor, unquote(left), unquote(right))
+    quote do: :erlang.bor(unquote(left), unquote(right))
   end
 
   @doc """
   Bitwise xor.
   """
   defmacro bxor(left, right) do
-    quote do: __op__(:bxor, unquote(left), unquote(right))
+    quote do: :erlang.bxor(unquote(left), unquote(right))
   end
 
   @doc """
   Bitwise xor as operator.
   """
   defmacro left ^^^ right do
-    quote do: __op__(:bxor, unquote(left), unquote(right))
+    quote do: :erlang.bxor(unquote(left), unquote(right))
   end
 
   @doc """
   Arithmetic bitshift left.
   """
   defmacro bsl(left, right) do
-    quote do: __op__(:bsl, unquote(left), unquote(right))
+    quote do: :erlang.bsl(unquote(left), unquote(right))
   end
 
   @doc """
   Arithmetic bitshift left as operator.
   """
   defmacro left <<< right do
-    quote do: __op__(:bsl, unquote(left), unquote(right))
+    quote do: :erlang.bsl(unquote(left), unquote(right))
   end
 
   @doc """
   Arithmetic bitshift right.
   """
   defmacro bsr(left, right) do
-    quote do: __op__(:bsr, unquote(left), unquote(right))
+    quote do: :erlang.bsr(unquote(left), unquote(right))
   end
 
   @doc """
   Arithmetic bitshift right as operator.
   """
   defmacro left >>> right do
-    quote do: __op__(:bsr, unquote(left), unquote(right))
+    quote do: :erlang.bsr(unquote(left), unquote(right))
   end
 end

lib/elixir/lib/code.ex

@@ -1,29 +1,29 @@
 defmodule Code do
-  defexception LoadError, file: nil do
-    def message(exception) do
-      "could not load #{exception.file}"
+  defexception LoadError, [:file, :message] do
+    def exception(opts) do
+      file = opts[:file]
+      LoadError[message: "could not load #{file}", file: file]
     end
   end
 
   @moduledoc """
-  The Code module is responsible for managing code compilation,
-  code evaluation and code loading.
+  Utilities for managing code compilation, code evaluation and code loading.
 
-  It complements [Erlang's code module](http://www.erlang.org/doc/man/code.html)
+  This module complements [Erlang's code module](http://www.erlang.org/doc/man/code.html)
   to add behavior which is specific to Elixir.
-
   """
 
   @doc """
-  Returns all loaded files.
+  List all loaded files.
   """
   def loaded_files do
     :elixir_code_server.call :loaded
   end
 
   @doc """
-  Removes the given files from the loaded files list.
-  The modules defined in the file are not removed,
+  Remove files from the loaded files list.
+
+  The modules defined in the file are not removed;
   calling this function only removes them from the list,
   allowing them to be required again.
   """
@@ -32,7 +32,8 @@ defmodule Code do
   end
 
   @doc """
-  Appends a path to the Erlang VM code path.
+  Append a path to the Erlang VM code path.
+
   The path is expanded with `Path.expand/1` before being appended.
   """
   def append_path(path) do
@@ -40,7 +41,8 @@ defmodule Code do
   end
 
   @doc """
-  Prepends a path to the Erlang VM code path.
+  Prepend a path to the Erlang VM code path.
+
   The path is expanded with `Path.expand/1` before being prepended.
   """
   def prepend_path(path) do
@@ -48,7 +50,8 @@ defmodule Code do
   end
 
   @doc """
-  Deletes a path from the Erlang VM code path.
+  Delete a path from the Erlang VM code path.
+
   The path is expanded with `Path.expand/1` before being deleted.
   """
   def delete_path(path) do
@@ -56,9 +59,12 @@ defmodule Code do
   end
 
   @doc """
-  Evaluates the contents given by `string`. The second argument is 
-  a keyword list of variable bindings, followed by a keyword list of
-  environment options. Those options can be:
+  Evaluate the contents given by `string`. 
+
+  The `binding` argument is a keyword list of variable bindings. 
+  The `opts` argument is a keyword list of environment options. 
+  
+  Those options can be:
 
   * `:file` - the file to be considered in the evaluation
   * `:line` - the line on which the script starts
@@ -83,9 +89,12 @@ defmodule Code do
   etc.
 
   Returns a tuple of the form `{ value, binding }`,
-  where `value` is the the value returned from evaluating `string`; `binding`
-  is a keyword list with the value of all variable bindings after evaluating
-  `string`. If an error occurs while evaluating `string` an exception will be raised.
+  where `value` is the the value returned from evaluating `string`.
+  If an error occurs while evaluating `string` an exception will be raised.
+
+  `binding` is a keyword list with the value of all variable bindings
+  after evaluating `string`. The binding key is usually an atom, but it
+  may be a tuple for variables defined in a different context.
 
   ## Examples
 
@@ -95,7 +104,7 @@ defmodule Code do
       iex> Code.eval_string("c = a + b", [a: 1, b: 2], __ENV__)
       {3, [a: 1, b: 2, c: 3]}
 
-      iex> Code.eval_string("a = a + b", [a: 1, b: 2])          
+      iex> Code.eval_string("a = a + b", [a: 1, b: 2])
       {3, [a: 3, b: 2]}
 
   For convenience, you can pass `__ENV__` as the `opts` argument and
@@ -118,12 +127,12 @@ defmodule Code do
   end
 
   defp do_eval_string(string, binding, opts) when is_list(binding) do
-    { value, binding, _scope } = :elixir.eval to_char_list(string), binding, opts
+    { value, binding, _env, _scope } = :elixir.eval to_char_list(string), binding, opts
     { value, binding }
   end
 
   @doc """
-  Evaluates the quoted contents.
+  Evaluate the quoted contents.
 
   See `eval_string/3` for a description of arguments and return values.
 
@@ -153,8 +162,7 @@ defmodule Code do
   end
 
   defp do_eval_quoted(quoted, binding, opts) when is_list(binding) do
-    { value, binding, _scope } =
-      :elixir.eval_quoted [quoted], binding, opts
+    { value, binding, _env, _scope } = :elixir.eval_quoted quoted, binding, opts
     { value, binding }
   end
 
@@ -166,10 +174,10 @@ defmodule Code do
   end
 
   defp validate_requires(kind, requires) do
-    valid = is_list(requires) and Enum.all?(requires, is_atom(&1))
+    valid = is_list(requires) and Enum.all?(requires, &is_atom(&1))
 
     unless valid do
-      raise ArgumentError, "expected :#{kind} option given to eval in the format: [module]"
+      raise ArgumentError, message: "expected :#{kind} option given to eval in the format: [module]"
     end
   end
 
@@ -179,7 +187,7 @@ defmodule Code do
     end)
 
     unless valid do
-      raise ArgumentError, "expected :#{kind} option given to eval in the format: [{ module, module }]"
+      raise ArgumentError, message: "expected :#{kind} option given to eval in the format: [{ module, module }]"
     end
   end
 
@@ -191,12 +199,14 @@ defmodule Code do
     end)
 
     unless valid do
-      raise ArgumentError, "expected :#{kind} option given to eval in the format: [{ module, [{ name, arity }] }]"
+      raise ArgumentError, message: "expected :#{kind} option given to eval in the format: [{ module, [{ name, arity }] }]"
     end
   end
 
   @doc """
-  Converts the given string to its quoted form. Returns `{ :ok, quoted_form }`
+  Convert the given string to its quoted form. 
+  
+  Returns `{ :ok, quoted_form }`
   if it succeeds, `{ :error, { line, error, token } }` otherwise.
 
   ## Options
@@ -209,25 +219,27 @@ defmodule Code do
   * `:existing_atoms_only` - When `true`, raises an error
     when non-existing atoms are found by the tokenizer.
 
-  ## Macro.to_string/1
+  ## Macro.to_string/2
 
   The opposite of converting a string to its quoted form is
-  `Macro.to_string/1`, which converts a quoted form to a string/binary
+  `Macro.to_string/2`, which converts a quoted form to a string/binary
   representation.
   """
   def string_to_quoted(string, opts // []) when is_list(opts) do
     file = Keyword.get opts, :file, "nofile"
     line = Keyword.get opts, :line, 1
-    res  = :elixir_translator.forms(to_char_list(string), line, file, opts)
+    res  = :elixir.string_to_quoted(to_char_list(string), line, file, opts)
 
     case res do
-      { :ok, forms } -> { :ok, unpack_quote(line, forms) }
+      { :ok, forms } -> { :ok, forms }
       _ -> res
     end
   end
 
   @doc """
-  Converts the given string to its quoted form. It returns the ast if it succeeds,
+  Convert the given string to its quoted form. 
+  
+  It returns the ast if it succeeds,
   raises an exception otherwise. The exception is a `TokenMissingError`
   in case a token is missing (usually because the expression is incomplete),
   `SyntaxError` otherwise.
@@ -237,16 +249,13 @@ defmodule Code do
   def string_to_quoted!(string, opts // []) when is_list(opts) do
     file = Keyword.get opts, :file, "nofile"
     line = Keyword.get opts, :line, 1
-    res  = :elixir_translator.forms!(to_char_list(string), line, file, opts)
-    unpack_quote(line, res)
+    :elixir.string_to_quoted!(to_char_list(string), line, file, opts)
   end
 
-  defp unpack_quote(_line, []),                              do: nil
-  defp unpack_quote(_line, [forms]) when not is_list(forms), do: forms
-  defp unpack_quote(line, forms),                            do: { :__block__, [line: line], forms }
-
   @doc """
-  Loads the given `file`. Accepts `relative_to` as an argument to tell where
+  Load the given file. 
+  
+  Accepts `relative_to` as an argument to tell where
   the file is located. If the file was already required/loaded, loads it again.
   It returns a list of tuples `{ ModuleName, <<byte_code>> }`, one tuple for each
   module defined in the file.
@@ -266,9 +275,11 @@ defmodule Code do
   end
 
   @doc """
-  Requires the given `file`. Accepts `relative_to` as an argument to tell where
+  Require the given `file`. 
+  
+  Accepts `relative_to` as an argument to tell where
   the file is located. The return value is the same as that of `load_file/2`. If
-  the file was already required/loaded, doesn't do anything and returns nil.
+  the file was already required/loaded, doesn't do anything and returns `nil`.
 
   Notice that if `require_file` is invoked by different processes concurrently,
   the first process to invoke `require_file` acquires a lock and the remaining
@@ -294,7 +305,8 @@ defmodule Code do
   end
 
   @doc """
-  Loads the compilation options from the code server.
+  Load the compilation options from the code server.
+
   Check `compiler_options/1` for more information.
   """
   def compiler_options do
@@ -302,8 +314,9 @@ defmodule Code do
   end
 
   @doc """
-  Sets compilation options. These options are global
-  since they are stored by Elixir's Code Server.
+  Set compilation options. 
+
+  These options are global since they are stored by Elixir's Code Server.
 
   Available options are:
 
@@ -312,7 +325,7 @@ defmodule Code do
 
   * `:debug_info` - when `true`, retain debug information in the compiled module.
     This allows a developer to reconstruct the original source
-    code, for such reasons, `false` by default;
+    code, `false` by default;
 
   * `:ignore_module_conflict` - when `true`, override modules that were already defined
     without raising errors, `false` by default;
@@ -325,28 +338,34 @@ defmodule Code do
   end
 
   @doc """
-  Compiles the given string and returns a list of tuples where
+  Compile the given string. 
+
+  Returns a list of tuples where
   the first element is the module name and the second one is its
   binary.
 
   For compiling many files at once, check `Kernel.ParallelCompiler.files/2`.
   """
   def compile_string(string, file // "nofile") when is_binary(file) do
-    :elixir_compiler.string String.to_char_list!(string), file
+    :elixir_compiler.string to_char_list(string), file
   end
 
   @doc """
-  Compiles the quoted expression and returns a list of tuples where
+  Compile the quoted expression. 
+  
+  Returns a list of tuples where
   the first element is the module name and the second one is its
   binary.
   """
   def compile_quoted(quoted, file // "nofile") when is_binary(file) do
-    :elixir_compiler.quoted [quoted], file
+    :elixir_compiler.quoted quoted, file
   end
 
   @doc """
-  Ensures the given module is loaded. If the module is already
-  loaded, it works as no-op. If the module was not yet loaded,
+  Ensure the given module is loaded. 
+  
+  If the module is already
+  loaded, this works as no-op. If the module was not yet loaded,
   it tries to load it.
 
   If it succeeds loading the module, it returns
@@ -362,7 +381,7 @@ defmodule Code do
   modules need to be loaded upfront or explicitly.
 
   Therefore, this function is used to check if a module is loaded
-  before using it and allows one to react accordingly. For example, the `URI` 
+  before using it and allows one to react accordingly. For example, the `URI`
   module uses this function to check if a specific parser exists for a given
   URI scheme.
 
@@ -387,17 +406,21 @@ defmodule Code do
   end
 
   @doc """
-  Similar to `ensure_loaded/1`, but returns a boolean in case
-  it could be ensured or not.
+  Ensure the given module is loaded.
+
+  Similar to `ensure_loaded/1`, but returns `true` if the module
+  is already loaded or was successfully loaded. Returns `false` otherwise.
   """
   def ensure_loaded?(module) do
     match?({ :module, ^module }, ensure_loaded(module))
   end
 
   @doc """
-  Ensures the given module is compiled and loaded. If the module
+  Ensure the given module is compiled and loaded. 
+  
+  If the module
   is already loaded, it works as no-op. If the module was not
-  loaded yet, it checks if it needs to be compiled first and 
+  loaded yet, it checks if it needs to be compiled first and
   then tries to load it.
 
   If it succeeds loading the module, it returns
@@ -425,8 +448,11 @@ defmodule Code do
   end
 
   @doc """
-  Similar to `ensure_compiled/1`, but returns a boolean in case
-  it could be ensured or not.
+  Ensure the given module is compiled and loaded. 
+
+  Similar to `ensure_compiled/1`, but returns `true` if the module
+  is already loaded or was successfully loaded and compiled. 
+  Returns `false` otherwise.
   """
   def ensure_compiled?(module) do
     match?({ :module, ^module }, ensure_compiled(module))

lib/elixir/lib/dict.ex

@@ -31,6 +31,11 @@ defmodule Dict do
         IO.puts "#{k}: #{v}"
       end)
 
+  ## Match
+
+  Dictionaries are required to implement all operations
+  using the match (`===`) operator. Any deviation from
+  this behaviour should be avoided and explicitly documented.
   """
 
   use Behaviour
@@ -65,7 +70,6 @@ defmodule Dict do
   defcallback update(t, key, value, (value -> value)) :: t
   defcallback update!(t, key, (value -> value)) :: t | no_return
   defcallback values(t) :: list(value)
-  defcallback reduce(t, any, ({key, value}, any -> any)) :: any
 
   defmacrop target(dict) do
     quote do
@@ -97,7 +101,7 @@ defmodule Dict do
   end
 
   @doc """
-  Returns a list of all values in `dict`
+  Returns a list of all values in `dict`.
   The values are not guaranteed to be in any order.
 
   ## Examples
@@ -420,7 +424,7 @@ defmodule Dict do
 
   @doc """
   Returns a new dict where only the keys in `keys` from `dict` are
-  included. 
+  included.
   Any non-member keys are ignored.
 
   ## Examples
@@ -458,8 +462,8 @@ defmodule Dict do
   end
 
   @doc """
-  Check if two dicts are equal. If the dicts are of different types, they are
-  first converted to lists.
+  Check if two dicts are equal using `===`. If the dicts are
+  of different types, they are first converted to lists.
 
   ## Examples
 

lib/elixir/lib/dict/behaviour.ex

@@ -1,9 +1,11 @@
 defmodule Dict.Behaviour do
   @moduledoc """
+  Default implementations for some required functions in the `Dict` module.
+
   This module makes it easier to create your own `Dict` compliant
   module, by providing default implementations for some required functions.
-  
-  Usage:
+
+  ## Example
 
       defmodule MyDict do
         use Dict.Behaviour
@@ -13,17 +15,39 @@ defmodule Dict.Behaviour do
         # override default implementations if needed
       end
 
-  The client module must contain following functions: `size/1`, `fetch/2`, 
-  `put/3`, `update/4`, `delete/2` and `reduce/3`. All of them are part of
-  the Dict behaviour, so no extra functions are actually required.
+  The client module must contain the following functions:
 
-  Based on these functions, `Dict.Behaviour` generates default implementations 
-  for other functions such as `drop`, `take`, etc. All of the functions are
-  defined as overridable, so you can provide your own implementation if
-  needed.
+  * `delete/2`
+  * `fetch/2`
+  * `put/3`
+  * `reduce/3`
+  * `size/1`
+  * `update/4`
 
-  If you implement `new/0` and `new/1` functions, you can also test your custom 
-  module via `Dict` doctests:
+  All of which are part of the `Dict` behaviour, so no extra functions are
+  actually required.
+
+  Based on these functions, `Dict.Behaviour` generates default implementations
+  for the following functions:
+
+  * `drop/2`
+  * `equal?/2`
+  * `fetch!/2`
+  * `get/2`
+  * `get/3`
+  * `has_key?/2`
+  * `keys/1`
+  * `merge/2`
+  * `merge/3`
+  * `put_new/3`
+  * `take/2`
+  * `to_list/1`
+  * `values/1`
+
+  All of these functions are defined as overridable, so you can provide your own
+  implementation if needed.
+
+  Note you can also test your custom module via `Dict`'s doctests:
 
       defmodule MyDict do
         def new(keywords // []) do
@@ -38,49 +62,42 @@ defmodule Dict.Behaviour do
       end
 
   """
-
   defmacro __using__(_) do
     quote do
       @behaviour Dict
-      
+
       def get(dict, key, default // nil) do
         case fetch(dict, key) do
           { :ok, value } -> value
           :error -> default
         end
       end
-      defoverridable get: 2, get: 3
 
       def fetch!(dict, key) do
         case fetch(dict, key) do
           { :ok, value } -> value
-          :error -> raise(KeyError, key: key)
+          :error -> raise KeyError, key: key
         end
       end
-      defoverridable fetch!: 2
- 
+
       def has_key?(dict, key) do
         match? { :ok, _ }, fetch(dict, key)
       end
-      defoverridable has_key?: 2
- 
+
       def put_new(dict, key, value) do
         update(dict, key, value, fn(v) -> v end)
       end
-      defoverridable put_new: 3
- 
+
       def drop(dict, []), do: dict
 
       def drop(dict, [key|keys]) do
         drop(delete(dict, key), keys)
       end
-      defoverridable drop: 2
- 
+
       def take(dict, keys) do
         take(dict, keys, new)
       end
-      defoverridable take: 2
- 
+
       defp take(_dict, [], acc), do: acc
       defp take(dict, [key|keys], acc) do
         case fetch(dict, key) do
@@ -88,38 +105,49 @@ defmodule Dict.Behaviour do
           :error -> take(dict, keys, acc)
         end
       end
- 
-      def to_list(dict), do: reduce(dict, [], &[&1|&2]) |> Enum.reverse
-      defoverridable to_list: 1
- 
-      def keys(dict), do: reduce(dict, [], fn({k, _}, acc) -> [k | acc] end) |> Enum.reverse
-      defoverridable keys: 1
- 
-      def values(dict), do: reduce(dict, [], fn({_, v}, acc) -> [v | acc] end) |> Enum.reverse
-      defoverridable values: 1
- 
+
+      def to_list(dict) do
+        reduce(dict, { :cont, [] }, fn
+          kv, acc -> { :cont, [kv|acc] }
+        end) |> elem(1) |> :lists.reverse
+      end
+
+      def keys(dict) do
+        reduce(dict, { :cont, [] }, fn
+          {k, _}, acc -> { :cont, [k|acc] }
+        end) |> elem(1) |> :lists.reverse
+      end
+
+      def values(dict) do
+        reduce(dict, { :cont, [] }, fn
+          {_, v}, acc -> { :cont, [v|acc] }
+        end) |> elem(1) |> :lists.reverse
+      end
+
       def equal?(dict1, dict2) do
+        import Kernel, except: [size: 1]
         case size(dict1) == size(dict2) do
           false -> false
-          true -> 
-            try do
-              reduce(dict1, nil, fn({ k, v }, _acc) ->
-                unless fetch(dict2, k) == { :ok, v }, do: throw(:error)
-              end)
-              true
-            catch
-              :error -> false
-            end
+          true  ->
+            reduce(dict1, { :cont, true }, fn({ k, v }, _acc) ->
+              unless fetch(dict2, k) == { :ok, v } do
+                { :halt, false }
+              else
+                { :cont, true }
+              end
+            end) |> elem(1)
         end
       end
-      defoverridable equal?: 2
- 
+
       def merge(dict, enumerable, callback // fn(_k, _v1, v2) -> v2 end) do
         Enum.reduce(enumerable, dict, fn({key, value}, acc) ->
           update(acc, key, value, fn(v1) -> callback.(key, v1, value) end)
         end)
       end
-      defoverridable merge: 2, merge: 3
+
+      defoverridable merge: 2, merge: 3, equal?: 2, to_list: 1, keys: 1,
+                     values: 1, take: 2, drop: 2, get: 2, get: 3, fetch!: 2,
+                     has_key?: 2, put_new: 3
     end
   end
-end
+end

lib/elixir/lib/exception.ex

@@ -1,3 +1,7 @@
+# Some exceptions implement `message/1` instead of `exception/1` mostly
+# for bootstrap reasons. It is recommended for applications to implement
+# `exception/1` instead of `message/1` as described in `defexception/3` docs.
+
 defexception RuntimeError,      message: "runtime error"
 defexception ArgumentError,     message: "argument error"
 defexception ArithmeticError,   message: "bad argument in arithmetic expression"
@@ -5,19 +9,22 @@ defexception SystemLimitError,  message: "a system limit has been reached"
 
 defexception SyntaxError, [file: nil, line: nil, description: "syntax error"] do
   def message(exception) do
-    "#{Exception.format_file_line(exception.file, exception.line)}#{exception.description}"
+    Exception.format_file_line(Path.relative_to_cwd(exception.file), exception.line) <>
+      exception.description
   end
 end
 
 defexception TokenMissingError, [file: nil, line: nil, description: "expression is incomplete"] do
   def message(exception) do
-    "#{Exception.format_file_line(exception.file, exception.line)}#{exception.description}"
+    Exception.format_file_line(Path.relative_to_cwd(exception.file), exception.line) <>
+      exception.description
   end
 end
 
 defexception CompileError, [file: nil, line: nil, description: "compile error"] do
   def message(exception) do
-    "#{Exception.format_file_line(exception.file, exception.line)}#{exception.description}"
+    Exception.format_file_line(Path.relative_to_cwd(exception.file), exception.line) <>
+      exception.description
   end
 end
 
@@ -39,9 +46,16 @@ defexception CaseClauseError, [actual: nil] do
   end
 end
 
+defexception TryClauseError, [actual: nil] do
+  def message(exception) do
+    "no try clause matching: #{inspect(exception.actual)}"
+  end
+end
+
 defexception BadArityError, [function: nil, args: nil] do
   def message(exception) do
-    "bad arity error: #{inspect(exception.function)} called with #{inspect(exception.args)}"
+    args = Enum.map_join(exception.args, ", ", &inspect/1)
+    "bad arity error: #{inspect(exception.function)} called with (#{args})"
   end
 end
 
@@ -94,25 +108,56 @@ defexception Enum.OutOfBoundsError, message: "out of bounds error"
 
 defexception Enum.EmptyError, message: "empty error"
 
+defexception File.Error, [reason: nil, action: "", path: nil] do
+  def message(exception) do
+    formatted = iolist_to_binary(:file.format_error(reason exception))
+    "could not #{action exception} #{path exception}: #{formatted}"
+  end
+end
+
+defexception File.CopyError, [reason: nil, action: "", source: nil, destination: nil, on: nil] do
+  def message(exception) do
+    formatted = iolist_to_binary(:file.format_error(reason exception))
+    location  = if on = on(exception), do: ". #{on}", else: ""
+    "could not #{action exception} from #{source exception} to " <>
+      "#{destination exception}#{location}: #{formatted}"
+  end
+end
+
 defmodule Exception do
   @moduledoc """
-  Several convenience functions to work with and pretty print
+  Convenience functions to work with and pretty print
   exceptions and stacktraces.
+
+  Notice that stacktraces in Elixir are updated on errors.
+  For example, at any given moement, `System.stacktrace`
+  will return the stacktrace for the last error that ocurred
+  in the current process.
+
+  That said, many of the functions in this module will
+  automatically calculate the stacktrace based on the caller,
+  when invoked without arguments, changing the value of
+  `System.stacktrace`. If instead you want to format the
+  stacktrace of the latest error, you should instead explicitly
+  pass the `System.stacktrace` as argument.
   """
 
   @doc """
   Normalizes an exception, converting Erlang exceptions
-  to Elixir exceptions. It takes the kind spilled by
-  `catch` as an argument as a convenience for converting only
-  `:errors`, ignorning the others.
+  to Elixir exceptions.
+
+  It takes the `kind` spilled by `catch` as an argument and
+  normalizes only `:error`, returning the untouched payload
+  for others.
   """
   def normalize(:error, exception), do: normalize(exception)
   def normalize(_kind, other), do: other
 
   @doc """
   Normalizes an exception, converting Erlang exceptions
-  to Elixir exceptions. Useful when interfacing Erlang
-  code with Elixir code.
+  to Elixir exceptions.
+
+  Useful when interfacing Erlang code with Elixir code.
   """
   def normalize(exception) when is_exception(exception) do
     exception
@@ -146,6 +191,10 @@ defmodule Exception do
     CaseClauseError[actual: actual]
   end
 
+  def normalize({ :try_clause, actual }) do
+    TryClauseError[actual: actual]
+  end
+
   def normalize(:undef) do
     { mod, fun, arity } = from_stacktrace(:erlang.get_stacktrace)
     UndefinedFunctionError[module: mod, function: fun, arity: arity]
@@ -166,41 +215,45 @@ defmodule Exception do
 
   @doc """
   Receives a tuple representing a stacktrace entry and formats it.
-
-  The current working directory may be given as an argument,
-  otherwise one is automatically retrieved.
   """
-  def format_stacktrace_entry(entry, cwd // nil)
+  def format_stacktrace_entry(entry)
 
   # From Macro.Env.stacktrace
-  def format_stacktrace_entry({ module, :__MODULE__, 0, file_line }, cwd) do
-    "#{format_location(file_line, cwd)}#{inspect module} (module)"
+  def format_stacktrace_entry({ module, :__MODULE__, 0, location }) do
+    format_location(location) <> inspect(module) <> " (module)"
   end
 
   # From :elixir_compiler_*
-  def format_stacktrace_entry({ _module, :__MODULE__, 2, file_line }, cwd) do
-    "#{format_location(file_line, cwd)}(module)"
+  def format_stacktrace_entry({ _module, :__MODULE__, 1, location }) do
+    format_location(location) <> "(module)"
   end
 
   # From :elixir_compiler_*
-  def format_stacktrace_entry({ _module, :__FILE__, 2, file_line }, cwd) do
-    "#{format_location(file_line, cwd)}(file)"
+  def format_stacktrace_entry({ _module, :__FILE__, 1, location }) do
+    format_location(location) <> "(file)"
   end
 
-  def format_stacktrace_entry({module, fun, arity, file_line}, cwd) do
-    "#{format_location(file_line, cwd)}#{format_mfa(module, fun, arity)}"
+  def format_stacktrace_entry({module, fun, arity, location}) do
+    format_application(module) <> format_location(location) <> format_mfa(module, fun, arity)
   end
 
-  def format_stacktrace_entry({fun, arity, file_line}, cwd) do
-    "#{format_location(file_line, cwd)}#{format_fa(fun, arity)}"
+  def format_stacktrace_entry({fun, arity, location}) do
+    format_location(location) <> format_fa(fun, arity)
+  end
+
+  defp format_application(module) do
+    case :application.get_application(module) do
+      { :ok, app } -> "(" <> atom_to_binary(app) <> ") "
+      :undefined   -> ""
+    end
   end
 
   @doc """
   Formats the stacktrace.
 
   A stacktrace must be given as an argument. If not, this function
-  calculates the current stacktrace and formats it. As a consequence,
-  the value of `System.stacktrace` is changed.
+  calculates a new stacktrace based on the caller and formats it. As
+  a consequence, the value of `System.stacktrace` is changed.
   """
   def format_stacktrace(trace // nil) do
     trace = trace || try do
@@ -209,16 +262,19 @@ defmodule Exception do
       :stacktrace -> Enum.drop(:erlang.get_stacktrace, 1)
     end
 
-    cwd = System.cwd
-
     case trace do
       [] -> "\n"
-      s  -> "    " <> Enum.map_join(s, "\n    ", &format_stacktrace_entry(&1, cwd)) <> "\n"
+      s  -> "    " <> Enum.map_join(s, "\n    ", &format_stacktrace_entry(&1)) <> "\n"
     end
   end
 
   @doc """
   Formats the caller, i.e. the first entry in the stacktrace.
+
+  A stacktrace must be given as an argument. If not, this function
+  calculates a new stacktrace based on the caller and formats it. As
+  a consequence, the value of `System.stacktrace` is changed.
+
   Notice that due to tail call optimization, the stacktrace
   may not report the direct caller of the function.
   """
@@ -266,6 +322,8 @@ defmodule Exception do
       "Foo.bar/1"
       iex> Exception.format_mfa Foo, :bar, []
       "Foo.bar()"
+      iex> Exception.format_mfa nil, :bar, []
+      "nil.bar()"
 
   """
   def format_mfa(module, fun, arity) do
@@ -287,9 +345,6 @@ defmodule Exception do
   Formats the given file and line as shown in stacktraces.
   If any of the values are nil, they are omitted.
 
-  The current working directory may be given as an argument,
-  otherwise one is automatically retrieved.
-
   ## Examples
 
       iex> Exception.format_file_line("foo", 1)
@@ -302,11 +357,8 @@ defmodule Exception do
       ""
 
   """
-  def format_file_line(file, line, cwd // nil) do
+  def format_file_line(file, line) do
     if file do
-      file = to_string(file)
-      file = if cwd, do: Path.relative_to(file, cwd), else: Path.relative_to_cwd(file)
-
       if line && line != 0 do
         "#{file}:#{line}: "
       else
@@ -317,8 +369,8 @@ defmodule Exception do
     end
   end
 
-  defp format_location(opts, cwd) do
-    format_file_line Keyword.get(opts, :file), Keyword.get(opts, :line), cwd
+  defp format_location(opts) do
+    format_file_line Keyword.get(opts, :file), Keyword.get(opts, :line)
   end
 
   defp from_stacktrace([{ module, function, args, _ }|_]) when is_list(args) do

lib/elixir/lib/file.ex

@@ -31,29 +31,6 @@ defrecord File.Stat, Record.extract(:file_info, from_lib: "kernel/include/file.h
   """
 end
 
-defexception File.Error, [reason: nil, action: "", path: nil] do
-  def message(exception) do
-    formatted = iolist_to_binary(:file.format_error(reason exception))
-    "could not #{action exception} #{path exception}: #{formatted}"
-  end
-end
-
-defexception File.CopyError, [reason: nil, action: "", source: nil, destination: nil, on: nil] do
-  def message(exception) do
-    formatted = iolist_to_binary(:file.format_error(reason exception))
-    location  = if on = on(exception), do: ". #{on}", else: ""
-    "could not #{action exception} from #{source exception} to " <>
-      "#{destination exception}#{location}: #{formatted}"
-  end
-end
-
-defexception File.IteratorError, reason: nil do
-  def message(exception) do
-    formatted = iolist_to_binary(:file.format_error(reason exception))
-    "error during file iteration: #{formatted}"
-  end
-end
-
 defmodule File do
   @moduledoc """
   This module contains functions to manipulate files.
@@ -70,9 +47,9 @@ defmodule File do
   in the `IO` module. By default, a file is opened in binary mode
   which requires the functions `IO.binread/2` and `IO.binwrite/2`
   to interact with the file. A developer may pass `:utf8` as an
-  option when opening the file and then all other functions
-  from `IO` are available, since they work directly with Unicode
-  data.
+  option when opening the file, then the slower `IO.read/2` and
+  `IO.write/2` functions must be used as they are responsible for
+  doing the proper conversions and data guarantees.
 
   Most of the functions in this module return `:ok` or
   `{ :ok, result }` in case of success, `{ :error, reason }`
@@ -108,7 +85,7 @@ defmodule File do
 
   ## Examples
 
-      File.regular? __FILE__ #=> true
+      File.regular? __ENV__.file #=> true
 
   """
   def regular?(path) do
@@ -488,7 +465,7 @@ defmodule File do
         case F.list_dir(src) do
           { :ok, files } ->
             case mkdir(dest) do
-              success in [:ok, { :error, :eexist }] ->
+              success when success in [:ok, { :error, :eexist }] ->
                 Enum.reduce(files, [dest|acc], fn(x, acc) ->
                   do_cp_r(FN.join(src, x), FN.join(dest, x), callback, acc)
                 end)
@@ -567,6 +544,8 @@ defmodule File do
   * :enospc - There is a no space left on the device.
   * :eacces - Missing permission for writing the file or searching one of the parent directories.
   * :eisdir - The named file is a directory.
+
+  Check `File.open/2` for existing modes.
   """
   def write(path, content, modes // []) do
     F.write_file(path, content, modes)
@@ -670,7 +649,7 @@ defmodule File do
     do_rm_rf(path, { :ok, [] })
   end
 
-  defp do_rm_rf(path, { :ok, acc } = entry) do
+  defp do_rm_rf(path, { :ok, _ } = entry) do
     case safe_list_dir(path) do
       { :ok, files } when is_list(files) ->
         res =
@@ -945,7 +924,7 @@ defmodule File do
   """
   def ls(path // ".") do
     case F.list_dir(path) do
-      { :ok, file_list } -> { :ok, Enum.map(file_list, to_string(&1)) }
+      { :ok, file_list } -> { :ok, Enum.map(file_list, &to_string(&1)) }
       { :error, _ } = error -> error
     end
   end
@@ -975,39 +954,72 @@ defmodule File do
   end
 
   @doc """
-  Opens the given `file` with the given `mode` and returns
-  a stream for each `:line` (default) or for a given number
+  Opens the file at the given `path` with the given `modes` and
+  returns a stream for each `:line` (default) or for a given number
   of bytes given by `line_or_bytes`.
 
-  The returned stream will fail for the same reasons as `File.open!`.
-  Note that the file is opened only when streaming begins.
+  The returned stream will fail for the same reasons as
+  `File.open!/2`. Note that the file is opened only and every time
+  streaming begins.
+
+  Note that stream by default uses `IO.binread/2` unless
+  the file is opened with an encoding, then the slower `IO.read/2`
+  is used to do the proper data conversion and guarantees.
   """
-  def stream!(file, mode // [], line_or_bytes // :line) do
-    fn(acc, fun) ->
-      device = open!(file, mode)
-      try do
-        IO.stream(device, line_or_bytes, acc, fun)
-      after
-        F.close(device)
+  def stream!(path, modes // [], line_or_bytes // :line) do
+    modes = open_defaults(modes, true)
+    bin   = nil? List.keyfind(modes, :encoding, 0)
+
+    start_fun =
+      fn ->
+        case F.open(path, modes) do
+          { :ok, device }    -> device
+          { :error, reason } ->
+            raise File.Error, reason: reason, action: "stream", path: to_string(path)
+        end
       end
-    end
+
+    next_fun =
+      case bin do
+        true  -> &IO.do_binstream(&1, line_or_bytes)
+        false -> &IO.do_stream(&1, line_or_bytes)
+      end
+
+    Stream.resource(start_fun, next_fun, &F.close/1)
   end
 
   @doc """
-  Opens the given `file` with the given `mode` and returns
-  a binstream for each `:line` (default) or for a given number
-  of bytes given by `line_or_bytes`.
+  Receives a stream and returns a new stream that will open the file
+  at the given `path` for write  with the extra `modes` and write
+  each value to the file.
 
-  The returned stream will fail for the same reasons as `File.open!`.
-  Note that the file is opened only when streaming begins.
+  The returned stream will fail for the same reasons as
+  `File.open!/2`. Note that the file is opened only and every time
+  streaming begins.
+
+  Note that stream by default uses `IO.binwrite/2` unless
+  the file is opened with an encoding, then the slower `IO.write/2`
+  is used to do the proper data conversion and guarantees.
   """
-  def binstream!(file, mode // [], line_or_bytes // :line) do
-    fn(fun, acc) ->
-      device = open!(file, mode)
-      try do
-        IO.binstream(device, line_or_bytes, fun, acc)
-      after
-        F.close(device)
+  def stream_to!(stream, path, modes // []) do
+    modes = open_defaults([:write|List.delete(modes, :write)], true)
+    bin   = nil? List.keyfind(modes, :encoding, 0)
+
+    fn acc, f ->
+      case F.open(path, modes) do
+        { :ok, device } ->
+          each =
+            case bin do
+              true  -> &IO.binwrite(device, &1)
+              false -> &IO.write(device, &1)
+            end
+
+          stream
+          |> Stream.each(each)
+          |> Stream.after(fn -> F.close(device) end)
+          |> Enumerable.Stream.Lazy.reduce(acc, f)
+        { :error, reason } ->
+          raise File.Error, reason: reason, action: "stream_to", path: to_string(path)
       end
     end
   end
@@ -1053,7 +1065,7 @@ defmodule File do
   end
 
   @doc """
-  Changes the owner given by the user id `gid`
+  Changes the owner given by the user id `uid`
   for a given `file`. Returns `:ok` on success,
   or `{:error, reason}` on failure.
   """
@@ -1064,8 +1076,8 @@ defmodule File do
   @doc """
   Same as `chown/2`, but raises an exception in case of failure. Otherwise `:ok`.
   """
-  def chown!(file, gid) do
-    case chown(file, gid) do
+  def chown!(file, uid) do
+    case chown(file, uid) do
       :ok -> :ok
       { :error, reason } ->
         raise File.Error, reason: reason, action: "change owner for", path: to_string(file)

lib/elixir/lib/float.ex

@@ -0,0 +1,162 @@
+defmodule Float do
+  @moduledoc """
+  Functions for working with floating point numbers.
+  """
+
+  @doc """
+  Parses a binary into a float.
+
+  If successful, returns a tuple of the form `{ float, remainder_of_binary }`.
+  Otherwise `:error`.
+
+  ## Examples
+
+      iex> Float.parse("34")
+      {34.0,""}
+      iex> Float.parse("34.25")
+      {34.25,""}
+      iex> Float.parse("56.5xyz")
+      {56.5,"xyz"}
+      iex> Float.parse("pi")
+      :error
+
+  """
+  @spec parse(binary) :: { float, binary } | :error
+  def parse("-" <> binary) do
+    case parse_unsign(binary) do
+      :error -> :error
+      { number, remainder } -> { -number, remainder }
+    end
+  end
+
+  def parse(binary) do
+    parse_unsign(binary)
+  end
+
+  defp parse_unsign("-" <> _), do: :error
+  defp parse_unsign(binary) when is_binary(binary) do
+    case Integer.parse binary do
+      :error -> :error
+      { integer_part, after_integer } -> parse_unsign after_integer, integer_part
+    end
+  end
+
+  # Dot followed by digit is required afterwards or we are done
+  defp parse_unsign(<< ?., char, rest :: binary >>, int) when char in ?0..?9 do
+    parse_unsign(rest, char - ?0, 1, int)
+  end
+
+  defp parse_unsign(rest, int) do
+    { :erlang.float(int), rest }
+  end
+
+  # Handle decimal points
+  defp parse_unsign(<< char, rest :: binary >>, float, decimal, int) when char in ?0..?9 do
+    parse_unsign rest, 10 * float + (char - ?0), decimal + 1, int
+  end
+
+  defp parse_unsign(<< ?e, after_e :: binary >>, float, decimal, int) do
+    case Integer.parse after_e do
+      :error ->
+        # Note we rebuild the binary here instead of breaking it apart at
+        # the function clause because the current approach copies a binary
+        # just on this branch. If we broke it apart in the function clause,
+        # the copy would happen when calling Integer.parse/1.
+        { floatify(int, float, decimal), << ?e, after_e :: binary >> }
+      { exponential, after_exponential } ->
+        { floatify(int, float, decimal, exponential), after_exponential }
+    end
+  end
+
+  defp parse_unsign(bitstring, float, decimal, int) do
+    { floatify(int, float, decimal), bitstring }
+  end
+
+  defp floatify(int, float, decimal, exponential // 0) do
+    multiplier = if int < 0, do: -1.0, else: 1.0
+
+    # Try to ensure the minimum amount of rounding errors
+    result = multiplier * (abs(int) * :math.pow(10, decimal) + float) * :math.pow(10, exponential - decimal)
+
+    # Try avoiding stuff like this:
+    # iex(1)> 0.0001 * 75
+    # 0.007500000000000001
+    # Due to IEEE 754 floating point standard
+    # http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html
+
+    final_decimal_places = decimal - exponential
+    if final_decimal_places > 0 do
+      decimal_power_round = :math.pow(10,  final_decimal_places)
+      trunc(result * decimal_power_round) / decimal_power_round
+    else
+      result
+    end
+  end
+
+  @doc """
+  Round a float to the largest integer less than or equal to `num`
+
+  ## Examples
+
+      iex> Float.floor(34)
+      34
+      iex> Float.floor(34.25)
+      34
+      iex> Float.floor(-56.5)
+      -57
+
+  """
+  @spec floor(float | integer) :: integer
+  def floor(num) when is_integer(num), do: num
+  def floor(num) when is_float(num) do
+    truncated = :erlang.trunc(num)
+    case :erlang.abs(num - truncated) do
+      x when x > 0 and num < 0 -> truncated - 1
+      _ -> truncated
+    end
+  end
+
+  @doc """
+  Round a float to the largest integer greater than or equal to `num`
+
+  ## Examples
+
+      iex> Float.ceil(34)
+      34
+      iex> Float.ceil(34.25)
+      35
+      iex> Float.ceil(-56.5)
+      -56
+
+  """
+  @spec ceil(float | integer) :: integer
+  def ceil(num) when is_integer(num), do: num
+  def ceil(num) when is_float(num) do
+    truncated = :erlang.trunc(num)
+    case :erlang.abs(num - truncated) do
+      x when x > 0 and num > 0 -> truncated + 1
+      _ -> truncated
+    end
+  end
+
+  @doc """
+  Rounds a floating point value to an arbitrary number of fractional digits
+  (between 0 and 15).
+
+  ## Examples
+
+      iex> Float.round(5.5674, 3)
+      5.567
+      iex> Float.round(5.5675, 3)
+      5.568
+      iex> Float.round(-5.5674, 3)
+      -5.567
+      iex> Float.round(-5.5675, 3)
+      -5.568
+  """
+  @spec round(float, integer) :: float
+  def round(number, precision) when is_float(number) and is_integer(precision) and precision in 0..15 do
+    Kernel.round(number * :math.pow(10, precision)) / :math.pow(10, precision)
+  end
+
+end

lib/elixir/lib/gen_event/behaviour.ex

@@ -25,7 +25,6 @@ defmodule GenEvent.Behaviour do
         def handle_call(:notifications, notifications) do
           {:ok, Enum.reverse(notifications), []}
         end
-
       end
 
       { :ok, pid } = :gen_event.start_link

lib/elixir/lib/gen_fsm/behaviour.ex

@@ -21,78 +21,67 @@ defmodule GenFSM.Behaviour do
   works - we hereby declare a full disclaimer for any lawsuits that the
   behaviour of the CVM in its original state might encur.
 
-
       defmodule MyFsm do
         use GenFSM.Behaviour
 
-          # keeping track of what is going on inside the CVM.
-          # 3 is the target price for a cup of coffee
-          defrecord StateData, coins: 0, price: 3
+        # keeping track of what is going on inside the CVM.
+        # 3 is the target price for a cup of coffee
+        defrecord StateData, coins: 0, price: 3
 
-          #
-          # API functions
-          #
+        # API functions
 
-          def start_link() do
-            :gen_fsm.start_link({:local, :cvm}, __MODULE__, [], [])
-          end
-
-          def insert_coin() do
-            :gen_fsm.send_event(:cvm, :coin)
-          end
-
-          def request_coffee() do
-            :gen_fsm.send_event(:cvm, :request_coffee)
-          end
-
-          #
-          # Callbacks
-          #
-
-          def init(_args) do
-            { :ok, :short_paid, StateData.new }
-          end
-
-
-          def short_paid(:coin, state_data = StateData[coins: c, price: p]) 
-           when c + 1 < p do
-            { :next_state, :short_paid, state_data.coins(c + 1) }
-          end
-
-          def short_paid(:coin, state_data) do
-            { :next_state, :paid_in_full, state_data.update_coins(&1 + 1) }
-          end
-
-          def short_paid(:request_coffee, state_data) do
-            { :next_state, :requested_short_paid, state_data }
-          end
-
-
-          def requested_short_paid(:request_coffee, state_data) do
-            {:next_state, :requested_short_paid, state_data }
-          end
-
-          def requested_short_paid(:coin, state_data=StateData[coins: c, price: p]) 
-           when c+1 < p do
-            { :next_state, :requested_short_paid, state_data.coins(c + 1) }
-          end
-
-          def requested_short_paid(:coin, _state_data) do
-            IO.puts "Here's your coffee!"
-            { :next_state, :short_paid, StateData.new }
-          end
-
-
-          def paid_in_full(:coin, state_data) do
-            { :next_state, :paid_in_full, state_data.update_coins(&1 + 1) }
-          end
-
-          def paid_in_full(:request_coffee, _state_data) do
-            IO.puts "Here's your coffee!"
-            { :next_state, :short_paid, StateData.new }
-          end
+        def start_link() do
+          :gen_fsm.start_link({:local, :cvm}, __MODULE__, [], [])
         end
 
+        def insert_coin() do
+          :gen_fsm.send_event(:cvm, :coin)
+        end
+
+        def request_coffee() do
+          :gen_fsm.send_event(:cvm, :request_coffee)
+        end
+
+        # Callbacks
+
+        def init(_args) do
+          { :ok, :short_paid, StateData.new }
+        end
+
+        def short_paid(:coin, state_data = StateData[coins: c, price: p]) when c + 1 < p do
+          { :next_state, :short_paid, state_data.coins(c + 1) }
+        end
+
+        def short_paid(:coin, state_data) do
+          { :next_state, :paid_in_full, &state_data.update_coins(&1 + 1) }
+        end
+
+        def short_paid(:request_coffee, state_data) do
+          { :next_state, :requested_short_paid, state_data }
+        end
+
+        def requested_short_paid(:request_coffee, state_data) do
+          { :next_state, :requested_short_paid, state_data }
+        end
+
+        def requested_short_paid(:coin, state_data=StateData[coins: c, price: p]) when c+1 < p do
+          { :next_state, :requested_short_paid, state_data.coins(c + 1) }
+        end
+
+        def requested_short_paid(:coin, _state_data) do
+          IO.puts "Here's your coffee!"
+          { :next_state, :short_paid, StateData.new }
+        end
+
+        def paid_in_full(:coin, state_data) do
+          { :next_state, :paid_in_full, &state_data.update_coins(&1 + 1) }
+        end
+
+        def paid_in_full(:request_coffee, _state_data) do
+          IO.puts "Here's your coffee!"
+          { :next_state, :short_paid, StateData.new }
+        end
+      end
 
       { :ok, _pid } = MyFsm.start_link()
 
@@ -138,15 +127,15 @@ defmodule GenFSM.Behaviour do
   for you. The list of callbacks are:
 
   * `init(args)` - invoked when the FSM is started;
-  * `handle_sync_event(event, from, state_name, state_data)` - invoked to 
+  * `handle_sync_event(event, from, state_name, state_data)` - invoked to
   handle `sync_send_all_state_event` messages;
-  * `handle_event(event, state_name, state_data)` - invoked to handle 
+  * `handle_event(event, state_name, state_data)` - invoked to handle
   `send_all_state_event` messages;
-  * `handle_info(msg, state_name, state_data)` - handle all other 
+  * `handle_info(msg, state_name, state_data)` - handle all other
   messages which are normally received by processes;
-  * `terminate(reason, state_name, state_data)` - called when the FSM 
+  * `terminate(reason, state_name, state_data)` - called when the FSM
   is about to terminate, useful for cleaning up;
-  * `code_change(old_vsn, state, extra)` - called when the application 
+  * `code_change(old_vsn, state, extra)` - called when the application
   code is being upgraded live (hot code swap);
 
   Unlike `GenServer` and `GenEvent`, the callback `init/1` is not
@@ -154,9 +143,9 @@ defmodule GenFSM.Behaviour do
 
   For each state you need to define either or both of these:
 
-  * `state_name(event, state_data)` - invoked to handle 
+  * `state_name(event, state_data)` - invoked to handle
   `send_event` messages;
-  * `state_name(event, from, state_data)`- invoked to handle 
+  * `state_name(event, from, state_data)`- invoked to handle
   `sync_send_event` messages;
 
   If you send asynchronous events you only need to implement the
@@ -180,13 +169,13 @@ defmodule GenFSM.Behaviour do
       @behavior :gen_fsm
 
       @doc false
-      def handle_event(_event, state_name, state_data) do
-        { :next_state, state_name, state_data }
+      def handle_event(event, state_name, state_data) do
+        { :stop, {:bad_event, state_name, event}, state_data }
       end
 
       @doc false
-      def handle_sync_event(_event, from, state_name, state_data) do
-        { :reply, :default_implementation, state_name, state_data }
+      def handle_sync_event(event, from, state_name, state_data) do
+        { :stop, {:bad_sync_event, state_name, event}, state_data }
       end
 
       @doc false

lib/elixir/lib/gen_server/behaviour.ex

@@ -94,8 +94,8 @@ defmodule GenServer.Behaviour do
       end
 
       @doc false
-      def handle_call(_request, _from, state) do
-        { :noreply, state }
+      def handle_call(request, _from, state) do
+        { :stop, { :bad_call, request }, state }
       end
 
       @doc false
@@ -104,8 +104,8 @@ defmodule GenServer.Behaviour do
       end
 
       @doc false
-      def handle_cast(_msg, state) do
-        { :noreply, state }
+      def handle_cast(msg, state) do
+        { :stop, { :bad_cast, msg }, state }
       end
 
       @doc false

lib/elixir/lib/hash_dict.ex

@@ -107,12 +107,6 @@ defmodule HashDict do
     dict
   end
 
-  @doc false
-  def update(dict, key, fun) when is_function(fun, 1) do
-    IO.write "HashDict.update/3 is deprecated, please use HashDict.update!/3 instead\n#{Exception.format_stacktrace}"
-    update!(dict, key, fun)
-  end
-
   def update!(dict, key, fun) when is_function(fun, 1) do
     case dict_put(dict, key, { :update, nil, fun }) do
       { dict, 0 } ->
@@ -159,13 +153,13 @@ defmodule HashDict do
 
   def merge(dict1, dict2, callback) when is_record(dict1, HashDict) and is_record(dict2, HashDict) and elem(dict1, 1) < elem(dict2, 1) do
     dict_fold dict1, dict2, fn [k|v1], acc ->
-      update(acc, k, v1, callback.(k, v1, &1))
+      update(acc, k, v1, &callback.(k, v1, &1))
     end
   end
 
   def merge(dict1, dict2, callback) when is_record(dict1, HashDict) and is_record(dict2, HashDict) do
     dict_fold dict2, dict1, fn [k|v2], acc ->
-      update(acc, k, v2, callback.(k, &1, v2))
+      update(acc, k, v2, &callback.(k, &1, v2))
     end
   end
 
@@ -194,7 +188,8 @@ defmodule HashDict do
   end
 
   def reduce(trie(root: root, depth: depth), acc, fun) do
-    node_reduce(root, depth, acc, fun, @node_size)
+    node_reduce(root, depth, acc, fun, @node_size,
+                fn { :cont, acc }, _f -> { :done, acc } end)
   end
 
   ## Dict-wide functions
@@ -334,13 +329,10 @@ defmodule HashDict do
   end
 
   # Reduces the bucket
-  defp bucket_reduce([[k|v]|t], acc, fun) do
-    bucket_reduce(t, fun.({ k, v }, acc), fun)
-  end
-
-  defp bucket_reduce([], acc, _fun) do
-    acc
-  end
+  defp bucket_reduce(_, { :halt, acc }, _fun),        do: { :halted, acc }
+  defp bucket_reduce(list, { :suspend, acc }, fun),   do: { :suspended, acc, &bucket_reduce(list, &1, fun) }
+  defp bucket_reduce([[k|v]|t], { :cont, acc }, fun), do: bucket_reduce(t, fun.({ k, v }, acc), fun)
+  defp bucket_reduce([], { :cont, acc }, _fun),       do: { :done, acc }
 
   defp bucket_fold(bucket, acc, fun) do
     :lists.foldl(fun, acc, bucket)
@@ -405,17 +397,29 @@ defmodule HashDict do
   end
 
   # Reduces a node recursively
-  defp node_reduce(bucket, -1, acc, fun, _) do
-    bucket_reduce(bucket, acc, fun)
+  defp node_reduce(_, -1, { :halt, acc }, _fun, _count, _next) do
+    { :halted, acc }
   end
 
-  defp node_reduce(node, depth, acc, fun, count) when count >= 1 do
-    acc = node_reduce(:erlang.element(count, node), depth - 1, acc, fun, @node_size)
-    node_reduce(node, depth, acc, fun, count - 1)
+  defp node_reduce(list, -1, { :suspend, acc }, fun, count, next) do
+    { :suspended, acc, &node_reduce(list, -1, &1, fun, count, next) }
   end
 
-  defp node_reduce(_node, _, acc, _fun, 0) do
-    acc
+  defp node_reduce([[k|v]|t], -1, { :cont, acc }, fun, _count, next) do
+    node_reduce(t, -1, fun.({ k, v }, acc), fun, _count, next)
+  end
+
+  defp node_reduce([], -1, { :cont, _ } = acc, fun, _count, next) do
+    next.(acc, fun)
+  end
+
+  defp node_reduce(node, depth, acc, fun, count, next) when count >= 1 do
+    node_reduce(:erlang.element(count, node), depth - 1, acc, fun, @node_size,
+                &node_reduce(node, depth, &1, &2, count - 1, next))
+  end
+
+  defp node_reduce(_node, _, acc, fun, 0, next) do
+    next.(acc, fun)
   end
 
   # Folds a node recursively
@@ -474,9 +478,9 @@ end
 
 defimpl Enumerable, for: HashDict do
   def reduce(dict, acc, fun),  do: HashDict.reduce(dict, acc, fun)
-  def member?(dict, { k, v }), do: match?({ :ok, ^v }, HashDict.fetch(dict, k))
-  def member?(_dict, _),       do: false
-  def count(dict),             do: HashDict.size(dict)
+  def member?(dict, { k, v }), do: { :ok, match?({ :ok, ^v }, HashDict.fetch(dict, k)) }
+  def member?(_dict, _),       do: { :ok, false }
+  def count(dict),             do: { :ok, HashDict.size(dict) }
 end
 
 defimpl Access, for: HashDict do

lib/elixir/lib/hash_set.ex

@@ -103,7 +103,7 @@ defmodule HashSet do
   end
 
   def to_list(set) do
-    set_fold(set, [], [&1|&2]) |> :lists.reverse
+    set_fold(set, [], &[&1|&2]) |> :lists.reverse
   end
 
   def put(set, member) do
@@ -136,11 +136,12 @@ defmodule HashSet do
 
   @doc false
   def reduce(ordered(bucket: bucket), acc, fun) do
-    :lists.foldl(fun, acc, bucket)
+    bucket_reduce(bucket, acc, fun)
   end
 
-  def reduce(trie() = set, acc, fun) do
-    set_fold(set, acc, fun)
+  def reduce(trie(root: root, depth: depth), acc, fun) do
+    node_reduce(root, depth, acc, fun, @node_size,
+                fn { :cont, acc }, _f -> { :done, acc } end)
   end
 
   ## HashSet-wide functions
@@ -225,29 +226,21 @@ defmodule HashSet do
   end
 
   defp set_equal?(set1, set2) do
-    try do
-      reduce(set1, true, fn member, acc ->
-        case member?(set2, member) do
-          true -> acc
-          _    -> throw(:error)
-        end
-      end)
-    catch
-      :error -> false
-    end
+    reduce(set1, { :cont, true }, fn member, acc ->
+      case member?(set2, member) do
+        true -> { :cont, acc }
+        _    -> { :halt, false }
+      end
+    end) |> elem(1)
   end
 
   defp set_disjoint?(set1, set2) do
-    try do
-      reduce(set1, true, fn member, acc ->
-        case member?(set2, member) do
-          false -> acc
-          _     -> throw(:error)
-        end
-      end)
-    catch
-      :error -> false
-    end
+    reduce(set1, { :cont, true }, fn member, acc ->
+      case member?(set2, member) do
+        false -> { :cont, acc }
+        _     -> { :halt, false }
+      end
+    end) |> elem(1)
   end
 
   defp set_fold(ordered(bucket: bucket), acc, fun) do
@@ -311,6 +304,11 @@ defmodule HashSet do
     { [], 0 }
   end
 
+  defp bucket_reduce(_, { :halt, acc }, _fun),      do: { :halted, acc }
+  defp bucket_reduce(list, { :suspend, acc }, fun), do: { :suspended, acc, &bucket_reduce(list, &1, fun) }
+  defp bucket_reduce([h|t], { :cont, acc }, fun),   do: bucket_reduce(t, fun.(h, acc), fun)
+  defp bucket_reduce([], { :cont, acc }, _fun),     do: { :done, acc }
+
   defp bucket_fold(bucket, acc, fun) do
     :lists.foldl(fun, acc, bucket)
   end
@@ -396,6 +394,33 @@ defmodule HashSet do
     end
   end
 
+  # Reduces a node recursively
+  defp node_reduce(_, -1, { :halt, acc }, _fun, _count, _next) do
+    { :halted, acc }
+  end
+
+  defp node_reduce(list, -1, { :suspend, acc }, fun, count, next) do
+    { :suspended, acc, &node_reduce(list, -1, &1, fun, count, next) }
+  end
+
+  defp node_reduce([h|t], -1, { :cont, acc }, fun, _count, next) do
+    node_reduce(t, -1, fun.(h, acc), fun, _count, next)
+  end
+
+  defp node_reduce([], -1, { :cont, _ } = acc, fun, _count, next) do
+    next.(acc, fun)
+  end
+
+  defp node_reduce(node, depth, acc, fun, count, next) when count >= 1 do
+    node_reduce(:erlang.element(count, node), depth - 1, acc, fun, @node_size,
+                &node_reduce(node, depth, &1, &2, count - 1, next))
+  end
+
+  defp node_reduce(_node, _, acc, fun, 0, next) do
+    next.(acc, fun)
+  end
+
+  # Folds a node recursively
   defp node_fold(bucket, -1, acc, fun, _) do
     bucket_fold(bucket, acc, fun)
   end
@@ -476,17 +501,10 @@ defmodule HashSet do
   defp each_contract(acc, [m|bucket]), do: [m|each_contract(acc, bucket)]
   defp each_contract([], bucket), do: bucket
   defp each_contract(acc, []), do: acc
-
-  @doc false
-  def inspect_depth(trie(depth: d)), do: d
-  @doc false
-  def inspect_contract(trie(contract_on: c)), do: c
-  @doc false
-  def inspect_expand(trie(expand_on: e)), do: e
 end
 
 defimpl Enumerable, for: HashSet do
   def reduce(set, acc, fun), do: HashSet.reduce(set, acc, fun)
-  def member?(set, v),       do: HashSet.member?(set, v)
-  def count(set),            do: HashSet.size(set)
+  def member?(set, v),       do: { :ok, HashSet.member?(set, v) }
+  def count(set),            do: { :ok, HashSet.size(set) }
 end

lib/elixir/lib/inspect.ex

@@ -1,7 +1,13 @@
 import Kernel, except: [inspect: 1]
 import Inspect.Algebra
 
-defrecord Inspect.Opts, raw: false, limit: 50, pretty: false, width: 80
+defrecord Inspect.Opts,
+  records: true,
+  binaries: :infer,
+  char_lists: :infer,
+  limit: 50,
+  pretty: false,
+  width: 80
 
 defprotocol Inspect do
   @moduledoc """
@@ -13,11 +19,9 @@ defprotocol Inspect do
   followed by the inspecting options, represented by the record
   `Inspect.Opts`.
 
-  Inspection is done using the functions available in
-  `Inspect.Algebra` and by calling `Kernel.inspect/2` recursively
-  passing the `Inspect.Opts` as an argument. When `Kernel.inspect/2`
-  receives an `Inspect.Opts` record as the second argument, it returns
-  the underlying algebra document instead of the formatted string.
+  Inspection is done using the functions available in `Inspect.Algebra`.
+
+  ## Examples
 
   Many times, inspecting a structure can be implemented in function
   of existing entities. For example, here is `HashSet`'s `inspect`
@@ -27,7 +31,7 @@ defprotocol Inspect do
         import Inspect.Algebra
 
         def inspect(dict, opts) do
-          concat ["#HashSet<", Kernel.inspect(HashSet.to_list(dict), opts), ">"]
+          concat ["#HashSet<", to_doc(HashSet.to_list(dict), opts), ">"]
         end
       end
 
@@ -35,14 +39,24 @@ defprotocol Inspect do
   concatenates algebra documents together. In the example above,
   it is concatenating the string `"HashSet<"` (all strings are
   valid algebra documents that keep their formatting when pretty
-  printed), the document returned by `Kernel.inspect/2` and the
+  printed), the document returned by `Inspect.Algebra.to_doc/2` and the
   other string `">"`.
 
   Since regular strings are valid entities in an algebra document,
   an implementation of inspect may simply return a string,
   although that will devoid it of any pretty-printing.
-  """
 
+  ## Error handling
+
+  In case there is an error while your structure is being inspected,
+  Elixir will automatically fall back to tuple inspection for records.
+  You can however access the underlying error by invoking the Inspect
+  implementation directly. For example, to test Inspect.HashSet above,
+  you just need to do:
+
+      Inspect.HashSet.inspect(HashSet.new, Inspect.Opts.new)
+
+  """
   def inspect(thing, opts)
 end
 
@@ -86,7 +100,7 @@ defimpl Inspect, for: Atom do
       valid_ref_identifier?(binary) ->
         "Elixir." <> rest = binary
         rest
-      atom in [:{}, :[], :<<>>] ->
+      atom in [:{}, :<<>>] ->
         ":" <> binary
       atom in Macro.binary_ops or atom in Macro.unary_ops ->
         ":" <> binary
@@ -150,8 +164,8 @@ defimpl Inspect, for: BitString do
       "<<0, 1, 2>>"
 
   """
-  def inspect(thing, opts) when is_binary(thing) do
-    if String.printable?(thing) do
+  def inspect(thing, Inspect.Opts[binaries: bins] = opts) when is_binary(thing) do
+    if bins == :as_strings or (bins == :infer and String.printable?(thing)) do
       << ?", escape(thing, ?") :: binary, ?" >>
     else
       inspect_bitstring(thing, opts)
@@ -205,11 +219,29 @@ defimpl Inspect, for: BitString do
   defp escape(<<?\v, t :: binary>>, char, binary) do
     escape(t, char, << binary :: binary, ?\\, ?v >>)
   end
+  defp escape(<<h :: utf8, t :: binary>>, char, binary) do
+    head = << h :: utf8 >>
+    if String.printable?(head) do
+      escape(t, char, append(head, binary))
+    else
+      << byte :: size(8), h :: binary >> = head
+      t = << h :: binary, t :: binary >>
+      escape(t, char, << binary :: binary, octify(byte) :: binary >>)
+    end
+  end
   defp escape(<<h, t :: binary>>, char, binary) do
-    escape(t, char, << binary :: binary, h >>)
+    escape(t, char, << binary :: binary, octify(h) :: binary >>)
   end
   defp escape(<<>>, _char, binary), do: binary
 
+  defp octify(byte) do
+    << hi :: size(2), mi :: size(3), lo :: size(3) >> = << byte >>
+    << ?\\, ?0 + hi, ?0 + mi, ?0 + lo >>
+  end
+
+  defp append(<<h, t :: binary>>, binary), do: append(t, << binary :: binary, h >>)
+  defp append(<<>>, binary), do: binary
+
   ## Bitstrings
 
   defp inspect_bitstring(bitstring, Inspect.Opts[] = opts) do
@@ -262,21 +294,21 @@ defimpl Inspect, for: List do
 
   def inspect([], _opts), do: "[]"
 
-  def inspect(thing, Inspect.Opts[] = opts) do
+  def inspect(thing, Inspect.Opts[char_lists: lists] = opts) do
     cond do
-      :io_lib.printable_list(thing) ->
+      lists == :as_char_lists or (lists == :infer and :io_lib.printable_list(thing)) ->
         << ?', Inspect.BitString.escape(String.from_char_list!(thing), ?') :: binary, ?' >>
-      keyword?(thing) && not opts.raw ->
-        surround_many("[", thing, "]", opts.limit, keyword(&1, opts))
+      keyword?(thing) ->
+        surround_many("[", thing, "]", opts.limit, &keyword(&1, opts))
       true ->
-        surround_many("[", thing, "]", opts.limit, Kernel.inspect(&1, opts))
+        surround_many("[", thing, "]", opts.limit, &to_doc(&1, opts))
     end
   end
 
   defp keyword({key, value}, opts) do
     concat(
       key_to_binary(key) <> ": ",
-      Kernel.inspect(value, opts)
+      to_doc(value, opts)
     )
   end
 
@@ -314,24 +346,24 @@ defimpl Inspect, for: Tuple do
 
   def inspect({}, _opts), do: "{}"
 
-  def inspect(tuple, opts) do
-    unless opts.raw do
+  def inspect(tuple, Inspect.Opts[] = opts) do
+    if opts.records do
       record_inspect(tuple, opts)
-    end || surround_many("{", tuple_to_list(tuple), "}", opts.limit, Kernel.inspect(&1, opts))
+    else
+      surround_many("{", tuple_to_list(tuple), "}", opts.limit, &to_doc(&1, opts))
+    end
   end
 
   ## Helpers
 
-  defp record_inspect(record, opts) do
+  defp record_inspect(record, Inspect.Opts[] = opts) do
     [name|tail] = tuple_to_list(record)
 
     if is_atom(name) && (fields = record_fields(name)) && (length(fields) == size(record) - 1) do
-      if Enum.first(tail) == :__exception__ do
-        surround_record(name, tl(fields), tl(tail), opts)
-      else
-        surround_record(name, fields, tail, opts)
-      end
-    end || surround_many("{", [name|tail], "}", opts.limit, Kernel.inspect(&1, opts))
+      surround_record(name, fields, tail, opts)
+    else
+      surround_many("{", [name|tail], "}", opts.limit, &to_doc(&1, opts))
+    end
   end
 
   defp record_fields(name) do
@@ -346,27 +378,46 @@ defimpl Inspect, for: Tuple do
     end
   end
 
-  defp surround_record(name, fields, tail, opts) do
-    fields = lc { field, _ } inlist fields, do: field
-
+  defp surround_record(name, fields, tail, Inspect.Opts[] = opts) do
     concat(
       Inspect.Atom.inspect(name, opts),
-      surround_many("[", Enum.zip(fields, tail), "]", opts.limit, keyword(&1, opts))
+      surround_many("[", zip_fields(fields, tail), "]", opts.limit, &keyword(&1, opts))
     )
   end
 
+  defp zip_fields([{ key, _ }|tk], [value|tv]) do
+    case atom_to_binary(key) do
+      "_" <> _ -> zip_fields(tk, tv)
+      key -> [{ key, value }|zip_fields(tk, tv)]
+    end
+  end
+
+  defp zip_fields([], []) do
+    []
+  end
+
   defp keyword({ k, v }, opts) do
-    concat(
-      atom_to_binary(k, :utf8) <> ": ",
-      Kernel.inspect(v, opts)
-    )
+    concat(k <> ": ", to_doc(v, opts))
   end
 end
 
-defimpl Inspect, for: Number do
+defimpl Inspect, for: Integer do
   @doc """
-  Represents the number as a string.
+  Represents the integer as a string.
 
+  ## Examples
+
+      iex> inspect(1)
+      "1"
+
+  """
+  def inspect(thing, _opts) do
+    integer_to_binary(thing)
+  end
+end
+
+defimpl Inspect, for: Float do
+  @doc """
   Floats are represented using the shortened, correctly rounded string
   that converts to float when read back with `binary_to_float/1`. This
   is done via the Erlang implementation of _Printing Floating-Point
@@ -375,14 +426,10 @@ defimpl Inspect, for: Number do
 
   ## Examples
 
-      iex> inspect(1)
-      "1"
+      iex> inspect(1.0)
+      "1.0"
 
   """
-  def inspect(thing, _opts) when is_integer(thing) do
-    integer_to_binary(thing)
-  end
-
   def inspect(thing, _opts) do
     iolist_to_binary(:io_lib_format.fwrite_g(thing))
   end
@@ -399,11 +446,11 @@ defimpl Inspect, for: Regex do
 
   """
   def inspect(regex, opts) when size(regex) == 5 do
-    concat ["%r", Kernel.inspect(Regex.source(regex), opts), Regex.opts(regex)]
+    concat ["%r", to_doc(Regex.source(regex), opts), Regex.opts(regex)]
   end
 
-  def inspect(other, opts) do
-    Kernel.inspect(other, opts.raw(true))
+  def inspect(other, Inspect.Opts[] = opts) do
+    to_doc(other, opts.records(false))
   end
 end
 
@@ -446,7 +493,7 @@ end
 
 defimpl Inspect, for: PID do
   def inspect(pid, _opts) do
-    "#PID" <> iolist_to_binary(pid_to_list(pid))
+    "#PID" <> iolist_to_binary(:erlang.pid_to_list(pid))
   end
 end
 

lib/elixir/lib/inspect/algebra.ex

@@ -71,9 +71,6 @@ defmodule Inspect.Algebra do
   @nesting 1
   @break " "
 
-  defp repeat(_, 0), do: ""
-  defp repeat(s, i), do: :lists.duplicate(i, s)
-
   # Functional interface to `doc` records
 
   @type t :: :doc_nil | doc_cons_t | doc_nest_t | doc_break_t | doc_group_t | binary
@@ -82,6 +79,48 @@ defmodule Inspect.Algebra do
   defrecordp :doc_break, str: " " :: binary
   defrecordp :doc_group, doc: :doc_nil :: t
 
+  defmacrop is_doc(doc) do
+    if __CALLER__.in_guard? do
+      do_is_doc(doc)
+    else
+      var = quote do: doc
+      quote do
+        unquote(var) = unquote(doc)
+        unquote(do_is_doc(var))
+      end
+    end
+  end
+
+  defp do_is_doc(doc) do
+    quote do
+      is_binary(unquote(doc)) or
+      unquote(doc) == :doc_nil or
+      (is_tuple(unquote(doc)) and
+       elem(unquote(doc), 0) in [:doc_cons, :doc_nest, :doc_break, :doc_group])
+    end
+  end
+
+  @doc """
+  Converts an Elixir structure to an algebra document
+  according to the inspect protocol.
+  """
+  @spec to_doc(any, Inspect.Opts.t) :: t
+  def to_doc(arg, opts) when is_record(opts, Inspect.Opts) do
+    if is_tuple(arg) do
+      if elem(opts, Inspect.Opts.__record__(:index, :records)) do
+        try do
+          Inspect.inspect(arg, opts)
+        catch
+          _, _ -> Inspect.Tuple.inspect(arg, opts)
+        end
+      else
+        Inspect.Tuple.inspect(arg, opts)
+      end
+    else
+      Inspect.inspect(arg, opts)
+    end
+  end
+
   @doc """
   Returns `:doc_nil` which is a document entity used to represent
   nothingness. Takes no arguments.
@@ -107,13 +146,17 @@ defmodule Inspect.Algebra do
 
   """
   @spec concat(t, t) :: doc_cons_t
-  def concat(x, y), do: doc_cons(left: x, right: y)
+  def concat(x, y) when is_doc(x) and is_doc(y) do
+    doc_cons(left: x, right: y)
+  end
 
   @doc """
   Concatenates a list of documents.
   """
   @spec concat([t]) :: doc_cons_t
-  def concat(docs), do: folddoc(docs, concat(&1, &2))
+  def concat(docs) do
+    folddoc(docs, &concat(&1, &2))
+  end
 
   @doc """
   Nests document entity `x` positions deep. Nesting will be
@@ -127,8 +170,13 @@ defmodule Inspect.Algebra do
 
   """
   @spec nest(t, non_neg_integer) :: doc_nest_t
-  def nest(x, 0),                    do: x
-  def nest(x, i) when is_integer(i), do: doc_nest(indent: i, doc: x)
+  def nest(x, 0) when is_doc(x) do
+    x
+  end
+
+  def nest(x, i) when is_doc(x) and is_integer(i) do
+    doc_nest(indent: i, doc: x)
+  end
 
   @doc %S"""
   Document entity representing a break. This break can
@@ -198,7 +246,9 @@ defmodule Inspect.Algebra do
 
   """
   @spec group(t) :: doc_group_t
-  def group(d), do: doc_group(doc: d)
+  def group(d) when is_doc(d) do
+    doc_group(doc: d)
+  end
 
   @doc """
   Inserts a mandatory single space between two document entities.
@@ -240,7 +290,7 @@ defmodule Inspect.Algebra do
       "A!B"
 
   """
-  @spec folddoc([any], ((any, [t]) -> t)) :: t
+  @spec folddoc([t], ((t, t) -> t)) :: t
   def folddoc([], _), do: empty
   def folddoc([doc], _), do: doc
   def folddoc([d|ds], f), do: f.(d, folddoc(ds, f))
@@ -263,7 +313,7 @@ defmodule Inspect.Algebra do
   """
   @spec surround(binary, t, binary) :: t
   def surround(left, doc, right) do
-    group concat [left, nest(doc, @nesting), right]
+    group concat left, concat(nest(doc, @nesting), right)
   end
 
   @doc %S"""
@@ -273,40 +323,46 @@ defmodule Inspect.Algebra do
 
   ## Examples
 
-      iex> doc = Inspect.Algebra.surround_many("[", Enum.to_list(1..5), "]", :infinity, integer_to_binary(&1))
+      iex> doc = Inspect.Algebra.surround_many("[", Enum.to_list(1..5), "]", :infinity, &integer_to_binary(&1))
       iex> Inspect.Algebra.pretty(doc, 5)
       "[1,\n 2,\n 3,\n 4,\n 5]"
 
-      iex> doc = Inspect.Algebra.surround_many("[", Enum.to_list(1..5), "]", 3, integer_to_binary(&1))
+      iex> doc = Inspect.Algebra.surround_many("[", Enum.to_list(1..5), "]", 3, &integer_to_binary(&1))
       iex> Inspect.Algebra.pretty(doc, 20)
       "[1, 2, 3, ...]"
 
+      iex> doc = Inspect.Algebra.surround_many("[", Enum.to_list(1..5), "]", 3, &integer_to_binary(&1), "!")
+      iex> Inspect.Algebra.pretty(doc, 20)
+      "[1! 2! 3! ...]"
+
   """
-  @spec surround_many(binary, [any], binary, integer | :infinity, (term -> t)) :: t
-  def surround_many(left, [], right, _, _fun) do
+  @spec surround_many(binary, [any], binary, integer | :infinity, (term -> t), binary) :: t
+  def surround_many(left, docs, right, limit, fun, separator // @surround_separator)
+
+  def surround_many(left, [], right, _, _fun, _) do
     concat(left, right)
   end
 
-  def surround_many(left, docs, right, limit, fun) do
-    surround(left, surround_many(docs, limit, fun), right)
+  def surround_many(left, docs, right, limit, fun, sep) do
+    surround(left, surround_many(docs, limit, fun, sep), right)
   end
 
-  defp surround_many(_, 0, _fun) do
+  defp surround_many(_, 0, _fun, _sep) do
     "..."
   end
 
-  defp surround_many([h], _limit, fun) do
+  defp surround_many([h], _limit, fun, _sep) do
     fun.(h)
   end
 
-  defp surround_many([h|t], limit, fun) when is_list(t) do
+  defp surround_many([h|t], limit, fun, sep) when is_list(t) do
     glue(
-      concat(fun.(h), @surround_separator),
-      surround_many(t, decrement(limit), fun)
+      concat(fun.(h), sep),
+      surround_many(t, decrement(limit), fun, sep)
     )
   end
 
-  defp surround_many([h|t], _limit, fun) do
+  defp surround_many([h|t], _limit, fun, _sep) do
     glue(
       concat(fun.(h), @tail_separator),
       fun.(t)
@@ -334,57 +390,44 @@ defmodule Inspect.Algebra do
 
   # Rendering and internal helpers
 
-  # Records representing __simple__ documents, already on a fixed layout
-  # Those are generalized by `sdoc` type.
-  @type sdoc :: :s_nil | s_text_t | s_line_t
-  defrecordp :s_text, str: "" :: binary, sdoc: :s_nil :: sdoc
-  defrecordp :s_line, indent: 1 :: non_neg_integer, sdoc: :s_nil :: sdoc
-
   # Record representing the document mode to be rendered: flat or broken
   @typep mode :: :flat | :break
 
   @doc false
   @spec fits?(integer, [{ integer, mode, t }]) :: boolean
-  def fits?(:infinity, _),                                  do: true # no pretty printing
   def fits?(w, _) when w < 0,                               do: false
   def fits?(_, []),                                         do: true
   def fits?(w, [{_, _, :doc_nil} | t]),                     do: fits?(w, t)
   def fits?(w, [{i, m, doc_cons(left: x, right: y)} | t]),  do: fits?(w, [{i, m, x} | [{i, m, y} | t]])
   def fits?(w, [{i, m, doc_nest(indent: j, doc: x)} | t]),  do: fits?(w, [{i + j, m, x} | t])
+  def fits?(w, [{i, _, doc_group(doc: x)} | t]),            do: fits?(w, [{i, :flat, x} | t])
   def fits?(w, [{_, _, s} | t]) when is_binary(s),          do: fits?((w - byte_size s), t)
   def fits?(w, [{_, :flat, doc_break(str: s)} | t]),        do: fits?((w - byte_size s), t)
   def fits?(_, [{_, :break, doc_break(str: _)} | _]),       do: true
-  def fits?(w, [{i, _, doc_group(doc: x)} | t]),            do: fits?(w, [{i, :flat, x} | t])
 
   @doc false
-  @spec format(integer, integer, [{ integer, mode, t }]) :: atom | tuple
-  def format(_, _, []),                                        do: :s_nil
+  @spec format(integer | :infinity, integer, [{ integer, mode, t }]) :: [binary]
+  def format(_, _, []),                                        do: []
   def format(w, k, [{_, _, :doc_nil} | t]),                    do: format(w, k, t)
   def format(w, k, [{i, m, doc_cons(left: x, right: y)} | t]), do: format(w, k, [{i, m, x} | [{i, m, y} | t]])
   def format(w, k, [{i, m, doc_nest(indent: j, doc: x)} | t]), do: format(w, k, [{i + j, m, x} | t])
-  def format(w, k, [{_, _, s} | t]) when is_binary(s),         do: s_text(str: s, sdoc: format(w, (k + byte_size s), t))
   def format(w, k, [{i, m, doc_group(doc: x)} | t]),           do: format(w, k, [{i, m, x} | t])
-  def format(w, k, [{_, :flat, doc_break(str: s)} | t]),       do: s_text(str: s, sdoc: format(w, (k + byte_size s), t))
+  def format(w, k, [{_, _, s} | t]) when is_binary(s),         do: [s | format(w, (k + byte_size s), t)]
+  def format(w, k, [{_, :flat, doc_break(str: s)} | t]),       do: [s | format(w, (k + byte_size s), t)]
   def format(w, k, [{i, :break, doc_break(str: s)} | t]) do
     k = k + byte_size(s)
-    if fits?(w - k, t) do
-      s_text(str: s, sdoc: format(w, k, t))
+
+    if w == :infinity or fits?(w - k, t) do
+      [s | format(w, k, t)]
     else
-      s_line(indent: i, sdoc: format(w, i, t))
+      prefix = :binary.copy(" ", i)
+      [@newline, prefix | format(w, i, t)]
     end
   end
 
   @doc false
-  @spec render(sdoc) :: binary
+  @spec render([binary]) :: binary
   def render(sdoc) do
-    iolist_to_binary do_render sdoc
-  end
-
-  @spec do_render(sdoc) :: [binary]
-  defp do_render(:s_nil), do: [""]
-  defp do_render(s_text(str: s, sdoc: d)), do: [s | do_render(d)]
-  defp do_render(s_line(indent: i, sdoc: d)) do
-    prefix = repeat " ", i
-    [@newline | [prefix | do_render d]]
+    iolist_to_binary sdoc
   end
 end

lib/elixir/lib/integer.ex

@@ -3,17 +3,68 @@ defmodule Integer do
   Functions for working with integers.
   """
 
+  import Bitwise
+
   @doc """
-  Returns true if `n` is an odd number, otherwise false.
+  Determines if an integer is odd.
+
+  Returns `true` if `n` is an odd number, otherwise `false`.
+  Implemented as a macro so it is allowed in guard clauses.
   """
   defmacro odd?(n) do
-    quote do: rem(unquote(n), 2) != 0
+    quote do: (unquote(n) &&& 1) == 1
   end
 
   @doc """
-  Returns true if `n` is an even number, otherwise false.
+  Determines if an integer is even.
+
+  Returns `true` if `n` is an even number, otherwise `false`.
+  Implemented as a macro so it is allowed in guard clauses.
   """
   defmacro even?(n) do
-    quote do: rem(unquote(n), 2) == 0
+    quote do: (unquote(n) &&& 1) == 0
+  end
+
+  @doc """
+  Converts a binary to an integer. 
+  
+  If successful, returns a tuple of the form `{ integer, remainder_of_binary }`.
+  Otherwise `:error`.
+
+  ## Examples
+
+      iex> Integer.parse("34")
+      {34,""}
+      iex> Integer.parse("34.5")
+      {34,".5"}
+      iex> Integer.parse("three")
+      :error
+
+  """
+  @spec parse(binary) :: { integer, binary } | :error
+  def parse(<< ?-, bin :: binary >>) do
+    case do_parse(bin) do
+      :error -> :error
+      { number, remainder } -> { -number, remainder }
+    end
+  end
+
+  def parse(<< ?+, bin :: binary >>) do
+    do_parse(bin)
+  end
+
+  def parse(bin) when is_binary(bin) do
+    do_parse(bin)
+  end
+
+  defp do_parse(<< char, bin :: binary >>) when char in ?0..?9, do: do_parse(bin, char - ?0)
+  defp do_parse(_), do: :error
+
+  defp do_parse(<< char, rest :: binary >>, acc) when char in ?0..?9 do
+    do_parse rest, 10 * acc + (char - ?0)
+  end
+
+  defp do_parse(bitstring, acc) do
+    { acc, bitstring }
   end
 end

lib/elixir/lib/io.ex

@@ -1,28 +1,33 @@
+defexception IO.StreamError, [:reason, :message] do
+  def exception(opts) do
+    reason    = opts[:reason]
+    formatted = iolist_to_binary(:file.format_error(reason))
+    IO.StreamError[message: "error during streaming: #{formatted}", reason: reason]
+  end
+end
+
 defmodule IO do
   @moduledoc """
-  Module responsible for doing IO. Many functions in this
-  module expects an IO device and an io data encoded in UTF-8.
+  Functions handling IO.
 
+  Many functions in this module expects an IO device as argument.
   An IO device must be a pid or an atom representing a process.
   For convenience, Elixir provides `:stdio` and `:stderr` as
   shortcuts to Erlang's `:standard_io` and `:standard_error`.
 
-  An io data can be:
+  The majority of the functions expect data encoded in UTF-8
+  and will do a conversion to string, via the `String.Chars`
+  protocol (as shown in typespecs).
 
-  * A list of integers representing a string. Any unicode
-    character must be represented with one entry in the list,
-    this entry being an integer with the codepoint value;
-
-  * A binary in which unicode characters are represented
-    with many bytes (Elixir's default representation);
-
-  * A list of binaries or a list of char lists (as described above);
-
-  * If none of the above, `to_string` is invoked on the
-    given argument;
+  The functions starting with `bin*` expects iodata as arguments,
+  i.e. iolists or binaries with no particular encoding.
 
   """
 
+  @type device   :: atom | pid
+  @type chardata :: char_list | String.Chars.t
+  @type nodata :: { :error, term } | :eof
+
   import :erlang, only: [group_leader: 0]
 
   defmacrop is_iolist(data) do
@@ -43,6 +48,7 @@ defmodule IO do
     for instance `{:error, :estale}` if reading from an
     NFS file system.
   """
+  @spec read(device, :line | non_neg_integer) :: chardata | nodata
   def read(device // group_leader, chars_or_line)
 
   def read(device, :line) do
@@ -65,6 +71,7 @@ defmodule IO do
     for instance `{:error, :estale}` if reading from an
     NFS file system.
   """
+  @spec binread(device, :line | non_neg_integer) :: iodata | nodata
   def binread(device // group_leader, chars_or_line)
 
   def binread(device, :line) do
@@ -98,8 +105,9 @@ defmodule IO do
       #=> "error"
 
   """
-  def write(device // group_leader(), item) when is_iolist(item) do
-    :io.put_chars map_dev(device), item
+  @spec write(device, chardata) :: :ok
+  def write(device // group_leader(), item) do
+    :io.put_chars map_dev(device), to_chardata(item)
   end
 
   @doc """
@@ -108,6 +116,7 @@ defmodule IO do
 
   Check `write/2` for more information.
   """
+  @spec binwrite(device, iodata) :: :ok | { :error, term }
   def binwrite(device // group_leader(), item) when is_iolist(item) do
     :file.write map_dev(device), item
   end
@@ -117,9 +126,10 @@ defmodule IO do
   but adds a newline at the end. The argument is expected
   to be a chardata.
   """
-  def puts(device // group_leader(), item) when is_iolist(item) do
+  @spec puts(device, chardata) :: :ok
+  def puts(device // group_leader(), item) do
     erl_dev = map_dev(device)
-    :io.put_chars erl_dev, [item, ?\n]
+    :io.put_chars erl_dev, [to_chardata(item), ?\n]
   end
 
   @doc """
@@ -135,6 +145,7 @@ defmodule IO do
       IO.inspect Process.list
 
   """
+  @spec inspect(term, Keyword.t) :: term
   def inspect(item, opts // []) do
     inspect group_leader(), item, opts
   end
@@ -142,6 +153,7 @@ defmodule IO do
   @doc """
   Inspects the item with options using the given device.
   """
+  @spec inspect(device, term, Keyword.t) :: term
   def inspect(device, item, opts) when is_list(opts) do
     opts = Keyword.put_new(opts, :pretty, true)
 
@@ -160,7 +172,7 @@ defmodule IO do
   Gets a number of bytes from the io device. If the
   io device is a unicode device, `count` implies
   the number of unicode codepoints to be retrieved.
-  Otherwise, `count` is the number of raw bytes to be retrieved. 
+  Otherwise, `count` is the number of raw bytes to be retrieved.
   It returns:
 
   * `data` - The input characters.
@@ -171,6 +183,8 @@ defmodule IO do
     for instance `{:error, :estale}` if reading from an
     NFS file system.
   """
+  @spec getn(chardata, pos_integer) :: chardata | nodata
+  @spec getn(device, chardata) :: chardata | nodata
   def getn(prompt, count // 1)
 
   def getn(prompt, count) when is_integer(count) do
@@ -187,8 +201,9 @@ defmodule IO do
   the number of unicode codepoints to be retrieved.
   Otherwise, `count` is the number of raw bytes to be retrieved.
   """
+  @spec getn(device, chardata, pos_integer) :: chardata | nodata
   def getn(device, prompt, count) do
-    :io.get_chars(map_dev(device), prompt, count)
+    :io.get_chars(map_dev(device), to_chardata(prompt), count)
   end
 
   @doc """
@@ -203,73 +218,69 @@ defmodule IO do
     for instance `{:error, :estale}` if reading from an
     NFS file system.
   """
+  @spec gets(device, chardata) :: chardata | nodata
   def gets(device // group_leader(), prompt) do
-    :io.get_line(map_dev(device), prompt)
+    :io.get_line(map_dev(device), to_chardata(prompt))
   end
 
   @doc """
   Converts the io device into a Stream. The device is
-  iterated line by line if :line is given or by a given
+  iterated line by line if `:line` is given or by a given
   number of codepoints.
 
-  This reads the io as utf-8. Check out
+  This reads the IO as utf-8. Check out
   `IO.binstream/2` to handle the IO as a raw binary.
 
+  Note that an IO stream has side effects and every time
+  you go over the stream you may get different results.
+
   ## Examples
 
   Here is an example on how we mimic an echo server
   from the command line:
 
-      Enum.each IO.stream(:stdio, :line), IO.write(&1)
+      Enum.each IO.stream(:stdio, :line), &IO.write(&1)
 
   """
-  def stream(device, line_or_bytes) do
-    stream(map_dev(device), line_or_bytes, &1, &2)
-  end
-
-  @doc false
-  def stream(device) do
-    IO.write "IO.stream(device) is deprecated, please use IO.stream(device, :line) instead\n#{Exception.format_stacktrace}"
-    stream(device, :line)
+  @spec stream(device, :line | pos_integer) :: Enumerable.t
+  def stream(device, line_or_codepoints) do
+    Stream.unfold(map_dev(device), &do_stream(&1, line_or_codepoints))
   end
 
   @doc """
   Converts the io device into a Stream. The device is
-  iterated line by line.
+  iterated line by line or by a number of bytes. This
+  reads the IO as a raw binary.
 
-  This reads the io as a raw binary.
+  Note that an IO stream has side effects and every time
+  you go over the stream you may get different results.
   """
+  @spec binstream(device, :line | pos_integer) :: Enumerable.t
   def binstream(device, line_or_bytes) do
-    binstream(map_dev(device), line_or_bytes, &1, &2)
+    Stream.unfold(map_dev(device), &do_binstream(&1, line_or_bytes))
   end
 
   @doc false
-  def binstream(device) do
-    IO.write "IO.binstream(device) is deprecated, please use IO.binstream(device, :line) instead\n#{Exception.format_stacktrace}"
-    binstream(device, :line)
-  end
-
-  @doc false
-  def stream(device, what, acc, fun) do
+  def do_stream(device, what) do
     case read(device, what) do
       :eof ->
-        acc
+        nil
       { :error, reason } ->
-        raise File.IteratorError, reason: reason
+        raise IO.StreamError, reason: reason
       data ->
-        stream(device, what, fun.(data, acc), fun)
+        { data, device }
     end
   end
 
   @doc false
-  def binstream(device, what, acc, fun) do
+  def do_binstream(device, what) do
     case binread(device, what) do
       :eof ->
-        acc
+        nil
       { :error, reason } ->
-        raise File.IteratorError, reason: reason
+        raise IO.StreamError, reason: reason
       data ->
-        binstream(device, what, fun.(data, acc), fun)
+        { data, device }
     end
   end
 
@@ -277,4 +288,7 @@ defmodule IO do
   defp map_dev(:stdio),  do: :standard_io
   defp map_dev(:stderr), do: :standard_error
   defp map_dev(other),   do: other
+
+  defp to_chardata(list) when is_list(list), do: list
+  defp to_chardata(other), do: to_string(other)
 end

lib/elixir/lib/io/ansi.ex

@@ -16,9 +16,9 @@ end
 
 defmodule IO.ANSI do
   @moduledoc """
-  This module provides functionality to render ANSI escape sequences
+  Functionality to render ANSI escape sequences
   (http://en.wikipedia.org/wiki/ANSI_escape_code) —  characters embedded
-  in the text used to control formatting, color, and other output options
+  in text used to control formatting, color, and other output options
   on video text terminals.
   """
 
@@ -28,7 +28,7 @@ defmodule IO.ANSI do
   Checks whether the default I/O device is a terminal or a file.
 
   Used to identify whether printing ANSI escape sequences will likely
-  be printed as intended.
+  be displayed as intended.
   """
   @spec terminal? :: boolean
   @spec terminal?(:io.device) :: boolean
@@ -144,18 +144,19 @@ defmodule IO.ANSI do
   The format for referring to sequences is `%{red}` and `%{red,bright}` (for
   multiple sequences).
 
-  It will also append a %{reset} to the string. If you don't want this
-  behaviour, use `escape_fragment/1` and `escape_fragment/2`.
+  It will also append a `%{reset}` to the string. If you don't want this
+  behaviour, use `escape_fragment/2`.
 
   An optional boolean parameter can be passed to enable or disable
-  emitting actual ANSI codes. When false, no ANSI codes will emitted.
+  emitting actual ANSI codes. When `false`, no ANSI codes will emitted.
   By default, standard output will be checked if it is a terminal capable
-  of handling these sequences (using `terminal?/0` function)
+  of handling these sequences (using `terminal?/1` function)
 
-  ## Example
+  ## Examples
+
+      iex> IO.ANSI.escape("Hello %{red,bright,green}yes", true)
+      "Hello \e[31m\e[1m\e[32myes\e[0m"
 
-    iex> IO.ANSI.escape("Hello %{red,bright,green}yes", true)
-    "Hello \e[31m\e[1m\e[32myes\e[0m"
   """
   @spec escape(String.t, emit :: boolean) :: String.t
   def escape(string, emit // terminal?) do
@@ -174,16 +175,16 @@ defmodule IO.ANSI do
   multiple sequences).
 
   An optional boolean parameter can be passed to enable or disable
-  emitting actual ANSI codes. When false, no ANSI codes will emitted.
+  emitting actual ANSI codes. When `false`, no ANSI codes will emitted.
   By default, standard output will be checked if it is a terminal capable
-  of handling these sequences (using `terminal?/0` function)
+  of handling these sequences (using `terminal?/1` function)
 
-  ## Example
+  ## Examples
 
-    iex> IO.ANSI.escape_fragment("Hello %{red,bright,green}yes", true)
-    "Hello \e[31m\e[1m\e[32myes"
-    iex> IO.ANSI.escape_fragment("%{reset}bye", true)
-    "\e[0mbye"
+      iex> IO.ANSI.escape_fragment("Hello %{red,bright,green}yes", true)
+      "Hello \e[31m\e[1m\e[32myes"
+      iex> IO.ANSI.escape_fragment("%{reset}bye", true)
+      "\e[0mbye"
 
   """
   @spec escape_fragment(String.t, emit :: boolean) :: String.t

lib/elixir/lib/io/ansi/docs.ex

@@ -0,0 +1,377 @@
+defmodule IO.ANSI.Docs do
+  @moduledoc false
+
+  @bullets [?*, ?-, ?+]
+
+  @default_colors [ enabled: true,
+                    doc_code: "cyan,bright",
+                    doc_inline_code: "cyan",
+                    doc_headings: "yellow,bright",
+                    doc_title: "reverse,yellow,bright",
+                    doc_bold: "bright",
+                    doc_underline: "underline" ]
+
+  @doc """
+  Prints the head of the documentation (i.e. the function signature).
+
+  See `print/3` for docs on the supported options.
+  """
+  def print_heading(heading, colors // @default_colors) do
+    IO.puts IO.ANSI.reset
+    width   = column_width()
+    padding = div(width + String.length(heading), 2)
+    heading = heading |> String.rjust(padding) |> String.ljust(width)
+    write(:doc_title, heading, colors)
+  end
+
+  @doc """
+  Prints the documentation body.
+
+  In addition to the priting string, takes a truth value for whether to use ANSI
+  escape codes, and a keyword list for the printing color settings. Supported
+  keys for the color settings are:
+
+  * `:enabled`         - toggles coloring on and off (true)
+  * `:doc_code`        - code blocks (cyan, bright)
+  * `:doc_inline_code` - inline code (cyan)
+  * `:doc_headings`    - h1 and h2 headings (yellow, bright)
+  * `:doc_title`       - top level heading (reverse, yellow, bright)
+  * `:doc_bold`        - bold text (bright)
+  * `:doc_underline`   - underlined text (underline)
+
+  Values for the color settings are strings with comma-separated attributes.
+  Supported attributes are:
+
+  * Colors:     `black  red  green  yellow  blue  magenta  cyan  white`
+  * Intensity:  `normal  bright`
+  * Decoration: `underline  reverse`
+  """
+  def print(doc, colors // @default_colors) do
+    doc
+    |> String.split(["\r\n","\n"], trim: false)
+    |> Enum.map(&String.rstrip/1)
+    |> process("", colors)
+  end
+
+  defp process([], _indent, _colors), do: nil
+
+  defp process(["# " <> heading | rest], _indent, colors) do
+    write_h1(String.strip(heading), colors)
+    process(rest, "", colors)
+  end
+
+  defp process(["## " <> heading | rest], _indent, colors) do
+    write_h2(String.strip(heading), colors)
+    process(rest, "", colors)
+  end
+
+  defp process(["### " <> heading | rest], indent, colors) do
+    write_h3(String.strip(heading), indent, colors)
+    process(rest, indent, colors)
+  end
+
+  defp process(["" | rest], indent, colors) do
+    process(rest, indent, colors)
+  end
+
+  defp process(["    " <> line | rest], indent, colors) do
+    process_code(rest, [line], indent, colors)
+  end
+
+  defp process([line | rest], indent, colors) do
+    { stripped, count } = strip_spaces(line, 0)
+    case stripped do
+      <<bullet, ?\s, item :: binary >> when bullet in @bullets ->
+        process_list(item, rest, count, indent, colors)
+      _ ->
+        process_text(rest, [line], indent, false, colors)
+    end
+  end
+
+  defp strip_spaces(" " <> line, acc) do
+    strip_spaces(line, acc + 1)
+  end
+
+  defp strip_spaces(rest, acc) do
+    { rest, acc }
+  end
+
+  ## Headings
+
+  defp write_h1(heading, colors) do
+    write_h2(String.upcase(heading), colors)
+  end
+
+  defp write_h2(heading, colors) do
+    write(:doc_headings, heading, colors)
+  end
+
+  defp write_h3(heading, indent, colors) do
+    IO.write(indent)
+    write(:doc_headings, heading, colors)
+  end
+
+  ## Lists
+
+  defp process_list(line, rest, count, indent, colors) do
+    IO.write indent <> "• "
+    { contents, rest, done } = process_list_next(rest, count, false, [])
+    process_text(contents, [line], indent <> "  ", true, colors)
+    if done, do: IO.puts(IO.ANSI.reset)
+    process(rest, indent, colors)
+  end
+
+  # Process the thing after a list item entry. It can be either:
+  #
+  # * Continuation of the list
+  # * A nested list
+  # * The end of the list
+  #
+  defp process_list_next([" " <> _ = line | rest], count, _done, acc) do
+    case list_next(line, count) do
+      :done    -> { Enum.reverse(acc), [line|rest], false }
+      chopped  -> process_list_next(rest, count, false, [chopped|acc])
+    end
+  end
+
+  defp process_list_next([<<bullet, ?\s, _ :: binary>> | _] = rest, _count, _done, acc) when bullet in @bullets do
+    { Enum.reverse(acc), rest, false }
+  end
+
+  defp process_list_next(["" | rest], count, _done, acc) do
+    process_list_next(rest, count, true, [""|acc])
+  end
+
+  defp process_list_next(rest, _count, done, acc) do
+    { Enum.reverse(acc), rest, done }
+  end
+
+  defp list_next(<<bullet, ?\s, _ :: binary>>, 0) when bullet in @bullets, do: :done
+  defp list_next(line, 0),          do: chop(line, 2)
+  defp list_next(" " <> line, acc), do: list_next(line, acc - 1)
+  defp list_next(line, _acc),       do: line
+
+  defp chop(" " <> line, acc) when acc > 0, do: chop(line, acc - 1)
+  defp chop(line, _acc), do: line
+
+  ## Text (paragraphs / lists)
+
+  defp process_text(doc=["" | _], para, indent, from_list, colors) do
+    write_text(Enum.reverse(para), indent, from_list, colors)
+    process(doc, indent, colors)
+  end
+
+  defp process_text([], para, indent, from_list, colors) do
+    write_text(Enum.reverse(para), indent, from_list, colors)
+  end
+
+  defp process_text([line | rest], para, indent, true, colors) do
+    { stripped, count } = strip_spaces(line, 0)
+    case stripped do
+      <<bullet, ?\s, item :: binary>> when bullet in @bullets ->
+        write_text(Enum.reverse(para), indent, true, colors)
+        process_list(item, rest, count, indent, colors)
+      _ ->
+        process_text(rest, [line | para], indent, true, colors)
+    end
+  end
+
+  defp process_text([line | rest], para, indent, from_list, colors) do
+    process_text(rest, [line | para], indent, from_list, colors)
+  end
+
+  defp write_text(lines, indent, from_list, colors) do
+    lines
+    |> Enum.join(" ")
+    |> handle_links
+    |> handle_inline(nil, [], [], colors)
+    |> String.split(%r{\s})
+    |> write_with_wrap(column_width() - size(indent), indent, from_list)
+
+    unless from_list, do: IO.puts(IO.ANSI.reset)
+  end
+
+  ## Code blocks
+
+  defp process_code([], code, indent, colors) do
+    write_code(code, indent, colors)
+  end
+
+  # Blank line between code blocks
+  defp process_code([ "", "    " <> line | rest ], code, indent, colors) do
+    process_code(rest, [line, "" | code], indent, colors)
+  end
+
+  defp process_code([ "    " <> line | rest ], code, indent, colors) do
+    process_code(rest, [line|code], indent, colors)
+  end
+
+  defp process_code(rest, code, indent, colors) do
+    write_code(code, indent, colors)
+    process(rest, indent, colors)
+  end
+
+  defp write_code(code, indent, colors) do
+    write(:doc_code, "#{indent}┃ #{Enum.join(Enum.reverse(code), "\n#{indent}┃ ")}", colors)
+  end
+
+  ## Helpers
+
+  defp write(style, string, colors) do
+    IO.puts color(style, colors) <> string <> IO.ANSI.reset
+    IO.puts IO.ANSI.reset
+  end
+
+  defp write_with_wrap([], _available, _indent, _first) do
+    :ok
+  end
+
+  defp write_with_wrap(words, available, indent, first) do
+    { words, rest } = take_words(words, available, [])
+    IO.puts (if first, do: "", else: indent) <> Enum.join(words, " ")
+    write_with_wrap(rest, available, indent, false)
+  end
+
+  defp take_words([word|words], available, acc) do
+    available = available - length_without_escape(word, 0)
+
+    cond do
+      # It fits, take one for space and continue decreasing
+      available > 0 ->
+        take_words(words, available - 1, [word|acc])
+
+      # No space but we got no words
+      acc == [] ->
+        { [word], words }
+
+      # Otherwise
+      true ->
+        { Enum.reverse(acc), [word|words] }
+    end
+  end
+
+  defp take_words([], _available, acc) do
+    { Enum.reverse(acc), [] }
+  end
+
+  defp length_without_escape(<< ?\e, ?[, _, _, ?m, rest :: binary >>, count) do
+    length_without_escape(rest, count)
+  end
+
+  defp length_without_escape(<< ?\e, ?[, _, ?m, rest :: binary >>, count) do
+    length_without_escape(rest, count)
+  end
+
+  defp length_without_escape(rest, count) do
+    case String.next_grapheme(rest) do
+      { _, rest }  -> length_without_escape(rest, count + 1)
+      nil -> count
+    end
+  end
+
+  defp handle_links(text) do
+    text
+    |> remove_square_brackets_in_link
+    |> escape_underlines_in_link
+  end
+
+  defp escape_underlines_in_link(text) do
+    case Regex.match?(%r{.*(https?\S*)}, text) do
+      true ->
+        Regex.replace(%r{_}, text, "\\\\_")
+      _ ->
+        text
+    end
+  end
+
+  defp remove_square_brackets_in_link(text) do
+    Regex.replace(%r{\[(.*?)\]\((.*?)\)}, text, "\\1 (\\2)")
+  end
+
+  # Single inline quotes.
+  @single [?`, ?_, ?*]
+
+  # ` does not require space in between
+  @spaced [?_, ?*]
+
+  # Clauses for handling spaces
+  defp handle_inline(<<?*, ?*, ?\s, rest :: binary>>, nil, buffer, acc, colors) do
+    handle_inline(rest, nil, [?\s, ?*, ?*|buffer], acc, colors)
+  end
+
+  defp handle_inline(<<mark, ?\s, rest :: binary>>, nil, buffer, acc, colors) when mark in @spaced do
+    handle_inline(rest, nil, [?\s, mark|buffer], acc, colors)
+  end
+
+  defp handle_inline(<<?\s, ?*, ?*, rest :: binary>>, limit, buffer, acc, colors) do
+    handle_inline(rest, limit, [?*, ?*, ?\s|buffer], acc, colors)
+  end
+
+  defp handle_inline(<<?\s, mark, rest :: binary>>, limit, buffer, acc, colors) when mark in @spaced do
+    handle_inline(rest, limit, [mark, ?\s|buffer], acc, colors)
+  end
+
+  # Clauses for handling escape
+  defp handle_inline(<<?\\, ?\\, rest :: binary>>, limit, buffer, acc, colors) do
+    handle_inline(rest, limit, [?\\|buffer], acc, colors)
+  end
+
+  defp handle_inline(<<?\\, ?*, ?*, rest :: binary>>, limit, buffer, acc, colors) do
+    handle_inline(rest, limit, [?*, ?*|buffer], acc, colors)
+  end
+
+  # A escape is not valid inside `
+  defp handle_inline(<<?\\, mark, rest :: binary>>, limit, buffer, acc, colors)
+      when mark in [?_, ?*, ?`] and not(mark == limit and mark == ?`) do
+    handle_inline(rest, limit, [mark|buffer], acc, colors)
+  end
+
+  # Inline start
+  defp handle_inline(<<?*, ?*, rest :: binary>>, nil, buffer, acc, colors) when rest != "" do
+    handle_inline(rest, ?d, ["**"], [Enum.reverse(buffer)|acc], colors)
+  end
+
+  defp handle_inline(<<mark, rest :: binary>>, nil, buffer, acc, colors) when rest != "" and mark in @single do
+    handle_inline(rest, mark, [<<mark>>], [Enum.reverse(buffer)|acc], colors)
+  end
+
+  # Inline end
+  defp handle_inline(<<?*, ?*, rest :: binary>>, ?d, buffer, acc, colors) do
+    handle_inline(rest, nil, [], [inline_buffer(buffer, colors)|acc], colors)
+  end
+
+  defp handle_inline(<<mark, rest :: binary>>, mark, buffer, acc, colors) when mark in @single do
+    handle_inline(rest, nil, [], [inline_buffer(buffer, colors)|acc], colors)
+  end
+
+  defp handle_inline(<<char, rest :: binary>>, mark, buffer, acc, colors) do
+    handle_inline(rest, mark, [char|buffer], acc, colors)
+  end
+
+  defp handle_inline(<<>>, _mark, buffer, acc, _colors) do
+    iolist_to_binary Enum.reverse([Enum.reverse(buffer)|acc])
+  end
+
+  defp inline_buffer(buffer, colors) do
+    [h|t] = Enum.reverse([IO.ANSI.reset|buffer])
+    [color_for(h, colors)|t]
+  end
+
+  defp color_for("`", colors),  do: color(:doc_inline_code, colors)
+  defp color_for("_", colors),  do: color(:doc_underline, colors)
+  defp color_for("*", colors),  do: color(:doc_bold, colors)
+  defp color_for("**", colors), do: color(:doc_bold, colors)
+
+  defp color(style, colors) do
+    color = colors[style]
+    enabled = colors[:enabled]
+    IO.ANSI.escape_fragment("%{#{color}}", enabled)
+  end
+
+  defp column_width() do
+    case :io.columns do
+      { :ok, width } -> min(width, 80)
+      _              -> 80
+    end
+  end
+end

lib/elixir/lib/kernel/cli.ex

@@ -15,12 +15,12 @@ defmodule Kernel.CLI do
     System.argv(argv)
 
     run fn ->
-      command_results = Enum.map(Enum.reverse(config.commands), process_command(&1, config))
+      command_results = Enum.map(Enum.reverse(config.commands), &process_command(&1, config))
       command_errors  = lc { :error, msg } inlist command_results, do: msg
       errors          = Enum.reverse(config.errors) ++ command_errors
 
       if errors != [] do
-        Enum.each(errors, IO.puts(:stderr, &1))
+        Enum.each(errors, &IO.puts(:stderr, &1))
         System.halt(1)
       end
     end, config.halt
@@ -77,7 +77,7 @@ defmodule Kernel.CLI do
   defp shared_option?(list, config, callback) do
     case process_shared(list, config) do
       { [h|hs], _ } when h == hd(list) ->
-        new_config = config.update_errors ["#{h} : Unknown option" | &1]
+        new_config = config.update_errors &["#{h} : Unknown option" | &1]
         callback.(hs, new_config)
       { new_list, new_config } ->
         callback.(new_list, new_config)
@@ -104,6 +104,10 @@ defmodule Kernel.CLI do
     prune_stacktrace(t)
   end
 
+  defp prune_stacktrace([{ __MODULE__, :wrapper, 1, _ }|_]) do
+    []
+  end
+
   defp prune_stacktrace([h|t]) do
     [h|prune_stacktrace(t)]
   end
@@ -120,17 +124,17 @@ defmodule Kernel.CLI do
   end
 
   defp process_shared(["-pa", h|t], config) do
-    Enum.each Path.wildcard(Path.expand(h)), Code.prepend_path(&1)
+    Enum.each Path.wildcard(Path.expand(h)), &Code.prepend_path(&1)
     process_shared t, config
   end
 
   defp process_shared(["-pz", h|t], config) do
-    Enum.each Path.wildcard(Path.expand(h)), Code.append_path(&1)
+    Enum.each Path.wildcard(Path.expand(h)), &Code.append_path(&1)
     process_shared t, config
   end
 
   defp process_shared(["--app", h|t], config) do
-    process_shared t, config.update_commands [{:app, h}|&1]
+    process_shared t, config.update_commands &[{:app, h}|&1]
   end
 
   defp process_shared(["--no-halt"|t], config) do
@@ -138,15 +142,15 @@ defmodule Kernel.CLI do
   end
 
   defp process_shared(["-e", h|t], config) do
-    process_shared t, config.update_commands [{:eval, h}|&1]
+    process_shared t, config.update_commands &[{:eval, h}|&1]
   end
 
   defp process_shared(["-r", h|t], config) do
-    process_shared t, config.update_commands [{:require, h}|&1]
+    process_shared t, config.update_commands &[{:require, h}|&1]
   end
 
   defp process_shared(["-pr", h|t], config) do
-    process_shared t, config.update_commands [{:parallel_require, h}|&1]
+    process_shared t, config.update_commands &[{:parallel_require, h}|&1]
   end
 
   defp process_shared([erl, _|t], config) when erl in ["--erl", "--sname", "--name", "--cookie"] do
@@ -176,15 +180,15 @@ defmodule Kernel.CLI do
   end
 
   defp process_argv(["-S", h|t], config) do
-    { config.update_commands([{:script, h}|&1]), t }
+    { config.update_commands(&[{:script, h}|&1]), t }
   end
 
   defp process_argv([h|t] = list, config) do
     case h do
       "-" <> _ ->
-        shared_option? list, config, process_argv(&1, &2)
+        shared_option? list, config, &process_argv(&1, &2)
       _ ->
-        { config.update_commands([{:file, h}|&1]), t }
+        { config.update_commands(&[{:file, h}|&1]), t }
     end
   end
 
@@ -203,21 +207,21 @@ defmodule Kernel.CLI do
   end
 
   defp process_compiler(["--no-docs"|t], config) do
-    process_compiler t, config.update_compiler_options([{:docs, false}|&1])
+    process_compiler t, config.update_compiler_options(&[{:docs, false}|&1])
   end
 
   defp process_compiler(["--no-debug-info"|t], config) do
-    process_compiler t, config.update_compiler_options([{:debug_info, false}|&1])
+    process_compiler t, config.update_compiler_options(&[{:debug_info, false}|&1])
   end
 
   defp process_compiler(["--ignore-module-conflict"|t], config) do
-    process_compiler t, config.update_compiler_options([{:ignore_module_conflict, true}|&1])
+    process_compiler t, config.update_compiler_options(&[{:ignore_module_conflict, true}|&1])
   end
 
   defp process_compiler(["--warnings-as-errors"|t], config) do
-    process_compiler t, config.update_compiler_options([{:warnings_as_errors, true}|&1])
+    process_compiler t, config.update_compiler_options(&[{:warnings_as_errors, true}|&1])
   end
-  
+
   defp process_compiler(["--verbose"|t], config) do
     process_compiler t, config.verbose_compile(true)
   end
@@ -225,15 +229,15 @@ defmodule Kernel.CLI do
   defp process_compiler([h|t] = list, config) do
     case h do
       "-" <> _ ->
-        shared_option? list, config, process_compiler(&1, &2)
+        shared_option? list, config, &process_compiler(&1, &2)
       _ ->
         pattern = if :filelib.is_dir(h), do: "#{h}/**/*.ex", else: h
-        process_compiler t, config.update_compile [pattern|&1]
+        process_compiler t, config.update_compile &[pattern|&1]
     end
   end
 
   defp process_compiler([], config) do
-    { config.update_commands([{:compile, config.compile}|&1]), [] }
+    { config.update_commands(&[{:compile, config.compile}|&1]), [] }
   end
 
   # Process iex options
@@ -253,15 +257,15 @@ defmodule Kernel.CLI do
   end
 
   defp process_iex(["-S", h|t], config) do
-    { config.update_commands([{:script, h}|&1]), t }
+    { config.update_commands(&[{:script, h}|&1]), t }
   end
 
   defp process_iex([h|t] = list, config) do
     case h do
       "-" <> _ ->
-        shared_option? list, config, process_iex(&1, &2)
+        shared_option? list, config, &process_iex(&1, &2)
       _ ->
-        { config.update_commands([{:file, h}|&1]), t }
+        { config.update_commands(&[{:file, h}|&1]), t }
     end
   end
 
@@ -273,16 +277,14 @@ defmodule Kernel.CLI do
 
   defp process_command({:cookie, h}, _config) do
     if Node.alive? do
-      Node.set_cookie(binary_to_atom(h))
-      :ok
+      wrapper fn -> Node.set_cookie(binary_to_atom(h)) end
     else
       { :error, "--cookie : Cannot set cookie if the node is not alive (set --name or --sname)" }
     end
   end
 
   defp process_command({:eval, expr}, _config) when is_binary(expr) do
-    Code.eval_string(expr, [])
-    :ok
+    wrapper fn -> Code.eval_string(expr, []) end
   end
 
   defp process_command({:app, app}, _config) when is_binary(app) do
@@ -296,8 +298,7 @@ defmodule Kernel.CLI do
 
   defp process_command({:script, file}, _config) when is_binary(file) do
     if exec = find_elixir_executable(file) do
-      Code.require_file(exec)
-      :ok
+      wrapper fn -> Code.require_file(exec) end
     else
       { :error, "-S : Could not find executable #{file}" }
     end
@@ -305,8 +306,7 @@ defmodule Kernel.CLI do
 
   defp process_command({:file, file}, _config) when is_binary(file) do
     if :filelib.is_regular(file) do
-      Code.require_file(file)
-      :ok
+      wrapper fn -> Code.require_file(file) end
     else
       { :error, "No file named #{file}" }
     end
@@ -315,11 +315,10 @@ defmodule Kernel.CLI do
   defp process_command({:require, pattern}, _config) when is_binary(pattern) do
     files = Path.wildcard(pattern)
     files = Enum.uniq(files)
-    files = Enum.filter files, :filelib.is_regular(&1)
+    files = Enum.filter files, &:filelib.is_regular(&1)
 
     if files != [] do
-      Enum.map files, Code.require_file(&1)
-      :ok
+      wrapper fn -> Enum.map files, &Code.require_file(&1) end
     else
       { :error, "-r : No files matched pattern #{pattern}" }
     end
@@ -328,11 +327,10 @@ defmodule Kernel.CLI do
   defp process_command({:parallel_require, pattern}, _config) when is_binary(pattern) do
     files = Path.wildcard(pattern)
     files = Enum.uniq(files)
-    files = Enum.filter files, :filelib.is_regular(&1)
+    files = Enum.filter files, &:filelib.is_regular(&1)
 
     if files != [] do
-      Kernel.ParallelRequire.files(files)
-      :ok
+      wrapper fn -> Kernel.ParallelRequire.files(files) end
     else
       { :error, "-pr : No files matched pattern #{pattern}" }
     end
@@ -341,20 +339,50 @@ defmodule Kernel.CLI do
   defp process_command({:compile, patterns}, config) do
     :filelib.ensure_dir(:filename.join(config.output, "."))
 
-    files = Enum.map patterns, Path.wildcard(&1)
-    files = Enum.uniq(Enum.concat(files))
-    files = Enum.filter files, :filelib.is_regular(&1)
-
-    if files != [] do
-      Code.compiler_options(config.compiler_options)
-      Kernel.ParallelCompiler.files_to_path(files, config.output,
-        each_file: fn file -> if config.verbose_compile do IO.puts "Compiled #{file}" end end)
-      :ok
-    else
-      { :error, "--compile : No files matched patterns #{Enum.join(patterns, ",")}" }
+    case match_regular_files(patterns) do
+      { :ok, [] } ->
+        { :error, "No files matched provided patterns" }
+      { :ok, files } ->
+        wrapper fn ->
+          Code.compiler_options(config.compiler_options)
+          Kernel.ParallelCompiler.files_to_path(files, config.output,
+            each_file: fn file -> if config.verbose_compile do IO.puts "Compiled #{file}" end end)
+        end
+      { :missing, missing } ->
+        { :error, "No files matched pattern(s) #{Enum.join(missing, ",")}" }
     end
   end
 
+  defp match_regular_files(patterns) do
+    matched_files = Enum.map patterns, fn(pattern) ->
+      case Path.wildcard(pattern) do
+        []    -> { :missing, pattern }
+        files -> { :ok, files }
+      end
+    end
+
+    files = Enum.filter_map matched_files,
+       fn(match) -> elem(match, 0) == :ok end,
+       &elem(&1, 1)
+
+    missing_patterns = Enum.filter_map matched_files,
+       fn(match) -> elem(match, 0) == :missing end,
+       &elem(&1, 1)
+
+    if missing_patterns == [] do
+      files = Enum.uniq(Enum.concat(files))
+      files = Enum.filter files, &:filelib.is_regular(&1)
+      { :ok, files }
+    else
+      { :missing,  Enum.uniq(missing_patterns) }
+    end
+  end
+
+  defp wrapper(fun) do
+    fun.()
+    :ok
+  end
+
   defp find_elixir_executable(file) do
     if exec = System.find_executable(file) do
       # If we are on Windows, the executable is going to be

lib/elixir/lib/kernel/error_handler.ex

@@ -18,9 +18,19 @@ defmodule Kernel.ErrorHandler do
       { :module, _ } -> []
       { :error, _ } ->
         parent = :erlang.get(:elixir_compiler_pid)
-        parent <- { :waiting, self(), module }
+        ref    = :erlang.make_ref
+        send parent, { :waiting, self(), ref, module }
+        :erlang.garbage_collect(self)
         receive do
-          { :release, ^parent } -> :ok
+          { ^ref, :ready } ->
+            :ok
+          { ^ref, :release } ->
+            # On release, get rid of the compiler pid,
+            # no further allow elixir_ensure_compiled directives
+            # and revert to the original error handler.
+            :erlang.erase(:elixir_compiler_pid)
+            :erlang.erase(:elixir_ensure_compiled)
+            :erlang.process_flag(:error_handler, :error_handler)
         end
     end
   end

lib/elixir/lib/kernel/lexical_tracker.ex

@@ -0,0 +1,175 @@
+# This is a module Elixir responsible for tracking
+# the usage of aliases, imports and requires in the Elixir scope.
+#
+# The implementation simply stores dispatch information in an
+# ETS table and then consults this table once compilation is done.
+#
+# Note that since this is required for bootstrap, we can't use
+# any of the `GenServer.Behaviour` conveniences.
+defmodule Kernel.LexicalTracker do
+  @timeout  30_000
+  @behaviour :gen_server
+
+  @import 2
+  @alias  3
+
+  @doc """
+  Returns all remotes linked to in this lexical scope.
+  """
+  def remotes(arg) do
+    # If the module is compiled from a function, its lexical
+    # scope may be long gone, so it has no associated PID.
+    if pid = to_pid(arg) do
+      ets = :gen_server.call(pid, :ets, @timeout)
+      :ets.match(ets, { :"$1", :_, :_ }) |> List.flatten
+    else
+      []
+    end
+  end
+
+  defp to_pid(pid) when is_pid(pid),  do: pid
+  defp to_pid(mod) when is_atom(mod) do
+    table = :elixir_module.data_table(mod)
+    [{ _, val }] = :ets.lookup(table, :__lexical_tracker)
+    val
+  end
+
+  # Internal API
+
+  # Starts the tracker and returns its pid.
+  @doc false
+  def start_link do
+    { :ok, pid } = :gen_server.start_link(__MODULE__, [], [])
+    pid
+  end
+
+  @doc false
+  def stop(pid) do
+    :gen_server.cast(pid, :stop)
+  end
+
+  @doc false
+  def add_import(pid, module, line, warn) do
+    :gen_server.cast(pid, { :add_import, module, line, warn })
+  end
+
+  @doc false
+  def add_alias(pid, module, line, warn) do
+    :gen_server.cast(pid, { :add_alias, module, line, warn })
+  end
+
+  @doc false
+  def remote_dispatch(pid, module) do
+    :gen_server.cast(pid, { :remote_dispatch, module })
+  end
+
+  @doc false
+  def import_dispatch(pid, module) do
+    :gen_server.cast(pid, { :import_dispatch, module })
+  end
+
+  @doc false
+  def alias_dispatch(pid, module) do
+    :gen_server.cast(pid, { :alias_dispatch, module })
+  end
+
+  @doc false
+  def collect_unused_imports(pid) do
+    unused(pid, @import)
+  end
+
+  @doc false
+  def collect_unused_aliases(pid) do
+    unused(pid, @alias)
+  end
+
+  defp unused(pid, pos) do
+    ets = :gen_server.call(pid, :ets, @timeout)
+    :ets.foldl(fn
+      { module, _, _ } = tuple, acc when is_integer(:erlang.element(pos, tuple)) ->
+        [{ module, :erlang.element(pos, tuple) }|acc]
+      _, acc ->
+        acc
+    end, [], ets) |> Enum.sort
+  end
+
+  # Callbacks
+
+
+  def init([]) do
+    { :ok, :ets.new(:lexical, [:protected]) }
+  end
+
+  def handle_call(:ets, _from, d) do
+    { :reply, d, d }
+  end
+
+  def handle_call(request, _from, d) do
+    { :stop, { :bad_call, request }, d }
+  end
+
+  def handle_cast({ :remote_dispatch, module }, d) do
+    add_module(d, module)
+    { :noreply, d }
+  end
+
+  def handle_cast({ :import_dispatch, module }, d) do
+    add_dispatch(d, module, @import)
+    { :noreply, d }
+  end
+
+  def handle_cast({ :alias_dispatch, module }, d) do
+    add_dispatch(d, module, @alias)
+    { :noreply, d }
+  end
+
+  def handle_cast({ :add_import, module, line, warn }, d) do
+    add_directive(d, module, line, warn, @import)
+    { :noreply, d }
+  end
+
+  def handle_cast({ :add_alias, module, line, warn }, d) do
+    add_directive(d, module, line, warn, @alias)
+    { :noreply, d }
+  end
+
+  def handle_cast(:stop, d) do
+    { :stop, :normal, d }
+  end
+
+  def handle_cast(msg, d) do
+    { :stop, { :bad_cast, msg }, d }
+  end
+
+  def handle_info(_msg, d) do
+    { :noreply, d }
+  end
+
+  def terminate(_reason, _d) do
+    :ok
+  end
+
+  def code_change(_old, d, _extra) do
+    { :ok, d }
+  end
+
+  # Callbacks helpers
+
+  # In the table we keep imports and aliases.
+  # If the value is false, it was not imported/aliased
+  # If the value is true, it was imported/aliased
+  # If the value is a line, it was imported/aliased and has a pending warning
+  defp add_module(d, module) do
+    :ets.insert_new(d, { module, false, false })
+  end
+
+  defp add_dispatch(d, module, pos) do
+    :ets.update_element(d, module, { pos, true })
+  end
+
+  defp add_directive(d, module, line, warn, pos) do
+    add_module(d, module)
+    marker = if warn, do: line, else: true
+    :ets.update_element(d, module, { pos, marker })
+  end
+end

lib/elixir/lib/kernel/parallel_compiler.ex

@@ -20,13 +20,8 @@ defmodule Kernel.ParallelCompiler do
   * `:each_file` - for each file compiled, invokes the callback passing the file
   * `:each_module` - for each module compiled, invokes the callback
                      passing the file, module and the module bytecode
-  * `:each_waiting` - every time a module waits for another module to be compiled,
-                      this callback is invoked with the module we are waiting
-
-  The compiler doesn't care about the return values of the callbacks except
-  by `each_waiting`, which must return nil or a file as a hint where the source
-  could be found.
 
+  The compiler doesn't care about the return values of the callbacks.
   Returns the modules generated by each compiled file.
   """
   def files(files, callbacks // [])
@@ -47,14 +42,15 @@ defmodule Kernel.ParallelCompiler do
 
   defp spawn_compilers(files, path, callbacks) do
     Code.ensure_loaded(Kernel.ErrorHandler)
-    :elixir_code_server.cast(:reset_warnings)
+    compiler_pid = self()
+    :elixir_code_server.cast({ :reset_warnings, compiler_pid })
     schedulers = max(:erlang.system_info(:schedulers_online), 2)
 
     result = spawn_compilers(files, files, path, callbacks, [], [], schedulers, [])
 
     # In case --warning-as-errors is enabled and there was a warning,
-    # compilation status will be set to error and we fail with Kernel.CompilationError
-    case :elixir_code_server.call(:compilation_status) do
+    # compilation status will be set to error and we fail with CompileError
+    case :elixir_code_server.call({ :compilation_status, compiler_pid }) do
       :ok    -> result
       :error -> raise CompileError, [], []
     end
@@ -66,29 +62,42 @@ defmodule Kernel.ParallelCompiler do
     wait_for_messages(files, original, output, callbacks, waiting, queued, schedulers, result)
   end
 
+  # Release waiting processes
+  defp spawn_compilers([h|t], original, output, callbacks, waiting, queued, schedulers, result) when is_pid(h) do
+    { ^h, ref, _ } = List.keyfind(waiting, h, 0)
+    send h, { ref, :ready }
+    waiting = List.keydelete(waiting, h, 0)
+    spawn_compilers(t, original, output, callbacks, waiting, queued, schedulers, result)
+  end
+
   # Spawn a compiler for each file in the list until we reach the limit
   defp spawn_compilers([h|t], original, output, callbacks, waiting, queued, schedulers, result) do
     parent = self()
 
-    child  = spawn_link fn ->
-      :erlang.put(:elixir_compiler_pid, parent)
-      :erlang.put(:elixir_ensure_compiled, true)
-      :erlang.process_flag(:error_handler, Kernel.ErrorHandler)
+    { pid, ref } =
+      :erlang.spawn_monitor fn ->
+        # Notify Code.ensure_compiled/2 that we should
+        # attempt to compile the module by doing a dispatch.
+        :erlang.put(:elixir_ensure_compiled, true)
 
-      try do
-        if output do
-          :elixir_compiler.file_to_path(h, output)
-        else
-          :elixir_compiler.file(h)
-        end
-        parent <- { :compiled, self(), h }
-      catch
-        kind, reason ->
-          parent <- { :failure, self(), kind, reason, System.stacktrace }
+        # Set the elixir_compiler_pid used by our custom Kernel.ErrorHandler.
+        :erlang.put(:elixir_compiler_pid, parent)
+        :erlang.process_flag(:error_handler, Kernel.ErrorHandler)
+
+        exit(try do
+          if output do
+            :elixir_compiler.file_to_path(h, output)
+          else
+            :elixir_compiler.file(h)
+          end
+          { :compiled, h }
+        catch
+          kind, reason ->
+            { :failure, kind, reason, System.stacktrace }
+        end)
       end
-    end
 
-    spawn_compilers(t, original, output, callbacks, waiting, [{child, h}|queued], schedulers, result)
+    spawn_compilers(t, original, output, callbacks, waiting, [{pid, ref, h}|queued], schedulers, result)
   end
 
   # No more files, nothing waiting, queue is empty, we are done
@@ -96,7 +105,10 @@ defmodule Kernel.ParallelCompiler do
 
   # Queued x, waiting for x: POSSIBLE ERROR! Release processes so we get the failures
   defp spawn_compilers([], original, output, callbacks, waiting, queued, schedulers, result) when length(waiting) == length(queued) do
-    Enum.each queued, fn { child, _ } -> child <- { :release, self() } end
+    Enum.each queued, fn { child, _, _ } ->
+      { ^child, ref, _ } = List.keyfind(waiting, child, 0)
+      send child, { ref, :release }
+    end
     wait_for_messages([], original, output, callbacks, waiting, queued, schedulers, result)
   end
 
@@ -108,47 +120,61 @@ defmodule Kernel.ParallelCompiler do
   # Wait for messages from child processes
   defp wait_for_messages(files, original, output, callbacks, waiting, queued, schedulers, result) do
     receive do
-      { :compiled, child, file } ->
-        if callback = Keyword.get(callbacks, :each_file) do
-          callback.(file)
-        end
-        new_queued  = List.keydelete(queued, child, 0)
-
-        # Sometimes we may have spurious entries in the waiting
-        # list because someone invoked try/rescue UndefinedFunctionError
-        new_waiting = List.keydelete(waiting, child, 0)
-        spawn_compilers(files, original, output, callbacks, new_waiting, new_queued, schedulers, result)
-      { :module_available, child, file, module, binary } ->
+      { :module_available, child, ref, file, module, binary } ->
         if callback = Keyword.get(callbacks, :each_module) do
           callback.(file, module, binary)
         end
 
         # Release the module loader which is waiting for an ack
-        child <- { self, :ack }
+        send child, { ref, :ack }
 
-        new_waiting = release_waiting_processes(module, waiting)
-        new_result  = [module|result]
-        wait_for_messages(files, original, output, callbacks, new_waiting, queued, schedulers, new_result)
-      { :waiting, child, on } ->
-        # If one of the callbacks is each_waiting and we haven't seen
-        # the hinted file before, add it to the list to be processed.
-        if (callback = Keyword.get(callbacks, :each_waiting)) &&
-           (hint = callback.(on)) &&
-           not(hint in original) do
-          files    = [hint|files]
-          original = [hint|original]
+        available  = lc { pid, _, waiting_module } inlist waiting,
+                        waiting_module == module,
+                        do: pid
+
+        spawn_compilers(available ++ files, original, output, callbacks,
+                        waiting, queued, schedulers, [module|result])
+
+      { :waiting, child, ref, on } ->
+        # Oops, we already got this module. Do not put it on waiting.
+        if :lists.member(on, result) do
+          send child, { :release, ref }
+        else
+          waiting = [{ child, ref, on }|waiting]
+        end
+        spawn_compilers(files, original, output, callbacks, waiting, queued, schedulers, result)
+
+      { :DOWN, _down_ref, :process, down_pid, { :compiled, file } } ->
+        if callback = Keyword.get(callbacks, :each_file) do
+          callback.(file)
         end
 
-        new_waiting = :orddict.store(child, on, waiting)
-        spawn_compilers(files, original, output, callbacks, new_waiting, queued, schedulers, result)
-      { :failure, child, kind, reason, stacktrace } ->
+        # Sometimes we may have spurious entries in the waiting
+        # list because someone invoked try/rescue UndefinedFunctionError
+        new_waiting = List.keydelete(waiting, down_pid, 0)
+        new_queued  = List.keydelete(queued, down_pid, 0)
+        spawn_compilers(files, original, output, callbacks, new_waiting, new_queued, schedulers, result)
+
+      { :DOWN, down_ref, :process, _down_pid, { :failure, kind, reason, stacktrace } } ->
+        handle_failure(down_ref, kind, reason, stacktrace, files, waiting, queued)
+        wait_for_messages(files, original, output, callbacks, waiting, queued, schedulers, result)
+
+      { :DOWN, down_ref, :process, _down_pid, other } ->
+        handle_failure(down_ref, :exit, other, [], files, waiting, queued)
+        wait_for_messages(files, original, output, callbacks, waiting, queued, schedulers, result)
+    end
+  end
+
+  defp handle_failure(ref, kind, reason, stacktrace, files, waiting, queued) do
+    case List.keyfind(queued, ref, 1) do
+      { child, ^ref, file } ->
         if many_missing?(child, files, waiting, queued) do
           IO.puts "== Compilation failed =="
           IO.puts "Compilation failed on the following files:\n"
 
-          Enum.each Enum.reverse(queued), fn { pid, file } ->
+          Enum.each Enum.reverse(queued), fn { pid, _ref, file } ->
             case List.keyfind(waiting, pid, 0) do
-              { _, mod } -> IO.puts "* #{file} is missing module #{inspect mod}"
+              { ^pid, _, mod } -> IO.puts "* #{file} is missing module #{inspect mod}"
               _ -> :ok
             end
           end
@@ -156,29 +182,18 @@ defmodule Kernel.ParallelCompiler do
           IO.puts "\nThe first failure is shown below..."
         end
 
-        {^child, file} = List.keyfind(queued, child, 0)
         IO.puts "== Compilation error on file #{file} =="
         :erlang.raise(kind, reason, stacktrace)
+      _ ->
+        :ok
     end
   end
 
   defp many_missing?(child, files, waiting, queued) do
     waiting_length = length(waiting)
 
-    match?({ ^child, _ }, List.keyfind(waiting, child, 0)) and
+    match?({ ^child, _, _ }, List.keyfind(waiting, child, 0)) and
       waiting_length > 1 and files == [] and
       waiting_length == length(queued)
   end
-
-  # Release waiting processes that are waiting for the given module
-  defp release_waiting_processes(module, waiting) do
-    Enum.filter waiting, fn { child, waiting_module } ->
-      if waiting_module == module do
-        child <- { :release, self() }
-        false
-      else
-        true
-      end
-    end
-  end
 end

lib/elixir/lib/kernel/parallel_require.ex

@@ -35,7 +35,7 @@ defmodule Kernel.ParallelRequire do
     ensure_compiled = :erlang.get(:elixir_ensure_compiled)
     { :error_handler, handler } = :erlang.process_info(parent, :error_handler)
 
-    child = spawn_link fn ->
+    { pid, ref } = :erlang.spawn_monitor fn ->
       if compiler_pid != :undefined do
         :erlang.put(:elixir_compiler_pid, compiler_pid)
       end
@@ -46,25 +46,35 @@ defmodule Kernel.ParallelRequire do
 
       :erlang.process_flag(:error_handler, handler)
 
-      try do
+      exit(try do
         new = Code.require_file(h) || []
-        parent <- { :required, self, Enum.map(new, &elem(&1, 0)), h }
+        { :required, Enum.map(new, &elem(&1, 0)), h }
       catch
         kind, reason ->
-          parent <- { :failure, self, kind, reason, System.stacktrace }
-      end
+          { :failure, kind, reason, System.stacktrace }
+      end)
     end
 
-    spawn_requires(t, [child|waiting], callback, schedulers, result)
+    spawn_requires(t, [{ pid, ref }|waiting], callback, schedulers, result)
   end
 
   defp wait_for_messages(files, waiting, callback, schedulers, result) do
     receive do
-      { :required, child, mods, file } ->
-        callback.(file)
-        spawn_requires(files, List.delete(waiting, child), callback, schedulers, mods ++ result)
-      { :failure, _child, kind, reason, stacktrace } ->
-        :erlang.raise(kind, reason, stacktrace)
+      { :DOWN, ref, :process, pid, status } ->
+        tuple = { pid, ref }
+        if tuple in waiting do
+          case status do
+            { :required, mods, file } ->
+              callback.(file)
+              result  = mods ++ result
+              waiting = List.delete(waiting, tuple)
+            { :failure, kind, reason, stacktrace } ->
+              :erlang.raise(kind, reason, stacktrace)
+            other ->
+              :erlang.raise(:exit, other, [])
+          end
+        end
+        spawn_requires(files, waiting, callback, schedulers, result)
     end
   end
-end
+end

lib/elixir/lib/kernel/record_rewriter.ex

@@ -75,7 +75,7 @@ defmodule Kernel.RecordRewriter do
           if :lists.member({ function, length(args) + 1 }, optimizable) do
             case record_field_info(function) do
               { kind, field } ->
-                if index = Enum.find_index(fields, field == &1) do
+                if index = Enum.find_index(fields, &(field == &1)) do
                   optimize_record_accessor_call(line, res, kind, field, index, left, args)
                 end
               nil ->
@@ -111,7 +111,7 @@ defmodule Kernel.RecordRewriter do
 
   defp optimize_record_other_call(line, record, res, function, left, args) do
     call = { :call, line,
-      { :remote, line, { :atom, line, record }, { :atom, 0, function } },
+      { :remote, line, { :atom, 0, record }, { :atom, 0, function } },
       args ++ [left]
     }
     { call, res }
@@ -128,7 +128,7 @@ defmodule Kernel.RecordRewriter do
     { { :call, call_line, remote, [integer, tuple, value] }, dict, res }
   end
 
-  defp optimize_expr({ :call, call_line, { :remote, line, left, right }, args } = call, module, dict) do
+  defp optimize_expr({ :call, call_line, { :remote, line, left, right }, args }, module, dict) do
     { left, dict, res } = optimize_expr(left, module, dict)
     { right, dict, _ } = optimize_expr(right, module, dict)
     { args, dict } = optimize_args(args, module, dict)
@@ -352,7 +352,7 @@ defmodule Kernel.RecordRewriter do
   end
 
   defp extract_vars({ :tuple, _, args }, vars) do
-    [Enum.map(args, extract_vars(&1, []))|vars]
+    [Enum.map(args, &extract_vars(&1, []))|vars]
   end
 
   defp extract_vars(_, vars) do

lib/elixir/lib/kernel/typespec.ex

@@ -2,56 +2,95 @@ defmodule Kernel.Typespec do
   @moduledoc """
   Provides macros and functions for working with typespecs.
 
-  The attributes `@type`, `@opaque`, `@typep`, `@spec` and
-  `@callback` available in modules are handled by the equivalent
-  macros defined by this module.
+  Elixir comes with a notation for declaring types and specifications. Elixir is
+  dynamically typed, as such typespecs are never used by the compiler to
+  optimize or modify code. Still, using typespecs is useful as documentation and
+  tools such as [Dialyzer](http://www.erlang.org/doc/man/dialyzer.html) can
+  analyze the code with typespecs to find bugs.
 
-  ## Defining a type
+  The attributes `@type`, `@opaque`, `@typep`, `@spec` and `@callback` available
+  in modules are handled by the equivalent macros defined by this module. See
+  sub-sections "Defining a type" and "Defining a specification" below.
 
-      @type type_name :: type
-      @typep type_name :: type
-      @opaque type_name :: type
+  ## Types and their syntax
 
-  For more details, see documentation for `deftype`, `deftypep` and `defopaque`
-  below.
+  The type syntax provided by Elixir is fairly similar to the one in
+  [Erlang](http://www.erlang.org/doc/reference_manual/typespec.html).
 
-  ## Defining a specification
+  Most of the built-in types provided in Erlang (for example, `pid()`) are
+  expressed the same way: `pid()` or simply `pid`. Parametrized types are also
+  supported (`list(integer)`) and so are remote types (`Enum.t`).
 
-      @spec function_name(type, type) :: type
-      @callback function_name(type, type) :: type
+  Integers and atom literals are allowed as types (ex. `1`, `:atom` or
+  `false`). All other types are built of unions of predefined types. Certain
+  shorthands are allowed, such as `[...]`, `<<>>` and `{...}`.
 
-  For more details, see documentation for `defspec` and `defcallback` below.
+  ### Predefined types
 
-  ## Types
+      Type :: any         # the top type, the set of all terms
+            | none        # the bottom type, contains no terms
+            | pid
+            | port
+            | reference
+            | Atom
+            | Bitstring
+            | float
+            | Fun
+            | Integer
+            | List
+            | Tuple
+            | Union
+            | UserDefined # Described in section "Defining a type"
 
-  The type syntax provided by Elixir is fairly similar to the one
-  in Erlang.
+      Atom :: atom
+            | ElixirAtom # `:foo`, `:bar`, ...
 
-  Most of the built-in types provided in Erlang (for example, `pid()`)
-  are expressed the same way: `pid()` or simply `pid`. Parametrized types
-  are also supported (`list(integer())`) and so are remote types (`Enum.t`).
+      Bitstring :: <<>>
+                 | << _ :: M >>             # M is a positive integer
+                 | << _ :: _ * N >>         # N is a positive integer
+                 | << _ :: M, _ :: _ * N >>
 
-  Certain data type shortcuts (`[...]`, `<<>>` and `{...}`) are supported as
-  well.
+      Fun :: (... -> any)    # any function
+           | (... -> Type)   # any arity, returning Type
+           | (() -> Type))
+           | (TList -> Type)
 
-  Main differences lie in how bit strings and functions are defined:
+      Integer :: integer
+               | ElixirInteger                # ..., -1, 0, 1, ... 42 ...
+               | ElixirInteger..ElixirInteger # an integer range
 
-  ### Bit Strings
+      List :: list(Type)                        # proper list ([]-terminated)
+            | improper_list(Type1, Type2)       # Type1=contents, Type2=termination
+            | maybe_improper_list(Type1, Type2) # Type1 and Type2 as above
+            | nonempty_list(Type)               # proper non-empty list
+            | []                                # empty list
+            | [Type]                            # shorthand for list(Type)
+            | [Type, ...]                       # shorthand for nonempty_list(Type)
+
+      Tuple :: tuple     # a tuple of any size
+             | {}        # empty tuple
+             | { TList }
+
+      TList :: Type
+             | Type, TList
+
+      Union :: Type1 | Type2
+
+  ### Bit strings
 
   Bit string with a base size of 3:
 
-      <<_ :: 3>>
+      << _ :: 3 >>
 
   Bit string with a unit size of 8:
 
-      <<_ :: _ * 8>>
+      << _ :: _ * 8 >>
 
   ### Anonymous functions
 
   Any anonymous function:
 
       ((...) -> any)
-      or
       (... -> any)
 
   Anonymous function with arity of zero:
@@ -61,27 +100,98 @@ defmodule Kernel.Typespec do
   Anonymous function with some arity:
 
       ((type, type) -> type)
-      or
       (type, type -> type)
 
+  ## Built-in types
+
+  Built-in type         | Defined as
+  :-------------------- | :---------
+  `term`                | `any`
+  `binary`              | `<< _ :: _ * 8 >>`
+  `bitstring`           | `<< _ :: _ * 1 >>`
+  `boolean`             | `false` &#124; `true`
+  `byte`                | `0..255`
+  `char`                | `0..0xffff`
+  `number`              | `integer` &#124; `float`
+  `list`                | `[any]`
+  `maybe_improper_list` | `maybe_improper_list(any, any)`
+  `nonempty_list`       | `nonempty_list(any)`
+  `iodata`              | `iolist` &#124; `binary`
+  `iolist`              | `maybe_improper_list(byte` &#124; `binary` &#124; `iolist, binary` &#124; `[])`
+  `module`              | `atom`
+  `mfa`                 | `{ atom, atom, arity }`
+  `arity`               | `0..255`
+  `node`                | `atom`
+  `timeout`             | `:infinity` &#124; `non_neg_integer`
+  `no_return`           | `none`
+  `fun`                 | `(... -> any)`
+
+  Some built-in types cannot be expressed with valid syntax according to the
+  language defined above.
+
+  Built-in type     | Can be interpreted as
+  :---------------- | :--------------------
+  `non_neg_integer` | `0..`
+  `pos_integer`     | `1..`
+  `neg_integer`     | `..-1`
+
+  Types defined in other modules are referred to as "remote types", they are
+  referenced as `Module.type_name` (ex. `Enum.t` or `String.t`).
+
+  ## Defining a type
+
+      @type type_name :: type
+      @typep type_name :: type
+      @opaque type_name :: type
+
+  A type defined with `@typep` is private. An opaque type, defined with
+  `@opaque` is a type where the internal structure of the type will not be
+  visible, but the type is still public.
+
+  Types can be parametrised by defining variables as parameters, these variables
+  can then be used to define the type.
+
+      @type dict(key, value) :: [{ key, value }]
+
+  Types can also be defined for records, see `defrecord/3`.
+
+  ## Defining a specification
+
+      @spec function_name(type1, type2) :: return_type
+      @callback function_name(type1, type2) :: return_type
+
+  Callbacks are used to define the callbacks functions of behaviours (see
+  `Behaviour`).
+
+  Guards can be used to restrict type variables given as arguments to the
+  function.
+
+      @spec function(arg) :: [arg] when arg: atom
+
+  Type variables with no restriction can also be defined.
+
+      @spec function(arg) :: [arg] when arg: var
+
+  Specifications can be overloaded just like ordinary functions.
+
+      @spec function(integer) :: atom
+      @spec function(atom)    :: integer
+
   ## Notes
 
-  Elixir discourages the use of type `string()` as it might be confused
-  with binaries which are referred to as "strings" in Elixir (as opposed to
-  character lists). In order to use the type that is called `string()` in Erlang,
-  one has to use the `char_list()` type which is a synonym for `string()`. If you
-  use `string()`, you'll get a warning from the compiler.
+  Elixir discourages the use of type `string` as it might be confused with
+  binaries which are referred to as "strings" in Elixir (as opposed to character
+  lists). In order to use the type that is called `string` in Erlang, one has to
+  use the `char_list` type which is a synonym for `string`. If you use `string`,
+  you'll get a warning from the compiler.
 
-  If you want to refer to the "string" type (the one operated by functions in the
-  String module), use `String.t()` type instead.
-
-  See http://www.erlang.org/doc/reference_manual/typespec.html
-  for more information.
+  If you want to refer to the "string" type (the one operated on by functions in
+  the `String` module), use `String.t` type instead.
   """
 
   @doc """
   Defines a type.
-  This macro is the one responsible for handling the attribute `@type`.
+  This macro is responsible for handling the attribute `@type`.
 
   ## Examples
 
@@ -90,13 +200,13 @@ defmodule Kernel.Typespec do
   """
   defmacro deftype(type) do
     quote do
-      Kernel.Typespec.deftype(:type, unquote(Macro.escape type), __ENV__)
+      Kernel.Typespec.deftype(:type, unquote(Macro.escape(type, unquote: true)), __ENV__)
     end
   end
 
   @doc """
   Defines an opaque type.
-  This macro is the one responsible for handling the attribute `@opaque`.
+  This macro is responsible for handling the attribute `@opaque`.
 
   ## Examples
 
@@ -105,13 +215,13 @@ defmodule Kernel.Typespec do
   """
   defmacro defopaque(type) do
     quote do
-      Kernel.Typespec.deftype(:opaque, unquote(Macro.escape type), __ENV__)
+      Kernel.Typespec.deftype(:opaque, unquote(Macro.escape(type, unquote: true)), __ENV__)
     end
   end
 
   @doc """
   Defines a private type.
-  This macro is the one responsible for handling the attribute `@typep`.
+  This macro is responsible for handling the attribute `@typep`.
 
   ## Examples
 
@@ -120,13 +230,13 @@ defmodule Kernel.Typespec do
   """
   defmacro deftypep(type) do
     quote do
-      Kernel.Typespec.deftype(:typep, unquote(Macro.escape type), __ENV__)
+      Kernel.Typespec.deftype(:typep, unquote(Macro.escape(type, unquote: true)), __ENV__)
     end
   end
 
   @doc """
   Defines a spec.
-  This macro is the one responsible for handling the attribute `@spec`.
+  This macro is responsible for handling the attribute `@spec`.
 
   ## Examples
 
@@ -135,13 +245,13 @@ defmodule Kernel.Typespec do
   """
   defmacro defspec(spec) do
     quote do
-      Kernel.Typespec.defspec(:spec, unquote(Macro.escape spec), __ENV__)
+      Kernel.Typespec.defspec(:spec, unquote(Macro.escape(spec, unquote: true)), __ENV__)
     end
   end
 
   @doc """
   Defines a callback.
-  This macro is the one responsible for handling the attribute `@callback`.
+  This macro is responsible for handling the attribute `@callback`.
 
   ## Examples
 
@@ -150,7 +260,7 @@ defmodule Kernel.Typespec do
   """
   defmacro defcallback(spec) do
     quote do
-      Kernel.Typespec.defspec(:callback, unquote(Macro.escape spec), __ENV__)
+      Kernel.Typespec.defspec(:callback, unquote(Macro.escape(spec, unquote: true)), __ENV__)
     end
   end
 
@@ -159,7 +269,7 @@ defmodule Kernel.Typespec do
   @doc """
   Defines a `type`, `typep` or `opaque` by receiving Erlang's typespec.
   """
-  def define_type(module, kind, { name, _, vars } = type) when kind in [:type, :typep, :opaque] do
+  def define_type(caller, kind, { name, _, vars } = type) when kind in [:type, :typep, :opaque] do
     { kind, export } =
       case kind do
         :type   -> { :type, true }
@@ -167,12 +277,35 @@ defmodule Kernel.Typespec do
         :opaque -> { :opaque, true }
       end
 
+    module = caller.module
+    arity  = length(vars)
+
     Module.compile_typespec module, kind, type
-    if export, do:
-      Module.compile_typespec(module, :export_type, [{ name, length(vars) }])
+
+    if export do
+      Module.compile_typespec(module, :export_type, [{ name, arity }])
+    end
+
+    define_doc(caller, kind, name, arity, export)
     type
   end
 
+  defp define_doc(caller, kind, name, arity, export) do
+    module = caller.module
+    doc    = Module.get_attribute(module, :typedoc)
+
+    if doc do
+      if export do
+        Module.add_doc(module, caller.line, kind, { name, arity }, doc)
+      else
+        :elixir_errors.warn "#{caller.file}:#{caller.line}: type #{name}/#{arity} is private, " <>
+                            "@typedoc's are always discarded for private types\n"
+      end
+    end
+
+    Module.delete_attribute(module, :typedoc)
+  end
+
   @doc """
   Defines a `spec` by receiving Erlang's typespec.
   """
@@ -193,9 +326,9 @@ defmodule Kernel.Typespec do
   for modules being compiled.
   """
   def defines_type?(module, name, arity) do
-    finder = match?({ ^name, _, vars } when length(vars) == arity, &1)
-    :lists.any(finder, Module.get_attribute(module, :type)) or
-      :lists.any(finder, Module.get_attribute(module, :opaque))
+    finder = &match?({ ^name, _, vars } when length(vars) == arity, &1)
+    Enum.any?(Module.get_attribute(module, :type), finder) or
+      Enum.any?(Module.get_attribute(module, :opaque), finder)
   end
 
   @doc """
@@ -204,7 +337,7 @@ defmodule Kernel.Typespec do
   """
   def defines_spec?(module, name, arity) do
     tuple = { name, arity }
-    :lists.any(match?(^tuple, &1), Module.get_attribute(module, :spec))
+    Enum.any?(Module.get_attribute(module, :spec), &match?(^tuple, &1))
   end
 
   @doc """
@@ -213,15 +346,28 @@ defmodule Kernel.Typespec do
   """
   def defines_callback?(module, name, arity) do
     tuple = { name, arity }
-    :lists.any(match?(^tuple, &1), Module.get_attribute(module, :callback))
+    Enum.any?(Module.get_attribute(module, :callback), &match?(^tuple, &1))
   end
 
   @doc """
   Converts a spec clause back to Elixir AST.
   """
   def spec_to_ast(name, { :type, line, :fun, [{:type, _, :product, args}, result] }) do
-    args = lc arg inlist args, do: typespec_to_ast(arg)
-    { :::, [line: line], [{ name, [line: line], args }, typespec_to_ast(result)] }
+    meta = [line: line]
+    body = { name, meta, Enum.map(args, &typespec_to_ast/1) }
+
+    vars = args ++ [result]
+      |> Enum.flat_map(&collect_vars/1)
+      |> Enum.uniq
+      |> Enum.map(&{ &1, { :var, meta, nil } })
+
+    result = if vars == [] do
+      typespec_to_ast(result)
+    else
+      { :when, meta, [typespec_to_ast(result), vars] }
+    end
+
+    { :::, meta, [body, result] }
   end
 
   def spec_to_ast(name, { :type, line, :fun, [] }) do
@@ -229,15 +375,25 @@ defmodule Kernel.Typespec do
   end
 
   def spec_to_ast(name, { :type, line, :bounded_fun, [{ :type, _, :fun, [{ :type, _, :product, args }, result] }, constraints] }) do
-    [h|t] =
-      lc {:type, line, :constraint, [{:atom, _, :is_subtype}, [var, type]]} inlist constraints do
-        { :is_subtype, [line: line], [typespec_to_ast(var), typespec_to_ast(type)] }
+    guards =
+      lc {:type, _, :constraint, [{:atom, _, :is_subtype}, [{ :var, _, var }, type]]} inlist constraints do
+        { var, typespec_to_ast(type) }
       end
 
-    args = lc arg inlist args, do: typespec_to_ast(arg)
-    guards = Enum.reduce t, h, fn(x, acc) -> { :and, line, [acc, x] } end
+    meta = [line: line]
 
-    { :::, [line: line], [{ :when, [line: line], [{ name, [line: line], args }, guards] }, typespec_to_ast(result)] }
+    vars = args ++ [result]
+      |> Enum.flat_map(&collect_vars/1)
+      |> Enum.uniq
+      |> Kernel.--(Keyword.keys(guards))
+      |> Enum.map(&{ &1, { :var, meta, nil } })
+
+    args = lc arg inlist args, do: typespec_to_ast(arg)
+
+    { :::, meta, [
+      { name, [line: line], args },
+      { :when, meta, [typespec_to_ast(result), guards ++ vars] }
+    ] }
   end
 
   @doc """
@@ -256,15 +412,36 @@ defmodule Kernel.Typespec do
   end
 
   @doc """
-  Returns all types available from the module's beam code.
+  Returns all type docs available from the module's beam code.
 
-  It is returned as a list of tuples where the first
-  element is the type (`:typep`, `:type` and `:opaque`).
+  The result is returned as a list of tuples where the first element is the pair of type
+  name and arity and the second element is the documentation.
 
-  The module has to have a corresponding beam file on the disk which can be
+  The module must have a corresponding beam file which can be
   located by the runtime system.
   """
-  def beam_types(module) do
+  @spec beam_typedocs(module | binary) :: [tuple] | nil
+  def beam_typedocs(module) when is_atom(module) or is_binary(module) do
+    case abstract_code(module) do
+      { :ok, abstract_code } ->
+        type_docs = lc { :attribute, _, :typedoc, tup } inlist abstract_code, do: tup
+        List.flatten(type_docs)
+      _ ->
+        nil
+    end
+  end
+
+  @doc """
+  Returns all types available from the module's beam code.
+
+  The result is returned as a list of tuples where the first
+  element is the type (`:typep`, `:type` and `:opaque`).
+
+  The module must have a corresponding beam file which can be
+  located by the runtime system.
+  """
+  @spec beam_types(module | binary) :: [tuple] | nil
+  def beam_types(module) when is_atom(module) or is_binary(module) do
     case abstract_code(module) do
       { :ok, abstract_code } ->
         exported_types = lc { :attribute, _, :export_type, types } inlist abstract_code, do: types
@@ -273,38 +450,40 @@ defmodule Kernel.Typespec do
         lc { :attribute, _, kind, { name, _, args } = type } inlist abstract_code, kind in [:opaque, :type] do
           cond do
             kind == :opaque -> { :opaque, type }
-            :lists.member({ name, length(args) }, exported_types) -> { :type, type }
+            { name, length(args) } in exported_types -> { :type, type }
             true -> { :typep, type }
           end
         end
       _ ->
-        []
+        nil
     end
   end
 
   @doc """
   Returns all specs available from the module's beam code.
 
-  It is returned as a list of tuples where the first
+  The result is returned as a list of tuples where the first
   element is spec name and arity and the second is the spec.
 
-  The module has to have a corresponding beam file on the disk which can be
+  The module must have a corresponding beam file which can be
   located by the runtime system.
   """
-  def beam_specs(module) do
+  @spec beam_specs(module | binary) :: [tuple] | nil
+  def beam_specs(module) when is_atom(module) or is_binary(module) do
     from_abstract_code(module, :spec)
   end
 
   @doc """
   Returns all callbacks available from the module's beam code.
 
-  It is returned as a list of tuples where the first
+  The result is returned as a list of tuples where the first
   element is spec name and arity and the second is the spec.
 
-  The module has to have a corresponding beam file on the disk which can be
-  located by the runtime system.
+  The module must have a corresponding beam file
+  which can be located by the runtime system.
   """
-  def beam_callbacks(module) do
+  @spec beam_callbacks(module | binary) :: [tuple] | nil
+  def beam_callbacks(module) when is_atom(module) or is_binary(module) do
     from_abstract_code(module, :callback)
   end
 
@@ -312,8 +491,8 @@ defmodule Kernel.Typespec do
     case abstract_code(module) do
       { :ok, abstract_code } ->
         lc { :attribute, _, abs_kind, value } inlist abstract_code, kind == abs_kind, do: value
-      _ ->
-        []
+      :error ->
+        nil
     end
   end
 
@@ -322,7 +501,7 @@ defmodule Kernel.Typespec do
       {:ok, { _, [{ :abstract_code, { _raw_abstract_v1, abstract_code } }] } } ->
         { :ok, abstract_code }
       _ ->
-        []
+        :error
     end
   end
 
@@ -351,7 +530,7 @@ defmodule Kernel.Typespec do
 
   def deftype(_kind, other, caller) do
     type_spec = Macro.to_string(other)
-    compile_error caller, "invalid type specification #{type_spec}"
+    compile_error caller, "invalid type specification: #{type_spec}"
   end
 
   defp do_deftype(kind, { name, _, args }, definition, caller) do
@@ -368,23 +547,27 @@ defmodule Kernel.Typespec do
     vars = lc { :var, _, _ } = var inlist args, do: var
     type = { name, spec, vars }
 
-    define_type(caller.module, kind, type)
+    define_type(caller, kind, type)
   end
 
   @doc false
-  def defspec(type, {:::, _, [{ :when, _, [{ name, meta, args }, constraints_guard] }, return] }, caller) do
+  def defspec(type, { :::, meta, [{ name, _, args }, return_and_guard] }, caller) do
     if is_atom(args), do: args = []
-    constraints = guard_to_constraints(constraints_guard, caller)
-    spec = { :type, line(meta), :fun, fn_args(meta, args, return, Keyword.keys(constraints), caller) }
-    spec = { :type, line(meta), :bounded_fun, [spec, Keyword.values(constraints)] }
-    code = { { name, Kernel.length(args) }, spec }
-    Module.compile_typespec(caller.module, type, code)
-    code
-  end
+    { return, guard } = split_return_and_guard(return_and_guard)
+
+    unless Keyword.keyword?(guard) do
+      guard = Macro.to_string(guard)
+      compile_error caller, "expected keywords as guard in function type specification, got: #{guard}"
+    end
+
+    vars = Keyword.keys(guard)
+    constraints = guard_to_constraints(guard, vars, meta, caller)
+
+    spec = { :type, line(meta), :fun, fn_args(meta, args, return, vars, caller) }
+    if constraints != [] do
+      spec = { :type, line(meta), :bounded_fun, [spec, constraints] }
+    end
 
-  def defspec(type, {:::, _, [{ name, meta, args }, return]}, caller) do
-    if is_atom(args), do: args = []
-    spec = { :type, line(meta), :fun, fn_args(meta, args, return, [], caller) }
     code = { { name, Kernel.length(args) }, spec }
     Module.compile_typespec(caller.module, type, code)
     code
@@ -392,23 +575,67 @@ defmodule Kernel.Typespec do
 
   def defspec(_type, other, caller) do
     spec = Macro.to_string(other)
-    compile_error caller, "invalid function type specification #{spec}"
+    compile_error caller, "invalid function type specification: #{spec}"
   end
 
-  defp guard_to_constraints({ :is_subtype, meta, [{ name, _, _ }, type] }, caller) do
+  defp split_return_and_guard({ :when, _, [return, guard] }) do
+    { return, guard }
+  end
+
+  defp split_return_and_guard({ :|, meta, [left, right] }) do
+    { return, guard } = split_return_and_guard(right)
+    { { :|, meta, [left, return] }, guard }
+  end
+
+  defp split_return_and_guard(other) do
+    { other, [] }
+  end
+
+  defp guard_to_constraints(guard, vars, meta, caller) do
     line = line(meta)
-    contraints = [{ :atom, line, :is_subtype }, [{:var, line, name}, typespec(type, [], caller)]]
-    [{ name, { :type, line, :constraint, contraints } }]
-  end
 
-  defp guard_to_constraints({ :and, _, [left, right] }, caller) do
-    guard_to_constraints(left, caller) ++ guard_to_constraints(right, caller)
+    Enum.reduce(guard, [], fn
+      { _name, { :var, _, context } }, acc when is_atom(context) ->
+        acc
+      { name, type }, acc ->
+        constraint = [{ :atom, line, :is_subtype }, [{:var, line, name}, typespec(type, vars, caller)]]
+        type = { :type, line, :constraint, constraint }
+        [type|acc]
+    end) |> Enum.reverse
   end
 
   ## To AST conversion
 
+  defp collect_vars({ :ann_type, _line, args }) when is_list(args) do
+    []
+  end
+
+  defp collect_vars({ :type, _line, _kind, args }) when is_list(args) do
+    Enum.flat_map(args, &collect_vars/1)
+  end
+
+  defp collect_vars({ :remote_type, _line, args }) when is_list(args) do
+    Enum.flat_map(args, &collect_vars/1)
+  end
+
+  defp collect_vars({ :typed_record_field, _line, type }) do
+    collect_vars(type)
+  end
+
+  defp collect_vars({:paren_type, _line, [type]}) do
+    collect_vars(type)
+  end
+
+  defp collect_vars({:var, _line, var}) do
+    [erl_to_ex_var(var)]
+  end
+
+  defp collect_vars(_) do
+    []
+  end
+
   defp typespec_to_ast({ :type, line, :tuple, :any }) do
-    typespec_to_ast({:type, line, :tuple, []})
+    { :tuple, [line: line], [] }
   end
 
   defp typespec_to_ast({ :type, line, :tuple, args }) do
@@ -416,8 +643,8 @@ defmodule Kernel.Typespec do
     { :{}, [line: line], args }
   end
 
-  defp typespec_to_ast({ :type, _line, :list, [arg] }) do
-    case unpack_typespec_kw(arg, []) do
+  defp typespec_to_ast({ :type, _line, :list, [{ :type, _, :union, unions } = arg] }) do
+    case unpack_typespec_kw(unions, []) do
       { :ok, ast } -> ast
       :error -> [typespec_to_ast(arg)]
     end
@@ -441,17 +668,16 @@ defmodule Kernel.Typespec do
 
   defp typespec_to_ast({ :type, line, :union, args }) do
     args = lc arg inlist args, do: typespec_to_ast(arg)
-    Enum.reduce tl(args), hd(args),
-      fn(arg, expr) -> { :|, [line: line], [expr, arg] } end
+    Enum.reduce Enum.reverse(args), fn(arg, expr) -> { :|, [line: line], [arg, expr] } end
   end
 
   defp typespec_to_ast({ :type, line, :fun, [{:type, _, :product, args}, result] }) do
     args = lc arg inlist args, do: typespec_to_ast(arg)
-    { :->, [line: line], [{args, [line: line], typespec_to_ast(result)}] }
+    [{ :->, [line: line], [args, typespec_to_ast(result)] }]
   end
 
   defp typespec_to_ast({ :type, line, :fun, [args, result] }) do
-    { :->, [line: line], [{[typespec_to_ast(args)], [line: line], typespec_to_ast(result)}] }
+    [{ :->, [line: line], [[typespec_to_ast(args)], typespec_to_ast(result)] }]
   end
 
   defp typespec_to_ast({ :type, line, :fun, [] }) do
@@ -468,14 +694,11 @@ defmodule Kernel.Typespec do
   end
 
   defp typespec_to_ast({ :var, line, var }) do
-    var =
-      case atom_to_binary(var) do
-        <<"_", c :: [binary, size(1)], rest :: binary>> ->
-          binary_to_atom("_#{String.downcase(c)}#{rest}")
-        <<c :: [binary, size(1)], rest :: binary>> ->
-          binary_to_atom("#{String.downcase(c)}#{rest}")
-      end
-    { var, line, nil }
+    { erl_to_ex_var(var), line, nil }
+  end
+
+  defp typespec_to_ast({ :op, line, op, arg }) do
+    { op, [line: line], [typespec_to_ast(arg)] }
   end
 
   # Special shortcut(s)
@@ -517,6 +740,15 @@ defmodule Kernel.Typespec do
 
   defp typespec_to_ast(other), do: other
 
+  defp erl_to_ex_var(var) do
+    case atom_to_binary(var) do
+      <<"_", c :: [binary, size(1)], rest :: binary>> ->
+        binary_to_atom("_#{String.downcase(c)}#{rest}")
+      <<c :: [binary, size(1)], rest :: binary>> ->
+        binary_to_atom("#{String.downcase(c)}#{rest}")
+    end
+  end
+
   ## From AST conversion
 
   defp line(meta) do
@@ -528,7 +760,7 @@ defmodule Kernel.Typespec do
 
   # Handle unions
   defp typespec({ :|, meta, [_, _] } = exprs, vars, caller) do
-    exprs = Enum.reverse(collect_union(exprs))
+    exprs = collect_union(exprs)
     union = lc e inlist exprs, do: typespec(e, vars, caller)
     { :type, line(meta), :union, union }
   end
@@ -562,11 +794,7 @@ defmodule Kernel.Typespec do
   end
 
   # Handle funs
-  defp typespec({:->, meta, [{[{:fun, _, arguments}], cmeta, return}]}, vars, caller) when is_list(arguments) do
-    typespec({:->, meta, [{arguments, cmeta, return}]}, vars, caller)
-  end
-
-  defp typespec({:->, meta, [{arguments, _, return}]}, vars, caller) when is_list(arguments) do
+  defp typespec([{:->, meta, [arguments, return]}], vars, caller) when is_list(arguments) do
     args = fn_args(meta, arguments, return, vars, caller)
     { :type, line(meta), :fun, args }
   end
@@ -600,11 +828,7 @@ defmodule Kernel.Typespec do
   end
 
   # Handle tuples
-  defp typespec({:tuple, meta, atom}, vars, caller) when is_atom(atom) do
-    typespec({:{}, meta, []}, vars, caller)
-  end
-
-  defp typespec({:{}, meta, []}, _, _) do
+  defp typespec({:tuple, meta, args}, _vars, _caller) when args == [] or is_atom(args) do
     { :type, line(meta), :tuple, :any }
   end
 
@@ -613,6 +837,10 @@ defmodule Kernel.Typespec do
     { :type, line(meta), :tuple, args }
   end
 
+  defp typespec({ left, right }, vars, caller) do
+    typespec({ :{}, [], [left, right] }, vars, caller)
+  end
+
   # Handle blocks
   defp typespec({:__block__, _meta, [arg]}, vars, caller) do
     typespec(arg, vars, caller)
@@ -620,7 +848,7 @@ defmodule Kernel.Typespec do
 
   # Handle variables or local calls
   defp typespec({name, meta, atom}, vars, caller) when is_atom(atom) do
-    if :lists.member(name, vars) do
+    if name in vars do
       { :var, line(meta), name }
     else
       typespec({name, meta, []}, vars, caller)
@@ -629,7 +857,7 @@ defmodule Kernel.Typespec do
 
   # Handle local calls
   defp typespec({:string, meta, arguments}, vars, caller) do
-    IO.write "warning: string() type use is discouraged. For character lists, use " <>
+    :elixir_errors.warn "warning: string() type use is discouraged. For character lists, use " <>
       "char_list() type, for strings, String.t()\n#{Exception.format_stacktrace(caller.stacktrace)}"
     arguments = lc arg inlist arguments, do: typespec(arg, vars, caller)
     { :type, line(meta), :string, arguments }
@@ -669,18 +897,14 @@ defmodule Kernel.Typespec do
     typespec({ :nonempty_list, [], [spec] }, vars, caller)
   end
 
-  defp typespec([h|t] = l, vars, caller) do
-    union = Enum.reduce(t, validate_kw(h, l, caller), fn(x, acc) ->
-      { :|, [], [acc, validate_kw(x, l, caller)] }
+  defp typespec(list, vars, caller) do
+    [h|t] = Enum.reverse(list)
+    union = Enum.reduce(t, validate_kw(h, list, caller), fn(x, acc) ->
+      { :|, [], [validate_kw(x, list, caller), acc] }
     end)
     typespec({ :list, [], [union] }, vars, caller)
   end
 
-  defp typespec(t, vars, caller) when is_tuple(t) do
-    args = lc e inlist tuple_to_list(t), do: typespec(e, vars, caller)
-    { :type, 0, :tuple, args }
-  end
-
   ## Helpers
 
   defp compile_error(caller, desc) do
@@ -692,12 +916,12 @@ defmodule Kernel.Typespec do
     { :remote_type, line(meta), [ remote, name, arguments ] }
   end
 
-  defp collect_union({ :|, _, [a, b] }), do: [b|collect_union(a)]
+  defp collect_union({ :|, _, [a, b] }), do: [a|collect_union(b)]
   defp collect_union(v), do: [v]
 
   defp validate_kw({ key, _ } = t, _, _caller) when is_atom(key), do: t
   defp validate_kw(_, original, caller) do
-    compile_error(caller, "unexpected list #{Macro.to_string original} in typespec")
+    compile_error(caller, "unexpected list in typespec: #{Macro.to_string original}")
   end
 
   defp fn_args(meta, args, return, vars, caller) do
@@ -720,15 +944,12 @@ defmodule Kernel.Typespec do
     {:var, line(meta), name}
   end
 
-  defp unpack_typespec_kw({ :type, _, :union, [
-         next,
-         { :type, _, :tuple, [{ :atom, _, atom }, type] }
-       ] }, acc) do
-    unpack_typespec_kw(next, [{atom, typespec_to_ast(type)}|acc])
+  defp unpack_typespec_kw([{ :type, _, :tuple, [{ :atom, _, atom }, type] }|t], acc) do
+    unpack_typespec_kw(t, [{atom, typespec_to_ast(type)}|acc])
   end
 
-  defp unpack_typespec_kw({ :type, _, :tuple, [{ :atom, _, atom }, type] }, acc) do
-    { :ok, [{atom, typespec_to_ast(type)}|acc] }
+  defp unpack_typespec_kw([], acc) do
+    { :ok, :lists.reverse(acc) }
   end
 
   defp unpack_typespec_kw(_, _acc) do

lib/elixir/lib/keyword.ex

@@ -4,18 +4,30 @@ defmodule Keyword do
   of the tuple is an atom and the second element can be
   any value.
 
-  A keyword may have duplicated keys, so it is not strictly
+  A keyword may have duplicated keys so it is not strictly
   a dictionary. However most of the functions in this module
-  allows it to behave exactly as a dictionary. For example,
-  `Keyword.get` will get the first entry matching the given
-  key, regardless if duplicated entries exist. Similarly,
-  `Keyword.put` and `Keyword.delete` ensure all duplicated
-  entries for a given key are removed when invoked.
+  behaves exactly as a dictionary and mimic the API defined
+  by the `Dict` behaviour.
+
+  For example, `Keyword.get` will get the first entry matching
+  the given key, regardless if duplicated entries exist.
+  Similarly, `Keyword.put` and `Keyword.delete` ensure all
+  duplicated entries for a given key are removed when invoked.
+  A handful of functions exist to handle duplicated keys, in
+  particular, `from_enum` allows creating a new keywords without
+  removing duplicated keys, `get_values` returns all values for
+  a given key and `delete_first` deletes just one of the existing
+  entries.
+
+  Since a keyword list is simply a list, all the operations defined
+  in `Enum` and `List` can also be applied.
   """
 
   @type key :: atom
   @type value :: any
+
   @type t :: [{key, value}]
+  @type t(value) :: [{ key, value }]
 
   @doc """
   Creates a Keyword from an enum. Unlike `Keyword.new`
@@ -23,16 +35,12 @@ defmodule Keyword do
   duplicated entries.
   """
   @spec from_enum(Enum.t) :: t
-  def from_enum(enum) when is_list(enum) do
-    enum
-  end
-
   def from_enum(enum) do
-    Enum.map(enum, fn(x) -> x end)
+    Enum.to_list(enum)
   end
 
   @doc """
-  Checks if the given argument is a keywords list or not
+  Checks if the given argument is a keywords list or not.
   """
   @spec keyword?(term) :: boolean
   def keyword?([{ key, _value } | rest]) when is_atom(key) do
@@ -203,6 +211,27 @@ defmodule Keyword do
     lc { _, value } inlist keywords, do: value
   end
 
+  @doc """
+  Deletes the entry in the keyword list for a `key` with `value`.
+  If no `key` with `value` exists, returns the keyword list unchanged.
+
+  ## Examples
+
+      iex> Keyword.delete([a: 1, b: 2], :a, 1)
+      [b: 2]
+
+      iex> Keyword.delete([a: 1, b: 2, a: 3], :a, 3)
+      [a: 1, b: 2]
+
+      iex> Keyword.delete([b: 2], :a, 5)
+      [b: 2]
+
+  """
+  @spec delete(t, key, value) :: t
+  def delete(keywords, key, value) when is_atom(key) do
+    lc { k, v } = tuple inlist keywords, key != k or value != v, do: tuple
+  end
+
   @doc """
   Deletes all entries in the keyword list for a specific `key`.
   If the `key` does not exist, returns the keyword list unchanged.
@@ -248,7 +277,7 @@ defmodule Keyword do
   Puts the given `value` under `key`.
 
   If a previous value is already stored, all entries are
-  removed and the value is overriden.
+  removed and the value is overridden.
 
   ## Examples
 
@@ -356,22 +385,16 @@ defmodule Keyword do
     :lists.keymember(key, 1, keywords)
   end
 
-  @doc false
-  def update(dict, key, fun) when is_function(fun, 1) do
-    IO.write "Keyword.update/3 is deprecated, please use Keyword.update!/3 instead\n#{Exception.format_stacktrace}"
-    update!(dict, key, fun)
-  end
-
   @doc """
   Updates the `key` with the given function. If the `key` does
   not exist, raises `KeyError`.
 
   ## Examples
 
-      iex> Keyword.update!([a: 1], :a, &1 * 2)
+      iex> Keyword.update!([a: 1], :a, &(&1 * 2))
       [a: 2]
 
-      iex> Keyword.update!([a: 1], :b, &1 * 2)
+      iex> Keyword.update!([a: 1], :b, &(&1 * 2))
       ** (KeyError) key not found: :b
 
   """
@@ -394,10 +417,10 @@ defmodule Keyword do
 
   ## Examples
 
-      iex> Keyword.update([a: 1], :a, 13, &1 * 2)
+      iex> Keyword.update([a: 1], :a, 13, &(&1 * 2))
       [a: 2]
 
-      iex> Keyword.update([a: 1], :b, 11, &1 * 2)
+      iex> Keyword.update([a: 1], :b, 11, &(&1 * 2))
       [a: 1, b: 11]
 
   """
@@ -413,4 +436,128 @@ defmodule Keyword do
   def update([], key, initial, _fun) when is_atom(key) do
     [{key, initial}]
   end
+
+  @doc """
+  Splits the given keywords in two given the given keys.
+  Duplicated keys are preserved in the split keyword list.
+
+  ## Examples
+
+      iex> d = [a: 1, b: 2, c: 3, d: 4]
+      iex> Keyword.split(d, [:a, :c, :e])
+      { [a: 1, c: 3], [b: 2, d: 4] }
+
+      iex> d = [a: 1, b: 2, c: 3, d: 4, a: 5]
+      iex> Keyword.split(d, [:a, :c, :e])
+      { [a: 1, c: 3, a: 5], [b: 2, d: 4] }
+
+  """
+  def split(dict, keys) do
+    acc = { [], [] }
+
+    { take, drop } = Enum.reduce dict, acc, fn({ k, v }, { take, drop }) ->
+      case :lists.member(k, keys) do
+        true  -> { [{k, v}|take], drop }
+        false -> { take, [{k, v}|drop] }
+      end
+    end
+
+    { Enum.reverse(take), Enum.reverse(drop) }
+  end
+
+  @doc """
+  Takes the given keys from the dict.
+  Duplicated keys are preserved in the new keyword list.
+
+  ## Examples
+
+      iex> d = [a: 1, b: 2, c: 3, d: 4]
+      iex> Keyword.take(d, [:a, :c, :e])
+      [a: 1, c: 3]
+
+      iex> d = [a: 1, b: 2, c: 3, d: 4, a: 5]
+      iex> Keyword.take(d, [:a, :c, :e])
+      [a: 1, c: 3, a: 5]
+
+  """
+  def take(dict, keys) do
+    lc { k, _ } = tuple inlist dict, :lists.member(k, keys), do: tuple
+  end
+
+  @doc """
+  Drops the given keys from the dict.
+  Duplicated keys are preserved in the new keyword list.
+
+  ## Examples
+
+      iex> d = [a: 1, b: 2, c: 3, d: 4]
+      iex> Keyword.drop(d, [:b, :d])
+      [a: 1, c: 3]
+
+      iex> d = [a: 1, b: 2, c: 3, d: 4, a: 5]
+      iex> Keyword.drop(d, [:b, :d])
+      [a: 1, c: 3, a: 5]
+
+  """
+  def drop(dict, keys) do
+    lc { k, _ } = tuple inlist dict, not :lists.member(k, keys), do: tuple
+  end
+
+  @doc """
+  Returns the first value associated with `key` in the keyword
+  list as well as the keyword list without `key`.
+
+  All duplicated entries are removed. See `pop_first/3` for
+  removing only the first entry.
+
+  ## Examples
+
+      iex> Keyword.pop [a: 1], :a
+      {1,[]}
+
+      iex> Keyword.pop [a: 1], :b
+      {nil,[a: 1]}
+
+      iex> Keyword.pop [a: 1], :b, 3
+      {3,[a: 1]}
+
+      iex> Keyword.pop [a: 1], :b, 3
+      {3,[a: 1]}
+
+      iex> Keyword.pop [a: 1, a: 2], :a
+      {1,[]}
+
+  """
+  def pop(dict, key, default // nil) do
+    { get(dict, key, default), delete(dict, key) }
+  end
+
+  @doc """
+  Returns the first value associated with `key` in the keyword
+  list as well as the keyword list without that particular ocurrence
+  of `key`.
+
+  Duplicated entries are not removed.
+
+  ## Examples
+
+      iex> Keyword.pop_first [a: 1], :a
+      {1,[]}
+
+      iex> Keyword.pop_first [a: 1], :b
+      {nil,[a: 1]}
+
+      iex> Keyword.pop_first [a: 1], :b, 3
+      {3,[a: 1]}
+
+      iex> Keyword.pop_first [a: 1], :b, 3
+      {3,[a: 1]}
+
+      iex> Keyword.pop_first [a: 1, a: 2], :a
+      {1,[a: 2]}
+
+  """
+  def pop_first(dict, key, default // nil) do
+    { get(dict, key, default), delete_first(dict, key) }
+  end
 end

lib/elixir/lib/list.ex

@@ -4,6 +4,12 @@ defmodule List do
   and cannot be part of the Enum protocol. In general,
   favor using the Enum API instead of List.
 
+  Some functions in this module expect an index. Index
+  access for list is linear. Negative indexes are also
+  supported but they imply the list will be iterated twice,
+  one to calculate the proper index and another to the
+  operation.
+
   A decision was taken to delegate most functions to
   Erlang's standard library but follow Elixir's convention
   of receiving the target (in this case, a list) as the
@@ -12,22 +18,10 @@ defmodule List do
 
   @compile :inline_list_funcs
 
-  @doc false
-  def concat(list) when is_list(list) do
-    IO.write "List.concat/1 is deprecated, please use Enum.concat/1 instead\n#{Exception.format_stacktrace}"
-    :lists.append(list)
-  end
-
-  @doc false
-  def concat(list, elements) when is_list(list) and is_list(elements) do
-    IO.write "List.concat/2 is deprecated, please use Enum.concat/2 instead\n#{Exception.format_stacktrace}"
-    list ++ elements
-  end
-
   @doc """
-  Deletes the given item from the list. Returns a list without the item.
-  If the item occurs more than once in the list, just the first occurrence
-  is removed.
+  Deletes the given item from the list. Returns a list without
+  the item. If the item occurs more than once in the list, just
+  the first occurrence is removed.
 
   ## Examples
 
@@ -38,6 +32,7 @@ defmodule List do
       [1, 2, 3]
 
   """
+  @spec delete(list, any) :: list
   def delete(list, item) do
     :lists.delete(item, list)
   end
@@ -53,6 +48,7 @@ defmodule List do
       iex> List.duplicate([1, 2], 2)
       [[1,2],[1,2]]
   """
+  @spec duplicate(elem, non_neg_integer) :: [elem] when elem: var
   def duplicate(elem, n) do
     :lists.duplicate(n, elem)
   end
@@ -66,12 +62,13 @@ defmodule List do
       [1,2,3]
 
   """
+  @spec flatten(deep_list) :: list when deep_list: [any | deep_list]
   def flatten(list) do
     :lists.flatten(list)
   end
 
   @doc """
-  Flattens the given `list` of nested lists. 
+  Flattens the given `list` of nested lists.
   The list `tail` will be added at the end of
   the flattened list.
 
@@ -81,6 +78,7 @@ defmodule List do
       [1,2,3,4,5]
 
   """
+  @spec flatten(deep_list, [elem]) :: [elem] when elem: var, deep_list: [elem | deep_list]
   def flatten(list, tail) do
     :lists.flatten(list, tail)
   end
@@ -98,6 +96,7 @@ defmodule List do
       2
 
   """
+  @spec foldl([elem], acc, (elem, acc -> acc)) :: acc when elem: var, acc: var
   def foldl(list, acc, function) when is_list(list) and is_function(function) do
     :lists.foldl(function, acc, list)
   end
@@ -112,10 +111,28 @@ defmodule List do
       -2
 
   """
+  @spec foldr([elem], acc, (elem, acc -> acc)) :: acc when elem: var, acc: var
   def foldr(list, acc, function) when is_list(list) and is_function(function) do
     :lists.foldr(function, acc, list)
   end
 
+  @doc """
+  Returns the first element in `list` or `nil` if `list` is empty.
+
+  ## Examples
+
+      iex> List.first([])
+      nil
+      iex> List.first([1])
+      1
+      iex> List.first([1, 2, 3])
+      1
+
+  """
+  @spec first([elem]) :: nil | elem when elem: var
+  def first([]),    do: nil
+  def first([h|_]), do: h
+
   @doc """
   Returns the last element in `list` or `nil` if `list` is empty.
 
@@ -129,11 +146,10 @@ defmodule List do
       3
 
   """
-  def last([]), do: nil
-
-  def last(list) do
-    :lists.last(list)
-  end
+  @spec last([elem]) :: nil | elem when elem: var
+  def last([]),    do: nil
+  def last([h]),   do: h
+  def last([_|t]), do: last(t)
 
   @doc """
   Receives a list of tuples and returns the first tuple
@@ -152,6 +168,7 @@ defmodule List do
       nil
 
   """
+  @spec keyfind([tuple], any, non_neg_integer, any) :: any
   def keyfind(list, key, position, default // nil) do
     :lists.keyfind(key, position + 1, list) || default
   end
@@ -173,6 +190,7 @@ defmodule List do
       false
 
   """
+  @spec keymember?([tuple], any, non_neg_integer) :: any
   def keymember?(list, key, position) do
     :lists.keymember(key, position + 1, list)
   end
@@ -187,6 +205,7 @@ defmodule List do
       [a: 3, b: 2]
 
   """
+  @spec keyreplace([tuple], any, non_neg_integer, tuple) :: [tuple]
   def keyreplace(list, key, position, new_tuple) do
     :lists.keyreplace(key, position + 1, list, new_tuple)
   end
@@ -204,6 +223,7 @@ defmodule List do
       [a: 5, b: 3, c: 1]
 
   """
+  @spec keysort([tuple], non_neg_integer) :: [tuple]
   def keysort(list, position) do
     :lists.keysort(position + 1, list)
   end
@@ -222,6 +242,7 @@ defmodule List do
       [a: 1, b: 2, c: 3]
 
   """
+  @spec keystore([tuple], any, non_neg_integer, tuple) :: [tuple]
   def keystore(list, key, position, new_tuple) do
     :lists.keystore(key, position + 1, list, new_tuple)
   end
@@ -234,15 +255,16 @@ defmodule List do
   ## Examples
 
       iex> List.keydelete([a: 1, b: 2], :a, 0)
-      [{ :b, 2 }]
+      [b: 2]
 
       iex> List.keydelete([a: 1, b: 2], 2, 1)
-      [{ :a, 1 }]
+      [a: 1]
 
       iex> List.keydelete([a: 1, b: 2], :c, 0)
-      [{ :a, 1 }, { :b, 2 }]
+      [a: 1, b: 2]
 
   """
+  @spec keydelete([tuple], any, non_neg_integer) :: [tuple]
   def keydelete(list, key, position) do
     :lists.keydelete(key, position + 1, list)
   end
@@ -264,6 +286,7 @@ defmodule List do
       []
 
   """
+  @spec wrap(list | any) :: list
   def wrap(list) when is_list(list) do
     list
   end
@@ -288,6 +311,7 @@ defmodule List do
       [{1, 3, 5}]
 
   """
+  @spec zip([list]) :: [tuple]
   def zip([]), do: []
   def zip(list_of_lists) when is_list(list_of_lists) do
     do_zip(list_of_lists, [])
@@ -306,14 +330,15 @@ defmodule List do
       [[1, 2, 3], [:a, :b, :c]]
 
   """
+  @spec unzip([tuple]) :: [list]
   def unzip(list) when is_list(list) do
-    :lists.map tuple_to_list(&1), zip(list)
+    :lists.map &tuple_to_list/1, zip(list)
   end
 
   @doc """
-  Returns a list with `value` inserted at the specified `index`. Note that `index`
-  is capped at the list length. Negative indices indicate an offset from the
-  end of the list.
+  Returns a list with `value` inserted at the specified `index`.
+  Note that `index` is capped at the list length. Negative indices
+  indicate an offset from the end of the list.
 
   ## Examples
 
@@ -330,6 +355,7 @@ defmodule List do
       [0, 1, 2, 3]
 
   """
+  @spec insert_at(list, integer, any) :: list
   def insert_at(list, index, value) do
     if index < 0 do
       do_insert_at(list, length(list) + index + 1, value)
@@ -339,9 +365,9 @@ defmodule List do
   end
 
   @doc """
-  Returns a list with a replaced value at the specified `index`. Negative indices
-  indicate an offset from the end of the list. If `index` is out of bounds, the
-  original `list` is returned.
+  Returns a list with a replaced value at the specified `index`.
+  Negative indices indicate an offset from the end of the list.
+  If `index` is out of bounds, the original `list` is returned.
 
   ## Examples
 
@@ -358,6 +384,7 @@ defmodule List do
       [1, 2, 3]
 
   """
+  @spec replace_at(list, integer, any) :: list
   def replace_at(list, index, value) do
     if index < 0 do
       do_replace_at(list, length(list) + index, value)
@@ -366,6 +393,61 @@ defmodule List do
     end
   end
 
+  @doc """
+  Returns a list with an updated value at the specified `index`.
+  Negative indices indicate an offset from the end of the list.
+  If `index` is out of bounds, the original `list` is returned.
+
+  ## Examples
+
+      iex> List.update_at([1, 2, 3], 0, &(&1 + 10))
+      [11, 2, 3]
+
+      iex> List.update_at([1, 2, 3], 10, &(&1 + 10))
+      [1, 2, 3]
+
+      iex> List.update_at([1, 2, 3], -1, &(&1 + 10))
+      [1, 2, 13]
+
+      iex> List.update_at([1, 2, 3], -10, &(&1 + 10))
+      [1, 2, 3]
+
+  """
+  @spec update_at([elem], integer, (elem -> any)) :: list when elem: var
+  def update_at(list, index, fun) do
+    if index < 0 do
+      do_update_at(list, length(list) + index, fun)
+    else
+      do_update_at(list, index, fun)
+    end
+  end
+
+  @doc """
+  Produces a new list by removing the value at the specified `index`.
+  Negative indices indicate an offset from the end of the list.
+  If `index` is out of bounds, the original `list` is returned.
+
+  ## Examples
+
+      iex> List.delete_at([1, 2, 3], 0)
+      [2, 3]
+
+      iex List.delete_at([1, 2, 3], 10)
+      [1, 2, 3]
+
+      iex> List.delete_at([1, 2, 3], -1)
+      [1, 2]
+
+  """
+  @spec delete_at(list, integer) :: list
+  def delete_at(list, index) do
+    if index < 0 do
+      do_delete_at(list, length(list) + index)
+    else
+      do_delete_at(list, index)
+    end
+  end
+
   ## Helpers
 
   # replace_at
@@ -400,6 +482,42 @@ defmodule List do
     [ h | do_insert_at(t, index - 1, value) ]
   end
 
+  # update_at
+
+  defp do_update_at([value|list], 0, fun) do
+    [ fun.(value) | list ]
+  end
+
+  defp do_update_at(list, index, _fun) when index < 0 do
+    list
+  end
+
+  defp do_update_at([h|t], index, fun) do
+    [ h | do_update_at(t, index - 1, fun) ]
+  end
+
+  defp do_update_at([], _index, _fun) do
+    []
+  end
+
+  # delete_at
+
+  defp do_delete_at([], _index) do
+    []
+  end
+
+  defp do_delete_at([_|t], 0) do
+    t
+  end
+
+  defp do_delete_at(list, index) when index < 0 do
+    list
+  end
+
+  defp do_delete_at([h|t], index) do
+    [h | do_delete_at(t, index-1)]
+  end
+
   # zip
 
   defp do_zip(list, acc) do

lib/elixir/lib/list/chars.ex

@@ -9,8 +9,6 @@ defprotocol List.Chars do
   by Kernel invokes this protocol.
   """
 
-  @only [BitString, List, Atom, Number, Record]
-
   def to_char_list(thing)
 end
 
@@ -38,13 +36,15 @@ defimpl List.Chars, for: List do
   def to_char_list(list), do: list
 end
 
-defimpl List.Chars, for: Number do
-  @digits 20
-  @limit  :math.pow(10, @digits)
-
-  def to_char_list(thing) when is_integer(thing) do
+defimpl List.Chars, for: Integer do
+  def to_char_list(thing) do
     integer_to_list(thing)
   end
+end
+
+defimpl List.Chars, for: Float do
+  @digits 20
+  @limit  :math.pow(10, @digits)
 
   def to_char_list(thing) when thing > @limit do
     float_to_list(thing, scientific: @digits)

lib/elixir/lib/list_dict.ex

@@ -2,6 +2,10 @@ defmodule ListDict do
   @moduledoc """
   A Dict implementation that works on lists of two-items tuples.
 
+  This dictionary is only recommended for keeping a small amount
+  of values. Other dict alternatives are more viable for keeping
+  any other amount than a handful.
+
   For more information about the functions and their APIs, please
   consult the `Dict` module.
   """
@@ -38,28 +42,25 @@ defmodule ListDict do
     length(dict)
   end
 
-  def has_key?(dict, key) do
-    :lists.keymember(key, 1, dict)
-  end
+  def has_key?(dict, key)
+  def has_key?([{ key, _ }|_], key), do: true
+  def has_key?([{ _, _ }|t], key), do: has_key?(t, key)
+  def has_key?([], _key), do: false
 
-  def get(dict, key, default // nil) do
-    case :lists.keyfind(key, 1, dict) do
-      { ^key, value } -> value
-      false -> default
-    end
-  end
+  def get(dict, key, default // nil)
+  def get([{ key, value }|_], key, _default), do: value
+  def get([{ _, _ }|t], key, default), do: get(t, key, default)
+  def get([], _key, default), do: default
 
-  def fetch(dict, key) do
-    case :lists.keyfind(key, 1, dict) do
-      { ^key, value } -> { :ok, value }
-      false -> :error
-    end
-  end
+  def fetch(dict, key)
+  def fetch([{ key, value }|_], key), do: { :ok, value }
+  def fetch([{ _, _ }|t], key), do: fetch(t, key)
+  def fetch([], _key), do: :error
 
   def fetch!(dict, key) do
-    case :lists.keyfind(key, 1, dict) do
-      { ^key, value } -> value
-      false -> raise(KeyError, key: key)
+    case fetch(dict, key) do
+      { :ok, value } -> value
+      :error -> raise(KeyError, key: key)
     end
   end
 
@@ -72,26 +73,26 @@ defmodule ListDict do
   end
 
   def put_new(dict, key, val) do
-    case :lists.keyfind(key, 1, dict) do
-      { ^key, _ } -> dict
+    case has_key?(dict, key) do
+      true  -> dict
       false -> [{key, val}|dict]
     end
   end
 
-  def delete(dict, key) do
-    lc { k, _ } = tuple inlist dict, key != k, do: tuple
-  end
+  def delete(dict, key)
+  def delete([{ key, _ }|t], key), do: t
+  def delete([{ _, _ } = h|t], key), do: [h|delete(t, key)]
+  def delete([], _key), do: []
 
-  def merge(dict, enum, callback // fn(_k, _v1, v2) -> v2 end)
-
-  def merge(dict1, dict2, fun) do
-    Enum.reduce dict2, dict1, fn { k, v2 }, acc ->
-      update(acc, k, v2, fn(v1) -> fun.(k, v1, v2) end)
+  def merge(dict, enum, callback // fn(_k, _v1, v2) -> v2 end) do
+    Enum.reduce enum, dict, fn { k, v2 }, acc ->
+      update(acc, k, v2, fn(v1) -> callback.(k, v1, v2) end)
     end
   end
 
   def split(dict, keys) do
-    acc = { new(), new() }
+    acc = { [], [] }
+
     {take, drop} = Enum.reduce dict, acc, fn({ k, v }, { take, drop }) ->
       if :lists.member(k, keys) do
         { [{k, v}|take], drop }
@@ -99,7 +100,7 @@ defmodule ListDict do
         { take, [{k, v}|drop] }
       end
     end
-    
+
     {Enum.reverse(take), Enum.reverse(drop)}
   end
 
@@ -111,12 +112,6 @@ defmodule ListDict do
     lc { k, _ } = tuple inlist dict, not :lists.member(k, keys), do: tuple
   end
 
-  @doc false
-  def update(dict, key, fun) when is_function(fun, 1) do
-    IO.write "ListDict.update/3 is deprecated, please use ListDict.update!/3 instead\n#{Exception.format_stacktrace}"
-    update!(dict, key, fun)
-  end
-
   def update!([{key, value}|dict], key, fun) do
     [{key, fun.(value)}|delete(dict, key)]
   end
@@ -144,7 +139,7 @@ defmodule ListDict do
   def empty(_dict), do: []
 
   def equal?(dict, other) do
-    :lists.keysort(1, dict) == :lists.keysort(1, other)
+    :lists.keysort(1, dict) === :lists.keysort(1, other)
   end
 
   def to_list(dict), do: dict

lib/elixir/lib/macro.ex

@@ -2,79 +2,163 @@ import Kernel, except: [to_string: 1]
 
 defmodule Macro do
   @moduledoc """
-  This module provides conveniences for working with macros.
+  Conveniences for working with macros.
   """
 
-  @type t :: { t, t } | { t, Keyword.t, t } | atom | number | binary | list
+  @typedoc "Abstract Syntax Tree (AST)"
+  @type t :: expr | { t, t } | atom | number | binary | pid | fun | [t]
+
+  @typedoc "Expr node (remaining ones are literals)"
+  @type expr :: { expr | atom, Keyword.t, atom | [t] }
+
+  @binary_ops [:===, :!==,
+    :==, :!=, :<=, :>=,
+    :&&, :||, :<>, :++, :--, :**, ://, :::, :<-, :.., :|>, :=~,
+    :<, :>, :->,
+    :+, :-, :*, :/, :=, :|, :.,
+    :and, :or, :xor, :when, :in, :inlist, :inbits,
+    :<<<, :>>>, :|||, :&&&, :^^^, :~~~]
 
   @doc false
-  defmacro binary_ops do
-    [ :===, :!==,
-      :==, :!=, :<=, :>=,
-      :&&, :||, :<>, :++, :--, :**, ://, :::, :<-, :.., :|>, :=~,
-      :<, :>, :->,
-      :+, :-, :*, :/, :=, :|, :.,
-      :and, :or, :xor, :when, :in, :inlist, :inbits,
-      :<<<, :>>>, :|||, :&&&, :^^^, :~~~ ]
-  end
+  defmacro binary_ops, do: @binary_ops
+
+  @unary_ops [:!, :@, :^, :not, :+, :-, :~~~, :&]
 
   @doc false
-  defmacro unary_ops do
-    [:!, :@, :^, :not, :+, :-, :~~~, :&]
-  end
+  defmacro unary_ops, do: @unary_ops
 
-  @typep precedence :: integer # Higher means binds stronger
-  @spec binary_op_props(atom) :: { :left|:right, precedence }
+  @spec binary_op_props(atom) :: { :left | :right, precedence :: integer }
   defp binary_op_props(o) do
     case o do
-      :::                                                       -> {:right, 30}
-      :when                                                     -> {:right, 40}
-      o when o in [:inlist, :inbits]                            -> {:left, 50}
-      ://                                                       -> {:right, 60}
-      :|                                                        -> {:left, 70}
+      :::                                                       -> {:right, 40}
+      o when o in [:inlist, :inbits, ://, :|]                   -> {:right, 50}
+      :when                                                     -> {:right, 70}
       :=                                                        -> {:right, 80}
       o when o in [:||, :|||, :or, :xor]                        -> {:left, 130}
       o when o in [:&&, :&&&, :and]                             -> {:left, 140}
       o when o in [:==, :!=, :<, :<=, :>=, :>, :=~, :===, :!==] -> {:left, 150}
       o when o in [:<-, :|>, :<<<, :>>>]                        -> {:right, 160}
       :in                                                       -> {:left, 170}
-      :..                                                       -> {:left, 200}
+      o when o in [:++, :--, :**, :.., :<>]                     -> {:right, 200}
       o when o in [:+, :-]                                      -> {:left, 210}
       o when o in [:*, :/]                                      -> {:left, 220}
-      o when o in [:<>]                                         -> {:right, 230}
       :^^^                                                      -> {:left, 250}
       :.                                                        -> {:left, 310}
     end
   end
 
   @doc """
-  Receives an expresion representing a possible definition
-  and extracts its arguments. It returns a tuple with the
-  function name and the arguments list or `:error` if not
-  a valid call syntax.
+  Breaks a pipeline expression into a list.
 
-  This is useful for macros that want to provide the same
-  argument syntax available in def/defp/defmacro and friends.
+  Raises if the pipeline is ill-formed.
+  """
+  @spec unpipe(Macro.t) :: [Macro.t]
+  def unpipe({ :|> , _, [left, right] }) do
+    [left|unpipe(right)]
+  end
+
+  def unpipe(other) do
+    [other]
+  end
+
+  @doc """
+  Pipes `expr` into the `call_expr` as the
+  argument in the given `position`.
+  """
+  @spec pipe(Macro.t, Macro.t, integer) :: Macro.t | no_return
+  def pipe(expr, call_args, integer // 0)
+
+  def pipe(expr, { call, line, atom }, integer) when is_atom(atom) do
+    { call, line, List.insert_at([], integer, expr) }
+  end
+
+  def pipe(expr, { call, line, args }, integer) when is_list(args) do
+    { call, line, List.insert_at(args, integer, expr) }
+  end
+
+  def pipe(expr, call_args, _integer) do
+    raise ArgumentError,
+      message: "cannot pipe #{to_string expr} into #{to_string call_args}"
+  end
+
+  @doc """
+  Recurs the quoted expression applying the given function to
+  each metadata node.
+
+  This is often useful to remove information like lines and
+  hygienic counters from the expression for either storage or
+  comparison.
 
   ## Examples
 
-      extract_args(quote do: foo)        == { :foo, [] }
-      extract_args(quote do: foo())      == { :foo, [] }
-      extract_args(quote do: foo(1, 2, 3)) == { :foo, [1, 2, 3] }
-      extract_args(quote do: 1.(1, 2, 3))  == :error
+      iex> quoted = quote line: 10, do: sample()
+      {:sample, [line: 10], []}
+      iex> Macro.update_meta(quoted, &Keyword.delete(&1, :line))
+      {:sample, [], []}
 
   """
-  @spec extract_args(Macro.t) :: { atom, [Macro.t] } | :error
-  def extract_args(expr) do
-    :elixir_clauses.extract_args(expr)
+  @spec update_meta(t, (Keyword.t -> Keyword.t)) :: t
+  def update_meta(quoted, fun)
+
+  def update_meta({ left, meta, right }, fun) when is_list(meta) do
+    { update_meta(left, fun), fun.(meta), update_meta(right, fun) }
   end
 
+  def update_meta({ left, right }, fun) do
+    { update_meta(left, fun), update_meta(right, fun) }
+  end
+
+  def update_meta(list, fun) when is_list(list) do
+    lc x inlist list, do: update_meta(x, fun)
+  end
+
+  def update_meta(other, _fun) do
+    other
+  end
+
+  @doc """
+  Decomposes a local or remote call into its remote part (when provided),
+  function name and argument list.
+
+  Returns `:error` when an invalid call syntax is provided.
+
+  ## Examples
+
+      iex> Macro.decompose_call(quote do: foo)
+      { :foo, [] }
+
+      iex> Macro.decompose_call(quote do: foo())
+      { :foo, [] }
+
+      iex> Macro.decompose_call(quote do: foo(1, 2, 3))
+      { :foo, [1, 2, 3] }
+
+      iex> Macro.decompose_call(quote do: Elixir.M.foo(1, 2, 3))
+      { { :__aliases__, [], [:Elixir, :M] }, :foo, [1, 2, 3] }
+
+      iex> Macro.decompose_call(quote do: 42)
+      :error
+
+  """
+  @spec decompose_call(Macro.t) :: { atom, [Macro.t] } | { Macro.t, atom, [Macro.t] } | :error
+  def decompose_call({ { :., _, [remote, function] }, _, args }) when is_tuple(remote) or is_atom(remote),
+    do: { remote, function, args }
+
+  def decompose_call({ name, _, args }) when is_atom(name) and is_atom(args),
+    do: { name, [] }
+
+  def decompose_call({ name, _, args }) when is_atom(name) and is_list(args),
+    do: { name, args }
+
+  def decompose_call(_),
+    do: :error
+
   @doc """
   Recursively escapes a value so it can be inserted
   into a syntax tree.
 
   One may pass `unquote: true` to `escape/2`
-  which leaves unquote statements unescaped, effectively
+  which leaves `unquote` statements unescaped, effectively
   unquoting the contents on escape.
 
   ## Examples
@@ -89,26 +173,16 @@ defmodule Macro do
       1
 
   """
-  @spec escape(Macro.t) :: Macro.t
-  @spec escape(Macro.t, Keyword.t) :: Macro.t
+  @spec escape(term) :: Macro.t
+  @spec escape(term, Keyword.t) :: Macro.t
   def escape(expr, opts // []) do
-    :elixir_quote.escape(expr, Keyword.get(opts, :unquote, false)) |> elem(0)
-  end
-
-  @doc false
-  def unescape_binary(chars) do
-    IO.write "Macro.unescape_binary/1 is deprecated, please use Macro.unescape_string/1 instead\n#{Exception.format_stacktrace}"
-    unescape_string(chars)
-  end
-
-  @doc false
-  def unescape_binary(chars, map) do
-    IO.write "Macro.unescape_binary/2 is deprecated, please use Macro.unescape_string/2 instead\n#{Exception.format_stacktrace}"
-    unescape_string(chars, map)
+    elem(:elixir_quote.escape(expr, Keyword.get(opts, :unquote, false)), 0)
   end
 
   @doc %S"""
-  Unescape the given chars. This is the unescaping behavior
+  Unescape the given chars.
+
+  This is the unescaping behavior
   used by default in Elixir single- and double-quoted strings.
   Check `unescape_string/2` for information on how to customize
   the escaping map.
@@ -118,7 +192,7 @@ defmodule Macro do
   also escaped according to the latin1 set they represent.
 
   This function is commonly used on sigil implementations
-  (like `%r`, `%b` and others) which receive a raw, unescaped
+  (like `%r`, `%s` and others) which receive a raw, unescaped
   string.
 
   ## Examples
@@ -127,7 +201,7 @@ defmodule Macro do
       "example\n"
 
   In the example above, we pass a string with `\n` escaped
-  and we return a version with it unescaped.
+  and return a version with it unescaped.
   """
   @spec unescape_string(String.t) :: String.t
   def unescape_string(chars) do
@@ -136,13 +210,14 @@ defmodule Macro do
 
   @doc %S"""
   Unescape the given chars according to the map given.
+
   Check `unescape_string/1` if you want to use the same map
   as Elixir single- and double-quoted strings.
 
   ## Map
 
   The map must be a function. The function receives an integer
-  representing the number of the characters it wants to unescape.
+  representing the codepoint of the character it wants to unescape.
   Here is the default mapping function implemented by Elixir:
 
       def unescape_map(?a), do: ?\a
@@ -163,18 +238,18 @@ defmodule Macro do
   ## Octals
 
   Octals will by default be escaped unless the map function
-  returns false for ?0.
+  returns `false` for `?0`.
 
   ## Hex
 
   Hexadecimals will by default be escaped unless the map function
-  returns false for ?x.
+  returns `false` for `?x`.
 
   ## Examples
 
-  Using the unescape_map defined above is easy:
+  Using the `unescape_map` function defined above is easy:
 
-      Macro.unescape_string "example\\n", unescape_map(&1)
+      Macro.unescape_string "example\\n", &unescape_map(&1)
 
   """
   @spec unescape_string(String.t, (non_neg_integer -> non_neg_integer | false)) :: String.t
@@ -184,12 +259,13 @@ defmodule Macro do
 
   @doc """
   Unescape the given tokens according to the default map.
+
   Check `unescape_string/1` and `unescape_string/2` for more
   information about unescaping.
 
   Only tokens that are binaries are unescaped, all others are
   ignored. This function is useful when implementing your own
-  sigils. Check the implementation of `Kernel.sigil_b`
+  sigils. Check the implementation of `Kernel.sigil_s/2`
   for examples.
   """
   @spec unescape_tokens([Macro.t]) :: [Macro.t]
@@ -199,6 +275,7 @@ defmodule Macro do
 
   @doc """
   Unescape the given tokens according to the given map.
+
   Check `unescape_tokens/1` and `unescape_string/2` for more information.
   """
   @spec unescape_tokens([Macro.t], (non_neg_integer -> non_neg_integer | false)) :: [Macro.t]
@@ -221,7 +298,7 @@ defmodule Macro do
 
   # Variables
   def to_string({ var, _, atom } = ast, fun) when is_atom(atom) do
-    fun.(ast, atom_to_binary(var, :utf8))
+    fun.(ast, atom_to_binary(var))
   end
 
   # Aliases
@@ -249,36 +326,43 @@ defmodule Macro do
 
   # Tuple containers
   def to_string({ :{}, _, args } = ast, fun) do
-    fun.(ast, "{" <> Enum.map_join(args, ", ", &to_string(&1, fun)) <> "}")
-  end
-
-  # List containers
-  def to_string({ :[], _, args } = ast, fun) do
-    fun.(ast, "[" <> Enum.map_join(args, ", ", &to_string(&1, fun)) <> "]")
+    tuple = "{" <> Enum.map_join(args, ", ", &to_string(&1, fun)) <> "}"
+    fun.(ast, tuple)
   end
 
   # Fn keyword
-  def to_string({ :fn, _, [[do: { :->, _, [{_, _, tuple}] } = arrow]] } = ast, fun)
+  def to_string({ :fn, _, [{ :->, _, [_, tuple] }] = arrow } = ast, fun)
       when not is_tuple(tuple) or elem(tuple, 0) != :__block__ do
     fun.(ast, "fn " <> arrow_to_string(arrow, fun) <> " end")
   end
 
-  def to_string({ :fn, _, [[do: { :->, _, [_] } = block]] } = ast, fun) do
+  def to_string({ :fn, _, [{ :->, _, _ }] = block } = ast, fun) do
     fun.(ast, "fn " <> block_to_string(block, fun) <> "\nend")
   end
 
-  def to_string({ :fn, _, [[do: block]] } = ast, fun) do
+  def to_string({ :fn, _, block } = ast, fun) do
     block = adjust_new_lines block_to_string(block, fun), "\n  "
     fun.(ast, "fn\n  " <> block <> "\nend")
   end
 
   # left -> right
-  def to_string({ :->, _, _ } = ast, fun) do
+  def to_string([{ :->, _, _ }|_] = ast, fun) do
     fun.(ast, "(" <> arrow_to_string(ast, fun, true) <> ")")
   end
 
+  # left when right
+  def to_string({ :when, _, [left, right] } = ast, fun) do
+    if right != [] and Keyword.keyword?(right) do
+      right = kw_list_to_string(right, fun)
+    else
+      right = fun.(ast, op_to_string(right, fun, :when, :right))
+    end
+
+    fun.(ast, op_to_string(left, fun, :when, :left) <> " when " <> right)
+  end
+
   # Binary ops
-  def to_string({ op, _, [left, right] } = ast, fun) when op in binary_ops do
+  def to_string({ op, _, [left, right] } = ast, fun) when op in @binary_ops do
     fun.(ast, op_to_string(left, fun, op, :left) <> " #{op} " <> op_to_string(right, fun, op, :right))
   end
 
@@ -293,23 +377,19 @@ defmodule Macro do
     fun.(ast, "not " <> to_string(arg, fun))
   end
 
-  def to_string({ op, _, [arg] } = ast, fun) when op in unary_ops do
-    fun.(ast, atom_to_binary(op, :utf8) <> to_string(arg, fun))
+  def to_string({ op, _, [arg] } = ast, fun) when op in @unary_ops do
+    fun.(ast, atom_to_binary(op) <> to_string(arg, fun))
   end
 
   # Access
   def to_string({ { :., _, [Kernel, :access] }, _, [left, right] } = ast, fun) do
-    fun.(ast, if right != [] and Keyword.keyword?(right) do
-      to_string(left, fun) <> to_string(right, fun)
-    else
-      to_string(left, fun) <> "[" <> to_string(right, fun) <> "]"
-    end)
+    fun.(ast, to_string(left, fun) <> to_string(right, fun))
   end
 
   # All other calls
   def to_string({ target, _, args } = ast, fun) when is_list(args) do
     { list, last } = :elixir_utils.split_last(args)
-    fun.(ast, case is_kw_blocks?(last) do
+    fun.(ast, case kw_blocks?(last) do
       true  -> call_to_string_with_args(target, list, fun) <> kw_blocks_to_string(last, fun)
       false -> call_to_string_with_args(target, args, fun)
     end)
@@ -321,51 +401,58 @@ defmodule Macro do
   end
 
   # Lists
-  def to_string(list = ast, fun) when is_list(list) do
-    if Keyword.keyword?(list) do
-      fun.(ast, "[" <> kw_list_to_string(list, fun) <> "]")
-    else
-      to_string({ :[], [], list }, fun)
-    end
+  def to_string(list, fun) when is_list(list) do
+    fun.(list, cond do
+      list == [] ->
+        "[]"
+      :io_lib.printable_list(list) ->
+        "'" <> Inspect.BitString.escape(String.from_char_list!(list), ?') <> "'"
+      Keyword.keyword?(list) ->
+        "[" <> kw_list_to_string(list, fun) <> "]"
+      true ->
+        "[" <> Enum.map_join(list, ", ", &to_string(&1, fun)) <> "]"
+    end)
   end
 
   # All other structures
-  def to_string(other, fun), do: fun.(other, inspect(other, raw: true))
+  def to_string(other, fun), do: fun.(other, inspect(other, records: false))
 
   # Block keywords
-  defmacrop kw_keywords, do: [:do, :catch, :rescue, :after, :else]
+  @kw_keywords [:do, :catch, :rescue, :after, :else]
 
-  defp is_kw_blocks?([_|_] = kw) do
-    Enum.all?(kw, match?({x, _} when x in kw_keywords, &1))
+  defp kw_blocks?([_|_] = kw) do
+    Enum.all?(kw, &match?({x, _} when x in @kw_keywords, &1))
   end
-  defp is_kw_blocks?(_), do: false
+  defp kw_blocks?(_), do: false
 
-  defp module_to_string(atom, _fun) when is_atom(atom), do: inspect(atom, raw: true)
+  defp module_to_string(atom, _fun) when is_atom(atom), do: inspect(atom, records: false)
   defp module_to_string(other, fun), do: call_to_string(other, fun)
 
-  defp call_to_string(atom, _fun) when is_atom(atom), do: atom_to_binary(atom, :utf8)
+  defp call_to_string(atom, _fun) when is_atom(atom), do: atom_to_binary(atom)
   defp call_to_string({ :., _, [arg] }, fun),         do: module_to_string(arg, fun) <> "."
   defp call_to_string({ :., _, [left, right] }, fun), do: module_to_string(left, fun) <> "." <> call_to_string(right, fun)
   defp call_to_string(other, fun),                    do: to_string(other, fun)
 
   defp call_to_string_with_args(target, args, fun) do
-    { list, last } = :elixir_utils.split_last(args)
     target = call_to_string(target, fun)
+    args = args_to_string(args, fun)
+    target <> "(" <> args <> ")"
+  end
 
-    case last != [] and Keyword.keyword?(last) do
-      true  ->
-        args = Enum.map_join(list, ", ", &to_string(&1, fun))
-        if list != [], do: args = args <> ", "
-        args = args <> kw_list_to_string(last, fun)
-        target <> "(" <> args <> ")"
-      false ->
-        args = Enum.map_join(args, ", ", &to_string(&1, fun))
-        target <> "(" <> args <> ")"
+  defp args_to_string(args, fun) do
+    { list, last } = :elixir_utils.split_last(args)
+
+    if last != [] and Keyword.keyword?(last) do
+      args = Enum.map_join(list, ", ", &to_string(&1, fun))
+      if list != [], do: args = args <> ", "
+      args <> kw_list_to_string(last, fun)
+    else
+      Enum.map_join(args, ", ", &to_string(&1, fun))
     end
   end
 
   defp kw_blocks_to_string(kw, fun) do
-    Enum.reduce(kw_keywords, " ", fn(x, acc) ->
+    Enum.reduce(@kw_keywords, " ", fn(x, acc) ->
       case Keyword.has_key?(kw, x) do
         true  -> acc <> kw_block_to_string(x, Keyword.get(kw, x), fun)
         false -> acc
@@ -375,11 +462,11 @@ defmodule Macro do
 
   defp kw_block_to_string(key, value, fun) do
     block = adjust_new_lines block_to_string(value, fun), "\n  "
-    atom_to_binary(key, :utf8) <> "\n  " <> block <> "\n"
+    atom_to_binary(key) <> "\n  " <> block <> "\n"
   end
 
-  defp block_to_string({ :->, _, exprs }, fun) do
-    Enum.map_join(exprs, "\n", fn({ left, _, right }) ->
+  defp block_to_string([{ :->, _, _ }|_] = block, fun) do
+    Enum.map_join(block, "\n", fn({ :->, _, [left, right] }) ->
       left = comma_join_or_empty_paren(left, fun, false)
       left <> "->\n  " <> adjust_new_lines block_to_string(right, fun), "\n  "
     end)
@@ -401,7 +488,7 @@ defmodule Macro do
     "(" <> to_string(expr, fun) <> ")"
   end
 
-  defp op_to_string({ op, _, [_, _] } = expr, fun, parent_op, side) when op in binary_ops do
+  defp op_to_string({ op, _, [_, _] } = expr, fun, parent_op, side) when op in @binary_ops do
     { parent_assoc, parent_prec } = binary_op_props(parent_op)
     { _, prec }                   = binary_op_props(op)
     cond do
@@ -419,8 +506,8 @@ defmodule Macro do
 
   defp op_to_string(expr, fun, _, _), do: to_string(expr, fun)
 
-  defp arrow_to_string({ :->, _, pairs }, fun, paren // false) do
-    Enum.map_join(pairs, "; ", fn({ left, _, right }) ->
+  defp arrow_to_string(pairs, fun, paren // false) do
+    Enum.map_join(pairs, "; ", fn({ :->, _, [left, right] }) ->
       left = comma_join_or_empty_paren(left, fun, paren)
       left <> "-> " <> to_string(right, fun)
     end)
@@ -443,18 +530,19 @@ defmodule Macro do
   end
 
   @doc """
-  Receives a AST node and expands it once. The following contents are expanded:
+  Receives an AST node and expands it once.
+
+  The following contents are expanded:
 
   * Macros (local or remote);
   * Aliases are expanded (if possible) and return atoms;
-  * All pseudo-variables (`__FILE__`, `__MODULE__`, etc);
+  * Pseudo-variables (`__ENV__`, `__MODULE__` and `__DIR__`);
   * Module attributes reader (`@foo`);
 
-  In case the expression cannot be expanded, it returns the expression
+  If the expression cannot be expanded, it returns the expression
   itself. Notice that `expand_once/2` performs the expansion just
   once and it is not recursive. Check `expand/2` for expansion
-  until the node no longer represents a macro and `expand_all/2`
-  for recursive expansion.
+  until the node can no longer be expanded.
 
   ## Examples
 
@@ -499,7 +587,7 @@ defmodule Macro do
       end
 
   The final module name will be `MyHelpers.Module` and not
-  `My.Module`. With `Macro.expand`, such aliases are taken
+  `My.Module`. With `Macro.expand/2`, such aliases are taken
   into consideration. Local and remote macros are also
   expanded. We could rewrite our macro above to use this
   function as:
@@ -518,192 +606,155 @@ defmodule Macro do
 
   """
   def expand_once(ast, env) do
-    expand_once(ast, env, nil) |> elem(0)
+    elem(do_expand_once(ast, env), 0)
   end
 
-  defp expand_once({ :__aliases__, _, _ } = original, env, cache) do
-    case :elixir_aliases.expand(original, env.aliases, env.macro_aliases) do
+  defp do_expand_once({ :__aliases__, _, _ } = original, env) do
+    case :elixir_aliases.expand(original, env.aliases, env.macro_aliases, env.lexical_tracker) do
       receiver when is_atom(receiver) ->
-        :elixir_tracker.record_alias(receiver, env.module)
-        { receiver, true, cache }
+        :elixir_lexical.record_remote(receiver, env.lexical_tracker)
+        { receiver, true }
       aliases ->
-        aliases = lc alias inlist aliases, do: (expand_once(alias, env, cache) |> elem(0))
+        aliases = lc alias inlist aliases, do: elem(do_expand_once(alias, env), 0)
 
-        case :lists.all(is_atom(&1), aliases) do
+        case :lists.all(&is_atom/1, aliases) do
           true ->
             receiver = :elixir_aliases.concat(aliases)
-            :elixir_tracker.record_alias(receiver, env.module)
-            { receiver, true, cache }
-          false -> { original, false, cache }
+            :elixir_lexical.record_remote(receiver, env.lexical_tracker)
+            { receiver, true }
+          false ->
+            { original, false }
         end
     end
   end
 
   # Expand @ calls
-  defp expand_once({ :@, _, [{ name, _, args }] } = original, env, cache) when is_atom(args) or args == [] do
+  defp do_expand_once({ :@, _, [{ name, _, args }] } = original, env) when is_atom(args) or args == [] do
     case (module = env.module) && Module.open?(module) do
-      true  -> { Module.get_attribute(module, name), true, cache }
-      false -> { original, false, cache }
+      true  -> { Module.get_attribute(module, name), true }
+      false -> { original, false }
     end
   end
 
   # Expand pseudo-variables
-  defp expand_once({ :__MODULE__, _, atom }, env, cache) when is_atom(atom),
-    do: { env.module, true, cache }
-  defp expand_once({ :__FILE__, _, atom }, env, cache)   when is_atom(atom),
-    do: { env.file, true, cache }
-  defp expand_once({ :__DIR__, _, atom }, env, cache)    when is_atom(atom),
-    do: { :filename.dirname(env.file), true, cache }
-  defp expand_once({ :__ENV__, _, atom }, env, cache)    when is_atom(atom),
-    do: { env, true, cache }
+  defp do_expand_once({ :__MODULE__, _, atom }, env) when is_atom(atom),
+    do: { env.module, true }
+  defp do_expand_once({ :__DIR__, _, atom }, env)    when is_atom(atom),
+    do: { :filename.dirname(env.file), true }
+
+  defp do_expand_once({ :__FILE__, _, atom }, env) when is_atom(atom) do
+    IO.write "__FILE__ is deprecated, please use __DIR__ or __ENV__.file instead\n#{Exception.format_stacktrace}"
+    { env.file, true }
+  end
+
+  defp do_expand_once({ :__ENV__, _, atom }, env)    when is_atom(atom),
+    do: { { :{}, [], tuple_to_list(env) }, true }
+  defp do_expand_once({ { :., _, [{ :__ENV__, _, atom }, field] }, _, [] } = original, env) when
+      is_atom(atom) and is_atom(field) do
+    case :erlang.function_exported(Macro.Env, field, 1) do
+      true  -> { apply(env, field, []), true }
+      false -> { original, false }
+    end
+  end
 
   # Expand possible macro import invocation
-  defp expand_once({ atom, line, args } = original, env, cache) when is_atom(atom) do
-    args = case is_atom(args) do
-      true  -> []
-      false -> args
+  defp do_expand_once({ atom, meta, context } = original, env)
+      when is_atom(atom) and is_list(meta) and is_atom(context) do
+    if :lists.member({ atom, Keyword.get(meta, :counter, context) }, env.vars) do
+      { original, false }
+    else
+      case do_expand_once({ atom, meta, [] }, env) do
+        { _, true } = exp -> exp
+        { _, false }      -> { original, false }
+      end
+    end
+  end
+
+  defp do_expand_once({ atom, meta, args } = original, env)
+      when is_atom(atom) and is_list(args) and is_list(meta) do
+    module = env.module
+
+    extra  = if function_exported?(module, :__info__, 1) do
+      [{ module, module.__info__(:macros) }]
+    else
+      []
     end
 
-    case not is_partial?(args) do
-      false -> { original, false, cache }
-      true  ->
-        module = env.module
+    arity = length(args)
 
-        extra  = if function_exported?(module, :__info__, 1) do
-          [{ module, module.__info__(:macros) }]
-        else
-          []
-        end
+    if :elixir_import.special_form(atom, arity) do
+      { original, false }
+    else
+      expand = :elixir_dispatch.expand_import(meta, { atom, length(args) }, args,
+                                              :elixir_env.ex_to_env(env), extra)
 
-        cache  = to_erl_env(env, cache)
-        expand = :elixir_dispatch.expand_import(line, { atom, length(args) }, args,
-          env.module, extra, cache)
-
-        case expand do
-          { :ok, _, expanded } -> { expanded, true, cache }
-          { :error, _ }        -> { original, false, cache }
-        end
+      case expand do
+        { :ok, receiver, quoted } ->
+          next = :elixir_counter.next
+          { :elixir_quote.linify_with_context_counter(0, { receiver, next }, quoted), true }
+        { :ok, _receiver } ->
+          { original, false }
+        :error ->
+          { original, false }
+      end
     end
   end
 
   # Expand possible macro require invocation
-  defp expand_once({ { :., _, [left, right] }, line, args } = original, env, cache) when is_atom(right) do
-    { receiver, _, _ } = expand_once(left, env, cache)
+  defp do_expand_once({ { :., _, [left, right] }, meta, args } = original, env) when is_atom(right) do
+    { receiver, _ } = do_expand_once(left, env)
 
-    case is_atom(receiver) and not is_partial?(args) do
-      false -> { original, false, cache }
+    case is_atom(receiver) do
+      false -> { original, false }
       true  ->
-        cache  = to_erl_env(env, cache)
-        expand = :elixir_dispatch.expand_require(line, receiver, { right, length(args) },
-          args, env.module, cache)
+        expand = :elixir_dispatch.expand_require(meta, receiver, { right, length(args) },
+          args, :elixir_env.ex_to_env(env))
 
         case expand do
-          { :ok, _receiver, expanded } -> { expanded, true, cache }
-          { :error, _ }                -> { original, false, cache }
+          { :ok, receiver, quoted } ->
+            next = :elixir_counter.next
+            { :elixir_quote.linify_with_context_counter(0, { receiver, next }, quoted), true }
+          :error ->
+            { original, false }
         end
     end
   end
 
   # Anything else is just returned
-  defp expand_once(other, _env, cache), do: { other, false, cache }
-
-  defp to_erl_env(env, nil),    do: :elixir_scope.to_erl_env(env)
-  defp to_erl_env(_env, cache), do: cache
-
-  defp is_partial?(args) do
-    :lists.any(match?({ :&, _, [_] }, &1), args)
-  end
+  defp do_expand_once(other, _env), do: { other, false }
 
   @doc """
-  Receives a AST node and expands it until it no longer represents
-  a macro. Check `expand_once/2` for more information on how
-  expansion works and `expand_all/2` for recursive expansion.
+  Receives an AST node and expands it until it no longer represents
+  a macro.
+
+  Check `expand_once/2` for more information on how
+  expansion works.
   """
   def expand(tree, env) do
-    expand(tree, env, nil) |> elem(0)
+    expand_until({ tree, true }, env)
   end
 
-  @doc false # Used internally by Elixir
-  def expand(tree, env, cache) do
-    expand_until({ tree, true, cache }, env)
+  defp expand_until({ tree, true }, env) do
+    expand_until(do_expand_once(tree, env), env)
   end
 
-  defp expand_until({ tree, true, cache }, env) do
-    expand_until(expand_once(tree, env, cache), env)
-  end
-
-  defp expand_until({ tree, false, cache }, _env) do
-    { tree, cache }
-  end
-
-  @doc %S"""
-  Receives a AST node and expands it until it no longer represents
-  a macro. Then it expands all of its children recursively.
-
-  The documentation for `expand_once/2` goes into more detail
-  on how expansion works. Keep in mind that `expand_all/2` is
-  naive as it doesn't mutate the environment. Take this example:
-
-      quoted = quote do: __MODULE__
-      Macro.to_string Macro.expand_all(quoted, __ENV__)
-      #=> "nil"
-
-  The example above, works as expected. Outside of a module,
-  `__MODULE__` returns `nil`. Now consider this variation:
-
-      quoted = quote do
-        defmodule Foo, do: __MODULE__
-      end
-      Macro.to_string Macro.expand_all(quoted, __ENV__)
-      #=> "defmodule(Foo) do\n  nil\nend"
-
-  One would expect `__MODULE__` to be expanded to `Foo`
-  but that is not the case since all the expansion is done
-  based on the environment `__ENV__`. This behaviour is on
-  purpose, as the proper expansion would require passing through
-  the whole Elixir compiler by actually executing the code, which
-  is beyond the scope of this function.
-  """
-  def expand_all(tree, env) do
-    expand_all(tree, env, nil) |> elem(0)
-  end
-
-  @doc false # Used internally by Elixir
-  def expand_all(tree, env, cache) do
-    expand_all_until(expand(tree, env, cache), env)
-  end
-
-  defp expand_all_until({ { left, meta, right }, cache }, env) do
-    { left, cache }  = expand_all(left, env, cache)
-    { right, cache } = expand_all(right, env, cache)
-    { { left, meta, right }, cache }
-  end
-
-  defp expand_all_until({ { left, right }, cache }, env) do
-    { left, cache }  = expand_all(left, env, cache)
-    { right, cache } = expand_all(right, env, cache)
-    { { left, right }, cache }
-  end
-
-  defp expand_all_until({ list, cache }, env) when is_list(list) do
-    :lists.mapfoldl(expand_all(&1, env, &2), cache, list)
-  end
-
-  defp expand_all_until({ other, cache }, _env) do
-    { other, cache }
+  defp expand_until({ tree, false }, _env) do
+    tree
   end
 
   @doc """
-  Recurs the quoted expression checking if all sub-terms are
-  safe (i.e. they represent data structures and don't actually
-  evaluate code) and returns `:ok` unless a given term is unsafe,
+  Recursively traverses the quoted expression checking if all sub-terms are
+  safe.
+
+  Terms are considered safe if they represent data structures and don't actually
+  evaluate code. Returns `:ok` unless a given term is unsafe,
   which is returned as `{ :unsafe, term }`.
   """
   def safe_term(terms) do
     do_safe_term(terms) || :ok
   end
 
-  defp do_safe_term({ local, _, terms }) when local in [:{}, :[], :__aliases__] do
+  defp do_safe_term({ local, _, terms }) when local in [:{}, :__aliases__] do
     do_safe_term(terms)
   end
 
@@ -712,7 +763,7 @@ defmodule Macro do
   end
 
   defp do_safe_term({ left, right }), do: do_safe_term(left) || do_safe_term(right)
-  defp do_safe_term(terms) when is_list(terms),  do: Enum.find_value(terms, do_safe_term(&1))
+  defp do_safe_term(terms) when is_list(terms),  do: Enum.find_value(terms, &do_safe_term(&1))
   defp do_safe_term(terms) when is_tuple(terms), do: { :unsafe, terms }
   defp do_safe_term(_), do: nil
 end

lib/elixir/lib/macro/env.ex

@@ -12,36 +12,45 @@ defmodule Macro.Env do
   * `function` - a tuple as `{ atom, integer` }, where the first
     element is the function name and the seconds its arity. Returns
     `nil` if not inside a function
-  * `aliases` -  a list of two item tuples, where the first
-    item is the aliased name and the second the actual name
   * `context` - the context of the environment. It can be nil
     (default context), inside a guard or inside an assign
+  * `aliases` -  a list of two item tuples, where the first
+    item is the aliased name and the second the actual name
   * `requires` - the list of required modules
   * `functions` - a list of functions imported from each module
   * `macros` - a list of macros imported from each module
-  * `context_modules` - a list of modules defined in the current context
   * `macro_aliases` - a list of aliases defined inside the current macro
-  * `vars` - a list keeping all defined varaibles as { var, context }
+  * `context_modules` - a list of modules defined in the current context
+  * `vars` - a list keeping all defined variables as { var, context }
+  * `export_vars` - a list keeping all variables to be exported in a construct (may be nil)
+  * `lexical_tracker` - PID to the lexical tracker which is responsible to keep user info
+  * `local` - the module to expand local functions to
   """
 
   @type name_arity :: { atom, non_neg_integer }
   @type file :: binary
   @type line :: non_neg_integer
   @type aliases :: [{ module, module }]
+  @type macro_aliases :: [{ module, { integer, module } }]
   @type context :: :match | :guard | nil
   @type requires :: [module]
   @type functions :: [{ module, [name_arity] }]
   @type macros :: [{ module, [name_arity] }]
   @type context_modules :: [module]
-  @type vars :: [{ atom, atom }]
+  @type vars :: [{ atom, atom | non_neg_integer }]
+  @type export_vars :: vars | nil
+  @type lexical_tracker :: pid
+  @type local :: module | nil
 
-  fields = [:module, :file, :line, :function, :aliases, :context, :requires,
-            :functions, :macros, :context_modules, :macro_aliases, :vars]
+  fields = [:module, :file, :line, :function, :context, :requires, :aliases, :functions,
+            :macros, :macro_aliases, :context_modules, :vars, :export_vars, :lexical_tracker,
+            :local]
 
   types  = quote do: [module: module, file: file, line: line,
-    function: name_arity, aliases: aliases, requires: requires,
-    functions: functions, macros: macros, context_modules: context_modules,
-    macro_aliases: aliases, vars: vars]
+    function: name_arity, context: context, requires: requires, aliases: aliases,
+    functions: functions, macros: macros,  macro_aliases: aliases,
+    context_modules: context_modules, vars: vars, export_vars: export_vars,
+    lexical_tracker: lexical_tracker, local: local]
 
   Record.deffunctions(fields, __MODULE__)
   Record.deftypes(fields, types, __MODULE__)
@@ -55,13 +64,13 @@ defmodule Macro.Env do
   end
 
   @doc """
-  Returns wether the compilation environment is currently
+  Returns whether the compilation environment is currently
   inside a guard.
   """
   def in_guard?(record), do: context(record) == :guard
 
   @doc """
-  Returns wether the compilation environment is currently
+  Returns whether the compilation environment is currently
   inside a match clause.
   """
   def in_match?(record), do: context(record) == :match
@@ -72,7 +81,7 @@ defmodule Macro.Env do
   def stacktrace(record) do
     cond do
       nil?(record.module) ->
-        [{ :elixir_compiler, :__FILE__, 2, location(record) }]
+        [{ :elixir_compiler, :__FILE__, 1, location(record) }]
       nil?(record.function) ->
         [{ module(record), :__MODULE__, 0, location(record) }]
       true ->

lib/elixir/lib/module.ex

@@ -29,7 +29,7 @@ defmodule Module do
       When just a module is provided, the function is assumed to be
       `__after_compile__/2`.
 
-      **Example**
+      ### Example
 
           defmodule M do
             @after_compile __MODULE__
@@ -44,18 +44,20 @@ defmodule Module do
       A hook that will be invoked before the module is compiled.
 
       Accepts a module or a tuple `{ <module>, <function/macro atom> }`. The
-      function/macro must take one argument: the module environment. If it\'s a
+      function/macro must take one argument: the module environment. If it's a
       macro, its returned value will be injected at the end of the module definition
       before the compilation starts.
 
       When just a module is provided, the function/macro is assumed to be
       `__before_compile__/1`.
 
-      **Example**
+      Note: differently from `@after_compile`, the callback function/macro must
+      be placed in a separate module (because when the callback is invoked,
+      the current module does not yet exist).
 
-          defmodule M do
-            @before_compile __MODULE__
+      ### Example
 
+          defmodule A do
             defmacro __before_compile__(_env) do
               quote do
                 def hello, do: "world"
@@ -63,11 +65,15 @@ defmodule Module do
             end
           end
 
-  * `@behaviour`   (notice the british spelling)
+          defmodule B do
+            @before_compile A
+          end
+
+  * `@behaviour`   (notice the British spelling)
 
       Specify an OTP or user-defined behaviour.
 
-      **Example**
+      ### Example
 
           defmodule M do
             @behaviour gen_event
@@ -85,7 +91,7 @@ defmodule Module do
       See http://www.erlang.org/doc/man/compile.html for the list of supported
       options.
 
-      **Example**
+      ### Example
 
             defmodule M do
               @compile { :inline, myfun: 1 }
@@ -106,7 +112,7 @@ defmodule Module do
 
       Can be invoked more than once.
 
-      **Example**
+      ### Example
 
             defmodule M do
               @doc "Hello world"
@@ -129,7 +135,7 @@ defmodule Module do
 
       Accepts a string. Can be used more than once.
 
-      **Example**
+      ### Example
 
             defmodule M do
               @doc "Hello world"
@@ -147,7 +153,7 @@ defmodule Module do
       `@moduledoc false` will make the module invisible to the
       documentation extraction tools like ExDoc.
 
-      **Example**
+      ### Example
 
             defmodule M do
               @moduledoc """
@@ -177,13 +183,13 @@ defmodule Module do
       When just a module is provided, the function is assumed to be
       `__on_definition__/6`.
 
-      Note that you can\'t provide the current module to `@on_definition`
+      Note that you can't provide the current module to `@on_definition`
       because the hook function will not be defined in time. Finally, since
       the `on_definition` hook is executed inside the context of the defined
       function (i.e. `env.function` returns the current function), the hook
       can only be a function, not a macro.
 
-      **Example**
+      ### Example
 
             defmodule H do
               def on_def(_env, kind, name, args, guards, body) do
@@ -216,7 +222,7 @@ defmodule Module do
       must have arity 0 (no arguments) and has to return `:ok`, otherwise the
       loading of the module will be aborted.
 
-      **Example**
+      ### Example
 
             defmodule M do
               @on_load :load_check
@@ -238,7 +244,7 @@ defmodule Module do
 
       Specify the module version. Accepts any valid Elixir value.
 
-      **Example**
+      ### Example
 
             defmodule M do
               @vsn "1.0"
@@ -292,7 +298,7 @@ defmodule Module do
 
   In addition to the above, you may also pass to `__info__/1` any atom supported
   by Erlang's `module_info` function which also gets defined for each compiled
-  module. See http://erlang.org/doc/reference_manual/modules.html#id74571 for
+  module. See http://erlang.org/doc/reference_manual/modules.html#id69430 for
   more information.
   """
   def __info__(kind)
@@ -346,13 +352,16 @@ defmodule Module do
 
   def eval_quoted(module, quoted, binding, opts) do
     assert_not_compiled!(:eval_quoted, module)
-    :elixir_module.eval_quoted(module, quoted, binding, opts)
+    :elixir_def.reset_last(module)
+    { value, binding, _env, _scope } =
+      :elixir.eval_quoted quoted, binding, Keyword.put(opts, :module, module)
+    { value, binding }
   end
 
   @doc """
-  Creates a module with the given name and given by
+  Creates a module with the given name and defined by
   the given quoted expressions. The line where the module
-  is defined and its file can be given as options.
+  is defined and its file can be passed as options.
 
   ## Examples
 
@@ -365,7 +374,7 @@ defmodule Module do
 
       Hello.world #=> true
 
-  ## Differences with `defmodule`
+  ## Differences from `defmodule`
 
   `Module.create` works similarly to `defmodule` and
   return the same results. While one could also use
@@ -385,13 +394,11 @@ defmodule Module do
   end
 
   def create(module, quoted, opts) when is_atom(module) do
-    line = Keyword.get(opts, :line, 1)
-    :elixir_module.compile(line, module, quoted, [], :elixir.scope_for_eval(opts))
+    :elixir_module.compile(module, quoted, [], :elixir.env_for_eval(opts))
   end
 
   @doc """
-  Concatenates the list of aliases and returns a new alias.
-  It handles char lists, binaries and atoms.
+  Concatenates a list of aliases and returns a new alias.
 
   ## Examples
 
@@ -399,17 +406,15 @@ defmodule Module do
       Foo.Bar
       iex> Module.concat([Foo, "Bar"])
       Foo.Bar
-      iex> Module.concat([Foo, 'Bar'])
-      Foo.Bar
 
   """
+  @spec concat([binary | atom]) :: atom
   def concat(list) when is_list(list) do
     :elixir_aliases.concat(list)
   end
 
   @doc """
-  Concatenates the two given aliases and returns a new alias.
-  It handles char lists, binaries and atoms.
+  Concatenates two aliases and returns a new alias.
 
   ## Examples
 
@@ -417,18 +422,17 @@ defmodule Module do
       Foo.Bar
       iex> Module.concat(Foo, "Bar")
       Foo.Bar
-      iex> Module.concat(Foo, 'Bar')
-      Foo.Bar
 
   """
+  @spec concat(binary | atom, binary | atom) :: atom
   def concat(left, right) do
     :elixir_aliases.concat([left, right])
   end
 
   @doc """
-  Concatenates the list aliases and returns a new alias only
+  Concatenates a list of aliases and returns a new alias only
   if the alias was already referenced. If the alias was not
-  referenced yet, fails with ArgumentError.
+  referenced yet, fails with `ArgumentError`.
   It handles char lists, binaries and atoms.
 
   ## Examples
@@ -440,14 +444,15 @@ defmodule Module do
       List.Chars
 
   """
+  @spec safe_concat([binary | atom]) :: atom | no_return
   def safe_concat(list) when is_list(list) do
     :elixir_aliases.safe_concat(list)
   end
 
   @doc """
-  Concatenates the two aliases and returns a new alias only
+  Concatenates two aliases and returns a new alias only
   if the alias was already referenced. If the alias was not
-  referenced yet, fails with ArgumentError.
+  referenced yet, fails with `ArgumentError`.
   It handles char lists, binaries and atoms.
 
   ## Examples
@@ -459,6 +464,7 @@ defmodule Module do
       List.Chars
 
   """
+  @spec safe_concat(binary | atom, binary | atom) :: atom | no_return
   def safe_concat(left, right) do
     :elixir_aliases.safe_concat([left, right])
   end
@@ -477,10 +483,11 @@ defmodule Module do
   end
 
   @doc """
-  Attaches documentation to a given function. It expects
-  the module the function belongs to, the line (a non negative
-  integer), the kind (def or defmacro), a tuple representing
-  the function and its arity and the documentation, which should
+  Attaches documentation to a given function or type. It expects
+  the module the function/type belongs to, the line (a non negative
+  integer), the kind (`def` or `defmacro`), a tuple representing
+  the function and its arity, the function signature (the signature
+  should be omitted for types) and the documentation, which should
   be either a binary or a boolean.
 
   ## Examples
@@ -491,12 +498,14 @@ defmodule Module do
       end
 
   """
-  def add_doc(_module, _line, kind, _tuple, _signature, doc) when kind in [:defp, :defmacrop] do
+  def add_doc(module, line, kind, tuple, signature // [], doc)
+
+  def add_doc(_module, _line, kind, _tuple, _signature, doc) when kind in [:defp, :defmacrop, :typep] do
     if doc, do: { :error, :private_doc }, else: :ok
   end
 
   def add_doc(module, line, kind, tuple, signature, doc) when
-      kind in [:def, :defmacro] and (is_binary(doc) or is_boolean(doc) or doc == nil) do
+      kind in [:def, :defmacro, :type, :opaque] and (is_binary(doc) or is_boolean(doc) or doc == nil) do
     assert_not_compiled!(:add_doc, module)
     table = docs_table_for(module)
 
@@ -575,7 +584,7 @@ defmodule Module do
 
   @doc """
   Checks if the module defines the given function or macro.
-  Use `defines?/3` to assert for an specific type.
+  Use `defines?/3` to assert for a specific type.
 
   ## Examples
 
@@ -593,8 +602,8 @@ defmodule Module do
   end
 
   @doc """
-  Checks if the module defines a function or macro with the
-  given `kind`. `kind` can be either `:def`, `:defp`,
+  Checks if the module defines a function or macro of the
+  given `kind`. `kind` can be any of `:def`, `:defp`,
   `:defmacro` or `:defmacrop`.
 
   ## Examples
@@ -616,7 +625,7 @@ defmodule Module do
   end
 
   @doc """
-  Return all functions defined in the given module.
+  Return all functions defined in `module`.
 
   ## Examples
 
@@ -633,7 +642,7 @@ defmodule Module do
   end
 
   @doc """
-  Returns all functions defined in te given module according
+  Returns all functions defined in `module`, according
   to its kind.
 
   ## Examples
@@ -652,7 +661,7 @@ defmodule Module do
   end
 
   @doc """
-  Makes the given functions in the given module overridable.
+  Makes the given functions in `module` overridable.
   An overridable function is lazily defined, allowing a
   developer to customize it. See `Kernel.defoverridable` for
   more information and documentation.
@@ -667,7 +676,12 @@ defmodule Module do
           raise "Cannot make function #{name}/#{arity} overridable because it was not defined"
         clause ->
           :elixir_def.delete_definition(module, tuple)
-          neighbours = Module.DispatchTracker.yank(module, tuple)
+
+          neighbours = if loaded?(Module.LocalsTracker) do
+            Module.LocalsTracker.yank(module, tuple)
+          else
+            []
+          end
 
           old    = get_attribute(module, :__overridable)
           merged = :orddict.update(tuple, fn({ count, _, _, _ }) ->
@@ -680,10 +694,10 @@ defmodule Module do
   end
 
   @doc """
-  Returns true if the given tuple in module is marked as overridable.
+  Returns `true` if `tuple` in `module` is marked as overridable.
   """
   def overridable?(module, tuple) do
-    !! List.keyfind(get_attribute(module, :__overridable), tuple, 0)
+    !!List.keyfind(get_attribute(module, :__overridable), tuple, 0)
   end
 
   @doc """
@@ -733,7 +747,10 @@ defmodule Module do
 
   Notice the third argument is used to indicate if a warning
   should be emitted when the attribute was not previously defined.
-  This is true for `@foo` attributes but false for direct calls.
+  `warn` may also be a stacktrace, which will be used in the
+  warning. The default value for `warn` is false for direct calls
+  but the `@foo` macro sets it to the proper stacktrace automatically,
+  warning every time `@foo` is used but not set previously.
 
   ## Examples
 
@@ -747,6 +764,7 @@ defmodule Module do
       end
 
   """
+  @spec get_attribute(module, atom, warn :: boolean | [tuple]) :: term
   def get_attribute(module, key, warn // false) when is_atom(key) do
     assert_not_compiled!(:get_attribute, module)
     table = data_table_for(module)
@@ -756,18 +774,22 @@ defmodule Module do
       [] ->
         acc = :ets.lookup_element(table, :__acc_attributes, 2)
 
-        if :lists.member(key, acc) do
-          []
-        else
-          warn && :elixir_errors.warn "#{Exception.format_caller} undefined module attribute @#{key}, " <>
-            "please remove access to @#{key} or explicitly set it to nil before access\n"
-          nil
+        cond do
+          :lists.member(key, acc) ->
+            []
+          warn ->
+            stack = is_list(warn) and warn
+            :elixir_errors.warn "#{Exception.format_caller(stack)} undefined module attribute @#{key}, " <>
+              "please remove access to @#{key} or explicitly set it to nil before access\n"
+            nil
+          true ->
+            nil
         end
     end
   end
 
   @doc """
-  Deletes all attributes that matches the given key.
+  Deletes all attributes that match the given key.
 
   ## Examples
 
@@ -799,7 +821,7 @@ defmodule Module do
   * `:persist` - The attribute will be persisted in the Erlang
     Abstract Format. Useful when interfacing with Erlang libraries.
 
-  By default, both options are false.
+  By default, both options are `false`.
 
   ## Examples
 
@@ -924,4 +946,6 @@ defmodule Module do
       raise ArgumentError,
         message: "could not call #{fun} on module #{inspect module} because it was already compiled"
   end
+
+  defp loaded?(module), do: is_tuple :code.is_loaded(module)
 end

lib/elixir/lib/module/locals_tracker.ex

@@ -5,44 +5,33 @@
 # ## Implementation
 #
 # The implementation uses the digraph module to track
-# all dependencies. The graph starts with three main
-# vertices:
+# all dependencies. The graph starts with one main vertice:
 #
 # * `:local` - points to local functions
-# * `:import` - points to imported modules
-# * `:warn` - points to imported modules that should be warned
-# * `:remote` - points to remote modules
 #
-# Besides those, we have can the following vertices:
+# We also have can the following vertices:
 #
-# * `Module` - a module that was invoked via an import or remotely
+# * `Module` - a module that was invoked via an import
 # * `{ name, arity }` - a local function/arity pair
 # * `{ :import, name, arity }` - an invoked function/arity import
-# * `{ :remote, name, arity }` - an remotely invoked function/arity
 #
 # Each of those vertices can associate to other vertices
 # as described below:
 #
 # * `Module`
-#   * in neighbours: `:import`, `:remote`, `:warn`,
-#     `{ :import, name, arity }` and `{ :remote, name arity }`
-#   * out neighbours: `:warn`
+#   * in neighbours:  `{ :import, name, arity }`
 #
 # * `{ name, arity }`
 #   * in neighbours: `:local`, `{ name, arity }`
-#   * out neighbours: `{ :import, name, arity }` and `{ :remote, name arity }`
+#   * out neighbours: `{ :import, name, arity }`
 #
 # * `{ :import, name, arity }`
 #   * in neighbours: `{ name, arity }`
 #   * out neighbours: `Module`
 #
-# * `{ :remote, name, arity }`
-#   * in neighbours: `{ name, arity }`
-#   * out neighbours: `Module`
-#
 # Note that since this is required for bootstrap, we can't use
 # any of the `GenServer.Behaviour` conveniences.
-defmodule Module.DispatchTracker do
+defmodule Module.LocalsTracker do
   @moduledoc false
 
   @timeout  30_000
@@ -54,53 +43,9 @@ defmodule Module.DispatchTracker do
 
   @type local :: { name, arity }
   @type import :: { :import, name, arity }
-  @type remote :: { :remote, name, arity }
 
   # Public API
 
-  @doc """
-  Receives a dispatch or a module and returns all dispatches
-  that calls it.
-
-  In case the argument is a module, the response will be
-  made by import and remote dispatches.
-
-  In case the argument is another dispatch, the response
-  will be made by local dispatches.
-
-  This function is not recursive, so if A dispatches to
-  B which dispatches to C, A does not appear in the result,
-  only B.
-  """
-  @spec dispatches_to(ref, module) :: [import | remote]
-  @spec dispatches_to(ref, local | import | remote) :: [local]
-  def dispatches_to(ref, dispatch) do
-    d = :gen_server.call(to_pid(ref), :digraph, @timeout)
-    :digraph.in_neighbours(d, dispatch) |> only_tuples
-  end
-
-  @doc """
-  Receives a local and returns all dispatches from that local.
-
-  This function is not recursive, so if A dispatches to
-  B which dispatches to C, C does not appear in the result,
-  only B.
-  """
-  @spec dispatches_from(ref, local) :: [local | import | remote]
-  def dispatches_from(ref, { name, arity }) do
-    d = :gen_server.call(to_pid(ref), :digraph, @timeout)
-    :digraph.out_neighbours(d, { name, arity }) |> only_tuples
-  end
-
-  @doc """
-  Returns all the modules which were imported.
-  """
-  @spec imports(ref) :: [module]
-  def imports(ref) do
-    d = :gen_server.call(to_pid(ref), :digraph, @timeout)
-    :digraph.out_neighbours(d, :import)
-  end
-
   @doc """
   Returns all imported modules that had the given
   `{ name, arity }` invoked.
@@ -111,34 +56,6 @@ defmodule Module.DispatchTracker do
     :digraph.out_neighbours(d, { :import, name, arity })
   end
 
-  @doc """
-  Returns all the aliases defined in the given module.
-  """
-  @spec aliases(ref) :: [module]
-  def aliases(ref) do
-    d = :gen_server.call(to_pid(ref), :digraph, @timeout)
-    :digraph.out_neighbours(d, :alias)
-  end
-
-  @doc """
-  Returns all the modules which were remotely dispatched to.
-  """
-  @spec remotes(ref) :: [module]
-  def remotes(ref) do
-    d = :gen_server.call(to_pid(ref), :digraph, @timeout)
-    :digraph.out_neighbours(d, :remote)
-  end
-
-  @doc """
-  Returns all modules that had the given `{ name, arity }`
-  invoked remotely.
-  """
-  @spec remotes_with_dispatch(ref, name_arity) :: [module]
-  def remotes_with_dispatch(ref, { name, arity }) do
-    d = :gen_server.call(to_pid(ref), :digraph, @timeout)
-    :digraph.out_neighbours(d, { :remote, name, arity })
-  end
-
   @doc """
   Returns all locals that are reachable.
 
@@ -157,20 +74,16 @@ defmodule Module.DispatchTracker do
     neighbours = (lc { _, _ } = t inlist neighbours, do: t) |> :ordsets.from_list
     remaining  = :ordsets.subtract(neighbours, vertices)
     vertices   = :ordsets.union(neighbours, vertices)
-    :lists.foldl(reduce_reachable(d, &1, &2), vertices, remaining)
+    :lists.foldl(&reduce_reachable(d, &1, &2), vertices, remaining)
   end
 
   defp to_pid(pid) when is_pid(pid),  do: pid
   defp to_pid(mod) when is_atom(mod) do
     table = :elixir_module.data_table(mod)
-    [{ _, val }] = :ets.lookup(table, :__dispatch_tracker)
+    [{ _, val }] = :ets.lookup(table, :__locals_tracker)
     val
   end
 
-  defp only_tuples(list) do
-    lc x inlist list, is_tuple(x), do: x
-  end
-
   # Internal API
 
   # Starts the tracker and returns its pid.
@@ -206,50 +119,10 @@ defmodule Module.DispatchTracker do
     :gen_server.cast(pid, { :add_local, from, to })
   end
 
-  # Adds an alias.
-  @doc false
-  def add_alias(pid, module) when is_atom(module) do
-    :gen_server.cast(pid, { :add_alias, module })
-  end
-
-  # Adds a remote dispatch to the given target.
-  @doc false
-  def add_remote(pid, function, module, target) when is_atom(module) and is_tuple(target) do
-    :gen_server.cast(pid, { :add_external, :remote, function, module, target })
-  end
-
   # Adds a import dispatch to the given target.
   @doc false
   def add_import(pid, function, module, target) when is_atom(module) and is_tuple(target) do
-    :gen_server.cast(pid, { :add_external, :import, function, module, target })
-  end
-
-  # Associates a module with a warn. This adds the given
-  # module and associates it with the `:import` vertex
-  # permanently, even if warn is false.
-  @doc false
-  def add_warnable(pid, module, warn, line) when is_atom(module) and is_boolean(warn) do
-    :gen_server.cast(pid, { :add_warnable, module, warn, line })
-  end
-
-  # Collect all unused imports where warn has been set to true.
-  def collect_unused_imports(pid) do
-    d = :gen_server.call(pid, :digraph, @timeout)
-    warnable = :digraph.out_neighbours(d, :warn)
-
-    lc mod inlist warnable, not has_imports?(d, mod), line = get_warn_line(d, mod) do
-      { mod, line }
-    end
-  end
-
-  defp get_warn_line(d, mod) do
-    [edge] = :digraph.out_edges(d, mod)
-    { ^edge, ^mod, :warn, line } = :digraph.edge(d, edge)
-    line
-  end
-
-  defp has_imports?(d, mod) do
-    Enum.any?(:digraph.in_neighbours(d, mod), match?({ :import, _, _ }, &1))
+    :gen_server.cast(pid, { :add_import, function, module, target })
   end
 
   # Yanks a local node. Returns its in and out vertices in a tuple.
@@ -258,6 +131,8 @@ defmodule Module.DispatchTracker do
     :gen_server.call(to_pid(pid), { :yank, local }, @timeout)
   end
 
+  # Reattach a previously yanked node
+  @doc false
   def reattach(pid, kind, tuple, neighbours) do
     pid = to_pid(pid)
     add_definition(pid, kind, tuple)
@@ -282,7 +157,7 @@ defmodule Module.DispatchTracker do
   @doc false
   def collect_unused_locals(pid, private) do
     reachable = reachable(pid)
-    :lists.foldl(collect_unused_locals(&1, &2, reachable), [], private)
+    :lists.foldl(&collect_unused_locals(&1, &2, reachable), [], private)
   end
 
   defp collect_unused_locals({ tuple, kind, 0 }, acc, reachable) do
@@ -311,6 +186,16 @@ defmodule Module.DispatchTracker do
     end
   end
 
+  @doc false
+  def cache_env(pid, env) do
+    :gen_server.call(pid, { :cache_env, env }, @timeout)
+  end
+
+  @doc false
+  def get_cached_env(pid, ref) do
+    :gen_server.call(pid, { :get_cached_env, ref }, @timeout)
+  end
+
   # Stops the gen server
   @doc false
   def stop(pid) do
@@ -322,101 +207,92 @@ defmodule Module.DispatchTracker do
   def init([]) do
     d = :digraph.new([:protected])
     :digraph.add_vertex(d, :local)
-    :digraph.add_vertex(d, :alias)
-    :digraph.add_vertex(d, :import)
-    :digraph.add_vertex(d, :remote)
-    :digraph.add_vertex(d, :warn)
-    { :ok, d }
+    { :ok, { d, [] } }
   end
 
-  def handle_call({ :yank, local }, _from, d) do
+  def handle_call({ :cache_env, env }, _from, { d, cache }) do
+    case cache do
+      [{i,^env}|_] ->
+        { :reply, i, { d, cache } }
+      t ->
+        i = length(t)
+        { :reply, i, { d, [{i,env}|t] } }
+    end
+  end
+
+  def handle_call({ :get_cached_env, ref }, _from, { _, cache } = state) do
+    { ^ref, env } = :lists.keyfind(ref, 1, cache)
+    { :reply, env, state }
+  end
+
+  def handle_call({ :yank, local }, _from, { d, _ } = state) do
     in_vertices  = :digraph.in_neighbours(d, local)
     out_vertices = :digraph.out_neighbours(d, local)
     :digraph.del_vertex(d, local)
-    { :reply, { in_vertices, out_vertices }, d }
+    { :reply, { in_vertices, out_vertices }, state }
   end
 
-  def handle_call(:digraph, _from, d) do
-    { :reply, d, d }
+  def handle_call(:digraph, _from, { d, _ } = state) do
+    { :reply, d, state }
   end
 
-  def handle_call(_request, _from, d) do
-    { :noreply, d }
+  def handle_call(request, _from, state) do
+    { :stop, { :bad_call, request }, state }
   end
 
-  def handle_info(_msg, d) do
-    { :noreply, d }
+  def handle_info(_msg, state) do
+    { :noreply, state }
   end
 
-  def handle_cast({ :add_local, from, to }, d) do
+  def handle_cast({ :add_local, from, to }, { d, _ } = state) do
     handle_add_local(d, from, to)
-    { :noreply, d }
+    { :noreply, state }
   end
 
-  def handle_cast({ :add_alias, module }, d) do
-    :digraph.add_vertex(d, module)
-    replace_edge!(d, :alias, module)
-    { :noreply, d }
+  def handle_cast({ :add_import, function, module, { name, arity } }, { d, _ } = state) do
+    handle_import(d, function, module, name, arity)
+    { :noreply, state }
   end
 
-  def handle_cast({ :add_external, kind, function, module, { name, arity } }, d) do
-    handle_import_or_remote(d, kind, function, module, name, arity)
-    { :noreply, d }
-  end
-
-  def handle_cast({ :add_warnable, module, warn, line }, d) do
-    :digraph.add_vertex(d, module)
-    replace_edge!(d, :import, module)
-
-    if warn do
-      :digraph.add_edge(d, :warn, module, line)
-      :digraph.add_edge(d, module, :warn, line)
-    else
-      :digraph.del_path(d, :warn, module)
-    end
-    { :noreply, d }
-  end
-
-  def handle_cast({ :add_definition, kind, tuple }, d) do
+  def handle_cast({ :add_definition, kind, tuple }, { d, _ } = state) do
     handle_add_definition(d, kind, tuple)
-    { :noreply, d }
+    { :noreply, state }
   end
 
-  def handle_cast({ :add_defaults, kind, { name, arity }, defaults }, d) do
+  def handle_cast({ :add_defaults, kind, { name, arity }, defaults }, { d, _ } = state) do
     lc i inlist :lists.seq(arity - defaults, arity - 1) do
       handle_add_definition(d, kind, { name, i })
       handle_add_local(d, { name, i }, { name, i + 1 })
     end
-    { :noreply, d }
+    { :noreply, state }
   end
 
-  def handle_cast({ :reattach, tuple, { in_neigh, out_neigh } }, d) do
+  def handle_cast({ :reattach, tuple, { in_neigh, out_neigh } }, { d, _ } = state) do
     lc from inlist in_neigh, do: replace_edge(d, from, tuple)
     lc to inlist out_neigh,  do: replace_edge(d, tuple, to)
-    { :noreply, d }
+    { :noreply, state }
   end
 
-  def handle_cast(:stop, d) do
-    { :stop, :normal, d }
+  def handle_cast(:stop, state) do
+    { :stop, :normal, state }
   end
 
-  def handle_cast(_msg, d) do
-    { :noreply, d }
+  def handle_cast(msg, state) do
+    { :stop, { :bad_cast, msg }, state }
   end
 
-  def terminate(_reason, _d) do
+  def terminate(_reason, _state) do
     :ok
   end
 
-  def code_change(_old, d, _extra) do
-    { :ok, d }
+  def code_change(_old, state, _extra) do
+    { :ok, state }
   end
 
-  defp handle_import_or_remote(d, kind, function, module, name, arity) do
+  defp handle_import(d, function, module, name, arity) do
     :digraph.add_vertex(d, module)
-    replace_edge!(d, kind, module)
 
-    tuple = { kind, name, arity }
+    tuple = { :import, name, arity }
     :digraph.add_vertex(d, tuple)
     replace_edge!(d, tuple, module)
 

lib/elixir/lib/option_parser.ex

@@ -33,7 +33,7 @@ defmodule OptionParser do
 
   Extra information about switches can be given as arguments, too.
   This is useful when a switch must behave as a boolean
-  or if duplicated switches should be kept, overriden or accumulated.
+  or if duplicated switches should be kept, overridden or accumulated.
 
   The following types are supported:
 
@@ -156,12 +156,12 @@ defmodule OptionParser do
         :boolean in kinds ->
           { nil, value in [true, "true"] }
         :integer in kinds ->
-          case String.to_integer(value) do
+          case Integer.parse(value) do
             { value, "" } -> { nil, value }
             _ -> { option, value }
           end
         :float in kinds ->
-          case String.to_float(value) do
+          case Float.parse(value) do
             { value, "" } -> { nil, value }
             _ -> { option, value }
           end

lib/elixir/lib/process.ex

@@ -264,8 +264,8 @@ defmodule Process do
 
   @doc """
   Associates the name with a pid or a port identifier. name, which must
-  be an atom, can be used instead of the pid / port identifier in the
-  send operator (name <- message).
+  be an atom, can be used instead of the pid / port identifier with the
+  `Kernel.send/2` function.
 
   See http://www.erlang.org/doc/man/erlang.html#register-2 for more info.
   """
@@ -304,11 +304,11 @@ defmodule Process do
   end
 
   @doc """
-  Sets the group leader of Pid to GroupLeader. Typically, this is used when a processes
+  Sets the group leader of `pid` to `leader`. Typically, this is used when a processes
   started from a certain shell should have another group leader than `:init`.
   """
-  @spec group_leader(leader :: pid, pid) :: true
-  def group_leader(leader, pid) do
+  @spec group_leader(pid, leader :: pid) :: true
+  def group_leader(pid, leader) do
     :erlang.group_leader(leader, pid)
   end
 

lib/elixir/lib/protocol.ex

@@ -1,58 +1,90 @@
 defmodule Protocol do
   @moduledoc false
 
-  # We need to use :lists because Enum is not available yet
-  require :lists, as: L
-
-  @doc """
-  Handle `defprotocol`. It will define a function for each
-  protocol plus two extra functions:
-
-  * `__protocol__/1` - returns the protocol name when :name is given,
-                       and a keyword list with the protocol functions
-                       when :functions is given;
-
-  * `__impl_for__/1` - receives one argument and returns a module
-                       that implements the protocol for the given
-                       data type. If no implementation matches, returns nil;
-
-  * `__impl_for__!/1` - same as above but raises an error if an implementation is not found
-  """
+  # Callback for defprotocol.
+  @doc false
   def defprotocol(name, [do: block]) do
     quote do
       defmodule unquote(name) do
         # We don't allow function definition inside protocols
         import Kernel, except: [
-          defmacrop: 1, defmacrop: 2, defmacrop: 4,
-          defmacro: 1, defmacro: 2, defmacro: 4,
-          defp: 1, defp: 2, defp: 4,
-          def: 1, def: 2, def: 4
+          defmacrop: 1, defmacrop: 2, defmacro: 1, defmacro: 2,
+          defp: 1, defp: 2, def: 1, def: 2
         ]
 
         # Import the new dsl that holds the new def
         import Protocol.DSL, only: :macros
 
+        # Compile with debug info for consolidation
+        @compile :debug_info
+
         # Set up a clear slate to store defined functions
         @functions []
-        @only nil
-        @except nil
+        @fallback_to_any false
 
         # Invoke the user given block
         unquote(block)
 
-        # Define callbacks and meta information
-        { conversions, fallback, returns_nil } = Protocol.conversions_for(__MODULE__, @only, @except)
-        Protocol.impl_for(conversions, fallback, returns_nil, __ENV__)
-        Protocol.meta(@functions, conversions, fallback, returns_nil, __ENV__)
+        # Finalize expansion
+        unquote(after_defprotocol)
       end
     end
   end
 
-  @doc """
-  Implement the given protocol for the given module.
-  It also defines a `__impl__` function which
-  returns the protocol being implemented.
-  """
+  defp after_defprotocol do
+    quote unquote: false do
+      { arg, bodies, rec } = Protocol.impl_for(__MODULE__)
+
+      @spec impl_for(term) :: module | nil
+      Kernel.def impl_for(data)
+
+      lc { guard, body } inlist bodies do
+        Kernel.def impl_for(unquote(arg)) when unquote(guard), do: unquote(body)
+      end
+
+      @spec impl_for!(term) :: module | no_return
+      Kernel.def impl_for!(data) do
+        impl_for(data) || raise(Protocol.UndefinedError, protocol: __MODULE__, value: data)
+      end
+
+      # Handle special Record type
+      Kernel.defp rec_impl_for(unquote(arg)), do: unquote(rec)
+
+      # Handle special Any type
+      if @fallback_to_any do
+        Kernel.defp any_impl_for do
+          try do
+            __MODULE__.Any.__impl__(:name)
+          catch
+            :error, :undef, [[{ __MODULE__.Any, :__impl__, [:name], _ }|_]|_] ->
+              nil
+          end
+        end
+      else
+        Kernel.defp any_impl_for, do: nil
+      end
+
+      # Inline both helpers
+      @compile { :inline, any_impl_for: 0, rec_impl_for: 1 }
+
+      if :code.ensure_loaded(Kernel.Typespec) == { :module, Kernel.Typespec } and
+         not Kernel.Typespec.defines_type?(__MODULE__, :t, 0) do
+        @type t :: term
+      end
+
+      # Store information as an attribute so it
+      # can be read without loading the module.
+      Module.register_attribute(__MODULE__, :protocol, persist: true)
+      @protocol [fallback_to_any: !!@fallback_to_any, consolidated: false]
+
+      @doc false
+      Kernel.def __protocol__(:name),      do: __MODULE__
+      Kernel.def __protocol__(:functions), do: unquote(:lists.sort(@functions))
+    end
+  end
+
+  # Callback for defimpl.
+  @doc false
   def defimpl(protocol, opts) do
     do_defimpl(protocol, :lists.keysort(1, opts))
   end
@@ -76,6 +108,10 @@ defmodule Protocol do
 
         unquote(block)
 
+        Module.register_attribute(__MODULE__, :impl, persist: true)
+        @impl [protocol: @protocol, for: @for]
+
+        @doc false
         def __impl__(:name),     do: __MODULE__
         def __impl__(:protocol), do: @protocol
         def __impl__(:for),      do: @for
@@ -100,203 +136,88 @@ defmodule Protocol do
     end
   end
 
-  # Implements the function that detects the protocol and returns
-  # the module to dispatch to. Returns module.Record for records
-  # which should be properly handled by the dispatching function.
+  # Builtin types.
   @doc false
-  def impl_for(conversions, fallback, returns_nil, env) do
-    contents = lc kind inlist conversions do
-      each_impl_for(kind, conversions, fallback)
-    end
-
-    if returns_nil do
-      contents = contents ++ [quote do
-        defp __raw_impl__(_) do
-          nil
-        end
-      end]
-    end
-
-    Module.eval_quoted env.module, contents, [], env.location
+  def builtin do
+    [ Tuple, Atom, List, BitString, Integer, Float,
+      Function, PID, Port, Reference, Any ]
   end
 
-  # Defines meta information about the protocol and internal callbacks.
+  # Implements the function that detects the protocol and
+  # returns the module to dispatch to.
   @doc false
-  def meta(functions, conversions, fallback, returns_nil, env) do
-    any     = L.keyfind(Any, 1, conversions) != false
-    records = L.keyfind(Record, 1, conversions) != false
+  def impl_for(current) do
+    arg = quote(do: arg)
+    all = [Record|builtin]
 
-    meta = quote location: :keep do
-      unless Kernel.Typespec.defines_type?(__MODULE__, :t, 0) do
-        @type t :: unquote(generate_type(conversions, any))
-      end
-
-      @doc false
-      def __protocol__(:name),      do: __MODULE__
-      def __protocol__(:functions), do: unquote(:lists.sort(functions))
-    end
-
-    impl_for = cond do
-      records and fallback ->
-        quote location: :keep do
-          def __impl_for__(arg) do
-            case __raw_impl__(arg) do
-              __MODULE__.Record ->
-                target = Module.concat(__MODULE__, :erlang.element(1, arg))
-                try do
-                  target.__impl__
-                  target
-                catch
-                  :error, :undef, [[{ ^target, :__impl__, [], _ }|_]|_] ->
-                    unquote(fallback)
-                end
-              other ->
-                other
-            end
-          end
-        end
-      records ->
-        quote location: :keep do
-          def __impl_for__(arg) do
-            case __raw_impl__(arg) do
-              __MODULE__.Record ->
-                Module.concat(__MODULE__, :erlang.element(1, arg))
-              other ->
-                other
-            end
-          end
-        end
-      true ->
-        quote location: :keep do
-          def __impl_for__(arg), do: __raw_impl__(arg)
-        end
-    end
-
-    impl_bang = if returns_nil do
-      quote do
-        def __impl_for__!(arg) do
-          __impl_for__(arg) || raise(Protocol.UndefinedError, protocol: __MODULE__, value: arg)
-        end
-      end
-    else
-      quote do
-        def __impl_for__!(arg), do: __impl_for__(arg)
-      end
-    end
-
-    Module.eval_quoted env.module, [meta, impl_for, impl_bang], [], env.location
+    { arg,
+      lc(mod inlist all, do: impl_for(current, mod, arg)),
+      rec_impl_for(current, arg) }
   end
 
-  # Returns the default conversions according to the given
-  # only/except options.
-  @doc false
-  def conversions_for(module, only, except) do
-    kinds = all_types
+  defp rec_impl_for(current, arg) do
+    fallback = impl_for(current, Tuple, arg) |> elem(1)
 
-    conversions =
-      if only do
-        L.map(fn i -> L.keyfind(i, 1, kinds) end, only)
-      else
-        except = except || [Any]
-        L.foldl(fn i, list -> L.keydelete(i, 1, list) end, kinds, except)
-      end
-
-    fallback = cond do
-      L.keyfind(Tuple, 1, conversions) ->
-        Module.concat module, Tuple
-      L.keyfind(Any, 1, conversions) ->
-        Module.concat module, Any
-      true ->
-        nil
-    end
-
-    # If any is not in the list and we don't implement all
-    # protocols, we need to handle nil cases.
-    returns_nil = L.keyfind(Any, 1, conversions) == false and length(conversions) < 10
-    { conversions, fallback, returns_nil }
-  end
-
-  ## Helpers
-
-  defp generate_type(_conversions, true) do
-    quote(do: any)
-  end
-
-  defp generate_type(conversions, false) do
-    or_function     = fn({ _, _, x }, acc) -> { :|, [], [acc, x] } end
-    { _, _, first } = hd(conversions)
-    :lists.foldl(or_function, first, tl(conversions))
-  end
-
-  defp all_types do
-    [ { Record,    :is_record,    quote do: tuple },
-      { Tuple,     :is_tuple,     quote do: tuple },
-      { Atom,      :is_atom,      quote do: atom },
-      { List,      :is_list,      quote do: list },
-      { BitString, :is_bitstring, quote do: <<>> },
-      { Number,    :is_number,    quote do: number },
-      { Function,  :is_function,  quote do: (... -> any) },
-      { PID,       :is_pid,       quote do: pid },
-      { Port,      :is_port,      quote do: port },
-      { Reference, :is_reference, quote do: reference },
-      { Any,       :is_any,       quote do: any } ]
-  end
-
-  # Returns a quoted expression that allows to check
-  # if the first item in the tuple is a built-in or not.
-  defp is_builtin?([{h, _, _}]) do
     quote do
-      first == unquote(h)
-    end
-  end
-
-  defp is_builtin?([{h, _, _}|t]) do
-    quote do
-      first == unquote(h) or unquote(is_builtin?(t))
-    end
-  end
-
-  # We don't have a fallback, so we assume what was given
-  # as a record is indeed a record
-  defp each_impl_for({ _, :is_record, _ }, _conversions, nil) do
-    quote do
-      defp __raw_impl__(arg) when is_record(arg) do
-        __MODULE__.Record
+      target = Module.concat(unquote(current), unquote(arg))
+      try do
+        target.__impl__(:name)
+      catch
+        :error, :undef, [[{ ^target, :__impl__, [:name], _ }|_]|_] ->
+          unquote(fallback)
       end
     end
   end
 
-  # Specially handle records in the case we have fallbacks.
-  defp each_impl_for({ _, :is_record, _ }, conversions, fallback) do
-    quote do
-      defp __raw_impl__(arg) when is_record(arg) do
-        first = :erlang.element(1, arg)
-        case unquote(is_builtin?(conversions)) do
-          true  -> unquote(fallback)
-          false ->
-            case atom_to_list(first) do
-              'Elixir.' ++ _ -> __MODULE__.Record
-              _              -> unquote(fallback)
-            end
-        end
+  defp impl_for(current, Record, arg) do
+    fallback = impl_for(current, Tuple, arg) |> elem(1)
+
+    dispatch = quote do
+      atom = :erlang.element(1, unquote(arg))
+
+      case not(atom in unquote(builtin)) and match?('Elixir.' ++ _, atom_to_list(atom)) do
+        true  -> rec_impl_for(atom)
+        false -> unquote(fallback)
       end
     end
+
+    quote do
+      { is_record(unquote(arg)), unquote(dispatch) }
+    end
   end
 
-  # Special case any as we don't need to generate a guard.
-  defp each_impl_for({ _, :is_any, _ }, _, _) do
+  defp impl_for(current, Tuple, arg),     do: impl_with_fallback(Tuple, :is_tuple, current, arg)
+  defp impl_for(current, Atom, arg),      do: impl_with_fallback(Atom, :is_atom, current, arg)
+  defp impl_for(current, List, arg),      do: impl_with_fallback(List, :is_list, current, arg)
+  defp impl_for(current, BitString, arg), do: impl_with_fallback(BitString, :is_bitstring, current, arg)
+  defp impl_for(current, Integer, arg),   do: impl_with_fallback(Integer, :is_integer, current, arg)
+  defp impl_for(current, Float, arg),     do: impl_with_fallback(Float, :is_float, current, arg)
+  defp impl_for(current, Function, arg),  do: impl_with_fallback(Function, :is_function, current, arg)
+  defp impl_for(current, PID, arg),       do: impl_with_fallback(PID, :is_pid, current, arg)
+  defp impl_for(current, Port, arg),      do: impl_with_fallback(Port, :is_port, current, arg)
+  defp impl_for(current, Reference, arg), do: impl_with_fallback(Reference, :is_reference, current, arg)
+
+  defp impl_for(_current, Any, _arg) do
+    { true, quote(do: any_impl_for) }
+  end
+
+  # Defines an implementation with fallback to the given module.
+  defp impl_with_fallback(mod, guard, current, arg) do
     quote do
-      defp __raw_impl__(_) do
-        __MODULE__.Any
-      end
+      { :erlang.unquote(guard)(unquote(arg)),
+        unquote(with_fallback(Module.concat(current, mod))) }
     end
   end
 
-  # Generate all others protocols.
-  defp each_impl_for({ kind, fun, _ }, _, _) do
+  # Tries to dispatch to a given target, fallbacks to the
+  # given `fallback` implementation if the target does not exist.
+  defp with_fallback(target) when is_atom(target) do
     quote do
-      defp __raw_impl__(arg) when unquote(fun)(arg) do
-        __MODULE__.unquote(kind)
+      try do
+        unquote(target).__impl__(:name)
+      catch
+        :error, :undef, [[{ unquote(target), :__impl__, [:name], _ }|_]|_] ->
+          any_impl_for
       end
     end
   end
@@ -305,108 +226,22 @@ end
 defmodule Protocol.DSL do
   @moduledoc false
 
-  # We need to use :lists because Enum is not available yet
-  require :lists, as: L
-
   @doc false
-  def args_and_body(module, name, arity) do
-    # Generate arguments according the arity. The arguments
-    # are named xa, xb and so forth. We cannot use string
-    # interpolation to generate the arguments because of compile
-    # dependencies, so we use the <<>> instead.
-    args = lc i inlist :lists.seq(1, arity) do
-      { binary_to_atom(<<?x, i + 64>>), [], __MODULE__ }
-    end
-
-    { conversions, fallback, returns_nil } = conversions_for(module)
-    clauses = [default_clause(name, args)]
-
-    if returns_nil do
-      clauses = [nil_clause()|clauses]
-    end
-
-    if L.keyfind(Record, 1, conversions) do
-      clauses = [record_clause(name, args, fallback)|clauses]
-    end
-
-    body =
-      quote do
-        case __raw_impl__(xA), do: unquote({ :->, [], clauses })
-      end
-
-    { args, body }
+  defmacro def({ _, _, args }) when args == [] or is_atom(args) do
+    raise ArgumentError, message: "protocol functions expect at least one argument"
   end
 
-  @doc false
-  def callback_from_spec(module, name, arity) do
-    tuple = { name, arity }
-    specs = Module.get_attribute(module, :spec)
-
-    found = lc { k, v } inlist specs, k == tuple do
-      Kernel.Typespec.define_callback(module, tuple, v)
-      true
-    end
-
-    found != []
-  end
-
-  defp conversions_for(module) do
-    only   = Module.get_attribute(module, :only)
-    except = Module.get_attribute(module, :except)
-    Protocol.conversions_for(module, only, except)
-  end
-
-  defp default_clause(name, args) do
-    { [quote do: other],
-      [],
-      quote(do: apply(other, unquote(name), [unquote_splicing(args)])) }
-  end
-
-  defp nil_clause() do
-    { [nil],
-      [],
-      quote(do: raise(Protocol.UndefinedError, protocol: __MODULE__, value: xA)) }
-  end
-
-  defp record_clause(name, args, nil) do
-    { [quote(do: __MODULE__.Record)],
-      [],
-      quote(do: Module.concat(__MODULE__, :erlang.element(1, xA)).unquote(name)(unquote_splicing(args))) }
-  end
-
-  defp record_clause(name, args, fallback) do
-    arity = length(args)
-
-    { [quote(do: __MODULE__.Record)],
-      [],
-      quote do
-        target = Module.concat(__MODULE__, :erlang.element(1, xA))
-        try do
-          target.unquote(name)(unquote_splicing(args))
-        catch
-          :error, :undef, [[{ ^target, name, args, _ }|_]|_] when
-              name == unquote(name) and length(args) == unquote(arity) ->
-            apply unquote(fallback), name, [unquote_splicing(args)]
-        end
-      end }
-  end
-
-  defmacro def(expression) do
-    case expression do
-      { _, _, args } when args == [] or is_atom(args) ->
-        raise ArgumentError, message: "protocol functions expect at least one argument"
-      { name, _, args } when is_atom(name) and is_list(args) ->
-        :ok
-      _ ->
-        raise ArgumentError, message: "invalid args for defprotocol"
-    end
-
+  defmacro def({ name, _, args }) when is_atom(name) and is_list(args) do
     arity = length(args)
 
     type_args = lc _ inlist :lists.seq(2, arity), do: quote(do: term)
     type_args = [quote(do: t) | type_args]
 
-    meta = quote do
+    call_args = lc i inlist :lists.seq(2, arity),
+                  do: { binary_to_atom(<<?x, i + 64>>), [], __MODULE__ }
+    call_args = [quote(do: t) | call_args]
+
+    quote do
       name  = unquote(name)
       arity = unquote(arity)
 
@@ -416,17 +251,37 @@ defmodule Protocol.DSL do
       # signature that will be used by docs
       Kernel.def unquote(name)(unquote_splicing(args))
 
+      # Generate the actual implementation
+      Kernel.def unquote(name)(unquote_splicing(call_args)) do
+        impl_for!(t).unquote(name)(unquote_splicing(call_args))
+      end
+
       # Convert the spec to callback if possible,
       # otherwise generate a dummy callback
-      Protocol.DSL.callback_from_spec(__MODULE__, name, arity) ||
+      Protocol.DSL.__spec__?(__MODULE__, name, arity) ||
         @callback unquote(name)(unquote_splicing(type_args)) :: term
     end
+  end
 
-    impl = quote bind_quoted: [name: name, arity: arity] do
-      { args, body } = Protocol.DSL.args_and_body(__MODULE__, name, arity)
-      Kernel.def unquote(name)(unquote_splicing(args)), do: unquote(body)
+  defmacro def(_) do
+    raise ArgumentError, message: "invalid args for def inside defprotocol"
+  end
+
+  @doc false
+  def __spec__?(module, name, arity) do
+    case :code.ensure_loaded(Kernel.Typespec) do
+      { :module, Kernel.Typespec } ->
+        tuple = { name, arity }
+        specs = Module.get_attribute(module, :spec)
+
+        found = lc { k, v } inlist specs, k == tuple do
+          Kernel.Typespec.define_callback(module, tuple, v)
+          true
+        end
+
+        found != []
+      { :error, _ } ->
+        true
     end
-
-    [meta, impl]
   end
 end

lib/elixir/lib/protocol/consolidation.ex

@@ -0,0 +1,266 @@
+defmodule Protocol.Consolidation do
+  @moduledoc """
+  Module responsible for consolidating protocols and helpers for
+  extracting protocols and implementations from code paths for
+  consolidation.
+  """
+
+  @doc """
+  Extract all protocols from the given paths.
+
+  The paths can be either a char list or a string. Internally
+  they are worked on as char lists, so passing them as lists
+  avoid extra conversion.
+
+  ## Examples
+
+      # Get Elixir's ebin and retrieve all protocols
+      iex> path = :code.lib_dir(:elixir, :ebin)
+      iex> mods = Protocol.Consolidation.extract_protocols([path])
+      iex> Enumerable in mods
+      true
+
+  """
+  @spec extract_protocols([char_list | String.t]) :: [atom]
+  def extract_protocols(paths) do
+    extract_matching_by_attribute paths, 'Elixir.',
+      fn module, attributes ->
+        case attributes[:protocol] do
+          [fallback_to_any: _, consolidated: _] -> module
+          _ -> nil
+        end
+      end
+  end
+
+  @doc """
+  Extract all types implemented for the given protocol from
+  the given paths.
+
+  The paths can be either a char list or a string. Internally
+  they are worked on as char lists, so passing them as lists
+  avoid extra conversion.
+
+  ## Examples
+
+      # Get Elixir's ebin and retrieve all protocols
+      iex> path = :code.lib_dir(:elixir, :ebin)
+      iex> mods = Protocol.Consolidation.extract_impls(Enumerable, [path])
+      iex> List in mods
+      true
+
+  """
+  @spec extract_impls(module, [char_list | String.t]) :: [atom]
+  def extract_impls(protocol, paths) when is_atom(protocol) do
+    prefix = atom_to_list(protocol) ++ '.'
+    extract_matching_by_attribute paths, prefix, fn
+      _mod, attributes ->
+        case attributes[:impl] do
+          [protocol: ^protocol, for: for] -> for
+          _ -> nil
+        end
+    end
+  end
+
+  defp extract_matching_by_attribute(paths, prefix, callback) do
+    lc path inlist paths,
+       file inlist list_dir(path),
+       mod = extract_from_file(path, file, prefix, callback),
+       do: mod
+  end
+
+  defp list_dir(path) when is_list(path) do
+    case :file.list_dir(path) do
+      { :ok, files } -> files
+      _ -> []
+    end
+  end
+
+  defp list_dir(path), do: list_dir(to_char_list(path))
+
+  defp extract_from_file(path, file, prefix, callback) do
+    if :lists.prefix(prefix, file) and Path.extname(file) == '.beam' do
+      extract_from_beam(Path.join(path, file), callback)
+    end
+  end
+
+  defp extract_from_beam(file, callback) do
+    case :beam_lib.chunks(file, [:attributes]) do
+      {:ok, { module, [attributes: attributes] } } ->
+        callback.(module, attributes)
+       _ ->
+         nil
+    end
+  end
+
+  defmacrop if_ok(expr, call) do
+    quote do
+      case unquote(expr) do
+        { :ok, var } -> unquote(Macro.pipe(quote(do: var), call))
+        other -> other
+      end
+    end
+  end
+
+  @doc """
+  Receives a protocol and a list of implementations and
+  consolidates the given protocol. Consolidation happens
+  by changing the protocol `impl_for` in the abstract
+  format to have fast lookup rules.
+
+  It returns the updated version of the protocol bytecode.
+  A given bytecode or protocol implementation can be checked
+  to be consolidated or not by analyzing the protocol
+  attribute:
+
+      Enumerable.__info__(:attributes)[:protocol]
+
+  If the first element of the tuple is true, it means
+  the protocol was consolidated.
+
+  This function does not load the protocol at any point
+  nor loads the new bytecode for the compiled module.
+  """
+  @spec apply_to(module, [module]) ::
+    { :ok, binary } |
+    { :error, :not_a_protocol } |
+    { :error, :no_beam_info }
+  def apply_to(protocol, types) when is_atom(protocol) do
+    ensure_protocol(protocol)
+    |> if_ok(change_debug_info types)
+    |> if_ok(compile)
+  end
+
+  # Ensure the given module is loaded and is a protocol.
+  defp ensure_protocol(protocol) do
+    case :beam_lib.chunks(beam_file(protocol), [:abstract_code, :attributes]) do
+      { :ok, { ^protocol, [abstract_code: { _raw, abstract_code },
+                           attributes: attributes] } } ->
+        case attributes[:protocol] do
+          [fallback_to_any: any, consolidated: _] ->
+            { :ok, { protocol, any, abstract_code } }
+          _ ->
+            { :error, :not_a_protocol }
+        end
+      _ ->
+        { :error, :no_beam_info }
+    end
+  end
+
+  defp beam_file(module) when is_atom(module) do
+    case :code.which(module) do
+      :non_existing -> module
+      file -> file
+    end
+  end
+
+  # Change the debug information to the optimized
+  # impl_for/1 dispatch version.
+  defp change_debug_info({ protocol, any, code }, types) do
+    types   = if any, do: types, else: List.delete(types, Any)
+    records = types -- Protocol.builtin
+    builtin = Protocol.builtin -- (Protocol.builtin -- types)
+    builtin = if records != [], do: [Record|builtin], else: builtin
+    change_impl_for(code, protocol, builtin, records, false, [])
+  end
+
+  defp change_impl_for([{ :attribute, line, :protocol, _ }|t], protocol, builtin, records, _, acc) do
+    attr = [fallback_to_any: Any in builtin, consolidated: true]
+    change_impl_for(t, protocol, builtin, records, true,
+                    [{ :attribute, line, :protocol, attr }|acc])
+  end
+
+  defp change_impl_for([{ :function, line, :impl_for, 1, _ }|t], protocol, builtin, records, is_protocol, acc) do
+    clauses = lc type inlist builtin, do: clause_for(type, protocol, line)
+
+    unless Any in builtin do
+      clauses = clauses ++ [fallback_clause_for(nil, protocol, line)]
+    end
+
+    change_impl_for(t, protocol, builtin, records, is_protocol,
+                    [{ :function, line, :impl_for, 1, clauses }|acc])
+  end
+
+  defp change_impl_for([{ :function, line, :rec_impl_for, 1, _ }|t], protocol, builtin, records, is_protocol, acc) do
+    fallback = if Tuple in builtin, do: Module.concat(protocol, Tuple)
+    clauses  = lc type inlist records, do: record_clause_for(type, protocol, line)
+    clauses  = clauses ++ [fallback_clause_for(fallback, protocol, line)]
+
+    change_impl_for(t, protocol, builtin, records, is_protocol,
+                    [{ :function, line, :rec_impl_for, 1, clauses }|acc])
+  end
+
+  defp change_impl_for([h|t], protocol, info, types, is_protocol, acc) do
+    change_impl_for(t, protocol, info, types, is_protocol, [h|acc])
+  end
+
+  defp change_impl_for([], protocol, _info, _types, is_protocol, acc) do
+    if is_protocol do
+      { :ok, { protocol, Enum.reverse(acc) } }
+    else
+      { :error, :not_a_protocol }
+    end
+  end
+
+  defp clause_for(Tuple, protocol, line),     do: builtin_clause_for(Tuple, :is_tuple, protocol, line)
+  defp clause_for(Atom, protocol, line),      do: builtin_clause_for(Atom, :is_atom, protocol, line)
+  defp clause_for(List, protocol, line),      do: builtin_clause_for(List, :is_list, protocol, line)
+  defp clause_for(BitString, protocol, line), do: builtin_clause_for(BitString, :is_bitstring, protocol, line)
+  defp clause_for(Integer, protocol, line),   do: builtin_clause_for(Integer, :is_integer, protocol, line)
+  defp clause_for(Float, protocol, line),     do: builtin_clause_for(Float, :is_float, protocol, line)
+  defp clause_for(Function, protocol, line),  do: builtin_clause_for(Function, :is_function, protocol, line)
+  defp clause_for(PID, protocol, line),       do: builtin_clause_for(PID, :is_pid, protocol, line)
+  defp clause_for(Port, protocol, line),      do: builtin_clause_for(Port, :is_port, protocol, line)
+  defp clause_for(Reference, protocol, line), do: builtin_clause_for(Reference, :is_reference, protocol, line)
+
+  defp clause_for(Any, protocol, line) do
+    {:clause, line, [{:var, line, :_}], [],
+      [{ :atom, line, Module.concat(protocol, Any) }]}
+  end
+
+  defp clause_for(Record, _protocol, line) do
+    {:clause, line, [{:var, line, :x}],
+      [[{:op, line, :andalso,
+          {:call, line,
+            {:remote, line, {:atom, line, :erlang}, {:atom, line, :is_tuple}},
+            [{:var, line, :x}]},
+          {:call, line,
+            {:remote, line, {:atom, line, :erlang}, {:atom, line, :is_atom}},
+            [{:call, line,
+              {:remote, line, {:atom, line, :erlang}, {:atom, line, :element}},
+              [{:integer, line, 1}, {:var, line, :x}]
+            }]},
+      }]],
+      [{:call, line,
+          {:atom, line, :rec_impl_for},
+          [{:call, line,
+            {:remote, line, {:atom, line, :erlang}, {:atom, line, :element}},
+            [{:integer, line, 1}, {:var, line, :x}]}]}]}
+  end
+
+  defp builtin_clause_for(mod, guard, protocol, line) do
+    {:clause, line,
+      [{:var, line, :x}],
+      [[{:call, line,
+          {:remote, line, {:atom, line, :erlang}, {:atom, line, guard}},
+          [{:var, line, :x}],
+      }]],
+      [{:atom, line, Module.concat(protocol, mod)}]}
+  end
+
+  defp record_clause_for(other, protocol, line) do
+    {:clause, line, [{:atom, line, other}], [],
+      [{:atom, line, Module.concat(protocol, other)}]}
+  end
+
+  defp fallback_clause_for(value, _protocol, line) do
+    {:clause, line, [{:var, line, :_}], [],
+      [{ :atom, line, value }]}
+  end
+
+  # Finally compile the module and emit its bytecode.
+  defp compile({ protocol, code }) do
+    opts = if Code.compiler_options[:debug_info], do: [:debug_info], else: []
+    { :ok, ^protocol, binary, _warnings } = :compile.forms(code, [:return|opts])
+    { :ok, binary }
+  end
+end

lib/elixir/lib/range.ex

@@ -6,9 +6,9 @@ end
 
 defprotocol Range.Iterator do
   @doc """
-  Reduces the range based on the type of the first argument.
+  Returns the function that calculates the next item.
   """
-  def reduce(first, range, acc, fun)
+  def next(first, range)
 
   @doc """
   Count how many items are in the range.
@@ -17,56 +17,65 @@ defprotocol Range.Iterator do
 end
 
 defimpl Enumerable, for: Range do
-  def reduce(Range[first: first] = range, acc, fun) do
-    Range.Iterator.reduce(first, range, acc, fun)
+  def reduce(first .. last = range, acc, fun) do
+    reduce(first, last, acc, fun, Range.Iterator.next(first, range), last >= first)
   end
 
-  def member?(Range[first: first, last: last], value) do
-    value in first..last
+  defp reduce(_x, _y, { :halt, acc }, _fun, _next, _up) do
+    { :halted, acc }
   end
 
-  def count(Range[first: first] = range) do
-    Range.Iterator.count(first, range)
+  defp reduce(x, y, { :suspend, acc }, fun, next, up) do
+    { :suspended, acc, &reduce(x, y, &1, fun, next, up) }
+  end
+
+  defp reduce(x, y, { :cont, acc }, fun, next, true) when x <= y do
+    reduce(next.(x), y, fun.(x, acc), fun, next, true)
+  end
+
+  defp reduce(x, y, { :cont, acc }, fun, next, false) when x >= y do
+    reduce(next.(x), y, fun.(x, acc), fun, next, false)
+  end
+
+  defp reduce(_, _, { :cont, acc }, _fun, _next, _up) do
+    { :done, acc }
+  end
+
+  def member?(first .. last, value) do
+    if first <= last do
+      { :ok, first <= value and value <= last }
+    else
+      { :ok, last <= value and value <= first }
+    end
+  end
+
+  def count(first .. _ = range) do
+    { :ok, Range.Iterator.count(first, range) }
   end
 end
 
-defimpl Range.Iterator, for: Number do
-  def reduce(first, Range[last: last], acc, fun) when is_integer(first) and is_integer(last) do
-    reducer = if last >= first do
-      fn(acc, fun) -> do_reducer_up(first, last, acc, fun) end
+defimpl Range.Iterator, for: Integer do
+  def next(first, Range[last: last]) when is_integer(last) do
+    if last >= first do
+      &(&1 + 1)
     else
-      fn(acc, fun) -> do_reducer_down(first, last, acc, fun) end
+      &(&1 - 1)
     end
-    Enumerable.Function.reduce(reducer, acc, fun)
   end
 
-  defp do_reducer_up(counter, last, acc, _fun) when counter > last do
-    acc
-  end
-
-  defp do_reducer_up(counter, last, acc, fun) do
-    do_reducer_up(counter + 1, last, fun.(counter, acc), fun)
-  end
-
-  defp do_reducer_down(counter, last, acc, _fun) when counter < last do
-    acc
-  end
-
-  defp do_reducer_down(counter, last, acc, fun) do
-    do_reducer_down(counter - 1, last, fun.(counter, acc), fun)
-  end
-
-  def count(first, Range[last: last]) when is_integer(first) and is_integer(last) and last >= first do
-    last - first + 1
-  end
-
-  def count(first, Range[last: last]) when is_integer(first) and is_integer(last) do
-    first - last + 1
+  def count(first, Range[last: last]) when is_integer(last) do
+    if last >= first do
+      last - first + 1
+    else
+      first - last + 1
+    end
   end
 end
 
 defimpl Inspect, for: Range do
+  import Inspect.Algebra
+
   def inspect(Range[first: first, last: last], opts) do
-    Inspect.Algebra.concat [Kernel.inspect(first, opts), "..", Kernel.inspect(last, opts)]
+    concat [to_doc(first, opts), "..", to_doc(last, opts)]
   end
 end

lib/elixir/lib/record.ex

@@ -1,19 +1,19 @@
 defmodule Record do
   @moduledoc %S"""
-  Convenience functions for working with Records.
+  Functions for working with Records.
 
   A record is a tagged tuple which contains one or more elements
   where the first element is an atom. We can manually create a record
   by simply defining such a tuple:
 
-      iex> record = { User, "josé", 25 }
+      iex> record = { User, "José", 25 }
       iex> is_record(record, User)
       true
 
   However, manually constructing tuples can be quite error prone.
-  In case we need to add a new field to our User, it would require
+  If we need to add a new field to our User, it would require
   us to carefully change all places where the tuple is used.
-  Furthermore, as more and more items are added to the tuple, they
+  Furthermore, as more items are added to the tuple, they
   lose semantic value.
 
   This module solves these problems by allowing us to name each
@@ -38,7 +38,7 @@ defmodule Record do
   In the example above, `defrecordp` is going to generate a set of
   macros named `user` that allows us to create, update and match
   on a record. Our record is going to have two fields, a `name` with
-  default value of "José" and `age` of 25.
+  default value of "José" and `age` with default value 25.
 
   Let's see some examples:
 
@@ -75,10 +75,7 @@ defmodule Record do
   As the name says, `defrecordp` is useful when you don't want to
   expose the record definition. The `user` macro used above, for
   example, is only available inside the `User` module and nowhere else.
-  You can find more information in `Kernel.defrecordp/2` docs.
-
-  In general, though, records are used as part of a module's public
-  interface. For such use cases, Elixir provides record modules.
+  You can find more information in `Kernel.defrecordp/3` docs.
 
   ## defrecord
 
@@ -97,11 +94,11 @@ defmodule Record do
       user()        #=> User[name: "José", age: 25]
       user(age: 26) #=> User[name: "José", age: 26]
 
-  Notice that now, since the record definition is accessible, Elixir
+  Notice that now since the record definition is accessible, Elixir
   shows the record nicely formatted, no longer as a simple tuple. We
-  can get the raw formatting by passing `raw: true` to `inspect`:
+  can get the raw formatting by passing `records: false` to `inspect`:
 
-      inspect user(), raw: true
+      inspect user(), records: false
       { User, "José", 25 }
 
   Since working with external records is common, Elixir allows
@@ -112,9 +109,9 @@ defmodule Record do
       User[]        #=> User[name: "José", age: 25]
       User[age: 26] #=> User[name: "José", age: 26]
 
-  The macro name is replaced by the module name and the parenthesis
-  are replaced by brackets. When the shortcut syntax is used, there
-  is no need to import the record.
+  The macro name is replaced by the module name and the parentheses
+  are replaced by brackets. When this syntax is used, there is no
+  need to import the record.
 
   Before we sum up the differences between `defrecord` and
   `defrecordp`, there is one last functionality introduced by
@@ -123,12 +120,12 @@ defmodule Record do
   ## Runtime access
 
   All the functionality discussed above happens at compilation time.
-  This means that both `user(user, age: 26)` and `User[user, age: 26]`
-  will be transformed into a simple tuple operation at compile time.
+  This means that both `user(age: 26)` and `User[age: 26]` are expanded
+  into a tuple at compile time.
 
   However, there are some situations where we want to set or update
-  fields dynamically. `defrecord` (and `defrecord` only) supports
-  it out of the box:
+  fields dynamically. `defrecord` (and not `defrecordp` ) supports
+  this behaviour out of the box:
 
       defrecord User, name: "José", age: 25
 
@@ -138,14 +135,23 @@ defmodule Record do
       user.update(age: 26)
       #=> User[name: "Hello", age: 26]
 
-  All the calls above happen at runtime. It gives Elixir
-  records flexibility at the cost of performance since
-  there is more work happening at runtime.
+  All the calls above happen at runtime. It gives Elixir records
+  flexibility at the cost of performance since there is more work
+  happening at runtime.
 
-  To sum up, `defrecordp` should be used when you don't want
-  to expose the record information while `defrecord` should be used
-  whenever you want to share a record within your code or with other
-  libraries or whenever you need to dynamically set or update fields.
+  The above calls (`new` and `update`) can accept both atom and string
+  keys for field names, however not both at the same time. This feature
+  allows to "sanitize" untrusted dictionaries and initialize/update
+  records without using `Kernel.binary_to_existing_atom/1`.
+
+  To sum up, `defrecordp` should be used when you don't want to expose
+  the record information while `defrecord` should be used whenever you
+  want to share a record within your code or with other libraries or
+  whenever you need to dynamically set or update fields.
+
+  The standard library contains excellent examples of both use cases,
+  with [`HashDict`](HashDict.html) being implemented with `defrecordp`
+  and [`Range`](Range.html) with `defrecord`.
 
   You can learn more about records in the `Kernel.defrecord/3` docs. Now
   let's discuss the usefulness of combining records with protocols.
@@ -170,22 +176,30 @@ defmodule Record do
         end
       end
 
-  Now we can explicitly convert our WeekDate:
+  Now we can explicitly convert our `WeekDate`:
 
       to_string WeekDate[year: 2013, week: 26, week_day: 4]
       "2013-W26-4"
 
-  A protocol can be implemented for any record defined via `defrecord`.
+  A protocol can be implemented for any record, whether
+  generated with `defrecordp` or `defrecord`.
   """
 
   @type t :: tuple
 
   @doc """
-  Extract record information from an Erlang file and
-  return the fields as a list of tuples.
+  Extract record information from an Erlang file.
+
+  Returns the fields as a list of tuples.
 
   ## Examples
 
+      Record.extract(:file_info, from_lib: "kernel/include/file.hrl")
+      #=> [size: :undefined, type: :undefined, access: :undefined, atime: :undefined,
+           mtime: :undefined, ctime: :undefined, mode: :undefined, links: :undefined,
+           major_device: :undefined, minor_device: :undefined, inode: :undefined,
+           uid: :undefined, gid: :undefined]
+
       defrecord FileInfo, Record.extract(:file_info, from_lib: "kernel/include/file.hrl")
 
   """
@@ -194,8 +208,9 @@ defmodule Record do
   end
 
   @doc """
-  Imports a public record definition as a set of private macros,
-  as defined by `Kernel.defrecordp/2`. This is useful when one
+  Import a public record definition as a set of private macros.
+
+  Macros defined as in `Kernel.defrecordp/3`. This is useful when one
   desires to manipulate a record via a set of macros instead
   of the regular access syntax.
 
@@ -231,14 +246,14 @@ defmodule Record do
       unquoted_fields = unquote(fields)
 
       defmodule unquote(name) do
-        import Elixir.Record.DSL
+        import Record.DSL
 
         @record_fields []
         @record_types  []
 
-        Elixir.Record.deffunctions(unquoted_fields, __ENV__)
+        Record.deffunctions(unquoted_fields, __ENV__)
         value = unquote(block)
-        Elixir.Record.deftypes(@record_fields, @record_types, __ENV__)
+        Record.deftypes(@record_fields, @record_types, __ENV__)
         value
       end
     end
@@ -253,35 +268,58 @@ defmodule Record do
   defp record_check!(_), do: :ok
 
   @doc false
-  def defrecordp(name, tag, fields) when is_atom(name) and is_atom(tag) and is_list(fields) do
-    { fields, types, def_type } = recordp_split(fields, [], [], false)
-    type = binary_to_atom(atom_to_binary(name) <> "_t")
-    tag  = tag || name
+  def defrecordp(name, tag, fields) do
+    case recordp_split(fields, [], [], false) do
+      { :ok, fields, types, def_type } ->
+        types = Macro.escape(types)
 
-    quote do
-      Record.defmacros(unquote(name), unquote(fields), __ENV__, unquote(tag))
+        # bind_quoted isn't available when bootstrapping record
+        quoted = quote [unquote: false] do
+          Record.defmacros(name, fields, __ENV__, tag)
 
-      if unquote(def_type) do
-        @typep unquote(type)() :: { unquote(tag), unquote_splicing(types) }
-      end
+          if def_type do
+            type = binary_to_atom(atom_to_binary(name) <> "_t")
+            @typep unquote(type)() :: { unquote(tag || name), unquote_splicing(types) }
+          end
+        end
+
+        quote do
+          def_type = unquote(def_type)
+          fields = unquote(fields)
+          types = unquote(types)
+          tag = unquote(tag)
+          name = unquote(name)
+          unquote(quoted)
+        end
+
+      :error ->
+        quote do
+          name = unquote(name)
+          Record.defmacros(name, unquote(fields), __ENV__, unquote(tag))
+        end
     end
   end
 
   defp recordp_split([{ field, { :::, _, [default, type] }}|t], defaults, types, _) do
-    recordp_split t, [{ field, default }|defaults], [type|types], true
+    recordp_split(t, [{ field, default }|defaults], [type|types], true)
   end
 
   defp recordp_split([other|t], defaults, types, def_type) do
-    recordp_split t, [other|defaults], [quote(do: term)|types], def_type
+    recordp_split(t, [other|defaults], [quote(do: term)|types], def_type)
   end
 
   defp recordp_split([], defaults, types, def_type) do
-    { :lists.reverse(defaults), :lists.reverse(types), def_type }
+    { :ok, :lists.reverse(defaults), :lists.reverse(types), def_type }
+  end
+
+  defp recordp_split(_, _, _, _) do
+    :error
   end
 
   @doc """
-  Defines record functions skipping the module definition.
-  This is called directly by `defrecord`. It expects the record
+  Define record functions skipping the module definition.
+
+  This is called directly by `Kernel.defrecord/3`. It expects the record
   values, a set of options and the module environment.
 
   ## Examples
@@ -308,7 +346,7 @@ defmodule Record do
 
     contents = [quote(do: @record_fields unquote(escaped))|contents]
 
-    # Special case for bootstraping purposes
+    # Special case for bootstrapping purposes
     if env == Macro.Env do
       Module.eval_quoted(env, contents, [], [])
     else
@@ -317,7 +355,7 @@ defmodule Record do
   end
 
   @doc """
-  Defines types and specs for the record.
+  Define types and specs for the record.
   """
   def deftypes(values, types, env) do
     types  = types || []
@@ -331,16 +369,22 @@ defmodule Record do
       accessor_specs(values, 1, [])
     ]
 
-    if env == Macro.Env do
-      Module.eval_quoted(env, contents, [], [])
-    else
-      Module.eval_quoted(env.module, contents, [], env.location)
+    # We need to handle bootstraping
+    cond do
+      :code.ensure_loaded(Kernel.Typespec) != { :module, Kernel.Typespec } ->
+        nil
+      env == Macro.Env ->
+        Module.eval_quoted(env, contents, [], [])
+      true ->
+        Module.eval_quoted(env.module, contents, [], env.location)
     end
   end
 
   @doc """
-  Defines macros for manipulating records. This is called
-  directly by `defrecordp`. It expects the macro name, the
+  Define macros for manipulating records.
+
+  This is called
+  directly by `Kernel.defrecordp/3`. It expects the macro name, the
   record values and the environment.
 
   ## Examples
@@ -350,7 +394,9 @@ defmodule Record do
       end
 
   """
-  def defmacros(name, values, env, tag // nil) do
+  def defmacros(name, values, env, tag // nil)
+      when is_atom(name) and is_list(values) and is_atom(tag) do
+
     escaped = lc value inlist values do
       { key, value } = convert_value(value)
       { key, Macro.escape(value) }
@@ -390,7 +436,7 @@ defmodule Record do
   end
 
   # Store fields that can be optimized and that cannot be
-  # optimized as they are overriden
+  # optimized as they are overridden
   @doc false
   def __on_definition__(env, kind, name, args, _guards, _body) do
     tuple     = { name, length(args) }
@@ -414,7 +460,7 @@ defmodule Record do
   @doc false
   def access(atom, fields, keyword, caller) do
     unless is_keyword(keyword) do
-      raise ArgumentError, message: "expected contents inside brackets to be a Keyword"
+      raise ArgumentError, message: "expected contents inside brackets to be a keyword list, got: #{inspect keyword}"
     end
 
     in_match = caller.in_match?
@@ -441,10 +487,10 @@ defmodule Record do
 
     case remaining do
       [] ->
-        quote do: { unquote_splicing([atom|match]) }
+        { :{}, [], [atom|match] }
       _  ->
         keys = lc { key, _ } inlist remaining, do: key
-        raise ArgumentError, message: "record #{inspect atom} does not have the keys: #{inspect keys}"
+        raise ArgumentError, message: "record #{inspect atom} does not have the key: #{inspect hd(keys)}"
     end
   end
 
@@ -627,9 +673,8 @@ defmodule Record do
     # an ordered dict of options (opts) and it will try to fetch
     # the given key from the ordered dict, falling back to the
     # default value if one does not exist.
-    selective = lc { k, v } inlist values do
-      quote do: Keyword.get(opts, unquote(k), unquote(v))
-    end
+    atom_selective   = lc { k, v } inlist values, do: initialize_lookup(k, v)
+    string_selective = lc { k, v } inlist values, do: initialize_lookup(atom_to_binary(k), v)
 
     quote do
       @doc false
@@ -637,7 +682,17 @@ defmodule Record do
 
       @doc false
       def new([]), do: { __MODULE__, unquote_splicing(defaults) }
-      def new(opts) when is_list(opts), do: { __MODULE__, unquote_splicing(selective) }
+      def new([{key, _}|_] = opts) when is_atom(key), do: { __MODULE__, unquote_splicing(atom_selective) }
+      def new([{key, _}|_] = opts) when is_binary(key), do: { __MODULE__, unquote_splicing(string_selective) }
+    end
+  end
+
+  defp initialize_lookup(k, v) do
+    quote do
+      case :lists.keyfind(unquote(k), 1, opts) do
+        false -> unquote(v)
+        {_, v} -> v
+      end
     end
   end
 
@@ -658,7 +713,7 @@ defmodule Record do
     quote do
       @doc false
       def to_keywords(record) do
-        unquote(:orddict.from_list(sorted))
+        unquote(sorted)
       end
     end
   end
@@ -728,15 +783,14 @@ defmodule Record do
   # Define an updater method that receives a
   # keyword list and updates the record.
   defp updater(values) do
-    fields =
-      lc {key, _default} inlist values do
-        index = find_index(values, key, 1)
-        quote do
-          Keyword.get(keywords, unquote(key), elem(record, unquote(index)))
-        end
-      end
+    atom_fields =
+      lc {key, _default} inlist values, do: updater_lookup(key, key, values)
 
-    contents = quote do: { __MODULE__, unquote_splicing(fields) }
+    string_fields =
+      lc {key, _default} inlist values, do: updater_lookup(atom_to_binary(key), key, values)
+
+    atom_contents = quote do: { __MODULE__, unquote_splicing(atom_fields) }
+    string_contents = quote do: { __MODULE__, unquote_splicing(string_fields) }
 
     quote do
       @doc false
@@ -744,8 +798,22 @@ defmodule Record do
         record
       end
 
-      def update(keywords, record) do
-        unquote(contents)
+      def update([{key, _}|_] = keywords, record) when is_atom(key) do
+        unquote(atom_contents)
+      end
+      def update([{key, _}|_] = keywords, record) when is_binary(key) do
+        unquote(string_contents)
+      end
+    end
+  end
+
+  defp updater_lookup(k, key, values) do
+    index = find_index(values, key, 0)
+
+    quote do
+      case :lists.keyfind(unquote(k), 1, keywords) do
+        false -> :erlang.element(unquote(index + 2), record)
+        {_, value} -> value
       end
     end
   end
@@ -767,7 +835,7 @@ defmodule Record do
 
   defp core_specs(values) do
     types   = lc { _, _, spec } inlist values, do: spec
-    options = if values == [], do: [], else: [options_specs(values)]
+    options = lc { k, _, v } inlist values, do: { k, v }
 
     quote do
       unless Kernel.Typespec.defines_type?(__MODULE__, :t, 0) do
@@ -775,7 +843,7 @@ defmodule Record do
       end
 
       unless Kernel.Typespec.defines_type?(__MODULE__, :options, 0) do
-        @type options :: unquote(options)
+        @type options :: unquote(options) | [{String.t, term}]
       end
 
       @spec new :: t
@@ -788,12 +856,6 @@ defmodule Record do
     end
   end
 
-  defp options_specs([{ k, _, v }|t]) do
-    :lists.foldl fn { k, _, v }, acc ->
-      { :|, [], [{ k, v }, acc] }
-    end, { k, v }, t
-  end
-
   defp accessor_specs([{ :__exception__, _, _ }|t], 1, acc) do
     accessor_specs(t, 2, acc)
   end
@@ -814,7 +876,7 @@ defmodule Record do
 
   ## Helpers
 
-  defp is_keyword(list) when is_list(list), do: :lists.all(is_keyword_tuple(&1), list)
+  defp is_keyword(list) when is_list(list), do: :lists.all(&is_keyword_tuple/1, list)
   defp is_keyword(_), do: false
 
   defp is_keyword_tuple({ x, _ }) when is_atom(x), do: true
@@ -870,6 +932,7 @@ defmodule Record.DSL do
 
   @doc """
   Defines the type for each field in the record.
+
   Expects a keyword list.
   """
   defmacro record_type(opts) when is_list(opts) do

lib/elixir/lib/record/extractor.ex

@@ -6,10 +6,11 @@ defmodule Record.Extractor do
   def retrieve(name, from: file) when is_binary(file) do
     file = String.to_char_list!(file)
 
-    case :code.where_is_file(file) do
-      :non_existing -> realfile = file
-      realfile -> :ok
-    end
+    realfile =
+      case :code.where_is_file(file) do
+        :non_existing -> file
+        realfile -> realfile
+      end
 
     retrieve_record(name, realfile)
   end
@@ -19,7 +20,7 @@ defmodule Record.Extractor do
   def retrieve(name, from_lib: file) when is_binary(file) do
     [app|path] = :filename.split(String.to_char_list!(file))
 
-    case :code.lib_dir(app) do
+    case :code.lib_dir(list_to_atom(app)) do
       { :error, _ } ->
         raise ArgumentError, message: "lib file #{file} could not be found"
       libpath ->
@@ -88,4 +89,4 @@ defmodule Record.Extractor do
     { :value, record, _ } = :erl_eval.expr(record_ast, [])
     record
   end
-end
+end

lib/elixir/lib/regex.ex

@@ -1,22 +1,21 @@
 defmodule Regex do
   @moduledoc %S"""
-  Regular expressions for Elixir built on top of the re module
+  Regular expressions for Elixir built on top of the `re` module
   in the Erlang Standard Library. More information can be found
-  in the re documentation: http://www.erlang.org/doc/man/re.html
+  in the [`re` documentation](http://www.erlang.org/doc/man/re.html).
 
-  Regular expressions in Elixir can be created using Regex.compile!
-  or using the special form with `%r`:
+  Regular expressions in Elixir can be created using `Regex.compile!/2`
+  or using the special form with [`%r`](Kernel.html#sigil_r/2):
 
       # A simple regular expressions that matches foo anywhere in the string
       %r/foo/
 
-      # A regular expression with case insensitive options and handling for unicode chars
-      %r/foo/iu
+      # A regular expression with case insensitive options
+      %r/foo/i
 
-  The re module provides several options, the ones available in Elixir, followed by
+  The `re` module provides several options, the ones available in Elixir, followed by
   their shortcut in parenthesis, are:
 
-  * `unicode` (u) - enable unicode specific patterns like \p
   * `caseless` (i) - add case insensitivity
   * `dotall` (s) - causes dot to match newlines and also set newline to anycrlf.
     The new line setting can be overridden by setting `(*CR)` or `(*LF)` or
@@ -27,8 +26,8 @@ defmodule Regex do
     allow `#` to delimit comments
   * `firstline` (f) - forces the unanchored pattern to match before or at the first
     newline, though the matched text may continue over the newline
-  * `ungreedy` (r) - invert the "greediness" of the regexp
-  * `groups` (g) - compile with info about groups available
+  * `ungreedy` (r) - inverts the "greediness" of the regexp
+  * `groups` (g) - compiles with info about groups available
 
   The options not available are:
 
@@ -44,7 +43,7 @@ defmodule Regex do
   """
 
   defrecordp :regex, Regex, [:re_pattern, :source, :options, :groups]
-  @type t :: { Regex, term, binary, binary, [atom] }
+  @type t :: { Regex, term, binary, binary, [atom] | nil }
 
   defexception CompileError, message: "regex could not be compiled"
 
@@ -53,12 +52,20 @@ defmodule Regex do
 
   The given options can either be a binary with the characters
   representing the same regex options given to the `%r` sigil,
-  or a list of options, as expected by the Erlang `re` docs.
+  or a list of options, as expected by the [Erlang `re` docs](http://www.erlang.org/doc/man/re.html).
 
   It returns `{ :ok, regex }` in case of success,
   `{ :error, reason }` otherwise.
+
+  ## Examples
+
+      iex> Regex.compile("foo")
+      {:ok, %r"foo"}
+      iex> Regex.compile("*foo")
+      {:error, {'nothing to repeat', 0}}
+
   """
-  @spec compile(binary, binary | [term]) :: t
+  @spec compile(binary, binary | [term]) :: { :ok, t } | { :error, any }
   def compile(source, options // "")
 
   def compile(source, options) when is_binary(options) do
@@ -67,7 +74,9 @@ defmodule Regex do
         { :error, { :invalid_option, rest } }
 
       translated_options ->
-        compile(source, translated_options, options)
+        # Always use the unicode option, we don't have a latin1 legacy like
+        # Erlang.
+        compile(source, [:unicode|translated_options], options)
     end
   end
 
@@ -154,15 +163,15 @@ defmodule Regex do
 
   ## Examples
 
-      iex> Regex.captures(%r/c(?<foo>d)/g, "abcd")
+      iex> Regex.named_captures(%r/c(?<foo>d)/g, "abcd")
       [foo: "d"]
-      iex> Regex.captures(%r/a(?<foo>b)c(?<bar>d)/g, "abcd")
+      iex> Regex.named_captures(%r/a(?<foo>b)c(?<bar>d)/g, "abcd")
       [foo: "b", bar: "d"]
-      iex> Regex.captures(%r/a(?<foo>b)c(?<bar>d)/g, "efgh")
+      iex> Regex.named_captures(%r/a(?<foo>b)c(?<bar>d)/g, "efgh")
       nil
 
   """
-  def captures(regex(groups: groups) = regex, string, options // []) do
+  def named_captures(regex(groups: groups) = regex, string, options // []) do
     options = Keyword.put_new(options, :capture, :groups)
     results = run(regex, string, options)
     if results, do: Enum.zip(groups, results)
@@ -249,13 +258,14 @@ defmodule Regex do
 
     options = [{ :capture, captures, return }, :global]
     case :re.run(string, compiled, options) do
+      :match -> []
       :nomatch -> []
       { :match, results } -> results
     end
   end
 
   @doc """
-  Split the given target into the number of parts specified.
+  Splits the given target into the number of parts specified.
   If no number of parts is given, it defaults to `:infinity`.
 
   ## Examples
@@ -265,6 +275,10 @@ defmodule Regex do
       ["a","b-c"]
       iex> Regex.split(%r/-/, "abc")
       ["abc"]
+      iex> Regex.split(%r//, "abc")
+      ["a", "b", "c", ""]
+      iex> Regex.split(%r//, "abc", trim: true)
+      ["a", "b", "c"]
   """
 
   def split(regex, string, options // [])
@@ -354,7 +368,10 @@ defmodule Regex do
   defp return_for(element) when is_binary(element), do: :binary
   defp return_for(element) when is_list(element),   do: :list
 
-  defp translate_options(<<?u, t :: binary>>), do: [:unicode|translate_options(t)]
+  defp translate_options(<<?u, t :: binary>>) do
+    IO.write "The /u flag for regular expressions is no longer needed\n#{Exception.format_stacktrace}"
+    translate_options(t)
+  end
   defp translate_options(<<?i, t :: binary>>), do: [:caseless|translate_options(t)]
   defp translate_options(<<?x, t :: binary>>), do: [:extended|translate_options(t)]
   defp translate_options(<<?f, t :: binary>>), do: [:firstline|translate_options(t)]

lib/elixir/lib/set.ex

@@ -15,6 +15,21 @@ defmodule Set do
   For simplicity's sake, in the examples below every time
   `new` is used, it implies one of the module-specific
   calls like above.
+
+  ## Protocols
+
+  Sets are required to implement the `Enumerable` protocol,
+  allowing one to write:
+
+      Enum.each(set, fn k ->
+        IO.inspect k
+      end)
+
+  ## Match
+
+  Sets are required to implement all operations
+  using the match (`===`) operator. Any deviation from
+  this behaviour should be avoided and explicitly documented.
   """
 
   use Behaviour

lib/elixir/lib/stream/reducers.ex

@@ -0,0 +1,188 @@
+defmodule Stream.Reducers do
+  # Collection of reducers shared by Enum and Stream.
+  @moduledoc false
+
+  defmacro chunk(n, step, limit, f // nil) do
+    quote do
+      fn entry, acc(h, { buffer, count }, t) ->
+        buffer = [entry|buffer]
+        count  = count + 1
+
+        new =
+          if count >= unquote(limit) do
+            left = count - unquote(step)
+            { Enum.take(buffer, left), left }
+          else
+            { buffer, count }
+          end
+
+        if count == unquote(n) do
+          cont_with_acc(unquote(f), :lists.reverse(buffer), h, new, t)
+        else
+          { :cont, acc(h, new, t) }
+        end
+      end
+    end
+  end
+
+  defmacro chunk_by(callback, f // nil) do
+    quote do
+      fn
+        entry, acc(h, { buffer, value }, t) ->
+          new_value = unquote(callback).(entry)
+          if new_value == value do
+            { :cont, acc(h, { [entry|buffer], value }, t) }
+          else
+            cont_with_acc(unquote(f), :lists.reverse(buffer), h, { [entry], new_value }, t)
+          end
+        entry, acc(h, nil, t) ->
+          { :cont, acc(h, { [entry], unquote(callback).(entry) }, t) }
+      end
+    end
+  end
+
+  defmacro drop(f // nil) do
+    quote do
+      fn
+        _entry, acc(h, n, t) when n > 0 ->
+          { :cont, acc(h, n-1, t) }
+        entry, acc(h, n, t) ->
+          cont_with_acc(unquote(f), entry, h, n, t)
+      end
+    end
+  end
+
+  defmacro drop_while(callback, f // nil) do
+    quote do
+      fn entry, acc(h, bool, t) = orig ->
+        if bool and unquote(callback).(entry) do
+          { :cont, orig }
+        else
+          cont_with_acc(unquote(f), entry, h, false, t)
+        end
+      end
+    end
+  end
+
+  defmacro filter(callback, f // nil) do
+    quote do
+      fn(entry, acc) ->
+        if unquote(callback).(entry) do
+          cont(unquote(f), entry, acc)
+        else
+          { :cont, acc }
+        end
+      end
+    end
+  end
+
+  defmacro filter_map(filter, mapper, f // nil) do
+    quote do
+      fn(entry, acc) ->
+        if unquote(filter).(entry) do
+          cont(unquote(f), unquote(mapper).(entry), acc)
+        else
+          { :cont, acc }
+        end
+      end
+    end
+  end
+
+  defmacro map(callback, f // nil) do
+    quote do
+      fn(entry, acc) ->
+        cont(unquote(f), unquote(callback).(entry), acc)
+      end
+    end
+  end
+
+  defmacro reject(callback, f // nil) do
+    quote do
+      fn(entry, acc) ->
+        unless unquote(callback).(entry) do
+          cont(unquote(f), entry, acc)
+        else
+          { :cont, acc }
+        end
+      end
+    end
+  end
+
+  defmacro scan_2(callback, f // nil) do
+    quote do
+      fn
+        entry, acc(h, :first, t) ->
+          cont_with_acc(unquote(f), entry, h, { :ok, entry }, t)
+        entry, acc(h, { :ok, acc }, t) ->
+          value = unquote(callback).(entry, acc)
+          cont_with_acc(unquote(f), value, h, { :ok, value }, t)
+      end
+    end
+  end
+
+  defmacro scan_3(callback, f // nil) do
+    quote do
+      fn(entry, acc(h, acc, t)) ->
+        value = unquote(callback).(entry, acc)
+        cont_with_acc(unquote(f), value, h, value, t)
+      end
+    end
+  end
+
+  defmacro take(f // nil) do
+    quote do
+      fn(entry, acc(h, n, t) = orig) ->
+        if n >= 1 do
+          cont_with_acc(unquote(f), entry, h, n-1, t)
+        else
+          { :halt, orig }
+        end
+      end
+    end
+  end
+
+  defmacro take_every(nth, f // nil) do
+    quote do
+      fn
+        entry, acc(h, n, t) when n === :first
+                            when n === unquote(nth) ->
+          cont_with_acc(unquote(f), entry, h, 1, t)
+        entry, acc(h, n, t) ->
+          { :cont, acc(h, n+1, t) }
+      end
+    end
+  end
+
+  defmacro take_while(callback, f // nil) do
+    quote do
+      fn(entry, acc) ->
+        if unquote(callback).(entry) do
+          cont(unquote(f), entry, acc)
+        else
+          { :halt, acc }
+        end
+      end
+    end
+  end
+
+  defmacro uniq(callback, f // nil) do
+    quote do
+      fn(entry, acc(h, prev, t) = acc) ->
+        value = unquote(callback).(entry)
+        if :lists.member(value, prev) do
+          { :cont, acc }
+        else
+          cont_with_acc(unquote(f), entry, h, [value|prev], t)
+        end
+      end
+    end
+  end
+
+  defmacro with_index(f // nil) do
+    quote do
+      fn(entry, acc(h, counter, t)) ->
+        cont_with_acc(unquote(f), { entry, counter }, h, counter + 1, t)
+      end
+    end
+  end
+end

lib/elixir/lib/string.ex

@@ -1,3 +1,5 @@
+import Kernel, except: [length: 1]
+
 defmodule String do
   @moduledoc %S"""
   A String in Elixir is a UTF-8 encoded binary.
@@ -5,7 +7,7 @@ defmodule String do
   ## String and binary operations
 
   The functions in this module act according to the
-  Unicode Standard, version 6.2.0. For example,
+  Unicode Standard, version 6.3.0. For example,
   `capitalize/1`, `downcase/1`, `strip/1` are provided by this
   module.
 
@@ -16,7 +18,7 @@ defmodule String do
   * `Kernel.binary_part/3` - retrieves part of the binary
   * `Kernel.bit_size/1` and `Kernel.byte_size/1` - size related functions
   * `Kernel.is_bitstring/1` and `Kernel.is_binary/1` - type checking function
-  * Plus a number of conversion functions, like `Kernel.binary_to_atom/2`,
+  * Plus a number of conversion functions, like `Kernel.binary_to_atom/1`,
     `Kernel.binary_to_integer/2`, `Kernel.binary_to_term/1` and their inverses,
     like `Kernel.integer_to_binary/2`
 
@@ -30,10 +32,14 @@ defmodule String do
   For example, the character "é" is represented with two
   bytes:
 
-      iex> string = "é"
-      ...> byte_size(string)
+      iex> byte_size("é")
       2
 
+  However, this module returns the proper length:
+
+      iex> String.length("é")
+      1
+
   Furthermore, this module also presents the concept of
   graphemes, which are multiple characters that may be
   "perceived as a single character" by readers. For example,
@@ -43,6 +49,8 @@ defmodule String do
       iex> string = "\x{0065}\x{0301}"
       ...> byte_size(string)
       3
+      iex> String.length(string)
+      1
 
   Although the example above is made of two characters, it is
   perceived by users as one.
@@ -57,6 +65,11 @@ defmodule String do
   Standard, but does not contain any of the locale specific
   behaviour.
 
+  More information about graphemes can be found in the [Unicode
+  Standard Annex #29](http://www.unicode.org/reports/tr29/).
+  This current Elixir version implements Extended Grapheme Cluster
+  algorithm.
+
   ## Integer codepoints
 
   Although codepoints could be represented as integers, this
@@ -136,10 +149,10 @@ defmodule String do
   def printable?(<<?\a, t :: binary>>), do: printable?(t)
 
   def printable?(<<>>), do: true
-  def printable?(_),    do: false
+  def printable?(b) when is_binary(b), do: false
 
   @doc """
-  Splits a string on substrings at each Unicode whitespace
+  Divides a string into substrings at each Unicode whitespace
   occurrence with leading and trailing whitespace ignored.
 
   ## Examples
@@ -155,7 +168,7 @@ defmodule String do
   @spec split(t) :: [t]
   defdelegate split(binary), to: String.Unicode
 
-  @doc """
+  @doc %S"""
   Divides a string into substrings based on a pattern,
   returning a list of these substrings. The pattern can
   be a string, a list of strings or a regular expression.
@@ -168,6 +181,8 @@ defmodule String do
 
   ## Examples
 
+  Splitting with a string pattern:
+
       iex> String.split("a,b,c", ",")
       ["a", "b", "c"]
       iex> String.split("a,b,c", ",", global: false)
@@ -175,15 +190,30 @@ defmodule String do
       iex> String.split(" a b c ", " ", trim: true)
       ["a", "b", "c"]
 
+  A list of patterns:
+
       iex> String.split("1,2 3,4", [" ", ","])
       ["1", "2", "3", "4"]
 
+  A regular expression:
+
       iex> String.split("a,b,c", %r{,})
       ["a", "b", "c"]
       iex> String.split("a,b,c", %r{,}, global: false)
       ["a", "b,c"]
-      iex> String.split("a,b", %r{\\.})
-      ["a,b"]
+      iex> String.split(" a b c ", %r{\s}, trim: true)
+      ["a", "b", "c"]
+
+  Splitting on empty patterns returns codepoints:
+
+      iex> String.split("abc", %r{})
+      ["a", "b", "c", ""]
+      iex> String.split("abc", "")
+      ["a", "b", "c", ""]
+      iex> String.split("abc", "", trim: true)
+      ["a", "b", "c"]
+      iex> String.split("abc", "", global: false)
+      ["a", "bc"]
 
   """
   @spec split(t, t | [t] | Regex.t) :: [t]
@@ -192,6 +222,8 @@ defmodule String do
 
   def split("", _pattern, _options), do: [""]
 
+  def split(binary, "", options), do: split(binary, %r"", options)
+
   def split(binary, pattern, options) when is_regex(pattern) do
     Regex.split(pattern, binary, options)
   end
@@ -435,13 +467,13 @@ defmodule String do
   end
 
   defp do_justify(subject, len, padding, type) when is_integer(padding) do
-    subject_len = String.length(subject)
+    subject_len = length(subject)
 
     cond do
       subject_len >= len ->
         subject
       subject_len < len ->
-        fill = String.duplicate(<<padding :: utf8>>, len - subject_len)
+        fill = duplicate(<<padding :: utf8>>, len - subject_len)
 
         case type do
           :left  -> subject <> fill
@@ -465,7 +497,7 @@ defmodule String do
       "a-b,c"
 
   The pattern can also be a regex. In those cases, one can give `\N`
-  in the `replacement` string to access a specific catpure in the regex:
+  in the `replacement` string to access a specific capture in the regex:
 
       iex> String.replace("a,b,c", %r/,(.)/, ",\\1\\1")
       "a,bb,cc"
@@ -523,14 +555,14 @@ defmodule String do
   """
   @spec reverse(t) :: t
   def reverse(string) do
-    do_reverse(String.Unicode.next_grapheme(string), [])
+    do_reverse(next_grapheme(string), [])
   end
 
   defp do_reverse({grapheme, rest}, acc) do
-    do_reverse(String.Unicode.next_grapheme(rest), [grapheme|acc])
+    do_reverse(next_grapheme(rest), [grapheme|acc])
   end
 
-  defp do_reverse(:no_grapheme, acc), do: iolist_to_binary(acc)
+  defp do_reverse(nil, acc), do: iolist_to_binary(acc)
 
   @doc """
   Returns a binary `subject` duplicated `n` times.
@@ -570,7 +602,7 @@ defmodule String do
   Returns the next codepoint in a String.
 
   The result is a tuple with the codepoint and the
-  remaining of the string or `:no_codepoint` in case
+  remaining of the string or `nil` in case
   the string reached its end.
 
   As with other functions in the String module, this
@@ -584,7 +616,8 @@ defmodule String do
       { "j", "osé" }
 
   """
-  @spec next_codepoint(t) :: {codepoint, t} | :no_codepoint
+  @compile { :inline, next_codepoint: 1 }
+  @spec next_codepoint(t) :: {codepoint, t} | nil
   defdelegate next_codepoint(string), to: String.Unicode
 
   @doc %S"""
@@ -642,30 +675,10 @@ defmodule String do
   def valid_character?(<<_ :: utf8>> = codepoint), do: valid?(codepoint)
   def valid_character?(_), do: false
 
-  @doc %S"""
-  Checks whether `str` is a valid codepoint.
-
-  Note that the empty string is considered invalid, as are
-  strings containing multiple codepoints.
-
-  ## Examples
-
-      iex> String.valid_codepoint?("a")
-      true
-      iex> String.valid_codepoint?("ø")
-      true
-      iex> String.valid_codepoint?(<<0xffff :: 16>>)
-      false
-      iex> String.valid_codepoint?("asdf")
-      false
-
-  """
-  @spec valid_codepoint?(codepoint) :: boolean
-  def valid_codepoint?(<<_ :: utf8>>), do: true
-  def valid_codepoint?(_), do: false
-
   @doc """
-  Returns unicode graphemes in the string.
+  Returns unicode graphemes in the string as per Extended Grapheme
+  Cluster algorithm outlined in the [Unicode Standard Annex #29,
+  Unicode Text Segmentation](http://www.unicode.org/reports/tr29/).
 
   ## Examples
 
@@ -674,13 +687,13 @@ defmodule String do
 
   """
   @spec graphemes(t) :: [grapheme]
-  defdelegate graphemes(string), to: String.Unicode
+  defdelegate graphemes(string), to: String.Graphemes
 
   @doc """
   Returns the next grapheme in a String.
 
   The result is a tuple with the grapheme and the
-  remaining of the string or `:no_grapheme` in case
+  remaining of the string or `nil` in case
   the String reached its end.
 
   ## Examples
@@ -689,8 +702,9 @@ defmodule String do
       { "j", "osé" }
 
   """
-  @spec next_grapheme(t) :: { grapheme, t } | :no_grapheme
-  defdelegate next_grapheme(string), to: String.Unicode
+  @compile { :inline, next_grapheme: 1 }
+  @spec next_grapheme(t) :: { grapheme, t } | nil
+  defdelegate next_grapheme(string), to: String.Graphemes
 
   @doc """
   Returns the first grapheme from an utf8 string,
@@ -708,7 +722,7 @@ defmodule String do
   def first(string) do
     case next_grapheme(string) do
       { char, _ } -> char
-      :no_grapheme -> nil
+      nil -> nil
     end
   end
 
@@ -733,7 +747,7 @@ defmodule String do
     do_last(next_grapheme(rest), char)
   end
 
-  defp do_last(:no_grapheme, last_char), do: last_char
+  defp do_last(nil, last_char), do: last_char
 
   @doc """
   Returns the number of unicode graphemes in an utf8 string.
@@ -755,7 +769,7 @@ defmodule String do
     1 + do_length(next_grapheme(rest))
   end
 
-  defp do_length(:no_grapheme), do: 0
+  defp do_length(nil), do: 0
 
   @doc """
   Returns the grapheme in the `position` of the given utf8 `string`.
@@ -782,7 +796,7 @@ defmodule String do
   end
 
   def at(string, position) when position < 0 do
-    real_pos = do_length(next_grapheme(string)) - abs(position)
+    real_pos = length(string) - abs(position)
     case real_pos >= 0 do
       true  -> do_at(next_grapheme(string), real_pos, 0)
       false -> nil
@@ -797,7 +811,7 @@ defmodule String do
     char
   end
 
-  defp do_at(:no_grapheme, _, _), do: nil
+  defp do_at(nil, _, _), do: nil
 
   @doc """
   Returns a substring starting at the offset given by the first, and
@@ -827,7 +841,7 @@ defmodule String do
   @spec slice(t, integer, integer) :: grapheme | nil
 
   def slice(string, start, 0) do
-    case abs(start) <= String.length(string) do
+    case abs(start) <= length(string) do
       true -> ""
       false -> nil
     end
@@ -838,13 +852,72 @@ defmodule String do
   end
 
   def slice(string, start, len) when start < 0 and len >= 0 do
-    real_start_pos = do_length(next_grapheme(string)) - abs(start)
+    real_start_pos = length(string) - abs(start)
     case real_start_pos >= 0 do
       true -> do_slice(next_grapheme(string), real_start_pos, real_start_pos + len - 1, 0, "")
       false -> nil
     end
   end
 
+  @doc """
+  Returns a substring from the offset given by the start of the
+  range to the offset given by the end of the range.
+
+  If the start of the range is not a valid offset for the given
+  string or if the range is in reverse order, returns `nil`.
+
+  ## Examples
+
+      iex> String.slice("elixir", 1..3)
+      "lix"
+      iex> String.slice("elixir", 1..10)
+      "lixir"
+      iex> String.slice("elixir", 10..3)
+      nil
+
+      iex> String.slice("elixir", -4..-1)
+      "ixir"
+      iex> String.slice("elixir", 2..-1)
+      "ixir"
+      iex> String.slice("elixir", -4..6)
+      "ixir"
+      iex> String.slice("elixir", -1..-4)
+      nil
+      iex> String.slice("elixir", -10..-7)
+      nil
+
+      iex> String.slice("a", 0..1500)
+      "a"
+      iex> String.slice("a", 1..1500)
+      ""
+      iex> String.slice("a", 2..1500)
+      nil
+
+  """
+  @spec slice(t, Range.t) :: t | nil
+
+  def slice(string, range)
+
+  def slice(string, first..last) when first >= 0 and last >= 0 do
+    do_slice(next_grapheme(string), first, last, 0, "")
+  end
+
+  def slice(string, first..last) do
+    total = length(string)
+
+    if first < 0 do
+      first = total + first
+    end
+
+    if last < 0 do
+      last = total + last
+    end
+
+    if first > 0 do
+      do_slice(next_grapheme(string), first, last, 0, "")
+    end
+  end
+
   defp do_slice(_, start_pos, last_pos, _, _) when start_pos > last_pos do
     nil
   end
@@ -861,75 +934,17 @@ defmodule String do
     acc <> char
   end
 
-  defp do_slice(:no_grapheme, start_pos, _, current_pos, acc) when start_pos == current_pos do
+  defp do_slice(nil, start_pos, _, current_pos, acc) when start_pos == current_pos do
     acc
   end
 
-  defp do_slice(:no_grapheme, _, _, _, acc) do
+  defp do_slice(nil, _, _, _, acc) do
     case acc do
       "" -> nil
       _ -> acc
     end
   end
 
-  @doc """
-  Converts a string to an integer. If successful, returns a
-  tuple of the form `{integer, remainder of string}`. If unsuccessful,
-  returns `:error`.
-
-  ## Examples
-
-      iex> String.to_integer("34")
-      {34,""}
-      iex> String.to_integer("34.5")
-      {34,".5"}
-      iex> String.to_integer("three")
-      :error
-
-  """
-  @spec to_integer(t) :: {integer, t} | :error
-
-  def to_integer(string) do
-    {result, remainder} = :string.to_integer(:binary.bin_to_list(string))
-    case result do
-      :error -> :error
-      _ -> {result, :binary.list_to_bin(remainder)}
-    end
-  end
-
-  @doc """
-  Converts a string to a float. If successful, returns a
-  tuple of the form `{float, remainder of string}`.
-  If unsuccessful, returns `:error`.
-
-  ## Examples
-
-      iex> String.to_float("34")
-      {34.0,""}
-      iex> String.to_float("34.25")
-      {34.25,""}
-      iex> String.to_float("56.5xyz")
-      {56.5,"xyz"}
-      iex> String.to_float("pi")
-      :error
-
-  """
-  @spec to_float(t) :: {integer, t} | :error
-
-  def to_float(string) do
-    charlist = :binary.bin_to_list(string)
-    {result, remainder} = :string.to_float(charlist)
-    case result do
-      :error ->
-        {int_result, int_remainder} = :string.to_integer(charlist)
-        case int_result do
-          :error -> :error
-          _ -> {:erlang.float(int_result), :binary.list_to_bin(int_remainder)}
-        end
-      _ -> {result, :binary.list_to_bin(remainder)}
-    end
-  end
-
   @doc """
   Returns `true` if `string` starts with any of the prefixes given, otherwise
   `false`. `prefixes` can be either a single prefix or a list of prefixes.
@@ -947,7 +962,7 @@ defmodule String do
   @spec starts_with?(t, t | [t]) :: boolean
 
   def starts_with?(string, prefixes) when is_list(prefixes) do
-    Enum.any?(prefixes, do_starts_with(string, &1))
+    Enum.any?(prefixes, &do_starts_with(string, &1))
   end
 
   def starts_with?(string, prefix) do
@@ -983,7 +998,7 @@ defmodule String do
   @spec ends_with?(t, t | [t]) :: boolean
 
   def ends_with?(string, suffixes) when is_list(suffixes) do
-    Enum.any?(suffixes, do_ends_with(string, &1))
+    Enum.any?(suffixes, &do_ends_with(string, &1))
   end
 
   def ends_with?(string, suffix) do
@@ -1022,7 +1037,7 @@ defmodule String do
   @spec contains?(t, t | [t]) :: boolean
 
   def contains?(string, matches) when is_list(matches) do
-    Enum.any?(matches, do_contains(string, &1))
+    Enum.any?(matches, &do_contains(string, &1))
   end
 
   def contains?(string, match) do
@@ -1041,9 +1056,12 @@ defmodule String do
     raise ArgumentError
   end
 
-  defexception UnicodeConversionError, encoded: nil, rest: nil, kind: nil do
-    def message(exception) do
-      "#{exception.kind} #{detail(exception.rest)}"
+  defexception UnicodeConversionError, [:encoded, :message] do
+    def exception(opts) do
+      UnicodeConversionError[
+        encoded: opts[:encoded],
+        message: "#{opts[:kind]} #{detail(opts[:rest])}"
+      ]
     end
 
     defp detail(rest) when is_binary(rest) do

lib/elixir/lib/string/chars.ex

@@ -14,8 +14,6 @@ defprotocol String.Chars do
   as `"foo" <> to_string(bar)`.
   """
 
-  @only [BitString, List, Number, Atom, Record]
-
   def to_string(thing)
 end
 
@@ -29,7 +27,7 @@ defimpl String.Chars, for: Atom do
   end
 
   def to_string(atom) do
-    atom_to_binary(atom, :utf8)
+    atom_to_binary(atom)
   end
 end
 
@@ -67,14 +65,19 @@ defimpl String.Chars, for: List do
   def to_string(char_list), do: String.from_char_list!(char_list)
 end
 
-defimpl String.Chars, for: Number do
+defimpl String.Chars, for: Integer do
   @doc """
-  Simply converts the number (integer or a float) to a binary.
+  Simply converts the integer to a string.
   """
-  def to_string(thing) when is_integer(thing) do
+  def to_string(thing) do
     integer_to_binary(thing)
   end
+end
 
+defimpl String.Chars, for: Float do
+  @doc """
+  Simply converts the float to a string.
+  """
   def to_string(thing) do
     iolist_to_binary(:io_lib_format.fwrite_g(thing))
   end

lib/elixir/lib/system.ex

@@ -9,8 +9,8 @@ defmodule System do
     message: "could not get a current working directory, the current location is not accessible"
 
   @moduledoc """
-  The System module provides access to some variables used or
-  maintained by the VM and to functions that interact strongly
+  The System module provides access to variables used or
+  maintained by the VM and to functions that interact directly
   with the VM or the host system.
   """
 
@@ -28,8 +28,8 @@ defmodule System do
 
   # Read and strip the version from the `VERSION` file.
   defmacrop get_version do
-    case read_stripped("VERSION") do
-      ""   -> raise CompileError, message: "could not read the version number from VERSION"
+    case read_stripped(:filename.join(__DIR__, "../../../VERSION")) do
+      ""   -> raise ArgumentError, message: "could not read the version number from VERSION"
       data -> data
     end
   end
@@ -38,7 +38,7 @@ defmodule System do
   # the most recent tag. If that is not available, tries to read the commit hash
   # from .git/HEAD. If that fails, returns an empty string.
   defmacrop get_describe do
-    dirpath = ".git"
+    dirpath = :filename.join(__DIR__, "../../../.git")
     case :file.read_file_info(dirpath) do
       { :ok, _ } ->
         if :os.find_executable('git') do
@@ -57,13 +57,17 @@ defmodule System do
   end
 
   @doc """
+  Elixir version information.
+
   Returns Elixir's version as binary.
   """
   @spec version() :: String.t
   def version, do: get_version
 
   @doc """
-  Returns a keywords list with version, git tag info and date.
+  Elixir build information.
+
+  Returns a keyword list with Elixir version, git tag info and compilation date.
   """
   @spec build_info() :: Keyword.t
   def build_info do
@@ -71,7 +75,9 @@ defmodule System do
   end
 
   @doc """
-  Returns the list of command-line arguments passed to the program.
+  List command line arguments.
+
+  Returns the list of command line arguments passed to the program.
   """
   @spec argv() :: [String.t]
   def argv do
@@ -79,15 +85,19 @@ defmodule System do
   end
 
   @doc """
-  Changes the list of command-line arguments. Use it with caution,
-  as it destory any previous argv information.
+  Modify command line arguments.
+
+  Changes the list of command line arguments. Use it with caution,
+  as it destroys any previous argv information.
   """
   @spec argv([String.t]) :: :ok
-  def argv(argv) do
-    :elixir_code_server.cast({ :argv, argv })
+  def argv(args) do
+    :elixir_code_server.cast({ :argv, args })
   end
 
   @doc """
+  Current working directory.
+
   Returns the current working directory or `nil` if one
   is not available.
   """
@@ -99,6 +109,8 @@ defmodule System do
   end
 
   @doc """
+  Current working directory, exception on error.
+
   Returns the current working directory or raises `System.NoAccessCwdError`.
   """
   def cwd! do
@@ -106,8 +118,10 @@ defmodule System do
   end
 
   @doc """
-  Returns the user home (platform independent).
-  It returns `nil` if no user home is set.
+  User home directory.
+
+  Returns the user home directory (platform independent).
+  Returns `nil` if no user home is set.
   """
   def user_home do
     case :os.type() do
@@ -117,6 +131,8 @@ defmodule System do
   end
 
   @doc """
+  User home directory, exception on error.
+
   Same as `user_home/0` but raises `System.NoHomeError`
   instead of returning `nil` if no user home is set.
   """
@@ -139,8 +155,10 @@ defmodule System do
   end
 
   @doc %S"""
+  Writable temporary directory.
+
   Returns a writable temporary directory.
-  It searches for directories in the following order:
+  Searches for directories in the following order:
 
   1. The directory named by the TMPDIR environment variable
   2. The directory named by the TEMP environment variable
@@ -159,7 +177,9 @@ defmodule System do
   end
 
   @doc """
-  Same as `tmp_dir` but raises `System.NoTmpDirError`
+  Writable temporary directory, exception on error.
+
+  Same as `tmp_dir/0` but raises `System.NoTmpDirError`
   instead of returning `nil` if no temp dir is set.
   """
   def tmp_dir! do
@@ -189,18 +209,22 @@ defmodule System do
   end
 
   @doc """
+  Register a program exit handler function.
+
   Registers a function that will be invoked
   at the end of program execution. Useful for
-  invoking a hook in a "script" mode.
+  invoking a hook in "script" mode.
 
-  The function must expect the exit status code
-  as argument.
+  The function must receive the exit status code
+  as an argument.
   """
   def at_exit(fun) when is_function(fun, 1) do
     :elixir_code_server.cast { :at_exit, fun }
   end
 
   @doc """
+  Execute a system command.
+
   Executes `command` in a command shell of the target OS,
   captures the standard output of the command and returns
   the result as a binary.
@@ -222,6 +246,8 @@ defmodule System do
   end
 
   @doc """
+  Locate an executable on the system.
+
   This function looks up an executable program given
   its name using the environment variable PATH on Unix
   and Windows. It also considers the proper executable
@@ -248,6 +274,8 @@ defmodule System do
   end
 
   @doc """
+  System environment variables.
+
   Returns a list of all environment variables. Each variable is given as a
   `{name, value}` tuple where both `name` and `value` are strings.
   """
@@ -261,6 +289,8 @@ defmodule System do
   end
 
   @doc """
+  Environment variable value.
+
   Returns the value of the environment variable
   `varname` as a binary, or `nil` if the environment
   variable is undefined.
@@ -274,6 +304,8 @@ defmodule System do
   end
 
   @doc """
+  Erlang VM process identifier.
+
   Returns the process identifier of the current Erlang emulator
   in the format most commonly used by the operating system environment.
 
@@ -283,6 +315,8 @@ defmodule System do
   def get_pid, do: iolist_to_binary(:os.getpid)
 
   @doc """
+  Set an environment variable value.
+
   Sets a new `value` for the environment variable `varname`.
   """
   @spec put_env(binary, binary) :: :ok
@@ -292,6 +326,8 @@ defmodule System do
   end
 
   @doc """
+  Set multiple environment variables.
+
   Sets a new value for each environment variable corresponding
   to each key in `dict`.
   """
@@ -301,22 +337,24 @@ defmodule System do
   end
 
   @doc """
-  Gets Elixir's stacktrace.
+  Last exception stacktrace.
 
-  Notice the Erlang VM (and therefore this function) does not
+  Note that the Erlang VM (and therefore this function) does not
   return the current stacktrace but rather the stacktrace of the
   latest exception.
   """
   def stacktrace do
-    filter_stacktrace :erlang.get_stacktrace
+    :erlang.get_stacktrace
   end
 
   @doc """
-  Halts the Erlang runtime system where the first argument status must be a
+  Halt the Erlang runtime system.
+
+  Halts the Erlang runtime system where the argument `status` must be a
   non-negative integer, the atom `:abort` or a binary.
 
   * If an integer, the runtime system exits with the integer value which
-    is returned to the Operating System;
+    is returned to the operating system;
 
   * If `:abort`, the runtime system aborts producing a core dump, if that is
     enabled in the operating system;
@@ -347,9 +385,4 @@ defmodule System do
   def halt(status) when is_binary(status) do
     :erlang.halt(:binary.bin_to_list(status))
   end
-
-  ## Helpers
-
-  defp filter_stacktrace([{ Kernel, :raise, _, _ }|t]), do: t
-  defp filter_stacktrace(t), do: t
 end

lib/elixir/lib/tuple.ex

@@ -0,0 +1,60 @@
+defmodule Tuple do
+  @moduledoc """
+  Functions for working with tuples.
+  """
+
+  @doc """
+  Create a new tuple.
+
+  Creates a tuple of size `size` containing the
+  given `data` at every position.
+
+  ## Examples
+
+      iex> Tuple.duplicate(:hello, 3)
+      { :hello, :hello, :hello }
+
+  """
+  @spec duplicate(term, non_neg_integer) :: tuple
+  def duplicate(data, size) do
+    :erlang.make_tuple(size, data)
+  end
+
+  @doc """
+  Insert an element into a tuple.
+
+  Inserts `value` into `tuple` at the given zero-based `index`.
+  Raises an `ArgumentError` if `index` is greater than the
+  length of `tuple`.
+
+  ## Examples
+
+      iex> tuple = { :bar, :baz }
+      ...> Tuple.insert_at(tuple, 0, :foo)
+      { :foo, :bar, :baz }
+
+  """
+  @spec insert_at(tuple, non_neg_integer, term) :: tuple
+  def insert_at(tuple, index, term) do
+    :erlang.insert_element(index + 1, tuple, term)
+  end
+
+  @doc """
+  Remove an element from a tuple.
+
+  Deletes the element at the zero-based `index` from `tuple`.
+  Raises an `ArgumentError` if `index` is greater than 
+  or equal to the length of `tuple`.
+
+  ## Examples
+
+      iex> tuple = { :foo, :bar, :baz }
+      ...> Tuple.delete_at(tuple, 0)
+      { :bar, :baz }
+
+  """
+  @spec delete_at(tuple, non_neg_integer) :: tuple
+  def delete_at(tuple, index) do
+    :erlang.delete_element(index + 1, tuple)
+  end
+end

lib/elixir/lib/uri.ex

@@ -31,8 +31,18 @@ defmodule URI do
 
   @doc """
   Returns the default port for a given scheme.
+
   If the scheme is unknown to URI, returns `nil`.
   Any scheme may be registered via `default_port/2`.
+
+  ## Examples
+
+      iex> URI.default_port("ftp")
+      21
+
+      iex> URI.default_port("ponzi")
+      nil
+
   """
   def default_port(scheme) when is_binary(scheme) do
     { :ok, dict } = :application.get_env(:elixir, :uri)
@@ -48,20 +58,43 @@ defmodule URI do
   end
 
   @doc """
+  Encodes an enumerable into a query string.
+
   Takes an enumerable (containing a sequence of two-item tuples)
-  and returns a string of the form "k=v&k2=v2..." where keys and values are
-  URL encoded as per `encode/1`. Keys and values can be any term
-  that implements the `String.Chars` protocol (i.e. can be converted
+  and returns a string of the form "key1=value1&key2=value2..." where
+  keys and values are URL encoded as per `encode/1`. Keys and values can
+  be any term that implements the `String.Chars` protocol (i.e. can be converted
   to a binary).
+
+  ## Examples
+
+      iex> hd = HashDict.new([{"foo", 1}, {"bar", "2"}])
+      iex> URI.encode_query(hd)
+      "bar=2&foo=1"
+
   """
-  def encode_query(l), do: Enum.map_join(l, "&", pair(&1))
+  def encode_query(l), do: Enum.map_join(l, "&", &pair/1)
 
   @doc """
-  Given a query string of the form "key1=value1&key=value2...", produces an
-  orddict with one entry for each key-value pair. Each key and value will be a
-  binary. It also does percent-unescaping of both keys and values.
+  Decodes a query string into a `HashDict`.
+
+  Given a query string of the form "key1=value1&key2=value2...", produces a
+  `HashDict` with one entry for each key-value pair. Each key and value will be a
+  binary. Keys and values will be percent-unescaped.
 
   Use `query_decoder/1` if you want to iterate over each value manually.
+
+  ## Examples
+
+      iex> URI.decode_query("foo=1&bar=2") |> Dict.to_list
+      [{"bar", "2"}, {"foo", "1"}]
+
+      iex> hd = HashDict.new()
+      iex> URI.decode_query("foo=1&bar=2", hd) |> HashDict.keys
+      ["bar", "foo"]
+      iex> URI.decode_query("foo=1&bar=2", hd) |> HashDict.values
+      ["2", "1"]
+
   """
   def decode_query(q, dict // HashDict.new) when is_binary(q) do
     Enum.reduce query_decoder(q), dict, fn({ k, v }, acc) -> Dict.put(acc, k, v) end
@@ -70,22 +103,26 @@ defmodule URI do
   @doc """
   Returns an iterator function over the query string that decodes
   the query string in steps.
+
+  ## Examples
+
+      iex> URI.query_decoder("foo=1&bar=2") |> Enum.map &(&1)
+      [{"foo", "1"}, {"bar", "2"}]
+
   """
   def query_decoder(q) when is_binary(q) do
-    fn(acc, fun) ->
-      do_decoder(q, acc, fun)
-    end
+    Stream.unfold(q, &do_decoder/1)
   end
 
-  defp do_decoder("", acc, _fun) do
-    acc
+  defp do_decoder("") do
+    nil
   end
 
-  defp do_decoder(q, acc, fun) do
-    next =
+  defp do_decoder(q) do
+    { first, next } =
       case :binary.split(q, "&") do
-        [first, rest] -> rest
-        [first]       -> ""
+        [first, rest] -> { first, rest }
+        [first]       -> { first, "" }
       end
 
     current =
@@ -94,7 +131,7 @@ defmodule URI do
         [ key ]        -> { decode(key), nil }
       end
 
-    do_decoder(next, fun.(current, acc), fun)
+    { current, next }
   end
 
   defp pair({k, v}) do
@@ -102,7 +139,13 @@ defmodule URI do
   end
 
   @doc """
-  Percent (URL) encodes a URI.
+  Percent-escape a URI.
+
+  ## Example
+
+      iex> URI.encode("http://elixir-lang.com/getting_started/2.html")
+      "http%3A%2F%2Felixir-lang.com%2Fgetting_started%2F2.html"
+
   """
   def encode(s), do: bc(<<c>> inbits s, do: <<percent(c) :: binary>>)
 
@@ -124,7 +167,13 @@ defmodule URI do
   defp hex(n), do: <<n + ?A - 10>>
 
   @doc """
-  Unpercent (URL) decodes a URI.
+  Percent-unescape a URI.
+
+  ## Examples
+
+      iex> URI.decode("http%3A%2F%2Felixir-lang.com")
+      "http://elixir-lang.com"
+
   """
   def decode(<<?%, hex1, hex2, tail :: binary >>) do
     << bsl(hex2dec(hex1), 4) + hex2dec(hex2) >> <> decode(tail)
@@ -137,6 +186,7 @@ defmodule URI do
   def decode(<<>>), do: <<>>
 
   defp hex2dec(n) when n in ?A..?F, do: n - ?A + 10
+  defp hex2dec(n) when n in ?a..?f, do: n - ?a + 10
   defp hex2dec(n) when n in ?0..?9, do: n - ?0
 
   defp check_plus(?+), do: 32
@@ -145,20 +195,18 @@ defmodule URI do
   @doc """
   Parses a URI into components.
 
-  URIs have portions that are handled specially for the
-  particular scheme of the URI. For example, http and https
-  have different default ports. Sometimes the parsing
-  of portions themselves are different. This parser
-  is extensible via behavior modules. If you have a
-  module named `URI.MYSCHEME` with a function called
-  `parse` that takes a single argument, the generically
-  parsed URI, that function will be called when this
-  parse function is passed a URI of that scheme. This
-  allows you to build on top of what the URI library
-  currently offers. You also need to define `default_port`
-  which takes no arguments and returns the default port
-  for that particular scheme. Take a look at `URI.HTTPS` for an
-  example of one of these extension modules.
+  URIs have portions that are handled specially for the particular
+  scheme of the URI. For example, http and https have different
+  default ports. Such values can be accessed and registered via
+  `URI.default_port/1` and `URI.default_port/2`.
+
+  ## Examples
+
+      iex> URI.parse("http://elixir-lang.org/")
+      URI.Info[scheme: "http", path: "/", query: nil, fragment: nil,
+               authority: "elixir-lang.org", userinfo: nil,
+               host: "elixir-lang.org", port: 80]
+
   """
   def parse(s) when is_binary(s) do
     # From http://tools.ietf.org/html/rfc3986#appendix-B

lib/elixir/lib/version.ex

@@ -1,15 +1,13 @@
-defmodule Mix.Version do
+defmodule Version do
   @moduledoc %S"""
-  This module provides functions for parsing and matching
-  versions with requirements.
+  Functions for parsing and matching versions against requirements.
 
-  A version is a string or a `Mix.Version.Schema` generated
-  after parsing via `Mix.Version.parse/1`. A requirement is
-  a string that follows a specific format.
+  A version is a string in a specific format or a `Version.Schema`
+  generated after parsing via `Version.parse/1`.
 
-  `Mix.Version` parsing and requirements follows
+  `Version` parsing and requirements follow
   [SemVer 2.0 schema](http://semver.org/) and you will get
-  the most of Mix' version system by following it. In order
+  the most of Mix's version system by following it. In order
   to support integration with projects that may
   follow different versioning schemas, Elixir won't choke
   on unknown versions, however you won't be able to use
@@ -33,7 +31,7 @@ defmodule Mix.Version do
 
   Requirements allow you to specify which versions of a given
   dependency you are willing to work against. It supports common
-  operators like `>=`, `<=`, `>`, `==` and friends that would
+  operators like `>=`, `<=`, `>`, `==` and friends that
   work as one would expect:
 
       # Only version 2.0.0
@@ -54,8 +52,8 @@ defmodule Mix.Version do
 
   """
 
-  @type t :: String.t | Mix.Version.Schema.t
-  @type requirement :: String.t | Mix.Version.Requirement.t
+  @type t :: String.t | Version.Schema.t
+  @type requirement :: String.t | Version.Requirement.t
 
   @type matchable :: { major :: String.t | non_neg_integer,
                        minor :: non_neg_integer | nil,
@@ -67,23 +65,37 @@ defmodule Mix.Version do
   defrecord Schema, major: 0, minor: 0, patch: 0, pre: nil, build: nil, source: nil
   defrecord Requirement, source: nil, matchspec: nil
 
-  defexception InvalidRequirement, reason: :invalid_requirement do
-    def message(InvalidRequirement[reason: reason]) when is_binary(reason) do
-      { first, rest } = String.next_grapheme(reason)
-      String.downcase(first) <> rest
-    end
+  defexception InvalidRequirement, [:message] do
+    def exception(opts) do
+      message =
+        if is_binary opts[:reason] do
+          { first, rest } = String.next_grapheme(opts[:reason])
+          String.downcase(first) <> rest
+        else
+          "invalid version specification"
+        end
 
-    def message(InvalidRequirement[]) do
-      "invalid version specification"
+      InvalidRequirement[message: message]
     end
   end
 
   @doc """
-  Checks if the given version matches the specification.
+  Check if the given version matches the specification.
+
+  Returns `true` if `version` satisfies `requirement`, `false` otherwise.
+  Raises a `Version.InvalidRequirement` exception if `requirement` is not parseable.
+
+  ## Examples
+
+      iex> Version.match?("2.0", ">1.0")
+      true
+      iex> Version.match?("2.0", "==1.0")
+      false
+
   """
   @spec match?(t, requirement) :: boolean
   def match?(version, requirement) when is_binary(requirement) do
-    case Mix.Version.Parser.parse_requirement(requirement) do
+    case Version.Parser.parse_requirement(requirement) do
       { :ok, req } ->
         match?(version, req)
       { :error, reason } ->
@@ -105,24 +117,38 @@ defmodule Mix.Version do
   end
 
   @doc """
-  Checks if a version string is compatible with [semver](http://semver.org/).
+  Check if a version string is compatible with [semver](http://semver.org/).
+
+  ## Examples
+
+      iex> Version.valid?("2.0")
+      true
+      iex> Version.valid?("invalid")
+      false
+
   """
   @spec valid?(String.t | Schema.t) :: boolean
   def valid?(string) when is_binary(string) do
-    Mix.Version.Parser.valid_version?(string)
+    Version.Parser.valid_version?(string)
   end
 
-  def valid?(Mix.Version.Schema[major: nil]), do: false
-  def valid?(Mix.Version.Schema[]),           do: true
+  def valid?(Version.Schema[major: nil]), do: false
+  def valid?(Version.Schema[]),           do: true
 
   @doc """
-  Parse a version into a matchable value.
+  Parse a version string into a `Version.Schema`.
+
+  ## Examples
+
+      iex> Version.parse("2.0.1-alpha1")
+      #Version.Schema<2.0.1-alpha1>
+
   """
-  @spec parse(String.t) :: { :ok, Schema.t } | { :error, term }
+  @spec parse(String.t) :: Schema.t
   def parse(string) when is_binary(string) do
-    case Mix.Version.Parser.parse_version(string) do
+    case Version.Parser.parse_version(string) do
       { :ok, matchable } -> from_matchable(matchable).source(string).build(get_build(string))
-      { :error, _ } -> Mix.Version.Schema[source: string]
+      { :error, _ } -> Version.Schema[source: string]
     end
   end
 
@@ -137,19 +163,25 @@ defmodule Mix.Version do
   end
 
   @doc """
-  Get the matchable representation.
+  Convert a version to a `Version.matchable`
+
+  ## Examples
+
+      iex> Version.to_matchable("2.0.1-alpha.1")
+      {2, 0, 1, ["alpha", 1]}
+
   """
-  @spec to_matchable(String.t | Schema.t) :: Mix.Version.matchable
+  @spec to_matchable(String.t | Schema.t) :: Version.matchable
   def to_matchable(Schema[major: nil, source: source]) do
     { source, nil, nil, [] }
   end
 
-  def to_matchable(Mix.Version.Schema[major: major, minor: minor, patch: patch, pre: nil]) do
+  def to_matchable(Version.Schema[major: major, minor: minor, patch: patch, pre: nil]) do
     { major, minor, patch, [] }
   end
 
-  def to_matchable(Mix.Version.Schema[major: major, minor: minor, patch: patch, pre: pre]) do
-    { major, minor, patch, Mix.Version.Parser.parse_pre(pre) }
+  def to_matchable(Version.Schema[major: major, minor: minor, patch: patch, pre: pre]) do
+    { major, minor, patch, Version.Parser.parse_pre(pre) }
   end
 
   def to_matchable(string) do
@@ -157,11 +189,17 @@ defmodule Mix.Version do
   end
 
   @doc """
-  Convert a matchable to a `Mix.Version`.
+  Convert a matchable to a `Version.Schema`.
+
+  ## Examples
+
+      iex> Version.from_matchable({2, 0, 1, ["alpha", 1]})
+      #Version.Schema<2.0.1-alpha.1>
+
   """
-  @spec from_matchable(Mix.Version.matchable) :: Schema.t
+  @spec from_matchable(Version.matchable) :: Schema.t
   def from_matchable({ source, nil, nil, nil }) when is_binary(source) do
-    Mix.Version.Schema[source: source]
+    Version.Schema[source: source]
   end
 
   def from_matchable({ major, minor, patch, pre }) do
@@ -184,7 +222,7 @@ defmodule Mix.Version do
       end
     end
 
-    Mix.Version.Schema[major: major, minor: minor, patch: patch, pre: pre, source: source]
+    Version.Schema[major: major, minor: minor, patch: patch, pre: pre, source: source]
   end
 
   defmodule Parser.DSL do
@@ -251,12 +289,12 @@ defmodule Mix.Version do
     end
 
     deflexer acc do
-      Enum.filter(Enum.reverse(acc), &1 != :' ')
+      Enum.filter(Enum.reverse(acc), &(&1 != :' '))
     end
 
     @version_regex %r/^(\d+)(?:\.(\d+)(?:\.(\d+))?)?(?:\-([^\s]+))?(?:\+[^\d]+)?$/
 
-    @spec parse_requirement(String.t) :: { :ok, Mix.Version.Requirement.t } | { :error, binary | atom }
+    @spec parse_requirement(String.t) :: { :ok, Version.Requirement.t } | { :error, binary | atom }
     def parse_requirement(source) do
       lexed = lexer(source, [])
 
@@ -280,7 +318,7 @@ defmodule Mix.Version do
     defp nillify(""), do: nil
     defp nillify(o),  do: o
 
-    @spec parse_version(String.t) :: { :ok, Mix.Version.matchable } | { :error, :invalid_version }
+    @spec parse_version(String.t) :: { :ok, Version.matchable } | { :error, :invalid_version }
     def parse_version(string) when is_binary(string) do
       if valid_version?(string) do
         destructure [_, major, minor, patch, pre], Regex.run(@version_regex, string)
@@ -364,18 +402,18 @@ defmodule Mix.Version do
     end
 
     defp approximate(version) do
-      Mix.Version.from_matchable(case Regex.run(@version_regex, version) do
+      Version.from_matchable(case Regex.run(@version_regex, version) do
         [_, major] ->
-          { binary_to_integer(major) + 1, 0, 0, [] }
+          { binary_to_integer(major) + 1, 0, 0, [0] }
 
         [_, major, _] ->
-          { binary_to_integer(major) + 1, 0, 0, [] }
+          { binary_to_integer(major) + 1, 0, 0, [0] }
 
         [_, major, minor, _] ->
-          { binary_to_integer(major), binary_to_integer(minor) + 1, 0, [] }
+          { binary_to_integer(major), binary_to_integer(minor) + 1, 0, [0] }
 
         [_, major, minor, _, _] ->
-          { binary_to_integer(major), binary_to_integer(minor) + 1, 0, [] }
+          { binary_to_integer(major), binary_to_integer(minor) + 1, 0, [0] }
       end)
     end
 
@@ -387,19 +425,19 @@ defmodule Mix.Version do
     end
 
     defp to_condition([:'==', version | _]) do
-      version = Mix.Version.to_matchable(version)
+      version = Version.to_matchable(version)
 
       { :'==', :'$_', { :const, version } }
     end
 
     defp to_condition([:'!=', version | _]) do
-      version = Mix.Version.to_matchable(version)
+      version = Version.to_matchable(version)
 
       { :'/=', :'$_', { :const, version } }
     end
 
     defp to_condition([:'~>', version | _]) do
-      from = Mix.Version.parse(version)
+      from = Version.parse(version)
       to   = approximate(version)
 
       { :andalso, to_condition([:'>=', to_string(from)]),
@@ -407,7 +445,7 @@ defmodule Mix.Version do
     end
 
     defp to_condition([:'>', version | _]) do
-      { major, minor, patch, pre } = Mix.Version.to_matchable(version)
+      { major, minor, patch, pre } = Version.to_matchable(version)
 
       { :andalso, { :not, { :is_binary, :'$1' } },
                   { :orelse, { :'>', {{ :'$1', :'$2', :'$3' }},
@@ -423,7 +461,7 @@ defmodule Mix.Version do
     end
 
     defp to_condition([:'>=', version | _]) do
-      matchable = Mix.Version.to_matchable(version)
+      matchable = Version.to_matchable(version)
 
       { :orelse, { :andalso, { :not, { :is_binary, :'$1' } },
                              { :'==', :'$_', { :const, matchable } } },
@@ -431,7 +469,7 @@ defmodule Mix.Version do
     end
 
     defp to_condition([:'<', version | _]) do
-      { major, minor, patch, pre } = Mix.Version.to_matchable(version)
+      { major, minor, patch, pre } = Version.to_matchable(version)
 
       { :andalso, { :not, { :is_binary, :'$1' } },
                   { :orelse, { :'<', {{ :'$1', :'$2', :'$3' }},
@@ -447,7 +485,7 @@ defmodule Mix.Version do
     end
 
     defp to_condition([:'<=', version | _]) do
-      matchable = Mix.Version.to_matchable(version)
+      matchable = Version.to_matchable(version)
 
       { :orelse, { :andalso, { :not, { :is_binary, :'$1' } },
                              { :'==', :'$_', { :const, matchable } } },
@@ -468,26 +506,26 @@ defmodule Mix.Version do
   end
 end
 
-defimpl String.Chars, for: Mix.Version.Schema do
-  def to_string(Mix.Version.Schema[source: source]) do
+defimpl String.Chars, for: Version.Schema do
+  def to_string(Version.Schema[source: source]) do
     source
   end
 end
 
-defimpl Inspect, for: Mix.Version.Schema do
+defimpl Inspect, for: Version.Schema do
   def inspect(self, _opts) do
-    "#Mix.Version.Schema<" <> to_string(self) <> ">"
+    "#Version.Schema<" <> to_string(self) <> ">"
   end
 end
 
-defimpl String.Chars, for: Mix.Version.Requirement do
+defimpl String.Chars, for: Version.Requirement do
   def to_string({ _, source, _ }) do
     source
   end
 end
 
-defimpl Inspect, for: Mix.Version.Requirement do
+defimpl Inspect, for: Version.Requirement do
   def inspect({ _, source, _ }, _opts) do
-    "#Mix.Version.Requirement<" <> source <> ">"
+    "#Version.Requirement<" <> source <> ">"
   end
 end

lib/elixir/priv/NamedSequences.txt

@@ -1,420 +0,0 @@
-LATIN CAPITAL LETTER A WITH MACRON AND GRAVE;0100 0300
-LATIN SMALL LETTER A WITH MACRON AND GRAVE;0101 0300
-LATIN CAPITAL LETTER E WITH VERTICAL LINE BELOW;0045 0329
-LATIN SMALL LETTER E WITH VERTICAL LINE BELOW;0065 0329
-LATIN CAPITAL LETTER E WITH VERTICAL LINE BELOW AND GRAVE;00C8 0329
-LATIN SMALL LETTER E WITH VERTICAL LINE BELOW AND GRAVE;00E8 0329
-LATIN CAPITAL LETTER E WITH VERTICAL LINE BELOW AND ACUTE;00C9 0329
-LATIN SMALL LETTER E WITH VERTICAL LINE BELOW AND ACUTE;00E9 0329
-LATIN CAPITAL LETTER E WITH CIRCUMFLEX AND MACRON;00CA 0304
-LATIN SMALL LETTER E WITH CIRCUMFLEX AND MACRON;00EA 0304
-LATIN CAPITAL LETTER E WITH CIRCUMFLEX AND CARON;00CA 030C
-LATIN SMALL LETTER E WITH CIRCUMFLEX AND CARON;00EA 030C
-LATIN CAPITAL LETTER I WITH MACRON AND GRAVE;012A 0300
-LATIN SMALL LETTER I WITH MACRON AND GRAVE;012B 0300
-LATIN SMALL LETTER I WITH DOT ABOVE AND ACUTE;0069 0307 0301
-LATIN SMALL LETTER NG WITH TILDE ABOVE;006E 0360 0067
-LATIN CAPITAL LETTER O WITH VERTICAL LINE BELOW;004F 0329
-LATIN SMALL LETTER O WITH VERTICAL LINE BELOW;006F 0329
-LATIN CAPITAL LETTER O WITH VERTICAL LINE BELOW AND GRAVE;00D2 0329
-LATIN SMALL LETTER O WITH VERTICAL LINE BELOW AND GRAVE;00F2 0329
-LATIN CAPITAL LETTER O WITH VERTICAL LINE BELOW AND ACUTE;00D3 0329
-LATIN SMALL LETTER O WITH VERTICAL LINE BELOW AND ACUTE;00F3 0329
-LATIN CAPITAL LETTER S WITH VERTICAL LINE BELOW;0053 0329
-LATIN SMALL LETTER S WITH VERTICAL LINE BELOW;0073 0329
-LATIN CAPITAL LETTER U WITH MACRON AND GRAVE;016A 0300
-LATIN SMALL LETTER U WITH MACRON AND GRAVE;016B 0300
-LATIN CAPITAL LETTER A WITH OGONEK AND ACUTE;0104 0301
-LATIN SMALL LETTER A WITH OGONEK AND ACUTE;0105 0301
-LATIN CAPITAL LETTER A WITH OGONEK AND TILDE;0104 0303
-LATIN SMALL LETTER A WITH OGONEK AND TILDE;0105 0303
-LATIN CAPITAL LETTER E WITH OGONEK AND ACUTE;0118 0301
-LATIN SMALL LETTER E WITH OGONEK AND ACUTE;0119 0301
-LATIN CAPITAL LETTER E WITH OGONEK AND TILDE;0118 0303
-LATIN SMALL LETTER E WITH OGONEK AND TILDE;0119 0303
-LATIN CAPITAL LETTER E WITH DOT ABOVE AND ACUTE;0116 0301
-LATIN SMALL LETTER E WITH DOT ABOVE AND ACUTE;0117 0301
-LATIN CAPITAL LETTER E WITH DOT ABOVE AND TILDE;0116 0303
-LATIN SMALL LETTER E WITH DOT ABOVE AND TILDE;0117 0303
-LATIN SMALL LETTER I WITH DOT ABOVE AND GRAVE;0069 0307 0300
-LATIN SMALL LETTER I WITH DOT ABOVE AND TILDE;0069 0307 0303
-LATIN CAPITAL LETTER I WITH OGONEK AND ACUTE;012E 0301
-LATIN SMALL LETTER I WITH OGONEK AND DOT ABOVE AND ACUTE;012F 0307 0301
-LATIN CAPITAL LETTER I WITH OGONEK AND TILDE;012E 0303
-LATIN SMALL LETTER I WITH OGONEK AND DOT ABOVE AND TILDE;012F 0307 0303
-LATIN CAPITAL LETTER J WITH TILDE;004A 0303
-LATIN SMALL LETTER J WITH DOT ABOVE AND TILDE;006A 0307 0303
-LATIN CAPITAL LETTER L WITH TILDE;004C 0303
-LATIN SMALL LETTER L WITH TILDE;006C 0303
-LATIN CAPITAL LETTER M WITH TILDE;004D 0303
-LATIN SMALL LETTER M WITH TILDE;006D 0303
-LATIN CAPITAL LETTER R WITH TILDE;0052 0303
-LATIN SMALL LETTER R WITH TILDE;0072 0303
-LATIN CAPITAL LETTER U WITH OGONEK AND ACUTE;0172 0301
-LATIN SMALL LETTER U WITH OGONEK AND ACUTE;0173 0301
-LATIN CAPITAL LETTER U WITH OGONEK AND TILDE;0172 0303
-LATIN SMALL LETTER U WITH OGONEK AND TILDE;0173 0303
-LATIN CAPITAL LETTER U WITH MACRON AND ACUTE;016A 0301
-LATIN SMALL LETTER U WITH MACRON AND ACUTE;016B 0301
-LATIN CAPITAL LETTER U WITH MACRON AND TILDE;016A 0303
-LATIN SMALL LETTER U WITH MACRON AND TILDE;016B 0303
-LATIN SMALL LETTER AE WITH GRAVE;00E6 0300
-LATIN SMALL LETTER OPEN O WITH GRAVE;0254 0300
-LATIN SMALL LETTER OPEN O WITH ACUTE;0254 0301
-LATIN SMALL LETTER TURNED V WITH GRAVE;028C 0300
-LATIN SMALL LETTER TURNED V WITH ACUTE;028C 0301
-LATIN SMALL LETTER SCHWA WITH GRAVE;0259 0300
-LATIN SMALL LETTER SCHWA WITH ACUTE;0259 0301
-LATIN SMALL LETTER HOOKED SCHWA WITH GRAVE;025A 0300
-LATIN SMALL LETTER HOOKED SCHWA WITH ACUTE;025A 0301
-BENGALI LETTER KHINYA;0995 09CD 09B7
-TAMIL CONSONANT K;  0B95 0BCD
-TAMIL CONSONANT NG; 0B99 0BCD
-TAMIL CONSONANT C;  0B9A 0BCD
-TAMIL CONSONANT NY; 0B9E 0BCD
-TAMIL CONSONANT TT; 0B9F 0BCD
-TAMIL CONSONANT NN; 0BA3 0BCD
-TAMIL CONSONANT T;  0BA4 0BCD
-TAMIL CONSONANT N;  0BA8 0BCD
-TAMIL CONSONANT P;  0BAA 0BCD
-TAMIL CONSONANT M;  0BAE 0BCD
-TAMIL CONSONANT Y;  0BAF 0BCD
-TAMIL CONSONANT R;  0BB0 0BCD
-TAMIL CONSONANT L;  0BB2 0BCD
-TAMIL CONSONANT V;  0BB5 0BCD
-TAMIL CONSONANT LLL;0BB4 0BCD
-TAMIL CONSONANT LL; 0BB3 0BCD
-TAMIL CONSONANT RR; 0BB1 0BCD
-TAMIL CONSONANT NNN;0BA9 0BCD
-TAMIL CONSONANT J;  0B9C 0BCD
-TAMIL CONSONANT SH; 0BB6 0BCD
-TAMIL CONSONANT SS; 0BB7 0BCD
-TAMIL CONSONANT S;  0BB8 0BCD
-TAMIL CONSONANT H;  0BB9 0BCD
-TAMIL CONSONANT KSS;0B95 0BCD 0BB7 0BCD
-TAMIL SYLLABLE KAA; 0B95 0BBE
-TAMIL SYLLABLE KI;  0B95 0BBF
-TAMIL SYLLABLE KII; 0B95 0BC0
-TAMIL SYLLABLE KU;  0B95 0BC1
-TAMIL SYLLABLE KUU; 0B95 0BC2
-TAMIL SYLLABLE KE;  0B95 0BC6
-TAMIL SYLLABLE KEE; 0B95 0BC7
-TAMIL SYLLABLE KAI; 0B95 0BC8
-TAMIL SYLLABLE KO;  0B95 0BCA
-TAMIL SYLLABLE KOO; 0B95 0BCB
-TAMIL SYLLABLE KAU; 0B95 0BCC
-TAMIL SYLLABLE NGAA; 0B99 0BBE
-TAMIL SYLLABLE NGI;  0B99 0BBF
-TAMIL SYLLABLE NGII; 0B99 0BC0
-TAMIL SYLLABLE NGU;  0B99 0BC1
-TAMIL SYLLABLE NGUU; 0B99 0BC2
-TAMIL SYLLABLE NGE;  0B99 0BC6
-TAMIL SYLLABLE NGEE; 0B99 0BC7
-TAMIL SYLLABLE NGAI; 0B99 0BC8
-TAMIL SYLLABLE NGO;  0B99 0BCA
-TAMIL SYLLABLE NGOO; 0B99 0BCB
-TAMIL SYLLABLE NGAU; 0B99 0BCC
-TAMIL SYLLABLE CI;  0B9A 0BBF
-TAMIL SYLLABLE CII; 0B9A 0BC0
-TAMIL SYLLABLE CU;  0B9A 0BC1
-TAMIL SYLLABLE CUU; 0B9A 0BC2
-TAMIL SYLLABLE CE;  0B9A 0BC6
-TAMIL SYLLABLE CEE; 0B9A 0BC7
-TAMIL SYLLABLE CAI; 0B9A 0BC8
-TAMIL SYLLABLE CO;  0B9A 0BCA
-TAMIL SYLLABLE COO; 0B9A 0BCB
-TAMIL SYLLABLE CAU; 0B9A 0BCC
-TAMIL SYLLABLE NYAA; 0B9E 0BBE
-TAMIL SYLLABLE NYI;  0B9E 0BBF
-TAMIL SYLLABLE NYII; 0B9E 0BC0
-TAMIL SYLLABLE NYU;  0B9E 0BC1
-TAMIL SYLLABLE NYUU; 0B9E 0BC2
-TAMIL SYLLABLE NYE;  0B9E 0BC6
-TAMIL SYLLABLE NYEE; 0B9E 0BC7
-TAMIL SYLLABLE NYAI; 0B9E 0BC8
-TAMIL SYLLABLE NYO;  0B9E 0BCA
-TAMIL SYLLABLE NYOO; 0B9E 0BCB
-TAMIL SYLLABLE NYAU; 0B9E 0BCC
-TAMIL SYLLABLE TTAA; 0B9F 0BBE
-TAMIL SYLLABLE TTI;  0B9F 0BBF
-TAMIL SYLLABLE TTII; 0B9F 0BC0
-TAMIL SYLLABLE TTU;  0B9F 0BC1
-TAMIL SYLLABLE TTUU; 0B9F 0BC2
-TAMIL SYLLABLE TTE;  0B9F 0BC6
-TAMIL SYLLABLE TTEE; 0B9F 0BC7
-TAMIL SYLLABLE TTAI; 0B9F 0BC8
-TAMIL SYLLABLE TTO;  0B9F 0BCA
-TAMIL SYLLABLE TTOO; 0B9F 0BCB
-TAMIL SYLLABLE TTAU; 0B9F 0BCC
-TAMIL SYLLABLE NNAA; 0BA3 0BBE
-TAMIL SYLLABLE NNI;  0BA3 0BBF
-TAMIL SYLLABLE NNII; 0BA3 0BC0
-TAMIL SYLLABLE NNU;  0BA3 0BC1
-TAMIL SYLLABLE NNUU; 0BA3 0BC2
-TAMIL SYLLABLE NNE;  0BA3 0BC6
-TAMIL SYLLABLE NNEE; 0BA3 0BC7
-TAMIL SYLLABLE NNAI; 0BA3 0BC8
-TAMIL SYLLABLE NNO;  0BA3 0BCA
-TAMIL SYLLABLE NNOO; 0BA3 0BCB
-TAMIL SYLLABLE NNAU; 0BA3 0BCC
-TAMIL SYLLABLE TAA; 0BA4 0BBE
-TAMIL SYLLABLE TI;  0BA4 0BBF
-TAMIL SYLLABLE TII; 0BA4 0BC0
-TAMIL SYLLABLE TU;  0BA4 0BC1
-TAMIL SYLLABLE TUU; 0BA4 0BC2
-TAMIL SYLLABLE TE;  0BA4 0BC6
-TAMIL SYLLABLE TEE; 0BA4 0BC7
-TAMIL SYLLABLE TAI; 0BA4 0BC8
-TAMIL SYLLABLE TO;  0BA4 0BCA
-TAMIL SYLLABLE TOO; 0BA4 0BCB
-TAMIL SYLLABLE TAU; 0BA4 0BCC
-TAMIL SYLLABLE NAA; 0BA8 0BBE
-TAMIL SYLLABLE NI;  0BA8 0BBF
-TAMIL SYLLABLE NII; 0BA8 0BC0
-TAMIL SYLLABLE NU;  0BA8 0BC1
-TAMIL SYLLABLE NUU; 0BA8 0BC2
-TAMIL SYLLABLE NE;  0BA8 0BC6
-TAMIL SYLLABLE NEE; 0BA8 0BC7
-TAMIL SYLLABLE NAI; 0BA8 0BC8
-TAMIL SYLLABLE NO;  0BA8 0BCA
-TAMIL SYLLABLE NOO; 0BA8 0BCB
-TAMIL SYLLABLE NAU; 0BA8 0BCC
-TAMIL SYLLABLE PAA; 0BAA 0BBE
-TAMIL SYLLABLE PI;  0BAA 0BBF
-TAMIL SYLLABLE PII; 0BAA 0BC0
-TAMIL SYLLABLE PU;  0BAA 0BC1
-TAMIL SYLLABLE PUU; 0BAA 0BC2
-TAMIL SYLLABLE PE;  0BAA 0BC6
-TAMIL SYLLABLE PEE; 0BAA 0BC7
-TAMIL SYLLABLE PAI; 0BAA 0BC8
-TAMIL SYLLABLE PO;  0BAA 0BCA
-TAMIL SYLLABLE POO; 0BAA 0BCB
-TAMIL SYLLABLE PAU; 0BAA 0BCC
-TAMIL SYLLABLE MAA; 0BAE 0BBE
-TAMIL SYLLABLE MI;  0BAE 0BBF
-TAMIL SYLLABLE MII; 0BAE 0BC0
-TAMIL SYLLABLE MU;  0BAE 0BC1
-TAMIL SYLLABLE MUU; 0BAE 0BC2
-TAMIL SYLLABLE ME;  0BAE 0BC6
-TAMIL SYLLABLE MEE; 0BAE 0BC7
-TAMIL SYLLABLE MAI; 0BAE 0BC8
-TAMIL SYLLABLE MO;  0BAE 0BCA
-TAMIL SYLLABLE MOO; 0BAE 0BCB
-TAMIL SYLLABLE MAU; 0BAE 0BCC
-TAMIL SYLLABLE YAA; 0BAF 0BBE
-TAMIL SYLLABLE YI;  0BAF 0BBF
-TAMIL SYLLABLE YII; 0BAF 0BC0
-TAMIL SYLLABLE YU;  0BAF 0BC1
-TAMIL SYLLABLE YUU; 0BAF 0BC2
-TAMIL SYLLABLE YE;  0BAF 0BC6
-TAMIL SYLLABLE YEE; 0BAF 0BC7
-TAMIL SYLLABLE YAI; 0BAF 0BC8
-TAMIL SYLLABLE YO;  0BAF 0BCA
-TAMIL SYLLABLE YOO; 0BAF 0BCB
-TAMIL SYLLABLE YAU; 0BAF 0BCC
-TAMIL SYLLABLE RAA; 0BB0 0BBE
-TAMIL SYLLABLE RI;  0BB0 0BBF
-TAMIL SYLLABLE RII; 0BB0 0BC0
-TAMIL SYLLABLE RU;  0BB0 0BC1
-TAMIL SYLLABLE RUU; 0BB0 0BC2
-TAMIL SYLLABLE RE;  0BB0 0BC6
-TAMIL SYLLABLE REE; 0BB0 0BC7
-TAMIL SYLLABLE RAI; 0BB0 0BC8
-TAMIL SYLLABLE RO;  0BB0 0BCA
-TAMIL SYLLABLE ROO; 0BB0 0BCB
-TAMIL SYLLABLE RAU; 0BB0 0BCC
-TAMIL SYLLABLE LAA; 0BB2 0BBE
-TAMIL SYLLABLE LI;  0BB2 0BBF
-TAMIL SYLLABLE LII; 0BB2 0BC0
-TAMIL SYLLABLE LU;  0BB2 0BC1
-TAMIL SYLLABLE LUU; 0BB2 0BC2
-TAMIL SYLLABLE LE;  0BB2 0BC6
-TAMIL SYLLABLE LEE; 0BB2 0BC7
-TAMIL SYLLABLE LAI; 0BB2 0BC8
-TAMIL SYLLABLE LO;  0BB2 0BCA
-TAMIL SYLLABLE LOO; 0BB2 0BCB
-TAMIL SYLLABLE LAU; 0BB2 0BCC
-TAMIL SYLLABLE VAA; 0BB5 0BBE
-TAMIL SYLLABLE VI;  0BB5 0BBF
-TAMIL SYLLABLE VII; 0BB5 0BC0
-TAMIL SYLLABLE VU;  0BB5 0BC1
-TAMIL SYLLABLE VUU; 0BB5 0BC2
-TAMIL SYLLABLE VE;  0BB5 0BC6
-TAMIL SYLLABLE VEE; 0BB5 0BC7
-TAMIL SYLLABLE VAI; 0BB5 0BC8
-TAMIL SYLLABLE VO;  0BB5 0BCA
-TAMIL SYLLABLE VOO; 0BB5 0BCB
-TAMIL SYLLABLE VAU; 0BB5 0BCC
-TAMIL SYLLABLE LLLAA; 0BB4 0BBE
-TAMIL SYLLABLE LLLI;  0BB4 0BBF
-TAMIL SYLLABLE LLLII; 0BB4 0BC0
-TAMIL SYLLABLE LLLU;  0BB4 0BC1
-TAMIL SYLLABLE LLLUU; 0BB4 0BC2
-TAMIL SYLLABLE LLLE;  0BB4 0BC6
-TAMIL SYLLABLE LLLEE; 0BB4 0BC7
-TAMIL SYLLABLE LLLAI; 0BB4 0BC8
-TAMIL SYLLABLE LLLO;  0BB4 0BCA
-TAMIL SYLLABLE LLLOO; 0BB4 0BCB
-TAMIL SYLLABLE LLLAU; 0BB4 0BCC
-TAMIL SYLLABLE LLAA; 0BB3 0BBE
-TAMIL SYLLABLE LLI;  0BB3 0BBF
-TAMIL SYLLABLE LLII; 0BB3 0BC0
-TAMIL SYLLABLE LLU;  0BB3 0BC1
-TAMIL SYLLABLE LLUU; 0BB3 0BC2
-TAMIL SYLLABLE LLE;  0BB3 0BC6
-TAMIL SYLLABLE LLEE; 0BB3 0BC7
-TAMIL SYLLABLE LLAI; 0BB3 0BC8
-TAMIL SYLLABLE LLO;  0BB3 0BCA
-TAMIL SYLLABLE LLOO; 0BB3 0BCB
-TAMIL SYLLABLE LLAU; 0BB3 0BCC
-TAMIL SYLLABLE RRAA; 0BB1 0BBE
-TAMIL SYLLABLE RRI;  0BB1 0BBF
-TAMIL SYLLABLE RRII; 0BB1 0BC0
-TAMIL SYLLABLE RRU;  0BB1 0BC1
-TAMIL SYLLABLE RRUU; 0BB1 0BC2
-TAMIL SYLLABLE RRE;  0BB1 0BC6
-TAMIL SYLLABLE RREE; 0BB1 0BC7
-TAMIL SYLLABLE RRAI; 0BB1 0BC8
-TAMIL SYLLABLE RRO;  0BB1 0BCA
-TAMIL SYLLABLE RROO; 0BB1 0BCB
-TAMIL SYLLABLE RRAU; 0BB1 0BCC
-TAMIL SYLLABLE NNNAA; 0BA9 0BBE
-TAMIL SYLLABLE NNNI;  0BA9 0BBF
-TAMIL SYLLABLE NNNII; 0BA9 0BC0
-TAMIL SYLLABLE NNNU;  0BA9 0BC1
-TAMIL SYLLABLE NNNUU; 0BA9 0BC2
-TAMIL SYLLABLE NNNE;  0BA9 0BC6
-TAMIL SYLLABLE NNNEE; 0BA9 0BC7
-TAMIL SYLLABLE NNNAI; 0BA9 0BC8
-TAMIL SYLLABLE NNNO;  0BA9 0BCA
-TAMIL SYLLABLE NNNOO; 0BA9 0BCB
-TAMIL SYLLABLE NNNAU; 0BA9 0BCC
-TAMIL SYLLABLE JAA; 0B9C 0BBE
-TAMIL SYLLABLE JI;  0B9C 0BBF
-TAMIL SYLLABLE JII; 0B9C 0BC0
-TAMIL SYLLABLE JU;  0B9C 0BC1
-TAMIL SYLLABLE JUU; 0B9C 0BC2
-TAMIL SYLLABLE JE;  0B9C 0BC6
-TAMIL SYLLABLE JEE; 0B9C 0BC7
-TAMIL SYLLABLE JAI; 0B9C 0BC8
-TAMIL SYLLABLE JO;  0B9C 0BCA
-TAMIL SYLLABLE JOO; 0B9C 0BCB
-TAMIL SYLLABLE JAU; 0B9C 0BCC
-TAMIL SYLLABLE SHAA; 0BB6 0BBE
-TAMIL SYLLABLE SHI;  0BB6 0BBF
-TAMIL SYLLABLE SHII; 0BB6 0BC0
-TAMIL SYLLABLE SHU;  0BB6 0BC1
-TAMIL SYLLABLE SHUU; 0BB6 0BC2
-TAMIL SYLLABLE SHE;  0BB6 0BC6
-TAMIL SYLLABLE SHEE; 0BB6 0BC7
-TAMIL SYLLABLE SHAI; 0BB6 0BC8
-TAMIL SYLLABLE SHO;  0BB6 0BCA
-TAMIL SYLLABLE SHOO; 0BB6 0BCB
-TAMIL SYLLABLE SHAU; 0BB6 0BCC
-TAMIL SYLLABLE SSAA; 0BB7 0BBE
-TAMIL SYLLABLE SSI;  0BB7 0BBF
-TAMIL SYLLABLE SSII; 0BB7 0BC0
-TAMIL SYLLABLE SSU;  0BB7 0BC1
-TAMIL SYLLABLE SSUU; 0BB7 0BC2
-TAMIL SYLLABLE SSE;  0BB7 0BC6
-TAMIL SYLLABLE SSEE; 0BB7 0BC7
-TAMIL SYLLABLE SSAI; 0BB7 0BC8
-TAMIL SYLLABLE SSO;  0BB7 0BCA
-TAMIL SYLLABLE SSOO; 0BB7 0BCB
-TAMIL SYLLABLE SSAU; 0BB7 0BCC
-TAMIL SYLLABLE SAA; 0BB8 0BBE
-TAMIL SYLLABLE SI;  0BB8 0BBF
-TAMIL SYLLABLE SII; 0BB8 0BC0
-TAMIL SYLLABLE SU;  0BB8 0BC1
-TAMIL SYLLABLE SUU; 0BB8 0BC2
-TAMIL SYLLABLE SE;  0BB8 0BC6
-TAMIL SYLLABLE SEE; 0BB8 0BC7
-TAMIL SYLLABLE SAI; 0BB8 0BC8
-TAMIL SYLLABLE SO;  0BB8 0BCA
-TAMIL SYLLABLE SOO; 0BB8 0BCB
-TAMIL SYLLABLE SAU; 0BB8 0BCC
-TAMIL SYLLABLE HAA; 0BB9 0BBE
-TAMIL SYLLABLE HI;  0BB9 0BBF
-TAMIL SYLLABLE HII; 0BB9 0BC0
-TAMIL SYLLABLE HU;  0BB9 0BC1
-TAMIL SYLLABLE HUU; 0BB9 0BC2
-TAMIL SYLLABLE HE;  0BB9 0BC6
-TAMIL SYLLABLE HEE; 0BB9 0BC7
-TAMIL SYLLABLE HAI; 0BB9 0BC8
-TAMIL SYLLABLE HO;  0BB9 0BCA
-TAMIL SYLLABLE HOO; 0BB9 0BCB
-TAMIL SYLLABLE HAU; 0BB9 0BCC
-TAMIL SYLLABLE KSSA;  0B95 0BCD 0BB7
-TAMIL SYLLABLE KSSAA; 0B95 0BCD 0BB7 0BBE
-TAMIL SYLLABLE KSSI;  0B95 0BCD 0BB7 0BBF
-TAMIL SYLLABLE KSSII; 0B95 0BCD 0BB7 0BC0
-TAMIL SYLLABLE KSSU;  0B95 0BCD 0BB7 0BC1
-TAMIL SYLLABLE KSSUU; 0B95 0BCD 0BB7 0BC2
-TAMIL SYLLABLE KSSE;  0B95 0BCD 0BB7 0BC6
-TAMIL SYLLABLE KSSEE; 0B95 0BCD 0BB7 0BC7
-TAMIL SYLLABLE KSSAI; 0B95 0BCD 0BB7 0BC8
-TAMIL SYLLABLE KSSO;  0B95 0BCD 0BB7 0BCA
-TAMIL SYLLABLE KSSOO; 0B95 0BCD 0BB7 0BCB
-TAMIL SYLLABLE KSSAU; 0B95 0BCD 0BB7 0BCC
-TAMIL SYLLABLE SHRII; 0BB6 0BCD 0BB0 0BC0
-SINHALA CONSONANT SIGN YANSAYA;0DCA 200D 0DBA
-SINHALA CONSONANT SIGN RAKAARAANSAYA;0DCA 200D 0DBB
-SINHALA CONSONANT SIGN REPAYA;0DBB 0DCA 200D
-GEORGIAN LETTER U-BRJGU;10E3 0302
-KHMER CONSONANT SIGN COENG KA;17D2 1780
-KHMER CONSONANT SIGN COENG KHA;17D2 1781
-KHMER CONSONANT SIGN COENG KO;17D2 1782
-KHMER CONSONANT SIGN COENG KHO;17D2 1783
-KHMER CONSONANT SIGN COENG NGO;17D2 1784
-KHMER CONSONANT SIGN COENG CA;17D2 1785
-KHMER CONSONANT SIGN COENG CHA;17D2 1786
-KHMER CONSONANT SIGN COENG CO;17D2 1787
-KHMER CONSONANT SIGN COENG CHO;17D2 1788
-KHMER CONSONANT SIGN COENG NYO;17D2 1789
-KHMER CONSONANT SIGN COENG DA;17D2 178A
-KHMER CONSONANT SIGN COENG TTHA;17D2 178B
-KHMER CONSONANT SIGN COENG DO;17D2 178C
-KHMER CONSONANT SIGN COENG TTHO;17D2 178D
-KHMER CONSONANT SIGN COENG NA;17D2 178E
-KHMER CONSONANT SIGN COENG TA;17D2 178F
-KHMER CONSONANT SIGN COENG THA;17D2 1790
-KHMER CONSONANT SIGN COENG TO;17D2 1791
-KHMER CONSONANT SIGN COENG THO;17D2 1792
-KHMER CONSONANT SIGN COENG NO;17D2 1793
-KHMER CONSONANT SIGN COENG BA;17D2 1794
-KHMER CONSONANT SIGN COENG PHA;17D2 1795
-KHMER CONSONANT SIGN COENG PO;17D2 1796
-KHMER CONSONANT SIGN COENG PHO;17D2 1797
-KHMER CONSONANT SIGN COENG MO;17D2 1798
-KHMER CONSONANT SIGN COENG YO;17D2 1799
-KHMER CONSONANT SIGN COENG RO;17D2 179A
-KHMER CONSONANT SIGN COENG LO;17D2 179B
-KHMER CONSONANT SIGN COENG VO;17D2 179C
-KHMER CONSONANT SIGN COENG SHA;17D2 179D
-KHMER CONSONANT SIGN COENG SSA;17D2 179E
-KHMER CONSONANT SIGN COENG SA;17D2 179F
-KHMER CONSONANT SIGN COENG HA;17D2 17A0
-KHMER CONSONANT SIGN COENG LA;17D2 17A1
-KHMER VOWEL SIGN COENG QA;17D2 17A2
-KHMER INDEPENDENT VOWEL SIGN COENG QU;17D2 17A7
-KHMER INDEPENDENT VOWEL SIGN COENG RY;17D2 17AB
-KHMER INDEPENDENT VOWEL SIGN COENG RYY;17D2 17AC
-KHMER INDEPENDENT VOWEL SIGN COENG QE;17D2 17AF
-KHMER VOWEL SIGN OM;17BB 17C6
-KHMER VOWEL SIGN AAM;17B6 17C6
-HIRAGANA LETTER BIDAKUON NGA;304B 309A
-HIRAGANA LETTER BIDAKUON NGI;304D 309A
-HIRAGANA LETTER BIDAKUON NGU;304F 309A
-HIRAGANA LETTER BIDAKUON NGE;3051 309A
-HIRAGANA LETTER BIDAKUON NGO;3053 309A
-KATAKANA LETTER BIDAKUON NGA;30AB 309A
-KATAKANA LETTER BIDAKUON NGI;30AD 309A
-KATAKANA LETTER BIDAKUON NGU;30AF 309A
-KATAKANA LETTER BIDAKUON NGE;30B1 309A
-KATAKANA LETTER BIDAKUON NGO;30B3 309A
-KATAKANA LETTER AINU CE;30BB 309A
-KATAKANA LETTER AINU TU;30C4 309A
-KATAKANA LETTER AINU TO;30C8 309A
-KATAKANA LETTER AINU P;31F7 309A
-MODIFIER LETTER EXTRA-HIGH EXTRA-LOW CONTOUR TONE BAR;02E5 02E9
-MODIFIER LETTER EXTRA-LOW EXTRA-HIGH CONTOUR TONE BAR;02E9 02E5

lib/elixir/rebar.config

@@ -24,4 +24,4 @@
             {verbose, false}
            ]}.
 
-{require_otp_vsn,"(R16B).*"}.
+{require_otp_vsn,"(R16B|R17A).*"}.

lib/elixir/src/elixir.erl

@@ -1,10 +1,11 @@
+%% Main entry point for Elixir functions. All of those functions are
+%% private to the Elixir compiler and reserved to be used by Elixir only.
 -module(elixir).
 -behaviour(application).
 -export([main/1, start_cli/0,
-  scope_for_eval/1, scope_for_eval/2,
-  eval/2, eval/3, eval/4,
-  eval_quoted/2, eval_quoted/3, eval_quoted/4,
-  eval_forms/3, translate_forms/3]).
+  string_to_quoted/4, 'string_to_quoted!'/4,
+  env_for_eval/1, env_for_eval/2, quoted_to_erl/2, quoted_to_erl/3,
+  eval/2, eval/3, eval_forms/3, eval_forms/4, eval_quoted/3]).
 -include("elixir.hrl").
 
 %% Top level types
@@ -12,7 +13,7 @@
 -type char_list() :: string().
 -type as_boolean(T) :: T.
 
-% OTP APPLICATION API
+%% OTP Application API
 
 -export([start/2, stop/1, config_change/3]).
 
@@ -23,7 +24,7 @@ start(_Type, _Args) ->
     io:setopts(standard_io, [binary,{encoding,unicode}]),
     io:setopts(standard_error, [binary,{encoding,unicode}])
   end),
-  elixir_sup:start_link([]).
+  elixir_sup:start_link().
 
 stop(_S) ->
   ok.
@@ -45,96 +46,129 @@ start_cli() ->
 
 %% EVAL HOOKS
 
-scope_for_eval(Opts) ->
-  scope_for_eval(#elixir_scope{
-    file = <<"nofile">>,
+env_for_eval(Opts) ->
+  env_for_eval(#elixir_env{
     local = nil,
-    aliases = [],
     requires = elixir_dispatch:default_requires(),
     functions = elixir_dispatch:default_functions(),
     macros = elixir_dispatch:default_macros()
   }, Opts).
 
-scope_for_eval(Scope, Opts) ->
+env_for_eval(Env, Opts) ->
+  Line = case lists:keyfind(line, 1, Opts) of
+    { line, RawLine } when is_integer(RawLine) -> RawLine;
+    false -> Env#elixir_env.line
+  end,
+
   File = case lists:keyfind(file, 1, Opts) of
     { file, RawFile } when is_binary(RawFile) -> RawFile;
-    false -> Scope#elixir_scope.file
+    false -> Env#elixir_env.file
   end,
 
   Local = case lists:keyfind(delegate_locals_to, 1, Opts) of
     { delegate_locals_to, LocalOpt } -> LocalOpt;
-    false -> Scope#elixir_scope.local
+    false -> Env#elixir_env.local
   end,
 
   Aliases = case lists:keyfind(aliases, 1, Opts) of
     { aliases, AliasesOpt } -> AliasesOpt;
-    false -> Scope#elixir_scope.aliases
+    false -> Env#elixir_env.aliases
   end,
 
   Requires = case lists:keyfind(requires, 1, Opts) of
     { requires, List } -> ordsets:from_list(List);
-    false -> Scope#elixir_scope.requires
+    false -> Env#elixir_env.requires
   end,
 
   Functions = case lists:keyfind(functions, 1, Opts) of
     { functions, FunctionsOpt } -> FunctionsOpt;
-    false -> Scope#elixir_scope.functions
+    false -> Env#elixir_env.functions
   end,
 
   Macros = case lists:keyfind(macros, 1, Opts) of
     { macros, MacrosOpt } -> MacrosOpt;
-    false -> Scope#elixir_scope.macros
+    false -> Env#elixir_env.macros
   end,
 
-  Scope#elixir_scope{
-    file=File, local=Local,
+  Module = case lists:keyfind(module, 1, Opts) of
+    { module, ModuleOpt } when is_atom(ModuleOpt) -> ModuleOpt;
+    false -> nil
+  end,
+
+  Env#elixir_env{
+    file=File, local=Local, module=Module,
     macros=Macros, functions=Functions,
-    requires=Requires, aliases=Aliases }.
+    requires=Requires, aliases=Aliases, line=Line
+  }.
 
 %% String evaluation
 
-eval(String, Binding) -> eval(String, Binding, []).
+eval(String, Binding) ->
+  eval(String, Binding, []).
 
-eval(String, Binding, Opts) ->
-  case lists:keyfind(line, 1, Opts) of
-    false -> Line = 1;
-    { line, Line } -> []
-  end,
-  eval(String, Binding, Line, scope_for_eval(Opts)).
-
-eval(String, Binding, Line, #elixir_scope{file=File} = S) when
+eval(String, Binding, Opts) when is_list(Opts) ->
+  eval(String, Binding, env_for_eval(Opts));
+eval(String, Binding, #elixir_env{line=Line,file=File} = E) when
     is_list(String), is_list(Binding), is_integer(Line), is_binary(File) ->
-  Forms = elixir_translator:'forms!'(String, Line, File, []),
-  eval_forms(Forms, Binding, S).
+  Forms = 'string_to_quoted!'(String, Line, File, []),
+  eval_forms(Forms, Binding, E).
 
 %% Quoted evaluation
 
-eval_quoted(Tree, Binding) -> eval_quoted(Tree, Binding, []).
+eval_quoted(Tree, Binding, Opts) when is_list(Opts) ->
+  eval_quoted(Tree, Binding, env_for_eval(Opts));
+eval_quoted(Tree, Binding, #elixir_env{line=Line} = E) ->
+  eval_forms(elixir_quote:linify(Line, Tree), Binding, E).
 
-eval_quoted(Tree, Binding, Opts) ->
-  case lists:keyfind(line, 1, Opts) of
-    { line, Line } -> [];
-    false -> Line = 1
-  end,
-  eval_quoted(Tree, Binding, Line, scope_for_eval(Opts)).
+%% Handle forms evaluation. The main difference to
+%% to eval_quoted is that it does not linefy the given
+%% args.
 
-eval_quoted(Tree, Binding, Line, #elixir_scope{} = S) when is_integer(Line) ->
-  eval_forms(elixir_quote:linify(Line, Tree), Binding, S).
+eval_forms(Tree, Binding, #elixir_env{} = E) ->
+  eval_forms(Tree, Binding, E, elixir_env:env_to_scope(E));
+eval_forms(Tree, Binding, Opts) when is_list(Opts) ->
+  eval_forms(Tree, Binding, env_for_eval(Opts)).
+eval_forms(Tree, Binding, Env, Scope) ->
+  { ParsedBinding, ParsedScope } = elixir_scope:load_binding(Binding, Scope),
+  ParsedEnv = Env#elixir_env{vars=[K || {K,_} <- ParsedScope#elixir_scope.vars]},
+  { Erl, NewEnv, NewScope } = quoted_to_erl(Tree, ParsedEnv, ParsedScope),
 
-%% Handle forms evaluation internally, it is an
-%% internal API not meant for external usage.
-
-translate_forms(Tree, Binding, Opts) when is_list(Opts) ->
-  translate_forms(Tree, Binding, scope_for_eval(Opts));
-
-translate_forms(Tree, Binding, #elixir_scope{} = Scope) ->
-  elixir_translator:translate(Tree, elixir_scope:vars_from_binding(Scope, Binding)).
-
-eval_forms(Tree, Binding, Scope) ->
-  { ParseTree, NewScope } = translate_forms(Tree, Binding, Scope),
-  case ParseTree of
-    [] -> { nil, Binding, NewScope };
+  case Erl of
+    { atom, _, Atom } ->
+      { Atom, Binding, NewEnv, NewScope };
     _  ->
-      {value, Value, NewBinding} = erl_eval:exprs(ParseTree, elixir_scope:binding_for_eval(Binding, nil)),
-      {Value, elixir_scope:binding_from_vars(NewScope, NewBinding), NewScope }
+      { value, Value, NewBinding } = erl_eval:expr(Erl, ParsedBinding),
+      { Value, elixir_scope:dump_binding(NewBinding, NewScope), NewEnv, NewScope }
+  end.
+
+%% Converts a quoted expression to erlang abstract format
+
+quoted_to_erl(Quoted, Env) ->
+  quoted_to_erl(Quoted, Env, elixir_env:env_to_scope(Env)).
+
+quoted_to_erl(Quoted, Env, Scope) ->
+  { Expanded, NewEnv } = elixir_exp:expand(Quoted, Env),
+  { Erl, NewScope }    = elixir_translator:translate(Expanded, Scope),
+  { Erl, NewEnv, NewScope }.
+
+%% Converts a given string (char list) into quote expression
+
+string_to_quoted(String, StartLine, File, Opts) when is_integer(StartLine), is_binary(File) ->
+  case elixir_tokenizer:tokenize(String, StartLine, [{ file, File }|Opts]) of
+    { ok, _Line, Tokens } ->
+      try elixir_parser:parse(Tokens) of
+        { ok, Forms } -> { ok, Forms };
+        { error, { Line, _, [Error, Token] } } -> { error, { Line, Error, Token } }
+      catch
+        { error, { Line, _, [Error, Token] } } -> { error, { Line, Error, Token } }
+      end;
+    { error, Reason, _Rest, _SoFar } -> { error, Reason }
+  end.
+
+'string_to_quoted!'(String, StartLine, File, Opts) ->
+  case string_to_quoted(String, StartLine, File, Opts) of
+    { ok, Forms } ->
+      Forms;
+    { error, { Line, Error, Token } } ->
+      elixir_errors:parse_error(Line, File, Error, Token)
   end.

lib/elixir/src/elixir_aliases.erl

@@ -1,85 +1,90 @@
 -module(elixir_aliases).
--export([nesting_alias/2, last/1, concat/1, safe_concat/1,
-  format_error/1, ensure_loaded/3, ensure_loaded/4, expand/3, store/4]).
+-export([inspect/1, last/1, concat/1, safe_concat/1, format_error/1,
+         ensure_loaded/3, expand/4, store/7]).
 -include("elixir.hrl").
 
-%% Store an alias in the given scope
-store(_Meta, New, New, S) -> S;
-store(Meta, New, Old, S) ->
-  SA = S#elixir_scope{
-    aliases=orddict:store(New, Old, S#elixir_scope.aliases)
-  },
-
-  case lists:keymember(context, 1, Meta) of
-    true ->
-      SA#elixir_scope{
-        macro_aliases=orddict:store(New, Old, S#elixir_scope.macro_aliases)
-      };
-    false ->
-      SA
+inspect(Atom) when is_atom(Atom) ->
+  case elixir_compiler:get_opt(internal) of
+    true -> atom_to_binary(Atom, utf8);
+    false -> 'Elixir.Inspect.Atom':inspect(Atom)
   end.
 
+%% Store an alias in the given scope
+store(_Meta, New, New, _TKV, Aliases, MacroAliases, _Lexical) ->
+  { Aliases, MacroAliases };
+store(Meta, New, Old, TKV, Aliases, MacroAliases, Lexical) ->
+  record_warn(Meta, New, TKV, Lexical),
+  { store_alias(New, Old, Aliases),
+    store_macro_alias(Meta, New, Old, MacroAliases) }.
+
+store_alias(New, Old, Aliases) ->
+  lists:keystore(New, 1, Aliases, { New, Old }).
+store_macro_alias(Meta, New, Old, Aliases) ->
+  case lists:keymember(context, 1, Meta) andalso
+       lists:keyfind(counter, 1, Meta) of
+    { counter, Counter } when is_integer(Counter) ->
+      lists:keystore(New, 1, Aliases, { New, { Counter, Old } });
+    _ ->
+      Aliases
+  end.
+
+record_warn(Meta, Ref, Opts, Lexical) ->
+  Warn =
+    case lists:keyfind(warn, 1, Opts) of
+      { warn, false } -> false;
+      { warn, true } -> true;
+      false -> not lists:keymember(context, 1, Meta)
+    end,
+  elixir_lexical:record_alias(Ref, ?line(Meta), Warn, Lexical).
+
 %% Expand an alias. It returns an atom (meaning that there
 %% was an expansion) or a list of atoms.
 
-expand({ '__aliases__', Meta, _ } = Alias, Aliases, MacroAliases) ->
+expand({ '__aliases__', _Meta, ['Elixir'|_] = List }, _Aliases, _MacroAliases, _LexicalTracker) ->
+  concat(List);
+
+expand({ '__aliases__', Meta, _ } = Alias, Aliases, MacroAliases, LexicalTracker) ->
   case lists:keyfind(alias, 1, Meta) of
     { alias, false } ->
-      expand(Alias, MacroAliases);
+      expand(Alias, MacroAliases, LexicalTracker);
     { alias, Atom } when is_atom(Atom) ->
-      case expand(Alias, MacroAliases) of
-        OtherAtom when is_atom(OtherAtom) -> OtherAtom;
-        OtherAliases when is_list(OtherAliases) -> Atom
-      end;
+      Atom;
     false ->
-      expand(Alias, Aliases)
+      expand(Alias, Aliases, LexicalTracker)
   end.
 
-expand({ '__aliases__', _Meta, [H] }, Aliases) when H /= 'Elixir' ->
-  case expand_one(H, Aliases) of
-    false -> [H];
-    Atom  -> Atom
-  end;
-
-expand({ '__aliases__', _Meta, [H|T] }, Aliases) when is_atom(H) ->
-  case H of
-    'Elixir' ->
-      concat(T);
-    _ ->
-      case expand_one(H, Aliases) of
-        false -> [H|T];
-        Atom  -> concat([Atom|T])
+expand({ '__aliases__', Meta, [H|T] }, Aliases, LexicalTracker) when is_atom(H) ->
+  Lookup  = list_to_atom("Elixir." ++ atom_to_list(H)),
+  Counter = case lists:keyfind(counter, 1, Meta) of
+    { counter, C } -> C;
+    _ -> nil
+  end,
+  case lookup(Lookup, Aliases, Counter) of
+    Lookup -> [H|T];
+    Atom ->
+      elixir_lexical:record_alias(Lookup, LexicalTracker),
+      case T of
+        [] -> Atom;
+        _  -> concat([Atom|T])
       end
   end;
 
-expand({ '__aliases__', _Meta, List }, _Aliases) ->
+expand({ '__aliases__', _Meta, List }, _Aliases, _LexicalTracker) ->
   List.
 
-expand_one(H, Aliases) ->
-  Lookup = list_to_atom("Elixir." ++ atom_to_list(H)),
-  case lookup(Lookup, Aliases) of
-    Lookup -> false;
-    Else   -> Else
-  end.
-
 %% Ensure a module is loaded before its usage.
 
-ensure_loaded(Line, Ref, S) ->
-  ensure_loaded(Line, S#elixir_scope.file, Ref, S#elixir_scope.context_modules).
-
-ensure_loaded(_Line, _File, 'Elixir.Kernel', _FileModules) ->
-  ok;
-
-ensure_loaded(Line, File, Ref, FileModules) ->
+ensure_loaded(_Meta, 'Elixir.Kernel', _E) -> ok;
+ensure_loaded(Meta, Ref, E) ->
   try
     Ref:module_info(compile)
   catch
     error:undef ->
-      Kind = case lists:member(Ref, FileModules) of
+      Kind = case lists:member(Ref, E#elixir_env.context_modules) of
         true  -> scheduled_module;
         false -> unloaded_module
       end,
-      elixir_errors:form_error(Line, File, ?MODULE, { Kind, Ref })
+      elixir_errors:form_error(Meta, E#elixir_env.file, ?MODULE, { Kind, Ref })
   end.
 
 %% Receives an atom and returns the last bit as an alias.
@@ -92,74 +97,48 @@ last([$.|_], Acc) -> Acc;
 last([H|T], Acc) -> last(T, [H|Acc]);
 last([], Acc) -> Acc.
 
-%% Gets two modules names and return an alias
-%% which can be passed down to the alias directive
-%% and it will create a proper shortcut representing
-%% the given nesting.
-%%
-%% Examples:
-%%
-%%     nesting_alias('Elixir.Foo.Bar', 'Elixir.Foo.Bar.Baz.Bat')
-%%     { 'Elixir.Baz', 'Elixir.Foo.Bar.Baz' }
-%%
-%% When passed to alias, the example above will generate an
-%% alias like:
-%%
-%%     'Elixir.Baz' => 'Elixir.Foo.Bar.Baz'
-%%
-nesting_alias(nil, _Full) -> false;
-
-nesting_alias(Prefix, Full) ->
-  PrefixList = list_nesting(Prefix),
-  FullList   = list_nesting(Full),
-  (PrefixList /= []) andalso do_nesting(PrefixList, FullList, []).
-
-do_nesting([X|PreTail], [X|Tail], Acc) ->
-  do_nesting(PreTail, Tail, [X|Acc]);
-do_nesting([], [H|_], Acc) ->
-  { list_to_atom("Elixir." ++ H), concat(lists:reverse([H|Acc])) };
-do_nesting(_, _, _Acc) ->
-  false.
-
-list_nesting(Atom) ->
-  case string:tokens(atom_to_list(Atom), ".") of
-    ["Elixir"|T] -> T;
-    _ -> []
-  end.
-
 %% Receives a list of atoms, binaries or lists
 %% representing modules and concatenates them.
 
-concat(Args) -> list_to_atom(raw_concat(Args)).
-safe_concat(Args) -> list_to_existing_atom(raw_concat(Args)).
+concat(Args)      -> binary_to_atom(do_concat(Args), utf8).
+safe_concat(Args) -> binary_to_existing_atom(do_concat(Args), utf8).
 
-raw_concat(['Elixir'|Args]) -> do_concat(Args);
-raw_concat([nil|Args])      -> do_concat(Args);
-raw_concat(Args)            -> do_concat(Args).
+do_concat([H|T]) when is_atom(H), H /= nil ->
+  do_concat([atom_to_binary(H, utf8)|T]);
+do_concat([<<"Elixir.", _/binary>>=H|T]) ->
+  do_concat(T, H);
+do_concat([<<"Elixir">>=H|T]) ->
+  do_concat(T, H);
+do_concat(T) ->
+  do_concat(T, <<"Elixir">>).
 
-do_concat(Args) ->
-  Aliases = [to_partial(Arg) || Arg <- Args],
-  "Elixir" ++ lists:concat(Aliases).
+do_concat([nil|T], Acc) ->
+  do_concat(T, Acc);
+do_concat([H|T], Acc) when is_atom(H) ->
+  do_concat(T, <<Acc/binary, $., (to_partial(atom_to_binary(H, utf8)))/binary>>);
+do_concat([H|T], Acc) when is_binary(H) ->
+  do_concat(T, <<Acc/binary, $., (to_partial(H))/binary>>);
+do_concat([], Acc) ->
+  Acc.
 
-to_partial(Arg) when is_binary(Arg) -> to_partial(binary_to_list(Arg));
-to_partial(Arg) when is_atom(Arg)   -> to_partial(atom_to_list(Arg));
-to_partial("Elixir." ++ Arg)        -> [$.|Arg];
-to_partial([$.|_] = Arg)            -> Arg;
-to_partial(Arg) when is_list(Arg)   -> [$.|Arg].
+to_partial(<<"Elixir.", Arg/binary>>) -> Arg;
+to_partial(<<".", Arg/binary>>)       -> Arg;
+to_partial(Arg) when is_binary(Arg)   -> Arg.
 
 %% Lookup an alias in the current scope.
 
-lookup(Else, Dict) ->
-  case orddict:find(Else, Dict) of
-    { ok, Value } when Value /= Else -> lookup(Value, Dict);
+lookup(Else, Dict, Counter) ->
+  case lists:keyfind(Else, 1, Dict) of
+    { Else, { Counter, Value } } -> lookup(Value, Dict, Counter);
+    { Else, Value } when is_atom(Value) -> lookup(Value, Dict, Counter);
     _ -> Else
   end.
 
 %% Errors
 
 format_error({unloaded_module, Module}) ->
-  io_lib:format("module ~ts is not loaded and could not be found", [elixir_errors:inspect(Module)]);
+  io_lib:format("module ~ts is not loaded and could not be found", [elixir_aliases:inspect(Module)]);
 
 format_error({scheduled_module, Module}) ->
   io_lib:format("module ~ts is not loaded but was defined. This happens because you are trying to use a module in the same context it is defined. Try defining the module outside the context that requires it.",
-    [elixir_errors:inspect(Module)]).
+    [inspect(Module)]).

lib/elixir/src/elixir_bitstring.erl

@@ -0,0 +1,202 @@
+%% Handle translation of Elixir literals to Erlang AST.
+-module(elixir_bitstring).
+-export([translate/3, expand/3]).
+-include("elixir.hrl").
+
+%% Expansion
+
+expand(Meta, Args, E) ->
+  case E#elixir_env.context of
+    match ->
+      { EArgs, EA } = expand_bitstr(fun elixir_exp:expand/2, Args, [], E),
+      { { '<<>>', Meta, EArgs }, EA };
+    _ ->
+      { EArgs, { EC, EV } } = expand_bitstr(fun elixir_exp:expand_arg/2, Args, [], { E, E }),
+      { { '<<>>', Meta, EArgs }, elixir_env:mergea(EV, EC) }
+  end.
+
+expand_bitstr(_Fun, [], Acc, E) ->
+  { lists:reverse(Acc), E };
+expand_bitstr(Fun, [{'::',Meta,[Left,Right]}|T], Acc, E) ->
+  { ELeft, EL } = Fun(Left, E),
+
+  %% Variables defined outside the binary can be accounted
+  %% on subparts, however we can't assign new variables.
+  case E of
+    { ER, _ } -> ok;                    %% expand_arg,  no assigns
+    _ -> ER = E#elixir_env{context=nil} %% expand_each, revert assigns
+  end,
+
+  ERight = expand_bit_info(Meta, Right, ER),
+  expand_bitstr(Fun, T, [{'::',Meta,[ELeft,ERight]}|Acc], EL);
+
+expand_bitstr(Fun, [H|T], Acc, E) ->
+  { Expr, ES } = Fun(H, E),
+  expand_bitstr(Fun, T, [Expr|Acc], ES).
+
+%% Expand bit info
+
+expand_bit_info(Meta, Info, E) when is_list(Info) ->
+  expand_bit_info(Meta, Info, default, [], E);
+
+expand_bit_info(Meta, Info, E) ->
+  expand_bit_info(Meta, [Info], E).
+
+expand_bit_info(Meta, [{ Expr, ExprMeta, Args }|T], Size, Types, E) when is_atom(Expr) ->
+  ListArgs = if is_atom(Args) -> []; is_list(Args) -> Args end,
+  case expand_bit_type_or_size(Expr, ListArgs) of
+    type ->
+      { EArgs, EE } = elixir_exp:expand_args(ListArgs, E),
+      expand_bit_info(Meta, T, Size, [{ Expr, [], EArgs }|Types], EE);
+    size ->
+      case Size of
+        default -> ok;
+        _ -> elixir_errors:compile_error(Meta, E#elixir_env.file, "duplicated size definition in bitstring")
+      end,
+      { EArgs, EE } = elixir_exp:expand_args(ListArgs, E),
+      expand_bit_info(Meta, T, { Expr, [], EArgs }, Types, EE);
+    none ->
+      handle_unknown_bit_info(Meta, { Expr, ExprMeta, ListArgs }, T, Size, Types, E)
+  end;
+
+expand_bit_info(Meta, [Int|T], Size, Types, E) when is_integer(Int) ->
+  expand_bit_info(Meta, [{ size, [], [Int] }|T], Size, Types, E);
+
+expand_bit_info(Meta, [Expr|_], _Size, _Types, E) ->
+  elixir_errors:compile_error(Meta, E#elixir_env.file,
+    "unknown bitstring specifier ~ts", ['Elixir.Kernel':inspect(Expr)]);
+
+expand_bit_info(_Meta, [], Size, Types, _) ->
+  case Size of
+    default -> lists:reverse(Types);
+    _ -> [Size|lists:reverse(Types)]
+  end.
+
+expand_bit_type_or_size(binary, [])    -> type;
+expand_bit_type_or_size(integer, [])   -> type;
+expand_bit_type_or_size(float, [])     -> type;
+expand_bit_type_or_size(bitstring, []) -> type;
+expand_bit_type_or_size(bytes, [])     -> type;
+expand_bit_type_or_size(bits, [])      -> type;
+expand_bit_type_or_size(utf8, [])      -> type;
+expand_bit_type_or_size(utf16, [])     -> type;
+expand_bit_type_or_size(utf32, [])     -> type;
+expand_bit_type_or_size(signed, [])    -> type;
+expand_bit_type_or_size(unsigned, [])  -> type;
+expand_bit_type_or_size(big, [])       -> type;
+expand_bit_type_or_size(little, [])    -> type;
+expand_bit_type_or_size(native, [])    -> type;
+expand_bit_type_or_size(unit, [_])     -> type;
+expand_bit_type_or_size(size, [_])     -> size;
+expand_bit_type_or_size(_, _)          -> none.
+
+handle_unknown_bit_info(Meta, { _, ExprMeta, _ } = Expr, T, Size, Types, E) ->
+  case 'Elixir.Macro':expand(Expr, elixir_env:env_to_ex({ ?line(ExprMeta), E })) of
+    Expr ->
+      elixir_errors:compile_error(ExprMeta, E#elixir_env.file,
+        "unknown bitstring specifier ~ts", ['Elixir.Macro':to_string(Expr)]);
+    Other ->
+      List = case is_list(Other) of true -> Other; false -> [Other] end,
+      expand_bit_info(Meta, List ++ T, Size, Types, E)
+  end.
+
+%% Translation
+
+translate(Meta, Args, S) ->
+  case S#elixir_scope.context of
+    match ->
+      build_bitstr(fun elixir_translator:translate/2, Args, Meta, S);
+    _ ->
+      build_bitstr(fun(X, Acc) -> elixir_translator:translate_arg(X, Acc, S) end, Args, Meta, S)
+  end.
+
+build_bitstr(Fun, Exprs, Meta, S) ->
+  { Final, FinalS } = build_bitstr_each(Fun, Exprs, Meta, S, []),
+  { { bin, ?line(Meta), lists:reverse(Final) }, FinalS }.
+
+build_bitstr_each(_Fun, [], _Meta, S, Acc) ->
+  { Acc, S };
+
+build_bitstr_each(Fun, [{'::',_,[H,V]}|T], Meta, S, Acc) ->
+  { Size, Types } = extract_bit_info(Meta, V, S#elixir_scope{context=nil}),
+  build_bitstr_each(Fun, T, Meta, S, Acc, H, Size, Types);
+
+build_bitstr_each(Fun, [H|T], Meta, S, Acc) ->
+  build_bitstr_each(Fun, T, Meta, S, Acc, H, default, default).
+
+build_bitstr_each(Fun, T, Meta, S, Acc, H, default, Types) when is_binary(H) ->
+  Element =
+    case types_allow_splice(Types, []) of
+      true ->
+        %% See explanation in elixir_utils:elixir_to_erl/1 to know
+        %% why we can simply convert the binary to a list.
+        { bin_element, ?line(Meta), { string, 0, binary_to_list(H) }, default, default };
+      false ->
+        case types_require_conversion(Types) of
+          true ->
+            { bin_element, ?line(Meta), { string, 0, elixir_utils:characters_to_list(H) }, default, Types };
+          false ->
+            elixir_errors:compile_error(Meta, S#elixir_scope.file, "invalid types for literal string in <<>>. "
+              "Accepted types are: little, big, utf8, utf16, utf32, bits, bytes, binary, bitstring")
+        end
+    end,
+
+  build_bitstr_each(Fun, T, Meta, S, [Element|Acc]);
+
+build_bitstr_each(_Fun, _T, Meta, S, _Acc, H, _Size, _Types) when is_binary(H) ->
+  elixir_errors:compile_error(Meta, S#elixir_scope.file, "size is not supported for literal string in <<>>");
+
+build_bitstr_each(_Fun, _T, Meta, S, _Acc, H, _Size, _Types) when is_list(H); is_atom(H) ->
+  elixir_errors:compile_error(Meta, S#elixir_scope.file, "invalid literal ~ts in <<>>",
+    ['Elixir.Macro':to_string(H)]);
+
+build_bitstr_each(Fun, T, Meta, S, Acc, H, Size, Types) ->
+  { Expr, NS } = Fun(H, S),
+
+  case Expr of
+    { bin, _, Elements } ->
+      case (Size == default) andalso types_allow_splice(Types, Elements) of
+        true  -> build_bitstr_each(Fun, T, Meta, NS, lists:reverse(Elements) ++ Acc);
+        false -> build_bitstr_each(Fun, T, Meta, NS, [{ bin_element, ?line(Meta), Expr, Size, Types }|Acc])
+      end;
+    _ ->
+      build_bitstr_each(Fun, T, Meta, NS, [{ bin_element, ?line(Meta), Expr, Size, Types }|Acc])
+  end.
+
+types_require_conversion([End|T]) when End == little; End == big -> types_require_conversion(T);
+types_require_conversion([UTF|T]) when UTF == utf8; UTF == utf16; UTF == utf32 -> types_require_conversion(T);
+types_require_conversion([]) -> true;
+types_require_conversion(_) -> false.
+
+types_allow_splice([bytes], Elements)  -> lists:all(fun has_default_size/1, Elements);
+types_allow_splice([binary], Elements) -> lists:all(fun has_default_size/1, Elements);
+types_allow_splice([bits], _)          -> true;
+types_allow_splice([bitstring], _)     -> true;
+types_allow_splice(default, _)         -> true;
+types_allow_splice(_, _)               -> false.
+
+has_default_size({ bin_element, _, _, default, _ }) -> true;
+has_default_size({ bin_element, _, _, _, _ })       -> false.
+
+%% Extra bitstring specifiers
+
+extract_bit_info(Meta, [{ size, _, [Arg] }|T], S) ->
+  case elixir_translator:translate(Arg, S) of
+    { { Kind, _, _ } = Size, _ } when Kind == integer; Kind == var ->
+      { Size, extract_bit_type(Meta, T, S) };
+    _ ->
+      elixir_errors:compile_error(Meta, S#elixir_scope.file,
+        "size in bitstring expects an integer or a variable as argument, got: ~ts", ['Elixir.Macro':to_string(Arg)])
+  end;
+extract_bit_info(Meta, T, S) ->
+  { default, extract_bit_type(Meta, T, S) }.
+
+extract_bit_type(Meta, [{ unit, _, [Arg] }|T], S) when is_integer(Arg) ->
+  [{ unit, Arg }|extract_bit_type(Meta, T, S)];
+extract_bit_type(Meta, [{ unit, _, [Arg] }|_], S) ->
+  elixir_errors:compile_error(Meta, S#elixir_scope.file,
+    "unit in bitstring expects an integer as argument, got: ~ts", ['Elixir.Macro':to_string(Arg)]);
+extract_bit_type(Meta, [{ Other, _, [] }|T], S) ->
+  [Other|extract_bit_type(Meta, T, S)];
+extract_bit_type(_Meta, [], _S) ->
+  [].

lib/elixir/src/elixir_bootstrap.erl

@@ -0,0 +1,51 @@
+%% An Erlang module that behaves like an Elixir module
+%% used for bootstraping.
+-module(elixir_bootstrap).
+-export(['MACRO-def'/2, 'MACRO-def'/3, 'MACRO-defp'/3, 'MACRO-defmodule'/3,
+         'MACRO-defmacro'/2, 'MACRO-defmacro'/3, 'MACRO-defmacrop'/3,
+         'MACRO-@'/2, '__info__'/1]).
+-define(kernel, 'Elixir.Kernel').
+
+%% Mock out @ to be a no-op unless Kernel is defined.
+'MACRO-@'(Caller, Tree) ->
+  unless_loaded('MACRO-@', [Caller, Tree], fun() -> nil end).
+
+'MACRO-def'(Caller, Call) -> 'MACRO-def'(Caller, Call, nil).
+'MACRO-def'(Caller, Call, Expr) -> define(Caller, def, Call, Expr).
+'MACRO-defp'(Caller, Call, Expr) -> define(Caller, defp, Call, Expr).
+
+'MACRO-defmacro'(Caller, Call) -> 'MACRO-defmacro'(Caller, Call, nil).
+'MACRO-defmacro'(Caller, Call, Expr) -> define(Caller, defmacro, Call, Expr).
+'MACRO-defmacrop'(Caller, Call, Expr) -> define(Caller, defmacrop, Call, Expr).
+
+'MACRO-defmodule'(_Caller, Alias, [{do,Block}]) ->
+  { Escaped, _ } = elixir_quote:escape(Block, false),
+  Args = [Alias, Escaped, [], env()],
+  { { '.', [], [elixir_module, compile] }, [], Args }.
+
+'__info__'(functions) ->
+  [];
+'__info__'(macros) ->
+  [{ '@', 1 },
+   {def,1},
+   {def,2},
+   {defmacro,1},
+   {defmacro,2},
+   {defmacrop,2},
+   {defmodule,2},
+   {defp,2}].
+
+define({Line,E}, Kind, Call, Expr) ->
+  { EscapedCall, UC } = elixir_quote:escape(Call, true),
+  { EscapedExpr, UE } = elixir_quote:escape(Expr, true),
+  Args = [Line, Kind, not(UC or UE), EscapedCall, EscapedExpr, elixir_locals:cache_env(E)],
+  { { '.', [], [elixir_def, store_definition] }, [], Args }.
+
+unless_loaded(Fun, Args, Callback) ->
+  case code:is_loaded(?kernel) of
+    { _, _} -> apply(?kernel, Fun, Args);
+    false   -> Callback()
+  end.
+
+env() ->
+  { '__ENV__', [], nil }.

lib/elixir/src/elixir_clauses.erl

@@ -1,134 +1,109 @@
 %% Handle code related to args, guard and -> matching for case,
 %% fn, receive and friends. try is handled in elixir_try.
 -module(elixir_clauses).
--export([
-  assigns/3, assigns_block/5, assigns_block/6, extract_splat_guards/1,
-  get_pairs/4, get_pairs/5, match/3, extract_args/1, extract_guards/1]).
+-export([match/3, clause/6, clauses/3, get_pairs/2, get_pairs/3,
+  extract_splat_guards/1, extract_guards/1]).
 -include("elixir.hrl").
 
 %% Get pairs from a clause.
 
-get_pairs(Meta, Key, Clauses, S) ->
-  get_pairs(Meta, Key, Clauses, S, false).
-
-get_pairs(Meta, Key, Clauses, S, AllowNil) ->
+get_pairs(Key, Clauses) ->
+  get_pairs(Key, Clauses, false).
+get_pairs(Key, Clauses, AllowNil) ->
   case lists:keyfind(Key, 1, Clauses) of
-    { Key, { '->', _, Pairs } } ->
-      [{ Key, PMeta, Left, Right } || { Left, PMeta, Right } <- Pairs];
+    { Key, Pairs } when is_list(Pairs) ->
+      [{ Key, Meta, Left, Right } || { '->', Meta, [Left, Right] } <- Pairs];
     { Key, nil } when AllowNil ->
       [];
-    { Key, _ } ->
-      elixir_errors:syntax_error(Meta, S#elixir_scope.file, "expected pairs with -> for key ~ts", [Key]);
-    _ ->
+    false ->
       []
   end.
 
-% Function for translating assigns.
+%% Translate matches
 
-assigns(Fun, Args, #elixir_scope{context=Context, temp_vars=TempVars,
+match(Fun, Args, #elixir_scope{context=Context, match_vars=MatchVars,
     backup_vars=BackupVars, vars=Vars} = S) when Context /= match ->
-  { Result, NewS } = assigns(Fun, Args, S#elixir_scope{context=match,
-                       temp_vars=ordsets:new(), backup_vars=Vars}),
+  { Result, NewS } = match(Fun, Args, S#elixir_scope{context=match,
+                       match_vars=ordsets:new(), backup_vars=Vars}),
   { Result, NewS#elixir_scope{context=Context,
-      temp_vars=TempVars, backup_vars=BackupVars} };
+      match_vars=MatchVars, backup_vars=BackupVars} };
+match(Fun, Args, S) -> Fun(Args, S).
 
-assigns(Fun, Args, S) -> Fun(Args, S).
+%% Translate clauses with args, guards and expressions
 
-%% Function for translating a block that is preceeded by an
-%% assignment and optional guards. This is used by def* and fn.
+clause(Line, Fun, Args, Expr, Guards, S) when is_integer(Line) ->
+  { TArgs, SA } = match(Fun, Args, S#elixir_scope{extra_guards=[]}),
+  { TExpr, SE } = elixir_translator:translate(Expr, SA#elixir_scope{extra_guards=nil}),
 
-assigns_block(Line, Fun, BareArgs, Exprs, S) ->
-  { Args, Guards } = extract_guards(BareArgs),
-  assigns_block(Line, Fun, Args, Exprs, Guards, S).
+  SG    = SA#elixir_scope{context=guard, extra_guards=nil},
+  Extra = SA#elixir_scope.extra_guards,
 
-assigns_block(Line, Fun, Args, Exprs, Guards, S) when is_integer(Line) ->
-  { TArgs, SA }  = assigns(Fun, Args, S#elixir_scope{extra_guards=[]}),
-  { TExprs, SE } = elixir_translator:translate(Exprs, SA#elixir_scope{extra_guards=nil}),
-
-  FArgs   = listify(TArgs),
-  SG      = SA#elixir_scope{context=guard, extra_guards=nil},
-  Extra   = SA#elixir_scope.extra_guards,
-
-  FGuards = case Guards of
+  TGuards = case Guards of
     [] -> case Extra of [] -> []; _ -> [Extra] end;
     _  -> [translate_guard(Line, Guard, Extra, SG) || Guard <- Guards]
   end,
 
-  % Uncompact expressions from the block.
-  case TExprs of
-    [{ block, _, FExprs }] -> [];
-    _ -> FExprs = TExprs
-  end,
-
-  { { clause, Line, FArgs, FGuards, FExprs }, SE }.
+  { { clause, Line, TArgs, TGuards, unblock(TExpr) }, SE }.
 
 % Translate/Extract guards from the given expression.
 
 translate_guard(Line, Guard, Extra, S) ->
-  [element(1, elixir_translator:translate_each(elixir_quote:linify(Line, Guard), S))|Extra].
+  [element(1, elixir_translator:translate(elixir_quote:linify(Line, Guard), S))|Extra].
 
-extract_guards({ 'when', _, [Left, Right] }) -> { Left, extract_or_clauses(Right, []) };
+extract_guards({ 'when', _, [Left, Right] }) -> { Left, extract_or_guards(Right) };
 extract_guards(Else) -> { Else, [] }.
 
-extract_or_clauses({ 'when', _, [Left, Right] }, Acc) -> extract_or_clauses(Right, [Left|Acc]);
-extract_or_clauses(Term, Acc) -> [Term|Acc].
-
-% Extract name and args from the given expression.
-
-extract_args({ { '.', _, [Name] }, _, Args }) when is_atom(Name), is_list(Args) -> { Name, Args };
-extract_args({ Name, _, Args }) when is_atom(Name), is_atom(Args) -> { Name, [] };
-extract_args({ Name, _, Args }) when is_atom(Name), is_list(Args) -> { Name, Args };
-extract_args(_) -> error.
+extract_or_guards({ 'when', _, [Left, Right] }) -> [Left|extract_or_guards(Right)];
+extract_or_guards(Term) -> [Term].
 
 % Extract guards when multiple left side args are allowed.
 
 extract_splat_guards([{ 'when', _, [_,_|_] = Args }]) ->
   { Left, Right } = elixir_utils:split_last(Args),
-  { Left, extract_or_clauses(Right, []) };
+  { Left, extract_or_guards(Right) };
 extract_splat_guards(Else) ->
   { Else, [] }.
 
 % Function for translating macros with match style like case and receive.
 
-match(Meta, Clauses, #elixir_scope{clause_vars=C1} = S) ->
-  { TC, TS } = do_match(Meta, Clauses, S#elixir_scope{clause_vars=orddict:new()}),
-  C2 = TS#elixir_scope.clause_vars,
-  { TC, TS#elixir_scope{clause_vars=elixir_scope:merge_clause_vars(C1, C2)} }.
+clauses(Meta, Clauses, #elixir_scope{export_vars=CV} = S) ->
+  { TC, TS } = do_clauses(Meta, Clauses, S#elixir_scope{export_vars=[]}),
+  { TC, TS#elixir_scope{export_vars=elixir_scope:merge_opt_vars(CV, TS#elixir_scope.export_vars)} }.
 
-do_match(_Meta, [], S) ->
+do_clauses(_Meta, [], S) ->
   { [], S };
 
-do_match(_Meta, [DecoupledClause], S) ->
-  { TDecoupledClause, TS } = each_clause(DecoupledClause, S),
-  { [TDecoupledClause], TS };
+% do_clauses(_Meta, [DecoupledClause], S) ->
+%   { TDecoupledClause, TS } = each_clause(DecoupledClause, S),
+%   { [TDecoupledClause], TS };
 
-do_match(Meta, DecoupledClauses, S) ->
+do_clauses(Meta, DecoupledClauses, S) ->
   % Transform tree just passing the variables counter forward
   % and storing variables defined inside each clause.
-  Transformer = fun(X, {Acc, CV}) ->
-    { TX, TAcc } = each_clause(X, Acc),
-    { TX, { merge_clauses_scope(S, TAcc), [TAcc#elixir_scope.clause_vars|CV] } }
+  Transformer = fun(X, {SAcc, VAcc}) ->
+    { TX, TS } = each_clause(X, SAcc),
+    { TX,
+      { elixir_scope:mergec(S, TS), [TS#elixir_scope.export_vars|VAcc] } }
   end,
 
-  { TClauses, { TS, ReverseCV } } = lists:mapfoldl(Transformer, {S, []}, DecoupledClauses),
+  { TClauses, { TS, ReverseCV } } =
+    lists:mapfoldl(Transformer, {S, []}, DecoupledClauses),
 
   % Now get all the variables defined inside each clause
   CV = lists:reverse(ReverseCV),
-  AllVars = lists:foldl(fun(KV, Acc) ->
-    elixir_scope:merge_clause_vars(Acc, KV)
-  end, orddict:new(), CV),
+  AllVars = lists:foldl(fun elixir_scope:merge_vars/2, [], CV),
 
   % Create a new scope that contains a list of all variables
   % defined inside all the clauses. It returns this new scope and
   % a list of tuples where the first element is the variable name,
   % the second one is the new pointer to the variable and the third
   % is the old pointer.
-  { FinalVars, FS } = lists:mapfoldl(fun({ Key, Ref }, Acc) ->
-    normalize_vars(Key, Ref, Acc)
+  { FinalVars, FS } = lists:mapfoldl(fun({ Key, Val }, Acc) ->
+    normalize_vars(Key, Val, Acc)
   end, TS, AllVars),
 
   % Expand all clauses by adding a match operation at the end
-  % that assigns variables missing in one clause to the others.
+  % that defines variables missing in one clause to the others.
   expand_clauses(?line(Meta), TClauses, CV, FinalVars, [], FS).
 
 expand_clauses(Line, [Clause|T], [ClauseVars|V], FinalVars, Acc, S) ->
@@ -149,7 +124,8 @@ expand_clauses(Line, [Clause|T], [ClauseVars|V], FinalVars, Acc, S) ->
             { match, _, { var, _, UserVarName } = UserVar, _ } when UserVarName /= '_' ->
               { [UserVar,MatchExpr,Final|RawClauseExprs], S };
             _ ->
-              { StorageVar, SS } = elixir_scope:build_erl_var(Line, S),
+              { VarName, _, SS } = elixir_scope:build_var('_', S),
+              StorageVar  = { var, Line, VarName },
               StorageExpr = { match, Line, StorageVar, Final },
               { [StorageVar,MatchExpr,StorageExpr|RawClauseExprs], SS }
           end;
@@ -167,21 +143,17 @@ expand_clauses(_Line, [], [], _FinalVars, Acc, S) ->
 % Handle each key/value clause pair and translate them accordingly.
 
 each_clause({ do, Meta, [Condition], Expr }, S) ->
-  assigns_block(?line(Meta), fun elixir_translator:translate_each/2, Condition, [Expr], S);
+  { Arg, Guards } = extract_guards(Condition),
+  clause(?line(Meta), fun elixir_translator:translate_many/2, [Arg], Expr, Guards, S);
 
 each_clause({ else, Meta, [Condition], Expr }, S) ->
-  assigns_block(?line(Meta), fun elixir_translator:translate_each/2, Condition, [Expr], S);
+  { Arg, Guards } = extract_guards(Condition),
+  clause(?line(Meta), fun elixir_translator:translate_many/2, [Arg], Expr, Guards, S);
 
 each_clause({ 'after', Meta, [Condition], Expr }, S) ->
-  { TCondition, SC } = elixir_translator:translate_each(Condition, S),
-  { TBody, SB } = elixir_translator:translate([Expr], SC),
-  { { clause, ?line(Meta), [TCondition], [], TBody }, SB };
-
-each_clause({ Key, Meta, [_|_], _ }, S) when Key == do; Key == 'after' ->
-  elixir_errors:syntax_error(Meta, S#elixir_scope.file, "too many arguments given for ~ts", [Key]);
-
-each_clause({ Key, Meta, _, _ }, S) ->
-  elixir_errors:syntax_error(Meta, S#elixir_scope.file, "invalid key ~ts", [Key]).
+  { TCondition, SC } = elixir_translator:translate(Condition, S),
+  { TExpr, SB } = elixir_translator:translate(Expr, SC),
+  { { clause, ?line(Meta), [TCondition], [], unblock(TExpr) }, SB }.
 
 % Check if the given expression is a match tuple.
 % This is a small optimization to allow us to change
@@ -189,55 +161,51 @@ each_clause({ Key, Meta, _, _ }, S) ->
 
 has_match_tuple({'receive', _, _, _, _}) ->
   true;
-
 has_match_tuple({'receive', _, _}) ->
   true;
-
 has_match_tuple({'case', _, _, _}) ->
   true;
-
 has_match_tuple({match, _, _, _}) ->
   true;
-
 has_match_tuple({'fun', _, { clauses, _ }}) ->
   false;
-
 has_match_tuple(H) when is_tuple(H) ->
   has_match_tuple(tuple_to_list(H));
-
 has_match_tuple(H) when is_list(H) ->
   lists:any(fun has_match_tuple/1, H);
-
 has_match_tuple(_) -> false.
 
 % Normalize the given var in between clauses
 % by picking one value as reference and retriving
 % its previous value.
 
-normalize_vars(Key, Value, #elixir_scope{vars=Vars,clause_vars=ClauseVars} = S) ->
-  FS = S#elixir_scope{
+normalize_vars(Key, Value, #elixir_scope{vars=Vars,export_vars=ClauseVars} = S) ->
+  VS = S#elixir_scope{
     vars=orddict:store(Key, Value, Vars),
-    clause_vars=orddict:store(Key, Value, ClauseVars)
+    export_vars=orddict:store(Key, Value, ClauseVars)
   },
 
   Expr = case orddict:find(Key, Vars) of
-    { ok, OldValue } -> { var, 0, OldValue };
+    { ok, { PreValue, _ } } -> { var, 0, PreValue };
     error -> { atom, 0, nil }
   end,
 
-  { { Key, Value, Expr }, FS }.
+  { { Key, Value, Expr }, VS }.
 
 % Generate match vars by checking if they were updated
 % or not and assigning the previous value.
 
-generate_match_vars([{ Key, NewValue, OldValue }|T], ClauseVars, Left, Right) ->
+generate_match_vars([{ Key, Value, Expr }|T], ClauseVars, Left, Right) ->
   case orddict:find(Key, ClauseVars) of
-    { ok, NewValue } ->
+    { ok, Value } ->
       generate_match_vars(T, ClauseVars, Left, Right);
-    { ok, ClauseValue } ->
-      generate_match_vars(T, ClauseVars, [{ var, 0, NewValue }|Left], [{ var, 0, ClauseValue }|Right]);
+    { ok, Clause } ->
+      generate_match_vars(T, ClauseVars,
+        [{ var, 0, element(1, Value) }|Left],
+        [{ var, 0, element(1, Clause) }|Right]);
     error ->
-      generate_match_vars(T, ClauseVars, [{ var, 0, NewValue }|Left], [OldValue|Right])
+      generate_match_vars(T, ClauseVars,
+        [{ var, 0, element(1, Value) }|Left], [Expr|Right])
   end;
 
 generate_match_vars([], _ClauseVars, Left, Right) ->
@@ -249,15 +217,5 @@ generate_match(Line, [Left], [Right]) ->
 generate_match(Line, LeftVars, RightVars) ->
   { match, Line, { tuple, Line, LeftVars }, { tuple, Line, RightVars } }.
 
-listify(Expr) when not is_list(Expr) -> [Expr];
-listify(Expr) -> Expr.
-
-%% We don't use umergec because imports, aliases and
-%% what not are not passed from one clause to the other.
-merge_clauses_scope(S1, S2) ->
-  S1#elixir_scope{
-    counter=S2#elixir_scope.counter,
-    extra_guards=S2#elixir_scope.extra_guards,
-    super=S1#elixir_scope.super orelse S2#elixir_scope.super,
-    caller=S1#elixir_scope.caller orelse S2#elixir_scope.caller
-  }.
+unblock({ 'block', _, Exprs }) -> Exprs;
+unblock(Exprs) -> [Exprs].

lib/elixir/src/elixir_code_server.erl

@@ -3,32 +3,33 @@
 -export([start_link/0, init/1, handle_call/3, handle_cast/2,
   handle_info/2, terminate/2, code_change/3]).
 -behavior(gen_server).
+
+-define(timeout, 30000).
 -record(elixir_code_server, {
+  compilation_status=[],
   argv=[],
-  compilation_status=ok,
   loaded=[],
   at_exit=[],
-  pool=[],
-  counter=0,
+  pool={[],0},
   compiler_options=[{docs,true},{debug_info,true},{warnings_as_errors,false}],
-  waiting=[]
+  erl_compiler_options=nil
 }).
 
 call(Args) ->
-  gen_server:call(?MODULE, Args, get_timeout()).
+  gen_server:call(?MODULE, Args, ?timeout).
 
 cast(Args) ->
   gen_server:cast(?MODULE, Args).
 
-get_timeout() ->
-  30000.
-
 %% Callbacks
 
 start_link() ->
-  { ok, _ } = gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
+  gen_server:start_link({local, ?MODULE}, ?MODULE, ok, []).
 
-init(_args) ->
+init(ok) ->
+  code:ensure_loaded('Elixir.Macro.Env'),
+  code:ensure_loaded('Elixir.Module.LocalsTracker'),
+  code:ensure_loaded('Elixir.Kernel.LexicalTracker'),
   { ok, #elixir_code_server{} }.
 
 handle_call({ acquire, Path }, From, Config) ->
@@ -59,20 +60,31 @@ handle_call(argv, _From, Config) ->
 handle_call(compiler_options, _From, Config) ->
   { reply, Config#elixir_code_server.compiler_options, Config };
 
-handle_call(compilation_status, _From, Config) ->
-  { reply, Config#elixir_code_server.compilation_status, Config };
+handle_call({ compilation_status, CompilerPid }, _From, Config) ->
+  CompilationStatusList    = Config#elixir_code_server.compilation_status,
+  CompilationStatusListNew = orddict:erase(CompilerPid, CompilationStatusList),
+  CompilationStatus        = orddict:fetch(CompilerPid, CompilationStatusList),
+  { reply, CompilationStatus, Config#elixir_code_server{compilation_status=CompilationStatusListNew} };
 
 handle_call(retrieve_module_name, _From, Config) ->
   case Config#elixir_code_server.pool of
-    [H|T] ->
-      { reply, module_tuple(H), Config#elixir_code_server{pool=T} };
-    [] ->
-      Counter = Config#elixir_code_server.counter,
-      { reply, module_tuple(Counter), Config#elixir_code_server{counter=Counter+1} }
+    { [H|T], Counter } ->
+      { reply, module_tuple(H), Config#elixir_code_server{pool={T,Counter}} };
+    { [], Counter } ->
+      { reply, module_tuple(Counter), Config#elixir_code_server{pool={[],Counter+1}} }
   end;
 
-handle_call(_Request, _From, Config) ->
-  { reply, undef, Config }.
+handle_call(erl_compiler_options, _From, Config) ->
+  case Config#elixir_code_server.erl_compiler_options of
+    nil ->
+      Opts = erl_compiler_options(),
+      { reply, Opts, Config#elixir_code_server{erl_compiler_options=Opts} };
+    Opts ->
+      { reply, Opts, Config }
+  end;
+
+handle_call(Request, _From, Config) ->
+  { stop, { badcall, Request }, Config }.
 
 handle_cast({ at_exit, AtExit }, Config) ->
   { noreply, Config#elixir_code_server{at_exit=[AtExit|Config#elixir_code_server.at_exit]} };
@@ -84,14 +96,18 @@ handle_cast({ compiler_options, Options }, Config) ->
   Final = orddict:merge(fun(_,_,V) -> V end, Config#elixir_code_server.compiler_options, Options),
   { noreply, Config#elixir_code_server{compiler_options=Final} };
 
-handle_cast(register_warning, Config) ->
+handle_cast({ register_warning, CompilerPid }, Config) ->
+  CompilationStatusCurrent = Config#elixir_code_server.compilation_status,
+  CompilationStatusNew     = orddict:store(CompilerPid, error, CompilationStatusCurrent),
   case orddict:find(warnings_as_errors, Config#elixir_code_server.compiler_options) of
-    { ok, true } -> { noreply, Config#elixir_code_server{compilation_status=error} };
+    { ok, true } -> { noreply, Config#elixir_code_server{compilation_status=CompilationStatusNew} };
     _ -> { noreply, Config }
   end;
 
-handle_cast(reset_warnings, Config) ->
-  { noreply, Config#elixir_code_server{compilation_status=ok} };
+handle_cast({ reset_warnings, CompilerPid }, Config) ->
+  CompilationStatusCurrent = Config#elixir_code_server.compilation_status,
+  CompilationStatusNew     = orddict:store(CompilerPid, ok, CompilationStatusCurrent),
+  { noreply, Config#elixir_code_server{compilation_status=CompilationStatusNew} };
 
 handle_cast({ loaded, Path }, Config) ->
   Current = Config#elixir_code_server.loaded,
@@ -112,11 +128,11 @@ handle_cast({ unload_files, Files }, Config) ->
   Unloaded = lists:foldl(fun(File, Acc) -> orddict:erase(File, Acc) end, Current, Files),
   { noreply, Config#elixir_code_server{loaded=Unloaded} };
 
-handle_cast({ return_module_name, I }, #elixir_code_server{pool=Pool} = Config) ->
-  { noreply, Config#elixir_code_server{pool=[I|Pool]} };
+handle_cast({ return_module_name, H }, #elixir_code_server{pool={T,Counter}} = Config) ->
+  { noreply, Config#elixir_code_server{pool={[H|T],Counter}} };
 
-handle_cast(_Request, Config) ->
-  { noreply, Config }.
+handle_cast(Request, Config) ->
+  { stop, { badcast, Request }, Config }.
 
 handle_info(_Request, Config) ->
   { noreply, Config }.
@@ -127,4 +143,25 @@ terminate(_Reason, _Config) ->
 code_change(_Old, Config, _Extra) ->
   { ok, Config }.
 
-module_tuple(I) -> { list_to_atom("elixir_compiler_" ++ integer_to_list(I)), I }.
+module_tuple(I) ->
+  { list_to_atom("elixir_compiler_" ++ integer_to_list(I)), I }.
+
+erl_compiler_options() ->
+  Key = "ERL_COMPILER_OPTIONS",
+  case os:getenv(Key) of
+    false -> [];
+    Str when is_list(Str) ->
+      case erl_scan:string(Str) of
+        {ok,Tokens,_} ->
+          case erl_parse:parse_term(Tokens ++ [{dot, 1}]) of
+            {ok,List} when is_list(List) -> List;
+            {ok,Term} -> [Term];
+            {error,_Reason} ->
+              io:format("Ignoring bad term in ~ts\n", [Key]),
+              []
+          end;
+        {error, {_,_,_Reason}, _} ->
+          io:format("Ignoring bad term in ~ts\n", [Key]),
+          []
+      end
+  end.

lib/elixir/src/elixir_compiler.erl

@@ -1,27 +1,21 @@
 -module(elixir_compiler).
--export([get_opts/0, get_opt/1, get_opt/2, string/2, quoted/2, file/1, file_to_path/2]).
--export([core/0, module/3, eval_forms/4, format_error/1]).
+-export([get_opt/1, string/2, quoted/2, file/1, file_to_path/2]).
+-export([core/0, module/4, eval_forms/3, format_error/1]).
 -include("elixir.hrl").
 
 %% Public API
 
-%% Get compilation options.
-
-get_opt(Key) -> get_opt(Key, get_opts()).
-
-get_opt(Key, Dict) ->
+get_opt(Key) ->
+  Dict = elixir_code_server:call(compiler_options),
   case lists:keyfind(Key, 1, Dict) of
     false -> false;
     { Key, Value } -> Value
   end.
 
-get_opts() ->
-  elixir_code_server:call(compiler_options).
-
-%% Compiles the given string.
+%% Compilation entry points.
 
 string(Contents, File) when is_list(Contents), is_binary(File) ->
-  Forms = elixir_translator:'forms!'(Contents, 1, File, []),
+  Forms = elixir:'string_to_quoted!'(Contents, 1, File, []),
   quoted(Forms, File).
 
 quoted(Forms, File) when is_binary(File) ->
@@ -29,71 +23,121 @@ quoted(Forms, File) when is_binary(File) ->
 
   try
     put(elixir_compiled, []),
-    eval_forms(Forms, 1, [], elixir:scope_for_eval([{file,File}])),
+    elixir_lexical:run(File, fun
+      (Pid) ->
+        Env = elixir:env_for_eval([{line,1},{file,File}]),
+        eval_forms(Forms, [], Env#elixir_env{lexical_tracker=Pid})
+    end),
     lists:reverse(get(elixir_compiled))
   after
     put(elixir_compiled, Previous)
   end.
 
-%% Compile a file, return a tuple of module names and binaries.
-
 file(Relative) when is_binary(Relative) ->
   File = filename:absname(Relative),
   { ok, Bin } = file:read_file(File),
   string(elixir_utils:characters_to_list(Bin), File).
 
-%% Compiles a file to the given path (directory).
-
 file_to_path(File, Path) when is_binary(File), is_binary(Path) ->
   Lists = file(File),
   [binary_to_path(X, Path) || X <- Lists],
   Lists.
 
-%% Evaluates the contents/forms by compiling them to an Erlang module.
+%% Evaluation
+
+eval_forms(Forms, Vars, E) ->
+  case (E#elixir_env.module == nil) andalso allows_fast_compilation(Forms) of
+    true  -> eval_compilation(Forms, Vars, E);
+    false -> code_loading_compilation(Forms, Vars, E)
+  end.
+
+eval_compilation(Forms, Vars, E) ->
+  Binding = [{ Key, Value } || { _Name, _Kind, Key, Value } <- Vars],
+  { Result, _Binding, EE, _S } = elixir:eval_forms(Forms, Binding, E),
+  { Result, EE }.
+
+code_loading_compilation(Forms, Vars, #elixir_env{line=Line} = E) ->
+  Dict = [{ { Name, Kind }, { Value, 0 } } || { Name, Kind, Value, _ } <- Vars],
+  S = elixir_env:env_to_scope_with_vars(E, Dict),
+  { Expr, EE, _S } = elixir:quoted_to_erl(Forms, E, S),
 
-eval_forms(Forms, Line, Vars, S) ->
   { Module, I } = retrieve_module_name(),
-  { Exprs, FS } = elixir_translator:translate(Forms, S),
-
-  Fun  = module_form_fun(S#elixir_scope.module),
-  Form = module_form(Fun, Exprs, Line, S#elixir_scope.file, Module, Vars),
-  Args = [X || { _, _, _, X } <- Vars],
+  Fun  = code_fun(E#elixir_env.module),
+  Form = code_mod(Fun, Expr, Line, E#elixir_env.file, Module, Vars),
+  Args = list_to_tuple([V || { _, _, _, V } <- Vars]),
 
   %% Pass { native, false } to speed up bootstrap
   %% process when native is set to true
-  { module(Form, S#elixir_scope.file, [{native,false}], true, fun(_, Binary) ->
-    Res = Module:Fun(S#elixir_scope.module, Args),
-    code:delete(Module),
-
+  module(Form, E#elixir_env.file, [{native,false}], true, fun(_, Binary) ->
     %% If we have labeled locals, anonymous functions
     %% were created and therefore we cannot ditch the
     %% module
-    case beam_lib:chunks(Binary, [labeled_locals]) of
-      { ok, { _, [{ labeled_locals, []}] } } ->
-        code:purge(Module),
-        return_module_name(I);
-      _ ->
-        ok
-    end,
+    Purgeable =
+      case beam_lib:chunks(Binary, [labeled_locals]) of
+        { ok, { _, [{ labeled_locals, []}] } } -> true;
+        _ -> false
+      end,
+    dispatch_loaded(Module, Fun, Args, Purgeable, I, EE)
+  end).
 
-    Res
-  end), FS }.
+dispatch_loaded(Module, Fun, Args, Purgeable, I, E) ->
+  Res = Module:Fun(Args),
+  code:delete(Module),
+  if Purgeable ->
+      code:purge(Module),
+      return_module_name(I);
+     true ->
+       ok
+  end,
+  { Res, E }.
 
-%% Internal API
+code_fun(nil) -> '__FILE__';
+code_fun(_)   -> '__MODULE__'.
+
+code_mod(Fun, Expr, Line, File, Module, Vars) when is_binary(File), is_integer(Line) ->
+  Tuple    = { tuple, Line, [{ var, Line, K } || { _, _, K, _ } <- Vars] },
+  Relative = elixir_utils:relative_to_cwd(File),
+
+  [
+    { attribute, Line, file, { elixir_utils:characters_to_list(Relative), 1 } },
+    { attribute, Line, module, Module },
+    { attribute, Line, export, [{ Fun, 1 }, { '__RELATIVE__', 0 }] },
+    { function, Line, Fun, 1, [
+      { clause, Line, [Tuple], [], [Expr] }
+    ] },
+    { function, Line, '__RELATIVE__', 0, [
+      { clause, Line, [], [], [elixir_utils:elixir_to_erl(Relative)] }
+    ] }
+  ].
+
+retrieve_module_name() ->
+  elixir_code_server:call(retrieve_module_name).
+
+return_module_name(I) ->
+  elixir_code_server:cast({ return_module_name, I }).
+
+allows_fast_compilation({ '__block__', _, Exprs }) ->
+  lists:all(fun allows_fast_compilation/1, Exprs);
+allows_fast_compilation({defmodule,_,_}) -> true;
+allows_fast_compilation(_) -> false.
+
+%% INTERNAL API
 
 %% Compile the module by forms based on the scope information
 %% executes the callback in case of success. This automatically
 %% handles errors and warnings. Used by this module and elixir_module.
-module(Forms, File, Callback) ->
-  Options = case get_opt(debug_info) of
-    true -> [debug_info];
-    _ -> []
-  end,
-  module(Forms, File, Options, false, Callback).
+module(Forms, File, Opts, Callback) ->
+  Final =
+    case (get_opt(debug_info) == true) orelse
+         lists:member(debug_info, Opts) of
+      true  -> [debug_info];
+      false -> []
+    end,
+  module(Forms, File, Final, false, Callback).
 
 module(Forms, File, RawOptions, Bootstrap, Callback) when
     is_binary(File), is_list(Forms), is_list(RawOptions), is_boolean(Bootstrap), is_function(Callback) ->
-  { Options, SkipNative } = compile_opts(Forms, RawOptions),
+  { Options, SkipNative } = compiler_options(Forms, RawOptions),
   Listname = elixir_utils:characters_to_list(File),
 
   case compile:noenv_forms([no_auto_import()|Forms], [return,{source,Listname}|Options]) of
@@ -110,44 +154,14 @@ module(Forms, File, RawOptions, Bootstrap, Callback) when
       format_errors(Errors)
   end.
 
-%% Compile core files for bootstrap.
-%% Invoked from the Makefile.
-
-core() ->
-  application:start(elixir),
-  elixir_code_server:cast({ compiler_options, [{docs,false},{internal,true}] }),
-  [core_file(File) || File <- core_main()].
-
-%% HELPERS
-
-compile_opts(Forms, Options) ->
-  EnvOptions = env_default_opts(),
+compiler_options(Forms, Options) ->
+  EnvOptions = elixir_code_server:call(erl_compiler_options),
   SkipNative = lists:member(native, EnvOptions) and contains_on_load(Forms),
   case SkipNative or lists:member([{native,false}], Options) of
     true  -> { Options ++ lists:delete(native, EnvOptions), SkipNative };
     false -> { Options ++ EnvOptions, SkipNative }
   end.
 
-env_default_opts() ->
-  Key = "ERL_COMPILER_OPTIONS",
-  case os:getenv(Key) of
-    false -> [];
-    Str when is_list(Str) ->
-      case erl_scan:string(Str) of
-        {ok,Tokens,_} ->
-          case erl_parse:parse_term(Tokens ++ [{dot, 1}]) of
-            {ok,List} when is_list(List) -> List;
-            {ok,Term} -> [Term];
-            {error,_Reason} ->
-              io:format("Ignoring bad term in ~ts\n", [Key]),
-              []
-          end;
-        {error, {_,_,_Reason}, _} ->
-          io:format("Ignoring bad term in ~ts\n", [Key]),
-          []
-      end
-  end.
-
 contains_on_load([{ attribute, _, on_load, _ }|_]) -> true;
 contains_on_load([_|T]) -> contains_on_load(T);
 contains_on_load([]) -> false.
@@ -156,46 +170,13 @@ no_auto_import() ->
   Bifs = [{ Name, Arity } || { Name, Arity } <- erlang:module_info(exports), erl_internal:bif(Name, Arity)],
   { attribute, 0, compile, { no_auto_import, Bifs } }.
 
-module_form_fun(nil) -> '__FILE__';
-module_form_fun(_)   -> '__MODULE__'.
 
-module_form(Fun, Exprs, Line, File, Module, Vars) when
-    is_binary(File), is_integer(Line), is_list(Exprs), is_atom(Module) ->
-  Cons = lists:foldr(fun({ _, _, Var, _ }, Acc) ->
-    { cons, Line, { var, Line, Var }, Acc }
-  end, { nil, Line }, Vars),
+%% CORE HANDLING
 
-  Args = [{ var, Line, '_@MODULE'}, Cons],
-  Relative = elixir_utils:relative_to_cwd(File),
-
-  [
-    { attribute, Line, file, { elixir_utils:characters_to_list(File), 1 } },
-    { attribute, Line, module, Module },
-    { attribute, Line, export, [{ Fun, 2 }, { '__RELATIVE__', 0 }] },
-    { function, Line, Fun, length(Args), [
-      { clause, Line, Args, [], Exprs }
-    ] },
-    { function, Line, '__RELATIVE__', 0, [
-      { clause, Line, [], [], [elixir_utils:elixir_to_erl(Relative)] }
-    ] }
-  ].
-
-%% Generate module names from code server.
-
-retrieve_module_name() ->
-  elixir_code_server:call(retrieve_module_name).
-
-return_module_name(I) ->
-  elixir_code_server:cast({ return_module_name, I }).
-
-%% Receives a module Binary and outputs it in the given path.
-
-binary_to_path({ModuleName, Binary}, CompilePath) ->
-  Path = filename:join(CompilePath, atom_to_list(ModuleName) ++ ".beam"),
-  ok = file:write_file(Path, Binary),
-  Path.
-
-%% CORE FILES COMPILATION
+core() ->
+  application:start(elixir),
+  elixir_code_server:cast({ compiler_options, [{docs,false},{internal,true}] }),
+  [core_file(File) || File <- core_main()].
 
 core_file(File) ->
   try
@@ -214,44 +195,54 @@ core_main() ->
     <<"lib/elixir/lib/keyword.ex">>,
     <<"lib/elixir/lib/module.ex">>,
     <<"lib/elixir/lib/list.ex">>,
-    <<"lib/elixir/lib/kernel/typespec.ex">>,
     <<"lib/elixir/lib/record.ex">>,
     <<"lib/elixir/lib/macro.ex">>,
     <<"lib/elixir/lib/macro/env.ex">>,
     <<"lib/elixir/lib/exception.ex">>,
     <<"lib/elixir/lib/code.ex">>,
+    <<"lib/elixir/lib/module/locals_tracker.ex">>,
     <<"lib/elixir/lib/protocol.ex">>,
+    <<"lib/elixir/lib/stream/reducers.ex">>,
     <<"lib/elixir/lib/enum.ex">>,
     <<"lib/elixir/lib/inspect/algebra.ex">>,
     <<"lib/elixir/lib/inspect.ex">>,
     <<"lib/elixir/lib/range.ex">>,
+    <<"lib/elixir/lib/regex.ex">>,
     <<"lib/elixir/lib/string.ex">>,
     <<"lib/elixir/lib/string/chars.ex">>,
     <<"lib/elixir/lib/io.ex">>,
     <<"lib/elixir/lib/path.ex">>,
     <<"lib/elixir/lib/system.ex">>,
+    <<"lib/elixir/lib/kernel/typespec.ex">>,
     <<"lib/elixir/lib/kernel/cli.ex">>,
     <<"lib/elixir/lib/kernel/error_handler.ex">>,
     <<"lib/elixir/lib/kernel/parallel_compiler.ex">>,
     <<"lib/elixir/lib/kernel/record_rewriter.ex">>,
-    <<"lib/elixir/lib/module/dispatch_tracker.ex">>
+    <<"lib/elixir/lib/kernel/lexical_tracker.ex">>
   ].
 
+binary_to_path({ModuleName, Binary}, CompilePath) ->
+  Path = filename:join(CompilePath, atom_to_list(ModuleName) ++ ".beam"),
+  ok = file:write_file(Path, Binary),
+  Path.
+
 %% ERROR HANDLING
 
 format_error({ skip_native, Module }) ->
   io_lib:format("skipping native compilation for ~ts because it contains on_load attribute",
-    [elixir_errors:inspect(Module)]).
+    [elixir_aliases:inspect(Module)]).
 
 format_errors([]) ->
   exit({ nocompile, "compilation failed but no error was raised" });
 
 format_errors(Errors) ->
   lists:foreach(fun ({File, Each}) ->
-    lists:foreach(fun (Error) -> elixir_errors:handle_file_error(File, Error) end, Each)
+    BinFile = elixir_utils:characters_to_binary(File),
+    lists:foreach(fun (Error) -> elixir_errors:handle_file_error(BinFile, Error) end, Each)
   end, Errors).
 
 format_warnings(Bootstrap, Warnings) ->
   lists:foreach(fun ({File, Each}) ->
-    lists:foreach(fun (Warning) -> elixir_errors:handle_file_warning(Bootstrap, File, Warning) end, Each)
+    BinFile = elixir_utils:characters_to_binary(File),
+    lists:foreach(fun (Warning) -> elixir_errors:handle_file_warning(Bootstrap, BinFile, Warning) end, Each)
   end, Warnings).

lib/elixir/src/elixir_counter.erl

@@ -0,0 +1,38 @@
+-module(elixir_counter).
+-export([start_link/0, init/1, handle_call/3, handle_cast/2,
+  handle_info/2, terminate/2, code_change/3, next/0]).
+-behavior(gen_server).
+
+-define(timeout, 1000).     %% 1 second
+-define(limit, 4294967296). %% 2^32
+
+next() ->
+  gen_server:call(?MODULE, next, ?timeout).
+
+start_link() ->
+  gen_server:start_link({local, ?MODULE}, ?MODULE, 0, []).
+
+init(Counter) ->
+  { ok, Counter }.
+
+handle_call(next, _From, Counter) ->
+  { reply, Counter, bump(Counter) };
+handle_call(Request, _From, Counter) ->
+  { stop, { badcall, Request }, Counter }.
+
+handle_cast(Request, Counter) ->
+  { stop, { badcast, Request }, Counter }.
+
+handle_info(_Request, Counter) ->
+  { noreply, Counter }.
+
+terminate(_Reason, _Counter) ->
+  ok.
+
+code_change(_Old, Counter, _Extra) ->
+  { ok, Counter }.
+
+bump(Counter) when Counter < ?limit ->
+  Counter + 1;
+bump(_Counter) ->
+  0.

lib/elixir/src/elixir_def.erl

@@ -1,19 +1,9 @@
 % Holds the logic responsible for functions definition (def(p) and defmacro(p)).
 -module(elixir_def).
--export([table/1,
-  clauses_table/1,
-  setup/1,
-  cleanup/1,
-  reset_last/1,
-  lookup_definition/2,
-  delete_definition/2,
-  wrap_definition/6,
-  wrap_definition/8,
-  store_definition/7,
-  store_definition/9,
-  store_each/8,
-  unwrap_stored_definitions/2,
-  format_error/1]).
+-export([table/1, clauses_table/1, setup/1,
+  cleanup/1, reset_last/1, lookup_definition/2,
+  delete_definition/2, store_definition/6, unwrap_definitions/1,
+  store_each/8, format_error/1]).
 -include("elixir.hrl").
 
 -define(attr, '__def_table').
@@ -55,176 +45,148 @@ delete_definition(Module, Tuple) ->
   ets:delete(table(Module), Tuple),
   ets:delete(clauses_table(Module), Tuple).
 
-%% Wraps the function into a call to store_definition once the function
-%% definition is read. The function is compiled into a meta tree to ensure
-%% we will receive the full function.
-%%
-%% We need to wrap functions instead of eagerly defining them to ensure
-%% functions inside branches won't propagate, for example:
-%%
-%%   if false do
-%%     def bar, do: 1
-%%   else
-%%     def bar, do: 2
-%%   end
-%%
-%% If we just analyzed the compiled structure (i.e. the function availables
-%% before evaluating the function body), we would see both definitions.
-
-wrap_definition(Kind, Meta, Call, Expr, CheckClauses, S) ->
-  do_wrap_definition(Kind, Meta, CheckClauses, [
-    Call, Expr, elixir_scope:serialize(S)
-  ]).
-
-wrap_definition(Kind, Meta, Name, Args, Guards, Expr, CheckClauses, S) ->
-  do_wrap_definition(Kind, Meta, CheckClauses, [
-    Name, Args, Guards, Expr, elixir_scope:serialize(S)
-  ]).
-
-do_wrap_definition(Kind, Meta, CheckClauses, Extra) ->
-  Line   = ?line(Meta),
-  Invoke =
-    [{atom, Line, Kind},
-     {integer, Line, Line},
-     {atom, Line, CheckClauses},
-     {var, Line, '_@MODULE'}] ++ Extra,
-
-  ?wrap_call(Line, ?MODULE, store_definition, Invoke).
-
 % Invoked by the wrap definition with the function abstract tree.
 % Each function is then added to the function table.
 
-store_definition(Kind, Line, CheckClauses, Module, Call, Body, RawS) ->
-  S = elixir_scope:deserialize(RawS),
+store_definition(Line, Kind, CheckClauses, Call, Body, Pos) ->
+  E = (elixir_locals:get_cached_env(Pos))#elixir_env{line=Line},
   { NameAndArgs, Guards } = elixir_clauses:extract_guards(Call),
 
-  { Name, Args } = case elixir_clauses:extract_args(NameAndArgs) of
-    error ->
-      Format = [Kind, 'Elixir.Macro':to_string(NameAndArgs)],
-      elixir_errors:syntax_error(Line, S#elixir_scope.file, "invalid syntax in ~ts ~ts", Format);
-    Tuple ->
-      Tuple
+  { Name, Args } = case NameAndArgs of
+    { N, _, A } when is_atom(N), is_atom(A) -> { N, [] };
+    { N, _, A } when is_atom(N), is_list(A) -> { N, A };
+    _ -> elixir_errors:form_error(Line, E#elixir_env.file, ?MODULE, { invalid_def, Kind, NameAndArgs })
   end,
 
-  assert_no_aliases_name(Line, Name, Args, S),
-  store_definition(Kind, Line, CheckClauses, Module, Name, Args, Guards, Body, S).
+  %% Now that we have verified the call format,
+  %% extract meta information like file and context.
+  { _, Meta, _ } = Call,
+  DoCheckClauses = (not lists:keymember(context, 1, Meta)) andalso (CheckClauses),
 
-store_definition(Kind, Line, _CheckClauses, nil, _Name, _Args, _Guards, _Body, #elixir_scope{file=File}) ->
-  elixir_errors:form_error(Line, File, ?MODULE, { no_module, Kind });
+  %% Check if there is a file information in the definition.
+  %% If so, we assume this come from another source and we need
+  %% to linify taking into account keep line numbers.
+  { File, Key }  = case lists:keyfind(file, 1, Meta) of
+    { file, Bin } when is_binary(Bin) -> { Bin, keep };
+    _ -> { nil, line }
+  end,
 
-store_definition(Kind, Line, CheckClauses, Module, Name, Args, Guards, Body, #elixir_scope{} = DS) ->
+  LinifyArgs   = elixir_quote:linify(Line, Key, Args),
+  LinifyGuards = elixir_quote:linify(Line, Key, Guards),
+  LinifyBody   = elixir_quote:linify(Line, Key, Body),
+
+  assert_no_aliases_name(Line, Name, Args, E),
+  store_definition(Line, Kind, DoCheckClauses, Name,
+                   LinifyArgs, LinifyGuards, LinifyBody, File, E).
+
+store_definition(Line, Kind, CheckClauses, Name, Args, Guards, Body, MetaFile, #elixir_env{module=Module} = ER) ->
   Arity = length(Args),
   Tuple = { Name, Arity },
-  S = DS#elixir_scope{module=Module, function=Tuple},
-  elixir_tracker:record_definition(Tuple, Kind, Module),
+  E = ER#elixir_env{function=Tuple},
+  elixir_locals:record_definition(Tuple, Kind, Module),
 
-  CO = elixir_compiler:get_opts(),
-  Location = retrieve_file(Line, Module, S, CO),
-  run_on_definition_callbacks(Kind, Line, Module, Name, Args, Guards, Body, S, CO),
-
-  { Function, Defaults, TS } = translate_definition(Kind, Line, Name, Args, Guards, Body, S),
+  Location = retrieve_location(Line, MetaFile, Module),
+  run_on_definition_callbacks(Kind, Line, Module, Name, Args, Guards, Body, E),
+  { Function, Defaults, Super } = translate_definition(Kind, Line, Name, Args, Guards, Body, E),
 
   DefaultsLength = length(Defaults),
-  elixir_tracker:record_defaults(Tuple, Kind, Module, DefaultsLength),
+  elixir_locals:record_defaults(Tuple, Kind, Module, DefaultsLength),
 
-  File  = TS#elixir_scope.file,
-  Table = table(Module),
+  File   = E#elixir_env.file,
+  Table  = table(Module),
   CTable = clauses_table(Module),
 
-  compile_super(Module, TS),
-  check_previous_defaults(Table, Line, Name, Arity, Kind, DefaultsLength, S),
+  compile_super(Module, Super, E),
+  check_previous_defaults(Table, Line, Name, Arity, Kind, DefaultsLength, E),
 
   store_each(CheckClauses, Kind, File, Location,
     Table, CTable, DefaultsLength, Function),
   [store_each(false, Kind, File, Location, Table, CTable, 0,
     default_function_for(Kind, Name, Default)) || Default <- Defaults],
 
-  { Name, Arity };
-
-store_definition(Kind, Line, CheckClauses, Module, Name, Args, Guards, Body, RawS) ->
-  store_definition(Kind, Line, CheckClauses, Module, Name, Args, Guards, Body, elixir_scope:deserialize(RawS)).
+  { Name, Arity }.
 
 %% @on_definition
 
-run_on_definition_callbacks(Kind, Line, Module, Name, Args, Guards, Expr, S, CO) ->
-  case elixir_compiler:get_opt(internal, CO) of
+run_on_definition_callbacks(Kind, Line, Module, Name, Args, Guards, Expr, E) ->
+  case elixir_compiler:get_opt(internal) of
     true ->
       ok;
     _ ->
-      Env = elixir_scope:to_ex_env({ Line, S }),
+      Env = elixir_env:env_to_ex({ Line, E }),
       Callbacks = 'Elixir.Module':get_attribute(Module, on_definition),
       [Mod:Fun(Env, Kind, Name, Args, Guards, Expr) || { Mod, Fun } <- Callbacks]
   end.
 
-%% Retrieve @file or fallback to default
+%% Retrieve location from meta file or @file, otherwise nil
 
-retrieve_file(Line, Module, S, CO) ->
-  case elixir_compiler:get_opt(internal, CO) of
-    true -> { elixir_utils:characters_to_list(S#elixir_scope.file), Line };
-    _ ->
-      case 'Elixir.Module':get_attribute(Module, file) of
-        nil  -> { elixir_utils:characters_to_list(S#elixir_scope.file), Line };
-        Else ->
-          'Elixir.Module':delete_attribute(Module, file),
-          { Else, 0 }
-      end
+retrieve_location(Line, File, Module) ->
+  case not(elixir_compiler:get_opt(internal)) andalso
+       'Elixir.Module':get_attribute(Module, file) of
+    X when X == nil; X == false ->
+      if
+        is_binary(File) ->
+          { elixir_utils:characters_to_list(elixir_utils:relative_to_cwd(File)), Line };
+        true ->
+          nil
+      end;
+    X ->
+      'Elixir.Module':delete_attribute(Module, file),
+      { X, 0 }
   end.
 
 %% Compile super
 
-compile_super(Module, #elixir_scope{function=Function, super=true}) ->
+compile_super(Module, true, #elixir_env{function=Function}) ->
   elixir_def_overridable:store(Module, Function, true);
-
-compile_super(_Module, _S) -> ok.
+compile_super(_Module, _, _E) -> ok.
 
 %% Translate the given call and expression given
 %% and then store it in memory.
 
-translate_definition(Kind, Line, Name, RawArgs, RawGuards, RawBody, S) when is_integer(Line) ->
-  Args    = elixir_quote:linify(Line, RawArgs),
-  Guards  = elixir_quote:linify(Line, RawGuards),
-  Body    = elixir_quote:linify(Line, RawBody),
-  Arity   = length(Args),
+translate_definition(Kind, Line, Name, Args, Guards, Body, E) when is_integer(Line) ->
+  Arity = length(Args),
 
   %% Macros receive a special argument on invocation. Notice it does
   %% not affect the arity of the stored function, but the clause
   %% already contains it.
-  ExtendedArgs = case is_macro(Kind) of
+  AllArgs = case is_macro(Kind) of
     true  -> [{ '_@CALLER', [{line,Line}], nil }|Args];
     false -> Args
   end,
 
-  { Unpacked, Defaults } = elixir_def_defaults:unpack(Kind, Name, ExtendedArgs, S),
-  { Clauses, TS } = translate_clause(Line, Kind, Unpacked, Guards, Body, S),
+  { EArgs, EGuards, EBody, _ } = elixir_exp_clauses:def(fun elixir_def_defaults:expand/2,
+                                   AllArgs, Guards, expr_from_body(Line, Body), E),
+
+  S = elixir_env:env_to_scope(E),
+  { Unpacked, Defaults } = elixir_def_defaults:unpack(Kind, Name, EArgs, S),
+  { Clauses, Super } = translate_clause(Body, Line, Kind, Unpacked, EGuards, EBody, S),
 
   Function = { function, Line, Name, Arity, Clauses },
-  { Function, Defaults, TS }.
+  { Function, Defaults, Super }.
 
-translate_clause(_Line, _Kind, _Unpacked, [], nil, S) ->
-  { [], S };
-
-translate_clause(Line, Kind, _Unpacked, _Guards, nil, #elixir_scope{file=File}) ->
-  elixir_errors:syntax_error(Line, File, "missing do keyword in ~ts", [Kind]);
-
-translate_clause(Line, Kind, Unpacked, Guards, Body, S) ->
-  Expr = expr_from_body(Line, Body),
-
-  { TClause, TS } = elixir_clauses:assigns_block(Line,
-    fun elixir_translator:translate/2, Unpacked, [Expr], Guards, S),
+translate_clause(nil, _Line, _Kind, _Args, [], _Body, _S) ->
+  { [], false };
+translate_clause(nil, Line, Kind, _Args, _Guards, _Body, #elixir_scope{file=File}) ->
+  elixir_errors:form_error(Line, File, ?MODULE, { missing_do, Kind });
+translate_clause(_, Line, Kind, Args, Guards, Body, S) ->
+  { TClause, TS } = elixir_clauses:clause(Line,
+    fun elixir_translator:translate_many/2, Args, Body, Guards, S),
 
   %% Set __CALLER__ if used
   FClause = case is_macro(Kind) andalso TS#elixir_scope.caller of
     true  ->
       FBody = { 'match', Line,
         { 'var', Line, '__CALLER__' },
-        ?wrap_call(Line, elixir_scope, to_ex_env, [{ var, Line, '_@CALLER' }])
+        ?wrap_call(Line, elixir_env, env_to_ex, [{ var, Line, '_@CALLER' }])
       },
       setelement(5, TClause, [FBody|element(5, TClause)]);
     false -> TClause
   end,
 
-  { [FClause], TS }.
+  { [FClause], TS#elixir_scope.super }.
 
+expr_from_body(_Line, nil)            -> nil;
 expr_from_body(_Line, [{ do, Expr }]) -> Expr;
 expr_from_body(Line, Else)            -> { 'try', [{line,Line}], [Else] }.
 
@@ -235,20 +197,20 @@ is_macro(_)         -> false.
 % Unwrap the functions stored in the functions table.
 % It returns a list of all functions to be exported, plus the macros,
 % and the body of all functions.
-unwrap_stored_definitions(File, Module) ->
+unwrap_definitions(Module) ->
   Table = table(Module),
   CTable = clauses_table(Module),
   ets:delete(Table, last),
-  unwrap_stored_definition(ets:tab2list(Table), CTable, File, [], [], [], [], [], [], []).
+  unwrap_definition(ets:tab2list(Table), CTable, [], [], [], [], [], [], []).
 
-unwrap_stored_definition([Fun|T], CTable, File, All, Exports, Private, Def, Defmacro, Functions, Tail) ->
+unwrap_definition([Fun|T], CTable, All, Exports, Private, Def, Defmacro, Functions, Tail) ->
   Tuple   = element(1, Fun),
   Clauses = [Clause || { _, Clause } <- ets:lookup(CTable, Tuple)],
 
   { NewFun, NewExports, NewPrivate, NewDef, NewDefmacro } =
     case Clauses of
       [] -> { false, Exports, Private, Def, Defmacro };
-      _  -> unwrap_stored_definition(element(2, Fun), Tuple, Fun, Exports, Private, Def, Defmacro)
+      _  -> unwrap_definition(element(2, Fun), Tuple, Fun, Exports, Private, Def, Defmacro)
     end,
 
   { NewFunctions, NewTail } = case NewFun of
@@ -257,39 +219,35 @@ unwrap_stored_definition([Fun|T], CTable, File, All, Exports, Private, Def, Defm
       { Functions, Tail };
     _ ->
       NewAll = [Tuple|All],
-      function_for_stored_definition(NewFun, Clauses, File, Functions, Tail)
+      function_for_stored_definition(NewFun, Clauses, Functions, Tail)
   end,
 
-  unwrap_stored_definition(T, CTable, File, NewAll, NewExports, NewPrivate,
+  unwrap_definition(T, CTable, NewAll, NewExports, NewPrivate,
     NewDef, NewDefmacro, NewFunctions, NewTail);
-
-unwrap_stored_definition([], _CTable, _File, All, Exports, Private, Def, Defmacro, Functions, Tail) ->
+unwrap_definition([], _CTable, All, Exports, Private, Def, Defmacro, Functions, Tail) ->
   { All, Exports, Private, ordsets:from_list(Def),
     ordsets:from_list(Defmacro), lists:reverse(Tail ++ Functions) }.
 
-unwrap_stored_definition(def, Tuple, Fun, Exports, Private, Def, Defmacro) ->
+unwrap_definition(def, Tuple, Fun, Exports, Private, Def, Defmacro) ->
   { Fun, [Tuple|Exports], Private, [Tuple|Def], Defmacro };
-
-unwrap_stored_definition(defmacro, { Name, Arity } = Tuple, Fun, Exports, Private, Def, Defmacro) ->
+unwrap_definition(defmacro, { Name, Arity } = Tuple, Fun, Exports, Private, Def, Defmacro) ->
   Macro = { ?elixir_macro(Name), Arity + 1 },
   { setelement(1, Fun, Macro), [Macro|Exports], Private, Def, [Tuple|Defmacro] };
-
-unwrap_stored_definition(defp, Tuple, Fun, Exports, Private, Def, Defmacro) ->
+unwrap_definition(defp, Tuple, Fun, Exports, Private, Def, Defmacro) ->
   %% { Name, Arity }, Kind, Line, Check, Defaults
   Info = { Tuple, defp, element(3, Fun), element(5, Fun), element(7, Fun) },
   { Fun, Exports, [Info|Private], Def, Defmacro };
-
-unwrap_stored_definition(defmacrop, Tuple, Fun, Exports, Private, Def, Defmacro) ->
+unwrap_definition(defmacrop, Tuple, Fun, Exports, Private, Def, Defmacro) ->
   %% { Name, Arity }, Kind, Line, Check, Defaults
   Info  = { Tuple, defmacrop, element(3, Fun), element(5, Fun), element(7, Fun) },
   { false, Exports, [Info|Private], Def, Defmacro }.
 
 %% Helpers
 
-function_for_stored_definition({{Name,Arity}, _, Line, _, _, {File, _}, _}, Clauses, File, Functions, Tail) ->
+function_for_stored_definition({{Name,Arity}, _, Line, _, _, nil, _}, Clauses, Functions, Tail) ->
   { [{ function, Line, Name, Arity, Clauses }|Functions], Tail };
 
-function_for_stored_definition({{Name,Arity}, _, Line, _, _, Location, _}, Clauses, _File, Functions, Tail) ->
+function_for_stored_definition({{Name,Arity}, _, Line, _, _, Location, _}, Clauses, Functions, Tail) ->
   { Functions, [
     { function, Line, Name, Arity, Clauses },
     { attribute, Line, file, Location } | Tail
@@ -349,10 +307,10 @@ check_valid_defaults(Line, File, Name, Arity, Kind, _, 0) ->
 check_valid_defaults(Line, File, Name, Arity, Kind, _, _) ->
   elixir_errors:form_error(Line, File, ?MODULE, { clauses_with_defaults, { Kind, Name, Arity } }).
 
-check_previous_defaults(Table, Line, Name, Arity, Kind, Defaults, S) ->
+check_previous_defaults(Table, Line, Name, Arity, Kind, Defaults, E) ->
   Matches = ets:match(Table, { { Name, '$2' }, '$1', '_', '_', '_', '_', '$3' }),
   [ begin
-      elixir_errors:form_error(Line, S#elixir_scope.file, ?MODULE,
+      elixir_errors:form_error(Line, E#elixir_env.file, ?MODULE,
         { defs_with_defaults, Name, { Kind, Arity }, { K, A } })
     end || [K, A, D] <- Matches, A /= Arity, D /= 0, defaults_conflict(A, D, Arity, Defaults)].
 
@@ -360,9 +318,8 @@ defaults_conflict(A, D, Arity, Defaults) ->
   ((Arity >= (A - D)) andalso (Arity < A)) orelse
     ((A >= (Arity - Defaults)) andalso (A < Arity)).
 
-assert_no_aliases_name(Line, '__aliases__', [Atom], #elixir_scope{file=File}) when is_atom(Atom) ->
-  Message = "function names should start with lowercase characters or underscore, invalid name ~ts",
-  elixir_errors:syntax_error(Line, File, Message, [Atom]);
+assert_no_aliases_name(Line, '__aliases__', [Atom], #elixir_env{file=File}) when is_atom(Atom) ->
+  elixir_errors:form_error(Line, File, ?MODULE, { no_alias, Atom });
 
 assert_no_aliases_name(_Meta, _Aliases, _Args, _S) ->
   ok.
@@ -392,4 +349,13 @@ format_error({ungrouped_clause,{Kind,Name,Arity,OrigLine,OrigFile}}) ->
     [Kind, Kind, Name, Arity, OrigFile, OrigLine]);
 
 format_error({changed_kind,{Name,Arity,Previous,Current}}) ->
-  io_lib:format("~ts ~ts/~B already defined as ~ts", [Current, Name, Arity, Previous]).
+  io_lib:format("~ts ~ts/~B already defined as ~ts", [Current, Name, Arity, Previous]);
+
+format_error({no_alias, Atom}) ->
+  io_lib:format("function names should start with lowercase characters or underscore, invalid name ~ts", [Atom]);
+
+format_error({invalid_def, Kind, NameAndArgs}) ->
+  io_lib:format("invalid syntax in ~ts ~ts", [Kind, 'Elixir.Macro':to_string(NameAndArgs)]);
+
+format_error({missing_do, Kind}) ->
+  io_lib:format("missing do keyword in ~ts", [Kind]).

lib/elixir/src/elixir_def_defaults.erl

@@ -1,8 +1,18 @@
 % Handle default clauses for function definitions.
 -module(elixir_def_defaults).
--export([unpack/4]).
+-export([expand/2, unpack/4]).
 -include("elixir.hrl").
 
+expand(Args, E) ->
+  lists:mapfoldl(fun
+    ({ '//', Meta, [Left, Right] }, Acc) ->
+      { ELeft, EL } = elixir_exp:expand(Left, Acc),
+      { ERight, _ } = elixir_exp:expand(Right, Acc#elixir_env{context=nil}),
+      { { '//', Meta, [ELeft, ERight] }, EL };
+    (Left, Acc) ->
+      elixir_exp:expand(Left, Acc)
+  end, E, Args).
+
 unpack(Kind, Name, Args, S) ->
   unpack_each(Kind, Name, Args, [], [], S).
 
@@ -15,12 +25,12 @@ unpack_each(Kind, Name, [{'//', Line, [Expr, _]}|T] = List, Acc, Clauses, S) ->
   Base = build_match(Acc, Line, []),
   { Args, Invoke } = extract_defaults(List, Line, length(Base), [], []),
 
-  { DefArgs, SA }   = elixir_clauses:assigns(fun elixir_translator:translate/2, Base ++ Args, S),
-  { InvokeArgs, _ } = elixir_translator:translate_args(Base ++ Invoke, SA),
+  { DefArgs, SA }  = elixir_clauses:match(fun elixir_translator:translate_many/2, Base ++ Args, S),
+  { DefInvoke, _ } = elixir_translator:translate_args(Base ++ Invoke, SA),
 
   Call = { call, Line,
     { atom, Line, name_for_kind(Kind, Name) },
-    InvokeArgs
+    DefInvoke
   },
 
   Clause = { clause, Line, DefArgs, [], [Call] },

lib/elixir/src/elixir_def_local.erl

@@ -1,62 +0,0 @@
-%% Module responsible for local invocation of macros and functions.
--module(elixir_def_local).
--export([macro_for/3, function_for/3]).
--include("elixir.hrl").
-
-macro_for(_Tuple, _All, #elixir_scope{module=nil}) -> false;
-
-macro_for(Tuple, All, #elixir_scope{module=Module}) ->
-  try elixir_def:lookup_definition(Module, Tuple) of
-    { { Tuple, Kind, Line, _, _, _, _ }, Clauses } when Kind == defmacro, Clauses /= [];
-                                                        All, Kind == defmacrop, Clauses /= [] ->
-      fun() -> get_function(Line, Module, Clauses) end;
-    _ ->
-      false
-  catch
-    error:badarg -> false
-  end.
-
-function_for(Module, Name, Arity) ->
-  Tuple = { Name, Arity },
-  case elixir_def:lookup_definition(Module, Tuple) of
-    { { Tuple, _, Line, _, _, _, _ }, Clauses } when Clauses /= [] ->
-      %% There is no need to record such calls
-      %% since they come from funtions that were
-      %% already analyzed at compilation time.
-      get_function(Line, Module, Clauses);
-    _ ->
-      [_|T] = erlang:get_stacktrace(),
-      erlang:raise(error, undef, [{Module,Name,Arity,[]}|T])
-  end.
-
-get_function(Line, Module, Clauses) ->
-  RewrittenClauses = [rewrite_clause(Clause, Module) || Clause <- Clauses],
-  Fun = { 'fun', Line, {clauses, RewrittenClauses } },
-  { value, Result, _Binding } = erl_eval:exprs([Fun], []),
-  Result.
-
-rewrite_clause({ call, Line, { atom, Line, RawName }, Args }, Module) ->
-  Remote = { remote, Line,
-    { atom, Line, ?MODULE },
-    { atom, Line, function_for }
-  },
-
-  %% If we have a macro, its arity in the table is
-  %% actually one less than in the function call
-  { Name, Arity } = case atom_to_list(RawName) of
-    "MACRO-" ++ Rest -> { list_to_atom(Rest), length(Args) - 1 };
-    _ -> { RawName, length(Args) }
-  end,
-
-  FunCall = { call, Line, Remote, [
-    { atom, Line, Module }, { atom, Line, Name }, { integer, Line, Arity }
-  ] },
-  { call, Line, FunCall, Args };
-
-rewrite_clause(Tuple, Module) when is_tuple(Tuple) ->
-  list_to_tuple(rewrite_clause(tuple_to_list(Tuple), Module));
-
-rewrite_clause(List, Module) when is_list(List) ->
-  [rewrite_clause(Item, Module) || Item <- List];
-
-rewrite_clause(Else, _) -> Else.

lib/elixir/src/elixir_def_overridable.erl

@@ -1,7 +1,7 @@
 % Holds the logic responsible for defining overridable functions and handling super.
 -module(elixir_def_overridable).
--export([store_pending/1, is_defined/2, ensure_defined/4,
-  assign_args/3, name/2, store/3, format_error/1]).
+-export([store_pending/1, ensure_defined/4,
+  name/2, store/3, format_error/1]).
 -include("elixir.hrl").
 
 overridable(Module) ->
@@ -12,32 +12,13 @@ overridable(Module, Value) ->
 
 %% Check if an overridable function is defined.
 
-is_defined(Module, Tuple) ->
+ensure_defined(Meta, Module, Tuple, S) ->
   Overridable = overridable(Module),
   case orddict:find(Tuple, Overridable) of
-    { ok, { _, _, _, _ } } -> true;
-    _ -> false
+    { ok, { _, _, _, _ } } -> ok;
+    _ -> elixir_errors:form_error(Meta, S#elixir_scope.file, ?MODULE, { no_super, Module, Tuple })
   end.
 
-ensure_defined(Meta, Module, Tuple, S) ->
-  case is_defined(Module, Tuple) of
-    true -> ok;
-    _    -> elixir_errors:form_error(Meta, S#elixir_scope.file, ?MODULE, { no_super, Module, Tuple })
-  end.
-
-%% Assign pseudo variables to the given vars.
-
-assign_args(Line, Args, S) when is_integer(Line) ->
-  { FArgs, _ } = lists:mapfoldl(fun(X, Acc) -> assign_args(Line, X, Acc, S) end, 1, Args),
-  FArgs.
-
-assign_args(Line, X, Acc, _) ->
-  Match = { match, Line, X, { var, Line, super_arg(Acc) } },
-  { Match, Acc + 1 }.
-
-super_arg(Counter) ->
-  ?atom_concat(['_@S', Counter]).
-
 %% Gets the name based on the function and stored overridables
 
 name(Module, Function) ->
@@ -62,7 +43,12 @@ store(Module, Function, GenerateName) ->
         false -> { Kind, Name }
       end,
 
-      'Elixir.Module.DispatchTracker':reattach(Module, Kind, { Name, Arity }, Neighbours),
+      case code:is_loaded('Elixir.Module.LocalsTracker') of
+        { _, _ } ->
+          'Elixir.Module.LocalsTracker':reattach(Module, Kind, { Name, Arity }, Neighbours);
+        _ ->
+          ok
+      end,
 
       Def = { function, Line, FinalName, Arity, Clauses },
       elixir_def:store_each(false, FinalKind, File, Location,
@@ -81,7 +67,7 @@ format_error({ no_super, Module, { Name, Arity } }) ->
   Bins   = [format_fa(X) || { X, { _, _, _, _ } } <- overridable(Module)],
   Joined = 'Elixir.Enum':join(Bins, <<", ">>),
   io_lib:format("no super defined for ~ts/~B in module ~ts. Overridable functions available are: ~ts",
-    [Name, Arity, elixir_errors:inspect(Module), Joined]).
+    [Name, Arity, elixir_aliases:inspect(Module), Joined]).
 
 format_fa({ Name, Arity }) ->
   A = atom_to_binary(Name, utf8),

lib/elixir/src/elixir_dispatch.erl

@@ -2,32 +2,33 @@
 %% This module access the information stored on the scope
 %% by elixir_import and therefore assumes it is normalized (ordsets)
 -module(elixir_dispatch).
--export([default_macros/0, default_functions/0, default_requires/0,
-  dispatch_require/6, dispatch_import/5,
+-export([dispatch_import/5, dispatch_require/6,
   require_function/5, import_function/4,
-  expand_import/6, expand_require/6, find_import/4,
-  format_error/1, in_erlang_functions/0, in_erlang_macros/0]).
+  expand_import/5, expand_require/5,
+  default_functions/0, default_macros/0, default_requires/0,
+  find_import/4, format_error/1]).
 -include("elixir.hrl").
-
 -import(ordsets, [is_element/2]).
--define(builtin, 'Elixir.Kernel').
+-define(kernel, 'Elixir.Kernel').
 
 default_functions() ->
-  [ { ?builtin, elixir_imported_functions() } ].
+  [ { ?kernel, elixir_imported_functions() } ].
 default_macros() ->
-  [ { ?builtin, elixir_imported_macros() } ].
+  [ { ?kernel, elixir_imported_macros() } ].
 default_requires() ->
   [ 'Elixir.Integer', 'Elixir.Kernel', 'Elixir.Kernel.Typespec', 'Elixir.Record' ].
 
-find_import(Meta, Name, Arity, S) ->
+find_import(Meta, Name, Arity, E) ->
   Tuple = { Name, Arity },
 
-  case find_dispatch(Meta, Tuple, S) of
+  case find_dispatch(Meta, Tuple, [], E) of
     { function, Receiver } ->
-      elixir_tracker:record_import(Tuple, Receiver, S#elixir_scope.module, S#elixir_scope.function),
+      elixir_lexical:record_import(Receiver, E#elixir_env.lexical_tracker),
+      elixir_locals:record_import(Tuple, Receiver, E#elixir_env.module, E#elixir_env.function),
       Receiver;
     { macro, Receiver } ->
-      elixir_tracker:record_import(Tuple, Receiver, S#elixir_scope.module, S#elixir_scope.function),
+      elixir_lexical:record_import(Receiver, E#elixir_env.lexical_tracker),
+      elixir_locals:record_import(Tuple, Receiver, E#elixir_env.module, E#elixir_env.function),
       Receiver;
     _ ->
       false
@@ -35,236 +36,200 @@ find_import(Meta, Name, Arity, S) ->
 
 %% Function retrieval
 
-import_function(Meta, Name, Arity, S) ->
+import_function(Meta, Name, Arity, E) ->
   Tuple = { Name, Arity },
-  case find_dispatch(Meta, Tuple, S) of
+  case find_dispatch(Meta, Tuple, [], E) of
     { function, Receiver } ->
-      elixir_tracker:record_import(Tuple, Receiver, S#elixir_scope.module, S#elixir_scope.function),
-      remote_function(Meta, Receiver, Name, Arity, S);
+      elixir_lexical:record_import(Receiver, E#elixir_env.lexical_tracker),
+      elixir_locals:record_import(Tuple, Receiver, E#elixir_env.module, E#elixir_env.function),
+      remote_function(Receiver, Name, Arity, E);
     { macro, _Receiver } ->
       false;
     { import, Receiver } ->
-      require_function(Meta, Receiver, Name, Arity, S);
+      require_function(Meta, Receiver, Name, Arity, E);
     false ->
       case elixir_import:special_form(Name, Arity) of
         true  -> false;
         false ->
-          elixir_tracker:record_local(Tuple, S#elixir_scope.module, S#elixir_scope.function),
-          { { 'fun', ?line(Meta), { function, Name, Arity } }, S }
+          elixir_locals:record_local(Tuple, E#elixir_env.module, E#elixir_env.function),
+          { local, Name, Arity }
       end
   end.
 
-require_function(Meta, Receiver, Name, Arity, S) ->
+require_function(_Meta, Receiver, Name, Arity, E) ->
+  case is_element({ Name, Arity }, get_optional_macros(Receiver)) of
+    true  -> false;
+    false -> remote_function(Receiver, Name, Arity, E)
+  end.
+
+remote_function(Receiver, Name, Arity, E) ->
+  elixir_lexical:record_remote(Receiver, E#elixir_env.lexical_tracker),
+  Tuple = { Name, Arity },
+  Final =
+    case (Receiver == ?kernel) andalso is_element(Tuple, in_erlang_functions()) of
+      true  -> erlang;
+      false -> Receiver
+    end,
+  { remote, Final, Name, Arity }.
+
+%% Dispatches
+
+dispatch_import(Meta, Name, Args, E, Callback) ->
+  Arity = length(Args),
+  case expand_import(Meta, { Name, Arity }, Args, E, []) of
+    { ok, Receiver, Quoted } ->
+      expand_quoted(Meta, Receiver, Name, Arity, Quoted, E);
+    { ok, Receiver } ->
+      elixir_exp:expand({ { '.', [], [Receiver, Name] }, Meta, Args }, E);
+    error ->
+      Callback()
+  end.
+
+dispatch_require(Meta, Receiver, Name, Args, E, Callback) when is_atom(Receiver) ->
+  Arity = length(Args),
   Tuple = { Name, Arity },
 
-  case is_element(Tuple, get_optional_macros(Receiver)) of
-    true  -> false;
+  case (Receiver == ?kernel) andalso is_element(Tuple, in_erlang_functions()) of
+    true  -> Callback(erlang);
     false ->
-      elixir_tracker:record_remote(Tuple, Receiver, S#elixir_scope.module, S#elixir_scope.function),
-      remote_function(Meta, Receiver, Name, Arity, S)
-  end.
-
-%% Function dispatch
-
-dispatch_import(Meta, Name, Args, S, Callback) ->
-  Module = S#elixir_scope.module,
-  Arity  = length(Args),
-  Tuple  = { Name, Arity },
-
-  case find_dispatch(Meta, Tuple, S) of
-    { function, Receiver } ->
-      elixir_tracker:record_import(Tuple, Receiver, Module, S#elixir_scope.function),
-      Endpoint = case (Receiver == ?builtin) andalso is_element(Tuple, in_erlang_functions()) of
-        true  -> erlang;
-        false -> Receiver
-      end,
-      elixir_translator:translate_each({ { '.', Meta, [Endpoint, Name] }, Meta, Args }, S);
-    { import, Receiver } ->
-      elixir_tracker:record_remote(Tuple, Receiver, S#elixir_scope.module, S#elixir_scope.function),
-      elixir_translator:translate_each({ { '.', Meta, [Receiver, Name] }, [{require,false}|Meta], Args }, S);
-    Result ->
-      case do_expand_import(Meta, Tuple, Args, Module, S, Result) of
-        { error, noexpansion } ->
-          Callback();
-        { error, internal } ->
-          elixir_tracker:record_import(Tuple, ?builtin, Module, S#elixir_scope.function),
-          elixir_macros:translate({ Name, Meta, Args }, S);
-        { ok, _Receiver, Tree } ->
-          translate_expansion(Meta, Tree, S)
-      end
-  end.
-
-dispatch_require(Meta, Receiver, Name, Args, S, Callback) ->
-  Module = S#elixir_scope.module,
-  Arity  = length(Args),
-  Tuple  = { Name, Arity },
-
-  case (Receiver == ?builtin) andalso is_element(Tuple, in_erlang_functions()) of
-    true ->
-      elixir_tracker:record_remote(Tuple, Receiver, S#elixir_scope.module, S#elixir_scope.function),
-      { TArgs, SA } = elixir_translator:translate_args(Args, S),
-      { ?wrap_call(?line(Meta), erlang, Name, TArgs), SA };
-    false ->
-      case expand_require(Meta, Receiver, Tuple, Args, Module, S) of
-        { error, noexpansion } ->
-          Callback();
-        { error, internal } ->
-          elixir_tracker:record_remote(Tuple, ?builtin, S#elixir_scope.module, S#elixir_scope.function),
-          elixir_macros:translate({ Name, Meta, Args }, S);
-        { ok, _Receiver, Tree } ->
-          translate_expansion(Meta, Tree, S)
-      end
-  end.
-
-%% Macros expansion
-
-expand_import(Meta, Tuple, Args, Module, Extra, S) ->
-  Result = find_dispatch(Meta, Tuple, Extra, S),
-  do_expand_import(Meta, Tuple, Args, Module, S, Result).
-
-do_expand_import(Meta, { Name, Arity } = Tuple, Args, Module, S, Result) ->
-  Function = S#elixir_scope.function,
-
-  Fun = (Function /= nil) andalso (Function /= Tuple) andalso
-        elixir_def_local:macro_for(Tuple, true, S),
-
-  case Fun of
-    false ->
-      do_expand_import_no_local(Meta, Tuple, Args, Module, S, Result);
-    _ ->
-      case Result of
-        { Kind, Receiver } when Kind == import; Receiver == Module ->
-          do_expand_import_no_local(Meta, Tuple, Args, Module, S, Result);
-        { _, Receiver } ->
-          Error = { macro_conflict, { Receiver, Name, Arity } },
-          elixir_errors:form_error(Meta, S#elixir_scope.file, ?MODULE, Error);
-        false ->
-          elixir_tracker:record_local(Tuple, Module, S#elixir_scope.function),
-          { ok, Module, expand_macro_fun(Meta, Fun(), Module, Name, Args, Module, S) }
-      end
-  end.
-
-do_expand_import_no_local(Meta, { Name, Arity } = Tuple, Args, Module, S, Result) ->
-  case Result of
-    { macro, ?builtin } ->
-      case is_element(Tuple, in_erlang_macros()) of
-        true  -> { error, internal };
-        false ->
-          elixir_tracker:record_import(Tuple, ?builtin, Module, S#elixir_scope.function),
-          { ok, ?builtin, expand_macro_named(Meta, ?builtin, Name, Arity, Args, Module, S) }
-      end;
-    { macro, Receiver } ->
-      elixir_tracker:record_import(Tuple, Receiver, Module, S#elixir_scope.function),
-      { ok, Receiver, expand_macro_named(Meta, Receiver, Name, Arity, Args, Module, S) };
-    { import, Receiver } ->
-      expand_require(Meta, Receiver, Tuple, Args, Module, S);
-    _ ->
-      { error, noexpansion }
-  end.
-
-expand_require(Meta, ?builtin, { Name, Arity } = Tuple, Args, Module, S) ->
-  case is_element(Tuple, in_erlang_macros()) of
-    true  -> { error, internal };
-    false ->
-      case is_element(Tuple, in_elixir_macros()) of
-        true  ->
-          elixir_tracker:record_remote(Tuple, ?builtin, S#elixir_scope.module, S#elixir_scope.function),
-          { ok, ?builtin, expand_macro_named(Meta, ?builtin, Name, Arity, Args, Module, S) };
-        false ->
-          { error, noexpansion }
+      case expand_require(Meta, Receiver, Tuple, Args, E) of
+        { ok, Receiver, Quoted } -> expand_quoted(Meta, Receiver, Name, Arity, Quoted, E);
+        error -> Callback(Receiver)
       end
   end;
 
-expand_require(Meta, Receiver, { Name, Arity } = Tuple, Args, Module, S) ->
-  Function = S#elixir_scope.function,
+dispatch_require(_Meta, Receiver, _Name, _Args, _E, Callback) ->
+  Callback(Receiver).
 
-  Fun = (Module == Receiver) andalso (Function /= nil) andalso
-        (Function /= Tuple) andalso elixir_def_local:macro_for(Tuple, false, S),
+%% Macros expansion
 
-  case Fun of
-    false ->
-      case is_element(Tuple, get_optional_macros(Receiver)) of
-        true  ->
-          elixir_tracker:record_remote(Tuple, Receiver, Module, Function),
-          { ok, Receiver, expand_macro_named(Meta, Receiver, Name, Arity, Args, Module, S) };
-        false -> { error, noexpansion }
-      end;
+expand_import(Meta, { Name, Arity } = Tuple, Args, E, Extra) ->
+  Module   = E#elixir_env.module,
+  Dispatch = find_dispatch(Meta, Tuple, Extra, E),
+  Function = E#elixir_env.function,
+  Local    = (Function /= nil) andalso (Function /= Tuple) andalso
+              elixir_locals:macro_for(Module, Name, Arity),
+
+  case Dispatch of
+    %% In case it is an import, or the receive is the same as the
+    %% current module (happens in bootstraping), we dispatch the import.
+    { Kind, Receiver } when Kind == import; Receiver == Module ->
+      do_expand_import(Meta, Tuple, Args, Module, E, Dispatch);
+
+    %% There is a local and an import. This is a conflict.
+    { _, Receiver } when Local /= false ->
+      Error = { macro_conflict, { Receiver, Name, Arity } },
+      elixir_errors:form_error(Meta, E#elixir_env.file, ?MODULE, Error);
+
+    %% There is no local. Dispatch the import.
+    _ when Local == false ->
+      do_expand_import(Meta, Tuple, Args, Module, E, Dispatch);
+
+    %% Dispatch to the local.
     _ ->
-      elixir_tracker:record_local(Tuple, Module, Function),
-      { ok, Receiver, expand_macro_fun(Meta, Fun(), Receiver, Name, Args, Module, S) }
+      elixir_locals:record_local(Tuple, Module, Function),
+      { ok, Module, expand_macro_fun(Meta, Local(), Module, Name, Args, E) }
+  end.
+
+do_expand_import(Meta, { Name, Arity } = Tuple, Args, Module, E, Result) ->
+  case Result of
+    { function, Receiver } ->
+      elixir_lexical:record_import(Receiver, E#elixir_env.lexical_tracker),
+      elixir_locals:record_import(Tuple, Receiver, Module, E#elixir_env.function),
+      Endpoint = case (Receiver == ?kernel) andalso is_element(Tuple, in_erlang_functions()) of
+        true  -> erlang;
+        false -> Receiver
+      end,
+      { ok, Endpoint };
+    { macro, Receiver } ->
+      elixir_lexical:record_import(Receiver, E#elixir_env.lexical_tracker),
+      elixir_locals:record_import(Tuple, Receiver, Module, E#elixir_env.function),
+      { ok, Receiver, expand_macro_named(Meta, Receiver, Name, Arity, Args, E) };
+    { import, Receiver } ->
+      case expand_require([{require,false}|Meta], Receiver, Tuple, Args, E) of
+        { ok, _, _ } = Response -> Response;
+        error -> { ok, Receiver }
+      end;
+    false ->
+      error
+  end.
+
+expand_require(Meta, Receiver, { Name, Arity } = Tuple, Args, E) ->
+  Module = E#elixir_env.module,
+
+  case is_element(Tuple, get_optional_macros(Receiver)) of
+    true ->
+      Requires = E#elixir_env.requires,
+      case (Receiver == Module) orelse is_element(Receiver, Requires) orelse skip_require(Meta) of
+        true  ->
+          elixir_lexical:record_remote(Receiver, E#elixir_env.lexical_tracker),
+          { ok, Receiver, expand_macro_named(Meta, Receiver, Name, Arity, Args, E) };
+        false ->
+          Info = { unrequired_module, { Receiver, Name, length(Args), Requires } },
+          elixir_errors:form_error(Meta, E#elixir_env.file, ?MODULE, Info)
+      end;
+    false ->
+      error
   end.
 
 %% Expansion helpers
 
-expand_macro_fun(Meta, Fun, Receiver, Name, Args, Module, S) ->
-  Requires = S#elixir_scope.requires,
-  case (Receiver == Module) or is_element(Receiver, Requires) or skip_require(Meta) of
-    true  -> ok;
-    false ->
-      Tuple = { unrequired_module, { Receiver, Name, length(Args), Requires } },
-      elixir_errors:form_error(Meta, S#elixir_scope.file, ?MODULE, Tuple)
-  end,
-
+expand_macro_fun(Meta, Fun, Receiver, Name, Args, E) ->
   Line = ?line(Meta),
-  SArg = {Line,S},
+  EArg = { Line, E },
 
   try
-    apply(Fun, [SArg|Args])
+    apply(Fun, [EArg|Args])
   catch
     Kind:Reason ->
-      Info = { Receiver, Name, length(Args), location(Line, S) },
-      erlang:raise(Kind, Reason, munge_stacktrace(Info, erlang:get_stacktrace(), SArg))
+      Info = [{ Receiver, Name, length(Args), location(Line, E) }, mfa(Line, E)],
+      erlang:raise(Kind, Reason, prune_stacktrace(Info, erlang:get_stacktrace(), EArg))
   end.
 
-expand_macro_named(Meta, Receiver, Name, Arity, Args, Module, S) ->
+expand_macro_named(Meta, Receiver, Name, Arity, Args, E) ->
   ProperName  = ?elixir_macro(Name),
   ProperArity = Arity + 1,
   Fun         = fun Receiver:ProperName/ProperArity,
-  expand_macro_fun(Meta, Fun, Receiver, Name, Args, Module, S).
+  expand_macro_fun(Meta, Fun, Receiver, Name, Args, E).
 
-translate_expansion(Meta, Tree, S) ->
+expand_quoted(Meta, Receiver, Name, Arity, Quoted, E) ->
   Line = ?line(Meta),
+  Next = elixir_counter:next(),
 
   try
-    elixir_translator:translate_each(
-      elixir_quote:linify(Line, Tree),
-      S
-    )
+    elixir_exp:expand(
+      elixir_quote:linify_with_context_counter(Line, { Receiver, Next }, Quoted),
+      E)
   catch
     Kind:Reason ->
-      erlang:raise(Kind, Reason, munge_stacktrace(mfa(Line, S), erlang:get_stacktrace(), nil))
+      Info = [{ Receiver, Name, Arity, location(Line, E) }, mfa(Line, E)],
+      erlang:raise(Kind, Reason, prune_stacktrace(Info, erlang:get_stacktrace(), nil))
   end.
 
-mfa(Line, #elixir_scope{module=nil} = S) ->
-  { elixir_compiler, '__FILE__', 2, location(Line, S) };
+mfa(Line, #elixir_env{module=nil} = E) ->
+  { elixir_compiler, '__FILE__', 2, location(Line, E) };
+mfa(Line, #elixir_env{module=Module, function=nil} = E) ->
+  { Module, '__MODULE__', 0, location(Line, E) };
+mfa(Line, #elixir_env{module=Module, function={ Name, Arity }} = E) ->
+  { Module, Name, Arity, location(Line, E) }.
 
-mfa(Line, #elixir_scope{module=Module,function=nil} = S) ->
-  { Module, '__MODULE__', 0, location(Line, S) };
-
-mfa(Line, #elixir_scope{module=Module,function={ Name, Arity }} = S) ->
-  { Module, Name, Arity, location(Line, S) }.
-
-location(Line, S) ->
-  [{ file, S#elixir_scope.file }, { line, Line }].
+location(Line, E) ->
+  [{ file, elixir_utils:characters_to_list(E#elixir_env.file) }, { line, Line }].
 
 %% Helpers
 
 skip_require(Meta) ->
   lists:keyfind(require, 1, Meta) == { require, false }.
 
-find_dispatch(Meta, Tuple, S) ->
-  find_dispatch(Meta, Tuple, [], S).
-
-find_dispatch(Meta, Tuple, Extra, S) ->
-  case is_import(Meta, Tuple, S) of
+find_dispatch(Meta, Tuple, Extra, E) ->
+  case is_import(Meta) of
     { import, _ } = Import ->
       Import;
     false ->
-      Functions = S#elixir_scope.functions,
-      Macros = Extra ++ S#elixir_scope.macros,
-      File = S#elixir_scope.file,
-      FunMatch = find_dispatch(Tuple, Functions),
-      MacMatch = find_dispatch(Tuple, Macros),
+      Funs = E#elixir_env.functions,
+      Macs = Extra ++ E#elixir_env.macros,
+      FunMatch = find_dispatch(Tuple, Funs),
+      MacMatch = find_dispatch(Tuple, Macs),
 
       case { FunMatch, MacMatch } of
         { [], [Receiver] } -> { macro, Receiver };
@@ -274,21 +239,18 @@ find_dispatch(Meta, Tuple, Extra, S) ->
           { Name, Arity } = Tuple,
           [First, Second|_] = FunMatch ++ MacMatch,
           Error = { ambiguous_call, { First, Second, Name, Arity } },
-          elixir_errors:form_error(Meta, File, ?MODULE, Error)
+          elixir_errors:form_error(Meta, E#elixir_env.file, ?MODULE, Error)
       end
   end.
 
 find_dispatch(Tuple, List) ->
   [Receiver || { Receiver, Set } <- List, is_element(Tuple, Set)].
 
-is_import(Meta, Tuple, S) ->
+is_import(Meta) ->
   case lists:keyfind(import, 1, Meta) of
     { import, _ } = Import ->
       case lists:keyfind(context, 1, Meta) of
-        { context, Context } ->
-          not_an_import(Tuple, Context, S#elixir_scope.macro_functions)
-            andalso not_an_import(Tuple, Context, S#elixir_scope.macro_macros)
-            andalso Import;
+        { context, _ } -> Import;
         false ->
           false
       end;
@@ -296,58 +258,30 @@ is_import(Meta, Tuple, S) ->
       false
   end.
 
-not_an_import(Tuple, Context, Dict) ->
-  case orddict:find(Context, Dict) of
-    { ok, Pairs } ->
-      not lists:any(fun({ _, List }) -> lists:member(Tuple, List) end, Pairs);
-    error ->
-      true
-  end.
-
 %% We've reached the invoked macro, skip it with the rest
-munge_stacktrace(Info, [{ _, _, [S|_], _ }|_], S) ->
-  [Info];
-
+prune_stacktrace(Info, [{ _, _, [E|_], _ }|_], E) ->
+  Info;
 %% We've reached the elixir_dispatch internals, skip it with the rest
-munge_stacktrace(Info, [{ elixir_dispatch, _, _, _ }|_], _) ->
-  [Info];
-
-munge_stacktrace(Info, [H|T], S) ->
-  [H|munge_stacktrace(Info, T, S)];
-
-munge_stacktrace(Info, [], _) ->
-  [Info].
-
-remote_function(Meta, Receiver, Name, Arity, S) ->
-  Line  = ?line(Meta),
-
-  Final =
-    case (Receiver == ?builtin) andalso is_element({ Name, Arity }, in_erlang_functions()) of
-      true  -> erlang;
-      false -> Receiver
-    end,
-
-  { { 'fun', Line, { function,
-    { atom, Line, Final },
-    { atom, Line, Name },
-    { integer, Line, Arity}
-  } }, S }.
-
+prune_stacktrace(Info, [{ Mod, _, _, _ }|_], _) when Mod == elixir_dispatch; Mod == elixir_exp ->
+  Info;
+prune_stacktrace(Info, [H|T], E) ->
+  [H|prune_stacktrace(Info, T, E)];
+prune_stacktrace(Info, [], _) ->
+  Info.
 
 %% ERROR HANDLING
 
-format_error({ unrequired_module, { Receiver, Name, Arity, Required }}) ->
-  String = 'Elixir.Enum':join([elixir_errors:inspect(R) || R <- Required], <<", ">>),
-  io_lib:format("tried to invoke macro ~ts.~ts/~B but module was not required. Required: ~ts",
-    [elixir_errors:inspect(Receiver), Name, Arity, String]);
-
+format_error({ unrequired_module, { Receiver, Name, Arity, _Required }}) ->
+  Module = elixir_aliases:inspect(Receiver),
+  io_lib:format("you must require ~ts before invoking the macro ~ts.~ts/~B",
+    [Module, Module, Name, Arity]);
 format_error({ macro_conflict, { Receiver, Name, Arity } }) ->
-  io_lib:format("call to local macro ~ts/~B conflicts with imported ~ts.~ts/~B",
-    [Name, Arity, elixir_errors:inspect(Receiver), Name, Arity]);
-
+  io_lib:format("call to local macro ~ts/~B conflicts with imported ~ts.~ts/~B, "
+    "please rename the local macro or remove the conflicting import",
+    [Name, Arity, elixir_aliases:inspect(Receiver), Name, Arity]);
 format_error({ ambiguous_call, { Mod1, Mod2, Name, Arity }}) ->
   io_lib:format("function ~ts/~B imported from both ~ts and ~ts, call is ambiguous",
-    [Name, Arity, elixir_errors:inspect(Mod1), elixir_errors:inspect(Mod2)]).
+    [Name, Arity, elixir_aliases:inspect(Mod1), elixir_aliases:inspect(Mod2)]).
 
 %% INTROSPECTION
 
@@ -367,36 +301,27 @@ get_optional_macros(Receiver) ->
 
 elixir_imported_functions() ->
   try
-    ?builtin:'__info__'(functions)
+    ?kernel:'__info__'(functions)
   catch
     error:undef -> in_erlang_functions()
   end.
 
 elixir_imported_macros() ->
   try
-    ?builtin:'__info__'(macros)
-  catch
-    error:undef -> in_erlang_macros()
-  end.
-
-%% Macros imported from Kernel module. Sorted on compilation.
-
-in_elixir_macros() ->
-  try
-    ?builtin:'__info__'(macros)
+    ?kernel:'__info__'(macros)
   catch
     error:undef -> []
   end.
 
 %% Functions imported from Erlang module. MUST BE SORTED.
+%% TODO: Inline operators here as well once bug in erl_eval is fixed.
 in_erlang_functions() ->
   [
     { abs, 1 },
-    { atom_to_binary, 2 },
+    { apply, 2 },
+    { apply, 3 },
     { atom_to_list, 1 },
     { binary_part, 3 },
-    { binary_to_atom, 2 },
-    { binary_to_existing_atom, 2 },
     { binary_to_float, 1 },
     { binary_to_float, 2 },
     { binary_to_integer, 1 },
@@ -410,7 +335,9 @@ in_erlang_functions() ->
     { exit, 1 },
     % { float, 1 },
     { float_to_binary, 1 },
+    % { float_to_binary, 2 },
     { float_to_list, 1 },
+    % { float_to_list, 2 },
     { hd, 1 },
     { integer_to_binary, 1 },
     { integer_to_binary, 2 },
@@ -418,7 +345,6 @@ in_erlang_functions() ->
     { integer_to_list, 2 },
     { iolist_size, 1 },
     { iolist_to_binary, 1 },
-    { is_alive, 0 },
     { is_atom, 1 },
     { is_binary, 1 },
     { is_bitstring, 1 },
@@ -440,7 +366,6 @@ in_erlang_functions() ->
     { list_to_float, 1 },
     { list_to_integer, 1 },
     { list_to_integer, 2 },
-    { list_to_pid, 1 },
     { list_to_tuple, 1 },
     { make_ref, 0 },
     { max, 2 },
@@ -448,9 +373,9 @@ in_erlang_functions() ->
     { node, 0 },
     { node, 1 },
     % { now, 0 },
-    { pid_to_list, 1 },
     { round, 1 },
     { self, 0 },
+    { send, 2 },
     { size, 1 },
     { spawn, 1 },
     { spawn, 3 },
@@ -466,50 +391,3 @@ in_erlang_functions() ->
     { tuple_size, 1 },
     { tuple_to_list, 1 }
   ].
-
-%% Macros implemented in Erlang. MUST BE SORTED.
-in_erlang_macros() ->
-  [
-    {'!',1},
-    {'!=',2},
-    {'!==',2},
-    {'*',2},
-    {'+',1},
-    {'+',2},
-    {'++',2},
-    {'-',1},
-    {'-',2},
-    {'--',2},
-    {'/',2},
-    {'<',2},
-    {'<-',2},
-    {'<=',2},
-    {'==',2},
-    {'===',2},
-    {'>',2},
-    {'>=',2},
-    {'@',1},
-    {'and',2},
-    {apply,2},
-    {apply,3},
-    {'case',2},
-    {def,1},
-    {def,2},
-    {def,4},
-    {defmacro,1},
-    {defmacro,2},
-    {defmacro,4},
-    {defmacrop,1},
-    {defmacrop,2},
-    {defmacrop,4},
-    {defmodule,2},
-    {defp,1},
-    {defp,2},
-    {defp,4},
-    {in,2},
-    {'not',1},
-    {'or',2},
-    {'receive',1},
-    {'try',1},
-    {'xor',2}
-  ].

lib/elixir/src/elixir_env.erl

@@ -0,0 +1,49 @@
+-module(elixir_env).
+-include("elixir.hrl").
+-export([ex_to_env/1, env_to_scope/1, env_to_scope_with_vars/2, env_to_ex/1]).
+-export([mergea/2, mergev/2, merge_vars/2, merge_opt_vars/2]).
+
+%% Conversion in between #elixir_env, #elixir_scope and Macro.Env
+
+env_to_ex({ Line, #elixir_env{} = Env }) ->
+  erlang:setelement(1, Env#elixir_env{line=Line}, 'Elixir.Macro.Env').
+
+ex_to_env(Env) when element(1, Env) == 'Elixir.Macro.Env' ->
+  erlang:setelement(1, Env, elixir_env).
+
+env_to_scope(#elixir_env{module=Module,file=File,function=Function,context=Context}) ->
+  #elixir_scope{module=Module,file=File,function=Function,context=Context}.
+
+env_to_scope_with_vars(#elixir_env{} = Env, Vars) ->
+  (env_to_scope(Env))#elixir_scope{
+    vars=orddict:from_list(Vars),
+    counter=[{'_',length(Vars)}]
+  }.
+
+%% SCOPE MERGING
+
+%% Receives two scopes and return a new scope based on the second
+%% with their variables merged.
+mergev(E1, E2) when is_list(E1) ->
+  E2#elixir_env{
+    vars=merge_vars(E1, E2#elixir_env.vars),
+    export_vars=merge_opt_vars(E1, E2#elixir_env.export_vars)
+  };
+mergev(E1, E2) ->
+  E2#elixir_env{
+    vars=merge_vars(E1#elixir_env.vars, E2#elixir_env.vars),
+    export_vars=merge_opt_vars(E1#elixir_env.export_vars, E2#elixir_env.export_vars)
+  }.
+
+%% Receives two scopes and return the later scope
+%% keeping the variables from the first (counters,
+%% imports and everything else are passed forward).
+
+mergea(E1, E2) ->
+  E2#elixir_env{vars=E1#elixir_env.vars}.
+
+merge_vars(V1, V2) -> ordsets:union(V1, V2).
+
+merge_opt_vars(_V1, nil) -> nil;
+merge_opt_vars(nil, _V2) -> nil;
+merge_opt_vars(V1, V2)   -> ordsets:union(V1, V2).

lib/elixir/src/elixir_errors.erl

@@ -1,53 +1,40 @@
 % A bunch of helpers to help to deal with errors in Elixir source code.
 % This is not exposed in the Elixir language.
 -module(elixir_errors).
--export([syntax_error/3, syntax_error/4,
-  compile_error/3, compile_error/4, inspect/1,
-  form_error/4, parse_error/4, assert_module_scope/3,
-  assert_no_function_scope/3, assert_function_scope/3,
-  assert_no_match_scope/3, assert_no_guard_scope/3,
-  assert_no_match_or_guard_scope/3, warn/1,
+-export([compile_error/3, compile_error/4,
+  form_error/4, parse_error/4, warn/1,
   handle_file_warning/2, handle_file_warning/3, handle_file_error/2,
   deprecation/3, deprecation/4]).
 -include("elixir.hrl").
 
 warn(Warning) ->
-  elixir_code_server:cast(register_warning),
+  CompilerPid = get(elixir_compiler_pid),
+  if
+    CompilerPid =/= undefined ->
+      elixir_code_server:cast({ register_warning, CompilerPid });
+    true -> false
+  end,
   io:put_chars(standard_error, Warning).
 
-%% Handle inspecting for exceptions
+%% Raised during expansion/translation/compilation.
 
-inspect(Atom) when is_atom(Atom) ->
-  'Elixir.Inspect.Atom':inspect(Atom).
+-spec form_error(list(), binary(), module(), any()) -> no_return().
 
-%% Raised during macros translation.
+form_error(Meta, File, Module, Desc) ->
+  compile_error(Meta, File, format_error(Module, Desc)).
 
--spec syntax_error(non_neg_integer() | list(), file:filename(), binary() | string()) -> no_return().
--spec syntax_error(non_neg_integer() | list(), file:filename(), binary() | string(), list()) -> no_return().
-
-syntax_error(Meta, File, Message) when is_list(Message) ->
-  syntax_error(Meta, File, iolist_to_binary(Message));
-
-syntax_error(Meta, File, Message) when is_binary(Message) ->
-  raise(Meta, File, 'Elixir.SyntaxError', Message).
-
-syntax_error(Meta, File, Format, Args)  ->
-  Message = io_lib:format(Format, Args),
-  raise(Meta, File, 'Elixir.SyntaxError', elixir_utils:characters_to_binary(Message)).
+-spec compile_error(list(), binary(), iolist()) -> no_return().
+-spec compile_error(list(), binary(), iolist(), list()) -> no_return().
 
 compile_error(Meta, File, Message) when is_list(Message) ->
-  compile_error(Meta, File, iolist_to_binary(Message));
-
-compile_error(Meta, File, Message) when is_binary(Message) ->
-  raise(Meta, File, 'Elixir.CompileError', Message).
-
-compile_error(Meta, File, Format, Args)  ->
-  Message = io_lib:format(Format, Args),
   raise(Meta, File, 'Elixir.CompileError', elixir_utils:characters_to_binary(Message)).
 
+compile_error(Meta, File, Format, Args) when is_list(Format)  ->
+  compile_error(Meta, File, io_lib:format(Format, Args)).
+
 %% Raised on tokenizing/parsing
 
--spec parse_error(non_neg_integer() | list(), file:filename(), iolist() | atom(), [] | iolist()) -> no_return().
+-spec parse_error(non_neg_integer() | list(), file:filename_all(), iolist() | atom(), [] | iolist()) -> no_return().
 
 parse_error(Meta, File, Error, []) ->
   Message = case Error of
@@ -73,14 +60,6 @@ parse_error(Meta, File, Error, Token) ->
   Message = <<BinError / binary, BinToken / binary >>,
   raise(Meta, File, 'Elixir.SyntaxError', Message).
 
-%% Raised during compilation
-
--spec form_error(non_neg_integer() | list(), file:filename(), module(), any()) -> no_return().
-
-form_error(Meta, File, Module, Desc) ->
-  Message = iolist_to_binary(format_error(Module, Desc)),
-  raise(Meta, File, 'Elixir.CompileError', Message).
-
 %% Shows a deprecation message
 
 deprecation(Meta, File, Message) -> deprecation(Meta, File, Message, []).
@@ -126,14 +105,14 @@ handle_file_warning(_, File, {Line,erl_lint,{undefined_behaviour_func,{Fun,Arity
 
   Kind    = protocol_or_behaviour(Module),
   Raw     = "undefined ~ts ~ts ~ts/~B (for ~ts ~ts)",
-  Message = io_lib:format(Raw, [Kind, DefKind, Def, DefArity, Kind, inspect(Module)]),
+  Message = io_lib:format(Raw, [Kind, DefKind, Def, DefArity, Kind, elixir_aliases:inspect(Module)]),
   warn(file_format(Line, File, Message));
 
 handle_file_warning(_, File, {Line,erl_lint,{undefined_behaviour,Module}}) ->
   case elixir_compiler:get_opt(internal) of
     true  -> [];
     false ->
-      Message = io_lib:format("behaviour ~ts undefined", [inspect(Module)]),
+      Message = io_lib:format("behaviour ~ts undefined", [elixir_aliases:inspect(Module)]),
       warn(file_format(Line, File, Message))
   end;
 
@@ -158,7 +137,7 @@ handle_file_warning(_, File, {Line,Module,Desc}) ->
 handle_file_warning(File, Desc) ->
   handle_file_warning(false, File, Desc).
 
--spec handle_file_error(file:filename(), {non_neg_integer(), module(), any()}) -> no_return().
+-spec handle_file_error(file:filename_all(), {non_neg_integer(), module(), any()}) -> no_return().
 
 handle_file_error(File, {Line,erl_lint,{unsafe_var,Var,{In,_Where}}}) ->
   Translated = case In of
@@ -169,35 +148,13 @@ handle_file_error(File, {Line,erl_lint,{unsafe_var,Var,{In,_Where}}}) ->
   Message = io_lib:format("cannot define variable ~ts inside ~ts", [format_var(Var), Translated]),
   raise(Line, File, 'Elixir.CompileError', iolist_to_binary(Message));
 
+handle_file_error(File, {Line,erl_lint,{spec_fun_undefined,{M,F,A}}}) ->
+  Message = io_lib:format("spec for undefined function ~ts.~ts/~B", [elixir_aliases:inspect(M), F, A]),
+  raise(Line, File, 'Elixir.CompileError', iolist_to_binary(Message));
+
 handle_file_error(File, {Line,Module,Desc}) ->
   form_error(Line, File, Module, Desc).
 
-%% Assertions
-
-assert_no_function_scope(_Meta, _Kind, #elixir_scope{function=nil}) -> [];
-assert_no_function_scope(Meta, Kind, S) ->
-  syntax_error(Meta, S#elixir_scope.file, "cannot invoke ~ts inside a function", [Kind]).
-
-assert_no_match_or_guard_scope(Meta, Kind, S) ->
-  assert_no_match_scope(Meta, Kind, S),
-  assert_no_guard_scope(Meta, Kind, S).
-
-assert_no_match_scope(Meta, Kind, #elixir_scope{context=match} = S) ->
-  syntax_error(Meta, S#elixir_scope.file, "cannot invoke ~ts inside match clause", [Kind]);
-assert_no_match_scope(_Meta, _Kind, _S) -> [].
-
-assert_no_guard_scope(Meta, Kind, #elixir_scope{context=guard} = S) ->
-  syntax_error(Meta, S#elixir_scope.file, "cannot invoke ~ts inside guard", [Kind]);
-assert_no_guard_scope(_Meta, _Kind, _S) -> [].
-
-assert_module_scope(Meta, Kind, #elixir_scope{module=nil,file=File}) ->
-  syntax_error(Meta, File, "cannot invoke ~ts outside module", [Kind]);
-assert_module_scope(_Meta, _Kind, #elixir_scope{module=Module}) -> Module.
-
-assert_function_scope(Meta, Kind, #elixir_scope{function=nil,file=File}) ->
-  syntax_error(Meta, File, "cannot invoke ~ts outside function", [Kind]);
-assert_function_scope(_Meta, _Kind, #elixir_scope{function=Function}) -> Function.
-
 %% Helpers
 
 raise(Meta, File, Kind, Message) when is_list(Meta) ->
@@ -206,7 +163,7 @@ raise(Meta, File, Kind, Message) when is_list(Meta) ->
 raise(none, File, Kind, Message) ->
   raise(0, File, Kind, Message);
 
-raise(Line, File, Kind, Message) when is_integer(Line) ->
+raise(Line, File, Kind, Message) when is_integer(Line), is_binary(File) ->
   %% Populate the stacktrace so we can raise it
   try
     throw(ok)
@@ -214,13 +171,13 @@ raise(Line, File, Kind, Message) when is_integer(Line) ->
     ok -> ok
   end,
   Stacktrace = erlang:get_stacktrace(),
-  Exception = Kind:new([{description, Message}, {file, iolist_to_binary(File)}, {line, Line}]),
+  Exception = Kind:new([{description, Message}, {file, File}, {line, Line}]),
   erlang:raise(error, Exception, tl(Stacktrace)).
 
-file_format(0, File, Message) ->
+file_format(0, File, Message) when is_binary(File) ->
   io_lib:format("~ts: ~ts~n", [elixir_utils:relative_to_cwd(File), Message]);
 
-file_format(Line, File, Message) ->
+file_format(Line, File, Message) when is_binary(File) ->
   io_lib:format("~ts:~w: ~ts~n", [elixir_utils:relative_to_cwd(File), Line, Message]).
 
 format_var(Var) ->

lib/elixir/src/elixir_exp.erl

@@ -0,0 +1,562 @@
+-module(elixir_exp).
+-export([expand/2, expand_many/2, expand_args/2, expand_arg/2]).
+-import(elixir_errors, [compile_error/3, compile_error/4]).
+-include("elixir.hrl").
+
+%% =
+
+expand({ '=', Meta, [Left, Right] }, E) ->
+  assert_no_guard_scope(Meta, '=', E),
+  { ERight, ER } = expand(Right, E),
+  { ELeft, EL }  = elixir_exp_clauses:match(fun expand/2, Left, E),
+  { { '=', Meta, [ELeft, ERight] }, elixir_env:mergev(EL, ER) };
+
+%% Literal operators
+
+expand({ '{}', Meta, Args }, E) ->
+  { EArgs, EA } = expand_args(Args, E),
+  { { '{}', Meta, EArgs }, EA };
+
+expand({ '<<>>', Meta, Args }, E) ->
+  elixir_bitstring:expand(Meta, Args, E);
+
+%% Other operators
+
+expand({ '__op__', Meta, [_, _] = Args }, E) ->
+  { EArgs, EA } = expand_args(Args, E),
+  { { '__op__', Meta, EArgs }, EA };
+
+expand({ '__op__', Meta, [_, _, _] = Args }, E) ->
+  { EArgs, EA } = expand_args(Args, E),
+  { { '__op__', Meta, EArgs }, EA };
+
+expand({ '->', Meta, _Args }, E) ->
+  compile_error(Meta, E#elixir_env.file, "unhandled operator ->");
+
+%% __block__
+
+expand({ '__block__', _Meta, [] }, E) ->
+  { nil, E };
+expand({ '__block__', _Meta, [Arg] }, E) ->
+  expand(Arg, E);
+expand({ '__block__', Meta, Args }, E) when is_list(Args) ->
+  { EArgs, EA } = expand_many(Args, E),
+  { { '__block__', Meta, EArgs }, EA };
+
+%% __aliases__
+
+expand({ '__aliases__', _, _ } = Alias, E) ->
+  case elixir_aliases:expand(Alias, E#elixir_env.aliases,
+                             E#elixir_env.macro_aliases, E#elixir_env.lexical_tracker) of
+    Receiver when is_atom(Receiver) ->
+      elixir_lexical:record_remote(Receiver, E#elixir_env.lexical_tracker),
+      { Receiver, E };
+    Aliases ->
+      { EAliases, EA } = expand_args(Aliases, E),
+
+      case lists:all(fun is_atom/1, EAliases) of
+        true ->
+          Receiver = elixir_aliases:concat(EAliases),
+          elixir_lexical:record_remote(Receiver, E#elixir_env.lexical_tracker),
+          { Receiver, EA };
+        false ->
+          { { { '.', [], [elixir_aliases, concat] }, [], [EAliases] }, EA }
+      end
+  end;
+
+%% alias
+
+expand({ alias, Meta, [Ref] }, E) ->
+  expand({ alias, Meta, [Ref,[]] }, E);
+expand({ alias, Meta, [Ref, KV] }, E) ->
+  assert_no_match_or_guard_scope(Meta, alias, E),
+  { ERef, ER } = expand(Ref, E),
+  { EKV, ET }  = expand_opts(Meta, alias, [as, warn], no_alias_opts(KV), ER),
+
+  if
+    is_atom(ERef) ->
+      { { alias, Meta, [ERef, EKV] },
+        expand_alias(Meta, true, ERef, EKV, ET) };
+    true ->
+      compile_error(Meta, E#elixir_env.file,
+        "invalid arguments for alias, expected a compile time atom or alias as argument")
+  end;
+
+expand({ require, Meta, [Ref] }, E) ->
+  expand({ require, Meta, [Ref, []] }, E);
+expand({ require, Meta, [Ref, KV] }, E) ->
+  assert_no_match_or_guard_scope(Meta, require, E),
+
+  { ERef, ER } = expand(Ref, E),
+  { EKV, ET }  = expand_opts(Meta, require, [as, warn], no_alias_opts(KV), ER),
+
+  if
+    is_atom(ERef) ->
+      elixir_aliases:ensure_loaded(Meta, ERef, ET),
+      { { require, Meta, [ERef, EKV] },
+        expand_require(Meta, ERef, EKV, ET) };
+    true ->
+      compile_error(Meta, E#elixir_env.file,
+        "invalid arguments for require, expected a compile time atom or alias as argument")
+  end;
+
+expand({ import, Meta, [Left] }, E) ->
+  expand({ import, Meta, [Left, []]}, E);
+
+expand({ import, Meta, [Ref, KV] }, E) ->
+  assert_no_match_or_guard_scope(Meta, import, E),
+  { ERef, ER } = expand(Ref, E),
+  { EKV, ET }  = expand_opts(Meta, import, [only, except, warn], KV, ER),
+
+  if
+    is_atom(ERef) ->
+      elixir_aliases:ensure_loaded(Meta, ERef, ET),
+      { Functions, Macros } = elixir_import:import(Meta, ERef, EKV, ET),
+      { { import, Meta, [ERef, EKV] },
+        expand_require(Meta, ERef, EKV, ET#elixir_env{functions=Functions, macros=Macros}) };
+    true ->
+      compile_error(Meta, E#elixir_env.file, "invalid name for import, expected a compile time atom or alias")
+  end;
+
+%% Pseudo vars
+
+expand({ '__MODULE__', _, Atom }, E) when is_atom(Atom) ->
+  { E#elixir_env.module, E };
+expand({ '__FILE__', Meta, Atom }, E) when is_atom(Atom) ->
+  elixir_errors:deprecation(Meta, E#elixir_env.file, "__FILE__ is deprecated, please use __DIR__ or __ENV__.file instead"),
+  { E#elixir_env.file, E };
+expand({ '__DIR__', _, Atom }, E) when is_atom(Atom) ->
+  { filename:dirname(E#elixir_env.file), E };
+expand({ '__CALLER__', _, Atom } = Caller, E) when is_atom(Atom) ->
+  { Caller, E };
+expand({ '__ENV__', Meta, Atom }, E) when is_atom(Atom) ->
+  Env = elixir_env:env_to_ex({ ?line(Meta), E }),
+  { { '{}', [], tuple_to_list(Env) }, E };
+expand({ { '.', DotMeta, [{ '__ENV__', Meta, Atom }, Field] }, CallMeta, [] }, E) when is_atom(Atom), is_atom(Field) ->
+  Env = elixir_env:env_to_ex({ ?line(Meta), E }),
+  case erlang:function_exported('Elixir.Macro.Env', Field, 1) of
+    true  -> { Env:Field(), E };
+    false -> { { { '.', DotMeta, [{ '{}', [], tuple_to_list(Env) }, Field] }, CallMeta, [] }, E }
+  end;
+
+%% Quote
+
+expand({ Unquote, Meta, [_] }, E) when Unquote == unquote; Unquote == unquote_splicing ->
+  compile_error(Meta, E#elixir_env.file, "~p called outside quote", [Unquote]);
+
+expand({ quote, Meta, [Opts] }, E) when is_list(Opts) ->
+  case lists:keyfind(do, 1, Opts) of
+    { do, Do } ->
+      expand({ quote, Meta, [lists:keydelete(do, 1, Opts), [{do,Do}]] }, E);
+    false ->
+      compile_error(Meta, E#elixir_env.file, "missing do keyword in quote")
+  end;
+
+expand({ quote, Meta, [_] }, E) ->
+  compile_error(Meta, E#elixir_env.file, "invalid arguments for quote");
+
+expand({ quote, Meta, [KV, Do] }, E) when is_list(Do) ->
+  Exprs =
+    case lists:keyfind(do, 1, Do) of
+      { do, Expr } -> Expr;
+      false -> compile_error(Meta, E#elixir_scope.file, "missing do keyword in quote")
+    end,
+
+  ValidOpts   = [hygiene, context, var_context, location, line, unquote, bind_quoted],
+  { EKV, ET } = expand_opts(Meta, quote, ValidOpts, KV, E),
+
+  Hygiene = case lists:keyfind(hygiene, 1, EKV) of
+    { hygiene, List } when is_list(List) ->
+      List;
+    false ->
+      []
+  end,
+
+  Context = case lists:keyfind(context, 1, EKV) of
+    { context, Atom } when is_atom(Atom) ->
+      Atom;
+    { context, _ } ->
+      compile_error(Meta, E#elixir_env.file, "invalid :context for quote, expected a compile time atom or an alias");
+    false ->
+      case E#elixir_env.module of
+        nil -> 'Elixir';
+        Mod -> Mod
+      end
+  end,
+
+  Vars    = lists:keyfind(vars, 1, Hygiene) /= { vars, false },
+  Aliases = lists:keyfind(aliases, 1, Hygiene) /= { aliases, false },
+  Imports = lists:keyfind(imports, 1, Hygiene) /= { imports, false },
+
+  Keep = lists:keyfind(location, 1, EKV) == { location, keep },
+  Line = proplists:get_value(line, EKV, false),
+
+  { Binding, DefaultUnquote } = case lists:keyfind(bind_quoted, 1, EKV) of
+    { bind_quoted, BQ } -> { BQ, false };
+    false -> { nil, true }
+  end,
+
+  Unquote = case lists:keyfind(unquote, 1, EKV) of
+    { unquote, Bool } when is_boolean(Bool) -> Bool;
+    false -> DefaultUnquote
+  end,
+
+  Q = #elixir_quote{vars_hygiene=Vars, line=Line, keep=Keep, unquote=Unquote,
+        aliases_hygiene=Aliases, imports_hygiene=Imports, context=Context},
+
+  { Quoted, _Q } = elixir_quote:quote(Exprs, Binding, Q, ET),
+  expand(Quoted, ET);
+
+expand({ quote, Meta, [_, _] }, E) ->
+  compile_error(Meta, E#elixir_env.file, "invalid arguments for quote");
+
+%% Functions
+
+expand({ '&', _, [Arg] } = Original, E) when is_integer(Arg) ->
+  { Original, E };
+expand({ '&', Meta, [Arg] }, E) ->
+  assert_no_match_or_guard_scope(Meta, '&', E),
+  case elixir_fn:capture(Meta, Arg, E) of
+    { local, Fun, Arity } ->
+      { { '&', Meta, [{ '/', [], [{ Fun, [], nil }, Arity] }] }, E };
+    { expanded, Expr, EE } ->
+      expand(Expr, EE)
+  end;
+
+expand({ fn, Meta, Pairs }, E) ->
+  assert_no_match_or_guard_scope(Meta, fn, E),
+  elixir_fn:expand(Meta, Pairs, E);
+
+%% Case/Receive/Try
+
+expand({'case', Meta, [Expr, KV]}, E) ->
+  assert_no_match_or_guard_scope(Meta, 'case', E),
+  { EExpr, EE } = expand(Expr, E),
+  { EClauses, EC } = elixir_exp_clauses:'case'(Meta, KV, EE),
+  { { 'case', Meta, [EExpr, EClauses] }, EC };
+
+expand({'receive', Meta, [KV]}, E) ->
+  assert_no_match_or_guard_scope(Meta, 'receive', E),
+  { EClauses, EC } = elixir_exp_clauses:'receive'(Meta, KV, E),
+  { { 'receive', Meta, [EClauses] }, EC };
+
+expand({'try', Meta, [KV]}, E) ->
+  assert_no_match_or_guard_scope(Meta, 'try', E),
+  { EClauses, EC } = elixir_exp_clauses:'try'(Meta, KV, E),
+  { { 'try', Meta, [EClauses] }, EC };
+
+%% Comprehensions
+
+expand({ Kind, Meta, Args }, E) when is_list(Args), (Kind == lc) orelse (Kind == bc) ->
+  expand_comprehension(Meta, Kind, Args, E);
+
+%% Super
+
+expand({ super, Meta, Args }, E) when is_list(Args) ->
+  assert_no_match_or_guard_scope(Meta, super, E),
+  { EArgs, EA } = expand_args(Args, E),
+  { { super, Meta, EArgs }, EA };
+
+%% Vars
+
+expand({ '^', Meta, [Arg] }, #elixir_env{context=match} = E) ->
+  case expand(Arg, E) of
+    { { Name, _, Kind } = EArg, EA } when is_atom(Name), is_atom(Kind) ->
+      { { '^', Meta, [EArg] }, EA };
+    _ ->
+    Msg = "invalid argument for unary operator ^, expected an existing variable, got: ^~ts",
+    compile_error(Meta, E#elixir_env.file, Msg, ['Elixir.Macro':to_string(Arg)])
+  end;
+expand({ '^', Meta, [Arg] }, E) ->
+  compile_error(Meta, E#elixir_env.file,
+    "cannot use ^~ts outside of match clauses", ['Elixir.Macro':to_string(Arg)]);
+
+expand({ '_', _, Kind } = Var, E) when is_atom(Kind) ->
+  { Var, E };
+expand({ Name, Meta, Kind } = Var, #elixir_env{context=match, export_vars=Export} = E) when is_atom(Name), is_atom(Kind) ->
+  Pair      = { Name, var_kind(Meta, Kind) },
+  NewVars   = ordsets:add_element(Pair, E#elixir_env.vars),
+  NewExport = case (Export /= nil) andalso (lists:keyfind(export, 1, Meta) /= { export, false }) of
+    true  -> ordsets:add_element(Pair, Export);
+    false -> Export
+  end,
+  { Var, E#elixir_env{vars=NewVars, export_vars=NewExport} };
+expand({ Name, Meta, Kind } = Var, #elixir_env{vars=Vars} = E) when is_atom(Name), is_atom(Kind) ->
+  case lists:member({ Name, var_kind(Meta, Kind) }, Vars) of
+    true ->
+      { Var, E };
+    false ->
+      VarMeta = lists:keyfind(var, 1, Meta),
+      if
+        VarMeta == { var, true } ->
+          Extra = case Kind of
+            nil -> "";
+            _   -> io_lib:format(" (context ~ts)", [elixir_aliases:inspect(Kind)])
+          end,
+
+          compile_error(Meta, E#elixir_env.file, "expected var ~ts~ts to expand to an existing "
+                        "variable or be a part of a match", [Name, Extra]);
+        E#elixir_env.context == guard ->
+          compile_error(Meta, E#elixir_env.file, "unknown variable ~ts or cannot invoke "
+                        "function ~ts/0 inside guard", [Name, Name]);
+        true ->
+          expand({ Name, Meta, [] }, E)
+      end
+  end;
+
+%% Local calls
+
+expand({ Atom, Meta, Args }, E) when is_atom(Atom), is_list(Meta), is_list(Args) ->
+  assert_no_ambiguous_op(Atom, Meta, Args, E),
+
+  elixir_dispatch:dispatch_import(Meta, Atom, Args, E, fun() ->
+    expand_local(Meta, Atom, Args, E)
+  end);
+
+%% Remote calls
+
+expand({ { '.', DotMeta, [Left, Right] }, Meta, Args }, E)
+    when (is_tuple(Left) orelse is_atom(Left)), is_atom(Right), is_list(Meta), is_list(Args) ->
+  { ELeft, EL } = expand(Left, E),
+
+  elixir_dispatch:dispatch_require(Meta, ELeft, Right, Args, EL, fun(Receiver) ->
+    expand_remote(Receiver, DotMeta, Right, Meta, Args, E, EL)
+  end);
+
+%% Anonymous calls
+
+expand({ { '.', DotMeta, [Expr] }, Meta, Args }, E) when is_list(Args) ->
+  { EExpr, EE } = expand(Expr, E),
+  if
+    is_atom(EExpr) ->
+      compile_error(Meta, E#elixir_env.file, "invalid function call :~ts.()", [EExpr]);
+    true ->
+      { EArgs, EA } = expand_args(Args, elixir_env:mergea(E, EE)),
+      { { { '.', DotMeta, [EExpr] }, Meta, EArgs }, elixir_env:mergev(EE, EA) }
+  end;
+
+%% Invalid calls
+
+expand({ _, Meta, Args } = Invalid, E) when is_list(Meta) and is_list(Args) ->
+  compile_error(Meta, E#elixir_env.file, "invalid call ~ts",
+    ['Elixir.Macro':to_string(Invalid)]);
+
+expand({ _, _, _ } = Tuple, E) ->
+  compile_error([{line,0}], E#elixir_env.file, "invalid quoted expression: ~ts",
+    ['Elixir.Kernel':inspect(Tuple, [{records,false}])]);
+
+%% Literals
+
+expand({ Left, Right }, E) ->
+  { [ELeft, ERight], EE } = expand_args([Left, Right], E),
+  { { ELeft, ERight }, EE };
+
+expand(List, #elixir_env{context=match} = E) when is_list(List) ->
+  expand_list(List, fun expand/2, E, []);
+
+expand(List, E) when is_list(List) ->
+  { EArgs, { EC, EV } } = expand_list(List, fun expand_arg/2, {E, E}, []),
+  { EArgs, elixir_env:mergea(EV, EC) };
+
+expand(Function, E) when is_function(Function) ->
+  case (erlang:fun_info(Function, type) == { type, external }) andalso
+       (erlang:fun_info(Function, env) == { env, [] }) of
+    true ->
+      { Function, E };
+    false ->
+      compile_error([{line,0}], E#elixir_env.file,
+        "invalid quoted expression: ~ts", ['Elixir.Kernel':inspect(Function)])
+  end;
+
+expand(Other, E) when is_number(Other); is_atom(Other); is_binary(Other); is_pid(Other) ->
+  { Other, E };
+
+expand(Other, E) ->
+  compile_error([{line,0}], E#elixir_env.file,
+    "invalid quoted expression: ~ts", ['Elixir.Kernel':inspect(Other)]).
+
+%% Helpers
+
+expand_list([{ '|', Meta, [_, _] = Args }], Fun, Acc, List) ->
+  { EArgs, EAcc } = lists:mapfoldl(Fun, Acc, Args),
+  expand_list([], Fun, EAcc, [{ '|', Meta, EArgs }|List]);
+expand_list([H|T], Fun, Acc, List) ->
+  { EArg, EAcc } = Fun(H, Acc),
+  expand_list(T, Fun, EAcc, [EArg|List]);
+expand_list([], _Fun, Acc, List) ->
+  { lists:reverse(List), Acc }.
+
+expand_many(Args, E) ->
+  lists:mapfoldl(fun expand/2, E, Args).
+
+%% Variables in arguments are not propagated from one
+%% argument to the other. For instance:
+%%
+%%   x = 1
+%%   foo(x = x + 2, x)
+%%   x
+%%
+%% Should be the same as:
+%%
+%%   foo(3, 1)
+%%   3
+%%
+%% However, lexical information is.
+expand_arg(Arg, Acc) when is_number(Arg); is_atom(Arg); is_binary(Arg); is_pid(Arg); is_function(Arg) ->
+  { Arg, Acc };
+expand_arg(Arg, { Acc1, Acc2 }) ->
+  { EArg, EAcc } = expand(Arg, Acc1),
+  { EArg, { elixir_env:mergea(Acc1, EAcc), elixir_env:mergev(Acc2, EAcc) } }.
+
+expand_args(Args, #elixir_env{context=match} = E) ->
+  lists:mapfoldl(fun expand/2, E, Args);
+expand_args(Args, E) ->
+  { EArgs, { EC, EV } } = lists:mapfoldl(fun expand_arg/2, {E, E}, Args),
+  { EArgs, elixir_env:mergea(EV, EC) }.
+
+%% Match/var helpers
+
+var_kind(Meta, Kind) ->
+  case lists:keyfind(counter, 1, Meta) of
+    { counter, Counter } -> Counter;
+    false -> Kind
+  end.
+
+%% Locals
+
+assert_no_ambiguous_op(Name, Meta, [Arg], E) ->
+  case lists:keyfind(ambiguous_op, 1, Meta) of
+    { ambiguous_op, Kind } ->
+      case lists:member({ Name, Kind }, E#elixir_env.vars) of
+        true ->
+          compile_error(Meta, E#elixir_env.file, "\"~ts ~ts\" looks like a function call but "
+                        "there is a variable named \"~ts\", please use explicit parenthesis or even spaces",
+                        [Name, 'Elixir.Macro':to_string(Arg), Name]);
+        false ->
+          ok
+      end;
+    _ ->
+      ok
+  end;
+assert_no_ambiguous_op(_Atom, _Meta, _Args, _E) ->
+  ok.
+
+expand_local(Meta, Name, Args, #elixir_env{local=nil, function=nil} = E) ->
+  { EArgs, EA } = expand_args(Args, E),
+  { { Name, Meta, EArgs }, EA };
+expand_local(Meta, Name, Args, #elixir_env{local=nil, module=Module, function=Function} = E) ->
+  elixir_locals:record_local({ Name, length(Args) }, Module, Function),
+  { EArgs, EA } = expand_args(Args, E),
+  { { Name, Meta, EArgs }, EA };
+expand_local(Meta, Name, Args, E) ->
+  expand({ { '.', Meta, [E#elixir_env.local, Name] }, Meta, Args }, E).
+
+%% Remote
+
+expand_remote(Receiver, DotMeta, Right, Meta, Args, E, EL) ->
+  if
+    is_atom(Receiver) -> elixir_lexical:record_remote(Receiver, E#elixir_env.lexical_tracker);
+    true -> ok
+  end,
+  { EArgs, EA } = expand_args(Args, E),
+  { { { '.', DotMeta, [Receiver, Right] }, Meta, EArgs }, elixir_env:mergev(EL, EA) }.
+
+%% Lexical helpers
+
+expand_opts(Meta, Kind, Allowed, Opts, E) ->
+  { EOpts, EE } = expand(Opts, E),
+  validate_opts(Meta, Kind, Allowed, EOpts, EE),
+  { EOpts, EE }.
+
+validate_opts(Meta, Kind, Allowed, Opts, E) when is_list(Opts) ->
+  [begin
+    compile_error(Meta, E#elixir_env.file,
+                  "unsupported option ~ts given to ~s", ['Elixir.Kernel':inspect(Key), Kind])
+  end || { Key, _ } <- Opts, not lists:member(Key, Allowed)];
+
+validate_opts(Meta, Kind, _Allowed, _Opts, S) ->
+  compile_error(Meta, S#elixir_scope.file, "invalid options for ~s, expected a keyword list", [Kind]).
+
+no_alias_opts(KV) when is_list(KV) ->
+  case lists:keyfind(as, 1, KV) of
+    { as, As } -> lists:keystore(as, 1, KV, { as, no_alias_expansion(As) });
+    false -> KV
+  end;
+no_alias_opts(KV) -> KV.
+
+no_alias_expansion({ '__aliases__', Meta, [H|T] }) when (H /= 'Elixir') and is_atom(H) ->
+  { '__aliases__', Meta, ['Elixir',H|T] };
+no_alias_expansion(Other) ->
+  Other.
+
+expand_require(Meta, Ref, KV, E) ->
+  RE = E#elixir_env{requires=ordsets:add_element(Ref, E#elixir_env.requires)},
+  expand_alias(Meta, false, Ref, KV, RE).
+
+expand_alias(Meta, IncludeByDefault, Ref, KV, #elixir_env{context_modules=Context} = E) ->
+  New = expand_as(lists:keyfind(as, 1, KV), Meta, IncludeByDefault, Ref, E),
+
+  %% Add the alias to context_modules if defined is true.
+  %% This is used by defmodule in order to store the defined
+  %% module in context modules.
+  NewContext =
+    case lists:keyfind(defined, 1, Meta) of
+      { defined, Mod } when is_atom(Mod) -> [Mod|Context];
+      false -> Context
+    end,
+
+  { Aliases, MacroAliases } = elixir_aliases:store(Meta, New, Ref, KV, E#elixir_env.aliases,
+                                E#elixir_env.macro_aliases, E#elixir_env.lexical_tracker),
+
+  E#elixir_env{aliases=Aliases, macro_aliases=MacroAliases, context_modules=NewContext}.
+
+expand_as({ as, true }, _Meta, _IncludeByDefault, Ref, _E) ->
+  elixir_aliases:last(Ref);
+expand_as({ as, false }, _Meta, _IncludeByDefault, Ref, _E) ->
+  Ref;
+expand_as({ as, Atom }, Meta, _IncludeByDefault, _Ref, E) when is_atom(Atom) ->
+  case length(string:tokens(atom_to_list(Atom), ".")) of
+    1 -> compile_error(Meta, E#elixir_env.file,
+           "invalid value for keyword :as, expected an alias, got atom: ~ts", [elixir_aliases:inspect(Atom)]);
+    2 -> Atom;
+    _ -> compile_error(Meta, E#elixir_env.file,
+           "invalid value for keyword :as, expected an alias, got nested alias: ~ts", [elixir_aliases:inspect(Atom)])
+  end;
+expand_as(false, _Meta, IncludeByDefault, Ref, _E) ->
+  if IncludeByDefault -> elixir_aliases:last(Ref);
+     true -> Ref
+  end;
+expand_as({ as, Other }, Meta, _IncludeByDefault, _Ref, E) ->
+  compile_error(Meta, E#elixir_env.file,
+    "invalid value for keyword :as, expected an alias, got: ~ts", ['Elixir.Macro':to_string(Other)]).
+
+%% Comprehensions
+
+expand_comprehension(Meta, Kind, Args, E) ->
+  case elixir_utils:split_last(Args) of
+    { Cases, [{do,Expr}] } ->
+      { ECases, EC } = lists:mapfoldl(fun expand_comprehension_clause/2, E, Cases),
+      { EExpr, _ }   = expand(Expr, EC),
+      { { Kind, Meta, ECases ++ [[{do,EExpr}]] }, E };
+    _ ->
+      compile_error(Meta, E#elixir_env.file, "missing do keyword in comprehension ~ts", [Kind])
+  end.
+
+expand_comprehension_clause({Gen, Meta, [Left, Right]}, E) when Gen == inbits; Gen == inlist ->
+  { ERight, ER } = expand(Right, E),
+  { ELeft, EL }  = elixir_exp_clauses:match(fun expand/2, Left, E),
+  { { Gen, Meta, [ELeft, ERight] }, elixir_env:mergev(EL, ER) };
+expand_comprehension_clause(X, E) ->
+  expand(X, E).
+
+%% Assertions
+
+assert_no_match_or_guard_scope(Meta, Kind, E) ->
+  assert_no_match_scope(Meta, Kind, E),
+  assert_no_guard_scope(Meta, Kind, E).
+assert_no_match_scope(Meta, _Kind, #elixir_env{context=match,file=File}) ->
+  compile_error(Meta, File, "invalid expression in match");
+assert_no_match_scope(_Meta, _Kind, _E) -> [].
+assert_no_guard_scope(Meta, _Kind, #elixir_env{context=guard,file=File}) ->
+  compile_error(Meta, File, "invalid expression in guard");
+assert_no_guard_scope(_Meta, _Kind, _E) -> [].

lib/elixir/src/elixir_exp_clauses.erl

@@ -0,0 +1,200 @@
+%% Handle code related to args, guard and -> matching for case,
+%% fn, receive and friends. try is handled in elixir_try.
+-module(elixir_exp_clauses).
+-export([match/3, clause/5, 'case'/3, 'receive'/3, 'try'/3, def/5]).
+-import(elixir_errors, [compile_error/3, compile_error/4]).
+-include("elixir.hrl").
+
+match(Fun, Expr, #elixir_env{context=Context} = E) ->
+  { EExpr, EE } = Fun(Expr, E#elixir_env{context=match}),
+  { EExpr, EE#elixir_env{context=Context} }.
+
+def(Fun, Args, Guards, Body, E) ->
+  { EArgs, EA }   = match(Fun, Args, E),
+  { EGuards, EG } = guard(Guards, EA#elixir_env{context=guard}),
+  { EBody, EB }   = elixir_exp:expand(Body, EG#elixir_env{context=E#elixir_env.context}),
+  { EArgs, EGuards, EBody, EB }.
+
+clause(Meta, Kind, Fun, { '->', ClauseMeta, [_, _] } = Clause, E) when is_function(Fun, 3) ->
+  clause(Meta, Kind, fun(X, Acc) -> Fun(ClauseMeta, X, Acc) end, Clause, E);
+clause(_Meta, _Kind, Fun, { '->', Meta, [Left, Right] }, E) ->
+  { ELeft, EL }  = head(Fun, Left, E),
+  { ERight, ER } = elixir_exp:expand(Right, EL),
+  { { '->', Meta, [ELeft, ERight] }, ER };
+clause(Meta, Kind, _Fun, _, E) ->
+  compile_error(Meta, E#elixir_env.file, "expected -> clauses in ~ts", [Kind]).
+
+head(Fun, [{ 'when', Meta, [_,_|_] = All }], E) ->
+  { Args, Guard } = elixir_utils:split_last(All),
+  { EArgs, EA }   = match(Fun, Args, E),
+  { EGuard, EG }  = guard(Guard, EA#elixir_env{context=guard}),
+  { [{ 'when', Meta, EArgs ++ [EGuard] }], EG#elixir_env{context=E#elixir_env.context} };
+head(Fun, Args, E) ->
+  match(Fun, Args, E).
+
+guard({ 'when', Meta, [Left, Right] }, E) ->
+  { ELeft, EL }  = guard(Left, E),
+  { ERight, ER } = guard(Right, EL),
+  { { 'when', Meta, [ELeft, ERight] }, ER };
+guard(Other, E) ->
+  elixir_exp:expand(Other, E).
+
+%% Case
+
+'case'(Meta, [], E) ->
+  compile_error(Meta, E#elixir_env.file, "missing do keyword in case");
+'case'(Meta, KV, E) when not is_list(KV) ->
+  compile_error(Meta, E#elixir_env.file, "invalid arguments for case");
+'case'(Meta, KV, E) ->
+  EE = E#elixir_env{export_vars=[]},
+  { EClauses, EVars } = lists:mapfoldl(fun(X, Acc) -> do_case(Meta, X, Acc, EE) end, [], KV),
+  { EClauses, elixir_env:mergev(EVars, E) }.
+
+do_case(Meta, { 'do', _ } = Do, Acc, E) ->
+  expand_with_export(Meta, 'case', expand_arg(Meta, 'case', 'do'), Do, Acc, E);
+do_case(Meta, { Key, _ }, _Acc, E) ->
+  compile_error(Meta, E#elixir_env.file, "unexpected keyword ~ts in case", [Key]).
+
+%% Receive
+
+'receive'(Meta, [], E) ->
+  compile_error(Meta, E#elixir_env.file, "missing do or after keyword in receive");
+'receive'(Meta, KV, E) when not is_list(KV) ->
+  compile_error(Meta, E#elixir_env.file, "invalid arguments for receive");
+'receive'(Meta, KV, E) ->
+  EE = E#elixir_env{export_vars=[]},
+  { EClauses, EVars } = lists:mapfoldl(fun(X, Acc) -> do_receive(Meta, X, Acc, EE) end, [], KV),
+  { EClauses, elixir_env:mergev(EVars, E) }.
+
+do_receive(_Meta, { 'do', nil } = Do, Acc, _E) ->
+  { Do, Acc };
+do_receive(Meta, { 'do', _ } = Do, Acc, E) ->
+  expand_with_export(Meta, 'receive', expand_arg(Meta, 'receive', 'do'), Do, Acc, E);
+do_receive(_Meta, { 'after', [{ '->', Meta, [[Left], Right] }] }, Acc, E) ->
+  { ELeft, EL }  = elixir_exp:expand(Left, E),
+  { ERight, ER } = elixir_exp:expand(Right, EL),
+  EClause = { 'after', [{ '->', Meta, [[ELeft], ERight] }] },
+  { EClause, elixir_env:merge_vars(Acc, ER#elixir_env.export_vars) };
+do_receive(Meta, { 'after', _ }, _Acc, E) ->
+  compile_error(Meta, E#elixir_env.file, "expected a single -> clause for after in receive");
+do_receive(Meta, { Key, _ }, _Acc, E) ->
+  compile_error(Meta, E#elixir_env.file, "unexpected keyword ~ts in receive", [Key]).
+
+%% Try
+
+'try'(Meta, [], E) ->
+  compile_error(Meta, E#elixir_env.file, "missing do keywords in try");
+'try'(Meta, KV, E) when not is_list(KV) ->
+  elixir_errors:compile_error(Meta, E#elixir_env.file, "invalid arguments for try");
+'try'(Meta, KV, E) ->
+  { lists:map(fun(X) -> do_try(Meta, X, E) end, KV), E }.
+
+do_try(_Meta, { 'do', Expr }, E) ->
+  { EExpr, _ } = elixir_exp:expand(Expr, E),
+  { 'do', EExpr };
+do_try(_Meta, { 'after', Expr }, E) ->
+  { EExpr, _ } = elixir_exp:expand(Expr, E),
+  { 'after', EExpr };
+do_try(Meta, { 'else', _ } = Else, E) ->
+  expand_without_export(Meta, 'try', expand_arg(Meta, 'try', 'else'), Else, E);
+do_try(Meta, { 'catch', _ } = Catch, E) ->
+  expand_without_export(Meta, 'try', fun elixir_exp:expand_many/2, Catch, E);
+do_try(Meta, { 'rescue', _ } = Rescue, E) ->
+  expand_without_export(Meta, 'try', fun expand_rescue/3, Rescue, E);
+do_try(Meta, { Key, _ }, E) ->
+  compile_error(Meta, E#elixir_env.file, "unexpected keyword ~ts in try", [Key]).
+
+expand_rescue(Meta, [Arg], E) ->
+  case expand_rescue(Arg, E) of
+    { EArg, EA } ->
+      { [EArg], EA };
+    false ->
+      compile_error(Meta, E#elixir_env.file, "invalid rescue clause. The clause should "
+        "match on an alias, a variable or be in the `var in [alias]` format")
+  end;
+expand_rescue(Meta, _, E) ->
+  compile_error(Meta, E#elixir_env.file, "expected one arg for rescue clauses (->) in try").
+
+%% rescue var => var in _
+expand_rescue({ Name, _, Atom } = Var, E) when is_atom(Name), is_atom(Atom) ->
+  expand_rescue({ in, [], [Var, { '_', [], E#elixir_env.module }] }, E);
+
+%% rescue var in [Exprs]
+expand_rescue({ in, Meta, [Left, Right] }, E) ->
+  { ERight, ER } = elixir_exp:expand(Right, E#elixir_env{context=nil}),
+  { ELeft, EL }  = elixir_exp:expand(Left, ER#elixir_env{context=match}),
+
+  case ELeft of
+    { Name, _, Atom } when is_atom(Name), is_atom(Atom) ->
+      case normalize_rescue(ERight) of
+        false -> false;
+        Other -> { { in, Meta, [ELeft, Other] }, EL }
+      end;
+    _ ->
+      false
+  end;
+
+%% rescue Error => _ in [Error]
+expand_rescue(Arg, E) ->
+  expand_rescue({ in, [], [{ '_', [], E#elixir_env.module }, Arg] }, E).
+
+normalize_rescue({ '_', _, Atom } = N) when is_atom(Atom) -> N;
+normalize_rescue(Atom) when is_atom(Atom) -> [Atom];
+normalize_rescue(Other) ->
+  is_list(Other)
+    andalso lists:all(fun is_var_or_atom/1, Other)
+    andalso Other.
+
+is_var_or_atom({ Name, _, Atom }) when is_atom(Name), is_atom(Atom) -> true;
+is_var_or_atom(Atom) when is_atom(Atom) -> true;
+is_var_or_atom(_) -> false.
+
+%% Expansion helpers
+
+export_vars({ Left, Meta, Right }) when is_atom(Left), is_list(Meta), is_atom(Right) ->
+  { Left, [{export,false}|Meta], Right };
+export_vars({ Left, Meta, Right }) ->
+  { export_vars(Left), Meta, export_vars(Right) };
+export_vars({ Left, Right }) ->
+  { export_vars(Left), export_vars(Right) };
+export_vars(List) when is_list(List) ->
+  [export_vars(X) || X <- List];
+export_vars(Other) ->
+  Other.
+
+%% Returns a function that expands arguments
+%% considering we have at maximum one entry.
+expand_arg(Meta, Kind, Key) ->
+  fun
+    ([Arg], E) ->
+      { EArg, EA } = elixir_exp:expand(Arg, E),
+      { [EArg], EA };
+    (_, E) ->
+      compile_error(Meta, E#elixir_env.file, "expected one arg for ~ts clauses (->) in ~ts", [Key, Kind])
+  end.
+
+%% Expands all -> pairs in a given key keeping the overall vars.
+expand_with_export(Meta, Kind, Fun, { Key, Clauses }, Acc, E) when is_list(Clauses) ->
+  EFun =
+    case lists:keyfind(export_all, 1, Meta) of
+      { export_all, true } -> Fun;
+      _ -> fun(ExportArgs, ExportE) -> Fun(export_vars(ExportArgs), ExportE) end
+    end,
+  Transformer = fun(Clause, Vars) ->
+    { EClause, EC } = clause(Meta, Kind, EFun, Clause, E),
+    { EClause, elixir_env:merge_vars(Vars, EC#elixir_env.export_vars) }
+  end,
+  { EClauses, EVars } = lists:mapfoldl(Transformer, Acc, Clauses),
+  { { Key, EClauses }, EVars };
+expand_with_export(Meta, Kind, _Fun, { Key, _ }, _Acc, E) ->
+  compile_error(Meta, E#elixir_env.file, "expected -> clauses for ~ts in ~ts", [Key, Kind]).
+
+%% Expands all -> pairs in a given key but do not keep the overall vars.
+expand_without_export(Meta, Kind, Fun, { Key, Clauses }, E) when is_list(Clauses) ->
+  Transformer = fun(Clause) ->
+    { EClause, _ } = clause(Meta, Kind, Fun, Clause, E),
+    EClause
+  end,
+  { Key, lists:map(Transformer, Clauses) };
+expand_without_export(Meta, Kind, _Fun, { Key, _ }, E) ->
+  compile_error(Meta, E#elixir_env.file, "expected -> clauses for ~ts in ~ts", [Key, Kind]).

lib/elixir/src/elixir_fn.erl

@@ -1,13 +1,13 @@
 -module(elixir_fn).
--export([fn/3, capture/3]).
--import(elixir_scope, [umergec/2]).
--import(elixir_errors, [syntax_error/3, syntax_error/4, compile_error/4]).
+-export([translate/3, capture/3, expand/3]).
+-import(elixir_errors, [compile_error/3, compile_error/4]).
 -include("elixir.hrl").
 
-fn(Meta, Clauses, S) ->
-  Transformer = fun({ ArgsWithGuards, CMeta, Expr }, Acc) ->
+translate(Meta, Clauses, S) ->
+  Transformer = fun({ '->', CMeta, [ArgsWithGuards, Expr] }, Acc) ->
     { Args, Guards } = elixir_clauses:extract_splat_guards(ArgsWithGuards),
-    elixir_clauses:assigns_block(?line(CMeta), fun translate_fn_match/2, Args, [Expr], Guards, umergec(S, Acc))
+    { TClause, TS  } = elixir_clauses:clause(?line(CMeta), fun translate_fn_match/2, Args, Expr, Guards, Acc),
+    { TClause, elixir_scope:mergef(S, TS) }
   end,
 
   { TClauses, NS } = lists:mapfoldl(Transformer, S, Clauses),
@@ -15,21 +15,32 @@ fn(Meta, Clauses, S) ->
 
   case length(lists:usort(Arities)) of
     1 ->
-      { { 'fun', ?line(Meta), { clauses, TClauses } }, umergec(S, NS) };
+      { { 'fun', ?line(Meta), { clauses, TClauses } }, NS };
     _ ->
-      syntax_error(Meta, S#elixir_scope.file,
-                   "cannot mix clauses with different arities in function definition")
+      compile_error(Meta, S#elixir_scope.file,
+                    "cannot mix clauses with different arities in function definition")
   end.
 
 translate_fn_match(Arg, S) ->
-  { TArg, TS } = elixir_translator:translate(Arg, S#elixir_scope{extra=fn_match}),
+  { TArg, TS } = elixir_translator:translate_many(Arg, S#elixir_scope{extra=fn_match}),
   { TArg, TS#elixir_scope{extra=S#elixir_scope.extra} }.
 
-capture(Meta, { '/', _, [{ { '.', _, [_, F] } = Dot, RequireMeta , [] }, A] }, S) when is_atom(F), is_integer(A) ->
-  Args = [{ '&', [], [X] } || X <- lists:seq(1, A)],
-  capture_require(Meta, { Dot, RequireMeta, Args }, S, true);
+%% Expansion
 
-capture(Meta, { '/', _, [{ F, _, C }, A] }, S) when is_atom(F), is_integer(A), is_atom(C) ->
+expand(Meta, Clauses, E) when is_list(Clauses) ->
+  Transformer = fun(Clause) ->
+    { EClause, _ } = elixir_exp_clauses:clause(Meta, fn, fun elixir_exp:expand_many/2, Clause, E),
+    EClause
+  end,
+  { { fn, Meta, lists:map(Transformer, Clauses) }, E }.
+
+%% Capture
+
+capture(Meta, { '/', _, [{ { '.', _, [_, F] } = Dot, RequireMeta , [] }, A] }, E) when is_atom(F), is_integer(A) ->
+  Args = [{ '&', [], [X] } || X <- lists:seq(1, A)],
+  capture_require(Meta, { Dot, RequireMeta, Args }, E, true);
+
+capture(Meta, { '/', _, [{ F, _, C }, A] }, E) when is_atom(F), is_integer(A), is_atom(C) ->
   ImportMeta =
     case lists:keyfind(import_fa, 1, Meta) of
       { import_fa, { Receiver, Context } } ->
@@ -40,112 +51,102 @@ capture(Meta, { '/', _, [{ F, _, C }, A] }, S) when is_atom(F), is_integer(A), i
       false -> Meta
     end,
   Args = [{ '&', [], [X] } || X <- lists:seq(1, A)],
-  capture_import(Meta, { F, ImportMeta, Args }, S, true);
+  capture_import(Meta, { F, ImportMeta, Args }, E, true);
 
-capture(Meta, { { '.', _, [_, Fun] }, _, Args } = Expr, S) when is_atom(Fun), is_list(Args) ->
-  capture_require(Meta, Expr, S, is_sequential_and_not_empty(Args));
+capture(Meta, { { '.', _, [_, Fun] }, _, Args } = Expr, E) when is_atom(Fun), is_list(Args) ->
+  capture_require(Meta, Expr, E, is_sequential_and_not_empty(Args));
 
-capture(Meta, { { '.', _, [_] }, _, Args } = Expr, S) when is_list(Args) ->
-  do_capture(Meta, Expr, S, false);
+capture(Meta, { { '.', _, [_] }, _, Args } = Expr, E) when is_list(Args) ->
+  do_capture(Meta, Expr, E, false);
 
-capture(Meta, { '__block__', _, _ } = Expr, S) ->
+capture(Meta, { '__block__', _, [Expr] }, E) ->
+  capture(Meta, Expr, E);
+
+capture(Meta, { '__block__', _, _ } = Expr, E) ->
   Message = "invalid args for &, block expressions are not allowed, got: ~ts",
-  syntax_error(Meta, S#elixir_scope.file, Message, ['Elixir.Macro':to_string(Expr)]);
+  compile_error(Meta, E#elixir_env.file, Message, ['Elixir.Macro':to_string(Expr)]);
 
-capture(Meta, { Atom, _, Args } = Expr, S) when is_atom(Atom), is_list(Args) ->
-  capture_import(Meta, Expr, S, is_sequential_and_not_empty(Args));
+capture(Meta, { Atom, _, Args } = Expr, E) when is_atom(Atom), is_list(Args) ->
+  capture_import(Meta, Expr, E, is_sequential_and_not_empty(Args));
 
-capture(Meta, { Left, Right }, S) ->
-  capture(Meta, { '{}', Meta, [Left, Right] }, S);
+capture(Meta, { Left, Right }, E) ->
+  capture(Meta, { '{}', Meta, [Left, Right] }, E);
 
-capture(Meta, List, S) when is_list(List) ->
-  capture(Meta, { '[]', Meta, List }, S);
+capture(Meta, List, E) when is_list(List) ->
+  do_capture(Meta, List, E, is_sequential_and_not_empty(List));
 
-capture(Meta, Arg, S) when is_integer(Arg) ->
-  compile_error(Meta, S#elixir_scope.file, "unhandled &~B outside of a capture", [Arg]);
+capture(Meta, Arg, E) ->
+  invalid_capture(Meta, Arg, E).
 
-capture(Meta, Arg, S) ->
-  invalid_capture(Meta, Arg, S).
+capture_import(Meta, { Atom, ImportMeta, Args } = Expr, E, Sequential) ->
+  Res = Sequential andalso
+        elixir_dispatch:import_function(ImportMeta, Atom, length(Args), E),
+  handle_capture(Res, Meta, Expr, E, Sequential).
 
-%% Helpers
+capture_require(Meta, { { '.', _, [Left, Right] }, RequireMeta, Args } = Expr, E, Sequential) ->
+  { Mod, EE } = elixir_exp:expand(Left, E),
+  Res = Sequential andalso is_atom(Mod) andalso
+        elixir_dispatch:require_function(RequireMeta, Mod, Right, length(Args), EE),
+  handle_capture(Res, Meta, Expr, EE, Sequential).
 
-capture_import(Meta, { Atom, ImportMeta, Args } = Expr, S, Sequential) ->
-  case Sequential andalso
-       elixir_dispatch:import_function(ImportMeta, Atom, length(Args), S) of
-    false -> do_capture(Meta, Expr, S, Sequential);
-    Else  -> Else
+handle_capture({ local, Fun, Arity }, _Meta, _Expr, _E, _Sequential) ->
+  { local, Fun, Arity };
+handle_capture({ remote, Receiver, Fun, Arity }, Meta, _Expr, E, _Sequential) ->
+  Tree = { { '.', [], [erlang, make_fun] }, Meta, [Receiver, Fun, Arity] },
+  { expanded, Tree, E };
+handle_capture(false, Meta, Expr, E, Sequential) ->
+  do_capture(Meta, Expr, E, Sequential).
+
+do_capture(Meta, Expr, E, Sequential) ->
+  case do_escape(Expr, elixir_counter:next(), E, []) of
+    { _, [] } when not Sequential ->
+      invalid_capture(Meta, Expr, E);
+    { EExpr, EDict } ->
+      EVars = validate(Meta, EDict, 1, E),
+      Fn = { fn, Meta, [{ '->', Meta, [EVars, EExpr]}]},
+      { expanded, Fn, E }
   end.
 
-capture_require(Meta, { { '.', _, [Left, Right] }, RequireMeta, Args } = Expr, S, Sequential) ->
-  { Mod, SE } = 'Elixir.Macro':expand_all(Left, elixir_scope:to_ex_env({ ?line(Meta), S }), S),
-
-  case Sequential andalso is_atom(Mod) andalso
-       elixir_dispatch:require_function(RequireMeta, Mod, Right, length(Args), SE) of
-    false -> do_capture(Meta, Expr, S, Sequential);
-    Else  -> Else
-  end.
-
-do_capture(Meta, Expr, S, Sequential) ->
-  case do_escape(Expr, S, []) of
-    { _, _, [] } when not Sequential ->
-      invalid_capture(Meta, Expr, S);
-    { TExpr, TS, TDict } ->
-      TVars = validate(Meta, TDict, 1, S),
-      fn(Meta, [{ TVars, Meta, TExpr }], TS)
-  end.
-
-invalid_capture(Meta, Arg, S) ->
+invalid_capture(Meta, Arg, E) ->
   Message = "invalid args for &, expected an expression in the format of &Mod.fun/arity, "
             "&local/arity or a capture containing at least one argument as &1, got: ~ts",
-  syntax_error(Meta, S#elixir_scope.file, Message, ['Elixir.Macro':to_string(Arg)]).
+  compile_error(Meta, E#elixir_env.file, Message, ['Elixir.Macro':to_string(Arg)]).
 
-validate(Meta, [{ Pos, Var }|T], Pos, S) ->
-  [Var|validate(Meta, T, Pos + 1, S)];
+validate(Meta, [{ Pos, Var }|T], Pos, E) ->
+  [Var|validate(Meta, T, Pos + 1, E)];
 
-validate(Meta, [{ Pos, _ }|_], Expected, S) ->
-  compile_error(Meta, S#elixir_scope.file, "capture &~B cannot be defined without &~B", [Pos, Expected]);
+validate(Meta, [{ Pos, _ }|_], Expected, E) ->
+  compile_error(Meta, E#elixir_env.file, "capture &~B cannot be defined without &~B", [Pos, Expected]);
 
-validate(_Meta, [], _Pos, _S) ->
+validate(_Meta, [], _Pos, _E) ->
   [].
 
-do_escape({ '&', Meta, [Pos] }, S, Dict) when is_integer(Pos), Pos > 0 ->
-  case orddict:find(Pos, Dict) of
-    { ok, Var } ->
-      { Var, S, Dict };
-    error ->
-      { Var, SC } = elixir_scope:build_ex_var(?line(Meta), S),
-      { Var, SC, orddict:store(Pos, Var, Dict) }
-  end;
+do_escape({ '&', _, [Pos] }, Counter, _E, Dict) when is_integer(Pos), Pos > 0 ->
+  Var = { list_to_atom([$x, $@+Pos]), [{ counter, Counter }], elixir_fn },
+  { Var, orddict:store(Pos, Var, Dict) };
 
-do_escape({ '&', Meta, [Pos] }, S, _Dict) when is_integer(Pos) ->
-  compile_error(Meta, S#elixir_scope.file, "capture &~B is not allowed", [Pos]);
+do_escape({ '&', Meta, [Pos] }, _Counter, E, _Dict) when is_integer(Pos) ->
+  compile_error(Meta, E#elixir_env.file, "capture &~B is not allowed", [Pos]);
 
-do_escape({ '&', Meta, _ } = Arg, S, _Dict) ->
+do_escape({ '&', Meta, _ } = Arg, _Counter, E, _Dict) ->
   Message = "nested captures via & are not allowed: ~ts",
-  compile_error(Meta, S#elixir_scope.file, Message, ['Elixir.Macro':to_string(Arg)]);
+  compile_error(Meta, E#elixir_env.file, Message, ['Elixir.Macro':to_string(Arg)]);
 
-do_escape({ Left, Meta, Right }, S0, Dict0) ->
-  { TLeft, S1, Dict1 }  = do_escape(Left, S0, Dict0),
-  { TRight, S2, Dict2 } = do_escape(Right, S1, Dict1),
-  { { TLeft, Meta, TRight }, S2, Dict2 };
+do_escape({ Left, Meta, Right }, Counter, E, Dict0) ->
+  { TLeft, Dict1 }  = do_escape(Left, Counter, E, Dict0),
+  { TRight, Dict2 } = do_escape(Right, Counter, E, Dict1),
+  { { TLeft, Meta, TRight }, Dict2 };
 
-do_escape({ Left, Right }, S0, Dict0) ->
-  { TLeft, S1, Dict1 }  = do_escape(Left, S0, Dict0),
-  { TRight, S2, Dict2 } = do_escape(Right, S1, Dict1),
-  { { TLeft, TRight }, S2, Dict2 };
+do_escape({ Left, Right }, Counter, E, Dict0) ->
+  { TLeft, Dict1 }  = do_escape(Left, Counter, E, Dict0),
+  { TRight, Dict2 } = do_escape(Right, Counter, E, Dict1),
+  { { TLeft, TRight }, Dict2 };
 
-do_escape(List, S, Dict) when is_list(List) ->
-  do_escape_list(List, S, Dict, []);
+do_escape(List, Counter, E, Dict) when is_list(List) ->
+  lists:mapfoldl(fun(X, Acc) -> do_escape(X, Counter, E, Acc) end, Dict, List);
 
-do_escape(Other, S, Dict) ->
-  { Other, S, Dict }.
-
-do_escape_list([H|T], S, Dict, Acc) ->
-  { TH, TS, TDict } = do_escape(H, S, Dict),
-  do_escape_list(T, TS, TDict, [TH|Acc]);
-
-do_escape_list([], S, Dict, Acc) ->
-  { lists:reverse(Acc), S, Dict }.
+do_escape(Other, _Counter, _E, Dict) ->
+  { Other, Dict }.
 
 is_sequential_and_not_empty([])   -> false;
 is_sequential_and_not_empty(List) -> is_sequential(List, 1).

lib/elixir/src/elixir_import.erl

@@ -7,56 +7,55 @@
 
 %% IMPORT
 
-import(Meta, Ref, Opts, S) ->
-  SI =
+import(Meta, Ref, Opts, E) ->
+  Res =
     case keyfind(only, Opts) of
       { only, functions } ->
-        import_functions(Meta, Ref, Opts, S);
+        { import_functions(Meta, Ref, Opts, E),
+          E#elixir_env.macros };
       { only, macros } ->
-        import_macros(true, Meta, Ref, Opts, S);
+        { E#elixir_env.functions,
+          import_macros(true, Meta, Ref, Opts, E) };
       { only, List } when is_list(List) ->
-        import_macros(false, Meta, Ref, Opts, import_functions(Meta, Ref, Opts, S));
+        { import_functions(Meta, Ref, Opts, E),
+          import_macros(false, Meta, Ref, Opts, E) };
       false ->
-        import_macros(false, Meta, Ref, Opts, import_functions(Meta, Ref, Opts, S))
+        { import_functions(Meta, Ref, Opts, E),
+          import_macros(false, Meta, Ref, Opts, E) }
     end,
-  record_warn(Meta, Ref, Opts, SI),
-  SI.
 
-import_functions(Meta, Ref, Opts, S) ->
-  { Functions, Temp } = calculate(Meta, Ref, Opts, S#elixir_scope.functions,
-    S#elixir_scope.macro_functions, fun() -> get_functions(Ref) end, S),
-  S#elixir_scope{functions=Functions, macro_functions=Temp}.
+  record_warn(Meta, Ref, Opts, E),
+  Res.
 
-import_macros(Force, Meta, Ref, Opts, S) ->
-  Existing = fun() ->
+import_functions(Meta, Ref, Opts, E) ->
+  calculate(Meta, Ref, Opts, E#elixir_env.functions, E, fun() -> get_functions(Ref) end).
+
+import_macros(Force, Meta, Ref, Opts, E) ->
+  calculate(Meta, Ref, Opts, E#elixir_env.macros, E, fun() ->
     case Force of
-      true  -> get_macros(Meta, Ref, S);
+      true  -> get_macros(Meta, Ref, E);
       false -> get_optional_macros(Ref)
     end
-  end,
-  { Macros, Temp } = calculate(Meta, Ref, Opts, S#elixir_scope.macros,
-    S#elixir_scope.macro_macros, Existing, S),
-  S#elixir_scope{macros=Macros, macro_macros=Temp}.
+  end).
 
-record_warn(_Meta, _Ref, _Opts, #elixir_scope{module=nil}) -> false;
-record_warn(Meta, Ref, Opts, #elixir_scope{module=Module}) ->
+record_warn(Meta, Ref, Opts, E) ->
   Warn =
     case keyfind(warn, Opts) of
       { warn, false } -> false;
       { warn, true } -> true;
       false -> not lists:keymember(context, 1, Meta)
     end,
-  elixir_tracker:record_warn(Ref, Warn, ?line(Meta), Module).
+  elixir_lexical:record_import(Ref, ?line(Meta), Warn, E#elixir_env.lexical_tracker).
 
 %% Calculates the imports based on only and except
 
-calculate(Meta, Key, Opts, Old, Temp, Existing, S) ->
+calculate(Meta, Key, Opts, Old, E, Existing) ->
   New = case keyfind(only, Opts) of
     { only, Only } when is_list(Only) ->
       case Only -- get_exports(Key) of
         [{Name,Arity}|_] ->
           Tuple = { invalid_import, { Key, Name, Arity } },
-          elixir_errors:form_error(Meta, S#elixir_scope.file, ?MODULE, Tuple);
+          elixir_errors:form_error(Meta, E#elixir_env.file, ?MODULE, Tuple);
         _ ->
           intersection(Only, Existing())
       end;
@@ -77,25 +76,10 @@ calculate(Meta, Key, Opts, Old, Temp, Existing, S) ->
   Final = remove_internals(Set),
 
   case Final of
-    [] ->
-      { keydelete(Key, Old),
-        if_quoted(Meta, Temp, fun(Value) -> keydelete(Key, Value) end) };
+    [] -> keydelete(Key, Old);
     _  ->
-      ensure_no_special_form_conflict(Meta, S#elixir_scope.file, Key, Final),
-      { [{ Key, Final }|keydelete(Key, Old)],
-        if_quoted(Meta, Temp, fun(Value) -> [{ Key, Final }|keydelete(Key, Value)] end) }
-  end.
-
-if_quoted(Meta, Temp, Callback) ->
-  case lists:keyfind(context, 1, Meta) of
-    { context, Context } ->
-      Current = case orddict:find(Context, Temp) of
-        { ok, Value } -> Value;
-        error -> []
-      end,
-      orddict:store(Context, Callback(Current), Temp);
-    _ ->
-      Temp
+      ensure_no_special_form_conflict(Meta, E#elixir_env.file, Key, Final),
+      [{ Key, Final }|keydelete(Key, Old)]
   end.
 
 %% Retrieve functions and macros from modules
@@ -114,13 +98,13 @@ get_functions(Module) ->
     error:undef -> Module:module_info(exports)
   end.
 
-get_macros(Meta, Module, S) ->
+get_macros(Meta, Module, E) ->
   try
     Module:'__info__'(macros)
   catch
     error:undef ->
       Tuple = { no_macros, Module },
-      elixir_errors:form_error(Meta, S#elixir_scope.file, ?MODULE, Tuple)
+      elixir_errors:form_error(Meta, E#elixir_env.file, ?MODULE, Tuple)
   end.
 
 get_optional_macros(Module)  ->
@@ -151,14 +135,14 @@ ensure_no_special_form_conflict(_Meta, _File, _Key, []) -> ok.
 
 format_error({invalid_import,{Receiver, Name, Arity}}) ->
   io_lib:format("cannot import ~ts.~ts/~B because it doesn't exist",
-    [elixir_errors:inspect(Receiver), Name, Arity]);
+    [elixir_aliases:inspect(Receiver), Name, Arity]);
 
 format_error({special_form_conflict,{Receiver, Name, Arity}}) ->
   io_lib:format("cannot import ~ts.~ts/~B because it conflicts with Elixir special forms",
-    [elixir_errors:inspect(Receiver), Name, Arity]);
+    [elixir_aliases:inspect(Receiver), Name, Arity]);
 
 format_error({ no_macros, Module }) ->
-  io_lib:format("could not load macros from module ~ts", [elixir_errors:inspect(Module)]).
+  io_lib:format("could not load macros from module ~ts", [elixir_aliases:inspect(Module)]).
 
 %% LIST HELPERS
 
@@ -197,12 +181,10 @@ special_form('^',1) -> true;
 special_form('=',2) -> true;
 special_form('__op__',2) -> true;
 special_form('__op__',3) -> true;
-special_form('__scope__',2) -> true;
 special_form('__block__',_) -> true;
 special_form('->',_) -> true;
 special_form('<<>>',_) -> true;
 special_form('{}',_) -> true;
-special_form('[]',_) -> true;
 special_form('alias',1) -> true;
 special_form('alias',2) -> true;
 special_form('require',1) -> true;
@@ -212,7 +194,6 @@ special_form('import',2) -> true;
 special_form('__ENV__',0) -> true;
 special_form('__CALLER__',0) -> true;
 special_form('__MODULE__',0) -> true;
-special_form('__FILE__',0) -> true;
 special_form('__DIR__',0) -> true;
 special_form('__aliases__',_) -> true;
 special_form('quote',1) -> true;
@@ -221,10 +202,9 @@ special_form('unquote',1) -> true;
 special_form('unquote_splicing',1) -> true;
 special_form('fn',_) -> true;
 special_form('super',_) -> true;
-special_form('super?',0) -> true;
 special_form('bc',_) -> true;
 special_form('lc',_) -> true;
-special_form('var!',1) -> true;
-special_form('var!',2) -> true;
-special_form('alias!',1) -> true;
+special_form('case',2) -> true;
+special_form('try',2) -> true;
+special_form('receive',1) -> true;
 special_form(_, _) -> false.

lib/elixir/src/elixir_interpolation.erl

@@ -6,62 +6,70 @@ unescape_tokens/1, unescape_tokens/2, unescape_map/1]).
 
 %% Extract string interpolations
 
-extract(Line, File, Interpol, String, Last) ->
-  extract(Line, File, Interpol, String, [], [], [], Last).
+extract(Line, Raw, Interpol, String, Last) ->
+  %% Ignore whatever is in the scope and enable terminator checking.
+  Scope = Raw#elixir_tokenizer{terminators=[], check_terminators=true},
+  extract(Line, Scope, Interpol, String, [], 0, [], Last).
 
-extract(Line, File, _Interpol, [], Buffer, [], Output, []) ->
-  finish_extraction(Line, File, Buffer, Output, []);
+%% Terminators
 
-extract(Line, _File, _Interpol, [], _Buffer, [], _Output, Last) ->
-  { error, { Line, io_lib:format("missing terminator: ~ts", [[Last]]), [] } };
+extract(Line, _Scope, _Interpol, [], Buffer, 0, Output, []) ->
+  finish_extraction(Line, Buffer, Output, []);
 
-extract(Line, File, _Interpol, [Last|Remaining], Buffer, [], Output, Last) ->
-  finish_extraction(Line, File, Buffer, Output, Remaining);
+extract(Line, _Scope, _Interpol, [], _Buffer, 0, _Output, Last) ->
+  { error, { string, Line, io_lib:format("missing terminator: ~ts", [[Last]]), [] } };
 
-extract(Line, _File, _Interpol, End, _Buffer, _Search, _Output, Last) when End == [Last]; End == [] ->
-  { error, { Line, io_lib:format("missing terminator: ~ts", [[Last]]), [] } };
+extract(Line, _Scope, _Interpol, [Last|Remaining], Buffer, 0, Output, Last) ->
+  finish_extraction(Line, Buffer, Output, Remaining);
 
-extract(Line, File, Interpol, [$\n|Rest], Buffer, Search, Output, Last) ->
-  extract(Line+1, File, Interpol, Rest, [$\n|Buffer], Search, Output, Last);
+extract(Line, _Scope, _Interpol, [], _Buffer, _Search, _Output, Last) ->
+  { error, { string, Line, io_lib:format("missing terminator: ~ts", [[Last]]), [] } };
 
-extract(Line, File, Interpol, [$\\, $#, ${|Rest], Buffer, [], Output, Last) ->
-  extract(Line, File, Interpol, Rest, [${,$#|Buffer], [], Output, Last);
+%% Going through the string
 
-extract(Line, File, Interpol, [$\\,Char|Rest], Buffer, [], Output, Last) ->
-  extract(Line, File, Interpol, Rest, [Char,$\\|Buffer], [], Output, Last);
+extract(Line, Scope, Interpol, [$\n|Rest], Buffer, Search, Output, Last) ->
+  extract(Line+1, Scope, Interpol, Rest, [$\n|Buffer], Search, Output, Last);
 
-extract(Line, File, true, [$#, ${|Rest], Buffer, [], Output, Last) ->
-  NewOutput = build_interpol(s, Line, File, Buffer, Output),
-  extract(Line, File, true, Rest, [], [$}], NewOutput, Last);
+extract(Line, Scope, Interpol, [$\\, $#, ${|Rest], Buffer, Search, Output, Last) ->
+  extract(Line, Scope, Interpol, Rest, [${,$#|Buffer], Search, Output, Last);
 
-extract(Line, File, true, [$}|Rest], Buffer, [$}], Output, Last) ->
-  NewOutput = build_interpol(i, Line, File, Buffer, Output),
-  extract(Line, File, true, Rest, [], [], NewOutput, Last);
+extract(Line, Scope, Interpol, [$\\,Char|Rest], Buffer, Search, Output, Last) ->
+  extract(Line, Scope, Interpol, Rest, [Char,$\\|Buffer], Search, Output, Last);
 
-%% Check for available separators inside interpolation
+extract(Line, Scope, true, [$#, ${|Rest], Buffer, Search, Output, Last) ->
+  Output1 = build_string(Line, Buffer, Output),
 
-extract(Line, File, Interpol, [C|Rest], Buffer, [C|Search], Output, Last) when C == $); C == $]; C == $}; C == $>; C == $"; C == $' ->
-  extract(Line, File, Interpol, Rest, [C|Buffer], Search, Output, Last);
+  case elixir_tokenizer:tokenize(Rest, Line, Scope) of
+    { error, { EndLine, _, "}" }, [$}|NewRest], Tokens } ->
+      Output2 = build_interpol(Line, Tokens, Output1),
+      extract(EndLine, Scope, true, NewRest, [], Search, Output2, Last);
+    { error, Reason, _, _ } ->
+      { error, Reason };
+    { ok, _EndLine, _ } ->
+      { error, { string, Line, "missing interpolation terminator: }", [] } }
+  end;
 
-extract(Line, File, Interpol, [C|Rest], Buffer, [_|_] = Search, Output, Last) when C == $"; C == $' ->
-  extract(Line, File, Interpol, Rest, [C|Buffer], [C|Search], Output, Last);
+%% Matching () [] {} <> inside sigils
 
-extract(Line, File, Interpol, [$<|Rest], Buffer, [_|_] = Search, Output, Last) ->
-  extract(Line, File, Interpol, Rest, [$<|Buffer], [$>|Search], Output, Last);
+extract(Line, Scope, Interpol, [$(|Rest], Buffer, Search, Output, $)) ->
+  extract(Line, Scope, Interpol, Rest, [$(|Buffer], Search + 1, Output, $));
 
-extract(Line, File, Interpol, [${|Rest], Buffer, [_|_] = Search, Output, Last) ->
-  extract(Line, File, Interpol, Rest, [${|Buffer], [$}|Search], Output, Last);
+extract(Line, Scope, Interpol, [$[|Rest], Buffer, Search, Output, $]) ->
+  extract(Line, Scope, Interpol, Rest, [$[|Buffer], Search + 1, Output, $]);
 
-extract(Line, File, Interpol, [$[|Rest], Buffer, [_|_] = Search, Output, Last) ->
-  extract(Line, File, Interpol, Rest, [$[|Buffer], [$]|Search], Output, Last);
+extract(Line, Scope, Interpol, [${|Rest], Buffer, Search, Output, $}) ->
+  extract(Line, Scope, Interpol, Rest, [${|Buffer], Search + 1, Output, $});
 
-extract(Line, File, Interpol, [$(|Rest], Buffer, [_|_] = Search, Output, Last) ->
-  extract(Line, File, Interpol, Rest, [$(|Buffer], [$)|Search], Output, Last);
+extract(Line, Scope, Interpol, [$<|Rest], Buffer, Search, Output, $>) ->
+  extract(Line, Scope, Interpol, Rest, [$<|Buffer], Search + 1, Output, $>);
 
-%% Else
+extract(Line, Scope, Interpol, [Last|Rest], Buffer, Search, Output, Last) ->
+  extract(Line, Scope, Interpol, Rest, [Last|Buffer], Search - 1, Output, Last);
 
-extract(Line, File, Interpol, [Char|Rest], Buffer, Search, Output, Last) ->
-  extract(Line, File, Interpol, Rest, [Char|Buffer], Search, Output, Last).
+%% Catch all clause
+
+extract(Line, Scope, Interpol, [Char|Rest], Buffer, Search, Output, Last) ->
+  extract(Line, Scope, Interpol, Rest, [Char|Buffer], Search, Output, Last).
 
 %% Unescape a series of tokens as returned by extract.
 
@@ -130,7 +138,13 @@ unescape_chars(<<>>, _Map, _Octals, _Hex, Acc) -> Acc.
 
 append_escaped(Rest, Map, List, Octal, Hex, Acc, Base) ->
   Codepoint = list_to_integer(List, Base),
-  unescape_chars(Rest, Map, Octal, Hex, <<Acc/binary, Codepoint/utf8>>).
+  try <<Acc/binary, Codepoint/utf8>> of
+    Binary -> unescape_chars(Rest, Map, Octal, Hex, Binary)
+  catch
+    error:badarg ->
+      Msg = <<"invalid or reserved unicode codepoint ", (integer_to_binary(Codepoint))/binary>>,
+      error('Elixir.ArgumentError':exception([{message,Msg}]))
+  end.
 
 % Unescape Helpers
 
@@ -148,30 +162,16 @@ unescape_map(E)  -> E.
 
 % Extract Helpers
 
-finish_extraction(Line, File, Buffer, Output, Remaining) ->
-  case build_interpol(s, Line, File, Buffer, Output) of
+finish_extraction(Line, Buffer, Output, Remaining) ->
+  case build_string(Line, Buffer, Output) of
     []    -> Final = [<<>>];
     Final -> []
   end,
   { Line, lists:reverse(Final), Remaining }.
 
-build_interpol(_Kind, _Line, _File, [], Output) ->
-  Output;
+build_string(_Line, [], Output) -> Output;
+build_string(_Line, Buffer, Output) ->
+  [elixir_utils:characters_to_binary(lists:reverse(Buffer))|Output].
 
-build_interpol(s, _Line, _File, Buffer, Output) ->
-  [elixir_utils:characters_to_binary(lists:reverse(Buffer))|Output];
-
-build_interpol(i, Line, File, Buffer, Output) ->
-  [wrap_interpol(Line, forms(lists:reverse(Buffer), Line, File))| Output].
-
-wrap_interpol(Line, Form) ->
-  Meta = [{line,Line}],
-  { '::', Meta, [{ { '.', Meta, ['Elixir.Kernel', to_string] }, Meta, [Form]}, { binary, Meta, nil }]}.
-
-forms(String, StartLine, File) ->
-  case elixir_translator:forms(String, StartLine, File, []) of
-    { ok, [] } -> nil;
-    { ok, [Forms] } when not is_list(Forms) -> Forms;
-    { ok, Forms } -> { '__block__', [{line,StartLine}], Forms };
-    { error, Tuple } -> throw({ interpolation_error, Tuple })
-  end.
+build_interpol(Line, Buffer, Output) ->
+  [{ Line, lists:reverse(Buffer) }|Output].

lib/elixir/src/elixir_lexical.erl

@@ -0,0 +1,79 @@
+%% Module responsible for tracking lexical information.
+-module(elixir_lexical).
+-export([run/2,
+  record_alias/4, record_alias/2,
+  record_import/4, record_import/2,
+  record_remote/2, format_error/1
+]).
+-include("elixir.hrl").
+
+-define(tracker, 'Elixir.Kernel.LexicalTracker').
+
+run(File, Callback) ->
+  case code:is_loaded(?tracker) of
+    { file, _ } ->
+      Pid = ?tracker:start_link(),
+      try
+        Callback(Pid)
+      after
+        warn_unused_aliases(File, Pid),
+        warn_unused_imports(File, Pid),
+        unlink(Pid), ?tracker:stop(Pid)
+      end;
+    false ->
+      Callback(nil)
+  end.
+
+%% RECORD
+
+record_alias(Module, Line, Warn, Ref) ->
+  if_tracker(Ref, fun(Pid) ->
+    ?tracker:add_alias(Pid, Module, Line, Warn),
+    true
+  end).
+
+record_import(Module, Line, Warn, Ref) ->
+  if_tracker(Ref, fun(Pid) ->
+    ?tracker:add_import(Pid, Module, Line, Warn),
+    true
+  end).
+
+record_alias(Module, Ref) ->
+  if_tracker(Ref, fun(Pid) ->
+    ?tracker:alias_dispatch(Pid, Module),
+    true
+  end).
+
+record_import(Module, Ref) ->
+  if_tracker(Ref, fun(Pid) ->
+    ?tracker:import_dispatch(Pid, Module),
+    true
+  end).
+
+record_remote(Module, Ref) ->
+  if_tracker(Ref, fun(Pid) ->
+    ?tracker:remote_dispatch(Pid, Module),
+    true
+  end).
+
+%% HELPERS
+
+if_tracker(nil, _Callback) -> false;
+if_tracker(Pid, Callback) when is_pid(Pid) -> Callback(Pid).
+
+%% ERROR HANDLING
+
+warn_unused_imports(File, Pid) ->
+  [ begin
+      elixir_errors:handle_file_warning(File, { L, ?MODULE, { unused_import, M } })
+    end || { M, L } <- ?tracker:collect_unused_imports(Pid)].
+
+warn_unused_aliases(File, Pid) ->
+  [ begin
+      elixir_errors:handle_file_warning(File, { L, ?MODULE, { unused_alias, M } })
+    end || { M, L } <- ?tracker:collect_unused_aliases(Pid)].
+
+format_error({unused_alias, Module}) ->
+  io_lib:format("unused alias ~ts", [elixir_aliases:inspect(Module)]);
+format_error({unused_import, Module}) ->
+  io_lib:format("unused import ~ts", [elixir_aliases:inspect(Module)]).

lib/elixir/src/elixir_literal.erl

@@ -1,214 +0,0 @@
-%% Handle translation of Elixir literals to Erlang AST.
--module(elixir_literal).
--export([translate/2]).
--import(elixir_translator, [translate_each/2, translate_args/2]).
--import(elixir_scope, [umergec/2]).
--include("elixir.hrl").
-
-translate({ '<<>>', Meta, Args } = Original, S) when is_list(Args) ->
-  case elixir_partials:handle(Original, S, allow_tail) of
-    error ->
-      case S#elixir_scope.context of
-        match ->
-          build_bitstr(fun elixir_translator:translate_each/2, Args, Meta, S);
-        _ ->
-          { TArgs, { SC, SV } } = build_bitstr(fun elixir_translator:translate_arg/2, Args, Meta, { S, S }),
-          { TArgs, umergec(SV, SC) }
-      end;
-    Else -> Else
-  end;
-
-translate({ '{}', Meta, Args } = Original, S) when is_list(Args) ->
-  case elixir_partials:handle(Original, S) of
-    error ->
-      { TArgs, SE } = translate_args(Args, S),
-      { { tuple, ?line(Meta), TArgs }, SE };
-    Else -> Else
-  end;
-
-translate({ '[]', _Meta, [] }, S) ->
-  { { nil, 0 }, S };
-
-translate({ '[]', Meta, Args } = Original, S) when is_list(Args) ->
-  case elixir_partials:handle(Original, S, allow_tail) of
-    error ->
-      [RTail|RArgs] = lists:reverse(Args),
-
-      case RTail of
-        {'|',_,[Left,Right]} ->
-          RExprs = [Left|RArgs],
-          TailFun = fun(ST) -> translate_each(Right, ST) end;
-        _ ->
-          RExprs = [RTail|RArgs],
-          TailFun = fun(ST) -> { { nil, ?line(Meta) }, ST } end
-      end,
-
-      { Exprs, SE } = translate_args(lists:reverse(RExprs), S),
-      { Tail, ST }  = TailFun(SE),
-      { elixir_utils:list_to_cons(?line(Meta), Exprs, Tail), ST };
-    Else -> Else
-  end;
-
-translate({ Left, Right }, S) ->
-  translate({ '{}', [], [Left, Right]}, S);
-
-translate(Args, S) when is_list(Args) ->
-  translate({ '[]', [], Args }, S);
-
-translate(Tuple, S) when is_tuple(Tuple) ->
-  elixir_errors:compile_error(0, S#elixir_scope.file,
-    "tuples in quoted expressions must have 2 or 3 items, invalid quoted expression: ~ts",
-    ['Elixir.Kernel':inspect(Tuple)]);
-
-translate(Other, S) ->
-  { elixir_utils:elixir_to_erl(0, Other, S), S }.
-
-%% Helpers
-
-% Build a bitstring taking into accounts the following types:
-%
-% * If a bitstring or a list is given, we just append its items
-% * If '::' is given, extract the bitstring information
-% * All the other types are simply translated and handled with Erlang's default
-%
-build_bitstr(Fun, Exprs, Meta, S) ->
-  { Final, FinalS } = build_bitstr_each(Fun, Exprs, Meta, S, []),
-  { { bin, ?line(Meta), lists:reverse(Final) }, FinalS }.
-
-build_bitstr_each(_Fun, [], _Meta, S, Acc) ->
-  { Acc, S };
-
-build_bitstr_each(Fun, [{'::',_,[H,V]}|T], Meta, S, Acc) ->
-  %% Variables defined outside the binary can be accounted
-  %% on subparts, however we can't assign new variables.
-  case S of
-    { ES, _ } -> [];                      %% translate_arg,  no assigns
-    _ -> ES = S#elixir_scope{context=nil} %% translate_each, revert assigns
-  end,
-
-  { Size, Types } = extract_bit_values(Meta, V, ES),
-  build_bitstr_each(Fun, T, Meta, S, Acc, H, Size, Types);
-
-build_bitstr_each(Fun, [H|T], Meta, S, Acc) ->
-  build_bitstr_each(Fun, T, Meta, S, Acc, H, default, default).
-
-build_bitstr_each(Fun, T, Meta, S, Acc, H, Size, Types) ->
-  { Expr, NS } = Fun(H, S),
-
-  AllowString = types_allow_string(Types),
-  AllowSplice = types_allow_splice(Types),
-  AllowAny    = (AllowString orelse AllowSplice) andalso (Size == default),
-
-  case AllowAny andalso Expr of
-    { bin, _, [{ bin_element, 0, { string, 0, String }, default, default }] } when AllowString ->
-      build_bitstr_each(Fun, T, Meta, NS, [{ bin_element, ?line(Meta), { string, 0, String }, Size, Types }|Acc]);
-    { bin, _, Elements } when AllowSplice ->
-      build_bitstr_each(Fun, T, Meta, NS, lists:reverse(Elements) ++ Acc);
-    { cons, _, _, _ } = Cons ->
-      build_bitstr_each(Fun, T, Meta, NS, rehash_cons(Cons, Size, Types, []) ++ Acc);
-    { nil, _ } ->
-      build_bitstr_each(Fun, T, Meta, NS, Acc);
-    _ ->
-      build_bitstr_each(Fun, T, Meta, NS, [{ bin_element, ?line(Meta), Expr, Size, Types }|Acc])
-  end.
-
-rehash_cons({ nil, _ }, _Size, _Types, Acc) -> Acc;
-rehash_cons({ cons, Line, Left, Right }, Size, Types, Acc) ->
-  rehash_cons(Right, Size, Types, [{ bin_element, Line, Left, Size, Types }|Acc]).
-
-types_allow_string([End|T]) when End == little; End == big -> types_allow_string(T);
-types_allow_string([UTF|T]) when UTF == utf8; UTF == utf16; UTF == utf32 -> types_allow_string(T);
-types_allow_string([]) -> true;
-types_allow_string(_) -> false.
-
-types_allow_splice(default) -> true;
-types_allow_splice([bytes]) -> true;
-types_allow_splice([binary]) -> true;
-types_allow_splice([bits]) -> true;
-types_allow_splice([bitstring]) -> true;
-types_allow_splice(_) -> false.
-
-%% Extra bitstring specifiers
-
-extract_bit_values(Meta, V, S) when is_list(V) ->
-  extract_bit_values(Meta, V, default, [], S);
-
-extract_bit_values(Meta, V, S) ->
-  extract_bit_values(Meta, [V], S).
-
-extract_bit_values(Meta, [{ Value, _Meta, Atom }|T] = All, Size, Types, S)
-    when is_atom(Value) andalso (is_atom(Atom) orelse Atom == []) ->
-  case extract_bit_type(Value, Types) of
-    { error, unknown } ->
-      handle_unknown_specifier(Meta, All, Size, Types, S);
-    NewTypes when is_list(NewTypes) ->
-      extract_bit_values(Meta, T, Size, NewTypes, S)
-  end;
-
-extract_bit_values(Meta, [{ Value, CallMeta, [_] = Args }|T] = All, Size, Types, S) when is_atom(Value) ->
-  { TArgs, _ } = elixir_translator:translate_args(Args, S),
-  case extract_bit_type_or_size(Value, TArgs, Size, Types) of
-    { error, unknown } ->
-      handle_unknown_specifier(Meta, All, Size, Types, S);
-    { error, Msg } ->
-      elixir_errors:syntax_error(CallMeta, S#elixir_scope.file, Msg);
-    { NewSize, NewTypes } ->
-      extract_bit_values(Meta, T, NewSize, NewTypes, S)
-  end;
-
-extract_bit_values(Meta, [Size|T], default, Types, S) when is_integer(Size) andalso Size >= 0 ->
-  extract_bit_values(Meta, T, {integer, Meta, Size}, Types, S);
-
-extract_bit_values(Meta, [{ '|', _, [Left, Right] }], Size, Types, S) ->
-  { Expanded, ES } = 'Elixir.Macro':expand_all(Right, elixir_scope:to_ex_env({ ?line(Meta), S }), S),
-  extract_bit_values(Meta, [Left|join_expansion(Expanded,[])], Size, Types, ES);
-
-extract_bit_values(Meta, [_|_] = All, Size, Types, S) ->
-  handle_unknown_specifier(Meta, All, Size, Types, S);
-
-extract_bit_values(_Meta, [], Size, [], _S) ->
-  { Size, default };
-
-extract_bit_values(_Meta, [], Size, Types, _S) ->
-  { Size, lists:reverse(Types) }.
-
-extract_bit_type(Value, Types) when
-    Value == binary; Value == integer; Value == float; Value == bitstring;
-    Value == bytes; Value == bits;
-    Value == utf8; Value == utf16; Value == utf32;
-    Value == signed; Value == unsigned;
-    Value == big; Value == little; Value == native ->
-  [Value|Types];
-
-extract_bit_type(_Value, _Types) ->
-  { error, unknown }.
-
-extract_bit_type_or_size(size, [{ Kind, _, _ } = Arg], default, Types) when Kind == var; Kind == integer ->
-  { Arg, Types };
-
-extract_bit_type_or_size(size, _Args, default, _Types) ->
-  { error, "size in bitstring expects an integer or a variable as argument" };
-
-extract_bit_type_or_size(size, _Args, _Other, _Types) ->
-  { error, "duplicated size definition for bitstring" };
-
-extract_bit_type_or_size(unit, [{ integer, _, Arg }], Size, Types) ->
-  { Size, [{ unit, Arg }|Types] };
-
-extract_bit_type_or_size(unit, _Args, _Other, _Types) ->
-  { error, "unit in bitstring expects an integer as argument" };
-
-extract_bit_type_or_size(_Value, _Args, _Other, _Types) ->
-  { error, unknown }.
-
-handle_unknown_specifier(Meta, [H|T], Size, Types, S) ->
-  case 'Elixir.Macro':expand_all(H, elixir_scope:to_ex_env({ ?line(Meta), S }), S) of
-    { H, ES } ->
-      elixir_errors:syntax_error(Meta, ES#elixir_scope.file, "unknown bitstring specifier ~ts", ['Elixir.Macro':to_string(H)]);
-    { E, ES } ->
-      extract_bit_values(Meta, join_expansion(E,T), Size, Types, ES)
-  end.
-
-join_expansion({ '__block__', _, [Expanded] }, Tail) -> join_expansion(Expanded, Tail);
-join_expansion(Expanded, Tail) when is_list(Expanded) -> Expanded ++ Tail;
-join_expansion(Expanded, Tail) -> [Expanded|Tail].

lib/elixir/src/elixir_locals.erl

@@ -0,0 +1,184 @@
+%% Module responsible for tracking invocations of module calls.
+-module(elixir_locals).
+-export([
+  setup/1, cleanup/1, cache_env/1, get_cached_env/1,
+  record_local/2, record_local/3, record_import/4,
+  record_definition/3, record_defaults/4,
+  ensure_no_function_conflict/4, warn_unused_local/3, format_error/1
+]).
+-export([macro_for/3, function_for/3]).
+
+-include("elixir.hrl").
+-define(attr, '__locals_tracker').
+-define(tracker, 'Elixir.Module.LocalsTracker').
+
+macro_for(nil, _Name, _Arity) -> false;
+macro_for(Module, Name, Arity) ->
+  Tuple = { Name, Arity },
+  try elixir_def:lookup_definition(Module, Tuple) of
+    { { Tuple, Kind, Line, _, _, _, _ }, [_|_] = Clauses }
+        when Kind == defmacro; Kind == defmacrop ->
+      fun() -> get_function(Line, Module, Clauses) end;
+    _ ->
+      false
+  catch
+    error:badarg -> false
+  end.
+
+function_for(Module, Name, Arity) ->
+  Tuple = { Name, Arity },
+  case elixir_def:lookup_definition(Module, Tuple) of
+    { { Tuple, _, Line, _, _, _, _ }, Clauses } when Clauses /= [] ->
+      %% There is no need to record such calls
+      %% since they come from funtions that were
+      %% already analyzed at compilation time.
+      get_function(Line, Module, Clauses);
+    _ ->
+      [_|T] = erlang:get_stacktrace(),
+      erlang:raise(error, undef, [{Module,Name,Arity,[]}|T])
+  end.
+
+get_function(Line, Module, Clauses) ->
+  RewrittenClauses = [rewrite_clause(Clause, Module) || Clause <- Clauses],
+  Fun = { 'fun', Line, {clauses, RewrittenClauses } },
+  { value, Result, _Binding } = erl_eval:exprs([Fun], []),
+  Result.
+
+rewrite_clause({ call, Line, { atom, Line, RawName }, Args }, Module) ->
+  Remote = { remote, Line,
+    { atom, Line, ?MODULE },
+    { atom, Line, function_for }
+  },
+
+  %% If we have a macro, its arity in the table is
+  %% actually one less than in the function call
+  { Name, Arity } = case atom_to_list(RawName) of
+    "MACRO-" ++ Rest -> { list_to_atom(Rest), length(Args) - 1 };
+    _ -> { RawName, length(Args) }
+  end,
+
+  FunCall = { call, Line, Remote, [
+    { atom, Line, Module }, { atom, Line, Name }, { integer, Line, Arity }
+  ] },
+  { call, Line, FunCall, Args };
+
+rewrite_clause(Tuple, Module) when is_tuple(Tuple) ->
+  list_to_tuple(rewrite_clause(tuple_to_list(Tuple), Module));
+
+rewrite_clause(List, Module) when is_list(List) ->
+  [rewrite_clause(Item, Module) || Item <- List];
+
+rewrite_clause(Else, _) -> Else.
+
+%% TRACKING
+
+setup(Module) ->
+  case code:is_loaded(?tracker) of
+    { file, _ } -> ets:insert(Module, { ?attr, ?tracker:start_link() });
+    false -> ok
+  end.
+
+cleanup(Module) ->
+  if_tracker(Module, fun(Pid) -> unlink(Pid), ?tracker:stop(Pid) end).
+
+record_local(Tuple, Module) when is_atom(Module) ->
+  if_tracker(Module, fun(Pid) ->
+    ?tracker:add_local(Pid, Tuple),
+    true
+  end).
+record_local(Tuple, _Module, Function)
+  when Function == nil; Function == Tuple -> false;
+record_local(Tuple, Module, Function) ->
+  if_tracker(Module, fun(Pid) ->
+    ?tracker:add_local(Pid, Function, Tuple),
+    true
+  end).
+
+record_import(_Tuple, Receiver, Module, _Function)
+  when Module == nil; Module == Receiver -> false;
+record_import(Tuple, Receiver, Module, Function) ->
+  if_tracker(Module, fun(Pid) ->
+    ?tracker:add_import(Pid, Function, Receiver, Tuple),
+    true
+  end).
+
+record_definition(Tuple, Kind, Module) ->
+  if_tracker(Module, fun(Pid) ->
+    ?tracker:add_definition(Pid, Kind, Tuple),
+    true
+  end).
+
+record_defaults(_Tuple, _Kind, _Module, 0) ->
+  true;
+record_defaults(Tuple, Kind, Module, Defaults) ->
+  if_tracker(Module, fun(Pid) ->
+    ?tracker:add_defaults(Pid, Kind, Tuple, Defaults),
+    true
+  end).
+
+if_tracker(Module, Callback) ->
+  try ets:lookup_element(Module, ?attr, 2) of
+    Pid -> Callback(Pid)
+  catch
+    error:badarg -> false
+  end.
+
+%% CACHING
+
+cache_env(#elixir_env{module=Module} = RE) ->
+  E = RE#elixir_env{line=nil,vars=[]},
+  try ets:lookup_element(Module, ?attr, 2) of
+    Pid ->
+      { Pid, ?tracker:cache_env(Pid, E) }
+  catch
+    error:badarg ->
+      { Escaped, _ } = elixir_quote:escape(E, false),
+      Escaped
+  end;
+cache_env(ExEnv) ->
+  cache_env(elixir_env:ex_to_env(ExEnv)).
+
+get_cached_env({Pid,Ref}) -> ?tracker:get_cached_env(Pid, Ref);
+get_cached_env(Env) -> Env.
+
+%% ERROR HANDLING
+
+ensure_no_function_conflict(Meta, File, Module, AllDefined) ->
+  if_tracker(Module, fun(Pid) ->
+    [ begin
+        elixir_errors:form_error(Meta, File, ?MODULE, { function_conflict, Error })
+      end || Error <- ?tracker:collect_imports_conflicts(Pid, AllDefined) ]
+  end),
+  ok.
+
+warn_unused_local(File, Module, Private) ->
+  if_tracker(Module, fun(Pid) ->
+    Args = [ { Fun, Kind, Defaults } ||
+             { Fun, Kind, _Line, true, Defaults } <- Private],
+
+    Unused = ?tracker:collect_unused_locals(Pid, Args),
+
+    [ begin
+        { _, _, Line, _, _ } = lists:keyfind(element(2, Error), 1, Private),
+        elixir_errors:handle_file_warning(File, { Line, ?MODULE, Error })
+      end || Error <- Unused ]
+  end).
+
+format_error({function_conflict,{Receivers, Name, Arity}}) ->
+  io_lib:format("imported ~ts.~ts/~B conflicts with local function",
+    [elixir_aliases:inspect(hd(Receivers)), Name, Arity]);
+
+format_error({unused_args,{Name, Arity}}) ->
+  io_lib:format("default arguments in ~ts/~B are never used", [Name, Arity]);
+
+format_error({unused_args,{Name, Arity},1}) ->
+  io_lib:format("the first default argument in ~ts/~B is never used", [Name, Arity]);
+
+format_error({unused_args,{Name, Arity},Count}) ->
+  io_lib:format("the first ~B default arguments in ~ts/~B are never used", [Count, Name, Arity]);
+
+format_error({unused_def,{Name, Arity},defp}) ->
+  io_lib:format("function ~ts/~B is unused", [Name, Arity]);
+
+format_error({unused_def,{Name, Arity},defmacrop}) ->
+  io_lib:format("macro ~ts/~B is unused", [Name, Arity]).

lib/elixir/src/elixir_macros.erl

@@ -1,333 +0,0 @@
-%% Those macros behave like they belong to Kernel,
-%% but do not since they need to be implemented in Erlang.
--module(elixir_macros).
--export([translate/2]).
--import(elixir_translator, [translate_each/2, translate_args/2, translate_apply/7]).
--import(elixir_scope, [umergec/2, umergea/2]).
--import(elixir_errors, [compile_error/3, syntax_error/3, syntax_error/4,
-  assert_no_function_scope/3, assert_module_scope/3, assert_no_match_or_guard_scope/3]).
-
--include("elixir.hrl").
--define(opt_in_types(Kind), Kind == atom orelse Kind == integer orelse Kind == float).
-
-%% Operators
-
-translate({ '+', _Meta, [Expr] }, S) when is_number(Expr) ->
-  translate_each(Expr, S);
-
-translate({ '-', _Meta, [Expr] }, S) when is_number(Expr) ->
-  translate_each(-1 * Expr, S);
-
-translate({ Op, Meta, Exprs }, S) when is_list(Exprs),
-    Op == '<-' orelse Op == '--' ->
-  assert_no_match_or_guard_scope(Meta, Op, S),
-  translate_each({ '__op__', Meta, [Op|Exprs] }, S);
-
-translate({ Op, Meta, Exprs }, S) when is_list(Exprs),
-    Op == '+'   orelse Op == '-'   orelse Op == '*'   orelse Op == '/' orelse
-    Op == '++'  orelse Op == 'not' orelse Op == 'and' orelse Op == 'or' orelse
-    Op == 'xor' orelse Op == '<'   orelse Op == '>'   orelse Op == '<=' orelse
-    Op == '>='  orelse Op == '=='  orelse Op == '!='  orelse Op == '===' orelse
-    Op == '!==' ->
-  translate_each({ '__op__', Meta, [Op|Exprs] }, S);
-
-translate({ '!', Meta, [{ '!', _, [Expr] }] }, S) ->
-  { TExpr, SE } = translate_each(Expr, S),
-  elixir_utils:convert_to_boolean(?line(Meta), TExpr, true, S#elixir_scope.context == guard, SE);
-
-translate({ '!', Meta, [Expr] }, S) ->
-  { TExpr, SE } = translate_each(Expr, S),
-  elixir_utils:convert_to_boolean(?line(Meta), TExpr, false, S#elixir_scope.context == guard, SE);
-
-translate({ in, Meta, [Left, Right] }, #elixir_scope{extra_guards=nil} = S) ->
-  { _, TExpr, TS } = translate_in(Meta, Left, Right, S),
-  { TExpr, TS };
-
-translate({ in, Meta, [Left, Right] }, #elixir_scope{extra_guards=Extra} = S) ->
-  { TVar, TExpr, TS } = translate_in(Meta, Left, Right, S),
-  { TVar, TS#elixir_scope{extra_guards=[TExpr|Extra]} };
-
-%% @
-
-translate({'@', Meta, [{ Name, _, [Arg] }]}, S) when Name == typep; Name == type; Name == spec; Name == callback; Name == opaque ->
-  case elixir_compiler:get_opt(internal) of
-    true  -> { { nil, ?line(Meta) }, S };
-    false ->
-      Call = { { '.', Meta, ['Elixir.Kernel.Typespec', spec_to_macro(Name)] }, Meta, [Arg] },
-      translate_each(Call, S)
-  end;
-
-translate({'@', Meta, [{ Name, _, Args }]}, S) ->
-  assert_module_scope(Meta, '@', S),
-
-  case is_reserved_data(Name) andalso elixir_compiler:get_opt(internal) of
-    true ->
-      { { nil, ?line(Meta) }, S };
-    _ ->
-      case Args of
-        [Arg] ->
-          case S#elixir_scope.function of
-            nil ->
-              translate_each({
-                { '.', Meta, ['Elixir.Module', put_attribute] },
-                  Meta,
-                  [ { '__MODULE__', Meta, false }, Name, Arg ]
-              }, S);
-            _  ->
-              syntax_error(Meta, S#elixir_scope.file,
-                "cannot dynamically set attribute @~ts inside a function", [Name])
-          end;
-        _ when is_atom(Args) or (Args == []) ->
-          case S#elixir_scope.function of
-            nil ->
-              translate_each({
-                { '.', Meta, ['Elixir.Module', get_attribute] },
-                Meta,
-                [ { '__MODULE__', Meta, false }, Name, true ]
-              }, S);
-            _ ->
-              Contents = 'Elixir.Module':get_attribute(S#elixir_scope.module, Name),
-              { elixir_utils:elixir_to_erl(?line(Meta), Contents, S), S }
-          end;
-        _ ->
-          syntax_error(Meta, S#elixir_scope.file, "expected 0 or 1 argument for @~ts, got: ~p", [Name, length(Args)])
-      end
-  end;
-
-%% Case
-
-translate({'case', Meta, [Expr, KV]}, S) when is_list(KV) ->
-  assert_no_match_or_guard_scope(Meta, 'case', S),
-  Clauses = elixir_clauses:get_pairs(Meta, do, KV, S),
-  { TExpr, NS } = translate_each(Expr, S),
-
-  RClauses = case elixir_utils:returns_boolean(TExpr) of
-    true  -> rewrite_case_clauses(Clauses);
-    false -> Clauses
-  end,
-
-  { TClauses, TS } = elixir_clauses:match(Meta, RClauses, NS),
-  { { 'case', ?line(Meta), TExpr, TClauses }, TS };
-
-%% Try
-
-translate({'try', Meta, [Clauses]}, S) when is_list(Clauses) ->
-  assert_no_match_or_guard_scope(Meta, 'try', S),
-
-  Do = proplists:get_value('do', Clauses, nil),
-  { TDo, SB } = elixir_translator:translate_each(Do, S#elixir_scope{noname=true}),
-
-  Catch = [Tuple || { X, _ } = Tuple <- Clauses, X == 'rescue' orelse X == 'catch'],
-  { TCatch, SC } = elixir_try:clauses(Meta, Catch, umergea(S, SB)),
-
-  After = proplists:get_value('after', Clauses, nil),
-  { TAfter, SA } = elixir_translator:translate_each(After, umergea(S, SC)),
-
-  Else = elixir_clauses:get_pairs(Meta, else, Clauses, S),
-  { TElse, SE } = elixir_clauses:match(Meta, Else, umergea(S, SA)),
-
-  SF = (umergec(S, SE))#elixir_scope{noname=S#elixir_scope.noname},
-  { { 'try', ?line(Meta), pack(TDo), TElse, TCatch, pack(TAfter) }, SF };
-
-%% Receive
-
-translate({'receive', Meta, [KV] }, S) when is_list(KV) ->
-  assert_no_match_or_guard_scope(Meta, 'receive', S),
-  Do = elixir_clauses:get_pairs(Meta, do, KV, S, true),
-
-  case lists:keyfind('after', 1, KV) of
-    false ->
-      { TClauses, SC } = elixir_clauses:match(Meta, Do, S),
-      { { 'receive', ?line(Meta), TClauses }, SC };
-    _ ->
-      After = elixir_clauses:get_pairs(Meta, 'after', KV, S),
-      { TClauses, SC } = elixir_clauses:match(Meta, Do ++ After, S),
-      { FClauses, TAfter } = elixir_utils:split_last(TClauses),
-      { _, _, [FExpr], _, FAfter } = TAfter,
-      { { 'receive', ?line(Meta), FClauses, FExpr, FAfter }, SC }
-  end;
-
-%% Definitions
-
-translate({defmodule, Meta, [Ref, KV]}, S) when is_list(KV) ->
-  { TRef, _ } = translate_each(Ref, S),
-
-  Block = case lists:keyfind(do, 1, KV) of
-    { do, DoValue } -> DoValue;
-    false -> syntax_error(Meta, S#elixir_scope.file, "missing do keyword in defmodule")
-  end,
-
-  { FRef, FS } = case TRef of
-    { atom, _, Module } ->
-      FullModule = expand_module(Ref, Module, S),
-
-      RS = case elixir_aliases:nesting_alias(S#elixir_scope.module, FullModule) of
-        { New, Old } -> elixir_aliases:store(Meta, New, Old, S);
-        false -> S
-      end,
-
-      {
-        { atom, Meta, FullModule },
-        RS#elixir_scope{context_modules=[FullModule|S#elixir_scope.context_modules]}
-      };
-    _ ->
-      { TRef, S }
-  end,
-
-  MS = FS#elixir_scope{local=nil},
-  { elixir_module:translate(Meta, FRef, Block, MS), FS };
-
-translate({Kind, Meta, [Call]}, S) when ?defs(Kind) ->
-  translate({Kind, Meta, [Call, nil]}, S);
-
-translate({Kind, Meta, [Call, Expr]}, S) when ?defs(Kind) ->
-  assert_module_scope(Meta, Kind, S),
-  assert_no_function_scope(Meta, Kind, S),
-  { TCall, QC, SC } = elixir_quote:erl_escape(Call, true, S),
-  { TExpr, QE, SE } = elixir_quote:erl_escape(Expr, true, SC),
-  CheckClauses = (not lists:keymember(context, 1, Meta)) andalso
-                   (not QC#elixir_quote.unquoted) andalso (not QE#elixir_quote.unquoted),
-  { elixir_def:wrap_definition(Kind, Meta, TCall, TExpr, CheckClauses, SE), SE };
-
-translate({Kind, Meta, [Name, Args, Guards, Expr]}, S) when ?defs(Kind) ->
-  elixir_errors:deprecation(Meta, S#elixir_scope.file, "~ts/4 is deprecated, please use ~ts/2 instead", [Kind, Kind]),
-  assert_module_scope(Meta, Kind, S),
-  assert_no_function_scope(Meta, Kind, S),
-  { TName, SN }   = translate_each(Name, S),
-  { TArgs, SA }   = translate_each(Args, SN),
-  { TGuards, SG } = translate_each(Guards, SA),
-  { TExpr, SE }   = translate_each(Expr, SG),
-  { elixir_def:wrap_definition(Kind, Meta, TName, TArgs, TGuards, TExpr, false, SE), SE };
-
-%% Apply - Optimize apply by checking what doesn't need to be dispatched dynamically
-
-translate({ apply, Meta, [Left, Right, Args] }, S) when is_list(Args) ->
-  { TLeft,  SL } = translate_each(Left, S),
-  { TRight, SR } = translate_each(Right, umergec(S, SL)),
-  translate_apply(Meta, TLeft, TRight, Args, S, SL, SR);
-
-translate({ apply, Meta, Args }, S) ->
-  { TArgs, NS } = translate_args(Args, S),
-  { ?wrap_call(?line(Meta), erlang, apply, TArgs), NS };
-
-translate({ Name, Meta, Args }, S) ->
-  syntax_error(Meta, S#elixir_scope.file,
-               "invalid arguments for macro ~ts/~B", [Name, length(Args)]).
-
-%% Helpers
-
-translate_in(Meta, Left, Right, S) ->
-  Line = ?line(Meta),
-
-  { TLeft, SL } = case Left of
-    { '_', _, Atom } when is_atom(Atom) ->
-      elixir_scope:build_erl_var(Line, S);
-    _ ->
-      translate_each(Left, S)
-  end,
-
-  { TRight, SR } = translate_each(Right, SL),
-
-  Cache = (S#elixir_scope.context == nil),
-
-  { Var, SV } = case Cache of
-    true  -> elixir_scope:build_erl_var(Line, SR);
-    false -> { TLeft, SR }
-  end,
-
-  { TCache, TExpr } = case TRight of
-    { nil, _ } ->
-      Expr = { atom, Line, false },
-      { Cache, Expr };
-    { cons, _, _, _ } ->
-      [H|T] = elixir_utils:cons_to_list(TRight),
-      Expr = lists:foldr(fun(X, Acc) ->
-        { op, Line, 'orelse', { op, Line, '==', Var, X }, Acc }
-      end, { op, Line, '==', Var, H }, T),
-      { Cache, Expr };
-    { string, _, [H|T] } ->
-      Expr = lists:foldl(fun(X, Acc) ->
-        { op, Line, 'orelse', { op, Line, '==', Var, { integer, Line, X } }, Acc }
-      end, { op, Line, '==', Var, { integer, Line, H } }, T),
-      { Cache, Expr };
-    { tuple, _, [{ atom, _, 'Elixir.Range' }, Start, End] } ->
-      Expr = case { Start, End } of
-        { { K1, _, StartInt }, { K2, _, EndInt } } when ?opt_in_types(K1), ?opt_in_types(K2), StartInt =< EndInt ->
-          increasing_compare(Line, Var, Start, End);
-        { { K1, _, _ }, { K2, _, _ } } when ?opt_in_types(K1), ?opt_in_types(K2) ->
-          decreasing_compare(Line, Var, Start, End);
-        _ ->
-          { op, Line, 'orelse',
-            { op, Line, 'andalso',
-              { op, Line, '=<', Start, End},
-              increasing_compare(Line, Var, Start, End) },
-            { op, Line, 'andalso',
-              { op, Line, '<', End, Start},
-              decreasing_compare(Line, Var, Start, End) } }
-      end,
-      { Cache, Expr };
-    _ ->
-      case Cache of
-        true ->
-          { false, ?wrap_call(Line, 'Elixir.Enum', 'member?', [TRight, TLeft]) };
-        false ->
-          compile_error(Meta, S#elixir_scope.file, "invalid args for operator in, it expects a "
-                        "compile time list or range on the right side when used in guard expressions")
-      end
-  end,
-
-  case TCache of
-    true  -> { Var, { block, Line, [ { match, Line, Var, TLeft }, TExpr ] }, SV };
-    false -> { Var, TExpr, SV }
-  end.
-
-increasing_compare(Line, Var, Start, End) ->
-  { op, Line, 'andalso',
-    { op, Line, '>=', Var, Start },
-    { op, Line, '=<', Var, End } }.
-
-decreasing_compare(Line, Var, Start, End) ->
-  { op, Line, 'andalso',
-    { op, Line, '=<', Var, Start },
-    { op, Line, '>=', Var, End } }.
-
-rewrite_case_clauses([{do,Meta1,[{in,_,[{'_',_,_},[false,nil]]}],False},{do,Meta2,[{'_',_,_}],True}]) ->
-  [{do,Meta1,[false],False},{do,Meta2,[true],True}];
-
-rewrite_case_clauses(Clauses) ->
-  Clauses.
-
-%% defmodule :foo
-expand_module(Raw, _Module, _S) when is_atom(Raw) ->
-  Raw;
-
-%% defmodule Hello
-expand_module({ '__aliases__', _, [H] }, _Module, S) ->
-  elixir_aliases:concat([S#elixir_scope.module, H]);
-
-%% defmodule Hello.World
-expand_module({ '__aliases__', _, _ } = Alias, Module, S) ->
-  case elixir_aliases:expand(Alias, S#elixir_scope.aliases, S#elixir_scope.macro_aliases) of
-    Atom when is_atom(Atom) ->
-      Module;
-    Aliases when is_list(Aliases) ->
-      elixir_aliases:concat([S#elixir_scope.module, Module])
-  end;
-
-%% defmodule Elixir.Hello.World
-expand_module(_Raw, Module, S) ->
-  elixir_aliases:concat([S#elixir_scope.module, Module]).
-
-is_reserved_data(moduledoc) -> true;
-is_reserved_data(doc)       -> true;
-is_reserved_data(_)         -> false.
-
-spec_to_macro(type)     -> deftype;
-spec_to_macro(typep)    -> deftypep;
-spec_to_macro(opaque)   -> defopaque;
-spec_to_macro(spec)     -> defspec;
-spec_to_macro(callback) -> defcallback.
-
-% Pack a list of expressions from a block.
-pack({ 'block', _, Exprs }) -> Exprs;
-pack(Expr)                  -> [Expr].