Release v0.8.2

Implement GenEvent.Behaviour

.gitignore

@@ -1,5 +1,5 @@
 /.eunit/*
-/.full
+/.release
 /lib/*/ebin/*
 /lib/*/tmp
 /lib/*/test/tmp
@@ -10,4 +10,7 @@
 /deps/*
 /ebin
 /rel/elixir
-erl_crash.dump
+erl_crash.dump
+.dialyzer_plt
+.dialyzer.base_plt
+.*.swp

.travis.yml

@@ -1,10 +1,13 @@
 language: erlang
-script: "make compile && make .full test"
+script: "make compile && make test"
 notifications:
   irc: "irc.freenode.org#elixir-lang"
   recipients:
     - jose.valim@plataformatec.com.br
+    - yrashk@gmail.com
 otp_release:
+  - R16B
+  - R15B03
   - R15B02
   - R15B01
   - R15B

CHANGELOG.md

@@ -1,3 +1,170 @@
+# v0.8.2 (2013-04-20)
+
+* enhancements
+  * [ExUnit] Use ANSI escape codes in CLI output
+  * [ExUnit] Include suite run time on CLI results
+  * [ExUnit] Add support to doctests, allowing test cases to be generated from code samples
+  * [File] Add `File.ls` and `File.ls!`
+  * [IEx] Support `pwd` and `cd` helpers
+  * [Kernel] Better error reporting for invalid bitstring generators
+  * [Kernel] Improve meta-programming by allowing `unquote` on `def/2`, `defp/2`, `defmacro/2` and `defmacrop/2`
+  * [Kernel] Add support to R16B new functions: `insert_elem/3` and `delete_elem/2`
+  * [Kernel] Import conflicts are now lazily handled. If two modules import the same functions, it will fail only if the function is invoked
+  * [Mix] Support `--cover` on mix test and `test_coverage` on Mixfiles
+  * [Record] Each record now provides `Record.options` with the options supported by its `new` and `update` functions
+
+* bug fix
+  * [Binary] inspect no longer escapes standalone hash `#`
+  * [IEx] The `h` helper can now retrieve docs for special forms
+  * [Kernel] Record optimizations were not being triggered in functions inside the record module
+  * [Kernel] Aliases defined inside macros should be carried over
+  * [Kernel] Fix a bug where nested records could not use the Record[] syntax
+  * [Path] Fix a bug on `Path.expand` when expanding paths starting with `~`
+
+* deprecations
+  * [Kernel] `setelem/3` is deprecated in favor of `set_elem/3`
+  * [Kernel] `function(:is_atom, 1)` is deprecated in favor of `function(is_atom/1)`
+
+* backwards incompatible changes
+  * [Kernel] `unquote` now only applies to the closest quote. If your code contains a quote that contains another quote that calls unquote, it will no longer work. Use `Macro.escape` instead and pass your quoted contents up in steps, for example:
+
+            quote do
+              quote do: unquote(x)
+            end
+
+      should become:
+
+            quote do
+              unquote(Macro.escape(x))
+            end
+
+# v0.8.1 (2013-02-17)
+
+* enhancements
+  * [ExUnit] Tests can now receive metadata set on setup/teardown callbacks
+  * [ExUnit] Add support to ExUnit.CaseTemplate to share callbacks in between test cases
+  * [IO] Add `IO.ANSI` to make it easy to write ANSI escape codes
+  * [Kernel] Better support for Unicode lists
+  * [Kernel] Reduce variables footprint in `case`/`receive` clauses
+  * [Kernel] Disable native compilation when on_load attributes is present to work around an Erlang bug
+  * [Macro] `Macro.expand` also considers macros from the current `__ENV__` module
+  * [Mix] Improve support for compilation of `.erl` files
+  * [Mix] Add support for compilation of `.yrl` and `.xrl` files
+  * [OptionParser] Switches are now overridden by default but can be kept in order if chosen
+  * [Typespec] Better error reporting for invalid typespecs
+
+* bug fix
+  * [Mix] Allow Mix projects to be generated with just one letter
+
+* backwards incompatible changes
+  * [Kernel] `before_compile` and `after_compile` callbacks now receive the environment as first argument instead of the module
+
+* deprecations
+  * [ExUnit] Explicitly defined test/setup/teardown functions are deprecated
+  * [Kernel] Tidy up and clean `quote` API
+  * [Kernel] Old `:local.(args)` syntax is deprecated
+  * [Process] `Process.self` is deprecated in favor `Kernel.self`
+
+# v0.8.0 (2013-01-28)
+
+* enhancements
+  * [Binary] Support `<< "string" :: utf8 >>` as in Erlang
+  * [Binary] Support `\a` escape character in binaries
+  * [Binary] Support syntax shortcut for specifying size in bit syntax
+  * [CLI] Support `--app` option to start an application and its dependencies
+  * [Dict] Support `put_new` in `Dict` and `Keyword`
+  * [Dict] Add `List.Dict` and a faster `HashDict` implementation
+  * [ExUnit] ExUnit now supports multiple runs in the same process
+  * [ExUnit] Failures in ExUnit now shows a tailored stacktrace
+  * [ExUnit] Introduce `ExUnit.ExpectationError` to provide better error messages
+  * [Kernel] Introduce `Application.Behaviour` to define application module callbacks
+  * [Kernel] Introduce `Supervisor.Behaviour` to define supervisors callbacks
+  * [Kernel] More optimizations were added to Record handling
+  * [Kernel] `?\x` and `?\` are now supported ways to retrieve a codepoint
+  * [Kernel] Octal numbers can now be defined as `0777`
+  * [Kernel] Improve macros hygiene regarding variables, aliases and imports
+  * [Mix] Mix now starts the current application before run, iex, test and friends
+  * [Mix] Mix now provides basic support for compiling `.erl` files
+  * [Mix] `mix escriptize` only generates escript if necessary and accept `--force` and `--no-compile` as options
+  * [Path] Introduce `Path` module to hold filesystem paths related functions
+  * [String] Add `String.capitalize` and `String.slice`
+  * [System] Add `System.tmp_dir`, `System.cwd` and `System.user_home`
+
+* bug fix
+  * [Kernel] `import` with `only` accepts functions starting with underscore
+  * [String] `String.first` and `String.last` return nil for empty binaries
+  * [String] `String.rstrip` and `String.lstrip` now verify if argument is a binary
+  * [Typespec] Support `...` inside typespec's lists
+
+* backwards incompatible changes
+  * [Kernel] The AST now allows metadata to be attached to each node. This means the second item in the AST is no longer an integer (representing the line), but a keywords list. Code that relies on the line information from AST or that manually generate AST nodes need to be properly updated
+
+* deprecations
+  * [Dict] Deprecate `Binary.Dict` and `OrdDict` in favor of `HashDict` and `List.Dict`
+  * [File] Deprecate path related functions in favor of the module `Path`
+  * [Kernel] The `/>` operator has been deprecated in favor of `|>`
+  * [Mix] `Mix.Project.sources` is deprecated in favor of `Mix.Project.config_files`
+  * [Mix] `mix iex` is no longer functional, please use `iex -S mix`
+  * [OptionParser] `:flags` option was deprecated in favor of `:switches` to support many types
+
+# v0.7.2 (2012-12-04)
+
+* enhancements
+  * [CLI] `--debug-info` is now true by default
+  * [ExUnit] Make ExUnit exit happen in two steps allowing developers to add custom `at_exit` hooks
+  * [IEx] Many improvements to helpers functions `h/1`, `s/1` and others
+  * [Kernel] Functions defined with `fn` can now handle many clauses
+  * [Kernel] Raise an error if clauses with different arities are defined in the same function
+  * [Kernel] `function` macro now accepts arguments in `M.f/a` and `f/a` formats
+  * [Macro] Improvements to `Macro.to_binary`
+  * [Mix] Mix now echoes the output as it comes when executing external commands such as git or rebar
+  * [Mix] Mix now validates `application` callback's values
+  * [Record] Record accessors are now optimized and can be up to 6x faster in some cases
+  * [String] Support `\xXX` and `\x{HEX}` escape sequences in strings, char lists and regexes
+
+* bug fix
+  * [Bootstrap] Compiling Elixir source no longer fails if environment variables contain utf-8 entries
+  * [IEx] IEx will now wait for all command line options to be processed before starting
+  * [Kernel] Ensure proper stacktraces when showing deprecations
+
+* deprecations
+  * [Enum] `Enum.qsort` is deprecated in favor of `Enum.sort`
+  * [ExUnit] `assert left in right` is deprecated in favor of `assert left inlist right`
+  * [List] `List.sort` and `List.uniq` have been deprecated in favor of their `Enum` counterparts
+  * [Record] Default-based generated functions are deprecated
+  * [Typespec] Enhancements and deprecations to the `@spec/@callback` and the fun type syntax
+
+# v0.7.1 (2012-11-18)
+
+* enhancements
+  * [IEx] Only show documented functions and also show docs for default generated functions
+  * [IO] Add `IO.binread`, `IO.binwrite` and `IO.binreadline` to handle raw binary file operations
+  * [ExUnit] Add support for user configuration at `HOME/.ex_unit.exs`
+  * [ExUnit] Add support for custom formatters via a well-defined behaviour
+  * [Kernel] Add support for `defrecordp`
+  * [Kernel] Improved dialyzer support
+  * [Kernel] Improved error messages when creating functions with aliases names
+  * [Mix] Improve SCM behaviour to allow more robust integration
+  * [Mix] Changing deps information on `mix.exs` forces users to fetch new dependencies
+  * [Mix] Support (parallel) requires on mix run
+  * [Mix] Support `-q` when running tests to compile only changed files
+  * [String] Support `String.downcase` and `String.upcase` according to Unicode 6.2.0
+  * [String] Add support for graphemes in `String.length`, `String.at` and others
+  * [Typespec] Support `@opaque` as attribute
+  * [Typespec] Define a default type `t` for protocols and records
+  * [Typespec] Add support for the access protocol in typespecs
+
+* bug fix
+  * [Kernel] Fix an issue where variables inside clauses remained unassigned
+  * [Kernel] Ensure `defoverridable` functions can be referred in many clauses
+  * [Kernel] Allow keywords as function names when following a dot (useful when integrating with erlang libraries)
+  * [File] File is opened by default on binary mode instead of utf-8
+
+* deprecations
+  * [Behaviour] `defcallback/1` is deprecated in favor of `defcallback/2` which matches erlang `@callbacks`
+  * [Enum] `Enum.times` is deprecated in favor of using ranges
+  * [System] `halt` moved to `System` module
+
 # v0.7.0 (2012-10-20)
 
 * enhancements
@@ -58,7 +225,7 @@
 
 * deprecations
   * [Access] The semantics of the access protocol were reduced from a broad query API to simple data structure key-based access
-  * [ExUnit] Some assertions are deprecated in favor of simply using `assert()` 
+  * [ExUnit] Some assertions are deprecated in favor of simply using `assert()`
   * [File] `File.read_info` is deprecated in favor of `File.stat`
   * [IO] `IO.print` is deprecated in favor of `IO.write`
   * [Kernel] Deprecate `__LINE__` and `__FUNCTION__` in favor of `__ENV__.line` and `__ENV__.function`
@@ -99,4 +266,4 @@
 
 # v0.5.0 (2012-05-24)
 
-* First official release
+* First official release

LICENSE

@@ -1,4 +1,4 @@
-Copyright 2012 Plataformatec.
+Copyright 2012-2013 Plataformatec.
 
   Licensed under the Apache License, Version 2.0 (the "License");
   you may not use this file except in compliance with the License.

Makefile

@@ -1,31 +1,36 @@
-REBAR:=$(shell echo `pwd`/rebar)
-ELIXIRC:=bin/elixirc --ignore-module-conflict $(ELIXIRC_OPTS)
-ERLC:=erlc -I lib/elixir/include
-ERL:=erl -I lib/elixir/include -noshell -env ERL_LIBS $ERL_LIBS:lib
-FULLFLAG:=.full
-VERSION:=0.7.0
+REBAR := $(shell echo `pwd`/rebar)
+ELIXIRC := bin/elixirc --ignore-module-conflict $(ELIXIRC_OPTS)
+ERLC := erlc -I lib/elixir/include
+ERL := erl -I lib/elixir/include -noshell -pa lib/elixir/ebin
+VERSION := $(strip $(shell cat VERSION))
+INSTALL_PATH := /usr/local
 
-.PHONY: 1
+.PHONY: install compile erlang elixir dialyze test clean docs release_docs release_zip release_erl
 .NOTPARALLEL: compile
 
 #==> Templates
-define TASK_TEMPLATE
+
+define APP_TEMPLATE
 $(1): lib/$(1)/ebin/Elixir-$(2).beam lib/$(1)/ebin/$(1).app
 
 lib/$(1)/ebin/$(1).app:
-	@ cd lib/$(1) && ../../bin/elixir ../../bin/mix compile.app
+	@ cd lib/$(1) && ../../bin/elixir -e "Mix.Server.start_link(:dev)" -r mix.exs -e "Mix.Task.run('compile.app')"
 
-lib/$(1)/ebin/Elixir-$(2).beam: $(wildcard lib/$(1)/lib/*.ex) $(wildcard lib/$(1)/lib/*/*.ex) $(wildcard lib/$(1)/lib/*/*/*.ex) $$(FORCE)
+lib/$(1)/ebin/Elixir-$(2).beam: $(wildcard lib/$(1)/lib/*.ex) $(wildcard lib/$(1)/lib/*/*.ex) $(wildcard lib/$(1)/lib/*/*/*.ex)
 	@ echo "==> $(1) (compile)"
 	@ $$(ELIXIRC) "lib/$(1)/lib/**/*.ex" -o lib/$(1)/ebin
 
 test_$(1): $(1)
 	@ echo "==> $(1) (exunit)"
-	@ cd lib/$(1) && time ../../bin/elixir -r "test/test_helper.exs" -pr "test/**/*_test.exs";
+	@ cd lib/$(1) && ../../bin/elixir -r "test/test_helper.exs" -pr "test/**/*_test.exs";
 endef
 
 #==> Compilation tasks
+
 KERNEL:=lib/elixir/ebin/Elixir-Kernel.beam
+UNICODE:=lib/elixir/ebin/Elixir-String-Unicode.beam
+
+default: compile
 
 compile: lib/elixir/src/elixir.app.src erlang elixir
 
@@ -36,73 +41,107 @@ lib/elixir/src/elixir.app.src: src/elixir.app.src
 erlang:
 	@ cd lib/elixir && $(REBAR) compile
 
-# We need to compile only EEx (without the app)
-# file so we can compile Mix
-elixir: kernel lib/eex/ebin/Elixir-EEx.beam mix ex_unit eex iex
+# Since Mix depends on EEx and EEx depends on
+# Mix, we first compile EEx without the .app
+# file, then mix and then compile eex fully
+elixir: kernel unicode lib/eex/ebin/Elixir-EEx.beam mix ex_unit eex iex
 
-kernel: $(KERNEL)
-$(KERNEL): lib/elixir/lib/*.ex lib/elixir/lib/*/*.ex $(FORCE)
-	@ if [ -f $(KERNEL) ]; then                                 \
-		echo "==> kernel (compile)";                            \
-		$(ELIXIRC) "lib/elixir/lib/**/*.ex" -o lib/elixir/ebin; \
-	else                                                        \
-		echo "==> bootstrap (compile)";                         \
-		$(ERL) -s elixir_compiler core -s erlang halt;          \
+kernel: $(KERNEL) VERSION
+$(KERNEL): lib/elixir/lib/*.ex lib/elixir/lib/*/*.ex
+	@ if [ ! -f $(KERNEL) ]; then                       \
+		echo "==> bootstrap (compile)";                 \
+		$(ERL) -s elixir_compiler core -s erlang halt;  \
 	fi
+	@ echo "==> kernel (compile)";
+	@ $(ELIXIRC) "lib/elixir/lib/**/*.ex" -o lib/elixir/ebin;
 	@ rm -rf lib/elixir/ebin/elixir.app
 	@ cd lib/elixir && $(REBAR) compile
 
-$(eval $(call TASK_TEMPLATE,ex_unit,ExUnit))
-$(eval $(call TASK_TEMPLATE,eex,EEx))
-$(eval $(call TASK_TEMPLATE,mix,Mix))
-$(eval $(call TASK_TEMPLATE,iex,IEx))
+unicode: $(UNICODE)
+$(UNICODE): lib/elixir/priv/unicode.ex lib/elixir/priv/UnicodeData.txt lib/elixir/priv/NamedSequences.txt
+	@ echo "==> unicode (compile)";
+	@ echo "This step can take up to a minute to compile in order to embed the Unicode database"
+	@ $(ELIXIRC) lib/elixir/priv/unicode.ex -o lib/elixir/ebin;
+
+$(eval $(call APP_TEMPLATE,ex_unit,ExUnit))
+$(eval $(call APP_TEMPLATE,eex,EEx))
+$(eval $(call APP_TEMPLATE,mix,Mix))
+$(eval $(call APP_TEMPLATE,iex,IEx))
+
+install: compile
+	@ echo "==> elixir (install)"
+	for dir in lib/*; do \
+		install -m755 -d $(INSTALL_PATH)/lib/elixir/$$dir/ebin; \
+		install -m644 $$dir/ebin/* $(INSTALL_PATH)/lib/elixir/$$dir/ebin; \
+	done
+	install -m755 -d $(INSTALL_PATH)/lib/elixir/bin
+	install -m755 $(filter-out %.bat, $(wildcard bin/*)) $(INSTALL_PATH)/lib/elixir/bin
+	install -m755 -d $(INSTALL_PATH)/bin
+	ln -sf $(INSTALL_PATH)/lib/elixir/bin/* $(INSTALL_PATH)/bin
 
 clean:
 	@ cd lib/elixir && $(REBAR) clean
-	rm -rf .full
 	rm -rf ebin
 	rm -rf lib/*/ebin
 	rm -rf lib/*/test/tmp
 	rm -rf lib/mix/test/fixtures/git_repo
-	rm -rf lib/mix/tmp
+	rm -rf lib/*/tmp
+	rm -rf lib/elixir/src/elixir.app.src
+	rm -rf lib/elixir/src/*_lexer.erl
+	rm -rf lib/elixir/src/*_parser.erl
+	rm -rf lib/elixir/test/ebin
 
 #==> Release tasks
-$(FULLFLAG): $(wildcard lib/*/ebin/*)
-	make ELIXIRC_OPTS="--debug-info" FORCE=1
-	touch $(FULLFLAG)
 
-zip: $(FULLFLAG)
-	rm -rf v$(VERSION).zip
-	zip -9 -r v$(VERSION).zip bin CHANGELOG.md LEGAL lib/*/ebin LICENSE README.md rel
-
-docs: $(FULLFLAG)
+docs: compile
 	mkdir -p ebin
 	rm -rf docs
 	cp -R -f lib/*/ebin/*.beam ./ebin
-	bin/elixir ../exdoc/bin/exdoc
+	bin/elixir ../exdoc/bin/exdoc "Elixir" "$(VERSION)" -m Kernel -u "https://github.com/elixir-lang/elixir/blob/master/%{path}#L%{line}"
 	rm -rf ebin
 
+release_zip: compile
+	rm -rf v$(VERSION).zip
+	zip -9 -r v$(VERSION).zip bin CHANGELOG.md LEGAL lib/*/ebin LICENSE README.md rel VERSION
+
 release_docs: docs
 	cd ../elixir-lang.github.com && git checkout master
 	rm -rf ../elixir-lang.github.com/docs/master
-	mv output ../elixir-lang.github.com/docs/master
+	mv docs ../elixir-lang.github.com/docs/master
 
-release_erl: $(FULLFLAG)
+release_erl: compile
 	@ rm -rf rel/elixir
 	@ cd rel && ../rebar generate
 
 #==> Tests tasks
+
 test: test_erlang test_elixir
 
 test_erlang: compile
 	@ echo "==> elixir (eunit)"
 	@ mkdir -p lib/elixir/test/ebin
 	@ $(ERLC) -pa lib/elixir/ebin -o lib/elixir/test/ebin lib/elixir/test/erlang/*.erl
-	@ time $(ERL) -pa lib/elixir/test/ebin -s test_helper test -s erlang halt;
+	@ $(ERL) -pa lib/elixir/test/ebin -s test_helper test -s erlang halt;
 	@ echo
 
-test_elixir: test_kernel test_mix test_ex_unit test_eex test_iex
+test_elixir: test_kernel test_doctest test_mix test_ex_unit test_eex test_iex
+
+test_doctest: compile
+	@ echo "==> doctest (exunit)"
+	@ cd lib/elixir && ../../bin/elixir -r "test/doctest.exs";
 
 test_kernel: compile
 	@ echo "==> kernel (exunit)"
-	@ cd lib/elixir && time ../../bin/elixir -r "test/elixir/test_helper.exs" -pr "test/elixir/**/*_test.exs";
+	@ cd lib/elixir && ../../bin/elixir -r "test/elixir/test_helper.exs" -pr "test/elixir/**/*_test.exs";
+
+.dialyzer.base_plt:
+	@ echo "==> Adding Erlang/OTP basic applications to a new base PLT"
+	@ dialyzer --output_plt .dialyzer.base_plt --build_plt --apps erts kernel stdlib compiler syntax_tools inets crypto ssl
+
+dialyze: .dialyzer.base_plt
+	@ rm -f .dialyzer_plt
+	@ cp .dialyzer.base_plt .dialyzer_plt
+	@ echo "==> Adding Elixir to PLT..."
+	@ dialyzer --plt .dialyzer_plt --add_to_plt -r lib/elixir/ebin lib/ex_unit/ebin lib/mix/ebin lib/iex/ebin lib/eex/ebin
+	@ echo "==> Dialyzing Elixir..."
+	@ dialyzer --plt .dialyzer_plt -r lib/elixir/ebin lib/ex_unit/ebin lib/mix/ebin lib/iex/ebin lib/eex/ebin

README.md

@@ -30,6 +30,10 @@ If you want to contribute, Elixir code is divided in applications inside the `li
 
 * `ex_unit` - Simple test framework that ships with Elixir;
 
+* `iex` — IEx, Elixir's interactive shell
+
+* `mix` — Elixir's build tool
+
 We usually keep a list of features and bugs [in the issue tracker][2].
 
 # Important links

RELEASE.md

@@ -2,26 +2,26 @@
 
 This document simply outlines the release process:
 
-1) Remove .dev extension from current versions
+1) Remove `.dev` extension from VERSION
 
-2) Run `make clean test` to ensure all tests pass from scratch
+2) Run `make clean test` to ensure all tests pass from scratch and the CI is green
 
 3) Ensure CHANGELOG is updated and tag release version with timestamp in it
 
 4) Commit changes above and update stable branch
 
-5) Create tag from stable branch
+5) Create tag from master branch
 
-6) Release new docs, update elixir-lang.org
+6) Release new docs with `make release_docs`, update elixir-lang.org
 
-7) Push new zip to Elixir's downloads page
+7) Release new zip with `make release_zip`, push new zip to Elixir's elixir-lang.org/packages.html
 
-8) After release, bump versions and add .dev back
+8) Push package to expm with `expm publish package.exs`
+
+9) After release, bump versions and add `.dev` back
 
 ## Places where version is mentioned
 
+* VERSION
+* CHANGELOG
 * src/elixir.app.src
-* lib/elixir/lib/system.ex
-* rel/reltool.config
-* Makefile
-* CHANGELOG

VERSION

@@ -0,0 +1 @@
+0.8.2

bin/elixir

@@ -1,18 +1,18 @@
 #!/bin/sh
-if [ $# -eq 0 ]; then
+if [ $# -eq 0 ] || [ $1 = "--help" ] || [ $1 = "-h" ]; then
   echo "Usage: `basename $0` [options] [.exs file] [data]
 
-  -v                Prints version and exit
+  -v                Prints version
   -e \"command\"      Evaluates the given command (*)
   -r \"file\"         Requires the given files/patterns (*)
   -S \"script\"       Finds and executes the given script (*)
   -pr \"file\"        Requires the given files/patterns in parallel (*)
   -pa \"path\"        Prepends the given path to Erlang code path (*)
   -pz \"path\"        Appends the given path to Erlang code path (*)
-  --erl \"switches\"  Switches to be passed down to erlang
+  --app \"app\"       Start the given app and its dependencies (*)
+  --erl \"switches\"  Switches to be passed down to erlang (*)
   --name \"name\"     Makes and assigns a name to the distributed node
   --sname \"name\"    Makes and assigns a short name to the distributed node
-  --remsh \"name\"    Connects to a node using a remote shell (with iex)
   --detached        Starts the Erlang VM detached from console
   --no-halt         Does not halt the Erlang VM after execution
 
@@ -68,10 +68,11 @@ SELF=$(readlink_f "$0")
 SCRIPT_PATH=$(dirname "$SELF")
 
 if [ -f "$HOME/.elixirrc" ]; then . "$HOME/.elixirrc"; fi
+if [ "$ELIXIR_NO_CLI" != "1" ]; then ERL="$ERL -s elixir start_cli"; fi
 
 if [ -f "$SCRIPT_PATH/../releases/RELEASES" ] && [ -f "$SCRIPT_PATH/erl" ]
 then
-  exec "$SCRIPT_PATH"/erl -env ERL_LIBS $ERL_LIBS:"$SCRIPT_PATH/../lib" -boot elixir -noshell $ELIXIR_ERL_OPTS $ERL -s elixir start_cli -extra "$@"
+  exec "$SCRIPT_PATH"/erl -env ERL_LIBS $ERL_LIBS:"$SCRIPT_PATH/../lib" -boot elixir -noshell $ELIXIR_ERL_OPTS $ERL -extra "$@"
 else
-  exec erl -env ERL_LIBS $ERL_LIBS:"$SCRIPT_PATH/../lib" -noshell $ELIXIR_ERL_OPTS $ERL -s elixir start_cli -extra "$@"
+  exec erl -env ERL_LIBS $ERL_LIBS:"$SCRIPT_PATH/../lib" -noshell $ELIXIR_ERL_OPTS $ERL -extra "$@"
 fi

bin/elixir.bat

@@ -1,22 +1,25 @@
 @echo off
-if "%*" == "" (
-	goto documentation
+set argc=0
+for %%x in (%*) do set /A argc+=1
+if %argc% == 0 (
+  goto documentation
 ) else (
-	goto run 
-) 
+  goto run
+)
 :documentation
 echo Usage: %~nx0 [options] [.exs file] [data]
 echo.
-echo  -v              Prints version and exit
+echo  -v            Prints version and exit
 echo  -e command    Evaluates the given command (*)
 echo  -r command    Requires the given file/pattern (*)
 echo  -pr command   Requires the given file/pattern in parallel (*)
-echo  -pa path     Prepend the given path to Erlang code path (*)
+echo  -pa path      Prepend the given path to Erlang code path (*)
 echo  -pz path      Append the given path to Erlang code path (*)
-echo  --no-halt       Do not halt the Erlang VM after execution
+echo  --app app     Start the given app and its dependencies (*)
+echo  --no-halt     Do not halt the Erlang VM after execution
 echo.
 echo ** Options marked with (*) can be given more than once
 echo ** Options given after the .exs file or -- are passed down to the executed code
 echo ** Options can be passed to the erlang runtime using ELIXIR_ERL_OPTS
 :run
-erl -env ERL_LIBS %ERL_LIBS%;"%~dp0\..\lib" -noshell %ELIXIR_ERL_OPTS% -s elixir start_cli -extra %*
+erl -env ERL_LIBS %ERL_LIBS%;"%~dp0\..\lib" -noshell %ELIXIR_ERL_OPTS% -s elixir start_cli -extra %*

bin/elixirc

@@ -2,9 +2,9 @@
 if [ $# -eq 0 ]; then
   echo "Usage: `basename $0` [elixir switches] [compiler switches] [.ex files]
 
-  -o              The directory to output compiled files
-  --no-docs       Do not attach documentation with compiled code
-  --debug-info    Attach debug info to compiled modules
+  -o               The directory to output compiled files
+  --no-docs        Do not attach documentation to compiled modules
+  --no-debug-info  Do not attach debug info to compiled modules
   --ignore-module-conflict
 
 ** Options given after -- are passed down to the executed code

bin/elixirc.bat

@@ -1,22 +1,21 @@
 @echo off
-if "%*" == "" (
-	goto documentation
+set argc=0
+for %%x in (%*) do set /A argc+=1
+if %argc% == 0 (
+  goto documentation
 ) else (
-	goto run 
-) 
+  goto run
+)
 :documentation
-echo Usage: %~nx0 [switches] [.ex files]
+echo Usage: %~nx0 [elixir switches] [compiler switches] [.ex files]
 echo.
-echo  -v              Prints version and exit
-echo  -o              The directory to output compiled files
-echo  -pa path      Prepend the given path to Erlang code path (*)
-echo  -pz path      Append the given path to Erlang code path (*)
-echo  --no-docs       Do not attach documentation with compiled code
-echo  --debug-info    Attach debug info to compiled modules
+echo  -o               The directory to output compiled files
+echo  --no-docs        Do not attach documentation to compiled modules
+echo  --no-debug-info  Do not attach debug info to compiled modules
 echo  --ignore-module-conflict
 echo.
 echo ** Options marked with (*) can be given more than once
 echo ** Options given after -- are passed down to the executed code
 echo ** Options can be passed to the erlang runtime using ELIXIR_ERL_OPTS" >&2
 :run
-call "%~dp0\elixir.bat" --compile %*
+call "%~dp0\elixir.bat" --compile %*

bin/iex

@@ -1,4 +1,26 @@
 #!/bin/sh
+if [ $# -gt 0 ] && ([ $1 = "--help" ] || [ $1 = "-h" ]); then
+  echo "Usage: `basename $0` [options] [.exs file] [data]
+
+  -v                Prints version
+  -e \"command\"      Evaluates the given command (*)
+  -r \"file\"         Requires the given files/patterns (*)
+  -S \"script\"       Finds and executes the given script (*)
+  -pr \"file\"        Requires the given files/patterns in parallel (*)
+  -pa \"path\"        Prepends the given path to Erlang code path (*)
+  -pz \"path\"        Appends the given path to Erlang code path (*)
+  --app \"app\"       Start the given app and its dependencies (*)
+  --erl \"switches\"  Switches to be passed down to erlang (*)
+  --name \"name\"     Makes and assigns a name to the distributed node
+  --sname \"name\"    Makes and assigns a short name to the distributed node
+  --remsh \"name\"    Connects to a node using a remote shell
+
+** Options marked with (*) can be given more than once
+** Options given after the .exs file or -- are passed down to the executed code
+** Options can be passed to the erlang runtime using ELIXIR_ERL_OPTS or --erl" >&2
+  exit 1
+fi
+
 readlink_f () {
   cd "$(dirname "$1")" > /dev/null
   local filename="$(basename "$1")"
@@ -11,4 +33,4 @@ readlink_f () {
 
 SELF=$(readlink_f "$0")
 SCRIPT_PATH=$(dirname "$SELF")
-exec "$SCRIPT_PATH"/elixir --no-halt -e "IEx.cli" "$@"
+ELIXIR_NO_CLI=1 exec "$SCRIPT_PATH"/elixir --no-halt --erl "-user Elixir-IEx-CLI" "$@"

bin/iex.bat

@@ -1,2 +1,2 @@
 @echo off
-call "%~dp0\elixir.bat" --no-halt -e "IEx.cli" %*
+call "%~dp0\elixir.bat" --no-halt -e "IEx.start" %*

bin/mix

@@ -1,3 +1,3 @@
 #!/usr/bin/env elixir
 Mix.start
-Mix.CLI.run
+Mix.CLI.run

bin/mix.bat

@@ -1,2 +1,2 @@
 @echo off
-call "%~dp0\elixir.bat" "%~dp0\mix" %*
+call "%~dp0\elixir.bat" "%~dp0\mix" %*

lib/eex/lib/eex.ex

@@ -189,9 +189,9 @@ defmodule EEx do
 
   @doc false
   def function_from_quoted(module, kind, name, args, source, info) do
-    args  = Enum.map args, fn arg -> { arg, 0, nil } end
+    args  = Enum.map args, fn arg -> { arg, [], nil } end
     quote = quote do
-      unquote(kind).(unquote(name).(unquote_splicing(args)), do: unquote(source))
+      unquote(kind)(unquote(name)(unquote_splicing(args)), do: unquote(source))
     end
     Module.eval_quoted module, quote, [], info
   end

lib/eex/lib/eex/compiler.ex

@@ -68,7 +68,7 @@ defmodule EEx.Compiler do
     else
       key = length(state.dict)
       placeholder = '__EEX__(' ++ integer_to_list(key) ++ ');'
-      { current ++ placeholder ++ new_lines ++ chars, state.prepend_dict([{key, buffer}]) }
+      { current ++ placeholder ++ new_lines ++ chars, state.update_dict([{key, buffer}|&1]) }
     end
   end
 
@@ -90,7 +90,7 @@ defmodule EEx.Compiler do
 
   defp maybe_block([]),    do: nil
   defp maybe_block([h]),   do: h
-  defp maybe_block(other), do: { :__block__, 0, other }
+  defp maybe_block(other), do: { :__block__, [], other }
 
   # Changes placeholder to real expression
 

lib/eex/lib/eex/tokenizer.ex

@@ -112,7 +112,7 @@ defmodule EEx.Tokenizer do
         else
           :middle_expr
         end
-      error ->
+      _error ->
         :middle_expr
     end
   end

lib/eex/test/eex_test.exs

@@ -10,7 +10,7 @@ defmodule EExText.Compiled do
   { :erlang, 1, 2 }.tuple_to_list
   EEx.function_from_string :def, :string_sample, "<%= a + b %>", [:a, :b]
 
-  filename = File.expand_path("../fixtures/eex_template_with_bindings.eex", __FILE__)
+  filename = Path.expand("../fixtures/eex_template_with_bindings.eex", __FILE__)
   EEx.function_from_file :defp, :private_file_sample, filename, [:bar]
   def file_sample(arg), do: private_file_sample(arg)
 
@@ -271,13 +271,13 @@ foo
   end
 
   test "evaluates the source from a given file" do
-    filename = File.expand_path("../fixtures/eex_template.eex", __FILE__)
+    filename = Path.expand("../fixtures/eex_template.eex", __FILE__)
     result = EEx.eval_file(filename)
     assert result == "foo bar.\n"
   end
 
   test "evaluates the source from a given file with bindings" do
-    filename = File.expand_path("../fixtures/eex_template_with_bindings.eex", __FILE__)
+    filename = Path.expand("../fixtures/eex_template_with_bindings.eex", __FILE__)
     result = EEx.eval_file(filename, [bar: 1])
     assert result == "foo 1\n"
   end

lib/elixir/include/elixir.hrl

@@ -1,39 +1,56 @@
--define(ELIXIR_WRAP_CALL(Line, Module, Function, Args),
+-define(wrap_call(Line, Module, Function, Args),
   { call, Line,
     { remote, Line, { atom, Line, Module }, { atom, Line, Function } },
     Args
   }).
 
--define(ELIXIR_ATOM_CONCAT(Atoms), list_to_atom(lists:concat(Atoms))).
--define(ELIXIR_MACRO(Macro), list_to_atom(lists:concat(['MACRO-',Macro]))).
+-define(atom_concat(Atoms), list_to_atom(lists:concat(Atoms))).
+-define(elixir_macro(Macro), list_to_atom(lists:concat(['MACRO-',Macro]))).
+-define(line(Opts), elixir_tree_helpers:get_line(Opts)).
 
 -record(elixir_scope, {
-  context=nil,                                   %% can be assign, guards or nil
-  noname=false,                                  %% when true, don't add new names (used by try)
-  check_clauses=true,                            %% when true, check def clauses ordering
-  super=false,                                   %% when true, it means super was invoked
-  caller=false,                                  %% when true, it means caller was invoked
-  name_args=false,                               %% when true, it means arguments should be named
-  macro=[],                                      %% a stack with macros nesting
-  module=nil,                                    %% the current module
-  function=nil,                                  %% the current function
-  recur=nil,                                     %% the current loop function to be recurred
-  vars=[],                                       %% a dict of defined variables and their alias
-  temp_vars=[],                                  %% a dict of all variables defined in a particular assign
-  clause_vars=[],                                %% a dict of all variables defined in a particular clause
-  quote_vars=[],                                 %% a dict of all quoted variables
-  extra_guards=nil,                              %% extra guards from args expansion
-  counter=0,                                     %% a counter for the variables defined
-  file=(<<"nofile">>),                           %% the current scope filename
-  local=nil,                                     %% the scope to evaluate local functions against
-  aliases=[],                                    %% an orddict with aliases by new -> old names
-  requires=elixir_dispatch:default_requires(),   %% a set with modules required
-  macros=elixir_dispatch:default_macros(),       %% a list with macros imported by module
-  functions=elixir_dispatch:default_functions(), %% a list with functions imported by module
-  scheduled=[]}).                                %% scheduled modules to be loaded
+  context=nil,             %% can be assign, guards or nil
+  noname=false,            %% when true, don't add new names (used by try)
+  check_clauses=true,      %% when true, check def clauses ordering
+  super=false,             %% when true, it means super was invoked
+  caller=false,            %% when true, it means caller was invoked
+  name_args=false,         %% when true, it means arguments should be named
+  module=nil,              %% the current module
+  function=nil,            %% the current function
+  vars=[],                 %% a dict of defined variables and their alias
+  temp_vars=[],            %% a dict of all variables defined in a particular assign
+  clause_vars=nil,         %% a dict of all variables defined in a particular clause
+  extra_guards=nil,        %% extra guards from args expansion
+  counter=[],              %% a counter for the variables defined
+  local=nil,               %% the scope to evaluate local functions against
+  scheduled=[],            %% scheduled modules to be loaded
+  macro_aliases=[],        %% keep aliases defined inside a macro
+  aliases,                 %% an orddict with aliases by new -> old names
+  file,                    %% the current scope filename
+  requires,                %% a set with modules required
+  macro_macros=[],         %% a list with macros imported from module inside a macro
+  macros,                  %% a list with macros imported from module
+  macro_functions=[],      %% a list with functions imported from module inside a macro
+  functions                %% a list with functions imported from module
+}).
 
 -record(elixir_quote, {
   line=0,
-  marker=quoted,
+  vars_hygiene=nil,
+  aliases_hygiene=true,
+  imports_hygiene=true,
   unquote=true
-}).
+}).
+
+%% Used in tokenization and interpolation
+
+-define(is_digit(S), S >= $0 andalso S =< $9).
+-define(is_hex(S), ?is_digit(S) orelse (S >= $A andalso S =< $F) orelse (S >= $a andalso S =< $f)).
+-define(is_bin(S), S >= $0 andalso S =< $1).
+-define(is_octal(S), S >= $0 andalso S =< $7).
+-define(is_leading_octal(S), S >= $0 andalso S =< $3).
+-define(is_upcase(S), S >= $A andalso S =< $Z).
+-define(is_downcase(S), S >= $a andalso S =< $z).
+-define(is_word(S), ?is_digit(S) orelse ?is_upcase(S) orelse ?is_downcase(S)).
+-define(is_quote(S), S == $" orelse S == $').
+-define(is_space(S), S == $\s; S == $\r; S == $\t; S == $\n).

lib/elixir/lib/access.ex

@@ -5,11 +5,10 @@ defprotocol Access do
   The Access protocol is the underlying protocol invoked
   when the brackets syntax is used. For instance, `foo[bar]`
   is translated to `access foo, bar` which, by default,
-  invokes `Access.access` protocol.
+  invokes the `Access.access` protocol.
 
   This protocol is limited and is implemented only for the
-  following built-in types: keywords, records, atoms and
-  functions.
+  following built-in types: keywords, records and functions.
   """
 
   @only [List, Function, Record, Atom]
@@ -17,24 +16,32 @@ defprotocol Access do
   @doc """
   Receives the element being accessed and the access item.
   """
-  def access(element, qualifier)
+  def access(container, key)
 end
 
 defimpl Access, for: List do
   @doc """
-  Access the given key in a keyword list.
+  Access the given key in a tuple list.
 
   ## Examples
 
-      keywords = [a: 1, b: 2]
-      keywords[:a] #=> 1
+      iex> keywords = [a: 1, b: 2]
+      ...> keywords[:a]
+      1
+
+      iex> star_ratings = [{1.0, "★"}, {1.5, "★☆"}, {2.0, "★★"}]
+      ...> star_ratings[1.5]
+      "★☆"
 
   """
+  def access([], _key), do: nil
 
-  def access(list, atom) when is_atom(atom) do
-    Keyword.get(list, atom)
+  def access(list, key) do
+    case :lists.keyfind(key, 1, list) do
+      { ^key, value } -> value
+      false -> nil
+    end
   end
-
 end
 
 defimpl Access, for: Atom do

lib/elixir/lib/application/behaviour.ex

@@ -0,0 +1,105 @@
+defmodule Application.Behaviour do
+  @moduledoc """
+  This module is a convenience to define application module callbacks.
+
+  In Erlang/OTP, an application is a component that can be started
+  and stopped as a unit, and which can be re-used in other systems
+  as well.
+
+  The first step to achieve this is to define an application specification.
+  For example, if your application is named `:my_app`, an app specification
+  should exist at `ebin/my_app.app`. This file is usually defined by
+  build tools like Mix.
+
+  Then, with the app specification in hands, we must also define an
+  application module callback that controls how to start and stop
+  such applications. This module is about defining such callbacks.
+
+  There are two callbacks required to be implemented:
+
+  1. `start(type, args)` - It must return `{ :ok, pid }` or
+     `{ :ok, pid, state }`, where `pid` is the process identifier
+     of the supervisor tree root;
+
+  2. `stop(state)` receives the state returned by `start` and should
+     do any necessary cleaning up. Notice that shutting down the supervisor
+     is automatically handled by the VM;
+
+  When using this module, it simply tags the module behaviour as
+  `:application` and defines a default `stop/1` callback. The `start/2`
+  still needs to be defined by the user.
+
+  You can learn more about the `:application` module, the application
+  specification and the application module callbacks below:
+
+  http://www.erlang.org/doc/man/application.html
+  http://www.erlang.org/doc/design_principles/applications.html
+  http://learnyousomeerlang.com/building-otp-applications
+
+  ## Example
+
+      defmodule MyApp do
+        use Application.Behaviour
+
+        def start(_type, args) do
+          MyApp.Sup.start_link(args)
+        end
+      end
+
+  """
+
+
+  # Starts the given application and all of its dependencies that
+  # have not been started yet recursively.
+  #
+  # ## Supported types
+  #
+  # When starting an application, a type can be given:
+  #
+  # * `:permanent` - If a permanent application terminates, all other
+  #    applications and the runtime system are also terminated;
+  # * `:transient` - If a transient application terminates with reason
+  #    `:normal`, this is reported but no other applications are terminated.
+  #    If a transient application terminates abnormally, all other
+  #    applications and the runtime system are also terminated;
+  # * `:temporary` -  If a temporary application terminates, this is reported
+  #    but no other applications are terminated.
+  #
+  # The type only applies to the application being started. Its dependencies
+  # are all started with default type (which is :temporary).
+  #
+  # Note that transient mode is of little practical use, since when a
+  # supervision tree terminates, the reason is set to shutdown, not normal.
+  #
+  # ## Examples
+  #
+  #     Application.Behaviour.start(:my_app)
+  #
+  @doc false
+  def start(app, type // :temporary) do
+    case :application.start(app, type) do
+      { :error, { :not_started, dep } } ->
+        case start(dep) do
+          :ok   -> start(app, type)
+          other -> other
+        end
+      { :error, { :already_started, _ } } ->
+        :ok
+      other ->
+        other
+    end
+  end
+
+  @doc false
+  defmacro __using__(_) do
+    quote location: :keep do
+      @behavior :application
+
+      def stop(_state) do
+        :ok
+      end
+
+      defoverridable [stop: 1]
+    end
+  end
+end

lib/elixir/lib/behaviour.ex

@@ -12,13 +12,13 @@ defmodule Behaviour do
         use Behaviour
 
         @doc "Parses the given URL"
-        defcallback parse(arg)
+        defcallback parse(uri_info :: URI.Info.t) :: URI.Info.t
 
         @doc "Defines a default port"
-        defcallback default_port()
+        defcallback default_port() :: integer
       end
 
-  And then a specific protocol may use it as:
+  And then a specific module may use it as:
 
       defmodule URI.HTTP do
         @behaviour URI.Parser
@@ -32,46 +32,69 @@ defmodule Behaviour do
 
   ## Implementation
 
-  Internally, Erlang call `behaviour_info(:callbacks)`
-  to obtain all functions that a behaviour should
-  implemented. Therefore, all this module does is
-  to define `behaviour_info(:callbacks)` with the
-  `defcallback` definitions.
+  Behaviours since Erlang R15 must be defined via
+  `@callback` attributes. `defcallback` is a simple
+  mechanism that defines the `@callback` attribute
+  according to the type specification and also allows
+  docs and defines a custom function signature.
+
+  The callbacks and their documentation can be retrieved
+  via the `__behaviour__` callback function.
   """
 
   @doc """
-  Annotates the given function is a callback. `defcallback` is
-  slightly different than simple using `def` because, even if
-  `defcallback` contains default values, a default function
-  won't be generated, which would happen with `def`.
+  Defines a callback according to the given type specification.
   """
+  defmacro defcallback({ :::, _, [fun, return] }) do
+    do_defcallback(fun, return, __CALLER__)
+  end
+
   defmacro defcallback(fun) do
-    { name, args } = :elixir_clauses.extract_args(fun)
-    length = length(args)
+    do_defcallback(fun, quote(do: term), __CALLER__)
+  end
 
-    { args, defaults } = Enum.map_reduce(args, 0, function do
-      { ://, _, [expr, _] }, acc ->
-        { expr, acc + 1 }
-      expr, acc ->
-        { expr, acc }
-    end)
+  defp do_defcallback(fun, return, caller) do
+    case Macro.extract_args(fun) do
+      { name, args } -> :ok
+      :error ->
+        raise ArgumentError, message: "invalid syntax in defcallback #{Macro.to_binary(fun)}"
+    end
 
-    attributes = Enum.map (length - defaults)..length, fn(i) ->
-      quote do
-        @__behaviour_callbacks { unquote(name), unquote(i) }
-      end
+    arity = length(args)
+
+    Enum.each args, fn
+      { :::, _, [left, right] } ->
+        ensure_not_default(left)
+        ensure_not_default(right)
+        left
+      other ->
+        ensure_not_default(other)
+        other
     end
 
     quote do
-      __block__ unquote(attributes)
-      def unquote(name)(unquote_splicing(args))
+      @callback unquote(name)(unquote_splicing(args)) :: unquote(return)
+      Behaviour.store_docs __MODULE__, unquote(caller.line), unquote(name), unquote(arity)
     end
   end
 
+  defp ensure_not_default({ ://, _, [_, _] }) do
+    raise ArgumentError, message: "default arguments // not supported in defcallback"
+  end
+
+  defp ensure_not_default(_), do: :ok
+
+  @doc false
+  def store_docs(module, line, name, arity) do
+    doc = Module.get_attribute module, :doc
+    Module.delete_attribute module, :doc
+    Module.put_attribute module, :__behaviour_docs, { { name, arity }, line, doc }
+  end
+
   @doc false
   defmacro __using__(_) do
     quote do
-      Module.register_attribute(__MODULE__, :__behaviour_callbacks, accumulate: true)
+      Module.register_attribute(__MODULE__, :__behaviour_docs, accumulate: true)
       @before_compile unquote(__MODULE__)
       import unquote(__MODULE__)
     end
@@ -81,8 +104,12 @@ defmodule Behaviour do
   defmacro __before_compile__(_) do
     quote do
       @doc false
-      def behaviour_info(:callbacks) do
-        @__behaviour_callbacks
+      def __behaviour__(:callbacks) do
+        __MODULE__.behaviour_info(:callbacks)
+      end
+
+      def __behaviour__(:docs) do
+        @__behaviour_docs
       end
     end
   end

lib/elixir/lib/binary/chars.ex

@@ -51,8 +51,10 @@ defimpl Binary.Chars, for: List do
 
   ## Examples
 
-      to_binary 'foo' #=> "foo"
-      to_binary ["foo", 'bar'] #=> "foobar"
+      iex> to_binary 'foo'
+      "foo"
+      iex> to_binary ["foo", 'bar']
+      "foobar"
 
   """
   def to_binary(thing) do

lib/elixir/lib/binary/dict.ex

@@ -1,81 +0,0 @@
-defmodule Binary.Dict do
-  @moduledoc """
-  This module implements a dictionary that forces the keys to be
-  converted to binaries on insertion. Currently it is implemented
-  using an `OrdDict`, but this may change in the future.
-
-  Check the `Dict` module for examples and documentation.
-  """
-
-  use Dict.Common
-
-  defmacrop dict(data) do
-    quote do
-      { Binary.Dict, unquote(data) }
-    end
-  end
-
-  def keys(dict(data)) do
-    lc { k, _ } inlist data, do: k
-  end
-
-  def values(dict(data)) do
-    lc { _, v } inlist data, do: v
-  end
-
-  def size(dict(data)) do
-    length(data)
-  end
-
-  def has_key?(dict(data), key) do
-    :orddict.is_key to_binary(key), data
-  end
-
-  def get(dict(data), key, default) do
-    case :orddict.find(to_binary(key), data) do
-      {:ok, value} -> value
-      :error       -> default
-    end
-  end
-
-  def put(dict(data), key, value) do
-    dict(:orddict.store to_binary(key), value, data)
-  end
-
-  def delete(dict(data), key) do
-    dict(:orddict.erase to_binary(key), data)
-  end
-
-  def merge(dict(d1), dict(d2), fun) do
-    dict(:orddict.merge fun, d1, d2)
-  end
-
-  def merge(dict(_) = d1, d2, fun) do
-    merge(d1, new(d2), fun)
-  end
-
-  def update(dict(data), key, fun) do
-    dict(:orddict.update to_binary(key), fun, data)
-  end
-
-  def update(dict(data), key, initial, fun) do
-    dict(:orddict.update to_binary(key), fun, initial, data)
-  end
-
-  def empty(_) do
-    dict([])
-  end
-
-  def to_list(dict(data)) do
-    data
-  end
-end
-
-defimpl Enum.Iterator, for: Binary.Dict do
-  def iterator({ Binary.Dict, data }), do: data
-  def count({ Binary.Dict, data }),    do: length(data)
-end
-
-defimpl Access, for: Binary.Dict do
-  def access(dict, key), do: Binary.Dict.get(dict, key, nil)
-end

lib/elixir/lib/binary/inspect.ex

@@ -11,7 +11,7 @@ defprotocol Binary.Inspect do
   printing.
   """
 
-  @only [BitString, List, Tuple, Atom, Number, Any]
+  @only [BitString, List, Tuple, Atom, Number, Function, PID, Port, Reference]
 
   def inspect(thing, opts)
 end
@@ -54,45 +54,52 @@ defmodule Binary.Inspect.Utils do
 
   ## escape
 
-  # It is considerably faster to loop the binary
-  # and convert it to a list as we go compared
-  # to looping the binary and creating a binary
-  # as we go.
   def escape(other, char) do
-    list_to_binary [char|do_escape(other, char)]
+    b = do_escape(other, char, <<>>)
+    << char, b :: binary, char >>
   end
 
-  defp do_escape(<<char, t :: binary>>, char) do
-    [?\\, char | do_escape(t, char)]
+  @compile {:inline, do_escape: 3}
+  defp do_escape(<<>>, _char, binary), do: binary
+  defp do_escape(<< char, t :: binary >>, char, binary) do
+    do_escape(t, char, << binary :: binary, ?\\, char >>)
   end
-
-  defp do_escape(<<h, t :: binary>>, char) when
-    h == ?#  or h == ?\b or
-    h == ?\d or h == ?\e or
-    h == ?\f or h == ?\n or
-    h == ?\r or h == ?\\ or
-    h == ?\t or h == ?\v do
-    [?\\, escape_map(h) | do_escape(t, char)]
+  defp do_escape(<<?#, ?{, t :: binary>>, char, binary) do
+    do_escape(t, char, << binary :: binary, ?\\, ?#, ?{ >>)
   end
-
-  defp do_escape(<<h, t :: binary>>, char) do
-    [h | do_escape(t,char)]
+  defp do_escape(<<?\a, t :: binary>>, char, binary) do
+    do_escape(t, char, << binary :: binary, ?\\, ?a >>)
   end
-
-  defp do_escape(<<>>, char) do
-    [char]
+  defp do_escape(<<?\b, t :: binary>>, char, binary) do
+    do_escape(t, char, << binary :: binary, ?\\, ?b >>)
+  end
+  defp do_escape(<<?\d, t :: binary>>, char, binary) do
+    do_escape(t, char, << binary :: binary, ?\\, ?d >>)
+  end
+  defp do_escape(<<?\e, t :: binary>>, char, binary) do
+    do_escape(t, char, << binary :: binary, ?\\, ?e >>)
+  end
+  defp do_escape(<<?\f, t :: binary>>, char, binary) do
+    do_escape(t, char, << binary :: binary, ?\\, ?f >>)
+  end
+  defp do_escape(<<?\n, t :: binary>>, char, binary) do
+    do_escape(t, char, << binary :: binary, ?\\, ?n >>)
+  end
+  defp do_escape(<<?\r, t :: binary>>, char, binary) do
+    do_escape(t, char, << binary :: binary, ?\\, ?r >>)
+  end
+  defp do_escape(<<?\\, t :: binary>>, char, binary) do
+    do_escape(t, char, << binary :: binary, ?\\, ?\\ >>)
+  end
+  defp do_escape(<<?\t, t :: binary>>, char, binary) do
+    do_escape(t, char, << binary :: binary, ?\\, ?t >>)
+  end
+  defp do_escape(<<?\v, t :: binary>>, char, binary) do
+    do_escape(t, char, << binary :: binary, ?\\, ?v >>)
+  end
+  defp do_escape(<<h, t :: binary>>, char, binary) do
+    do_escape(t, char, << binary :: binary, h >>)
   end
-
-  defp escape_map(?#),  do: ?#
-  defp escape_map(?\b), do: ?b
-  defp escape_map(?\d), do: ?d
-  defp escape_map(?\e), do: ?e
-  defp escape_map(?\f), do: ?f
-  defp escape_map(?\n), do: ?n
-  defp escape_map(?\r), do: ?r
-  defp escape_map(?\\), do: ?\\
-  defp escape_map(?\t), do: ?t
-  defp escape_map(?\v), do: ?v
 end
 
 defimpl Binary.Inspect, for: Atom do
@@ -109,9 +116,12 @@ defimpl Binary.Inspect, for: Atom do
 
   ## Examples
 
-      inspect(:foo)    #=> ":foo"
-      inspect(nil)     #=> "nil"
-      inspect(Foo.Bar) #=> "Foo.Bar"
+      iex> inspect(:foo)
+      ":foo"
+      iex> inspect(nil)
+      "nil"
+      iex> inspect(Foo.Bar)
+      "Foo.Bar"
 
   """
 
@@ -128,8 +138,7 @@ defimpl Binary.Inspect, for: Atom do
       valid_atom_identifier?(binary) ->
         ":" <> binary
       valid_ref_identifier?(binary) ->
-        "Elixir-" <> rest = binary
-        bc <<r>> inbits rest, do: <<to_dot(r)>>
+        Module.to_binary(atom)
       atom in Macro.binary_ops or atom in Macro.unary_ops ->
         ":" <> binary
       true ->
@@ -139,9 +148,6 @@ defimpl Binary.Inspect, for: Atom do
 
   # Detect if atom is an atom alias (Elixir-Foo-Bar-Baz)
 
-  defp to_dot(?-), do: ?.
-  defp to_dot(l),  do: l
-
   defp valid_ref_identifier?("Elixir" <> rest) do
     valid_ref_piece?(rest)
   end
@@ -188,8 +194,10 @@ defimpl Binary.Inspect, for: BitString do
 
   ## Examples
 
-      inspect("bar")   #=> "bar"
-      inspect("f\"oo") #=> "f\"oo"
+      iex> inspect("bar")
+      "\"bar\""
+      iex> inspect("f\"oo")
+      "\"f\\\"oo\""
 
   """
 
@@ -253,9 +261,12 @@ defimpl Binary.Inspect, for: List do
 
   ## Examples
 
-      inspect('bar')       #=> 'bar'
-      inspect([0|'bar'])   #=> "[0,98,97,114]"
-      inspect([:foo,:bar]) #=> "[:foo, :bar]"
+      iex> inspect('bar')
+      "'bar'"
+      iex> inspect([0|'bar'])
+      "[0,98,97,114]"
+      iex> inspect([:foo,:bar])
+      "[:foo,:bar]"
 
   """
 
@@ -263,22 +274,15 @@ defimpl Binary.Inspect, for: List do
 
   def inspect(thing, opts) do
     cond do
-      printable?(thing) ->
-        escape(list_to_binary(thing), ?')
-      keywords?(thing) ->
+      :io_lib.printable_list(thing) ->
+        escape(:unicode.characters_to_binary(thing), ?')
+      Keyword.keyword?(thing) ->
         "[" <> join_keywords(thing, opts) <> "]"
       true ->
         container_join(thing, "[", "]", opts)
     end
   end
 
-  ## keywords?
-  defp keywords?([]), do: true
-  defp keywords?([{ key, _value } | rest]) when is_atom(key) do
-    keywords?(rest)
-  end
-  defp keywords?(_other), do: false
-
   defp join_keywords(thing, opts) do
     Enum.join(lc {key, value} inlist thing do
       key_to_binary(key, opts) <> ": " <> Binary.Inspect.inspect(value, opts)
@@ -291,19 +295,6 @@ defimpl Binary.Inspect, for: List do
       other -> other
     end
   end
-
-  ## printable?
-
-  defp printable?([c|cs]) when is_integer(c) and c in 32..126 do
-    printable?(cs)
-  end
-
-  defp printable?([c|cs]) when c in [?\n, ?\r, ?\t, ?\v, ?\b, ?\f, ?\e] do
-    printable?(cs)
-  end
-
-  defp printable?([]), do: true
-  defp printable?(_),  do: false
 end
 
 defimpl Binary.Inspect, for: Tuple do
@@ -315,8 +306,10 @@ defimpl Binary.Inspect, for: Tuple do
 
   ## Examples
 
-      inspect({1,2,3})            #=> "{1,2,3}"
-      inspect(ArgumentError.new)  #=> ArgumentError[message: "argument error"]
+      iex> inspect({1,2,3})
+      "{1,2,3}"
+      iex> inspect(ArgumentError.new)
+      "ArgumentError[message: \\\"argument error\\\"]"
 
   """
 
@@ -334,9 +327,7 @@ defimpl Binary.Inspect, for: Tuple do
     name = elem(tuple, 0)
 
     if is_atom(name) and match?("Elixir-" <> _, atom_to_binary(name)) do
-      if name in [BitString, List, Tuple, Atom, Number, Any] do
-        record_inspect(tuple, opts)
-      else
+      unless name in [BitString, List, Tuple, Atom, Number, Any] do
         try do
           target = Module.concat(Binary.Inspect, name)
           target.inspect(tuple, opts)
@@ -348,19 +339,16 @@ defimpl Binary.Inspect, for: Tuple do
     end
   end
 
-  defp record_inspect(exception, opts) when is_exception(exception) do
-    list = tuple_to_list(exception)
-    [name,_|tail] = list
-    [_|fields]    = name.__record__(:fields)
-    record_join(name, fields, tail, opts)
-  end
-
   defp record_inspect(record, opts) do
     list = tuple_to_list(record)
     [name|tail] = list
 
     if (fields = record_fields(name)) && (length(fields) == size(record) - 1) do
-      record_join(name, fields, tail, opts)
+      if Enum.first(tail) == :__exception__ do
+        record_join(name, tl(fields), tl(tail), opts)
+      else
+        record_join(name, fields, tail, opts)
+      end
     end
   end
 
@@ -399,7 +387,8 @@ defimpl Binary.Inspect, for: Number do
 
   ## Examples
 
-      inspect(1) #=> "1"
+      iex> inspect(1)
+      "1"
 
   """
 
@@ -418,27 +407,57 @@ defimpl Binary.Inspect, for: Regex do
 
   ## Examples
 
-      inspect(%r/foo/m) #=> "%r\"foo\"m"
+      iex> inspect(%r/foo/m)
+      "%r\"foo\"m"
 
   """
 
-  def inspect(thing, _) do
-    "%r" <> Binary.Inspect.inspect(Regex.source(thing), []) <> Regex.opts(thing)
+  def inspect(regex, _opts) when size(regex) == 5 do
+    "%r" <> Binary.Inspect.inspect(Regex.source(regex), []) <> Regex.opts(regex)
+  end
+
+  def inspect(other, opts) do
+    Binary.Inspect.inspect other, Keyword.put(opts, :raw, true)
   end
 end
 
-defimpl Binary.Inspect, for: Any do
+defimpl Binary.Inspect, for: Function do
   @moduledoc """
-  For all other terms not implemented, we use the default
-  Erlang representation.
-
-  ## Examples
-
-      inspect Process.self #=> "<0.35.0>"
-
+  Inspect functions, when possible, in a literal form.
   """
 
-  def inspect(thing, _) do
-    iolist_to_binary :io_lib.format('~p', [thing])
+  def inspect(function, _opts) do
+    fun_info = :erlang.fun_info(function)
+    if fun_info[:type] == :external and fun_info[:env] == [] do
+      "function(#{Kernel.inspect(fun_info[:module])}.#{fun_info[:name]}/#{fun_info[:arity]})"
+    else
+      '#Fun' ++ rest = :erlang.fun_to_list(function)
+      "#Function" <> list_to_binary(rest)
+    end
   end
-end
+end
+
+defimpl Binary.Inspect, for: PID do
+  @moduledoc "Inspect PIDs"
+
+  def inspect(pid, _) do
+    "#PID" <> list_to_binary pid_to_list(pid)
+  end
+end
+
+defimpl Binary.Inspect, for: Port do
+  @moduledoc "Inspect ports"
+
+  def inspect(port, _) do
+    list_to_binary :erlang.port_to_list(port)
+  end
+end
+
+defimpl Binary.Inspect, for: Reference do
+  @moduledoc "Inspect references"
+
+  def inspect(ref, _) do
+    '#Ref' ++ rest = :erlang.ref_to_list(ref)
+    "#Reference" <> list_to_binary(rest)
+  end
+end

lib/elixir/lib/bitwise.ex

@@ -6,15 +6,19 @@ defmodule Bitwise do
   The easiest way to use is to simply import them into
   your module:
 
-      use Bitwise
+      iex> use Bitwise
+      ...> bnot 1
+      -2
 
-      bnot 1   #=> -2
-      1 &&& 1  #=> 1
+      iex> use Bitwise
+      ...> 1 &&& 1
+      1
 
   You can select to include only or skip operators by passing options:
 
-      use Bitwise, only_operators: true
-      1 &&& 1 #=> 1
+      iex> use Bitwise, only_operators: true
+      ...> 1 &&& 1
+      1
 
   """
 

lib/elixir/lib/code.ex

@@ -1,4 +1,10 @@
 defmodule Code do
+  defexception LoadError, file: nil do
+    def message(exception) do
+      "could not load #{exception.file}"
+    end
+  end
+
   @moduledoc """
   The Code module is responsible to manage code compilation,
   code evaluation and code loading.
@@ -14,7 +20,7 @@ defmodule Code do
   Returns all the loaded files.
   """
   def loaded_files do
-    server_call :loaded
+    :elixir_code_server.call :loaded
   end
 
   @doc """
@@ -24,56 +30,78 @@ defmodule Code do
   allowing it to be required again.
   """
   def unload_files(files) do
-    server_cast { :unload_files, files }
+    :elixir_code_server.cast { :unload_files, files }
   end
 
   @doc """
   Appends a path to Erlang VM code path.
-  The path is expanded with `File.expand_path` before added.
+  The path is expanded with `Path.expand` before added.
   """
   def append_path(path) do
-    :code.add_pathz(File.expand_path to_char_list(path))
+    :code.add_pathz(Path.expand to_char_list(path))
   end
 
   @doc """
   Prepends a path to Erlang VM code path.
-  The path is expanded with `File.expand_path` before added.
+  The path is expanded with `Path.expand` before added.
   """
   def prepend_path(path) do
-    :code.add_patha(File.expand_path to_char_list(path))
+    :code.add_patha(Path.expand to_char_list(path))
   end
 
   @doc """
   Deletes a path from Erlang VM code path.
-  The path is expanded with `File.expand_path` before deleted.
+  The path is expanded with `Path.expand` before deleted.
   """
   def delete_path(path) do
-    :code.del_path(File.expand_path to_char_list(path))
+    :code.del_path(Path.expand to_char_list(path))
   end
 
   @doc """
-  Evalutes the contents given by string. The second argument is the binding
-  (which should be a Keyword) followed by a keyword list of options. The
-  options can be:
+  Evalutes the contents given by string. The second argument is the
+  binding (which should be a keyword) followed by a keyword list of
+  environment options. Those options can be:
 
   * `:file` - the file to be considered in the evaluation
   * `:line` - the line the script starts
   * `:delegate_locals_to` - delegate local calls to the given module,
-    otherwise functions are evaluated inside Erlang's default scope.
+    the default is to not delegate
+
+  Besides, the following scope values can be configured:
+
+  * `:aliases` - a list of tuples with the alias and its target
+  * `:requires` - a list of modules required
+  * `:functions` - a list of tuples where the first element is a module
+    and the second a list of imported function names and arity
+  * `:macros` - a list of tuples where the first element is a module
+    and the second a list of imported macro names and arity
+
+  Notice that setting any ov the values above overrides Elixir default
+  values. For example, setting `:requires` to `[]`, will no longer
+  automatically required the `Kernel` module, in the same way setting
+  `:macros` will no longer auto-import `Kernel` macros as `if`, `case`,
+  etc.
 
   ## Examples
 
-      Code.eval "a + b", [a: 1, b: 2], file: __ENV__.file, line: __ENV__.line
-      #=> { 3, [ {:a, 1}, {:b, 2} ] }
+      iex> Code.eval "a + b", [a: 1, b: 2], file: __ENV__.file, line: __ENV__.line
+      { 3, [ {:a, 1}, {:b, 2} ] }
 
-  When passing the __ENV__'s file and line, we could simply get
-  the location which already returns both fields as a keyword list:
+  For convenience, you can my pass `__ENV__` as argument and
+  all imports, requires and aliases will be automatically carried
+  over:
 
-      Code.eval "a + b", [a: 1, b: 2], __ENV__.location
-      #=> { 3, [ {:a, 1}, {:b, 2} ] }
+      iex> Code.eval "a + b", [a: 1, b: 2], __ENV__
+      { 3, [ {:a, 1}, {:b, 2} ] }
 
   """
-  def eval(string, binding // [], opts // []) do
+  def eval(string, binding // [], opts // [])
+
+  def eval(string, binding, Macro.Env[] = env) do
+    eval(string, binding, env.to_keywords)
+  end
+
+  def eval(string, binding, opts) do
     { value, binding, _scope } =
       :elixir.eval :unicode.characters_to_list(string), binding, opts
     { value, binding }
@@ -82,34 +110,27 @@ defmodule Code do
   @doc """
   Evalutes the quoted contents.
 
-  ## Options
-
-  This function accepts a list of options. The supported
-  options are:
-
-  * `:file` - The filename to be used in stacktraces
-    and the file reported in the __ENV__ variable.
-
-  * `:line` - The line reported in the __ENV__ variable.
+  This function accepts a list of environment options.
+  Check `Code.eval` for more information.
 
   ## Examples
 
-      contents = quote hygiene: false, do: a + b
+      iex> contents = quote hygiene: [vars: false], do: a + b
+      ...> Code.eval_quoted contents, [a: 1, b: 2], file: __ENV__.file, line: __ENV__.line
+      { 3, [ {:a, 1}, {:b, 2} ] }
 
-      Code.eval_quoted contents, [a: 1, b: 2], file: __ENV__.file, line: __ENV__.line
-      #=> { 3, [ {:a, 1}, {:b, 2} ] }
+  For convenience, you can my pass `__ENV__` as argument and
+  all options will be automatically extracted from the environment:
 
-  For convenience, you can also pass the current __ENV__ and
-  the proper information will be extracted:
-
-      Code.eval_quoted contents, [a: 1, b: 2], __ENV__
-      #=> { 3, [ {:a, 1}, {:b, 2} ] }
+      iex> contents = quote hygiene: [vars: false], do: a + b
+      ...> Code.eval_quoted contents, [a: 1, b: 2], __ENV__
+      { 3, [ {:a, 1}, {:b, 2} ] }
 
   """
   def eval_quoted(quoted, binding // [], opts // [])
 
   def eval_quoted(quoted, binding, Macro.Env[] = env) do
-    eval_quoted(quoted, binding, env.location)
+    eval_quoted(quoted, binding, env.to_keywords)
   end
 
   def eval_quoted(quoted, binding, opts) do
@@ -119,8 +140,8 @@ defmodule Code do
   end
 
   @doc """
-  Converts the given string to AST. It returns { :ok, ast }
-  if it succeeds, { :error, { line, error, token } } otherwise.
+  Converts the given string to AST. It returns `{ :ok, ast }`
+  if it succeeds, `{ :error, { line, error, token } }` otherwise.
 
   ## Options
 
@@ -161,7 +182,7 @@ defmodule Code do
 
   defp unpack_ast(_line, []),                              do: nil
   defp unpack_ast(_line, [forms]) when not is_list(forms), do: forms
-  defp unpack_ast(line, forms),                            do: { :__block__, line, forms }
+  defp unpack_ast(line, forms),                            do: { :__block__, [line: line], forms }
 
   @doc """
   Loads the given `file`. Accepts `relative_to` as an argument
@@ -177,9 +198,9 @@ defmodule Code do
   """
   def load_file(file, relative_to // nil) when is_binary(file) do
     file = find_file(file, relative_to)
-    server_call { :acquire, file }
+    :elixir_code_server.call { :acquire, file }
     loaded = :elixir_compiler.file file
-    server_cast { :loaded, file }
+    :elixir_code_server.cast { :loaded, file }
     loaded
   end
 
@@ -199,14 +220,14 @@ defmodule Code do
   def require_file(file, relative_to // nil) when is_binary(file) do
     file = find_file(file, relative_to)
 
-    case server_call({ :acquire, file }) do
+    case :elixir_code_server.call({ :acquire, file }) do
       :loaded  ->
         nil
       { :queued, ref }  ->
         receive do { :elixir_code_server, ^ref, :loaded } -> nil end
       :proceed ->
         loaded = :elixir_compiler.file file
-        server_cast { :loaded, file }
+        :elixir_code_server.cast { :loaded, file }
         loaded
     end
   end
@@ -216,7 +237,7 @@ defmodule Code do
   Check compiler_options/1 for more information.
   """
   def compiler_options do
-    server_call :compiler_options
+    :elixir_code_server.call :compiler_options
   end
 
   @doc """
@@ -235,7 +256,7 @@ defmodule Code do
 
   """
   def compiler_options(opts) do
-    server_call { :compiler_options, opts }
+    :elixir_code_server.cast { :compiler_options, opts }
   end
 
   @doc """
@@ -246,7 +267,16 @@ defmodule Code do
   For compiling many files at once, check `Kernel.ParallelCompiler`.
   """
   def compile_string(string, file // "nofile") when is_binary(file) do
-    :elixir_compiler.string :unicode.characters_to_list(string), to_binary(file)
+    :elixir_compiler.string :unicode.characters_to_list(string), file
+  end
+
+  @doc """
+  Compiles the quoted expression and returns a list of tuples where
+  the first element is the module name and the second one is its
+  binary.
+  """
+  def compile_quoted(quoted, file // "nofile") when is_binary(file) do
+    :elixir_compiler.quoted [quoted], file
   end
 
   @doc """
@@ -319,7 +349,7 @@ defmodule Code do
           :undefined -> error
           _ ->
             try do
-              module.__info__(:self)
+              module.__info__(:module)
               { :module, module }
             rescue
               UndefinedFunctionError -> error
@@ -342,32 +372,16 @@ defmodule Code do
   # Finds the file given the relative_to path.
   # If the file is found, returns its path in binary, fails otherwise.
   defp find_file(file, relative_to) do
-    file = to_binary(file)
-
     file = if relative_to do
-      File.expand_path(file, relative_to)
+      Path.expand(file, relative_to)
     else
-      File.expand_path(file)
+      Path.expand(file)
     end
 
     if File.regular?(file) do
       file
     else
-      prefix = "#{file}.exs"
-      if File.regular?(prefix) do
-        IO.write "[WARNING] Passing a file without .exs extension to Code.load_file or Code.require_file is deprecated, please pass the full name instead\n#{Exception.formatted_stacktrace}"
-        prefix
-      else
-        raise ArgumentError, message: "could not load #{file}"
-      end
+      raise LoadError, file: file
     end
   end
-
-  defp server_call(args) do
-    :gen_server.call(:elixir_code_server, args)
-  end
-
-  defp server_cast(args) do
-    :gen_server.cast(:elixir_code_server, args)
-  end
 end

lib/elixir/lib/dict.ex

@@ -1,36 +1,74 @@
 defmodule Dict do
-  @moduledoc """
+  @moduledoc %B"""
   This module specifies the Dict API expected to be
   implemented by different dictionaries. It also provides
-  functions that redirect to the underlying Dict based on
-  the tuple signature.
-
-  The keyword list used throughout Elixir cannot be
-  manipulated via the Dict module, you must use the
-  Keyword module instead. This distinction is intentional:
-  the Dict module is meant to work on structures that work
-  as storage.
+  functions that redirect to the underlying Dict, allowing
+  a developer to work with different Dict implementations
+  using one API.
 
   To create a new dict, use the `new` functions defined
   by each dict type:
 
-      OrdDict.new [{:a, 1}, {:b, 2}]
       HashDict.new  #=> creates an empty HashDict
 
   For simplicity's sake, in the examples below everytime
   `new` is used, it implies one of the module-specific
-  calls like the two above. Likewise, when the result of
-  a function invocation is shown in the form `[a: 1, b: 2]`,
-  it implies that the returned value is actually of the
-  same dict type as the input one.
+  calls like above. Likewise, when the result of a function
+  invocation is shown in the form `[a: 1, b: 2]`, it implies
+  that the returned value is actually of the same dict type
+  as the input one.
+
+  ## Protocols
+
+  Besides implementing the functions in this module, all
+  dictionaries are also required to implement the `Access`
+  protocol:
+
+      iex> dict = HashDict.new
+      ...> dict = Dict.put dict, :hello, :world
+      ...> dict[:hello]
+      :world
+
+  And also the `Enum.Iterator` protocol, allowing one to write:
+
+      Enum.each dict, fn { k, v } ->
+        IO.puts "#{k}: #{v}"
+      end
 
   """
 
-  @doc false
-  def behaviour_info(:callbacks) do
-    [ delete: 2, empty: 1, get: 3, has_key?: 2,
-      keys: 1, merge: 3, put: 3, size: 1, to_list: 1,
-      update: 3, update: 4, values: 1 ]
+  use Behaviour
+
+  @type key :: any
+  @type value :: any
+  @type t :: tuple | list
+
+  defcallback delete(t, key) :: t
+  defcallback empty(t) :: t
+  defcallback get(t, key) :: value
+  defcallback get(t, key, value) :: value
+  defcallback get!(t, key) :: value | no_return
+  defcallback has_key?(t, key) :: boolean
+  defcallback keys(t) :: list(key)
+  defcallback merge(t, t) :: t
+  defcallback merge(t, t, (key, value, value -> value)) :: t
+  defcallback put(t, key, value) :: t
+  defcallback put_new(t, key, value) :: t
+  defcallback size(t) :: non_neg_integer()
+  defcallback to_list(t) :: list()
+  defcallback update(t, key, (value -> value)) :: t | no_return
+  defcallback update(t, key, value, (value -> value)) :: t
+  defcallback values(t) :: list(value)
+
+  defmacrop target(dict) do
+    quote do
+      cond do
+        is_tuple(unquote(dict)) ->
+          elem(unquote(dict), 0)
+        is_list(unquote(dict)) ->
+          List.Dict
+      end
+    end
   end
 
   @doc """
@@ -40,12 +78,14 @@ defmodule Dict do
 
   ## Examples
 
-      d = new [a: 1, b: 2]
-      Dict.keys d  #=> [:a,:b]
+      iex> d = HashDict.new [a: 1, b: 2]
+      ...> Enum.sort Dict.keys d
+      [:a,:b]
 
   """
+  @spec keys(t) :: [key]
   def keys(dict) do
-    elem(dict, 0).keys(dict)
+    target(dict).keys(dict)
   end
 
   @doc """
@@ -53,12 +93,14 @@ defmodule Dict do
 
   ## Examples
 
-      d = new [a: 1, b: 2]
-      Dict.values d  #=> [1,2]
+      iex> d = HashDict.new [a: 1, b: 2]
+      ...> Enum.sort Dict.values d
+      [1,2]
 
   """
+  @spec values(t) :: [value]
   def values(dict) do
-    elem(dict, 0).values(dict)
+    target(dict).values(dict)
   end
 
   @doc """
@@ -66,12 +108,14 @@ defmodule Dict do
 
   ## Examples
 
-      d = new [a: 1, b: 2]
-      Dict.size d  #=> 2
+      iex> d = HashDict.new [a: 1, b: 2]
+      ...> Dict.size d
+      2
 
   """
+  @spec size(t) :: non_neg_integer
   def size(dict) do
-    elem(dict, 0).size(dict)
+    target(dict).size(dict)
   end
 
   @doc """
@@ -79,13 +123,18 @@ defmodule Dict do
 
   ## Examples
 
-      d = new [a: 1]
-      Dict.has_key?(d, :a)  #=> true
-      Dict.has_key?(d, :b)  #=> false
+      iex> d = HashDict.new [a: 1]
+      ...> Dict.has_key?(d, :a)
+      true
+
+      iex> d = HashDict.new [a: 1]
+      ...> Dict.has_key?(d, :b)
+      false
 
   """
+  @spec has_key?(t, key) :: boolean
   def has_key?(dict, key) do
-    elem(dict, 0).has_key?(dict, key)
+    target(dict).has_key?(dict, key)
   end
 
   @doc """
@@ -94,14 +143,40 @@ defmodule Dict do
 
   ## Examples
 
-      d = new [a: 1]
-      Dict.get d, :a     #=> 1
-      Dict.get d, :b     #=> nil
-      Dict.get d, :b, 3  #=> 3
+      iex> d = HashDict.new [a: 1]
+      ...> Dict.get d, :a
+      1
+
+      iex> d = HashDict.new [a: 1]
+      ...> Dict.get d, :b
+      nil
+
+      iex> d = HashDict.new [a: 1]
+      ...> Dict.get d, :b, 3
+      3
+  """
+  @spec get(t, key, value) :: value
+  def get(dict, key, default // nil) do
+    target(dict).get(dict, key, default)
+  end
+
+  @doc """
+  Returns the value associated with `key` in `dict`. If `dict` does not
+  contain `key`, it raises `KeyError`.
+
+  ## Examples
+
+      iex> d = HashDict.new [a: 1]
+      ...> Dict.get d, :a
+      1
+      iex> d = HashDict.new [a: 1]
+      ...> Dict.get! d, :b
+      ** (KeyError) key not found: :b
 
   """
-  def get(dict, key, default // nil) do
-    elem(dict, 0).get(dict, key, default)
+  @spec get!(t, key) :: value | no_return
+  def get!(dict, key) do
+    target(dict).get!(dict, key)
   end
 
   @doc """
@@ -110,13 +185,31 @@ defmodule Dict do
 
   ## Examples
 
-      d = new [a: 1, b: 2]
-      Dict.put d, :a, 3
-      #=> [a: 3, b: 2]
+      iex> d = HashDict.new [a: 1, b: 2]
+      ...> d = Dict.put d, :a, 3
+      ...> Dict.get d, :a
+      3
 
   """
+  @spec put(t, key, value) :: t
   def put(dict, key, val) do
-    elem(dict, 0).put(dict, key, val)
+    target(dict).put(dict, key, val)
+  end
+
+  @doc """
+  Puts the given `value` under `key` in `dict` unless `key` already exists.
+
+  ## Examples
+
+      iex> d = HashDict.new [a: 1, b: 2]
+      ...> d = Dict.put_new d, :a, 3
+      ...> Dict.get d, :a
+      1
+
+  """
+  @spec put_new(t, key, value) :: t
+  def put_new(dict, key, val) do
+    target(dict).put_new(dict, key, val)
   end
 
   @doc """
@@ -125,51 +218,58 @@ defmodule Dict do
 
   ## Examples
 
-      d = new [a: 1, b: 2]
-      Dict.delete d, :a      #=> [b: 2]
+      iex> d = HashDict.new [a: 1, b: 2]
+      ...> d = Dict.delete d, :a
+      ...> Dict.get d, :a
+      nil
 
-      d = new [b: 2]
-      Dict.delete d, :a      #=> [b: 2]
+      iex> d = HashDict.new [b: 2]
+      ...> Dict.delete(d, :a) == d
+      true
 
   """
+  @spec delete(t, key) :: t
   def delete(dict, key) do
-    elem(dict, 0).delete(dict, key)
+    target(dict).delete(dict, key)
   end
 
   @doc """
-  Merges two dicts into one. If the dicts have duplicated entries,
-  the one given as second argument wins. In case the second argument
-  is not of the same kind as the first one, it is converted to the
-  same kind before merging as long as it implements the `Enum` protocol.
+  Merges the given enum into the dict. In case one of the enum entries
+  alread exist in the dict, it is given higher preference.
 
   ## Examples
 
-      d1 = new [a: 1, b: 2]
-      d2 = new [a: 3, d: 4]
-      Dict.merge d1, d2
-      #=> [a: 3, b: 2, d: 4]
+      iex> d1 = HashDict.new [a: 1, b: 2]
+      ...> d2 = HashDict.new [a: 3, d: 4]
+      ...> d = Dict.merge d1, d2
+      ...> [a: Dict.get(d, :a), b: Dict.get(d, :b), d: Dict.get(d, :d)]
+      [a: 3, b: 2, d: 4]
 
   """
-  def merge(dict1, dict2) do
-    merge(dict1, dict2, fn(_k, _v1, v2) -> v2 end)
+  @spec merge(t, t) :: t
+  def merge(dict, enum) do
+    merge(dict, enum, fn(_k, _v1, v2) -> v2 end)
   end
 
   @doc """
-  Merges two dicts into one. If the dicts have duplicated entries, the given
-  function is invoked to solve conflicts.
+  Merges the given enum into the dict. In case one of the enum entries
+  alread exist in the dict, the given function is invoked to solve
+  conflicts.
 
   ## Examples
 
-      d1 = new [a: 1, b: 2]
-      d2 = new [a: 3, d: 4]
-      Dict.merge d1, d2, fn _k, v1, v2 ->
-        v1 + v2
-      end
-      #=> [a: 4, b: 2, d: 4]
+      iex> d1 = HashDict.new [a: 1, b: 2]
+      ...> d2 = HashDict.new [a: 3, d: 4]
+      ...> d = Dict.merge d1, d2, fn _k, v1, v2 ->
+      ...>   v1 + v2
+      ...> end
+      ...> [a: Dict.get(d, :a), b: Dict.get(d, :b), d: Dict.get(d, :d)]
+      [a: 4, b: 2, d: 4]
 
   """
-  def merge(dict1, dict2, fun) do
-    elem(dict1, 0).merge(dict1, dict2, fun)
+  @spec merge(t, t, (key, value, value -> value)) :: t
+  def merge(dict, enum, fun) do
+    target(dict).merge(dict, enum, fun)
   end
 
   @doc """
@@ -178,13 +278,15 @@ defmodule Dict do
 
   ## Examples
 
-      d = new [a: 1, b: 2]
-      Dict.update d, :a, fn val -> -val end
-      #=> [a: -1, b: 2]
+      iex> d = HashDict.new [a: 1, b: 2]
+      ...> d = Dict.update d, :a, fn val -> -val end
+      ...> Dict.get d, :a
+      -1
 
   """
+  @spec update(t, key, (value -> value)) :: t
   def update(dict, key, fun) do
-    elem(dict, 0).update(dict, key, fun)
+    target(dict).update(dict, key, fun)
   end
 
   @doc """
@@ -194,27 +296,31 @@ defmodule Dict do
 
   ## Examples
 
-      d = new [a: 1, b: 2]
-      Dict.update d, :c, 3, fn val -> -val end
-      #=> [a: 1, b: 2, c: 3]
+      iex> d = HashDict.new [a: 1, b: 2]
+      ...> d = Dict.update d, :c, 3, fn val -> -val end
+      ...> Dict.get d, :c
+      3
 
   """
+  @spec update(t, key, value, (value -> value)) :: t
   def update(dict, key, initial, fun) do
-    elem(dict, 0).update(dict, key, initial, fun)
+    target(dict).update(dict, key, initial, fun)
   end
 
   @doc """
   Returns an empty dict of the same type as `dict`.
   """
+  @spec empty(t) :: t
   def empty(dict) do
-    elem(dict, 0).empty(dict)
+    target(dict).empty(dict)
   end
 
   @doc """
   Returns a list of key-value pairs stored in `dict`.
   No particular order is enforced.
   """
+  @spec to_list(t) :: list
   def to_list(dict) do
-    elem(dict, 0).to_list(dict)
+    target(dict).to_list(dict)
   end
 end

lib/elixir/lib/dict/common.ex

@@ -1,47 +0,0 @@
-defmodule Dict.Common do
-  @moduledoc false
-
-  defmacro __using__(_) do
-    quote do
-      @behavior Dict
-
-      @doc """
-      Creates a new empty dict.
-      """
-      def new do
-        empty(nil)
-      end
-
-      @doc """
-      Creates a new dict from a list of pairs.
-
-      ## Examples
-
-          #{inspect(__MODULE__)}.new [{:b,1},{:a,2}]
-          #=> [a: 1, b: 2]
-
-      """
-      def new(pairs) do
-        Enum.reduce pairs, new, fn { k, v }, dict ->
-          put(dict, k, v)
-        end
-      end
-
-      @doc """
-      Creates a new dict from a list of elements with the
-      help of the transformation function.
-
-      ## Examples
-
-          #{inspect(__MODULE__)}.new ["a", "b"], fn x -> {x, x} end
-          #=> ["a": "a", "b": "b"]
-      """
-      def new(list, transform) when is_function(transform) do
-        Enum.reduce list, new(), fn i, dict ->
-          { k, v } = transform.(i)
-          put(dict, k, v)
-        end
-      end
-    end
-  end
-end

lib/elixir/lib/exception.ex

@@ -1,13 +1,14 @@
 defmodule Exception do
   @moduledoc """
   Several convenience functions to work and pretty print
-  exceptions and backtraces.
+  exceptions and stacktraces.
   """
 
-  # Normalize an exception converting Erlang exceptions
-  # to Elixir style exceptions. This is meant to be used
-  # internally.
-  @doc false
+  @doc """
+  Normalizes an exception converting Erlang exceptions
+  to Elixir exceptions. Useful when interfacing Erlang
+  code with Elixir code.
+  """
   def normalize(exception) when is_exception(exception) do
     exception
   end
@@ -41,11 +42,11 @@ defmodule Exception do
   end
 
   def normalize(:undef) do
-    UndefinedFunctionError.new from_stacktrace(System.stacktrace)
+    UndefinedFunctionError.new from_stacktrace(:erlang.get_stacktrace)
   end
 
   def normalize(:function_clause) do
-    FunctionClauseError.new from_stacktrace(System.stacktrace)
+    FunctionClauseError.new from_stacktrace(:erlang.get_stacktrace)
   end
 
   def normalize({ :badarg, payload }) do
@@ -56,7 +57,72 @@ defmodule Exception do
     ErlangError.new original: other
   end
 
-  # Check the given module is a valid record.
+  @doc """
+  Receives a tuple representing a stacktrace entry and formats it.
+  The current working directory may be given as argument, which
+  is used to prettify the stacktrace.
+  """
+  def format_entry(entry, cwd // nil)
+
+  # From Macro.Env.stacktrace
+  def format_entry({ module, :__MODULE__, 0, file_line }, cwd) do
+    "#{format_file_line(file_line, cwd)}#{inspect module} (module)"
+  end
+
+  # From :elixir_compiler
+  def format_entry({ _module, :__MODULE__, 2, file_line }, cwd) do
+    "#{format_file_line(file_line, cwd)}(module)"
+  end
+
+  # From :elixir_compiler
+  def format_entry({ _module, :__FILE__, 2, file_line }, cwd) do
+    "#{format_file_line(file_line, cwd)}(file)"
+  end
+
+  def format_entry({module, fun, arity, file_line}, cwd) do
+    "#{format_file_line(file_line, cwd)}#{format_module_fun_arity(module, fun, arity)}"
+  end
+
+  def format_entry({fun, arity, file_line}, cwd) do
+    "#{format_file_line(file_line, cwd)}#{format_fun_arity(fun, arity)}"
+  end
+
+  @doc """
+  Formats the stacktrace.
+
+  A stacktrace must be given as argument. If not, this function
+  calculates the current stacktrace and formats it. As consequence,
+  the value of `System.stacktrace` is changed.
+  """
+  def format_stacktrace(trace // nil)
+
+  def format_stacktrace(trace) do
+    trace = trace || try do
+      throw(:stacktrace)
+    catch
+      :stacktrace -> Enum.drop(:erlang.get_stacktrace, 1)
+    end
+
+    case trace do
+      [] -> "\n"
+      s  -> "    " <> Enum.map_join(s, "\n    ", format_entry(&1)) <> "\n"
+    end
+  end
+
+  @doc """
+  Prints the current stacktrace to standard output.
+
+  A stacktrace must be given as argument. If not, this function
+  calculates the current stacktrace and formats it. As consequence,
+  the value of `System.stacktrace` is changed.
+  """
+  def print_stacktrace(trace // nil) do
+    IO.write format_stacktrace(trace)
+  end
+
+  ## Helpers
+
+  # Check the given module is a valid exception record.
   @doc false
   def check!(module) do
     unless :erlang.function_exported(module, :message, 1) do
@@ -64,16 +130,27 @@ defmodule Exception do
     end
   end
 
-  @doc """
-  Receives a module, fun and arity and returns a string
-  representing such invocation. Arity may also be a list
-  of arguments. It follows the same syntax as in stacktraces.
-  """
-  def format_module_fun_arity(module, fun, arity) do
-    case inspect(fun) do
-      << ?:, fun :: binary >> -> :ok
-      fun -> :ok
+  # Format fun and arity
+  @doc false
+  def format_fun_arity(fun, arity) do
+    if is_list(arity) do
+      inspected = lc x inlist arity, do: inspect(x)
+      "#{inspect fun}(#{Enum.join(inspected, ", ")})"
+    else
+      "#{inspect fun}/#{arity}"
     end
+  end
+
+  # Receives a module, fun and arity and returns a string
+  # representing such invocation. Arity may also be a list
+  # of arguments. It follows the same syntax as in stacktraces.
+  @doc false
+  def format_module_fun_arity(module, fun, arity) do
+    fun =
+      case inspect(fun) do
+        << ?:, erl :: binary >> -> erl
+        elixir -> elixir
+      end
 
     if is_list(arity) do
       inspected = lc x inlist arity, do: inspect(x)
@@ -83,37 +160,20 @@ defmodule Exception do
     end
   end
 
-  @doc """
-  Returns the stacktrace as a binary formatted as per `format_stacktrace/1`.
-  """
-  def formatted_stacktrace(trace // System.stacktrace) do
-    case trace do
-      [] -> ""
-      s  -> "    " <> Enum.map_join(s, "\n    ", format_stacktrace(&1)) <> "\n"
-    end
+  # Format file and line for exception printing.
+  @doc false
+  def format_file_line(opts, cwd) do
+    format_file_line Keyword.get(opts, :file), Keyword.get(opts, :line), cwd
   end
 
-  @doc """
-  Formats each line in the stacktrace.
-  """
-  def format_stacktrace({module, fun, arity, file_line}) do
-    "#{format_file_line(file_line)}#{format_module_fun_arity(module, fun, arity)}"
-  end
-
-  @doc """
-  Formats file and line information present in stacktraces.
-  Expect them to be given in a keyword list.
-  """
-  def format_file_line(file_line) do
-    format_file_line(Keyword.get(file_line, :file), Keyword.get(file_line, :line))
-  end
-
-  @doc """
-  Formats the given file and line.
-  """
-  def format_file_line(file, line) do
+  def format_file_line(file, line, cwd) do
     if file do
       file = to_binary(file)
+
+      if cwd do
+        file = Path.relative_to(file, cwd)
+      end
+
       if line && line != 0 do
         "#{file}:#{line}: "
       else
@@ -124,8 +184,6 @@ defmodule Exception do
     end
   end
 
-  ## Helpers
-
   defp from_stacktrace([{ module, function, arity, _ }|_]) do
     [module: module, function: function, arity: arity]
   end
@@ -140,25 +198,25 @@ defexception SystemLimitError,  message: "a system limit has been reached"
 
 defexception SyntaxError, [file: nil, line: nil, description: "syntax error"] do
   def message(exception) do
-    "#{Exception.format_file_line(exception.file, exception.line)}#{exception.description}"
+    "#{Exception.format_file_line(exception.file, exception.line, nil)}#{exception.description}"
   end
 end
 
 defexception TokenMissingError, [file: nil, line: nil, description: "expression is incomplete"] do
   def message(exception) do
-    "#{Exception.format_file_line(exception.file, exception.line)}#{exception.description}"
+    "#{Exception.format_file_line(exception.file, exception.line, nil)}#{exception.description}"
   end
 end
 
 defexception CompileError, [file: nil, line: nil, description: "compile error"] do
   def message(exception) do
-    "#{Exception.format_file_line(exception.file, exception.line)}#{exception.description}"
+    "#{Exception.format_file_line(exception.file, exception.line, nil)}#{exception.description}"
   end
 end
 
 defexception BadFunctionError, [actual: nil] do
   def message(exception) do
-    "bad function: #{inspect(exception.actual)}"
+    "expected a function, got: #{inspect(exception.actual)}"
   end
 end
 
@@ -217,7 +275,7 @@ defexception ErlangError, [original: nil] do
   end
 end
 
-defexception Keyword.KeyError, key: nil do
+defexception KeyError, key: nil do
   def message(exception) do
     "key not found: #{inspect exception.key}"
   end

lib/elixir/lib/file.ex

@@ -54,13 +54,20 @@ end
 
 defmodule File do
   @moduledoc """
-  This module contains function to manipulate files,
-  filenames and the filesystem. Many of the functions
-  that interact with the filesystem have their naming
-  based on its UNIX variants. For example, deleting a
-  file is done with `File.rm`. Getting its stats with
-  `File.stat`. If you want to read or write to a file
-  in chunks, check the IO module.
+  This module contains function to manipulate files.
+  Many of the functions that interact with the filesystem
+  have their naming based on its UNIX variants. For
+  example, deleting a file is done with `File.rm`.
+  Getting its stats with `File.stat`.
+
+  In order to write and read files, one must use the
+  functions in the IO module. By default, a file is
+  opened in binary mode which requires the functions
+  `IO.binread`, `IO.binwrite` and `IO.binreadline` to
+  interact with the file. A developer may pass `:utf8`
+  as an option when opening the file and then all other
+  functions from IO are available, since they work directly
+  with Unicode data.
 
   Most of the functions in this module return `:ok`
   or `{ :ok, result }` in case of success, `{ :error, reason }`
@@ -98,40 +105,6 @@ defmodule File do
   alias :filename, as: FN
   alias :filelib,  as: FL
 
-  @doc """
-  Expands the path by returning its absolute name and expanding
-  any `.` and `..` characters.
-
-  If the given `path` is a char list, returns a char list.
-  Otherwise returns a binary.
-
-  ## Examples
-
-      File.expand_path("/foo/bar/../bar") == "/foo/bar"
-
-  """
-  def expand_path(path) do
-    normalize FN.absname(path)
-  end
-
-  @doc """
-  Expands the path to the relative location and expanding
-  any `.` and `..` characters. If the path is already an
-  absolute path, the relative location is ignored.
-
-  If the given `path` is a char list, returns a char list.
-  Otherwise returns a binary.
-
-  ## Examples
-
-      File.expand_path("foo/bar/../bar", "/baz") == "/baz/foo/bar"
-      File.expand_path("/foo/bar/../bar", "/baz") == "/foo/bar"
-
-  """
-  def expand_path(path, relative_to) do
-    normalize FN.absname(FN.absname(path, relative_to))
-  end
-
   @doc """
   Returns true if the path is a regular file.
 
@@ -172,159 +145,6 @@ defmodule File do
     match?({ :ok, _ }, F.read_file_info(path))
   end
 
-  @doc """
-  Returns the last component of the path or the path
-  itself if it does not contain any directory separators.
-
-  If the given `path` is a char list, returns a char list.
-  Otherwise returns a binary.
-
-  ## Examples
-
-      File.basename("foo")
-      #=> "foo"
-
-      File.basename("foo/bar")
-      #=> "bar"
-
-      File.basename("/")
-      #=> ""
-
-  """
-  def basename(path) do
-    FN.basename(path)
-  end
-
-  @doc """
-  Returns the last component of `path` with the `extension`
-  stripped. This function should be used to remove a specific
-  extension which might, or might not, be there.
-
-  If the given `path` is a char list, returns a char list.
-  Otherwise returns a binary.
-
-  ## Examples
-
-      File.basename("~/foo/bar.ex", ".ex")
-      #=> "bar"
-      File.basename("~/foo/bar.exs", ".ex")
-      #=> "bar.exs"
-      File.basename("~/foo/bar.old.ex", ".ex")
-      #=> "bar.old"
-
-  """
-  def basename(path, extension) do
-    FN.basename(path, extension)
-  end
-
-  @doc """
-  Return the `directory` component of `path`.
-
-  If the given `path` is a char list, returns a char list.
-  Otherwise returns a binary.
-
-  ## Examples
-
-      File.dirname("/foo/bar.ex")
-      #=> "foo"
-
-  """
-  def dirname(path) do
-    FN.dirname(path)
-  end
-
-  @doc """
-  Return the `extension` of the last component of `path`.
-
-  If the given `path` is a char list, returns a char list.
-  Otherwise returns a binary.
-
-  ## Examples
-
-      File.extname("foo.erl")
-      #=> ".erl"
-      File.extname("~/foo/bar")
-      #=> ""
-
-  """
-  def extname(path) do
-    FN.extension(path)
-  end
-
-  @doc """
-  Returns the `path` with the `extension` stripped.
-
-  If the given `path` is a char list, returns a char list.
-  Otherwise returns a binary.
-
-  ## Examples
-
-      File.rootname("/foo/bar")
-      #=> "/foo/bar"
-      File.rootname("/foo/bar.ex")
-      #=> "/foo/bar"
-
-  """
-  def rootname(path) do
-    FN.rootname(path)
-  end
-
-  @doc """
-  Returns the `path` with the `extension` stripped. This function should be used to
-  remove a specific extension which might, or might not, be there.
-
-  If the given `path` is a char list, returns a char list.
-  Otherwise returns a binary.
-
-  ## Examples
-
-      File.rootname("/foo/bar.erl", ".erl")
-      #=> "/foo/bar"
-      File.rootname("/foo/bar.erl", ".ex")
-      #=> "/foo/bar.erl"
-
-  """
-  def rootname(path, extension) do
-   FN.rootname(path, extension)
-  end
-
-  @doc """
-  Returns a string with one or more paths components joint by the path separator.
-  This function should be used to convert a list of strings in a path.
-
-  If the given `paths` are a char list, returns a char list.
-  Otherwise returns a binary.
-
-  ## Examples
-
-      File.join(["~", "foo"])
-      #=> "~/foo"
-      File.join(["foo"])
-      #=> "foo"
-      File.join(["/", "foo", "bar"])
-      #=> "/foo/bar"
-
-  """
-  def join(paths) do
-    FN.join(paths)
-  end
-
-  @doc """
-  Join two paths.
-
-  If the given paths are a char list, returns a char list.
-  Otherwise returns a binary.
-
-  ## Examples
-
-      File.join("foo", "bar")
-      #=> "foo/bar"
-
-  """
-  def join(left, right) do
-    FN.join(left, right)
-  end
-
   @doc """
   Tries to create the directory `path`. Missing parent directories are not created.
   Returns `:ok` if successful, or `{:error, reason}` if an error occurs.
@@ -349,7 +169,7 @@ defmodule File do
     case mkdir(path) do
       :ok -> :ok
       { :error, reason } ->
-        raise File.Error, reason: reason, action: "make directory", path: to_binary(path)
+        raise File.Error, reason: reason, action: "make directory", path: :unicode.characters_to_binary(path)
     end
   end
 
@@ -364,7 +184,7 @@ defmodule File do
   * :enotdir - A component of `path` is not a directory.
   """
   def mkdir_p(path) do
-    FL.ensure_dir(join(path, "."))
+    FL.ensure_dir(FN.join(path, "."))
   end
 
   @doc """
@@ -374,7 +194,7 @@ defmodule File do
     case mkdir_p(path) do
       :ok -> :ok
       { :error, reason } ->
-        raise File.Error, reason: reason, action: "make directory (with -p)", path: to_binary(path)
+        raise File.Error, reason: reason, action: "make directory (with -p)", path: :unicode.characters_to_binary(path)
     end
   end
 
@@ -407,70 +227,10 @@ defmodule File do
       { :ok, binary } ->
         binary
       { :error, reason } ->
-        raise File.Error, reason: reason, action: "read file", path: to_binary(path)
+        raise File.Error, reason: reason, action: "read file", path: :unicode.characters_to_binary(path)
     end
   end
 
-  @doc """
-  Returns a list with the path splitted by the path separator.
-  If an empty string is given, then it returns the root path.
-
-  ## Examples
-
-       File.split("")
-       #=> ["/"]
-       File.split("foo")
-       #=> ["foo"]
-       File.split("/foo/bar")
-       #=> ["/", "foo", "bar"]
-
-  """
-  def split(path) do
-    FN.split(path)
-  end
-
-  @doc """
-  Traverses files and directories according to the given `glob` expression.
-
-  The wildcard string looks like an ordinary filename, except that certain
-  "wildcard characters" are interpreted in a special way. The following
-  characters are special:
-
-  * `?` - Matches one character.
-  * `*` - Matches any number of characters up to the end of
-          the filename, the next dot, or the next slash.
-  * `**` - Two adjacent <c>*</c>'s used as a single pattern will
-           match all files and zero or more directories and subdirectories.
-  * `[char1,char2,...]` - Matches any of the characters listed. Two characters
-                          separated by a hyphen will match a range of characters.
-  * `{item1,item2,...}` - Matches one of the alternatives.
-
-  Other characters represent themselves. Only filenames that have exactly
-  the same character in the same position will match. Note that matching
-  is case-sensitive; i.e. "a" will not match "A".
-
-  ## Examples
-
-  Imagine you have a directory called `projects` with three Elixir projects
-  inside of it: `elixir`, `exdoc` and `dynamo`. You can find all `.beam` files
-  inside their ebin directories all projects as follows:
-
-      File.wildcard("projects/*/ebin/**/*.beam")
-
-  If you want to search for both `.beam` and `.app` files, you could do:
-
-      File.wildcard("projects/*/ebin/**/*.{beam,app}")
-
-  """
-  def wildcard(glob) when is_binary(glob) do
-    paths = :elixir_glob.wildcard binary_to_list(glob)
-    Enum.map paths, list_to_binary(&1)
-  end
-
-  def wildcard(glob) when is_list(glob) do
-    :elixir_glob.wildcard(glob)
-  end
-
   @doc """
   Returns information about the `path`. If it exists, it
   returns a `{ :ok, info }` tuple, where info is  as a
@@ -502,7 +262,7 @@ defmodule File do
     case stat(path, opts) do
       {:ok, info}      -> info
       {:error, reason} ->
-        raise File.Error, reason: reason, action: "read file stats", path: to_binary(path)
+        raise File.Error, reason: reason, action: "read file stats", path: :unicode.characters_to_binary(path)
     end
   end
 
@@ -511,7 +271,7 @@ defmodule File do
   path. Returns `:ok` or `{ :error, reason }`.
   """
   def write_stat(path, File.Stat[] = stat, opts // []) do
-    F.write_file_info(path, setelem(stat, 0, :file_info), opts)
+    F.write_file_info(path, set_elem(stat, 0, :file_info), opts)
   end
 
   @doc """
@@ -522,7 +282,7 @@ defmodule File do
     case write_stat(path, stat, opts) do
       :ok -> :ok
       { :error, reason } ->
-        raise File.Error, reason: reason, action: "write file stats", path: to_binary(path)
+        raise File.Error, reason: reason, action: "write file stats", path: :unicode.characters_to_binary(path)
     end
   end
 
@@ -545,7 +305,7 @@ defmodule File do
     case touch(path, time) do
       :ok -> :ok
       { :error, reason } ->
-        raise File.Error, reason: reason, action: "touch", path: to_binary(path)
+        raise File.Error, reason: reason, action: "touch", path: :unicode.characters_to_binary(path)
     end
   end
 
@@ -577,7 +337,7 @@ defmodule File do
       { :ok, bytes_count } -> bytes_count
       { :error, reason } ->
         raise File.CopyError, reason: reason, action: "copy",
-          source: to_binary(source), destination: to_binary(destination)
+          source: :unicode.characters_to_binary(source), destination: :unicode.characters_to_binary(destination)
     end
   end
 
@@ -607,7 +367,7 @@ defmodule File do
       output =
         if dir?(destination) do
           mkdir(destination)
-          join(destination, basename(source))
+          FN.join(destination, FN.basename(source))
         else
           destination
         end
@@ -628,7 +388,7 @@ defmodule File do
       :ok -> :ok
       { :error, reason } ->
         raise File.CopyError, reason: reason, action: "copy recursively",
-          source: to_binary(source), destination: to_binary(destination)
+          source: :unicode.characters_to_binary(source), destination: :unicode.characters_to_binary(destination)
     end
   end
 
@@ -682,7 +442,7 @@ defmodule File do
     output =
       if dir?(destination) || dir?(source) do
         mkdir(destination)
-        join(destination, basename(source))
+        FN.join(destination, FN.basename(source))
       else
         destination
       end
@@ -702,7 +462,7 @@ defmodule File do
       { :ok, files } -> files
       { :error, reason } ->
         raise File.CopyError, reason: reason, action: "copy recursively",
-          source: to_binary(source), destination: to_binary(destination)
+          source: :unicode.characters_to_binary(source), destination: :unicode.characters_to_binary(destination)
     end
   end
 
@@ -718,7 +478,7 @@ defmodule File do
             case mkdir(dest) do
               success in [:ok, { :error, :eexist }] ->
                 Enum.reduce(files, [dest|acc], fn(x, acc) ->
-                  do_cp_r(join(src, x), join(dest, x), callback, acc)
+                  do_cp_r(FN.join(src, x), FN.join(dest, x), callback, acc)
                 end)
               reason -> reason
             end
@@ -807,7 +567,7 @@ defmodule File do
     case F.write_file(path, content, modes) do
       :ok -> :ok
       { :error, reason } ->
-        raise File.Error, reason: reason, action: "write to file", path: to_binary(path)
+        raise File.Error, reason: reason, action: "write to file", path: :unicode.characters_to_binary(path)
     end
   end
 
@@ -826,7 +586,7 @@ defmodule File do
 
   ## Examples
 
-      File.rm('foo.txt')
+      File.rm('file.txt')
       #=> :ok
 
       File.rm('tmp_dir/')
@@ -844,7 +604,7 @@ defmodule File do
     case rm(path) do
       :ok -> :ok
       { :error, reason } ->
-        raise File.Error, reason: reason, action: "remove file", path: to_binary(path)
+        raise File.Error, reason: reason, action: "remove file", path: :unicode.characters_to_binary(path)
     end
   end
 
@@ -857,7 +617,7 @@ defmodule File do
       File.rddir('tmp_dir')
       #=> :ok
 
-      File.rmdir('foo.txt')
+      File.rmdir('file.txt')
       #=> {:error, :enotdir}
 
   """
@@ -872,7 +632,7 @@ defmodule File do
     case rmdir(path) do
       :ok -> :ok
       { :error, reason } ->
-        raise File.Error, reason: reason, action: "remove directory", path: to_binary(path)
+        raise File.Error, reason: reason, action: "remove directory", path: :unicode.characters_to_binary(path)
     end
   end
 
@@ -903,7 +663,7 @@ defmodule File do
       { :ok, files } ->
         res =
           Enum.reduce files, entry, fn(file, tuple) ->
-            do_rm_rf(join(path, file), tuple)
+            do_rm_rf(FN.join(path, file), tuple)
           end
 
         case res do
@@ -945,14 +705,20 @@ defmodule File do
     case rm_rf(path) do
       { :ok, files } -> files
       { :error, reason } ->
-        raise File.Error, reason: reason, action: "remove files and directories recursively from", path: to_binary(path)
+        raise File.Error, reason: reason, action: "remove files and directories recursively from", path: :unicode.characters_to_binary(path)
     end
   end
 
   @doc """
   Opens the given `path` according to the given list of modes.
 
-  By default, the file is opened in read mode, as a binary with utf8 encoding.
+  In order to write and read files, one must use the functions
+  in the IO module. By default, a file is opened in binary mode
+  which requires the functions `IO.binread`, `IO.binwrite` and
+  `IO.binreadline` to interact with the file. A developer may pass
+  `:utf8` as an option when opening the file and then all other
+  functions from IO are available, since they work directly with
+  Unicode data.
 
   The allowed modes:
 
@@ -974,6 +740,12 @@ defmodule File do
                      Note that the file size obtained with `stat/1` will most probably not
                      match the number of bytes that can be read from a compressed file.
 
+  * `:utf8` - This option denotes how data is actually stored in the disk file and
+              makes the file perform automatic translation of characters to and from utf-8.
+              If data is sent to a file in a format that cannot be converted to the utf-8
+              or if data is read by a function that returns data in a format that cannot cope
+              with the character range of the data, an error occurs and the file will be closed.
+
   If a function is given to modes (instead of a list), it dispatches to `open/3`.
 
   Check `http://www.erlang.org/doc/man/file.html#open-2` for more information about
@@ -1000,7 +772,7 @@ defmodule File do
   def open(path, modes // [])
 
   def open(path, modes) when is_list(modes) do
-    F.open(path, open_defaults(modes, true, true))
+    F.open(path, open_defaults(modes, true))
   end
 
   def open(path, function) when is_function(function) do
@@ -1023,7 +795,7 @@ defmodule File do
 
   ## Examples
 
-      File.open!("foo.txt", [:read, :write], fn(file) ->
+      File.open("file.txt", [:read, :write], fn(file) ->
         IO.readline(file)
       end)
 
@@ -1048,7 +820,7 @@ defmodule File do
     case open(path, modes) do
       { :ok, device }    -> device
       { :error, reason } ->
-        raise File.Error, reason: reason, action: "open", path: to_binary(path)
+        raise File.Error, reason: reason, action: "open", path: :unicode.characters_to_binary(path)
     end
   end
 
@@ -1060,7 +832,7 @@ defmodule File do
     case open(path, modes, function) do
       { :ok, device }    -> device
       { :error, reason } ->
-        raise File.Error, reason: reason, action: "open", path: to_binary(path)
+        raise File.Error, reason: reason, action: "open", path: :unicode.characters_to_binary(path)
     end
   end
 
@@ -1072,7 +844,7 @@ defmodule File do
   """
   def cwd() do
     case F.get_cwd do
-      { :ok, cwd } -> { :ok, list_to_binary(cwd) }
+      { :ok, cwd } -> { :ok, :unicode.characters_to_binary(cwd) }
       { :error, _ } = error -> error
     end
   end
@@ -1082,7 +854,7 @@ defmodule File do
   """
   def cwd!() do
     case F.get_cwd do
-      { :ok, cwd } -> list_to_binary(cwd)
+      { :ok, cwd } -> :unicode.characters_to_binary(cwd)
       { :error, reason } ->
           raise File.Error, reason: reason, action: "get current working directory"
     end
@@ -1103,7 +875,7 @@ defmodule File do
     case F.set_cwd(path) do
       :ok -> :ok
       { :error, reason } ->
-          raise File.Error, reason: reason, action: "set current working directory to", path: to_binary(path)
+          raise File.Error, reason: reason, action: "set current working directory to", path: :unicode.characters_to_binary(path)
     end
   end
 
@@ -1125,6 +897,31 @@ defmodule File do
     end
   end
 
+  @doc """
+  Returns list of files in the given directory.
+
+  It returns `{ :ok, [files] }` in case of success,
+  `{ :error, reason }` otherwise.
+  """
+  def ls(path // ".") do
+    case F.list_dir(path) do
+      { :ok, file_list } -> { :ok, Enum.map file_list, :unicode.characters_to_binary(&1) }
+      { :error, _ } = error -> error
+    end
+  end
+
+  @doc """
+  The same as `ls/1` but raises `File.Error`
+  in case of an error.
+  """
+  def ls!(dir // ".") do
+    case ls(dir) do
+      { :ok, value } -> value
+      { :error, reason } ->
+        raise File.Error, reason: reason, action: "list directory", path: :unicode.characters_to_binary(dir)
+    end
+  end
+
   @doc """
   Closes the file referenced by `io_device`. It mostly returns `:ok`, except
   for some severe errors such as out of memory.
@@ -1133,15 +930,22 @@ defmodule File do
   `close/1` might return an old write error and not even try to close the file.
   See `open/2`.
   """
+  def close(io_device) when is_function(io_device) do
+    { iterator, _ } = io_device.()
+    iterator.(:close)
+  end
+
   def close(io_device) do
     F.close(io_device)
   end
 
   @doc """
-  Convert the file device into an iterator that can be
+  Converts the file device into an iterator that can be
   passed into `Enum`. The device is iterated line
-  by line lazily, at the end of iteration the file is
-  closed.
+  by line, at the end of iteration the file is closed.
+
+  This reads the file as utf-8. CHeck out `File.biniterator`
+  to handle the file as a raw binary.
 
   ## Examples
 
@@ -1149,7 +953,8 @@ defmodule File do
   quotes per single quotes and write each line to a target file
   is shown below:
 
-      source = File.iterator("README.md")
+      { :ok, device } = File.open("README.md")
+      source = File.iterator(device)
       File.open "NEWREADME.md", [:write], fn(target) ->
         Enum.each source, fn(line) ->
           IO.write target, Regex.replace(%r/"/, line, "'")
@@ -1165,29 +970,24 @@ defmodule File do
 
   def iterator(device) do
     fn ->
-      function = fn(_) ->
-        case :io.get_line(device, "") do
-          :eof ->
-            close(device)
-            :stop
-          { :error, reason } ->
-            raise File.IteratorError, reason: reason
-          data ->
-            { strip_ending(data), :ok }
-        end
+      function = fn
+        :ok ->
+          case :io.get_line(device, '') do
+            :eof ->
+              close(device)
+              :stop
+            { :error, reason } ->
+              raise File.IteratorError, reason: reason
+            data ->
+              { data, :ok }
+          end
+        :close ->
+          close(device)
       end
-      { function, function.(:start) }
+      { function, :ok }
     end
   end
 
-  defp strip_ending("\r\n"), do: <<>>
-  defp strip_ending("\n"),   do: <<>>
-  defp strip_ending(""),     do: <<>>
-
-  defp strip_ending(<< h, t :: binary >>) do
-    << h, strip_ending(t) :: binary >>
-  end
-
   @doc """
   Opens the given `file` with the given `mode` and
   returns its iterator. Fails for the same reasons
@@ -1205,46 +1005,74 @@ defmodule File do
   cannot be opened.
   """
   def iterator!(file, mode // []) do
-    open!(file, mode) /> iterator
+    open!(file, mode) |> iterator
+  end
+
+  @doc """
+  Converts the file device into an iterator that can
+  be passed into `Enum` to iterate line by line as a
+  binary. Check `iterator/1` for more information.
+  """
+  def biniterator(device)
+
+  def biniterator(file) when is_binary(file) or is_list(file) do
+    biniterator(file, [])
+  end
+
+  def biniterator(device) do
+    fn ->
+      function = fn
+        :ok ->
+          case :file.read_line(device) do
+            :eof ->
+              close(device)
+              :stop
+            { :error, reason } ->
+              raise File.IteratorError, reason: reason
+            { :ok, data } ->
+              { data, :ok }
+          end
+        :close ->
+          close(device)
+      end
+      { function, :ok }
+    end
+  end
+
+  @doc """
+  Opens the given `file` with the given `mode` and
+  returns its biniterator. Fails for the same reasons
+  as `File.open`.
+  """
+  def biniterator(file, mode) do
+    case open(file, mode) do
+      { :ok, device } -> { :ok, biniterator(device) }
+      error -> error
+    end
+  end
+
+  @doc """
+  Same as `biniterator/2` but raises if the file
+  cannot be opened.
+  """
+  def biniterator!(file, mode // []) do
+    open!(file, mode) |> biniterator
   end
 
   ## Helpers
 
-  # Normalize the given path by removing "..".
-  defp normalize(path), do: normalize(split(path), [])
-
-  defp normalize([top|t], [_|acc]) when top in ["..", '..'] do
-    normalize t, acc
+  defp open_defaults([:charlist|t], _add_binary) do
+    open_defaults(t, false)
   end
 
-  defp normalize([top|t], acc) when top in [".", '.'] do
-    normalize t, acc
+  defp open_defaults([:utf8|t], add_binary) do
+    open_defaults([{ :encoding, :utf8 }|t], add_binary)
   end
 
-  defp normalize([h|t], acc) do
-    normalize t, [h|acc]
+  defp open_defaults([h|t], add_binary) do
+    [h|open_defaults(t, add_binary)]
   end
 
-  defp normalize([], acc) do
-    join Enum.reverse(acc)
-  end
-
-  defp open_defaults([:charlist|t], add_encoding, _add_binary) do
-    open_defaults(t, add_encoding, false)
-  end
-
-  defp open_defaults([{:encoding, _} = h|t], _add_encoding, add_binary) do
-    [h|open_defaults(t, false, add_binary)]
-  end
-
-  defp open_defaults([h|t], add_encoding, add_binary) do
-    [h|open_defaults(t, add_encoding, add_binary)]
-  end
-
-  defp open_defaults([], add_encoding, add_binary) do
-    options = []
-    if add_encoding, do: options = [{:encoding, :unicode}|options]
-    if add_binary,   do: options = [:binary|options]
-    options
-  end
+  defp open_defaults([], true),  do: [:binary]
+  defp open_defaults([], false), do: []
 end

lib/elixir/lib/gen_event/behaviour.ex

@@ -0,0 +1,90 @@
+defmodule GenEvent.Behaviour do
+  @moduledoc """
+  This module is a convenience to define GenEvent callbacks in Elixir.
+
+  GenEvent is an OTP behaviour that encapsulates event handling functionality.
+
+  ## Example
+
+  Bellow follows an example of a GenEvent that stores notifications
+  until they are fetched:
+
+      defmodule MyEventHandler do
+        use GenEvent.Behaviour
+
+        # Callbacks
+
+        def init(_) do
+          { :ok, [] }
+        end
+
+        def handle_event({:notification, x}, notifications) do
+          { :ok, [x|notifications] }
+        end
+
+        def handle_call(:notifications, notifications) do
+          {:ok, Enum.reverse(notifications), []}
+        end
+
+      end
+
+      { :ok, pid } = :gen_event.start_link
+
+      :gen_event.add_handler(pid, MyEventHandler, [])
+
+      :gen_event.notify(pid, {:notification, 1})
+      :gen_event.notify(pid, {:notification, 2})
+      
+      :gen_event.call(pid, MyEventHandler, :notifications)
+      #=> [1, 2]
+
+      :gen_event.call(pid, MyEventHandler, :notifications)
+      #=> []
+
+  Notice we never call the server callbacks directly, they are called
+  by OTP whenever we interact with the server. 
+
+  Starting and sending messages to gen_event is done
+  via Erlang's `:gen_event` module. For more information,
+  please refer to the following:
+
+  http://www.erlang.org/doc/man/gen_event.html
+  http://learnyousomeerlang.com/event-handlers
+  """
+
+  @doc false
+  defmacro __using__(_) do
+    quote location: :keep do
+      @behavior :gen_event
+
+      def init(args) do
+        { :ok, args }
+      end
+
+      def handle_event(_event, state) do
+        { :ok, state }
+      end
+
+      def handle_call(_request, state) do
+        { :ok, :ok, state }
+      end
+
+      def handle_info(_msg, state) do
+        { :ok, state }
+      end
+
+      def terminate(reason, state) do
+        :ok
+      end
+
+      def code_change(_old, state, _extra) do
+        { :ok, state }
+      end
+
+      defoverridable [init: 1, 
+                      handle_event: 2,
+                      handle_call: 2, handle_info: 2,
+                      terminate: 2, code_change: 3]
+    end
+  end
+end

lib/elixir/lib/gen_server/behaviour.ex

@@ -1,55 +1,86 @@
-defmodule GenServer.Behavior do
-  @moduledoc false
-
-  defmacro __using__(_) do
-    IO.puts "Using GenServer.Behavior is deprecated, use GenServer.Behaviour instead"
-    quote do
-      use GenServer.Behaviour
-    end
-  end
-end
-
 defmodule GenServer.Behaviour do
   @moduledoc """
-  By using this module, you get default GenServer callbacks
-  for `init`, `handle_call`, `handle_info`, `handle_cast`,
-  `terminate` and `code_change`. Since these functions are
-  defined as overridable, they can be customized and fallback
-  to the default behaviour by calling `super`.
+  This module is a convenience to define GenServer callbacks in Elixir.
 
-  This module also tags the behavior as :gen_server. For more
-  information on gen_server, please refer to the Erlang
-  documentation:
-
-  http://www.erlang.org/doc/man/gen_server.html
-  http://www.erlang.org/doc/design_principles/gen_server_concepts.html
+  A server is responsible to react to messages received from client
+  and a GenServer is an OTP behaviour that encapsulates common server
+  functionalities.
 
   ## Example
 
+  Bellow follows an example of a GenServer that push and pop items
+  into a stack:
+
       defmodule MyServer do
         use GenServer.Behaviour
 
         # Callbacks
 
-        def handle_call(:peek, _from, [h|_] = state) do
-          { :reply, h, state }
+        def handle_call(:pop, _from, [h|t]) do
+          { :reply, h, t }
         end
 
-        # Default behaviour
-        def handle_call(request, from, config) do
-          super(request, from, config)
+        def handle_call(_request, _from, _config) do
+          # Call the default implementation from GenServer.Behaviour
+          super
         end
 
-        def handle_cast({ :push, item }, state) do
-          { :noreply, [item|state] }
+        def handle_cast({ :push, item }, config) do
+          { :noreply, [item|config] }
         end
 
-        # Default cast behaviour
-        def handle_cast(request, config) do
-          super(request, config)
+        def handle_cast(_request, _config) do
+          super
         end
       end
 
+      { :ok, pid } = :gen_server.start_link(MyServer, [:hello], [])
+
+      :gen_server.call(pid, :pop)
+      #=> :hello
+
+      :gen_server.cast(pid, { :push, :world })
+      #=> :ok
+
+      :gen_server.call(pid, :pop)
+      #=> :world
+
+  Notice we never call the server callbacks directly, they are called
+  by OTP whenever we interact with the server. **cast** messages are
+  asynchronous while **call** ones are synchronous.  In the case of
+  GenServer's, there are 8 different values a callback such as
+  `handle_call` or `handle_cast` can return:
+
+      { :reply, reply, new_state }
+      { :reply, reply, new_state, timeout }
+      { :reply, reply, new_state, :hibernate }
+      { :noreply, new_state }
+      { :noreply, new_state, timeout }
+      { :noreply, new_state, :hibernate }
+      { :stop, reason, new_state }
+      { :stop, reason, reply, new_state }
+
+  There are 6 callbacks required to be implemented in a GenServer. The
+  `GenServer.Behaviour` module defines all of them automatically, but
+  allows us to customize the ones we need. The list of callbacks are:
+
+  * `init(args)` - invoked when the server is started;
+  * `handle_call(msg, from, state)` - invoked to handle call messages;
+  * `handle_cast(msg, state)` - invoked to handle cast messages;
+  * `handle_info(msg, state)` - handle all other messages which are
+     normally received by processes;
+  * `terminate(reason, state)` - called when the server is about to
+     terminate, useful for cleaning up;
+  * `code_change(old_vsn, state, extra)` - called when the application
+    code is being upgraded live (hot code swap);
+
+  Starting and sending messages to the gen_server is done
+  via Erlang's `:gen_server` module. For more information,
+  please refer to the following:
+
+  http://www.erlang.org/doc/man/gen_server.html
+  http://www.erlang.org/doc/design_principles/gen_server_concepts.html
+  http://learnyousomeerlang.com/clients-and-servers
   """
 
   @doc false
@@ -62,7 +93,7 @@ defmodule GenServer.Behaviour do
       end
 
       def handle_call(_request, _from, state) do
-        { :reply, :undef, state }
+        { :noreply, state }
       end
 
       def handle_info(_msg, state) do

lib/elixir/lib/hash_dict.ex

@@ -1,89 +1,484 @@
 defmodule HashDict do
   @moduledoc """
-  This module implements a dictionary type based on
-  hashing of the keys. It is a simple wrapper around
-  [Erlang's dict module](http://www.erlang.org/doc/man/dict.html)
-  and exposed via the `Dict` module.
+  A key-value store.
 
-  Check the `Dict` module for examples and documentation.
+  The `HashDict` is meant to work well with both small and
+  large set of keys and it is an implementation of the `Dict`
+  behaviour. For more information about the functions and
+  their APIs, please consult the `Dict` module.
   """
 
-  use Dict.Common
+  @behaviour Dict
 
-  defmacrop dict(data) do
-    quote do
-      { HashDict, unquote(data) }
+  # A dictionary (key-value) implementation based on dynamic hashing.
+  #
+  # This implementation is based on hash tries. We first start with
+  # a set of 8 buckets and expand when the density is about 5 entries
+  # per bucket. We use bit shifting to make rehashing faster on
+  # expansion.
+  #
+  # Compared to dict, it provides many enhancements:
+  #
+  # 1. HashDict buckets are ordered sets, this gives us faster access
+  #    and modification times
+  #
+  # 2. It uses phash2 to calculate the hash (instead of phash)
+  #
+  # 3. The dictionary first starts with a single bucket, instead of
+  #    a set of 8 buckets. This allow us to skip hashing altogher
+  #    for small dictionaries, providing faster operations and
+  #    reducing memory consumption
+  #
+  # 4. Once we reach 8 elements, the dictionary is promoted to a
+  #    set of buckets
+
+  # The ordered record contains a single bucket
+  ordered_threshold = 8
+
+  defrecordp :ordered,
+    size: 0,
+    bucket: []
+
+  # The bucketed record contains a series of buckets.
+  expand_load = 5
+  contract_load = 2
+  node_bitmap = 0b111
+  node_shift = 3
+  node_size = 8
+  node_template = :erlang.make_tuple(node_size, [])
+  node_escaped = Macro.escape(node_template)
+
+  defrecordp :trie,
+    size: 0,
+    depth: 0,
+    expand_on: node_size * expand_load,
+    contract_on: contract_load,
+    root: node_template
+
+  import Bitwise
+
+  # Let's inline common instructions
+  @compile { :inline, bucket_hash: 1, bucket_index: 1, bucket_nth_index: 2, bucket_next: 1 }
+
+  @doc """
+  Creates a new empty dict.
+  """
+  @spec new :: Dict.t
+  def new do
+    ordered()
+  end
+
+  @doc """
+  Creates a new dict from the given enumerable.
+
+  ## Examples
+
+      HashDict.new [{:b,1},{:a,2}]
+      #=> #HashDict<[a: 2, b: 1]>
+
+  """
+  @spec new(list({key :: term, value :: term})) :: Dict.t
+  def new(pairs) do
+    Enum.reduce pairs, ordered(), fn { k, v }, dict ->
+      put(dict, k, v)
     end
   end
 
-  @doc false
-  def keys(dict(data)) do
-    :dict.fetch_keys data
-  end
+  @doc """
+  Creates a new dict from the enumerable with the
+  help of the transformation function.
 
-  @doc false
-  def values(dict(data)) do
-    :dict.fold fn _key, value, acc ->
-      [value|acc]
-    end, [], data
-  end
+  ## Examples
 
-  @doc false
-  def size(dict(data)) do
-    :dict.size data
-  end
+      HashDict.new ["a", "b"], fn x -> {x, x} end
+      #=> #HashDict<[{"a","a"},{"b","b"}]>
 
-  @doc false
-  def has_key?(dict(data), key) do
-    :dict.is_key key, data
-  end
-
-  @doc false
-  def get(dict(data), key, default) do
-    case :dict.find(key, data) do
-      {:ok, value} -> value
-      :error       -> default
+  """
+  @spec new(list, (term -> {key :: term, value ::term})) :: Dict.t
+  def new(list, transform) when is_function(transform) do
+    Enum.reduce list, new(), fn i, dict ->
+      { k, v } = transform.(i)
+      put(dict, k, v)
     end
   end
 
-  @doc false
-  def put(dict(data), key, value) do
-    dict(:dict.store key, value, data)
+  @doc """
+  Puts the given key and value in the dict.
+  """
+  def put(dict, key, value) do
+    { dict, _ } = dict_put(dict, key, { :put, value })
+    dict
   end
 
-  @doc false
-  def delete(dict(data), key) do
-    dict(:dict.erase key, data)
+  @doc """
+  Puts the given value under key in the dictionary
+  only if one does not exist yet.
+  """
+  def put_new(dict, key, value) do
+    update(dict, key, value, fn(v) -> v end)
   end
 
-  @doc false
-  def merge(dict(d1), dict(d2), fun) do
-    dict(:dict.merge fun, d1, d2)
+  @doc """
+  Updates the key in the dictionary according
+  to the given function. Raises if the key does
+  not exist in the dictionary.
+  """
+  def update(dict, key, fun) when is_function(fun, 1) do
+    case dict_put(dict, key, { :update, nil, fun }) do
+      { dict, 0 } ->
+        dict
+      { _dict, 1 } ->
+        raise KeyError, key: key
+    end
   end
 
-  @doc false
-  def merge(dict(_) = d1, d2, fun) do
-    merge(d1, new(d2), fun)
+  @doc """
+  Updates the key in the dictionary according
+  to the given function. Adds initial value if
+  the key does not exist in the dicionary.
+  """
+  def update(dict, key, initial, fun) when is_function(fun, 1) do
+    { dict, _ } = dict_put(dict, key, { :update, initial, fun })
+    dict
   end
 
-  @doc false
-  def update(dict(data), key, fun) do
-    dict(:dict.update key, fun, data)
+  @doc """
+  Gets the value under key from the dict.
+  """
+  def get(dict, key, default // nil) do
+    case dict_get(dict, key) do
+      { ^key, value } -> value
+      false -> default
+    end
   end
 
-  @doc false
-  def update(dict(data), key, initial, fun) do
-    dict(:dict.update key, fun, initial, data)
+  @doc """
+  Gets the value under key from the dict,
+  raises KeyError if such key does not exist.
+  """
+  def get!(dict, key) when is_tuple(dict) do
+    case dict_get(dict, key) do
+      { ^key, value } -> value
+      false -> raise(KeyError, key: key)
+    end
   end
 
-  @doc false
+  @doc """
+  Checks if the dict has the given key.
+  """
+  def has_key?(dict, key) do
+    match? { ^key, _ }, dict_get(dict, key)
+  end
+
+  @doc """
+  Deletes a value from the dict.
+  """
+  def delete(ordered(bucket: bucket, size: size) = dict, key) do
+    case bucket_delete(bucket, key) do
+      { _, 0 } ->
+        dict
+      { new_bucket, -1 } ->
+        ordered(dict, size: size - 1, bucket: new_bucket)
+    end
+  end
+
+  def delete(trie(root: root, size: size, depth: depth) = dict, key) do
+    pos = bucket_hash(key)
+    case node_delete(root, depth, pos, key) do
+      { _, 0 } ->
+        dict
+      { root, -1 } ->
+        if depth > 0 and trie(dict, :contract_on) == size do
+          root = node_contract(root, depth, depth - 1)
+          trie(dict,
+            root: root,
+            size: size - 1,
+            depth: depth - 1,
+            contract_on: div(size, unquote(node_size)),
+            expand_on: div(trie(dict, :expand_on), unquote(node_size)))
+        else
+          trie(dict, size: size - 1, root: root)
+        end
+    end
+  end
+
+  @doc """
+  Returns the dict size.
+  """
+  def size(dict) do
+    elem(dict, 1)
+  end
+
+  @doc """
+  Returns an empty dict.
+  """
   def empty(_) do
-    dict(:dict.new)
+    ordered()
   end
 
-  @doc false
-  def to_list(dict(data)) do
-    :dict.to_list data
+  @doc """
+  Converts the dict to a list.
+  """
+  def to_list(ordered(bucket: bucket)) do
+    bucket
+  end
+
+  def to_list(dict) do
+    dict_fold(dict, [], [&1|&2])
+  end
+
+  @doc """
+  Get all keys in the dict.
+  """
+  def keys(dict) do
+    dict_fold(dict, [], fn { k, _ }, acc -> [k|acc] end)
+  end
+
+  @doc """
+  Get all values in the dict.
+  """
+  def values(dict) do
+    dict_fold(dict, [], fn { _, v }, acc -> [v|acc] end)
+  end
+
+  @doc """
+  Merges two dictionaries.
+  """
+  def merge(dict, enum, callback // fn(_k, _v1, v2) -> v2 end)
+
+  def merge(dict1, dict2, callback) when is_record(dict1, HashDict) and is_record(dict2, HashDict) and elem(dict1, 1) < elem(dict2, 1) do
+    dict_fold dict1, dict2, fn { k, v1 }, acc ->
+      update(acc, k, v1, callback.(k, v1, &1))
+    end
+  end
+
+  def merge(dict1, dict2, callback) when is_record(dict1, HashDict) and is_record(dict2, HashDict) do
+    dict_fold dict2, dict1, fn { k, v2 }, acc ->
+      update(acc, k, v2, callback.(k, &1, v2))
+    end
+  end
+
+  def merge(dict1, dict2, callback) when is_record(dict1, HashDict) do
+    Enum.reduce dict2, dict1, fn { k, v2 }, acc ->
+      update(acc, k, v2, callback.(k, &1, v2))
+    end
+  end
+
+  ## Dict-wide functions
+
+  defp dict_get(ordered(bucket: bucket), key) do
+    :lists.keyfind(key, 1, bucket)
+  end
+
+  defp dict_get(trie(root: root, depth: depth), key) do
+    :lists.keyfind(key, 1, node_bucket(root, depth, bucket_hash(key)))
+  end
+
+  defp dict_fold(ordered(bucket: bucket), acc, fun) do
+    bucket_fold(bucket, acc, fun)
+  end
+
+  defp dict_fold(trie(root: root, depth: depth), acc, fun) do
+    node_fold(root, depth, acc, fun, unquote(node_size))
+  end
+
+  defp dict_put(ordered(size: unquote(ordered_threshold), bucket: bucket), key, value) do
+    root = node_relocate(bucket, 0)
+    dict_put(trie(size: unquote(ordered_threshold), root: root), key, value)
+  end
+
+  defp dict_put(ordered(size: size, bucket: bucket) = dict, key, value) do
+    { new, count } = bucket_put(bucket, key, value)
+    { ordered(dict, size: size + count, bucket: new), count }
+  end
+
+  defp dict_put(trie(root: root, depth: depth, size: size, expand_on: size, contract_on: contract_on) = dict, key, value) do
+    root = node_expand(root, depth, depth + 1)
+    dict = trie(dict, root: root, depth: depth + 1,
+      expand_on: size * unquote(node_size), contract_on: contract_on * unquote(node_size))
+    dict_put(dict, key, value)
+  end
+
+  defp dict_put(trie(root: root, size: size, depth: depth) = dict, key, value) do
+    pos = bucket_hash(key)
+    { root, count } = node_put(root, depth, pos, key, value)
+    { trie(dict, size: size + count, root: root), count }
+  end
+
+  ## Bucket helpers
+
+  # Puts a value in the bucket
+  defp bucket_put([{k,_}=e|bucket], key, { :put, value }) when key < k do
+    { [{key,value},e|bucket], 1 }
+  end
+
+  defp bucket_put([{k,_}=e|bucket], key, { :update, initial, _fun }) when key < k do
+    { [{key,initial},e|bucket], 1 }
+  end
+
+  defp bucket_put([{k,_}=e|bucket], key, value) when key > k do
+    { rest, count } = bucket_put(bucket, key, value)
+    { [e|rest], count }
+  end
+
+  defp bucket_put([{_,_}|bucket], key, { :put, value }) do
+    { [{key,value}|bucket], 0 }
+  end
+
+  defp bucket_put([{_,value}|bucket], key, { :update, _initial, fun }) do
+    { [{key,fun.(value)}|bucket], 0 }
+  end
+
+  defp bucket_put([], key, { :put, value }) do
+    { [{key,value}], 1 }
+  end
+
+  defp bucket_put([], key, { :update, initial, _fun }) do
+    { [{key,initial}], 1 }
+  end
+
+  # Puts a value in the bucket without returning
+  # the operation value
+  defp bucket_put!([{k,_}=e|bucket], key, value) when key < k, do: [{key,value},e|bucket]
+  defp bucket_put!([{k,_}=e|bucket], key, value) when key > k, do: [e|bucket_put!(bucket, key, value)]
+  defp bucket_put!([{_,_}|bucket], key, value), do: [{key,value}|bucket]
+  defp bucket_put!([], key, value), do: [{key,value}]
+
+  # Deletes a key from the bucket
+  defp bucket_delete([{k,_}|_] = bucket, key) when key < k do
+    { bucket, 0 }
+  end
+
+  defp bucket_delete([{k,_}=e|bucket], key) when key > k do
+    { rest, count } = bucket_delete(bucket, key)
+    { [e|rest], count }
+  end
+
+  defp bucket_delete([{_,_}|bucket], _key) do
+    { bucket, -1 }
+  end
+
+  defp bucket_delete([], _key) do
+    { [], 0 }
+  end
+
+  # Folds the bucket
+  defp bucket_fold(bucket, acc, fun) do
+    :lists.foldl(fun, acc, bucket)
+  end
+
+  defp bucket_hash(key) do
+    :erlang.phash2(key)
+  end
+
+  defp bucket_index(hash) do
+    hash &&& unquote(node_bitmap)
+  end
+
+  defp bucket_nth_index(hash, n) do
+    (hash >>> (unquote(node_shift) * n)) &&& unquote(node_bitmap)
+  end
+
+  defp bucket_next(hash) do
+    hash >>> unquote(node_shift)
+  end
+
+  ## Node helpers
+
+  # Gets a bucket from the node
+  defp node_bucket(node, 0, hash) do
+    elem(node, bucket_index(hash))
+  end
+
+  defp node_bucket(node, depth, hash) do
+    child = elem(node, bucket_index(hash))
+    node_bucket(child, depth - 1, bucket_next(hash))
+  end
+
+  # Puts a key-value into a node
+  defp node_put(node, 0, hash, key, value) do
+    pos = bucket_index(hash)
+    { new, count } = bucket_put(elem(node, pos), key, value)
+    { set_elem(node, pos, new), count }
+  end
+
+  defp node_put(node, depth, hash, key, value) do
+    pos = bucket_index(hash)
+    { new, count } = node_put(elem(node, pos), depth - 1, bucket_next(hash), key, value)
+    { set_elem(node, pos, new), count }
+  end
+
+  # Deletes a key from the bucket
+  defp node_delete(node, 0, hash, key) do
+    pos = bucket_index(hash)
+    case bucket_delete(elem(node, pos), key) do
+      { _, 0 }    -> { node, 0 }
+      { new, -1 } -> { set_elem(node, pos, new), -1 }
+    end
+  end
+
+  defp node_delete(node, depth, hash, key) do
+    pos = bucket_index(hash)
+    case node_delete(elem(node, pos), depth - 1, bucket_next(hash), key) do
+      { _, 0 }    -> { node, 0 }
+      { new, -1 } -> { set_elem(node, pos, new), -1 }
+    end
+  end
+
+  # Folds a node recursively
+  defp node_fold(bucket, -1, acc, fun, _) do
+    bucket_fold(bucket, acc, fun)
+  end
+
+  defp node_fold(node, depth, acc, fun, count) when count >= 1 do
+    acc = node_fold(:erlang.element(count, node), depth - 1, acc, fun, unquote(node_size))
+    node_fold(node, depth, acc, fun, count - 1)
+  end
+
+  defp node_fold(_node, _, acc, _fun, 0) do
+    acc
+  end
+
+  # Node resizing
+  defp node_expand({ b1, b2, b3, b4, b5, b6, b7, b8 }, 0, n) do
+    { node_relocate(b1, n), node_relocate(b2, n), node_relocate(b3, n),
+      node_relocate(b4, n), node_relocate(b5, n), node_relocate(b6, n),
+      node_relocate(b7, n), node_relocate(b8, n) }
+  end
+
+  defp node_expand({ b1, b2, b3, b4, b5, b6, b7, b8 }, depth, n) do
+    depth = depth - 1
+    { node_expand(b1, depth, n), node_expand(b2, depth, n), node_expand(b3, depth, n),
+      node_expand(b4, depth, n), node_expand(b5, depth, n), node_expand(b6, depth, n),
+      node_expand(b7, depth, n), node_expand(b8, depth, n) }
+  end
+
+  defp node_contract({ b1, b2, b3, b4, b5, b6, b7, b8 }, depth, n) when depth > 1 do
+    depth = depth - 1
+    { node_contract(b1, depth, n), node_contract(b2, depth, n), node_contract(b3, depth, n),
+      node_contract(b4, depth, n), node_contract(b5, depth, n), node_contract(b6, depth, n),
+      node_contract(b7, depth, n), node_contract(b8, depth, n) }
+  end
+
+  defp node_contract({ b1, b2, b3, b4, b5, b6, b7, b8 }, 1, n) do
+    unquote(node_escaped)
+      |> each_contract(b1, n) |> each_contract(b2, n) |> each_contract(b3, n)
+      |> each_contract(b4, n) |> each_contract(b5, n) |> each_contract(b6, n)
+      |> each_contract(b7, n) |> each_contract(b8, n)
+  end
+
+  defp each_contract(acc, { b1, b2, b3, b4, b5, b6, b7, b8 }, n) do
+    acc
+      |> node_relocate(b1, n) |> node_relocate(b2, n) |> node_relocate(b3, n)
+      |> node_relocate(b4, n) |> node_relocate(b5, n) |> node_relocate(b6, n)
+      |> node_relocate(b7, n) |> node_relocate(b8, n)
+  end
+
+  defp node_relocate(node // unquote(node_escaped), bucket, n) do
+    :lists.foldl fn { key, value }, acc ->
+      pos = key |> bucket_hash() |> bucket_nth_index(n)
+      set_elem(acc, pos, bucket_put!(elem(acc, pos), key, value))
+    end, node, bucket
   end
 end
 
@@ -94,4 +489,12 @@ end
 
 defimpl Access, for: HashDict do
   def access(dict, key), do: HashDict.get(dict, key, nil)
-end
+end
+
+defimpl Binary.Inspect, for: HashDict do
+  import Kernel, except: [inspect: 2]
+
+  def inspect(dict, opts) do
+    "#HashDict<" <> Binary.Inspect.inspect(HashDict.to_list(dict), opts) <> ">"
+  end
+end

lib/elixir/lib/io.ex

@@ -3,7 +3,7 @@ defmodule IO do
   Module responsible for doing IO. Many functions in this
   module expects an IO device and an io data encoded in UTF-8.
 
-  An IO device must be a pid is an atom representing a process.
+  An IO device must be a pid or an atom representing a process.
   For convenience, Elixir provides `:stdio` and `:stderr` as
   shortcut to Erlang's `:standard_io` and `:standard_error`.
 
@@ -23,6 +23,8 @@ defmodule IO do
 
   """
 
+  import :erlang, only: [group_leader: 0]
+
   @doc """
   Reads `count` bytes from the IO device. It returns:
 
@@ -34,12 +36,25 @@ defmodule IO do
     for instance {:error, :estale} if reading from an
     NFS file system.
   """
-  def read(device // :stdio, count) do
+  def read(device // group_leader(), count) do
     :io.get_chars(map_dev(device), "", count)
   end
 
   @doc """
-  Read a line from the IO device. It returns:
+  Reads `count` bytes from the IO device as binary,
+  no unicode conversion happens.
+
+  Check `read/2` for more information.
+  """
+  def binread(device // group_leader(), count) do
+    case :file.read(map_dev(device), count) do
+      { :ok, data } -> data
+      other -> other
+    end
+  end
+
+  @doc """
+  Reads a line from the IO device. It returns:
 
   * `data` - The input characters.
 
@@ -52,10 +67,23 @@ defmodule IO do
   This function does the same as `gets/2`,
   except the prompt is not required as argument.
   """
-  def readline(device // :stdio) do
+  def readline(device // group_leader()) do
     :io.get_line(map_dev(device), "")
   end
 
+  @doc """
+  Reads a line from the IO device as binary,
+  no unicode conversion happens.
+
+  Check `readline/1` for more information.
+  """
+  def binreadline(device // group_leader()) do
+    case :file.read_line(map_dev(device)) do
+      { :ok, data } -> data
+      other -> other
+    end
+  end
+
   @doc """
   Writes the given argument to the given device.
   By default the device is the standard output.
@@ -73,26 +101,47 @@ defmodule IO do
       #=> "error"
 
   """
-  def write(device // :stdio, item) do
+  def write(device // group_leader(), item) do
     :io.put_chars map_dev(device), to_iodata(item)
   end
 
+  @doc """
+  Writes the given argument to the given device
+  as a binary, no unicode conversion happens.
+
+  Check `write/2` for more information.
+  """
+  def binwrite(device // group_leader(), item) do
+    :file.write map_dev(device), to_iodata(item)
+  end
+
   @doc """
   Writes the argument to the device, similarly to write
   but adds a new line at the end. The argument is expected
   to be a chardata.
   """
-  def puts(device // :stdio, item) do
+  def puts(device // group_leader(), item) do
     erl_dev = map_dev(device)
-    :io.put_chars erl_dev, to_iodata(item)
-    :io.nl(erl_dev)
+    :io.put_chars erl_dev, [to_iodata(item), ?\n]
   end
 
   @doc """
   Inspects and writes the given argument to the device
-  followed by a new line. Returns the item given.
+  followed by a new line. A set of options can be given.
+
+  ## Examples
+
+      IO.inspect Process.list
+
   """
-  def inspect(device // :stdio, item, opts // []) do
+  def inspect(item, opts // []) do
+    inspect group_leader(), item, opts
+  end
+
+  @doc """
+  Inspects the item with options using the given device.
+  """
+  def inspect(device, item, opts) do
     puts device, Binary.Inspect.inspect(item, opts)
     item
   end
@@ -108,7 +157,7 @@ defmodule IO do
     for instance {:error, :estale} if reading from an
     NFS file system.
   """
-  def getb(device // :stdio, prompt, count // 1) do
+  def getb(device // group_leader(), prompt, count // 1) do
     :io.get_chars(map_dev(device), to_iodata(prompt), count)
   end
 
@@ -124,7 +173,7 @@ defmodule IO do
     for instance {:error, :estale} if reading from an
     NFS file system.
   """
-  def gets(device // :stdio, prompt) do
+  def gets(device // group_leader(), prompt) do
     :io.get_line(map_dev(device), to_iodata(prompt))
   end
 

lib/elixir/lib/io/ansi.ex

@@ -0,0 +1,233 @@
+defmodule IO.ANSI.Sequence do
+  @moduledoc false
+
+  defmacro defsequence(name, code) do
+    binary_name = atom_to_binary(name)
+
+    quote do
+      def unquote(name)() do
+        "\e[#{unquote(code)}m"
+      end
+
+      defp escape_sequence(<< unquote(binary_name), rest :: binary >>) do
+        { "\e[#{unquote(code)}m", rest }
+      end
+    end
+  end
+end
+
+defmodule IO.ANSI do
+  @moduledoc """
+  This module provides functionality to render ANSI escape sequences
+  (http://en.wikipedia.org/wiki/ANSI_escape_code) —  characters embedded
+  in the text used to control formatting, color, and other output options
+  on video text terminals.
+
+  Please be aware that in Erlang/OTP versions prior to R16, you will not
+  be able to render ANSI escape sequences in iex or erlang shell
+  """
+
+  import IO.ANSI.Sequence
+
+  @doc """
+  Checks whether the default I/O device is a terminal or a file.
+
+  Used to identify whether printing ANSI escape sequences will likely
+  be printed as intended.
+
+  Please note that invoked while in shell (iex) in Erlang/OTP
+  prior to R16, terminal?/0 will always return false because
+  Erlang shell did not support ANSI escape sequences up until
+  R16.
+  """
+  @spec terminal? :: boolean
+  @spec terminal?(:io.device) :: boolean
+  def terminal?(device // :erlang.group_leader) do
+    if :erlang.system_info(:otp_release) < 'R16' and
+       Process.whereis(:user) != device do
+      # Shell prior to R16 doesn't support ANSI escape
+      # sequences
+      false
+    else
+      match?({:ok, _}, :io.columns(device))
+    end
+  end
+
+  @doc "Resets all attributes"
+  defsequence :reset, 0
+
+  @doc "Bright (increased intensity) or Bold"
+  defsequence :bright, 1
+
+  @doc "Faint (decreased intensity), not widely supported"
+  defsequence :faint, 2
+
+  @doc "Italic: on. Not widely supported. Sometimes treated as inverse."
+  defsequence :italic, 3
+
+  @doc "Underline: Single"
+  defsequence :underline, 4
+
+  @doc "Blink: Slow. Less than 150 per minute"
+  defsequence :blink_slow, 5
+
+  @doc "Blink: Rapid. MS-DOS ANSI.SYS; 150 per minute or more; not widely supported"
+  defsequence :blink_rapid, 6
+
+  @doc "Image: Negative. Swap foreground and background"
+  defsequence :inverse, 7
+
+  @doc "Image: Negative. Swap foreground and background"
+  defsequence :reverse, 7
+
+  @doc "Conceal. Not widely supported"
+  defsequence :conceal, 8
+
+  @doc "Crossed-out. Characters legible, but marked for deletion. Not widely supported."
+  defsequence :crossed_out, 9
+
+  @doc "Sets primary (default) font"
+  defsequence :primary_font, 10
+
+  lc font_n inlist [1,2,3,4,5,6,7,8,9] do
+    Module.eval_quoted __ENV__, (quote do
+      @doc "Sets alternative font #{unquote(font_n)}"
+      defsequence(unquote(:"font_#{font_n}"), unquote(font_n + 10))
+    end)
+  end
+
+  @doc "Normal color or intensity"
+  defsequence :normal, 22
+
+  @doc "Not italic"
+  defsequence :not_italic, 23
+
+  @doc "Underline: None"
+  defsequence :no_underline, 24
+
+  @doc "Blink: off"
+  defsequence :blink_off, 25
+
+  colors = [:black, :red, :green, :yellow, :blue, :magenta, :cyan, :white]
+  colors = Enum.zip(0..(length(colors)-1), colors)
+
+  lc { code, color } inlist colors do
+    Module.eval_quoted __ENV__, (quote do
+      @doc "Sets foreground color to #{unquote(color)}"
+      defsequence unquote(color), unquote(code + 30)
+
+      @doc "Sets background color to #{unquote(color)}"
+      defsequence unquote(:"#{color}_background"), unquote(code + 40)
+    end)
+  end
+
+  @doc "Default text color"
+  defsequence :default_color, 39
+
+  @doc "Default background color"
+  defsequence :default_background, 49
+
+  @doc "Framed"
+  defsequence :framed, 51
+
+  @doc "Encircled"
+  defsequence :encircled, 52
+
+  @doc "Overlined"
+  defsequence :overlined, 53
+
+  @doc "Not framed or encircled"
+  defsequence :not_framed_encircled, 54
+
+  @doc "Not overlined"
+  defsequence :not_overlined, 55
+
+  # Catch spaces between codes
+  defp escape_sequence(<< ?\s, rest :: binary >>) do
+    escape_sequence(rest)
+  end
+
+  defp escape_sequence(other) do
+    [spec|_] = String.split(other, %r/(,|\})/)
+    raise ArgumentError, message: "invalid ANSI sequence specification: #{spec}"
+  end
+
+  @doc %B"""
+  Escapes a string coverting named ANSI sequences into actual ANSI codes.
+
+  The format for referring sequences is `%{red}` and `%{red,bright}` (for
+  multiple sequences)
+
+  It will also force a %{reset} to get appended to every string. If you don't
+  want this behaviour, use `escape_fragment/1` and `escape_fragment/2`.
+
+  An optional boolean parameter can be passed to enable or disable
+  emitting actual ANSI codes. When false, no ANSI codes will emitted.
+  By default, standard output will be checked if it is a terminal capable
+  of handling these sequences (using `terminal?/0` function)
+
+  ## Example
+
+    iex> IO.ANSI.escape "Hello %{red,bright,green}yes"
+    "Hello \e[31m\e[1m\e[32myes\e[0m"
+  """
+  @spec escape(String.t, emit :: boolean) :: String.t
+  def escape(string, emit // terminal?) do
+    {rendered, emitted} = do_escape(string, false, emit, false, [])
+    if emitted and emit do
+      rendered <> reset
+    else
+      rendered
+    end
+  end
+
+  @doc %B"""
+  Escapes a string coverting named ANSI sequences into actual ANSI codes.
+
+  The format for referring sequences is `%{red}` and `%{red,bright}` (for
+  multiple sequences)
+
+  An optional boolean parameter can be passed to enable or disable
+  emitting actual ANSI codes. When false, no ANSI codes will emitted.
+  By default, standard output will be checked if it is a terminal capable
+  of handling these sequences (using `terminal?/0` function)
+
+  ## Example
+
+    iex> IO.ANSI.escape "Hello %{red,bright,green}yes"
+    "Hello \e[31m\e[1m\e[32myes\e[0m"
+
+  """
+  @spec escape_fragment(String.t, emit :: boolean) :: String.t
+  def escape_fragment(string, emit // terminal?) do
+    {rendered, _emitted} = do_escape(string, false, emit, false, [])
+    rendered
+  end
+
+  defp do_escape(<< ?%, ?{, rest :: binary >>, false, emit, _emitted, acc) do
+    do_escape_sequence(rest, emit, acc)
+  end
+  defp do_escape(<< ?,, rest :: binary >>, true, emit, _emitted, acc) do
+    do_escape_sequence(rest, emit, acc)
+  end
+  defp do_escape(<< ?\s, rest :: binary >>, true, emit, emitted, acc) do
+    do_escape(rest, true, emit, emitted, acc)
+  end
+  defp do_escape(<< ?}, rest :: binary >>, true, emit, emitted, acc) do
+    do_escape(rest, false, emit, emitted, acc)
+  end
+  defp do_escape(<< x :: [binary, size(1)], rest :: binary>>, false, emit, emitted, acc) do
+    do_escape(rest, false, emit, emitted, [x|acc])
+  end
+  defp do_escape("", false, _emit, emitted, acc) do
+    {list_to_binary(Enum.reverse(acc)), emitted}
+  end
+
+  defp do_escape_sequence(rest, emit, acc) do
+    {code, rest} = escape_sequence(rest)
+    if emit do
+      acc = [code|acc]
+    end
+    do_escape(rest, true, emit, true, acc)
+  end
+end

lib/elixir/lib/kernel/cli.ex

@@ -1,52 +1,67 @@
-defrecord Kernel.CLI.Config, commands: [], output: ".",
-                             compile: [], halt: true, compiler_options: []
-
 defmodule Kernel.CLI do
-  @moduledoc false
+  @moduledoc """
+  Module responsible for controlling Elixir's CLI
+  """
 
-  # Invoked directly from erlang boot process. It parses all argv
-  # options and execute them in the order they are specified.
-  def process_argv(options) do
-    { config, argv } = process_options(options, Kernel.CLI.Config.new)
+  defrecord Config, commands: [], output: ".",
+                    compile: [], halt: true, compiler_options: []
 
+  # This is the API invoked by Elixir boot process.
+  @doc false
+  def main(argv) do
     argv = lc arg inlist argv, do: list_to_binary(arg)
-    :gen_server.call(:elixir_code_server, { :argv, argv })
 
-    all_commands = Enum.reverse(config.commands)
+    { config, argv } = process_argv(argv, Kernel.CLI.Config.new)
+    :elixir_code_server.cast({ :argv, argv })
 
+    run fn ->
+      Enum.map Enum.reverse(config.commands), process_command(&1, config)
+    end, config.halt
+  end
+
+  @doc """
+  Runs the given function by catching any failure
+  and printing them to stdout. `at_exit` hooks are
+  also invoked before exiting.
+
+  This function is used by Elixir's CLI and also
+  by escripts generated by Elixir.
+  """
+  def run(fun, halt // true) do
     try do
-      Enum.map all_commands, process_command(&1, config)
-      if config.halt do
+      fun.()
+      if halt do
         at_exit(0)
-        halt(0)
+        System.halt(0)
       end
     rescue
       exception ->
         at_exit(1)
         trace = System.stacktrace
         IO.puts :stderr, "** (#{inspect exception.__record__(:name)}) #{exception.message}"
-        IO.puts Exception.formatted_stacktrace(trace)
-        halt(1)
+        IO.puts Exception.format_stacktrace(trace)
+        System.halt(1)
     catch
       :exit, reason when is_integer(reason) ->
         at_exit(reason)
-        halt(reason)
+        System.halt(reason)
       :exit, :normal ->
         at_exit(0)
-        halt(0)
+        System.halt(0)
       kind, reason ->
         at_exit(1)
         trace = System.stacktrace
         IO.puts :stderr, "** (#{kind}) #{inspect(reason)}"
-        IO.puts Exception.formatted_stacktrace(trace)
-        halt(1)
+        IO.puts Exception.format_stacktrace(trace)
+        System.halt(1)
     end
   end
 
   ## Private
 
   defp at_exit(status) do
-    hooks = :gen_server.call(:elixir_code_server, :at_exit)
+    hooks = :elixir_code_server.call(:flush_at_exit)
+
     lc hook inlist hooks do
       try do
         hook.(status)
@@ -54,24 +69,28 @@ defmodule Kernel.CLI do
         exception ->
           trace = System.stacktrace
           IO.puts :stderr, "** (#{inspect exception.__record__(:name)}) #{exception.message}"
-          IO.puts Exception.formatted_stacktrace(trace)
+          IO.puts Exception.format_stacktrace(trace)
       catch
         kind, reason ->
           trace = System.stacktrace
           IO.puts :stderr, "** #{kind} #{inspect(reason)}"
-          IO.puts Exception.formatted_stacktrace(trace)
+          IO.puts Exception.format_stacktrace(trace)
       end
     end
+
+    # If an at_exit callback adds a
+    # new hook we need to invoke it.
+    unless hooks == [], do: at_exit(status)
   end
 
   defp invalid_option(option) do
-    IO.puts(:stderr, "Unknown option #{list_to_binary(option)}")
-    halt(1)
+    IO.puts(:stderr, "Unknown option #{option}")
+    System.halt(1)
   end
 
   defp shared_option?(list, config, callback) do
     case process_shared(list, config) do
-      { [h|t], _ } when h == hd(list) ->
+      { [h|_], _ } when h == hd(list) ->
         invalid_option h
       { new_list, new_config } ->
         callback.(new_list, new_config)
@@ -80,43 +99,44 @@ defmodule Kernel.CLI do
 
   # Process shared options
 
-  defp process_shared(['-v'|t], config) do
+  defp process_shared(["-v"|t], config) do
     IO.puts "Elixir #{System.version}"
     process_shared t, config
   end
 
-  defp process_shared(['--no-halt'|t], config) do
+  defp process_shared(["--app",h|t], config) do
+    process_shared t, config.update_commands [{:app,h}|&1]
+  end
+
+  defp process_shared(["--no-halt"|t], config) do
     process_shared t, config.halt(false)
   end
 
-  defp process_shared(['-e',h|t], config) do
-    process_shared t, config.prepend_commands [eval: h]
+  defp process_shared(["-e",h|t], config) do
+    process_shared t, config.update_commands [{:eval,h}|&1]
   end
 
-  defp process_shared(['-pa',h|t], config) do
-    Enum.each File.wildcard(File.expand_path(h)), Code.prepend_path(&1)
+  defp process_shared(["-pa",h|t], config) do
+    Enum.each Path.wildcard(Path.expand(h)), Code.prepend_path(&1)
     process_shared t, config
   end
 
-  defp process_shared(['-pz',h|t], config) do
-    Enum.each File.wildcard(File.expand_path(h)), Code.append_path(&1)
+  defp process_shared(["-pz",h|t], config) do
+    Enum.each Path.wildcard(Path.expand(h)), Code.append_path(&1)
     process_shared t, config
   end
 
-  defp process_shared(['-r',h|t], config) do
-    h = list_to_binary(h)
-    config = Enum.reduce File.wildcard(h), config, fn path, config ->
-      config.prepend_commands [require: path]
-    end
-    process_shared t, config
+  defp process_shared(["-r",h|t], config) do
+    process_shared t, Enum.reduce(Path.wildcard(h), config, fn path, config ->
+      config.update_commands [{:require,path}|&1]
+    end)
   end
 
-  defp process_shared(['-pr',h|t], config) do
-    h = list_to_binary(h)
-    process_shared t, config.prepend_commands [parallel_require: h]
+  defp process_shared(["-pr",h|t], config) do
+    process_shared t, config.update_commands [{:parallel_require,h}|&1]
   end
 
-  defp process_shared([erl,_|t], config) when erl in ['--erl', '--sname', '--remsh', '--name'] do
+  defp process_shared([erl,_|t], config) when erl in ["--erl", "--sname", "--remsh", "--name"] do
     process_shared t, config
   end
 
@@ -126,79 +146,87 @@ defmodule Kernel.CLI do
 
   # Process init options
 
-  def process_options(['--'|t], config) do
+  defp process_argv(["--"|t], config) do
     { config, t }
   end
 
-  def process_options(['--compile'|t], config) do
+  defp process_argv(["--compile"|t], config) do
     process_compiler t, config
   end
 
-  def process_options(['-S',h|t], config) do
+  defp process_argv(["-S",h|t], config) do
     exec = System.find_executable(h)
     if exec do
-      { config.prepend_commands([require: list_to_binary(exec)]), t }
+      { config.update_commands([{:require,exec}|&1]), t }
     else
       IO.puts(:stderr, "Could not find executable #{h}")
-      halt(1)
+      System.halt(1)
     end
   end
 
-  def process_options([h|t] = list, config) do
+  defp process_argv([h|t] = list, config) do
     case h do
-      '-' ++ _ ->
-        shared_option? list, config, process_options(&1, &2)
+      "-" <> _ ->
+        shared_option? list, config, process_argv(&1, &2)
       _ ->
-        h = list_to_binary(h)
-        { config.prepend_commands([require: h]), t }
+        { config.update_commands([{:require,h}|&1]), t }
     end
   end
 
-  def process_options([], config) do
+  defp process_argv([], config) do
     { config, [] }
   end
 
   # Process compiler options
 
-  defp process_compiler(['--'|t], config) do
+  defp process_compiler(["--"|t], config) do
     { config, t }
   end
 
-  defp process_compiler(['-o',h|t], config) do
-    process_compiler t, config.output(list_to_binary(h))
+  defp process_compiler(["-o",h|t], config) do
+    process_compiler t, config.output(h)
   end
 
-  defp process_compiler(['--no-docs'|t], config) do
-    process_compiler t, config.merge_compiler_options(docs: false)
+  defp process_compiler(["--no-docs"|t], config) do
+    process_compiler t, config.update_compiler_options([{:docs,false}|&1])
   end
 
-  defp process_compiler(['--debug-info'|t], config) do
-    process_compiler t, config.merge_compiler_options(debug_info: true)
+  defp process_compiler(["--no-debug-info"|t], config) do
+    process_compiler t, config.update_compiler_options([{:debug_info,false}|&1])
   end
 
-  defp process_compiler(['--ignore-module-conflict'|t], config) do
-    process_compiler t, config.merge_compiler_options(ignore_module_conflict: true)
+  defp process_compiler(["--ignore-module-conflict"|t], config) do
+    process_compiler t, config.update_compiler_options([{:ignore_module_conflict,true}|&1])
   end
 
   defp process_compiler([h|t] = list, config) do
     case h do
-      '-' ++ _ ->
+      "-" <> _ ->
         shared_option? list, config, process_compiler(&1, &2)
       _ ->
-        h = list_to_binary(h)
         pattern = if File.dir?(h), do: "#{h}/**/*.ex", else: h
-        process_compiler t, config.prepend_compile [pattern]
+        process_compiler t, config.update_compile [pattern|&1]
     end
   end
 
   defp process_compiler([], config) do
-    { config.prepend_commands([compile: config.compile]), [] }
+    { config.update_commands([{:compile,config.compile}|&1]), [] }
   end
 
   # Process commands
 
-  defp process_command({:eval, expr}, _config) when is_list(expr) do
-    :elixir.eval(expr, [])
+  defp process_command({:eval, expr}, _config) when is_binary(expr) do
+    Code.eval(expr, [])
+  end
+
+  defp process_command({:app, app}, _config) when is_binary(app) do
+    case Application.Behaviour.start(binary_to_atom(app)) do
+      { :error, reason } ->
+        IO.puts(:stderr, "Could not start application #{app}: #{inspect reason}")
+        System.halt(1)
+      :ok ->
+        :ok
+    end
   end
 
   defp process_command({:require, file}, _config) when is_binary(file) do
@@ -206,8 +234,8 @@ defmodule Kernel.CLI do
   end
 
   defp process_command({:parallel_require, pattern}, _config) when is_binary(pattern) do
-    files = File.wildcard(pattern)
-    files = List.uniq(files)
+    files = Path.wildcard(pattern)
+    files = Enum.uniq(files)
     files = Enum.filter files, File.regular?(&1)
     Kernel.ParallelRequire.files(files)
   end
@@ -215,8 +243,8 @@ defmodule Kernel.CLI do
   defp process_command({:compile, patterns}, config) do
     File.mkdir_p(config.output)
 
-    files = Enum.map patterns, File.wildcard(&1)
-    files = List.uniq(List.concat(files))
+    files = Enum.map patterns, Path.wildcard(&1)
+    files = Enum.uniq(List.concat(files))
     files = Enum.filter files, File.regular?(&1)
 
     Code.compiler_options(config.compiler_options)

lib/elixir/lib/kernel/error_handler.ex

@@ -17,7 +17,7 @@ defmodule Kernel.ErrorHandler do
     case Code.ensure_loaded(module) do
       { :module, _ } -> []
       { :error, _ } ->
-        parent = Process.get(:elixir_compiler_pid)
+        parent = :erlang.get(:elixir_compiler_pid)
         parent <- { :waiting, self(), module }
         receive do
           { :release, ^parent } -> :ok

lib/elixir/lib/kernel/parallel_compiler.ex

@@ -19,7 +19,7 @@ defmodule Kernel.ParallelCompiler do
   with its name can be optionally given as argument.
   """
   def files(files, callback // default_callback) do
-    files_to_path(files, nil, callback)
+    spawn_compilers(files, nil, callback)
   end
 
   @doc """
@@ -27,24 +27,29 @@ defmodule Kernel.ParallelCompiler do
   Read files/2 for more information.
   """
   def files_to_path(files, path, callback // default_callback) when is_binary(path) do
+    spawn_compilers(files, path, callback)
+  end
+
+  defp spawn_compilers(files, path, callback) do
     Code.ensure_loaded(Kernel.ErrorHandler)
-    spawn_compilers(files, path, callback, [], [], [])
+    schedulers = max(:erlang.system_info(:schedulers_online), 2)
+    spawn_compilers(files, path, callback, [], [], schedulers, [])
   end
 
   # We already have 4 currently running, don't spawn new ones
-  defp spawn_compilers(files, output, callback, waiting, queued, result) when
-      length(queued) - length(waiting) >= 4 do
-    wait_for_messages(files, output, callback, waiting, queued, result)
+  defp spawn_compilers(files, output, callback, waiting, queued, schedulers, result) when
+      length(queued) - length(waiting) >= schedulers do
+    wait_for_messages(files, output, callback, waiting, queued, schedulers, result)
   end
 
   # Spawn a compiler for each file in the list until we reach the limit
-  defp spawn_compilers([h|t], output, callback, waiting, queued, result) do
+  defp spawn_compilers([h|t], output, callback, waiting, queued, schedulers, result) do
     parent = self()
 
     child  = spawn_link fn ->
-      Process.put(:elixir_compiler_pid, parent)
-      Process.put(:elixir_ensure_compiled, true)
-      Process.flag(:error_handler, Kernel.ErrorHandler)
+      :erlang.put(:elixir_compiler_pid, parent)
+      :erlang.put(:elixir_ensure_compiled, true)
+      :erlang.process_flag(:error_handler, Kernel.ErrorHandler)
 
       try do
         if output do
@@ -59,25 +64,25 @@ defmodule Kernel.ParallelCompiler do
       end
     end
 
-    spawn_compilers(t, output, callback, waiting, [{child,h}|queued], result)
+    spawn_compilers(t, output, callback, waiting, [{child,h}|queued], schedulers, result)
   end
 
   # No more files, nothing waiting, queue is empty, we are done
-  defp spawn_compilers([], _output, _callback, [], [], result), do: result
+  defp spawn_compilers([], _output, _callback, [], [], _schedulers, result), do: result
 
   # Queued x, waiting for x: POSSIBLE ERROR! Release processes so we get the failures
-  defp spawn_compilers([], output, callback, waiting, queued, result) when length(waiting) == length(queued) do
+  defp spawn_compilers([], output, callback, waiting, queued, schedulers, result) when length(waiting) == length(queued) do
     Enum.each queued, fn { child, _ } -> child <- { :release, self() } end
-    wait_for_messages([], output, callback, waiting, queued, result)
+    wait_for_messages([], output, callback, waiting, queued, schedulers, result)
   end
 
   # No more files, but queue and waiting are not full or do not match
-  defp spawn_compilers([], output, callback, waiting, queued, result) do
-    wait_for_messages([], output, callback, waiting, queued, result)
+  defp spawn_compilers([], output, callback, waiting, queued, schedulers, result) do
+    wait_for_messages([], output, callback, waiting, queued, schedulers, result)
   end
 
   # Wait for messages from child processes
-  defp wait_for_messages(files, output, callback, waiting, queued, result) do
+  defp wait_for_messages(files, output, callback, waiting, queued, schedulers, result) do
     receive do
       { :compiled, child, file } ->
         callback.(file)
@@ -85,25 +90,43 @@ defmodule Kernel.ParallelCompiler do
         # Sometimes we may have spurious entries in the waiting
         # list because someone invoked try/rescue UndefinedFunctionError
         new_waiting = List.keydelete(waiting, child, 0)
-        spawn_compilers(files, output, callback, new_waiting, new_queued, result)
-      { :module_available, child, module, binary } ->
+        spawn_compilers(files, output, callback, new_waiting, new_queued, schedulers, result)
+      { :module_available, _child, module, binary } ->
         new_waiting = release_waiting_processes(module, waiting)
         new_result  = [{module, binary}|result]
-        wait_for_messages(files, output, callback, new_waiting, queued, new_result)
+        wait_for_messages(files, output, callback, new_waiting, queued, schedulers, new_result)
       { :waiting, child, on } ->
         new_waiting = OrdDict.store(child, on, waiting)
-        spawn_compilers(files, output, callback, new_waiting, queued, result)
+        spawn_compilers(files, output, callback, new_waiting, queued, schedulers, result)
       { :failure, child, kind, reason, stacktrace } ->
-        extra = if match?({^child, module}, List.keyfind(waiting, child, 0)) do
-          " (undefined module #{inspect module})"
+        if many_missing?(child, files, waiting, queued) do
+          IO.puts "== Compilation failed =="
+          IO.puts "Compilation failed on the following files:\n"
+
+          Enum.each Enum.reverse(queued), fn { pid, file } ->
+            case List.keyfind(waiting, pid, 0) do
+              { _, mod } -> IO.puts "* #{file} is missing module #{inspect mod}"
+              _ -> :ok
+            end
+          end
+
+          IO.puts "\nThe first failure is shown below..."
         end
 
-      {^child, file} = List.keyfind(queued, child, 0)
-      IO.puts "== Compilation error on file #{file}#{extra} =="
-      :erlang.raise(kind, reason, stacktrace)
+        {^child, file} = List.keyfind(queued, child, 0)
+        IO.puts "== Compilation error on file #{file} =="
+        :erlang.raise(kind, reason, stacktrace)
     end
   end
 
+  defp many_missing?(child, files, waiting, queued) do
+    waiting_length = length(waiting)
+
+    match?({ ^child, _ }, List.keyfind(waiting, child, 0)) and
+      waiting_length > 1 and files == [] and
+      waiting_length == length(queued)
+  end
+
   # Release waiting processes that are waiting for the given module
   defp release_waiting_processes(module, waiting) do
     Enum.filter waiting, fn { child, waiting_module } ->

lib/elixir/lib/kernel/parallel_require.ex

@@ -12,38 +12,55 @@ defmodule Kernel.ParallelRequire do
   can be optionally given as argument.
   """
   def files(files, callback // default_callback) do
-    spawn_requires(files, [], callback, [])
+    schedulers = max(:erlang.system_info(:schedulers_online), 2)
+    spawn_requires(files, [], callback, schedulers, [])
   end
 
-  defp spawn_requires([], [], _callback, result),     do: result
-  defp spawn_requires([], waiting, callback, result), do: wait_for_messages([], waiting, callback, result)
+  defp spawn_requires([], [], _callback, _schedulers, result), do: result
 
-  defp spawn_requires(files, waiting, callback, result) when length(waiting) >= 4 do
-    wait_for_messages(files, waiting, callback, result)
+  defp spawn_requires([], waiting, callback, schedulers, result) do
+    wait_for_messages([], waiting, callback, schedulers, result)
   end
 
-  defp spawn_requires([h|t], waiting, callback, result) do
+  defp spawn_requires(files, waiting, callback, schedulers, result) when length(waiting) >= schedulers do
+    wait_for_messages(files, waiting, callback, schedulers, result)
+  end
+
+  defp spawn_requires([h|t], waiting, callback, schedulers, result) do
     parent = self
 
-    child  = spawn_link fn ->
+    compiler_pid    = :erlang.get(:elixir_compiler_pid)
+    ensure_compiled = :erlang.get(:elixir_ensure_compiled)
+    { :error_handler, handler } = :erlang.process_info(parent, :error_handler)
+
+    child = spawn_link fn ->
+      if compiler_pid != :undefined do
+        :erlang.put(:elixir_compiler_pid, compiler_pid)
+      end
+
+      if ensure_compiled != :undefined do
+        :erlang.put(:elixir_ensure_compiled, ensure_compiled)
+      end
+
+      :erlang.process_flag(:error_handler, handler)
+
       try do
-        new    = Code.require_file(h)
-        result = (new || []) ++ result
-        callback.(h)
-        parent <- { :required, self }
+        new = Code.require_file(h)
+        parent <- { :required, self, new, h }
       catch
         kind, reason ->
           parent <- { :failure, self, kind, reason, System.stacktrace }
       end
     end
 
-    spawn_requires(t, [child|waiting], callback, result)
+    spawn_requires(t, [child|waiting], callback, schedulers, result)
   end
 
-  defp wait_for_messages(files, waiting, callback, result) do
+  defp wait_for_messages(files, waiting, callback, schedulers, result) do
     receive do
-      { :required, child } ->
-        spawn_requires(files, List.delete(waiting, child), callback, result)
+      { :required, child, new, file } ->
+        callback.(file)
+        spawn_requires(files, List.delete(waiting, child), callback, schedulers, (new || []) ++ result)
       { :failure, _child, kind, reason, stacktrace } ->
         :erlang.raise(kind, reason, stacktrace)
     end

lib/elixir/lib/kernel/record_rewriter.ex

@@ -0,0 +1,394 @@
+# This is an optimization Elixir runs on function clauses.
+# Whenever a variables matches against a record (a tagged
+# tuple), this information is stored in order to optimize
+# record calls.
+defmodule Kernel.RecordRewriter do
+  @moduledoc false
+
+  def optimize_clause(module, clause) do
+    optimize_clause(clause, module, :orddict.new)
+  end
+
+  ## Clause
+
+  defp optimize_clause({ :clause, line, args, guards, body }, module, dict) do
+    { args, dict } = optimize_args(args, module, dict)
+    { body, dict, res } = optimize_body(body, module, dict, [])
+    { { :clause, line, args, guards, body }, dict, res }
+  end
+
+  defp optimize_args(args, module, dict) do
+    Enum.map_reduce args, dict, fn(arg, acc) ->
+      { new_arg, new_acc, _res } = optimize_expr(arg, module, acc)
+      { new_arg, new_acc }
+    end
+  end
+
+  defp optimize_body([], _module, dict, _acc) do
+    { [], dict, nil }
+  end
+
+  defp optimize_body([h], module, dict, acc) do
+    { new_expr, new_dict, new_res } = optimize_expr(h, module, dict)
+    { Enum.reverse([new_expr|acc]), new_dict, new_res }
+  end
+
+  defp optimize_body([h|t], module, dict, acc) do
+    { new_expr, new_dict, _ } = optimize_expr(h, module, dict)
+    optimize_body(t, module, new_dict, [new_expr|acc])
+  end
+
+  ## Record helpers
+
+  defp record_fields(record, record) do
+    fields      = Module.get_attribute(record, :record_fields)
+    optimizable = Module.get_attribute(record, :record_optimizable)
+    unless nil?(fields) or nil?(optimizable) do
+      { (lc { k, _ } inlist fields, do: k), optimizable }
+    end
+  end
+
+  defp record_fields(_module, record) do
+    if Code.ensure_loaded?(record) && function_exported?(record, :__record__, 1) do
+      try do
+        fields      = lc { k, _ } inlist record.__record__(:fields), do: k
+        optimizable = record.__record__(:optimizable)
+        { fields, optimizable }
+      rescue
+        [UndefinedFunctionError, FunctionClauseError] -> nil
+      end
+    end
+  end
+
+  defp record_field_info(function) do
+    case atom_to_list(function) do
+      'update' -> nil
+      'update_' ++ _field -> nil
+      _ -> { :accessor, function }
+    end
+  end
+
+  defp optimize_call(line, module, { record, _ } = res, left, { :atom, _, function }, args) do
+    case record_fields(module, record) do
+      { fields, optimizable } ->
+        opt_call =
+          if List.member?(optimizable, { function, length(args) + 1 }) do
+            case record_field_info(function) do
+              { kind, field } ->
+                if index = Enum.find_index(fields, field == &1) do
+                  optimize_record_accessor_call(line, res, kind, field, index, left, args)
+                end
+              nil ->
+                optimize_record_other_call(line, record, res, function, left, args)
+            end
+          end
+
+        opt_call || optimize_record_other_call(line, record, nil, function, left, args)
+      nil ->
+        nil
+    end
+  end
+
+  defp optimize_call(_line, _module, _res, _left, _right, _args) do
+    nil
+  end
+
+  defp optimize_record_accessor_call(line, _res, :accessor, _field, index, left, []) do
+    call = { :call, line,
+      { :remote, line, { :atom, 0, :erlang }, { :atom, 0, :element } },
+      [{ :integer, 0, index + 2 }, left]
+    }
+    { call, nil }
+  end
+
+  defp optimize_record_accessor_call(line, res, :accessor, _field, index, left, [arg]) do
+    call = { :call, line,
+      { :remote, line, { :atom, 0, :erlang }, { :atom, 0, :setelement } },
+      [{ :integer, 0, index + 2 }, left, arg]
+    }
+    { call, res }
+  end
+
+  defp optimize_record_other_call(line, record, res, function, left, args) do
+    call = { :call, line,
+      { :remote, line, { :atom, line, record }, { :atom, 0, function } },
+      args ++ [left]
+    }
+    { call, res }
+  end
+
+  ## Expr
+
+  defp optimize_expr({ :call, call_line, { :remote, line, left, right }, args }, module, dict) do
+    { left, dict, res } = optimize_expr(left, module, dict)
+    { right, dict, _ } = optimize_expr(right, module, dict)
+    { args, dict } = optimize_args(args, module, dict)
+
+    case optimize_call(call_line, module, res, left, right, args) do
+      { call, call_res } ->
+        { call, dict, call_res }
+      nil ->
+        { { :call, call_line, { :remote, line, left, right }, args }, dict, nil }
+    end
+  end
+
+  defp optimize_expr({ :call, line, expr, args }, module, dict) do
+    { expr, dict, _ } = optimize_expr(expr, module, dict)
+    { args, dict } = optimize_args(args, module, dict)
+    { { :call, line, expr, args }, dict, nil }
+  end
+
+  defp optimize_expr({ :match, line, left, right }, module, dict) do
+    { left,  dict, left_res }  = optimize_expr(left, module, dict)
+    { right, dict, right_res } = optimize_expr(right, module, dict)
+
+    match = { :match, line, left, right }
+
+    if left_res do
+      dict = assign_vars(extract_vars(right, []), dict, left_res)
+    end
+
+    if right_res do
+      dict = assign_vars(extract_vars(left, []), dict, right_res)
+    end
+
+    { match, dict, right_res || left_res }
+  end
+
+  defp optimize_expr({ :op, line, op, left, right }, module, dict) do
+    { left, dict, _ }  = optimize_expr(left, module, dict)
+    { right, dict, _ } = optimize_expr(right, module, dict)
+    { { :op, line, op, left, right }, dict, nil }
+  end
+
+  defp optimize_expr({ :op, line, op, expr }, module, dict) do
+    { expr, dict, _ } = optimize_expr(expr, module, dict)
+    { { :op, line, op, expr }, dict, nil }
+  end
+
+  defp optimize_expr({ :bin, line, elements }, module, dict) do
+    { elements, dict } = optimize_args(elements, module, dict)
+    { { :bin, line, elements }, dict, nil }
+  end
+
+  defp optimize_expr({ :bin_element, line, expr, type1, type2 }, module, dict) do
+    { expr, dict, _ } = optimize_expr(expr, module, dict)
+    { { :bin_element, line, expr, type1, type2 }, dict, nil }
+  end
+
+  defp optimize_expr({ :cons, line, left, right }, module, dict) do
+    { left, dict, _ }  = optimize_expr(left, module, dict)
+    { right, dict, _ } = optimize_expr(right, module, dict)
+    { { :cons, line, left, right }, dict, nil }
+  end
+
+  defp optimize_expr({ :block, line, args }, module, dict) do
+    { args, dict, res } = optimize_body(args, module, dict, [])
+    { { :block, line, args }, dict, res }
+  end
+
+  defp optimize_expr({ :tuple, line, args }, module, dict) do
+    { args, dict, args_res } = optimize_tuple_args(args, module, dict)
+    args_res = if Enum.any?(args_res), do: args_res, else: nil
+
+    res =
+      case args do
+        [{ :atom, _, atom }|_] -> atom
+        _ -> nil
+      end
+
+    { { :tuple, line, args }, dict, { res, args_res } }
+  end
+
+  defp optimize_expr({ :var, _, name } = var, _module, dict) do
+    case :orddict.find(name, dict) do
+      { :ok, res } -> { var, dict, res }
+      :error       -> { var, dict, nil }
+    end
+  end
+
+  defp optimize_expr({ :case, line, expr, clauses }, module, dict) do
+    { expr, dict, _ } = optimize_expr(expr, module, dict)
+    tuples  = lc clause inlist clauses, do: optimize_clause(clause, module, dict)
+    clauses = lc { clause, _, _ } inlist tuples, do: clause
+    dict    = join_dict(tuples)
+    res     = join_result(tuples)
+    { { :case, line, expr, clauses }, dict, res }
+  end
+
+  defp optimize_expr({ :receive, line, clauses }, module, dict) do
+    tuples  = lc clause inlist clauses, do: optimize_clause(clause, module, dict)
+    clauses = lc { clause, _, _ } inlist tuples, do: clause
+    dict    = join_dict(tuples)
+    res     = join_result(tuples)
+    { { :receive, line, clauses }, dict, res }
+  end
+
+  defp optimize_expr({ :receive, line, clauses, after_key, after_value }, module, dict) do
+    tuples  = lc clause inlist clauses, do: optimize_clause(clause, module, dict)
+    clauses = lc { clause, _, _ } inlist tuples, do: clause
+
+    { after_key, dict, _ } = optimize_expr(after_key, module, dict)
+    { after_value, dict, res } = optimize_body(after_value, module, dict, [])
+
+    dict = join_dict(tuples, dict)
+    res  = join_result(tuples, res)
+
+    { { :receive, line, clauses, after_key, after_value }, dict, res }
+  end
+
+  defp optimize_expr({ :try, line, body, elses, catches, try_after }, module, dict) do
+    tuples  = lc clause inlist catches, do: optimize_clause(clause, module, dict)
+    catches = lc { clause, _, _ } inlist tuples, do: clause
+
+    tuples  = lc clause inlist elses, do: optimize_clause(clause, module, dict)
+    elses   = lc { clause, _, _ } inlist tuples, do: clause
+
+    { body, _, res } = optimize_body(body, module, dict, [])
+    res = join_result(tuples, res)
+
+    { try_after, _, _ } = optimize_body(try_after, module, dict, [])
+    { { :try, line, body, elses, catches, try_after }, dict, res }
+  end
+
+  defp optimize_expr({ :fun, line, { :function, receiver, name, arity } }, module, dict) do
+    { receiver, dict, _ } = optimize_expr(receiver, module, dict)
+    { name, dict, _ }     = optimize_expr(name, module, dict)
+    { arity, dict, _ }    = optimize_expr(arity, module, dict)
+    { { :fun, line, { :function, receiver, name, arity } }, dict, nil }
+  end
+
+  defp optimize_expr({ :fun, line, { :clauses, clauses } }, module, dict) do
+    clauses = lc clause inlist clauses do
+      { clause, _, _ } = optimize_clause(clause, module, dict)
+      clause
+    end
+
+    { { :fun, line, { :clauses, clauses } }, dict, nil }
+  end
+
+  defp optimize_expr({ comprehension, line, expr, args }, module, dict) when comprehension in [:lc, :bc] do
+    { args, new_dict } = optimize_args(args, module, dict)
+    { expr, _, _ } = optimize_expr(expr, module, new_dict)
+    { { comprehension, line, expr, args }, dict, nil }
+  end
+
+  defp optimize_expr({ generate, line, left, right }, module, dict) when generate in [:generate, :b_generate] do
+    { left, dict, _ }  = optimize_expr(left, module, dict)
+    { right, dict, _ } = optimize_expr(right, module, dict)
+    { { generate, line, left, right }, dict, nil }
+  end
+
+  defp optimize_expr(other, _module, dict) when elem(other, 0) in [:string, :atom, :integer, :float, :nil, :fun] do
+    { other, dict, nil }
+  end
+
+  ## Helpers
+
+  defp optimize_tuple_args(args, module, dict) do
+    { final_args, { final_dict, final_acc } } =
+      Enum.map_reduce args, { dict, [] }, fn(arg, { acc_dict, acc_res }) ->
+        { new_arg, new_acc, res } = optimize_expr(arg, module, acc_dict)
+        { new_arg, { new_acc, [res|acc_res] } }
+      end
+
+    { final_args, final_dict, Enum.reverse(final_acc) }
+  end
+
+  defp assign_vars([key|t], dict, { _, value } = res) when is_list(key) and is_list(value) and length(key) == length(value) do
+    assign_vars t, assign_nested_vars(key, dict, value), res
+  end
+
+  defp assign_vars([key|t], dict, { value, _ } = res) when is_atom(key) and value != nil do
+    dict =
+      case :orddict.find(key, dict) do
+        { :ok, ^res } ->
+          dict
+        { :ok, { ^value, _ } } ->
+          :orddict.store(key, { value, nil }, dict)
+        { :ok, _ } ->
+          # We are overriding a type of an existing variable,
+          # which means the source code is invalid.
+          :orddict.store(key, nil, dict)
+        :error ->
+          :orddict.store(key, res, dict)
+      end
+
+    assign_vars t, dict, res
+  end
+
+  defp assign_vars([_|t], dict, res) do
+    assign_vars t, dict, res
+  end
+
+  defp assign_vars([], dict, _res) do
+    dict
+  end
+
+  defp assign_nested_vars([vars|vt], dict, [res|rt]) do
+    assign_nested_vars(vt, assign_vars(vars, dict, res), rt)
+  end
+
+  defp assign_nested_vars([], dict, []) do
+    dict
+  end
+
+  defp extract_vars({ :match, _, left, right }, vars) do
+    vars = extract_vars(right, vars)
+    extract_vars(left, vars)
+  end
+
+  defp extract_vars({ :var, _, name }, vars) do
+    [name|vars]
+  end
+
+  defp extract_vars({ :tuple, _, args }, vars) do
+    [Enum.map(args, extract_vars(&1, []))|vars]
+  end
+
+  defp extract_vars(_, vars) do
+    vars
+  end
+
+  defp join_dict([{ _, dict, _ }|t]) do
+    join_dict(t, dict)
+  end
+
+  defp join_dict([{ _, dict, _ }|t], other) do
+    other = Enum.reduce other, other, fn
+      { key, { value, _ } = res }, acc ->
+        case :orddict.find(key, dict) do
+          { :ok, ^res } -> acc
+          { :ok, { ^value, _ } } -> :orddict.store(key, { value, nil }, acc)
+          { :ok, _ } -> :orddict.store(key, nil, acc)
+          :error -> :orddict.erase(key, acc)
+        end
+    end
+
+    join_dict(t, other)
+  end
+
+  defp join_dict([], other) do
+    other
+  end
+
+  defp join_result([{ _, _, res }|t]) do
+    join_result(t, res)
+  end
+
+  defp join_result([{ _, _, res }|t], res) do
+    join_result(t, res)
+  end
+
+  defp join_result([{ _, _, { res, _ } }|t], { res, _ }) do
+    join_result(t, { res, nil })
+  end
+
+  defp join_result([{ _, _, _ }|_], _res) do
+    nil
+  end
+
+  defp join_result([], res) do
+    res
+  end
+end

lib/elixir/lib/kernel/special_forms.ex

@@ -1,8 +1,12 @@
 defmodule Kernel.SpecialForms do
   @moduledoc """
-  In this module we define Elixir special forms. Those are called
-  special forms because they cannot be overridden by the developer
-  and sometimes have lexical scope (like `alias`, `import`, etc).
+  In this module we define Elixir special forms. Special forms
+  cannot be overriden by the developer and are the basic
+  building blocks of Elixir code.
+
+  Some of those forms are lexical (like `alias`, `import`, etc).
+  The macros `{}`, `[]` and `<<>>` are also special forms used
+  to define data structures, respectively tuples, lists and binaries.
 
   This module also documents Elixir's pseudo variables (`__MODULE__`,
   `__FILE__`, `__ENV__` and `__CALLER__`). Pseudo variables return
@@ -22,6 +26,7 @@ defmodule Kernel.SpecialForms do
 
       :{}.(1,2,3)
       { 1, 2, 3 }
+
   """
   defmacro :{}.(args)
 
@@ -32,6 +37,7 @@ defmodule Kernel.SpecialForms do
 
       :[].(1,2,3)
       [ 1, 2, 3 ]
+
   """
   defmacro :[].(args)
 
@@ -40,8 +46,126 @@ defmodule Kernel.SpecialForms do
 
   ## Examples
 
-      :<<>>.(1,2,3)
+      iex> << 1, 2, 3 >>
       << 1, 2, 3 >>
+
+  ## Bitstring types
+
+  A bitstring may contain many parts and those may have
+  specific types. Most of the time, Elixir will figure out
+  the part's type and won't require any work from you:
+
+      iex> <<102, "oo">>
+      "foo"
+
+  Above we have two parts: the first is an integer and the
+  second is a binary. If we use any other Elixir expression,
+  Elixir can no longer guess the type:
+
+      iex> rest = "oo"
+      ...> <<102, rest>>
+      ** (ArgumentError) argument error
+
+  When a variable or expression is given as a binary part,
+  Elixir defaults the type of that part to an unsigned
+  little-endian integer. In the example above, since we haven't
+  specified a type, Elixir expected an integer but we passed a
+  binary, resulting in `ArgumentError`. We can solve this by
+  explicitly tagging it as a binary:
+
+      <<102, rest :: binary>>
+
+  The type can be integer, float, binary, bytes, bitstring,
+  bits, utf8, utf16 or utf32, e.g.:
+
+      <<102 :: float, rest :: binary>>
+
+  Integer can be any arbitrary precision integer. A float is an
+  IEEE 754 binary32 or binary64 floating point number. A bitstring
+  is an arbitrary series of bits. A binary is a special case of
+  bitstring that has a total size divisible by 8.
+
+  The utf8, utf16, and utf32 types are for UTF code points.
+
+  The bits type is an alias for bitstring. The bytes type is an
+  alias for binary.
+
+  The signedness can also be given as signed or unsigned. The
+  signedness only matters for matching. If unspecified, it
+  defaults to unsigned. Example:
+
+      iex> <<-100 :: signed, _rest :: binary>> = <<-100, "foo">>
+      <<156,102,111,111>>
+
+  This match would have failed if we did not specify that the
+  value -100 is signed. If we're matching into a variable instead
+  of a value, the signedness won't be checked; rather, the number
+  will simply be interpreted as having the given (or implied)
+  signedness, e.g.:
+
+      iex> <<val, _rest :: binary>> = <<-100, "foo">>
+      ...> val
+      156
+
+  Here, `val` is interpreted as unsigned.
+
+  Signedness is only relevant on integers.
+
+  The endianness of a part can be big, little or native (the
+  latter meaning it will be resolved at VM load time). Passing
+  many options can be done by giving a list:
+
+      <<102 :: [integer, native], rest :: binary>>
+
+  Or:
+
+      <<102 :: [unsigned, big, integer], rest :: binary>>
+
+  And so on.
+
+  Endianness only makes sense for integers and some UTF code
+  point types (utf16 and utf32).
+
+  Finally, we can also specify size and unit for each part. The
+  unit is multiplied by the size to give the effective size of
+  the part:
+
+      iex> <<102, _rest :: [size(2), unit(8)]>> = "foo"
+      "foo"
+
+      iex> <<102, _rest :: size(16)>> = "foo"
+      "foo"
+
+      iex> <<102, _rest :: size(32)>> = "foo"
+      ** (MatchError) no match of right hand side value: "foo"
+
+  In the example above, the first two expressions matches
+  because the string "foo" takes 24 bits and we are matching
+  against a part of 24 bits as well, 8 of which are taken by
+  the integer 102 and the remaining 16 bits are specified on
+  the rest. On the last example, we expect a rest with size 32,
+  which won't match.
+
+  Size and unit are not applicable to utf8, utf16, and utf32.
+
+  The default size for integers is 8. For floats, it is 64. For
+  binaries, it is the size of the binary. Only the last binary
+  in a binary match can use the default size (all others must
+  have their size specified explicitly). Bitstrings do not have
+  a default size.
+
+  Size can also be specified using a syntax shortcut. Instead of
+  writing `size(8)`, one can write just `8` and it will be interpreted
+  as `size(8)`
+
+      iex> << 1 :: 3 >> == << 1 :: size(3) >>
+      true
+
+  The default unit for integers, floats, and bitstrings is 1. For
+  binaries, it is 8.
+
+  For floats, unit * size must result in 32 or 64, corresponding
+  to binary32 and binary64, respectively.
   """
   defmacro :<<>>.(args)
 
@@ -56,7 +180,7 @@ defmodule Kernel.SpecialForms do
         alias MyKeyword, as: Keyword
       end
 
-  In the example above, we have set up `MyOrdict` to be alias
+  In the example above, we have set up `MyKeyword` to be alias
   as `Keyword`. So now, any reference to `Keyword` will be
   automatically replaced by `MyKeyword`.
 
@@ -122,8 +246,9 @@ defmodule Kernel.SpecialForms do
   import those functions and reference them as local functions,
   for example:
 
-      import List
-      flatten([1,[2],3]) #=> [1,2,3]
+      iex> import List
+      ...> flatten([1,[2],3])
+      [1,2,3]
 
   ## Selector
 
@@ -139,8 +264,8 @@ defmodule Kernel.SpecialForms do
 
       import :all, List
 
-  It can also be customized to import only functions or only
-  macros:
+  It can also be customized to import only all functions or
+  all macros:
 
       import :functions, List
       import :macros, List
@@ -179,6 +304,13 @@ defmodule Kernel.SpecialForms do
   All imported modules are also required by default. `import`
   also accepts `as:` as an option so it automatically sets up
   an alias. Please check `alias` for more information.
+
+  ## Ambiguous function/macro names
+
+  If two modules `A` and `B` are imported and they both contain
+  a `foo` function with an arity of `1`, an error is only emitted
+  if an ambiguous call to `foo/1` is actually made; that is, the
+  errors are emitted lazily, not eagerly.
   """
   defmacro import(module, opts)
 
@@ -208,14 +340,14 @@ defmodule Kernel.SpecialForms do
 
   ## Examples
 
-      quote do: sum(1, 2, 3)
-      #=> { :sum, 0, [1, 2, 3] }
+      iex> quote do: sum(1, 2, 3)
+      { :sum, [], [1, 2, 3] }
 
-  ## Homoiconicity
+  ## Explanation
 
-  Elixir is an homoiconic language. Any Elixir program can be
-  represented using its own data structures. The building block
-  of Elixir homoiconicity is a tuple with three elements, for example:
+  Any Elixir code can be represented using Elixir data structures.
+  The building block of Elixir homoiconicity is a tuple with three
+  elements, for example:
 
       { :sum, 1, [1, 2, 3] }
 
@@ -227,8 +359,17 @@ defmodule Kernel.SpecialForms do
   * The second element of the tuple is always an integer
     representing the line number;
   * The third element of the tuple are the arguments for the
-    function call. The third argument may be an atom, meaning
-    that it may be a variable.
+    function call. The third argument may be an atom, which is
+    usually a variable (or a local call);
+
+  ## Options
+
+  * `:unquote` - When false, disables unquoting. Useful when you have a quote
+                 inside another quote and want to control which quote is
+                 able to unquote;
+  * `:location` - When set to `:keep`, keeps the current line and file on quotes.
+                  Read the Stacktrace information section below for more information;
+  * `:hygiene` - Allows a developer to disable hygiene selectively;
 
   ## Macro literals
 
@@ -244,9 +385,19 @@ defmodule Kernel.SpecialForms do
 
   ## Hygiene
 
-  Elixir macros are hygienic regarding to variables. This means
-  a variable defined in a macro cannot affect the scope where
-  the macro is included. Consider the following example:
+  Elixir macros are hygienic via means of deferred resolution.
+
+  This means aliases and imports defined inside the quoted refer
+  to the context that defined the macro and not the context
+  where the macro is expanded.
+
+  Furthermore, variables inside quote are also hygienic: a
+  variable defined in a macro cannot affect the variables where
+  the macro is expanded.
+
+  ### Hygiene in variables
+
+  Consider the following example:
 
       defmodule Hygiene do
         defmacro no_interference do
@@ -261,10 +412,10 @@ defmodule Kernel.SpecialForms do
       a #=> 10
 
   In the example above, `a` returns 10 even if the macro
-  is apparently setting it to 1 because the variables defined
-  in the macro does not affect the context the macro is
-  executed. If you want to set or get a variable, you can do
-  it with the help of the `var!` macro:
+  is apparently setting it to 1 because variables defined
+  in the macro does not affect the context the macro is executed.
+  If you want to set or get a variable in the user context, you
+  can do it with the help of the `var!` macro:
 
       defmodule NoHygiene do
         defmacro interference do
@@ -276,31 +427,145 @@ defmodule Kernel.SpecialForms do
 
       a = 10
       NoHygiene.interference
-      a #=> 11
+      a #=> 1
 
-  Notice that aliases are not hygienic in Elixir, ambiguity
-  must be solved by prepending Elixir:
+  It is important to understand that quoted variables are scoped
+  to the module they are defined. That said, even if two modules
+  define the same quoted variable `a`, their values are going
+  to be independent:
 
-      quote do
-        Elixir.Foo #=> Access the root Foo
-        Foo        #=> Access the Foo alias in the current module
-                       (if any is set), then fallback to Elixir.Foo
+      defmodule Hygiene1 do
+        defmacro var1 do
+          quote do: a = 1
+        end
       end
 
-  ## Options
+      defmodule Hygiene2 do
+        defmacro var2 do
+          quote do: a = 2
+        end
+      end
 
-  * `:hygiene` - When false, disables hygiene;
-  * `:unquote` - When false, disables unquoting. Useful when you have a quote
-    inside another quote and want to control which quote is able to unquote;
-  * `:location` - When set to `:keep`, keeps the current line and file on quotes.
-                  Read the Stacktrace information section below for more information;
+  Calling macros `var1` and `var2` are not going to change their
+  each other values for `a`. This is useful because quoted
+  variables from different modules cannot conflict. If you desire
+  to explicitly access a variable from another module, we can once
+  again use `var!` macro, but explicitly passing a second argument:
+
+      # Access the variable a from Hygiene1
+      quote do: var!(a, Hygiene1) = 2
+
+  Hygiene for variables can be disabled overall as:
+
+      quote hygiene: [vars: false], do: x
+
+  ### Hygiene in aliases
+
+  Aliases inside quote are hygienic by default.
+  Consider the following example:
+
+      defmodule Hygiene do
+        alias HashDict, as: D
+
+        defmacro no_interference do
+          quote do: D.new
+        end
+      end
+
+      require Hygiene
+      Hygiene.no_interference #=> #HashDict<[]>
+
+  Notice that, even though the alias `D` is not available
+  in the context the macro is expanded, the code above works
+  because `D` still expands to `HashDict`.
+
+  In some particular cases you may want to access an alias
+  or a module defined in the caller. In such scenarios, you
+  can access it by disabling hygiene with `hygiene: [aliases: false]`
+  or by using the `alias!` macro inside the quote:
+
+      defmodule Hygiene do
+        # This will expand to Elixir.Nested.hello
+        defmacro no_interference do
+          quote do: Nested.hello
+        end
+
+        # This will expand to Nested.hello for
+        # whatever is Nested in the caller
+        defmacro interference do
+          quote do: alias!(Nested).hello
+        end
+      end
+
+      defmodule Parent do
+        defmodule Nested do
+          def hello, do: "world"
+        end
+
+        require Hygiene
+        Hygiene.no_interference
+        #=> ** (UndefinedFunctionError) ...
+
+        Hygiene.interference
+        #=> "world"
+      end
+
+
+  ## Hygiene in imports
+
+  Similar to aliases, imports in Elixir hygienic. Consider the
+  following code:
+
+      defmodule Hygiene do
+        defmacrop get_size do
+          quote do
+            size("hello")
+          end
+        end
+
+        def return_size do
+          import Kernel, except: [size: 1]
+          get_size
+        end
+      end
+
+      Hygiene.return_size #=> 5
+
+  Notice how `return_size` returns 5 even though the `size/1`
+  function is not imported.
+
+  Elixir is smart enough to delay the resolution to the latest
+  moment possible. So, if you call `size("hello")` inside quote,
+  but no `size/1` function is available, it is then expanded on
+  the caller:
+
+      defmodule Lazy do
+        defmacrop get_size do
+          import Kernel, except: [size: 1]
+
+          quote do
+            size([a: 1, b: 2])
+          end
+        end
+
+        def return_size do
+          import Kernel, except: [size: 1]
+          import Dict, only: [size: 1]
+          get_size
+        end
+      end
+
+      Lazy.return_size #=> 2
+
+  As in aliases, imports expansion can be explicitly disabled
+  via the `hygiene: [imports: false]` option.
 
   ## Stacktrace information
 
   One of Elixir goals is to provide proper stacktrace whenever there is an
   exception. In order to work properly with macros, the default behavior
-  in quote is to set the line to 0. When a macro is invoked and the quoted
-  expressions is expanded, 0 is replaced by the line of the call site.
+  in quote is to not set a line. When a macro is invoked and the quoted
+  expressions is expanded, the call site line is inserted.
 
   This is a good behavior for the majority of the cases, except if the macro
   is defining new functions. Consider this example:
@@ -313,8 +578,8 @@ defmodule Kernel.SpecialForms do
   However, if there is an exception in any of these functions, we want
   the stacktrace to point to the `GenServer.Behaviour` and not the line
   that calls `use GenServer.Behaviour`. For this reason, there is an
-  option called `:location` that when set to `:keep` keeps these proper
-  semantics:
+  option called `:location` that when set to `:keep` keeps the original
+  line and file lines instead of setting them to 0:
 
       quote location: :keep do
         def handle_call(request, _from, state) do
@@ -324,87 +589,71 @@ defmodule Kernel.SpecialForms do
 
   It is important to warn though that `location: :keep` evaluates the
   code as if it was defined inside `GenServer.Behaviour` file, in
-  particular, the macro `__FILE__` will always point to
-  `GenServer.Behaviour` file.
+  particular, the macro `__FILE__` and exceptions happening inside
+  the quote will always point to `GenServer.Behaviour` file.
   """
-  defmacro quote(opts, do: contents)
+  defmacro quote(opts, block)
 
   @doc """
-  Unquotes the given expression from inside a macro.
-
-  ## Examples
-
-  Imagine the situation you have a variable `name` and
-  you want to inject it inside some quote. The first attempt
-  would be:
-
-      value = 13
-      quote do: sum(1, value, 3)
-
-  Which would then return:
-
-      { :sum, 0, [1, { :value, 0, quoted }, 3] }
-
-  Which is not the expected result. For this, we use unquote:
-
-      value = 13
-      quote do: sum(1, unquote(value), 3)
-      #=> { :sum, 0, [1, 13, 3] }
-
+  When used inside quoting, marks that the variable should
+  not be hygienized. The argument can be either a variable
+  node (i.e. a tuple with three elements where the last
+  one is an atom) or an atom representing the variable name.
+  Check `quote/2` for more information.
   """
-  defmacro unquote(expr)
+  defmacro var!(var)
 
   @doc """
-  Unquotes the given list expanding its arguments. Similar
-  to unquote.
-
-  ## Examples
-
-      values = [2,3,4]
-      quote do: sum(1, unquote_splicing(values), 5)
-      #=> { :sum, 0, [1, 2, 3, 4, 5] }
-
+  Defines a variable in the given context.
+  Check `quote/2` for more information.
   """
-  defmacro unquote_splicing(expr)
+  defmacro var!(var, context)
+
+  @doc """
+  When used inside quoting, marks that the alias should not
+  be hygienezed. This means the alias will be expanded when
+  the macro is expanded.
+  """
+  defmacro alias!(alias)
 
   @doc """
   List comprehensions allow you to quickly build a list from another list:
 
-      lc n inlist [1,2,3,4], do: n * 2
-      #=> [2,4,6,8]
+      iex> lc n inlist [1,2,3,4], do: n * 2
+      [2,4,6,8]
 
   A comprehension accepts many generators and also filters. Generators
   are defined using both `inlist` and `inbits` operators, allowing you
   to loop lists and bitstrings:
 
       # A list generator:
-      lc n inlist [1,2,3,4], do: n * 2
-      #=> [2,4,6,8]
+      iex> lc n inlist [1,2,3,4], do: n * 2
+      [2,4,6,8]
 
       # A bit string generator:
-      lc <<n>> inbits <<1,2,3,4>>, do: n * 2
-      #=> [2,4,6,8]
+      iex> lc <<n>> inbits <<1,2,3,4>>, do: n * 2
+      [2,4,6,8]
 
       # A generator from a variable:
-      list = [1,2,3,4]
-      lc n inlist list, do: n * 2
-      #=> [2,4,6,8]
+      iex> list = [1,2,3,4]
+      ...> lc n inlist list, do: n * 2
+      [2,4,6,8]
 
       # A comprehension with two generators
-      lc x inlist [1,2], y inlist [2,3], do: x*y
-      #=> [2,3,4,6]
+      iex> lc x inlist [1,2], y inlist [2,3], do: x*y
+      [2,3,4,6]
 
   Filters can also be given:
 
       # A comprehension with a generator and a filter
-      lc n inlist [1,2,3,4,5,6], rem(n, 2) == 0, do: n
-      #=> [2,4,6]
+      iex> lc n inlist [1,2,3,4,5,6], rem(n, 2) == 0, do: n
+      [2,4,6]
 
   Bit string generators are quite useful when you need to
   organize bit string streams:
 
       iex> pixels = <<213,45,132,64,76,32,76,0,0,234,32,15>>
-      iex> lc <<r:8,g:8,b:8>> inbits pixels, do: {r,g,b}
+      iex> lc <<r::8,g::8,b::8>> inbits pixels, do: {r,g,b}
       [{213,45,132},{64,76,32},{76,0,0},{234,32,15}]
 
   """
@@ -416,7 +665,7 @@ defmodule Kernel.SpecialForms do
   be a bitstring. For example, here is how to remove all
   spaces from a string:
 
-      bc <<c>> inbits " hello world ", c != ? , do: <<c>>
+      iex> bc <<c>> inbits " hello world ", c != ? , do: <<c>>
       "helloworld"
 
   """
@@ -427,8 +676,8 @@ defmodule Kernel.SpecialForms do
   of expressions in Elixir. This special form is private
   and should not be invoked directly:
 
-      quote do: (1; 2; 3)
-      #=> { :__block__, 0, [1,2,3] }
+      iex> quote do: (1; 2; 3)
+      { :__block__, [], [1,2,3] }
 
   """
   defmacro __block__(args)
@@ -440,7 +689,7 @@ defmodule Kernel.SpecialForms do
   it belonged to another file.
 
       quote location: :keep, do: 1
-      #=> { :__scope__, 1,[[file: "iex"], 1] }
+      #=> { :__scope__, 1,[[file: "iex"],[do: 1]] }
 
   Check `quote/1` for more information.
   """
@@ -448,11 +697,39 @@ defmodule Kernel.SpecialForms do
 
   @doc """
   This is the special form used to hold aliases information.
-  At compilation time, it is usually compiled to an atom:
+  It is usually compiled to an atom:
 
-      quote do: Foo.Bar
-      { :__aliases__, 0, [:Foo,:Bar] }
+      quote do: Foo.Bar #=>
+      { :__aliases__, [], [:Foo,:Bar] }
+
+  Elixir represents `Foo.Bar` as `__aliases__` so calls can be
+  unambiguously identified by the operator `:.`. For example:
+
+      quote do: Foo.bar #=>
+      {{:.,[],[{:__aliases__,[],[:Foo]},:bar]},[],[]}
+
+  Whenever an expression iterator sees a `:.` as the tuple key,
+  it can be sure that it represents a call and the second argument
+  is the list is an atom.
+
+  On the other hand, aliases holds some properties:
+
+  1) The head element of aliases can be any term;
+
+  2) The tail elements of aliases are guaranteed to always be atoms;
+
+  3) When the head element of aliases is the atom :Elixir, no expansion happen;
+
+  4) When the head element of aliases is not an atom, it is expanded at runtime:
+
+        quote do: some_var.Foo
+        {:__aliases__,[],[{:some_var,[],:quoted},:Bar]}
+
+     Since `some_var` is not available at compilation time, the compiler
+     expands such expression to:
+
+        Module.concat [some_var, Foo]
 
   """
   defmacro __aliases__(args)
-end
+end

lib/elixir/lib/kernel/typespec.ex

@@ -1,140 +1,648 @@
 defmodule Kernel.Typespec do
   @moduledoc """
-  This is the module that converts Elixir typespecs
-  to Erlang typespecs syntax. Everytime @spec, @type
-  and @typep are used they proxy to the functions
-  in this module.
+  Holds macros and functions for working with typespecs.
+
+  The attributes `@type`, `@opaque`, `@typep`, `@spec` and
+  `@callback` available in modules are handled by the equivalent
+  macros defined by this module.
+
+  ## Defining a type
+
+  @type type_name :: type
+  @typep type_name :: type
+  @opaque type_name :: type
+
+  For more details, see documentation for deftype, deftypep and defopaque in
+  Kernel.Typespec
+
+  ## Defining a specification
+
+  @spec function_name(type, type) :: type
+  @callback function_name(type, type) :: type
+
+  For more details, see documentation for defspec and defcallback in
+  Kernel.Typespec
+
+  ## Types
+
+  The type syntax provided by Elixir is fairly similar to the one
+  in Erlang.
+
+  Most of the built-in types provided in Erlang (for example, `pid()`)
+  are expressed the same way: `pid()` or simply `pid`. Parametrized types
+  are also supported: `list(integer())` and so are remote types: `Enum.t`.
+
+  Certain data type shortcuts ([...], <<>> and {...}) are supported as well.
+
+  Main differences lie in how bit strings and functions are defined:
+
+  ### Bit Strings
+
+  Bit string with a base size of 3:
+
+      <<_ :: 3>>
+
+  Bit string with a unit size of 8:
+
+      <<_ :: _ * 8>>
+
+  ### Functions
+
+  Any function:
+
+      (fun(...) -> any)
+      or
+      ((...) -> any)
+      or
+      (... -> any)
+
+  Function with arity of zero:
+
+      (fun() -> type)
+      or
+      (() -> type)
+
+  Function with some arity:
+
+      (fun(type, type) -> type)
+      or
+      ((type, type) -> type)
+      or
+      (type, type -> type)
+
+  ## Notes
+
+  Elixir discourages the use of type `string()` as it might be confused
+  with binaries which are referred to as "strings" in Elixir (as opposed to
+  character lists). In order to use the type that is called `string()` in Erlang,
+  one has to use the `char_list()` type which is a synonym for `string()`. If you
+  use `string()`, you'll get a warning from the compiler.
+
+  If you want to refer to the "string" type (the one operated by functions in the
+  String module), use `String.t()` type instead.
+
+  See http://www.erlang.org/doc/reference_manual/typespec.html
+  for more information.
   """
 
-  defmacro deftype(name, options // []) do
-    _deftype(name, true, __CALLER__, options)
-  end
+  @doc """
+  Defines a type.
+  This macro is the one responsible to handle the attribute @type.
 
-  defmacro deftypep(name) do
-    _deftype(name, false, __CALLER__, [])
-  end
+  ## Examples
 
-  defmacro defspec(spec, block) do
-    _defspec(:spec, __CALLER__, spec, block)
-  end
+      @type my_type :: atom
 
-  defmacro defcallback(spec, block) do
-    _defspec(:callback, __CALLER__, spec, block)
+  """
+  defmacro deftype(type) do
+    quote do
+      Kernel.Typespec.deftype(:type, unquote(Macro.escape type), __ENV__)
+    end
   end
 
   @doc """
-  Get the types defined for the given module. This function
-  is only available for modules being compiled. If the module
-  was already compiled, you need to loop its attributes
-  to get such information.
+  Defines an opaque type.
+  This macro is the one responsible to handle the attribute @opaque.
+
+  ## Examples
+
+      @opaque my_type :: atom
+
   """
-  def get_types(module) do
-    Module.get_attribute(module, :type) ++ Module.get_attribute(module, :opaque)
+  defmacro defopaque(type) do
+    quote do
+      Kernel.Typespec.deftype(:opaque, unquote(Macro.escape type), __ENV__)
+    end
   end
 
   @doc """
-  Get the specs defined for the given module. This function
-  is only available for modules being compiled. If the module
-  was already compiled, you need to loop its attributes
-  to get such information.
+  Defines a private type.
+  This macro is the one responsible to handle the attribute @typep.
+
+  ## Examples
+
+      @typep my_type :: atom
+
   """
-  def get_specs(module) do
-    specs = :ets.tab2list(spec_table_for(module))
-    keys  = :lists.ukeysort(1, specs)
-    lc { k, _ } inlist keys, do: { k, :proplists.append_values(k, specs) }
+  defmacro deftypep(type) do
+    quote do
+      Kernel.Typespec.deftype(:typep, unquote(Macro.escape type), __ENV__)
+    end
   end
 
-  ## Typespec conversion
+  @doc """
+  Defines a spec.
+  This macro is the one responsible to handle the attribute @spec.
+
+  ## Examples
+
+      @spec add(number, number) :: number
+
+  """
+  defmacro defspec(spec) do
+    quote do
+      Kernel.Typespec.defspec(:spec, unquote(Macro.escape spec), __ENV__)
+    end
+  end
+
+  @doc """
+  Defines a callback.
+  This macro is the one responsible to handle the attribute @callback.
+
+  ## Examples
+
+      @callback add(number, number) :: number
+
+  """
+  defmacro defcallback(spec) do
+    quote do
+      Kernel.Typespec.defspec(:callback, unquote(Macro.escape spec), __ENV__)
+    end
+  end
+
+  ## Helpers
+
+  @doc """
+  Defines a `type`, `typep` or `opaque` by receiving Erlang's typespec.
+  """
+  def define_type(module, kind, { name, _, vars } = type) when kind in [:type, :typep, :opaque] do
+    { kind, export } =
+      case kind do
+        :type   -> { :type, true }
+        :typep  -> { :type, false }
+        :opaque -> { :opaque, true }
+      end
+
+    Module.compile_typespec module, kind, type
+    if export, do:
+      Module.compile_typespec module, :export_type, [{ name, length(vars) }]
+    type
+  end
+
+  @doc """
+  Defines a `spec` by receiving Erlang's typespec.
+  """
+  def define_spec(module, tuple, definition) do
+    Module.compile_typespec module, :spec, { tuple, definition }
+  end
+
+  @doc """
+  Defines a `callback` by receiving Erlang's typespec.
+  """
+  def define_callback(module, tuple, definition) do
+    Module.compile_typespec module, :callback, { tuple, definition }
+  end
+
+  @doc """
+  Returns true if the current module defines a given type
+  (private, opaque or not). This function is only available
+  for modules being compiled.
+  """
+  def defines_type?(module, name, arity) do
+    finder = match?({ ^name, _, vars } when length(vars) == arity, &1)
+    :lists.any(finder, Module.get_attribute(module, :type)) or
+      :lists.any(finder, Module.get_attribute(module, :opaque))
+  end
+
+  @doc """
+  Returns true if the current module defines a given spec.
+  This function is only available for modules being compiled.
+  """
+  def defines_spec?(module, name, arity) do
+    tuple = { name, arity }
+    :lists.any(match?(^tuple, &1), Module.get_attribute(module, :spec))
+  end
+
+  @doc """
+  Returns true if the current module defines a callback.
+  This function is only available for modules being compiled.
+  """
+  def defines_callback?(module, name, arity) do
+    tuple = { name, arity }
+    :lists.any(match?(^tuple, &1), Module.get_attribute(module, :callback))
+  end
+
+  @doc """
+  Converts a spec clause back to Elixir AST.
+  """
+  def spec_to_ast(name, { :type, line, :fun, [{:type, _, :product, args},result] }) do
+    args = lc arg inlist args, do: typespec_to_ast(arg)
+    { :::, [line: line], [{ name, [line: line], args }, typespec_to_ast(result)] }
+  end
+
+  def spec_to_ast(name, { :type, line, :fun, [] }) do
+    { :::, [line: line], [{ name, [line: line], [] }, quote(do: term)] }
+  end
+
+  def spec_to_ast(name, { :type, line, :bounded_fun, [{ :type, _, :fun, [{ :type, _, :product, args }, result] }, constraints] }) do
+    [h|t] =
+      lc {:type, line, :constraint, [{:atom, _, :is_subtype}, [var, type]]} inlist constraints do
+        { :is_subtype, [line: line], [typespec_to_ast(var), typespec_to_ast(type)] }
+      end
+
+    args = lc arg inlist args, do: typespec_to_ast(arg)
+    guards = Enum.reduce t, h, fn(x, acc) -> { :and, line, [acc, x] } end
+
+    { :::, [line: line], [{ :when, [line: line], [{ name, [line: line], args }, guards] }, typespec_to_ast(result)] }
+  end
+
+  @doc """
+  Converts a type clause back to Elixir AST.
+  """
+  def type_to_ast({ { :record, record }, fields, args }) when is_atom(record) do
+    fields = lc field inlist fields, do: typespec_to_ast(field)
+    args = lc arg inlist args, do: typespec_to_ast(arg)
+    type = { :{}, [], [record|fields] }
+    quote do: unquote(record)(unquote_splicing(args)) :: unquote(type)
+  end
+
+  def type_to_ast({ name, type, args }) do
+    args = lc arg inlist args, do: typespec_to_ast(arg)
+    quote do: unquote(name)(unquote_splicing(args)) :: unquote(typespec_to_ast(type))
+  end
+
+  @doc """
+  Returns all types available from the beam.
+
+  It is returned as a list of tuples where the first
+  element is the type (`:typep`, `:type` and `:opaque`).
+
+  The module has to have a corresponding beam file
+  on the file system.
+  """
+  def beam_types(module) do
+    case abstract_code(module) do
+      { :ok, abstract_code } ->
+        exported_types = lc { :attribute, _, :export_type, types } inlist abstract_code, do: types
+        exported_types = List.flatten(exported_types)
+
+        lc { :attribute, _, kind, { name, _, args } = type } inlist abstract_code, kind in [:opaque, :type] do
+          cond do
+            kind == :opaque -> { :opaque, type }
+            List.member?(exported_types, { name, length(args) }) -> { :type, type }
+            true -> { :typep, type }
+          end
+        end
+      _ ->
+        []
+    end
+  end
+
+  @doc """
+  Returns all specs available from the beam.
+
+  It is returned as a list of tuples where the first
+  element is spec name and arity and the second is the spec.
+
+  The module has to have a corresponding beam file
+  on the file system.
+  """
+  def beam_specs(module) do
+    from_abstract_code(module, :spec)
+  end
+
+  @doc """
+  Returns all callbacks available from the beam.
+
+  It is returned as a list of tuples where the first
+  element is spec name and arity and the second is the spec.
+
+  The module has to have a corresponding beam file
+  on the file system.
+  """
+  def beam_callbacks(module) do
+    from_abstract_code(module, :callback)
+  end
+
+  defp from_abstract_code(module, kind) do
+    case abstract_code(module) do
+      { :ok, abstract_code } ->
+        lc { :attribute, _, abs_kind, value } inlist abstract_code, kind == abs_kind, do: value
+      _ ->
+        []
+    end
+  end
+
+  defp abstract_code(module) do
+    case :beam_lib.chunks(abstract_code_beam(module), [:abstract_code]) do
+      {:ok, { _, [{ :abstract_code, { _raw_abstract_v1, abstract_code } }] } } ->
+        { :ok, abstract_code }
+      _ ->
+        []
+    end
+  end
+
+  defp abstract_code_beam(module) when is_atom(module) do
+    case :code.which(module) do
+      :non_existing -> module
+      file -> file
+    end
+  end
+
+  defp abstract_code_beam(binary) when is_binary(binary) do
+    binary
+  end
+
+  ## Macro callbacks
+
+  @doc false
+  def deftype(kind, { :::, _, [type, definition] }, caller) do
+    do_deftype(kind, type, definition, caller)
+  end
+
+  def deftype(kind, {name, _meta, args} = type, caller) when 
+                                                        is_atom(name) and
+                                                        not is_list(args) do
+    do_deftype(kind, type, { :term, [line: caller.line], nil }, caller)
+  end
+
+  def deftype(_kind, other, _caller) do
+    type_spec = Macro.to_binary(other)
+    raise ArgumentError, message: "invalid type specification #{type_spec}"
+  end
+
+  defp do_deftype(kind, { name, _, args }, definition, caller) do
+    args =
+      if is_atom(args) do
+        []
+      else
+        lc(arg inlist args, do: variable(arg))
+      end
+
+    vars = lc { :var, _, var } inlist args, do: var
+    spec = typespec(definition, vars, caller)
+
+    vars = lc { :var, _, _ } = var inlist args, do: var
+    type = { name, spec, vars }
+
+    define_type(caller.module, kind, type)
+  end
+
+  @doc false
+  def defspec(type, {:::, _, [{ :when, _, [{ name, meta, args }, constraints_guard] }, return] }, caller) do
+    if is_atom(args), do: args = []
+    constraints = guard_to_constraints(constraints_guard, caller)
+    spec = { :type, line(meta), :fun, fn_args(meta, args, return, Keyword.keys(constraints), caller) }
+    spec = { :type, line(meta), :bounded_fun, [spec, Keyword.values(constraints)] }
+    code = { { name, Kernel.length(args) }, spec }
+    Module.compile_typespec(caller.module, type, code)
+    code
+  end
+
+  def defspec(type, {:::, _, [{ name, meta, args }, return]}, caller) do
+    if is_atom(args), do: args = []
+    spec = { :type, line(meta), :fun, fn_args(meta, args, return, [], caller) }
+    code = { { name, Kernel.length(args) }, spec }
+    Module.compile_typespec(caller.module, type, code)
+    code
+  end
+
+  def defspec(_type, other, _caller) do
+    spec = Macro.to_binary(other)
+    raise ArgumentError, message: "invalid function type specification #{spec}"
+  end
+
+  defp guard_to_constraints({ :is_subtype, meta, [{ name, _, _ }, type] }, caller) do
+    line = line(meta)
+    contraints = [{ :atom, line, :is_subtype }, [{:var, line, name}, typespec(type, [], caller)]]
+    [{ name, { :type, line, :constraint, contraints } }]
+  end
+
+  defp guard_to_constraints({ :and, _, [left, right] }, caller) do
+    guard_to_constraints(left, caller) ++ guard_to_constraints(right, caller)
+  end
+
+  ## To AST conversion
+
+  defp typespec_to_ast({ :type, line, :tuple, :any }) do
+    typespec_to_ast({:type, line, :tuple, []})
+  end
+
+  defp typespec_to_ast({ :type, line, :tuple, args }) do
+    args = lc arg inlist args, do: typespec_to_ast(arg)
+    { :{}, [line: line], args }
+  end
+
+  defp typespec_to_ast({ :type, line, :binary, [arg1, arg2] }) do
+    [arg1, arg2] = lc arg inlist [arg1, arg2], do: typespec_to_ast(arg)
+    cond do
+      arg2 == 0 ->
+        quote line: line, do: <<_ :: unquote(arg1)>>
+      arg1 == 0 ->
+        quote line: line, do: <<_ :: _ * unquote(arg2)>>
+      true ->
+        quote line: line, do: <<_ :: unquote(arg1) * unquote(arg2)>>
+    end
+  end
+
+  defp typespec_to_ast({ :type, line, :union, args }) do
+    args = lc arg inlist args, do: typespec_to_ast(arg)
+    Enum.reduce tl(args), hd(args),
+      fn(arg, expr) -> { :|, [line: line], [expr, arg] } end
+  end
+
+  defp typespec_to_ast({ :type, line, :fun, [{:type, _, :product, args},result] }) do
+    args = lc arg inlist args, do: typespec_to_ast(arg)
+    { :"->", [line: line], [{args, typespec_to_ast(result)}] }
+  end
+
+  defp typespec_to_ast({ :type, line, :fun, [args, result] }) do
+    { :"->", [line: line], [{[typespec_to_ast(args)], typespec_to_ast(result)}] }
+  end
+
+  defp typespec_to_ast({ :type, line, :fun, [] }) do
+    typespec_to_ast({ :type, line, :fun, [{:type, line, :any}, {:type,line,:any, []} ] })
+  end
+
+  defp typespec_to_ast({ :type, line, :range, [left, right] }) do
+    { :"..", [line: line], [typespec_to_ast(left), typespec_to_ast(right)] }
+  end
+
+  defp typespec_to_ast({ :type, line, name, args }) do
+    args = lc arg inlist args, do: typespec_to_ast(arg)
+    { name, [line: line], args }
+  end
+
+  defp typespec_to_ast({ :var, line, var }) do
+    var =
+    case atom_to_binary(var) do
+      <<"_", c :: [binary, size(1)], rest :: binary>> ->
+        binary_to_atom("_#{String.downcase(c)}#{rest}")
+      <<c :: [binary, size(1)], rest :: binary>> ->
+        binary_to_atom("#{String.downcase(c)}#{rest}")
+    end
+    { var, line, nil }
+  end
+
+  # special shortcut(s)
+  defp typespec_to_ast({ :remote_type, line, [{:atom, _, :elixir}, {:atom, _, :char_list}, []] }) do
+    typespec_to_ast({:type, line, :char_list, []})
+  end
+
+  defp typespec_to_ast({ :remote_type, line, [{:atom, _, :elixir}, {:atom, _, :as_boolean}, [arg]] }) do
+    typespec_to_ast({:type, line, :as_boolean, [arg]})
+  end
+
+
+  defp typespec_to_ast({ :remote_type, line, [mod, name, args] }) do
+    args = lc arg inlist args, do: typespec_to_ast(arg)
+    dot  = { :., [line: line], [typespec_to_ast(mod), typespec_to_ast(name)] }
+    { dot, [line: line], args }
+  end
+
+  defp typespec_to_ast({ :ann_type, line, [var, type] }) do
+    { :::, [line: line], [typespec_to_ast(var), typespec_to_ast(type)] }
+  end
+
+  defp typespec_to_ast({ :typed_record_field,
+                         { :record_field, line, { :atom, line1, name }},
+                         type }) do
+    typespec_to_ast({ :ann_type, line, [{ :var, line1, name }, type] })
+  end
+
+  defp typespec_to_ast({:type, _, :any}) do
+    quote do: ...
+  end
+
+  defp typespec_to_ast({:paren_type, _, [type]}) do
+    typespec_to_ast(type)
+  end
+
+  defp typespec_to_ast({ t, _line, atom }) when is_atom(t) do
+    atom
+  end
+
+  defp typespec_to_ast(other), do: other
+
+  ## From AST conversion
+
+  defp line(meta) do
+    case :lists.keyfind(:line, 1, meta) do
+      { :line, line } -> line
+      false -> 0
+    end
+  end
 
   # Handle unions
-  defp typespec({ :|, line, [_,_] } = exprs, vars, caller) do
-    exprs = Enum.reverse(collect_union(exprs))
+  defp typespec({ :|, meta, [_,_] } = exprs, vars, caller) do
+    exprs = :lists.reverse(collect_union(exprs))
     union = lc e inlist exprs, do: typespec(e, vars, caller)
-    { :type, line, :union, union }
+    { :type, line(meta), :union, union }
   end
 
   # Handle binaries
-  defp typespec({:<<>>, line, []}, _,_) do
-     {:type, line, :binary, [{:integer, line, 0}, {:integer, line, 0}]}
+  defp typespec({:<<>>, meta, []}, _,_) do
+    {:type, line(meta), :binary, [{:integer, line(meta), 0}, {:integer, line(meta), 0}]}
   end
 
-  defp typespec({:<<>>, line, [{:|, _, [{:_, line1, atom}, {:*, _, [{:_, line2, atom}, unit]}]}]}, _, _) when is_atom(atom) do
-     {:type, line, :binary, [{:integer, line1, 0}, {:integer, line2, unit}]}
+  defp typespec({:<<>>, meta, [{:::, _, [{:_, meta1, atom}, {:*, _, [{:_, meta2, atom}, unit]}]}]}, _, _) when is_atom(atom) do
+    {:type, line(meta), :binary, [{:integer, line(meta1), 0}, {:integer, line(meta2), unit}]}
   end
 
-  defp typespec({:<<>>, line, [{:|, line1, [{:_, line2, atom}, base]}]}, _, _) when is_atom(atom) do
-     {:type, line, :binary, [{:integer, line1, base}, {:integer, line2, 0}]}
+  defp typespec({:<<>>, meta, [{:::, meta1, [{:_, meta2, atom}, base]}]}, _, _) when is_atom(atom) do
+    {:type, line(meta), :binary, [{:integer, line(meta1), base}, {:integer, line(meta2), 0}]}
   end
 
   # Handle ranges
-  defp typespec({:"..", line, args}, vars, caller) do
-    typespec({:range, line, args}, vars, caller)
+  defp typespec({:"..", meta, args}, vars, caller) do
+    typespec({:range, meta, args}, vars, caller)
+  end
+
+  # Handle special forms
+  defp typespec({:__MODULE__, _, atom}, vars, caller) when is_atom(atom) do
+    typespec(caller.module, vars, caller)
   end
 
-  # Handle aliases
   defp typespec({:__aliases__, _, _} = alias, vars, caller) do
     atom = Macro.expand alias, caller
     typespec(atom, vars, caller)
   end
 
+  # Handle funs
+
+  defp typespec({:->, meta, [{[{:fun, _, arguments}], return}]}, vars, caller) when is_list(arguments) do
+    typespec({:->, meta, [{arguments, return}]}, vars, caller)
+  end
+
+  defp typespec({:->, meta, [{arguments, return}]}, vars, caller) when is_list(arguments) do
+    args = fn_args(meta, arguments, return, vars, caller)
+    { :type, line(meta), :fun, args }
+  end
+
   # Handle type operator
-  defp typespec({:"::", line, [var, expr] }, vars, caller) do
+  defp typespec({:"::", meta, [var, expr] }, vars, caller) do
     left  = typespec(var, [elem(var, 0)|vars], caller)
     right = typespec(expr, vars, caller)
-    { :ann_type, line, [left, right] }
+    { :ann_type, line(meta), [left, right] }
   end
 
   # Handle unary ops
-  defp typespec({op, line, [integer]}, _, _) when op in [:+, :-] and is_integer(integer) do
-    { :op, line, op, {:integer, line, integer} }
+  defp typespec({op, meta, [integer]}, _, _) when op in [:+, :-] and is_integer(integer) do
+    { :op, line(meta), op, {:integer, line(meta), integer} }
+  end
+
+  # Handle access macro
+  defp typespec({{:., meta, [Kernel, :access]}, meta1, [target, args]}, vars, caller) do
+    access = {{:., meta, [Kernel, :access]}, meta1,
+              [target, args ++ [_: { :any, [], [] }]]}
+    typespec(Macro.expand(access, caller), vars, caller)
   end
 
   # Handle remote calls
-  defp typespec({{:., line, [remote, name]}, _, args}, vars, caller) do
+  defp typespec({{:., meta, [remote, name]}, _, args}, vars, caller) do
     remote = Macro.expand remote, caller
-    unless is_atom(remote), do: raise(ArgumentError, message: "Invalid remote in typespec")
-    remote_type({typespec(remote, vars, caller), line, typespec(name, vars, caller), args}, vars, caller)
+    unless is_atom(remote), do: raise ArgumentError, message: "invalid remote in typespec"
+    remote_type({typespec(remote, vars, caller), meta, typespec(name, vars, caller), args}, vars, caller)
   end
 
   # Handle tuples
-  defp typespec({:tuple, line, atom}, vars, caller) when is_atom(atom) do
-    typespec({:{}, line, []}, vars, caller)
+  defp typespec({:tuple, meta, atom}, vars, caller) when is_atom(atom) do
+    typespec({:{}, meta, []}, vars, caller)
   end
 
-  defp typespec({:{}, line, []}, _, _) do
-    { :type, line, :tuple, :any }
+  defp typespec({:{}, meta, []}, _, _) do
+    { :type, line(meta), :tuple, :any }
   end
 
-  defp typespec({:{}, line, t}, vars, caller) when is_list(t) do
+  defp typespec({:{}, meta, t}, vars, caller) when is_list(t) do
     args = lc e inlist t, do: typespec(e, vars, caller)
-    { :type, line, :tuple, args }
+    { :type, line(meta), :tuple, args }
   end
 
-  # Handle funs
-  defp typespec({:fun, line, arguments}, vars, caller) when is_list(arguments) do
-    args =
-      case Enum.reverse(arguments) do
-        [[{:do,h}]|t] -> fn_args(line, Enum.reverse(t), h, vars, caller)
-        [] -> []
-        _  -> [fn_args(line, arguments, vars, caller)]
-      end
+  # Handle blocks
 
-    { :type, line, :fun, args }
+  defp typespec({:__block__, _meta, [arg]}, vars, caller) do
+    typespec(arg, vars, caller)
   end
 
   # Handle variables or local calls
-  defp typespec({name, line, atom}, vars, caller) when is_atom(atom) do
+  defp typespec({name, meta, atom}, vars, caller) when is_atom(atom) do
     if List.member?(vars, name) do
-       { :var, line, name }
+      { :var, line(meta), name }
     else
-      typespec({name, line, []}, vars, caller)
+      typespec({name, meta, []}, vars, caller)
     end
   end
 
   # Handle local calls
-  defp typespec({name, line, arguments}, vars, caller) do
+  defp typespec({:string, meta, arguments}, vars, caller) do
+    IO.write "warning: string() type use is discouraged. For character lists, use " <>
+      "char_list() type, for strings, String.t()\n#{Exception.format_stacktrace(caller.stacktrace)}"
     arguments = lc arg inlist arguments, do: typespec(arg, vars, caller)
-    { :type, line, name, arguments }
+    { :type, line(meta), :string, arguments }
+  end
+
+  defp typespec({:char_list, _meta, arguments}, vars, caller) do
+    typespec((quote do: :elixir.char_list(unquote_splicing(arguments))), vars, caller)
+  end
+
+  defp typespec({:as_boolean, _meta, arguments}, vars, caller) do
+    typespec((quote do: :elixir.as_boolean(unquote_splicing(arguments))), vars, caller)
+  end
+
+  defp typespec({name, meta, arguments}, vars, caller) do
+    arguments = lc arg inlist arguments, do: typespec(arg, vars, caller)
+    { :type, line(meta), name, arguments }
   end
 
   # Handle literals
@@ -147,15 +655,19 @@ defmodule Kernel.Typespec do
   end
 
   defp typespec([], vars, caller) do
-    typespec({ nil, 0, [] }, vars, caller)
+    typespec({ nil, [], [] }, vars, caller)
   end
 
   defp typespec([spec], vars, caller) do
-    typespec({ :list, 0, [spec] }, vars, caller)
+    typespec({ :list, [], [spec] }, vars, caller)
+  end
+
+  defp typespec([spec, {:"...", _, quoted}], vars, caller) when is_atom(quoted) do
+    typespec({ :nonempty_list, [], [spec] }, vars, caller)
   end
 
   defp typespec(l, _, _) when is_list(l) do
-    raise(ArgumentError, message: "Unexpected list #{inspect l}")
+    raise ArgumentError, message: "Unexpected list #{inspect l}"
   end
 
   defp typespec(t, vars, caller) when is_tuple(t) do
@@ -165,81 +677,31 @@ defmodule Kernel.Typespec do
 
   ## Helpers
 
-  defp remote_type({remote, line, name, arguments}, vars, caller) do
+  defp remote_type({remote, meta, name, arguments}, vars, caller) do
     arguments = lc arg inlist arguments, do: typespec(arg, vars, caller)
-    { :remote_type, line, [ remote, name, arguments ] }
+    { :remote_type, line(meta), [ remote, name, arguments ] }
   end
 
   defp collect_union({ :|, _, [a, b] }), do: [b|collect_union(a)]
   defp collect_union(v), do: [v]
 
-  defp fn_args(line, args, return, vars, caller) do
-    [fn_args(line, args, vars, caller), typespec(return, vars, caller)]
+  defp fn_args(meta, args, return, vars, caller) do
+    case [fn_args(meta, args, vars, caller), typespec(return, vars, caller)] do
+      [{:type,_,:any},{:type,_,:any,[]}] -> []
+      x -> x
+    end
   end
 
-  defp fn_args(line, [{:"...", _, _}], _vars, _caller) do
-    { :type, line, :any }
+  defp fn_args(meta, [{:"...", _, _}], _vars, _caller) do
+    { :type, line(meta), :any }
   end
 
-  defp fn_args(line, args, vars, caller) do
+  defp fn_args(meta, args, vars, caller) do
     args = lc arg inlist args, do: typespec(arg, vars, caller)
-    { :type, line, :product, args }
+    { :type, line(meta), :product, args }
   end
 
-  def _deftype({:"::", _, [name, definition]}, export, caller, opts) do
-    _deftype(name, definition, export, caller, opts)
+  defp variable({name, meta, _}) do
+    {:var, line(meta), name}
   end
-
-  def _deftype(name, export, caller, opts) do
-    _deftype(name, { :term, caller.line, nil }, export, caller, opts)
-  end
-
-  defp _deftype({name, _, args}, definition, export, caller, options) do
-    args = if is_atom(args), do: [], else: lc(arg inlist args, do: variable(arg))
-    vars = lc {:var, _, var} inlist args, do: var
-    spec = typespec(definition, vars, caller)
-
-    vars = lc ({:var, _, _} = var) inlist args, do: var
-    attr = if options[:opaque], do: :opaque, else: :type
-
-    export = if export do
-      quote do: Module.compile_type(__MODULE__, :export_type, [{name, length(vars)}])
-    else
-      nil
-    end
-
-    quote do
-      name = unquote(name)
-      spec = unquote(Macro.escape(spec))
-      vars = unquote(Macro.escape(vars))
-      type = { name, spec, vars }
-      Module.compile_type __MODULE__, unquote(attr), type
-      unquote(export)
-      { unquote(attr), type }
-    end
-  end
-
-  defp _defspec(type, caller, {name, line, args},[{:do,return}]) do
-    if is_atom(args), do: args = []
-    spec  = { :type, line, :fun, fn_args(line, args, return, [], caller) }
-    code  = Macro.escape { {type, { name, length(args) }}, [spec] }
-    table = spec_table_for(caller.module)
-
-    quote do
-      code = unquote(code)
-      :ets.insert(unquote(table), code)
-      code
-    end
-  end
-
-  defp spec_table_for(module) do
-    table = list_to_atom :lists.concat([:s, module])
-    unless table == :ets.info(table, :name), do:
-      raise(ArgumentError, message: "cannot manage specs for #{inspect module} because it was already compiled")
-    table
-  end
-
-  defp variable({name, line, _}) do
-    {:var, line, name}
-  end
-end
+end

lib/elixir/lib/keyword.ex

@@ -13,11 +13,16 @@ defmodule Keyword do
   entries for a given key are removed when invoked.
   """
 
+  @type key :: atom
+  @type value :: any
+  @type t :: [{key, value}]
+
   @doc """
   Creates a Keyword from enum. Differently from `Keyword.new`
   which behaves as a dict, `Keyword.from_enum` do not remove
   duplicated entries.
   """
+  @spec from_enum(Enum.t) :: t
   def from_enum(enum) when is_list(enum) do
     enum
   end
@@ -26,9 +31,24 @@ defmodule Keyword do
     Enum.map(enum, fn(x) -> x end)
   end
 
+  @doc """
+  Checks if the given argument is a keywords list or not
+  """
+  @spec keyword?(term) :: boolean
+  def keyword?([{ key, _value } | rest]) when is_atom(key) do
+    case atom_to_list(key) do
+      'Elixir-' ++ _ -> false
+      _ -> keyword?(rest)
+    end
+  end
+
+  def keyword?([]),     do: true
+  def keyword?(_other), do: false
+
   @doc """
   Returns an empty keyword list, i.e. an empty list.
   """
+  @spec new :: t
   def new do
     []
   end
@@ -39,10 +59,11 @@ defmodule Keyword do
 
   ## Examples
 
-      Keyword.new [{:b,1},{:a,2}]
-      #=> [a: 2, b: 1]
+      iex> Keyword.new [{:b,1},{:a,2}]
+      [a: 2, b: 1]
 
   """
+  @spec new(Enum.t) :: t
   def new(pairs) do
     Enum.reduce pairs, [], fn { k, v }, keywords ->
       put(keywords, k, v)
@@ -56,10 +77,11 @@ defmodule Keyword do
 
   ## Examples
 
-      Keyword.new [:a, :b], fn x -> {x,x} end
-      #=> [a: :a, b: :b]
+      iex> Enum.sort Keyword.new [:a, :b], fn x -> {x,x} end
+      [a: :a, b: :b]
 
   """
+  @spec new(Enum.t, ({key, value} -> {key, value})) :: t
   def new(pairs, transform) do
     Enum.reduce pairs, [], fn i, keywords ->
       { k, v } = transform.(i)
@@ -78,12 +100,17 @@ defmodule Keyword do
 
   ## Examples
 
-      Keyword.get [a: 1], :a      #=> 1
-      Keyword.get [a: 1], :b      #=> nil
-      Keyword.get [a: 1], :b, 3   #=> 3
+      iex> Keyword.get [a: 1], :a
+      1
+      iex> Keyword.get [a: 1], :b
+      nil
+      iex> Keyword.get [a: 1], :b, 3
+      3
 
   """
-  def get(keywords, key, default // nil) do
+  @spec get(t, key) :: value
+  @spec get(t, key, value) :: value
+  def get(keywords, key, default // nil) when is_atom(key) do
     case :lists.keyfind(key, 1, keywords) do
       { ^key, value } -> value
       false -> default
@@ -96,14 +123,17 @@ defmodule Keyword do
 
   ## Examples
 
-      Keyword.get! [a: 1], :a      #=> 1
-      Keyword.get! [a: 1], :b      #=> raises KeyError[key: :b]
+      iex> Keyword.get! [a: 1], :a
+      1
+      iex> Keyword.get! [a: 1], :b
+      ** (KeyError) key not found: :b
 
   """
-  def get!(keywords, key) do
+  @spec get!(t, key) :: value | no_return
+  def get!(keywords, key) when is_atom(key) do
     case :lists.keyfind(key, 1, keywords) do
       { ^key, value } -> value
-      false -> raise(Keyword.KeyError, key: key)
+      false -> raise(KeyError, key: key)
     end
   end  
 
@@ -112,11 +142,12 @@ defmodule Keyword do
 
   ## Examples
 
-      Keyword.get_values [a: 1, a: 2], :a
-      #=> [1,2]
+      iex> Keyword.get_values [a: 1, a: 2], :a
+      [1,2]
 
   """
-  def get_values(keywords, key) do
+  @spec get_values(t, key) :: [value]
+  def get_values(keywords, key) when is_atom(key) do
     lc { k, v } inlist keywords, key == k, do: v
   end
 
@@ -126,9 +157,11 @@ defmodule Keyword do
 
   ## Examples
 
-      Keyword.keys [a: 1, b: 2] #=> [:a,:b]
+      iex> Keyword.keys [a: 1, b: 2]
+      [:a,:b]
 
   """
+  @spec keys(t) :: [key]
   def keys(keywords) do
     lc { key, _ } inlist keywords, do: key
   end
@@ -138,9 +171,11 @@ defmodule Keyword do
 
   ## Examples
 
-      Keyword.values [a: 1, b: 2] #=> [1,2]
+      iex> Keyword.values [a: 1, b: 2]
+      [1,2]
 
   """
+  @spec values(t) :: [value]
   def values(keywords) do
     lc { _, value } inlist keywords, do: value
   end
@@ -153,28 +188,50 @@ defmodule Keyword do
 
   ## Examples
 
-      Keyword.delete [a: 1, b: 2], :a   #=> [b: 2]
-      Keyword.delete [b: 2], :a         #=> [b: 2]
+      iex> Keyword.delete [a: 1, b: 2], :a
+      [b: 2]
+      iex> Keyword.delete [b: 2], :a
+      [b: 2]
 
   """
-  def delete(keywords, key) do
+  @spec delete(t, key) :: t
+  def delete(keywords, key) when is_atom(key) do
     lc { k, _ } = tuple inlist keywords, key != k, do: tuple
   end
 
   @doc """
-  Sets the given `value` under `key`.
+  Puts the given `value` under `key`.
 
   If a previous value is already stored, all entries are
   removed and the value is overriden.
 
   ## Examples
 
-      Keyword.put [a: 1, b: 2], :a, 3
-      #=> [a: 3, b: 2]
+      iex> Keyword.put [a: 1, b: 2], :a, 3
+      [a: 3, b: 2]
 
   """
-  def put(list, key, value) do
-    [{key, value}|delete(list, key)]
+  @spec put(t, key, value) :: t
+  def put(keywords, key, value) when is_atom(key) do
+    [{key, value}|delete(keywords, key)]
+  end
+
+  @doc """
+  Puts the given `value` under `key` unless the entry `key`
+  already exists.
+
+  ## Examples
+
+      iex> Keyword.put_new [a: 1, b: 2], :a, 3
+      [a: 1, b: 2]
+
+  """
+  @spec put_new(t, key, value) :: t
+  def put_new(keywords, key, value) when is_atom(key) do
+    case :lists.keyfind(key, 1, keywords) do
+      { ^key, _ } -> keywords
+      false -> [{key,value}|keywords]
+    end
   end
 
   @doc """
@@ -183,10 +240,11 @@ defmodule Keyword do
 
   ## Examples
 
-      Keyword.equal? [a: 1, b: 2], [b: 2, a: 1]
-      #=> true
+      iex> Keyword.equal? [a: 1, b: 2], [b: 2, a: 1]
+      true
 
   """
+  @spec equal?(t, t) :: boolean
   def equal?(left, right) do
     :lists.sort(left) == :lists.sort(right)
   end
@@ -197,10 +255,11 @@ defmodule Keyword do
 
   ## Examples
 
-      Keyword.merge [a: 1, b: 2], [a: 3, d: 4]
-      #=> [a:3, b:2, d: 4]
+      iex> Enum.sort Keyword.merge [a: 1, b: 2], [a: 3, d: 4]
+      [a: 3, b: 2, d: 4]
 
   """
+  @spec merge(t, t) :: t
   def merge(d1, d2) do
     d2 ++ lc({ k, _ } = tuple inlist d1, not has_key?(d2, k), do: tuple)
   end
@@ -211,12 +270,13 @@ defmodule Keyword do
 
   ## Examples
 
-      Keyword.merge [a: 1, b: 2], [a: 3, d: 4], fn _k, v1, v2 ->
-        v1 + v2
-      end
-      #=> [a:4, b:2, d: 4]
+      iex> Keyword.merge [a: 1, b: 2], [a: 3, d: 4], fn _k, v1, v2 ->
+      ...>  v1 + v2
+      iex> end
+      [a: 4, b: 2, d: 4]
 
   """
+  @spec merge(t, t, (key, value, value -> value)) :: t
   def merge(d1, d2, fun) do
     do_merge(d2, d1, fun)
   end
@@ -229,39 +289,35 @@ defmodule Keyword do
     acc
   end
 
-  @doc false
-  def key?(list, key) do
-    IO.write "[WARNING] Keyword.key? is deprecated, please use Keyword.has_key? instead\n#{Exception.formatted_stacktrace}"
-    has_key?(list, key)
-  end
-
   @doc """
   Returns whether a given key exists in the given keywords.
 
   ## Examples
 
-      Keyword.has_key?([a: 1], :a)
-      #=> true
-      Keyword.has_key?([a: 1], :b)
-      #=> false
+      iex> Keyword.has_key?([a: 1], :a)
+      true
+      iex> Keyword.has_key?([a: 1], :b)
+      false
 
   """
-  def has_key?(keywords, key) do
+  @spec has_key?(t, key) :: boolean
+  def has_key?(keywords, key) when is_atom(key) do
     :lists.keymember(key, 1, keywords)
   end
 
   @doc """
   Updates the key with the given function. If the key does
-  not exist, raises `Keyword.KeyError`.
+  not exist, raises `KeyError`.
 
   ## Examples
 
-      Keyword.update([a: 1], :a, &1 * 2)
-      #=> [a: 2]
-      Keyword.update([a: 1], :b, &1 * 2)
-      #=> Keyword.KeyError
+      iex> Keyword.update([a: 1], :a, &1 * 2)
+      [a: 2]
+      iex> Keyword.update([a: 1], :b, &1 * 2)
+      ** (KeyError) key not found: :b
 
   """
+  @spec update(t, key, (value -> value)) :: t | no_return
   def update([{key, value}|keywords], key, fun) do
     [{key, fun.(value)}|delete(keywords, key)]
   end
@@ -271,7 +327,7 @@ defmodule Keyword do
   end
 
   def update([], key, _fun) when is_atom(key) do
-    raise(Keyword.KeyError, key: key)
+    raise(KeyError, key: key)
   end
 
   @doc """
@@ -280,12 +336,13 @@ defmodule Keyword do
 
   ## Examples
 
-      Keyword.update([a: 1], :a, 13, &1 * 2)
-      #=> [a: 2]
-      Keyword.update([a: 1], :b, 11, &1 * 2)
-      #=> [a: 1, b: 11]
+      iex> Keyword.update([a: 1], :a, 13, &1 * 2)
+      [a: 2]
+      iex> Keyword.update([a: 1], :b, 11, &1 * 2)
+      [a: 1, b: 11]
 
   """
+  @spec update(t, key, value, (value -> value)) :: t  
   def update([{key, value}|keywords], key, _initial, fun) do
     [{key, fun.(value)}|delete(keywords, key)]
   end

lib/elixir/lib/list.ex

@@ -15,8 +15,8 @@ defmodule List do
 
   ## Examples
 
-      List.concat [[1,[2],3], [4], [5,6]]
-      #=> [1,[2],3,4,5,6]
+      iex> List.concat [[1,[2],3], [4], [5,6]]
+      [1,[2],3,4,5,6]
 
   """
   def concat(list) when is_list(list) do
@@ -32,8 +32,8 @@ defmodule List do
 
   ## Examples
 
-      List.concat [1,2,3], [4,5,6]
-      #=> [1,2,3,4,5,6]
+      iex> List.concat [1,2,3], [4,5,6]
+      [1,2,3,4,5,6]
 
   """
   def concat(list, elements) when is_list(list) and is_list(elements) do
@@ -47,8 +47,8 @@ defmodule List do
 
   ## Examples
 
-      List.delete([1,2,3], 1)
-      #=> [2,3]
+      iex> List.delete([1,2,3], 1)
+      [2,3]
 
   """
   def delete(list, item) do
@@ -60,11 +60,11 @@ defmodule List do
 
   ## Examples
 
-      List.duplicate "hello", 3
-      #=> ["hello","hello","hello"]
+      iex> List.duplicate "hello", 3
+      ["hello","hello","hello"]
 
-      List.duplicate [1,2], 2
-      #=> [[1,2],[1,2]]
+      iex> List.duplicate [1,2], 2
+      [[1,2],[1,2]]
   """
   def duplicate(elem, n) do
     :lists.duplicate(n, elem)
@@ -77,11 +77,11 @@ defmodule List do
 
   ## Examples
 
-      List.flatten [1,[[2],3]]
-      #=> [1,2,3]
+      iex> List.flatten [1,[[2],3]]
+      [1,2,3]
 
-      List.flatten [1,[[2],3]], [4,5]
-      #=> [1,2,3,4,5]
+      iex> List.flatten [1,[[2],3]], [4,5]
+      [1,2,3,4,5]
 
   """
   def flatten(list) do
@@ -98,11 +98,11 @@ defmodule List do
 
   ## Examples
 
-      List.foldl [5,5], 10, fn x, acc -> x + acc end
-      #=> 20
+      iex> List.foldl [5,5], 10, fn x, acc -> x + acc end
+      20
 
-      List.foldl [1,2,3,4], 0, fn x, acc -> x - acc end
-      #=> 2
+      iex> List.foldl [1,2,3,4], 0, fn x, acc -> x - acc end
+      2
 
   """
   def foldl(list, acc, function) when is_list(list) and is_function(function) do
@@ -115,31 +115,25 @@ defmodule List do
 
   ## Examples
 
-      List.foldr [1,2,3,4], 0, fn x, acc -> x - acc end
-      #=> -2
+      iex> List.foldr [1,2,3,4], 0, fn x, acc -> x - acc end
+      -2
 
   """
   def foldr(list, acc, function) when is_list(list) and is_function(function) do
     :lists.foldr(function, acc, list)
   end
 
-  @doc false
-  def reverse(list) do
-    IO.write "[WARNING] List.reverse/1 is deprecated, please use Enum.reverse/1 instead\n#{Exception.formatted_stacktrace}"
-    :lists.reverse(list)
-  end
-
   @doc """
   Returns the last element in `list` or nil if the `list` is empty.
 
   ## Examples
 
-      List.last []
-      #=> nil
-      List.last [1]
-      #=> 1
-      List.last [1, 2, 3]
-      #=> 3
+      iex> List.last []
+      nil
+      iex> List.last [1]
+      1
+      iex> List.last [1, 2, 3]
+      3
 
   """
   def last([]), do: nil
@@ -155,11 +149,11 @@ defmodule List do
 
   ## Examples
 
-      List.member? [1,2,3], 1
-      #=> true
+      iex> List.member? [1,2,3], 1
+      true
 
-      List.member? [1,2,3], 0
-      #=> false
+      iex> List.member? [1,2,3], 0
+      false
 
   """
   def member?(list, term) do
@@ -173,14 +167,14 @@ defmodule List do
 
   ## Examples
 
-      List.keyfind([a: 1, b: 2], :a, 0)
-      #=> { :a, 1 }
+      iex> List.keyfind([a: 1, b: 2], :a, 0)
+      { :a, 1 }
 
-      List.keyfind([a: 1, b: 2], 2, 1)
-      #=> { :b, 2 }
+      iex> List.keyfind([a: 1, b: 2], 2, 1)
+      { :b, 2 }
 
-      List.keyfind([a: 1, b: 2], :c, 0)
-      #=> nil
+      iex> List.keyfind([a: 1, b: 2], :c, 0)
+      nil
 
   """
   def keyfind(list, key, position, default // nil) do
@@ -194,14 +188,14 @@ defmodule List do
 
   ## Examples
 
-      List.keymember?([a: 1, b: 2], :a, 0)
-      #=> true
+      iex> List.keymember?([a: 1, b: 2], :a, 0)
+      true
 
-      List.keymember?([a: 1, b: 2], 2, 1)
-      #=> true
+      iex> List.keymember?([a: 1, b: 2], 2, 1)
+      true
 
-      List.keymember?([a: 1, b: 2], :c, 0)
-      #=> false
+      iex> List.keymember?([a: 1, b: 2], :c, 0)
+      false
 
   """
   def keymember?(list, key, position) do
@@ -214,8 +208,8 @@ defmodule List do
 
   ## Examples
 
-      List.keyreplace([a: 1, b: 2], :a, 0, { :a, 3 })
-      #=> [a: 3, b: 2]
+      iex> List.keyreplace([a: 1, b: 2], :a, 0, { :a, 3 })
+      [a: 3, b: 2]
 
   """
   def keyreplace(list, key, position, new_tuple) do
@@ -229,8 +223,8 @@ defmodule List do
 
   ## Examples
 
-      List.keystore([a: 1, b: 2], :a, 0, { :a, 3 })
-      #=> [a: 3, b: 2]
+      iex> List.keystore([a: 1, b: 2], :a, 0, { :a, 3 })
+      [a: 3, b: 2]
 
   """
   def keystore(list, key, position, new_tuple) do
@@ -244,101 +238,20 @@ defmodule List do
 
   ## Examples
 
-      List.keydelete([a: 1, b: 2], :a, 0)
-      #=> [{ :b, 2 }]
+      iex> List.keydelete([a: 1, b: 2], :a, 0)
+      [{ :b, 2 }]
 
-      List.keydelete([a: 1, b: 2], 2, 1)
-      #=> [{ :a, 1 }]
+      iex> List.keydelete([a: 1, b: 2], 2, 1)
+      [{ :a, 1 }]
 
-      List.keydelete([a: 1, b: 2], :c, 0)
-      #=> [{ :a, 1 }, { :b, 2 }]
+      iex> List.keydelete([a: 1, b: 2], :c, 0)
+      [{ :a, 1 }, { :b, 2 }]
 
   """
   def keydelete(list, key, position) do
     :lists.keydelete(key, position + 1, list)
   end
 
-  @doc """
-  Returns a list of integers in the given range (both ends included when
-  possible). An optional step can be provided as well (defaults to 1).
-
-  If first > last and no step is provided, the numbers will be in descending
-  order.
-
-  ## Examples
-
-      List.range 1, 3     #=> [1,2,3]
-      List.range 1, 8, 2  #=> [1,3,5,7]
-      List.range 1, 0     #=> []
-      List.range 3, 1     #=> [3,2,1]
-      List.range 5, 1, -2 #=> [5, 3, 1]
-
-  """
-  def range(first, last, step // nil)
-
-  def range(first, last, step) when is_integer(first) and is_integer(last) and first <= last do
-    step = case step do
-      nil ->
-        :lists.seq(first, last, 1)
-      x when x < 0 ->
-        []
-      _ ->
-        :lists.seq(first, last, step)
-    end
-  end
-
-  def range(first, last, step) when is_integer(first) and is_integer(last) and first > last do
-    step = case step do
-      nil ->
-        :lists.seq(first, last, -1)
-      x when x > 0 ->
-        []
-      _ ->
-        :lists.seq(first, last, step)
-    end
-  end
-
-  @doc """
-  Sorts the list by comparing each term. For an alternative
-  sorting algorithm, check `Enum.qsort`.
-
-  ## Examples
-
-      List.sort [3, 4, 2, 1, 7]
-      #=> [1, 2, 3, 4, 7]
-
-  """
-  def sort(list) do
-    :lists.sort list
-  end
-
-  @doc """
-  Sorts the list according to an ordering function. fun(a, b) should
-  return true if `a` compares less than or equal to `b`, `false` otherwise.
-
-  ## Examples
-
-      List.sort [3, 4, 2, 1, 7], fn a, b -> b <= a end
-      #=> [7, 4, 3, 2, 1]
-
-  """
-  def sort(list, fun) do
-    :lists.sort fun, list
-  end
-
-  @doc """
-  Returns a list without duplicated items.
-
-  ## Examples
-
-      List.uniq [1,2,3,2,1]
-      #=> [1,2,3]
-
-  """
-  def uniq(list) when is_list(list) do
-    do_uniq(list, [])
-  end
-
   @doc """
   Wraps the argument in a list.
   If the argument is already a list, returns the list.
@@ -346,7 +259,8 @@ defmodule List do
 
   ## Examples
 
-      List.wrap [1,2,3] #=> [1,2,3]
+      iex> List.wrap [1,2,3]
+      [1,2,3]
 
   """
   def wrap(list) when is_list(list) do
@@ -361,22 +275,16 @@ defmodule List do
     [other]
   end
 
-  @doc false
-  def zip(item1, item2) do
-    IO.write "[WARNING] List.zip/2 is deprecated, please use Enum.zip/2 instead\n#{Exception.formatted_stacktrace}"
-    do_zip(to_list(item1), to_list(item2), [])
-  end
-
   @doc """
   Zips corresponding elements from each list in `list_of_lists`.
 
   ## Examples
 
-      List.zip [[1, 2], [3, 4], [5, 6]]
-      #=> [{1, 3, 5}, {2, 4, 6}]
+      iex> List.zip [[1, 2], [3, 4], [5, 6]]
+      [{1, 3, 5}, {2, 4, 6}]
 
-      List.zip [[1, 2], [3], [5, 6]]
-      #=> [{1, 3, 5}]
+      iex> List.zip [[1, 2], [3], [5, 6]]
+      [{1, 3, 5}]
 
   """
   def zip([]), do: []
@@ -390,53 +298,28 @@ defmodule List do
 
   ## Examples
 
-      List.unzip [{1, 2}, {3, 4}]
-      #=> [[1, 3], [2, 4]]
+      iex> List.unzip [{1, 2}, {3, 4}]
+      [[1, 3], [2, 4]]
 
-      List.unzip [{1, :a, "apple"}, {2, :b, "banana"}, {3, :c}]
-      #=> [[1, 2, 3], [:a, :b, :c]]
+      iex> List.unzip [{1, :a, "apple"}, {2, :b, "banana"}, {3, :c}]
+      [[1, 2, 3], [:a, :b, :c]]
 
   """
   def unzip(list) when is_list(list) do
     :lists.map tuple_to_list(&1), zip(list)
   end
 
-  ## Private
-
-  # uniq
-
-  defp do_uniq([h|t], acc) do
-    case :lists.member(h, acc) do
-      true ->
-        do_uniq(t, acc)
-      false ->
-        [h|do_uniq(t, [h|acc])]
-    end
-  end
-
-  defp do_uniq([], _acc) do
-    []
-  end
+  ## Helpers
 
   # zip
 
-  defp do_zip([h1|t1], [h2|t2], acc) do
-    do_zip t1, t2, [{h1, h2}|acc]
-  end
-
-  defp do_zip(_, _, acc) do
-    :lists.reverse acc
-  end
-
   defp do_zip(list, acc) do
     converter = fn x, acc -> do_zip_each(to_list(x), acc) end
     {mlist, heads} = :lists.mapfoldl converter, [], list
 
     case heads do
-      nil ->
-        :lists.reverse acc
-      _ ->
-      do_zip mlist, [list_to_tuple(:lists.reverse(heads))|acc]
+      nil -> :lists.reverse acc
+      _   -> do_zip mlist, [list_to_tuple(:lists.reverse(heads))|acc]
     end
   end
 

lib/elixir/lib/list/dict.ex

@@ -0,0 +1,155 @@
+defmodule List.Dict do
+  @moduledoc """
+  A Dict implementation that works on lists of two-items tuples.
+
+  For more information about the functions and their APIs, please
+  consult the `Dict` module.
+  """
+
+  @doc """
+  Returns a new `List.Dict`, i.e. an empty list.
+  """
+  def new, do: []
+
+  @doc """
+  Creates a new `List.Dict` from the given pairs.
+  """
+  def new(pairs) do
+    Enum.map pairs, fn({ k, v }) -> { k, v } end
+  end
+
+  @doc """
+  Creates a new `List.Dict` from the given pairs
+  via the given transformation function.
+  """
+  def new(list, transform) when is_function(transform) do
+    Enum.map list, transform
+  end
+
+  @doc """
+  Returns all keys in the dict.
+  """
+  def keys(dict) do
+    lc { key, _ } inlist dict, do: key
+  end
+
+  @doc """
+  Returns all values in the dict.
+  """
+  def values(dict) do
+    lc { _, value } inlist dict, do: value
+  end
+
+  @doc """
+  Returns the dict size.
+  """
+  def size(dict) do
+    length(dict)
+  end
+
+  @doc """
+  Returns true if the dict has the given key.
+  """
+  def has_key?(dict, key) do
+    :lists.keymember(key, 1, dict)
+  end
+
+  @doc """
+  Returns the value under key from the given
+  dict or default if no key is set.
+  """
+  def get(dict, key, default // nil) do
+    case :lists.keyfind(key, 1, dict) do
+      { ^key, value } -> value
+      false -> default
+    end
+  end
+
+  @doc """
+  Returns the value under the given key
+  raises `KeyError` if the key does not exist.
+  """
+  def get!(dict, key) do
+    case :lists.keyfind(key, 1, dict) do
+      { ^key, value } -> value
+      false -> raise(KeyError, key: key)
+    end
+  end
+
+  @doc """
+  Puts the given key-value pair in the dict.
+  """
+  def put(dict, key, val) do
+    [{key, val}|delete(dict, key)]
+  end
+
+  @doc """
+  Puts the given key-value pair in the dict
+  if no entry exists yet.
+  """
+  def put_new(dict, key, val) do
+    case :lists.keyfind(key, 1, dict) do
+      { ^key, _ } -> dict
+      false -> [{key,val}|dict]
+    end
+  end
+
+  @doc """
+  Deletes the entry under the given key from the dict.
+  """
+  def delete(dict, key) do
+    lc { k, _ } = tuple inlist dict, key != k, do: tuple
+  end
+
+  @doc """
+  Merges the given Enumerable into the dict.
+  """
+  def merge(dict, enum, callback // fn(_k, _v1, v2) -> v2 end)
+
+  def merge(dict1, dict2, fun) do
+    Enum.reduce dict2, dict1, fn { k, v2 }, acc ->
+      update(acc, k, v2, fn(v1) -> fun.(k, v1, v2) end)
+    end
+  end
+
+  @doc """
+  Updates the key in the dict according to the given function.
+  """
+  def update([{key, value}|dict], key, fun) do
+    [{key, fun.(value)}|delete(dict, key)]
+  end
+
+  def update([{_, _} = e|dict], key, fun) do
+    [e|update(dict, key, fun)]
+  end
+
+  def update([], key, _fun) do
+    raise(KeyError, key: key)
+  end
+
+  @doc """
+  Updates the key in the dict according to the given function
+  or uses the given initial value if no entry exists.
+  """
+  def update([{key, value}|dict], key, _initial, fun) do
+    [{key, fun.(value)}|delete(dict, key)]
+  end
+
+  def update([{_, _} = e|dict], key, initial, fun) do
+    [e|update(dict, key, initial, fun)]
+  end
+
+  def update([], key, initial, _fun) do
+    [{key, initial}]
+  end
+
+  @doc """
+  Returns an empty `List.Dict`.
+  """
+  def empty(_dict), do: []
+
+  @doc """
+  Converts the dict to a list.
+  """
+  def to_list(dict), do: dict
+end

lib/elixir/lib/macro.ex

@@ -13,7 +13,7 @@ defmodule Macro do
     [
       :===, :!==,
       :==, :!=, :<=, :>=,
-      :&&, :||, :<>, :++, :--, :**, ://, :::, :<-, :.., :/>, :=~,
+      :&&, :||, :<>, :++, :--, :**, ://, :::, :<-, :.., :|>, :=~,
       :<, :>,
       :+, :-, :*, :/, :=, :|, :.,
       :and, :or, :xor, :when, :in, :inlist, :inbits,
@@ -30,31 +30,112 @@ defmodule Macro do
   end
 
   @doc """
-  Recursively escapes the given value so it can be inserted
-  into a syntax tree. Structures that are valid syntax nodes
-  (like atoms, integers, binaries) are represented by themselves.
+  Receives an expresion representing a possible definition
+  and extracts its arguments. It returns a tuple with the
+  function name and the arguments list or `:error` if not
+  a valid call syntax.
+
+  This is useful for macros that want to provide the same
+  arguments syntax available in def/defp/defmacro and friends.
 
   ## Examples
 
-      Macro.escape(:foo)
-      #=> :foo
+      extract_args(quote do: foo)        == { :foo, [] }
+      extract_args(quote do: foo())      == { :foo, [] }
+      extract_args(quote do: foo(1,2,3)) == { :foo, [1,2,3] }
+      extract_args(quote do: 1.(1,2,3))  == :error
 
-      Macro.escape({ :a, :b, :c })
-      #=> { :{}, 0, [:a, :b, :c] }
   """
-  def escape({ left, right }) do
-    { escape(left), escape(right) }
+  def extract_args(expr) do
+    :elixir_clauses.extract_args(expr)
   end
 
-  def escape(tuple) when is_tuple(tuple) do
-    { :{}, 0, escape(tuple_to_list(tuple)) }
+  @doc """
+  Recursively escapes a value so it can be inserted
+  into a syntax tree.
+
+  ## Examples
+
+      iex> Macro.escape(:foo)
+      :foo
+
+      iex> Macro.escape({ :a, :b, :c })
+      { :{}, [], [:a, :b, :c] }
+
+  """
+  def escape(expr) do
+    do_escape(expr, false)
   end
 
-  def escape(list) when is_list(list) do
-    lc item inlist list, do: escape(item)
+  defp do_escape({ { { :., meta, [left, :unquote] }, _, [expr] }, _, args }, true) do
+    all = [do_escape(meta, true), do_escape(left, true), expr, do_escape(args, true)]
+    quote do
+      apply :elixir_quote, :unquote, [unquote_splicing(all), __FILE__]
+    end
   end
 
-  def escape(other), do: other
+  defp do_escape({ { :., meta, [left, :unquote] }, _, [expr] }, true) do
+    all = [do_escape(meta, true), do_escape(left, true), expr, nil]
+    quote do
+      apply :elixir_quote, :unquote, [unquote_splicing(all), __FILE__]
+    end
+  end
+
+  defp do_escape({ :unquote, _meta, [expr] }, true) do
+    expr
+  end
+
+  defp do_escape({ left, right }, unquote) when
+      (not is_tuple(left) or elem(left, 1) != :unquote_splicing) and
+      (not is_tuple(right) or elem(right, 1) != :unquote_splicing) do
+    { do_escape(left, unquote), do_escape(right, unquote) }
+  end
+
+  defp do_escape({ :quote, meta, args }, true) when length(args) in 1..2 do
+    { :{}, [], do_escape([:quote, meta, args], false) }
+  end
+
+  defp do_escape(tuple, unquote) when is_tuple(tuple) do
+    { :{}, [], do_escape(tuple_to_list(tuple), unquote) }
+  end
+
+  defp do_escape(list, true) when is_list(list) do
+    do_splice(Enum.reverse(list))
+  end
+
+  defp do_escape(list, false) when is_list(list) do
+    lc item inlist list, do: do_escape(item, false)
+  end
+
+  defp do_escape(other, _unquote), do: other
+
+  defp do_splice([{ :|, meta, [{ :unquote_splicing, _, [left] }, right] }|t]) do
+    spliced = { :++, meta, [do_splice(t, [], []), left] }
+    { :++, meta, [spliced, right] }
+  end
+
+  defp do_splice(list) do
+    do_splice(list, [], [])
+  end
+
+  defp do_splice([{ :unquote_splicing, _meta, [expr] }|t], buffer, acc) do
+    do_splice(t, [], do_splice_join(do_splice_join(expr, buffer), acc))
+  end
+
+  defp do_splice([h|t], buffer, acc) do
+    do_splice t, [do_escape(h, true)|buffer], acc
+  end
+
+  defp do_splice([], buffer, acc), do: do_splice_join(buffer, acc)
+
+  defp do_splice_join([], right),   do: right
+  defp do_splice_join(left, []),    do: left
+  defp do_splice_join(left, right), do: { :++, [], [left, right] }
+
+  @doc false
+  def escape_quoted(expr) do
+    do_escape(expr, true)
+  end
 
   @doc %B"""
   Unescape the given chars. This is the unescaping behavior
@@ -62,17 +143,17 @@ defmodule Macro do
   Check `unescape_binary/2` for information on how to customize
   the escaping map.
 
-  In this setup, Elixir will escape the following: `\b`, `\d`,
-  `\e`, `\f`, `\n`, `\r`, `\s`, `\t` and `\v`. Octals are also
-  escaped according to the latin1 set they represent.
+  In this setup, Elixir will escape the following: `\a`, `\b`,
+  `\d`, `\e`, `\f`, `\n`, `\r`, `\s`, `\t` and `\v`. Octals are
+  also escaped according to the latin1 set they represent.
 
   This function is commonly used on sigil implementations
   (like `%r`, `%b` and others).
 
   ## Examples
 
-      Macro.unescape_binary "example\\n"
-      #=> "example\n"
+      iex> Macro.unescape_binary "example\\n"
+      "example\n"
 
   In the example above, we pass a string with `\n` escaped
   and we return a version with it unescaped.
@@ -83,8 +164,8 @@ defmodule Macro do
 
   @doc %B"""
   Unescape the given chars according to the map given.
-  Check `unescape/1` if you want to use the same map as
-  Elixir single- and double-quoted strings.
+  Check `unescape_binary/1` if you want to use the same map
+  as Elixir single- and double-quoted strings.
 
   ## Map
 
@@ -92,6 +173,7 @@ defmodule Macro do
   representing the number of the characters it wants to unescape.
   Here is the default mapping function implemented by Elixir:
 
+      def unescape_map(?a), do: ?\a
       def unescape_map(?b), do: ?\b
       def unescape_map(?d), do: ?\d
       def unescape_map(?e), do: ?\e
@@ -111,6 +193,11 @@ defmodule Macro do
   Octals will by default be escaped unless the map function
   returns false for ?0.
 
+  ## Hex
+
+  Octals will by default be escaped unless the map function
+  returns false for ?x.
+
   ## Examples
 
   Using the unescape_map defined above is easy:
@@ -124,11 +211,13 @@ defmodule Macro do
 
   @doc """
   Unescape the given tokens according to the default map.
-  Check `unescape/1` and `unescape/2` for more information
-  about unescaping. Only tokens that are binaries are
-  unescaped, all others are ignored. This method is useful
-  when implementing your own sigils. Check the implementation
-  of `Kernel.__b__` for examples.
+  Check `unescape_binary/1` and `unescape_binary/2` for more
+  information about unescaping.
+
+  Only tokens that are binaries are unescaped, all others are
+  ignored. This function is useful when implementing your own
+  sigils. Check the implementation of `Kernel.__b__`
+  for examples.
   """
   def unescape_tokens(tokens) do
     :elixir_interpolation.unescape_tokens(tokens)
@@ -136,7 +225,7 @@ defmodule Macro do
 
   @doc """
   Unescape the given tokens according to the given map.
-  Check `unescape_tokens/1` and `unescaped/2` for more information.
+  Check `unescape_tokens/1` and `unescape_binary/2` for more information.
   """
   def unescape_tokens(tokens, map) do
     :elixir_interpolation.unescape_tokens(tokens, map)
@@ -147,8 +236,8 @@ defmodule Macro do
 
   ## Examples
 
-      Macro.to_binary(quote do: foo.bar(1, 2, 3))
-      #=> "foo.bar(1, 2, 3)"
+      iex> Macro.to_binary(quote do: foo.bar(1, 2, 3))
+      "foo.bar(1, 2, 3)"
 
   """
   def to_binary(tree)
@@ -189,21 +278,40 @@ defmodule Macro do
   end
 
   # Fn keyword
-  def to_binary({ :fn, _, [[do: block]] }) do
+  def to_binary({ :fn, _, [[do: { :->, _, [{_,tuple}] } = arrow]] })
+      when not is_tuple(tuple) or elem(tuple, 0) != :__block__ do
+    "fn " <> arrow_to_binary(arrow) <> " end"
+  end
+
+  def to_binary({ :fn, _, [[do: { :->, _, [_] } = block]] }) do
     "fn " <> block_to_binary(block) <> "\nend"
   end
 
+  def to_binary({ :fn, _, [[do: block]] }) do
+    block = adjust_new_lines block_to_binary(block), "\n  "
+    "fn\n  " <> block <> "\nend"
+  end
+
   # Partial call
   def to_binary({ :&, _, [num] }) do
     "&#{num}"
   end
 
+  # left -> right
+  def to_binary({ :->, _, _ } = arrow) do
+    "(" <> arrow_to_binary(arrow, true) <> ")"
+  end
+
   # Binary ops
   def to_binary({ op, _, [left, right] }) when op in binary_ops do
     op_to_binary(left) <> " #{op} " <> op_to_binary(right)
   end
 
   # Unary ops
+  def to_binary({ :not, _, [arg] })  do
+    "not " <> to_binary(arg)
+  end
+
   def to_binary({ op, _, [arg] }) when op in unary_ops do
     atom_to_binary(op, :utf8) <> to_binary(arg)
   end
@@ -219,12 +327,12 @@ defmodule Macro do
 
   # Two-item tuples
   def to_binary({ left, right }) do
-    to_binary({ :{}, 0, [left, right] })
+    to_binary({ :{}, [], [left, right] })
   end
 
   # Lists
   def to_binary(list) when is_list(list) do
-    to_binary({ :[], 0, list })
+    to_binary({ :[], [], list })
   end
 
   # All other structures
@@ -236,11 +344,14 @@ defmodule Macro do
   defp is_kw_blocks?([_|_] = kw) do
     Enum.all?(kw, match?({x, _} when x in kw_keywords, &1))
   end
-  defp is_kw_blocks?(_),          do: false
+  defp is_kw_blocks?(_), do: false
+
+  defp module_to_binary(atom) when is_atom(atom), do: Binary.Inspect.inspect(atom, raw: true)
+  defp module_to_binary(other), do: call_to_binary(other)
 
   defp call_to_binary(atom) when is_atom(atom),  do: atom_to_binary(atom, :utf8)
-  defp call_to_binary({ :., _, [arg] }),         do: call_to_binary(arg) <> "."
-  defp call_to_binary({ :., _, [left, right] }), do: call_to_binary(left) <> "." <> call_to_binary(right)
+  defp call_to_binary({ :., _, [arg] }),         do: module_to_binary(arg) <> "."
+  defp call_to_binary({ :., _, [left, right] }), do: module_to_binary(left) <> "." <> call_to_binary(right)
   defp call_to_binary(other),                    do: to_binary(other)
 
   defp call_to_binary_with_args(target, args) do
@@ -264,8 +375,8 @@ defmodule Macro do
 
   defp block_to_binary({ :->, _, exprs }) do
     Enum.map_join(exprs, "\n", fn({ left, right }) ->
-      left = Enum.map_join(left, ", ", to_binary(&1))
-      left <> " ->\n  " <> adjust_new_lines block_to_binary(right), "\n  "
+      left = comma_join_or_empty_paren(left, false)
+      left <> "->\n  " <> adjust_new_lines block_to_binary(right), "\n  "
     end)
   end
 
@@ -281,6 +392,20 @@ defmodule Macro do
 
   defp op_to_binary(expr), do: to_binary(expr)
 
+  defp arrow_to_binary({ :->, _, pairs }, paren // false) do
+    Enum.map_join(pairs, "; ", fn({ left, right }) ->
+      left = comma_join_or_empty_paren(left, paren)
+      left <> "-> " <> to_binary(right)
+    end)
+  end
+
+  defp comma_join_or_empty_paren([], true),  do: "() "
+  defp comma_join_or_empty_paren([], false), do: ""
+
+  defp comma_join_or_empty_paren(left, _) do
+    Enum.map_join(left, ", ", to_binary(&1)) <> " "
+  end
+
   defp adjust_new_lines(block, replacement) do
     bc <<x>> inbits block do
       << case x == ?\n do
@@ -297,13 +422,17 @@ defmodule Macro do
   * Macros (local or remote);
   * Aliases are expanded (if possible) and return atoms;
   * All pseudo-variables (__FILE__, __MODULE__, etc);
+  * Module attributes reader (@foo);
 
   In case the expression cannot be expanded, it returns the expression itself.
 
   Notice that `Macro.expand` is not recursive and it does not
-  expand child expressions. For example, `!some_macro` will expand as:
+  expand child expressions. In this example
+
+    Macro.expand(quote(do: !some_macro), __ENV__)
+
+  `!some_macro` will expand to something like:
 
-      iex> IO.puts Macro.to_binary Macro.expand(quote(do: !some_macro), __ENV__)
       case some_macro do
         false -> true
         nil   -> true
@@ -343,7 +472,7 @@ defmodule Macro do
   The compilation will fail because `My.Module` when quoted
   is not an atom, but a syntax tree as follow:
 
-      {:__aliases__, 0, [:My, :Module] }
+      {:__aliases__, [], [:My, :Module] }
 
   That said, we need to expand the aliases node above to an
   atom, so we can retrieve its length. Expanding the node is
@@ -375,35 +504,25 @@ defmodule Macro do
       end
 
   """
-  def expand(aliases, env)
-
-  # The first case we handle is __aliases__. In case
-  # aliases just contain one item, we are sure it is
-  # an atom, so we just expand it based on the aliases
-  # dict.
-  def expand({ :__aliases__, _, [h] }, env) when h != Elixir do
-    expand_alias(h, env)
+  def expand(aliases, env) do
+    expand(aliases, env, nil)
   end
 
-  # In case aliases contains more than one item, we need
-  # to loop them checking if they are all atoms or not.
-  # Macros and pseudo-variables are then expanded.
-  def expand({ :__aliases__, _, [h|t] } = original, env) do
-    aliases = case h do
-      x when is_atom(x) and x != Elixir -> [expand_alias(x, env)|t]
-      _                                 -> [h|t]
-    end
+  defp expand({ :__aliases__, _, _ } = original, env, cache) do
+    case :elixir_aliases.expand(original, env.aliases, []) do
+      atom when is_atom(atom) -> atom
+      aliases ->
+        aliases = lc alias inlist aliases, do: expand(alias, env, cache)
 
-    aliases = lc alias inlist aliases, do: expand(alias, env)
-
-    case :lists.all(is_atom(&1), aliases) do
-      true  -> :elixir_aliases.concat(aliases)
-      false -> original
+        case :lists.all(is_atom(&1), aliases) do
+          true  -> :elixir_aliases.concat(aliases)
+          false -> original
+        end
     end
   end
 
   # Expand @ calls
-  def expand({ :@, _, [{ name, _, args }] } = original, env) when is_atom(args) or args == [] do
+  defp expand({ :@, _, [{ name, _, args }] } = original, env, _cache) when is_atom(args) or args == [] do
     case (module = env.module) && Module.open?(module) do
       true  -> Module.get_attribute(module, name)
       false -> original
@@ -411,12 +530,12 @@ defmodule Macro do
   end
 
   # Expand pseudo-variables
-  def expand({ :__MODULE__, _, atom }, env) when is_atom(atom), do: env.module
-  def expand({ :__FILE__, _, atom }, env)   when is_atom(atom), do: env.file
-  def expand({ :__ENV__, _, atom }, env)    when is_atom(atom), do: env
+  defp expand({ :__MODULE__, _, atom }, env, _cache) when is_atom(atom), do: env.module
+  defp expand({ :__FILE__, _, atom }, env, _cache)   when is_atom(atom), do: env.file
+  defp expand({ :__ENV__, _, atom }, env, _cache)    when is_atom(atom), do: env
 
   # Expand possible macro import invocation
-  def expand({ atom, line, args } = original, env) when is_atom(atom) do
+  defp expand({ atom, line, args } = original, env, cache) when is_atom(atom) do
     args = case is_atom(args) do
       true  -> []
       false -> args
@@ -425,8 +544,16 @@ defmodule Macro do
     case not is_partial?(args) do
       false -> original
       true  ->
+        module = env.module
+
+        extra  = if function_exported?(module, :__info__, 1) do
+          [{ module, module.__info__(:macros) }]
+        else
+          []
+        end
+
         expand = :elixir_dispatch.expand_import(line, { atom, length(args) }, args,
-          env.module, env.function, env.requires, env.macros, env)
+          env.module, extra, to_erl_env(env, cache))
         case expand do
           { :ok, _, expanded } -> expanded
           { :error, _ }     -> original
@@ -435,23 +562,26 @@ defmodule Macro do
   end
 
   # Expand possible macro require invocation
-  def expand({ { :., _, [left, right] }, line, args } = original, env) when is_atom(right) do
+  defp expand({ { :., _, [left, right] }, line, args } = original, env, cache) when is_atom(right) do
     receiver = expand(left, env)
 
     case is_atom(receiver) and not is_partial?(args) do
       false -> original
       true  ->
         expand = :elixir_dispatch.expand_require(line, receiver, { right, length(args) },
-          args, env.module, env.function, env.requires, env)
+          args, env.module, to_erl_env(env, cache))
         case expand do
-          { :ok, expanded } -> expanded
-          { :error, _ }     -> original
+          { :ok, _receiver, expanded } -> expanded
+          { :error, _ }                -> original
         end
     end
   end
 
   # Anything else is just returned
-  def expand(other, _env), do: other
+  defp expand(other, _env, _cache), do: other
+
+  defp to_erl_env(env, nil),    do: :elixir_scope.to_erl_env(env)
+  defp to_erl_env(_env, cache), do: cache
 
   ## Helpers
 
@@ -459,11 +589,6 @@ defmodule Macro do
     :lists.any(match?({ :&, _, [_] }, &1), args)
   end
 
-  defp expand_alias(h, env) do
-    atom = list_to_atom('Elixir-' ++ atom_to_list(h))
-    :elixir_aliases.lookup(atom, env.aliases)
-  end
-
   @doc """
   Recurs the quoted expression checking if all sub terms are
   safe (i.e. they represented data structured and don't actually
@@ -474,16 +599,16 @@ defmodule Macro do
     do_safe_term(terms) || :ok
   end
 
-  def do_safe_term({ local, _, terms }) when local in [:{}, :[], :__aliases__] do
+  defp do_safe_term({ local, _, terms }) when local in [:{}, :[], :__aliases__] do
     do_safe_term(terms)
   end
 
-  def do_safe_term({ unary, _, [term] }) when unary in [:+, :-] do
+  defp do_safe_term({ unary, _, [term] }) when unary in [:+, :-] do
     do_safe_term(term)
   end
 
-  def do_safe_term({ left, right }), do: do_safe_term(left) || do_safe_term(right)
-  def do_safe_term(terms) when is_list(terms),  do: Enum.find_value(terms, do_safe_term(&1))
-  def do_safe_term(terms) when is_tuple(terms), do: { :unsafe, terms }
-  def do_safe_term(_), do: nil
-end
+  defp do_safe_term({ left, right }), do: do_safe_term(left) || do_safe_term(right)
+  defp do_safe_term(terms) when is_list(terms),  do: Enum.find_value(terms, do_safe_term(&1))
+  defp do_safe_term(terms) when is_tuple(terms), do: { :unsafe, terms }
+  defp do_safe_term(_), do: nil
+end

lib/elixir/lib/macro/env.ex

@@ -1,55 +1,44 @@
 defmodule Macro.Env do
-  Record.deffunctions([:module, :file, :line, :function,
-    :aliases, :context, :requires, :macros], __MODULE__)
-
   @moduledoc """
-  A record that contains compile time environment information,
-  It can be accessed at any time by calling __ENV__.
+  A record that holds compile time environment information.
+
+  The current environment can be accessed at any time as
+  `__ENV__`. Inside macros, the caller environment can be
+  accessed as `__CALLER__`. It contains the following fields:
+
+  * `module` - the current module name.
+  * `file` - the current file name as a binary
+  * `line` - the current line as an integer
+  * `function` - a tuple as `{ atom, integer` }, where the first
+    element is the function name and the seconds its arity. Returns
+    `nil` if not inside a function
+  * `aliases` -  a list of two item tuples, where the first
+    item is the aliased name and the second the actual name
+  * `context` - the context of the environment. It can be nil
+    (default context), inside a guard or inside an assign
+  * `requires` - the list of required modules
+  * `functions` - a list of functions imported from each module
+  * `macros` - a list of macros imported from each module
   """
 
-  @doc """
-  Returns the current module name.
-  """
-  def module(record)
+  @type name_arity :: { atom, non_neg_integer }
+  @type file :: binary
+  @type line :: non_neg_integer
+  @type aliases :: [{ module, module }]
+  @type context :: :match | :guard | nil
+  @type requires :: [module]
+  @type functions :: [{ module, [name_arity] }]
+  @type macros :: [{ module, [name_arity] }]
 
-  @doc """
-  Returns the current file name as a binary.
-  """
-  def file(record)
+  fields = [:module, :file, :line, :function,
+            :aliases, :context, :requires, :functions, :macros]
 
-  @doc """
-  Returns the current line as an integer.
-  """
-  def line(record)
+  types  = quote do: [module: module, file: file, line: line,
+    function: name_arity, aliases: aliases, requires: requires,
+    functions: functions, macros: macros]
 
-  @doc """
-  Returns a tuple as { Atom, Integer }, where the first element
-  is the function name and the seconds its arity. Returns `nil`
-  if not inside a function.
-  """
-  def function(record)
-
-  @doc """
-  Returns a list of two item tuples, where the first
-  item is the aliased name and the second the actual name.
-  """
-  def aliases(record)
-
-  @doc """
-  Returns the context of the environment. It can be nil
-  (default context), inside a guard or inside an assign.
-  """
-  def context(record)
-
-  @doc """
-  Returns the list of required modules.
-  """
-  def requires(record)
-
-  @doc """
-  Returns a list of macros imported from each module.
-  """
-  def macros(record)
+  Record.deffunctions(fields, __MODULE__)
+  Record.deftypes(fields, types, __MODULE__)
 
   @doc """
   Returns a keyword list containing the file and line
@@ -69,5 +58,20 @@ defmodule Macro.Env do
   Returns wether the compilation environment is currently
   inside a match clause.
   """
-  def in_match?(record), do: context(record) == :assign
+  def in_match?(record), do: context(record) == :match
+
+  @doc """
+  Returns the environment stacktrace.
+  """
+  def stacktrace(record) do
+    cond do
+      nil?(record.module) ->
+        [{ :elixir_compiler, :__FILE__, 2, location(record) }]
+      nil?(record.function) ->
+        [{ module(record), :__MODULE__, 0, location(record) }]
+      true ->
+        { name, arity } = record.function
+        [{ module(record), name, arity, location(record) }]
+    end
+  end
 end

lib/elixir/lib/module.ex

@@ -1,6 +1,12 @@
 defmodule Module do
   require :ets, as: ETS
 
+  defmacrop is_env(env) do
+    quote do
+      is_tuple(unquote(env)) and size(unquote(env)) > 1 and elem(unquote(env), 0) == Macro.Env
+    end
+  end
+
   @moduledoc """
   This module provides many functions to deal with modules during
   compilation time. It allows a developer to dynamically attach
@@ -23,43 +29,40 @@ defmodule Module do
 
   @doc """
   Evalutes the quotes contents in the given module context.
+
+  A list of environment options can also be given as argument.
+  Check `Code.eval` for more information.
+
   Raises an error if the module was already compiled.
 
-  ## Options
-
-  This function accepts a list of options. The supported
-  options are:
-
-  * `:file` - The filename to be used in stacktraces
-    and the file reported in the __ENV__ variable.
-
-  * `:line` - The line reported in the __ENV__ variable.
-
   ## Examples
 
       defmodule Foo do
         contents = quote do: (def sum(a, b), do: a + b)
-        Module.eval_quoted __MODULE__, contents, []
+        Module.eval_quoted __MODULE__, contents
       end
 
       Foo.sum(1, 2) #=> 3
 
-  This function also accepts a `Macro.Env` as first argument. This
-  is useful to evalute the quoted contents inside an existing
-  environment (considering the environemnt module, line and file):
+  For convenience, you can my pass `__ENV__` as argument and
+  all options will be automatically extracted from the environment:
 
       defmodule Foo do
         contents = quote do: (def sum(a, b), do: a + b)
-        Module.eval_quoted __ENV__, contents, []
+        Module.eval_quoted __MODULE__, contents, [], __ENV__
       end
 
       Foo.sum(1, 2) #=> 3
 
   """
-  def eval_quoted(env, quoted, binding // [], opts // [])
+  def eval_quoted(module, quoted, binding // [], opts // [])
 
-  def eval_quoted(Macro.Env[module: module] = env, quoted, binding, opts) do
-    eval_quoted(module, quoted, binding, Keyword.merge(env.location, opts))
+  def eval_quoted(env, quoted, binding, opts) when is_env(env) do
+    eval_quoted(env.module, quoted, binding, Keyword.merge(env.to_keywords, opts))
+  end
+
+  def eval_quoted(module, quoted, binding, env) when is_env(env) do
+    eval_quoted(module, quoted, binding, env.to_keywords)
   end
 
   def eval_quoted(module, quoted, binding, opts) do
@@ -79,7 +82,7 @@ defmodule Module do
           def world, do: true
         end
 
-      Module.create(Hello, contents, __ENV__)
+      Module.create(Hello, contents, __ENV__.location)
 
       Hello.world #=> true
 
@@ -91,20 +94,19 @@ defmodule Module do
   function is preferred when the module body is given
   by a quoted expression.
 
-  Another important distinction is that `defmodule`
-  blends into the scope it is invoked, allowing you
-  to access all variables, imports and requires from
-  the module. `Module.create`, on the other hand, creates
-  a new scope so imports, requires, etc are not inherited.
+  Another important distinction is that `Module.create`
+  allows you to control the environment variables used
+  when defining the module, while `defmodule` automatically
+  shares the same environment.
   """
   def create(module, quoted, opts // [])
 
-  def create(module, quoted, Macro.Env[] = env) do
-    create(module, quoted, env.location)
+  def create(module, quoted, env) when is_env(env) do
+    create(module, quoted, env.to_keywords)
   end
 
   def create(module, quoted, opts) when is_atom(module) do
-    line = opts[:line] || 1
+    line = Keyword.get(opts, :line, 1)
     :elixir_module.compile(line, module, quoted, [], :elixir.scope_for_eval(opts))
   end
 
@@ -114,9 +116,12 @@ defmodule Module do
 
   ## Examples
 
-      Module.concat [Foo, Bar]    #=> Foo.Bar
-      Module.concat [Foo, "Bar"]  #=> Foo.Bar
-      Module.concat [Foo, 'Bar']  #=> Foo.Bar
+      iex> Module.concat [Foo, Bar]
+      Foo.Bar
+      iex> Module.concat [Foo, "Bar"]
+      Foo.Bar
+      iex> Module.concat [Foo, 'Bar']
+      Foo.Bar
 
   """
   def concat(list) when is_list(list) do
@@ -129,9 +134,12 @@ defmodule Module do
 
   ## Examples
 
-      Module.concat Foo, Bar    #=> Foo.Bar
-      Module.concat Foo, "Bar"  #=> Foo.Bar
-      Module.concat Foo, 'Bar'  #=> Foo.Bar
+      iex> Module.concat Foo, Bar
+      Foo.Bar
+      iex> Module.concat Foo, "Bar"
+      Foo.Bar
+      iex> Module.concat Foo, 'Bar'
+      Foo.Bar
 
   """
   def concat(left, right) do
@@ -146,11 +154,11 @@ defmodule Module do
 
   ## Examples
 
-      Module.safe_concat [Unknown, Module]
-      #=> ArgumentError
+      iex> Module.safe_concat [Unknown, Module]
+      ** (ArgumentError) argument error
 
-      Module.safe_concat [List, Chars]
-      #=> List.Chars
+      iex> Module.safe_concat [List, Chars]
+      List.Chars
 
   """
   def safe_concat(list) when is_list(list) do
@@ -165,11 +173,11 @@ defmodule Module do
 
   ## Examples
 
-      Module.safe_concat Unknown, Module
-      #=> ArgumentError
+      iex> Module.safe_concat Unknown, Module
+      ** (ArgumentError) argument error
 
-      Module.safe_concat List, Chars
-      #=> List.Chars
+      iex> Module.safe_concat List, Chars
+      List.Chars
 
   """
   def safe_concat(left, right) do
@@ -196,7 +204,7 @@ defmodule Module do
   end
 
   def add_doc(module, line, kind, tuple, signature, doc) when
-      kind in [:def, :defmacro, :defcallback] and (is_binary(doc) or is_boolean(doc) or doc == nil) do
+      kind in [:def, :defmacro] and (is_binary(doc) or is_boolean(doc) or doc == nil) do
     assert_not_compiled!(:add_doc, module)
     table = docs_table_for(module)
 
@@ -208,11 +216,11 @@ defmodule Module do
       [] ->
         ETS.insert(table, { tuple, line, kind, signature, doc })
         :ok
-      [{ tuple, line, old_kind, old_sign, old_doc }] when old_doc == nil or doc == nil or old_doc == doc ->
+      [{ tuple, line, _old_kind, old_sign, old_doc }] when old_doc == nil or doc == nil or old_doc == doc ->
         ETS.insert(table, {
           tuple,
           line,
-          merge_kind(old_kind, kind),
+          kind,
           merge_signatures(old_sign, signature, 1),
           doc || old_doc
         })
@@ -249,10 +257,6 @@ defmodule Module do
 
   # Merge
 
-  defp merge_kind(:defcallback, new), do: new
-  defp merge_kind(old, :defcallback), do: old
-  defp merge_kind(_old, new),         do: new
-
   defp merge_signatures([h1|t1], [h2|t2], i) do
     [merge_signature(h1, h2, i)|merge_signatures(t1, t2, i + 1)]
   end
@@ -371,8 +375,8 @@ defmodule Module do
           ETS.delete(table, tuple)
 
           old    = get_attribute(module, :__overridable)
-          new    = [ { tuple, { 1, [clause] } } ]
-          merged = :orddict.merge(fn(_k, { count, v1 }, _v2) -> { count + 1, [clause|v1] } end, old, new)
+          new    = [ { tuple, { 1, clause, false } } ]
+          merged = :orddict.merge(fn(_k, { count, _, _ }, _v2) -> { count + 1, clause, false } end, old, new)
 
           put_attribute(module, :__overridable, merged)
         _ ->
@@ -386,8 +390,7 @@ defmodule Module do
   Returns true if the given tuple in module is marked as overridable.
   """
   def overridable?(module, tuple) do
-    key = List.keyfind(get_attribute(module, :__overridable), tuple, 0)
-    match? { _, { _, [_|_] } }, key
+    !! List.keyfind(get_attribute(module, :__overridable), tuple, 0)
   end
 
   @doc """
@@ -421,12 +424,6 @@ defmodule Module do
     ETS.insert(table, { key, new })
   end
 
-  @doc false
-  def add_attribute(module, key, value) when is_atom(key) do
-    IO.write "[WARNING] Module.add_attribute is deprecated, please use Module.put_attribute instead\n#{Exception.formatted_stacktrace}"
-    put_attribute(module, key, value)
-  end
-
   @doc """
   Gets the given attribute from a module. If the attribute
   was marked as accumulate with `Module.register_attribute`,
@@ -456,12 +453,6 @@ defmodule Module do
     end
   end
 
-  @doc false
-  def read_attribute(module, key) when is_atom(key) do
-    IO.write "[WARNING] Module.read_attribute is deprecated, please use Module.get_attribute instead\n#{Exception.formatted_stacktrace}"
-    get_attribute(module, key)
-  end
-
   @doc """
   Deletes all attributes that matches the given key.
 
@@ -489,7 +480,8 @@ defmodule Module do
   When registering an attribute, two options can be given:
 
   * `:accumulate` - Several calls to the same attribute will
-    accumulate instead of override the previous one;
+    accumulate instead of override the previous one. New attributes
+    are always added to the top of the accumulated list.
 
   * `:persist` - The attribute will be persisted in the Erlang
     Abstract Format. Useful when interfacing with Erlang libraries.
@@ -536,19 +528,31 @@ defmodule Module do
 
   """
   def split(module) do
-    tl(String.split(to_binary(module), "-"))
+    tl(String.split(Binary.Chars.to_binary(module), "-"))
   end
 
+  @doc """
+  Convert a module name to binary without the Elixir prefix.
+  """
+  def to_binary(Elixir), do: "Elixir"
+
+  def to_binary(module) do
+    "Elixir-" <> rest = Binary.Chars.to_binary(module)
+    bc <<r>> inbits rest, do: <<to_dot(r)>>
+  end
+
+  defp to_dot(?-), do: ?.
+  defp to_dot(l),  do: l
+
   @doc false
   # Used internally to compile documentation. This function
   # is private and must be used only internally.
-  def compile_doc(env, kind, name, args, _guards, body) do
+  def compile_doc(env, kind, name, args, _guards, _body) do
     module = env.module
     line   = env.line
     arity  = length(args)
     pair   = { name, arity }
     doc    = get_attribute(module, :doc)
-    kind   = if kind == :def and body == nil and module != Kernel, do: :defcallback, else: kind
 
     case add_doc(module, line, kind, pair, args, doc) do
       :ok ->
@@ -565,7 +569,7 @@ defmodule Module do
   @doc false
   # Used internally to compile types. This function
   # is private and must be used only internally.
-  def compile_type(module, key, value) when is_atom(key) do
+  def compile_typespec(module, key, value) when is_atom(key) do
     assert_not_compiled!(:put_attribute, module)
     table = data_table_for(module)
 
@@ -589,7 +593,7 @@ defmodule Module do
     atom
   end
 
-  defp normalize_attribute(:file, Macro.Env[file: file, line: line]),       do: { binary_to_list(file), line}
+  defp normalize_attribute(:file, env) when is_env(env),                    do: { binary_to_list(env.file), env.line }
   defp normalize_attribute(:file, { binary, line }) when is_binary(binary), do: { binary_to_list(binary), line }
   defp normalize_attribute(:file, other) when not is_tuple(other),          do: normalize_attribute(:file, { other, 1 })
 

lib/elixir/lib/node.ex

@@ -3,9 +3,12 @@ defmodule Node do
   Functions related to Erlang nodes.
   """
 
+  @type t :: atom
+
   @doc """
   Returns the current node. It returns the same as the built-in node().
   """
+  @spec self :: t
   def self do
     :erlang.node()
   end
@@ -14,6 +17,7 @@ defmodule Node do
   Returns true if the local node is alive; that is, if the node can be
   part of a distributed system. Otherwise, it returns false.
   """
+  @spec alive? :: boolean
   def alive? do
     :erlang.is_alive()
   end
@@ -22,6 +26,7 @@ defmodule Node do
   Returns a list of all visible nodes in the system, excluding
   the local node. Same as list(visible).
   """
+  @spec list :: [t]
   def list do
     :erlang.nodes()
   end
@@ -30,9 +35,11 @@ defmodule Node do
   Returns a list of nodes according to argument given. The result
   returned when the argument is a list, is the list of nodes
   satisfying the disjunction(s) of the list elements.
-  
+
   See http://www.erlang.org/doc/man/erlang.html#nodes-1 for more info.
   """
+  @typep list_arg :: :visible | :hidden | :connected | :this | :known
+  @spec list(list_arg | [list_arg]) :: [t]
   def list(args) do
     :erlang.nodes(args)
   end
@@ -43,6 +50,7 @@ defmodule Node do
 
   See http://www.erlang.org/doc/man/erlang.html#monitor_node-2 for more info.
   """
+  @spec monitor(t, boolean) :: true
   def monitor(node, flag) do
     :erlang.monitor_node(node, flag)
   end
@@ -53,6 +61,7 @@ defmodule Node do
 
   See http://www.erlang.org/doc/man/erlang.html#monitor_node-3 for more info.
   """
+  @spec monitor(t, boolean, [:allow_passive_connect]) :: true
   def monitor(node, flag, options) do
     :erlang.monitor_node(node, flag, options)
   end
@@ -62,34 +71,74 @@ defmodule Node do
   the local node has crashed. This BIF is mainly used in the Erlang network
   authentication protocols. Returns true if disconnection succeeds, otherwise
   false. If the local node is not alive, the function returns ignored.
-  
+
   See http://www.erlang.org/doc/man/erlang.html#disconnect_node-1 for more info.
   """
+  @spec disconnect(t) :: boolean | :ignored
   def disconnect(node) do
     :erlang.disconnect_node(node)
   end
 
+  @doc """
+  Establishes a connection to Node. Returns true if successful,
+  false if not, and the atom `:ignored` if the local node is not
+  alive.
+
+  See http://erlang.org/doc/man/net_kernel.html#connect_node-1 for more info.
+  """
+  @spec connect(t) :: boolean | :ignored
+  def connect(node) do
+    :net_kernel.connect_node(node)
+  end
+
   @doc """
   Returns the pid of a new process started by the application of `fun`
   on `node`. If `node` does not exist, a useless pid is returned.
 
-  Check http://www.erlang.org/doc/man/erlang.html#spawn_opt-4 for
+  Check http://www.erlang.org/doc/man/erlang.html#spawn-2 for
   the list of available options.
   """
-  def spawn(node, fun, opts // []) do
+  @spec spawn(t, (() -> any)) :: pid
+  def spawn(node, fun) do
+    :erlang.spawn(node, fun)
+  end
+
+  @doc """
+  Returns the pid of a new process started by the application of `fun`
+  on `node`. If `node` does not exist, a useless pid is returned.
+
+  Check http://www.erlang.org/doc/man/erlang.html#spawn_opt-3 for
+  the list of available options.
+  """
+  @spec spawn(t, (() -> any), Process.spawn_opts) :: pid | {pid, reference}
+  def spawn(node, fun, opts) do
     :erlang.spawn_opt(node, fun, opts)
   end
 
   @doc """
   Returns the pid of a new process started by the application of
-  `module.function(args)` on `node`. If `node` does not exists, a useless
-  pid is returned.
+  `module.function(args)` on `node`. If `node` does not exists,
+  a useless pid is returned.
 
-  Check http://www.erlang.org/doc/man/erlang.html#spawn_opt-4 for
+  Check http://www.erlang.org/doc/man/erlang.html#spawn-4 for
   the list of available options.
   """
-  def spawn(node, module, fun, args, opts // []) do
-    :erlang.spawn(node, module, fun, args, opts)
+  @spec spawn(t, module, atom, [any]) :: pid
+  def spawn(node, module, fun, args) do
+    :erlang.spawn(node, module, fun, args)
+  end
+
+  @doc """
+  Returns the pid of a new process started by the application of
+  `module.function(args)` on `node`. If `node` does not exists,
+  a useless pid is returned.
+
+  Check http://www.erlang.org/doc/man/erlang.html#spawn_opt-5 for
+  the list of available options.
+  """
+  @spec spawn(t, module, atom, [any], Process.spawn_opts) :: pid | {pid, reference}
+  def spawn(node, module, fun, args, opts) do
+    :erlang.spawn_opt(node, module, fun, args, opts)
   end
 
   @doc """
@@ -99,6 +148,7 @@ defmodule Node do
   (and due to the link, an exit signal with exit reason :noconnection will be
   received).
   """
+  @spec spawn_link(t, (() -> any)) :: pid
   def spawn_link(node, fun) do
     :erlang.spawn_link(node, fun)
   end
@@ -110,6 +160,7 @@ defmodule Node do
   pid is returned (and due to the link, an exit signal with exit reason
   :noconnection will be received).
   """
+  @spec spawn_link(t, module, atom, [any]) :: pid
   def spawn_link(node, module, fun, args) do
     :erlang.spawn_link(node, module, fun, args)
   end

lib/elixir/lib/option_parser.ex

@@ -5,47 +5,57 @@ defmodule OptionParser do
 
   ## Examples
 
-      OptionParser.parse(["--debug"])
-      #=> { [debug: true], [] }
+      iex> OptionParser.parse(["--debug"])
+      { [debug: true], [] }
 
-      OptionParser.parse(["--source", "lib"])
-      #=> { [source: "lib"], [] }
+      iex> OptionParser.parse(["--source", "lib"])
+      { [source: "lib"], [] }
 
-      OptionParser.parse(["--source", "lib", "test/enum_test.exs", "--verbose"])
-      #=> { [source: "lib", verbose: true], ["test/enum_test.exs"] }
+      iex> OptionParser.parse(["--source", "lib", "test/enum_test.exs", "--verbose"])
+      { [source: "lib", verbose: true], ["test/enum_test.exs"] }
 
   ## Aliases
 
   A set of aliases can be given as second argument:
 
-      OptionParser.parse(["-d"], aliases: [d: :debug])
-      #=> { [debug: true], [] }
+      iex> OptionParser.parse(["-d"], aliases: [d: :debug])
+      { [debug: true], [] }
 
-  ## Flags
+  ## Switches
 
-  A set of flags can be given as argument too. Those are considered
-  boolean and never consume the next value unless it is a boolean:
+  Extra information about switches can be given as argument too.
+  This is useful in order to say a switch must behave as a boolean
+  or if duplicated switches should be kept, overriden or accumulated.
 
-      OptionParser.parse(["--unlock path/to/file"], flags: [:unlock])
-      #=> { [unlock: true], ["path/to/file"] }
+  The following types are supported:
 
-      OptionParser.parse(["--unlock false path/to/file"], flags: [:unlock])
-      #=> { [unlock: false], ["path/to/file"] }
+  * `:boolean` - Mark the given switch as boolean. Boolean switches
+                 never consumes the following value unless it is
+                 true or false;
+
+  The following extra options are supported:
+
+  * `:keep` - Keep duplicated items in the list instead of overriding;
+
+  Examples:
+
+      iex> OptionParser.parse(["--unlock", "path/to/file"], switches: [unlock: :boolean])
+      { [unlock: true], ["path/to/file"] }
+
+      iex> OptionParser.parse(["--unlock", "false", "path/to/file"], switches: [unlock: :boolean])
+      { [unlock: false], ["path/to/file"] }
 
   ## Negation switches
 
   Any switches starting with `--no-` are always considered to be
-  booleans and never parse the next value.
+  booleans and never parse the next value:
 
-      OptionParser.parse(["--no-op path/to/file"])
-      #=> { [no_op: true], ["path/to/file"] }
+      iex> OptionParser.parse(["--no-op", "path/to/file"])
+      { [no_op: true], ["path/to/file"] }
 
   """
   def parse(argv, opts // []) when is_list(argv) and is_list(opts) do
-    aliases = opts[:aliases] || []
-    flags   = opts[:flags]   || []
-    dict    = Keyword.new(flags, fn(k) -> { k, false } end)
-    parse(argv, aliases, flags, dict, [], true)
+    parse(argv, opts, true)
   end
 
   @doc """
@@ -56,59 +66,79 @@ defmodule OptionParser do
 
   ## Example
 
-      OptionParser.parse_head(["--source", "lib", "test/enum_test.exs", "--verbose"])
-      #=> { [source: "lib"], ["test/enum_test.exs", "--verbose"] }
+      iex> OptionParser.parse_head(["--source", "lib", "test/enum_test.exs", "--verbose"])
+      { [source: "lib"], ["test/enum_test.exs", "--verbose"] }
 
   """
   def parse_head(argv, opts // []) when is_list(argv) and is_list(opts) do
-    aliases = opts[:aliases] || []
-    flags   = opts[:flags]   || []
-    dict    = Keyword.new(flags, fn(k) -> { k, false } end)
-    parse(argv, aliases, flags, dict, [], false)
+    parse(argv, opts, false)
   end
 
   ## Helpers
 
-  defp parse(["-" <> option|t], aliases, flags, dict, args, all) do
+  defp parse(argv, opts, bool) do
+    aliases  = opts[:aliases]  || []
+    switches = opts[:switches] || []
+    parse(argv, aliases, switches, bool)
+  end
+
+  defp parse(argv, aliases, switches, all) do
+    parse(argv, aliases, switches, [], [], all)
+  end
+
+  defp parse(["-" <> option|t], aliases, switches, dict, args, all) do
     { option, value } = normalize_option(option, aliases)
+    kind = switches[option]
 
     if value == nil do
-      { value, t } = if is_flag?(flags, option) do
-        flag_from_tail(t)
-      else
-        value_from_tail(t)
-      end
+      { value, t } =
+        if is_switch_a? :boolean, kind do
+          boolean_from_tail(t)
+        else
+          value_from_tail(t)
+        end
     end
 
-    dict = store_option dict, option, value
-    parse(t, aliases, flags, dict, args, all)
+    dict = store_option dict, option, value, kind
+    parse(t, aliases, switches, dict, args, all)
   end
 
-  defp parse([], _, _, dict, args, true) do
-    { dict, Enum.reverse(args) }
+  defp parse([h|t], aliases, switches, dict, args, true) do
+    parse(t, aliases, switches, dict, [h|args], true)
   end
 
-  defp parse([h|t], aliases, flags, dict, args, true) do
-    parse(t, aliases, flags, dict, [h|args], true)
+  defp parse([], _, switches, dict, args, true) do
+    { reverse_dict(dict, switches), Enum.reverse(args) }
   end
 
-  defp parse(value, _, _, dict, _args, false) do
-    { dict, value }
+  defp parse(value, _, switches, dict, _args, false) do
+    { reverse_dict(dict, switches), value }
   end
 
-  defp flag_from_tail([h|t]) when h in ["false", "true"], do: { h, t }
-  defp flag_from_tail(t)                                , do: { true, t }
+  defp boolean_from_tail([h|t]) when h in ["false", "true"], do: { h, t }
+  defp boolean_from_tail(t)                                , do: { true, t }
 
   defp value_from_tail(["-" <> _|_] = t), do: { true, t }
   defp value_from_tail([h|t]),            do: { h, t }
   defp value_from_tail([]),               do: { true, [] }
 
-  defp store_option(dict, option, value) when value in ["false", "true"] do
-    store_option(dict, option, binary_to_atom(value))
+  defp store_option(dict, option, value, switches) when value in ["true", "false"] do
+    store_option dict, option, binary_to_atom(value), switches
   end
 
-  defp store_option(dict, option, value) do
-    Keyword.put dict, option, value
+  defp store_option(dict, option, value, kind) do
+    if is_switch_a? :keep, kind do
+      [{ option, value }|dict]
+    else
+      [{ option, value }|Keyword.delete(dict, option)]
+    end
+  end
+
+  defp reverse_dict(dict, switches) do
+    switches = lc { k, v } inlist switches,
+                  is_switch_a?(:boolean, v),
+                  not Keyword.has_key?(dict, k), do: { k, false }
+    Enum.reverse switches ++ dict
   end
 
   defp normalize_option(<<?-, option :: binary>>, aliases) do
@@ -118,7 +148,7 @@ defmodule OptionParser do
   defp normalize_option(option, aliases) do
     { option, value } = split_option(option)
     if is_no?(option), do: value = true
-    atom = option /> to_underscore /> binary_to_atom
+    atom = option |> to_underscore |> binary_to_atom
     { aliases[atom] || atom, value }
   end
 
@@ -136,5 +166,7 @@ defmodule OptionParser do
   defp is_no?("no-" <> _), do: true
   defp is_no?(_),          do: false
 
-  defp is_flag?(flags, option), do: List.member?(flags, option)
-end
+  defp is_switch_a?(kind, list) when is_list(list), do: List.member?(list, kind)
+  defp is_switch_a?(kind, kind), do: true
+  defp is_switch_a?(_, _),       do: false
+end

lib/elixir/lib/ord_dict.ex

@@ -1,95 +0,0 @@
-defmodule OrdDict do
-  @moduledoc """
-  This module implements a dictionary type that stores items
-  as a list of tuples. It is a simple wrapper around
-  [Erlang's orddict module](http://www.erlang.org/doc/man/orddict.html)
-  and exposed via the `Dict` module.
-
-  Check the `Dict` module for examples and documentation.
-  """
-
-  use Dict.Common
-
-  defmacrop dict(data) do
-    quote do
-      { OrdDict, unquote(data) }
-    end
-  end
-
-  @doc false
-  def keys(dict(data)) do
-    lc { k, _ } inlist data, do: k
-  end
-
-  @doc false
-  def values(dict(data)) do
-    lc { _, v } inlist data, do: v
-  end
-
-  @doc false
-  def size(dict(data)) do
-    length(data)
-  end
-
-  @doc false
-  def has_key?(dict(data), key) do
-    :orddict.is_key key, data
-  end
-
-  @doc false
-  def get(dict(data), key, default) do
-    case :orddict.find(key, data) do
-      {:ok, value} -> value
-      :error       -> default
-    end
-  end
-
-  @doc false
-  def put(dict(data), key, value) do
-    dict(:orddict.store key, value, data)
-  end
-
-  @doc false
-  def delete(dict(data), key) do
-    dict(:orddict.erase key, data)
-  end
-
-  @doc false
-  def merge(dict(d1), dict(d2), fun) do
-    dict(:orddict.merge fun, d1, d2)
-  end
-
-  @doc false
-  def merge(dict(_) = d1, d2, fun) do
-    merge(d1, new(d2), fun)
-  end
-
-  @doc false
-  def update(dict(data), key, fun) do
-    dict(:orddict.update key, fun, data)
-  end
-
-  @doc false
-  def update(dict(data), key, initial, fun) do
-    dict(:orddict.update key, fun, initial, data)
-  end
-
-  @doc false
-  def empty(_) do
-    dict([])
-  end
-
-  @doc false
-  def to_list(dict(data)) do
-    data
-  end
-end
-
-defimpl Enum.Iterator, for: OrdDict do
-  def iterator({ OrdDict, data }), do: data
-  def count({ OrdDict, data }),    do: length(data)
-end
-
-defimpl Access, for: OrdDict do
-  def access(dict, key), do: OrdDict.get(dict, key, nil)
-end

lib/elixir/lib/path.ex

@@ -0,0 +1,523 @@
+defmodule Path do
+  @moduledoc """
+  This module provides conveniences for manipulating or
+  retrieving filesystem paths.
+
+  The functions on this module may receive a char list or
+  a binary as argument and will return the given type.
+
+  The majority of the functions in this module do not
+  interact with the file system, unless some few functions
+  that needs to query the filesystem to retrieve paths
+  (like `Path.wildcard` and `Path.expand`).
+  """
+
+  alias :filename, as: FN
+
+  @type t :: char_list | atom | binary
+  @type r :: char_list | binary
+
+  @doc """
+  Converts the given filename and returns an absolute name.
+  Differently from `Path.expand/1`, no attempt is made to
+  resolve `..`, `.` or `~`.
+
+  ## Unix examples
+
+      Path.absname("foo")
+      #=> "/usr/local/foo"
+
+      Path.absname("../x")
+      #=> "/usr/local/../x"
+
+  ## Windows
+
+      Path.absname("foo").
+      "D:/usr/local/foo"
+      Path.absname("../x").
+      "D:/usr/local/../x"
+
+  """
+  def absname(path) do
+    FN.absname(path, get_cwd(path))
+  end
+
+  @doc """
+  Converts the given filename and returns an absolute name
+  relative to the given location. If the path is already
+  an absolute path, the relative path is ignored.
+
+  Differently from `Path.expand/2`, no attempt is made to
+  resolve `..`, `.` or `~`.
+
+  ## Examples
+
+      iex> Path.absname("foo", "bar")
+      "bar/foo"
+
+      iex> Path.absname("../x", "bar")
+      "bar/../x"
+
+  """
+  def absname(path, relative_to) do
+    FN.absname(path, relative_to)
+  end
+
+  @doc """
+  Expands the path by returning its absolute name and expanding
+  any `.` and `..` characters.
+
+  ## Examples
+
+      iex> Path.expand("/foo/bar/../bar")
+      "/foo/bar"
+
+  """
+  def expand(path) do
+    normalize FN.absname(expand_home(path), get_cwd(path))
+  end
+
+  @doc """
+  Expands the path to the relative location and expanding
+  any `.` and `..` characters. If the path is already an
+  absolute path, the relative location is ignored.
+
+  ## Examples
+
+      iex> Path.expand("foo/bar/../bar", "/baz")
+      "/baz/foo/bar"
+      iex> Path.expand("/foo/bar/../bar", "/baz")
+      "/foo/bar"
+
+  """
+  def expand(path, relative_to) do
+    normalize FN.absname(FN.absname(expand_home(path), relative_to), get_cwd(path))
+  end
+
+  @doc """
+  Returns the path type.
+
+  ## Unix examples
+
+      Path.type("/usr/local/bin")   #=> :absolute
+      Path.type("usr/local/bin")    #=> :relative
+      Path.type("../usr/local/bin") #=> :relative
+
+  ## Windows examples
+
+      Path.type("D:/usr/local/bin") #=> :absolute
+      Path.type("usr/local/bin")    #=> :relative
+      Path.type("D:bar.ex")         #=> :volumerelative
+      Path.type("/bar/foo.ex")      #=> :volumerelative
+
+  """
+  def type(name) when is_list(name) or is_binary(name) do
+    case :os.type() do
+      { :win32, _ } -> win32_pathtype(name)
+      _             -> unix_pathtype(name)
+    end |> elem(0)
+  end
+
+  @doc """
+  Forces the path to be a relative path.
+
+  ## Unix examples
+
+      Path.relative("/usr/local/bin")   #=> "usr/local/bin"
+      Path.relative("usr/local/bin")    #=> "usr/local/bin"
+      Path.relative("../usr/local/bin") #=> "../usr/local/bin"
+
+  ## Windows examples
+
+      Path.relative("D:/usr/local/bin") #=> "usr/local/bin"
+      Path.relative("usr/local/bin")    #=> "usr/local/bin"
+      Path.relative("D:bar.ex")         #=> "bar.ex"
+      Path.relative("/bar/foo.ex")      #=> "bar/foo.ex"
+
+  """
+  def relative(name) do
+    case :os.type() do
+      { :win32, _ } -> win32_pathtype(name)
+      _             -> unix_pathtype(name)
+    end |> elem(1)
+  end
+
+  defp unix_pathtype(<<?/, relative :: binary>>), do:
+    { :absolute, relative }
+  defp unix_pathtype([?/|relative]), do:
+    { :absolute, relative }
+  defp unix_pathtype([list|rest]) when is_list(list), do:
+    unix_pathtype(list ++ rest)
+  defp unix_pathtype([atom|rest]) when is_atom(atom), do:
+    unix_pathtype(atom_to_list(atom) ++ rest)
+  defp unix_pathtype(relative), do:
+    { :relative, relative }
+
+  @slash [?/, ?\\]
+
+  defp win32_pathtype([list|rest]) when is_list(list), do:
+    win32_pathtype(list++rest)
+  defp win32_pathtype([atom|rest]) when is_atom(atom), do:
+    win32_pathtype(atom_to_list(atom)++rest)
+  defp win32_pathtype([char, list|rest]) when is_list(list), do:
+    win32_pathtype([char|list++rest])
+  defp win32_pathtype(<<c1, c2, relative :: binary>>) when c1 in @slash and c2 in @slash, do:
+    { :absolute, relative }
+  defp win32_pathtype(<<c, relative :: binary>>) when c in @slash, do:
+    { :volumerelative, relative }
+  defp win32_pathtype(<<_letter, ?:, c, relative :: binary>>) when c in @slash, do:
+    { :absolute, relative }
+  defp win32_pathtype(<<_letter, ?:, relative :: binary>>), do:
+    { :volumerelative, relative }
+
+  defp win32_pathtype([c1, c2 | relative]) when c1 in @slash and c2 in @slash, do:
+    { :absolute, relative }
+  defp win32_pathtype([c | relative]) when c in @slash, do:
+    { :volumerelative, relative }
+  defp win32_pathtype([c1, c2, list|rest]) when is_list(list), do:
+    win32_pathtype([c1, c2|list++rest])
+  defp win32_pathtype([_letter, ?:, c | relative]) when c in @slash, do:
+    { :absolute, relative }
+  defp win32_pathtype([_letter, ?: | relative]), do:
+    { :volumerelative, relative }
+  defp win32_pathtype(relative), do:
+    { :relative, relative }
+
+  @doc """
+  Returns the given `path` relative to the given `from` path.
+
+  This function does not query the filesystem, so it assumes
+  no symlinks in between the paths.
+
+  In case a direct relative path cannot be found, it returns
+  the original path.
+
+  ## Examples
+
+      iex> Path.relative_to("/usr/local/foo", "/usr/local")
+      "foo"
+      iex> Path.relative_to("/usr/local/foo", "/")
+      "usr/local/foo"
+      iex> Path.relative_to("/usr/local/foo", "/etc")
+      "/usr/local/foo"
+
+  """
+  def relative_to(path, from) when is_list(path) and is_binary(from) do
+    path = filename_string_to_binary(path)
+    relative_to(FN.split(path), FN.split(from), path)
+  end
+
+  def relative_to(path, from) when is_binary(path) and is_list(from) do
+    relative_to(FN.split(path), FN.split(filename_string_to_binary(from)), path)
+  end
+
+  def relative_to(path, from) do
+    relative_to(FN.split(path), FN.split(from), path)
+  end
+
+  defp relative_to([h|t1], [h|t2], original) do
+    relative_to(t1, t2, original)
+  end
+
+  defp relative_to([_|_] = l1, [], _original) do
+    FN.join(l1)
+  end
+
+  defp relative_to(_, _, original) do
+    original
+  end
+
+  @doc """
+  Returns the last component of the path or the path
+  itself if it does not contain any directory separators.
+
+  ## Examples
+
+      iex> Path.basename("foo")
+      "foo"
+
+      iex> Path.basename("foo/bar")
+      "bar"
+
+      iex> Path.basename("/")
+      ""
+
+  """
+  def basename(path) do
+    FN.basename(path)
+  end
+
+  @doc """
+  Returns the last component of `path` with the `extension`
+  stripped. This function should be used to remove a specific
+  extension which may, or may not, be there.
+
+  ## Examples
+
+      iex> Path.basename("~/foo/bar.ex", ".ex")
+      "bar"
+      iex> Path.basename("~/foo/bar.exs", ".ex")
+      "bar.exs"
+      iex> Path.basename("~/foo/bar.old.ex", ".ex")
+      "bar.old"
+
+  """
+  def basename(path, extension) do
+    FN.basename(path, extension)
+  end
+
+  @doc """
+  Return the `directory` component of `path`.
+
+  ## Examples
+
+      Path.dirname("/foo/bar.ex")
+      #=> "foo"
+
+  """
+  def dirname(path) do
+    FN.dirname(path)
+  end
+
+  @doc """
+  Return the `extension` of the last component of `path`.
+
+  ## Examples
+
+      iex> Path.extname("foo.erl")
+      ".erl"
+      iex> Path.extname("~/foo/bar")
+      ""
+
+  """
+  def extname(path) do
+    FN.extension(path)
+  end
+
+  @doc """
+  Returns the `path` with the `extension` stripped.
+
+  ## Examples
+
+      iex> Path.rootname("/foo/bar")
+      "/foo/bar"
+      iex> Path.rootname("/foo/bar.ex")
+      "/foo/bar"
+
+  """
+  def rootname(path) do
+    FN.rootname(path)
+  end
+
+  @doc """
+  Returns the `path` with the `extension` stripped. This function should be used to
+  remove a specific extension which might, or might not, be there.
+
+  ## Examples
+
+      iex> Path.rootname("/foo/bar.erl", ".erl")
+      "/foo/bar"
+      iex> Path.rootname("/foo/bar.erl", ".ex")
+      "/foo/bar.erl"
+
+  """
+  def rootname(path, extension) do
+    FN.rootname(path, extension)
+  end
+
+  @doc """
+  Returns a string with one or more paths components joint by the path separator.
+  This function should be used to convert a list of strings in a path.
+
+  ## Examples
+
+      iex> Path.join(["~", "foo"])
+      "~/foo"
+      iex> Path.join(["foo"])
+      "foo"
+      iex> Path.join(["/", "foo", "bar"])
+      "/foo/bar"
+
+  """
+  def join([name1, name2|rest]), do:
+    join([join(name1, name2)|rest])
+  def join([name]) when is_list(name), do:
+    binary_to_filename_string(do_join(filename_string_to_binary(name), <<>>, [], major_os_type()))
+  def join([name]) when is_binary(name), do:
+    do_join(name, <<>>, [], major_os_type())
+
+  @doc """
+  Joins two paths.
+
+  ## Examples
+
+      iex> Path.join("foo", "bar")
+      "foo/bar"
+
+  """
+  def join(left, right) when is_binary(left) and is_binary(right), do:
+    do_join(left, Path.relative(right), [], major_os_type())
+
+  def join(left, right) when is_binary(left) and is_list(right), do:
+    join(left, filename_string_to_binary(right))
+
+  def join(left, right) when is_list(left) and is_binary(right), do:
+    join(filename_string_to_binary(left), right)
+
+  def join(left, right) when is_list(left) and is_list(right), do:
+    binary_to_filename_string join(filename_string_to_binary(left), filename_string_to_binary(right))
+
+  def join(left, right) when is_atom(left), do:
+    join(atom_to_binary(left), right)
+
+  def join(left, right) when is_atom(right), do:
+    join(left, atom_to_binary(right))
+
+  defp major_os_type do
+    case :os.type do
+      { maj, _ } -> maj
+      maj -> maj
+    end
+  end
+
+  defp do_join(<<uc_letter, ?:, rest :: binary>>, relativename, [], :win32) when uc_letter in ?A..?Z, do:
+    do_join(rest, relativename, [?:, uc_letter+?a-?A], :win32)
+  defp do_join(<<?\\,rest :: binary>>, relativename, result, :win32), do:
+    do_join(<<?/,rest :: binary>>, relativename, result, :win32)
+  defp do_join(<<?/,rest :: binary>>, relativename, [?., ?/|result], os_type), do:
+    do_join(rest, relativename, [?/|result], os_type)
+  defp do_join(<<?/,rest :: binary>>, relativename, [?/|result], os_type), do:
+    do_join(rest, relativename, [?/|result], os_type)
+  defp do_join(<<>>, <<>>, result, os_type), do:
+    list_to_binary(maybe_remove_dirsep(result, os_type))
+  defp do_join(<<>>, relativename, [?:|rest], :win32), do:
+    do_join(relativename, <<>>, [?:|rest], :win32)
+  defp do_join(<<>>, relativename, [?/|result], os_type), do:
+    do_join(relativename, <<>>, [?/|result], os_type)
+  defp do_join(<<>>, relativename, result, os_type), do:
+    do_join(relativename, <<>>, [?/|result], os_type)
+  defp do_join(<<char,rest :: binary>>, relativename, result, os_type) when is_integer(char), do:
+    do_join(rest, relativename, [char|result], os_type)
+
+  defp maybe_remove_dirsep([?/, ?:, letter], :win32), do:
+    [letter, ?:, ?/]
+  defp maybe_remove_dirsep([?/], _), do:
+    [?/]
+  defp maybe_remove_dirsep([?/|name], _), do:
+    :lists.reverse(name)
+  defp maybe_remove_dirsep(name, _), do:
+    :lists.reverse(name)
+
+  @doc """
+  Returns a list with the path splitted by the path separator.
+  If an empty string is given, then it returns the root path.
+
+  ## Examples
+
+       iex> Path.split("")
+       ["/"]
+       iex> Path.split("foo")
+       ["foo"]
+       iex> Path.split("/foo/bar")
+       ["/", "foo", "bar"]
+
+  """
+  def split(path) do
+    FN.split(path)
+  end
+
+  @doc """
+  Traverses paths according to the given `glob` expression.
+
+  The wildcard looks like an ordinary path, except that certain
+  "wildcard characters" are interpreted in a special way. The
+  following characters are special:
+
+  * `?` - Matches one character.
+  * `*` - Matches any number of characters up to the end of
+          the filename, the next dot, or the next slash.
+  * `**` - Two adjacent <c>*</c>'s used as a single pattern will
+           match all files and zero or more directories and subdirectories.
+  * `[char1,char2,...]` - Matches any of the characters listed. Two characters
+                          separated by a hyphen will match a range of characters.
+  * `{item1,item2,...}` - Matches one of the alternatives.
+
+  Other characters represent themselves. Only paths that have
+  exactly the same character in the same position will match. Note
+  that matching is case-sensitive; i.e. "a" will not match "A".
+
+  ## Examples
+
+  Imagine you have a directory called `projects` with three Elixir projects
+  inside of it: `elixir`, `exdoc` and `dynamo`. You can find all `.beam` files
+  inside their ebin directories all projects as follows:
+
+      Path.wildcard("projects/*/ebin/**/*.beam")
+
+  If you want to search for both `.beam` and `.app` files, you could do:
+
+      Path.wildcard("projects/*/ebin/**/*.{beam,app}")
+
+  """
+  def wildcard(glob) when is_binary(glob) do
+    paths = :elixir_glob.wildcard :unicode.characters_to_list(glob)
+    Enum.map paths, :unicode.characters_to_binary(&1)
+  end
+
+  def wildcard(glob) when is_list(glob) do
+    :elixir_glob.wildcard glob
+  end
+
+  ## Helpers
+
+  defp get_cwd(path) when is_list(path), do: System.cwd! |> binary_to_filename_string
+  defp get_cwd(_), do: System.cwd!
+
+  defp binary_to_filename_string(binary) do
+    case :unicode.characters_to_list(binary) do
+      { :error, _, _ } ->
+        :erlang.error(:badarg)
+      list when is_list(list) ->
+        list
+    end
+  end
+
+  defp filename_string_to_binary(list) do
+    case :unicode.characters_to_binary(:filename.flatten(list), :unicode, :file.native_name_encoding()) do
+      { :error, _, _ } ->
+        :erlang.error(:badarg)
+      bin when is_binary(bin) ->
+        bin
+    end
+  end
+
+  # Normalize the given path by expanding "..", "." and "~".
+
+  defp expand_home(<<?~, rest :: binary>>) do
+    System.user_home! <> rest
+  end
+
+  defp expand_home('~' ++ rest) do
+    (System.user_home! |> binary_to_filename_string) ++ rest
+  end
+
+  defp expand_home(other), do: other
+
+  defp normalize(path), do: normalize(FN.split(path), [])
+
+  defp normalize([top|t], [_|acc]) when top in ["..", '..'] do
+    normalize t, acc
+  end
+
+  defp normalize([top|t], acc) when top in [".", '.'] do
+    normalize t, acc
+  end
+
+  defp normalize([h|t], acc) do
+    normalize t, [h|acc]
+  end
+
+  defp normalize([], acc) do
+    join Enum.reverse(acc)
+  end
+end

lib/elixir/lib/process.ex

@@ -14,22 +14,22 @@ defmodule Process do
 
   `pid` must refer to a process at the local node.
   """
+  @spec alive?(pid) :: boolean
   def alive?(pid) do
     :erlang.is_process_alive(pid)
   end
 
-  @doc """
-  Returns the current process.
-  """
+  @doc false
   def self do
+    IO.puts "Process.self is deprecated, please use Kernel.self instead"
+    Exception.print_stacktrace
     :erlang.self()
   end
 
   @doc """
-  Returns all key-values in the dictionary
-  with no specific ordering (i.e. they are
-  not a keyword list).
+  Returns all key-values in the dictionary.
   """
+  @spec get :: [{term, term}]
   def get do
     :erlang.get()
   end
@@ -37,6 +37,8 @@ defmodule Process do
   @doc """
   Returns the value for the given key.
   """
+  @spec get(term) :: term
+  @spec get(term, default :: term) :: term
   def get(key, default // nil) do
     case :erlang.get(key) do
       :undefined ->
@@ -49,6 +51,7 @@ defmodule Process do
   @doc """
   Returns all keys that have the given `value`.
   """
+  @spec get_keys(term) :: [term]
   def get_keys(value) do
     :erlang.get_keys(value)
   end
@@ -56,6 +59,7 @@ defmodule Process do
   @doc """
   Stores the given key-value in the process dictionary.
   """
+  @spec put(term, term) :: term | nil
   def put(key, value) do
     nillify :erlang.put(key, value)
   end
@@ -63,6 +67,7 @@ defmodule Process do
   @doc """
   Deletes all items in the dictionary.
   """
+  @spec delete :: [{term, term}]
   def delete() do
     :erlang.erase()
   end
@@ -70,6 +75,7 @@ defmodule Process do
   @doc """
   Deletes the given key from the dictionary.
   """
+  @spec delete(term) :: term | nil
   def delete(key) do
     nillify :erlang.erase(key)
   end
@@ -97,18 +103,34 @@ defmodule Process do
       Process.exit(pid, :kill)
 
   """
-  def exit(pid, status) do
-    :erlang.exit(pid, status)
+  @spec exit(pid, term) :: true
+  def exit(pid, reason) do
+    :erlang.exit(pid, reason)
   end
 
   @doc """
   Returns the pid of a new process started by the application of `fun`.
-
-  It behaves exactly the same as the `Kernel.spawn/1` function except
-  it also accepts extra options, for the list of available options
-  check http://www.erlang.org/doc/man/erlang.html#spawn_opt-4
+  It behaves exactly the same as `Kernel.spawn/1`.
   """
-  def spawn(fun, opts // []) do
+  @spec spawn((() -> any)) :: pid
+  def spawn(fun) do
+    :erlang.spawn(fun)
+  end
+
+  @type spawn_opt  :: :link | :monitor | {:priority, :low | :normal | :high} |
+                      {:fullsweep_after, non_neg_integer} |
+                      {:min_heap_size, non_neg_integer} |
+                      {:min_bin_vheap_size, non_neg_integer}
+  @type spawn_opts :: [spawn_opt]
+
+  @doc """
+  Returns the pid of a new process started by the application of `fun`.
+
+  It also accepts extra options, for the list of available options
+  check http://www.erlang.org/doc/man/erlang.html#spawn_opt-2
+  """
+  @spec spawn((() -> any), spawn_opts) :: pid | {pid, reference}
+  def spawn(fun, opts) do
     :erlang.spawn_opt(fun, opts)
   end
 
@@ -117,12 +139,24 @@ defmodule Process do
   `module.function(args)`. The new process created will be placed in the system
   scheduler queue and be run some time later.
 
-  It behaves exactly the same as the `Kernel.spawn/3` function except
-  it also accepts extra options, for the list of available options
+  It behaves exactly the same as the `Kernel.spawn/3` function.
+  """
+  @spec spawn(module, atom, [any]) :: pid
+  def spawn(mod, fun, args) do
+    :erlang.spawn(mod, fun, args)
+  end
+
+  @doc """
+  Returns the pid of a new process started by the application of
+  `module.function(args)`. The new process created will be placed in the system
+  scheduler queue and be run some time later.
+
+  It also accepts extra options, for the list of available options
   check http://www.erlang.org/doc/man/erlang.html#spawn_opt-4
 
   """
-  def spawn(mod, fun, args, opts // []) do
+  @spec spawn(module, atom, [any], spawn_opts) :: pid | {pid, reference}
+  def spawn(mod, fun, args, opts) do
     :erlang.spawn_opt(mod, fun, args, opts)
   end
 
@@ -131,6 +165,7 @@ defmodule Process do
   A link is created between the calling process and the new
   process, atomically.
   """
+  @spec spawn_link((() -> any)) :: pid
   def spawn_link(fun) do
     :erlang.spawn_link(fun)
   end
@@ -140,6 +175,7 @@ defmodule Process do
   `module.function(args)`. A link is created between the calling process
   and the new process, atomically. Otherwise works like spawn/3.
   """
+  @spec spawn_link(module, atom, [any]) :: pid
   def spawn_link(mod, fun, args) do
     :erlang.spawn_link(mod, fun, args)
   end
@@ -148,6 +184,7 @@ defmodule Process do
   Returns the pid of a new process started by the application of `fun`
   and reference for a monitor created to the new process.
   """
+  @spec spawn_monitor((() -> any)) :: {pid, reference}
   def spawn_monitor(fun) do
     :erlang.spawn_monitor(fun)
   end
@@ -157,6 +194,7 @@ defmodule Process do
   and the process is monitored at the same time. Returns the pid and a
   reference for the monitor. Otherwise works like spawn/3.
   """
+  @spec spawn_monitor(module, atom, [any]) :: {pid, reference}
   def spawn_monitor(mod, fun, args) do
     :erlang.spawn_monitor(mod, fun, args)
   end
@@ -167,6 +205,7 @@ defmodule Process do
 
   See http://www.erlang.org/doc/man/erlang.html#monitor-2 for more info.
   """
+  @spec monitor(pid | {reg_name :: atom, node :: atom} | reg_name :: atom) :: reference
   def monitor(item) do
     :erlang.monitor(:process, item)
   end
@@ -178,6 +217,8 @@ defmodule Process do
 
   See http://www.erlang.org/doc/man/erlang.html#demonitor-2 for more info.
   """
+  @spec demonitor(reference) :: true
+  @spec demonitor(reference, options :: [:flush | :info]) :: boolean
   def demonitor(monitor_ref, options // []) do
     :erlang.demonitor(monitor_ref, options)
   end
@@ -192,6 +233,7 @@ defmodule Process do
 
   See http://www.erlang.org/doc/man/erlang.html#processes-0 for more info.
   """
+  @spec list :: [pid]
   def list do
     :erlang.processes()
   end
@@ -202,6 +244,7 @@ defmodule Process do
 
   See http://www.erlang.org/doc/man/erlang.html#link-1 for more info.
   """
+  @spec link(pid | port) :: true
   def link(pid) do
     :erlang.link(pid)
   end
@@ -213,6 +256,7 @@ defmodule Process do
 
   See http://www.erlang.org/doc/man/erlang.html#unlink-1 for more info.
   """
+  @spec unlink(pid | port) :: true
   def unlink(pid) do
     :erlang.unlink(pid)
   end
@@ -224,6 +268,7 @@ defmodule Process do
 
   See http://www.erlang.org/doc/man/erlang.html#register-2 for more info.
   """
+  @spec register(pid | port, atom) :: true
   def register(pid, name) do
     :erlang.register(name, pid)
   end
@@ -233,6 +278,7 @@ defmodule Process do
 
   See http://www.erlang.org/doc/man/erlang.html#unregister-1 for more info.
   """
+  @spec unregister(atom) :: true
   def unregister(name) do
     :erlang.unregister(name)
   end
@@ -243,6 +289,7 @@ defmodule Process do
 
   See http://www.erlang.org/doc/man/erlang.html#whereis-1 for more info.
   """
+  @spec whereis(atom) :: pid | port | nil
   def whereis(name) do
     nillify :erlang.whereis(name)
   end
@@ -250,6 +297,7 @@ defmodule Process do
   @doc """
   Returns the pid of the group leader for the process which evaluates the function.
   """
+  @spec group_leader :: pid
   def group_leader do
     :erlang.group_leader
   end
@@ -258,6 +306,7 @@ defmodule Process do
   Sets the group leader of Pid to GroupLeader. Typically, this is used when a processes
   started from a certain shell should have another group leader than `:init`.
   """
+  @spec group_leader(leader :: pid, pid) :: true
   def group_leader(leader, pid) do
     :erlang.group_leader(leader, pid)
   end
@@ -265,16 +314,21 @@ defmodule Process do
   @doc """
   Returns a list of names which have been registered using register/2.
   """
+  @spec registered :: [atom]
   def registered do
     :erlang.registered()
   end
 
+  @typep process_flag :: :trap_exit | :error_handler | :min_heap_size |
+                         :min_bin_vheap_size | :priority | :save_calls |
+                         :sensitive
   @doc """
   Sets certain flags for the process which calls this function.
   Returns the old value of the flag.
 
   See http://www.erlang.org/doc/man/erlang.html#process_flag-2 for more info.
   """
+  @spec flag(process_flag, term) :: term
   def flag(flag, value) do
     :erlang.process_flag(flag, value)
   end
@@ -286,6 +340,7 @@ defmodule Process do
 
   See http://www.erlang.org/doc/man/erlang.html#process_flag-3 for more info.
   """
+  @spec flag(pid, process_flag, term) :: term
   def flag(pid, flag, value) do
     :erlang.process_flag(pid, flag, value)
   end
@@ -296,6 +351,7 @@ defmodule Process do
 
   See http://www.erlang.org/doc/man/erlang.html#process_info-1 for more info.
   """
+  @spec info(pid) :: Keyword.t
   def info(pid) do
     :erlang.process_info(pid)
   end
@@ -306,6 +362,7 @@ defmodule Process do
 
   See http://www.erlang.org/doc/man/erlang.html#process_info-2 for more info.
   """
+  @spec info(pid, atom) :: {atom, term}
   def info(pid, spec) do
     :erlang.process_info(pid, spec)
   end

lib/elixir/lib/protocol.ex

@@ -23,13 +23,14 @@ defmodule Protocol do
       defmodule unquote(name) do
         # We don't allow function definition inside protocols
         import Kernel, except: [
+          defmacrop: 1, defmacrop: 2, defmacrop: 4,
           defmacro: 1, defmacro: 2, defmacro: 4,
           defp: 1, defp: 2, defp: 4,
           def: 1, def: 2, def: 4
         ]
 
         # Import the new dsl that holds the new def
-        import Protocol.DSL
+        import :macros, Protocol.DSL
 
         # Set up a clear slate to store defined functions
         @functions []
@@ -38,9 +39,9 @@ defmodule Protocol do
         unquote(block)
 
         # Define callbacks and meta information
-        { conversions, fallback } = Protocol.conversions_for(__MODULE__, @only, @except)
-        Protocol.impl_for(conversions, fallback, __ENV__)
-        Protocol.meta(@functions, fallback, __ENV__)
+        { conversions, fallback, returns_nil } = Protocol.conversions_for(__MODULE__, @only, @except)
+        Protocol.impl_for(conversions, fallback, returns_nil, __ENV__)
+        Protocol.meta(@functions, conversions, fallback, returns_nil, __ENV__)
       end
     end
   end
@@ -74,10 +75,9 @@ defmodule Protocol do
     end
   end
 
-  @doc """
-  Check if the given module is a protocol. Raises an error
-  if not loaded or not a protocol.
-  """
+  # Check if the given module is a protocol. Raises an error
+  # if not loaded or not a protocol.
+  @doc false
   def assert_protocol(module) do
     case Code.ensure_compiled(module) do
       { :module, ^module } -> nil
@@ -92,60 +92,16 @@ defmodule Protocol do
     end
   end
 
-  @doc """
-  Defines meta information about the protocol and internal callbacks.
-  """
-  def meta(functions, fallback, env) do
-    contents = quote do
-      def __protocol__(:name),      do: __MODULE__
-      def __protocol__(:functions), do: unquote(:lists.sort(functions))
-
-      @doc false
-      def behaviour_info(:callbacks), do: [{ :__impl__, 0 }|__protocol__(:functions)]
-
-      def __impl_for__(arg) do
-        case __raw_impl__(arg) do
-          __MODULE__.Record ->
-            target = Module.concat(__MODULE__, :erlang.element(1, arg))
-            try do
-              target.__impl__
-              target
-            catch
-              :error, :undef, [[{ ^target, :__impl__, [], _ }|_]|_] ->
-                __fallback__
-            end
-          other ->
-            other
-        end
-      end
-
-      def __impl_for__!(arg) do
-        if module = __impl_for__(arg) do
-          module
-        else
-          raise Protocol.UndefinedError, protocol: __MODULE__, structure: arg
-        end
-      end
-
-      defp __fallback__, do: unquote(fallback)
-    end
-
-    Module.eval_quoted env, contents
-  end
-
-  @doc """
-  Implements the function that detects the protocol and returns
-  the module to dispatch to. Returns module.Record for records
-  which should be properly handled by the dispatching function.
-  """
-  def impl_for(conversions, fallback, env) do
+  # Implements the function that detects the protocol and returns
+  # the module to dispatch to. Returns module.Record for records
+  # which should be properly handled by the dispatching function.
+  @doc false
+  def impl_for(conversions, fallback, returns_nil, env) do
     contents = lc kind inlist conversions do
-      each_impl_for(kind, if fallback, do: conversions)
+      each_impl_for(kind, conversions, fallback)
     end
 
-    # If we don't implement all protocols and any is not in the
-    # list, we need to add a final clause that returns nil.
-    if !L.keyfind(Any, 1, conversions) && length(conversions) < 10 do
+    if returns_nil do
       contents = contents ++ [quote do
         defp __raw_impl__(_) do
           nil
@@ -153,13 +109,79 @@ defmodule Protocol do
       end]
     end
 
-    Module.eval_quoted env, contents
+    Module.eval_quoted env.module, contents, [], env.location
   end
 
-  @doc """
-  Returns the default conversions according to the given
-  only/except options.
-  """
+  # Defines meta information about the protocol and internal callbacks.
+  @doc false
+  def meta(functions, conversions, fallback, returns_nil, env) do
+    any     = L.keyfind(Any, 1, conversions) != false
+    records = L.keyfind(Record, 1, conversions) != false
+
+    meta = quote location: :keep do
+      unless Kernel.Typespec.defines_type?(__MODULE__, :t, 0) do
+        @type t :: unquote(generate_type(conversions, any))
+      end
+
+      @doc false
+      def __protocol__(:name),      do: __MODULE__
+      def __protocol__(:functions), do: unquote(:lists.sort(functions))
+    end
+
+    impl_for = cond do
+      records and fallback ->
+        quote location: :keep do
+          def __impl_for__(arg) do
+            case __raw_impl__(arg) do
+              __MODULE__.Record ->
+                target = Module.concat(__MODULE__, :erlang.element(1, arg))
+                try do
+                  target.__impl__
+                  target
+                catch
+                  :error, :undef, [[{ ^target, :__impl__, [], _ }|_]|_] ->
+                    unquote(fallback)
+                end
+              other ->
+                other
+            end
+          end
+        end
+      records ->
+        quote location: :keep do
+          def __impl_for__(arg) do
+            case __raw_impl__(arg) do
+              __MODULE__.Record ->
+                Module.concat(__MODULE__, :erlang.element(1, arg))
+              other ->
+                other
+            end
+          end
+        end
+      true ->
+        quote location: :keep do
+          def __impl_for__(arg), do: __raw_impl__(arg)
+        end
+    end
+
+    impl_bang = if returns_nil do
+      quote do
+        def __impl_for__!(arg) do
+          __impl_for__(arg) || raise(Protocol.UndefinedError, protocol: __MODULE__, structure: arg)
+        end
+      end
+    else
+      quote do
+        def __impl_for__!(arg), do: __impl_for__(arg)
+      end
+    end
+
+    Module.eval_quoted env.module, [meta, impl_for, impl_bang], [], env.location
+  end
+
+  # Returns the default conversions according to the given
+  # only/except options.
+  @doc false
   def conversions_for(module, only, except) do
     kinds = all_types
 
@@ -180,65 +202,73 @@ defmodule Protocol do
         nil
     end
 
-    { conversions, fallback }
+    # If any is not in the list and we don't implement all
+    # protocols, we need to handle nil cases.
+    returns_nil = L.keyfind(Any, 1, conversions) == false and length(conversions) < 10
+    { conversions, fallback, returns_nil }
   end
 
   ## Helpers
 
+  defp generate_type(_conversions, true) do
+    quote(do: any)
+  end
+
+  defp generate_type(conversions, false) do
+    or_function     = fn({ _, _, x }, acc) -> { :|, [], [acc, x] } end
+    { _, _, first } = hd(conversions)
+    :lists.foldl(or_function, first, tl(conversions))
+  end
+
   defp all_types do
-    [
-      { Record,    :is_record },
-      { Tuple,     :is_tuple },
-      { Atom,      :is_atom },
-      { List,      :is_list },
-      { BitString, :is_bitstring },
-      { Number,    :is_number },
-      { Function,  :is_function },
-      { PID,       :is_pid },
-      { Port,      :is_port },
-      { Reference, :is_reference },
-      { Any,       :is_any }
-    ]
+    [ { Record,    :is_record,    quote do: tuple },
+      { Tuple,     :is_tuple,     quote do: tuple },
+      { Atom,      :is_atom,      quote do: atom },
+      { List,      :is_list,      quote do: list },
+      { BitString, :is_bitstring, quote do: <<>> },
+      { Number,    :is_number,    quote do: number },
+      { Function,  :is_function,  quote do: (... -> any) },
+      { PID,       :is_pid,       quote do: pid },
+      { Port,      :is_port,      quote do: port },
+      { Reference, :is_reference, quote do: reference },
+      { Any,       :is_any,       quote do: any } ]
   end
 
   # Returns a quoted expression that allows to check
   # if the first item in the tuple is a built-in or not.
-  defp is_builtin?([{h,_}]) do
+  defp is_builtin?([{h,_,_}]) do
     quote do
       first == unquote(h)
     end
   end
 
-  defp is_builtin?([{h,_}|t]) do
+  defp is_builtin?([{h,_,_}|t]) do
     quote do
       first == unquote(h) or unquote(is_builtin?(t))
     end
   end
 
-  # Handle records when we don't have fallbacks.
-  # It simply gets the first element of the tuple.
-  # This case assumes that, whenever a tuple is given
-  # it is meant to be a record, so we don't need extra
-  # checks.
-  defp each_impl_for({ _, :is_record }, nil) do
+  # We don't have a fallback, so we assume what was given
+  # as a record is indeed a record
+  defp each_impl_for({ _, :is_record, _ }, _conversions, nil) do
     quote do
-      defp __raw_impl__(arg) when is_tuple(arg) and is_atom(:erlang.element(1, arg)) do
+      defp __raw_impl__(arg) when is_record(arg) do
         __MODULE__.Record
       end
     end
   end
 
   # Specially handle records in the case we have fallbacks.
-  defp each_impl_for({ _, :is_record }, conversions) do
+  defp each_impl_for({ _, :is_record, _ }, conversions, fallback) do
     quote do
-      defp __raw_impl__(arg) when is_tuple(arg) and is_atom(:erlang.element(1, arg)) do
+      defp __raw_impl__(arg) when is_record(arg) do
         first = :erlang.element(1, arg)
         case unquote(is_builtin?(conversions)) do
-          true  -> __fallback__
+          true  -> unquote(fallback)
           false ->
             case atom_to_list(first) do
               'Elixir-' ++ _ -> __MODULE__.Record
-              _              -> __fallback__
+              _              -> unquote(fallback)
             end
         end
       end
@@ -246,7 +276,7 @@ defmodule Protocol do
   end
 
   # Special case any as we don't need to generate a guard.
-  defp each_impl_for({ _, :is_any }, _) do
+  defp each_impl_for({ _, :is_any, _ }, _, _) do
     quote do
       defp __raw_impl__(_) do
         __MODULE__.Any
@@ -255,10 +285,10 @@ defmodule Protocol do
   end
 
   # Generate all others protocols.
-  defp each_impl_for({ kind, fun }, _) do
+  defp each_impl_for({ kind, fun, _ }, _, _) do
     quote do
-      defp __raw_impl__(arg) when unquote(fun).(arg) do
-        Module.concat __MODULE__, unquote(kind)
+      defp __raw_impl__(arg) when unquote(fun)(arg) do
+        __MODULE__.unquote(kind)
       end
     end
   end
@@ -267,6 +297,90 @@ end
 defmodule Protocol.DSL do
   @moduledoc false
 
+  # We need to use :lists because Enum is not available yet
+  require :lists, as: L
+
+  @doc false
+  def args_and_body(module, name, arity) do
+    # Generate arguments according the arity. The arguments
+    # are named xa, xb and so forth. We cannot use string
+    # interpolation to generate the arguments because of compile
+    # dependencies, so we use the <<>> instead.
+    args = lc i inlist :lists.seq(1, arity) do
+      { binary_to_atom(<<?x, i + 64>>), [], __MODULE__ }
+    end
+
+    { conversions, fallback, returns_nil } = conversions_for(module)
+    clauses = [default_clause(name, args)]
+
+    if returns_nil do
+      clauses = [nil_clause()|clauses]
+    end
+
+    if L.keyfind(Record, 1, conversions) do
+      clauses = [record_clause(name, args, fallback)|clauses]
+    end
+
+    body =
+      quote do
+        case __raw_impl__(xA), do: unquote({ :->, [], clauses })
+      end
+
+    { args, body }
+  end
+
+  @doc false
+  def callback_from_spec(module, name, arity) do
+    tuple = { name, arity }
+    specs = Module.get_attribute(module, :spec)
+
+    found = lc { k, v } inlist specs, k == tuple do
+      Kernel.Typespec.define_callback(module, tuple, v)
+      true
+    end
+
+    found != []
+  end
+
+  defp conversions_for(module) do
+    only   = Module.get_attribute(module, :only)
+    except = Module.get_attribute(module, :except)
+    Protocol.conversions_for(module, only, except)
+  end
+
+  defp default_clause(name, args) do
+    quote do: { [other], apply(other, unquote(name), [unquote_splicing(args)]) }
+  end
+
+  defp nil_clause() do
+    quote do
+      { [nil], raise(Protocol.UndefinedError, protocol: __MODULE__, structure: xA) }
+    end
+  end
+
+  defp record_clause(name, args, nil) do
+    quote do
+      { [__MODULE__.Record],
+        Module.concat(__MODULE__, :erlang.element(1, xA)).unquote(name)(unquote_splicing(args)) }
+    end
+  end
+
+  defp record_clause(name, args, fallback) do
+    arity = length(args)
+
+    quote do
+      { [__MODULE__.Record],
+        (target = Module.concat(__MODULE__, :erlang.element(1, xA))
+        try do
+          target.unquote(name)(unquote_splicing(args))
+        catch
+          :error, :undef, [[{ ^target, name, args, _ }|_]|_] when
+              name == unquote(name) and length(args) == unquote(arity) ->
+            apply unquote(fallback), name, [unquote_splicing(args)]
+        end) }
+    end
+  end
+
   defmacro def(expression) do
     case expression do
       { _, _, args } when args == [] or is_atom(args) ->
@@ -279,43 +393,26 @@ defmodule Protocol.DSL do
 
     arity = length(args)
 
-    # Generate arguments according the arity. The arguments
-    # are named xa, xb and so forth. We cannot use string
-    # interpolation to generate the arguments because of compile
-    # dependencies, so we use the <<>> instead.
-    generated = lc i inlist :lists.seq(1, arity) do
-      { binary_to_atom(<<?x, i + 64>>), 0, :quoted }
-    end
+    type_args = lc _ inlist :lists.seq(2, arity), do: quote(do: term)
+    type_args = [quote(do: t) | type_args]
 
     quote do
-      # Append new function to the list
-      @functions [unquote({name, arity})|@functions]
+      name  = unquote(name)
+      arity = unquote(arity)
+
+      @functions [{name, arity}|@functions]
 
       # Generate a fake definition with the user
       # signature that will be used by docs
-      Kernel.def unquote(name).(unquote_splicing(args))
+      Kernel.def unquote(name)(unquote_splicing(args))
 
-      Kernel.def unquote(name).(unquote_splicing(generated)) do
-        case __raw_impl__(xA) do
-          __MODULE__.Record ->
-            target = Module.concat(__MODULE__, :erlang.element(1, xA))
-            try do
-              target.unquote(name)(unquote_splicing(generated))
-            catch
-              :error, :undef, [[{ ^target, unquote(name), args, _ }|_]|_] when length(args) == unquote(arity) ->
-                case __fallback__ do
-                  nil ->
-                    raise Protocol.UndefinedError, protocol: __MODULE__, structure: xA
-                  other ->
-                    apply other, unquote(name), [unquote_splicing(generated)]
-                end
-            end
-          nil ->
-            raise Protocol.UndefinedError, protocol: __MODULE__, structure: xA
-          other ->
-            apply other, unquote(name), [unquote_splicing(generated)]
-        end
-      end
+      { args, body } = Protocol.DSL.args_and_body(__MODULE__, name, arity)
+      Kernel.def unquote(name), args, [], do: body
+
+      # Convert the spec to callback if possible,
+      # otherwise generate a dummy callback
+      Protocol.DSL.callback_from_spec(__MODULE__, name, arity) ||
+        @callback unquote(name)(unquote_splicing(type_args)) :: term
     end
   end
 end

lib/elixir/lib/range.ex

@@ -32,19 +32,31 @@ defimpl Enum.Iterator, for: Range do
 end
 
 defimpl Range.Iterator, for: Number do
-  def iterator(first, Range[first: f, last: last]) when is_integer(first) and is_integer(last) and last < f do
-    fn(current) ->
-      if current < last, do: :stop, else: { current, current - 1 }
-    end
-  end
-
-  def iterator(first, Range[last: last]) when is_integer(first) and is_integer(last) do
+  def iterator(first, Range[last: last]) when is_integer(first) and is_integer(last) and last >= first do
     fn(current) ->
       if current > last, do: :stop, else: { current, current + 1 }
     end
   end
 
-  def count(first, Range[last: last]) when is_integer(first) and is_integer(last) do
+  def iterator(first, Range[last: last]) when is_integer(first) and is_integer(last) do
+    fn(current) ->
+      if current < last, do: :stop, else: { current, current - 1 }
+    end
+  end
+
+  def count(first, Range[last: last]) when is_integer(first) and is_integer(last) and last >= first do
     last - first + 1
   end
-end
+
+  def count(first, Range[last: last]) when is_integer(first) and is_integer(last) do
+    first - last + 1
+  end
+end
+
+defimpl Binary.Inspect, for: Range do
+  import Kernel, except: [inspect: 2]
+
+  def inspect(Range[first: first, last: last], opts) do
+    Binary.Inspect.inspect(first, opts) <> ".." <> Binary.Inspect.inspect(last, opts)
+  end
+end

lib/elixir/lib/record.ex

@@ -18,7 +18,7 @@ defmodule Record do
 
   @doc """
   Main entry point for records definition. It defines a module
-  with the given `name` and the fields specified in `modules`.
+  with the given `name` and the fields specified in `values`.
   This is invoked directly by `Kernel.defrecord`, so check it
   for more information and documentation.
   """
@@ -29,125 +29,193 @@ defmodule Record do
     quote do
       defmodule unquote(name) do
         @moduledoc false
-        Record.deffunctions(unquote(values), unquote(opts), __ENV__)
-        unquote(block)
+        import Record.DSL
+
+        @record_fields []
+        @record_types  []
+
+        values = unquote(values)
+        opts   = unquote(opts)
+        Record.deffunctions(values, __ENV__)
+        value = unquote(block)
+        Record.deftypes(values, @record_types, __ENV__)
+        value
       end
     end
   end
 
+  @doc """
+  Import public record definition as a set of private macros (as defined by defrecordp/2)
+
+  ## Usage
+
+     Record.import Record.Module, as: macro_name
+
+  ## Example
+
+     defmodule Test do
+       Record.import File.Stat, as: :file_stat
+
+       def size(file_stat(size: size)), do: size
+     end
+
+  """
+  defmacro import(module, as: name) do
+    quote do
+      Record.defmacros(unquote(name), unquote(module).__record__(:fields), __ENV__, unquote(module))
+    end
+  end
+
+  @doc """
+  Main entry point for private records definition. It defines
+  a set of macros with the given `name` and the fields specified
+  in `values`. This is invoked directly by `Kernel.defrecordp`,
+  so check it for more information and documentation.
+  """
+  def defrecordp(name, fields) do
+    quote do
+      Record.defmacros(unquote(name), unquote(fields), __ENV__)
+    end
+  end
+
   @doc """
   Defines record functions skipping the module definition.
-  This is called directly by `defrecord`. It expects the
-  module environment, the module values and a keyword list
-  of options.
+  This is called directly by `defrecord`. It expects the record
+  values, a set of options and the module environment.
 
   ## Examples
 
       defmodule CustomRecord do
         Record.deffunctions [:name, :age], __ENV__
+        Record.deftypes [:name, :age], [name: :binary, age: :integer], __ENV__
       end
 
   """
-  def deffunctions(values, opts // [], env) do
-    values     = lc value inlist values, do: convert_value(value)
-    escaped    = Macro.escape(values)
-    extensions = Keyword.get(opts, :extensions, Record.Extensions)
+  def deffunctions(values, env) do
+    values  = lc value inlist values, do: convert_value(value)
+    escaped = Macro.escape(values)
 
     contents = [
       reflection(escaped),
       initializer(escaped),
       indexes(escaped),
-      readers(values, 1, []),
       conversions(values),
-      writers(values, 1, []),
+      record_optimizable(),
       updater(values),
-      extensions(values, 1, [], extensions)
+      accessors(values, 1),
+      switch_recorder()
     ]
 
+    contents = [quote(do: @record_fields unquote(escaped))|contents]
+
     # Special case for bootstraping purposes
     if env == Macro.Env do
-      :elixir_module.eval_quoted(env, contents, [], [])
+      Module.eval_quoted(env, contents, [], [])
     else
-      contents = [quote(do: @__record__ unquote(escaped))|contents]
-      Module.eval_quoted(env, contents)
+      Module.eval_quoted(env.module, contents, [], env.location)
     end
   end
 
   @doc """
-  Defines three macros for reading and writing records values.
-  These macros are private to the current module and are
-  basically a simple mechanism for manipulating tuples when
-  there isn't an interest in exposing the record as a whole.
-  In some ways, it is similar to Erlang records, since it is
-  only available at compilation time.
+  Defines types and specs for the record.
+  """
+  def deftypes(values, types, env) do
+    types  = types || []
+    values = lc value inlist values do
+      { name, default } = convert_value(value)
+      { name, default, find_spec(types, name) }
+    end
+
+    contents = [
+      core_specs(values),
+      accessor_specs(values, 1, [])
+    ]
+
+    if env == Macro.Env do
+      Module.eval_quoted(env, contents, [], [])
+    else
+      Module.eval_quoted(env.module, contents, [], env.location)
+    end
+  end
+
+  @doc """
+  Defines macros for manipulating records. This is called
+  directly by `defrecordp`. It expects the macro name, the
+  record values and the environment.
 
   ## Examples
 
-      defmodule CustomModule do
-        Record.defmacros :_user, [:name, :age], __ENV__
-
-        def new(name, age) do
-          _user(name: name, age: age)
-        end
-
-        def name(user, name) do
-          _user(user, name: name)
-        end
-
-        def age(user) do
-          _user(user, :age)
-        end
-
-        def to_keywords(user) do
-          _user(user)
-        end
-
-        def name_and_age(user) do
-         _user(user, [:name, :age])
-        end
-
-        def age_and_name(user) do
-         _user(user, [:age, :name])
-        end
+      defmodule CustomRecord do
+        Record.defmacros :user, [:name, :age], __ENV__
       end
 
   """
-  def defmacros(name, values, env) do
-    escaped = lc value inlist values, do: Macro.escape(convert_value(value))
+  def defmacros(name, values, env, tag // nil) do
+    escaped = lc value inlist values do
+      { key, value } = convert_value(value)
+      { key, Macro.escape(value) }
+    end
 
     contents = quote do
-      defmacrop unquote(name).(record) when is_tuple(record) do
-        Record.to_keywords(__CALLER__, __MODULE__, unquote(escaped), record)
+
+      defmacrop unquote(name)() do
+        Record.access(unquote(tag) || __MODULE__, unquote(escaped), [], __CALLER__)
       end
 
-      defmacrop unquote(name).(args) do
-        Record.access(__CALLER__, __MODULE__, unquote(escaped), args)
+      defmacrop unquote(name)(record) when is_tuple(record) do
+        Record.to_keywords(unquote(tag) || __MODULE__, unquote(escaped), record)
       end
 
-      defmacrop unquote(name).(record, key) when is_atom(key) do
-        Record.get(__CALLER__, __MODULE__, unquote(escaped), record, key)
+      defmacrop unquote(name)(args) do
+        Record.access(unquote(tag) || __MODULE__, unquote(escaped), args, __CALLER__)
       end
 
-      defmacrop unquote(name).(record, args) do
-        Record.dispatch(__CALLER__, __MODULE__, unquote(escaped), record, args)
+      defmacrop unquote(name)(record, key) when is_atom(key) do
+        Record.get(unquote(tag) || __MODULE__, unquote(escaped), record, key)
+      end
+
+      defmacrop unquote(name)(record, args) do
+        Record.dispatch(unquote(tag) || __MODULE__, unquote(escaped), record, args, __CALLER__)
       end
     end
 
-    Module.eval_quoted(env, contents)
+    Module.eval_quoted(env.module, contents, [], env.location)
   end
 
-  defp convert_value(atom) when is_atom(atom) do
-    { atom, nil }
+  ## Callbacks
+
+  # Store all optimizable fields in the record as well
+  @doc false
+  defmacro __before_compile__(_) do
+    quote do
+      def __record__(:optimizable), do: @record_optimizable
+    end
   end
 
-  defp convert_value(other), do: other
+  # Store fields that can be optimized and that cannot be
+  # optimized as they are overriden
+  @doc false
+  def __on_definition__(env, kind, name, args, _guards, _body) do
+    tuple     = { name, length(args) }
+    module    = env.module
+    functions = Module.get_attribute(module, :record_optimizable)
+
+    functions =
+      if kind in [:def] and Module.get_attribute(module, :record_optimized) do
+        [tuple|functions]
+      else
+        List.delete(functions, tuple)
+      end
+
+    Module.put_attribute(module, :record_optimizable, functions)
+  end
 
   # Implements the access macro used by records.
   # It returns a quoted expression that defines
   # a record or a match in case the record is
   # inside a match.
   @doc false
-  def access(caller, atom, fields, keyword) do
+  def access(atom, fields, keyword, caller) do
     unless is_keyword(keyword) do
       raise "expected contents inside brackets to be a Keyword"
     end
@@ -155,7 +223,7 @@ defmodule Record do
     in_match = caller.in_match?
 
     has_underscore_value = Keyword.has_key?(keyword, :_)
-    underscore_value     = Keyword.get(keyword, :_, { :_, 0, nil })
+    underscore_value     = Keyword.get(keyword, :_, { :_, [], nil })
     keyword              = Keyword.delete keyword, :_
 
     iterator = fn({field, default}, each_keyword) ->
@@ -175,7 +243,7 @@ defmodule Record do
     { match, remaining } = :lists.mapfoldl(iterator, keyword, fields)
 
     case remaining do
-      [] -> { :{}, caller.line, [atom|match] }
+      [] -> { :{}, [line: caller.line], [atom|match] }
       _  ->
         keys = lc { key, _ } inlist remaining, do: key
         raise "record #{inspect atom} does not have the keys: #{inspect keys}"
@@ -184,11 +252,11 @@ defmodule Record do
 
   # Dispatch the call to either update or to_list depending on the args given.
   @doc false
-  def dispatch(caller, atom, fields, record, args) do
+  def dispatch(atom, fields, record, args, caller) do
     if is_keyword(args) do
-      update(caller, atom, fields, record, args)
+      update(atom, fields, record, args, caller)
     else
-      to_list(caller, atom, fields, record, args)
+      to_list(atom, fields, record, args)
     end
   end
 
@@ -196,7 +264,7 @@ defmodule Record do
   # It returns a quoted expression that represents
   # the access given by the keywords.
   @doc false
-  defp update(caller, atom, fields, var, keyword) do
+  defp update(atom, fields, var, keyword, caller) do
     unless is_keyword(keyword) do
       raise "expected contents inside brackets to be a Keyword"
     end
@@ -221,7 +289,7 @@ defmodule Record do
   # It returns a quoted expression that represents
   # getting the value of a given field.
   @doc false
-  def get(_caller, atom, fields, var, key) do
+  def get(atom, fields, var, key) do
     index = find_index(fields, key, 0)
     if index do
       quote do
@@ -236,12 +304,12 @@ defmodule Record do
   # It returns a quoted expression that represents
   # converting record to keywords list.
   @doc false
-  def to_keywords(_caller, _atom, fields, record) do
-    Enum.map Keyword.from_enum(fields),
-      fn({key, _default}) ->
+  def to_keywords(_atom, fields, record) do
+    Enum.map fields,
+      fn { key, _default } ->
         index = find_index(fields, key, 0)
         quote do
-          {unquote(key), :erlang.element(unquote(index + 2), unquote(record))}
+          { unquote(key), :erlang.element(unquote(index + 2), unquote(record)) }
         end
       end
   end
@@ -250,7 +318,7 @@ defmodule Record do
   # It returns a quoted expression that represents
   # extracting given fields from record.
   @doc false
-  defp to_list(_caller, atom, fields, record, keys) do
+  defp to_list(atom, fields, record, keys) do
     Enum.map keys,
       fn(key) ->
         index = find_index(fields, key, 0)
@@ -262,11 +330,7 @@ defmodule Record do
       end
   end
 
-  defp is_keyword(list) when is_list(list), do: :lists.all(is_keyword_tuple(&1), list)
-  defp is_keyword(_), do: false
-
-  defp is_keyword_tuple({ x, _ }) when is_atom(x), do: true
-  defp is_keyword_tuple(_), do: false
+  ## Function generation
 
   # Define __record__/1 and __record__/2 as reflection functions
   # that returns the record names and fields.
@@ -284,7 +348,10 @@ defmodule Record do
   #
   defp reflection(values) do
     quote do
+      @doc false
       def __record__(kind, _),      do: __record__(kind)
+
+      @doc false
       def __record__(:name),        do: __MODULE__
       def __record__(:fields),      do: unquote(values)
     end
@@ -320,7 +387,10 @@ defmodule Record do
     end
 
     quote do
+      @doc false
       def new(), do: new([])
+
+      @doc false
       def new([]), do: { __MODULE__, unquote_splicing(defaults) }
       def new(opts) when is_list(opts), do: { __MODULE__, unquote_splicing(selective) }
       def new(tuple) when is_tuple(tuple), do: :erlang.setelement(1, tuple, __MODULE__)
@@ -352,7 +422,11 @@ defmodule Record do
     end
     quote do
       unquote(quoted)
+
+      @doc false
       def __index__(_), do: nil
+
+      @doc false
       def __index__(key, _), do: __index__(key)
     end
   end
@@ -372,17 +446,14 @@ defmodule Record do
     end
 
     quote do
+      @doc false
       def to_keywords(record) do
         unquote(:orddict.from_list(sorted))
       end
     end
   end
 
-  defp find_index([{ k, _ }|_], k, i), do: i
-  defp find_index([{ _, _ }|t], k, i), do: find_index(t, k, i + 1)
-  defp find_index([], _k, _i), do: nil
-
-  # Implement readers. For a declaration like:
+  # Implement accessors. For a declaration like:
   #
   #     defrecord FileInfo, atime: nil, mtime: nil
   #
@@ -396,220 +467,185 @@ defmodule Record do
   #       elem(record, 2)
   #     end
   #
-  defp readers([{ key, _default }|t], i, acc) do
-    contents = quote do
-      def unquote(key).(record) do
-        :erlang.element(unquote(i + 1), record)
-      end
-    end
-
-    readers(t, i + 1, [contents | acc])
-  end
-
-  defp readers([], _i, acc), do: acc
-
-  # Implement writers. For a declaration like:
-  #
-  #     defrecord FileInfo, atime: nil, mtime: nil
-  #
-  # It will define four methods:
-  #
   #     def :atime.(value, record) do
-  #       setelem(record, 1, value)
+  #       set_elem(record, 1, value)
   #     end
   #
   #     def :mtime.(record) do
-  #       setelem(record, 2, value)
+  #       set_elem(record, 2, value)
   #     end
   #
-  defp writers([{ key, _default }|t], i, acc) do
+  #     def :atime.(callback, record) do
+  #       set_elem(record, 1, callback.(elem(record, 1)))
+  #     end
+  #
+  #     def :mtime.(callback, record) do
+  #       set_elem(record, 2, callback.(elem(record, 2)))
+  #     end
+  #
+  defp accessors([{ :__exception__, _ }|t], 1) do
+    accessors(t, 2)
+  end
+
+  defp accessors([{ key, _default }|t], i) do
+    update = binary_to_atom "update_" <> atom_to_binary(key)
+
     contents = quote do
-      def unquote(key).(value, record) do
+      @doc false
+      def unquote(key)(record) do
+        :erlang.element(unquote(i + 1), record)
+      end
+
+      @doc false
+      def unquote(key)(value, record) do
         :erlang.setelement(unquote(i + 1), record, value)
       end
+
+      @doc false
+      def unquote(update)(function, record) do
+        :erlang.setelement(unquote(i + 1), record,
+          function.(:erlang.element(unquote(i + 1), record)))
+      end
     end
 
-    writers(t, i + 1, [contents | acc])
+    [contents|accessors(t, i + 1)]
   end
 
-  defp writers([], _i, acc), do: acc
+  defp accessors([], _i) do
+    []
+  end
 
-  # Defines update/2
+  # Define an updater method that receives a
+  # keyword list and updates the record.
   defp updater(values) do
     fields =
       lc {key, _default} inlist values do
+        index = find_index(values, key, 1)
         quote do
-          Keyword.get(keywords,
-                      unquote(key),
-                      elem(record, unquote(find_index(values, key, 1))))
+          Keyword.get(keywords, unquote(key), elem(record, unquote(index)))
         end
       end
-    contents = {:{}, 0, [(quote do: __MODULE__)|fields]}
+
+    contents = { :{}, [], [(quote do: __MODULE__)|fields] }
+
     quote do
+      @doc false
+      def update([], record) do
+        record
+      end
+
       def update(keywords, record) do
         unquote(contents)
       end
     end
   end
 
-  # Defines extra functions from the definition.
-  defp extensions([{ key, default }|t], i, acc, extensions) do
-    functions = extensions.functions_for(key, default, i)
-    extensions(t, i + 1, [functions | acc], extensions)
+  defp record_optimizable do
+    quote do
+      @record_optimized true
+      @record_optimizable []
+      @before_compile { unquote(__MODULE__), :__before_compile__ }
+      @on_definition { unquote(__MODULE__), :__on_definition__ }
+    end
   end
 
-  defp extensions([], _i, acc, _), do: acc
+  defp switch_recorder do
+    quote do: @record_optimized false
+  end
+
+  ## Types/specs generation
+
+  defp core_specs(values) do
+    types   = lc { _, _, spec } inlist values, do: spec
+    options = if values == [], do: [], else: [options_specs(values)]
+
+    quote do
+      unless Kernel.Typespec.defines_type?(__MODULE__, :t, 0) do
+        @type t :: { __MODULE__, unquote_splicing(types) }
+      end
+
+      unless Kernel.Typespec.defines_type?(__MODULE__, :options, 0) do
+        @type options :: unquote(options)
+      end
+
+      @spec new :: t
+      @spec new(options | tuple) :: t
+      @spec to_keywords(t) :: options
+      @spec update(options, t) :: t
+      @spec __index__(atom) :: non_neg_integer | nil
+    end
+  end
+
+  defp options_specs([{ k, _, v }|t]) do
+    :lists.foldl fn { k, _, v }, acc ->
+      { :|, [], [{ k, v }, acc] }
+    end, { k, v }, t
+  end
+
+  defp accessor_specs([{ :__exception__, _, _ }|t], 1, acc) do
+    accessor_specs(t, 2, acc)
+  end
+
+  defp accessor_specs([{ key, _default, spec }|t], i, acc) do
+    update = binary_to_atom "update_" <> atom_to_binary(key)
+
+    contents = quote do
+      @spec unquote(key)(t) :: unquote(spec)
+      @spec unquote(key)(unquote(spec), t) :: t
+      @spec unquote(update)((unquote(spec) -> unquote(spec)), t) :: t
+    end
+
+    accessor_specs(t, i + 1, [contents | acc])
+  end
+
+  defp accessor_specs([], _i, acc), do: acc
+
+  ## Helpers
+
+  defp is_keyword(list) when is_list(list), do: :lists.all(is_keyword_tuple(&1), list)
+  defp is_keyword(_), do: false
+
+  defp is_keyword_tuple({ x, _ }) when is_atom(x), do: true
+  defp is_keyword_tuple(_), do: false
+
+  defp convert_value(atom) when is_atom(atom), do: { atom, nil }
+
+  defp convert_value({ atom, other }) when is_atom(atom) and is_function(other), do:
+    raise ArgumentError, message: "record field default value #{inspect atom} cannot be a function"
+
+  defp convert_value({ atom, other }) when is_atom(atom) and (is_reference(other) or is_pid(other) or is_port(other)), do:
+    raise ArgumentError, message: "record field default value #{inspect atom} cannot be a reference, pid or port"
+
+  defp convert_value({ atom, _ } = tuple) when is_atom(atom), do: tuple
+
+  defp convert_value({ field, _ }), do:
+    raise ArgumentError, message: "record field name has to be an atom, got #{inspect field}"
+
+  defp find_index([{ k, _ }|_], k, i), do: i
+  defp find_index([{ _, _ }|t], k, i), do: find_index(t, k, i + 1)
+  defp find_index([], _k, _i), do: nil
+
+  defp find_spec(types, name) do
+    matches = lc { k, v } inlist types, name == k, do: v
+    case matches do
+      [h|_] -> h
+      _     -> quote do: term
+    end
+  end
 end
 
-defmodule Record.Extractor do
+defmodule Record.DSL do
   @moduledoc false
 
-  # Retrieve a record definition from an Erlang file using
-  # the same lookup as the *include* attribute from Erlang modules.
-  def retrieve(name, from: string) do
-    file = to_char_list(string)
+  @doc """
+  Defines the type for each field in the record.
+  Expects a keyword list.
+  """
+  defmacro record_type(opts) when is_list(opts) do
+    escaped = lc { k, v } inlist opts, do: { k, Macro.escape(v) }
 
-    case :code.where_is_file(file) do
-      :non_existing -> realfile = file
-      realfile -> nil
+    quote do
+      @record_types Keyword.merge(@record_types || [], unquote(escaped))
     end
-
-    retrieve_record(name, realfile)
-  end
-
-  # Retrieve a record definition from an Erlang file using
-  # the same lookup as the *include_lib* attribute from Erlang modules.
-  def retrieve(name, from_lib: file) do
-    [app|path] = :filename.split(to_char_list(file))
-    case :code.lib_dir(to_char_list(app)) do
-      { :error, _ } ->
-        raise ArgumentError, "Lib file #{to_binary(file)} could not be found"
-      libpath ->
-        retrieve_record name, :filename.join([libpath|path])
-    end
-  end
-
-  # Retrieve the record with the given name from the given file
-  defp retrieve_record(name, file) do
-    records = retrieve_from_file(file)
-    if record = List.keyfind(records, name, 0) do
-      parse_record(record)
-    else
-      raise ArgumentError, "No record #{name} found at #{to_binary(file)}"
-    end
-  end
-
-  # Parse the given file and retrieve all existent records.
-  defp retrieve_from_file(file) do
-    lc { :attribute, _, :record, record } inlist read_file(file), do: record
-  end
-
-  # Read a file and return its abstract syntax form that also
-  # includes record and other preprocessor modules. This is done
-  # by using Erlang's epp_dodger.
-  defp read_file(file) do
-    case :epp_dodger.quick_parse_file(file) do
-      { :ok, form } ->
-        form
-      other ->
-        raise "Error parsing file #{to_binary(file)}, got: #{inspect(other)}"
-    end
-  end
-
-  # Parse a tuple with name and fields and returns a
-  # list of second order tuples where the first element
-  # is the field and the second is its default value.
-  defp parse_record({ _name, fields }) do
-    cons = List.foldr fields, { nil, 0 }, fn f, acc ->
-      { :cons, 0, parse_field(f), acc }
-    end
-    { :value, list, _ } = :erl_eval.expr(cons, [])
-    list
-  end
-
-  defp parse_field({ :typed_record_field, record_field, _type }) do
-    parse_field(record_field)
-  end
-
-  defp parse_field({ :record_field, _, key }) do
-    { :tuple, 0, [key, {:atom, 0, :nil}] }
-  end
-
-  defp parse_field({ :record_field, _, key, value }) do
-    { :tuple, 0, [key, value] }
   end
 end
-
-defmodule Record.Extensions do
-  @moduledoc false
-
-  # Main entry point. It defines both default functions
-  # via `default_for` and extensions via `extension_for`.
-  def functions_for(key, default, i) do
-    [ default_for(key, default, i),
-      extension_for(key, default, i) ]
-  end
-
-  # Skip the __exception__ for defexception.
-  def default_for(:__exception__, _default, _i) do
-    nil
-  end
-
-  # Define the default functions for each field.
-  def default_for(key, _default, i) do
-    bin_update = "update_" <> atom_to_binary(key)
-    update     = binary_to_atom(bin_update)
-
-    quote do
-      def unquote(update).(function, record) do
-        current = :erlang.element(unquote(i + 1), record)
-        :erlang.setelement(unquote(i + 1), record, function.(current))
-      end
-    end
-  end
-
-  # Define extensions based on the default type.
-  def extension_for(key, default, i) when is_list(default) do
-    bin_key = atom_to_binary(key)
-    prepend = :"prepend_#{bin_key}"
-    merge   = :"merge_#{bin_key}"
-
-    quote do
-      def unquote(prepend).(value, record) do
-        current = :erlang.element(unquote(i + 1), record)
-        :erlang.setelement(unquote(i + 1), record, value ++ current)
-      end
-
-      def unquote(merge).(value, record) do
-        current = :erlang.element(unquote(i + 1), record)
-        :erlang.setelement(unquote(i + 1), record, Keyword.merge(current, value))
-      end
-    end
-  end
-
-  def extension_for(key, default, i) when is_number(default) do
-    bin_key   = atom_to_binary(key)
-    increment = :"increment_#{bin_key}"
-
-    quote do
-      def unquote(increment).(value // 1, record) do
-        current = :erlang.element(unquote(i + 1), record)
-        :erlang.setelement(unquote(i + 1), record, current + value)
-      end
-    end
-  end
-
-  def extension_for(key, default, i) when is_boolean(default) do
-    bin_key = atom_to_binary(key)
-    toggle = :"toggle_#{bin_key}"
-
-    quote do
-      def unquote(toggle).(record) do
-        current = :erlang.element(unquote(i + 1), record)
-        :erlang.setelement(unquote(i + 1), record, not current)
-       end
-    end
-  end
-
-  def extension_for(_, _, _), do: nil
-end

lib/elixir/lib/record/extractor.ex

@@ -0,0 +1,79 @@
+defmodule Record.Extractor do
+  @moduledoc false
+
+  # Retrieve a record definition from an Erlang file using
+  # the same lookup as the *include* attribute from Erlang modules.
+  def retrieve(name, from: string) do
+    file = to_char_list(string)
+
+    case :code.where_is_file(file) do
+      :non_existing -> realfile = file
+      realfile -> nil
+    end
+
+    retrieve_record(name, realfile)
+  end
+
+  # Retrieve a record definition from an Erlang file using
+  # the same lookup as the *include_lib* attribute from Erlang modules.
+  def retrieve(name, from_lib: file) do
+    [app|path] = :filename.split(to_char_list(file))
+
+    case :code.lib_dir(to_char_list(app)) do
+      { :error, _ } ->
+        raise ArgumentError, "lib file #{to_binary(file)} could not be found"
+      libpath ->
+        retrieve_record name, :filename.join([libpath|path])
+    end
+  end
+
+  # Retrieve the record with the given name from the given file
+  defp retrieve_record(name, file) do
+    records = retrieve_from_file(file)
+    if record = List.keyfind(records, name, 0) do
+      parse_record(record)
+    else
+      raise ArgumentError, "no record #{name} found at #{to_binary(file)}"
+    end
+  end
+
+  # Parse the given file and retrieve all existent records.
+  defp retrieve_from_file(file) do
+    lc { :attribute, _, :record, record } inlist read_file(file), do: record
+  end
+
+  # Read a file and return its abstract syntax form that also
+  # includes record and other preprocessor modules. This is done
+  # by using Erlang's epp_dodger.
+  defp read_file(file) do
+    case :epp_dodger.quick_parse_file(file) do
+      { :ok, form } ->
+        form
+      other ->
+        raise "error parsing file #{to_binary(file)}, got: #{inspect(other)}"
+    end
+  end
+
+  # Parse a tuple with name and fields and returns a
+  # list of tuples where the first element is the field
+  # and the second is its default value.
+  defp parse_record({ _name, fields }) do
+    cons = List.foldr fields, { nil, 0 }, fn f, acc ->
+      { :cons, 0, parse_field(f), acc }
+    end
+    { :value, list, _ } = :erl_eval.expr(cons, [])
+    list
+  end
+
+  defp parse_field({ :typed_record_field, record_field, _type }) do
+    parse_field(record_field)
+  end
+
+  defp parse_field({ :record_field, _, key }) do
+    { :tuple, 0, [key, {:atom, 0, :undefined}] }
+  end
+
+  defp parse_field({ :record_field, _, key, value }) do
+    { :tuple, 0, [key, value] }
+  end
+end

lib/elixir/lib/regex.ex

@@ -16,7 +16,7 @@ defmodule Regex do
   The re module provides several options, the one available in Elixir, followed by
   their shortcut in parenthesis, are:
 
-  * unicode (u) - used when you want to match against specific unicode characters
+  * unicode (u) - enable unicode specific patterns like \p
   * caseless (i) - add case insensitivity
   * dotall (s) - causes dot to match newlines and also set newline to anycrlf.
     The new line setting can be overwritten by setting `(*CR)` or `(*LF)` or
@@ -43,6 +43,9 @@ defmodule Regex do
   a binary, a char list will return a char list).
   """
 
+  defrecordp :regex, [:re_pattern, :source, :options, :groups]
+  @type t :: { Regex, term, term, term, term }
+
   defexception CompileError, message: "regex could not be compiled"
 
   @doc """
@@ -51,15 +54,15 @@ defmodule Regex do
   It returns `{ :ok, regex }` in case of success,
   `{ :error, reason }` otherwise.
   """
-  def compile(source, options // "") do
-    source  = to_binary(source)
+  def compile(source, options // "") when is_binary(source) do
     options = to_binary(options)
     opts    = translate_options(options)
     re_opts = opts -- [:groups]
     groups  = if opts != re_opts, do: parse_groups(source)
+
     case :re.compile(source, re_opts) do
-      { :ok, compiled } ->
-        { :ok, { Regex, compiled, source, options, groups } }
+      { :ok, re_pattern } ->
+        { :ok, regex(re_pattern: re_pattern, source: source, options: options, groups: groups) }
       error ->
         error
     end
@@ -83,11 +86,13 @@ defmodule Regex do
 
   ## Examples
 
-      Regex.index %r/c(d)/, "abcd"  #=> 3
-      Regex.index %r/e/, "abcd"     #=> nil
+      iex> Regex.index %r/c(d)/, "abcd"
+      2
+      iex> Regex.index %r/e/, "abcd"
+      nil
 
   """
-  def index({ Regex, compiled, _, _, _ }, string) do
+  def index(regex(re_pattern: compiled), string) do
     case :re.run(string, compiled, [{ :capture, :first, :index }]) do
       :nomatch -> nil
       { :match, [{index,_}] } -> index
@@ -99,11 +104,13 @@ defmodule Regex do
 
   ## Examples
 
-      Regex.match? %r/foo/, "foo" #=> true
-      Regex.match? %r/foo/, "bar" #=> false
+      iex> Regex.match? %r/foo/, "foo"
+      true
+      iex> Regex.match? %r/foo/, "bar"
+      false
 
   """
-  def match?({ Regex, compiled, _, _, _ }, string) do
+  def match?(regex(re_pattern: compiled), string) do
     :re.run(string, compiled, [{ :capture, :none }]) == :match
   end
 
@@ -113,17 +120,20 @@ defmodule Regex do
 
   ## Examples
 
-      Regex.run %r/c(d)/, "abcd"  #=> ["cd", "d"]
-      Regex.run %r/e/, "abcd"     #=> nil
+      iex> Regex.run %r/c(d)/, "abcd"
+      ["cd", "d"]
+      iex> Regex.run %r/e/, "abcd"
+      nil
 
   """
   def run(regex, string, options // [])
-  def run({ Regex, compiled, _, _, groups }, string, options) do
+
+  def run(regex(re_pattern: compiled, groups: groups), string, options) do
     return = Keyword.get(options, :return, return_for(string))
 
     captures =
       case Keyword.get(options, :capture, :all) do
-        :groups -> groups || raise "Regex was not compiled with g"
+        :groups -> groups || raise ArgumentError, message: "regex was not compiled with g"
         others  -> others
       end
 
@@ -135,18 +145,20 @@ defmodule Regex do
 
   @doc """
   Returns the given captures as a list of tuples.
+  Requires the regex to be compiled with the groups option.
 
   ## Examples
 
-      Regex.captures %r/c(?<foo>d)/g, "abcd"  #=> [{:foo, ["d"]}]
+      iex> Regex.captures %r/c(?<foo>d)/g, "abcd"
+      [foo: "d"]
 
   """
-  def captures({ Regex, _, _, _, groups } = regex, string, options // []) do
+  def captures(regex(groups: groups) = regex, string, options // []) do
     unless captures = Keyword.get(options, :capture) do
       captures = if groups do
-        List.sort(groups)
+        Enum.sort(groups)
       else
-        raise "Regex was not compiled with g"
+        raise ArgumentError, message: "regex was not compiled with g"
       end
       options  = Keyword.put(options, :capture, captures)
     end
@@ -155,29 +167,9 @@ defmodule Regex do
   end
 
   @doc """
-  Returns a list with the match indexes in the given string.
-  The matches are tuples where the first element is the index
-  (zero indexed) the match happened and the second is the length
-  of the match.
-
-  ## Examples
-
-      Regex.indexes %r/c(d)/, "abcd"  #=> [{2,2},{3,1}]
-      Regex.indexes %r/e/, "abcd"     #=> nil
-
+  Returns the underlying `re_pattern` in the regular expression.
   """
-  def indexes({ Regex, compiled, _, _, _ }, string) do
-    IO.write "[WARNING] Regex.indexes is deprecated, please use Regex.run with return: :index as option instead\n#{Exception.formatted_stacktrace}"
-    case :re.run(string, compiled, [{ :capture, :all, :index }]) do
-      :nomatch -> nil
-      { :match, results } -> results
-    end
-  end
-
-  @doc """
-  Returns the underlying re_pattern in the regular expression.
-  """
-  def re_pattern({ Regex, compiled, _, _, _ }) do
+  def re_pattern(regex(re_pattern: compiled)) do
     compiled
   end
 
@@ -186,23 +178,25 @@ defmodule Regex do
 
   ## Examples
 
-      Regex.source %r(foo) #=> "foo"
+      iex> Regex.source %r(foo)
+      "foo"
 
   """
-  def source({ Regex, _, source, _, _ }) do
+  def source(regex(source: source)) do
     source
   end
 
   @doc """
-  Returns the regex options as a list.
+  Returns the regex options as a string.
 
   ## Examples
 
-      Regex.opts %r(foo)m #=> 'm'
+      iex> Regex.opts %r(foo)m
+      "m"
 
   """
-  def opts({ Regex, _, _, opts, _ }) do
-    opts
+  def opts(regex(options: options)) do
+    options
   end
 
   @doc """
@@ -210,10 +204,11 @@ defmodule Regex do
 
   ## Examples
 
-      Regex.groups %r/(?<foo>foo)/g #=> ["foo"]
+      iex> Regex.groups %r/(?<foo>foo)/g
+      [:foo]
 
   """
-  def groups({ Regex, _, _, _, groups }) do
+  def groups(regex(groups: groups)) do
     groups
   end
 
@@ -225,14 +220,18 @@ defmodule Regex do
 
   ## Examples
 
-      Regex.scan %r/c(d|e)/, "abcd abce"   #=> [["d"], ["e"]]
-      Regex.scan %r/c(?:d|e)/, "abcd abce" #=> ["cd", "ce"]
-      Regex.scan %r/e/, "abcd"             #=> []
+      iex> Regex.scan %r/c(d|e)/, "abcd abce"
+      [["d"], ["e"]]
+      iex> Regex.scan %r/c(?:d|e)/, "abcd abce"
+      ["cd", "ce"]
+      iex> Regex.scan %r/e/, "abcd"
+      []
 
   """
   def scan(regex, string, options // [])
-  def scan({ Regex, compiled, _, _, _ }, string, options) do
-    return = options[:return] || return_for(string)
+
+  def scan(regex(re_pattern: compiled), string, options) do
+    return  = Keyword.get(options, :return, return_for(string))
     options = [{ :capture, :all, return }, :global]
     case :re.run(string, compiled, options) do
       :nomatch -> []
@@ -247,21 +246,16 @@ defmodule Regex do
 
   def split(regex, string, options // [])
 
-  def split(regex, string, options) when is_integer(options) or is_atom(options) do
-    IO.write "[WARNING] Passing an integer or atom to Regex.split/3 is deprecated, pass a :parts option instead\n#{Exception.formatted_stacktrace}"
-    split(regex, string, parts: options)
-  end
-
-  def split({ Regex, compiled, _, _, _ }, string, options) do
+  def split(regex(re_pattern: compiled), string, options) do
     parts =
       cond do
-        options[:global] == false -> 2
-        p = options[:parts]       -> p
-        true                      -> :infinity
+        Keyword.get(options, :global) == false -> 2
+        p = Keyword.get(options, :parts)       -> p
+        true                                   -> :infinity
       end
 
-    return = options[:return] || return_for(string)
-    opts   = [{ :return, return }, { :parts, parts }]
+    return = Keyword.get(options, :return, return_for(string))
+    opts   = [return: return, parts: parts]
     :re.split(string, compiled, opts)
   end
 
@@ -276,26 +270,25 @@ defmodule Regex do
 
   ## Examples
 
-      Regex.replace(%r/d/, "abc", "d")       #=> "abc"
-      Regex.replace(%r/b/, "abc", "d")       #=> "adc"
-      Regex.replace(%r/b/, "abc", "[&]")     #=> "a[b]c"
-      Regex.replace(%r/b/, "abc", "[\\&]")   #=> "a[&]c"
-      Regex.replace(%r/(b)/, "abc", "[\\1]") #=> "a[b]c"
+      iex> Regex.replace(%r/d/, "abc", "d")
+      "abc"
+      iex> Regex.replace(%r/b/, "abc", "d")
+      "adc"
+      iex> Regex.replace(%r/b/, "abc", "[&]")
+      "a[b]c"
+      iex> Regex.replace(%r/b/, "abc", "[\\&]")
+      "a[&]c"
+      iex> Regex.replace(%r/(b)/, "abc", "[\\1]")
+      "a[b]c"
 
   """
-  def replace({ Regex, compiled, _, _, _ }, string, replacement, options // []) do
-    opts   = if options[:global] != false, do: [:global], else: []
-    return = options[:return] || return_for(string)
+  def replace(regex(re_pattern: compiled), string, replacement, options // []) do
+    opts   = if Keyword.get(options, :global) != false, do: [:global], else: []
+    return = Keyword.get(options, :return, return_for(string))
     opts   = [{ :return, return }|opts]
     :re.replace(string, compiled, replacement, opts)
   end
 
-  @doc false
-  def replace_all({ Regex, compiled, _, _, _ }, string, replacement) do
-    IO.write "[WARNING] Regex.replace_all is deprecated, simply use Regex.replace instead\n#{Exception.formatted_stacktrace}"
-    :re.replace(string, compiled, replacement, [{ :return, return_for(string) }, :global])
-  end
-
   # Helpers
 
   @doc false
@@ -305,6 +298,7 @@ defmodule Regex do
   def unescape_map(?r), do: ?\r
   def unescape_map(?t), do: ?\t
   def unescape_map(?v), do: ?\v
+  def unescape_map(?a), do: ?\a
   def unescape_map(_),  do: false
 
   # Private Helpers
@@ -326,7 +320,7 @@ defmodule Regex do
     lc result inlist results do
       case result do
         [t] -> t
-        [h|t] -> t
+        [_|t] -> t
       end
     end
   end

lib/elixir/lib/string.ex

@@ -1,81 +1,128 @@
 defmodule String do
-  @moduledoc """
-  A string in Elixir is a utf-8 binary. This module
-  contains function to work with utf-8 data and its
-  codepoints.
+  @moduledoc %B"""
+  A String in Elixir is a UTF-8 encoded binary.
 
-  For working with raw binaries, use Erlang's :binary
+  ## String and binary operations
+
+  The functions in this module act according to the
+  Unicode Standard, version 6.2.0. For example,
+  `titlecase`, `downcase`, `strip` are provided by this
   module.
+
+  Besides this module, Elixir provides more low-level
+  operations that works directly with binaries. Some
+  of those can be found in the `Kernel` module, as:
+
+  * `binary_part/2` and `binary_part/3` - retrieves part of the binary
+  * `bit_size/1` and `byte_size/1` - size related functions
+  * `is_bitstring/1` and `is_binary/1` - type checking function
+  * Plus a bunch of conversion functions, like `binary_to_atom/2`,
+    `binary_to_integer/2`, `binary_to_term/1` and their opposite
+    like `integer_to_binary/2`
+
+  Finally, [the `:binary` module](http://erlang.org/doc/man/binary.html)
+  provides a couple other functions that works on the byte level.
+
+  ## Codepoints and graphemes
+
+  As per the Unicode Standard, a codepoint is an Unicode
+  Character, which may be represented by one or more bytes.
+  For example, the character "é" is represented with two
+  bytes:
+
+      iex> string = "é"
+      ...> byte_size(string)
+      2
+
+  Furthermore, this module also presents the concept of
+  graphemes, which are multiple characters that may be
+  "perceived as a single character" by readers. For example,
+  the same "é" character written above could be represented
+  by the letter "e" followed by the accent ́:
+
+      iex> string = "\x{0065}\x{0301}"
+      ...> byte_size(string)
+      3
+
+  Although the example above is made of two characters, it is
+  perceived by users as one.
+
+  Graphemes can also be two characters that are interpreted
+  as one by some languages. For example, some languages may
+  consider "ch" as a grapheme. However, since this information
+  depends on the locale, it is not taken into account by this
+  module.
+
+  In general, the functions in this module rely on the Unicode
+  Standard, but does not contain any of the locale specific
+  behaviour.
+
+  ## Integer codepoints
+
+  Although codepoints could be represented as integers, this
+  module represents all codepoints as strings. For example:
+
+      iex> String.codepoints "josé"
+      ["j", "o", "s", "é"]
+
+  There are a couple of ways to retrieve a character integer
+  codepoint. One may use the `?` special macro:
+
+      iex> ?j
+      106
+      iex> ?é
+      233
+
+  Or also via pattern matching:
+
+      iex> << eacute :: utf8 >> = "é"
+      ...> eacute
+      233
+
+  As we have seen above, codepoints can be inserted into
+  a string by their hexadecimal code:
+
+      "jos\x{0065}\x{0301}" #=>
+      "josé"
+
+  ## Self-synchronization
+
+  The UTF-8 encoding is self-synchronizing. This means that
+  if malformed data (i.e., data that is not possible according
+  to the definition of the encoding) is encountered, only one
+  codepoint needs to be rejected.
+
+  This module relies on this behaviour to ignore such invalid
+  characters. For example, `String.length` is going to return
+  a correct result even if an invalid codepoint is fed into it.
+
+  In other words, this module expects invalid data to be detected
+  when retrieving data from the external source. For example, a
+  driver that reads strings from a database will be the one
+  responsible to check the validity of the encoding.
   """
 
+  @type t :: binary
+  @type codepoint :: t
+  @type grapheme :: t
+
   @doc """
   Checks if a string is printable considering it is encoded
   as UTF-8. Returns true if so, false otherwise.
 
   ## Examples
 
-      String.printable?("abc") #=> true
+      iex> String.printable?("abc")
+      true
 
   """
+  @spec printable?(t) :: boolean
 
-  # Allow basic ascii chars
-  def printable?(<<c, t :: binary>>) when c in ?\s..?~ do
-    printable?(t)
-  end
-
-  # From 16#A0 to 16#BF
-  def printable?(<<194, c, t :: binary>>) when c in 160..191 do
-    printable?(t)
-  end
-
-  # From 16#C0 to 16#7FF
-  def printable?(<<m, o1, t :: binary>>) when m in 195..223 and o1 in 128..191 do
-    printable?(t)
-  end
-
-  # From 16#800 to 16#CFFF
-  def printable?(<<m, o1, o2, t :: binary>>) when m in 224..236 and
-      o1 >= 128 and o1 < 192 and o2 >= 128 and o2 < 192 do
-    printable?(t)
-  end
-
-  # From 16#D000 to 16#D7FF
-  def printable?(<<237, o1, o2, t :: binary>>) when
-      o1 >= 128 and o1 < 160 and o2 >= 128 and o2 < 192 do
-    printable?(t)
-  end
-
-  # Reject 16#FFFF and 16#FFFE
-  def printable?(<<239, 191, o>>) when o == 190 or o == 191 do
-    false
-  end
-
-  # From 16#E000 to 16#EFFF
-  def printable?(<<m, o1, o2, t :: binary>>) when (m == 238 or m == 239) and
-      o1 in 128..191 and o2 in 128..191 do
-    printable?(t)
-  end
-
-  # From 16#F000 to 16#FFFD
-  def printable?(<<239, o1, o2, t :: binary>>) when
-      o1 in 128..191 and o2 in 128..191 do
-    printable?(t)
-  end
-
-  # From 16#10000 to 16#3FFFF
-  def printable?(<<240, o1, o2, o3, t :: binary>>) when
-      o1 in 144..191 and o2 in 128..191 and o3 in 128..191 do
-    printable?(t)
-  end
-
-  # Reject 16#110000 onwards
-  def printable?(<<244, o1, _, _, _ :: binary>>) when o1 >= 144 do
-    false
-  end
-
-  # From 16#4000 to 16#10FFFF
-  def printable?(<<m, o1, o2, o3, t :: binary>>) when m in 241..244 and
-      o1 in 128..191 and o2 in 128..191 and o3 in 128..191 do
+  def printable?(<< h :: utf8, t :: binary >>)
+      when h in ?\040..?\176
+      when h in 0xA0..0xD7FF
+      when h in 0xE000..0xFFFD
+      when h in 0x10000..0x10FFFF do
     printable?(t)
   end
 
@@ -86,37 +133,57 @@ defmodule String do
   def printable?(<<?\b, t :: binary>>), do: printable?(t)
   def printable?(<<?\f, t :: binary>>), do: printable?(t)
   def printable?(<<?\e, t :: binary>>), do: printable?(t)
+  def printable?(<<?\a, t :: binary>>), do: printable?(t)
 
   def printable?(<<>>), do: true
   def printable?(_),    do: false
 
-
   @doc """
   Divides a string into sub string based on a pattern,
   returning a list of these sub string. The pattern can
   be a string, a list of strings or a regular expression.
 
-  The string is split into two parts by default, unless
-  `global` option is true. If a pattern is not specified,
-  the string is split on whitespace occurrences.
+  The string is split into as many parts as possible by
+  default, unless the `global` option is set to false.
+  If a pattern is not specified, the string is split on
+  Unicode whitespace occurrences with leading and trailing
+  whitespace ignored.
 
   It returns a list with the original string if the pattern
   can't be matched.
 
   ## Examples
 
-      String.split("a,b,c", ",")  #=> ["a", "b", "c"]
-      String.split("a,b,c", ",", global: false)  #=> ["a", "b,c"]
+      iex> String.split("foo bar")
+      ["foo", "bar"]
+      iex> String.split("foo" <> <<194,133>> <> "bar")
+      ["foo", "bar"]
+      iex> String.split(" foo bar ")
+      ["foo", "bar"]
 
-      String.split("foo bar")     #=> ["foo", "bar"]
-      String.split("1,2 3,4", [" ", ","]) #=> ["1", "2", "3", "4"]
+      iex> String.split("a,b,c", ",")
+      ["a", "b", "c"]
+      iex> String.split("a,b,c", ",", global: false)
+      ["a", "b,c"]
 
-      String.split("a,b,c", %r{,}) #=> ["a", "b", "c"]
-      String.split("a,b,c", %r{,}, global: false)  #=> ["a", "b,c"]
-      String.split("a,b", %r{\.})   #=> ["a,b"]
+      iex> String.split("1,2 3,4", [" ", ","])
+      ["1", "2", "3", "4"]
+
+      iex> String.split("a,b,c", %r{,})
+      ["a", "b", "c"]
+      iex> String.split("a,b,c", %r{,}, global: false)
+      ["a", "b,c"]
+      iex> String.split("a,b", %r{\\.})
+      ["a,b"]
 
   """
-  def split(binary, pattern // " ", options // [])
+  @spec split(t) :: [t]
+  @spec split(t, t | [t] | Regex.t) :: [t]
+  @spec split(t, t | [t] | Regex.t, Keyword.t) :: [t]
+
+  defdelegate split(binary), to: String.Unicode
+
+  def split(binary, pattern, options // [])
 
   def split(binary, pattern, options) when is_regex(pattern) do
     Regex.split(pattern, binary, global: options[:global])
@@ -128,99 +195,134 @@ defmodule String do
   end
 
   @doc """
-  Convert all characters on the given string to upper case.
+  Convert all characters on the given string to upcase.
 
   ## Examples
 
-      String.upcase("abcd") #=> "ABCD"
-      String.upcase("ab 123 xpto") #=> "AB 123 XPTO"
-      String.upcase("josé") #=> "JOSÉ"
+      iex> String.upcase("abcd")
+      "ABCD"
+      iex> String.upcase("ab 123 xpto")
+      "AB 123 XPTO"
+      iex> String.upcase("josé")
+      "JOSÉ"
 
   """
-  def upcase(<<>>), do: <<>>
+  @spec upcase(t) :: t
+  defdelegate upcase(binary), to: String.Unicode
 
-  def upcase(<<195, c, t :: binary>>) when c in 160..191 do
-    <<195, c - 32, upcase(t) :: binary>>
-  end
+  @doc """
+  Convert all characters on the given string to downcase.
 
-  def upcase(<<c, t :: binary>>) when c in ?a..?z do
-    <<c  - 32, upcase(t) :: binary>>
-  end
+  ## Examples
 
-  def upcase(<<c, t :: binary>>) do
-    <<c , upcase(t) :: binary>>
+      iex> String.downcase("ABCD")
+      "abcd"
+      iex> String.downcase("AB 123 XPTO")
+      "ab 123 xpto"
+      iex> String.downcase("JOSÉ")
+      "josé"
+
+  """
+  @spec downcase(t) :: t
+  defdelegate downcase(binary), to: String.Unicode
+
+  @doc """
+  Converts the first character in the given string to
+  titlecase and the remaining to downcase.
+
+  This relies on the titlecase information provided
+  by the Unicode Standard. Note this function makes
+  no attempt in capitalizing all words in the string
+  (usually known as titlecase).
+
+  ## Examples
+
+      iex> String.capitalize("abcd")
+      "Abcd"
+      iex> String.capitalize("fin")
+      "Fin"
+      iex> String.capitalize("josé")
+      "José"
+
+  """
+  @spec capitalize(t) :: t
+  def capitalize(string) when is_binary(string) do
+    { char, rest } = String.Unicode.titlecase_once(string)
+    char <> downcase(rest)
   end
 
   @doc """
-  Convert all characters on the given string to down case.
+  Returns a string where trailing Unicode whitespace
+  has been removed.
 
   ## Examples
 
-      String.downcase("ABCD") #=> "abcd"
-      String.downcase("AB 123 XPTO") #=> "ab 123 xpto"
-      String.downcase("JOSÉ") #=> "josé"
+      iex> String.rstrip("   abc  ")
+      "   abc"
 
   """
-  def downcase(<<>>), do: <<>>
-
-  def downcase(<<195, c, t :: binary>>) when c in 128..159 do
-    <<195, c + 32, downcase(t) :: binary>>
-  end
-
-  def downcase(<<c, t :: binary>>) when c in ?A..?Z do
-    <<c + 32, downcase(t) :: binary>>
-  end
-
-  def downcase(<<c, t :: binary>>) do
-    <<c , downcase(t) :: binary>>
-  end
+  @spec rstrip(t) :: t
+  defdelegate rstrip(binary), to: String.Unicode
 
   @doc """
-  Returns a string where trailing char have been
-  removed. If no `char` is passed `space`is used.
+  Returns a string where trailing `char` have been removed.
 
   ## Examples
 
-      String.rstrip("   abc  ")      #=> "   abc"
-      String.rstrip("   abc _", ?_)  #=> "   abc "
+      iex> String.rstrip("   abc _", ?_)
+      "   abc "
 
   """
-  def rstrip(string, char // ?\s)
+  @spec rstrip(t, char) :: t
+
+  def rstrip("", _char), do: ""
 
   # Do a quick check before we traverse the whole
   # binary. :binary.last is a fast operation (it
   # does not traverse the whole binary).
   def rstrip(string, char) do
     if :binary.last(string) == char do
-      rstrip(string, "", char)
+      do_rstrip(string, "", char)
     else
       string
     end
   end
 
-  defp rstrip(<<char, string :: binary>>, buffer, char) do
-    rstrip(string, <<char, buffer :: binary>>, char)
+  defp do_rstrip(<<char, string :: binary>>, buffer, char) do
+    do_rstrip(string, <<char, buffer :: binary>>, char)
   end
 
-  defp rstrip(<<char, string :: binary>>, buffer, another_char) do
-    <<buffer :: binary, char, rstrip(string, "", another_char) :: binary>>
+  defp do_rstrip(<<char, string :: binary>>, buffer, another_char) do
+    <<buffer :: binary, char, do_rstrip(string, "", another_char) :: binary>>
   end
 
-  defp rstrip(<<>>, _, _) do
+  defp do_rstrip(<<>>, _, _) do
     <<>>
   end
 
   @doc """
-  Returns a string where leading char have been
-  removed. If no `char` is passed `space`is used.
+  Returns a string where leading Unicode whitespace
+  has been removed.
 
   ## Examples
 
-      String.lstrip("   abc  ")       #=> "abc  "
-      String.lstrip("_  abc  _", ?_)  #=> "  abc  _"
+      iex> String.lstrip("   abc  ")
+      "abc  "
 
   """
-  def lstrip(string, char // ?\s)
+  defdelegate lstrip(binary), to: String.Unicode
+
+  @doc """
+  Returns a string where leading `char` have been removed.
+
+  ## Examples
+
+      iex> String.lstrip("_  abc  _", ?_)
+      "  abc  _"
+
+  """
+
+  @spec lstrip(t, char) :: t
 
   def lstrip(<<char, rest :: binary>>, char) do
     <<lstrip(rest, char) :: binary>>
@@ -231,38 +333,63 @@ defmodule String do
   end
 
   @doc """
-  Returns a string where leading/trailing char have been
-  removed. If no `char` is passed `space`is used.
+  Returns a string where leading/trailing Unicode whitespace
+  has been removed.
 
   ## Examples
 
-      String.strip("   abc  ")       #=> "abc"
-      String.strip("a  abc  a", ?a)  #=> "  abc  "
+      iex> String.strip("   abc  ")
+      "abc"
 
   """
-  def strip(string, char // ?\s) do
+  @spec strip(t) :: t
+
+  def strip(string) do
+    rstrip(lstrip(string))
+  end
+
+  @doc """
+  Returns a string where leading/trailing `char` have been
+  removed.
+
+  ## Examples
+
+      iex> String.strip("a  abc  a", ?a)
+      "  abc  "
+
+  """
+  @spec strip(t, char) :: t
+
+  def strip(string, char) do
     rstrip(lstrip(string, char), char)
   end
 
   @doc """
   Returns a new binary based on `subject` by replacing the parts
-  matching `pattern` for `replacement`. If `options` is specified
-  with `[global: true]`, then it will replace all matches, otherwise
-  it will replace just the first one.
+  matching `pattern` for `replacement`. By default, it replaces
+  all entries, except if the `global` option is set to false.
 
-  For the replaced part must be used in `replacement`, then the
-  position or the positions where it is to be inserted must be specified by using
-  the option `insert_replaced`.
+  If the replaced part must be used in `replacement`, then the
+  position or the positions where it is to be inserted must be
+  specified by using the option `insert_replaced`.
 
   ## Examples
 
-      String.replace("a,b,c", ",", "-") #=> "a-b-c"
-      String.replace("a,b,c", ",", "-", global: false) #=> "a-b,c"
-      String.replace("a,b,c", "b", "[]", insert_replaced: 1) #=> "a,[b],c"
-      String.replace("a,b,c", ",", "[]", insert_replaced: 2) #=> "a[],b[],c"
-      String.replace("a,b,c", ",", "[]", insert_replaced: [1,1]) #=> "a[,,]b[,,]c"
+      iex> String.replace("a,b,c", ",", "-")
+      "a-b-c"
+      iex> String.replace("a,b,c", ",", "-", global: false)
+      "a-b,c"
+      iex> String.replace("a,b,c", "b", "[]", insert_replaced: 1)
+      "a,[b],c"
+      iex> String.replace("a,b,c", ",", "[]", insert_replaced: 2)
+      "a[],b[],c"
+      iex> String.replace("a,b,c", ",", "[]", insert_replaced: [1,1])
+      "a[,,]b[,,]c"
 
   """
+  @spec replace(t, t, t) :: t
+  @spec replace(t, t, t, Keyword.t) :: t
+
   def replace(subject, pattern, replacement, options // []) do
     opts = translate_replace_options(options)
     :binary.replace(subject, pattern, replacement, opts)
@@ -283,143 +410,262 @@ defmodule String do
 
   ## Examples
 
-      String.duplicate("abc", 1) #=> "abc"
-      String.duplicate("abc", 2) #=> "abcabc"
+      iex> String.duplicate("abc", 1)
+      "abc"
+      iex> String.duplicate("abc", 2)
+      "abcabc"
 
   """
+  @spec duplicate(t, pos_integer) :: t
   def duplicate(subject, n) when is_integer(n) and n > 0 do
     :binary.copy(subject, n)
   end
 
   @doc """
-  Returns a list with codepoints from an utf8 string.
+  Returns all codepoints in the string.
 
   ## Examples
 
-      String.codepoints("josé")         #=> ["j", "o", "s", "é"]
-      String.codepoints("оптими зации") #=> ["о","п","т","и","м","и"," ","з","а","ц","и","и"]
-      String.codepoints("ἅἪῼ")          #=> ["ἅ","Ἢ","ῼ"]
+      iex> String.codepoints("josé")
+      ["j", "o", "s", "é"]
+      iex> String.codepoints("оптими зации")
+      ["о","п","т","и","м","и"," ","з","а","ц","и","и"]
+      iex> String.codepoints("ἅἪῼ")
+      ["ἅ","Ἢ","ῼ"]
 
   """
-  def codepoints(string) do
-    do_codepoints(codepoint(string))
-  end
-
-  defp do_codepoints({char, rest}) do
-    [char|do_codepoints(codepoint(rest))]
-  end
-
-  defp do_codepoints(:no_codepoint), do: []
+  @spec codepoints(t) :: [codepoint]
+  defdelegate codepoints(string), to: String.Unicode
 
   @doc """
-  Returns the first codepoint from an utf8 string.
+  Returns the next codepoint in a String.
+
+  The result is a tuple with the codepoint and the
+  remaining of the string or `:no_codepoint` in case
+  the string reached its end.
+
+  As the other functions in the String module, this
+  function does not check for the validity of the codepoint.
+  That said, if an invalid codepoint is found, it will
+  be returned by this function.
 
   ## Examples
 
-      String.first("elixir")  #=> "e"
-      String.first("եոգլի") #=> "ե"
+      iex> String.next_codepoint("josé")
+      { "j", "osé" }
 
   """
+  @spec next_codepoint(t) :: {codepoint, t} | :no_codepoint
+  defdelegate next_codepoint(string), to: String.Unicode
+
+  @doc %B"""
+  Checks whether `str` is a valid codepoint.
+
+  Note that the empty string is considered invalid, as are
+  strings containing multiple codepoints.
+
+  ## Examples
+
+      iex> String.valid_codepoint?("a")
+      true
+      iex> String.valid_codepoint?("ø")
+      true
+      iex> String.valid_codepoint?("\xffff")
+      false
+      iex> String.valid_codepoint?("asdf")
+      false
+
+  """
+  @spec valid_codepoint?(codepoint) :: boolean
+  def valid_codepoint?(<<_ :: utf8>>), do: true
+  def valid_codepoint?(_), do: false
+
+  @doc """
+  Returns unicode graphemes in the string.
+
+  ## Examples
+
+      iex> String.graphemes("Ā̀stute")
+      ["Ā̀","s","t","u","t","e"]
+
+  """
+  @spec graphemes(t) :: [grapheme]
+  defdelegate graphemes(string), to: String.Unicode
+
+  @doc """
+  Returns the next grapheme in a String.
+
+  The result is a tuple with the grapheme and the
+  remaining of the string or `:no_grapheme` in case
+  the String reached its end.
+
+  ## Examples
+
+      iex> String.next_grapheme("josé")
+      { "j", "osé" }
+
+  """
+  @spec next_grapheme(t) :: grapheme | :no_grapheme
+  defdelegate next_grapheme(string), to: String.Unicode
+
+  @doc """
+  Returns the first grapheme from an utf8 string.
+
+  ## Examples
+
+      iex> String.first("elixir")
+      "e"
+      iex> String.first("եոգլի")
+      "ե"
+
+  """
+  @spec first(t) :: grapheme | nil
   def first(string) do
-    case codepoint(string) do
+    case next_grapheme(string) do
       { char, _ } -> char
-      :no_codepoint -> ""
+      :no_grapheme -> nil
     end
   end
 
   @doc """
-  Returns the last codepoint from an utf8 string.
+  Returns the last grapheme from an utf8 string.
 
   ## Examples
 
-      String.last("elixir")  #=> "r"
-      String.last("եոգլի") #=> "ի"
+      iex> String.last("elixir")
+      "r"
+      iex> String.last("եոգլի")
+      "ի"
 
   """
+  @spec last(t) :: grapheme | nil
   def last(string) do
-    do_last(codepoint(string), "")
+    do_last(next_grapheme(string), nil)
   end
 
   defp do_last({char, rest}, _) do
-    do_last(codepoint(rest), char)
+    do_last(next_grapheme(rest), char)
   end
 
-  defp do_last(:no_codepoint, last_char), do: last_char
+  defp do_last(:no_grapheme, last_char), do: last_char
 
   @doc """
-  Returns the number of codepoint in an utf8 string.
+  Returns the number of unicode graphemes in an utf8 string.
 
   ## Examples
 
-      String.length("elixir")  #=> 6
-      String.length("եոգլի") #=> 5
+      iex> String.length("elixir")
+      6
+      iex> String.length("եոգլի")
+      5
 
   """
+  @spec length(t) :: non_neg_integer
   def length(string) do
-    do_length(codepoint(string))
+    do_length(next_grapheme(string))
   end
 
   defp do_length({_, rest}) do
-    1 + do_length(codepoint(rest))
+    1 + do_length(next_grapheme(rest))
   end
 
-  defp do_length(:no_codepoint), do: 0
+  defp do_length(:no_grapheme), do: 0
 
   @doc """
-  Returns the codepoint in the `position` of the given utf8 `string`.
+  Returns the grapheme in the `position` of the given utf8 `string`.
   If `position` is greater than `string` length, than it returns `nil`.
 
   ## Examples
 
-      String.at("elixir", 0) #=> "1"
-      String.at("elixir", 1) #=> "l"
-      String.at("elixir", 10) #=> nil
-      String.at("elixir", -1) #=> "r"
-      String.at("elixir", -10) #=> "nil"
+      iex> String.at("elixir", 0)
+      "e"
+      iex> String.at("elixir", 1)
+      "l"
+      iex> String.at("elixir", 10)
+      nil
+      iex> String.at("elixir", -1)
+      "r"
+      iex> String.at("elixir", -10)
+      nil
 
   """
+  @spec at(t, integer) :: grapheme | nil
+
   def at(string, position) when position >= 0 do
-    do_at(codepoint(string), position, 0)
+    do_at(next_grapheme(string), position, 0)
   end
 
   def at(string, position) when position < 0 do
-    real_pos = do_length(codepoint(string)) - abs(position)
+    real_pos = do_length(next_grapheme(string)) - abs(position)
     case real_pos >= 0 do
-      true -> do_at(codepoint(string), real_pos, 0)
-      false -> ""
+      true  -> do_at(next_grapheme(string), real_pos, 0)
+      false -> nil
     end
   end
 
   defp do_at({_ , rest}, desired_pos, current_pos) when desired_pos > current_pos do
-    do_at(codepoint(rest), desired_pos, current_pos + 1)
+    do_at(next_grapheme(rest), desired_pos, current_pos + 1)
   end
 
   defp do_at({char, _}, desired_pos, current_pos) when desired_pos == current_pos do
     char
   end
 
-  defp do_at(:no_codepoint, _, _), do: ""
+  defp do_at(:no_grapheme, _, _), do: nil
 
-  # Private implementation which returns the first codepoint
-  # of any given utf8 string and the rest of it
-  # If an empty string is given, :no_codepoint is returned.
-  defp codepoint(<<194, char, rest :: binary>>)
-    when char in 161..191,
-    do: { <<194, char>>, rest }
+  @doc """
+  Returns a substring starting at the offset given by the first, and
+  a length given by the second.
+  If the offset is greater than string length, than it returns nil.
 
-  defp codepoint(<<first, char, rest :: binary>>)
-    when first in 195..223 and char in 128..191,
-    do: { <<first, char>>, rest }
+  ## Examples
 
-  defp codepoint(<<first, second, char, rest :: binary>>)
-    when first == 224 and second in 160..191 and char in 128..191,
-    do: { <<first, second, char>>, rest }
+      iex> String.slice("elixir", 1, 3)
+      "lix"
+      iex> String.slice("elixir", 1, 10)
+      "lixir"
+      iex> String.slice("elixir", 10, 3)
+      nil
+      iex> String.slice("elixir", -4, 4)
+      "ixir"
+      iex> String.slice("elixir", -10, 3)
+      nil
 
-  defp codepoint(<<first, second, char, rest :: binary>>)
-    when first in 225..239 and second in 128..191 and char in 128..191,
-    do: { <<first, second, char>>, rest }
+  """
+  @spec slice(t, integer, integer) :: grapheme | nil
 
-  defp codepoint(<<other, rest :: binary>>), do: { <<other>>, rest }
+  def slice(string, start, len) when start >= 0 do
+    do_slice(next_grapheme(string), start, start + len - 1, 0, "")
+  end
 
-  defp codepoint(<<>>), do: :no_codepoint
+  def slice(string, start, len) when start < 0 do
+    real_start_pos = do_length(next_grapheme(string)) - abs(start)
+    case real_start_pos >= 0 do
+      true -> do_slice(next_grapheme(string), real_start_pos, real_start_pos + len - 1, 0, "")
+      false -> nil
+    end
+  end
+
+  defp do_slice(_, start_pos, last_pos, _, _) when start_pos > last_pos do
+    nil
+  end
+
+  defp do_slice({_, rest}, start_pos, last_pos, current_pos, acc) when current_pos < start_pos do
+    do_slice(next_grapheme(rest), start_pos, last_pos, current_pos + 1, acc)
+  end
+
+  defp do_slice({char, rest}, start_pos, last_pos, current_pos, acc) when current_pos >= start_pos and current_pos < last_pos do
+    do_slice(next_grapheme(rest), start_pos, last_pos, current_pos + 1, acc <> char)
+  end
+
+  defp do_slice({char, _}, start_pos, last_pos, current_pos, acc) when current_pos >= start_pos and current_pos == last_pos do
+    acc <> char
+  end
+
+  defp do_slice(:no_grapheme, _, _, _, acc) do
+    case acc do
+      "" -> nil
+      _ -> acc
+    end
+  end
 end

lib/elixir/lib/supervisor/behaviour.ex

@@ -0,0 +1,183 @@
+defmodule Supervisor.Behaviour do
+  @moduledoc """
+  This module is a convenience to define Supervisor
+  callbacks in Elixir. By using this module, you get
+  the module behaviour automatically tagged as
+  `:supervisor` and some helper functions are imported
+  to make defining supervisors easier.
+
+  For more information on supervisors, please check the
+  remaining functions defined in this module or refer to
+  the following:
+
+  http://www.erlang.org/doc/man/supervisor.html
+  http://www.erlang.org/doc/design_principles/sup_princ.html
+  http://learnyousomeerlang.com/supervisors
+
+  ## Example
+
+      defmodule ExUnit.Sup do
+        use Supervisor.Behaviour
+
+        def init(user_options) do
+          tree = [ worker(ExUnit.Runner, [user_options]) ]
+          supervise(tree, strategy: :one_for_one)
+        end
+      end
+
+      { :ok, pid } = :supervisor.start_link(MyServer, [])
+
+  """
+
+  @doc false
+  defmacro __using__(_) do
+    quote location: :keep do
+      @behaviour :supervisor
+      import unquote(__MODULE__)
+    end
+  end
+
+  @doc """
+  Receives a list of children (worker or supervisors) to
+  supervise and a set of options. Returns a tuple containing
+  the supervisor specification.
+
+  ## Examples
+
+      supervise children, strategy: :one_for_one
+
+  ## Options
+
+  * `:strategy` - the restart strategy option It can be either
+    `:one_for_one`, `:rest_for_one`, `:one_for_all` and
+    `:simple_one_for_one`;
+
+  * `:max_restarts` - the maximum amount of restarts allowed in
+    a time frame. Defaults to 5;
+
+  * `:max_seconds` - the time frame in which max_restarts applies.
+    Defaults to 5;
+
+  The `:strategy` option is required and by default maximum 5 restarts
+  are allowed in 5 seconds.
+
+  ## Strategies
+
+  * `:one_for_one` - If a child process terminates, only that
+    process is restarted;
+
+  * `:one_for_all` - If a child process terminates, all other child
+    processes are terminated and then all child processes, including
+    the terminated one, are restarted;
+
+  * `:rest_for_one` - If a child process terminates, the "rest" of
+    the child processes, i.e. the child processes after the terminated
+    process in start order, are terminated. Then the terminated child
+    process and the rest of the child processes are restarted;
+
+  * `:simple_one_for_one` - Similar to `:one_for_one` but suits better
+    when dynamically attaching children;
+
+  """
+  def supervise(children, options) do
+    unless strategy = options[:strategy] do
+      raise ArgumentError, message: "expected :strategy option to be given to supervise"
+    end
+
+    maxR = Keyword.get(options, :max_restarts, 5)
+    maxS = Keyword.get(options, :max_seconds, 5)
+
+    { :ok, { { strategy, maxR, maxS }, children } }
+  end
+
+  @child_doc """
+  ## Options
+
+  * `:id` - a name used to identify the child specification
+    internally by the supervisor. Defaults to the module name;
+
+  * `:function` - the function to invoke on the child to start it.
+    Defaults to `:start_link`;
+
+  * `:restart` - defines when the child process should restart.
+    Defaults to `:permanent`;
+
+  * `:shutdown` - defines how a child process should be terminated.
+    Defaults to `5000`;
+
+  * `:modules` - it should be a list with one element `[module]`,
+    where module is the name of the callback module only if the
+    child process is a supervisor, gen_server or gen_fsm. If the
+    child process is a gen_event, modules should be `:dynamic`.
+    Defaults to a list with the given module;
+
+  ## Restart values
+
+  The following restart values are supported:
+
+  * `:permanent` - the child process is always restarted;
+
+  * `:temporary` - the child process is never restarted (not even
+    when the supervisor's strategy is `:rest_for_one` or `:one_for_all`);
+
+  * `:transient` - the child process is restarted only if it
+    terminates abnormally, i.e. with another exit reason than
+    `:normal`, `:shutdown` or `{ :shutdown, term }`;
+
+  ## Shutdown values
+
+  The following shutdown values are supported:
+
+  * `:brutal_kill` - the child process is unconditionally terminated
+    using `exit(child, :kill)`;
+
+  * `:infinity` - if the child process is a supervisor, it is a mechanism
+    to give the subtree enough time to shutdown. It can also be used with
+    workers with care;
+
+  * Finally, it can also be any integer meaning that the supervisor tells
+    the child process to terminate by calling `exit(child, :shutdown)` and
+    then waits for an exit signal back. If no exit signal is received within
+    the specified time (in miliseconds), the child process is unconditionally
+    terminated using `exit(child, :kill)`;
+  """
+
+  @doc """
+  Defines the given `module` as a worker which will be started
+  with the given arguments.
+
+      worker ExUnit.Runner, [], restart: :permanent
+
+  By default, the function `:start_link` is invoked on the given module.
+
+  #{@child_doc}
+  """
+  def worker(module, args, options // []) do
+    child(:worker, module, args, options)
+  end
+
+  @doc """
+  Defines the given `module` as a supervisor which will be started
+  with the given arguments.
+
+      supervisor ExUnit.Runner, [], restart: :permanent
+
+  By default, the function `:start_link` is invoked on the given module.
+
+  #{@child_doc}
+  """
+  def supervisor(module, args, options // []) do
+    child(:supervisor, module, args, options)
+  end
+
+  defp child(type, module, args, options) do
+    id       = Keyword.get(options, :id, module)
+    modules  = Keyword.get(options, :modules, [module])
+    function = Keyword.get(options, :function, :start_link)
+    restart  = Keyword.get(options, :restart, :permanent)
+    shutdown = Keyword.get(options, :shutdown, 5000)
+
+    { id, { module, function, args },
+      restart, shutdown, type, modules }
+  end
+end

lib/elixir/lib/system.ex

@@ -1,14 +1,29 @@
 defmodule System do
+  defexception NoHomeError,
+    message: "could not find the user home, please set the HOME environment variable"
+
+  defexception NoTmpDirError,
+    message: "could not get a writable temporary directory, please set the TMPDIR environment variable"
+
+  defexception NoAccessCwdError,
+    message: "could not get a current working directory, the current location is not accessible"
+
   @moduledoc """
   The System module provides access to some variables used or
   maintained by the VM and to functions that interact strongly
   with the VM or the host system.
   """
 
+  # Read and strip the version from the `VERSION` file.
+  defmacrop get_version do
+    Regex.replace %r/^\s+|\s+$/, File.read!("VERSION"), ""
+  end
+
   # Tries to run `git describe --always --tags`. In case of success
   # returns the most recent tag, otherwise returns an empty string.
   defmacrop get_describe do
-    if :os.find_executable('git') do
+    dotgit = Path.join(File.cwd!, ".git")
+    if :os.find_executable('git') && File.exists?(dotgit) do
       data = :os.cmd('git describe --always --tags')
       Regex.replace %r/\n/, to_binary(data), ""
     else
@@ -24,11 +39,13 @@ defmodule System do
   @doc """
   Returns Elixir's version as binary.
   """
-  def version, do: "0.7.0"
+  @spec version() :: String.t
+  def version, do: get_version
 
   @doc """
   Returns a keywords list with version, git tag info and date.
   """
+  @spec build_info() :: Keyword.t
   def build_info do
     [version: version, tag: get_describe, date: get_date]
   end
@@ -36,20 +53,109 @@ defmodule System do
   @doc """
   Returns the list of command-line arguments passed to the program.
   """
+  @spec argv() :: [String.t]
   def argv do
-    :gen_server.call(:elixir_code_server, :argv)
+    :elixir_code_server.call :argv
+  end
+
+  @doc """
+  Returns the current working directory or nil if one
+  is not available.
+  """
+  def cwd do
+    case :file.get_cwd do
+      { :ok, list } -> :unicode.characters_to_binary(list)
+      _ -> nil
+    end
+  end
+
+  @doc """
+  Returns the current working directory or raises `System.NoAccessCwdError`.
+  """
+  def cwd! do
+    cwd || raise NoAccessCwdError
+  end
+
+  @doc """
+  Returns the user home (platform independent).
+  It returns nil if no user home is set.
+  """
+  def user_home do
+    get_unix_home || get_windows_home
+  end
+
+  @doc """
+  Same as `user_home` but raises `System.NoHomeError`
+  instead of returning nil if no user home is set.
+  """
+  def user_home! do
+    user_home || raise NoHomeError
+  end
+
+  defp get_unix_home do
+    get_env("HOME")
+  end
+
+  defp get_windows_home do
+    get_env("USERPROFILE") || (
+      hd = get_env("HOMEDRIVE")
+      hp = get_env("HOMEPATH")
+      hd && hp && hd <> hp
+    )
+  end
+
+  @doc %B"""
+  Returns a writable temporary directory.
+  It searches for directories in the following order:
+
+  1. The directory named by the TMPDIR environment variable
+  2. The directory named by the TEMP environment variable
+  3. The directory named by the TMP environment variable
+  4. `C:\TMP` on Windows or `/tmp` on Unix
+  5.  As a last resort, the current working directory
+
+  Returns nil if none of the above are writable.
+  """
+  def tmp_dir do
+    write_env_tmp_dir('TMPDIR') ||
+      write_env_tmp_dir('TEMP')  ||
+      write_env_tmp_dir('TMP') ||
+      write_tmp_dir("/tmp")     ||
+      ((cwd = cwd()) && write_tmp_dir(cwd))
+  end
+
+  @doc """
+  Same as `tmp_dir` but raises `System.NoTmpDirError`
+  instead of returning nil if no temp dir is set.
+  """
+  def tmp_dir! do
+    tmp_dir || raise NoTmpDirError
+  end
+
+  defp write_env_tmp_dir(env) do
+    case System.get_env(env) do
+      nil -> nil
+      tmp -> write_tmp_dir tmp
+    end
+  end
+
+  defp write_tmp_dir(dir) do
+    case File.stat(dir) do
+      { :ok, File.Stat[type: :directory, access: access] } when access in [:read_write, :write] -> dir
+      { :error, _ } -> nil
+    end
   end
 
   @doc """
   Registers a function that will be invoked
   at the end of program execution. Useful for
-  invoking a hook on scripted mode.
+  invoking a hook in a "script" mode.
 
   The function must expect the exit status code
   as argument.
   """
   def at_exit(fun) when is_function(fun, 1) do
-    server_call { :at_exit, fun }
+    :elixir_code_server.cast { :at_exit, fun }
   end
 
   @doc """
@@ -60,6 +166,8 @@ defmodule System do
   If `command` is a char list, a char list is returned.
   Returns a binary otherwise.
   """
+  @spec cmd(char_list) :: char_list
+  @spec cmd(String.t) :: String.t
   def cmd(command) when is_list(command) do
     :os.cmd(command)
   end
@@ -76,6 +184,8 @@ defmodule System do
   If `command` is a char list, a char list is returned.
   Returns a binary otherwise.
   """
+  @spec find_executable(char_list) :: char_list | nil
+  @spec find_executable(String.t) :: String.t | nil
   def find_executable(command) when is_list(command) do
     :os.find_executable(command) || nil
   end
@@ -92,8 +202,9 @@ defmodule System do
   given as a single string of the format "VarName=Value", where VarName is the
   name of the variable and Value its value.
   """
+  @spec get_env() :: [{String.t, String.t}]
   def get_env do
-    Enum.map :os.getenv, list_to_binary &1
+    Enum.map :os.getenv, :unicode.characters_to_binary &1
   end
 
   @doc """
@@ -101,10 +212,11 @@ defmodule System do
   `varname` as a binary, or nil if the environment
   variable is undefined.
   """
+  @spec get_env(String.t) :: String.t | nil
   def get_env(varname) do
     case :os.getenv(to_char_list(varname)) do
       false -> nil
-      other -> list_to_binary(other)
+      other -> :unicode.characters_to_binary(other)
     end
   end
 
@@ -114,40 +226,91 @@ defmodule System do
 
   See http://www.erlang.org/doc/man/os.html#getpid-0 for more info.
   """
+  @spec get_pid() :: String.t
   def get_pid, do: list_to_binary(:os.getpid)
 
   @doc """
   Sets a new `value` for the environment variable `varname`.
   """
-  def put_env(varname, value) do
-   :os.putenv to_char_list(varname), to_char_list(value)
+  @spec put_env(String.t, String.t | char_list) :: :ok
+  def put_env(varname, value) when is_binary(value) or is_list(value) do
+   :os.putenv to_char_list(varname), :unicode.characters_to_list(value)
+   :ok
   end
 
   @doc """
   Sets a new value for each environment variable corresponding
   to each key in `dict`.
   """
+  @spec put_env(Dict.t) :: :ok
   def put_env(dict) do
     Enum.each dict, fn {key, val} -> put_env key, val end
   end
 
   @doc """
-  Get the stacktrace.
+  Gets Elixir's stacktrace.
+
+  Notice the Erlang VM (and therefore this function) does not
+  return the current stacktrace but rather the stacktrace of the
+  latest exception.
   """
   def stacktrace do
     filter_stacktrace :erlang.get_stacktrace
   end
 
+  @doc """
+  Halts the Erlang runtime system where the first argument status must be a
+  non-negative integer, the atom `:abort` or any type that can be converted
+  to a char list.
+
+  * If an integer, the runtime system exits with the integer value which
+    is returned to the Operating System;
+
+  * If `:abort`, the runtime system aborts producing a core dump, if that is
+    enabled in the operating system;
+
+  * If a char list, an erlang crash dump is produced with status as slogan,
+    and then the runtime system exits with status code 1;
+
+  Note that on many platforms, only the status codes 0-255 are supported
+  by the operating system.
+
+  For integer status, Erlang runtime system closes all ports and allows async
+  threads to finish their operations before exiting. To exit without such
+  flushing, pass options [flush: false] instead.
+
+  For more information, check: http://www.erlang.org/doc/man/erlang.html#halt-2
+
+  ## Examples
+
+      System.halt(0)
+      System.halt(1, flush: false)
+      System.halt(:abort)
+
+  """
+  @spec halt() :: no_return
+  @spec halt(non_neg_integer | List.Chars.t | :abort) :: no_return
+  @spec halt(non_neg_integer | List.Chars.t | :abort, [] | [flush: false]) :: no_return
+  def halt(status // 0, options // [])
+
+  def halt(status, options) when is_integer(status) or status == :abort do
+    do_halt(status, options)
+  end
+
+  def halt(status, options) do
+    do_halt(to_char_list(status), options)
+  end
+
+  # Support R15B
+  if List.member?(:erlang.module_info(:exports), { :halt, 2 }) do
+    defp do_halt(status, options),  do: :erlang.halt(status, options)
+  else
+    IO.puts "Using limited halt support. Upgrade to R15B01 or later is recommended."
+    defp do_halt(status, _options), do: :erlang.halt(status)
+  end
+
   ## Helpers
 
-  # Filter stacktrace by removing internal BOOTSTRAP calls.
-  defp filter_stacktrace([{ Kernel, :raise, _, _ }|t]), do: filter_stacktrace(t)
-  defp filter_stacktrace([{ _mod, :BOOTSTRAP, _, info }|t]),
-    do: filter_stacktrace([{ Kernel, :defmodule, 2, info }|t])
-  defp filter_stacktrace([h|t]), do: [h|filter_stacktrace(t)]
-  defp filter_stacktrace([]), do: []
-
-  defp server_call(args) do
-    :gen_server.call(:elixir_code_server, args)
-  end
+  defp filter_stacktrace([{ Kernel, :raise, _, _ }|t]), do: t
+  defp filter_stacktrace(t), do: t
 end

lib/elixir/lib/tuple.ex

@@ -1,2 +1,3 @@
 defmodule Tuple do
+  @moduledoc false
 end

lib/elixir/lib/uri.ex

@@ -28,7 +28,7 @@ defmodule URI do
 
   Use decoder/1 if you want to customize or iterate each value manually.
   """
-  def decode_query(q, dict // OrdDict.new) do
+  def decode_query(q, dict // HashDict.new) when is_binary(q) do
     Enum.reduce query_decoder(q), dict, fn({ k, v }, acc) -> Dict.put(acc, k, v) end
   end
 
@@ -36,8 +36,8 @@ defmodule URI do
   Returns an iterator function over the query string that decodes
   the query string in steps.
   """
-  def query_decoder(q) do
-    fn -> { do_decoder(&1), do_decoder(to_binary(q)) } end
+  def query_decoder(q) when is_binary(q) do
+    fn -> { do_decoder(&1), q } end
   end
 
   defp do_decoder("") do
@@ -81,14 +81,9 @@ defmodule URI do
     <<c>>
   end
 
-  defp percent(c), do: escape_byte(c)
-
-  defp escape_byte(c), do: "%" <> hex(c)
+  defp percent(c), do: "%" <> hex(bsr(c, 4)) <> hex(band(c, 15))
 
   defp hex(n) when n <= 9, do: <<n + ?0>>
-  defp hex(n) when n > 15 do
-    hex(bsr(n, 4)) <> hex(band(n, 15))
-  end
   defp hex(n), do: <<n + ?A - 10>>
 
   @doc """
@@ -128,10 +123,10 @@ defmodule URI do
   for that particular scheme. Take a look at URI.HTTPS for an
   example of one of these extension modules.
   """
-  def parse(s) do
+  def parse(s) when is_binary(s) do
     # From http://tools.ietf.org/html/rfc3986#appendix-B
     regex = %r/^(([^:\/?#]+):)?(\/\/([^\/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))?/
-    parts = nillify(Regex.run(regex, to_binary(s)))
+    parts = nillify(Regex.run(regex, s))
 
     destructure [_, _, scheme, _, authority, path, _, query, _, fragment], parts
     { userinfo, host, port } = split_authority(authority)
@@ -149,12 +144,12 @@ defmodule URI do
     if scheme do
       module =
         try do
-          Module.safe_concat(URI, :string.to_upper(binary_to_list(scheme)))
+          Module.safe_concat(URI, String.upcase(scheme))
         rescue
           ArgumentError -> nil
         end
 
-      if module && match?({:module,^module}, Code.ensure_loaded(module)) do
+      if module && Code.ensure_loaded?(module) do
         module.parse(default_port(info, module))
       else
         info
@@ -173,7 +168,7 @@ defmodule URI do
     s = s || ""
     components = Regex.run %r/(^(.*)@)?([^:]*)(:(\d*))?/, s
     destructure [_, _, userinfo, host, _, port], nillify(components)
-    port = if port, do: list_to_integer(binary_to_list(port))
+    port = if port, do: binary_to_integer(port)
     { userinfo, host, port }
   end
 

lib/elixir/lib/uri/parser.ex

@@ -9,10 +9,10 @@ defmodule URI.Parser do
   @doc """
   Responsible for parsing extra URL information.
   """
-  defcallback parse(uri_info)
+  defcallback parse(uri_info :: URI.Info.t) :: URI.Info.t
 
   @doc """
   Responsible for returning the default port.
   """
-  defcallback default_port()
+  defcallback default_port() :: integer
 end

lib/elixir/priv/NamedSequences.txt

@@ -0,0 +1,420 @@
+LATIN CAPITAL LETTER A WITH MACRON AND GRAVE;0100 0300
+LATIN SMALL LETTER A WITH MACRON AND GRAVE;0101 0300
+LATIN CAPITAL LETTER E WITH VERTICAL LINE BELOW;0045 0329
+LATIN SMALL LETTER E WITH VERTICAL LINE BELOW;0065 0329
+LATIN CAPITAL LETTER E WITH VERTICAL LINE BELOW AND GRAVE;00C8 0329
+LATIN SMALL LETTER E WITH VERTICAL LINE BELOW AND GRAVE;00E8 0329
+LATIN CAPITAL LETTER E WITH VERTICAL LINE BELOW AND ACUTE;00C9 0329
+LATIN SMALL LETTER E WITH VERTICAL LINE BELOW AND ACUTE;00E9 0329
+LATIN CAPITAL LETTER E WITH CIRCUMFLEX AND MACRON;00CA 0304
+LATIN SMALL LETTER E WITH CIRCUMFLEX AND MACRON;00EA 0304
+LATIN CAPITAL LETTER E WITH CIRCUMFLEX AND CARON;00CA 030C
+LATIN SMALL LETTER E WITH CIRCUMFLEX AND CARON;00EA 030C
+LATIN CAPITAL LETTER I WITH MACRON AND GRAVE;012A 0300
+LATIN SMALL LETTER I WITH MACRON AND GRAVE;012B 0300
+LATIN SMALL LETTER I WITH DOT ABOVE AND ACUTE;0069 0307 0301
+LATIN SMALL LETTER NG WITH TILDE ABOVE;006E 0360 0067
+LATIN CAPITAL LETTER O WITH VERTICAL LINE BELOW;004F 0329
+LATIN SMALL LETTER O WITH VERTICAL LINE BELOW;006F 0329
+LATIN CAPITAL LETTER O WITH VERTICAL LINE BELOW AND GRAVE;00D2 0329
+LATIN SMALL LETTER O WITH VERTICAL LINE BELOW AND GRAVE;00F2 0329
+LATIN CAPITAL LETTER O WITH VERTICAL LINE BELOW AND ACUTE;00D3 0329
+LATIN SMALL LETTER O WITH VERTICAL LINE BELOW AND ACUTE;00F3 0329
+LATIN CAPITAL LETTER S WITH VERTICAL LINE BELOW;0053 0329
+LATIN SMALL LETTER S WITH VERTICAL LINE BELOW;0073 0329
+LATIN CAPITAL LETTER U WITH MACRON AND GRAVE;016A 0300
+LATIN SMALL LETTER U WITH MACRON AND GRAVE;016B 0300
+LATIN CAPITAL LETTER A WITH OGONEK AND ACUTE;0104 0301
+LATIN SMALL LETTER A WITH OGONEK AND ACUTE;0105 0301
+LATIN CAPITAL LETTER A WITH OGONEK AND TILDE;0104 0303
+LATIN SMALL LETTER A WITH OGONEK AND TILDE;0105 0303
+LATIN CAPITAL LETTER E WITH OGONEK AND ACUTE;0118 0301
+LATIN SMALL LETTER E WITH OGONEK AND ACUTE;0119 0301
+LATIN CAPITAL LETTER E WITH OGONEK AND TILDE;0118 0303
+LATIN SMALL LETTER E WITH OGONEK AND TILDE;0119 0303
+LATIN CAPITAL LETTER E WITH DOT ABOVE AND ACUTE;0116 0301
+LATIN SMALL LETTER E WITH DOT ABOVE AND ACUTE;0117 0301
+LATIN CAPITAL LETTER E WITH DOT ABOVE AND TILDE;0116 0303
+LATIN SMALL LETTER E WITH DOT ABOVE AND TILDE;0117 0303
+LATIN SMALL LETTER I WITH DOT ABOVE AND GRAVE;0069 0307 0300
+LATIN SMALL LETTER I WITH DOT ABOVE AND TILDE;0069 0307 0303
+LATIN CAPITAL LETTER I WITH OGONEK AND ACUTE;012E 0301
+LATIN SMALL LETTER I WITH OGONEK AND DOT ABOVE AND ACUTE;012F 0307 0301
+LATIN CAPITAL LETTER I WITH OGONEK AND TILDE;012E 0303
+LATIN SMALL LETTER I WITH OGONEK AND DOT ABOVE AND TILDE;012F 0307 0303
+LATIN CAPITAL LETTER J WITH TILDE;004A 0303
+LATIN SMALL LETTER J WITH DOT ABOVE AND TILDE;006A 0307 0303
+LATIN CAPITAL LETTER L WITH TILDE;004C 0303
+LATIN SMALL LETTER L WITH TILDE;006C 0303
+LATIN CAPITAL LETTER M WITH TILDE;004D 0303
+LATIN SMALL LETTER M WITH TILDE;006D 0303
+LATIN CAPITAL LETTER R WITH TILDE;0052 0303
+LATIN SMALL LETTER R WITH TILDE;0072 0303
+LATIN CAPITAL LETTER U WITH OGONEK AND ACUTE;0172 0301
+LATIN SMALL LETTER U WITH OGONEK AND ACUTE;0173 0301
+LATIN CAPITAL LETTER U WITH OGONEK AND TILDE;0172 0303
+LATIN SMALL LETTER U WITH OGONEK AND TILDE;0173 0303
+LATIN CAPITAL LETTER U WITH MACRON AND ACUTE;016A 0301
+LATIN SMALL LETTER U WITH MACRON AND ACUTE;016B 0301
+LATIN CAPITAL LETTER U WITH MACRON AND TILDE;016A 0303
+LATIN SMALL LETTER U WITH MACRON AND TILDE;016B 0303
+LATIN SMALL LETTER AE WITH GRAVE;00E6 0300
+LATIN SMALL LETTER OPEN O WITH GRAVE;0254 0300
+LATIN SMALL LETTER OPEN O WITH ACUTE;0254 0301
+LATIN SMALL LETTER TURNED V WITH GRAVE;028C 0300
+LATIN SMALL LETTER TURNED V WITH ACUTE;028C 0301
+LATIN SMALL LETTER SCHWA WITH GRAVE;0259 0300
+LATIN SMALL LETTER SCHWA WITH ACUTE;0259 0301
+LATIN SMALL LETTER HOOKED SCHWA WITH GRAVE;025A 0300
+LATIN SMALL LETTER HOOKED SCHWA WITH ACUTE;025A 0301
+BENGALI LETTER KHINYA;0995 09CD 09B7
+TAMIL CONSONANT K;  0B95 0BCD
+TAMIL CONSONANT NG; 0B99 0BCD
+TAMIL CONSONANT C;  0B9A 0BCD
+TAMIL CONSONANT NY; 0B9E 0BCD
+TAMIL CONSONANT TT; 0B9F 0BCD
+TAMIL CONSONANT NN; 0BA3 0BCD
+TAMIL CONSONANT T;  0BA4 0BCD
+TAMIL CONSONANT N;  0BA8 0BCD
+TAMIL CONSONANT P;  0BAA 0BCD
+TAMIL CONSONANT M;  0BAE 0BCD
+TAMIL CONSONANT Y;  0BAF 0BCD
+TAMIL CONSONANT R;  0BB0 0BCD
+TAMIL CONSONANT L;  0BB2 0BCD
+TAMIL CONSONANT V;  0BB5 0BCD
+TAMIL CONSONANT LLL;0BB4 0BCD
+TAMIL CONSONANT LL; 0BB3 0BCD
+TAMIL CONSONANT RR; 0BB1 0BCD
+TAMIL CONSONANT NNN;0BA9 0BCD
+TAMIL CONSONANT J;  0B9C 0BCD
+TAMIL CONSONANT SH; 0BB6 0BCD
+TAMIL CONSONANT SS; 0BB7 0BCD
+TAMIL CONSONANT S;  0BB8 0BCD
+TAMIL CONSONANT H;  0BB9 0BCD
+TAMIL CONSONANT KSS;0B95 0BCD 0BB7 0BCD
+TAMIL SYLLABLE KAA; 0B95 0BBE
+TAMIL SYLLABLE KI;  0B95 0BBF
+TAMIL SYLLABLE KII; 0B95 0BC0
+TAMIL SYLLABLE KU;  0B95 0BC1
+TAMIL SYLLABLE KUU; 0B95 0BC2
+TAMIL SYLLABLE KE;  0B95 0BC6
+TAMIL SYLLABLE KEE; 0B95 0BC7
+TAMIL SYLLABLE KAI; 0B95 0BC8
+TAMIL SYLLABLE KO;  0B95 0BCA
+TAMIL SYLLABLE KOO; 0B95 0BCB
+TAMIL SYLLABLE KAU; 0B95 0BCC
+TAMIL SYLLABLE NGAA; 0B99 0BBE
+TAMIL SYLLABLE NGI;  0B99 0BBF
+TAMIL SYLLABLE NGII; 0B99 0BC0
+TAMIL SYLLABLE NGU;  0B99 0BC1
+TAMIL SYLLABLE NGUU; 0B99 0BC2
+TAMIL SYLLABLE NGE;  0B99 0BC6
+TAMIL SYLLABLE NGEE; 0B99 0BC7
+TAMIL SYLLABLE NGAI; 0B99 0BC8
+TAMIL SYLLABLE NGO;  0B99 0BCA
+TAMIL SYLLABLE NGOO; 0B99 0BCB
+TAMIL SYLLABLE NGAU; 0B99 0BCC
+TAMIL SYLLABLE CI;  0B9A 0BBF
+TAMIL SYLLABLE CII; 0B9A 0BC0
+TAMIL SYLLABLE CU;  0B9A 0BC1
+TAMIL SYLLABLE CUU; 0B9A 0BC2
+TAMIL SYLLABLE CE;  0B9A 0BC6
+TAMIL SYLLABLE CEE; 0B9A 0BC7
+TAMIL SYLLABLE CAI; 0B9A 0BC8
+TAMIL SYLLABLE CO;  0B9A 0BCA
+TAMIL SYLLABLE COO; 0B9A 0BCB
+TAMIL SYLLABLE CAU; 0B9A 0BCC
+TAMIL SYLLABLE NYAA; 0B9E 0BBE
+TAMIL SYLLABLE NYI;  0B9E 0BBF
+TAMIL SYLLABLE NYII; 0B9E 0BC0
+TAMIL SYLLABLE NYU;  0B9E 0BC1
+TAMIL SYLLABLE NYUU; 0B9E 0BC2
+TAMIL SYLLABLE NYE;  0B9E 0BC6
+TAMIL SYLLABLE NYEE; 0B9E 0BC7
+TAMIL SYLLABLE NYAI; 0B9E 0BC8
+TAMIL SYLLABLE NYO;  0B9E 0BCA
+TAMIL SYLLABLE NYOO; 0B9E 0BCB
+TAMIL SYLLABLE NYAU; 0B9E 0BCC
+TAMIL SYLLABLE TTAA; 0B9F 0BBE
+TAMIL SYLLABLE TTI;  0B9F 0BBF
+TAMIL SYLLABLE TTII; 0B9F 0BC0
+TAMIL SYLLABLE TTU;  0B9F 0BC1
+TAMIL SYLLABLE TTUU; 0B9F 0BC2
+TAMIL SYLLABLE TTE;  0B9F 0BC6
+TAMIL SYLLABLE TTEE; 0B9F 0BC7
+TAMIL SYLLABLE TTAI; 0B9F 0BC8
+TAMIL SYLLABLE TTO;  0B9F 0BCA
+TAMIL SYLLABLE TTOO; 0B9F 0BCB
+TAMIL SYLLABLE TTAU; 0B9F 0BCC
+TAMIL SYLLABLE NNAA; 0BA3 0BBE
+TAMIL SYLLABLE NNI;  0BA3 0BBF
+TAMIL SYLLABLE NNII; 0BA3 0BC0
+TAMIL SYLLABLE NNU;  0BA3 0BC1
+TAMIL SYLLABLE NNUU; 0BA3 0BC2
+TAMIL SYLLABLE NNE;  0BA3 0BC6
+TAMIL SYLLABLE NNEE; 0BA3 0BC7
+TAMIL SYLLABLE NNAI; 0BA3 0BC8
+TAMIL SYLLABLE NNO;  0BA3 0BCA
+TAMIL SYLLABLE NNOO; 0BA3 0BCB
+TAMIL SYLLABLE NNAU; 0BA3 0BCC
+TAMIL SYLLABLE TAA; 0BA4 0BBE
+TAMIL SYLLABLE TI;  0BA4 0BBF
+TAMIL SYLLABLE TII; 0BA4 0BC0
+TAMIL SYLLABLE TU;  0BA4 0BC1
+TAMIL SYLLABLE TUU; 0BA4 0BC2
+TAMIL SYLLABLE TE;  0BA4 0BC6
+TAMIL SYLLABLE TEE; 0BA4 0BC7
+TAMIL SYLLABLE TAI; 0BA4 0BC8
+TAMIL SYLLABLE TO;  0BA4 0BCA
+TAMIL SYLLABLE TOO; 0BA4 0BCB
+TAMIL SYLLABLE TAU; 0BA4 0BCC
+TAMIL SYLLABLE NAA; 0BA8 0BBE
+TAMIL SYLLABLE NI;  0BA8 0BBF
+TAMIL SYLLABLE NII; 0BA8 0BC0
+TAMIL SYLLABLE NU;  0BA8 0BC1
+TAMIL SYLLABLE NUU; 0BA8 0BC2
+TAMIL SYLLABLE NE;  0BA8 0BC6
+TAMIL SYLLABLE NEE; 0BA8 0BC7
+TAMIL SYLLABLE NAI; 0BA8 0BC8
+TAMIL SYLLABLE NO;  0BA8 0BCA
+TAMIL SYLLABLE NOO; 0BA8 0BCB
+TAMIL SYLLABLE NAU; 0BA8 0BCC
+TAMIL SYLLABLE PAA; 0BAA 0BBE
+TAMIL SYLLABLE PI;  0BAA 0BBF
+TAMIL SYLLABLE PII; 0BAA 0BC0
+TAMIL SYLLABLE PU;  0BAA 0BC1
+TAMIL SYLLABLE PUU; 0BAA 0BC2
+TAMIL SYLLABLE PE;  0BAA 0BC6
+TAMIL SYLLABLE PEE; 0BAA 0BC7
+TAMIL SYLLABLE PAI; 0BAA 0BC8
+TAMIL SYLLABLE PO;  0BAA 0BCA
+TAMIL SYLLABLE POO; 0BAA 0BCB
+TAMIL SYLLABLE PAU; 0BAA 0BCC
+TAMIL SYLLABLE MAA; 0BAE 0BBE
+TAMIL SYLLABLE MI;  0BAE 0BBF
+TAMIL SYLLABLE MII; 0BAE 0BC0
+TAMIL SYLLABLE MU;  0BAE 0BC1
+TAMIL SYLLABLE MUU; 0BAE 0BC2
+TAMIL SYLLABLE ME;  0BAE 0BC6
+TAMIL SYLLABLE MEE; 0BAE 0BC7
+TAMIL SYLLABLE MAI; 0BAE 0BC8
+TAMIL SYLLABLE MO;  0BAE 0BCA
+TAMIL SYLLABLE MOO; 0BAE 0BCB
+TAMIL SYLLABLE MAU; 0BAE 0BCC
+TAMIL SYLLABLE YAA; 0BAF 0BBE
+TAMIL SYLLABLE YI;  0BAF 0BBF
+TAMIL SYLLABLE YII; 0BAF 0BC0
+TAMIL SYLLABLE YU;  0BAF 0BC1
+TAMIL SYLLABLE YUU; 0BAF 0BC2
+TAMIL SYLLABLE YE;  0BAF 0BC6
+TAMIL SYLLABLE YEE; 0BAF 0BC7
+TAMIL SYLLABLE YAI; 0BAF 0BC8
+TAMIL SYLLABLE YO;  0BAF 0BCA
+TAMIL SYLLABLE YOO; 0BAF 0BCB
+TAMIL SYLLABLE YAU; 0BAF 0BCC
+TAMIL SYLLABLE RAA; 0BB0 0BBE
+TAMIL SYLLABLE RI;  0BB0 0BBF
+TAMIL SYLLABLE RII; 0BB0 0BC0
+TAMIL SYLLABLE RU;  0BB0 0BC1
+TAMIL SYLLABLE RUU; 0BB0 0BC2
+TAMIL SYLLABLE RE;  0BB0 0BC6
+TAMIL SYLLABLE REE; 0BB0 0BC7
+TAMIL SYLLABLE RAI; 0BB0 0BC8
+TAMIL SYLLABLE RO;  0BB0 0BCA
+TAMIL SYLLABLE ROO; 0BB0 0BCB
+TAMIL SYLLABLE RAU; 0BB0 0BCC
+TAMIL SYLLABLE LAA; 0BB2 0BBE
+TAMIL SYLLABLE LI;  0BB2 0BBF
+TAMIL SYLLABLE LII; 0BB2 0BC0
+TAMIL SYLLABLE LU;  0BB2 0BC1
+TAMIL SYLLABLE LUU; 0BB2 0BC2
+TAMIL SYLLABLE LE;  0BB2 0BC6
+TAMIL SYLLABLE LEE; 0BB2 0BC7
+TAMIL SYLLABLE LAI; 0BB2 0BC8
+TAMIL SYLLABLE LO;  0BB2 0BCA
+TAMIL SYLLABLE LOO; 0BB2 0BCB
+TAMIL SYLLABLE LAU; 0BB2 0BCC
+TAMIL SYLLABLE VAA; 0BB5 0BBE
+TAMIL SYLLABLE VI;  0BB5 0BBF
+TAMIL SYLLABLE VII; 0BB5 0BC0
+TAMIL SYLLABLE VU;  0BB5 0BC1
+TAMIL SYLLABLE VUU; 0BB5 0BC2
+TAMIL SYLLABLE VE;  0BB5 0BC6
+TAMIL SYLLABLE VEE; 0BB5 0BC7
+TAMIL SYLLABLE VAI; 0BB5 0BC8
+TAMIL SYLLABLE VO;  0BB5 0BCA
+TAMIL SYLLABLE VOO; 0BB5 0BCB
+TAMIL SYLLABLE VAU; 0BB5 0BCC
+TAMIL SYLLABLE LLLAA; 0BB4 0BBE
+TAMIL SYLLABLE LLLI;  0BB4 0BBF
+TAMIL SYLLABLE LLLII; 0BB4 0BC0
+TAMIL SYLLABLE LLLU;  0BB4 0BC1
+TAMIL SYLLABLE LLLUU; 0BB4 0BC2
+TAMIL SYLLABLE LLLE;  0BB4 0BC6
+TAMIL SYLLABLE LLLEE; 0BB4 0BC7
+TAMIL SYLLABLE LLLAI; 0BB4 0BC8
+TAMIL SYLLABLE LLLO;  0BB4 0BCA
+TAMIL SYLLABLE LLLOO; 0BB4 0BCB
+TAMIL SYLLABLE LLLAU; 0BB4 0BCC
+TAMIL SYLLABLE LLAA; 0BB3 0BBE
+TAMIL SYLLABLE LLI;  0BB3 0BBF
+TAMIL SYLLABLE LLII; 0BB3 0BC0
+TAMIL SYLLABLE LLU;  0BB3 0BC1
+TAMIL SYLLABLE LLUU; 0BB3 0BC2
+TAMIL SYLLABLE LLE;  0BB3 0BC6
+TAMIL SYLLABLE LLEE; 0BB3 0BC7
+TAMIL SYLLABLE LLAI; 0BB3 0BC8
+TAMIL SYLLABLE LLO;  0BB3 0BCA
+TAMIL SYLLABLE LLOO; 0BB3 0BCB
+TAMIL SYLLABLE LLAU; 0BB3 0BCC
+TAMIL SYLLABLE RRAA; 0BB1 0BBE
+TAMIL SYLLABLE RRI;  0BB1 0BBF
+TAMIL SYLLABLE RRII; 0BB1 0BC0
+TAMIL SYLLABLE RRU;  0BB1 0BC1
+TAMIL SYLLABLE RRUU; 0BB1 0BC2
+TAMIL SYLLABLE RRE;  0BB1 0BC6
+TAMIL SYLLABLE RREE; 0BB1 0BC7
+TAMIL SYLLABLE RRAI; 0BB1 0BC8
+TAMIL SYLLABLE RRO;  0BB1 0BCA
+TAMIL SYLLABLE RROO; 0BB1 0BCB
+TAMIL SYLLABLE RRAU; 0BB1 0BCC
+TAMIL SYLLABLE NNNAA; 0BA9 0BBE
+TAMIL SYLLABLE NNNI;  0BA9 0BBF
+TAMIL SYLLABLE NNNII; 0BA9 0BC0
+TAMIL SYLLABLE NNNU;  0BA9 0BC1
+TAMIL SYLLABLE NNNUU; 0BA9 0BC2
+TAMIL SYLLABLE NNNE;  0BA9 0BC6
+TAMIL SYLLABLE NNNEE; 0BA9 0BC7
+TAMIL SYLLABLE NNNAI; 0BA9 0BC8
+TAMIL SYLLABLE NNNO;  0BA9 0BCA
+TAMIL SYLLABLE NNNOO; 0BA9 0BCB
+TAMIL SYLLABLE NNNAU; 0BA9 0BCC
+TAMIL SYLLABLE JAA; 0B9C 0BBE
+TAMIL SYLLABLE JI;  0B9C 0BBF
+TAMIL SYLLABLE JII; 0B9C 0BC0
+TAMIL SYLLABLE JU;  0B9C 0BC1
+TAMIL SYLLABLE JUU; 0B9C 0BC2
+TAMIL SYLLABLE JE;  0B9C 0BC6
+TAMIL SYLLABLE JEE; 0B9C 0BC7
+TAMIL SYLLABLE JAI; 0B9C 0BC8
+TAMIL SYLLABLE JO;  0B9C 0BCA
+TAMIL SYLLABLE JOO; 0B9C 0BCB
+TAMIL SYLLABLE JAU; 0B9C 0BCC
+TAMIL SYLLABLE SHAA; 0BB6 0BBE
+TAMIL SYLLABLE SHI;  0BB6 0BBF
+TAMIL SYLLABLE SHII; 0BB6 0BC0
+TAMIL SYLLABLE SHU;  0BB6 0BC1
+TAMIL SYLLABLE SHUU; 0BB6 0BC2
+TAMIL SYLLABLE SHE;  0BB6 0BC6
+TAMIL SYLLABLE SHEE; 0BB6 0BC7
+TAMIL SYLLABLE SHAI; 0BB6 0BC8
+TAMIL SYLLABLE SHO;  0BB6 0BCA
+TAMIL SYLLABLE SHOO; 0BB6 0BCB
+TAMIL SYLLABLE SHAU; 0BB6 0BCC
+TAMIL SYLLABLE SSAA; 0BB7 0BBE
+TAMIL SYLLABLE SSI;  0BB7 0BBF
+TAMIL SYLLABLE SSII; 0BB7 0BC0
+TAMIL SYLLABLE SSU;  0BB7 0BC1
+TAMIL SYLLABLE SSUU; 0BB7 0BC2
+TAMIL SYLLABLE SSE;  0BB7 0BC6
+TAMIL SYLLABLE SSEE; 0BB7 0BC7
+TAMIL SYLLABLE SSAI; 0BB7 0BC8
+TAMIL SYLLABLE SSO;  0BB7 0BCA
+TAMIL SYLLABLE SSOO; 0BB7 0BCB
+TAMIL SYLLABLE SSAU; 0BB7 0BCC
+TAMIL SYLLABLE SAA; 0BB8 0BBE
+TAMIL SYLLABLE SI;  0BB8 0BBF
+TAMIL SYLLABLE SII; 0BB8 0BC0
+TAMIL SYLLABLE SU;  0BB8 0BC1
+TAMIL SYLLABLE SUU; 0BB8 0BC2
+TAMIL SYLLABLE SE;  0BB8 0BC6
+TAMIL SYLLABLE SEE; 0BB8 0BC7
+TAMIL SYLLABLE SAI; 0BB8 0BC8
+TAMIL SYLLABLE SO;  0BB8 0BCA
+TAMIL SYLLABLE SOO; 0BB8 0BCB
+TAMIL SYLLABLE SAU; 0BB8 0BCC
+TAMIL SYLLABLE HAA; 0BB9 0BBE
+TAMIL SYLLABLE HI;  0BB9 0BBF
+TAMIL SYLLABLE HII; 0BB9 0BC0
+TAMIL SYLLABLE HU;  0BB9 0BC1
+TAMIL SYLLABLE HUU; 0BB9 0BC2
+TAMIL SYLLABLE HE;  0BB9 0BC6
+TAMIL SYLLABLE HEE; 0BB9 0BC7
+TAMIL SYLLABLE HAI; 0BB9 0BC8
+TAMIL SYLLABLE HO;  0BB9 0BCA
+TAMIL SYLLABLE HOO; 0BB9 0BCB
+TAMIL SYLLABLE HAU; 0BB9 0BCC
+TAMIL SYLLABLE KSSA;  0B95 0BCD 0BB7
+TAMIL SYLLABLE KSSAA; 0B95 0BCD 0BB7 0BBE
+TAMIL SYLLABLE KSSI;  0B95 0BCD 0BB7 0BBF
+TAMIL SYLLABLE KSSII; 0B95 0BCD 0BB7 0BC0
+TAMIL SYLLABLE KSSU;  0B95 0BCD 0BB7 0BC1
+TAMIL SYLLABLE KSSUU; 0B95 0BCD 0BB7 0BC2
+TAMIL SYLLABLE KSSE;  0B95 0BCD 0BB7 0BC6
+TAMIL SYLLABLE KSSEE; 0B95 0BCD 0BB7 0BC7
+TAMIL SYLLABLE KSSAI; 0B95 0BCD 0BB7 0BC8
+TAMIL SYLLABLE KSSO;  0B95 0BCD 0BB7 0BCA
+TAMIL SYLLABLE KSSOO; 0B95 0BCD 0BB7 0BCB
+TAMIL SYLLABLE KSSAU; 0B95 0BCD 0BB7 0BCC
+TAMIL SYLLABLE SHRII; 0BB6 0BCD 0BB0 0BC0
+SINHALA CONSONANT SIGN YANSAYA;0DCA 200D 0DBA
+SINHALA CONSONANT SIGN RAKAARAANSAYA;0DCA 200D 0DBB
+SINHALA CONSONANT SIGN REPAYA;0DBB 0DCA 200D
+GEORGIAN LETTER U-BRJGU;10E3 0302
+KHMER CONSONANT SIGN COENG KA;17D2 1780
+KHMER CONSONANT SIGN COENG KHA;17D2 1781
+KHMER CONSONANT SIGN COENG KO;17D2 1782
+KHMER CONSONANT SIGN COENG KHO;17D2 1783
+KHMER CONSONANT SIGN COENG NGO;17D2 1784
+KHMER CONSONANT SIGN COENG CA;17D2 1785
+KHMER CONSONANT SIGN COENG CHA;17D2 1786
+KHMER CONSONANT SIGN COENG CO;17D2 1787
+KHMER CONSONANT SIGN COENG CHO;17D2 1788
+KHMER CONSONANT SIGN COENG NYO;17D2 1789
+KHMER CONSONANT SIGN COENG DA;17D2 178A
+KHMER CONSONANT SIGN COENG TTHA;17D2 178B
+KHMER CONSONANT SIGN COENG DO;17D2 178C
+KHMER CONSONANT SIGN COENG TTHO;17D2 178D
+KHMER CONSONANT SIGN COENG NA;17D2 178E
+KHMER CONSONANT SIGN COENG TA;17D2 178F
+KHMER CONSONANT SIGN COENG THA;17D2 1790
+KHMER CONSONANT SIGN COENG TO;17D2 1791
+KHMER CONSONANT SIGN COENG THO;17D2 1792
+KHMER CONSONANT SIGN COENG NO;17D2 1793
+KHMER CONSONANT SIGN COENG BA;17D2 1794
+KHMER CONSONANT SIGN COENG PHA;17D2 1795
+KHMER CONSONANT SIGN COENG PO;17D2 1796
+KHMER CONSONANT SIGN COENG PHO;17D2 1797
+KHMER CONSONANT SIGN COENG MO;17D2 1798
+KHMER CONSONANT SIGN COENG YO;17D2 1799
+KHMER CONSONANT SIGN COENG RO;17D2 179A
+KHMER CONSONANT SIGN COENG LO;17D2 179B
+KHMER CONSONANT SIGN COENG VO;17D2 179C
+KHMER CONSONANT SIGN COENG SHA;17D2 179D
+KHMER CONSONANT SIGN COENG SSA;17D2 179E
+KHMER CONSONANT SIGN COENG SA;17D2 179F
+KHMER CONSONANT SIGN COENG HA;17D2 17A0
+KHMER CONSONANT SIGN COENG LA;17D2 17A1
+KHMER VOWEL SIGN COENG QA;17D2 17A2
+KHMER INDEPENDENT VOWEL SIGN COENG QU;17D2 17A7
+KHMER INDEPENDENT VOWEL SIGN COENG RY;17D2 17AB
+KHMER INDEPENDENT VOWEL SIGN COENG RYY;17D2 17AC
+KHMER INDEPENDENT VOWEL SIGN COENG QE;17D2 17AF
+KHMER VOWEL SIGN OM;17BB 17C6
+KHMER VOWEL SIGN AAM;17B6 17C6
+HIRAGANA LETTER BIDAKUON NGA;304B 309A
+HIRAGANA LETTER BIDAKUON NGI;304D 309A
+HIRAGANA LETTER BIDAKUON NGU;304F 309A
+HIRAGANA LETTER BIDAKUON NGE;3051 309A
+HIRAGANA LETTER BIDAKUON NGO;3053 309A
+KATAKANA LETTER BIDAKUON NGA;30AB 309A
+KATAKANA LETTER BIDAKUON NGI;30AD 309A
+KATAKANA LETTER BIDAKUON NGU;30AF 309A
+KATAKANA LETTER BIDAKUON NGE;30B1 309A
+KATAKANA LETTER BIDAKUON NGO;30B3 309A
+KATAKANA LETTER AINU CE;30BB 309A
+KATAKANA LETTER AINU TU;30C4 309A
+KATAKANA LETTER AINU TO;30C8 309A
+KATAKANA LETTER AINU P;31F7 309A
+MODIFIER LETTER EXTRA-HIGH EXTRA-LOW CONTOUR TONE BAR;02E5 02E9
+MODIFIER LETTER EXTRA-LOW EXTRA-HIGH CONTOUR TONE BAR;02E9 02E5

lib/elixir/priv/SpecialCasing.txt

@@ -0,0 +1,103 @@
+00DF; 00DF; 0053 0073; 0053 0053; # LATIN SMALL LETTER SHARP S
+0130; 0069 0307; 0130; 0130; # LATIN CAPITAL LETTER I WITH DOT ABOVE
+FB00; FB00; 0046 0066; 0046 0046; # LATIN SMALL LIGATURE FF
+FB01; FB01; 0046 0069; 0046 0049; # LATIN SMALL LIGATURE FI
+FB02; FB02; 0046 006C; 0046 004C; # LATIN SMALL LIGATURE FL
+FB03; FB03; 0046 0066 0069; 0046 0046 0049; # LATIN SMALL LIGATURE FFI
+FB04; FB04; 0046 0066 006C; 0046 0046 004C; # LATIN SMALL LIGATURE FFL
+FB05; FB05; 0053 0074; 0053 0054; # LATIN SMALL LIGATURE LONG S T
+FB06; FB06; 0053 0074; 0053 0054; # LATIN SMALL LIGATURE ST
+0587; 0587; 0535 0582; 0535 0552; # ARMENIAN SMALL LIGATURE ECH YIWN
+FB13; FB13; 0544 0576; 0544 0546; # ARMENIAN SMALL LIGATURE MEN NOW
+FB14; FB14; 0544 0565; 0544 0535; # ARMENIAN SMALL LIGATURE MEN ECH
+FB15; FB15; 0544 056B; 0544 053B; # ARMENIAN SMALL LIGATURE MEN INI
+FB16; FB16; 054E 0576; 054E 0546; # ARMENIAN SMALL LIGATURE VEW NOW
+FB17; FB17; 0544 056D; 0544 053D; # ARMENIAN SMALL LIGATURE MEN XEH
+0149; 0149; 02BC 004E; 02BC 004E; # LATIN SMALL LETTER N PRECEDED BY APOSTROPHE
+0390; 0390; 0399 0308 0301; 0399 0308 0301; # GREEK SMALL LETTER IOTA WITH DIALYTIKA AND TONOS
+03B0; 03B0; 03A5 0308 0301; 03A5 0308 0301; # GREEK SMALL LETTER UPSILON WITH DIALYTIKA AND TONOS
+01F0; 01F0; 004A 030C; 004A 030C; # LATIN SMALL LETTER J WITH CARON
+1E96; 1E96; 0048 0331; 0048 0331; # LATIN SMALL LETTER H WITH LINE BELOW
+1E97; 1E97; 0054 0308; 0054 0308; # LATIN SMALL LETTER T WITH DIAERESIS
+1E98; 1E98; 0057 030A; 0057 030A; # LATIN SMALL LETTER W WITH RING ABOVE
+1E99; 1E99; 0059 030A; 0059 030A; # LATIN SMALL LETTER Y WITH RING ABOVE
+1E9A; 1E9A; 0041 02BE; 0041 02BE; # LATIN SMALL LETTER A WITH RIGHT HALF RING
+1F50; 1F50; 03A5 0313; 03A5 0313; # GREEK SMALL LETTER UPSILON WITH PSILI
+1F52; 1F52; 03A5 0313 0300; 03A5 0313 0300; # GREEK SMALL LETTER UPSILON WITH PSILI AND VARIA
+1F54; 1F54; 03A5 0313 0301; 03A5 0313 0301; # GREEK SMALL LETTER UPSILON WITH PSILI AND OXIA
+1F56; 1F56; 03A5 0313 0342; 03A5 0313 0342; # GREEK SMALL LETTER UPSILON WITH PSILI AND PERISPOMENI
+1FB6; 1FB6; 0391 0342; 0391 0342; # GREEK SMALL LETTER ALPHA WITH PERISPOMENI
+1FC6; 1FC6; 0397 0342; 0397 0342; # GREEK SMALL LETTER ETA WITH PERISPOMENI
+1FD2; 1FD2; 0399 0308 0300; 0399 0308 0300; # GREEK SMALL LETTER IOTA WITH DIALYTIKA AND VARIA
+1FD3; 1FD3; 0399 0308 0301; 0399 0308 0301; # GREEK SMALL LETTER IOTA WITH DIALYTIKA AND OXIA
+1FD6; 1FD6; 0399 0342; 0399 0342; # GREEK SMALL LETTER IOTA WITH PERISPOMENI
+1FD7; 1FD7; 0399 0308 0342; 0399 0308 0342; # GREEK SMALL LETTER IOTA WITH DIALYTIKA AND PERISPOMENI
+1FE2; 1FE2; 03A5 0308 0300; 03A5 0308 0300; # GREEK SMALL LETTER UPSILON WITH DIALYTIKA AND VARIA
+1FE3; 1FE3; 03A5 0308 0301; 03A5 0308 0301; # GREEK SMALL LETTER UPSILON WITH DIALYTIKA AND OXIA
+1FE4; 1FE4; 03A1 0313; 03A1 0313; # GREEK SMALL LETTER RHO WITH PSILI
+1FE6; 1FE6; 03A5 0342; 03A5 0342; # GREEK SMALL LETTER UPSILON WITH PERISPOMENI
+1FE7; 1FE7; 03A5 0308 0342; 03A5 0308 0342; # GREEK SMALL LETTER UPSILON WITH DIALYTIKA AND PERISPOMENI
+1FF6; 1FF6; 03A9 0342; 03A9 0342; # GREEK SMALL LETTER OMEGA WITH PERISPOMENI
+1F80; 1F80; 1F88; 1F08 0399; # GREEK SMALL LETTER ALPHA WITH PSILI AND YPOGEGRAMMENI
+1F81; 1F81; 1F89; 1F09 0399; # GREEK SMALL LETTER ALPHA WITH DASIA AND YPOGEGRAMMENI
+1F82; 1F82; 1F8A; 1F0A 0399; # GREEK SMALL LETTER ALPHA WITH PSILI AND VARIA AND YPOGEGRAMMENI
+1F83; 1F83; 1F8B; 1F0B 0399; # GREEK SMALL LETTER ALPHA WITH DASIA AND VARIA AND YPOGEGRAMMENI
+1F84; 1F84; 1F8C; 1F0C 0399; # GREEK SMALL LETTER ALPHA WITH PSILI AND OXIA AND YPOGEGRAMMENI
+1F85; 1F85; 1F8D; 1F0D 0399; # GREEK SMALL LETTER ALPHA WITH DASIA AND OXIA AND YPOGEGRAMMENI
+1F86; 1F86; 1F8E; 1F0E 0399; # GREEK SMALL LETTER ALPHA WITH PSILI AND PERISPOMENI AND YPOGEGRAMMENI
+1F87; 1F87; 1F8F; 1F0F 0399; # GREEK SMALL LETTER ALPHA WITH DASIA AND PERISPOMENI AND YPOGEGRAMMENI
+1F88; 1F80; 1F88; 1F08 0399; # GREEK CAPITAL LETTER ALPHA WITH PSILI AND PROSGEGRAMMENI
+1F89; 1F81; 1F89; 1F09 0399; # GREEK CAPITAL LETTER ALPHA WITH DASIA AND PROSGEGRAMMENI
+1F8A; 1F82; 1F8A; 1F0A 0399; # GREEK CAPITAL LETTER ALPHA WITH PSILI AND VARIA AND PROSGEGRAMMENI
+1F8B; 1F83; 1F8B; 1F0B 0399; # GREEK CAPITAL LETTER ALPHA WITH DASIA AND VARIA AND PROSGEGRAMMENI
+1F8C; 1F84; 1F8C; 1F0C 0399; # GREEK CAPITAL LETTER ALPHA WITH PSILI AND OXIA AND PROSGEGRAMMENI
+1F8D; 1F85; 1F8D; 1F0D 0399; # GREEK CAPITAL LETTER ALPHA WITH DASIA AND OXIA AND PROSGEGRAMMENI
+1F8E; 1F86; 1F8E; 1F0E 0399; # GREEK CAPITAL LETTER ALPHA WITH PSILI AND PERISPOMENI AND PROSGEGRAMMENI
+1F8F; 1F87; 1F8F; 1F0F 0399; # GREEK CAPITAL LETTER ALPHA WITH DASIA AND PERISPOMENI AND PROSGEGRAMMENI
+1F90; 1F90; 1F98; 1F28 0399; # GREEK SMALL LETTER ETA WITH PSILI AND YPOGEGRAMMENI
+1F91; 1F91; 1F99; 1F29 0399; # GREEK SMALL LETTER ETA WITH DASIA AND YPOGEGRAMMENI
+1F92; 1F92; 1F9A; 1F2A 0399; # GREEK SMALL LETTER ETA WITH PSILI AND VARIA AND YPOGEGRAMMENI
+1F93; 1F93; 1F9B; 1F2B 0399; # GREEK SMALL LETTER ETA WITH DASIA AND VARIA AND YPOGEGRAMMENI
+1F94; 1F94; 1F9C; 1F2C 0399; # GREEK SMALL LETTER ETA WITH PSILI AND OXIA AND YPOGEGRAMMENI
+1F95; 1F95; 1F9D; 1F2D 0399; # GREEK SMALL LETTER ETA WITH DASIA AND OXIA AND YPOGEGRAMMENI
+1F96; 1F96; 1F9E; 1F2E 0399; # GREEK SMALL LETTER ETA WITH PSILI AND PERISPOMENI AND YPOGEGRAMMENI
+1F97; 1F97; 1F9F; 1F2F 0399; # GREEK SMALL LETTER ETA WITH DASIA AND PERISPOMENI AND YPOGEGRAMMENI
+1F98; 1F90; 1F98; 1F28 0399; # GREEK CAPITAL LETTER ETA WITH PSILI AND PROSGEGRAMMENI
+1F99; 1F91; 1F99; 1F29 0399; # GREEK CAPITAL LETTER ETA WITH DASIA AND PROSGEGRAMMENI
+1F9A; 1F92; 1F9A; 1F2A 0399; # GREEK CAPITAL LETTER ETA WITH PSILI AND VARIA AND PROSGEGRAMMENI
+1F9B; 1F93; 1F9B; 1F2B 0399; # GREEK CAPITAL LETTER ETA WITH DASIA AND VARIA AND PROSGEGRAMMENI
+1F9C; 1F94; 1F9C; 1F2C 0399; # GREEK CAPITAL LETTER ETA WITH PSILI AND OXIA AND PROSGEGRAMMENI
+1F9D; 1F95; 1F9D; 1F2D 0399; # GREEK CAPITAL LETTER ETA WITH DASIA AND OXIA AND PROSGEGRAMMENI
+1F9E; 1F96; 1F9E; 1F2E 0399; # GREEK CAPITAL LETTER ETA WITH PSILI AND PERISPOMENI AND PROSGEGRAMMENI
+1F9F; 1F97; 1F9F; 1F2F 0399; # GREEK CAPITAL LETTER ETA WITH DASIA AND PERISPOMENI AND PROSGEGRAMMENI
+1FA0; 1FA0; 1FA8; 1F68 0399; # GREEK SMALL LETTER OMEGA WITH PSILI AND YPOGEGRAMMENI
+1FA1; 1FA1; 1FA9; 1F69 0399; # GREEK SMALL LETTER OMEGA WITH DASIA AND YPOGEGRAMMENI
+1FA2; 1FA2; 1FAA; 1F6A 0399; # GREEK SMALL LETTER OMEGA WITH PSILI AND VARIA AND YPOGEGRAMMENI
+1FA3; 1FA3; 1FAB; 1F6B 0399; # GREEK SMALL LETTER OMEGA WITH DASIA AND VARIA AND YPOGEGRAMMENI
+1FA4; 1FA4; 1FAC; 1F6C 0399; # GREEK SMALL LETTER OMEGA WITH PSILI AND OXIA AND YPOGEGRAMMENI
+1FA5; 1FA5; 1FAD; 1F6D 0399; # GREEK SMALL LETTER OMEGA WITH DASIA AND OXIA AND YPOGEGRAMMENI
+1FA6; 1FA6; 1FAE; 1F6E 0399; # GREEK SMALL LETTER OMEGA WITH PSILI AND PERISPOMENI AND YPOGEGRAMMENI
+1FA7; 1FA7; 1FAF; 1F6F 0399; # GREEK SMALL LETTER OMEGA WITH DASIA AND PERISPOMENI AND YPOGEGRAMMENI
+1FA8; 1FA0; 1FA8; 1F68 0399; # GREEK CAPITAL LETTER OMEGA WITH PSILI AND PROSGEGRAMMENI
+1FA9; 1FA1; 1FA9; 1F69 0399; # GREEK CAPITAL LETTER OMEGA WITH DASIA AND PROSGEGRAMMENI
+1FAA; 1FA2; 1FAA; 1F6A 0399; # GREEK CAPITAL LETTER OMEGA WITH PSILI AND VARIA AND PROSGEGRAMMENI
+1FAB; 1FA3; 1FAB; 1F6B 0399; # GREEK CAPITAL LETTER OMEGA WITH DASIA AND VARIA AND PROSGEGRAMMENI
+1FAC; 1FA4; 1FAC; 1F6C 0399; # GREEK CAPITAL LETTER OMEGA WITH PSILI AND OXIA AND PROSGEGRAMMENI
+1FAD; 1FA5; 1FAD; 1F6D 0399; # GREEK CAPITAL LETTER OMEGA WITH DASIA AND OXIA AND PROSGEGRAMMENI
+1FAE; 1FA6; 1FAE; 1F6E 0399; # GREEK CAPITAL LETTER OMEGA WITH PSILI AND PERISPOMENI AND PROSGEGRAMMENI
+1FAF; 1FA7; 1FAF; 1F6F 0399; # GREEK CAPITAL LETTER OMEGA WITH DASIA AND PERISPOMENI AND PROSGEGRAMMENI
+1FB3; 1FB3; 1FBC; 0391 0399; # GREEK SMALL LETTER ALPHA WITH YPOGEGRAMMENI
+1FBC; 1FB3; 1FBC; 0391 0399; # GREEK CAPITAL LETTER ALPHA WITH PROSGEGRAMMENI
+1FC3; 1FC3; 1FCC; 0397 0399; # GREEK SMALL LETTER ETA WITH YPOGEGRAMMENI
+1FCC; 1FC3; 1FCC; 0397 0399; # GREEK CAPITAL LETTER ETA WITH PROSGEGRAMMENI
+1FF3; 1FF3; 1FFC; 03A9 0399; # GREEK SMALL LETTER OMEGA WITH YPOGEGRAMMENI
+1FFC; 1FF3; 1FFC; 03A9 0399; # GREEK CAPITAL LETTER OMEGA WITH PROSGEGRAMMENI
+1FB2; 1FB2; 1FBA 0345; 1FBA 0399; # GREEK SMALL LETTER ALPHA WITH VARIA AND YPOGEGRAMMENI
+1FB4; 1FB4; 0386 0345; 0386 0399; # GREEK SMALL LETTER ALPHA WITH OXIA AND YPOGEGRAMMENI
+1FC2; 1FC2; 1FCA 0345; 1FCA 0399; # GREEK SMALL LETTER ETA WITH VARIA AND YPOGEGRAMMENI
+1FC4; 1FC4; 0389 0345; 0389 0399; # GREEK SMALL LETTER ETA WITH OXIA AND YPOGEGRAMMENI
+1FF2; 1FF2; 1FFA 0345; 1FFA 0399; # GREEK SMALL LETTER OMEGA WITH VARIA AND YPOGEGRAMMENI
+1FF4; 1FF4; 038F 0345; 038F 0399; # GREEK SMALL LETTER OMEGA WITH OXIA AND YPOGEGRAMMENI
+1FB7; 1FB7; 0391 0342 0345; 0391 0342 0399; # GREEK SMALL LETTER ALPHA WITH PERISPOMENI AND YPOGEGRAMMENI
+1FC7; 1FC7; 0397 0342 0345; 0397 0342 0399; # GREEK SMALL LETTER ETA WITH PERISPOMENI AND YPOGEGRAMMENI
+1FF7; 1FF7; 03A9 0342 0345; 03A9 0342 0399; # GREEK SMALL LETTER OMEGA WITH PERISPOMENI AND YPOGEGRAMMENI

lib/elixir/priv/unicode.ex

@@ -0,0 +1,216 @@
+# This file has its own compilation step because
+# it needs to parse String.Unicode data and
+# compile a digested module.
+defmodule String.Unicode do
+  @moduledoc false
+
+  def version, do: {6,2,0}
+
+  to_binary = fn
+    "" ->
+      nil
+    codepoints ->
+      codepoints = :binary.split(codepoints, " ", [:global])
+      Enum.reduce codepoints, "", fn(codepoint, acc) ->
+        acc <> << binary_to_integer(codepoint, 16) :: utf8 >>
+      end
+  end
+
+  data_path = Path.expand("../UnicodeData.txt", __FILE__)
+
+  { codes, whitespace } = Enum.reduce File.iterator!(data_path), { [], [] }, fn(line, { cacc, wacc }) ->
+    [ codepoint, _name, _category,
+      _class, bidi, _decomposition,
+      _numeric_1, _numeric_2, _numeric_3,
+      _bidi_mirror, _unicode_1, _iso,
+      upper, lower, title ] = :binary.split(line, ";", [:global])
+
+    title = :binary.part(title, 0, size(title) - 1)
+
+    cond do
+      upper != "" or lower != "" or title != "" ->
+        { [{ to_binary.(codepoint), to_binary.(upper), to_binary.(lower), to_binary.(title) } | cacc], wacc }
+      bidi in ["B", "S", "WS"] ->
+        { cacc, [to_binary.(codepoint) | wacc] }
+      true ->
+        { cacc, wacc }
+    end
+  end
+
+  special_path = Path.expand("../SpecialCasing.txt", __FILE__)
+
+  codes = Enum.reduce File.iterator!(special_path), codes, fn(line, acc) ->
+    [ codepoint, lower, title, upper, _comment ] = :binary.split(line, "; ", [:global])
+    key = to_binary.(codepoint)
+    :lists.keystore(key, 1, acc, { key, to_binary.(upper), to_binary.(lower), to_binary.(title) })
+  end
+
+  seqs_path = Path.expand("../NamedSequences.txt", __FILE__)
+
+  seqs = Enum.map File.iterator!(seqs_path), fn(line) ->
+    [ _name, codepoints ] = :binary.split(line, ";", [:global])
+    codepoints = :binary.split(codepoints, " ", [:global])
+    codepoints = Enum.map codepoints, Regex.replace(%r/\s+/, &1, "")
+    codepoints = Enum.filter codepoints, fn(x) -> size(x) > 0 end
+    Enum.map codepoints, to_binary.(&1)
+  end
+
+  # Downcase
+
+  def downcase(""), do: ""
+
+  lc { codepoint, _upper, lower, _title } inlist codes, lower && lower != codepoint do
+    def downcase(unquote(codepoint) <> rest) do
+      unquote(lower) <> downcase(rest)
+    end
+  end
+
+  def downcase(<< char, rest :: binary >>) do
+    << char >> <> downcase(rest)
+  end
+
+  # Upcase
+
+  def upcase(""), do: ""
+
+  lc { codepoint, upper, _lower, _title } inlist codes, upper && upper != codepoint do
+    def upcase(unquote(codepoint) <> rest) do
+      unquote(upper) <> upcase(rest)
+    end
+  end
+
+  def upcase(<< char, rest :: binary >>) do
+    << char >> <> upcase(rest)
+  end
+
+  # Titlecase once
+
+  def titlecase_once(""), do: { "", "" }
+
+  lc { codepoint, _upper, _lower, title } inlist codes, title && title != codepoint do
+    def titlecase_once(unquote(codepoint) <> rest) do
+      { unquote(title), rest }
+    end
+  end
+
+  def titlecase_once(<< char, rest :: binary >>) do
+    { << char >>, rest }
+  end
+
+  # Strip
+
+  def lstrip(""), do: ""
+
+  lc codepoint inlist whitespace do
+    def lstrip(unquote(codepoint) <> rest) do
+      lstrip(rest)
+    end
+  end
+
+  def lstrip(other) when is_binary(other), do: other
+
+  def rstrip(""), do: ""
+
+  def rstrip(string) when is_binary(string) do
+    do_rstrip(string, "")
+  end
+
+  lc codepoint inlist whitespace do
+    defp do_rstrip(unquote(codepoint) <> rest, buffer) do
+      do_rstrip(rest, unquote(codepoint) <> buffer)
+    end
+  end
+
+  defp do_rstrip(<< char, rest :: binary >>, buffer) do
+    << buffer :: binary, char, do_rstrip(rest, "") :: binary >>
+  end
+
+  defp do_rstrip(<<>>, _), do: <<>>
+
+  # Split
+
+  def split(""), do: ""
+
+  def split(string) when is_binary(string) do
+    :lists.reverse do_split(string, "", [])
+  end
+
+  lc codepoint inlist whitespace do
+    defp do_split(unquote(codepoint) <> rest, buffer, acc) do
+      if buffer != "" do
+        do_split(rest, "", [buffer | acc])
+      else
+        do_split(rest, buffer, acc)
+      end
+    end
+  end
+
+  defp do_split(<< char, rest :: binary >>, buffer, acc) do
+    do_split(rest, << buffer :: binary, char >>, acc)
+  end
+
+  defp do_split(<<>>, buffer, acc) do
+    if buffer != "" do
+      [buffer | acc]
+    else
+      acc
+    end
+  end
+
+  # Graphemes
+
+  lc codepoints inlist seqs do
+    def next_grapheme(<< unquote_splicing(codepoints), t :: binary >>) do
+      { << unquote_splicing(codepoints) >>, t }
+    end
+  end
+
+  def next_grapheme(<<>>) do
+    :no_grapheme
+  end
+
+  def next_grapheme(binary) when is_binary(binary) do
+    case next_codepoint(binary) do
+      :no_codepoint -> :no_grapheme
+      other -> other
+    end
+  end
+
+  def graphemes(binary) when is_binary(binary) do
+    do_graphemes(next_grapheme(binary))
+  end
+
+  defp do_graphemes({ c, rest }) do
+    [c|do_graphemes(next_grapheme(rest))]
+  end
+
+  defp do_graphemes(:no_grapheme) do
+    []
+  end
+
+  # Codepoints
+
+  def next_codepoint(<< cp :: utf8, rest :: binary >>) do
+    { <<cp :: utf8>>, rest }
+  end
+
+  def next_codepoint(<< cp, rest :: binary >>) do
+    { <<cp>>, rest }
+  end
+
+  def next_codepoint(<<>>) do
+    :no_codepoint
+  end
+
+  def codepoints(binary) when is_binary(binary) do
+    do_codepoints(next_codepoint(binary))
+  end
+
+  defp do_codepoints({ c, rest }) do
+    [c|do_codepoints(next_codepoint(rest))]
+  end
+
+  defp do_codepoints(:no_codepoint) do
+    []
+  end
+end

lib/elixir/rebar.config

@@ -16,7 +16,7 @@
             %% warn_missing_spec,
             %% warn_untyped_record,
             %% warnings_as_errors,
-            no_debug_info
+            debug_info
            ]}.
 
 {yrl_opts, [
@@ -24,4 +24,4 @@
             {verbose, false}
            ]}.
 
-{require_otp_vsn,"R15"}.
+{require_otp_vsn,"(R15|R16).*"}.

lib/elixir/src/elixir.erl

@@ -1,21 +1,28 @@
 -module(elixir).
 -behaviour(application).
--export([main/1, start_cli/0, start_app/0,
+-export([main/1, start_cli/0,
   scope_for_eval/1, eval/2, eval/3, eval/4,
   eval_quoted/2, eval_quoted/3, eval_quoted/4,
-  eval_forms/3]).
+  eval_forms/3, translate_forms/3]).
 -include("elixir.hrl").
 -compile({parse_transform, elixir_transform}).
 
+%% Top level types
+-export_type([char_list/0, as_boolean/1]).
+-type char_list() :: string().
+-type as_boolean(T) :: T.
 
 % OTP APPLICATION API
 
 -export([start/2, stop/1, config_change/3]).
 
 start(_Type, _Args) ->
-  %% Set the shell to unicode so printing inside files work
-  io:setopts(standard_io, [{encoding,unicode}]),
-  io:setopts(standard_error, [{encoding,unicode}]),
+  %% Set the shell to unicode so printing inside scripts work
+  %% Those can take a while, so let's do it in a new process
+  spawn(fun() ->
+    io:setopts(standard_io, [{encoding,unicode}]),
+    io:setopts(standard_error, [{encoding,unicode}])
+  end),
   elixir_sup:start_link([]).
 
 stop(_S) ->
@@ -27,40 +34,52 @@ config_change(_Changed, _New, _Remove) ->
 %% escript entry point
 
 main(Args) ->
-  start_app(),
-  'Elixir.Kernel.CLI':process_argv(Args).
+  application:start(?MODULE),
+  'Elixir.Kernel.CLI':main(Args).
 
-%% ELIXIR ENTRY POINTS
-
-% Start the Elixir app. This is the proper way to boot Elixir from
-% inside an Erlang process.
-
-start_app() ->
-  case lists:keyfind(?MODULE, 1, application:loaded_applications()) of
-    false -> application:start(?MODULE);
-    _ -> ok
-  end.
-
-% Boot and process given options. Invoked by Elixir's script.
+%% Boot and process given options. Invoked by Elixir's script.
 
 start_cli() ->
-  start_app(),
-  'Elixir.Kernel.CLI':process_argv(init:get_plain_arguments()).
+  application:start(?MODULE),
+  'Elixir.Kernel.CLI':main(init:get_plain_arguments()).
 
 %% EVAL HOOKS
 
 scope_for_eval(Opts) ->
-  File = case lists:keyfind(file, 1, Opts) of
-    { file, F } -> to_binary(F);
-    false -> <<"nofile">>
+  case lists:keyfind(file, 1, Opts) of
+    { file, RawFile } -> File = to_binary(RawFile);
+    false -> File = <<"nofile">>
   end,
 
-  Local = case lists:keyfind(delegate_locals_to, 1, Opts) of
-    { delegate_locals_to, L } -> L;
-    false -> nil
+  case lists:keyfind(delegate_locals_to, 1, Opts) of
+    { delegate_locals_to, Local } -> Local;
+    false -> Local = nil
   end,
 
-  #elixir_scope{file=File,local=Local}.
+  case lists:keyfind(aliases, 1, Opts) of
+    { aliases, Aliases } -> Aliases;
+    false -> Aliases = []
+  end,
+
+  case lists:keyfind(requires, 1, Opts) of
+    { requires, List } -> Requires = ordsets:from_list(List);
+    false -> Requires = elixir_dispatch:default_requires()
+  end,
+
+  case lists:keyfind(functions, 1, Opts) of
+    { functions, Functions } -> Functions;
+    false -> Functions = elixir_dispatch:default_functions()
+  end,
+
+  case lists:keyfind(macros, 1, Opts) of
+    { macros, Macros } -> Macros;
+    false -> Macros = elixir_dispatch:default_macros()
+  end,
+
+  #elixir_scope{
+    file=File, local=Local,
+    macros=Macros, functions=Functions,
+    requires=Requires, aliases=Aliases }.
 
 %% String evaluation
 
@@ -89,21 +108,25 @@ eval_quoted(Tree, Binding, Opts) ->
   end,
   eval_quoted(Tree, Binding, Line, scope_for_eval(Opts)).
 
-eval_quoted(Tree, Binding, Line, #elixir_scope{} = S) ->
+eval_quoted(Tree, Binding, Line, #elixir_scope{} = S) when is_integer(Line) ->
   eval_forms(elixir_quote:linify(Line, Tree), Binding, S).
 
 %% Handle forms evaluation internally, it is an
 %% internal API not meant for external usage.
 
-eval_forms(Tree, Binding, RawScope) ->
-  Scope = RawScope#elixir_scope{
+translate_forms(Tree, Binding, Opts) when is_list(Opts) ->
+  translate_forms(Tree, Binding, scope_for_eval(Opts));
+
+translate_forms(Tree, Binding, #elixir_scope{} = Scope) ->
+  elixir_translator:translate(Tree, Scope#elixir_scope{
     vars=binding_dict(Binding),
     temp_vars=[],
-    quote_vars=[],
-    clause_vars=[],
-    counter=0
-  },
-  { ParseTree, NewScope } = elixir_translator:translate(Tree, Scope),
+    clause_vars=nil,
+    counter=[]
+  }).
+
+eval_forms(Tree, Binding, Scope) ->
+  { ParseTree, NewScope } = translate_forms(Tree, Binding, Scope),
   case ParseTree of
     [] -> { nil, Binding, NewScope };
     _  ->
@@ -117,15 +140,17 @@ to_binary(Bin)  when is_binary(Bin) -> Bin;
 to_binary(List) when is_list(List) -> list_to_binary(List).
 
 binding_dict(List) -> binding_dict(List, orddict:new()).
-binding_dict([{H,_}|T], Dict) -> binding_dict(T, orddict:store(H, H, Dict));
+binding_dict([{{H,Kind},_}|T], Dict) -> binding_dict(T, orddict:store({ H, Kind }, H, Dict));
+binding_dict([{H,_}|T], Dict) -> binding_dict(T, orddict:store({ H, nil }, H, Dict));
 binding_dict([], Dict) -> Dict.
 
 final_binding(Binding, Vars) -> final_binding(Binding, [], Binding, Vars).
 final_binding([{Var,_}|T], Acc, Binding, Vars) ->
-  case atom_to_list(Var) of
-    "_@" ++ _ -> final_binding(T, Acc, Binding, Vars);
-    _ ->
-      RealName = orddict:fetch(Var, Vars),
+  case lists:member($@, atom_to_list(Var)) of
+    true  ->
+      final_binding(T, Acc, Binding, Vars);
+    false ->
+      RealName  = orddict:fetch({ Var, nil }, Vars),
       RealValue = proplists:get_value(RealName, Binding, nil),
       final_binding(T, [{Var, RealValue}|Acc], Binding, Vars)
   end;

lib/elixir/src/elixir_aliases.erl

@@ -1,10 +1,54 @@
 -module(elixir_aliases).
--export([first/1, last/1, concat/1, safe_concat/1, lookup/2,
-  format_error/1, ensure_loaded/3]).
+-export([nesting_alias/2, last/1, concat/1, safe_concat/1,
+  format_error/1, ensure_loaded/3, expand/3]).
 -include("elixir.hrl").
 -compile({parse_transform, elixir_transform}).
 
+%% Expand an alias. It returns an atom (meaning that there
+%% was an expansion) or a list of atoms.
+
+expand({ '__aliases__', Meta, _ } = Alias, Aliases, MacroAliases) ->
+  case lists:keyfind(alias, 1, Meta) of
+    { alias, false } ->
+      expand(Alias, MacroAliases);
+    { alias, Atom } when is_atom(Atom) ->
+      case expand(Alias, MacroAliases) of
+        OtherAtom when is_atom(OtherAtom) -> OtherAtom;
+        OtherAliases when is_list(OtherAliases) -> Atom
+      end;
+    false ->
+      expand(Alias, Aliases)
+  end.
+
+expand({ '__aliases__', _Meta, [H] }, Aliases) when H /= 'Elixir' ->
+  case expand_one(H, Aliases) of
+    false -> [H];
+    Atom  -> Atom
+  end;
+
+expand({ '__aliases__', _Meta, [H|T] }, Aliases) when is_atom(H) ->
+  case H of
+    'Elixir' ->
+      concat(T);
+    _ ->
+      case expand_one(H, Aliases) of
+        false -> [H|T];
+        Atom  -> concat([Atom|T])
+      end
+  end;
+
+expand({ '__aliases__', _Meta, List }, _Aliases) ->
+  List.
+
+expand_one(H, Aliases) ->
+  Lookup = list_to_atom("Elixir-" ++ atom_to_list(H)),
+  case lookup(Lookup, Aliases) of
+    Lookup -> false;
+    Else   -> Else
+  end.
+
 %% Ensure a module is loaded before its usage.
+
 ensure_loaded(_Line, 'Elixir.Kernel', _S) ->
   ok;
 
@@ -20,18 +64,7 @@ ensure_loaded(Line, Ref, S) ->
       elixir_errors:form_error(Line, S#elixir_scope.file, ?MODULE, { Kind, Ref })
   end.
 
-%% Receives an atom and returns the first alias.
-
-first(Atom) ->
-  First = first(atom_to_list(Atom), []),
-  list_to_atom("Elixir-" ++ First).
-
-first("Elixir-" ++ Rest, []) -> first(Rest, []);
-first([$-|_], Acc) -> lists:reverse(Acc);
-first([H|T], Acc) -> first(T, [H|Acc]);
-first([], Acc) -> lists:reverse(Acc).
-
-%% Receives an atom and returns the last alias.
+%% Receives an atom and returns the last bit as an alias.
 
 last(Atom) ->
   Last = last(lists:reverse(atom_to_list(Atom)), []),
@@ -41,8 +74,43 @@ last([$-|_], Acc) -> Acc;
 last([H|T], Acc) -> last(T, [H|Acc]);
 last([], Acc) -> Acc.
 
-%% Receives a list of atoms representing modules
-%% and concatenate them.
+%% Gets two modules names and return an alias
+%% which can be passed down to the alias directive
+%% and it will create a proper shortcut representing
+%% the given nesting.
+%%
+%% Examples:
+%%
+%%     nesting_alias('Elixir.Foo.Bar', 'Elixir.Foo.Bar.Baz.Bat')
+%%     { 'Elixir.Baz', 'Elixir.Foo.Bar.Baz' }
+%%
+%% When passed to alias, the example above will generate an
+%% alias like:
+%%
+%%     'Elixir.Baz' => 'Elixir.Foo.Bar.Baz'
+%%
+nesting_alias(nil, _Full) -> false;
+
+nesting_alias(Prefix, Full) ->
+  PrefixList = list_nesting(Prefix),
+  FullList   = list_nesting(Full),
+  (PrefixList /= []) andalso do_nesting(PrefixList, FullList, []).
+
+do_nesting([X|PreTail], [X|Tail], Acc) ->
+  do_nesting(PreTail, Tail, [X|Acc]);
+do_nesting([], [H|_], Acc) ->
+  { list_to_atom("Elixir-" ++ H), concat(lists:reverse([H|Acc])) };
+do_nesting(_, _, _Acc) ->
+  false.
+
+list_nesting(Atom) ->
+  case string:tokens(atom_to_list(Atom), "-") of
+    ["Elixir"|T] -> T;
+    _ -> []
+  end.
+
+%% Receives a list of atoms, binaries or lists
+%% representing modules and concatenates them.
 
 concat(Args) -> list_to_atom(raw_concat(Args)).
 safe_concat(Args) -> list_to_existing_atom(raw_concat(Args)).
@@ -66,7 +134,7 @@ dot_to_dash(List) ->
     _  -> X
    end || X <- List].
 
-%% Lookup an alias in the current scope
+%% Lookup an alias in the current scope.
 
 lookup(Else, Dict) ->
   case orddict:find(Else, Dict) of
@@ -77,8 +145,8 @@ lookup(Else, Dict) ->
 %% Errors
 
 format_error({unloaded_module, Module}) ->
-  io_lib:format("module ~s is not loaded and could not be found", [elixir_errors:inspect(Module)]);
+  io_lib:format("module ~ts is not loaded and could not be found", [elixir_errors:inspect(Module)]);
 
 format_error({scheduled_module, Module}) ->
-  io_lib:format("module ~s is not loaded but was defined. This happens because you are trying to use a module in the same context it is defined. Try defining the module outside the context that requires it.",
+  io_lib:format("module ~ts is not loaded but was defined. This happens because you are trying to use a module in the same context it is defined. Try defining the module outside the context that requires it.",
     [elixir_errors:inspect(Module)]).

lib/elixir/src/elixir_clauses.erl

@@ -9,25 +9,25 @@
 
 %% Get pairs from a clause.
 
-get_pairs(Line, Key, Clauses, S) ->
-  get_pairs(Line, Key, Clauses, S, false).
+get_pairs(Meta, Key, Clauses, S) ->
+  get_pairs(Meta, Key, Clauses, S, false).
 
-get_pairs(Line, Key, Clauses, S, AllowNil) ->
+get_pairs(Meta, Key, Clauses, S, AllowNil) ->
   case lists:keyfind(Key, 1, Clauses) of
     { Key, { '->', _, Pairs } } ->
       [{ Key, Left, Right } || { Left, Right } <- Pairs];
     { Key, nil } when AllowNil ->
       [];
     { Key, _ } ->
-      elixir_errors:syntax_error(Line, S#elixir_scope.file, "expected pairs with -> for key ~s", [Key]);
+      elixir_errors:syntax_error(Meta, S#elixir_scope.file, "expected pairs with -> for key ~ts", [Key]);
     _ ->
       []
   end.
 
 % Function for translating assigns.
 
-assigns(Fun, Args, #elixir_scope{context=Context} = S) when Context /= assign ->
-  { Result, NewS } = assigns(Fun, Args, S#elixir_scope{context=assign, temp_vars=orddict:new()}),
+assigns(Fun, Args, #elixir_scope{context=Context} = S) when Context /= match ->
+  { Result, NewS } = assigns(Fun, Args, S#elixir_scope{context=match, temp_vars=orddict:new()}),
   { Result, NewS#elixir_scope{context=Context} };
 
 assigns(Fun, Args, S) -> Fun(Args, S).
@@ -39,7 +39,7 @@ assigns_block(Line, Fun, BareArgs, Exprs, S) ->
   { Args, Guards } = extract_guards(BareArgs),
   assigns_block(Line, Fun, Args, Exprs, Guards, S).
 
-assigns_block(Line, Fun, Args, Exprs, Guards, S) ->
+assigns_block(Line, Fun, Args, Exprs, Guards, S) when is_integer(Line) ->
   { TArgs, SA }  = assigns(Fun, Args, S#elixir_scope{extra_guards=[]}),
   { TExprs, SE } = elixir_translator:translate(Exprs, SA#elixir_scope{extra_guards=nil}),
 
@@ -75,7 +75,8 @@ extract_or_clauses(Term, Acc) -> [Term|Acc].
 
 extract_args({ { '.', _, [Name] }, _, Args }) when is_atom(Name), is_list(Args) -> { Name, Args };
 extract_args({ Name, _, Args }) when is_atom(Name), is_atom(Args) -> { Name, [] };
-extract_args({ Name, _, Args }) when is_atom(Name), is_list(Args) -> { Name, Args }.
+extract_args({ Name, _, Args }) when is_atom(Name), is_list(Args) -> { Name, Args };
+extract_args(_) -> error.
 
 % Extract guards when it is in the last element of the args
 
@@ -87,91 +88,98 @@ extract_last_guards(Args) ->
 
 % Function for translating macros with match style like case and receive.
 
-match(Line, DecoupledClauses, RawS) ->
-  S = RawS#elixir_scope{clause_vars=orddict:new()},
+match(Meta, Clauses, #elixir_scope{clause_vars=C1} = S) ->
+  { TC, TS } = do_match(Meta, Clauses, S#elixir_scope{clause_vars=orddict:new()}),
+  C2 = TS#elixir_scope.clause_vars,
+  { TC, TS#elixir_scope{clause_vars=elixir_scope:merge_clause_vars(C1, C2)} }.
 
-  case DecoupledClauses of
-    [DecoupledClause] ->
-      { TDecoupledClause, TS } = each_clause(Line, DecoupledClause, S),
-      { [TDecoupledClause], TS };
-    _ ->
-      % Transform tree just passing the variables counter forward
-      % and storing variables defined inside each clause.
-      Transformer = fun(X, {Acc, CV}) ->
-        { TX, TAcc } = each_clause(Line, X, Acc),
-        { TX, { umergec(S, TAcc), [TAcc#elixir_scope.clause_vars|CV] } }
-      end,
+do_match(_Meta, [], S) ->
+  { [], S };
 
-      { TClauses, { TS, RawCV } } = lists:mapfoldl(Transformer, {S, []}, DecoupledClauses),
+do_match(Meta, [DecoupledClause], S) ->
+  { TDecoupledClause, TS } = each_clause(Meta, DecoupledClause, S),
+  { [TDecoupledClause], TS };
 
-      % Now get all the variables defined inside each clause
-      CV = lists:reverse(RawCV),
-      NewVars = lists:umerge([lists:sort(orddict:fetch_keys(X)) || X <- CV]),
-
-      case NewVars of
-        [] -> { TClauses, TS };
-        _  ->
-          % Create a new scope that contains a list of all variables
-          % defined inside all the clauses. It returns this new scope and
-          % a list of tuples where the first element is the variable name,
-          % the second one is the new pointer to the variable and the third
-          % is the old pointer.
-          { FinalVars, FS } = lists:mapfoldl(fun normalize_vars/2, TS, NewVars),
-
-          % Defines a tuple that will be used as left side of the match operator
-          LeftVars = [{var, Line, NewValue} || { _, _, NewValue, _ } <- FinalVars],
-
-          % Expand all clauses by adding a match operation at the end
-          % that assigns variables missing in one clause to the others.
-          expand_clauses(Line, TClauses, CV, LeftVars, FinalVars, [], FS)
-      end
-  end.
-
-expand_clauses(Line, [Clause|T], [ClauseVars|V], LeftVars, FinalVars, Acc, S) ->
-  RightVars = [normalize_clause_var(Var, Kind, OldValue, ClauseVars) || { Var, Kind, _, OldValue } <- FinalVars],
-
-  AssignExpr = generate_match(Line, LeftVars, RightVars),
-  ClauseExprs = element(5, Clause),
-  [Final|RawClauseExprs] = lists:reverse(ClauseExprs),
-
-  % If the last sentence has a match clause, we need to assign its value
-  % in the variable list. If not, we insert the variable list before the
-  % final clause in order to keep it tail call optimized.
-  { FinalClauseExprs, FS } = case has_match_tuple(Final) of
-    true ->
-      case Final of
-        { match, _, { var, _, UserVarName } = UserVar, _ } when UserVarName /= '_' ->
-          { [UserVar,AssignExpr,Final|RawClauseExprs], S };
-        _ ->
-          { StorageVar, SS } = elixir_scope:build_erl_var(Line, S),
-          StorageExpr = { match, Line, StorageVar, Final },
-          { [StorageVar,AssignExpr,StorageExpr|RawClauseExprs], SS }
-      end;
-    false ->
-      { [Final,AssignExpr|RawClauseExprs], S }
+do_match(Meta, DecoupledClauses, S) ->
+  % Transform tree just passing the variables counter forward
+  % and storing variables defined inside each clause.
+  Transformer = fun(X, {Acc, CV}) ->
+    { TX, TAcc } = each_clause(Meta, X, Acc),
+    { TX, { umergec(S, TAcc), [TAcc#elixir_scope.clause_vars|CV] } }
   end,
 
-  FinalClause = setelement(5, Clause, lists:reverse(FinalClauseExprs)),
-  expand_clauses(Line, T, V, LeftVars, FinalVars, [FinalClause|Acc], FS);
+  { TClauses, { TS, ReverseCV } } = lists:mapfoldl(Transformer, {S, []}, DecoupledClauses),
 
-expand_clauses(_Line, [], [], _LeftVars, _FinalVars, Acc, S) ->
+  % Now get all the variables defined inside each clause
+  CV = lists:reverse(ReverseCV),
+  AllVars = lists:foldl(fun(KV, Acc) ->
+    elixir_scope:merge_clause_vars(Acc, KV)
+  end, orddict:new(), CV),
+
+  % Create a new scope that contains a list of all variables
+  % defined inside all the clauses. It returns this new scope and
+  % a list of tuples where the first element is the variable name,
+  % the second one is the new pointer to the variable and the third
+  % is the old pointer.
+  { FinalVars, FS } = lists:mapfoldl(fun({ Key, Ref }, Acc) ->
+    normalize_vars(Key, Ref, Acc)
+  end, TS, AllVars),
+
+  % Expand all clauses by adding a match operation at the end
+  % that assigns variables missing in one clause to the others.
+  expand_clauses(?line(Meta), TClauses, CV, FinalVars, [], FS).
+
+expand_clauses(Line, [Clause|T], [ClauseVars|V], FinalVars, Acc, S) ->
+  case generate_match_vars(FinalVars, ClauseVars, [], []) of
+    { [], [] } ->
+      expand_clauses(Line, T, V, FinalVars, [Clause|Acc], S);
+    { Left, Right } ->
+      MatchExpr   = generate_match(Line, Left, Right),
+      ClauseExprs = element(5, Clause),
+      [Final|RawClauseExprs] = lists:reverse(ClauseExprs),
+
+      % If the last sentence has a match clause, we need to assign its value
+      % in the variable list. If not, we insert the variable list before the
+      % final clause in order to keep it tail call optimized.
+      { FinalClauseExprs, FS } = case has_match_tuple(Final) of
+        true ->
+          case Final of
+            { match, _, { var, _, UserVarName } = UserVar, _ } when UserVarName /= '_' ->
+              { [UserVar,MatchExpr,Final|RawClauseExprs], S };
+            _ ->
+              { StorageVar, SS } = elixir_scope:build_erl_var(Line, S),
+              StorageExpr = { match, Line, StorageVar, Final },
+              { [StorageVar,MatchExpr,StorageExpr|RawClauseExprs], SS }
+          end;
+        false ->
+          { [Final,MatchExpr|RawClauseExprs], S }
+      end,
+
+      FinalClause = setelement(5, Clause, lists:reverse(FinalClauseExprs)),
+      expand_clauses(Line, T, V, FinalVars, [FinalClause|Acc], FS)
+  end;
+
+expand_clauses(_Line, [], [], _FinalVars, Acc, S) ->
   { lists:reverse(Acc), S }.
 
 % Handle each key/value clause pair and translate them accordingly.
 
-each_clause(Line, { do, [Condition], Expr }, S) ->
-  assigns_block(Line, fun elixir_translator:translate_each/2, Condition, [Expr], S);
+each_clause(Meta, { do, [Condition], Expr }, S) ->
+  assigns_block(?line(Meta), fun elixir_translator:translate_each/2, Condition, [Expr], S);
 
-each_clause(Line, { 'after', [Condition], Expr }, S) ->
+each_clause(Meta, { else, [Condition], Expr }, S) ->
+  assigns_block(?line(Meta), fun elixir_translator:translate_each/2, Condition, [Expr], S);
+
+each_clause(Meta, { 'after', [Condition], Expr }, S) ->
   { TCondition, SC } = elixir_translator:translate_each(Condition, S),
   { TBody, SB } = elixir_translator:translate([Expr], SC),
-  { { clause, Line, [TCondition], [], TBody }, SB };
+  { { clause, ?line(Meta), [TCondition], [], TBody }, SB };
 
-each_clause(Line, { Key, [_|_], _ }, S) when Key == do; Key == 'after' ->
-  elixir_errors:syntax_error(Line, S#elixir_scope.file, "too many arguments given for ~s", [Key]);
+each_clause(Meta, { Key, [_|_], _ }, S) when Key == do; Key == 'after' ->
+  elixir_errors:syntax_error(Meta, S#elixir_scope.file, "too many arguments given for ~ts", [Key]);
 
-each_clause(Line, { Key, _, _ }, S) ->
-  elixir_errors:syntax_error(Line, S#elixir_scope.file, "invalid key ~s", [Key]).
+each_clause(Meta, { Key, _, _ }, S) ->
+  elixir_errors:syntax_error(Meta, S#elixir_scope.file, "invalid key ~ts", [Key]).
 
 % Check if the given expression is a match tuple.
 % This is a small optimization to allow us to change
@@ -197,38 +205,38 @@ has_match_tuple(H) when is_list(H) ->
 
 has_match_tuple(_) -> false.
 
-% Normalize the given var checking its existence in the scope var dictionary.
+% Normalize the given var in between clauses
+% by picking one value as reference and retriving
+% its previous value.
 
-normalize_vars({ Var, quoted }, S) ->
-  normalize_vars(Var, quoted, #elixir_scope.quote_vars, S);
+normalize_vars(Key, Value, #elixir_scope{vars=Vars,clause_vars=ClauseVars} = S) ->
+  FS = S#elixir_scope{
+    vars=orddict:store(Key, Value, Vars),
+    clause_vars=orddict:store(Key, Value, ClauseVars)
+  },
 
-normalize_vars({ Var, Kind }, S) ->
-  normalize_vars(Var, Kind, #elixir_scope.vars, S).
-
-normalize_vars(Var, Kind, Index, #elixir_scope{clause_vars=ClauseVars} = S) ->
-  Vars = element(Index, S),
-  { { _, _, NewValue }, S1 } = elixir_scope:build_erl_var(0, S),
-
-  S2 = setelement(Index, S1, orddict:store(Var, NewValue, Vars)),
-  S3 = S2#elixir_scope{clause_vars=orddict:store({ Var, Kind }, NewValue, ClauseVars)},
-
-  Expr = case orddict:find(Var, Vars) of
+  Expr = case orddict:find(Key, Vars) of
     { ok, OldValue } -> { var, 0, OldValue };
     error -> { atom, 0, nil }
   end,
 
-  { { Var, Kind, NewValue, Expr }, S3 }.
+  { { Key, Value, Expr }, FS }.
 
-% Normalize a var by checking if it was defined in the clause.
-% If so, use it, otherwise use from main scope.
+% Generate match vars by checking if they were updated
+% or not and assigning the previous value.
 
-normalize_clause_var(Var, Kind, OldValue, ClauseVars) ->
-  case orddict:find({ Var, Kind }, ClauseVars) of
-    { ok, ClauseValue } -> { var, 0, ClauseValue };
-    error -> OldValue
-  end.
+generate_match_vars([{ Key, NewValue, OldValue }|T], ClauseVars, Left, Right) ->
+  case orddict:find(Key, ClauseVars) of
+    { ok, NewValue } ->
+      generate_match_vars(T, ClauseVars, Left, Right);
+    { ok, ClauseValue } ->
+      generate_match_vars(T, ClauseVars, [{ var, 0, NewValue }|Left], [{ var, 0, ClauseValue }|Right]);
+    error ->
+      generate_match_vars(T, ClauseVars, [{ var, 0, NewValue }|Left], [OldValue|Right])
+  end;
 
-%% generate_match
+generate_match_vars([], _ClauseVars, Left, Right) ->
+  { Left, Right }.
 
 generate_match(Line, [Left], [Right]) ->
   { match, Line, Left, Right };
@@ -236,7 +244,5 @@ generate_match(Line, [Left], [Right]) ->
 generate_match(Line, LeftVars, RightVars) ->
   { match, Line, { tuple, Line, LeftVars }, { tuple, Line, RightVars } }.
 
-%% Listify
-
 listify(Expr) when not is_list(Expr) -> [Expr];
 listify(Expr) -> Expr.

lib/elixir/src/elixir_code_server.erl

@@ -1,4 +1,5 @@
 -module(elixir_code_server).
+-export([call/1, cast/1]).
 -export([start_link/0, init/1, handle_call/3, handle_cast/2,
   handle_info/2, terminate/2, code_change/3]).
 -behavior(gen_server).
@@ -6,14 +7,27 @@
   argv=[],
   loaded=[],
   at_exit=[],
-  compiler_options=[{docs,true}]
+  pool=[],
+  counter=0,
+  compiler_options=[{docs,true},{debug_info,true}],
+  waiting=[]
 }).
 
+call(Args) ->
+  gen_server:call(?MODULE, Args, get_timeout()).
+
+cast(Args) ->
+  gen_server:cast(?MODULE, Args).
+
+get_timeout() ->
+  30000.
+
+%% Callbacks
+
 start_link() ->
-  { ok, _ } = gen_server:start_link({local, elixir_code_server}, ?MODULE, [], []).
+  { ok, _ } = gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
 
 init(_args) ->
-  process_flag(trap_exit, true),
   { ok, #elixir_code_server{} }.
 
 handle_call({ acquire, Path }, From, Config) ->
@@ -29,31 +43,43 @@ handle_call({ acquire, Path }, From, Config) ->
       { reply, proceed, Config#elixir_code_server{loaded=Queued} }
   end;
 
-handle_call({ at_exit, AtExit }, _From, Config) ->
-  { reply, ok, Config#elixir_code_server{at_exit=[AtExit|Config#elixir_code_server.at_exit]} };
-
-handle_call({ argv, Argv }, _From, Config) ->
-  { reply, ok, Config#elixir_code_server{argv=Argv} };
-
-handle_call({ compiler_options, Options }, _From, Config) ->
-  Final = orddict:merge(fun(_,_,V) -> V end, Config#elixir_code_server.compiler_options, Options),
-  { reply, ok, Config#elixir_code_server{compiler_options=Final} };
-
 handle_call(loaded, _From, Config) ->
   { reply, [F || { F, true } <- Config#elixir_code_server.loaded], Config };
 
 handle_call(at_exit, _From, Config) ->
   { reply, Config#elixir_code_server.at_exit, Config };
 
+handle_call(flush_at_exit, _From, Config) ->
+  { reply, Config#elixir_code_server.at_exit, Config#elixir_code_server{at_exit=[]} };
+
 handle_call(argv, _From, Config) ->
   { reply, Config#elixir_code_server.argv, Config };
 
 handle_call(compiler_options, _From, Config) ->
   { reply, Config#elixir_code_server.compiler_options, Config };
 
+handle_call(retrieve_module_name, _From, Config) ->
+  case Config#elixir_code_server.pool of
+    [H|T] ->
+      { reply, module_tuple(H), Config#elixir_code_server{pool=T} };
+    [] ->
+      Counter = Config#elixir_code_server.counter,
+      { reply, module_tuple(Counter), Config#elixir_code_server{counter=Counter+1} }
+  end;
+
 handle_call(_Request, _From, Config) ->
   { reply, undef, Config }.
 
+handle_cast({ at_exit, AtExit }, Config) ->
+  { noreply, Config#elixir_code_server{at_exit=[AtExit|Config#elixir_code_server.at_exit]} };
+
+handle_cast({ argv, Argv }, Config) ->
+  { noreply, Config#elixir_code_server{argv=Argv} };
+
+handle_cast({ compiler_options, Options }, Config) ->
+  Final = orddict:merge(fun(_,_,V) -> V end, Config#elixir_code_server.compiler_options, Options),
+  { noreply, Config#elixir_code_server{compiler_options=Final} };
+
 handle_cast({ loaded, Path }, Config) ->
   Current = Config#elixir_code_server.loaded,
   case orddict:find(Path, Current) of
@@ -73,16 +99,19 @@ handle_cast({ unload_files, Files }, Config) ->
   Unloaded = lists:foldl(fun(File, Acc) -> orddict:erase(File, Acc) end, Current, Files),
   { noreply, Config#elixir_code_server{loaded=Unloaded} };
 
+handle_cast({ return_module_name, I }, #elixir_code_server{pool=Pool} = Config) ->
+  { noreply, Config#elixir_code_server{pool=[I|Pool]} };
+
 handle_cast(_Request, Config) ->
   { noreply, Config }.
 
 handle_info(_Request, Config) ->
   { noreply, Config }.
 
-terminate(Reason, Config) ->
-  io:format("[FATAL] ~p crashed:\n~p~n", [?MODULE, Reason]),
-  io:format("[FATAL] ~p snapshot:\n~p~n", [?MODULE, Config]),
+terminate(_Reason, _Config) ->
   ok.
 
 code_change(_Old, Config, _Extra) ->
-  { ok, Config }.
+  { ok, Config }.
+
+module_tuple(I) -> { list_to_atom("elixir_compiler_" ++ integer_to_list(I)), I }.

lib/elixir/src/elixir_compiler.erl

@@ -1,6 +1,6 @@
 -module(elixir_compiler).
 -export([get_opts/0, get_opt/1, get_opt/2, string/2, file/1, file_to_path/2]).
--export([core/0, module/3, eval_forms/5]).
+-export([core/0, module/3, eval_forms/4, format_error/1]).
 -include("elixir.hrl").
 -compile({parse_transform, elixir_transform}).
 
@@ -17,17 +17,20 @@ get_opt(Key, Dict) ->
   end.
 
 get_opts() ->
-  gen_server:call(elixir_code_server, compiler_options).
+  elixir_code_server:call(compiler_options).
 
 %% Compiles the given string.
 
 string(Contents, File) when is_list(Contents), is_binary(File) ->
+  Forms = elixir_translator:'forms!'(Contents, 1, File, []),
+  quoted(Forms, File).
+
+quoted(Forms, File) when is_binary(File) ->
   Previous = get(elixir_compiled),
 
   try
     put(elixir_compiled, []),
-    Forms = elixir_translator:'forms!'(Contents, 1, File, []),
-    eval_forms(Forms, 1, File, [], #elixir_scope{file=File}),
+    eval_forms(Forms, 1, [], elixir:scope_for_eval([{file,File}])),
     lists:reverse(get(elixir_compiled))
   after
     put(elixir_compiled, Previous)
@@ -49,34 +52,31 @@ file_to_path(File, Path) when is_binary(File), is_binary(Path) ->
 
 %% Evaluates the contents/forms by compiling them to an Erlang module.
 
-eval_forms(Forms, Line, Module, Vars, #elixir_scope{module=nil} = S) ->
-  eval_forms(Forms, Line, Module, nil, Vars, S);
-
-eval_forms(Forms, Line, Module, Vars, #elixir_scope{module=Value} = S) ->
-  eval_forms(Forms, Line, Module, Value, Vars, S).
-
-eval_forms(Forms, Line, RawModule, Value, Vars, S) ->
-  case (Value == nil) andalso allows_fast_compilation(Forms) of
-    true  -> eval_compilation(Forms, Vars, S);
-    false -> code_loading_compilation(Forms, Line, RawModule, Value, Vars, S)
-  end.
-
-eval_compilation(Forms, Vars, S) ->
-  Binding = [{ Var, Value } || { _, Var, Value } <- Vars],
-  { Result, _Binding, FS } = elixir:eval_forms(Forms, [{'_@MODULE',nil}|Binding], S),
-  { Result, FS }.
-
-code_loading_compilation(Forms, Line, RawModule, Value, Vars, S) ->
-  Module        = escape_module(RawModule),
+eval_forms(Forms, Line, Vars, S) ->
+  { Module, I } = retrieve_module_name(),
   { Exprs, FS } = elixir_translator:translate(Forms, S),
-  ModuleForm    = module_form(Exprs, Line, S#elixir_scope.file, Module, Vars),
 
-  Args = [X || { _, _, X } <- Vars],
+  Fun  = module_form_fun(S#elixir_scope.module),
+  Form = module_form(Fun, Exprs, Line, S#elixir_scope.file, Module, Vars),
+  Args = [X || { _, _, _, X } <- Vars],
 
-  { module(ModuleForm, S#elixir_scope.file, [], true, fun(Mod, _) ->
-    Res = Mod:'BOOTSTRAP'(Value, Args),
-    code:purge(Module),
+  %% Pass { native, false } to speed up bootstrap
+  %% process when native is set to true
+  { module(Form, S#elixir_scope.file, [{native,false}], true, fun(_, Binary) ->
+    Res = Module:Fun(S#elixir_scope.module, Args),
     code:delete(Module),
+
+    %% If we have labeled locals, anonymous functions
+    %% were created and therefore we cannot ditch the
+    %% module
+    case beam_lib:chunks(Binary, [labeled_locals]) of
+      { ok, { _, [{ labeled_locals, []}] } } ->
+        code:purge(Module),
+        return_module_name(I);
+      _ ->
+        ok
+    end,
+
     Res
   end), FS }.
 
@@ -92,11 +92,17 @@ module(Forms, S, Callback) ->
   end,
   module(Forms, S#elixir_scope.file, Options, false, Callback).
 
-module(Forms, File, Options, Bootstrap, Callback) when
-    is_binary(File), is_list(Forms), is_list(Options), is_boolean(Bootstrap), is_function(Callback) ->
+module(Forms, File, RawOptions, Bootstrap, Callback) when
+    is_binary(File), is_list(Forms), is_list(RawOptions), is_boolean(Bootstrap), is_function(Callback) ->
+  { Options, SkipNative } = compile_opts(Forms, RawOptions),
   Listname = binary_to_list(File),
-  case compile:forms([no_auto_import()|Forms], [return,{source,Listname}|Options]) of
-    {ok, ModuleName, Binary, Warnings} ->
+
+  case compile:noenv_forms([no_auto_import()|Forms], [return,{source,Listname}|Options]) of
+    {ok, ModuleName, Binary, RawWarnings} ->
+      Warnings = case SkipNative of
+        true  -> [{?MODULE,[{0,?MODULE,{skip_native,ModuleName}}]}|RawWarnings];
+        false -> RawWarnings
+      end,
       format_warnings(Bootstrap, File, Warnings),
       code:load_binary(ModuleName, Listname, Binary),
       Callback(ModuleName, Binary);
@@ -109,23 +115,54 @@ module(Forms, File, Options, Bootstrap, Callback) when
 %% Invoked from the Makefile.
 
 core() ->
-  elixir:start_app(),
-  gen_server:call(elixir_code_server, { compiler_options, [{docs,false},{internal,true}] }),
-  [core_file(File) || File <- core_main()],
-  AllLists = [filelib:wildcard(Wildcard) || Wildcard <- core_list()],
-  Files = lists:append(AllLists) -- core_main(),
-  [core_file(File) || File <- 'Elixir.List':uniq(Files)].
+  application:start(elixir),
+  elixir_code_server:cast({ compiler_options, [{docs,false},{internal,true}] }),
+  [core_file(File) || File <- core_main()].
 
 %% HELPERS
 
+compile_opts(Forms, Options) ->
+  EnvOptions = env_default_opts(),
+  SkipNative = lists:member(native, EnvOptions) and contains_on_load(Forms),
+  case SkipNative or lists:member([{native,false}], Options) of
+    true  -> { Options ++ lists:delete(native, EnvOptions), SkipNative };
+    false -> { Options ++ EnvOptions, SkipNative }
+  end.
+
+env_default_opts() ->
+  Key = "ERL_COMPILER_OPTIONS",
+  case os:getenv(Key) of
+    false -> [];
+    Str when is_list(Str) ->
+      case erl_scan:string(Str) of
+        {ok,Tokens,_} ->
+          case erl_parse:parse_term(Tokens ++ [{dot, 1}]) of
+            {ok,List} when is_list(List) -> List;
+            {ok,Term} -> [Term];
+            {error,_Reason} ->
+              io:format("Ignoring bad term in ~ts\n", [Key]),
+              []
+          end;
+        {error, {_,_,_Reason}, _} ->
+          io:format("Ignoring bad term in ~ts\n", [Key]),
+          []
+      end
+  end.
+
+contains_on_load([{ attribute, _, on_load, _ }|_]) -> true;
+contains_on_load([_|T]) -> contains_on_load(T);
+contains_on_load([]) -> false.
+
 no_auto_import() ->
-  { attribute, 0, compile, {
-    no_auto_import, erlang:module_info(exports) } }.
+  Bifs = [{ Name, Arity } || { Name, Arity } <- erlang:module_info(exports), erl_internal:bif(Name, Arity)],
+  { attribute, 0, compile, { no_auto_import, Bifs } }.
 
-module_form(Exprs, Line, File, Module, Vars) when
-    is_binary(File), is_list(Exprs), is_integer(Line), is_atom(Module) ->
+module_form_fun(nil) -> '__FILE__';
+module_form_fun(_)   -> '__MODULE__'.
 
-  Cons = lists:foldr(fun({ _, Var, _ }, Acc) ->
+module_form(Fun, Exprs, Line, File, Module, Vars) when
+    is_binary(File), is_integer(Line), is_list(Exprs), is_atom(Module) ->
+  Cons = lists:foldr(fun({ _, _, Var, _ }, Acc) ->
     { cons, Line, { var, Line, Var }, Acc }
   end, { nil, Line }, Vars),
 
@@ -134,37 +171,19 @@ module_form(Exprs, Line, File, Module, Vars) when
   [
     { attribute, Line, file, { binary_to_list(File), 1 } },
     { attribute, Line, module, Module },
-    { attribute, Line, export, [{ 'BOOTSTRAP', 2 }] },
-    { function, Line, 'BOOTSTRAP', length(Args), [
+    { attribute, Line, export, [{ Fun, 2 }] },
+    { function, Line, Fun, length(Args), [
       { clause, Line, Args, [], Exprs }
     ] }
   ].
 
-%% Fast compilation is available?
+%% Generate module names from code server.
 
-allows_fast_compilation([{defmodule,_,_}|T]) -> allows_fast_compilation(T);
-allows_fast_compilation([]) -> true;
-allows_fast_compilation(_) -> false.
+retrieve_module_name() ->
+  elixir_code_server:call(retrieve_module_name).
 
-%% Escape the module name, removing slashes, dots,
-%% so it can be loaded by Erlang.
-
-escape_module(Module) when is_atom(Module) ->
-  escape_module(atom_to_list(Module));
-
-escape_module(Module) when is_binary(Module) ->
-  escape_module(binary_to_list(Module));
-
-escape_module(Module) when is_list(Module) ->
-  list_to_atom(escape_each(Module)).
-
-escape_each([H|T]) when H >= $A, H =< $Z; H >= $a, H =< $z; H >= $0, H =< $9; H == $- ->
-  [H|escape_each(T)];
-
-escape_each([_|T]) ->
-  [$_|escape_each(T)];
-
-escape_each([]) -> [].
+return_module_name(I) ->
+  elixir_code_server:cast({ return_module_name, I }).
 
 %% Receives a module Binary and outputs it in the given path.
 
@@ -179,50 +198,52 @@ core_file(File) ->
   try
     Lists = file(list_to_binary(File)),
     [binary_to_path(X, "lib/elixir/ebin") || X <- Lists],
-    io:format("Compiled ~s~n", [File])
+    io:format("Compiled ~ts~n", [File])
   catch
     Kind:Reason ->
       io:format("~p: ~p~nstacktrace: ~p~n", [Kind, Reason, erlang:get_stacktrace()]),
       exit(1)
   end.
 
-core_list() ->
-  [
-    "lib/elixir/lib/range.ex",
-    "lib/elixir/lib/behaviour.ex",
-    "lib/elixir/lib/uri/parser.ex",
-    "lib/elixir/lib/elixir/formatter.ex",
-    "lib/elixir/lib/dict.ex",
-    "lib/elixir/lib/dict/common.ex",
-    "lib/elixir/lib/access.ex",
-    "lib/elixir/lib/range.ex",
-    "lib/elixir/lib/*/*.ex",
-    "lib/elixir/lib/*.ex"
-  ].
-
 core_main() ->
   [
     "lib/elixir/lib/kernel.ex",
     "lib/elixir/lib/keyword.ex",
+    "lib/elixir/lib/list.ex",
+    "lib/elixir/lib/kernel/typespec.ex",
+    "lib/elixir/lib/module.ex",
     "lib/elixir/lib/record.ex",
+    "lib/elixir/lib/record/extractor.ex",
     "lib/elixir/lib/macro.ex",
     "lib/elixir/lib/macro/env.ex",
-    "lib/elixir/lib/module.ex",
-    "lib/elixir/lib/list.ex",
+    "lib/elixir/lib/exception.ex",
     "lib/elixir/lib/code.ex",
     "lib/elixir/lib/protocol.ex",
     "lib/elixir/lib/enum.ex",
-    "lib/elixir/lib/exception.ex",
     "lib/elixir/lib/binary/inspect.ex",
     "lib/elixir/lib/binary/chars.ex",
     "lib/elixir/lib/list/chars.ex",
-    "lib/elixir/lib/gen_server/behaviour.ex"
+    "lib/elixir/lib/io.ex",
+    "lib/elixir/lib/file.ex",
+    "lib/elixir/lib/path.ex",
+    "lib/elixir/lib/access.ex",
+    "lib/elixir/lib/regex.ex",
+    "lib/elixir/lib/system.ex",
+    "lib/elixir/lib/process.ex",
+    "lib/elixir/lib/kernel/cli.ex",
+    "lib/elixir/lib/kernel/error_handler.ex",
+    "lib/elixir/lib/kernel/parallel_compiler.ex",
+    "lib/elixir/lib/kernel/record_rewriter.ex"
   ].
 
 %% ERROR HANDLING
 
+format_error({ skip_native, Module }) ->
+  io_lib:format("skipping native compilation for ~ts because it contains on_load attribute",
+    [elixir_errors:inspect(Module)]).
+
 format_errors(_File, []) ->
-  exit({nocompile, "compilation failed but no error was raised"});
+  exit({ nocompile, "compilation failed but no error was raised" });
 
 format_errors(File, Errors) ->
   lists:foreach(fun ({_, Each}) ->

lib/elixir/src/elixir_def.erl

@@ -4,22 +4,24 @@
   build_table/1,
   delete_table/1,
   reset_last/1,
+  wrap_definition/5,
   wrap_definition/7,
+  store_definition/6,
   store_definition/8,
-  store_each/8,
-  unwrap_stored_definitions/1,
+  store_each/7,
+  unwrap_stored_definitions/2,
   format_error/1]).
 -include("elixir.hrl").
 -compile({parse_transform, elixir_transform}).
 
 %% Table management functions. Called internally.
 
-table(Module) -> ?ELIXIR_ATOM_CONCAT([f, Module]).
+table(Module) -> ?atom_concat([f, Module]).
 
 build_table(Module) ->
   FunctionTable = table(Module),
   ets:new(FunctionTable, [set, named_table, public]),
-  ets:insert(FunctionTable, { last, [] }),
+  reset_last(Module),
   FunctionTable.
 
 delete_table(Module) ->
@@ -44,44 +46,63 @@ reset_last(Module) ->
 %%
 %% If we just analyzed the compiled structure (i.e. the function availables
 %% before evaluating the function body), we would see both definitions.
-wrap_definition(Kind, Line, Name, Args, Guards, Expr, S) ->
-  MetaS = elixir_scope:serialize(S),
 
-  Invoke = [
-    {atom, Line, Kind},
-    {integer, Line, Line},
-    {var, Line, '_@MODULE'},
-    Name,
-    Args,
-    Guards,
-    Expr,
-    MetaS
-  ],
+wrap_definition(Kind, Meta, Call, Expr, S) ->
+  do_wrap_definition(Kind, Meta, [
+    Call, Expr, elixir_scope:serialize(S)
+  ]).
 
-  ?ELIXIR_WRAP_CALL(Line, ?MODULE, store_definition, Invoke).
+wrap_definition(Kind, Meta, Name, Args, Guards, Expr, S) ->
+  do_wrap_definition(Kind, Meta, [
+    Name, Args, Guards, Expr, elixir_scope:serialize(S)
+  ]).
+
+do_wrap_definition(Kind, Meta, Extra) ->
+  Line   = ?line(Meta),
+  Invoke =
+    [{atom, Line, Kind},
+     {integer, Line, Line},
+     {var, Line, '_@MODULE'}] ++ Extra,
+
+  ?wrap_call(Line, ?MODULE, store_definition, Invoke).
 
 % Invoked by the wrap definition with the function abstract tree.
 % Each function is then added to the function table.
+store_definition(Kind, Line, Module, Call, Body, RawS) ->
+  S = elixir_scope:deserialize(RawS),
+  { NameAndArgs, Guards } = elixir_clauses:extract_guards(Call),
+
+  { Name, Args } = case elixir_clauses:extract_args(NameAndArgs) of
+    error ->
+      Format = [Kind, 'Elixir.Macro':to_binary(NameAndArgs)],
+      elixir_errors:syntax_error(Line, S#elixir_scope.file, "invalid syntax in ~ts ~ts", Format);
+    Tuple ->
+      Tuple
+  end,
+
+  assert_no_aliases_name(Line, Name, Args, S),
+  store_definition(Kind, Line, Module, Name, Args, Guards, Body, RawS).
 
 store_definition(Kind, Line, nil, _Name, _Args, _Guards, _Body, RawS) ->
   S = elixir_scope:deserialize(RawS),
-  elixir_errors:syntax_error(Line, S#elixir_scope.file, "cannot define function outside module, invalid scope for ~s", [Kind]);
+  elixir_errors:syntax_error(Line, S#elixir_scope.file, "cannot define function outside module, invalid scope for ~ts", [Kind]);
 
 store_definition(Kind, Line, Module, Name, Args, Guards, Body, RawS) ->
+  do_store_definition(Kind, Line, Module, Name, Args, Guards, Body, elixir_scope:deserialize(RawS)).
+
+do_store_definition(Kind, Line, Module, Name, Args, Guards, Body, DS) ->
   Arity = length(Args),
-  DS    = elixir_scope:deserialize(RawS),
   S     = DS#elixir_scope{function={Name,Arity}, module=Module},
   Expr  = def_body(Line, Body),
 
   CO = elixir_compiler:get_opts(),
-  Location = retrieve_file(Module, CO),
+  Location = retrieve_file(Line, Module, S, CO),
   run_on_definition_callbacks(Kind, Line, Module, Name, Args, Guards, Body, S, CO),
 
   { Function, Defaults, TS } = translate_definition(Kind, Line, Name, Args, Guards, Expr, S),
 
   File  = TS#elixir_scope.file,
   Table = table(Module),
-  Stack = TS#elixir_scope.macro,
 
   %% Store function
   if
@@ -90,17 +111,17 @@ store_definition(Kind, Line, Module, Name, Args, Guards, Body, RawS) ->
       compile_super(Module, TS),
       CheckClauses = S#elixir_scope.check_clauses,
       store_each(CheckClauses, Kind, File, Location,
-        Stack, Table, length(Defaults), Function)
+        Table, length(Defaults), Function)
   end,
 
-  [store_each(false, Kind, File, Location, Stack, Table, 0,
-    function_for_default(Kind, Name, Default)) || Default <- Defaults],
+  [store_each(false, Kind, File, Location, Table, 0,
+    default_function_for(Kind, Name, Default)) || Default <- Defaults],
 
   { Name, Arity }.
 
 def_body(_Line, nil)            -> nil;
 def_body(_Line, [{ do, Expr }]) -> Expr;
-def_body(Line, Else)            -> { 'try', Line, [Else] }.
+def_body(Line, Else)            -> { 'try', [{line,Line}], [Else] }.
 
 %% @on_definition
 
@@ -110,18 +131,18 @@ run_on_definition_callbacks(Kind, Line, Module, Name, Args, Guards, Expr, S, CO)
       ok;
     _ ->
       Env = elixir_scope:to_ex_env({ Line, S }),
-      Callbacks = 'Elixir.Module':get_attribute(Module, on_definition),
-      [Mod:Fun(Env, Kind, Name, Args, Guards, Expr) || { Mod, Fun } <- Callbacks]
+      elixir_module:eval_callbacks(Line, Module, on_definition,
+        [Env, Kind, Name, Args, Guards, Expr], S)
   end.
 
-%% Retrieve @file
+%% Retrieve @file or fallback to default
 
-retrieve_file(Module, CO) ->
+retrieve_file(Line, Module, S, CO) ->
   case elixir_compiler:get_opt(internal, CO) of
-    true -> [];
+    true -> { binary_to_list(S#elixir_scope.file), Line };
     _ ->
       case 'Elixir.Module':get_attribute(Module, file) of
-        nil  -> [];
+        nil  -> { binary_to_list(S#elixir_scope.file), Line };
         Else ->
           'Elixir.Module':delete_attribute(Module, file),
           Else
@@ -138,7 +159,7 @@ compile_super(_Module, _S) -> ok.
 %% Translate the given call and expression given
 %% and then store it in memory.
 
-translate_definition(Kind, Line, Name, RawArgs, RawGuards, RawExpr, S) ->
+translate_definition(Kind, Line, Name, RawArgs, RawGuards, RawExpr, S) when is_integer(Line) ->
   Args    = elixir_quote:linify(Line, RawArgs),
   Guards  = elixir_quote:linify(Line, RawGuards),
   Expr    = elixir_quote:linify(Line, RawExpr),
@@ -149,7 +170,7 @@ translate_definition(Kind, Line, Name, RawArgs, RawGuards, RawExpr, S) ->
   %% not affect the arity of the stored function, but the clause
   %% already contains it.
   ExtendedArgs = case IsMacro of
-    true  -> [{ '_@CALLER', Line, nil }|Args];
+    true  -> [{ '_@CALLER', [{line,Line}], nil }|Args];
     false -> Args
   end,
 
@@ -171,7 +192,7 @@ translate_definition(Kind, Line, Name, RawArgs, RawGuards, RawExpr, S) ->
     true  ->
       FBody = { 'match', Line,
         { 'var', Line, '__CALLER__' },
-        ?ELIXIR_WRAP_CALL(Line, elixir_scope, to_ex_env, [{ var, Line, '_@CALLER' }])
+        ?wrap_call(Line, elixir_scope, to_ex_env, [{ var, Line, '_@CALLER' }])
       },
       setelement(5, NClause, [FBody|element(5, NClause)]);
     false -> NClause
@@ -187,52 +208,53 @@ is_macro(_)         -> false.
 % Unwrap the functions stored in the functions table.
 % It returns a list of all functions to be exported, plus the macros,
 % and the body of all functions.
-unwrap_stored_definitions(Module) ->
+unwrap_stored_definitions(File, Module) ->
   Table = table(Module),
   ets:delete(Table, last),
-  unwrap_stored_definition(ets:tab2list(Table), [], [], [], [], [], {[],[]}).
+  unwrap_stored_definition(ets:tab2list(Table), File, [], [], [], [], [], {[],[]}).
 
-unwrap_stored_definition([Fun|T], Exports, Private, Def, Defmacro, Defmacrop, Functions) when element(2, Fun) == def ->
+unwrap_stored_definition([Fun|T], File, Exports, Private, Def, Defmacro, Defmacrop, Functions) when element(2, Fun) == def ->
   Tuple = element(1, Fun),
   unwrap_stored_definition(
-    T, [Tuple|Exports], Private, [Tuple|Def], Defmacro, Defmacrop,
-    function_for_stored_definition(Fun, Functions)
+    T, File, [Tuple|Exports], Private, [Tuple|Def], Defmacro, Defmacrop,
+    function_for_stored_definition(Fun, File, Functions)
   );
 
-unwrap_stored_definition([Fun|T], Exports, Private, Def, Defmacro, Defmacrop, Functions) when element(2, Fun) == defmacro ->
+unwrap_stored_definition([Fun|T], File, Exports, Private, Def, Defmacro, Defmacrop, Functions) when element(2, Fun) == defmacro ->
   { Name, Arity } = Tuple = element(1, Fun),
-  Macro = { ?ELIXIR_MACRO(Name), Arity + 1 },
+  Macro = { ?elixir_macro(Name), Arity + 1 },
 
   unwrap_stored_definition(
-    T, [Macro|Exports], Private, Def, [Tuple|Defmacro], Defmacrop,
-    function_for_stored_definition(setelement(1, Fun, Macro), Functions)
+    T, File, [Macro|Exports], Private, Def, [Tuple|Defmacro], Defmacrop,
+    function_for_stored_definition(setelement(1, Fun, Macro), File, Functions)
   );
 
-unwrap_stored_definition([Fun|T], Exports, Private, Def, Defmacro, Defmacrop, Functions) when element(2, Fun) == defp ->
+unwrap_stored_definition([Fun|T], File, Exports, Private, Def, Defmacro, Defmacrop, Functions) when element(2, Fun) == defp ->
   unwrap_stored_definition(
-    T, Exports, [element(1, Fun)|Private], Def, Defmacro, Defmacrop,
-    function_for_stored_definition(Fun, Functions)
+    T, File, Exports, [element(1, Fun)|Private], Def, Defmacro, Defmacrop,
+    function_for_stored_definition(Fun, File, Functions)
   );
 
-unwrap_stored_definition([Fun|T], Exports, Private, Def, Defmacro, Defmacrop, Functions) when element(2, Fun) == defmacrop ->
+unwrap_stored_definition([Fun|T], File, Exports, Private, Def, Defmacro, Defmacrop, Functions) when element(2, Fun) == defmacrop ->
+  Tuple = element(1, Fun),
   unwrap_stored_definition(
-    T, Exports, [element(1, Fun)|Private], Def, Defmacro,
-    [{ element(1, Fun), element(3, Fun) }|Defmacrop], Functions
+    T, File, Exports, [Tuple|Private], Def, Defmacro,
+    [{ Tuple, element(3, Fun), element(5, Fun) }|Defmacrop], Functions
   );
 
-unwrap_stored_definition([], Exports, Private, Def, Defmacro, Defmacrop, {Functions,Tail}) ->
+unwrap_stored_definition([], _File, Exports, Private, Def, Defmacro, Defmacrop, {Functions,Tail}) ->
   { Exports, Private, ordsets:from_list(Def), ordsets:from_list(Defmacro),
     ordsets:from_list(Defmacrop), lists:reverse(Tail ++ Functions) }.
 
 %% Helpers
 
-function_for_stored_definition({{Name, Arity}, _, Line, _, [], _, _, Clauses}, {Functions,Tail}) ->
+function_for_stored_definition({{Name, Arity}, _, Line, _, _, { File, _ }, _, Clauses}, File, {Functions,Tail}) ->
   {
     [{ function, Line, Name, Arity, lists:reverse(Clauses) }|Functions],
     Tail
   };
 
-function_for_stored_definition({{Name, Arity}, _, Line, _, Location, _, _, Clauses}, {Functions,Tail}) ->
+function_for_stored_definition({{Name, Arity}, _, Line, _, _, Location, _, Clauses}, _File, {Functions,Tail}) ->
   {
     Functions,
     [
@@ -241,33 +263,33 @@ function_for_stored_definition({{Name, Arity}, _, Line, _, Location, _, _, Claus
     ]
   }.
 
-function_for_default(Kind, Name, { clause, Line, Args, _Guards, _Exprs } = Clause)
+default_function_for(Kind, Name, { clause, Line, Args, _Guards, _Exprs } = Clause)
     when Kind == defmacro; Kind == defmacrop ->
   { function, Line, Name, length(Args) - 1, [Clause] };
 
-function_for_default(_, Name, { clause, Line, Args, _Guards, _Exprs } = Clause) ->
+default_function_for(_, Name, { clause, Line, Args, _Guards, _Exprs } = Clause) ->
   { function, Line, Name, length(Args), [Clause] }.
 
 %% Store each definition in the table.
 %% This function also checks and emit warnings in case
 %% the kind, of the visibility of the function changes.
 
-store_each(Check, Kind, File, Location, Stack, Table, Defaults, {function, Line, Name, Arity, Clauses}) ->
+store_each(Check, Kind, File, Location, Table, Defaults, {function, Line, Name, Arity, Clauses}) ->
   case ets:lookup(Table, {Name, Arity}) of
-    [{{Name, Arity}, StoredKind, _, _, StoredLocation, StoredStack, StoredDefaults, StoredClauses}] ->
+    [{{Name, Arity}, StoredKind, _, _, StoredCheck, StoredLocation, StoredDefaults, StoredClauses}] ->
       FinalLocation = StoredLocation,
       FinalDefaults = Defaults + StoredDefaults,
       FinalClauses  = Clauses ++ StoredClauses,
       check_valid_kind(Line, File, Name, Arity, Kind, StoredKind),
       check_valid_defaults(Line, File, Name, Arity, FinalDefaults),
-      Check andalso (Stack == StoredStack) andalso check_valid_clause(Line, File, Name, Arity, Table);
+      (Check and StoredCheck) andalso check_valid_clause(Line, File, Name, Arity, Table);
     [] ->
       FinalLocation = Location,
       FinalDefaults = Defaults,
       FinalClauses  = Clauses,
       Check andalso ets:insert(Table, { last, { Name, Arity } })
   end,
-  ets:insert(Table, {{Name, Arity}, Kind, Line, File, FinalLocation, Stack, FinalDefaults, FinalClauses}).
+  ets:insert(Table, {{Name, Arity}, Kind, Line, File, Check, FinalLocation, FinalDefaults, FinalClauses}).
 
 %% Validations
 
@@ -289,13 +311,20 @@ check_valid_defaults(_Line, _File, _Name, _Arity, 0) -> [];
 check_valid_defaults(Line, File, Name, Arity, _) ->
   elixir_errors:handle_file_warning(File, { Line, ?MODULE, { clauses_with_docs, { Name, Arity } } }).
 
+assert_no_aliases_name(Line, '__aliases__', [Atom], #elixir_scope{file=File}) when is_atom(Atom) ->
+  Message = "function names should start with lowercase characters or underscore, invalid name ~ts",
+  elixir_errors:syntax_error(Line, File, Message, [atom_to_binary(Atom, utf8)]);
+
+assert_no_aliases_name(_Meta, _Aliases, _Args, _S) ->
+  ok.
+
 %% Format errors
 
 format_error({clauses_with_docs,{Name,Arity}}) ->
-  io_lib:format("function ~s/~B has default values and multiple clauses, use a separate clause for declaring defaults", [Name, Arity]);
+  io_lib:format("function ~ts/~B has default values and multiple clauses, use a separate clause for declaring defaults", [Name, Arity]);
 
 format_error({changed_clause,{{Name,Arity},{ElseName,ElseArity}}}) ->
-  io_lib:format("function ~s/~B does not match previous clause ~s/~B", [Name, Arity, ElseName, ElseArity]);
+  io_lib:format("function ~ts/~B does not match previous clause ~ts/~B", [Name, Arity, ElseName, ElseArity]);
 
 format_error({changed_kind,{Name,Arity,Previous,Current}}) ->
-  io_lib:format("~s ~s/~B already defined as ~s", [Current, Name, Arity, Previous]).
+  io_lib:format("~ts ~ts/~B already defined as ~ts", [Current, Name, Arity, Previous]).

lib/elixir/src/elixir_def_defaults.erl

@@ -15,8 +15,7 @@ unpack_each(Kind, Name, [{'//', Line, [Expr, _]}|T] = List, Acc, Clauses, S) ->
   Base = build_match(Acc, Line, []),
   { Args, Invoke } = extract_defaults(List, Line, length(Base), [], []),
 
-  SM = S#elixir_scope{counter=length(Base)},
-  { DefArgs, SA }   = elixir_clauses:assigns(fun elixir_translator:translate/2, Base ++ Args, SM),
+  { DefArgs, SA }   = elixir_clauses:assigns(fun elixir_translator:translate/2, Base ++ Args, S),
   { InvokeArgs, _ } = elixir_translator:translate_args(Base ++ Invoke, SA),
 
   Call = { call, Line,
@@ -39,7 +38,7 @@ extract_defaults([{'//', _, [_Expr, Default]}|T], Line, Counter, NewArgs, NewInv
   extract_defaults(T, Line, Counter, NewArgs, [Default|NewInvoke]);
 
 extract_defaults([_|T], Line, Counter, NewArgs, NewInvoke) ->
-  H = { ?ELIXIR_ATOM_CONCAT(["_@", Counter]), Line, nil },
+  H = { ?atom_concat(["_@D", Counter]), Line, nil },
   extract_defaults(T, Line, Counter + 1, [H|NewArgs], [H|NewInvoke]);
 
 extract_defaults([], _Line, _Counter, NewArgs, NewInvoke) ->
@@ -50,10 +49,10 @@ extract_defaults([], _Line, _Counter, NewArgs, NewInvoke) ->
 build_match([], _Line, Acc) -> Acc;
 
 build_match([_|T], Line, Acc) ->
-  Var = { ?ELIXIR_ATOM_CONCAT(["_@", length(T)]), Line, nil },
+  Var = { ?atom_concat(["_@D", length(T)]), Line, nil },
   build_match(T, Line, [Var|Acc]).
 
 % Given the invoked function name based on the kind
 
-name_for_kind(Kind, Name) when Kind == defmacro; Kind == defmacrop -> ?ELIXIR_MACRO(Name);
+name_for_kind(Kind, Name) when Kind == defmacro; Kind == defmacrop -> ?elixir_macro(Name);
 name_for_kind(_Kind, Name) -> Name.

lib/elixir/src/elixir_def_local.erl

@@ -69,9 +69,10 @@ rewrite_clause(Else, _) -> Else.
 
 %% Error handling
 
-check_unused_local_macros(File, Recorded, PMacros) ->
+check_unused_local_macros(File, Recorded, Defmacrop) ->
   [elixir_errors:handle_file_warning(File,
-    { Line, ?MODULE, { unused_macro, Fun } }) || { Fun, Line } <- PMacros, not lists:member(Fun, Recorded)].
+    { Line, ?MODULE, { unused_macro, Fun } }) || { Fun, Line, Check } <- Defmacrop,
+      Check, not lists:member(Fun, Recorded)].
 
 format_error({unused_macro,{Name, Arity}}) ->
-  io_lib:format("macro ~s/~B is unused", [Name, Arity]).
+  io_lib:format("macro ~ts/~B is unused", [Name, Arity]).

lib/elixir/src/elixir_def_overridable.erl

@@ -16,19 +16,20 @@ overridable(Module, Value) ->
 is_defined(Module, Tuple) ->
   Overridable = overridable(Module),
   case orddict:find(Tuple, Overridable) of
-    { ok, { _, [_|_] } } -> true;
+    { ok, { _, _, _ } } -> true;
     _ -> false
   end.
 
-ensure_defined(Line, Module, Tuple, S) ->
+ensure_defined(Meta, Module, Tuple, S) ->
   case is_defined(Module, Tuple) of
     true -> [];
-    _    -> elixir_errors:form_error(Line, S#elixir_scope.file, ?MODULE, { no_super, Module, Tuple })
+    _    -> elixir_errors:form_error(Meta, S#elixir_scope.file, ?MODULE, { no_super, Module, Tuple })
   end.
 
 %% Retrieve args defined for the given arity.
 
-retrieve_args(Line, Arity, S) ->
+retrieve_args(Meta, Arity, S) ->
+  Line = ?line(Meta),
   {
     [ { var, Line, super_arg(X) } || X <- lists:seq(1, Arity) ],
     S#elixir_scope{name_args=true}
@@ -36,7 +37,7 @@ retrieve_args(Line, Arity, S) ->
 
 %% Assign pseudo variables to the given vars.
 
-assign_args(Line, Args, S) ->
+assign_args(Line, Args, S) when is_integer(Line) ->
   { FArgs, _ } = lists:mapfoldl(fun(X, Acc) -> assign_args(Line, X, Acc, S) end, 1, Args),
   FArgs.
 
@@ -45,7 +46,7 @@ assign_args(Line, X, Acc, _) ->
   { Match, Acc + 1 }.
 
 super_arg(Counter) ->
-  ?ELIXIR_ATOM_CONCAT(['_@S', Counter]).
+  ?atom_concat(['_@S', Counter]).
 
 %% Gets the name based on the function and stored overridables
 
@@ -53,39 +54,41 @@ name(Module, Function) ->
   name(Module, Function, overridable(Module)).
 
 name(_Module, { Name, _ } = Function, Overridable) ->
-  { Count, _ } = orddict:fetch(Function, Overridable),
-  ?ELIXIR_ATOM_CONCAT([Name, " (overridable ", Count, ")"]).
+  { Count, _, _ } = orddict:fetch(Function, Overridable),
+  ?atom_concat([Name, " (overridable ", Count, ")"]).
 
 %% Store
 
 store(Module, Function, GenerateName) ->
   Overridable = overridable(Module),
-  { Count, [H|T] } = orddict:fetch(Function, Overridable),
-  overridable(Module, orddict:store(Function, { Count, T }, Overridable)),
+  case orddict:fetch(Function, Overridable) of
+    { _Count, _Clause, true } -> ok;
+    { Count, Clause, false } ->
+      overridable(Module, orddict:store(Function, { Count, Clause, true }, Overridable)),
+      { { Name, Arity }, Kind, Line, File, _Check, Location, Defaults, Clauses } = Clause,
 
-  { { Name, Arity }, Kind, Line, File, Location, Stack, Defaults, Clauses } = H,
+      { FinalKind, FinalName } = case GenerateName of
+        true  -> { defp, name(Module, Function, Overridable) };
+        false -> { Kind, Name }
+      end,
 
-  { FinalKind, FinalName } = case GenerateName of
-    true  -> { defp, name(Module, Function, Overridable) };
-    false -> { Kind, Name }
-  end,
-
-  Def = { function, Line, FinalName, Arity, Clauses },
-  elixir_def:store_each(false, FinalKind, File, Location,
-    Stack, elixir_def:table(Module), Defaults, Def).
+      Def = { function, Line, FinalName, Arity, Clauses },
+      elixir_def:store_each(false, FinalKind, File, Location,
+        elixir_def:table(Module), Defaults, Def)
+  end.
 
 %% Store pending declarations that were not manually made concrete.
 
 store_pending(Module) ->
-  [store(Module, X, false) || { X, { _, [_|_] } } <- overridable(Module),
+  [store(Module, X, false) || { X, { _, _, false } } <- overridable(Module),
     not 'Elixir.Module':'defines?'(Module, X)].
 
 %% Error handling
 
 format_error({ no_super, Module, { Name, Arity } }) ->
-  Bins   = [ format_fa(X) || { X, { _, [_|_] } } <- overridable(Module)],
+  Bins   = [ format_fa(X) || { X, { _, _, _ } } <- overridable(Module)],
   Joined = 'Elixir.Enum':join(Bins, <<", ">>),
-  io_lib:format("no super defined for ~s/~B in module ~s. Overridable functions available are: ~s",
+  io_lib:format("no super defined for ~ts/~B in module ~ts. Overridable functions available are: ~ts",
     [Name, Arity, elixir_errors:inspect(Module), Joined]).
 
 format_fa({ Name, Arity }) ->

lib/elixir/src/elixir_dispatch.erl

@@ -5,7 +5,7 @@
 -export([default_macros/0, default_functions/0, default_requires/0,
   dispatch_require/6, dispatch_import/5,
   require_function/5, import_function/4,
-  expand_import/8, expand_require/8,
+  expand_import/6, expand_require/6, find_import/4,
   format_error/1, in_erlang_functions/0, in_erlang_macros/0]).
 -include("elixir.hrl").
 -compile({parse_transform, elixir_transform}).
@@ -18,169 +18,243 @@ default_functions() ->
 default_macros() ->
   [ { ?BUILTIN, ordsets:union(in_elixir_macros(), in_erlang_macros()) } ].
 default_requires() ->
-  [ ?BUILTIN, 'Elixir.Kernel.Typespec' ].
+  [ ?BUILTIN, 'Elixir.Kernel.Typespec', 'Elixir.Record' ].
+
+find_import(Meta, Name, Arity, S) ->
+  Tuple = { Name, Arity },
+
+  case find_dispatch(Meta, Tuple, S) of
+    { function, Receiver } -> Receiver;
+    { macro, Receiver } -> Receiver;
+    _ -> false
+  end.
 
 %% Function retrieval
 
-import_function(Line, Name, Arity, S) ->
+import_function(Meta, Name, Arity, S) ->
   Tuple = { Name, Arity },
-  case find_dispatch(Tuple, S#elixir_scope.functions) of
-    false ->
-      case find_dispatch(Tuple, S#elixir_scope.macros) of
-        false -> { { 'fun', Line, { function, Name, Arity } }, S };
-        _ -> false
-      end;
-    Receiver ->
+  case find_dispatch(Meta, Tuple, S) of
+    { function, Receiver } ->
       elixir_import:record(import, Tuple, Receiver, S#elixir_scope.module),
-      remote_function(Line, Receiver, Name, Arity, S)
+      remote_function(Meta, Receiver, Name, Arity, S);
+    { macro, _Receiver } ->
+      false;
+    { import, Receiver } ->
+      require_function(Meta, Receiver, Name, Arity, S);
+    nomatch ->
+      { { 'fun', ?line(Meta), { function, Name, Arity } }, S }
   end.
 
-require_function(Line, Receiver, Name, Arity, S) ->
+require_function(Meta, Receiver, Name, Arity, S) ->
   Tuple = { Name, Arity },
 
   case is_element(Tuple, get_optional_macros(Receiver)) of
     true  -> false;
-    false -> remote_function(Line, Receiver, Name, Arity, S)
+    false -> remote_function(Meta, Receiver, Name, Arity, S)
   end.
 
 %% Function dispatch
 
-dispatch_import(Line, Name, Args, S, Callback) ->
+dispatch_import(Meta, Name, Args, S, Callback) ->
   Module = S#elixir_scope.module,
   Arity  = length(Args),
   Tuple  = { Name, Arity },
 
-  case find_dispatch(Tuple, S#elixir_scope.functions) of
-    false ->
-      case expand_import(Line, Tuple, Args, Module, S#elixir_scope.function,
-          S#elixir_scope.requires, S#elixir_scope.macros, S) of
-        { error, noexpansion } ->
-          Callback();
-        { error, internal } ->
-          elixir_import:record(import, Tuple, ?BUILTIN, Module),
-          elixir_macros:translate({ Name, Line, Args }, S);
-        { ok, Receiver, Tree } ->
-          translate_expansion(Line, Tree, Receiver, Name, Arity, S)
-      end;
-    Receiver ->
+  case find_dispatch(Meta, Tuple, S) of
+    { function, Receiver } ->
       elixir_import:record(import, Tuple, Receiver, Module),
       Endpoint = case (Receiver == ?BUILTIN) andalso is_element(Tuple, in_erlang_functions()) of
         true  -> erlang;
         false -> Receiver
       end,
-      elixir_translator:translate_each({ { '.', Line, [Endpoint, Name] }, Line, Args }, S)
+      elixir_translator:translate_each({ { '.', Meta, [Endpoint, Name] }, Meta, Args }, S);
+    { import, Receiver } ->
+      elixir_translator:translate_each({ { '.', Meta, [Receiver, Name] }, [{require,false}|Meta], Args }, S);
+    Result ->
+      case do_expand_import(Meta, Tuple, Args, Module, S, Result) of
+        { error, noexpansion } ->
+          Callback();
+        { error, internal } ->
+          elixir_import:record(import, Tuple, ?BUILTIN, Module),
+          elixir_macros:translate({ Name, Meta, Args }, S);
+        { ok, _Receiver, Tree } ->
+          translate_expansion(Meta, Tree, S)
+      end
   end.
 
-dispatch_require(Line, Receiver, Name, Args, S, Callback) ->
+dispatch_require(Meta, Receiver, Name, Args, S, Callback) ->
   Module = S#elixir_scope.module,
   Arity  = length(Args),
   Tuple  = { Name, Arity },
 
-  case (Receiver == Module) andalso is_element(Tuple, in_erlang_functions()) of
+  case (Receiver == ?BUILTIN) andalso is_element(Tuple, in_erlang_functions()) of
     true ->
-      elixir_translator:translate_each({ { '.', Line, [erlang, Name] }, Line, Args }, S);
+      { TArgs, SA } = elixir_translator:translate_args(Args, S),
+      { ?wrap_call(?line(Meta), erlang, Name, TArgs), SA };
     false ->
-      case expand_require(Line, Receiver, Tuple, Args, Module,
-          S#elixir_scope.function, S#elixir_scope.requires, S) of
+      case expand_require(Meta, Receiver, Tuple, Args, Module, S) of
         { error, noexpansion } ->
           Callback();
         { error, internal } ->
-          elixir_macros:translate({ Name, Line, Args }, S);
-        { ok, Tree } ->
-          translate_expansion(Line, Tree, Receiver, Name, Arity, S)
+          elixir_macros:translate({ Name, Meta, Args }, S);
+        { ok, _Receiver, Tree } ->
+          translate_expansion(Meta, Tree, S)
       end
   end.
 
 %% Macros expansion
 
-expand_import(Line, { Name, Arity } = Tuple, Args, Module, Function, Requires, Macros, SEnv) ->
-  case find_dispatch(Tuple, Macros) of
-    false ->
-      Fun = (Function /= Tuple) andalso
+expand_import(Meta, Tuple, Args, Module, Extra, S) ->
+  Result = find_dispatch(Meta, Tuple, Extra, S),
+  do_expand_import(Meta, Tuple, Args, Module, S, Result).
+
+do_expand_import(Meta, { Name, Arity } = Tuple, Args, Module, S, Result) ->
+  case Result of
+    { macro, ?BUILTIN } ->
+      case is_element(Tuple, in_erlang_macros()) of
+        true  -> { error, internal };
+        false ->
+          elixir_import:record(import, Tuple, ?BUILTIN, Module),
+          { ok, ?BUILTIN, expand_macro_named(Meta, ?BUILTIN, Name, Arity, Args, Module, S) }
+      end;
+    { macro, Receiver } ->
+      elixir_import:record(import, Tuple, Receiver, Module),
+      { ok, Receiver, expand_macro_named(Meta, Receiver, Name, Arity, Args, Module, S) };
+    { import, Receiver } ->
+      expand_require(Meta, Receiver, Tuple, Args, Module, S);
+    _ ->
+      Fun = (S#elixir_scope.function /= Tuple) andalso
         elixir_def_local:macro_for(Tuple, true, Module),
       case Fun of
         false -> { error, noexpansion };
         _ ->
           elixir_import:record(import, Tuple, Module, Module),
-          { ok, Module, expand_macro_fun(Line, Fun, Module, Name, Args, Module, Requires, SEnv) }
-      end;
-    ?BUILTIN ->
-      case is_element(Tuple, in_erlang_macros()) of
-        true  -> { error, internal };
-        false ->
-          elixir_import:record(import, Tuple, ?BUILTIN, Module),
-          { ok, ?BUILTIN, expand_macro_named(Line, ?BUILTIN, Name, Arity, Args, Module, Requires, SEnv) }
-      end;
-    Receiver ->
-      elixir_import:record(import, Tuple, Receiver, Module),
-      { ok, Receiver, expand_macro_named(Line, Receiver, Name, Arity, Args, Module, Requires, SEnv) }
+          { ok, Module, expand_macro_fun(Meta, Fun, Module, Name, Args, Module, S) }
+      end
   end.
 
-expand_require(Line, ?BUILTIN, { Name, Arity } = Tuple, Args, Module, _Function, Requires, SEnv) ->
+expand_require(Meta, ?BUILTIN, { Name, Arity } = Tuple, Args, Module, S) ->
   case is_element(Tuple, in_erlang_macros()) of
     true  -> { error, internal };
     false ->
       case is_element(Tuple, in_elixir_macros()) of
-        true  -> { ok, expand_macro_named(Line, ?BUILTIN, Name, Arity, Args, Module, Requires, SEnv) };
+        true  -> { ok, ?BUILTIN, expand_macro_named(Meta, ?BUILTIN, Name, Arity, Args, Module, S) };
         false -> { error, noexpansion }
       end
   end;
 
-expand_require(Line, Receiver, { Name, Arity } = Tuple, Args, Module, Function, Requires, SEnv) ->
-  Fun = (Module == Receiver) andalso (Function /= Tuple) andalso
+expand_require(Meta, Receiver, { Name, Arity } = Tuple, Args, Module, S) ->
+  Fun = (Module == Receiver) andalso (S#elixir_scope.function /= Tuple) andalso
     elixir_def_local:macro_for(Tuple, false, Module),
 
   case Fun of
     false ->
       case is_element(Tuple, get_optional_macros(Receiver)) of
-        true  -> { ok, expand_macro_named(Line, Receiver, Name, Arity, Args, Module, Requires, SEnv) };
+        true  -> { ok, Receiver, expand_macro_named(Meta, Receiver, Name, Arity, Args, Module, S) };
         false -> { error, noexpansion }
       end;
     _ ->
       elixir_import:record(import, Tuple, Receiver, Module),
-      { ok, expand_macro_fun(Line, Fun, Receiver, Name, Args, Module, Requires, SEnv) }
+      { ok, Receiver, expand_macro_fun(Meta, Fun, Receiver, Name, Args, Module, S) }
   end.
 
 %% Expansion helpers
 
-expand_macro_fun(Line, Fun, Receiver, Name, Args, Module, Requires, SEnv) ->
-  case (Receiver == Module) or is_element(Receiver, Requires) of
+expand_macro_fun(Meta, Fun, Receiver, Name, Args, Module, S) ->
+  Requires = S#elixir_scope.requires,
+  case (Receiver == Module) or is_element(Receiver, Requires) or skip_require(Meta) of
     true  -> ok;
     false ->
       Tuple = { unrequired_module, { Receiver, Name, length(Args), Requires } },
-      elixir_errors:form_error(Line, elixir_scope:filename(SEnv), ?MODULE, Tuple)
+      elixir_errors:form_error(Meta, S#elixir_scope.file, ?MODULE, Tuple)
   end,
 
-  SArg = {Line,SEnv},
+  Line = ?line(Meta),
+  SArg = {Line,S},
 
   try
     apply(Fun, [SArg|Args])
   catch
     Kind:Reason ->
-      Info = { Receiver, Name, length(Args), [{ file, elixir_scope:filename(SEnv) }, { line, Line }] },
+      Info = { Receiver, Name, length(Args), [{ file, S#elixir_scope.file }, { line, Line }] },
       erlang:raise(Kind, Reason, munge_stacktrace(Info, erlang:get_stacktrace(), SArg))
   end.
 
-expand_macro_named(Line, Receiver, Name, Arity, Args, Module, Requires, SEnv) ->
-  ProperName  = ?ELIXIR_MACRO(Name),
+expand_macro_named(Meta, Receiver, Name, Arity, Args, Module, S) ->
+  ProperName  = ?elixir_macro(Name),
   ProperArity = Arity + 1,
   Fun         = fun Receiver:ProperName/ProperArity,
-  expand_macro_fun(Line, Fun, Receiver, Name, Args, Module, Requires, SEnv).
+  expand_macro_fun(Meta, Fun, Receiver, Name, Args, Module, S).
 
-translate_expansion(Line, Tree, Receiver, Name, Arity, S) ->
-  NewS = S#elixir_scope{macro=[{Line,Receiver,Name,Arity}|S#elixir_scope.macro]},
-  { TTree, TS } = elixir_translator:translate_each(elixir_quote:linify(Line, Tree), NewS),
-  { TTree, TS#elixir_scope{macro=S#elixir_scope.macro} }.
+translate_expansion(Meta, Tree, S) ->
+  { TR, TS } = elixir_translator:translate_each(
+    elixir_quote:linify(?line(Meta), Tree),
+    S#elixir_scope{
+      check_clauses=false,
+      macro_functions=[],
+      macro_macros=[],
+      macro_aliases=[]
+    }
+  ),
+  { TR, TS#elixir_scope{
+    check_clauses=S#elixir_scope.check_clauses,
+    macro_functions=merge_imports(S#elixir_scope.macro_functions, TS#elixir_scope.macro_functions),
+    macro_macros=merge_imports(S#elixir_scope.macro_macros, TS#elixir_scope.macro_macros),
+    macro_aliases=merge_aliases(S#elixir_scope.macro_aliases, TS#elixir_scope.macro_aliases)
+  } }.
+
+merge_aliases(A1, A2) ->
+  orddict:merge(fun(_K,_V1,V2) -> V2 end, A1, A2).
+
+merge_imports(A1, A2) ->
+  A2 ++ lists:foldl(fun({ Key, _ }, Acc) -> lists:keydelete(Key, 1, Acc) end, A1, A2).
 
 %% Helpers
 
-find_dispatch(Tuple, [{ Name, Values }|T]) ->
-  case is_element(Tuple, Values) of
-    true  -> Name;
-    false -> find_dispatch(Tuple, T)
-  end;
+skip_require(Meta) ->
+  lists:keyfind(require, 1, Meta) == { require, false }.
 
-find_dispatch(_Tuple, []) -> false.
+find_dispatch(Meta, Tuple, S) ->
+  find_dispatch(Meta, Tuple, [], S).
+
+find_dispatch(Meta, Tuple, Extra, S) ->
+  case is_import(Meta, Tuple, S) of
+    { import, _ } = Import ->
+      Import;
+    false ->
+      Functions = S#elixir_scope.functions,
+      Macros = Extra ++ S#elixir_scope.macros,
+      File = S#elixir_scope.file,
+      FunMatch = find_dispatch(Tuple, Functions),
+      MacMatch = find_dispatch(Tuple, Macros),
+
+      case { FunMatch, MacMatch } of
+        { [], [Receiver] } -> { macro, Receiver };
+        { [Receiver], [] } -> { function, Receiver };
+        { [], [] } -> nomatch;
+        _ ->
+          { Name, Arity } = Tuple,
+          [First, Second|_] = FunMatch ++ MacMatch,
+          Error = { ambiguous_call, { First, Second, Name, Arity } },
+          elixir_errors:form_error(Meta, File, ?MODULE, Error)
+      end
+  end.
+
+find_dispatch(Tuple, List) ->
+  [Receiver || { Receiver, Set } <- List, is_element(Tuple, Set)].
+
+is_import(Meta, Tuple, S) ->
+  case lists:keyfind(import, 1, Meta) of
+    { import, _ } = Import ->
+      not_an_import(Tuple, S#elixir_scope.macro_functions)
+        andalso not_an_import(Tuple, S#elixir_scope.macro_macros)
+        andalso Import;
+    false ->
+      false
+  end.
+
+not_an_import(Tuple, Pairs) ->
+  not lists:any(fun({ _, List }) -> lists:member(Tuple, List) end, Pairs).
 
 munge_stacktrace(Info, [{ _, _, [S|_], _ }|_], S) ->
   [Info];
@@ -196,16 +270,22 @@ munge_stacktrace(_, [], _) ->
 
 %% ERROR HANDLING
 
-format_error({ unrequired_module,{Receiver, Name, Arity, Required }}) ->
+format_error({ unrequired_module, { Receiver, Name, Arity, Required }}) ->
   String = string:join([elixir_errors:inspect(R) || R <- Required], ", "),
-  io_lib:format("tried to invoke macro ~s.~s/~B but module was not required. Required: ~s",
-    [elixir_errors:inspect(Receiver), Name, Arity, String]).
+  io_lib:format("tried to invoke macro ~ts.~ts/~B but module was not required. Required: ~ts",
+    [elixir_errors:inspect(Receiver), Name, Arity, String]);
+
+format_error({ ambiguous_call, { Mod1, Mod2, Name, Arity }}) ->
+  io_lib:format("function ~ts/~B imported from both ~ts and ~ts, call is ambiguous",
+    [Name, Arity, elixir_errors:inspect(Mod1), elixir_errors:inspect(Mod2)]).
 
 %% INTROSPECTION
 
-remote_function(Line, Receiver, Name, Arity, S) ->
+remote_function(Meta, Receiver, Name, Arity, S) ->
+  Line  = ?line(Meta),
+
   Final =
-    case Receiver == ?BUILTIN andalso is_element({ Name, Arity }, in_erlang_functions()) of
+    case (Receiver == ?BUILTIN) andalso is_element({ Name, Arity }, in_erlang_functions()) of
       true  -> erlang;
       false -> Receiver
     end,
@@ -234,7 +314,7 @@ get_optional_macros(Receiver) ->
 
 in_elixir_functions() ->
   try
-    ?BUILTIN:'__info__'(functions) -- [{'__info__',1}]
+    ?BUILTIN:'__info__'(functions)
   catch
     error:undef -> []
   end.
@@ -268,9 +348,6 @@ in_erlang_functions() ->
     { exit, 1 },
     { float, 1 },
     { float_to_list, 1 },
-    { halt, 0 },
-    { halt, 1 },
-    { halt, 2 },
     { hd, 1 },
     { integer_to_list, 1 },
     { integer_to_list, 2 },
@@ -373,6 +450,5 @@ in_erlang_macros() ->
     {'or',2},
     {'receive',1},
     {'try',1},
-    {'var!',1},
     {'xor',2}
-  ].
+  ].

lib/elixir/src/elixir_errors.erl

@@ -4,8 +4,8 @@
 -export([syntax_error/3, syntax_error/4, inspect/1,
   form_error/4, parse_error/4, assert_module_scope/3,
   assert_no_function_scope/3, assert_function_scope/3,
-  assert_no_assign_scope/3, assert_no_guard_scope/3,
-  assert_no_assign_or_guard_scope/3,
+  assert_no_match_scope/3, assert_no_guard_scope/3,
+  assert_no_match_or_guard_scope/3,
   handle_file_warning/2, handle_file_warning/3, handle_file_error/2,
   deprecation/3, deprecation/4, file_format/3]).
 -include("elixir.hrl").
@@ -26,29 +26,34 @@ to_dot(L)  -> L.
 
 %% Raised during macros translation.
 
-syntax_error(Line, File, Message) when is_list(Message) ->
-  syntax_error(Line, File, iolist_to_binary(Message));
+-spec syntax_error(non_neg_integer() | list(), file:filename(), binary() | string()) -> no_return().
+-spec syntax_error(non_neg_integer() | list(), file:filename(), binary() | string(), list()) -> no_return().
 
-syntax_error(Line, File, Message) when is_binary(Message) ->
-  raise(Line, File, 'Elixir.SyntaxError', Message).
+syntax_error(Meta, File, Message) when is_list(Message) ->
+  syntax_error(Meta, File, iolist_to_binary(Message));
 
-syntax_error(Line, File, Format, Args)  ->
+syntax_error(Meta, File, Message) when is_binary(Message) ->
+  raise(Meta, File, 'Elixir.SyntaxError', Message).
+
+syntax_error(Meta, File, Format, Args)  ->
   Message = io_lib:format(Format, Args),
-  raise(Line, File, 'Elixir.SyntaxError', iolist_to_binary(Message)).
+  raise(Meta, File, 'Elixir.SyntaxError', iolist_to_binary(Message)).
 
 %% Raised on tokenizing/parsing
 
-parse_error(Line, File, Error, []) ->
+-spec parse_error(non_neg_integer() | list(), file:filename(), iolist() | atom(), [] | iolist()) -> no_return().
+
+parse_error(Meta, File, Error, []) ->
   Message = case Error of
     "syntax error before: " -> <<"syntax error: expression is incomplete">>;
     _ -> iolist_to_binary(Error)
   end,
-  raise(Line, File, 'Elixir.TokenMissingError', Message);
+  raise(Meta, File, 'Elixir.TokenMissingError', Message);
 
-parse_error(Line, File, "syntax error before: ", "'end'") ->
-  raise(Line, File, 'Elixir.SyntaxError', <<"unexpected token: end">>);
+parse_error(Meta, File, "syntax error before: ", "'end'") ->
+  raise(Meta, File, 'Elixir.SyntaxError', <<"unexpected token: end">>);
 
-parse_error(Line, File, Error, Token) ->
+parse_error(Meta, File, Error, Token) ->
   BinError = if
     is_atom(Error) -> atom_to_binary(Error, utf8);
     true -> iolist_to_binary(Error)
@@ -60,20 +65,22 @@ parse_error(Line, File, Error, Token) ->
   end,
 
   Message = <<BinError / binary, BinToken / binary >>,
-  raise(Line, File, 'Elixir.SyntaxError', Message).
+  raise(Meta, File, 'Elixir.SyntaxError', Message).
 
 %% Raised during compilation
 
-form_error(Line, File, Module, Desc) ->
+-spec form_error(non_neg_integer(), file:filename(), module(), any()) -> no_return().
+
+form_error(Meta, File, Module, Desc) ->
   Message = iolist_to_binary(format_error(Module, Desc)),
-  raise(Line, File, 'Elixir.CompileError', Message).
+  raise(Meta, File, 'Elixir.CompileError', Message).
 
 %% Shows a deprecation message
 
-deprecation(Line, File, Message) -> deprecation(Line, File, Message, []).
+deprecation(Meta, File, Message) -> deprecation(Meta, File, Message, []).
 
-deprecation(Line, File, Message, Args) ->
-  io:format(file_format(Line, File, io_lib:format(Message, Args))).
+deprecation(Meta, File, Message, Args) ->
+  io:format(file_format(?line(Meta), File, io_lib:format(Message, Args))).
 
 %% Handle warnings and errors (called during module compilation)
 
@@ -90,18 +97,27 @@ handle_file_warning(_, _File, { _Line, v3_kernel, bad_call }) -> [];
 %% Rewrite
 handle_file_warning(_, File, {Line,erl_lint,{undefined_behaviour_func,{Fun,Arity},Module}}) ->
   Kind    = protocol_or_behaviour(Module),
-  Raw     = "undefined ~s function ~s/~B (for ~s ~s)",
+  Raw     = "undefined ~ts function ~ts/~B (for ~ts ~ts)",
   Message = io_lib:format(Raw, [Kind, Fun, Arity, Kind, inspect(Module)]),
   io:format(file_format(Line, File, Message));
 
 handle_file_warning(_, File, {Line,erl_lint,{undefined_behaviour,Module}}) ->
-  Raw = io_lib:format("behaviour ~s undefined", [inspect(Module)]),
+  case elixir_compiler:get_opt(internal) of
+    true  -> [];
+    false ->
+      Message = io_lib:format("behaviour ~ts undefined", [inspect(Module)]),
+      io:format(file_format(Line, File, Message))
+  end;
 
-  Message = case erlang:function_exported(Module, behavior_info, 1) of
-    true  -> Raw ++ " (maybe you meant behaviour_info instead of behavior_info?)";
-    false -> Raw
-  end,
+handle_file_warning(_, _File, {Line,erl_lint,{unused_var,_Var}}) when Line =< 0 ->
+  [];
 
+handle_file_warning(_, File, {Line,erl_lint,{unused_var,Var}}) ->
+  Message = format_error(erl_lint, { unused_var, format_var(Var) }),
+  io:format(file_format(Line, File, Message));
+
+handle_file_warning(_, File, {Line,erl_lint,{shadowed_var,Var,Where}}) ->
+  Message = format_error(erl_lint, { shadowed_var, format_var(Var), Where }),
   io:format(file_format(Line, File, Message));
 
 %% Default behavior
@@ -112,45 +128,59 @@ handle_file_warning(_, File, {Line,Module,Desc}) ->
 handle_file_warning(File, Desc) ->
   handle_file_warning(false, File, Desc).
 
+-spec handle_file_error(file:filename(), {non_neg_integer(), module(), any()}) -> no_return().
+
 handle_file_error(File, {Line,Module,Desc}) ->
   form_error(Line, File, Module, Desc).
 
 %% Assertions
 
-assert_no_function_scope(_Line, _Kind, #elixir_scope{function=nil}) -> [];
-assert_no_function_scope(Line, Kind, S) ->
-  syntax_error(Line, S#elixir_scope.file, "cannot invoke ~s inside a function", [Kind]).
+assert_no_function_scope(_Meta, _Kind, #elixir_scope{function=nil}) -> [];
+assert_no_function_scope(Meta, Kind, S) ->
+  syntax_error(Meta, S#elixir_scope.file, "cannot invoke ~ts inside a function", [Kind]).
 
-assert_no_assign_or_guard_scope(Line, Kind, S) ->
-  assert_no_assign_scope(Line, Kind, S),
-  assert_no_guard_scope(Line, Kind, S).
+assert_no_match_or_guard_scope(Meta, Kind, S) ->
+  assert_no_match_scope(Meta, Kind, S),
+  assert_no_guard_scope(Meta, Kind, S).
 
-assert_no_assign_scope(Line, Kind, #elixir_scope{context=assign} = S) ->
-  syntax_error(Line, S#elixir_scope.file, "cannot invoke ~s inside assign", [Kind]);
-assert_no_assign_scope(_Line, _Kind, _S) -> [].
+assert_no_match_scope(Meta, Kind, #elixir_scope{context=match} = S) ->
+  syntax_error(Meta, S#elixir_scope.file, "cannot invoke ~ts inside match clause", [Kind]);
+assert_no_match_scope(_Meta, _Kind, _S) -> [].
 
-assert_no_guard_scope(Line, Kind, #elixir_scope{context=guard} = S) ->
-  syntax_error(Line, S#elixir_scope.file, "cannot invoke ~s inside guard", [Kind]);
-assert_no_guard_scope(_Line, _Kind, _S) -> [].
+assert_no_guard_scope(Meta, Kind, #elixir_scope{context=guard} = S) ->
+  syntax_error(Meta, S#elixir_scope.file, "cannot invoke ~ts inside guard", [Kind]);
+assert_no_guard_scope(_Meta, _Kind, _S) -> [].
 
-assert_module_scope(Line, Kind, #elixir_scope{module=nil,file=File}) ->
-  syntax_error(Line, File, "cannot invoke ~s outside module", [Kind]);
-assert_module_scope(_Line, _Kind, #elixir_scope{module=Module}) -> Module.
+assert_module_scope(Meta, Kind, #elixir_scope{module=nil,file=File}) ->
+  syntax_error(Meta, File, "cannot invoke ~ts outside module", [Kind]);
+assert_module_scope(_Meta, _Kind, #elixir_scope{module=Module}) -> Module.
 
-assert_function_scope(Line, Kind, #elixir_scope{function=nil,file=File}) ->
-  syntax_error(Line, File, "cannot invoke ~s outside function", [Kind]);
-assert_function_scope(_Line, _Kind, #elixir_scope{function=Function}) -> Function.
+assert_function_scope(Meta, Kind, #elixir_scope{function=nil,file=File}) ->
+  syntax_error(Meta, File, "cannot invoke ~ts outside function", [Kind]);
+assert_function_scope(_Meta, _Kind, #elixir_scope{function=Function}) -> Function.
 
 %% Helpers
 
-raise(Line, File, Kind, Message) ->
+raise(Meta, File, Kind, Message) when is_list(Meta) ->
+  raise(?line(Meta), File, Kind, Message);
+
+raise(none, File, Kind, Message) ->
+  raise(0, File, Kind, Message);
+
+raise(Line, File, Kind, Message) when is_integer(Line) ->
   Stacktrace = erlang:get_stacktrace(),
   Exception = Kind:new([{description, Message}, {file, iolist_to_binary(File)}, {line, Line}]),
   erlang:raise(error, Exception, Stacktrace).
 
+file_format(0, File, Message) ->
+  io_lib:format("~ts: ~ts~n", [File, Message]);
+
 file_format(Line, File, Message) ->
   io_lib:format("~ts:~w: ~ts~n", [File, Line, Message]).
 
+format_var(Var) ->
+  list_to_atom(lists:takewhile(fun(X) -> X /= $@ end, atom_to_list(Var))).
+
 format_error([], Desc) ->
   io_lib:format("~p", [Desc]);
 

lib/elixir/src/elixir_import.erl

@@ -6,7 +6,7 @@
   build_table/1, delete_table/1, record/4]).
 -include("elixir.hrl").
 
-table(Module) -> ?ELIXIR_ATOM_CONCAT([i, Module]).
+table(Module) -> ?atom_concat([i, Module]).
 
 build_table(Module) ->
   ets:new(table(Module), [set, named_table, public]).
@@ -34,30 +34,30 @@ recorded_locals(Module) ->
 %% Update the scope to consider the imports for aliases
 %% based on the given options and selector.
 
-import(Line, Ref, Opts, Selector, S) ->
+import(Meta, Ref, Opts, Selector, S) ->
   IncludeAll = (Selector == all) or (Selector == default),
 
   SF = case IncludeAll or (Selector == functions) of
     false -> S;
     true  ->
-      FunctionsFun = fun() -> remove_underscored(Selector, get_functions(Ref)) end,
-      Functions = calculate(Line, Ref, Opts,
-        S#elixir_scope.functions, FunctionsFun, S),
-      S#elixir_scope{functions=Functions}
+      FunctionsFun = fun(K) -> remove_underscored(K andalso Selector, get_functions(Ref)) end,
+      { Functions, TempF } = calculate(Meta, Ref, Opts,
+        S#elixir_scope.functions, S#elixir_scope.macro_functions, FunctionsFun, S),
+      S#elixir_scope{functions=Functions, macro_functions=TempF}
   end,
 
   SM = case IncludeAll or (Selector == macros) of
     false -> SF;
     true  ->
-      MacrosFun = fun() ->
+      MacrosFun = fun(K) ->
         case IncludeAll of
-          true  -> remove_underscored(Selector, get_optional_macros(Ref));
-          false -> get_macros(Line, Ref, SF)
+          true  -> remove_underscored(K andalso Selector, get_optional_macros(Ref));
+          false -> get_macros(Meta, Ref, SF)
         end
       end,
-      Macros = calculate(Line, Ref, Opts,
-        SF#elixir_scope.macros, MacrosFun, SF),
-      SF#elixir_scope{macros=Macros}
+      { Macros, TempM } = calculate(Meta, Ref, Opts,
+        SF#elixir_scope.macros, SF#elixir_scope.macro_macros, MacrosFun, SF),
+      SF#elixir_scope{macros=Macros, macro_macros=TempM}
   end,
 
   SM.
@@ -66,67 +66,65 @@ import(Line, Ref, Opts, Selector, S) ->
 
 %% Calculates the imports based on only and except
 
-calculate(Line, Key, Opts, Old, AvailableFun, S) ->
+calculate(Meta, Key, Opts, Old, Temp, AvailableFun, S) ->
   File = S#elixir_scope.file,
-  All = keydelete(Key, Old),
 
   New = case keyfind(only, Opts) of
     { only, RawOnly } ->
-      Only = expand_fun_arity(Line, only, RawOnly, S),
+      Only = expand_fun_arity(Meta, only, RawOnly, S),
       case Only -- get_exports(Key) of
         [{Name,Arity}|_] ->
           Tuple = { invalid_import, { Key, Name, Arity } },
-          elixir_errors:form_error(Line, File, ?MODULE, Tuple);
-        _ -> intersection(Only, AvailableFun())
+          elixir_errors:form_error(Meta, File, ?MODULE, Tuple);
+        _ ->
+          intersection(Only, AvailableFun(false))
       end;
     false ->
       case keyfind(except, Opts) of
-        false -> AvailableFun()   ;     
-        { except, [] } -> AvailableFun();
+        false -> AvailableFun(true);
+        { except, [] } -> AvailableFun(true);
         { except, RawExcept } ->
-          Except = expand_fun_arity(Line, except, RawExcept, S),
+          Except = expand_fun_arity(Meta, except, RawExcept, S),
           case keyfind(Key, Old) of
-            false -> AvailableFun() -- Except;
-            {Key,ToRemove} -> ToRemove -- Except
+            false -> AvailableFun(true) -- Except;
+            {Key,OldImports} -> OldImports -- Except
           end
       end
   end,
 
   %% Normalize the data before storing it
-  Set     = ordsets:from_list(New),
-  Final   = remove_internals(Set),
+  Set   = ordsets:from_list(New),
+  Final = remove_internals(Set),
 
   case Final of
-    [] -> All;
+    [] -> { keydelete(Key, Old), keydelete(Key, Temp) };
     _  ->
-      ensure_no_conflicts(Line, File, Final, keydelete(Key, S#elixir_scope.macros)),
-      ensure_no_conflicts(Line, File, Final, keydelete(Key, S#elixir_scope.functions)),
-      ensure_no_in_erlang_macro_conflict(Line, File, Key, Final, internal_conflict),
-      [{ Key, Final }|All]
+      ensure_no_in_erlang_macro_conflict(Meta, File, Key, Final, internal_conflict),
+      { [{ Key, Final }|keydelete(Key, Old)], [{ Key, Final }|keydelete(Key, Temp)] }
   end.
 
 %% Ensure we are expanding macros and stuff
 
-expand_fun_arity(Line, Kind, Value, S) ->
+expand_fun_arity(Meta, Kind, Value, S) ->
   { TValue, _S } = elixir_translator:translate_each(Value, S),
-  cons_to_keywords(Line, Kind, TValue, S).
+  cons_to_keywords(Meta, Kind, TValue, S).
 
-cons_to_keywords(Line, Kind, { cons, _, Left, { nil, _ } }, S) ->
-  [tuple_to_fun_arity(Line, Kind, Left, S)];
+cons_to_keywords(Meta, Kind, { cons, _, Left, { nil, _ } }, S) ->
+  [tuple_to_fun_arity(Meta, Kind, Left, S)];
 
-cons_to_keywords(Line, Kind, { cons, _, Left, Right }, S) ->
-  [tuple_to_fun_arity(Line, Kind, Left, S)|cons_to_keywords(Line, Kind, Right, S)];
+cons_to_keywords(Meta, Kind, { cons, _, Left, Right }, S) ->
+  [tuple_to_fun_arity(Meta, Kind, Left, S)|cons_to_keywords(Meta, Kind, Right, S)];
 
-cons_to_keywords(Line, Kind, _, S) ->
-  elixir_errors:syntax_error(Line, S#elixir_scope.file,
-    "invalid value for :~s, expected a list with functions and arities", [Kind]).
+cons_to_keywords(Meta, Kind, _, S) ->
+  elixir_errors:syntax_error(Meta, S#elixir_scope.file,
+    "invalid value for :~ts, expected a list with functions and arities", [Kind]).
 
-tuple_to_fun_arity(_Line, _Kind, { tuple, _, [{ atom, _, Atom }, { integer, _, Integer }] }, _S) ->
+tuple_to_fun_arity(_Meta, _Kind, { tuple, _, [{ atom, _, Atom }, { integer, _, Integer }] }, _S) ->
   { Atom, Integer };
 
-tuple_to_fun_arity(Line, Kind, _, S) ->
-  elixir_errors:syntax_error(Line, S#elixir_scope.file,
-    "invalid value for :~s, expected a list with functions and arities", [Kind]).
+tuple_to_fun_arity(Meta, Kind, _, S) ->
+  elixir_errors:syntax_error(Meta, S#elixir_scope.file,
+    "invalid value for :~ts, expected a list with functions and arities", [Kind]).
 
 %% Retrieve functions and macros from modules
 
@@ -144,13 +142,13 @@ get_functions(Module) ->
     error:undef -> Module:module_info(exports)
   end.
 
-get_macros(Line, Module, S) ->
+get_macros(Meta, Module, S) ->
   try
     Module:'__info__'(macros)
   catch
     error:undef ->
       Tuple = { no_macros, Module },
-      elixir_errors:form_error(Line, S#elixir_scope.file, ?MODULE, Tuple)
+      elixir_errors:form_error(Meta, S#elixir_scope.file, ?MODULE, Tuple)
   end.
 
 get_optional_macros(Module)  ->
@@ -170,13 +168,13 @@ get_optional_macros(Module)  ->
 %% Elixir "implemented in Erlang" macro. Checking if a local
 %% conflicts with an import is automatically done by Erlang.
 
-ensure_no_local_conflict(Line, File, Module, AllDefined) ->
-  ensure_no_in_erlang_macro_conflict(Line, File, Module, AllDefined, local_conflict).
+ensure_no_local_conflict(Meta, File, Module, AllDefined) ->
+  ensure_no_in_erlang_macro_conflict(Meta, File, Module, AllDefined, local_conflict).
 
 %% Find conlicts in the given list of functions with
 %% the recorded set of imports.
 
-ensure_no_import_conflict(Line, File, Module, AllDefined) ->
+ensure_no_import_conflict(Meta, File, Module, AllDefined) ->
   Table = table(Module),
   Matches = [X || X <- AllDefined, ets:member(Table, X)],
 
@@ -184,7 +182,7 @@ ensure_no_import_conflict(Line, File, Module, AllDefined) ->
     [{Name,Arity}|_] ->
       Key = ets:lookup_element(Table, {Name, Arity }, 2),
       Tuple = { import_conflict, { Key, Name, Arity } },
-      elixir_errors:form_error(Line, File, ?MODULE, Tuple);
+      elixir_errors:form_error(Meta, File, ?MODULE, Tuple);
     [] ->
       ok
   end.
@@ -192,7 +190,7 @@ ensure_no_import_conflict(Line, File, Module, AllDefined) ->
 %% Ensure the given functions don't clash with any
 %% of Elixir non overridable macros.
 
-ensure_no_in_erlang_macro_conflict(Line, File, Key, [{Name,Arity}|T], Reason) ->
+ensure_no_in_erlang_macro_conflict(Meta, File, Key, [{Name,Arity}|T], Reason) ->
   Values = lists:filter(fun({X,Y}) ->
     (Name == X) andalso ((Y == '*') orelse (Y == Arity))
   end, non_overridable_macros()),
@@ -200,50 +198,34 @@ ensure_no_in_erlang_macro_conflict(Line, File, Key, [{Name,Arity}|T], Reason) ->
   case Values /= [] of
     true  ->
       Tuple = { Reason, { Key, Name, Arity } },
-      elixir_errors:form_error(Line, File, ?MODULE, Tuple);
-    false -> ensure_no_in_erlang_macro_conflict(Line, File, Key, T, Reason)
+      elixir_errors:form_error(Meta, File, ?MODULE, Tuple);
+    false -> ensure_no_in_erlang_macro_conflict(Meta, File, Key, T, Reason)
   end;
 
-ensure_no_in_erlang_macro_conflict(_Line, _File, _Key, [], _) -> ok.
-
-%% Find conlicts in the given list of functions with the set of imports.
-%% Used internally to ensure a newly imported fun or macro does not
-%% conflict with an already imported set.
-
-ensure_no_conflicts(Line, File, Functions, [{Key,Value}|T]) ->
-  Filtered = lists:filter(fun(X) -> lists:member(X, Functions) end, Value),
-  case Filtered of
-    [{Name,Arity}|_] ->
-      Tuple = { already_imported, { Key, Name, Arity } },
-      elixir_errors:form_error(Line, File, ?MODULE, Tuple);
-    [] ->
-      ensure_no_conflicts(Line, File, Functions, T)
-  end;
-
-ensure_no_conflicts(_Line, _File, _Functions, _S) -> ok.
+ensure_no_in_erlang_macro_conflict(_Meta, _File, _Key, [], _) -> ok.
 
 %% ERROR HANDLING
 
 format_error({already_imported,{Receiver, Name, Arity}}) ->
-  io_lib:format("function ~s/~B already imported from ~s", [Name, Arity, elixir_errors:inspect(Receiver)]);
+  io_lib:format("function ~ts/~B already imported from ~ts", [Name, Arity, elixir_errors:inspect(Receiver)]);
 
 format_error({invalid_import,{Receiver, Name, Arity}}) ->
-  io_lib:format("cannot import ~s.~s/~B because it doesn't exist",
+  io_lib:format("cannot import ~ts.~ts/~B because it doesn't exist",
     [elixir_errors:inspect(Receiver), Name, Arity]);
 
 format_error({import_conflict,{Receiver, Name, Arity}}) ->
-  io_lib:format("imported ~s.~s/~B conflicts with local function",
+  io_lib:format("imported ~ts.~ts/~B conflicts with local function",
     [elixir_errors:inspect(Receiver), Name, Arity]);
 
 format_error({local_conflict,{_, Name, Arity}}) ->
-  io_lib:format("cannot define local ~s/~B because it conflicts with Elixir internal macros", [Name, Arity]);
+  io_lib:format("cannot define local ~ts/~B because it conflicts with Elixir special forms", [Name, Arity]);
 
 format_error({internal_conflict,{Receiver, Name, Arity}}) ->
-  io_lib:format("cannot import ~s.~s/~B because it conflicts with Elixir internal macros",
+  io_lib:format("cannot import ~ts.~ts/~B because it conflicts with Elixir special forms",
     [elixir_errors:inspect(Receiver), Name, Arity]);
 
 format_error({ no_macros, Module }) ->
-  io_lib:format("could not load macros from module ~s", [elixir_errors:inspect(Module)]).
+  io_lib:format("could not load macros from module ~ts", [elixir_errors:inspect(Module)]).
 
 %% LIST HELPERS
 
@@ -317,5 +299,8 @@ non_overridable_macros() ->
     {'super','*'},
     {'super?',0},
     {'bc','*'},
-    {'lc','*'}
+    {'lc','*'},
+    {'var!',1},
+    {'var!',2},
+    {'alias!',1}
   ].

lib/elixir/src/elixir_interpolation.erl

@@ -2,7 +2,6 @@
 -module(elixir_interpolation).
 -export([extract/5, unescape_chars/1, unescape_chars/2,
 unescape_tokens/1, unescape_tokens/2, unescape_map/1]).
--define(is_octal(S), S >= $0 andalso S =< $7).
 -include("elixir.hrl").
 -compile({parse_transform, elixir_transform}).
 
@@ -20,8 +19,8 @@ extract(Line, _File, _Interpol, [], _Buffer, [], _Output, Last) ->
 extract(Line, File, _Interpol, [Last|Remaining], Buffer, [], Output, Last) ->
   finish_extraction(Line, File, Buffer, Output, Remaining);
 
-extract(Line, _File, _Interpol, [Last], _Buffer, Search, _Output, Last) ->
-  { error, { Line, io_lib:format("unexpected end of string: ~ts", [[hd(Search)]]), [Last] } };
+extract(Line, _File, _Interpol, End, _Buffer, _Search, _Output, Last) when End == [Last]; End == [] ->
+  { error, { Line, io_lib:format("missing terminator: ~ts", [[Last]]), [] } };
 
 extract(Line, File, Interpol, [$\n|Rest], Buffer, Search, Output, Last) ->
   extract(Line+1, File, Interpol, Rest, [$\n|Buffer], Search, Output, Last);
@@ -40,14 +39,17 @@ extract(Line, File, true, [$}|Rest], Buffer, [$}], Output, Last) ->
   NewOutput = build_interpol(i, Line, File, Buffer, Output),
   extract(Line, File, true, Rest, [], [], NewOutput, Last);
 
-%% Check for available separators "", {}, [] and () inside interpolation
+%% Check for available separators inside interpolation
 
-extract(Line, File, Interpol, [C|Rest], Buffer, [C|Search], Output, Last) when C == $); C == $]; C == $}; C == $"; C == $' ->
+extract(Line, File, Interpol, [C|Rest], Buffer, [C|Search], Output, Last) when C == $); C == $]; C == $}; C == $>; C == $"; C == $' ->
   extract(Line, File, Interpol, Rest, [C|Buffer], Search, Output, Last);
 
 extract(Line, File, Interpol, [C|Rest], Buffer, [_|_] = Search, Output, Last) when C == $"; C == $' ->
   extract(Line, File, Interpol, Rest, [C|Buffer], [C|Search], Output, Last);
 
+extract(Line, File, Interpol, [$<|Rest], Buffer, [_|_] = Search, Output, Last) ->
+  extract(Line, File, Interpol, Rest, [$<|Buffer], [$>|Search], Output, Last);
+
 extract(Line, File, Interpol, [${|Rest], Buffer, [_|_] = Search, Output, Last) ->
   extract(Line, File, Interpol, Rest, [${|Buffer], [$}|Search], Output, Last);
 
@@ -75,45 +77,65 @@ unescape_token(Other, _Map) -> Other.
 
 % Unescape chars. For instance, "\" "n" (two chars) needs to be converted to "\n" (one char).
 
--define(to_octal(List),
-
-).
-
 unescape_chars(String) ->
   unescape_chars(String, fun unescape_map/1).
 
 unescape_chars(String, Map) ->
-  Octals = case Map($0) of
-    false -> false;
-    _ -> true
-  end,
-  unescape_chars(String, Map, Octals, <<>>).
+  Octals = Map($0) /= false,
+  Hex    = Map($x) /= false,
+  unescape_chars(String, Map, Octals, Hex, <<>>).
 
-unescape_chars(<<$\\,A,B,C,Rest/binary>>, Map, true, Acc) when ?is_octal(A), ?is_octal(B), ?is_octal(C) ->
-  to_octal(Rest, Map, [A,B,C], Acc);
+unescape_chars(<<$\\,A,B,C,Rest/binary>>, Map, true, Hex, Acc) when ?is_octal(A), A =< $3, ?is_octal(B), ?is_octal(C) ->
+  append_escaped(Rest, Map, [A,B,C], true, Hex, Acc, 8);
 
-unescape_chars(<<$\\,A,B,Rest/binary>>, Map, true, Acc) when ?is_octal(A), ?is_octal(B) ->
-  to_octal(Rest, Map, [A,B], Acc);
+unescape_chars(<<$\\,A,B,Rest/binary>>, Map, true, Hex, Acc) when ?is_octal(A), ?is_octal(B) ->
+  append_escaped(Rest, Map, [A,B], true, Hex, Acc, 8);
 
-unescape_chars(<<$\\,A,Rest/binary>>, Map, true, Acc) when ?is_octal(A) ->
-  to_octal(Rest, Map, [A], Acc);
+unescape_chars(<<$\\,A,Rest/binary>>, Map, true, Hex, Acc) when ?is_octal(A) ->
+  append_escaped(Rest, Map, [A], true, Hex, Acc, 8);
 
-unescape_chars(<<$\\,Escaped,Rest/binary>>, Map, Octals, Acc) ->
+unescape_chars(<<$\\,P,A,B,Rest/binary>>, Map, Octal, true, Acc) when (P == $x orelse P == $X), ?is_hex(A), ?is_hex(B) ->
+  append_escaped(Rest, Map, [A,B], Octal, true, Acc, 16);
+
+unescape_chars(<<$\\,P,A,Rest/binary>>, Map, Octal, true, Acc) when (P == $x orelse P == $X), ?is_hex(A) ->
+  append_escaped(Rest, Map, [A], Octal, true, Acc, 16);
+
+unescape_chars(<<$\\,P,${,A,$},Rest/binary>>, Map, Octal, true, Acc) when (P == $x orelse P == $X), ?is_hex(A) ->
+  append_escaped(Rest, Map, [A], Octal, true, Acc, 16);
+
+unescape_chars(<<$\\,P,${,A,B,$},Rest/binary>>, Map, Octal, true, Acc) when (P == $x orelse P == $X), ?is_hex(A), ?is_hex(B) ->
+  append_escaped(Rest, Map, [A,B], Octal, true, Acc, 16);
+
+unescape_chars(<<$\\,P,${,A,B,C,$},Rest/binary>>, Map, Octal, true, Acc) when (P == $x orelse P == $X), ?is_hex(A), ?is_hex(B), ?is_hex(C) ->
+  append_escaped(Rest, Map, [A,B,C], Octal, true, Acc, 16);
+
+unescape_chars(<<$\\,P,${,A,B,C,D,$},Rest/binary>>, Map, Octal, true, Acc) when (P == $x orelse P == $X), ?is_hex(A), ?is_hex(B), ?is_hex(C), ?is_hex(D) ->
+  append_escaped(Rest, Map, [A,B,C,D], Octal, true, Acc, 16);
+
+unescape_chars(<<$\\,P,${,A,B,C,D,E,$},Rest/binary>>, Map, Octal, true, Acc) when (P == $x orelse P == $X), ?is_hex(A), ?is_hex(B), ?is_hex(C), ?is_hex(D), ?is_hex(E) ->
+    append_escaped(Rest, Map, [A,B,C,D,E], Octal, true, Acc, 16);
+
+unescape_chars(<<$\\,P,${,A,B,C,D,E,F,$},Rest/binary>>, Map, Octal, true, Acc) when (P == $x orelse P == $X), ?is_hex(A), ?is_hex(B), ?is_hex(C), ?is_hex(D), ?is_hex(E), ?is_hex(F) ->
+  append_escaped(Rest, Map, [A,B,C,D,E,F], Octal, true, Acc, 16);
+
+unescape_chars(<<$\\,Escaped,Rest/binary>>, Map, Octals, Hex, Acc) ->
   case Map(Escaped) of
-    false -> unescape_chars(Rest, Map, Octals, <<Acc/binary, $\\, Escaped>>);
-    Other -> unescape_chars(Rest, Map, Octals, <<Acc/binary, Other>>)
+    false -> unescape_chars(Rest, Map, Octals, Hex, <<Acc/binary, $\\, Escaped>>);
+    Other -> unescape_chars(Rest, Map, Octals, Hex, <<Acc/binary, Other>>)
   end;
 
-unescape_chars(<<Char, Rest/binary>>, Map, Octals, Acc) ->
-  unescape_chars(Rest, Map, Octals, <<Acc/binary, Char>>);
+unescape_chars(<<Char, Rest/binary>>, Map, Octals, Hex, Acc) ->
+  unescape_chars(Rest, Map, Octals, Hex, <<Acc/binary, Char>>);
 
-unescape_chars(<<>>, _Map, _Octals, Acc) -> Acc.
+unescape_chars(<<>>, _Map, _Octals, _Hex, Acc) -> Acc.
 
-to_octal(Rest, Map, Octal, Acc) ->
-  unescape_chars(Rest, Map, true, <<Acc/binary, (list_to_integer(Octal, 8))/integer>>).
+append_escaped(Rest, Map, List, Octal, Hex, Acc, Base) ->
+  Codepoint = list_to_integer(List, Base),
+  unescape_chars(Rest, Map, Octal, Hex, <<Acc/binary, Codepoint/utf8>>).
 
 % Unescape Helpers
 
+unescape_map($a) -> 7;
 unescape_map($b) -> $\b;
 unescape_map($d) -> $\d;
 unescape_map($e) -> $\e;
@@ -147,12 +169,13 @@ wrap_interpol(_Line, Form) when is_binary(Form) ->
   Form;
 
 wrap_interpol(Line, Form) ->
-  { '::', Line, [{ { '.', Line, ['Elixir.Binary.Chars', to_binary] }, Line, [Form]}, { binary, Line, nil }]}.
+  Meta = [{line,Line}],
+  { '::', Meta, [{ { '.', Meta, ['Elixir.Binary.Chars', to_binary] }, Meta, [Form]}, { binary, Meta, nil }]}.
 
 forms(String, StartLine, File) ->
   case elixir_translator:forms(String, StartLine, File, []) of
     { ok, [] } -> nil;
     { ok, [Forms] } when not is_list(Forms) -> Forms;
-    { ok, Forms } -> { '__block__', StartLine, Forms };
+    { ok, Forms } -> { '__block__', [{line,StartLine}], Forms };
     { error, Tuple } -> throw({ interpolation_error, Tuple })
-  end.
+  end.

lib/elixir/src/elixir_literal.erl

@@ -5,31 +5,31 @@
 -include("elixir.hrl").
 -compile({parse_transform, elixir_transform}).
 
-translate({ '<<>>', Line, Args } = Original, S) when is_list(Args) ->
+translate({ '<<>>', Meta, Args } = Original, S) when is_list(Args) ->
   case elixir_partials:handle(Original, S, allow_tail) of
     error ->
       case S#elixir_scope.context of
-        assign ->
-          build_bitstr(fun elixir_translator:translate_each/2, Args, Line, S);
+        match ->
+          build_bitstr(fun elixir_translator:translate_each/2, Args, Meta, S);
         _ ->
-          { TArgs, { SC, SV } } = build_bitstr(fun elixir_translator:translate_arg/2, Args, Line, { S, S }),
+          { TArgs, { SC, SV } } = build_bitstr(fun elixir_translator:translate_arg/2, Args, Meta, { S, S }),
           { TArgs, umergec(SV, SC) }
       end;
     Else -> Else
   end;
 
-translate({ '{}', Line, Args } = Original, S) when is_list(Args) ->
+translate({ '{}', Meta, Args } = Original, S) when is_list(Args) ->
   case elixir_partials:handle(Original, S) of
     error ->
       { TArgs, SE } = translate_args(Args, S),
-      { {tuple, Line, TArgs}, SE };
+      { { tuple, ?line(Meta), TArgs }, SE };
     Else -> Else
   end;
 
-translate({ '[]', _Line, [] }, S) ->
+translate({ '[]', _Meta, [] }, S) ->
   { { nil, 0 }, S };
 
-translate({ '[]', _Line, Args } = Original, S) when is_list(Args) ->
+translate({ '[]', _Meta, Args } = Original, S) when is_list(Args) ->
   case elixir_partials:handle(Original, S, allow_tail) of
     error ->
       [RTail|RArgs] = lists:reverse(Args),
@@ -47,7 +47,7 @@ translate({ '[]', _Line, Args } = Original, S) when is_list(Args) ->
       end,
 
       { FExprs, FS } = case S#elixir_scope.context of
-        assign ->
+        match ->
           elixir_tree_helpers:build_reverse_list(fun elixir_translator:translate_each/2, Exprs, 0, ListS, Tail);
         _ ->
           { TArgs, { SC, SV } } =
@@ -60,10 +60,10 @@ translate({ '[]', _Line, Args } = Original, S) when is_list(Args) ->
   end;
 
 translate({ Left, Right }, S) ->
-  translate({ '{}', 0, [Left, Right]}, S);
+  translate({ '{}', [], [Left, Right]}, S);
 
 translate(Args, S) when is_list(Args) ->
-  translate({ '[]', 0, Args }, S);
+  translate({ '[]', [], Args }, S);
 
 translate(Number, S) when is_integer(Number) ->
   { { integer, 0, Number }, S };
@@ -76,109 +76,115 @@ translate(Atom, S) when is_atom(Atom) ->
 
 translate(Bitstring, S) when is_bitstring(Bitstring) ->
   { elixir_tree_helpers:abstract_syntax(Bitstring), S }.
-  
+
 %% Helpers
 
 % Build a bitstring taking into accounts the following types:
 %
 % * If a bitstring or a list is given, we just append its items
-% * If '|' is given, extract the bitstring information
+% * If '::' is given, extract the bitstring information
 % * All the other types are simply translated and handled with Erlang's default
 %
-build_bitstr(Fun, Exprs, Line, S) ->
-  { Final, FinalS } = build_bitstr_each(Fun, Exprs, Line, S, []),
-  { { bin, Line, lists:reverse(Final) }, FinalS }.
+build_bitstr(Fun, Exprs, Meta, S) ->
+  { Final, FinalS } = build_bitstr_each(Fun, Exprs, Meta, S, []),
+  { { bin, ?line(Meta), lists:reverse(Final) }, FinalS }.
 
-build_bitstr_each(_Fun, [], _Line, S, Acc) ->
+build_bitstr_each(_Fun, [], _Meta, S, Acc) ->
   { Acc, S };
 
-build_bitstr_each(Fun, [H|T], Line, S, Acc) when is_list(H) ->
-  { NewAcc, NewS } = build_bitstr_each(Fun, H, Line, S, Acc),
-  build_bitstr_each(Fun, T, Line, NewS, NewAcc);
-
-build_bitstr_each(Fun, [H|T], Line, S, Acc) when is_bitstring(H) ->
-  { bin, _, Elements } = elixir_tree_helpers:abstract_syntax(H),
-  NewAcc = lists:foldl(fun(Element, FinalAcc) -> [Element|FinalAcc] end, Acc, Elements),
-  build_bitstr_each(Fun, T, Line, S, NewAcc);
-
-build_bitstr_each(Fun, [{'::',_,[H,V]}|T], Line, S, Acc) ->
-  { Expr, NS } = Fun(H, S),
-
+build_bitstr_each(Fun, [{'::',_,[H,V]}|T], Meta, S, Acc) ->
   %% Variables defined outside the binary can be accounted
   %% on subparts, however we can't assign new variables.
-  case NS of
-    { ES, _ } -> [];                       %% translate_arg,  no assigns
-    _ -> ES = NS#elixir_scope{context=nil} %% translate_each, revert assigns
+  case S of
+    { ES, _ } -> [];                      %% translate_arg,  no assigns
+    _ -> ES = S#elixir_scope{context=nil} %% translate_each, revert assigns
   end,
 
-  { Size, Types } = extract_bit_values(Line, V, ES),
-  build_bitstr_each(Fun, T, Line, NS, [{ bin_element, Line, Expr, Size, Types }|Acc]);
+  { Size, Types } = extract_bit_values(Meta, V, ES),
+  build_bitstr_each(Fun, T, Meta, S, Acc, H, Size, Types);
 
-%% Deprecated syntax
-build_bitstr_each(Fun, [{'|',_,[H,V]}|T], Line, S, Acc) ->
+build_bitstr_each(Fun, [H|T], Meta, S, Acc) ->
+  build_bitstr_each(Fun, T, Meta, S, Acc, H, default, default).
+
+build_bitstr_each(Fun, T, Meta, S, Acc, H, Size, Types) when is_list(H) ->
+  case is_default_or_utf(Types) of
+    true ->
+      { NewAcc, NewS } = lists:foldl(fun(L, { LA, LS }) ->
+        { FL, FS } = Fun(L, LS),
+        { [{ bin_element, ?line(Meta), FL, Size, Types }|LA], FS }
+      end, { Acc, S }, H),
+      build_bitstr_each(Fun, T, Meta, NewS, NewAcc);
+    false ->
+      build_bitstr_default(Fun, T, Meta, S, Acc, H, Size, Types)
+  end;
+
+build_bitstr_each(Fun, T, Meta, S, Acc, H, Size, Types) when is_bitstring(H) ->
+  case is_default_or_utf(Types) of
+    true ->
+      Line = ?line(Meta),
+      { bin, _, Elements } = elixir_tree_helpers:abstract_syntax(H),
+      NewAcc = lists:foldl(fun({ bin_element, _, Expr, _, _ }, FinalAcc) ->
+        [{ bin_element, Line, Expr, Size, Types }|FinalAcc]
+      end, Acc, Elements),
+      build_bitstr_each(Fun, T, Meta, S, NewAcc);
+    false ->
+      build_bitstr_default(Fun, T, Meta, S, Acc, H, Size, Types)
+  end;
+
+build_bitstr_each(Fun, T, Meta, S, Acc, H, Size, Types) ->
+  build_bitstr_default(Fun, T, Meta, S, Acc, H, Size, Types).
+
+build_bitstr_default(Fun, T, Meta, S, Acc, H, Size, Types) ->
   { Expr, NS } = Fun(H, S),
+  build_bitstr_each(Fun, T, Meta, NS, [{ bin_element, ?line(Meta), Expr, Size, Types }|Acc]).
 
-  %% Just variables defined outside the binary can be accounted on subparts
-  case NS of
-    { ES, _ } -> [];
-    ES -> []
-  end,
-
-  elixir_errors:deprecation(Line, ES#elixir_scope.file, "Old bitstring syntax with | is deprecated, use the new one with :: instead"),
-
-  %% Assigns can be made in subparts
-  { Int, Types } = extract_bin_values(Line, V, default, [], ES#elixir_scope{context=nil}),
-
-  Final = case Types of
-    [] -> default;
-    _  -> lists:reverse(Types)
-  end,
-
-  build_bitstr_each(Fun, T, Line, NS, [{ bin_element, Line, Expr, Int, Final }|Acc]);
-
-build_bitstr_each(Fun, [H|T], Line, S, Acc) ->
-  { Expr, NS } = Fun(H, S),
-  build_bitstr_each(Fun, T, Line, NS, [{ bin_element, Line, Expr, default, default }|Acc]).
+is_default_or_utf(default) -> true;
+is_default_or_utf([UTF|_]) when UTF == utf8; UTF == utf16; UTF == utf32 -> true;
+is_default_or_utf([_|T]) -> is_default_or_utf(T);
+is_default_or_utf([]) -> false.
 
 %% Extra bitstring specifiers
 
-extract_bit_values(Line, V, S) when is_list(V) ->
-  extract_bit_values(Line, V, default, [], S);
+extract_bit_values(Meta, V, S) when is_list(V) ->
+  extract_bit_values(Meta, V, default, [], S);
 
-extract_bit_values(Line, V, S) ->
-  extract_bit_values(Line, [V], S).
+extract_bit_values(Meta, V, S) ->
+  extract_bit_values(Meta, [V], S).
 
-extract_bit_values(Line, [{ Value, _CallLine, Atom }|T] = All, Size, Types, S)
+extract_bit_values(Meta, [{ Value, _Meta, Atom }|T] = All, Size, Types, S)
     when is_atom(Value) andalso (is_atom(Atom) orelse Atom == []) ->
   case extract_bit_type(Value, Types) of
     { error, unknown } ->
-      handle_unknown_specifier(Line, All, Size, Types, S);
+      handle_unknown_specifier(Meta, All, Size, Types, S);
     NewTypes when is_list(NewTypes) ->
-      extract_bit_values(Line, T, Size, NewTypes, S)
+      extract_bit_values(Meta, T, Size, NewTypes, S)
   end;
 
-extract_bit_values(Line, [{ Value, CallLine, [_] = Args }|T] = All, Size, Types, S) when is_atom(Value) ->
+extract_bit_values(Meta, [{ Value, CallMeta, [_] = Args }|T] = All, Size, Types, S) when is_atom(Value) ->
   { TArgs, _ } = elixir_translator:translate_args(Args, S),
   case extract_bit_type_or_size(Value, TArgs, Size, Types) of
     { error, unknown } ->
-      handle_unknown_specifier(Line, All, Size, Types, S);
+      handle_unknown_specifier(Meta, All, Size, Types, S);
     { error, Msg } ->
-      elixir_errors:syntax_error(CallLine, S#elixir_scope.file, Msg);
+      elixir_errors:syntax_error(CallMeta, S#elixir_scope.file, Msg);
     { NewSize, NewTypes } ->
-      extract_bit_values(Line, T, NewSize, NewTypes, S)
+      extract_bit_values(Meta, T, NewSize, NewTypes, S)
   end;
 
-extract_bit_values(Line, [{ '|', _, [Left, Right] }], Size, Types, S) ->
-  Expanded = 'Elixir.Macro':expand(Right, elixir_scope:to_ex_env({ Line, S })),
-  extract_bit_values(Line, [Left|join_expansion(Expanded,[])], Size, Types, S);
+extract_bit_values(Meta, [Size|T], default, Types, S) when is_integer(Size) andalso Size >= 0 ->
+  extract_bit_values(Meta, T, {integer, Meta, Size}, Types, S);
 
-extract_bit_values(Line, [_|_] = All, Size, Types, S) ->
-  handle_unknown_specifier(Line, All, Size, Types, S);
+extract_bit_values(Meta, [{ '|', _, [Left, Right] }], Size, Types, S) ->
+  Expanded = 'Elixir.Macro':expand(Right, elixir_scope:to_ex_env({ ?line(Meta), S })),
+  extract_bit_values(Meta, [Left|join_expansion(Expanded,[])], Size, Types, S);
 
-extract_bit_values(_Line, [], Size, [], _S) ->
+extract_bit_values(Meta, [_|_] = All, Size, Types, S) ->
+  handle_unknown_specifier(Meta, All, Size, Types, S);
+
+extract_bit_values(_Meta, [], Size, [], _S) ->
   { Size, default };
 
-extract_bit_values(_Line, [], Size, Types, _S) ->
+extract_bit_values(_Meta, [], Size, Types, _S) ->
   { Size, lists:reverse(Types) }.
 
 extract_bit_type(Value, Types) when
@@ -210,36 +216,14 @@ extract_bit_type_or_size(unit, _Args, _Other, _Types) ->
 extract_bit_type_or_size(_Value, _Args, _Other, _Types) ->
   { error, unknown }.
 
-handle_unknown_specifier(Line, [H|T], Size, Types, S) ->
-  case 'Elixir.Macro':expand(H, elixir_scope:to_ex_env({ Line, S })) of
+handle_unknown_specifier(Meta, [H|T], Size, Types, S) ->
+  case 'Elixir.Macro':expand(H, elixir_scope:to_ex_env({ ?line(Meta), S })) of
     H ->
-      elixir_errors:syntax_error(Line, S#elixir_scope.file, "unknown bitstring specifier ~s", ['Elixir.Macro':to_binary(H)]);
+      elixir_errors:syntax_error(Meta, S#elixir_scope.file, "unknown bitstring specifier ~ts", ['Elixir.Macro':to_binary(H)]);
     E ->
-      extract_bit_values(Line, join_expansion(E,T), Size, Types, S)
+      extract_bit_values(Meta, join_expansion(E,T), Size, Types, S)
   end.
 
 join_expansion({ '__block__', _, [Expanded] }, Tail) -> join_expansion(Expanded, Tail);
 join_expansion(Expanded, Tail) when is_list(Expanded) -> Expanded ++ Tail;
 join_expansion(Expanded, Tail) -> [Expanded|Tail].
-
-%% Deprecated specifiers
-
-extract_bin_values(Line, { '-', _Line, [Left, Right] }, Int, Types, S) ->
-  { LInt, LTypes } = extract_bin_values(Line, Left, Int, Types, S),
-  extract_bin_values(Line, Right, LInt, LTypes, S);
-
-extract_bin_values(Line, Value, default, Types, _S) when is_integer(Value) ->
-  { { integer, Line, Value }, Types };
-
-extract_bin_values(_Line, { Value, _, Atom } = Expr, default, Types, S) when is_atom(Value), is_atom(Atom) ->
-  Translated = element(1, elixir_translator:translate_each(Expr, S)),
-  { Translated, Types };
-
-extract_bin_values(Line, Value, _Int, _Types, S) when is_integer(Value) ->
-  elixir_errors:syntax_error(Line, S#elixir_scope.file, "duplicated size specifier ~p in <<>>", [Value]);
-
-extract_bin_values(_Line, Value, Int, Types, _S) when is_atom(Value); is_tuple(Value), tuple_size(Value) == 2 ->
-  { Int, [Value|Types] };
-
-extract_bin_values(Line, _Value, _Int, _Types, S) ->
-  elixir_errors:syntax_error(Line, S#elixir_scope.file, "invalid specifier for <<>>").

lib/elixir/src/elixir_macros.erl

@@ -5,129 +5,149 @@
 -import(elixir_translator, [translate_each/2, translate_args/2, translate_apply/7]).
 -import(elixir_scope, [umergec/2]).
 -import(elixir_errors, [syntax_error/3, syntax_error/4,
-  assert_no_function_scope/3, assert_module_scope/3, assert_no_assign_or_guard_scope/3]).
+  assert_no_function_scope/3, assert_module_scope/3, assert_no_match_or_guard_scope/3]).
 -include("elixir.hrl").
 
--define(FUNS(), Kind == def; Kind == defp; Kind == defmacro; Kind == defmacrop).
+-define(FUNS(Kind), Kind == def; Kind == defp; Kind == defmacro; Kind == defmacrop).
+-define(IN_TYPES(Kind), Kind == atom orelse Kind == integer orelse Kind == float).
+
 -compile({parse_transform, elixir_transform}).
 
 %% Operators
 
-translate({ '+', _Line, [Expr] }, S) when is_number(Expr) ->
+translate({ '+', _Meta, [Expr] }, S) when is_number(Expr) ->
   translate_each(Expr, S);
 
-translate({ '-', _Line, [Expr] }, S) when is_number(Expr) ->
+translate({ '-', _Meta, [Expr] }, S) when is_number(Expr) ->
   translate_each(-1 * Expr, S);
 
-translate({ Op, Line, Exprs }, S) when is_list(Exprs),
+translate({ Op, Meta, Exprs }, S) when is_list(Exprs),
     Op == '<-' orelse Op == '--' ->
-  assert_no_assign_or_guard_scope(Line, Op, S),
-  translate_each({ '__op__', Line, [Op|Exprs] }, S);
+  assert_no_match_or_guard_scope(Meta, Op, S),
+  translate_each({ '__op__', Meta, [Op|Exprs] }, S);
 
-translate({ Op, Line, Exprs }, S) when is_list(Exprs),
+translate({ Op, Meta, Exprs }, S) when is_list(Exprs),
     Op == '+'   orelse Op == '-'   orelse Op == '*'   orelse Op == '/' orelse
     Op == '++'  orelse Op == 'not' orelse Op == 'and' orelse Op == 'or' orelse
     Op == 'xor' orelse Op == '<'   orelse Op == '>'   orelse Op == '<=' orelse
     Op == '>='  orelse Op == '=='  orelse Op == '!='  orelse Op == '===' orelse
     Op == '!==' ->
-  translate_each({ '__op__', Line, [Op|Exprs] }, S);
+  translate_each({ '__op__', Meta, [Op|Exprs] }, S);
 
-translate({ '!', Line, [{ '!', _, [Expr] }] }, S) ->
+translate({ '!', Meta, [{ '!', _, [Expr] }] }, S) ->
   { TExpr, SE } = translate_each(Expr, S),
-  { elixir_tree_helpers:convert_to_boolean(Line, TExpr, true, S#elixir_scope.context == guard), SE };
+  elixir_tree_helpers:convert_to_boolean(?line(Meta), TExpr, true, S#elixir_scope.context == guard, SE);
 
-translate({ '!', Line, [Expr] }, S) ->
+translate({ '!', Meta, [Expr] }, S) ->
   { TExpr, SE } = translate_each(Expr, S),
-  { elixir_tree_helpers:convert_to_boolean(Line, TExpr, false, S#elixir_scope.context == guard), SE };
+  elixir_tree_helpers:convert_to_boolean(?line(Meta), TExpr, false, S#elixir_scope.context == guard, SE);
 
-translate({ in, Line, [Left, Right] }, #elixir_scope{extra_guards=nil} = S) ->
-  { _, TExpr, TS } = translate_in(Line, Left, Right, S),
+translate({ in, Meta, [Left, Right] }, #elixir_scope{extra_guards=nil} = S) ->
+  { _, TExpr, TS } = translate_in(Meta, Left, Right, S),
   { TExpr, TS };
 
-translate({ in, Line, [Left, Right] }, #elixir_scope{extra_guards=Extra} = S) ->
-  { TVar, TExpr, TS } = translate_in(Line, Left, Right, S),
+translate({ in, Meta, [Left, Right] }, #elixir_scope{extra_guards=Extra} = S) ->
+  { TVar, TExpr, TS } = translate_in(Meta, Left, Right, S),
   { TVar, TS#elixir_scope{extra_guards=[TExpr|Extra]} };
 
 %% Functions
 
-translate({ function, Line, [[{do,{ '->',_,Pairs}}]] }, S) ->
-  assert_no_assign_or_guard_scope(Line, 'function', S),
-  elixir_translator:translate_fn(Line, Pairs, S);
+translate({ function, Meta, [[{do,{ '->',_,Pairs}}]] }, S) ->
+  assert_no_match_or_guard_scope(Meta, 'function', S),
+  elixir_translator:translate_fn(Meta, Pairs, S);
 
-translate({ function, Line, [_] }, S) ->
-  assert_no_assign_or_guard_scope(Line, 'function', S),
-  syntax_error(Line, S#elixir_scope.file, "invalid args for function");
+translate({ function, _, [{ '/', _, [{{ '.', Meta, [M, F] }, _ , []}, A]}] }, S) when is_atom(F), is_integer(A) ->
+  translate({ function, Meta, [M, F, A] }, S);
 
-translate({ function, Line, [_, _] = Args }, S) ->
-  assert_no_assign_or_guard_scope(Line, 'function', S),
+translate({ function, MetaFA, [{ '/', _, [{F, Meta, C}, A]}] }, S) when is_atom(F), is_integer(A), is_atom(C) ->
+  assert_no_match_or_guard_scope(Meta, 'function', S),
+
+  WrappedMeta =
+    case (lists:keyfind(import, 1, Meta) == false) andalso lists:keyfind(import_fa, 1, MetaFA) of
+      { import_fa, Receiver } -> [{ import, Receiver }|Meta];
+      false -> Meta
+    end,
+
+  case elixir_dispatch:import_function(WrappedMeta, F, A, S) of
+    false -> syntax_error(WrappedMeta, S#elixir_scope.file, "cannot convert a macro to a function");
+    Else  -> Else
+  end;
+
+translate({ function, Meta, [_] }, S) ->
+  assert_no_match_or_guard_scope(Meta, 'function', S),
+  syntax_error(Meta, S#elixir_scope.file, "invalid args for function");
+
+translate({ function, Meta, [_, _] = Args }, S) ->
+  assert_no_match_or_guard_scope(Meta, 'function', S),
 
   case translate_args(Args, S) of
     { [{atom,_,Name}, {integer,_,Arity}], SA } ->
-      case elixir_dispatch:import_function(Line, Name, Arity, SA) of
-        false -> syntax_error(Line, S#elixir_scope.file, "cannot convert a macro to a function");
+      elixir_errors:deprecation(Meta, S#elixir_scope.file, "function(:~ts, ~B) is deprecated, please use function(~ts/~B)", [Name, Arity, Name, Arity]),
+      case elixir_dispatch:import_function(Meta, Name, Arity, SA) of
+        false -> syntax_error(Meta, S#elixir_scope.file, "cannot convert a macro to a function");
         Else  -> Else
       end;
     _ ->
-      syntax_error(Line, S#elixir_scope.file, "cannot dynamically retrieve local function. use function(module, fun, arity) instead")
+      syntax_error(Meta, S#elixir_scope.file, "cannot dynamically retrieve local function, use function/3 instead")
   end;
 
-translate({ function, Line, [_,_,_] = Args }, S) when is_list(Args) ->
-  assert_no_assign_or_guard_scope(Line, 'function', S),
+translate({ function, Meta, [_,_,_] = Args }, S) when is_list(Args) ->
+  assert_no_match_or_guard_scope(Meta, 'function', S),
   { [A,B,C], SA } = translate_args(Args, S),
-  { { 'fun', Line, { function, A, B, C } }, SA };
+  { { 'fun', ?line(Meta), { function, A, B, C } }, SA };
 
 %% @
 
-translate({'@', Line, [{ Name, _, Args }]}, S) when Name == typep; Name == type; Name == spec; Name == callback ->
+translate({'@', Meta, [{ Name, _, [Arg] }]}, S) when Name == typep; Name == type; Name == spec; Name == callback; Name == opaque ->
   case elixir_compiler:get_opt(internal) of
-    true  -> { { nil, Line }, S };
+    true  -> { { nil, ?line(Meta) }, S };
     false ->
-      Call = { { '.', Line, ['Elixir.Kernel.Typespec', spec_to_macro(Name)] }, Line, Args },
+      Call = { { '.', Meta, ['Elixir.Kernel.Typespec', spec_to_macro(Name)] }, Meta, [Arg] },
       translate_each(Call, S)
   end;
 
-translate({'@', Line, [{ Name, _, Args }]}, S) ->
-  assert_module_scope(Line, '@', S),
+translate({'@', Meta, [{ Name, _, Args }]}, S) ->
+  assert_module_scope(Meta, '@', S),
 
   case is_reserved_data(Name) andalso elixir_compiler:get_opt(internal) of
     true ->
-      { { nil, Line }, S };
+      { { nil, ?line(Meta) }, S };
     _ ->
       case Args of
         [Arg] ->
           case S#elixir_scope.function of
             nil ->
               translate_each({
-                { '.', Line, ['Elixir.Module', put_attribute] },
-                  Line,
-                  [ { '__MODULE__', Line, false }, Name, Arg ]
+                { '.', Meta, ['Elixir.Module', put_attribute] },
+                  Meta,
+                  [ { '__MODULE__', Meta, false }, Name, Arg ]
               }, S);
             _  ->
-              syntax_error(Line, S#elixir_scope.file,
-                "cannot dynamically set attribute @~s inside a function", [Name])
+              syntax_error(Meta, S#elixir_scope.file,
+                "cannot dynamically set attribute @~ts inside a function", [Name])
           end;
         _ when is_atom(Args) or (Args == []) ->
           case S#elixir_scope.function of
             nil ->
               translate_each({
-                { '.', Line, ['Elixir.Module', get_attribute] },
-                Line,
-                [ { '__MODULE__', Line, false }, Name ]
+                { '.', Meta, ['Elixir.Module', get_attribute] },
+                Meta,
+                [ { '__MODULE__', Meta, false }, Name ]
               }, S);
             _ ->
               Contents = 'Elixir.Module':get_attribute(S#elixir_scope.module, Name),
               { elixir_tree_helpers:abstract_syntax(Contents), S }
           end;
         _ ->
-          syntax_error(Line, S#elixir_scope.file, "expected 0 or 1 argument for @~s, got: ~p", [Name, length(Args)])
+          syntax_error(Meta, S#elixir_scope.file, "expected 0 or 1 argument for @~ts, got: ~p", [Name, length(Args)])
       end
   end;
 
 %% Case
 
-translate({'case', Line, [Expr, KV]}, S) ->
-  assert_no_assign_or_guard_scope(Line, 'case', S),
-  Clauses = elixir_clauses:get_pairs(Line, do, KV, S),
+translate({'case', Meta, [Expr, KV]}, S) ->
+  assert_no_match_or_guard_scope(Meta, 'case', S),
+  Clauses = elixir_clauses:get_pairs(Meta, do, KV, S),
   { TExpr, NS } = translate_each(Expr, S),
 
   RClauses = case elixir_tree_helpers:returns_boolean(TExpr) of
@@ -135,123 +155,116 @@ translate({'case', Line, [Expr, KV]}, S) ->
     false -> Clauses
   end,
 
-  { TClauses, TS } = elixir_clauses:match(Line, RClauses, NS),
-  { { 'case', Line, TExpr, TClauses }, TS };
+  { TClauses, TS } = elixir_clauses:match(Meta, RClauses, NS),
+  { { 'case', ?line(Meta), TExpr, TClauses }, TS };
 
 %% Try
 
-translate({'try', Line, [Clauses]}, RawS) ->
+translate({'try', Meta, [Clauses]}, RawS) ->
   S = RawS#elixir_scope{noname=true},
-  assert_no_assign_or_guard_scope(Line, 'try', S),
+  assert_no_match_or_guard_scope(Meta, 'try', S),
 
-  Do = proplists:get_value('do', Clauses, []),
-  { TDo, SB } = elixir_translator:translate([Do], S),
+  Do = proplists:get_value('do', Clauses, nil),
+  { TDo, SB } = elixir_translator:translate_each(Do, S),
 
   Catch = [Tuple || { X, _ } = Tuple <- Clauses, X == 'rescue' orelse X == 'catch'],
-  { TCatch, SC } = elixir_try:clauses(Line, Catch, umergec(S, SB)),
+  { TCatch, SC } = elixir_try:clauses(Meta, Catch, umergec(S, SB)),
 
-  { TAfter, SA } = case lists:keyfind('after', 1, Clauses) of
-    { 'after', After } -> elixir_translator:translate([After], umergec(S, SC));
-    false -> { [], SC }
-  end,
+  After = proplists:get_value('after', Clauses, nil),
+  { TAfter, SA } = elixir_translator:translate_each(After, umergec(S, SC)),
 
-  { { 'try', Line, unpack(TDo), [], TCatch, unpack(TAfter) }, umergec(RawS, SA) };
+  Else = elixir_clauses:get_pairs(Meta, else, Clauses, S),
+  { TElse, SE } = elixir_clauses:match(Meta, Else, umergec(S, SA)),
+
+  { { 'try', ?line(Meta), pack(TDo), TElse, TCatch, pack(TAfter) }, umergec(RawS, SE) };
 
 %% Receive
 
-translate({'receive', Line, [KV] }, S) ->
-  assert_no_assign_or_guard_scope(Line, 'receive', S),
-  Do = elixir_clauses:get_pairs(Line, do, KV, S, true),
+translate({'receive', Meta, [KV] }, S) ->
+  assert_no_match_or_guard_scope(Meta, 'receive', S),
+  Do = elixir_clauses:get_pairs(Meta, do, KV, S, true),
 
   case lists:keyfind('after', 1, KV) of
     false ->
-      { TClauses, SC } = elixir_clauses:match(Line, Do, S),
-      { { 'receive', Line, TClauses }, SC };
+      { TClauses, SC } = elixir_clauses:match(Meta, Do, S),
+      { { 'receive', ?line(Meta), TClauses }, SC };
     _ ->
-      After = elixir_clauses:get_pairs(Line, 'after', KV, S),
-      { TClauses, SC } = elixir_clauses:match(Line, Do ++ After, S),
+      After = elixir_clauses:get_pairs(Meta, 'after', KV, S),
+      { TClauses, SC } = elixir_clauses:match(Meta, Do ++ After, S),
       { FClauses, TAfter } = elixir_tree_helpers:split_last(TClauses),
       { _, _, [FExpr], _, FAfter } = TAfter,
-      { { 'receive', Line, FClauses, FExpr, FAfter }, SC }
+      { { 'receive', ?line(Meta), FClauses, FExpr, FAfter }, SC }
   end;
 
 %% Definitions
 
-translate({defmodule, Line, [Ref, KV]}, S) ->
+translate({defmodule, Meta, [Ref, KV]}, S) ->
   { TRef, _ } = translate_each(Ref, S),
 
   Block = case lists:keyfind(do, 1, KV) of
     { do, DoValue } -> DoValue;
-    false -> syntax_error(Line, S#elixir_scope.file, "expected do: argument in defmodule")
+    false -> syntax_error(Meta, S#elixir_scope.file, "expected do: argument in defmodule")
   end,
 
   { FRef, FS } = case TRef of
     { atom, _, Module } ->
-      NewModule = module_ref(Ref, Module, S#elixir_scope.module),
+      FullModule = expand_module(Ref, Module, S),
 
-      RS = case Module == NewModule of
-        true  -> S;
+      RS = case elixir_aliases:nesting_alias(S#elixir_scope.module, FullModule) of
+        { New, Old } ->
+          S#elixir_scope{
+            aliases=orddict:store(New, Old, S#elixir_scope.aliases),
+            macro_aliases=orddict:store(New, Old, S#elixir_scope.macro_aliases)};
         false ->
-          element(2, translate_each({
-            alias, Line, [NewModule, [{as, elixir_aliases:first(Module)}]]
-          }, S))
+          S
       end,
 
       {
-        { atom, Line, NewModule },
-        RS#elixir_scope{scheduled=[NewModule|S#elixir_scope.scheduled]}
+        { atom, Meta, FullModule },
+        RS#elixir_scope{scheduled=[FullModule|S#elixir_scope.scheduled]}
       };
     _ ->
       { TRef, S }
   end,
 
-  { elixir_module:translate(Line, FRef, Block, S), FS };
+  MS = FS#elixir_scope{check_clauses=true,local=nil},
+  { elixir_module:translate(Meta, FRef, Block, MS), FS };
 
-translate({Kind, Line, [Call]}, S) when ?FUNS() ->
-  translate({Kind, Line, [Call, nil]}, S);
+translate({Kind, Meta, [Call]}, S) when ?FUNS(Kind) ->
+  translate({Kind, Meta, [Call, nil]}, S);
 
-translate({Kind, Line, [Call, Expr]}, S) when ?FUNS() ->
-  assert_module_scope(Line, Kind, S),
-  assert_no_function_scope(Line, Kind, S),
-  { TCall, Guards } = elixir_clauses:extract_guards(Call),
-  { Name, Args }    = elixir_clauses:extract_args(TCall),
-  TName             = elixir_tree_helpers:abstract_syntax(Name),
-  TArgs             = elixir_tree_helpers:abstract_syntax(Args),
-  TGuards           = elixir_tree_helpers:abstract_syntax(Guards),
-  TExpr             = elixir_tree_helpers:abstract_syntax(Expr),
-  { elixir_def:wrap_definition(Kind, Line, TName, TArgs, TGuards, TExpr, S), S };
+translate({Kind, Meta, [Call, Expr]}, S) when ?FUNS(Kind) ->
+  assert_module_scope(Meta, Kind, S),
+  assert_no_function_scope(Meta, Kind, S),
+  { TCall, SC } = elixir_quote:user_quote(Call, S),
+  { TExpr, SE } = elixir_quote:user_quote(Expr, SC),
+  { elixir_def:wrap_definition(Kind, Meta, TCall, TExpr, SE), SE };
 
-translate({Kind, Line, [Name, Args, Guards, Expr]}, S) when ?FUNS() ->
-  assert_module_scope(Line, Kind, S),
-  assert_no_function_scope(Line, Kind, S),
+translate({Kind, Meta, [Name, Args, Guards, Expr]}, S) when ?FUNS(Kind) ->
+  assert_module_scope(Meta, Kind, S),
+  assert_no_function_scope(Meta, Kind, S),
   { TName, NS }   = translate_each(Name, S),
   { TArgs, AS }   = translate_each(Args, NS),
   { TGuards, GS } = translate_each(Guards, AS),
   { TExpr, ES }   = translate_each(Expr, GS),
-  { elixir_def:wrap_definition(Kind, Line, TName, TArgs, TGuards, TExpr, ES), ES };
+  { elixir_def:wrap_definition(Kind, Meta, TName, TArgs, TGuards, TExpr, ES), ES };
 
 %% Apply - Optimize apply by checking what doesn't need to be dispatched dynamically
 
-translate({ apply, Line, [Left, Right, Args] }, S) when is_list(Args) ->
+translate({ apply, Meta, [Left, Right, Args] }, S) when is_list(Args) ->
   { TLeft,  SL } = translate_each(Left, S),
   { TRight, SR } = translate_each(Right, umergec(S, SL)),
-  translate_apply(Line, TLeft, TRight, Args, S, SL, SR);
+  translate_apply(Meta, TLeft, TRight, Args, S, SL, SR);
 
-translate({ apply, Line, Args }, S) ->
+translate({ apply, Meta, Args }, S) ->
   { TArgs, NS } = translate_args(Args, S),
-  { ?ELIXIR_WRAP_CALL(Line, erlang, apply, TArgs), NS };
-
-%% Handle forced variables
-
-translate({ 'var!', _, [{Name, Line, Atom}] }, S) when is_atom(Name), is_atom(Atom) ->
-  elixir_scope:translate_var(Line, Name, nil, S);
-
-translate({ 'var!', Line, [_] }, S) ->
-  syntax_error(Line, S#elixir_scope.file, "invalid args for var!").
+  { ?wrap_call(?line(Meta), erlang, apply, TArgs), NS }.
 
 %% Helpers
 
-translate_in(Line, Left, Right, S) ->
+translate_in(Meta, Left, Right, S) ->
+  Line = ?line(Meta),
+
   { TLeft, SL } = case Left of
     { '_', _, Atom } when is_atom(Atom) ->
       elixir_scope:build_erl_var(Line, S);
@@ -271,24 +284,30 @@ translate_in(Line, Left, Right, S) ->
   Expr = case TRight of
     { cons, _, _, _ } ->
       [H|T] = elixir_tree_helpers:cons_to_list(TRight),
-      lists:foldl(fun(X, Acc) ->
-        { op, Line, 'orelse', Acc, { op, Line, '==', Var, X } }
+      lists:foldr(fun(X, Acc) ->
+        { op, Line, 'orelse', { op, Line, '==', Var, X }, Acc }
       end, { op, Line, '==', Var, H }, T);
+    { string, _, [H|T] } ->
+      lists:foldl(fun(X, Acc) ->
+        { op, Line, 'orelse', { op, Line, '==', Var, { integer, Line, X } }, Acc }
+      end, { op, Line, '==', Var, { integer, Line, H } }, T);
     { tuple, _, [{ atom, _, 'Elixir.Range' }, Start, End] } ->
-      { op, Line, 'orelse',
-        { op, Line, 'andalso',
-          { op, Line, '=<', Start, End},      
-          { op, Line, 'andalso',
-            { op, Line, '>=', Var, Start },
-            { op, Line, '=<', Var, End } } },
-        { op, Line, 'andalso',
-          { op, Line, '<', End, Start},
-          { op, Line, 'andalso',          
-            { op, Line, '=<', Var, Start },
-            { op, Line, '>=', Var, End } } }
-      };
+      case { Start, End } of
+        { { K1, _, StartInt }, { K2, _, EndInt } } when ?IN_TYPES(K1), ?IN_TYPES(K2), StartInt =< EndInt ->
+          increasing_compare(Line, Var, Start, End);
+        { { K1, _, _ }, { K2, _, _ } } when ?IN_TYPES(K1), ?IN_TYPES(K2) ->
+          decreasing_compare(Line, Var, Start, End);
+        _ ->
+          { op, Line, 'orelse',
+            { op, Line, 'andalso',
+              { op, Line, '=<', Start, End},
+              increasing_compare(Line, Var, Start, End) },
+            { op, Line, 'andalso',
+              { op, Line, '<', End, Start},
+              decreasing_compare(Line, Var, Start, End) } }
+      end;
     _ ->
-      syntax_error(Line, S#elixir_scope.file, "invalid args for operator in")
+      syntax_error(Meta, S#elixir_scope.file, "invalid args for operator in, it expects an explicit array or an explicit range on the right side")
   end,
 
   case Cache of
@@ -296,20 +315,35 @@ translate_in(Line, Left, Right, S) ->
     false -> { Var, Expr, SV }
   end.
 
+increasing_compare(Line, Var, Start, End) ->
+  { op, Line, 'andalso',
+    { op, Line, '>=', Var, Start },
+    { op, Line, '=<', Var, End } }.
+
+decreasing_compare(Line, Var, Start, End) ->
+  { op, Line, 'andalso',
+    { op, Line, '=<', Var, Start },
+    { op, Line, '>=', Var, End } }.
+
 rewrite_case_clauses([{do,[{in,_,[{'_',_,_},[false,nil]]}],False},{do,[{'_',_,_}],True}]) ->
   [{do,[false],False},{do,[true],True}];
 
 rewrite_case_clauses(Clauses) ->
   Clauses.
 
-module_ref(_Raw, Module, nil) ->
+expand_module(Raw, Module, #elixir_scope{module=Nesting}) when is_atom(Raw); Nesting == nil ->
   Module;
 
-module_ref({ '__aliases__', _, ['Elixir'|_]}, Module, _Nesting) ->
-  Module;
+expand_module({ '__aliases__', _, List } = Alias, Module, S) when List /= ['Elixir'] ->
+  case elixir_aliases:expand(Alias, S#elixir_scope.aliases, S#elixir_scope.macro_aliases) of
+    Atom when is_atom(Atom) ->
+      Module;
+    Aliases when is_list(Aliases) ->
+      elixir_aliases:concat([S#elixir_scope.module, Module])
+  end;
 
-module_ref(_F, Module, Nesting) ->
-  elixir_aliases:concat([Nesting, Module]).
+expand_module(_Raw, Module, S) ->
+  elixir_aliases:concat([S#elixir_scope.module, Module]).
 
 is_reserved_data(moduledoc) -> true;
 is_reserved_data(doc)       -> true;
@@ -317,9 +351,10 @@ is_reserved_data(_)         -> false.
 
 spec_to_macro(type)     -> deftype;
 spec_to_macro(typep)    -> deftypep;
+spec_to_macro(opaque)   -> defopaque;
 spec_to_macro(spec)     -> defspec;
 spec_to_macro(callback) -> defcallback.
 
-% Unpack a list of expressions from a block.
-unpack([{ '__block__', _, Exprs }]) -> Exprs;
-unpack(Exprs)                       -> Exprs.
+% Pack a list of expressions from a block.
+pack({ 'block', _, Exprs }) -> Exprs;
+pack(Expr)                  -> [Expr].

lib/elixir/src/elixir_module.erl

@@ -1,5 +1,6 @@
 -module(elixir_module).
--export([translate/4, compile/5, data_table/1, eval_quoted/4, format_error/1]).
+-export([translate/4, compile/5, data_table/1, eval_quoted/4,
+         format_error/1, eval_callbacks/5]).
 -include("elixir.hrl").
 -compile({parse_transform, elixir_transform}).
 
@@ -14,7 +15,7 @@ eval_quoted(Module, Quoted, RawBinding, Opts) ->
     false -> Line = 1
   end,
 
-  { Value, FinalBinding, _Scope } = elixir:eval_quoted([Quoted], Binding, Line, Scope),
+  { Value, FinalBinding, _Scope } = elixir:eval_quoted([Quoted], Binding, Line, Scope#elixir_scope{check_clauses=false}),
   { Value, FinalBinding }.
 
 scope_for_eval(Module, #elixir_scope{} = S) ->
@@ -25,17 +26,13 @@ scope_for_eval(Module, Opts) ->
 
 binding_for_eval(Module, Binding) -> [{'_@MODULE',Module}|Binding].
 
-
 %% TABLE METHODS
 
 data_table(Module) ->
   Module.
 
 docs_table(Module) ->
-  ?ELIXIR_ATOM_CONCAT([o, Module]).
-
-spec_table(Module) ->
-  ?ELIXIR_ATOM_CONCAT([s, Module]).
+  ?atom_concat([o, Module]).
 
 %% TRANSFORMATION FUNCTIONS
 
@@ -43,44 +40,38 @@ spec_table(Module) ->
 %% The abstract form for extra arguments may be given and they
 %% will be passed to the invoked function.
 
-translate(Line, Ref, Block, S) ->
-  MetaBlock = elixir_tree_helpers:abstract_syntax(Block),
-  MetaS     = elixir_scope:serialize(S),
+translate(Meta, Ref, Block, S) ->
+  Line            = ?line(Meta),
+  MetaBlock       = elixir_tree_helpers:abstract_syntax(Block),
+  { MetaS, Vars } = elixir_scope:serialize_with_vars(Line, S),
 
-  Vars = orddict:fold(fun(Key, Value, { Acc, Counter }) ->
-    { { cons, Line, { tuple, Line, [
-      { atom, Line, Key },
-      { atom, Line, ?ELIXIR_ATOM_CONCAT(["_@", Counter]) },
-      { var,  Line, Value }
-    ] }, Acc }, Counter + 1 }
-  end, { { nil, Line }, 0 }, S#elixir_scope.vars),
-
-  Args = [{integer, Line, Line}, Ref, MetaBlock, element(1, Vars), MetaS],
-  ?ELIXIR_WRAP_CALL(Line, ?MODULE, compile, Args).
+  Args = [{integer, Line, Line}, Ref, MetaBlock, Vars, MetaS],
+  ?wrap_call(Line, ?MODULE, compile, Args).
 
 %% The compilation hook.
 
 compile(Line, Module, Block, Vars, #elixir_scope{} = S) when is_atom(Module) ->
   C = elixir_compiler:get_opts(),
   File = S#elixir_scope.file,
+  FileList = binary_to_list(File),
 
   check_module_availability(Line, File, Module, C),
   build(Line, File, Module),
 
   try
     Result = eval_form(Line, Module, Block, Vars, S),
-    { Export, Private, Def, Defmacro, Defmacrop, Functions } = elixir_def:unwrap_stored_definitions(Module),
+    { Export, Private, Def, Defmacro, Defmacrop, Functions } = elixir_def:unwrap_stored_definitions(FileList, Module),
 
     { All, Forms0 } = functions_form(Line, File, Module, Export, Private, Def, Defmacro, Defmacrop, Functions, C),
-    Forms1          = attributes_form(Line, File, Module, Forms0),
-    Forms2          = specs_form(Line, Module, Defmacro, Defmacrop, Forms1),
+    Forms1          = specs_form(Line, Module, Defmacro, Defmacrop, Forms0, C),
+    Forms2          = attributes_form(Line, File, Module, Forms1),
 
     elixir_import:ensure_no_local_conflict(Line, File, Module, All),
     elixir_import:ensure_no_import_conflict(Line, File, Module, All),
 
     Final = [
-      {attribute, Line, file, {File,Line}},
-      {attribute, Line, module, Module} | Forms2
+      { attribute, Line, file, { FileList, Line } },
+      { attribute, Line, module, Module } | Forms2
     ],
 
     Binary = load_form(Line, Final, S),
@@ -88,7 +79,6 @@ compile(Line, Module, Block, Vars, #elixir_scope{} = S) when is_atom(Module) ->
   after
     ets:delete(data_table(Module)),
     ets:delete(docs_table(Module)),
-    ets:delete(spec_table(Module)),
     elixir_def:delete_table(Module),
     elixir_import:delete_table(Module)
   end;
@@ -97,8 +87,8 @@ compile(Line, Other, _Block, _Vars, #elixir_scope{file=File}) ->
   elixir_errors:form_error(Line, File, ?MODULE, { invalid_module, Other });
 
 compile(Line, Module, Block, Vars, RawS) ->
-  Dict = [{ X, Y } || { X, Y, _ } <- Vars],
-  S = elixir_scope:deserialize(RawS, Dict),
+  Dict = [{ { Name, Kind }, Value } || { Name, Kind, Value, _ } <- Vars],
+  S = elixir_scope:deserialize_with_vars(RawS, Dict),
   compile(Line, Module, Block, Vars, S).
 
 %% Hook that builds both attribute and functions and set up common hooks.
@@ -131,10 +121,6 @@ build(Line, File, Module) ->
   DocsTable = docs_table(Module),
   ets:new(DocsTable, [ordered_set, named_table, public]),
 
-  %% Holds specs in a format it is easier to query.
-  SpecTable = spec_table(Module),
-  ets:new(SpecTable, [duplicate_bag, named_table, public]),
-
   %% We keep a separated table for function definitions
   %% and another one for imports. We keep them in different
   %% tables for organization and speed purpose (since the
@@ -145,15 +131,21 @@ build(Line, File, Module) ->
 %% Receives the module representation and evaluates it.
 
 eval_form(Line, Module, Block, Vars, RawS) ->
-  Temp = ?ELIXIR_ATOM_CONCAT(["COMPILE-",Module]),
   S = scope_for_eval(Module, RawS),
-  { Value, NewS } = elixir_compiler:eval_forms([Block], Line, Temp, Vars, S),
+  { Value, NewS } = elixir_compiler:eval_forms([Block], Line, Vars, S),
+  elixir_def_overridable:store_pending(Module),
+  Env = elixir_scope:to_ex_env({ Line, S }),
+  eval_callbacks(Line, Module, before_compile, [Env], NewS),
   elixir_def_overridable:store_pending(Module),
-  eval_callbacks(Line, Module, before_compile, [Module], NewS),
   Value.
 
 %% Return the form with exports and function declarations.
-functions_form(Line, File, Module, Export, Private, Def, Defmacro, Defmacrop, Functions, C) ->
+functions_form(Line, File, Module, Export, Private, Def, Defmacro, Defmacrop, RawFunctions, C) ->
+  Functions = case elixir_compiler:get_opt(internal, C) of
+    true  -> RawFunctions;
+    false -> record_rewrite_functions(Module, RawFunctions)
+  end,
+
   { FinalExport, FinalFunctions } =
     add_info_function(Line, File, Module, Export, Functions, Def, Defmacro, C),
 
@@ -164,6 +156,17 @@ functions_form(Line, File, Module, Export, Private, Def, Defmacro, Defmacrop, Fu
     {attribute, Line, export, lists:sort(FinalExport)} | FinalFunctions
   ] }.
 
+record_rewrite_functions(Module, Functions) ->
+  lists:map(fun
+    ({ function, Line, Name, Arity, Clauses }) ->
+      Rewriten = [begin
+        { C, _, _ } = 'Elixir.Kernel.RecordRewriter':optimize_clause(Module, Clause),
+        C
+      end || Clause <- Clauses],
+      { function, Line, Name, Arity, Rewriten };
+    (Other) -> Other
+  end, Functions).
+
 %% Add attributes handling to the form
 
 attributes_form(Line, _File, Module, Current) ->
@@ -188,36 +191,43 @@ attributes_form(Line, _File, Module, Current) ->
 
 %% Specs
 
-specs_form(Line, Module, Defmacro, Defmacrop, Forms) ->
-  { Specs, Callbacks } = ets:foldl(fun(X, Acc) ->
-    translate_spec(X, Defmacro, Defmacrop, Acc)
-  end, { [], [] }, spec_table(Module)),
+specs_form(Line, Module, Defmacro, DefmacropWithLine, Forms, C) ->
+  Defmacrop = [Tuple || { Tuple, _, _ } <- DefmacropWithLine],
+  case elixir_compiler:get_opt(internal, C) of
+    true -> Forms;
+    _    ->
+      Callbacks = 'Elixir.Module':get_attribute(Module, callback),
+      Specs     = [translate_spec(Spec, Defmacro, Defmacrop) ||
+                    Spec <- 'Elixir.Module':get_attribute(Module, spec)],
 
-  Temp = specs_attributes(Line, spec, Forms, Specs),
-  specs_attributes(Line, callback, Temp, Callbacks).
+      'Elixir.Module':delete_attribute(Module, spec),
+      'Elixir.Module':delete_attribute(Module, callback),
+
+      Temp = specs_attributes(Line, spec, Forms, Specs),
+      specs_attributes(Line, callback, Temp, Callbacks)
+  end.
 
 specs_attributes(Line, Type, Forms, Specs) ->
-  Keys = lists:ukeysort(1, Specs),
+  Keys = lists:foldl(fun({ Tuple, Value }, Acc) ->
+                       lists:keystore(Tuple, 1, Acc, { Tuple, Value } )
+                     end, [], Specs),
   lists:foldl(fun({ Tuple, _ }, Acc) ->
-    [{ attribute, Line, Type, { Tuple, proplists:append_values(Tuple, Specs) } }|Acc]
+    Values = [V || { K, V } <- Specs, K == Tuple],
+    [{ attribute, Line, Type, { Tuple, Values } }|Acc]
   end, Forms, Keys).
 
-translate_spec({ { spec, Spec }, Rest }, Defmacro, Defmacrop, { Specs, Callbacks } = Acc) ->
+translate_spec({ Spec, Rest }, Defmacro, Defmacrop) ->
   case ordsets:is_element(Spec, Defmacrop) of
-    true  -> Acc;
+    true  -> { Spec, Rest };
     false ->
       case ordsets:is_element(Spec, Defmacro) of
         true ->
           { Name, Arity } = Spec,
-          New = { { ?ELIXIR_MACRO(Name), Arity + 1 }, [spec_for_macro(X) || X <- Rest] },
-          { [New|Specs], Callbacks };
+          { { ?elixir_macro(Name), Arity + 1 }, spec_for_macro(Rest) };
         false ->
-          { [{ Spec, Rest }|Specs], Callbacks }
+          { Spec, Rest }
       end
-  end;
-
-translate_spec({ { callback, Spec }, Rest }, _, _, { Specs, Callbacks }) ->
-  { Specs, [{ Spec, Rest }|Callbacks] }.
+  end.
 
 spec_for_macro({ type, Line, 'fun', [{ type, _, product, Args }|T] }) ->
   NewArgs = [{type,Line,term,[]}|Args],
@@ -230,7 +240,8 @@ spec_for_macro(Else) -> Else.
 load_form(Line, Forms, S) ->
   elixir_compiler:module(Forms, S, fun(Module, Binary) ->
     EvalS = scope_for_eval(Module, S),
-    eval_callbacks(Line, Module, after_compile, [Module, Binary], EvalS),
+    Env = elixir_scope:to_ex_env({ Line, EvalS }),
+    eval_callbacks(Line, Module, after_compile, [Env, Binary], EvalS),
 
     case get(elixir_compiled) of
       Current when is_list(Current) ->
@@ -266,56 +277,49 @@ add_info_function(Line, File, Module, Export, Functions, Def, Defmacro, C) ->
     true  -> elixir_errors:form_error(Line, File, ?MODULE, {internal_function_overridden, Pair});
     false ->
       Docs = elixir_compiler:get_opt(docs, C),
-      Contents = { function, Line, '__info__', 1, [
-        functions_clause(Line, Def),
-        macros_clause(Line, Module, Def, Defmacro),
-        docs_clause(Line, Module, Docs),
+      Contents = { function, 0, '__info__', 1, [
+        functions_clause(Def),
+        macros_clause(Module, Def, Defmacro),
+        docs_clause(Module, Docs),
         moduledoc_clause(Line, Module, Docs),
-        compile_clause(Line),
-        self_clause(Line, Module),
-        else_clause(Line)
+        module_clause(Module),
+        else_clause()
       ] },
       { [Pair|Export], [Contents|Functions] }
   end.
 
-functions_clause(Line, Def) ->
-  All = ordsets:add_element({'__info__',1}, Def),
-  { clause, Line, [{ atom, Line, functions }], [], [elixir_tree_helpers:abstract_syntax(All)] }.
+functions_clause(Def) ->
+  { clause, 0, [{ atom, 0, functions }], [], [elixir_tree_helpers:abstract_syntax(Def)] }.
 
-macros_clause(Line, Module, Def, Defmacro) ->
+macros_clause(Module, Def, Defmacro) ->
   All = handle_builtin_macros(Module, Def, Defmacro),
-  { clause, Line, [{ atom, Line, macros }], [], [elixir_tree_helpers:abstract_syntax(All)] }.
+  { clause, 0, [{ atom, 0, macros }], [], [elixir_tree_helpers:abstract_syntax(All)] }.
 
 handle_builtin_macros('Elixir.Kernel', Def, Defmacro) ->
   ordsets:subtract(ordsets:union(Defmacro,
     elixir_dispatch:in_erlang_macros()), Def);
 handle_builtin_macros(_, _Def, Defmacro) -> Defmacro.
 
-self_clause(Line, Module) ->
-  { clause, Line, [{ atom, Line, self }], [], [{ atom, Line, Module }] }.
+module_clause(Module) ->
+  { clause, 0, [{ atom, 0, module }], [], [{ atom, 0, Module }] }.
 
-docs_clause(Line, Module, true) ->
+docs_clause(Module, true) ->
   Docs = ordsets:from_list(ets:tab2list(docs_table(Module))),
-  { clause, Line, [{ atom, Line, docs }], [], [elixir_tree_helpers:abstract_syntax(Docs)] };
+  { clause, 0, [{ atom, 0, docs }], [], [elixir_tree_helpers:abstract_syntax(Docs)] };
 
-docs_clause(Line, _Module, _) ->
-  { clause, Line, [{ atom, Line, docs }], [], [{ atom, Line, nil }] }.
+docs_clause(_Module, _) ->
+  { clause, 0, [{ atom, 0, docs }], [], [{ atom, 0, nil }] }.
 
 moduledoc_clause(Line, Module, true) ->
   Docs = 'Elixir.Module':get_attribute(Module, moduledoc),
-  { clause, Line, [{ atom, Line, moduledoc }], [], [elixir_tree_helpers:abstract_syntax({ Line, Docs })] };
+  { clause, 0, [{ atom, 0, moduledoc }], [], [elixir_tree_helpers:abstract_syntax({ Line, Docs })] };
 
-moduledoc_clause(Line, _Module, _) ->
-  { clause, Line, [{ atom, Line, moduledoc }], [], [{ atom, Line, nil }] }.
+moduledoc_clause(_Line, _Module, _) ->
+  { clause, 0, [{ atom, 0, moduledoc }], [], [{ atom, 0, nil }] }.
 
-compile_clause(Line) ->
-  Info = { call, Line, { atom, Line, module_info }, [{ atom, Line, compile }] },
-  WrappedInfo = ?ELIXIR_WRAP_CALL(Line, 'Elixir.Keyword', 'from_enum', [Info]),
-  { clause, Line, [{ atom, Line, compile }], [], [WrappedInfo] }.
-
-else_clause(Line) ->
-  Info = { call, Line, { atom, Line, module_info }, [{ var, Line, atom }] },
-  { clause, Line, [{ var, Line, atom }], [], [Info] }.
+else_clause() ->
+  Info = { call, 0, { atom, 0, module_info }, [{ var, 0, atom }] },
+  { clause, 0, [{ var, 0, atom }], [], [Info] }.
 
 % HELPERS
 
@@ -323,16 +327,13 @@ eval_callbacks(Line, Module, Name, Args, RawS) ->
   S         = RawS#elixir_scope{check_clauses=false},
   Binding   = binding_for_eval(Module, []),
   Callbacks = lists:reverse(ets:lookup_element(data_table(Module), Name, 2)),
-  Requires  = S#elixir_scope.requires,
+  Meta      = [{line,Line},{require,false}],
 
   lists:foreach(fun({M,F}) ->
-    Expr  = { { '.', Line, [M,F] }, Line, Args },
-    Scope = case ordsets:is_element(M, Requires) of
-      true  -> S;
-      false -> S#elixir_scope{requires=ordsets:add_element(M, Requires)}
-    end,
-
-    { Tree, _ } = elixir_translator:translate_each(Expr, Scope),
+    { Tree, _ } = elixir_dispatch:dispatch_require(Meta, M, F, Args, S, fun() ->
+      apply(M, F, Args),
+      { { nil, 0 }, S }
+    end),
 
     try
       erl_eval:exprs([Tree], Binding)
@@ -346,14 +347,14 @@ eval_callbacks(Line, Module, Name, Args, RawS) ->
 % ERROR HANDLING
 
 format_error({ internal_function_overridden, { Name, Arity } }) ->
-  io_lib:format("function ~s/~B is internal and should not be overridden", [Name, Arity]);
+  io_lib:format("function ~ts/~B is internal and should not be overridden", [Name, Arity]);
 
 format_error({ invalid_module, Module}) ->
   io_lib:format("invalid module name: ~p", [Module]);
 
 format_error({ module_defined, Module }) ->
-  io_lib:format("redefining module ~s", [elixir_errors:inspect(Module)]);
+  io_lib:format("redefining module ~ts", [elixir_errors:inspect(Module)]);
 
 format_error({ module_in_definition, Module }) ->
-  io_lib:format("cannot define module ~s because it is currently being defined",
+  io_lib:format("cannot define module ~ts because it is currently being defined",
     [elixir_errors:inspect(Module)]).

lib/elixir/src/elixir_parser.yrl

@@ -1,39 +1,36 @@
-% Grammar for the Elixir language done with yecc
-% Copyright (C) 2011 Jose Valim
-
 Nonterminals
   grammar expr_list
-  expr block_expr fn_expr bracket_expr call_expr bracket_at_expr max_expr
+  expr paren_expr block_expr fn_expr bracket_expr call_expr bracket_at_expr max_expr
   base_expr matched_expr matched_op_expr unmatched_expr op_expr
   add_op mult_op unary_op two_op pipeline_op bin_concat_op
-  match_op send_op default_op when_op pipe_op in_op inc_op stab_op range_op
+  match_op send_op default_op when_op pipe_op in_op inc_op range_op
   andand_op oror_op and_op or_op comp_expr_op colon_colon_op three_op at_op
-  open_paren close_paren
+  open_paren close_paren empty_paren
   open_bracket close_bracket
   open_curly close_curly
   open_bit close_bit
   base_comma_expr comma_expr optional_comma_expr matched_comma_expr
-  call_args call_args_parens call_args_no_parens parens_call
-  stab_expr stab_expr_list
+  call_args call_args_parens call_args_parens_not_one call_args_no_parens parens_call
+  stab stab_op stab_expr stab_maybe_expr
   kw_eol kw_expr kw_comma kw_base
   matched_kw_expr matched_kw_comma matched_kw_base
   dot_op dot_ref dot_identifier dot_op_identifier dot_do_identifier
   dot_paren_identifier dot_punctuated_identifier dot_bracket_identifier
   var list bracket_access bit_string tuple
-  fn_block do_block do_eol end_eol block_eol block_item block_list
+  do_block fn_eol do_eol end_eol block_eol block_item block_list
   .
 
 Terminals
   identifier kw_identifier punctuated_identifier
   bracket_identifier paren_identifier do_identifier block_identifier
-  fn fn_paren 'end' '__aliases__'
+  fn 'end' aliases
   number signed_number atom bin_string list_string sigil
   dot_call_op comp_op op_identifier
   'not' 'and' 'or' 'xor' 'when' 'in' 'inlist' 'inbits' 'do'
   'true' 'false' 'nil'
   '=' '+' '-' '*' '/' '++' '--' '**' '//'
   '(' ')' '[' ']' '{' '}' '<<' '>>' '::'
-  eol ','  '&' '|'  '.' '^' '@' '<-' '<>' '->' '/>' '=~'
+  eol ','  '&' '|'  '.' '^' '@' '<-' '<>' '->' '|>' '/>' '=~'
   '&&' '||' '!' '...' '..'
   '<<<' '>>>' '&&&' '|||' '^^^' '~~~'
   .
@@ -41,10 +38,10 @@ Terminals
 Rootsymbol grammar.
 
 Left       5 do.
-Right     10 '->'.
+Right     10 stab_op.
 Left      20 ','.  % Solve nested call_args conflicts
-Right     30 when_op.
-Right     40 colon_colon_op.
+Right     30 colon_colon_op.
+Right     40 when_op.
 Right     50 default_op.
 Left      60 pipe_op.
 Left      70 inc_op.
@@ -80,8 +77,12 @@ grammar -> '$empty' : [nil].
 
 % Note expressions are on reverse order
 expr_list -> expr : ['$1'].
+expr_list -> empty_paren : [nil].
 expr_list -> expr_list eol expr : ['$3'|'$1'].
 
+paren_expr -> empty_paren : nil.
+paren_expr -> expr : '$1'.
+
 expr -> matched_expr : '$1'.
 expr -> unmatched_expr : '$1'.
 
@@ -140,14 +141,12 @@ matched_op_expr -> colon_colon_op matched_expr : { '$1', '$2' }.
 matched_op_expr -> comp_expr_op matched_expr : { '$1', '$2' }.
 
 block_expr -> parens_call call_args_parens do_block : build_identifier('$1', '$2' ++ '$3').
-block_expr -> parens_call call_args_parens call_args_parens do_block : { build_identifier('$1', '$2'), ?line('$1'), '$3' ++ '$4' }.
+block_expr -> parens_call call_args_parens call_args_parens do_block : build_nested_parens('$1', '$2', '$3' ++ '$4').
 block_expr -> dot_punctuated_identifier call_args_no_parens do_block : build_identifier('$1', '$2' ++ '$3').
 block_expr -> dot_do_identifier do_block : build_identifier('$1', '$2').
 block_expr -> dot_identifier call_args_no_parens do_block : build_identifier('$1', '$2' ++ '$3').
 
-fn_expr -> fn_paren call_args_parens fn_block : build_fn('$1', '$2', '$3').
-fn_expr -> fn call_args_no_parens fn_block : build_fn('$1', '$2', '$3').
-fn_expr -> fn fn_block : build_fn('$1', [], '$2').
+fn_expr -> fn_eol stab end_eol : build_fn('$1', build_stab(lists:reverse('$2'))).
 fn_expr -> call_expr : '$1'.
 
 call_expr -> dot_punctuated_identifier call_args_no_parens : build_identifier('$1', '$2').
@@ -160,11 +159,10 @@ call_expr -> max_expr : '$1'.
 
 max_expr -> bracket_expr : '$1'.
 max_expr -> parens_call call_args_parens : build_identifier('$1', '$2').
-max_expr -> parens_call call_args_parens call_args_parens : { build_identifier('$1', '$2'), ?line('$1'), '$3' }.
+max_expr -> parens_call call_args_parens call_args_parens : build_nested_parens('$1', '$2', '$3').
 max_expr -> dot_ref : '$1'.
 max_expr -> base_expr : '$1'.
-max_expr -> open_paren ')' : build_block([]).
-max_expr -> open_paren expr_list close_paren : build_block('$2').
+max_expr -> open_paren stab close_paren : build_stab(lists:reverse('$2')).
 
 bracket_expr -> dot_bracket_identifier bracket_access : build_access(build_identifier('$1', nil), '$2').
 bracket_expr -> max_expr bracket_access : build_access('$1', '$2').
@@ -174,29 +172,30 @@ bracket_at_expr -> at_op max_expr bracket_access : build_access(build_unary_op('
 bracket_at_expr -> bracket_at_expr bracket_access : build_access('$1', '$2').
 
 base_expr -> number : ?exprs('$1').
-base_expr -> signed_number : { element(4, '$1'), ?line('$1'), ?exprs('$1') }.
+base_expr -> signed_number : { element(4, '$1'), [{line,?line('$1')}], ?exprs('$1') }.
 base_expr -> atom : build_atom('$1').
 base_expr -> list : '$1'.
 base_expr -> tuple : '$1'.
 base_expr -> 'true' : ?op('$1').
 base_expr -> 'false' : ?op('$1').
 base_expr -> 'nil' : ?op('$1').
-base_expr -> '__aliases__' : '$1'.
+base_expr -> aliases : { '__aliases__', [{line,?line('$1')}], ?exprs('$1') }.
 base_expr -> bin_string  : build_bin_string('$1').
 base_expr -> list_string : build_list_string('$1').
 base_expr -> bit_string : '$1'.
-base_expr -> '&' : '$1'.
-base_expr -> '...' : { ?op('$1'), ?line('$1'), [] }.
+base_expr -> '&' : { '&', [{line,?line('$1')}], ?exprs('$1') }.
+base_expr -> '...' : { ?op('$1'), [{line,?line('$1')}], [] }.
 base_expr -> sigil : build_sigil('$1').
 
 %% Blocks
 
-fn_block -> '->' grammar 'end' : build_block('$2', false).
-
 do_block -> do_eol 'end' : [[{do,nil}]].
-do_block -> do_eol stab_expr_list end_eol : [[{ do, build_stab(lists:reverse('$2')) }]].
+do_block -> do_eol stab end_eol : [[{ do, build_stab(lists:reverse('$2')) }]].
 do_block -> do_eol block_list 'end' : [[{ do, nil }|'$2']].
-do_block -> do_eol stab_expr_list eol block_list 'end' : [[{ do, build_stab(lists:reverse('$2')) }|'$4']].
+do_block -> do_eol stab eol block_list 'end' : [[{ do, build_stab(lists:reverse('$2')) }|'$4']].
+
+fn_eol -> 'fn' : '$1'.
+fn_eol -> 'fn' eol : '$1'.
 
 do_eol -> 'do' : '$1'.
 do_eol -> 'do' eol : '$1'.
@@ -207,13 +206,18 @@ end_eol -> eol 'end' : '$2'.
 block_eol -> block_identifier : '$1'.
 block_eol -> block_identifier eol : '$1'.
 
-stab_expr_list -> stab_expr : ['$1'].
-stab_expr_list -> stab_expr_list eol stab_expr : ['$3'|'$1'].
+stab -> stab_expr : ['$1'].
+stab -> stab eol stab_expr : ['$3'|'$1'].
 
 stab_expr -> expr : '$1'.
-stab_expr -> comma_expr stab_op expr : build_op('$2', lists:reverse('$1'), '$3').
+stab_expr -> stab_op stab_maybe_expr : build_op('$1', [], '$2').
+stab_expr -> call_args_no_parens stab_op stab_maybe_expr : build_op('$2', '$1', '$3').
+stab_expr -> call_args_parens_not_one stab_op stab_maybe_expr : build_op('$2', '$1', '$3').
 
-block_item -> block_eol stab_expr_list eol : { ?exprs('$1'), build_stab(lists:reverse('$2')) }.
+stab_maybe_expr -> 'expr' : '$1'.
+stab_maybe_expr -> '$empty' : nil.
+
+block_item -> block_eol stab eol : { ?exprs('$1'), build_stab(lists:reverse('$2')) }.
 block_item -> block_eol : { ?exprs('$1'), nil }.
 
 block_list -> block_item : ['$1'].
@@ -228,6 +232,8 @@ open_paren -> '(' eol  : '$1'.
 close_paren -> ')'     : '$1'.
 close_paren -> eol ')' : '$2'.
 
+empty_paren -> open_paren ')' : nil.
+
 open_bracket  -> '['     : '$1'.
 open_bracket  -> '[' eol : '$1'.
 close_bracket -> ']'     : '$1'.
@@ -264,6 +270,8 @@ two_op -> '**' eol : '$1'.
 
 pipeline_op -> '=~' : '$1'.
 pipeline_op -> '=~' eol : '$1'.
+pipeline_op -> '|>' : '$1'.
+pipeline_op -> '|>' eol : '$1'.
 pipeline_op -> '/>' : '$1'.
 pipeline_op -> '/>' eol : '$1'.
 
@@ -354,7 +362,7 @@ dot_op -> '.' eol : '$1'.
 dot_identifier -> identifier : '$1'.
 dot_identifier -> matched_expr dot_op identifier : build_dot('$2', '$1', '$3').
 
-dot_ref -> matched_expr dot_op '__aliases__' : build_dot_ref('$2', '$1', '$3').
+dot_ref -> matched_expr dot_op aliases : build_dot_ref('$2', '$1', '$3').
 
 dot_op_identifier -> op_identifier : '$1'.
 dot_op_identifier -> matched_expr dot_op op_identifier : build_dot('$2', '$1', '$3').
@@ -372,7 +380,7 @@ dot_punctuated_identifier -> punctuated_identifier : '$1'.
 dot_punctuated_identifier -> matched_expr dot_op punctuated_identifier : build_dot('$2', '$1', '$3').
 
 parens_call -> dot_paren_identifier : '$1'.
-parens_call -> matched_expr dot_call_op : { '.', ?line('$2'), ['$1'] }. % Fun/local calls
+parens_call -> matched_expr dot_call_op : { '.', [{line,?line('$2')}], ['$1'] }. % Fun/local calls
 
 % Function calls
 
@@ -383,17 +391,21 @@ call_args_no_parens -> matched_comma_expr : lists:reverse('$1').
 call_args_no_parens -> matched_kw_base : ['$1'].
 call_args_no_parens -> matched_comma_expr ',' matched_kw_base : lists:reverse(['$3'|'$1']).
 
-base_comma_expr -> expr ',' : ['$1'].
-base_comma_expr -> base_comma_expr expr ',' : ['$2'|'$1'].
+call_args_parens_not_one -> open_paren ')' : [].
+call_args_parens_not_one -> open_paren matched_kw_base close_paren : ['$2'].
+call_args_parens_not_one -> open_paren matched_expr ',' call_args_no_parens close_paren : ['$2'|'$4'].
 
-comma_expr -> expr : ['$1'].
+base_comma_expr -> paren_expr ',' : ['$1'].
+base_comma_expr -> base_comma_expr paren_expr ',' : ['$2'|'$1'].
+
+comma_expr -> paren_expr : ['$1'].
 comma_expr -> kw_base : ['$1'].
 comma_expr -> base_comma_expr : '$1'.
-comma_expr -> base_comma_expr expr : ['$2'|'$1'].
+comma_expr -> base_comma_expr paren_expr : ['$2'|'$1'].
 comma_expr -> base_comma_expr kw_base : ['$2'|'$1'].
 
-optional_comma_expr -> expr : '$1'.
-optional_comma_expr -> expr ',' : '$1'.
+optional_comma_expr -> paren_expr : '$1'.
+optional_comma_expr -> paren_expr ',' : '$1'.
 
 call_args -> comma_expr : lists:reverse('$1').
 
@@ -404,7 +416,7 @@ call_args_parens -> open_paren call_args close_paren : '$2'.
 
 kw_eol  -> kw_identifier : '$1'.
 kw_eol  -> kw_identifier eol : '$1'.
-kw_expr -> kw_eol expr : { ?exprs('$1'),'$2' }.
+kw_expr -> kw_eol paren_expr : { ?exprs('$1'),'$2' }.
 
 kw_comma -> kw_expr ',' : ['$1'].
 kw_comma -> kw_comma kw_expr ',' : ['$2'|'$1'].
@@ -414,6 +426,7 @@ kw_base  -> kw_comma : lists:reverse('$1').
 kw_base  -> kw_comma kw_expr : lists:reverse(['$2'|'$1']).
 
 matched_kw_expr  -> kw_eol matched_expr : {?exprs('$1'),'$2'}.
+matched_kw_expr  -> kw_eol empty_paren : {?exprs('$1'),nil}.
 matched_kw_comma -> matched_kw_expr : ['$1'].
 matched_kw_comma -> matched_kw_expr ',' matched_kw_comma : ['$1'|'$3'].
 matched_kw_base  -> matched_kw_comma : '$1'.
@@ -427,18 +440,18 @@ bracket_access -> open_bracket kw_base close_bracket : { '$2', ?line('$1') }.
 list -> open_bracket ']' : [].
 list -> open_bracket kw_base close_bracket : '$2'.
 list -> open_bracket optional_comma_expr close_bracket : ['$2'].
-list -> open_bracket expr ',' call_args close_bracket : ['$2'|'$4'].
+list -> open_bracket paren_expr ',' call_args close_bracket : ['$2'|'$4'].
 
 % Tuple
 
 tuple -> open_curly '}' : build_tuple('$1', []).
 tuple -> open_curly optional_comma_expr close_curly : build_tuple('$1', ['$2']).
-tuple -> open_curly expr ',' call_args close_curly :  build_tuple('$1', ['$2'|'$4']).
+tuple -> open_curly paren_expr ',' call_args close_curly :  build_tuple('$1', ['$2'|'$4']).
 
 % Bitstrings
 
-bit_string -> open_bit '>>' : { '<<>>', ?line('$1'), [] }.
-bit_string -> open_bit call_args close_bit : { '<<>>', ?line('$1'), '$2' }.
+bit_string -> open_bit '>>' : { '<<>>', [{line,?line('$1')}], [] }.
+bit_string -> open_bit call_args close_bit : { '<<>>', [{line,?line('$1')}], '$2' }.
 
 Erlang code.
 
@@ -456,62 +469,62 @@ Erlang code.
 %% Operators
 
 build_op({ _, _, _ } = Op, Left, Right) ->
-  { ?exprs(Op), ?line(Op), [Left, Right] };
+  { ?exprs(Op), [{line,?line(Op)}], [Left, Right] };
 
 build_op({ BOp, Line }, { UOp, _, [Left] }, Right) when ?rearrange_bop(BOp), ?rearrange_uop(UOp) ->
-  { UOp, Line, [{ BOp, Line, [Left, Right] }] };
+  { UOp, [{line,Line}], [{ BOp, [{line,Line}], [Left, Right] }] };
 
 build_op(Op, Left, Right) ->
-  { ?op(Op), ?line(Op), [Left, Right] }.
+  { ?op(Op), [{line,?line(Op)}], [Left, Right] }.
 
 build_unary_op(Op, Expr) ->
-  { ?op(Op), ?line(Op), [Expr] }.
+  { ?op(Op), [{line,?line(Op)}], [Expr] }.
 
 build_tuple(_Marker, [Left, Right]) ->
   { Left, Right };
 
 build_tuple(Marker, Args) ->
-  { '{}', ?line(Marker), Args }.
+  { '{}', [{line,?line(Marker)}], Args }.
 
 %% Blocks
 
-build_block(Exprs) -> build_block(Exprs, true).
-
-build_block([], _)                                         -> nil;
-build_block([nil], _)                                      -> { '__block__', 0, [nil] };
-build_block([{Op,_,[_]}]=Exprs, _) when ?rearrange_uop(Op) -> { '__block__', 0, Exprs };
-build_block([Expr], _) when not is_list(Expr)              -> Expr;
-build_block(Exprs, true)                                   -> { '__block__', 0, lists:reverse(Exprs) };
-build_block(Exprs, false)                                  -> { '__block__', 0, Exprs }.
+build_block([nil])                                      -> { '__block__', [], [nil] };
+build_block([{Op,_,[_]}]=Exprs) when ?rearrange_uop(Op) -> { '__block__', [], Exprs };
+build_block([Expr]) when not is_list(Expr)              -> Expr;
+build_block(Exprs)                                      -> { '__block__', [], Exprs }.
 
 %% Dots
 
-build_dot_ref(Dot, { '__aliases__', _, Left }, { '__aliases__', _, Right }) ->
-  { '__aliases__', ?line(Dot), Left ++ Right };
+build_dot_ref(Dot, { '__aliases__', _, Left }, { 'aliases', _, Right }) ->
+  { '__aliases__', [{line,?line(Dot)}], Left ++ Right };
 
-build_dot_ref(Dot, Other, { '__aliases__', _, Right }) ->
-  { '__aliases__', ?line(Dot), [Other|Right] }.
+build_dot_ref(Dot, Other, { 'aliases', _, Right }) ->
+  { '__aliases__', [{line,?line(Dot)}], [Other|Right] }.
 
 build_dot(Dot, Left, Right) ->
-  { '.', ?line(Dot), [Left, extract_identifier(Right)] }.
+  { '.', [{line,?line(Dot)}], [Left, extract_identifier(Right)] }.
 
 %% Identifiers
 
-build_identifier({ '.', Line, _ } = Dot, Args) ->
+build_nested_parens(Dot, Args1, Args2) ->
+  Identifier = build_identifier(Dot, Args1),
+  { Identifier, ?line(Identifier), Args2 }.
+
+build_identifier({ '.', Meta, _ } = Dot, Args) ->
   FArgs = case Args of
     nil -> [];
     _ -> Args
   end,
-  { Dot, Line, FArgs };
+  { Dot, Meta, FArgs };
 
-build_identifier({ Keyword, Line }, Args) when Keyword == fn; Keyword == fn_paren ->
-  { fn, Line, Args };
+build_identifier({ Keyword, Line }, Args) when Keyword == fn ->
+  { fn, [{line,Line}], Args };
 
 build_identifier({ op_identifier, Line, Identifier }, Args) ->
-  { '__ambiguousop__', Line, [{ Identifier, Line, nil }|Args] };
+  { '__ambiguousop__', [{line,Line}], [{ Identifier, [{line,Line}], nil }|Args] };
 
 build_identifier({ _, Line, Identifier }, Args) ->
-  { Identifier, Line, Args }.
+  { Identifier, [{line,Line}], Args }.
 
 extract_identifier({ Kind, _, Identifier }) when
     Kind == identifier; Kind == punctuated_identifier; Kind == bracket_identifier;
@@ -522,45 +535,49 @@ extract_identifier(Other) -> Other.
 
 %% Fn
 
-build_fn(Op, Args, Expr) ->
-  Line = ?line(Op),
-  Stab = [{do, { '->', Line, [{ Args, Expr }] }}],
-  { fn, Line, [Stab] }.
+build_fn(Op, Stab) ->
+  { fn, [{line,?line(Op)}], [[{ do, Stab }]] }.
 
 %% Access
 
 build_access(Expr, Access) ->
-  { access, ?line(Access), [ Expr, ?op(Access) ] }.
+  Meta = [{line,?line(Access)}],
+  { { '.', Meta, ['Elixir-Kernel', access] }, Meta, [ Expr, ?op(Access) ] }.
 
 %% Interpolation aware
 
 build_sigil({ sigil, Line, Sigil, Parts, Modifiers }) ->
-  { list_to_atom([$_,$_,Sigil,$_,$_]), Line, [ { '<<>>', Line, Parts }, Modifiers ] }.
+  Meta = [{line,Line}],
+  { list_to_atom([$_,$_,Sigil,$_,$_]), Meta, [ { '<<>>', Meta, Parts }, Modifiers ] }.
 
 build_bin_string({ bin_string, _Line, [H] }) when is_binary(H) -> H;
-build_bin_string({ bin_string, Line, Args }) -> { '<<>>', Line, Args }.
+build_bin_string({ bin_string, Line, Args }) -> { '<<>>', [{line,Line}], Args }.
 
-build_list_string({ list_string, _Line, [H] }) when is_binary(H) -> binary_to_list(H);
-build_list_string({ list_string, Line, Args }) -> { binary_to_list, Line, [{ '<<>>', Line, Args}] }.
+build_list_string({ list_string, _Line, [H] }) when is_binary(H) -> unicode:characters_to_list(H);
+build_list_string({ list_string, Line, Args }) ->
+  Meta = [{line,Line}],
+  { { '.', Meta, [unicode, characters_to_list] }, Meta, [{ '<<>>', Meta, Args}] }.
 
 build_atom({ atom, _Line, Atom }) when is_atom(Atom) -> Atom;
-build_atom({ atom, Line, Args }) -> { binary_to_atom, Line, [{ '<<>>', Line, Args }, utf8] }.
+build_atom({ atom, Line, Args }) ->
+  Meta = [{line,Line}],
+  { { '.', Meta, [erlang, binary_to_atom] }, Meta, [{ '<<>>', Meta, Args }, utf8] }.
 
 %% Keywords
 
-build_stab([{ '->', Line, [Left, Right] }|T]) ->
-  { '->', Line, build_stab(T, Left, [Right], []) };
+build_stab([{ '->', Meta, [Left, Right] }|T]) ->
+  { '->', Meta, build_stab(T, Left, [Right], []) };
 
 build_stab(Else) ->
-  build_block(Else, false).
+  build_block(Else).
 
 build_stab([{ '->', _, [Left, Right] }|T], Marker, Temp, Acc) ->
-  H = { Marker, build_block(Temp) },
+  H = { Marker, build_block(lists:reverse(Temp)) },
   build_stab(T, Left, [Right], [H|Acc]);
 
 build_stab([H|T], Marker, Temp, Acc) ->
   build_stab(T, Marker, [H|Temp], Acc);
 
 build_stab([], Marker, Temp, Acc) ->
-  H = { Marker, build_block(Temp) },
+  H = { Marker, build_block(lists:reverse(Temp)) },
   lists:reverse([H|Acc]).

lib/elixir/src/elixir_partials.erl

@@ -6,28 +6,28 @@
 handle(Original, S) ->
   handle(Original, S, default).
 
-handle({ _, Line, Args } = Original, S, Opt) when is_list(Args), S#elixir_scope.context /= assign ->
+handle({ _, Meta, Args } = Original, S, Opt) when is_list(Args), S#elixir_scope.context /= match ->
   case convert(Args, S, Opt) of
     { Call, Def, SC } when Def /= [] ->
-      Final = validate(Line, Def, SC),
+      Final = validate(Meta, Def, SC),
       Block = setelement(3, Original, Call),
-      elixir_translator:translate_fn(Line, [{ Final, Block }], SC);
+      elixir_translator:translate_fn(Meta, [{ Final, Block }], SC);
     _ -> error
   end;
 
 handle(_Original, _S, _Opt) ->
   error.
 
-validate(Line, Def, S) ->
-  validate(Line, lists:sort(Def), 1, S).
+validate(Meta, Def, S) ->
+  validate(Meta, lists:sort(Def), 1, S).
 
-validate(Line, [{ Pos, Item }|T], Pos, S) ->
-  [Item|validate(Line, T, Pos + 1, S)];
+validate(Meta, [{ Pos, Item }|T], Pos, S) ->
+  [Item|validate(Meta, T, Pos + 1, S)];
 
-validate(Line, [{ Pos, _ }|_], Expected, S) ->
-  elixir_errors:syntax_error(Line, S#elixir_scope.file, "partial variable &~w cannot be defined without &~w", [Pos, Expected]);
+validate(Meta, [{ Pos, _ }|_], Expected, S) ->
+  elixir_errors:syntax_error(Meta, S#elixir_scope.file, "partial variable &~w cannot be defined without &~w", [Pos, Expected]);
 
-validate(_Line, [], _Pos, _S) ->
+validate(_Meta, [], _Pos, _S) ->
   [].
 
 %% This function receives arguments and then checks
@@ -37,14 +37,14 @@ validate(_Line, [], _Pos, _S) ->
 %% function definition and the third one is the new scope.
 convert(List, S, Opt) -> convert(List, S, Opt, [], []).
 
-convert([{'|', Line, [_, _] = Args}|T], S, allow_tail, CallAcc, DefAcc) ->
+convert([{'|', Meta, [_, _] = Args}|T], S, allow_tail, CallAcc, DefAcc) ->
   { NewArgs, NewDef, NewS } = convert(Args, S, allow_tail, [], DefAcc),
-  convert(T, NewS, allow_tail, [{ '|', Line, NewArgs}|CallAcc], NewDef);
+  convert(T, NewS, allow_tail, [{ '|', Meta, NewArgs}|CallAcc], NewDef);
 
-convert([{'&', Line, [Pos]}|T], S, Opt, CallAcc, DefAcc) ->
+convert([{'&', Meta, [Pos]}|T], S, Opt, CallAcc, DefAcc) ->
   case lists:keyfind(Pos, 1, DefAcc) of
     false ->
-      { Var, SC } = elixir_scope:build_ex_var(Line, S),
+      { Var, SC } = elixir_scope:build_ex_var(?line(Meta), S),
       convert(T, SC, Opt, [Var|CallAcc], [{Pos,Var}|DefAcc]);
     {Pos,Var} ->
       convert(T, S, Opt, [Var|CallAcc], DefAcc)

lib/elixir/src/elixir_quote.erl

@@ -1,56 +1,129 @@
 -module(elixir_quote).
--export([quote/3, linify/2]).
+-export([quote/3, linify/2, unquote/5, user_quote/2]).
 -include("elixir.hrl").
 
 %% Apply the line from site call on quoted contents.
+linify(Line, Exprs) when is_integer(Line) ->
+  do_linify(Line, Exprs).
 
-linify(Line, { Left, 0, Right }) ->
-  { linify(Line, Left), Line, linify(Line, Right) };
+do_linify(Line, { Left, Meta, Right }) ->
+  NewMeta = case ?line(Meta) of
+    0 -> lists:keystore(line, 1, Meta, { line, Line });
+    _ -> Meta
+  end,
+  { do_linify(Line, Left), NewMeta, do_linify(Line, Right) };
 
-linify(Line, Tuple) when is_tuple(Tuple) ->
-  list_to_tuple(linify(Line, tuple_to_list(Tuple)));
+do_linify(Line, { Left, Right }) ->
+  { do_linify(Line, Left), do_linify(Line, Right) };
 
-linify(Line, List) when is_list(List) ->
-  [linify(Line, X) || X <- List];
+do_linify(Line, List) when is_list(List) ->
+  [do_linify(Line, X) || X <- List];
 
-linify(_, Else) -> Else.
+do_linify(_, Else) -> Else.
 
-%% Translation
+%% Some expressions cannot be unquoted at compilation time.
+%% This function is responisble for doing runtime unquoting.
+unquote(Meta, Left, { '__aliases__', _, Args }, nil, _File) ->
+  { '__aliases__', Meta, [Left|Args] };
 
-quote({ 'unquote_splicing', Line, _ } = Expr, Q, S) ->
-  do_quote({ '__block__', Line, [Expr] }, Q, S);
+unquote(Meta, Left, Right, nil, _File) when is_atom(Right) ->
+  case atom_to_list(Right) of
+    "Elixir-" ++ _ ->
+      { '__aliases__', Meta, [Left, Right] };
+    _ ->
+      { { '.', Meta, [Left, Right] }, Meta, [] }
+  end;
+
+unquote(Meta, Left, Right, Args, _File) when is_atom(Right) ->
+  { { '.', Meta, [Left, Right] }, Meta, Args };
+
+unquote(Meta, _Left, _Right, _Args, File) ->
+  elixir_errors:syntax_error(Meta, File, "expected unquote after dot to return an atom or an alias").
+
+%% Similar to quote, but the given code is meant to come directly from the user.
+%% Basically, lines are kept and hygiene mechanisms are disabled.
+user_quote(Expr, S) ->
+  quote(Expr, #elixir_quote{
+    line=keep,
+    vars_hygiene=nil,
+    aliases_hygiene=false,
+    imports_hygiene=false,
+    unquote=true
+  }, S).
+
+%% Quotes an expression into Erlang's AST
+
+quote({ 'unquote_splicing', Meta, [_] } = Expr, #elixir_quote{unquote=true} = Q, S) ->
+  do_quote({ '__block__', Meta, [Expr] }, Q, S);
 
 quote(Else, Q, S) ->
   do_quote(Else, Q, S).
 
-do_quote({ unquote, _Line, [Expr] }, #elixir_quote{unquote=true}, S) ->
+do_quote({ quote, _, Args } = Tuple, #elixir_quote{unquote=true} = Q, S) when length(Args) == 1; length(Args) == 2 ->
+  do_quote_tuple(Tuple, Q#elixir_quote{unquote=false}, S);
+
+do_quote({ unquote, _Meta, [Expr] }, #elixir_quote{unquote=true}, S) ->
   elixir_translator:translate_each(Expr, S);
 
-do_quote({ { { '.', Line, [Left, unquote] }, _Line, [Expr] }, CallLine, Args }, Q, S) ->
-  Rewritten = { { '.', Line, [Left, { unquote, Line, [Expr] }] }, CallLine, Args },
-  do_quote(Rewritten, Q, S);
+do_quote({ function, Meta, [{ '/', _, [{F, _, C}, A]}] = Args },
+    #elixir_quote{imports_hygiene=true} = Q, S) when is_atom(F), is_integer(A), is_atom(C) ->
+  do_quote_fa(function, Meta, Args, F, A, Q, S);
 
-do_quote({ { '.', Line, [Left, unquote] }, _Line, [Expr] }, Q, S) ->
-  Rewritten = { { '.', Line, [Left, { unquote, Line, [Expr] }] }, Line, [] },
-  do_quote(Rewritten, Q, S);
+do_quote({ { '.', _, [_, function] } = Target, Meta, [{ '/', _, [{F, _, C}, A]}] = Args },
+    #elixir_quote{imports_hygiene=true} = Q, S) when is_atom(F), is_integer(A), is_atom(C) ->
+  do_quote_fa(Target, Meta, Args, F, A, Q, S);
 
-do_quote({ Left, Line, nil }, Q, S) when is_atom(Left) ->
+do_quote({ 'alias!', _Meta, [Expr] }, Q, S) ->
+  do_quote(Expr, Q#elixir_quote{aliases_hygiene=false}, S);
+
+do_quote({ '__aliases__', Meta, [H|T] } = Alias, #elixir_quote{aliases_hygiene=true} = Q, S) when is_atom(H) and (H /= 'Elixir') ->
+  Annotation = case elixir_aliases:expand(Alias, S#elixir_scope.aliases, S#elixir_scope.macro_aliases) of
+    Atom when is_atom(Atom) -> Atom;
+    Aliases when is_list(Aliases) -> false
+  end,
+
+  Line = ?line(Meta),
+  { TAliases, SA } = do_quote([H|T], Q, S),
+
+  { { tuple, Line, [
+    { atom, Line, '__aliases__' },
+    meta([{alias,Annotation}|Meta], Q),
+    TAliases
+  ] }, SA };
+
+do_quote({ { { '.', Meta, [Left, unquote] }, _, [Expr] }, _, Args }, #elixir_quote{unquote=true} = Q, S) ->
+  All  = [Left, { unquote, Meta, [Expr] }, Args, S#elixir_scope.file],
+  { TAll, TS } = lists:mapfoldl(fun(X, Acc) -> do_quote(X, Q, Acc) end, S, All),
+  { ?wrap_call(?line(Meta), elixir_quote, unquote, [meta(Meta, Q)|TAll]), TS };
+
+do_quote({ { '.', Meta, [Left, unquote] }, _, [Expr] }, #elixir_quote{unquote=true} = Q, S) ->
+  All = [Left, { unquote, Meta, [Expr] }, nil, S#elixir_scope.file],
+  { TAll, TS } = lists:mapfoldl(fun(X, Acc) -> do_quote(X, Q, Acc) end, S, All),
+  { ?wrap_call(?line(Meta), elixir_quote, unquote, [meta(Meta, Q)|TAll]), TS };
+
+do_quote({ Left, Meta, nil }, Q, S) when is_atom(Left) ->
+  Line  = ?line(Meta),
   Tuple = { tuple, Line, [
     { atom, Line, Left },
-    line(Line, Q),
-    { atom, Line, Q#elixir_quote.marker }
+    meta(Meta, Q),
+    { atom, Line, Q#elixir_quote.vars_hygiene }
   ] },
   { Tuple, S };
 
-do_quote({ Left, Line, Right }, Q, S) ->
-  { TLeft, LS } = do_quote(Left, Q, S),
-  { TRight, RS } = do_quote(Right, Q, LS),
+do_quote({ Name, Meta, ArgsOrAtom } = Tuple, #elixir_quote{imports_hygiene=true} = Q, S) when is_atom(Name) ->
+  Arity = case is_atom(ArgsOrAtom) of
+    true  -> 0;
+    false -> length(ArgsOrAtom)
+  end,
 
-  %% We need to remove line numbers from quoted exprs
-  %% otherwise the line number quotes in the macro will
-  %% get mixed with the unquoted contents
-  Tuple = { tuple, Line, [TLeft, line(Line, Q), TRight] },
-  { Tuple, RS };
+  case (lists:keyfind(import, 1, Meta) == false) andalso
+      elixir_dispatch:find_import(Meta, Name, Arity, S) of
+    false    -> do_quote_tuple(Tuple, Q, S);
+    Receiver -> do_quote_tuple({ Name, [{import,Receiver}|Meta], ArgsOrAtom }, Q, S)
+  end;
+
+do_quote({ _, _, _ } = Tuple, Q, S) ->
+  do_quote_tuple(Tuple, Q, S);
 
 % Handle two item tuples but still allow them to be spliced.
 do_quote({ Left, Right }, Q, S) when
@@ -58,26 +131,46 @@ do_quote({ Left, Right }, Q, S) when
   not is_tuple(Right) orelse (element(1, Right) /= unquote_splicing) ->
   { TLeft, LS } = do_quote(Left, Q, S),
   { TRight, RS } = do_quote(Right, Q, LS),
-  { { tuple, int_line(Q), [TLeft, TRight] }, RS };
+  { { tuple, line(Q), [TLeft, TRight] }, RS };
 
 do_quote({ Left, Right }, Q, S) ->
-  do_quote({ '{}', int_line(Q), [Left, Right] }, Q, S);
+  do_quote({ '{}', [{line,line(Q)}], [Left, Right] }, Q, S);
 
 do_quote(List, Q, S) when is_list(List) ->
   splice(List, Q, [], [], S);
 
 do_quote(Number, Q, S) when is_integer(Number) ->
-  { { integer, int_line(Q), Number }, S };
+  { { integer, line(Q), Number }, S };
 
 do_quote(Number, Q, S) when is_float(Number) ->
-  { { float, int_line(Q), Number }, S };
+  { { float, line(Q), Number }, S };
 
 do_quote(Atom, Q, S) when is_atom(Atom) ->
-  { { atom, int_line(Q), Atom }, S };
+  { { atom, line(Q), Atom }, S };
 
 do_quote(Bitstring, _Q, S) when is_bitstring(Bitstring) ->
   { elixir_tree_helpers:abstract_syntax(Bitstring), S }.
 
+do_quote_tuple({ Left, Meta, Right }, Q, S) ->
+  { TLeft, LS } = do_quote(Left, Q, S),
+  { TRight, RS } = do_quote(Right, Q, LS),
+
+  %% We need to remove line numbers from quoted exprs
+  %% otherwise the line number quotes in the macro will
+  %% get mixed with the unquoted contents
+  Tuple = { tuple, ?line(Meta), [TLeft, meta(Meta, Q), TRight] },
+  { Tuple, RS }.
+
+do_quote_fa(Target, Meta, Args, F, A, Q, S) ->
+  NewMeta =
+    case (lists:keyfind(import_fa, 1, Meta) == false) andalso
+        elixir_dispatch:find_import(Meta, F, A, S) of
+      false    -> Meta;
+      Receiver -> [{ import_fa, Receiver }|Meta]
+    end,
+
+  do_quote_tuple({ Target, NewMeta, Args }, Q, S).
+
 % Loop through the list finding each unquote_splicing entry.
 
 splice([{ unquote_splicing, _, [Args] }|T], #elixir_quote{unquote=true} = Q, Buffer, Acc, S) ->
@@ -85,10 +178,10 @@ splice([{ unquote_splicing, _, [Args] }|T], #elixir_quote{unquote=true} = Q, Buf
   { TArgs, TS } = elixir_translator:translate_each(Args, NewS),
   splice(T, Q, [], [TArgs|NewAcc], TS);
 
-splice([{ '|', Line, [{ unquote_splicing, _, [_] } = Left, Right] }], #elixir_quote{unquote=true} = Q, Buffer, Acc, S) ->
+splice([{ '|', Meta, [{ unquote_splicing, _, [_] } = Left, Right] }], #elixir_quote{unquote=true} = Q, Buffer, Acc, S) ->
   { TLeft, SL }  = splice([Left], Q, Buffer, Acc, S),
   { TRight, SR } = do_quote(Right, Q, SL),
-  { { op, Line, '++', TLeft, TRight }, SR };
+  { { op, ?line(Meta), '++', TLeft, TRight }, SR };
 
 splice([H|T], Q, Buffer, Acc, S) ->
   splice(T, Q, [H|Buffer], Acc, S);
@@ -97,12 +190,10 @@ splice([], Q, Buffer, Acc, S) ->
   { NewAcc, NewS } = from_buffer_to_acc(Buffer, Q, Acc, S),
   case NewAcc of
     [] ->
-      { { nil, int_line(Q) }, NewS };
-    [List] ->
-      { List, NewS };
-    _ ->
-      List = elixir_tree_helpers:build_simple_reverse_list(int_line(Q), NewAcc),
-      { ?ELIXIR_WRAP_CALL(int_line(Q), lists, append, [List]), NewS }
+      { { nil, line(Q) }, NewS };
+    [H|T] ->
+      Line = line(Q),
+      { lists:foldl(fun(L, R) -> { op, Line, '++', L, R } end, H, T), NewS }
   end.
 
 from_buffer_to_acc([], _Q, Acc, S) ->
@@ -110,14 +201,19 @@ from_buffer_to_acc([], _Q, Acc, S) ->
 
 from_buffer_to_acc(Buffer, Q, Acc, S) ->
   { New, NewS } = elixir_tree_helpers:build_reverse_list(
-    fun(X, AccS) -> do_quote(X, Q, AccS) end, Buffer, int_line(Q), S),
+    fun(X, AccS) -> do_quote(X, Q, AccS) end, Buffer, line(Q), S),
   { [New|Acc], NewS }.
 
-line(Line, #elixir_quote{line=keep})  -> { integer, Line, Line };
-line(_Line, #elixir_quote{line=Line}) -> Line.
+meta(Meta, #elixir_quote{line=keep}) ->
+  elixir_tree_helpers:abstract_syntax(Meta);
+meta(Meta, #elixir_quote{line=nil}) ->
+  elixir_tree_helpers:abstract_syntax(lists:keydelete(line, 1, Meta));
+meta(Meta, #elixir_quote{line=Line} = Q) ->
+  { cons,
+    line(Q),
+    { tuple, line(Q), [{ atom, line(Q), line }, Line] },
+    elixir_tree_helpers:abstract_syntax(lists:keydelete(line, 1, Meta))
+  }.
 
-int_line(Q) ->
-  case line(0, Q) of
-    { integer, _, Line } -> Line;
-    _ -> 0
-  end.
+line(#elixir_quote{line={ integer, _, Line }}) -> Line;
+line(_) -> 0.

lib/elixir/src/elixir_scope.erl

@@ -1,100 +1,132 @@
 %% Convenience functions used to manipulate scope
 %% and its variables.
 -module(elixir_scope).
--export([translate_var/4,
+-export([translate_var/5,
   build_erl_var/2, build_ex_var/2,
-  serialize/1, deserialize/1, deserialize/2,
-  to_erl_env/1, to_ex_env/1, filename/1,
-  umergev/2, umergec/2
-  ]).
+  serialize/1, deserialize/1,
+  serialize_with_vars/2, deserialize_with_vars/2,
+  to_erl_env/1, to_ex_env/1,
+  umergev/2, umergec/2, merge_clause_vars/2
+]).
 -include("elixir.hrl").
 -compile({parse_transform, elixir_transform}).
 
-translate_var(Line, Name, Kind, S) ->
+translate_var(Meta, Name, Kind, S, Callback) ->
+  Line = ?line(Meta),
   Vars = S#elixir_scope.vars,
 
   case Name of
-    '_' -> { {var, Line, Name}, S };
+    '_' ->
+      { { var, Line, Name }, S };
     _ ->
       case S#elixir_scope.context of
-        assign ->
+        match ->
           TempVars = S#elixir_scope.temp_vars,
-          case { orddict:is_key(Name, Vars), orddict:find(Name, TempVars) } of
+          case { orddict:is_key({ Name, Kind }, Vars), orddict:find(Name, TempVars) } of
             { true, { ok, Kind } } ->
-              { {var, Line, orddict:fetch(Name, Vars) }, S };
+              { { var, Line, orddict:fetch({ Name, Kind }, Vars) }, S };
             { Else, _ } ->
-              { NewVar, NS } = case Else or S#elixir_scope.noname of
-                true -> build_erl_var(Line, S);
-                false -> { {var, Line, Name}, S }
+              { NewVar, NS } = if
+                Kind /= nil -> build_erl_var(Line, S);
+                Else -> build_erl_var(Line, Name, Name, S);
+                S#elixir_scope.noname -> build_erl_var(Line, Name, Name, S);
+                true -> { { var, Line, Name }, S }
               end,
               RealName = element(3, NewVar),
               ClauseVars = S#elixir_scope.clause_vars,
               { NewVar, NS#elixir_scope{
-                vars=orddict:store(Name, RealName, Vars),
+                vars=orddict:store({ Name, Kind }, RealName, Vars),
                 temp_vars=orddict:store(Name, Kind, TempVars),
-                clause_vars=orddict:store({ Name, Kind }, RealName, ClauseVars)
+                clause_vars=if
+                  ClauseVars == nil -> nil;
+                  true -> orddict:store({ Name, Kind }, RealName, ClauseVars)
+                end
               } }
           end;
         _ ->
-          case orddict:is_key(Name, Vars) of
-            false -> elixir_translator:translate_each({Name, Line, []}, S);
-            true  -> { {var, Line, orddict:fetch(Name, Vars) }, S }
+          case orddict:find({ Name, Kind }, Vars) of
+            { ok, VarName } -> { { var, Line, VarName }, S };
+            error -> Callback()
           end
       end
   end.
 
 % Handle variables translation
 
-build_erl_var(Line, #elixir_scope{counter=Counter} = S) ->
-  NS = S#elixir_scope{counter=Counter+1},
-  Var = { var, Line, ?ELIXIR_ATOM_CONCAT(["_@", Counter]) },
+build_ex_var(Line, S)  -> build_ex_var(Line, '', "_", S).
+build_erl_var(Line, S) -> build_erl_var(Line, '', "_", S).
+
+build_var_counter(Key, #elixir_scope{counter=Counter} = S) ->
+  New = orddict:update_counter(Key, 1, Counter),
+  { orddict:fetch(Key, New), S#elixir_scope{counter=New} }.
+
+build_erl_var(Line, Key, Name, S) when is_integer(Line) ->
+  { Counter, NS } = build_var_counter(Key, S),
+  Var = { var, Line, ?atom_concat([Name, "@", Counter]) },
   { Var, NS }.
 
-build_ex_var(Line, #elixir_scope{counter=Counter} = S) ->
-  NS = S#elixir_scope{counter=Counter+1},
-  Var = { ?ELIXIR_ATOM_CONCAT(["_@", Counter]), Line, nil },
+build_ex_var(Line, Key, Name, S) when is_integer(Line) ->
+  Context = case S#elixir_scope.module of
+    nil -> 'Elixir';
+    Mod -> Mod
+  end,
+
+  { Counter, NS } = build_var_counter(Key, S),
+  Var = { ?atom_concat([Name, "@", Counter]), [{line,Line}], Context },
   { Var, NS }.
 
 % Handle Macro.Env conversion
 
-to_erl_env(Scope) ->
-  elixir_tree_helpers:abstract_syntax(to_ex_env(Scope)).
-
-to_ex_env({ Line, Tuple }) when element(1, Tuple) == 'Elixir.Macro.Env' ->
-  setelement(4, Tuple, Line);
+to_erl_env({ 'Elixir.Macro.Env', Module, File, _Line, Function, Aliases, Context, Requires, Functions, Macros }) ->
+  #elixir_scope{module=Module,file=File,
+    function=Function,aliases=Aliases,context=Context,
+    requires=Requires,macros=Macros,functions=Functions}.
 
 to_ex_env({ Line, #elixir_scope{module=Module,file=File,
     function=Function,aliases=Aliases,context=Context,
-    requires=Requires,macros=Macros} }) ->
-  { 'Elixir.Macro.Env', Module, File, Line, Function, Aliases, Context, Requires, Macros }.
-
-filename(#elixir_scope{file=File}) -> File;
-filename(Other) -> element(3, Other).
+    requires=Requires,macros=Macros,functions=Functions} }) when is_integer(Line) ->
+  { 'Elixir.Macro.Env', Module, File, Line, Function, Aliases, Context, Requires, Functions, Macros }.
 
 % Provides a tuple with only the scope information we want to serialize.
 
 serialize(S) ->
   elixir_tree_helpers:abstract_syntax(
-    { S#elixir_scope.file, S#elixir_scope.functions, S#elixir_scope.check_clauses, S#elixir_scope.macro,
-      S#elixir_scope.requires, S#elixir_scope.macros, S#elixir_scope.aliases, S#elixir_scope.scheduled }
+    { S#elixir_scope.file, S#elixir_scope.functions, S#elixir_scope.check_clauses,
+      S#elixir_scope.requires, S#elixir_scope.macros, S#elixir_scope.aliases,
+      S#elixir_scope.macro_functions, S#elixir_scope.macro_macros, S#elixir_scope.macro_aliases,
+      S#elixir_scope.scheduled }
   ).
 
+serialize_with_vars(Line, S) when is_integer(Line) ->
+  { Vars, _ } = orddict:fold(fun({ Key, Kind }, Value, { Acc, Counter }) ->
+    { { cons, Line, { tuple, Line, [
+      { atom, Line, Key },
+      { atom, Line, Kind },
+      { atom, Line, ?atom_concat(["_@", Counter]) },
+      { var,  Line, Value }
+    ] }, Acc }, Counter + 1 }
+  end, { { nil, Line }, 0 }, S#elixir_scope.vars),
+  { serialize(S), Vars }.
+
 % Fill in the scope with the variables serialization set in serialize_scope.
 
-deserialize(Tuple) -> deserialize(Tuple, []).
+deserialize(Tuple) -> deserialize_with_vars(Tuple, []).
 
-deserialize({ File, Functions, CheckClauses, Macro, Requires, Macros, Aliases, Scheduled }, Vars) ->
+deserialize_with_vars({ File, Functions, CheckClauses, Requires, Macros,
+                        Aliases, MacroFunctions, MacroMacros, MacroAliases, Scheduled }, Vars) ->
   #elixir_scope{
     file=File,
     functions=Functions,
     check_clauses=CheckClauses,
-    macro=Macro,
     requires=Requires,
     macros=Macros,
     aliases=Aliases,
+    macro_functions=MacroFunctions,
+    macro_macros=MacroMacros,
+    macro_aliases=MacroAliases,
     scheduled=Scheduled,
     vars=orddict:from_list(Vars),
-    counter=length(Vars)
+    counter=[{'',length(Vars)}]
   }.
 
 % Receives two scopes and return a new scope based on the second
@@ -103,14 +135,11 @@ deserialize({ File, Functions, CheckClauses, Macro, Requires, Macros, Aliases, S
 umergev(S1, S2) ->
   V1 = S1#elixir_scope.vars,
   V2 = S2#elixir_scope.vars,
-  Q1 = S1#elixir_scope.quote_vars,
-  Q2 = S2#elixir_scope.quote_vars,
   C1 = S1#elixir_scope.clause_vars,
   C2 = S2#elixir_scope.clause_vars,
   S2#elixir_scope{
     vars=orddict:merge(fun var_merger/3, V1, V2),
-    quote_vars=orddict:merge(fun var_merger/3, Q1, Q2),
-    clause_vars=orddict:merge(fun clause_var_merger/3, C1, C2)
+    clause_vars=merge_clause_vars(C1, C2)
   }.
 
 % Receives two scopes and return a new scope based on the first
@@ -127,16 +156,21 @@ umergec(S1, S2) ->
 
 % Merge variables trying to find the most recently created.
 
-clause_var_merger({ Var, _ }, K1, K2) ->
-  var_merger(Var, K1, K2).
+merge_clause_vars(nil, _C2) -> nil;
+merge_clause_vars(_C1, nil) -> nil;
+merge_clause_vars(C1, C2)   ->
+  orddict:merge(fun var_merger/3, C1, C2).
 
-var_merger(Var, Var, K2) -> K2;
-var_merger(Var, K1, Var) -> K1;
+var_merger({ Var, _ }, Var, K2) -> K2;
+var_merger({ Var, _ }, K1, Var) -> K1;
 var_merger(_Var, K1, K2) ->
-  V1 = var_number(atom_to_list(K1)),
-  V2 = var_number(atom_to_list(K2)),
-  if V1 > V2 -> K1;
-     true -> K2
+  V1 = var_number(atom_to_list(K1), []),
+  V2 = var_number(atom_to_list(K2), []),
+  case V1 > V2 of
+    true  -> K1;
+    false -> K2
   end.
 
-var_number([_,_|T]) -> list_to_integer(T).
+var_number([$@|T], _Acc) -> var_number(T, []);
+var_number([H|T], Acc)   -> var_number(T, [H|Acc]);
+var_number([], Acc)      -> list_to_integer(lists:reverse(Acc)).

lib/elixir/src/elixir_tokenizer.erl

@@ -10,16 +10,6 @@
   existing_atoms_only=false
 }).
 
--define(is_digit(S), S >= $0 andalso S =< $9).
--define(is_hex(S), ?is_digit(S) orelse (S >= $A andalso S =< $F) orelse (S >= $a andalso S =< $f)).
--define(is_bin(S), S >= $0 andalso S =< $1).
--define(is_octal(S), S >= $0 andalso S =< $7).
--define(is_upcase(S), S >= $A andalso S =< $Z).
--define(is_downcase(S), S >= $a andalso S =< $z).
--define(is_word(S), ?is_digit(S) orelse ?is_upcase(S) orelse ?is_downcase(S)).
--define(is_quote(S), S == $" orelse S == $').
--define(is_space(S), S == $\s; S == $\r; S == $\t; S == $\n).
-
 -define(container2(T1, T2),
   T1 == ${, T2 == $};
   T1 == $[, T2 == $]
@@ -58,6 +48,7 @@
   T1 == $<, T2 == $-;
   T1 == $-, T2 == $>;
   T1 == $., T2 == $.;
+  T1 == $|, T2 == $>;
   T1 == $/, T2 == $>;
   T1 == $=, T2 == $~
 ).
@@ -116,7 +107,7 @@ tokenize([], _Line, #scope{terminators=[]}, Tokens) ->
 
 tokenize([], EndLine, #scope{terminators=[{ Start, StartLine }|_]}, _Tokens) ->
   End     = terminator(Start),
-  Message = io_lib:format("missing terminator: ~s (for \"~s\" starting at line ~B)", [End, Start, StartLine]),
+  Message = io_lib:format("missing terminator: ~ts (for \"~ts\" starting at line ~B)", [End, Start, StartLine]),
   { error, { EndLine, Message, [] } };
 
 % Base integers
@@ -125,14 +116,14 @@ tokenize([$0,X,H|T], Line, Scope, Tokens) when (X == $x orelse X == $X), ?is_hex
   { Rest, Number } = tokenize_hex([H|T], []),
   tokenize(Rest, Line, Scope, [{ number, Line, Number }|Tokens]);
 
-tokenize([$0,O,H|T], Line, Scope, Tokens) when (O == $o orelse O == $O), ?is_octal(H) ->
-  { Rest, Number } = tokenize_octal([H|T], []),
-  tokenize(Rest, Line, Scope, [{ number, Line, Number }|Tokens]);
-
 tokenize([$0,B,H|T], Line, Scope, Tokens) when (B == $b orelse B == $B), ?is_bin(H) ->
   { Rest, Number } = tokenize_bin([H|T], []),
   tokenize(Rest, Line, Scope, [{ number, Line, Number }|Tokens]);
 
+tokenize([$0,H|T], Line, Scope, Tokens) when ?is_octal(H) ->
+  { Rest, Number } = tokenize_octal([H|T], []),
+  tokenize(Rest, Line, Scope, [{ number, Line, Number }|Tokens]);
+
 % Comments
 
 tokenize([$#|String], Line, Scope, Tokens) ->
@@ -157,11 +148,55 @@ tokenize([$%,S,H|T], Line, #scope{file=File} = Scope, Tokens) when not(?is_word(
       tokenize(Final, NewLine, Scope, [{ sigil, Line, S, Parts, Modifiers }|Tokens]);
     Error ->
       Sigil = [$%,S,H],
-      interpolation_error(Error, " (for sigil ~s starting at line ~B)", [Sigil, Line])
+      interpolation_error(Error, " (for sigil ~ts starting at line ~B)", [Sigil, Line])
   end;
 
 % Char tokens
 
+tokenize([$?,$\\,P,${,A,B,C,D,E,F,$}|T], Line, Scope, Tokens) when (P == $x orelse P == $X), ?is_hex(A), ?is_hex(B), ?is_hex(C), ?is_hex(D), ?is_hex(E), ?is_hex(F) ->
+  [Char] = unicode:characters_to_list(elixir_interpolation:unescape_chars(list_to_binary([$\\,P,${,A,B,C,D,E,F,$}]))),
+  tokenize(T, Line, Scope, [{ number, Line, Char }|Tokens]);
+
+tokenize([$?,$\\,P,${,A,B,C,D,E,$}|T], Line, Scope, Tokens) when (P == $x orelse P == $X), ?is_hex(A), ?is_hex(B), ?is_hex(C), ?is_hex(D), ?is_hex(E) ->
+  [Char] = unicode:characters_to_list(elixir_interpolation:unescape_chars(list_to_binary([$\\,P,${,A,B,C,D,E,$}]))),
+  tokenize(T, Line, Scope, [{ number, Line, Char }|Tokens]);
+
+tokenize([$?,$\\,P,${,A,B,C,D,$}|T], Line, Scope, Tokens) when (P == $x orelse P == $X), ?is_hex(A), ?is_hex(B), ?is_hex(C), ?is_hex(D) ->
+  [Char] = unicode:characters_to_list(elixir_interpolation:unescape_chars(list_to_binary([$\\,P,${,A,B,C,D,$}]))),
+  tokenize(T, Line, Scope, [{ number, Line, Char }|Tokens]);
+
+tokenize([$?,$\\,P,${,A,B,C,$}|T], Line, Scope, Tokens) when (P == $x orelse P == $X), ?is_hex(A), ?is_hex(B), ?is_hex(C) ->
+  [Char] = unicode:characters_to_list(elixir_interpolation:unescape_chars(list_to_binary([$\\,P,${,A,B,C,$}]))),
+  tokenize(T, Line, Scope, [{ number, Line, Char }|Tokens]);
+
+tokenize([$?,$\\,P,${,A,B,$}|T], Line, Scope, Tokens) when (P == $x orelse P == $X), ?is_hex(A), ?is_hex(B) ->
+  [Char] = unicode:characters_to_list(elixir_interpolation:unescape_chars(list_to_binary([$\\,P,${,A,B,$}]))),
+  tokenize(T, Line, Scope, [{ number, Line, Char }|Tokens]);
+
+tokenize([$?,$\\,P,${,A,$}|T], Line, Scope, Tokens) when (P == $x orelse P == $X), ?is_hex(A) ->
+  [Char] = unicode:characters_to_list(elixir_interpolation:unescape_chars(list_to_binary([$\\,P,${,A,$}]))),
+  tokenize(T, Line, Scope, [{ number, Line, Char }|Tokens]);
+
+tokenize([$?,$\\,P,A,B|T], Line, Scope, Tokens) when (P == $x orelse P == $X), ?is_hex(A), ?is_hex(B) ->
+  [Char] = unicode:characters_to_list(elixir_interpolation:unescape_chars(list_to_binary([$\\,P,A,B]))),
+  tokenize(T, Line, Scope, [{ number, Line, Char }|Tokens]);
+
+tokenize([$?,$\\,P,A|T], Line, Scope, Tokens) when (P == $x orelse P == $X), ?is_hex(A) ->
+  [Char] = unicode:characters_to_list(elixir_interpolation:unescape_chars(list_to_binary([$\\,P,A]))),
+  tokenize(T, Line, Scope, [{ number, Line, Char }|Tokens]);
+
+tokenize([$?,$\\,A,B,C|T], Line, Scope, Tokens) when ?is_octal(A), A =< $3,?is_octal(B), ?is_octal(C) ->
+  [Char] = unicode:characters_to_list(elixir_interpolation:unescape_chars(list_to_binary([$\\,A,B,C]))),
+  tokenize(T, Line, Scope, [{ number, Line, Char }|Tokens]);
+
+tokenize([$?,$\\,A,B|T], Line, Scope, Tokens) when ?is_octal(A), ?is_octal(B) ->
+  [Char] = unicode:characters_to_list(elixir_interpolation:unescape_chars(list_to_binary([$\\,A,B]))),
+  tokenize(T, Line, Scope, [{ number, Line, Char }|Tokens]);
+
+tokenize([$?,$\\,A|T], Line, Scope, Tokens) when ?is_octal(A) ->
+  [Char] = unicode:characters_to_list(elixir_interpolation:unescape_chars(list_to_binary([$\\,A]))),
+  tokenize(T, Line, Scope, [{ number, Line, Char }|Tokens]);
+
 tokenize([$?,$\\,H|T], Line, Scope, Tokens) ->
   Char = elixir_interpolation:unescape_map(H),
   tokenize(T, Line, Scope, [{ number, Line, Char }|Tokens]);
@@ -169,15 +204,21 @@ tokenize([$?,$\\,H|T], Line, Scope, Tokens) ->
 tokenize([$?,Char|T], Line, Scope, Tokens) ->
   tokenize(T, Line, Scope, [{ number, Line, Char }|Tokens]);
 
-% Dot identifier
+% Dot identifier/operators
 
 tokenize("..." ++ Rest, Line, Scope, Tokens) ->
   Token = tokenize_call_identifier(identifier, Line, '...', Rest),
   tokenize(Rest, Line, Scope, [Token|Tokens]);
 
-% Dot operators
-
 % ## Containers
+
+tokenize([$.,T|Tail], Line, Scope, Tokens) when ?is_space(T) ->
+  case [T|Tail] of
+    [$\r,$\n|Rest] -> tokenize([$.|Rest], Line + 1, Scope, Tokens);
+    [$\n|Rest]     -> tokenize([$.|Rest], Line + 1, Scope, Tokens);
+    [_|Rest]       -> tokenize([$.|Rest], Line, Scope, Tokens)
+  end;
+
 tokenize(".<<>>" ++ Rest, Line, Scope, Tokens) ->
   handle_call_identifier(Rest, Line, '<<>>', Scope, Tokens);
 
@@ -237,7 +278,7 @@ tokenize([$:,H|T], Line, #scope{file=File} = Scope, Tokens) when ?is_quote(H) ->
   case elixir_interpolation:extract(Line, File, true, T, H) of
     { NewLine, Parts, Rest } ->
       Token = case unescape_tokens(Parts) of
-        [Part]    -> { atom, Line, unsafe_to_atom(Part, Scope) };
+        [Part] when is_list(Part) or is_binary(Part) -> { atom, Line, unsafe_to_atom(Part, Scope) };
         Unescaped -> { atom, Line, Unescaped }
       end,
       tokenize(Rest, NewLine, Scope, [Token|Tokens]);
@@ -245,7 +286,10 @@ tokenize([$:,H|T], Line, #scope{file=File} = Scope, Tokens) when ?is_quote(H) ->
       interpolation_error(Error, " (for atom starting at line ~B)", [Line])
   end;
 
-% Atom operators
+% Atom identifiers/operators
+
+tokenize(":..." ++ Rest, Line, Scope, Tokens) ->
+  tokenize(Rest, Line, Scope, [{ atom, Line, '...' }|Tokens]);
 
 % ## Containers
 tokenize(":<<>>" ++ Rest, Line, Scope, Tokens) ->
@@ -269,10 +313,10 @@ tokenize([$:,T|Rest], Line, Scope, Tokens) when ?comp1(T); ?op1(T); T == $&; T =
 % End of line
 
 tokenize(";" ++ Rest, Line, Scope, []) ->
-  tokenize(Rest, Line, Scope, eol(Line, $;, []));
+  tokenize(Rest, Line, Scope, eol(Line, ';', []));
 
 tokenize(";" ++ Rest, Line, Scope, [Top|Tokens]) when element(1, Top) /= eol ->
-  tokenize(Rest, Line, Scope, eol(Line, $;, [Top|Tokens]));
+  tokenize(Rest, Line, Scope, eol(Line, ';', [Top|Tokens]));
 
 tokenize("\\\n" ++ Rest, Line, Scope, Tokens) ->
   tokenize(Rest, Line + 1, Scope, Tokens);
@@ -281,10 +325,10 @@ tokenize("\\\r\n" ++ Rest, Line, Scope, Tokens) ->
   tokenize(Rest, Line + 1, Scope, Tokens);
 
 tokenize("\n" ++ Rest, Line, Scope, Tokens) ->
-  tokenize(Rest, Line + 1, Scope, eol(Line, $\n, Tokens));
+  tokenize(Rest, Line + 1, Scope, eol(Line, newline, Tokens));
 
 tokenize("\r\n" ++ Rest, Line, Scope, Tokens) ->
-  tokenize(Rest, Line + 1, Scope, eol(Line, $\n, Tokens));
+  tokenize(Rest, Line + 1, Scope, eol(Line, newline, Tokens));
 
 % Stand-alone tokens
 
@@ -350,7 +394,7 @@ tokenize([H|_] = String, Line, Scope, Tokens) when ?is_upcase(H) ->
     [$:|T] when hd(T) /= $: ->
       tokenize(T, Line, Scope, [{ kw_identifier, Line, Atom }|Tokens]);
     _ ->
-      tokenize(Rest, Line, Scope, [{ '__aliases__', Line, [Atom] }|Tokens])
+      tokenize(Rest, Line, Scope, [{ aliases, Line, [Atom] }|Tokens])
   end;
 
 % Identifier
@@ -358,13 +402,15 @@ tokenize([H|_] = String, Line, Scope, Tokens) when ?is_upcase(H) ->
 tokenize([H|_] = String, Line, Scope, Tokens) when ?is_downcase(H); H == $_ ->
   { Rest, { Kind, _, Identifier } } = tokenize_any_identifier(String, Line, [], Scope),
   case handle_keyword(Line, Kind, Identifier, Tokens) of
-    false ->
+    nomatch ->
       tokenize(Rest, Line, Scope, [{ Kind, Line, Identifier }|Tokens]);
-    [Check|T] ->
+    { ok, [Check|T] } ->
       case handle_terminator(Check, Scope) of
         { error, _ } = Error -> Error;
         New -> tokenize(Rest, Line, New, [Check|T])
-      end
+      end;
+    { error, Token } ->
+      { error, { Line, "syntax error before: ", Token } }
   end;
 
 % Ambiguous unary/binary operators tokens
@@ -414,7 +460,7 @@ handle_strings(T, Line, H, #scope{file=File} = Scope, Tokens) ->
           Atom = unsafe_to_atom(unescape_chars(Bin), Scope),
           tokenize(Rest, NewLine, Scope, [{ kw_identifier, Line, Atom }|Tokens]);
         _ ->
-          { error, { Line, "invalid interpolation in key", [$"|T] } }
+          { error, { Line, "invalid interpolation in key ", [$"|T] } }
       end;
     { NewLine, Parts, Rest } ->
       Token = { string_type(H),Line,unescape_tokens(Parts) },
@@ -496,7 +542,7 @@ extract_heredoc(Line0, Rest0, Marker) ->
 
 heredoc_error(ErrorLine, StartLine, Marker) ->
   Terminator = [Marker, Marker, Marker],
-  Message    = io_lib:format("missing terminator: ~s (for heredoc starting at line ~B)", [Terminator, StartLine]),
+  Message    = io_lib:format("missing terminator: ~ts (for heredoc starting at line ~B)", [Terminator, StartLine]),
   { error, { ErrorLine, Message, [] } }.
 
 %% Remove spaces from heredoc based on the position of the final quotes.
@@ -687,7 +733,7 @@ tokenize_call_identifier(Kind, Line, Atom, Rest) ->
 
 verify_kw_and_space(_Line, _Atom, [H|_], #scope{}) when ?is_space(H) -> ok;
 verify_kw_and_space(Line, Atom, _, #scope{file=File}) ->
-  io:format("~ts:~w: keyword argument ~s: must be followed by space~n", [File, Line, Atom]).
+  io:format("~ts:~w: keyword argument ~ts: must be followed by space~n", [File, Line, Atom]).
 
 next_is_block([Space|Tokens]) when Space == $\t; Space == $\s ->
   next_is_block(Tokens);
@@ -702,13 +748,13 @@ next_is_block([$d,$o|_]) ->
 next_is_block(_) ->
   false.
 
-add_token_with_nl(Left, [{eol,_,$\n}|T]) -> [Left|T];
+add_token_with_nl(Left, [{eol,_,newline}|T]) -> [Left|T];
 add_token_with_nl(Left, T) -> [Left|T].
 
 % Error handling
 
 interpolation_error({ error, { Line, Message, Token } }, Extension, Args) ->
-  { error, { Line, io_lib:format("~s" ++ Extension, [Message|Args]), Token } }.
+  { error, { Line, io_lib:format("~ts" ++ Extension, [Message|Args]), Token } }.
 
 % Terminators
 
@@ -729,7 +775,7 @@ handle_terminator(Token, #scope{terminators=Terminators} = Scope) ->
     New -> Scope#scope{terminators=New}
   end.
 
-check_terminator({ S, Line }, Terminators) when S == 'fn'; S == 'fn_paren' ->
+check_terminator({ S, Line }, Terminators) when S == 'fn' ->
   [{ fn, Line }|Terminators];
 
 check_terminator({ S, _ } = New, Terminators) when
@@ -752,7 +798,7 @@ check_terminator({ E, _ }, [{ S, _ }|Terminators]) when
 check_terminator({ E, Line }, [{ Start, StartLine }|_]) when
     E == 'end'; E == ')'; E == ']'; E == '}'; E == '>>' ->
   End     = terminator(Start),
-  Message = io_lib:format("missing terminator: ~s (for \"~s\" starting at line ~B)", [End, Start, StartLine]),
+  Message = io_lib:format("missing terminator: ~ts (for \"~ts\" starting at line ~B)", [End, Start, StartLine]),
   { error, { Line, Message, [] } };
 
 check_terminator({ E, Line }, []) when
@@ -770,34 +816,41 @@ terminator('{')  -> '}';
 terminator('<<') -> '>>'.
 
 % Keywords check
-handle_keyword(Line, paren_identifier, fn, Tokens) ->
-  [{ 'fn_paren', Line }|Tokens];
+handle_keyword(Line, Identifier, Atom, [{ '.', _ }|_] = Tokens) ->
+  { ok, [{ Identifier, Line, Atom }|Tokens] };
 
-handle_keyword(Line, Identifier, Atom, Tokens) when Identifier ==  bracket_identifier; Identifier == paren_identifier ->
+handle_keyword(Line, Identifier, Atom, Tokens) when
+    Identifier ==  identifier; Identifier == do_identifier;
+    Identifier ==  bracket_identifier; Identifier == paren_identifier ->
   case keyword(Atom) of
-    true -> [{ Atom, Line }|Tokens];
-    _    -> [{ Identifier, Line, Atom }|Tokens]
+    true  -> { ok, [{ Atom, Line }|Tokens] };
+    op    -> { ok, add_token_with_nl({ Atom, Line }, Tokens) };
+    block -> { ok, [{ block_identifier, Line, Atom }|Tokens] };
+    do    ->
+      case do_keyword_valid(Tokens) of
+        true  -> { ok, [{ Atom, Line }|Tokens] };
+        false -> { error, "do" }
+      end;
+    false -> nomatch
   end;
 
-handle_keyword(Line, Identifier, Atom, Tokens) when Identifier ==  identifier; Identifier == do_identifier ->
-  case keyword(Atom) of
-    true  -> [{ Atom, Line }|Tokens];
-    op    -> add_token_with_nl({ Atom, Line }, Tokens);
-    block -> [{ block_identifier, Line, Atom }|Tokens];
-    false -> false
-  end;
+handle_keyword(_, _, _, _) -> nomatch.
 
-handle_keyword(_, _, _, _) -> false.
+do_keyword_valid([{ Atom, _ }|_]) ->
+  is_boolean(keyword(Atom));
+do_keyword_valid(_) -> true.
 
 % Keywords
 keyword('fn')    -> true;
-keyword('do')    -> true;
 keyword('end')   -> true;
 keyword('true')  -> true;
 keyword('false') -> true;
 keyword('nil')   -> true;
 keyword('not')   -> true;
 
+% Special handling for do
+keyword('do')    -> do;
+
 % Bin operator keywords
 keyword('and')    -> op;
 keyword('or')     -> op;
@@ -813,4 +866,4 @@ keyword('else')   -> block;
 keyword('rescue') -> block;
 keyword('catch')  -> block;
 
-keyword(_) -> false.
+keyword(_) -> false.

lib/elixir/src/elixir_translator.erl

@@ -7,7 +7,7 @@
 -import(elixir_scope, [umergev/2, umergec/2]).
 -import(elixir_errors, [syntax_error/3, syntax_error/4, parse_error/4,
   assert_function_scope/3, assert_module_scope/3, assert_no_guard_scope/3,
-  assert_no_assign_or_guard_scope/3]).
+  assert_no_match_or_guard_scope/3]).
 -include("elixir.hrl").
 -compile({parse_transform, elixir_transform}).
 
@@ -35,14 +35,17 @@ translate(Forms, S) ->
 %% Those macros are "low-level". They are the basic mechanism
 %% that makes the language work and cannot be partially applied
 %% nor overwritten.
+%%
+%% =, ^, import, require and alias could be made non-special
+%% forms without causing any side effects.
 
 %% Assignment operator
 
-translate_each({'=', Line, [Left, Right]}, S) ->
-  assert_no_guard_scope(Line, '=', S),
+translate_each({'=', Meta, [Left, Right]}, S) ->
+  assert_no_guard_scope(Meta, '=', S),
   { TRight, SR } = translate_each(Right, S),
   { TLeft, SL } = elixir_clauses:assigns(fun translate_each/2, Left, SR),
-  { { match, Line, TLeft, TRight }, SL };
+  { { match, ?line(Meta), TLeft, TRight }, SL };
 
 %% Containers
 
@@ -51,51 +54,51 @@ translate_each({ C, _, _ } = Original, S) when C == '[]'; C == '{}'; C == '<<>>'
 
 %% Blocks and scope rewriters
 
-translate_each({ '__block__', Line, [] }, S) ->
-  { { atom, Line, nil }, S };
+translate_each({ '__block__', Meta, [] }, S) ->
+  { { atom, ?line(Meta), nil }, S };
 
 translate_each({ '__block__', _Line, [Arg] }, S) ->
   translate_each(Arg, S);
 
-translate_each({ '__block__', Line, Args }, S) when is_list(Args) ->
+translate_each({ '__block__', Meta, Args }, S) when is_list(Args) ->
   { TArgs, NS } = translate(Args, S),
-  { { block, Line, TArgs }, NS };
+  { { block, ?line(Meta), TArgs }, NS };
 
-translate_each({ '__scope__', _Line, [[{file,File}],Expr] }, S) ->
+translate_each({ '__scope__', _Meta, [[{file,File}],[{do,Expr}]] }, S) ->
   Old = S#elixir_scope.file,
   { TExpr, TS } = translate_each(Expr, S#elixir_scope{file=File}),
   { TExpr, TS#elixir_scope{file=Old} };
 
 %% Erlang op
 
-translate_each({ '__op__', Line, [Op, Expr] }, S) when is_atom(Op) ->
+translate_each({ '__op__', Meta, [Op, Expr] }, S) when is_atom(Op) ->
   { TExpr, NS } = translate_each(Expr, S),
-  { { op, Line, convert_op(Op), TExpr }, NS };
+  { { op, ?line(Meta), convert_op(Op), TExpr }, NS };
 
-translate_each({ '__op__', Line, [Op, Left, Right] }, S) when is_atom(Op) ->
+translate_each({ '__op__', Meta, [Op, Left, Right] }, S) when is_atom(Op) ->
   { [TLeft, TRight], NS }  = translate_args([Left, Right], S),
-  { { op, Line, convert_op(Op), TLeft, TRight }, NS };
+  { { op, ?line(Meta), convert_op(Op), TLeft, TRight }, NS };
 
-translate_each({ '__ambiguousop__', Line, [Var, H|T] }, S) ->
-  { Name, _, _ } = Var,
+translate_each({ '__ambiguousop__', Meta, [Var, H|T] }, S) ->
+  { Name, _, Kind } = Var,
 
-  case orddict:find(Name, S#elixir_scope.vars) of
-    error -> translate_each({ Name, Line, [H|T] }, S);
+  case orddict:find({ Name, Kind }, S#elixir_scope.vars) of
+    error -> translate_each({ Name, Meta, [H|T] }, S);
     _ ->
       case T of
         [] -> translate_each(rellocate_ambiguous_op(H, Var), S);
-        _  -> syntax_error(Line, S#elixir_scope.file, "Many arguments given to ~s, but ~s is a variable. Use even spaces to solve ambiguity.", [Name, Name])
+        _  -> syntax_error(Meta, S#elixir_scope.file, "Many arguments given to ~s, but ~s is a variable. Use even spaces to solve ambiguity.", [Name, Name])
       end
   end;
 
 %% Lexical
 
-translate_each({ alias, Line, [Ref] }, S) ->
-  translate_each({ alias, Line, [Ref,[]] }, S);
+translate_each({ alias, Meta, [Ref] }, S) ->
+  translate_each({ alias, Meta, [Ref,[]] }, S);
 
-translate_each({ alias, Line, [Ref, KV] }, S) ->
-  assert_no_assign_or_guard_scope(Line, alias, S),
-  validate_opts(Line, alias, [as], KV, S),
+translate_each({ alias, Meta, [Ref, KV] }, S) ->
+  assert_no_match_or_guard_scope(Meta, alias, S),
+  validate_opts(Meta, alias, [as], KV, S),
 
   { TRef, SR } = translate_each(Ref, S),
 
@@ -103,14 +106,15 @@ translate_each({ alias, Line, [Ref, KV] }, S) ->
     { atom, _, Old } ->
       { New, SF } = case lists:keyfind(as, 1, KV) of
         Opt when Opt == { as, true }; Opt == false ->
-          { elixir_aliases:last(Old), SR };        
+          { elixir_aliases:last(Old), SR };
         { as, false } ->
           { Old, SR };
         { as, Other } ->
-          { TOther, SA } = translate_each(Other, SR#elixir_scope{aliases=[]}),
+          { TOther, SA } = translate_each(no_alias_expansion(Other),
+            SR#elixir_scope{aliases=[],macro_aliases=[]}),
           case TOther of
             { atom, _, Atom } -> { Atom, SA };
-            _ -> syntax_error(Line, S#elixir_scope.file,
+            _ -> syntax_error(Meta, S#elixir_scope.file,
                    "invalid args for alias, expected an atom or alias as argument")
           end
       end,
@@ -118,28 +122,29 @@ translate_each({ alias, Line, [Ref, KV] }, S) ->
       %% Avoid creating aliases if first == last
       %% unecessarily polluting the aliases dict
       case New == Old of
-        true  -> { { nil, Line }, SF };
+        true  -> { { nil, ?line(Meta) }, SF };
         false ->
           case string:tokens(atom_to_list(New), "-") of
             [_,_] -> [];
-            _ -> syntax_error(Line, S#elixir_scope.file,
+            _ -> syntax_error(Meta, S#elixir_scope.file,
                    "invalid args for alias, cannot create nested alias ~s", [elixir_errors:inspect(New)])
           end,
 
-          { { nil, Line }, SF#elixir_scope{
-            aliases=orddict:store(New, Old, S#elixir_scope.aliases)
+          { { nil, ?line(Meta) }, SF#elixir_scope{
+            aliases=orddict:store(New, Old, S#elixir_scope.aliases),
+            macro_aliases=orddict:store(New, Old, S#elixir_scope.macro_aliases)
           } }
       end;
     _ ->
-      syntax_error(Line, S#elixir_scope.file, "invalid args for alias, expected an atom or alias as argument")
+      syntax_error(Meta, S#elixir_scope.file, "invalid args for alias, expected an atom or alias as argument")
   end;
 
-translate_each({ require, Line, [Ref] }, S) ->
-  translate_each({ require, Line, [Ref, []] }, S);
+translate_each({ require, Meta, [Ref] }, S) ->
+  translate_each({ require, Meta, [Ref, []] }, S);
 
-translate_each({ require, Line, [Ref, KV] }, S) ->
-  assert_no_assign_or_guard_scope(Line, 'require', S),
-  validate_opts(Line, require, [as], KV, S),
+translate_each({ require, Meta, [Ref, KV] }, S) ->
+  assert_no_match_or_guard_scope(Meta, 'require', S),
+  validate_opts(Meta, require, [as], KV, S),
 
   { TRef, SR } = translate_each(Ref, S),
 
@@ -150,243 +155,296 @@ translate_each({ require, Line, [Ref, KV] }, S) ->
 
   case TRef of
     { atom, _, Old } ->
-      elixir_aliases:ensure_loaded(Line, Old, SR),
+      elixir_aliases:ensure_loaded(Meta, Old, SR),
 
       SF = SR#elixir_scope{
         requires=ordsets:add_element(Old, S#elixir_scope.requires)
       },
 
-      translate_each({ alias, Line, [Ref, [{ as, As }]] }, SF);
+      translate_each({ alias, Meta, [Ref, [{ as, As }]] }, SF);
     _ ->
-      syntax_error(Line, S#elixir_scope.file, "invalid args for require, expected an atom or alias as argument")
+      syntax_error(Meta, S#elixir_scope.file, "invalid args for require, expected an atom or alias as argument")
   end;
 
-translate_each({ import, Line, [Left] }, S) ->
-  translate_each({ import, Line, [Left, []]}, S);
+translate_each({ import, Meta, [Left] }, S) ->
+  translate_each({ import, Meta, [Left, []]}, S);
 
-translate_each({ import, Line, [Left,Opts] }, S) when is_list(Opts) ->
-  translate_each({ import, Line, [default, Left, Opts]}, S);
+translate_each({ import, Meta, [Left,Opts] }, S) when is_list(Opts) ->
+  translate_each({ import, Meta, [default, Left, Opts]}, S);
 
-translate_each({ import, Line, [Selector, Left] }, S) ->
-  translate_each({ import, Line, [Selector, Left, []]}, S);
+translate_each({ import, Meta, [Selector, Left] }, S) ->
+  translate_each({ import, Meta, [Selector, Left, []]}, S);
 
-translate_each({ import, Line, [Left, Right, Opts] }, S) ->
-  assert_no_assign_or_guard_scope(Line, 'import', S),
+translate_each({ import, Meta, [Left, Right, Opts] }, S) ->
+  assert_no_match_or_guard_scope(Meta, 'import', S),
   { TSelector, SL } = translate_each(Left, S),
   { TRef, SR } = translate_each(Right, SL),
 
   Selector = case TSelector of
     { atom, _,  SelectorAtom } -> SelectorAtom;
-    _ -> syntax_error(Line, S#elixir_scope.file, "invalid selector for import, expected an atom")
+    _ -> syntax_error(Meta, S#elixir_scope.file, "invalid selector for import, expected an atom")
   end,
 
   Ref = case TRef of
     { atom, _, RefAtom } -> RefAtom;
-    _ -> syntax_error(Line, S#elixir_scope.file, "invalid name for import, expected an atom or alias")
+    _ -> syntax_error(Meta, S#elixir_scope.file, "invalid name for import, expected an atom or alias")
   end,
 
-  validate_opts(Line, import, [as, only, except], Opts, S),
+  validate_opts(Meta, import, [as, only, except], Opts, S),
 
   As = case lists:keyfind(as, 1, Opts) of
     false -> false;
     { as, Value } -> Value
   end,
 
-  elixir_aliases:ensure_loaded(Line, Ref, SR),
-  SF = elixir_import:import(Line, Ref, Opts, Selector, SR),
-  translate_each({ require, Line, [Ref, [{ as, As }]] }, SF);
+  elixir_aliases:ensure_loaded(Meta, Ref, SR),
+  SF = elixir_import:import(Meta, Ref, Opts, Selector, SR),
+  translate_each({ require, Meta, [Ref, [{ as, As }]] }, SF);
 
 %% Pseudo variables
 
-translate_each({ '__MODULE__', Line, Atom }, S) when is_atom(Atom) ->
-  { { atom, Line, S#elixir_scope.module }, S };
+translate_each({ '__MODULE__', Meta, Atom }, S) when is_atom(Atom) ->
+  { { atom, ?line(Meta), S#elixir_scope.module }, S };
 
-translate_each({ '__FILE__', _Line, Atom }, S) when is_atom(Atom) ->
+translate_each({ '__FILE__', _Meta, Atom }, S) when is_atom(Atom) ->
   translate_each(S#elixir_scope.file, S);
 
-translate_each({ '__ENV__', Line, Atom }, S) when is_atom(Atom) ->
-  { elixir_scope:to_erl_env({ Line, S }), S };
+translate_each({ '__ENV__', Meta, Atom }, S) when is_atom(Atom) ->
+  Env = elixir_scope:to_ex_env({ ?line(Meta), S }),
+  { elixir_tree_helpers:abstract_syntax(Env), S };
 
-translate_each({ '__CALLER__', Line, Atom }, S) when is_atom(Atom) ->
-  { { var, Line, '__CALLER__' }, S#elixir_scope{caller=true} };
+translate_each({ '__CALLER__', Meta, Atom }, S) when is_atom(Atom) ->
+  { { var, ?line(Meta), '__CALLER__' }, S#elixir_scope{caller=true} };
 
 %% Aliases
 
-translate_each({ '__aliases__', Line, [H] }, S) when H /= 'Elixir'  ->
-  Atom = list_to_atom("Elixir-" ++ atom_to_list(H)),
-  { { atom, Line, elixir_aliases:lookup(Atom, S#elixir_scope.aliases) }, S };
+translate_each({ '__aliases__', Meta, _ } = Alias, S) ->
+  case elixir_aliases:expand(Alias, S#elixir_scope.aliases, S#elixir_scope.macro_aliases) of
+    Atom when is_atom(Atom) -> { { atom, ?line(Meta), Atom }, S };
+    Aliases ->
+      { TAliases, SA } = translate_args(Aliases, S),
 
-translate_each({ '__aliases__', Line, [H|T] }, S) ->
-  Aliases = if
-    is_atom(H) andalso (H /= 'Elixir') ->
-      Atom = list_to_atom("Elixir-" ++ atom_to_list(H)),
-      [elixir_aliases:lookup(Atom, S#elixir_scope.aliases)|T];
-    true ->
-      [H|T]
-  end,
-
-  { TAliases, SA } = translate_args(Aliases, S),
-
-  case lists:all(fun is_atom_tuple/1, TAliases) of
-    true ->
-      Atoms = [Atom || { atom, _, Atom } <- TAliases],
-      { { atom, Line, elixir_aliases:concat(Atoms) }, SA };
-    false ->
-      Args = [elixir_tree_helpers:build_simple_list(Line, TAliases)],
-      { ?ELIXIR_WRAP_CALL(Line, elixir_aliases, concat, Args), SA }
+      case lists:all(fun is_atom_tuple/1, TAliases) of
+        true ->
+          Atoms = [Atom || { atom, _, Atom } <- TAliases],
+          { { atom, ?line(Meta), elixir_aliases:concat(Atoms) }, SA };
+        false ->
+          Args = [elixir_tree_helpers:build_simple_list(?line(Meta), TAliases)],
+          { ?wrap_call(?line(Meta), elixir_aliases, concat, Args), SA }
+      end
   end;
 
 %% Quoting
 
-translate_each({ Unquote, Line, _Args }, S) when Unquote == unquote; Unquote == unquote_splicing ->
-  syntax_error(Line, S#elixir_scope.file, "~p called outside quote", [Unquote]);
+translate_each({ Unquote, Meta, [_|_] }, S) when Unquote == unquote; Unquote == unquote_splicing ->
+  syntax_error(Meta, S#elixir_scope.file, "~p called outside quote", [Unquote]);
 
-translate_each({ quote, Line, [Left, Right] }, S) ->
-  translate_each({ quote, Line, [orddict:from_list(Left ++ Right)] }, S);
+translate_each({ quote, Meta, [Left, Right] }, S) ->
+  translate_each({ quote, Meta, [orddict:from_list(Left ++ Right)] }, S);
 
-translate_each({ quote, GivenLine, [T] }, S) when is_list(T) ->
+translate_each({ quote, Meta, [T] }, S) when is_list(T) ->
   Exprs =
-  case lists:keyfind(do, 1, T) of
-    {do, E} -> E;
+    case lists:keyfind(do, 1, T) of
+      { do, E } -> E;
+      false ->
+        syntax_error(Meta, S#elixir_scope.file, "invalid args for quote")
+    end,
+
+  Hygiene = case lists:keyfind(hygiene, 1, T) of
+    { hygiene, true } ->
+      elixir_errors:deprecation(Meta, S#elixir_scope.file, "hygiene: true is deprecated, please set it to [] instead", []),
+      [];
+    { hygiene, List } when is_list(List) ->
+      List;
+    { hygiene, false } ->
+      elixir_errors:deprecation(Meta, S#elixir_scope.file, "hygiene: false is deprecated, please set it to [vars: false] instead", []),
+      [{vars,false}, {imports,false}, {aliases,false}];
     false ->
-      syntax_error(GivenLine, S#elixir_scope.file, "invalid args for quote")
+      []
   end,
 
-  Marker = case lists:keyfind(hygiene, 1, T) of
-    { hygiene, false } -> nil;
-    _ -> quoted
+  Vars = case lists:keyfind(var_context, 1, T) of
+    { var_context, VarContext } ->
+      expand_var_context(Meta, VarContext, "invalid argument given for var_context in quote", S);
+    false ->
+      case lists:keyfind(vars, 1, Hygiene) of
+        { vars, false } -> nil;
+        _ ->
+          case S#elixir_scope.module of
+            nil -> 'Elixir';
+            Mod -> Mod
+          end
+      end
   end,
 
-  { DefaultLine, WrappedExprs } = case lists:keyfind(location, 1, T) of
-    { location, keep } ->
-      Scoped = { '__scope__', GivenLine, [[{file,S#elixir_scope.file}],Exprs] },
-      { keep, Scoped };
-    _ ->
-      { 0, Exprs}
+  GetExpansion = fun(Key, New) ->
+    case lists:keyfind(Key, 1, T) of
+      { Key, Bool } when is_boolean(Bool) ->
+        elixir_errors:deprecation(Meta, S#elixir_scope.file, "the ~s option for quote is deprecated, please use hygiene: [~s: ~s] instead", [Key, New, Bool]),
+        Bool;
+      false ->
+        not(lists:keyfind(New, 1, Hygiene) == { New, false })
+    end
+  end,
+
+  Aliases = GetExpansion(aliases_hygiene, aliases),
+  Imports = GetExpansion(imports_hygiene, imports),
+
+  { DefaultLine, DefaultFile } = case lists:keyfind(location, 1, T) of
+    { location, keep }  -> { keep, keep };
+    false -> { nil, nil }
   end,
 
   Line = case lists:keyfind(line, 1, T) of
-    { line, Value } -> Value;
-    _ -> DefaultLine
+    { line, LineValue } -> LineValue;
+    false -> DefaultLine
   end,
 
   { TLine, SL } = case Line of
     keep -> { keep, S };
+    nil  -> { nil, S };
     _    -> translate_each(Line, S)
   end,
 
+  File = case lists:keyfind(file, 1, T) of
+    { file, FileValue } -> FileValue;
+    false -> DefaultFile
+  end,
+
+  TFile = case File of
+    keep -> S#elixir_scope.file;
+    _    -> File
+  end,
+
+  TExprs = if
+    is_binary(TFile) ->
+      { '__scope__', Meta, [[{file,TFile}],[{do,Exprs}]] };
+    File == nil ->
+      Exprs;
+    true ->
+      syntax_error(Meta, S#elixir_scope.file, "invalid args for quote, expected :file to be a binary")
+  end,
+
   Unquote = case lists:keyfind(unquote, 1, T) of
     { unquote, false } -> false;
     _ -> true
   end,
 
-  elixir_quote:quote(WrappedExprs, #elixir_quote{marker=Marker, line=TLine, unquote=Unquote}, SL);
+  Q = #elixir_quote{vars_hygiene=Vars, line=TLine, unquote=Unquote,
+        aliases_hygiene=Aliases, imports_hygiene=Imports},
+
+  elixir_quote:quote(TExprs, Q, SL);
+
+translate_each({ quote, Meta, [_] }, S) ->
+  syntax_error(Meta, S#elixir_scope.file, "invalid args for quote");
+
+translate_each({ 'alias!', _Meta, [Arg] }, S) ->
+  translate_each(Arg, S);
+
+translate_each({ 'var!', Meta, [Arg] }, S) ->
+  translate_each({ 'var!', Meta, [Arg, nil] }, S);
+
+translate_each({ 'var!', Meta, [{Name, _, Atom}, Kind] }, S) when is_atom(Name), is_atom(Atom) ->
+  translate_each({ 'var!', Meta, [Name, Kind] }, S);
+
+translate_each({ 'var!', Meta, [Name, Kind] }, S) when is_atom(Name) ->
+  Expanded = expand_var_context(Meta, Kind, "invalid second argument for var!", S),
+  elixir_scope:translate_var(Meta, Name, Expanded, S, fun() ->
+    syntax_error(Meta, S#elixir_scope.file, "expected var!(~ts) to expand to an existing variable or be a part of a match", [Name])
+  end);
+
+translate_each({ 'var!', Meta, [_, _] }, S) ->
+  syntax_error(Meta, S#elixir_scope.file, "invalid first argument for var!, expected a var or an atom");
 
-translate_each({ quote, GivenLine, [_] }, S) ->
-  syntax_error(GivenLine, S#elixir_scope.file, "invalid args for quote");
-  
 %% Functions
 
-translate_each({ fn, Line, [[{do, { '->', _, Pairs }}]] }, S) ->
-  assert_no_assign_or_guard_scope(Line, 'fn', S),
-  translate_fn(Line, Pairs, S);
+translate_each({ fn, Meta, [[{do, { '->', _, Pairs }}]] }, S) ->
+  assert_no_match_or_guard_scope(Meta, 'fn', S),
+  translate_fn(Meta, Pairs, S);
 
 %% Comprehensions
 
-translate_each({ Kind, Line, Args }, S) when is_list(Args), (Kind == lc) orelse (Kind == bc) ->
-  translate_comprehension(Line, Kind, Args, S);
+translate_each({ Kind, Meta, Args }, S) when is_list(Args), (Kind == lc) orelse (Kind == bc) ->
+  translate_comprehension(Meta, Kind, Args, S);
 
 %% Super (exceptionally supports partial application)
 
-translate_each({ super, Line, Args } = Original, S) ->
+translate_each({ super, Meta, Args } = Original, S) ->
   case elixir_partials:handle(Original, S) of
     error ->
-      assert_no_assign_or_guard_scope(Line, super, S),
-      Module = assert_module_scope(Line, super, S),
-      Function = assert_function_scope(Line, super, S),
-      elixir_def_overridable:ensure_defined(Line, Module, Function, S),
+      assert_no_match_or_guard_scope(Meta, super, S),
+      Module = assert_module_scope(Meta, super, S),
+      Function = assert_function_scope(Meta, super, S),
+      elixir_def_overridable:ensure_defined(Meta, Module, Function, S),
 
       { _, Arity } = Function,
 
       { TArgs, TS } = if
         is_atom(Args) ->
-          elixir_def_overridable:retrieve_args(Line, Arity, S);
+          elixir_def_overridable:retrieve_args(Meta, Arity, S);
         length(Args) == Arity ->
           translate_args(Args, S);
         true ->
-          syntax_error(Line, S#elixir_scope.file, "super must be called with the same number of arguments as the current function")
+          syntax_error(Meta, S#elixir_scope.file, "super must be called with the same number of arguments as the current function")
       end,
 
       Super = elixir_def_overridable:name(Module, Function),
-      { { call, Line, { atom, Line, Super }, TArgs }, TS#elixir_scope{super=true} };
+      { { call, ?line(Meta), { atom, ?line(Meta), Super }, TArgs }, TS#elixir_scope{super=true} };
     Else -> Else
   end;
 
-translate_each({ 'super?', Line, [] }, S) ->
-  Module = assert_module_scope(Line, 'super?', S),
-  Function = assert_function_scope(Line, 'super?', S),
+translate_each({ 'super?', Meta, [] }, S) ->
+  Module = assert_module_scope(Meta, 'super?', S),
+  Function = assert_function_scope(Meta, 'super?', S),
   Bool = elixir_def_overridable:is_defined(Module, Function),
-  { { atom, Line, Bool }, S };
+  { { atom, ?line(Meta), Bool }, S };
 
 %% Variables
 
-translate_each({ '^', Line, [ { Name, _, Args } ] }, S) ->
-  Dict = case Args of
-    nil    -> S#elixir_scope.vars;
-    quoted -> S#elixir_scope.quote_vars;
-    _ ->
-      syntax_error(Line, S#elixir_scope.file, "cannot use ^ with expression at ^~s, ^ must be used only with variables", [Name])
-  end,
-
-  Result = case S#elixir_scope.context of
-    assign ->
-      case orddict:find(Name, Dict) of
-        error -> "unbound variable ^~s";
-        { ok, Value } -> { {var, Line, Value}, S }
-      end;
-    _ -> "cannot access variable ^~s outside of assignment"
-  end,
-
-  case is_list(Result) of
-    true ->
-      syntax_error(Line, S#elixir_scope.file, Result, [Name]);
-    false ->
-      Result
+translate_each({ '^', Meta, [ { Name, _, Kind } ] }, #elixir_scope{context=match} = S) when is_atom(Name), is_atom(Kind) ->
+  case orddict:find({ Name, Kind }, S#elixir_scope.vars) of
+    { ok, Value } ->
+      { { var, Meta, Value }, S };
+    error ->
+      syntax_error(Meta, S#elixir_scope.file, "unbound variable ^~ts", [Name])
   end;
 
-translate_each({ Name, Line, quoted }, S) when is_atom(Name) ->
-  NewS = S#elixir_scope{vars=S#elixir_scope.quote_vars,noname=true},
-  { TVar, VS } = elixir_scope:translate_var(Line, Name, quoted, NewS),
-  { TVar, VS#elixir_scope{
-    quote_vars=VS#elixir_scope.vars,
-    noname=S#elixir_scope.noname,
-    vars=S#elixir_scope.vars
-  } };
+translate_each({ '^', Meta, [ { Name, _, Kind } ] }, S) when is_atom(Name), is_atom(Kind) ->
+  syntax_error(Meta, S#elixir_scope.file,
+    "cannot use ^~ts outside of match clauses", [Name]);
 
-translate_each({ Name, Line, nil }, S) when is_atom(Name) ->
-  elixir_scope:translate_var(Line, Name, nil, S);
+translate_each({ '^', Meta, [ Expr ] }, S) ->
+  syntax_error(Meta, S#elixir_scope.file,
+    "the unary operator ^ can only be used with variables, invalid expression ^~ts", ['Elixir.Macro':to_binary(Expr)]);
+
+translate_each({ Name, Meta, Kind }, S) when is_atom(Name), is_atom(Kind) ->
+  elixir_scope:translate_var(Meta, Name, Kind, S, fun() ->
+    translate_each({ Name, Meta, [] }, S)
+  end);
 
 %% Local calls
 
-translate_each({ Atom, Line, Args } = Original, S) when is_atom(Atom) ->
+translate_each({ Atom, Meta, Args } = Original, S) when is_atom(Atom) ->
   case elixir_partials:is_sequential(Args) andalso
-       elixir_dispatch:import_function(Line, Atom, length(Args), S) of
+       elixir_dispatch:import_function(Meta, Atom, length(Args), S) of
     false ->
       case elixir_partials:handle(Original, S) of
         error ->
           Callback = fun() ->
             case S#elixir_scope.context of
-              guard ->
+              match ->
                 Arity = length(Args),
                 File  = S#elixir_scope.file,
                 case Arity of
-                  0 -> syntax_error(Line, File, "unknown variable ~s or cannot invoke local ~s/~B inside guard", [Atom, Atom, Arity]);
-                  _ -> syntax_error(Line, File, "cannot invoke local ~s/~B inside guard", [Atom, Arity])
+                  0 -> syntax_error(Meta, File, "unknown variable ~ts or cannot invoke local ~ts/~B inside guard", [Atom, Atom, Arity]);
+                  _ -> syntax_error(Meta, File, "cannot invoke local ~ts/~B inside guard", [Atom, Arity])
                 end;
-              _ -> translate_local(Line, Atom, Args, S)
+              _ ->
+                translate_local(Meta, Atom, Args, S)
             end
           end,
-          elixir_dispatch:dispatch_import(Line, Atom, Args, S, Callback);
+
+          elixir_dispatch:dispatch_import(Meta, Atom, Args, S, Callback);
         Else  -> Else
       end;
     Else -> Else
@@ -394,30 +452,22 @@ translate_each({ Atom, Line, Args } = Original, S) when is_atom(Atom) ->
 
 %% Remote calls
 
-translate_each({ { '.', _, [Left, Right] }, Line, Args } = Original, S) when is_atom(Right) ->
+translate_each({ { '.', _, [Left, Right] }, Meta, Args } = Original, S) when is_atom(Right) ->
   { TLeft,  SL } = translate_each(Left, S),
 
   Fun = (element(1, TLeft) == atom) andalso elixir_partials:is_sequential(Args) andalso
-    elixir_dispatch:require_function(Line, element(3, TLeft), Right, length(Args), SL),
+    elixir_dispatch:require_function(Meta, element(3, TLeft), Right, length(Args), SL),
 
   case Fun of
     false ->
       case elixir_partials:handle(Original, S) of
         error ->
           { TRight, SR } = translate_each(Right, umergec(S, SL)),
-          Callback = fun() -> translate_apply(Line, TLeft, TRight, Args, S, SL, SR) end,
+          Callback = fun() -> translate_apply(Meta, TLeft, TRight, Args, S, SL, SR) end,
 
           case TLeft of
-            { atom, _, 'Elixir.Erlang' } ->
-              elixir_errors:deprecation(Line, S#elixir_scope.file, "Erlang. syntax is deprecated"),
-              case Args of
-                [] -> { { atom, Line, Right }, S };
-                _ ->
-                  Message = "invalid args for Erlang.~s expression",
-                  syntax_error(Line, S#elixir_scope.file, Message, [Right])
-              end;
             { atom, _, Receiver } ->
-              elixir_dispatch:dispatch_require(Line, Receiver, Right, Args, umergev(SL, SR), Callback);
+              elixir_dispatch:dispatch_require(Meta, Receiver, Right, Args, umergev(SL, SR), Callback);
             _ ->
               Callback()
           end;
@@ -428,24 +478,31 @@ translate_each({ { '.', _, [Left, Right] }, Line, Args } = Original, S) when is_
 
 %% Anonymous function calls
 
-translate_each({ { '.', _, [Expr] }, Line, Args } = Original, S) ->
+translate_each({ { '.', _, [Expr] }, Meta, Args } = Original, S) ->
   { TExpr, SE } = translate_each(Expr, S),
   case TExpr of
     { atom, _, Atom } ->
-      translate_each({ Atom, Line, Args }, S);
+      case atom_to_list(Atom) of
+        [H|_] when (H >= $a andalso H =< $z); H == $_ ->
+          elixir_errors:deprecation(Meta, S#elixir_scope.file, "the :~s.() syntax is deprecated, please use ~s() instead", [Atom, Atom]);
+        _ ->
+          elixir_errors:deprecation(Meta, S#elixir_scope.file, "the :~s.() syntax is deprecated", [Atom])
+      end,
+
+      translate_each({ Atom, Meta, Args }, S);
     _ ->
       case elixir_partials:handle(Original, S) of
         error ->
           { TArgs, SA } = translate_args(Args, umergec(S, SE)),
-          { {call, Line, TExpr, TArgs}, umergev(SE, SA) };
+          { {call, ?line(Meta), TExpr, TArgs}, umergev(SE, SA) };
         Else -> Else
       end
   end;
 
 %% Invalid calls
 
-translate_each({ Invalid, Line, _Args }, S) ->
-  syntax_error(Line, S#elixir_scope.file, "unexpected parenthesis after ~ts",
+translate_each({ Invalid, Meta, _Args }, S) ->
+  syntax_error(Meta, S#elixir_scope.file, "unexpected parenthesis after ~ts",
     ['Elixir.Macro':to_binary(Invalid)]);
 
 %% Literals
@@ -455,34 +512,54 @@ translate_each(Literal, S) ->
 
 %% Helpers
 
-validate_opts(Line, Kind, Allowed, Opts, S) when is_list(Opts) ->
+validate_opts(Meta, Kind, Allowed, Opts, S) when is_list(Opts) ->
   [begin
-    syntax_error(Line, S#elixir_scope.file, "unsupported option ~s given to ~s", [Key, Kind])
+    syntax_error(Meta, S#elixir_scope.file, "unsupported option ~s given to ~s", [Key, Kind])
   end || { Key, _ } <- Opts, not lists:member(Key, Allowed)];
 
-validate_opts(Line, Kind, _Allowed, _Opts, S) ->
-  syntax_error(Line, S#elixir_scope.file, "invalid options for ~s, expected a keyword list", [Kind]).
+validate_opts(Meta, Kind, _Allowed, _Opts, S) ->
+  syntax_error(Meta, S#elixir_scope.file, "invalid options for ~s, expected a keyword list", [Kind]).
+
+translate_fn(Meta, Clauses, S) ->
+  Line = ?line(Meta),
 
-translate_fn(Line, Clauses, S) ->
   Transformer = fun({ ArgsWithGuards, Expr }, Acc) ->
     { Args, Guards } = elixir_clauses:extract_last_guards(ArgsWithGuards),
     elixir_clauses:assigns_block(Line, fun elixir_translator:translate/2, Args, [Expr], Guards, umergec(S, Acc))
   end,
 
   { TClauses, NS } = lists:mapfoldl(Transformer, S, Clauses),
-  { { 'fun', Line, {clauses, TClauses} }, umergec(S, NS) }.
+  Arities = [length(Args) || { clause, _Line, Args, _Guards, _Exprs } <- TClauses],
 
-translate_local(Line, Name, Args, #elixir_scope{local=nil} = S) ->
+  case length(lists:usort(Arities)) of
+    1 ->
+      { { 'fun', Line, { clauses, TClauses } }, umergec(S, NS) };
+    _ ->
+      syntax_error(Meta, S#elixir_scope.file, "cannot mix clauses with different arities in function definition")
+  end.
+
+translate_local(Meta, Name, Args, #elixir_scope{local=nil} = S) ->
+  Line = ?line(Meta),
   { TArgs, NS } = translate_args(Args, S),
   { { call, Line, { atom, Line, Name }, TArgs }, NS };
 
-translate_local(Line, Name, Args, S) ->
-  { TArgs, NS } = translate_args(Args, S),
-  Remote = { remote, Line,
-    { atom, Line, S#elixir_scope.local },
-    { atom, Line, Name }
-  },
-  { { call, Line, Remote, TArgs }, NS }.
+translate_local(Meta, Name, Args, S) ->
+  translate_each({ { '.', Meta, [S#elixir_scope.local, Name] }, Meta, Args }, S).
+
+expand_var_context(_Meta, Atom, _Msg, _S) when is_atom(Atom) -> Atom;
+expand_var_context(Meta, Alias, Msg, S) ->
+  case translate_each(Alias, S) of
+    { { atom, _, Atom }, _ } ->
+      Atom;
+    _ ->
+      syntax_error(Meta, S#elixir_scope.file, "~ts, expected a compile time available alias or an atom", [Msg])
+  end.
+
+no_alias_expansion({ '__aliases__', Meta, [H|T] }) when (H /= 'Elixir') and is_atom(H) ->
+  { '__aliases__', Meta, ['Elixir',H|T] };
+
+no_alias_expansion(Other) ->
+  Other.
 
 %% Translate args
 
@@ -503,9 +580,9 @@ translate_local(Line, Name, Args, S) ->
 
 translate_arg(Arg, { Acc, S }) ->
   { TArg, TAcc } = translate_each(Arg, Acc),
-  { TArg, { umergec(S, TAcc), umergev(S, TAcc) } }.
+  { TArg, { umergec(Acc, TAcc), umergev(S, TAcc) } }.
 
-translate_args(Args, #elixir_scope{context=assign} = S) ->
+translate_args(Args, #elixir_scope{context=match} = S) ->
   translate(Args, S);
 
 translate_args(Args, S) ->
@@ -515,15 +592,15 @@ translate_args(Args, S) ->
 %% Translate apply
 %% Used by both apply and external function invocation macros.
 
-translate_apply(Line, TLeft, TRight, Args, S, SL, SR) ->
+translate_apply(Meta, TLeft, TRight, Args, S, SL, SR) ->
+  Line = ?line(Meta),
+
   Optimize = case (Args == []) orelse lists:last(Args) of
     { '|', _, _ } -> false;
     _ ->
-      case { TLeft, TRight } of
-        { { Kind, _, _ }, { atom, _, _ } } when Kind == var; Kind == tuple; Kind == atom ->
-          true;
-        _ ->
-          false
+      case TRight of
+        { atom, _, _ } -> true;
+        _ -> false
       end
   end,
 
@@ -535,7 +612,7 @@ translate_apply(Line, TLeft, TRight, Args, S, SL, SR) ->
     false ->
       { TArgs, SA } = translate_each(Args, umergec(S, SR)),
       FS = umergev(SL, umergev(SR,SA)),
-      { ?ELIXIR_WRAP_CALL(Line, erlang, apply, [TLeft, TRight, TArgs]), FS }
+      { ?wrap_call(Line, erlang, apply, [TLeft, TRight, TArgs]), FS }
   end.
 
 %% __op__ helpers
@@ -554,47 +631,46 @@ convert_op(Else)   ->  Else.
 is_atom_tuple({ atom, _, _ }) -> true;
 is_atom_tuple(_) -> false.
 
-rellocate_ambiguous_op({ Op, Line, [Expr] }, Var) when Op == '+'; Op == '-' ->
-  { Op, Line, [Var, Expr] };
+rellocate_ambiguous_op({ Op, Meta, [Expr] }, Var) when Op == '+'; Op == '-' ->
+  { Op, Meta, [Var, Expr] };
 
-rellocate_ambiguous_op({ Call, Line, [H|T] }, Var) ->
-  { Call, Line, [rellocate_ambiguous_op(H, Var)|T] }.
+rellocate_ambiguous_op({ Call, Meta, [H|T] }, Var) ->
+  { Call, Meta, [rellocate_ambiguous_op(H, Var)|T] }.
 
 %% Comprehensions
 
-translate_comprehension(Line, Kind, Args, S) ->
+translate_comprehension(Meta, Kind, Args, S) ->
   case elixir_tree_helpers:split_last(Args) of
     { Cases, [{do,Expr}] } ->
-      { TCases, SC } = lists:mapfoldl(fun translate_comprehension_clause/2, S, Cases),
-      { TExpr, SE }  = translate_comprehension_do(Line, Kind, Expr, SC),
-      { { Kind, Line, TExpr, TCases }, umergec(S, SE) };
+      { TCases, SC } = lists:mapfoldl(fun(C, Acc) -> translate_comprehension_clause(Meta, C, Acc) end, S, Cases),
+      { TExpr, SE }  = translate_comprehension_do(Meta, Kind, Expr, SC),
+      { { Kind, ?line(Meta), TExpr, TCases }, umergec(S, SE) };
     _ ->
-      syntax_error(Line, S#elixir_scope.file, "keyword argument :do missing for comprehension ~s", [Kind])
+      syntax_error(Meta, S#elixir_scope.file, "keyword argument :do missing for comprehension ~s", [Kind])
   end.
 
-translate_comprehension_do(_Line, bc, { '<<>>', _, _ } = Expr, S) ->
+translate_comprehension_do(_Meta, bc, { '<<>>', _, _ } = Expr, S) ->
   translate_each(Expr, S);
 
-translate_comprehension_do(Line, bc, _Expr, S) ->
-  syntax_error(Line, S#elixir_scope.file, "a bit comprehension expects a bit string << >> to be returned");
+translate_comprehension_do(Meta, bc, _Expr, S) ->
+  syntax_error(Meta, S#elixir_scope.file, "a bit comprehension expects a bit string << >> to be returned");
 
-translate_comprehension_do(_Line, _Kind, Expr, S) ->
+translate_comprehension_do(_Meta, _Kind, Expr, S) ->
   translate_each(Expr, S).
 
-translate_comprehension_clause({inbits, Line, [Left, Right]}, S) ->
+translate_comprehension_clause(_Meta, {inbits, Meta, [{ '<<>>', _, _} = Left, Right]}, S) ->
   { TRight, SR } = translate_each(Right, S),
   { TLeft, SL  } = elixir_clauses:assigns(fun elixir_translator:translate_each/2, Left, SR),
-  { { b_generate, Line, TLeft, TRight }, SL };
+  { { b_generate, ?line(Meta), TLeft, TRight }, SL };
 
-translate_comprehension_clause({inlist, Line, [Left, Right]}, S) ->
+translate_comprehension_clause(_Meta, {inbits, Meta, [_Left, _Right]}, S) ->
+  syntax_error(Meta, S#elixir_scope.file, "a bit comprehension expects a bit string << >> to be used in inbits generators");
+
+translate_comprehension_clause(_Meta, {inlist, Meta, [Left, Right]}, S) ->
   { TRight, SR } = translate_each(Right, S),
   { TLeft, SL  } = elixir_clauses:assigns(fun elixir_translator:translate_each/2, Left, SR),
-  { { generate, Line, TLeft, TRight }, SL };
+  { { generate, ?line(Meta), TLeft, TRight }, SL };
 
-translate_comprehension_clause(X, S) ->
+translate_comprehension_clause(Meta, X, S) ->
   { TX, TS } = translate_each(X, S),
-  Line = case X of
-    { _, L, _ } -> L;
-    _ -> 0
-  end,
-  { elixir_tree_helpers:convert_to_boolean(Line, TX, true, false), TS }.
+  elixir_tree_helpers:convert_to_boolean(?line(Meta), TX, true, false, TS).

lib/elixir/src/elixir_tree_helpers.erl

@@ -5,11 +5,17 @@
 -module(elixir_tree_helpers).
 -compile({parse_transform, elixir_transform}).
 -export([abstract_syntax/1, split_last/1, cons_to_list/1,
-  convert_to_boolean/4, returns_boolean/1,
+  convert_to_boolean/5, returns_boolean/1, get_line/1,
   build_list/4, build_list/5, build_simple_list/2,
   build_reverse_list/4, build_reverse_list/5, build_simple_reverse_list/2]).
 -include("elixir.hrl").
 
+get_line(Opts) ->
+  case lists:keyfind(line, 1, Opts) of
+    { line, Line } when is_integer(Line) -> Line;
+    false -> 0
+  end.
+
 split_last([])         -> { [], [] };
 split_last(List)       -> split_last(List, []).
 split_last([H], Acc)   -> { lists:reverse(Acc), H };
@@ -37,14 +43,14 @@ cons_to_list({ cons, _, Left, Right }) ->
 build_list(Fun, Exprs, Line, S) ->
   build_list(Fun, Exprs, Line, S, {nil, Line}).
 
-build_list(Fun, Exprs, Line, S, Tail) ->
+build_list(Fun, Exprs, Line, S, Tail) when is_integer(Line) ->
   build_list_each(Fun, lists:reverse(Exprs), Line, S, Tail).
 
 % Same as build_list, but the list given is in reverse other.
 build_reverse_list(Fun, Exprs, Line, S) ->
   build_list_each(Fun, Exprs, Line, S, {nil,Line}).
 
-build_reverse_list(Fun, Exprs, Line, S, Tail) ->
+build_reverse_list(Fun, Exprs, Line, S, Tail) when is_integer(Line) ->
   build_list_each(Fun, Exprs, Line, S, Tail).
 
 % Builds a simple list, without translatation, just by generating the cons-cell.
@@ -84,34 +90,47 @@ returns_boolean({ call, _, { remote, _, { atom, _, erlang }, { atom, _, Fun } },
 returns_boolean({ call, _, { remote, _, { atom, _, erlang }, { atom, _, Fun } }, [_,_] }) when
   Fun == is_function -> true;
 
+returns_boolean({ call, _, { remote, _, { atom, _, erlang }, { atom, _, Fun } }, [_,_,_] }) when
+  Fun == function_exported -> true;
+
+returns_boolean({ atom, _, Bool }) when Bool == true; Bool == false -> true;
+
+returns_boolean({ 'case', _, _, Clauses }) ->
+  lists:all(fun
+    ({clause,_,_,_,[Expr]}) -> returns_boolean(Expr);
+    (_) -> false
+  end, Clauses);
+
 returns_boolean(_) -> false.
 
-convert_to_boolean(Line, Expr, Bool, InGuard) ->
+convert_to_boolean(Line, Expr, Bool, InGuard, S) when is_integer(Line) ->
   case { returns_boolean(Expr), Bool } of
-    { true, true }  -> Expr;
-    { true, false } -> { op, Line, 'not', Expr };
-    _               -> do_convert_to_boolean(Line, Expr, Bool, InGuard)
+    { true, true }  -> { Expr, S };
+    { true, false } -> { { op, Line, 'not', Expr }, S };
+    _               -> do_convert_to_boolean(Line, Expr, Bool, InGuard, S)
   end.
 
-do_convert_to_boolean(Line, Expr, Bool, false) ->
-  Any   = [{var, Line, '_'}],
-  False = [{atom,Line,false}],
-  Nil   = [{atom,Line,nil}],
+%% Notice we use a temporary var and bundle nil
+%% and false checks in the same clause since
+%% it makes dialyzer happy.
+do_convert_to_boolean(Line, Expr, Bool, false, S) ->
+  Any         = { var, Line, '_' },
+  { Var, TS } = elixir_scope:build_erl_var(Line, S),
+  OrElse      = do_guarded_convert_to_boolean(Line, Var, 'orelse', '=='),
 
-  FalseResult = [{atom,Line,not Bool}],
-  TrueResult  = [{atom,Line,Bool}],
+  FalseResult = { atom,Line,not Bool },
+  TrueResult  = { atom,Line,Bool },
 
-  { 'case', Line, Expr, [
-    { clause, Line, False, [], FalseResult },
-    { clause, Line, Nil, [], FalseResult },
-    { clause, Line, Any, [], TrueResult }
-  ] };
+  { { 'case', Line, Expr, [
+    { clause, Line, [Var], [[OrElse]], [FalseResult] },
+    { clause, Line, [Any], [], [TrueResult] }
+  ] }, TS };
 
-do_convert_to_boolean(Line, Expr, true, true) ->
-  do_guarded_convert_to_boolean(Line, Expr, 'andalso', '/=');
+do_convert_to_boolean(Line, Expr, true, true, S) ->
+  { do_guarded_convert_to_boolean(Line, Expr, 'andalso', '/='), S };
 
-do_convert_to_boolean(Line, Expr, false, true) ->
-  do_guarded_convert_to_boolean(Line, Expr, 'orelse', '==').
+do_convert_to_boolean(Line, Expr, false, true, S) ->
+  { do_guarded_convert_to_boolean(Line, Expr, 'orelse', '=='), S }.
 
 do_guarded_convert_to_boolean(Line, Expr, Op, Comp) ->
   Left  = { op, Line, Comp, Expr, { atom, Line, false } },

lib/elixir/src/elixir_try.erl

@@ -1,136 +1,144 @@
 -module(elixir_try).
--export([clauses/3]).
+-export([clauses/3, format_error/1]).
 -import(elixir_scope, [umergec/2]).
 -include("elixir.hrl").
 -compile({parse_transform, elixir_transform}).
 
-clauses(Line, Clauses, S) ->
-  Catch  = elixir_clauses:get_pairs(Line, 'catch', Clauses, S),
-  Rescue = elixir_clauses:get_pairs(Line, rescue, Clauses, S),
-  Transformer = fun(X, Acc) -> each_clause(Line, X, umergec(S, Acc)) end,
+clauses(Meta, Clauses, S) ->
+  Catch  = elixir_clauses:get_pairs(Meta, 'catch', Clauses, S),
+  Rescue = elixir_clauses:get_pairs(Meta, rescue, Clauses, S),
+  Transformer = fun(X, Acc) -> each_clause(Meta, X, umergec(S, Acc)) end,
   lists:mapfoldl(Transformer, S, Rescue ++ Catch).
 
-each_clause(Line, { 'catch', Raw, Expr }, S) ->
+each_clause(Meta, { 'catch', Raw, Expr }, S) ->
   { Args, Guards } = elixir_clauses:extract_last_guards(Raw),
 
   Final = case Args of
-    [X]     -> [throw, X, { '_', Line, nil }];
-    [X,Y]   -> [X, Y, { '_', Line, nil }];
+    [X]     -> [throw, X, { '_', Meta, nil }];
+    [X,Y]   -> [X, Y, { '_', Meta, nil }];
     [_,_,_] -> Args;
     _       ->
-      elixir_errors:syntax_error(Line, S#elixir_scope.file, "too many arguments given for catch")
+      elixir_errors:syntax_error(Meta, S#elixir_scope.file, "too many arguments given for catch")
   end,
 
-  Condition = { '{}', Line, Final },
-  elixir_clauses:assigns_block(Line, fun elixir_translator:translate_each/2, Condition, [Expr], Guards, S);
+  Condition = { '{}', Meta, Final },
+  elixir_clauses:assigns_block(?line(Meta), fun elixir_translator:translate_each/2, Condition, [Expr], Guards, S);
 
-each_clause(Line, { rescue, [Condition], Expr }, S) ->
-  case normalize_rescue(Line, Condition, S) of
+each_clause(Meta, { rescue, [Condition|T], Expr }, S) ->
+  case normalize_rescue(Meta, Condition, S) of
     { Left, Right } ->
       case Left of
         { '_', _, _ } ->
-          { ClauseVar, CS } = elixir_scope:build_ex_var(Line, S),
-          { Clause, _ } = rescue_guards(Line, ClauseVar, Right, S),
-          each_clause(Line, { 'catch', [error, Clause], Expr }, CS);
+          { ClauseVar, CS } = elixir_scope:build_ex_var(?line(Meta), S),
+          { Clause, _ } = rescue_guards(Meta, ClauseVar, Right, S),
+          each_clause(Meta, { 'catch', [error, Clause|T], Expr }, CS);
         _ ->
-          { Clause, Safe } = rescue_guards(Line, Left, Right, S),
+          { Clause, Safe } = rescue_guards(Meta, Left, Right, S),
           case Safe of
             true ->
-              each_clause(Line, { 'catch', [error, Clause], Expr }, S);
+              each_clause(Meta, { 'catch', [error, Clause|T], Expr }, S);
             false ->
-              { ClauseVar, CS }  = elixir_scope:build_ex_var(Line, S),
-              { FinalClause, _ } = rescue_guards(Line, ClauseVar, Right, S),
-              Match = { '=', Line, [
+              { ClauseVar, CS }  = elixir_scope:build_ex_var(?line(Meta), S),
+              { FinalClause, _ } = rescue_guards(Meta, ClauseVar, Right, S),
+              Match = { '=', Meta, [
                 Left,
-                { { '.', Line, ['Elixir.Exception', normalize] }, Line, [ClauseVar] }
+                { { '.', Meta, ['Elixir.Exception', normalize] }, Meta, [ClauseVar] }
               ] },
-              FinalExpr = prepend_to_block(Line, Match, Expr),
-              each_clause(Line, { 'catch', [error, FinalClause], FinalExpr }, CS)
+              FinalExpr = prepend_to_block(Meta, Match, Expr),
+              each_clause(Meta, { 'catch', [error, FinalClause|T], FinalExpr }, CS)
           end
       end;
     _ ->
-      Result = each_clause(Line, { 'catch', [error, Condition], Expr }, S),
-      validate_rescue_access(Line, Condition, S),
-      Result
+      validate_rescue_access(Meta, Condition, S),
+      each_clause(Meta, { 'catch', [error, Condition|T], Expr }, S)
   end;
 
-each_clause(Line, {rescue,_,_}, S) ->
-  elixir_errors:syntax_error(Line, S#elixir_scope.file, "too many arguments given for rescue");
+each_clause(Meta, {rescue,_,_}, S) ->
+  elixir_errors:syntax_error(Meta, S#elixir_scope.file, "too many arguments given for rescue");
 
-each_clause(Line, {Key,_,_}, S) ->
-  elixir_errors:syntax_error(Line, S#elixir_scope.file, "invalid key ~s in try", [Key]).
+each_clause(Meta, {Key,_,_}, S) ->
+  elixir_errors:syntax_error(Meta, S#elixir_scope.file, "invalid key ~ts in try", [Key]).
 
 %% Helpers
 
 %% rescue [Error] -> _ in [Error]
-normalize_rescue(Line, List, S) when is_list(List) ->
-  normalize_rescue(Line, { in, Line, [{ '_', Line, nil }, List] }, S);
+normalize_rescue(Meta, List, S) when is_list(List) ->
+  normalize_rescue(Meta, { in, Meta, [{ '_', Meta, nil }, List] }, S);
 
 %% rescue var -> var in _
-normalize_rescue(_, { Name, Line, Atom } = Rescue, S) when is_atom(Name), is_atom(Atom), Name /= '_' ->
-  normalize_rescue(Line, { in, Line, [Rescue, { '_', Line, nil }] }, S);
+normalize_rescue(_, { Name, Meta, Atom } = Rescue, S) when is_atom(Name), is_atom(Atom), Name /= '_' ->
+  normalize_rescue(Meta, { in, Meta, [Rescue, { '_', Meta, nil }] }, S);
 
 %% rescue var in [Exprs]
-normalize_rescue(_, { in, Line, [Left, Right] }, S) ->
+normalize_rescue(_, { in, Meta, [Left, Right] }, S) ->
   case Right of
     { '_', _, _ } ->
       { Left, nil };
     _ when is_list(Right) ->
       is_valid_rescue_list(Right, S) andalso { Left, Right };
     _ ->
-      Expanded = 'Elixir.Macro':expand(Right, elixir_scope:to_ex_env({ Line, S })),
+      Expanded = 'Elixir.Macro':expand(Right, elixir_scope:to_ex_env({ ?line(Meta), S })),
       case is_valid_rescue_list(Expanded, S) of
         true  -> { Left, Expanded };
         false ->
-          elixir_errors:syntax_error(Line, S#elixir_scope.file, "invalid use of operator \"in\" in rescue inside try")
+          elixir_errors:syntax_error(Meta, S#elixir_scope.file, "invalid use of operator \"in\" in rescue inside try")
       end
   end;
 
-normalize_rescue(Line, Condition, S) ->
-  case elixir_translator:translate_each(Condition, S#elixir_scope{context=assign}) of
+normalize_rescue(_, { '=', Meta, [{ '__aliases__', _, _ } = Alias, { Name, _, Atom } = Var] }, S)
+    when is_atom(Name) and is_atom(Atom) ->
+  elixir_errors:handle_file_warning(S#elixir_scope.file, { Meta, ?MODULE, { rescue_no_match, Var, Alias } }),
+  false;
+
+normalize_rescue(_, { '=', Meta, [{ Name, _, Atom } = Var, { '__aliases__', _, _ } = Alias] }, S)
+    when is_atom(Name) and is_atom(Atom) ->
+  elixir_errors:handle_file_warning(S#elixir_scope.file, { Meta, ?MODULE, { rescue_no_match, Var, Alias } }),
+  false;
+
+normalize_rescue(Meta, Condition, S) ->
+  case elixir_translator:translate_each(Condition, S#elixir_scope{context=match}) of
     { { atom, _, Atom }, _ } ->
-      normalize_rescue(Line, { in, Line, [{ '_', Line, nil }, [Atom]] }, S);
+      normalize_rescue(Meta, { in, Meta, [{ '_', Meta, nil }, [Atom]] }, S);
     _ ->
       false
   end.
 
-
 %% Convert rescue clauses into guards.
 rescue_guards(_, Var, nil, _) -> { Var, false };
 
-rescue_guards(Line, Var, Guards, S) ->
-  { RawElixir, RawErlang } = rescue_each_var(Line, Var, Guards),
-  { Elixir, Erlang, Safe } = rescue_each_ref(Line, Var, Guards, RawElixir, RawErlang, RawErlang == [], S),
+rescue_guards(Meta, Var, Guards, S) ->
+  { RawElixir, RawErlang } = rescue_each_var(Meta, Var, Guards),
+  { Elixir, Erlang, Safe } = rescue_each_ref(Meta, Var, Guards, RawElixir, RawErlang, RawErlang == [], S),
 
   Final = case Elixir == [] of
     true  -> Erlang;
     false ->
-      IsTuple     = { is_tuple, Line, [Var] },
-      IsException = { '==', Line, [
-        { { '.', Line, [erlang, element] }, Line, [2, Var] }, '__exception__'
+      IsTuple     = { is_tuple, Meta, [Var] },
+      IsException = { '==', Meta, [
+        { { '.', Meta, [erlang, element] }, Meta, [2, Var] }, '__exception__'
       ] },
-      OrElse = join(Line, 'or', Elixir),
-      [join(Line, 'and', [IsTuple, IsException, OrElse])|Erlang]
+      OrElse = join(Meta, 'or', Elixir),
+      [join(Meta, 'and', [IsTuple, IsException, OrElse])|Erlang]
   end,
   {
-    { 'when', Line, [Var, reverse_join(Line, 'when', Final)] },
+    { 'when', Meta, [Var, reverse_join(Meta, 'when', Final)] },
     Safe
   }.
 
 %% Handle variables in the right side of rescue.
 
-rescue_each_var(Line, ClauseVar, Guards) ->
+rescue_each_var(Meta, ClauseVar, Guards) ->
   Vars = [Var || Var <- Guards, is_var(Var)],
 
   case Vars == [] of
     true  -> { [], [] };
     false ->
-      Elixir = [exception_compare(Line, ClauseVar, Var) || Var <- Vars],
+      Elixir = [exception_compare(Meta, ClauseVar, Var) || Var <- Vars],
       Erlang = lists:map(fun(Rescue) ->
-        Compares = [{ '==', Line, [Rescue, Var] } || Var <- Vars],
-        { 'and', Line, [
-          erlang_rescue_guard_for(Line, ClauseVar, Rescue),
-          join(Line, 'or', Compares)
+        Compares = [{ '==', Meta, [Rescue, Var] } || Var <- Vars],
+        { 'and', Meta, [
+          erlang_rescue_guard_for(Meta, ClauseVar, Rescue),
+          join(Meta, 'or', Compares)
         ] }
       end, erlang_rescues()),
       { Elixir, Erlang }
@@ -141,30 +149,30 @@ rescue_each_var(Line, ClauseVar, Guards) ->
 %% method for optimization.
 
 %% Ignore variables
-rescue_each_ref(Line, Var, [{ Name, _, Atom }|T], Elixir, Erlang, Safe, S) when is_atom(Name), is_atom(Atom) ->
-  rescue_each_ref(Line, Var, T, Elixir, Erlang, Safe, S);
+rescue_each_ref(Meta, Var, [{ Name, _, Atom }|T], Elixir, Erlang, Safe, S) when is_atom(Name), is_atom(Atom) ->
+  rescue_each_ref(Meta, Var, T, Elixir, Erlang, Safe, S);
 
-rescue_each_ref(Line, Var, [H|T], Elixir, Erlang, _Safe, S) when
+rescue_each_ref(Meta, Var, [H|T], Elixir, Erlang, _Safe, S) when
   H == 'Elixir.UndefinedFunctionError'; H == 'Elixir.ErlangError';
   H == 'Elixir.ArgumentError'; H == 'Elixir.ArithmeticError';
   H == 'Elixir.BadArityError'; H == 'Elixir.BadFunctionError';
   H == 'Elixir.MatchError'; H == 'Elixir.CaseClauseError';
   H == 'Elixir.FunctionClauseError'; H == 'Elixir.SystemLimitError' ->
-  Expr = { 'or', Line, [
-    erlang_rescue_guard_for(Line, Var, H),
-    exception_compare(Line, Var, H)
+  Expr = { 'or', Meta, [
+    erlang_rescue_guard_for(Meta, Var, H),
+    exception_compare(Meta, Var, H)
   ] },
-  rescue_each_ref(Line, Var, T, Elixir, [Expr|Erlang], false, S);
+  rescue_each_ref(Meta, Var, T, Elixir, [Expr|Erlang], false, S);
 
-rescue_each_ref(Line, Var, [H|T], Elixir, Erlang, Safe, S) when is_atom(H) ->
-  rescue_each_ref(Line, Var, T, [exception_compare(Line, Var, H)|Elixir], Erlang, Safe, S);
+rescue_each_ref(Meta, Var, [H|T], Elixir, Erlang, Safe, S) when is_atom(H) ->
+  rescue_each_ref(Meta, Var, T, [exception_compare(Meta, Var, H)|Elixir], Erlang, Safe, S);
 
-rescue_each_ref(Line, Var, [H|T], Elixir, Erlang, Safe, S) ->
+rescue_each_ref(Meta, Var, [H|T], Elixir, Erlang, Safe, S) ->
   case elixir_translator:translate_each(H, S) of
     { { atom, _, Atom }, _ } ->
-      rescue_each_ref(Line, Var, [Atom|T], Elixir, Erlang, Safe, S);
+      rescue_each_ref(Meta, Var, [Atom|T], Elixir, Erlang, Safe, S);
     _ ->
-      rescue_each_ref(Line, Var, T, [exception_compare(Line, Var, H)|Elixir], Erlang, Safe, S)
+      rescue_each_ref(Meta, Var, T, [exception_compare(Meta, Var, H)|Elixir], Erlang, Safe, S)
   end;
 
 rescue_each_ref(_, _, [], Elixir, Erlang, Safe, _) ->
@@ -179,73 +187,73 @@ erlang_rescues() ->
     'Elixir.SystemLimitError', 'Elixir.ErlangError'
   ].
 
-erlang_rescue_guard_for(Line, Var, List) when is_list(List) ->
-  join(Line, 'or', [erlang_rescue_guard_for(Line, Var, X) || X <- List]);
+erlang_rescue_guard_for(Meta, Var, List) when is_list(List) ->
+  join(Meta, 'or', [erlang_rescue_guard_for(Meta, Var, X) || X <- List]);
 
-erlang_rescue_guard_for(Line, Var, 'Elixir.UndefinedFunctionError') ->
-  { '==', Line, [Var, undef] };
+erlang_rescue_guard_for(Meta, Var, 'Elixir.UndefinedFunctionError') ->
+  { '==', Meta, [Var, undef] };
 
-erlang_rescue_guard_for(Line, Var, 'Elixir.FunctionClauseError') ->
-  { '==', Line, [Var, function_clause] };
+erlang_rescue_guard_for(Meta, Var, 'Elixir.FunctionClauseError') ->
+  { '==', Meta, [Var, function_clause] };
 
-erlang_rescue_guard_for(Line, Var, 'Elixir.SystemLimitError') ->
-  { '==', Line, [Var, system_limit] };
+erlang_rescue_guard_for(Meta, Var, 'Elixir.SystemLimitError') ->
+  { '==', Meta, [Var, system_limit] };
 
-erlang_rescue_guard_for(Line, Var, 'Elixir.ArithmeticError') ->
-  { '==', Line, [Var, badarith] };
+erlang_rescue_guard_for(Meta, Var, 'Elixir.ArithmeticError') ->
+  { '==', Meta, [Var, badarith] };
 
-erlang_rescue_guard_for(Line, Var, 'Elixir.BadArityError') ->
-  { 'and', Line, [
-    { is_tuple, Line, [Var] },
-    exception_compare(Line, Var, badarity)
+erlang_rescue_guard_for(Meta, Var, 'Elixir.BadArityError') ->
+  { 'and', Meta, [
+    { is_tuple, Meta, [Var] },
+    exception_compare(Meta, Var, badarity)
   ] };
 
-erlang_rescue_guard_for(Line, Var, 'Elixir.BadFunctionError') ->
-  { 'and', Line, [
-    { is_tuple, Line, [Var] },
-    exception_compare(Line, Var, badfun)
+erlang_rescue_guard_for(Meta, Var, 'Elixir.BadFunctionError') ->
+  { 'and', Meta, [
+    { is_tuple, Meta, [Var] },
+    exception_compare(Meta, Var, badfun)
   ] };
 
-erlang_rescue_guard_for(Line, Var, 'Elixir.MatchError') ->
-  { 'and', Line, [
-    { is_tuple, Line, [Var] },
-    exception_compare(Line, Var, badmatch)
+erlang_rescue_guard_for(Meta, Var, 'Elixir.MatchError') ->
+  { 'and', Meta, [
+    { is_tuple, Meta, [Var] },
+    exception_compare(Meta, Var, badmatch)
   ] };
 
-erlang_rescue_guard_for(Line, Var, 'Elixir.CaseClauseError') ->
-  { 'and', Line, [
-    { is_tuple, Line, [Var] },
-    exception_compare(Line, Var, case_clause)
+erlang_rescue_guard_for(Meta, Var, 'Elixir.CaseClauseError') ->
+  { 'and', Meta, [
+    { is_tuple, Meta, [Var] },
+    exception_compare(Meta, Var, case_clause)
   ] };
 
-erlang_rescue_guard_for(Line, Var, 'Elixir.ArgumentError') ->
-  { 'or', Line, [
-    { '==', Line, [Var, badarg] },
-    { 'and', Line, [
-      { is_tuple, Line, [Var] },
-      exception_compare(Line, Var, badarg)
+erlang_rescue_guard_for(Meta, Var, 'Elixir.ArgumentError') ->
+  { 'or', Meta, [
+    { '==', Meta, [Var, badarg] },
+    { 'and', Meta, [
+      { is_tuple, Meta, [Var] },
+      exception_compare(Meta, Var, badarg)
     ] }
   ] };
 
-erlang_rescue_guard_for(Line, Var, 'Elixir.ErlangError') ->
-  IsNotTuple  = { 'not', Line, [{ is_tuple, Line, [Var] }] },
-  IsException = { '!=', Line, [
-    { { '.', Line, [ erlang, element ] }, Line, [2, Var] }, '__exception__'
+erlang_rescue_guard_for(Meta, Var, 'Elixir.ErlangError') ->
+  IsNotTuple  = { 'not', Meta, [{ is_tuple, Meta, [Var] }] },
+  IsException = { '!=', Meta, [
+    { { '.', Meta, [ erlang, element ] }, Meta, [2, Var] }, '__exception__'
   ] },
-  { 'or', Line, [IsNotTuple, IsException] }.
+  { 'or', Meta, [IsNotTuple, IsException] }.
 
 %% Validate rescue access
 
-validate_rescue_access(Line, { '=', _, [Left, Right] }, S) ->
-  validate_rescue_access(Line, Left, S),
-  validate_rescue_access(Line, Right, S);
+validate_rescue_access(Meta, { '=', _, [Left, Right] }, S) ->
+  validate_rescue_access(Meta, Left, S),
+  validate_rescue_access(Meta, Right, S);
 
-validate_rescue_access(Line, { 'access', _, [Element, _] }, S) ->
+validate_rescue_access(Meta, { { '.', _, ['Elixir.Kernel', 'access'] }, _, [Element, _] }, S) ->
   case elixir_translator:translate_each(Element, S) of
     { { atom, _, Atom }, _ } ->
       case lists:member(Atom, erlang_rescues()) of
         false -> [];
-        true -> elixir_errors:syntax_error(Line, S#elixir_scope.file, "cannot (yet) pattern match against erlang exceptions")
+        true -> elixir_errors:syntax_error(Meta, S#elixir_scope.file, "cannot (yet) pattern match against erlang exceptions")
       end;
     _ -> []
   end;
@@ -254,6 +262,12 @@ validate_rescue_access(_, _, _) -> [].
 
 %% Helpers
 
+format_error({ rescue_no_match, Var, Alias }) ->
+  VarBinary   = 'Elixir.Macro':to_binary(Var),
+  AliasBinary = 'Elixir.Macro':to_binary(Alias),
+  Message = "rescue clause (~ts = ~ts) can never match, maybe you meant to write: ~ts in [~ts] ?",
+  io_lib:format(Message, [AliasBinary, VarBinary, VarBinary, AliasBinary]).
+
 is_var({ Name, _, Atom }) when is_atom(Name), is_atom(Atom) -> true;
 is_var(_) -> false.
 
@@ -264,20 +278,20 @@ is_valid_rescue_list(Right, S) when is_list(Right) ->
   { TRight, _ } = elixir_translator:translate(Right, S),
   lists:all(fun is_erl_var_or_atom/1, TRight).
 
-exception_compare(Line, Var, Expr) ->
-  { '==', Line, [
-    { { '.', Line, [ erlang, element ] }, Line, [1, Var] },
+exception_compare(Meta, Var, Expr) ->
+  { '==', Meta, [
+    { { '.', Meta, [ erlang, element ] }, Meta, [1, Var] },
     Expr
   ] }.
 
-join(Line, Kind, [H|T]) ->
-  lists:foldl(fun(X, Acc) -> { Kind, Line, [Acc, X] } end, H, T).
+join(Meta, Kind, [H|T]) ->
+  lists:foldl(fun(X, Acc) -> { Kind, Meta, [Acc, X] } end, H, T).
 
-reverse_join(Line, Kind, [H|T]) ->
-  lists:foldl(fun(X, Acc) -> { Kind, Line, [X, Acc] } end, H, T).
+reverse_join(Meta, Kind, [H|T]) ->
+  lists:foldl(fun(X, Acc) -> { Kind, Meta, [X, Acc] } end, H, T).
 
-prepend_to_block(_Line, Expr, { '__block__', Line, Args }) ->
-  { '__block__', Line, [Expr|Args] };
+prepend_to_block(_Meta, Expr, { '__block__', Meta, Args }) ->
+  { '__block__', Meta, [Expr|Args] };
 
-prepend_to_block(Line, Expr, Args) ->
-  { '__block__', Line, [Expr, Args] }.
+prepend_to_block(Meta, Expr, Args) ->
+  { '__block__', Meta, [Expr, Args] }.

lib/elixir/test/doctest.exs

@@ -0,0 +1,28 @@
+ExUnit.start []
+
+defmodule KernelTest do
+  use ExUnit.Case, async: true
+  doctest Kernel
+  doctest Kernel.SpecialForms
+  doctest Access.List
+  doctest Binary.Chars.List
+  doctest Binary.Inspect.Atom
+  doctest Binary.Inspect.BitString
+  doctest Binary.Inspect.List
+  doctest Binary.Inspect.Tuple
+  doctest Binary.Inspect.Number
+  doctest Binary.Inspect.Regex
+  doctest Bitwise
+  doctest Code
+  doctest Dict
+  doctest Enum
+  doctest IO.ANSI
+  doctest Regex
+  doctest String
+  doctest OptionParser
+  doctest Path
+  doctest Module
+  doctest Macro
+  doctest List
+  doctest Keyword
+end

lib/elixir/test/elixir/behaviour_test.exs

@@ -6,16 +6,42 @@ defmodule BehaviourTest do
   defmodule Sample do
     use Behaviour
 
-    @doc "Doc"
-    defcallback foo(foo, bar // [], bat // [])
+    @doc "Foo"
+    defcallback foo(atom, binary) :: binary
+
+    @doc "Bar"
+    defcallback bar(External.hello, my_var :: binary) :: binary
   end
 
-  test :docs_are_defined do
-    assert [_, _] = Sample.__info__(:docs)
+  test :docs do
+    docs = Enum.sort(Sample.__behaviour__(:docs))
+    assert docs == [
+      {{:bar,2},13,"Bar"},
+      {{:foo,2},10,"Foo"}
+    ]
   end
 
   test :callbacks do
-    assert [{:foo,3},{:foo,2},{:foo,1}] = Sample.behaviour_info(:callbacks)
+    assert Sample.__behaviour__(:callbacks) == [foo: 2, bar: 2]
+  end
+
+  test :specs do
+    assert length(Keyword.get_values(Sample.module_info[:attributes], :callback)) == 2
+  end
+
+  test :default_is_not_supported do
+    assert_raise ArgumentError, fn ->
+      defmodule WithDefault do
+        use Behaviour
+        defcallback hello(num // 0 :: integer) :: integer
+      end
+    end
+
+    assert_raise ArgumentError, fn ->
+      defmodule WithDefault do
+        use Behaviour
+        defcallback hello(num :: integer // 0) :: integer
+      end
+    end
   end
 end
-