Commit Graph
686 Commits
Author SHA1 Message Date
Jechol Lee 360ed75d17 Mix test parameters into the per-test :rand seed (#15869)
ExUnit seeds each test process with {phash2(module), phash2(name), seed}.
parameterize re-runs the module per parameter set without changing the
test name, so every parameter set of a test started from the same :rand
state and drew the same random sequence.

Hash the parameters together with the name when they are present. Tests
without parameters keep their previous seed, so existing --seed
reproductions are unaffected.

Assisted-by: Claude Code:claude-fable-5-1

Signed-off-by: Jechol Lee <mr.jechol@gmail.com>
2026-09-08 13:11:39 +02:00
Tobias Pfeiffer 936078255d Do not overlap sibling failures in parameterized tests failures manifest
Fixes #15820
2026-09-02 21:22:03 +02:00
Kuba Migdał 4d242a8c3d Sanitize ExUnit formatters against invalid UTF-8 (#15821) 2026-09-01 15:05:16 +02:00
Tiago Moraes dbd2fadba1 Simplify CaptureLog formatter test (#15810) 2026-08-30 19:06:32 +02:00
José Valim 3389f84059 Fix more warnings during test suite runs 2026-08-25 13:53:28 +02:00
Alexander Gubarev e20ca7e788 Clarify ExUnit --slowest-modules switch header (#15660) 2026-07-23 18:36:26 +02:00
José Valim 5d2ac0ca69 Make monitored test check more reliable 2026-07-23 09:00:30 +02:00
Alexander Gubarev 51ac6a5c0f Fix refute_in_delta boundary logic and negative delta (#15623) 2026-07-16 19:09:17 +02:00
Alexander Gubarev f2ad598ca3 Fix ExUnit StringIO processes leak when named device capture failed (#15621) 2026-07-16 17:29:21 +02:00
Jean Klingler bec8d39796 Add :migrate_atom_interpolations option to formatter (#15531) 2026-07-02 08:04:25 +09:00
José Valim 1ac9007bae Fix race condition in test 2026-06-17 10:53:12 +02:00
Jean Klingler 83e9b66e65 Add to_unsafe_atom/1 and soft-deprecate to_atom/1 (#15494) 2026-06-16 20:50:11 +09:00
Steffen Deusch 1cb62b731c Add trace helper to ExUnit.Assertions (#15432) 2026-06-08 14:17:55 +02:00
Evadne Wu b58e843528 Scope ExUnit diff width tests (#15396) 2026-05-16 01:27:33 +02:00
José Valim aae39c87dc Fix warnings 2026-05-12 14:49:36 +02:00
Gary Rennie 5b97e9c132 Show remaining runs when using --repeat-until-failure (#15270)
The --repeat-until-failure can be very useful for finding flaky tests,
however sometimes it is hard to gauge how long it has left to run before
it can be considered successful.

This commit adds a small output showing the number of remaining runs
when the `--repeat-until-failure` flag is used.

It looks like:

```
Running ExUnit with seed: 382462, max_cases: 32, remaining_runs: 3
Excluding tags: [windows: true]

.......
Finished in 0.02 seconds (0.00s async, 0.02s sync)

Result: 7 passed
Running ExUnit with seed: 422319, max_cases: 32, remaining_runs: 2
Excluding tags: [windows: true]

.......
Finished in 0.00 seconds (0.00s async, 0.00s sync)
```
2026-04-15 09:46:55 +02:00
José Valim 03b9fde67a Allow test names longer than 255 characters 2026-03-29 09:13:46 +02:00
José Valim a24d3ff221 Avoid false positives when diffing bitstrings, closes #15204 2026-03-25 08:20:05 +01:00
José Valim 2e7a49d426 Abort diffing on binary patterns, closes #15191 2026-03-13 18:25:07 +01:00
José Valim a064a00601 Fix CI and improve ExUnit results 2026-03-11 11:19:51 +01:00
Andrea Leopardi 352bc46c8e Improve ExUnit failure/success output (#15168) 2026-03-11 08:27:58 +01:00
Jean Klingler 2817a70680 Remove mentions to OTP27+ versions from docs, comments and tests (#15165)
* Remove mentions of "since OTP27+"

* Remove test code for OTP26-
2026-03-10 15:39:03 +09:00
José Valim 0ea809e720 Check redundant clauses for definitions (#15124) 2026-03-02 17:19:51 +01:00
Michał Łępicki 6d5f43318c Allow specifying SEED when running tests (#15111)
this is helpful when reproducing flaky test failures
2026-02-25 09:12:15 +01:00
José Valim c35f651309 Analyze conditional orelse and when 2026-01-03 13:38:38 +01:00
José Valim f1bbb2cd32 Infer types from guards and do post-inference on stdlib (#15032)
We now infer types from guards although we don't
yet infer conditional types (such as on `or`).
2026-01-02 19:48:45 +01:00
José Valim 47aab3b37c Ensure parallel matches in assert propagate type information, closes #14921 2025-11-18 15:19:57 -08:00
Austin Ziegler 5732283ada Add :formatter to ExUnit.CaptureLog options (#14914)
Mostly modified `:log_capture_on` handler for ExUnit.CaptureServer to
respect a `:formatter` option instead `Logger.default_formatter/1`.

Updates were made to documentation and tests to reflect this change.
2025-11-17 12:34:31 -08:00
Łukasz Samson ee469e56e6 Fix crash when format_kind_reason is executed with TestModule (#14901)
Closes #14900.
2025-11-13 13:05:09 +01:00
José Valim 7eb5b3ae17 Reset ansi escapes before newlines in Logger
Closes #14855
Closes #14841
2025-10-24 12:19:14 +02:00
José Valim 977efeac99 Do not crash on empty test unit groups, closes #14754 2025-10-07 11:12:33 +02:00
José Valim c6d3e48474 Fix regression on direct raise of ExUnit.AssertionError 2025-10-04 21:40:52 +02:00
Nathan Long 95265ba1c2 ExUnit sets a process label for each test (#14758) 2025-09-09 21:53:07 +02:00
José Valim 848fc1d6df Raise if message in AssertionError is not a binary
Closes #14695.
2025-08-31 10:04:37 +02:00
Eksperimental 9ae79d1385 Format numbers with underscores to improve readability (#14735) 2025-08-30 19:25:48 +02:00
Eksperimental 927f91c9fe Fix order in ExUnit results when listing pinned variables (#14723)
The pinned variables were returned in a random order (often reversed):

     test/ex_unit_pinned_variables_order_test.exs:23
     match (=) failed
     The following variables were pinned:
       var_d = "four"
       var_c = "three"
       var_b = "two"
       var_a = "one"
     code:  assert %{a: ^var_d, b: ^var_c, c: ^var_b, d: ^var_a} = build(var_a, var_b, var_c, var_d)
     left:  %{a: ^var_d, b: ^var_c, c: ^var_b, d: ^var_a}
     right: %{a: "one", b: "two", c: "three", d: "four"}
     stacktrace:
       test/ex_unit_pinned_variables_order_test.exs:29: (test)

This fix sorts them alphabetically.

This bug was introduced in 884e93391e when the pinned vars were now accumulated in a map (instead
of a list).

A repo replicating the issue can be found here:
- https://github.com/eksperimental-debug/elixir_debug/tree/ex-unit-pinned-variables-order
- https://github.com/eksperimental-debug/elixir_debug/blob/ex-unit-pinned-variables-order/ex_unit_pinned_variables_order/test/ex_unit_pinned_variables_order_test.exs
2025-08-27 08:06:14 +02:00
José Valim 09dcbf7566 Introduce deprecations scheduled for Elixir v1.20 2025-06-07 21:55:06 +02:00
José Valim 51e160e6f8 Add OTP 28 to CI (#14510) 2025-05-21 13:48:13 +02:00
José Valim 87dd05e5fd Preserve files with no line filter on mix test
Before this patch, if you invoked:

    mix test test/foo_test.exs:13 test/bar_test.exs

We would include a location filter only for foo_test.exs
and no tests in bar_test.exs would run. We now ensure
that the full location of files with line filters are
included as tags, as long as any file is included.

Closes #14445.
2025-04-25 15:33:43 +02:00
José Valim 920a6a0a42 Consistenly apply timeouts across suites (#14444)
Closes #14443.
2025-04-18 12:11:12 +02:00
Rudolf Manusadzhian 7e6e7008f9 Add group to the test context (#14440) 2025-04-17 09:39:44 +02:00
Jonatan Männchen f3e8ceb5aa Implement Test Coverage Reporting (#14343) 2025-04-07 22:37:01 +02:00
Steffen Deusch b08eea70e5 Prevent exits from leaking in ExUnit.CaptureServer (#14348)
When a process that traps exits used ExUnit's log capture functionality,
it would receive stray exits from the Tasks started inside it that could
lead to issues if not handled.
2025-03-19 16:51:28 +01:00
Jean Klingler b3cc618199 Fix regression when diffing nested improper lists (#14292)
Close https://github.com/elixir-lang/elixir/issues/14291
2025-02-21 23:08:36 +09:00
José Valim c1e79d189c Support Erlang/OTP 28 2025-02-20 18:10:23 +01:00
Daniel Kukula c65f9555aa Fix ExUnit list diff bug (#14275) 2025-02-19 08:32:15 +01:00
Daniel Kukula f1bf1b8602 Use newlines and pretty printing in several builtin exceptions (#14269) 2025-02-18 11:39:58 +01:00
Jonatan Männchen f7bbe81ea7 Inline licence info & copyright for test files (#14257) 2025-02-07 20:54:14 +01:00
Jean Klingler 4087560237 Support ellipsis in doctest exceptions (#14233) 2025-01-28 18:48:47 +09:00
eeff 5354a1fc8d Add :inspect_opts option for doctest (#14225)
Closes #14219
2025-01-24 21:29:21 +01:00