-
Notifications
You must be signed in to change notification settings - Fork 3.4k
Add parameterized tests #13618
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add parameterized tests #13618
Conversation
Sometimes you want to run the same tests but with different parameters. In ExUnit, it is possible to do by returning a `:parameterize` key in your `setup_all` context. The value must be a list of maps which will be the parameters merged into the test context. For example, Elixir has a module called `Registry`, which can have type `:unique` or `:duplicate`, and can control its concurrency factor using the `:partitions` option. If you have a number of tests that *behave the same* across all of those values, I can parameterize those tests with: setup_all do parameters = for kind <- [:unique, :duplicate], partitions <- [1, 8], do: %{kind: kind, partitions: partitions} [parameterize: parameters] end Look at the changes to registry_test.exs in this pull request as an example. As a benefit, tests now run faster too, as there is less code to compile. Use parameterized tests with care: * Abuse of parameterized tests may make your test suite considerably slower * If you use parameterized tests and then find yourself adding conditionals in your tests to deal with different parameters, then parameterized tests may be the wrong solution to your problem. Consider creating separated tests and sharing logic between them using regular functions
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sweet 🍬
Co-authored-by: Andrea Leopardi <[email protected]>
I have pushed a new implementation that allows parameterized tests to run concurrently. Thank you @sorentwo and @wojtekmach for the feedback. |
{parameterize, opts} = Keyword.pop(opts, :parameterize, nil) | ||
|
||
unless parameterize == nil or (is_list(parameterize) and Enum.all?(parameterize, &is_map/1)) do |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure if truly better:
{parameterize, opts} = Keyword.pop(opts, :parameterize, nil) | |
unless parameterize == nil or (is_list(parameterize) and Enum.all?(parameterize, &is_map/1)) do | |
{parameterize, opts} = Keyword.pop(opts, :parameterize) | |
if parameterize && not (is_list(parameterize) and Enum.all?(parameterize, &is_map/1)) do |
💚 💙 💜 💛 ❤️ |
This is the same behavor as pre elixir-lang#13618
Sometimes you want to run the same tests but with different parameters.
In ExUnit, it is possible to do so by passing a
:parameterize
key toExUnit.Case
. The value must be a list of maps which will be theparameters merged into the test context.
For example, Elixir has a module called
Registry
, which can have type:unique
or:duplicate
, and can control its concurrency factor usingthe
:partitions
option. If you have a number of tests that behave thesame across all of those values, you can parameterize those tests with:
Then, in your tests, you can access the parameters as part of the context:
Use parameterized tests with care:
Although parameterized tests run concurrently when
async: true
is also given,abuse of parameterized tests may make your test suite slower
If you use parameterized tests and then find yourself adding conditionals
in your tests to deal with different parameters, then parameterized tests
may be the wrong solution to your problem. Consider creating separated
tests and sharing logic between them using regular functions