-
Notifications
You must be signed in to change notification settings - Fork 159
Prevent double wrapping of inherited Hypothesis tests #239
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Prevent double wrapping of inherited Hypothesis tests #239
Conversation
… and renamed it to "test_base". This allows us to add more tests for the Hypothesis integration while at the same time keep different tests tidy by using different files. Signed-off-by: Michael Seifert <[email protected]>
5639a37
to
927f376
Compare
Pytest-asyncio identifies Hypothesis test cases by their `is_hypothesis_test` flag. When setting up an async Hypothesis test pytest-asyncio replaces the function's `hypothesis.inner_test` attribute. The the top level function never changes. When a Hypothesis test case is defined in a base class and inherited by subclasses, the test is collected in each subclass. Since the top-level Hypothesis test never changes, its inner test will be wrapped multiple times. Double wrapping leads to execution errors caused by stale (closed) event loops in all test executions after the first. This change adds an `original_test_function` attribute to the async function wrapper, in order to keep track of the original Hypothesis test. When re-wrapping would occur in subclasses pytest-asyncio wraps the original test function rather than the wrapper function. Closes pytest-dev#231 Signed-off-by: Michael Seifert <[email protected]>
Signed-off-by: Michael Seifert <[email protected]>
This is an important information for downstream packagers. Signed-off-by: Michael Seifert <[email protected]>
927f376
to
dc74a20
Compare
Codecov Report
@@ Coverage Diff @@
## master #239 +/- ##
=======================================
Coverage 95.83% 95.83%
=======================================
Files 2 2
Lines 144 144
=======================================
Hits 138 138
Misses 6 6
Continue to review full report at Codecov.
|
Rebased to master. #220 addresses the same issue in a slightly different way. I adjusted the PR accordingly. The PR now:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
When looking at #231 I thought that something is wrong with the setup and teardown of the
event_loop
fixture. Running pytest with--show-setup
proved that this is not the case, though:I dug further and noticed that regular async tests are executed just fine. Only tests that were using Hypothesis raised errors when they were executed in a subclass. Eventually, it turned out that subclassing caused multiple synchronous function wrappers to be layered around inherited Hypothesis tests. Here's why:
Pytest-asyncio identifies Hypothesis test cases by their
is_hypothesis_test
attribute. When setting up an async Hypothesis test pytest-asyncio replaces the function'shypothesis.inner_test
attribute. The the top level function never changes.When a Hypothesis test case is defined in a base class and inherited by subclasses, the test is collected in each subclass. Since the top-level Hypothesis test never changes, its inner test will be wrapped multiple times.
Double wrapping leads to execution errors caused by stale (closed) event loops in all test executions after the first. This also explains why a module scoped fixture could serve as a workaround for the OP in #231.
This change adds an
original_test_function
attribute to the async function wrapper, in order to keep track of the original Hypothesis test. When re-wrapping would occur in subclasses pytest-asyncio wraps the original test function rather than the wrapper function.Closes #231