pytest-asyncio 0.22.0 will cause pytest INTERNALERROR when encounter module level pytest.skip

[kkpattern]

After upgrading to version 0.22.0, we noticed that when pytest-asyncio collects the tests, if there is a module-level pytest.skip, it causes an INTERNALERROR instead of skipping the test file.

Minimal Repo

import pytest
try:
    import numpy
except ImportError:
    pytest.skip("numpy not installed", allow_module_level=True)

With pytest-asyncio 0.21.1, running python -m pytest will output:

===================================================== test session starts =====================================================
platform linux -- Python 3.11.3, pytest-7.4.3, pluggy-1.3.0
rootdir: /path/to/test/test_pytest_asyncio
configfile: pytest.ini
plugins: asyncio-0.21.1
asyncio: mode=Mode.AUTO
collected 0 items / 1 skipped

===================================================== 1 skipped in 0.01s ======================================================

With pytest-asyncio 0.22.0, running python -m pytest will output:

===================================================== test session starts =====================================================platform linux -- Python 3.11.3, pytest-7.4.3, pluggy-1.3.0
rootdir: /path/to/test/test_pytest_asyncio
configfile: pytest.ini
plugins: asyncio-0.22.0
asyncio: mode=Mode.AUTO
collected 0 items
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/path/to/test/test_pytest_asyncio/test.py", line 3, in <module>
INTERNALERROR>     import numpy
INTERNALERROR> ModuleNotFoundError: No module named 'numpy'
INTERNALERROR>
INTERNALERROR> During handling of the above exception, another exception occurred:
INTERNALERROR>
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/path/to/test/.venv/lib/python3.11/site-packages/_pytest/main.py", line 271, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>                          ^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/path/to/test/.venv/lib/python3.11/site-packages/_pytest/main.py", line 324, in _main
INTERNALERROR>     config.hook.pytest_collection(session=session)
INTERNALERROR>   File "/path/to/test/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 493, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/path/to/test/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 115, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/path/to/test/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 152, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/path/to/test/.venv/lib/python3.11/site-packages/pluggy/_result.py", line 114, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/path/to/test/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 77, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/path/to/test/.venv/lib/python3.11/site-packages/_pytest/main.py", line 335, in pytest_collection
INTERNALERROR>     session.perform_collect()
INTERNALERROR>   File "/path/to/test/.venv/lib/python3.11/site-packages/_pytest/main.py", line 675, in perform_collect
INTERNALERROR>     self.items.extend(self.genitems(node))
INTERNALERROR>   File "/path/to/test/.venv/lib/python3.11/site-packages/_pytest/main.py", line 842, in genitems
INTERNALERROR>     rep = collect_one_node(node)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/path/to/test/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 546, in collect_one_node
INTERNALERROR>     ihook.pytest_collectstart(collector=collector)
INTERNALERROR>   File "/path/to/test/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 493, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/path/to/test/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 115, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/path/to/test/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 113, in _multicall
INTERNALERROR>     raise exception.with_traceback(exception.__traceback__)
INTERNALERROR>   File "/path/to/test/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 77, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/path/to/test/.venv/lib/python3.11/site-packages/pytest_asyncio/plugin.py", line 552, in pytest_collectstart
INTERNALERROR>     marks = get_unpacked_marks(collector.obj, consider_mro=True)
INTERNALERROR>                                ^^^^^^^^^^^^^
INTERNALERROR>   File "/path/to/test/.venv/lib/python3.11/site-packages/_pytest/python.py", line 310, in obj
INTERNALERROR>     self._obj = obj = self._getobj()
INTERNALERROR>                       ^^^^^^^^^^^^^^
INTERNALERROR>   File "/path/to/test/.venv/lib/python3.11/site-packages/_pytest/python.py", line 528, in _getobj
INTERNALERROR>     return self._importtestmodule()
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/path/to/test/.venv/lib/python3.11/site-packages/_pytest/python.py", line 617, in _importtestmodule
INTERNALERROR>     mod = import_path(self.path, mode=importmode, root=self.config.rootpath)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/path/to/test/.venv/lib/python3.11/site-packages/_pytest/pathlib.py", line 567, in import_path
INTERNALERROR>     importlib.import_module(module_name)
INTERNALERROR>   File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
INTERNALERROR>     return _bootstrap._gcd_import(name[level:], package, level)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
INTERNALERROR>   File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
INTERNALERROR>   File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
INTERNALERROR>   File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
INTERNALERROR>   File "/path/to/test/.venv/lib/python3.11/site-packages/_pytest/assertion/rewrite.py", line 186, in exec_module
INTERNALERROR>     exec(co, module.__dict__)
INTERNALERROR>   File "/path/to/test/test_pytest_asyncio/test.py", line 5, in <module>
INTERNALERROR>     pytest.skip("numpy not installed", allow_module_level=True)
INTERNALERROR>   File "/path/to/test/.venv/lib/python3.11/site-packages/_pytest/outcomes.py", line 179, in skip
INTERNALERROR>     raise Skipped(msg=reason, allow_module_level=allow_module_level)
INTERNALERROR> Skipped: numpy not installed

==================================================== no tests ran in 0.01s ====================================================

[seifertm]

Thanks, great reproducer and bug report!

pytest-asyncio does have tests for pytest.skip, but none use allow_module_level=True. We'll have to check what's wrong.

[seifertm]

The code triggering the internal error no longer exists on main. Pytest-asyncio v0.22.0 has been yanked from PyPI and there are no plans for a patch release for v0.22.