Require explicit marks for async fixtures

[pipermerriam]

pytest-asyncio interprets all async def based fixtures as things that should be run by the asyncio event loop. This appears to make it impossible for a test suite to have side-by-side tests for both libraries.

import asyncio
import trio

import pytest

import pytest_trio


@pytest_trio.trio_fixture
async def trio_thing():
    await trio.sleep(0)
    return 'trio'


@pytest.fixture
async def asyncio_thing():
    await asyncio.sleep(0)
    return 'asyncio'


@pytest.mark.asyncio
async def test_something_asyncio(asyncio_thing):
    await asyncio.sleep(0)
    assert asyncio_thing == 'asyncio'


@pytest.mark.trio
async def test_something_trio(trio_thing):
    await trio.sleep(0)
    assert trio_thing == 'trio'

This results in:

$ pytest test_it.py -s
=========================================================================================== test session starts ===========================================================================================
platform linux -- Python 3.6.5, pytest-4.0.2, py-1.8.0, pluggy-0.9.0 -- /home/piper/python-environments/lahja/bin/python
cachedir: .pytest_cache
rootdir: /home/piper/projects/lahja, inifile: pytest.ini
plugins: xdist-1.25.0, trio-0.5.2, forked-1.0.2, asyncio-0.10.0
collected 2 items

test_it.py::test_something_asyncio PASSED
test_it.py::test_something_trio ERROR

================================================================================================= ERRORS ==================================================================================================
__________________________________________________________________________________ ERROR at setup of test_something_trio __________________________________________________________________________________

    @pytest_trio.trio_fixture
    async def trio_thing():
>       await trio.sleep(0)


test_it.py:11:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../../python-environments/lahja/lib/python3.6/site-packages/trio/_timeouts.py:83: in sleep
    await _core.checkpoint()
../../../python-environments/lahja/lib/python3.6/site-packages/trio/_core/_run.py:1697: in checkpoint
    with CancelScope(deadline=-inf):
../../../python-environments/lahja/lib/python3.6/site-packages/trio/_core/_ki.py:165: in wrapper
    return fn(*args, **kwargs)
../../../python-environments/lahja/lib/python3.6/site-packages/trio/_core/_run.py:179: in __enter__
    task = _core.current_task()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def current_task():
        """Return the :class:`Task` object representing the current task.

        Returns:
          Task: the :class:`Task` that called :func:`current_task`.

        """

        try:
            return GLOBAL_RUN_CONTEXT.task
        except AttributeError:
>           raise RuntimeError("must be called from async context") from None
E           RuntimeError: must be called from async context


../../../python-environments/lahja/lib/python3.6/site-packages/trio/_core/_run.py:1645: RuntimeError

[asvetlov]

See also #61

[agronholm]

Would it not be possible for the plugin to only act on fixtures if the requesting test is marked with @pytest.mark.asyncio? AnyIO checks for its own marker but I'm not sure if it works with autouse=True fixtures.

[njsmith]

A user just asked for support because they couldn't get pytest-trio working, and it turned out to be because they had a copy of pytest-asyncio installed as well. Their reproducer had an async fixture in it, so I'm pretty sure that this issue is the cause.

https://gitter.im/python-trio/general?at=5d005d341edaa30adf0746fd

[njsmith]

I just had a quick go at fixing this, by adding the following code to the top of pytest-asyncio's pytest_fixture_setup:

    if 'asyncio' not in request.keywords:
        yield
        return

The idea being that pytest-asyncio would only apply its special handling to fixtures if they're being requested by an asyncio test. This seems like the simplest hack that would fix most of the surprising crashes.

It almost works – 1 failed, 36 passed. The failing test is test_async_fixture_scope, which uses a module-scoped async fixture. It turns out that for module-scoped fixtures, AFAICT there's no way to get at the markers on the original functions that are requesting the module-scoped fixture. At least using public APIs... all the tests pass if we use

    if 'asyncio' not in request.keywords and 'asyncio' not in request._parent_request.keywords:
        yield
        return

but that requires poking around in one of pytest's private APIs.

CC: @RonnyPfannschmidt @nicoddemus – Any suggestions?

[pipermerriam]

We've had to divide our test suite up into separate runs because of this issue. It would be very nice to be able to run them side-by-side. I've opened #125 with an approach similar to how pytest-trio does it and am happy to polish that once I get some confirmation from maintainers they are 👍 with the general direction.

[njsmith]

@pipermerriam Oh cool, I hadn't seen that!

It might make sense to add in some sniffing logic like in my comment above? The nice thing about that is that in many cases it's backwards-compatible without users having to do anything, AND ALSO it fixes the pytest-trio breakage without users having to do anything. Without this kind of heuristic, then I think one case or the other requires user intervention.

[pipermerriam]

👍 I'll look into how I can combine the two to make for a better user experience. My goal was to set things out on a path that pointed in a direction where both play nicely by default but eventually having that mechanism be via an explicit fixture mark so that there's as little implicit inference as possible.

[Tinche]

I don't have a whole lot of time to dedicate to this, but I'd be happy to defer to @njsmith or @nicoddemus for any solutions and merge them in :)

[nicoddemus]

@njsmith this is easier in more modern pytest (3.6+ IIRC):

    asyncio_marks = list(request.node.iter_markers('asyncio'))
    if not asyncio_marks:
        yield
        return

This also covers your other example using _parent_request. 👍

[njsmith]

Here's another user who got bit by this: python-trio/pytest-trio#85

[florimondmanca]

This issue caught me recently as well — relevant discussions in agronholm/anyio#74 and agronholm/anyio#75 (comment).

If #125 needs a helping hand I’d be happy to provide some @pipermerriam. :)

[graingert]

@florimondmanca I'm getting burnt by this issue again so had a go at fixing the conflicts in #125 but didn't get very far