diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 0fc6e61049a44..fc08a1b164caf 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -127,6 +127,12 @@ repos: types: [python] files: ^pandas/tests/ exclude: ^pandas/tests/extension/ + - id: unwanted-patters-pytest-xfail + name: Check for use of pytest.xfail + entry: pytest\.xfail + language: pygrep + types: [python] + files: ^pandas/tests/ - id: inconsistent-namespace-usage name: 'Check for inconsistent use of pandas namespace in tests' entry: python scripts/check_for_inconsistent_pandas_namespace.py diff --git a/doc/source/development/code_style.rst b/doc/source/development/code_style.rst index 9477a9ac79dd6..19d83eb75e5bd 100644 --- a/doc/source/development/code_style.rst +++ b/doc/source/development/code_style.rst @@ -161,6 +161,46 @@ For example: # wrong from common import test_base +Testing +======= + +Failing tests +-------------- + +See https://docs.pytest.org/en/latest/skipping.html for background. + +Do not use ``pytest.xfail`` +--------------------------- + +Do not use this method. It has the same behavior as ``pytest.skip``, namely +it immediately stops the test and does not check if the test will fail. If +this is the behavior you desire, use ``pytest.skip`` instead. + +Using ``pytest.mark.xfail`` +--------------------------- + +Use this method if a test is known to fail but the manner in which it fails +is not meant to be captured. It is common to use this method for a test that +exhibits buggy behavior or a non-implemented feature. If +the failing test has flaky behavior, use the argument ``strict=False``. This +will make it so pytest does not fail if the test happens to pass. + +Prefer the decorator ``@pytest.mark.xfail`` and the argument ``pytest.param`` +over usage within a test so that the test is appropriately marked during the +collection phase of pytest. For xfailing a test that involves multiple +parameters, a fixture, or a combination of these, it is only possible to +xfail during the testing phase. To do so, use the ``request`` fixture: + +.. code-block:: python + + import pytest + + def test_xfail(request): + request.node.add_marker(pytest.mark.xfail(reason="Indicate why here")) + +xfail is not to be used for tests involving failure due to invalid user arguments. +For these tests, we need to verify the correct exception type and error message +is being raised, using ``pytest.raises`` instead. Miscellaneous =============