Skip to content

Commit 4b124de

Browse files
authored
DOC: Update xdist to provide more speed-up options and context (#48958) (#48535) (#48984)
1 parent 0106c26 commit 4b124de

File tree

1 file changed

+56
-10
lines changed

1 file changed

+56
-10
lines changed

doc/source/development/contributing_codebase.rst

+56-10
Original file line numberDiff line numberDiff line change
@@ -790,14 +790,11 @@ Or with one of the following constructs::
790790
pytest pandas/tests/[test-module].py::[TestClass]
791791
pytest pandas/tests/[test-module].py::[TestClass]::[test_method]
792792

793-
Using `pytest-xdist <https://pypi.org/project/pytest-xdist>`_, one can
794-
speed up local testing on multicore machines. To use this feature, you will
795-
need to install ``pytest-xdist`` via::
796-
797-
pip install pytest-xdist
798-
799-
The ``-n`` flag then can be specified when running ``pytest`` to parallelize a test run
800-
across the number of specified cores or ``auto`` to utilize all the available cores on your machine.
793+
Using `pytest-xdist <https://pypi.org/project/pytest-xdist>`_, which is
794+
included in our 'pandas-dev' environment, one can speed up local testing on
795+
multicore machines. The ``-n`` number flag then can be specified when running
796+
pytest to parallelize a test run across the number of specified cores or auto to
797+
utilize all the available cores on your machine.
801798

802799
.. code-block:: bash
803800
@@ -807,8 +804,57 @@ across the number of specified cores or ``auto`` to utilize all the available co
807804
# Utilizes all available cores
808805
pytest -n auto pandas
809806
810-
This can significantly reduce the time it takes to locally run tests before
811-
submitting a pull request.
807+
If you'd like to speed things along further a more advanced use of this
808+
command would look like this
809+
810+
.. code-block:: bash
811+
812+
pytest pandas -n 4 -m "not slow and not network and not db and not single_cpu" -r sxX
813+
814+
In addition to the multithreaded performance increase this improves test
815+
speed by skipping some tests using the ``-m`` mark flag:
816+
817+
- slow: any test taking long (think seconds rather than milliseconds)
818+
- network: tests requiring network connectivity
819+
- db: tests requiring a database (mysql or postgres)
820+
- single_cpu: tests that should run on a single cpu only
821+
822+
You might want to enable the following option if it's relevant for you:
823+
824+
- arm_slow: any test taking long on arm64 architecture
825+
826+
These markers are defined `in this toml file <https://github.com/pandas-dev/pandas/blob/main/pyproject.toml>`_
827+
, under ``[tool.pytest.ini_options]`` in a list called ``markers``, in case
828+
you want to check if new ones have been created which are of interest to you.
829+
830+
The ``-r`` report flag will display a short summary info (see `pytest
831+
documentation <https://docs.pytest.org/en/4.6.x/usage.html#detailed-summary-report>`_)
832+
. Here we are displaying the number of:
833+
834+
- s: skipped tests
835+
- x: xfailed tests
836+
- X: xpassed tests
837+
838+
The summary is optional and can be removed if you don't need the added
839+
information. Using the parallelization option can significantly reduce the
840+
time it takes to locally run tests before submitting a pull request.
841+
842+
If you require assistance with the results,
843+
which has happened in the past, please set a seed before running the command
844+
and opening a bug report, that way we can reproduce it. Here's an example
845+
for setting a seed on windows
846+
847+
.. code-block:: bash
848+
849+
set PYTHONHASHSEED=314159265
850+
pytest pandas -n 4 -m "not slow and not network and not db and not single_cpu" -r sxX
851+
852+
On Unix use
853+
854+
.. code-block:: bash
855+
856+
export PYTHONHASHSEED=314159265
857+
pytest pandas -n 4 -m "not slow and not network and not db and not single_cpu" -r sxX
812858
813859
For more, see the `pytest <https://docs.pytest.org/en/latest/>`_ documentation.
814860

0 commit comments

Comments
 (0)