BUG 1.3.3: pytest is failing when pytest-randomly pytest plugin is used

[kloczek]

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the master branch of pandas.

Reproducible Example

run pytest

Issue Description

pytest is failing with unrecognized arguments: --strict-data-files

+ /usr/bin/pytest -ra
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: --strict-data-files
  inifile: /home/tkloczko/rpmbuild/BUILD/pandas-1.3.3/pyproject.toml
  rootdir: /home/tkloczko/rpmbuild/BUILD/pandas-1.3.3

Expected Behavior

pytest should run test suite

Installed Versions

N/A

[kloczek]

[tkloczko@barrel SPECS]$ pytest --version
pytest 6.2.5

[debnathshoham]

I think you should run pytest pandas
Refer here for more details.

[kloczek]

With that hint it is a bit better but now I have 29 failed units.
Full log is in attachment.
python-pandas_pytest.log.txt
Just summary:

FAILED pandas/tests/generic/test_finalize.py::test_binops[rsub-args4-right] - AssertionError: assert {} == {'a': 1}
FAILED pandas/tests/extension/decimal/test_decimal.py::TestGetitem::test_getitem_boolean_array_mask - AssertionError
FAILED pandas/tests/extension/decimal/test_decimal.py::TestGetitem::test_getitem_mask - AssertionError
FAILED pandas/tests/extension/decimal/test_decimal.py::TestGetitem::test_take - AssertionError
FAILED pandas/tests/extension/decimal/test_decimal.py::TestGetitem::test_getitem_ellipsis_and_slice - AssertionError: ExtensionArray are different
FAILED pandas/tests/extension/decimal/test_decimal.py::TestGroupby::test_grouping_grouper - AssertionError: ExtensionArray are different
FAILED pandas/tests/extension/decimal/test_decimal.py::TestReshaping::test_set_frame_overwrite_object - AssertionError
FAILED pandas/tests/extension/decimal/test_decimal.py::TestReshaping::test_concat[False] - AssertionError
FAILED pandas/tests/extension/decimal/test_decimal.py::TestReshaping::test_concat[True] - AssertionError
FAILED pandas/tests/extension/decimal/test_decimal.py::TestSetitem::test_setitem_empty_indexer[True] - AssertionError: Attributes of Series are different
FAILED pandas/tests/extension/decimal/test_decimal.py::TestSetitem::test_setitem_empty_indexer[False] - AssertionError: ExtensionArray are different
FAILED pandas/tests/extension/decimal/test_decimal.py::TestMethods::test_equals[False-DataFrame] - AssertionError
FAILED pandas/tests/extension/decimal/test_decimal.py::TestMethods::test_factorize_equivalence[-2] - AssertionError
FAILED pandas/tests/extension/decimal/test_decimal.py::TestMethods::test_equals[False-Series] - AssertionError
FAILED pandas/tests/extension/decimal/test_decimal.py::TestMethods::test_equals[True-DataFrame] - AssertionError
FAILED pandas/tests/extension/decimal/test_decimal.py::TestMethods::test_factorize_equivalence[-1] - AssertionError
FAILED pandas/tests/extension/decimal/test_decimal.py::TestMethods::test_equals[True-Series] - AssertionError
FAILED pandas/tests/extension/decimal/test_decimal.py::TestConstructors::test_dataframe_constructor_from_dict[False] - AssertionError
FAILED pandas/tests/extension/decimal/test_decimal.py::TestConstructors::test_from_sequence_from_cls - AssertionError: ExtensionArray are different
FAILED pandas/tests/extension/decimal/test_decimal.py::TestConstructors::test_pandas_array - AssertionError: ExtensionArray are different
FAILED pandas/tests/extension/decimal/test_decimal.py::TestConstructors::test_empty - NotImplementedError: Default 'empty' implementation is invalid for dtype='decimal'
FAILED pandas/tests/extension/decimal/test_decimal.py::TestConstructors::test_dataframe_constructor_from_dict[True] - AssertionError
FAILED pandas/tests/extension/decimal/test_decimal.py::TestDtype::test_eq - AssertionError
FAILED pandas/tests/extension/decimal/test_decimal.py::TestDtype::test_eq_with_str - AssertionError
FAILED pandas/tests/io/test_clipboard.py::test_raw_roundtrip[\U0001f44d...] - pandas.io.clipboard.PyperclipException:
FAILED pandas/tests/io/test_clipboard.py::test_raw_roundtrip[abcd...] - pandas.io.clipboard.PyperclipException:
FAILED pandas/tests/io/test_clipboard.py::test_raw_roundtrip[\u03a9\u0153\u2211\xb4...] - pandas.io.clipboard.PyperclipException:
FAILED pandas/tests/arrays/test_array.py::test_array[data29-decimal-expected29] - AssertionError: ExtensionArray are different
FAILED pandas/tests/test_common.py::test_serializable[obj95] - _pickle.PicklingError: Can't pickle <function read_html at 0x7ff00d1f19d0>: it's not the same object as pa...
==================================== 29 failed, 164645 passed, 16174 skipped, 827 xfailed, 4 xpassed, 30 warnings in 1799.16s (0:29:59) ====================================

[MarcoGorelli]

These tests pass just fine

Please follow the contributing guide for how to set up your environment

To avoid wasting time like in matplotlib/matplotlib#20312, I'll quote their reply:

I've now wasted more than enough time on basically confirming how Python import semantics work, so unless you can come up with some concrete reason why something is wrong, I'm not going to bother any further.

[kloczek]

These tests pass just fine

Did you had a look on exect failing units messages?

Please follow the contributing guide for how to set up your environment

This ticket is not about contribution to the code.
This ticket is about fauiling test suite.

To avoid wasting time like in matplotlib/matplotlib#20312, I'll quote their reply:

I've now wasted more than enough time on basically confirming how Python import semantics work, so unless you can come up with some concrete reason why something is wrong, I'm not going to bother any further.

In that case it was wrong testing methodology usuusing python3 -Bm pytest. It should be used pytest script because python3 -Bm pytest automatically adds to the sys.path upfront current directory. Byu this pytest was not testing what was installed in </install/prefix> but in current directory.

If you will look one more time in the attached log you will see that in fseconfd line is /usr/bin/pytest -ra pandas.
Between those two cases there is no any connections.

May I ask you to provide list of installed modules wqith version in your test env?

[kloczek]

Just rerun pytest and had completly different set of fails.
I'm using pytest-randomly pytest plugin.
Looks like pandas teest suite has some dependencies between test units and that plugin exposes that kind of issue.
With such dependencies running pytest pandas <exac/file.py> or pytest pandas <exac/file.py>::><exact_unit> may fail.
So it would be rally good to reopen that ticket ..

On diagnosing that kind of issue can be used https://github.com/mrbean-bremen/pytest-find-dependencies/

[MarcoGorelli]

I'm using pytest-randomly pytest plugin.

changed the title BUG 1.3.3: pytest is failing with unrecognized arguments: --strict-data-files BUG 1.3.3: pytest is failing when pytest-randomly pytest plugin is used

OK, now this is a more specific and actionable report, thanks - let's open a PR which uses pytest-randomly and see what happens

For the future, if your setup is different to what's advised in the contributing guide, you should say so right away

[MarcoGorelli]

closing in favour of #34373

If there's an issue other than pytest-randomly which pandas should be aware of, please do open a new issue - thanks!