CI: fix failing script/tests

[jbrockmendel]

• closes #xxxx
• tests added / passed
• Ensure all linting tests pass, see here for how to run them
• whatsnew entry

[MarcoGorelli]

Hi @jbrockmendel ,

bit confused here, don't these tests pass already? I just tried pull master and they seem fine:

$ cd scripts/
$ pytest
======================================== test session starts =========================================
platform linux -- Python 3.8.6, pytest-6.2.2, py-1.9.0, pluggy-0.13.1
rootdir: /home/marco/pandas-marco, configfile: setup.cfg
plugins: hypothesis-6.8.1, cov-2.11.1, instafail-0.4.1, monkeytype-1.1.0, forked-1.2.0, xdist-2.2.1, asyncio-0.14.0
collected 72 items                                                                                   

tests/test_inconsistent_namespace_check.py ........                                            [ 11%]
tests/test_no_bool_in_generic.py ....                                                          [ 16%]
tests/test_validate_docstrings.py .....................................                        [ 68%]
tests/test_validate_unwanted_patterns.py .......................                               [100%]

========================================= 72 passed in 1.38s =========================================

Also, tokenize-rt is a dependency of pyupgrade so I didn't think it would be necessary to include it in the environment file

---

EDIT

Running from the root directory also works:

$ pytest scripts/
======================================== test session starts =========================================
platform linux -- Python 3.8.6, pytest-6.2.2, py-1.9.0, pluggy-0.13.1
rootdir: /home/marco/pandas-marco, configfile: setup.cfg
plugins: hypothesis-6.8.1, cov-2.11.1, instafail-0.4.1, monkeytype-1.1.0, forked-1.2.0, xdist-2.2.1, asyncio-0.14.0
collected 72 items                                                                                   

scripts/tests/test_inconsistent_namespace_check.py ........                                    [ 11%]
scripts/tests/test_no_bool_in_generic.py ....                                                  [ 16%]
scripts/tests/test_validate_docstrings.py .....................................                [ 68%]
scripts/tests/test_validate_unwanted_patterns.py .......................                       [100%]

========================================= 72 passed in 1.37s =========================================

As for how to run them, I think the import validate_docstrings in test_validate_docstrings.py (which predates me) suggests they should be run from within scripts

Also, this is already checked here:

pandas/.github/workflows/ci.yml

Lines 67 to 69 in b172a60

	- name: Testing docstring validation script
	run: pytest --capture=no --strict-markers scripts
	if: always()

[jbrockmendel]

Also, tokenize-rt is a dependency of pyupgrade so I didn't think it would be necessary to include it in the environment file

Makes sense, will revert

[jbrockmendel]

bit confused here, don't these tests pass already? I just tried pull master and they seem fine:

what flake8 version are you using?

[jbrockmendel]

Running from the root directory also works:

$ pytest scripts/
______________________________________ ERROR collecting scripts/tests/test_inconsistent_namespace_check.py ______________________________________
ImportError while importing test module '~/pd/pandas/scripts/tests/test_inconsistent_namespace_check.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/local/Cellar/[email protected]/3.9.1_8/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
scripts/tests/test_inconsistent_namespace_check.py:3: in <module>
    from scripts.check_for_inconsistent_pandas_namespace import (
E   ModuleNotFoundError: No module named 'scripts'

$ cd scripts
$ pytest
______________________________________ ERROR collecting scripts/tests/test_inconsistent_namespace_check.py ______________________________________
ImportError while importing test module '~/pd/pandas/scripts/tests/test_inconsistent_namespace_check.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/local/Cellar/[email protected]/3.9.1_8/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_inconsistent_namespace_check.py:3: in <module>
    from scripts.check_for_inconsistent_pandas_namespace import (
E   ModuleNotFoundError: No module named 'scripts'

[jreback]

cool merge when ready @MarcoGorelli @jbrockmendel

[MarcoGorelli]

bit confused here, don't these tests pass already? I just tried pull master and they seem fine:

what flake8 version are you using?

Ah yeah, I had 3.8.4, hadn't done conda env update -f environment.yml...I thought we'd solved the issues with inconsistent linter versions, but this check uses the conda env version. IMO it makes sense to move it to pre-commit too so this doesn't happen again, will work on that later

Thanks for fixing this + explaining!

---

Just to clarify, it's not #40439 which broke the check, but the fact that flake8 isn't pinned in environment.yml, as that's the version used by this check

[simonjayhawkins]

@meeseeksdev backport 1.2.x