Skip to content

CI: linting check to ensure lib.NoDefault is only used for typing #53901

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Jun 29, 2023

Conversation

Charlie-XIAO
Copy link
Contributor

@Charlie-XIAO Charlie-XIAO commented Jun 28, 2023

In this PR, I added a pre-commit hook to check that pandas._libs.lib.NoDefault is used only for typing. This is invoked by #53877. Before that PR, this new hook correctly catches all misused NoDefault:

(pandas-dev) D:\>pre-commit run unwanted-patterns-nodefault-not-used-for-typing --all-files
Check for pandas._libs.lib.NoDefault not used for typing.......................Failed   
- hook id: unwanted-patterns-nodefault-not-used-for-typing
- exit code: 1

pandas/tests/reshape/test_pivot_multilevel.py:38:NoDefault is not used for typing       
pandas/tests/reshape/test_pivot_multilevel.py:75:NoDefault is not used for typing       
pandas/core/frame.py:8831:NoDefault is not used for typing
pandas/core/frame.py:8831:NoDefault is not used for typing
pandas/core/reshape/pivot.py:514:NoDefault is not used for typing
pandas/core/reshape/pivot.py:515:NoDefault is not used for typing
pandas/core/reshape/pivot.py:529:NoDefault is not used for typing
pandas/core/reshape/pivot.py:525:NoDefault is not used for typing
pandas/core/reshape/pivot.py:530:NoDefault is not used for typing
pandas/core/reshape/pivot.py:535:NoDefault is not used for typing
pandas/core/reshape/pivot.py:542:NoDefault is not used for typing
pandas/core/reshape/pivot.py:572:NoDefault is not used for typing

After that PR, the check passes normally:

(pandas-dev) D:\>pre-commit run unwanted-patterns-nodefault-not-used-for-typing --all-files
Check for pandas._libs.lib.NoDefault not used for typing.......................Passed

In fact I don't work with ASTs very often so not sure if this check covers all possibilties. Please let me know if any modification is needed. (The runtime is a bit long but seems acceptable, approximately 10s to check all files.)

@MarcoGorelli MarcoGorelli added Code Style Code style, linting, code_checks Typing type annotations, mypy/pyright type checking labels Jun 28, 2023
@MarcoGorelli MarcoGorelli added this to the 2.1 milestone Jun 28, 2023
@MarcoGorelli
Copy link
Member

nice idea

I think you're repeatedly walking up and down the same paths - do you want to check scripts/no_bool_in_generic.py? that one checks for annotations, maybe there's some logic there that can be reused?

@Charlie-XIAO
Copy link
Contributor Author

Charlie-XIAO commented Jun 28, 2023

Improved from approx. 10s for all files to approx. 4s @MarcoGorelli Thanks for your suggestion!

@MarcoGorelli
Copy link
Member

Well done!

Would appreciate it if we could add a tiny little test case to scripts/tests/test_validate_unwanted_patterns.py too

Other than that, looks good!

@Charlie-XIAO
Copy link
Contributor Author

Charlie-XIAO commented Jun 28, 2023

Didn't notice that there are tests for linting scripts, thanks for the reminder @MarcoGorelli. Added some simple tests now.

Copy link
Member

@MarcoGorelli MarcoGorelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice, great work, looks good to me on green

@mroeschke mroeschke merged commit 69acdb2 into pandas-dev:main Jun 29, 2023
@mroeschke
Copy link
Member

Nice thanks @Charlie-XIAO

Daquisu pushed a commit to Daquisu/pandas that referenced this pull request Jul 8, 2023
…andas-dev#53901)

* CI add liniting to check NoDefault used only for typing

* minor modification

* reduce cost

* tests added for the new linting check

* types_or -> types because only python

* rephrase pre-commit hook name

* rephrase more

* fix failing tests:

* retrigger checks

* retrigger checks
@Charlie-XIAO Charlie-XIAO deleted the lint-nodefault-only-typing branch September 23, 2023 13:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Code Style Code style, linting, code_checks Typing type annotations, mypy/pyright type checking
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants