Skip to content

STYLE make black local hook run twice as fast #49947

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Nov 29, 2022

Conversation

MarcoGorelli
Copy link
Member

@MarcoGorelli MarcoGorelli commented Nov 28, 2022

  • closes #xxxx (Replace xxxx with the GitHub issue number)
  • Tests added and passed if fixing a bug or adding a new feature
  • All code checks passed.
  • Added type annotations to new arguments/methods/functions.
  • Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

Since https://ichard26.github.io/blog/2022/05/compiling-black-with-mypyc-part-1/ , black runs twice as fast when the wheels are installed, compared with when it's built from source in the pre-commit hook


EDIT: CI timings show 53.84s vs 131.85s, so this is more than a 2x improvement

@@ -36,6 +36,8 @@ jobs:

- name: Run pre-commit
uses: pre-commit/[email protected]
with:
extra_args: --verbose --all-files
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

verbose just shows the timings for each hook. --all-files is the default extra_args

@@ -85,7 +85,7 @@ dependencies:
- cxx-compiler

# code checks
- black=22.3.0
- black=22.10.0
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at some point I'll address #46561

@MarcoGorelli MarcoGorelli added the Code Style Code style, linting, code_checks label Nov 28, 2022
@jbrockmendel
Copy link
Member

Just checking: this doesn’t assume anything about my dev encironment?

@MarcoGorelli
Copy link
Member Author

Just checking: this doesn’t assume anything about my dev encironment?

that's right!

@akx
Copy link
Contributor

akx commented Nov 29, 2022

This could maybe also be implemented by adding black==22.10.0 to the additional_dependencies of the nonlocal black hook, so precommit would also install the mypyced module...

@MarcoGorelli
Copy link
Member Author

This could maybe also be implemented by adding black==22.10.0 to the additional_dependencies of the nonlocal black hook, so precommit would also install the mypyced module...

that doesn't work unfortunately, I get

[INFO] Installing environment for https://github.com/python/black.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
An unexpected error has occurred: CalledProcessError: command: ('/home/marcogorelli/.cache/pre-commit/repom_n0drrc/py_env-python3.8/bin/python', '-mpip', 'install', '.', 'black==22.10.0')
return code: 1
expected return code: 0
stdout:
    Processing /home/marcogorelli/.cache/pre-commit/repom_n0drrc
      Installing build dependencies: started
      Installing build dependencies: finished with status 'done'
      Getting requirements to build wheel: started
      Getting requirements to build wheel: finished with status 'done'
      Preparing metadata (pyproject.toml): started
      Preparing metadata (pyproject.toml): finished with status 'done'
    
    The conflict is caused by:
        The user requested black 0.1.dev1+g27d2014 (from /home/marcogorelli/.cache/pre-commit/repom_n0drrc)
        The user requested black==22.10.0
    
    To fix this you could try to:
    1. loosen the range of package versions you've specified
    2. remove package versions to allow pip attempt to solve the dependency conflict
    
    
stderr:
    ERROR: Cannot install black 0.1.dev1+g27d2014 (from /home/marcogorelli/.cache/pre-commit/repom_n0drrc) and black==22.10.0 because these package versions have conflicting dependencies.
    ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
    
Check the log at /home/marcogorelli/.cache/pre-commit/pre-commit.log

@akx
Copy link
Contributor

akx commented Nov 29, 2022

@MarcoGorelli Ah, I guess black uses setuptools_scm or something so non-tagged versions are always dev+stuff (which makes sense). Thanks for giving it a shot!

@mroeschke mroeschke added this to the 2.0 milestone Nov 29, 2022
@mroeschke mroeschke merged commit 32b4222 into pandas-dev:main Nov 29, 2022
@mroeschke
Copy link
Member

Thanks @MarcoGorelli

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Code Style Code style, linting, code_checks
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants