Skip to content

Fix type annotation in pandas.compat.numpy.function #25961

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

vaibhavhrt
Copy link
Contributor

@vaibhavhrt vaibhavhrt commented Apr 2, 2019

Fix type errors as indicated by mypy + type sme other variables that cought my eye
@vaibhavhrt
Copy link
Contributor Author

@WillAyd I think this is my third open PR for typing. Should I wait for these to get reviewed before creating new issues and PR?

@WillAyd
Copy link
Member

WillAyd commented Apr 2, 2019 via email

@vaibhavhrt
Copy link
Contributor Author

Ok, I will wait for at least one of them to get merged before working on new issues.

@WillAyd WillAyd added the Typing type annotations, mypy/pyright type checking label Apr 2, 2019
@codecov
Copy link

codecov bot commented Apr 2, 2019

Codecov Report

Merging #25961 into master will decrease coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #25961      +/-   ##
==========================================
- Coverage   91.82%   91.81%   -0.01%     
==========================================
  Files         175      175              
  Lines       52581    52582       +1     
==========================================
- Hits        48280    48277       -3     
- Misses       4301     4305       +4
Flag Coverage Δ
#multiple 90.36% <100%> (ø) ⬆️
#single 41.89% <100%> (-0.08%) ⬇️
Impacted Files Coverage Δ
pandas/compat/numpy/function.py 87.97% <100%> (+0.06%) ⬆️
pandas/io/gbq.py 75% <0%> (-12.5%) ⬇️
pandas/core/frame.py 96.79% <0%> (-0.12%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 00c119c...b02dfaf. Read the comment docs.

@codecov
Copy link

codecov bot commented Apr 2, 2019

Codecov Report

Merging #25961 into master will decrease coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #25961      +/-   ##
==========================================
- Coverage   91.98%   91.98%   -0.01%     
==========================================
  Files         175      175              
  Lines       52375    52376       +1     
==========================================
- Hits        48179    48176       -3     
- Misses       4196     4200       +4
Flag Coverage Δ
#multiple 90.53% <100%> (ø) ⬆️
#single 40.7% <100%> (-0.15%) ⬇️
Impacted Files Coverage Δ
pandas/compat/numpy/function.py 87.97% <100%> (+0.06%) ⬆️
pandas/io/gbq.py 78.94% <0%> (-10.53%) ⬇️
pandas/core/frame.py 96.9% <0%> (-0.12%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 444ba64...a647e6f. Read the comment docs.

@gfyoung
Copy link
Member

gfyoung commented Apr 2, 2019

IMO, these type hints in this PR are a little over-fitted to this file. These default argument dictionaries are meant to map strings to anything that numpy throws at us from its signatures upstream. It would be more prudent to actually declare these types as OrderedDict[str, Any] to indicate this.

@WillAyd
Copy link
Member

WillAyd commented Apr 2, 2019

Yea was thinking the same thing, though I think maybe wrapping the containers in Final would be a good indication that these aren't meant to be modified:

https://mypy.readthedocs.io/en/latest/final_attrs.html?highlight=immutable#details-of-using-final

@gfyoung
Copy link
Member

gfyoung commented Apr 2, 2019

though I think maybe wrapping the containers in Final would be a good indication that these aren't meant to be modified

That's a fair proposition.

@gfyoung gfyoung closed this Apr 2, 2019
@gfyoung gfyoung reopened this Apr 2, 2019
@gfyoung
Copy link
Member

gfyoung commented Apr 2, 2019

My bad - fat fingers 😂

@vaibhavhrt
Copy link
Contributor Author

@WillAyd @gfyoung does this looks good:

CLIP_DEFAULTS = dict(out=None)  # type Final[Dict[str, Any]]

@gfyoung
Copy link
Member

gfyoung commented Apr 2, 2019

@vaibhavhrt : That looks good!

If you could make similar modifications to your other changes, that would be great.

@WillAyd
Copy link
Member

WillAyd commented Apr 2, 2019

@gfyoung think should convert dict to OrderedDict too? Not sure if a distinction between those is intentional so maybe make consistent?

@vaibhavhrt
Copy link
Contributor Author

@WillAyd Final needs to be imported from typing_extensions(https://pypi.org/project/typing-extensions/). But I don't think we have that in requirements. I usually use pipenv so I am not sure how to add a new package to requirements in this project. Would it be enough to add typing_extensions to requirements-dev.txt

@WillAyd
Copy link
Member

WillAyd commented Apr 2, 2019

Well that's disappointing...not sure we want to add a third party library just yet. Was that not added until 3.7

@vaibhavhrt
Copy link
Contributor Author

vaibhavhrt commented Apr 2, 2019

mypy pandas/compat/numpy/function.py

runs without any error but at from typing_extensions import Final my linter says unresolved import 'typing_extensions' Python(unresolved-import). I am guessing tests might fail.

@gfyoung
Copy link
Member

gfyoung commented Apr 2, 2019

think should convert dict to OrderedDict too? Not sure if a distinction between those is intentional so maybe make consistent?

@WillAyd : OrderedDict is used when there are multiple arguments (we need to maintain input order in the same way that numpy would define those arguments in its signature). For single-argument compatibility, dict or OrderedDict should fine, though the latter might have some slight overhead.

@WillAyd
Copy link
Member

WillAyd commented Apr 3, 2019

@jreback do you have any thoughts on adding typing_extensions as a dependency to pandas?

https://github.com/python/typing/tree/master/typing_extensions

On the plus side maintained by Guido and could help smooth over compat issues leading up to 3.7 (where typing is no longer provisional). On the downside would be another dependency

From there I think we could use Final, ChainMap, OrderedDict, Type, etc...

Side note - I know we've used Type in a few places already but reading more up on it that wasn't added until 3.5.2 so I think that might have broken the code base for 3.5 and 3.5.1

@jreback
Copy link
Contributor

jreback commented Apr 3, 2019

not a big deal to add typing_extensions

it’s pretty common now-a-days

let’s do that in a separate patch

@vaibhavhrt
Copy link
Contributor Author

Let me know when typing_extension has been added. @WillAyd if you want I can create an issue and PR to add typing_extensions to dependencies. But I am not sure how you are handling dependencies so so I might need some help.

@WillAyd
Copy link
Member

WillAyd commented Apr 3, 2019

Sure give it a go! You should be able to see the PR to add Mypy which you can use for reference

@jbrockmendel
Copy link
Member

@vaibhavhrt can you merge master

@vaibhavhrt
Copy link
Contributor Author

@jbrockmendel ok I will merge master. I have few more commits to push in this PR. But before I do that #25975 needs to be merged. The dependency typing_extensions is needed for this PR. But that PR is having some problems with CI configuration. Can you take a look and tell me how to fix that.

@jreback
Copy link
Contributor

jreback commented Apr 16, 2019

@vaibhavhrt we won't be merging that PR mentioned #25961 (comment) as is, see the comments

@vaibhavhrt
Copy link
Contributor Author

vaibhavhrt commented Apr 16, 2019 via email

Copy link
Member

@WillAyd WillAyd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's move forward without Final - can you merge master and address comments?

@WillAyd
Copy link
Member

WillAyd commented Apr 24, 2019 via email

@pep8speaks
Copy link

pep8speaks commented Apr 24, 2019

Hello @vaibhavhrt! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2019-04-25 06:44:26 UTC

@vaibhavhrt
Copy link
Contributor Author

@WillAyd two tests are failing, but only thing I did was add comments, how can it make tests fail.

Copy link
Member

@WillAyd WillAyd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. If you merge master one more time should resolve CI failures

@vaibhavhrt
Copy link
Contributor Author

@jreback I have fixed everything, you can review and merge it.

@jreback jreback added this to the 0.25.0 milestone Apr 26, 2019
@jreback jreback merged commit 77855a1 into pandas-dev:master Apr 26, 2019
@jreback
Copy link
Contributor

jreback commented Apr 26, 2019

thanks!

@vaibhavhrt vaibhavhrt deleted the Fix-Type-Annotation-pandas.compat.numpy.function branch April 26, 2019 09:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Typing type annotations, mypy/pyright type checking
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fix Type Annotations in pandas/compat/numpy/function.py
6 participants