Typing: Support New Mypy Semantic Analyzer #27070

alimcmaster1 · 2019-06-26T23:44:49Z

closes Support MyPy New Semantic Analyzer #27018

WillAyd · 2019-06-26T23:54:08Z

Does this cause Mypy failures? Wasn’t think we’d add this as a setting as much as just fix the errors that using it would cause

alimcmaster1 · 2019-06-26T23:57:33Z

^ I'm just checking that it does cause failures. This way we can verify i've actually fixed the issues

codecov · 2019-06-27T00:18:48Z

Codecov Report

Merging #27070 into master will decrease coverage by <.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #27070      +/-   ##
==========================================
- Coverage   92.04%   92.03%   -0.01%     
==========================================
  Files         180      180              
  Lines       50714    50714              
==========================================
- Hits        46680    46675       -5     
- Misses       4034     4039       +5

Flag	Coverage Δ
#multiple	`90.67% <ø> (ø)`	⬆️
#single	`41.86% <ø> (-0.11%)`	⬇️

Impacted Files	Coverage Δ
pandas/io/gbq.py	`88.88% <0%> (-11.12%)`	⬇️
pandas/core/frame.py	`96.89% <0%> (-0.12%)`	⬇️
pandas/util/testing.py	`90.84% <0%> (-0.11%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d94146c...c098d1b. Read the comment docs.

codecov · 2019-06-27T00:18:55Z

Codecov Report

Merging #27070 into master will increase coverage by 0.16%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #27070      +/-   ##
==========================================
+ Coverage   91.87%   92.03%   +0.16%     
==========================================
  Files         180      180              
  Lines       50834    50714     -120     
==========================================
- Hits        46703    46675      -28     
+ Misses       4131     4039      -92

Flag	Coverage Δ
#multiple	`90.67% <ø> (+0.16%)`	⬆️
#single	`41.86% <ø> (-0.12%)`	⬇️

Impacted Files	Coverage Δ
pandas/io/gbq.py	`88.88% <0%> (-11.12%)`	⬇️
pandas/io/excel/_openpyxl.py	`84.71% <0%> (-3.23%)`	⬇️
pandas/core/arrays/integer.py	`96.3% <0%> (-1.32%)`	⬇️
pandas/core/internals/construction.py	`95.95% <0%> (-0.8%)`	⬇️
pandas/core/dtypes/cast.py	`90.72% <0%> (-0.34%)`	⬇️
pandas/core/arrays/sparse.py	`94.19% <0%> (-0.31%)`	⬇️
pandas/core/config_init.py	`95.8% <0%> (-0.28%)`	⬇️
pandas/core/indexes/base.py	`96.92% <0%> (-0.21%)`	⬇️
pandas/core/arrays/timedeltas.py	`88.82% <0%> (-0.2%)`	⬇️
pandas/core/groupby/groupby.py	`97.17% <0%> (-0.16%)`	⬇️
... and 40 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a173cac...04b49ed. Read the comment docs.

WillAyd · 2019-06-27T02:11:12Z

Gotcha. Just FYI you can also run mypy --new-semantic-analyzer pandas locally to see

alimcmaster1 · 2019-06-30T16:03:37Z

first two errors removed, unsure of how to deal with the mypy error

pandas/io/json/__init__.py:5: error: Name 'json' is not defined

Due to the fact we have: ( in our init.py)

del json, normalize, table_schema  # noqa

Any advice @WillAyd ?

WillAyd · 2019-06-30T16:07:40Z

Does converting the imports to abolute help at all? Seems like kind of a strange init file - maybe modeling it after pandas.io.excel would help and allow for removal of noqa

pandas/io/json/__init__.py

pandas/io/json/_json.py

pep8speaks · 2019-06-30T19:41:41Z

Hello @alimcmaster1! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2019-07-07 20:44:20 UTC

WillAyd · 2019-06-30T19:47:47Z

Just remove the noqa - shouldn't need them

alimcmaster1 · 2019-06-30T20:45:41Z

Latest test failures seem unrelated: ( currently looking )

pandas\tests\indexes\multi\test_format.py:14: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <contextlib._GeneratorContextManager object at 0x000002299F625860>
type = None, value = None, traceback = None

    def __exit__(self, type, value, traceback):
        if type is None:
            try:
>               next(self.gen)
E               AssertionError: Did not see expected warning of class 'FutureWarning'.

WillAyd · 2019-07-01T16:48:57Z

Code changes look good. Can you remove changes to mypy.ini and any requirements files?

@jreback any objection to renaming of files in json folder?

jreback · 2019-07-01T16:56:10Z

no objections

alimcmaster1 · 2019-07-01T20:25:43Z

Thanks for the review @WillAyd .

Just for my understanding what’s your motivation for removing the changes to mypy.ini it’s whats validating the mypy semantic analysis in CI - to avoid anyone introducing further errors?

Thanks

WillAyd · 2019-07-01T20:38:39Z

Just for my understanding what’s your motivation for removing the changes to mypy.ini it’s whats validating the mypy semantic analysis in CI - to avoid anyone introducing further errors?

Because we don't need to really pin a dependency here. Any of these changes are backwards compatible we are just preventing future mypy versions from breaking; no need to force a min version

jorisvandenbossche · 2019-07-03T02:47:32Z

pandas/io/json/__init__.py

+    read_json,
+    to_json,
+    json_normalize,
+    build_table_schema


those should be strings I think?

…mcmali-mypy

WillAyd

lgtm - over to you @jorisvandenbossche

jreback · 2019-07-05T18:48:00Z

@alimcmaster1 isn't the commit adding the semantic analyzer removed? IOW this doesn't look like it is triggering it.

pandas/core/arrays/interval.py

pandas/core/indexes/timedeltas.py

WillAyd · 2019-07-05T18:50:32Z

I asked to remove the semantic analyzer from the config because it naturally becomes the default in the next Mypy release. Figured no need to set then and not setting maintains better backwards compat

jreback · 2019-07-05T18:51:37Z

I asked to remove the semantic analyzer from the config because it naturally becomes the default in the next Mypy release. Figured no need to set then and not setting maintains better backwards compat

ok, then we should set a min on mypy (when this comes out)? or at least on our CI do that?

WillAyd · 2019-07-05T18:57:45Z

We don’t have to. The new analyzer is more strict so anything done here works with or without it. But yea if we wanted to play it safe could set min version to next release when it comes out

…

Sent from my iPhone

On Jul 5, 2019, at 11:51 AM, Jeff Reback ***@***.***> wrote: I asked to remove the semantic analyzer from the config because it naturally becomes the default in the next Mypy release. Figured no need to set then and not setting maintains better backwards compat ok, then we should set a min on mypy (when this comes out)? or at least on our CI do that? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

pandas/io/json/_json.py

alimcmaster1 · 2019-07-07T21:37:25Z

Updated as per your comments and green @jreback

jreback · 2019-07-08T01:36:09Z

thanks @alimcmaster1

jreback added the Typing type annotations, mypy/pyright type checking label Jun 27, 2019

jreback added this to the 0.25.0 milestone Jun 27, 2019

alimcmaster1 marked this pull request as ready for review June 30, 2019 16:03

WillAyd requested changes Jun 30, 2019

View reviewed changes

pandas/io/json/__init__.py Show resolved Hide resolved

pandas/io/json/_json.py Outdated Show resolved Hide resolved

alimcmaster1 added 6 commits June 30, 2019 22:11

Add semantic analyser

f64719c

Remove already def errors

1c8ceda

Convert __init__ structure to match io.excel

0dc57ef

Add back noqa and add __all__

7e1681e

Add back noqa and add __all__

a961e65

Add back noqa and add __all__

0cfc9f2

alimcmaster1 and others added 2 commits July 2, 2019 22:41

Remove mypy version requirements

29acba4

Merge branch 'master' into PR_TOOL_MERGE_PR_27070

d5f5845

jorisvandenbossche reviewed Jul 3, 2019

View reviewed changes

jorisvandenbossche changed the title ~~Add semantic analyser~~ Typing: Add mypy semantic analyser Jul 3, 2019

jreback removed this from the 0.25.0 milestone Jul 3, 2019

Merge branch 'master' of https://github.com/alimcmaster1/pandas into …

a29e245

…mcmali-mypy

alimcmaster1 added 3 commits July 5, 2019 01:22

__all__ as strings

6ae4040

black formatting

3a744b4

fix import

917135c

WillAyd changed the title ~~Typing: Add mypy semantic analyser~~ Typing: Support New Mypy Semantic Analyzer Jul 5, 2019

WillAyd approved these changes Jul 5, 2019

View reviewed changes

jreback requested changes Jul 5, 2019

View reviewed changes

pandas/core/arrays/interval.py Show resolved Hide resolved

pandas/core/indexes/timedeltas.py Show resolved Hide resolved

jreback approved these changes Jul 5, 2019

View reviewed changes

jreback added this to the 0.25.0 milestone Jul 5, 2019

jreback requested changes Jul 6, 2019

View reviewed changes

pandas/io/json/_json.py Outdated Show resolved Hide resolved

Rename method

0b34836

jreback approved these changes Jul 8, 2019

View reviewed changes

jreback merged commit 845d4b8 into pandas-dev:master Jul 8, 2019

alimcmaster1 deleted the mcmali-mypy branch December 25, 2019 20:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Typing: Support New Mypy Semantic Analyzer #27070

Typing: Support New Mypy Semantic Analyzer #27070

alimcmaster1 commented Jun 26, 2019

WillAyd commented Jun 26, 2019

alimcmaster1 commented Jun 26, 2019

codecov bot commented Jun 27, 2019

codecov bot commented Jun 27, 2019 •

edited

Loading

WillAyd commented Jun 27, 2019

alimcmaster1 commented Jun 30, 2019

WillAyd commented Jun 30, 2019

pep8speaks commented Jun 30, 2019 •

edited

Loading

WillAyd commented Jun 30, 2019

alimcmaster1 commented Jun 30, 2019 •

edited

Loading

WillAyd commented Jul 1, 2019

jreback commented Jul 1, 2019

alimcmaster1 commented Jul 1, 2019

WillAyd commented Jul 1, 2019

jorisvandenbossche Jul 3, 2019

alimcmaster1 Jul 5, 2019

WillAyd left a comment

jreback commented Jul 5, 2019

WillAyd commented Jul 5, 2019

jreback commented Jul 5, 2019

WillAyd commented Jul 5, 2019 via email

alimcmaster1 commented Jul 7, 2019

jreback commented Jul 8, 2019

Typing: Support New Mypy Semantic Analyzer #27070

Typing: Support New Mypy Semantic Analyzer #27070

Conversation

alimcmaster1 commented Jun 26, 2019

WillAyd commented Jun 26, 2019

alimcmaster1 commented Jun 26, 2019

codecov bot commented Jun 27, 2019

Codecov Report

codecov bot commented Jun 27, 2019 • edited Loading

Codecov Report

WillAyd commented Jun 27, 2019

alimcmaster1 commented Jun 30, 2019

WillAyd commented Jun 30, 2019

pep8speaks commented Jun 30, 2019 • edited Loading

Comment last updated at 2019-07-07 20:44:20 UTC

WillAyd commented Jun 30, 2019

alimcmaster1 commented Jun 30, 2019 • edited Loading

WillAyd commented Jul 1, 2019

jreback commented Jul 1, 2019

alimcmaster1 commented Jul 1, 2019

WillAyd commented Jul 1, 2019

jorisvandenbossche Jul 3, 2019

Choose a reason for hiding this comment

alimcmaster1 Jul 5, 2019

Choose a reason for hiding this comment

WillAyd left a comment

Choose a reason for hiding this comment

jreback commented Jul 5, 2019

WillAyd commented Jul 5, 2019

jreback commented Jul 5, 2019

WillAyd commented Jul 5, 2019 via email

alimcmaster1 commented Jul 7, 2019

jreback commented Jul 8, 2019

codecov bot commented Jun 27, 2019 •

edited

Loading

pep8speaks commented Jun 30, 2019 •

edited

Loading

alimcmaster1 commented Jun 30, 2019 •

edited

Loading