TYPING: Enable --check-untyped-defs for MyPy

[simonjayhawkins]

• closes Enable --check-untyped-defs for MyPy #27568

I'll open a new issue for resolution of mypy errors per module, see https://gitter.im/pydata/pandas?at=5dc5cdf2091dd14a0e86103a and #28926 for process used for test modules.

cc @WillAyd @jbrockmendel

[jbrockmendel]

i have no objection, but am too ignorant on this topic to give a "lgtm"

[WillAyd]

On board with the change. Only consideration point is if we want this much customization in setup.cfg or if we should reinstate mypy.ini . Not sure it really matters but cc @jreback

[jbrockmendel]

i like having setup.cfg as a one-stop-shop. IIUC the idea is that the stuff added here will be whittled down over time anyway

[simonjayhawkins]

we had a brief discussion on this in #28914, the result #28914 (comment)

changing this now would disrupt open PRs for #28926, so would probably be better to put this on hold and clear them first.

[simonjayhawkins]

I'll close this until a decision on #29493 (comment) is reached. Too much of a lag between generating the list and merging could cause CI failures.

[simonjayhawkins]

@jreback DO NOT MERGE if any PRs adding type annotations have been added since 2ec7f2f. also once this is merged do not merge any PRs adding type annotations unless they have merged this patch.

[simonjayhawkins]

list will be different for mypy 0.730, so closing til #29653 is merged.

[xhochy]

@simonjayhawkins This can be reopened/reviewed/merged as #29653 is merged now?

[simonjayhawkins]

@xhochy I'll reopen it later today for discussion. few things to consider:

mypy 0.750 is now released so may want to bump version on ci to stay current

check_untyped_defs with 0.740+ gives many more errors to resolve so may want to stick with 0.730 for a while. (from the blog http://mypy-lang.blogspot.com/ The self argument to methods without annotations is now correctly given the object’s type (instead of Any) when using check_untyped_defs.)

also need to consider the process to use for bumping mypy on ci once check_untyped_defs is enabled. Simplest solution would be just to update blacklist to exclude new failures and address in seperate PRs and accept than the coverage of check_untyped_defs takes a temporary hit on each mypy bump.

[xhochy]

also need to consider the process to use for bumping mypy on ci once check_untyped_defs is enabled. Simplest solution would be just to update blacklist to exclude new failures and address in seperate PRs and accept than the coverage of check_untyped_defs takes a temporary hit on each mypy bump.

I would expect that long-term the number of new failures with mypy updates will decrease. We will have probably more typing in pandas, i.e. less untyped defs and mypy will also mature more and more where each release will bring in less change and thus far fewer new errors. Still, for roughly the next 6-12 months I would expect that we will have to go with the blacklist-new-failure-and-fix-in-followup-PRs way.

[WillAyd]

@simonjayhawkins I am fine with this so if no other feedback let's move forward - does the list of exclusions need refreshing?

[simonjayhawkins]

OK, i'll give it 24hours and if no objections, i'll refresh the list and merge on green.

[TomAugspurger]

+1