Skip to content

TYP: Update pyright #56892

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jan 16, 2024
Merged

TYP: Update pyright #56892

merged 4 commits into from
Jan 16, 2024

Conversation

twoertwein
Copy link
Member

@twoertwein twoertwein commented Jan 15, 2024

Newer releases of pyright infer return types of partially annotated functions, which uncovers quite a few difficult-to-fix type errors (some functions had no return types because it would create too many typing errors elsewhere).

(pyright can also try to infer return types for completely unannotated functions, but that is disabled for now in pyright_reportGeneralTypeIssues.json as it would trigger more difficult-to-fix cases.)

@@ -69,16 +69,26 @@ def fast_multiget(
mapping: dict,
keys: np.ndarray, # object[:]
default=...,
) -> np.ndarray: ...
) -> ArrayLike: ...
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe_convert_objects returns an ExtensionArray or an np.ndarray. Most functions call maybe_convert_objects only when convert=True and otherwise return an np.ndarray (see the overloads).

def unstack(self, level: IndexLabel = -1, fill_value=None, sort: bool = True):
def unstack(
self, level: IndexLabel = -1, fill_value=None, sort: bool = True
) -> DataFrame | Series:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

returns a Series when a DataFrame has a non-MultiIndex index (this is also documented; a series with a non-MultiIndex raises instead). Pyright's magic was able to infer that DataFrame | Series can be returned, unfortunately, this requires quite a few ignore codes.

@@ -500,13 +500,13 @@ def _minmax(self, meth: str):

return self.start + self.step * no_steps

def min(self, axis=None, skipna: bool = True, *args, **kwargs) -> int:
def min(self, axis=None, skipna: bool = True, *args, **kwargs) -> float:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit sad: _minmax can return np.nan which is a float

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we still annotate int | float here (and for _minmax). I think we had previously discussed wanting to be a bit more explicit with float capturing int

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed it!

@@ -456,7 +456,7 @@ def _str_lstrip(self, to_strip=None):
def _str_rstrip(self, to_strip=None):
return self._str_map(lambda x: x.rstrip(to_strip))

def _str_removeprefix(self, prefix: str) -> Series:
def _str_removeprefix(self, prefix: str):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was simply wrong

@@ -112,7 +114,8 @@ def _plot( # type: ignore[override]
*,
bins,
**kwds,
):
# might return a subset from the possible return types of Axes.hist(...)[2]?
) -> BarContainer | Polygon | list[BarContainer | Polygon]:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Matplotlib has type annotations and lists the above types as possible return types. I think it only returns BarContainer in test tests. As Polygon could be returned, I opted to reflect that in the signature as it could uncover runtime issues (but I'm not familiar enough to make further decisions)

@@ -637,7 +638,7 @@ def _str_map(
# error: Argument 1 to "dtype" has incompatible type
# "Union[ExtensionDtype, str, dtype[Any], Type[object]]"; expected
# "Type[object]"
dtype=np.dtype(dtype), # type: ignore[arg-type]
dtype=np.dtype(cast(type, dtype)),
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

map_infer_mask requires that dtype is np.dtype but np.dtype is a generic type: since the call to it seems invalid to mypy, it cannot infer the correct generic type of np.dtype which creates the existing type error. This PR uses overloads for map_infer_mask (convert=False returns np.ndarray): since the call to it seems invalid, mypy picks the last overload, which is not correct here - so I cast the argument for np.dtype to avoid this mess.

@twoertwein twoertwein added the Typing type annotations, mypy/pyright type checking label Jan 15, 2024
@twoertwein twoertwein marked this pull request as ready for review January 15, 2024 22:34
@twoertwein twoertwein requested a review from WillAyd as a code owner January 15, 2024 22:34
@mroeschke mroeschke added this to the 3.0 milestone Jan 16, 2024
@mroeschke mroeschke merged commit 0e11d6d into pandas-dev:main Jan 16, 2024
@mroeschke
Copy link
Member

Thanks @twoertwein

@twoertwein twoertwein deleted the pyright branch January 17, 2024 02:50
pmhatre1 pushed a commit to pmhatre1/pandas-pmhatre1 that referenced this pull request May 7, 2024
* TYP: Update pyright

* isort

* int | float
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Typing type annotations, mypy/pyright type checking
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants