Skip to content

TYP/CI: bump mypy&pyright #46905

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
May 15, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/code-checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ jobs:

- name: Install pyright
# note: keep version in sync with .pre-commit-config.yaml
run: npm install -g [email protected].245
run: npm install -g [email protected].247

- name: Build Pandas
id: build
Expand Down
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ repos:
types: [python]
stages: [manual]
# note: keep version in sync with .github/workflows/code-checks.yml
additional_dependencies: ['[email protected].245']
additional_dependencies: ['[email protected].247']
- repo: local
hooks:
- id: flake8-rst
Expand Down
2 changes: 1 addition & 1 deletion doc/source/whatsnew/v1.5.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -360,7 +360,7 @@ If installed, we now require:
+-----------------+-----------------+----------+---------+
| Package | Minimum Version | Required | Changed |
+=================+=================+==========+=========+
| mypy (dev) | 0.941 | | X |
| mypy (dev) | 0.950 | | X |
+-----------------+-----------------+----------+---------+


Expand Down
2 changes: 1 addition & 1 deletion environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ dependencies:
- flake8-bugbear=21.3.2 # used by flake8, find likely bugs
- flake8-comprehensions=3.7.0 # used by flake8, linting of unnecessary comprehensions
- isort>=5.2.1 # check that imports are in the right order
- mypy=0.941
- mypy=0.950
- pre-commit>=2.9.2
- pycodestyle # used by flake8
- pyupgrade
Expand Down
4 changes: 3 additions & 1 deletion pandas/_config/localization.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,9 @@ def set_locale(
locale.setlocale(lc_var, new_locale)
normalized_locale = locale.getlocale()
if all(x is not None for x in normalized_locale):
yield ".".join(normalized_locale)
# error: Argument 1 to "join" of "str" has incompatible type
# "Tuple[Optional[str], Optional[str]]"; expected "Iterable[str]"
yield ".".join(normalized_locale) # type: ignore[arg-type]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mypy doesn't seem to understand all(...) to narrow the type.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this works, but let's not do this yet! (until we can use it elsewhere)

        def all_not_none(arg: Iterable[str | None]) -> TypeGuard[Iterable[str]]:
            return all(x is not None for x in arg)

        if all_not_none(normalized_locale):

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking through the code there are a couple of other places this pattern is used that do not use all_not_none from pandas/core/common.py

maybe change this one here, then in the future we only need the TypeGuard in one place.

    try:
        import pandas.core.common as com

        locale.setlocale(lc_var, new_locale)
        normalized_locale = locale.getlocale()
        if com.all_not_none(*normalized_locale):

note need to prevent circular import.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have there been discussions about using typing_extensions? Then we could already use TypeGuard.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no we don't use typing_extensions but we can import from typing in pandas._typing with a guard so that it resolves to Any in older python versions.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have there been discussions about using typing_extensions?

As long as typing_extensions is not a run-time dependency, I would not be adverse to it being added as a dev dependency.

else:
yield new_locale
finally:
Expand Down
4 changes: 2 additions & 2 deletions pandas/_libs/tslibs/timestamps.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ class Timestamp(datetime):
def time(self) -> _time: ...
def timetz(self) -> _time: ...
def replace(
self,
self: _DatetimeT,
year: int = ...,
month: int = ...,
day: int = ...,
Expand All @@ -123,7 +123,7 @@ class Timestamp(datetime):
microsecond: int = ...,
tzinfo: _tzinfo | None = ...,
fold: int = ...,
) -> datetime: ...
) -> _DatetimeT: ...
def astimezone(self: _DatetimeT, tz: _tzinfo | None = ...) -> _DatetimeT: ...
def ctime(self) -> str: ...
def isoformat(self, sep: str = ..., timespec: str = ...) -> str: ...
Expand Down
2 changes: 1 addition & 1 deletion pandas/core/arrays/masked.py
Original file line number Diff line number Diff line change
Expand Up @@ -957,7 +957,7 @@ def equals(self, other) -> bool:
return array_equivalent(left, right, dtype_equal=True)

def _quantile(
self: BaseMaskedArrayT, qs: npt.NDArray[np.float64], interpolation: str
self, qs: npt.NDArray[np.float64], interpolation: str
) -> BaseMaskedArray:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pyright didn't previously find this. I assume the return value can be any BaseMaskedArray.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep. maybe add type annotations to _maybe_mask_result also.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can be follow-on. I suspect that _maybe_mask_result should probably return the same type, but since we use explicit Subclass constructors this would not be the case if we subclass further.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will leave this out - it would require quite a few ignore comments if we do something like:

    @overload
    def _maybe_mask_result(self, result: ArrayLike, mask: ArrayLike) -> ArrayLike:
        ...

    @overload
    def _maybe_mask_result(
        self, result: tuple[ArrayLike, ArrayLike], mask: ArrayLike
    ) -> tuple[ArrayLike, ArrayLike]:
        ...

    def _maybe_mask_result(
        self, result: ArrayLike | tuple[ArrayLike, ArrayLike], mask: ArrayLike
    ) -> ArrayLike | tuple[ArrayLike, ArrayLike]:

For example FloatingArray and BooleanArray require an ndarray not an ExtensionArray.

"""
Dispatch to quantile_with_mask, needed because we do not have
Expand Down
25 changes: 8 additions & 17 deletions pandas/io/parsers/base_parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
from enum import Enum
import itertools
from typing import (
Any,
Callable,
DefaultDict,
Hashable,
Expand Down Expand Up @@ -1027,26 +1028,14 @@ def _get_empty_meta(
# Convert `dtype` to a defaultdict of some kind.
# This will enable us to write `dtype[col_name]`
# without worrying about KeyError issues later on.
dtype_dict: defaultdict[Hashable, Any]
if not is_dict_like(dtype):
# if dtype == None, default will be object.
default_dtype = dtype or object
# error: Argument 1 to "defaultdict" has incompatible type "Callable[[],
# Union[ExtensionDtype, str, dtype[Any], Type[object], Dict[Hashable,
# Union[ExtensionDtype, Union[str, dtype[Any]], Type[str], Type[float],
# Type[int], Type[complex], Type[bool], Type[object]]]]]"; expected
# "Optional[Callable[[], Union[ExtensionDtype, str, dtype[Any],
# Type[object]]]]"
# error: Incompatible return value type (got "Union[ExtensionDtype, str,
# dtype[Any], Type[object], Dict[Hashable, Union[ExtensionDtype, Union[str,
# dtype[Any]], Type[str], Type[float], Type[int], Type[complex], Type[bool],
# Type[object]]]]", expected "Union[ExtensionDtype, str, dtype[Any],
# Type[object]]")
dtype = defaultdict(
lambda: default_dtype # type: ignore[arg-type, return-value]
)
dtype_dict = defaultdict(lambda: default_dtype)
else:
dtype = cast(dict, dtype)
dtype = defaultdict(
dtype_dict = defaultdict(
lambda: object,
{columns[k] if is_integer(k) else k: v for k, v in dtype.items()},
)
Expand All @@ -1063,14 +1052,16 @@ def _get_empty_meta(
if (index_col is None or index_col is False) or index_names is None:
index = Index([])
else:
data = [Series([], dtype=dtype[name]) for name in index_names]
data = [Series([], dtype=dtype_dict[name]) for name in index_names]
index = ensure_index_from_sequences(data, names=index_names)
index_col.sort()

for i, n in enumerate(index_col):
columns.pop(n - i)

col_dict = {col_name: Series([], dtype=dtype[col_name]) for col_name in columns}
col_dict = {
col_name: Series([], dtype=dtype_dict[col_name]) for col_name in columns
}

return index, columns, col_dict

Expand Down
2 changes: 1 addition & 1 deletion requirements-dev.txt
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ flake8==4.0.1
flake8-bugbear==21.3.2
flake8-comprehensions==3.7.0
isort>=5.2.1
mypy==0.941
mypy==0.950
pre-commit>=2.9.2
pycodestyle
pyupgrade
Expand Down