Skip to content

887 select_dtypes stubs fixing #900

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 2, 2024
Merged

Conversation

mutricyl
Copy link
Contributor

@mutricyl mutricyl commented Apr 2, 2024

I tried include with overloads limitations stated in select_dtypes documentation:

ValueError

  • If both of include and exclude are empty
  • If include and exclude have overlapping elements
  • If any kind of string dtype is passed in.

I managed to cover somehow the first tick covering when neither of include or exclude is provided but I am not sure that it is worth the introduced complexity in stubs.

Copy link
Collaborator

@Dr-Irv Dr-Irv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, this looks good. I have ideas for you to try to catch the case of passing str or str as the dtype, as well as the 2 empty list case, so see if that works.

I don't think that overlapping lists can be detected with static typing.

"datetime64[ns]",
]
)
AstypeArgExtList: TypeAlias = AstypeArgExt | list[AstypeArgExt]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if you added the following here as the first 2 overloads, you could catch the case of the string arguments not being allowed:

@overload
def select_dtypes(self, include: StrDtypeArg, exclude: AstypeArgExtList | None = ...) -> Never: ...
@overload
def select_dtypes(self, include: AstypeArgExtList | None = ..., exclude: StrDtypeArg) -> Never: ...

And if you added this, I think that you'd catch the case of 2 empty lists:

@overload
def select_dtypes(self, include: list[Never], exclude: list[Never]) -> Never: ...

You import Never from typing_extensions

There's a possibility that either mypy and/or pyright will complain about overlapping overloads, but you can add #type: ignore and # pyright: ignore as needed in the stubs to ignore that complaint.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is implemented. Never is actually stronger than I expected since pylance read the stubs and visually show that the program will stop there.

@mutricyl
Copy link
Contributor Author

mutricyl commented Apr 2, 2024

@Dr-Irv your comments are implemented, you can review this PR one more time.
Regards

Copy link
Collaborator

@Dr-Irv Dr-Irv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @mutricyl . Nice use of assert_never()

@Dr-Irv Dr-Irv merged commit 0cf1cb9 into pandas-dev:main Apr 2, 2024
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

In DataFrame.select_dtypes type values like np.number or np.datetime64 should be allowed
2 participants