Skip to content

Change check() function to test actual types in Series and Index #546

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Feb 21, 2023

Conversation

Dr-Irv
Copy link
Collaborator

@Dr-Irv Dr-Irv commented Feb 20, 2023

While reviewing #372 , it uncovered that by having check() use next(iter()), if you have a Series([1,2,3], dtype=np.int8), it was not checking that the actual type of the first element was np.int8, since Series.__iter__() returns a python scalar.

So this corrects that, allowing us to be more precise in the type checking of the values inside the results, since we want to check that astype() is doing what we expect.

Also, was able to allow pytables to work with 3.11, since they released a compatible version.

Uncovered an odd issue with bitwise operations on lists of int

In some cases, used np.integer because it is a supertype that works across platforms. Other times, np.int_ worked fine.

@Dr-Irv Dr-Irv requested a review from twoertwein February 20, 2023 23:10
Copy link
Member

@twoertwein twoertwein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! Hope you had a somewhat automated approach for all these changes :)

iloc vs __iter__: I would have been fine keeping the current approach as it is not clear what the "correct" type is.

@Dr-Irv
Copy link
Collaborator Author

Dr-Irv commented Feb 21, 2023

Looks good to me! Hope you had a somewhat automated approach for all these changes :)

Unfortunately not! It was a bit of wack-a-mole.

iloc vs __iter__: I would have been fine keeping the current approach as it is not clear what the "correct" type is.

Agreed, but I think we should be doing this level of checking when we get to things like Int8Dtype() .

@twoertwein twoertwein merged commit 949885e into pandas-dev:main Feb 21, 2023
@twoertwein
Copy link
Member

Thanks @Dr-Irv !

btw, the next mypy release (1.1?) will suggest using pandas-stubs python/mypy#14328 (comment)

@Dr-Irv Dr-Irv deleted the fixcheck branch February 27, 2023 22:17
twoertwein pushed a commit to twoertwein/pandas-stubs that referenced this pull request Apr 1, 2023
…das-dev#546)

* fix check function to check actual type, not using __iter__()

* change np.int32 to np.int_

* for bitwise use np.integer

* change tables reference in pyproject.toml

* for nightly, use np.integer rather than np.int64

* make timedelta division type result dependent on pandas version

* fix mypy issue of assigning timedeltadiv return type
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants