ENH: Select numeric ExtensionDtypes with DataFrame.select_dtypes #35341
Conversation
|
Hello @andrewgsavage! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found: There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻 Comment last updated at 2020-10-19 13:14:33 UTC |
pandas/core/frame.py
Outdated
| ] | ||
| if np.number in dtypes_set: | ||
| extracted_dtypes.extend([]) |
There was a problem hiding this comment.
not looked at this in detail, but from the name of the function, i'm assuming that this is not the correct location for this special casing.
There is another issue regarding extension arrays and dtypes, #9581. maybe a compat function for np.issubdtype could help, where we could add the special cases for EAs.
There was a problem hiding this comment.
OK I've added function to pandas\compat\numpy_init_.py
| if numeric: | ||
| assert df.select_dtypes(np.number).shape == df.shape | ||
| else: | ||
| assert df.select_dtypes(np.number).shape != df.shape |
There was a problem hiding this comment.
the extension tests are predominantly testing specific EAs. could you add a test to pandas\tests\extension\base\dtype.py instead and override as necessary for the different EAs.
There was a problem hiding this comment.
My intention isn't to test the different EAs, but to test a dummy EA that emulates an external EA, like PintArray.
The most similar tests I could find were in pandas\tests\extension\test_common.py. Perhaps I should add the tests there?
simonjayhawkins
added
Dtype Conversions
pandas/compat/numpy/__init__.py
Outdated
| @@ -5,6 +5,8 @@ | |||
|
|
|||
| import numpy as np | |||
|
|
|||
| from pandas.core.dtypes.common import is_extension_array_dtype | |||
There was a problem hiding this comment.
if there's any viable way to avoid this, we'd rather not import from pandas.core here
pandas/compat/numpy/__init__.py
Outdated
| @@ -62,6 +63,17 @@ def np_array_datetime64_compat(arr, *args, **kwargs): | |||
| return np.array(arr, *args, **kwargs) | |||
|
|
|||
|
|
|||
| def np_issubclass_compat(unique_dtype, dtypes_set): | |||
There was a problem hiding this comment.
this wouldn't g in here, better in pandas/core/dtypes/common.py
with a full doc-string & examples
There was a problem hiding this comment.
this is almost is_numeric, but with the twist about np.number
There was a problem hiding this comment.
I've moved it and added doc string and examples. I'm not aware of any pandas dtypes that have _is_numeric = True and don't inherit from np.number.
Should is_numeric return True for ExtensionDtypes that have _is_numeric = True and don't inherit from np.number?
| pass | ||
|
|
||
|
|
||
| def test_select_dtypes_numeric(): |
There was a problem hiding this comment.
all of this should go in tests/frame/methods/test_select_dtypes.py
|
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this. |
|
@andrewgsavage can you merge master to resolve conflicts. Also needs a release note. (put in doc\source\whatsnew\v1.2.0.rst for now, but my need to moved as 1.2rc is scheduled before the end of the month) |
|
Closing in favor of #38246 |
black pandasgit diff upstream/master -u -- "*.py" | flake8 --diff