Skip to content

REF: Changed ExtensionDtype inheritance #20363

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

TomAugspurger
Copy link
Contributor

is_extension_array_dtype(dtype) was incorrect for dtypes that haven't
implemented the new interface yet. This is because they indirectly subclassed
ExtensionDtype.

This PR changes the hierarchy so that PandasExtensionDtype doesn't subclass
ExtensionDtype. As we implement the interface, like Categorical, we'll add
ExtensionDtype as a base class.

Before:

DatetimeTZDtype <- PandasExtensionDtype <- ExtensionDtype (wrong)
CategoricalDtype <- PandasExtensionDtype <- ExtensionDtype (right)

After:

DatetimeTZDtype <- PandasExtensionDtype
                                        \
                                         - _DtypeOpsMixin
                                        /
                   ExtensionDtype ------

CategoricalDtype - PandasExtensionDtype -
                \                        \
                 \                        -_DtypeOpsMixin
                  \                      /
                   ExtensionDtype -------

Once all our extension dtypes have implemented the interface we can go back
to the simple, linear inheritance structure.

`is_extension_array_dtype(dtype)` was incorrect for dtypes that haven't
implemented the new interface yet. This is because they indirectly subclassed
ExtensionDtype.

This PR changes the hierarchy so that PandasExtensionDtype doesn't subclass
ExtensionDtype. As we implement the interface, like Categorical, we'll add
ExtensionDtype as a base class.

Before:

```
DatetimeTZDtype <- PandasExtensionDtype <- ExtensionDtype (wrong)
CategoricalDtype <- PandasExtensionDtype <- ExtensionDtype (right)

After:

DatetimeTZDtype <- PandasExtensionDtype
                                        \
                                         - _DtypeOpsMixin
                                        /
                   ExtensionDtype ------

CategoricalDtype - PandasExtensionDtype -
                \                        \
                 \                        -_DtypeOpsMixin
                  \                      /
                   ExtensionDtype -------

```

Once all our extension dtypes have implemented the interface we can go back
to the simple, linear inheritance structure.
@TomAugspurger TomAugspurger added Dtype Conversions Unexpected or buggy dtype conversions ExtensionArray Extending pandas with custom dtypes or arrays. labels Mar 15, 2018
@TomAugspurger TomAugspurger added this to the 0.23.0 milestone Mar 15, 2018
Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add a test that asserts is_extension_array (or not) for each of the current pandas extension types (and so should only be True for CategoricalDtype, but false for Interval, Period, Datetimewtz)

@codecov
Copy link

codecov bot commented Mar 16, 2018

Codecov Report

Merging #20363 into master will increase coverage by 0.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #20363      +/-   ##
==========================================
+ Coverage   91.78%   91.79%   +0.01%     
==========================================
  Files         152      152              
  Lines       49196    49186      -10     
==========================================
- Hits        45153    45152       -1     
+ Misses       4043     4034       -9
Flag Coverage Δ
#multiple 90.18% <100%> (+0.01%) ⬆️
#single 41.83% <88.46%> (+0.01%) ⬆️
Impacted Files Coverage Δ
pandas/core/dtypes/dtypes.py 95.9% <100%> (ø) ⬆️
pandas/core/dtypes/base.py 91.89% <100%> (+0.22%) ⬆️
pandas/core/dtypes/common.py 94.64% <100%> (ø) ⬆️
pandas/core/dtypes/cast.py 87.68% <100%> (ø) ⬆️
pandas/core/internals.py 95.54% <100%> (ø) ⬆️
pandas/core/indexes/datetimes.py 95.64% <0%> (ø) ⬆️
pandas/core/generic.py 95.85% <0%> (ø) ⬆️
pandas/core/series.py 93.84% <0%> (ø) ⬆️
pandas/core/indexes/datetimelike.py 96.72% <0%> (ø) ⬆️
pandas/errors/__init__.py 92.3% <0%> (ø) ⬆️
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 87f5654...da2451e. Read the comment docs.

@TomAugspurger
Copy link
Contributor Author

Added tests for each extension dtype. Will have to move from not to is as we migrate them.

@TomAugspurger TomAugspurger merged commit 8a58303 into pandas-dev:master Mar 16, 2018
@TomAugspurger TomAugspurger deleted the extension-dtype-heirarchy branch March 16, 2018 14:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Dtype Conversions Unexpected or buggy dtype conversions ExtensionArray Extending pandas with custom dtypes or arrays.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants