Skip to content

round method fails when columns are CategoricalIndex of IntervalIndex #30063

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
tdpetrou opened this issue Dec 4, 2019 · 3 comments · Fixed by #30181
Closed

round method fails when columns are CategoricalIndex of IntervalIndex #30063

tdpetrou opened this issue Dec 4, 2019 · 3 comments · Fixed by #30181
Assignees
Labels
Bug Categorical Categorical Data Type Indexing Related to indexing on series/frames, not to indexes themselves Interval Interval data type
Milestone

Comments

@tdpetrou
Copy link
Contributor

tdpetrou commented Dec 4, 2019

Code Sample, a copy-pastable example if possible

cols = pd.interval_range(0, 2)
data = np.random.rand(5, 2)

df = pd.DataFrame(data, columns=cols)
df.round() # works

cols2 = pd.CategoricalIndex(cols)
df2 = pd.DataFrame(data, columns=cols2)
df2.round() # fails

TypeError: No matching signature found

Problem description

The round method fails whenever the columns are CategoricalIndex made from an IntervalIndex. The pd.cut function creates these types of value and they can appear in columns when using pivot_table.

Expected Output

Should round as normal.

Output of pd.show_versions()

INSTALLED VERSIONS

commit : None
python : 3.7.3.final.0
python-bits : 64
OS : Darwin
OS-release : 18.7.0
machine : x86_64
processor : i386
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 0.25.3
numpy : 1.17.3
pytz : 2019.3
dateutil : 2.8.0
pip : 19.3.1
setuptools : 42.0.2.post20191201
Cython : 0.29.14
pytest : 5.3.1
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : 4.4.2
html5lib : None
pymysql : 0.9.3
psycopg2 : None
jinja2 : 2.10.3
IPython : 7.10.0
pandas_datareader: None
bs4 : 4.8.1
bottleneck : None
fastparquet : None
gcsfs : None
lxml.etree : 4.4.2
matplotlib : 3.1.2
numexpr : None
odfpy : None
openpyxl : 3.0.2
pandas_gbq : None
pyarrow : None
pytables : None
s3fs : None
scipy : 1.3.2
sqlalchemy : 1.3.11
tables : None
xarray : None
xlrd : 1.2.0
xlwt : None
xlsxwriter : None

@jbrockmendel
Copy link
Member

Looks like IntervalIndex.get_indexer will need to check for EAs before calling self._engine.get_indexer. It wouldn't surprise me if other EAs cause similar problems.

PR to fix would be welcome.

@jbrockmendel jbrockmendel added Bug Interval Interval data type Indexing Related to indexing on series/frames, not to indexes themselves labels Dec 4, 2019
@jorisvandenbossche jorisvandenbossche added the ExtensionArray Extending pandas with custom dtypes or arrays. label Dec 5, 2019
@jorisvandenbossche jorisvandenbossche added this to the Contributions Welcome milestone Dec 5, 2019
@sameshl
Copy link
Contributor

sameshl commented Dec 8, 2019

take

@sameshl sameshl removed their assignment Dec 9, 2019
@jschendel
Copy link
Member

jschendel commented Dec 10, 2019

A more direct example of this is failing due to IntervalIndex.get_indexer not supporting categoricals:

In [2]: ii = pd.interval_range(0, 2)

In [3]: cat = pd.Categorical(ii)

In [4]: ii.get_indexer(cat)
---------------------------------------------------------------------------
TypeError: No matching signature found

Need some special handling in get_indexer for this since categoricals can contain valid values of varying types, i.e. only intervals, only points, mixed intervals and points.

It wouldn't surprise me if other EAs cause similar problems

Yes, get_indexer fails similarly for homogeneous Period targets but should be fine for other EAs: there's already support for datetime/timedelta, and since ensure_index is called other EAs should get coerced to an object dtype Index, which goes down a separate path.

I'll open a new issue for the above since the fix is somewhat different than with categoricals and I want to keep those PRs disjoint.

@jschendel jschendel added Categorical Categorical Data Type and removed ExtensionArray Extending pandas with custom dtypes or arrays. labels Dec 10, 2019
@jschendel jschendel self-assigned this Dec 10, 2019
@jreback jreback modified the milestones: Contributions Welcome, 1.0 Dec 11, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Categorical Categorical Data Type Indexing Related to indexing on series/frames, not to indexes themselves Interval Interval data type
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants