Skip to content

BUG: After reindex(), astype() raises exception for categorical series #17444

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
junjiecai opened this issue Sep 6, 2017 · 3 comments · Fixed by #29342
Closed

BUG: After reindex(), astype() raises exception for categorical series #17444

junjiecai opened this issue Sep 6, 2017 · 3 comments · Fixed by #29342
Labels
good first issue Needs Tests Unit test(s) needed to prevent regressions
Milestone

Comments

@junjiecai
Copy link

junjiecai commented Sep 6, 2017

Hi

I meet an unexpected behavior for categorical series

After reindex() a categorical series, astype() will raise Exception

For example, code below raise ValueError: Wrong number of dimensions

s = Series([1, 2, 3], index = [2, 0, 1])
(s.astype('category')
 .reindex([0, 1, 2])
 .astype(str))

However, if I reindex() first then set the series categorical, astype() works as expected.
Code below works fine

s = Series([1, 2, 3], index = [2, 0, 1])
(s.reindex([0, 1, 2])
 .astype('category')
 .astype(str))

I think both code should behave the same way

@jreback
Copy link
Contributor

jreback commented Sep 6, 2017

hmm this is a bug, not sure exactly, but I think something is getting created in an odd way.

@jreback jreback changed the title After reindex(), astype() raises exception for categorical series BUG: After reindex(), astype() raises exception for categorical series Sep 6, 2017
@jreback jreback added Categorical Categorical Data Type Difficulty Intermediate Reshaping Concat, Merge/Join, Stack/Unstack, Explode labels Sep 6, 2017
@jreback jreback added this to the Next Major Release milestone Sep 6, 2017
@alanbato
Copy link
Contributor

I'm looking into it, and it seems that the bug is due to its CategoricalBlock step having ndim evaluating to 2, whereas its values.ndim evaluate to 1. I think it has to do with how a Block object has an ndim of 2 by default.
Maybe we aren't sending the ndim parameter and we should, or is this supposed to be correct?

@mroeschke
Copy link
Member

Looks to work on master. Could use a test.

In [30]: s = Series([1, 2, 3], index = [2, 0, 1])
    ...: (s.astype('category')
    ...:  .reindex([0, 1, 2])
    ...:  .astype(str))
Out[30]:
0    2
1    3
2    1
dtype: object

In [31]: pd.__version__
Out[31]: '0.26.0.dev0+565.g8c5941cd5'

@mroeschke mroeschke added good first issue Needs Tests Unit test(s) needed to prevent regressions and removed Categorical Categorical Data Type Difficulty Intermediate Reshaping Concat, Merge/Join, Stack/Unstack, Explode labels Oct 15, 2019
@jreback jreback modified the milestones: Contributions Welcome, 1.0 Nov 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Needs Tests Unit test(s) needed to prevent regressions
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants