Skip to content

BUG: assignment with boolean index not propogating dtype #13169

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jreback opened this issue May 13, 2016 · 2 comments
Closed

BUG: assignment with boolean index not propogating dtype #13169

jreback opened this issue May 13, 2016 · 2 comments
Labels
Bug Closing Candidate May be closeable, needs more eyeballs Dtype Conversions Unexpected or buggy dtype conversions Indexing Related to indexing on series/frames, not to indexes themselves

Comments

@jreback
Copy link
Contributor

jreback commented May 13, 2016

In [5]: df = DataFrame({'A' : [1,2]})

In [6]: df['B'] = Index([True,False])

In [7]: df['C'] = Index([True,False]).values

In [8]: df
Out[8]: 
   A      B      C
0  1   True   True
1  2  False  False

In [9]: df.dtypes
Out[9]: 
A     int64
B    object
C    object
dtype: object

In [10]: df = DataFrame({'A' : [1,2]})

In [11]: df['B'] = Index([True,False])

# you have to force the coercion
In [12]: df['C'] = Index([True,False]).values.astype(bool)

In [13]: Index([True,False])
Out[13]: Index([True, False], dtype='object')

In [14]: Index([True,False]).values
Out[14]: array([True, False], dtype=object)

In [15]: df.dtypes
Out[15]: 
A     int64
B    object
C      bool
dtype: object

Since we don't have a boolean Index type (we should, but separate issue), the assignment of an Index leaves the resulting dtype as object rather than bool. This can be fixed on a special case basis here, though maybe better in _sanitize_array.

But I think this might be non-performant to even check with infer_dtype. So you have to be a bit hacky about this, e.g. call is_bool_array first, then you can coerce if it IS (say its NOT, IOW its a bunch of strings, then this is problematic).

@jreback jreback added Bug Reshaping Concat, Merge/Join, Stack/Unstack, Explode Dtype Conversions Unexpected or buggy dtype conversions Difficulty Intermediate labels May 13, 2016
@jreback jreback added this to the Next Major Release milestone May 13, 2016
jreback added a commit to jreback/pandas that referenced this issue May 27, 2016
closes pandas-dev#8596

xref to pandas-dev#13169 as assignment of Index of bools not retaining dtype
@mroeschke mroeschke added Indexing Related to indexing on series/frames, not to indexes themselves and removed Reshaping Concat, Merge/Join, Stack/Unstack, Explode labels Apr 30, 2021
@jbrockmendel
Copy link
Member

Not clear that there's anything wrong here, just an inconvenient consequence of not having a bool-dtype Index.

@jbrockmendel jbrockmendel added the Closing Candidate May be closeable, needs more eyeballs label Jan 1, 2022
@mroeschke
Copy link
Member

Now that EA's are supported in the Index, I think using the 'boolean masked-EA should be a sufficient work around here, so closing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Closing Candidate May be closeable, needs more eyeballs Dtype Conversions Unexpected or buggy dtype conversions Indexing Related to indexing on series/frames, not to indexes themselves
Projects
None yet
Development

No branches or pull requests

3 participants