Skip to content

BUG: RangeIndex.astype('category') #41263

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 5, 2021
Merged

BUG: RangeIndex.astype('category') #41263

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
812eca4
BUG: RangeIndex.astype('category')
topper-123 May 2, 2021
359befa
add gh number
topper-123 May 2, 2021
7840030
add explicit test
topper-123 May 3, 2021
ebf9feb
simplify test
topper-123 May 3, 2021
d6edb52
fixing assert_index_equals with exact=True
topper-123 May 4, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v1.3.0.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -780,6 +780,7 @@ Indexing
- Bug in :meth:`DatetimeIndex.insert` when inserting ``np.datetime64("NaT")`` into a timezone-aware index incorrectly treating the timezone-naive value as timezone-aware (:issue:`39769`)
- Bug in incorrectly raising in :meth:`Index.insert`, when setting a new column that cannot be held in the existing ``frame.columns``, or in :meth:`Series.reset_index` or :meth:`DataFrame.reset_index` instead of casting to a compatible dtype (:issue:`39068`)
- Bug in :meth:`RangeIndex.append` where a single object of length 1 was concatenated incorrectly (:issue:`39401`)
- Bug in :meth:`RangeIndex.astype` where when converting to :class:`CategoricalIndex`, the categories became a :class:`Int64Index` instead of a :class:`RangeIndex` (:issue:`41263`)
- Bug in setting ``numpy.timedelta64`` values into an object-dtype :class:`Series` using a boolean indexer (:issue:`39488`)
- Bug in setting numeric values into a into a boolean-dtypes :class:`Series` using ``at`` or ``iat`` failing to cast to object-dtype (:issue:`39582`)
- Bug in :meth:`DataFrame.__setitem__` and :meth:`DataFrame.iloc.__setitem__` raising ``ValueError`` when trying to index with a row-slice and setting a list as values (:issue:`40440`)
Expand Down
4 changes: 1 addition & 3 deletions pandas/core/indexes/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -907,9 +907,7 @@ def astype(self, dtype, copy=True):
elif is_categorical_dtype(dtype):
from pandas.core.indexes.category import CategoricalIndex

return CategoricalIndex(
self._values, name=self.name, dtype=dtype, copy=copy
)
return CategoricalIndex(self, name=self.name, dtype=dtype, copy=copy)

elif is_extension_array_dtype(dtype):
return Index(np.asarray(self), name=self.name, dtype=dtype, copy=copy)
Expand Down
6 changes: 3 additions & 3 deletions pandas/tests/indexes/common.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -667,19 +667,19 @@ def test_astype_category(self, copy, name, ordered, simple_index):
# standard categories
dtype = CategoricalDtype(ordered=ordered)
result = idx.astype(dtype, copy=copy)
expected = CategoricalIndex(idx.values, name=name, ordered=ordered)
expected = CategoricalIndex(idx, name=name, ordered=ordered)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe have a test explicitly for RangeIndex?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tests.indexes.ranges.test_range.py::TestRangeIndex inherits from this base class, so it's tested that way. Do you have something else in mind?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some tm.assert_foo functions dont raise when we assert an Int64Index equals a RangeIndex. So off the top my head it isn't obvious that this edited test would fail in master. im asking for a test that is super-obviously tied to this bugfix

tm.assert_index_equal(result, expected)

# non-standard categories
dtype = CategoricalDtype(idx.unique().tolist()[:-1], ordered)
result = idx.astype(dtype, copy=copy)
expected = CategoricalIndex(idx.values, name=name, dtype=dtype)
expected = CategoricalIndex(idx, name=name, dtype=dtype)
tm.assert_index_equal(result, expected)

if ordered is False:
# dtype='category' defaults to ordered=False, so only test once
result = idx.astype("category", copy=copy)
expected = CategoricalIndex(idx.values, name=name)
expected = CategoricalIndex(idx, name=name)
tm.assert_index_equal(result, expected)

def test_is_unique(self, simple_index):
Expand Down
23 changes: 23 additions & 0 deletions pandas/tests/indexes/ranges/test_range.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@

import pandas as pd
from pandas import (
CategoricalDtype,
CategoricalIndex,
Float64Index,
Index,
Int64Index,
Expand Down Expand Up @@ -533,3 +535,24 @@ def test_isin_range(self, base):
result = base.isin(values)
expected = np.array([True, False])
tm.assert_numpy_array_equal(result, expected)

@pytest.mark.parametrize("copy", [True, False])
@pytest.mark.parametrize("name", [None, "foo"])
@pytest.mark.parametrize("ordered", [True, False])
def test_astype_category(self, copy, name, ordered, simple_index):
super().test_astype_category(copy, name, ordered, simple_index)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm we don't usually call the super functions as they are already called so this is confusing. i think you can remove. (assume my assertion is true)


# GH 41263
idx = simple_index if not name else simple_index.rename(name)

dtypes = [CategoricalDtype(ordered=ordered)]
if ordered is False:
# dtype='category' defaults to ordered=False, so only test once
dtypes.append("category")

for dtype in dtypes:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you not simply construct the expected result and use assert_index_equal(exact=True)

Copy link
Contributor Author

@topper-123 topper-123 May 4, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's better. It however exposed a bug because assert_index_equal(exact=True) didn't actually work as expected.

I fixed that bug in assert_index_equal and changed these tests to use exact=True.

# standard categories
result = idx.astype(dtype, copy=copy)
assert isinstance(result, CategoricalIndex)
assert isinstance(result.categories, RangeIndex)
assert (result.categories == idx).all()