Skip to content

REGR: Index.map adding back tz to tz-agnostic result #57475

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v2.2.1.rst
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ Fixed regressions
- Fixed regression in :meth:`DataFrameGroupBy.idxmin`, :meth:`DataFrameGroupBy.idxmax`, :meth:`SeriesGroupBy.idxmin`, :meth:`SeriesGroupBy.idxmax` where values containing the minimum or maximum value for the dtype could produce incorrect results (:issue:`57040`)
- Fixed regression in :meth:`ExtensionArray.to_numpy` raising for non-numeric masked dtypes (:issue:`56991`)
- Fixed regression in :meth:`Index.join` raising ``TypeError`` when joining an empty index to a non-empty index containing mixed dtype values (:issue:`57048`)
- Fixed regression in :meth:`Index.map` that would not change the dtype when the provided mapping would change data from tz-aware to tz-agnostic or tz-agnostic to tz-aware (:issue:`57192`)
- Fixed regression in :meth:`Series.pct_change` raising a ``ValueError`` for an empty :class:`Series` (:issue:`57056`)
- Fixed regression in :meth:`Series.to_numpy` when dtype is given as float and the data contains NaNs (:issue:`57121`)

Expand Down
18 changes: 10 additions & 8 deletions pandas/core/indexes/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -6124,14 +6124,16 @@ def map(self, mapper, na_action: Literal["ignore"] | None = None):
# empty
dtype = self.dtype

# e.g. if we are floating and new_values is all ints, then we
# don't want to cast back to floating. But if we are UInt64
# and new_values is all ints, we want to try.
same_dtype = lib.infer_dtype(new_values, skipna=False) == self.inferred_type
if same_dtype:
new_values = maybe_cast_pointwise_result(
new_values, self.dtype, same_dtype=same_dtype
)
if self.inferred_type != "datetime64":
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure what other dtypes might arise here - need to check

# e.g. if we are floating and new_values is all ints, then we
# don't want to cast back to floating. But if we are UInt64
# and new_values is all ints, we want to try.
# GH#57192 - we skip datetime64 because inference from values is reliable
same_dtype = lib.infer_dtype(new_values, skipna=False) == self.inferred_type
if same_dtype:
new_values = maybe_cast_pointwise_result(
new_values, self.dtype, same_dtype=same_dtype
)

return Index._with_infer(new_values, dtype=dtype, copy=False, name=self.name)

Expand Down
13 changes: 13 additions & 0 deletions pandas/tests/indexes/datetimes/methods/test_map.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,3 +45,16 @@ def test_index_map(self, name):
)
exp_index = MultiIndex.from_product(((2018,), range(1, 7)), names=[name, name])
tm.assert_index_equal(index, exp_index)

@pytest.mark.parametrize("input_tz", ["UTC", None])
@pytest.mark.parametrize("output_tz", ["UTC", None])
def test_mapping_tz_to_tz_agnostic(self, input_tz, output_tz):
# GH#57192
index = date_range("2018-01-01", periods=6, freq="ME", tz=input_tz)
expected = date_range("2018-01-01", periods=6, freq="ME", tz=output_tz)
if input_tz == "UTC" and output_tz == "UTC":
method = "tz_convert"
else:
method = "tz_localize"
result = index.map(lambda x: getattr(x, method)(output_tz))
tm.assert_index_equal(result, expected)