Skip to content

BUG: setting dt64 values into Series[int] incorrectly casting dt64->int #39266

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jan 28, 2021
1 change: 1 addition & 0 deletions doc/source/whatsnew/v1.3.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -281,6 +281,7 @@ Indexing
- Bug in :meth:`DataFrame.loc` dropping levels of :class:`MultiIndex` when :class:`DataFrame` used as input has only one row (:issue:`10521`)
- Bug in setting ``timedelta64`` values into numeric :class:`Series` failing to cast to object dtype (:issue:`39086`)
- Bug in setting :class:`Interval` values into a :class:`Series` or :class:`DataFrame` with mismatched :class:`IntervalDtype` incorrectly casting the new values to the existing dtype (:issue:`39120`)
- Bug in setting ``datetime64`` values into a :class:`Series` with integer-dtype incorrect casting the datetime64 values to integers (:issue:`39266`)

Missing
^^^^^^^
Expand Down
12 changes: 12 additions & 0 deletions pandas/core/internals/blocks.py
Original file line number Diff line number Diff line change
Expand Up @@ -991,6 +991,18 @@ def setitem(self, indexer, value):

# set
else:
if (
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

elif?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, we still want to fall through to L1006

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure but i thin it is much more clear to use an elif and then simply repeat that line in the else

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated + green

self.is_object
and not is_ea_value
and arr_value.dtype.kind in ["m", "M"]
):
# https://github.com/numpy/numpy/issues/12550
# numpy will incorrect cast to int if we're not careful
if is_list_like(value):
value = list(value)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

np.asarray?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

until numpy#12550 is fixed this is the only way i found to get this to work

else:
value = [value] * len(values[indexer])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe use np.full?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above re numpy#12550


values[indexer] = value

if transpose:
Expand Down
21 changes: 21 additions & 0 deletions pandas/tests/series/indexing/test_setitem.py
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,27 @@ def test_setitem_callable_other(self):
tm.assert_series_equal(ser, expected)


class TestSetitemCasting:
@pytest.mark.parametrize("dtype", ["M8[ns]", "m8[ns]"])
def test_setitem_dt64_into_int_series(self, dtype):
# dont cast dt64 to int when doing this setitem
orig = Series([1, 2, 3])

val = np.datetime64("2021-01-18 13:25:00", "ns")
if dtype == "m8[ns]":
val = val - val

ser = orig.copy()
ser[:-1] = val
expected = Series([val, val, 3], dtype=object)
tm.assert_series_equal(ser, expected)
assert isinstance(ser[0], type(val))

ser = orig.copy()
ser[:-1] = np.array([val, val])
tm.assert_series_equal(ser, expected)


@pytest.mark.parametrize(
"obj,expected,key",
[
Expand Down