Skip to content

BUG: Fix issue with datetime[ns, tz] input in Block.setitem GH32395 #32479

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 25 commits into from
May 25, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
198474d
BUG: Fix issue with datetime[ns, tz] input in Block.setitem GH32395
h-vishal Mar 5, 2020
6eba1c9
Added tests for series, moved tests to test_loc.py
h-vishal Mar 6, 2020
11b4f29
Add dtype to series object in tests to suppress warning
h-vishal Mar 6, 2020
1b3cba1
Fix whatsnew
h-vishal Mar 6, 2020
d83cff9
Merge remote-tracking branch 'remotes/upstream/master' into issue-32395
h-vishal Mar 6, 2020
4e66228
Merge remote-tracking branch 'remotes/upstream/master' into issue-32395
h-vishal Mar 7, 2020
ac85aa3
Merge remote-tracking branch 'remotes/upstream/master' into issue-32395
h-vishal Mar 9, 2020
2d25c7a
Merge remote-tracking branch 'remotes/upstream/master' into issue-32395
h-vishal Mar 27, 2020
3e78985
Move whatsnew to 1.1.0
h-vishal Mar 27, 2020
7e600c7
Genralise fix for other extension array types
h-vishal Apr 2, 2020
05a788c
Merge remote-tracking branch 'remotes/upstream/master' into issue-32395
h-vishal Apr 2, 2020
b136ce3
Add new whatsnew
h-vishal Apr 3, 2020
d433d7c
Fix doc
h-vishal Apr 3, 2020
00199f6
Fix version number in whatsnew
h-vishal Apr 3, 2020
01f01fd
Merge remote-tracking branch 'remotes/upstream/master' into issue-32395
h-vishal Apr 3, 2020
8cfa045
TST: Fix precision test in tests.computation.test_eval.check_alignment()
h-vishal Apr 3, 2020
4f8fccd
Revert "TST: Fix precision test in
h-vishal Apr 4, 2020
0f8a913
CLN: Test style tests.indexing.test_loc
h-vishal Apr 4, 2020
9d4d1df
Merge remote-tracking branch 'remotes/upstream/master' into issue-32395
h-vishal Apr 4, 2020
a01a676
Modify tests after review
h-vishal Apr 6, 2020
26f8ed3
Merge remote-tracking branch 'remotes/upstream/master' into issue-32395
h-vishal Apr 6, 2020
3caf161
Merge remote-tracking branch 'upstream/master' into issue-32395
simonjayhawkins May 7, 2020
8d9e6d7
Merge remote-tracking branch 'upstream/master' into issue-32395
simonjayhawkins May 20, 2020
fbc31c8
nits
simonjayhawkins May 20, 2020
c7e7cf5
update tests
simonjayhawkins May 20, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v1.1.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -726,6 +726,7 @@ Indexing
- Bug in :meth:`DataFrame.iloc.__setitem__` creating a new array instead of overwriting ``Categorical`` values in-place (:issue:`32831`)
- Bug in :class:`Interval` where a :class:`Timedelta` could not be added or subtracted from a :class:`Timestamp` interval (:issue:`32023`)
- Bug in :meth:`DataFrame.copy` _item_cache not invalidated after copy causes post-copy value updates to not be reflected (:issue:`31784`)
- Fixed regression in :meth:`DataFrame.loc` and :meth:`Series.loc` throwing an error when a ``datetime64[ns, tz]`` value is provided (:issue:`32395`)
- Bug in `Series.__getitem__` with an integer key and a :class:`MultiIndex` with leading integer level failing to raise ``KeyError`` if the key is not present in the first level (:issue:`33355`)
- Bug in :meth:`DataFrame.iloc` when slicing a single column-:class:`DataFrame`` with ``ExtensionDtype`` (e.g. ``df.iloc[:, :1]``) returning an invalid result (:issue:`32957`)
- Bug in :meth:`DatetimeIndex.insert` and :meth:`TimedeltaIndex.insert` causing index ``freq`` to be lost when setting an element into an empty :class:`Series` (:issue:33573`)
Expand Down
7 changes: 7 additions & 0 deletions pandas/core/internals/blocks.py
Original file line number Diff line number Diff line change
Expand Up @@ -824,8 +824,10 @@ def setitem(self, indexer, value):
if is_extension_array_dtype(getattr(value, "dtype", None)):
# We need to be careful not to allow through strings that
# can be parsed to EADtypes
is_ea_value = True
arr_value = value
else:
is_ea_value = False
arr_value = np.array(value)

if transpose:
Expand Down Expand Up @@ -853,6 +855,11 @@ def setitem(self, indexer, value):
values[indexer] = value
return self.make_block(Categorical(self.values, dtype=arr_value.dtype))

elif exact_match and is_ea_value:
# GH#32395 if we're going to replace the values entirely, just
# substitute in the new array
return self.make_block(arr_value)

# if we are an exact match (ex-broadcasting),
# then use the resultant dtype
elif exact_match:
Expand Down
28 changes: 28 additions & 0 deletions pandas/tests/extension/base/setitem.py
Original file line number Diff line number Diff line change
Expand Up @@ -301,3 +301,31 @@ def test_setitem_preserves_views(self, data):
data[0] = data[1]
assert view1[0] == data[1]
assert view2[0] == data[1]

def test_setitem_dataframe_column_with_index(self, data):
# https://github.com/pandas-dev/pandas/issues/32395
df = expected = pd.DataFrame({"data": pd.Series(data)})
result = pd.DataFrame(index=df.index)
result.loc[df.index, "data"] = df["data"]
self.assert_frame_equal(result, expected)

def test_setitem_dataframe_column_without_index(self, data):
# https://github.com/pandas-dev/pandas/issues/32395
df = expected = pd.DataFrame({"data": pd.Series(data)})
result = pd.DataFrame(index=df.index)
result.loc[:, "data"] = df["data"]
self.assert_frame_equal(result, expected)

def test_setitem_series_with_index(self, data):
# https://github.com/pandas-dev/pandas/issues/32395
ser = expected = pd.Series(data, name="data")
result = pd.Series(index=ser.index, dtype=np.object, name="data")
result.loc[ser.index] = ser
self.assert_series_equal(result, expected)

def test_setitem_series_without_index(self, data):
# https://github.com/pandas-dev/pandas/issues/32395
ser = expected = pd.Series(data, name="data")
result = pd.Series(index=ser.index, dtype=np.object, name="data")
result.loc[:] = ser
self.assert_series_equal(result, expected)