Skip to content

BUG: Inserting array of same size with Series.loc raises ValueError #38266

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Dec 5, 2020
2 changes: 2 additions & 0 deletions doc/source/whatsnew/v1.2.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -677,6 +677,8 @@ Indexing
- Bug in :meth:`DataFrame.loc` and :meth:`DataFrame.__getitem__` raising ``KeyError`` when columns were :class:`MultiIndex` with only one level (:issue:`29749`)
- Bug in :meth:`Series.__getitem__` and :meth:`DataFrame.__getitem__` raising blank ``KeyError`` without missing keys for :class:`IntervalIndex` (:issue:`27365`)
- Bug in setting a new label on a :class:`DataFrame` or :class:`Series` with a :class:`CategoricalIndex` incorrectly raising ``TypeError`` when the new label is not among the index's categories (:issue:`38098`)
- Bug in :meth:`Series.loc` and :meth:`Series.iloc` raising ``ValueError`` when inserting a listlike ``np.array``, ``list`` or ``tuple`` in an ``object`` Series of equal length (:issue:`37748`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you list 3 issues in the top of PR, what's missing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I added a whastnew note for #38271 and one for #37748.
In the PR, I listed #37486 because it is a particular case of #37748 (iloc on a Series with a list, both of size 1).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pls add the issue number to the note for #37748 then

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

- Bug in :meth:`Series.loc` and :meth:`Series.iloc` setting all the values of an ``object`` Series with those of a listlike ``ExtensionArray`` instead of inserting it (:issue:`38271`)

Missing
^^^^^^^
Expand Down
12 changes: 12 additions & 0 deletions pandas/_testing.py
Original file line number Diff line number Diff line change
Expand Up @@ -1755,6 +1755,18 @@ def assert_equal(left, right, **kwargs):
raise NotImplementedError(type(left))


def assert_python_equal(left, right):
"""
Check left and right are equal w.r.t the ``==`` operator.

Parameters
----------
left : object
right : object
"""
assert left == right
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, I'm guessing this is too much, but I failed to find helper funcs to assert list/tuple equality.
I'll welcome advice on how to write the tests :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah don't do this, just do assert left == right pytest already handles this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok



def box_expected(expected, box_cls, transpose=True):
"""
Helper function to wrap the expected output of a test in a given box_class.
Expand Down
3 changes: 3 additions & 0 deletions pandas/core/indexers.py
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,9 @@ def is_scalar_indexer(indexer, ndim: int) -> bool:
-------
bool
"""
if ndim == 1 and is_integer(indexer):
# GH37748: allow indexer to be an integer for Series
return True
if isinstance(indexer, tuple):
if len(indexer) == ndim:
return all(
Expand Down
6 changes: 6 additions & 0 deletions pandas/tests/indexing/test_indexers.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,12 @@ def test_is_scalar_indexer():

assert not is_scalar_indexer(slice(None), 1)

indexer = 0
assert is_scalar_indexer(indexer, 1)

indexer = (0,)
assert is_scalar_indexer(indexer, 1)


class TestValidateIndices:
def test_validate_indices_ok(self):
Expand Down
29 changes: 29 additions & 0 deletions pandas/tests/indexing/test_loc.py
Original file line number Diff line number Diff line change
Expand Up @@ -2072,3 +2072,32 @@ def test_loc_setitem_dt64tz_values(self):
s2["a"] = expected
result = s2["a"]
assert result == expected

@pytest.mark.parametrize(
"array_fn,assert_fn",
[
(np.zeros, tm.assert_numpy_array_equal),
(pd.array, tm.assert_extension_array_equal),
(list, tm.assert_python_equal),
(tuple, tm.assert_python_equal),
],
)
@pytest.mark.parametrize("size", [0, 4, 5, 6])
def test_loc_iloc_setitem_with_listlike(self, size, array_fn, assert_fn):
# GH37748
# testing insertion, in Series of size N (here 5), of listlike
# of size 0, N-1, N, N+1

expected = array_fn([0] * size)

ser = Series(0, index=list("abcde"), dtype="object")

ser.loc["a"] = expected
result = ser[0]
assert_fn(result, expected)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason for not checking the whole Series? This would also avoid your helper function

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point, thanks


ser = Series(0, index=list("abcde"), dtype="object")

ser.iloc[0] = expected
result = ser[0]
assert_fn(result, expected)