BUG: Setting values with ILOC and list like indexers raises

[phofl]

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• (optional) I have confirmed this bug exists on the master branch of pandas.

---

Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example

import pandas as pd


df = pd.DataFrame({"flag": ["x", "y"], "value": [1, 2]})
df.iloc[[True, False], 1] = df.iloc[[True, False], 1] * 2
print(df)

Problem description

This worked on 1.0.5 and returned

  flag  value
0    x      2
1    y      2

On master this raises:

Traceback (most recent call last):
  File "/home/developer/.config/JetBrains/PyCharm2020.2/scratches/scratch_5.py", line 221, in <module>
    df.iloc[[True, False], 1] = x
  File "/home/developer/PycharmProjects/pandas/pandas/core/indexing.py", line 681, in __setitem__
    iloc._setitem_with_indexer(indexer, value)
  File "/home/developer/PycharmProjects/pandas/pandas/core/indexing.py", line 1756, in _setitem_with_indexer
    self._setitem_single_column(ilocs[0], value, pi)
  File "/home/developer/PycharmProjects/pandas/pandas/core/indexing.py", line 1800, in _setitem_single_column
    ser._mgr = ser._mgr.setitem(indexer=pi, value=value)
  File "/home/developer/PycharmProjects/pandas/pandas/core/internals/managers.py", line 532, in setitem
    return self.apply("setitem", indexer=indexer, value=value)
  File "/home/developer/PycharmProjects/pandas/pandas/core/internals/managers.py", line 397, in apply
    applied = getattr(b, f)(**kwargs)
  File "/home/developer/PycharmProjects/pandas/pandas/core/internals/blocks.py", line 923, in setitem
    check_setitem_lengths(indexer, value, values)
  File "/home/developer/PycharmProjects/pandas/pandas/core/indexers.py", line 158, in check_setitem_lengths
    raise ValueError(
ValueError: cannot set using a list-like indexer with a different length than the value

Process finished with exit code 1

Was this change of behavior intended?

Expected Output

I would expect, that this would work and returns the results from 1.0.5

Output of pd.show_versions()

master

[dsaxton]

Thanks @phofl

3da053c is the first bad commit
commit 3da053c
Author: jbrockmendel [email protected]
Date: Mon Feb 17 16:15:45 2020 -0800

BUG: fix length_of_indexer with boolean mask (#31897)

pandas/core/indexers.py | 8 +++++++-
pandas/core/indexing.py | 28 +++++++---------------------
pandas/tests/indexing/test_indexers.py | 11 +++++++++++
3 files changed, 25 insertions(+), 22 deletions(-)
create mode 100644 pandas/tests/indexing/test_indexers.py

#31897
cc @jbrockmendel

[jbrockmendel]

Not 100% sure, but it looks like before we get to that point in Block.setitem we want to have value = 2 instead if value = np.array([2]). It'll take some more digging to see exactly where that unpacking should take place.

[phofl]

cc @jbrockmendel

I looked a bit into this and found, that

pandas/pandas/core/indexing.py

Line 1673 in 18b4864

pi = plane_indexer[0] if lplane_indexer == 1 else plane_indexer

casts the plane_indexer to a list, when only one element from the indexer is True. This seems to be the reason why

df = pd.DataFrame({"flag": ["x", "y", "z"], "value": [1, 3, 4]})
df.iloc[[True, False, True], 1] = df.iloc[[True, False, True], 1] * 2

works, while

df = pd.DataFrame({"flag": ["x", "y", "z"], "value": [1, 3, 4]})
df.iloc[[True, False, False], 1] = df.iloc[[True, False, False], 1] * 2

does not. The check raising the error is only ran, when pi is a list. This line does not seem to be necessary. I ran all tests in indexing and indexes without failure after deleting it. Do you remember why you introduced it in #31837?