BUG: DataFrame.drop - Tuple Cannot Be Used as List-Like

[ghost]

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of pandas.
• (optional) I have confirmed this bug exists on the master branch of pandas.

---

Code Sample

import pandas as pd

data = {"column_a": [5, 10, 15], "column_b": ["five", "ten", "fifteen"]}
index = [0, 1, 2]
frame = pd.DataFrame(data, index=index)

frame.drop(tuple([0, 1]))

Problem description

DataFrame.drop (https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.drop.html) should take a parameter 'labels' of 'single label or list-like' for the 'Index or column labels to drop.'

According to documentation for api.types.is_list_like (https://pandas.pydata.org/docs/reference/api/pandas.api.types.is_list_like.html) "Objects that are considered list-like are for example Python lists, tuples, sets, NumPy arrays, and Pandas Series."

We should be able to drop using a Tuple of Index labels. However, this leads to a 'KeyError' indicating the the entire tuple is not found as a value in the Index.

Expected Output

I expect this should return a DataFrame with the dropped Indexes - the same as if dropping with a List, Set, Array or Series of [0, 1].

frame.drop(tuple([0, 1])) == frame.drop([0, 1])

expected_data = {"column_a": [15], "column_b": ["fifteen"]}
expected_index = [2]
expected_output = pd.DataFrame(data, index=index)

Output of pd.show_versions()

INSTALLED VERSIONS

commit : 71f01a8
python : 3.8.10.final.0
python-bits : 64
OS : Darwin
OS-release : 19.6.0
Version : Darwin Kernel Version 19.6.0: Mon Apr 12 20:57:45 PDT 2021; root:xnu-6153.141.28.1~1/RELEASE_X86_64
machine : x86_64
processor : i386
byteorder : little
LC_ALL : en_US.UTF-8
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.4.0.dev0+310.g71f01a885f
numpy : 1.21.1
pytz : 2021.1
dateutil : 2.8.2
pip : 21.1.3
setuptools : 52.0.0.post20210125
Cython : 0.29.24
pytest : 6.2.4
hypothesis : 6.14.4
sphinx : 4.1.2
blosc : 1.10.4
feather : None
xlsxwriter : 1.4.4
lxml.etree : 4.6.3
html5lib : 1.1
pymysql : None
psycopg2 : None
jinja2 : 3.0.1
IPython : 7.25.0
pandas_datareader: None
bs4 : 4.9.3
bottleneck : 1.3.2
fsspec : 2021.05.0
fastparquet : 0.6.3
gcsfs : 2021.05.0
matplotlib : 3.4.2
numexpr : 2.7.3
odfpy : None
openpyxl : 3.0.7
pandas_gbq : None
pyarrow : 4.0.1
pyxlsb : None
s3fs : 2021.05.0
scipy : 1.7.0
sqlalchemy : 1.4.22
tables : 3.6.1
tabulate : 0.8.9
xarray : 0.18.2
xlrd : 2.0.1
xlwt : 1.3.0
numba : 0.53.1

[ghost]

Didn't see anything Open for this under Issues with the 'DataFrame' label and mention of 'drop' or 'tuple'. Noticed this when trying to write tests for another Pull Request #42746.

[rhshadrach]

Thanks for the report! Not sure if this is a documentation issue (i.e. tuples should not be counted as list-likes in drop) or an implementation one. I think you will find not much support for having tuples in an index by pandas, but that is something we'd like to see improved. cc @phofl @jbrockmendel for any thoughts.

[ghost]

@rhshadrach I took some time to look into this a bit more and I think I have a better understanding now. I believe the problem is that to a user, a tuple can be both a label or a list-like (which are both accepted in the documentation), but here we always consider it a label and the user is unaware.

In DataFrame.drop() we call pandas/core/common.py index_labels_to_array to "Transform label or iterable of labels to array, for use in Index." I see that the commit that created this (cf8d776) specifically wanted tuples to "pass single label to drop" and never an array of values.

I think this goes back to what you were saying about having support for tuples in an Index by pandas. This is my personal interpretation.

1. Tuple Supported in Index - Either it allows both options and it differentiate whether the tuple is supposed to be a single label or a list-like, or it is only allowed as a label. I think realistically there are many other list-likes which can be used.
2. Tuple Not-Supported in Index - Since tuples should not be value in an Index it always treat them as a list-like.

What do you think?

def index_labels_to_array(labels, dtype: NpDtype | None = None) -> np.ndarray:
    """
        Transform label or iterable of labels to array, for use in Index.

    Parameters
    ----------
    dtype : dtype
        If specified, use as dtype of the resulting array, otherwise infer.

    Returns
    -------
    array
    """
    if isinstance(labels, (str, tuple)):
        labels = [labels]

    if not isinstance(labels, (list, np.ndarray)):
        try:
            labels = list(labels)
        except TypeError:  # non-iterable
            labels = [labels]

    labels = asarray_tuplesafe(labels, dtype=dtype)

    return labels

[phofl]

I don not think that this is desireable. If we interpret tuples as list like we would run into value dependent behavior, because in case of MultiIndexes, the tuple elements correspond to levels

df = pd.DataFrame([[1, 2, 3], [4, 5, 6]], index=MultiIndex.from_arrays([[10, 20], [30, 40]]))
df.drop(index=(10, 30))

[ghost]

@phofl Yah, I see how this is intended for MultiIndexes now in https://pandas.pydata.org/pandas-docs/stable/user_guide/advanced.html and pandas/tests/frame/methods/test_drop.py::TestDataFrameDrop::test_mixed_depth_drop.

I can close this is everyone is okay.

[phofl]

Yes agreed

[rhshadrach]

I think docs still need clarification?

[ghost]

@rhshadrach What did you have in mind? Would love to help if I can.

I did notice a few places where I think we can cleanly add clarification such as the Docstring and "Drop columns and/or rows of MultiIndex DataFrame" Examples. #42777

[rhshadrach]

@mikephung122 The documentation seems to me to indicate that tuples will behave like multiple labels:

labels: single label or list-like
Index or column labels to drop.

It think it should be indicated that a tuple will not be treated as a list like here be changing the 2nd line to:

Index or column labels to drop. A tuple will be used as a single label an not treated as a list-like.

Would you be interested in putting up a PR to fix?

[ghost]

@rhshadrach Yah of course, I can take this.

I also noticed that the reference to https://pandas.pydata.org/pandas-docs/stable/user_guide/advanced.html in the Docstring https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.drop.html isn't working as a link.

Do you think its worth adding to the Docstring and Examples as well? Or did you only want to modify it in the labels parameter description like you specified.

[ghost]

take

[rhshadrach]

An example seems like a good idea here to me.