BUG: drop on axis with duplicate values doesn't raise when label is absent

[vinu2003]

Code Sample, a copy-pastable example if possible

>>> items
             item   name
store1      dress   V
store2       food   A
store2      water   C

CASE 1 : axis set to column and given a value which is not available in the structure. Gives ValueError.

>>> items.drop('abcd',axis=1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/site-packages/pandas/core/generic.py", line 2530, in drop
    obj = obj._drop_axis(labels, axis, level=level, errors=errors)
  File "/usr/local/lib/python3.6/site-packages/pandas/core/generic.py", line 2562, in _drop_axis
    new_axis = axis.drop(labels, errors=errors)
  File "/usr/local/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 3744, in drop
    labels[mask])
ValueError: labels ['abcd'] not contained in axis

CASE 2 : axis set to row, and given a value which is not available in structure should behave in the same way as the above case does.
Instead it just prints the structure table. Shouldn’t it be throwing ValueError ?

>>> items.drop('abcd',axis=0)
             item   name
store1      dress   V
store2       food   A
store2      water   C
>>> 

Problem description

When provided invalid column name(axis=1) it throws "ValueError". Same happens with row as well(axis=0). But when there are duplicate indices in the DataFrame it doesn't.

Expected Output

>>> items.drop('d',axis=0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/site-packages/pandas/core/generic.py", line 2530, in drop
    obj = obj._drop_axis(labels, axis, level=level, errors=errors)
  File "/usr/local/lib/python3.6/site-packages/pandas/core/generic.py", line 2562, in _drop_axis
    new_axis = axis.drop(labels, errors=errors)
  File "/usr/local/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 3744, in drop
    labels[mask])
ValueError: labels ['d'] not contained in axis

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.6.4.final.0 python-bits: 64 OS: Darwin OS-release: 17.3.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: en_AU.UTF-8 LOCALE: en_AU.UTF-8

pandas: 0.22.0
pytest: None
pip: 9.0.1
setuptools: 36.5.0
Cython: 0.27.3
numpy: 1.13.3
scipy: None
pyarrow: None
xarray: None
IPython: None
sphinx: None
patsy: None
dateutil: 2.6.1
pytz: 2017.3
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

[jorisvandenbossche]

@vinu2003 Thanks for the report! As you mentioned on the mailing list, this is triggered by the fact there is a duplicate index and is indeed a bug.
Related to #18561 (I also mentioned this bug there), but since it is not exactly the same keeping this open as a separate issue.

I reformatted your post a bit, please make sure in the future to use correct markdown syntax (so it 'looks' good, github provides some help if you edit/post a comment)

[jorisvandenbossche]

Reproducible example:

In [18]: df = pd.DataFrame({"A": [1, 1, 1], "B": [1, 1, 1]}, index=['a', 'b', 'c'])

In [19]: df.drop("d", axis=0)
...
ValueError: labels ['d'] not contained in axis

In [20]: df = pd.DataFrame({"A": [1, 1, 1], "B": [1, 1, 1]}, index=['a', 'b', 'b'])

In [21]: df.drop("d", axis=0)
Out[21]: 
   A  B
a  1  1
b  1  1
b  1  1

[aschade]

I'm going to take a look since I don't see any other PR's for this issue.

[vinu2003]

Thanks Joris.
I will make sure that i follow the markdown syntax from next time.

[vinu2003]

Hi Alex, Correct me if I am wrong. It should be "ValueError" as it raises with drop with axis=0 without duplicate index. So with duplicate as well, I thought it should raise "ValueError". Please correct me if I am kissing something here. Kind regards, Vinodhini Balusamy

…

Sent from my iPhone

On 14 Jan 2018, at 12:19 am, Alexander Michael Schade ***@***.***> wrote: @jorisvandenbossche Should this raise KeyError though? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[vinu2003]

I just saw your code changes Alex. So for both axis 0 and 1 , you are making as "KeyError" instead of "ValueError".

…

Sent from my iPhone

On 15 Jan 2018, at 4:06 pm, Vinodhini Balusamy ***@***.***> wrote: Hi Alex, Correct me if I am wrong. It should be "ValueError" as it raises with drop with axis=0 without duplicate index. So with duplicate as well, I thought it should raise "ValueError". Please correct me if I am kissing something here. Kind regards, Vinodhini Balusamy Sent from my iPhone > On 14 Jan 2018, at 12:19 am, Alexander Michael Schade ***@***.***> wrote: > > @jorisvandenbossche Should this raise KeyError though? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub, or mute the thread. >

[jorisvandenbossche]

Yes, there is discussion in the PR (#19230 (comment)) that KeyError would actually be better as ValueError

[vinu2003]

Yes. That makes sense.

…

Sent from my iPhone

On 15 Jan 2018, at 7:49 pm, Joris Van den Bossche ***@***.***> wrote: Yes, there is discussion in the PR (#19230 (comment)) that KeyError would actually be better as ValueError — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.