Inconsistent handling of duplicate axes + mixed dtypes in DataFrame.where

[batterseapower]

Code Sample, a copy-pastable example if possible

result = pd.DataFrame([
    [0, np.nan]
], columns=pd.Index(['A', 'A']))

mask = pd.DataFrame([[True, True]], *result.axes)

a = result.astype(object).where(mask) # works
b = result.astype('f8').where(mask) # works
c = result.T.where(mask.T).T # works
d = result.where(mask) # fails: "cannot reindex from a duplicate axis"

Problem description

It doesn't make sense that a, b and c work but d doesn't. The dtype of a column shouldn't affect whether or not masking suceeds.

Expected Output

a.astype('f8').equals(b.astype('f8')) and b.astype('f8').equals(c.astype('f8')) and c.astype('f8').equals(d.astype('f8'))

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.6.1.final.0 python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 63 Stepping 2, GenuineIntel byteorder: little LC_ALL: None LANG: None LOCALE: None.None

pandas: 0.24.1
pytest: 3.1.2
pip: 19.0.2
setuptools: 39.0.1
Cython: 0.27.2
numpy: 1.16.1
scipy: 1.2.1
pyarrow: 0.9.0
xarray: None
IPython: 6.1.0
sphinx: None
patsy: 0.4.1
dateutil: 2.8.0
pytz: 2018.9
blosc: None
bottleneck: 1.3.0.dev0
tables: 3.4.4
numexpr: 2.6.9
feather: None
matplotlib: 2.2.2
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml.etree: 3.8.0
bs4: 4.6.0
html5lib: 0.9999999
sqlalchemy: 1.1.11
pymysql: None
psycopg2: None
jinja2: 2.9.6
s3fs: None
fastparquet: 0.1.5
pandas_gbq: None
pandas_datareader: None
gcsfs: None

[gfyoung]

cc @toobaz

@batterseapower : What is the display function that you are calling in your first two "working" examples?

[batterseapower]

@gfyoung ah - that’s from Ipython and not important. I’ve removed it from the example.

[mroeschke]

This looks okay on master. Could use a test

In [10]:     result = pd.DataFrame([
    ...:     [0, np.nan]
    ...: ], columns=pd.Index(['A', 'A']))
    ...:
    ...: mask = pd.DataFrame([[True, True]], *result.axes)
    ...:
    ...: a = result.astype(object).where(mask) # works
    ...: b = result.astype('f8').where(mask) # works
    ...: c = result.T.where(mask.T).T # works
    ...: d = result.where(mask)

In [11]: d
Out[11]:
   A   A
0  0 NaN

[Bhavay-2001]

hey @mroeschke, I would like to work on this issue please can u help me with that. Thanks

[mroeschke]

@Bhavay192 you can view our contributing guide here https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#contributing-your-changes-to-pandas

[Bhavay-2001]

Hey @mroeschke, thanks for replying could you please tell me what tests needs to be performed??

[mroeschke]

A unit test needs to be written for the code above following this guide: https://pandas.pydata.org/pandas-docs/stable/development/contributing_codebase.html#writing-tests

[mitlabence]

take