pd.concat inconsistent with non-unique index

[martinfleis]

Code Sample, a copy-pastable example if possible

Following works:

df4 = pd.DataFrame([['bird', 'polly'], ['monkey', 'george'], ['a', 'b']],
                   columns=['animal', 'name'])
df1 = pd.DataFrame([['a', 1], ['b', 2]],
                   columns=['letter', 'number'])

df4.index = [0, 1, 1]

pd.concat([df1, df4], axis=1)

	letter	number	animal	name
0	a	1	bird	polly
1	b	2	monkey	george
1	b	2	a	b

But if the order of index values is the opposite, it raises ValueError:

df4 = pd.DataFrame([['bird', 'polly'], ['monkey', 'george'], ['a', 'b']],
                   columns=['animal', 'name'])
df1 = pd.DataFrame([['a', 1], ['b', 2]],
                   columns=['letter', 'number'])

df4.index = [1, 0, 0]

pd.concat([df1, df4], axis=1)

ValueError: Shape of passed values is (3, 4), indices imply (2, 4)

Problem description

As shown in the example, concat behaves inconsistently if index has duplicated value. It depends on the order of values if it succeeds or not. It is causing issues in geopandas (geopandas/geopandas#1223). I know that duplicated index is not ideal thing to have, but this seems to be a bug.

Expected Output

Expected output would be concatenated gdf as in the example with sorted index.

Output of pd.show_versions()

INSTALLED VERSIONS

commit : None
python : 3.8.0.final.0
python-bits : 64
OS : Darwin
OS-release : 19.2.0
machine : x86_64
processor : i386
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 0.25.3
numpy : 1.17.3
pytz : 2019.3
dateutil : 2.8.1
pip : 19.3.1
setuptools : 42.0.2.post20191201
Cython : None
pytest : 5.3.1
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.10.3
IPython : 7.10.1
pandas_datareader: None
bs4 : None
bottleneck : None
fastparquet : None
gcsfs : None
lxml.etree : None
matplotlib : 3.1.2
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
s3fs : None
scipy : 1.3.2
sqlalchemy : None
tables : None
xarray : None
xlrd : 1.2.0
xlwt : None
xlsxwriter : None

[charlesdong1991]

thanks for the issue, seems a duplicate of #20565

A PR is about to finish up to deal with it!

[phofl]

I think this behaves now as expected

   animal    name letter  number
0    bird   polly      a       1
1  monkey  george      b       2
1       a       b      b       2

for the raising case

[DEBADRIBASAK]

I see that this issue is still there in pandas version 1.2.3. I am looking for a contribution. Can I take this?