BUG: .loc/.iloc/.at/iat setitem coerces a pd.Series to np.ndarray with object dtype

[rosagold]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd


s1 = pd.Series(dtype='O', index=['a', 'b'])
s1['a'] = pd.Series()
s1.loc['b'] = pd.Series()
# >> s1
# a    Series([], dtype: float64)
# b                            []        <--------------- ?? wrong cast to `np.ndarray`
# dtype: object


# It works if the index is **not** specified
s2 = pd.Series(dtype='O', index=None)
s2['a'] = pd.Series()
s2.loc['b'] = pd.Series()
# >> s2
# a    Series([], dtype: float64)
# b    Series([], dtype: float64)
# dtype: object

Issue Description

A series with object dtype and a predefined index, cast items of type pd.Series to np.ndarray instead of preserving the series(-type). This apply to all accessors loc, iloc, at, iat and only works correctly with __setitem__..

Expected Behavior

The elements in a series of dtype object should not be casted in any case.

Installed Versions

INSTALLED VERSIONS

commit : 87cfe4e
python : 3.10.6.final.0
python-bits : 64
OS : Linux
OS-release : 5.15.0-48-generic
Version : #54-Ubuntu SMP Fri Aug 26 13:26:29 UTC 2022
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8
pandas : 1.5.0
numpy : 1.23.3
pytz : 2022.2.1
dateutil : 2.8.2
setuptools : 59.8.0
pip : 22.2.2
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
brotli : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : 3.6.0
numba : 0.56.2
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : 9.0.0
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : 1.9.1
snappy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
zstandard : None
tzdata : None

[rosagold]

BTW: The bug/issue was introduced with 1.5. In 1.4.4 and earlier versions it worked as expected.

[phofl]

commit 63ae9aac0555009b2e83bdb7a764d51c72929c90
Author: jbrockmendel <[email protected]>
Date:   Mon Jan 31 16:04:54 2022 -0800

    REF: use try/except pattern in Block.setitem, EABackedBlock.setitem (#45742)

cc @jbrockmendel
#45742

[rhkarls]

I came across this as well, trying to insert at Series in at a specified DataFrame index. Example if useful:

import pandas as pd

df = pd.DataFrame({'A':pd.Series(dtype=float),
                   'B': pd.Series(dtype=object)})

series = pd.Series(index=pd.date_range('2020-01-01','2020-01-02', freq='D'), data=1)

df.at[0,'A'] = 55.5 
df.at[0,'B'] = series 

With pandas 1.4.4:

df.at[0,'B']
Out: 
2020-01-01    1
2020-01-02    1
Freq: D, dtype: int64

With pandas 1.5.1:

df.at[0,'B']
Out: array([1, 1], dtype=int64)

Note that the 'A' column is needed, else it gives ValueError: Incompatible indexer with Series when executing df.at[0,'B'] = series, both for 1.4.4 and 1.5.x

[xr-chen]

take

[jbrockmendel]

#49306 may solve this, but folks, trying to put a Series object inside a Series/DataFrame is a recipe for a bad time.

[MarcoGorelli]

moving to the 2.1 milestone