BUG: inconsisntent construction results between pd.array and pd.Series for dtype=str

[koizumihiroo]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd

pd.array([1, None], dtype=str)
# <NumpyExtensionArray>
# ['1', 'None']
# Length: 2, dtype: str128

pd.Series([1, None], dtype=str).array
# <NumpyExtensionArray>
# ['1', None]
# Length: 2, dtype: object

pd.DataFrame([1, None], dtype=str)[0].array
# <NumpyExtensionArray>
# ['1', None]
# Length: 2, dtype: object

Issue Description

While pd.Series returns object dtype when including None value in data for dtype=str, pd.array convert it to "None" string value. I am not sure this is intended or already discussed, but seems curious.

As investigating in construction process, pd.array pass dtype as is (str) but pd.Sereis call (via lib.ensure_string_array) dtype as "object"

using pd.array

# https://github.com/pandas-dev/pandas/blob/v2.2.1/pandas/core/arrays/numpy_.py#L127
result = np.asarray(scalars, dtype=dtype)

using pd.Sereis

# https://github.com/pandas-dev/pandas/blob/v2.2.1/pandas/core/construction.py#L800-L802
# lib.ensure_string_array(arr, convert_na_value=False, copy=copy).reshape(shape)
# https://github.com/pandas-dev/pandas/blob/v2.2.1/pandas/_libs/lib.pyx#L766
result = np.asarray(arr, dtype="object")

Expected Behavior

Although there might be historical reasons behind this divergent behavior, striving for consistency between pd.array and pd.Series for dtype=str would be more intuitive.

Installed Versions

/tmp/pandas-report/.venv/lib/python3.11/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils. warnings.warn("Setuptools is replacing distutils.")

INSTALLED VERSIONS

commit : bdc79c1
python : 3.11.6.final.0
python-bits : 64
OS : Linux
OS-release : 5.15.133.1-microsoft-standard-WSL2
Version : #1 SMP Thu Oct 5 21:02:42 UTC 2023
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : C.UTF-8
LOCALE : en_US.UTF-8

pandas : 2.2.1
numpy : 1.26.4
pytz : 2024.1
dateutil : 2.8.2
setuptools : 65.5.0
pip : 23.2.1
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 3.1.3
IPython : 8.22.1
pandas_datareader : None
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : None
bottleneck : None
dataframe-api-compat : None
fastparquet : None
fsspec : 2024.2.0
gcsfs : 2024.2.0
matplotlib : None
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : 15.0.0
pyreadstat : None
python-calamine : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : 2.0.27
tables : None
tabulate : None
xarray : None
xlrd : None
zstandard : None
tzdata : 2024.1
qtpy : None
pyqt5 : None

[koizumihiroo]

I also confirmed the issue on the main branch version '3.0.0.dev0+480.gc71244ad21' with NumPy version '1.26.4' (installing NumPy-2.0.0.dev0 with Pandas-3.0.0.dev0+480.gc71244ad21 resulted in pandas import error without pyarrow installation).

[DuarteMarques510]

take

[DuarteMarques510]

here's what i know, coming from a beginner in contributing to open-source, np.asarray is where the change/cast happens in the array, so we can't change anything in there. The only thing i can think of now is trying to cast each element and then create the numpy array with different parameters.

[DuarteMarques510]

and should a cast fail, the element should be kept as is

[DuarteMarques510]

but when it comes to a None, it can be casted to some dtypes (like string in the example above), that's why its possible to obtain the result with a 'None'

[DuarteMarques510]

i have developed a temporary solution but possibly not what one might look for, i separate the None's from the rest of the data and call np.asarray with dtype=<given_dtype> for those values, then i also generate an array for the None's by using np.asarray with dtype="object" only if there are None's in the given set. If there are None's the final output will state the dtype as object like in both pd.Series(...).array and pd.Dataframe(...)[0].array

[DuarteMarques510]

I have developed now a solution that could be the one… I’m moving on the testing phase before making a pull request

[DuarteMarques510]

i have now created a new test that will fail with the current version of the https://github.com/pandas-dev/pandas/blob/v2.2.1/pandas/core/arrays/numpy_.py#L116 of the _from_sequence classmethod and will pass under my solution but its compromising 3 other tests, working on it...

[DuarteMarques510]

I have addapted my solution for those compromised tests, and here's my veredict:
i had trouble passing the test where shared memory is asserted (https://github.com/pandas-dev/pandas/blob/main/pandas/tests/arrays/test_array.py#L269), after adapting, this is the result: for the example tested, there is no problem, but in order to process None correctly, shared memory will not be possible because of the separating of the none's from the rest of the values