API: astype("string") behavior

[simonjayhawkins]

in #39908 (comment) @jorisvandenbossche wrote

---

Bringing the "deferred" storage mode lookup for StringDtype discussion (originally here: #39908 (comment)) in the main thread, and trying to recap.

Currently, doing pd.StringDtype() (without specifying the storage), will already look up the option. In the default case, you get:

>>> pd.StringDtype().storage
'python'

which also means that pandas_dtype() already "fully initializes" the string dtype:

>>> pd.api.types.pandas_dtype("string")
string[python]

As a consequence, doing astype("string") will actually convert the values if your string dtype doesn't match the globab setting:

>>> s = pd.Series(['a', 'b'], dtype=pd.StringDtype(storage="pyarrow"))
>>> s.dtype
string[pyarrow]
>>> s.astype("string").dtype
string[python]

While I think it could make sense for .astype("string") to mean: "ensure I have a string dtype", and thus don't convert to another storage backend if I already had a string dtype to start with.

We do something similar for CategoricalDtype ("category" means a categorical dtype with no categories, but astype("category") does not remove your categories, it preserves any existing categorical dtype as is).

We could still have the astype("string") behave in the way I suggest by special casing this in the astype implementations (as suggested in #39908 (comment))), but I think that's something we would ideally avoid (any astype implementation accepting string dtype values as input would need to handle this case?)