ENH: Don't remove the ability to expand a Series to a DataFrame by applying a function that returns Series

[n-splv]

Feature Type

• Adding new functionality to pandas

• Changing existing functionality in pandas

• Removing existing functionality in pandas

Problem Description

Being able to expand a Series by applying a function to its cells is very convenient, but in 2.1 in produces a
FutureWarning: Returning a DataFrame from Series.apply when the supplied function returns a Series is deprecated and will be removed in a future version

Feature Description

Right now I can do this:

def expand_cell(s: Union[str, np.ndarray, list]) -> pd.Series:

    result = process_cell(s)
    data = {
        'foo': result.foo,
        'bar': result.bar,
        ...
    }
    return pd.Series(data)
        
df = pd.concat([
    df,
    df[column].apply(expand_cell),
], axis=1)

I tried to get rid of the FutureWarning, but changing the expand_cell return value to be pd.Series(data).to_frame() didn't help, because in such case the result of apply() is a Series with each cell containing a DataFrame.

I might be missing something, but as of right now the only working solution which doesn't raise a Warning seems be the replacement of the concise df[column].apply(expand_cell) with a more ugly pd.DataFrame([expand_cell(c) for c in df[column]]).

Alternative Solutions

Keep everything as is, I guess? :)

Additional Context

No response

[lithomas1]

cc @topper-123

[rhshadrach]

Additional discussion on this is taking place on the PR #52123.

[topper-123]

There is also a more current discussion here: #52116.

The reason it was removed was partly that it didn't fit into how apply works in other contexts ( DataFrame.apply etc.), but also partly because it is very very slow (See perf example in OP in #52116). My recommendation is to do it this without apply e.g. in a loop and then concat the results, but that hasn't been communicated very well + there isn't 100 % agreement on this as seen in #52116.

Slightly related, we should really implement series.explode(axis=1), so series with listlikes could be easily be turned into dataframes. AFAIK there isn't easy functionality for doing that currently.

[rhshadrach]

I think we should revert the deprecation for the time being.

[miroSalted]

Are you going to remove the warning as well?