Skip to content

fix Series.apply(..., by_row) #53584

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

topper-123
Copy link
Contributor

@topper-123 topper-123 commented Jun 10, 2023

Fixes #53400 (comment) by making the by_row param take no_default as its default parameter.

@rhshadrach
Copy link
Member

rhshadrach commented Jun 10, 2023

This moves in the right direction, but I think we still run into issues with the deprecation.

df = pd.DataFrame({'a': [1, 1, 2], 'b': [3, 4, 5]})

# On 2.0.x
print(df.apply([lambda x: 1]))
#          a        b
#   <lambda> <lambda>
# 0        1        1
# 1        1        1
# 2        1        1

# On main
print(df.apply([lambda x: 1]))
#           a  b
# <lambda>  1  1

Here, pandas is internally calling agg, which then splits up the DataFrame.agg into Series.agg on each column, and then this calls Series.apply. So we end up in Series.apply but with no way for the user to specify by_row=False.

One way around this would be to stop breaking up the DataFrame.agg into Series.agg calls and instead breakup the list func into individual DataFrame.agg calls, but then we run into #45658. If we try to do this without changing the behavior, we end up losing dtype information because of the transpose.

The only other way I see forward is to add a by_row argument to DataFrame.apply that is only used for list-likes (and maybe dict-likes?). This seems not straightforward from a user perspective. Maybe I'm overlooking another alternative?

@topper-123
Copy link
Contributor Author

I've made a new PR for this, see #53601, so I'm closing this.

@topper-123 topper-123 closed this Jun 11, 2023
@topper-123 topper-123 deleted the fix_series_Apply_by_row branch June 11, 2023 14:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants