Skip to content

REF: Paths through DataFrameGroupBy.agg #52362

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
rhshadrach opened this issue Apr 2, 2023 · 1 comment · May be fixed by #57706
Open

REF: Paths through DataFrameGroupBy.agg #52362

rhshadrach opened this issue Apr 2, 2023 · 1 comment · May be fixed by #57706
Labels
Apply Apply, Aggregate, Transform, Map Groupby Refactor Internal refactoring of code

Comments

@rhshadrach
Copy link
Member

In groupby, agg may take various paths. While some branching may be necessary, this can lead to inconsistent results and should be avoided if possible. The following methods take different paths to do the main part of the computation. This issue is to investigate these methods and reduce branching if possible.

Once the deprecation in #7155 is enforced, the apply method does not branch in this fashion. A similar issue for transform is #42907.

DataFrameGroupBy.agg

  • _python_agg_general
  • _aggregate_frame
  • DataFrameGroupBy.agg with func wrapped as a list

DataFrameGroupBy._python_agg_general

  • _ptyhon_apply_general
  • BaseGrouper.agg_series

cc @topper-123

@rhshadrach rhshadrach added Refactor Internal refactoring of code Groupby Apply Apply, Aggregate, Transform, Map labels Apr 2, 2023
@topper-123
Copy link
Contributor

I agree that this too complex currently and Pandas as project would benefit a lot from getting this simplified.

I haven't looked into the these methods lately so haven't any clear image of how much it possible to simplify this/cut down on paths without breaking backward compatibility, but I do think we need to look seriously into how to simplify these.

@rhshadrach rhshadrach linked a pull request Mar 2, 2024 that will close this issue
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Apply Apply, Aggregate, Transform, Map Groupby Refactor Internal refactoring of code
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants