Skip to content

REF: Avoid special-casing in core.apply #39993

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
rhshadrach opened this issue Feb 23, 2021 · 3 comments
Open

REF: Avoid special-casing in core.apply #39993

rhshadrach opened this issue Feb 23, 2021 · 3 comments
Labels
Apply Apply, Aggregate, Transform, Map Refactor Internal refactoring of code

Comments

@rhshadrach
Copy link
Member

Ref #39935 (comment)

Currently in apply, there are many separate places where post-processing logic is applied. Once this is logic has been better consolidated, we can avoid special-casing certain ops (e.g. "size" for DataFrame).

@rhshadrach rhshadrach added Refactor Internal refactoring of code Apply Apply, Aggregate, Transform, Map labels Feb 23, 2021
@rhshadrach rhshadrach added this to the Someday milestone Feb 23, 2021
@jbrockmendel
Copy link
Member

Tangentially related: we've deprecated silent dropping of nuisance columns in groupby, resample, and DataFrame reductions, but apparently can also do so in core.apply. Would it make sense to deprecate it there too?

@rhshadrach
Copy link
Member Author

Yes, I think so.

@rhshadrach
Copy link
Member Author

@jbrockmendel I think we should also be doing this for transform. Here is an issue caused by fallback because of a typeerror:

df = pd.DataFrame(
    {'a' : ['x', 'y'], 'b' :['z', 'w'], 'c' : [1, 2], 'd' : [2., 5.]}
)
gb1 = df.groupby('a')
gb2 = gb1[['c', 'd']]
print(gb1.transform(lambda x: x.max() - x.min()))
print(gb2.transform(lambda x: x.max() - x.min()))
   c    d
0  0  0.0
1  0  0.0
     c    d
0  0.0  0.0
1  0.0  0.0

@mroeschke mroeschke removed this from the Someday milestone Oct 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Apply Apply, Aggregate, Transform, Map Refactor Internal refactoring of code
Projects
None yet
Development

No branches or pull requests

3 participants