Skip to content

Non-Reducing Return Values in Rolling .apply #4130

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
hayd opened this issue Jul 5, 2013 · 3 comments
Closed

Non-Reducing Return Values in Rolling .apply #4130

hayd opened this issue Jul 5, 2013 · 3 comments
Labels
Apply Apply, Aggregate, Transform, Map Enhancement Window rolling, ewma, expanding

Comments

@hayd
Copy link
Contributor

hayd commented Jul 5, 2013

related #2813, #4964 (somewhat)

At the moment rolling_apply (et al) the passed function has to return a value (unlike apply which can return a Series and overall return a DataFrame).

If you could pass it a Series you could, for example, do this:

from pandas.core.algorithms import value_counts
s = Series([1, 1, 2, 3, 1, 2, 3, 2, 1])

pd.rolling_apply(s, 3, value_counts)

from the mailing list.

@jreback
Copy link
Contributor

jreback commented Jul 5, 2013

Here's the example, only issue is that you need to test first iteration to see what structure is returned

In [5]: s = Series([1, 1, 2, 3, 1, 2, 3, 2, 1])

In [6]: def f(x):
   ...:     c = x.value_counts()
   ...:     return 100*c/c.sum()
   ...: 

In [7]: concat([ f(s.iloc[i:i+3]) for i in range(len(s)-3) ],axis=1)
Out[7]: 
    0   1   2   3   4   5
1  66  33  33  33  33 NaN
2  33  33  33  33  33  66
3 NaN  33  33  33  33  33

@WillAyd
Copy link
Member

WillAyd commented Jul 6, 2018

A series can be produced now via the keyword argument raw=False, though the original issue does not work on master with a relatively cryptic message:

>>> 
s.rolling(3).apply(lambda x: x[:2], raw=False) 
TypeError: cannot convert the series to <class 'float'>

Not sure it is even feasible to support non-reducing functions like value_counts provided in the original message, but keeping open in case others have an idea on how to support

@WillAyd WillAyd changed the title ENH: return series in rolling apply Non-Reducing Return Values in Rolling .apply Jul 6, 2018
@datapythonista datapythonista modified the milestones: Contributions Welcome, Someday Jul 8, 2018
@mroeschke mroeschke added the Window rolling, ewma, expanding label Oct 20, 2019
@mroeschke mroeschke removed API Design Numeric Operations Arithmetic, Comparison, and Logical operations Reshaping Concat, Merge/Join, Stack/Unstack, Explode labels May 8, 2020
@mroeschke mroeschke added the Apply Apply, Aggregate, Transform, Map label Oct 5, 2020
@mroeschke mroeschke removed this from the Someday milestone Oct 13, 2022
@mroeschke
Copy link
Member

Looks like there hasn't been a lot of demand for this feature and would increase the complexity greatly if apply could return arbitrary shaped results for each window, so closing for now. Can reopen if there's more interest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Apply Apply, Aggregate, Transform, Map Enhancement Window rolling, ewma, expanding
Projects
None yet
Development

No branches or pull requests

6 participants