Non-fixed/variable size moving windows in .rolling and .ewm

[benjamin-dirtydirty-be]

Hi everyone.

I'm trying to code adaptive lookback indicators in Python/pandas.

import pandas as pd
import numpy as np

def ALB(src, period, TrendParam):
  change = src.diff(period).abs()
  volatility = src.diff().abs().rolling(window=period).sum()
  ER = change / volatility
  Alpha = 2.0 / (period + 1)
  VER = np.power((ER - (2 * ER - 1) / 2.0 * (1 - TrendParam) + 0.5), 2)
  VAlpha = Alpha * VER
  VLength = round((2.0 / VAlpha) - 1)
  return VLength

def percentrank(src, period):
  count = src.rolling(window=period, min_periods=1).apply(lambda x: np.count_nonzero(np.where(x[-1] > x[:-1])), raw=True)
  return 100 * (count / period)

time = pd.date_range(start='2020-01-17', periods=1000, freq='min', name='time')
price = pd.Index(['price'])
df = pd.DataFrame(np.random.rand(1000, 1), time, price)
df['ALB'] = ALB(src=df['price'], period=60, TrendParam=1).fillna(0).astype(int)
df['PCTRNK'] = percentrank(df['price'], period=df['ALB'])
print(df)

The code above results in a "ValueError: window must be an integer"

Traceback (most recent call last):
  File "<input>", line 22, in <module>
  File "<input>", line 15, in percentrank
  File "C:\Users\dd\PycharmProjects\rolling\venv\lib\site-packages\pandas\core\generic.py", line 10769, in rolling
    closed=closed,
  File "C:\Users\dd\PycharmProjects\rolling\venv\lib\site-packages\pandas\core\window.py", line 2849, in rolling
    return Rolling(obj, **kwds)
  File "C:\Users\dd\PycharmProjects\rolling\venv\lib\site-packages\pandas\core\window.py", line 93, in __init__
    self.validate()
  File "C:\Users\dd\PycharmProjects\rolling\venv\lib\site-packages\pandas\core\window.py", line 1738, in validate
    raise ValueError("window must be an integer")
ValueError: window must be an integer

When using a fixed size window of 60

df['PCTRNK'] = percentrank(df['price'], period=60)

I get the following output:

                        price  ALB     PCTRNK
time                                         
2020-01-17 00:00:00  0.728757    0   0.000000
2020-01-17 00:01:00  0.532295    0   0.000000
2020-01-17 00:02:00  0.899849    0   1.666667
2020-01-17 00:03:00  0.742664    0   1.666667
2020-01-17 00:04:00  0.334157    0   0.000000
                       ...  ...        ...
2020-01-17 16:35:00  0.781244  226  75.000000
2020-01-17 16:36:00  0.190402  240  16.666667
2020-01-17 16:37:00  0.505205  220  50.000000
2020-01-17 16:38:00  0.791318  233  80.000000
2020-01-17 16:39:00  0.241293  215  18.333333

I would like to use ALB as window (and min_periods) input for the .rolling part of the percentrank function, row by row.

I think it would make sense if we were able to use a non-fixed size moving window in .rolling and also in .ewm.
So ideally the parameters window and min_periods (in .rolling) could also accept Series.

This has already been mentioned in issue #26920. but the issue has been closed by @TomAugspurger, stating it looks like a duplicate of #23002.

I guess there is a (inefficient, naive) way of doing it, but I'm puzzled how to make it work.
A 'workaround' is mentioned here I think, but I'm still struggling.

Maybe someone can help?

[mroeschke]

If you download the 1.0rc, you can try out the new feature of defining your own BaseIndexer that allows you to define a function that dictates the start and end bounds of each rolling/ewm window.

https://pandas.pydata.org/pandas-docs/version/1.0.0/user_guide/computation.html#stats-custom-rolling-window

[benjamin-dirtydirty-be]

@mroeschke thank you for the link.

'Custom window rolling' is part of 1.0 now, but I'm also struggling with this new functionality.

Maybe someone knows how to tackle my problem, with the new Custom window rolling function or any other way?

[mroeschke]

You can check out the example in https://pandas.pydata.org/pandas-docs/version/1.0.0/user_guide/computation.html#stats-custom-rolling-window on how to create your custom windows.

For further usage questions, we recommend using StackOverflow. Going to close this issue for now.