Sliding Window w/ Step Size & Get Back Windows

[drcrook1]

Code Sample, a copy-pastable example if possible

Below is likely a very poor implementation of this (it is super super slow; approximately 65 seconds for my data set with 75 windows to do and my implementation is also very memory intensive). Basically, I want to define a window size and a step size and get back a list of dataframes in order of appearance.

I need to use this because I will effectively be doing this twice. First I need to get my major windows and then in each window, I need to generate more, smaller windows as well as more data inside there.

Additionally, I need to be able to align my windows to the right, left or center and have an option to remove windows which do not have the correct sizes in them.

My dataframe is multi-indexed as well (the primary index is what I've been using to slice and that works great).

the df.rolling() seems very close but the primary draw back is I can not define a step size and I also have not been able to figure out how to get the actual windows out of it prefferably as a dataframe.

def get_slice(slicer, df, copy=True):
    vals = df.index.levels[0][:slicer]
    if(copy):
        return df.loc[pd.IndexSlice[vals,:], :].copy()
    else:
        return df.loc[pd.IndexSlice[vals,:], :]

def get_window_walk_list(df, n, slide_size, copy=True):
    """
    Takes a dataframe, a window size and a slide size.
    Returns a list of dataframes which have that size following that slice.
    """
    print("Entered window walk list")
    df.sort_index(ascending=True, inplace=True)
    list_df = [get_slice(i + n, df, copy) for i in range(0, len(df.index.levels[0]) - n, slide_size)]
    if(list_df[-1].shape[0] < df.shape[0]):
        print("appending latest")
        list_df.append(df[:].copy())
    list_df = list_df[::-1]
    return list_df

[WillAyd]

There's a lot going on here but I think this is more of a usage question than anything else, which would be better served for SO.

[drcrook1]

possibly, Ideally, I would like to have something like this:

windows_list = df.rolling(100, step_size=10).get_windows()

[WillAyd]

Ah OK fair enough though still keeping this closed as a duplicate of #15354 then. LMK if I am overlooking something

[drcrook1]

Ah; yeah; I saw that one. So it is like that one, except I also need to be able to actually access the windows from it as well.

Ideally it would also deal with multi-index and you specify which level it is that you care about for the window size and steps.

windows_list = df.rolling(100,step_size=10,level=0).get_windows()

Reason being I'm not applying simple mean, std etc over the windows. I've got some extremely complex computations which are happening against those windows and the sub windows which I'll create inside each window.

[WillAyd]

Any reason you can't just pass your function to .apply? Would be the idiomatic approach here IIUC