Skip to content

BUG: rolling aggregate() with a list of functions along axis 1 raises ValueError #46132

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
3 tasks done
mvashishtha opened this issue Feb 23, 2022 · 5 comments
Open
3 tasks done
Assignees
Labels
Apply Apply, Aggregate, Transform, Map Bug Window rolling, ewma, expanding

Comments

@mvashishtha
Copy link

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
import numpy as np

df = pd.DataFrame([[1, 2], [3, 4]])
# rolling across axis=0 works fine:
print(df.rolling(window=2, axis=0, min_periods=1).aggregate([np.sum, np.mean]))
# This causes a ValueError
print(df.rolling(window=2, axis=1, min_periods=1).aggregate([np.sum, np.mean]))

Issue Description

I start with the dataframe:

   0  1
0  1  2
1  3  4

Rolling over axis=0 gives the rolling sum and mean for each column:

     0         1
   sum mean  sum mean
0  1.0  1.0  2.0  2.0
1  4.0  2.0  6.0  3.0

Rolling over axis=1 should give rolling sum and mean for each row, but instead raises a ValueError: No axis named 1 for object type Series:

Show stack trace
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
File ~/pandas/pandas/core/generic.py:535, in NDFrame._get_axis_number(cls, axis)
    534 try:
--> 535     return cls._AXIS_TO_AXIS_NUMBER[axis]
    536 except KeyError:

KeyError: 1

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
Input In [20], in <module>
      4 df = pd.DataFrame([[1, 2], [3, 4]])
      5 # rolling across axis=0 works fine:
----> 6 df.rolling(window=2, axis=1, min_periods=1).aggregate([np.sum, np.mean])

File ~/pandas/pandas/core/window/rolling.py:1765, in Rolling.aggregate(self, func, *args, **kwargs)
   1727 @doc(
   1728     _shared_docs["aggregate"],
   1729     see_also=dedent(
   (...)
   1763 )
   1764 def aggregate(self, func, *args, **kwargs):
-> 1765     return super().aggregate(func, *args, **kwargs)

File ~/pandas/pandas/core/window/rolling.py:635, in BaseWindow.aggregate(self, func, *args, **kwargs)
    634 def aggregate(self, func, *args, **kwargs):
--> 635     result = ResamplerWindowApply(self, func, args=args, kwargs=kwargs).agg()
    636     if result is None:
    637         return self.apply(func, raw=False, args=args, kwargs=kwargs)

File ~/pandas/pandas/core/apply.py:171, in Apply.agg(self)
    168     return self.agg_dict_like()
    169 elif is_list_like(arg):
    170     # we require a list, but not a 'str'
--> 171     return self.agg_list_like()
    173 if callable(arg):
    174     f = com.get_cython_func(arg)

File ~/pandas/pandas/core/apply.py:367, in Apply.agg_list_like(self)
    365 indices = []
    366 for index, col in enumerate(selected_obj):
--> 367     colg = obj._gotitem(col, ndim=1, subset=selected_obj.iloc[:, index])
    368     try:
    369         # Capture and suppress any warnings emitted by us in the call
    370         # to agg below, but pass through any warnings that were
    371         # generated otherwise.
    372         # This is necessary because of https://bugs.python.org/issue29672
    373         # See GH #43741 for more details
    374         with warnings.catch_warnings(record=True) as record:

File ~/pandas/pandas/core/window/rolling.py:289, in BaseWindow._gotitem(self, key, ndim, subset)
    284 if subset.ndim == 2 and (
    285     (is_scalar(key) and key in subset) or is_list_like(key)
    286 ):
    287     selection = key
--> 289 new_win = type(self)(subset, selection=selection, **kwargs)
    290 return new_win

File ~/pandas/pandas/core/window/rolling.py:141, in BaseWindow.__init__(self, obj, window, min_periods, center, win_type, axis, on, closed, method, selection)
    139 # TODO(2.0): Change this back to self.win_type once deprecation is enforced
    140 self._win_type = win_type
--> 141 self.axis = obj._get_axis_number(axis) if axis is not None else None
    142 self.method = method
    143 self._win_freq_i8 = None

File ~/pandas/pandas/core/generic.py:537, in NDFrame._get_axis_number(cls, axis)
    535     return cls._AXIS_TO_AXIS_NUMBER[axis]
    536 except KeyError:
--> 537     raise ValueError(f"No axis named {axis} for object type {cls.__name__}")

ValueError: No axis named 1 for object type Series

Expected Behavior

I expect the last rolling call to give a dataframe with the rolling sums and means of each row:

          0    1
0 sum   1.0  3.0
  mean  1.0  1.5
1 sum   3.0  7.0
  mean  3.0  3.5

Installed Versions

INSTALLED VERSIONS

commit : b4e578d
python : 3.9.10.final.0
python-bits : 64
OS : Darwin
OS-release : 21.3.0
Version : Darwin Kernel Version 21.3.0: Wed Jan 5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_X86_64
machine : x86_64
processor : i386
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.5.0.dev0+122.gb4e578db85
numpy : 1.22.1
pytz : 2021.3
dateutil : 2.8.2
pip : 22.0.3
setuptools : 60.8.2
Cython : 0.29.27
pytest : 7.0.0
hypothesis : None
sphinx : 4.4.0
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : 2.9.3
jinja2 : 3.0.3
IPython : 8.0.0
pandas_datareader: None
bs4 : None
bottleneck : None
fastparquet : None
fsspec : 2022.01.0
gcsfs : None
matplotlib : 3.5.1
numba : None
numexpr : 2.8.1
odfpy : None
openpyxl : 3.0.9
pandas_gbq : 0.16.0
pyarrow : 6.0.1
pyreadstat : None
pyxlsb : None
s3fs : 2022.01.0
scipy : 1.7.3
sqlalchemy : 1.4.31
tables : 3.7.0
tabulate : None
xarray : 0.20.2
xlrd : 2.0.1
xlwt : None
zstandard : None

@mvashishtha mvashishtha added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Feb 23, 2022
@mroeschke mroeschke added Apply Apply, Aggregate, Transform, Map Window rolling, ewma, expanding and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Feb 24, 2022
@mvashishtha mvashishtha changed the title rolling aggregate() with a list of functions along axis 1 raises ValueError BUG: rolling aggregate() with a list of functions along axis 1 raises ValueError Feb 24, 2022
@xr-chen
Copy link
Contributor

xr-chen commented Apr 11, 2022

take

@sqali
Copy link
Contributor

sqali commented Jan 11, 2023

take

@sqali
Copy link
Contributor

sqali commented Jan 11, 2023

Is this still raising errors or fixed?

@anmyachev
Copy link
Contributor

@jbrockmendel now I see another exception: NotImplementedError: axis other than 0 is not supported instead of ValueError.

I guess this is no longer a bug, but expected behavior. Should we close the issue?

@jbrockmendel
Copy link
Member

I'd expect it to be deprecated, not raise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Apply Apply, Aggregate, Transform, Map Bug Window rolling, ewma, expanding
Projects
None yet
6 participants