BUG: resample apply is actually aggregate

[endremborza]

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• (optional) I have confirmed this bug exists on the master branch of pandas.

---

this piece of code does it, I don't really understand why these things are OK to be the same function here

pandas/pandas/core/resample.py

Line 308 in 7ffbf1a

apply = aggregate

Code Sample, a copy-pastable example

df = pd.DataFrame([[1,2],[3,4], [5,6]], index=pd.date_range("2020-01-30", "2020-02-01"))

df.resample("1M").apply(lambda df: df.mean().mean())

Problem description

current behavior calls the function on all columns, in all groups, as in aggregate, but is should on all dfs of the groups as in apply

Expected Output

2020-01-31    2.5
2020-02-29    5.5
Freq: M, dtype: float64

Output of pd.show_versions()

INSTALLED VERSIONS

commit : 2a7d332
python : 3.8.5.final.0
python-bits : 64
OS : Linux
OS-release : 5.2.1-1.el7.elrepo.x86_64
Version : #1 SMP Sun Jul 14 08:15:04 EDT 2019
machine : x86_64
processor :
byteorder : little
LC_ALL : None
LANG : C.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.1.2
numpy : 1.19.1
pytz : 2020.1
dateutil : 2.8.1
pip : 20.1.1
setuptools : 47.1.0
Cython : None
pytest : 4.6.11
hypothesis : 5.29.0
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : 2.8.5 (dt dec pq3 ext lo64)
jinja2 : 2.11.2
IPython : 7.17.0
pandas_datareader: None
bs4 : None
bottleneck : 1.3.2
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : 3.3.1
numexpr : 2.7.1
odfpy : None
openpyxl : 3.0.5
pandas_gbq : None
pyarrow : None
pytables : None
pyxlsb : None
s3fs : None
scipy : 1.5.2
sqlalchemy : 1.3.19
tables : 3.6.1
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : None

[simonjayhawkins]

Thanks @endremborza for the report.

master gives the expected output

>>> pd.__version__
'1.3.0.dev0+42.g36c4d5ccf1'
>>>
>>> df = pd.DataFrame(
...     [[1, 2], [3, 4], [5, 6]], index=pd.date_range("2020-01-30", "2020-02-01")
... )
>>>
>>> df.resample("1M").apply(lambda df: df.mean().mean())
2020-01-31    2.5
2020-02-29    5.5
Freq: M, dtype: float64
>>>

[endremborza]

well, I don't understand how that's possible, as this is the same in master:

agg = aggregate
apply = aggregate

anyway, my bad for not checking. this seems to handle something similar:
055651b#diff-345e833a2b0533a66aee6d2440b4522f06f789321519218d88ff3add50894cf0

[endremborza]

However, as far as I can tell, the solution I linked in the commit does not solve this:

df.resample("1M").apply(lambda df: df.corr().mean())

as this would give TypeError, not Data, Key or Attribute, which are handled.

[simonjayhawkins]

master gives the expected output

hmm, I have bottleneck installed in my dev environment.

in a clean environment with 1.2.0rc0 gives the result in the op, installing bottleneck and gives expected output.

[slonik-az]

As of pandas version 1.3.2 Resampler's apply is still an alias for aggregate that caused me couple of hours of time trying to figure out why df.resample(...).apply(...) is semantically different from df.groupby(...).apply(...).

We already have agg as an alias for aggregate. Having apply as yet another alias for the same serves no useful purpose and makes for an inconsistent API.

Please, either fix apply or remove it from Resampler class altogether.

To see the problem for yourself with any version of Python:

>>> import pandas.core.resample
>>> pandas.core.resample.Resampler.apply is pandas.core.resample.Resampler.aggregate
True

Both functions are bound to the same object.

[simonjayhawkins]

As of pandas version 1.3.2 Resampler's apply is still an alias for aggregate

hence the open status of this issue

Please, either fix apply or remove it from Resampler class altogether.

pandas, like many open source projects, is nearly all volunteer

since there are 3000+ open issue most patches must come from the community

you are welcome to submit a Pull Request and the core team will review.

[jreback]

this alias exists for compatibility with grouping which uses the same convention - not even sure why this is a problem

this is unlikely to change

[endremborza]

what do you mean grouping has the same convention?

they are in no way aliases

[rhshadrach]

I think we should be working to differentiate the behavior between .agg and .apply across various pandas objects, and especially improve the behavior of .agg so that it reliably aggregates (xref: #35725).

Should we expose a resample(...).apply, similar to resample(...).agg but calling groupby's apply under the hood? Another alternative would be to deprecate resample(...).apply.

cc @jreback @mroeschke

[mroeschke]

Should we expose a resample(...).apply, similar to resample(...).agg but calling groupby's apply under the hood?

I would be OK with this in the hope for agg to always aggregate