BUG: groupby apply does not work as expected if df is modified in the function

[47482-pawansingh]

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• (optional) I have confirmed this bug exists on the master branch of pandas.

---

Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example

# Creating some df
df = pd.DataFrame({'idx': [1,1,1,2,2], 
                        'ds': [20210101, 20210108, 20210115, 20210101, 20210108], 
                        'units':[1,2,3, 10,20]})

def grp_apply_bug(part_df):
    part_df.sort_values("ds", inplace=True)
    return part_df.units.sum()

print(df.groupby("idx").apply(lambda x: grp_apply_bug(x)))
>>> idx
      1    6
      2    6

# expected result
>>> idx
       1   6
       2   6

## If I modify the function to either copy the data or dont use `inplace=True`, apply works perfectly
def grp_apply_bug(part_df):
    part_df = part_df.sort_values("ds")
    return part_df.units.sum()

print(df.groupby("idx").apply(lambda x: grp_apply_bug(x)))
>>> idx
        1    6
        2  30
    

If the function which is called in the apply post groupby we use inplace=True, then apply ends up using the first group data for all the groups

[aggregation is wrong which will lead to unecessary bug]

Expected Output

>>> idx
        1  6
        2 30

Output of pd.show_versions()

INSTALLED VERSIONS

commit : 7d32926
python : 3.7.9.final.0
python-bits : 64
OS : Darwin
OS-release : 19.6.0
Version : Darwin Kernel Version 19.6.0: Tue Nov 10 00:10:30 PST 2020; root:xnu-6153.141.10~1/RELEASE_X86_64
machine : x86_64
processor : i386
byteorder : little
LC_ALL : None
LANG : None
LOCALE : None.UTF-8

pandas : 1.2.2
numpy : 1.20.1
pytz : 2021.1
dateutil : 2.8.1
pip : 21.0.1
setuptools : 49.6.0.post20210108
Cython : 0.29.14
pytest : 5.4.2
hypothesis : None
sphinx : 3.0.4
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : 2.8.5 (dt dec pq3 ext lo64)
jinja2 : 2.11.3
IPython : 7.20.0
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : 3.3.2
numexpr : None
odfpy : None
openpyxl : 3.0.6
pandas_gbq : None
pyarrow : None
pyxlsb : None
s3fs : None
scipy : 1.6.0
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : None

[phofl]

Hi, thanks for you're report. This is not intendet to work (see #12653). Objects should not be modified with apply.

[47482-pawansingh]

Any reason why it works when we modify an object and assign it to itself, but doesn't work when the same is done using inplace=True.

[phofl]

You are not modifying the underlying object in this case, while changing this inplace does. Assigning it to itself leaves the object we are grouping over unchanged