Skip to content

BUG: df.groupby().aggregate() doesn't preserve subclass type #59667

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
3 tasks done
jaredight opened this issue Aug 30, 2024 · 3 comments
Closed
3 tasks done

BUG: df.groupby().aggregate() doesn't preserve subclass type #59667

jaredight opened this issue Aug 30, 2024 · 3 comments
Assignees
Labels
Apply Apply, Aggregate, Transform, Map Bug Closing Candidate May be closeable, needs more eyeballs Groupby Subclassing Subclassing pandas objects

Comments

@jaredight
Copy link

jaredight commented Aug 30, 2024

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

pandas as pd

class MySeries(pd.Series):
    pass

class MyDataFrame(pd.DataFrame):
    @property
    def _constructor(self):
        return MyDataFrame
    _constructor_sliced = MySeries

MySeries._constructor_expanddim = MyDataFrame

df = MyDataFrame({"a": reversed(range(10)), "b": list('aaaabbbccc')})

assert isinstance(df.groupby("b").agg({"a": "sum"}), MyDataFrame)

Traceback

AssertionError                            Traceback (most recent call last)
<ipython-input-7-e355635bac0a> in <cell line: 14>()
     12 df = MyDataFrame({"a": reversed(range(10)), "b": list('aaaabbbccc')})
     13 
---> 14 assert isinstance(df.groupby("b").agg({"a": "sum"}), MyDataFrame)

AssertionError: 

Issue Description

The df.groupby().agg() operation does not preserve subclass type. See this StackOverflow question for more details.

Other relevant issues:

Expected Behavior

Expected behavior is that df.groupby().agg() would be the same class type as df.

Installed Versions

INSTALLED VERSIONS

commit : d9cdd2e
python : 3.10.12.final.0
python-bits : 64
OS : Linux
OS-release : 6.1.85+
Version : #1 SMP PREEMPT_DYNAMIC Thu Jun 27 21:05:47 UTC 2024
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : en_US.UTF-8
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 2.2.2
numpy : 1.26.4
pytz : 2024.1
dateutil : 2.8.2
setuptools : 71.0.4
pip : 24.1.2
Cython : 3.0.11
pytest : 7.4.4
hypothesis : None
sphinx : 5.0.2
blosc : None
feather : None
xlsxwriter : None
lxml.etree : 4.9.4
html5lib : 1.1
pymysql : None
psycopg2 : 2.9.9
jinja2 : 3.1.4
IPython : 7.34.0
pandas_datareader : 0.10.0
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : 4.12.3
bottleneck : None
dataframe-api-compat : None
fastparquet : None
fsspec : 2024.6.1
gcsfs : 2024.6.1
matplotlib : 3.7.1
numba : 0.60.0
numexpr : 2.10.1
odfpy : None
openpyxl : 3.1.5
pandas_gbq : 0.23.1
pyarrow : 14.0.2
pyreadstat : None
python-calamine : None
pyxlsb : None
s3fs : None
scipy : 1.13.1
sqlalchemy : 2.0.32
tables : 3.8.0
tabulate : 0.9.0
xarray : 2024.6.0
xlrd : 2.0.1
zstandard : None
tzdata : 2024.1
qtpy : None
pyqt5 : None

@jaredight jaredight added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Aug 30, 2024
@mroeschke mroeschke added Groupby Subclassing Subclassing pandas objects Apply Apply, Aggregate, Transform, Map and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Aug 30, 2024
@AshmitGupta
Copy link

take

@rhshadrach
Copy link
Member

@jaredight - you get the desired behavior by overriding _constructor and _constructor_expanddim in MySeries. See here:

https://pandas.pydata.org/docs/development/extending.html#override-constructor-properties

@rhshadrach rhshadrach added the Closing Candidate May be closeable, needs more eyeballs label Sep 2, 2024
@mroeschke
Copy link
Member

Thanks for the report but as explained this is supported with the correct overrides so closing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Apply Apply, Aggregate, Transform, Map Bug Closing Candidate May be closeable, needs more eyeballs Groupby Subclassing Subclassing pandas objects
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants