Skip to content

Remove FrozenNDArray #29335

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from
Closed

Conversation

WillAyd
Copy link
Member

@WillAyd WillAyd commented Nov 2, 2019

  • closes #xxxx
  • tests added / passed
  • passes black pandas
  • passes git diff upstream/master -u -- "*.py" | flake8 --diff
  • whatsnew entry

@WillAyd WillAyd added the Deprecate Functionality to remove in pandas label Nov 2, 2019
@WillAyd WillAyd added this to the 1.0 milestone Nov 2, 2019
@WillAyd
Copy link
Member Author

WillAyd commented Nov 2, 2019

@jbrockmendel this is partially a culprit for the circular references you would get when trying to import pandas.core.reshape.concat. To try and frame the issue,

  1. pandas.core.base requires concat in some places, but if you try to move this to global imports THEN
  2. concat requires an import of pandas.core.index WHICH
  3. requires imports from pandas.core.indexes.frozen WHICH
  4. require the import of PandasObject from pandas.core.base

But PandasObject is not defined by the time step 4 comes around, hence the circular import

In a follow up could maybe question if FrozenList really needs to inherit from PandasObject, but will leave that to a separate discussion

Copy link
Member

@gfyoung gfyoung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really thought these changes would have been trivial, but I am somewhat confused by the many test failures that this removal triggered...

@jbrockmendel
Copy link
Member

Not having looked at the test failures yet, two thoughts: 1) im on board for removing this if we can and 2) not needing to subclass PandasObject is also plausible

@WillAyd
Copy link
Member Author

WillAyd commented Nov 7, 2019

Closing to clear queue but plan on coming back to this

@WillAyd WillAyd closed this Nov 7, 2019
@WillAyd WillAyd deleted the remove-frozenndarray branch January 16, 2020 00:33
@mspacek
Copy link

mspacek commented May 13, 2020

This seems to have caused the problem described here:

https://stackoverflow.com/questions/61641738/pandas-1-0-cannot-pickle-load-dict-containing-dataframe-with-multiindex

which I'm now also experiencing. I dumped a MultiIndex dataframe containing ndarrays to a pickle on disk under pandas 0.25.x in Python 3.6, and now I'm getting:

AttributeError: Can't get attribute 'FrozenNDArray' on <module 'pandas.core.indexes.frozen'

when trying to load it in pandas 1.0.3 (still on Python 3.6). Any suggestions/workarounds? Should I instead open up a new issue?

This solved issue seems related, but is for Python 2.7:

#31988

This comment in the original rationale for getting rid of FrozenNDArray mentions pandas.compat.pickle_compat.py, which seems relevant:

#9031 (comment)

@WillAyd
Copy link
Member Author

WillAyd commented May 13, 2020

Can you open a dedicated issue for that? I think need to define some pickle combat for that particular use case but can discuss on its own thread

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Deprecate Functionality to remove in pandas
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants