Skip to content

BUG: pickling subset of Arrow-backed data would serialize the entire data #49078

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Oct 20, 2022
Merged

BUG: pickling subset of Arrow-backed data would serialize the entire data #49078

merged 4 commits into from
Oct 20, 2022

Conversation

mroeschke
Copy link
Member

@mroeschke mroeschke added IO Pickle read_pickle, to_pickle Arrow pyarrow functionality labels Oct 13, 2022
@mroeschke mroeschke added this to the 2.0 milestone Oct 13, 2022
phofl
phofl reviewed Oct 14, 2022
View reviewed changes
Copy link
Member

@phofl phofl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we bumpy the minimum pyarrow version so that combine_chunks is available everywhere? This would help in another pr too

@jbrockmendel
Copy link
Member

Should we bumpy the minimum pyarrow version so that combine_chunks is available everywhere?

For 2.0 let's bump it as high as we can.

@mroeschke mroeschke mentioned this pull request Oct 14, 2022
Merged
5 tasks
@mroeschke mroeschke merged commit 0e88f4f into pandas-dev:main Oct 20, 2022
@mroeschke mroeschke deleted the bug/pickle/pyarrow branch October 20, 2022 16:19
phofl pushed a commit to phofl/pandas that referenced this pull request Oct 21, 2022
@mroeschke @phofl
BUG: pickling subset of Arrow-backed data would serialize the entire …
5aab71e 
…data (pandas-dev#49078)

* BUG: pickling subset of Arrow-backed data would serialize the entire data

* Use data
@ian-r-rose ian-r-rose mentioned this pull request Oct 31, 2022
Closed
3 tasks
noatamir pushed a commit to noatamir/pandas that referenced this pull request Nov 9, 2022
@mroeschke @noatamir
BUG: pickling subset of Arrow-backed data would serialize the entire …
f015b19 
…data (pandas-dev#49078)

* BUG: pickling subset of Arrow-backed data would serialize the entire data

* Use data
@jrbourbeau jrbourbeau mentioned this pull request Dec 9, 2022
Merged
@jrbourbeau jrbourbeau mentioned this pull request Mar 13, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@phofl phofl phofl approved these changes

Assignees
No one assigned
Labels
Arrow pyarrow functionality IO Pickle read_pickle, to_pickle
Projects
None yet
Milestone
2.0
Development

Successfully merging this pull request may close these issues.

ENH: Serialize view of ArrowStringArray
3 participants
@mroeschke @jbrockmendel @phofl