Skip to content

ENH: meth 'DataFrame.to_pickle' and func 'read_pickle' to accept URL GH#30163 #30301

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Jan 9, 2020

Conversation

suzutomato
Copy link
Contributor

@suzutomato suzutomato commented Dec 17, 2019

@jreback
Copy link
Contributor

jreback commented Dec 17, 2019

@suzutomato i see multiple PRs that you are opening and then closing
don’t open new ones if u make a change, rather push to the same one

this needs tests

Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needs tests

@jbrockmendel jbrockmendel added the IO Pickle read_pickle, to_pickle label Dec 17, 2019
@bashtage
Copy link
Contributor

It feels like getting pickles over networks is antithetical to their recommended use due to the security implications.

@suzutomato suzutomato changed the title BUG: func 'to_pickle' and 'read_pickle' where not accepting URL GH30299 BUG: func 'to_pickle' and 'read_pickle' where not accepting URL GH#30163 Dec 21, 2019
@suzutomato suzutomato requested a review from jreback December 25, 2019 21:18
@suzutomato
Copy link
Contributor Author

Sorry for the mess I've made, as well as the delay on this PR.
Tests were added.

2 Azure pipelines were failed/canceled due to timeout. I'll push agin later if that's the best way to rerun them.
Thank you,

@jreback
Copy link
Contributor

jreback commented Jan 1, 2020

can you merge master and ping on green.

@suzutomato
Copy link
Contributor Author

@jreback
Thank you for the comment, updated and merged.
Now all is green. ptal.

Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good a few comments, ping on green.

@pep8speaks
Copy link

pep8speaks commented Jan 4, 2020

Hello @suzutomato! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2020-01-09 12:17:29 UTC

@suzutomato
Copy link
Contributor Author

@jreback Thank you for review. Updated accordingly, please take a look. Let me know if I misunderstood anything.

@jreback jreback added this to the 1.0 milestone Jan 5, 2020
@TomAugspurger
Copy link
Contributor

It feels like getting pickles over networks is antithetical to their recommended use due to the security implications.

That was my initial reaction too. Do we want to encourage this?

@jreback
Copy link
Contributor

jreback commented Jan 6, 2020

It feels like getting pickles over networks is antithetical to their recommended use due to the security implications.

That was my initial reaction too. Do we want to encourage this?

this is a pretty common pattern actually;
security is out of scope as this is up to the user

@TomAugspurger
Copy link
Contributor

Fair enough. We do have a warning in the read_pickle docs. May want to re-emphasize that in the filepath_or_buffer section.

@suzutomato suzutomato requested a review from jreback January 7, 2020 18:31
Copy link
Contributor

@TomAugspurger TomAugspurger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, aside from a doc question.

Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. @TomAugspurger ok here?

@TomAugspurger
Copy link
Contributor

Fixed the merge conflict. Good to merge when CI passes.

@jreback jreback merged commit 1c3d64b into pandas-dev:master Jan 9, 2020
@jreback
Copy link
Contributor

jreback commented Jan 9, 2020

thanks @suzutomato very nice!

@suzutomato
Copy link
Contributor Author

Thank you @jreback and @TomAugspurger for guidance and your patience!
It was fun!

@suzutomato suzutomato deleted the bug30299-fix branch January 10, 2020 03:29
@suzutomato suzutomato changed the title BUG: func 'to_pickle' and 'read_pickle' where not accepting URL GH#30163 ENH: meth 'DataFrame.to_pickle' and func 'read_pickle' to accept URL GH#30163 Jan 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
IO Pickle read_pickle, to_pickle
Projects
None yet
Development

Successfully merging this pull request may close these issues.

pandas.read_pickle can take buffer object as well, not only str
7 participants