Skip to content

CLN deprecate save&load in favour of to_pickle&read_pickle #3787

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 15, 2013
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions RELEASE.rst
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,8 @@ pandas 0.11.1
- removed ``Excel`` support to ``pandas.io.excel``
- added top-level ``pd.read_sql`` and ``to_sql`` DataFrame methods
- removed ``clipboard`` support to ``pandas.io.clipboard``
- replace top-level and instance methods ``save`` and ``load`` with top-level ``read_pickle`` and
``to_pickle`` instance method, ``save`` and ``load`` will give deprecation warning.
- the ``method`` and ``axis`` arguments of ``DataFrame.replace()`` are
deprecated
- Implement ``__nonzero__`` for ``NDFrame`` objects (GH3691_, GH3696_)
Expand Down
14 changes: 5 additions & 9 deletions doc/source/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,12 @@ Input/Output
Pickling
~~~~~~~~

.. currentmodule:: pandas.core.common
.. currentmodule:: pandas.io.pickle

.. autosummary::
:toctree: generated/

load
save
read_pickle

Flat File
~~~~~~~~~
Expand Down Expand Up @@ -378,8 +377,7 @@ Serialization / IO / Conversion
:toctree: generated/

Series.from_csv
Series.load
Series.save
Series.to_pickle
Series.to_csv
Series.to_dict
Series.to_sparse
Expand Down Expand Up @@ -601,8 +599,7 @@ Serialization / IO / Conversion
DataFrame.from_items
DataFrame.from_records
DataFrame.info
DataFrame.load
DataFrame.save
DataFrame.to_pickle
DataFrame.to_csv
DataFrame.to_hdf
DataFrame.to_dict
Expand Down Expand Up @@ -770,8 +767,7 @@ Serialization / IO / Conversion
:toctree: generated/

Panel.from_dict
Panel.load
Panel.save
Panel.to_pickle
Panel.to_excel
Panel.to_sparse
Panel.to_frame
Expand Down
40 changes: 0 additions & 40 deletions doc/source/basics.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1207,46 +1207,6 @@ While float dtypes are unchanged.
casted
casted.dtypes

.. _basics.serialize:

Pickling and serialization
--------------------------

All pandas objects are equipped with ``save`` methods which use Python's
``cPickle`` module to save data structures to disk using the pickle format.

.. ipython:: python

df
df.save('foo.pickle')

The ``load`` function in the ``pandas`` namespace can be used to load any
pickled pandas object (or any other pickled object) from file:


.. ipython:: python

load('foo.pickle')

There is also a ``save`` function which takes any object as its first argument:

.. ipython:: python

save(df, 'foo.pickle')
load('foo.pickle')

.. ipython:: python
:suppress:

import os
os.remove('foo.pickle')

.. warning::

Loading pickled data received from untrusted sources can be unsafe.

See: http://docs.python.org/2.7/library/pickle.html


Working with package options
----------------------------
Expand Down
37 changes: 37 additions & 0 deletions doc/source/io.rst
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ object.
* ``read_html``
* ``read_stata``
* ``read_clipboard``
* ``read_pickle``

The corresponding ``writer`` functions are object methods that are accessed like ``df.to_csv()``

Expand All @@ -50,6 +51,7 @@ The corresponding ``writer`` functions are object methods that are accessed like
* ``to_html``
* ``to_stata``
* ``to_clipboard``
* ``to_pickle``

.. _io.read_csv_table:

Expand Down Expand Up @@ -1442,7 +1444,42 @@ We can see that we got the same content back, which we had earlier written to th
You may need to install xclip or xsel (with gtk or PyQt4 modules) on Linux to use these methods.


.. _io.serialize:

Pickling and serialization
--------------------------

All pandas objects are equipped with ``to_pickle`` methods which use Python's
``cPickle`` module to save data structures to disk using the pickle format.

.. ipython:: python

df
df.to_pickle('foo.pkl')

The ``read_pickle`` function in the ``pandas`` namespace can be used to load
any pickled pandas object (or any other pickled object) from file:


.. ipython:: python

read_pickle('foo.pkl')

.. ipython:: python
:suppress:

import os
os.remove('foo.pkl')

.. warning::

Loading pickled data received from untrusted sources can be unsafe.

See: http://docs.python.org/2.7/library/pickle.html

.. note::

These methods were previously ``save`` and ``load``, now deprecated.

.. _io.excel:

Expand Down
3 changes: 2 additions & 1 deletion pandas/core/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
import numpy as np

from pandas.core.algorithms import factorize, match, unique, value_counts
from pandas.core.common import isnull, notnull, save, load
from pandas.core.common import isnull, notnull
from pandas.core.categorical import Categorical, Factor
from pandas.core.format import (set_printoptions, reset_printoptions,
set_eng_float_format)
Expand All @@ -28,6 +28,7 @@

# legacy
from pandas.core.daterange import DateRange # deprecated
from pandas.core.common import save, load # deprecated, remove in 0.12
import pandas.core.datetools as datetools

from pandas.core.config import get_option, set_option, reset_option,\
Expand Down
85 changes: 37 additions & 48 deletions pandas/core/common.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,6 @@
"""
Misc tools for implementing data structures
"""
# XXX: HACK for NumPy 1.5.1 to suppress warnings
try:
import cPickle as pickle
except ImportError: # pragma: no cover
import pickle

import itertools
from datetime import datetime
Expand Down Expand Up @@ -1668,49 +1663,6 @@ def _all_none(*args):
return True


def save(obj, path):
"""
Pickle (serialize) object to input file path

Parameters
----------
obj : any object
path : string
File path
"""
f = open(path, 'wb')
try:
pickle.dump(obj, f, protocol=pickle.HIGHEST_PROTOCOL)
finally:
f.close()


def load(path):
"""
Load pickled pandas object (or any other pickled object) from the specified
file path

Warning: Loading pickled data received from untrusted sources can be unsafe.
See: http://docs.python.org/2.7/library/pickle.html

Parameters
----------
path : string
File path

Returns
-------
unpickled : type of object stored in file
"""
try:
with open(path,'rb') as fh:
return pickle.load(fh)
except:
if not py3compat.PY3:
raise
with open(path,'rb') as fh:
return pickle.load(fh, encoding='latin1')

class UTF8Recoder:
"""
Iterator that reads an encoded stream and reencodes the input to UTF-8
Expand Down Expand Up @@ -2109,3 +2061,40 @@ def console_encode(object, **kwds):
"""
return pprint_thing_encoded(object,
get_option("display.encoding"))

def load(path): # TODO remove in 0.12
"""
Load pickled pandas object (or any other pickled object) from the specified
file path

Warning: Loading pickled data received from untrusted sources can be unsafe.
See: http://docs.python.org/2.7/library/pickle.html

Parameters
----------
path : string
File path

Returns
-------
unpickled : type of object stored in file
"""
import warnings
warnings.warn("load is deprecated, use read_pickle", FutureWarning)
from pandas.io.pickle import read_pickle
return read_pickle(path)

def save(obj, path): # TODO remove in 0.12
'''
Pickle (serialize) object to input file path

Parameters
----------
obj : any object
path : string
File path
'''
import warnings
warnings.warn("save is deprecated, use obj.to_pickle", FutureWarning)
from pandas.io.pickle import to_pickle
return to_pickle(obj, path)
27 changes: 22 additions & 5 deletions pandas/core/generic.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,29 @@ class PandasObject(object):
_AXIS_ALIASES = {}
_AXIS_NAMES = dict((v, k) for k, v in _AXIS_NUMBERS.iteritems())

def save(self, path):
com.save(self, path)
def to_pickle(self, path):
"""
Pickle (serialize) object to input file path

@classmethod
def load(cls, path):
return com.load(path)
Parameters
----------
path : string
File path
"""
from pandas.io.pickle import to_pickle
return to_pickle(self, path)

def save(self, path): # TODO remove in 0.12
import warnings
from pandas.io.pickle import to_pickle
warnings.warn("save is deprecated, use to_pickle", FutureWarning)
return to_pickle(self, path)

def load(self, path): # TODO remove in 0.12
import warnings
from pandas.io.pickle import read_pickle
warnings.warn("load is deprecated, use pd.read_pickle", FutureWarning)
return read_pickle(path)

def __hash__(self):
raise TypeError('{0!r} objects are mutable, thus they cannot be'
Expand Down
1 change: 1 addition & 0 deletions pandas/io/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,4 @@
from pandas.io.html import read_html
from pandas.io.sql import read_sql
from pandas.io.stata import read_stata
from pandas.io.pickle import read_pickle
48 changes: 48 additions & 0 deletions pandas/io/pickle.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# XXX: HACK for NumPy 1.5.1 to suppress warnings
try:
import cPickle as pickle
except ImportError: # pragma: no cover
import pickle

def to_pickle(obj, path):
"""
Pickle (serialize) object to input file path

Parameters
----------
obj : any object
path : string
File path
"""
f = open(path, 'wb')
try:
pickle.dump(obj, f, protocol=pickle.HIGHEST_PROTOCOL)
finally:
f.close()

def read_pickle(path):
"""
Load pickled pandas object (or any other pickled object) from the specified
file path

Warning: Loading pickled data received from untrusted sources can be unsafe.
See: http://docs.python.org/2.7/library/pickle.html

Parameters
----------
path : string
File path

Returns
-------
unpickled : type of object stored in file
"""
try:
with open(path,'rb') as fh:
return pickle.load(fh)
except:
from pandas.util import py3compat
if not py3compat.PY3:
raise
with open(path,'rb') as fh:
return pickle.load(fh, encoding='latin1')
1 change: 1 addition & 0 deletions pandas/sparse/tests/test_sparse.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@

from numpy import nan
import numpy as np
import pandas as pd
dec = np.testing.dec

from pandas.util.testing import (assert_almost_equal, assert_series_equal,
Expand Down
2 changes: 1 addition & 1 deletion pandas/tests/test_index.py
Original file line number Diff line number Diff line change
Expand Up @@ -1080,7 +1080,7 @@ def test_legacy_v2_unpickle(self):
pth, _ = os.path.split(os.path.abspath(__file__))
filepath = os.path.join(pth, 'data', 'mindex_073.pickle')

obj = com.load(filepath)
obj = pd.read_pickle(filepath)

obj2 = MultiIndex.from_tuples(obj.values)
self.assert_(obj.equals(obj2))
Expand Down
Loading