Skip to content

Commit 69d29ba

Browse files
committed
Do not use bytes when loading back pickled objects
This causes an issue with unicode strings in py2. Instead I am using something similar to: https://github.com/pandas-dev/pandas/blob/master/pandas/io/pickle.py#L157 where the default is the better option rather than forcing it to use bytes.
1 parent 3a8d42a commit 69d29ba

File tree

1 file changed

+7
-6
lines changed

1 file changed

+7
-6
lines changed

arctic/store/_pickle_store.py

+7-6
Original file line numberDiff line numberDiff line change
@@ -61,13 +61,14 @@ def read(self, mongoose_lib, version, symbol, **kwargs):
6161
return pickle_compat_load(io.BytesIO(data))
6262
else:
6363
try:
64-
return pickle_compat_load(io.BytesIO(data), encoding='bytes')
64+
# The default encoding is ascii.
65+
return pickle_compat_load(io.BytesIO(data))
6566
except UnicodeDecodeError as ue:
66-
logger.error("Could not load Pickled object in PY3 with bytes encoding: %s" % ue)
67-
# Using encoding='latin1' is required for unpickling NumPy arrays and instances of datetime, date and
68-
# time pickled by Python 2: https://docs.python.org/3/library/pickle.html#pickle.load
69-
encoding = kwargs.get('encoding', 'latin_1') # Check if someone has manually specified encoding.
70-
return pickle_compat_load(io.BytesIO(data), encoding=encoding)
67+
# Using encoding='latin1' is required for unpickling NumPy arrays and instances of datetime, date
68+
# and time pickled by Python 2: https://docs.python.org/3/library/pickle.html#pickle.load
69+
logger.info("Could not Unpickle with ascii, Using latin1.")
70+
encoding = kwargs.get('encoding', 'latin_1') # Check if someone has manually specified encoding.
71+
return pickle_compat_load(io.BytesIO(data), encoding=encoding)
7172
return version['data']
7273

7374
@staticmethod

0 commit comments

Comments
 (0)