Skip to content

Commit 3d594bc

Browse files
Licht-Txhochy
authored andcommitted
ARROW-2273: [Python] Raise NotImplementedError when pandas Sparse types serializing
This fixes [ARROW-2273](https://issues.apache.org/jira/browse/ARROW-2273). `pandas` Sparse types are planned to be deprecated in pandas future releases (pandas-dev/pandas#19239). `SparseDataFrame` and `SparseSeries` are naive implementation and have many bugs. IMO, this is not the right time to support these in `pyarrow`. Author: Licht-T <[email protected]> Closes #1997 from Licht-T/add-pandas-sparse-unsupported-msg and squashes the following commits: 64e24ce <Licht-T> ENH: Raise NotImplementedError when pandas Sparse types serializing pandas Sparse types are planned to be deprecated in pandas future releases. pandas-dev/pandas#19239
1 parent 5bdfff8 commit 3d594bc

File tree

1 file changed

+18
-0
lines changed

1 file changed

+18
-0
lines changed

python/pyarrow/serialization.py

+18
Original file line numberDiff line numberDiff line change
@@ -81,13 +81,31 @@ def _register_custom_pandas_handlers(context):
8181

8282
import pyarrow.pandas_compat as pdcompat
8383

84+
sparse_type_error_msg = (
85+
'{0} serialization is not supported.\n'
86+
'Note that {0} is planned to be deprecated '
87+
'in pandas future releases.\n'
88+
'See https://github.com/pandas-dev/pandas/issues/19239 '
89+
'for more information.'
90+
)
91+
8492
def _serialize_pandas_dataframe(obj):
93+
if isinstance(obj, pd.SparseDataFrame):
94+
raise NotImplementedError(
95+
sparse_type_error_msg.format('SparseDataFrame')
96+
)
97+
8598
return pdcompat.dataframe_to_serialized_dict(obj)
8699

87100
def _deserialize_pandas_dataframe(data):
88101
return pdcompat.serialized_dict_to_dataframe(data)
89102

90103
def _serialize_pandas_series(obj):
104+
if isinstance(obj, pd.SparseSeries):
105+
raise NotImplementedError(
106+
sparse_type_error_msg.format('SparseSeries')
107+
)
108+
91109
return _serialize_pandas_dataframe(pd.DataFrame({obj.name: obj}))
92110

93111
def _deserialize_pandas_series(data):

0 commit comments

Comments
 (0)