_combine_const() in pandas.sparse.frame does not have uniform method signature with pandas.core.frame

[Ritaja]

Code Sample

>>> import pandas as pd
>>> df = pd.DataFrame(np.zeros((4,5),dtype=int))
>>> sparse = df.to_sparse()
>>> sparse.loc[[1]] != 0

Expected Output

0 1 2 3 4
1 False False False False False

Observed Output

TypeError: _combine_const() got an unexpected keyword argument 'raise_on_error'

output of pd.show_versions()

INSTALLED VERSIONS
commit: None
python: 2.7.6.final.0
python-bits: 64
OS: Linux
OS-release: 4.2.0-35-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8

pandas: 0.18.0
nose: None
pip: 8.1.1
setuptools: 20.9.0
Cython: None
numpy: 1.10.4
scipy: 0.16.1
statsmodels: None
xarray: None
IPython: None
sphinx: 1.3.1
patsy: None
dateutil: 2.5.2
pytz: 2016.3
blosc: None
bottleneck: None
tables: None
numexpr: None
matplotlib: 1.3.1
openpyxl: None
xlrd: None
xlwt: 1.0.0
xlsxwriter: None
lxml: None
bs4: None
html5lib: 0.999
httplib2: 0.8
apiclient: None
sqlalchemy: 1.0.11
pymysql: None
psycopg2: 2.6.1 (dt dec pq3 ext lo64)
jinja2: 2.8
boto: None

Reason

pandas.core.frame implements _combine_const(self, other, func, raise_on_error=True)

while pandas.sparse.frame implements _combine_const(self, other, func)

and the missing argument in the signature is the cause of the error when using sparse frames. This is raised due to the call res = self._combine_const(other, func, raise_on_error=False) from _comp_method_FRAME at pandas.core.ops

[jreback]

yeah guess we don't actually have an issue for this.

[jreback]

pull-requests welcome!

[Ritaja]

I was thinking of working on this, however I have a small doubt: I noticed that pandas.sparse.frame has an unimplemented method astype.

usage:

in method _comp_method_FRAME we return res.fillna(True).astype(bool)

In scope of the current scenario, it would work to use astype from pandas.core.generic so we could do away with the unimplemented method in pandas.sparse.frame. I am sure it was put there for a reason, I just can't see an immediate usage due to my novice pandas background.

Any help is appreciated :)

[jreback]

sparse does not have too much support for many dtypes atm; its not very general. but it is moving in the right directlion mostly by @sinhrks. you are delving into a deep dark area of pandas.

[Ritaja]

@jreback I see. So how would you advice to address the current scenario, bypassing a complete solution for parse.sparse.frame astype().

[jreback]

I think @sinhrks would have better short-term advice.

[sinhrks]

I think we should have SparseDataFrame.astype to keep the impl same (xref #667). I have a draft of astype, and going to finish it by next release(0.18.2).

• https://github.com/sinhrks/pandas/tree/sparse_dtype

Until then, adding more tests is appreciated.

[Ritaja]

@sinhrks So the fix for inconsistent signatures should wait for the next release (0.18.2) ? Forgive me for paraphrasing.