BUG: array ufuncs with more than 2 inputs fails with 2 or more pd.Series and scalar arguments

[DocOtak]

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• (optional) I have confirmed this bug exists on the master branch of pandas.

---

Code Sample, a copy-pastable example

(note that this requires the gsw library to be installed)

>>> import gsw
>>> import pandas as pd
>>> gsw.SA_from_SP(pd.Series([30,30,30]), pd.Series([1000,1000,1000]), 0, 0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/abarna/.pyenv/versions/gsw/lib/python3.8/site-packages/gsw/_utilities.py", line 62, in wrapper
    ret = f(*newargs, **kw)
  File "/Users/abarna/.pyenv/versions/gsw/lib/python3.8/site-packages/gsw/_wrapped_ufuncs.py", line 3245, in SA_from_SP
    return _gsw_ufuncs.sa_from_sp(SP, p, lon, lat)
  File "/Users/abarna/.pyenv/versions/gsw/lib/python3.8/site-packages/pandas/core/generic.py", line 1935, in __array_ufunc__
    return arraylike.array_ufunc(self, ufunc, method, *inputs, **kwargs)
  File "/Users/abarna/.pyenv/versions/gsw/lib/python3.8/site-packages/pandas/core/arraylike.py", line 284, in array_ufunc
    raise NotImplementedError(
NotImplementedError: Cannot apply ufunc <ufunc 'sa_from_sp'> to mixed DataFrame and Series inputs.

Problem description

My group uses pandas with the gsw library to do some oceanographic calculations. gsw provides ufuncs to do these calculations. When more than one pd.Series is mixed with the scalar arguments at the end, it results in the NotImplementedError you see above. Also, gsw library itself was only recently updated to defer to the input's __array_ufunc__ method if it is present.

Expected Output

This is the result of some manual broadcasting:

>>> gsw.SA_from_SP(pd.Series([30,30,30]), pd.Series([1000,1000,1000]), pd.Series([0, 0, 0]), pd.Series([0, 0, 0]))
0    30.145599
1    30.145599
2    30.145599
dtype: float64

Output of pd.show_versions()

INSTALLED VERSIONS

commit : 7d32926
python : 3.8.2.final.0
python-bits : 64
OS : Darwin
OS-release : 20.3.0
Version : Darwin Kernel Version 20.3.0: Thu Jan 21 00:07:06 PST 2021; root:xnu-7195.81.3~1/RELEASE_X86_64
machine : x86_64
processor : i386
byteorder : little
LC_ALL : en_US.UTF-8
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.2.2
numpy : 1.20.1
pytz : 2021.1
dateutil : 2.8.1
pip : 19.2.3
setuptools : 41.2.0
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : None

[mroeschke]

Sorry for the delay. Would you be able to reproduce this with a more minimal example without the gsw library? If there is a bug, we would need to be able to add a regression test without this dependency

[DocOtak]

Hi @mroeschke no worries about the delay. Here is a reproducible example using only numpy and pandas. I do need to make a toy ufunc to demonstrate this:

Setup:

import numpy as np
import pandas as pd

def add3(x,y,z):
    return x + y + z
ufunc = np.frompyfunc(add3, 3, 1)

s = pd.Series(np.array([1,2]))

Demonstration that it works as expected:

>>> ufunc(s, 1, 1)

0    3
1    4
dtype: object

And the failing case:

>>> ufunc(s, s, 1)
---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
/var/folders/dl/m9b7rnmj455_fz5kdwgdbbcr0000gn/T/ipykernel_85664/928580685.py in <module>
      7 
      8 s = pd.Series(np.array([1,2]))
----> 9 ufunc(s, s, 1)

~/.pyenv/versions/3.9.6/envs/jupyter/lib/python3.9/site-packages/pandas/core/generic.py in __array_ufunc__(self, ufunc, method, *inputs, **kwargs)
   2027         self, ufunc: np.ufunc, method: str, *inputs: Any, **kwargs: Any
   2028     ):
-> 2029         return arraylike.array_ufunc(self, ufunc, method, *inputs, **kwargs)
   2030 
   2031     # ideally we would define this to avoid the getattr checks, but

~/.pyenv/versions/3.9.6/envs/jupyter/lib/python3.9/site-packages/pandas/core/arraylike.py in array_ufunc(self, ufunc, method, *inputs, **kwargs)
    290             # well. Previously this raised an internal ValueError. We might
    291             # support it someday, so raise a NotImplementedError.
--> 292             raise NotImplementedError(
    293                 "Cannot apply ufunc {} to mixed DataFrame and Series "
    294                 "inputs.".format(ufunc)

NotImplementedError: Cannot apply ufunc <ufunc 'add3 (vectorized)'> to mixed DataFrame and Series inputs.

[mroeschke]

Great thanks for the minimal example!

[miettal]

I face same issue. Is there workaround other than version fixing.

[as-ec]

Hi, I also have the same issue using np.clip and passing in a number of series. As a result I'm stuck in my upgrade path at pandas 1.1.5.

Is there a plan to fix this in the pipeline of work or convenient reformulation that will work as a work around? I definitely don't know enough to submit a PR for a fix. Sorry.

Thanks for your help,

Andy

[jreback]

@as-ec bugs get fixed when the community submit pull requests

the core team can provide review

[anthonyaag]

If approved in time, would it be possible to get the above fix also on 1.5.1?