Skip to content

REGR: ufunc with DataFrame input not passing all kwargs #40878

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Apr 12, 2021
1 change: 1 addition & 0 deletions doc/source/whatsnew/v1.2.4.rst
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ Fixed regressions
- Fixed regression in :meth:`DataFrame.where` not returning a copy in the case of an all True condition (:issue:`39595`)
- Fixed regression in :meth:`DataFrame.replace` raising ``IndexError`` when ``regex`` was a multi-key dictionary (:issue:`39338`)
- Fixed regression in repr of floats in an ``object`` column not respecting ``float_format`` when printed in the console or outputted through :meth:`DataFrame.to_string`, :meth:`DataFrame.to_html`, and :meth:`DataFrame.to_latex` (:issue:`40024`)
- Fixed regression in ``numpy`` ufuncs such as ``np.add`` not passing through all arguments for 2-dimensional input (:issue:`40662`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Fixed regression in ``numpy`` ufuncs such as ``np.add`` not passing through all arguments for 2-dimensional input (:issue:`40662`)
- Fixed regression in NumPy ufuncs such as ``np.add`` not passing through all arguments for 2-dimensional input (:issue:`40662`)

and maybe replace 2-dimensional input with :class:DataFrame?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep will do


.. ---------------------------------------------------------------------------

Expand Down
2 changes: 1 addition & 1 deletion pandas/core/arraylike.py
Original file line number Diff line number Diff line change
Expand Up @@ -357,7 +357,7 @@ def reconstruct(result):
# * len(inputs) > 1 is doable when we know that we have
# aligned blocks / dtypes.
inputs = tuple(np.asarray(x) for x in inputs)
result = getattr(ufunc, method)(*inputs)
result = getattr(ufunc, method)(*inputs, **kwargs)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the else clause is reached for a DataFrame when not (len(inputs) > 1 or ufunc.nout > 1).

The result = mgr.apply(getattr(ufunc, method)) there for __call__ also fails to pass along **kwargs

Is it straightforward to add to the paramterised here to exercise that path.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, I think so...will let you know if I run into any issues

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a test, but had to xfail because out is not written to correctly. The written result becomes transposed with the block-wise apply, I'm guessing because of the following relationship:

arr = np.array([[1, 2], [3, 4]])
df = pd.DataFrame(arr)
print(df.values)
print(df._mgr.blocks[0].values)

gives

[[1 2]
 [3 4]]
[[1 3]
 [2 4]]

Copy link
Member

@jorisvandenbossche jorisvandenbossche Apr 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, basically when out is specified, I think we should simply not call mgr.apply, see #39275, #39260 for a recent similar case where certain additional keyword arguments cannot be handled on a block-by-block case.

elif self.ndim == 1:
# ufunc(series, ...)
inputs = tuple(extract_array(x, extract_numpy=True) for x in inputs)
Expand Down
16 changes: 16 additions & 0 deletions pandas/tests/frame/test_ufunc.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,22 @@ def test_binary_input_dispatch_binop(dtype):
tm.assert_frame_equal(result, expected)


def test_ufunc_passes_args():
# GH#40662
arr = np.array([[1, 2], [3, 4]])
df = pd.DataFrame(arr)
result = np.zeros_like(df)
np.add(df, 1, out=result)

expected = arr + 1
tm.assert_numpy_array_equal(result, expected)

result = np.zeros_like(df)
np.add(df, 1, out=result, where=[[False, True], [True, False]])
expected = np.array([[0, 3], [4, 0]])
tm.assert_numpy_array_equal(result, expected)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you also test the return type/values of the operation itself.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done



@pytest.mark.parametrize("dtype_a", dtypes)
@pytest.mark.parametrize("dtype_b", dtypes)
def test_binary_input_aligns_columns(request, dtype_a, dtype_b):
Expand Down