Skip to content

BUG: arithmetic ops not propagating mask when combining mask arrays and numpy #50322

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Dec 19, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v2.0.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -804,6 +804,7 @@ Timezones
Numeric
^^^^^^^
- Bug in :meth:`DataFrame.add` cannot apply ufunc when inputs contain mixed DataFrame type and Series type (:issue:`39853`)
- Bug in arithmetic operations on :class:`Series` not propagating mask when combining masked dtypes and numpy dtypes (:issue:`45810`, :issue:`42630`)
- Bug in DataFrame reduction methods (e.g. :meth:`DataFrame.sum`) with object dtype, ``axis=1`` and ``numeric_only=False`` would not be coerced to float (:issue:`49551`)
- Bug in :meth:`DataFrame.sem` and :meth:`Series.sem` where an erroneous ``TypeError`` would always raise when using data backed by an :class:`ArrowDtype` (:issue:`49759`)

Expand Down
6 changes: 5 additions & 1 deletion pandas/core/arrays/masked.py
Original file line number Diff line number Diff line change
Expand Up @@ -609,9 +609,13 @@ def _propagate_mask(
if other is libmissing.NA:
# GH#45421 don't alter inplace
mask = mask | True
elif is_list_like(other) and len(other) == len(mask):
mask = mask | isna(other)
else:
mask = self._mask | mask
return mask
# Incompatible return value type (got "Optional[ndarray[Any, dtype[bool_]]]",
# expected "ndarray[Any, dtype[bool_]]")
return mask # type: ignore[return-value]

def _arith_method(self, other, op):
op_name = op.__name__
Expand Down
19 changes: 19 additions & 0 deletions pandas/tests/series/test_arithmetic.py
Original file line number Diff line number Diff line change
Expand Up @@ -307,6 +307,25 @@ def test_arithmetic_with_duplicate_index(self):
expected = Series(Timedelta("9 hours"), index=[2, 2, 3, 3, 4])
tm.assert_series_equal(result, expected)

def test_masked_and_non_masked_propagate_na(self):
# GH#45810
ser1 = Series([0, np.nan], dtype="float")
ser2 = Series([0, 1], dtype="Int64")
result = ser1 * ser2
expected = Series([0, pd.NA], dtype="Float64")
tm.assert_series_equal(result, expected)

def test_mask_div_propagate_na_for_non_na_dtype(self):
# GH#42630
ser1 = Series([15, pd.NA, 5, 4], dtype="Int64")
ser2 = Series([15, 5, np.nan, 4])
result = ser1 / ser2
expected = Series([1.0, pd.NA, pd.NA, 1.0], dtype="Float64")
tm.assert_series_equal(result, expected)

result = ser2 / ser1
tm.assert_series_equal(result, expected)


# ------------------------------------------------------------------
# Comparisons
Expand Down