Skip to content

TST: Tests for replace method when column contains pd.NA (#47480) #49805

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 11 commits into from
Closed
5 changes: 4 additions & 1 deletion pandas/core/missing.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,12 +88,15 @@ def mask_missing(arr: ArrayLike, values_to_mask) -> npt.NDArray[np.bool_]:

# GH 21977
mask = np.zeros(arr.shape, dtype=bool)
arr_na_mask = ~isna(arr)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we avoid doing this when not necessary? I think we only need this with object dtype

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vs-araujo can you do this

for x in nonna:
if is_numeric_v_string_like(arr, x):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pls keep this check. it'll avoid some warnings and will be a fastpath

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer if you pre-compute a mask in case we have object dtype before entering the loop.
Then you can add another if clause to only update the values that are not na

new_mask[~object_na_mask] = arr[~object_na_mask] == x

# GH#29553 prevent numpy deprecation warnings
pass
else:
new_mask = arr == x
# GH#47480
new_mask = np.zeros_like(arr, dtype=bool)
new_mask[arr_na_mask] = arr[arr_na_mask] == x
if not isinstance(new_mask, np.ndarray):
# usually BooleanArray
new_mask = new_mask.to_numpy(dtype=bool, na_value=False)
Expand Down
8 changes: 8 additions & 0 deletions pandas/tests/frame/methods/test_replace.py
Original file line number Diff line number Diff line change
Expand Up @@ -1503,6 +1503,14 @@ def test_replace_value_none_dtype_numeric(self, val):
result = df.replace({val: None})
tm.assert_frame_equal(result, expected)

@pytest.mark.parametrize("dtype", [None, "Int64", "Float64"])
def test_replace_df_containing_na(self, dtype):
# GH#47480
df = DataFrame({"A": [4, 1, pd.NA], "B": [1, 3, 2]}, dtype=dtype)
df.replace(to_replace=1, value=100, inplace=True)
expected = DataFrame({"A": [4, 100, pd.NA], "B": [100, 3, 2]}, dtype=dtype)
tm.assert_frame_equal(df, expected)


class TestDataFrameReplaceRegex:
@pytest.mark.parametrize(
Expand Down
8 changes: 8 additions & 0 deletions pandas/tests/series/methods/test_replace.py
Original file line number Diff line number Diff line change
Expand Up @@ -674,3 +674,11 @@ def test_replace_value_none_dtype_numeric(self, val):
result = ser.replace(val, None)
expected = pd.Series([1, None], dtype=object)
tm.assert_series_equal(result, expected)

@pytest.mark.parametrize("dtype", [None, "Int64", "Float64"])
def test_replace_series_containing_na(self, dtype):
# GH#47480
df = pd.Series([4, 1, pd.NA, 1], dtype=dtype)
df.replace(to_replace=1, value=100, inplace=True)
expected = pd.Series([4, 100, pd.NA, 100], dtype=dtype)
tm.assert_series_equal(df, expected)