Skip to content

BUG: Handle NA in assert_numpy_array_equal #31910

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Feb 13, 2020
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions doc/source/whatsnew/v1.0.2.rst
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,10 @@ Bug fixes

- Using ``pd.NA`` with :meth:`DataFrame.to_json` now correctly outputs a null value instead of an empty object (:issue:`31615`)

**Testing**

- Fixed bug where :meth:`assert_numpy_array_equal` would raise a ``TypeError`` when encountering ``pd.NA`` (:issue:`31881`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one actually isn't exposed via API so can remove note


.. ---------------------------------------------------------------------------

.. _whatsnew_102.contributors:
Expand Down
5 changes: 5 additions & 0 deletions pandas/_libs/lib.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -571,6 +571,11 @@ def array_equivalent_object(left: object[:], right: object[:]) -> bool:
if PyArray_Check(x) and PyArray_Check(y):
if not array_equivalent_object(x, y):
return False
elif x is C_NA or y is C_NA:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can just simplify by doing an XOR and returning False when that is the case. PyObject_RichCompareBool on the next condition will return True when both objects have the same id anyway

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes, good idea

if x is C_NA and y is C_NA:
return True
else:
return False
elif not (PyObject_RichCompareBool(x, y, Py_EQ) or
(x is None or is_nan(x)) and (y is None or is_nan(y))):
return False
Expand Down
17 changes: 17 additions & 0 deletions pandas/tests/util/test_assert_numpy_array_equal.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
import numpy as np
import pytest

from pandas._libs.missing import NA
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in tests you can import this from main pandas namespace


from pandas import Timestamp
import pandas._testing as tm

Expand Down Expand Up @@ -175,3 +177,18 @@ def test_numpy_array_equal_copy_flag(other_type, check_same):
tm.assert_numpy_array_equal(a, other, check_same=check_same)
else:
tm.assert_numpy_array_equal(a, other, check_same=check_same)


def test_numpy_array_equal_contains_na():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice - can you also add a test using nulls_fixture that ensures equality passes?

# https://github.com/pandas-dev/pandas/issues/31881
a = np.array([True, False])
b = np.array([True, NA])

msg = """numpy array are different

numpy array values are different \\(50.0 %\\)
\\[left\\]: \\[True, False\\]
\\[right\\]: \\[True, <NA>\\]"""

with pytest.raises(AssertionError, match=msg):
tm.assert_numpy_array_equal(a, b)