BUG: Handle NA in assert_numpy_array_equal #31910
Conversation
pandas/_libs/lib.pyx
Outdated
| @@ -571,6 +571,11 @@ def array_equivalent_object(left: object[:], right: object[:]) -> bool: | |||
| if PyArray_Check(x) and PyArray_Check(y): | |||
| if not array_equivalent_object(x, y): | |||
| return False | |||
| elif x is C_NA or y is C_NA: | |||
There was a problem hiding this comment.
I think you can just simplify by doing an XOR and returning False when that is the case. PyObject_RichCompareBool on the next condition will return True when both objects have the same id anyway
doc/source/whatsnew/v1.0.2.rst
Outdated
| @@ -32,6 +32,10 @@ Bug fixes | |||
|
|
|||
| - Using ``pd.NA`` with :meth:`DataFrame.to_json` now correctly outputs a null value instead of an empty object (:issue:`31615`) | |||
|
|
|||
| **Testing** | |||
|
|
|||
| - Fixed bug where :meth:`assert_numpy_array_equal` would raise a ``TypeError`` when encountering ``pd.NA`` (:issue:`31881`) | |||
There was a problem hiding this comment.
This one actually isn't exposed via API so can remove note
| @@ -175,3 +177,18 @@ def test_numpy_array_equal_copy_flag(other_type, check_same): | |||
| tm.assert_numpy_array_equal(a, other, check_same=check_same) | |||
| else: | |||
| tm.assert_numpy_array_equal(a, other, check_same=check_same) | |||
|
|
|||
|
|
|||
| def test_numpy_array_equal_contains_na(): | |||
There was a problem hiding this comment.
Nice - can you also add a test using nulls_fixture that ensures equality passes?
| def test_numpy_array_equal_identical_na(nulls_fixture): | ||
| a = np.array([nulls_fixture]) | ||
|
|
||
| tm.assert_numpy_array_equal(a, a) |
There was a problem hiding this comment.
Different NA-likes are not considered equal? Eg np.array([np.nan]) vs np.array([pd.NA]) ?
I think for now it would be good to keep those not equal (since they behave differently), but if so, we should best add a test for that as well.
There was a problem hiding this comment.
Yes, those are comparing unequal, will add a test
| @@ -1,6 +1,8 @@ | |||
| import numpy as np | |||
| import pytest | |||
|
|
|||
| from pandas._libs.missing import NA | |||
There was a problem hiding this comment.
in tests you can import this from main pandas namespace
Co-Authored-By: Joris Van den Bossche <[email protected]>
Co-Authored-By: Joris Van den Bossche <[email protected]>
|
I think worth back porting to 1.0.2 |
Yes, it's internal only, but since other fixes that will be backported might want to rely on this testing functionality, let's backport |
|
@dsaxton Thanks! |
Co-authored-by: Daniel Saxton <[email protected]>
black pandasgit diff upstream/master -u -- "*.py" | flake8 --diffI think in the the case where we have NA we want to use identity rather than equality for checking when two arrays are equal.