Skip to content

BUG: equals/assert_numpy_array_equals with non-singleton NAs #39650

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 8 commits into from
Feb 10, 2021
Merged

BUG: equals/assert_numpy_array_equals with non-singleton NAs #39650

merged 8 commits into from
Feb 10, 2021

Conversation

jbrockmendel
Copy link
Member

  • closes #xxxx
  • tests added / passed
  • Ensure all linting tests pass, see here for how to run them
  • whatsnew entry

jreback
jreback approved these changes Feb 7, 2021
View reviewed changes
Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm.

@jreback jreback added Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate Testing pandas testing functions or related to the test suite labels Feb 7, 2021
@jreback jreback added this to the 1.3 milestone Feb 7, 2021
@jbrockmendel jbrockmendel mentioned this pull request Feb 7, 2021
Merged
4 tasks
jorisvandenbossche
jorisvandenbossche reviewed Feb 8, 2021
View reviewed changes
Copy link
Member

@jorisvandenbossche jorisvandenbossche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this change behaviour of how NA-likes are currently handled in equals and assert methods?

Eg currently we have see None or np.nan as equivalent:

In [54]: pd.Series([1, None], dtype=object).equals(pd.Series([1, np.nan], dtype=object))
Out[54]: True

pandas/tests/series/methods/test_equals.py
right = Series([np.float64("NaN")], dtype=object)
assert left.equals(right)
assert Index(left).equals(Index(right))
assert left.array.equals(right.array)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also add tests for the case of non-matching NAs?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure

@jbrockmendel
Copy link
Member Author

Eg currently we have see None or np.nan as equivalent:

good catch, this changes that. will update to keep current behavior (though id be OK with deprecating it separately)

jreback
jreback reviewed Feb 8, 2021
View reviewed changes
pandas/_libs/lib.pyx Outdated
(x is None or is_nan(x)) and (y is None or is_nan(y))):
elif not (
PyObject_RichCompareBool(x, y, Py_EQ)
or is_matching_na(x, y)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of this an you add a keyword to is_matching_na? (actually would make it easier to deprecate then), maybe allow_equivalent=True?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated + greenish

jorisvandenbossche
jorisvandenbossche approved these changes Feb 10, 2021
View reviewed changes
Copy link
Member

@jorisvandenbossche jorisvandenbossche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need a whatsnew?

@jbrockmendel
Copy link
Member Author

whatsnew added + green

@jreback jreback merged commit 602502e into pandas-dev:master Feb 10, 2021
@jbrockmendel jbrockmendel deleted the bug-is_matching_na branch February 10, 2021 20:36
@simonjayhawkins simonjayhawkins mentioned this pull request Nov 1, 2021
Closed
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jreback jreback jreback approved these changes

@jorisvandenbossche jorisvandenbossche jorisvandenbossche approved these changes

Assignees
No one assigned
Labels
Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate Testing pandas testing functions or related to the test suite
Projects
None yet
Milestone
1.3
Development

Successfully merging this pull request may close these issues.
3 participants
@jbrockmendel @jreback @jorisvandenbossche