ENH: Handle numpy-like arrays in assert_almost_equal

[znicholls]

• closes #xxxx
• tests added / passed
• Ensure all linting tests pass, see here for how to run them
• whatsnew entry

[znicholls]

At this point, the test fails because assert_almost_equal doesn't have any real way to support numpy-like (but not numpy) arrays. I've put in a tweak which gets numpy-like arrays to go to the right branch of assert_almost_equal, however the comparison fails in array_equivalent. The problem is that the inputs in array_equivalent are converted using np.asarray. After this operation, they are identified as object type (even though their dtype attribute isn't object), hence array_equivalent goes into the is_string_dtype branch so _array_equivalent_object tries to zip the things being compared and then things explode because a numpy-scalar isn't iterable.

[znicholls]

I added an __array__ method to the mock numpy-like object which seems to have fixed the issue in #39790 (comment). So, the question is, do the changes to testing made here look ok?

[jreback]

At this point, the test fails because assert_almost_equal doesn't have any real way to support numpy-like (but not numpy) arrays. I've put in a tweak which gets numpy-like arrays to go to the right branch of assert_almost_equal, however the comparison fails in array_equivalent. The problem is that the inputs in array_equivalent are converted using np.asarray. After this operation, they are identified as object type (even though their dtype attribute isn't object), hence array_equivalent goes into the is_string_dtype branch so _array_equivalent_object tries to zip the things being compared and then things explode because a numpy-scalar isn't iterable.

sound like you need to add a function which can return true if not is_array but has size / dtype; and then use it in array_equivalent. call it is_arraylike?

but stepping back here. I thought Pint is using the EA interface; if this is the case then no changes should be needed here at all. if this is not the case, then can you explain why?

[znicholls]

I thought Pint is using the EA interface; if this is the case then no changes should be needed here at all

Because the comparisons should be defined elsewhere?

In any case, it does appear this is the case. I think I've started at the wrong spot. I'll try again in a different MR (feel free to close for now, I can re-open if needed).

[znicholls]

@jreback I have dug deeper on this. The test in which this is an issue is

pandas/pandas/tests/extension/base/reshaping.py

Line 290 in 9a52a81

def test_unstack(self, data, index, obj):

Casting both expected and result to object causes the raise to be hit. I am guessing we need to do something similar to what is done for JSONArray and overwrite assert_series_equal and assert_frame_equal like is done here

pandas/pandas/tests/extension/json/test_json.py

Line 96 in 9a52a81

def assert_frame_equal(cls, left, right, *args, **kwargs):

?

[znicholls]

Or we just skip that test entirely like

pandas/pandas/tests/extension/json/test_json.py

Line 184 in 9a52a81

@pytest.mark.xfail(reason="dict for NA")

[jbrockmendel]

The problem is that the inputs in array_equivalent are converted using np.asarray. After this operation, they are identified as object type (even though their dtype attribute isn't object)

do the inputs have an __array__ method? id expect that to govern what dtype np.asarray gets you

[jbrockmendel]

arent the is_array checks redundant here?

[znicholls]

Probably, I had kept them in case there was some edge case I hadn't thought of but I can also remove them

[znicholls]

do the inputs have an __array__ method? id expect that to govern what dtype np.asarray gets you

Yep sorry I realised that after the fact (comment here #39790 (comment), fix here ad5f714)

[jbrockmendel]

I added an array method to the mock numpy-like object which seems to have fixed the issue in #39790 (comment). So, the question is, do the changes to testing made here look ok?

so with the __array__ method, are the test changes needed? they're going to hurt performance a small-but-frequent amount

[znicholls]

The test changes are still needed. The reason is that you need something over and above is_array because is_array is equivalent to isinstance(val, np.ndarray). The array like implementations are not numpy arrays so they fail that check (even though they have an __array__ method).

[github-actions]

This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.

[simonjayhawkins]

@znicholls Thanks for the PR. closing as stale.