Deprecation/FutureWarnings raised by == on NumPy arrays in NumPy version1.9

[unutbu]

This continues a discussion started at the end of GH 8512

Summary: NumPy is changing the behavior of ==. It no longer checks for is relationships between elements. Therefore,

In older versions of NumPy:

array([nan], dtype=object) == array([nan], dtype=object)
array([ True], dtype=bool)

In future NumPy, I think this will happen:

array([nan], dtype=object) == array([nan], dtype=object)
array([ False], dtype=bool)

In current NumPy version 1.9, a DeprecationWarning or FutureWarning is issued.

One place where this issue affects Pandas is

nosetests -v pandas/tests/test_series.py:TestSeries.test_replace

The problem originates in internals.py/_possibly_compare which is called like this:

_possibly_compare(np.array([1, np.nan], dtype='O'), 'foo', operater.eq)
_possibly_compare(np.array(['2000-01-02T19:00'], dtype='datetime64[ns]'), 1, operater.eq)

Both of these raise DeprecationWarnings.

a is always an array, b is a scalar. AFAICT, the intent of
_possibly_compare is to return a boolean mask where elements in a equal b.
Since we want a boolean mask, array_equivalent is not a help here since that
just returns a single bool.

[unutbu]

Above, a and b refer to _possibly_compare(a, b, operator.eq)

[unutbu]

@jreback: I think this problem would be a lot easier to fix if in future versions of NumPy, ndarray.__eq__ would simply return False instead of TypeError for things like

np.array(['2000-01-02T19:00'], dtype='<M8[ns]') == 1

False is what it returns now, so I think the majority of those DeprecationWarnings would be moot.

Meanwhile, the FutureWarnings (I think -- I should check this some more --) only occur when comparing NaNs in object arrays. We (hopefully) already address this issue with things like isnull and array_equivalent, so again the FutureWarnings should be moot.

What do you think? Do you think we should ask the NumPy developers to change the TypeErrors to False?

[jreback]

why don't u post a numpy issue and see why they are doing it this way

I agree it would be more consistent to simply return FAlse

[unutbu]

@jreback: NumPy issue: numpy/numpy#5188 (comment)