API: .equals for Extension Arrays

[jreback]

We don't have a good method of testing for equality between EA's

In [19]: from pandas.tests.extension.decimal.array import DecimalArray                                                                                                                                                                                           

In [20]: from decimal import Decimal                                                                                                                                                                                                                             

In [21]: x = DecimalArray([Decimal('1'),Decimal('Nan')])                                                                                                                                                                                                         

In [22]: x == x                                                                                                                                                                                                                                                  
Out[22]: array([ True, False])

In [23]: x = pd.Series([1,np.nan], dtype='Int64').array                                                                                                                                                                                                          

In [24]: x == x                                                                                                                                                                                                                                                  
Out[24]: array([ True, False])

These happen to work with Series now because the null checks are handled at a higher level.

In [26]: x.equals(x)                                                                                                                                                                                                                                             
Out[26]: True

In [27]: x = pd.Series(DecimalArray([Decimal('1'),Decimal('Nan')]))                                                                                                                                                                                              

In [28]: x.equals(x)                                                                                                                                                                                                                                             
Out[28]: True

we could provide a default implementation that should just work in the presence of NA.

def equals(self, other):
    return ((self == other) | (self.isna() == other.isna())).all()

actually we should also implement a default __eq__

[jreback]

cc @jorisvandenbossche @TomAugspurger

[jreback]

actually we define __eq__ if you subclass from ExtensionArrayScalarMixin, but we need to change to use the above (and __ne__).

[jorisvandenbossche]

+1 on adding equals and such a default implementation.

We might only need to add __eq__ as a required abstract method, to ensure people implement it (otherwise the default python one will always return True or False based on identity, and thus give wrong results for such an equals method)

[TomAugspurger]

+1 for `.equals`, and making `__eq__` abstract so that the expected array-like `__eq__` must be implemented.

…

On Thu, Jun 27, 2019 at 2:13 PM Joris Van den Bossche < ***@***.***> wrote: +1 on adding equals and such a default implementation. We might only need to add __eq__ as a required abstract method, to ensure people implement it (otherwise the default python one will always return True or False based on identity, and thus give wrong results for such an equals method) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#27081?email_source=notifications&email_token=AAKAOIVGP7LWBI23LK4Q53LP4UGMFA5CNFSM4H36BQPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYYDB7Y#issuecomment-506474751>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAKAOIRFMUGIIV7TO5FSW4DP4UGMFANCNFSM4H36BQPA> .

[jorisvandenbossche]

Removing the milestone as I think this is not 0.25.0 critical?

[jreback]

no this is slightly non trivial

[zys5945]

Should the __eq__ on ExtensionScalarOpsMixin be changed to take care nan, or should ExtensionArray expose a .equals() that does that?

I believe DecimalArray is the only class that inherits ExtensionScalarOpsMixin so to cover all EA's I think a .equals() on EA is more appropriate?

@jreback

[VijayantSoni]

Hi, As this is labeled good first issue, I would like to take this on. Is this still needed ?