__or__ (and others?) should return NotImplemented for custom classes

[adamklein]

Code Sample, a copy-pastable example if possible

In [1]: class Test:
   ...:     def __ror__(self, other):
   ...:         print('here')
   ...:         return self
   ...: 

In [2]: {} | Test()
here
Out[2]: <__main__.Test at 0x7f37a1601ba8>

In [3]: import pandas

In [4]: pandas.DataFrame() | Test()
Out[4]: 
Empty DataFrame
Columns: []
Index: []

Problem description

I would expect DataFrame to return NotImplemented in this case, so we can properly delegate to __ror__.

Expected Output

Out[3]: <__main__.Test at 0x7f37a1601ba8>

Output of pd.show_versions()

In [4]: pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.6.6.final.0
python-bits: 64
OS: Linux
OS-release: 4.14.67-ts1
machine: x86_64
processor:
byteorder: little
LC_ALL: None
LANG: en_US.utf8
LOCALE: en_US.UTF-8

pandas: 0.24.2
pytest: 4.4.1
pip: 19.0.3
setuptools: 40.8.0
Cython: 0.29.6
numpy: 1.15.4
scipy: 1.2.1
pyarrow: 0.11.1
xarray: None
IPython: 7.4.0
sphinx: 2.0.1
patsy: 0.5.1
dateutil: 2.8.0
pytz: 2018.9
blosc: None
bottleneck: None
tables: 3.5.1
numexpr: 2.6.9
feather: None
matplotlib: 3.0.3
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: 1.1.5
lxml.etree: None
bs4: 4.7.1
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: 0.2.0
fastparquet: None
pandas_gbq: None
pandas_datareader: None
gcsfs: None

[gfyoung]

cc @jreback

[TomAugspurger]

I didn't know this, but apparently DataFrame.__or__ will fill fill missing values with other, for non series / frame.

In [6]: pd.DataFrame({"A": [np.nan]}) | Test()
here
Out[6]:
      A
0  True

This is a bit strange, but is it worth deprecating?

[WillAyd]

I think this is happening here:

pandas/pandas/core/ops.py

Line 2167 in ee6b131

self = self.fillna(fill_value)

There's a comment elsewhere in the module about how the bool_method objects are not tested, so I think this is a pretty deep bug.

Not sure what the best solution is yet but providing as FYI in case it helps others to diagnose

[jbrockmendel]

This can be done by implementing __pandas_priority__ on your custom class. Closed by #48347.