Mismatch in Series/DataFrame boolean ops behavior

[jbrockmendel]

Based on test_bool_ops_df_compat from tests.series.test_operators:

s1 = pd.Series([True, False, np.nan, True], dtype=object)
s2 = pd.Series([True, True, False, np.nan], dtype=object)

>>> s1 & s2
0     True
1    False
2    False
3    False
dtype: bool

>>> s1.to_frame() & s2.to_frame()
       0
0   True
1  False
2    NaN
3    NaN

The DataFrame behavior strikes me as More Correct. Is the mismatch intentional?

[WillAyd]

I wouldn't think so and agree with your evaluation

[TomAugspurger]

Agreed.

(totally different code-path, but the index version seems buggy as well)

In [12]: pd.Index(s1) & pd.Index(s2)
Out[12]: Index([True, True, False, nan], dtype='object')

I believe we already have an issue for multiset ops being incorrect.

[h-vetinari]

I believe we already have an issue for multiset ops being incorrect.

Not sure I understand the multiset aspect, but the code above with the non-commutativity of boolean operations is the subject of #6528 and #13806.

Self-quoting from the former about the Series case:

The main issue seems that in pandas.core.ops.wrapper (https://github.com/pandas-dev/pandas/blob/master/pandas/core/ops.py#L1572), after aligning the Series, only other is filled (transforming NaN to False), but self isn't, which introduces this symmetry break.

[jbrockmendel]

Closed by #29531 (that adds the test, not sure when the actual fix was)