Series/DataFrame.isna/notna with sparse data densifies

[jorisvandenbossche]

In [1]: df = pd.SparseDataFrame({'A': [np.nan, np.nan, 1, 2, np.nan], 
   ...:                          'B': [0, np.nan, np.nan, 2, np.nan]})                                                                                                          

In [2]: res = df.notna()                                                                                                                                                        

In [3]: res                                                                                                                                                                     
Out[3]: 
       A      B
0  False   True
1  False  False
2   True  False
3   True   True
4  False  False

In [4]: res['A']                                                                                                                                                                
Out[4]: 
0    False
1    False
2     True
3     True
4    False
Name: A, dtype: Sparse[bool, False]
BlockIndex
Block locations: array([2], dtype=int32)
Block lengths: array([2], dtype=int32)

In [5]: df2 = pd.DataFrame(df)                                                                                                                                                  

In [6]: res2 = df2.notna()                                                                                                                                                      

In [7]: res2                                                                                                                                                                    
Out[7]: 
       A      B
0  False   True
1  False  False
2   True  False
3   True   True
4  False  False

In [8]: res2['A']                                                                                                                                                               
Out[8]: 
0    False
1    False
2     True
3     True
4    False
Name: A, dtype: bool

The result of isna / notna can be of type Sparse[bool, False] here as well.

[JustinZhengBC]

Similar to #23744, I think the issue is the constructor as pd.DataFrame can only return that class so it cannot self-cast to the SparseDataFrame class. Using df2 = pd.SparseDataFrame(df) achieves expected results

[jorisvandenbossche]

See #23744 (comment) for an answer.

[mzeitlin11]

This looks ok on master:

In [3]: df = pd.DataFrame({"a": [np.nan, 1.0], "b": [np.nan, 1.0]}, dtype=pd.Spa
   ...: rseDtype("float"))

In [4]: df.isna().dtypes
Out[4]:
a    Sparse[bool, True]
b    Sparse[bool, True]
dtype: object

In [5]: df["a"].isna().dtype
Out[5]: Sparse[bool, True]

And is hit by the TestMissing class of extension tests, so going to close