BUG: incorrect type when indexing sparse dataframe with iterable

[suvayu]

closes #34526
closes #34540

• 2 tests added / passed
• passes black pandas
• passes git diff upstream/master -u -- "*.py" | flake8 --diff
• whatsnew entry

The problem arose when indexing with an iterable. If the result consisted of columns which originally only had sparse values, the dtype was solely inferred from the fill_value, which defaults to 0. Hence, the resulting columns have the dtype Sparse[int64, 0]. This commit changes the type inference logic to use the numpy type promotion rules between the underlying subtype of the SparseArray.dtype and the type of the fill value.

[jreback]

looks fine, can you create a test_sparse.py (in this directory) and move other sparse indexing tests as well (e.g. the one right above, maybe more)

[suvayu]

@jreback Thanks for the review! I'll try fix the bug in the next few days.

[suvayu]

For the type inference logic, I wasn't sure which to use, SparseArray.dtype.subtype or the sparse values themselves (SparseArray.sp_values). I went with the former because it seemed conceptually cleaner.

[suvayu]

Hi @jreback, just bringing this to your attention. I wasn't sure if you noticed it's ready for review (I used the "request review" option in the side panel). Apologies if this is already on your list.

[TomAugspurger]

@suvayu can you add a whatsnew to 1.1.0.rst in the bug fixes section?

[TomAugspurger]

The changes look good, just need a release note. Make sure to include references to both issues fixed.

[suvayu]

Hi @TomAugspurger, I added the the whatsnew entry. There was a merge conflict with master (another entry in the same section), so I had to force push after resolving it.

[TomAugspurger]

Thanks @suvayu!