PERF: enhance MultiIndex.remove_unused_levels when no levels are unused

[toobaz]

• closes Unstack performance regression  #19289
• tests passed
• passes git diff upstream/master -u -- "*.py" | flake8 --diff
• whatsnew entry

This uses np.bincount rather than algos.unique to check if there are unused levels: since levels indexes are adjacent integers, the former is faster (~50% in my tests). It then fallbacks to algos.unique if there are unused levels.

The result is that asv tests on --bench reshape give

       before           after         ratio
     [d5d5a718]       [499ae6bf]
-     1.43±0.01ms      1.19±0.01ms     0.83  reshape.SparseIndex.time_unstack

There is a penalty (of the same order of magnitude) for cases in which there are unused levels, but I think this is worth because:

• there is room for cythonizing and bringing this penalty close to zero in the average case (see comment in code)
• I think the case of no unused level is much more frequent
• if people want to avoid this penalty they can just clean their index from unused levels once for all
• in any case, asv tests on --bench multiindex_object (where the index in Duplicates always has unused levels) do not find any change in performance

[codecov]

Codecov Report

Merging #20703 into master will decrease coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master   #20703      +/-   ##
==========================================
- Coverage   91.84%   91.83%   -0.01%     
==========================================
  Files         153      153              
  Lines       49277    49279       +2     
==========================================
  Hits        45257    45257              
- Misses       4020     4022       +2

Flag	Coverage Δ
#multiple	90.22% <100%> (-0.01%)	⬇️
#single	41.9% <0%> (-0.01%)	⬇️

Impacted Files	Coverage Δ
pandas/core/indexes/multi.py	95.06% <100%> (ø)	⬆️
pandas/util/testing.py	84.38% <0%> (-0.21%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d5d5a71...c61dd9a. Read the comment docs.

[jreback]

thanks @toobaz I closed the issue, but I suppose can revisit a re-factorting of unstack and/or cythonize removing unused levels at some point.