You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Converting a sparse Series to a scipy.sparse.coo_matrix could be much faster. I think the get_indexer function defined in _to_ijv adds unnecessary complexity.
Describe the solution you'd like
It can be much faster by accessing the codes attribute of the multiindex, as follows:
for a two-level multiindex. It should be straightforward to extend to more levels I think.
API breaking implications
None
Describe alternatives you've considered
None
Additional context
To give an example, I started digging into this problem because I had a 2-level-MultiIndexed Series with 61M rows, that is to be converted to a 1M x 1500 sparse matrix. Making the conversion using to_coo() took 10min, making it as described above took half a second.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem?
Converting a sparse Series to a
scipy.sparse.coo_matrix
could be much faster. I think theget_indexer
function defined in_to_ijv
adds unnecessary complexity.Describe the solution you'd like
It can be much faster by accessing the
codes
attribute of the multiindex, as follows:for a two-level multiindex. It should be straightforward to extend to more levels I think.
API breaking implications
None
Describe alternatives you've considered
None
Additional context
To give an example, I started digging into this problem because I had a 2-level-MultiIndexed Series with 61M rows, that is to be converted to a 1M x 1500 sparse matrix. Making the conversion using
to_coo()
took 10min, making it as described above took half a second.The text was updated successfully, but these errors were encountered: