Skip to content

PERF: perf improvments in indexing with object dtypes (GH5968) #5973

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jan 16, 2014

Conversation

jreback
Copy link
Contributor

@jreback jreback commented Jan 16, 2014

closes #5968

-------------------------------------------------------------------------------
Test name                                    | head[ms] | base[ms] |  ratio   |
-------------------------------------------------------------------------------
frame_iloc_big                               |   0.2836 |   2.8454 |   0.0997 |

@ghost
Copy link

ghost commented Jan 16, 2014

That's a real boon for someone out there.

@ghost
Copy link

ghost commented Jan 16, 2014

Hey now, it might be christmas again. frame_reindex is everywhere.

frame_dtypes                                 |   0.3270 |  66.4640 |   0.0049 |
frame_repr_wide                              |  22.0180 | 693.6400 |   0.0317 |
frame_iloc_big                               |   0.3091 |   3.5690 |   0.0866 |
frame_reindex_axis1                          | 327.7626 | 488.3307 |   0.6712 |
frame_reindex_both_axes                      |  47.7920 |  65.5576 |   0.7290 |
frame_reindex_upcast                         |  12.5876 |  16.9796 |   0.7413 |
packers_write_pickle                         |   1.0316 |   1.3046 |   0.7908 |
frame_reindex_both_axes_ix                   |  50.0793 |  60.2030 |   0.8318 |

jreback added a commit that referenced this pull request Jan 16, 2014
PERF: perf improvments in indexing with object dtypes (GH5968)
@jreback jreback merged commit 52a139e into pandas-dev:master Jan 16, 2014
@jtratner
Copy link
Contributor

Nice work! What did you use to profile this and figure out where the issue was occurring?

@jreback
Copy link
Contributor Author

jreback commented Jan 17, 2014

oh started with @y-p putting up the issue about repr of a large frame being slow

first prob was the dtypes taking a non constant time ; 2ns was iloc

I stepped thru it to figure out

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

df.dtypes.values is not O(1) and repr(df) is therefore slow for large frames
2 participants