Skip to content

Performance regressions since 0.8.0 #2336

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wesm opened this issue Nov 23, 2012 · 1 comment
Closed

Performance regressions since 0.8.0 #2336

wesm opened this issue Nov 23, 2012 · 1 comment
Labels
Performance Memory or execution speed performance
Milestone
0.10

Comments

@wesm
Copy link
Member

wesm commented Nov 23, 2012

Should investigate some of these ones at the bottom. The pivot benchmark is related to refactoring I made in the unstack code, and is not so simple to fix:


Results:
                                           t_head t_baseline  ratio
name                                                               
indexing_panel_subset                      3.2588  1997.1172 0.0016
groupby_frame_median                      10.2211   181.5090 0.0563
datetimeindex_add_offset                   0.3107     2.2951 0.1354
read_csv_thou_vb                          41.6154   205.6429 0.2024
frame_constructor_ndarray                  0.0409     0.0888 0.4604
read_csv_vb                               23.5146    50.1700 0.4687
read_csv_standard                         14.4659    29.6830 0.4873
reindex_multiindex                         1.4252     2.7399 0.5202
groupby_frame_singlekey_integer            3.8736     6.6471 0.5828
groupby_first                              7.2411    12.2543 0.5909
groupby_last                               7.6421    12.7714 0.5984
panel_from_dict_same_index                30.3418    49.3051 0.6154
panel_from_dict_equiv_indexes             30.5921    49.1175 0.6228
groupby_simple_compress_timing            53.7012    85.8531 0.6255
reindex_frame_level_reindex                1.8341     2.7030 0.6786
append_frame_single_homogenous             0.3622     0.4860 0.7453
timeseries_add_irregular                  22.9419    30.2414 0.7586
write_csv_standard                       359.9060   458.5490 0.7849
stat_ops_level_series_sum                  3.7688     4.6419 0.8119
groupby_series_simple_cython               6.5816     7.9662 0.8262
dataframe_reindex_columns                  0.3087     0.3718 0.8303
groupby_multi_size                        39.3789    45.9909 0.8562
frame_get_numeric_data                     0.0842     0.0981 0.8589
reshape_stack_simple                       2.5648     2.9615 0.8660
frame_reindex_axis1                        3.3879     3.8442 0.8813
timeseries_asof_nan                        8.5555     9.6283 0.8886
timeseries_asof                            9.2042    10.3367 0.8904
stat_ops_level_frame_sum                   4.6151     5.0289 0.9177
frame_reindex_both_axes_ix                 0.7154     0.7768 0.9209
indexing_dataframe_boolean_rows            0.2306     0.2500 0.9225
stats_rank_average_int                    24.2966    26.2871 0.9243
frame_to_csv                             474.3190   511.4419 0.9274
sort_level_zero                            6.2975     6.7836 0.9283
reindex_frame_level_align                  1.8621     1.9765 0.9421
dataframe_reindex_daterange                0.3935     0.4141 0.9503
frame_ctor_nested_dict_int64             135.1471   142.0300 0.9515
join_dataframe_index_single_key_bigger     8.2120     8.6290 0.9517
timeseries_large_lookup_value              0.0248     0.0260 0.9526
join_dataframe_index_single_key_small      7.7202     8.0596 0.9579
indexing_dataframe_boolean_rows_object     0.4611     0.4768 0.9670
timeseries_sort_index                     21.2128    21.9074 0.9683
frame_boolean_row_select                   0.2998     0.3096 0.9683
read_table_multiple_date                2367.1391  2435.5052 0.9719
read_table_multiple_date_baseline       1086.0040  1116.9188 0.9723
series_align_left_monotonic               17.5190    17.9150 0.9779
groupby_multi_series_op                   19.7853    20.2238 0.9783
series_align_int64_index                  37.7741    38.5680 0.9794
panel_from_dict_two_different_indexes     99.2441   101.2650 0.9800
frame_fillna_many_columns_pad             16.8927    17.2210 0.9809
reindex_daterange_backfill                 0.1794     0.1813 0.9894
groupby_multi_cython                      22.6835    22.9242 0.9895
reindex_daterange_pad                      0.1835     0.1853 0.9901
reindex_fillna_pad                         0.1220     0.1228 0.9940
frame_fancy_lookup                         2.1757     2.1861 0.9953
index_int64_intersection                  38.0053    38.1207 0.9970
index_int64_union                         88.7940    89.0169 0.9975
groupby_multi_python                      67.7776    67.8983 0.9982
frame_ctor_nested_dict                    96.4491    96.5813 0.9986
stat_ops_series_std                        0.2840     0.2827 1.0044
series_value_counts_int64                  2.6949     2.6773 1.0066
timeseries_timestamp_downsample_mean       5.9724     5.9310 1.0070
sort_level_one                             6.2288     6.1651 1.0103
groupby_apply_dict_return                 40.3767    39.8300 1.0137
stats_rank2d_axis0_average                26.6424    26.1975 1.0170
concat_series_axis1                       73.4706    72.2370 1.0171
groupby_frame_cython_many_columns          5.5423     5.4274 1.0212
stats_rank_average                        35.6590    34.8816 1.0223
join_dataframe_index_multi                28.8276    28.1839 1.0228
reindex_fillna_backfill                    0.1190     0.1160 1.0259
frame_fillna_inplace                      19.4959    18.9305 1.0299
timeseries_asof_single                     0.0586     0.0567 1.0333
stats_rank2d_axis1_average                18.5875    17.8737 1.0399
join_dataframe_index_single_key_bigger    21.5043    20.6628 1.0407
groupby_pivot_table                       24.8929    23.6659 1.0518
timeseries_slice_minutely                  0.0583     0.0553 1.0549
timeseries_timestamp_tzinfo_cons           0.0177     0.0168 1.0590
groupby_multi_different_functions         19.3980    18.1736 1.0674
read_csv_comment_vb                      106.1771    99.0222 1.0723
match_strings                              0.4124     0.3835 1.0753
series_constructor_ndarray                 0.0110     0.0103 1.0773
groupby_indices                           11.7223    10.7914 1.0863
timeseries_1min_5min_ohlc                  0.6142     0.5541 1.1086
append_frame_single_mixed                  1.7186     1.5501 1.1087
reshape_unstack_simple                     3.3168     2.9830 1.1119
frame_ctor_list_of_dict                  109.4160    98.0361 1.1161
groupby_multi_different_numpy_functions   20.6145    18.4240 1.1189
timeseries_1min_5min_mean                  0.6310     0.5584 1.1300
frame_reindex_axis0                        3.0153     2.5576 1.1790
frame_reindex_both_axes                    0.7285     0.6088 1.1967
series_ctor_from_dict                      3.7974     3.1420 1.2086
stat_ops_level_frame_sum_multiple         12.9655    10.3905 1.2478
sparse_series_to_frame                   208.4851   146.2591 1.4255
stat_ops_level_series_sum_multiple        12.6471     8.4403 1.4984
panel_from_dict_all_different_indexes    181.9479   104.8920 1.7346
reshape_pivot_time_series                318.2459   148.5360 2.1426
frame_fancy_lookup_all                   125.4880    36.3095 3.4561

Columns: test_name | head_time [ms] | baseline_time [ms] | ratio

- a Ratio of 1.30 means HEAD is 30% slower then the Baseline.

Head [5af2f95] : ENH: accelerate label compression. accept >= 10 micros in test_perf
Baseline [a901af2] : RLS: version 0.8.0
wesm added a commit that referenced this issue Nov 27, 2012
@wesm
PERF: performance regression in unioning multiple DatetimeIndex objec…
a87194b 
…ts. #2336
wesm added a commit that referenced this issue Nov 27, 2012
@wesm
ENH: be more careful about performance in list conversion when passed…
7f32698 
… to _ensure_index. per #2336
wesm added a commit that referenced this issue Nov 28, 2012
@wesm
PERF: fix iteritems performance regression in unique case. #2336
a25ffc3
@wesm
Copy link
Member Author

wesm commented Nov 28, 2012

I've fixed the regressions to my satisfaction

@wesm wesm closed this as completed Nov 28, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Performance Memory or execution speed performance
Projects
None yet
Milestone
0.10
Development

No branches or pull requests
1 participant
@wesm