Skip to content

PERF/REGR: symmetric_difference revert most of #41833 #42318

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

jbrockmendel
Copy link
Member

@jbrockmendel jbrockmendel commented Jun 30, 2021

aimed at #42239

@simonjayhawkins simonjayhawkins added Performance Memory or execution speed performance setops union, intersection, difference, symmetric_difference labels Jun 30, 2021
@simonjayhawkins simonjayhawkins added this to the 1.3 milestone Jun 30, 2021
@simonjayhawkins
Copy link
Member

(pandas-dev) simon@T3630:~/pandas/asv_bench (perf-symmetric_difference)$ asv compare v1.3.0.dev0 4c878d244a369cc7689f6c01c1a8c4d77b405cc5 |grep SetOperations.time_operation.*symmetric_difference
      15.0±0.09ms         16.1±2ms     1.08  index_object.SetOperations.time_operation('date_string', 'symmetric_difference')
+     3.75±0.03ms      4.24±0.07ms     1.13  index_object.SetOperations.time_operation('datetime', 'symmetric_difference')
      2.84±0.04ms       2.91±0.3ms     1.03  index_object.SetOperations.time_operation('int', 'symmetric_difference')
       13.8±0.2ms       14.9±0.2ms     1.07  index_object.SetOperations.time_operation('strings', 'symmetric_difference')
+         422±3ms          670±6ms     1.59  multiindex_object.SetOperations.time_operation('monotonic', 'datetime', 'symmetric_difference')
+      42.2±0.2ms       60.1±0.5ms     1.42  multiindex_object.SetOperations.time_operation('monotonic', 'int', 'symmetric_difference')
+        42.2±2ms       48.6±0.6ms     1.15  multiindex_object.SetOperations.time_operation('monotonic', 'string', 'symmetric_difference')
+         423±3ms          652±7ms     1.54  multiindex_object.SetOperations.time_operation('non_monotonic', 'datetime', 'symmetric_difference')
+      42.6±0.5ms       58.9±0.4ms     1.38  multiindex_object.SetOperations.time_operation('non_monotonic', 'int', 'symmetric_difference')
+        42.4±1ms       48.3±0.5ms     1.14  multiindex_object.SetOperations.time_operation('non_monotonic', 'string', 'symmetric_difference')

@simonjayhawkins
Copy link
Member

comparing this PR to master

(pandas-dev) simon@T3630:~/pandas/asv_bench (perf-symmetric_difference)$ asv compare master 4c878d244a369cc7689f6c01c1a8c4d77b405cc5 |grep SetOperations.time_operation.*symmetric_difference
-      31.1±0.7ms         16.1±2ms     0.52  index_object.SetOperations.time_operation('date_string', 'symmetric_difference')
-      7.25±0.1ms      4.24±0.07ms     0.58  index_object.SetOperations.time_operation('datetime', 'symmetric_difference')
-      5.16±0.3ms       2.91±0.3ms     0.57  index_object.SetOperations.time_operation('int', 'symmetric_difference')
-      28.7±0.7ms       14.9±0.2ms     0.52  index_object.SetOperations.time_operation('strings', 'symmetric_difference')
-      1.13±0.01s          670±6ms     0.59  multiindex_object.SetOperations.time_operation('monotonic', 'datetime', 'symmetric_difference')
-         105±3ms       60.1±0.5ms     0.57  multiindex_object.SetOperations.time_operation('monotonic', 'int', 'symmetric_difference')
-      81.5±0.5ms       48.6±0.6ms     0.60  multiindex_object.SetOperations.time_operation('monotonic', 'string', 'symmetric_difference')
-      1.14±0.03s          652±7ms     0.57  multiindex_object.SetOperations.time_operation('non_monotonic', 'datetime', 'symmetric_difference')
-         103±1ms       58.9±0.4ms     0.57  multiindex_object.SetOperations.time_operation('non_monotonic', 'int', 'symmetric_difference')
-        81.9±1ms       48.3±0.5ms     0.59  multiindex_object.SetOperations.time_operation('non_monotonic', 'string', 'symmetric_difference')

updating to close issue

@simonjayhawkins
Copy link
Member

odd. https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=62375&view=results

will merge as it appears tests were successful

@simonjayhawkins simonjayhawkins merged commit be2efa9 into pandas-dev:master Jun 30, 2021
@lumberbot-app

This comment has been minimized.

@simonjayhawkins
Copy link
Member

simonjayhawkins commented Jun 30, 2021

updating to close issue

on 1.3.x Index.symmetric_difference is overridden for MultiIndex so we will need to recheck timings

simonjayhawkins pushed a commit to simonjayhawkins/pandas that referenced this pull request Jun 30, 2021
@jbrockmendel jbrockmendel deleted the perf-symmetric_difference branch June 30, 2021 14:25
simonjayhawkins added a commit that referenced this pull request Jun 30, 2021
…1833 (#42324)

* Backport PR #42318: PERF/REGR: symmetric_difference revert most of #41833

* remove if self._is_multi block

* remove MultiIndex symmetric_difference

Co-authored-by: jbrockmendel <[email protected]>
JulianWgs pushed a commit to JulianWgs/pandas that referenced this pull request Jul 3, 2021
…as-dev#42318)

* PERF/REGR: symmetric_difference revert most of 41833

* mypy fixup

Co-authored-by: Simon Hawkins <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Performance Memory or execution speed performance setops union, intersection, difference, symmetric_difference
Projects
None yet
Development

Successfully merging this pull request may close these issues.

PERF: regression in datetimelike symmetric_difference set op
2 participants