Skip to content

ENH: preserve RangeIndex in factorize #38034

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Nov 26, 2020

Conversation

jbrockmendel
Copy link
Member

  • closes #xxxx
  • tests added / passed
  • passes black pandas
  • passes git diff upstream/master -u -- "*.py" | flake8 --diff
  • whatsnew entry

@jreback jreback added the Index Related to the Index class or subclasses label Nov 26, 2020
@jreback
Copy link
Contributor

jreback commented Nov 26, 2020

minor comment, any perf implications?

@jbrockmendel
Copy link
Member Author

any perf implications?

In [2]: ri = pd.Index(range(10**5))

In [3]: %timeit ri.factorize()
2.28 ms ± 120 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)   # <-- master
33.2 µs ± 199 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)  # <-- PR

In [5]: %timeit mi = pd.MultiIndex.from_arrays([ri, ri, ri])
18.2 ms ± 1.26 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)  # <-- master
308 µs ± 5.21 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)  # <-- PR

In [6]: mi = pd.MultiIndex.from_arrays([ri, ri, ri])

In [7]: %timeit mi.get_loc((10, 10, 10))
13.1 µs ± 65.4 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)  # <-- master
11 µs ± 116 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)  # <-- PR

@jreback jreback added this to the 1.2 milestone Nov 26, 2020
@jreback
Copy link
Contributor

jreback commented Nov 26, 2020

any perf implications?

In [2]: ri = pd.Index(range(10**5))

In [3]: %timeit ri.factorize()
2.28 ms ± 120 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)   # <-- master
33.2 µs ± 199 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)  # <-- PR

In [5]: %timeit mi = pd.MultiIndex.from_arrays([ri, ri, ri])
18.2 ms ± 1.26 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)  # <-- master
308 µs ± 5.21 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)  # <-- PR

In [6]: mi = pd.MultiIndex.from_arrays([ri, ri, ri])

In [7]: %timeit mi.get_loc((10, 10, 10))
13.1 µs ± 65.4 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)  # <-- master
11 µs ± 116 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)  # <-- PR

nice ok, imagine these progate to other things in a + way.

@jreback jreback added the Performance Memory or execution speed performance label Nov 26, 2020
@jbrockmendel
Copy link
Member Author

rebased+green

@jreback jreback merged commit 090d6a1 into pandas-dev:master Nov 26, 2020
@jbrockmendel jbrockmendel deleted the enh-range-factorize branch November 26, 2020 22:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Index Related to the Index class or subclasses Performance Memory or execution speed performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants