Skip to content

PERF: quantile now operates per block boosting perf / fix quantile with nan #13122

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

PERF: quantile now operates per block boosting perf / fix quantile with nan #13122

wants to merge 1 commit into from

Conversation

jreback
Copy link
Contributor

@jreback jreback commented May 9, 2016

closes #11623
closes #13098
There is a slight API change in that the returned Series are now named by the quantile.

this PR

In [1]: df = DataFrame(np.random.randn(1000, 3), columns=list('ABC'))

In [2]: %timeit df.quantile([0.1, 0.5], axis=1)
1000 loops, best of 3: 830 µs per loop

0.18.1

In [3]: df.quantile([0.1, 0.5], axis=1)
In [1]: df = DataFrame(np.random.randn(1000, 3), columns=list('ABC'))

In [2]: %timeit df.quantile([0.1, 0.5], axis=1)
1 loop, best of 3: 317 ms per loop

@jreback jreback added Performance Memory or execution speed performance Numeric Operations Arithmetic, Comparison, and Logical operations Clean labels May 9, 2016
@jreback jreback added this to the 0.18.2 milestone May 9, 2016
@jreback
Copy link
Contributor Author

jreback commented May 9, 2016

cc @sinhrks

@jreback jreback force-pushed the quantile branch 4 times, most recently from e0a2d03 to eacc055 Compare May 10, 2016 01:14
@jreback
Copy link
Contributor Author

jreback commented May 10, 2016

@sinhrks this also resolves some thorny issues with .replace and datetime w/tz (now the conversion happens at a much lower level).

@jreback jreback force-pushed the quantile branch 2 times, most recently from 44ffd60 to fb6b853 Compare May 10, 2016 01:58
@jreback jreback mentioned this pull request May 10, 2016
Closed
sinhrks
sinhrks reviewed May 11, 2016
View reviewed changes
pandas/core/internals.py
@@ -1260,32 +1267,117 @@ def equals(self, other):
return False
return array_equivalent(self.values, other.values)

def quantile(self, qs, mgr=None, **kwargs):
def quantile(self, qs, interpolation='linear', axis=0, mgr=None, **kwargs):
Copy link
Member

@sinhrks sinhrks May 11, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kwargs is not used anymore?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

umm no.

@sinhrks
Copy link
Member

sinhrks commented May 11, 2016

Thanks. I've at a loss how to achieve the behavior you did in reduction.

BTW, we don't seem to test .quantile(axis=1) with mixed dtype currently. It's nice to be added.

@jreback jreback mentioned this pull request May 11, 2016
Closed
@jreback jreback force-pushed the quantile branch 2 times, most recently from 12509b0 to c815bdb Compare May 12, 2016 00:21
@jreback
Copy link
Contributor Author

jreback commented May 12, 2016

ok added test for mixed, though not a whole lot you can do with it.

@jreback jreback closed this in 4de83d2 May 12, 2016
@jreback jreback mentioned this pull request May 31, 2016
Closed
@jreback jreback mentioned this pull request Jun 7, 2016
Closed
@jreback jreback mentioned this pull request Jun 17, 2016
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
Clean Numeric Operations Arithmetic, Comparison, and Logical operations Performance Memory or execution speed performance
Projects
None yet
Milestone
0.19.0
Development

Successfully merging this pull request may close these issues.

REGRP: Series.quantile returns NaN Pandas quantile function very slow
2 participants
@jreback @sinhrks