Skip to content

Rewrite pandas.util.testing.assert_almost_equal for performance #4398

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wesm opened this issue Jul 29, 2013 · 3 comments · Fixed by #5219
Closed

Rewrite pandas.util.testing.assert_almost_equal for performance #4398

wesm opened this issue Jul 29, 2013 · 3 comments · Fixed by #5219
Labels
Performance Memory or execution speed performance Testing pandas testing functions or related to the test suite
Milestone
0.13

Comments

@wesm
Copy link
Member

wesm commented Jul 29, 2013

I don't know if anyone's profiled the test suite, but we'd get major performance boosts if we wrote a Cython version of assert_almost_equal. Is there another open issue for this already?
@cpcloud
Copy link
Member

cpcloud commented Jul 29, 2013

@wesm i think you have one somewhere

@cpcloud
Copy link
Member

cpcloud commented Jul 29, 2013

i can't find it though maybe it was something else

@danbirken danbirken mentioned this issue Oct 15, 2013
Merged
danbirken added a commit to danbirken/pandas that referenced this issue Oct 21, 2013
@danbirken
TST/PERF: Re-write assert_almost_equal() in cython pandas-dev#4398
3b50b52 
Add a testing.pyx cython file, and port assert_almost_equal() from
python to cython.

On my machine this brings a modest gain to the suite of "not slow" tests
(160s -> 140s), but on assert_almost_equal() heavy tests, like
test_expressions.py, it shows a large improvement (14s -> 4s).
danbirken added a commit to danbirken/pandas that referenced this issue Oct 21, 2013
@danbirken
TST: Fix edge cases in assert_almost_equal() + tests pandas-dev#4398
81ef690 
Many of the edge cases were related to ordering of the items, but in
some cases there were also issues with type checking.  This fixes both
of those issues and massively expands the testing for this function.
@danbirken
Copy link
Contributor

FYI: #3150 is the duplicate of this one that was theorized to exist in the initial message

@jreback jreback mentioned this issue Oct 21, 2013
Closed
jreback added a commit that referenced this issue Oct 21, 2013
@jreback
Merge pull request #5219 from danbirken/c_assert_almost_equal_simple
1023b79 
TST/PERF: Re-write assert_almost_equal() in cython #4398
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Performance Memory or execution speed performance Testing pandas testing functions or related to the test suite
Projects
None yet
Milestone
0.13
Development

Successfully merging a pull request may close this issue.

TST/PERF: Re-write assert_almost_equal() in cython #4398 danbirken/pandas
3 participants
@danbirken @wesm @cpcloud