Skip to content

PERF: perf issue with dropna on frame #5815

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 1, 2014

Conversation

jreback
Copy link
Contributor

@jreback jreback commented Jan 1, 2014

took out the apply on count and just compute directly

-------------------------------------------------------------------------------
Test name                                    | head[ms] | base[ms] |  ratio   |
-------------------------------------------------------------------------------
frame_dropna_axis1_any                       | 147.5154 | 334.2137 |   0.4414 |
frame_dropna_axis1_all                       | 251.1443 | 437.9021 |   0.5735 |
frame_dropna_axis0_all                       |  80.6900 |  80.8613 |   0.9979 |
frame_dropna_axis0_any                       |  58.6040 |  54.6887 |   1.0716 |
-------------------------------------------------------------------------------
Test name                                    | head[ms] | base[ms] |  ratio   |
-------------------------------------------------------------------------------

Ratio < 1.0 means the target commit is faster then the baseline.
Seed used: 1234

Target [c6e300d] : PERF: perf issue with dropna on frame
Base   [5e176a9] : Merge pull request #5738 from y-p/PR_json_pr_ver

BLD: ci/print_versions.py learned to output json

jreback added a commit that referenced this pull request Jan 1, 2014
PERF: perf issue with dropna on frame
@jreback jreback merged commit f7aeaeb into pandas-dev:master Jan 1, 2014
@ghost
Copy link

ghost commented Jan 2, 2014

Broke builds on SPARC.

http://nipy.bic.berkeley.edu/tgrid?category=pandas

@jtratner
Copy link
Contributor

jtratner commented Jan 2, 2014

just 32bit vs. 64bit error. This is why it will be very nice when we can get an 32 bit build going on Travis/somewhere else.

@jtratner
Copy link
Contributor

jtratner commented Jan 2, 2014

that said, I don't have time to fix this tonight, but I'm assuming @jreback can fix quickly.

Btw, @y-p another thing about versioning, all of the builds show 0.13_... in their filename, which I just find confusing (and makes it seem like bugs are with 0.13 rather than 0.14).

@ghost
Copy link

ghost commented Jan 2, 2014

What does "just an error" mean?

It's possible to setup a 32bit environment on travis, it's just annoying to develop the scripts
because the turnaround to testing them is ~10 minutes. I've struggled with travis enough, you're
welcome to it. It'll have to be at the expense of one of the other builds or the test time will spike again.

re versioning, yeah. go ahead and open a PR.

@ghost
Copy link

ghost commented Jan 2, 2014

...rather then 0.14 or 0.13.1, we can't know.

@ghost
Copy link

ghost commented Jan 2, 2014

It would be bad if we had 50 versions prefixed "0.14.0dev-" after which we release a 0.13.1
that's a later commit. I think that's worse then getting used to 0.13-post notation, but you
and @jreback care deeply about this, so It's best if you just made it work they way you like.
I'll live.

@jtratner
Copy link
Contributor

jtratner commented Jan 2, 2014

@y-p there's no 32 bit build allowed on Travis. I see your point on 0.14 vs. 0.13.1 - probably just needs to say "dev" in the name instead.

@jtratner
Copy link
Contributor

jtratner commented Jan 2, 2014

and "just an error", I mean that issues with which int dtype should come up in 32 bit crop up a lot because we aren't regularly testing on 32 bit machines.

I'll put up a PR about versioning tomorrow once I figure out how directory names/version names are set...

@ghost
Copy link

ghost commented Jan 2, 2014

@jtratner, who can stop you from installing a 32bit python every run? it may even be relatively painless
with miniconda, not sure.

Go for it, I'm now convinced this versioning stuff is important enough.

@jtratner
Copy link
Contributor

jtratner commented Jan 2, 2014

Ooh! Interesting idea for sure...

@jreback
Copy link
Contributor Author

jreback commented Jan 2, 2014

I think lets rename to 0.13.1 then no confusion if we later change it to 0.14
we have a free option (but the other way, what we r doing now IS confusing)

also makes the git version look fine for now (if it has dev in it)

@jreback
Copy link
Contributor Author

jreback commented Jan 2, 2014

re running 32-bit - looks possible but tricky

http://askubuntu.com/questions/60094/compile-32-bit-on-64-bitsystem

ironically if sparc builds are automatic at least easy to catch

almost always these are just test dtype issue - easy to fix

@jreback
Copy link
Contributor Author

jreback commented Jan 2, 2014

fixed by #5820

@ghost
Copy link

ghost commented Jan 3, 2014

I think a 64bit gcc can emit 32 bit code without too much trouble, would need to install 32bit libc
and so on which is a little broken on debian if memory serves.

Don't put effort into this for now. I've been slowly working on something to improve our CI situation.
I just suck at CSS, it'll get there.

@jreback, i'm sold, fix the versions. I won't say a word, honest.

@jreback
Copy link
Contributor Author

jreback commented Jan 3, 2014

@jtratner change to 0.13.1 for now ok?

can always change later back to 0.14

it's hard to say until we really release into the wild if we r going to need 0.13.1
or since sparc/windows now building automatically

should be somewhat straightforward to actually release

@jtratner
Copy link
Contributor

jtratner commented Jan 3, 2014

Fine with me

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants