Skip to content

CLN: remove need for *VectorData c-structures in hashtable.pyx #14879

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jreback opened this issue Dec 14, 2016 · 1 comment
Closed

CLN: remove need for *VectorData c-structures in hashtable.pyx #14879

jreback opened this issue Dec 14, 2016 · 1 comment
Labels
Clean Performance Memory or execution speed performance

Comments

@jreback
Copy link
Contributor

jreback commented Dec 14, 2016

post #14859

See what I did for StringHashTable here

I think we should just remove entirely the *VectorData structures and simply create them in a single pass at the end.

I don't think this will actually have a perf penalty, but should see if anything changes.

This will make the code easier to understand and still provide all of the GIL release benefits (because we actually need to hold the GIL to resize), if we use the method above, then can simply allocate the unique array all at once, then fill it with a single expression that doesn't require resizing, because we know the nuniques at this point.

@jreback jreback added Clean Difficulty Advanced Performance Memory or execution speed performance labels Dec 14, 2016
@jreback jreback added this to the Next Major Release milestone Dec 14, 2016
jreback added a commit to jreback/pandas that referenced this issue Dec 15, 2016
@jreback
Copy link
Contributor Author

jreback commented Dec 15, 2016

closing this. perf degrades somehat so may not be worth this.

@jreback jreback closed this as completed Dec 15, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Clean Performance Memory or execution speed performance
Projects
None yet
Development

No branches or pull requests

1 participant