Skip to content

PERF: Optimize _categorical_convert CSV parser when categories are known ahead of time #17743

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
TomAugspurger opened this issue Oct 2, 2017 · 0 comments
Open

PERF: Optimize _categorical_convert CSV parser when categories are known ahead of time #17743

TomAugspurger opened this issue Oct 2, 2017 · 0 comments
Labels
Categorical Categorical Data Type IO CSV read_csv, to_csv Performance Memory or execution speed performance

Comments

@TomAugspurger
Copy link
Contributor

TomAugspurger commented Oct 2, 2017 •
edited
Loading

We should be able to speed up

pandas/pandas/_libs/parsers.pyx

Line 1612 in 2310faa

cdef _categorical_convert(parser_t *parser, int64_t col,
when the users passes a CategoricalDtype.

xref #17643
@TomAugspurger TomAugspurger added Categorical Categorical Data Type IO CSV read_csv, to_csv Performance Memory or execution speed performance labels Oct 2, 2017
@TomAugspurger TomAugspurger added this to the Next Major Release milestone Oct 2, 2017
@TomAugspurger TomAugspurger mentioned this issue Oct 2, 2017
Merged
@mroeschke mroeschke removed this from the Contributions Welcome milestone Oct 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Categorical Categorical Data Type IO CSV read_csv, to_csv Performance Memory or execution speed performance
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@TomAugspurger @mroeschke