REF: move union_categoricals call outside of cython #40964
Conversation
| return result | ||
|
|
||
|
|
||
| def ensure_dtype_objs(dtype): |
There was a problem hiding this comment.
don't we do this more generally elsewhere? e.g. astype should be relocated
There was a problem hiding this comment.
if we did it would probably be in io.common and i dont see it there
|
gentle ping; lots of annotations to be done around these functions |
| warning_columns = [] | ||
|
|
||
| result = {} | ||
| for name in names: |
There was a problem hiding this comment.
do we not have a generic pandas concat call to handle this? or is that for a followup
There was a problem hiding this comment.
do we not have a generic pandas concat call to handle this? or is that for a followup
im hopeful we can use something more generic, but havent tried it out yet (comment on L336)
|
id like to get this in before the upcoming PR doing annotations for io.parsers, which has recently become a blocker for #41059 |
| """ | ||
| # Conserve intermediate space | ||
| # Caller is responsible for concatenating chunks, | ||
| # see c_parser_wrapper._concatenatve_chunks |
|
thanks, can you catch @simonjayhawkins comment in next pass. |
There's no real perf bump to calling union_categoricals inside cython, better to do it in the python code where we can e.g. get the benefit of mypy. Plus gets us closer to dependency structure goals.