Skip to content

CLN: consolidate compression inference as much as possible #12688

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jreback opened this issue Mar 22, 2016 · 5 comments
Closed

CLN: consolidate compression inference as much as possible #12688

jreback opened this issue Mar 22, 2016 · 5 comments
Labels
Clean IO CSV read_csv, to_csv
Milestone

Comments

@jreback
Copy link
Contributor

jreback commented Mar 22, 2016

we have 3 copies of the compression inference routines in parser.py, parser.py, and io/common.py. See how much we can reduce this (some is necessary as sometimes we need to actually return a file handle or need to read in and re-wrap).

xref #12175

@jreback jreback added this to the 0.18.1 milestone Mar 22, 2016
@jreback
Copy link
Contributor Author

jreback commented Mar 22, 2016

cc @lababidi

@jreback
Copy link
Contributor Author

jreback commented May 31, 2016

@gfyoung if you are looking for something to do :>

I think we can blow away _wrap_compresed in parsers/io/parsers.py and incorporate any such functionaility in io/common.py/_get_handle

there is also compression handling in parser.pyx but I think this might be harder to remove (in favor of _get_handle, though maybe if we pass an argument might work).

xref #13317

@gfyoung
Copy link
Member

gfyoung commented May 31, 2016

Haha. Thanks @jreback for the mention. I'll take a closer look when I have the time. 😄

@lababidi
Copy link

@jreback I got the following error when attempting to test:

TypeError: assert_almost_equal() got an unexpected keyword argument 'check_dtype'```

@jreback
Copy link
Contributor Author

jreback commented May 31, 2016

you need to rebuild extensions

python setup.py build_ext --inplace

@lababidi lababidi mentioned this issue Jun 1, 2016
4 tasks
@jorisvandenbossche jorisvandenbossche modified the milestones: Next Major Release, 0.19.0 Sep 1, 2016
dhimmel added a commit to dhimmel/pandas that referenced this issue Dec 13, 2016
dhimmel added a commit to dhimmel/pandas that referenced this issue Dec 13, 2016
@jreback jreback modified the milestones: 0.20.0, Next Major Release Dec 13, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Clean IO CSV read_csv, to_csv
Projects
None yet
4 participants