Skip to content

BUG: Check complib values #10341

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

BUG: Check complib values #10341

wants to merge 1 commit into from

Conversation

bashtage
Copy link
Contributor

Add check for complib when opening a HDFStore

closes #4582
closes #8874

BUG: Check complib values
59a7fa0 
Add check for complib when opening a HDFStore

closes pandas-dev#4582
closes pandas-dev#8874
jreback
jreback reviewed Jun 12, 2015
View reviewed changes
pandas/io/pytables.py
@@ -389,6 +389,10 @@ def __init__(self, path, mode=None, complevel=None, complib=None,
except ImportError as ex: # pragma: no cover
raise ImportError('HDFStore requires PyTables, "{ex}" problem importing'.format(ex=str(ex)))

if complib not in (None, 'blosc', 'bzip2', 'lzo', 'zlib'):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should iterrogate the PyTables/Filters class, as more things seem to be valid http://www.pytables.org/usersguide/libref/helper_classes.html#filtersclassdescr

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but then reading over the old issue. I am not sure these blosc compressors work (though not entirely sure why)

@bashtage
Copy link
Contributor Author

Using multiple compressors, e.g. blosc:zlib, resulted in corruption for me - also not sure why. So it seems safer to just prevent them being used and to enforce what the docs say are supported compressors.

@jreback
Copy link
Contributor

jreback commented Jun 12, 2015

ok that's fine, move to a class variable though _compressors? to make maintenance easier in the future

@jreback jreback added Bug Error Reporting Incorrect or improved errors from pandas IO HDF5 read_hdf, HDFStore labels Jun 12, 2015
@jreback jreback added this to the 0.16.2 milestone Jun 12, 2015
@jreback
Copy link
Contributor

jreback commented Jun 12, 2015

ping when green. I am going to cut 0.16.2 soon, so this will be last one :)

@jreback
Copy link
Contributor

jreback commented Jun 12, 2015

merged via 3cdd1df

thanks!

@jreback jreback closed this Jun 12, 2015
This was referenced Jun 12, 2015
Closed
Closed
@bashtage bashtage deleted the complib-check branch June 15, 2015 13:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
Bug Error Reporting Incorrect or improved errors from pandas IO HDF5 read_hdf, HDFStore
Projects
None yet
Milestone
0.16.2
Development

Successfully merging this pull request may close these issues.

HDF corrupts data when using complib='blosc:zlib' ER: raise on invalid compression options in HDFStore
2 participants
@bashtage @jreback