Skip to content

pd.HDFStore doesn't accept pathlib objects #14705

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
TomAugspurger opened this issue Nov 21, 2016 · 5 comments
Closed

pd.HDFStore doesn't accept pathlib objects #14705

TomAugspurger opened this issue Nov 21, 2016 · 5 comments
Labels
good first issue IO Data IO issues that don't fit into a more specific label IO HDF5 read_hdf, HDFStore

Comments

@TomAugspurger
Copy link
Contributor

Code Sample, a copy-pastable example if possible

# Your code here
In [1]: from pathlib import Path

In [2]: pd.HDFStore(Path('store.h5'))
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-7a495ee2d608> in <module>()
----> 1 pd.HDFStore(Path('store.h5'))

/Users/tom.augspurger/Envs/py3/lib/python3.5/site-packages/pandas/pandas/io/pytables.py in __init__(self, path, mode, complevel, complib, fletcher32, **kwargs)
    446         self._fletcher32 = fletcher32
    447         self._filters = None
--> 448         self.open(mode=mode, **kwargs)
    449
    450     @property

/Users/tom.augspurger/Envs/py3/lib/python3.5/site-packages/pandas/pandas/io/pytables.py in open(self, mode, **kwargs)
    585
    586         try:
--> 587             self._handle = tables.open_file(self._path, self._mode, **kwargs)
    588         except (IOError) as e:  # pragma: no cover
    589             if 'can not be written' in str(e):

/Users/tom.augspurger/Envs/py3/lib/python3.5/site-packages/tables/file.py in open_file(filename, mode, title, root_uep, filters, **kwargs)
    316
    317     # Finally, create the File instance, and return it
--> 318     return File(filename, mode, title, root_uep, filters, **kwargs)
    319
    320 openFile = previous_api(open_file)

/Users/tom.augspurger/Envs/py3/lib/python3.5/site-packages/tables/file.py in __init__(self, filename, mode, title, root_uep, filters, **kwargs)
    782
    783         # Now, it is time to initialize the File extension
--> 784         self._g_new(filename, mode, **params)
    785
    786         # Check filters and set PyTables format version for new files.

tables/hdf5extension.pyx in tables.hdf5extension.File._g_new (tables/hdf5extension.c:3781)()

tables/utilsextension.pyx in tables.utilsextension.encode_filename (tables/utilsextension.c:5345)()

TypeError: Expected bytes, got PosixPath

Expected Output

same as str for path

xref #11033 for the original Pathlib implementation
xref #13823 for fspath

@TomAugspurger TomAugspurger added IO Data IO issues that don't fit into a more specific label IO HDF5 read_hdf, HDFStore labels Nov 21, 2016
@TomAugspurger TomAugspurger added this to the 0.20.0 milestone Nov 21, 2016
@TomAugspurger
Copy link
Contributor Author

Maybe this should be fixed upstream in pytables now that I think of it...

@jreback
Copy link
Contributor

jreback commented Nov 22, 2016

this was set for the read_*/to_* functions here: 62bed0e

so this should be quite easy to do.

@tadiseshan
Copy link
Contributor

I haven't worked with pathlib before, but I'd like to start taking a look and attempt a pr!

@CianciuStyles
Copy link
Contributor

Hello, I've just checked out the latest version of the repo, and the problem seems to be already fixed:

(pandas-dev) C:\Users\fcian\Documents\Code\pandas>python
Python 3.6.5 |Anaconda, Inc.| (default, Mar 29 2018, 13:32:41) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
>>> pandas.show_versions()

INSTALLED VERSIONS
------------------
commit: 3885cedb884a8b22e0875d9ffbc8d28123d82b48
python: 3.6.5.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 94 Stepping 3, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None

pandas: 0.23.0.dev0+747.g3885ced
pytest: 3.5.0
pip: 9.0.3
setuptools: 39.0.1
Cython: 0.28.1
numpy: 1.14.2
scipy: 1.0.1
pyarrow: 0.9.0
xarray: 0.10.2
IPython: 6.3.1
sphinx: 1.7.2
patsy: 0.5.0
dateutil: 2.7.2
pytz: 2018.3
blosc: None
bottleneck: 1.2.1
tables: 3.4.2
numexpr: 2.6.4
feather: 0.4.0
matplotlib: 2.2.2
openpyxl: 2.5.2
xlrd: 1.1.0
xlwt: 1.3.0
xlsxwriter: 1.0.2
lxml: 4.2.1
bs4: 4.6.0
html5lib: 1.0.1
sqlalchemy: 1.2.6
pymysql: 0.8.0
psycopg2: None
jinja2: 2.10
s3fs: 0.1.4
fastparquet: 0.1.5
pandas_gbq: None
pandas_datareader: None

>>> from pathlib import Path
>>> pandas.HDFStore(Path('store.h5'))
<class 'pandas.io.pytables.HDFStore'>
File path: store.h5

I think this issue can be safely closed.

@TomAugspurger
Copy link
Contributor Author

TomAugspurger commented Apr 14, 2018

Yes, and this is tested here. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue IO Data IO issues that don't fit into a more specific label IO HDF5 read_hdf, HDFStore
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants