COMPAT: Add Pathlib, py.path support for read_hdf #12930
Conversation
| _PATHLIB_INSTALLED = True | ||
| except ImportError: | ||
| _PATHLIB_INSTALLED = False | ||
|
|
|
pls add a whatsnew note (you can put in Enhancements) |
jreback
added
IO Data
quintusdias
force-pushed
the
issue11773
branch
3 times, most recently
from
9cd0ef6 to
e7f609c
Compare
Current coverage is 83.91%@@ master #12930 diff @@
==========================================
Files 136 136
Lines 49918 49927 +9
Methods 0 0
Messages 0 0
Branches 0 0
==========================================
+ Hits 41885 41895 +10
+ Misses 8033 8032 -1
Partials 0 0
|
|
Ok, updates made. On the whatsnew item, there was already a pathlib item for read_csv down in the bugfixes section so I put the read_hdf item there. |
| @@ -270,7 +272,8 @@ def read_hdf(path_or_buf, key=None, **kwargs): | |||
|
|
|||
| Parameters | |||
| ---------- | |||
| path_or_buf : path (string), or buffer to read from | |||
| path_or_buf : path (string), buffer, or path object (pathlib.Path or | |||
| py._path.local.LocalPath) to read from | |||
There was a problem hiding this comment.
add a versionadded tag here (for the last part)
| @@ -4,3 +4,4 @@ numpy | |||
| cython | |||
| nose | |||
| flake8 | |||
| pytables | |||
quintusdias
force-pushed
the
issue11773
branch
3 times, most recently
from
1c575ef to
7232661
Compare
| store = HDFStore(path_or_buf, **kwargs) | ||
| auto_close = True | ||
| else: | ||
| raise NotImplementedError('Support for generic buffers has not ' |
There was a problem hiding this comment.
can you add a tests for this
There was a problem hiding this comment.
test_pytables.TestHDFStore.test_read_hdf_errors should be hitting the else-branch (NotImplementedError), as the path_or_buf object is a regular file pointer (io.TextWrapper) there.
pytables.TestHDFStore.test_read_from_pathlib_path should be hitting the if-branch on python3 (and on python2 if pathlib is installed).
I apologize for my tardiness in responding to these...
|
if you can rebase / update |
quintusdias
force-pushed
the
issue11773
branch
2 times, most recently
from
a9113cc to
3190fa2
Compare
|
Ok, rebased. |
| @@ -32,6 +32,8 @@ Other enhancements | |||
| - The ``.tz_localize()`` method of ``DatetimeIndex`` and ``Timestamp`` has gained the ``errors`` keyword, so you can potentially coerce nonexistent timestamps to ``NaT``. The default behaviour remains to raising a ``NonExistentTimeError`` (:issue:`13057`) | |||
|
|
|||
| - ``Index`` now supports ``.str.extractall()`` which returns ``DataFrame``, see :ref:`Extract all matches in each subject (extractall) <text.extractall>` (:issue:`10008`, :issue:`13156`) | |||
| - ``pd.read_hdf()`` now accepts path objects (e.g. ``pathlib.Path``, ``py.path.local``) for the file path, in line with other ``read_*`` functions (:issue:`11773`) | |||
There was a problem hiding this comment.
you an just make this a single line (the first one) and say .to_hdf()/from_hdf()
|
@quintusdias lgtm. minor comments on docs. pls update / ping when green. |
|
Comments addressed |
| @@ -259,7 +261,13 @@ def to_hdf(path_or_buf, key, value, mode=None, complevel=None, complib=None, | |||
| complib=complib) as store: | |||
| f(store) | |||
| else: | |||
| f(path_or_buf) | |||
| path_or_buf = _stringify_path(path_or_buf) | |||
There was a problem hiding this comment.
I think is redundant, just adding a path_or_buf = _stringify_path(path_or_buf) (before the if should allow this2 to work, rather than adding another test for if instance(path_or_buf, string_tyeps) (which is repeated).
Tests are roundtrip. Closes pandas-dev#11773
|
Changes made. |
|
thanks @quintusdias |
git diff upstream/master | flake8 --diffCloses #11773
Travis-ci test JOB_NAME=27_slow_nnet_LOCALE consistently fails due to locale issue... zh_CN.GB18030??