Skip to content

Fixed Value Error when doing HDFStore.Select of contiguous mixed-data #17670

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 28, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v0.21.0.txt
Original file line number Diff line number Diff line change
Expand Up @@ -580,6 +580,7 @@ I/O
- Bug in :func:`read_html` where import check fails when run in multiple threads (:issue:`16928`)
- Bug in :func:`read_csv` where automatic delimiter detection caused a ``TypeError`` to be thrown when a bad line was encountered rather than the correct error message (:issue:`13374`)
- Bug in ``DataFrame.to_html()`` with ``notebook=True`` where DataFrames with named indices or non-MultiIndex indices had undesired horizontal or vertical alignment for column or row labels, respectively (:issue:`16792`)
- Bug in :func:`HDFStore.select` when reading a contiguous mixed-data table featuring VLArray (:issue:`17021`)

Plotting
^^^^^^^^
Expand Down
5 changes: 2 additions & 3 deletions pandas/io/pytables.py
Original file line number Diff line number Diff line change
Expand Up @@ -2441,13 +2441,12 @@ def read_array(self, key, start=None, stop=None):
""" read an array for the specified node (off of group """
import tables
node = getattr(self.group, key)
data = node[start:stop]
attrs = node._v_attrs

transposed = getattr(attrs, 'transposed', False)

if isinstance(node, tables.VLArray):
ret = data[0]
ret = node[0][start:stop]
else:
dtype = getattr(attrs, 'value_type', None)
shape = getattr(attrs, 'shape', None)
Expand All @@ -2456,7 +2455,7 @@ def read_array(self, key, start=None, stop=None):
# length 0 axis
ret = np.empty(shape, dtype=dtype)
else:
ret = data
ret = node[start:stop]

if dtype == u('datetime64'):

Expand Down
13 changes: 13 additions & 0 deletions pandas/tests/io/test_pytables.py
Original file line number Diff line number Diff line change
Expand Up @@ -4387,6 +4387,19 @@ def test_path_pathlib(self):
lambda p: pd.read_hdf(p, 'df'))
tm.assert_frame_equal(df, result)

@pytest.mark.parametrize('start, stop', [(0, 2), (1, 2), (None, None)])
def test_contiguous_mixed_data_table(self, start, stop):
# GH 17021
# ValueError when reading a contiguous mixed-data table ft. VLArray
df = DataFrame({'a': Series([20111010, 20111011, 20111012]),
'b': Series(['ab', 'cd', 'ab'])})

with ensure_clean_store(self.path) as store:
store.append('test_dataset', df)

result = store.select('test_dataset', start=start, stop=stop)
assert_frame_equal(df[start:stop], result)

def test_path_pathlib_hdfstore(self):
df = tm.makeDataFrame()

Expand Down