Skip to content

Commit 2eef865

Browse files
joshamjreback
authored andcommitted
BUG: fix HDFStore.append with all empty strings error (GH12242) (#23435)
1 parent 555adc2 commit 2eef865

File tree

3 files changed

+12
-1
lines changed

3 files changed

+12
-1
lines changed

doc/source/whatsnew/v0.24.0.txt

+1
Original file line numberDiff line numberDiff line change
@@ -1272,6 +1272,7 @@ Notice how we now instead output ``np.nan`` itself instead of a stringified form
12721272
- Bug in :meth:`detect_client_encoding` where potential ``IOError`` goes unhandled when importing in a mod_wsgi process due to restricted access to stdout. (:issue:`21552`)
12731273
- Bug in :func:`to_string()` that broke column alignment when ``index=False`` and width of first column's values is greater than the width of first column's header (:issue:`16839`, :issue:`13032`)
12741274
- Bug in :func:`DataFrame.to_csv` where a single level MultiIndex incorrectly wrote a tuple. Now just the value of the index is written (:issue:`19589`).
1275+
- Bug in :meth:`HDFStore.append` when appending a :class:`DataFrame` with an empty string column and ``min_itemsize`` < 8 (:issue:`12242`)
12751276

12761277
Plotting
12771278
^^^^^^^^

pandas/io/pytables.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -4637,7 +4637,7 @@ def _convert_string_array(data, encoding, errors, itemsize=None):
46374637
# create the sized dtype
46384638
if itemsize is None:
46394639
ensured = ensure_object(data.ravel())
4640-
itemsize = libwriters.max_len_string_array(ensured)
4640+
itemsize = max(1, libwriters.max_len_string_array(ensured))
46414641

46424642
data = np.asarray(data, dtype="S%d" % itemsize)
46434643
return data

pandas/tests/io/test_pytables.py

+10
Original file line numberDiff line numberDiff line change
@@ -1482,6 +1482,16 @@ def check_col(key, name, size):
14821482
pytest.raises(ValueError, store.append, 'df',
14831483
df, min_itemsize={'foo': 20, 'foobar': 20})
14841484

1485+
def test_append_with_empty_string(self):
1486+
1487+
with ensure_clean_store(self.path) as store:
1488+
1489+
# with all empty strings (GH 12242)
1490+
df = DataFrame({'x': ['a', 'b', 'c', 'd', 'e', 'f', '']})
1491+
store.append('df', df[:-1], min_itemsize={'x': 1})
1492+
store.append('df', df[-1:], min_itemsize={'x': 1})
1493+
tm.assert_frame_equal(store.select('df'), df)
1494+
14851495
def test_to_hdf_with_min_itemsize(self):
14861496

14871497
with ensure_clean_path(self.path) as path:

0 commit comments

Comments
 (0)