Wrong error message in HDFStore.append

[bryanfree66]

Updated pytables.py to clarify error message when appending dataframe with None item in previously string-only column

• closes Wrong error message in HDFStore.append when appending dataframe with None item in previously string-only column #16300
• tests passed
• passes git diff upstream/master -u -- "*.py" | flake8 --diff
• whatsnew entry

[datapythonista]

@bryanfree66 can you run the issue case, and report the new error message please?

store = pd.HDFStore('teststore.h5', 'w')
chunk = pd.DataFrame({'V1':['a','b','c','d','e'], 'data': range(5)})
store.append('df', chunk)
chunk = pd.DataFrame({'V1':[None, None], 'data': [3, 5]})
store.append('df', chunk)

[bryanfree66]

import pandas as pd
store = pd.HDFStore('teststore.h5', 'w')
chunk = pd.DataFrame({'V1':['a','b','c','d','e'], 'data': range(5)})
store.append('df', chunk)
chunk = pd.DataFrame({'V1':[None, None], 'data': [3, 5]})
store.append('df', chunk)
Traceback (most recent call last):
File "", line 1, in
File "/Users/bryan/code/pandas/pandas/io/pytables.py", line 988, in append
**kwargs)
File "/Users/bryan/code/pandas/pandas/io/pytables.py", line 1415, in _write_to_group
s.write(obj=value, append=append, complib=complib, **kwargs)
File "/Users/bryan/code/pandas/pandas/io/pytables.py", line 3996, in write
**kwargs)
File "/Users/bryan/code/pandas/pandas/io/pytables.py", line 3667, in create_axes
raise e
File "/Users/bryan/code/pandas/pandas/io/pytables.py", line 3662, in create_axes
info=self.info)
File "/Users/bryan/code/pandas/pandas/io/pytables.py", line 2001, in set_atom
errors)
File "/Users/bryan/code/pandas/pandas/io/pytables.py", line 2048, in set_atom_string
eci = existing_col.validate_col(itemsize)
File "/Users/bryan/code/pandas/pandas/io/pytables.py", line 1739, in validate_col
c.itemsize))
ValueError: Trying to store a string with len [3] in the column [V1], but
this column has a limit of [1]!
Consider using min_itemsize to preset the sizes on these columns

[jreback]

needs a test

[datapythonista]

ValueError: Trying to store a string with len [3] in the column [V1], but
this column has a limit of [1]!
Consider using min_itemsize to preset the sizes on these columns

I don't think this error is correct. The example doesn't try to store a trying with len 3, is trying to store a None. Isn't that right?

[bryanfree66]

Yeah, I noticed another issue too. I’m going to cancel this PR and work on fixing it properly.

…

On Aug 24, 2018, 9:22 AM -0500, Marc Garcia ***@***.***>, wrote: ValueError: Trying to store a string with len [3] in the column [V1], but this column has a limit of [1]! Consider using min_itemsize to preset the sizes on these columns I don't think this error is correct. The example doesn't try to store a trying with len 3, is trying to store a None. Isn't that right? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.