DOC: cookbook for in-memory HDFStores usage

[joshlk]

When writing a DataFrame to hdf format ('to_hdf' method) you cannot pass a file handle instead of a string. i.e. you can't:

with open('test.hdf') as f:
    df.to_hdf(f, "df")

This raises an error:

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "C:\Anaconda\lib\site-packages\pandas\core\generic.py", line 896, in to_hdf
    return pytables.to_hdf(path_or_buf, key, self, **kwargs)
  File "C:\Anaconda\lib\site-packages\pandas\io\pytables.py", line 286, in to_hdf
    f(path_or_buf)
  File "C:\Anaconda\lib\site-packages\pandas\io\pytables.py", line 279, in <lambda>
    f = lambda store: store.put(key, value, **kwargs)
AttributeError: 'file' object has no attribute 'put'

This is inconsistent with the 'to_csv' method where you can do this. In the docs it states that you can pass a file buffer to both 'to_hdf' and 'to_csv'.

pd.show_versions(False)

INSTALLED VERSIONS
------------------
commit: None
python: 2.7.5.final.0
python-bits: 64
OS: Windows
OS-release: 7
machine: AMD64
processor: Intel64 Family 6 Model 45 Stepping 7, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None

pandas: 0.14.1
nose: 1.3.4
Cython: 0.22
numpy: 1.9.2
scipy: 0.15.1
statsmodels: None
IPython: 3.0.0
sphinx: 1.2.3
patsy: 0.3.0
scikits.timeseries: None
dateutil: 2.4.1
pytz: 2015.2
bottleneck: None
tables: 3.1.1
numexpr: 2.3.1
matplotlib: 1.4.3
openpyxl: 1.8.5
xlrd: 0.9.3
xlwt: 0.7.5
xlsxwriter: 0.6.7
lxml: 3.4.2
bs4: 4.3.2
html5lib: None
httplib2: 0.9
apiclient: None
rpy2: 2.5.6
sqlalchemy: 0.9.9
pymysql: None
psycopg2: None

[bashtage]

I think the current implementation in master HDFStore or a path. I also think HDFStores can be created with a generic buffer.

[jreback]

we have a tests for this and this would make a nice cookbook recipe.

you still need to give it a string path, but it wont' actually generate a file, see the docs here for how to actually get the image (e.g. the bytes). I am not sure that you can write directly to a Bytes type of object, but if you use this method you can say send the bytes over the wire for reconstruction (though this is simpler with say msgpack support``)

store = HDFStore(path,mode='a',driver='H5FD_CORE',driver_core_backing_store=0)
store['df'] = df
store.append('df2',df)
store.close

[joshlk]

The doc string currently states:

DataFrame.to_hdf(path_or_buf, key, **kwargs):
path_or_buf : the path (string) or buffer to put the store

Should it be changed so its more specific and says you should provide a HDFStore?

[jreback]

yes could change the arg names / doc-string to be more clear

[jreback]

@joshlk going to leave this one open if you (or someone!) wants to add a cookbook recipe for using in-memory stores