Skip to content

BUG: validate multi index names in HDFStore writing #5527

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
bluefir opened this issue Nov 15, 2013 · 2 comments · Fixed by #5634
Closed

BUG: validate multi index names in HDFStore writing #5527

bluefir opened this issue Nov 15, 2013 · 2 comments · Fixed by #5634
Labels
Bug IO HDF5 read_hdf, HDFStore
Milestone

Comments

@bluefir
Copy link

bluefir commented Nov 15, 2013

store_id_map

<class 'pandas.io.pytables.HDFStore'>
File path: C:\output\identifier_map.h5
/identifier_map frame_table (typ->appendable_multi,nrows->26779823,ncols->9,indexers->[index],dc->[RefIdentifierID])

store_id_map.select('identifier_map')

KeyError Traceback (most recent call last)
in ()
----> 1 store_id_map.select('identifier_map')

C:\Python27\lib\site-packages\pandas\io\pytables.pyc in select(self, key, where, start, stop, columns, iterator, chunksize, auto_close, *_kwargs)
456 return TableIterator(self, func, nrows=s.nrows, start=start, stop=stop, chunksize=chunksize, auto_close=auto_close)
457
--> 458 return TableIterator(self, func, nrows=s.nrows, start=start, stop=stop, auto_close=auto_close).get_values()
459
460 def select_as_coordinates(self, key, where=None, start=None, stop=None, *_kwargs):

C:\Python27\lib\site-packages\pandas\io\pytables.pyc in get_values(self)
982
983 def get_values(self):
--> 984 results = self.func(self.start, self.stop)
985 self.close()
986 return results

C:\Python27\lib\site-packages\pandas\io\pytables.pyc in func(_start, _stop)
449 # what we are actually going to do for a chunk
450 def func(_start, _stop):
--> 451 return s.read(where=where, start=_start, stop=_stop, columns=columns, **kwargs)
452
453 if iterator or chunksize is not None:

C:\Python27\lib\site-packages\pandas\io\pytables.pyc in read(self, columns, *_kwargs)
3259 columns.insert(0, n)
3260 df = super(AppendableMultiFrameTable, self).read(columns=columns, *_kwargs)
-> 3261 df.set_index(self.levels, inplace=True)
3262 return df
3263

C:\Python27\lib\site-packages\pandas\core\frame.pyc in set_index(self, keys, drop, append, inplace, verify_integrity)
2827 names.append(None)
2828 else:
-> 2829 level = frame[col].values
2830 names.append(col)
2831 if drop:

C:\Python27\lib\site-packages\pandas\core\frame.pyc in getitem(self, key)
2001 # get column
2002 if self.columns.is_unique:
-> 2003 return self._get_item_cache(key)
2004
2005 # duplicate columns

C:\Python27\lib\site-packages\pandas\core\generic.pyc in _get_item_cache(self, item)
665 return cache[item]
666 except Exception:
--> 667 values = self._data.get(item)
668 res = self._box_item_values(item, values)
669 cache[item] = res

C:\Python27\lib\site-packages\pandas\core\internals.pyc in get(self, item)
1653 def get(self, item):
1654 if self.items.is_unique:
-> 1655 _, block = self._find_block(item)
1656 return block.get(item)
1657 else:

C:\Python27\lib\site-packages\pandas\core\internals.pyc in _find_block(self, item)
1933
1934 def _find_block(self, item):
-> 1935 self._check_have(item)
1936 for i, block in enumerate(self.blocks):
1937 if item in block:

C:\Python27\lib\site-packages\pandas\core\internals.pyc in _check_have(self, item)
1940 def _check_have(self, item):
1941 if item not in self.items:
-> 1942 raise KeyError('no item named %s' % com.pprint_thing(item))
1943
1944 def reindex_axis(self, new_axis, method=None, axis=0, copy=True):

KeyError: u'no item named None'

I am at a loss. What does that mean? I successfully saved the DataFrame but I cannot read it back.

@bluefir
Copy link
Author

bluefir commented Nov 16, 2013

Never mind. One of the columns of the MultiIndex was not named, i.e. it had None as a name. Apparently HDFStore chokes on that. If so, you guys might want to raise an exception when we try to save a frame like that.

@bluefir bluefir closed this as completed Nov 16, 2013
@jtratner
Copy link
Contributor

That's definitely a bug there, so we should address that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug IO HDF5 read_hdf, HDFStore
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants