TST: HDFStore tests don't clean up files after KeyboardInterrupt

[ghost]

Cleanup code exists in the tearDown method, but a KeyboardInterrupt
either doesn't run the tearDown, or maybe the file handle isn't closed so
the os.remove fails.

One example which left behind a file in the root of the source tree.

λ nosetests pandas/io/tests/test_pytables.py
.S......SSSS..^CTraceback (most recent call last):
  File "/usr/local/bin/nosetests", line 9, in <module>
    load_entry_point('nose==1.2.1', 'console_scripts', 'nosetests-2.7')()
  File "/usr/local/lib/python2.7/dist-packages/nose/core.py", line 118, in __init__
    **extra_args)
  File "/usr/lib/python2.7/unittest/main.py", line 95, in __init__
    self.runTests()
  File "/usr/local/lib/python2.7/dist-packages/nose/core.py", line 197, in runTests
    result = self.testRunner.run(self.test)
  File "/usr/local/lib/python2.7/dist-packages/nose/core.py", line 61, in run
    test(result)
  File "/usr/local/lib/python2.7/dist-packages/nose/suite.py", line 176, in __call__
    return self.run(*arg, **kw)
  File "/usr/local/lib/python2.7/dist-packages/nose/suite.py", line 223, in run
    test(orig)
  File "/usr/local/lib/python2.7/dist-packages/nose/suite.py", line 176, in __call__
    return self.run(*arg, **kw)
  File "/usr/local/lib/python2.7/dist-packages/nose/suite.py", line 223, in run
    test(orig)
  File "/usr/local/lib/python2.7/dist-packages/nose/suite.py", line 176, in __call__
    return self.run(*arg, **kw)
  File "/usr/local/lib/python2.7/dist-packages/nose/suite.py", line 223, in run
    test(orig)
  File "/usr/local/lib/python2.7/dist-packages/nose/case.py", line 45, in __call__
    return self.run(*arg, **kwarg)
  File "/usr/local/lib/python2.7/dist-packages/nose/case.py", line 133, in run
    self.runTest(result)
  File "/usr/local/lib/python2.7/dist-packages/nose/case.py", line 151, in runTest
    test(result)
  File "/usr/lib/python2.7/unittest/case.py", line 396, in __call__
    return self.run(*args, **kwds)
  File "/usr/lib/python2.7/unittest/case.py", line 332, in run
    testMethod()
  File "/home/user1/src/pandas/pandas/io/tests/test_pytables.py", line 1690, in test_coordinates
    self.store.append('df1', df1, data_columns=['A', 'B'])
  File "/home/user1/src/pandas/pandas/io/pytables.py", line 532, in append
    self._write_to_group(key, value, table=True, append=True, **kwargs)
  File "/home/user1/src/pandas/pandas/io/pytables.py", line 788, in _write_to_group
    s.write(obj = value, append=append, complib=complib, **kwargs)
  File "/home/user1/src/pandas/pandas/io/pytables.py", line 2502, in write
    self.set_attrs()
  File "/home/user1/src/pandas/pandas/io/pytables.py", line 1984, in set_attrs
    self.attrs.table_type = self.table_type
  File "/usr/local/lib/python2.7/dist-packages/tables/attributeset.py", line 444, in __setattr__
    self._g__setattr(name, value)
  File "/usr/local/lib/python2.7/dist-packages/tables/attributeset.py", line 386, in _g__setattr
    self._g_setAttr(self._v_node, name, stvalue)
  File "hdf5Extension.pyx", line 419, in tables.hdf5Extension.AttributeSet._g_setAttr (tables/hdf5Extension.c:3928)
  File "/usr/local/lib/python2.7/dist-packages/tables/atom.py", line 368, in from_dtype
    return class_.from_kind('string', itemsize, dtype.shape, dflt)
  File "/usr/local/lib/python2.7/dist-packages/tables/atom.py", line 456, in from_kind
    kwargs['itemsize'] = itemsize
KeyboardInterrupt
Closing remaining open files: __26Weihln5J__.h5... done

[jreback]

yup....test need to be run on a clean (non-empty) file in case of ctrl-c.....so if they fail or user interrupts then the .h5 are left behind.....

how do you think we should handle this?

[ghost]

Moving the cleanup to a try/finally around each test body would solve the problem.
If it's not an open handle issue, maybe capture that in a context manager in util.testing?

with ensure_cleanup(path):
   foo

[jreback]

cheating a bit, but seems to work...GH #2755

[ghost]

hmm. race condition if break occurs between closing of file and invocation of tearDown?

sidenote - I usually squash back-and-forth commits with "git rebase -i"

[jreback]

ok...back to context manager
interesting that on windows, pytables would spawn threads to do work, so that if you ctrl-c in the middle then it couldn't close the file
thks for the hints!

[jreback]

@y-p this should be closable as PR was merged in...ok?

[ghost]

ok, thanks for fixing it.

[ghost]

@jreback - oh, sorry. Go ahead and exercise your new and well-earned bit.

:)

[jreback]

thanks!