TypeError: 'TextFileReader' object is not an iterator

[toobaz]

csv = pd.read_csv(file_path, chunksize=10)
chunk = next(csv)

yields

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-21-ec4c9826eb96> in <module>()
      1 csv = pd.read_csv(file_path, chunksize=10)
----> 2 chunk = next(csv)

TypeError: TextFileReader object is not an iterator

which I didn't expect. Sure, I can do csv = iter(pd.read_csv(file_path, chunksize=10)), but since I don't see any advantage (i.e. indexing) in having a non-iterator iterable, maybe read_csv could directly return an iterator? In other words,

In [3]: csv = pd.read_csv(file_path, chunksize=100)

In [4]: print(len(list(iter(csv))), len(list(iter(csv))))
(23, 0)

is maybe not what people expect from a non-iterator iterable.

(Even) I should be able to submit a PR. The only question is whether 1) this is worth fixing 2) this is worth fixing in pandas rather than in pytables.

[jreback]

TextFileReader and the HDFStore are iterables, not iterators, kind of like lists, so that they are reusable, see a good description here

from a practical point of view, I don't actually think we have a way of reusing these as the point is that these are out-of-core by definition.

So I suppose adding `.next()`` would be ok.

See if anything breaks.