BUG: windows with TemporaryFile an read_csv #13398

dcloses #13398

Author: Matthieu Brucher <[email protected]>

Closes #13481 from mbrucher/issue-13398 and squashes the following commits:

8b52631 [Matthieu Brucher] Yet another small update for more general regex
0d54151 [Matthieu Brucher] Simplified
5871625 [Matthieu Brucher] Grammar
aa3f0aa [Matthieu Brucher] lint change
1c33fb5 [Matthieu Brucher] Simplified test and added what's new note.
d8ceb57 [Matthieu Brucher] lint changes
fd20aaf [Matthieu Brucher] Moved the test to the Python parser test file
98e476e [Matthieu Brucher] Using same way of referencing as just above, consistency.
119fb65 [Matthieu Brucher] Added reference to original issue in the test + test the result itself (assuming that previous test is OK)
5af8465 [Matthieu Brucher] Adding a test with Python engine
d8decae [Matthieu Brucher] #13398 Change the way of reading back to readline (consistent with the test before entering the function)

Author: mbrucher

doc/source/whatsnew/v0.18.2.txt

@@ -496,6 +496,7 @@ Bug Fixes
 - Bug in ``pd.read_csv()`` in which the ``nrows`` argument was not properly validated for both engines (:issue:`10476`)
 - Bug in ``pd.read_csv()`` with ``engine='python'`` in which infinities of mixed-case forms were not being interpreted properly (:issue:`13274`)
 - Bug in ``pd.read_csv()`` with ``engine='python'`` in which trailing ``NaN`` values were not being parsed (:issue:`13320`)
+- Bug in ``pd.read_csv()`` with ``engine='python'`` when reading from a tempfile.TemporaryFile on Windows with Python 3 (:issue:`13398`)
 - Bug in ``pd.read_csv()`` that prevents ``usecols`` kwarg from accepting single-byte unicode strings (:issue:`13219`)
 - Bug in ``pd.read_csv()`` that prevents ``usecols`` from being an empty set (:issue:`13402`)
 - Bug in ``pd.read_csv()`` with ``engine=='c'`` in which null ``quotechar`` was not accepted even though ``quoting`` was specified as ``None`` (:issue:`13411`)

pandas/io/parsers.py

@@ -1868,7 +1868,7 @@ class MyDialect(csv.Dialect):
 
         else:
             def _read():
-                line = next(f)
+                line = f.readline()
                 pat = re.compile(sep)
                 yield pat.split(line.strip())
                 for line in f:

pandas/io/tests/parser/python_parser_only.py

@@ -171,3 +171,17 @@ def test_read_table_buglet_4x_multiindex(self):
             columns=list('abcABC'), index=list('abc'))
         actual = self.read_table(StringIO(data), sep='\s+')
         tm.assert_frame_equal(actual, expected)
+
+    def test_temporary_file(self):
+        # GH13398
+        data1 = "0 0"
+
+        from tempfile import TemporaryFile
+        new_file = TemporaryFile("w+")
+        new_file.write(data1)
+        new_file.flush()
+        new_file.seek(0)
+
+        result = self.read_csv(new_file, sep=r"\s*", header=None)
+        expected = DataFrame([[0, 0]])
+        tm.assert_frame_equal(result, expected)