Skip to content

BUG: windows with TemporaryFile an read_csv #13398 #13481

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 11 commits into from
Closed

BUG: windows with TemporaryFile an read_csv #13398 #13481

Changes from 1 commit
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
d8decae
#13398
mbrucher Jun 18, 2016
5af8465
Adding a test with Python engine
mbrucher Jun 18, 2016
119fb65
Added reference to original issue in the test + test the result itsel…
mbrucher Jun 18, 2016
98e476e
Using same way of referencing as just above, consistency.
mbrucher Jun 18, 2016
fd20aaf
Moved the test to the Python parser test file
mbrucher Jun 18, 2016
d8ceb57
lint changes
mbrucher Jun 18, 2016
1c33fb5
Simplified test and added what's new note.
mbrucher Jun 19, 2016
aa3f0aa
lint change
mbrucher Jun 19, 2016
5871625
Grammar
mbrucher Jun 19, 2016
0d54151
Simplified
mbrucher Jun 19, 2016
8b52631
Yet another small update for more general regex
mbrucher Jun 19, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion pandas/io/tests/test_common.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
from pandas.io import common
from pandas.compat import is_platform_windows, StringIO

from pandas import read_csv, concat
from pandas import read_csv, read_table, concat

try:
from pathlib import Path
Expand Down Expand Up @@ -90,6 +90,15 @@ def test_iterator(self):
expected.index = [0 for i in range(len(expected))]
tm.assert_frame_equal(concat(it), expected.iloc[1:])

def test_temporary_file(self):
from tempfile import TemporaryFile
new_file = TemporaryFile("w+")
new_file.write("0 0")
new_file.flush()
new_file.seek(0)

dataframe = read_table(new_file, sep=r"\s+", header=None, engine="python")


Copy link
Member

@gfyoung gfyoung Jun 18, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. No need to import read_table for this. read_csv and read_table are absolutely identical (except for the default delimiter), and read_csv will also trigger this error.

  2. You should check the result of the read_ operation to make sure you aren't getting garbage out.

  3. Make reference the issue (the one you raised) under the def ... statement

4) Do not specify engine='python' - we want to test that both the C and Python engines both work with TemporaryFile, even though the issue was specific to the Python engine. This is better coverage.

  1. Actually, since you do need that regex sep, move the test into python_parser_only.py- I forgot that initially when I said that common.py was the right place to go.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You meqn reference the issue in the test (comment)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly:

def test_temporary_file(self):
    # see gh-13398
    ...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are those tests run? They all fail when called explicitely (and they should).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, they do. If you look at test_parsers.py, all of them get imported as test cases.

Copy link
Contributor Author

@mbrucher mbrucher Jun 18, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of the errors I get:

> nosetests -s pandas\io\tests\parser\python_parser_only.py:PythonParserTests
> EEEEEEEE
> ======================================================================
> ERROR: pandas.io.tests.parser.python_parser_only.PythonParserTests.test_BytesIO_input
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "G:\Anaconda3\lib\site-packages\nose\case.py", line 198, in runTest
>     self.test(*self.arg)
>   File "G:\Informatique\pandas\pandas\io\tests\parser\python_parser_only.py", line 84, in test_BytesIO_input
>     result = self.read_table(data, sep="::", encoding='cp1255')
> AttributeError: 'PythonParserTests' object has no attribute 'read_table'
> 
> ======================================================================
> ERROR: pandas.io.tests.parser.python_parser_only.PythonParserTests.test_decompression_regex_sep
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "G:\Anaconda3\lib\site-packages\nose\case.py", line 198, in runTest
>     self.test(*self.arg)
>   File "G:\Informatique\pandas\pandas\io\tests\parser\python_parser_only.py", line 133, in test_decompression_regex_sep
>     data = open(self.csv1, 'rb').read()
> AttributeError: 'PythonParserTests' object has no attribute 'csv1'
> 
> ======================================================================
> ERROR: pandas.io.tests.parser.python_parser_only.PythonParserTests.test_negative_skipfooter_raises
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "G:\Anaconda3\lib\site-packages\nose\case.py", line 198, in runTest
>     self.test(*self.arg)
>   File "G:\Informatique\pandas\pandas\io\tests\parser\python_parser_only.py", line 34, in test_negative_skipfooter_raises
>     self.read_csv(StringIO(text), skipfooter=-1)
> AttributeError: 'PythonParserTests' object has no attribute 'read_csv'

And they are proper issues in the test file.

class TestMMapWrapper(tm.TestCase):

Expand Down