BUG: Fix file descriptor leak #32598
Conversation
gfyoung
added
API - Consistency
| @@ -314,3 +316,15 @@ def test_malformed_skipfooter(python_parser_only): | |||
| msg = "Expected 3 fields in line 4, saw 5" | |||
| with pytest.raises(ParserError, match=msg): | |||
| parser.read_csv(StringIO(data), header=1, comment="#", skipfooter=1) | |||
|
|
|||
|
|
|||
| def test_file_descriptor_leak(python_parser_only): | |||
There was a problem hiding this comment.
Can't we also test this against the C parser (and use the all_parsers fixture instead)?
There was a problem hiding this comment.
No problem, done.
| def test_file_descriptor_leak(python_parser_only): | ||
| # GH 31488 | ||
| parser = python_parser_only | ||
| with open("empty.csv", "w"): |
There was a problem hiding this comment.
Can we try using our ensure_clean utility instead?
| proc = psutil.Process() | ||
| with pytest.raises(EmptyDataError): | ||
| parser.read_csv("empty.csv") | ||
| assert not proc.open_files() |
There was a problem hiding this comment.
I would check that the list of open files does not contain our file, not that it's empty (more vulnerable to failure if the testing environment happens to have other files open unrelated to our test).
There was a problem hiding this comment.
I have now changed the test to a before/after check that triggers when the results differ.
| pass | ||
|
|
||
| proc = psutil.Process() | ||
| with pytest.raises(EmptyDataError): |
There was a problem hiding this comment.
Check the error message as well.
datapythonista
changed the title
roberthdevries
force-pushed
the
fix-31488-read_csv-file-leak-on-EmptyDataError
branch
2 times, most recently
from
0e5012f to
830ed1d
Compare
| # GH 31488 | ||
| import psutil | ||
|
|
||
| proc = psutil.Process() |
There was a problem hiding this comment.
can you add a comment here very similar to what you posted above (e.g. why using psutil open files to check)
roberthdevries
force-pushed
the
fix-31488-read_csv-file-leak-on-EmptyDataError
branch
from
830ed1d to
bc238a0
Compare
Also add a comment why combining tm.ensure_clean and td.check_file_leaks don't play along nicely together
roberthdevries
force-pushed
the
fix-31488-read_csv-file-leak-on-EmptyDataError
branch
from
bc238a0 to
d08bc10
Compare
|
thanks @roberthdevries |
roberthdevries
deleted the
fix-31488-read_csv-file-leak-on-EmptyDataError
branch
black pandasgit diff upstream/master -u -- "*.py" | flake8 --diff