EHN: to_csv compression accepts file-like object #21249
Conversation
minggli
force-pushed
the
enhancement/fh_compression
branch
from
bd8977d to
00712b5
Compare
Codecov Report
@@ Coverage Diff @@
## master #21249 +/- ##
==========================================
+ Coverage 91.84% 91.84% +<.01%
==========================================
Files 153 153
Lines 49538 49540 +2
==========================================
+ Hits 45499 45501 +2
Misses 4039 4039
Continue to review full report at Codecov.
|
pandas/tests/test_common.py
Outdated
| columns=['X', 'Y', 'Z']), | ||
| Series(100 * [0.123456, 0.234567, 0.567567], name='X')]) | ||
| @pytest.mark.parametrize('method', ['to_csv']) | ||
| def test_compression_size_fh(obj, method, compression): |
There was a problem hiding this comment.
can you make a new fixture alongside the existing to just have the compression options (exclude None), e.g. maybe compression_only (and can change the above test to use as well)
pandas/tests/test_common.py
Outdated
| with tm.ensure_clean() as filename: | ||
| with open(filename, 'w') as fh: | ||
| getattr(obj, method)(fh, compression=compression) | ||
| compressed = os.path.getsize(filename) |
There was a problem hiding this comment.
can you asssert that the fh is still open (inside the with block)?
There was a problem hiding this comment.
🤔 not sure how to do it here. with compression, the original handle is first written and closed and then compressed handle is created in the same name with entirety of strings of original handle enclosed. so assert fh.closed will pass inside the with block at the moment.
jreback
added
Output-Formatting
|
thanks @minggli nice patch! note - we might be able to use this compression_only fixture in some of the parser compression tests (eg. where we would skip on None) |
git diff upstream/master -u -- "*.py" | flake8 --diffHandle an unsupported case when a file-like object instead of path passed into to_csv with compression. According to documentation, compression keyword requires it to be a filename.
At the moment, when a handle is passed, it appears to be uncompressed.
Tentative enhancement.