TST: Clean up DataFrame.to_csv compression tests

[reidy-p]

xref #19226

Parametrized some of the compression tests in tests/frame/test_to_csv.py and used the new decompress_file function.

[jreback]

so if you want to take this one step further, you can make compression a fixture (put it in pandas.tests.conftest) the top-level one, then you can simply use the compression arg and it will just work.

[jreback]

I would also change decompress_file to be a contextmanger so you can do

with tm.decompress_file(filename, compress) as fh:
    ......

[pep8speaks]

Hello @reidy-p! Thanks for updating the PR.

Cheers ! There are no PEP8 issues in this Pull Request. 🍻

Comment last updated on January 20, 2018 at 11:31 Hours UTC

[jreback]

looks very nice. not passing though :< ping on green.

[reidy-p]

@jreback the failing tests seem to be caused by the following lines of code which have a pytest.mark.parametrize parameter with the name compression which is the same as the new fixture name:

pandas/pandas/tests/io/parser/test_network.py

Lines 16 to 26 in f0cd23c

	@pytest.mark.network
	@pytest.mark.parametrize(
	"compression,extension", [
	('gzip', '.gz'), ('bz2', '.bz2'), ('zip', '.zip'),
	pytest.param('xz', '.xz', marks=td.skip_if_no_lzma)
	]
	)
	@pytest.mark.parametrize('mode', ['explicit', 'infer'])
	@pytest.mark.parametrize('engine', ['python', 'c'])
	def test_compressed_urls(salaries_table, compression, extension, mode, engine):
	check_compressed_urls(salaries_table, compression, extension, mode, engine)

Do you want to try to combine this into the new compression fixture? It could get a bit messy because the compression here differs from the compression covered by the new fixture but I'll see what I can do.

[jreback]

so best would be to fix that
or could just rename the arg for now

[codecov]

Codecov Report

Merging #19273 into master will not change coverage.
The diff coverage is 100%.

@@           Coverage Diff           @@
##           master   #19273   +/-   ##
=======================================
  Coverage   91.53%   91.53%           
=======================================
  Files         150      150           
  Lines       48739    48739           
=======================================
  Hits        44612    44612           
  Misses       4127     4127

Flag	Coverage Δ
#multiple	89.9% <100%> (ø)	⬆️
#single	41.73% <50%> (ø)	⬆️

Impacted Files	Coverage Δ
pandas/util/testing.py	84.45% <100%> (ø)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1245f06...8077702. Read the comment docs.

[reidy-p]

@jreback it’s green now. I just changed the argument name for now but I will continue with cleaning the compression tests.

[jreback]

thanks @reidy-p keep em coming!

[jorisvandenbossche]

Why does this add a new conftest.py file instead of adding it to the existing one? (which is one level up)

[jreback]

@reidy-p yeah move this 1 level high in next PR (it’s already there just add the contents)

[reidy-p]

Sure. Thanks @jorisvandenbossche