Skip to content

Commit 0cd736c

Browse files
gfyoungtm9k1
authored andcommitted
ERR: Fail-fast with incompatible skipfooter combos (pandas-dev#23711)
* ERR: Fail-fast with incompatible skipfooter combos * Don't create the iterator and error immediately if the skipfooter parameter is passed in. * Raise the correct error message when nrows is passed in with skipfooter. * Fix doc lint errors
1 parent 3e501f0 commit 0cd736c

File tree

3 files changed

+22
-11
lines changed

3 files changed

+22
-11
lines changed

doc/source/whatsnew/v0.24.0.rst

+1
Original file line numberDiff line numberDiff line change
@@ -1370,6 +1370,7 @@ Notice how we now instead output ``np.nan`` itself instead of a stringified form
13701370
- Bug in :func:`DataFrame.to_string()` that caused representations of :class:`DataFrame` to not take up the whole window (:issue:`22984`)
13711371
- Bug in :func:`DataFrame.to_csv` where a single level MultiIndex incorrectly wrote a tuple. Now just the value of the index is written (:issue:`19589`).
13721372
- Bug in :meth:`HDFStore.append` when appending a :class:`DataFrame` with an empty string column and ``min_itemsize`` < 8 (:issue:`12242`)
1373+
- Bug in :func:`read_csv()` in which incorrect error messages were being raised when ``skipfooter`` was passed in along with ``nrows``, ``iterator``, or ``chunksize`` (:issue:`23711`)
13731374
- Bug in :meth:`read_csv()` in which :class:`MultiIndex` index names were being improperly handled in the cases when they were not provided (:issue:`23484`)
13741375
- Bug in :meth:`read_html()` in which the error message was not displaying the valid flavors when an invalid one was provided (:issue:`23549`)
13751376
- Bug in :meth:`read_excel()` in which ``index_col=None`` was not being respected and parsing index columns anyway (:issue:`20480`)

pandas/io/parsers.py

+6-5
Original file line numberDiff line numberDiff line change
@@ -787,6 +787,12 @@ def __init__(self, f, engine=None, **kwds):
787787
stacklevel=2)
788788
kwds[param] = dialect_val
789789

790+
if kwds.get("skipfooter"):
791+
if kwds.get("iterator") or kwds.get("chunksize"):
792+
raise ValueError("'skipfooter' not supported for 'iteration'")
793+
if kwds.get("nrows"):
794+
raise ValueError("'skipfooter' not supported with 'nrows'")
795+
790796
if kwds.get('header', 'infer') == 'infer':
791797
kwds['header'] = 0 if kwds.get('names') is None else None
792798

@@ -1054,11 +1060,6 @@ def _failover_to_python(self):
10541060

10551061
def read(self, nrows=None):
10561062
nrows = _validate_integer('nrows', nrows)
1057-
1058-
if nrows is not None:
1059-
if self.options.get('skipfooter'):
1060-
raise ValueError('skipfooter not supported for iteration')
1061-
10621063
ret = self._engine.read(nrows)
10631064

10641065
# May alter columns / col_dict

pandas/tests/io/parser/common.py

+15-6
Original file line numberDiff line numberDiff line change
@@ -537,12 +537,21 @@ def test_iterator(self):
537537
assert len(result) == 3
538538
tm.assert_frame_equal(pd.concat(result), expected)
539539

540-
# skipfooter is not supported with the C parser yet
541-
if self.engine == 'python':
542-
# test bad parameter (skipfooter)
543-
reader = self.read_csv(StringIO(self.data1), index_col=0,
544-
iterator=True, skipfooter=1)
545-
pytest.raises(ValueError, reader.read, 3)
540+
@pytest.mark.parametrize("kwargs", [
541+
dict(iterator=True,
542+
chunksize=1),
543+
dict(iterator=True),
544+
dict(chunksize=1)
545+
])
546+
def test_iterator_skipfooter_errors(self, kwargs):
547+
msg = "'skipfooter' not supported for 'iteration'"
548+
with pytest.raises(ValueError, match=msg):
549+
self.read_csv(StringIO(self.data1), skipfooter=1, **kwargs)
550+
551+
def test_nrows_skipfooter_errors(self):
552+
msg = "'skipfooter' not supported with 'nrows'"
553+
with pytest.raises(ValueError, match=msg):
554+
self.read_csv(StringIO(self.data1), skipfooter=1, nrows=5)
546555

547556
def test_pass_names_with_index(self):
548557
lines = self.data1.split('\n')

0 commit comments

Comments
 (0)