Skip to content

BUG: fix skiprows callable infinite loop #45586

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Feb 6, 2022
Merged

BUG: fix skiprows callable infinite loop #45586

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
3f8f7c6
BUG: fix skiprows callable infinite loop
bram2000 Jan 20, 2022
e563d93
BUG: seperate new test
bram2000 Jan 24, 2022
0e09f5a
add whatsnew entry
bram2000 Jan 24, 2022
30cfec6
make note user-centric
bram2000 Jan 24, 2022
831f69c
add engine
bram2000 Jan 24, 2022
ac235d4
move note to IO section
bram2000 Jan 27, 2022
9df6e01
modify comment
bram2000 Jan 27, 2022
c1acd08
Merge branch 'main' into skiprows-function-fix
jreback Jan 28, 2022
85d5ee1
add test for csv
bram2000 Jan 28, 2022
a1f9187
lint
bram2000 Jan 28, 2022
3e41e7d
formatting
bram2000 Jan 28, 2022
196b055
fix whatsnew
bram2000 Jan 28, 2022
3abc40a
test not valid for pyxlsb
bram2000 Jan 28, 2022
ecd25b9
Merge branch 'main' into skiprows-function-fix
bram2000 Jan 31, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions pandas/io/parsers/python_parser.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -667,6 +667,8 @@ def _is_line_empty(self, line: list[Scalar]) -> bool:
def _next_line(self) -> list[Scalar]:
if isinstance(self.data, list):
while self.skipfunc(self.pos):
if self.pos >= len(self.data):
break
self.pos += 1

while True:
Expand Down
16 changes: 16 additions & 0 deletions pandas/tests/io/excel/test_readers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1148,6 +1148,22 @@ def test_read_excel_skiprows(self, request, read_ext):
)
tm.assert_frame_equal(actual, expected)

actual = pd.read_excel(
"testskiprows" + read_ext,
sheet_name="skiprows_list",
skiprows=lambda x: x not in [1, 3, 5],
)
expected = DataFrame(
[
[1, 2.5, pd.Timestamp("2015-01-01"), True],
# [2, 3.5, pd.Timestamp("2015-01-02"), False],
[3, 4.5, pd.Timestamp("2015-01-03"), False],
# [4, 5.5, pd.Timestamp("2015-01-04"), True],
],
columns=["a", "b", "c", "d"],
)
tm.assert_frame_equal(actual, expected)

actual = pd.read_excel(
"testskiprows" + read_ext,
sheet_name="skiprows_list",
Expand Down