Skip to content

BUG: Standardize malformed row handling in Python engine #15913

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 6, 2017

Conversation

gfyoung
Copy link
Member

@gfyoung gfyoung commented Apr 6, 2017

Wrap next(source) when source is a file-buffer object to allow for more uniform error message displaying in the Python engine. Closes #15910.

elif 'newline inside string' in msg:
msg = ('EOF inside string starting with '
'line ' + str(kwargs['row_num']))
raise Exception(msg)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a plain Exception? shouldn't it be a ValueError?

Copy link
Member Author

@gfyoung gfyoung Apr 6, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copied from where it was originally, which according to the blame was refactored by you! 😄

In any case, I agree: we should just remove that line and raise csv.Error now

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm don't remember that :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah csv.Error sounds fine

make a note of this in the whatsnew though

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, done.

@codecov
Copy link

codecov bot commented Apr 6, 2017

Codecov Report

Merging #15913 into master will increase coverage by <.01%.
The diff coverage is 95.65%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #15913      +/-   ##
==========================================
+ Coverage   90.95%   90.96%   +<.01%     
==========================================
  Files         145      145              
  Lines       49534    49531       -3     
==========================================
- Hits        45056    45055       -1     
+ Misses       4478     4476       -2
Flag Coverage Δ
#multiple 88.72% <95.65%> (ø) ⬆️
#single 40.63% <4.34%> (-0.03%) ⬇️
Impacted Files Coverage Δ
pandas/io/parsers.py 95.65% <95.65%> (+0.12%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ba30e3a...884387e. Read the comment docs.

@gfyoung gfyoung force-pushed the invalid-quote-skipfooter branch from e0157d6 to 884387e Compare April 6, 2017 03:07
@gfyoung
Copy link
Member Author

gfyoung commented Apr 6, 2017

@jreback : Everything is green and ready to go.

@jreback jreback added IO CSV read_csv, to_csv Bug labels Apr 6, 2017
@jreback jreback added this to the 0.20.0 milestone Apr 6, 2017
@jreback jreback merged commit a0b089e into pandas-dev:master Apr 6, 2017
@jreback
Copy link
Contributor

jreback commented Apr 6, 2017

thanks @gfyoung

@gfyoung gfyoung deleted the invalid-quote-skipfooter branch April 6, 2017 14:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug IO CSV read_csv, to_csv
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants