Skip to content

BUG: read_fwf inference should respect skiprows (#11256) #14028

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

dsm054
Copy link
Contributor

@dsm054 dsm054 commented Aug 18, 2016

Fix the fact that we don't skip the rows when inferring colspecs by passing skiprows down the chain until it's needed.

self.f = f
self.buffer = None
self.delimiter = '\r\n' + delimiter if delimiter else '\n\r\t '
self.comment = comment
if skiprows is None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you don't need this here (as you do it below)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't sure if the path I was using was the only way these functions were called, but I can remove all but one.

@jreback jreback added API Design IO CSV read_csv, to_csv labels Aug 18, 2016
@jreback
Copy link
Contributor

jreback commented Sep 9, 2016

can you rebase / update?

@jorisvandenbossche
Copy link
Member

@dsm054 Do you have time to update this?

@jreback
Copy link
Contributor

jreback commented Dec 26, 2016

can you rebase / update

pls move release note to 0.20.0

@dsm054
Copy link
Contributor Author

dsm054 commented Jan 10, 2017

Okay, will have another look at this now while rooting for Clemson.

@dsm054 dsm054 force-pushed the bugfix/fwf_skiprows branch from 11aa4e3 to b5b3e66 Compare January 10, 2017 02:01
@codecov-io
Copy link

codecov-io commented Jan 10, 2017

Current coverage is 84.77% (diff: 95.00%)

Merging #14028 into master will increase coverage by <.01%

@@             master     #14028   diff @@
==========================================
  Files           145        145          
  Lines         51212      51220     +8   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43413      43421     +8   
  Misses         7799       7799          
  Partials          0          0          

Powered by Codecov. Last update e1a4144...b5b3e66

@jreback jreback added this to the 0.20.0 milestone Jan 10, 2017
@jreback jreback closed this in aa03e7f Jan 10, 2017
@jreback
Copy link
Contributor

jreback commented Jan 10, 2017

thanks @dsm054

if you want to do a followup to handle comment= in the same way would be great!

and I guess skip_blank_lines as well?

AnkurDedania pushed a commit to AnkurDedania/pandas that referenced this pull request Mar 21, 2017
Fix the fact that we don't skip the rows when inferring colspecs by
passing skiprows down the chain until it's needed.  - [X] closes
pandas-dev#11256 - [X] 3 tests added / passed - [X] passes `git diff
upstream/master | flake8 --diff` - [X] whatsnew entry

Author: D.S. McNeil <[email protected]>

Closes pandas-dev#14028 from dsm054/bugfix/fwf_skiprows and squashes the following commits:

b5b3e66 [D.S. McNeil] BUG: read_fwf inference should respect skiprows (pandas-dev#11256)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API Design IO CSV read_csv, to_csv
Projects
None yet
Development

Successfully merging this pull request may close these issues.

read_fwf does not take into account skiprows when inferring column width
4 participants