Feature Request: skipcols in .read_csv

[pylang]

I'd like to read a set of csv files but exclude specific columns. read_csv currently has a usecols keyword, but it requires writing a list of all the columns present. This is a bit tedious and more importantly, not all files have the same columns, so usecols would not work in general cases, whereas a complimentary function would work. Can a skipcols keyword be added to 0.17 that accepts a list of column names and reads all but those columns into a DataFrame? Thanks.

xref #4749
xref #8985
xref #6710

[terrytangyuan]

So read_csv() is defined by calling _make_parser_function() which calls _read(). Any instructions would be appreciated. It's a bit confusing to me. @jreback

[jreback]

parser is a bit complicated. see how usecols is used.

[terrytangyuan]

It looks like the code related to usecols needs to be re-factored before I add things on top of it. I won't be able to re-factor coz I might break a lot of internal things. @jreback

[TomAugspurger]

Is there a spot in the code where when know all the columns before starting to parse the rows? If so you can assign usecols=set(all_cols) - set(skipcols) (would need to fixup the ordering afterwards) and go from there.

[terrytangyuan]

Yeah I did something similar but stopped due to other code related to
usecols. I'll look into it again. Thanks.
On Sep 2, 2015 8:38 AM, "Tom Augspurger" notifications@github.com wrote:

Is there a spot in the code where when know all the columns before
starting to parse the rows? If so you can take set usecols=set(all_cols)

• set(skipcols) (would need to fixup the ordering afterwards) and go from
  there.

—
Reply to this email directly or view it on GitHub
#10882 (comment).

[pylang]

Any progress on this addition? Thanks.