When parsing a CSV file without an index, if the list with columns names is too short or too long, one gets a "Index contains duplicate entries" exception.

[mcobzarenco]

Hi Wes,

Just a minor bug submisson:
When parsing a CSV file without an index, if the list with columns names is too short or too long, one gets a "Index contains duplicate entries" exception.

E.g.:

The file test.csv contains:
abc,1,a
abc,2,b
def,3,c

Running:
table = pandas.read_csv('reports/9/test.csv', header=None, names=['a', 'b'],index_col=None)

Results in:

Exception                                 Traceback (most recent call last)
/home/marius/Code/otsquant/analysis/ccapital/<ipython-input-70-7f388b2776b2> in <module>()
----> 1 table = pandas.read_csv('reports/9/test.csv', header=None, names=['a', 'b'],index_col=None)

/usr/lib64/python2.7/site-packages/pandas/io/parsers.pyc in read_csv(filepath_or_buffer, sep, header, index_col, names, skiprows, na_values, parse_dates, date_parser, nrows, iterator, chunksize, skip_footer, converters, verbose, delimiter, encoding)
    125         return parser
    126 
--> 127     return parser.get_chunk()
    128 
    129 @Appender(_read_table_doc)

/usr/lib64/python2.7/site-packages/pandas/io/parsers.pyc in get_chunk(self, rows)
    467         if not index._verify_integrity():
    468             dups = index.get_duplicates()
--> 469             raise Exception('Index has duplicates: %s' % str(dups))
    470 
    471         if len(self.columns) != len(zipped_content):

Exception: Index has duplicates: ['abc']

Many thanks,
Marius

[adamklein]

Under the maxim that explicit is better than implicit, to suppress index inference I added the default "infer_index=True"; by setting to false, the right error will occur.

[wesm]

Hmm, maybe just should raise a better error like "Tried to use columns 1-X as index but had duplicates"

[mcobzarenco]

I'm not sure, I'm thinking that if you set index_col=None, you definitely
don't want an index from the file. Maybe just raise an exception "not
enough column labels"?
On Feb 28, 2012 7:09 PM, "Wes McKinney" <
[email protected]>
wrote:

Hmm, maybe just should raise a better error like "Tried to use columns 1-X
as index but had duplicates"

---

Reply to this email directly or view it on GitHub:
#835 (comment)

[adamklein]

@wesm, fine by me, easy change.

[wesm]

@aristotle137 the trouble is that there are cases where you have N - 1 values in the first row and N values in the subsequent rows (with indices) and you want it to "just work". I would rather support this fairly common use case rather than make those people's lives difficult in order to raise a better error message.

[anirbanspace]

My code worked when i used 'sep' parameter. As following:

df = pd.read_csv(loc_train_labels,header=None,sep=";",names=['Class'])