Skip to content

read_csv: Inconsistent behaviour with parse_dates if the first row is short #11019

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jmopp opened this issue Sep 7, 2015 · 2 comments · Fixed by #44620
Closed

read_csv: Inconsistent behaviour with parse_dates if the first row is short #11019

jmopp opened this issue Sep 7, 2015 · 2 comments · Fixed by #44620
Labels
Error Reporting Incorrect or improved errors from pandas IO CSV read_csv, to_csv Testing pandas testing functions or related to the test suite
Milestone

Comments

@jmopp
Copy link

jmopp commented Sep 7, 2015

I suspect this is related to #6710.

import pandas
from io import StringIO
t = "a,b,c,d\n1970-01-01,2,3,4"
s2 = "a,b,c\n1970-01-01,2,3\n1980-01-01,2,3,4"
s = "a,b,c\n1970-01-01,2,3,4"
pandas.read_csv(StringIO(t), parse_dates=['a'])
pandas.read_csv(StringIO(s2), parse_dates=['a'])
pandas.read_csv(StringIO(s), parse_dates=['a'])

Calling read_csv with parse_dates=['a'] does the correct thing for t, raises a CParserError for s2, and raises TypeError: 'NoneType' object has no attribute '__getitem__' for s.

I would expect a CParserError for s as well.

Pandas version: 0.16.2

@jreback
Copy link
Contributor

jreback commented Sep 8, 2015

yep, could raise a better error message here. pull-requests are welcome.

@jreback jreback added Error Reporting Incorrect or improved errors from pandas IO CSV read_csv, to_csv Difficulty Intermediate labels Sep 8, 2015
@jreback jreback added this to the Next Major Release milestone Sep 8, 2015
@mroeschke mroeschke added the Bug label Apr 18, 2021
@phofl
Copy link
Member

phofl commented Nov 25, 2021

This works as expected now. The s case is inferring the date as index, which happens if rows are longer than header. Also it is not parsed as date, so ok

@phofl phofl closed this as completed Nov 25, 2021
@phofl phofl reopened this Nov 25, 2021
@phofl phofl added Testing pandas testing functions or related to the test suite and removed Bug labels Nov 25, 2021
@jreback jreback modified the milestones: Contributions Welcome, 1.4 Nov 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Error Reporting Incorrect or improved errors from pandas IO CSV read_csv, to_csv Testing pandas testing functions or related to the test suite
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants