Skip to content

fix to_datetime default error suppression #8894

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

cowpig
Copy link

@cowpig cowpig commented Nov 25, 2014

No description provided.

@jreback
Copy link
Contributor

jreback commented Nov 25, 2014

  • its not nice to change kwarg names without a deprecation cycle.
  • why do you think this is a good idea?
  • I think you could only realistically do this with an option that you could optionally set,
    e.g. change the kwarg to None then if its None use the default value of the option (I am talking
    about pd.set_option(....) here)
  • this would need a fair amount of tests for this change

@jreback jreback added the Datetime Datetime data dtype label Nov 26, 2014
@cowpig
Copy link
Author

cowpig commented Nov 26, 2014

I can see how switching to a boolean can break things, but ignoring errors by default is quite bad. How about I start by just changing the default?

@jreback
Copy link
Contributor

jreback commented Nov 26, 2014

@cowpig well, the problem is pd.to_datetime will not barf on valid but non-parseable datelikes now.

eg.

In [27]: pd.to_datetime('19MAY11',errors='raise')
TypeError: 'NoneType' object is not iterable

In [28]: pd.to_datetime('19MAY11',errors='raise',coerce=True)
Out[28]: NaT

In [29]: pd.to_datetime('19MAY11',errors='raise',format='%d%b%y')
Out[29]: Timestamp('2011-05-19 00:00:00')

So that error message is pretty bad. If you chane the default, then see what error message happens, and could think about doing that. I am not averse, its just that this affects lots of things possibly in subtle way.

@jreback jreback added this to the 0.16.0 milestone Nov 29, 2014
@vfilimonov
Copy link
Contributor

I think it belongs here:

date format "%Y%m%d" does not raise errors even when errors='raise' and silently returns datetime.datetime for out of bounds timestamp:

>>> pd.to_datetime('1300-01-01', format='%Y-%m-%d', errors='raise')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users//.virtual_envs/system/lib/python2.7/site-packages/pandas/tseries/tools.py", line 318, in to_datetime
    return _convert_listlike(np.array([ arg ]), box, format)[0]
  File "/Users//.virtual_envs/system/lib/python2.7/site-packages/pandas/tseries/tools.py", line 306, in _convert_listlike
    raise e
pandas.tslib.OutOfBoundsDatetime: Out of bounds nanosecond timestamp: 1300-01-01 00:00:00
>>> pd.to_datetime('13000101', format='%Y%m%d', errors='raise')
datetime.datetime(1300, 1, 1, 0, 0)

@jreback jreback modified the milestones: 0.16.0, Next Major Release Mar 3, 2015
@jreback
Copy link
Contributor

jreback commented May 9, 2015

closing pls reopen if/when updated

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API Design Datetime Datetime data dtype
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants