Skip to content

BUG: Fixed float parsing with unit when using pd.to_datetime (GH13834) #13847

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

wcwagner
Copy link
Contributor

@wcwagner wcwagner commented Jul 29, 2016

@codecov-io
Copy link

codecov-io commented Jul 30, 2016

Current coverage is 85.30% (diff: 100%)

Merging #13847 into master will decrease coverage by <.01%

@@             master     #13847   diff @@
==========================================
  Files           139        139          
  Lines         50157      50157          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          42785      42784     -1   
- Misses         7372       7373     +1   
  Partials          0          0          

Powered by Codecov. Last update b7abef4...54a26ee

@sinhrks sinhrks added Bug Datetime Datetime data dtype Dtype Conversions Unexpected or buggy dtype conversions labels Jul 30, 2016
@@ -855,3 +855,5 @@ Bug Fixes

- Bug in ``.to_excel()`` when DataFrame contains a MultiIndex which contains a label with a NaN value (:issue:`13511`)
- Bug in ``pd.read_csv`` in Python 2.x with non-UTF8 encoded, multi-character separated data (:issue:`3404`)

- Bug in ``pd.to_datetime()`` did not cast floats correctly, resulting in truncated datetime (:issue:`13845`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sinhrks sinhrks added this to the 0.19.0 milestone Jul 30, 2016
@@ -2095,7 +2095,7 @@ cpdef array_with_unit_to_datetime(ndarray values, unit, errors='coerce'):
# if we have nulls that are not type-compat
# then need to iterate
try:
iresult = values.astype('i8')
iresult = values.astype('i8', casting='no')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should actually be casting='same_kind'. You can also add copy=False to avoid a copy in the case where the values are already int64.

In [35]: a = np.array([1, 2, 3], dtype='int32')

In [36]: a.astype('int64', casting='no')
TypeError                                 Traceback (most recent call last)
<ipython-input-36-efb09bba64c8> in <module>()
----> 1 a.astype('int64', casting='no')

TypeError: Cannot cast array from dtype('int32') to dtype('int64') according to the rule 'no'

In [37]: a.astype('int64', casting='same_kind')
Out[37]: array([1, 2, 3], dtype=int64)

@jreback
Copy link
Contributor

jreback commented Aug 3, 2016

@wcwagner can you update according to @chris-b1 suggestion

@wcwagner
Copy link
Contributor Author

wcwagner commented Aug 4, 2016

Having trouble w/ my laptop recently, so sorry for slow response. Please lmk if there's anything else, I will try to fix anything asap.

Thanks for the suggestion @chris-b1

@wcwagner
Copy link
Contributor Author

wcwagner commented Aug 9, 2016

@jreback everything good here?

@@ -972,3 +972,5 @@ Bug Fixes

- Bug in ``Index`` raises ``KeyError`` displaying incorrect column when column is not in the df and columns contains duplicate values (:issue:`13822`)
- Bug in ``Period`` and ``PeriodIndex`` creating wrong dates when frequency has combined offset aliases (:issue:`13874`)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

be much more explict here, this is only when unit specified.

@jreback
Copy link
Contributor

jreback commented Aug 9, 2016

lgtm. ping when fixed / green.

@wcwagner
Copy link
Contributor Author

wcwagner commented Aug 9, 2016

@jreback

@jreback
Copy link
Contributor

jreback commented Aug 9, 2016

just about to merge this and I saw:
which don't look right

In [8]: pd.to_datetime(1000.1,unit='D').value
Out[8]: 86400000000000000

In [9]: pd.to_datetime(1000.1,unit='s').value
Out[9]: 1000000000000

@jreback jreback closed this in 49f99ac Aug 9, 2016
@jreback
Copy link
Contributor

jreback commented Aug 9, 2016

@wcwagner nvm my last.....didn't compile the extension :<

thanks @wcwagner

@wcwagner wcwagner deleted the bug/13834 branch August 9, 2016 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Datetime Datetime data dtype Dtype Conversions Unexpected or buggy dtype conversions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG]: Wrong unix timestamp parsing with floating point using pd.to_datetime
5 participants