DOC: timeseries.rst floating point precision (#15817) #15919
Conversation
mchwalisz
commented
Apr 6, 2017
mchwalisz
commented
- closes to_datetime() ns precision inconsistencies between s and ns #15817
Codecov Report
@@ Coverage Diff @@
## master #15919 +/- ##
==========================================
- Coverage 90.97% 90.96% -0.01%
==========================================
Files 145 145
Lines 49475 49475
==========================================
- Hits 45008 45007 -1
- Misses 4467 4468 +1
Continue to review full report at Codecov.
|
pandas/tseries/tools.py
Outdated
| @@ -237,6 +237,10 @@ def to_datetime(arg, errors='raise', dayfirst=False, yearfirst=False, | |||
| integer or float number. This will be based off the origin. | |||
| Example, with unit='ms' and origin='unix' (the default), this | |||
| would calculate the number of milliseconds to the unix epoch start. | |||
|
|
|||
| Warning: For float arg, precision rounding might happen. To prevent | |||
| unexpected behavior use a fixed-width exact type. | |||
There was a problem hiding this comment.
this is true only for ns (make that clear); i would put this in the Notes section
There was a problem hiding this comment.
If you mean for unit='ns' I don't agree. You can use unit='s' and have result saved up to mili seconds with rounding error from float. as in timeseries example
There was a problem hiding this comment.
right. ok in any even still put it in the Notes section.
doc/source/timeseries.rst
Outdated
| @@ -265,6 +265,22 @@ Typical epoch stored units | |||
| pd.to_datetime([1349720105100, 1349720105200, 1349720105300, | |||
| 1349720105400, 1349720105500 ], unit='ms') | |||
|
|
|||
| Conversion of float epoch times can lead to inaccurate and unexpected results. | |||
There was a problem hiding this comment.
i would make this a note
also units are rounded to the unit precision so the first example is misleading
There was a problem hiding this comment.
you mean the line as note or the whole section?
also unit is taken as the base but precision is taken from float. First example is exactly the problem I've had at the beginning.
In [36]: pd.to_datetime(1490195805.433502912, unit='s')
Out[36]: Timestamp('2017-03-22 15:16:45.433503')
It neither s nor ns in the example.
There was a problem hiding this comment.
right, that was the example, ok then.
There was a problem hiding this comment.
I mean this whole section you added to make a note (or could do a warning).
doc/source/timeseries.rst
Outdated
| pd.to_datetime(1490195805.433502912, unit='s') | ||
|
|
||
| pd.to_datetime(1490195805433502912, unit='ns') | ||
|
|
||
| These *work*, but the results may be unexpected. |
There was a problem hiding this comment.
you may have to adjust this line after
mchwalisz
force-pushed
the
timeseries-precision
branch
from
93f8716 to
cd10e92
Compare
|
I've moved the warning in |
|
lgtm. @jorisvandenbossche |
doc/source/timeseries.rst
Outdated
| .. warning:: | ||
|
|
||
| Conversion of float epoch times can lead to inaccurate and unexpected results. | ||
| Python floats are unbounded precision, with about 15 significant digits (depends on |
There was a problem hiding this comment.
What do you mean with "unbounded precision"? that seems to contradict a bit with "15 digits"
doc/source/timeseries.rst
Outdated
|
|
||
| Conversion of float epoch times can lead to inaccurate and unexpected results. | ||
| Python floats are unbounded precision, with about 15 significant digits (depends on | ||
| operating system). Rounding in conversion from to high precision ``Timestamp`` is |
There was a problem hiding this comment.
There seems to be something wrong in the "in conversion from to high precision Timestamp" part (it doesn't read fluent to me)
There was a problem hiding this comment.
flaot is missing. will correct it in a sec.
There was a problem hiding this comment.
pls emphasize that this is only on input of very high precision float data.
mchwalisz
force-pushed
the
timeseries-precision
branch
from
cd10e92 to
e25033e
Compare
doc/source/timeseries.rst
Outdated
|
|
||
| .. note:: | ||
|
|
||
| Epoch times will be rounded to the nearest nanosecond. | ||
| These *work*, but the results may be unexpected. |
There was a problem hiding this comment.
this part is now hanging, so add something like. numeric input is in referece to the epoch time (actually may want to move this mini-example (just this last part here) to the 'origin' section.)
mchwalisz
force-pushed
the
timeseries-precision
branch
from
e25033e to
7b82e8b
Compare
|
thanks! |
