Skip to content

float precision issue with rolling sum #19308

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
paulgliu opened this issue Jan 18, 2018 · 3 comments
Closed

float precision issue with rolling sum #19308

paulgliu opened this issue Jan 18, 2018 · 3 comments
Labels
Dtype Conversions Unexpected or buggy dtype conversions Duplicate Report Duplicate issue or pull request Reshaping Concat, Merge/Join, Stack/Unstack, Explode

Comments

@paulgliu
Copy link

Code Sample, a copy-pastable example if possible

import pandas as pd
df = pd.DataFrame({'foo': [1.0001] * 5 + [0.]*5})
df
foo
0 1.0001
1 1.0001
2 1.0001
3 1.0001
4 1.0001
5 0.0000
6 0.0000
7 0.0000
8 0.0000
9 0.0000
df.rolling(2).sum()
foo
0 NaN
1 2.000200e+00
2 2.000200e+00
3 2.000200e+00
4 2.000200e+00
5 1.000100e+00
6 4.440892e-16
7 4.440892e-16
8 4.440892e-16
9 4.440892e-16

Problem description

Rolling sum of zeros sometimes gives a very small negative number, which is not the desired behavior.

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.5.4.final.0
python-bits: 64
OS: Linux
OS-release: 4.1.35-pv-ts2
machine: x86_64
processor:
byteorder: little
LC_ALL: en_US.utf8
LANG: en_US.utf8
LOCALE: en_US.UTF-8

pandas: 0.20.3
pytest: 3.2.1
pip: 9.0.1
setuptools: 36.4.0
Cython: 0.26
numpy: 1.12.1
scipy: 0.19.1
xarray: None
IPython: 6.1.0
sphinx: 1.6.3
patsy: 0.4.1
dateutil: 2.6.1
pytz: 2017.2
blosc: None
bottleneck: 1.2.1
tables: 3.4.2
numexpr: 2.6.2
feather: None
matplotlib: 2.0.2
openpyxl: 2.4.8
xlrd: 1.1.0
xlwt: None
xlsxwriter: 0.9.8
lxml: None
bs4: 4.6.0
html5lib: 0.9999999
sqlalchemy: 1.1.13
pymysql: None
psycopg2: None
jinja2: 2.9.6
s3fs: None
pandas_gbq: None
pandas_datareader: None

@mroeschke
Copy link
Member

xref #13254

@jreback
Copy link
Contributor

jreback commented Jan 18, 2018

duplicate.

@jreback jreback closed this as completed Jan 18, 2018
@jreback jreback added Reshaping Concat, Merge/Join, Stack/Unstack, Explode Dtype Conversions Unexpected or buggy dtype conversions Duplicate Report Duplicate issue or pull request labels Jan 18, 2018
@jreback jreback added this to the No action milestone Jan 18, 2018
@jreback
Copy link
Contributor

jreback commented Jan 18, 2018

thanks for the report @liuguangyu0220 ideally we would use kahan summation to avoid this problem. welcome to take a PR to do so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Dtype Conversions Unexpected or buggy dtype conversions Duplicate Report Duplicate issue or pull request Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

No branches or pull requests

3 participants