Skip to content

to_json float precision bug #15716

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
amirshavit opened this issue Mar 17, 2017 · 4 comments
Closed

to_json float precision bug #15716

amirshavit opened this issue Mar 17, 2017 · 4 comments
Labels
IO JSON read_json, to_json, json_normalize
Milestone

Comments

@amirshavit
Copy link

amirshavit commented Mar 17, 2017

xref #15864 for more tests

Code Sample, a copy-pastable example if possible

>>> pd.DataFrame([dict(a_float=0.99999999999999944)]).to_json(double_precision=14)
'{"a_float":{"0":1.0}}'
>>> pd.DataFrame([dict(a_float=0.99999999999999944)]).to_json(double_precision=15)
'{"a_float":{"0":0.1}}'

Problem description

...

Expected Output

'{"a_float":{"0":1.0}}'

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 2.7.12.final.0 python-bits: 64 OS: Linux OS-release: 4.4.0-66-generic machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_GB.UTF-8 LOCALE: None.None

pandas: 0.19.2
nose: 1.3.7
pip: 9.0.1
setuptools: 34.3.2
Cython: 0.24.1
numpy: 1.12.0
scipy: 0.18.1
statsmodelsdouble_precision: 0.6.1
xarray: None
IPython: 5.1.0
sphinx: 1.4.6
patsy: 0.4.1
dateutil: 2.6.0
pytz: 2016.10
blosc: None
bottleneck: 1.1.0
tables: 3.2.3.1
numexpr: 2.6.1
matplotlib: 1.5.3
openpyxl: 2.3.2
xlrd: 1.0.0
xlwt: 1.1.2
xlsxwriter: 0.9.3
lxml: 3.6.4
bs4: 4.5.1
html5lib: None
httplib2: 0.10.3
apiclient: 1.5.3
sqlalchemy: 1.1.2
pymysql: None
psycopg2: None
jinja2: 2.7.3
boto: 2.42.0
pandas_datareader: None

@TomAugspurger
Copy link
Contributor

I think this is expected. You have 17 decimal places (len('99999999999999944')), and the max is 15.

@amirshavit
Copy link
Author

I think 1.0 is expected, not 0.1

@TomAugspurger
Copy link
Contributor

Ha, indeed. Sorry I missed that. Interested in taking a look at what's going on? Probably somewhere around here. That's all implemented in C, so maybe a bit tricky.

@TomAugspurger TomAugspurger added Difficulty Intermediate IO JSON read_json, to_json, json_normalize labels Mar 17, 2017
@TomAugspurger TomAugspurger added this to the 0.20.0 milestone Mar 17, 2017
@jreback jreback modified the milestones: Next Major Release, 0.20.0 Mar 17, 2017
funnycrab pushed a commit to funnycrab/pandas that referenced this issue Apr 2, 2017
This is a fix attempt for issue pandas-dev#15716 as well as pandas-dev#15864.

Note that whenever the frac is incremented, there is a chance that its
value may hit the value of pow10.
@jreback jreback modified the milestones: 0.20.0, Next Major Release Apr 3, 2017
@jreback jreback closed this as completed in 7059d89 Apr 3, 2017
@dblyon
Copy link

dblyon commented Jun 30, 2020

The following solved the issue for me:
json.dumps(df.to_dict(orient='records'))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
IO JSON read_json, to_json, json_normalize
Projects
None yet
Development

No branches or pull requests

4 participants