Skip to content

BUG: to_json for DataFrame containing Path objects crash with infinite recursion #36211

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
2 of 3 tasks
rikardn opened this issue Sep 8, 2020 · 2 comments
Open
2 of 3 tasks
Labels
Bug Error Reporting Incorrect or improved errors from pandas IO JSON read_json, to_json, json_normalize

Comments

@rikardn
Copy link

rikardn commented Sep 8, 2020

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • (optional) I have confirmed this bug exists on the master branch of pandas.


When trying to convert a DataFrame that contains Path objects to json it crashes:

import pandas as pd
from pathlib import Path

df = pd.DataFrame({'a': [Path('m1')]})
df.to_json()

output

Traceback (most recent call last):

  File "<ipython-input-67-05359facf0e6>", line 1, in <module>
    df.to_json()

  File "/home/rikard/.local/lib/python3.7/site-packages/pandas/core/generic.py", line 2305, in to_json
    indent=indent,

  File "/home/rikard/.local/lib/python3.7/site-packages/pandas/io/json/_json.py", line 84, in to_json
    indent=indent,

  File "/home/rikard/.local/lib/python3.7/site-packages/pandas/io/json/_json.py", line 144, in write
    self.indent,

  File "/home/rikard/.local/lib/python3.7/site-packages/pandas/io/json/_json.py", line 244, in _write
    indent,

  File "/home/rikard/.local/lib/python3.7/site-packages/pandas/io/json/_json.py", line 166, in _write
    indent=indent,

OverflowError: Maximum recursion level reached

It might be that Path objects are not supported by to_json, but I wouldn't expect an infinite recursion.

INSTALLED VERSIONS

commit : f2ca0a2
python : 3.7.5.final.0
python-bits : 64
OS : Linux
OS-release : 5.3.0-64-generic
Version : #58-Ubuntu SMP Fri Jul 10 19:33:51 UTC 2020
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.1.1
numpy : 1.16.2
pytz : 2019.2
dateutil : 2.7.3
pip : 18.1
setuptools : 41.1.0
Cython : None
pytest : 3.7.4
hypothesis : None
sphinx : 2.4.4
blosc : None
feather : None
xlsxwriter : None
lxml.etree : 4.5.0
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.11.2
IPython : 7.10.2
pandas_datareader: None
bs4 : 4.9.0
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : 3.1.2
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
pyxlsb : None
s3fs : None
scipy : 1.1.0
sqlalchemy : None
tables : None
tabulate : None
xarray : 0.14.1
xlrd : None
xlwt : None
numba : None

@rikardn rikardn added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Sep 8, 2020
@MarcoGorelli
Copy link
Member

Can confirm this reproduces on master, thanks @rikardn for the report

@rhshadrach rhshadrach added IO JSON read_json, to_json, json_normalize Error Reporting Incorrect or improved errors from pandas and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Sep 16, 2020
@Rijgersberg
Copy link

Can confirm. Same problem, and the infinite recursion error was a bit of a head-scratcher.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Error Reporting Incorrect or improved errors from pandas IO JSON read_json, to_json, json_normalize
Projects
None yet
Development

No branches or pull requests

4 participants