Skip to content

pd.read_json cryptic error on missing file #5655

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ghost opened this issue Dec 6, 2013 · 8 comments
Closed

pd.read_json cryptic error on missing file #5655

ghost opened this issue Dec 6, 2013 · 8 comments
Labels
API Design Error Reporting Incorrect or improved errors from pandas IO JSON read_json, to_json, json_normalize
Milestone

Comments

@ghost
Copy link

ghost commented Dec 6, 2013

In [12]: pd.read_json("No Such File")
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-12-f479cae2f65a> in <module>()
----> 1 pd.read_json("No Such File")

/home/user1/src/pandas/pandas/io/json.pyc in read_json(path_or_buf, orient, typ, dtype, convert_axes, convert_dates, keep_default_dates, numpy, precise_float, date_unit)
    188         obj = FrameParser(json, orient, dtype, convert_axes, convert_dates,
    189                           keep_default_dates, numpy, precise_float,
--> 190                           date_unit).parse()
    191 
    192     if typ == 'series' or obj is None:

/home/user1/src/pandas/pandas/io/json.pyc in parse(self)
    256 
    257         else:
--> 258             self._parse_no_numpy()
    259 
    260         if self.obj is None:

/home/user1/src/pandas/pandas/io/json.pyc in _parse_no_numpy(self)
    473         if orient == "columns":
    474             self.obj = DataFrame(
--> 475                 loads(json, precise_float=self.precise_float), dtype=None)
    476         elif orient == "split":
    477             decoded = dict((str(k), v)

ValueError: Expected object or value

read_json interprets strings which are not filenames as json data, then fails to parse them
if the filename names a path that doesn't exist (due to typo, or being in wrong directory for example).

That overloading makes it impossible nasty to distinguish two distinct error cases, e.g.
missing file and malformed json. Dubious API choice to my tastes.

in any case, catch both errors and return a saner message "missing file or malformed input" etc.

@jreback
Copy link
Contributor

jreback commented Dec 6, 2013

error could be better, but this is legit, the user passed a non-existant file, so it must be a json-string (that cannot be parsed).

@ghost
Copy link
Author

ghost commented Dec 6, 2013

Right, what I meant by that description is that I'm aware the behavior is by choice but wish
it hadn't been : "If I make a typo, i must mean JSON" is also what this implies. These
eagerly overloaded signatures often create pathologies where there needn't be any.

Nevermind, that's the API, but a better error message would help.

@jreback
Copy link
Contributor

jreback commented Dec 6, 2013

TLTC (too-late-too-cancel)....

@ghost
Copy link
Author

ghost commented Dec 6, 2013

:)

@jreback
Copy link
Contributor

jreback commented Dec 6, 2013

same type of error also will occur with msgpack

@ghost
Copy link
Author

ghost commented Dec 6, 2013

Safety in numbers?

Oh, yeah, better error message there too then I guess.

@jreback
Copy link
Contributor

jreback commented Dec 6, 2013

haha

@ghost
Copy link
Author

ghost commented Dec 19, 2013

Can't fix similar problem in read_msgpack because

pd.read_msgpack("NoSuchFile")

Decodes just fine.

Can't fix it in read_json either because sometimes the Exception error message
carries valuable info on how the json is broken. Rewriting the exception message
based on the specific error message raise is fugly. No.

Sorry, but that's a really pernicious API.

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API Design Error Reporting Incorrect or improved errors from pandas IO JSON read_json, to_json, json_normalize
Projects
None yet
Development

No branches or pull requests

1 participant