BUG: closes #44312: fixes unwanted TypeError with nullable nested metadata in json_normalize #44325
Conversation
qdbp
force-pushed
the
gh-44312-fix-typerror-on-missing-nested-meta-fields
branch
2 times, most recently
from
d25e6ca to
ba914c2
Compare
qdbp
changed the title
qdbp
force-pushed
the
gh-44312-fix-typerror-on-missing-nested-meta-fields
branch
3 times, most recently
from
022cbb6 to
5346a3a
Compare
| @@ -389,6 +389,8 @@ def _pull_field( | |||
| try: | |||
| if isinstance(spec, list): | |||
| for field in spec: | |||
| if result is None: | |||
There was a problem hiding this comment.
While this fixes the issue, the typing suggests result should be dict[str, Any] from js. Is the typing incorrect or the issue more upstream?
There was a problem hiding this comment.
It's the Any value type on the js dict: once we're one level down we're no longer guaranteed that result is a Dict. It could be (and in this case is) None.
There was a problem hiding this comment.
Proper json typing requires recursive types, which seem to be, as of very recently, tentatively supported: python/mypy#731 (comment). This is a whole other kettle of fish, though, that will no doubt reveal a plethora of similar soundness issues.
There was a problem hiding this comment.
Got it. Could you add an informative message in KeyError and test it if possible?
qdbp
force-pushed
the
gh-44312-fix-typerror-on-missing-nested-meta-fields
branch
from
5346a3a to
04eddf9
Compare
… metadata field is missing
qdbp
force-pushed
the
gh-44312-fix-typerror-on-missing-nested-meta-fields
branch
from
04eddf9 to
1f1917d
Compare
|
thanks @qdbp |
… metadata field is missing (pandas-dev#44325)
Fixes TypeError crash in json_normalize when using errors="ignore" and extracting nullable nested metadata.