Move json_normalize to pd namespace #27615

vishwakftw · 2019-07-26T17:12:09Z

Added a whatsnew entry
Imported pandas.io.json.json_normalize in init.py
closes Move json_normalize to pd namespace #27586
whatsnew entry

- Added a whatsnew entry - Imported pandas.io.json.json_normalize in __init__.py

WillAyd · 2019-07-26T17:22:12Z

Makes sense for the move. @TomAugspurger do you know how we've messaged things like this to users in the past to not import from pandas.io.json anymore?

TomAugspurger · 2019-07-26T19:06:42Z

I don't recall on messaging. Probably something "Accessing json_normalize from pandas.io has been deprecated and will be removed in a future version. Uses pandas.json_normalize instead."

But we'll need to make a pandas.io._json_normalize that has the implementation, and pandas.io.json_normalize will warn then call that. pandas/__init__.py will import io._json_normalize as json_normalize.

pandas/__init__.py

doc/source/whatsnew/v1.0.0.rst

vishwakftw · 2019-07-26T22:37:20Z

Thanks for review @jreback, please allow me to get back on this by the end of the weekend if that’s ok. I’ll also incorporate @TomAugspurger’s comment on deprecation as well.

TomAugspurger · 2019-07-26T22:38:48Z

Thanks! No rush, we’re still a ways out from 1.0.

…

On Jul 26, 2019, at 17:37, Vishwak Srinivasan ***@***.***> wrote: Thanks for review @jreback, please allow me to get back on this by the end of the weekend if that’s ok. I’ll also incorporate @TomAugspurger’s comment on deprecation as well. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

- json_normalize --> _json_normalize - json_normalize inside io.json._normalize is a deprecated method - update whatsnew docs - update io docs - fix broken test_api

pep8speaks · 2019-07-28T16:44:59Z

Hello @vishwakftw! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2019-12-18 19:02:46 UTC

TomAugspurger · 2019-07-29T11:23:40Z

Looks like the docs need to be updated: https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=15186

The line numbers may be a bit off, but hopefully close enough.

doc/source/whatsnew/v1.0.0.rst

- Update whatsnew entry

TomAugspurger · 2019-07-29T14:47:10Z

Can you update doc/source/reference/io.rst so that the reference docs for json_normalize are for the top-level?

It seems that build_table_scheme also is exposed. Would you be willing to do a similar change for it? Either here or as a followup.

vishwakftw · 2019-07-29T14:54:34Z

Can you update doc/source/reference/io.rst so that the reference docs for json_normalize are for the top-level?

Yes, will do that.

It seems that build_table_scheme also is exposed. Would you be willing to do a similar change for it? Either here or as a followup.

Would you like me to expose build_table_schema in the pd namespace?

TomAugspurger · 2019-07-29T15:04:04Z

Possibly. I didn't closely follow the issue discussing json_nomalize, but we should be consistent. @jreback thoughts?

…

On Mon, Jul 29, 2019 at 9:54 AM Vishwak Srinivasan ***@***.***> wrote: Can you update doc/source/reference/io.rst so that the reference docs for json_normalize are for the top-level? Yes, will do that. It seems that build_table_scheme also is exposed. Would you be willing to do a similar change for it? Either here or as a followup. Would you like me to expose build_table_schema in the pd namespace? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#27615?email_source=notifications&email_token=AAKAOIVMVRVD4WW6VZDGKPTQB4ADJA5CNFSM4IHFVMN2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3A67OY#issuecomment-516026299>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAKAOIQ2DKANOQIP4S5GFA3QB4ADJANCNFSM4IHFVMNQ> .

- Update entry in reference/io.rst

jorisvandenbossche · 2019-07-30T10:30:33Z

I also just commented on the issue about build_table_schema (#27586), but I am not sure that that warrants a top-level function.

WillAyd · 2019-07-31T21:05:53Z

build_table_schema doesn't deserve a top level function as it's not generally useful for end users (it's one part of actually writing out in the table format)

It's unfortunate that json_normalize leaked publicly the way it is now but it certainly has usage and enhancement requests coming in for it, hence this PR

TomAugspurger · 2019-07-31T21:09:44Z

I wouldn't say "leaked publicly" though, right? That's the intentional, documented location for it.

…

On Wed, Jul 31, 2019 at 4:06 PM William Ayd ***@***.***> wrote: build_table_schema doesn't deserve a top level function as it's not generally useful for end users (it's one part of actually writing out in the table format) It's unfortunate that json_normalize leaked publicly the way it is now but it certainly has usage and enhancement requests coming in for it, hence this PR — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#27615?email_source=notifications&email_token=AAKAOIUX3GSP7YXWHME5HYLQCH5DTA5CNFSM4IHFVMN2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3IR2NQ#issuecomment-517020982>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAKAOISWNHKHZYOLLMDUJ23QCH5DTANCNFSM4IHFVMNQ> .

WillAyd · 2019-07-31T21:16:34Z

Can’t speak to the entire history of intention but looks like was added alongside read_json in #10301 but never moved to the top level whenever read_json was In any case I don’t think whatever we do with this function needs to dictate what happens for build_table_schema

…

On Jul 31, 2019, at 2:09 PM, Tom Augspurger ***@***.***> wrote: I wouldn't say "leaked publicly" though, right? That's the intentional, documented location for it. On Wed, Jul 31, 2019 at 4:06 PM William Ayd ***@***.***> wrote: > build_table_schema doesn't deserve a top level function as it's not > generally useful for end users (it's one part of actually writing out in > the table format) > > It's unfortunate that json_normalize leaked publicly the way it is now > but it certainly has usage and enhancement requests coming in for it, hence > this PR > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#27615?email_source=notifications&email_token=AAKAOIUX3GSP7YXWHME5HYLQCH5DTA5CNFSM4IHFVMN2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3IR2NQ#issuecomment-517020982>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AAKAOISWNHKHZYOLLMDUJ23QCH5DTANCNFSM4IHFVMNQ> > . > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#27615?email_source=notifications&email_token=AAEU4UIDSHL3FAQU6J7LRO3QCH5RZA5CNFSM4IHFVMN2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3ISDQI#issuecomment-517022145>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEU4UL5H3YJORQL5ITYKOTQCH5RZANCNFSM4IHFVMNQ>.

…nto json_normalize-to-pd

WillAyd · 2019-08-23T13:29:03Z

@vishwakftw can you check doc failure and merge master?

vishwakftw · 2019-08-26T22:07:39Z

@WillAyd I can't seem to understand where the doc is failing. Could you help?

WillAyd · 2019-08-26T22:24:58Z

Hmm I guess the old build logs got cleared. Just restarted it so error should show again in say 20 minutes

…on_normalize-to-pd

vishwakftw · 2019-11-07T21:10:26Z

@WillAyd I was unsure about how to deprecate on import, and was wondering if you had an idea (#27615).

WillAyd · 2019-11-07T22:15:14Z

not sure; this might be OK as is if you can get CI to pass will take another look

vishwakftw · 2019-11-08T22:28:19Z

I'm not sure if the CI failures are relevant.

WillAyd · 2019-11-09T00:58:24Z

Looks like you need to run black on the changes to pass CI

I’ll take a look at the rest this weekend

…nto json_normalize-to-pd

WillAyd

lgtm. Over to @jreback

WillAyd · 2019-12-17T17:47:59Z

@vishwakftw can you fix conflicts and ping on green

…on_normalize-to-pd

…nto json_normalize-to-pd

vishwakftw · 2019-12-17T18:01:41Z

@WillAyd Sure, I've rebased and will ping you once green.

vishwakftw · 2019-12-17T18:54:29Z

@WillAyd There is one failure, I'm not sure if it is related:

=================================== FAILURES ===================================

______________________________ test_scikit_learn _______________________________

[gw1] linux -- Python 3.6.9 /home/travis/miniconda3/envs/pandas-dev/bin/python

df =    A

0  1

1  2

2  3

    @pytest.mark.filterwarnings("ignore:can't:ImportWarning")

    def test_scikit_learn(df):

    

>       sklearn = import_module("sklearn")  # noqa

pandas/tests/test_downstream.py:72: 

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

pandas/tests/test_downstream.py:20: in import_module

    return importlib.import_module(name)

../../../miniconda3/envs/pandas-dev/lib/python3.6/importlib/__init__.py:126: in import_module

    return _bootstrap._gcd_import(name[level:], package, level)

<frozen importlib._bootstrap>:994: in _gcd_import

    ???

<frozen importlib._bootstrap>:971: in _find_and_load

    ???

<frozen importlib._bootstrap>:955: in _find_and_load_unlocked

    ???

<frozen importlib._bootstrap>:665: in _load_unlocked

    ???

<frozen importlib._bootstrap_external>:678: in exec_module

    ???

<frozen importlib._bootstrap>:219: in _call_with_frames_removed

    ???

../../../miniconda3/envs/pandas-dev/lib/python3.6/site-packages/sklearn/__init__.py:75: in <module>

    from .utils._show_versions import show_versions

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    """

    # License: BSD 3 clause

    

    import platform

    import sys

    import importlib

    

>   from ._openmp_helpers import _openmp_parallelism_enabled

E   ImportError: dlopen: cannot load any more object with static TLS

../../../miniconda3/envs/pandas-dev/lib/python3.6/site-packages/sklearn/utils/_show_versions.py:12: ImportError

WillAyd · 2019-12-18T18:35:41Z

Should have been resolved with changes to CI yesterday. Looks like another merge conflict - if you clean up and repush should be green now I think

…on_normalize-to-pd

vishwakftw · 2019-12-18T19:35:38Z

@WillAyd the CI is green now

WillAyd · 2019-12-18T19:53:25Z

Great job with this @vishwakftw !

vishwakftw · 2019-12-18T19:54:11Z

Thank you @WillAyd. Apologies for the delay in getting this merged.

Move json_normalize to pd namespace

ad922ce

- Added a whatsnew entry - Imported pandas.io.json.json_normalize in __init__.py

WillAyd added the API Design label Jul 26, 2019

jreback requested changes Jul 26, 2019

View reviewed changes

pandas/__init__.py Outdated Show resolved Hide resolved

doc/source/whatsnew/v1.0.0.rst Outdated Show resolved Hide resolved

doc/source/whatsnew/v1.0.0.rst Outdated Show resolved Hide resolved

Code review - I

ea57d98

- json_normalize --> _json_normalize - json_normalize inside io.json._normalize is a deprecated method - update whatsnew docs - update io docs - fix broken test_api

vishwakftw added 3 commits July 28, 2019 22:18

Fix lint

84b0543

Fix failing tests due to usage of deprecated function

471d457

Fix lint using black and isort, sorry

c28b637

TomAugspurger reviewed Jul 29, 2019

View reviewed changes

doc/source/whatsnew/v1.0.0.rst Outdated Show resolved Hide resolved

vishwakftw added 2 commits July 29, 2019 19:35

Code Review - II

8b5dba4

- Update whatsnew entry

Code Review - II

c023b68

- Update whatsnew entry

Code Review - III

9ec93e8

- Update entry in reference/io.rst

Merge branch 'json_normalize-to-pd' of github.com:vishwakftw/pandas i…

53ef1bf

…nto json_normalize-to-pd

vishwakftw added 2 commits August 27, 2019 11:31

Merge branch 'master' of https://github.com/pandas-dev/pandas into js…

ce1e0e3

…on_normalize-to-pd

Possible doc failure fix

94b195e

vishwakftw added 2 commits November 8, 2019 14:30

Fix lint error

3af6f03

Merge branch 'master' into json_normalize-to-pd

7402563

vishwakftw added 2 commits November 9, 2019 01:36

Black reformat

6476942

Merge branch 'json_normalize-to-pd' of github.com:vishwakftw/pandas i…

f415dec

…nto json_normalize-to-pd

WillAyd approved these changes Nov 10, 2019

View reviewed changes

Merge branch 'master' into json_normalize-to-pd

746034c

vishwakftw added 2 commits December 17, 2019 12:58

Merge branch 'master' of https://github.com/pandas-dev/pandas into js…

b60698f

…on_normalize-to-pd

Merge branch 'json_normalize-to-pd' of github.com:vishwakftw/pandas i…

7555d37

…nto json_normalize-to-pd

Merge branch 'master' of https://github.com/pandas-dev/pandas into js…

ecc64a8

…on_normalize-to-pd

WillAyd merged commit 5c8bd04 into pandas-dev:master Dec 18, 2019

vishwakftw deleted the json_normalize-to-pd branch December 18, 2019 19:54

vishwakftw restored the json_normalize-to-pd branch December 18, 2019 19:54

vishwakftw deleted the json_normalize-to-pd branch December 18, 2019 19:54

vishwakftw restored the json_normalize-to-pd branch December 18, 2019 19:55

proost pushed a commit to proost/pandas that referenced this pull request Dec 19, 2019

Move json_normalize to pd namespace (pandas-dev#27615)

ca8fe9c

proost pushed a commit to proost/pandas that referenced this pull request Dec 19, 2019

Move json_normalize to pd namespace (pandas-dev#27615)

a29ad15

AlexKirko pushed a commit to AlexKirko/pandas that referenced this pull request Dec 29, 2019

Move json_normalize to pd namespace (pandas-dev#27615)

bdfd3ad

rhshadrach mentioned this pull request Sep 10, 2021

DOC: pandas.json_normalize missing from API Reference portion of documentation #42540

Closed

mroeschke mentioned this pull request Jan 6, 2023

DEPR: Remove pandas.io.json.json_normalize in favor of pd.json_normalize #50608

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move json_normalize to pd namespace #27615

Move json_normalize to pd namespace #27615

vishwakftw commented Jul 26, 2019

WillAyd commented Jul 26, 2019

TomAugspurger commented Jul 26, 2019

vishwakftw commented Jul 26, 2019

TomAugspurger commented Jul 26, 2019 via email

pep8speaks commented Jul 28, 2019 •

edited

Loading

TomAugspurger commented Jul 29, 2019

TomAugspurger commented Jul 29, 2019

vishwakftw commented Jul 29, 2019

TomAugspurger commented Jul 29, 2019 via email

jorisvandenbossche commented Jul 30, 2019

WillAyd commented Jul 31, 2019

TomAugspurger commented Jul 31, 2019 via email

WillAyd commented Jul 31, 2019 via email

WillAyd commented Aug 23, 2019

vishwakftw commented Aug 26, 2019

WillAyd commented Aug 26, 2019

vishwakftw commented Nov 7, 2019

WillAyd commented Nov 7, 2019

vishwakftw commented Nov 8, 2019

WillAyd commented Nov 9, 2019

WillAyd left a comment

WillAyd commented Dec 17, 2019

vishwakftw commented Dec 17, 2019

vishwakftw commented Dec 17, 2019

WillAyd commented Dec 18, 2019

vishwakftw commented Dec 18, 2019

WillAyd commented Dec 18, 2019

vishwakftw commented Dec 18, 2019

Move json_normalize to pd namespace #27615

Move json_normalize to pd namespace #27615

Conversation

vishwakftw commented Jul 26, 2019

WillAyd commented Jul 26, 2019

TomAugspurger commented Jul 26, 2019

vishwakftw commented Jul 26, 2019

TomAugspurger commented Jul 26, 2019 via email

pep8speaks commented Jul 28, 2019 • edited Loading

Comment last updated at 2019-12-18 19:02:46 UTC

TomAugspurger commented Jul 29, 2019

TomAugspurger commented Jul 29, 2019

vishwakftw commented Jul 29, 2019

TomAugspurger commented Jul 29, 2019 via email

jorisvandenbossche commented Jul 30, 2019

WillAyd commented Jul 31, 2019

TomAugspurger commented Jul 31, 2019 via email

WillAyd commented Jul 31, 2019 via email

WillAyd commented Aug 23, 2019

vishwakftw commented Aug 26, 2019

WillAyd commented Aug 26, 2019

vishwakftw commented Nov 7, 2019

WillAyd commented Nov 7, 2019

vishwakftw commented Nov 8, 2019

WillAyd commented Nov 9, 2019

WillAyd left a comment

Choose a reason for hiding this comment

WillAyd commented Dec 17, 2019

vishwakftw commented Dec 17, 2019

vishwakftw commented Dec 17, 2019

WillAyd commented Dec 18, 2019

vishwakftw commented Dec 18, 2019

WillAyd commented Dec 18, 2019

vishwakftw commented Dec 18, 2019

pep8speaks commented Jul 28, 2019 •

edited

Loading