Skip to content

DEPR: keep_default_dates and convert_dates in read_json #59161

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mroeschke opened this issue Jul 1, 2024 · 6 comments
Open

DEPR: keep_default_dates and convert_dates in read_json #59161

mroeschke opened this issue Jul 1, 2024 · 6 comments
Labels
Deprecate Functionality to remove in pandas Dtype Conversions Unexpected or buggy dtype conversions IO JSON read_json, to_json, json_normalize

Comments

@mroeschke
Copy link
Member

read_json has convert_dates, keep_default_dates and dtypes that all try to perform dtype conversions on the result from the ujson parser. Conflicts can arise when a column matching the specification of keep_default_dates matches a column specified in dtypes.

The proposal is to deprecate keep_default_dates and convert_dates and only use dtype's current definition of

  • True: attempt dtype conversion on all column (and refactor to use infer_dtype instead of astype guessing)
  • False: perform no dtype conversion
  • dict: dict keys that match column labels will be cast to type in the dict value
@mroeschke mroeschke added Dtype Conversions Unexpected or buggy dtype conversions IO JSON read_json, to_json, json_normalize Deprecate Functionality to remove in pandas labels Jul 1, 2024
@mroeschke mroeschke changed the title DEPR: Dtype casting in read_json DEPR: keep_default_dates and convert_dates in read_json Jul 1, 2024
@Animesh-Shukla
Copy link

taking this issue

@phershbe
Copy link
Contributor

@Animesh-Shukla are you still working on this? Your take was from a few weeks ago so I think that I am going to take this since I contributed on here a while ago and want to start again. If you still want it or want to work on it together, let me know!

@Animesh-Shukla
Copy link

you can take it @phershbe

@phershbe
Copy link
Contributor

take

@phershbe
Copy link
Contributor

I have most of a draft pull request mentioned here, but some tests are not passing so I am working on identifying the reasons and fixing them

@phershbe phershbe removed their assignment Aug 22, 2024
@phershbe
Copy link
Contributor

I created a pull request but have struggled with this issue, I don't want to keep in blocked so I unassigned myself in case anybody else would like to work on it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Deprecate Functionality to remove in pandas Dtype Conversions Unexpected or buggy dtype conversions IO JSON read_json, to_json, json_normalize
Projects
None yet
Development

No branches or pull requests

3 participants