BUG: Parse missing values using read_json with dtype=False to NaN instead of None (GH28501) #37834
Conversation
…tead of None (GH28501)
jreback
added
the
Missing-data
There was a problem hiding this comment.
yep this is way more consistent, thanks @avinashpancham
cc @WillAyd merge when ready
|
Nice PR! Looks like there is a merge conflict in the whatsnew - if you can fix that up we can get this merged |
|
Merged master, but CI fails due to unrelated reasons. Will retry with new updates from master later today |
|
@avinashpancham can you merge master once again |
doc/source/whatsnew/v1.2.0.rst
Outdated
| @@ -624,6 +624,7 @@ I/O | |||
| - Bug in :func:`read_html` was raising a ``TypeError`` when supplying a ``pathlib.Path`` argument to the ``io`` parameter (:issue:`37705`) | |||
| - :meth:`to_excel` and :meth:`to_markdown` support writing to fsspec URLs such as S3 and Google Cloud Storage (:issue:`33987`) | |||
| - Bug in :meth:`read_fw` was not skipping blank lines (even with ``skip_blank_lines=True``) (:issue:`37758`) | |||
| - Parse missing values using :func:`read_json` with ``dtype=False`` to NaN instead of None (:issue:`28501`) | |||
There was a problem hiding this comment.
use double backticks around NaN and None
|
@jreback merged master and updated whatsnew entry |
|
hmm precommit checks are failing, can you merge master and ping on green |
|
Still some issues, will try again tomorrow |
avinashpancham
force-pushed
the
GH28501
branch
from
14c9f3f to
9fc5989
Compare
|
thanks @avinashpancham |
|
I'm not sure that there was consensus in the issue discussion on the correct behaviour or whether it is a bug. Maybe worth considering moving the release note into the breaking changes section in case anyone is depending on this behaviour. |
type dtype option is actually pretty odd anyhow (and should deprecate) |
|
I think most voices in the issue actually argued that the current behaviour was correct, and so IMO we should not have changed it to start with. |
|
there was exactly 1 comment |
|
Will opened the issue, and 2 people commented on it arguing not to change it (and a third one commented as well, but off-topic), no one else argued in favor of the change. Anyway, I don't think counting the numbers is that important, but I agree with @simonjayhawkins that there was certainly no clear consensus |
|
On the None vs NaN, I am personally quite ambivalent, but when disabling parsing I think we should not return float dtype for this case. |
|
you can revert if u want this was removing a special case though - it was very odd |
black pandasgit diff upstream/master -u -- "*.py" | flake8 --diff