Skip to content

BUG: Add trailing trailing newline in to_json #36898

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Oct 10, 2020

Conversation

Rohith295
Copy link
Contributor

@Rohith295 Rohith295 commented Oct 5, 2020

@jbrockmendel
Copy link
Member

can you add a test for this

@dsaxton dsaxton changed the title To json issue BUG: Add trailing trailing newline in to_json Oct 5, 2020
@dsaxton dsaxton added Bug IO JSON read_json, to_json, json_normalize labels Oct 5, 2020
@Rohith295
Copy link
Contributor Author

can you add a test for this

@jbrockmendel Yes. Will do that

@Rohith295
Copy link
Contributor Author

Rohith295 commented Oct 6, 2020

@jbrockmendel added a test for the bug. And also fixed the breaking tests.

@jbrockmendel
Copy link
Member

cc @WillAyd

Copy link
Member

@WillAyd WillAyd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems reasonable - @jreback

@Rohith295
Copy link
Contributor Author

Can anyone of you complete the PR. @jbrockmendel @WillAyd . There will be again a lot of conflicts if delayed 👍

@jreback jreback added this to the 1.2 milestone Oct 10, 2020
@jreback jreback merged commit 60f08a4 into pandas-dev:master Oct 10, 2020
@jreback
Copy link
Contributor

jreback commented Oct 10, 2020

thanks @Rohith295

kesmit13 pushed a commit to kesmit13/pandas that referenced this pull request Nov 2, 2020
bencap added a commit to seattleflu/id3c that referenced this pull request Jul 29, 2022
We recently upgraded from Pandas 1.1.5 to Pandas 1.4.3. In Pandas 1.2, a bug was fixed that prevented trailing
whitespaces from being added to the end of dumped ndjson output. See [here](pandas-dev/pandas#36898) for additional information.

This change uses `rstrip` on the dumped ndjson output so that all trailing whitespace is removed before this
data is dumped to stdout (or a destination file). Note that this trailing newline affected our assumptions
about the structure of NDJSON files and thus prevented files created after the upgrade from being parsed without
error.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug IO JSON read_json, to_json, json_normalize
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: to_json(lines=True) does not add a trailing newline
5 participants