Skip to content

Compress CSV responses from the API #281

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 20, 2020
Merged

Compress CSV responses from the API #281

merged 1 commit into from
Nov 20, 2020

Conversation

capnrefsmmat
Copy link
Contributor

#210 added format=csv, but these responses are not currently compressed like JSON responses are.

I'd like to switch to CSV format in the R client, so compression is important to add here.

#210 added format=csv, but these responses are not currently compressed like JSON responses are.

I'd like to switch to CSV format in the R client, so compression is important to add here.
Copy link
Contributor

@korlaxxalrok korlaxxalrok left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, @capnrefsmmat. This seems like a straightforward change.

Additionally, sorry for the MEGA LATE review.

@capnrefsmmat
Copy link
Contributor Author

No problem. Safe to merge for auto-deploy?

@korlaxxalrok
Copy link
Contributor

@capnrefsmmat Yeah, lets merge it today, but maybe be on the lookout for any breakage :)

@capnrefsmmat capnrefsmmat merged commit cd24d88 into main Nov 20, 2020
@capnrefsmmat capnrefsmmat deleted the compress-csv branch November 20, 2020 15:11
@capnrefsmmat
Copy link
Contributor Author

Confirmed that delphi.cmu.edu and api.covidcast.cmu.edu are now serving compressed responses. On one day of JHU confirmed_incidence_num:

  • JSON response: 532 kB, compressed to 14.7 kB
  • CSV response: 183.1 kB, compressed to 11.8 kB

So gzip is great, but for clients that understand CSV natively, the CSV format will be great for having much less to parse.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants