-
Notifications
You must be signed in to change notification settings - Fork 67
Apply "quick fix" data ingestion performance improvements #187
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…pdates. Also fixed/improved setUp in integrations/server/test_covidcast_meta.py and added skeleton-code for new `missing_*` rows.
Changed column names back to timestamp1, timestamp2 for testing purposes.
tests changed as i pulled the serialization down a layer
* Implemented the Quick Fix: Direction should be updated to NA wherever there are no historical data to compute the stdev from. * Modified the integration test to test for this behavior
* Updated column names back to value_updated_timestamp and direction_updated_timestamp.
change signal type from VARCHAR(32) to VARCHAR(64) in temporary tables.
Partitioning Direction Update
Tests conflict resolution
Meta cache breakdown
* Database schema * Signal name length guard in CSV uploader * Integration test verifies signal length 32<x<64 succeeds, x>64 fails
Increase maximum signal length for COVIDcast to 64 characters
If I understand the delphi-epidata/src/server/api.php Lines 982 to 991 in dff64d0
|
@sgratzl Yes, though I'm going to call that out of scope for this PR, mostly because staging this batch of changes has been more work than expected. Once it's merged we can do a followup to drop the subquery from the API server code. |
Fixes cmu-delphi/covidcast-indicators#158
Results:
Requires the following setup in prod before merging:
is_wip
andis_latest_issue
for existing rows