Add covid_hosp_facility lookup integration tests #1030
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* mixed-facility-metadata inputs * facility-metadata updates
melange396
approved these changes
Nov 17, 2022
| # ...and string of VALUES placeholders | ||
| values_str = ','.join( ['%s'] * len(ak_cols) ) | ||
| # use aggregate key table alias | ||
| ak_table = self.table_name + '_key' | ||
| # assemble full SQL statement | ||
| ak_insert_sql = f'INSERT INTO `{ak_table}` ({ak_cols_str}) VALUES ({values_str}) ON DUPLICATE KEY UPDATE {ak_updates_str}' | ||
| ak_insert_sql = f'INSERT INTO `{ak_table}` ({ak_cols_str}) VALUES ({values_str}) as v ON DUPLICATE KEY UPDATE {ak_updates_str}' |
There was a problem hiding this comment.
Suggested change
| ak_insert_sql = f'INSERT INTO `{ak_table}` ({ak_cols_str}) VALUES ({values_str}) as v ON DUPLICATE KEY UPDATE {ak_updates_str}' | |
| ak_insert_sql = f'INSERT INTO `{ak_table}` ({ak_cols_str}) VALUES ({values_str}) AS v ON DUPLICATE KEY UPDATE {ak_updates_str}' |
Comment on lines
181
to
182
| .sort_values(self.KEY_COLS)[ak_cols] | ||
| .drop_duplicates()) |
There was a problem hiding this comment.
very nice, so any later-dated rows for the same PK will go into the INSERT...ON DUPLICATE after the earlier ones, and thus the newest data will be preserved in the ak_table! it might be worth leaving a note to that effect? also (im beginning to hate this data set) should we apply the dataframe_typemap before this sort, because '2020-10-30'<'2020/09/07' but not '20201030'<'20200907'?
There was a problem hiding this comment.
should we apply the dataframe_typemap before this sort
Good call. We've never seen dates without delimiters in the source data, but we have seen both slashes and dashes, and
>>> "2/3/4" > "2-3-5"
True
>>> "2/3/4" > "2/3/5"
False
😩
krivard
added a commit
that referenced
this pull request
Feb 27, 2023
* Switch to executemany * Add new limited_geocode datatype * Fix test prep missing from #1030
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Extends #1016
Prerequisites:
Unless it is a documentation hotfix it should be merged against thedevbranchdevSummary
Integration tests