Timezone issue described in #593 #595
Conversation
| dropdate_dt = (datetime.now() - timedelta(days=1,hours=16)) | ||
| dropdate_dt = dropdate_dt.replace(hour=0,minute=0,second=0,microsecond=0) |
There was a problem hiding this comment.
Can you say more about what this does? If they post more than an hour earlier or later, what will happen?
There was a problem hiding this comment.
If we run this script at 4:01 and they had posted the files already at 3, then the script will execute properly.
If they consistently post the files late, then I'm pretty sure the code will throw some sort of "File not found" error. It uses dropdate_dt to determine the filenames, so it will expect there to be files from yesterday, but the code can only download files from two days ago.
If CHNG posts the files at 5pm one day and at 4pm the next, and we run the code between 4pm and 5pm, the code will throw an assertion error because it found too many files dropped within the last 24 hours.
This code followed the outline of the HSP download scheme, but maybe a simpler solution is to compute the expected filename beforehand and check if it's been uploaded to their server yet. This way the errors would be more intelligible. For now, if we run the script after 8, everything should work properly.
There was a problem hiding this comment.
compute the expected filename beforehand
Yep, this sounds ideal going forward. I'll put this PR through as-is and make a tracking issue for the refactor.
Description
See #593. When we download the CHNG files, the timestamps are automatically converted into the code's timezone, in our case 4pm. We shouldn't run the indicator between 4-6pm Eastern just to be safe.
Changelog
Fixes