Skip to content

NCHS fix removing None files #603

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Dec 4, 2020
Merged

NCHS fix removing None files #603

merged 1 commit into from
Dec 4, 2020

Conversation

eujing
Copy link
Contributor

@eujing eujing commented Dec 4, 2020

Description

When doing archive diffing at the daily level and the daily_cache has actually been populated, we may get empty diffs for files that did not change, and these return a None instead of a file path.

The current code might try to remove these None objects as so:

Traceback (most recent call last):
  File "/home/indicators/.pyenv/versions/3.8.2/lib/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/indicators/.pyenv/versions/3.8.2/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/indicators/runtime/nchs_mortality/delphi_nchs_mortality/__main__.py", line 11, in <module>
    run_module()  # pragma: no cover
  File "/home/indicators/runtime/nchs_mortality/delphi_nchs_mortality/run.py", line 87, in run_module
    arch_diffs(params, daily_arch_diff)
  File "/home/indicators/runtime/nchs_mortality/delphi_nchs_mortality/archive_diffs.py", line 82, in arch_diffs
    remove(diff_file)
TypeError: remove: path should be string, bytes or os.PathLike, not NoneType

This PR updates the tests to make sure this case gets tested, and updates the code to ensure we do not try to remove None objects.

Changelog

Itemize code/test/documentation changes and files added/removed.

  • delphi_nchs_mortality/archive_diffs.py: Check for None diff files.
  • tests/conftest.py: Populate the test daily_cache directory with some files to test empty diffs.
  • tests/test_data/weekly_202025_state_wip_deaths_covid_incidence_prop.csv: File that should have no diff from what is generated.

Fixes

  • No issue created for this yet.

@eujing eujing requested a review from jingjtang December 4, 2020 16:35
Copy link
Contributor

@jingjtang jingjtang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@krivard krivard merged commit e1546d9 into main Dec 4, 2020
@krivard krivard deleted the fix-nchs_mortality branch December 4, 2020 22:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants