Skip to content

Add NAN code support to Safegraph #845

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 13 commits into from

Conversation

dshemetov
Copy link
Contributor

@dshemetov dshemetov commented Feb 19, 2021

Description

Work as part #838. This indicator was not filtering nan values and has no privacy thresholding, so the change was simple.

Changelog

  • Add three missingness columns, default to not missing for "sample_size" and write data insufficient to "stderr" in case it's missing (which occurs only if sample size is 1).
  • No region exceptions that I'm aware of

Fixes

* update export utility to export, validate, and test the missing cols
* add deletion coding to the archiver, make it expect missing cols, and
  let it handle comparisons between missing and non-missing CSVs
* add missingness columns to safegraph
* add data insufficient if the stderr is missing
* add tests
dshemetov added 11 commits May 3, 2021 16:55
* update export utility to export, validate, and test the missing cols
* handle deleted rows: replaced with nan values
* handle deleted files: replace with an empty CSV file
* handle comparisons between CSVs with/without missing cols
* update export utility to export, validate, and test the missing cols
* handle deleted rows: replaced with nan values
* handle deleted files: replace with an empty CSV file
* handle comparisons between CSVs with/without missing cols
@dshemetov dshemetov closed this Aug 31, 2021
@dshemetov
Copy link
Contributor Author

Closed because Safegraph has been deactivated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant