Skip to content

FB-survey validation with a generic design to include other pipelines #155

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 154 commits into from
Nov 19, 2020
Merged

FB-survey validation with a generic design to include other pipelines #155

merged 154 commits into from
Nov 19, 2020

Conversation

amartyabasu
Copy link
Contributor

@amartyabasu amartyabasu commented Jul 20, 2020 •
edited by krivard
Loading

Closes #59

  • The new validator reads the daily-files from the disk after pipeline execution. The validations for a given signal-type and geo-value on given day are done against a range of previous days data, that get fetched from the API.
  • The validator does basic checks related to different columns, column types, missing_dates, numeric-constraints, null values.
  • It also checks for upward and downward spikes of data, variations in mean differences of column values grouped by geo_ids

@krivard krivard requested a review from jsharpna July 20, 2020 18:19
@krivard krivard mentioned this pull request Jul 21, 2020
Closed
4 tasks
krivard
krivard reviewed Aug 7, 2020
View reviewed changes
Copy link
Contributor

@krivard krivard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't run this yet, but it looks like it's not quite ready to go.

@krivard krivard marked this pull request as draft September 8, 2020 15:21
@krivard
Copy link
Contributor

krivard commented Sep 8, 2020

This doesn't seem to run, and is full of temp code -- converting to draft to revise

@krivard
Copy link
Contributor

krivard commented Sep 8, 2020

@nmdefries

@krivard krivard force-pushed the fb-package-validation branch from 773d279 to cd8541f Compare September 8, 2020 19:15
@krivard krivard mentioned this pull request Oct 9, 2020
Closed
@nmdefries nmdefries requested a review from krivard October 10, 2020 16:42
@nmdefries nmdefries marked this pull request as ready for review October 10, 2020 16:52
krivard
krivard reviewed Oct 12, 2020
View reviewed changes
Copy link
Contributor

@krivard krivard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just based on a read-through, not running or testing

@nmdefries nmdefries self-assigned this Oct 13, 2020
@krivard krivard removed the request for review from jsharpna October 15, 2020 20:55
nmdefries and others added 22 commits November 2, 2020 12:18
@nmdefries
thread API calls. update plans
be920b1
@nmdefries
use semaphore to improve API threading. Fix error in reference data s…
213fa60 
…ubsetting.
@nmdefries
replace read_geo_signal_combo_files() by querying all-frames. Remove …
6622bee 
…unused vars. Simplify API thread methods. Update plans
@JedGrabman
First draft of geo_id validator
5d8f220 
Code to validate that all geo_id values are valid, by comparing against a list of known values.
@JedGrabman
Fix typo (geo_value -> geo_id)
f2562e6
@JedGrabman
Minor fixes
ac28c33 
- Clarify format vs. value checks
- move files from csv/ -> static/
@JedGrabman
Allow setting static directory in template
0ea0a79
@JedGrabman
rename directory (dev -> scripts)
69881a4 
Renamed directory and file (unique_geoids.R) is now expected to be run from within the directory instead of from one level up.
@nmdefries
update unittests with function name changes
9ac11d8
@nmdefries
update plans
4bbe787
@nmdefries
update plans
e29e8a7
@JedGrabman @nmdefries
Pass through filename for error message
06ba36c 
Co-authored-by: nmdefries <[email protected]>
@nmdefries
update plans. Create starter-issue section
373af74
@nmdefries
update plans
61b0b99
@nmdefries
update plans
82feea8
@nmdefries
update plans
8984aac
@JedGrabman
Add file for national values
538d98e
@JedGrabman
Allow uppercase version of geo_id, but with warning
45046cb
@JedGrabman
Add tests for geo_id value checks
ad32bd3
@nmdefries
Add small error-message changes to tests
42ad65e
@nmdefries
Merge pull request #426 from JedGrabman/jedgrabman/geo_id_validation
728a060 
Find Unexpected Values for geo_id compared to historical geo_ids seen
@nmdefries
fix tests
fb15373
JedGrabman and others added 5 commits November 9, 2020 13:38
@JedGrabman
Make geo_id checks robust
22e88e2 
Automatically determine signals and data sources to use for retrieving geo_values. This adds robustness at the cost of efficiency.
@nmdefries
Merge pull request #470 from JedGrabman/jedgrabman/geo_id_validation
eb737f4 
Make geo_id list creation robust
@nmdefries
update plans with feedback from modeling meeting
b700138
@nmdefries
Merge branch 'main' into fb-package-validation
af05d99
@nmdefries
update plans, small clean-up items
e2c86a3
@krivard krivard merged commit 8a20533 into main Nov 19, 2020
@krivard krivard deleted the fb-package-validation branch February 9, 2021 19:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@krivard krivard krivard left review comments

@nmdefries nmdefries nmdefries left review comments

@benjaminysmith benjaminysmith Awaiting requested review from benjaminysmith

Assignees

@nmdefries nmdefries

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Port the Facebook validation pipeline to be generic and automatable
6 participants
@amartyabasu @krivard @nmdefries @benjaminysmith @Ananya-Joshi @JedGrabman