diff --git a/docs/api/README.md b/docs/api/README.md index ebe3876f7..64fd15809 100644 --- a/docs/api/README.md +++ b/docs/api/README.md @@ -1,5 +1,5 @@ --- -title: Epidata API (Other Epidemics) +title: Epidata API (Other Diseases) nav_order: 3 has_children: true --- diff --git a/docs/api/afhsb.md b/docs/api/afhsb.md index 9931ccc6c..3f25e5097 100644 --- a/docs/api/afhsb.md +++ b/docs/api/afhsb.md @@ -1,6 +1,6 @@ --- title: AFHSB -parent: Epidata API (Other Epidemics) +parent: Epidata API (Other Diseases) --- # AFHSB diff --git a/docs/api/cdc.md b/docs/api/cdc.md index d4237c695..1773b8f9d 100644 --- a/docs/api/cdc.md +++ b/docs/api/cdc.md @@ -1,6 +1,6 @@ --- title: CDC -parent: Epidata API (Other Epidemics) +parent: Epidata API (Other Diseases) --- # CDC diff --git a/docs/api/covid_hosp.md b/docs/api/covid_hosp.md index 6ae4abab9..735170abb 100644 --- a/docs/api/covid_hosp.md +++ b/docs/api/covid_hosp.md @@ -1,6 +1,6 @@ --- title: COVID-19 Reported Patient Impact and Hospital Capacity by State Timeseries -parent: Epidata API (Other Epidemics) +parent: Epidata API (Other Diseases) --- # COVID-19 Hospitalization diff --git a/docs/api/covidcast-signals/google-symptoms.md b/docs/api/covidcast-signals/google-symptoms.md new file mode 100644 index 000000000..5cac67bc5 --- /dev/null +++ b/docs/api/covidcast-signals/google-symptoms.md @@ -0,0 +1,85 @@ +--- +title: Google Symptoms +parent: Data Sources and Signals +grand_parent: COVIDcast API +--- + +# Google Symptoms +{: .no_toc} + +* **Source name:** `google-symptoms` +* **First issued:** 30 November 2020 +* **Number of data revisions since 19 May 2020:** 0 +* **Date of last change:** Never +* **Available for:** county, MSA, HRR, state (see [geography coding docs](../covidcast_geography.md)) +* **License:** [CC BY](../covidcast_licensing.md#creative-commons-attribution) + +This data source is based on the [COVID-19 Search Trends symptoms +dataset](https://github.com/google-research/open-covid-19-data/tree/master/data/exports/search_trends_symptoms_dataset). Using +this search data, we estimate the volume of searches mapped to symptoms related +to COVID-19 such as _anosmia_ (lack of smell) and _ageusia_(lack of taste). The +resulting daily dataset for each region shows the relative frequency of searches +for each symptom. The signals are measured in arbitrary units that are +normalized for population and scaled by the maximum value of the normalized +popularity within a geographic region across a specific time range. **Thus, +values are NOT comparable across geographic regions**. Larger numbers represent +higher numbers of symptom-related searches. + +| Signal | Description | +| --- | --- | +| `anosmia_raw_search` | Google search volume for anosmia-related searches, in arbitrary units that are normalized for population | +| `anosmia_smoothed_search` | Google search volume for anosmia-related searches, in arbitrary units that are normalized for population, smoothed by 7-day average | +| `ageusia_raw_search` | Google search volume for ageusia-related searches, in arbitrary units that are normalized for population | +| `ageusia_smoothed_search` | Google search volume for ageusia-related searches, in arbitrary units that are normalized for population, smoothed by 7-day average | +| `sum_anosmia_ageusia_raw_search` | The sum of Google search volume for anosmia and ageusia related searches, in an arbitrary units that are normalized for population | +| `sum_anosmia_ageusia_smoothed_search` | The sum of Google search volume for anosmia and ageusia related searches, in an arbitrary units that are normalized for population, smoothed by 7-day average | + + +## Table of contents +{: .no_toc .text-delta} + +1. TOC +{:toc} +## Estimation +The `sum_anosmia_ageusia_raw_search` signals are simply the raw sum of the + values of `anosmia_raw_search` and `ageusia_raw_search`, but not the union of + anosmia and ageusia related searches. This is because the data volume is + calculated based on search queries. A single search query can be mapped to more + than one symptom. Currently, Google does not provide _intersection/union_ + data. Users should be careful when considering such signals. + +## Limitation +When daily volume in a region does not meet quality or privacy thresholds, set +by Google, no value will be reported. Since Google uses differential privacy, +there is artificial noise added to the raw datasets to avoid identifying any +individual persons without affecting the quality of results. + +The data is normalized by the total number of Search users in certain regions +for a certain time period and is scaled considering the maximum value of the +normalized popularity across the entire published time range for that region +over all symptoms. The values of symptom popularity are **NOT** comparable +across geographic regions. Due to the scaling step, most of the values should be +in the range 0-1. However, since the scaling factor is calculated and stored at +a certain time point, the symptom popularity released after that time point is +likely to exceed the previously-observed maximum value which results in values +larger than 1. + + +## Geographical Aggregation +The state-level and county-level `raw_search` signals for specific symptoms such +as _anosmia_ and _ageusia_ are taken directly from the [COVID-19 Search Trends +symptoms +dataset](https://github.com/google-research/open-covid-19-data/tree/master/data/exports/search_trends_symptoms_dataset) +without changes. We aggregate the county-level data to the MSA and HRR levels +using the population-weighted average. For MSAs/HRRs that include counties that +have no data provided due to quality or privacy issues for a certain day, we +simply assume the values to be 0 during aggregation. The values for MSAs/HRRs +with no counties having non-NaN values will not be reported. Thus, the resulting +MSA/HRR level data does not fully match the _actual_ MSA/HRR level data (which +we are not provided). + + +## Lag and Backfill +Google does not update the search data daily, but has an uncertain update +frequency. The delay can range from 1 day to 10 days or even more. We check for +updates every day and provide the most up-to-date data. diff --git a/docs/api/covidcast-signals/safegraph.md b/docs/api/covidcast-signals/safegraph.md index e1a112bc4..3c14d33cc 100644 --- a/docs/api/covidcast-signals/safegraph.md +++ b/docs/api/covidcast-signals/safegraph.md @@ -4,23 +4,35 @@ parent: Data Sources and Signals grand_parent: COVIDcast Epidata API --- -# SafeGraph Mobility - +# SafeGraph +{: .no_toc} * **Source name:** `safegraph` +* **Available for:** county, MSA, HRR, state (see [geography coding docs](../covidcast_geography.md)) +* **License:** [CC BY](../covidcast_licensing.md#creative-commons-attribution) + +This data source uses data reported by [SafeGraph](https://www.safegraph.com/) +using anonymized location data from mobile phones. SafeGraph provides several +different datasets to eligible researchers. We surface signals from two such +datasets. + +## Table of contents +{: .no_toc .text-delta} + +1. TOC +{:toc} + +## SafeGraph Social Distancing Metrics + * **First issued:** 23 June 2020 * **Number of data revisions since 23 June 2020:** 1 * **Date of last change:** 3 November 2020 -* **Available for:** county, state (see [geography coding docs](../covidcast_geography.md)) -* **License:** [CC BY](../covidcast_licensing.md#creative-commons-attribution) -This data source uses data reported by [SafeGraph](https://www.safegraph.com/) -using anonymized location data from mobile phones. SafeGraph provides [social -distancing metrics](https://docs.safegraph.com/docs/social-distancing-metrics) -to eligible researchers who obtain an API key. SafeGraph provides this data for -individual census block groups, using differential privacy to protect the -privacy of individual people in the data. +Data source based on [social distancing +metrics](https://docs.safegraph.com/docs/social-distancing-metrics). SafeGraph +provides this data for individual census block groups, using differential +privacy to protect individual people's data privacy. -Delphi creates features of the Safegraph data at the census block group level, +Delphi creates features of the SafeGraph data at the census block group level, then aggregates these features to the county and state levels. The aggregated data is freely available through the COVIDcast API. @@ -34,6 +46,10 @@ documentation](https://docs.safegraph.com/docs/social-distancing-metrics). | `full_time_work_prop` | The fraction of mobile devices that spent more than 6 hours at a location other than their home during the daytime (SafeGraph's `full_time_work_behavior_devices / device_count`) | | `part_time_work_prop` | The fraction of devices that spent between 3 and 6 hours at a location other than their home during the daytime (SafeGraph's `part_time_work_behavior_devices / device_count`) | | `median_home_dwell_time` | The median time spent at home for all devices at this location for this time period, in minutes | +| `completely_home_prop_7dav` | Offers a 7-day trailing window average of the `completely_home_prop`. | +| `full_time_work_prop_7dav` | Offers a 7-day trailing window average of the`full_time_work_prop`. | +| `part_time_work_prop_7dav` | Offers a 7-day trailing window average of the`part_time_work_prop`.| +| `median_home_dwell_time_7dav` | Offers a 7-day trailing window average of the `median_home_dwell_time`.| After computing each metric on the census block group (CBG) level, we aggregate to the county-level by taking the mean over CBGs in a county to obtain the value @@ -43,8 +59,63 @@ doing so, we make the simplifying assumption that each CBG contributes an iid observation to the county-level distribution. `n` also serves as the sample size. The same method is used for aggregation to states. -## Lag +SafeGraph's signals measure mobility each day, which causes strong day-of-week +effects: weekends have substantially different values than weekdays. Users +interested in long-term trends, rather than mobility on one specific day, may +prefer the `7dav` signals since averaging over the preceding 7 days removes +these day-of-week effects. + +### Lag SafeGraph provides this data with a three-day lag, meaning estimates for a specific day are only available three days later. It may take up to an additional day for SafeGraph's data to be ingested into the COVIDcast API. + + +## SafeGraph Weekly Patterns + +* **First issued:** 30 November 2020 +* **Number of data revisions since 23 June 2020:** 0 +* **Date of last change:** never + +Data source based on [Weekly +Patterns](https://docs.safegraph.com/docs/weekly-patterns) dataset. SafeGraph +provides this data for different points of interest +([POIs](https://docs.safegraph.com/v4.0/docs#section-core-places)) considering +individual census block groups, using differential privacy to protect individual +people's data privacy. + +Delphi gathers the number of daily visits to POIs of certain types(bars, +restaurants, etc.) from SafeGraph's Weekly Patterns data at the 5-digit ZipCode +level, then aggregates and reports these features to the county, MSA, HRR, and +state levels. The aggregated data is freely available through the COVIDcast API. + +For precise definitions of the quantities below, consult the [SafeGraph Weekly +Patterns documentation](https://docs.safegraph.com/docs/weekly-patterns). + +| Signal | Description | +| --- | --- | +| `bars_visit_num` | The number of daily visits to bar-related POIs in a certain region | +| `bars_visit_prop` | The number of daily visits to bar-related POIs in a certain region, per 100,000 population | +| `restaurants_visit_num` | The number of daily visits to restaurant-related POIs in a certain region | +| `restaurants_visit_prop` | The number of daily visits to restaurant-related POIs in a certain region, per 100,000 population | + +SafeGraph delivers the number of daily visits to U.S. POIs, the details of which +are described in the [Places +Manual](https://readme.safegraph.com/docs/places-manual#section-placekey) +dataset. Delphi aggregates the number of visits to certain types of places, +such as bars (places with [NAICS code = +722410](https://www.census.gov/cgi-bin/sssd/naics/naicsrch?input=722410&search=2017+NAICS+Search&search=2017)) +and restaurants (places with [NAICS code = +722511](https://www.census.gov/cgi-bin/sssd/naics/naicsrch)). For example, +Adagio Teas is coded as a bar because it serves alcohol, while Napkin Burger is +considered to be a full-service restaurant. More information on NAICS codes is +available from the [US Census Bureau: North American Industry Classification +System](https://www.census.gov/eos/www/naics/index.html). + +### Lag + +SafeGraph provides newly updated data for the previous week every Wednesday, +meaning estimates for a specific day are only available 3-9 days later. It may +take up to an additional day for SafeGraph's data to be ingested into the +COVIDcast API. diff --git a/docs/api/delphi.md b/docs/api/delphi.md index 05626f1a1..11a4dc896 100644 --- a/docs/api/delphi.md +++ b/docs/api/delphi.md @@ -1,6 +1,6 @@ --- title: Delphi Forecasts -parent: Epidata API (Other Epidemics) +parent: Epidata API (Other Diseases) --- # Delphi Forecasts diff --git a/docs/api/dengue_nowcast.md b/docs/api/dengue_nowcast.md index c7a90b700..7561a8429 100644 --- a/docs/api/dengue_nowcast.md +++ b/docs/api/dengue_nowcast.md @@ -1,6 +1,6 @@ --- title: Dengue Nowcast -parent: Epidata API (Other Epidemics) +parent: Epidata API (Other Diseases) --- # Delphi's Dengue Nowcast diff --git a/docs/api/dengue_sensors.md b/docs/api/dengue_sensors.md index bb1ff6fbc..cd218786c 100644 --- a/docs/api/dengue_sensors.md +++ b/docs/api/dengue_sensors.md @@ -1,6 +1,6 @@ --- title: Dengue Digital Surveillance -parent: Epidata API (Other Epidemics) +parent: Epidata API (Other Diseases) --- # Dengue Digital Surveillance Sensors diff --git a/docs/api/ecdc_ili.md b/docs/api/ecdc_ili.md index 58e9eab09..19539166c 100644 --- a/docs/api/ecdc_ili.md +++ b/docs/api/ecdc_ili.md @@ -1,6 +1,6 @@ --- title: ECDC ILI -parent: Epidata API (Other Epidemics) +parent: Epidata API (Other Diseases) --- # ECDC ILI diff --git a/docs/api/flusurv.md b/docs/api/flusurv.md index ca2309ee4..6ea97d32a 100644 --- a/docs/api/flusurv.md +++ b/docs/api/flusurv.md @@ -1,6 +1,6 @@ --- title: Flusurv -parent: Epidata API (Other Epidemics) +parent: Epidata API (Other Diseases) --- # FluSurv diff --git a/docs/api/fluview.md b/docs/api/fluview.md index 316863d75..930c9b4fd 100644 --- a/docs/api/fluview.md +++ b/docs/api/fluview.md @@ -1,6 +1,6 @@ --- title: FluView -parent: Epidata API (Other Epidemics) +parent: Epidata API (Other Diseases) --- # FluView diff --git a/docs/api/fluview_clinical.md b/docs/api/fluview_clinical.md index f861ed88d..7366225ab 100644 --- a/docs/api/fluview_clinical.md +++ b/docs/api/fluview_clinical.md @@ -1,6 +1,6 @@ --- title: FluView Clinical -parent: Epidata API (Other Epidemics) +parent: Epidata API (Other Diseases) --- # FluView Clinical diff --git a/docs/api/fluview_meta.md b/docs/api/fluview_meta.md index 3cfc567dd..e81be6e71 100644 --- a/docs/api/fluview_meta.md +++ b/docs/api/fluview_meta.md @@ -1,6 +1,6 @@ --- title: FluView metadata -parent: Epidata API (Other Epidemics) +parent: Epidata API (Other Diseases) --- # FluView metadata diff --git a/docs/api/gft.md b/docs/api/gft.md index 92653dd7b..69d825372 100644 --- a/docs/api/gft.md +++ b/docs/api/gft.md @@ -1,6 +1,6 @@ --- title: Google Flu Trends -parent: Epidata API (Other Epidemics) +parent: Epidata API (Other Diseases) --- # Google Flu Trends diff --git a/docs/api/ght.md b/docs/api/ght.md index 981db0455..3d3f6bb69 100644 --- a/docs/api/ght.md +++ b/docs/api/ght.md @@ -1,6 +1,6 @@ --- title: Google Health Trends -parent: Epidata API (Other Epidemics) +parent: Epidata API (Other Diseases) --- # Google Health Trends diff --git a/docs/api/kcdc_ili.md b/docs/api/kcdc_ili.md index edf60c7e1..6b611fa8b 100644 --- a/docs/api/kcdc_ili.md +++ b/docs/api/kcdc_ili.md @@ -1,6 +1,6 @@ --- title: KCDC ILI -parent: Epidata API (Other Epidemics) +parent: Epidata API (Other Diseases) --- # KCDC ILI diff --git a/docs/api/meta.md b/docs/api/meta.md index f1c7eb7e3..db5beabd9 100644 --- a/docs/api/meta.md +++ b/docs/api/meta.md @@ -1,6 +1,6 @@ --- title: Metadata -parent: Epidata API (Other Epidemics) +parent: Epidata API (Other Diseases) --- # API Metadata diff --git a/docs/api/meta_afhsb.md b/docs/api/meta_afhsb.md index 84572f39c..2791a0efe 100644 --- a/docs/api/meta_afhsb.md +++ b/docs/api/meta_afhsb.md @@ -1,6 +1,6 @@ --- title: AFHSB Metadata -parent: Epidata API (Other Epidemics) +parent: Epidata API (Other Diseases) --- # AFHSB Metadata diff --git a/docs/api/meta_norostat.md b/docs/api/meta_norostat.md index 7e17958bc..f8ba73094 100644 --- a/docs/api/meta_norostat.md +++ b/docs/api/meta_norostat.md @@ -1,6 +1,6 @@ --- title: NoroSTAT Metadata -parent: Epidata API (Other Epidemics) +parent: Epidata API (Other Diseases) --- # NoroSTAT Metadata diff --git a/docs/api/nidss_dengue.md b/docs/api/nidss_dengue.md index b0aa10155..c253097f0 100644 --- a/docs/api/nidss_dengue.md +++ b/docs/api/nidss_dengue.md @@ -1,6 +1,6 @@ --- title: NIDSS Dengue -parent: Epidata API (Other Epidemics) +parent: Epidata API (Other Diseases) --- # NIDSS Dengue diff --git a/docs/api/nidss_flu.md b/docs/api/nidss_flu.md index 5ba181977..8b19eabc8 100644 --- a/docs/api/nidss_flu.md +++ b/docs/api/nidss_flu.md @@ -1,6 +1,6 @@ --- title: NIDSS Flu -parent: Epidata API (Other Epidemics) +parent: Epidata API (Other Diseases) --- # NIDSS Flu diff --git a/docs/api/norostat.md b/docs/api/norostat.md index 7af7ca6e8..d4d88fd5b 100644 --- a/docs/api/norostat.md +++ b/docs/api/norostat.md @@ -1,6 +1,6 @@ --- title: NoroSTAT -parent: Epidata API (Other Epidemics) +parent: Epidata API (Other Diseases) --- # NoroSTAT diff --git a/docs/api/nowcast.md b/docs/api/nowcast.md index 34d2d47a1..85c832dd0 100644 --- a/docs/api/nowcast.md +++ b/docs/api/nowcast.md @@ -1,6 +1,6 @@ --- title: ILI Nearby Nowcast -parent: Epidata API (Other Epidemics) +parent: Epidata API (Other Diseases) --- # ILI Nearby Nowcast diff --git a/docs/api/paho_dengue.md b/docs/api/paho_dengue.md index 7ea71f4b9..99254715a 100644 --- a/docs/api/paho_dengue.md +++ b/docs/api/paho_dengue.md @@ -1,6 +1,6 @@ --- title: PAHO Dengue -parent: Epidata API (Other Epidemics) +parent: Epidata API (Other Diseases) --- # PAHO Dengue diff --git a/docs/api/quidel.md b/docs/api/quidel.md index 2830f183b..e90a59a4f 100644 --- a/docs/api/quidel.md +++ b/docs/api/quidel.md @@ -1,6 +1,6 @@ --- title: Quidel -parent: Epidata API (Other Epidemics) +parent: Epidata API (Other Diseases) --- # Quidel diff --git a/docs/api/sensors.md b/docs/api/sensors.md index a6a7e4f99..24ae001e4 100644 --- a/docs/api/sensors.md +++ b/docs/api/sensors.md @@ -1,6 +1,6 @@ --- title: Digital Surveillance Sensors -parent: Epidata API (Other Epidemics) +parent: Epidata API (Other Diseases) --- # Digital Surveillance Sensors diff --git a/docs/api/twitter.md b/docs/api/twitter.md index 95d00525a..0accbc660 100644 --- a/docs/api/twitter.md +++ b/docs/api/twitter.md @@ -1,6 +1,6 @@ --- title: Twitter Stream -parent: Epidata API (Other Epidemics) +parent: Epidata API (Other Diseases) --- # Twitter Stream diff --git a/docs/api/wiki.md b/docs/api/wiki.md index ac852b73c..558b0ee9d 100644 --- a/docs/api/wiki.md +++ b/docs/api/wiki.md @@ -1,6 +1,6 @@ --- title: Wikipedia Access -parent: Epidata API (Other Epidemics) +parent: Epidata API (Other Diseases) --- # Wikipedia Access