diff --git a/.github/workflows/release_main.yml b/.github/workflows/release_main.yml index 3adc85c3f..1e4e061b6 100644 --- a/.github/workflows/release_main.yml +++ b/.github/workflows/release_main.yml @@ -57,7 +57,7 @@ jobs: - name: Build Assets run: npm pack - name: Upload Release Asset - uses: AButler/upload-release-assets@v2.0 + uses: AButler/upload-release-assets@v3.0 with: files: "www-main-*.tgz" repo-token: ${{ secrets.GITHUB_TOKEN }} diff --git a/README.md b/README.md index e94923272..0bfae9ebb 100644 --- a/README.md +++ b/README.md @@ -83,6 +83,7 @@ As an alternative you can use Docker and Docker Compose: 1. `docker-compose up -d` to create a docker container for the current environment. 1. `docker-compose exec r bash` to jump into the container. 1. `micromamba activate www-main` to activate the environment +2. `export API_KEY=y0urAp1kEy` to add your own api key as env variable. Now you have the environment ready to start converting .Rmd blog files to html. #### Commands diff --git a/content/about/_index.md b/content/about/_index.md index fa260b7b5..61e72ac20 100644 --- a/content/about/_index.md +++ b/content/about/_index.md @@ -34,3 +34,79 @@ We're a research group based out of Carnegie Mellon University dedicated to deve ### Who is our audience? Public health authorities (federal, state, local), the healthcare industry, the public and private sectors, fellow researchers working on epidemic tracking and forecasting, data journalists, and the general public. + +### Milestones + +* **September 2023.** We were selected to be a Center of Innovation in Outbreak Analytics and Disease Modeling by the Centers for Disease Control and Prevention’s [Center for Forecasting and Outbreak Analytics](https://www.cdc.gov/forecast-outbreak-analytics/index.html). We are one of thirteen centers serving as the core of [Insight Net](https://www.cdc.gov/forecast-outbreak-analytics/partners/insightnet/index.html). + +* **March 2023.** With the sunsetting of regular COVID-19 Case reporting by both JHU CSSE and USAFacts, we are no longer focusing on case tracking and forecasting and are now prioritizing COVID and influenza-related hospitalizations, and on gearing up for tracking other circulating and emerging pathogens. + +* **February 2023.** We commenced a new project supporting the [CDC’s Center for Outbreak Forecasting and Analytics](https://www.cdc.gov/forecast-outbreak-analytics/index.html), which includes, among other goals, the prototyping of federated epidemic surveillance. + +* **September 2022.** We launched [Epidata v4](https://delphi.cmu.edu/blog/2022/12/14/introducing-epidata-v4/), prioritizing fast access to the most up-to-date data while retaining the deep data revision history needed by researchers. + +* **June 2022.** We ended data collection for the [COVID-19 Trends and Impact Survey (CTIS)](https://delphi.cmu.edu/covid19/ctis/) after more than two years, during which we received 29.5 million survey responses in the US (and [well over 100m globally](https://covidmap.umd.edu/)). + +* **April 2022.** We received the [Allen Newell Award for Research Excellence in SCS at CMU](https://www.cs.cmu.edu/events/newell-award), which is awarded annually and recognizes an outstanding body of work that epitomizes Allen Newell's research style. + +* **April 2022.** We received [the Policy Impact Award and the Warren J. Mitofsky Innovators Award from the the American Association of Public Opinion Research (AAPOR)](https://www.cs.cmu.edu/news/2022/delphi-aapor-awards), along with our partners from the University of Maryland Social Data Science Center and Meta, for our work on the [COVID-19 Trends and Impact Survey](https://delphi.cmu.edu/covid19/ctis/) (CTIS). + +* **January 2022.** We added selected signals from the White House COVID-19 Data Strategy and Execution Workgroup’s [Community Profile Report (CPR)](https://healthdata.gov/Health/COVID-19-Community-Profile-Report/gqxm-d9w9), including hospital admission and vaccination rates. This data source was discontinued in February 2023. + +* **January 2022.** After the CDC paused flu forecasting for the 2020-2021 season due to too little flu activity, we restarted generating flu forecasts for the 2021-2022 flu season. + +* **December 2021.** Our work was highlighted in a Proceedings of the National Academy of Sciences Special Feature, ["Beyond Cases and Deaths: The Benefits of Auxiliary Data Streams In Tracking the COVID-19 Pandemic"](https://www.pnas.org/topic/548) + +* **May 2021.** We received the 2021 [Statistical Partnerships Among Academe, Industry, and Government (SPAIG) award](https://www.cmu.edu/dietrich/news/news-stories/2021/may/spaig-covid.html) from the American Statistical Association (ASA) along with our COVIDcast collaborators. + +* **April 2021.** We launched our redesigned website, which included [COVIDcast 2.0](https://delphi.cmu.edu/covidcast/). + +* **March 2021.** We added PCR testing data from [COVID Act Now](https://covidactnow.org/). These data were discontinued in December 2021. + +* **January 2021.** We added signals on adult and pediatric COVID hospitalizations from the U.S. Department of Health & Human Services. In particular, we include the sum of all confirmed adult and pediatric COVID-19 hospital admissions. This sum is used as the "ground truth" for hospitalizations by the [COVID-19 Forecast Hub](https://covid19forecasthub.org/). + +* **December 2020.** We added national provisional death counts, from the National Center for Health Statistics (NCHS). [These data are](https://www.cdc.gov/nchs/nvss/vsrr/COVID19/index.htm)[ based on death certificate data received and coded by NCHS](https://www.cdc.gov/nchs/nvss/vsrr/COVID19/index.htm). + +* **December 2020.** We added inpatient and outpatient COVID-19 signals based on aggregated statistics from medical claims, provided to us by [Change Healthcare](https://www.changehealthcare.com/). + +* **November 2020.** Shifting from our original Google Health Trends data source, we added our Google Symptoms signals, which estimate the volume of searches mapped to symptom sets related to COVID-19. + +* **October 2020.** [Thirteen volunteer Googlers joined our group](https://www.cmu.edu/news/stories/archives/2020/september/covidcast-google.html) for six months via [Google.org Fellowships](https://www.google.org/our-approach/), bringing in professional experience and dramatically increasing our productivity. + +* **September 2020.** [The COVID-19 Symptom Data Challenge was launched](https://healthpolicy.duke.edu/events/covid-19-symptom-data-challenge). Sponsored by Delphi along with Meta, University of Maryland, the Duke Margolis Center for Health Policy, and Resolve to Save Lives, the Challenge asked participants to "enable earlier detection and improved situational awareness of the outbreak" using [CTIS data](https://delphi.cmu.edu/covidcast/survey-results/). + +* **August 2020.** We built upon our pre-pandemic relationship with Quidel and added signals based on positivity rates of their COVID-19 antigen tests. This data source was discontinued in August 2023. + +* **June 2020.** We added several new data sources, including: + + * Anonymized location data from mobile phones from [Safegraph](https://www.safegraph.com/). This source was discontinued in July 2022. + + * County-level confirmed COVID-19 case and death data from [USAFacts](https://usafacts.org/). This source was deactivated in January 2023. + + * Inpatient and outpatient COVID-19 hospitalization signals we derived from aggregated statistics from medical claims, provided to us by [Optum](https://www.optum.com/). + +* **May 2020.** We added signals for COVID-19 Cases and Deaths, mirrored from the [Center for Systems Science and Engineering](https://systems.jhu.edu/research/public-health/ncov/) at Johns Hopkins University. + +* **April 2020.** We partnered with Meta to launch the [COVID-19 Trends and Impact Survey (CTIS)](https://delphi.cmu.edu/covid19/ctis/) to monitor in real-time the spread and impact of the COVID-19 pandemic in the United States. + +* **April 2020.** We added signals from two new data sources: + + * Signals related to COVID-related doctor visits, derived from aggregated statistics from medical claims, provided to us by [Optum](https://www.optum.com/). + + * Data from Google Health Trends, which estimate the volume of COVID-related searches in a given location, on a given day. This data source was discontinued in March 2021. + +* **April 2020.** We began supporting and advising the U.S. CDC’s community-driven COVID-19 forecasting effort, including creating and maintaining an ensemble forecast from the models submitted to the [COVID-19 Forecast Hub](https://covid19forecasthub.org/), and a [forecast evaluation dashboard](https://delphi.cmu.edu/forecast-eval/). + +* **March 2020.** Working with Brett Slatkin (head of Google Surveys) and Hal Varian (Google’s Chief Economist), we launched our [Google Symptom Survey](https://delphi.cmu.edu/blog/2020/09/18/covid-19-symptom-surveys-through-google/), which ended in May 2020. + +* **March 2020.** We launched [COVIDcast](https://delphi.cmu.edu/covidcast/), the nation’s largest public repository of diverse, real-time indicators of COVID-19 activity, freely accessible through the [Epidata API](https://cmu-delphi.github.io/delphi-epidata/api/covidcast.html), which is updated daily with the latest data. + +* **2019.** We became a CDC National Center of Excellence for Influenza Forecasting, one of two nationally (and a 5-year designation). + +* **2016.** We developed and deployed [influenza nowcasts](https://delphi.cmu.edu/nowcast/) for the CDC, state departments of public health, and the public. + +* **2016.** We developed and deployed the [Epidata API](https://cmu-delphi.github.io/delphi-epidata/), which provides real-time access to epidemiological surveillance data. + +* **2013.** We began supporting the U.S. CDC’s Influenza Division in advancing and growing a [scientific community around influenza forecasting](https://www.cdc.gov/flu/weekly/flusight/index.html). We’ve been [perennial leaders in forecasting accuracy](https://www.cs.cmu.edu/~roni/CDC%20Flu%20Challenge%202014-2018%20Results.pdf) ever since. + + diff --git a/content/blog/2020-08-26-fb-survey.Rmd b/content/blog/2020-08-26-fb-survey.Rmd index 598ba02cc..0a49d9489 100644 --- a/content/blog/2020-08-26-fb-survey.Rmd +++ b/content/blog/2020-08-26-fb-survey.Rmd @@ -47,9 +47,6 @@ output: toc: true --- -```{r, echo=FALSE} -options(covidcast.auth = Sys.getenv("API_KEY")) -``` Since April 2020, in collaboration with Facebook, partner universities, and public health officials, @@ -140,6 +137,8 @@ library(covidcast) library(dplyr) library(gridExtra) +options(covidcast.auth = Sys.getenv("API_KEY")) # for more on API keys, see: https://cmu-delphi.github.io/delphi-epidata/api/api_keys.html + # Fetch Facebook % CLI signal and JHU confirmed case incidence proportion at # the state level start_day = "2020-06-15" diff --git a/content/blog/2020-08-26-fb-survey.html b/content/blog/2020-08-26-fb-survey.html index 0516c82b5..758fb7b4d 100644 --- a/content/blog/2020-08-26-fb-survey.html +++ b/content/blog/2020-08-26-fb-survey.html @@ -47,20 +47,19 @@ toc: true --- -
@@ -141,6 +140,8 @@

Short Background

library(dplyr) library(gridExtra) +options(covidcast.auth = Sys.getenv("API_KEY")) # for more on API keys, see: https://cmu-delphi.github.io/delphi-epidata/api/api_keys.html + # Fetch Facebook % CLI signal and JHU confirmed case incidence proportion at # the state level start_day = "2020-06-15" diff --git a/content/blog/2020-08-28-api.Rmd b/content/blog/2020-08-28-api.Rmd index 3828a5b36..c8675d28b 100644 --- a/content/blog/2020-08-28-api.Rmd +++ b/content/blog/2020-08-28-api.Rmd @@ -282,9 +282,10 @@ and ask "What was known _as of_ this date?" This is important because estimates can change for _weeks_ as new data arrives: -```{r q-versioning, warning=FALSE, message=FALSE, cache=TRUE} +```{r q-versioning, warning=FALSE, message=FALSE, cache=TRUE, eval=FALSE} library(covidcast) library(dplyr) +options(covidcast.auth = Sys.getenv("API_KEY")) # for more on API keys, see: https://cmu-delphi.github.io/delphi-epidata/api/api_keys.html query_date <- "2020-08-01" covidcast_signal( data_source = "quidel", @@ -300,6 +301,7 @@ covidcast_signal( col.names = c("Test date", "Positivity rate (%)", "Sample size", "Issued on", "Lag (days)")) ``` +*November 2023 update: Quidel data is no longer publicly available, so the table generated by the code chunk above has been removed.* Many data sources are subject to revisions: @@ -359,6 +361,7 @@ that are due to COVID-19 in several states. ```{r dv-graph, message=FALSE, cache=TRUE} library(covidcast) +options(covidcast.auth = Sys.getenv("API_KEY")) # for more on API keys, see: https://cmu-delphi.github.io/delphi-epidata/api/api_keys.html hosp <- covidcast_signal( data_source = "hospital-admissions", signal = "smoothed_adj_covid19_from_claims", start_day = "2020-03-01", end_day = "2020-08-30", @@ -398,14 +401,17 @@ this is the `fb-survey` data source's `smoothed_hh_cmnty_cli` signal. (Click the "Code" button to see the Python code used to produce this example.) ```{python python-data, dev='svg'} +import matplotlib.pyplot as plt import covidcast from datetime import date -import matplotlib.pyplot as plt +import os + +covidcast.use_api_key(os.environ['API_KEY']) # for more on API keys, see: https://cmu-delphi.github.io/delphi-epidata/api/api_keys.html data = covidcast.signal("fb-survey", "smoothed_hh_cmnty_cli", date(2020, 9, 8), date(2020, 9, 8), geo_type="state") -covidcast.plot_choropleth(data, figsize=(7, 5)) +covidcast.plot(data, plot_type="choropleth", figsize=(7, 5)) plt.title("% who know someone who is sick, Sept 8, 2020") ``` diff --git a/content/blog/2020-08-28-api.html b/content/blog/2020-08-28-api.html index 5a16146ff..8866b34e8 100644 --- a/content/blog/2020-08-28-api.html +++ b/content/blog/2020-08-28-api.html @@ -51,15 +51,14 @@ toc: true --- -
@@ -280,6 +279,7 @@

Tracking Observations and Revisions

can change for weeks as new data arrives:

library(covidcast)
 library(dplyr)
+options(covidcast.auth = Sys.getenv("API_KEY")) # for more on API keys, see: https://cmu-delphi.github.io/delphi-epidata/api/api_keys.html
 query_date <- "2020-08-01"
 covidcast_signal(
   data_source = "quidel",
@@ -294,182 +294,7 @@ 

Tracking Observations and Revisions

knitr::kable("html", digits = 2, col.names = c("Test date", "Positivity rate (%)", "Sample size", "Issued on", "Lag (days)"))
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
-Test date - -Positivity rate (%) - -Sample size - -Issued on - -Lag (days) -
-2020-08-01 - -1.01 - -198 - -2020-08-06 - -5 -
-2020-08-01 - -0.97 - -206 - -2020-08-07 - -6 -
-2020-08-01 - -1.41 - -284 - -2020-08-10 - -9 -
-2020-08-01 - -1.38 - -290 - -2020-08-12 - -11 -
-2020-08-01 - -1.33 - -377 - -2020-08-16 - -15 -
-2020-08-01 - -1.53 - -459 - -2020-08-19 - -18 -
-2020-08-01 - -1.47 - -477 - -2020-08-20 - -19 -
-2020-08-01 - -1.46 - -479 - -2020-08-26 - -25 -
-2020-08-01 - -1.36 - -513 - -2020-08-30 - -29 -
+

November 2023 update: Quidel data is no longer publicly available, so the table generated by the code chunk above has been removed.

Many data sources are subject to revisions: