Skip to content

Files

Latest commit

minhkhulminhkhul
JHU deactivation: geomap
Apr 12, 2023
a2f809e · Apr 12, 2023

History

History
57 lines (37 loc) · 5.74 KB

README.md

File metadata and controls

57 lines (37 loc) · 5.74 KB

Geocoding data processing pipeline

Authors: Jingjing Tang, James Sharpnack, Dmitry Shemetov

Usage

Requires the following source files below.

Run the following to build the crosswalk tables in covidcast-indicators/_delph_utils_python/delph_utils/data

$ python geo_data_proc.py

You can see consistency checks and diffs with old sources in ./consistency_checks.ipynb

Geo Codes

We support the following geocodes.

  • The ZIP code and the FIPS code are the most granular geocodes we support.
    • The ZIP code is a US postal code used by the USPS and the FIPS code is an identifier for US counties and other associated territories. The ZIP code is five digit code (with leading zeros).
    • The FIPS code is a five digit code (with leading zeros), where the first two digits are a two-digit state code and the last three are a three-digit county code (see this US Census Bureau page for detailed information).
  • The Metropolitan Statistical Area (MSA) code refers to regions around cities (these are sometimes referred to as CBSA codes). More information on these can be found at the US Census Bureau.
    • We are reserving 10001-10099 for states codes of the form 100XX where XX is the FIPS code for the state (the current smallest CBSA is 10100). In the case that the CBSA codes change then it should be verified that these are not used.
  • State codes are a series of equivalent identifiers for US state. They include the state name, the state number (state_id), and the state two-letter abbreviation (state_code). The state number is the state FIPS code. See here for more.
  • The Hospital Referral Region (HRR) and the Hospital Service Area (HSA). More information here. FIPS codes depart in some special cases, so we produce manual changes listed below.

Source files

The source files are requested from a government URL when geo_data_proc.py is run (see the top of said script for the URLs). Below we describe the locations to find updated versions of the source files, if they are ever needed.

  • ZIP -> FIPS (county) population tables available from US Census. This file contains the population of the intersections between ZIP and FIPS regions, allowing the creation of a population-weighted transform between the two. As of 4 February 2022, this source did not include population information for 24 ZIPs that appear in our indicators. We have added those values manually using information available from the zipdatamaps website.
  • ZIP -> HRR -> HSA crosswalk file comes from the 2018 version at the Dartmouth Atlas Project.
  • FIPS -> MSA crosswalk file comes from the September 2018 version of the delineation files at the US Census Bureau.
  • State Code -> State ID -> State Name comes from the ANSI standard at the US Census. The first two digits of a FIPS codes should match the state code here.

Derived files

The rest of the crosswalk tables are derived from the mappings above. We provide crosswalk functions from granular to coarser codes, but not the other way around. This is because there is no information gained when crosswalking from coarse to granular.

Deprecated source files

  • ZIP to FIPS to HRR to states: 02_20_uszips.csv comes from a version of the table here modified by Jingjing to include population weights.
    • The 02_20_uszips.csv file is based on the newest consensus data including 5-digit zipcode, fips code, county name, state, population, HRR, HSA (I downloaded the original file from here. This file matches best to the most recent (2020) situation in terms of the population. But there still exist some matching problems. I manually checked and corrected those lines (~20) with zip-codes. The mapping from 5-digit zipcode to HRR is based on the file in 2017 version downloaded from here.
  • ZIP -> FIPS is provided by huduser.gov for zip -> fips?
  • FIPS county population data from US Census Bureau. Details of Bedford, Virginia counting here.
  • CBSA -> FIPS crosswalk from here (the file is cbsatocountycrosswalk.csv).
  • MSA tables from March 2020 here. This file seems to differ in a few fips codes from the source for the 02_20_uszip file which Jingjing constructed. There are at least 10 additional fips in 03_20_msa that are not in the uszip file, and one of the msa codes seems to be incorrect: 49020 (a google search confirms that it is incorrect in uszip and correct in the census data).
  • MSA tables from 2019 here

Notes

  • The NAs in the coding currently zero-fills.