Skip to content

Refactor safegraph to use geo utils #313

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Oct 30, 2020
Merged

Refactor safegraph to use geo utils #313

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
ddb331d
Refactor to use GeoMapper for fips to state
chinandrew Oct 15, 2020
0e1bd78
Change join behavior to match original
chinandrew Oct 15, 2020
ca7ccd0
Remove newline
chinandrew Oct 15, 2020
a30dd99
Merge branch 'main' into geo_refactor_safegraph
chinandrew Oct 20, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion safegraph/delphi_safegraph/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,4 @@

from __future__ import absolute_import

from . import geo
from . import process
66 changes: 0 additions & 66 deletions safegraph/delphi_safegraph/geo.py
View file
Open in desktop

This file was deleted.

16 changes: 11 additions & 5 deletions safegraph/delphi_safegraph/process.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@
import pandas as pd
import covidcast

from .constants import HOME_DWELL, COMPLETELY_HOME, FULL_TIME_WORK, PART_TIME_WORK
from .geo import FIPS_TO_STATE, VALID_GEO_RESOLUTIONS
from delphi_utils import GeoMapper

from .constants import HOME_DWELL, COMPLETELY_HOME, FULL_TIME_WORK, PART_TIME_WORK, GEO_RESOLUTIONS

# Magic number for modular arithmetic; CBG -> FIPS
MOD = 10000000
Expand Down Expand Up @@ -176,11 +177,16 @@ def aggregate(df, signal_names, geo_resolution='county'):
if geo_resolution == 'county':
df['geo_id'] = df['county_fips']
elif geo_resolution == 'state':
df['geo_id'] = df['county_fips'].apply(lambda x:
FIPS_TO_STATE[x[:2]])
gmpr = GeoMapper()
df = gmpr.add_geocode(df,
from_col='county_fips',
from_code='fips',
new_code='state_id',
new_col='geo_id',
dropna=False)
else:
raise ValueError(
f'`geo_resolution` must be one of {VALID_GEO_RESOLUTIONS}.')
f'`geo_resolution` must be one of {GEO_RESOLUTIONS}.')

# Aggregation and signal creation
grouped_df = df.groupby(['geo_id'])[signal_names]
Expand Down