[jhu-csse] Figure out how to report cases and deaths not assigned to a county

[capnrefsmmat]

Jacob noticed that Providence, RI has no deaths in the JHU data on our map, despite having a large population and many cases. Alden determined that in the source file, all deaths in Rhode Island are reported in the county "Unassigned", rather than in a specific county. We don't report these deaths in any county or even in the state-level map.

There seem to be numerous other states that also have an "Unassigned" county containing some deaths.

How should we report these deaths? Distribute them among the counties in the state by population? I didn't find documentation from JHU saying why some deaths are unassigned.

This will require documentation in our API docs, and potentially in the release notes.

cc @statsmaths @huisaddison

[capnrefsmmat]

Jacob has noted there are other differences between JHU and USAFacts, which is apparently what CDC uses. Here are his comparisons: https://delphi-org.slack.com/archives/CV1SYBC90/p1590952989173200

Not all of these differences can be accounted for by the "Unassigned" cases.

[jacobbien]

It looks like Alex knows this already from his post above, but I just want to emphasize something that only became apparent to us today (thanks to a plot from @ajgreen93) that this also affects state level reporting of JHU deaths. For example, the state of RI has zero deaths reported on our map. I haven't looked into this, but I'd expect this issue may also explain why Maine, New Hampshire, and Vermont have such low counts.

[krivard]

We will start reporting unassigned cases and deaths in a megacounty for the state, and include these counts in the state aggregations.

[jingjtang]

It's not clear to me about how to calculate the prop based on population for unassigned cases/death. So the megacounties are added only for sensors:_num for county/hrr/msa levels. As for states, there is no such a problem. The code is ready for review here: cmu-delphi/covidcast-indicators#68 @capnrefsmmat @krivard

[krivard]

That's tricky; for RI we would want prop to be over the state population, but indeed it's not clear what the denominator should be for the other states where "Unassigned" appears. @jacobbien which signals are you using, num or prop?

[jacobbien]

For JHU death, we are using num. For JHU cases, we are using both num and prop. I agree that it's not clear how prop could be calculated for "unassigned". From our standpoint I guess we'd just have to deal with an NA there. Thanks!

…

On Mon, Jun 8, 2020 at 12:55 PM krivard ***@***.***> wrote: That's tricky; for RI we would want prop to be over the state population, but indeed it's not clear what the denominator should be for the other states where "Unassigned" appears. @jacobbien <https://urldefense.com/v3/__https://github.com/jacobbien__;!!LIr3w8kk_Xxm!4-rH6zngYUPApmFPmkr-2ahLevJritcab8FoAnxhjf82u20Y8VFw-FYI69K3$> which signals are you using, num or prop? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://urldefense.com/v3/__https://github.com/cmu-delphi/covidcast-indicators/issues/58*issuecomment-640853071__;Iw!!LIr3w8kk_Xxm!4-rH6zngYUPApmFPmkr-2ahLevJritcab8FoAnxhjf82u20Y8VFw-K9iTuH1$>, or unsubscribe <https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AAPKMIEQCO7F6XWQH5BEM3DRVU64RANCNFSM4NOVYY2Q__;!!LIr3w8kk_Xxm!4-rH6zngYUPApmFPmkr-2ahLevJritcab8FoAnxhjf82u20Y8VFw-OturR-k$> .

[krivard]

For viz: "Unassigned" is ambiguous. For RI, all real counties always report zero, so we should just treat the whole state as a big megacounty and report the Unassigned figure.

For everywhere else, we don't know what counties "Unassigned" counts were drawn from. We could consider not reporting all zero-count counties, so that they would all be shown as part of the Unassigned/megacounty, but it's not clear that's the right call.

[krivard]

This is almost done in #68, waiting on guidance for a solution for mega MSAs and HRRs

[krivard]

Prioritizing #69 for now.