Decide if we should switch to the weighted Facebook signals on the public maps

[capnrefsmmat]

These are in principle bias-corrected, so they are "better"; however, they take an additional day to compute, since we have to wait for weights to come back from Facebook. We should decide what we think of this tradeoff and then decide if we want to change over to the weighted estimates.

[krivard]

This should wait until we're ready to switch over to the new codebase, since that's the only one that does weighted community metrics.

(specifically waiting on #59 and #150, possibly others)

[ryantibs]

@capnrefsmmat @krivard I was looking at the Facebook data just now (through our new and improved R package!) and was reminded of this. I'm guessing we're still stalled on this, because we're waiting on the new codebase (the other issues Katie flagged)?

That said, I was looking at the meta data and I noticed that there's some signal names I don't recognize.

> meta = covidcast_meta()
> meta %>% filter(data_source=="fb-survey") %>% select(signal) %>% unique
                    signal
1                  raw_cli
5            raw_community
9         raw_hh_cmnty_cli
13                 raw_ili
17      raw_nohh_cmnty_cli
21                raw_wcli
25                raw_wili
29            smoothed_cli
33      smoothed_community
37   smoothed_hh_cmnty_cli
41            smoothed_ili
45 smoothed_nohh_cmnty_cli
49           smoothed_wcli
53           smoothed_wili
57    unsmoothed_community

What are the *_community signals? (Where * = raw, smoothed, unsmoothed). Are these defunct?

While I'm on this topic (sorry to go on a bit of a tangent here, but it's related), look at the following.

> meta %>% filter(data_source=="jhu-csse") %>% select(signal) %>% unique
                             signal
1     confirmed_7dav_cumulative_num
5    confirmed_7dav_cumulative_prop
9      confirmed_7dav_incidence_num
13    confirmed_7dav_incidence_prop
17 confirmed_7day_avg_cumulative_nu
21 confirmed_7day_avg_cumulative_pr
25 confirmed_7day_avg_incidence_num
29 confirmed_7day_avg_incidence_pro
33         confirmed_cumulative_num
37        confirmed_cumulative_prop
41          confirmed_incidence_num
45         confirmed_incidence_prop
49       deaths_7dav_cumulative_num
53      deaths_7dav_cumulative_prop
57        deaths_7dav_incidence_num
61       deaths_7dav_incidence_prop
65   deaths_7day_avg_cumulative_num
69  deaths_7day_avg_cumulative_prop
73    deaths_7day_avg_incidence_num
77   deaths_7day_avg_incidence_prop
81            deaths_cumulative_num
85           deaths_cumulative_prop
89             deaths_incidence_num
93            deaths_incidence_prop

Seems like there must be stuff here that's defunct: we have both 7dav and 7day_avg, for example.

Is this generally known/intended behavior with the API signals, or is it just that we haven't "cleaned up" some signals that we should have?

[krivard]

This is actually from adding data versioning; those signal names were deleted. We’re working on a way to mark deletions in a way that preserves all data history. So far I have permanently removed only the records from the doctor-visits source that include private data, and left everything else in. It does indeed clutter the meta signal, but since meta correctly indicates the stop date for each of them, it seemed an okay compromise.

…

On Mon, Jul 27, 2020 at 11:45 PM ryantibs ***@***.***> wrote: @capnrefsmmat <https://github.com/capnrefsmmat> @krivard <https://github.com/krivard> I was looking at the Facebook data just now (through our new and improved R package!) and was reminded of this. I'm guessing we're still stalled on this, because we're waiting on the new codebase (the other issues Katie flagged)? That said, I was looking at the meta data and I noticed that there's some signal names I don't recognize. > meta = covidcast_meta() > meta %>% filter(data_source=="fb-survey") %>% select(signal) %>% unique signal 1 raw_cli 5 raw_community 9 raw_hh_cmnty_cli 13 raw_ili 17 raw_nohh_cmnty_cli 21 raw_wcli 25 raw_wili 29 smoothed_cli 33 smoothed_community 37 smoothed_hh_cmnty_cli 41 smoothed_ili 45 smoothed_nohh_cmnty_cli 49 smoothed_wcli 53 smoothed_wili 57 unsmoothed_community What are the *_community signals? (Where * = raw, smoothed, unsmoothed). Are these defunct? While I'm on this topic (sorry to go on a bit of a tangent here, but it's related), look at the following. > meta %>% filter(data_source=="jhu-csse") %>% select(signal) %>% unique signal 1 confirmed_7dav_cumulative_num 5 confirmed_7dav_cumulative_prop 9 confirmed_7dav_incidence_num 13 confirmed_7dav_incidence_prop 17 confirmed_7day_avg_cumulative_nu 21 confirmed_7day_avg_cumulative_pr 25 confirmed_7day_avg_incidence_num 29 confirmed_7day_avg_incidence_pro 33 confirmed_cumulative_num 37 confirmed_cumulative_prop 41 confirmed_incidence_num 45 confirmed_incidence_prop 49 deaths_7dav_cumulative_num 53 deaths_7dav_cumulative_prop 57 deaths_7dav_incidence_num 61 deaths_7dav_incidence_prop 65 deaths_7day_avg_cumulative_num 69 deaths_7day_avg_cumulative_prop 73 deaths_7day_avg_incidence_num 77 deaths_7day_avg_incidence_prop 81 deaths_cumulative_num 85 deaths_cumulative_prop 89 deaths_incidence_num 93 deaths_incidence_prop Seems like there must be stuff here that's defunct: we have both *7dav* and *7day_avg*, for example. Is this generally known/intended behavior with the API signals, or is it just that we haven't "cleaned up" some signals that we should have? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#139 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAI24CSQMKHHY6Z3XKHSADLR5ZCUVANCNFSM4OVYDHIA> .

[ryantibs]

I see. In principle I like the idea of data versioning, but I'm not sure I like the idea of having "everything ever created" appear in the metadata.

For example, if I created a new signal today, and deleted it tomorrow, would it appear in the meta data from every day going forward? I think so, according to your rule. And would that be a good thing?

So, two questions:

1. Do we want to pass through and clean up truly "transient/temporary" signals like the ones I pointed out? I would say yes. There's a big difference between these signals, and the Google survey signals or the Quidel signals, for example. We stopped supporting both, but there's no real reason for somebody to have to look through many-different-named versions 7 day averaged signals. Whereas there's a reason somebody might want to look back at the Google survey and Quidel signals.

2. Do we want to create two versions of the metadata function? The first one, which would be the standard one (which already exists) returns, say, only the currently supported signals. A second one returns, say, a "complete" list of all our signals, supported and non-supported.