NSSP create combined signals based on sum of flu + covid + rsv

[minhkhul]

From Nov 9th 2024, combined signals are no longer updated on nssp. We can technically generate these signals ourselves still by (need to check if it really is that simple) summing the result of the three other signals.

[aysim319]

using national data to compare between the combined values generated from source and doing a raw sum of the other three columns, shows that they are not a raw sum, but have some unknown denomination/calculations that goes through the source side.

[melange396]

I wanted to see if this was an artifact in the national-level data... It seems to appear at all levels, when comparable, in a relatively large (>20%) number of records, though many records do agree (~40%). The vast majority of the differences are between ~0 and +~1/3 (though they range from -1.67 to 8.33 across all data), which corresponds to the majority of percent differences in ~0% - 4% (ranging from -15% - 80% in total) (note the latter numbers are percent difference between the calculated and given "combined" disease sums, not to be confused with the values themselves, which represent percentages of visits). I figured it was possible that some of these discrepancies could be explained by patients who present with more than one pathogen, but it seems unlikely that information is captured all that often, and it does not explain all of the differences (some are negative).

My analysis code and its output:

output:

count breakdown:
393856 : total number of data points
148691 : no 'percent_visits_combined' value in source data (nothing to compare to)
94805 : calculated and given values for combined visits agree
45220 : some component values unspecified, calculation would be incomplete
48019 : discrepancy explainable by rounding errors
57121 : problematic difference between calculated and given values for combined visits

histogram of problematic differences:
(array([    6,     0,    15,   294, 50388,  5279,   816,   174,    45,
          41,    26,    18,     4,     2,     0,    11,     0,     0,
           0,     0,     0,     0,     0,     0,     2]), array([-1.67, -1.27, -0.87, -0.47, -0.07,  0.33,  0.73,  1.13,  1.53,
        1.93,  2.33,  2.73,  3.13,  3.53,  3.93,  4.33,  4.73,  5.13,
        5.53,  5.93,  6.33,  6.73,  7.13,  7.53,  7.93,  8.33]))

histogram of percent differences (of calculated combined value) where problematic differences occur:
(array([   11,    39,   156,   422, 47097,  7130,  1402,   411,   195,
         113,    49,    43,    17,     7,     3,    15,     3,     0,
           7,     0,     0,     0,     0,     0,     1]), array([-15.02347418, -11.22253521,  -7.42159624,  -3.62065728,
         0.18028169,   3.98122066,   7.78215962,  11.58309859,
        15.38403756,  19.18497653,  22.98591549,  26.78685446,
        30.58779343,  34.38873239,  38.18967136,  41.99061033,
        45.7915493 ,  49.59248826,  53.39342723,  57.1943662 ,
        60.99530516,  64.79624413,  68.5971831 ,  72.39812207,
        76.19906103,  80.        ]))

source:

import requests
import numpy

r = requests.get("https://data.cdc.gov/resource/rdmq-nq56.json?$limit=400000")
data = r.json()

no_combined = 0
agree = 0
incomplete = 0
rounding = 0
problematic = 0

diff_h = []
pct_diff_h = []

for point in data:
    if 'percent_visits_combined' in point:
        combd = float(point['percent_visits_combined'])
        cov   = float(point.get('percent_visits_covid', 0))
        flu   = float(point.get('percent_visits_influenza', 0))
        rsv   = float(point.get('percent_visits_rsv', 0))
        calcd = cov + flu + rsv
        diff  = calcd - combd
        if diff == 0:
            ##print("given and calculated values agree!")
            agree += 1
        elif any(v==0 for v in (combd, cov, flu, rsv)):
            ##print("incomplete records?")
            incomplete += 1
        elif abs(diff) < 0.030001:
            ##print("rounding error?")
            rounding += 1
        else:
            pct_diff = diff / calcd * 100
            ##print(" ".join([point['week_end'][:10]]+[point[f] for f in "fips geography county".split()]), cov, flu, rsv, combd, calcd, diff, pct_diff)
            diff_h.append(diff)
            pct_diff_h.append(pct_diff)
            problematic += 1
    else:
        no_combined += 1


print("count breakdown:")
print(len(data), ": total number of data points")
print(no_combined, ": no 'percent_visits_combined' value in source data (nothing to compare to)")
print(agree, ": calculated and given values for combined visits agree")
print(incomplete, ": some component values unspecified, calculation would be incomplete")
print(rounding, ": discrepancy explainable by rounding errors")
print(problematic, ": problematic difference between calculated and given values for combined visits")
print()
print("histogram of problematic differences:")
print(numpy.histogram(diff_h, bins=25))
print()
print("histogram of percent differences (of calculated combined value) where problematic differences occur:")
print(numpy.histogram(pct_diff_h, bins=25))
print()

[melange396]

This is currently blocked; We are waiting for Roni to get an explanation from his NSSP contact.

[RoniRos]

NSSP communication continues to be problematic. Let's just note in our documentation that the combined indicator, when available, does not match the sum of the individual indicators. And let's not create our own combined version.