Methodology

Published

February 13, 2025

Introduction

Here we provide details on our methodology for calibrating and running our forecast, as well as a description of the technologies powering the epiworld-forecasts tool.

Forecast Methodology

We use case count data published weekly by the Utah DHHS. The calibration and SIR connected model are run using the epiworldR package.

COVID-19 Cases in Utah (last 90 days)

Every week, Utah DHHS publishes COVID-19 surveillance data on their Coronavirus Dashboard which includes reported case counts for each day starting March 18, 2020. Our forecast is calibrated on case counts from the last 90 days.

Calibrating the Forecasting Model

We use epiworldR’s implementation of Likelihood-Free Markhov Chain Monte Carlo (LFMCMC) to calibrate the SIR connected model (using ModelSIRCONN from epiworldR) by estimating the following model parameters:

  • Recovery rate
  • Transmission rates for each season (spring, summer, fall, winter)
  • Contact rates for weekdays and weekends

The LFMCMC simulation runs for 6,000 iterations. In each iteration, it does the following:

  • Proposes a new set of parameter values
  • Runs the model with 10,000 agents and the proposed parameters
  • Compares the output to the UDHHS data and records the result

We note that while the population of Utah over 3 million, we only need to run our model with 10,000 agents. This is because we don’t expect more than 10,000 COVID-19 cases and the model automatically scales the contact rate to account for the difference between the model population and Utah’s population.

When the simulation is finished, we use a burn-in period of n = 2,000 (33% of simulation iterations). The epiworldR results printout (below) shows the mean parameter/statistic value, the 95% credible interval (in [ ]), and the initial/observed value (in ( )).

___________________________________________

LIKELIHOOD-FREE MARKOV CHAIN MONTE CARLO

N Samples (total) : 6000
N Samples (after burn-in period) : 4000
Elapsed t : 5.00m

Parameters:
  -Recovery rate              :  0.20 [ 0.15,  0.32] (initial :  0.14)
  -Transmission rate (spring) :  0.02 [ 0.01,  0.04] (initial :  0.05)
  -Transmission rate (summer) :  0.13 [ 0.04,  0.32] (initial :  0.04)
  -Transmission rate (fall)   :  0.03 [ 0.02,  0.04] (initial :  0.06)
  -Transmission rate (winter) :  0.04 [ 0.03,  0.06] (initial :  0.07)
  -Contact rate (weekday)     :  9.52 [ 9.19,  9.86] (initial :  10.00)
  -Contact rate (weekend)     :  2.21 [ 1.34,  3.07] (initial :  2.00)

Statistics:
  -Time to peak               :   51.87 [  21.00,   71.00] (Observed:   64.00)
  -Size of peak               :  177.15 [ 124.00,  261.00] (Observed:  173.00)
  -Mean (cases)               :   59.08 [  44.49,   73.76] (Observed:   65.36)
  -Standard deviation (cases) :   50.27 [  34.20,   71.93] (Observed:   27.73)
___________________________________________

Here is the posterior distribution of the LFMCMC samples with vertical lines representing the initial parameter values.

COVID-19 Forecast

We can now run the forecast. Our model prevalence is set according to the reported case counts of the most recent day of the UDHHS data. We then take a sample of n = 200 from the LFMCMC accepted parameters (after the burn-in period) and run the SIR connected model with the new prevalence for each set of parameters. Each simulation is for two weeks, giving us a 14-day forecast of COVID-19 in Utah. The forecast mean is shown below along with the 50% and 95% confidence intervals. The actual case counts are plotted in black, while the forecast is plotted in blue.

Technologies

epiworld-forecasts is built with the following technologies:

  • epiworldR: Fast agent-based modeling R package for disease simulations
  • Docker: The forecast runs inside a Docker container which has all the needed packages. This container is built in a separate workflow and pushed to the GitHub Container Registry.
  • GitHub Actions: The forecast runs on a schedule through GitHub Actions
  • Quarto: Generates the HTML report that is published to GitHub Pages
  • GitHub: Version control, hosting the source code repository and website