The study, conducted by a large team of researchers, including biocomplexity scientist Matteo Convertino of Japan’s Hokkaido University, analyzed the forecasts of 14 predictive models submitted by 11 teams to the US-based Centers for Disease Control and Prevention (CDC) as part of its 2015-2016 influenza season forecasting challenge.
The CDC launched the annual challenge in 2013, encouraging academics and private industry researchers to forecast the timing, peak, and intensity of the flu season in the US. Previous efforts were directed toward forecasting Dengue fever. The general aim of the challenge is to improve influenza forecasting in order to better inform public health responses to seasonal epidemics and future pandemics.
Results from analyses of the submissions of the 2015-16 season show that forecasting skill, measured using a logarithmic score, was generally highest among the teams and their models for seasonal peak intensity and short-term forecasts, but was generally low for timing of season onset and peak week.
Forecasting skill was higher among teams that had participated in the challenge before, and also among teams that combined more than one model to develop their forecasts. When the researchers combined all team forecasts into a single ensemble model, they found it performed better compared to the results of each individual model.
“The results highlight the continuing challenge of improving forecast accuracy for more seasons and at lead times of several weeks or more; forecasts that would be of even more utility for public health officials,” the researchers write. The results show that the CDC forecasting challenge provides unique insight into epidemic forecasting, they say.
The forecasting models used by the teams did provide valuable data, but future forecasts could be further improved as teams gain more experience and by using combined model approaches, they conclude.
“As the only ongoing infectious disease forecasting challenge in the United States, the CDC influenza forecasting challenge sets a model for other infectious diseases by identifying data and resource constraints that limit model development, establishing best practices for forecast submission and evaluation, identifying areas where forecasts can be improved, tying forecasting efforts to real public health needs, and assessing their performance related to those needs,” say the researchers.
