Announcement of Requirements and Registration for the Predict the Influenza Season Challenge, 70303-70305 [2013-28198]

Agencies

• Centers for Disease Control and Prevention
• DEPARTMENT OF HEALTH AND HUMAN SERVICES

[Federal Register Volume 78, Number 227 (Monday, November 25, 2013)]
[Notices]
[Pages 70303-70305]
From the Federal Register Online via the Government Printing Office [www.gpo.gov]
[FR Doc No: 2013-28198]



[[Page 70303]]

-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Centers for Disease Control and Prevention


Announcement of Requirements and Registration for the Predict the 
Influenza Season Challenge

AGENCY: Centers for Disease Control and Prevention (CDC), Department of 
Health and Human Services (HHS).
    Award Approving Official: Thomas R. Frieden, MD, MPH, Director, 
Centers for Disease Control and Prevention, and Administrator, Agency 
for Toxic Substances and Disease Registry.

ACTION: Notice.

-----------------------------------------------------------------------

SUMMARY: The Centers for Disease Control and Prevention (CDC) located 
within the Department of Health and Human Services (HHS) announces the 
launch of the Predict the Influenza Season Challenge.
    Each year annual epidemics of influenza occur in the United States. 
However, variations in the timing and intensity of the season occur 
each year. Early insights into the timing of the beginning, the peak, 
and the intensity of the influenza season would be very useful in 
planning vaccination campaigns, targeting resources and therefore 
reducing costs associated with influenza prevention and control, and 
communicating prevention messages to the public. Mathematical and 
statistical models can be useful in predicting the timing and impact of 
the influenza season, but no models published to date have successfully 
predicted key influenza season milestones with sufficient accuracy.
    The Influenza Division, National Center for Immunization and 
Respiratory Diseases, CDC would like to invest in innovation through 
the research and development of mathematical and statistical models 
that use digital surveillance data (e.g. Twitter, internet search data, 
web surveys, etc.) to predict the timing, peak, and intensity of the 
upcoming influenza season. By hosting this challenge, the Influenza 
Division is able to garner the technical innovation required to 
accurately forecast the influenza season for less resources than would 
be required through more traditional mechanisms. Furthermore, this 
challenge will allow the Influenza Division to foster competition and 
receive and evaluate multiple influenza season forecasts from a number 
of scientists using a variety of different methodologies, which would 
not be possible if a single entity were funded. Accurate influenza 
forecasts have the potential to reduce long-term costs by more 
efficiently utilizing resources that are available to track influenza 
and implement control measures during the influenza season.
    The challenge could be solved by applying any mathematical, 
statistical, or other approach to predictive modeling. This challenge 
will provide the Influenza Division with methods that advance the 
science of prediction modeling, enhance the understanding of influenza 
modeling and the use of digital data for influenza surveillance, and 
improve the implementation of prevention and control measures for 
seasonal influenza.
    The historical national surveillance data that could be used to 
enable training and correlation model development by competitors are 
available at https://gis.cdc.gov/grasp/fluview/fluportaldashboard.html 
and are updated every Friday at noon. The competitors' predictions and 
methodology describing their models will comprise their initial 
challenge entry. Eight subsequent bi-weekly submissions of predictions 
based on the submitted methodology are also required. Predictions must 
be national in scope but may also include HHS region predictions; all 
predictions must be scaled to ILINet. Competitors' models will be 
evaluated based on the methodology and how well the predictions match 
the 2013-14 influenza season as measured by the U.S. Outpatient 
Influenza-like Illness Surveillance Network (ILINet: https://www.cdc.gov/flu/weekly/overview.htm#Outpatient).
    The competition will award a $75,000 prize and singular recognition 
to the person or team that most closely predicts the influenza season.

DATES: Registration opens on November 23 and closes December 2, 2013. 
Register by sending email to flucontest@cdc.gov and include your first 
and last name, your address, and your phone number. Please see 
``Registration Process for Participants'' below for additional details.
    The Competition Submission Period is from December 1, 2013-March 
27, 2014. The first submission of the prediction must be received by 
December 2, 2013. Subsequent submissions will be required biweekly 
until the close of the submission period and must be received by 11:59 
p.m. Eastern on December 19, 2013; January 2, 2014; January 16, 2014; 
January 30, 2014; February 13, 2014; February 27, 2014; March 13, 2014; 
and March 27, 2014.
    Judging will take place between March 28, 2014 and May 30, 2014. 
Winners will be announced by June 20, 2014.

FOR FURTHER INFORMATION CONTACT: Matthew Biggerstaff, MPH, Centers for 
Disease Control and Prevention, 1600 Clifton Rd NE MS A32; Atlanta, GA 
30333, Phone: 404-639-3747, Email: flucontest@cdc.gov.

SUPPLEMENTARY INFORMATION: 
    Subject of Challenge Competition: Entrants of the Predict the 
Influenza Season Challenge are asked to predict the beginning, the 
peak, and the intensity of the 2013-2014 influenza season at the 
national level and at each or any Health and Human Services (HHS) 
region level(s) in the United States by developing mathematical and 
statistical models that utilize digital surveillance data (e.g. Twitter 
data, mining internet search term data, internet-based surveys). If 
there are questions about the eligibility of different data sources, 
please contact Matthew Biggerstaff at 404-639-3747 or 
flucontest@cdc.gov.
    Eligibility Rules for Participating in the Competition:
    To be eligible to win a prize under this challenge, an individual 
or entity--
    (1) Must be at least 18 years old;
    (2) Shall have registered to participate in the competition under 
the rules promulgated by HHS/CDC;
    (3) Shall have complied with all the requirements under this 
section;
    (4) In the case of a private entity, shall be incorporated in and 
maintain a primary place of business in the United States, and in the 
case of an individual, whether participating singly or in a group, 
shall be a citizen or permanent resident of the United States; and
    (5) May not be a Federal entity or Federal employee acting within 
the scope of their employment.
    (6) Shall not be an HHS employee working on their applications or 
submissions during assigned duty hours.
    (7) Shall not be an employee of or contractor at HHS/CDC.
    (8) Federal grantees may not use Federal funds to develop COMPETES 
Act challenge applications unless consistent with the purpose of their 
grant award.
    (9) Federal contractors may not use Federal funds from a contract 
to develop COMPETES Act challenge applications or to fund efforts in 
support of a COMPETES Act challenge submission.
    An individual or entity shall not be deemed ineligible because the 
individual or entity used Federal facilities or consulted with Federal

[[Page 70304]]

employees during a competition if the facilities and employees are made 
available to all individuals and entities participating in the 
competition on an equal basis.
    By participating in this challenge, an individual or organization 
agrees to assume any and all risks related to participating in the 
challenge. Individuals or organizations also agree to waive claims 
against the Federal Government and its related entities, except in the 
case of willful misconduct, when participating in the challenge, 
including claims for injury; death; damage; or loss of property, money, 
or profits, and including those risks caused by negligence or other 
causes.
    By participating in this challenge, individuals or organizations 
agree to protect the Federal Government against third party claims for 
damages arising from or related to challenge activities.
    Individuals or organizations are not required to hold liability 
insurance related to participation in this challenge.
    Registration Process for Participants: To register, please send an 
email to flucontest@cdc.gov and include your first and last name, your 
address, and your phone number. The subject line of the email 
submission must be labeled ``Predict the Flu Challenge [Competitors 
Name] [Date of Submission]''. Additionally, this email should include 
the following statements: ``By participating in this competition, I 
agree to assume any and all risks and waive claims against the Federal 
Government and its related entities, except in the case of willful 
misconduct, for any injury, death, damage, or loss of property, 
revenue, or profits, whether direct, indirect, or consequential, 
arising from my participation in this prize contest, whether the 
injury, death, damage, or loss arises through negligence or otherwise. 
Additionally, I agree to indemnify the Federal Government against third 
party claims for damages arising from or related to competition 
activities.'' Registration opens on November 23, 2013, and closes 11:59 
p.m. EST on December 2, 2013.
    The Competition Submission Period is from December 1, 2013-March 
27, 2014. The first submission of the prediction must be received by 
December 2, 2013 and include a narrative describing the methodology of 
the prediction model and the results of that model (figures, tables, or 
narratives) using digital surveillance data (e.g. Twitter data, mining 
internet search term data, internet-based surveys) that predicts the 
beginning, the peak, and the intensity of the 2013-2014 influenza 
season at the national level in the United States. The methodology and 
data source(s) submitted to CDC must match the actual methods utilized 
by the competitors when making their influenza season predictions. In 
addition to the national-level predictions, competitors may also submit 
predictions of the beginning, the peak, and the intensity of the 2013-
2014 influenza seasons for any of the 10 HHS regions.
    Subsequent submissions will be required biweekly until the close of 
the submission period and must include the updated results of the 
prediction model (figures, tables, or narratives) at the national level 
in the United States using digital surveillance data; predictions for 
any of the 10 HHS regions will also be accepted, and submitting 
predictions for the 10 regions can potentially add to the final scores 
competitors. Subsequent submissions that include the updated 
predictions of the beginning, peak, and the intensity of the 2013-2014 
influenza season must be received by 11:59 p.m. Eastern on December 19, 
2013; January 2, 2014; January 16, 2014; January 30, 2014; February 13, 
2014; February 27, 2014; March 13, 2014; and March 27, 2014. A total of 
9 submissions must be received over the course of the contest to be 
eligible for the contest prize; late submissions will not be 
considered.
    Contest Prize: For the contest, one $75,000 prize will be awarded. 
The winning competitor will be recognized on the CDC influenza Web 
page.
    Payment of the Prize: Prizes awarded under this competition will be 
paid by electronic funds transfer and may be subject to Federal income 
taxes. HHS will comply with the Internal Revenue Service withholding 
and reporting requirements, where applicable.
    Basis Upon Which Winner Will Be Selected: A total of 9 submissions 
must be received over the course of the contest to be eligible for 
judging; late submissions will not be considered. The Competitors' 
model methodology and nine biweekly predictions will comprise their 
challenge entry. The selection of the winner for this challenge will be 
based on an evaluation of the methodology used to make the prediction 
and the accuracy of the prediction compared to the U.S. Outpatient 
Influenza-like Illness Surveillance Network (ILINet: https://www.cdc.gov/flu/weekly/) at the national level and any of the included 
10 HHS regions. National and HHS-level predictions must be scaled to 
ILINet.
    Submissions will be judged by a panel of reviewers that may include 
CDC staff, public health officials, and/or academics from noncompeting 
colleges or universities. Judges will score submissions on a scale of 0 
to 150 using the six criteria identified below. Based on the average 
score for each submission, the top submissions will be recommended to a 
Lead Judge in CDC's Influenza Division. The Lead Judge will assess the 
top submissions using the criteria below and select the first place 
entry. Judging will take place between March 28, 2014 and May 30, 2014.
i. Methodology (25)
     Are key materials to support the correct interpretation of 
the predictions by the judges, such as concepts, sources, and methods, 
provided as part of the submission?
     Are the predictions scaled to ILINet? Are the data and 
results presented clearly, allowing the judges who may not be experts 
in mathematical modeling to evaluate the model and its outputs?
     What are the data sources used to make the prediction? Is 
the data source representative? If not, which groups are 
underrepresented in the data? Are there any impacts caused by these 
exclusions?
     Is there a clear description of how the search terms were 
selected (if relevant)?
     Are there any serious accuracy or methodological problems 
with the prediction approach?
     Are measures provided that give an indication of how 
certain contestants are about their predictions? (e.g. confidence 
intervals, probability of the prediction occurring)?
ii. Predicting the Start of the Influenza Season (10)
     Does the model accurately predict the start of the 
influenza season at the national level and any of the included 10 HHS 
regions? The start of the season will be defined as the week when the 
percentage of visits reported through ILINet crosses the baseline value 
for three consecutive weeks. ILINet baseline values for the United 
States and the 10 HHS regions for the 2013-2014 influenza season are 
available at https://www.cdc.gov/flu/weekly/overview.htm#Outpatient.
iii. Predicting the Peak Week of the Influenza Season (10)
     Does the model accurately predict the peak week of ILINet 
at the national level and any of the included 10 HHS regions? The peak 
week will be defined as the surveillance week that the ILINet 
percentage is the highest for the 2013-14 influenza season in the 
United States and the 10 separate HHS regions.

[[Page 70305]]

iv. Predicting the Intensity of the Influenza Season (25)
     Does the model predict the intensity of the influenza 
season at the national and any of the included HHS regional levels? The 
intensity will be defined as the number of weeks that ILINet remains 
above baseline and the highest numeric value that the ILINet percentage 
reaches in the United States and the 10 separate HHS regions.
v. Timeliness and Reliability of the Predictions (20)
     How many weeks before each prediction milestone (the start 
and the peak week) was the most accurate prediction made?
     Did the prediction of each milestone vary widely between 
the Competitors' submissions?
vi. Geography (10)
     How comprehensively are the geographic regions of the 
United States represented in the source data?
vii. Optional HHS Regional Predictions (Up to 50 Bonus Points)
     HHS regional predictions will be evaluated separately 
using the six criteria identified above. Each submitted regional 
prediction will add between 0 and 5 points to the competitor's final 
score, depending on the evaluation of the accuracy, timeliness, and 
reliability of the prediction.
    Additional Information: The historical national surveillance data 
that could be used to enable training and correlation model development 
by competitors are available at https://gis.cdc.gov/grasp/fluview/fluportaldashboard.html and are updated every Friday at noon.
    The Competitors will not have to transfer their exclusive 
intellectual property rights to the CDC. Instead, the Competitors will 
grant to CDC non-exclusive license to practice their solutions.
    Compliance With Rules and Contacting Contest Winners: Finalists and 
the contest winners must comply with all terms and conditions of these 
official rules, and winning is contingent upon fulfilling all 
requirements herein. The initial finalists will be notified by email, 
telephone, or mail after the date of the judging. Awards may be subject 
to Federal income taxes, and the Department of Health and Human 
Services will comply with the Internal Revenue Service withholding and 
reporting requirements, where applicable.
    Privacy: If contestants choose to provide the CDC with personal 
information by registering or filling out the submission form through 
the Challenge.gov Web site, that information is used to respond to 
contestants in matters regarding their submission, announcements of 
entrants, finalists, and winners of the contest. Information is not 
collected for commercial marketing. Registering through the 
Challenge.gov Web site is not required, however. Registrants may submit 
an email to flucontest@cdc.gov as noted in ``Registration Process for 
Participants.'' Winners are permitted to cite that they won this 
contest.
    General Conditions: The CDC reserves the right to cancel, suspend, 
and/or modify the contest, or any part of it, for any reason, at CDC's 
sole discretion.
    Participation in this contest constitutes a contestants' full and 
unconditional agreement to abide by the contest's official rules found 
at www.Challenge.gov.

    Authority: 15 U.S.C. 3719.

    Dated: November 19, 2013.
Stacey Hoffman,
Acting Director, Division of Executive Secretariat, Centers for Disease 
Control and Prevention.
[FR Doc. 2013-28198 Filed 11-22-13; 8:45 am]
BILLING CODE 4163-18-P