Challenge Competition: Announcement of AHRQ Challenge on Integrating Healthcare System Data With Systematic Review Findings, 58350-58353 [2022-20703]

Agencies

• Agency for Healthcare Research and Quality
• DEPARTMENT OF HEALTH AND HUMAN SERVICES

[Federal Register Volume 87, Number 185 (Monday, September 26, 2022)]
[Notices]
[Pages 58350-58353]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2022-20703]


=======================================================================
-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Agency for Healthcare Research and Quality


Challenge Competition: Announcement of AHRQ Challenge on 
Integrating Healthcare System Data With Systematic Review Findings

AGENCY: Agency for Healthcare Research and Quality (AHRQ), HHS.

ACTION: Notice.

-----------------------------------------------------------------------

SUMMARY: The Agency for Healthcare Research and Quality (AHRQ) is 
announcing a challenge competition to explore the resources and 
infrastructure needed to integrate real-world data from healthcare 
systems into systematic review findings. This healthcare systems data 
can augment study findings synthesized in systematic reviews in a 
number of ways, including by filling evidence gaps identified in the 
systematic review to strengthen the available evidence, and by 
examining the applicability of systematic review findings to real-world 
populations, including population subgroups not examined in published 
studies. Ultimately, this work will help AHRQ understand if and how 
sources of data and information outside of traditional systematic 
reviews, particularly from healthcare systems themselves, could be used 
alongside systematic reviews to improve healthcare decision making, 
healthcare delivery and potentially patient outcomes. This challenge 
competition will start on (September 26, 2022) and will be completed in 
two phases, with cash prizes awarded at the end of Phase 2 to all of 
those proceeding to Phase 2 and to the winners of Phase 2. The winner 
and runner-up from Phase 2 will be posted on the AHRQ website.

DATES: Phase 1 Submission Deadline on January 9, 2023 and Phase 2 
Submission Deadline on July 10, 2023.

ADDRESSES: Submit your responses electronically via: https://www.challenge.gov/?challenge=ahrq-challenge-on-integrating-healthcare-system-data-with-systematic-review-findings.

FOR FURTHER INFORMATION CONTACT: Suchitra Iyer, Director, Technology 
Assessment Program; Email: [email protected], Telephone: 301-
427-1550.

SUPPLEMENTARY INFORMATION:

Problem Statement

    The Agency for Healthcare Research and Quality (AHRQ), U.S. 
Department of Health and Human Services (HHS), is announcing a 
challenge competition to explore the feasibility, resources and 
infrastructure needed to integrate real world data from healthcare 
systems into systematic review findings. The statutory authority for 
this challenge competition is Section 105 of the America COMPETES 
Reauthorization Act of 2010.
    AHRQ's Evidence-based Practice Center (EPC) Program produces 
systematic reviews which synthesize information from the peer reviewed 
literature and provide the state of the science on available healthcare 
technologies (such as pharmaceuticals and medical devices) as well as 
healthcare delivery strategies. The process of creating these reviews 
is stakeholder driven, methodologically rigorous, and transparent. 
Reviews are used to inform healthcare decisions, including 
recommendations in clinical practice guidelines as well as national 
coverage determinations by Medicare. AHRQ also supports healthcare 
systems in their efforts to improve the quality of care and optimize 
patient outcomes; systematic reviews are scoped to address issues of 
priority to healthcare systems. Yet, due to limitations in the 
literature base, EPC systematic reviews may be inconclusive or only 
represent a narrow patient population, making it difficult to 
generalize or implement the findings within heterogeneous healthcare 
systems.
    Systematic reviews may also lack contextual details that can inform 
successful implementation. Improving healthcare delivery (and thus 
patient outcomes) often entails addressing issues beyond the benefits 
or harms of an intervention, traditionally the objective of a 
systematic review. Traditional reviews may not explain gaps in uptake 
or use of a clinical service and questions about how best to implement 
a given clinical service (e.g., details around implementation of a 
service or intervention).
    A recent EPC Program methods report (https://effectivehealthcare.ahrq.gov/products/unpublished-health-data/methods-report) articulates specific scenarios with examples of where 
healthcare system data may most effectively complement systematic 
reviews (i.e., to improve the strength, applicability, and 
implementation of evidence). In one example, investigators at the Mayo 
Clinic found that published evidence on outcomes following total 
pancreatectomy was sparse, so they supplemented a meta-analysis of 
published studies with their own unpublished healthcare system data, 
which more than doubled the sample size and improved the strength of 
evidence available (https://pubmed.ncbi.nlm.nih.gov/20681992/).
    In another instance, secondary analyses of Veterans Affairs (VA) 
data (https://pubmed.ncbi.nlm.nih.gov/35810550/) confirmed the 
applicability to the VA of findings from a published systematic review 
(https://www.hsrd.research.va.gov/publications/esp/robot-gen-surg.cfm).
    The recent EPC methods report also outlines important limitations 
and considerations when using unpublished healthcare system data 
alongside systematic reviews, such as relevant limitations in data 
quality. However, the report did not address the necessary resources, 
skills, partnerships, and processes required to utilize healthcare 
system data alongside systematic reviews to strengthen the 
actionability of systematic reviews.
    This Challenge therefore invites applicants to conduct an analyses 
of healthcare system data to supplement an existing AHRQ EPC Program 
systematic review. This will help AHRQ understand if and how sources of 
data

[[Page 58351]]

and information outside of traditional systematic reviews, particularly 
from healthcare systems themselves, could be used alongside systematic 
reviews to improve decision making, healthcare delivery, and 
potentially patient outcomes.

Challenge Goal

    The AHRQ EPC Program is interested in learning how analysis of 
real-world data collected by healthcare systems can be used in 
conjunction with findings from an AHRQ systematic review to inform 
healthcare decision-making in the context of a specific local setting.
    The goal of this challenge is to explore and determine the 
feasibility, resources, and infrastructure needed to incorporate 
unpublished healthcare system data into systematic review findings. 
Ideally, these data will enable the healthcare system to make decisions 
about which practices to incorporate locally and how to overcome 
barriers to implement the evidence to improve clinical practice, 
healthcare system operations and, potentially, health outcomes. The 
winner and runner-up from Phase 2 will be posted on the AHRQ website.

Challenge Design

     All evidence reports (systematic reviews, rapid reviews, 
and technical briefs) published on the AHRQ website since January 2018 
(https://effectivehealthcare.ahrq.gov/about) may be considered for this 
Challenge.
     Healthcare systems and other provider groups interested in 
implementing evidence into practice at their sites may apply.
     An organization may choose to address no more than 2 
systematic reviews, submitting a separate proposal for each review.
     Teams should have expertise in the clinical topic, 
evidence-based practice, data analysis, and quality improvement.
    This Challenge consists of two phases:

Phase 1: Proposal

    Elicit written proposals on a topic for which an AHRQ evidence 
report (systematic review, rapid review or technical brief) has been 
published since January 2018 (https://effectivehealthcare.ahrq.gov/about) and that is relevant to a dilemma in the applicant's healthcare 
system. Each proposal is to be written in the form of a narrative that:
    1. Explicitly states the rationale (i.e., addressing evidence gaps 
to strengthen available evidence, examining applicability of findings 
to real-world patients) to complement findings from an EPC report with 
analysis of health system data, including a discussion of possible 
limitations of the analysis.
    2. Develops and describes an analytic plan for use of healthcare 
system data [EHR data from an individual healthcare system or networks 
of healthcare systems (for example, PCORnet, Epic's Cosmos, etc.)].
    3. Provides an approach for decision-making based on the results of 
the data analysis and the evidence report, and an evaluation of the 
decision-making process and results.
    4. Describes potential challenges, barriers, and strategies to 
successfully complete the analysis.
    5. Lists team members, their role, area of expertise and hours on 
project.
    A total of 5 proposals will be selected as winners for Phase 1.

Phase 2: Analyses

    Healthcare systems selected as winners in Phase 1 will be invited 
to provide a written narrative that:
    1. Includes a complete analysis of internal real-world data and 
appraisal of the analysis for risk of bias using a formal tool such as 
the ROBINS I (https://www.bmj.com/content/355/bmj.i4919).
    2. Specifies how the findings from unpublished data support, 
refute, and/or otherwise complement findings from the published 
evidence examined in the systematic review. If the unpublished data 
conflicts with the AHRQ review's conclusions, discuss possible reasons 
for the discrepancy [e.g., challenges with internal validity of 
healthcare system data analysis related to study design and methods 
used, or challenges with external validity with respect to population 
sub-groups (gender, race/ethnicity, multimorbidity) examined in the 
healthcare system data].
    3. Describes how the unpublished data informed decision-making 
(e.g., adapt, adopt, abandon, prioritize).
    4. Reports on solutions to any barriers encountered to using 
healthcare system data alongside a published evidence review, including 
barriers with access to healthcare system data, interoperability of 
data sources, and data analysis.
    5. Briefly describes potential approaches to implement or 
deimplement the evidence, including use of clinical advisories, 
clinical pathways, clinical decision support, or any other method. The 
plan should describe anticipated risks and barriers and strategies for 
successful implementation. Decisions made against uptake should be 
justified.

Timeline and Prize Amounts

    AHRQ is hosting this challenge as a two-phase competition. All 
costs associated with developing and submitting proposals as well as 
conducting the analysis of real-world data will be the responsibility 
of the Challenge participant. Cash prizes will be awarded only after 
the projects are evaluated and determined acceptable at the end of 
Phase 2.

Timeline

     September 26, 2022--Challenge launch.
     January 9, 2023--Submissions for Phase 1 (written 
proposals) are due. AHRQ will complete the review of the proposals 
within 6-8 weeks of closing the announcement.
     March 10, 2023--AHRQ will announce the Phase 1 winners. 
Phase 2 of the Challenge will commence once the Phase 1 winners are 
announced and notified by March 10, 2023. The AHRQ team will schedule a 
live, virtual technical assistance webinar with all winners of Phase 1 
to discuss scope of content, accessibility/compliance with Section 508, 
and address questions that the winners may have.
     March 10, 2023--Phase 2 participants will have at least 
120 calendar days from notification to conduct and submit their 
analyses as described in their proposal(s). The deadline to submit the 
analysis is July 10, 2023.
     Fall 2023--The final winners of Phase 2 of the competition 
will be announced in October 2023.

Prize Amounts

    The top five entries in phase one will be invited to participate in 
Phase 2. Upon completion of Phase 2, each of the top five entries will 
each receive cash awards of $50,000. Additionally, the first-place 
winner from Phase 2 will be awarded an additional $150,000 and the 
runner-up will be awarded an additional $100,000. The winner and 
runner-up from Phase 2 will be posted on the AHRQ website.

How To Enter the Challenge

    Participants can register by visiting the Challenge.gov website 
(https://www.challenge.gov). Participants should carefully review 
challenge information and submission requirements on the website, 
including the intellectual property rules and assessment criteria. 
Participants are encouraged to follow the Challenge on Challenge.gov to 
obtain any updates and reminders of upcoming deadlines.

Submission Requirements

Phase 1

    The submitted proposals must be written in US English and submitted

[[Page 58352]]

using Challenge.gov no later than January 9, 2023. Applicants or 
applicant organizations shall submit no more than 2 proposals, and no 
proposal shall describe more than one topic. Each proposal will be no 
more than 2,000 words, double spaced, 11-point type size, with 1-inch 
margins. Include in proposals plans for meeting Section 508 
accessibility standards.

Phase 2

    Analyses shall be a journal article-style written narrative in U.S. 
English in no more than 4,000 words and submitted using the 
Challenge.gov website no later than July 10, 2023.

Review Process

    All submissions will be reviewed by at least two subject matter 
experts from within or outside the federal government who will score 
the proposals based on the assessment criteria and provide a brief 
comment about the submission. The scores/comments on Phase 1 and Phase 
2 submissions will be compiled, and a ranked summary provided to AHRQ 
staff. AHRQ will select winners based on quantitative and qualitative 
assessments.

Evaluation Criteria for Selecting Winning Applications

Assessment Criteria for Phase 1

1. Approach (40 points)
    Does the proposal sufficiently describe:
    a. the context and rationale for why the EPC report has been 
chosen, and how the analyses of unpublished data will complement the 
findings from the systematic review?
    b. the data source and the analytic approach? Does the described 
analytic approach provide the right balance of feasibility, rigor, and 
innovation for the project?
2. Impact (40 points)
    Does the proposal clearly and concisely describe:
    a. potential routes for uptake of evidence within the healthcare 
system?
    b. method to measure whether the uptake will have an impact on 
healthcare delivery, quality of care or patient outcomes?
    c. anticipated risks, barriers, and strategies to successfully 
complete the project?
3. Team (20 points)
    Does the team have the right combination of expertise to support 
the proposed technical approach?
    Compliance (pass/fail)--Does the Phase 1 proposal adequately 
address required accessibility standards (Section 508)?

Assessment Criteria for Phase 2

    Participants in Phase 2 may be disqualified if their submitted 
analyses deviate from their Phase 1winning proposals.
1. Analysis (40 points)
    a. Are the description and discussion of the findings from the 
analysis comprehensive?
    b. To what degree is the alignment/nonalignment between healthcare 
system data, the AHRQ systematic review, and the healthcare system's 
decision-making explained?
2. Description of the Process (40 points)
    a. Are the risks, barriers, challenges encountered, solutions, and 
required infrastructure well described?
3. Future Plans (20 points)
    a. Is the preliminary plan for implementation and evaluation 
appropriate and feasible?
    Compliance (pass/fail)--Does the Phase 1 proposal adequately 
address required accessibility standards (Section 508)?

Eligibility Rules for Participating in the Challenge

    To be eligible under this Challenge, an individual (whether 
participating singly or in a group) or entity:
    1. Shall have registered (Challenge.gov) to participate in the 
Challenge.
    2. Shall have complied with the rules set forth in this 
announcement for participation in this Challenge.
    3. Shall be incorporated and maintain a primary place of business 
in the United States (in the case of a private entity), and in the case 
of an individual, whether participating singly or in a group, shall be 
a citizen or permanent resident of the United States.
    4. May not be a Federal entity or Federal employee acting within 
the scope of their employment. (All Federal employees should consult 
with their agency Ethics Official to determine whether the federal 
ethics rules will limit or prohibit the acceptance of a prize).
    5. May not be an employee of AHRQ or any other company, 
organization, or individual involved with the design, production, 
execution, judging, or distribution of the Challenge, or their 
immediate family (spouse, parents and step-parents, siblings and step-
siblings, and children and step-children), or household members (people 
who share the same residence at least 3 months out of the year).
    6. May not use Federal funds from a grant to develop Challenge 
applications unless consistent with the purpose of the grant award.
    7. May not use Federal funds from a contract to develop Challenge 
applications or to fund efforts in support of a Challenge submission.
    8. Shall not be deemed ineligible because the individual or entity 
used Federal facilities or consulted with Federal employees during a 
competition if the facilities and employees are made equitably 
available to all individuals and entities participating in the 
competition.
    9. Shall not be required to purchase liability insurance as a 
condition of participation in this competition.

Additional Rules of Participation

    By participating in this Challenge, each individual (whether 
participating singly or in a group) or entity:
    1. Agrees to follow all applicable Federal, State, and local laws, 
regulations, and policies.
    2. Agrees to comply with all terms and conditions of participation 
in this Challenge.
    3. Agrees that the submission will not use HHS or AHRQ logos or 
official seals, except as authorized by HHS or AHRQ. Notwithstanding 
this authorized use of AHRQ/HHS branding, participants will not claim 
endorsement by AHRQ/HHS.
    4. Understands that all materials submitted to AHRQ as part of a 
submission become AHRQ records. Any confidential commercial or 
financial information contained in a submission must be clearly 
designated as such at the time of submission.
    5. Agrees that a winning submission may only be announced by AHRQ 
and in a time, place, and manner determined by AHRQ in its sole 
discretion. Winners will be permitted to publicize and publish their 
winning submissions in accordance with instructions provided by AHRQ. 
The winner and runner-up from Phase 2 will be posted on the AHRQ 
website.
    6. Agrees that the submission must not infringe upon copyright or 
any other rights of any third party.
    7. Agrees to assume any and all risks and waive claims against the 
Federal Government and its related entities, except in the case of 
willful misconduct, for any injury, death, damage, or loss of property, 
revenue, or profits, whether direct, indirect, or consequential, 
arising from participation in this prize contest, whether the injury, 
death, damage, or loss arises through negligence or otherwise.
    8. Agrees to indemnify the Federal Government against third-party 
claims

[[Page 58353]]

for damages arising from or related to Challenge activities.
    9. Understands that circulation of findings could be worldwide, and 
that the Federal Government will not compensate the participants for 
any use; winners shall receive a one-time cash prize as set forth in 
this announcement. The winner and runner-up from Phase 2 will be posted 
on the AHRQ website.
    10. Understands that AHRQ reserves the right to cancel, suspend, 
and/or modify this prize contest, or any part of it, for any reason, in 
AHRQ's sole discretion. AHRQ also reserves the right not to award any 
prizes if no entries are deemed worthy.
    11. Understands that AHRQ will not select a winner that is named on 
the Excluded Parties List System (EPLS).

Intellectual Property (IP) Rights

    1. Each participant retains title and full ownership in and to 
their submission. Participants expressly reserve all intellectual 
property rights not expressly granted.
    2. By participating in the Challenge, each participant (whether 
participating singly or in a group) acknowledges that he or she is the 
sole author or owner of, or has a right to use, any copyrightable works 
that the submission comprises, that the works are wholly original with 
the participant (or is an improved version of an existing work that the 
participant has sufficient rights to use and improve), and that the 
submission does not infringe any copyright or any other rights of any 
third party of which participant is aware.
    3. In addition, each participant (whether participating singly or 
in a group) grants to the U.S. Government a paid-up, nonexclusive, 
royalty-free, irrevocable worldwide license in perpetuity, and the 
right to reproduce, publish, post, link to, share, display publicly (on 
the web or elsewhere) and prepare derivative works, including the right 
to authorize others to do so on behalf of the U.S. Government.
    4. Each participant must clearly delineate any intellectual 
property and/or confidential commercial information contained in a 
submission that the participant wishes to protect as proprietary data, 
in accordance with Additional Rules of Participation No. 4.
    5. If the submission includes any third-party works (such as third-
party content or open-source code), the participant must be able to 
provide, upon request, documentation of all appropriate licenses and 
releases for use of such third-party works. If the participant cannot 
provide documentation of all required licenses and releases, AHRQ 
reserves the right, in its sole discretion, to disqualify the 
submission.

    Dated: September 20, 2022.
Marquita Cullom,
Associate Director.
[FR Doc. 2022-20703 Filed 9-23-22; 8:45 am]
BILLING CODE 4160-90-P