Agency Information Collection Activities: Proposed Collection; Comment Request, 6194-6197 [2020-02057]

Download as PDF 6194 Federal Register / Vol. 85, No. 23 / Tuesday, February 4, 2020 / Notices of 6,000 patient and 6,000 clinician surveys. Audio recordings of up to 260 clinical encounters will be obtained, with burden not to exceed 10 minutes to obtain patient informed consent. Two clinician surveys will be conducted, one immediately following SHARE training and one following the second observation period, one month following SHARE training. These will be conducted with no more than 100 clinicians and will require no more than 10 minutes to complete. Exhibit 2 shows the estimated cost burden over 3 years, based on the respondents’ time to participate in these research activities. The total cost burden is estimated to be $19,688. EXHIBIT 1—ESTIMATED BURDEN HOURS OVER 3 YEARS Number of respondents Type of information collection Number of responses per respondent Hours per response Total burden hours Card survey (patient) ....................................................................................... Card survey (clinician) ..................................................................................... Audio recorded encounters ............................................................................. Clinician survey immediately following training ............................................... Clinician survey one month following training ................................................. 6,000 6,000 260 100 100 1 1 1 1 1 2/60 1/60 10/60 10/60 10/60 200 100 44 17 17 Totals ........................................................................................................ 12,460 na na 378 * May include telephone non-response follow-up in which case the burden will not change EXHIBIT 2—ESTIMATED COST BURDEN OVER 3 YEARS Number of respondents Type of information collection Total burden hours Average hourly wage rate * Total cost burden Card survey (patient) ....................................................................................... Card survey (clinician) ..................................................................................... Audio recorded encounters ............................................................................. Clinician survey immediately following training ............................................... Clinician survey one month following training ................................................. 6,000 6,000 260 100 100 200 100 44 17 17 $24.98 101.43 24.98 101.43 101.43 $4,996 10,143 1,100 1,725 1,725 Totals ........................................................................................................ 12,460 378 na 19,689 * Based upon the average wages for 29–1060 Physicians and Surgeons (broad) and 00–0000 All Occupations, ‘‘National Compensation Survey: Occupational Wages in the United States, May 2018,’’ U.S. Department of Labor, Bureau of Labor Statistics https://www.bls.gov/oes/current/ oes_nat.htm#29-0000. khammond on DSKJM1Z7X2PROD with NOTICES Request for Comments In accordance with the Paperwork Reduction Act, comments on AHRQ’s information collection are requested with regard to any of the following: (a) Whether the proposed collection of information is necessary for the proper performance of AHRQ’s health care research and health care information dissemination functions, including whether the information will have practical utility; (b) the accuracy of AHRQ’s estimate of burden (including hours and costs) of the proposed collection(s) of information; (c) ways to enhance the quality, utility and clarity of the information to be collected; and (d) ways to minimize the burden of the collection of information upon the respondents, including the use of automated collection techniques or other forms of information technology. Comments submitted in response to this notice will be summarized and included in the Agency’s subsequent request for OMB approval of the proposed information collection. All comments will become a matter of public record. VerDate Sep<11>2014 17:48 Feb 03, 2020 Jkt 250001 Dated: January 29, 2020. Virginia L. Mackay-Smith, Associate Director. [FR Doc. 2020–02116 Filed 2–3–20; 8:45 am] BILLING CODE 4160–90–P DEPARTMENT OF HEALTH AND HUMAN SERVICES Agency for Healthcare Research and Quality Agency Information Collection Activities: Proposed Collection; Comment Request Agency for Healthcare Research and Quality, HHS. AGENCY: ACTION: Notice This notice announces the intention of the Agency for Healthcare Research and Quality (AHRQ) to request that the Office of Management and Budget (OMB) approve the proposed information collection project ‘‘Evaluating the Implementation of Products by Learning Health Systems to Inform and Encourage Use of AHRQ Evidence Reports.’’ SUMMARY: PO 00000 Frm 00054 Fmt 4703 Sfmt 4703 Comments on this notice must be received by 60 days after date of publication. ADDRESSES: Written comments should be submitted to: Doris Lefkowitz, Reports Clearance Officer, AHRQ, by email at doris.lefkowitz@AHRQ.hhs.gov. Copies of the proposed collection plans, data collection instruments, and specific details on the estimated burden can be obtained from the AHRQ Reports Clearance Officer. FOR FURTHER INFORMATION CONTACT: Doris Lefkowitz, AHRQ Reports Clearance Officer, (301) 427–1477, or by email at doris.lefkowitz@AHRQ.hhs.gov. SUPPLEMENTARY INFORMATION: DATES: Proposed Project Evaluating the Implementation of Products by Learning Health Systems To Inform and Encourage Use of AHRQ Evidence Reports AHRQ’s Evidence-based Practice Center (EPC) Program has 20 years of experience in synthesizing research to inform evidence-based health care practice, delivery, policies, and research. The AHRQ EPC program is committed to partnering with E:\FR\FM\04FEN1.SGM 04FEN1 khammond on DSKJM1Z7X2PROD with NOTICES Federal Register / Vol. 85, No. 23 / Tuesday, February 4, 2020 / Notices organizations to make sure its evidence reports can be used in practice. Historically, most of its evidence reports have been used by clinical professional organizations to support the development of clinical practice guidelines or Federal agencies to inform their program planning and research priorities. To improve the uptake and relevance of the AHRQ EPC’s evidence reports, specifically for health systems, AHRQ has contracted with the American Institutes for Research (AIR) to obtain feedback from learning health systems (LHSs) to assist the AHRQ EPC program in developing and disseminating evidence reports that can be used to improve the quality and effectiveness of patient care. Even if an EPC evidence report topic addresses LHS-specific evidence needs, the density of the information in an evidence report may preclude its easy review by busy LHS leaders and decisionmakers. AHRQ understands that to facilitate use by LHSs, complex evidence reports must be translated into a format that promotes LHS evidencebased decision making and can be contextualized within each LHS’ own system-generated evidence. Such translational products, for the purposes of this notice, are referred to simply as ‘‘products.’’ The purpose of this information collection is to support a process evaluation of use and implementation of two such products into LHS decisionmaking processes, workflows, and clinical care. The evaluation has the following goals: 1. Document how LHSs prioritize filling evidence gaps, make decisions about using evidence, and implement tools to support and promote evidence use in clinical care. 2. Assess the contextual factors that may influence implementation success; associated implementation resources, barriers and facilitators; and satisfaction of LHS leaders and clinical staff. 3. Provide the AHRQ EPC program with necessary insights about the perspectives, needs, and preferences of LHS leaders and clinical staff as related to decisions and implementation of products into practice. This study is being conducted by AHRQ through its contractor, the American Institutes for Research (AIR), pursuant to AHRQ’s statutory authority to conduct and support research on, and disseminate information on, health care and on systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness and value of healthcare services. 42 U.S.C 299a(a)(1). VerDate Sep<11>2014 17:48 Feb 03, 2020 Jkt 250001 Method of Collection To achieve the goals of this project, the following data collection activities will be implemented: 1. Key informant interviews with health system leaders, clinicians and staff; and 2. compilation and coding of notes from ‘‘implementation support’’ meetings (‘‘check-ins’’) between an implementation facilitator and site champions who are implementing the products. Brief Background on the Products To Be Implemented by LHSs in This Study AHRQ is funding the development of two products that are specifically intended to make the findings from EPC evidence reports more accessible and usable by health systems. These are the products that will be offered to LHSs for potential implementation during this project. They include a ‘‘triage tool’’ and a ‘‘data visualization tool’’ that have been designed to support LHS use of AHRQ evidence reports. The LHS triage tool presents high-level results of evidence reports that enable leaders within LHSs to quickly understand the relevance of the reports to their organization, share high-level information with key stakeholders (e.g., healthcare executives), and link to more granular data from the report. The data visualization tool presents data from the evidence review and individual studies in a dynamic, interactive website. The evaluation will capture the anticipated variation in how the LHS might use the products and the unique experience of LHSs. Key Informant Interviews There will be two rounds of key informant interviews: (1) In-person preliminary interviews will be conducted early in the implementation period (months 1–3) with LHS leaders and clinicians and will focus on health systems’ rationale for selecting each product and early experiences with its roll-out into practice; (2) remote followup interviews will be conducted via telephone later in the implementation period (months 10–11) with two sets of stakeholders: (a) LHS leaders and (b) clinicians/staff (hereafter, ‘‘clinical staff’’) actively implementing the product. These follow-up interviews will focus on health systems’ experiences implementing their selected product(s). All interviews (preliminary and follow-up) will be 60 minutes in duration, recorded with permission of the key informants, and transcribed for analysis. Up to 88 total interviews will be conducted across the two rounds of PO 00000 Frm 00055 Fmt 4703 Sfmt 4703 6195 key informant interviews. Assuming the same LHS leaders participate in the preliminary and follow-up interviews, the key informant interviews will involve 4–5 LHS leaders and clinical staff from each of the eleven LHSs implementing the study. Additional detail about the information collection components is provided below. 1. In-person preliminary interviews. The preliminary interviews will include 2–3 LHS leaders/decisionmakers at each of eleven implementation sites for a maximum of 33 interviews in the first round of data collection. The interviews will be conducted during implementation site visits that are occurring early in the project to support the health systems’ testing and/or roll out of the products into clinical workflows. Specific topics explored in the preliminary interviews include LHSs’ decision to participate in implementation, decision considerations for the selected product, experiences leading the implementation, and early experiences and perceptions of the selected product(s). To limit respondent burden, we will use the implementation site visits as an opportunity for conducting the preliminary interviews, thereby limiting the need to schedule additional time with respondents for a phone interview. If a respondent has limited availability during the site visit, however, we may need to do the preliminary interview remotely or substitute the respondent with another qualified staff member who is available during the implementation site visit. 2. Remote follow-up interviews. The follow-up interviews will include the 2–3 LHS leaders/decisionmakers from the preliminary interviews (maximim n = 33), along with 2 additional clinical staff (n = 22) at each of eleven implementation sites for a maximum of 55 follow-up interviews. Specific topics explored in the follow-up interviews include LHS leaders’ and clinical staff’s experiences with each product as well as their perceptions of the relative advantage, acceptability/compatibility, appropriateness, and feasibility of using the product; implementation fidelity (i.e., if the implementation went as planned), reach, barriers and facilitators, and associated costs; any outcomes of implementing the product (e.g., achieved any intended systemic changes); and likely sustainability of continuing to use the product in practice. The two sets of in-depth qualitative interviews will allow for a nuanced exploration of both what LHSs value about the products and what it takes to successfully implement such tools into practice. The research on implementation and uptake of products to promote use of evidence in LHS settings is sparse, thus it is important to use a data collection strategy for the evaluation that will yield rich information about the experience of health systems, LHS decisionmakers, and the staff implementing the tools into practice. A quantitative survey would not yield the depth of individual E:\FR\FM\04FEN1.SGM 04FEN1 6196 Federal Register / Vol. 85, No. 23 / Tuesday, February 4, 2020 / Notices feedback that is needed to capture the experience of implementing these tools and the unique contexts of the health systems. Thus, interviews are the preferred method of systematically collecting these data. khammond on DSKJM1Z7X2PROD with NOTICES Implementation Support Meetings/ ‘‘Check-ins’’ In addition to key informant interviews, which will be conducted only at the beginning and end of implementation, AHRQ will gather information throughout the implementation period by using monthly implementation support meetings between implementation facilitators and site champions as an ongoing opportunity to ask key questions about implementation progress. Although the primary goal of these check-in meetings is to provide technical assistance with implementation and recommendations for handling emergent challenges in the implementation process, they will also be a source of rich information for the evaluation. Because these meetings occur in real time as the implementation unfolds, they will reduce the potential biases (e.g., selective memory, recency effects, forgetting details about key events and their sequence) associated with only collecting data at the beginning or end of the implementation period. These check-in meetings will occur by telephone and are intended to monitor implementation progress, provide support to health systems, and discuss next steps. AIR implementation facilitators for each site will schedule telephone conference calls with site champions (N = 11), during which structured notes will be taken. These notes will be supplemented with relevant information from other touchpoints between the facilitators and champions (e.g., ad hoc calls, email exchanges, and voluntary participation in monthly shared learning events) as they naturally occur. Notetakers will capture and document information related to key implementation domains as these topics arise in check-in meetings and other facilitator/champion encounters throughout implementation. Estimated Annual Respondent Burden Exhibit 1 shows the total estimated annualized burden of 214.5 hours for the two rounds of key informant interviews and implementation ‘‘checkins’’ combined. For the key informant interviews (totaling 154 hours), burden is included for: (1) LHS leaders/ decisionmakers participating in the preliminary interviews (a maximum of 33 hours), (2) LHS leaders/ decisionmakers participating in the follow-up interviews (a maximum of 33 hours), (3) clinical staff participating in the follow-up interviews (a maximum of 22 hours), (4) interviewee review of materials, consent forms, and logistics in advance of their respective interviews (i.e., 16.5 + 5.5 = 22 hours) and (5) time for designated LHS staff (e.g., the LHS member, a designated site liaison, selected interviewees) to recommend key informants, coordinate implementation support, and help with scheduling of in-person preliminary interviews and remote follow-up interviews (44 hours). Also included in Exhibit 1 is the estimated annualized burden hours for monthly check-ins between implementation facilitators and LHS champions for informal technical assistance support and the quick status probes on implementation progress (a maximum of 60.5 hours). These annualized burden estimates for the key informant interviews and the coaching sessions are further explained below. Key Informant Interviews: Expanded Detail on Burden Estimates We estimate 1 hour for each key informant interview for: (1) LHS leaders/decisionmakers participating in the preliminary interviews (a maximum of 33 hours), (2) LHS leaders/ decisionmakers participating in the follow-up interviews (a maximum of 33 hours), (3) clinical staff participating in the follow-up interviews (a maximum of 22 hours), (Total interview burden = 1.00 hour × maximum of 88 interviews = 88 hours). We estimate an additional 15 minutes (0.25 hours) will be needed for key informants to prepare for their respective interview(s) (Total interview preparation burden = 0.25 hours × maximum of 88 interviews = 22 hours; of which 16.5 hours is for leaders/ decisionmakers to prepare for both preliminary and follow-up interviews and 5.5 is for clinical staff to prepare for their participation in the follow-up interviews only). Finally we estimate time for LHS leaders and staff to identify interview candidates, facilitate recruitment, coordinate implementation support, and assist with interview scheduling (4.00 hours per each of 11 LHSs; Total staff assistance burden = 4.00 hours × 11 sites = 44 hours). The ‘‘staff assistance’’ burden involves the following: • In each of the eleven LHS organizations implementing the product(s), the LHS member (and/or site liaison/champion) will identify prospective key informants (i.e., other LHS leaders/decisionmakers and appropriate clinical staff), with additional key informants subsequently identified through snowball sampling. • Designated LHS staff (i.e., LHS member, designee and/or site liaison/ champion) will provide needed contact information to the AIR evaluation team for outreach and recruitment of the prospective key informant interview candidates, assist with interview scheduling, and coordinate implementation support with the AIR team. We will develop standardized email messages to reach out to interview candidates and a written overview of the project, the evaluation, and the purpose of the interview. We will coordinate scheduling of both the implementation support check-ins and the 60-minute interviews at the most convenient time, considering the needs of the LHS leadership and staff. For the preliminary interviews, if prospective interviewees are not available during our site visit, we will ask for suggestions of other LHS staff who meet our recruitment criteria or arrange a telephone interview, if needed. Implementation Support Meetings/ Check-Ins: Expanded Detail on Burden Estimates We estimate 60.5 hours for the monthly check-ins between implementation facilitators and LHS champions. This includes an average of 30 minutes of implementation support/ check-in meetings per each of the 11 LHSs for each month of implementation (11 months). (11 months × 0.5 hours = 5.5 hours). Across LHSs, the estimated burden associated with check-ins is approximately 61 hours across the implementation period (5.5 hours × 11 LHSs = 60.5 hours). EXHIBIT 1—ESTIMATED ANNUALIZED BURDEN HOURS Number of respondents * Form name In-person preliminary interviews with LHS leaders/decisionmakers ............... VerDate Sep<11>2014 17:48 Feb 03, 2020 Jkt 250001 PO 00000 Frm 00056 Fmt 4703 Sfmt 4703 Number of responses per respondent ** 33 E:\FR\FM\04FEN1.SGM 1 04FEN1 Hours per response 1.00 Total burden hours 33 6197 Federal Register / Vol. 85, No. 23 / Tuesday, February 4, 2020 / Notices EXHIBIT 1—ESTIMATED ANNUALIZED BURDEN HOURS—Continued Number of respondents * Form name Number of responses per respondent Hours per response Total burden hours Remote follow-up interviews with LHS leaders/decisionmakers ..................... Remote follow-up interviews with clinical staff ................................................ Review of materials prior to BOTH preliminary and follow-up interviews— LHS leaders/decisionmakers ....................................................................... Review of materials prior to interviews—clinical staff ..................................... Interview scheduling and other staff assistance .............................................. Implementation check-ins: Brief monthly implementation progress checks, documented for the evaluation as structured notes on implementation topics naturally occurring in coach/champion encounters ........................... ** 33 22 1 1 1.00 1.00 33 22 33 22 11 2 1 1 0.25 0.25 4.00 16.5 5.5 44 11 11 0.5 60.5 Total .......................................................................................................... ........................ ........................ ........................ *** 214.5 * The numbers in this column give the maximum number of respondents for each listed activity based on a range in the number of recruits per site (e.g., ‘‘2–3 LHS leaders/decisionmakers’’). The balance may shift some between LHS leaders/decisionmakers and clinical staff depending on implementation team and leadership composition at each site. In any case, 88 interviews (33+33+22=88) is a maximum possible in the event each of the 11 sites contributes 3 ‘‘LHS leaders/decsionmakers’’ (likely the same people for preliminary and follow-up interviews) and 2 additional clinical staff (for follow-up interviews only) as key informants. It is more likely that the total number of interviews will be around 80. ** These are likely to be the same 33 respondents in both preliminary and follow-up interviews. *** Total maximum burdened hours estimate based on maximum of 88 interviews. Costs associated with the estimated annualized burden hours are provided in Exhibit 2. EXHIBIT 2—ESTIMATED ANNUALIZED COST BURDEN Number of respondents * Form name Average hourly wage rate ** Total cost burden 33 33 22 33 33 22 a $94.47 33 22 11 16.5 5.5 44 a 94.47 c 20.34 1,558.76 286.72 894.96 11 60.5 a 94.47 5,715.44 ........................ ........................ ........................ 15,837.76 In-person preliminary interviews with leaders/decisionmakers ....................... Remote follow-up interviews with leaders/decisionmakers ............................. Remote follow-up interviews with clinical staff ................................................ Review of materials prior to BOTH preliminary and follow-up interviews— LHS leaders/decisionmakers ....................................................................... Review of materials prior to interviews—clinical staff ..................................... Interview scheduling and other staff assistance c ............................................ Implementation check-ins (documented for the evaluation as structured notes on implementation progress) ............................................................. Total .......................................................................................................... Total burden hours a 94.47 b 52.13 b 52.13 $3,117.51 3,117.51 1,146.86 * The numbers in this column give the maximum number of respondents for each listed activity based on a range in the number of recruits per site (e.g., ‘‘2–3 LHS leaders/decisionmakers’’). As noted in the comment to Exhibit 1, the balance may shift some between LHS leaders/decisionmakers and clinical staff depending on implementation team and leadership composition at each site. In any case, 88 interviews (33+33+22=88) is a maximum possible. ** National Compensation Survey: Occupational wages in the United States May 2018 ‘‘U.S. Department of Labor, Bureau of Labor Statistics.’’ a Based on the mean wages for Internists, General. 29–1063; annual salary of $196,490. b Based on the mean wages for Physician Assistants, 29–1071; annual salary of $108,430. c Based on the mean wages for Secretaries and Administrative Assistants, 43–6010; annual salary of $42,320. khammond on DSKJM1Z7X2PROD with NOTICES Request for Comments In accordance with the Paperwork Reduction Act, comments on AHRQ’s information collection are requested with regard to any of the following: (a) Whether the proposed collection of information is necessary for the proper performance of AHRQ’s health care research and health care information dissemination functions, including whether the information will have practical utility; (b) the accuracy of VerDate Sep<11>2014 17:48 Feb 03, 2020 Jkt 250001 AHRQ’s estimate of burden (including hours and costs) of the proposed collection(s) of information; (c) ways to enhance the quality, utility and clarity of the information to be collected; and (d) ways to minimize the burden of the collection of information upon the respondents, including the use of automated collection techniques or other forms of information technology. Comments submitted in response to this notice will be summarized and PO 00000 Frm 00057 Fmt 4703 Sfmt 4703 included in the Agency’s subsequent request for OMB approval of the proposed information collection. All comments will become a matter of public record. Dated: January 29, 2020. Virginia L. Mackay-Smith, Associate Director. [FR Doc. 2020–02057 Filed 2–3–20; 8:45 am] BILLING CODE 4160–90–P E:\FR\FM\04FEN1.SGM 04FEN1

Agencies

[Federal Register Volume 85, Number 23 (Tuesday, February 4, 2020)]
[Notices]
[Pages 6194-6197]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2020-02057]


-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Agency for Healthcare Research and Quality


Agency Information Collection Activities: Proposed Collection; 
Comment Request

AGENCY: Agency for Healthcare Research and Quality, HHS.

ACTION: Notice

-----------------------------------------------------------------------

SUMMARY: This notice announces the intention of the Agency for 
Healthcare Research and Quality (AHRQ) to request that the Office of 
Management and Budget (OMB) approve the proposed information collection 
project ``Evaluating the Implementation of Products by Learning Health 
Systems to Inform and Encourage Use of AHRQ Evidence Reports.''

DATES: Comments on this notice must be received by 60 days after date 
of publication.

ADDRESSES: Written comments should be submitted to: Doris Lefkowitz, 
Reports Clearance Officer, AHRQ, by email at 
[email protected].
    Copies of the proposed collection plans, data collection 
instruments, and specific details on the estimated burden can be 
obtained from the AHRQ Reports Clearance Officer.

FOR FURTHER INFORMATION CONTACT: Doris Lefkowitz, AHRQ Reports 
Clearance Officer, (301) 427-1477, or by email at 
[email protected].

SUPPLEMENTARY INFORMATION: 

Proposed Project

Evaluating the Implementation of Products by Learning Health Systems To 
Inform and Encourage Use of AHRQ Evidence Reports

    AHRQ's Evidence-based Practice Center (EPC) Program has 20 years of 
experience in synthesizing research to inform evidence-based health 
care practice, delivery, policies, and research. The AHRQ EPC program 
is committed to partnering with

[[Page 6195]]

organizations to make sure its evidence reports can be used in 
practice. Historically, most of its evidence reports have been used by 
clinical professional organizations to support the development of 
clinical practice guidelines or Federal agencies to inform their 
program planning and research priorities. To improve the uptake and 
relevance of the AHRQ EPC's evidence reports, specifically for health 
systems, AHRQ has contracted with the American Institutes for Research 
(AIR) to obtain feedback from learning health systems (LHSs) to assist 
the AHRQ EPC program in developing and disseminating evidence reports 
that can be used to improve the quality and effectiveness of patient 
care.
    Even if an EPC evidence report topic addresses LHS-specific 
evidence needs, the density of the information in an evidence report 
may preclude its easy review by busy LHS leaders and decisionmakers. 
AHRQ understands that to facilitate use by LHSs, complex evidence 
reports must be translated into a format that promotes LHS evidence-
based decision making and can be contextualized within each LHS' own 
system-generated evidence. Such translational products, for the 
purposes of this notice, are referred to simply as ``products.''
    The purpose of this information collection is to support a process 
evaluation of use and implementation of two such products into LHS 
decisionmaking processes, workflows, and clinical care. The evaluation 
has the following goals:
    1. Document how LHSs prioritize filling evidence gaps, make 
decisions about using evidence, and implement tools to support and 
promote evidence use in clinical care.
    2. Assess the contextual factors that may influence implementation 
success; associated implementation resources, barriers and 
facilitators; and satisfaction of LHS leaders and clinical staff.
    3. Provide the AHRQ EPC program with necessary insights about the 
perspectives, needs, and preferences of LHS leaders and clinical staff 
as related to decisions and implementation of products into practice.
    This study is being conducted by AHRQ through its contractor, the 
American Institutes for Research (AIR), pursuant to AHRQ's statutory 
authority to conduct and support research on, and disseminate 
information on, health care and on systems for the delivery of such 
care, including activities with respect to the quality, effectiveness, 
efficiency, appropriateness and value of healthcare services. 42 U.S.C 
299a(a)(1).

Method of Collection

    To achieve the goals of this project, the following data collection 
activities will be implemented:
    1. Key informant interviews with health system leaders, clinicians 
and staff; and
    2. compilation and coding of notes from ``implementation support'' 
meetings (``check-ins'') between an implementation facilitator and site 
champions who are implementing the products.

Brief Background on the Products To Be Implemented by LHSs in This 
Study

    AHRQ is funding the development of two products that are 
specifically intended to make the findings from EPC evidence reports 
more accessible and usable by health systems. These are the products 
that will be offered to LHSs for potential implementation during this 
project. They include a ``triage tool'' and a ``data visualization 
tool'' that have been designed to support LHS use of AHRQ evidence 
reports. The LHS triage tool presents high-level results of evidence 
reports that enable leaders within LHSs to quickly understand the 
relevance of the reports to their organization, share high-level 
information with key stakeholders (e.g., healthcare executives), and 
link to more granular data from the report. The data visualization tool 
presents data from the evidence review and individual studies in a 
dynamic, interactive website. The evaluation will capture the 
anticipated variation in how the LHS might use the products and the 
unique experience of LHSs.

Key Informant Interviews

    There will be two rounds of key informant interviews: (1) In-person 
preliminary interviews will be conducted early in the implementation 
period (months 1-3) with LHS leaders and clinicians and will focus on 
health systems' rationale for selecting each product and early 
experiences with its roll-out into practice; (2) remote follow-up 
interviews will be conducted via telephone later in the implementation 
period (months 10-11) with two sets of stakeholders: (a) LHS leaders 
and (b) clinicians/staff (hereafter, ``clinical staff'') actively 
implementing the product. These follow-up interviews will focus on 
health systems' experiences implementing their selected product(s). All 
interviews (preliminary and follow-up) will be 60 minutes in duration, 
recorded with permission of the key informants, and transcribed for 
analysis. Up to 88 total interviews will be conducted across the two 
rounds of key informant interviews. Assuming the same LHS leaders 
participate in the preliminary and follow-up interviews, the key 
informant interviews will involve 4-5 LHS leaders and clinical staff 
from each of the eleven LHSs implementing the study. Additional detail 
about the information collection components is provided below.

    1. In-person preliminary interviews. The preliminary interviews 
will include 2-3 LHS leaders/decisionmakers at each of eleven 
implementation sites for a maximum of 33 interviews in the first 
round of data collection. The interviews will be conducted during 
implementation site visits that are occurring early in the project 
to support the health systems' testing and/or roll out of the 
products into clinical workflows. Specific topics explored in the 
preliminary interviews include LHSs' decision to participate in 
implementation, decision considerations for the selected product, 
experiences leading the implementation, and early experiences and 
perceptions of the selected product(s). To limit respondent burden, 
we will use the implementation site visits as an opportunity for 
conducting the preliminary interviews, thereby limiting the need to 
schedule additional time with respondents for a phone interview. If 
a respondent has limited availability during the site visit, 
however, we may need to do the preliminary interview remotely or 
substitute the respondent with another qualified staff member who is 
available during the implementation site visit.
    2. Remote follow-up interviews. The follow-up interviews will 
include the 2-3 LHS leaders/decisionmakers from the preliminary 
interviews (maximim n = 33), along with 2 additional clinical staff 
(n = 22) at each of eleven implementation sites for a maximum of 55 
follow-up interviews. Specific topics explored in the follow-up 
interviews include LHS leaders' and clinical staff's experiences 
with each product as well as their perceptions of the relative 
advantage, acceptability/compatibility, appropriateness, and 
feasibility of using the product; implementation fidelity (i.e., if 
the implementation went as planned), reach, barriers and 
facilitators, and associated costs; any outcomes of implementing the 
product (e.g., achieved any intended systemic changes); and likely 
sustainability of continuing to use the product in practice.

    The two sets of in-depth qualitative interviews will allow for a 
nuanced exploration of both what LHSs value about the products and what 
it takes to successfully implement such tools into practice. The 
research on implementation and uptake of products to promote use of 
evidence in LHS settings is sparse, thus it is important to use a data 
collection strategy for the evaluation that will yield rich information 
about the experience of health systems, LHS decisionmakers, and the 
staff implementing the tools into practice. A quantitative survey would 
not yield the depth of individual

[[Page 6196]]

feedback that is needed to capture the experience of implementing these 
tools and the unique contexts of the health systems. Thus, interviews 
are the preferred method of systematically collecting these data.

Implementation Support Meetings/``Check-ins''

    In addition to key informant interviews, which will be conducted 
only at the beginning and end of implementation, AHRQ will gather 
information throughout the implementation period by using monthly 
implementation support meetings between implementation facilitators and 
site champions as an ongoing opportunity to ask key questions about 
implementation progress. Although the primary goal of these check-in 
meetings is to provide technical assistance with implementation and 
recommendations for handling emergent challenges in the implementation 
process, they will also be a source of rich information for the 
evaluation. Because these meetings occur in real time as the 
implementation unfolds, they will reduce the potential biases (e.g., 
selective memory, recency effects, forgetting details about key events 
and their sequence) associated with only collecting data at the 
beginning or end of the implementation period.
    These check-in meetings will occur by telephone and are intended to 
monitor implementation progress, provide support to health systems, and 
discuss next steps. AIR implementation facilitators for each site will 
schedule telephone conference calls with site champions (N = 11), 
during which structured notes will be taken. These notes will be 
supplemented with relevant information from other touchpoints between 
the facilitators and champions (e.g., ad hoc calls, email exchanges, 
and voluntary participation in monthly shared learning events) as they 
naturally occur. Notetakers will capture and document information 
related to key implementation domains as these topics arise in check-in 
meetings and other facilitator/champion encounters throughout 
implementation.

Estimated Annual Respondent Burden

    Exhibit 1 shows the total estimated annualized burden of 214.5 
hours for the two rounds of key informant interviews and implementation 
``check-ins'' combined. For the key informant interviews (totaling 154 
hours), burden is included for: (1) LHS leaders/decisionmakers 
participating in the preliminary interviews (a maximum of 33 hours), 
(2) LHS leaders/decisionmakers participating in the follow-up 
interviews (a maximum of 33 hours), (3) clinical staff participating in 
the follow-up interviews (a maximum of 22 hours), (4) interviewee 
review of materials, consent forms, and logistics in advance of their 
respective interviews (i.e., 16.5 + 5.5 = 22 hours) and (5) time for 
designated LHS staff (e.g., the LHS member, a designated site liaison, 
selected interviewees) to recommend key informants, coordinate 
implementation support, and help with scheduling of in-person 
preliminary interviews and remote follow-up interviews (44 hours). Also 
included in Exhibit 1 is the estimated annualized burden hours for 
monthly check-ins between implementation facilitators and LHS champions 
for informal technical assistance support and the quick status probes 
on implementation progress (a maximum of 60.5 hours). These annualized 
burden estimates for the key informant interviews and the coaching 
sessions are further explained below.

Key Informant Interviews: Expanded Detail on Burden Estimates

    We estimate 1 hour for each key informant interview for: (1) LHS 
leaders/decisionmakers participating in the preliminary interviews (a 
maximum of 33 hours), (2) LHS leaders/decisionmakers participating in 
the follow-up interviews (a maximum of 33 hours), (3) clinical staff 
participating in the follow-up interviews (a maximum of 22 hours), 
(Total interview burden = 1.00 hour x maximum of 88 interviews = 88 
hours). We estimate an additional 15 minutes (0.25 hours) will be 
needed for key informants to prepare for their respective interview(s) 
(Total interview preparation burden = 0.25 hours x maximum of 88 
interviews = 22 hours; of which 16.5 hours is for leaders/
decisionmakers to prepare for both preliminary and follow-up interviews 
and 5.5 is for clinical staff to prepare for their participation in the 
follow-up interviews only). Finally we estimate time for LHS leaders 
and staff to identify interview candidates, facilitate recruitment, 
coordinate implementation support, and assist with interview scheduling 
(4.00 hours per each of 11 LHSs; Total staff assistance burden = 4.00 
hours x 11 sites = 44 hours). The ``staff assistance'' burden involves 
the following:
     In each of the eleven LHS organizations implementing the 
product(s), the LHS member (and/or site liaison/champion) will identify 
prospective key informants (i.e., other LHS leaders/decisionmakers and 
appropriate clinical staff), with additional key informants 
subsequently identified through snowball sampling.
     Designated LHS staff (i.e., LHS member, designee and/or 
site liaison/champion) will provide needed contact information to the 
AIR evaluation team for outreach and recruitment of the prospective key 
informant interview candidates, assist with interview scheduling, and 
coordinate implementation support with the AIR team.
    We will develop standardized email messages to reach out to 
interview candidates and a written overview of the project, the 
evaluation, and the purpose of the interview. We will coordinate 
scheduling of both the implementation support check-ins and the 60-
minute interviews at the most convenient time, considering the needs of 
the LHS leadership and staff. For the preliminary interviews, if 
prospective interviewees are not available during our site visit, we 
will ask for suggestions of other LHS staff who meet our recruitment 
criteria or arrange a telephone interview, if needed.

Implementation Support Meetings/Check-Ins: Expanded Detail on Burden 
Estimates

    We estimate 60.5 hours for the monthly check-ins between 
implementation facilitators and LHS champions. This includes an average 
of 30 minutes of implementation support/check-in meetings per each of 
the 11 LHSs for each month of implementation (11 months). (11 months x 
0.5 hours = 5.5 hours). Across LHSs, the estimated burden associated 
with check-ins is approximately 61 hours across the implementation 
period (5.5 hours x 11 LHSs = 60.5 hours).

                                  Exhibit 1--Estimated Annualized Burden Hours
----------------------------------------------------------------------------------------------------------------
                                                                     Number of
                    Form name                        Number of     responses per     Hours per     Total burden
                                                   respondents *    respondent       response          hours
----------------------------------------------------------------------------------------------------------------
In-person preliminary interviews with LHS                  ** 33               1            1.00              33
 leaders/decisionmakers.........................

[[Page 6197]]

 
Remote follow-up interviews with LHS leaders/              ** 33               1            1.00              33
 decisionmakers.................................
Remote follow-up interviews with clinical staff.              22               1            1.00              22
Review of materials prior to BOTH preliminary                 33               2            0.25            16.5
 and follow-up interviews--LHS leaders/
 decisionmakers.................................
Review of materials prior to interviews--                     22               1            0.25             5.5
 clinical staff.................................
Interview scheduling and other staff assistance.              11               1            4.00              44
Implementation check-ins: Brief monthly                       11              11             0.5            60.5
 implementation progress checks, documented for
 the evaluation as structured notes on
 implementation topics naturally occurring in
 coach/champion encounters......................
rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
    Total.......................................  ..............  ..............  ..............       *** 214.5
----------------------------------------------------------------------------------------------------------------
* The numbers in this column give the maximum number of respondents for each listed activity based on a range in
  the number of recruits per site (e.g., ``2-3 LHS leaders/decisionmakers''). The balance may shift some between
  LHS leaders/decisionmakers and clinical staff depending on implementation team and leadership composition at
  each site. In any case, 88 interviews (33+33+22=88) is a maximum possible in the event each of the 11 sites
  contributes 3 ``LHS leaders/decsionmakers'' (likely the same people for preliminary and follow-up interviews)
  and 2 additional clinical staff (for follow-up interviews only) as key informants. It is more likely that the
  total number of interviews will be around 80.
** These are likely to be the same 33 respondents in both preliminary and follow-up interviews.
*** Total maximum burdened hours estimate based on maximum of 88 interviews.

    Costs associated with the estimated annualized burden hours are 
provided in Exhibit 2.

                                   Exhibit 2--Estimated Annualized Cost Burden
----------------------------------------------------------------------------------------------------------------
                                                                                      Average
                    Form name                        Number of     Total burden     hourly wage     Total cost
                                                   respondents *       hours          rate **         burden
----------------------------------------------------------------------------------------------------------------
In-person preliminary interviews with leaders/                33              33      \a\ $94.47       $3,117.51
 decisionmakers.................................
Remote follow-up interviews with leaders/                     33              33       \a\ 94.47        3,117.51
 decisionmakers.................................
Remote follow-up interviews with clinical staff.              22              22       \b\ 52.13        1,146.86
Review of materials prior to BOTH preliminary                 33            16.5       \a\ 94.47        1,558.76
 and follow-up interviews--LHS leaders/
 decisionmakers.................................
Review of materials prior to interviews--                     22             5.5       \b\ 52.13          286.72
 clinical staff.................................
Interview scheduling and other staff assistance               11              44       \c\ 20.34          894.96
 \c\............................................
Implementation check-ins (documented for the                  11            60.5       \a\ 94.47        5,715.44
 evaluation as structured notes on
 implementation progress).......................
                                                 ---------------------------------------------------------------
    Total.......................................  ..............  ..............  ..............       15,837.76
----------------------------------------------------------------------------------------------------------------
* The numbers in this column give the maximum number of respondents for each listed activity based on a range in
  the number of recruits per site (e.g., ``2-3 LHS leaders/decisionmakers''). As noted in the comment to Exhibit
  1, the balance may shift some between LHS leaders/decisionmakers and clinical staff depending on
  implementation team and leadership composition at each site. In any case, 88 interviews (33+33+22=88) is a
  maximum possible.
** National Compensation Survey: Occupational wages in the United States May 2018 ``U.S. Department of Labor,
  Bureau of Labor Statistics.''
\a\ Based on the mean wages for Internists, General. 29-1063; annual salary of $196,490.
\b\ Based on the mean wages for Physician Assistants, 29-1071; annual salary of $108,430.
\c\ Based on the mean wages for Secretaries and Administrative Assistants, 43-6010; annual salary of $42,320.

Request for Comments

    In accordance with the Paperwork Reduction Act, comments on AHRQ's 
information collection are requested with regard to any of the 
following: (a) Whether the proposed collection of information is 
necessary for the proper performance of AHRQ's health care research and 
health care information dissemination functions, including whether the 
information will have practical utility; (b) the accuracy of AHRQ's 
estimate of burden (including hours and costs) of the proposed 
collection(s) of information; (c) ways to enhance the quality, utility 
and clarity of the information to be collected; and (d) ways to 
minimize the burden of the collection of information upon the 
respondents, including the use of automated collection techniques or 
other forms of information technology.
    Comments submitted in response to this notice will be summarized 
and included in the Agency's subsequent request for OMB approval of the 
proposed information collection. All comments will become a matter of 
public record.

    Dated: January 29, 2020.
Virginia L. Mackay-Smith,
Associate Director.
[FR Doc. 2020-02057 Filed 2-3-20; 8:45 am]
 BILLING CODE 4160-90-P