Agency Information Collection Activities: Proposed Collection; Comment Request, 20497-20500 [2020-07664]
Download as PDF
<![CDATA[
Federal Register / Vol. 85, No. 71 / Monday, April 13, 2020 / Notices
ACTION:
Formations of, Acquisitions by, and
Mergers of Bank Holding Companies
SUMMARY:
The companies listed in this notice
have applied to the Board for approval,
pursuant to the Bank Holding Company
Act of 1956 (12 U.S.C. 1841 et seq.)
(BHC Act), Regulation Y (12 CFR part
225), and all other applicable statutes
and regulations to become a bank
holding company and/or to acquire the
assets or the ownership of, control of, or
the power to vote shares of a bank or
bank holding company and all of the
banks and nonbanking companies
owned by the bank holding company,
including the companies listed below.
The applications listed below, as well
as other related filings required by the
Board, if any, are available for
immediate inspection at the Federal
Reserve Bank indicated. The
applications will also be available for
inspection at the offices of the Board of
Governors. Interested persons may
express their views in writing on the
standards enumerated in the BHC Act
(12 U.S.C. 1842(c)).
Comments regarding each of these
applications must be received at the
Reserve Bank indicated or the offices of
the Board of Governors, Ann E.
Misback, Secretary of the Board, 20th
Street and Constitution Avenue NW,
Washington DC 20551–0001, not later
than May 12, 2020.
A. Federal Reserve Bank of San
Francisco (Sebastian Astrada, Director,
Applications) 101 Market Street, San
Francisco, California 94105–1579:
1. GUVJEC Investment Corporation,
Baltimore, Maryland; to become a bank
holding company by acquiring
Farmington Bancorp, Bothell,
Washington, and thereby indirectly
acquire Farmington State Bank,
Farmington, Washington.
Board of Governors of the Federal Reserve
System, April 7, 2020.
Yao-Chin Chao,
Assistant Secretary of the Board.
[FR Doc. 2020–07639 Filed 4–10–20; 8:45 am]
BILLING CODE P
DEPARTMENT OF HEALTH AND
HUMAN SERVICES
khammond on DSKJM1Z7X2PROD with NOTICES
Notice.
FEDERAL RESERVE SYSTEM
Agency for Healthcare Research and
Quality
Agency Information Collection
Activities: Proposed Collection;
Comment Request
Agency for Healthcare Research
and Quality, HHS.
AGENCY:
VerDate Sep<11>2014
17:57 Apr 10, 2020
Jkt 250001
This notice announces the
intention of the Agency for Healthcare
Research and Quality (AHRQ) to request
that the Office of Management and
Budget (OMB) approve the proposed
information collection project
‘‘Evaluating the Implementation of
Products by AHRQ’s Learning Health
Systems to Inform and Encourage Use of
AHRQ Evidence Reports.’’ This
proposed information collection was
previously published in the Federal
Register on February 4, 2020 and
allowed 60 days for public comment.
AHRQ did not receive comments from
members of the public. The purpose of
this notice is to allow an additional 30
days for public comment.
DATES: Comments on this notice must be
received by 30 days after date of
publication of this notice.
ADDRESSES: Written comments and
recommendations for the proposed
information collection should be sent
within 30 days of publication of this
notice to www.reginfo.gov/public/do/
PRAMain. Find this particular
information collection by selecting
‘‘Currently under 30-day Review—Open
for Public Comments’’ or by using the
search function.
FOR FURTHER INFORMATION CONTACT:
Doris Lefkowitz, AHRQ Reports
Clearance Officer, (301) 427–1477, or by
email at doris.lefkowitz@AHRQ.hhs.gov.
SUPPLEMENTARY INFORMATION:
Proposed Project
Evaluating the Implementation of
Products by Learning Health Systems To
Inform and Encourage Use of AHRQ
Evidence Reports
AHRQ’s Evidence-based Practice
Center (EPC) Program has 20 years of
experience in synthesizing research to
inform evidence-based health care
practice, delivery, policies, and
research. The AHRQ EPC program is
committed to partnering with
organizations to make sure its evidence
reports can be used in practice.
Historically, most of its evidence reports
have been used by clinical professional
organizations to support the
development of clinical practice
guidelines or Federal agencies to inform
their program planning and research
priorities. To improve the uptake and
relevance of the AHRQ EPC’s evidence
reports, specifically for health systems,
AHRQ has contracted with the
American Institutes for Research (AIR)
to obtain feedback from learning health
systems (LHSs) to assist the AHRQ EPC
program in developing and
disseminating evidence reports that can
PO 00000
Frm 00031
Fmt 4703
Sfmt 4703
20497
be used to improve the quality and
effectiveness of patient care.
Even if an EPC evidence report topic
addresses LHS-specific evidence needs,
the density of the information in an
evidence report may preclude its easy
review by busy LHS leaders and
decisionmakers. AHRQ understands
that to facilitate use by LHSs, complex
evidence reports must be translated into
a format that promotes LHS evidencebased decision making and can be
contextualized within each LHS’ own
system-generated evidence. Such
translational products, for the purposes
of this notice, are referred to simply as
‘‘products.’’
The purpose of this information
collection is to support a process
evaluation of use and implementation of
two such products into LHS
decisionmaking processes, workflows,
and clinical care. The evaluation has the
following goals:
1. Document how LHSs prioritize
filling evidence gaps, make decisions
about using evidence, and implement
tools to support and promote evidence
use in clinical care.
2. Assess the contextual factors that
may influence implementation success;
associated implementation resources,
barriers and facilitators; and satisfaction
of LHS leaders and clinical staff.
3. Provide the AHRQ EPC program
with necessary insights about the
perspectives, needs, and preferences of
LHS leaders and clinical staff as related
to decisions and implementation of
products into practice.
This study is being conducted by
AHRQ through its contractor, the
American Institutes for Research (AIR),
pursuant to AHRQ’s statutory authority
to conduct and support research on, and
disseminate information on, health care
and on systems for the delivery of such
care, including activities with respect to
the quality, effectiveness, efficiency,
appropriateness and value of healthcare
services. 42 U.S.C. 299a(a)(1).
Method of Collection
To achieve the goals of this project,
the following data collection activities
will be implemented:
1. Key informant interviews with
health system leaders, clinicians and
staff; and
2. compilation and coding of notes
from ‘‘implementation support’’
meetings (‘‘check-ins’’) between an
implementation facilitator and site
champions who are implementing the
products.
E:\FR\FM\13APN1.SGM
13APN1
20498
Federal Register / Vol. 85, No. 71 / Monday, April 13, 2020 / Notices
Brief Background on the Products To Be
Implemented by LHSs in This Study
AHRQ is funding the development of
two products that are specifically
intended to make the findings from EPC
evidence reports more accessible and
usable by health systems. These are the
products that will be offered to LHSs for
potential implementation during this
project. They include a ‘‘triage tool’’ and
a ‘‘data visualization tool’’ that have
been designed to support LHS use of
AHRQ evidence reports. The LHS triage
tool presents high-level results of
evidence reports that enable leaders
within LHSs to quickly understand the
relevance of the reports to their
organization, share high-level
information with key stakeholders (e.g.,
healthcare executives), and link to more
granular data from the report. The data
visualization tool presents data from the
evidence review and individual studies
in a dynamic, interactive website. The
evaluation will capture the anticipated
variation in how the LHS might use the
products and the unique experience of
LHSs.
khammond on DSKJM1Z7X2PROD with NOTICES
Key Informant Interviews
There will be two rounds of key
informant interviews: (1) In-person
preliminary interviews will be
conducted early in the implementation
period (months 1–3) with LHS leaders
and clinicians and will focus on health
systems’ rationale for selecting each
product and early experiences with its
roll-out into practice; (2) remote followup interviews will be conducted via
telephone later in the implementation
period (months 10–11) with two sets of
stakeholders: (a) LHS leaders and (b)
clinicians/staff (hereafter, ‘‘clinical
staff’’) actively implementing the
product. These follow-up interviews
will focus on health systems’
experiences implementing their selected
product(s). All interviews (preliminary
and follow-up) will be 60-minutes in
duration, recorded with permission of
the key informants, and transcribed for
analysis. Up to 88 total interviews will
be conducted across the two rounds of
key informant interviews. Assuming the
same LHS leaders participate in the
preliminary and follow-up interviews,
the key informant interviews will
involve 4–5 LHS leaders and clinical
staff from each of the eleven LHSs
implementing the study. Additional
detail about the information collection
components is provided below.
1. In-person preliminary interviews.
The preliminary interviews will include
2–3 LHS leaders/decisionmakers at each
of eleven implementation sites for a
maximum of 33 interviews in the first
VerDate Sep<11>2014
17:57 Apr 10, 2020
Jkt 250001
round of data collection. The interviews
will be conducted during
implementation site visits that are
occurring early in the project to support
the health systems’ testing and/or roll
out of the products into clinical
workflows. Specific topics explored in
the preliminary interviews include
LHSs’ decision to participate in
implementation, decision
considerations for the selected product,
experiences leading the
implementation, and early experiences
and perceptions of the selected
product(s). To limit respondent burden,
we will use the implementation site
visits as an opportunity for conducting
the preliminary interviews, thereby
limiting the need to schedule additional
time with respondents for a phone
interview. If a respondent has limited
availability during the site visit,
however, we may need to do the
preliminary interview remotely or
substitute the respondent with another
qualified staff member who is available
during the implementation site visit.
2. Remote follow-up interviews. The
follow-up interviews will include the 2–
3 LHS leaders/decisionmakers from the
preliminary interviews (maximim n =
33), along with 2 additional clinical staff
(n = 22) at each of eleven
implementation sites for a maximum of
55 follow-up interviews. Specific topics
explored in the follow-up interviews
include LHS leaders’ and clinical staff’s
experiences with each product as well
as their perceptions of the relative
advantage, acceptability/compatibility,
appropriateness, and feasibility of using
the product; implementation fidelity
(i.e., if the implementation went as
planned), reach, barriers and facilitators,
and associated costs; any outcomes of
implementing the product (e.g.,
achieved any intended systemic
changes); and likely sustainability of
continuing to use the product in
practice.
The two sets of in-depth qualitative
interviews will allow for a nuanced
exploration of both what LHSs value
about the products and what it takes to
successfully implement such tools into
practice. The research on
implementation and uptake of products
to promote use of evidence in LHS
settings is sparse, thus it is important to
use a data collection strategy for the
evaluation that will yield rich
information about the experience of
health systems, LHS decisionmakers,
and the staff implementing the tools
into practice. A quantitative survey
would not yield the depth of individual
feedback that is needed to capture the
experience of implementing these tools
and the unique contexts of the health
PO 00000
Frm 00032
Fmt 4703
Sfmt 4703
systems. Thus, interviews are the
preferred method of systematically
collecting this data.
Implementation Support Meetings/
‘‘Check-Ins’’
In addition to key informant
interviews, which will be conducted
only at the beginning and end of
implementation, AHRQ will gather
information throughout the
implementation period by using
monthly implementation support
meetings between implementation
facilitators and site champions as an
ongoing opportunity to ask key
questions about implementation
progress. Although the primary goal of
these check-in meetings is to provide
technical assistance with
implementation and recommendations
for handling emergent challenges in the
implementation process, they will also
be a source of rich information for the
evaluation. Because these meetings
occur in real time as the implementation
unfolds, they will reduce the potential
biases (e.g., selective memory, recency
effects, forgetting details about key
events and their sequence) associated
with only collecting data at the
beginning or end of the implementation
period.
These check-in meetings will occur by
telephone and are intended to monitor
implementation progress, provide
support to health systems, and discuss
next steps. AIR implementation
facilitators for each site will schedule
telephone conference calls with site
champions (N = 11), during which
structured notes will be taken. These
notes will be supplemented with
relevant information from other
touchpoints between the facilitators and
champions (e.g., ad hoc calls, email
exchanges, and voluntary participation
in monthly shared learning events) as
they naturally occur. Notetakers will
capture and document information
related to key implementation domains
as these topics arise in check-in
meetings and other facilitator/champion
encounters throughout implementation.
Estimated Annual Respondent Burden
Exhibit 1 shows the total estimated
annualized burden of 214.5 hours for
the two rounds of key informant
interviews and implementation ‘‘checkins’’ combined. For the key informant
interviews (totaling 154 hours), burden
is included for: (1) LHS leaders/
decisionmakers participating in the
preliminary interviews (a maximum of
33 hours), (2) LHS leaders/
decisionmakers participating in the
follow-up interviews (a maximum of 33
hours), (3) clinical staff participating in
E:\FR\FM\13APN1.SGM
13APN1
20499
Federal Register / Vol. 85, No. 71 / Monday, April 13, 2020 / Notices
the follow-up interviews (a maximum of
22 hours), (4) interviewee review of
materials, consent forms, and logistics
in advance of their respective interviews
(i.e., 16.5 + 5.5 = 22 hours) and (5) time
for designated LHS staff (e.g., the LHS
member, a designated site liaison,
selected interviewees) to recommend
key informants, coordinate
implementation support, and help with
scheduling of in-person preliminary
interviews and remote follow-up
interviews (44 hours). Also included in
Exhibit 1 is the estimated annualized
burden hours for monthly check-ins
between implementation facilitators and
LHS champions for informal technical
assistance support and the quick status
probes on implementation progress (a
maximum of 60.5 hours). These
annualized burden estimates for the key
informant interviews and the coaching
sessions are further explained below.
Key Informant Interviews: Expanded
Detail on Burden Estimates
We estimate 1 hour for each key
informant interview for: (1) LHS
leaders/decisionmakers participating in
the preliminary interviews (a maximum
of 33 hours), (2) LHS leaders/
decisionmakers participating in the
follow-up interviews (a maximum of 33
hours), (3) clinical staff participating in
the follow-up interviews (a maximum of
22 hours), (Total interview burden =
1.00 hour × maximum of 88 interviews
= 88 hours). We estimate an additional
15 minutes (0.25 hours) will be needed
for key informants to prepare for their
respective interview(s) (Total interview
preparation burden = 0.25 hours ×
maximum of 88 interviews = 22 hours;
of which 16.5 hours is for leaders/
decisionmakers to prepare for both
preliminary and follow-up interviews
and 5.5 is for clinical staff to prepare for
their participation in the follow-up
interviews only). Finally we estimate
time for LHS leaders and staff to
identify interview candidates, facilitate
recruitment, coordinate implementation
support, and assist with interview
scheduling (4.00 hours per each of 11
LHSs; Total staff assistance burden =
4.00 hours × 11 sites = 44 hours). The
‘‘staff assistance’’ burden involves the
following:
• In each of the eleven LHS
organizations implementing the
product(s), the LHS member (and/or site
liaison/champion) will identify
prospective key informants (i.e., other
LHS leaders/decisionmakers and
appropriate clinical staff), with
additional key informants subsequently
identified through snowball sampling.
• Designated LHS staff (i.e., LHS
member, designee and/or site liaison/
champion) will provide needed contact
information to the AIR evaluation team
for outreach and recruitment of the
prospective key informant interview
candidates, assist with interview
scheduling, and coordinate
implementation support with the AIR
team.
We will develop standardized email
messages to reach out to interview
candidates and a written overview of
the project, the evaluation, and the
purpose of the interview. We will
coordinate scheduling of both the
implementation support check-ins and
the 60-minute interviews at the most
convenient time, considering the needs
of the LHS leadership and staff. For the
preliminary interviews, if prospective
interviewees are not available during
our site visit, we will ask for suggestions
of other LHS staff who meet our
recruitment criteria or arrange a
telephone interview, if needed.
Implementation Support Meetings/
Check-Ins: Expanded Detail on Burden
Estimates
We estimate 60.5 hours for the
monthly check-ins between
implementation facilitators and LHS
champions. This includes an average of
30 minutes of implementation support/
check-in meetings per each of the 11
LHSs for each month of implementation
(11 months). (11 months × 0.5 hours =
5.5 hours). Across LHSs, the estimated
burden associated with check-ins is
approximately 61 hours across the
implementation period (5.5 hours × 11
LHSs = 60.5 hours).
EXHIBIT 1—ESTIMATED ANNUALIZED BURDEN HOURS
Number of
respondents *
Form name
Number of
responses per
respondent
Hours per
response
Total burden
hours
In-person preliminary interviews with LHS leaders/decisionmakers ...............
Remote follow-up interviews with LHS leaders/decisionmakers .....................
Remote follow-up interviews with clinical staff ................................................
Review of materials prior to BOTH preliminary and follow-up interviews—
LHS leaders/decisionmakers .......................................................................
Review of materials prior to interviews—clinical staff .....................................
Interview scheduling and other staff assistance ..............................................
Implementation check-ins: Brief monthly implementation progress checks,
documented for the evaluation as structured notes on implementation
topics naturally occurring in coach/champion encounters ...........................
** 33
** 33
22
1
1
1
1.00
1.00
1.00
33
33
22
33
22
11
2
1
1
0.25
0.25
4.00
16.5
5.5
44
11
11
0.5
60.5
Total ..........................................................................................................
165
........................
........................
*** 214.5
khammond on DSKJM1Z7X2PROD with NOTICES
* The numbers in this column give the maximum number of respondents for each listed activity based on a range in the number of recruits per
site (e.g., ‘‘2–3 LHS leaders/decisionmakers’’). The balance may shift some between LHS leaders/decisionmakers and clinical staff depending on
implementation team and leadership composition at each site. In any case, 88 interviews (33 + 33 + 22 = 88) is a maximum possible in the event
each of the 11 sites contributes 3 ‘‘LHS leaders/decisionmakers’’ (likely the same people for preliminary and follow-up interviews) and 2 additional clinical staff (for follow-up interviews only) as key informants. It is more likely that the total number of interviews will be around 80.
** These are likely to be the same 33 respondents in both preliminary and follow-up interviews.
*** Total maximum burdened hours estimate based on maximum of 88 interviews.
Costs associated with the estimated
annualized burden hours are provided
in Exhibit 2.
VerDate Sep<11>2014
17:57 Apr 10, 2020
Jkt 250001
PO 00000
Frm 00033
Fmt 4703
Sfmt 4703
E:\FR\FM\13APN1.SGM
13APN1
20500
Federal Register / Vol. 85, No. 71 / Monday, April 13, 2020 / Notices
EXHIBIT 2—ESTIMATED ANNUALIZED COST BURDEN
Number of
respondents *
Form name
In-person preliminary interviews with leaders/decisionmakers .......................
Remote follow-up interviews with leaders/decisionmakers .............................
Remote follow-up interviews with clinical staff ................................................
Review of materials prior to BOTH preliminary and follow-up interviews—
LHS leaders/decisionmakers .......................................................................
Review of materials prior to interviews—clinical staff .....................................
Interview scheduling and other staff assistance c ............................................
Implementation check-ins (documented for the evaluation as structured
notes on implementation progress) .............................................................
Total ..........................................................................................................
Total burden
hours
Average
hourly wage
rate **
Total cost
burden
33
33
22
33
33
22
a $94.47
33
22
11
16.5
5.5
44
a 94.47
c 20.34
1,558.76
286.72
894.96
11
60.5
a 94.47
5,715.44
165
........................
........................
15,837.76
a 94.47
b 52.13
b 52.13
$3,117.51
3,117.51
1,146.86
* The numbers in this column give the maximum number of respondents for each listed activity based on a range in the number of recruits per
site (e.g., ‘‘2–3 LHS leaders/decisionmakers’’). As noted in the comment to Exhibit 1, the balance may shift some between LHS leaders/decisionmakers and clinical staff depending on implementation team and leadership composition at each site. In any case, 88 interviews (33 + 33 + 22 =
88) is a maximum possible.
** National Compensation Survey: Occupational wages in the United States May 2018 ‘‘U.S. Department of Labor, Bureau of Labor Statistics.’’
a Based on the mean wages for Internists, General. 29–1063; annual salary of $196,490.
b Based on the mean wages for Physician Assistants, 29–1071; annual salary of $108,430.
c Based on the mean wages for Secretaries and Administrative Assistants, 43–6010; annual salary of $42,320.
Request for Comments
In accordance with the Paperwork
Reduction Act, comments on AHRQ’s
information collection are requested
with regard to any of the following: (a)
Whether the proposed collection of
information is necessary for the proper
performance of AHRQ’s health care
research and health care information
dissemination functions, including
whether the information will have
practical utility; (b) the accuracy of
AHRQ’s estimate of burden (including
hours and costs) of the proposed
collection(s) of information; (c) ways to
enhance the quality, utility and clarity
of the information to be collected; and
(d) ways to minimize the burden of the
collection of information upon the
respondents, including the use of
automated collection techniques or
other forms of information technology.
Comments submitted in response to
this notice will be summarized and
included in the Agency’s subsequent
request for OMB approval of the
proposed information collection. All
comments will become a matter of
public record.
Dated: April 7, 2020.
Virginia L. Mackay-Smith,
Associate Director.
khammond on DSKJM1Z7X2PROD with NOTICES
[FR Doc. 2020–07664 Filed 4–10–20; 8:45 am]
BILLING CODE 4160–90–P
VerDate Sep<11>2014
17:57 Apr 10, 2020
Jkt 250001
DEPARTMENT OF HEALTH AND
HUMAN SERVICES
Agency for Healthcare Research and
Quality
Agency Information Collection
Activities: Proposed Collection;
Comment Request
Agency for Healthcare Research
and Quality, HHS.
ACTION: Notice.
AGENCY:
This notice announces the
intention of the Agency for Healthcare
Research and Quality (AHRQ) to request
that the Office of Management and
Budget (OMB) approve the proposed
information collection project:
‘‘Consumer Assessment of Healthcare
Providers and Systems (CAHPS) Health
Plan Survey Database.’’ In accordance
with the Paperwork Reduction Act of
1995, AHRQ invites the public to
comment on this proposed information
collection. This proposed information
collection was previously published in
the Federal Register on January 28,
2020 and allowed 60 days for public
comment. AHRQ did not receive
comments from members of the public.
The purpose of this notice is to allow an
additional 30 days for public comment.
DATES: Comments on this notice must be
received by 30 days after date of
publication of this notice.
ADDRESSES: Written comments and
recommendations for the proposed
information collection should be sent
within 30 days of publication of this
notice to www.reginfo.gov/public/do/
PRAMain. Find this particular
information collection by selecting
‘‘Currently under 30-day Review—Open
SUMMARY:
PO 00000
Frm 00034
Fmt 4703
Sfmt 4703
for Public Comments’’ or by using the
search function.
FOR FURTHER INFORMATION CONTACT:
Doris Lefkowitz, AHRQ Reports
Clearance Officer, (301) 427–1477, or by
email at doris.lefkowitz@AHRQ.hhs.gov.
SUPPLEMENTARY INFORMATION:
Proposed Project
Consumer Assessment of Healthcare
Providers and Systems (CAHPS) Health
Plan Survey Database
AHRQ requests that OMB reapprove
AHRQ’s collection of information for
the AHRQ Consumer Assessment of
Healthcare Providers and Systems
(CAHPS) Health Plan Survey Database:
OMB Control number 0935–0165,
expiration May 31, 2020 (the CAHPS
Health Plan Database). The CAHPS
Health Plan Database consists of data
from the AHRQ CAHPS Health Plan
Survey. Health plans in the U.S. are
asked to voluntarily submit data from
the survey to AHRQ, through its
contractor, Westat. The CAHPS Health
Plan Database was developed by AHRQ
in 1998 in response to requests from
health plans, purchasers, and the
Centers for Medicare & Medicaid
Services (CMS) to provide comparative
data to support public reporting of
health plan ratings, health plan
accreditation and quality improvement.
This research has the following goals:
(1) To maintain the CAHPS Health
Plan Database using data from AHRQ’s
standardized CAHPS Health Plan
Survey to provide results to health care
purchasers, consumers, regulators and
policy makers across the country.
(2) To offer several products and
services, including aggregated results
presented through an Online Reporting
E:\FR\FM\13APN1.SGM
13APN1
]]>Agencies
[Federal Register Volume 85, Number 71 (Monday, April 13, 2020)]
[Notices]
[Pages 20497-20500]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2020-07664]
=======================================================================
-----------------------------------------------------------------------
DEPARTMENT OF HEALTH AND HUMAN SERVICES
Agency for Healthcare Research and Quality
Agency Information Collection Activities: Proposed Collection;
Comment Request
AGENCY: Agency for Healthcare Research and Quality, HHS.
ACTION: Notice.
-----------------------------------------------------------------------
SUMMARY: This notice announces the intention of the Agency for
Healthcare Research and Quality (AHRQ) to request that the Office of
Management and Budget (OMB) approve the proposed information collection
project ``Evaluating the Implementation of Products by AHRQ's Learning
Health Systems to Inform and Encourage Use of AHRQ Evidence Reports.''
This proposed information collection was previously published in the
Federal Register on February 4, 2020 and allowed 60 days for public
comment. AHRQ did not receive comments from members of the public. The
purpose of this notice is to allow an additional 30 days for public
comment.
DATES: Comments on this notice must be received by 30 days after date
of publication of this notice.
ADDRESSES: Written comments and recommendations for the proposed
information collection should be sent within 30 days of publication of
this notice to www.reginfo.gov/public/do/PRAMain. Find this particular
information collection by selecting ``Currently under 30-day Review--
Open for Public Comments'' or by using the search function.
FOR FURTHER INFORMATION CONTACT: Doris Lefkowitz, AHRQ Reports
Clearance Officer, (301) 427-1477, or by email at
[email protected].
SUPPLEMENTARY INFORMATION:
Proposed Project
Evaluating the Implementation of Products by Learning Health Systems To
Inform and Encourage Use of AHRQ Evidence Reports
AHRQ's Evidence-based Practice Center (EPC) Program has 20 years of
experience in synthesizing research to inform evidence-based health
care practice, delivery, policies, and research. The AHRQ EPC program
is committed to partnering with organizations to make sure its evidence
reports can be used in practice. Historically, most of its evidence
reports have been used by clinical professional organizations to
support the development of clinical practice guidelines or Federal
agencies to inform their program planning and research priorities. To
improve the uptake and relevance of the AHRQ EPC's evidence reports,
specifically for health systems, AHRQ has contracted with the American
Institutes for Research (AIR) to obtain feedback from learning health
systems (LHSs) to assist the AHRQ EPC program in developing and
disseminating evidence reports that can be used to improve the quality
and effectiveness of patient care.
Even if an EPC evidence report topic addresses LHS-specific
evidence needs, the density of the information in an evidence report
may preclude its easy review by busy LHS leaders and decisionmakers.
AHRQ understands that to facilitate use by LHSs, complex evidence
reports must be translated into a format that promotes LHS evidence-
based decision making and can be contextualized within each LHS' own
system-generated evidence. Such translational products, for the
purposes of this notice, are referred to simply as ``products.''
The purpose of this information collection is to support a process
evaluation of use and implementation of two such products into LHS
decisionmaking processes, workflows, and clinical care. The evaluation
has the following goals:
1. Document how LHSs prioritize filling evidence gaps, make
decisions about using evidence, and implement tools to support and
promote evidence use in clinical care.
2. Assess the contextual factors that may influence implementation
success; associated implementation resources, barriers and
facilitators; and satisfaction of LHS leaders and clinical staff.
3. Provide the AHRQ EPC program with necessary insights about the
perspectives, needs, and preferences of LHS leaders and clinical staff
as related to decisions and implementation of products into practice.
This study is being conducted by AHRQ through its contractor, the
American Institutes for Research (AIR), pursuant to AHRQ's statutory
authority to conduct and support research on, and disseminate
information on, health care and on systems for the delivery of such
care, including activities with respect to the quality, effectiveness,
efficiency, appropriateness and value of healthcare services. 42 U.S.C.
299a(a)(1).
Method of Collection
To achieve the goals of this project, the following data collection
activities will be implemented:
1. Key informant interviews with health system leaders, clinicians
and staff; and
2. compilation and coding of notes from ``implementation support''
meetings (``check-ins'') between an implementation facilitator and site
champions who are implementing the products.
[[Page 20498]]
Brief Background on the Products To Be Implemented by LHSs in This
Study
AHRQ is funding the development of two products that are
specifically intended to make the findings from EPC evidence reports
more accessible and usable by health systems. These are the products
that will be offered to LHSs for potential implementation during this
project. They include a ``triage tool'' and a ``data visualization
tool'' that have been designed to support LHS use of AHRQ evidence
reports. The LHS triage tool presents high-level results of evidence
reports that enable leaders within LHSs to quickly understand the
relevance of the reports to their organization, share high-level
information with key stakeholders (e.g., healthcare executives), and
link to more granular data from the report. The data visualization tool
presents data from the evidence review and individual studies in a
dynamic, interactive website. The evaluation will capture the
anticipated variation in how the LHS might use the products and the
unique experience of LHSs.
Key Informant Interviews
There will be two rounds of key informant interviews: (1) In-person
preliminary interviews will be conducted early in the implementation
period (months 1-3) with LHS leaders and clinicians and will focus on
health systems' rationale for selecting each product and early
experiences with its roll-out into practice; (2) remote follow-up
interviews will be conducted via telephone later in the implementation
period (months 10-11) with two sets of stakeholders: (a) LHS leaders
and (b) clinicians/staff (hereafter, ``clinical staff'') actively
implementing the product. These follow-up interviews will focus on
health systems' experiences implementing their selected product(s). All
interviews (preliminary and follow-up) will be 60-minutes in duration,
recorded with permission of the key informants, and transcribed for
analysis. Up to 88 total interviews will be conducted across the two
rounds of key informant interviews. Assuming the same LHS leaders
participate in the preliminary and follow-up interviews, the key
informant interviews will involve 4-5 LHS leaders and clinical staff
from each of the eleven LHSs implementing the study. Additional detail
about the information collection components is provided below.
1. In-person preliminary interviews. The preliminary interviews
will include 2-3 LHS leaders/decisionmakers at each of eleven
implementation sites for a maximum of 33 interviews in the first round
of data collection. The interviews will be conducted during
implementation site visits that are occurring early in the project to
support the health systems' testing and/or roll out of the products
into clinical workflows. Specific topics explored in the preliminary
interviews include LHSs' decision to participate in implementation,
decision considerations for the selected product, experiences leading
the implementation, and early experiences and perceptions of the
selected product(s). To limit respondent burden, we will use the
implementation site visits as an opportunity for conducting the
preliminary interviews, thereby limiting the need to schedule
additional time with respondents for a phone interview. If a respondent
has limited availability during the site visit, however, we may need to
do the preliminary interview remotely or substitute the respondent with
another qualified staff member who is available during the
implementation site visit.
2. Remote follow-up interviews. The follow-up interviews will
include the 2-3 LHS leaders/decisionmakers from the preliminary
interviews (maximim n = 33), along with 2 additional clinical staff (n
= 22) at each of eleven implementation sites for a maximum of 55
follow-up interviews. Specific topics explored in the follow-up
interviews include LHS leaders' and clinical staff's experiences with
each product as well as their perceptions of the relative advantage,
acceptability/compatibility, appropriateness, and feasibility of using
the product; implementation fidelity (i.e., if the implementation went
as planned), reach, barriers and facilitators, and associated costs;
any outcomes of implementing the product (e.g., achieved any intended
systemic changes); and likely sustainability of continuing to use the
product in practice.
The two sets of in-depth qualitative interviews will allow for a
nuanced exploration of both what LHSs value about the products and what
it takes to successfully implement such tools into practice. The
research on implementation and uptake of products to promote use of
evidence in LHS settings is sparse, thus it is important to use a data
collection strategy for the evaluation that will yield rich information
about the experience of health systems, LHS decisionmakers, and the
staff implementing the tools into practice. A quantitative survey would
not yield the depth of individual feedback that is needed to capture
the experience of implementing these tools and the unique contexts of
the health systems. Thus, interviews are the preferred method of
systematically collecting this data.
Implementation Support Meetings/``Check-Ins''
In addition to key informant interviews, which will be conducted
only at the beginning and end of implementation, AHRQ will gather
information throughout the implementation period by using monthly
implementation support meetings between implementation facilitators and
site champions as an ongoing opportunity to ask key questions about
implementation progress. Although the primary goal of these check-in
meetings is to provide technical assistance with implementation and
recommendations for handling emergent challenges in the implementation
process, they will also be a source of rich information for the
evaluation. Because these meetings occur in real time as the
implementation unfolds, they will reduce the potential biases (e.g.,
selective memory, recency effects, forgetting details about key events
and their sequence) associated with only collecting data at the
beginning or end of the implementation period.
These check-in meetings will occur by telephone and are intended to
monitor implementation progress, provide support to health systems, and
discuss next steps. AIR implementation facilitators for each site will
schedule telephone conference calls with site champions (N = 11),
during which structured notes will be taken. These notes will be
supplemented with relevant information from other touchpoints between
the facilitators and champions (e.g., ad hoc calls, email exchanges,
and voluntary participation in monthly shared learning events) as they
naturally occur. Notetakers will capture and document information
related to key implementation domains as these topics arise in check-in
meetings and other facilitator/champion encounters throughout
implementation.
Estimated Annual Respondent Burden
Exhibit 1 shows the total estimated annualized burden of 214.5
hours for the two rounds of key informant interviews and implementation
``check-ins'' combined. For the key informant interviews (totaling 154
hours), burden is included for: (1) LHS leaders/decisionmakers
participating in the preliminary interviews (a maximum of 33 hours),
(2) LHS leaders/decisionmakers participating in the follow-up
interviews (a maximum of 33 hours), (3) clinical staff participating in
[[Page 20499]]
the follow-up interviews (a maximum of 22 hours), (4) interviewee
review of materials, consent forms, and logistics in advance of their
respective interviews (i.e., 16.5 + 5.5 = 22 hours) and (5) time for
designated LHS staff (e.g., the LHS member, a designated site liaison,
selected interviewees) to recommend key informants, coordinate
implementation support, and help with scheduling of in-person
preliminary interviews and remote follow-up interviews (44 hours). Also
included in Exhibit 1 is the estimated annualized burden hours for
monthly check-ins between implementation facilitators and LHS champions
for informal technical assistance support and the quick status probes
on implementation progress (a maximum of 60.5 hours). These annualized
burden estimates for the key informant interviews and the coaching
sessions are further explained below.
Key Informant Interviews: Expanded Detail on Burden Estimates
We estimate 1 hour for each key informant interview for: (1) LHS
leaders/decisionmakers participating in the preliminary interviews (a
maximum of 33 hours), (2) LHS leaders/decisionmakers participating in
the follow-up interviews (a maximum of 33 hours), (3) clinical staff
participating in the follow-up interviews (a maximum of 22 hours),
(Total interview burden = 1.00 hour x maximum of 88 interviews = 88
hours). We estimate an additional 15 minutes (0.25 hours) will be
needed for key informants to prepare for their respective interview(s)
(Total interview preparation burden = 0.25 hours x maximum of 88
interviews = 22 hours; of which 16.5 hours is for leaders/
decisionmakers to prepare for both preliminary and follow-up interviews
and 5.5 is for clinical staff to prepare for their participation in the
follow-up interviews only). Finally we estimate time for LHS leaders
and staff to identify interview candidates, facilitate recruitment,
coordinate implementation support, and assist with interview scheduling
(4.00 hours per each of 11 LHSs; Total staff assistance burden = 4.00
hours x 11 sites = 44 hours). The ``staff assistance'' burden involves
the following:
In each of the eleven LHS organizations implementing the
product(s), the LHS member (and/or site liaison/champion) will identify
prospective key informants (i.e., other LHS leaders/decisionmakers and
appropriate clinical staff), with additional key informants
subsequently identified through snowball sampling.
Designated LHS staff (i.e., LHS member, designee and/or
site liaison/champion) will provide needed contact information to the
AIR evaluation team for outreach and recruitment of the prospective key
informant interview candidates, assist with interview scheduling, and
coordinate implementation support with the AIR team.
We will develop standardized email messages to reach out to
interview candidates and a written overview of the project, the
evaluation, and the purpose of the interview. We will coordinate
scheduling of both the implementation support check-ins and the 60-
minute interviews at the most convenient time, considering the needs of
the LHS leadership and staff. For the preliminary interviews, if
prospective interviewees are not available during our site visit, we
will ask for suggestions of other LHS staff who meet our recruitment
criteria or arrange a telephone interview, if needed.
Implementation Support Meetings/Check-Ins: Expanded Detail on Burden
Estimates
We estimate 60.5 hours for the monthly check-ins between
implementation facilitators and LHS champions. This includes an average
of 30 minutes of implementation support/check-in meetings per each of
the 11 LHSs for each month of implementation (11 months). (11 months x
0.5 hours = 5.5 hours). Across LHSs, the estimated burden associated
with check-ins is approximately 61 hours across the implementation
period (5.5 hours x 11 LHSs = 60.5 hours).
Exhibit 1--Estimated Annualized Burden Hours
----------------------------------------------------------------------------------------------------------------
Number of
Form name Number of responses per Hours per Total burden
respondents * respondent response hours
----------------------------------------------------------------------------------------------------------------
In-person preliminary interviews with LHS ** 33 1 1.00 33
leaders/decisionmakers.........................
Remote follow-up interviews with LHS leaders/ ** 33 1 1.00 33
decisionmakers.................................
Remote follow-up interviews with clinical staff. 22 1 1.00 22
Review of materials prior to BOTH preliminary 33 2 0.25 16.5
and follow-up interviews--LHS leaders/
decisionmakers.................................
Review of materials prior to interviews-- 22 1 0.25 5.5
clinical staff.................................
Interview scheduling and other staff assistance. 11 1 4.00 44
Implementation check-ins: Brief monthly 11 11 0.5 60.5
implementation progress checks, documented for
the evaluation as structured notes on
implementation topics naturally occurring in
coach/champion encounters......................
---------------------------------------------------------------
Total....................................... 165 .............. .............. *** 214.5
----------------------------------------------------------------------------------------------------------------
* The numbers in this column give the maximum number of respondents for each listed activity based on a range in
the number of recruits per site (e.g., ``2-3 LHS leaders/decisionmakers''). The balance may shift some between
LHS leaders/decisionmakers and clinical staff depending on implementation team and leadership composition at
each site. In any case, 88 interviews (33 + 33 + 22 = 88) is a maximum possible in the event each of the 11
sites contributes 3 ``LHS leaders/decisionmakers'' (likely the same people for preliminary and follow-up
interviews) and 2 additional clinical staff (for follow-up interviews only) as key informants. It is more
likely that the total number of interviews will be around 80.
** These are likely to be the same 33 respondents in both preliminary and follow-up interviews.
*** Total maximum burdened hours estimate based on maximum of 88 interviews.
Costs associated with the estimated annualized burden hours are
provided in Exhibit 2.
[[Page 20500]]
Exhibit 2--Estimated Annualized Cost Burden
----------------------------------------------------------------------------------------------------------------
Number of Total burden Average hourly Total cost
Form name respondents * hours wage rate ** burden
----------------------------------------------------------------------------------------------------------------
In-person preliminary interviews with leaders/ 33 33 \a\ $94.47 $3,117.51
decisionmakers.................................
Remote follow-up interviews with leaders/ 33 33 \a\ 94.47 3,117.51
decisionmakers.................................
Remote follow-up interviews with clinical staff. 22 22 \b\ 52.13 1,146.86
Review of materials prior to BOTH preliminary 33 16.5 \a\ 94.47 1,558.76
and follow-up interviews--LHS leaders/
decisionmakers.................................
Review of materials prior to interviews-- 22 5.5 \b\ 52.13 286.72
clinical staff.................................
Interview scheduling and other staff assistance 11 44 \c\ 20.34 894.96
\c\............................................
Implementation check-ins (documented for the 11 60.5 \a\ 94.47 5,715.44
evaluation as structured notes on
implementation progress).......................
---------------------------------------------------------------
Total....................................... 165 .............. .............. 15,837.76
----------------------------------------------------------------------------------------------------------------
* The numbers in this column give the maximum number of respondents for each listed activity based on a range in
the number of recruits per site (e.g., ``2-3 LHS leaders/decisionmakers''). As noted in the comment to Exhibit
1, the balance may shift some between LHS leaders/decisionmakers and clinical staff depending on
implementation team and leadership composition at each site. In any case, 88 interviews (33 + 33 + 22 = 88) is
a maximum possible.
** National Compensation Survey: Occupational wages in the United States May 2018 ``U.S. Department of Labor,
Bureau of Labor Statistics.''
\a\ Based on the mean wages for Internists, General. 29-1063; annual salary of $196,490.
\b\ Based on the mean wages for Physician Assistants, 29-1071; annual salary of $108,430.
\c\ Based on the mean wages for Secretaries and Administrative Assistants, 43-6010; annual salary of $42,320.
Request for Comments
In accordance with the Paperwork Reduction Act, comments on AHRQ's
information collection are requested with regard to any of the
following: (a) Whether the proposed collection of information is
necessary for the proper performance of AHRQ's health care research and
health care information dissemination functions, including whether the
information will have practical utility; (b) the accuracy of AHRQ's
estimate of burden (including hours and costs) of the proposed
collection(s) of information; (c) ways to enhance the quality, utility
and clarity of the information to be collected; and (d) ways to
minimize the burden of the collection of information upon the
respondents, including the use of automated collection techniques or
other forms of information technology.
Comments submitted in response to this notice will be summarized
and included in the Agency's subsequent request for OMB approval of the
proposed information collection. All comments will become a matter of
public record.
Dated: April 7, 2020.
Virginia L. Mackay-Smith,
Associate Director.
[FR Doc. 2020-07664 Filed 4-10-20; 8:45 am]
BILLING CODE 4160-90-P
