Agency Information Collection Activities: Proposed Collection; Comment Request, 74785-74788 [2011-30795]
Download as PDF
<![CDATA[
emcdonald on DSK5VPTVN1PROD with NOTICES
Federal Register / Vol. 76, No. 231 / Thursday, December 1, 2011 / Notices
higher per-foot dockage fee than other
vessels’’ is unlawfully discriminatory.
Complainant asserted this claim in the
United States District Court for the
District of Alaska, which court referred
the claim to the Commission upon
Respondent’s motion in that court
‘‘alleging that the dispute was within
the Commission’s primary jurisdiction.’’
Thus Complainant alleges that
Respondent has violated the Shipping
Act of 1984 ‘‘by unreasonably
prejudicing and disadvantaging Minto
and unreasonably preferring and
advantaging others in violation of 46
U.S.C. 41106(2), and by failing to
establish, observe, and enforce just and
reasonable regulations and practices
relating to or connected with receiving,
handling, storing, or delivering
property, in violation of 46 U.S.C.
41102. Complainant also presents its
state law discrimination claim at the
direction of the District Court.
Complainant requests that
Respondent be ordered ‘‘after due
hearing, to answer the charges herein, to
cease and desist from the aforesaid
violations of the Shipping Act, to
establish and put in force such practices
as the Commission determines to be
lawful and reasonable, and to pay Minto
reparations for PARN’s violations of the
Act, including the amount of the actual
injury, plus interest, costs and attorneys
fees, and any other damages to be
determined; and that the Commission
order any such other relief as it
determines proper.’’ The full text of the
complaint can be found in the
Commission’s Electronic Reading Room
at https://www.fmc.gov.
This proceeding has been assigned to
the Office of Administrative Law Judges.
Hearing in this matter, if any is held,
shall commence within the time
limitations prescribed in 46 CFR 502.61,
and only after consideration has been
given by the parties and the presiding
officer to the use of alternative forms of
dispute resolution. The hearing shall
include oral testimony and crossexamination in the discretion of the
presiding officer only upon proper
showing that there are genuine issues of
material fact that cannot be resolved on
the basis of sworn statements, affidavits,
depositions, or other documents or that
the nature of the matter in issue is such
that an oral hearing and crossexamination are necessary for the
development of an adequate record.
Pursuant to the further terms of 46 CFR
502.61, the initial decision of the
presiding officer in this proceeding shall
be issued by November 23, 2012 and the
VerDate Mar<15>2010
17:34 Nov 30, 2011
Jkt 226001
final decision of the Commission shall
be issued by March 25, 2013.
Karen V. Gregory,
Secretary.
[FR Doc. 2011–30895 Filed 11–30–11; 8:45 am]
BILLING CODE 6730–01–P
DEPARTMENT OF HEALTH AND
HUMAN SERVICES
Agency for Healthcare Research and
Quality
Agency Information Collection
Activities: Proposed Collection;
Comment Request
Agency for Healthcare Research
and Quality, HHS.
ACTION: Notice.
AGENCY:
This notice announces the
intention of the Agency for Healthcare
Research and Quality (AHRQ) to request
that the Office of Management and
Budget (OMB) approve the proposed
information collection project: ‘‘Use of
Deliberative Methods to Enhance Public
Engagement in the Agency for
Healthcare Research and Quality’s
(AHRQ’s) Effective Healthcare (EHC)
Program and Comparative Effectiveness
Research (CER) Enterprise.’’ In
accordance with the Paperwork
Reduction Act, 44 U.S.C. 3501–3521,
AHRQ invites the public to comment on
this proposed information collection.
DATES: Comments on this notice must be
received by January 30, 2012.
ADDRESSES: Written comments should
be submitted to: Doris Lefkowitz,
Reports Clearance Officer, AHRQ, by
email at doris.lefkowitz@AHRQ.hhs.gov.
Copies of the proposed collection
plans, data collection instruments, and
specific details on the estimated burden
can be obtained from the AHRQ Reports
Clearance Officer.
FOR FURTHER INFORMATION CONTACT:
Doris Lefkowitz, AHRQ Reports
Clearance Officer, (301) 427–1477, or by
email at dorislefkowitz@AHRQ.hhs.gov.
SUPPLEMENTARY INFORMATION:
SUMMARY:
Proposed Project
Use of Deliberative Methods To Enhance
Public Engagement in the Agency for
Healthcare Research and Quality’s
(AHRQ’s) Effective Healthcare (EHC)
Program and Comparative Effectiveness
Research (CER) Enterprise
With this project, AHRQ seeks
evidence on the feasibility and
usefulness of public deliberation as an
approach to obtaining public input on
questions related to the conduct and use
PO 00000
Frm 00015
Fmt 4703
Sfmt 4703
74785
of comparative effectiveness research
(CER). Although stakeholder
engagement has been central to the
Effective Healthcare (EHC) prop-am to
date, public input has not traditionally
been used to inform and guide broad
strategies related to the use of evidence
to inform decisions. This study would
provide a research base to address this
gap. This project closely ties to AHRQ’s
efforts to improve the rigor of methods,
as it will generate methodological
evidence through a randomized
controlled experiment comparing five
distinct methods of public deliberation
to find the most effective approaches for
involving the general public, including
members of AHRQ’s priority
populations, in questions related to the
research enterprise. Public deliberation
is a strategy for engaging lay people in
informing decisions when these
decisions require consideration of
values and ethics in addition to
scientific evidence. It includes three
core elements:
(1) Convening a group of people
(either in person or via online
technologies to connect people in
remote locations),
(2) Educating the participants on the
relevant issue(s) through dissemination
of educational materials and/or the use
of content experts, and
(3) Having the participants engage in
a reason-based discussion, or
deliberation, on all sides of the issue(s).
AHRQ wishes to study the
effectiveness of public deliberation,
because it offers the opportunity to
obtain public input on complex topics
in an environment that encourages
participants to educate themselves
about the topic and discuss it in a
thoughtful, respectful manner.
Information about the topic is
intentionally neutral and respectful of
the full range of underlying values and
experience with healthcare issues in the
population. This approach is designed
to improve upon the sometimes
superficial or ‘‘top of mind’’ responses
that are often provided by public
opinion surveys. AHRQ views public
deliberation as a potential source of
higher quality public input on issues
fundamental to the Agency’s mission,
such as the best and most effective ways
to use comparative effectiveness
research, than has heretofore been
available.
Several distinct deliberative methods
have been developed and used
previously. They share the three core
elements of public deliberation, but
differ on key features of implementation
such as duration, whether they take
place in-person or online, and the use
of content experts. Although there is
E:\FR\FM\01DEN1.SGM
01DEN1
74786
Federal Register / Vol. 76, No. 231 / Thursday, December 1, 2011 / Notices
emcdonald on DSK5VPTVN1PROD with NOTICES
considerable theoretical and case study
literature endorsing the value of public
deliberation, there has been little
empirical research about its
effectiveness and even less about the
comparative merits of different
deliberative methods (Community
Forum Deliberative Methods Literature
Review, 2010).
The objectives of this study are to:
1. Obtain informed and deliberated
input from lay people on important
questions underlying AHRQ’s research
program; and
2. Expand the evidence base for the
use of public deliberation methods for
exploring issues relevant to healthcare
research by comparing the outcomes of
five distinct deliberative methods to a
control condition and to each other.
This study is being conducted by
AHRQ through its contractor, the
American Institutes of Research (AIR),
pursuant to AHRQ’s statutory authority
to (1) promote healthcare quality
improvement by conducting and
supporting both research that develops
and presents scientific evidence
regarding all aspects of healthcare and
the synthesis and dissemination of
available scientific evidence for use by
policymakers, among others, and (2)
conduct and support research, provide
technical assistance, and disseminate
information on healthcare and on
systems for the delivery of such care.
See 42 U.S.C. 299(b)(1)(A), (D), (F), and
(G); 42 U.S.C. 299(b)(2); 42 U.S.C.
299a(a)(1)–(4).
Method of Collection
To achieve the objectives of this study
the following activities and data
collections will be implemented:
(1) Participant recruitment—A short
screening questionnaire, including a
brief overview of the study, will be used
to recruit persons for the study.
(2) Educational Materials—
Educational materials are designed to
inform participants about the topics that
are being deliberated and will be
provided to all 1,685 participants
recruited before the implementation of
any of the methods, but after the
administration of the Knowledge and
Attitudes Pre-test Survey (described
below). Additional content provided
during the deliberative method sessions
includes an overview of the study and
the background materials needed by
participants to competently deliberate
the issues. For two methods (ODP and
IDP; see below) educational materials to
be used during the sessions will be sent
to participants before the sessions (but
after administration of the pre-test).
(3) Deliberative Discussion Groups
and Control Group—The purpose of the
VerDate Mar<15>2010
17:34 Nov 30, 2011
Jkt 226001
discussion groups is to obtain informed
and deliberated input from lay people
on an important set of issues underlying
healthcare research. Participants will be
randomly assigned to one of the five
deliberative methods or a control
condition. The five methods were
selected because they have been
previously implemented and vary on
key features that may affect the
scalability and effectiveness of the
methods, including: duration (from two
hours to three days), mode of
implementation (online versus in
person), role of content experts, and
time between sessions allowing
participants to seek additional
information on the issues and
communicate informally with other
participants. The subject of the
deliberations is the use of research
evidence in healthcare decision-making.
This deliberative topic encompasses
several themes or ‘‘variations’’ that will
be elaborated in the deliberations:
1. Use of evidence to encourage better
healthcare: Is evidence useful (or, what
kind of evidence is useful) to a
physician and a patient who are
considering a test or treatment that has
been found to be ineffective, less
effective than another, riskier than
another, or for which effectiveness has
not been demonstrated?
2. Use of evidence to encourage better
value: Is evidence useful (or, what kind
of evidence is useful) to a physician and
a patient who are considering a test or
treatment that is effective even though
an equally effective but less expensive
alternative is available?
3. Decision-making when evidence
shows more complex trade-offs: Is
evidence useful (or, what kind of
evidence is useful) in treatment
decisions that involve the balancing of
effectiveness, risk, and value?
The issues involved in each variation
will be discussed in the context of
specific comparative effectiveness
research (CER) examples. These
‘‘vignettes’’ illustrate the issues and
elicit participants’ input on the issues
and the values employed by participants
in the deliberations.
(4) Knowledge and Attitudes Pre-test
Survey—This survey will measure
knowledge of and attitudes about the
health issues discussed in the
deliberations. It will be administered to
deliberation participants and controls
before educational materials are sent or
the methods are implemented.
As described, study participants will
be provided with educational materials
related to the deliberative topic. In order
to assess whether or not participants
were sufficiently informed on the topics
addressed in the materials, the
PO 00000
Frm 00016
Fmt 4703
Sfmt 4703
Knowledge and Attitudes Survey
contains items assessing knowledge of
medical research and medical evidence,
of comparative effectiveness research,
and of healthcare costs. The attitudinal
questions refer to the use of medical
evidence in healthcare decision making.
They include attitudes about health care
decision-making when research findings
can provide no support for, or conflict
with patient and doctor preferences for
particular treatments.
The questionnaire will also gather
demographic and other information
necessary to characterize the study
sample, test the success of the
randomization, and define population
subgroups for which variation in
outcomes will be examined. The
demographic variables also will be used
to control for participant and group
characteristics that may influence the
outcomes. Even though the design
involves randomization, and these
characteristics should be balanced
across groups, including them in the
statistical models guards against
inadequate results from randomization.
The variables to be measured in the
Knowledge and Attitudes Pre-test
Survey include:
• Sociodemographic characteristics:
Gender, age, marital status, education,
employment status, household
income, race/ethnicity, priority
population, languages spoken (in
addition to English)
• General health status
• Recent experience with the healthcare
system (e.g., seeing a healthcare
provider more than three times for the
same condition in the last 12 months)
• Health insurance coverage
• Health information-seeking behavior
(e.g., the extent to which people seek
healthcare information or rely on their
doctors to provide information)
(5) Knowledge and Attitudes Post-test
Survey—This survey will measure
knowledge of and attitudes about the
issues discussed in the deliberations
after the deliberations take place. It will
be administered to deliberation
participants and controls within one
week following conclusion of the
deliberative methods and will include
the same knowledge and attitude
questions as the pre-test questionnaire.
(6) Deliberative Experience Survey—
As described above, the five deliberative
methods being tested vary in terms of
duration, mode, use of educational
materials, and time between deliberative
sessions. A one-time survey will be
administered to participants in the
deliberative methods after
implementation of the experimental
conditions to compare deliberative
E:\FR\FM\01DEN1.SGM
01DEN1
74787
Federal Register / Vol. 76, No. 231 / Thursday, December 1, 2011 / Notices
methods to each other. Levels of
discourse quality and implementation
quality achieved will be assessed. Using
multi-item scales, the survey will
measure the following:
Discourse quality
• Equal participation in the
discussions
• Respect for others’ opinions and
tolerance of differing perspectives
• Appreciation of perspectives other
than their own
• Reasoned justification of ideas:
Sharing the reasoning or rationale
for positions, opinions, beliefs, or
preferences
Implementation quality
• Quality of group facilitation
• Quality. of the educational
materials provided
• Quality of the experts
• Transparency of the process and
use of the results
• Participants’ perceived value of
method
• Participants’ view of the influence
the results will have on programs
In sum, information collection in this
study will entail qualitative transcript
review and quantitative surveys. This
information will be used to describe and
summarize the input obtained from the
participants in the deliberative groups
concerning the use of evidence,
presenting the findings in reports for
AHRQ and the public.
The information from the surveys also
will be used to expand the evidence
base for public deliberation. The
experiment is designed to: (1) Compare
the effectiveness of the five deliberative
methods to the control condition and to
each other, (2) compare the quality of
the discourse achieved by the
deliberative methods to each other, (3)
assess the quality of implementation of
the five methods, and (4) test for
variation in effectiveness and discourse
quality by features of the deliberations
and for population subgroups defined
by sociodemogiaphic characteristics of
the participants.
Estimated Annual Respondent Burden
Exhibit 1 shows the estimated
annualized burden associated with the
respondents’ time to participate in this
research. The total annualized burden
hours are estimated to be 11,647 hours.
The burden estimate comprises the
following activities:
Participant Recruitment—The
screening questionnaire and recruitment
letter and materials will be sent to 1,685
participants. We estimate that it will
take 15 minutes to complete the
questionnaire and review the
recruitment letter and materials.
Educational materials—Educational
materials will be provided to all 1,685
participants recruited before the
implementation of any of the methods.
We estimate that it will take up to 1
hour to review the materials.
Short Citizens’ Deliberation (SCD):
This method will be tested with 192
participants (12 groups). Participants
will attend a single, 2-hour in-person
meeting.
Online Deliberative Polling® (ODP):
This method will be tested with 288
participants (24 groups) and will consist
of 4 online sessions over the course of
4 weeks; in total, this method will take
about 5 hours per person.
In-Person Deliberative Polling® (IDP):
This method will be tested with 288
participants (16 groups); participants
will attend a single in-person meeting,
lasting a full day.
Citizens’ Panel (CP): This method will
be tested with 96 participants (4
groups); participants will attend a 3-day,
in-person meeting.
Interrupted Deliberation (ID): This
method will be tested with 192
participants (12 groups). Participants
will attend 2 in-person meetings, lasting
3 hours each, a week apart. Between
meetings, participants will be asked to
access an online platform. In total, this
method will take about 6 hours per
person.
Knowledge and Attitudes Pre-test
Survey: This survey will be
administered to 1,685 participants and
will take an estimated 30 minutes to
complete.
Knowledge and Attitudes Post-test
Survey: This survey will be
administered to 1,685 participants and
will take an estimated 20 minutes to
complete.
Deliberative Experience Survey: This
survey will be administered to 1,056
deliberative methods participants at the
conclusion of the deliberative method. It
will take about 15 minutes to complete.
EXHIBIT 1—ESTIMATED ANNUALIZED BURDEN HOURS
Number of
responses per
respondent
Number of
respondents
Form name/Deliberative method
Hours per
response
Total burden
hours
Recruitment and Consent Materials ................................................
Short Citizens’ Deliberation (SCD) ..................................................
Online Deliberative Polling® (ODP) ................................................
In-Person Deliberative Polling® (IDP) .............................................
Citizens’ Panel .................................................................................
Interrupted Deliberation (ID) ............................................................
Educational Materials ......................................................................
Knowledge and Attitudes Pretest Survey ........................................
Knowledge and Attitudes Posttest Survey ......................................
Deliberative Experience Survey ......................................................
1685
192
288
288
96
192
1685
1685
1685
1056
1
1
1
1
1
1
1
1
1
1
15/60
2
5
9
24
6
1
30/60
20/60
15/60
421
384
1440
2592
2304
1152
1685
843
562
264
Total ..........................................................................................
8852
N/A
N/A
11647
emcdonald on DSK5VPTVN1PROD with NOTICES
EXHIBIT 2—ESTIMATED ANNUALIZED COST BURDEN
Number of
respondents
Form name/Deliberative method
Recruitment and Consent Materials ................................................
Short Citizens’ Deliberation (SCD) ..................................................
Online Deliberative Polling® (ODP) ................................................
In-Person Deliberative Polling® (IDP) .............................................
Citizens’ Panel .................................................................................
Interrupted Deliberation (ID) ............................................................
VerDate Mar<15>2010
17:34 Nov 30, 2011
Jkt 226001
PO 00000
Frm 00017
Fmt 4703
Total burden
hours
1685
192
288
288
96
192
Sfmt 4703
Average hourly
wage rate
421
384
1440
2592
2304
1152
E:\FR\FM\01DEN1.SGM
$21.35
21.35
21.35
21.35
21.35
21.35
01DEN1
Total cost burden
$8,988
8,198
30,744
55,339
49,190
24,595
74788
Federal Register / Vol. 76, No. 231 / Thursday, December 1, 2011 / Notices
EXHIBIT 2—ESTIMATED ANNUALIZED COST BURDEN—Continued
Number of
respondents
Form name/Deliberative method
Total burden
hours
Average hourly
wage rate
Total cost burden
Educational Materials ......................................................................
Knowledge and Attitudes Pretest Survey ........................................
Knowledge and Attitudes Post-test Survey .....................................
Deliberative Experience Survey ......................................................
1685
1685
1685
1056
1685
843
562
264
21.35
$21.35
21.35
21.35
35,975
$17,998
11,999
5,636
Total ..........................................................................................
8852
N/A
N/A
248,662
* Based upon the mean of the wages for 00–000 All Occupations ($21.35), May 2010 National Occupational Employment and Wage Estimates.
United States, ‘‘U.S. Department of Labor, Bureau of Labor Statistics.’’ https://www.bls.gov/oes/current/oes_nat.htm#00–0000.
Estimated Annual Costs to the Federal
Government
Exhibit 3 below breaks down the costs
related to this study. These are the costs
associated with the portion of the
contract awarded to AIR to conduct the
experiment. Since the implementation
and evaluation periods will span 24
months, the costs have been annualized
by taking the total cost and dividing by
2.
EXHIBIT 3—ESTIMATED ANNUALIZED COST TO THE FEDERAL GOVERNMENT
Cost component
Total cost
Project Management ........................................................................................................................................
Technical Expert Panel ....................................................................................................................................
Technology Tools ............................................................................................................................................
Develop Educational Materials ........................................................................................................................
Evaluation Plan ................................................................................................................................................
Implement Methods .........................................................................................................................................
Conceptual Framework ....................................................................................................................................
Data Processing and Analysis .........................................................................................................................
Reporting .........................................................................................................................................................
Overhead .........................................................................................................................................................
Total ..........................................................................................................................................................
emcdonald on DSK5VPTVN1PROD with NOTICES
Request for Comments
In accordance with the Paperwork
Reduction Act, comments on AHRQ’s
information collection are requested
with regard to any of the following: (a)
Whether the proposed collection of
information is necessary for the proper
performance of AHRQ healthcare
research and healthcare information
dissemination functions, including
whether the information will have
practical utility; (b) the accuracy of
AHRQ’s estimate of burden (including
hours and costs) of the proposed
collection(s) of information; (c) ways to
enhance the quality, utility, and clarity
of the information to be collected; and
(d) ways to minimize the burden of the
collection of information upon the
respondents, including the use of
automated collection techniques or
other forms of information technology.
Comments submitted in response to
this notice will be summarized and
included in the Agency’s subsequent
request for OMB approval of the
proposed information collection. All
comments will become a matter of
public record.
VerDate Mar<15>2010
17:34 Nov 30, 2011
Jkt 226001
Dated: November 16, 2011.
Carolyn Clancy,
Director.
[FR Doc. 2011–30795 Filed 11–30–11; 8:45 am]
BILLING CODE 4160–90–M
DEPARTMENT OF HEALTH AND
HUMAN SERVICES
Agency for Healthcare Research and
Quality
Annualized cost
$60,106
117,793
177,580
368,624
214,566
1,624,169
50,195
566,846
135,693
1,281,340
4,596,914
$30,053
58,896
88,790
184,312
107,283
812,085
25,098
283,423
67,847
640,670
2,298,457
Rule), 42 CFR part 3, authorizes AHRQ,
on behalf of the Secretary of HHS, to list
as a PSO an entity that attests that it
meets the statutory and regulatory
requirements for listing. A PSO can be
‘‘delisted’’ by the Secretary if it is found
to no longer meet the requirements of
the Patient Safety Act and patient Safety
Rule, including when a PSO chooses to
voluntarily relinquish its status as a
PSO for any reason.
Patient Safety Organizations:
Voluntary Relinquishment From
HealthWatch, Inc.
Agency for Healthcare Research
and Quality (AHRQ), HHS.
ACTION: Notice of delisting.
The directories for both listed
and delisted PSOs are ongoing and
reviewed weekly by AHRQ. The
delisting was effective at 12 Midnight
ET (2400) on November 1, 2011.
ADDRESSES:
DATES:
AGENCY:
AHRQ has accepted a
notification of voluntary relinquishment
from HealthWatch, Inc. of its status as
a Patient Safety Organization (PSO). The
Patient Safety and Quality Improvement
Act of 2005 (Patient Safety Act), Public
Law 109–41, 42 U.S.C. 299b–21—b–26,
provides for the formation of PSOs,
which collect, aggregate, and analyze
confidential information regarding the
quality and safety of health care
delivery. The Patient Safety and Quality
Improvement Final Rule (Patient Safety
SUMMARY:
PO 00000
Frm 00018
Fmt 4703
Sfmt 4703
Both directories can be
accessed electronically at the following
HHS Web site: https://
www.pso.AHRQ.gov/.
FOR FURTHER INFORMATION CONTACT:
Susan Grinder, Center for Quality
Improvement and Patient Safety, AHRQ,
540 Gaither Road, Rockville, MD 20850;
Telephone (toll free): (866) 403–3697;
Telephone (local): (301) 427–1111; TTY
(toll free): (866) 438–7231; TTY (local):
(301) 427–1130; Email:
pso@AHRQ.hhs.gov.
SUPPLEMENTARY INFORMATION:
E:\FR\FM\01DEN1.SGM
01DEN1
]]>Agencies
[Federal Register Volume 76, Number 231 (Thursday, December 1, 2011)]
[Notices]
[Pages 74785-74788]
From the Federal Register Online via the Government Printing Office [www.gpo.gov]
[FR Doc No: 2011-30795]
=======================================================================
-----------------------------------------------------------------------
DEPARTMENT OF HEALTH AND HUMAN SERVICES
Agency for Healthcare Research and Quality
Agency Information Collection Activities: Proposed Collection;
Comment Request
AGENCY: Agency for Healthcare Research and Quality, HHS.
ACTION: Notice.
-----------------------------------------------------------------------
SUMMARY: This notice announces the intention of the Agency for
Healthcare Research and Quality (AHRQ) to request that the Office of
Management and Budget (OMB) approve the proposed information collection
project: ``Use of Deliberative Methods to Enhance Public Engagement in
the Agency for Healthcare Research and Quality's (AHRQ's) Effective
Healthcare (EHC) Program and Comparative Effectiveness Research (CER)
Enterprise.'' In accordance with the Paperwork Reduction Act, 44 U.S.C.
3501-3521, AHRQ invites the public to comment on this proposed
information collection.
DATES: Comments on this notice must be received by January 30, 2012.
ADDRESSES: Written comments should be submitted to: Doris Lefkowitz,
Reports Clearance Officer, AHRQ, by email at
doris.lefkowitz@AHRQ.hhs.gov.
Copies of the proposed collection plans, data collection
instruments, and specific details on the estimated burden can be
obtained from the AHRQ Reports Clearance Officer.
FOR FURTHER INFORMATION CONTACT: Doris Lefkowitz, AHRQ Reports
Clearance Officer, (301) 427-1477, or by email at
dorislefkowitz@AHRQ.hhs.gov.
SUPPLEMENTARY INFORMATION:
Proposed Project
Use of Deliberative Methods To Enhance Public Engagement in the Agency
for Healthcare Research and Quality's (AHRQ's) Effective Healthcare
(EHC) Program and Comparative Effectiveness Research (CER) Enterprise
With this project, AHRQ seeks evidence on the feasibility and
usefulness of public deliberation as an approach to obtaining public
input on questions related to the conduct and use of comparative
effectiveness research (CER). Although stakeholder engagement has been
central to the Effective Healthcare (EHC) prop-am to date, public input
has not traditionally been used to inform and guide broad strategies
related to the use of evidence to inform decisions. This study would
provide a research base to address this gap. This project closely ties
to AHRQ's efforts to improve the rigor of methods, as it will generate
methodological evidence through a randomized controlled experiment
comparing five distinct methods of public deliberation to find the most
effective approaches for involving the general public, including
members of AHRQ's priority populations, in questions related to the
research enterprise. Public deliberation is a strategy for engaging lay
people in informing decisions when these decisions require
consideration of values and ethics in addition to scientific evidence.
It includes three core elements:
(1) Convening a group of people (either in person or via online
technologies to connect people in remote locations),
(2) Educating the participants on the relevant issue(s) through
dissemination of educational materials and/or the use of content
experts, and
(3) Having the participants engage in a reason-based discussion, or
deliberation, on all sides of the issue(s).
AHRQ wishes to study the effectiveness of public deliberation,
because it offers the opportunity to obtain public input on complex
topics in an environment that encourages participants to educate
themselves about the topic and discuss it in a thoughtful, respectful
manner. Information about the topic is intentionally neutral and
respectful of the full range of underlying values and experience with
healthcare issues in the population. This approach is designed to
improve upon the sometimes superficial or ``top of mind'' responses
that are often provided by public opinion surveys. AHRQ views public
deliberation as a potential source of higher quality public input on
issues fundamental to the Agency's mission, such as the best and most
effective ways to use comparative effectiveness research, than has
heretofore been available.
Several distinct deliberative methods have been developed and used
previously. They share the three core elements of public deliberation,
but differ on key features of implementation such as duration, whether
they take place in-person or online, and the use of content experts.
Although there is
[[Page 74786]]
considerable theoretical and case study literature endorsing the value
of public deliberation, there has been little empirical research about
its effectiveness and even less about the comparative merits of
different deliberative methods (Community Forum Deliberative Methods
Literature Review, 2010).
The objectives of this study are to:
1. Obtain informed and deliberated input from lay people on
important questions underlying AHRQ's research program; and
2. Expand the evidence base for the use of public deliberation
methods for exploring issues relevant to healthcare research by
comparing the outcomes of five distinct deliberative methods to a
control condition and to each other.
This study is being conducted by AHRQ through its contractor, the
American Institutes of Research (AIR), pursuant to AHRQ's statutory
authority to (1) promote healthcare quality improvement by conducting
and supporting both research that develops and presents scientific
evidence regarding all aspects of healthcare and the synthesis and
dissemination of available scientific evidence for use by policymakers,
among others, and (2) conduct and support research, provide technical
assistance, and disseminate information on healthcare and on systems
for the delivery of such care. See 42 U.S.C. 299(b)(1)(A), (D), (F),
and (G); 42 U.S.C. 299(b)(2); 42 U.S.C. 299a(a)(1)-(4).
Method of Collection
To achieve the objectives of this study the following activities
and data collections will be implemented:
(1) Participant recruitment--A short screening questionnaire,
including a brief overview of the study, will be used to recruit
persons for the study.
(2) Educational Materials--Educational materials are designed to
inform participants about the topics that are being deliberated and
will be provided to all 1,685 participants recruited before the
implementation of any of the methods, but after the administration of
the Knowledge and Attitudes Pre-test Survey (described below).
Additional content provided during the deliberative method sessions
includes an overview of the study and the background materials needed
by participants to competently deliberate the issues. For two methods
(ODP and IDP; see below) educational materials to be used during the
sessions will be sent to participants before the sessions (but after
administration of the pre-test).
(3) Deliberative Discussion Groups and Control Group--The purpose
of the discussion groups is to obtain informed and deliberated input
from lay people on an important set of issues underlying healthcare
research. Participants will be randomly assigned to one of the five
deliberative methods or a control condition. The five methods were
selected because they have been previously implemented and vary on key
features that may affect the scalability and effectiveness of the
methods, including: duration (from two hours to three days), mode of
implementation (online versus in person), role of content experts, and
time between sessions allowing participants to seek additional
information on the issues and communicate informally with other
participants. The subject of the deliberations is the use of research
evidence in healthcare decision-making. This deliberative topic
encompasses several themes or ``variations'' that will be elaborated in
the deliberations:
1. Use of evidence to encourage better healthcare: Is evidence
useful (or, what kind of evidence is useful) to a physician and a
patient who are considering a test or treatment that has been found to
be ineffective, less effective than another, riskier than another, or
for which effectiveness has not been demonstrated?
2. Use of evidence to encourage better value: Is evidence useful
(or, what kind of evidence is useful) to a physician and a patient who
are considering a test or treatment that is effective even though an
equally effective but less expensive alternative is available?
3. Decision-making when evidence shows more complex trade-offs: Is
evidence useful (or, what kind of evidence is useful) in treatment
decisions that involve the balancing of effectiveness, risk, and value?
The issues involved in each variation will be discussed in the
context of specific comparative effectiveness research (CER) examples.
These ``vignettes'' illustrate the issues and elicit participants'
input on the issues and the values employed by participants in the
deliberations.
(4) Knowledge and Attitudes Pre-test Survey--This survey will
measure knowledge of and attitudes about the health issues discussed in
the deliberations. It will be administered to deliberation participants
and controls before educational materials are sent or the methods are
implemented.
As described, study participants will be provided with educational
materials related to the deliberative topic. In order to assess whether
or not participants were sufficiently informed on the topics addressed
in the materials, the Knowledge and Attitudes Survey contains items
assessing knowledge of medical research and medical evidence, of
comparative effectiveness research, and of healthcare costs. The
attitudinal questions refer to the use of medical evidence in
healthcare decision making. They include attitudes about health care
decision-making when research findings can provide no support for, or
conflict with patient and doctor preferences for particular treatments.
The questionnaire will also gather demographic and other
information necessary to characterize the study sample, test the
success of the randomization, and define population subgroups for which
variation in outcomes will be examined. The demographic variables also
will be used to control for participant and group characteristics that
may influence the outcomes. Even though the design involves
randomization, and these characteristics should be balanced across
groups, including them in the statistical models guards against
inadequate results from randomization.
The variables to be measured in the Knowledge and Attitudes Pre-
test Survey include:
Sociodemographic characteristics: Gender, age, marital status,
education, employment status, household income, race/ethnicity,
priority population, languages spoken (in addition to English)
General health status
Recent experience with the healthcare system (e.g., seeing a
healthcare provider more than three times for the same condition in the
last 12 months)
Health insurance coverage
Health information-seeking behavior (e.g., the extent to which
people seek healthcare information or rely on their doctors to provide
information)
(5) Knowledge and Attitudes Post-test Survey--This survey will
measure knowledge of and attitudes about the issues discussed in the
deliberations after the deliberations take place. It will be
administered to deliberation participants and controls within one week
following conclusion of the deliberative methods and will include the
same knowledge and attitude questions as the pre-test questionnaire.
(6) Deliberative Experience Survey--As described above, the five
deliberative methods being tested vary in terms of duration, mode, use
of educational materials, and time between deliberative sessions. A
one-time survey will be administered to participants in the
deliberative methods after implementation of the experimental
conditions to compare deliberative
[[Page 74787]]
methods to each other. Levels of discourse quality and implementation
quality achieved will be assessed. Using multi-item scales, the survey
will measure the following:
Discourse quality
Equal participation in the discussions
Respect for others' opinions and tolerance of differing
perspectives
Appreciation of perspectives other than their own
Reasoned justification of ideas: Sharing the reasoning or
rationale for positions, opinions, beliefs, or preferences
Implementation quality
Quality of group facilitation
Quality. of the educational materials provided
Quality of the experts
Transparency of the process and use of the results
Participants' perceived value of method
Participants' view of the influence the results will have
on programs
In sum, information collection in this study will entail
qualitative transcript review and quantitative surveys. This
information will be used to describe and summarize the input obtained
from the participants in the deliberative groups concerning the use of
evidence, presenting the findings in reports for AHRQ and the public.
The information from the surveys also will be used to expand the
evidence base for public deliberation. The experiment is designed to:
(1) Compare the effectiveness of the five deliberative methods to the
control condition and to each other, (2) compare the quality of the
discourse achieved by the deliberative methods to each other, (3)
assess the quality of implementation of the five methods, and (4) test
for variation in effectiveness and discourse quality by features of the
deliberations and for population subgroups defined by sociodemogiaphic
characteristics of the participants.
Estimated Annual Respondent Burden
Exhibit 1 shows the estimated annualized burden associated with the
respondents' time to participate in this research. The total annualized
burden hours are estimated to be 11,647 hours. The burden estimate
comprises the following activities:
Participant Recruitment--The screening questionnaire and
recruitment letter and materials will be sent to 1,685 participants. We
estimate that it will take 15 minutes to complete the questionnaire and
review the recruitment letter and materials.
Educational materials--Educational materials will be provided to
all 1,685 participants recruited before the implementation of any of
the methods. We estimate that it will take up to 1 hour to review the
materials.
Short Citizens' Deliberation (SCD): This method will be tested with
192 participants (12 groups). Participants will attend a single, 2-hour
in-person meeting.
Online Deliberative Polling[supreg] (ODP): This method will be
tested with 288 participants (24 groups) and will consist of 4 online
sessions over the course of 4 weeks; in total, this method will take
about 5 hours per person.
In-Person Deliberative Polling[supreg] (IDP): This method will be
tested with 288 participants (16 groups); participants will attend a
single in-person meeting, lasting a full day.
Citizens' Panel (CP): This method will be tested with 96
participants (4 groups); participants will attend a 3-day, in-person
meeting.
Interrupted Deliberation (ID): This method will be tested with 192
participants (12 groups). Participants will attend 2 in-person
meetings, lasting 3 hours each, a week apart. Between meetings,
participants will be asked to access an online platform. In total, this
method will take about 6 hours per person.
Knowledge and Attitudes Pre-test Survey: This survey will be
administered to 1,685 participants and will take an estimated 30
minutes to complete.
Knowledge and Attitudes Post-test Survey: This survey will be
administered to 1,685 participants and will take an estimated 20
minutes to complete.
Deliberative Experience Survey: This survey will be administered to
1,056 deliberative methods participants at the conclusion of the
deliberative method. It will take about 15 minutes to complete.
Exhibit 1--Estimated Annualized Burden Hours
----------------------------------------------------------------------------------------------------------------
Number of
Form name/Deliberative method Number of responses per Hours per Total burden
respondents respondent response hours
----------------------------------------------------------------------------------------------------------------
Recruitment and Consent Materials....... 1685 1 15/60 421
Short Citizens' Deliberation (SCD)...... 192 1 2 384
Online Deliberative Polling[reg] (ODP).. 288 1 5 1440
In-Person Deliberative Polling[reg] 288 1 9 2592
(IDP)..................................
Citizens' Panel......................... 96 1 24 2304
Interrupted Deliberation (ID)........... 192 1 6 1152
Educational Materials................... 1685 1 1 1685
Knowledge and Attitudes Pretest Survey.. 1685 1 30/60 843
Knowledge and Attitudes Posttest Survey. 1685 1 20/60 562
Deliberative Experience Survey.......... 1056 1 15/60 264
-----------------------------------------------------------------------
Total............................... 8852 N/A N/A 11647
----------------------------------------------------------------------------------------------------------------
Exhibit 2--Estimated Annualized Cost Burden
----------------------------------------------------------------------------------------------------------------
Number of Total burden Average hourly Total cost
Form name/Deliberative method respondents hours wage rate burden
----------------------------------------------------------------------------------------------------------------
Recruitment and Consent Materials....... 1685 421 $21.35 $8,988
Short Citizens' Deliberation (SCD)...... 192 384 21.35 8,198
Online Deliberative Polling[reg] (ODP).. 288 1440 21.35 30,744
In-Person Deliberative Polling[reg] 288 2592 21.35 55,339
(IDP)..................................
Citizens' Panel......................... 96 2304 21.35 49,190
Interrupted Deliberation (ID)........... 192 1152 21.35 24,595
[[Page 74788]]
Educational Materials................... 1685 1685 21.35 35,975
Knowledge and Attitudes Pretest Survey.. 1685 843 $21.35 $17,998
Knowledge and Attitudes Post-test Survey 1685 562 21.35 11,999
Deliberative Experience Survey.......... 1056 264 21.35 5,636
-----------------------------------------------------------------------
Total............................... 8852 N/A N/A 248,662
----------------------------------------------------------------------------------------------------------------
* Based upon the mean of the wages for 00-000 All Occupations ($21.35), May 2010 National Occupational
Employment and Wage Estimates. United States, ``U.S. Department of Labor, Bureau of Labor Statistics.'' https://www.bls.gov/oes/current/oes_nat.htm#00-0000.
Estimated Annual Costs to the Federal Government
Exhibit 3 below breaks down the costs related to this study. These
are the costs associated with the portion of the contract awarded to
AIR to conduct the experiment. Since the implementation and evaluation
periods will span 24 months, the costs have been annualized by taking
the total cost and dividing by 2.
Exhibit 3--Estimated Annualized Cost to the Federal Government
------------------------------------------------------------------------
Cost component Total cost Annualized cost
------------------------------------------------------------------------
Project Management.................. $60,106 $30,053
Technical Expert Panel.............. 117,793 58,896
Technology Tools.................... 177,580 88,790
Develop Educational Materials....... 368,624 184,312
Evaluation Plan..................... 214,566 107,283
Implement Methods................... 1,624,169 812,085
Conceptual Framework................ 50,195 25,098
Data Processing and Analysis........ 566,846 283,423
Reporting........................... 135,693 67,847
Overhead............................ 1,281,340 640,670
Total........................... 4,596,914 2,298,457
------------------------------------------------------------------------
Request for Comments
In accordance with the Paperwork Reduction Act, comments on AHRQ's
information collection are requested with regard to any of the
following: (a) Whether the proposed collection of information is
necessary for the proper performance of AHRQ healthcare research and
healthcare information dissemination functions, including whether the
information will have practical utility; (b) the accuracy of AHRQ's
estimate of burden (including hours and costs) of the proposed
collection(s) of information; (c) ways to enhance the quality, utility,
and clarity of the information to be collected; and (d) ways to
minimize the burden of the collection of information upon the
respondents, including the use of automated collection techniques or
other forms of information technology.
Comments submitted in response to this notice will be summarized
and included in the Agency's subsequent request for OMB approval of the
proposed information collection. All comments will become a matter of
public record.
Dated: November 16, 2011.
Carolyn Clancy,
Director.
[FR Doc. 2011-30795 Filed 11-30-11; 8:45 am]
BILLING CODE 4160-90-M
