Agency Information Collection Activities; Submission for Office of Management and Budget Review; Comment Request; Targeted Mechanism of Action Presentations in Prescription Drug Promotion, 58110-58116 [2022-20623]
Download as PDF
<![CDATA[
lotter on DSK11XQN23PROD with NOTICES1
58110
Federal Register / Vol. 87, No. 184 / Friday, September 23, 2022 / Notices
for the operation of both businesses and
their employees. Both Anchor Foods
companies were located in Westbury,
New York.
From on or about February 2011 and
continuing through January 2014, Mr.
Tuccillo knowingly and willfully
conspired with Anchor Foods, Roy
Tuccillo, Jr., and others to import giant
squid from Peru to Mr. Tuccillo’s
companies’ location in Westbury, New
York and repackage and sell that squid
falsely labeled and identified as
‘‘octopus.’’ Mr. Tuccillo sold the falsely
labeled squid in interstate commerce to
grocery stores in New Jersey, Texas, and
Massachusetts. Mr. Tuccillo used email
and fax to sell and receive payments for
the squid falsely labeled as octopus. In
total, Mr. Tuccillo’s companies made
$1,128,388.50 worth of fraudulent sales
of squid.
As a result of this conviction, FDA
sent Mr. Tuccillo, by certified mail on
June 6, 2022, a notice proposing to
debar him for a period of 5 years from
importing articles of food or offering
such articles for import into the United
States. The proposal was based on a
finding under section 306(b)(1)(C) of the
FD&C Act that Mr. Tuccillo’s felony
conviction of conspiracy to commit wire
fraud in violation of 18 U.S.C. 371 and
1343, constitutes conduct relating to the
importation into the United States of an
article of food because Mr. Tuccillo
knowingly and willfully conspired with
Anchor Foods, Roy Tuccillo, Jr., and
others to import giant squid from Peru
to his companies’ location in Westbury,
New York and repackage and sell that
squid falsely labeled and identified as
‘‘octopus’’ in interstate commerce, using
email and fax to sell and receive
payments for the falsely labeled squid.
The proposal was also based on a
determination, after consideration of the
relevant factors set forth in section
306(c)(3) of the FD&C Act (21 U.S.C.
335a(c)(3)), that Mr. Tuccillo should be
subject to a 5-year period of debarment.
The proposal also offered Mr. Tuccillo
an opportunity to request a hearing,
providing him 30 days from the date of
receipt of the letter in which to file the
request, and advised him that failure to
request a hearing constituted a waiver of
the opportunity for a hearing and of any
contentions concerning this action. Mr.
Tuccillo failed to respond within the
timeframe prescribed by regulation and
has, therefore, waived his opportunity
for a hearing and waived any
contentions concerning his debarment
(21 CFR part 12).
II. Findings and Order
Therefore, the Assistant
Commissioner, Office of Human and
VerDate Sep<11>2014
17:04 Sep 22, 2022
Jkt 256001
Animal Food Operations, under section
306(b)(1)(C) of the FD&C Act, under
authority delegated to the Assistant
Commissioner, finds that Mr. Tuccillo
has been convicted of a felony count
under Federal law for conduct relating
to the importation into the United States
of an article of food and that he is
subject to a 5-year period of debarment.
As a result of the foregoing finding,
Mr. Tuccillo is debarred for a period of
5 years from importing articles of food
or offering such articles for import into
the United States, applicable (see
DATES). Pursuant to section 301(cc) of
the FD&C Act (21 U.S.C. 331(cc)), the
importing or offering for import into the
United States of an article of food by,
with the assistance of, or at the direction
of Roy Tuccillo, Sr., is a prohibited act.
Any application by Mr. Tuccillo for
termination of debarment under section
306(d)(1) of the FD&C Act should be
identified with Docket No. FDA–2022–
N–0316 and sent to the Dockets
Management Staff (ADDRESSES). The
public availability of information in
these submissions is governed by 21
CFR 10.20.
Publicly available submissions will be
placed in the docket and will be
viewable at https://www.regulations.gov
or at the Dockets Management Staff (see
ADDRESSES) between 9 a.m. and 4 p.m.,
Monday through Friday, 240–402–7500.
Dated: September 19, 2022.
Lauren K. Roth,
Associate Commissioner for Policy.
[FR Doc. 2022–20710 Filed 9–22–22; 8:45 am]
BILLING CODE 4164–01–P
DEPARTMENT OF HEALTH AND
HUMAN SERVICES
Food and Drug Administration
[Docket No. FDA–2021–N–1050]
Agency Information Collection
Activities; Submission for Office of
Management and Budget Review;
Comment Request; Targeted
Mechanism of Action Presentations in
Prescription Drug Promotion
AGENCY:
Food and Drug Administration,
HHS.
ACTION:
Notice.
The Food and Drug
Administration (FDA, Agency, or we) is
announcing that a proposed collection
of information has been submitted to the
Office of Management and Budget
(OMB) for review and clearance under
the Paperwork Reduction Act of 1995.
DATES: Submit written comments
(including recommendations) on the
SUMMARY:
PO 00000
Frm 00057
Fmt 4703
Sfmt 4703
collection of information by October 24,
2022.
ADDRESSES: To ensure that comments on
the information collection are received,
OMB recommends that written
comments be submitted to https://
www.reginfo.gov/public/do/PRAMain.
Find this particular information
collection by selecting ‘‘Currently under
Review—Open for Public Comments’’ or
by using the search function. The title
of this information collection is
‘‘Targeted Mechanism of Action
Presentations in Prescription Drug
Promotion.’’ Also include the FDA
docket number found in brackets in the
heading of this document.
FOR FURTHER INFORMATION CONTACT:
JonnaLynn Capezzuto, Office of
Operations, Food and Drug
Administration, Three White Flint
North, 10A–12M, 11601 Landsdown St.,
North Bethesda, MD 20852, 301–796–
3794, PRAStaff@fda.hhs.gov.
For copies of the questionnaire: Office
of Prescription Drug Promotion (OPDP)
Research Team, DTCresearch@
fda.hhs.gov.
SUPPLEMENTARY INFORMATION: In
compliance with 44 U.S.C. 3507, FDA
has submitted the following proposed
collection of information to OMB for
review and clearance.
Targeted Mechanism of Action
Presentations in Prescription Drug
Promotion
OMB Control Number 0910—NEW
I. Background
Section 1701(a)(4) of the Public
Health Service Act (42 U.S.C.
300u(a)(4)) authorizes FDA to conduct
research relating to health information.
Section 1003(d)(2)(C) of the Federal
Food, Drug, and Cosmetic Act (FD&C
Act) (21 U.S.C. 393(d)(2)(C)) authorizes
FDA to conduct research relating to
drugs and other FDA-regulated products
in carrying out the provisions of the
FD&C Act.
The Office of Prescription Drug
Promotion’s (OPDP) mission is to
protect the public health by helping to
ensure that prescription drug promotion
is truthful, balanced, and accurately
communicated. OPDP’s research
program provides scientific evidence to
help ensure that our policies related to
prescription drug promotion will have
the greatest benefit to public health.
Toward that end, we have consistently
conducted research to evaluate the
aspects of prescription drug promotion
that are most central to our mission. Our
research focuses in particular on three
main topic areas: advertising features,
including content and format; target
E:\FR\FM\23SEN1.SGM
23SEN1
Federal Register / Vol. 87, No. 184 / Friday, September 23, 2022 / Notices
populations; and research quality.
Through the evaluation of advertising
features, we assess how elements such
as graphics, format, and disease and
product characteristics impact the
communication and understanding of
prescription drug risks and benefits.
Focusing on target populations allows
us to evaluate how understanding of
prescription drug risks and benefits may
vary as a function of audience, and our
focus on research quality aims at
maximizing the quality of research data
through analytical methodology
development and investigation of
sampling and response issues. This
study will inform the first two topic
areas, advertising features and target
populations.
Because we recognize the strength of
data and the confidence in the robust
nature of the findings are improved
through the results of multiple
converging studies, we continue to
develop evidence to inform our
thinking. We evaluate the results from
our studies within the broader context
of research and findings from other
sources, and this larger body of
knowledge collectively informs our
policies as well as our research program.
Our research is documented on our
home page, which can be found at:
https://www.fda.gov/about-fda/centerdrug-evaluation-and-research-cder/
office-prescription-drug-promotionopdp-research. The website includes
links to the latest Federal Register
notices and peer-reviewed publications
produced by our office.
In 2014, OPDP conducted focus
groups designed to provide insights on
how consumers and healthcare
providers (HCPs), including physicians,
nurse practitioners, and physician
assistants, interpret the term ‘‘targeted’’
in prescription drug promotional
materials. Although diverse views were
voiced, there appeared to be some
tendency toward the impression that
products with promotional materials
using this term would be safer and more
effective than other similar treatments.
OPDP is also now conducting a
nationally representative survey
regarding the ways in which consumers
and primary care physicians (PCPs)
interpret terms and phrases commonly
used in prescription drug promotional
materials, including assessment of
impressions of the terms ‘‘targeted’’ and
‘‘targeted mechanism of action’’
(targeted MoA) (86 FR 24867, May 10,
2021). Building upon this line of
research, the proposed study will
investigate the influence of targeted
MoA claims, graphics, and disclosures
that provide context about a drug’s
targeted MoA, utilizing an experimental
design with both consumer and HCP
samples. The experimental approach
described here is intended to
complement and augment the prior
research by facilitating assessment of
causality. Specifically, the proposed
study will explore how varied targeted
MoA presentations affect consumer and
HCP understanding of the MoA of a
drug, perception of drug benefits and
risks, attention to risk information, and
interest in the drug.
Table 1 depicts the study design.
Participants will be randomly assigned
to one of 12 experimental conditions in
58111
which the presence versus absence of:
(1) a targeted MoA claim, (2) a graphic
depicting a targeted MoA, and (3) a
disclosure that provides context about
the targeted MoA of the drug are varied
in a branded website for a fictitious
prescription drug indicated to treat
bladder cancer and cancers of the
urinary tract (renal pelvis, ureter, or
urethra) that have spread or cannot be
removed by surgery. We selected cancer
as the medical condition for study given
the prevalence of targeted MoA
presentations in promotional materials
for prescription drugs indicated to treat
various forms of cancer. Notably, there
will be three variations related to the
targeted MoA graphic: (1) no graphic, (2)
an inaccurate graphic (graphic 1)
showing only the effect of the drug on
cancerous cells but not on healthy cells,
and (3) an accurate graphic (graphic 2)
that will show the effect of the drug on
both cancerous and healthy cells. The
design will be replicated in both the
consumer and HCP samples with
stimuli specifically created for each
audience. Draft stimuli were informed
by, but not identical to, actual targeted
MoA presentations from a marketplace
evaluation conducted under FDA
guidance. Draft stimuli were also
informed by an FDA subject matter
expert’s review. Following exposure to
the stimuli, the participants will
complete a questionnaire designed to
assess relevant outcome measures. A
copy of the questionnaire is available
upon request. All aspects of this study
will be completed online. Participation
is estimated to take approximately 20
minutes, excluding the screener’s time.
TABLE 1—STUDY DESIGN
Targeted MoA graphic
Sample
Targeted
MoA claim
Disclosure
HCP .......................................................
Present .........................
Absent ..........................
Consumer ..............................................
Present .........................
Absent ..........................
lotter on DSK11XQN23PROD with NOTICES1
1 Each
Present ..........................
Absent ...........................
Present ..........................
Absent ...........................
Present ..........................
Absent ...........................
Present ..........................
Absent ...........................
Present
(graphic 1—
inaccurate)
Present
(graphic 2—
accurate)
Absent
1D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D
D symbol represents an experimental condition.
For the HCP sample, we will recruit
oncologists, PCPs with oncology
experience, and nurse practitioners and
physician assistants who specialize in
oncology. We will also recruit a general
population sample of adult volunteers
18 years or older for the consumer
sample. A general population, rather
VerDate Sep<11>2014
17:04 Sep 22, 2022
Jkt 256001
than a diagnosed consumer sample, was
selected because of concerns about
being able to recruit enough participants
for this particular study if we selected
a cancer-specific sample.
We will ask consumers to consider a
hypothetical scenario in which they
have recently been diagnosed with
PO 00000
Frm 00058
Fmt 4703
Sfmt 4703
cancer and are actively looking for
available treatments. HCPs will be asked
to consider a scenario in which they are
actively looking for available treatments
for a patient who has been diagnosed
with cancer. We will also ask consumers
if they have ever been diagnosed with
cancer. HCP participants will be drawn
E:\FR\FM\23SEN1.SGM
23SEN1
lotter on DSK11XQN23PROD with NOTICES1
58112
Federal Register / Vol. 87, No. 184 / Friday, September 23, 2022 / Notices
from online HCP panels, and general
population consumer participants will
be drawn from online consumer panels.
Informed by power analyses, we will
recruit a sample of 540 HCPs and 540
consumers for the main study.
In the Federal Register of October 28,
2021 (86 FR 59736), FDA published a
60-day notice requesting public
comment on the proposed collection of
information. FDA received five
comments that were Paperwork
Reduction Act (PRA) related. Within the
submissions, FDA received multiple
comments that the Agency has
addressed in this notice. For brevity,
some public comments are paraphrased
and, therefore, may not state the exact
language used by the commenter. All
comments were considered even if they
were not fully captured by our
paraphrasing in this document. One
submission (ID number FDA–2021–N–
1050–0002) was read and considered
but was outside the scope of the
research and is not addressed further.
Comments and responses are numbered
here for organizational purposes only.
(Comment 1) One comment stated
that FDA has already investigated how
HCPs and consumers interpret the terms
‘‘targeted’’ and ‘‘targeted mechanism of
action.’’
(Response 1) Prior qualitative
research 1 looked at how consumers and
HCPs interpret the term ‘‘targeted’’ in
prescription drug promotional
materials. This initial qualitative
research suggested that products using
the term ‘‘targeted’’ may appear safer or
more effective than other similar
treatments but did not fully explore the
implications of those interpretations.
Robust empirical evidence is needed to
understand how complex concepts,
such as ‘‘targeted’’ and ‘‘targeted MoA,’’
are interpreted or whether they lead to
inaccurate inferences about a drug’s
efficacy and side effects when presented
to consumers and HCPs in prescription
drug promotion. The present research
seeks to extend previous studies by
investigating the effects of including a
graphic and by exploring whether the
inclusion of a disclosure statement can
help to clarify the information. It is
possible that the presence of targeted
MoA graphics affects the impressions of
the product, which we are assessing in
this study. It is also possible that any
inflated perceptions consumers or HCPs
may have based on the MoA claim or
graphics can be adjusted by adding a
1 See ‘‘Focus Groups to Investigate Specific
Terminology in Prescription Drug Promotion
(completed in 2014),’’ available at https://
www.fda.gov/aboutfda/centersoffices/
officeofmedicalproductsandtobacco/cder/
ucm090276.htm.
VerDate Sep<11>2014
17:04 Sep 22, 2022
Jkt 256001
disclosure. These are the questions this
research is aiming to address through an
experimental design. We conducted a
literature review, which found that only
two published articles (Refs. 1 and 2)
have focused on assessing the impact of
exposure to MoA presentations in
prescription drug promotion. We also
conducted a marketplace evaluation,
which found that these types of
presentations are widespread in the
prescription drug promotion
marketplace. Together, this preliminary
work highlights the importance of this
study and the need for experimental
research that examines the effect of
targeted MoA presentations in
prescription drug promotion on both
consumers and HCPs.
(Comment 2) Two comments
proposed recruiting cancer patients
rather than general population
consumers because, according to one
comment, cancer patients are more
likely to be exposed to promotional
materials regarding cancer products and
may be more familiar with cancerrelated terms than the general
population. The comments also
suggested that being diagnosed with a
life-threatening illness may influence
perception of risk/benefit and interest in
a drug. One comment encouraged the
Agency to look for ways to mitigate such
bias, and the other specifically proposed
that the Agency focus the research on a
target consumer respondent sample of
those who have had a cancer diagnosis
and allow the screening criteria to
straddle across multiple cancer
diagnoses.
(Response 2) We chose a general
population sample because of concerns
about being able to recruit enough
participants if we selected a cancerspecific sample. However, we agree that
in a future study, a small, carefully
designed replication study with cancer
patients could be valuable. We will also
ask participants if they have been
diagnosed with cancer and control for
any impact that a diagnosis of prior
cancer may have.
(Comment 3) One comment objected
that access to the specific study stimuli
and questionnaire was not provided.
(Response 3) We have described the
purpose of the study, the design, and
the population of interest and have
provided the questionnaire to numerous
individuals upon request. We provided
the disclosure language in the
questionnaire. Our full stimuli are
under development during the PRA
process. We do not make draft stimuli
public during this time because of
concerns that this may contaminate our
participant pool and compromise our
research.
PO 00000
Frm 00059
Fmt 4703
Sfmt 4703
(Comment 4) Two comments
suggested that the research assumes that
all targeted MoA claims that do not
include a discussion of off-target effects
are misleading and that it is misleading
to suggest that targeted therapies are
safer or more effective. The comments
noted that this assumption would be
overly broad and simplified and may
result in biased results.
(Response 4) This research does not
assume that any specific presentation is
or is not misleading. Rather, this
research aims to understand whether
variations in MoA presentations of a
targeted drug (e.g., presenting an
inaccurate graphic depicting a drug’s
MoA without a disclosure relative to an
accurate graphic depicting the MoA)
may affect consumer and HCP
perceptions of the drug. In this way, the
research will provide more information
to help determine whether these
audiences are misled by the tested
presentations.
(Comment 5) Two comments focused
on the proposed graphics. One
expressed concern about the ability of a
graphic to depict a targeted MoA
accurately (particularly as it refers to the
impact on off-target healthy cells) and to
convey a truthful and non-misleading
representation. The other comment
proposed changes to the inaccurate
graphic in terms of how it depicted
healthy and cancer cells.
(Response 5) We tested candidate
graphics in cognitive interviews to
confirm that the audience interpreted
the graphics as intended. The graphics
were also reviewed by medical
professionals, and we consulted with a
doctoral-trained researcher who
publishes extensively on the effects of
graphic presentations in health
communication and advertising.
(Comment 6) One comment noted that
it is unclear what proportion of the
sample will be oncologists versus PCPs
with oncology experience. The
comment also stated that while PCPs
may have a role in the cancer patient’s
journey and may provide input along
the way to diagnosis, as well as during
the management phase of treatment,
they are not routine decision makers for
new treatments or treatment changes.
(Response 6) HCPs of all types are
exposed to prescription drug promotion.
Depending on location (e.g., rural areas)
and type of clinical setting, some nononcologists may consider oncologic
prescription drugs to treat their patients.
We agree that oncologists are the most
relevant population to study in this
research. However, we also want to
know whether specific education and
experience influence the processing of
claims, graphics, and disclosures. We
E:\FR\FM\23SEN1.SGM
23SEN1
lotter on DSK11XQN23PROD with NOTICES1
Federal Register / Vol. 87, No. 184 / Friday, September 23, 2022 / Notices
intend to use PCPs as a control group to
understand whether specific advanced
training influences the understanding of
MoA claims, graphics, and associated
disclosures. Further, including PCPs
with oncology experience alongside
oncologists has yielded useful data in
prior studies (Ref. 3). The sample will
be equally distributed across
oncologists, PCPs with oncology
experience, and nurse practitioners and
physician assistants with oncology
experience.
(Comment 7) One comment stated
that the study should only recruit nurse
practitioners and physician assistants
who specialize in oncology.
(Response 7) We agree. Only nurse
practitioners and physician assistants
who specialize in oncology are eligible
for the study.
(Comment 8) One comment noted that
the instructions at the top of the
questionnaire ask participants to ‘‘make
your best guess’’ based on the web page
they just viewed. The comment stated
that respondents should not be asked to
guess as their response and argued that
these instructions undermine the
importance of the participants’ answers.
(Response 8) The instructions are
displayed before perceived efficacy and
risk questions where consumer
participants are told, ‘‘Most people
don’t know how a prescription drug will
affect them until they’ve taken the drug.
But we’d like you to make your best
guess based on the web page you just
saw. Please answer the following
questions based on what you saw on the
web page.’’ HCPs are told, ‘‘Please
answer the following questions based on
what you saw on the web page rather
than prior knowledge of this class of
medications.’’
These instructions have been
cognitively tested in prior studies, as
well as in the present study, and we
found no evidence that these
instructions undermined the perceived
importance of participants’ answers.
Instead, the instructions helped to
indicate that we wanted participants to
form an opinion and that they did not
need to base their opinion on prior
knowledge to do so.
(Comment 9) One comment suggested
that the recall questions (questions 6
through 11) and especially the ‘‘foil’’
responses could bias the responses to
the questions that follow them and
recommended locating the recall
questions after other questions.
(Response 9) We always approach
question ordering carefully, attempting
to balance several considerations,
including the reduction of bias from one
question to another, the flow, and the
importance of each item. In this case,
VerDate Sep<11>2014
17:04 Sep 22, 2022
Jkt 256001
we are prioritizing measures of specific
claim comprehension over other more
general questions in our questionnaire,
which is why questions 6 through 11 are
placed earlier in the questionnaire.
Answering recall and comprehension
questions first will allow consumers and
HCPs to provide a more accurate
response and will allow us to better
understand whether the information
was comprehended. We did not
encounter any issues with recall
questions influencing responses to
questions found later in the survey
during cognitive interviews.
(Comment 10) One comment
recommended using a consistent scale
throughout the survey. Another
suggested changing questions 12, 13, 14,
16, 17, 18, 19, 20, 22, and 23 to 7-point
scales to add a midpoint.
(Response 10) We use true/false/don’t
know or yes/no/don’t know response
options for the comprehension
questions and Likert-type scales for
perceptions and opinion questions.
Using one scale throughout the survey
would not necessarily provide better
data. For nearly all Likert-type
questions, we use 6-point scales with
the endpoints labeled. Some of these
questions with Likert-type scales are
validated questions; for these, we have
maintained the response options from
the validated measures. Other questions
were altered from validated measures,
and similarly, we preferred to maintain
the Likert-type scales that the original
measure had. We will change question
5 from a 7-point to a 6-point scale to
increase consistency. We will retain the
5-point scales with all response options
labeled for the two validated scales for
beliefs about medications and trust in
prescription drug materials.
Regarding the inclusion of a
midpoint, this is a matter of debate in
the literature and has never been
resolved. Based on input from cognitive
interviews and in response to public
comments, we will be adding a neutral
point to the comparative efficacy and
risk questions (i.e., questions 17 through
23), which will change these questions
to be 7-point response options with
endpoints and midpoint labeled.
(Comment 11) Two comments stated
that the 6-point scales do not allow the
respondent to pick neither agree/
disagree/unknown. One comment noted
that this is a concern for most 6-point
scale questions but particularly for
questions 17 through 23, which
compare the study drug to other
medications. The comments
recommended either an anchored
neutral middle point on the scale or a
box for uncertain/do not know
responses.
PO 00000
Frm 00060
Fmt 4703
Sfmt 4703
58113
(Response 11) There are benefits and
drawbacks to including a neutral or ‘‘no
reaction’’ response in survey research,
and the decision to use a neutral
midpoint depends on the goal of the
measures (Refs. 4 and 5). For questions
assessing comprehension of the MoA
claim, we included a ‘‘do not know’’
option as this response would indicate
some level of uncertainty about the
MoA, and that uncertainty itself would
be meaningful and actionable
information. However, when assessing
perceptions and attitudes about the
claim, graphic, or disclosure, our
objective is to force a selection.
Inclusion of a neutral response option in
these instances could potentially
encourage satisficing—cuing
participants to select a neutral response
when there is uncertainty (Ref. 7). For
the comparative risk and efficacy
questions (questions 17 through 23), we
will include a midpoint based on results
from cognitive interviews; however,
these interviews did not point to the
need to include a midpoint for the other
questions.
(Comment 12) Questions 17 through
23 ask about the efficacy and risks of the
study drug compared to other
prescription drugs for the same
indication. One comment contended
that, without prior knowledge of the
efficacy and risks of the prescription
drugs on the market, it would be
difficult for respondents to make a fully
informed conclusion. Another comment
asserted that the comparative risk and
efficacy questions should be revised to
establish a clear comparator, such as
chemotherapy. Finally, a comment
recommended removing these questions
as consumers should not be assessing a
drug’s safety or efficacy compared to
other drugs.
(Response 12) There are instances in
the clinical setting when consumers will
discuss the safety and risk information
of a drug compared to others (e.g., if a
patient switches from one drug to
another or if a family member asks the
consumer to talk to their doctor about
another drug). We acknowledge that in
a clinical setting, patients and HCPs
may use additional information to make
decisions about how a drug compares to
another. However, the intent of
questions 17 through 23 is to
understand whether exposure to
different presentations of the MoA
claim, graphics, and disclosure results
in different comprehension or
perceptions, such as perception of
comparative risks and efficacy. Except
for the varied presentations, all
participants will have the same level of
information regarding the MoA of the
drug. So, we would expect that all
E:\FR\FM\23SEN1.SGM
23SEN1
lotter on DSK11XQN23PROD with NOTICES1
58114
Federal Register / Vol. 87, No. 184 / Friday, September 23, 2022 / Notices
participants would be equally informed
of the drug, and differences among
study conditions could be attributed to
the experimental manipulations.
Additionally, any subjective
experiences outside the experiment
setting should be evenly distributed
across study conditions as a function of
random assignment; therefore, they
should not have any impact on the
outcomes of the study. Still, cognitive
interviews indicated that HCPs and
consumers preferred that a midpoint be
added to the response scale for these
questions, which we added in the
revised questionnaire. Based on
cognitive interviews, we also revised the
questions to include the phrase
‘‘compared to other similar prescription
drugs that are for/treat bladder cancer.’’
We will also review these questions and
make any necessary adjustments based
on pre-testing results.
(Comment 13) One comment stated
that the questionnaire does not consider
the HCP respondents’ baseline
understanding or expectations of
targeted treatments.
(Response 13) We expect that any
knowledge or expectations of targeted
treatments that consumers and HCPs
already have outside of the experiment
setting should be evenly distributed
across study conditions as a function of
random assignment; therefore, observed
differences between conditions are
unlikely to be caused by these
individual differences. However, we
added an item that assesses HCPs’
knowledge of targeted therapies for
cancer treatments.
(Comment 14) One comment
encouraged FDA to disseminate all final
results of completed research related to
this topic.
(Response 14) FDA’s research is
documented on our homepage, which
can be found at https://www.fda.gov/
about-fda/center-drug-evaluation-andresearch-cder/office-prescription-drugpromotion-opdp-research. The website
includes links to the latest Federal
Register notices and peer-reviewed
publications produced by our office.
The Agency also anticipates
disseminating the results of this study
after the final analyses of the data are
completed. The exact timing and nature
of any such dissemination has not been
determined, but dissemination of
research results often occurs through
presentations at trade and academic
conferences, publications, articles, and
postings on FDA’s website.
(Comment 15) One comment
recommended that certain populations,
such as those who work in
pharmaceutical marketing or for the
U.S. Department of Health and Human
VerDate Sep<11>2014
17:04 Sep 22, 2022
Jkt 256001
Services (HHS), be excluded from the
study.
(Response 15) We agree. Participants
will be excluded from participation if
they work for a pharmaceutical,
advertising, or market research company
or are employed by HHS.
(Comment 16) One comment
recommended that participants who are
unable to recall key elements of the
stimuli, such as indication, risk
elements, presence of claim, and
presence of disclaimers, be excluded
from the study because they are not able
to appropriately assess the MoA
presentations.
(Response 16) The fact that a
consumer or HCP is not able to recall
certain information does not mean they
did not see that information or
subconsciously process it (Ref. 6).
Therefore, we do not plan to exclude
anyone based on their self-reported
recall of elements in the stimuli.
(Comment 17) One comment
suggested that participants should be
asked questions 30 through 34 as part of
a pre-test and be stratified based on
their responses.
(Response 17) Typically, stratified
randomization is used if there are
prognostic variables that correlate with
outcome measures and researchers are
concerned about such factors not being
evenly distributed across groups (Ref.
8). We have no reason to expect that the
aforementioned factors would have a
strong association with the outcome
measures, nor do we have reason to
believe that we will not achieve
adequate balance of prognostic variables
given the large sample size proposed for
this study (Ref. 8). Random assignment
will help to produce groups that are, on
average, probabilistically similar to each
other. Because randomization
eliminates most other sources of
systematic variation, we can be
reasonably confident that any effect that
is found is the result of the intervention
and not some preexisting differences
between the groups (Ref. 9). However,
we have included questions 30 through
34 to assess the association of factors
such as health literacy, prior cancer
diagnosis, or familiarity with cancer
treatment options with our outcomes
and statistically control for those
variables if necessary.
(Comment 18) One comment
suggested that in order to ensure that
differences in risk assessment across
stimuli are due to the manipulation of
MoA information, the prominence of the
risk presentation should be
standardized across the 12 versions of
the stimuli and displayed in accordance
with FDA’s guidance document entitled
‘‘Presenting Risk Information in
PO 00000
Frm 00061
Fmt 4703
Sfmt 4703
Prescription Drug and Medical Device
Promotion.’’ 2 The comment also
encouraged the use of qualifiers to
delineate which side effects are
considered serious.
(Response 18) In creating the stimuli,
we created one web page that was the
basis for all the stimuli. The risk
presentation was standardized across
the experimental conditions, and we
kept FDA’s guidance in mind when
displaying stimuli. Regarding the
suggested use of qualifiers to delineate
which side effects are considered
‘‘serious,’’ we again note that we kept
FDA’s guidance in mind with respect to
the risk presentation.
(Comment 19) One comment noted
that the disclosure for patients should
be reworded as follows to prevent
implied bias: ‘‘[Drug X] delivers
medicine directly to cancer cells and
can also harm healthy cells.’’
(Response 19) We revised the
statement to read ‘‘[Drug X] could also
affect healthy cells.’’ With this change,
the consumer disclosure is consistent
with the content of the disclosure
shown to HCPs.
(Comment 20) One comment asserted
that most promotional materials in the
real world qualify MoA statements with
language mirroring the labeling (e.g.,
‘‘Pre-clinical studies demonstrate . . .’’)
and recommended that the research
materials be updated to include similar
qualifying language.
(Response 20) The addition of such
language may create an imbalance of
information across the various
experimental conditions and could
confound interpretation of the results.
As such, we did not include the
qualifying language mentioned above.
(Comment 21) One comment
suggested that study participants should
be allowed to refer back to the product
website as often as needed rather than
only being permitted to view it once.
(Response 21) As a practice, we often
purposely do not permit study
participants to refer back to the product
website as often as needed for these
types of studies. Rather, for this study,
we will instruct participants to read the
website carefully and alert them that
they will be answering several questions
about the content that they just saw and
that they cannot return to the website.
The goal of this study is not to assess
participants’ comprehension of verbatim
information in the stimuli, for which
2 The draft guidance for industry ‘‘Presenting Risk
Information in Prescription Drug and Medical
Device Promotion’’ (May 2009) is available on the
FDA guidance web page at https://www.fda.gov/
regulatory-information/search-fda-guidancedocuments. When finalized this guidance will
represent FDA’s thinking on this issue.
E:\FR\FM\23SEN1.SGM
23SEN1
Federal Register / Vol. 87, No. 184 / Friday, September 23, 2022 / Notices
repeated exposures to stimuli may be
more appropriate in another study.
Rather, the present study is interested in
gist understanding of the information.
Allowing for multiple exposures to the
stimuli could potentially influence
study outcomes and confound
interpretation of the study results. A
large literature supports presence of a
‘‘mere exposure effect’’ in social science
research, where more exposure
enhances processing and increases
positive affect toward stimuli (Refs. 10
and 11).
(Comment 22) One comment
recommended removing question 16
(i.e., risk-benefit tradeoff) for consumers
because consumers may not have the
experience or background to assess a
drug’s benefit-risk profile. The comment
also suggested that this question ignores
the role of prescribers in informing
patients of the relevant risks and
benefits of prescription medications.
(Response 22) We disagree that
consumers do not form their own
perceptions about risk-benefit tradeoffs
after seeing direct-to-consumer (DTC)
promotional materials and before any
discussion with an HCP. Consumers
often wish to participate in shared
decision making with HCPs when
selecting prescription drugs and may
request specific prescription drugs from
their HCPs based on promotions they
have seen in the marketplace. Because
the information consumers receive
through DTC prescription drug
58115
promotion can impact these requests, it
is important to investigate how the
information in prescription drug
promotional pieces impacts consumer
attention, understanding, and
perceptions. In addition, the purpose of
these questions is to assess perceived
benefit and risk based on the
promotional material shown. The
question includes instructions
indicating that judgments should be
reached based on the information on the
prescription drug website. As such, we
plan to ask participants about their
perceptions of the risk-benefit tradeoff
using question 16, which is a common
and validated item in DTC research.
FDA estimates the burden of this
collection of information as follows:
TABLE 2—ESTIMATED ANNUAL REPORTING BURDEN 1
Number of
respondents 2
Activity
Pretest:
General population: pretest screener completes (assumes 75% eligible).
General population: number of completes, pretest ......
HCP: pretest screener completes (assumes 60% eligible).
HCP: number of completes, pretest .............................
Main Study:
General population: number of main study screener
completes (assumes 75% eligible).
General population: number of completes, main study
HCP: number of main study screener completes (assumes 60% eligible).
HCP: number of completes, main study ......................
Total ......................................................................
Number of
responses per
respondent
Average
burden per
response 3
Total annual
responses
Total
hours
528
1
528
0.08 (5 min.) ......
42.2
396
660
1
1
396
660
0.33 (20 min.) ....
0.08 (5 min.) ......
130.7
52.8
396
1
396
0.33 (20 min.) ....
130.7
792
1
792
0.08 (5 min.) ......
63.4
594
990
1
1
594
990
0.33 (20 min.) ....
0.08 (5 min.) ......
196.0
79.2
594
1
594
0.33 (20 min.) ....
196.0
........................
........................
........................
............................
891
1 There
are no capital costs or operating and maintenance costs associated with this collection of information.
with most online and mail surveys, it is always possible that some participants are in the process of completing the survey when the target
number is reached and that those surveys will be completed and received before the survey is closed out. To account for this, we have estimated approximately 10 percent overage for both samples in the study.
3 Burden estimates of less than 1 hour are expressed as a fraction of an hour in decimal format.
2 As
lotter on DSK11XQN23PROD with NOTICES1
II. References
The following references marked with
an asterisk (*) are on display at the
Dockets Management Staff (see
ADDRESSES) and are available for
viewing by interested persons between
9 a.m. and 4 p.m., Monday through
Friday; they also are available
electronically at https://
www.regulations.gov. References
without asterisks are not on public
display at https://www.regulations.gov
because they have copyright restriction.
Some may be available at the website
address, if listed. References without
asterisks are available for viewing only
at the Dockets Management Staff. FDA
has verified the website addresses, as of
the date this document publishes in the
Federal Register, but websites are
subject to change over time.
VerDate Sep<11>2014
17:04 Sep 22, 2022
Jkt 256001
1. O’Donoghue, A.C., Williams, P.A.,
Sullivan, H.W., et al. ‘‘Effects of
Comparative Claims in Prescription Drug
Direct-to-Consumer Advertising on
Consumer Perceptions and Recall.’’
Social Science & Medicine, 120:1–11,
2014.
*2. Sullivan, H.W., O’Donoghue, A.C., Gard
Read, J., et al. ‘‘Testimonials and
Informational Videos on Branded
Prescription Drug websites:
Experimental Study to Assess Influence
on Consumer Knowledge and
Perceptions.’’ Journal of Medical internet
Research, 20(1):e13, 2018.
3. Boudewyns, V., O’Donoghue, A.C., Paquin,
R.S., et al. ‘‘Physician Interpretation of
Data of Uncertain Clinical Utility in
Oncology Prescription Drug Promotion.’’
The Oncologist, 26(12):1071–1078, 2021.
4. Moors, G. ‘‘Exploring the Effect of a
Middle Response Category on Response
Style in Attitude Measurement.’’ Quality
& Quantity, 42(6):779–794, 2008.
5. Sturgis, P., Roberts, C., and Smith, P.
PO 00000
Frm 00062
Fmt 4703
Sfmt 4703
‘‘Middle Alternatives Revisited: How the
Neither/nor Response Acts as a Way of
Saying ‘I Don’t Know?’’’ Sociological
Methods & Research, 43(1):15–38, 2014.
6. Shapiro, S. and Krishnan, H.S. ‘‘MemoryBased Measures for Assessing
Advertising Effects: A Comparison of
Explicit and Implicit Memory Effects.’’
Journal of Advertising, 30(3):1–13, 2001.
7. Krosnick, J.A. ‘‘Questionnaire Design.’’ In:
Vannette, D., Krosnick, J. (eds). The
Palgrave Handbook of Survey Research
(pp. 439–455). Palgrave Macmillan,
Cham, 2018.
8. Friedman, L.M., Furberg, C.D., DeMets,
D.L., et al. Fundamentals of Clinical
Trials. 5th ed. New York, NY: Spring
Science-Business Media, 1998.
9. Fisher, R.A. The Design of Experiments.
Edinburgh, United Kingdom: Oliver &
Boyd, 1935.
10. Bornstein, R.F. ‘‘Exposure and Affect:
Overview and Meta-analysis of Research,
1968–1987.’’ Psychological Bulletin,
106(2):265–289, 1989.
E:\FR\FM\23SEN1.SGM
23SEN1
58116
Federal Register / Vol. 87, No. 184 / Friday, September 23, 2022 / Notices
11. Bornstein, R.F. and D’Agostino, P.R. ‘‘The
Attribution and Discounting of
Perceptual Fluency: Preliminary Tests of
a Perceptual Fluency/Attributional
Model of the Mere Exposure Effect.’’
Social Cognition, 12(2):103–128, 1994.
Dated: September 19, 2022.
Lauren K. Roth,
Associate Commissioner for Policy.
[FR Doc. 2022–20623 Filed 9–22–22; 8:45 am]
BILLING CODE 4164–01–P
DEPARTMENT OF HEALTH AND
HUMAN SERVICES
Food and Drug Administration
[Docket Nos. FDA–2020–E–2364 and FDA–
2020–E–2365]
Determination of Regulatory Review
Period for Purposes of Patent
Extension; NURTEC ODT
AGENCY:
Food and Drug Administration,
HHS.
ACTION:
Notice.
The Food and Drug
Administration (FDA or the Agency) has
determined the regulatory review period
for NURTEC ODT and is publishing this
notice of that determination as required
by law. FDA has made the
determination because of the
submission of an application to the
Director of the U.S. Patent and
Trademark Office (USPTO), Department
of Commerce, for the extension of a
patent which claims that human drug
product.
SUMMARY:
Anyone with knowledge that any
of the dates as published (see
SUPPLEMENTARY INFORMATION) are
incorrect must submit either electronic
or written comments and ask for a
redetermination by November 22, 2022.
Furthermore, any interested person may
petition FDA for a determination
regarding whether the applicant for
extension acted with due diligence
during the regulatory review period by
March 22, 2023. See ‘‘Petitions’’ in the
SUPPLEMENTARY INFORMATION section for
more information.
ADDRESSES: You may submit comments
as follows. Please note that late,
untimely filed comments will not be
considered. The https://
www.regulations.gov electronic filing
system will accept comments until
11:59 p.m. Eastern Time at the end of
November 22, 2022. Comments received
by mail/hand delivery/courier (for
written/paper submissions) will be
considered timely if they are received
on or before that date.
lotter on DSK11XQN23PROD with NOTICES1
DATES:
VerDate Sep<11>2014
17:04 Sep 22, 2022
Jkt 256001
Electronic Submissions
Submit electronic comments in the
following way:
• Federal eRulemaking Portal:
https://www.regulations.gov. Follow the
instructions for submitting comments.
Comments submitted electronically,
including attachments, to https://
www.regulations.gov will be posted to
the docket unchanged. Because your
comment will be made public, you are
solely responsible for ensuring that your
comment does not include any
confidential information that you or a
third party may not wish to be posted,
such as medical information, your or
anyone else’s Social Security number, or
confidential business information, such
as a manufacturing process. Please note
that if you include your name, contact
information, or other information that
identifies you in the body of your
comments, that information will be
posted on https://www.regulations.gov.
• If you want to submit a comment
with confidential information that you
do not wish to be made available to the
public, submit the comment as a
written/paper submission and in the
manner detailed (see ‘‘Written/Paper
Submissions’’ and ‘‘Instructions’’).
Written/Paper Submissions
Submit written/paper submissions as
follows:
• Mail/Hand Delivery/Courier (for
written/paper submissions): Dockets
Management Staff (HFA–305), Food and
Drug Administration, 5630 Fishers
Lane, Rm. 1061, Rockville, MD 20852.
• For written/paper comments
submitted to the Dockets Management
Staff, FDA will post your comment, as
well as any attachments, except for
information submitted, marked and
identified, as confidential, if submitted
as detailed in ‘‘Instructions.’’
Instructions: All submissions received
must include the Docket Nos. FDA–
2020–E–2364 and FDA–2020–E–2365
for ‘‘Determination of Regulatory
Review Period for Purposes of Patent
Extension; NURTEC ODT.’’ Received
comments, those filed in a timely
manner (see ADDRESSES), will be placed
in the docket and, except for those
submitted as ‘‘Confidential
Submissions,’’ publicly viewable at
https://www.regulations.gov or at the
Dockets Management Staff between 9
a.m. and 4 p.m., Monday through
Friday, 240–402–7500.
• Confidential Submissions—To
submit a comment with confidential
information that you do not wish to be
made publicly available, submit your
comments only as a written/paper
submission. You should submit two
PO 00000
Frm 00063
Fmt 4703
Sfmt 4703
copies total. One copy will include the
information you claim to be confidential
with a heading or cover note that states
‘‘THIS DOCUMENT CONTAINS
CONFIDENTIAL INFORMATION.’’ The
Agency will review this copy, including
the claimed confidential information, in
its consideration of comments. The
second copy, which will have the
claimed confidential information
redacted/blacked out, will be available
for public viewing and posted on
https://www.regulations.gov. Submit
both copies to the Dockets Management
Staff. If you do not wish your name and
contact information to be made publicly
available, you can provide this
information on the cover sheet and not
in the body of your comments and you
must identify this information as
‘‘confidential.’’ Any information marked
as ‘‘confidential’’ will not be disclosed
except in accordance with § 10.20 (21
CFR 10.20) and other applicable
disclosure law. For more information
about FDA’s posting of comments to
public dockets, see 80 FR 56469,
September 18, 2015, or access the
information at: https://
www.govinfo.gov/content/pkg/FR-201509-18/pdf/2015-23389.pdf.
Docket: For access to the docket to
read background documents or the
electronic and written/paper comments
received, go to https://
www.regulations.gov and insert the
docket numbers, found in brackets in
the heading of this document, into the
‘‘Search’’ box and follow the prompts
and/or go to the Dockets Management
Staff, 5630 Fishers Lane, Rm. 1061,
Rockville, MD 20852, 240–402–7500.
FOR FURTHER INFORMATION CONTACT:
Beverly Friedman, Office of Regulatory
Policy, Food and Drug Administration,
10903 New Hampshire Ave., Bldg. 51,
Rm. 6250, Silver Spring, MD 20993,
301–796–3600.
SUPPLEMENTARY INFORMATION:
I. Background
The Drug Price Competition and
Patent Term Restoration Act of 1984
(Pub. L. 98–417) and the Generic
Animal Drug and Patent Term
Restoration Act (Pub. L. 100–670)
generally provide that a patent may be
extended for a period of up to 5 years
so long as the patented item (human
drug or biologic product, animal drug
product, medical device, food additive,
or color additive) was subject to
regulatory review by FDA before the
item was marketed. Under these acts, a
product’s regulatory review period
forms the basis for determining the
amount of extension an applicant may
receive.
E:\FR\FM\23SEN1.SGM
23SEN1
]]>Agencies
[Federal Register Volume 87, Number 184 (Friday, September 23, 2022)]
[Notices]
[Pages 58110-58116]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2022-20623]
-----------------------------------------------------------------------
DEPARTMENT OF HEALTH AND HUMAN SERVICES
Food and Drug Administration
[Docket No. FDA-2021-N-1050]
Agency Information Collection Activities; Submission for Office
of Management and Budget Review; Comment Request; Targeted Mechanism of
Action Presentations in Prescription Drug Promotion
AGENCY: Food and Drug Administration, HHS.
ACTION: Notice.
-----------------------------------------------------------------------
SUMMARY: The Food and Drug Administration (FDA, Agency, or we) is
announcing that a proposed collection of information has been submitted
to the Office of Management and Budget (OMB) for review and clearance
under the Paperwork Reduction Act of 1995.
DATES: Submit written comments (including recommendations) on the
collection of information by October 24, 2022.
ADDRESSES: To ensure that comments on the information collection are
received, OMB recommends that written comments be submitted to https://www.reginfo.gov/public/do/PRAMain. Find this particular information
collection by selecting ``Currently under Review--Open for Public
Comments'' or by using the search function. The title of this
information collection is ``Targeted Mechanism of Action Presentations
in Prescription Drug Promotion.'' Also include the FDA docket number
found in brackets in the heading of this document.
FOR FURTHER INFORMATION CONTACT: JonnaLynn Capezzuto, Office of
Operations, Food and Drug Administration, Three White Flint North, 10A-
12M, 11601 Landsdown St., North Bethesda, MD 20852, 301-796-3794,
[email protected].
For copies of the questionnaire: Office of Prescription Drug
Promotion (OPDP) Research Team, [email protected].
SUPPLEMENTARY INFORMATION: In compliance with 44 U.S.C. 3507, FDA has
submitted the following proposed collection of information to OMB for
review and clearance.
Targeted Mechanism of Action Presentations in Prescription Drug
Promotion
OMB Control Number 0910--NEW
I. Background
Section 1701(a)(4) of the Public Health Service Act (42 U.S.C.
300u(a)(4)) authorizes FDA to conduct research relating to health
information. Section 1003(d)(2)(C) of the Federal Food, Drug, and
Cosmetic Act (FD&C Act) (21 U.S.C. 393(d)(2)(C)) authorizes FDA to
conduct research relating to drugs and other FDA-regulated products in
carrying out the provisions of the FD&C Act.
The Office of Prescription Drug Promotion's (OPDP) mission is to
protect the public health by helping to ensure that prescription drug
promotion is truthful, balanced, and accurately communicated. OPDP's
research program provides scientific evidence to help ensure that our
policies related to prescription drug promotion will have the greatest
benefit to public health. Toward that end, we have consistently
conducted research to evaluate the aspects of prescription drug
promotion that are most central to our mission. Our research focuses in
particular on three main topic areas: advertising features, including
content and format; target
[[Page 58111]]
populations; and research quality. Through the evaluation of
advertising features, we assess how elements such as graphics, format,
and disease and product characteristics impact the communication and
understanding of prescription drug risks and benefits. Focusing on
target populations allows us to evaluate how understanding of
prescription drug risks and benefits may vary as a function of
audience, and our focus on research quality aims at maximizing the
quality of research data through analytical methodology development and
investigation of sampling and response issues. This study will inform
the first two topic areas, advertising features and target populations.
Because we recognize the strength of data and the confidence in the
robust nature of the findings are improved through the results of
multiple converging studies, we continue to develop evidence to inform
our thinking. We evaluate the results from our studies within the
broader context of research and findings from other sources, and this
larger body of knowledge collectively informs our policies as well as
our research program. Our research is documented on our home page,
which can be found at: https://www.fda.gov/about-fda/center-drug-evaluation-and-research-cder/office-prescription-drug-promotion-opdp-research. The website includes links to the latest Federal Register
notices and peer-reviewed publications produced by our office.
In 2014, OPDP conducted focus groups designed to provide insights
on how consumers and healthcare providers (HCPs), including physicians,
nurse practitioners, and physician assistants, interpret the term
``targeted'' in prescription drug promotional materials. Although
diverse views were voiced, there appeared to be some tendency toward
the impression that products with promotional materials using this term
would be safer and more effective than other similar treatments. OPDP
is also now conducting a nationally representative survey regarding the
ways in which consumers and primary care physicians (PCPs) interpret
terms and phrases commonly used in prescription drug promotional
materials, including assessment of impressions of the terms
``targeted'' and ``targeted mechanism of action'' (targeted MoA) (86 FR
24867, May 10, 2021). Building upon this line of research, the proposed
study will investigate the influence of targeted MoA claims, graphics,
and disclosures that provide context about a drug's targeted MoA,
utilizing an experimental design with both consumer and HCP samples.
The experimental approach described here is intended to complement and
augment the prior research by facilitating assessment of causality.
Specifically, the proposed study will explore how varied targeted MoA
presentations affect consumer and HCP understanding of the MoA of a
drug, perception of drug benefits and risks, attention to risk
information, and interest in the drug.
Table 1 depicts the study design. Participants will be randomly
assigned to one of 12 experimental conditions in which the presence
versus absence of: (1) a targeted MoA claim, (2) a graphic depicting a
targeted MoA, and (3) a disclosure that provides context about the
targeted MoA of the drug are varied in a branded website for a
fictitious prescription drug indicated to treat bladder cancer and
cancers of the urinary tract (renal pelvis, ureter, or urethra) that
have spread or cannot be removed by surgery. We selected cancer as the
medical condition for study given the prevalence of targeted MoA
presentations in promotional materials for prescription drugs indicated
to treat various forms of cancer. Notably, there will be three
variations related to the targeted MoA graphic: (1) no graphic, (2) an
inaccurate graphic (graphic 1) showing only the effect of the drug on
cancerous cells but not on healthy cells, and (3) an accurate graphic
(graphic 2) that will show the effect of the drug on both cancerous and
healthy cells. The design will be replicated in both the consumer and
HCP samples with stimuli specifically created for each audience. Draft
stimuli were informed by, but not identical to, actual targeted MoA
presentations from a marketplace evaluation conducted under FDA
guidance. Draft stimuli were also informed by an FDA subject matter
expert's review. Following exposure to the stimuli, the participants
will complete a questionnaire designed to assess relevant outcome
measures. A copy of the questionnaire is available upon request. All
aspects of this study will be completed online. Participation is
estimated to take approximately 20 minutes, excluding the screener's
time.
Table 1--Study Design
--------------------------------------------------------------------------------------------------------------------------------------------------------
Targeted MoA graphic
--------------------------------------------------
Sample Disclosure Targeted MoA claim Present Present
(graphic 1-- (graphic 2-- Absent
inaccurate) accurate)
--------------------------------------------------------------------------------------------------------------------------------------------------------
HCP...................................... Present..................... Present..................... \1\ [ssquf] [ssquf] [ssquf]
Absent...................... [ssquf] [ssquf] [ssquf]
Absent...................... Present..................... [ssquf] [ssquf] [ssquf]
Absent...................... [ssquf] [ssquf] [ssquf]
Consumer................................. Present..................... Present..................... [ssquf] [ssquf] [ssquf]
Absent...................... [ssquf] [ssquf] [ssquf]
Absent...................... Present..................... [ssquf] [ssquf] [ssquf]
Absent...................... [ssquf] [ssquf] [ssquf]
--------------------------------------------------------------------------------------------------------------------------------------------------------
\1\ Each [ssquf] symbol represents an experimental condition.
For the HCP sample, we will recruit oncologists, PCPs with oncology
experience, and nurse practitioners and physician assistants who
specialize in oncology. We will also recruit a general population
sample of adult volunteers 18 years or older for the consumer sample. A
general population, rather than a diagnosed consumer sample, was
selected because of concerns about being able to recruit enough
participants for this particular study if we selected a cancer-specific
sample.
We will ask consumers to consider a hypothetical scenario in which
they have recently been diagnosed with cancer and are actively looking
for available treatments. HCPs will be asked to consider a scenario in
which they are actively looking for available treatments for a patient
who has been diagnosed with cancer. We will also ask consumers if they
have ever been diagnosed with cancer. HCP participants will be drawn
[[Page 58112]]
from online HCP panels, and general population consumer participants
will be drawn from online consumer panels. Informed by power analyses,
we will recruit a sample of 540 HCPs and 540 consumers for the main
study.
In the Federal Register of October 28, 2021 (86 FR 59736), FDA
published a 60-day notice requesting public comment on the proposed
collection of information. FDA received five comments that were
Paperwork Reduction Act (PRA) related. Within the submissions, FDA
received multiple comments that the Agency has addressed in this
notice. For brevity, some public comments are paraphrased and,
therefore, may not state the exact language used by the commenter. All
comments were considered even if they were not fully captured by our
paraphrasing in this document. One submission (ID number FDA-2021-N-
1050-0002) was read and considered but was outside the scope of the
research and is not addressed further. Comments and responses are
numbered here for organizational purposes only.
(Comment 1) One comment stated that FDA has already investigated
how HCPs and consumers interpret the terms ``targeted'' and ``targeted
mechanism of action.''
(Response 1) Prior qualitative research \1\ looked at how consumers
and HCPs interpret the term ``targeted'' in prescription drug
promotional materials. This initial qualitative research suggested that
products using the term ``targeted'' may appear safer or more effective
than other similar treatments but did not fully explore the
implications of those interpretations. Robust empirical evidence is
needed to understand how complex concepts, such as ``targeted'' and
``targeted MoA,'' are interpreted or whether they lead to inaccurate
inferences about a drug's efficacy and side effects when presented to
consumers and HCPs in prescription drug promotion. The present research
seeks to extend previous studies by investigating the effects of
including a graphic and by exploring whether the inclusion of a
disclosure statement can help to clarify the information. It is
possible that the presence of targeted MoA graphics affects the
impressions of the product, which we are assessing in this study. It is
also possible that any inflated perceptions consumers or HCPs may have
based on the MoA claim or graphics can be adjusted by adding a
disclosure. These are the questions this research is aiming to address
through an experimental design. We conducted a literature review, which
found that only two published articles (Refs. 1 and 2) have focused on
assessing the impact of exposure to MoA presentations in prescription
drug promotion. We also conducted a marketplace evaluation, which found
that these types of presentations are widespread in the prescription
drug promotion marketplace. Together, this preliminary work highlights
the importance of this study and the need for experimental research
that examines the effect of targeted MoA presentations in prescription
drug promotion on both consumers and HCPs.
---------------------------------------------------------------------------
\1\ See ``Focus Groups to Investigate Specific Terminology in
Prescription Drug Promotion (completed in 2014),'' available at
https://www.fda.gov/aboutfda/centersoffices/officeofmedicalproductsandtobacco/cder/ucm090276.htm.
---------------------------------------------------------------------------
(Comment 2) Two comments proposed recruiting cancer patients rather
than general population consumers because, according to one comment,
cancer patients are more likely to be exposed to promotional materials
regarding cancer products and may be more familiar with cancer-related
terms than the general population. The comments also suggested that
being diagnosed with a life-threatening illness may influence
perception of risk/benefit and interest in a drug. One comment
encouraged the Agency to look for ways to mitigate such bias, and the
other specifically proposed that the Agency focus the research on a
target consumer respondent sample of those who have had a cancer
diagnosis and allow the screening criteria to straddle across multiple
cancer diagnoses.
(Response 2) We chose a general population sample because of
concerns about being able to recruit enough participants if we selected
a cancer-specific sample. However, we agree that in a future study, a
small, carefully designed replication study with cancer patients could
be valuable. We will also ask participants if they have been diagnosed
with cancer and control for any impact that a diagnosis of prior cancer
may have.
(Comment 3) One comment objected that access to the specific study
stimuli and questionnaire was not provided.
(Response 3) We have described the purpose of the study, the
design, and the population of interest and have provided the
questionnaire to numerous individuals upon request. We provided the
disclosure language in the questionnaire. Our full stimuli are under
development during the PRA process. We do not make draft stimuli public
during this time because of concerns that this may contaminate our
participant pool and compromise our research.
(Comment 4) Two comments suggested that the research assumes that
all targeted MoA claims that do not include a discussion of off-target
effects are misleading and that it is misleading to suggest that
targeted therapies are safer or more effective. The comments noted that
this assumption would be overly broad and simplified and may result in
biased results.
(Response 4) This research does not assume that any specific
presentation is or is not misleading. Rather, this research aims to
understand whether variations in MoA presentations of a targeted drug
(e.g., presenting an inaccurate graphic depicting a drug's MoA without
a disclosure relative to an accurate graphic depicting the MoA) may
affect consumer and HCP perceptions of the drug. In this way, the
research will provide more information to help determine whether these
audiences are misled by the tested presentations.
(Comment 5) Two comments focused on the proposed graphics. One
expressed concern about the ability of a graphic to depict a targeted
MoA accurately (particularly as it refers to the impact on off-target
healthy cells) and to convey a truthful and non-misleading
representation. The other comment proposed changes to the inaccurate
graphic in terms of how it depicted healthy and cancer cells.
(Response 5) We tested candidate graphics in cognitive interviews
to confirm that the audience interpreted the graphics as intended. The
graphics were also reviewed by medical professionals, and we consulted
with a doctoral-trained researcher who publishes extensively on the
effects of graphic presentations in health communication and
advertising.
(Comment 6) One comment noted that it is unclear what proportion of
the sample will be oncologists versus PCPs with oncology experience.
The comment also stated that while PCPs may have a role in the cancer
patient's journey and may provide input along the way to diagnosis, as
well as during the management phase of treatment, they are not routine
decision makers for new treatments or treatment changes.
(Response 6) HCPs of all types are exposed to prescription drug
promotion. Depending on location (e.g., rural areas) and type of
clinical setting, some non-oncologists may consider oncologic
prescription drugs to treat their patients. We agree that oncologists
are the most relevant population to study in this research. However, we
also want to know whether specific education and experience influence
the processing of claims, graphics, and disclosures. We
[[Page 58113]]
intend to use PCPs as a control group to understand whether specific
advanced training influences the understanding of MoA claims, graphics,
and associated disclosures. Further, including PCPs with oncology
experience alongside oncologists has yielded useful data in prior
studies (Ref. 3). The sample will be equally distributed across
oncologists, PCPs with oncology experience, and nurse practitioners and
physician assistants with oncology experience.
(Comment 7) One comment stated that the study should only recruit
nurse practitioners and physician assistants who specialize in
oncology.
(Response 7) We agree. Only nurse practitioners and physician
assistants who specialize in oncology are eligible for the study.
(Comment 8) One comment noted that the instructions at the top of
the questionnaire ask participants to ``make your best guess'' based on
the web page they just viewed. The comment stated that respondents
should not be asked to guess as their response and argued that these
instructions undermine the importance of the participants' answers.
(Response 8) The instructions are displayed before perceived
efficacy and risk questions where consumer participants are told,
``Most people don't know how a prescription drug will affect them until
they've taken the drug. But we'd like you to make your best guess based
on the web page you just saw. Please answer the following questions
based on what you saw on the web page.'' HCPs are told, ``Please answer
the following questions based on what you saw on the web page rather
than prior knowledge of this class of medications.''
These instructions have been cognitively tested in prior studies,
as well as in the present study, and we found no evidence that these
instructions undermined the perceived importance of participants'
answers. Instead, the instructions helped to indicate that we wanted
participants to form an opinion and that they did not need to base
their opinion on prior knowledge to do so.
(Comment 9) One comment suggested that the recall questions
(questions 6 through 11) and especially the ``foil'' responses could
bias the responses to the questions that follow them and recommended
locating the recall questions after other questions.
(Response 9) We always approach question ordering carefully,
attempting to balance several considerations, including the reduction
of bias from one question to another, the flow, and the importance of
each item. In this case, we are prioritizing measures of specific claim
comprehension over other more general questions in our questionnaire,
which is why questions 6 through 11 are placed earlier in the
questionnaire. Answering recall and comprehension questions first will
allow consumers and HCPs to provide a more accurate response and will
allow us to better understand whether the information was comprehended.
We did not encounter any issues with recall questions influencing
responses to questions found later in the survey during cognitive
interviews.
(Comment 10) One comment recommended using a consistent scale
throughout the survey. Another suggested changing questions 12, 13, 14,
16, 17, 18, 19, 20, 22, and 23 to 7-point scales to add a midpoint.
(Response 10) We use true/false/don't know or yes/no/don't know
response options for the comprehension questions and Likert-type scales
for perceptions and opinion questions. Using one scale throughout the
survey would not necessarily provide better data. For nearly all
Likert-type questions, we use 6-point scales with the endpoints
labeled. Some of these questions with Likert-type scales are validated
questions; for these, we have maintained the response options from the
validated measures. Other questions were altered from validated
measures, and similarly, we preferred to maintain the Likert-type
scales that the original measure had. We will change question 5 from a
7-point to a 6-point scale to increase consistency. We will retain the
5-point scales with all response options labeled for the two validated
scales for beliefs about medications and trust in prescription drug
materials.
Regarding the inclusion of a midpoint, this is a matter of debate
in the literature and has never been resolved. Based on input from
cognitive interviews and in response to public comments, we will be
adding a neutral point to the comparative efficacy and risk questions
(i.e., questions 17 through 23), which will change these questions to
be 7-point response options with endpoints and midpoint labeled.
(Comment 11) Two comments stated that the 6-point scales do not
allow the respondent to pick neither agree/disagree/unknown. One
comment noted that this is a concern for most 6-point scale questions
but particularly for questions 17 through 23, which compare the study
drug to other medications. The comments recommended either an anchored
neutral middle point on the scale or a box for uncertain/do not know
responses.
(Response 11) There are benefits and drawbacks to including a
neutral or ``no reaction'' response in survey research, and the
decision to use a neutral midpoint depends on the goal of the measures
(Refs. 4 and 5). For questions assessing comprehension of the MoA
claim, we included a ``do not know'' option as this response would
indicate some level of uncertainty about the MoA, and that uncertainty
itself would be meaningful and actionable information. However, when
assessing perceptions and attitudes about the claim, graphic, or
disclosure, our objective is to force a selection. Inclusion of a
neutral response option in these instances could potentially encourage
satisficing--cuing participants to select a neutral response when there
is uncertainty (Ref. 7). For the comparative risk and efficacy
questions (questions 17 through 23), we will include a midpoint based
on results from cognitive interviews; however, these interviews did not
point to the need to include a midpoint for the other questions.
(Comment 12) Questions 17 through 23 ask about the efficacy and
risks of the study drug compared to other prescription drugs for the
same indication. One comment contended that, without prior knowledge of
the efficacy and risks of the prescription drugs on the market, it
would be difficult for respondents to make a fully informed conclusion.
Another comment asserted that the comparative risk and efficacy
questions should be revised to establish a clear comparator, such as
chemotherapy. Finally, a comment recommended removing these questions
as consumers should not be assessing a drug's safety or efficacy
compared to other drugs.
(Response 12) There are instances in the clinical setting when
consumers will discuss the safety and risk information of a drug
compared to others (e.g., if a patient switches from one drug to
another or if a family member asks the consumer to talk to their doctor
about another drug). We acknowledge that in a clinical setting,
patients and HCPs may use additional information to make decisions
about how a drug compares to another. However, the intent of questions
17 through 23 is to understand whether exposure to different
presentations of the MoA claim, graphics, and disclosure results in
different comprehension or perceptions, such as perception of
comparative risks and efficacy. Except for the varied presentations,
all participants will have the same level of information regarding the
MoA of the drug. So, we would expect that all
[[Page 58114]]
participants would be equally informed of the drug, and differences
among study conditions could be attributed to the experimental
manipulations. Additionally, any subjective experiences outside the
experiment setting should be evenly distributed across study conditions
as a function of random assignment; therefore, they should not have any
impact on the outcomes of the study. Still, cognitive interviews
indicated that HCPs and consumers preferred that a midpoint be added to
the response scale for these questions, which we added in the revised
questionnaire. Based on cognitive interviews, we also revised the
questions to include the phrase ``compared to other similar
prescription drugs that are for/treat bladder cancer.'' We will also
review these questions and make any necessary adjustments based on pre-
testing results.
(Comment 13) One comment stated that the questionnaire does not
consider the HCP respondents' baseline understanding or expectations of
targeted treatments.
(Response 13) We expect that any knowledge or expectations of
targeted treatments that consumers and HCPs already have outside of the
experiment setting should be evenly distributed across study conditions
as a function of random assignment; therefore, observed differences
between conditions are unlikely to be caused by these individual
differences. However, we added an item that assesses HCPs' knowledge of
targeted therapies for cancer treatments.
(Comment 14) One comment encouraged FDA to disseminate all final
results of completed research related to this topic.
(Response 14) FDA's research is documented on our homepage, which
can be found at https://www.fda.gov/about-fda/center-drug-evaluation-and-research-cder/office-prescription-drug-promotion-opdp-research. The
website includes links to the latest Federal Register notices and peer-
reviewed publications produced by our office. The Agency also
anticipates disseminating the results of this study after the final
analyses of the data are completed. The exact timing and nature of any
such dissemination has not been determined, but dissemination of
research results often occurs through presentations at trade and
academic conferences, publications, articles, and postings on FDA's
website.
(Comment 15) One comment recommended that certain populations, such
as those who work in pharmaceutical marketing or for the U.S.
Department of Health and Human Services (HHS), be excluded from the
study.
(Response 15) We agree. Participants will be excluded from
participation if they work for a pharmaceutical, advertising, or market
research company or are employed by HHS.
(Comment 16) One comment recommended that participants who are
unable to recall key elements of the stimuli, such as indication, risk
elements, presence of claim, and presence of disclaimers, be excluded
from the study because they are not able to appropriately assess the
MoA presentations.
(Response 16) The fact that a consumer or HCP is not able to recall
certain information does not mean they did not see that information or
subconsciously process it (Ref. 6). Therefore, we do not plan to
exclude anyone based on their self-reported recall of elements in the
stimuli.
(Comment 17) One comment suggested that participants should be
asked questions 30 through 34 as part of a pre-test and be stratified
based on their responses.
(Response 17) Typically, stratified randomization is used if there
are prognostic variables that correlate with outcome measures and
researchers are concerned about such factors not being evenly
distributed across groups (Ref. 8). We have no reason to expect that
the aforementioned factors would have a strong association with the
outcome measures, nor do we have reason to believe that we will not
achieve adequate balance of prognostic variables given the large sample
size proposed for this study (Ref. 8). Random assignment will help to
produce groups that are, on average, probabilistically similar to each
other. Because randomization eliminates most other sources of
systematic variation, we can be reasonably confident that any effect
that is found is the result of the intervention and not some
preexisting differences between the groups (Ref. 9). However, we have
included questions 30 through 34 to assess the association of factors
such as health literacy, prior cancer diagnosis, or familiarity with
cancer treatment options with our outcomes and statistically control
for those variables if necessary.
(Comment 18) One comment suggested that in order to ensure that
differences in risk assessment across stimuli are due to the
manipulation of MoA information, the prominence of the risk
presentation should be standardized across the 12 versions of the
stimuli and displayed in accordance with FDA's guidance document
entitled ``Presenting Risk Information in Prescription Drug and Medical
Device Promotion.'' \2\ The comment also encouraged the use of
qualifiers to delineate which side effects are considered serious.
---------------------------------------------------------------------------
\2\ The draft guidance for industry ``Presenting Risk
Information in Prescription Drug and Medical Device Promotion'' (May
2009) is available on the FDA guidance web page at https://www.fda.gov/regulatory-information/search-fda-guidance-documents.
When finalized this guidance will represent FDA's thinking on this
issue.
---------------------------------------------------------------------------
(Response 18) In creating the stimuli, we created one web page that
was the basis for all the stimuli. The risk presentation was
standardized across the experimental conditions, and we kept FDA's
guidance in mind when displaying stimuli. Regarding the suggested use
of qualifiers to delineate which side effects are considered
``serious,'' we again note that we kept FDA's guidance in mind with
respect to the risk presentation.
(Comment 19) One comment noted that the disclosure for patients
should be reworded as follows to prevent implied bias: ``[Drug X]
delivers medicine directly to cancer cells and can also harm healthy
cells.''
(Response 19) We revised the statement to read ``[Drug X] could
also affect healthy cells.'' With this change, the consumer disclosure
is consistent with the content of the disclosure shown to HCPs.
(Comment 20) One comment asserted that most promotional materials
in the real world qualify MoA statements with language mirroring the
labeling (e.g., ``Pre-clinical studies demonstrate . . .'') and
recommended that the research materials be updated to include similar
qualifying language.
(Response 20) The addition of such language may create an imbalance
of information across the various experimental conditions and could
confound interpretation of the results. As such, we did not include the
qualifying language mentioned above.
(Comment 21) One comment suggested that study participants should
be allowed to refer back to the product website as often as needed
rather than only being permitted to view it once.
(Response 21) As a practice, we often purposely do not permit study
participants to refer back to the product website as often as needed
for these types of studies. Rather, for this study, we will instruct
participants to read the website carefully and alert them that they
will be answering several questions about the content that they just
saw and that they cannot return to the website. The goal of this study
is not to assess participants' comprehension of verbatim information in
the stimuli, for which
[[Page 58115]]
repeated exposures to stimuli may be more appropriate in another study.
Rather, the present study is interested in gist understanding of the
information. Allowing for multiple exposures to the stimuli could
potentially influence study outcomes and confound interpretation of the
study results. A large literature supports presence of a ``mere
exposure effect'' in social science research, where more exposure
enhances processing and increases positive affect toward stimuli (Refs.
10 and 11).
(Comment 22) One comment recommended removing question 16 (i.e.,
risk-benefit tradeoff) for consumers because consumers may not have the
experience or background to assess a drug's benefit-risk profile. The
comment also suggested that this question ignores the role of
prescribers in informing patients of the relevant risks and benefits of
prescription medications.
(Response 22) We disagree that consumers do not form their own
perceptions about risk-benefit tradeoffs after seeing direct-to-
consumer (DTC) promotional materials and before any discussion with an
HCP. Consumers often wish to participate in shared decision making with
HCPs when selecting prescription drugs and may request specific
prescription drugs from their HCPs based on promotions they have seen
in the marketplace. Because the information consumers receive through
DTC prescription drug promotion can impact these requests, it is
important to investigate how the information in prescription drug
promotional pieces impacts consumer attention, understanding, and
perceptions. In addition, the purpose of these questions is to assess
perceived benefit and risk based on the promotional material shown. The
question includes instructions indicating that judgments should be
reached based on the information on the prescription drug website. As
such, we plan to ask participants about their perceptions of the risk-
benefit tradeoff using question 16, which is a common and validated
item in DTC research.
FDA estimates the burden of this collection of information as
follows:
Table 2--Estimated Annual Reporting Burden 1
----------------------------------------------------------------------------------------------------------------
Number of Number of
Activity respondents 2 responses per Total annual Average burden per Total hours
respondent responses response 3
----------------------------------------------------------------------------------------------------------------
Pretest:
General population: 528 1 528 0.08 (5 min.)........ 42.2
pretest screener
completes (assumes 75%
eligible).
General population: 396 1 396 0.33 (20 min.)....... 130.7
number of completes,
pretest.
HCP: pretest screener 660 1 660 0.08 (5 min.)........ 52.8
completes (assumes 60%
eligible).
HCP: number of 396 1 396 0.33 (20 min.)....... 130.7
completes, pretest.
Main Study:
General population: 792 1 792 0.08 (5 min.)........ 63.4
number of main study
screener completes
(assumes 75% eligible).
General population: 594 1 594 0.33 (20 min.)....... 196.0
number of completes,
main study.
HCP: number of main 990 1 990 0.08 (5 min.)........ 79.2
study screener
completes (assumes 60%
eligible).
HCP: number of 594 1 594 0.33 (20 min.)....... 196.0
completes, main study.
-----------------------------------------------------------------------------------
Total............... .............. .............. .............. ..................... 891
----------------------------------------------------------------------------------------------------------------
1 There are no capital costs or operating and maintenance costs associated with this collection of information.
2 As with most online and mail surveys, it is always possible that some participants are in the process of
completing the survey when the target number is reached and that those surveys will be completed and received
before the survey is closed out. To account for this, we have estimated approximately 10 percent overage for
both samples in the study.
3 Burden estimates of less than 1 hour are expressed as a fraction of an hour in decimal format.
II. References
The following references marked with an asterisk (*) are on display
at the Dockets Management Staff (see ADDRESSES) and are available for
viewing by interested persons between 9 a.m. and 4 p.m., Monday through
Friday; they also are available electronically at https://www.regulations.gov. References without asterisks are not on public
display at https://www.regulations.gov because they have copyright
restriction. Some may be available at the website address, if listed.
References without asterisks are available for viewing only at the
Dockets Management Staff. FDA has verified the website addresses, as of
the date this document publishes in the Federal Register, but websites
are subject to change over time.
1. O'Donoghue, A.C., Williams, P.A., Sullivan, H.W., et al.
``Effects of Comparative Claims in Prescription Drug Direct-to-
Consumer Advertising on Consumer Perceptions and Recall.'' Social
Science & Medicine, 120:1-11, 2014.
*2. Sullivan, H.W., O'Donoghue, A.C., Gard Read, J., et al.
``Testimonials and Informational Videos on Branded Prescription Drug
websites: Experimental Study to Assess Influence on Consumer
Knowledge and Perceptions.'' Journal of Medical internet Research,
20(1):e13, 2018.
3. Boudewyns, V., O'Donoghue, A.C., Paquin, R.S., et al. ``Physician
Interpretation of Data of Uncertain Clinical Utility in Oncology
Prescription Drug Promotion.'' The Oncologist, 26(12):1071-1078,
2021.
4. Moors, G. ``Exploring the Effect of a Middle Response Category on
Response Style in Attitude Measurement.'' Quality & Quantity,
42(6):779-794, 2008.
5. Sturgis, P., Roberts, C., and Smith, P. ``Middle Alternatives
Revisited: How the Neither/nor Response Acts as a Way of Saying `I
Don't Know?''' Sociological Methods & Research, 43(1):15-38, 2014.
6. Shapiro, S. and Krishnan, H.S. ``Memory-Based Measures for
Assessing Advertising Effects: A Comparison of Explicit and Implicit
Memory Effects.'' Journal of Advertising, 30(3):1-13, 2001.
7. Krosnick, J.A. ``Questionnaire Design.'' In: Vannette, D.,
Krosnick, J. (eds). The Palgrave Handbook of Survey Research (pp.
439-455). Palgrave Macmillan, Cham, 2018.
8. Friedman, L.M., Furberg, C.D., DeMets, D.L., et al. Fundamentals
of Clinical Trials. 5th ed. New York, NY: Spring Science-Business
Media, 1998.
9. Fisher, R.A. The Design of Experiments. Edinburgh, United
Kingdom: Oliver & Boyd, 1935.
10. Bornstein, R.F. ``Exposure and Affect: Overview and Meta-
analysis of Research, 1968-1987.'' Psychological Bulletin,
106(2):265-289, 1989.
[[Page 58116]]
11. Bornstein, R.F. and D'Agostino, P.R. ``The Attribution and
Discounting of Perceptual Fluency: Preliminary Tests of a Perceptual
Fluency/Attributional Model of the Mere Exposure Effect.'' Social
Cognition, 12(2):103-128, 1994.
Dated: September 19, 2022.
Lauren K. Roth,
Associate Commissioner for Policy.
[FR Doc. 2022-20623 Filed 9-22-22; 8:45 am]
BILLING CODE 4164-01-P
