Agency Information Collection Activities; Submission for Office of Management and Budget Review; Comment Request; Disclosures of Descriptive Presentations in Professional Oncology Prescription Drug Promotion, 56845-56852 [2018-24785]

Agencies

• Food and Drug Administration
• DEPARTMENT OF HEALTH AND HUMAN SERVICES

[Federal Register Volume 83, Number 220 (Wednesday, November 14, 2018)]
[Notices]
[Pages 56845-56852]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2018-24785]


=======================================================================
-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Food and Drug Administration

[Docket No. FDA-2017-N-1779]


Agency Information Collection Activities; Submission for Office 
of Management and Budget Review; Comment Request; Disclosures of 
Descriptive Presentations in Professional Oncology Prescription Drug 
Promotion

AGENCY: Food and Drug Administration, HHS.

ACTION: Notice.

-----------------------------------------------------------------------

SUMMARY: The Food and Drug Administration (FDA or Agency) is announcing 
that a proposed collection of information has been submitted to the 
Office of Management and Budget (OMB) for review and clearance under 
the Paperwork Reduction Act of 1995.

DATES: Fax written comments on the collection of information by 
December 14, 2018.

ADDRESSES: To ensure that comments on the information collection are 
received, OMB recommends that written comments be faxed to the Office 
of Information and Regulatory Affairs, OMB, Attn: FDA Desk Officer, 
Fax: 202-395-7285, or emailed to [email protected]. All 
comments should be identified with the OMB control number 0910-NEW and 
title ``Disclosures of Descriptive Presentations in Professional 
Oncology Prescription Drug Promotion.'' Also include the FDA docket 
number found in brackets in the heading of this document.

FOR FURTHER INFORMATION CONTACT: Ila S. Mizrachi, Office of Operations, 
Food and Drug Administration, Three White Flint North, 10A-12M, 11601 
Landsdown St., North Bethesda, MD 20852, 301-796-7726, 
[email protected].

SUPPLEMENTARY INFORMATION: In compliance with 44 U.S.C. 3507, FDA has 
submitted the following proposed collection of information to OMB for 
review and clearance.

Disclosures of Descriptive Presentations in Professional Oncology 
Prescription Drug Promotion

OMB Control Number--0910-NEW

I. Background

    Section 1701(a)(4) of the Public Health Service Act (42 U.S.C. 
300u(a)(4)) authorizes FDA to conduct research relating to health 
information. Section 1003(d)(2)(C) of the Federal

[[Page 56846]]

Food, Drug, and Cosmetic Act (FD&C Act) (21 U.S.C. 393(d)(2)(C)) 
authorizes FDA to conduct research relating to drugs and other FDA 
regulated products in carrying out the provisions of the FD&C Act.
    Under the FD&C Act and implementing regulations, promotional 
labeling and advertising about prescription drugs are generally 
required to be truthful, non-misleading, and to reveal facts material 
to the presentations made about the product being promoted (see 
sections 502(a) and (n), and 201(n) of the FD&C Act (21 U.S.C. 352(a) 
and (n), and 321(n)); see also 21 CFR 202.1). As a part of the ongoing 
evaluation of FDA's regulations in this area, FDA is proposing to study 
the impact of disclosures as they relate to presentations of 
preliminary and/or descriptive scientific and clinical data in 
promotional labeling and advertising for oncology products. The use of 
disclosures is one method of communicating information to healthcare 
professionals about scientific and clinical data, the limitations of 
that data, and practical utility of that information for use in 
treatment. These disclosures may influence prescriber comprehension and 
decision making and may affect how and what treatment they prescribe 
for their patients.
    Pharmaceutical companies market directly to physicians through 
means that include publishing advertisements in medical journals, 
exhibit booths at physician meetings or events, sending unsolicited 
promotional materials to doctors' offices, and presentations 
(``detailing'') by pharmaceutical representatives (Ref. 1). Research 
suggests that detail aids sometimes contain carefully extracted data 
from clinical studies that, taken out of context, can exaggerate the 
benefits of a drug (Ref. 2) or contribute to physicians prescribing the 
drug for an inappropriate patient population.
    Promotional labeling and advertising for cancer drugs deserve 
specific attention. Oncology drugs represented 26 percent of the 649 
compounds under clinical trial investigation from 2006 to 2011 (Ref. 
3). The past decade has seen a dramatic rise in the number of oncology 
drugs brought to market. In the past 18 months, over 22 percent of new 
drug approvals at FDA were new cancer drugs. In that time period, FDA 
approved 16 cancer drugs as new molecular entities or new therapeutic 
biologics out of a total of 72 (this does not include approvals of 
benign hematology products or biological license application approvals 
of blood reagents, or assays and anti-globulin products used in testing 
kits) (Refs. 4 and 5). Although overall survival remains the gold 
standard for demonstrating clinical benefit of a cancer drug, several 
additional endpoints including progression free survival, disease-free 
or recurrence-free survival, or durable response rate (including 
hematologic response endpoints) are accepted for either regular or 
accelerated approval depending on the magnitude of effect, safety 
profile, and disease context (Ref. 6). In addition to the endpoints 
upon which FDA approval of these products may be based, pharmaceutical 
companies typically assess many other endpoints to further explore the 
effects of their products. Some trials are designed to allow for formal 
statistical analyses of these additional endpoints; however, in many 
cases these endpoints are strictly exploratory and support only the 
reporting of descriptive results. For clinicians who are not 
specifically trained in clinical trial design, interpreting these 
endpoints may be challenging. Pharmaceutical companies invest heavily 
in the development and distribution of promotional materials to make 
oncologists aware of favorable clinical trial results.
    When communicating scientific and clinical data, a specific 
statement that modifies or qualifies a claim (referred to for the 
purposes of this document as a disclosure) could be used to convey the 
limitations of the data and practical utility of the information for 
treatment. Much of the prior research on disclosures in this topic area 
has been limited to the dietary supplement arena with consumers (Refs. 
7 to 10). Disclosures in professional pieces could influence prescriber 
comprehension as well as subsequent decision making; however, no 
published data exist regarding how prescribers use and understand 
scientific claims in conjunction with qualifying disclosures.
    The proposed study seeks to address the following research 
questions:
    1. Do disclosures mitigate potentially misleading presentations of 
preliminary and/or descriptive data in oncology drug product promotion?
    2. Does the language (technical, non-technical) of the disclosure 
influence the effectiveness of the disclosure?
    3. Does the presence of a general statement about the clinical 
utility of the data in addition to a specific disclosure influence 
processing of claims and disclosures?
    4. Do primary care physicians (PCPs) and oncologists differ in 
their processing of claims and disclosures about preliminary and/or 
descriptive data?
    5. Which disclosures do physicians prefer?
    To address these questions, FDA has designed a study that will be 
conducted in three independent phases, each phase examining a data 
display in a promotional piece for a unique oncology or hematology 
product. Independent variables will include: (1) Specific disclosure 
(technical, non-technical, none), (2) general statement (present, 
absent), and (3) specialty (PCPs, oncologists). Each phase will have 
the following design:

----------------------------------------------------------------------------------------------------------------
                                                                         Specific disclosure
             Sample               General statement ------------------------------------------------------------
                                                         Technical       Non-technical        No disclosure
----------------------------------------------------------------------------------------------------------------
Oncologists....................  Present...........                      Control.
                                 Absent............                      .......................
PCPs...........................  Present...........                      Control.
                                 Absent............                      .......................
----------------------------------------------------------------------------------------------------------------

    Specific disclosures will include material information specifically 
related to the particular data display in question. As such, each 
specific disclosure may include clinical or statistical information 
related to the trial design, the statistical analysis plan of the 
trial, or any other material statistical or clinical information 
necessary for evaluation or interpretation of the data. The team 
developing the disclosures includes social science analysts, 
pharmacists, oncological medical officers, statisticians, and an 
oncology nurse. An example of the general statement is ``This 
presentation includes exploratory information of uncertain clinical 
utility and should be interpreted cautiously when used to make 
treatment decisions.''

[[Page 56847]]

    Outcome (dependent) variables will focus on the assessment of the 
data display as a whole as well as attention to the disclosure, if 
present. Specifically, we will examine recognition of the clinical 
endpoint in the data display, comprehension of the data display, 
perceptions of the strength of the data, and the perceived credibility 
of the promotional piece. We will also look at attention to the 
specific disclosure and the general statement, prescriber decisions, 
and prescriber preferences. Preferences will be determined by a 
secondary task at the end of the questionnaire that shows each 
participant all disclosure options and asks them to choose their 
preferred version.
    Oncologists and PCPs will be recruited to participate via the 
internet. We plan to conduct one pretest with 90 participants and one 
study with 2,115 participants, both of which are expected to take 
approximately 20 minutes. Voluntary participants will view 
professionally developed promotional pieces that mimic currently 
available promotion and answer questions.
    In the Federal Register of Monday, June 19, 2017 (82 FR 27845), FDA 
published a 60-day notice requesting public comment on the proposed 
collection of information (see above). Comments received along with our 
responses to the comments are provided below. Comments that are not 
PRA-relevant or do not relate to the proposed study are not included. 
For brevity, some public comments are paraphrased and therefore may not 
reflect the exact language used by the commenter. We assure commenters 
that the entirety of their comments was considered even if not fully 
captured by our paraphrasing. The following acronyms are used here: FRN 
= Federal Register Notice; DTC = direct-to-consumer; HCP = healthcare 
professional; PCP = primary care physicians; FDA = Food and Drug 
Administration; OPDP = FDA's Office of Prescription Drug Promotion.
    The first public comment responder (regulations.gov tracking number 
1k1-8xz7-mwcd) included eight individual comments, to which we have 
responded.
    Comment 1: ``It is unclear why FDA has chosen to conduct a study 
focused on oncology therapeutics and those medical specialists who 
prescribe such products.'' [verbatim] All prescription drug products 
are treated the same according to regulations; therapeutic intent and 
prescriber type do not invoke alternate regulatory approaches.
    Response: As we described in the 60-day Federal Register notice, 
promotional activities for oncology drugs are frequent and pervasive. 
Promotional labeling and advertising for cancer drugs deserve specific 
attention. Oncology drugs represented 26 percent of the 649 compounds 
under clinical-trial investigation from 2006 to 2011 (Ref. 3). The past 
decade has seen a dramatic rise in the number of oncology drugs brought 
to market. In the past 18 months, over 22 percent of new drug approvals 
at FDA were new cancer drugs. In that time period, FDA approved 16 
cancer drugs as new molecular entities or new therapeutic biologics out 
of a total of 72 (this does not include approvals of benign hematology 
products or biological license application approvals of blood reagents, 
or assays and anti-globulin products used in testing kits) (Refs. 4 and 
5). Although overall survival remains the gold standard for 
demonstrating clinical benefit of a cancer drug, several additional 
endpoints including progression free survival, disease-free or 
recurrence-free survival, or response rate (including hematologic 
response endpoints) are accepted for either regular or accelerated 
approval depending on the magnitude of effect, safety profile, and 
disease context (Ref. 6). In addition to the endpoints upon which FDA 
approval may be based, pharmaceutical companies typically assess many 
other endpoints to further explore the effects of their products. Some 
trials are designed to allow for formal statistical analyses of these 
additional endpoints; however, in many cases these endpoints are 
strictly exploratory and support only the reporting of descriptive 
results. For clinicians who are not specifically trained in clinical 
trial design, interpreting these endpoints can be challenging. 
Pharmaceutical companies invest heavily in the development and 
distribution of promotional materials to educate oncologists about 
favorable clinical trial results.
    As another public comment responder (regulations.gov tracking 
number 1k1-8y3p-o6qb) notes, ``We agree with the FDA's assessment that 
dedicated research is necessary regarding oncology drug promotion, 
particularly given that a significant proportion of the drug 
development pipeline is comprised of oncology products . . .''
    Comment 2: FDA should use a more targeted approach, including a 
monadic design with 100 oncologists split into two experimental 
conditions.
    Response: To clarify the study design, we are testing two 
variations of disclosure (specific disclosure: Technical, non-
technical), two variations of general statement (general statement: 
Present or absent), plus a control (control: No specific disclosure). 
Participants will be healthcare professionals who are members of one of 
two medical populations and will be randomly assigned to one condition. 
Because we are examining the effects of multiple variables and their 
interactions, the necessary sample sizes will be larger than those 
suggested in this comment based on power analyses. We have, however, 
changed the study design based on multiple comments and will now 
examine only oncologists and primary care physicians.
    Comment 3: The length of the survey looks long--at 17 pages, it 
appears that it will take approximately 30-40 minutes to complete.
    Response: We have tested the survey in-house with individuals 
unfamiliar with the research project, and it appears that this survey 
will take approximately 15 minutes to complete.
    Comment 4: Instead of using recall as a measure, respondents should 
be allowed to have access to the materials while answering questions to 
better approximate their actual experiences.
    Response: It is an open question as to whether having the materials 
in front of them better approximates actual HCP experiences. In past 
discussions with HCPs, some have reported that they do refer back to 
materials that sales representatives leave, and others report that they 
do not receive leave-behind materials or do not refer to them again. In 
any case, we have a mixture of recall and comprehension questions in 
our questionnaire. For the recall questions, respondents will not be 
able to access the materials. They will, however, be able to review the 
materials while answering the comprehension questions.
    Comment 5: Why is FDA examining non-oncologists at all? Why are you 
screening out oncology for specialists in question SPECIALTY2?
    Response: HCPs of all types are exposed to prescription drug 
promotion. Depending on location (e.g., rural areas) and type of 
clinical setting, some non-oncologists may have a need to consider 
oncologic prescription drugs to treat their patients. We agree that 
oncologists are the most relevant population to study in this research. 
However, we also want to know whether specific education and experience 
influence the processing of claims, data, and disclosures. Upon further 
review, we agree that nurse practitioners and physician assistants 
without oncology experience are not a necessary group to investigate to 
answer our particular research questions. We intend to use PCPs as a 
control group to understand whether specific advanced training

[[Page 56848]]

influences the understanding of preliminary and/or descriptive oncology 
data. Some PCPs may have experience with oncology prescriptions, 
particularly in rural areas. We will not eliminate PCPs without 
oncology experience, but we will measure oncology prescribing 
experience and use this variable as a covariate in our studies.
    Comment 6: FDA should screen for the prescribing of oncologic 
products.
    Response: Although we do not intend to screen out physicians 
without oncology prescribing experience, we will measure this variable 
and use this information to determine whether it plays a role in the 
responses of PCPs.
    Comment 7: From this point (ENDPOINT) responses may be based on the 
ability of respondents to recall information vs. the absence/presence 
of disclosures. If FDA continues with this design, the Agency should be 
prepared to control for this in the study design.
    Response: Because this is an experimental design with random 
assignment to condition, any fatigue with questions that may affect the 
recall of information should fall out evenly across conditions. 
Therefore, any differences would be the result of our manipulations, in 
this case, the presence and form of disclosures. We have given thought 
to the ordering of the questions so that the most important questions 
are asked in the beginning of the survey rather than toward the end.
    Comment 8: The answer to this question (CAUTIOUS) may be influenced 
more by personal and subjective opinion vs. the content of the 
disclosure.
    Response: Because of the experimental design with random assignment 
to condition, personal and subjective opinions should be evenly and 
randomly spread across experimental conditions. However, upon further 
review, we have determined that this question has limited utility and 
we will delete it.
    The second public comment responder (regulations.gov tracking 
number 1k1-8y3p-o6qb) included one individual comment. They reported 
that they support the study specifically and OPDP's overall research 
efforts generally, and they agree that oncology deserves special 
attention. We thank this commenter for taking the time to provide this 
comment to us.
    The third public comment responder (regulations.gov tracking number 
1k1-8y5u-5vp0) included eight individual comments, to which we have 
responded.
    Comments 1 and 2: The commenter supports FDA social science 
research and this specific project, as well as the Disclosures study 
(Docket No. FDA-2017-N-0558). ``FDA's collective research indicates a 
considered, objective updating of the FDA's advertising regulations, 
including the use of disclosures to prevent misleading claims in 
advertisements for oncology products, is timely . . . . Enabling 
disclaimers would be one way to enable innovators to advertise new 
oncology therapeutics for their approved uses in ways which would be 
non-misleading.''
    Response: Thank you for your support.
    Comment 3: The commenter suggests making sure that primary care 
physicians and advanced practitioners have experience in the oncology 
field--otherwise, it seems useless to include less knowledgeable 
respondents whose answers are more speculative. Overall, they question 
whether advanced practitioners are appropriate for this study at all.
    Response: We have removed advanced practitioners from the design. 
We will measure the oncology prescribing experience of the PCPs in our 
sample, but we will not eliminate those who do not have specific 
oncology training. One of our research questions is whether specific 
training and experience in oncology influences the understanding of 
preliminary oncology data. To do that, we need to include a group of 
practitioners who may not have specific training and experience in 
oncology, but who are licensed practitioners permitted by law to 
prescribe oncology drugs, and who, in some cases, may do so.
    Comment 4: If the only data being presented for BENEFICIAL, 
EVIDENCE1 and EVIDENCE2 are the endpoints for the disclosure without 
presenting overall survival or more clinically validated data, we 
suggest removing these questions.
    Response: The pieces include other clinically validated data as 
would be typical in an existing piece for an oncology indication.
    Comment 5: Remove CONFUSING2 because it asks physicians to 
speculate.
    Response: As this item is a perception measure, as opposed to an 
accuracy measure, it is reasonable to consider some level of 
speculation. Moreover, in cognitive testing, HCPs responded without 
difficulty.
    Comment 6: For SCRIPT4, add an ``I don't know'' option instead of 
instructing respondents to ``make your best guess.''
    Response: This item was cognitively tested and participants 
expressed no difficulty answering it.
    Comment 7: Those who respond ``not at all familiar'' to FAMILIAR 
should skip BTKNOW1, BTKNOW2, and ACCEL.
    Response: We agree with this comment. Those who respond ``not at 
all familiar'' to FAMILIAR will skip the three items mentioned above.
    Comment 8: BTDV1 and BTDV2 present incomplete data and therefore it 
is unclear how this will be a useful question. The commenter suggests 
either adding an ``I need more information'' option or removing the 
question.
    Response: These items present incomplete data but we have provided 
enough data that HCPs should be able to make a choice. HCPs in 
cognitive testing exhibited no difficulty with the question. There is 
no existing data on perceptions of FDA's ``breakthrough'' designation 
and this item will provide at least rudimentary information. Please 
note that each respondent will see only one of the items. These items 
are carefully crafted to avoid order effects and alphabetical effects.
    The fourth public commenter (regulations.gov tracking number 1k1-
8y5u-koc0) included 15 individual comments, to which we have responded.
    Comment 1 (summarized): The commenter is concerned with the 
Agency's recent approaches to studies in this area. FDA has proposed to 
undertake projects in a variety of disparate topics without 
articulating a clear, overarching research agenda or adequate 
rationales on how the proposed research related to the goal of further 
protecting public health. Within the last year, the Agency has 
increased such efforts at an exponential pace. At times, FDA proposes 
new studies seemingly without fully appreciating its own previous 
research published on the Office of Prescription Drug Promotion (OPDP) 
website. Proposed studies are often unnecessary in light of existing 
data. The commenter suggests that the Agency publish a comprehensive 
list of its prescription drug advertising and promotion studies from 
the past five years and articulate a clear vision for its research 
priorities for the near future. Going forward, FDA should use such 
priorities to explain the necessity and utility of its proposed 
research and should provide a reasonable rationale for the proposed 
research.
    Response: OPDP's mission is to protect the public health by helping 
to ensure that prescription drug information is truthful, balanced, and 
accurately communicated, so that patients and healthcare providers can 
make informed decisions about

[[Page 56849]]

treatment options. OPDP's research program supports this mission by 
providing scientific evidence to help ensure that our policies related 
to prescription drug promotion will have the greatest benefit to public 
health. Toward that end, we have consistently conducted research to 
evaluate the aspects of prescription drug promotion that we believe are 
most central to our mission, focusing in particular on three main topic 
areas: Advertising features, including content and format; target 
populations; and research quality. Through the evaluation of 
advertising features we assess how elements such as graphics, format, 
and disease and product characteristics impact the communication and 
understanding of prescription drug risks and benefits; focusing on 
target populations allows us to evaluate how understanding of 
prescription drug risks and benefits may vary as a function of 
audience; and our focus on research quality aims at maximizing the 
quality of research data through analytical methodology development and 
investigation of sampling and response issues. Because we recognize the 
strength of data and the confidence in the robust nature of the 
findings is improved through the results of multiple converging 
studies, we continue to develop evidence to inform our thinking. We 
evaluate the results from our studies within the broader context of 
research and findings from other sources, and this larger body of 
knowledge collectively informs our policies as well as our research 
program. Our research is documented on our homepage, which can be found 
at: https://www.fda.gov/aboutfda/centersoffices/officeofmedicalproductsandtobacco/cder/ucm090276.htm. The website 
includes links to the latest Federal Register notices and peer-reviewed 
publications produced by our office. The website maintains information 
on studies we have conducted, dating back to a survey of DTC attitudes 
and behaviors conducted in 1999.
    Comment 2: FDA should provide more detail about the study to 
stakeholders. ``It is not clear from this description whether the study 
will yield useful information to evaluate whether disclosures provide 
appropriate contextual information in certain communications, whether 
such disclosures can be made more effective, and where the disclosures 
are necessary to ensure communications are truthful and non-
misleading.''
    Response: We have described the purpose of the study, the design, 
the population of interest, and have provided the questionnaire to 
numerous individuals upon request. These materials have proven 
sufficient for others to comment publicly, and for academic experts to 
peer-review the study successfully. Our full stimuli are under 
development during the PRA process. We do not make draft stimuli public 
during this time because of concerns that this may contaminate our 
participant pool and compromise the research.
    Comment 3: The Agency should wait until it has completed its 
broader study on disclosures more generally. This study is duplicative 
of other studies.
    Response: As we discussed in the 60-day Federal Register notice, 
oncological products deserve specific attention as they account for 
nearly a quarter of new drug approvals and can involve the assessment 
of complicated endpoints. Moreover, they have specific disclosures that 
are unique to their products and deserve particular study. The other 
disclosures study (Docket No. FDA-2017-N-0558) will provide important 
information about a variety of disclosures in different medical 
conditions. One research study cannot answer all questions or study all 
aspects of an issue. These two studies will be complementary but not 
redundant. Please also refer to our response to comment 1 from the 
first commenter above.
    Comment 4: Given that FDA grants approval based on certain 
preliminary and descriptive data, and that various limitations as to 
the underlying data must already be communicated to prescribers, there 
appears to be limited utility in researching disclosures regarding such 
data.
    Response: We disagree that FDA grants approval on preliminary or 
descriptive data. The evidentiary standard is substantial evidence. 
While we recognize that no single development program can answer all 
questions about a particular drug in all populations, it is not 
accurate to describe the evidence supporting approval as descriptive or 
preliminary. What is potentially unique about oncology products is that 
many are approved under accelerated approval, in which the substantial 
evidence of benefit is on a surrogate endpoint that is reasonably 
likely to predict a clinical outcome. There remains some residual 
uncertainty regarding whether the effect on a surrogate endpoint will 
directly correlate with a clinical benefit; however, there is a 
requirement that confirmatory evidence of clinical benefit be obtained 
after approval. This residual uncertainty about the relationship of the 
surrogate endpoint to the clinical benefit is communicated to 
prescribers through the FDA-required labeling (e.g., inclusion of a 
limitation of use in the Indications and Usage section of the FDA-
required labeling). In addition, reliance on a surrogate endpoint under 
accelerated approval is only done for serious diseases when the 
evidence indicates that the product provides a meaningful therapeutic 
benefit to patients over existing treatments (21 CFR 314.500).
    However, this study does not focus on endpoints that formed the 
basis for approval. This study focuses on promotional displays of 
preliminary and/or descriptive data. It has not been established 
whether and how current disclosure-type additions to promotion are 
adequately communicating the limitations around this type of data, and 
that is the purpose of the current study. Given the importance of these 
limitations, it is crucial to make sure that promotional materials 
directed at to prescribers convey limitations appropriately. Past 
research has shown that simply including a statement somewhere in a 
promotional piece does not grant it automatic usefulness (Refs. 7 to 
10).
    Comment 5: FDA notes that, ``[a]lthough overall survival remains 
the gold standard for demonstrating clinical benefit of a drug, several 
additional endpoints are accepted as surrogates . . . [including] 
disease-free survival, objective response rate, complete response rate, 
progression-free survival, and time to progression.'' The Agency 
further states that ``[f]or clinicians who are not specifically trained 
in clinical trial design, interpreting these endpoints may be 
challenging.'' FDA does not cite any sources for this claim, and there 
is no basis for thinking that clinicians do not have a thorough 
understanding of the data limitations described in presentations of 
preliminary or descriptive scientific and clinical data. This is 
especially true of oncologists.
    Response: This statement was not intended to be a claim, but rather 
a statement of concern. Studies report that physicians lack sufficient 
critical knowledge and skills to evaluate evidence based medicine (EBM) 
and may be influenced by the way study results are presented (Refs. 11 
to 13). FDA recently conducted a systematic review of research related 
to prescribers' training and critical appraisal skills related to 
clinical trials (Ref. 14). The study found that extant physician 
knowledge and skills regarding certain statistical concepts and trial 
designs were in the middle of the possible outcome score range, at 
levels below those considered mastery, even after

[[Page 56850]]

interventions designed to increase knowledge and skills. Evidence 
suggested that clinical credentials affect understanding and use of 
clinical data. Physicians with formal training in biostatistics, 
epidemiology, clinical research, or EBM demonstrated higher levels of 
knowledge and appraisal skills than those with usual medical education 
and training.
    Comment 6: The specific disclosures outlined by FDA include 
``clinical or statistical information related to the trial design, the 
statistical analysis plan of the trial, or any other material 
statistical or clinical information necessary for evaluation or 
interpretation of the data.'' The breadth of the proposed specific 
disclosures appears burdensome, unnecessary, and overwhelming for the 
purposes of the proposed survey.
    Response: These concepts were provided as examples of the types of 
information that may be necessary for the accurate evaluation or 
interpretation of the data. This statement was not meant to imply that 
all of these concepts would be included in disclosures used in this 
study.
    Comment 7: PCPs and non-oncology mid-level practitioners will 
provide much less utility in their survey responses regarding such 
disclosures.
    Response: We have changed the design. See previous comments and 
responses.
    Comment 8: The Agency proposes to conduct its survey via electronic 
media. FDA should consider testing non-electronic media, including 
printed sales aids, as these forms are often reviewed by the proposed 
study subjects.
    Response: To clarify, the stimuli presented will consist of mock 
print materials in .pdf format, administered via the internet. 
Conducting the study in person would require a greater expenditure of 
resources without appreciable benefits.
    Comment 9: The Agency should consider using a consistent sliding 
scale format for all survey responses. Just within pages 7-9 of the 
survey, FDA proposes numerous different schemes for survey responses: 
(1) ``Not at all beneficial--Extremely beneficial;'' (2) ``Completely 
agree--Completely disagree;'' (3) ``No evidence--Strong (or conclusive) 
evidence;'' (4) ``Not at all complex--Extremely complex;'' (5) ``Not at 
all confusing--Extremely confusing;'' and (6) additional responses in 
which subjects are asked to agree with certain statements. The variety 
in response options is confusing in format and could potentially 
introduce error. To the extent possible, FDA should make the response 
format consistent throughout the survey. Further, the Agency should 
ensure the sliding scale format consistently provides an odd number of 
responses to permit a ``neutral'' response. Certain questions (e.g., 
the IMPROVE question on page 7) provide six choices, not permitting a 
neutral response.
    Response: Although one scale throughout would be easier for 
respondents, it will not necessarily provide better data. When a series 
of adjacent questions have the same response options, respondents may 
use a response mechanism known as anchoring and adjusting when 
reporting (Ref. 15). Respondents use their response to the initial 
survey question on a topic as the ``cognitive anchor,'' and then adjust 
up or down based on subsequent questions (Ref. 16). Anchoring and 
adjusting is more likely to occur for questions when respondents have 
some level of uncertainty in their answer (Ref. 17), which would be 
expected in this study. Epley and Gilovich (Ref. 17) found that when 
respondents use an anchoring and adjusting strategy, they often adjust 
insufficiently. Respondents start with the response they used for the 
first item and then search for the next value that is ``close enough.'' 
This can result in responses to adjacent items being more similar than 
responses to the same items if they used an item-specific scale (Not at 
all beneficial to Extremely beneficial; Not at all complex to Extremely 
complex). Using the same scale across all survey questions would 
artificially increase the correlations between all questions making it 
more difficult to identify differences based on the stimuli or 
respondent characteristics. Furthermore, use of item-specific scales 
compared with agree-disagree scales reduces primacy effects (tendency 
of respondents to select options at the beginning of the list) (Ref. 
18), and increases reliability and validity (Ref. 19). Careful 
consideration was made to use agree-disagree scales only when item-
specific scales would not be appropriate (e.g., presenting patient 
vignettes) or unnecessarily complex (e.g., asking about ``complex 
terminology, statistical terms, or jargon,'' inquiring about ``strong'' 
evidence).
    In terms of neutral points, given the focus of the questions, we 
believe that offering a neutral response option is not necessary to 
measure opinions and attitudes accurately. Consequently, our objective 
is to force a selection and have participants make at least a weak 
commitment in either a positive or negative direction. Of concern is 
that offering a neutral midpoint could potentially encourage 
``satisficing''--cuing participants to choose a neutral response 
because it is offered (Ref. 20). Additionally, providing a midpoint 
leads to the loss of information regarding the direction in which 
people lean (Ref. 21). Research has found that neither format (either 
with or without a neutral point) is necessarily better or produces more 
valid or reliable results (Ref. 22). Instead, it should be left to the 
researcher to determine the goals of the study. During cognitive 
testing, a majority of participants were satisfied with the response 
options and all participants felt comfortable choosing a response in 
the absence of a midpoint.
    Use of a midpoint is an issue we have examined in previous studies 
and we determined that we achieve valid and reliable responses without 
a midpoint. To increase consistency with measures used in previous 
studies, and in support of the arguments presented above, we are opting 
to exclude a midpoint. Finally, if a participant does not feel that 
they can choose a response because of a lack of a neutral option, they 
will be able to skip the question.
    Comment 10: In the BENEFICIAL question on page 7 of the survey, it 
is unclear what relevance the subject's perception of clinical benefit 
of a drug has in studying FDA's proposed research purpose.
    Response: For prescription drug products, advertisers must ensure 
that both the benefits and limitations are appropriately conveyed. If 
limitations are not appropriately conveyed, viewers may have an 
inflated view of the benefits of the product, relative to its risks. 
This question investigates this issue.
    Comment 11: In a study setting, subjects may be prone to read and 
pay attention to more or all of the information presented. Subjects 
also are more aware of the importance of their responses. The Agency 
should address what efforts it will take to avoid response bias by 
study subjects.
    Response: We initially had this concern many years ago when OPDP 
began conducting research. However, since that time, we have seen no 
evidence of this bias. In fact, we often deal with the opposite 
problem--ensuring that respondents spend a minimum amount of time 
looking at mock materials. Moreover, cognitive testing participants 
have told us that they would not spend extra time on materials if they 
were answering questions without an interviewer in the room. 
Individuals, especially HCPs, are busy, and we believe our experiments 
do not overestimate the amount of time participants spend on actual 
materials.

[[Page 56851]]

    Comment 12: Although the draft survey did not contain Informed 
Consent text, the Agency should ensure that this text does not state or 
imply that the survey is being conducted on behalf of the U.S. Food and 
Drug Administration. Such a statement could potentially influence 
subjects' responses to study questions. Instead, this information might 
be provided at the conclusion of the study.
    Response: We will ensure that all materials reference the U.S. 
Department of Health and Human Services rather than FDA.
    Comment 13: The CAUTIOUS question on page 8 should be rephrased or 
omitted. Subjects may be biased to respond that they interpret all data 
with caution, regardless of the underlying scientific evidence 
presented in study stimuli.
    Response: We agree with this comment and will delete this item.
    Comment 14: The DECISIONS question on page 8 should be omitted. How 
survey participants ``feel about the data presented'' will be highly 
dependent on their external experience in making prescribing decisions. 
This question thus may lead to highly variable results.
    Response: Because this is an experimental design with random 
assignment to conditions, external experiences in making prescribing 
decisions should be randomly scattered across experimental conditions. 
Thus, we will be able to infer causation to our manipulations of 
disclosures if we find any differences across experimental conditions. 
We believe the presence and form of the disclosure may influence this 
dependent variable and believe it will reveal important information 
about how HCPs process the data.
    Comment 15: The PREFERENCE and PREFERWHY questions on page 16 
should be moved to the beginning of the survey or omitted altogether. 
Subjects' responses regarding their preference in sales aid disclosure 
statements will be heavily influenced by earlier portions of the 
survey.
    Response: We have given careful thought to the ordering of the 
questions in the questionnaire. Because preference is of secondary 
interest to us, we have included it after our primary outcome 
variables, so that it does not influence them. We recognize that prior 
questions may influence these measures and will interpret them with 
that caveat in mind.
    FDA estimates the burden of this collection of information as 
follows:

                                                     Table 1--Estimated Annual Reporting Burden \1\
--------------------------------------------------------------------------------------------------------------------------------------------------------
                                                                 Number of
                  Activity                       Number of    responses  per   Total annual         Average burden per response \2\         Total hours
                                                respondents      respondent      responses
--------------------------------------------------------------------------------------------------------------------------------------------------------
Pretest Study Screener Completes............             150               1             150  0.03 (2 minutes)..........................               5
Main Study Screener Completes...............           3,525               1           3,525  0.03 (2 minutes)..........................             106
Pretest Study...............................              90               1              90  0.33 (20 minutes).........................              30
Main Study..................................           2,115               1           2,115  0.33 (20 minutes).........................             698
                                             -----------------------------------------------------------------------------------------------------------
    Total...................................  ..............  ..............  ..............  ..........................................             839
--------------------------------------------------------------------------------------------------------------------------------------------------------
\1\ No capital costs or operating and maintenance costs are associated with collection of this information.
\2\ Burden estimates of less than 1 hour are expressed as a fraction of an hour in decimal format.

II. References

    The following references marked with an asterisk (*) are on display 
at the Dockets Management Staff (HFA-305), Food and Drug 
Administration, 5630 Fishers Lane, Rm. 1061, Rockville, MD 20852, and 
are available for viewing by interested persons between 9 a.m. and 4 
p.m., Monday through Friday; they also are available electronically at 
https://www.regulations.gov. References without asterisks are not on 
public display at https://www.regulations.gov because they have 
copyright restriction. Some may be available at the website address, if 
listed. References without asterisks are available for viewing only at 
the Dockets Management Staff. FDA has verified the website addresses, 
as of the date this document publishes in the Federal Register, but 
websites are subject to change over time.

*1. Johar, K., ``An Insider's Perspective: Defense of the 
Pharmaceutical Industry's Marketing Practices,'' Albany Law Review, 
76:299-334, 2012-2013.
*2. Wick, C., M. Egger, S. Trelle, et al., ``The Characteristics of 
Unsolicited Clinical Oncology Literature Provided by Pharmaceutical 
Industry,'' Annals of Oncology, 18(9):1580-1582, 2007.
* 3. Fisher, J.A., M.D. Cottingham, and C.A. Kalbaugh, ``Peering 
Into the Pharmaceutical `Pipeline': Investigational Drugs, Clinical 
Trials, and Industry Priorities,'' Social Science & Medicine, 
131:322-330, 2015.
4. Centerwatch, ``FDA Approved Drugs for Oncology,'' https://www.centerwatch.com/drug-information/fda-approved-drugs/therapeutic-area/12/oncology (accessed on October 5, 2018).
5. https://www.fda.gov/BiologicsBloodVaccines/DevelopmentApprovalProcess/BiologicalApprovalsbyYear/ucm596371.htm.
* 6. Beaver, J.A., L.J. Howie, L. Pelosof, et al., ``A 25-Year 
Experience of U.S. Food and Drug Administration Accelerated Approval 
of Malignant Hematology and Oncology Drugs and Biologics: A 
Review,'' JAMA Oncology, 4:849-856, 2018.
* 7. Dodge, T. and A. Kaufman, ``What Makes Consumers Think Dietary 
Supplements Are Safe and Effective? The Role of Disclaimers and FDA 
Approval,'' Health Psychology, 26:513-517, 2007.
* 8. Dodge, T., D. Litt, and A. Kaufman, ``Influence of the Dietary 
Supplement Health and Education Act on Consumer Beliefs About the 
Safety and Effectiveness of Dietary Supplements,'' Journal of Health 
Communication, 16(3):230-244, 2011.
* 9. Mason, M.J., D.L. Scammon, and X. Fang, ``The Impact of 
Warnings, Disclaimers, and Product Experience on Consumers' 
Perceptions of Dietary Supplements,'' The Journal of Consumer 
Affairs, 41(1):74-99, 2007.
* 10. France, K.R. and P.F. Bone, ``Policy Makers' Paradigms and 
Evidence From Consumer Interpretations of Dietary Supplement 
Labels,'' The Journal of Consumer Affairs, 39(1):27-51, 2005.
* 11. Ghosh, A.K. and K. Ghosh, ``Translating Evidence-Based 
Information into Effective Risk Communication: Current Challenges 
and Opportunities,'' The Journal of Laboratory and Clinical 
Medicine, 145(4):171-180, 2005.
* 12. Harewood, G.C. and L.M. Hendrick, ``Prospective, Controlled 
Assessment of the Impact of Formal Evidence-Based Medicine Teaching 
Workshop on Ability to Appraise the Medical Literature,'' Irish 
Journal of Medical Science, 179(1):91-94, 2010.
* 13. Fritsche, L., T. Greenhalgh, Y. Falck-Ytter, et al., ``Do 
Short Courses in Evidence Based Medicine Improve Knowledge and 
Skills? Validation of Berlin Questionnaire and Before and After 
Study of Courses in Evidence Based Medicine,'' British Medical 
Journal, 325(7376):1338-1341, 2002.
* 14. Kahwati, L., D. Carmondy, N. Berkman,

[[Page 56852]]

et al., ``Prescribers' Knowledge and Skills for Interpreting 
Research Results: A Systematic Review,'' Journal of Continuing 
Education in the Health Professions, 37(2):129-136, 2017.
* 15. Tversky, A. and D. Kahneman,'' Judgment Under Uncertainty: 
Heuristics and Biases,'' Science, 185(4157):1124-1131, 1974.
* 16. Gehlbach, H. and S. Barge, ``Anchoring and Adjusting in 
Questionnaire Responses,'' Basic and Applied Social Psychology, 
34(5):417-433, 2012.
* 17. Epley, N. and T. Gilovich, ``The Anchoring-and-Adjustment 
Heuristic: Why the Adjustments are Insufficient,'' Psychological 
Science, 17(4):311-318, 2006.
* 18. H[ouml]hne, J.K. and D. Krebs, ``Scale Direction Effects in 
Agree/Disagree and Item-Specific Questions: A Comparison of Question 
Formats,'' International Journal of Social Research Methodology, 
21(1):91-103, 2017.
* 19. Saris, W.E., M. Revilla, J.A. Krosnick, et al., ``Comparing 
Questions with Agree/Disagree Response Options to Questions with 
Item-Specific Response Options'' Survey Research Methods, 4:61-79, 
2010.
* 20. Krosnick, J.A. and S. Presser, ``Question and Questionnaire 
Design,'' In: Handbook of Survey Research (pp. 263-314). Bingley, 
United Kingdom: Emerald Group Publishing Limited, 2010.
21. Converse, J.M. and S. Presser, Survey Questions: Handcrafting 
the Standardized Questionnaire, (No. 63). Thousand Oaks, CA: SAGE 
Publications, 1986.
22. DeVellis, R.F., Scale Development: Theory and Applications, 
(Vol. 26). Thousand Oaks, CA: SAGE Publications, 2016.

    Dated: November 7, 2018.
Leslie Kux,
Associate Commissioner for Policy.
[FR Doc. 2018-24785 Filed 11-13-18; 8:45 am]
 BILLING CODE 4164-01-P