Center for Scientific Review Announcement of Requirements and Registration for New Methods To Detect Bias in Peer Review, 25603-25606 [2014-10196]

Agencies

• DEPARTMENT OF HEALTH AND HUMAN SERVICES
• National Institutes of Health

[Federal Register Volume 79, Number 86 (Monday, May 5, 2014)]
[Notices]
[Pages 25603-25606]
From the Federal Register Online via the Government Printing Office [www.gpo.gov]
[FR Doc No: 2014-10196]


-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

National Institutes of Health


Center for Scientific Review Announcement of Requirements and 
Registration for New Methods To Detect Bias in Peer Review


    Authority:  15 U.S.C. 3719.

SUMMARY: The Center for Scientific Review (CSR) is seeking ideas for 
the detection of bias in NIH Peer Review of grant applications in a 
challenge titled ``New Methods to Detect Bias in Peer Review.'' This 
notice provides information regarding requirements and registration for 
this challenge.

DATES: 
    Submission Period: May 5, 2014 through 11:59 p.m. Eastern Time, 
June 30, 2014.
    Judging Period: July 16, 2014 through August 29, 2014.
    Winners Announced: September 2, 2014.

ADDRESSES: Details on the NIH/CSR Peer Review process can be found on 
the Reviewer Resources tab at www.csr.nih.gov. For questions about this 
challenge, email

[[Page 25604]]

CSRDiversityPeerRev@mail.nih.gov or call at 301-300-3839.

FOR FURTHER INFORMATION CONTACT: Monica Basco, Center for Scientific 
Review, phone: 301-300-3839 or email: CSRDiversityPeerRev@mail.nih.gov.

SUPPLEMENTARY INFORMATION: The mission of CSR is to ensure that the 
National Institutes of Health (NIH) grant applications receive fair, 
independent, expert, and timely reviews so NIH can fund the most 
promising research. For this challenge, ``New Methods to Detect Bias in 
Peer Review,'' CSR is seeking ideas for the detection of bias in NIH 
Peer Review of grant applications. CSR is particularly interested in 
approaches, strategies, methodologies, and/or measures that would be 
sensitive to detecting bias among reviewers based on gender, race/
ethnicity, institutional affiliation, area of science, and amount of 
research experience of applicants.
    This challenge is consistent with peer review authority under 
sections 492 and 492A of the Public Health Service Act and federal 
regulations governing ``Scientific Peer Review of Research Grant 
Applications and Research and Development Contract Projects'' (42 CFR 
Part 52h). The challenge is part of a larger quality assessment 
activity related to peer review. Research findings (Ginther et al., 
2011; 2012) showed a discrepancy in success rates for NIH R01 grant 
funding between White applicants and Black applicants, raising the 
question of possible bias in the peer review process. This challenge 
aims to address that discrepancy by soliciting ideas for detecting 
potential bias in peer review. It directly supports the mission of CSR 
to ensure that the best and brightest minds have an equal opportunity 
to contribute to the realization of our national research goals.
    Subject of Challenge: The subject of this challenge is to seek 
ideas for the detection of bias in NIH Peer Review. The mission of the 
NIH is to seek fundamental knowledge about the nature and behavior of 
living systems and to apply that knowledge to enhance health, lengthen 
life, and reduce the burdens of illness and disability. NIH has a 
longstanding and time tested system of peer review to identify the most 
promising biomedical research to accomplish these aims. As the portal 
for NIH grant applications and their review for scientific and 
technical merit, CSR is engaged in a new initiative to closely examine 
the peer review process. Aims include the identification of procedures 
and practices that are most beneficial in accomplishing CSR's mission 
as well as identifying any aspects that might make the review process 
vulnerable to bias. The goal is to enhance the quality and validity of 
the peer review process.
    As background, every grant application submitted to NIH must 
undergo two levels of NIH Peer Review prior to funding. The first level 
of review is carried out by a Scientific Review Group (SRG) composed 
primarily of non-federal scientists who have expertise in relevant 
scientific disciplines and current research areas. The second level of 
review is performed by Institute and Center National Advisory Councils 
or Boards who make recommendations on priority areas of research, 
pending policy, and funding of particular applications. Councils are 
composed of both scientific and public representatives chosen for their 
expertise, interest, or activity in matters related to health and 
disease. Only applications that are recommended for approval by both 
the SRG and the Advisory Council may be recommended for funding. Final 
funding decisions are made by the director of the relevant NIH 
Institute or Center.
    CSR strives to ensure that NIH grant applications receive fair, 
independent, expert, and timely reviews--free from inappropriate 
influences--so NIH can fund the most promising research. However, 
recent studies (Ginther et al., 2011; 2012) have shown that African 
American researchers are less likely than White researchers to receive 
NIH R01 grant funding by at least 10 percentage points. An 
investigation of racial disparities in grant funding must include the 
exploration of potential bias in the peer review system.
    There are several challenges in the assessment of bias in peer 
review. Any reactive effects of assessing racial bias must be 
minimized. That is; detection strategies should not have a detrimental 
effect on reviewers by creating a sensitivity that did not previously 
exist. And, while the written critiques of reviewers may provide 
opportunities to identify biased comments, because of the confidential 
nature of peer review, the names and demographic characteristics of 
reviewers assigned to specific applications are not retained and not 
all grant applicants and reviewers provide sensitive demographic data 
such as race and ethnicity.
    In this challenge, participants are asked to submit their ideas for 
the detection of possible bias in the NIH Peer Review process. These 
ideas should provide approaches, strategies, methodologies, and/or 
measures that would be sensitive to detecting bias among reviewers due 
to gender, race/ethnicity, institutional affiliation, area of science, 
and prior funding of grant applicants (See complete submission 
requirements below).
    Eligibility Rules for Participating in the Challenge: The challenge 
is open to any individual, group of individuals, or entity (each 
referred to in this notice as a participant) who meets the eligibility 
criteria below. There is no limit to the number of entries a 
participant can submit.
    To be eligible to win a prize under this challenge:
    (1) The participant shall have registered to participate in the 
competition under the rules promulgated by CSR as described in this 
notice.
    (2) The participant (including each individual participating as a 
member of a group participant) shall have complied with all the 
requirements under this section.
    (3) In the case of a private entity, the entity shall be 
incorporated in and maintain a primary place of business in the United 
States, and in the case of an individual, whether participating singly 
or in a group, each shall be a citizen or permanent resident of the 
United States.
    (4) Individuals (whether competing alone or part of a group) who 
are younger than 18 must have their parent or legal guardian complete 
the Parental Consent Form. The form can be found on the Challenge Web 
page at www.csr.nih.gov.
    (5) The participant may not be a Federal entity or Federal employee 
acting within the scope of his or her employment.
    (6) The participant shall not be an HHS employee working on their 
applications or submissions during assigned duty hours.
    (7) The participant shall not be an employee of the National 
Institutes of Health, the Center for Scientific Review, a member of the 
Subcommittee on Peer Review or any other party involved with the 
design, production, execution, or distribution of the Challenge or 
their immediate family (spouse, parents or step-parents, siblings and 
step-siblings and children and step-children).
    (8) Federal grantees may not use Federal funds to develop COMPETES 
Act challenge applications unless consistent with the purpose of their 
grant award.
    (9) Federal contractors may not use Federal funds from a contract 
to develop COMPETES Act challenge applications or to fund efforts in 
support of a COMPETES Act challenge submission.
    (10) CSR reserves the right to cancel, suspend, modify the 
challenge and/or not award a prize if no submissions are deemed worthy.

[[Page 25605]]

    (11) CSR will claim no rights to intellectual property. By 
participating in this challenge, participant grants to CSR an 
irrevocable, paid-up, royalty-free, nonexclusive worldwide license to 
post, link to, share, and display publicly the submission(s) on the 
Web, newsletters or pamphlets, and other information products such as a 
future Funding Opportunity Announcement or other study to develop the 
methodology. In addition, CSR may incorporate proposed ideas into a 
future Request for Applications (RFA), Request for Proposals (RFP) or 
an implemented study to develop the methodology, but an award of a 
prize does not guarantee the proposed idea will be implemented.
    (12) By participating in this challenge, participant agrees that 
the submission is participant's original work and that all proposed 
ideas are participant's original effort. It is the responsibility of 
the participant to obtain any rights necessary to use, disclose, or 
reproduce any intellectual property owned by third parties and 
incorporated in the entry for all anticipated uses of the submission. 
Submissions must not violate or infringe upon the rights of other 
parties, including, but not limited to, privacy, publicity or 
intellectual property rights, or material that constitutes copyright or 
license infringement.
    (13) By participating in this challenge, each participant 
(including each individual making up a group participant) agrees to 
assume any and all risks and waive claims against the Federal 
Government and its related entities, except in the case of willful 
misconduct, for any injury, death, damage, or loss of property, 
revenue, or profits, whether direct, indirect, or consequential, 
arising from participation in this prize challenge, whether the injury, 
death, damage, or loss arises through negligence or otherwise.
    (14) Based on the subject matter of the challenge, the type of work 
that it will possibly require, as well as an analysis of the likelihood 
of any claims for death, bodily injury, or property damage, or loss 
potentially resulting from challenge participation, participants are 
not required to obtain liability insurance or demonstrate financial 
responsibility in order to participate in this challenge.
    (15) By participating in this challenge, each participant agrees to 
indemnify the Federal Government against third party claims for damages 
arising from or related to challenge activities.
    (16) An individual shall not be deemed ineligible because the 
individual used Federal facilities or consulted with Federal employees 
during this challenge if the facilities and employees are made 
available to all individuals participating in the challenge on an 
equitable basis.
    (17) In the case of groups, a single, individual group member will 
submit the submission on behalf of the group and certify that the 
submission meets all challenge rules.
    (18) The decision of the award approving official is final and 
cannot be contested. The award approving official is the Director of 
the Center for Scientific Review.
    Submission Process for Participants: Participants should submit all 
entry materials to CSRDiversityPeerReview@mail.nih.gov.
    Amount of the Prize: CSR may award up to a total of four prizes in 
two categories: Best Empirically-Based Submission and Most Creative 
Submission. In each of these two categories, CSR may award a first 
prize in the amount of $10,000 and a second prize in the amount of 
$5,000. Each submission is eligible for only one prize (i.e., a single 
submission cannot win more than one prize for this challenge).
    Prizes awarded under this challenge will be paid by electronic 
funds transfer and may be subject to Federal income taxes. HHS will 
comply with the Internal Revenue Service withholding and reporting 
requirements, where applicable. If a group or entity is selected as a 
winner, CSR will pay the prize to an individual representative of the 
group or entity designated in the cover letter required as part of the 
submission. To the extent applicable, it is this individual's 
responsibility to distribute the prize to group (or entity) members.
    Basis Upon Which Submissions Will Be Evaluated: After CSR receives 
and de-identifies the submissions, the submissions will be evaluated 
according to a two-stage process: (1) Technical merit will be evaluated 
for its potential to detect bias in peer review (High, Medium, Low 
Impact) by a panel of experts in fields relevant to peer review and 
reviewer bias, and (2) High Impact submissions will be evaluated and 
rank-ordered based on the judging criteria (see judging criteria below) 
by a panel of judges comprised of federal employees who will recommend 
the winning entries.
    The final awards will be approved by the Director of the Center for 
Scientific Review; provided, however, that CSR reserves the right to 
cancel, suspend, modify the challenge and/or not award a prize if no 
submissions are deemed worthy.
    The judging criteria for the best empirically based and most 
creative submissions are as follows:

Best Empirically-Based Submission

 Theoretically based and/or hypothesis driven
 Proposes an experimental design
 Well-grounded in peer reviewed empirical literature
 Proposes measurement methods
 Feasibility of implementation
 Related to the NIH Peer Review Process

Most Creative Submission

 Proposes novel concepts or translates existing concepts in a 
novel way
 Challenges existing paradigms
 The proposed project has potential to be translated for use in 
an experimental design
 Creative ways to apply ideas
 Implementation is feasible
 Relates to the NIH Peer Review Process
    Submission Requirements: This challenge is for the solicitation of 
ideas for the detection of bias in NIH Peer Review, therefore a full 
development of new measures is not required. The following materials 
must be uploaded to CSRDiversityPeerReview@mail.nih.gov or sent in 
hardcopy to the Office of the Director, Attention: Denise McGarrell, 
Center for Scientific Review, 6701 Rockledge Drive, Suite 3030, 
Bethesda, Maryland 20892 by the deadline. Incomplete submissions will 
not be considered. All submissions must be written in English.
     Cover sheet with title of the submission and the 
participant's name or names of group members and contact information. 
In the case of groups (and entities), indicate one group member 
responsible for corresponding with CSR. Also indicate which group 
member will be responsible for receiving the prize for distribution, as 
applicable, among group members.
     Challenge submission documents. Note: The 2-page challenge 
idea should be anonymous (i.e., not include identifying information of 
the participant). Submissions shall not exceed 2 single-spaced pages 
(not to include cover page, references or parental consent document, if 
applicable) and shall be constrained to no less than one inch margins 
and 11 pt. Ariel font. All submissions must be submitted in .doc (Word) 
format. Submissions should include the following sections:
    Aims: Describe the goals for your proposed approach to the 
detection of bias in peer review and the anticipated outcomes.

[[Page 25606]]

    Approach: Provide a detailed description of your proposed methods 
and procedures. Describe how you might measure the effectiveness of 
your plan in accomplishing your proposed aims.
    Implementation: Explain how your method might be implemented as 
part of NIH Peer Review. Include how your proposed method might be 
tested and, if effective, how it might be disseminated across the NIH.
     As applicable, the signed Parental Consent Document.
     Submissions not conforming to these specifications will be 
disqualified.

References

Ginther DK et al. (2011). Race, ethnicity, and NIH research awards. 
Science, 333 (1015-1019).
Ginther DK, Haak LL, Schaffer WT, & Kington R. (2012). Are race, 
ethnicity, and medical school affiliation associated with NIH R01 
type 1 award probability for physician investigators? Academic 
Medicine, 87 (11), 1516-1524.

    Dated: April 29, 2014.
Richard Nakamura,
Director, Center for Scientific Review, National Institutes of Health.
[FR Doc. 2014-10196 Filed 5-2-14; 8:45 am]
BILLING CODE 4140-01-P