Request for Information (RFI) on Recommendations for Improving NRSA Fellowship Review, 24821-24823 [2023-08603]

Agencies

• DEPARTMENT OF HEALTH AND HUMAN SERVICES
• National Institutes of Health

[Federal Register Volume 88, Number 78 (Monday, April 24, 2023)]
[Notices]
[Pages 24821-24823]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2023-08603]


-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

National Institutes of Health


Request for Information (RFI) on Recommendations for Improving 
NRSA Fellowship Review

AGENCY: National Institutes of Health, HHS.

ACTION: Request for information.

-----------------------------------------------------------------------

SUMMARY: The purpose of this Request for Information (RFI) is to 
solicit public input on proposed changes to the peer review of Ruth L. 
Kirschstein National Research Service Award (NRSA) fellowship 
applications that would restructure the review criteria and modify some 
sections of the Public Health Service (PHS) Fellowship Supplemental 
Form within the application. The goal of this effort is to facilitate 
the mission of NRSA fellowship peer review: to identify the most 
promising trainees and the excellent, individualized training programs 
that will help them become the outstanding scientists of the next 
generation. The proposed changes will allow peer reviewers to better 
evaluate the applicant's potential and the quality of the scientific 
training plan without undue influence of the sponsor's or institution's 
reputation; and ensure that the information provided in the application 
is aligned with the restructured criteria and targeted to the 
fellowship candidate's specific training needs.

DATES: Comments must be received by June 23, 2023 to ensure 
consideration.

ADDRESSES: Submissions can be sent electronically to: https://rfi.grants.nih.gov/?s=642ed5def0356688b20e6be3. NIH is specifically 
requesting public comment on the proposed changes to the peer review of 
NRSA fellowship applications that would restructure the review criteria 
and modify some sections of the PHS Fellowship Supplemental Form within 
the application described above and at: https://grants.nih.gov/policy/peer/improving-nrsa-fellowship. Response to this RFI is voluntary.

FOR FURTHER INFORMATION CONTACT: Questions about this request for 
information should be directed to Kristin Kramer, 6701 Rockledge Drive, 
Bethesda, MD 20817, 301-437-0911, [email protected].

SUPPLEMENTARY INFORMATION: 

Current Process

    The overall goal of the NIH Ruth L. Kirschstein National Research 
Service Award (NRSA) program is to help ensure that a diverse pool of 
highly trained scientists is available in appropriate scientific 
disciplines to address the Nation's biomedical, behavioral, and 
clinical research needs. NRSA fellowships support the training of pre-
and postdoctoral scientists, dual-degree investigators, and senior 
researchers. The first stage of NIH peer review serves to provide 
expert advice to NIH by assessing the likelihood that the fellowship 
will enhance the candidate's potential for, and commitment to, a 
productive independent scientific research career in a health-related 
field. The criteria for the review of NRSA fellowship applications 
derive from the NRSA regulation at 42 CFR part 66.106 (https://www.ecfr.gov/current/title-42/chapter-I/subchapter-E/part-66/subpart-A/section-66.106):
    (a) Within the limits of funds available, the Secretary shall make 
Awards to those applicants:
    (1) Who have satisfied the requirements of Sec.  66.105; and
    (2) Whose proposed research or training would, in the judgment of 
the Secretary, best promote the purposes of section 487(a)(1)(A) of the 
Act, taking into consideration among other pertinent factors:
    (i) The scientific, technical, or educational merit of the 
particular proposal;
    (ii) The availability of resources and facilities to carry it out;
    (iii) The qualifications and experience of the applicant; and
    (iv) The need for personnel in the subject area of the proposed 
research or training.
    The NIH peer review regulation does not address scoring. Scoring of 
all regulatory factors is determined by NIH policy. Currently, peer 
reviewers provide an Overall Impact Score (scored 1-9) that reflects 
their assessment of the likelihood that the fellowship will enhance the 
candidate's potential for, and commitment to, a productive independent 
scientific research career in a health-related field. Peer reviewers 
provide individual criterion scores for five criteria: (1) Applicant; 
(2) Sponsors and Collaborators; (3) Research Training Plan; (4) 
Training Potential; and (5) Institutional Environment and Commitment. 
Additional review criteria are evaluated and factored into the Overall 
Impact Score but are not given individual scores: Protections for Human 
Subjects; Inclusion of Women, Minorities, and Individuals Across the 
Lifespan; Vertebrate Animals; Biohazards; and Resubmission. Beyond 
these criteria, reviewers are asked to assess the following additional 
review considerations; these considerations are not considered in the 
Overall Impact Score: Training in the Responsible Conduct of Research, 
Applications from Foreign Organizations, Select Agents, Resource 
Sharing Plans, Budget and Period of Support, and Authentication of Key 
Biological and/or Chemical Resources.

Proposal Development

    NIH gathered input from many sources in forming this proposal. 
Unsolicited comments over a period of years conveyed persistent 
concerns that the NRSA fellowship review process disadvantages some 
highly-qualified, promising applicants. In response, the Center for 
Scientific Review (CSR) formed a working group to the CSR Advisory 
Council. To inform that group, CSR published a Review Matters blog at: 
https://www.csr.nih.gov/reviewmatters/2022/01/06/strengthening-fellowship-review/, inviting comments, which was cross-posted on the 
Office of Extramural Research blog, Open Mike at: https://nexus.od.nih.gov/all/2022/01/10/strengthening-fellowship-review/. The 
working group presented an interim report at: https://public.csr.nih.gov/sites/default/files/2019-10/Review_criteria_wg_CSRAC_interim_report_7April2020.pdf to the CSR 
Advisory Council, which adopted the recommendations, at public CSR 
Advisory Council meetings (March 2022 video https://videocast.nih.gov/watch=44677, slides https://public.csr.nih.gov/sites/default/files/2022-04/CSRAC_Fellowship_WG_interim_presentation.pdf; September 2022 
video https://videocast.nih.gov/watch=45767, slides https://public.csr.nih.gov/sites/default/files/2022-09/CSRAC_WG_on_Fellowship_Review_Sept_2022.pdf). Final recommendations 
from the CSR Advisory Council at: https://public.csr.nih.gov/sites/default/files/2022-11/CSRAC_Fellowship_review_WG_report_September_2022_final.pdf were 
considered by the CSR Director, as well as major internal NIH 
extramural-focused committees that included leadership from across NIH 
institutes and centers. Additional background information can be found 
at: https://grants.nih.gov/policy/peer/improving-nrsa-fellowship.

[[Page 24822]]

Recommendations for Improving NRSA Fellowship Review

Revise the Criteria Used To Evaluate NRSA Fellowship Applications

    As is currently the case, the Overall Impact Score (scored 1-9) 
will reflect the scientific and educational merit of the proposal and 
an assessment of the likelihood that the fellowship will enhance the 
applicant's potential for, and commitment to, an independent, 
productive research career in a health-related field. However, the 
current 5 scored criteria that inform the Overall Impact Score will be 
restructured into the following 3 scored criteria. Additional detail on 
proposed reviewer guidance can be found here: https://grants.nih.gov/policy/peer/improving-nrsa-fellowship/reviewer-instructions.

Criterion 1: Scientific Potential, Fellowship Goals, and Preparedness 
of the Applicant (Scored 1-9)

     Evaluate the breadth and depth of scientific understanding 
the applicant conveys in their statements. To what extent does the 
candidate articulate the importance of their science and demonstrate an 
ability to study that problem in a rigorous scientific manner.
     Evaluate the preparedness of the applicant to undertake 
the proposed training and their capacity to benefit from the 
fellowship. Evaluate their accomplishments in the context of their 
stage of training and the scientific opportunities they have had.
     Evaluate the applicant's scientific potential. Consider 
their trajectory in the context of their opportunities. Also consider 
other factors that bear on their potential to succeed, such as 
determination, persistence, and creativity.

Criterion 2: Science and Scientific Resources (Scored 1-9)

     Evaluate the quality of the proposed science. Assess the 
depth of understanding of the scientific background and the scientific 
rigor and feasibility of the approach.
     Evaluate the extent to which needed technical, scientific, 
and clinical resources are specified and are realistically available to 
the applicant.
     Assess whether the scientific expertise of the mentorship 
team is appropriate for the proposed science and whether the role of 
each mentor is clearly defined.
     Evaluate how well the proposed scientific project serves 
the applicant's training goals.
     Note that peer review of financial support for the 
proposed research will be eliminated.

Criterion 3: Training Plan and Training Resources (Scored 1-9)

     Evaluate whether the applicant clearly defines their 
career goals and whether the training plan is linked to them.
     Evaluate whether the applicant has clearly defined areas 
of needed growth. These could include specific scientific skills and 
knowledge and other professional needs such as communication, teaching, 
and mentorship skills.
     Evaluate the training environment for this applicant. 
Assess whether the necessary institutional training resources are well-
specified and available, specifically the practical availability of 
resources.
     Evaluate whether the trainee articulated a coherent and 
cohesive plan for interacting with sponsors and mentors.
     Assess whether the sponsor presents a strong pedagogical 
plan appropriate to the needs and goals of the applicant. Please 
include an evaluation of the training philosophy of the sponsor, their 
approach to training, time commitments and their accessibility.
     Evaluate and comment on what impact completion of the 
training plan will make in meeting the scientific development needs of 
the applicant and aid them in achieving their career goals.
    The Additional Review Criteria (e.g., Protections for Human 
Subjects; Inclusion of Women, Minorities and Individuals Across the 
Lifespan; etc.) would not change.
    The Additional Review Considerations (e.g., Training in the 
Responsible Conduct of Research, Resource Sharing Plans, Budget, etc.) 
would not change.
    Revising the criteria simplifies the task of reviewers by focusing 
their attention on just three key assessments: the scientific potential 
of the applicant, the science and scientific resources, and the 
training plan and training resources. The criteria are defined to give 
applicants from heterogeneous backgrounds a fair chance; reviewers are 
asked to evaluate applicant accomplishments and trajectory in the 
context of the opportunities they have had. In addition to evaluating 
applicant accomplishments, reviewers are asked to evaluate personal 
characteristics that contribute to success in science, factors such as 
determination, persistence, and creativity. The revised criteria are 
also expected to reduce bias in review by reducing any consideration of 
sponsor and institutional reputation and instead focusing review on 
their specific, realistic, and current contributions to the scientific 
needs, goals, and training of the specific trainee. NIH believes these 
changes will better enable peer review to identify those applications 
with the highest potential for producing productive research 
scientists, regardless of where the applicant started or the applicant 
institution.

Revise the Fellowship Supplemental Section of PHS SF424

    The NIH proposes to revise the following sections of the PHS 424 
Fellowship Supplement (https://grants.nih.gov/grants/how-to-apply-application-guide/forms-g/fellowship-forms-g.pdf): (1) Fellowship 
Applicant; (2) Sponsor(s), Collaborator(s), and Consultant(s); and (3) 
Letters of Reference. There are no proposed changes to the Research 
Training Plan section. Additionally, the revision would allow an 
optional Statement of Special Circumstance.
    The changes are intended to restructure the application so that the 
application content is better aligned with the review criteria, is less 
duplicative, and is easier for reviewers to assess. The changes 
emphasize substantive statements that pertain to the individual 
applicant trainee, require detailed accounts from sponsors explaining 
their preparation and approach to training, and their availability to 
the student. The changes would shorten the application by up to 2\1/2\ 
pages. The proposed changes for each section are described below:

1. Revised Applicant Section of the Fellowship Supplement

    Applicants would be asked to submit five statements:
    1. Statement of professional and fellowship goals.
    2. Fellowship qualifications.
    3. Self-assessment.
    4. Statement of scientific perspective.
    5. Activities planned under this award.
    Additionally, grades would no longer be required or allowed, 
however, applicants would be requested to include the titles of 
relevant courses completed.

2. Revised Sponsor and Co-Sponsor Section of the Fellowship Supplement

    Sponsors and Co-sponsors would be asked to submit three statements:
    1. Training plan, environment, and research facilities.
    2. Number of Fellows/Trainees to be supervised.
    3. Applicant's qualifications and potential for a research career.

[[Page 24823]]

3. Revised Instructions for Reference Letters

    NIH proposes to update the instructions for reference letters with 
more structure so that the resulting letters better assist reviewers in 
understanding the applicant's strengths, weaknesses, and potential to 
pursue a productive career in biomedical science. Writers would be 
instructed to respond to four questions addressing:
    1. Two to four most important characteristics that will contribute 
to applicant's success.
    2. Two to four areas of needed growth.
    3. Intellectual contributions made by the applicant during 
training.
    4. Overall assessment of readiness and potential.

4. Allow an Optional Statement of Special Circumstance

    NIH recommends allowing fellowship applicants to submit an optional 
Statement of Special Circumstance to address situations that may have 
hindered the trainee's progress, such as harassment, the COVID-19 
pandemic, or other personal or professional circumstances.
    Additional detail on proposed changes to the Fellowship Supplement 
can be found at: https://grants.nih.gov/policy/peer/improving-nrsa-fellowship/reviewer-instructions.

Submitting a Response

    Comments should be submitted electronically to the following web 
page at: https://rfi.grants.nih.gov/?s=642ed5def0356688b20e6be3.
    This RFI is for planning purposes only and should not be construed 
as a policy, solicitation for applications, or as an obligation on the 
part of the Government to provide support for any ideas identified in 
response to it. Please note that the Government will not pay for the 
preparation of any information submitted or for its use of that 
information.
    Please do not include any proprietary, classified, confidential, or 
sensitive information in your response. Responses will be compiled and 
a content analysis will be shared publicly after the close of the 
comment period. The NIH may use information gathered by this Notice to 
inform future policy development.

    Dated: April 18, 2023.
Tara A. Schwetz,
Acting Principle Deputy Director, National Institutes of Health.
[FR Doc. 2023-08603 Filed 4-21-23; 8:45 am]
BILLING CODE 4140-01-P