National Registry of Evidence-Based Programs and Practices, 38716-38718 [2015-16573]

Download as PDF 38716 Federal Register / Vol. 80, No. 129 / Tuesday, July 7, 2015 / Notices Substance Abuse and Mental Health Services Administration Center for Substance Abuse Treatment; Notice of Meeting Pursuant to Public Law 92–463, notice is hereby given that the Substance Abuse and Mental Health Services Administration’s (SAMHSA’s) Center for Substance Abuse Treatment (CSAT) National Advisory Council will meet on July 16, 2015, from 3:00 p.m.– 4:00 p.m. (EDT). The meeting will be closed to the public. The meeting will include discussion and evaluation of grant applications reviewed by Initial Review Groups, and involve an examination of confidential financial and business information as well as personal information concerning the applicants. Therefore, the meeting will be closed to the public, as determined by the SAMHSA Administrator, in accordance with Title 5 U.S.C. 552b(c)(4) and (6) and (c)(9)(B) and 5 U.S.C. App. 2, Section 10(d). The meeting will be held virtually. Meeting information and a roster of Council members may be obtained either by accessing the SAMHSA Council Web site at: http:// www.samhsa.gov/about-us/advisorycouncils/csat-national-advisory-council or by contacting LCDR Holly Berilla. Council Name: SAMHSA’s Center for Substance Abuse Treatment; National Advisory Council. Date/Time/Type: July 16, 2015, 3:00 p.m.– 4:00 p.m. EDT, Closed. Place: Virtual—Teleconference. Contact: LCDR Holly Berilla, Designated Federal Official, CSAT National Advisory Council, 1 Choke Cherry Road, Rockville, Maryland 20857 (mail), Telephone: (240) 276–1252, Fax: (240) 276–2252, Email: holly.berilla@samhsa.hhs.gov. Summer King, Statistician, SAMHSA. [FR Doc. 2015–16548 Filed 7–6–15; 8:45 am] BILLING CODE 4162–20–P DEPARTMENT OF HEALTH AND HUMAN SERVICES tkelley on DSK3SPTVN1PROD with NOTICES Substance Abuse and Mental Health Services Administration National Registry of Evidence-Based Programs and Practices Substance Abuse and Mental Health Services Administration (SAMHSA), HHS. ACTION: Notice regarding SAMHSA’s NREPP: Redesign of NREPP. AGENCY: VerDate Sep<11>2014 20:31 Jul 06, 2015 Jkt 235001 The mission of SAMHSA is to reduce the impact of substance abuse and mental illness on America’s communities. Established in 1992, the agency was directed by Congress to target effective substance abuse and mental health services to the people most in need, and to translate research in these areas more effectively and more rapidly into the general health care system. NREPP is a key public resource SAMHSA has developed to help meet this directive. This notice announces the redesign of NREPP to better align the registry with the standards and processes of other evidence-based repositories. A re-launch of the Web site with revised content is anticipated in late fall/early winter 2015. The notice explains the changes in how programs and practices will be identified for NREPP, how submissions will be screened and reviewed, and provides guidance on accessing updated information on the NREPP site. Potential applicants should be aware that this notice includes updated information relating to the eligibility of interventions for inclusion in NREPP and changes in the program and practice review process that supersedes guidance provided in earlier Federal Register notices. FOR FURTHER INFORMATION CONTACT: Carter Roeber, Ph.D., Social Science Analyst, Center for Behavioral Health Statistics and Quality, SAMHSA, 1 Choke Cherry Road, Room 2–1050, Rockville, MD 20857, telephone 240– 276–1488. SUPPLEMENTARY INFORMATION: SUMMARY: DEPARTMENT OF HEALTH AND HUMAN SERVICES Advancing Evidence-Based Programs and Practices Through Improved Decision Support Tools: Reconceptualizing NREPP Introduction SAMHSA’s NREPP is an evidencebased repository and review system designed to provide the public with reliable information about behavioral health interventions in the areas of mental health and substance abuse. Programs and practices that are accepted for inclusion in NREPP undergo a review process that provides information on the quality of research and the magnitude and direction of program or practice impact on individual outcomes. Materials for dissemination are reviewed to determine the type and extent of information available to support implementation. The results of these reviews are published on the NREPP Web site (http://nrepp.samhsa.gov). It should be noted that inclusion in NREPP indicates that some, but not PO 00000 Frm 00056 Fmt 4703 Sfmt 4703 necessarily all, of the evidence for a program or practice has been reviewed. In some cases, the quality of the research supporting the program or practice may have been determined to be poor or insufficient to earn a rating. Inclusion in NREPP does not constitute endorsement of an intervention as effective by SAMHSA. Moreover, since NREPP has not reviewed all behavioral health interventions, the use of NREPP as an exclusive or exhaustive list of interventions is not appropriate. Policymakers and funders in particular are discouraged from limiting providers and/or potential grantees to selecting exclusively from among NREPP interventions and from funding NREPP interventions regardless of the ratings the interventions receive. This notice announces changes to (1) the process for identifying new programs and practices for NREPP review, (2) the process for announcing open submission periods, (3) the minimum requirements to be considered for NREPP review, and (4) the review process. This notice also announces the intent to re-review currently posted NREPP programs and practices to comport with new review criteria and ratings. The re-review of programs and practices currently posted will take place over the course of the next several years, depending on available resources. A re-launch of the NREPP Web site will take place in phases and the first phase is planned for late fall/early winter of 2015. Identifying New Programs and Practices for NREPP Open submissions periods, during which applicants may submit materials for review, will continue to be used to identify new programs and practices for review (see below). Programs and practices addressing specific SAMHSA priorities may also be identified by SAMHSA or through environmental scans (including literature reviews, focus groups, public input, and interviews), as time and resources permit. Programs and practices related to priority areas may be reviewed before programs and practices identified through the open submission period. SAMHSA will be consulting with subject matter experts and leadership in underserved groups and populations, including American Indian/Alaska Native Tribes, regarding ways to incorporate traditional and culturallyspecific interventions into NREPP, in order to better meet the needs of groups whose efforts to promote behavioral health may not have not been routinely evaluated. Innovative, but perhaps less rigorously tested, programs and E:\FR\FM\07JYN1.SGM 07JYN1 Federal Register / Vol. 80, No. 129 / Tuesday, July 7, 2015 / Notices tkelley on DSK3SPTVN1PROD with NOTICES practices will be considered if specific to SAMHSA priority areas, as a parallel contribution to the standard review procedures. Stand-alone pharmacologic treatments are not eligible for review and should not be submitted to NREPP. The evidence base for pharmacologic treatments is reviewed and approved through the U.S. Food and Drug Administration (FDA). FDA approved pharmacotherapy interventions (onlabel use) are considered for NREPP review only when combined with one or more behavioral or psychosocial treatments. SAMHSA reserves the right to reject for review programs and practices whose goals or activities are determined to be inconsistent with SAMHSA’s mission, which is ‘‘to reduce the impact of substance abuse and mental illness on American communities.’’ Open Submission Periods SAMHSA accepts new applications for review during open submission periods. SAMHSA generally holds one open submission period a year but, depending on the number of reviews in progress and resources available may hold more or fewer within a calendar year. All future open submission periods will be announced on the NREPP Web site. Emails will also be sent announcing the open submission period to those on the NREPP listserv. Anyone wishing to be notified of future open submission periods can join the NREPP listserv by sending a request to nrepp@samhsa.hhs.gov. Applications can be submitted at any point during an open period. Program and practice developers, researchers, and others interested in submitting an intervention should read below for information about the new minimum requirements to be considered for an NREPP review. Additional future changes to the review process and criteria will be posted on the NREPP Web site as they are implemented. Therefore, before submitting a program or practice for NREPP review, applicants should examine the most recent information posted on the NREPP Web site about the review process and criteria and the most recent guidance for preparing an intervention for submission (see http:// www.nrepp.samhsa.gov/ ReviewSubmission.aspx). This guidance will be periodically updated to reflect the redesign of NREPP during the relaunch. The selection of interventions will take place after the closing of the open submission period, and applicants will be notified whether they have been VerDate Sep<11>2014 20:31 Jul 06, 2015 Jkt 235001 accepted after an initial screening to ensure that the application meets minimum requirements. The number of reviews conducted will depend on the availability of funds, with the timing of reviews to be determined by SAMHSA. In submitting an intervention, applicants should understand that if interventions are selected for review, the results of NREPP reviews are considered public information and will be posted on the NREPP Web site. Once a review is completed, the applicant will be provided with a summary document (‘‘the program or practice profile’’) that presents results of the review, ratings of program effectiveness, and descriptive information about the intervention. The applicant will have the opportunity to comment on the profile before it is posted but they will not have the option to refuse posting. Minimum Requirements To be considered for review, interventions must meet three minimum requirements: 1. Research or evaluation of the intervention has assessed mental health or substance use outcomes among individuals, communities, or populations OR other behavioral healthrelated outcomes on individuals, communities, or populations with or at risk of mental health issues or substance use problems. 2. Evidence of these outcomes has been demonstrated in at least one study using an experimental or quasiexperimental design. Experimental designs require random assignment, a control or comparison group, and preand post-intervention outcome assessments. Quasi-experimental designs do not require random assignment, but do require a comparison or control group and pre- and postintervention outcome assessments. Comparison/control groups must be a no-treatment control group, a wait-list control group, a treatment-as-usual comparison group, or an intervention that is presumed to be ineffective or substantially less effective than the intervention (e.g., a ‘‘placebo’’ control or, in cases in which providing no treatment might be considered unethical, less effective treatments, even if not treatment-as-usual, such as ‘‘supportive therapy’’). Studies with single-group, pretest-posttest designs or single-group, longitudinal/multiple time series do not meet this requirement, but will be considered to identify emerging programs and practices for consideration in the Learning Center. Comparative effectiveness trials, in which two interventions, both presumed to be equally effective, are PO 00000 Frm 00057 Fmt 4703 Sfmt 4703 38717 compared, and studies in which the effects of the same intervention on various subpopulations are compared or in which various doses or components of the same intervention are compared will be reviewed, but only for purposes of providing readers in the Learning Center the opportunity to determine whether a particular program or practice may have use or show promise in different settings or circumstances. 3. The results of these studies have been published in a peer-reviewed journal or other professional publication, or documented in a comprehensive evaluation report, published within the previous 25 years. Comprehensive evaluation reports must include a review of the literature, theoretical framework, purpose, methodology, findings/results with statistical analysis and p values for significant outcomes, discussion, and conclusions. Applicants are required to provide full-text documents at the time of submission that demonstrate the intervention meets these minimum requirements. Other research articles, published or unpublished evaluation reports, grant final reports, and replication studies may be submitted as additional supporting documentation. Note: Abstracts or links to partial articles are regarded as incomplete and will not be considered. NREPP will no longer require programs and practices to have developed implementation materials, training and support resources, and quality assurance procedures. However, programs and practices with such dissemination and/or implementation resources will be considered for prioritized review if within the priorities established by the SAMHSA review process, and the materials, along with the location of their availability, will be listed in the program or practice profile. Applicants submitting dissemination and implementation resources should include a brief narrative description of the materials that are being submitted. These materials may include, but are not limited to, treatment manuals, information for administrators, information for direct service staff, tested training curricula, mechanisms for ongoing supervision and consultation, protocols for gathering process and outcome data, ongoing monitoring of intervention fidelity, and processes for gathering feedback. Applicants should also provide the location of where the materials can be obtained. E:\FR\FM\07JYN1.SGM 07JYN1 38718 Federal Register / Vol. 80, No. 129 / Tuesday, July 7, 2015 / Notices Selection of Interventions for Review SAMHSA will select interventions for review from among submissions meeting the minimum requirements. In selecting interventions for review, SAMHSA may give special consideration to interventions that meet one or more of the following conditions: • More than one research study or evaluation has been conducted on the same or a similar target population that meets the minimum requirements. • The intervention targets underserved populations (e.g., minority populations, tribal communities or American Indian/Alaska Native populations, elderly individuals, young adults, individuals who are incarcerated, etc.). • Dissemination and implementation materials (e.g., program or practice manuals, training guides, measurement instruments, implementation fidelity tools) are available. Lower costs and nocost materials may be prioritized. • The intervention contributes to a content area in which few evidencebased interventions have been previously identified. Interventions that are not selected for review may be resubmitted by the applicant in a future open submission period. tkelley on DSK3SPTVN1PROD with NOTICES The Review Process The review process has been revised to improve the quality of the reviews and utility of information that NREPP can provide its users. In addition to articles and reports submitted by NREPP applicants, additional studies, articles, and evaluation reports regarding the interventions will be identified through literature searches. Studies and outcomes to be reviewed will be determined through the systematic application of standardized screening criteria, and the number of studies and outcomes to be reviewed will be expanded to more comprehensively represent the evidence base for the program or practice. Inclusion of studies and outcomes will no longer be limited to positive significant outcomes; all studies and outcomes that meet the standardized screening criteria will be reviewed, including those with negative and non-significant effects. Programs and practices will be assessed on the basis of evaluation studies of program or practice impact, information related to conceptual framework (that is, program or practice goals, theory of change, and program or practice components), and information about implementation fidelity (that is, whether a study employs quality assurance measures to declare that the program or practice is VerDate Sep<11>2014 20:31 Jul 06, 2015 Jkt 235001 delivered as intended to the program’s or practice’s target population). The methodological rigor (that is, internal validity, statistical validity, and measurement validity) of the research for each program or practice will be reviewed, as it pertains to each outcome examined, along with the magnitude and direction of the program’s or practice’s effect on each outcome. Based on this information, the program’s or practice’s effectiveness for each outcome will be rated, along with the rigor of the research examining the program or practice, and the ratings will be displayed on the NREPP Web site. In general, each NREPP evidence review will be conducted by two trained and certified reviewers. However, based on funding and available resources, SAMHSA use one reviewer for programs and practices being re-reviewed. When necessary, NREPP may conduct author queries to confirm or gather additional information needed for the review. Program and practice profiles will be developed on the basis of the information gathered. Applicants will have the opportunity to review the program or practice profile before it is posted on the NREPP site, but they will not have the option to refuse posting. Dissemination and implementation materials will no longer be rated as they were historically. Instead, descriptions of available materials for each program or practice, highlighting information that may be of most interest to NREPP users, will be included in the program or practice profile, along with information documenting the extent to which materials are available. Programs and practices currently posted on NREPP will be re-reviewed as time and resources permit but the rereviews of currently posted programs and practices will take place over the next few years. Detailed information about the revised review process will be available at http://www.nrepp.samhsa.gov after the re-launch of the new NREPP Web site. Enhancing the Learning Center NREPP’s Learning Center is a developing and underutilized component of the NREPP Web site. With the evolution and enhancement of the registry, SAMHSA seeks to bring greater recognition to both rigorously evaluated behavioral health interventions and those interventions that have been implemented, demonstrate promise, but have not necessarily been evaluated in a rigorous manner. To that end, the Learning Center is being significantly revamped to support stakeholder engagement and to become a shared PO 00000 Frm 00058 Fmt 4703 Sfmt 4703 learning environment for all stakeholders. SAMHSA recognizes that the successful promotion and dissemination of evidence-based programs and practices requires an environment that promotes community assessment, program and practice planning and evaluation, as well as guidance on the selection and implementation of programs and practices listed on NREPP. There are useful types of evaluation research, often conducted among underserved populations, which provide valuable insights for practitioners, but do not meet the minimum criteria required for experimental or quasi-experimental design. SAMHSA intends the Learning Center to be a forum for presenting research on emerging programs and practices, and exploring ways that preexperimental and qualitative research can complement and enrich findings from experimental and quasiexperimental research designs. An inventory of such programs and practices will be compiled and maintained within the Learning Center and will operate in parallel to the listing of reviewed programs and practices with experimental and quasiexperimental designs. In this way, SAMHSA intends to support programs and practices researched with the most rigorous approaches while also supporting the development of practicebased evidence, especially for certain populations and emerging practices that are critical to learning and improving behavioral health outcomes for persons with or at risk of developing behavioral health issues. Summer King, Statistician. [FR Doc. 2015–16573 Filed 7–6–15; 8:45 am] BILLING CODE 4162–20–P DEPARTMENT OF HOMELAND SECURITY U.S. Customs and Border Protection Notice of Issuance of Final Determination Concerning Wound Therapy System U.S. Customs and Border Protection, Department of Homeland Security. ACTION: Notice of final determination. AGENCY: This document provides notice that U.S. Customs and Border Protection (‘‘CBP’’) has issued a final determination concerning the country of origin of the PICO single use negative pressure wound therapy system manufactured and distributed by Smith SUMMARY: E:\FR\FM\07JYN1.SGM 07JYN1

Agencies

[Federal Register Volume 80, Number 129 (Tuesday, July 7, 2015)]
[Notices]
[Pages 38716-38718]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2015-16573]


-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Substance Abuse and Mental Health Services Administration


National Registry of Evidence-Based Programs and Practices

AGENCY: Substance Abuse and Mental Health Services Administration 
(SAMHSA), HHS.

ACTION: Notice regarding SAMHSA's NREPP: Redesign of NREPP.

-----------------------------------------------------------------------

SUMMARY: The mission of SAMHSA is to reduce the impact of substance 
abuse and mental illness on America's communities. Established in 1992, 
the agency was directed by Congress to target effective substance abuse 
and mental health services to the people most in need, and to translate 
research in these areas more effectively and more rapidly into the 
general health care system. NREPP is a key public resource SAMHSA has 
developed to help meet this directive. This notice announces the 
redesign of NREPP to better align the registry with the standards and 
processes of other evidence-based repositories. A re-launch of the Web 
site with revised content is anticipated in late fall/early winter 
2015.
    The notice explains the changes in how programs and practices will 
be identified for NREPP, how submissions will be screened and reviewed, 
and provides guidance on accessing updated information on the NREPP 
site. Potential applicants should be aware that this notice includes 
updated information relating to the eligibility of interventions for 
inclusion in NREPP and changes in the program and practice review 
process that supersedes guidance provided in earlier Federal Register 
notices.

FOR FURTHER INFORMATION CONTACT: Carter Roeber, Ph.D., Social Science 
Analyst, Center for Behavioral Health Statistics and Quality, SAMHSA, 1 
Choke Cherry Road, Room 2-1050, Rockville, MD 20857, telephone 240-276-
1488.

SUPPLEMENTARY INFORMATION: 

Advancing Evidence-Based Programs and Practices Through Improved 
Decision Support Tools: Reconceptualizing NREPP

Introduction

    SAMHSA's NREPP is an evidence-based repository and review system 
designed to provide the public with reliable information about 
behavioral health interventions in the areas of mental health and 
substance abuse. Programs and practices that are accepted for inclusion 
in NREPP undergo a review process that provides information on the 
quality of research and the magnitude and direction of program or 
practice impact on individual outcomes. Materials for dissemination are 
reviewed to determine the type and extent of information available to 
support implementation. The results of these reviews are published on 
the NREPP Web site (http://nrepp.samhsa.gov).
    It should be noted that inclusion in NREPP indicates that some, but 
not necessarily all, of the evidence for a program or practice has been 
reviewed. In some cases, the quality of the research supporting the 
program or practice may have been determined to be poor or insufficient 
to earn a rating. Inclusion in NREPP does not constitute endorsement of 
an intervention as effective by SAMHSA. Moreover, since NREPP has not 
reviewed all behavioral health interventions, the use of NREPP as an 
exclusive or exhaustive list of interventions is not appropriate. 
Policymakers and funders in particular are discouraged from limiting 
providers and/or potential grantees to selecting exclusively from among 
NREPP interventions and from funding NREPP interventions regardless of 
the ratings the interventions receive.
    This notice announces changes to (1) the process for identifying 
new programs and practices for NREPP review, (2) the process for 
announcing open submission periods, (3) the minimum requirements to be 
considered for NREPP review, and (4) the review process. This notice 
also announces the intent to re-review currently posted NREPP programs 
and practices to comport with new review criteria and ratings. The re-
review of programs and practices currently posted will take place over 
the course of the next several years, depending on available resources. 
A re-launch of the NREPP Web site will take place in phases and the 
first phase is planned for late fall/early winter of 2015.

Identifying New Programs and Practices for NREPP

    Open submissions periods, during which applicants may submit 
materials for review, will continue to be used to identify new programs 
and practices for review (see below). Programs and practices addressing 
specific SAMHSA priorities may also be identified by SAMHSA or through 
environmental scans (including literature reviews, focus groups, public 
input, and interviews), as time and resources permit. Programs and 
practices related to priority areas may be reviewed before programs and 
practices identified through the open submission period. SAMHSA will be 
consulting with subject matter experts and leadership in underserved 
groups and populations, including American Indian/Alaska Native Tribes, 
regarding ways to incorporate traditional and culturally-specific 
interventions into NREPP, in order to better meet the needs of groups 
whose efforts to promote behavioral health may not have not been 
routinely evaluated. Innovative, but perhaps less rigorously tested, 
programs and

[[Page 38717]]

practices will be considered if specific to SAMHSA priority areas, as a 
parallel contribution to the standard review procedures.
    Stand-alone pharmacologic treatments are not eligible for review 
and should not be submitted to NREPP. The evidence base for 
pharmacologic treatments is reviewed and approved through the U.S. Food 
and Drug Administration (FDA). FDA approved pharmacotherapy 
interventions (on-label use) are considered for NREPP review only when 
combined with one or more behavioral or psychosocial treatments.
    SAMHSA reserves the right to reject for review programs and 
practices whose goals or activities are determined to be inconsistent 
with SAMHSA's mission, which is ``to reduce the impact of substance 
abuse and mental illness on American communities.''

Open Submission Periods

    SAMHSA accepts new applications for review during open submission 
periods. SAMHSA generally holds one open submission period a year but, 
depending on the number of reviews in progress and resources available 
may hold more or fewer within a calendar year. All future open 
submission periods will be announced on the NREPP Web site. Emails will 
also be sent announcing the open submission period to those on the 
NREPP listserv. Anyone wishing to be notified of future open submission 
periods can join the NREPP listserv by sending a request to 
nrepp@samhsa.hhs.gov.
    Applications can be submitted at any point during an open period. 
Program and practice developers, researchers, and others interested in 
submitting an intervention should read below for information about the 
new minimum requirements to be considered for an NREPP review. 
Additional future changes to the review process and criteria will be 
posted on the NREPP Web site as they are implemented. Therefore, before 
submitting a program or practice for NREPP review, applicants should 
examine the most recent information posted on the NREPP Web site about 
the review process and criteria and the most recent guidance for 
preparing an intervention for submission (see http://www.nrepp.samhsa.gov/ReviewSubmission.aspx). This guidance will be 
periodically updated to reflect the redesign of NREPP during the re-
launch.
    The selection of interventions will take place after the closing of 
the open submission period, and applicants will be notified whether 
they have been accepted after an initial screening to ensure that the 
application meets minimum requirements. The number of reviews conducted 
will depend on the availability of funds, with the timing of reviews to 
be determined by SAMHSA. In submitting an intervention, applicants 
should understand that if interventions are selected for review, the 
results of NREPP reviews are considered public information and will be 
posted on the NREPP Web site. Once a review is completed, the applicant 
will be provided with a summary document (``the program or practice 
profile'') that presents results of the review, ratings of program 
effectiveness, and descriptive information about the intervention. The 
applicant will have the opportunity to comment on the profile before it 
is posted but they will not have the option to refuse posting.

Minimum Requirements

    To be considered for review, interventions must meet three minimum 
requirements:
    1. Research or evaluation of the intervention has assessed mental 
health or substance use outcomes among individuals, communities, or 
populations OR other behavioral health-related outcomes on individuals, 
communities, or populations with or at risk of mental health issues or 
substance use problems.
    2. Evidence of these outcomes has been demonstrated in at least one 
study using an experimental or quasi-experimental design. Experimental 
designs require random assignment, a control or comparison group, and 
pre- and post-intervention outcome assessments. Quasi-experimental 
designs do not require random assignment, but do require a comparison 
or control group and pre- and post-intervention outcome assessments. 
Comparison/control groups must be a no-treatment control group, a wait-
list control group, a treatment-as-usual comparison group, or an 
intervention that is presumed to be ineffective or substantially less 
effective than the intervention (e.g., a ``placebo'' control or, in 
cases in which providing no treatment might be considered unethical, 
less effective treatments, even if not treatment-as-usual, such as 
``supportive therapy''). Studies with single-group, pretest-posttest 
designs or single-group, longitudinal/multiple time series do not meet 
this requirement, but will be considered to identify emerging programs 
and practices for consideration in the Learning Center. Comparative 
effectiveness trials, in which two interventions, both presumed to be 
equally effective, are compared, and studies in which the effects of 
the same intervention on various subpopulations are compared or in 
which various doses or components of the same intervention are compared 
will be reviewed, but only for purposes of providing readers in the 
Learning Center the opportunity to determine whether a particular 
program or practice may have use or show promise in different settings 
or circumstances.
    3. The results of these studies have been published in a peer-
reviewed journal or other professional publication, or documented in a 
comprehensive evaluation report, published within the previous 25 
years. Comprehensive evaluation reports must include a review of the 
literature, theoretical framework, purpose, methodology, findings/
results with statistical analysis and p values for significant 
outcomes, discussion, and conclusions.
    Applicants are required to provide full-text documents at the time 
of submission that demonstrate the intervention meets these minimum 
requirements. Other research articles, published or unpublished 
evaluation reports, grant final reports, and replication studies may be 
submitted as additional supporting documentation. Note: Abstracts or 
links to partial articles are regarded as incomplete and will not be 
considered.
    NREPP will no longer require programs and practices to have 
developed implementation materials, training and support resources, and 
quality assurance procedures. However, programs and practices with such 
dissemination and/or implementation resources will be considered for 
prioritized review if within the priorities established by the SAMHSA 
review process, and the materials, along with the location of their 
availability, will be listed in the program or practice profile.
    Applicants submitting dissemination and implementation resources 
should include a brief narrative description of the materials that are 
being submitted. These materials may include, but are not limited to, 
treatment manuals, information for administrators, information for 
direct service staff, tested training curricula, mechanisms for ongoing 
supervision and consultation, protocols for gathering process and 
outcome data, ongoing monitoring of intervention fidelity, and 
processes for gathering feedback. Applicants should also provide the 
location of where the materials can be obtained.

[[Page 38718]]

Selection of Interventions for Review

    SAMHSA will select interventions for review from among submissions 
meeting the minimum requirements. In selecting interventions for 
review, SAMHSA may give special consideration to interventions that 
meet one or more of the following conditions:
     More than one research study or evaluation has been 
conducted on the same or a similar target population that meets the 
minimum requirements.
     The intervention targets underserved populations (e.g., 
minority populations, tribal communities or American Indian/Alaska 
Native populations, elderly individuals, young adults, individuals who 
are incarcerated, etc.).
     Dissemination and implementation materials (e.g., program 
or practice manuals, training guides, measurement instruments, 
implementation fidelity tools) are available. Lower costs and no-cost 
materials may be prioritized.
     The intervention contributes to a content area in which 
few evidence-based interventions have been previously identified.
    Interventions that are not selected for review may be resubmitted 
by the applicant in a future open submission period.

The Review Process

    The review process has been revised to improve the quality of the 
reviews and utility of information that NREPP can provide its users. In 
addition to articles and reports submitted by NREPP applicants, 
additional studies, articles, and evaluation reports regarding the 
interventions will be identified through literature searches. Studies 
and outcomes to be reviewed will be determined through the systematic 
application of standardized screening criteria, and the number of 
studies and outcomes to be reviewed will be expanded to more 
comprehensively represent the evidence base for the program or 
practice. Inclusion of studies and outcomes will no longer be limited 
to positive significant outcomes; all studies and outcomes that meet 
the standardized screening criteria will be reviewed, including those 
with negative and non-significant effects. Programs and practices will 
be assessed on the basis of evaluation studies of program or practice 
impact, information related to conceptual framework (that is, program 
or practice goals, theory of change, and program or practice 
components), and information about implementation fidelity (that is, 
whether a study employs quality assurance measures to declare that the 
program or practice is delivered as intended to the program's or 
practice's target population).
    The methodological rigor (that is, internal validity, statistical 
validity, and measurement validity) of the research for each program or 
practice will be reviewed, as it pertains to each outcome examined, 
along with the magnitude and direction of the program's or practice's 
effect on each outcome. Based on this information, the program's or 
practice's effectiveness for each outcome will be rated, along with the 
rigor of the research examining the program or practice, and the 
ratings will be displayed on the NREPP Web site.
    In general, each NREPP evidence review will be conducted by two 
trained and certified reviewers. However, based on funding and 
available resources, SAMHSA use one reviewer for programs and practices 
being re-reviewed. When necessary, NREPP may conduct author queries to 
confirm or gather additional information needed for the review. Program 
and practice profiles will be developed on the basis of the information 
gathered. Applicants will have the opportunity to review the program or 
practice profile before it is posted on the NREPP site, but they will 
not have the option to refuse posting.
    Dissemination and implementation materials will no longer be rated 
as they were historically. Instead, descriptions of available materials 
for each program or practice, highlighting information that may be of 
most interest to NREPP users, will be included in the program or 
practice profile, along with information documenting the extent to 
which materials are available.
    Programs and practices currently posted on NREPP will be re-
reviewed as time and resources permit but the re-reviews of currently 
posted programs and practices will take place over the next few years.
    Detailed information about the revised review process will be 
available at http://www.nrepp.samhsa.gov after the re-launch of the new 
NREPP Web site.

Enhancing the Learning Center

    NREPP's Learning Center is a developing and underutilized component 
of the NREPP Web site. With the evolution and enhancement of the 
registry, SAMHSA seeks to bring greater recognition to both rigorously 
evaluated behavioral health interventions and those interventions that 
have been implemented, demonstrate promise, but have not necessarily 
been evaluated in a rigorous manner. To that end, the Learning Center 
is being significantly revamped to support stakeholder engagement and 
to become a shared learning environment for all stakeholders. SAMHSA 
recognizes that the successful promotion and dissemination of evidence-
based programs and practices requires an environment that promotes 
community assessment, program and practice planning and evaluation, as 
well as guidance on the selection and implementation of programs and 
practices listed on NREPP. There are useful types of evaluation 
research, often conducted among underserved populations, which provide 
valuable insights for practitioners, but do not meet the minimum 
criteria required for experimental or quasi-experimental design. SAMHSA 
intends the Learning Center to be a forum for presenting research on 
emerging programs and practices, and exploring ways that pre-
experimental and qualitative research can complement and enrich 
findings from experimental and quasi-experimental research designs. An 
inventory of such programs and practices will be compiled and 
maintained within the Learning Center and will operate in parallel to 
the listing of reviewed programs and practices with experimental and 
quasi-experimental designs. In this way, SAMHSA intends to support 
programs and practices researched with the most rigorous approaches 
while also supporting the development of practice-based evidence, 
especially for certain populations and emerging practices that are 
critical to learning and improving behavioral health outcomes for 
persons with or at risk of developing behavioral health issues.

Summer King,
Statistician.
[FR Doc. 2015-16573 Filed 7-6-15; 8:45 am]
BILLING CODE 4162-20-P