Decisions Related to the Development of a Clearinghouse of Evidence-Based Practices in Accordance With the Family First Prevention Services Act of 2018, 29122-29124 [2018-13420]

Agencies

• DEPARTMENT OF HEALTH AND HUMAN SERVICES
• Administration for Children and Families

[Federal Register Volume 83, Number 121 (Friday, June 22, 2018)]
[Notices]
[Pages 29122-29124]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2018-13420]


-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Administration for Children and Families


Decisions Related to the Development of a Clearinghouse of 
Evidence-Based Practices in Accordance With the Family First Prevention 
Services Act of 2018

AGENCY: Administration for Children and Families, HHS.

ACTION: Request for public comment.

-----------------------------------------------------------------------

SUMMARY: The Administration for Children and Families, HHS, solicits 
comments by July 22, 2018 on initial criteria and potential candidate 
programs and services for review in a Clearinghouse of evidence-based 
practices in accordance with the Family First Prevention Services Act 
of 2018. The Clearinghouse will identify promising, supported, and 
well-supported practices for mental health and substance abuse 
prevention and treatment programs, in-home parent skill-based programs, 
and kinship navigator programs appropriate for children who are 
candidates for foster care pregnant or parenting foster youth, and the 
parents or kin caregivers of those children and youth.

SUPPLEMENTARY INFORMATION: Invitation to Comment: HHS invites comments 
regarding this Notice. To ensure that your comments have maximum 
effect, please identify clearly the section of this Notice that your 
comment addresses.

1.0 Background and Legislative Context

    The Family First Prevention Services Act (FFPSA) was signed into 
law as part of the Bipartisan Budget Act (H.R. 1892) on February 9, 
2018. FFPSA enables States to use Federal funds available under parts B 
and E of title IV of the Social Security Act to provide enhanced 
support to children and families and prevent foster care placements 
through the provision of evidence-based mental health and substance 
abuse prevention and treatment services, in-home parent skill-based 
programs, and kinship navigator services. As described in the statutory 
language, these services and programs are intended ``for children who 
are candidates for foster care or who are pregnant or parenting foster 
youth and the parents or kin caregivers of the children''.
    FFPSA requires an independent systematic review of evidence to 
designate programs and services as ``promising,'' ``supported,'' and 
``well-supported'' practices, defined as follows in section 
471(e)(4)(C):

     Promising Practice: ``A practice shall be considered to 
be a `promising practice' if the practice is superior to an 
appropriate comparison practice using conventional standards of 
statistical significance (in terms of demonstrated meaningful 
improvements in validated measures of important child and parent 
outcomes, such as mental health, substance abuse, and child safety 
and well-being), as established by the results or outcomes of at 
least one study that--(1) was rated by an independent systematic 
review for the quality of the study design and execution and 
determined to be well-designed and well-executed; and (2) utilized 
some form of control (such as an untreated group, a placebo group, 
or a wait list study).''
     Supported Practice: ``A practice shall be considered to 
be a `supported practice' if (I) the practice is superior to an 
appropriate comparison practice using conventional standards of 
statistical significance (in terms of demonstrated meaningful 
improvements in validated measures of important child and parent 
outcomes, such as mental health, substance abuse, and child safety 
and well-being), as established by the results or outcomes of at 
least one study that--(aa) was rated by an independent systematic 
review for the quality of the study design and execution and 
determined to be well-designed and well-executed; and (bb) was a 
rigorous random-controlled trial (or, if not available, a study 
using a rigorous quasi-experimental research design); and (cc) was 
carried out in a usual care or practice setting and (II) the study 
described in sub-clause (I) established that the practice has a 
sustained effect (when compared to a control group) for at least 6 
months beyond the end of treatment.''
     Well-supported Practice: ``A practice shall be 
considered to be a `well-supported practice' if (I) the practice is 
superior to an appropriate comparison practice using conventional 
standards of statistical significance (in terms of demonstrated 
meaningful improvements in validated measures of important child and 
parent outcomes, such as mental health, substance abuse, and child 
safety and well-being), as established by the results or outcomes of 
at least two studies that--(aa) were rated by an independent 
systematic review for the quality of the study design and execution 
and determined to be well-designed and well-executed; and (bb) were 
rigorous random-controlled trials (or, if not available, studies 
using a rigorous quasi-experimental research design); and (cc) were 
carried out in a usual care or practice setting and (II) at least 
one of the studies described in sub-clause (I) established that the 
practice has a sustained effect (when compared to a control group) 
for at least 1 year beyond the end of treatment.''

    In accordance with FFPSA, practices must also meet the following 
requirements:

     Book or manual: The practice has a book, manual, or 
other available writings that specify the components of the practice 
protocol and describe how to administer the practice.
     No empirical risk of harm: There is no empirical basis 
suggesting that, compared to its likely benefits, the practice 
constitutes a risk of harm to those receiving it.
     Weight of evidence supports benefits: If multiple 
outcome studies have been conducted, the overall weight of evidence 
supports the benefits of the practice.
     Reliable and valid outcome measures: Outcome measures 
are reliable and valid, and are administrated consistently and 
accurately across all those receiving the practice.
     No case data for severe or frequent risk of harm: There 
is no case data suggesting a risk of harm that was probably caused 
by the treatment and that was severe or frequent (section 
471(e)(4)(C)(ii) of the Act).

    In order to meet these requirements, the Administration for 
Children and Families (ACF) in the Department of Health and Human 
Services (HHS) intends to establish and maintain a public Clearinghouse 
of practices, including culturally specific, or location- or 
population-based adaptations of practices, identified via a systematic 
review of evidence on relevant programs and services. In accordance 
with FFPSA and building from other federal evidence reviews, HHS is 
developing initial criteria that will be used to designate programs and 
services as promising, supported, and well-supported practices. HHS 
will also identify a preliminary list of candidate services and 
programs that will be considered for systematic review.
    This Notice (1) identifies and requests comment on potential 
initial criteria for (a) identifying eligible programs and services for 
review by the Clearinghouse, (b) prioritizing eligible programs and 
services for review, (c) identifying eligible studies aligned with 
prioritized programs and services, (d) prioritizing eligible studies 
for rating, (e) rating studies, and (f) rating programs and services as 
promising, supported, and well-supported practices. This Notice (2) 
requests comment on potential programs and services that may meet the 
aforementioned criteria and that should be considered as candidates for 
systematic review. After comments are received, HHS will revise and 
publish the initial criteria and a preliminary list of candidate 
programs and services to be considered for review; and begin to conduct 
reviews. This Notice is one step in ensuring that activities associated 
with the development of a Clearinghouse are

[[Page 29123]]

transparent and build from the existing knowledge of States, federal 
agencies, researchers, evaluators, program and service developers, key 
stakeholders and experts, and the general public.

2.0 Initial Criteria

    2.1 Program or Service Eligibility Criteria. Programs or services 
may be eligible for inclusion in the Clearinghouse if they meet the 
following criteria developed in accordance with FFPSA statutory 
language [sections 471(e)(1)(B) and 471(e)(1)(c)]:

    2.1.1 Types of Programs and Services. HHS intends to limit 
eligibility to mental health and substance abuse prevention and 
treatment services, in-home parent skill-based programs (including 
parenting skills training, parent education, and individual and 
family counseling), and kinship navigator programs.
    2.1.2 Book/Manual/Writings Available. HHS intends to limit 
eligibility to programs or services that have a book, manual, or 
other available documentation that specifies the components of the 
practice protocol and describes how to administer the practice.
    2.2 Program or Service Prioritization Criteria. Timing and 
resources may not allow for a detailed review of all programs and 
services determined to be eligible by the criteria detailed in 
section 2.1 Program or Service Eligibility Criteria. Programs or 
services may be prioritized for review on the basis of the following 
criteria:
    2.2.1 Types of Programs and Services. As noted in 2.1.1. Types 
of Programs and Services, HHS intends to limit eligibility to mental 
health and substance abuse prevention and treatment services, in-
home parent skill-based programs (including parenting skills 
training, parent education, and individual and family counseling), 
or kinship navigator programs. This Notice requests comment on the 
scope of programs and services and topic areas of interest within 
the aforementioned categories that should be prioritized for 
inclusion.
    2.2.2 Target Population of Interest. HHS intends to prioritize 
programs or services for review that have been developed or used to 
target children and families involved in the child welfare system or 
populations similar to those involved in the child welfare system. 
This Notice requests comment on populations that may be considered 
``similar'' to those involved in the child welfare system.
    2.2.3 Target Outcomes. HHS intends to prioritize programs or 
services for review that aim to impact target outcomes. Target 
outcomes should be defined in accordance with FFPSA statutory 
language [section 471(e)(4)(C)] and include those outcomes that ``. 
. . prevent child abuse and neglect, and reduce the likelihood of 
foster care placement by supporting birth families and kinship 
families and improving targeted supports for pregnant and parenting 
youth and their children.'' These may include, but are not limited 
to, ``. . . important child and parent outcomes, such as mental 
health, substance abuse, and child safety and well-being.'' This 
Notice requests comment on which types of mental health, substance 
abuse, and child and family outcomes should be considered as `target 
outcomes' and requests research evidence to support recommendations 
of `target outcomes'. HHS does not intend to include access to 
service, satisfaction with programs and services, and referral to 
programs and services as `target outcomes'.
    2.2.4 Number of Impact Studies. HHS intends to prioritize 
programs or services with at least two studies with non-overlapping 
analytic samples and distinct implementations examining 
effectiveness/impact.
    2.2.5 In Use/Active. HHS intends to prioritize programs or 
services currently in use in the U.S. Programs or services that are 
no longer in operation or have no information available about active 
implementation will not be prioritized.
    2.2.6 Implementation and Fidelity Support. HHS intends to 
prioritize programs or services that have implementation training 
and staff support and/or fidelity monitoring tools and resources 
available to implementers in the United States.
    2.2.7 Trauma-Informed. HHS may also prioritize services and 
programs that have been implemented using a trauma-informed 
approach. FFPSA statutory language [section 471(e)(4)(B)] states, 
``The services or programs to be provided to or on behalf of a child 
are provided under an organizational structure and treatment 
framework that involves understanding, recognizing, and responding 
to the effects of all types of trauma and in accordance with 
recognized principles of a trauma-informed approach and trauma-
specific interventions to address trauma's consequences and 
facilitate healing.'' This Notice requests comment on the 
feasibility of prioritizing programs and services based on past 
implementation in accordance with trauma-informed principles.
    2.2.8 Delivery Setting for In-Home Parent Skill-Based Programs 
and Services. HHS intends to prioritize in-home parent skill-based 
programs and services where the primary service delivery strategy 
takes place in the caregivers' place of residence.
    2.3 Study Eligibility Criteria. HHS intends to engage in a broad 
literature search to identify studies examining prioritized programs 
and services. This search may include databases, websites, existing 
literature reviews, and meta-analyses. HHS intends to screen studies 
for eligibility using the following criteria:
    2.3.1 Impact Study. HHS intends to limit eligibility to studies 
included in government reports and peer-reviewed journal articles 
that assess effectiveness (i.e., impact) using quantitative methods.
    2.3.2 Target Outcomes. HHS intends to limit eligibility to 
studies that examine the impact of the service or program on at 
least one `target outcome', as described in section 2.2.3. Target 
Outcomes. As noted above, this Notice requests comment on specific 
outcomes in accordance with FFPSA statutory language that should be 
considered `target outcomes' and requests research evidence to 
support recommendations of `target outcomes'.
    2.3.3 Conducted in the U.S., U.K., Canada, New Zealand, or 
Australia. HHS intends to limit eligibility to studies conducted 
with samples in the U.S., U.K., Canada, New Zealand, or Australia to 
ensure that the evidence base reflects the populations where 
programs and services will be implemented.
    2.3.4 Study Published in English. HHS intends to limit 
eligibility to studies published in English.
    2.3.5 Published or Prepared in or after 1990. HHS intends to 
limit eligibility to studies published or prepared in or after 1990.
    2.3.6 Usual Care or Practice Setting. HHS intends to limit 
eligibility to studies carried out in a usual care or practice 
setting in accordance with FFPSA [section 471(e)(4)(C)]. This Notice 
requests comment on the definition of usual care or practice 
settings.
    2.4 Study Prioritization Criteria. Timing and resources may not 
allow for a detailed rating of all studies determined to be eligible 
by the criteria identified in section 2.3 Study Eligibility 
Criteria. HHS intends to conduct a high-level scan of eligible 
studies to determine which should be prioritized for rating. This 
Notice requests comment on criteria that can be used to prioritize 
eligible studies for rating.
    2.4.1 Implementation Period: FFPSA [section 471(e)(1)(A) and 
(B)] states that the Secretary may make a payment to a State for 
providing services or programs ``for not more than a 12-month 
period''. This Notice requests comment on whether studies with 
program or service implementation periods of longer than 12 months 
should be considered for review and if so, whether any other 
implementation period cutoff should be included as a study 
prioritization criterion.
    2.4.2 Sample of Interest. HHS intends to prioritize studies that 
include samples of children and families involved in the child 
welfare system or populations similar to those involved in the child 
welfare system. This Notice requests comment on populations that may 
be considered ``similar'' to those involved in the child welfare 
system.
    2.5 Study Rating Criteria. HHS intends to rate studies on the 
following criteria:
    2.5.1 Favorable Effects. HHS intends to rate studies based on 
whether they demonstrate at least one meaningful favorable effect 
(i.e., positive significant effect) on a `target outcome' as 
specified in section 2.3.2 Target Outcomes. A meaningful effect will 
be defined using conventional standards of statistical significance 
(i.e., two-tailed hypothesis test and a specified alpha level of p < 
.05). This Notice requests comment on whether and how ratings should 
consider the number or magnitude of favorable effects.
    2.5.2 Unfavorable Effects. HHS intends to rate studies based on 
the number of unfavorable effects (i.e., negative significant 
effects) on either `target' or non-target outcomes as specified in 
section 2.3.2 Target Outcomes. Effects will be defined using 
conventional standards of statistical significance (i.e., two-tailed 
hypothesis test and a specified alpha level of p < .05). This

[[Page 29124]]

Notice requests comment on whether and how studies should also be 
rated on the number of null effects on `target outcomes', and on 
whether and how ratings should consider the number or magnitude of 
unfavorable effects.
    2.5.3 Sustained Favorable Effect. HHS intends for studies with 
at least one favorable effect on a `target outcome', as determined 
by the criteria in 2.5.1 Favorable Effects, to be rated on whether 
or not they demonstrate a sustained favorable effect. As noted in 
section 471(e)(4)(C), a `supported practice' must have at least one 
study that demonstrates ``a sustained effect (when compared to a 
control group) for at least 6 months beyond the end of treatment'' 
and a `well-supported practice' must have at least one study that 
demonstrates ``a sustained effect (when compared to a control group) 
for at least 1 year beyond the end of treatment.'' HHS intends to 
classify studies as not demonstrating a sustained favorable effect 
(i.e., effects are demonstrated for less than 6 months), 
demonstrating a sustained favorable effect of 6 months or more (but 
less than 12 months), or demonstrating a sustained favorable effect 
of 12 months or more.
    2.5.4 Rigorous Study Design. HHS intends to rate studies as 
either high, moderate, or low on the rigor and appropriateness of 
their study design. Study designs that receive the highest rating 
will be either Randomized Controlled Trials (RCTs) or rigorous 
quasi-experimental designs. HHS defines randomized controlled trials 
as a study design in which sample members are assigned to the 
program or service and comparison groups by chance. Randomized 
control designs are often considered the ``gold standard'' of 
research design because personal characteristics (before the program 
or service begins) do not affect whether someone is assigned to the 
program or service or control group. HHS defines a quasi-
experimental design as a study design in which sample members are 
selected for the program or service and comparison groups in a 
nonrandom way. Similar to criteria considered in other federal 
evidence clearinghouses, rigorous study designs will be those that 
are appropriately powered, include an appropriate control group, 
maintain original assignment to study arms, and are appropriate to 
combat threats to internal validity. This Notice requests comment on 
threats to internal validity that should be considered. This Notice 
requests comment on appropriate thresholds for evaluating and 
assigning a rating to a study design.
    2.5.5 Rigorous Study Analysis. HHS intends to rate studies as 
either high, moderate, or low on the rigor and appropriateness of 
their analysis. Study analyses that receive the highest rating may 
be those that tested and established baseline equivalence, 
appropriately accounted for overall and differential sample 
attrition, appropriately accounted for multiple comparisons, and 
when necessary accounted for clustering. This Notice requests 
comment on appropriate thresholds for evaluating and assigning a 
rating to a study analysis.
    2.5.6 Reliability, Validity, and Systematic Administration of 
Outcome Measures. HHS intends to rate studies as either high, 
moderate, or low on the extent to which `target outcome' measures 
are reliable (i.e., the extent to which a measure produces the same 
results when used repeatedly), valid (i.e., the extent to which a 
measure captures what it is intended to measure), and were 
administered consistently and accurately across all those receiving 
the practice in accordance with FFPSA statutory language [section 
471(e)(4)(C)] or receiving the appropriate comparison practice. This 
Notice requests comment on appropriate thresholds for evaluating and 
assigning a rating to the reliability, validity, and administration 
of `target outcome' measures.
    2.6 Program or Service Rating Criteria. HHS intends for programs 
or services to be rated as promising, supported, or well-supported 
practices if they meet the below criteria that collectively assess 
the strength of evidence for a practice and build from the 
individual study criteria described in section 2.5 Study Rating 
Criteria. These criteria were developed in accordance with FFPSA 
statutory language [section 471(e)(4)(C)].
    2.6.1 Promising Practice: HHS intends to designate a program or 
service as a `promising practice' if the program or service has at 
least one study that demonstrates a favorable effect on a target 
outcome as described by criterion 2.5.1 Favorable Effects and 
achieves, at a minimum, moderate ratings on criteria 2.5.4 through 
2.5.6.
    2.6.2 Supported Practice: HHS intends to designate a program or 
service as a `supported practice' if the program or service has at 
least one study that demonstrates a favorable effect on a target 
outcome as described by 2.5.1 Favorable Effects, demonstrates a 
sustained favorable effect on a target outcome of at least 6 months 
beyond the end of treatment as described in Section 2.5.3 Sustained 
Favorable Effect, and achieves the high rating on criteria 2.5.4 
through 2.5.6.
    2.6.3 Well-Supported Practice: HHS intends to designate a 
program or service as a `well-supported practice' if the practice 
has at least two studies with non-overlapping analytic samples and 
distinct implementations that demonstrate favorable effects as 
described by 2.5.1 Favorable Effects, demonstrate sustained 
favorable effects of at least 12 months beyond the end of treatment 
as described in Section 2.5.3 Sustained Favorable Effect, and 
achieve the high rating on criteria 2.5.4 through 2.5.6.

    HHS does not intend to rate a program or service as a `promising', 
`supported', or `well-supported practice' if there is an empirical 
basis, as evidenced by multiple unfavorable effects on target or non-
target outcomes across reviewed studies, as described in 2.5.2 
Unfavorable Effects, that suggest the overall weight of evidence does 
not support the benefits of the program or service. This Notice 
requests comment on approaches for determining that promising, 
supported, and well-supported practices do not constitute a risk of 
harm. As described in FFPSA [section 471(e)(4)(C)], ``There is no 
empirical basis suggesting that, compared to its likely benefits, the 
practice constitutes a risk of harm to those receiving it'', ``If 
multiple outcome studies are conducted, the overall weight of evidences 
supports the benefits of the practice'', and ``There is no case data 
suggesting a risk of harm that was probably caused by the treatment and 
that was severe or frequent''.

3.0 Recommendations of Potential Candidate Programs and Services for 
Review

    This Notice requests comment on potential candidate programs and 
services to consider for the systematic evidence review. Comments 
should identify how recommended programs and services meet the criteria 
described in section 2.1 Program or Service Eligibility Criteria. These 
criteria include: Types of Programs and Services and Book/Manual/
Writings Available. Comments should also identify how recommended 
programs and services meet the criteria described in section 2.2 
Program or Service Prioritization Criteria. These criteria include: 
Types of Programs and Services, Target Population of Interest, Target 
Outcomes, Number of Impact Studies, In Use/Active, Implementation and 
Fidelity Support, Trauma-Informed, and Delivery Setting for In-Home 
Parent Skill-Based Programs and Services. In order to leverage new 
insights from the field, HHS may put forth additional future Notices 
requesting recommendations of potential candidate programs and services 
for review.

4.0 Submission of Comments

    Comments may be submitted until July 22, 2018 by email to 
[email protected].

Naomi Goldstein,
Deputy Assistant Secretary for Planning, Research, and Evaluation.
[FR Doc. 2018-13420 Filed 6-21-18; 8:45 am]
 BILLING CODE 4184-25-P