Decisions Related to the Development of a Clearinghouse of Evidence-Based Practices in Accordance With the Family First Prevention Services Act of 2018, 29122-29124 [2018-13420]

Download as PDF 29122 Federal Register / Vol. 83, No. 121 / Friday, June 22, 2018 / Notices Dated: June 19, 2018. Seema Verma, Administrator, Centers for Medicare & Medicaid Services. who are pregnant or parenting foster youth and the parents or kin caregivers of the children’’. FFPSA requires an independent systematic review of evidence to designate programs and services as ‘‘promising,’’ ‘‘supported,’’ and ‘‘wellsupported’’ practices, defined as follows in section 471(e)(4)(C): [FR Doc. 2018–13503 Filed 6–20–18; 8:45 am] BILLING CODE 4120–01–P DEPARTMENT OF HEALTH AND HUMAN SERVICES Administration for Children and Families Decisions Related to the Development of a Clearinghouse of Evidence-Based Practices in Accordance With the Family First Prevention Services Act of 2018 Administration for Children and Families, HHS. ACTION: Request for public comment. AGENCY: The Administration for Children and Families, HHS, solicits comments by July 22, 2018 on initial criteria and potential candidate programs and services for review in a Clearinghouse of evidence-based practices in accordance with the Family First Prevention Services Act of 2018. The Clearinghouse will identify promising, supported, and wellsupported practices for mental health and substance abuse prevention and treatment programs, in-home parent skill-based programs, and kinship navigator programs appropriate for children who are candidates for foster care pregnant or parenting foster youth, and the parents or kin caregivers of those children and youth. SUPPLEMENTARY INFORMATION: Invitation to Comment: HHS invites comments regarding this Notice. To ensure that your comments have maximum effect, please identify clearly the section of this Notice that your comment addresses. SUMMARY: daltland on DSKBBV9HB2PROD with NOTICES 1.0 Background and Legislative Context The Family First Prevention Services Act (FFPSA) was signed into law as part of the Bipartisan Budget Act (H.R. 1892) on February 9, 2018. FFPSA enables States to use Federal funds available under parts B and E of title IV of the Social Security Act to provide enhanced support to children and families and prevent foster care placements through the provision of evidence-based mental health and substance abuse prevention and treatment services, in-home parent skill-based programs, and kinship navigator services. As described in the statutory language, these services and programs are intended ‘‘for children who are candidates for foster care or VerDate Sep<11>2014 17:16 Jun 21, 2018 Jkt 244001 • Promising Practice: ‘‘A practice shall be considered to be a ‘promising practice’ if the practice is superior to an appropriate comparison practice using conventional standards of statistical significance (in terms of demonstrated meaningful improvements in validated measures of important child and parent outcomes, such as mental health, substance abuse, and child safety and wellbeing), as established by the results or outcomes of at least one study that—(1) was rated by an independent systematic review for the quality of the study design and execution and determined to be welldesigned and well-executed; and (2) utilized some form of control (such as an untreated group, a placebo group, or a wait list study).’’ • Supported Practice: ‘‘A practice shall be considered to be a ‘supported practice’ if (I) the practice is superior to an appropriate comparison practice using conventional standards of statistical significance (in terms of demonstrated meaningful improvements in validated measures of important child and parent outcomes, such as mental health, substance abuse, and child safety and wellbeing), as established by the results or outcomes of at least one study that—(aa) was rated by an independent systematic review for the quality of the study design and execution and determined to be welldesigned and well-executed; and (bb) was a rigorous random-controlled trial (or, if not available, a study using a rigorous quasiexperimental research design); and (cc) was carried out in a usual care or practice setting and (II) the study described in sub-clause (I) established that the practice has a sustained effect (when compared to a control group) for at least 6 months beyond the end of treatment.’’ • Well-supported Practice: ‘‘A practice shall be considered to be a ‘well-supported practice’ if (I) the practice is superior to an appropriate comparison practice using conventional standards of statistical significance (in terms of demonstrated meaningful improvements in validated measures of important child and parent outcomes, such as mental health, substance abuse, and child safety and well-being), as established by the results or outcomes of at least two studies that—(aa) were rated by an independent systematic review for the quality of the study design and execution and determined to be well-designed and wellexecuted; and (bb) were rigorous randomcontrolled trials (or, if not available, studies using a rigorous quasi-experimental research design); and (cc) were carried out in a usual care or practice setting and (II) at least one of the studies described in sub-clause (I) established that the practice has a sustained effect (when compared to a control group) for at least 1 year beyond the end of treatment.’’ PO 00000 Frm 00035 Fmt 4703 Sfmt 4703 In accordance with FFPSA, practices must also meet the following requirements: • Book or manual: The practice has a book, manual, or other available writings that specify the components of the practice protocol and describe how to administer the practice. • No empirical risk of harm: There is no empirical basis suggesting that, compared to its likely benefits, the practice constitutes a risk of harm to those receiving it. • Weight of evidence supports benefits: If multiple outcome studies have been conducted, the overall weight of evidence supports the benefits of the practice. • Reliable and valid outcome measures: Outcome measures are reliable and valid, and are administrated consistently and accurately across all those receiving the practice. • No case data for severe or frequent risk of harm: There is no case data suggesting a risk of harm that was probably caused by the treatment and that was severe or frequent (section 471(e)(4)(C)(ii) of the Act). In order to meet these requirements, the Administration for Children and Families (ACF) in the Department of Health and Human Services (HHS) intends to establish and maintain a public Clearinghouse of practices, including culturally specific, or location- or population-based adaptations of practices, identified via a systematic review of evidence on relevant programs and services. In accordance with FFPSA and building from other federal evidence reviews, HHS is developing initial criteria that will be used to designate programs and services as promising, supported, and well-supported practices. HHS will also identify a preliminary list of candidate services and programs that will be considered for systematic review. This Notice (1) identifies and requests comment on potential initial criteria for (a) identifying eligible programs and services for review by the Clearinghouse, (b) prioritizing eligible programs and services for review, (c) identifying eligible studies aligned with prioritized programs and services, (d) prioritizing eligible studies for rating, (e) rating studies, and (f) rating programs and services as promising, supported, and well-supported practices. This Notice (2) requests comment on potential programs and services that may meet the aforementioned criteria and that should be considered as candidates for systematic review. After comments are received, HHS will revise and publish the initial criteria and a preliminary list of candidate programs and services to be considered for review; and begin to conduct reviews. This Notice is one step in ensuring that activities associated with the development of a Clearinghouse are E:\FR\FM\22JNN1.SGM 22JNN1 Federal Register / Vol. 83, No. 121 / Friday, June 22, 2018 / Notices transparent and build from the existing knowledge of States, federal agencies, researchers, evaluators, program and service developers, key stakeholders and experts, and the general public. daltland on DSKBBV9HB2PROD with NOTICES 2.0 Initial Criteria 2.1 Program or Service Eligibility Criteria. Programs or services may be eligible for inclusion in the Clearinghouse if they meet the following criteria developed in accordance with FFPSA statutory language [sections 471(e)(1)(B) and 471(e)(1)(c)]: 2.1.1 Types of Programs and Services. HHS intends to limit eligibility to mental health and substance abuse prevention and treatment services, in-home parent skillbased programs (including parenting skills training, parent education, and individual and family counseling), and kinship navigator programs. 2.1.2 Book/Manual/Writings Available. HHS intends to limit eligibility to programs or services that have a book, manual, or other available documentation that specifies the components of the practice protocol and describes how to administer the practice. 2.2 Program or Service Prioritization Criteria. Timing and resources may not allow for a detailed review of all programs and services determined to be eligible by the criteria detailed in section 2.1 Program or Service Eligibility Criteria. Programs or services may be prioritized for review on the basis of the following criteria: 2.2.1 Types of Programs and Services. As noted in 2.1.1. Types of Programs and Services, HHS intends to limit eligibility to mental health and substance abuse prevention and treatment services, in-home parent skill-based programs (including parenting skills training, parent education, and individual and family counseling), or kinship navigator programs. This Notice requests comment on the scope of programs and services and topic areas of interest within the aforementioned categories that should be prioritized for inclusion. 2.2.2 Target Population of Interest. HHS intends to prioritize programs or services for review that have been developed or used to target children and families involved in the child welfare system or populations similar to those involved in the child welfare system. This Notice requests comment on populations that may be considered ‘‘similar’’ to those involved in the child welfare system. 2.2.3 Target Outcomes. HHS intends to prioritize programs or services for review that aim to impact target outcomes. Target outcomes should be defined in accordance with FFPSA statutory language [section 471(e)(4)(C)] and include those outcomes that ‘‘. . . prevent child abuse and neglect, and reduce the likelihood of foster care placement by supporting birth families and kinship families and improving targeted supports for pregnant and parenting youth and their children.’’ These may include, but are not limited to, ‘‘. . . important child and parent outcomes, such as mental health, substance abuse, and child safety and well- VerDate Sep<11>2014 17:16 Jun 21, 2018 Jkt 244001 being.’’ This Notice requests comment on which types of mental health, substance abuse, and child and family outcomes should be considered as ‘target outcomes’ and requests research evidence to support recommendations of ‘target outcomes’. HHS does not intend to include access to service, satisfaction with programs and services, and referral to programs and services as ‘target outcomes’. 2.2.4 Number of Impact Studies. HHS intends to prioritize programs or services with at least two studies with nonoverlapping analytic samples and distinct implementations examining effectiveness/ impact. 2.2.5 In Use/Active. HHS intends to prioritize programs or services currently in use in the U.S. Programs or services that are no longer in operation or have no information available about active implementation will not be prioritized. 2.2.6 Implementation and Fidelity Support. HHS intends to prioritize programs or services that have implementation training and staff support and/or fidelity monitoring tools and resources available to implementers in the United States. 2.2.7 Trauma-Informed. HHS may also prioritize services and programs that have been implemented using a trauma-informed approach. FFPSA statutory language [section 471(e)(4)(B)] states, ‘‘The services or programs to be provided to or on behalf of a child are provided under an organizational structure and treatment framework that involves understanding, recognizing, and responding to the effects of all types of trauma and in accordance with recognized principles of a trauma-informed approach and trauma-specific interventions to address trauma’s consequences and facilitate healing.’’ This Notice requests comment on the feasibility of prioritizing programs and services based on past implementation in accordance with trauma-informed principles. 2.2.8 Delivery Setting for In-Home Parent Skill-Based Programs and Services. HHS intends to prioritize in-home parent skillbased programs and services where the primary service delivery strategy takes place in the caregivers’ place of residence. 2.3 Study Eligibility Criteria. HHS intends to engage in a broad literature search to identify studies examining prioritized programs and services. This search may include databases, websites, existing literature reviews, and meta-analyses. HHS intends to screen studies for eligibility using the following criteria: 2.3.1 Impact Study. HHS intends to limit eligibility to studies included in government reports and peer-reviewed journal articles that assess effectiveness (i.e., impact) using quantitative methods. 2.3.2 Target Outcomes. HHS intends to limit eligibility to studies that examine the impact of the service or program on at least one ‘target outcome’, as described in section 2.2.3. Target Outcomes. As noted above, this Notice requests comment on specific outcomes in accordance with FFPSA statutory language that should be considered ‘target outcomes’ and requests research evidence to support recommendations of ‘target outcomes’. PO 00000 Frm 00036 Fmt 4703 Sfmt 4703 29123 2.3.3 Conducted in the U.S., U.K., Canada, New Zealand, or Australia. HHS intends to limit eligibility to studies conducted with samples in the U.S., U.K., Canada, New Zealand, or Australia to ensure that the evidence base reflects the populations where programs and services will be implemented. 2.3.4 Study Published in English. HHS intends to limit eligibility to studies published in English. 2.3.5 Published or Prepared in or after 1990. HHS intends to limit eligibility to studies published or prepared in or after 1990. 2.3.6 Usual Care or Practice Setting. HHS intends to limit eligibility to studies carried out in a usual care or practice setting in accordance with FFPSA [section 471(e)(4)(C)]. This Notice requests comment on the definition of usual care or practice settings. 2.4 Study Prioritization Criteria. Timing and resources may not allow for a detailed rating of all studies determined to be eligible by the criteria identified in section 2.3 Study Eligibility Criteria. HHS intends to conduct a high-level scan of eligible studies to determine which should be prioritized for rating. This Notice requests comment on criteria that can be used to prioritize eligible studies for rating. 2.4.1 Implementation Period: FFPSA [section 471(e)(1)(A) and (B)] states that the Secretary may make a payment to a State for providing services or programs ‘‘for not more than a 12-month period’’. This Notice requests comment on whether studies with program or service implementation periods of longer than 12 months should be considered for review and if so, whether any other implementation period cutoff should be included as a study prioritization criterion. 2.4.2 Sample of Interest. HHS intends to prioritize studies that include samples of children and families involved in the child welfare system or populations similar to those involved in the child welfare system. This Notice requests comment on populations that may be considered ‘‘similar’’ to those involved in the child welfare system. 2.5 Study Rating Criteria. HHS intends to rate studies on the following criteria: 2.5.1 Favorable Effects. HHS intends to rate studies based on whether they demonstrate at least one meaningful favorable effect (i.e., positive significant effect) on a ‘target outcome’ as specified in section 2.3.2 Target Outcomes. A meaningful effect will be defined using conventional standards of statistical significance (i.e., twotailed hypothesis test and a specified alpha level of p < .05). This Notice requests comment on whether and how ratings should consider the number or magnitude of favorable effects. 2.5.2 Unfavorable Effects. HHS intends to rate studies based on the number of unfavorable effects (i.e., negative significant effects) on either ‘target’ or non-target outcomes as specified in section 2.3.2 Target Outcomes. Effects will be defined using conventional standards of statistical significance (i.e., two-tailed hypothesis test and a specified alpha level of p < .05). This E:\FR\FM\22JNN1.SGM 22JNN1 daltland on DSKBBV9HB2PROD with NOTICES 29124 Federal Register / Vol. 83, No. 121 / Friday, June 22, 2018 / Notices Notice requests comment on whether and how studies should also be rated on the number of null effects on ‘target outcomes’, and on whether and how ratings should consider the number or magnitude of unfavorable effects. 2.5.3 Sustained Favorable Effect. HHS intends for studies with at least one favorable effect on a ‘target outcome’, as determined by the criteria in 2.5.1 Favorable Effects, to be rated on whether or not they demonstrate a sustained favorable effect. As noted in section 471(e)(4)(C), a ‘supported practice’ must have at least one study that demonstrates ‘‘a sustained effect (when compared to a control group) for at least 6 months beyond the end of treatment’’ and a ‘well-supported practice’ must have at least one study that demonstrates ‘‘a sustained effect (when compared to a control group) for at least 1 year beyond the end of treatment.’’ HHS intends to classify studies as not demonstrating a sustained favorable effect (i.e., effects are demonstrated for less than 6 months), demonstrating a sustained favorable effect of 6 months or more (but less than 12 months), or demonstrating a sustained favorable effect of 12 months or more. 2.5.4 Rigorous Study Design. HHS intends to rate studies as either high, moderate, or low on the rigor and appropriateness of their study design. Study designs that receive the highest rating will be either Randomized Controlled Trials (RCTs) or rigorous quasi-experimental designs. HHS defines randomized controlled trials as a study design in which sample members are assigned to the program or service and comparison groups by chance. Randomized control designs are often considered the ‘‘gold standard’’ of research design because personal characteristics (before the program or service begins) do not affect whether someone is assigned to the program or service or control group. HHS defines a quasi-experimental design as a study design in which sample members are selected for the program or service and comparison groups in a nonrandom way. Similar to criteria considered in other federal evidence clearinghouses, rigorous study designs will be those that are appropriately powered, include an appropriate control group, maintain original assignment to study arms, and are appropriate to combat threats to internal validity. This Notice requests comment on threats to internal validity that should be considered. This Notice requests comment on appropriate thresholds for evaluating and assigning a rating to a study design. 2.5.5 Rigorous Study Analysis. HHS intends to rate studies as either high, moderate, or low on the rigor and appropriateness of their analysis. Study analyses that receive the highest rating may be those that tested and established baseline equivalence, appropriately accounted for overall and differential sample attrition, appropriately accounted for multiple comparisons, and when necessary accounted for clustering. This Notice requests comment on appropriate thresholds for evaluating and assigning a rating to a study analysis. 2.5.6 Reliability, Validity, and Systematic Administration of Outcome Measures. HHS VerDate Sep<11>2014 17:16 Jun 21, 2018 Jkt 244001 intends to rate studies as either high, moderate, or low on the extent to which ‘target outcome’ measures are reliable (i.e., the extent to which a measure produces the same results when used repeatedly), valid (i.e., the extent to which a measure captures what it is intended to measure), and were administered consistently and accurately across all those receiving the practice in accordance with FFPSA statutory language [section 471(e)(4)(C)] or receiving the appropriate comparison practice. This Notice requests comment on appropriate thresholds for evaluating and assigning a rating to the reliability, validity, and administration of ‘target outcome’ measures. 2.6 Program or Service Rating Criteria. HHS intends for programs or services to be rated as promising, supported, or wellsupported practices if they meet the below criteria that collectively assess the strength of evidence for a practice and build from the individual study criteria described in section 2.5 Study Rating Criteria. These criteria were developed in accordance with FFPSA statutory language [section 471(e)(4)(C)]. 2.6.1 Promising Practice: HHS intends to designate a program or service as a ‘promising practice’ if the program or service has at least one study that demonstrates a favorable effect on a target outcome as described by criterion 2.5.1 Favorable Effects and achieves, at a minimum, moderate ratings on criteria 2.5.4 through 2.5.6. 2.6.2 Supported Practice: HHS intends to designate a program or service as a ‘supported practice’ if the program or service has at least one study that demonstrates a favorable effect on a target outcome as described by 2.5.1 Favorable Effects, demonstrates a sustained favorable effect on a target outcome of at least 6 months beyond the end of treatment as described in Section 2.5.3 Sustained Favorable Effect, and achieves the high rating on criteria 2.5.4 through 2.5.6. 2.6.3 Well-Supported Practice: HHS intends to designate a program or service as a ‘well-supported practice’ if the practice has at least two studies with non-overlapping analytic samples and distinct implementations that demonstrate favorable effects as described by 2.5.1 Favorable Effects, demonstrate sustained favorable effects of at least 12 months beyond the end of treatment as described in Section 2.5.3 Sustained Favorable Effect, and achieve the high rating on criteria 2.5.4 through 2.5.6. HHS does not intend to rate a program or service as a ‘promising’, ‘supported’, or ‘well-supported practice’ if there is an empirical basis, as evidenced by multiple unfavorable effects on target or non-target outcomes across reviewed studies, as described in 2.5.2 Unfavorable Effects, that suggest the overall weight of evidence does not support the benefits of the program or service. This Notice requests comment on approaches for determining that promising, supported, and wellsupported practices do not constitute a risk of harm. As described in FFPSA [section 471(e)(4)(C)], ‘‘There is no PO 00000 Frm 00037 Fmt 4703 Sfmt 4703 empirical basis suggesting that, compared to its likely benefits, the practice constitutes a risk of harm to those receiving it’’, ‘‘If multiple outcome studies are conducted, the overall weight of evidences supports the benefits of the practice’’, and ‘‘There is no case data suggesting a risk of harm that was probably caused by the treatment and that was severe or frequent’’. 3.0 Recommendations of Potential Candidate Programs and Services for Review This Notice requests comment on potential candidate programs and services to consider for the systematic evidence review. Comments should identify how recommended programs and services meet the criteria described in section 2.1 Program or Service Eligibility Criteria. These criteria include: Types of Programs and Services and Book/Manual/Writings Available. Comments should also identify how recommended programs and services meet the criteria described in section 2.2 Program or Service Prioritization Criteria. These criteria include: Types of Programs and Services, Target Population of Interest, Target Outcomes, Number of Impact Studies, In Use/ Active, Implementation and Fidelity Support, Trauma-Informed, and Delivery Setting for In-Home Parent Skill-Based Programs and Services. In order to leverage new insights from the field, HHS may put forth additional future Notices requesting recommendations of potential candidate programs and services for review. 4.0 Submission of Comments Comments may be submitted until July 22, 2018 by email to ffclearinghouse@acf.hhs.gov. Naomi Goldstein, Deputy Assistant Secretary for Planning, Research, and Evaluation. [FR Doc. 2018–13420 Filed 6–21–18; 8:45 am] BILLING CODE 4184–25–P DEPARTMENT OF HEALTH AND HUMAN SERVICES Food and Drug Administration [Docket No. FDA–2018–N–2066] Tobacco Products Scientific Advisory Committee; Notice of Meeting AGENCY: Food and Drug Administration, HHS. ACTION: Notice. The Food and Drug Administration (FDA) announces a SUMMARY: E:\FR\FM\22JNN1.SGM 22JNN1

Agencies

[Federal Register Volume 83, Number 121 (Friday, June 22, 2018)]
[Notices]
[Pages 29122-29124]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2018-13420]


-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Administration for Children and Families


Decisions Related to the Development of a Clearinghouse of 
Evidence-Based Practices in Accordance With the Family First Prevention 
Services Act of 2018

AGENCY: Administration for Children and Families, HHS.

ACTION: Request for public comment.

-----------------------------------------------------------------------

SUMMARY: The Administration for Children and Families, HHS, solicits 
comments by July 22, 2018 on initial criteria and potential candidate 
programs and services for review in a Clearinghouse of evidence-based 
practices in accordance with the Family First Prevention Services Act 
of 2018. The Clearinghouse will identify promising, supported, and 
well-supported practices for mental health and substance abuse 
prevention and treatment programs, in-home parent skill-based programs, 
and kinship navigator programs appropriate for children who are 
candidates for foster care pregnant or parenting foster youth, and the 
parents or kin caregivers of those children and youth.

SUPPLEMENTARY INFORMATION: Invitation to Comment: HHS invites comments 
regarding this Notice. To ensure that your comments have maximum 
effect, please identify clearly the section of this Notice that your 
comment addresses.

1.0 Background and Legislative Context

    The Family First Prevention Services Act (FFPSA) was signed into 
law as part of the Bipartisan Budget Act (H.R. 1892) on February 9, 
2018. FFPSA enables States to use Federal funds available under parts B 
and E of title IV of the Social Security Act to provide enhanced 
support to children and families and prevent foster care placements 
through the provision of evidence-based mental health and substance 
abuse prevention and treatment services, in-home parent skill-based 
programs, and kinship navigator services. As described in the statutory 
language, these services and programs are intended ``for children who 
are candidates for foster care or who are pregnant or parenting foster 
youth and the parents or kin caregivers of the children''.
    FFPSA requires an independent systematic review of evidence to 
designate programs and services as ``promising,'' ``supported,'' and 
``well-supported'' practices, defined as follows in section 
471(e)(4)(C):

     Promising Practice: ``A practice shall be considered to 
be a `promising practice' if the practice is superior to an 
appropriate comparison practice using conventional standards of 
statistical significance (in terms of demonstrated meaningful 
improvements in validated measures of important child and parent 
outcomes, such as mental health, substance abuse, and child safety 
and well-being), as established by the results or outcomes of at 
least one study that--(1) was rated by an independent systematic 
review for the quality of the study design and execution and 
determined to be well-designed and well-executed; and (2) utilized 
some form of control (such as an untreated group, a placebo group, 
or a wait list study).''
     Supported Practice: ``A practice shall be considered to 
be a `supported practice' if (I) the practice is superior to an 
appropriate comparison practice using conventional standards of 
statistical significance (in terms of demonstrated meaningful 
improvements in validated measures of important child and parent 
outcomes, such as mental health, substance abuse, and child safety 
and well-being), as established by the results or outcomes of at 
least one study that--(aa) was rated by an independent systematic 
review for the quality of the study design and execution and 
determined to be well-designed and well-executed; and (bb) was a 
rigorous random-controlled trial (or, if not available, a study 
using a rigorous quasi-experimental research design); and (cc) was 
carried out in a usual care or practice setting and (II) the study 
described in sub-clause (I) established that the practice has a 
sustained effect (when compared to a control group) for at least 6 
months beyond the end of treatment.''
     Well-supported Practice: ``A practice shall be 
considered to be a `well-supported practice' if (I) the practice is 
superior to an appropriate comparison practice using conventional 
standards of statistical significance (in terms of demonstrated 
meaningful improvements in validated measures of important child and 
parent outcomes, such as mental health, substance abuse, and child 
safety and well-being), as established by the results or outcomes of 
at least two studies that--(aa) were rated by an independent 
systematic review for the quality of the study design and execution 
and determined to be well-designed and well-executed; and (bb) were 
rigorous random-controlled trials (or, if not available, studies 
using a rigorous quasi-experimental research design); and (cc) were 
carried out in a usual care or practice setting and (II) at least 
one of the studies described in sub-clause (I) established that the 
practice has a sustained effect (when compared to a control group) 
for at least 1 year beyond the end of treatment.''

    In accordance with FFPSA, practices must also meet the following 
requirements:

     Book or manual: The practice has a book, manual, or 
other available writings that specify the components of the practice 
protocol and describe how to administer the practice.
     No empirical risk of harm: There is no empirical basis 
suggesting that, compared to its likely benefits, the practice 
constitutes a risk of harm to those receiving it.
     Weight of evidence supports benefits: If multiple 
outcome studies have been conducted, the overall weight of evidence 
supports the benefits of the practice.
     Reliable and valid outcome measures: Outcome measures 
are reliable and valid, and are administrated consistently and 
accurately across all those receiving the practice.
     No case data for severe or frequent risk of harm: There 
is no case data suggesting a risk of harm that was probably caused 
by the treatment and that was severe or frequent (section 
471(e)(4)(C)(ii) of the Act).

    In order to meet these requirements, the Administration for 
Children and Families (ACF) in the Department of Health and Human 
Services (HHS) intends to establish and maintain a public Clearinghouse 
of practices, including culturally specific, or location- or 
population-based adaptations of practices, identified via a systematic 
review of evidence on relevant programs and services. In accordance 
with FFPSA and building from other federal evidence reviews, HHS is 
developing initial criteria that will be used to designate programs and 
services as promising, supported, and well-supported practices. HHS 
will also identify a preliminary list of candidate services and 
programs that will be considered for systematic review.
    This Notice (1) identifies and requests comment on potential 
initial criteria for (a) identifying eligible programs and services for 
review by the Clearinghouse, (b) prioritizing eligible programs and 
services for review, (c) identifying eligible studies aligned with 
prioritized programs and services, (d) prioritizing eligible studies 
for rating, (e) rating studies, and (f) rating programs and services as 
promising, supported, and well-supported practices. This Notice (2) 
requests comment on potential programs and services that may meet the 
aforementioned criteria and that should be considered as candidates for 
systematic review. After comments are received, HHS will revise and 
publish the initial criteria and a preliminary list of candidate 
programs and services to be considered for review; and begin to conduct 
reviews. This Notice is one step in ensuring that activities associated 
with the development of a Clearinghouse are

[[Page 29123]]

transparent and build from the existing knowledge of States, federal 
agencies, researchers, evaluators, program and service developers, key 
stakeholders and experts, and the general public.

2.0 Initial Criteria

    2.1 Program or Service Eligibility Criteria. Programs or services 
may be eligible for inclusion in the Clearinghouse if they meet the 
following criteria developed in accordance with FFPSA statutory 
language [sections 471(e)(1)(B) and 471(e)(1)(c)]:

    2.1.1 Types of Programs and Services. HHS intends to limit 
eligibility to mental health and substance abuse prevention and 
treatment services, in-home parent skill-based programs (including 
parenting skills training, parent education, and individual and 
family counseling), and kinship navigator programs.
    2.1.2 Book/Manual/Writings Available. HHS intends to limit 
eligibility to programs or services that have a book, manual, or 
other available documentation that specifies the components of the 
practice protocol and describes how to administer the practice.
    2.2 Program or Service Prioritization Criteria. Timing and 
resources may not allow for a detailed review of all programs and 
services determined to be eligible by the criteria detailed in 
section 2.1 Program or Service Eligibility Criteria. Programs or 
services may be prioritized for review on the basis of the following 
criteria:
    2.2.1 Types of Programs and Services. As noted in 2.1.1. Types 
of Programs and Services, HHS intends to limit eligibility to mental 
health and substance abuse prevention and treatment services, in-
home parent skill-based programs (including parenting skills 
training, parent education, and individual and family counseling), 
or kinship navigator programs. This Notice requests comment on the 
scope of programs and services and topic areas of interest within 
the aforementioned categories that should be prioritized for 
inclusion.
    2.2.2 Target Population of Interest. HHS intends to prioritize 
programs or services for review that have been developed or used to 
target children and families involved in the child welfare system or 
populations similar to those involved in the child welfare system. 
This Notice requests comment on populations that may be considered 
``similar'' to those involved in the child welfare system.
    2.2.3 Target Outcomes. HHS intends to prioritize programs or 
services for review that aim to impact target outcomes. Target 
outcomes should be defined in accordance with FFPSA statutory 
language [section 471(e)(4)(C)] and include those outcomes that ``. 
. . prevent child abuse and neglect, and reduce the likelihood of 
foster care placement by supporting birth families and kinship 
families and improving targeted supports for pregnant and parenting 
youth and their children.'' These may include, but are not limited 
to, ``. . . important child and parent outcomes, such as mental 
health, substance abuse, and child safety and well-being.'' This 
Notice requests comment on which types of mental health, substance 
abuse, and child and family outcomes should be considered as `target 
outcomes' and requests research evidence to support recommendations 
of `target outcomes'. HHS does not intend to include access to 
service, satisfaction with programs and services, and referral to 
programs and services as `target outcomes'.
    2.2.4 Number of Impact Studies. HHS intends to prioritize 
programs or services with at least two studies with non-overlapping 
analytic samples and distinct implementations examining 
effectiveness/impact.
    2.2.5 In Use/Active. HHS intends to prioritize programs or 
services currently in use in the U.S. Programs or services that are 
no longer in operation or have no information available about active 
implementation will not be prioritized.
    2.2.6 Implementation and Fidelity Support. HHS intends to 
prioritize programs or services that have implementation training 
and staff support and/or fidelity monitoring tools and resources 
available to implementers in the United States.
    2.2.7 Trauma-Informed. HHS may also prioritize services and 
programs that have been implemented using a trauma-informed 
approach. FFPSA statutory language [section 471(e)(4)(B)] states, 
``The services or programs to be provided to or on behalf of a child 
are provided under an organizational structure and treatment 
framework that involves understanding, recognizing, and responding 
to the effects of all types of trauma and in accordance with 
recognized principles of a trauma-informed approach and trauma-
specific interventions to address trauma's consequences and 
facilitate healing.'' This Notice requests comment on the 
feasibility of prioritizing programs and services based on past 
implementation in accordance with trauma-informed principles.
    2.2.8 Delivery Setting for In-Home Parent Skill-Based Programs 
and Services. HHS intends to prioritize in-home parent skill-based 
programs and services where the primary service delivery strategy 
takes place in the caregivers' place of residence.
    2.3 Study Eligibility Criteria. HHS intends to engage in a broad 
literature search to identify studies examining prioritized programs 
and services. This search may include databases, websites, existing 
literature reviews, and meta-analyses. HHS intends to screen studies 
for eligibility using the following criteria:
    2.3.1 Impact Study. HHS intends to limit eligibility to studies 
included in government reports and peer-reviewed journal articles 
that assess effectiveness (i.e., impact) using quantitative methods.
    2.3.2 Target Outcomes. HHS intends to limit eligibility to 
studies that examine the impact of the service or program on at 
least one `target outcome', as described in section 2.2.3. Target 
Outcomes. As noted above, this Notice requests comment on specific 
outcomes in accordance with FFPSA statutory language that should be 
considered `target outcomes' and requests research evidence to 
support recommendations of `target outcomes'.
    2.3.3 Conducted in the U.S., U.K., Canada, New Zealand, or 
Australia. HHS intends to limit eligibility to studies conducted 
with samples in the U.S., U.K., Canada, New Zealand, or Australia to 
ensure that the evidence base reflects the populations where 
programs and services will be implemented.
    2.3.4 Study Published in English. HHS intends to limit 
eligibility to studies published in English.
    2.3.5 Published or Prepared in or after 1990. HHS intends to 
limit eligibility to studies published or prepared in or after 1990.
    2.3.6 Usual Care or Practice Setting. HHS intends to limit 
eligibility to studies carried out in a usual care or practice 
setting in accordance with FFPSA [section 471(e)(4)(C)]. This Notice 
requests comment on the definition of usual care or practice 
settings.
    2.4 Study Prioritization Criteria. Timing and resources may not 
allow for a detailed rating of all studies determined to be eligible 
by the criteria identified in section 2.3 Study Eligibility 
Criteria. HHS intends to conduct a high-level scan of eligible 
studies to determine which should be prioritized for rating. This 
Notice requests comment on criteria that can be used to prioritize 
eligible studies for rating.
    2.4.1 Implementation Period: FFPSA [section 471(e)(1)(A) and 
(B)] states that the Secretary may make a payment to a State for 
providing services or programs ``for not more than a 12-month 
period''. This Notice requests comment on whether studies with 
program or service implementation periods of longer than 12 months 
should be considered for review and if so, whether any other 
implementation period cutoff should be included as a study 
prioritization criterion.
    2.4.2 Sample of Interest. HHS intends to prioritize studies that 
include samples of children and families involved in the child 
welfare system or populations similar to those involved in the child 
welfare system. This Notice requests comment on populations that may 
be considered ``similar'' to those involved in the child welfare 
system.
    2.5 Study Rating Criteria. HHS intends to rate studies on the 
following criteria:
    2.5.1 Favorable Effects. HHS intends to rate studies based on 
whether they demonstrate at least one meaningful favorable effect 
(i.e., positive significant effect) on a `target outcome' as 
specified in section 2.3.2 Target Outcomes. A meaningful effect will 
be defined using conventional standards of statistical significance 
(i.e., two-tailed hypothesis test and a specified alpha level of p < 
.05). This Notice requests comment on whether and how ratings should 
consider the number or magnitude of favorable effects.
    2.5.2 Unfavorable Effects. HHS intends to rate studies based on 
the number of unfavorable effects (i.e., negative significant 
effects) on either `target' or non-target outcomes as specified in 
section 2.3.2 Target Outcomes. Effects will be defined using 
conventional standards of statistical significance (i.e., two-tailed 
hypothesis test and a specified alpha level of p < .05). This

[[Page 29124]]

Notice requests comment on whether and how studies should also be 
rated on the number of null effects on `target outcomes', and on 
whether and how ratings should consider the number or magnitude of 
unfavorable effects.
    2.5.3 Sustained Favorable Effect. HHS intends for studies with 
at least one favorable effect on a `target outcome', as determined 
by the criteria in 2.5.1 Favorable Effects, to be rated on whether 
or not they demonstrate a sustained favorable effect. As noted in 
section 471(e)(4)(C), a `supported practice' must have at least one 
study that demonstrates ``a sustained effect (when compared to a 
control group) for at least 6 months beyond the end of treatment'' 
and a `well-supported practice' must have at least one study that 
demonstrates ``a sustained effect (when compared to a control group) 
for at least 1 year beyond the end of treatment.'' HHS intends to 
classify studies as not demonstrating a sustained favorable effect 
(i.e., effects are demonstrated for less than 6 months), 
demonstrating a sustained favorable effect of 6 months or more (but 
less than 12 months), or demonstrating a sustained favorable effect 
of 12 months or more.
    2.5.4 Rigorous Study Design. HHS intends to rate studies as 
either high, moderate, or low on the rigor and appropriateness of 
their study design. Study designs that receive the highest rating 
will be either Randomized Controlled Trials (RCTs) or rigorous 
quasi-experimental designs. HHS defines randomized controlled trials 
as a study design in which sample members are assigned to the 
program or service and comparison groups by chance. Randomized 
control designs are often considered the ``gold standard'' of 
research design because personal characteristics (before the program 
or service begins) do not affect whether someone is assigned to the 
program or service or control group. HHS defines a quasi-
experimental design as a study design in which sample members are 
selected for the program or service and comparison groups in a 
nonrandom way. Similar to criteria considered in other federal 
evidence clearinghouses, rigorous study designs will be those that 
are appropriately powered, include an appropriate control group, 
maintain original assignment to study arms, and are appropriate to 
combat threats to internal validity. This Notice requests comment on 
threats to internal validity that should be considered. This Notice 
requests comment on appropriate thresholds for evaluating and 
assigning a rating to a study design.
    2.5.5 Rigorous Study Analysis. HHS intends to rate studies as 
either high, moderate, or low on the rigor and appropriateness of 
their analysis. Study analyses that receive the highest rating may 
be those that tested and established baseline equivalence, 
appropriately accounted for overall and differential sample 
attrition, appropriately accounted for multiple comparisons, and 
when necessary accounted for clustering. This Notice requests 
comment on appropriate thresholds for evaluating and assigning a 
rating to a study analysis.
    2.5.6 Reliability, Validity, and Systematic Administration of 
Outcome Measures. HHS intends to rate studies as either high, 
moderate, or low on the extent to which `target outcome' measures 
are reliable (i.e., the extent to which a measure produces the same 
results when used repeatedly), valid (i.e., the extent to which a 
measure captures what it is intended to measure), and were 
administered consistently and accurately across all those receiving 
the practice in accordance with FFPSA statutory language [section 
471(e)(4)(C)] or receiving the appropriate comparison practice. This 
Notice requests comment on appropriate thresholds for evaluating and 
assigning a rating to the reliability, validity, and administration 
of `target outcome' measures.
    2.6 Program or Service Rating Criteria. HHS intends for programs 
or services to be rated as promising, supported, or well-supported 
practices if they meet the below criteria that collectively assess 
the strength of evidence for a practice and build from the 
individual study criteria described in section 2.5 Study Rating 
Criteria. These criteria were developed in accordance with FFPSA 
statutory language [section 471(e)(4)(C)].
    2.6.1 Promising Practice: HHS intends to designate a program or 
service as a `promising practice' if the program or service has at 
least one study that demonstrates a favorable effect on a target 
outcome as described by criterion 2.5.1 Favorable Effects and 
achieves, at a minimum, moderate ratings on criteria 2.5.4 through 
2.5.6.
    2.6.2 Supported Practice: HHS intends to designate a program or 
service as a `supported practice' if the program or service has at 
least one study that demonstrates a favorable effect on a target 
outcome as described by 2.5.1 Favorable Effects, demonstrates a 
sustained favorable effect on a target outcome of at least 6 months 
beyond the end of treatment as described in Section 2.5.3 Sustained 
Favorable Effect, and achieves the high rating on criteria 2.5.4 
through 2.5.6.
    2.6.3 Well-Supported Practice: HHS intends to designate a 
program or service as a `well-supported practice' if the practice 
has at least two studies with non-overlapping analytic samples and 
distinct implementations that demonstrate favorable effects as 
described by 2.5.1 Favorable Effects, demonstrate sustained 
favorable effects of at least 12 months beyond the end of treatment 
as described in Section 2.5.3 Sustained Favorable Effect, and 
achieve the high rating on criteria 2.5.4 through 2.5.6.

    HHS does not intend to rate a program or service as a `promising', 
`supported', or `well-supported practice' if there is an empirical 
basis, as evidenced by multiple unfavorable effects on target or non-
target outcomes across reviewed studies, as described in 2.5.2 
Unfavorable Effects, that suggest the overall weight of evidence does 
not support the benefits of the program or service. This Notice 
requests comment on approaches for determining that promising, 
supported, and well-supported practices do not constitute a risk of 
harm. As described in FFPSA [section 471(e)(4)(C)], ``There is no 
empirical basis suggesting that, compared to its likely benefits, the 
practice constitutes a risk of harm to those receiving it'', ``If 
multiple outcome studies are conducted, the overall weight of evidences 
supports the benefits of the practice'', and ``There is no case data 
suggesting a risk of harm that was probably caused by the treatment and 
that was severe or frequent''.

3.0 Recommendations of Potential Candidate Programs and Services for 
Review

    This Notice requests comment on potential candidate programs and 
services to consider for the systematic evidence review. Comments 
should identify how recommended programs and services meet the criteria 
described in section 2.1 Program or Service Eligibility Criteria. These 
criteria include: Types of Programs and Services and Book/Manual/
Writings Available. Comments should also identify how recommended 
programs and services meet the criteria described in section 2.2 
Program or Service Prioritization Criteria. These criteria include: 
Types of Programs and Services, Target Population of Interest, Target 
Outcomes, Number of Impact Studies, In Use/Active, Implementation and 
Fidelity Support, Trauma-Informed, and Delivery Setting for In-Home 
Parent Skill-Based Programs and Services. In order to leverage new 
insights from the field, HHS may put forth additional future Notices 
requesting recommendations of potential candidate programs and services 
for review.

4.0 Submission of Comments

    Comments may be submitted until July 22, 2018 by email to 
[email protected].

Naomi Goldstein,
Deputy Assistant Secretary for Planning, Research, and Evaluation.
[FR Doc. 2018-13420 Filed 6-21-18; 8:45 am]
 BILLING CODE 4184-25-P