Updated Evaluation Policy; Cooperative Research or Demonstration Projects, 62175-62177 [2021-24493]

Agencies

• DEPARTMENT OF HEALTH AND HUMAN SERVICES
• Administration for Children and Families

[Federal Register Volume 86, Number 214 (Tuesday, November 9, 2021)]
[Notices]
[Pages 62175-62177]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2021-24493]


-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Administration for Children and Families


Updated Evaluation Policy; Cooperative Research or Demonstration 
Projects

AGENCY: Administration for Children and Families, HHS.

SUMMARY: The Administration for Children and Families is announcing 
updates to its evaluation policy for research or demonstration 
projects.

SUPPLEMENTARY INFORMATION: This evaluation policy builds on the 
Administration for Children and Families' (ACF) strong history of 
evaluation by outlining key principles to govern our planning, conduct, 
and use of evaluation. This policy reconfirms our commitment to 
conducting rigorous, relevant evaluations and to using evidence from 
evaluations to inform policy and practice. ACF seeks to promote rigor, 
relevance, transparency, independence, and ethics in the conduct of 
evaluations. This policy addresses each of these principles.
    The mission of ACF is to foster health and well-being by providing 
federal leadership, partnership, and resources for the compassionate 
and effective delivery of human services. Our vision is children, 
youth, families, individuals and communities who are resilient, safe, 
healthy, and economically secure. The importance of these goals demands 
that we continually innovate and improve, and that we evaluate our 
activities and those of our partners. Through evaluation, ACF and our 
partners can learn systematically so that we can make our services as 
effective, efficient, and equitable as possible.
    Evaluation produces one type of evidence. A learning organization 
with a culture of continuous improvement requires many types of 
evidence, including not only evaluation but also descriptive research 
studies, performance measures, financial and cost data, survey 
statistics, program administrative data, and feedback from service 
providers, participants, and other stakeholders. Further, continuous 
improvement requires systematic approaches to using information, such 
as regular data-driven reviews of performance and progress. Although 
this policy focuses on evaluation, the principles and many of the 
specifics

[[Page 62176]]

apply to the development and use of other types of evidence as well.
    This policy applies to all ACF-sponsored evaluations. While much of 
ACF's evaluation activity is overseen by the Office of Planning, 
Research, and Evaluation (OPRE), ACF program offices also sponsor 
evaluations through dedicated contracts or as part of their grant-
making. In order to promote quality, coordination and usefulness in 
ACF's evaluation activities, ACF program offices will consult with OPRE 
in developing evaluation activities. Program offices will discuss 
evaluation projects with OPRE in early stages to clarify evaluation 
questions and methodological options for addressing them, and as 
activities progress OPRE will review designs, plans, and reports. 
Program offices may also ask OPRE to design and oversee evaluation 
projects on their behalf or in collaboration with program office staff.
    Rigor: ACF is committed to using the most rigorous methods that are 
appropriate to both the evaluation questions and the populations, 
circumstances, and settings that are the focus of study; and that are 
feasible within budget and other constraints. Rigor is not restricted 
to impact evaluations, but is also necessary in implementation or 
process evaluations, descriptive studies, outcome evaluations, and 
formative evaluations; and in both qualitative and quantitative 
approaches. Rigor requires ensuring that inferences about cause and 
effect are well founded (internal validity); requires clarity about the 
populations, settings, or circumstances to which results can be 
generalized (external validity); and requires the use of measures that 
accurately capture the intended information (measurement reliability 
and validity).
    In assessing the effects of programs or services, ACF evaluations 
will use methods that isolate to the greatest extent possible the 
impacts of the programs or services from other influences such as 
trends over time, geographic variation, or pre-existing differences 
between participants and non-participants. For such causal questions, 
experimental approaches are preferred. When experimental approaches are 
not feasible, high-quality quasi-experiments offer an alternative. ACF 
will develop and use methods that are appropriate for understanding 
diverse populations, taking into account historical, contextual, and 
cultural factors. Where possible, evaluations will design data 
collections to allow disaggregation of data and analyses of sub-groups 
to support understanding of equity.
    ACF will recruit and maintain an evaluation workforce with the 
knowledge, training, and experience appropriate for planning and 
overseeing a rigorous evaluation portfolio. To accomplish this, ACF 
will recruit staff with advanced degrees and experience in a range of 
relevant disciplines such as program evaluation, policy analysis, 
economics, sociology, child development, etc. ACF will recruit staff 
with a range of backgrounds, lived experiences, and perspectives and 
with expertise in approaches appropriate for studying diverse 
populations. ACF will provide professional development opportunities so 
that staff can keep their skills current.
    ACF will ensure that contractors and grant recipients conducting 
evaluations have appropriate expertise through emphasizing the capacity 
for rigor in requests for proposal and funding opportunity 
announcements. This emphasis entails specifying expectations in 
criteria for the selection of grant recipients and contractors, and 
engaging reviewers with evaluation expertise. It also requires 
allocating sufficient resources for evaluation activities. ACF will 
generally require evaluation contractors to consult with external 
advisors who are leaders in relevant fields and who represent diverse 
backgrounds, lived experiences, and perspectives through the formation 
of technical work groups or other means; and to meaningfully engage 
stakeholders from programs and communities being studied throughout the 
evaluation lifecycle.
    Relevance: Evaluation priorities should take into account 
legislative requirements and Congressional interests and should reflect 
the interests and needs of ACF, HHS, and Administration leadership; ACF 
program office staff and leadership; ACF partners such as states, 
territories, tribes, and local grant recipients; service providers; the 
populations served; researchers; and other stakeholders. Stakeholders 
should have the opportunity to influence evaluation priorities to meet 
their interests and needs. Evaluations should be designed to examine 
questions relevant to the diverse populations that ACF programs serve, 
such as Black, Latino, and Indigenous and Native American persons, 
Asian Americans and Pacific Islanders and other persons of color; 
members of religious minorities; lesbian, gay, bisexual, transgender, 
and queer (LGBTQ+) persons; persons with disabilities; persons who live 
in rural areas; and persons otherwise adversely affected by persistent 
poverty or inequality. ACF will encourage diversity among those 
carrying out the work, through building awareness of opportunities and 
building evaluation capacity among under-represented groups. ACF will 
use inclusive and participatory practices in each phase of evaluation 
planning, execution, and dissemination, as appropriate and feasible.
    There must be strong partnerships among evaluation staff, program 
staff, policy-makers and service providers. Further, for new 
initiatives and demonstrations in particular, evaluations will be more 
feasible and useful when planned in concert with the planning of the 
initiative or demonstration, rather than as an afterthought. Given 
federal requirements related to procurement and information collection, 
it can take many months to award a grant or contract and begin 
collecting data. Thus, it is critical that planning for research and 
evaluation be integrated with planning for new initiatives.
    It is important for evaluators to disseminate findings in ways that 
are accessible and useful to policy-makers, service providers, the 
communities that ACF serves, and other stakeholders. OPRE and program 
offices will work in partnership to disseminate information about our 
research and evaluation activities and findings in a manner that is 
clear, accessible, and useful to our diverse range of audiences; this 
includes using plain language, using inclusive language, adhering to 
principles of clear communication, and developing products accessible 
to people with disabilities. ACF will require contractors to 
meaningfully engage stakeholders from the programs and communities 
involved in studies to improve clarity of presentations, accuracy of 
interpretations, and effectiveness of dissemination activities.
    It is ACF's policy to integrate both use of existing evidence and 
opportunities for further learning into all of our activities. Where an 
evidence base is lacking, we will build evidence through strong 
evaluations. Where evidence exists, we will use it. Discretionary 
funding opportunity announcements will require that successful 
applicants cooperate with any federal evaluations if selected to 
participate. As legally allowed, programs with waiver authorities 
should require rigorous evaluations as a condition of waivers. As 
appropriate, ACF will encourage, incentivize or require grant 
recipients to use existing evidence of effective strategies in 
designing or selecting service approaches. The emphasis on evidence is 
meant to support, not

[[Page 62177]]

inhibit, innovation, improvement, equity, and learning.
    Transparency: ACF will make information about planned and ongoing 
evaluations easily accessible, typically through posting on the web 
information about the contractor or grant recipient conducting the work 
and descriptions of the evaluation questions, methods to be used, and 
expected timeline for reporting results. ACF will present information 
about study designs, implementation, and findings at professional 
conferences.
    Study plans will be published in advance. ACF will release 
evaluation results regardless of the findings. Evaluation reports will 
describe the methods used, including strengths and weaknesses, and 
discuss the generalizability of the findings. Evaluation reports will 
present comprehensive results, including favorable, unfavorable, and 
null findings. ACF will release evaluation results timely--usually 
within two months of a report's completion.
    As appropriate and feasible, ACF will archive evaluation data for 
secondary use by interested researchers, typically through building 
requirements into contracts to prepare data sets for secondary use.
    Independence: Independence and objectivity are core principles of 
evaluation. Agency and program leadership, program staff, service 
providers, populations and communities studied, and others should 
participate actively in setting evaluation priorities, identifying 
evaluation questions, and assessing the implications of findings. 
However, it is important to insulate evaluation functions from undue 
influence and from both the appearance and the reality of bias. To 
promote objectivity, ACF protects independence in the design, 
execution, analysis, and reporting of evaluations. To this end:
     ACF will conduct evaluations through the competitive award 
of grants and contracts to external experts who are free from conflicts 
of interest.
     The Deputy Assistant Secretary for Planning, Research, and 
Evaluation reports directly to the Assistant Secretary for Children and 
Families; serves as ACF's Chief Evaluation Officer; has authority to 
approve the design of evaluation projects and analysis plans; and has 
authority to approve, release and disseminate evaluation reports.
    Ethics: ACF-sponsored evaluations will be conducted in an ethical 
and equitable manner and safeguard the dignity, rights, safety and 
privacy of participants. ACF-sponsored evaluations will comply with 
both the spirit and the letter of relevant requirements such as 
regulations governing research involving human subjects. ACF will 
expect contractors to meaningfully engage stakeholders from the 
programs and communities involved in studies to ensure programmatic, 
cultural, linguistic and historical nuances are accurately and 
respectfully addressed from the initial study design, through 
execution, analyses and reporting.
    Authority: 42 U.S.C. 1310.

JooYeun Chang,
Acting Assistant Secretary.
[FR Doc. 2021-24493 Filed 11-8-21; 8:45 am]
BILLING CODE 4184-79-P