Updated Defintions, Rules, and Procedures Related to Model Versions: Home Visiting Evidence of Effectiveness (HomVEE) Review, 47384-47388 [2020-16992]

Agencies

• DEPARTMENT OF HEALTH AND HUMAN SERVICES
• Administration for Children and Families

[Federal Register Volume 85, Number 151 (Wednesday, August 5, 2020)]
[Notices]
[Pages 47384-47388]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2020-16992]


-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Administration for Children and Families


Updated Defintions, Rules, and Procedures Related to Model 
Versions: Home Visiting Evidence of Effectiveness (HomVEE) Review

AGENCY: Administration for Children and Families, U.S. Department of 
Health and Human Services.

ACTION: Request for public comment.

-----------------------------------------------------------------------

SUMMARY: The Administration for Children and Families (ACF), within the 
U.S. Department of Health and Human Services (HHS), oversees the Home 
Visiting Evidence of Effectiveness (HomVEE) review, which is proposing 
to revise the procedures and standards that guide its work. The revised 
procedures and standards will be presented in two separate Federal 
Register notices. The current Federal Register notice seeks comments on 
proposed updated definitions, rules, and procedures related to handling 
home visiting model versions (commonly referred to in the home visiting 
research literature as adaptations) in the HomVEE review. Another 
Federal Register notice summarizes proposed changes and clarifications 
to HomVEE's procedures and standards for rating the quality of impact 
studies and determining which home visiting models meet HHS criteria 
for evidence of effectiveness. Readers are referred to the full text of 
the HomVEE Draft Version 2 Handbook on the HomVEE website (https://homvee.acf.hhs.gov/) for more details on all proposed changes.

DATES: Send comments on or before September 1, 2020.

ADDRESSES: Submit questions, comments, and supplementary documents to 
[email protected] with ``HomVEE model versions FRN comment'' in the 
subject line.

SUPPLEMENTARY INFORMATION: 
    Invitation to Comment: HHS invites comments regarding this notice. 
To ensure that your comments are clearly stated, please identify the 
section of this notice that your comments address.

1.0 Background

    To help policymakers, program administrators, model developers, 
researchers, and the public identify rigorous research and understand 
which early childhood home visiting models are effective, ACF's Office 
of Planning, Research, and Evaluation within HHS oversees the HomVEE 
review. HomVEE's mission is to conduct a thorough and transparent 
review of the research literature on home visiting for families with 
pregnant women and children from birth to kindergarten entry. The 
review team identifies well-designed research within that pool and 
extracts and summarizes the findings from that research.
    One critical use of HomVEE's results is to determine which home 
visiting models meet the HHS criteria for an ``evidence-based early 
childhood home visiting service delivery model'' (see Exhibit II.11 in 
the HomVEE Draft Version 2 Handbook), a key requirement of eligibility 
for implementation with the Maternal, Infant, and Early Childhood Home 
Visiting (MIECHV) Program funding. The MIECHV Program is administered 
by the Health Resources and Services Administration (HRSA) in 
partnership with ACF. Created in 2010, the MIECHV Program provides 
funding to states, territories, and tribal entities to implement home 
visiting models. MIECHV awardees have the flexibility to tailor the 
program to serve the specific needs of their communities. Through a 
needs assessment, awardees identify at-risk communities and select home 
visiting service delivery models that best meet state and/or local 
needs. As per MIECHV's authorizing statute, state and territory 
awardees must spend the majority of their MIECHV Program grants to 
implement evidence-based home visiting models, with up to 25 percent of 
funding available to implement promising approaches that will undergo 
rigorous evaluation.
    For the first time since its inception in 2009, HomVEE is proposing 
substantial revisions to several procedures and standards that guide 
the systematic review. The proposed revisions include (1) clarifying 
and updating standards and procedures (presented in a separate Federal 
Register notice) for rating the quality of impact studies that are used 
to determine which home visiting models meet HHS criteria for an 
``evidence-based early childhood home visiting service delivery model'' 
and (2) clarifying definitions, rules, and procedures for handling 
model versions (commonly referred to in the home visiting research 
literature as adaptations) in the HomVEE review process. The current 
Federal Register notice focuses on the latter set of revisions.

[[Page 47385]]

    HomVEE routinely encounters research in which an individual home 
visiting model has been changed or altered for various reasons, 
including to fit a new or different context (for example, adding a new 
curriculum targeted to a specific population) or to implement a new 
feature (for example, adding text messaging between visits). In order 
to ensure the transparency and accuracy of the HomVEE review, it is 
important to have clear definitions, rules, and procedures to classify 
research on different versions of home visiting models. Over the course 
of the past 2 years, a HomVEE workgroup composed of ACF and HRSA staff 
with contractor support met to discuss and develop the draft 
definitions, rules, and procedures outlined in this Federal Register 
notice. Their work was informed by a close review of the procedures of 
other federally sponsored systematic evidence reviews.
    Through this Federal Register notice, ACF seeks to provide a 
transparent account of how the review operates and to collect 
stakeholder input on draft definitions, rules, and procedures related 
to model versions. Because these procedural changes may affect critical 
decision making, and to better understand the implications of the 
changes for various stakeholders, ACF seeks public input on the draft 
changes. After a period of public comment, HomVEE will finalize and 
release the final definitions, rules, and procedures related to model 
versions.

2.0 Definitions, Rules, and Procedures for Reviewing Home Visiting 
Models and Model Versions

    First, this Federal Register notice presents HomVEE's definition of 
``early childhood home visiting model.'' Then, it discusses how HomVEE 
proposes to define and distinguish among various types of model 
versions.

2.1 Defining an Early Childhood Home Visiting Model

    In order to identify eligible research, HomVEE intends to apply the 
following definition of an early childhood home visiting model: HomVEE 
defines an early childhood home visiting model as an intervention that 
delivers a specified set of services (through a specified set of 
interactions). A model has a set of fidelity standards that describe 
how the model is to be implemented. These voluntary programs are 
interventions, either designed or adapted and tested for delivery in 
the home, in which trained home visitors meet with expectant parents or 
families with young children on a schedule that is defined or can be 
tailored to meet family needs. During the visits, home visitors aim to 
build strong, positive relationships with families who want and need 
support to improve child and family outcomes. Models reviewed by HomVEE 
must serve pregnant women or families with children from birth to 
kindergarten entry. In a model eligible for review, the primary service 
delivery strategy must be home visiting and there must be research on 
the model that examines its effects in at least one of eight outcome 
domains: Child development and school readiness, child health, family 
economic self-sufficiency, linkages and referrals, maternal health, 
positive parenting practices, reductions in child maltreatment, and 
reductions in juvenile delinquency, family violence, and crime.\1\ This 
definition of a home visiting model is for the purpose of sorting and 
classifying the research literature as part of the HomVEE review. It is 
not for the purposes of determining eligibility for MIECHV funding.
---------------------------------------------------------------------------

    \1\ These domains are inclusive of the benchmark domains and 
individual outcomes listed in the statute that authorized the 
Maternal, Infant, and Early Childhood Home Visiting (MIECHV) Program 
(Social Security Act, Section 511 [42 U.S.C. 711]).
---------------------------------------------------------------------------

2.1.1 Classifying Research on Early Childhood Home Visiting Models

2.1.1.1 Defining Model Versions

    To date, the HomVEE team has relied on model developers and 
manuscript authors to identify the specific version of the model they 
are examining in a research study. However, manuscripts vary in the 
extent to which their authors clearly identify the model version being 
tested. Moving forward, HomVEE intends to implement definitions, rules, 
and procedures to systematically differentiate versions of a model. 
This will help ensure that HomVEE treats all models equally and 
classifies research on model effectiveness accurately. These 
definitions, rules, and procedures are for the purpose of sorting and 
classifying the research literature as part of the HomVEE review. It is 
not for the purposes of determining eligibility for MIECHV funding.
    HomVEE proposes to apply the following definitions to differentiate 
models:
     Base model: The model as designed by the developer. (This 
includes implementations of the base model by the developer and 
replications of the base model by others.)
     Model version: Distinguished by a substantial change to a 
core feature of the base model (substantial change and core feature are 
defined below).
     Model iteration: Distinguished by a substantial change to 
a core feature of the base model that the model developer intends to 
represent a permanent update to the base model. The term model 
iteration is intended to capture changes that are part of the natural 
evolution of a model over time.
     Family of models: A base model and its iterations and 
versions.
    As indicated by the definitions, identifying substantial changes to 
core features or the addition or removal of core features is critical 
for differentiating base models and model versions. For the purposes of 
HomVEE's review, HomVEE conceptualizes home visiting models as being 
composed of a set of core features that include both content features 
(such as the topics covered in a curriculum) and implementation 
features (such as home visitor training and staff qualifications). Core 
features are integral to the model and will be defined as the 
following:
 Visit format (in-person, virtual, mixed)
 Home visit frequency
 Home visit length
 Number of home visits
 Program duration
 Required staffing structure
 Home visitor education and experience
 Home visitor supervision requirements
 Home visitor caseload
 Requirements for program certification
 Training requirements for home visitors (initial and ongoing)
 Trainer qualifications
 Curriculum
 Other features in the model's theory of change
    HomVEE further proposes to differentiate core features by whether 
the feature has a fixed requirement or flexible requirement. Core 
features with fixed requirements have specific guidelines that guide 
the implementation of the core feature, such as requiring a specific 
curriculum or delivering home visits with a specific frequency. Fixed 
requirements may also be a specific range, such as requiring home 
visits of 60-90 minutes, or minimum thresholds, such as a staff 
qualification of at least a B.A. degree. Flexible requirements do not 
have specific guidelines about implementation, such as no requirement 
for implementers to use a specific curriculum.

2.1.1.2 Identifying Core Features

    For model families that are candidates for review in a given year, 
HomVEE proposes to generate a list of core

[[Page 47386]]

features for each model and determine whether each core feature of each 
model has fixed or flexible requirements. HomVEE will first generate 
the list based on information available through manuscripts, the 
developer website, and/or the existing HomVEE implementation profile. 
HomVEE will then send the list to model developers to invite them to 
verify HomVEE's understanding of the model's core features.

2.1.1.3 Using Information on Core Features To Identify Model Versions

    Using a set of predetermined decision rules and the list of core 
features verified by model developers, HomVEE intends to identify 
substantial changes to core features to distinguish research on model 
versions and iterations from research on the base model. HomVEE 
welcomes feedback from the field on these decision rules. The rules 
draw on general guidance from the implementation science literature and 
current practices of other evidence reviews.
    Specifically, HomVEE will consider a change substantial if a fixed 
requirement related to a core feature is added or removed (see Table 
1). For example, if researchers or model implementers remove a 
curriculum module or add text message check-ins with families in 
between in-person visits, those changes would be considered 
substantial. In addition, HomVEE will consider as substantial those 
changes in which a fixed requirement becomes flexible or a flexible 
requirement becomes fixed. To illustrate, a base model may allow 
implementers to select their own curriculum that best fits their 
context. Researchers and model implementers may decide to modify that 
feature and require a specific curriculum. This would be considered a 
substantial change because a flexible requirement (no required 
curriculum) is now fixed (that is, a specific curriculum is now 
required).
    In addition, for core features that involve a specific frequency or 
number (such as number of home visits or home visitor caseloads), 
changes that modify the frequency by more than 50 percent will, in 
general, be considered substantial.\2\ To illustrate, a base model may 
require home visitors to offer families a minimum of 40 visits. 
Researchers and model implementers may change the home visit frequency 
requirements of the base model to offer families a minimum of 10 visits 
to better serve a population where 40 visits are not feasible. Given 
HomVEE's proposed decision rules, this would be considered a 
substantial change, because the visit frequency was changed by more 
than 50 percent.
---------------------------------------------------------------------------

    \2\ If a model under study lowers a required minimum threshold 
for a core feature, such as requiring at least 10 home visits 
instead of 20, this is a substantial change to the core feature and, 
therefore, is a model version or iteration. If a model under study 
exceeds a minimum requirement, this is not a substantial change to 
the core feature and therefore is a model replication.

                         Table 1--Proposed Decision Rules for Identifying Model Versions
----------------------------------------------------------------------------------------------------------------
                                                            Substantial changes to a
                                Non-substantial change to      core feature (model       Changes that requires
         Core feature            a core feature include:      version or iteration)    consultation with experts
                                                                    include:                    include:
----------------------------------------------------------------------------------------------------------------
Curriculum...................  None                        --Addition or subtraction   Any other changes.
                                                            of curriculum modules
                                                           --Flexible requirement
                                                            becomes fixed
                                                           --Fixed requirement
                                                            becomes flexible
                                                           --Change in required
                                                            curriculum (from one to
                                                            another)
Home visit frequency.........  --Intended frequency        --Intended frequency        --Intended frequency
                                changes by less than 25     changes by more than 50     changes by 25-50
                                percent                     percent                     percent.
                                                           --Flexible requirement
                                                            becomes fixed (such as
                                                            from no required
                                                            frequency to required
                                                            monthly visits)
                                                           --Fixed requirement
                                                            becomes flexible
Home visitor caseloads.......  --Intended caseload         --Intended caseload size    --Intended caseload size
                                changes by less than 25     changes by more than 50     changes by 25-50
                                percent                     percent                     percent.
                                                           --Flexible requirement
                                                            becomes fixed (such as
                                                            from no caseload guidance
                                                            to requiring one visitor
                                                            per 10 families)
                                                           --Fixed requirement
                                                            becomes flexible
Home visitor education and     None                        --Required degree type      None.
 experience.                                                changes (such as from
                                                            R.N. to M.S.W.)
                                                           --Flexible requirement
                                                            becomes fixed (such as
                                                            from no education
                                                            requirement to requiring
                                                            a Bachelor's degree)
                                                           --Fixed requirement
                                                            becomes flexible
Home visitor supervision.....  --Less than 25 percent      --More than 50 percent      --25-50 percent change in
                                change in intended          change in intended          intended supervision
                                supervision frequency       supervision frequency       frequency.

[[Page 47387]]

 
                                                           --Flexible requirement      --Mode of supervision
                                                            becomes fixed such as       (one-on-one, reflective
                                                            from no requirement for     supervision) added or
                                                            the frequency of            dropped.
                                                            supervision to requiring
                                                            monthly supervision
                                                            meetings)
                                                           --Fixed requirement
                                                            becomes flexible
Home visitor pre-service       --Less than 25 percent      --25-50 percent change in   --25-50 percent change in
 training.                      change in required          required training hours     required training hours.
                                training hours
                               --Change in training mode   --Addition or subtraction
                                (in person to virtual)      of training modules
                                that does not change
                                training content
                                                           --Flexible requirement
                                                            becomes fixed (such as
                                                            from no training
                                                            requirement to specified
                                                            training requirement)
                                                           --Fixed requirement
                                                            becomes flexible (such as
                                                            from specified training
                                                            requirement to no
                                                            training requirement)
Home visitor in-service        --Less than 25 percent      --25-50 percent change in   --25-50 percent change in
 training.                      change in required          required training hours     required training hours.
                                training hours
                               --Change in training mode   --Addition or subtraction
                                (in person to virtual)      of training modules
                                that does not change
                                training content
                                                           --Flexible requirement
                                                            becomes fixed (such as
                                                            from no training
                                                            requirement to specified
                                                            training requirement)
                                                           --Fixed requirement
                                                            becomes flexible (such as
                                                            from specified training
                                                            requirement to no
                                                            training requirement)
Program certification........  None                        --Program certification     Changes to certification
                                                            requirement added or        standards or procedures.
                                                            dropped
Number of home visits........  --Intended dosage changes   --Intended dosage changes   --Intended dosage changes
                                by less than 25 percent     by more than 50 percent     25-50 percent.
                                                           --Flexible requirement
                                                            becomes fixed (such as
                                                            from tailored number of
                                                            visits to fixed number of
                                                            visits)
                                                           --Fixed requirement
                                                            becomes flexible (such as
                                                            from fixed number of
                                                            visits to tailored number
                                                            of visits)
Program duration.............  --Intended duration         --Intended duration         --Intended duration
                                changes by less than 25     changes by more than 50     changes 25-50 percent.
                                percent                     percent
                                                           --Flexible requirement
                                                            becomes fixed (such as
                                                            from duration tailored to
                                                            family need to a fixed
                                                            number of months)
                                                           --Fixed requirement
                                                            becomes flexible (such as
                                                            from a fixed number of
                                                            months to duration
                                                            tailored to family need)
Required staffing structure..  --Less than 25 percent      --More than 50 percent      --25-50 percent change in
                                change in recommended       change in recommended       recommended supervisor
                                supervisor caseload         supervisor caseload         caseload.
                                                           --Flexible requirement      --Addition or elimination
                                                            becomes fixed (such as      of required non-
                                                            from no required            supervisory positions
                                                            structure to specified
                                                            structure)
                                                           --Fixed requirement
                                                            becomes flexible (such as
                                                            from specified structure
                                                            to no required structure)
Trainer qualifications.......  None                        --Addition or subtraction   --Any other changes.
                                                            of certification
                                                            requirement

[[Page 47388]]

 
                                                           --Flexible requirement
                                                            becomes fixed (such as
                                                            from no required
                                                            qualifications to
                                                            specified qualifications)
                                                           --Fixed requirement
                                                            becomes flexible (such as
                                                            from specified
                                                            qualifications to no
                                                            required qualifications)
Visit format (in person,       None                        --Change from one fixed     --Any changes involving
 virtual, mixed).                                           format to another           phone visits.
                               --Less than 25 percent      --More than 50 percent      --25-50 percent change in
                                change in required number   change in number of in-     number of in-person
                                of in-person visits         person visits               visits.
                                                           --Flexible requirement
                                                            becomes fixed (such as
                                                            from no required visit
                                                            format to required in-
                                                            person format)
                                                           --Fixed requirement
                                                            becomes flexible (such as
                                                            from required in-person
                                                            format to no required
                                                            visit format)
Visit length.................  None                        --More than 50 percent      --25-50 percent change in
                                                            change in required visit    required visit length.
                                                            length
                                                           --Flexible requirement
                                                            becomes fixed (such as
                                                            from no required length
                                                            to required length of one
                                                            hour)
                                                           --Fixed requirement
                                                            becomes flexible (such as
                                                            from required length of
                                                            one hour to no required
                                                            length)
Other features of the model's  None                        None                        --Any change.
 theory of change.
----------------------------------------------------------------------------------------------------------------

2.1.2 Prioritizing Early Childhood Home Visiting Models

    HomVEE proposes that early childhood home visiting models 
(including base models, model versions, and model iterations) continue 
to be eligible for review as long as they meet all of HomVEE's other 
criteria for inclusion as described in the HomVEE Draft Version 2 
Handbook. HomVEE will continue to select families of models for review 
by creating a prioritization score for each family of models using a 
combination of manuscript and model characteristics (see the HomVEE 
Draft Version 2 Handbook, Chapter II, Section A, for additional 
details). Also, after a family of models is selected for review, HomVEE 
will continue to review all previously unreviewed research on the base 
model, model iterations, and model versions.

2.2 Applying the HHS Criteria

    HomVEE proposes to review research on a model version and its base 
model separately. Therefore, a model version would need to 
independently meet HHS criteria for ``an evidence-based early childhood 
home visiting service delivery model.'' In contrast, HomVEE proposes 
that research on the base model and research on model iterations be 
reviewed together and contribute to HomVEE's determination of whether 
the base model meets HHS criteria.

2.3 Applying the New Definitions, Rules, and Procedures

    The new definitions, rules, and procedures about model versions 
will apply to all models that HomVEE considers for review, regardless 
of whether the model already meets HHS criteria for an ``evidence-based 
early childhood home visiting service delivery model.'' HomVEE 
anticipates implementing the new definitions, rules, and procedures to 
the HomVEE review beginning with the models prioritized for review in 
2021. The new definitions, rules, and procedures will be implemented 
with other models over time.

3.0 Request for Information

    Through this Federal Register notice, ACF is soliciting information 
from a broad array of stakeholders on the proposed revisions to 
HomVEE's procedures. Federal, state, and local decision makers rely on 
HomVEE to know which home visiting models are effective. New 
definitions, rules, and procedures about model versions may affect 
which models HomVEE determines to meet HHS criteria for ``an evidence-
based early childhood home visiting service delivery model.''
    Responses to this Federal Register notice will inform ACF's ongoing 
discussion about HomVEE's procedures and standards, with the aim of 
publishing a final HomVEE Version 2 Handbook by the end of 2020. This 
Federal Register notice is for information and planning purposes only 
and should not be construed as a solicitation or as an obligation on 
the part of ACF or HHS.

(Authority: Social Security Act Title V Sec.  511 [42 U.S.C. 711], 
as extended by the Bipartisan Budget Act of 2018 (Pub. L. 115-123) 
through fiscal year 2022)

John M. Sweet, Jr.,
ACF/OPRE Certifying Officer.
[FR Doc. 2020-16992 Filed 8-4-20; 8:45 am]
BILLING CODE 4184-74-P