Revised Single-Case Design Procedures and Standards: Home Visiting Evidence of Effectiveness (HomVEE) Review, 15221-15225 [2021-05840]

Download as PDF Federal Register / Vol. 86, No. 53 / Monday, March 22, 2021 / Notices The meeting will be held on Wednesday, June 9, 2021, from 8:30 a.m. to 6:00 p.m. EDT, and Thursday, June 10, 2021, from 8:30 a.m. to 5:00 p.m. EDT. ADDRESSES: The meeting will be held via web conference. FOR FURTHER INFORMATION CONTACT: Arielle Gatlin, Office of the Associate Director for Policy and Strategy; Centers for Disease Control and Prevention, 1600 Clifton Road NE, MS–V–25–5, Atlanta, GA 30329, phone: (404)498– 4512, email: CPSTF@cdc.gov. SUPPLEMENTARY INFORMATION: Meeting Accessibility: The CPSTF meeting will be held virtually via web conference. CDC will send web conference information to registrants upon receipt of their registration. All meeting attendees must register by June 2, 2021 to receive the web conference information for the June meeting. CDC will email web conference information from the CPSTF@cdc.gov mailbox. To register for the meeting, individuals should send an email to CPSTF@cdc.gov and include the following information: name, title, organization name, organization address, phone, and email. Public Comment: Individuals who would like to make public comments during the June meeting must state their desire to do so with their registration and provide their name and organizational affiliation (if any) and the topic to be addressed (if known). The requestor will receive instructions for the public comment process for this virtual meeting after the request is received. A public comment period follows the CPSTF’s discussion of each systematic review and will be limited, up to three minutes per person. Public comments will become part of the meeting summary. Background on the CPSTF: The CPSTF is an independent, nonfederal panel whose members are appointed by the CDC Director. CPSTF members represent a broad range of research, practice, and policy expertise in prevention, wellness, health promotion, and public health. The CPSTF was convened in 1996 by the Department of Health and Human Services (HHS) to identify community preventive programs, services, and policies that increase health, longevity, save lives and dollars, and improve Americans’ quality of life. CDC is mandated to provide ongoing administrative, research, and technical support for the operations of the CPSTF. During its meetings, the CPSTF considers the findings of systematic reviews of jbell on DSKJLSW7X2PROD with NOTICES DATES: VerDate Sep<11>2014 18:46 Mar 19, 2021 Jkt 253001 existing research and practice-based evidence and issues recommendations. CPSTF recommendations are not mandates for compliance or spending. Instead, they provide information about evidence-based options that decision makers and stakeholders can consider when they are determining what best meets the specific needs, preferences, available resources, and constraints of their jurisdictions and constituents. The CPSTF’s recommendations, along with the systematic reviews of the evidence on which they are based, are compiled in the The Community Guide. Matters proposed for discussion: The agenda will consist of deliberation on systematic reviews of literature and is open to the public. Topics will include HIV Prevention; Heart Disease and Stroke Prevention; and Nutrition, Physical Activity, and Obesity. Information regarding the start and end times for each day, and any updates to agenda topics, will be available on the Community Guide website (www.thecommunityguide.org) closer to the date of the meeting. The meeting agenda is subject to change without notice. Dated: March 16, 2021. Sandra Cashman, Executive Secretary, Centers for Disease Control and Prevention. [FR Doc. 2021–05810 Filed 3–19–21; 8:45 am] BILLING CODE 4163–18–P DEPARTMENT OF HEALTH AND HUMAN SERVICES Administration for Children and Families Revised Single-Case Design Procedures and Standards: Home Visiting Evidence of Effectiveness (HomVEE) Review Administration for Children and Families, U.S. Department of Health and Human Services. ACTION: Request for public comment. AGENCY: The Administration for Children and Families (ACF), within the U.S. Department of Health and Human Services (HHS), oversees the Home Visiting Evidence of Effectiveness (HomVEE) review, which is proposing to revise the procedures and standards that guide its work with single-case design (SCD) research. The current Federal Register notice (FRN) seeks comments on proposed changes related to revised procedures and standards for SCD research. Readers are referred to the full version of the HomVEE Version 2 Handbook on the HomVEE website for SUMMARY: PO 00000 Frm 00039 Fmt 4703 Sfmt 4703 15221 more details, particularly Appendix D (https://homvee.acf.hhs.gov/ publications/methods-standards). HomVEE will release an updated Handbook (Version 2.1) after consideration of public comments received in response to this FRN. DATES: Send comments on or before April 19, 2021. ADDRESSES: Submit questions, comments, and supplementary documents to HomVEE@acf.hhs.gov with ‘‘HomVEE SCD procedures and standards FRN comment’’ in the subject line. SUPPLEMENTARY INFORMATION: Invitation to Comment: HHS invites comments regarding this notice. To ensure that your comments are clearly stated, please identify the section of this notice that your comments address. 1.0 Background To help policymakers, program administrators, model developers, researchers, and the public identify rigorous research and understand which early childhood home visiting models are effective, ACF’s Office of Planning, Research, and Evaluation (OPRE) within HHS oversees the HomVEE review, in partnership with the Health Resources and Services Administration (HRSA). HomVEE’s mission is to conduct a thorough and transparent review of early childhood home visiting models that serve pregnant women and children from birth to kindergarten entry. The review identifies well-designed, wellexecuted research, then extracts and summarizes the findings from that research. One critical use of HomVEE’s results is to determine which home visiting models meet the HHS criteria for an ‘‘evidence-based early childhood home visiting service delivery model’’ (see Section 1.1 below), a key requirement of eligibility for implementation of the model with Maternal, Infant, and Early Childhood Home Visiting (MIECHV) Program funding. The MIECHV Program is administered by HRSA in partnership with ACF. Created in 2010, the MIECHV Program provides funding to states, territories, and tribal entities to implement home visiting models. MIECHV awardees have the flexibility to tailor the program to serve the specific needs of their communities. Through a needs assessment, awardees identify at-risk communities and select home visiting service delivery models that best meet state and/or local needs. In accordance with MIECHV’s authorizing statute, state and territory awardees must spend the majority of their MIECHV Program E:\FR\FM\22MRN1.SGM 22MRN1 15222 Federal Register / Vol. 86, No. 53 / Monday, March 22, 2021 / Notices grants to implement evidence-based home visiting models with up to 25 percent of funding available to implement promising approaches that will undergo rigorous evaluation. In December 2020, HomVEE released an updated Version 2 Handbook of Procedures and Evidence Standards (Sama-Miller et al. 2020). The updates incorporated input from methodological experts, other federal evidence reviews, and public comments on two FRNs released on August 5, 2020 (85 FR 47376 and 85 FR 47384). The HomVEE Version 2 Handbook adopted the What Works Clearinghouse (WWC) Version 4.1 Procedures and Standards Handbooks (2020a and 2020b), including revised SCD procedures and standards. This FRN proposes further changes to the HomVEE Version 2 Handbook to clarify how HomVEE would apply the WWC 4.1 procedures and standards that HomVEE already adopted. The proposed changes focus exclusively on SCD research, including: (1) Requirements for considering SCD research toward the HHS criteria for evidence-based models (discussed in Section 2 of the FRN). (2) Flexibilities for applying certain SCD standards (discussed in Section 3 of the FRN). A work group consisting of federal staff and HomVEE contractor staff, including methodological and home visiting experts, met to discuss and develop the proposed changes outlined in this FRN. Through this FRN, ACF seeks to gather stakeholder input on draft changes and to provide a transparent account of how the review operates. The remainder of Section 1 summarizes recent and current HomVEE procedures and standards with respect to SCD research. After a period of public comment (and consultation with selected experts outside HomVEE), HomVEE will release an updated Handbook (Version 2.1). Section 5 of this FRN specifies where in the updated Version 2.1 Handbook HomVEE would insert the proposed changes. jbell on DSKJLSW7X2PROD with NOTICES 1.1 HHS Criteria for an ‘‘EvidenceBased Early Childhood Home Visiting Service Delivery Model’’ As described in Section 2 of this FRN, HomVEE is proposing changes to requirements for how SCD research would be considered toward the HHS criteria. However, HomVEE is not proposing changes to the HHS criteria themselves. To meet HHS criteria for an ‘‘evidence-based early childhood home visiting service delivery model,’’ models VerDate Sep<11>2014 18:46 Mar 19, 2021 Jkt 253001 must meet at least one of the following criteria: (1) At least one high- or moderatequality impact study of the model finds favorable, statistically significant impacts in two or more of the eight outcome domains. (2) At least two high- or moderatequality impact studies of the model using non-overlapping analytic study samples find one or more favorable, statistically significant impacts in the same domain. In both cases, the impacts must either (1) be found in the full sample for the study, or (2) if found for subgroups but not for the full sample, be replicated in the same domain in two or more studies using non-overlapping analytic study samples. Additionally, following the MIECHV authorizing statute, if the model meets the above criteria based on findings from randomized controlled trials only, then two additional requirements apply. First, one or more favorable (statistically significant) impacts must be sustained for at least one year after program enrollment. Second, one or more favorable (statistically significant) impacts must be reported in a peerreviewed journal. Since HomVEE’s inception, research about SCD studies has had to satisfy certain requirements before manuscripts about those studies could be considered toward the HHS criteria. These requirements, known as the ‘‘5–3–20 rule,’’ are as follows: (1) At least five studies examining the intervention either met the WWC’s SCD standards without reservations or met those standards with reservations (corresponding to a ‘‘high’’ or ‘‘moderate’’ rating in HomVEE, respectively). (2) The SCDs were conducted by at least three research teams, with no overlapping authorship at three institutions. (3) The combined number of cases was at least 20. Beyond the 5–3–20 rule, the question of statistical significance is relevant to findings from SCD research because the HHS criteria focus on favorable, statistically significant findings. Authors of SCD manuscripts rarely describe their findings in terms of statistical significance, which HomVEE needs to apply the HHS criteria. However, the WWC Version 4.1 Procedures Handbook, adopted in the HomVEE Version 2 Handbook, introduces the design-comparable effect size (D–CES), from which statistical significance can be determined. PO 00000 Frm 00040 Fmt 4703 Sfmt 4703 2.0 Proposed Changes to How SCD Research Would Be Considered Toward HHS Criteria for an ‘‘Evidence-Based Early Childhood Home Visiting Service Delivery Model’’ The HomVEE Version 2 Handbook adopted the WWC Single-Case Design Procedures and Standards (Version 4.1). Therefore, the latest round of updates to each of the WWC and HomVEE procedures and standards included three key complementary changes. The changes, described in more detail in Subsections 2.1 through 2.3, are as follows: (1) The WWC no longer uses the ‘‘5– 3–20’’ rule, which established thresholds for when SCD research could contribute to evidence ratings. Consequently, HomVEE’s previous requirements for SCD research about an early childhood home visiting model to be considered toward the HHS criteria (see Section 1.1) no longer align with current best practices for systematic reviews. (2) Under HomVEE’s new procedures as defined in the Version 2 Handbook and adopted from the WWC, it is now possible to calculate effect sizes and determine statistical significance for SCD research using the D–CES. Consequently, HomVEE proposes to align the application of the HHS criteria for SCD research to the procedures already in effect for other eligible research designs. (3) Reviewers will query authors for numerical data for calculating the D– CES, a step that had not been relevant under prior versions of procedures for HomVEE (or the WWC). Because calculation of the D–CES requires numerical data from each time point, HomVEE proposes to query authors, as needed, for the information required to calculate the D–CES. 2.1 The 5–3–20 Rule No Longer Applies Additional requirements for considering results from SCD research toward the HHS criteria (the 5–3–20 rule) have been in effect since HomVEE’s inception; those requirements were consistent with the WWC’s then-current approach to SCD research. The 5–3–20 rule provided a threshold for determining when SCD research could contribute toward a decision about whether a model was evidence-based. Specifically, for SCD research to be considered toward the HHS criteria, the model’s SCD research has been required to consist of at least five studies with high or moderate internal validity, to be conducted by three independent research teams, and E:\FR\FM\22MRN1.SGM 22MRN1 Federal Register / Vol. 86, No. 53 / Monday, March 22, 2021 / Notices to include at least 20 cases. To date, no models reviewed by HomVEE have met the requirements of the 5–3–20 rule. The WWC Version 4.1 Procedures Handbook removed the 5–3–20 rule. Correspondingly, HomVEE now proposes to remove the 5–3–20 requirement for SCD research to be considered toward the HHS criteria. jbell on DSKJLSW7X2PROD with NOTICES 2.2 Effect Sizes and Statistical Significance Can Now Be Computed The HHS criteria for evidence-based models require evidence of favorable, statistically significant findings. However, HomVEE’s procedures did not previously specify how HomVEE would apply the HHS criteria to SCD research, for which researchers generally express findings in terms of visual patterns rather than statistical significance. Advances in meta-analysis have made it possible to calculate an effect size (the D–CES) and determine associated statistical significance for findings from most SCD research. Therefore, under its Version 4.1 procedures and standards, the WWC calculates an effect size from SCD studies, if feasible and appropriate, which is then treated as any other effect size when determining intervention ratings. Consistent with the WWC, the HomVEE Version 2 Handbook includes an explanation of how the review will calculate the D–CES for SCD findings. The D–CES is comparable to a standardized mean-difference effect size and can be interpreted similarly to effect sizes from group design impact studies, such as randomized controlled trials, regression discontinuity designs, and non-experimental group designs. Because HomVEE can calculate the D– CES and then use the D–CES to determine the statistical significance of a finding for most SCDs, it is now possible to align the application of the HHS criteria for SCD research to group design impact studies. HomVEE proposes to align the application of HHS criteria accordingly. A D–CES can be computed for SCDs using the following designs: (1) Multiple baseline designs focused on the same outcome across three or more cases. (2) Multiple probe designs focused on the same outcome across three or more cases. (3) Reversal/withdrawal designs focused on the same outcome across three or more cases. A D–CES cannot be computed for SCDs using the following designs: (1) Multiple baseline designs focused on the same case across three or more settings. VerDate Sep<11>2014 18:46 Mar 19, 2021 Jkt 253001 (2) Multiple probe designs focused on the same case across three or more settings. (3) Several reversal/withdrawal designs focused on the same case. HomVEE proposes to review all eligible SCD manuscripts. HomVEE also proposes to determine statistical significance based on the D–CES for all manuscripts for which a D–CES can be calculated. For SCD manuscripts for which a D–CES cannot be calculated, for either design or data reasons, HomVEE proposes to report on the rating of the manuscript and the findings reported in it. However, research for which a D–CES cannot be calculated will not be included in the summary of evidence that contributes to the assessment of whether a model meets HHS criteria. Without a D–CES, HomVEE cannot determine statistical significance of the manuscript’s findings. 2.3 Proposed Changes to Author Query Procedures SCD manuscripts frequently include graphical representation of data at each time point. To calculate the D–CES requires numerical data from each time point. HomVEE proposes to request numerical data via an author query, as needed. If the author does not provide the numerical data, then HomVEE proposes to use a software package, such as WebPlotDigitizer (Rohatgi 2020), to extract numerical data from graphical presentations. This approach is consistent with WWC procedures. 2.4 Summary of Proposed Changes to Requirements for SCD Research To Be Considered Toward the HHS Criteria To account for the elimination of the 5–3–20 rule and to promote consistency in how well-designed, well-executed impact studies are considered toward the HHS criteria, HomVEE proposes the following updates: (1) Eliminate the 5–3–20 additional requirement that SCD research must meet to be considered toward the HHS criteria and instead consider SCD research toward the HHS criteria in a manner consistent with other designs. (2) To support the use of SCD findings toward the HHS criteria, implement three procedural steps that align with the Version 4.1 practices of the WWC, as follows: a. Request numerical data from authors, if necessary. b. Calculate the D–CES, if possible, for those designs that are rated moderate or high. c. Calculate statistical significance for the D–CES. These proposed changes ensure that any impact study with high or moderate PO 00000 Frm 00041 Fmt 4703 Sfmt 4703 15223 quality, as determined by the application of HomVEE Version 2 standards, can contribute to the determination of whether a model meets the HHS criteria for an ‘‘evidence-based early childhood home visiting service deliver model.’’ 3.0 Flexibilities for Applying Certain SCD Standards HomVEE proposes to clarify the flexibilities related to the application of certain SCD standards consistent with current WWC practice. Such flexibilities are intended to ensure that HomVEE standards are not unnecessarily stringent as more SCD manuscripts are reviewed. HomVEE proposes to update the HomVEE Version 2 Handbook text to clarify how existing flexibilities may be applied in the HomVEE review process. 3.1. Flexibility Related to the Timing of Probe Point Collection When a Case Receives the Intervention The first flexibility concerns the collection of probe data points in a multiple probe SCD. The current standard is strict requiring that baseline probes for cases without the intervention be collected in the same session when another case starts the intervention. The purpose of the baseline probe collection is to assess whether cases not receiving the intervention have changes in outcomes prior to receiving the intervention. HomVEE, and the WWC, recognize the requirement that baseline probes for cases with and without the intervention take place in the exact same session may be overly restrictive. The goal of the requirement related to the timing of baseline data collection for cases not yet receiving the intervention is to provide support that any change in outcomes in the cases receiving the intervention is likely due to the intervention and not some external factor (internal validity). In alignment with WWC procedures, HomVEE currently grants exceptions to this standard for individual manuscripts or interventions in consultation with subject matter experts. HomVEE proposes making the requirement more flexible by clarifying that baseline probe points may be collected when the intervention is introduced or in subsequent baseline sessions. 3.2 Flexibilities Related to the Minimum Number of Effects Demonstrated or Data Points Required The other flexibilities relate to the demonstration of effects—either the minimum number of effects demonstrated or the data points required to demonstrate an effect. E:\FR\FM\22MRN1.SGM 22MRN1 15224 Federal Register / Vol. 86, No. 53 / Monday, March 22, 2021 / Notices HomVEE currently allows exceptions to each of these standards. These flexibilities are needed to accommodate possible nuances in outcomes that may be examined in SCDs — for example, outcomes that are challenging for researchers to collect without burdening families (such as outcomes based on skills that may be frustrating to be tested on repeatedly if they have not been taught) or outcomes that are dangerous to collect repeatedly without intervening (such as some child maltreatment outcomes). To facilitate a transparent review, HomVEE proposes to clarify the process for granting and documenting the application of these flexibilities. Specifically, HomVEE proposes to specify that, if warranted, the HomVEE team can grant exceptions in collaboration with subject matter experts, and the exception (and its rationale) will be documented clearly in the review and related dissemination efforts. 4.0 Timeline for HomVEE To Apply New Procedures and Standards HomVEE proposes to apply the new procedures and standards for SCD Topic 1 ..................... 2 ..................... 3 ..................... 4 ..................... 5 ..................... jbell on DSKJLSW7X2PROD with NOTICES 6.0 5.0 Summary of the Proposed Changes and Placement in the Version 2 Handbook Following the 2020 substantial update to HomVEE procedures and standards, HomVEE proposes additional changes focused specifically on SCD research. The table below summarizes the proposed changes to be made to the HomVEE Version 2 Handbook and the relevant handbook section(s) that HomVEE proposes to update. Description of proposed change and relevant FRN section Requirements for considering SCD studies no longer have additional requirements to meet. single-case design (SCD) The statistical significance of a finding (based on a designresearch toward HHS cricomparable effect size, or D-CESCES) will be considered teria for an ‘‘evidence-based for SCD research when applying the HHS criteria. Manuearly childhood home visscripts for which the D-CESCES cannot be calculated will iting service delivery model’’. be reviewed and reported but will not contribute to summaries of evidence. (See Sections 2.1, 2.2, and 2.4 for more details.). Author queries for numerical HomVEE will query authors for the numerical data underlying data for SCD manuscripts. graphical results in manuscripts, as needed. The numerical data are necessary for calculating the D-CESCES, which is used to determine statistical significance.. If the author does not provide numerical data, then HomVEE proposes to use a software package to extract numerical data from graphical representations. (See Section 2.3 for more details.). Additional baseline data point HomVEE will require multiple probe designs to have baseline requirements for multiple data in the same or subsequent baseline session for cases probe SCDs. not receiving the intervention when a case starts the intervention. (See Section 3.1 for more details.). Flexibility in the requirement HomVEE may grant an exception to this requirement in confor demonstration of three sultation with subject matter experts. If granted, it will be attempts of intervention efdocumented in the review and related dissemination effects at three different forts, including the rationale for the exception. (See Secpoints in time. tion 3.2 for more details.). Number of data points reHomVEE may grant an exception to this requirement in conquired. sultation with subject matter experts. If granted, it will be documented in the review and related dissemination efforts, including the rationale for the exception. (See Section 3.2 for more details.). Request for Information (RFI) Through this FRN, ACF is soliciting information from a broad array of stakeholders on the proposed revisions to HomVEE’s procedures and standards related to SCD research. Federal, state, and local decision makers rely on HomVEE to know which home visiting models are effective. Applying the HHS criteria to SCDs as they are applied to other quasi-experimental designs or randomized controlled trials means that well-designed, well-executed SCD research will be treated on par with other forms of well-designed, wellexecuted impact studies. Responses to this FRN will inform ACF’s ongoing discussion about HomVEE’s procedures and standards VerDate Sep<11>2014 research beginning with the 2021 review. 18:46 Mar 19, 2021 Jkt 253001 with the aim of publishing a final HomVEE Version 2.1 Handbook of Procedures and Standards in 2021. This RFI is for information and planning purposes only and should not be construed as a solicitation or as an obligation on the part of ACF or HHS. Authority: Social Security Act Title V § 511 [42 U.S.C. 711], as extended by the Bipartisan Budget Act of 2018 (Pub. L. 115– 123) through fiscal year 2022. Naomi Goldstein, Deputy Assistant Secretary for Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. References Rohatgi, A. WebPlotDigitizer, version 4.3. PO 00000 Frm 00042 Fmt 4703 Sfmt 4703 Relevant HomVEE Version 2 Handbook section Section B.2.c and Exhibit II.11. Section B.1.b.i Appendix D, Section D.1.. Appendix D, Section B.5, Footnote 80 and related statement. Appendix D, Section B.6, Footnote 81. Appendix D, Section B.6, Footnotes 82 and 83. 2020. Available at https://automeris.io/ WebPlotDigitizer. Accessed February 26, 2021. Sama-Miller, E., J. Lugo-Gil, J. Harding, L. Akers, and R. Coughlin. Home Visiting Evidence of Effectiveness (HomVEE) Systematic Review: Handbook of Procedures and Evidence Standards: Version 2, December 2020. Available at https://homvee.acf.hhs.gov/publications/ methods-standards. Accessed January 7, 2021. U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, What Works Clearinghouse. What Works Clearinghouse Procedures Handbook: Version 4.1., 2020a. Available at https://ies.ed.gov/ncee/wwc/Docs/ referenceresources/WWC-ProceduresHandbook-v4-1-508.pdf. Accessed June 19, 2020. E:\FR\FM\22MRN1.SGM 22MRN1 15225 Federal Register / Vol. 86, No. 53 / Monday, March 22, 2021 / Notices U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, What Works Clearinghouse. What Works Clearinghouse Standards Handbook: Version 4.1., 2020b. Available at https://ies.ed.gov/ncee/wwc/ Docs/referenceresources/WWCStandards-Handbook-v4-1-508.pdf. Accessed June 19, 2020. [FR Doc. 2021–05840 Filed 3–19–21; 8:45 am] BILLING CODE 4184–74–P DEPARTMENT OF HEALTH AND HUMAN SERVICES Administration for Children and Families Submission for OMB Review; Runaway and Homeless Youth Homeless Management Information System (RHY–HMIS; New Collection) Family and Youth Services Bureau (FYSB); Administration on Children, Youth and Families (ACYF); Administration for Children and Families; HHS. AGENCY: ACTION: Request for Public Comment. The Administration on Children, Youth and Families (ACYF), Family and Youth Services Bureau has a legislative requirement to collect and maintain client statistical records on the numbers and the characteristics of runaway and homeless youth, and youth at risk of family separation, who receive shelter and supportive services through the Runaway and Homeless Youth (RHY) Program funding. RHY data collection is integrated with the U.S. Department of Housing and Urban Development’s (HUD) Homeless Management Information System (HMIS). DATES: Comments due within 30 days of publication. OMB is required to make a decision concerning the collection of information between 30 and 60 days after publication of this document in the Federal Register. Therefore, a comment is best assured of having its full effect if OMB receives it within 30 days of publication. ADDRESSES: Written comments and recommendations for the proposed information collection should be sent within 30 days of publication of this notice to www.reginfo.gov/public/do/ SUMMARY: PRAMain. Find this particular information collection by selecting ‘‘Currently under 30-day Review—Open for Public Comments’’ or by using the search function. SUPPLEMENTARY INFORMATION: Description: The RHY Program has a requirement to collect information from all youth who receive shelter and supportive services with RHY funding. In April 2015, ACYF, through a formal Memorandum of Understanding, integrated the RHY data collection with HUD’s HMIS and HUD’s data standards along with other federal partners. HUD’s data standards has its own OMB clearance, but ACYF is requesting approval for the RHY data collection efforts as HUD’s will no longer include all federal partners. The data collection instrument includes universal data elements, which are collected by all federal partners and program specific elements, which are tailored to each program using HUD’s HMIS. Respondents: Youth who receive emergency and longer-term shelter and supportive services under RHY funding. RHY grantees who enter and upload data. ANNUAL BURDEN ESTIMATES Total number of respondents Instrument jbell on DSKJLSW7X2PROD with NOTICES RHY–HMIS: Basic Center Program (Intake) ....................... RHY–HMIS: Basic Center Program (Exit) ........................... RHY–HMIS: Transitional Living Program (including Maternity Group Home program and TLP Demonstration Programs; Intake) .................................................................. RHY–HMIS: Transitional Living Program (including Maternity Group Home program and TLP Demonstration Programs; Exit) ...................................................................... RHY–HMIS: Street Outreach Program (Contact) ................ RHY–HMIS: Street Outreach Program (Engagement) ........ RHY Funded Grantees (data entry) .................................... RHY Funded Grantees (data submission)—FY21 .............. RHY Funded Grantees (data submission)—FY22 & FY23 Total number of responses per respondent 0.38 0.33 46,740 40,590 15,580 13,530 18,000 1 0.38 6,840 2,280 18,000 108,000 30,000 279,000 611 611 1 1 1 2 2 8 0.33 0.5 0.28 0.36 0.16 0.16 5,940 54,000 8,400 200,880 196 782 1,980 18,000 2,800 66,960 65 261 Mary B. Jones, ACF/OPRE Certifying Officer. Pursuant to section 10(d) of the Federal Advisory Committee Act, as amended, notice is hereby given of the following meetings. The meetings will be closed to the public in accordance with the provisions set forth in sections 552b(c)(4) and 552b(c)(6), Title 5 U.S.C., VerDate Sep<11>2014 18:46 Mar 19, 2021 Jkt 253001 National Institutes of Health National Institute of Environmental Health Sciences; Notice of Closed Meetings PO 00000 Frm 00043 Annual burden hours 1 1 DEPARTMENT OF HEALTH AND HUMAN SERVICES BILLING CODE 4184–33–P Total burden hours 123,000 123,000 Estimated Total Annual Burden Hours: 121,456. Authority: Reconnecting Homeless Youth Act of 2008 (Pub.L. 110–378) through FY 2013 and more recently reauthorized by the Juvenile Justice Reform Act through FY 2019. [FR Doc. 2021–05876 Filed 3–19–21; 8:45 am] Average burden hours per response Fmt 4703 Sfmt 4703 as amended. The grant applications and the discussions could disclose confidential trade secrets or commercial property such as patentable material, and personal information concerning individuals associated with the grant applications, the disclosure of which would constitute a clearly unwarranted invasion of personal privacy. Name of Committee: National Institute of Environmental Health Sciences Special Emphasis Panel; Research Career Development of Scientists/Investigators in the Environmental Health Sciences. Date: April 8, 2021. Time: 10:30 a.m. to 4:30 p.m. E:\FR\FM\22MRN1.SGM 22MRN1

Agencies

[Federal Register Volume 86, Number 53 (Monday, March 22, 2021)]
[Notices]
[Pages 15221-15225]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2021-05840]


-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Administration for Children and Families


Revised Single-Case Design Procedures and Standards: Home 
Visiting Evidence of Effectiveness (HomVEE) Review

AGENCY: Administration for Children and Families, U.S. Department of 
Health and Human Services.

ACTION: Request for public comment.

-----------------------------------------------------------------------

SUMMARY: The Administration for Children and Families (ACF), within the 
U.S. Department of Health and Human Services (HHS), oversees the Home 
Visiting Evidence of Effectiveness (HomVEE) review, which is proposing 
to revise the procedures and standards that guide its work with single-
case design (SCD) research. The current Federal Register notice (FRN) 
seeks comments on proposed changes related to revised procedures and 
standards for SCD research. Readers are referred to the full version of 
the HomVEE Version 2 Handbook on the HomVEE website for more details, 
particularly Appendix D (https://homvee.acf.hhs.gov/publications/methods-standards). HomVEE will release an updated Handbook (Version 
2.1) after consideration of public comments received in response to 
this FRN.

DATES: Send comments on or before April 19, 2021.

ADDRESSES: Submit questions, comments, and supplementary documents to 
[email protected] with ``HomVEE SCD procedures and standards FRN 
comment'' in the subject line.

SUPPLEMENTARY INFORMATION: Invitation to Comment: HHS invites comments 
regarding this notice. To ensure that your comments are clearly stated, 
please identify the section of this notice that your comments address.

1.0 Background

    To help policymakers, program administrators, model developers, 
researchers, and the public identify rigorous research and understand 
which early childhood home visiting models are effective, ACF's Office 
of Planning, Research, and Evaluation (OPRE) within HHS oversees the 
HomVEE review, in partnership with the Health Resources and Services 
Administration (HRSA). HomVEE's mission is to conduct a thorough and 
transparent review of early childhood home visiting models that serve 
pregnant women and children from birth to kindergarten entry. The 
review identifies well-designed, well-executed research, then extracts 
and summarizes the findings from that research.
    One critical use of HomVEE's results is to determine which home 
visiting models meet the HHS criteria for an ``evidence-based early 
childhood home visiting service delivery model'' (see Section 1.1 
below), a key requirement of eligibility for implementation of the 
model with Maternal, Infant, and Early Childhood Home Visiting (MIECHV) 
Program funding.
    The MIECHV Program is administered by HRSA in partnership with ACF. 
Created in 2010, the MIECHV Program provides funding to states, 
territories, and tribal entities to implement home visiting models. 
MIECHV awardees have the flexibility to tailor the program to serve the 
specific needs of their communities. Through a needs assessment, 
awardees identify at-risk communities and select home visiting service 
delivery models that best meet state and/or local needs. In accordance 
with MIECHV's authorizing statute, state and territory awardees must 
spend the majority of their MIECHV Program

[[Page 15222]]

grants to implement evidence-based home visiting models with up to 25 
percent of funding available to implement promising approaches that 
will undergo rigorous evaluation.
    In December 2020, HomVEE released an updated Version 2 Handbook of 
Procedures and Evidence Standards (Sama-Miller et al. 2020). The 
updates incorporated input from methodological experts, other federal 
evidence reviews, and public comments on two FRNs released on August 5, 
2020 (85 FR 47376 and 85 FR 47384). The HomVEE Version 2 Handbook 
adopted the What Works Clearinghouse (WWC) Version 4.1 Procedures and 
Standards Handbooks (2020a and 2020b), including revised SCD procedures 
and standards.
    This FRN proposes further changes to the HomVEE Version 2 Handbook 
to clarify how HomVEE would apply the WWC 4.1 procedures and standards 
that HomVEE already adopted. The proposed changes focus exclusively on 
SCD research, including:
    (1) Requirements for considering SCD research toward the HHS 
criteria for evidence-based models (discussed in Section 2 of the FRN).
    (2) Flexibilities for applying certain SCD standards (discussed in 
Section 3 of the FRN).
    A work group consisting of federal staff and HomVEE contractor 
staff, including methodological and home visiting experts, met to 
discuss and develop the proposed changes outlined in this FRN. Through 
this FRN, ACF seeks to gather stakeholder input on draft changes and to 
provide a transparent account of how the review operates. The remainder 
of Section 1 summarizes recent and current HomVEE procedures and 
standards with respect to SCD research.
    After a period of public comment (and consultation with selected 
experts outside HomVEE), HomVEE will release an updated Handbook 
(Version 2.1). Section 5 of this FRN specifies where in the updated 
Version 2.1 Handbook HomVEE would insert the proposed changes.

1.1 HHS Criteria for an ``Evidence-Based Early Childhood Home Visiting 
Service Delivery Model''

    As described in Section 2 of this FRN, HomVEE is proposing changes 
to requirements for how SCD research would be considered toward the HHS 
criteria. However, HomVEE is not proposing changes to the HHS criteria 
themselves. To meet HHS criteria for an ``evidence-based early 
childhood home visiting service delivery model,'' models must meet at 
least one of the following criteria:
    (1) At least one high- or moderate-quality impact study of the 
model finds favorable, statistically significant impacts in two or more 
of the eight outcome domains.
    (2) At least two high- or moderate-quality impact studies of the 
model using non-overlapping analytic study samples find one or more 
favorable, statistically significant impacts in the same domain.
    In both cases, the impacts must either (1) be found in the full 
sample for the study, or (2) if found for subgroups but not for the 
full sample, be replicated in the same domain in two or more studies 
using non-overlapping analytic study samples.
    Additionally, following the MIECHV authorizing statute, if the 
model meets the above criteria based on findings from randomized 
controlled trials only, then two additional requirements apply. First, 
one or more favorable (statistically significant) impacts must be 
sustained for at least one year after program enrollment. Second, one 
or more favorable (statistically significant) impacts must be reported 
in a peer-reviewed journal.
    Since HomVEE's inception, research about SCD studies has had to 
satisfy certain requirements before manuscripts about those studies 
could be considered toward the HHS criteria. These requirements, known 
as the ``5-3-20 rule,'' are as follows:
    (1) At least five studies examining the intervention either met the 
WWC's SCD standards without reservations or met those standards with 
reservations (corresponding to a ``high'' or ``moderate'' rating in 
HomVEE, respectively).
    (2) The SCDs were conducted by at least three research teams, with 
no overlapping authorship at three institutions.
    (3) The combined number of cases was at least 20.
    Beyond the 5-3-20 rule, the question of statistical significance is 
relevant to findings from SCD research because the HHS criteria focus 
on favorable, statistically significant findings. Authors of SCD 
manuscripts rarely describe their findings in terms of statistical 
significance, which HomVEE needs to apply the HHS criteria. However, 
the WWC Version 4.1 Procedures Handbook, adopted in the HomVEE Version 
2 Handbook, introduces the design-comparable effect size (D-CES), from 
which statistical significance can be determined.

2.0 Proposed Changes to How SCD Research Would Be Considered Toward HHS 
Criteria for an ``Evidence-Based Early Childhood Home Visiting Service 
Delivery Model''

    The HomVEE Version 2 Handbook adopted the WWC Single-Case Design 
Procedures and Standards (Version 4.1). Therefore, the latest round of 
updates to each of the WWC and HomVEE procedures and standards included 
three key complementary changes. The changes, described in more detail 
in Subsections 2.1 through 2.3, are as follows:
    (1) The WWC no longer uses the ``5-3-20'' rule, which established 
thresholds for when SCD research could contribute to evidence ratings. 
Consequently, HomVEE's previous requirements for SCD research about an 
early childhood home visiting model to be considered toward the HHS 
criteria (see Section 1.1) no longer align with current best practices 
for systematic reviews.
    (2) Under HomVEE's new procedures as defined in the Version 2 
Handbook and adopted from the WWC, it is now possible to calculate 
effect sizes and determine statistical significance for SCD research 
using the D-CES. Consequently, HomVEE proposes to align the application 
of the HHS criteria for SCD research to the procedures already in 
effect for other eligible research designs.
    (3) Reviewers will query authors for numerical data for calculating 
the D-CES, a step that had not been relevant under prior versions of 
procedures for HomVEE (or the WWC). Because calculation of the D-CES 
requires numerical data from each time point, HomVEE proposes to query 
authors, as needed, for the information required to calculate the D-
CES.

2.1 The 5-3-20 Rule No Longer Applies

    Additional requirements for considering results from SCD research 
toward the HHS criteria (the 5-3-20 rule) have been in effect since 
HomVEE's inception; those requirements were consistent with the WWC's 
then-current approach to SCD research. The 5-3-20 rule provided a 
threshold for determining when SCD research could contribute toward a 
decision about whether a model was evidence-based. Specifically, for 
SCD research to be considered toward the HHS criteria, the model's SCD 
research has been required to consist of at least five studies with 
high or moderate internal validity, to be conducted by three 
independent research teams, and

[[Page 15223]]

to include at least 20 cases. To date, no models reviewed by HomVEE 
have met the requirements of the 5-3-20 rule.
    The WWC Version 4.1 Procedures Handbook removed the 5-3-20 rule. 
Correspondingly, HomVEE now proposes to remove the 5-3-20 requirement 
for SCD research to be considered toward the HHS criteria.

2.2 Effect Sizes and Statistical Significance Can Now Be Computed

    The HHS criteria for evidence-based models require evidence of 
favorable, statistically significant findings. However, HomVEE's 
procedures did not previously specify how HomVEE would apply the HHS 
criteria to SCD research, for which researchers generally express 
findings in terms of visual patterns rather than statistical 
significance.
    Advances in meta-analysis have made it possible to calculate an 
effect size (the D-CES) and determine associated statistical 
significance for findings from most SCD research. Therefore, under its 
Version 4.1 procedures and standards, the WWC calculates an effect size 
from SCD studies, if feasible and appropriate, which is then treated as 
any other effect size when determining intervention ratings. Consistent 
with the WWC, the HomVEE Version 2 Handbook includes an explanation of 
how the review will calculate the D-CES for SCD findings.
    The D-CES is comparable to a standardized mean-difference effect 
size and can be interpreted similarly to effect sizes from group design 
impact studies, such as randomized controlled trials, regression 
discontinuity designs, and non-experimental group designs. Because 
HomVEE can calculate the D-CES and then use the D-CES to determine the 
statistical significance of a finding for most SCDs, it is now possible 
to align the application of the HHS criteria for SCD research to group 
design impact studies. HomVEE proposes to align the application of HHS 
criteria accordingly.
    A D-CES can be computed for SCDs using the following designs:
    (1) Multiple baseline designs focused on the same outcome across 
three or more cases.
    (2) Multiple probe designs focused on the same outcome across three 
or more cases.
    (3) Reversal/withdrawal designs focused on the same outcome across 
three or more cases.
    A D-CES cannot be computed for SCDs using the following designs:
    (1) Multiple baseline designs focused on the same case across three 
or more settings.
    (2) Multiple probe designs focused on the same case across three or 
more settings.
    (3) Several reversal/withdrawal designs focused on the same case.
    HomVEE proposes to review all eligible SCD manuscripts. HomVEE also 
proposes to determine statistical significance based on the D-CES for 
all manuscripts for which a D-CES can be calculated. For SCD 
manuscripts for which a D-CES cannot be calculated, for either design 
or data reasons, HomVEE proposes to report on the rating of the 
manuscript and the findings reported in it. However, research for which 
a D-CES cannot be calculated will not be included in the summary of 
evidence that contributes to the assessment of whether a model meets 
HHS criteria. Without a D-CES, HomVEE cannot determine statistical 
significance of the manuscript's findings.

2.3 Proposed Changes to Author Query Procedures

    SCD manuscripts frequently include graphical representation of data 
at each time point. To calculate the D-CES requires numerical data from 
each time point. HomVEE proposes to request numerical data via an 
author query, as needed. If the author does not provide the numerical 
data, then HomVEE proposes to use a software package, such as 
WebPlotDigitizer (Rohatgi 2020), to extract numerical data from 
graphical presentations. This approach is consistent with WWC 
procedures.

2.4 Summary of Proposed Changes to Requirements for SCD Research To Be 
Considered Toward the HHS Criteria

    To account for the elimination of the 5-3-20 rule and to promote 
consistency in how well-designed, well-executed impact studies are 
considered toward the HHS criteria, HomVEE proposes the following 
updates:
    (1) Eliminate the 5-3-20 additional requirement that SCD research 
must meet to be considered toward the HHS criteria and instead consider 
SCD research toward the HHS criteria in a manner consistent with other 
designs.
    (2) To support the use of SCD findings toward the HHS criteria, 
implement three procedural steps that align with the Version 4.1 
practices of the WWC, as follows:
    a. Request numerical data from authors, if necessary.
    b. Calculate the D-CES, if possible, for those designs that are 
rated moderate or high.
    c. Calculate statistical significance for the D-CES.
    These proposed changes ensure that any impact study with high or 
moderate quality, as determined by the application of HomVEE Version 2 
standards, can contribute to the determination of whether a model meets 
the HHS criteria for an ``evidence-based early childhood home visiting 
service deliver model.''

3.0 Flexibilities for Applying Certain SCD Standards

    HomVEE proposes to clarify the flexibilities related to the 
application of certain SCD standards consistent with current WWC 
practice. Such flexibilities are intended to ensure that HomVEE 
standards are not unnecessarily stringent as more SCD manuscripts are 
reviewed. HomVEE proposes to update the HomVEE Version 2 Handbook text 
to clarify how existing flexibilities may be applied in the HomVEE 
review process.

3.1. Flexibility Related to the Timing of Probe Point Collection When a 
Case Receives the Intervention

    The first flexibility concerns the collection of probe data points 
in a multiple probe SCD. The current standard is strict requiring that 
baseline probes for cases without the intervention be collected in the 
same session when another case starts the intervention. The purpose of 
the baseline probe collection is to assess whether cases not receiving 
the intervention have changes in outcomes prior to receiving the 
intervention.
    HomVEE, and the WWC, recognize the requirement that baseline probes 
for cases with and without the intervention take place in the exact 
same session may be overly restrictive. The goal of the requirement 
related to the timing of baseline data collection for cases not yet 
receiving the intervention is to provide support that any change in 
outcomes in the cases receiving the intervention is likely due to the 
intervention and not some external factor (internal validity).
    In alignment with WWC procedures, HomVEE currently grants 
exceptions to this standard for individual manuscripts or interventions 
in consultation with subject matter experts. HomVEE proposes making the 
requirement more flexible by clarifying that baseline probe points may 
be collected when the intervention is introduced or in subsequent 
baseline sessions.

3.2 Flexibilities Related to the Minimum Number of Effects Demonstrated 
or Data Points Required

    The other flexibilities relate to the demonstration of effects--
either the minimum number of effects demonstrated or the data points 
required to demonstrate an effect.

[[Page 15224]]

HomVEE currently allows exceptions to each of these standards. These 
flexibilities are needed to accommodate possible nuances in outcomes 
that may be examined in SCDs -- for example, outcomes that are 
challenging for researchers to collect without burdening families (such 
as outcomes based on skills that may be frustrating to be tested on 
repeatedly if they have not been taught) or outcomes that are dangerous 
to collect repeatedly without intervening (such as some child 
maltreatment outcomes).
    To facilitate a transparent review, HomVEE proposes to clarify the 
process for granting and documenting the application of these 
flexibilities. Specifically, HomVEE proposes to specify that, if 
warranted, the HomVEE team can grant exceptions in collaboration with 
subject matter experts, and the exception (and its rationale) will be 
documented clearly in the review and related dissemination efforts.

4.0 Timeline for HomVEE To Apply New Procedures and Standards

    HomVEE proposes to apply the new procedures and standards for SCD 
research beginning with the 2021 review.

5.0 Summary of the Proposed Changes and Placement in the Version 2 
Handbook

    Following the 2020 substantial update to HomVEE procedures and 
standards, HomVEE proposes additional changes focused specifically on 
SCD research. The table below summarizes the proposed changes to be 
made to the HomVEE Version 2 Handbook and the relevant handbook 
section(s) that HomVEE proposes to update.

----------------------------------------------------------------------------------------------------------------
                                                            Description of proposed
                                        Topic               change and relevant FRN      Relevant HomVEE Version
                                                                    section                2 Handbook section
----------------------------------------------------------------------------------------------------------------
1...........................  Requirements for           SCD studies no longer have     Section B.2.c and
                               considering single-case    additional requirements to     Exhibit II.11.
                               design (SCD) research      meet. The statistical
                               toward HHS criteria for    significance of a finding
                               an ``evidence-based        (based on a design-
                               early childhood home       comparable effect size, or D-
                               visiting service           CESCES) will be considered
                               delivery model''.          for SCD research when
                                                          applying the HHS criteria.
                                                          Manuscripts for which the D-
                                                          CESCES cannot be calculated
                                                          will be reviewed and
                                                          reported but will not
                                                          contribute to summaries of
                                                          evidence. (See Sections 2.1,
                                                          2.2, and 2.4 for more
                                                          details.).
2...........................  Author queries for         HomVEE will query authors for  Section B.1.b.i Appendix
                               numerical data for SCD     the numerical data             D, Section D.1..
                               manuscripts.               underlying graphical results
                                                          in manuscripts, as needed.
                                                          The numerical data are
                                                          necessary for calculating
                                                          the D-CESCES, which is used
                                                          to determine statistical
                                                          significance..
                                                         If the author does not
                                                          provide numerical data, then
                                                          HomVEE proposes to use a
                                                          software package to extract
                                                          numerical data from
                                                          graphical representations.
                                                          (See Section 2.3 for more
                                                          details.).
3...........................  Additional baseline data   HomVEE will require multiple   Appendix D, Section B.5,
                               point requirements for     probe designs to have          Footnote 80 and related
                               multiple probe SCDs.       baseline data in the same or   statement.
                                                          subsequent baseline session
                                                          for cases not receiving the
                                                          intervention when a case
                                                          starts the intervention.
                                                          (See Section 3.1 for more
                                                          details.).
4...........................  Flexibility in the         HomVEE may grant an exception  Appendix D, Section B.6,
                               requirement for            to this requirement in         Footnote 81.
                               demonstration of three     consultation with subject
                               attempts of intervention   matter experts. If granted,
                               effects at three           it will be documented in the
                               different points in time.  review and related
                                                          dissemination efforts,
                                                          including the rationale for
                                                          the exception. (See Section
                                                          3.2 for more details.).
5...........................  Number of data points      HomVEE may grant an exception  Appendix D, Section B.6,
                               required.                  to this requirement in         Footnotes 82 and 83.
                                                          consultation with subject
                                                          matter experts. If granted,
                                                          it will be documented in the
                                                          review and related
                                                          dissemination efforts,
                                                          including the rationale for
                                                          the exception. (See Section
                                                          3.2 for more details.).
----------------------------------------------------------------------------------------------------------------

6.0 Request for Information (RFI)

    Through this FRN, ACF is soliciting information from a broad array 
of stakeholders on the proposed revisions to HomVEE's procedures and 
standards related to SCD research. Federal, state, and local decision 
makers rely on HomVEE to know which home visiting models are effective. 
Applying the HHS criteria to SCDs as they are applied to other quasi-
experimental designs or randomized controlled trials means that well-
designed, well-executed SCD research will be treated on par with other 
forms of well-designed, well-executed impact studies.
    Responses to this FRN will inform ACF's ongoing discussion about 
HomVEE's procedures and standards with the aim of publishing a final 
HomVEE Version 2.1 Handbook of Procedures and Standards in 2021. This 
RFI is for information and planning purposes only and should not be 
construed as a solicitation or as an obligation on the part of ACF or 
HHS.

    Authority: Social Security Act Title V Sec.  511 [42 U.S.C. 
711], as extended by the Bipartisan Budget Act of 2018 (Pub. L. 115-
123) through fiscal year 2022.

Naomi Goldstein,
Deputy Assistant Secretary for Planning, Research, and Evaluation, 
Administration for Children and Families, U.S. Department of Health and 
Human Services.

References

Rohatgi, A. WebPlotDigitizer, version 4.3. 2020. Available at 
https://automeris.io/WebPlotDigitizer. Accessed February 26, 2021.
Sama-Miller, E., J. Lugo-Gil, J. Harding, L. Akers, and R. Coughlin. 
Home Visiting Evidence of Effectiveness (HomVEE) Systematic Review: 
Handbook of Procedures and Evidence Standards: Version 2, December 
2020. Available at https://homvee.acf.hhs.gov/publications/methods-standards. Accessed January 7, 2021.
U.S. Department of Education, Institute of Education Sciences, 
National Center for Education Evaluation and Regional Assistance, 
What Works Clearinghouse. What Works Clearinghouse Procedures 
Handbook: Version 4.1., 2020a. Available at https://ies.ed.gov/ncee/wwc/Docs/referenceresources/WWC-Procedures-Handbook-v4-1-508.pdf. 
Accessed June 19, 2020.

[[Page 15225]]

U.S. Department of Education, Institute of Education Sciences, 
National Center for Education Evaluation and Regional Assistance, 
What Works Clearinghouse. What Works Clearinghouse Standards 
Handbook: Version 4.1., 2020b. Available at https://ies.ed.gov/ncee/wwc/Docs/referenceresources/WWC-Standards-Handbook-v4-1-508.pdf. 
Accessed June 19, 2020.

[FR Doc. 2021-05840 Filed 3-19-21; 8:45 am]
BILLING CODE 4184-74-P