Statewide Data Indicators and National Standards for Child and Family Services Reviews, 22604-22615 [2014-09001]

Download as PDF wreier-aviles on DSK5TPTVN1PROD with PROPOSALS 22604 Federal Register / Vol. 79, No. 78 / Wednesday, April 23, 2014 / Proposed Rules legume, subgroup 6B at 0.5 ppm; vegetable, legume, subgroup 6C at 0.2 ppm; vegetable, foliage of legume, subgroup 7A at 4 ppm; wheat, aspirated grain fraction at 2.5 ppm; wheat, bran at 0.5 ppm; wheat, germ at 0.5 ppm; wheat, grain at 0.5 ppm. Syngenta Crop Protection, LLC, has submitted practical analytical methodology for detecting and measuring levels of thiamethoxam in or on raw agricultural commodities. This method is based on crop specific cleanup procedures and determination by liquid chromatography with either ultraviolet (UV) or mass spectrometry (MS) detections. The limit of detection (LOC) for each analyte of this method is 1.25ng injected for samples analyzed by UV and 0.25 nanogram (ng) injected for samples analyzed by MS, and the limit quantification (LOQ) is 0.005 ppm for milk and juices, and 0.01 ppm for all other substrates. 2. PP 4F8237. (EPA–HQ–OPP–2014– 0156). Dow AgroSciences, LLC, 9330 Zionsville Rd., Indianapolis, IN 46268, requests to establish a tolerance in 40 CFR part 180 for residues of the insecticide, sulfoxaflor (N[methyloxido[1-[6-(trifluoromethyl)-3pyridinyl]ethyl]-g 4sulfanylidene]cyanamide), in or on alfalfa, forage at 7 parts per million (ppm); alfalfa, hay at 20 ppm; alfalfa, seed at 30 ppm; alfalfa, silage at 9 ppm; animal feed, nongrass, group 18, forage at 15 ppm; animal feed, nongrass, group 18, hay at 20 ppm; animal feed, nongrass, group 18, silage at 9 ppm; buckwheat, forage at 1 ppm; buckwheat, grain at 0.08 ppm; buckwheat, hay at 1.5 ppm; buckwheat, straw at 2 ppm; cacao bean, dried bean at 0.15 ppm; clover forage at 15 ppm; clover hay at 20 ppm; clover silage at 8 ppm; corn, field, forage at 0.5 ppm; corn, field, grain at 0.015 parts ppm; corn, field, stover at 0.8 ppm; corn, pop at 0.015 ppm; corn, pop, stover at 0.8 ppm; corn, sweet, at 0.01 ppm; corn, sweet, forage at 0.6 ppm; corn, sweet, stover at 0.7 ppm; millet, forage at 0.4 ppm; millet, grain at 0.3 ppm; oat, grain at 0.4 ppm; oat, hay at 1 ppm; oat, straw at 2 ppm; pineapple at 0.09 ppm; rye, forage at 1 ppm; rye, grain at 0.08 ppm; rye, hay at 1.5 ppm; rye, straw at 2 ppm; sorghum, forage at 0.4 ppm; sorghum, grain at 0.3 ppm; sorghum, stover at 0.9 ppm; teff, forage at 1 ppm; teff, grain at 0.08 ppm; teff, hay at 1.5 ppm; teff, straw at 2 ppm; teosinte, grain at 0.015 ppm; triticale, forage at 1 ppm; triticale, grain at 0.08 ppm; triticale, hay at 1.5 ppm; triticale, straw at 2 ppm. The residue profile of sulfoxaflor is adequately understood and an acceptable analytical method is available for enforcement purposes. VerDate Mar<15>2010 15:33 Apr 22, 2014 Jkt 232001 Analytical method 091116, ‘‘Enforcement Method for the Determination of Sulfoxaflor (XDE–208) and its Main Metabolites in Agricultural Commodities using Offline Solid-Phase Extraction and Liquid Chromatography with Tandem Mass Spectrometry Detection’’ was validated on a variety of plant matrices. The method was validated over the concentration range of 0.010–5.0 milligrams/kilograms (mg/ kg) with a validated limit of detection (LOD) of 0.003 mg/kg and limit of quantitation (LOQ) of 0.010 mg/kg. Amended Tolerance Frm 00009 Fmt 4702 List of Subjects in 40 CFR Part 180 Environmental protection, Agricultural commodities, Feed additives, Food additives, Pesticides and pests, Reporting and recordkeeping requirements. Dated: April 17, 2014. Lois Rossi, Director, Registration Division, Office of Pesticide Programs. [FR Doc. 2014–09257 Filed 4–22–14; 8:45 am] BILLING CODE 6560–50–P 3. PP 3F8205. (EPA–HQ–OPP–2013– 0758). Syngenta Crop Protection, LLC, P.O. Box 18300, Greensboro, NC 27419, requests to amend the tolerances in 40 CFR 180.565 for residues of the insecticide, thiamethoxam (3-[(2-chloro5-thiazolyl)methyl]tetrahydro-5-methylN-nitro-4H-1,3,5-oxadiazin-4-imine) and its metabolite (N-(2-chloro-thiazol-5ylmethyl)-N’-methyl-N’-nitroguanidine), by increasing the existing tolerances in or on alfalfa, forage from 0.05 to 10 parts per million (ppm); alfalfa, hay from 0.12 to 8 ppm; barley, grain from 0.4 to 0.9 ppm; barley, hay from 0.40 to 1.5 ppm; barley, straw from 0.40 to 3 ppm; corn, field, forage from 0.10 to 4 ppm; corn, field, stover from 0.05 to 4 ppm; corn, sweet, forage from 0.10 to 5 ppm; corn, sweet, kernel plus cob with husks removed from 0.02 to 0.03 ppm; corn, sweet, stover from 0.05 to 4 ppm; wheat, forage from 0.50 to 3 ppm; wheat, hay from 0.02 to 8 ppm; wheat, straw from 0.02 to 6 ppm. Concurrently, Syngenta Crop Protection, LLC, requests to amend the tolerances in 40 CFR 180.565 by removing tolerances for residues of the insecticide, thiamethoxam (3-[(2-chloro-5thiazolyl)methyl]tetrahydro-5-methyl-Nnitro-4H-1,3,5-oxadiazin-4-imine) in or on grain, cereal, group 15, except barley at 0.02 ppm; sunflower at 0.02 ppm; and vegetable, legume, group 6 at 0.02 ppm, upon approval of the tolerances listed under ‘‘New Tolerances’’ for PP 3F8205. Syngenta Crop Protection, LLC, has submitted practical analytical methodology for detecting and measuring levels of thiamethoxam in or on raw agricultural commodities. This method is based on crop specific cleanup procedures and determination by liquid chromatography with either ultraviolet (UV) or mass spectrometry (MS) detections. The limit of detection (LOC) for each analyte of this method is 1.25 ng injected for samples analyzed by UV and 0.25 nanogram (ng) injected for samples analyzed by MS, and the limit quantification (LOQ) is 0.005 ppm for PO 00000 milk and juices, and 0.01 ppm for all other substrates. Sfmt 4702 DEPARTMENT OF HEALTH AND HUMAN SERVICES Administration for Children and Families 45 CFR Part 1355 Statewide Data Indicators and National Standards for Child and Family Services Reviews Children’s Bureau (CB), Administration on Children, Youth and Families (ACYF), Administration for Children and Families (ACF), Department of Health and Human Services (HHS). ACTION: Notice of Statewide Data Indicators and National Standards for Child and Family Services Reviews. AGENCY: The Children’s Bureau reviews a state’s substantial conformity with titles IV–B and IV–E of the Social Security Act through the Child and Family Services Reviews (CFSRs). Statewide data indicators are used to inform the Children’s Bureau’s determination of a state’s substantial conformity relative to certain safety and permanency outcomes. This document advises the public of the Children’s Bureau’s plan to replace the statewide data indicators and the methods for calculating associated national standards on those indicators. We invite the public to comment on these indicators and methods before their use in CFSRs scheduled for Federal Fiscal Years (FFY) 2015 through FY 2018. DATES: Written comments must be submitted to the office listed in the ADDRESSES section below on or before May 23, 2014. ADDRESSES: Interested persons may submit written comments by any of the following methods: • Federal eRulemaking Portal: http:// www.regulations.gov. Follow the instructions for submitting comments. SUMMARY: E:\FR\FM\23APP1.SGM 23APP1 Federal Register / Vol. 79, No. 78 / Wednesday, April 23, 2014 / Proposed Rules • Mail or Courier Delivery: Miranda Lynch Thomas, Children’s Bureau, Administration on Children, Youth and Families, Administration for Children and Families, 1250 Maryland Avenue SW., 8th Floor, Washington, DC 20024. Instructions: If you choose to use an express, overnight, or other special delivery method, ensure that delivery may be made at the address listed under the ADDRESSES section. We urge interested parties to submit comments electronically to ensure that they are received in a timely manner. All comments received will be posted without change to http:// www.regulations.gov. This will include any personal information provided. FOR FURTHER INFORMATION CONTACT: Miranda Lynch Thomas, Children’s Bureau, 1250 Maryland Ave. SW., 8th Floor, Washington, DC 20024, (202) 205–8138. SUPPLEMENTARY INFORMATION: wreier-aviles on DSK5TPTVN1PROD with PROPOSALS Background The Children’s Bureau implemented the CFSRs in 2001 in response to a mandate in the Social Security Amendments of 1994 for the Department of Health and Human Services to issue regulations for the review of state child and family services programs under titles IV–B and IV–E of the Social Security Act (see section 1123A of the Social Security Act). The reviews are required for the Children’s Bureau to determine whether such programs are in substantial conformity with title IV–B and IV–E plan requirements, implementing regulations, and relevant title IV–B and IV–E plans. The review process, as regulated at 45 CFR 1355.31–37, grew out of extensive consultation with interested groups, individuals, and experts in the field of child welfare and related areas. The Children’s Bureau conducted the first round of CFSRs from 2001 through 2004 and the second round from 2007 through 2010. The third round is scheduled to begin in FFY 2015. Information about the initiation of this latest round can be found in CFSR Technical Bulletin #7 issued in March 2014 (see http://www.acf.hhs.gov/ programs/cb/resource/cfsr-technicalbulletin-7). The CFSRs enable the Children’s Bureau to: (1) Ensure conformity with federal child welfare requirements; (2) determine what is actually happening to children and families as they are engaged in child welfare services; and (3) assist states to enhance their capacity to help children and families achieve positive outcomes. The Children’s VerDate Mar<15>2010 15:33 Apr 22, 2014 Jkt 232001 Bureau conducts the reviews in partnership with state child welfare agency staff and other stakeholders involved in the provision of child welfare services. We have structured the reviews to help states identify strengths as well as areas needing improvement within their agencies and programs. The CFSR assesses state performance on seven outcomes and seven systemic factors. The seven outcomes focus on key items measuring safety, permanency, and well-being. The seven systemic factors focus on key state plan requirements of titles IV–B and IV–E that provide a foundation for child outcomes. States that the Children’s Bureau determines have not achieved substantial conformity in all the areas assessed in the review are required to develop and implement a program improvement plan within two years addressing the areas of nonconformity. The Children’s Bureau supports the states with technical assistance and monitors implementation of their program improvement plans. States that are unable to complete their plans successfully have some of their federal child welfare funds withheld. Most relevant to this document is the element of the reviews that provides for the Children’s Bureau to determine whether the state is in substantial conformity with certain child outcomes based on national standards we set for state performance on statewide data indicators. The regulations at 45 CFR 1355.34(b)(4) and (5) authorize us to add, amend, or suspend any of the statewide data indicators when appropriate, and to adjust the national standards when appropriate. Statewide data indicators are aggregate measures and we calculate them using administrative data available from a state’s submissions to the Adoption and Foster Care Analysis and Reporting System (AFCARS),1 the National Child Abuse and Neglect Data System (NCANDS),2 or a Children’s Bureauapproved alternate source for safetyrelated data. If a state is determined not to be in substantial conformity with a related outcome due to its performance compared to the national standard for an indicator, the state will include that 1 AFCARS collects case-level information from state and Tribal title IV–E agencies on all children in foster care and those who have been adopted with title IV–E agency involvement. Title IV–E agencies must submit AFCARS data to the Children’s Bureau twice a year. 2 NCANDS collects child-level information on every child who receives a response from a child protective services agency due to an allegation of abuse or neglect. States report this data to the Children’s Bureau voluntarily. In FFY 2013, all 50 states, the District of Columbia, and Puerto Rico submitted NCANDS data. PO 00000 Frm 00010 Fmt 4702 Sfmt 4702 22605 indicator in a program improvement plan. The Children’s Bureau establishes performance goals for each indicator included in a program improvement plan, based on the state’s prior performance, that the state is expected to reach by the end of the program improvement plan period. The goal to be achieved is relative to the state’s baseline performance at the beginning of the program improvement plan period. The Children’s Bureau views the CFSR as a dynamic process and has made ongoing improvements in the process to best meet state and federal needs. Most recently, we solicited feedback from the public (see 76 FR 18677, published April 5, 2011) about how they would envision a federal review process that meets the statutory requirements in section 1123A of the Social Security Act and holds child welfare agencies accountable for achieving positive outcomes for children and families and continuously improving the quality of their systems for doing so. In addition, we hired a consultant that specializes in child welfare measurement to work with Children’s Bureau data specialists. In 2013 we also tasked a contractor to the Children’s Bureau to convene a panel of child welfare administrators and data measurement experts to develop recommendations and feedback about specific aspects of the review process, including the statewide data indicators, national standards, and program improvement. The information from these experts along with public comments has shaped our plan for replacing the statewide data indicators that will be used in the CFSRs. Existing Statewide Data Indicators and Composite Measures and Planned Improvements To Address Feedback For CFSR Round 2, the Children’s Bureau developed six statewide data indicators and measures: two indicators related to safety and four composite measures related to permanency. The two safety-related statewide data indicators focused on recurrence of maltreatment and maltreatment of children in foster care and were used to inform an assessment of the state’s substantial conformity with the safety outcome that children are, first and foremost, protected from abuse and neglect. The four permanency-related data composites were used to inform the assessment of a state’s substantial conformity with the permanency outcome that children have permanency and stability in their living situations. The four permanency composites used during CFSR Round 2 were related to measures of timeliness and permanency E:\FR\FM\23APP1.SGM 23APP1 22606 Federal Register / Vol. 79, No. 78 / Wednesday, April 23, 2014 / Proposed Rules wreier-aviles on DSK5TPTVN1PROD with PROPOSALS of reunification; timeliness of adoptions, permanency for children in foster care for extended time periods, and placement stability.3 The Children’s Bureau established national standards for each of the six data indicators and composites. The Children’s Bureau thoroughly considered the alternatives available to us at the time and had a clear rationale for supporting the statewide data indicators, composites and methods chosen. However, we believe there is reason to make further improvements now given the additional concerns raised by stakeholders and the Children’s Bureau that we summarize below and throughout this document as we describe the proposed new indicators. Cohorts Used for Statewide Data Indicators: The Children’s Bureau is planning to calculate several statewide data indicators using entry cohorts to address concerns about the measurement techniques that relied on multiple exit cohorts in prior CFSR rounds. Some stakeholders noted that relying on exit cohorts and crosssectional (also referred to as point-intime) cohorts, rather than prospective entry cohorts, can statistically distort system performance outcomes because they represent biased sub-samples of all children who are served.4 For example, looking only at children who exit foster care in a given year ignores those children who did not leave; it is therefore not representative of all children in foster care who would be affected by state improvement efforts. Looking only at those children in foster care at the end of the year biases the sample to include more children with long lengths of stay in foster care. Entry cohorts, which include all children entering foster care in a given year, avoid these problems and provide a more complete assessment of overall system performance and recent practice trends. While entry cohorts have methodological advantages, they have limitations in terms of assessing state performance with regard to children who have been in foster care for a long time because of the length of time we measure for state improvements. For example, with an entry cohort approach, 3 More detailed information on the CFSR Round 2 measures can be found on the Children’s Bureau Web site at http://www.acf.hhs.gov/sites/default/ files/cb/data_indicators_for_the_second_round_of_ cfsrs.pdf. 4 See Testa, M., Poertner, J. Koh, E. (2008). Can AFCARS be rescued? Fixing the statistical yardstick that measures state child welfare performance. Urbana, IL: CFRC; and Testa, M. & Poertner, J. (Eds.). (2010). Fostering accountability: Using evidence to guide and improve child welfare policy. Oxford: Oxford University Press. VerDate Mar<15>2010 15:33 Apr 22, 2014 Jkt 232001 children who had already been in foster care for two or more years could only start being tracked in a third year. To address this limitation, the Children’s Bureau will still use other cohorts in some of its indicators. Composites: The Children’s Bureau plans to use individual indicators rather than composite measures for the purpose of establishing national standards in this round of review. We implemented the composite approach in CFSR Round 2 after consultation with states and national experts to support a more holistic approach to measuring state performance. The expansion to composites from the one-dimensional measures used in CFSR Round 1 allowed state performance on a particular domain to reflect broader performance, accounting for both strengths and weaknesses within the domain. Criticism of the composite measures used during CFSR Round 2 included the complexity of the composite scores for interpretation purposes. To address these concerns and clarify expectations with regard to national performance, the Children’s Bureau proposes to measure state performance with simplified statewide data indicators. We propose to maintain some of the advantages found with the composite approach by implementing companion measures during the program improvement plan period to provide an expanded and more effective measurement of a domain. New Proposed Statewide Data Indicators and Methods The Children’s Bureau plans for the new statewide data indicators to measure maltreatment in foster care and re-report of maltreatment as a component in evaluating Safety Outcome 1: Children are, first and foremost, protected from abuse and neglect. We also plan for statewide data indicators to measure the achievement of permanency in 12 months for children entering foster care, permanency in 12 months for children in foster care for 2 years or more, reentry to foster care, and placement stability. These four permanency indicators will be used as a component in evaluating Permanency Outcome 1: Children have permanency and stability in their living situations. Below is a description of each of the six proposed indicators including their definition, justification for inclusion, calculation and a discussion of relevant issues. Following the description of the indicators is information on the methods the Children’s Bureau plans to use for calculating the national PO 00000 Frm 00011 Fmt 4702 Sfmt 4702 standards and our approach to measuring a state’s program improvement on the indicators should we find that the state is not able to meet the national standard. We also provide additional information on how the Children’s Bureau will share data information with states through profiles and data quality issues that impact these indicators and methods. Attachment A provides a summary of each planned indicator including numerators, denominators, exclusions, and adjustments. Attachment B provides a comparison of the data measures used during CFSR Round 2 with the statewide data indicators we propose to use during Round 3. Attachment C provides information on the AFCARS and NCANDS data elements that are used to calculate the proposed indicators and national standards. Statewide Data Indicators for CFSR Safety Outcome 1: Children Are, First and Foremost, Protected From Abuse and Neglect Proposed Safety Performance Area 1: Maltreatment in Foster Care Indicator Definition: Of all children in foster care during a 12-month period, what is the rate of victimization per day of foster care? 5 The indicator includes all cases of substantiated or indicated maltreatment while in foster care and all days for all children in foster care at any point during a 12-month period. The denominator is all child days in foster care over a 12-month period, and the numerator is the number of instances of substantiated or indicated maltreatment among children in foster care over that same period. The definition of ‘‘children’’ for this indicator (and all indicators) includes those under the age of 18. This indicator includes all maltreatment types by any perpetrator, which may include foster parents, facility staff members, parents, or others. In addition, this indicator includes all days for all children in foster care at any point during a 12month period. Some states provide incident dates in their NCANDS data submissions. If a state provides incident dates, records with an incident date occurring before the date of removal will be excluded. Children in foster care for less than 8 days and any report that occurs within the first 7 days of removal are excluded from this indicator. This indicator is calculated using data that match 5 Rates are calculated per day of foster care. However, we will multiply the rate by 100,000 to produce larger and more meaningful numbers. E:\FR\FM\23APP1.SGM 23APP1 Federal Register / Vol. 79, No. 78 / Wednesday, April 23, 2014 / Proposed Rules children across AFCARS and NCANDS using the AFCARS identifier. Justification for Inclusion: This indicator provides a measure of whether the state child welfare agency is able to ensure that children do not experience abuse or neglect while in the state’s foster care system. The indicator holds states accountable for keeping children safe from harm while under the responsibility of the state, no matter who perpetrates the maltreatment in foster care. Relevant Issues and Discussion: During CFSR Round 2, the Children’s Bureau had a safety data indicator related to maltreatment in foster care. For that indicator, the counts of children not maltreated in foster care were derived by subtracting the NCANDS count of children maltreated by foster care providers from the total count of all children in foster care on the last day of the year, as reported in AFCARS. Because of improved reporting by states, we can now link AFCARS and NCANDS data using the child identifier and determine whether maltreatment occurred during a foster care episode, improving accuracy on this indicator by using entry cohorts instead of the retrospective method used in CFSR Round 2.6 This technique also allows us to expand the indicator to include maltreatment regardless of the perpetrator type. This measure uses the report date reported in NCANDS to determine if a child is victimized while in foster care, discounting the first week to allow for a potential lag time between an incidence of maltreatment and report of maltreatment. For those states that provide incident dates, an adjustment will be made if the data indicates that the incident occurred prior to the foster care episode. We encourage states to report incident dates in NCANDS, which will improve the accuracy of this indicator. The Children’s Bureau made this change in response to a suggestion from stakeholders with regard to the indicators used for the last round of reviews that we are now able to address. wreier-aviles on DSK5TPTVN1PROD with PROPOSALS Proposed Safety Performance Area 2: Re-Report of Maltreatment Indicator Definition: Of all children who received a screened-in report of maltreatment during a 12-month period (regardless of disposition type), what percent were reported again within 12 months from the date of initial report? The denominator is the number of children with at least one screened-in 6 Almost all states report AFCARS identifiers in the NCANDS data. For those states that do not, a Children’s Bureau-approved alternate source will be required to assess performance on this indicator. VerDate Mar<15>2010 15:33 Apr 22, 2014 Jkt 232001 report of alleged maltreatment in a 12month period and the numerator is the number of children in the denominator that had another screened-in report with a disposition within 12 months of their initial report. Screened-in reports that have a disposition reported are included, regardless of whether the disposition is that the child is a victim or a non-victim. This indicator is calculated using data from NCANDS. Justification for Inclusion: This indicator is included to provide an assessment of whether the agency took the necessary actions to prevent a future report of maltreatment for children previously the subject of a screened-in report to the agency. Relevant Issues and Discussion: During CFSR Round 2, the Children’s Bureau had a safety performance area related to repeat maltreatment. That measure was derived from calculating what percent of all children who were victims in a substantiated or indicated maltreatment allegation during the first 6 months of the reporting period were not victims in another substantiated or indicated maltreatment allegation within a 6-month period. We are expanding this measure to count all children with screened-in reports of alleged maltreatment. The Children’s Bureau believes that multiple reports regardless of whether maltreatment is substantiated or indicated is a viable measure of the agency’s attempts to prevent maltreatment based on research indicating that families with screened-in but unsubstantiated reports are at a high risk of re-report, in some cases as high as substantiated cases.7 Because reports are included regardless of disposition, this indicator includes both victims and non-victims. In addition, this indicator expands the time period examined to 12 months to include more children. The indicator also tracks such children for 12 months, as opposed to 6 months as in the prior indicator. The Children’s Bureau made this change in response to a suggestion from stakeholders with regard to the indicators used for the last round of reviews that we are now able to address with the improved quality of data reported in NCANDS. In addition, the contractor’s recommendations based on the expert panel convened in 2013 expressed support for a measure of screened-in reports to capture repeat maltreatment. Previous CFSR data measures focused on substantiated and indicated reports of maltreatment. The growing 7 See Drake, Jonson-Reid, Way, & Chung (2003). Substantiation and Recidivism. Child Maltreatment. Vol. 8, No. 4, 248–260. PO 00000 Frm 00012 Fmt 4702 Sfmt 4702 22607 implementation of differential response in the states (sometimes referred to as alternative response programs) where a substantial percentage of cases may bypass formal investigation altogether, however, makes a comparison of differential-response and nondifferential-response states difficult. In addition, states that initiate or expand differential response during an improvement period could show improvement on a substantiation-based measure of repeat maltreatment merely as an artifact of adopting differential response. An indicator based only on screened-in reports is not affected by differential response which contributed to our selecting this indicator. CFSR Permanency Outcome 1: Children Have Permanency and Stability in Their Living Situations The permanency-related statewide data indicators exclude children entering foster care at age 18 and older or who are already 18 and older on the first day of the period under review. Although the amendments to title IV–E of the Social Security Act made by the Fostering to Success and Increasing Adoptions Act of 2008 (Pub. L. 110– 351) permit states to provide foster care to youth who are age 18 and older, all states have not exercised such an option. Some states provide foster care to youth age 18 and older, however, there is no consistent inclusion of this population of youth across states and no consistent construct at this time for what achieving permanency means for such older youth. Therefore, the Children’s Bureau believes that it is appropriate to limit the permanency statewide data indicators to children under age 18 in this way to maintain consistency as we have in prior rounds. Proposed Permanency Performance Area 1: Permanency in 12 Months for Children Entering Foster Care Indicator Definition: Of all children who enter foster care in a 12-month period, what percent discharged to permanency within 12 months of entering foster care? The denominator is the number of children who enter foster care in a 12-month period and the numerator is the number of children in the denominator who discharged to permanency within 12 months of entering foster care. For the purposes of this indicator, permanency includes a child’s discharge from foster care to parent(s), living with relatives, guardianship, or adoption. Children who are in foster care for less than 8 days are excluded from this indicator. For children with multiple episodes during the same 12 month period, this E:\FR\FM\23APP1.SGM 23APP1 wreier-aviles on DSK5TPTVN1PROD with PROPOSALS 22608 Federal Register / Vol. 79, No. 78 / Wednesday, April 23, 2014 / Proposed Rules measure will only evaluate the first episode within the period. This indicator is calculated using data from AFCARS. Justification for Inclusion: This indicator provides a focus for the child welfare agency’s responsibility to reunify or place children in safe and permanent homes as soon as possible after removal. Relevant Issues and Discussion: During CFSR Round 2, the Children’s Bureau included a similar measure as part of one of the permanency composites. That measure assessed the percent of all children entering foster care for the first time in a 6-month period that left foster care to reunification (or living with a relative) within 12 months of entering foster care. For CFSR Round 3, the indicator will stand alone and not be combined into a composite measure. The indicator is also more expansive than the prior to include: (1) All permanency outcomes, including guardianship or adoption; (2) all children entering foster care rather than first removals only; and (3) children entering foster care over the course of 12 months rather than 6 months. The Children’s Bureau made this change in response to suggestions from stakeholders with regard to the indicators used for the last round of reviews that we are now able to address. The indicator’s expanded set of permanency outcomes recognizes that all forms of permanency represent equally successful outcomes for children. Although all permanency outcomes are included within this one indicator, states will still be able to analyze their data to determine which types of permanency they are achieving for children. The indicator’s expanded population recognizes the Children’s Bureau’s desire to measure performance for all children entering foster care rather than first removals only. The expansion to 12 months, as opposed to 6 months in the prior indicator will yield more stable estimates of performance. A 12 month period is important for this indicator as this cohort will also serve as the basis for the denominator in the re-entry into foster care indicator (discussed further below). Re-entry into foster care after a discharge from foster care is a rarer event that is better captured over a longer period to accommodate variability. In addition, including a full 12 month period lessens the effect of potential seasonal differences between 6 month periods. Please see the section on program improvement plans for more VerDate Mar<15>2010 15:33 Apr 22, 2014 Jkt 232001 information on how this indicator may be used in program improvement. Proposed Permanency Performance Area 2: Permanency in 12 Months for Children in Foster Care for 2 Years or More Indicator Definition: Of all children in foster care the first day of the year who had been in foster care (in that episode) for 2 years or longer, what percent discharged to permanency within the next 12 months? The denominator is the number of children in foster care on the first day of a 12-month period who had been in foster care (in that episode) for 2 or more years, and the numerator is the number of children in the denominator who discharged to permanency within 12 months of the first day. Permanency includes discharge to parent(s), living with relatives, guardianship, or adoption. Children in foster care for less than 8 days are excluded from this indicator. This indicator is calculated using data from AFCARS. Justification for Inclusion: The Children’s Bureau is committed to maintaining focus on the key outcome of achieving permanency for all children in foster care and shortening the time to permanency. With a second indicator of permanency in 12 months specifically focused on the cohort of children who have been in foster care for 2 or more years, the Children’s Bureau can monitor the effectiveness of the state child welfare agency in ensuring that states continue to emphasize permanency for children who have been in foster care for longer periods of time, to ensure they achieve permanency during the period under review. Relevant Issues and Discussion: During CFSR Round 2, this same individual measure was evaluated as a part of a composite. There are substantial numbers of children that cannot achieve permanency in 12 months, and those children may present different challenges. Such children may have a higher likelihood of achieving permanency goals such as adoption and guardianship than those achieving permanency in the first year. The Children’s Bureau continues this measure because of a commitment to monitor performance for children and youth who were already in foster care, and hold states accountable for attaining permanency for them, as well. Proposed Permanency Performance Area 3: Re-Entry to Foster Care Indicator Definition: Of all children who entered foster care in a 12-month period who were discharged within 12 PO 00000 Frm 00013 Fmt 4702 Sfmt 4702 months of that entry to reunification, living with a relative, or guardianship, what percent re-entered foster care within 12 months of their discharge? The denominator is the number of children who entered foster care in a 12month period who discharged within 12 months to reunification, living with a relative, or guardianship, and the numerator is the number of children in the denominator who re-entered foster care within 12 months of their discharge from foster care. Children in foster care for less than 8 days are excluded from this indicator. If a child re-enters foster care multiple times within 12 months of when they left, only the first re-entry into foster care is selected. This indicator is calculated using data from AFCARS. Justification for Inclusion: Although the Children’s Bureau believes that it is important to reunify children with their families as quickly as possible, we also believe that children should be reunified when safe and appropriate and with sufficient supports in place to prevent a subsequent removal. This indicator enables the Children’s Bureau to monitor the effectiveness of programs and practice that support reunification and other permanency goals. Relevant Issues and Discussion: During CFSR Round 2, this performance area was evaluated using a similar measure as a part of a composite. That measure was derived by calculating what percent of all children discharged from foster care to reunification or living with a relative in a 12-month period re-entered foster care in less than 12 months from the date of discharge. This indicator differs from the measure used for CFSR Round 2 by limiting the children eligible for re-entry to the entry cohort. The CFSR Round 2 measure counted all children who left foster care to reunify or live with a relative, regardless of when they entered foster care. The purpose of this focus on current practice is in keeping with the rationale that new interventions may best be monitored in an entry cohort. We also expanded the denominator to include children who leave foster care for guardianship in an effort to reflect a more comprehensive definition of permanency. The Children’s Bureau made this change in response to suggestions from stakeholders with regard to the indicators used for the last round of reviews that we are now able to address. The indicator attempts to capture the rate of ‘‘permanency’’ for children who leave foster care by measuring whether children re-enter foster care. For this indicator, adoption is not included as a permanency outcome because it is not E:\FR\FM\23APP1.SGM 23APP1 Federal Register / Vol. 79, No. 78 / Wednesday, April 23, 2014 / Proposed Rules wreier-aviles on DSK5TPTVN1PROD with PROPOSALS always possible to identify children who re-enter foster care following adoption. Please see the section on program improvement plans for more information on how this indicator may be used in program improvement. Proposed Permanency Performance Area 4: Placement Stability Indicator Definition: Of all children who enter foster care in a 12-month period, what is the rate of placement moves per day of foster care? 8 The denominator is, among children who enter foster care in a 12-month period, the total number of days these children were in foster care as of the end of the 12-month period. The numerator is, among children in the denominator, the total number of placement moves during the 12-month period. The initial placement in foster care is not counted, but all subsequent moves occurring within the 12-month period are included in the calculation. Children in foster care for less than 8 days and other settings a child may be placed in, that are not considered as placement settings for AFCARS purposes such as trial home visits, are not included in this indicator. This indicator is calculated using data from AFCARS. Justification for Inclusion: Placement stability is a critical component of the permanency and well-being of children in foster care. States are responsible for ensuring that children who are removed from their homes by the state experience stability while they are in foster care. Relevant Issues and Discussion: During CFSR Round 2, this performance area was evaluated using a similar measure as a part of a composite. That measure was derived from calculating what percent of all children served in foster care during the 12-month period had two or fewer placement settings. The new indicator controls for the length of time children spend in foster care so that we can examine moves per day of foster care, rather than children as the unit of analysis, as was employed during CFSR Round 2. The measure used for CFSR Round 2 was unable to differentiate between children who moved twice from children who moved more. The new indicator counts each move to accurately capture the rate of placement moves, rather than the number of children affected. The Children’s Bureau believes that placement stability is important to the permanency and well-being of children in foster care regardless of how long 8 Rates are calculated per day of foster care. However, we will multiply the rate by 1,000 to produce larger and more meaningful numbers. VerDate Mar<15>2010 15:33 Apr 22, 2014 Jkt 232001 they have been in foster care. Even so, our analysis of AFCARS data indicates that most placement moves occur within a child’s first 12 months of foster care, which is why we plan to focus this indicator on that time period. With this refined focus, the Children’s Bureau and states can monitor the period during which placement moves are most likely to occur and the state’s most recent performance. In the CFSR Round 2 measure, placement moves were monitored over the life of the case which meant that placement instability for a child in the early years of foster care placement would affected the assessment of the state’s CFSR performance in a more recent period under review. We also believe that by confining the indicator to this period of time, we are better able to measure a state’s improvement in a subsequent 12month period. The Children’s Bureau made this change in response to suggestions from stakeholders with regard to the indicators used for the last round of reviews that we are now able to address. National Standards National standards will be established for all indicators. By measuring state performance against national standards on statewide data indicators, the Children’s Bureau can assist states in continuously monitoring their performance on child outcomes and help practitioners and administrators better understand the entirety of their child welfare systems. We propose that the national standard for each indicator be set at the national observed performance for that particular indicator. The national standards will remain constant over the entire round of review, as has been the case in prior rounds. In CFSR round 2, national standards were based on the 75th percentile (approximately) of all states’ performance, with an adjustment for sampling error. For this round, we believe that the national observed performance, which will be similar to the average performance across all states, is a more reasonable benchmark and would appropriately challenge states to improve their performance. The national standard set at the national performance level for each indicator is a benchmark that is easily communicated to and understood by stakeholders, and a reasonable goal given the reality that states still need to improve practice in multiple areas. Setting the national standard at the national performance for each indicator is rooted in strategies central to an effective performance management PO 00000 Frm 00014 Fmt 4702 Sfmt 4702 22609 system focused on continuous quality improvement. Methodology: We propose that state performance on each statewide data indicator be assessed using a multi-level (i.e., hierarchical) model that riskadjusts for select child- and state-level characteristics. Multi-level modeling is a widely accepted statistical method that enables fair evaluation of relative performance among states with different case mixes. The technique calculates how much variance in performance is due to (1) children’s individual risk factors; (2) random measurement errors (due to modest sample sizes); and (3) the state’s long-run ability to achieve the desired outcome. Risk Adjustment: The model we propose to use will incorporate some risk adjustment. By incorporating risk adjustment, the multi-level model takes into account and controls for factors that differ across the states and that can affect outcomes regardless of the quality of services the state provides. The goal of risk adjustment is to minimize differences in outcomes that are due to factors over which states have little control, such as the age of children coming into foster care. For example, for the statewide data indicator of permanency in 12 months for children entering foster care, a state may discharge 40% of its children to permanency by 12 months. Forty percent is the state’s observed performance, and is simply the number of children discharged to permanency by 12 months divided by the number of children eligible for such an exit. But this state’s risk-adjusted performance might be 45%. That the state’s riskadjusted performance is higher than its observed performance means permanency was achieved for more children than expected, given the state’s case mix and how other states, on average, performed with a similar case mix. The Children’s Bureau will finalize risk adjustment variables after receiving public comments on this document. The contractor’s recommendations to us based on feedback from the expert panel convened in 2013 support the use of risk adjustment. The Children’s Bureau’s consideration of particular risk-adjustment variables will be based initially on the research literature, recommendations based on feedback from the expert panel and expert consultants, and the availability of data. The Children’s Bureau will test proposed variables and retain only those variables that have a statistically significant relationship to the outcome for each statewide data indicator. For example, the Children’s Bureau has E:\FR\FM\23APP1.SGM 23APP1 22610 Federal Register / Vol. 79, No. 78 / Wednesday, April 23, 2014 / Proposed Rules tested for possible inclusion child’s age, sex, number of prior removals, and interactions among these with the proposed statewide data indicators. The Children’s Bureau will consider riskadjustment variables at both the level of the child (e.g., age at entry) and the state (e.g., foster care entry rate). wreier-aviles on DSK5TPTVN1PROD with PROPOSALS Assessing State Performance To assess state performance, the Children’s Bureau proposes to estimate each state’s risk-adjusted performance and the corresponding 95% interval estimate. The Children’s Bureau can be 95% confident that a state’s true performance lies somewhere between the lower and upper limit of this interval. This interval also provides a way to judge whether a state’s performance is above or below the national average in a statistically meaningful way. The Children’s Bureau plans to compare each state’s interval estimate to the national observed performance, and assign each state to one of three groups: • ‘‘No different than national performance’’ if the 95% interval estimate surrounding the state’s riskadjusted performance includes the national observed performance. • ‘‘Higher than national performance’’ if the entire 95% interval estimate surrounding the state’s riskadjusted performance is higher than the national observed performance. • ‘‘Lower than national performance’’ if the entire 95% interval estimate surrounding the state’s risk-adjusted performance is lower than the national observed performance. Whether it is desirable for a state to be higher or lower than the national performance depends on the indicator. For the two permanency measures, a higher value is more desirable; for the remaining measures, a lower value is desirable. The methodology described above is similar to that used by the Centers for Medicare & Medicaid Services to measure hospital performance as part of its Hospital Inpatient Quality Reporting program.9 The methodology is also consistent with the use of such models in education and health care to distinguish statistically high- and lowperforming schools and hospitals.10 9 Yale New Haven Health Services Corporation, & Center for Outcomes Research & Evaluation. (2013). 2013 Measures Updates and Specifications Report: Hospital-Level 30-Day Risk-Standardized Readmission Measures for Acute Myocardial Infarction, Heart Failure, and Pneumonia (Version 6.0). Retrieved from http://www.qualitynet.org/dcs/ ContentServer?c=Page&pagename=QnetPublic/ Page/QnetTier4&cid=1219069855841. 10 See Goldstein & Spiegelhalter (2007). League Tables and Their Limitations. 159(3), 385–443; VerDate Mar<15>2010 15:33 Apr 22, 2014 Jkt 232001 Data Data Profiles: The Children’s Bureau will provide data profiles of state performance to each state before the state’s CFSR on the statewide data indicators and other contextual data available from AFCARS and NCANDS. This data profile will assist the state to develop its statewide assessment. In addition, the Children’s Bureau will provide data profiles semi-annually to assist states in measuring progress toward the goals identified in the program improvement plan. Data Quality: Setting national standards and measuring state performance on statewide data indicators for CFSR purposes relies upon the states submitting high-quality data to AFCARS and NCANDS. The Children’s Bureau has provided states with consultation and technical assistance before and throughout both rounds of reviews to address data quality issues. Additionally, the Children’s Bureau has provided states with tools for AFCARS and NCANDS that allow the agency to examine its data for accuracy and encourages states to incorporate these in their ongoing quality assurance process to review data. During the first two rounds of the CFSR, there have been occasions in which the Children’s Bureau cannot use a state’s data in aggregate calculations of the national standard. In isolated circumstances, these data quality issues have been significant enough to prevent us from relying on the state’s data as an accurate assessment of its performance on a statewide data indicator. The Children’s Bureau would like to be clear about the level of data quality issues that prevent state data from being used for CFSR purposes as described below. Data Quality: Excluding States From National Standards or State Performance We analyzed every data element from AFCARS and NCANDS that is relevant to each statewide data indicator (as listed in Attachment C) and performed data quality checks across files both over time as well as between files.11 Examples of these checks included looking for the presence of the same child identifier in the AFCARS and NCANDS file and reviewing for consistent reporting of a child in Normand & Shahian (2007). Statistical and Clinical Aspects of Hospital Outcomes Profiling. Statistical Science, 22(2), 206–226. 11 This data quality analysis is specific to indicators proposed for CFSR Round 3. It is a separate analysis from the standard data quality checks or validation that are performed when states submit their AFCARS or NCANDS data. PO 00000 Frm 00015 Fmt 4702 Sfmt 4702 AFCARS from the time the child entered foster through discharge and with an associated reason for discharge. This analysis revealed the scope of data quality issues present in current AFCARS and NCANDS submissions. Based on this analysis, we developed thresholds to identify data quality concerns and either accept or exclude the files when calculating national standards and state performance. For those data quality issues that are contained to one data file submission, we will consider a threshold of 5%. In other words, any state that has more than 5% of data missing or invalid 12 will be excluded from the model used to calculate the national standard (i.e., the national observed performance) and estimate states’ risk-adjusted performance. For cross-file checks, we are setting a higher threshold for exclusion. For the maltreatment in foster care measure, a state will be excluded from the national standards calculation and performance estimate if more than 10% of NCANDS victims are missing an AFCARS identifier. For the permanency indicators, a state will be excluded if it has more than 10% of dropped cases across two six month file submissions. Data Quality: Case-Level Exclusions For those states that do not exceed the data quality thresholds but still have identified data quality problems, we will include the state in national standards calculations and measure state performance but we will exclude child-level records with missing or invalid data on elements needed to determine the child’s outcome and perform the risk-adjustment. For example, if the risk-adjustment for an indicator includes age at entry, a child whose age at entry cannot be determined (due to a missing date of birth) will not be include in the analysis. We believe this exclusion will result in more accurate estimates of performance for those states with minor data quality issues. For each indicator, the Children’s Bureau will provide each state with a list of records that were excluded from the analyses. Program Improvement Plans States that fall below the national standard on any given indicator will be required to include that indicator in a program improvement plan. Regardless of which indicators a state is required to include in its program improvement plan, the Children’s Bureau will provide each state with a data profile that 12 For example, the date of birth is greater than the date of latest removal. E:\FR\FM\23APP1.SGM 23APP1 wreier-aviles on DSK5TPTVN1PROD with PROPOSALS Federal Register / Vol. 79, No. 78 / Wednesday, April 23, 2014 / Proposed Rules includes information on the state’s performance on all of the statewide data indicators. Companion Measures: For two of the statewide data indicators, permanency in 12 months for children entering foster care and re-entry to foster care, the Children’s Bureau proposes to consider performance for program improvement purposes in concert with the other indicator. This means that if a state has a program improvement plan that includes improving on the indicator permanency in 12 months for children entering foster care, the Children’s Bureau’s determination of whether the state has improved successfully will take into consideration its performance on the re-entry to foster care indicator as a companion measure. Specifically, the state must stay above a threshold for the companion re-entry to foster care indicator as well as achieve its goal on the permanency in 12 months for children entering foster care indicator, to successfully complete the program improvement plan. The reverse is also true. For details about threshold calculations, please see the section below. If a state must improve on the reentry to foster care indicator in its program improvement plan, it must also include the permanency in 12 months for children entering foster care indicator as a companion measure. Although the Children’s Bureau believes that it is important to reunify children with their families as quickly as possible, we also believe that children should be reunified when safe and appropriate and sufficient supports are in place to prevent a subsequent removal. The recommendations to us based on the expert panel convened in 2013 also support the use of companion measures in program improvement. Methods for Setting State Performance Baselines, Goals, and Thresholds: The key components for setting performance goals and monitoring progress over the course of a program improvement plan involve calculating baselines, goals, and thresholds. The Children’s Bureau methodology for specifying state improvement goals is statistically grounded, producing goals that are programmatically challenging, yet attainable. In addition, goals and thresholds should reflect each state’s own performance history and demonstrated capacity for change. In CFSR Round 2, the percent of improvement required for each state was the same for all states, and was applied to state-specific baselines. While this standardization was easy to communicate how it applied to each state, it also meant that high-performing VerDate Mar<15>2010 15:33 Apr 22, 2014 Jkt 232001 states were asked to make larger absolute improvements than lowerperforming states. This approach did not take into account the variability in performance that states have shown over time. For CFSR Round 3, the Children’s Bureau plans to set improvement goals relative to each state’s past performance. Setting Baselines: The Children’s Bureau plans to set the baseline for each statewide data indicator included in a program improvement plan at the state’s observed performance on that indicator for the most recent year of data available before the beginning of program improvement plan implementation. We also considered using the state’s threeyear average in this calculation, but that approach occasionally produced goals that fell below the state’s performance in the most recent year. Consistent with CFSR Round 2, we will use the most recent year as the baseline, because it represents a reasonable estimate of the state’s current performance. Setting Goals and Thresholds: We plan to set performance goals and thresholds will be based on the variability in the state’s observed performance in the three most recent years of data. There will be improvement factors, as in CFSR Round 2, but these will be driven by the variability in performance that the particular state has shown in the last three years. We will apply the improvement factors to the baseline to produce the concrete performance goal. The state’s amount of improvement required for a program improvement plan will be more than what is likely, in a statistical sense, to occur by chance alone. Conversely, we plan to set thresholds as the inverse of goals, which will identify a point by which a state is demonstrating a performance decline for companion measures that is more statistically than what might be expected by chance. We will use a technique called bootstrapping to develop goals and thresholds. The method calls for the Children’s Bureau to repeatedly sample a state’s past three years of performance estimates to construct a larger sample, and from that the calculation of a grand mean and standard deviation. The grand mean reflects that state’s ‘‘average’’ performance and the standard deviation reflects how much normal fluctuation in performance the Children’s Bureau might expect for that state, given its past performance. Then the standard deviation is used to calculate an estimate that would represent a level of change above and beyond the typical fluctuation that would otherwise be expected. The Children’s Bureau will PO 00000 Frm 00016 Fmt 4702 Sfmt 4702 22611 set the magnitude at four standard deviations from the grand mean. At that level we can say with confidence that— if we were to randomly estimate a state’s performance on the indicator (using their past performance), and did so 100 times—we would expect to see performance at this level less than 6% of the time (or fewer than 6 times out of 100). Six times out of 100 is rather rare, which is why we can treat it as representing a statistically meaningful change has occurred in the program.13 To determine exactly how much a state will need to improve, we must first calculate an improvement factor, which is the percentage difference between the grand mean and four standard deviations above the grand mean. We then apply that to the baseline, which is the observed performance in the most recent year. To demonstrate a sample calculation: A state may have a grand mean of 50%, a grand mean plus four standard deviations = 52%, and a year 3 value of 51. This will give us an improvement factor of 52/50 = 1.04. If that is applied to the baseline of 51%, the program improvement plan goal will be 51% × 1.04 = 53.04%. We will use a comparable technique to set thresholds for companion measures, subtracting rather than adding four standard deviations to the grand mean (when higher performance on an indicator is better), which can be used to identify a state’s decline in performance. To provide an example, if a goal was calculated to be three percentage points higher than the baseline percent, the threshold would be three percentage points below it. Thresholds are only relevant to companion measures. By design, states with less variation in performance from year to year have more modest goals, while those showing greater variation have more aggressive goals. Overall, we believe that the goals are reflective of each state’s own prior experience and performance levels, with goals that are achievable and substantively meaningful. We acknowledge that a few states with the 13 These guidelines are based on a theorem known as Chebyshev’s inequality. When all you have is an overall mean and standard deviation (which is what we have for each state’s performance), the theorem guarantees that a certain percent of values will be k standard deviations away from the mean. Specifically, at least 75% of the values will be within two standard deviations of the mean, at least 89% within three standard deviations, at least 94% within four standard deviations, at least 96% within five standard deviations, and at least 97% within six standard deviations (Chong et al. (2012). Improving Generalization Performance in Co-Evolutionary Learning. IEEE, vol. 16, no. 1, 70–85; Sheppard (2011). Environmental Study-Soil Sample Analysis for the Department of Ecology at Hanford). E:\FR\FM\23APP1.SGM 23APP1 22612 Federal Register / Vol. 79, No. 78 / Wednesday, April 23, 2014 / Proposed Rules lowest variation in performance for each indicator are assigned very modest goals, while a few states with the highest variation in performance are assigned very aggressive goals. To address these problems at the extreme ends, we propose to establish minimum and maximum improvement factors. Specifically, a floor will be set at the top of the bottom fifth, and the bottom of the top fifth, ordered by the size of the improvement factor. While the impact of this rule varies somewhat from indicator to indicator, overall we believe it provides a consistent basis for Category Measure title producing goals that are achievable and substantively meaningful. The inverse would be done for the thresholds. The contractor’s recommendations to us based on the feedback from the expert panel convened in 2013 support the setting of maximum and minimum thresholds for improvement goals at the level of performance of top and bottom quintiles. Process for Final Indicators We are interested in comments on all aspects of the statewide data indicators proposed and the methods to calculate national standards and program improvement. After considering the feedback to this docuemnt, we plan to publish a final list of indicators and methods that will be used in the CFSRs along with the actual national standards. (Authority: 42 U.S.C. 1320a–1a; 45 CFR 1355.31–37.) Mark Greenberg, Acting Commissioner, Administration on Children, Youth and Families. Attachment A: Proposed Statewide Data Indicators Measure description Denominator Numerator Exclusions Notes —If a state provides incident dates, records with an incident date occurring before the date of removal will be excluded. —Children in foster care <8 days. —Any report that occurs within the first 7 days of removal. None ...................... Cases are matched across AFCARS and NCANDS using AFCARS ID. Maltreatment in foster care. Of all children in foster care during a 12-month period, what is the rate of victimization per day of foster care? (From AFCARS) Among children in foster care during a 12month period, total number of days these children were in foster care as of the end of the 12month period a. (From NCANDS) Among children in the denominator, total number of substantiated or indicated reports of maltreatment (by any perpetrator) during the 12month period b. Safety ....... Re-report of maltreatment. (From NCANDS) Number of children in the denominator that had another screened-in report within 12 months of their initial report. Permanency in 12 months for children entering foster care. Of all children with a screened-in report of alleged maltreatment in a 12-month period (regardless of disposition), what percent had another screened-in report within 12 months of their initial report? Of all children who enter foster care in a 12-month period, what percent discharged to permanency within 12 months of entering foster care? c (From NCANDS) Number of children with at least one screened-in report of alleged maltreatment in a 12-month period. Permanency. (From AFCARS) Number of children who enter foster care in a 12-month period. Permanency. wreier-aviles on DSK5TPTVN1PROD with PROPOSALS Safety ....... Permanency in 12 months for children in foster care 2 years or more. (From AFCARS) Number of children in the denominator who discharged to permanency within 12 months of entering foster care or by the time they reached 18. (From AFCARS) Number of children in the denominator who discharged to permanency within 12 months of the 1st day or by the time they reached 18. VerDate Mar<15>2010 15:33 Apr 22, 2014 Of all children in foster care on the first day of a 12-month period, who had been in foster care (in that episode) for 2 or more years, what percent discharged to permanency within 12 months of the first day? Jkt 232001 PO 00000 (From AFCARS) Number of children in foster care on the first day of a 12month period who had been in foster care (in that episode) for 2 or more years. Frm 00017 Fmt 4702 Sfmt 4702 —Children in foster care <8 days. —Children who enter foster care at age 18 or more. —Children age 18 or more on the first day of the year. E:\FR\FM\23APP1.SGM 23APP1 Reports are included regardless of the type of disposition, so this indicator includes both victims and non-victims. Youth who turn 18 while in foster care who were included in the denominator will not be counted as having achieved permanency, regardless of discharge reason. Youth who turn 18 while in foster care who were included in the denominator will not be counted as having achieved permanency, regardless of discharge reason. 22613 Federal Register / Vol. 79, No. 78 / Wednesday, April 23, 2014 / Proposed Rules Category Measure description Measure title Permanency. Re-entry to foster care. Permanency. Placement Stability Denominator Of all children who enter foster care in a 12-month period who discharged within 12 months to reunification, live with relative, or guardianship, what percent reentered foster care within 12 months of their discharge? a Of all children who enter foster care in a 12-month period, what is the rate of placement moves per day of foster care? (From AFCARS) Number of children who enter foster care in a 12-month period who discharged within 12 months to reunification, live with relative, or guardianship. (From AFCARS) Number of children in the denominator who re-enter foster care within 12 months of their discharge. (From AFCARS) Among children who enter foster care in a 12month period, total number of days these children were in foster care as of the end of the 12month period d. Numerator Exclusions Notes —Children in foster care < 8 days. —Children who enter or exit foster care at age 18 or more. If a child has multiple re-entries to foster care within 12 months of their discharge, only his first reentry is selected. (From AFCARS) —Children in foster Among children care < 8 days. in the denomi—Children who nator, total numenter foster care ber of placement at age 18 or moves during the more. 12-month period e. The initial removal from home (and into foster care) is not counted as a placement move. a For example, if during the 12-month period there were two children in foster care, one child for 10 days (1st episode), the same child for 40 days (2nd episode), and the other child for 100 days (his only episode), the denominator would = 150 days (10+40+100). b For example, if during the 12-month period there were two children in foster care, and one child with 3 substantiated or indicated reports and the other with 1 such report, the numerator would = 4 reports (3+1). c If a child has multiple entries during the year, only his or her first entry is selected. d For example, if during the 12-month period two children entered foster care, one child for 10 days and the other child for 100 days, the denominator would be 110 days (10+100). e For example, if during the 12-month period two children entered foster care, and one child had 3 moves and the other had 1 move, the numerator would = 4 moves (3+1). Attachment B: Comparison of Data Measures—CFSR Round 2 and Round 3 Category Measure title Proposed CFSR round 3 indicator Comparable CFSR round 2 measure How and why it’s changed In the CFSR 2 measure, counts of children not maltreated in foster care are derived by subtracting the NCANDS count of children maltreated by foster care providers from the total count of all children placed in foster care, as reported in AFCARS. Because of improved reporting by states, we now link AFCARS and NCANDS data using the child ID and determine if maltreatment occurred during a foster care episode, improving accuracy on the indicator. This also allows us to expand the measure to include all types of perpetrators (including, for example, parents) under the assumption that states should be held accountable for keeping children safe from harm while in the care of the state, no matter who the perpetrator is. We are expanding the measure to count all children with screened-in reports of alleged maltreatment, because research suggests children with prior reports are at greater risk. In addition, by limiting only to victims, we could face measurement challenges as states implement Differential Response during a monitoring cycle, which could have an impact on numbers of substantiations. We also propose using a full 12-month period rather than only 6 months to capture the denominator, to create more stable estimates. Maltreatment in foster care. Of all children in foster care during a 12month period, what is the rate of victimization per day a of foster care? Of all children in foster care during the reporting period, what percent were not victims of substantiated or indicated maltreatment by a foster parent or facility staff member? Safety ....... wreier-aviles on DSK5TPTVN1PROD with PROPOSALS Safety ....... Re-report of maltreatment. Of all children with a screened-in report of alleged maltreatment in a 12-month period, what percent had another screened-in report within 12 months of their initial report? Of all children who were victims of substantiated or indicated maltreatment allegation during the first 6 months of the reporting period, what percent were not victims of another substantiated or indicated maltreatment allegation within a 6-month period? VerDate Mar<15>2010 15:33 Apr 22, 2014 Jkt 232001 PO 00000 Frm 00018 Fmt 4702 Sfmt 4702 E:\FR\FM\23APP1.SGM 23APP1 22614 Category Federal Register / Vol. 79, No. 78 / Wednesday, April 23, 2014 / Proposed Rules Measure title Proposed CFSR round 3 indicator Comparable CFSR round 2 measure How and why it’s changed Composite 1.3: Of all children entering foster care for the first time in a 6-month period, what percent discharged to reunification (or live with relative) within 12 months of entering foster care or by the time they reached 18? Composite 3.1: Of all children in foster care on the first day of a 12-month period who had been in foster care (in that episode) for 2 or more years, what percent discharged to permanency within 12 months of the first day or by the time they reached 18? Composite 1.4: Of all children discharged from foster care to reunification or live with a relative in a 12month period, what percent re-entered foster care in less than 12 months from the date of discharge? We now count all types of permanency (reunification, live with relative, adoption or guardianship) as having ‘met’ the indicator. We also expanded the measure to include all children who entered foster care that year; not just those on their first removal episode. We also expanded the window of time for the entry cohort to a full year instead of 6 months; this will yield more stable estimates. Permanency in 12 months for children entering foster care. Of all children who enter foster care in a 12month period, what percent discharged to permanency within 12 months of entering foster care? Permanency. Permanency in 12 months for children in foster care for 2 years or more. Of all children in foster care on the first day of a 12-month period who had been in foster care (in that episode) for 2 or more years, what percent discharged to permanency within 12 months of the first day? Permanency. Re-entry in 12 months ... Of all children who enter foster care in a 12month period and discharged within 12 months to reunification, live with relative, or guardianship, what percent re-entered foster care within 12 months of their date of discharge? Permanency. wreier-aviles on DSK5TPTVN1PROD with PROPOSALS Permanency. Placement stability ........ Of all children who enter foster care in a 12month period, what is the rate of placement moves per day b of foster care? Composite 4.1: Of all children served in foster care during the 12month period, what percent had two or fewer placement settings? Same measure; no change. The difference is that it is now evaluated on its own, rather than as just one part of a composite measure. By including the Entry Cohort Permanency indicator (listed above) as well as the Legacy Cohort Permanency indicator, we hold states accountable not only for children in their first year of foster care, but also those children/youth who have been in foster care for long periods of time. The new indicator is limited to those children who entered foster care during the year, whereas the CFSR Round 2 measure counted all children who discharged to reunification or live with relative, regardless of when they entered foster care. The purpose of this focus is in keeping with the rationale that new interventions may best be monitored in an entry cohort. This indicator will also be used as a companion measure with permanency in 12 months, to ensure that states working to improve permanency rates in their entry cohort do not see worsening performance on rates of re-entry to foster care. We also expanded the denominator to allow discharges to guardianship, in an effort to capture more discharges to permanency. Exits to adoption are not included because they cannot be tracked reliably, as some states issue new child identifiers if a child who was previously adopted enters foster care. The proposed indicator controls for length of time in foster care, so we are looking at moves per day of foster care, rather than children as the unit of analysis. The rationale for using an entry cohort rather than all children served is that our analysis shows children entering foster care tend to move much more than those children/youth in foster care for longer periods of time, whose placements may have stabilized. In CFSR Round 2 measure, moves that took place prior to the monitoring period were counted. Now we only count those moves that occur during the monitoring period. The initial placement is not counted. The CFSR Round 2 measure treated children who moved 2 times in an episode the same as children who moved 15 times; both were a failure to meet the measure. The new indicator counts each move, so it continues to hold states accountable for those children/youth who have already moved several times. a The rate may be expressed per 100,000 days because it is such a rare event. Using this metric gives us numbers greater than zero, which are easier to communicate. b The rate is expressed per 1,000 days to convert the rate to a metric that gives us numbers greater than zero. VerDate Mar<15>2010 15:33 Apr 22, 2014 Jkt 232001 PO 00000 Frm 00019 Fmt 4702 Sfmt 4702 E:\FR\FM\23APP1.SGM 23APP1 22615 Federal Register / Vol. 79, No. 78 / Wednesday, April 23, 2014 / Proposed Rules Attachment C: Data Elements Required for Statewide Data Indicators For instruction regarding AFCARS data elements, refer to http:// www.acf.hhs.gov/programs/cb/resource/ afcars-tb1. For instruction with regard to NCANDS data elements, refer to http://www.ndacan.cornell.edu/ datasets/pdfs_user_guides/178NCANDS-child2012v1-User-Guide-andCodebook.pdf. Permanency by 12 months Primary data elements required for calculation Re-entry by 12 months ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ NA NA NA ✓ ✓ ✓ NA NA ✓ NA NA NA ✓ ✓ NA NA NA ✓ ✓ NA NA ✓ ✓ NA NA NA NA NA NA ✓ ✓ NA ✓ NA NA ✓ NA NA NA NA ✓ NA NA NA NA ✓ NA NA NA NA ✓ NA NA NA NA NA NA NA NA NA NA NA NA ✓ ✓ ✓ AFCARS FC Element #1: a Title IV–E Agency .................... AFCARS FC Element #4: Record Number ......................... AFCARS FC Element #21: Date of Latest Removal ........... AFCARS FC Element #23: Date of Placement in Current Foster Care Setting .......................................................... AFCARS FC Element #24: Number of Placement Settings during this Removal Episode ........................................... AFCARS FC Element #56: Date of Discharge from FC ..... AFCARS FC Element #58: Reason for Discharge .............. NCANDS CF Element #4: Child ID ..................................... NCANDS CF Element #6: Report Date ............................... NCANDS CF Element #27: Child Maltreatment 1—Disposition Level b ................................................................. NCANDS CF Element #29: Child Maltreatment 2—Disposition Level ................................................................... NCANDS CF Element #31: Child Maltreatment 3—Disposition Level ................................................................... NCANDS CF Element #33: Child Maltreatment 4—Disposition Level ................................................................... NCANDS CF Element #34: Maltreatment death ................. NCANDS CF Element #145: AFCARS ID ........................... Placement stability Re-report of maltreatment Maltreatment in foster care a The elements are numbered by their position in the flat ASCII files submitted by states to these reporting systems. These numbering schema are specific to the files utilized by ACYF. Files obtained through the National Data Archive on Child Abuse and Neglect (NDACAN) may have a slightly different order. b Definition of ‘victim’ includes all children with a disposition level (for any of up to four maltreatments per child) of: a) Substantiated, or b) Indicated. These do not propose including differential response victims. Victims also include children who died as a result of maltreatment. Additional data elements required for risk-adjusted Analysis c Permanency by 12 months Re-entry by 12 months ✓ ✓ ✓ NA NA ✓ ✓ ✓ ✓ NA NA ✓ AFCARS FC Element #6: Child’s Date of Birth .................. AFCARS FC Element #7: Child Sex ................................... AFCARS FC Element #19: Total # of Removals ................ NCANDS CF Element #14: Child Age ................................ NCANDS CF Element #17: Child Sex ................................. US Census Bureau: Child Population, by State .................. Placement stability ✓ ✓ ✓ NA NA NA Re-report of maltreatment NA NA NA ✓ ✓ ✓ Maltreatment in foster care ✓ ✓ ✓ NA NA ✓ c In addition to those data elements used for risk adjustment, a few more are used to make necessary adjustments to outcomes; for example, we use the child’s current placement setting to determine if he or she is in Trial Home Visit before Reunification and, if so, time in foster care is adjusted consistent with adjustments for trial home visits used in CFSR Round 2. If a state provides NCANDS CF Element #146, Incident Date, an adjustment will be made to the maltreatment in foster care indicator to improve accuracy. BILLING CODE 4184–25–P and National Aeronautics and Space Administration (NASA). ACTION: Proposed rule. DEPARTMENT OF DEFENSE SUMMARY: [FR Doc. 2014–09001 Filed 4–22–14; 8:45 am] GENERAL SERVICES ADMINISTRATION wreier-aviles on DSK5TPTVN1PROD with PROPOSALS NATIONAL AERONAUTICS AND SPACE ADMINISTRATION 48 CFR Parts 4, 14, 15, and 52 [FAR Case 2014–001; Docket No. 2014– 0001; Sequence No. 1] RIN 9000–AM78 Federal Acquisition Regulation; Incorporating Section K in Contracts Department of Defense (DoD), General Services Administration (GSA), AGENCIES: VerDate Mar<15>2010 15:33 Apr 22, 2014 Jkt 232001 DoD, GSA, and NASA are proposing to amend the Federal Acquisition Regulation (FAR) to standardize the incorporation by reference of representations and certifications in contracts. DATES: Interested parties should submit written comments to the Regulatory Secretariat at one of the addressees shown below on or before June 23, 2014 to be considered in the formation of the final rule. ADDRESSES: Submit comments in response to FAR Case 2014–001 by any of the following methods: • Regulations.gov: http:// www.regulations.gov. Submit comments via the Federal eRulemaking portal by PO 00000 Frm 00020 Fmt 4702 Sfmt 4702 searching for ‘‘FAR Case 2014–001’’. Select the link ‘‘Comment Now’’ that corresponds with ‘‘FAR Case 2014– 001.’’ Follow the instructions provided at the ‘‘Comment Now’’ screen. Please include your name, company name (if any), and ‘‘FAR Case 2014–001’’ on your attached document. • Fax: 202–501–4067. • Mail: General Services Administration, Regulatory Secretariat (MVCB), ATTN: Hada Flowers, 1800 F Street NW., 2nd Floor, Washington, DC 20405. Instructions: Please submit comments only and cite FAR Case 2014–001, in all correspondence related to this case. All comments received will be posted without change to http:// www.regulations.gov, including any personal and/or business confidential information provided. E:\FR\FM\23APP1.SGM 23APP1

Agencies

[Federal Register Volume 79, Number 78 (Wednesday, April 23, 2014)]
[Proposed Rules]
[Pages 22604-22615]
From the Federal Register Online via the Government Printing Office [www.gpo.gov]
[FR Doc No: 2014-09001]


=======================================================================
-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Administration for Children and Families

45 CFR Part 1355


Statewide Data Indicators and National Standards for Child and 
Family Services Reviews

AGENCY: Children's Bureau (CB), Administration on Children, Youth and 
Families (ACYF), Administration for Children and Families (ACF), 
Department of Health and Human Services (HHS).

ACTION: Notice of Statewide Data Indicators and National Standards for 
Child and Family Services Reviews.

-----------------------------------------------------------------------

SUMMARY: The Children's Bureau reviews a state's substantial conformity 
with titles IV-B and IV-E of the Social Security Act through the Child 
and Family Services Reviews (CFSRs). Statewide data indicators are used 
to inform the Children's Bureau's determination of a state's 
substantial conformity relative to certain safety and permanency 
outcomes. This document advises the public of the Children's Bureau's 
plan to replace the statewide data indicators and the methods for 
calculating associated national standards on those indicators. We 
invite the public to comment on these indicators and methods before 
their use in CFSRs scheduled for Federal Fiscal Years (FFY) 2015 
through FY 2018.

DATES: Written comments must be submitted to the office listed in the 
ADDRESSES section below on or before May 23, 2014.

ADDRESSES: Interested persons may submit written comments by any of the 
following methods:
     Federal eRulemaking Portal: http://www.regulations.gov. 
Follow the instructions for submitting comments.

[[Page 22605]]

     Mail or Courier Delivery: Miranda Lynch Thomas, Children's 
Bureau, Administration on Children, Youth and Families, Administration 
for Children and Families, 1250 Maryland Avenue SW., 8th Floor, 
Washington, DC 20024.
    Instructions: If you choose to use an express, overnight, or other 
special delivery method, ensure that delivery may be made at the 
address listed under the ADDRESSES section. We urge interested parties 
to submit comments electronically to ensure that they are received in a 
timely manner. All comments received will be posted without change to 
http://www.regulations.gov. This will include any personal information 
provided.

FOR FURTHER INFORMATION CONTACT: Miranda Lynch Thomas, Children's 
Bureau, 1250 Maryland Ave. SW., 8th Floor, Washington, DC 20024, (202) 
205-8138.

SUPPLEMENTARY INFORMATION:

Background

    The Children's Bureau implemented the CFSRs in 2001 in response to 
a mandate in the Social Security Amendments of 1994 for the Department 
of Health and Human Services to issue regulations for the review of 
state child and family services programs under titles IV-B and IV-E of 
the Social Security Act (see section 1123A of the Social Security Act). 
The reviews are required for the Children's Bureau to determine whether 
such programs are in substantial conformity with title IV-B and IV-E 
plan requirements, implementing regulations, and relevant title IV-B 
and IV-E plans. The review process, as regulated at 45 CFR 1355.31-37, 
grew out of extensive consultation with interested groups, individuals, 
and experts in the field of child welfare and related areas.
    The Children's Bureau conducted the first round of CFSRs from 2001 
through 2004 and the second round from 2007 through 2010. The third 
round is scheduled to begin in FFY 2015. Information about the 
initiation of this latest round can be found in CFSR Technical Bulletin 
7 issued in March 2014 (see http://www.acf.hhs.gov/programs/cb/resource/cfsr-technical-bulletin-7).
    The CFSRs enable the Children's Bureau to: (1) Ensure conformity 
with federal child welfare requirements; (2) determine what is actually 
happening to children and families as they are engaged in child welfare 
services; and (3) assist states to enhance their capacity to help 
children and families achieve positive outcomes. The Children's Bureau 
conducts the reviews in partnership with state child welfare agency 
staff and other stakeholders involved in the provision of child welfare 
services. We have structured the reviews to help states identify 
strengths as well as areas needing improvement within their agencies 
and programs.
    The CFSR assesses state performance on seven outcomes and seven 
systemic factors. The seven outcomes focus on key items measuring 
safety, permanency, and well-being. The seven systemic factors focus on 
key state plan requirements of titles IV-B and IV-E that provide a 
foundation for child outcomes. States that the Children's Bureau 
determines have not achieved substantial conformity in all the areas 
assessed in the review are required to develop and implement a program 
improvement plan within two years addressing the areas of 
nonconformity. The Children's Bureau supports the states with technical 
assistance and monitors implementation of their program improvement 
plans. States that are unable to complete their plans successfully have 
some of their federal child welfare funds withheld.
    Most relevant to this document is the element of the reviews that 
provides for the Children's Bureau to determine whether the state is in 
substantial conformity with certain child outcomes based on national 
standards we set for state performance on statewide data indicators. 
The regulations at 45 CFR 1355.34(b)(4) and (5) authorize us to add, 
amend, or suspend any of the statewide data indicators when 
appropriate, and to adjust the national standards when appropriate. 
Statewide data indicators are aggregate measures and we calculate them 
using administrative data available from a state's submissions to the 
Adoption and Foster Care Analysis and Reporting System (AFCARS),\1\ the 
National Child Abuse and Neglect Data System (NCANDS),\2\ or a 
Children's Bureau-approved alternate source for safety-related data. If 
a state is determined not to be in substantial conformity with a 
related outcome due to its performance compared to the national 
standard for an indicator, the state will include that indicator in a 
program improvement plan. The Children's Bureau establishes performance 
goals for each indicator included in a program improvement plan, based 
on the state's prior performance, that the state is expected to reach 
by the end of the program improvement plan period. The goal to be 
achieved is relative to the state's baseline performance at the 
beginning of the program improvement plan period.
---------------------------------------------------------------------------

    \1\ AFCARS collects case-level information from state and Tribal 
title IV-E agencies on all children in foster care and those who 
have been adopted with title IV-E agency involvement. Title IV-E 
agencies must submit AFCARS data to the Children's Bureau twice a 
year.
    \2\ NCANDS collects child-level information on every child who 
receives a response from a child protective services agency due to 
an allegation of abuse or neglect. States report this data to the 
Children's Bureau voluntarily. In FFY 2013, all 50 states, the 
District of Columbia, and Puerto Rico submitted NCANDS data.
---------------------------------------------------------------------------

    The Children's Bureau views the CFSR as a dynamic process and has 
made ongoing improvements in the process to best meet state and federal 
needs. Most recently, we solicited feedback from the public (see 76 FR 
18677, published April 5, 2011) about how they would envision a federal 
review process that meets the statutory requirements in section 1123A 
of the Social Security Act and holds child welfare agencies accountable 
for achieving positive outcomes for children and families and 
continuously improving the quality of their systems for doing so. In 
addition, we hired a consultant that specializes in child welfare 
measurement to work with Children's Bureau data specialists. In 2013 we 
also tasked a contractor to the Children's Bureau to convene a panel of 
child welfare administrators and data measurement experts to develop 
recommendations and feedback about specific aspects of the review 
process, including the statewide data indicators, national standards, 
and program improvement. The information from these experts along with 
public comments has shaped our plan for replacing the statewide data 
indicators that will be used in the CFSRs.

Existing Statewide Data Indicators and Composite Measures and Planned 
Improvements To Address Feedback

    For CFSR Round 2, the Children's Bureau developed six statewide 
data indicators and measures: two indicators related to safety and four 
composite measures related to permanency. The two safety-related 
statewide data indicators focused on recurrence of maltreatment and 
maltreatment of children in foster care and were used to inform an 
assessment of the state's substantial conformity with the safety 
outcome that children are, first and foremost, protected from abuse and 
neglect. The four permanency-related data composites were used to 
inform the assessment of a state's substantial conformity with the 
permanency outcome that children have permanency and stability in their 
living situations. The four permanency composites used during CFSR 
Round 2 were related to measures of timeliness and permanency

[[Page 22606]]

of reunification; timeliness of adoptions, permanency for children in 
foster care for extended time periods, and placement stability.\3\ The 
Children's Bureau established national standards for each of the six 
data indicators and composites. The Children's Bureau thoroughly 
considered the alternatives available to us at the time and had a clear 
rationale for supporting the statewide data indicators, composites and 
methods chosen. However, we believe there is reason to make further 
improvements now given the additional concerns raised by stakeholders 
and the Children's Bureau that we summarize below and throughout this 
document as we describe the proposed new indicators.
---------------------------------------------------------------------------

    \3\ More detailed information on the CFSR Round 2 measures can 
be found on the Children's Bureau Web site at http://www.acf.hhs.gov/sites/default/files/cb/data_indicators_for_the_second_round_of_cfsrs.pdf.
---------------------------------------------------------------------------

    Cohorts Used for Statewide Data Indicators: The Children's Bureau 
is planning to calculate several statewide data indicators using entry 
cohorts to address concerns about the measurement techniques that 
relied on multiple exit cohorts in prior CFSR rounds. Some stakeholders 
noted that relying on exit cohorts and cross-sectional (also referred 
to as point-in-time) cohorts, rather than prospective entry cohorts, 
can statistically distort system performance outcomes because they 
represent biased sub-samples of all children who are served.\4\ For 
example, looking only at children who exit foster care in a given year 
ignores those children who did not leave; it is therefore not 
representative of all children in foster care who would be affected by 
state improvement efforts. Looking only at those children in foster 
care at the end of the year biases the sample to include more children 
with long lengths of stay in foster care. Entry cohorts, which include 
all children entering foster care in a given year, avoid these problems 
and provide a more complete assessment of overall system performance 
and recent practice trends.
---------------------------------------------------------------------------

    \4\ See Testa, M., Poertner, J. Koh, E. (2008). Can AFCARS be 
rescued? Fixing the statistical yardstick that measures state child 
welfare performance. Urbana, IL: CFRC; and Testa, M. & Poertner, J. 
(Eds.). (2010). Fostering accountability: Using evidence to guide 
and improve child welfare policy. Oxford: Oxford University Press.
---------------------------------------------------------------------------

    While entry cohorts have methodological advantages, they have 
limitations in terms of assessing state performance with regard to 
children who have been in foster care for a long time because of the 
length of time we measure for state improvements. For example, with an 
entry cohort approach, children who had already been in foster care for 
two or more years could only start being tracked in a third year. To 
address this limitation, the Children's Bureau will still use other 
cohorts in some of its indicators.
    Composites: The Children's Bureau plans to use individual 
indicators rather than composite measures for the purpose of 
establishing national standards in this round of review. We implemented 
the composite approach in CFSR Round 2 after consultation with states 
and national experts to support a more holistic approach to measuring 
state performance. The expansion to composites from the one-dimensional 
measures used in CFSR Round 1 allowed state performance on a particular 
domain to reflect broader performance, accounting for both strengths 
and weaknesses within the domain. Criticism of the composite measures 
used during CFSR Round 2 included the complexity of the composite 
scores for interpretation purposes.
    To address these concerns and clarify expectations with regard to 
national performance, the Children's Bureau proposes to measure state 
performance with simplified statewide data indicators. We propose to 
maintain some of the advantages found with the composite approach by 
implementing companion measures during the program improvement plan 
period to provide an expanded and more effective measurement of a 
domain.

New Proposed Statewide Data Indicators and Methods

    The Children's Bureau plans for the new statewide data indicators 
to measure maltreatment in foster care and re-report of maltreatment as 
a component in evaluating Safety Outcome 1: Children are, first and 
foremost, protected from abuse and neglect. We also plan for statewide 
data indicators to measure the achievement of permanency in 12 months 
for children entering foster care, permanency in 12 months for children 
in foster care for 2 years or more, re-entry to foster care, and 
placement stability. These four permanency indicators will be used as a 
component in evaluating Permanency Outcome 1: Children have permanency 
and stability in their living situations. Below is a description of 
each of the six proposed indicators including their definition, 
justification for inclusion, calculation and a discussion of relevant 
issues. Following the description of the indicators is information on 
the methods the Children's Bureau plans to use for calculating the 
national standards and our approach to measuring a state's program 
improvement on the indicators should we find that the state is not able 
to meet the national standard. We also provide additional information 
on how the Children's Bureau will share data information with states 
through profiles and data quality issues that impact these indicators 
and methods.
    Attachment A provides a summary of each planned indicator including 
numerators, denominators, exclusions, and adjustments. Attachment B 
provides a comparison of the data measures used during CFSR Round 2 
with the statewide data indicators we propose to use during Round 3. 
Attachment C provides information on the AFCARS and NCANDS data 
elements that are used to calculate the proposed indicators and 
national standards.

Statewide Data Indicators for CFSR Safety Outcome 1: Children Are, 
First and Foremost, Protected From Abuse and Neglect

Proposed Safety Performance Area 1: Maltreatment in Foster Care

    Indicator Definition: Of all children in foster care during a 12-
month period, what is the rate of victimization per day of foster care? 
\5\ The indicator includes all cases of substantiated or indicated 
maltreatment while in foster care and all days for all children in 
foster care at any point during a 12-month period. The denominator is 
all child days in foster care over a 12-month period, and the numerator 
is the number of instances of substantiated or indicated maltreatment 
among children in foster care over that same period. The definition of 
``children'' for this indicator (and all indicators) includes those 
under the age of 18. This indicator includes all maltreatment types by 
any perpetrator, which may include foster parents, facility staff 
members, parents, or others. In addition, this indicator includes all 
days for all children in foster care at any point during a 12-month 
period.
---------------------------------------------------------------------------

    \5\ Rates are calculated per day of foster care. However, we 
will multiply the rate by 100,000 to produce larger and more 
meaningful numbers.
---------------------------------------------------------------------------

    Some states provide incident dates in their NCANDS data 
submissions. If a state provides incident dates, records with an 
incident date occurring before the date of removal will be excluded. 
Children in foster care for less than 8 days and any report that occurs 
within the first 7 days of removal are excluded from this indicator. 
This indicator is calculated using data that match

[[Page 22607]]

children across AFCARS and NCANDS using the AFCARS identifier.
    Justification for Inclusion: This indicator provides a measure of 
whether the state child welfare agency is able to ensure that children 
do not experience abuse or neglect while in the state's foster care 
system. The indicator holds states accountable for keeping children 
safe from harm while under the responsibility of the state, no matter 
who perpetrates the maltreatment in foster care.
    Relevant Issues and Discussion: During CFSR Round 2, the Children's 
Bureau had a safety data indicator related to maltreatment in foster 
care. For that indicator, the counts of children not maltreated in 
foster care were derived by subtracting the NCANDS count of children 
maltreated by foster care providers from the total count of all 
children in foster care on the last day of the year, as reported in 
AFCARS. Because of improved reporting by states, we can now link AFCARS 
and NCANDS data using the child identifier and determine whether 
maltreatment occurred during a foster care episode, improving accuracy 
on this indicator by using entry cohorts instead of the retrospective 
method used in CFSR Round 2.\6\ This technique also allows us to expand 
the indicator to include maltreatment regardless of the perpetrator 
type. This measure uses the report date reported in NCANDS to determine 
if a child is victimized while in foster care, discounting the first 
week to allow for a potential lag time between an incidence of 
maltreatment and report of maltreatment. For those states that provide 
incident dates, an adjustment will be made if the data indicates that 
the incident occurred prior to the foster care episode. We encourage 
states to report incident dates in NCANDS, which will improve the 
accuracy of this indicator. The Children's Bureau made this change in 
response to a suggestion from stakeholders with regard to the 
indicators used for the last round of reviews that we are now able to 
address.
---------------------------------------------------------------------------

    \6\ Almost all states report AFCARS identifiers in the NCANDS 
data. For those states that do not, a Children's Bureau-approved 
alternate source will be required to assess performance on this 
indicator.
---------------------------------------------------------------------------

Proposed Safety Performance Area 2: Re-Report of Maltreatment

    Indicator Definition: Of all children who received a screened-in 
report of maltreatment during a 12-month period (regardless of 
disposition type), what percent were reported again within 12 months 
from the date of initial report? The denominator is the number of 
children with at least one screened-in report of alleged maltreatment 
in a 12-month period and the numerator is the number of children in the 
denominator that had another screened-in report with a disposition 
within 12 months of their initial report. Screened-in reports that have 
a disposition reported are included, regardless of whether the 
disposition is that the child is a victim or a non-victim. This 
indicator is calculated using data from NCANDS.
    Justification for Inclusion: This indicator is included to provide 
an assessment of whether the agency took the necessary actions to 
prevent a future report of maltreatment for children previously the 
subject of a screened-in report to the agency.
    Relevant Issues and Discussion: During CFSR Round 2, the Children's 
Bureau had a safety performance area related to repeat maltreatment. 
That measure was derived from calculating what percent of all children 
who were victims in a substantiated or indicated maltreatment 
allegation during the first 6 months of the reporting period were not 
victims in another substantiated or indicated maltreatment allegation 
within a 6-month period. We are expanding this measure to count all 
children with screened-in reports of alleged maltreatment. The 
Children's Bureau believes that multiple reports regardless of whether 
maltreatment is substantiated or indicated is a viable measure of the 
agency's attempts to prevent maltreatment based on research indicating 
that families with screened-in but unsubstantiated reports are at a 
high risk of re-report, in some cases as high as substantiated 
cases.\7\ Because reports are included regardless of disposition, this 
indicator includes both victims and non-victims. In addition, this 
indicator expands the time period examined to 12 months to include more 
children. The indicator also tracks such children for 12 months, as 
opposed to 6 months as in the prior indicator.
---------------------------------------------------------------------------

    \7\ See Drake, Jonson-Reid, Way, & Chung (2003). Substantiation 
and Recidivism. Child Maltreatment. Vol. 8, No. 4, 248-260.
---------------------------------------------------------------------------

    The Children's Bureau made this change in response to a suggestion 
from stakeholders with regard to the indicators used for the last round 
of reviews that we are now able to address with the improved quality of 
data reported in NCANDS. In addition, the contractor's recommendations 
based on the expert panel convened in 2013 expressed support for a 
measure of screened-in reports to capture repeat maltreatment.
    Previous CFSR data measures focused on substantiated and indicated 
reports of maltreatment. The growing implementation of differential 
response in the states (sometimes referred to as alternative response 
programs) where a substantial percentage of cases may bypass formal 
investigation altogether, however, makes a comparison of differential-
response and non-differential-response states difficult. In addition, 
states that initiate or expand differential response during an 
improvement period could show improvement on a substantiation-based 
measure of repeat maltreatment merely as an artifact of adopting 
differential response. An indicator based only on screened-in reports 
is not affected by differential response which contributed to our 
selecting this indicator.

CFSR Permanency Outcome 1: Children Have Permanency and Stability in 
Their Living Situations

    The permanency-related statewide data indicators exclude children 
entering foster care at age 18 and older or who are already 18 and 
older on the first day of the period under review. Although the 
amendments to title IV-E of the Social Security Act made by the 
Fostering to Success and Increasing Adoptions Act of 2008 (Pub. L. 110-
351) permit states to provide foster care to youth who are age 18 and 
older, all states have not exercised such an option. Some states 
provide foster care to youth age 18 and older, however, there is no 
consistent inclusion of this population of youth across states and no 
consistent construct at this time for what achieving permanency means 
for such older youth. Therefore, the Children's Bureau believes that it 
is appropriate to limit the permanency statewide data indicators to 
children under age 18 in this way to maintain consistency as we have in 
prior rounds.

Proposed Permanency Performance Area 1: Permanency in 12 Months for 
Children Entering Foster Care

    Indicator Definition: Of all children who enter foster care in a 
12-month period, what percent discharged to permanency within 12 months 
of entering foster care? The denominator is the number of children who 
enter foster care in a 12-month period and the numerator is the number 
of children in the denominator who discharged to permanency within 12 
months of entering foster care. For the purposes of this indicator, 
permanency includes a child's discharge from foster care to parent(s), 
living with relatives, guardianship, or adoption. Children who are in 
foster care for less than 8 days are excluded from this indicator. For 
children with multiple episodes during the same 12 month period, this

[[Page 22608]]

measure will only evaluate the first episode within the period. This 
indicator is calculated using data from AFCARS.
    Justification for Inclusion: This indicator provides a focus for 
the child welfare agency's responsibility to reunify or place children 
in safe and permanent homes as soon as possible after removal.
    Relevant Issues and Discussion: During CFSR Round 2, the Children's 
Bureau included a similar measure as part of one of the permanency 
composites. That measure assessed the percent of all children entering 
foster care for the first time in a 6-month period that left foster 
care to reunification (or living with a relative) within 12 months of 
entering foster care. For CFSR Round 3, the indicator will stand alone 
and not be combined into a composite measure. The indicator is also 
more expansive than the prior to include: (1) All permanency outcomes, 
including guardianship or adoption; (2) all children entering foster 
care rather than first removals only; and (3) children entering foster 
care over the course of 12 months rather than 6 months.
    The Children's Bureau made this change in response to suggestions 
from stakeholders with regard to the indicators used for the last round 
of reviews that we are now able to address. The indicator's expanded 
set of permanency outcomes recognizes that all forms of permanency 
represent equally successful outcomes for children. Although all 
permanency outcomes are included within this one indicator, states will 
still be able to analyze their data to determine which types of 
permanency they are achieving for children. The indicator's expanded 
population recognizes the Children's Bureau's desire to measure 
performance for all children entering foster care rather than first 
removals only. The expansion to 12 months, as opposed to 6 months in 
the prior indicator will yield more stable estimates of performance. A 
12 month period is important for this indicator as this cohort will 
also serve as the basis for the denominator in the re-entry into foster 
care indicator (discussed further below). Re-entry into foster care 
after a discharge from foster care is a rarer event that is better 
captured over a longer period to accommodate variability. In addition, 
including a full 12 month period lessens the effect of potential 
seasonal differences between 6 month periods.
    Please see the section on program improvement plans for more 
information on how this indicator may be used in program improvement.

Proposed Permanency Performance Area 2: Permanency in 12 Months for 
Children in Foster Care for 2 Years or More

    Indicator Definition: Of all children in foster care the first day 
of the year who had been in foster care (in that episode) for 2 years 
or longer, what percent discharged to permanency within the next 12 
months? The denominator is the number of children in foster care on the 
first day of a 12-month period who had been in foster care (in that 
episode) for 2 or more years, and the numerator is the number of 
children in the denominator who discharged to permanency within 12 
months of the first day. Permanency includes discharge to parent(s), 
living with relatives, guardianship, or adoption. Children in foster 
care for less than 8 days are excluded from this indicator. This 
indicator is calculated using data from AFCARS.
    Justification for Inclusion: The Children's Bureau is committed to 
maintaining focus on the key outcome of achieving permanency for all 
children in foster care and shortening the time to permanency. With a 
second indicator of permanency in 12 months specifically focused on the 
cohort of children who have been in foster care for 2 or more years, 
the Children's Bureau can monitor the effectiveness of the state child 
welfare agency in ensuring that states continue to emphasize permanency 
for children who have been in foster care for longer periods of time, 
to ensure they achieve permanency during the period under review.
    Relevant Issues and Discussion: During CFSR Round 2, this same 
individual measure was evaluated as a part of a composite. There are 
substantial numbers of children that cannot achieve permanency in 12 
months, and those children may present different challenges. Such 
children may have a higher likelihood of achieving permanency goals 
such as adoption and guardianship than those achieving permanency in 
the first year. The Children's Bureau continues this measure because of 
a commitment to monitor performance for children and youth who were 
already in foster care, and hold states accountable for attaining 
permanency for them, as well.

Proposed Permanency Performance Area 3: Re-Entry to Foster Care

    Indicator Definition: Of all children who entered foster care in a 
12-month period who were discharged within 12 months of that entry to 
reunification, living with a relative, or guardianship, what percent 
re-entered foster care within 12 months of their discharge? The 
denominator is the number of children who entered foster care in a 12-
month period who discharged within 12 months to reunification, living 
with a relative, or guardianship, and the numerator is the number of 
children in the denominator who re-entered foster care within 12 months 
of their discharge from foster care. Children in foster care for less 
than 8 days are excluded from this indicator. If a child re-enters 
foster care multiple times within 12 months of when they left, only the 
first re-entry into foster care is selected. This indicator is 
calculated using data from AFCARS.
    Justification for Inclusion: Although the Children's Bureau 
believes that it is important to reunify children with their families 
as quickly as possible, we also believe that children should be 
reunified when safe and appropriate and with sufficient supports in 
place to prevent a subsequent removal. This indicator enables the 
Children's Bureau to monitor the effectiveness of programs and practice 
that support reunification and other permanency goals.
    Relevant Issues and Discussion: During CFSR Round 2, this 
performance area was evaluated using a similar measure as a part of a 
composite. That measure was derived by calculating what percent of all 
children discharged from foster care to reunification or living with a 
relative in a 12-month period re-entered foster care in less than 12 
months from the date of discharge. This indicator differs from the 
measure used for CFSR Round 2 by limiting the children eligible for re-
entry to the entry cohort. The CFSR Round 2 measure counted all 
children who left foster care to reunify or live with a relative, 
regardless of when they entered foster care. The purpose of this focus 
on current practice is in keeping with the rationale that new 
interventions may best be monitored in an entry cohort. We also 
expanded the denominator to include children who leave foster care for 
guardianship in an effort to reflect a more comprehensive definition of 
permanency.
    The Children's Bureau made this change in response to suggestions 
from stakeholders with regard to the indicators used for the last round 
of reviews that we are now able to address. The indicator attempts to 
capture the rate of ``permanency'' for children who leave foster care 
by measuring whether children re-enter foster care. For this indicator, 
adoption is not included as a permanency outcome because it is not

[[Page 22609]]

always possible to identify children who re-enter foster care following 
adoption.
    Please see the section on program improvement plans for more 
information on how this indicator may be used in program improvement.

Proposed Permanency Performance Area 4: Placement Stability

    Indicator Definition: Of all children who enter foster care in a 
12-month period, what is the rate of placement moves per day of foster 
care? \8\ The denominator is, among children who enter foster care in a 
12-month period, the total number of days these children were in foster 
care as of the end of the 12-month period. The numerator is, among 
children in the denominator, the total number of placement moves during 
the 12-month period. The initial placement in foster care is not 
counted, but all subsequent moves occurring within the 12-month period 
are included in the calculation. Children in foster care for less than 
8 days and other settings a child may be placed in, that are not 
considered as placement settings for AFCARS purposes such as trial home 
visits, are not included in this indicator. This indicator is 
calculated using data from AFCARS.
---------------------------------------------------------------------------

    \8\ Rates are calculated per day of foster care. However, we 
will multiply the rate by 1,000 to produce larger and more 
meaningful numbers.
---------------------------------------------------------------------------

    Justification for Inclusion: Placement stability is a critical 
component of the permanency and well-being of children in foster care. 
States are responsible for ensuring that children who are removed from 
their homes by the state experience stability while they are in foster 
care.
    Relevant Issues and Discussion: During CFSR Round 2, this 
performance area was evaluated using a similar measure as a part of a 
composite. That measure was derived from calculating what percent of 
all children served in foster care during the 12-month period had two 
or fewer placement settings. The new indicator controls for the length 
of time children spend in foster care so that we can examine moves per 
day of foster care, rather than children as the unit of analysis, as 
was employed during CFSR Round 2. The measure used for CFSR Round 2 was 
unable to differentiate between children who moved twice from children 
who moved more. The new indicator counts each move to accurately 
capture the rate of placement moves, rather than the number of children 
affected.
    The Children's Bureau believes that placement stability is 
important to the permanency and well-being of children in foster care 
regardless of how long they have been in foster care. Even so, our 
analysis of AFCARS data indicates that most placement moves occur 
within a child's first 12 months of foster care, which is why we plan 
to focus this indicator on that time period. With this refined focus, 
the Children's Bureau and states can monitor the period during which 
placement moves are most likely to occur and the state's most recent 
performance. In the CFSR Round 2 measure, placement moves were 
monitored over the life of the case which meant that placement 
instability for a child in the early years of foster care placement 
would affected the assessment of the state's CFSR performance in a more 
recent period under review. We also believe that by confining the 
indicator to this period of time, we are better able to measure a 
state's improvement in a subsequent 12-month period. The Children's 
Bureau made this change in response to suggestions from stakeholders 
with regard to the indicators used for the last round of reviews that 
we are now able to address.

National Standards

    National standards will be established for all indicators. By 
measuring state performance against national standards on statewide 
data indicators, the Children's Bureau can assist states in 
continuously monitoring their performance on child outcomes and help 
practitioners and administrators better understand the entirety of 
their child welfare systems.
    We propose that the national standard for each indicator be set at 
the national observed performance for that particular indicator. The 
national standards will remain constant over the entire round of 
review, as has been the case in prior rounds. In CFSR round 2, national 
standards were based on the 75th percentile (approximately) of all 
states' performance, with an adjustment for sampling error. For this 
round, we believe that the national observed performance, which will be 
similar to the average performance across all states, is a more 
reasonable benchmark and would appropriately challenge states to 
improve their performance.
    The national standard set at the national performance level for 
each indicator is a benchmark that is easily communicated to and 
understood by stakeholders, and a reasonable goal given the reality 
that states still need to improve practice in multiple areas. Setting 
the national standard at the national performance for each indicator is 
rooted in strategies central to an effective performance management 
system focused on continuous quality improvement.
    Methodology: We propose that state performance on each statewide 
data indicator be assessed using a multi-level (i.e., hierarchical) 
model that risk-adjusts for select child- and state-level 
characteristics. Multi-level modeling is a widely accepted statistical 
method that enables fair evaluation of relative performance among 
states with different case mixes. The technique calculates how much 
variance in performance is due to (1) children's individual risk 
factors; (2) random measurement errors (due to modest sample sizes); 
and (3) the state's long-run ability to achieve the desired outcome.
    Risk Adjustment: The model we propose to use will incorporate some 
risk adjustment. By incorporating risk adjustment, the multi-level 
model takes into account and controls for factors that differ across 
the states and that can affect outcomes regardless of the quality of 
services the state provides. The goal of risk adjustment is to minimize 
differences in outcomes that are due to factors over which states have 
little control, such as the age of children coming into foster care. 
For example, for the statewide data indicator of permanency in 12 
months for children entering foster care, a state may discharge 40% of 
its children to permanency by 12 months. Forty percent is the state's 
observed performance, and is simply the number of children discharged 
to permanency by 12 months divided by the number of children eligible 
for such an exit. But this state's risk-adjusted performance might be 
45%. That the state's risk-adjusted performance is higher than its 
observed performance means permanency was achieved for more children 
than expected, given the state's case mix and how other states, on 
average, performed with a similar case mix.
    The Children's Bureau will finalize risk adjustment variables after 
receiving public comments on this document. The contractor's 
recommendations to us based on feedback from the expert panel convened 
in 2013 support the use of risk adjustment. The Children's Bureau's 
consideration of particular risk-adjustment variables will be based 
initially on the research literature, recommendations based on feedback 
from the expert panel and expert consultants, and the availability of 
data. The Children's Bureau will test proposed variables and retain 
only those variables that have a statistically significant relationship 
to the outcome for each statewide data indicator. For example, the 
Children's Bureau has

[[Page 22610]]

tested for possible inclusion child's age, sex, number of prior 
removals, and interactions among these with the proposed statewide data 
indicators. The Children's Bureau will consider risk-adjustment 
variables at both the level of the child (e.g., age at entry) and the 
state (e.g., foster care entry rate).

Assessing State Performance

    To assess state performance, the Children's Bureau proposes to 
estimate each state's risk-adjusted performance and the corresponding 
95% interval estimate. The Children's Bureau can be 95% confident that 
a state's true performance lies somewhere between the lower and upper 
limit of this interval. This interval also provides a way to judge 
whether a state's performance is above or below the national average in 
a statistically meaningful way.
    The Children's Bureau plans to compare each state's interval 
estimate to the national observed performance, and assign each state to 
one of three groups:
     ``No different than national performance'' if the 95% 
interval estimate surrounding the state's risk-adjusted performance 
includes the national observed performance.
     ``Higher than national performance'' if the entire 95% 
interval estimate surrounding the state's risk-adjusted performance is 
higher than the national observed performance.
     ``Lower than national performance'' if the entire 95% 
interval estimate surrounding the state's risk-adjusted performance is 
lower than the national observed performance.
    Whether it is desirable for a state to be higher or lower than the 
national performance depends on the indicator. For the two permanency 
measures, a higher value is more desirable; for the remaining measures, 
a lower value is desirable.
    The methodology described above is similar to that used by the 
Centers for Medicare & Medicaid Services to measure hospital 
performance as part of its Hospital Inpatient Quality Reporting 
program.\9\ The methodology is also consistent with the use of such 
models in education and health care to distinguish statistically high- 
and low-performing schools and hospitals.\10\
---------------------------------------------------------------------------

    \9\ Yale New Haven Health Services Corporation, & Center for 
Outcomes Research & Evaluation. (2013). 2013 Measures Updates and 
Specifications Report: Hospital-Level 30-Day Risk-Standardized 
Readmission Measures for Acute Myocardial Infarction, Heart Failure, 
and Pneumonia (Version 6.0). Retrieved from http://www.qualitynet.org/dcs/ContentServer?c=Page&pagename=QnetPublic/Page/QnetTier4&cid=1219069855841.
    \10\ See Goldstein & Spiegelhalter (2007). League Tables and 
Their Limitations. 159(3), 385-443; Normand & Shahian (2007). 
Statistical and Clinical Aspects of Hospital Outcomes Profiling. 
Statistical Science, 22(2), 206-226.
---------------------------------------------------------------------------

Data

    Data Profiles: The Children's Bureau will provide data profiles of 
state performance to each state before the state's CFSR on the 
statewide data indicators and other contextual data available from 
AFCARS and NCANDS. This data profile will assist the state to develop 
its statewide assessment. In addition, the Children's Bureau will 
provide data profiles semi-annually to assist states in measuring 
progress toward the goals identified in the program improvement plan.
    Data Quality: Setting national standards and measuring state 
performance on statewide data indicators for CFSR purposes relies upon 
the states submitting high-quality data to AFCARS and NCANDS. The 
Children's Bureau has provided states with consultation and technical 
assistance before and throughout both rounds of reviews to address data 
quality issues. Additionally, the Children's Bureau has provided states 
with tools for AFCARS and NCANDS that allow the agency to examine its 
data for accuracy and encourages states to incorporate these in their 
ongoing quality assurance process to review data.
    During the first two rounds of the CFSR, there have been occasions 
in which the Children's Bureau cannot use a state's data in aggregate 
calculations of the national standard. In isolated circumstances, these 
data quality issues have been significant enough to prevent us from 
relying on the state's data as an accurate assessment of its 
performance on a statewide data indicator. The Children's Bureau would 
like to be clear about the level of data quality issues that prevent 
state data from being used for CFSR purposes as described below.

Data Quality: Excluding States From National Standards or State 
Performance

    We analyzed every data element from AFCARS and NCANDS that is 
relevant to each statewide data indicator (as listed in Attachment C) 
and performed data quality checks across files both over time as well 
as between files.\11\ Examples of these checks included looking for the 
presence of the same child identifier in the AFCARS and NCANDS file and 
reviewing for consistent reporting of a child in AFCARS from the time 
the child entered foster through discharge and with an associated 
reason for discharge. This analysis revealed the scope of data quality 
issues present in current AFCARS and NCANDS submissions.
---------------------------------------------------------------------------

    \11\ This data quality analysis is specific to indicators 
proposed for CFSR Round 3. It is a separate analysis from the 
standard data quality checks or validation that are performed when 
states submit their AFCARS or NCANDS data.
---------------------------------------------------------------------------

    Based on this analysis, we developed thresholds to identify data 
quality concerns and either accept or exclude the files when 
calculating national standards and state performance. For those data 
quality issues that are contained to one data file submission, we will 
consider a threshold of 5%. In other words, any state that has more 
than 5% of data missing or invalid \12\ will be excluded from the model 
used to calculate the national standard (i.e., the national observed 
performance) and estimate states' risk-adjusted performance. For cross-
file checks, we are setting a higher threshold for exclusion. For the 
maltreatment in foster care measure, a state will be excluded from the 
national standards calculation and performance estimate if more than 
10% of NCANDS victims are missing an AFCARS identifier. For the 
permanency indicators, a state will be excluded if it has more than 10% 
of dropped cases across two six month file submissions.
---------------------------------------------------------------------------

    \12\ For example, the date of birth is greater than the date of 
latest removal.
---------------------------------------------------------------------------

Data Quality: Case-Level Exclusions

    For those states that do not exceed the data quality thresholds but 
still have identified data quality problems, we will include the state 
in national standards calculations and measure state performance but we 
will exclude child-level records with missing or invalid data on 
elements needed to determine the child's outcome and perform the risk-
adjustment. For example, if the risk-adjustment for an indicator 
includes age at entry, a child whose age at entry cannot be determined 
(due to a missing date of birth) will not be include in the analysis. 
We believe this exclusion will result in more accurate estimates of 
performance for those states with minor data quality issues. For each 
indicator, the Children's Bureau will provide each state with a list of 
records that were excluded from the analyses.

Program Improvement Plans

    States that fall below the national standard on any given indicator 
will be required to include that indicator in a program improvement 
plan. Regardless of which indicators a state is required to include in 
its program improvement plan, the Children's Bureau will provide each 
state with a data profile that

[[Page 22611]]

includes information on the state's performance on all of the statewide 
data indicators.
    Companion Measures: For two of the statewide data indicators, 
permanency in 12 months for children entering foster care and re-entry 
to foster care, the Children's Bureau proposes to consider performance 
for program improvement purposes in concert with the other indicator. 
This means that if a state has a program improvement plan that includes 
improving on the indicator permanency in 12 months for children 
entering foster care, the Children's Bureau's determination of whether 
the state has improved successfully will take into consideration its 
performance on the re-entry to foster care indicator as a companion 
measure. Specifically, the state must stay above a threshold for the 
companion re-entry to foster care indicator as well as achieve its goal 
on the permanency in 12 months for children entering foster care 
indicator, to successfully complete the program improvement plan. The 
reverse is also true. For details about threshold calculations, please 
see the section below. If a state must improve on the re-entry to 
foster care indicator in its program improvement plan, it must also 
include the permanency in 12 months for children entering foster care 
indicator as a companion measure. Although the Children's Bureau 
believes that it is important to reunify children with their families 
as quickly as possible, we also believe that children should be 
reunified when safe and appropriate and sufficient supports are in 
place to prevent a subsequent removal. The recommendations to us based 
on the expert panel convened in 2013 also support the use of companion 
measures in program improvement.
    Methods for Setting State Performance Baselines, Goals, and 
Thresholds: The key components for setting performance goals and 
monitoring progress over the course of a program improvement plan 
involve calculating baselines, goals, and thresholds. The Children's 
Bureau methodology for specifying state improvement goals is 
statistically grounded, producing goals that are programmatically 
challenging, yet attainable. In addition, goals and thresholds should 
reflect each state's own performance history and demonstrated capacity 
for change. In CFSR Round 2, the percent of improvement required for 
each state was the same for all states, and was applied to state-
specific baselines. While this standardization was easy to communicate 
how it applied to each state, it also meant that high-performing states 
were asked to make larger absolute improvements than lower-performing 
states. This approach did not take into account the variability in 
performance that states have shown over time. For CFSR Round 3, the 
Children's Bureau plans to set improvement goals relative to each 
state's past performance.
    Setting Baselines: The Children's Bureau plans to set the baseline 
for each statewide data indicator included in a program improvement 
plan at the state's observed performance on that indicator for the most 
recent year of data available before the beginning of program 
improvement plan implementation. We also considered using the state's 
three-year average in this calculation, but that approach occasionally 
produced goals that fell below the state's performance in the most 
recent year. Consistent with CFSR Round 2, we will use the most recent 
year as the baseline, because it represents a reasonable estimate of 
the state's current performance.
    Setting Goals and Thresholds: We plan to set performance goals and 
thresholds will be based on the variability in the state's observed 
performance in the three most recent years of data. There will be 
improvement factors, as in CFSR Round 2, but these will be driven by 
the variability in performance that the particular state has shown in 
the last three years. We will apply the improvement factors to the 
baseline to produce the concrete performance goal. The state's amount 
of improvement required for a program improvement plan will be more 
than what is likely, in a statistical sense, to occur by chance alone. 
Conversely, we plan to set thresholds as the inverse of goals, which 
will identify a point by which a state is demonstrating a performance 
decline for companion measures that is more statistically than what 
might be expected by chance.
    We will use a technique called bootstrapping to develop goals and 
thresholds. The method calls for the Children's Bureau to repeatedly 
sample a state's past three years of performance estimates to construct 
a larger sample, and from that the calculation of a grand mean and 
standard deviation. The grand mean reflects that state's ``average'' 
performance and the standard deviation reflects how much normal 
fluctuation in performance the Children's Bureau might expect for that 
state, given its past performance. Then the standard deviation is used 
to calculate an estimate that would represent a level of change above 
and beyond the typical fluctuation that would otherwise be expected. 
The Children's Bureau will set the magnitude at four standard 
deviations from the grand mean. At that level we can say with 
confidence that--if we were to randomly estimate a state's performance 
on the indicator (using their past performance), and did so 100 times--
we would expect to see performance at this level less than 6% of the 
time (or fewer than 6 times out of 100). Six times out of 100 is rather 
rare, which is why we can treat it as representing a statistically 
meaningful change has occurred in the program.\13\
---------------------------------------------------------------------------

    \13\ These guidelines are based on a theorem known as 
Chebyshev's inequality. When all you have is an overall mean and 
standard deviation (which is what we have for each state's 
performance), the theorem guarantees that a certain percent of 
values will be k standard deviations away from the mean. 
Specifically, at least 75% of the values will be within two standard 
deviations of the mean, at least 89% within three standard 
deviations, at least 94% within four standard deviations, at least 
96% within five standard deviations, and at least 97% within six 
standard deviations (Chong et al. (2012). Improving Generalization 
Performance in Co-Evolutionary Learning. IEEE, vol. 16, no. 1, 70-
85; Sheppard (2011). Environmental Study-Soil Sample Analysis for 
the Department of Ecology at Hanford).
---------------------------------------------------------------------------

    To determine exactly how much a state will need to improve, we must 
first calculate an improvement factor, which is the percentage 
difference between the grand mean and four standard deviations above 
the grand mean. We then apply that to the baseline, which is the 
observed performance in the most recent year. To demonstrate a sample 
calculation:

    A state may have a grand mean of 50%, a grand mean plus four 
standard deviations = 52%, and a year 3 value of 51. This will give 
us an improvement factor of 52/50 = 1.04. If that is applied to the 
baseline of 51%, the program improvement plan goal will be 51% x 
1.04 = 53.04%.

    We will use a comparable technique to set thresholds for companion 
measures, subtracting rather than adding four standard deviations to 
the grand mean (when higher performance on an indicator is better), 
which can be used to identify a state's decline in performance. To 
provide an example, if a goal was calculated to be three percentage 
points higher than the baseline percent, the threshold would be three 
percentage points below it. Thresholds are only relevant to companion 
measures.
    By design, states with less variation in performance from year to 
year have more modest goals, while those showing greater variation have 
more aggressive goals. Overall, we believe that the goals are 
reflective of each state's own prior experience and performance levels, 
with goals that are achievable and substantively meaningful. We 
acknowledge that a few states with the

[[Page 22612]]

lowest variation in performance for each indicator are assigned very 
modest goals, while a few states with the highest variation in 
performance are assigned very aggressive goals.
    To address these problems at the extreme ends, we propose to 
establish minimum and maximum improvement factors. Specifically, a 
floor will be set at the top of the bottom fifth, and the bottom of the 
top fifth, ordered by the size of the improvement factor. While the 
impact of this rule varies somewhat from indicator to indicator, 
overall we believe it provides a consistent basis for producing goals 
that are achievable and substantively meaningful. The inverse would be 
done for the thresholds. The contractor's recommendations to us based 
on the feedback from the expert panel convened in 2013 support the 
setting of maximum and minimum thresholds for improvement goals at the 
level of performance of top and bottom quintiles.

Process for Final Indicators

    We are interested in comments on all aspects of the statewide data 
indicators proposed and the methods to calculate national standards and 
program improvement. After considering the feedback to this docuemnt, 
we plan to publish a final list of indicators and methods that will be 
used in the CFSRs along with the actual national standards.

    (Authority: 42 U.S.C. 1320a-1a; 45 CFR 1355.31-37.)

Mark Greenberg,
Acting Commissioner, Administration on Children, Youth and Families.

Attachment A: Proposed Statewide Data Indicators

--------------------------------------------------------------------------------------------------------------------------------------------------------
      Category           Measure title       Measure description       Denominator            Numerator            Exclusions               Notes
--------------------------------------------------------------------------------------------------------------------------------------------------------
Safety.............  Maltreatment in        Of all children in    (From AFCARS) Among   (From NCANDS) Among   --If a state          Cases are matched
                      foster care.           foster care during    children in foster    children in the       provides incident     across AFCARS and
                                             a 12-month period,    care during a 12-     denominator, total    dates, records with   NCANDS using AFCARS
                                             what is the rate of   month period, total   number of             an incident date      ID.
                                             victimization per     number of days        substantiated or      occurring before
                                             day of foster care?   these children were   indicated reports     the date of removal
                                                                   in foster care as     of maltreatment (by   will be excluded.
                                                                   of the end of the     any perpetrator)     --Children in foster
                                                                   12-month period \a\.  during the 12-month   care <8 days.
                                                                                         period \b\.          --Any report that
                                                                                                               occurs within the
                                                                                                               first 7 days of
                                                                                                               removal.
Safety.............  Re-report of           Of all children with  (From NCANDS) Number  (From NCANDS) Number  None................  Reports are included
                      maltreatment.          a screened-in         of children with at   of children in the                          regardless of the
                                             report of alleged     least one screened-   denominator that                            type of
                                             maltreatment in a     in report of          had another                                 disposition, so
                                             12-month period       alleged               screened-in report                          this indicator
                                             (regardless of        maltreatment in a     within 12 months of                         includes both
                                             disposition), what    12-month period.      their initial                               victims and non-
                                             percent had another                         report.                                     victims.
                                             screened-in report
                                             within 12 months of
                                             their initial
                                             report?
Permanency.........  Permanency in 12       Of all children who   (From AFCARS) Number  (From AFCARS) Number  --Children in foster  Youth who turn 18
                      months for children    enter foster care     of children who       of children in the    care <8 days.         while in foster
                      entering foster care.  in a 12-month         enter foster care     denominator who      --Children who enter   care who were
                                             period, what          in a 12-month         discharged to         foster care at age    included in the
                                             percent discharged    period.               permanency within     18 or more.           denominator will
                                             to permanency                               12 months of                                not be counted as
                                             within 12 months of                         entering foster                             having achieved
                                             entering foster                             care or by the time                         permanency,
                                             care? \c\                                   they reached 18.                            regardless of
                                                                                                                                     discharge reason.
Permanency.........  Permanency in 12       Of all children in    (From AFCARS) Number  (From AFCARS) Number  --Children age 18 or  Youth who turn 18
                      months for children    foster care on the    of children in        of children in the    more on the first     while in foster
                      in foster care 2       first day of a 12-    foster care on the    denominator who       day of the year.      care who were
                      years or more.         month period, who     first day of a 12-    discharged to                               included in the
                                             had been in foster    month period who      permanency within                           denominator will
                                             care (in that         had been in foster    12 months of the                            not be counted as
                                             episode) for 2 or     care (in that         1st day or by the                           having achieved
                                             more years, what      episode) for 2 or     time they reached                           permanency,
                                             percent discharged    more years.           18.                                         regardless of
                                             to permanency                                                                           discharge reason.
                                             within 12 months of
                                             the first day?

[[Page 22613]]

 
Permanency.........  Re-entry to foster     Of all children who   (From AFCARS) Number  (From AFCARS) Number  --Children in foster  If a child has
                      care.                  enter foster care     of children who       of children in the    care < 8 days.        multiple re-entries
                                             in a 12-month         enter foster care     denominator who re-  --Children who enter   to foster care
                                             period who            in a 12-month         enter foster care     or exit foster care   within 12 months of
                                             discharged within     period who            within 12 months of   at age 18 or more.    their discharge,
                                             12 months to          discharged within     their discharge.                            only his first re-
                                             reunification, live   12 months to                                                      entry is selected.
                                             with relative, or     reunification, live
                                             guardianship, what    with relative, or
                                             percent re-entered    guardianship.
                                             foster care within
                                             12 months of their
                                             discharge? \a\
Permanency.........  Placement Stability..  Of all children who   (From AFCARS) Among   (From AFCARS) Among   --Children in foster  The initial removal
                                             enter foster care     children who enter    children in the       care < 8 days.        from home (and into
                                             in a 12-month         foster care in a 12-  denominator, total   --Children who enter   foster care) is not
                                             period, what is the   month period, total   number of placement   foster care at age    counted as a
                                             rate of placement     number of days        moves during the 12-  18 or more.           placement move.
                                             moves per day of      these children were   month period \e\.
                                             foster care?          in foster care as
                                                                   of the end of the
                                                                   12-month period \d\.
--------------------------------------------------------------------------------------------------------------------------------------------------------
\a\ For example, if during the 12-month period there were two children in foster care, one child for 10 days (1st episode), the same child for 40 days
  (2nd episode), and the other child for 100 days (his only episode), the denominator would = 150 days (10+40+100).
\b\ For example, if during the 12-month period there were two children in foster care, and one child with 3 substantiated or indicated reports and the
  other with 1 such report, the numerator would = 4 reports (3+1).
\c\ If a child has multiple entries during the year, only his or her first entry is selected.
\d\ For example, if during the 12-month period two children entered foster care, one child for 10 days and the other child for 100 days, the denominator
  would be 110 days (10+100).
\e\ For example, if during the 12-month period two children entered foster care, and one child had 3 moves and the other had 1 move, the numerator would
  = 4 moves (3+1).

Attachment B: Comparison of Data Measures--CFSR Round 2 and Round 3

----------------------------------------------------------------------------------------------------------------
                                            Proposed CFSR round     Comparable CFSR
      Category           Measure title          3 indicator         round 2 measure     How and why it's changed
----------------------------------------------------------------------------------------------------------------
Safety.............  Maltreatment in       Of all children in    Of all children in    In the CFSR 2 measure,
                      foster care.          foster care during    foster care during    counts of children not
                                            a 12-month period,    the reporting         maltreated in foster
                                            what is the rate of   period, what          care are derived by
                                            victimization per     percent were not      subtracting the NCANDS
                                            day \a\ of foster     victims of            count of children
                                            care?                 substantiated or      maltreated by foster
                                                                  indicated             care providers from the
                                                                  maltreatment by a     total count of all
                                                                  foster parent or      children placed in
                                                                  facility staff        foster care, as reported
                                                                  member?               in AFCARS. Because of
                                                                                        improved reporting by
                                                                                        states, we now link
                                                                                        AFCARS and NCANDS data
                                                                                        using the child ID and
                                                                                        determine if
                                                                                        maltreatment occurred
                                                                                        during a foster care
                                                                                        episode, improving
                                                                                        accuracy on the
                                                                                        indicator.
                                                                                       This also allows us to
                                                                                        expand the measure to
                                                                                        include all types of
                                                                                        perpetrators (including,
                                                                                        for example, parents)
                                                                                        under the assumption
                                                                                        that states should be
                                                                                        held accountable for
                                                                                        keeping children safe
                                                                                        from harm while in the
                                                                                        care of the state, no
                                                                                        matter who the
                                                                                        perpetrator is.
Safety.............  Re-report of          Of all children with  Of all children who   We are expanding the
                      maltreatment.         a screened-in         were victims of       measure to count all
                                            report of alleged     substantiated or      children with screened-
                                            maltreatment in a     indicated             in reports of alleged
                                            12-month period,      maltreatment          maltreatment, because
                                            what percent had      allegation during     research suggests
                                            another screened-in   the first 6 months    children with prior
                                            report within 12      of the reporting      reports are at greater
                                            months of their       period, what          risk.
                                            initial report?       percent were not     In addition, by limiting
                                                                  victims of another    only to victims, we
                                                                  substantiated or      could face measurement
                                                                  indicated             challenges as states
                                                                  maltreatment          implement Differential
                                                                  allegation within a   Response during a
                                                                  6-month period?       monitoring cycle, which
                                                                                        could have an impact on
                                                                                        numbers of
                                                                                        substantiations.
                                                                                       We also propose using a
                                                                                        full 12-month period
                                                                                        rather than only 6
                                                                                        months to capture the
                                                                                        denominator, to create
                                                                                        more stable estimates.

[[Page 22614]]

 
Permanency.........  Permanency in 12      Of all children who   Composite 1.3: Of     We now count all types of
                      months for children   enter foster care     all children          permanency
                      entering foster       in a 12-month         entering foster       (reunification, live
                      care.                 period, what          care for the first    with relative, adoption
                                            percent discharged    time in a 6-month     or guardianship) as
                                            to permanency         period, what          having `met' the
                                            within 12 months of   percent discharged    indicator.
                                            entering foster       to reunification     We also expanded the
                                            care?                 (or live with         measure to include all
                                                                  relative) within 12   children who entered
                                                                  months of entering    foster care that year;
                                                                  foster care or by     not just those on their
                                                                  the time they         first removal episode.
                                                                  reached 18?          We also expanded the
                                                                                        window of time for the
                                                                                        entry cohort to a full
                                                                                        year instead of 6
                                                                                        months; this will yield
                                                                                        more stable estimates.
Permanency.........  Permanency in 12      Of all children in    Composite 3.1: Of     Same measure; no change.
                      months for children   foster care on the    all children in       The difference is that
                      in foster care for    first day of a 12-    foster care on the    it is now evaluated on
                      2 years or more.      month period who      first day of a 12-    its own, rather than as
                                            had been in foster    month period who      just one part of a
                                            care (in that         had been in foster    composite measure.
                                            episode) for 2 or     care (in that        By including the Entry
                                            more years, what      episode) for 2 or     Cohort Permanency
                                            percent discharged    more years, what      indicator (listed above)
                                            to permanency         percent discharged    as well as the Legacy
                                            within 12 months of   to permanency         Cohort Permanency
                                            the first day?        within 12 months of   indicator, we hold
                                                                  the first day or by   states accountable not
                                                                  the time they         only for children in
                                                                  reached 18?           their first year of
                                                                                        foster care, but also
                                                                                        those children/youth who
                                                                                        have been in foster care
                                                                                        for long periods of
                                                                                        time.
Permanency.........  Re-entry in 12        Of all children who   Composite 1.4: Of     The new indicator is
                      months.               enter foster care     all children          limited to those
                                            in a 12-month         discharged from       children who entered
                                            period and            foster care to        foster care during the
                                            discharged within     reunification or      year, whereas the CFSR
                                            12 months to          live with a           Round 2 measure counted
                                            reunification, live   relative in a 12-     all children who
                                            with relative, or     month period, what    discharged to
                                            guardianship, what    percent re-entered    reunification or live
                                            percent re-entered    foster care in less   with relative,
                                            foster care within    than 12 months from   regardless of when they
                                            12 months of their    the date of           entered foster care. The
                                            date of discharge?    discharge?            purpose of this focus is
                                                                                        in keeping with the
                                                                                        rationale that new
                                                                                        interventions may best
                                                                                        be monitored in an entry
                                                                                        cohort. This indicator
                                                                                        will also be used as a
                                                                                        companion measure with
                                                                                        permanency in 12 months,
                                                                                        to ensure that states
                                                                                        working to improve
                                                                                        permanency rates in
                                                                                        their entry cohort do
                                                                                        not see worsening
                                                                                        performance on rates of
                                                                                        re-entry to foster care.
                                                                                       We also expanded the
                                                                                        denominator to allow
                                                                                        discharges to
                                                                                        guardianship, in an
                                                                                        effort to capture more
                                                                                        discharges to
                                                                                        permanency. Exits to
                                                                                        adoption are not
                                                                                        included because they
                                                                                        cannot be tracked
                                                                                        reliably, as some states
                                                                                        issue new child
                                                                                        identifiers if a child
                                                                                        who was previously
                                                                                        adopted enters foster
                                                                                        care.
Permanency.........  Placement stability.  Of all children who   Composite 4.1: Of     The proposed indicator
                                            enter foster care     all children served   controls for length of
                                            in a 12-month         in foster care        time in foster care, so
                                            period, what is the   during the 12-month   we are looking at moves
                                            rate of placement     period, what          per day of foster care,
                                            moves per day \b\     percent had two or    rather than children as
                                            of foster care?       fewer placement       the unit of analysis.
                                                                  settings?            The rationale for using
                                                                                        an entry cohort rather
                                                                                        than all children served
                                                                                        is that our analysis
                                                                                        shows children entering
                                                                                        foster care tend to move
                                                                                        much more than those
                                                                                        children/youth in foster
                                                                                        care for longer periods
                                                                                        of time, whose
                                                                                        placements may have
                                                                                        stabilized.
                                                                                       In CFSR Round 2 measure,
                                                                                        moves that took place
                                                                                        prior to the monitoring
                                                                                        period were counted. Now
                                                                                        we only count those
                                                                                        moves that occur during
                                                                                        the monitoring period.
                                                                                        The initial placement is
                                                                                        not counted.
                                                                                       The CFSR Round 2 measure
                                                                                        treated children who
                                                                                        moved 2 times in an
                                                                                        episode the same as
                                                                                        children who moved 15
                                                                                        times; both were a
                                                                                        failure to meet the
                                                                                        measure. The new
                                                                                        indicator counts each
                                                                                        move, so it continues to
                                                                                        hold states accountable
                                                                                        for those children/youth
                                                                                        who have already moved
                                                                                        several times.
----------------------------------------------------------------------------------------------------------------
\a\ The rate may be expressed per 100,000 days because it is such a rare event. Using this metric gives us
  numbers greater than zero, which are easier to communicate.
\b\ The rate is expressed per 1,000 days to convert the rate to a metric that gives us numbers greater than
  zero.


[[Page 22615]]

Attachment C: Data Elements Required for Statewide Data Indicators

    For instruction regarding AFCARS data elements, refer to http://www.acf.hhs.gov/programs/cb/resource/afcars-tb1.
    For instruction with regard to NCANDS data elements, refer to 
http://www.ndacan.cornell.edu/datasets/pdfs_user_guides/178-NCANDS-child2012v1-User-Guide-and-Codebook.pdf.

----------------------------------------------------------------------------------------------------------------
 Primary data elements required    Permanency by  Re-entry by 12     Placement     Re-report of    Maltreatment
         for calculation             12 months        months         stability     maltreatment   in foster care
----------------------------------------------------------------------------------------------------------------
AFCARS FC Element 1:            [check]         [check]         [check]              NA         [check]
 \a\ Title IV-E Agency..........
AFCARS FC Element 4:            [check]         [check]         [check]              NA         [check]
 Record Number..................
AFCARS FC Element 21:           [check]         [check]         [check]              NA         [check]
 Date of Latest Removal.........
AFCARS FC Element 23:                NA              NA         [check]              NA              NA
 Date of Placement in Current
 Foster Care Setting............
AFCARS FC Element 24:                NA              NA         [check]              NA              NA
 Number of Placement Settings
 during this Removal Episode....
AFCARS FC Element 56:           [check]         [check]         [check]              NA         [check]
 Date of Discharge from FC......
AFCARS FC Element 58:           [check]         [check]              NA              NA              NA
 Reason for Discharge...........
NCANDS CF Element 4:                 NA              NA              NA         [check]              NA
 Child ID.......................
NCANDS CF Element 6:                 NA              NA              NA         [check]         [check]
 Report Date....................
NCANDS CF Element 27:                NA              NA              NA              NA         [check]
 Child Maltreatment 1--
 Disposition Level \b\..........
NCANDS CF Element 29:                NA              NA              NA              NA         [check]
 Child Maltreatment 2--
 Disposition Level..............
NCANDS CF Element 31:                NA              NA              NA              NA         [check]
 Child Maltreatment 3--
 Disposition Level..............
NCANDS CF Element 33:                NA              NA              NA              NA         [check]
 Child Maltreatment 4--
 Disposition Level..............
NCANDS CF Element 34:                NA              NA              NA              NA         [check]
 Maltreatment death.............
NCANDS CF Element 145:               NA              NA              NA              NA         [check]
 AFCARS ID......................
----------------------------------------------------------------------------------------------------------------
\a\ The elements are numbered by their position in the flat ASCII files submitted by states to these reporting
  systems. These numbering schema are specific to the files utilized by ACYF. Files obtained through the
  National Data Archive on Child Abuse and Neglect (NDACAN) may have a slightly different order.
\b\ Definition of `victim' includes all children with a disposition level (for any of up to four maltreatments
  per child) of: a) Substantiated, or b) Indicated. These do not propose including differential response
  victims. Victims also include children who died as a result of maltreatment.


----------------------------------------------------------------------------------------------------------------
    Additional data elements
   required for risk-adjusted      Permanency by  Re-entry by 12     Placement     Re-report of    Maltreatment
          Analysis \c\               12 months        months         stability     maltreatment   in foster care
----------------------------------------------------------------------------------------------------------------
AFCARS FC Element 6:            [check]         [check]         [check]              NA         [check]
 Child's Date of Birth..........
AFCARS FC Element 7:            [check]         [check]         [check]              NA         [check]
 Child Sex......................
AFCARS FC Element 19:           [check]         [check]         [check]              NA         [check]
 Total  of Removals....
NCANDS CF Element 14:                NA              NA              NA         [check]              NA
 Child Age......................
NCANDS CF Element 17:                NA              NA              NA         [check]              NA
 Child Sex......................
US Census Bureau: Child                  [check]         [check]              NA         [check]         [check]
 Population, by State...........
----------------------------------------------------------------------------------------------------------------
\c\ In addition to those data elements used for risk adjustment, a few more are used to make necessary
  adjustments to outcomes; for example, we use the child's current placement setting to determine if he or she
  is in Trial Home Visit before Reunification and, if so, time in foster care is adjusted consistent with
  adjustments for trial home visits used in CFSR Round 2. If a state provides NCANDS CF Element 146,
  Incident Date, an adjustment will be made to the maltreatment in foster care indicator to improve accuracy.

[FR Doc. 2014-09001 Filed 4-22-14; 8:45 am]
BILLING CODE 4184-25-P