Notice of Proposed Information Collection Requests; Office of Planning, Evaluation and Policy Development; Study of Strategies for Improving the Quality of Local Grantee Program Evaluation, 35665-35666 [2012-14593]

Agencies

• DEPARTMENT OF EDUCATION

[Federal Register Volume 77, Number 115 (Thursday, June 14, 2012)]
[Notices]
[Pages 35665-35666]
From the Federal Register Online via the Government Printing Office [www.gpo.gov]
[FR Doc No: 2012-14593]



[[Page 35665]]

=======================================================================
-----------------------------------------------------------------------

DEPARTMENT OF EDUCATION


Notice of Proposed Information Collection Requests; Office of 
Planning, Evaluation and Policy Development; Study of Strategies for 
Improving the Quality of Local Grantee Program Evaluation

SUMMARY: This study is intended to inform the Department's decisions 
about how to structure future grant competitions; how to support 
evaluation and performance reporting activities among funded grantees, 
including technical assistance to improve the quality of evaluations 
and performance reporting; and how to make the best possible use of 
grantee evaluation findings.

DATES: Interested persons are invited to submit comments on or before 
August 13, 2012.

ADDRESSES: Written comments regarding burden and/or the collection 
activity requirements should be electronically mailed to 
ICDocketMgr@ed.gov or mailed to U.S. Department of Education, 400 
Maryland Avenue SW., LBJ, Washington, DC 20202-4537. Copies of the 
proposed information collection request may be accessed from https://edicsweb.ed.gov, by selecting the ``Browse Pending Collections'' link 
and by clicking on link number 04869. When you access the information 
collection, click on ``Download Attachments'' to view. Written requests 
for information should be addressed to U.S. Department of Education, 
400 Maryland Avenue SW., LBJ, Washington, DC 20202-4537. Requests may 
also be electronically mailed to ICDocketMgr@ed.gov or faxed to 202-
401-0920. Please specify the complete title of the information 
collection and OMB Control Number when making your request.
    Individuals who use a telecommunications device for the deaf (TDD) 
may call the Federal Information Relay Service (FIRS) at 1-800-877-
8339.

SUPPLEMENTARY INFORMATION: Section 3506 of the Paperwork Reduction Act 
of 1995 (44 U.S.C. Chapter 35) requires that Federal agencies provide 
interested parties an early opportunity to comment on information 
collection requests. The Director, Information Collection Clearance 
Division, Privacy, Information and Records Management Services, Office 
of Management, publishes this notice containing proposed information 
collection requests at the beginning of the Departmental review of the 
information collection. The Department of Education is especially 
interested in public comment addressing the following issues: (1) Is 
this collection necessary to the proper functions of the Department; 
(2) will this information be processed and used in a timely manner; (3) 
is the estimate of burden accurate; (4) how might the Department 
enhance the quality, utility, and clarity of the information to be 
collected; and (5) how might the Department minimize the burden of this 
collection on the respondents, including through the use of information 
technology. Please note that written comments received in response to 
this notice will be considered public records.
    Title of Collection: Study of Strategies for Improving the Quality 
of Local Grantee Program Evaluation.
    OMB Control Number: 1875-NEW.
    Type of Review: New.
    Total Estimated Number of Annual Responses: 20.
    Total Estimated Number of Annual Burden Hours: 20.
    Abstract: The U.S. Department of Education provides support to 
states, districts and schools through a number of competitive and 
formula grant programs. Through these programs, the federal government 
funds a wide array of activities, from professional development for 
teachers to turnaround efforts for failing schools. High-quality 
evaluation plays an essential role informing policy makers about 
program performance, outcomes and impact. Performance reporting with 
high-quality data can indicate whether a funded project is meeting its 
goals and taking place as planned. Project evaluations can explore how 
best to implement a particular educational practice, whether positive 
student outcomes were attained, or whether a particular educational 
intervention actually caused the outcomes observed.
    To date, the Department lacks comprehensive information about the 
quality or rigor of the performance reporting and evaluation activities 
its grantees are undertaking and whether the technical assistance 
provided has been useful in improving the quality of the performance 
reporting or evaluations. Accordingly, the focus of this study is to 
examine the influence of Department-funded technical assistance 
practices on the quality and rigor of grantee evaluations and 
performance reporting in two Department programs (described below). It 
will describe the technical assistance provided by the Department to 
support grantee performance reporting and evaluation; explore how 
grantees perceive the technical assistance has influenced their 
activities; assess the quality of performance reporting and evaluations 
undertaken; and determine how the findings from performance reporting 
and evaluations were used both by grantees and by the Department.
    This study will be based upon a systematic review of existing 
documentation as well as interviews with selected grantees and with 
federal staff and federal contractors involved in grant monitoring and 
in the provision of technical assistance to grantees. The interviews 
with selected grantees are the subject of this OMB clearance request.
    This study will focus on two grant programs within the Department's 
Office of Innovation and Improvement: the Charter Schools Program: 
State Educational Agencies (CSP SEA) program and the Voluntary Public 
School Choice (VPSC) program. A brief description of each program is 
provided below.
    1. Voluntary Public School Choice (authorized under the Elementary 
and Secondary Education Act of 1965, as amended, Title V, Part B, 
Subpart 3, (20 U.S.C. 7225-7225g)). The goal of the VPSC program is to 
support the creation and development of a large number of high-quality 
charter schools that are held accountable for enabling students to 
reach challenging state performance standards, and are open to all 
students. The program was first enacted as part of the No Child Left 
Behind Act of 2001 (Pub. L. No. 107-110, Sec.  115, Stat. 1425) to 
support the emergence and growth of choice initiatives across the 
country. VPSC's goal is to assist states and local school districts in 
creating, expanding, and implementing public school choice programs. 
The program has awarded two cycles of competitive grants to states, 
local education agencies, and partnerships that include public, 
nonprofit and for-profit organizations. In 2007, the most recent award 
year, the program awarded a total of 14 competitive grants to two 
states, eight school districts, a charter school, an intermediate 
school district, and KIPP schools in Texas.
    2. Charter Schools Program: State Educational Agencies (authorized 
under the Elementary and Secondary Education Act of 1965, Section 5201-
5211 (20 U.S.C. 7221a)). Federal support for charter schools began in 
1995 with the authorization of the CSP. The CSP SEA program awards 
competitive grants to state education agencies to plan, design, and 
implement new charter schools, as well as to disseminate information on 
successful charter schools. The key goals of the CSP SEA program are to 
increase the number of charter schools in operation across the nation 
and to increase the number of students who are achieving proficiency

[[Page 35666]]

on state assessments of math and reading. The CSP statute also 
addresses expanding the number of high-quality charter schools and 
encouraging states to provide support for facilities financing equal to 
what states provide for traditional public schools. Grants have been 
awarded to 40 states, including awards to 33 states since 2005. Grants 
are typically awarded for three years and may be renewed.
    This study will review all technical assistance provided to CSP SEA 
and VPSC grantees on performance reporting and evaluations and how 
grantees conduct these activities. All CSP SEA and VPSC grantees are 
required by the Department to conduct performance reporting. Although 
grantees are not required to conduct any particular type of evaluation, 
the study will review both impact evaluations and non-impact 
evaluations conducted by grantees. The study approach, with respect to 
the review of performance reporting, impact evaluations, and non-impact 
evaluations, is described below.

Performance Reporting

    The goal of performance reporting is to measure performance and 
track outcomes of the project's stated goals and objectives. The 
collection of accurate data on program performance is necessary for the 
reporting required by the Government Performance and Results Act 
(GPRA), which was passed by Congress in 1993 and updated through the 
GPRA Modernization Act of 2010. The latter will require even more 
frequent reporting--quarterly instead of annually.
    In addition to requiring grantees to collect annual data in support 
of GPRA reporting, the CSP SEA and VPSC programs encourage grantees to 
develop implementation and outcome measures in support of other program 
goals. Throughout this document, when we refer to grantee performance 
measures, we are referring to the measures grantees use not only for 
GPRA reporting, but also for reporting on other activities and 
outcomes.
    The CSP and VPSC programs provide technical assistance to grantees 
on developing appropriate objectives and performance measures and on 
obtaining quality data in support of those measures. Because all 
grantees conduct some kind of performance reporting, this study's 
examination of performance reporting encompasses all grantees. It will 
describe the type of technical assistance provided, categorize the 
types of performance measures that grantees address, determine whether 
the measures are responsive to the GPRA indicators defined for each 
program, review whether the initial set of performance measures changed 
as a result of the technical assistance received, and examine the 
quality and appropriateness of data collection for those measures.

Impact Evaluation

    While documenting implementation activities and outcomes can be 
useful to school and district administrators, it does not provide 
information on the effectiveness of funded interventions. The only 
evaluation designs that provide credible evidence about the impacts of 
interventions are rigorous experimental and quasi-experimental designs. 
Impact evaluations can provide guidance about what interventions should 
be considered for future funding and replication.
    This study will review the quality and rigor of all impact 
evaluations being conducted of higher-order outcomes, particularly 
student achievement, using criteria that were adapted from the What 
Works Clearinghouse review standards for grantees as part of the 
Department's Data Quality Initiative. These criteria were revised by 
Abt Associates as part of its annual review of Mathematics and Science 
Partnership final-year evaluations. The criteria are listed in Appendix 
A. The study will also examine the completeness and clarity of 
evaluation reports submitted as part of an impact evaluation.

Non-Impact Evaluation

    Grantees may choose to conduct non-impact evaluations to examine 
program outcomes and implementation processes. Non-impact evaluations 
may include both formative implementation and process evaluations that 
evaluate a program as it is unfolding, and summative descriptive 
evaluations that examine changes in final outcomes in a non-causal 
manner. A full framework of formative and summative evaluations is 
included in Appendix B.
    The main focus of the review of non-impact evaluations will be on 
those that focus on a change in higher-order outcomes using a one-group 
pre-post design. For these evaluations, the study will examine the 
appropriateness of data collection strategies for the design chosen and 
whether the findings of the study are described appropriately based on 
the design. The study will also describe other non-impact evaluations 
that grantees have undertaken, without commenting on their quality.
    Data collection, including conducting interviews and reviewing 
extant documents, is required to complete this study. Part A of this 
request discusses the justification for these data collection 
activities, while Part B describes the data collection and analysis 
procedures.

Darrin A. King,
Director, Information Collection Clearance Division, Privacy, 
Information and Records Management Services, Office of Management.
[FR Doc. 2012-14593 Filed 6-13-12; 8:45 am]
BILLING CODE 4000-01-P