Definitions and Selection Criteria That Apply to Direct Grant Programs, 35445-35451 [2017-15989]

Agencies

• DEPARTMENT OF EDUCATION

[Federal Register Volume 82, Number 145 (Monday, July 31, 2017)]
[Rules and Regulations]
[Pages 35445-35451]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2017-15989]


-----------------------------------------------------------------------

DEPARTMENT OF EDUCATION

34 CFR Parts 75 and 77

[Docket ID ED-2017-OII-0032]
RIN 1855-AA13


Definitions and Selection Criteria That Apply to Direct Grant 
Programs

AGENCY: Department of Education.

ACTION: Final rule with request for comments.

-----------------------------------------------------------------------

SUMMARY: The Secretary is issuing this rule in order to better align 
the regulations with the definition of ``evidence-based'' in the 
statutory authority. These changes mean that all competitive grant 
programs in the Department can continue to use the same provisions for 
evidence-based grant-making.

DATES: Effective date: These regulations are effective July 31, 2017. 
The incorporation by reference of certain publications listed in these 
regulations is approved by the Director of the Federal Register as of 
July 31, 2017.
    Comment due date: We will accept comments on or before August 30, 
2017. We will consider the comments received and may conduct additional 
rulemaking based on the comments.

ADDRESSES: Submit your comments through the Federal eRulemaking Portal 
or via postal mail, commercial delivery, or hand delivery. We will not 
accept comments submitted by fax or by email or those submitted after 
the comment period. To ensure that we do not receive duplicate copies, 
please submit your comments only once. In addition, please include the 
Docket ID at the top of your comments.
     Federal eRulemaking Portal: Go to www.regulations.gov to 
submit your comments electronically. Information on using 
Regulations.gov, including instructions for accessing agency documents, 
submitting comments, and viewing the docket, is available on the site 
under ``How to use Regulations.gov.''
     Postal Mail, Commercial Delivery, or Hand Delivery. If you 
mail or deliver your comments about these final regulations, address 
them to Kelly Terpak, U.S. Department of Education, 400 Maryland Avenue 
SW., Room 4W312, Washington, DC 20202-5900.
    Privacy Note: The Department's policy for comments received from 
members of the public is to make these submissions available for public 
viewing in their entirety on the Federal eRulemaking Portal at 
www.regulations.gov. Therefore, commenters should be careful to include 
in their comments only information that they wish to make publicly 
available.

FOR FURTHER INFORMATION CONTACT: Kelly Terpak, U.S. Department of 
Education, 400 Maryland Avenue SW., Room 4W312, Washington, DC 20202-
5900. Telephone: (202) 205-5231 or by email: kelly.terpak@ed.gov.
    If you use a telecommunications device for the deaf (TDD) or text 
telephone (TTY), call the Federal Relay Service (FRS), toll free, at 1-
800-877-8339.

SUPPLEMENTARY INFORMATION: As noted above, these regulations are 
effective on July 31, 2017. However, for grant award competitions 
announced by the Department in the Federal Register prior to the 
effective date of these regulations, unless the notice specifies 
otherwise, the provisions of 34 CFR parts 75 and 77 revised or removed 
through this notice of final regulations continue to apply to 
competitions and grants awarded under those notices inviting 
applications.

Invitation To Comment

    These regulations do not establish substantive policy changes, but 
instead make technical changes to existing regulations. Since these 
regulations make only technical changes, a comment period is not 
required. However, we are interested in whether you think we should 
make any changes in these regulations and thus we are inviting your 
comments. We will consider these comments in determining whether to 
make further technical changes to the regulations or engage in 
additional rulemaking. To ensure that your comments have maximum 
effect, we urge you to identify clearly the specific section or 
sections of the regulations that each of your comments addresses and to 
arrange your comments in the same order as the regulations. See 
ADDRESSES for instructions on how to submit comments.
    We invite you to assist us in complying with the specific 
requirements of Executive Orders 12866 and 13563 and their overall 
requirements of reducing regulatory burden that might result from these 
regulations. Please let us know of any additional ways we could reduce 
potential costs or increase potential benefits while preserving the 
effective and efficient administration of the Department's programs and 
activities.
    During and after the comment period, you may inspect all public 
comments about these regulations by accessing Regulations.gov. You may 
also inspect the comments in person in Room 6W245, 400 Maryland Avenue 
SW., Washington, DC, between the hours of 8:30 a.m. and 4:00 p.m., 
Eastern time, Monday through Friday of each week except Federal 
holidays. If you want to schedule time to inspect comments, please 
contact the person listed under FOR FURTHER INFORMATION CONTACT.
    Assistance to Individuals with Disabilities in Reviewing the 
Rulemaking Record: On request, we will provide an appropriate 
accommodation or auxiliary aid to an individual with a disability who 
needs assistance to review the comments or other documents in the 
public rulemaking record for these regulations. If you want to schedule 
an appointment for this type of aid, please contact the person listed 
under FOR FURTHER INFORMATION CONTACT.

Final Regulatory Changes

I. Selection Criteria

    Background: The regulations in subpart D of 34 CFR part 75 set 
forth the general requirements that govern the Department's selection 
of grantees for direct grant awards. For those direct grant programs 
that make discretionary grant awards, the Secretary uses selection 
criteria to evaluate applications submitted under those programs. The 
regulations establish a

[[Page 35446]]

menu of selection criteria that the Secretary may use in any Department 
discretionary grant competition.

34 CFR Part 75

Sec.  75.210 General Selection Criteria
    Current Regulations: Current Sec.  75.210(c) lists 29 factors under 
the ``Quality of the Project Design'' selection criterion. Section 
75.210(h) includes 12 factors under the ``Quality of the Project 
Evaluation'' selection criterion.
    Final Regulations and Reasons: We make the following changes to the 
selection criteria in Sec.  75.210(c) and (h):
    (1) Add one selection factor under the ``Quality of the Project 
Design'' criterion (Sec.  75.210(c)) to clarify that the Department may 
assess the extent to which an applicant's proposed project would 
represent a faithful adaptation of the evidence cited in support of its 
project. This factor is designed to assess whether projects would in 
fact implement the evidence cited as support, such that the project is 
``evidence-based'' as described in section 8101(21)(A) of the 
Elementary and Secondary Education Act (ESEA), as amended by the Every 
Student Succeeds Act (ESSA).
    (2) For clarification, add two selection factors under the 
``Quality of the Project Evaluation'' criterion (Sec.  75.210(h)) 
focused on (a) the qualifications of an applicant's evaluator; and (b) 
the sufficiency of resources to carry out the project evaluation.
    We also revise two factors under the ``Quality of the Project 
Design'' criterion (Sec.  75.210(c)) and four factors under the 
``Quality of the Project Evaluation'' criterion (Sec.  75.210(h)) to 
align terminology with the revised evidence definitions in 34 CFR part 
77. Specifically, the regulations:
    (1) Replace references to ``evidence of promise'' and ``strong 
theory'' with ``promising evidence'' and ``demonstrates a rationale,'' 
respectively.
    (2) Align terminology with the revised definitions in 34 CFR 
77.1(c) to include the term ``project component'' and clarify that the 
What Works Clearinghouse standards are described in the What Works 
Clearinghouse Handbook.
    We are making these revisions to improve the menu of selection 
criteria and factors by better aligning them to the evidence-related 
definitions in 34 CFR part 77. We make these revisions in conjunction 
with the amendments to the definitions in 34 CFR part 77, which, as 
discussed elsewhere in this document, we also revise to align with the 
evidence provisions in section 8101(21) of the ESEA, as amended by the 
ESSA, and for clarity. The final regulations do not change the way the 
Secretary uses the current and new selection criteria and factors. The 
Secretary will continue to use selection criteria that are consistent 
with the purpose of the program and permitted under the applicable 
statutes and regulations.

II. Evidence Preferences and Priorities

    Sec.  75.226 What procedures does the Secretary use if the 
Secretary decides to give special consideration to applications 
supported by strong evidence of effectiveness, moderate evidence of 
effectiveness, or evidence of promise?
    Current Regulations: Under Sec.  75.226, the Secretary may 
establish a competitive preference or absolute priority for projects 
supported by strong evidence of effectiveness, moderate evidence of 
effectiveness, or evidence of promise, as those terms are currently 
defined in 34 CFR part 77.
    Final Regulations and Reasons: The Secretary makes technical 
revisions to the title and text of this section to describe procedures 
for giving special consideration to applications supported by strong, 
moderate, or promising evidence, which are the evidence-related terms 
used in the ESEA. We include definitions for these terms elsewhere in 
this document.
    These technical changes ensure that discretionary grant programs 
authorized by the ESEA, as amended by the ESSA, can establish evidence-
based priorities under Sec.  75.226 and allow the Department the option 
to use one set of uniform evidence standards for all discretionary 
grant programs across each program's authorizing statute.

III. Evidence Definitions

    Background: Section 77.1(c) establishes definitions that, unless a 
statute or regulation provides otherwise, apply to the regulations in 
title 34 of the Code of Federal Regulations and can be used in 
Department grant competitions. This section includes a number of 
definitions that support the Department's use of evidence in grant 
competitions. The ESSA amended the ESEA to include a new definition of 
``evidence-based'' that necessitates changes to these definitions.

34 CFR Part 77

Sec.  77.1 Definitions That Apply to All Department Programs
    Current Regulations: Section 77.1(c) establishes definitions that, 
unless a statute or regulation provides otherwise, apply to the 
regulations in title 34 of the Code of Federal Regulations and can be 
used in Department grant competitions.
    Final Regulations and Reasons: We establish new, and revise some 
existing, definitions to (1) ensure alignment with provision in the 
ESEA, as amended by the ESSA, providing a single set of evidence 
definitions; and (2) make minor clarifying revisions to existing 
provisions. In these final regulations, we:
    (1) Add a definition of ``evidence-based'' that incorporates the 
four levels of evidence in section 8101(21)(A) of the ESEA, as amended 
by the ESSA.
    (2) Add a definition for ``project component'' as a single, 
clarifying term for what may be included in a project. The term 
clarifies that ``policy'' may be one component of a project; 
encompasses ``an activity, strategy, or intervention,'' to be 
consistent with the definition of ``evidence-based'' in section 
8101(21) of the ESEA, as amended by the ESSA; and includes ``process,'' 
``product,'' and ``practice,'' which were in the evidence definitions 
in 34 CFR 77.1(c) (e.g., strong evidence of effectiveness) prior to 
these final regulations.
    (3) Remove the definitions of ``large sample'' and ``multi-site 
sample'' and instead incorporate them into the new ``moderate 
evidence'' and ``strong evidence'' definitions, to streamline these 
definitions.
    (4) Replace the term ``strong theory'' with the term ``demonstrates 
a rationale,'' as this is the fourth level of evidence in the 
definition of ``evidence-based'' in section 8101(21) of the ESEA, as 
amended by the ESSA.
    (5) Replace the term ``evidence of promise'' with the term 
``promising evidence,'' to align with the definition of ``evidence-
based'' in section 8101(21) of the ESEA, as amended by the ESSA. In the 
definition of ``promising evidence,'' we clarify--
     How practice guides and intervention reports prepared by 
the What Works Clearinghouse (WWC), in alignment with the WWC standards 
incorporated in the definition, can provide promising evidence;
     How the Department already reviews single studies to 
determine whether they qualify under this level of evidence; and
     That certain quasi-experimental studies and experimental 
studies that do not meet WWC standards can qualify as promising 
evidence, as the previous ``evidence of promise'' definition implied.
     That correlational studies with statistical controls for 
selection bias must be well-designed and well-implemented to qualify as 
promising

[[Page 35447]]

evidence, as the ESEA, as amended by the ESSA, provides.
    (6) Replace the term ``moderate evidence of effectiveness'' with 
the term ``moderate evidence,'' which is used in the ESEA definition of 
``evidence-based.'' In the definition of ``moderate evidence,'' we 
clarify--
     How practice guides and intervention reports prepared by 
the WWC, in alignment with the WWC standards incorporated in the 
definition, can provide moderate evidence;
     How the Department already reviews single studies to 
determine whether they qualify under this level of evidence; and
     Through language regarding ``relevant findings,'' that 
there must be a link between the proposed activities, strategies, and 
interventions and specific statistically significant effects, as 
required under the definition of ``evidence-based'' in section 8101(21) 
of the ESEA, as amended by the ESSA.
    (7) Replace the term ``randomized controlled trial'' with the term 
``experimental study,'' to align with the definition of ``evidence-
based,'' in section 8101(21) specifically with regard to ``strong 
evidence.'' In this new definition of ``strong evidence,'' we clarify 
the types of studies that can qualify as experimental studies--
including, but not limited to, randomized controlled trials--as 
provided in the applicable WWC Handbook.
    (8) Replace the term ``strong evidence of effectiveness'' with the 
term ``strong evidence,'' which is used in the definition of 
``evidence-based'' in section 8101(21) of the ESEA, as amended by the 
ESSA. In the definition of ``strong evidence,'' we clarify--
     How practice guides and intervention reports prepared by 
the WWC, in alignment with the WWC standards incorporated in the 
definition, can provide promising evidence under the definition of 
``evidence-based'' in section 8101(21) of the ESEA, as amended by the 
ESSA;
     How the Department already reviews single studies to 
determine whether they qualify under this level of evidence; and
     Through language regarding ``relevant findings,'' that 
there must be a link between the proposed activities, strategies, and 
interventions and specific statistically significant effects, as 
required under the definition of ``evidence-based'' in section 8101(21) 
of the ESEA, as amended by the ESSA.
    (9) Replace the term ``What Works Clearinghouse Evidence 
Standards'' with the term ``What Works Clearinghouse Handbook,'' to 
clarify that the Handbook's procedures--not just standards--are 
relevant to evidence determinations, consistent with current practice. 
We also incorporate this Handbook, which provides a detailed 
description of the standards and procedures of the WWC, by reference. 
The WWC is an initiative of the U.S. Department of Education's National 
Center for Education Evaluation and Regional Assistance, within the 
Institute of Education Sciences (IES), which was established under the 
Education Sciences Reform Act of 2002. The WWC is an important part of 
IES's strategy to use rigorous and relevant research, evaluation, and 
statistics to inform decisions in the field of education. The WWC 
provides critical assessments of scientific evidence on the 
effectiveness of education programs, policies, products, and practices 
(referred to as ``interventions'') and a range of publications and 
tools summarizing this evidence. The WWC meets the need for credible, 
succinct information by reviewing research studies; assessing the 
quality of the research; summarizing the evidence of the effectiveness 
of programs, policies, products, and practices on student outcomes and 
other outcomes related to education; and disseminating its findings 
broadly. This Handbook is available to interested parties at the Web 
site address included in the regulation (https://ies.ed.gov/ncee/wwc/Handbooks).
    (10) Make minor clarifying changes to the definition of ``logic 
model'' so it is more easily understood.
    (11) Make minor clarifying changes to the definition of ``quasi-
experimental design study'' to align with terminology in the revised 
Sec.  77.1(c).
    (12) Make minor clarifying changes to the definition of ``relevant 
outcome'' to align with terminology in the revised Sec.  77.1(c).
Waiver of Proposed Rulemaking and Delayed Effective Date
    Under the Administrative Procedure Act (APA) (5 U.S.C. 553), the 
Department generally offers interested parties the opportunity to 
comment on proposed regulations. However, these regulations make 
technical changes only and do not establish substantive policy. The 
regulations are therefore exempt from notice and comment rulemaking 
under 5 U.S.C. 553(b)(3)(B). However, the Department is providing a 30-
day comment period and invites interested persons to participate in 
this rulemaking by submitting written comments. The Department will 
consider the comments received and may conduct additional rulemaking 
based on the comments.
    The APA also generally requires that regulations be published at 
least 30 days before their effective date, unless the agency has good 
cause to implement its regulations sooner (5 U.S.C. 553(d)(3)). Again, 
because these final regulations are merely technical, there is good 
cause to make them effective on the day they are published.

Executive Orders 12866, 13563, and 13771

Regulatory Impact Analysis

    Under Executive Order 12866, the Secretary must determine whether 
this regulatory action is ``significant'' and, therefore, subject to 
the requirements of the Executive order and subject to review by the 
Office of Management and Budget (OMB). Section 3(f) of Executive Order 
12866 defines a ``significant regulatory action'' as an action likely 
to result in a rule that may--
    (1) Have an annual effect on the economy of $100 million or more, 
or adversely affect a sector of the economy, productivity, competition, 
jobs, the environment, public health or safety, or State, local, or 
Tribal governments or communities in a material way (also referred to 
as an ``economically significant'' rule);
    (2) Create serious inconsistency or otherwise interfere with an 
action taken or planned by another agency;
    (3) Materially alter the budgetary impacts of entitlement grants, 
user fees, or loan programs or the rights and obligations of recipients 
thereof; or
    (4) Raise novel legal or policy issues arising out of legal 
mandates, the President's priorities, or the principles stated in the 
Executive order.
    This final regulatory action is not a significant regulatory action 
subject to review by OMB under section 3(f) of Executive Order 12866.
    Under Executive Order 13771, for each new regulation that the 
Department proposes for notice and comment or otherwise promulgates 
that is a significant regulatory action under Executive Order 12866, it 
must identify two deregulatory actions. For Fiscal Year 2017, any new 
incremental costs associated with a new regulation must be fully offset 
by the elimination of existing costs through deregulatory actions. The 
final regulations are not a significant regulatory action. Therefore, 
the requirements of Executive Order 13771 do not apply.
    We have also reviewed these regulations under Executive Order 
13563, which supplements and explicitly reaffirms the principles, 
structures, and definitions governing regulatory review established in

[[Page 35448]]

Executive Order 12866. To the extent permitted by law, Executive Order 
13563 requires that an agency--
    (1) Propose or adopt regulations only on a reasoned determination 
that their benefits justify their costs (recognizing that some benefits 
and costs are difficult to quantify);
    (2) Tailor its regulations to impose the least burden on society, 
consistent with obtaining regulatory objectives and taking into 
account--among other things and to the extent practicable--the costs of 
cumulative regulations;
    (3) In choosing among alternative regulatory approaches, select 
those approaches that maximize net benefits (including potential 
economic, environmental, public health and safety, and other 
advantages; distributive impacts; and equity);
    (4) To the extent feasible, specify performance objectives, rather 
than the behavior or manner of compliance a regulated entity must 
adopt; and
    (5) Identify and assess available alternatives to direct 
regulation, including economic incentives--such as user fees or 
marketable permits--to encourage the desired behavior, or provide 
information that enables the public to make choices.
    Executive Order 13563 also requires an agency ``to use the best 
available techniques to quantify anticipated present and future 
benefits and costs as accurately as possible.'' The Office of 
Information and Regulatory Affairs of OMB has emphasized that these 
techniques may include ``identifying changing future compliance costs 
that might result from technological innovation or anticipated 
behavioral changes.''
    We are issuing these final regulations only on a reasoned 
determination that their benefits justify their costs. In choosing 
among alternative regulatory approaches, we selected those approaches 
that maximize net benefits. Based on an analysis of anticipated costs 
and benefits, the Department believes that these final regulations are 
consistent with the principles in Executive Order 13563.
    We also have determined that this regulatory action does not unduly 
interfere with State, local, and Tribal governments in the exercise of 
their governmental functions.

Potential Costs and Benefits

    Under Executive Order 12866, we have assessed the potential costs 
and benefits of this regulatory action and have determined that these 
regulations would not impose additional costs. We believe any 
additional costs imposed by these final regulations will be negligible, 
primarily because they reflect technical changes which do not impose 
additional burden. Moreover, we believe any costs will be significantly 
outweighed by the potential benefits of making necessary clarifications 
and ensuring consistency among the Education Department General 
Administrative Regulations and section 8101(21) of ESEA, as amended by 
the ESSA.

Clarity of the Regulations

    Executive Order 12866 and the Presidential memorandum ``Plain 
Language in Government Writing'' require each agency to write 
regulations that are easy to understand.
    The Secretary invites comments on how to make these regulations 
easier to understand, including answers to questions such as the 
following:
     Are the requirements in the regulations clearly stated?
     Do the regulations contain technical terms or other 
wording that interferes with their clarity?
     Does the format of the regulations (grouping and order of 
sections, use of headings, paragraphing, etc.) aid or reduce their 
clarity?
     Would the regulations be easier to understand if we 
divided them into more (but shorter) sections? (A ``section'' is 
preceded by the symbol ``Sec.  '' and a numbered heading; for example, 
Sec.  75.210.)
     Could the description of the regulations in the 
SUPPLEMENTARY INFORMATION section of this preamble be more helpful in 
making the regulations easier to understand? If so, how?
     What else could we do to make the regulations easier to 
understand?
    To send any comments that concern how the Department could make 
these regulations easier to understand, see the instructions in the 
ADDRESSES section.

Regulatory Flexibility Act Certification

    The Secretary certifies that these regulations do not have a 
significant economic impact on a substantial number of small entities.

Paperwork Reduction Act of 1995

    The Paperwork Reduction Act of 1995 does not require you to respond 
to a collection of information unless it displays a valid OMB control 
number. We display the valid OMB control number assigned to a 
collection of information in final regulations at the end of the 
affected section of the regulations.

Intergovernmental Review

    This program is not subject to Executive Order 12372 and the 
regulations in 34 CFR part 79.
    Accessible Format: Individuals with disabilities can obtain this 
document in an accessible format (e.g., braille, large print, 
audiotape, or compact disc) on request to the program contact person 
listed under FOR FURTHER INFORMATION CONTACT.
    Electronic Access to This Document: The official version of this 
document is the document published in the Federal Register. Free 
internet access to the official edition of the Federal Register and the 
Code of Federal Regulations is available via the Federal Digital System 
at: www.gpo.gov/fdsys. At this site you can view this document, as well 
as all other documents of this Department published in the Federal 
Register, in text or Portable Document Format (PDF). To use PDF you 
must have Adobe Acrobat Reader, which is available free at the site.
    You may also access documents of the Department published in the 
Federal Register by using the article search feature at: 
www.federalregister.gov. Specifically, through the advanced search 
feature at this site, you can limit your search to documents published 
by the Department.

List of Subjects

34 CFR Part 75

    Accounting, Copyright, Education, Grant programs--education, 
Inventions and patents, Private schools, Reporting and recordkeeping 
requirements, Youth organizations.

34 CFR Part 77

    Education, Grant programs--education, Incorporation by reference.

    Dated: July 25, 2017.
Betsy DeVos,
Secretary of Education.

    For the reasons discussed in the preamble, the Secretary amends 
parts 75 and 77 of title 34 of the Code of Federal Regulations as 
follows:

PART 75--DIRECT GRANT PROGRAMS

0
1. The authority citation for part 75 continues to read as follows:

    Authority: 20 U.S.C. 1221e-3 and 3474, unless otherwise noted.

0
2. Section 75.210 is amended by:
0
a. Revising paragraphs (c)(2)(xxviii) and (xxix);
0
b. Adding paragraph (c)(2)(xxx); and
0
c. Revising paragraphs (h)(2)(viii) through (xii); and
0
d. Adding paragraph (h)(2)(xiii).
    The revisions and addition read as follows:

[[Page 35449]]

Sec.  75.210   General selection criteria.

* * * * *
    (c) * * *
    (2) * * *
    (xxviii) The extent to which the proposed project is supported by 
promising evidence (as defined in 34 CFR 77.1(c)).
    (xxix) The extent to which the proposed project demonstrates a 
rationale (as defined in 34 CFR 77.1(c)).
    (xxx) The extent to which the proposed project represents a 
faithful adaptation of the evidence cited in support of the proposed 
project.
* * * * *
    (h) * * *
    (2) * * *
    (viii) The extent to which the methods of evaluation will, if well 
implemented, produce evidence about the project's effectiveness that 
would meet the What Works Clearinghouse standards without reservations 
as described in the What Works Clearinghouse Handbook (as defined in 34 
CFR 77.1(c)).
    (ix) The extent to which the methods of evaluation will, if well 
implemented, produce evidence about the project's effectiveness that 
would meet the What Works Clearinghouse standards with or without 
reservations as described in the What Works Clearinghouse Handbook (as 
defined in 34 CFR 77.1(c)).
    (x) The extent to which the methods of evaluation will, if well 
implemented, produce promising evidence (as defined in 34 CFR 77.1(c)) 
about the project's effectiveness.
    (xi) The extent to which the evaluation plan clearly articulates 
the key project components, mediators, and outcomes, as well as a 
measurable threshold for acceptable implementation.
    (xii) The qualifications, including relevant training, experience, 
and independence, of the evaluator.
    (xiii) The extent to which the proposed project plan includes 
sufficient resources to conduct the project evaluation effectively.
* * * * *

0
3. Revise Sec.  75.226 to read as follows:


Sec.  75.226  What procedures does the Secretary use if the Secretary 
decides to give special consideration to applications supported by 
strong, moderate, or promising evidence?

    (a) As used in this section, ``strong evidence'' is defined in 34 
CFR 77.1(c).
    (b) As used in this section, ``moderate evidence'' is defined in 34 
CFR 77.1(c).
    (c) As used in this section, ``promising evidence'' is defined in 
34 CFR 77.1(c).
    (d) If the Secretary determines that special consideration of 
applications supported by strong, moderate, or promising evidence is 
appropriate, the Secretary may establish a separate competition under 
the procedures in 34 CFR 75.105(c)(3), or provide competitive 
preference under the procedures in 34 CFR 75.105(c)(2), for 
applications supported by--
    (1) Evidence that meets the conditions in the definition of 
``strong evidence'';
    (2) Evidence that meets the conditions in the definition of 
``moderate evidence''; or
    (3) Evidence that meets the conditions in the definition of 
``promising evidence.''

PART 77--DEFINITIONS THAT APPLY TO DEPARTMENT REGULATIONS

0
3. The authority citation for part 77 continues to read as follows:

    Authority: 20 U.S.C. 1221e-3 and 3474, unless otherwise noted.


0
4. Section 77.1(c) is amended by:
0
a. Adding, in alphabetical order, a definition for ``Demonstrates a 
rationale''.
0
b. Removing the definition of ``Evidence of promise''.
0
c. Adding, in alphabetical order, definitions for ``Evidence-based'' 
and ``Experimental study''.
0
d. Removing the definition of ``Large sample''.
0
e. Revising the definition of ``Logic model''.
0
f. Adding, in alphabetical order, a definition for ``Moderate 
evidence''.
0
g. Removing the definitions of ``Moderate evidence of effectiveness'' 
and ``Multi-site sample''.
0
h. Adding, in alphabetical order, definitions for ``Project component'' 
and ``Promising evidence''.
0
i. Revising the definitions of ``Quasi-experimental design study'' and 
``Relevant outcome''.
0
j. Adding, in alphabetical order, a definition for ``Strong evidence''.
0
k. Removing the definitions of ``Strong evidence of effectiveness'', 
``Strong theory'', and ``What Works Clearinghouse Evidence Standards''.
0
l. Adding, in alphabetical order, a definition for ``What Works 
Clearinghouse Handbook''.
    The additions and revisions read as follows:


Sec.  77.1  Definitions that apply to all Department programs.

* * * * *
    (c) * * *
    Demonstrates a rationale means a key project component included in 
the project's logic model is informed by research or evaluation 
findings that suggest the project component is likely to improve 
relevant outcomes.
* * * * *
    Evidence-based means the proposed project component is supported by 
one or more of strong evidence, moderate evidence, promising evidence, 
or evidence that demonstrates a rationale.
    Experimental study means a study that is designed to compare 
outcomes between two groups of individuals (such as students) that are 
otherwise equivalent except for their assignment to either a treatment 
group receiving a project component or a control group that does not. 
Randomized controlled trials, regression discontinuity design studies, 
and single-case design studies are the specific types of experimental 
studies that, depending on their design and implementation (e.g., 
sample attrition in randomized controlled trials and regression 
discontinuity design studies), can meet What Works Clearinghouse (WWC) 
standards without reservations as described in the WWC Handbook:
    (i) A randomized controlled trial employs random assignment of, for 
example, students, teachers, classrooms, or schools to receive the 
project component being evaluated (the treatment group) or not to 
receive the project component (the control group).
    (ii) A regression discontinuity design study assigns the project 
component being evaluated using a measured variable (e.g., assigning 
students reading below a cutoff score to tutoring or developmental 
education classes) and controls for that variable in the analysis of 
outcomes.
    (iii) A single-case design study uses observations of a single case 
(e.g., a student eligible for a behavioral intervention) over time in 
the absence and presence of a controlled treatment manipulation to 
determine whether the outcome is systematically related to the 
treatment.
* * * * *
    Logic model (also referred to as a theory of action) means a 
framework that identifies key project components of the proposed 
project (i.e., the active ``ingredients'' that are hypothesized to be 
critical to achieving the relevant outcomes) and describes the 
theoretical and operational relationships among the key project 
components and relevant outcomes.
* * * * *
    Moderate evidence means that there is evidence of effectiveness of 
a key project component in improving a relevant outcome for a sample 
that overlaps with the populations or settings proposed to receive that 
component, based on a relevant finding from one of the following:

[[Page 35450]]

    (i) A practice guide prepared by the WWC using version 2.1 or 3.0 
of the WWC Handbook reporting a ``strong evidence base'' or ``moderate 
evidence base'' for the corresponding practice guide recommendation;
    (ii) An intervention report prepared by the WWC using version 2.1 
or 3.0 of the WWC Handbook reporting a ``positive effect'' or 
``potentially positive effect'' on a relevant outcome based on a 
``medium to large'' extent of evidence, with no reporting of a 
``negative effect'' or ``potentially negative effect'' on a relevant 
outcome; or
    (iii) A single experimental study or quasi-experimental design 
study reviewed and reported by the WWC using version 2.1 or 3.0 of the 
WWC Handbook, or otherwise assessed by the Department using version 3.0 
of the WWC Handbook, as appropriate, and that--
    (A) Meets WWC standards with or without reservations;
    (B) Includes at least one statistically significant and positive 
(i.e., favorable) effect on a relevant outcome;
    (C) Includes no overriding statistically significant and negative 
effects on relevant outcomes reported in the study or in a 
corresponding WWC intervention report prepared under version 2.1 or 3.0 
of the WWC Handbook; and
    (D) Is based on a sample from more than one site (e.g., State, 
county, city, school district, or postsecondary campus) and includes at 
least 350 students or other individuals across sites. Multiple studies 
of the same project component that each meet requirements in paragraphs 
(iii)(A), (B), and (C) of this definition may together satisfy this 
requirement.
* * * * *
    Project component means an activity, strategy, intervention, 
process, product, practice, or policy included in a project. Evidence 
may pertain to an individual project component or to a combination of 
project components (e.g., training teachers on instructional practices 
for English learners and follow-on coaching for these teachers).
* * * * *
    Promising evidence means that there is evidence of the 
effectiveness of a key project component in improving a relevant 
outcome, based on a relevant finding from one of the following:
    (i) A practice guide prepared by WWC reporting a ``strong evidence 
base'' or ``moderate evidence base'' for the corresponding practice 
guide recommendation;
    (ii) An intervention report prepared by the WWC reporting a 
``positive effect'' or ``potentially positive effect'' on a relevant 
outcome with no reporting of a ``negative effect'' or ``potentially 
negative effect'' on a relevant outcome; or
    (iii) A single study assessed by the Department, as appropriate, 
that--
    (A) Is an experimental study, a quasi-experimental design study, or 
a well-designed and well-implemented correlational study with 
statistical controls for selection bias (e.g., a study using regression 
methods to account for differences between a treatment group and a 
comparison group); and
    (B) Includes at least one statistically significant and positive 
(i.e., favorable) effect on a relevant outcome.
* * * * *
    Quasi-experimental design study means a study using a design that 
attempts to approximate an experimental study by identifying a 
comparison group that is similar to the treatment group in important 
respects. This type of study, depending on design and implementation 
(e.g., establishment of baseline equivalence of the groups being 
compared), can meet WWC standards with reservations, but cannot meet 
WWC standards without reservations, as described in the WWC Handbook.
* * * * *
    Relevant outcome means the student outcome(s) or other outcome(s) 
the key project component is designed to improve, consistent with the 
specific goals of the program.
* * * * *
    Strong evidence means that there is evidence of the effectiveness 
of a key project component in improving a relevant outcome for a sample 
that overlaps with the populations and settings proposed to receive 
that component, based on a relevant finding from one of the following:
    (i) A practice guide prepared by the WWC using version 2.1 or 3.0 
of the WWC Handbook reporting a ``strong evidence base'' for the 
corresponding practice guide recommendation;
    (ii) An intervention report prepared by the WWC using version 2.1 
or 3.0 of the WWC Handbook reporting a ``positive effect'' on a 
relevant outcome based on a ``medium to large'' extent of evidence, 
with no reporting of a ``negative effect'' or ``potentially negative 
effect'' on a relevant outcome; or
    (iii) A single experimental study reviewed and reported by the WWC 
using version 2.1 or 3.0 of the WWC Handbook, or otherwise assessed by 
the Department using version 3.0 of the WWC Handbook, as appropriate, 
and that--
    (A) Meets WWC standards without reservations;
    (B) Includes at least one statistically significant and positive 
(i.e., favorable) effect on a relevant outcome;
    (C) Includes no overriding statistically significant and negative 
effects on relevant outcomes reported in the study or in a 
corresponding WWC intervention report prepared under version 2.1 or 3.0 
of the WWC Handbook; and
    (D) Is based on a sample from more than one site (e.g., State, 
county, city, school district, or postsecondary campus) and includes at 
least 350 students or other individuals across sites. Multiple studies 
of the same project component that each meet requirements in paragraphs 
(iii)(A), (B), and (C) of this definition may together satisfy this 
requirement.
* * * * *
    What Works Clearinghouse Handbook (WWC Handbook) means the 
standards and procedures set forth in the WWC Procedures and Standards 
Handbook, Version 3.0 or Version 2.1 (incorporated by reference, see 34 
CFR 77.2). Study findings eligible for review under WWC standards can 
meet WWC standards without reservations, meet WWC standards with 
reservations, or not meet WWC standards. WWC practice guides and 
intervention reports include findings from systematic reviews of 
evidence as described in the Handbook documentation.
* * * * *

0
5. Section 77.2 is added to read as follows:


Sec.  77.2   Incorporation by Reference.

    (a) Certain material is incorporated by reference into this part 
with the approval of the Director of the Federal Register under 5 
U.S.C. 552(a) and 1 CFR part 51. All approved material is available for 
inspection at Institute of Education Sciences, National Center for 
Education Evaluation and Regional Assistance by email at 
Contact.WWC@ed.gov, and is available from the sources listed below. It 
is also available for inspection at the National Archives and Records 
Administration (NARA). For information on the availability of this 
material at NARA, call 202-741-6030 or go to www.archives.gov/federal_register/code_of_federal_regulations/ibr_locations.html.
    (b) Institute of Education Sciences, 550 12th Street SW., 
Washington, DC 20202, (202) 245-6940, https://ies.ed.gov/ncee/wwc/Handbooks.
    (1) What Works Clearinghouse Procedures and Standards Handbook,

[[Page 35451]]

Version 3.0, March 2014, IBR approved for Sec.  77.1.
    (2) What Works Clearinghouse Procedures and Standards Handbook, 
Version 2.1, September 2011, IBR approved for Sec.  77.1.

[FR Doc. 2017-15989 Filed 7-27-17; 4:15 pm]
BILLING CODE 4000-01-P