Current through Reg. 50, No. 187; September 24, 2024
(1)
Purpose. The purpose of this rule is to provide districts choosing to use the
formulas for measuring student learning growth approved by the commissioner
with a framework for using and interpreting scores in school district
instructional personnel evaluation systems developed under Section
1012.34, F.S., and to provide
information for use in the approval of school leader preparation programs under
Section 1012.562, F.S.
(2) Definitions. For the purposes of this
rule, the following definitions apply.
(a)
"Confidence interval." A confidence interval is derived from the standard
error. It expresses the precision of a statistic as a range of values. An
individual teacher's VAM score is an estimate of that teacher's contributions
to student learning growth. The 95% confidence interval used in classification
represents a range of possible values that would include the teacher's VAM
score 95% of the time if VAM scores were repeatedly re-estimated with different
students for each teacher.
(b)
"Courses associated with statewide, standardized assessments" or "courses
associated with statewide, standardized assessments under Section
1008.22, F.S.," are those
courses which are assessed by statewide, standardized assessments and are
listed in the publication, "Florida VAM Course List," (effective April 2024),
which is incorporated herein by reference (
http://www.flrules.org/Gateway/reference.asp?No=Ref-16550).
A copy of the Florida VAM Course List may be obtained from the Florida
Department of Education, 325 West Gaines Street, Room 514, Tallahassee, FL
32399-0400.
(c) "Covariate." A
covariate is a variable or set of variables reflecting measured characteristics
used in computing a statistical model that controls for specific influences on
the outcome being modeled.
(d)
"Covariate adjustment model." A covariate adjustment model is a statistical
model that controls for the influence of one or more of the
covariates.
(e) "Expected score."
An expected score generated by a value-added model for a statewide,
standardized assessment is based on the student's prior statewide, standardized
assessment score history and covariates, as well as how other students in the
state actually performed on the assessment. For each individual student, the
expected score is the sum across all covariates of the value of the covariate
multiplied by that covariate's contribution to student learning as estimated by
the covariate adjustment model.
(f)
"Observed score." An observed score is the actual score a student received on
an assessment.
(g) "Staff
information system." The staff information system is the comprehensive
management information system maintained by the Department containing staff
data reported by school districts in accordance with Rule
6A-1.0014, F.A.C., (
http://www.flrules.org/Gateway/reference.asp?No=Ref-05723)
which is incorporated by reference herein. A copy of Rule
6A-1.0014, F.A.C., may be
obtained from the Florida Department of Education, 325 West Gaines Street, Room
514, Tallahassee, FL 32399-0400.
(h) "Standard error." A standard error is a
measure of the precision of a statistic. It is determined by both sample size
and sample variability.
(i)
"Student information system." The student information system is the
comprehensive management information system maintained by the Department
containing student data reported by school districts in accordance with Rule
6A-1.0014, F.A.C., as
incorporated in paragraph (2)(g) of this rule.
(j) "Value-added model" or "VAM." A
value-added model is a statistical model used for the purpose of determining an
individual teacher's contribution to student learning growth.
(3) Formulas for measuring student
learning growth.
(a) The English Language Arts
and Mathematics value-added models.
1. The
formula for measuring student learning growth using student English Language
Arts and Mathematics results approved by the commissioner is a covariate
adjustment value-added model.
The value-added model statistically establishes the expected
learning growth for each student, called an expected score. When a student's
actual performance differs from these expectations, a portion of that
difference is attributed to the teacher's and a portion is attributed to the
school's influence. Together, this information is used to compute a teacher's
value-added score. School value-added scores are the average of the teacher
value-added scores within the school, and are provided to districts that choose
to use them for the performance of students component of administrator
evaluations required under Section 1012.34(3)(a)1., F.S.
2. The formula for the model. A full
technical description of the data sources, formula, covariates, and methodology
for calculating VAM scores is provided in the publication, "Florida VAM
Methodology" (Effective August 2015) (
http://www.flrules.org/Gateway/reference.asp?No=Ref-05724),
which is incorporated herein by reference. A copy of "Florida VAM Methodology"
may be obtained from the Florida Department of Education, 325 West Gaines
Street, Room 514, Tallahassee, FL 32399-0400.
3. The covariates included in the value-added
model approved by the commissioner are:
a. The
number of subject-relevant courses in which a student is enrolled.
b. At least one (1) and up to two (2) prior
years of achievement scores on the applicable statewide, standardized
assessment for each student.
c. A
student's disabilities. The disabilities used within the model are limited to
language impaired; deaf or hard of hearing; visually impaired;
emotional/behavioral disabilities; specific learning disability; dual sensory
impaired; autism spectrum disorder; traumatic brain injured; other health
impaired; and other intellectual disability.
d. A student's English Language Learner (ELL)
status. This covariate is used to control for effects related to whether a
student is an English language learner and has been receiving English for
Speakers of Other Languages (ESOL) services for less than two (2) years; at
least two (2) years but less than four (4) years; at least four (4) years but
less than six (6) years; or six (6) years or longer.
e. A student's gifted status.
f. Student attendance.
g. Student mobility. This covariate is used
to control for effects related to changing schools during the school
year.
h. Difference from modal age
in grade. This covariate is used to control for effects related to differences
in a student's age from the most common age for students enrolled in the same
grade across the state.
i. Class
size. This covariate is used to control for effects related to the number of
students in a class.
j. Homogeneity
of students' entering test scores in the class. This covariate is used to
control for the variation in student proficiency within a classroom at the
beginning of the year.
4.
The formula produces a value-added score for a teacher. For English Language
Arts and Mathematics, this value-added score consists of two (2) parts:
a. The teacher effect. The teacher effect is
an estimate of a teacher's contributions to student achievement as measured by
scores on statewide, standardized assessments. It is based on the difference
between expected scores and actual scores for a teacher's students relative to
other teachers in the school, among students assessed in the same subject at
the same grade level during the same year.
b. The school component. The school component
is an estimate of the part of a student's performance that is common to
students within a school. It is based on the difference between expected scores
and actual scores for the school's students relative to other schools in the
state, among students assessed in the same subject at the same grade level
during the same year. It represents school-level factors influencing
performance of all students in a school among students assessed in the same
subject at the same grade level during the same year. Fifty (50) percent of the
school component shall be added to the teacher effect to create the teacher's
value-added score.
(b) The Algebra I value-added models.
1. The formula for measuring student learning
growth using student results from the statewide, standardized end-of-course
assessment in Algebra I pursuant to Section
1008.22, F.S., approved by the
commissioner is a covariate adjustment value-added model.
The value-added model statistically establishes the expected
learning growth for each student, called an expected score. When a student's
actual performance differs from these expectations, a portion of that
difference is attributed to the teacher's and a portion is attributed to the
school's influence. Together, this information is used to compute a teacher's
value-added score. School value-added scores are the average of the teacher
value-added scores within the school, and are provided for districts that
choose to use them for the performance of students component of administrator
evaluations required under Section
1012.34 (3)(a)1.,
F.S.
2. The formula for the
model. A full technical description of the data sources, formula, covariates,
and methodology for calculating VAM scores is provided in the publication,
Florida VAM Methodology.
3. The
covariates included in the Algebra I value-added model approved by the
commissioner include those listed under sub-subparagraphs (3)(a)3.a.-j., as
well as the following:
a. Mean prior test
score. Mean prior test score is the average of the most recent prior score on
the statewide, standardized assessment in Mathematics for all students within
the class.
b. Percent of students
in the class who are reported in the student information system as
Gifted.
c. Percent at modal age in
grade. Percent at modal age in grade is the percentage of students in the class
whose age on September 1 of the school year is the same as the modal age of all
students in the same grade.
4. The formula produces a value-added score
for a teacher. For Algebra I, the score is the teacher effect. The teacher
effect is an estimate of a teacher's contributions to student achievement as
measured by scores on statewide, standardized assessments. It is based on the
difference between expected scores and actual scores for a teacher's students
relative to other teachers in the school, among students assessed in the same
subject at the same grade level during the same year.
(4) Data Collected and Reported
for VAM.
(a) Data collection and reporting
procedures for purposes of VAM calculations shall be as provided in the
publication, Florida VAM Methodology. As set forth in this publication in more
detail, data from the Student Information System and Staff Information System
obtained from Surveys 2 and 3 are used in VAM calculations. School districts
submit Survey 2 and 3 data to the Department's Student Information System and
Staff Information System pursuant to Rule
6A-1.0014, F.A.C. (Comprehensive
Management Information System) and Rule
6A-1.0451, F.A.C. (Florida
Education Finance Program Student Membership Surveys) (
http://www.flrules.org/Gateway/reference.asp?No=Ref-05725).
These rules are incorporated by reference herein, and a copy of the rules may
be obtained from the Florida Department of Education, 325 West Gaines Street,
Room 514, Tallahassee, FL 32399-0400.
(b) Results provided to districts shall
include the following information for each statewide, standardized assessment
for which a formula has been adopted:
1. A
value-added score for each teacher and administrator based on the statewide,
standardized assessment associated with the course(s) that the teacher taught
during the current year or the school the administrator was assigned to during
the current year. This score shall be reported for each grade level and subject
area covered by the statewide assessment.
2. Three-year aggregate value-added scores
for each teacher, which includes data for the teacher from the current school
year and each of the two (2) prior years for which data are available, for a
total of at least one (1) and up to three (3) years of data for the teacher, as
follows:
a. Three-year aggregate English
Language Arts score. A combination of all value-added results for the teacher
from all grades and courses associated with the statewide, standardized
assessments in English Language Arts.
b. Three-year aggregate Mathematics score. A
combination of all value-added results for the teacher from all grades and
courses associated with the statewide, standardized assessments in
Mathematics.
c. Three-year
aggregate combined score. A combination of all value-added results associated
with the statewide, standardized assessments in English Language Arts and
Mathematics.
d. The aggregate score
shall be calculated by standardizing the value-added scores by converting them
to a proportion of a year's average growth within the grade and subject for the
year, and combining them across all grades and subjects for as many of the last
three (3) years as data are available. For districts choosing to use school
score(s) for administrator evaluations, districts may elect to combine multiple
one (1) year, two (2) year, and three (3) year aggregate scores that reflect
the school(s) to which the administrator was assigned and the years to which
they were assigned to those school(s) during the period.
3. The standard error for each value-added
score.
4. For each value-added
score that contains a school component, the school component and teacher
component reported separately.
(5) Classifying and Interpreting Scores.
(a) Data elements used by the department to
classify and interpret scores are as follows:
1. The statewide average year's growth for
students in each grade and subject. For each student learning growth formula,
an average year's growth for students across the state on the statewide
assessment is calculated, and once standardized, uses a threshold of zero (0)
to establish performance expectations. A score of zero (0) indicates that a
teacher's students scored no higher or lower, on average, than
expected.
2. The educator's
value-added score.
3. The
confidence interval. A confidence interval is computed using the standard error
associated with the educator's value-added score.
(b) Ratings. Districts may use the rating
provided by the department for the performance of students criterion in
performance evaluations under Section 1012.34, F.S., for classroom teachers of
courses associated with statewide, standardized assessments and administrators.
1. Ratings for Florida's value-added models.
The Department provides ratings for grade-specific and aggregate scores from
the English Language Arts, Mathematics, and Algebra I value-added
models.
2. The ratings for the
English Language Arts, Mathematics, and Algebra I value-added models are as
follows:
a. Highly Effective. A highly
effective rating is demonstrated by a value-added score of greater than zero
(0), where all of the scores contained within the associated 95-percent
confidence interval also lie above zero (0).
b. Effective. An effective rating is
demonstrated by the following:
(I) A
value-added score of zero (0);
(II)
A value-added score of greater than zero (0), where some portion of the range
of scores associated with a 95-percent confidence interval lies at or below
zero (0); or
(III) A value-added
score of less than zero (0), where some portion of the range of scores
associated with both the 68-percent and the 95-percent confidence interval lies
at or above zero (0).
c.
Needs Improvement, or Developing if the teacher has been teaching for fewer
than three (3) years. A needs improvement or developing rating is demonstrated
by a value-added score that is less than zero (0), where the entire 68-percent
confidence interval falls below zero (0), but where a portion of the 95-percent
confidence interval lies above zero (0).
d. Unsatisfactory. An unsatisfactory rating
is demonstrated by a value-added score of less than zero (0), where all of the
scores contained within the 95-percent confidence interval also lie below zero
(0).
Rulemaking Authority
1012.34 FS. Law Implemented
1012.34
FS.
New 9-9-15, Amended 4-30-18,
3-15-22.