ome subscribers to Phys-L might be interested in a recent post
"Value-Added Inequities: Should Value-Added Measures Be Used to
Evaluate Teachers?" [Hake (2011b)]. The abstract reads:
************************************************
ABSTRACT: In a previous post "The Ceiling Effect #2" [Hake (2011a)] I
noted that Sheila Tobias (2011), in a recent APS News editorial
"Teachers in the Crosshairs. . .", called attention to the
"Value-Added Inequity" (VAI) experienced by a Houston high-school
physics teacher who reported that her students entered her course
with high test scores. As a consequence her students achieved
relatively small pre-to-posttest gains (a consequence of the ceiling
effect) and she, in turn, received a relatively small value-added
bonus.
EDDRA2's Michael Martin responded that the Houston teacher's
Value-Added Inequity" (VAI) is "best appreciated as a grossly
unsophisticated process being employed by fundamentally incompetent
administrators." But EdResMeth's Tony Milanowski wrote that, as he
understood it, the model used in Houston was based on the report "SAS
EVAAS Statistical Models" [Wright et al., 2010 at
<http://bit.ly/hPrO7s>] which converts Item-Response-Theory-based
scale scores to Normal-Curve-Equivalents - but such sophistication
evidently did not prevent the VAI experienced by the Houston
high-school physics teacher.
Other VAI's were described by PhysLrnR's Boris Korsunsky and
Math-Teach's Haim. Boris wrote (paraphrasing): "My own bonus was
zero. Massachusetts used a value added method that took into account
a town's average parental income. Since my town is by far the
wealthiest in the state, we are often ranked 'below expectations' in
various state-produced rankings - even though Boston magazine has
repeatedly ranked us No.1 in the state. Similarly, Haim wrote
(paraphrasing): "The value-added formula for evaluating schools in
NYC resulted in Stuyvesant High School (one of the top academic
institutions in the U.S.) earning a 'B'."
Consistent with the above VAI's, reports by recognized experts
critical of the use of value added measures to grade teachers are:
(a)"Letter Report to the U.S. Department of Education on the Race to
the Top Fund" [NRC (2009)], (b) "Getting Value Out of Value-Added:
Report of a Workshop"[NRC (2010)], (c) "Problems With The Use of
Student Test Scores to Evaluate Teachers" [EPI (2010)], (d) "Grading
teachers on value-added measures falls short" [UCLA Today (2010)],
(e) "Error Rates in Measuring Teacher and School Performance Based on
Student Test Score Gains" [Schochet & Chiang (2010)], (f) "Hurdles
Emerge in Rising Effort to Rate Teachers"[Otterman (2010)].
Despite the above criticism, the Department of Education has designed
its "Race to the Top" scoring system to reward states that use
value-added calculations in teacher evaluations.
************************************************
"The [Race to the Top] initiative should support research based on
data that links student test scores with their teachers, but should
not prematurely promote the use of value-added approaches (which
evaluate teachers based on gains in their students' performance) to
reward or punish teachers."
"Letter Report to the U.S. Dept. of Education on the Race to the Top Fund"
[NRC (2009)]
REFERENCES [URL's shortened by <http://bit.ly/> and accessed on 18 Jan 2011.]
Hake, R.R. 2011a. "The Ceiling Effect #2" online on the OPEN! AERA-L
archives at <http://bit.ly/hUnHZe>. Post of 12 Jan 2011 16:19:49-0800
to AERA-L and Net-Gold. The abstract and link to the complete post
are being transmitted to various discussion lists are also online on
my blog "Hake'sEdStuff" at <http://bit.ly/gLWr7W> with a provision
for comments.
Hake, R.R. 2011b. "Value-Added Inequities: Should Value-Added
Measures Be Used to Evaluate Teachers?" online on the OPEN! AERA-L
archives at <http://bit.ly/fN1HmD>. Post of 18 Jan 2011
15:34:47-0800to AERA-L and Net-Gold. The abstract and link to the
complete post are being transmitted to various discussion lists are
also online on my blog "Hake'sEdStuff" at <http://bit.ly/h23shQ>
with a provision for comments.
NRC. 2009. National Research Council, Board on Testing and Assessment
(chaired by E.H. Haertel), "Letter Report to the U.S. Department of
Education on the Race to the Top Fund," online at
<http://bit.ly/dOg8v6>.