In response to my post "Teachers: the Archimedean Lever for Elevating
Public Schools" Hake (2005a), David Berliner (2005a) wrote:
"I served on the NAE . . .[National Academy of Education
<http://www.nae.nyu.edu/>] . . . report team. Its a pretty good
document. Not as revolutionary as the Flexner report for med, but
maybe as important if it takes hold. On the issue of teacher testing
you might find the following. . .[Berliner (2005b)]. . . worth
something. It is in sync with what the NAE recommends."
With regard to Berliner (2005b), in my post "Re: Measuring Teaching
Performance" [Hake (2005b)], I wrote [bracketed by lines
"HHHHHHHHHHH. . . .":
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
The above pre/post testing recommendation appears to conflict with
the opinion of psychologist David Berliner (2005b). Berliner writes:
". . . .measurement of [teachers'] success in promoting learning
through 'pay for performance' or 'value-added' assessments is so
filled with psychometric problems that no current system is
acceptable for assessing this dimension of teacher quality."
But the evident success of physics education researchers in the use
of pre/post testing to gauge student learning and thereby improve
introductory courses [most notably at Harvard [Crouch & Mazur (2002)
and MIT (Dori & Belcher (2004)] seems to contradict Berliner's claim,
suggesting that psychometric problems may not be as serious [Hake
(2004a), Scriven (2004)] as many believe. . .[see e.g., Suskie
(2004)].
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
As an aside, it should be realized that the pre/post testing being
done in physics makes use of reasonably well matched control groups -
just the traditional introductory courses, and has been carried out
by many different researchers in many different locations with the
virtually identical results - about a two-standard deviation
superiority of "interactive engagement" methods over "traditional
methods" in promoting students' understanding of Newtonian mechanics.
Where are Berliner's psychometric problems?
It's not clear to me how Berliner can state that "It. . . . [the
claimed "Near Impossibility of Testing For Teacher Quality"
(Berliner, 2005b)]. . . is in sync with what the NAE recommends,"
since, as I understand it, NAE is recommending that Congress pay for
the development of a national teacher test **using performance to
judge accomplishment.**
Three questions:
(a) How does the NAE define "teacher performance"?
(b) How does the NAE propose to measure "teacher performance"?
(c) Is the "teacher performance" to be measured somehow divorced from
the "teacher quality" that Berliner claims is nearly impossible to
measure?
For a recent book on value added assessment see Lissitz (2005). I
have not seen the book but I expect that the contributors - evidently
mostly from the PEP (Psychology/Education/Psychometric) community -
are all either oblivious or dismissive of the pre/post testing
research being carried out in such fields as astronomy, economics,
biology, chemistry, computer science, engineering, and physics [see,
e.g. Hake (2004a,b,c; 2005c)].
REFERENCES
Berliner, D. 2005a. "Teachers: the Archimedean Lever for Elevating
Public Schools" TeachingEdPsych post of 3 Jun 2005 22:24:59-0700;
online at
<https://listserv.temple.edu/cgi-bin/wa?A2=ind0506&L=teaching_edpsych&T=0&F=&S=&X=37DF86118237021F98&Y=rrhake%40earthlink.net&P=200>.
The encyclopedic URL indicates that TeachingEdPsych is one of the few
discussion lists that denies archive access to non-subscribers.
However, it takes only a few minutes to subscribe by following the
simple directions at
<http://listserv.temple.edu/archives/teaching_edpsych.html> / "Join
or leave the list (or change settings)" where "/" means "click on."
If you're busy, then subscribe using the "NOMAIL" option under
"Miscellaneous." Then, as a subscriber, you may access the archives
and/or post messages at any time, while receiving NO MAIL from the
list!
Berliner, D. 2005b. "The Near Impossibility of Testing For Teacher
Quality," Journal of Teacher Education 56(3): 205-213, online free to
subscribers ($25 for non-subscribers) at
<http://jte.sagepub.com/content/vol56/issue3/>.
Crouch, C.H. & E. Mazur. 2001. "Peer Instruction: Ten years of
experience and results," Am. J. Phys. 69: 970-977; online at
<http://mazur-www.harvard.edu/library.php>, search "All Education
Areas" for author "Crouch" (without the quotes).
Dori, Y.J. & J. Belcher, J. 2004. "How Does Technology-Enabled Active
Learning Affect Undergraduate Students' Understanding of
Electromagnetism Concepts?" To appear in The Journal of the Learning
Sciences 14(2), online at
<http://web.mit.edu/jbelcher/www/TEALref/TEAL_Dori&Belcher_JLS_10_01_2
004.pdf> (1 MB).
Hake, R.R. 2005a. "Teachers: the Archimedean Lever for Elevating
Public Schools," online at
<http://lists.asu.edu/cgi-bin/wa?A2=ind0506&L=aera-l&T=0&F=&S=&P=160>.
Post of 3 Jun 2005 17:18:11-0700 to AERA-C, AERA-G, AERA-GSL, AERA-H,
AERA-J, AERA-K, AERA-L, AP-physics, ASSESS, Biopi-L, Chemed-L,
EvalTalk, Math-Learn, Phys-L, PhysLrnR, Physhare, POD, STLHE-L,
TeachingEdPsych, and TIPS.
Hake, R.R. 2005b. "Re: Measuring Teaching Performance," online at
<http://lists.asu.edu/cgi-bin/wa?A2=ind0505&L=aera-j&T=0&O=A&P=962>.
Post of 13/14 May 2005 to AERA-C, AERA-D, AERA-J, AERA-K, AERA-L,
ASSESS, Biopi-L, Chemed-L, EvalTalk, Math-Learn, Phys-L, PhysLrnR
Physhare, POD, & STLHE-L, & TeachingEdPsych.
Hake, R.R. 2005c. "Carnegie Scholar Backs Pre/post Testing, online at
<http://lists.asu.edu/cgi-bin/wa?A2=ind0503&L=aera-d&P=R3965&I=-3>.
Post of 24 Mar 2005 01:23:45-0800 to AERA-C, AERA-D, AERA-J, AERA-K,
ASSESS, EDSTAT-L, EVALTALK, PhysLrnR, POD, and STLHE-L.
Lissitz, R.W., ed. 2005. "Value Added Models in Education: Theory
and Applications." JAM Press. Contents and ordering information are
at the Journal of Applied Measurement web site
<http://www.jampress.org> / "JAM Press Books!" where "/" means
"click on."