Some subscribers to Phys-L might be interested in a recent post "Re:
SETs under attack again. . ." [Hake (2013)]. The abstract reads:
**************************************
ABSTRACT: POD's Nira Hativa in her post "SETs under attack again. .
." at <http://bit.ly/11GDlpH> wrote (paraphrasing): "How would you
respond to the recent article in 'Psychology Today': 'Do the Best
Professors Get the Worst Ratings? Do students give low ratings to
teachers who instill deep learning?' by psychologist Nate Kornell
(2013)] at <http://bit.ly/190OXdi>.
On the basis of: (a) "Appearances Can Be Deceiving: Instructor
Fluency Increases Perceptions of Learning Without Increasing Actual
Learning" [Carpenter et al. (2013)] at <http://bit.ly/18Ruqvl>, and
(b) "Does Professor Quality Matter? Evidence from Random Assignment
of Students to Professors" [Carrell & West (2010)] at
<http://bit.ly/KtOnHp>, Kornell concluded that "STUDENT EVALUATIONS
ARE OF QUESTIONABLE VALUE."
Kornell's conclusion is consistent with Linda Nilson's (2013a) post
at <http://bit.ly/1asQQRL>, stating that "Carrell and West's article
is just one of several recent studies showing that student ratings
are *no longer* positively related to student learning [as found by
Cohen, 1981 at <http://bit.ly/11TYGjV>]."
NO LONGER? In "Re: Problems with Student Evaluations: Is Assessment
the Remedy?" [Hake (2002a)] at <http://bit.ly/hjt9ll> I wrote
(paraphrasing): "Neither Cohen nor any other SET champion has
countered the fatal objection of McKeachie (1987) at
<http://bit.ly/18Ou9t6> that the evidence for the validity of SETs as
gauges of the cognitive impact of courses rests for the most part on
measures of students' *lower-level* thinking as exhibited in course
grades or exams."
Nevertheless, the tired debate on SETs continues unabated. A search
of the archives of AERA-D, ASSESS, EvalTalk, Phys-L, PhysLrnR, POD, &
STLHE-L for "student evaluations" yielded hit totals of 762 on 5
April 2002 and 1987 on 17 June 2013. Has the tenor of the debate
changed over those 11 years? POD's Ed Nuhfer (2013) at
<http://bit.ly/15lk3IC> thinks so, pointing out that his 1990 opinion
"SETs Are Direct Measures of Student Satisfaction," once regarded as
heretical, now appears to be mainstream.
Hativa's post stimulated (as last count) 112 POD posts, none of them
mentioning pre/post testing with Concept Inventories
<http://bit.ly/dARkDY> (developed through arduous qualitative and
quantitative research by disciplinary experts) as a way to gauge
students' higher-order learning, despite the fact that such has been
occurring in physics education for almost 3 decades. In this post I
reference 19 failed attempts to inform academia that "SETs ARE *NOT*
VALID MEASURES OF STUDENTS' HIGHER-ORDER LEARNING."
**************************************
"The defenders of the SET process are generally found in the colleges
of education, in the national teachers' unions, and among those who
consult in the area. Their positive attitude toward SETs is
compatible with a holistic environment that consists of positive
research findings, currently accepted educational philosophy, and a
communication system largely centered within their own academic
disciplines. They are confident enough in their positive conclusions
to dismiss negative findings as 'myths' (Aleamoni, 1999; Marsh &
Roche, 2000) and to wonder why negative comments continue to be found
in the literature (Theall & Franklin, 2001). Because
instructional-related research is the province of their occupation,
it would be expected that the majority of the research on SETs would
come from those in educational disciplines."
- Dennis Clayson (2009)
"Physics educators have led the way in developing and using objective
tests to compare student learning gains in different types of
courses, and chemists, biologists, and others are now developing
similar instruments. These tests provide convincing evidence that
students assimilate new knowledge more effectively in courses
including active, inquiry-based, and collaborative learning, assisted
by information technology, than in traditional courses."
- William Wood & James Gentile (2003)
REFERENCES [URL shortened by <http://bit.ly/> and accessed on 20 June 2013.]
Hake, R.R. 2013. "Re: SETs under attack again. . .," online on the
OPEN! Net-Gold archives at <http://yhoo.it/19WKjA5>. Post of 20 Jun
2013 15:41:08-0700 to AERA-L and Net-Gold. The abstract and link to
the complete post are being distributed to various discussion lists
and are on my blog "Hake'sEdStuff" at <http://bit.ly/16m4cvQ>.