TOTAL HITS (as of 25 Apr 2002 14:14:14-0700) . . 762
TOTAL HITS (as of 21 Oct 2002 08:00:00-0700). . . . . . 845
Increase in 6 months . . . . . . . . . . . . . .83
The numbers following each discussion list URL give the numbers of
hits on "student evaluations" in the "search for" slot of that lists
archive's
search engine as of about six months ago (25 Apr 2002 14:14:14-0700)
and then today 21 Oct 2002 08:00:00-0700. Internet ethnographers
might wish to research the culture clashes apparent in some of these
posts.
In his Phys-L post of 0 Oct 2002 19:32:01-0500 titled "Student
Evaluations (Was: Lazy?)", John (Texas) Clement, contributing to an
enormous and ever growing archive of posts on the subject (see
above), wrote [[in the standard manner the square brackets "[. . .]
indicate MY insertions and NOT John's words; I bracket John's post by
"CLEMENT-CLEMENT-. . ." so that I don't have to use quotes within
quotes " ' ' "]]:
CLEMENT-CLEMENT-CLEMENT-CLEMENT-CLEMENT-CLEMENT-CLEMENT-CLEMENT
[Student evaluations are] . . . valid for what? The word valid has a
specific meaning in regards to a test. In this case it may only be
valid as a popularity contest.
The only way to test student performance is to have a separate test
in each comparable course which measures what the students actually
understand. I doubt the dean is either competent or has the time and
energy to make up such a test and make sure that it is administered.
Of course, the physics community has such tests as the FCI, FMCE,
Tufts Heat evaluation, Tufts Circuit evaluation, MPEX, and others . .
.[see e.g. NCSU 2002]. . . . those who are bothered by nasty deans
COULD ALWAYS GATHER A FEW RESEARCH ARTICLES WHICH SHOW THAT STUDENT
EVALUATIONS ARE WORTHLESS. . .[I think John may have meant "worthless
as a measure of the COGNITIVE impact of a course" - John, please
correct me if I'm wrong]. . . ., and give them to colleagues, the
deans, and to the university president. (My CAPS.)
CLEMENT-CLEMENT-CLEMENT-CLEMENT-CLEMENT-CLEMENT-CLEMENT-CLEMENT
For an educationally naive physicist's introduction to the vast
research literature of student evaluations, I suggest scanning Hake
(2002a). Quoting from that post:
HAKE-HAKE-HAKE-HAKE-HAKE-HAKE-HAKE-HAKE-HAKE-HAKE-HAKE-HAKE-HAKE
At the risk of replaying to deaf ears the same old record I reiterate
Lesson #3 of Hake (2002b) (see that article for the references other
than Wage & Buck):
"L3. High-Quality Standardized Tests Of The Cognitive And Affective
Impact Of Courses Are Essential For Gauging The Relative
Effectiveness Of Non-Traditional Educational Methods.
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
As far as I know, disciplines other than physics, astronomy (Adams et
al. 2000; Zeilik et al. 1997, 1998, 1999), and possibly economics
(Saunders 1991, Kennedy & Siegfried 1997, Chizmar & Ostrosky 1998,
Allgood and Walstad 1999) have yet to develop any such tests and
therefore cannot effectively gauge either the need for or the
efficacy of their reform efforts. . . .[more recently I have learned
of the considerable effort to develop such tests in engineering
education (Wage & Buck 2002)]. . . . In my opinion, all disciplines .
. .[even including mathematics - or are they too embroiled in the
Math Wars`!]. . . . should consider the construction of high-quality
standardized tests of essential introductory course concepts."
Because most disciplines. . .[even psychology!]. . . . have failed to
develop definitive tests to measure cognitive and affective course
impacts, seemingly simplistic statements from the pro Student
Evaluation of Teaching (SET) camp cannot always be immediately
dismissed. . . .[for example]. . . Michael Scriven (1988) [as quoted
by D'Apollonia & Abrami (1997)] stated that "student ratings are not
only A valid, but often THE ONLY valid, way to get much of the
information needed for most evaluations." (EMPHASIS in the
original.). . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . .
THE QUESTION IS "VALID FOR WHAT?" I think SET's can be "valid" in the
sense that they can be useful for gauging the AFFECTIVE impact of a
course and for providing diagnostic feedback TO TEACHERS [see, e.g.,
Hake & Swihart (1979)] to assist them in making mid-course
corrections. However IMHO, SET's are NOT valid in their widespread
use by ADMINISTRATORS to gauge the COGNITIVE impact of courses [see,
e.g., Williams & Ceci (1997); Hake (2000; 2002a,b); Johnson (2002)].
In fact the gross misuse of SET's as gauges of student learning is,
in my view, one of the institutional factors that thwarts substantive
educational reform (Hake 2002b, Lesson #12.)
HAKE-HAKE-HAKE-HAKE-HAKE-HAKE-HAKE-HAKE-HAKE-HAKE-HAKE-HAKE-HAKE
Hake, R.R. 2002a. "Re: Problems with Student Evaluations: Is
Assessment the Remedy?"
AERA-D/ASSESS/EvalTalk/Phys-L/PhysLrnR/POD/STLHE-H post of 25 Apr
2002 16:54:24-0700, online at
<http://lists.nau.edu/cgi-bin/wa?A2=ind0204&L=phys-l&P=R22161>.
Hake, R.R. 2002b. "Lessons from the physics education reform effort."
Conservation Ecology 5(2): 28; online at
<http://www.consecol.org/vol5/iss2/art28>. "Conservation Ecology," is
a FREE "peer-reviewed journal of integrative science and fundamental
policy research" with about 11,000 subscribers in about 108 countries.
NCSU. 2002. "Assessment Instrument Information Page, Physics
Education R & D Group, North Carolina State University"; online at
<http://www.ncsu.edu/per/TestInfo.html>.