Postings with the above title have recently appeared on both the Phys-L
and PhysLrnR nets. I am therefore sending this to both lists with
apologies to those who subscribe to both for the redundacy.=20
In a 11/17/97 PhysLrnR posting of the above title Kyle Forinash asks for
good references which show the usefulness of student evaluations in
determining teaching effectiveness. According to a quote being passed
around on his campus:
=93....student ratings are the single most valid source of data on
teaching effectiveness. In fact, as Marsh and Roche (1997) point out,
there is little evidence of the validity of any other sources of data.=94
If =93teaching effectiveness=94 is equated with student learning then, at
least for introductory physics, I think it is very doubtful that student
ratings are the most valid source of data on teaching effectiveness.=20
Halloun and Hestenes (1) found that four professors with good or
superlative student evaluations but who utilized traditional
passive-student lectures were almost totally ineffective in imparting
any conceptual understanding of Newtonian mechanics to students in
introductory physics courses at Arizona State University. Here student
learning was measured by pre- and post-test gains on the
Halloun-Hestenes Mechanics Diagnostic (MD) test of conceptual
understanding (1). More recent pre/post testing using the MD test and
the rather similar Force Concept Inventory (2) shows that much higher
gains can be achieved in interactive-engagement courses (2, 3). =20
Among other measures of student learning in introductory physics that
would appear superior to student evaluations are the (a) Mechanics
Baseline test (4), (b) Test of Understanding Graphs in Kinematics
(5),(c) Force and Motion Conceptual Evaluation (6), (d) Advanced
Placement exams.
As for references which purport to show the usefulness of student
evaluations in gauging student learning, Cohen (7) in a much quoted
meta-analysis of 41 studies on 68 separate multisection courses claimed
that =93the average correlation between an overall instructor rating and
student achievement was +0.43; the average correlation between an
overall course rating and student achievement was +0.47...the
results...provide strong support for the validity of student ratings as
measures of teaching effectiveness.=94 However, there are at least three
problems with Cohen=92s analysis: (a) the grading satisfaction hypothesis
discussed by Marsh (8) (students who are aware that they will receive
high grades reward their instructors with favorable evaluations) could
account, at least in part, for the positive correlation, (b) there was
no pretesting to disclose initial knowledge states, (c) the quality of
the =93achievement tests=94 was not examined (were they of the
plug-in-regurgitation type so common in introductory physics courses?).=20
With regard to deficiency =93c=94, Feldman (9), in a review and reanalysi=
s
of Cohen=92s data, points out that =93McKeachie.. (ref. 10)...has recentl=
y
reminded educational researchers and practitioners that the achievement
tests assessing student learning in the sorts of studies reviewed here
typically measure lower-level educational objectives such as memory of
facts and definitions rather than higher-level outcomes such as critical
thinking and problem solving that are usually taken as important in
higher education.=94
If =93teaching effectiveness=94 is gauged not just by student learning bu=
t
also by affective outcomes (e.g., students=92 enthusiasm, satisfaction,
motivation, interest, and appreciation) then, in my opinion, teaching
evaluations can serve useful purposes, especially as diagnostic tools
given relatively early in the semester (see e.g., ref. 11). =20
In a Phys-L posting of 11/19/97, John Gastineau wrote =93I don=92t happen=
to
have any students handy at the moment, but how about asking them....(if
they regard encouraging them to think as bothersome teaching which they
don=92t want in the course)?.... Who is teaching an articulate group of
pre-meds or other non-physical-science types?=94
=20
I and others have taught such groups at Indiana University for many
years. (3) The use of the evaluation described in ref. 11 in such
courses has been discussed (12). The average student rating of =93the lab
as a help in learning physics=94 was 3.36 where A=3Dexcellent=3D4, B=3DGo=
od=3D3,
C=3D2=3DFair, D=3D1=3DPoor, and E=3D0=3DFail. The labs emphasized collabo=
rative peer
instruction and Socratic dialogue(13). In a later evaluation of 3/95 we
asked students =93If given a choice as to lab formats, I would prefer=20
A. the present small group =91collaborative learning=92 and Socratic
dialogue method (percentage response =3D 60%);=20
B. a more traditional method in which students take and analyze data so
as to =93verify=94 certain physical principles (24%);=20
C. no preference or undecided (16%). =20
Thus, at least for pre-med type students at Indiana, strategies which
induce thinking seem to be appreciated.
Richard Hake, Emeritus Professor of Physics, Indiana University
24245 Hatters Street, Woodland Hills, CA 91367
<hake@ix.netcom.com>
REFERENCES
1. I. Halloun and D. Hestenes, =93The initial knowledge state of college
physics students,=94 Am. J. Phys. 53, 1043 (1985).
2. D. Hestenes, M. Wells, and G. Swackhamer, =93Force Concept Inventory,=94
Phys. Teach. 30, 141 (1992) [a revised 1995 version due to I. Halloun,
R.R. Hake, E.P. Mosca, and D. Hestenes is in E. Mazur,=20
=93Peer Instruction: A User=92s Manual=94 (Prentice Hall, 1997)and on the=
Web
(password protected) at <http://modeling.la.asu.edu/modeling.html>].
3. R.R. Hake, =93Interactive-engagement vs traditional methods: A
six-thousand-student survey of mechanics test data for introductory
physics courses,=94 Am. J. Phys., in press (Jan. 1998 ?) and on the Web a=
t
<http://carini.physics.indiana.edu/SDI/>; see also
<http://www.aahe.org/hake.htm>.
4. D. Hestenes and M. Wells, =93A Mechanics Baseline Test,=94 Phys. Teac=
h.
30, 159 (1992).
5. R.J. Beichner, =93Testing student interpretation of kinematics graphs,=
=94
Am. J. Phys. 62, 750 (1994).
6. See, e.g., D.R. Sokoloff and R.K. Thornton, =93Using Interactive
Lecture Demonstrations to Create an Active Learning Environment,=94 Phys.
Teach. 35, 340 (1997).=20
7. P.A. Cohen, =93Student ratings of Instruction and Student Achievement=
:
A Meta-analysis of Multisection Validity Studies,=94 Review of Educationa=
l
Research 51, 281 (1981). =20
8. H.W. Marsh, =93Students=92 Evaluations of University Teaching:
Dimensionality, Reliability, Validity, Potential Biases, and Utility,=94
J. of Educational Psychology 76, 707 (1984).
9. K.A. Feldman, =93The Association Between Student Ratings of Specific
Instructional Dimensions and Student Achievement: Refining and
Extending the Synthesis of Data from Multisection Validity Studies,=94
Research on Higher Education 30, 583 (1989).
10. W.J. McKeachie, =93Instructional Evaluation: Current Issues and
possible improvements,=94 J. of Higher Education 58(3), 344 (1987).
11. R.R. Hake and J.C. Swihart, =93Diagnostic Student Computerized
Evaluation of Multicomponent Courses (DISCOE), Teach. Learning (Indiana
University, January 1979, available on request).
12. S. Tobias and R.R. Hake =93Professors as physics students: What can
they teach us? Am. J. Phys. 56, 786 (1988).
13. R.R. Hake, =93Promoting student crossover to the Newtonian World,=94
Am. J. Phys. 55, 878 (1987); =93Socratic Pedagogy in the Introductory
Physics Lab,=94 Phys. Teach. 30, 546 (1992).