In response to my post "Re: pre/post testing to determine student
progress," [Hake (2006)], Eugene Komaroff [Director of the
Biostatistics and Clinical Trials at the Kessler Medical
Rehabilitation Research & Education Corporation of The New Jersey
Medical School] posted on AERA-D:
"Thank you for the most informative post on controversies surrounding
RCT in education. However, seems to me the problem is lack of
randomization, not a lack of comparison/control group, for drawing
causal conclusions with pre-post designs. I do not understand the
resistance to randomization. Why does random assignment to
intervention appear difficult to achieve in Educational Research -
ethics, politics?
Physicist Robert Ehrlich (2002) in an American Journal of Physics
(AJP) article "How do we know if we're doing a good job in physics
Teaching?" raised a similar point. In my response [Hake (2002a) I
wrote [bracketed by lines "HHHHHHHHHHH. . . .", see that article for
references other than Ehrlich (2002) and Hake (1998a,b; 2002b)]:
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
In his recent perspective ["How do we know if we're doing a good job
in physics Teaching?"], Robert Ehrlich set forth some good ideas,
but, in my opinion, faltered in his criticism of my survey article
[Hake (1998a,b)]
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Here are Ehrlich's remaining reasons 1, 2, and 4 for not accepting my
claims at face value, followed by brief rebuttals:
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
2. ". . . the IE versus non-IE comparison is hardly a double blind
one, because both Hake and the course instructor knew both the
category the course is being placed into (IE or non-IE), as well as
the FCI gain for that class."
In the case of a survey such as mine, it is not clear that blindness
to differences in T [Traditional] and IE [Interactive Engagement]
physics instruction could have been found in any potential surveyors,
physics teachers, or students who were not medically
institutionalized.
Non-double-blind education research experiments may be less
convincing than some double-blind medical experiments, but that
doesn't mean that the education results should necessarily be taken
at LESS than face value. In the case of my survey, I think that the
results merit acceptance at FULL face value, especially considering
the fact that normalized gain differences between T and IE courses
that are consistent with those I reported, have now been obtained by
physics education research groups [referenced in Hake (2002b)] at the
University of Maryland [Redish et al. (1997), Saul (1998), Redish and
Steinberg (1999), Redish (1999)], the University of Montana [Francis
et al. (1998)], Rennselaer and Tufts Universities [Cummings et al.
(1999)], North Carolina State University [Beichner et al. (1999)],
Hogskolan Dalarna - Sweden [Bernhard (2001)], Carnegie Mellon
University [Johnson (2001)], and City College of New York [Steinberg
and Donnelly (2002)].
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
And then in "Re: Should Randomized Control Trials Be the Gold
Standard of Educational Research? [Hake (2005)] I wrote:
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
Could physics education researchers (PER's) whose work is
predominately in UNDERGRADUATE education utilize RCT's? PER's deal
with populations of UP (university professors) and US (Undergraduate
Students). Most UP's demand autonomy in the way they teach courses
since they obviously know best how to lecture. Most of the US's (or
their parents) paid good money to be lectured at. No one that I know
of has been insane enough to even suggest that subjects from
populations UP and US be randomly assigned to different curricula in
a RCT, especially if one curriculum deemphasizes lectures. Also the
average UP, thrown into an IE course would be a total disaster. If
anyone has some ideas on how to accomplish an RTC among UP's and US's
while avoiding dismissal or execution please let me know. Of course
one could PAY the subjects, but this might bias the results towards
the greedy and/or impecunious.
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
Hake, R.R. 2002b. "Lessons from the physics education reform effort,"
Ecology and Society 5(2): 28; online at
<http://www.ecologyandsociety.org/vol5/iss2/art28/>. Ecology and
Society (formerly Conservation Ecology) is a free online
"peer-reviewed journal of integrative science and fundamental policy
research" with about 11,000 subscribers in about 108 countries.
Hake, R.R. 2005. Re: Should Randomized Control Trials Be the Gold
Standard of Educational Research? online at
<http://lists.asu.edu/cgi-bin/wa?A2=ind0504&L=aera-l&T=0&O=D&P=2100>.
Post of 17/18 Apr 2005 to AERA-C, AERA-D, AERA-G, AERA-H, AERA-J,
AERA-K, AERA-L, AP-Physics, ASSESS, Biopi-L, Chemed-L, EvalTalk,
Math-Learn, Phys-L, PhysLrnR, Physhare, POD, STLHE-L, & TIPS.
Hake, R.R. 2006. "Re: pre/post testing to determine student
progress," online at
<http://lists.asu.edu/cgi-bin/wa?A2=ind0609&L=aera-l&T=0&F=&S=&P=1718>.
Post of 24/25 Sept 2006 to AERA-A, AERA-B, AERA-C, AERA-D, AERA-I,
AERA-J, AERA-K, AERA-L, ASSESS, Chemed-L, Biopi-L, Biolab (rejected),
DrEd, EvalTalk, IFETS, Math-Learn, PsychTeacher (rejected),
TeachingEdPsych, Phys-L, PhysLnrR, POD, RUME, STLHE-L, & TIPS.