Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

Re: [Phys-l] pre/post testing to determine student progress #2



In response to my post "Re: pre/post testing to determine student progress," [Hake (2006)], Eugene Komaroff [Director of the Biostatistics and Clinical Trials at the Kessler Medical Rehabilitation Research & Education Corporation of The New Jersey Medical School] posted on AERA-D:

"Thank you for the most informative post on controversies surrounding RCT in education. However, seems to me the problem is lack of randomization, not a lack of comparison/control group, for drawing causal conclusions with pre-post designs. I do not understand the resistance to randomization. Why does random assignment to intervention appear difficult to achieve in Educational Research - ethics, politics?

Physicist Robert Ehrlich (2002) in an American Journal of Physics (AJP) article "How do we know if we're doing a good job in physics Teaching?" raised a similar point. In my response [Hake (2002a) I wrote [bracketed by lines "HHHHHHHHHHH. . . .", see that article for references other than Ehrlich (2002) and Hake (1998a,b; 2002b)]:

HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
In his recent perspective ["How do we know if we're doing a good job in physics Teaching?"], Robert Ehrlich set forth some good ideas, but, in my opinion, faltered in his criticism of my survey article [Hake (1998a,b)]
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . Here are Ehrlich's remaining reasons 1, 2, and 4 for not accepting my claims at face value, followed by brief rebuttals:
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . 2. ". . . the IE versus non-IE comparison is hardly a double blind one, because both Hake and the course instructor knew both the category the course is being placed into (IE or non-IE), as well as the FCI gain for that class."

In the case of a survey such as mine, it is not clear that blindness to differences in T [Traditional] and IE [Interactive Engagement] physics instruction could have been found in any potential surveyors, physics teachers, or students who were not medically institutionalized.

Non-double-blind education research experiments may be less convincing than some double-blind medical experiments, but that doesn't mean that the education results should necessarily be taken at LESS than face value. In the case of my survey, I think that the results merit acceptance at FULL face value, especially considering the fact that normalized gain differences between T and IE courses that are consistent with those I reported, have now been obtained by physics education research groups [referenced in Hake (2002b)] at the University of Maryland [Redish et al. (1997), Saul (1998), Redish and Steinberg (1999), Redish (1999)], the University of Montana [Francis et al. (1998)], Rennselaer and Tufts Universities [Cummings et al. (1999)], North Carolina State University [Beichner et al. (1999)], Hogskolan Dalarna - Sweden [Bernhard (2001)], Carnegie Mellon University [Johnson (2001)], and City College of New York [Steinberg and Donnelly (2002)].
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH


And then in "Re: Should Randomized Control Trials Be the Gold Standard of Educational Research? [Hake (2005)] I wrote:

HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
Could physics education researchers (PER's) whose work is predominately in UNDERGRADUATE education utilize RCT's? PER's deal with populations of UP (university professors) and US (Undergraduate Students). Most UP's demand autonomy in the way they teach courses since they obviously know best how to lecture. Most of the US's (or their parents) paid good money to be lectured at. No one that I know of has been insane enough to even suggest that subjects from populations UP and US be randomly assigned to different curricula in a RCT, especially if one curriculum deemphasizes lectures. Also the average UP, thrown into an IE course would be a total disaster. If anyone has some ideas on how to accomplish an RTC among UP's and US's while avoiding dismissal or execution please let me know. Of course one could PAY the subjects, but this might bias the results towards the greedy and/or impecunious.
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH


Richard Hake, Emeritus Professor of Physics, Indiana University
24245 Hatteras Street, Woodland Hills, CA 91367
<rrhake@earthlink.net>
<http://www.physics.indiana.edu/~hake>
<http://www.physics.indiana.edu/~sdi>


REFERENCES
Ehrlich, R. 2002. "How do we know if we're doing a good job in physics
Teaching?" Am. J. Phys. 70(1), 24-29 (2002); online to subscribers at <http://scitation.aip.org/dbt/dbt.jsp?KEY=AJPIAS&Volume=70&Issue=1>.

Hake, R.R. 1998a. "Interactive-engagement vs traditional methods: A
six-thousand-student survey of mechanics test data for introductory physics courses," Am. J. Phys. 66: 64-74; online as ref. 24 at
<http://www.physics.indiana.edu/~hake>, or simply click on <http://www.physics.indiana.edu/~sdi/ajpv3i.pdf> (84 kB).

Hake, R.R. 1998b. "Interactive-engagement methods in introductory mechanics courses," online as ref. 25 at <http://www.physics.indiana.edu/~hake>, or simply click on <http://www.physics.indiana.edu/~sdi/IEM-2b.pdf> (108 kB) - a crucial companion paper to Hake (1998a).

Hake, R.R. 2002a. "Comment on 'How do we know if we are doing a good job in physics teaching?' by Robert Ehrlich," Am. J. Phys. 70(10): 1058-1059; online as ref. 17 at <http://www.physics.indiana.edu/~hake> or download directly by clicking on <http://www.physics.indiana.edu/~hake/HakeOnEhrlich-2.pdf> (40 kB).

Hake, R.R. 2002b. "Lessons from the physics education reform effort," Ecology and Society 5(2): 28; online at
<http://www.ecologyandsociety.org/vol5/iss2/art28/>. Ecology and Society (formerly Conservation Ecology) is a free online "peer-reviewed journal of integrative science and fundamental policy research" with about 11,000 subscribers in about 108 countries.

Hake, R.R. 2005. Re: Should Randomized Control Trials Be the Gold Standard of Educational Research? online at
<http://lists.asu.edu/cgi-bin/wa?A2=ind0504&L=aera-l&T=0&O=D&P=2100>.
Post of 17/18 Apr 2005 to AERA-C, AERA-D, AERA-G, AERA-H, AERA-J,
AERA-K, AERA-L, AP-Physics, ASSESS, Biopi-L, Chemed-L, EvalTalk, Math-Learn, Phys-L, PhysLrnR, Physhare, POD, STLHE-L, & TIPS.

Hake, R.R. 2006. "Re: pre/post testing to determine student progress," online at
<http://lists.asu.edu/cgi-bin/wa?A2=ind0609&L=aera-l&T=0&F=&S=&P=1718>. Post of 24/25 Sept 2006 to AERA-A, AERA-B, AERA-C, AERA-D, AERA-I, AERA-J, AERA-K, AERA-L, ASSESS, Chemed-L, Biopi-L, Biolab (rejected), DrEd, EvalTalk, IFETS, Math-Learn, PsychTeacher (rejected), TeachingEdPsych, Phys-L, PhysLnrR, POD, RUME, STLHE-L, & TIPS.