| Chronology | Current Month | Current Thread | Current Date |
| [Year List] [Month List (current year)] | [Date Index] [Thread Index] | [Thread Prev] [Thread Next] | [Date Prev] [Date Next] |
Randomized control trials certainly have proven their worth in manyto
fields. I would not take any medicine that had not gone through such
clinical trials.
I think arguments could be made that it is difficult to carry out true
randomized control trials on teaching methods because it is difficult
control all the variables. However, I do think that randomizedcontrol
trials should be attempted on the various teaching methods that weuse,
to see if any statistically valid results can be obtained.own
I have used both active learning and direct instruction styles in my
teaching; and, have seen some differences in results -- but in mosta
cases the differences have not been spectacular.
Dr. Mark H. Shapiro
Professor of Physics, Emeritus
California State University, Fullerton
Phone: 714 278-3884
FAX: 714 278-5810
email: mshapiro@fullerton.edu
web: http://chaos.fullerton.edu/Shapiro.html
travel and family pictures:
http://community.webshots.com/user/mhshapiro
-----Original Message-----
From: Forum for Physics Educators [mailto:PHYS-L@list1.ucc.nau.edu] On
Behalf Of Richard Hake
Sent: Friday, April 15, 2005 8:29 PM
To: PHYS-L@LISTS.NAU.EDU
Subject: Should Randomized Control Trials Be the Gold Standard of
Educational Research ?
If you object to cross-posting as a way to tunnel through inter- and
intra-disciplinary barriers, please hit "delete" now. And if you
respond to this long (8 kB) post, please don't hit the reply button
unless you prune the original message normally contained in your
reply down to a few lines, otherwise you may inflict this entire post
yet again on suffering list subscribers.
In "Will the No Child Left Behind Act Promote Direct Instruction of
Science?" [Hake (2005)], I gave, as one of the seven reasons why
Direct Science Instruction threatens to predominate nationally under
the aegis of the No Child Left Behind Act, the following [bracketed
by lines "HHHHHH. . . ."]:
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
MOST INTERACTIVE ENGAGEMENT AND GUIDED INQUIRY METHODS HAVE NOT BEEN
TESTED IN RANDOMIZED CONTROL TRIALS (RCT'S), THE "GOLD STANDARD" OF
THE U.S. DEPT. OF EDUCATION (USDE)
That a single research method should be designated as the "gold
standard" for
evaluating an intervention's effectiveness appears antithetical to
the report of the NRC's Committee on Scientific Principles for
Education Research [Shavelson & Towne (2002) - ST]. ST state that
scientific research should "pose significant questions that can be
investigated empirically," and "use methods that permit direct
investigation of the questions."
Furthermore, the USDE's RCT gold standard is considered problematic by
widekB),
array of scholars. Taking issue with the RTC gold standard are
philosophers Dennis Phillips [Shavelson, Phillips, Towne, & Feuer
(2003)] and Michael Scrivin (2004); mathematicians Burkhardt &
Schoenfeld (2003); engineer Woodie Flowers [Zaritsky, Kelly, Flowers,
Rogers, Patrick (2003)]; and physicist Andre deSessa [Cobb, Confey,
diSessa, Lehrer, & Schauble (2003)].
In addition, the following organizations oppose the RTC gold standard:
(a) American Evaluation Association (AEA)
<http://www.eval.org/doestatement.htm>,
(b) American Education Research Association (AERA)
<http://www.eval.org/doeaera.htm>, and
(c) National Education Association
<http://www.eval.org/doe.nearesponse.pdf> (88 kB).
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
Two recent articles [Bhattacharjee (2005), Stipek (2005)] discuss the
pros and cons of RCT's and may be of interest to subscribers. I thank
Larry Woolf for bringing the Bhattacharjee reference to my attention.
Richard Hake, Emeritus Professor of Physics, Indiana University
24245 Hatteras Street, Woodland Hills, CA 91367
<rrhake@earthlink.net>
<http://www.physics.indiana.edu/~hake>
<http://www.physics.indiana.edu/~sdi>
REFERENCES
Bhattacharjee, Y. 2005. "Can Randomized Trials Answer The Question of
What Works?: A $120 million federal initiative to improve secondary
math education hopes to draw on an approach some researchers say may
not be ready for the classroom," Science 307: 1861-1863, 25 March,
currently online at
(a) <http://www.sciencemag.org/cgi/content/full/307/5717/1861>,
(b) <http://www.sciencemag.org/cgi/reprint/307/5717/1861.pdf> (208
and
(c) the archives of AERA-L (Politics and Policy in Education) with
academic referencing by R. Hake at
<http://lists.asu.edu/cgi-bin/wa?A2=ind0504&L=aera-l&T=0&O=D&P=1708>.
Burkhardt, H. & A.H. Schoenfeld. 2003. "Improving Educational
Research: Toward a More Useful, More Influential, and Better-Funded
Enterprise," Educational Researcher 32(9): 3-14; online at
<http://www.aera.net/publications/?id=401>.
Cobb, P., J. Confey, A. diSessa, R. Lehrer, L. Schauble. 2003.
"Design Experiments in Educational Research," Educational Researcher
32(1): 9-13; online at <http://www.aera.net/publications/?id=393>.
Hake, R.R. 2005. "Will the No Child Left Behind Act Promote Direct
Instruction of Science?" Am. Phys. Soc. 50: 851 (2005); APS March
Meeting, Los Angles, CA. 21-25 March; online as ref. 36 at
<http://www.physics.indiana.edu/~hake>, or download directly by
clicking on
<http://www.physics.indiana.edu/~hake/WillNCLBPromoteDSI-3.pdf> (256
kB).
Scriven, M. 2004. "Causation," unpublished; see, e.g. "Scriven on
randomized control groups," online at
<http://interversity.org/lists/arn-l/archives/Nov2003/msg00309.html>.
Shavelson, R.J. & L. Towne, eds. 2002. "Scientific Research in
Education," National Academy Press; online at
<http://www.nap.edu/catalog/10236.html>.
Shavelson, R.J., D.C. Phillips, L. Towne, and M.J. Feuer. 2003. "On
the Science of Education Design Studies," Educational Researcher
32(1): 25-28; online at <http://www.aera.net/publications/?id=393>.
Stipek, D. 2005. "Scientifically Based Practice," Education Week
24(28): 33-34; currently online at
(a) <http://www.edweek.org/ew/articles/2005/03/23/28stipek.h24.html>;
(b) as a Math-Learn post of 2 April 2005 10:56 am by Jerry Becker;
online at
<http://groups.yahoo.com/group/math-learn/message/7659>
(unfortunately one must subscribe to Math-Learn
<http://groups.yahoo.com/group/math-learn/> in order to access its
archives);
(c) as a post with a one-paragraph comment by Susan Ohanian at
<http://www.susanohanian.org/show_atrocities.html?id=3914>;
(d) the archives of AERA-L (Politics and Policy in Education) with
academic referencing by R. Hake at
<http://lists.asu.edu/cgi-bin/wa?A2=ind0504&L=aera-l&T=0&O=D&P=1840>.
Zaritsky, R., A.E. Kelly, W. Flowers, E. Rogers, & P. Patrick. 2003.
"Clinical Design Sciences: A View From Sister Design Efforts,"
Educational Researcher 32(1), 33-34; online at
<http://www.aera.net/publications/?id=393>.