Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

[Phys-L] Re: Should Randomized Control Trials Be the Gold Standard of Educational Research ?



If RCTs were able to answer the educational questions they might be useful,
but in general they are impractical, impossible, or at least expensive and
difficult in educational research. See my previous post for some of the
reasons.

Some questions might be answerable by such a methodology, but in general
parents would usually not allow this to be done if they thought that their
children were getting inferior education. Also consider the difficulty of
scheduling students for such a study. Would parents allow their children to
be randomly bussed to different schools for such a study? What if one of
the teachers is judged incompetent by the parents?

Most educational studies have results which are just significant, with the
actual effect size being fairly small. To see the effects you need a fairly
large sample. This does not include most of the PER studies and the other
studies I cited, which have shown very large effect sizes. It is easy to do
a simple 2 week educational experiment, and nobody complains, but a whole
year is difficult. Actually the experiments are going on all the time, but
they are called lesson plans, and are not very controlled. They put new
teachers into classrooms and provide little guidance.

Essentially there are other ways of studying the effects of treatments which
can also give valid results, so RCTs are not needed. For example Hake's
study is very valid, and shows an extremely large effect size. There is one
RCT which I can cite that was done by Feuerstein and it showed an estimated
effect size of around 1.0 for increase on standard IQ tests. The effect is
so large that it would like raising the survival rate of cancer patients
from 30% to 60%, yet this study is apparently unknown to the RCT
enthusiasts.

I think if you also look carefully at Hake's post, you will see he is
opposed to making RCT the gold standard with no other studies considered
equally as valid. If this is done, the current rate of progress will slow
from turtle's pace to snail's pace because of the difficulty and expense of
RCTs. Nobody has said that RCTs should not be done. But consider that
medical RCTs are very exact, and do not involve a huge number of patients,
with informed consent, and the patients can not tell what treatment they are
getting. In Physics RCTs are easily done because particles are assumed to
be identical, but in education this is not true.

John M. Clement
Houston, TX



This note lifted from the Medical Research Library website
at Brooklyn sets out the bones of Randomized Controls:

I am surprised that anyone here would oppose this methodology,
which is to my mind at least, a minimal level of ascent to a
scientific basis for comparison of methods and appears to be the
basis for upholding interactive participation as the only tutorial
method that gives superior results consistently, over chalk n talk
[also known as Direct Instruction]

Brian Whatcott

########################################
A randomized controlled study is one in which:
* There are two groups, one treatment group and one control group.
* The treatment group receives the treatment under investigation,
* and the control group receives either no treatment or some
* standard default treatment.
* Patients are randomly assigned to all groups.
Assigning patients at random reduces the risk of bias and increases the
probability that differences between the groups can be attributed to
the treatment.

Having a control group allows us to compare the treatment with alternative
choices. For instance, the statement that a particular medication cures
40% of cases tells us very little unless we also know how many cases get
better on their own! (Or with a different treatment).

With certain research questions, randomized controlled studies cannot
be done for ethical reasons. For instance, it would be unethical to
attempt
to measure the effect of smoking on health by asking one group to smoke
two packs a day and another group to abstain, since the smoking group
would be subject to unnecessary harm.

Randomized controlled trials are the standard method of answering
questions about the effectiveness of different therapies. If you have
a therapy question, first look for a randomized controlled trial, and
only go on to look for other types of studies if you don't find one.

########################################


At 10:28 PM 4/15/2005, Richard Hake, you wrote:

///

MOST INTERACTIVE ENGAGEMENT AND GUIDED INQUIRY
METHODS HAVE NOT BEEN
TESTED IN RANDOMIZED CONTROL TRIALS (RCT'S),
THE "GOLD STANDARD" OF
THE U.S. DEPT. OF EDUCATION (USDE)


Richard Hake, Emeritus Professor of Physics, Indiana University

///


Brian Whatcott Altus OK Eureka!
_______________________________________________
Phys-L mailing list
Phys-L@electron.physics.buffalo.edu
https://www.physics.buffalo.edu/mailman/listinfo/phys-l