PART 1
Those who dislike long posts (20 kB), academic references,
cross-posting, or have no interest in "Comprehensive School Reform
Models" are urged to immediately hit the DELETE button.
ABSTRACT: I attempt to correct a misunderstanding by EvalTalk's Jacob
Silver of my 30 Nov 2005 post titled "Re: Study Rates 22 Widely Used
Comprehensive School Reform Models." Jacob wrote: "I may be missing
something, but how will the overly rigorous
randomized-control-trial-derived data rip the guts out of
inquiry-based science education?" I list four things that Jacob (and
perhaps other subscribers) may have missed.
In response to my post "Re: Study Rates 22 Widely Used Comprehensive
School Reform Models" [Hake (2005a)], Jacob Silver (2005) wrote in
his EvalTalk post of 1 Dec 2005 titled "Re: Study Rates 22 Widely
Used Comprehensive School Reform Models":
"I may be missing something, but how will the overly rigorous
randomized control trial derived data rip the guts out of inquiry
based science education?"
I evidently did not clearly explain my position. I did not mean to
state or imply that "overly rigorous randomized-control-trial-derived
data [will] rip the guts out of inquiry-based science education."
What I wrote was more nuanced:
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
In my opinion the AIR (2005) report is yet further reason to fear
that the U.S. Dept. of Education's (USDE's) direct instruction
juggernaut, FUELED IN PART BY AN **UNSCIENTIFIC** ALLEGIANCE TO
RANDOMIZED CONTROL TRIALS [RCT's] AS THE GOLD STANDARD OF EDUCATIONAL
RESEARCH . . . [see "a" below]. . . , will eviscerate effective K-12
guided inquiry-based science education in the U.S. [see e.g., Hake
(2004; 2005b,c,d)]. BTW, "guided inquiry-based" does NOT mean the
seldom used boogeyman "pure discovery learning" researched by the
widely misinterpreted Klahr & Nigam (2004).
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
Among the things that Jacob may have missed are these four (a, b,c, d):
a. RCT's are NOT EVEN rigorous, let alone *overly* rigorous.
According to Cook & Payne: "In some quarters, particularly medical
ones, the randomized experiment . . .[commonly called randomized
control trial (RCT)]. . . is considered the causal 'gold standard.'
It is clearly not that in educational contexts, given the
difficulties with implementing and maintaining randomly created
groups, with the sometimes incomplete implementation of treatment
particulars, with the borrowing of some treatment particulars by
control group units, and with the limitations to external validity
that often follow from how the random assignment is achieved." For
further discussion of RCT problems see "The 2004 Claremont Debate:
Lipsey vs. Scriven [Donaldson & and Christie (2005)] and Hake (2005e).
b. According to my online Webster's Third New International
Dictionary, definition "1a" of "eviscerate" is "disembowel" (the
meaning assumed by Jacob Silver). But definition "1b" is
"devitalize," i.e., to deprive of life or vitality. IF the USDE were
threatening science education with a direct instruction (DI) dagger
then "disembowel" would be an appropriate meaning.
However, as indicated by the USDE's:
(1) financing of the *problematic* [see e.g., Kramer (2005), Hake
2005f)] AIR (2005) report,
(2) misinterpretation of Klahr & Nigam (2004) as generally supporting DI, and
(3) appointment of DI czar Douglas Carnine to the Technical Advisory
Group for the "What Works Clearinghouse,"
the USDE is riding DI as a juggernaut that devitalizes - OK - had I
been more metaphorically adept I would have written "the DI
juggernaut crushes."
c. My view is that it's the *USDE's DI juggernaut* that will
eviscerate "guided inquiry-based science education in the U.S.", NOT
randomized control trials per se.
d. The square bracketed reference "[see e.g., Hake (2004;
2005b,c,d)]", in the quote bracketed by "HHHHHHHHHHH. . ." above,
indicates that the explanation of why the DI juggernaut will
eviscerate "guided inquiry-based science education in the U.S." is
contained in Hake (2004; 2005b,c,d) [see REFERENCES below].
For example the abstract of "Will the No Child Left Behind Act
Promote Direct Instruction of Science?" [Hake (2005c)] reads:
"The No Child Left Behind (NCLB) Act requires testing in science
achievement starting in 2007. Will such testing tend to propagate
California's Direct Science Instruction (DSI) [Hake (2004)]
throughout the entire nation? After discussing the evidence for the
superiority of "interactive engagement" or "guided inquiry" methods
over DSI in conceptually difficult areas of science, I indicate seven
reasons why NCLB might promote DSI, and one reason - possible
*effective* intervention by the National Research Council. . .[see
e.g., Wilson & Bertenthal (2005)]. . . why it might not."
My seven reasons for why NCLB might promote Direct Science
Instruction are [see [Hake (2005c)] for references other than Hake
(2002, 2005e, 2005g), Heron & Meltzer (2005), Klahr and Nigam (2004),
Wieman & Perkins (2005)]:
1. Most interactive engagement and guided inquiry methods have not .
. .[usually for very good reasons (see, e.g. Hake 2005e)] . . . been
tested in randomized control trials (RCT's), the highly contested
"gold standard" of the U.S. Dept. of Education (USDE) and its "What
Works Clearinghouse" <http://www.w-w-c.org/>.
2. The heavily publicized [Adelson (2004), Begley (2004a,b),
Cavenaugh (2004a,b), Tweed (2004a,b), USDE (2004)] research of Klahr
and Nigam (2004) is widely misinterpreted as demonstrating the
general superiority of DSI.
3. It's easier to test for rote memorized material implanted by DSI than for
conceptual understanding of science and its methods induced by guided
inquiry or interactive engagement methods, as witness the inept
questions on California's STAR physics exam [Woolf (2005a,b,c), Hake
(2005g).
4. The U.S. Department of Education (USDE) is evidently bereft of
advisors from physical sciences, relying for advice on psychologists,
psychometricians, statisticians, economists, sociologists,
administrators, medical specialists, policy analysts, and education
specialists, most with a proclivity towards "Random Control Trials"
(RCT's).
5. Douglas Carnine (2000), dean of Direct Instructionists, is a
member of the Technical Advisory Group for the "What Works
Clearinghouse."
6. Psychologist Grover Whitehurst, director of the U.S. Education
Department's Institute of Education Sciences
<http://www.ed.gov/about/offices/list/ies/index.html?exp=0>),
evidently believes that "In science education, there is almost
nothing of proven efficacy" Begley [(2004b)]. . .[Whitehurst is
evidently oblivious of the physics education reform effort [Hake
(2002), Heron & Meltzer (2005), Wieman & Perkins (2005)]
7. Campbell's Law [Campbell (1975), Nichols & Berliner (2005)]: "The
more any quantitative social indicator is used for social decision
making, the more subject it will be to corruption pressures and the
more apt it will be to distort and corrupt the social processes it is
intended to monitor."