If you reply to this long (24kB) post please don't hit the reply
button unless you prune the copy of this post that may appear in your
reply down to a few relevant lines, otherwise the entire already
archived post may be needlessly resent to subscribers. [With humble
apologies to Brian Whatcott.]
************************************
ABSTRACT: I contest E.D. Hirsch's claims that: (a) the "Best
Practice" teaching methods recommended by Zemelman et al. are not
research based and are worst practice rather than best practice; and
(b) the 1980's research of sociologist J.S. Coleman purportedly
showing the inferiority of "progressive ideas" has never been
refuted.
************************************
In his book "Schools We Need & Why We Don't Have Them," E.D. Hirsch
(1996) presents a truncated version of the "more" and "less" list
that characterizes "best practice" reform teaching methods according
to the 1993 edition of "Best Practice" [Zemelman et al. (2005)]:
BEST PRACTICE
LESS whole-class teacher-directed instruction
LESS student passivity, sitting, listening, receiving
LESS attempts by teachers to cover large amounts of material
LESS rote memorization of facts and details
LESS stress on competition and grades
MORE experiential, inductive, hands-on learning
MORE active learning with all the attendant noise of students doing,
talking, collaborating
MORE deep study of a smaller number of topics
MORE responsibility transferred to students for their work:
goal-setting, record keeping, monitoring, evaluation
MORE choice for students; e.g., picking their own books, etc.
MORE attention to affective needs and varying cognitive styles of
students
MORE cooperative, collaborative activity.
Hirsch's next paragraph, on p. 173, is [my CAPS]:
"The authors praise the current consensus on these 'child-centered'
principles for being 'progressive, developmentally appropriate,
research based, and eminently teachable.' THESE CLAIMS ARE NOT,
HOWEVER 'RESEARCH BASED' IN THE WAY THE AUTHORS IMPLY. QUITE THE
CONTRARY. NO STUDIES OF CHILDREN'S LEARNING IN MAINSTREAM SCIENCE
SUPPORT THESE GENERALIZATIONS. With respect to effective learning,
the consensus in research is that THEIR RECOMMENDATIONS ARE WORST
PRACTICE, NOT 'BEST PRACTICE'."
Insofar as science education is concerned, Hirsch was either unaware
or dismissive of the mountain of evidence indicating that reform
teaching methods similar to those advocated by Zemelman et al.
(2005)] have been shown to be relatively effective in promoting
conceptual understanding of science.
In "Will the No Child Left Behind Act Promote Direct Instruction of
Science?" [Hake (2005a)] I list 12 sources of evidence for the
superiority of "interactive engagement" (IE) or "guided inquiry"
instruction over Hirsh's brand of direct instruction in conceptually
difficult areas of K-12 science education [bracketed by lines
"HHHHHHHHHH. . . "; SEE THAT ARTICLE FOR REFERENCES other than Hake
(2005b) and Halloun & Hestenes (1985a,b); slightly updated] :
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
1. Physics education research [recently reviewed by Hake (2005b)]
demonstrating a two-standard-deviation (2sd) superiority [cf. Bloom
(1984)] of "interactive engagement" (IE) methods over "traditional"
(T) methods in introductory mechanics classes in grades 9-16. Here:
(a) the 2sd difference is in the average *normalized* learning gain
<g> on pre/post tests of higher-order conceptual understanding
devised through arduous qualitative and quantitative research by
disciplinary experts such as Halloun & Hestenes (1985a,b);
(b) the half-century old average *normalized* learning gain
<g> = [<%post - <%pre>]/[100 % - <%pre>],
where angle brackets <. . . > indicate class averages, i.e., <g> is
simply the *actual* average gain divided by the *maximum* possible
average gain;
(c) IE methods are "designed at least in part to promote conceptual
understanding through interactive-engagement of students in heads-on
(always) and hands-on (usually) activities which yield immediate
feedback through discussion with peers and/or instructors"; and
(d) traditional (T) courses "make little or no use of IE methods,
relying primarily on passive-student lectures, recipe labs, and
algorithmic-problem exams."
If the "LESS" and "MORE" in the "BEST PRACTICE" list above refer to
deviation from the standard high-school and college level
introductory physics course, then most of the relatively effective IE
courses incorporate many of these strategies (first four "Lesses,"
first three and last "Mores") while the T courses incorporate almost
none of them.
2. Meta-meta-analysis of Lipsey & Wilson (1993): The eleven K-12
science-education studies listed in Table 1 of Lipsey & Wilson (where
the test group is characterized by reform methods) yield a total N =
888 students and average effect size <d> = 0.36 [Cohen (1988)]. Most
of these studies include grades 4 or 6 to 12 with the effect size
control group being traditional direct instruction and the
measurement unit being "achievement" or "learning" (presumably as
measured by tests). [Cohen's (1988, p. 24) rule of thumb - based on
typical results in social science research - that d = 0.2, 0.5, 0.8
imply respectively "small," "medium," and "large" effects. But Cohen
cautions that the adjectives "are relative, not only to each other,
but to the area of behavioral science or even more particularly to
the specific content and research method being employed in any given
investigation."]
3. Review by Lawrence Lowery (2003)
4. Review by Heidi Doss-Hammel (2004)
5. Review by Lopez & Schultz (2001)
6. References in AAAS (1993, 2004)
7. References in NRC (1996; 1997a,b; 1999, 2000, 2001, 2003)
6. References in Bransford et al. (2000)
9. References in Anderson (2002)
10. References in Jorgenson & Vanosdall (2002)
11. References in the recent NRC report by Duschl et al. (2007).
12. Review (in progress) of Levy & Century (2005)
It is important to note that none of the research included in 1-10
above includes the straw man of unguided "discovery learning" (UDL)
purportedly shown by Klahr & Nigam (2004) to be inferior to Direct
Science Instruction. Unfortunately, Duschl et al. (2007) (a) neglect
to explain that the UDL employed by Klahr & Nigam is well-known to be
ineffective for the average student and is almost never employed in
science instruction, (b) appear to be oblivious of "1" above, as is
the case for most of the psychology/education/psychometric (PEP)
community.
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
Aside from claiming that the reform methods of science instruction
recommended by Zemelman et al. ARE NOT research-based, does Hirsch
claim that his "Core Knowledge" brand <http://coreknowledge.org/CK/>
of direct instruction IS research based?
YES! For example, in a speech given at Harvard University in October
1999 and adapted by "Education Next," Hirsh (2001) praises the
research of sociologist James Coleman [bracketed by lines
"HHHHHHHHHHH. . . . ."; my CAPS; my inserts at ". . . [insert]. . .
.":
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
In the 1980s, the distinguished sociologist James Coleman. . . .
[<http://chronicle.uchicago.edu/950330/coleman.shtml>]. . .
conducted carefully controlled, large-sample research. . .[Hirsh
fails to provide references but he is apparently referring to Coleman
et al. (1982a,b) and Coleman & Hoffer (1987)]. . . demonstrated the
ineffectiveness of progressive methods in raising general academic
achievement and in closing the achievement gap between advantaged and
disadvantaged students. Coleman found that CATHOLIC SCHOOLS ACHIEVE
MORE EDUCATIONAL EQUITY THAN PUBLIC SCHOOLS BECAUSE they follow a
rich and demanding curriculum; provide a structured, orderly
environment; OFFER LOTS OF EXPLICIT INSTRUCTION, INCLUDING DRILL AND
PRACTICE; and expect every child to reach minimal goals in each
subject by the end of the year.
All of this stands in stark contrast to the progressive ideals of
unstructured, implicit teaching and "individually tailored"
instruction that now predominate in public schools. . . .[These may
be the progressive ideals as seen by Hirsch, but they are not the
progressive ideals of Dewey - see e.g., Ansbacher (2000) - that
undergird the relatively effective "Interactive Engagement" (IE)
methods referred to above.]. . . . As a result, disadvantaged
children prosper academically in Catholic schools, and the schools
narrow the gaps among races and social classes. When criticized for
condemning public schools, Coleman pointed out that the very same
democratic results were being achieved by the few public schools that
were also defying progressivist doctrine. Along with large-scale
international comparisons. . . [as far as I know most TIMSS analysts
believe that the relatively poor showing of U.S. students was due to
a science curriculum that's "a mile wide and an inch deep" rather
than their subjection to "progressivist doctrine"]. . . COLEMAN'S
WORK IS THE MOST RELIABLE OBSERVATIONAL DATA THAT WE HAVE REGARDING
THE VALIDITY OF PROGRESSIVE IDEAS, AND IT HAS NEVER BEEN REFUTED.
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
NEVER BEEN REFUTED? According to Kim & Placier (2004) [bracketed by
lines "K&P-K&P-K&P-. . . ."; see their article for references other
than Coleman et al. (1982a,b), Coleman & Hoffer (1987), and Witte
(1992); my CAPS]:
K&P-K&P-K&P-K&P-K&P-K&P-K&P-K&P-K&P-K&P-K&P-K&P
Comparison of academic achievement for Catholic versus public
secondary schools has been an active field of research for nearly 20
years, beginning with Coleman, Hoffer and Kilgore's (1982a, 1982b)
analysis of 1980 High School and Beyond (HSB) data . . .
[<http://nces.ed.gov/surveys/hsb/>]. . ., which found a positive
"Catholic school effect." This work has been grounded in social
capital theory, which explains the Catholic school advantage in terms
of the value for young people of being embedded in a network of
relationships, in this case a network based on religious association
(Coleman and Hoffer 1987). Subsequent studies have either lent
support, albeit sometimes qualified, to their findings (Bryk, Lee,
and Holland 1993; Gamoran 1992; Hoffer 2000; Hoffer, Greeley and
Coleman 1985; Jencks 1985; Jensen 1986; Keith 1985; Marsh 1991; Marsh
and Grayson 1990; Riordan 1985; Sander 1996) or CALLED THEM INTO
QUESTION (Alexander 1985; Gamoran 1996; Graetz 1990; LePore and
Warren 1997; Noell 1982; Willms 1985).
Coleman et al. (1982a) noted that findings of public-private school
comparisons could have implications for policy decisions and parent
choices-implications that have become even more salient today. A
decade later, however, WITTE (1992) ARGUED THAT IN STUDIES WITH
PROPER CONTROLS, ACHIEVEMENT DIFFERENCES BETWEEN PUBLIC AND PRIVATE
SCHOOLS WERE TOO SMALL AND UNCERTAIN TO HAVE POLICY IMPORT.
K&P-K&P-K&P-K&P-K&P-K&P-K&P-K&P-K&P-K&P-K&P-K&P
Witte's abstract reads as follows [bracketed by lines "WWWWWW. . ."; my CAPS]:
WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW
The purpose of this paper is to determine whether existing research
allows us to answer a crucial question: are student achievement
differences between public and private schools large enough and
certain enough to be of relevance to policymakers considering choice
proposals. To do this, I analyze research based on the "High School
and Beyond" study. The argument of this paper is that there is little
substantive evidence in this research that has policy relevance for
educational choice. Specifically I ARGUE THAT ANY DIFFERENCES IN
ACHIEVEMENT BETWEEN PUBLIC AND PRIVATE SCHOOLS THAT ARE STATISTICALLY
SIGNIFICANT AFTER PROPERLY MODELING ACHIEVEMENT GAINS ARE TRIVIAL IN
SIZE AND HIGHLY UNCERTAIN. I conclude by questioning "High School and
Beyond", or research similarly structured, will ever answer the
question of whether public or private schools produce greater student
achievement.
WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW
A somewhat less trenchant version of this post titled "Re: Top 10
List of Research-Based Recommendations" [Hake (2006)] has appeared on
the EDDRA, POD, and STLHE-L discussion lists.
Bloom, B.S. 1984. "The 2 Sigma Problem: The Search for Methods of
Group Instruction as Effective as One-to-One Tutoring," Educational
Researcher 13(6): 4-16. Bloom wrote: "Using the standard deviation
(sigma) of the control (conventional) class, it was typically found
that the average student under tutoring was about two standard
deviations above the average of the control class . . . The tutoring
process demonstrates that *most* of the students do have the
potential to reach this high level of learning. I believe an
important task of research and instruction is to seek ways of
accomplishing this under more practical and realistic conditions than
the one-to-one tutoring, which is too costly for most societies to
bear on a large scale.
This is the '2 sigma' problem."
Coleman, J.S., Hoffer, T., and Kilgore, S. 1982a. "Cognitive outcomes
in public and private schools," Sociology of Education, 55(2-3):
65-76.
Coleman, J.S., Hoffer, T., and Kilgore, S. 1982b. "High school
achievement: Public, Catholic and private schools compared." Basic
Books.
Coleman, J.S., and Hoffer, T. 1987. "Public and Private High Schools:
The Impact of Communities." Basic Books.
Duschl, R.A., H.A. Schweingruber, and A.W. Shouse, eds. 2007. "Taking
Science to School: Learning and Teaching Science in Grades K-8."
National Academies Press; information is online at
<http://www.nap.edu/catalog/11625.html>. See also the press release
at
<http://www8.nationalacademies.org/onpinews/newsitem.aspx?RecordID=11625>.
The NAP description reads: "What is science for a child? How do
children learn about science and how to do science? Drawing on a vast
array of work from neuroscience to classroom observation, "Taking
Science to School" provides a comprehensive picture of what we know
about teaching and learning science from kindergarten through eighth
grade. By looking at a broad range of questions, this book provides a
basic foundation for guiding science teaching and supporting students
in their learning." See also the report on Duschl et al. by Valerie
Strauss (2006) in the Washington Post.
Halloun, I. & D. Hestenes. 1985a. "The initial knowledge state of
college physics students." Am. J. Phys. 53: 1043-1055; online at
<http://modeling.asu.edu/R&E/Research.html>. Contains the "Mechanics
Diagnostic" test, precursor to the widely used "Force Concept
Inventory" [Hestenes et al. (1992)].
Hestenes, D., M. Wells, & G. Swackhamer, 1992. "Force Concept
Inventory," Phys. Teach. 30: 141-158; online (except for the test
itself) at
<http://modeling.asu.edu/R&E/Research.html>. The 1995 revision by
Halloun, Hake, Mosca, & Hestenes is online (password protected) at
the same URL, and is available in English, Spanish, German,
Malaysian, Chinese, Finnish, French, Turkish, Swedish, and Russian.
Hirsch, E.D. 1996. "The Schools We Need & Why We Don't Have Them."
Doubleday. Amazon.com information at <http://tinyurl.com/jz6aq>.
Strauss. V. 2004. "Back to Basics vs. Hands-On Instruction:
California Rethinks Science Labs." Washington Post, Tuesday, 3
February. page A12; online at <http://tinyurl.com/fmv83> (complete
report free to subscribers, $3.95 to non subscribers, a summary is
free to all).
Witte, J.F. 1992. "Private versus public school achievement: Are
there findings that should affect the educational choice debate?
Economics of Education Review 11(4): 371-394; online at the Elsevier
site <http://tinyurl.com/gym9v> [complete article free to subscribers
and $30 to non-subscribers, abstract free to all).