Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

[Phys-l] Best Practices in Science Education

If you reply to this long (24kB) post please don't hit the reply button unless you prune the copy of this post that may appear in your reply down to a few relevant lines, otherwise the entire already archived post may be needlessly resent to subscribers. [With humble apologies to Brian Whatcott.]

ABSTRACT: I contest E.D. Hirsch's claims that: (a) the "Best Practice" teaching methods recommended by Zemelman et al. are not research based and are worst practice rather than best practice; and (b) the 1980's research of sociologist J.S. Coleman purportedly showing the inferiority of "progressive ideas" has never been refuted. ************************************

In his book "Schools We Need & Why We Don't Have Them," E.D. Hirsch (1996) presents a truncated version of the "more" and "less" list that characterizes "best practice" reform teaching methods according to the 1993 edition of "Best Practice" [Zemelman et al. (2005)]:

LESS whole-class teacher-directed instruction
LESS student passivity, sitting, listening, receiving
LESS attempts by teachers to cover large amounts of material
LESS rote memorization of facts and details
LESS stress on competition and grades
MORE experiential, inductive, hands-on learning
MORE active learning with all the attendant noise of students doing,
talking, collaborating
MORE deep study of a smaller number of topics
MORE responsibility transferred to students for their work:
goal-setting, record keeping, monitoring, evaluation
MORE choice for students; e.g., picking their own books, etc.
MORE attention to affective needs and varying cognitive styles of
MORE cooperative, collaborative activity.

Hirsch's next paragraph, on p. 173, is [my CAPS]:

"The authors praise the current consensus on these 'child-centered' principles for being 'progressive, developmentally appropriate, research based, and eminently teachable.' THESE CLAIMS ARE NOT, HOWEVER 'RESEARCH BASED' IN THE WAY THE AUTHORS IMPLY. QUITE THE CONTRARY. NO STUDIES OF CHILDREN'S LEARNING IN MAINSTREAM SCIENCE SUPPORT THESE GENERALIZATIONS. With respect to effective learning, the consensus in research is that THEIR RECOMMENDATIONS ARE WORST PRACTICE, NOT 'BEST PRACTICE'."

Insofar as science education is concerned, Hirsch was either unaware or dismissive of the mountain of evidence indicating that reform teaching methods similar to those advocated by Zemelman et al. (2005)] have been shown to be relatively effective in promoting conceptual understanding of science.

In "Will the No Child Left Behind Act Promote Direct Instruction of Science?" [Hake (2005a)] I list 12 sources of evidence for the superiority of "interactive engagement" (IE) or "guided inquiry" instruction over Hirsh's brand of direct instruction in conceptually difficult areas of K-12 science education [bracketed by lines "HHHHHHHHHH. . . "; SEE THAT ARTICLE FOR REFERENCES other than Hake (2005b) and Halloun & Hestenes (1985a,b); slightly updated] :

1. Physics education research [recently reviewed by Hake (2005b)] demonstrating a two-standard-deviation (2sd) superiority [cf. Bloom (1984)] of "interactive engagement" (IE) methods over "traditional" (T) methods in introductory mechanics classes in grades 9-16. Here:

(a) the 2sd difference is in the average *normalized* learning gain <g> on pre/post tests of higher-order conceptual understanding devised through arduous qualitative and quantitative research by disciplinary experts such as Halloun & Hestenes (1985a,b);

(b) the half-century old average *normalized* learning gain
<g> = [<%post - <%pre>]/[100 % - <%pre>],
where angle brackets <. . . > indicate class averages, i.e., <g> is simply the *actual* average gain divided by the *maximum* possible average gain;

(c) IE methods are "designed at least in part to promote conceptual understanding through interactive-engagement of students in heads-on (always) and hands-on (usually) activities which yield immediate feedback through discussion with peers and/or instructors"; and

(d) traditional (T) courses "make little or no use of IE methods, relying primarily on passive-student lectures, recipe labs, and algorithmic-problem exams."

If the "LESS" and "MORE" in the "BEST PRACTICE" list above refer to deviation from the standard high-school and college level introductory physics course, then most of the relatively effective IE courses incorporate many of these strategies (first four "Lesses," first three and last "Mores") while the T courses incorporate almost none of them.
2. Meta-meta-analysis of Lipsey & Wilson (1993): The eleven K-12 science-education studies listed in Table 1 of Lipsey & Wilson (where the test group is characterized by reform methods) yield a total N = 888 students and average effect size <d> = 0.36 [Cohen (1988)]. Most of these studies include grades 4 or 6 to 12 with the effect size control group being traditional direct instruction and the measurement unit being "achievement" or "learning" (presumably as measured by tests). [Cohen's (1988, p. 24) rule of thumb - based on typical results in social science research - that d = 0.2, 0.5, 0.8 imply respectively "small," "medium," and "large" effects. But Cohen cautions that the adjectives "are relative, not only to each other, but to the area of behavioral science or even more particularly to the specific content and research method being employed in any given investigation."]

3. Review by Lawrence Lowery (2003)

4. Review by Heidi Doss-Hammel (2004)

5. Review by Lopez & Schultz (2001)

6. References in AAAS (1993, 2004)

7. References in NRC (1996; 1997a,b; 1999, 2000, 2001, 2003)

6. References in Bransford et al. (2000)

9. References in Anderson (2002)

10. References in Jorgenson & Vanosdall (2002)

11. References in the recent NRC report by Duschl et al. (2007).

12. Review (in progress) of Levy & Century (2005)

It is important to note that none of the research included in 1-10 above includes the straw man of unguided "discovery learning" (UDL) purportedly shown by Klahr & Nigam (2004) to be inferior to Direct Science Instruction. Unfortunately, Duschl et al. (2007) (a) neglect to explain that the UDL employed by Klahr & Nigam is well-known to be ineffective for the average student and is almost never employed in science instruction, (b) appear to be oblivious of "1" above, as is the case for most of the psychology/education/psychometric (PEP) community.

Aside from claiming that the reform methods of science instruction recommended by Zemelman et al. ARE NOT research-based, does Hirsch claim that his "Core Knowledge" brand <> of direct instruction IS research based?

YES! For example, in a speech given at Harvard University in October 1999 and adapted by "Education Next," Hirsh (2001) praises the research of sociologist James Coleman [bracketed by lines "HHHHHHHHHHH. . . . ."; my CAPS; my inserts at ". . . [insert]. . . .":

In the 1980s, the distinguished sociologist James Coleman. . . .
[<>]. . . conducted carefully controlled, large-sample research. . .[Hirsh fails to provide references but he is apparently referring to Coleman et al. (1982a,b) and Coleman & Hoffer (1987)]. . . demonstrated the ineffectiveness of progressive methods in raising general academic achievement and in closing the achievement gap between advantaged and disadvantaged students. Coleman found that CATHOLIC SCHOOLS ACHIEVE MORE EDUCATIONAL EQUITY THAN PUBLIC SCHOOLS BECAUSE they follow a rich and demanding curriculum; provide a structured, orderly environment; OFFER LOTS OF EXPLICIT INSTRUCTION, INCLUDING DRILL AND PRACTICE; and expect every child to reach minimal goals in each subject by the end of the year.

All of this stands in stark contrast to the progressive ideals of unstructured, implicit teaching and "individually tailored" instruction that now predominate in public schools. . . .[These may be the progressive ideals as seen by Hirsch, but they are not the progressive ideals of Dewey - see e.g., Ansbacher (2000) - that undergird the relatively effective "Interactive Engagement" (IE) methods referred to above.]. . . . As a result, disadvantaged children prosper academically in Catholic schools, and the schools narrow the gaps among races and social classes. When criticized for condemning public schools, Coleman pointed out that the very same democratic results were being achieved by the few public schools that were also defying progressivist doctrine. Along with large-scale international comparisons. . . [as far as I know most TIMSS analysts believe that the relatively poor showing of U.S. students was due to a science curriculum that's "a mile wide and an inch deep" rather than their subjection to "progressivist doctrine"]. . . COLEMAN'S WORK IS THE MOST RELIABLE OBSERVATIONAL DATA THAT WE HAVE REGARDING THE VALIDITY OF PROGRESSIVE IDEAS, AND IT HAS NEVER BEEN REFUTED.

NEVER BEEN REFUTED? According to Kim & Placier (2004) [bracketed by lines "K&P-K&P-K&P-. . . ."; see their article for references other than Coleman et al. (1982a,b), Coleman & Hoffer (1987), and Witte (1992); my CAPS]:

Comparison of academic achievement for Catholic versus public secondary schools has been an active field of research for nearly 20 years, beginning with Coleman, Hoffer and Kilgore's (1982a, 1982b) analysis of 1980 High School and Beyond (HSB) data . . . [<>]. . ., which found a positive "Catholic school effect." This work has been grounded in social capital theory, which explains the Catholic school advantage in terms of the value for young people of being embedded in a network of relationships, in this case a network based on religious association (Coleman and Hoffer 1987). Subsequent studies have either lent support, albeit sometimes qualified, to their findings (Bryk, Lee, and Holland 1993; Gamoran 1992; Hoffer 2000; Hoffer, Greeley and Coleman 1985; Jencks 1985; Jensen 1986; Keith 1985; Marsh 1991; Marsh and Grayson 1990; Riordan 1985; Sander 1996) or CALLED THEM INTO QUESTION (Alexander 1985; Gamoran 1996; Graetz 1990; LePore and Warren 1997; Noell 1982; Willms 1985).

Coleman et al. (1982a) noted that findings of public-private school comparisons could have implications for policy decisions and parent choices-implications that have become even more salient today. A decade later, however, WITTE (1992) ARGUED THAT IN STUDIES WITH PROPER CONTROLS, ACHIEVEMENT DIFFERENCES BETWEEN PUBLIC AND PRIVATE SCHOOLS WERE TOO SMALL AND UNCERTAIN TO HAVE POLICY IMPORT.

Witte's abstract reads as follows [bracketed by lines "WWWWWW. . ."; my CAPS]:

The purpose of this paper is to determine whether existing research allows us to answer a crucial question: are student achievement differences between public and private schools large enough and certain enough to be of relevance to policymakers considering choice proposals. To do this, I analyze research based on the "High School and Beyond" study. The argument of this paper is that there is little substantive evidence in this research that has policy relevance for educational choice. Specifically I ARGUE THAT ANY DIFFERENCES IN ACHIEVEMENT BETWEEN PUBLIC AND PRIVATE SCHOOLS THAT ARE STATISTICALLY SIGNIFICANT AFTER PROPERLY MODELING ACHIEVEMENT GAINS ARE TRIVIAL IN SIZE AND HIGHLY UNCERTAIN. I conclude by questioning "High School and Beyond", or research similarly structured, will ever answer the question of whether public or private schools produce greater student achievement.

A somewhat less trenchant version of this post titled "Re: Top 10 List of Research-Based Recommendations" [Hake (2006)] has appeared on the EDDRA, POD, and STLHE-L discussion lists.

Richard Hake, Emeritus Professor of Physics, Indiana University
24245 Hatteras Street, Woodland Hills, CA 91367

REFERENCES [Tiny URL's courtesy <>.]
Ansbacher, T. 2000. "An interview with John Dewey on science
education." The Physics Teacher 38(4): 224-227; freely online at
<> as a 1.3 MB pdf.

Bloom, B.S. 1984. "The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring," Educational Researcher 13(6): 4-16. Bloom wrote: "Using the standard deviation (sigma) of the control (conventional) class, it was typically found that the average student under tutoring was about two standard deviations above the average of the control class . . . The tutoring process demonstrates that *most* of the students do have the potential to reach this high level of learning. I believe an important task of research and instruction is to seek ways of accomplishing this under more practical and realistic conditions than the one-to-one tutoring, which is too costly for most societies to bear on a large scale.
This is the '2 sigma' problem."

Coleman, J.S., Hoffer, T., and Kilgore, S. 1982a. "Cognitive outcomes in public and private schools," Sociology of Education, 55(2-3): 65-76.

Coleman, J.S., Hoffer, T., and Kilgore, S. 1982b. "High school achievement: Public, Catholic and private schools compared." Basic Books.

Coleman, J.S., and Hoffer, T. 1987. "Public and Private High Schools: The Impact of Communities." Basic Books.

Duschl, R.A., H.A. Schweingruber, and A.W. Shouse, eds. 2007. "Taking Science to School: Learning and Teaching Science in Grades K-8." National Academies Press; information is online at <>. See also the press release at <>. The NAP description reads: "What is science for a child? How do children learn about science and how to do science? Drawing on a vast array of work from neuroscience to classroom observation, "Taking Science to School" provides a comprehensive picture of what we know about teaching and learning science from kindergarten through eighth grade. By looking at a broad range of questions, this book provides a basic foundation for guiding science teaching and supporting students in their learning." See also the report on Duschl et al. by Valerie Strauss (2006) in the Washington Post.

Hake, R.R. 2004. "Direct Science Instruction Suffers a Setback in California - Or Does It?" AAPT Announcer 34(2): 177; online as reference 33 at <>, or download directly by clicking on
<> (420 KB) [about 160 references and 180 hot-linked URL's]. A pdf version of the slides shown at the meeting is also available at ref. 33 or can be downloaded directly by clicking on <> (132 kB).

Hake, R.R. 2005a. "Will the No Child Left Behind Act Promote Direct Instruction of Science?" Am. Phys. Soc. 50: 851 (2005); APS March Meeting, Los Angles, CA. 21-25 March; online as ref. 36 at <>, or download directly by clicking on
<> (256 kB). See also Hake (2004).

Hake, R. R. 2005b. "The Physics Education Reform Effort: A Possible Model for Higher Education," online at <> (100 kB). This is a slightly edited version of an article that was (a) published in the National Teaching and Learning Forum 15(1), December 2005, online to
subscribers at <>, and (b)
disseminated by the Tomorrow's Professor list
<> as Msg. 698 on 14 Feb 2006.

Hake, R.R. 2006. "Re: Top 10 List of Research-Based Recommendations," online at
<>. Post of 27 Sep 2006 14:47:25-0700 to EDDRA, POD, and STLHE-L.

Halloun, I. & D. Hestenes. 1985a. "The initial knowledge state of college physics students." Am. J. Phys. 53: 1043-1055; online at
<>. Contains the "Mechanics Diagnostic" test, precursor to the widely used "Force Concept Inventory" [Hestenes et al. (1992)].

Halloun, I. & D. Hestenes. 1985b. "Common sense concepts about motion." Am. J. Phys. 53:1056-1065; online at <>.

Hestenes, D., M. Wells, & G. Swackhamer, 1992. "Force Concept Inventory," Phys. Teach. 30: 141-158; online (except for the test itself) at
<>. The 1995 revision by Halloun, Hake, Mosca, & Hestenes is online (password protected) at the same URL, and is available in English, Spanish, German, Malaysian, Chinese, Finnish, French, Turkish, Swedish, and Russian.

Hirsch, E.D. 1996. "The Schools We Need & Why We Don't Have Them." Doubleday. information at <>.

Hirsch, E.D. 2001. "Romancing the Child: Progressivism's philosophical roots," Education Next, 2001 issue #1; online at <>.

Kim. M.M. & and M. Placier. 2004. "Comparison of Academic Development in Catholic Versus Non-Catholic Private Secondary Schools," Education Policy Analysis Archives, February; online at <> and also <> (180kB).

Strauss. V. 2004. "Back to Basics vs. Hands-On Instruction: California Rethinks Science Labs." Washington Post, Tuesday, 3 February. page A12; online at <> (complete report free to subscribers, $3.95 to non subscribers, a summary is free to all).

Strauss, V. 2006. "Report Calls for Improvement in K-8 Science Education." Washington Post, 22 September, online at
<>, or more compactly at <>. See also Strauss's (2004) cogent report on the "Back to Basics vs. Hands-On War."

Witte, J.F. 1992. "Private versus public school achievement: Are there findings that should affect the educational choice debate? Economics of Education Review 11(4): 371-394; online at the Elsevier site <> [complete article free to subscribers and $30 to non-subscribers, abstract free to all).

Zemelman, S., H. Daniels, A. Hyde. 2005. "Best Practice: Today's Standards for Teaching and Learning in America's Schools." Heineman, third edition. information at <>. Heinean information at <>