Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

Re: [Phys-L] Seeking a study



D.J. Wagner initiated a thread "Seeking a study," on the January 2015 PhysLrnR archives <http://bit.ly/1BioSHS> which had grown to 14 posts on 18 Jan 2015 19:00-0700.

[To access the archives of PhysLnR one needs to subscribe :-(, but that takes only a few minutes by clicking on <http://bit.ly/nG318r> and then clicking on "Join or Leave PHYSLRNR-LIST." If you're busy, then subscribe using the "NOMAIL" option under "Miscellaneous." Then, as a subscriber, you may access the archives and/or post messages at any time, while receiving NO MAIL from the list!]

D.J. wrote :

"My colleague asked an interesting question, essentially if anyone has measured the 'background' effect of the first semester of college on the FCI or FMCE or such, i.e., is there experimental evidence for the common expectation of a zero change in diagnostic scores in the absence of instruction? Does anyone know of any studies that look at this?"

This well-known threat to internal validity is known as "maturation"- see e.g., page 57 of Shadish, Cook, & Campbell (2002). That it is inconsequential in the pre-post testing evidence examined in my article "Interactive-engagement vs traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses" [Hake (1998a)] is evident from the facts that (quoting from the abstract of Hake (1998a), emphasis added):

"A survey of pre/post test data using the Halloun-Hestenes Mechanics Diagnostic test or more recent Force Concept Inventory is reported for 62 introductory physics courses enrolling a total number of students N = 6542. A consistent analysis over diverse student populations in high schools, colleges, and universities is obtained if a rough measure of the average effectiveness of a course in promoting conceptual understanding is taken to be the average normalized gain <g>. The latter is defined as the ratio of the actual average gain (%<post> – %<pre>) to the maximum possible average gain (100 –%<pre>). Fourteen 'traditional' (T) courses (N = 2084) which made little or no use of interactive-engagement (IE) methods achieved an average gain <g> T-ave = 0.23 ± 0.04 (std dev). In sharp contrast, forty-eight courses (N = 4458) which made substantial use of IE methods achieved an average gain <g> IE-ave = 0.48 ± 0.14 (std dev), almost two standard deviations of <g> IE-ave above that of the traditional courses."

In the above (quoting from Hake (1998a): "For survey classification and analysis purposes I define:

(a) "Interactive Engagement" (IE) methods as those designed at least in part to promote conceptual understanding through active engagement of students in heads-on (always) and hands-on (usually) activities which yield immediate feedback through discussion with peers and/or instructors, all as judged by their literature descriptions;

(b) "Traditional" (T) courses as those reported by instructors to make little or no use of IE methods, relying primarily on passive-student lectures, recipe labs, and algorithmic problem exams;

(c) "Interactive Engagement" (IE) courses as those reported by instructors to make substantial use of IE methods.

The idea that greater average maturation of students in IE courses over those in T courses rather than the difference in pedagogy is responsible for the almost two standard deviation superiority of average normalized gains <g> of IE over T courses is hardly credible, considering that:

A. As stated in Hake (2008):

(a) within any one institution the test [Interactive Engagement(IE)] and control [Traditional (T)] groups were drawn from the same generic introductory course taken by relatively homogeneous groups of students, and

(b) IE-course teachers in all institutions are drawn from the same generic pool of introductory course physics teachers who, judging from the uniformly poor average normalized gains <g> they obtain in teaching traditional (T) courses, do not vary greatly in their ability to enhance student learning.



B. As stated in Hake (2011):

(a) As of 2006, average normalized gain differences between T and IE courses that are consistent with the work of Hake (1998a,b) and Figure 1 [of Hake (1998a] had been reported in at least 25 other research papers, as given and referenced in “Design-Based Research in Physics Education Research: A Review” [Hake (2008, p.12)], and

(b) As indicated in Hake (2008), this consistency of the results of many investigators in various institutions working with different student populations with the results of Hake (1998a,b))constitutes the most important single warrant for the validity of conclusion in Hake (1998a) that: "The conceptual and problem-solving test results strongly suggest that the classroom use of IE methods can increase mechanics-course effectiveness well beyond that obtained in traditional practice."



Richard Hake, Emeritus Professor of Physics, Indiana University. LINKS TO: Academia - <http://bit.ly/a8ixxm>; Articles - <http://bit.ly/a6M5y0>; Blog - <http://bit.ly/9yGsXh>; Facebook - <http://on.fb.me/XI7EKm>; GooglePlus - <http://bit.ly/KwZ6mE>; Google Scholar - <http://bit.ly/Wz2FP3>; LinkedIn - <http://linkd.in/14uycpW>; Socratic Dialogue Inducing (SDI) Labs - <http://bit.ly/9nGd3M>; Twitter - <http://bit.ly/juvd52>.



REFERENCES[All URLs accessed on 18 Jan 2015 and shortened by <http://bit.ly/>.

Hake, R.R. 1998a. "Interactive-engagement vs traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses," Am. J. Phys. 66: 64-74; online as an 84 kB pdf at <http://bit.ly/9484DG>. See also the crucial but generally ignored companion paper Hake (1998b).

Hake, R.R. 1998b. “Interactive-engagement methods in introductory mechanics courses,” online as a 108 kB pdf at <http://bit.ly/aH2JQN>. A crucial companion paper to Hake (1998a): average pre/post test scores, standard deviations, instructional methods, materials used, institutions, and instructors for each of the survey courses of Hake (1998a) are tabulated and referenced. In addition the paper includes: (a) case histories for the seven IE courses of Hake (1998a) whose effectiveness as gauged by pre-to-post test gains was close to those of T courses, (b) advice for implementing IE methods, and (c) suggestions for further research. Submitted on 6/19/98 to the “Physics Education Research Supplement” (PERS) of the American Journal of Physics, but rejected by its editor on the grounds that the very transparent, well organized, and crystal clear Physical-Review-type data tables were "impenetrable"!

Hake, R.R. 2008. "Design-Based Research in Physics Education Research: A Review," in Kelly, Lesh, & Baek (2008)]. A pre-publication version of that chapter is online as a 1.1 MB pdf at <http://bit.ly/9kORMZ>.Hake, R.R. 2011. "The Impact of Concept Inventories On Physics Education and It’s Relevance For Engineering Education," invited talk, 8 August, second annual NSF-sponsored "National Meeting on STEM Concept Inventories," Washington, D.C., online as an 8.7 MB pdf at <http://bit.ly/nmPY8F> and as ref. 64 at <http://bit.ly/a6M5y0>.

Kelly, A.E., R.A. Lesh, & J.Y. Baek. 2008. "Handbook of Design Research Methods in Education: Innovations in Science, Technology, Engineering, and Mathematics Learning and Teaching," Routledge. Publisher's information at <http://bit.ly/dkLabI>; Amazon.com information at <http://amzn.to/gtRpbU>.

Shadish, W.R., T.D. Cook, & D.T. Campbell. 2002. "Experimental and Quasi- Experimental Designs for Generalized Causal Inference." Houghton Mifflin. Amazon.com information at <http://amzn.to/JWWMVi>. A goldmine of references on social science research. Online as a 1.8 MB pdf at <http://bit.ly/1ywfO01>.