Chemed-L, Phys-L, Physhare, and STLHE-L with their respective
limitations of 150, 300, 599, and 150 lines per post have all been
mercifully shielded from my recent 800-line post:
Hake, R.R. 2003. "Re: Normalized Gain (was Inquiry method and
motivation)," post of 24 Nov 2003 17:12:05-0800 to EvalTalk,
Math-Learn, PhysLrnR, & POD; online at
<http://listserv.nd.edu/cgi-bin/wa?A2=ind0311&L=pod&O=D&P=18573>.
Later sent to AERA-D, ASSESS, and Biopi-L; and in abstract form to
Chemed-L, Phys-L, Physhare, STLHE-L, edstat, FYA, and AP-Physics.
If you're:
(a) not interested, please hit the delete button.
(b) interested in scanning the entire post, click on the above URL
(c) interested in scanning an abstract see the APPENDIX.
AKPPENDIX [Abstract of Hake (2003)]
In his Math-Learn post of 21 Nov 2003 titled "Re: Normalized Gain."
<Cmpalmer2@aol.com> asked three good questions:
1. Can you tell me more about normalized gain, and/or suggest a good
resource? . . . .
2. Is "normalized gain" valid on a non-standardized test?
3. Can I compare "normalized gain" even though I changed tests, a
couple of years ago, because we up-dated the text book? . . . . or
should I limit my comparisons to before the change and after the
change?"
In the following post I attempt to answer the above three questions
and also consider two others:
4. Are Clement's comments on the "average normalized gain" <g> and
the effect size "d" correct?
5. Is pre/post testing becoming more popular in disciplines other than physics?
I. DEFINITION OF THE AVERAGE NORMALIZED GAIN <g>
In Hake (2002b) I wrote (slightly edited):
"The half-century old average normalized gain <g> for a treatment has
been independently defined [Hovland et al. (1949), Gery (1972), Hake
(1998a)] as <g> = Gain/[Gain (maximum possible). In terms of %scores
<g> = (<%post> - <%pre>) / (100% - <%pre>)
Where angle brackets <. . . .> indicate the class average (preferably
only for students who have taken both the pre and post tests).
Thus, e.g., if a class averaged 40% on the pretest, and 60% on the
posttest then the class-average normalized gain <g> = (60% -
40%)/(100% - 40%) = 20%/60% = 0.33. Ever since the work of Hovland et
al. (1949) it's been known by pre/post cognoscente (up until about
1998 probably less than 100 people worldwide) that <g> IS A MUCH
BETTER INDICATOR OF THE EXTENT TO WHICH A TREATMENT IS EFFECTIVE THAN
IS EITHER GAIN OR POSTTEST, for example, if the treatment yields
<g> > 0.3 for a mechanics course, where <g> is calculated from
pre/post testing with the "Force Concept Inventory" (Hestenes et al.
1992) then the course could be considered as in the
"interactive-engagement zone" (Hake 1998a).
. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . .
V. IS PRE/POST TESTING BECOMING MORE POPULAR IN DISCIPLINES OTHER THAN PHYSICS?
The answer is "YES." The relatively early pre/post test work of:
(a) Halloun & Hestenes (1985a,b) in developing the "Mechanics
Diagnostic" test (precursor to the much-used "Force Concept
Inventory" (FCI) [Hestenes et al. (1992)]);
(b) Hake (1987) in physics;
(b) Paden & Moyer (1969) in economics;
(c) Sundberg & Moncada (1994) in biology;
(d) Milford (1996) in chemistry; and
(e) Zeilik et al. (1997) in astronomy.
has been followed by efforts to use valid and consistently reliable
such as the Force Concept Inventory (FCI) [Hestenes et al. (1992) in
a pre/post mode so as to formatively measure the need for, and the
results of, reform methods in:
(1) physics [for reviews see Hake (2002a,b)];
(2) biology [e.g., Lawson (2001), Anderson et al. (2002), Klymkowsky
et al. ( 2003), Roy (2001, 2003), Sundberg (2002), and Wood (2003);