Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

Re: Normalized Gain



Chemed-L, Phys-L, Physhare, and STLHE-L with their respective
limitations of 150, 300, 599, and 150 lines per post have all been
mercifully shielded from my recent 800-line post:

Hake, R.R. 2003. "Re: Normalized Gain (was Inquiry method and
motivation)," post of 24 Nov 2003 17:12:05-0800 to EvalTalk,
Math-Learn, PhysLrnR, & POD; online at
<http://listserv.nd.edu/cgi-bin/wa?A2=ind0311&L=pod&O=D&P=18573>.
Later sent to AERA-D, ASSESS, and Biopi-L; and in abstract form to
Chemed-L, Phys-L, Physhare, STLHE-L, edstat, FYA, and AP-Physics.

If you're:

(a) not interested, please hit the delete button.

(b) interested in scanning the entire post, click on the above URL

(c) interested in scanning an abstract see the APPENDIX.

Richard Hake, Emeritus Professor of Physics, Indiana University
24245 Hatteras Street, Woodland Hills, CA 91367
<rrhake@earthlink.net>
<http://www.physics.indiana.edu/~hake>
<http://www.physics.indiana.edu/~sdi>


AKPPENDIX [Abstract of Hake (2003)]
In his Math-Learn post of 21 Nov 2003 titled "Re: Normalized Gain."
<Cmpalmer2@aol.com> asked three good questions:

1. Can you tell me more about normalized gain, and/or suggest a good
resource? . . . .

2. Is "normalized gain" valid on a non-standardized test?

3. Can I compare "normalized gain" even though I changed tests, a
couple of years ago, because we up-dated the text book? . . . . or
should I limit my comparisons to before the change and after the
change?"

In the following post I attempt to answer the above three questions
and also consider two others:

4. Are Clement's comments on the "average normalized gain" <g> and
the effect size "d" correct?

5. Is pre/post testing becoming more popular in disciplines other than physics?

I. DEFINITION OF THE AVERAGE NORMALIZED GAIN <g>
In Hake (2002b) I wrote (slightly edited):

"The half-century old average normalized gain <g> for a treatment has
been independently defined [Hovland et al. (1949), Gery (1972), Hake
(1998a)] as <g> = Gain/[Gain (maximum possible). In terms of %scores

<g> = (<%post> - <%pre>) / (100% - <%pre>)

Where angle brackets <. . . .> indicate the class average (preferably
only for students who have taken both the pre and post tests).

Thus, e.g., if a class averaged 40% on the pretest, and 60% on the
posttest then the class-average normalized gain <g> = (60% -
40%)/(100% - 40%) = 20%/60% = 0.33. Ever since the work of Hovland et
al. (1949) it's been known by pre/post cognoscente (up until about
1998 probably less than 100 people worldwide) that <g> IS A MUCH
BETTER INDICATOR OF THE EXTENT TO WHICH A TREATMENT IS EFFECTIVE THAN
IS EITHER GAIN OR POSTTEST, for example, if the treatment yields
<g> > 0.3 for a mechanics course, where <g> is calculated from
pre/post testing with the "Force Concept Inventory" (Hestenes et al.
1992) then the course could be considered as in the
"interactive-engagement zone" (Hake 1998a).
. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . .
V. IS PRE/POST TESTING BECOMING MORE POPULAR IN DISCIPLINES OTHER THAN PHYSICS?

The answer is "YES." The relatively early pre/post test work of:
(a) Halloun & Hestenes (1985a,b) in developing the "Mechanics
Diagnostic" test (precursor to the much-used "Force Concept
Inventory" (FCI) [Hestenes et al. (1992)]);

(b) Hake (1987) in physics;

(b) Paden & Moyer (1969) in economics;

(c) Sundberg & Moncada (1994) in biology;

(d) Milford (1996) in chemistry; and

(e) Zeilik et al. (1997) in astronomy.

has been followed by efforts to use valid and consistently reliable
such as the Force Concept Inventory (FCI) [Hestenes et al. (1992) in
a pre/post mode so as to formatively measure the need for, and the
results of, reform methods in:

(1) physics [for reviews see Hake (2002a,b)];

(2) biology [e.g., Lawson (2001), Anderson et al. (2002), Klymkowsky
et al. ( 2003), Roy (2001, 2003), Sundberg (2002), and Wood (2003);

(3) chemistry [e.g., Bowen & Bunce (1997), ASU (2003), Gonzalez et al. (2003)];

(4) engineering [e.g., Evans et al. (2003)]; and

(5) computer science [e.g., Almstrum (2003)].

NOTE THAT MATHEMATICS IS ABSENT FROM THE ABOVE LISTING.