Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

What Does the FCI Tell Us? (was "Re: AP Physics Students")



Please excuse this cross-posting to discussion lists with archives at:

Phys-L <http://mailgate.nau.edu/archives/phys-l.html>,
PhysLrnR <http://listserv.boisestate.edu/archives/physlrnr.html>,
Physhare <http://lists.psu.edu/archives/physhare.html>.
AP Physics discussion list
<http://www.collegeboard.org/ap/listserv/tech.html>
(no easily searchable archives)

In his 4/25/01 Physhare post "Re: AP Physics Students," Donald Simsnek wrote:

"As to the fci, I personally am not sure just what it tells us. It
seems to show that most students have trouble with these concepts,
and that their courses haven't done a good job giving students
functional understanding. BUT IT DOESN'T CLEARLY AND SPECIFICALLY
SHOW HOW TO DO IT BETTER, EXCEPT MARGINALLY." (My CAPS and emphasis.)

Evidently Donald is unaware of studies(1-3) showing that:

A. The average normalized gain <g> affords a consistent analysis of
pre/post test data on conceptual understanding over diverse student
populations in high schools, colleges, and universities. The
correlation of the average normalized gain <g> with (%<pretest>) for
the 62 courses of refs. 1 & 2 is a very low +0.02. This constitutes
an experimental justification for the use of <g> as a comparative
measure of course effectiveness over diverse student populations with
widely varying average pretest scores. Here <g> is defined as

<g> = %<G> / %<G>max
= ( %<post > - %<pre >) /(100 - %<pre >),

where <post> and <pre> are the final (post) and initial (pre) class averages.

B. Fourteen Traditional (T) courses (2084 students) of the survey
yielded <<g>>14T = 0.23 + or - 0.04sd, where sd = standard deviation.
Considering the elemental nature of the MD/FCI (MD = Mechanics
Diagnostic - the precursor of the FCI) questions (many physics
teachers regard them as too easy to be used on examinations) and the
definition <g> = %<Gain>/ %<Gain>max, this suggests that traditional
(T) courses fail to convey much basic conceptual understanding of
Newtonian mechanics to the average student. Here the double angle
brackets "<<....>>" indicate an average of averages..

C. Forty-eight Interactive Engagement (IE) courses (4458 students) of
the survey yielded <<g>>48IE = 0.48 + or - 0.14sd. The <<g>>48IE is
over twice that of <<g>>14T , and is almost two sd's of <<g>>48IE
above that of the T courses, reminiscent of differences seen in
comparing instruction delivered to students in large groups with
one-on-one instruction (4). This suggests that IE courses can be much
more effective (NOT MARGINALLY MORE EFFECTIVE) than T courses in
enhancing conceptual understanding.

For survey classification and analysis purposes I OPERATIONALLY defined:

a. Interactive Engagement (IE) methods as those designed at least in
part to promote conceptual understanding through interactive
engagement of students in heads-on (always) and hands-on (usually)
activities which yield immediate feedback through discussion with
peers and/or instructors, all as judged by their literature
descriptions;

b. IE courses as those reported by instructors to make substantial
use of IE methods;

c. Traditional (T) courses as those reported by instructors to make
little or no use of IE methods, relying primarily on passive-student
lectures, recipe labs, and algorithmic-problem exams.

Richard Hake, Emeritus Professor of Physics, Indiana University
24245 Hatteras Street, Woodland Hills, CA 91367
<rrhake@earthlink.net>
<http://www.physics.indiana.edu/~hake>

REFERENCES
1. R.R. Hake, "Interactive-engagement vs traditional methods: A
six-thousand-student survey of mechanics test data for introductory
physics courses," Am. J. Phys. 66, 64-74 (1998); on the Web at
<http://www.physics.indiana.edu/~sdi/>.

2. R.R. Hake, "Interactive-engagement methods in introductory
mechanics courses," on the Web at
<http://www.physics.indiana.edu/~sdi/> and
submitted on 6/19/98 to the "Physics Education Research Supplement to
AJP"(PERS).

3. R.R. Hake, "Lessons from the Physics Education Reform Effort,"
submitted on 3/28/01 to "Conservation Ecology"
<http://www.consecol.org/Journal/>, a "peer-reviewed journal of
integrative science and fundamental policy research." On the web as
ref. 10 at <http://www.physics.indiana.edu/~hake>
[ConEc-Hake-O32601a.pdf, 3/26/01, 172K) (179 references, 98
hot-linked URL's). 164K). Gives references to articles by physics
education research groups whose FCI normalized gain results for
interactive-engagement and traditional courses are consistent with
those of (refs. 1 & 2). These groups are at: Univ. of Maryland
(Redish et al. 1997, Saul 1998, Redish & Steinberg 1999, Redish
1999); Univ. of Montana (Francis et al. 1998); Rennselaer and Tufts
(Cummings et al. 1999); North Carolina State Univ. (Beichner et al.
1999); and Hogskolan Dalarna -Sweden (Bernhard 1999). The consistency
of pre/post test results calls into serious question the common dour
appraisals of pre/post test designs [see, e.g., Cook &
Campbell(1979), Cronbach & Furby 1970)].

4. B.S. Bloom, "The 2 sigma problem: the search for methods of group
instruction as effective as one-to-on tutoring," Educational
Researcher 13(6): 4-16 (1984): "Using the standard deviation (sigma)
of the control (conventional) class, it was typically found that the
average student under tutoring was about two standard deviations
above the average of the control class .... The tutoring process
demonstrates that most of the students do have the potential to reach
this high level of learning. I believe an important task of research
and instruction is to seek ways of accomplishing this under more
practical and realistic conditions than the one-to-one tutoring,
which is too costly for most societies to bear on a large scale. This
is the '2 sigma' problem."