Is the FCI valid?

[Ben Crowell <bcrowell@SPRINTMAIL.COM>]

Jack Uretsky wrote:
> The fact that Hestenes thought that he was defining a minimum
acceptable
> level does not make it so.  The FCI, as I understand it, was never
> validated by any acceptable procedure, so the notion that a 70% FCI
> score is 70% of what's acceptable is unsupportable.

There are various definitions of validity used in the standardized
testing business. "Face" validity means that a knowledgeable person
believes from looking at it that, on the face of it, the test measures
what
it's meant to test. Not the most solid definition ever, but face
validity is
virtually the only kind used by teachers to construct tests and
assign grades in school.

Another kind of validity is verified by seeing if the test
correlates strongly with other measures of success. E.g. the
SAT is claimed to be valid because it correlates
with future success in college.

Since FCI scores don't correlate strongly, for example, with
students' ability to do Halliday-style problems, one could
say the FCI is demonstrably invalid.

I would however turn the argument around. I think the FCI has
better face validity than Halliday-style problems. We all
know that students try to plug and chug their way through
quantitative problems without understanding what the heck
they're doing. But I don't think anyone could argue that a
student understands physics if he/she answers wrong on an
FCI question about whether heavy objects fall faster than
light ones.

I would claim that the weak correlation between the FCI and
traditional methods of evaluation proves that traditional
methods are invalid.

Maybe we could get some comments from Phys-L-ers like Richard
Hake who have used the FCI extensively in their research
and therefore have had plenty of time to think about
these issues.

 Ben Crowell
 Fullerton College
 Online physics textbooks: www.lightandmatter.com