Re: Is the FCI valid?

[Jack Uretsky <jlu@HEP.ANL.GOV>]

Hi Ben-
        I think that we are not communicating.  I am addressing the
so-called "70% barrier".  Dick Hake treats this barrier as though
it signifies that students are only achieving 70% (roughly) of what
they would achieve as a result of an ideal course.
        You have focussed on my use of the word "validate".  I use
the word here, more or less, in your second sense - how does an FCI
grade correlate with "successful" teaching.  Hake's data provides
the only correlation  that we have available.  We may not, therefore,
draw any conclusion about teaching from the fact that scores top out
at 70%.
        I think that your descriptions of the meanings ascribed to
the word "validate" are in accord with my understanding.
                                Regards,
                                        Jack

Adam was by constitution and proclivity a scientist; I was the same, and
we loved to call ourselves by that great name...Our first memorable
scientific discovery was the law that water and like fluids run downhill,
not up.
                          Mark Twain, <Extract from Eve's Autobiography>

On Sat, 8 Jan 2000, Ben Crowell wrote:

> Jack Uretsky wrote:
> > The fact that Hestenes thought that he was defining a minimum
> acceptable
> > level does not make it so.  The FCI, as I understand it, was never
> > validated by any acceptable procedure, so the notion that a 70% FCI
> > score is 70% of what's acceptable is unsupportable.
>
> There are various definitions of validity used in the standardized
> testing business. "Face" validity means that a knowledgeable person
> believes from looking at it that, on the face of it, the test measures
> what
> it's meant to test. Not the most solid definition ever, but face
> validity is
> virtually the only kind used by teachers to construct tests and
> assign grades in school.
>
> Another kind of validity is verified by seeing if the test
> correlates strongly with other measures of success. E.g. the
> SAT is claimed to be valid because it correlates
> with future success in college.
>
> Since FCI scores don't correlate strongly, for example, with
> students' ability to do Halliday-style problems, one could
> say the FCI is demonstrably invalid.
>
> I would however turn the argument around. I think the FCI has
> better face validity than Halliday-style problems. We all
> know that students try to plug and chug their way through
> quantitative problems without understanding what the heck
> they're doing. But I don't think anyone could argue that a
> student understands physics if he/she answers wrong on an
> FCI question about whether heavy objects fall faster than
> light ones.
>
> I would claim that the weak correlation between the FCI and
> traditional methods of evaluation proves that traditional
> methods are invalid.
>
> Maybe we could get some comments from Phys-L-ers like Richard
> Hake who have used the FCI extensively in their research
> and therefore have had plenty of time to think about
> these issues.
>
>  Ben Crowell
>  Fullerton College
>  Online physics textbooks: www.lightandmatter.com