[Phys-L] Re: Testing assessments (was A Third law question)

[rtarara at SAINTMARYS.EDU (Rick Tarara)]

----- Original Message -----
From: "John Clement" <clement@HAL-PC.ORG>
 The anecdotal evidence you cite is certainly favorable, but such

> evidence can really only be supporting evidence.  I have no doubt that
> your
> program works better, but does it give 70% normalized gain?  At present
> the
> maximum gain has now been pushed to close to the 90% level by a few
> Modelers.


I really have a lot of trouble placing much weight on these normalized gains
from the FCI and FMCE.  To me the main problem is that these have been
virtually the same tests for more than a dozen years now.  There is no way
that instructors who use these tests year after year to evaluate their own
performance or the performance of their pedagogy of choice have not (at
least subconsciously) learned to teach to many of the specific questions of
the test.  As someone who uses the FCI in a more casual way, I know that I
tend to focus on points I know are key to the test.  So while 'teaching to
the test' is not necessarily bad, it does weaken (perhaps severely) the
value of the kinds of scores shown by John.  I have no doubt that curricula
that have been designed to get better scores on the FCI will do so and those
curricula (or techniques) not so designed will do more poorly.  My students
continually produce 60-70% normalized gains without the use of any
particular 'program'--but again I fully admit to pointing some of the
discussions towards the questions and content of the test.  What I find
somewhat sad, is that after at least 15 years of serious PER work, no newer
instruments, or better yet, no yearly updated instrument, has appeared that
could be widely used in the kinds of assessments for which the FCI and FMCE
continue to be used.  To be more convincing, we need assessment tools that
are not firmly emblazoned in the minds and memories of the instructors.  I'm
also waiting to get some students into my courses who have had 'reformed'
courses in HS and exhibit some real knowledge--which is partly why I give my
calc-level class the FCI on the first day of class.  I'm still waiting  ;-(

Rick