Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

Re: [Phys-l] [ncnaapt] New Physics CST Questions Released!


I would say there is more than just the observations that have been
correctly made I posted:
-----------------------------
The comments about number 4 are certainly correct. As to the amount of air
resistance, since the first object is moving faster at the end it could have
the same air resistance as the smaller object. Also one should never use
the word slower or faster in describing acceleration. It just produces
added confusion between velocity and acceleration.

I would say that the first 5 questions are pretty bad.

Number 5 is also a bad question. The answer could certainly be a conclusion
based on experience. Indeed one does not have to know theory to make the
conclusion, as experience with circuits is all that is necessary. The line
between hypothesis and conclusion can sometimes be fine. A hypothesis can
be a grand name for a guess. As to the obviously silly answer involving
theory, it may reinforce the usual misconception about theories. Usually
when questions are asked about these types of things the questions are very
bad. Even many good publications and authors get it wrong.

Number 6 is also poor because neglecting a known effect is not experimental
error, but experimenter error.

Number 10 is just a vocabulary question masquerading as a Newton's third or
second law question. It purports to test the second law, but really does
not.
----------------------------------
Number 15 purports to test student's understanding of the universal law of
gravity and its effect on objects on the Earth PH1.E. It is actually a
simple question which can be answered by observation with no understanding
of any theory.

The table at the end of the questions details what standards are being
tested. As is typical of many high stakes questions they do not test what
the writers claim is being tested. This has proven true for every question
I looked up in the table. (I admit to only looking up a couple, but 2/2 is
still poor) So on the basis of these types of poor questions students will
be denied graduation and teachers will be punished.

Perhaps the people who are responsible for these types of things should be
required to pass them themselves, and to identify which questions have no
real answer. Can the students who were punished by poor questions sue to
get compensation? And of course, can they even find out if the questions
were poor when they answered them? The only way to make the testing fair is
to release all questions after the tests have been given so that they can be
carefully scrutinized and publicly debated. But this will mean that the
quality of the questions are difficult to improve. The real problem is that
the good quality conceptual tests such as the FCI require more than 2
questions on each concept, and generally take 5 years to develop. High
stakes tests must be generated quickly and can not have enough questions on
each concept.

I think that this whole enterprise of high stakes testing is fatally flawed,
and can not be fixed by just making improvements in the questions. Since
these tests can often be passed by just memorizing things, they do not
really test what they are supposed to test. The standards are also too
broad to allow the students time to gain deep understanding of any subject.

Teachers and schools have some accountability because the parents can come
in and look at the tests. Students are always allowed to examine their
tests. By contrast the state tests often have no equivalent accountability.
I know of a teacher who had a parent conference where the parent brought
along a graduate student in the subject. The parent had not told the
teacher about the grad student. The grad student found no errors in the
testing process. I don't know if I would be as accommodating. My reaction
would be to tell the parent that I had not been told about the grad student,
so they would not be allowed at this conference. And if the average grad
student questioned my tests, or what I do in class, I am certain I could ask
questions that they could not answer about physics and pedagogy. But I know
parents would be reluctant to do this in light of my degrees. The teacher
who had this happen did not have advanced degrees, but was highly
experienced, and had lived in the countries where his subject, Spanish, was
spoken.

John M. Clement
Houston, TX

I think it is great that you are asking for critical feedback on the new
questions on the California Standards Test some of which are available
at
http://phyzblog.blogspot.com/2007/02/cst-questions-released.html .
It seems likely that such efforts will result in a higher quality test
in the future. Kudos! Here's my opinion on some of the newly released
questions:

1. The answer is a matter of opinion. Folks who are well-versed in
interpreting graphs, would prefer to view the angle vs. range graph but
folks who have trouble with graphs would probably find it easier to
interpret the table of values that would duplicate the given table of
values if it were entered into a spreadsheet or database that displayed
the table of values. I think that a question whose answer is a matter
of opinion should be left off of a standards test. I think it would be
better to display a graph of the data, or a best fit curve to the data
and ask the test taker to interpret the graph. You might ask whether a
launch at an angle five degrees greater than the launch angle that
yields maximum range would come closer to yielding a maximum range than
one that is five degrees less than the launch angle that yields maximum
range.

2. An accelerometer is a motion sensor. I don't know if it is still the
case (I suspect that it is) but back in the 70's I think that the
software on an ICBM was programmed to integrate accelerometer output to
determine the velocity of the payload at the current instant in time and
to integrate again to get the position at the current instant in time,
given initial velocity and position information. Values of the position
at successive instants could be used to generate a graph of position vs.
time. A sonic ranger available in some high school laboratories can
also be used to produce such a graph. A force probe with software
programmed to apply Newton's 2nd Law to an object of known mass could be
used to generate such a graph. The force probe or the accelerometer
would undoubtedly include a potential difference probe. Any one of the
four answers can be considered to be correct. Which is the best answer
is probably a matter of opinion. The question highlights what I
consider to be a problem with the California Standards test that is
shared by many tests. Ruling out many distracters sometimes requires a
lot of words. Test question writers seem to follow an unwritten rule
that the questions must be short. I think that critical reading skills
are an important part of physics improved critical reading skills should
be considered an important outcome of any physics course or program.
Being able to read a long question with words that unambiguously rule
out distracters is a skill that should be tested in a physics test.
Embrace the wordiness required to pose a good question. If this means
asking half as many questions, so be it.

10. This question and its answer reinforce the idea that weight is a
property of an object. How hard does the moon pull on the book when it
is near the surface of the moon? How hard does the earth pull on the
book when it is near the surface of the earth? A better answer than
weight, to the question posed on the test, would be the gravitational
force exerted on the book by the earth. Aside from being part of an
assessment tool, a standards test is a teaching tool. Reject any
question that can be expected to teach the student something that is
wrong.

Jeff Schnick