Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

Re: [Phys-l] What's the point of teaching to the test when it's questions are incorrect?



Same kind of thing happened in NJ. We were all told that all juniors would have to pass a science component on the required-to-graduate HSPA. We were given limited sample items. One that sticks out in my memory:

An airplane flies east. The wind blows north. Which of the following vectors represents the plane's velocity?

The answer choices included one aimed east, one aimed northeast and one aimed southeast. (I don't remember the other answer choice.) There was nothing in the problem to distinguish between whether they were giving or asking for the velocity relative to the air or to the ground. Some students chose the east vector. It says in the problem that the plane flew east. Can't argue with that. Others thought that it meant AIMS east, blown north, actually goes NE. Others thought, goes east despite northward wind, must have aimed SE...

Maybe the poor quality of the tests is the real reason that they limit our access to them. In any case, NJ has now abandoned that plan, after causing many districts to completely revise curriculum solely to meet the requirements of that test. I really wonder why we need a state department of ed...

-----Original Message-----
From: phys-l-bounces@carnot.physics.buffalo.edu [mailto:phys-l-bounces@carnot.physics.buffalo.edu] On Behalf Of John Clement
Sent: Friday, March 20, 2009 11:34 AM
To: 'Forum for Physics Educators'
Subject: [Phys-l] What's the point of teaching to the test when it's questions are incorrect?

Several years ago I brought up 3 questions from the TAKS test routinely
administered to children in TX. All 3 were extremely ambiguous and really
had no single answer.

Since then I have seen 10th graders tested with a district wide test with at
least one question that had no correct answer, and another where even the
teachers did not know what was being tested. There were about 30 questions
on this test. These were taken out of a TX test bank that was purchased as
preparation for the TAKS.

The one that was incorrect has a common misconception that is repeated in
the IPC book, and often appears even in college biology books.

The question went as follows (shortened):

Which of the following best explains where this energy is stored in the
food?
a. It is stored in the bonds that hold atoms and molecules together.
b. It is stored in vitamins and minerals.
c. It is stored in water molecules.
d. It is stored in the oxygen or carbon dioxide molecules that living
things breathe.

According to the book a. is the correct answer, but if you put food in a
sealed container you can not get energy by "breaking" the bonds. One could
just as easily ascribe the energy to the oxygen. You can not release energy
unless you have oxygen, so it is just as likely a candidate.

The only answer which makes sense is the energy is stored in the system or
combination of food and oxygen, not in the individual components. Potential
energy is often stored in a system, or one could think of it as being stored
in a field, such as the Earth-ball system. Promoting this misconception
makes students think you get energy by "breaking" bonds. Actually, the word
breaking is a very bad framing of the concept. A better framing is to say
the "stretching" of bonds, so that it becomes analogous to a rubber band.

Another one with ambiguous or wrong answers is:

The driver of a taxi stops to pick up a large family with many pieces of
heavy luggage. Based on Newton's second law of motion, the driver should
expect that the taxis will
a. Take longer to travel the same distance.
b. develop more stopping friction
c. require less energy to keep moving
d. take longer to both accelerate and stop

a,b, and d all have some correctness, but d directly relates to NTN2.
However the more time to stop may not be true in a locked wheel stop because
the friction increases proportional to the mass. The big problem with this
question is that it requires a very high level of thinking and less than 24%
of HS seniors are formal operational. It also expects that students have
connected mass and force with acceleration and not velocity, but
conventional physics courses are notably ineffective in doing this.

Here is one where it is often unclear what is being tested:

Because buoyant forces act in the opposite direction of gravity, objects
completely submerged in water ---
a. appear to weight more than they do in air
b. have a net force greater than their weight.
c. have a net force smaller than their weight
d. are unaffected by gravity when immersed.

Most teachers think it is about buoyant force, but it is really about net
force. a. would be automatically rejected by everyone. B. is possible if
the object is accelerating at a high rate, so the word at rest needs to be
in the statement. The completely submerged is a red herring. So when
reviewing he question, teachers will concentrate on the wrong part of the
question.

There was a question with a before and after picture unlabeled, but if you
thought the before picture was an after picture the correct answer changes.

Another one had a graph with a number of things going on, temperature vs
time for 2 substances from below freezing to after varporization. It had
answers such as substance A freezes before B or A boils at a higher
temperature than B... This type of question is calling on a very high level
cognitive skill, 2 variable reasoning. This type of reasoning is not very
high in HS seniors. So why not test this reasoning separately from
curriculum material? Separate out the curriculum from the Piagetian
reasoning skills. If you have only 30% capable of 2 variable reasoning, you
know the failure rate will be high, and teaching the question will not be
helpful unless identically the same question is used on the TAKS.

Then there was the typical refraction of light question which shows a ray
passing from air to water, ray going straight, backwards more toward normal,
forward away from the normal, forward toward the normal. The answer is cut
and dried, but if you look in the book it has a very confusing picture that
might have mislead students. It shows a pencil in a glass of water. The
view from the side shows the pencil being bent away from the normal in the
water. Students are asked to explain the picture, which is a very accurate
drawing probably from a photo. The problem with this illustration is the
way the pencil is bent, so if they remember this picture, they will get the
wrong answer. The bend in the pencil is due to the horizontal magnification
provided by the cylindrical glass of water. There is no way students could
analyze the picture because it is way tooo complicated. Even very bright
college students would be challenged by it.

The students are in a low performing school and generally come from low SES
families. The school probably has a 60% drop out rate. Students have
looked at these questions after the test, so none of this is confidential
information. The test bank contains questions supposedly similar or
identical in content to the standardized TX science test.

Oh, and if the course is supposed to raise student thinking about 2
variables, why not put that in the state requirements rather than sneaking
it into the test. But of course teaching physics or IPC does not do this as
shown by the Chinese-American comparison study. Teachers and test question
writers need to know and understand the Piagetian taxonomy of thinking. If
you do understand it, then you will understand what the question is really
testing, and not be fooled by surface easiness of it. On the surface it may
look easy, but when the types of reasoning are considered, it may be
cognitively extremely difficult. Every single science teacher needs
training in this type of thing just as much as they need to understand
content. This includes post-secondary teachers. A simple example is that
if students do not have compensation reasoning, they have extreme difficulty
in connecting increased density with sinking.

The standard TX IPC book has a huge number of errors, but then students
don't read the book, so I guess the errors don't matter. But why pay huge
sums for erroneous books which are never read? And why base questions on
the erroneous parts of the book?

John M. Clement
Houston, TX





_______________________________________________
Forum for Physics Educators
Phys-l@carnot.physics.buffalo.edu
https://carnot.physics.buffalo.edu/mailman/listinfo/phys-l