Executive summary: In kindergarten, they told me: "Check your work".
In graduate school, they used more words to say the same idea: "The
most important step toward preventing mistakes is to realize that
mistakes are possible."
This is central to any notion of critical thinking. It is important
to critically analyze your own work. "Publish in haste; repent at
leisure." I mention this because in the education literature, there
is a lot of talk about critical thinking ... yet in an astonishing
number of cases, it looks like nobody bothered to check the work.
IMHO, people who have not demonstrated any appreciable skill at
critical thinking ought not be telling other folks how to do it.
By all accounts, it was developed by Gregg Swackhammer, who is one of
the co-authors of the FCI. However, it appears that the ECI is not
nearly so well thought out.
If this were some weekly quiz written for the physics class at South Dogpatch
High School, I wouldn't bother to critique it ... but when a test is designed
as part of a multi-man-year study of educational methods, maybe somebody
ought to apply the "check your work" rule.
==) For one thing, I am mystified by ECI question 10. AFAICT, answer (c)
and answer (d) both seem absolutely correct. Help me out here: In physics
terms, does anybody know of any rational basis for preferring one answer
over the other? Also, in pedagogical terms, if a student chooses (c) over
(d) or vice versa, what does that tell you?
==) While we are at it, let's take a look at ECI question 2. To get started,
let's assume, hypothetically, that the drawing is to scale. Scaling laws are
a super-important part of physics, and have been since Day One. The salient
feature here is the high curvature of the bump at the middle. Given that
the radius of curvature of the bump is smaller than the drop from level A to
level C, simple physics and/or experience tells us that either the particle
flies off the bump and becomes airborne, /or/ that it has lost most of its
energy before reaching that point. The relationship between energy, curvature,
and acceleration is well known to any kid who ever paid attention while playing
with Hot Wheels, or while playing with a playground swing-set, et cetera. Or
you could do the calculation.
Meanwhile, we are told that the ball "rolls smoothly along the surface" so we
must conclude that the system is highly dissipative. This is consistent with
other questions on the test, where friction plays a dominant role.
On this basis, we conclude that answers (d) and (e) must be excluded. There
is AFAICT no clear basis for choosing amongst the remaining answers.
At this point it becomes a telepathy exam. The goal, evidently, is for the
student to read the teacher's mind. Under heavy time pressure, the student
must figure out what mistake the examiner had in mind when designing the test,
so that the student can choose the same mistake.
-- Maybe we are supposed to neglect friction ... but no, that's not possible,
because returning all the way to level A aka B is not one of the options.
-- Since this is an energy test, maybe we are supposed to forget everything
we know about force and acceleration.
-- Maybe the drawing is not to scale. OTOH if it's not to scale, we have to
guess the diameter of the steel ball. If the ball is small and the other
distances are large, then clearly (a) is the correct answer ... whereas if
the ball is large, (b) and (c) are viable contenders.
I do the telepathy calculation as follows: In how many ways could the
teacher screw up so as to make (d) the "approved" answer? In how many
ways could the teacher screw up so as to make (c) the "approved" answer?
Et cetera.
And people wonder why the "gain" and the retention are so small. Sheesh.
==) In ECI question 3, the telepathy exam continues.
Much depends on how you interpret the word "surroundings". When normal
people use the word, it refers to the immediate surroundings, as in "be
alert, be aware of your surroundings". It does not mean "the universe
as a whole". Using this interpretation, answer (c) appears to be the
best, in the sense that it represents a possible outcome, even if this
outcome is not uniquely guaranteed. Answer (e) is clearly not a smart
choice, for the following reason: On tests of kind, the "catchall" option
is almost never the approved response, because a wise-guy can always find
/some/ fault with the other choices, so hyper-technically the catchall
would always be the correct answer, but nobody likes a wise-guy.
One of the rules of doing science, and indeed of common sense, requires
us to consider all the possibilities. Maybe the teacher had in mind an
abnormally broad definition of "surroundings". In this case the question
is trivial. One of the rules of telepathy exams is that the teacher
usually does not like trivial answers. Along the same lines, if the
teacher wanted to include the E=constant case in the consideration, it
would have been as easy as π to include a graph of the constant function.
So there is no good reason to prefer this definition.
Note: Using 20/20 hindsight plus a PhD in physics, I am pretty sure
that the "approved" answer to this question is (e) ... but my point
remains: I cannot imagine how a student is supposed to reach that
answer.
==) In ECI question 8, it mentions "soccer" and an "ankle" injury. Help
me out here, folks. In what way do these details make it a better
question? Are they supposed to be "motivation", or are they meant to
provide helpful information, or perhaps a distraction?
I mention this because in the real world, questions are often underspecified,
and you need to look at the context in order to garner enough information
to solve the problem. This is the motivation for asking "rich context"
questions.
Alas, the details we see here are not rich context, they are bankrupt
context. It seems to me that as teachers, we ought to recognize the
distinction between real-world context and shaggy-dog stories. http://en.wikipedia.org/wiki/Shaggy_dog_story
While we are looking at question 8: I know what the "approved" answer
is, but like so many of the other questions, it involves wrong physics.
You could argue that the "approved" answer is /diametrically/ wrong,
if you read it carefully and literally.
Again it is a telepathy exam: The goal is to choose the answer that
is wrong in a slightly subtle way, by excluding the answers that are
so obviously wrong that the teacher would realize they are wrong.