On the physics: I've addressed the issue of the crudeness of the charge
distributions shown in our book, in particular the emphasis on field, with
the charge distribution continually labeled as very approximate, and
there's nothing I can add to that. Here is a fuller citation of the recent
excellent article on finding the charge distribution rather accurately,
which was cited by Bob LaMontagne: A semiquantitative treatment of surface
charges in DC circuits” by Rainer Mueller, American Journal of Physics 80
(9) Sept. 2012, pages 782-788. The surface charge model of circuits has
indeed been discussed (and demonstrated experimentally) in AJP for decades,
and there were advanced textbooks that mentioned surface charge (including
for example the textbook by Jefimenko), but for some reason it was little
known by physicists at the time we started teaching it ourselves in the
early 1990s. Perhaps others will do a better job of teaching it in the
future than we have, but responding to John's strong criticisms just might
not be feasible at the intro level, in a course which he apparently has
never taught. What we have succeeded in doing is making students and
instructors at the intro level conscious of the role of surface charge in
producing the field that drives the current, and we think that's important.
The person who first realized the crucial role of surface charge was
Kirchhoff, and you can read more about the interesting history in our
article "A unified treatment of electrostatics and circuits" found in the
"Articles and talks" section of matterandinteractions.org; the history is a
summary of a portion of a Ph.D. thesis done in France.
On measuring "success": It is certainly true that if one teaches nonsense
and the students are able to regurgitate that nonsense accurately, that's a
problem. Interestingly, the studies that I cited in support of the claim
that our treatment of the surface charge model is "successful" do not
suffer from that problem. The paper by Thatcher, Ganiel, and Boys posed
novel problems to students in interviews with students from a traditional
course and a course using the 1995 edition of our E&M book. These problems
were novel to both groups, and necessarily did not deal directly with the
surface charge model, yet the investigators found a sizable difference in
favor of the non-traditional group, both in a US university and an Israeli
high school.
The large-scale BEMA study used an assessment tool, Brief Electricity and
Magnetism Assessment, which Ruth Chabay and I created in the mid-1990s in
response to insistence from our Carnegie Mellon physics colleagues that we
prove that our approach to E&M was an improvement. We circulated a draft to
all the CMU faculty who had taught either intro or intermediate E&M, and
they gave us feedback designed to ensure that the test would be fair and
appropriate for students in either flavor of intro E&M; the final form was
agreed by all to be a fair, "lowest common denominator" test. That means
that surface charge among much else does not appear in BEMA. BEMA is
similar to the CSEM, which was developed at about the same time. Again, as
in the article by Thatcher et. al., there was a sizable difference in favor
of the non-traditional group.
I've presented real experimental evidence in support of the claim that the
M&I E&M curriculum is a significant improvement over traditional
instruction. Note too due to the lowest common denominator nature of these
comparisons, much that we see as important and valuable in the M&I approach
had to be omitted from the tests, not only surface charge. On the other
hand, it's possible to claim that at least in the BEMA measurement the M&I
students did well in spite of the stupid, wrong physics taught to them
about surface charge, that the improvement was the result of other aspects
of the curriculum that weren't so terribly wrong. In the Thatcher et. al.
study what seems to stand out is that our approach, in the context of
surface charge, talked about transients leading to a steady state, which
helped students think through novel situations of other kinds, so I would
claim that surface charge played a role, and I strongly suspect but cannot
prove that it played a role in the BEMA comparison as well.
Which brings me to making some comments on evidence for "success" in
physics education. As I've said, the creation of BEMA was the result of
insistence from colleagues at CMU that we compare performance in E&M
between traditional and our approaches. It took a LOT of work to create the
test, a LOT of work to administer the test (which involved among other
things a longitudinal study, which is difficult logistically), and a LOT of
work to cross-check with student grades in mechanics and calculus, and with
GPA. After all that work, the immediate reaction of some of our colleagues
to the final results was entirely dismissive, with instant rationalizations
showing why our approach could not have produced these results. That
experience made me highly sensitive to the great difficulty of making
educational measurements that will be taken seriously. I note that in this
Phys-L discussion no one has been unduly impressed with the evidence I've
presented, despite its relevance.
And actually, I sympathize with those dismissive CMU colleagues. A test
produces one or a few numbers, whereas personal judgements are highly
multidimensional and based on years of teaching experience, so even if the
test shows sizable results it's not inappropriate, based on experience, to
doubt those results because in educational measurements there is a very
large number of factors, many of them not even identified let alone
estimated or controlled for. For example, we had been at NCSU for awhile
before it was pointed out to us that, "Oh, and you should know that classes
taught at different times of day show significant differences because
students with high grades get to register early, and they pick the most
desirable times of the day."
Here's a stark counterexample. I remember reading the 1964 PRL paper by
Fitch and Cronin that showed CP violation in kaon decays. The effect was
very small, I think about a third of a percent asymmetry in the
measurements, so my first reaction was that there was no reason to believe
the result. But by the time I reached the end of the short paper I was
utterly convinced that the result was correct, and in fact later
measurements fell inside their error bars. The huge difference between this
and educational measurements is that they were able to identify and measure
or estimate all of the possible sources of error, both statistical and
systematic. This was possible because there is a small number of these
sources, and they are all known. In educational measurements there is a
huge number of sources of bias and error, many of them unknown.
I'll also comment that tests such as the FCI and BEMA are best used to
measure whether a curriculum is meeting its own goals, not for measuring
whether one content is better than another content. First, on
multidimensional grounds, you choose a curriculum (in the M&I case,
choosing between a 19th century perspective and a 20th century perspective)
and establish your goals. Then you try to teach it as well as you can,
exploiting all the PER research on pedagogy, but being attuned to the
likelihood that with a major change in content new pedagogy may be needed.
Then you use an assessment instrument tailored to your content to see how
successful you are in meeting your own goals, without reference to other
curricula.
In the case of BEMA, it was possible to create a lowest common denominator
test that can be fruitfully used with many curricula (for one thing, E&M is
inherently closer to the 20th century than is mechanics, though M&I does a
lot in E&M with the atomic nature of matter, and with polarization at the
micro level). In the case of mechanics, it is significantly harder to
create a lowest common denominator test to compare the traditional
curriculum with a curriculum where the ball-and-spring model of solids
plays a central role, where iterative approaches to motion are important,
where mechanics and thermal physics are entwined, where relativistic energy
plays a key role, and where the energetics of deformable systems is treated
correctly. Ruth's Ph.D. student Lin Ding created an assessment instrument
to measure whether the M&I goals related to energy are being met, and we
estimate that fully half of the questions would be unfair to students in a
traditional course because of being unfamiliar. The test can however be
used to measure whether the implementation of the M&I curriculum is
working, and whether it is working better this year than last year.