Gary Agin mentions that the ETS report to the department for the
Physics Major Field Test includes "assessment indicators" in five
subject areas of physics.
Before anyone gets too excited about this, I want to caution you that
ETS only provides these assessment indicators if your institution has
at least five students take the test. Bluffton College has never had
five physics graduates in one year... the highest I can remember is
three. From the statistics I've seen in the American Institute of
Physics survey of physics graduates, Bluffton is not unique... there
are many colleges and universities with fewer than five physics
graduates per year.
Thus, if you are thinking of using the ETS Major Field Test in physics,
but you do not have at least five graduates per year, do not plan on
getting any sub-field assessment indicators from ETS as the situation
exists today.
Gary also mentioned that the students receive subscores on
"introductory physics" and "modern physics." This is true, but I don't
find this breakdown very helpful. In this respect note that
"assessment indicators" and "subscores" are not the same thing.
Subscores are "earned" by each student and are always reported
(student by student, even if only one student). Assessment indicators
are reported as averages for your whole group and require five
students. Since the subfields of physics are used as assessment
indicators rather than subscores, (1) you don't get them if you have
fewer than five students, (2) if you do get them you don't know which
individual students were high or low in a particular area of physics.
Thus, I think ETS or someone needs to improve the test reporting to
give more subscores.
Since I chair a department that also contains chemistry and biology, I
can tell you the ETS situation is a little better in chemistry and
biology. For example, chemistry students get subscores for Inorganic
Chemistry, Organic Chemistry, Analytical Chemistry, and Physical Che
mistry. It is not uncommon for chemistry majors who are interested in
pre-medicine to do much better on organic chemistry than inorganic
chemistry. Likewise, it is not unusual for biology majors who are
interested in medical areas to do much better in anatomy and physiology
than in ecology. So when we see that type of thing going on, we know
it is not necessarily a reflection on our program as much as a
reflection on that particular student's interests. (Unless you want to
say we failed to "excite" that student in an area where s/he had little
interest.) Anyway, we believe knowing which individual students do
well or poorly in a specific subfield is valuable, and right now that
information is not available from ETS for the physics test.
One more bad thing... We find the ETS results (and perhaps any exit
exam) are most useful after we have about five or six years of data.
Even then we might have results for only 9-10 physics graduates. But
another bad thing happens... ETS changes the exams about every four or
five years, then cautions that we cannot compare scores across the
test-change boundary.
Clearly there are many of us with small numbers of physics graduates,
and we are being forced into some type of nationally-based assessment,
and we are still looking for the appropriate tool. Anybody out there
have strong influence on APS/AIP? Could they help us?
Michael D. Edmiston, Ph.D. Phone/voice-mail: 419-358-3270
Professor of Chemistry & Physics FAX: 419-358-3323
Chairman, Science Department E-Mail edmiston@bluffton.edu
Bluffton College
280 West College Avenue
Bluffton, OH 45817