Re: [Phys-L] other problems with what is (or isn't) on the test

[Hugh Haskell <haskellh@verizon.net>]

At 7:01 PM -0400 6/16/12, <cpreske@twcny.rr.com> wrote, in response 
to John Denker's comments:
> I would like to thank the thirty teachers from across NYS who worked 
> on the June 2012 Physics Regents over the last four years, those of 
> you who wrote questions, those who edited them, those who 
> constructed, reviewed and corrected the pretests and field tests and 
> those who helped  Bob and I finish all the work necessary  to 
> prepare the operational exam a fair and honest assessment for the 
> students of NYS. 
>
> To all of those who have critisized their work, please forward your 
> name, address and phone number so that whenever we have the ability 
> to bring new teachers in to work on future exams I will be able to 
> provide the Test Owner ( yes the one and only person at NYSED who 
> works on the exam that is a State employee)  a list of people who 
> obviously have the expertise that we could use in the future.
>
> To all the other teachers who appreciate the work of your fellow 
> teachers who have given of their time and energy for your students, 
> I thank you on their behalf. They do not do this work for the money 
> but for the benefit of all our students.
>
> If I have offended anyone, I am sorry but these people who work so 
> hard do not deserve the critisism that is heap on them for their 
> efforts. As I have said before there is no Albany there is onlly a 
> group of teachers who work tirelessly for you and your students and 
> their efforts should be commended.
I agree that the teachers who make up these tests are doing the best 
they can, and don't deserve the harsh criticism directed at them by 
John Denker. On the other hand I also agree with JD's conclusions 
about the tests. The problem is that "the best the test authors can 
do" isn't good enough.

As I learned over the years, making up good test questions is very 
difficult. I have created at least a thousand test and homework 
problems, most of which would not have passed Denker's mustert. On 
the other hand, the few that might have made that cut were seldom 
actually used, because I knew that the students couldn't answer them. 
It really does no good to use questions that the students will not be 
able to answer. It isn't good for their morale, not is it good for 
mine (they did poorly enough on the lower quality questions that I 
did use). All this at an elite school that presumably enrolled the 
best students in the state, and put them in a strong learning 
environment. The problem was that the students arrived in my 
classroom after ten or eleven years of seeing nothing but the 
regurgitative questions than Denker  so decries. I tried to make some 
inroads in their thinking processes, but as pretty much a voice in 
the wilderness it was a nearly hopeless task, and one that I probably 
wasn't well-qualified to undertake anyway. I was fortunate in that I 
was teaching in a school where we had seven physics teachers, half or 
more of whom held physics PhDs, and many of our students came from 
families with at lest one and often two parents who were 
academically-oriented professionals, so we were able to discuss many 
of these issues amongst ourselves, and over the years we did come up 
with some ideas to improve the thinking processes, but as one can 
imagine, the push-back from students was fierce, and mostly, they 
(and their parents), in general had the ear of the administration, so 
we were mostly tilting windmills in our efforts.

But, in general, our school systems are not oriented toward teaching 
thinking skills, and those teachers who try are often strongly 
discouraged from continuing. As are several others on this list, I 
have also been a member of the AP physics listserve for more than 15 
years. I have been impressed with the dedication of the teachers who 
participate on that list. They persevere in the face of daunting 
obstacles, not just from recalcitrant students but also from ignorant 
administrations and even more ignorant legislative bodies who think 
they know more about teaching than the teachers do.

That said, I also have some harsh words for the schools that 
purported to train them as physics teachers. The ignorance of some of 
the simplest principles of physics among many of the teachers on the 
list is astonishing, considering that they are supposed to be 
teaching the top students in their schools. Even more concerning to 
me is the apparent lack of the ability to think through a problem on 
their own. It does seem to me that any teacher who undertakes to 
teach an AP or IB or other elite course needs to be able to solve at 
least the overwhelming majority of the problems they either give 
themselves as homework or test (usually from a text or other 
pre-designed source), or see their students try to solve on 
externally designed tests (e.g., AP or Regents). Far too many of the 
queries that arise on the AP-Physics listserve are from teachers who 
are trying to solve some particular problems or improve their 
understanding of some basic principle. I admire their interest in 
improving their knowledge. They show their dedication and for that I 
applaud them. But most of the questions that are asked are at a level 
that the teachers should be at before they start teaching at the AP 
or IB level, and for that, we must fault the schools that prepared 
them.

Just as appalling as the nature of the questions raised, is the 
nature of the responses offered by many of the responding teachers. 
Far too often either a very specific "how to solve it" answer is 
given, or a particular formula is quoted as "being the solution."

Another area of concern to me is the number of threads that get 
started of some particular teaching approach, and the opinions 
expressed on the topic, which are often couched in algorithmic terms, 
which seem to be designed to give the students a series of "cookbook" 
solutions to problems that they can memorize and thus be able to 
solve whatever is thrown at them on the AP, IB or Regents exam they 
are facing. For example, a periodic debate arises on how to deal with 
centrifugal forces, and the opinion held are often quite vociferously 
promoted, especially the apparently widely taught myth that "there is 
no such thing as a centrifugal force." Many other questions end up 
being answered in terms of "canned" solutions, that, I guess, the 
teachers are supposed to learn and pass on to their students. This 
doesn't strike me as productive educational strategy. If the teachers 
are themselves required to figure out solutions for themselves, they 
will be better prepared to expect students to at least work toward 
that as a goal.

Having never worked at grading AP tests or any of the others, I don't 
know how much leeway is allowed students who stray from the 
"standard" solution to any given question, but apparently there is 
enough concern among the teachers that many of the threads generated 
on the AP-Physics list deal with what off-beat solutions might be 
expected, or do the teachers need to be sure that all students 
maintain an "approved" approach to problems that don't deviate from 
the commonly accepted approach.

In short, as much as I agree with the thrust of JD's comments (if not 
with the confrontational approach he takes), I think that we will not 
get better tests until we get better prepared students--who have been 
set on the road to thinking for themselves--and better-prepared 
teachers--how are ready to move students further along the road to 
independent thinking. Until that day, we are probably going to be 
stuck with students who have never been required to think for 
themselves and teachers who don't know how to teach students to think 
for themselves, and as a result we will have to live with the type of 
tests JD has well and truly excoriated.

Let me reiterate that I do not fault teachers for their lack of 
knowledge. Too many of them labor in isolation and really have no 
resources available to them to bring themselves up to the level they 
need to be. Their opportunities for professional development are 
often limited, and nearly always must be undertaken at their own 
expense. Only teacher's apparently are responsible for paying for 
their own professional development. In any case, I've certainly never 
heard of an administrator who had any problem getting the funds 
necessary to attend conferences or workshops--it usually comes from 
the teacher travel funds.

If we are going to fix these problems, it starts with the education 
schools and the agencies that fund our school systems, and we need to 
start making sure that are teachers are well-qualified professionals 
and that they get treated that way. Unfortunately that seems to be 
counter to the current political climate--on both sides of the aisle.

Hugh
--
Hugh Haskell
mailto:hugh@ieer.org
mailto:haskellh@verizon.net

I have been wondering for a long time why some of our own defense 
officials do not
put more emphasis on finding a good substitute for oil and worry less 
about where
more oil is to come from.  Our people are ingenious. New discoveries 
are all around
us, and when we have to make them, we nearly always do.

						Eleanor Roosevelt
						February 13, 1948