Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

Re: [Phys-L] Indicators of quality teaching : some necessities



On 06/28/2013 02:37 PM, Bruce Sherwood wrote:

It may be stating the obvious, but it's very difficult to carry out a
meaningful longitudinal study,

Yes, it's very difficult ... and yes, it "should" be obvious.
It has to be difficult. It's an experiment on human subjects!
Think of all the elaborate protocols that people go through for
drug trials. The researchers know that if they do any less,
the results will be wildly unreliable.

Experimenting on teaching methods is just as hard as drug
trials, only worse. It's worse because some of the protections
you would like to use are not available -- such as double
blinding.

which surely relates to why you don't see
many examples.

Yes, there are few examples of well-done experiments, which is
to be expected, given that they are expensive and difficult.
The paucity of well-done experiments is not what bothers me.
The problem is the super-abundance of shoddy experiments.
These are no bargain. They cost 100 times less, but they're
worth 1000 times less, if they're worth anything at all.

We're seeing extreme examples of looking under the lamp-post:
It's easy to do, but most of what folks should be looking for
is elsewhere.

=========

Note that there are two issues here:
1) Proper controls, including randomized controls, and
2) Longitudinal follow-up.

The former is more important; the latter is at best a
tactic, not a strategy, let alone a panacea. Follow-up
makes it possible for better controls to be applied ...
but it does not guarantee that they will be.

==================================


The rate of loss in the two groups appeared to be ...

The fact that we're talking about long-term loss instead
of long-term gain means there is a *lot* of work still
to be done.

Let's be clear: On a test of simple, fundamental concepts,
one would hope that students who come back in later years
would have more numerous ways of looking at each problem,
and more sophisticated ways, so that their scores would be
higher, not lower.

If you say loss is to be expected because the students
don't use the material in later years, that is IMHO a
huge red flag that says either
a) we're teaching the wrong material, or
b) we're using completely the wrong tests,
... or both.

You would think that the really basic physics would be
applicable to real life, every day. In this case, the
action item for us is to make sure students see the
connection to *real* things ... as opposed to monkey-
shooting.

As for the more advanced stuff, such as electromagnetism,
it is understandable that they would not need that every
day ... in which case the question arises, why bother
teaching it at all? The stock answer is that one hopes
they retain some subliminal understanding, so that if/when
they ever need it they can re-learn it in a hurry. Well,
if that is even partly true, then the longitudinal study
should measure how quickly they can re-learn it!

Remember: Be careful what you test for; you might
get it.

Yeah, I know it is really hard to measure how quickly
somebody re-learns something. That is 100% consistent
with what I've been saying: In this business, most of
the things that are worth doing are hard. Looking under
the lamp-post costs 100 times less, but it is worth 1000
times less, if it is worth anything at all.