Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

[Phys-L] Indicators of quality teaching : some necessities



Suppose you were conducting field trials of a new drug. There
are standard protocols that must be followed, involving proper
controls, randomization, double-blinding, and checking the
long-term (not just short-term) outcome. Everybody knows how
important such protocols are.

Education research is not fully analogous. For one thing, there
is usually no hope of double blinding; the students know whether
they are using textbook A or textbook B.

On the other hand, there are still plenty of parallels. For
starters, it is still super-important to look at the long-term
outcome. The trendy term for this is "longitudinal" studies.

As another point of terminology: The thing that people in this
forum have been calling "gain" is a slightly narrower version of
what the education literature calls "value added".

Longitudinal value-added modeling is trendy, and becoming steadily
more so. This is mostly a good thing, if you ask me. It doesn't
solve all the world's problems, but it is waaaay better than any
kind of non-longitudinal study. For almost every fault you can
find with longitudinal data, the non-longitudinal data has the
same fault, only worse.

In particular: There is a big difference between test-prep
and life-prep. There will always be an incentive to teach
to the test; this is either a good thing or a bad thing,
/depending on the test/. If the test is one or two years
down the road, there is much more of a wholesome incentive
to teach things of lasting value, rather than a perverse
incentive to cram for the test.

Add this to the long list of reasons why I roll my eyes when
I see PER papers that use pre-test / post-test "FCI gain" as
if it were an "indicator of quality teaching".

1) There's a lack of properly randomized controls.
2) There's a lack of longitudinal follow-up.
3) The test doesn't measure what I really care about, because
it is too narrowly focused.
4) It is a one-sided test: A low score is a reliable
indicator of lousy teaching, but a high score is not a
reliable indicator of quality teaching, because the test
is too simple. The gains that people brag about are so
low as to prove that the students do not understand
"conceptual physics". If they understood the fundamental
concepts, they would score much higher.

Issues (3) and (4) are specific to the FCI, but if you solved
these issues by switching to a more comprehensive, reasoning-
intensive test, you would still need to deal with issues (1)
and (2).

=========

I get literally millions of hits from
http://www.google.com/search?q=longitudinal+value-added