Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

[Physltest] [Phys-L] Re: "Effective" teaching methods



In his Phys-L post of 16 Nov 2004 13:59:57-0500 titled 16 Nov 2004
13:59:57-0500, John Denker wrote [bracketed by lines "DDDDDDDDD. . .
."; academic references added]:

DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD
In simplest terms, the Hawthorne Experiments taught us that

a) The placebo effect is strong. Just making a fuss over somebody has a big
effect.

b) It's easy to get fooled. It's hard to get the right answer.

It's quite a tale. Check out:

Sidebar #2 in
<http://www.bellsystemmemorial.com/westernelectric_history.html> [Bell
System Memorial (1969?)

also
<http://www.mgmtguru.com/mgt301/301_Lecture1Page9.htm> [Management Guru
(undated)]. . . .

IMHO if you haven't thought carefully about these experiments, you
don't know very much about
a) psychology, or
b) the design of experiments on human subjects.

Education involves a lot of (a) and education research involves a lot of (b).

The Hawthorne Experiments were *initially* briefly one of the
all-too-common cases where the investigators were fooled by the
placebo effect. But *eventually* it became one of the all-too-rare
cases where they stayed with it long enough to realize they'd been
fooled, and then to figure out what was
actually going on."

Another lesson that can be drawn is that a thousand little, quick,
cheap, sloppy experiments are not nearly as valuable as a few large,
extensive, expensive, well-run experiments.

One of the most appalling things is when people pay lip service to
the Hawthorne Experiments and then make the same old mistakes. The
same old nonsense has been going on for years. RPF [Richard P.
Feynman] said some pointed things about this at the 1974 commencement
(the "Cargo Cult Science" address). Check out
<http://www.lhup.edu/~dsimanek/cargocul.htmb> [Feynman (1974)]
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD

But do Denker's references [Bell System Memorial (1969?) and
Management Guru (undated)] really show that the investigators "stayed
with it long enough to realize they'd been fooled, and then to figure
out what was actually going on."

Even Management Guru (undated) states (My CAPS): "DESPITE MODERN
CRITICISM THAT THE RESEARCH WAS FLAWED and that incentives played a
larger role in improving worker productivity than the Hawthorne plant
researchers concluded, these studies changed the landscape of
management from Taylor's engineering approach to a social sciences
approach."

Counter evidence to Denker's claim that Hawthorne effect researchers
"stayed with it long enough to realize they'd been fooled, and then
to figure out what was actually going on" is also supplied by:

(a) Rice (undated) as supplied by Jack Uretsky in his Phys-L post of
16 Nov 2004 15:15:05-0600,

(b) Adair (1973),

(c) Bramel & Friend (1981),

(d) Gillespie (1988).

References b, c, and d are from Shadish et al. (2002). On page 79
under "Threats to Construct Validity" they write:

"Novelty and Disruptive Effects: Bracht and Glass (1968) suggested
that when an innovation is introduced, it can breed excitement,
energy, and enthusiasm that contribute to success, especially if
little innovation previously occurred [5]. . ."

Footnote [5] is:

"One instance of this threat is frequently called the "Hawthorne
Effect" after studies done at Western Electric Company's Hawthorne
site [Roethlisberger & Dickson (1939)]. In an early interpretation of
this research, it was thought that participants responded to the
attention being given to them by increasing their productivity,
whatever the treatment was. THIS INTERPRETATION HAS BEEN CALLED INTO
QUESTION [Adair (1973), Bramel & Friend (1981), Gillespie (1988)];
but the label "Hawthorne Effect" is likely to be used to describe it."

BTW, in Hake (1998a) I adduced evidence that the conclusion of that study:

"The conceptual and problem-solving test results strongly suggest
that the classroom use of "Interactive Engagement" methods can
increase mechanics-course effectiveness well beyond that obtained in
traditional practice" were not influenced by either the "Hawthorne
Effect" nor it's opposite the "John Henry Effect" [as they are
defined in standard texts such as Slavin (1992)].

The "cargo-cult" science described by Feynmann (1974) does not
characterize Hake (1998a,b) since the test groups [interactive
engagement (IE) courses] were compared to reasonably well matched
control groups [traditional (T) courses]. There was a nearly
two-standard deviation superiority of average normalized gains <g> of
the IE courses over those of the T courses.

Yet Herb Gottlieb (2004a,b) and John Denker (2004) would have us
believe that IE reform its just another one of those evanescent
cycles that so mislead physics instructors.


Richard Hake, Emeritus Professor of Physics, Indiana University
24245 Hatteras Street, Woodland Hills, CA 91367
<rrhake@earthlink.net>
<http://www.physics.indiana.edu/~hake>
<http://www.physics.indiana.edu/~sdi>

"Human beings, who are almost unique in
having the ability to learn from the
experience of others, are also remarkable
for their apparent disinclination to do
so."
Douglas Adams in "Last Chance to See"


REFERENCES
Adair, J.G. 1973. "The Hawthorne Effect: A reconsideration of a
methodological artifact," Journal of Applied Psychology 69: 334-345.

Bell System Memorial. 1969? "Western Electric - A Brief History"
online at
<http://www.bellsystemmemorial.com/westernelectric_history.html>. See
sidebar #2 [bracketed by lines "SSSSSSSS. . . ":

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
From 1924 until 1933, the Hawthorne plant was the site of a series of
experiments conducted under the auspices of the National Research
Council. The initial studies involved the impact of changes in
lighting levels on the productivity of several groups of workers. The
first two sets of tests showed that increased levels of supervision
played a much larger role in productivity increases than levels of
illumination.

The most involved of the experiments, the relay assembly test room
experiment, involved isolating six women, then measuring their
production, health, and social interactions in response to changes in
working conditions, such as the number and duration of rest periods,
length of the work day, and the amount of food they ate. Productivity
increased as each improvement was introduced, until the crucial
twelfth test, in which researchers removed the special conditions.
Productivity increased again! One of the researchers called the
twelfth test "the great eclairissement (sic), the new illumination,
that came from the research." The experiments raised the possibility
that, as Thomas J. Peters and Robert Waterman put it, "it is the
attention to employees, not work conditions per se, that has a
dominant impact on productivity."

The impact of the experiments has been felt worldwide, and by many
generations. In the 1950's, a number of Japanese executives visited
Western Electric and told their hosts that, "Management and the
Worker," . . . Roethlisberger & Dickson (1939). . . a book
summarizing the findings from the Hawthorne experiments, was required
reading in Japanese schools of management. The phrase "Hawthorne
effect" has come to mean any unexpected outcomes from
non-experimental variables in social or behavioral sciences. The
Hawthorne experiments have been elevated to what one historian calls
the "status of Creation myth" in many fields that study the
workplace, from sociology to psychology to anthropology.
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS

Bracht, G.H. & G.V. Glass. 1968. "The external validity of
experiments," American Educational Research Journal 5: 437-474.

Bramel, D. & R. Friend 1981. "Hawthorne, the myth of the docile
worker, and class bias in psychology, American Psychologist 36:
867-878.

Denker, J. 2004. "Re: Will (whatever) Promote (whatever)," Phys-L
post of 15 Nov 2004 21:59:17-0500; online at
<http://lists.nau.edu/cgi-bin/wa?A2=ind0411&L=phys-l&O=A&P=21186>.

Feynman, R. "Cargo Cult Science," online at
<http://www.lhup.edu/~dsimanek/cargocul.htm>. Feynman wrote:

FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
Other kinds of errors are more characteristic of poor science. When I
was at Cornell, I often talked to the people in the psychology
department. One of the students told me she wanted to do an
experiment that went something like this -- it had been found by
others that under certain circumstances, X, rats did something, A.
She was curious as to whether, if she changed the circumstances to Y,
they would still do A. So her proposal was to do the experiment under
circumstances Y and see if they still did A.

I explained to her that it was necessary first to repeat in her
laboratory the experiment of the other person -- to do it under
condition X to see if she could also get result A, and then change to
Y and see if A changed. Then she would know that the real difference
was the thing she thought she had under control.

She was very delighted with this new idea, and went to her professor.
And his reply was, no, you cannot do that, because the experiment has
already been done and you would be wasting time. This was in about
1947 or so, and it seems to have been the general policy then to not
try to repeat psychological experiments, but only to change the
conditions and see what happens.
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF

Gillespie, R. 1988. "The Hawthorne experiments and the politics of
experimentation," in J. Morawski, ed. "The rise of experimentation in
American psychology, pp. 114-137. Yale Univ. Press.

Gottlieb, H. 2004a. "Re: Will NCLB Promote Direct Instruction of Science?"
," Phys-L post of 15 Nov 2004 21:38:48-0500; online at
<http://lists.nau.edu/cgi-bin/wa?A2=ind0411&L=phys-l&O=A&P=20940>.

Gottlieb, H. 2004b. "Effective" teaching methods," Phys-L post of 15
Nov 2004 22:00:29-0500; online at
<http://lists.nau.edu/cgi-bin/wa?A2=ind0411&L=phys-l&O=A&P=21075>.
The same post at Gottlieb (2004b) but with a different title that
stimulated a 38 post Phys-L thread (as of 17 Nov 2004 19:23:08-0800).

Hake, R.R. 1998a. "Interactive-engagement vs traditional methods: A
six-thousand-student survey of mechanics test data for introductory
physics courses," Am. J. Phys. 66: 64-74; online as ref. 24 at
<http://www.physics.indiana.edu/~hake>.

Hake, R.R. 1998b. "Interactive-engagement methods in introductory
mechanics courses," online as ref. 25 at
<http://www.physics.indiana.edu/~hake>. Average pre/post test scores,
standard deviations, instructional methods, materials used,
institutions, and instructors for each of the survey courses of Hake
(1998a) are tabulated and referenced.

Management Guru. undated. "The Hawthorne Experiments: Management
Takes A New Direction,"
<http://www.mgmtguru.com/mgt301/301_Lecture1Page9.htm>. The
Management Guru wrote [bracketed by lines "MG-MG-MG. . ."]

MG-MG-MG-MG-MG-MG-MG-MG-MG-MG-MG-MG-MG-MG-MG
Despite modern criticism that the research was flawed and that
incentives played a larger role in improving worker productivity than
the Hawthorne plant researchers concluded. These studies changed the
landscape of management from Taylor's engineering approach to a
social sciences approach. Worker productivity would, henceforth, be
interpreted predominately in the United States in terms of social
group dynamics, motivation, leadership, and "human relations". The
practice of management could not be the aloof technician of Taylor's
Scientific Management, designing the job, selecting and training the
"right" worker, and rewarding for performance. The manager was an
immediate part of the social system in which work is performed,
responsible for leading, motivating, communicating, and designing the
social milieu in which work takes place. The studies also developed
the scholars that would continue to influence the American way of
thinking about management at Harvard Business School and elsewhere.
MG-MG-MG-MG-MG-MG-MG-MG-MG-MG-MG-MG-MG-MG-MG

Rice, B. undated. "The Hawthorne defect: Persistence of a flawed
theory," unpublished?, online at
<http://www.cs.unc.edu/~stotts/204/nohawth.html>. Berkeley Rise is a
senior editor of "Psychology Today."

Roethlisberger, F.S. & W.J. Dickson. 1939. Management and the
worker," Harvard Univ. Press.

Shadish, W.R., T.D. Cook, & D.T. Campbell. 2002. "Experimental and
Quasi-Experimental Designs for Generalized Causal Inference."
Houghton Mifflin. A goldmine of references to the social-science
literature of experimentation.

Slavin, R.E. 1992. "Research Methods in Education." Allyn and Bacon,
2nd ed. The "Hawthorne Effect" & its opposite the "John Henry Effect"
are treated on page 112.
_______________________________________________
Phys-L mailing list
Phys-L@electron.physics.buffalo.edu
https://www.physics.buffalo.edu/mailman/listinfo/phys-l