Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

[Phys-L] Re: Research on Student Response Systems



If you respond to this long (18kB) post, please don't hit the reply
button unless you prune the original message normally contained in
your reply down to a few lines, otherwise you may inflict this entire
post yet again on suffering list subscribers. Also, if you object to
cross-posting as a method of tunneling through intra-disciplinary
barriers please hit DELETE.

In his Phys-L post of 10 Feb 2005 titled "Research on Student
Response Systems," Mark Lucas (2005) wrote [my CAPS]:

"I have a couple of colleagues trying to track down SOLID STUDIES,
preferably at the college level but also in K-12, regarding the
learning effectiveness of using Student Response Systems in class.
Does anyone have references for such studies?"

Lucas's post stimulated a 20-post (as of 16 Feb 2005 08:51:00-0800)
thread accessible on the February archives of Phys-L at
<http://lists.nau.edu/archives/phys-l.html>.

For 73 references on "Student Response Systems," aka "personal
response systems," aka "Classroom Communication Systems," see my two
posts "Classroom Communication Systems" Hake (2004a,b). Some of the
references are to papers that discuss research on such systems. One
reference not included in Hake (2004a,b) is Jackson et al. (2005)
that I picked up from the DrEd archives at
<http://list.msu.edu/archives/dr-ed.html>.

In Hake (2004a) I made five points regarding Classroom Communication
Systems [bracketed by lines "HHHHHHH. . . ."; see that post for
references other than Hake (1998a,b; 2002a), Crouch & Mazur (2001),
Dori & Belcher (2004), and Novak et al. (1999)]:

HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
Classroom Communication Systems (CCS):

1. Are not by themselves magic bullets, but have been used - at least
in physics - to effectively promote "interactive engagement" (IE)
methods: "heads-on (always) and hands-on (usually) activities which
yield immediate feedback through discussion with peers and/or
instructors." IE methods were found to have a nearly two-standard
deviation superiority [cf. Bloom (1984)] over traditional (T) methods
in enhancing students' conceptual understanding of mechanics by Hake
(1998a,b; 2002a) and many other physics-education research groups as
referenced in Hake (2002a,b)].

2. Are best tried in combination with pre/post testing to assess
their impact on student learning relative to traditional methods, as
in the work of Mazur (1997), Crouch & Mazur (2001), Fagen et al.
(2002), Meltzer & Manivannan (2002), Dori & Belcher (2004), and Cheng
et al. (2004). Such pre/post testing using reasonably well matched
control groups (the traditional courses) does not meet the U.S. Dept.
of Education's (USDE's) PSEUDO "gold standard" of randomized control
trials, but [as argued in Hake (2004a)] would nevertheless probably
pass muster at the USDE's "What Works Clearing House"
<http://www.w-w-c.org/> as "quasi-experimental studies of especially
strong design" [see
<http://www.w-w-c.org/reviewprocess/standards.html>]. Despite rampant
pre/post paranoia [Hake (2000, 2004b)], pre/post assessments of
student learning are being more and more utilized in fields such as
astronomy, economics, biology, chemistry, computer science, and
engineering [see Hake (2004c)].

3. May allow a cost-effective Socratic approach [Hake (1992, 2002c,
2004d), Abrahamson (1998)] to instruction in large-enrollment
"lecture" sections. The Socratic potential of CCS has been generally
overlooked in the literature, possibly because of the gross
misunderstanding of the Socratic Method by academics (Hake 2004d).

4. May be more effective if used in conjunction with activities such
as "Just in Time Teaching" (JITT) [Novak et al. (1999), Crouch &
Mazur (2001)]. JITT induces study of and thinking about course
material PRIOR to the "lecture."

5. Could be considered as "Low Threshold Applications" (LTA's) for
the initiation of more effective education. LTA is a term evidently
coined by Steve Gilbert of the Teaching, Learning and Technology
Group <http://www.tltgroup.org/listserv/index.html> for applications
or activities that are not overly demanding of effort, resources, and
time [see <http://www.tltgroup.org/LTAs/Home.htm>.

For an exemplary use of "classroom communication systems" coupled
with "Just in Time Teaching" [Novak et al. (1999)] in introductory
physics education see Crouch & Mazur (2001).
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH

But BEWARE !! There are no "SOLID STUDIES" whatsoever !!! According
to Rick Tarara (2005) [bracketed by lines "TTTTTTT. . . .]"

TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT
The Hake . . .[1985a,b)]. . . paper illustrates most of the pitfalls
of such attempts, however. There are really no controls here since
the data are independently reported. Classifications of
'traditional' or 'interactive' are somewhat arbitrary with some
questionable assignments. The assessment tool is primarily the FCI
and there has been a lot of past discussion about the validity of
that -- in terms of dealing with a very narrow subset of physics,
being 'taught to' by those knowing it will be the assessment tool for
their techniques, etc. It is also where the lack of controls shows
up -- amount of time and emphasis spent on the topic for example. In
the end, this
collection of data (by an avowed advocate of interactive engagement
styles) points out how difficult it is to do meta-research in the
educational field (just research for that matter).

Even by the standards of psychological or sociological studies, most
projects are lacking - mostly because of the inherent difficulties.
All of which is simply to say that studies that proclaim this or that
as the best thing since peanut butter, or report some statistical
measure taken out to 4 significant figures, should be viewed with a
little skepticism. When I start seeing students arrive at college
who have gone through these 'new pedagogy' courses and actually know
some basic physics, then I'll get more excited about the
whole thing. Until then, as suggested earlier, I'll use my own experience
and common sense and incorporate those new techniques and tools that I deem
useful and fit best with my style.
TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT

It's unfortunate that some physicists, even those with established
reputations in traditional hard-core physics research [e.g., Arnold
Arons, John Belcher, Ken Heller, David Hestenes, Bob Hilborn, Bob
Karplus, Eric Mazur, Joe Redish, Fred Reif]; and (believe it or not)
even exemplars at Harvard [Crouch et al. (2001)] and MIT [Dori &
Belcher (2004)], do not possess Tarara's insight into the utter
futility of all physics education research (PER).

Even Cliff Swartz (1999), former editor of "The Physics Teacher" and
long-time acerbic critic of physics-education research, has failed to
reach Tarara's breakthrough realization that *all* PER is useless.
Swartz wrote [my CAPS and insertions at [. . . .]]:

CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
There is a variety of evidence, and claims of evidence, that each of
the latest fads . . .(constructivism, 'group' and 'peer' instruction,
'interaction') . . . produces superior learning and happier students.
In particular, students who interact with apparatus or lecture do
better on the "Force Concept Inventory" exam (Hestenes et al. 1992).
The evidence of Richard Hake's (1998a) metastatistical study is so
dramatic that the only surprising result is that many schools and
colleges are still teaching in old-fashioned ways. Perhaps the
interaction technique reduces coverage of topics, or perhaps the
method requires new teaching skills that teachers find awkward. . .
.[or perhaps there are too many Tarara's in physics teaching]. . ..AT
ANY RATE THE NEW METHODOLOGY IS NOT SWEEPING THE NATION.
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC

Richard Hake, Emeritus Professor of Physics, Indiana University
24245 Hatteras Street, Woodland Hills, CA 91367
<rrhake@earthlink.net>
<http://www.physics.indiana.edu/~hake>
<http://www.physics.indiana.edu/~sdi>

"Education is not rocket science, it's much harder."
George Nelson, astronaut, astrophysicist, and former director of the AAAS
Project 2061, as quoted by Redish (1999).

"The scientific method . . .is nothing more than doing one's
damnedest with one's mind, no holds barred. . ."
Percy Bridgeman (1950)


REFERENCES
Bridgeman, P. 1950. "Reflections of a Physicist." Philosophical Library.

Crouch, C.H. & E. Mazur. 2001. "Peer Instruction: Ten years of
experience and results," Am. J. Phys. 69: 970-977; online at
<http://mazur-www.harvard.edu/publications.php>. Search under "All
education areas" for author "Crouch."

Dori, Y.J. & J. Belcher, J. 2004. "How Does Technology-Enabled Active
Learning Affect Undergraduate Students' Understanding of
Electromagnetism Concepts?" To appear in The Journal of the Learning
Sciences 14(2), online at
<http://web.mit.edu/jbelcher/www/TEALref/TEAL_Dori&Belcher_JLS_10_01_2004.pdf>
(1 MB).

Hake, R.R. 1998a. "Interactive-engagement vs traditional methods: A
six-thousand-student survey of mechanics test data for introductory
physics courses," Am. J. Phys. 66: 64-74; online as ref. 24 at
<http://www.physics.indiana.edu/~hake>, or simply click on
<http://www.physics.indiana.edu/~sdi/ajpv3i.pdf> (84 kB). A
comparison of the pre- to post-test average normalized gain <g> for
62 introductory high-school, college, and university physics courses
enrolling a total 6542 students showed that fourteen "traditional"
(T) courses (N = 2084) which made little or no use of
interactive-engagement (IE) methods achieved an average gain <g>T-ave
= 0.23 plus or minus 0.04 (std dev), regardless of the experience,
enthusiasm, talents, and motivation of the lecturers. In sharp
contrast, forty-eight courses (N = 4458) which made substantial use
of IE methods achieved an average gain <g>IE-ave = 0.48 plus or minus
0.14 (std dev), almost two standard deviations of <g>IE-ave above
that of the traditional courses. Here: (a) the average normalized
gain <g> is the actual gain [<%post> - <%pre>] divided by the
maximum possible gain [100% - <%pre>] where the angle brackets
indicate the class averages; (b) IE courses are operationally defined
as those designed at least in part to promote conceptual
understanding through interactive engagement of students in heads-on
(always) and hands-on (usually) activities which yield immediate
feedback through discussion with peers and/or instructors; (c) T
courses are operationally defined courses as those reported by
instructors to make little or no use of IE methods, relying primarily
on passive-student lectures, recipe labs, and algorithmic problem
exams. More recently, normalized gain differences between T and IE
courses that are consistent with the work of Hake (1998a,b) have been
reported by many other physics education research groups as
referenced in Hake (2002a,b).

Hake, R.R. 1998b. "Interactive-engagement methods in introductory
mechanics courses," online as ref. 25 at
<http://www.physics.indiana.edu/~hake>, or simply click on
<http://www.physics.indiana.edu/~sdi/IEM-2b.pdf> (108 kB) Submitted
on 6/19/98 to the Physics Education Research Supplement (PERS) to Am.
J. Phys. but rejected by its editor on the grounds that the very
transparent Physical Review-type data tables were too complex! PER
suffers because it has no Physical-Review-type archival journal.
This paper is a crucial companion paper to Hake (1998a): average
pre/post test scores, standard deviations, instructional methods,
materials used, institutions, and instructors for each of the survey
courses of Hake (1998a) are tabulated and referenced. In addition the
paper includes: (a) case histories for the seven IE courses of Hake
(1998a) whose effectiveness as gauged by pre-to-post test gains was
close to those of T courses, (b) advice for implementing IE methods,
and (c) suggestions for further research.

Hake, R.R. 2002a. "Lessons from the physics education reform effort,"
Ecology and Society 5(2): 28; online at
<http://www.ecologyandsociety.org/vol5/iss2/art28/>. Ecology and Society
(formerly Conservation Ecology) is a free online "peer-reviewed
journal of integrative science and fundamental policy research" with
about 11,000 subscribers in about 108 countries,

Hake, R.R. 2002b. "Assessment of Physics Teaching Methods,
Proceedings of the UNESCO-ASPEN Workshop on Active Learning in
Physics, Univ. of Peradeniya, Sri Lanka, 2-4 Dec. 2002; also online
as ref. 29 at
<http://www.physics.indiana.edu/~hake/>.

Hake, R.R. 2004a. "Classroom Communication Systems," online at
<http://listserv.nd.edu/cgi-bin/wa?A2=ind0412&L=pod&O=D&P=24855>.
Post of 18 Dec 2004 15:00:07-0800 to POD, AERA-C, AERA-D, AERA-J,
AERA-K, ASSESS, DrEd, EvalTalk, PhysLrnR, and STLHE-L.

Hake, R.R. 2004b. "Classroom Communication Systems - Additional
References," online at
<http://listserv.nd.edu/cgi-bin/wa?A2=ind0412&L=pod&P=R22831&I=-3>
Post of 1 22 Dec 2004 11:03:41-0800 to POD, AERA-C, AERA-D, AERA-J,
AERA-K, ASSESS, DrEd, EvalTalk, PhysLrnR, and STLHE-L.

Jackson, M., A.C. Ganger, P.D. Bridge, & K. Ginsburg. 2005. "Wireless
Handheld Computers in the Undergraduate Medical Curriculum." Med Educ
Online 2005; 10:5; online as a 316 kB pdf at
<http://www.med-ed-online.org/pdf/t0000062.pdf>. For an abstract see
<http://www.med-ed-online.org/volume10.htm#t0000062>. The authors
give some good medically oriented references, but no references on
the general use of classroom communication systems outside medical
education.

Lucas, M. "Research on Student Response Systems," Phys-L post of 10
Feb 2005 17:51:49-0500; online at
<http://lists.nau.edu/cgi-bin/wa?A2=ind0502&L=phys-l&F=&S=&P=17789>.

Novak, G., E. Patterson, A. Gavrin, and W. Christian. 1999.
"Just-in-Time Teaching: Blending Active Learning and Web Technology."
Prentice-Hall; for an overview see
<http://webphysics.iupui.edu/jitt/jitt.html>; for implementation
information see <http://galileo.harvard.edu/galileo/sgm/jitt/>.

Redish, E.F. 1999. "Millikan lecture 1998: building a science of
teaching physics," Am. J. Phys. 67(7): 562-573; online at
<http://www.physics.umd.edu/rgroups/ripe/perg/cpt.html>.

Swartz, C.E.. 1999. Editorial: Demise of a shibboleth. Phys. Teach. 37: 330.

Tarara, R. 2005. "Re: Research on Student Response Systems," Phys-L
post of 13 Feb 2005 09:50:04 -0500; online at
<http://lists.nau.edu/cgi-bin/wa?A2=ind0502&L=phys-l&F=&S=&P=20726>.