Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

Re: [Phys-l] Review article on electronic instructional polling?

ABSTRACT: I comment on Derek Bruff's excellent "Classroom Response System Bibliography" with its references to classroom communication system (CCS) articles in biology, business, communications, computer science, earth science, economics, engineering, law, medicine, nursing, philosophy, physics, and political science. Unfortunately except for biology, computer science, earth science, economics, engineering, and physics, no standardized diagnostic tests appear to be available to gauge the need for, or effects of, CCS's (or any other type of reform pedagogy) on student learning. I give five pedagogical points regarding the use of CCS's and relay some RF keypad vendors from CCS guru Ray Burnstein.

If you reply to this long (17 kB) post please don't hit the reply button unless you prune the copy of this post that may appear in your reply down to a few relevant lines, otherwise the entire already archived post may be needlessly resent to subscribers.
Nora Bynum (2007) in her POD post of 15 Jun 2007 titled "Review article on electronic instructional polling?" wrote:

". . . is there a particular review article on the effectiveness of the use of electronic/handheld polling or audience response devices in instruction that folks on the listserve would recommend? "

Derek Bruff responded [bracketed by lines "BBBBB. . . ."; my insert at ". . . .[insert]. . . ."]:

As it turns out, I was just updating our center's bibliography on classroom response systems / audience response systems. The first two items on the page are recent, useful literature reviews:

. . . . [with a line break between "/technology/crs_b" and "iblio.htm"]. . . .

I hope that helps!

In my opinion, Derek's bibliography is very well done, providing, among other things, discipline specific articles (most of them hot-linked) for Biology, Business, Communications, Computer Science, Earth Science, Economics, Engineering, Law, Medicine, Nursing, Philosophy, Physics, and Political Science.

Unfortunately, as far as I know (please correct me if I'm wrong), except for biology, computer science, earth science, economics, engineering, and physics, no standardized diagnostic tests are available :-( to gauge the need for, or effects of, classroom communication systems (or any other type of reform pedagogy) on student learning. As I keep fruitlessly emphasizing [Hake (2005a, 2006a,b; 2007)], academia, in general, has ignored a salient lesson of the physics education reform effort [Hake (2002)]:

**High-Quality Standardized Tests of the Cognitive and Affective Impact of Courses Are Essential to Gauge the Relative Effectiveness of Non-Traditional Educational Methods.**

On a relatively trivial note, for some subscribers, easier access to Derek's bibliography might have been possible if:

1. The URL had been given as either:


or, even better:


2. The bare URL had been replaced by an academic reference e.g., Bruff (2007).

Regarding "1", suggestion #9 of my universally ignored "Fourteen Posting Suggestions" [Hake (2005b)] is:

"9. Give URL's as <http://......>. For most (but not all) server/mail systems: the "http://"; serves to hot-link URL's, and the angle brackets <. . .> serve to preserve hot-linking across line breaks. Note that unlike <>, its abbreviation <> is NOT hot-linked. CLICK ON ALL URL'S IN YOUR MESSAGE TO BE SURE THEY WORK."

Returning to classroom communication systems, although the references are now somewhat dated the posts "Handheld Devices" [Hake (2006c)], "Classroom Communication Systems" [Hake (2004a)] with 62 references, and "Classroom Communication Systems - Additional References" [Hake (2004b)] with 11 references might be of interest.

In the Hake (2004a] I wrote [SEE THAT POST FOR THE REFERENCES]:

Five points: Classroom Communication Systems (CCS):
1. Are not by themselves magic bullets, but have been used - at least in physics - to effectively promote "interactive engagement" (IE) methods: "heads-on (always) and hands-on (usually) activities which yield immediate feedback through discussion with peers and/or instructors." IE methods were found to have a nearly two-standard deviation superiority [cf. Bloom (1984)] over traditional (T) methods in enhancing students' conceptual understanding of mechanics by Hake (1998a,b; 2002a) and many other physics-education research groups as referenced in Hake (2002a,b)].

2. Are best tried in combination with pre/post testing to assess their impact on student learning relative to traditional methods, as in the work of Mazur (1997), Crouch & Mazur (2001), Fagen et al. (2002), Meltzer & Manivannan (2002), Dori & Belcher (2004), and Cheng et al. (2004). Such pre/post testing using reasonably well matched control groups (the traditional courses) does not meet the U.S. Dept. of Education's (USDE's) PSEUDO "gold standard" of randomized control trials, but [as argued in Hake (2004a)] would nevertheless probably pass muster at the USDE's "What Works Clearing House" <> as "quasi-experimental studies of especially strong design" [see <>]. Despite rampant pre/post paranoia [Hake (2000, 2004b)], pre/post assessments of student learning are being more and more utilized in fields such as astronomy, economics, biology, chemistry, computer science, and engineering [see Hake (2004c)].

3. May allow a cost-effective Socratic approach [Hake (1992, 2002c, 2004d), Abrahamson (1998)] to instruction in large-enrollment "lecture" sections. The Socratic potential of CCS has been generally overlooked in the literature, possibly because of the gross misunderstanding of the Socratic Method by academics (Hake 2004d).

4. May be more effective if used in conjunction with activities such as "Just in Time Teaching" (JITT) [Novak et al. (1999), Crouch & Mazur (2001)]. JITT induces study of and thinking about course material PRIOR to the "lecture."

5. Could be considered as "Low Threshold Applications" (LTA's) for the initiation of more effective education. LTA is a term evidently coined by Steve Gilbert of the Teaching, Learning and Technology Group <> for applications or activities that are not overly demanding of effort, resources, and time [see <>.

Those considering the adoption of CCS should be aware that the technology is advancing very rapidly. CCS guru Ray Burnstein (2001, 2003) wrote to me [Burnstein (2007)] in April 2007:

Let me present a list of RF keypad vendors and their e-mail addresses. There are more manufacturers but these are the principal vendors. RF keypads are widely available and cheap.

1. e-instruction/CPS - .......<>
2. iclicker -.........................<>
3. GTCO - .........................<>
4. H-ITT - ..........................<>
5. Qwizdom - .....................<>
6. Turning Point - ...............<>

To cull the sample I would eliminate #4 and #5. This still leaves a large sample to research. I played around with a lot of new keypad systems when I was visiting UTexas Austin, a year ago, but some of the keypad system models have even changed since then. . . . . . The Burnstein/Lederman (2003) paper. . . . .[ Bruff (2007) references that paper under "Vendor Comparison"]. . . . . . . on keypad systems is out of date as well as some more recent posters sessions presented at AAPT meetings. . . . [AAPT = American Association of Physics Teachers]. . . ..

Richard Hake, Emeritus Professor of Physics, Indiana University
24245 Hatteras Street, Woodland Hills, CA 91367

REFERENCES [Tiny URL's courtesy <>.]
Bruff, D. 2007. "Classroom Response System Bibliography" Vanderbilt Center for Teaching; online at

Burnstein, R.A. & L.M. Lederman. 2001. "Using wireless keypads in lecture classes." Phys. Teach. 39(1): 8-11; online at <>.

Burnstein, R.A. & L.M. Lederman. 2003. "Comparison of Different Commercial Wireless Keypad Systems," Phys. Teach. 41(5): 272-275; online at

Burnstein, R.A. 2007. Private communication of 11 Apr 2007 to R.R. Hake.

Bynum, N. 2007. "Review article on electronic instructional polling?" POD post of 15 Jun 2007 10:26:11-0400; online at <>.

Hake, R.R. 2002. "Lessons from the physics education reform effort," Ecology and Society 5(2): 28; online at <>. Ecology and Society (formerly Conservation Ecology) is a free online "peer-reviewed journal of integrative science and fundamental policy research" with about 11,000 subscribers in about 108 countries.

Hake, R.R. 2004a. "Classroom Communication Systems (was Sotl on Electronic Personal Response Systems?)," online at <>. Post of 18
Dec 2004 15:00:07-0800 to AERA-C, AERA-D, AERA-J, AERA-K, ASSESS, DrEd, EvalTalk, PhysLrnR, POD, and STLHE-L.

Hake, R.R. 2004b. "Classroom Communication Systems - Additional References," online at <> . Post of 22 Dec 2004 11:03:41-0800 to AERA-C, AERA-D, AERA-J, AERA-K, ASSESS, DrEd, EvalTalk, PhysLrnR, POD, and STLHE-L.

Hake, R. R. 2005a. "The Physics Education Reform Effort: A Possible Model for Higher Education?" online at <> (100 kB). This is a slightly edited version of an article that was (a) published in the National Teaching and Learning Forum 15(1), December, online to subscribers at <>, and (b) disseminated by the Tomorrow's Professor list <> as Msg. 698 on 14 Feb 2006. For an executive summary see Hake (2006a).

Hake, R.R. 2005b. "Fourteen Posting Suggestions," online at
<>. Post of 23 Jul 2005 11:38:29-0400 to AERA-C, AERA-G, AERA-GSL, AERA-H, AERA-I, AERA-J, AERA-K, AERA-L, ASSESS, EvalTalk, Math-Learn, PhysLrnR, POD, STLHE-L, TeachingEdPsych, and TIPS.

Hake, R.R. 2006a. "A Possible Model For Higher Education: The Physics Reform Effort (Author's Executive Summary)," Spark (American Astronomical Society Newsletter), June, online at <> (1.9MB). Scroll down about 4/5 of the way to the end of the newsletter.

Hake, R.R. 2006b. "Possible Palliatives for the Paralyzing Pre/Post Paranoia that Plagues Some PEP's" [PEP's = Psychometricians, Education specialists, and Psychologists], Journal of MultiDisciplinary Evaluation, Number 6, November, online at <>.

Hake, R.R. 2006c. "Handheld Devices," online at
<>. Post of 30 May 2006 14:51:28-0700 to AERA-L, ASSESS, EvalTalk, ITForum, PhysLrnR, & STLHE-L.

Hake, R.R. 2007. "Should We Measure Change? Yes!" online as ref. 43 at <>. To appear as a chapter in "Evaluation of Teaching and Student Learning in Higher Education," a Monograph of the American Evaluation Association <>. A severely truncated version appears at Hake (2006b).