If you reply to this long (17 kB) post please don't hit the reply
button unless you prune copy of this post that may appear in your
reply down to a few relevant lines, otherwise the entire already
archived post may be needlessly resent to subscribers.
*************************************
ABSTRACT
Anita Pincas (2008a) wrote that she was bothered by the use of
pre/post testing [extolled in "Can Scientific Research Enhance the
Art of Teaching?" [Hake (2007a)] because the post testing is
ordinarily carried out at the end of a course whereas "what has been
learned as a consequence of. . . [a course]. . . may not become
evident until very much later in the learner's life, for all kinds of
complex reasons." But who would argue that pedagogy that drastically
increases pre-to-post course gains over those attained by traditional
methods of instruction should be ignored because the post testing is
carried out directly after the course rather than "very much later in
the learner's life" ?
*************************************
In response to my post "Re: art of teaching" [Hake (2008a)] and its
reference to "Can Scientific Research Enhance the Art of Teaching?"
[Hake (2007a)], Anita Pincas (2008a) brought up two points in her
WBTOLL-L post to which I shall respond:
111111111111111111111111111
1. ". . . . I like your approach to teaching in term of input vis a
vis output [pre/post testing]. However, there is something that
bothers me a lot in this, namely the fact that what has been learned
as a consequence of a teaching event [or series of, in a course] may
not become evident until very much later in the learner's life, for
all kinds of complex reasons."
In physics education research pre-to-post course test gains have been
primarily used to gauge student learning in courses in Newtonian
mechanics - a conceptually difficult and counterintuitive subject.
It is not easy to promote student crossover from the intuitive
Aristotelian to the counterintuitive Newtonian World, but compared to
traditional passive-student lecture courses, "interactive engagement"
courses can yield normalized pre-to-post test gains which are about
two-standard deviations greater than those achieved by traditional
passive-student lecture courses [see Hake (1987; 1998a,b) and
confirmatory results in about 25 other independent peer-reviewed
publications as listed in Hake (2008b)].
However, I think it's unlikely that people will crossover to the
Newtonian World many years after a course without further guided
interactive engagement. The reason is that most ordinary real-world
experiences tend to reinforce Aristotelian rather than Newtonian
concepts, witness the Aristotelian outlook of most adults.
Nevertheless, I agree that what is learned in *some* courses, for
example psychology, "may not become evident until very much later in
the learner's life." Thus, in such courses where post testing
immediately follows the course, unimpressive pre-to-post test gains
may not necessarily mean that the course was of little benefit.
On the other hand, who would argue that pedagogy that drastically
increases pre-to-post course gains over those attained by traditional
methods of instruction should be ignored because the post testing is
carried out directly after the course rather than "very much later in
the learner's life" ?
Unfortunately, high-resolution pre/post test measurement of student
learning in most university courses is prevented by, among other
things, the pre/post paranoia that is rampant among some
psychologists, education specialists, and psychometricians (PEP's) -
see e.g., "Possible Palliatives for the Paralyzing Pre/Post Paranoia
that Plagues Some PEP's" [Hake (2006)], "Should We Measure Change?
Yes!" [Hake (2008c)], and "Can Distance and Classroom Learning Be
Increased?" [Hake (2008d)].
222222222222222222222222222222222222
2. "Agreed that an educational institution most frequently wants
'results now', but how can we also allow for later effects when
evaluating teaching, given that we cannot carry on post-testing?"
In a few instances, post testing HAS been done over periods of a few
years after the completion of mechanics courses. In "Six Lessons From
the Physics Education Reform Effort" [Hake (2007b)] I wrote
[bracketed by lines "HHHHHH. . . . ."; see that article for
references other than Chabay (1997), Francis et al. (1998), Hake
(1998a,b; 2005a), & Bernhard (2001)]:
HHHHHHHHHHHHHHHHHHHHHHHHHH
LESSON 1: THE USE OF INTERACTIVE ENGAGEMENT (IE) STRATEGIES CAN
INCREASE THE EFFECTIVENESS OF CONCEPTUALLY DIFFICULT COURSES WELL
BEYOND THAT OBTAINED BY TRADITIONAL (T) METHODS.
Education research in chemistry [Krause et al. (2004)]; engineering
[Froyd et al. (2006), Evans et al. (2003)]; and introductory science
education generally [Handelsman et al. (2004)], although neither as
extensive nor as systematic as that in physics [McDermott and Redish
(1999); Redish (1999); Thacker (2003); Heron & Meltzer (2005); Hake
(1998a,b; 2002a,b; 2005a; 2006a,b; 2007a,b); Wieman & Perkins (2005);
Wieman (2005)] is consistent with the latter in suggesting that, in
conceptually difficult areas, Interactive Engagement (IE) methods are
more effective than traditional T passive-student methods in
enhancing students' understanding. FURTHERMORE, THERE IS SOME
PRELIMINARY EVIDENCE THAT LEARNING IN IE PHYSICS COURSES IS
SUBSTANTIALLY RETAINED 1 TO 3 YEARS AFTER THE COURSES HAVE ENDED
[Chabay (1997), Francis et al. (1998), Bernhard (2001)].
I see no reason to doubt that enhanced understanding and retention
would result from greater use of IE methods in other science, and
even non-science, areas, but substantive research on this issue is
sorely needed - see e.g., "The Physics Education Reform Effort: A
Possible Model for Higher Education?" [Hake (2005a)].
HHHHHHHHHHHHHHHHHHHHHHHHHH
BTW, Anita's challenge to the value of pre/post testing is
reminiscent of her cogent challenge to the legitimacy of "Trigg's
Rules of Grammar" [Pincus (2008b,c)]. My thanks to Anita for her
helpful input (see the Burke signature quote).
"He . . . .[or she]. . . . that wrestles with us strengthens our
nerves, and sharpens our skill. Our antagonist is our helper."
Edmund Burke (1790)
". . . as typically used in existing studies, self reported gains or
growth do not permit the same level of internal validity as does
assessing gains by pretest-posttest design, where pretest and
posttest estimates are based on the same instrument."
Ernest Pascarella (2001)
REFERENCES
Bernhard, J. 2001. "Does active engagement curricula give long-lived
conceptual understanding?" pages 749-752 in R. Pinto and S. Surinach,
editors, "Physics teacher education beyond 2000." Elsevier, Paris,
France; online at
<http://staffwww.itn.liu.se/~jonbe/fou/didaktik/papers/girep2000_active.pdf>
(184 kB).
Hake, R.R. 1998a. "Interactive-engagement vs traditional methods: A
six thousand-student survey of mechanics test data for introductory
physics courses," Am. J. Phys. 66(1): 64-74; online at
<http://www.physics.indiana.edu/~sdi/ajpv3i.pdf> (84 kB). See also
the crucial companion paper Hake (1998b).
Hake, R.R. 1998b. "Interactive-engagement methods in introductory
mechanics courses," online at
<http://www.physics.indiana.edu/~sdi/IEM-2b.pdf> (108 kB) - a crucial
companion paper to Hake (1998a).
Hake, R. R. 2005a. "The Physics Education Reform Effort: A Possible
Model for Higher Education?" online at
<http://www.physics.indiana.edu/~hake/NTLF42.pdf> (100 kB). This is a
slightly edited version of an article that was (a) published in the
National Teaching and Learning Forum 15(1), December, online to
subscribers at
<http://www.ntlf.com/FTPSite/issues/v15n1/physics.htm>, and (b)
disseminated by the Tomorrow's Professor list
<http://ctl.stanford.edu/Tomprof/postings.html> as Msg. 698 on 14 Feb
2006. If your institution doesn't subscribe to the "National Teaching
and Learning Forum," then in should.
Hake, R.R. 2006. Possible Palliatives for the Paralyzing Pre/Post
Paranoia that Plagues Some PEP's, Journal of MultiDisciplinary
Evaluation 6, November, online at
<http://survey.ate.wmich.edu/jmde/index.php/jmde_1/issue/view/22>.
This even despite the admirable anti-alliteration advice at
psychologist Donald Zimmerman's site
<http://mypage.direct.ca/z/zimmerma/> to "Always assiduously and
attentively avoid awful, awkward, atrocious, appalling, artificial,
affected alliteration." This is a severely truncated version of
"Should We Measure Change? Yes! [Hake (2008c).
Hake, R.R. 2007a. "Can Scientific Research Enhance the Art of
Teaching?" invited talk, AAPT Greensboro meeting, 31 July, online at
<http://www.physics.indiana.edu/~hake/Sci&Art3.pdf> (1.2 MB). See
also Hake (2008a).
Hake, R.R. 2007b. "Six Lessons From the Physics Education Reform
Effort," Latin American Journal of Physics Education 1(1), September;
online at <http://journal.lapen.org.mx/sep07/HAKE%20Final.pdf> (124
kB).
Hake, R.R. 2008a. "Re: art of teaching," online at
<http://tinyurl.com/3ed3rl>. Post of 17 Sep 2008 to AERA-C, AERA-J,
AERA-K, AERA-L, AP-Physics, IFETS, NetGold, Physhare, Phys-L,
PhysLrnR, POD, STLHE-L, TIPS, & WBTOLL-L. For a guide to (and
critique of) discussion lists see Hake (2007c).
Hake, R.R. 2008b. "Design-Based Research in Physics Education
Research: A Review," in "Handbook of Design Research Methods in
Education: Innovations in Science, Technology, Engineering, and
Mathematics Learning and Teaching" [Kelly, Lesh, & Baek (2008)] -
publisher's information at <http://tinyurl.com/46k946>; a
pre-publication version of Hake's chapter is online at
<http://www.physics.indiana.edu/~hake/DBR-Physics3.pdf> (1.1 MB).
Hake, R.R. 2008d. "Can Distance and Classroom Learning Be Increased?"
IJ-SoTL 2(1): January; online at <http://tinyurl.com/2t5sro>. The
"International Journal of Scholarship of Teaching and Learning"
(IJ-SoTL) <http://www.georgiasouthern.edu/ijsotl/> is an open, peer
reviewed, international electronic journal containing articles,
essays, and discussions about the scholarship of teaching and
learning (SoTL) and its applications in higher/tertiary education
today.