Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

[Phys-l] The Myths of Innovation (Part 3)



If you reply to this long (24 kB) post please don't hit the reply button unless you prune the copy of this post that may appear in your reply down to a few relevant lines, otherwise the entire already archived post may be needlessly resent to subscribers.

***************************************************
ABSTRACT: I respond to comments by IFETS subscriber James Kariuki regarding innovation in higher education: (a) the training and support that's been available, and (b) the formidable barriers to innovation. Regarding the latter, I discuss the student resistance, as described by John Belcher, to the innovative TEAL program at MIT. Belcher's experience is consistent with James Rhem's essay "The High Risks of Improving Teaching"; Patti Thorn's dissertation "Bridging the Gap Between What Is Praised and What Is Practiced: Supporting the Work of Change as Anatomy & Physiology Instructors Introduce Active Learning into Their Undergraduate Classroom"; my own "Re: Problems with Student Evaluations: Is Assessment the Remedy?"; and with David Garvin's description of student outrage over Harvard's initiation of the case-based method.
***************************************************

In response to my post "The Myths of Innovation" (Part 1) [Hake (2007a)], James Kariuki (2007), in his IFETS post of 27 May 2007 wrote [bracketed by lines "KKKKKK. . . . ; slightly edited; my A, B]:

KKKKKKKKKKKKKKKKKKKKKKKKKKKKKK
A. I'm not sure how much training and support has been available in higher education for professors to use innovation, and how effective it is.
B. The knowledge and skills acquisition is very important and is strenuous, time consuming, expensive and tedious especially when it involves a complete change on the roles, routines and norms of the adopters (see Klein & Knight, 2005).
KKKKKKKKKKKKKKKKKKKKKKKKKKKKKK

REGARDING "A":
Some training and support for pedagogical innovation in U.S. higher education has come from education specialists in university teaching and learning centers (TLC's) [see, e.g., the listing at CTSE (2007)]. In fact, such support from MIT's excellent TLC <http://web.mit.edu/tll/> contributed to the success of John Belcher's TEAL (Technology-Enabled Active Learning) project, discussed in "Re: The Myths of Innovation" [Hake (2007b)].
However, the main impetus in physics has come from the bottom-up Physics Education Research (PER) Groups [over 50 U.S. PER groups are listed by Meltzer (2007)]. Unfortunately such education research groups are seldom found in other disciplines, even despite the encouragement of exemplary groups such as the "Scholarship of Teaching and Learning" [SOTL (2007)], the "International Society for Exploring Teaching and Learning" [ISETL (2007)], and the "International Society for Scholarship of Teaching and Learning" [ISSOTL (2007)].

The research and development work of PER groups in the U.S. has been reviewed by e.g., Redish (1999), Stokstad (2001), Heron & Meltzer (2005), and Hake (2007c). Stokstad's abstract reads: "Physicists are out in front in measuring how well students learn the basics, as science educators incorporate hands-on activities in hopes of making the introductory course a beginning rather than a finale. Figuring out what works is vitally important to the country, say U.S. educators. Each year, hundreds of thousands of U.S. students get their only exposure to science in an intro class -- and most leave without understanding how science works or with any desire to take further courses."
REGARDING "B":
That enumeration of barriers to innovation is consistent with the research of Henderson and Dancy (2006a,b), and with the pathologically slow diffusion of innovation in higher education. In addition, students themselves are sometimes barriers to innovation. In Hake (2007b), I omitted sections of the Ehrmann/Gilbert/McMartin (EGM) (2007) report that discussed the anti-TEAL comments of some MIT students, e.g. [bracketed by lines "EGM-EGM-EGM-. . . "' my insert at ". . . . .[insert]. . . . "]:

EGM-EGM-EGM-EGM-EGM-EGM-EGM-EGM-EGM
Whatever the reason, in spring 2003, students complained wherever they could; 150 students signed a petition to make TEAL course , at most, an option for freshman physics. According to a story . . . . .[LeBon (2003)]. . . . in "The Tech" the student newspaper, "8.02 TEAL does not provide us with the intellectual challenge and stimulation that can be expected from a course at MIT. We feel that the quality of our education has been compromised for the sake of 'trying something different.' We strongly advise that the traditional 8.02 course be reinstated as soon as possible."
EGM-EGM-EGM-EGM-EGM-EGM-EGM-EGM-EGM

John Belcher (2007), in a PhysLrnR post titled "Re: Student Resistance to Teaching Improvement," wrote [bracketed by lines "BBBBBB. . . . . ."; my insert at ". . . .[insert]. . . ."]:

BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
. . . . [Larry Smith (2007) asks]. . . . "Have people on this list encountered student resistance to active learning techniques, and what have they done about it?"

Yes I have, take a look at . . . . .["Improving Student Understanding with TEAL" [Belcher (2003)]. . . . . . (the links are live) for a blow by blow description, including links to various critical articles in the student newspaper.

This was written just after the initial large scale outing in Spring 2003 with 500 students. The student reception has gotten much better, because we made a lot of changes as outlined in. . . . .[Belcher (2003)]. . . ., but we still run about a point lower on a 7 point scale on "overall satisfaction" with the course than we did previously in lecture recitation format. I lectured in the lecture recitation format in the early 90's, to great praise, from the 40% of the students who attended lecture (I don't know what the other 60% thought, they were not there to be asked).
On the other hand in studio format we have doubled attendance from around 40% at the end of the term to around 80% at the end of the term, and those additional 40% who are weighing in with their opinion now were not there before (and wouldn't be there now, except we actually look to see if they are coming), so I would argue at this point that the student satisfaction is about the same as the lecture recitation format.
What saved this program given the initial student resistance was EVALUATION EVALUATION EVALUATION.
We had a professional evaluator, WE DID LOTS AND LOTS OF PRE TEST POST TEST TESTING [my CAPS], and we could demonstrate convincingly that the students were learning more -- the learning gains are double those of lecture recitation by standard measures (for the summative evaluation see . . . . [Dori & Belcher (2004)]. . . . . .
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB

That students may not always be the best judges of teaching effectiveness, as indicated above by Belcher, is consistent with e.g.:

a. James Rhem's (2006) essay "The High Risks of Improving Teaching" that drew on;
b. Patti Thorn's (2006) dissertation "Bridging the Gap Between What Is Praised and What Is Practiced: Supporting the Work of Change as Anatomy &Physiology Instructors Introduce Active Learning into Their Undergraduate Classroom";

c. My own post "Re: Problems with Student Evaluations: Is Assessment the Remedy?"[Hake (2002)]; and

d. The history of the case-based method as related in "Making the Case: Professional education for the world of practice" [Garvin (2003)] referred to in Hake (2004, 2007).

All this despite the blind faith of many university administrators that "Student Evaluations of Teaching" can be used to gauge the *cognitive* (as well as the affective) impact of courses.

David Garvin (2003) wrote:

GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG
Christopher Columbus Langdell, the pioneer of the case method, attended Harvard Law School from 1851 to 1854 - twice the usual term of study. He spent his extra time as a research assistant and librarian, holed up in the school's library reading legal decisions and developing an encyclopedic knowledge of court cases. . . .

In his course on contracts, he insisted that students read only original sources-cases-and draw their own conclusions. To assist them, he assembled a set of cases and published them, with only a brief two-page introduction. . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
Inducing general principles from a small selection of cases was a challenging task, and students were unlikely to succeed without help. To guide them, Langdell developed through trial and error what is now called the Socratic Method: an interrogatory style in which instructors question students closely about the facts of the case, the points at issue, judicial reasoning, underlying doctrines and principles, and comparisons with other cases. Students prepare for class knowing that they will have to do more than simply parrot back material they have memorized from lectures or textbooks; they will have to present their own interpretations and analysis, and face detailed follow-up questions from the instructor.

Langdell's innovations initially met with enormous resistance. MANY STUDENTS WERE OUTRAGED (my CAPS). During the first three years of his administration, as word spread of Harvard's new approach to legal education, enrollment at the school dropped from 165 to 117 students, leading Boston University to start a law school of its own. Alumni were in open revolt.
GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG


Richard Hake, Emeritus Professor of Physics, Indiana University
24245 Hatteras Street, Woodland Hills, CA 91367
<rrhake@earthlink.net>
<http://www.physics.indiana.edu/~hake>
<http://www.physics.indiana.edu/~sdi>

"The academic area is one of the most difficult areas to change in our society. We continue to use the same methods of instruction, particularly lectures, that have been used for hundreds of years. Little scientific research is done to test new approaches, and little systematic attention is given to the development of new methods. Universities that study many aspects of the world ignore the educational function in which they are engaging and from which a large part of their revenues are earned.
Richard Cyert, former president of Carnegie Mellon University, in Tuma & Reif (1980)


REFERENCES [Tiny URL's courtesy <http://tinyurl.com/create.php>.]
Agarwal, A. 2003. "The Real Deal on 8.02 TEAL." The Tech 123(16) 4 April, online at
<http://www-tech.mit.edu/V123/N16/arun16.16c.html>.

LeBon, L.E. 2003. "Students Petition Against TEAL." The Tech 123 (14). 21 March; online at
<http://www-tech.mit.edu/V123/N14/14802T.14n.html>. For further student condemnation of TEAL see Agarwal (2003).

Belcher, J.W. 2007. "Re: Student Resistance to Teaching Improvement," PhysLrnR post of 22 Jan 2007 06:49:56-0500; online at <http://tinyurl.com/3459fu>.

Belcher, J.W. 2003. "Improving Student Understanding with TEAL," MIT Faculty Newsletter XVI (2), October/November; online at <http://web.mit.edu/jbelcher/www/TEALref/fnlEditedLinks.pdf>.

CTSE. 2007. "Teaching and Learning Centers in the United States," Center for Teaching and Scholarly Excellence (Hofstra University); online at http://www.hofstra.edu/faculty/ctse/cte_links.cfm. I thank Richard Lyons's listing of online resources <http://www.developfaculty.com/online/index.html > for this reference.
Dori, Y.J. & J. Belcher. 2004. "How Does Technology-Enabled Active Learning Affect Undergraduate Students' Understanding of Electromagnetism Concepts?" The Journal of the Learning Sciences 14(2), online as a 1 MB pdf at <http://tinyurl.com/cqoqt>.

Ehrmann, S.C., S.W. Gilbert, and F. McMartin. 2007. "Factors Affecting the Adoption of Faculty-Developed Academic Software: A Study of Five iCampus Projects," online at <http://www.tltgroup.org/icampus/iCampus_Assessment_Full.pdf> (2.1 MB); the executive summary and table of contents is online at <http://www.tltgroup.org/icampus/exec_sum_icampus_assessment.pdf> (208 kB).

Garvin, D.A. 2003. "Making the Case: Professional education for the world of practice" Harvard Magazine, September/October; online at <http://www.harvardmagazine.com/on-line/090322.html>. Thanks to MIT's Lori Breslow and John Belcher for bringing this article to my attention.

Hake, R.R. 2002. "Re: Problems with Student Evaluations: Is Assessment the Remedy?" online at <http://www.physics.indiana.edu/~hake/AssessTheRem1.pdf> (72 kB).

Hake, R.R. 2004. "Re: Student resistance to changes in professional education practice," online at <http://listserv.nd.edu/cgi-bin/wa?A2=ind0410&L=pod&P=R1366&I=-3>. Post of 1 Oct 2004 12:30:24-0700 to AERA-I, AERA-J, AP-Physics, ASSESS, Dr-Ed, EvalTalk, PBL, Phys-L, PhysLrnR, POD, and STLHE-L.

Hake, R.R. 2006. "Eleven Quotes in Honor of Inertia," POD post of 13 Jun 2006 15:01:14-0700, online at <http://listserv.nd.edu/cgi-bin/wa?A2=ind0606&L=pod&P=R7910&I=-3>.

Hake, R.R. 2007a. "The Myths of Innovation," (Part 1), online at
<http://listserv.nd.edu/cgi-bin/wa?A2=ind0705&L=pod&O=D&P=14074>. Post of 26 May to AERA-A, B, C, J, L; ASSESS, Chemed-L, EdResMeth, EvalTalk, IFETS, Phys-L, PhysLrnR, POD, STLHE-L, & TIPS.

Hake, R.R. 2007b. "The Myths of Innovation," (Part 2), online at
<http://listserv.nd.edu/cgi-bin/wa?A2=ind0705&L=pod&O=D&P=14654>. Post of 28 May 2007 20:26:31 -0700 to ASSESS, POD, PhysLrnR, Phys-L, and IFETS. Abstract only to AERA-A, B, C, J, L; Chemed-L, EdResMeth, EvalTalk, STLHE-L, & TIPS.

Hake, R.R. 2007c. "Should We Measure Change? Yes!" online as ref. 43 at <http://www.physics.indiana.edu/~hake>. To appear as a chapter in "Evaluation of Teaching and Student Learning in Higher Education," a Monograph of the American Evaluation Association <http://www.eval.org/>.

Henderson, C. and M. Dancy. 2006a. "Physics Faculty and Educational Researchers: Divergent Expectations as Barriers to the Diffusion of Innovations," submitted in April 2006 to Am. J. Phys. (Physics Education Research Section); online at <http://homepages.wmich.edu/~chenders/Publications/DivergentExpectationsSubmitted.pdf>
(224 KB).

Henderson, C. & M. Dancy. 2006b. "Barriers to the Use of Research-Based Instructional Strategies: The Dual Role of Individual and Situational Characteristics," submitted in October 2006 to Physical Review Special Topics: Physics Education Research; online at
<http://homepages.wmich.edu/~chenders/Publications/SituationalPaperSubmitted.pdf> (184 KB).

Heron, P.R.L. & D.E. Meltzer. 2005. "The future of physics education research: Intellectual challenges and practical concerns," Am. J. Phys. 73(5): 459-462; online at
<http://www.physicseducation.net/docs/Heron-Meltzer.pdf> (56 kB).

ISETL. 2007. International Society for Exploring Teaching and Learning, information online at <http://www.isetl.org/index.cfm>.

ISSOTL. 2007. International Society for Scholarship of Teaching and Learning, information online at <http://www.issotl.org/>,

Kariuki, J. 2007. "Re: The Myths of Innovation." IFETS post of 27 May 2007 16:16:15 +0200. IFETS = International Forum of Educational Technology & Society <http://ifets.ieee.org/>, endorsed by IEEE Technical Committee on Learning Technology <http://lttf.ieee.org/>. Evidently IFETS runs on a LITE version of LISTSERV software with NO archives of the standard LISTSERV type, as benefits, e.g. POD at <http://listserv.nd.edu/archives/pod.html>.

Klein, K.J. & A.P. Knight. 2005. "Innovation implementation: Overcoming the challenge," Current Directions in Psychological Science, 14(5): 243-246; freely online at <http://www.blackwell-synergy.com/toc/cdir/14/5>. The abstract reads: "In changing work environments, innovation is imperative. Yet, many teams and organizations fail to realize the expected benefits of innovations that they adopt. A key reason is not innovation failure but implementation failure-the failure to gain targeted employees' skilled, consistent, and committed use of the innovation in question. . . .[or in the case of education, to gain students' use of the innovation]. . . . We review research on the implementation process, outlining the reasons why implementation is so challenging for many teams and organizations. We then describe the organizational characteristics that together enhance the likelihood of successful implementation, including a strong, positive climate for implementation; management support for innovation implementation; financial resource availability; and a learning orientation."

Meltzer, D. 2007. "Links to United States Physics Education Research Groups," online at <http://www.physicseducation.net/links/index.html>.

Redish, E.F. 1999. "Millikan Award Lecture 1998: Building a Science of Teaching Physics," Am. J. Phys. 67(7): 562-573; online at <http://www.physics.umd.edu/perg/papers/redish/millikan.htm>.

Rhem, J. 2006. "The High Risks of Improving Teaching," National Teaching and Learning Forum 15(6), online to subscribers at <http://www.ntlf.com/FTPSite/issues/v15n6/risks.htm>. If your institution doesn't have a subscription, then, in my opinion, it should. Rhem's essay was also disseminated by Rick Reis's "Tomorrow's Professor" list as Msg.#760 at <http://ctl.stanford.edu/Tomprof/postings/760.html> of 20 Nov 2006.

Smith, L. 2007. "Re: Student Resistance to Teaching Improvement," PhysLrnR post of 20 Jan 2007 11:08:49-0700; online at <http://tinyurl.com/2s8moe>.

SOTL. 2007. Scholarship of Teaching and Learning, information online at <http://www.carnegiefoundation.org/programs/sub.asp?key=21&subkey=72&topkey=21>.

Stokstad, E. 2001. "Reintroducing the Intro Course." Science 293: 1608-1610, 31 August, abstract online at <http://www.sciencemag.org/cgi/content/summary/293/5535/1608>. "Physicists are out in front in measuring how well students learn the basics, as science educators incorporate hands-on activities in hopes of making the introductory course a beginning rather than a finale. Figuring out what works is vitally important to the country, say U.S. educators. Each year, hundreds of thousands of U.S. students get their only exposure to science in an intro class--and most leave without understanding how science works or with any desire to take further courses."
Thorn, P.M.2003). "Bridging the gap between what is praised and what is practiced: supporting the work of change as anatomy & physiology instructors introduce active learning into their undergraduate classroom. Doctoral dissertation, The University of Texas at Austin; online at <http://hdl.handle.net/2152/1013>.

Tuma, D.T. & F. Reif, eds. 1980. "Problem Solving and Education: Issues in Teaching and Research," Lawrence Erlbaum.