Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

Re: [Phys-l] The Myths of Innovation



If you reply to this long (25 kB) post please don't hit the reply button unless you prune the copy of this post that may appear in your reply down to a few relevant lines, otherwise the entire already archived post may be needlessly resent to subscribers.

***************************************************
ABSTRACT: I attempt to summarize (a) Steve Ehrmann's ASSESS post regarding the Teaching, Learning, and Technology (TLT) Group's program to furnish inexpensive strategies to help large numbers of instructors make "low threshold" improvements in their teaching, and (b) the valuable Ehrmann, Gilbert, McMartin report "Factors Affecting the Adoption of Faculty-Developed Academic Software: A Study of Five iCampus Projects" on innovation and change at MIT.
***************************************************

In response to my post "The Myths of Innovation" [Hake (2007a)], Steve Ehrmann (2007) of the Teaching, Learning, and Technology (TLT) Group <http://www.tltgroup.org>, in his ASSESS post of 27 May 2007 wrote [bracketed by lines "EEEEEEEE. . . . ."; my insert at ". . . .[insert]. . . .]:

EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
I agree that there are many, many factors affecting the spread of innovation. We did a major study. . . . .["Factors Affecting the Adoption of Faculty-Developed Academic Software: A Study of Five iCampus Projects," Ehrmann, Gilbert, McMartin (2007)]. . . last year for MIT on factors affecting whether faculty-developed technology-based innovations in teaching got into long-term, widespread use at MIT and at other institutions. . . . . The report deals with just some of those factors
. . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
[Our investigation suggests that] few departments or institutions act as though they expect tangible gains from improvements made in their academic programs. . . . . If innovation (making it or importing it) isn't profitable, are there any other motives that are powerful enough to impel a program to take budget money away from other functions in order to provide such support, reward, hiring? If the answer to that question is 'no,' are there any other ways of accomplishing continual, documented improvement?

At the moment, The TLT Group . . . . .is focusing most of its effort here: inexpensive strategies to help large numbers of instructors make "low threshold" improvements: improvements that are so easy to adopt that people can do it within existing motivations and budgets. . . . [see at <http://www.tltgroup.org/ltas.htm>]. . . . . The typical image of innovation focuses on relatively expensive, high visibility shifts. But, in the political economy of higher education, that may be the wrong way to look.
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE

An excerpt from the cogent report by Ehrmann, Gilbert, McMartin (2007) follows [bracketed by lines EGM-EGM-EGM-EGM-. . . . ."; my insert at ". . . .[insert]. . . ."]:

EGM-EGM-EGM-EGM-EGM-EGM-EGM-EGM-EGM
Initiated in 1999, iCampus . . . .[<http://icampus.mit.edu/>] . . . . is a research collaboration between Microsoft Research and MIT whose goal is to create and demonstrate technologies with the potential for revolutionary change throughout the university curriculum. The program was made possible by a $25 million research grant from Microsoft to MIT and involves extensive collaboration between MIT and Microsoft staff.

The TLT Group has been asked, "In light of the experience of iCampus, especially those projects selected by MIT and Microsoft for close study, what can be learned about priorities for educational technology initiatives in the future and about how the spread of such innovations can be more effectively supported?"

In the past, many large-scale faculty-developed content-specific projects had had great success as pilot tests, but had failed to be widely used. So The TLT Group and iCampus decided to focus this study on five quite different projects that had already achieved some degree of institutionalization and wider use. Over 150 interviews were conducted with faculty members, staff and students at MIT and other institutions, and project documents were studied. The five
projects are:

1. iLabs. . . .[ <http://icampus.mit.edu/ilabs/>]. . . . - students can use web browsers to design experiments and collect data from distant laboratory equipment;

2. iMOAT . . . .[ <http://icampus.mit.edu/iMOAT/>]. . . . - the web is used to manage the process of large-scale assessment of student writing;

3. TEAL . . . . .[ <http://icampus.mit.edu/teal/>]. . . . - two terms of introductory physics have been redesigned around inquiry, discussion, experimentation, and visualization . . . .[see e.g. "How Does Technology-Enabled Active Learning Affect Undergraduate Students' Understanding of Electromagnetism Concepts?" (Dori & Belcher, 2004; Brehm, 2001; Belcher, 2001)];

4. XMAS - [<http://icampus.mit.edu/xmas/>]. . . .students can 'quote' video legally in their online discussions, presentations, and projects about films in courses such as Shakespeare;

5. xTutor. . . .[ <http://icampus.mit.edu/XTutor/>] . . . .is to be a tool kit for creating online courses; its strength is checking computer programming homework and providing feedback.
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
Chapter 5: TEAL: A CAMPAIGN TO IMPROVE CONCEPTUAL LEARNING IN FIRST YEAR PHYSICS AT MIT

There had long been dissatisfaction with freshman physics instruction at MIT. All majors at MIT, even in social sciences, management and humanities, are required to take two terms of physics. Various designs and experiments had been tried over the years to improve these physics courses, occasionally with two or even three variations of the course being taught simultaneously.

Meanwhile, since around 1980, change was in the wind nationally. For over two decades, physics education research had been yielding convincing evidence that a) students even at highly selective institutions such as Harvard and MIT were failing to master some of the most basic ideas in introductory physics, even when lectures and problem sessions were accompanied by traditional laboratory experiments. In the mid 1990's, a PBS television documentary, "Minds of
Our Own," . . . . .[ <http://www.learner.org/resources/series26.html>]. . . . drew on this research to dramatize problems in American education. Time and again these programs showed graduating seniors from MIT (and Harvard) still in their caps and gowns, giving ridiculously wrong answers about concepts in physics and other sciences that had been taught repeatedly in school and at MIT.

PERHAPS THE SINGLE MOST IMPORTANT DEVELOPMENT IN THIS CAMPAIGN TO UNDERSTAND AND IMPROVE LEARNING IN PHYSICS WAS THE DEVELOPMENT OF THE FORCE CONCEPT INVENTORY [my CAPS] by David Hestenes and his colleagues. . . . . .[Halloun & Hestenes (1985a,b); Hestenes et al. (1992)]. . . . This multiple choice test of conceptual understanding was carefully designed to reward conceptual understanding and to reveal the most common misconceptions. Many faculty members, from selective institutions to community colleges, were shocked by their students' poor performance on an exam that, beforehand, these faculty members had assumed most of their students would pass. This lack of understanding had previously been hidden from both faculty members and students by the students' ability to apply mathematics in routinized ways to routine problems found in problem sets, quizzes and exams.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The good news: this same stream of physics education R&D was demonstrating that nontraditional approaches to teaching could yield better understanding of those fundamental concepts. . . .[For a recent reviews see Heron & Meltzer (2006) and Hake (2007b)]. . . . .

The National Science Foundation and private foundations invested for years in the search for better approaches to physics education. That funding, and the resulting stream of encouraging findings and new program designs, helped leaders in the physics education movement gain in prestige. Their work began to draw favorable notice in publications from the National Academy of Sciences. . . .[see e.g., Fox & Hackerman (2003), McCray et al. (2003), Donovan & Pellegrino (2003)]. . . . . . and other authoritative groups. Investment in research was leading to documented improvements in learning. Some departments (though relatively few in institutions considered to be first rank research institutions) began to include a physics education specialist among their physics faculty. That is becoming more common, and more noticeable in the wider world of physics. Recently physics research was rocked by the news that Carl Wieman, a relatively recent and still young Nobel Laureate in Physics (2001) . . . . . . [<http://nobelprize.org/nobel_prizes/physics/laureates/2001/>]. . . . , had moved to the University of British Columbia, where he was shifting his research completely to physics education. . . [see e.g. "Transforming Physics Education" [Wieman & Perkins (2005)] and "Trading Research for Teaching" Epstein (2006).
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
John Belcher. . . . . [<http://web.mit.edu/physics/facultyandstaff/faculty/john_belcher.html>]. . . . . . . ., a professor of astrophysics without much previous background in teaching undergraduates, lectured second term physics (8.02, including electricity and magnetism, relativity, and related topics) for three years in the early 1990s. Although he received extremely good ratings from students by the third year, Belcher, too, was dissatisfied because of poor
attendance, a failure rate of 10-15%, and a sense that he was not reaching students. His response after the first year was to take acting classes, so that he could project better and make a better impression, and thereby increase attendance and understanding. That did not work.

So Belcher attended a conference of the American Association of Physics Teachers (AAPT). He learned that, for at least a decade, pioneers in physics education research had already been tackling these problems, and with some success.

Electricity and magnetism are among the topics in science that are most difficult to teach effectively. Students whose skills of reasoning and visualization have helped them succeed in the first term of physics often have problems with the complex and abstract visualization needed to master E&M.

Belcher was interested in applying his own ideas about making electromagnetic fields more visible and comprehensible to undergraduates. He wanted to do this by giving them simulations of fields, simulations that they could examine closely and manipulate.

Belcher submitted a major proposal to the US National Science Foundation, to develop computer simulations of electromagnetic radiation for use in 8.02. In a move that would have profound implications for physics instruction at MIT, NSF turned down Belcher's proposal but offered him a planning grant on the condition that Belcher team up with an expert in physics education research. So Belcher joined the physics education research community. His chosen expert was Bob Beichner. . . . .[ <http://www2.ncsu.edu/ncsu/pams/physics/People/beichner.html>]. . . . of North Carolina State, who was already at work developing SCALE-UP . . . . .[see, e.g., <http://www.ncsu.edu/per/scaleup.html> and Beichner & Saul (2006) ]. . . .

NC State's SCALE-UP stood on the shoulders of earlier programs such as Priscilla Laws's development of workshop physics at Dickinson College . . . .[see, e.g., Laws (1996)]. . . ., Rensselaer Polytechnic Institute's studio physics . . . .[see, e.g., Wilson (1994), Cummings et al. (1999)]. . . . , and an earlier program at NC State called IMPEC . . . .[see, e.g., Beichner et al. (1999)]. . . . .. IMPEC was an integrated approach to freshman engineering for which Beichner. . . .[and his colleagues]. . . . had developed physics instruction. SCALE-UP was a departure from all of these in its goal of handling sections of 100 students at a time. Earlier efforts were designed for sections of 20-70 students, which meant that their laboratory centered approaches could still have relatively high costs per student. To reduce costs per students to acceptable levels, SCALE-UP developed an approach that could support large sections that were nonetheless focused on experimentation, debate, and problem-solving. To perfect its approach, the SCALE-UP team systematically tried solution after solution, taking data, and adjusting its approach (e.g., on the design of worktables). Belcher would follow many of SCALE-Ups procedures for organizing large sections of students to do active learning, while using a different textbook, problems, and new simulation tools for visualizing electromagnetic fields.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Students in the scale-up of TEAL in spring 2003 . . . . .[achieved average normalized gains <g> = 0.52)]. . . . . far better than the lecture group . . . . [<g> = 0.27]. . . .This doubling of . . . .[normalized]. . . gains from traditional methods is also typical in other universities when IE . . . .[Interactive Engagement]. . . . methods are compared with more traditional lectures and laboratories. . . . .[Here the average normalized gain <g> is the actual average gain (<%posttest> - <%pretest>) divided by the maximum possible average gain (100% - <%pretest>)]. . . . . .
EGM-EGM-EGM-EGM-EGM-EGM-EGM-EGM-EGM


Richard Hake, Emeritus Professor of Physics, Indiana University
24245 Hatteras Street, Woodland Hills, CA 91367
<rrhake@earthlink.net>
<http://www.physics.indiana.edu/~hake>
<http://www.physics.indiana.edu/~sdi>

REFERENCES
Beichner, R., L. Bernold, E. Burniston, P. Dail, R. Felder, J. Gastineau, M. Gjertsen, and J. Risley. 1999. "Case study of the physics component of an integrated curriculum" Am; J. Phys. 67(7): S16-S24; online at <http://scitation.aip.org/dbt/dbt.jsp?KEY=AJPIAS&Volume=67&Issue=S1>.

Beichner, R.J & J.M. Saul. 2004. "Introduction to the SCALE-UP (Student-Centered Activities for Large Enrollment Undergraduate Programs) Project," in "Proceedings of the International School of Physics 'Enrico Fermi' Course CLVI in Varenna, Italy, M. Vicentini and E.F. Redish, eds. IOS Press; online at
<http://www.ncsu.edu/per/Articles/Varenna_SCALEUP_Paper.pdf> (1MB).

Belcher, J.W. 2001. "Studio Physics at MIT," online at <http://web.mit.edu/physics/facultyandstaff/faculty_documents/belcher_p@m_fall_01.pdf> (384 kB).

Brehm, D. 2001. "First-year physics course being transformed through experiment," MIT News Office, Dec. 19, Online at <http://web.mit.edu/newsoffice/2001/physics-1219.html>.

Cummings, K., J. Marx, R. Thornton, D. Kuhl. 1999. Evaluating innovations in studio physics. Physics Ed. Res. Supplement to Am. J. Phys. 67(7): S38-S44; online at <http://scitation.aip.org/dbt/dbt.jsp?KEY=AJPIAS&Volume=67&Issue=S1>.

Donovan, M.S. & J. Pellegrino, eds. 2003. Learning and Instruction: A SERP Research Agenda, National Academies Press; online at <http://books.nap.edu/catalog/10858.html>.
Dori, Y.J. & J. Belcher. 2004. "How Does Technology-Enabled Active Learning Affect
Undergraduate Students' Understanding of Electromagnetism Concepts?" The Journal of the
Learning Sciences 14(2), online as a 1 MB pdf at <http://tinyurl.com/cqoqt>.

Ehrmann, S. 2007. "Re: The Myths of Innovation," ASSESS post of 27 May 2007 09:20:26-0700; online at <http://tinyurl.com/yusw25>.

Ehrmann, S.C., S.W. Gilbert, and F. McMartin. 2007. "Factors Affecting the Adoption of Faculty-Developed Academic Software: A Study of Five iCampus Projects," online at <http://www.tltgroup.org/icampus/iCampus_Assessment_Full.pdf> (2.1 MB); the executive summary and table of contents is online at <http://www.tltgroup.org/icampus/iCampus_Assessment_Full.pdf> (208 kB). See also "Why do great teaching ideas spread so slowly? How can we speed that up?" [TLT-SWG Blog. 2007].

Epstein, D. 2006. "Trading Research for Teaching," Inside Higher Ed, 7 April, online at <http://insidehighered.com/news/2006/04/07/wieman>.

Fox, M.A., & N. Hackerman, eds. 2003. National Research Council, Committee on Undergraduate Science Education, National Academy Press; online at <http://www.nap.edu/catalog/10024.html>.

Hake, R.R. 2007a. "The Myths of Innovation," online at
<http://listserv.nd.edu/cgi-bin/wa?A2=ind0705&L=pod&O=D&P=14074>. Post of 26 May to AERA-A, B, C, J, L; ASSESS, Chemed-L, EdResMeth, EvalTalk, IFETS, Phys-L, PhysLrnR, POD, STLHE-L, & TIPS.

Hake, R.R. 2007b. "Should We Measure Change? Yes!" online as ref. 43 at at <http://www.physics.indiana.edu/~hake>. To appear as a chapter in "Evaluation of Teaching and Student Learning in Higher Education," a Monograph of the American Evaluation Association <http://www.eval.org/>.

Halloun, I. & D. Hestenes. 1985a. "The initial knowledge state of college physics students." Am. J. Phys. 53:1043-1055; online at <http://modeling.asu.edu/R&E/Research.html>. Contains the "Mechanics Diagnostic" test, precursor to the "Force Concept Inventory" [Hestenes et al. (1992)].

Halloun, I. & D. Hestenes. 1985b. "Common sense concepts about motion." Am. J. Phys. 53:1056-1065; online at <http://modeling.asu.edu/R&E/Research.html>.

Heron, P.R.L. & D.E. Meltzer. 2005. "The future of physics education research: Intellectual challenges and practical concerns," Am. J. Phys. 73(5): 459-462; online at
<http://www.physicseducation.net/docs/Heron-Meltzer.pdf> (56 kB).

Hestenes, D., M. Wells, & G. Swackhamer, 1992. "Force Concept Inventory," Phys. Teach. 30: 141-158; online (except for the test itself) at <http://modeling.asu.edu/R&E/Research.html>. The 1995 revision by Halloun, Hake, Mosca, & Hestenes is online (password protected) at the same URL, and is available in English, Spanish, German, Malaysian, Chinese, Finnish, French, Turkish, Swedish, and Russian.

Laws. P. 1997. "Millikan Lecture 1996: Promoting active learning based on physics education research in introductory physics courses," Am. J. Phys., 65 (1): 13-21; online at <http://scitation.aip.org/dbt/dbt.jsp?KEY=AJPIAS&Volume=65&Issue=1>.

McCray, R.A., R.L. DeHaan, J.A. Schuck, eds. 2003. "Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop," Committee on Undergraduate STEM Instruction, National Research Council, National Academy Press; online at <http://www.nap.edu/catalog/10711.html>.

TLT-SWG Blog. 2007. "Why do great teaching ideas spread so slowly? How can we speed that up?" online at
<http://tlt-swg.blogspot.com/2007/01/why-is-it-so-hard-to-disseminate-great.html>: The biggest barrier: typical faculty members get little preparation, little help and little reward for continually updating and improving all their courses. The best way to improve learning is not to invent your own ideas from scratch. It's to scan the world for successful ideas already tried out by like-minded colleagues teaching comparable courses. And if faculty were better prepared, supported, and rewarded they could do that."

Wieman, C. & K. Perkins. 2005. "Transforming Physics Education," Phys. Today 58(11): 36-41; online at <http://www.colorado.edu/physics/EducationIssues/papers/PhysicsTodayFinal.pdf> (292 kB).

Wilson, J.M. 1994. "The CUPLE Physics Studio," Phys. Teach. 32(9): 518-523 (1994); online at
<http://scitation.aip.org/dbt/dbt.jsp?KEY=PHTEAH&Volume=32&Issue=9>.