If you reply to this long (25 kB) post please don't hit the reply
button unless you prune the copy of this post that may appear in your
reply down to a few relevant lines, otherwise the entire already
archived post may be needlessly resent to subscribers.
***************************************************
ABSTRACT: I attempt to summarize (a) Steve Ehrmann's ASSESS post
regarding the Teaching, Learning, and Technology (TLT) Group's
program to furnish inexpensive strategies to help large numbers of
instructors make "low threshold" improvements in their teaching, and
(b) the valuable Ehrmann, Gilbert, McMartin report "Factors Affecting
the Adoption of Faculty-Developed Academic Software: A Study of Five
iCampus Projects" on innovation and change at MIT.
***************************************************
In response to my post "The Myths of Innovation" [Hake (2007a)],
Steve Ehrmann (2007) of the Teaching, Learning, and Technology (TLT)
Group <http://www.tltgroup.org>, in his ASSESS post of 27 May 2007
wrote [bracketed by lines "EEEEEEEE. . . . ."; my insert at ". . .
.[insert]. . . .]:
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
I agree that there are many, many factors affecting the spread of
innovation. We did a major study. . . . .["Factors Affecting the
Adoption of Faculty-Developed Academic Software: A Study of Five
iCampus Projects," Ehrmann, Gilbert, McMartin (2007)]. . . last year
for MIT on factors affecting whether faculty-developed
technology-based innovations in teaching got into long-term,
widespread use at MIT and at other institutions. . . . . The report
deals with just some of those factors
. . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
[Our investigation suggests that] few departments or institutions act
as though they expect tangible gains from improvements made in their
academic programs. . . . . If innovation (making it or importing it)
isn't profitable, are there any other motives that are powerful
enough to impel a program to take budget money away from other
functions in order to provide such support, reward, hiring? If the
answer to that question is 'no,' are there any other ways of
accomplishing continual, documented improvement?
At the moment, The TLT Group . . . . .is focusing most of its effort
here: inexpensive strategies to help large numbers of instructors
make "low threshold" improvements: improvements that are so easy to
adopt that people can do it within existing motivations and budgets.
. . . [see at <http://www.tltgroup.org/ltas.htm>]. . . . . The
typical image of innovation focuses on relatively expensive, high
visibility shifts. But, in the political economy of higher education,
that may be the wrong way to look.
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
An excerpt from the cogent report by Ehrmann, Gilbert, McMartin
(2007) follows [bracketed by lines EGM-EGM-EGM-EGM-. . . . ."; my
insert at ". . . .[insert]. . . ."]:
EGM-EGM-EGM-EGM-EGM-EGM-EGM-EGM-EGM
Initiated in 1999, iCampus . . . .[<http://icampus.mit.edu/>] . . . .
is a research collaboration between Microsoft Research and MIT whose
goal is to create and demonstrate technologies with the potential for
revolutionary change throughout the university curriculum. The
program was made possible by a $25 million research grant from
Microsoft to MIT and involves extensive collaboration between MIT and
Microsoft staff.
The TLT Group has been asked, "In light of the experience of iCampus,
especially those projects selected by MIT and Microsoft for close
study, what can be learned about priorities for educational
technology initiatives in the future and about how the spread of such
innovations can be more effectively supported?"
In the past, many large-scale faculty-developed content-specific
projects had had great success as pilot tests, but had failed to be
widely used. So The TLT Group and iCampus decided to focus this study
on five quite different projects that had already achieved some
degree of institutionalization and wider use. Over 150 interviews
were conducted with faculty members, staff and students at MIT and
other institutions, and project documents were studied. The five
projects are:
1. iLabs. . . .[ <http://icampus.mit.edu/ilabs/>]. . . . - students
can use web browsers to design experiments and collect data from
distant laboratory equipment;
2. iMOAT . . . .[ <http://icampus.mit.edu/iMOAT/>]. . . . - the web
is used to manage the process of large-scale assessment of student
writing;
3. TEAL . . . . .[ <http://icampus.mit.edu/teal/>]. . . . - two
terms of introductory physics have been redesigned around inquiry,
discussion, experimentation, and visualization . . . .[see e.g. "How
Does Technology-Enabled Active Learning Affect Undergraduate
Students' Understanding of Electromagnetism Concepts?" (Dori &
Belcher, 2004; Brehm, 2001; Belcher, 2001)];
4. XMAS - [<http://icampus.mit.edu/xmas/>]. . . .students can 'quote'
video legally in their online discussions, presentations, and
projects about films in courses such as Shakespeare;
5. xTutor. . . .[ <http://icampus.mit.edu/XTutor/>] . . . .is to be a
tool kit for creating online courses; its strength is checking
computer programming homework and providing feedback.
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
Chapter 5: TEAL: A CAMPAIGN TO IMPROVE CONCEPTUAL LEARNING IN FIRST
YEAR PHYSICS AT MIT
There had long been dissatisfaction with freshman physics instruction
at MIT. All majors at MIT, even in social sciences, management and
humanities, are required to take two terms of physics. Various
designs and experiments had been tried over the years to improve
these physics courses, occasionally with two or even three variations
of the course being taught simultaneously.
Meanwhile, since around 1980, change was in the wind nationally. For
over two decades, physics education research had been yielding
convincing evidence that a) students even at highly selective
institutions such as Harvard and MIT were failing to master some of
the most basic ideas in introductory physics, even when lectures and
problem sessions were accompanied by traditional laboratory
experiments. In the mid 1990's, a PBS television documentary, "Minds
of
Our Own," . . . . .[
<http://www.learner.org/resources/series26.html>]. . . . drew on this
research to dramatize problems in American education. Time and again
these programs showed graduating seniors from MIT (and Harvard) still
in their caps and gowns, giving ridiculously wrong answers about
concepts in physics and other sciences that had been taught
repeatedly in school and at MIT.
PERHAPS THE SINGLE MOST IMPORTANT DEVELOPMENT IN THIS CAMPAIGN TO
UNDERSTAND AND IMPROVE LEARNING IN PHYSICS WAS THE DEVELOPMENT OF THE
FORCE CONCEPT INVENTORY [my CAPS] by David Hestenes and his
colleagues. . . . . .[Halloun & Hestenes (1985a,b); Hestenes et al.
(1992)]. . . . This multiple choice test of conceptual understanding
was carefully designed to reward conceptual understanding and to
reveal the most common misconceptions. Many faculty members, from
selective institutions to community colleges, were shocked by their
students' poor performance on an exam that, beforehand, these faculty
members had assumed most of their students would pass. This lack of
understanding had previously been hidden from both faculty members
and students by the students' ability to apply mathematics in
routinized ways to routine problems found in problem sets, quizzes
and exams.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The good news: this same stream of physics education R&D was
demonstrating that nontraditional approaches to teaching could yield
better understanding of those fundamental concepts. . . .[For a
recent reviews see Heron & Meltzer (2006) and Hake (2007b)]. . . . .
The National Science Foundation and private foundations invested for
years in the search for better approaches to physics education. That
funding, and the resulting stream of encouraging findings and new
program designs, helped leaders in the physics education movement
gain in prestige. Their work began to draw favorable notice in
publications from the National Academy of Sciences. . . .[see e.g.,
Fox & Hackerman (2003), McCray et al. (2003), Donovan & Pellegrino
(2003)]. . . . . . and other authoritative groups. Investment in
research was leading to documented improvements in learning. Some
departments (though relatively few in institutions considered to be
first rank research institutions) began to include a physics
education specialist among their physics faculty. That is becoming
more common, and more noticeable in the wider world of physics.
Recently physics research was rocked by the news that Carl Wieman, a
relatively recent and still young Nobel Laureate in Physics (2001) .
. . . . .
[<http://nobelprize.org/nobel_prizes/physics/laureates/2001/>]. . . .
, had moved to the University of British Columbia, where he was
shifting his research completely to physics education. . . [see e.g.
"Transforming Physics Education" [Wieman & Perkins (2005)] and
"Trading Research for Teaching" Epstein (2006).
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
John Belcher. . . . .
[<http://web.mit.edu/physics/facultyandstaff/faculty/john_belcher.html>].
. . . . . . ., a professor of astrophysics without much previous
background in teaching undergraduates, lectured second term physics
(8.02, including electricity and magnetism, relativity, and related
topics) for three years in the early 1990s. Although he received
extremely good ratings from students by the third year, Belcher, too,
was dissatisfied because of poor
attendance, a failure rate of 10-15%, and a sense that he was not
reaching students. His response after the first year was to take
acting classes, so that he could project better and make a better
impression, and thereby increase attendance and understanding. That
did not work.
So Belcher attended a conference of the American Association of
Physics Teachers (AAPT). He learned that, for at least a decade,
pioneers in physics education research had already been tackling
these problems, and with some success.
Electricity and magnetism are among the topics in science that are
most difficult to teach effectively. Students whose skills of
reasoning and visualization have helped them succeed in the first
term of physics often have problems with the complex and abstract
visualization needed to master E&M.
Belcher was interested in applying his own ideas about making
electromagnetic fields more visible and comprehensible to
undergraduates. He wanted to do this by giving them simulations of
fields, simulations that they could examine closely and manipulate.
Belcher submitted a major proposal to the US National Science
Foundation, to develop computer simulations of electromagnetic
radiation for use in 8.02. In a move that would have profound
implications for physics instruction at MIT, NSF turned down
Belcher's proposal but offered him a planning grant on the condition
that Belcher team up with an expert in physics education research. So
Belcher joined the physics education research community. His chosen
expert was Bob Beichner. . . . .[
<http://www2.ncsu.edu/ncsu/pams/physics/People/beichner.html>]. . . .
of North Carolina State, who was already at work developing SCALE-UP
. . . . .[see, e.g., <http://www.ncsu.edu/per/scaleup.html> and
Beichner & Saul (2006) ]. . . .
NC State's SCALE-UP stood on the shoulders of earlier programs such
as Priscilla Laws's development of workshop physics at Dickinson
College . . . .[see, e.g., Laws (1996)]. . . ., Rensselaer
Polytechnic Institute's studio physics . . . .[see, e.g., Wilson
(1994), Cummings et al. (1999)]. . . . , and an earlier program at NC
State called IMPEC . . . .[see, e.g., Beichner et al. (1999)]. . . .
.. IMPEC was an integrated approach to freshman engineering for
which Beichner. . . .[and his colleagues]. . . . had developed
physics instruction. SCALE-UP was a departure from all of these in
its goal of handling sections of 100 students at a time. Earlier
efforts were designed for sections of 20-70 students, which meant
that their laboratory centered approaches could still have relatively
high costs per student. To reduce costs per students to acceptable
levels, SCALE-UP developed an approach that could support large
sections that were nonetheless focused on experimentation, debate,
and problem-solving. To perfect its approach, the SCALE-UP team
systematically tried solution after solution, taking data, and
adjusting its approach (e.g., on the design of worktables). Belcher
would follow many of SCALE-Ups procedures for organizing large
sections of students to do active learning, while using a different
textbook, problems, and new simulation tools for visualizing
electromagnetic fields.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Students in the scale-up of TEAL in spring 2003 . . . . .[achieved
average normalized gains <g> = 0.52)]. . . . . far better than the
lecture group . . . . [<g> = 0.27]. . . .This doubling of . . .
.[normalized]. . . gains from traditional methods is also typical in
other universities when IE . . . .[Interactive Engagement]. . . .
methods are compared with more traditional lectures and laboratories.
. . . .[Here the average normalized gain <g> is the actual average
gain (<%posttest> - <%pretest>) divided by the maximum possible
average gain (100% - <%pretest>)]. . . . . .
EGM-EGM-EGM-EGM-EGM-EGM-EGM-EGM-EGM
REFERENCES
Beichner, R., L. Bernold, E. Burniston, P. Dail, R. Felder, J.
Gastineau, M. Gjertsen, and J. Risley. 1999. "Case study of the
physics component of an integrated curriculum" Am; J. Phys. 67(7):
S16-S24; online at
<http://scitation.aip.org/dbt/dbt.jsp?KEY=AJPIAS&Volume=67&Issue=S1>.
Beichner, R.J & J.M. Saul. 2004. "Introduction to the SCALE-UP
(Student-Centered Activities for Large Enrollment Undergraduate
Programs) Project," in "Proceedings of the International School of
Physics 'Enrico Fermi' Course CLVI in Varenna, Italy, M. Vicentini
and E.F. Redish, eds. IOS Press; online at
<http://www.ncsu.edu/per/Articles/Varenna_SCALEUP_Paper.pdf> (1MB).
Donovan, M.S. & J. Pellegrino, eds. 2003. Learning and Instruction: A
SERP Research Agenda, National Academies Press; online at
<http://books.nap.edu/catalog/10858.html>.
Dori, Y.J. & J. Belcher. 2004. "How Does Technology-Enabled Active
Learning Affect
Undergraduate Students' Understanding of Electromagnetism Concepts?"
The Journal of the
Learning Sciences 14(2), online as a 1 MB pdf at <http://tinyurl.com/cqoqt>.
Ehrmann, S. 2007. "Re: The Myths of Innovation," ASSESS post of 27
May 2007 09:20:26-0700; online at <http://tinyurl.com/yusw25>.
Ehrmann, S.C., S.W. Gilbert, and F. McMartin. 2007. "Factors
Affecting the Adoption of Faculty-Developed Academic Software: A
Study of Five iCampus Projects," online at
<http://www.tltgroup.org/icampus/iCampus_Assessment_Full.pdf> (2.1
MB); the executive summary and table of contents is online at
<http://www.tltgroup.org/icampus/iCampus_Assessment_Full.pdf> (208
kB). See also "Why do great teaching ideas spread so slowly? How can
we speed that up?" [TLT-SWG Blog. 2007].
Fox, M.A., & N. Hackerman, eds. 2003. National Research Council,
Committee on Undergraduate Science Education, National Academy Press;
online at <http://www.nap.edu/catalog/10024.html>.
Hake, R.R. 2007b. "Should We Measure Change? Yes!" online as ref. 43
at at <http://www.physics.indiana.edu/~hake>. To appear as a chapter
in "Evaluation of Teaching and Student Learning in Higher Education,"
a Monograph of the American Evaluation Association
<http://www.eval.org/>.
Halloun, I. & D. Hestenes. 1985a. "The initial knowledge state of
college physics students." Am. J. Phys. 53:1043-1055; online at
<http://modeling.asu.edu/R&E/Research.html>. Contains the "Mechanics
Diagnostic" test, precursor to the "Force Concept Inventory"
[Hestenes et al. (1992)].
Heron, P.R.L. & D.E. Meltzer. 2005. "The future of physics education
research: Intellectual challenges and practical concerns," Am. J.
Phys. 73(5): 459-462; online at
<http://www.physicseducation.net/docs/Heron-Meltzer.pdf> (56 kB).
Hestenes, D., M. Wells, & G. Swackhamer, 1992. "Force Concept
Inventory," Phys. Teach. 30: 141-158; online (except for the test
itself) at <http://modeling.asu.edu/R&E/Research.html>. The 1995
revision by Halloun, Hake, Mosca, & Hestenes is online (password
protected) at the same URL, and is available in English, Spanish,
German, Malaysian, Chinese, Finnish, French, Turkish, Swedish, and
Russian.
McCray, R.A., R.L. DeHaan, J.A. Schuck, eds. 2003. "Improving
Undergraduate Instruction in Science, Technology, Engineering, and
Mathematics: Report of a Workshop," Committee on Undergraduate STEM
Instruction, National Research Council, National Academy Press;
online at <http://www.nap.edu/catalog/10711.html>.
TLT-SWG Blog. 2007. "Why do great teaching ideas spread so slowly?
How can we speed that up?" online at
<http://tlt-swg.blogspot.com/2007/01/why-is-it-so-hard-to-disseminate-great.html>:
The biggest barrier: typical faculty members get little preparation,
little help and little reward for continually updating and improving
all their courses. The best way to improve learning is not to invent
your own ideas from scratch. It's to scan the world for successful
ideas already tried out by like-minded colleagues teaching comparable
courses. And if faculty were better prepared, supported, and rewarded
they could do that."