******************************************
ABSTRACT: Mike La Lopa, in his POD post Re: SoTL (Scholarship of
Teaching and Learning) and Finance, suggests that business faculty
need information that will show that their teaching is ineffective
before changing their teaching methods. I think that the same can be
said of nearly ALL faculty, and that a convincing indicator of
teaching ineffectiveness is relatively low pre-to-post test gain on
valid and consistently reliable tests of conceptual understanding
developed by disciplinary experts. But are business (or any other)
faculty willing to undertake the arduous qualitative and quantitative
research required for such development?
******************************************
If you reply to this long post (19 kB - Sorry, it's all the fault of
Mick La Lopa) please don't hit the reply button unless you prune the
copy of this post that may appear in your reply down to a few
relevant lines, otherwise the entire already archived post may be
needlessly resent to subscribers.
Mick La Lopa, in his POD post of 11 Jun 2007 titled "Re: SoTL and
Finance" wrote [my inserts at ". . . . .[insert]. . . ."]:
"The. . . . .[business]. . . school folks are most likely Concrete
Sequentials (Thinkers) or Abstract Sequentials (Directors). . . .
[see, e.g., Gregorc (2006)]. . . You are correct that in both cases
they will need information to move them along especially the kind
that shows them that their teaching methods may be inadequate,
illogical, inefficient, ineffective, or inferior compared to others.
Once they figure that out for themselves -- you cannot do it for them
-- they will discover this great idea they had called SoTL . . .
.[(2007)]. . . once you are gone. Perhaps give them the SoTL
bibliography found at:
<http://www.carnegiefoundation.org/dynamic/downloads/file_1_196.pdf>."
In my opinion nearly *all* faculty (be they Concrete, Abstract, or
Incon Sequentials) need to find out that their teaching methods are
"inadequate, illogical, inefficient, ineffective, or inferior
compared to others," or compared to what might reasonably be
expected, before they will exert the effort to change their teaching
methods.
But how will they find out? Probably not from course exams, peer
assessment, student teaching evaluations, or even by studying the
citations in "The Scholarship of Teaching and Learning in Higher
Education: An Annotated Bibliography" [Hutchings et al. (2002)]. The
latter appear to contain nothing related to the *direct* measurement
of course effectiveness by pre/post testing, except indirectly though
the reference to Nelson (2000).
Lesson #3 of the generally ignored "Lessons of the Physics Education
Reform Effort" [Hake (2002)] is:
**High-Quality Standardized Tests of the Cognitive and Affective
Impact of Courses Are Essential to Gauge the Relative Effectiveness
of Non-Traditional Educational Methods.**
So great is the inertia of the educational establishment [see e.g.,
"Eleven Quotes in Honor of Inertia" (Hake, 2006c)] that three decades
of physics education research [see, e.g., McDermott & Redish (1999)]
demonstrating the futility of the passive-student lecture in
introductory courses was ignored until high-quality standardized
tests, that could be easily administered to thousands of students,
became available.
As a case in point: in sharp contrast to most faculty, Harvard's Eric
Mazur was aware of the Mechanics Diagnostic test of Halloun &
Hestenes (1998a,b), precursor to the widely employed Force Concept
Inventory [Hestenes et al. (1992)]. Mazur (1997, p. 4) wrote:
"When reading this. . . . .[the Mechanics Diagnostic test]. . . my
first reaction was 'Not my students. . .!' Intrigued, I decided to
test my own students' conceptual understanding, as well as that of
physics majors at Harvard. . . . . the results of the test came as a
shock: The students faired hardly better on the Halloun and Hestenes
test than on their midterm exam. Yet the Halloun and Hestenes test
is SIMPLE, whereas the material covered by the examination
(rotational dynamics, moments of inertia) if of far greater
difficulty, or so I thought."
Convinced by the pre/post test results that traditional methods of
instruction had little, if any, impact on students' conceptual
understanding, Mazur and his colleagues went on to develop and use
much more effective pedagogical methods, see. e.g., Crouch & Mazur
(2000). The same can be said for many other astronomy, economics,
biology, chemistry, geoscience, engineering, mathematics, and physics
faculty, as reviewed in Hake (2002; 2005; 2006a,b; 2007).
But are any business faculty willing to undertake the arduous
qualitative and quantitative research required to develop a valid and
consistently reliable BUCI (BUsiness Concept Inventory) counterpart
of the BCI [Biology Concept Inventory (Klymkowsky et al., 2003;
Klymkowsky, 2007)] or FCI [Force Concept Inventory (Hestenes et al.,
1992)]?
For that matter are *any* faculty (other than a few in math and
science) willing to do so?
Crouch, C.H. & E. Mazur. 2001. "Peer Instruction: Ten years of
experience and results," Am. J. Phys. 69: 970-977; online at
<http://tinyurl.com/sbys4>.
Flannery, M. 2001. "The Mind-Body Problem," American Biology Teacher
62(8): 610-615; online to subscribers at <http://tinyurl.com/yuus7z>.
Gregorc, A.F. 2006. "More Info on
Abstract/Random/Concrete/Sequential, Links, & some very helpful
Tables," online at
<http://www.floatingneutrinos.com/Message/arcs/links_on_abstractrandom.htm>.
Gregorc gives learning preferences for each of 4 "Mind Styles":
Concrete Sequentials, Abstract Sequentials, Abstract Randoms, and
Concrete Randoms.
Hake, R.R. 2002. "Lessons from the physics education reform effort,"
Ecology and Society 5(2): 28; online at
<http://www.ecologyandsociety.org/vol5/iss2/art28/>. Ecology and
Society (formerly Conservation Ecology) is a free online
"peer-reviewed journal of integrative science and fundamental policy
research" with about 11,000 subscribers in about 108 countries.
Hake, R.R. 2006a. "A Possible Model For Higher Education: The Physics
Reform Effort (Author's Executive Summary)," Spark (American
Astronomical Society Newsletter), June, online at
<http://www.aas.org/education/spark/SparkJune06.pdf> (1.9MB). Scroll
down about 4/5 of the way to the end of the newsletter.
Hake, R.R. 2006b. "Possible Palliatives for the Paralyzing Pre/Post
Paranoia that Plagues Some PEP's" [PEP's = Psychometricians,
Education specialists, and Psychologists], Journal of
MultiDisciplinary Evaluation, Number 6, November, online at
<http://evaluation.wmich.edu/jmde/JMDE_Num006.html>.
Hake, R.R. 2007. "Should We Measure Change? Yes!" online as ref. 43 at
<http://www.physics.indiana.edu/~hake>. To appear as a chapter in
"Evaluation of Teaching and Student Learning in Higher Education," a
Monograph of the American Evaluation Association
<http://www.eval.org/>. A severely truncated version appears at Hake
(2006b).
Halloun, I. & D. Hestenes. 1985a. "The initial knowledge state of
college physics students." Am. J. Phys. 53:1043-1055; online at
<http://modeling.asu.edu/R&E/Research.html>. Contains the "Mechanics
Diagnostic" test, precursor to the widely used Force Concept
Inventory [Hestenes et al. (1995)].
Hestenes, D., M. Wells, & G. Swackhamer, 1992. "Force Concept
Inventory," Phys. Teach. 30: 141-158; online (except for the test
itself) at <http://modeling.asu.edu/R&E/Research.html>. The 1995
revision by Halloun, Hake, Mosca, & Hestenes is online (password
protected) at the same URL, and is available in English, Spanish,
German, Malaysian, Chinese, Finnish, French, Turkish, Swedish, and
Russian.
Huber, M.T. & P. Huthchings. 2005. "The Advancement of Learning:
Building the Teaching Commons." Jossey-Bass. Information at
<http://www.carnegiefoundation.org/publications/pub.asp?key=43&subkey=1254>.
The only mention of pre/post testing that I can find is on page 50:
". . . .what [Maura Flannery (2001)] realized . . . was that she
needed windows into her students' learning that could not be provided
through traditional assessments and assignments. . . .she designed a
simple pretest that asked students at the beginning of the semester
about their attitudes and knowledge biology . . . . and invited them
to draw a cell and a molecule. At the end of the semester, when the
task was repeated, she was pleased to see more elaborate images. . .
." It would appear that Flannery may have been slightly ahead of her
time - see e.g., Klymkowsky et al. (2003) and Klymkowsky (2007). Lest
it be thought that the Carnegie Foundation is, except for reference
to Flannery, oblivious of pre/post testing. see the essay by
Carnegie Scholar Lloyd Bond (2006).
Klymkowsky, M.W., K. Garvin-Doxas, & M. Zeilik. 2003. "Bioliteracy
and Teaching Efficiency: What Biologists Can Learn from Physicists,"
Cell Biology Education 2: 155-161; online at
<http://www.lifescied.org/cgi/reprint/2/3/155>. The abstract reads:
The introduction of the Force Concept Inventory (FCI) by David
Hestenes and colleagues in 1992 produced a remarkable impact within
the community of physics teachers. An instrument to measure student
comprehension of the Newtonian concept of force, the FCI demonstrates
that active learning leads to far superior student conceptual
learning than didactic lectures. Compared to a working knowledge of
physics, biological literacy and illiteracy have an even more direct,
dramatic, and personal impact. They shape public research and
reproductive health policies, the
acceptance or rejection of technological advances, such as
vaccinations, genetically modified foods and gene therapies, and, on
the personal front, the reasoned evaluation of product claims and
lifestyle choices. While many students take biology courses at both
the secondary and the college levels, there is little in the way of
reliable and valid assessment of the effectiveness of biological
education. This lack has important consequences in terms of general
bioliteracy and, in turn, for our society. Here we describe the
beginning of a community effort to define what a bioliterate person
needs to know and to develop, validate, and disseminate a tiered
series of instruments collectively known as the Biology Concept
Inventory (BCI), which accurately measures student comprehension of
concepts in introductory, genetic, molecular, cell, and developmental
biology. The BCI should serve as a lever for moving our current
educational system in a direction that delivers a deeper conceptual
understanding of the fundamental ideas upon which biology and
biomedical sciences are based.
Klymkowsky, M.W. 2006. "Bioliteracy.net," online at <http://bioliteracy.net/>:
"Our goal is to generate, test and distribute the tools to determine
whether students are learning what teachers think they are teaching.
We assume that accurate and timely assessment of student knowledge
will pressure the educational world toward more effective teaching.
WHY? (a) Because basic understanding of the biological sciences
impacts our lives in more and more dramatic ways every year. (b) A
wide range of important personal, social, economic and political
decisions depend upon an accurate understanding of basic biology and
the means by which science generates, tests and extends our
knowledge."
Nelson, C. 2000. "Bibliography: How To Find Out More About College
Teaching and Its Scholarship: A Not Too Brief, Very Selective
Hyperlinked List." (College Pedagogy IS A Major Area Of
Scholarship!); online at
<http://php.indiana.edu/~nelson1/TCHNGBKS.html> . See especially "Two
Examples Showing That Different Pedagogy Really Matters." Neither
example made it into the Hutchings et al. (2002) bibliography. [The
Nelson citation given by Hutchings et al. (2002) has rotted.]
Wiggins, G. 1990. "The Truth May Make You Free, But the Test May Keep
You Imprisoned: Toward Assessment Worthy of the Liberal Arts," AAHE
Assessment Forum: 17-31; online at the Mathematical Association of
America (MAA) project "Supporting Assessment in Undergraduate
Mathematics" (SAUM) at <http://www.maa.org/saum/> / "Getting Started
With Assessment" where "/" means "click on," or download directly at
<http://www.maa.org/saum/articles/wiggins_appendix.html>.