Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

Re: [Phys-l] SoTL and Finance

ABSTRACT: Mike La Lopa, in his POD post Re: SoTL (Scholarship of Teaching and Learning) and Finance, suggests that business faculty need information that will show that their teaching is ineffective before changing their teaching methods. I think that the same can be said of nearly ALL faculty, and that a convincing indicator of teaching ineffectiveness is relatively low pre-to-post test gain on valid and consistently reliable tests of conceptual understanding developed by disciplinary experts. But are business (or any other) faculty willing to undertake the arduous qualitative and quantitative research required for such development?
If you reply to this long post (19 kB - Sorry, it's all the fault of Mick La Lopa) please don't hit the reply button unless you prune the copy of this post that may appear in your reply down to a few relevant lines, otherwise the entire already archived post may be needlessly resent to subscribers.

Mick La Lopa, in his POD post of 11 Jun 2007 titled "Re: SoTL and Finance" wrote [my inserts at ". . . . .[insert]. . . ."]:

"The. . . . .[business]. . . school folks are most likely Concrete Sequentials (Thinkers) or Abstract Sequentials (Directors). . . . [see, e.g., Gregorc (2006)]. . . You are correct that in both cases they will need information to move them along especially the kind that shows them that their teaching methods may be inadequate, illogical, inefficient, ineffective, or inferior compared to others. Once they figure that out for themselves -- you cannot do it for them -- they will discover this great idea they had called SoTL . . . .[(2007)]. . . once you are gone. Perhaps give them the SoTL bibliography found at: <>."

In my opinion nearly *all* faculty (be they Concrete, Abstract, or Incon Sequentials) need to find out that their teaching methods are "inadequate, illogical, inefficient, ineffective, or inferior compared to others," or compared to what might reasonably be expected, before they will exert the effort to change their teaching methods.

But how will they find out? Probably not from course exams, peer assessment, student teaching evaluations, or even by studying the citations in "The Scholarship of Teaching and Learning in Higher Education: An Annotated Bibliography" [Hutchings et al. (2002)]. The latter appear to contain nothing related to the *direct* measurement of course effectiveness by pre/post testing, except indirectly though the reference to Nelson (2000).

Lesson #3 of the generally ignored "Lessons of the Physics Education Reform Effort" [Hake (2002)] is:
**High-Quality Standardized Tests of the Cognitive and Affective Impact of Courses Are Essential to Gauge the Relative Effectiveness of Non-Traditional Educational Methods.**

So great is the inertia of the educational establishment [see e.g., "Eleven Quotes in Honor of Inertia" (Hake, 2006c)] that three decades of physics education research [see, e.g., McDermott & Redish (1999)] demonstrating the futility of the passive-student lecture in introductory courses was ignored until high-quality standardized tests, that could be easily administered to thousands of students, became available.

As a case in point: in sharp contrast to most faculty, Harvard's Eric Mazur was aware of the Mechanics Diagnostic test of Halloun & Hestenes (1998a,b), precursor to the widely employed Force Concept Inventory [Hestenes et al. (1992)]. Mazur (1997, p. 4) wrote:

"When reading this. . . . .[the Mechanics Diagnostic test]. . . my first reaction was 'Not my students. . .!' Intrigued, I decided to test my own students' conceptual understanding, as well as that of physics majors at Harvard. . . . . the results of the test came as a shock: The students faired hardly better on the Halloun and Hestenes test than on their midterm exam. Yet the Halloun and Hestenes test is SIMPLE, whereas the material covered by the examination (rotational dynamics, moments of inertia) if of far greater difficulty, or so I thought."

Convinced by the pre/post test results that traditional methods of instruction had little, if any, impact on students' conceptual understanding, Mazur and his colleagues went on to develop and use much more effective pedagogical methods, see. e.g., Crouch & Mazur (2000). The same can be said for many other astronomy, economics, biology, chemistry, geoscience, engineering, mathematics, and physics faculty, as reviewed in Hake (2002; 2005; 2006a,b; 2007).

But are any business faculty willing to undertake the arduous qualitative and quantitative research required to develop a valid and consistently reliable BUCI (BUsiness Concept Inventory) counterpart of the BCI [Biology Concept Inventory (Klymkowsky et al., 2003; Klymkowsky, 2007)] or FCI [Force Concept Inventory (Hestenes et al., 1992)]?
For that matter are *any* faculty (other than a few in math and science) willing to do so?

Richard Hake, Emeritus Professor of Physics, Indiana University
24245 Hatteras Street, Woodland Hills, CA 91367

"What we assess is what we value. We get what we assess, and if we don't assess it, we won't get it."
Lauren Resnick [quoted by Grant Wiggins (1990)]

REFERENCES [Tiny URL's courtesy <>.]
Bond. L. 2005. "Carnegie Perspectives: A different way to think about teaching and learning - Who has the lowest prices," online at
<>; see also the commentary at
<>, including that by Hake at
<>, and

Crouch, C.H. & E. Mazur. 2001. "Peer Instruction: Ten years of experience and results," Am. J. Phys. 69: 970-977; online at <>.

Flannery, M. 2001. "The Mind-Body Problem," American Biology Teacher 62(8): 610-615; online to subscribers at <>.

Gregorc, A.F. 2006. "More Info on Abstract/Random/Concrete/Sequential, Links, & some very helpful Tables," online at <>. Gregorc gives learning preferences for each of 4 "Mind Styles": Concrete Sequentials, Abstract Sequentials, Abstract Randoms, and Concrete Randoms.

Hake, R.R. 2002. "Lessons from the physics education reform effort," Ecology and Society 5(2): 28; online at <>. Ecology and Society (formerly Conservation Ecology) is a free online "peer-reviewed journal of integrative science and fundamental policy research" with about 11,000 subscribers in about 108 countries.

Hake, R. R. 2005. "The Physics Education Reform Effort: A Possible Model for Higher Education?" online at <> (100 kB). This is a slightly edited version of an article that was (a) published in the National Teaching and Learning Forum 15(1), December, online to subscribers at <>, and (b) disseminated by the Tomorrow's Professor list <> as Msg. 698 on 14 Feb 2006. For an executive summary see Hake (2006a).

Hake, R.R. 2006a. "A Possible Model For Higher Education: The Physics Reform Effort (Author's Executive Summary)," Spark (American Astronomical Society Newsletter), June, online at <> (1.9MB). Scroll down about 4/5 of the way to the end of the newsletter.

Hake, R.R. 2006b. "Possible Palliatives for the Paralyzing Pre/Post Paranoia that Plagues Some PEP's" [PEP's = Psychometricians, Education specialists, and Psychologists], Journal of MultiDisciplinary Evaluation, Number 6, November, online at <>.

Hake, R.R. 2006c. "Eleven Quotes in Honor of Inertia," online at
<>. Post of 13 Jun 2006 15:01:14-0700 to AERA-L, PhysLrnR, and POD.

Hake, R.R. 2007. "Should We Measure Change? Yes!" online as ref. 43 at
<>. To appear as a chapter in "Evaluation of Teaching and Student Learning in Higher Education," a Monograph of the American Evaluation Association <>. A severely truncated version appears at Hake (2006b).

Halloun, I. & D. Hestenes. 1985a. "The initial knowledge state of college physics students." Am. J. Phys. 53:1043-1055; online at <>. Contains the "Mechanics Diagnostic" test, precursor to the widely used Force Concept Inventory [Hestenes et al. (1995)].

Halloun, I. & D. Hestenes. 1985b. "Common sense concepts about motion." Am. J. Phys. 53:1056-1065; online at <>.

Hestenes, D., M. Wells, & G. Swackhamer, 1992. "Force Concept Inventory," Phys. Teach. 30: 141-158; online (except for the test itself) at <>. The 1995 revision by Halloun, Hake, Mosca, & Hestenes is online (password protected) at the same URL, and is available in English, Spanish, German, Malaysian, Chinese, Finnish, French, Turkish, Swedish, and Russian.

Huber, M.T. & P. Huthchings. 2005. "The Advancement of Learning: Building the Teaching Commons." Jossey-Bass. Information at <>. The only mention of pre/post testing that I can find is on page 50: ". . . .what [Maura Flannery (2001)] realized . . . was that she needed windows into her students' learning that could not be provided through traditional assessments and assignments. . . .she designed a simple pretest that asked students at the beginning of the semester about their attitudes and knowledge biology . . . . and invited them to draw a cell and a molecule. At the end of the semester, when the task was repeated, she was pleased to see more elaborate images. . . ." It would appear that Flannery may have been slightly ahead of her time - see e.g., Klymkowsky et al. (2003) and Klymkowsky (2007). Lest it be thought that the Carnegie Foundation is, except for reference to Flannery, oblivious of pre/post testing. see the essay by Carnegie Scholar Lloyd Bond (2006).

Hutchings, P. M. Babb, C. Bjork. 2002. "The Scholarship of Teaching and Learning in Higher Education: An Annotated Bibliography," online at <> (120kB). See also Huber & Hutchings (2005).

Klymkowsky, M.W., K. Garvin-Doxas, & M. Zeilik. 2003. "Bioliteracy and Teaching Efficiency: What Biologists Can Learn from Physicists," Cell Biology Education 2: 155-161; online at <>. The abstract reads:
The introduction of the Force Concept Inventory (FCI) by David Hestenes and colleagues in 1992 produced a remarkable impact within the community of physics teachers. An instrument to measure student comprehension of the Newtonian concept of force, the FCI demonstrates that active learning leads to far superior student conceptual learning than didactic lectures. Compared to a working knowledge of physics, biological literacy and illiteracy have an even more direct, dramatic, and personal impact. They shape public research and reproductive health policies, the
acceptance or rejection of technological advances, such as vaccinations, genetically modified foods and gene therapies, and, on the personal front, the reasoned evaluation of product claims and lifestyle choices. While many students take biology courses at both the secondary and the college levels, there is little in the way of reliable and valid assessment of the effectiveness of biological education. This lack has important consequences in terms of general bioliteracy and, in turn, for our society. Here we describe the beginning of a community effort to define what a bioliterate person needs to know and to develop, validate, and disseminate a tiered series of instruments collectively known as the Biology Concept Inventory (BCI), which accurately measures student comprehension of concepts in introductory, genetic, molecular, cell, and developmental biology. The BCI should serve as a lever for moving our current educational system in a direction that delivers a deeper conceptual understanding of the fundamental ideas upon which biology and biomedical sciences are based.

Klymkowsky, M.W. 2006. "," online at <>:
"Our goal is to generate, test and distribute the tools to determine whether students are learning what teachers think they are teaching. We assume that accurate and timely assessment of student knowledge will pressure the educational world toward more effective teaching. WHY? (a) Because basic understanding of the biological sciences impacts our lives in more and more dramatic ways every year. (b) A wide range of important personal, social, economic and political decisions depend upon an accurate understanding of basic biology and the means by which science generates, tests and extends our knowledge."

Mazur, E. 1997. "Peer Instruction: A User's Manual," Prentice Hall. Information at

McDermott, L.C. & E.F. Redish. 1999. "RL-PER1: Resource letter on physics education research," Am. J. Phys. 67(9): 755-767; online at <>.

Nelson, C. 2000. "Bibliography: How To Find Out More About College Teaching and Its Scholarship: A Not Too Brief, Very Selective Hyperlinked List." (College Pedagogy IS A Major Area Of Scholarship!); online at <> . See especially "Two Examples Showing That Different Pedagogy Really Matters." Neither example made it into the Hutchings et al. (2002) bibliography. [The Nelson citation given by Hutchings et al. (2002) has rotted.]

SOTL. 2007. Scholarship of Teaching and Learning, information online at <>.

Wiggins, G. 1990. "The Truth May Make You Free, But the Test May Keep You Imprisoned: Toward Assessment Worthy of the Liberal Arts," AAHE Assessment Forum: 17-31; online at the Mathematical Association of America (MAA) project "Supporting Assessment in Undergraduate Mathematics" (SAUM) at <> / "Getting Started With Assessment" where "/" means "click on," or download directly at <>.