If you object to cross-posting as a way to tunnel through inter- and
intra-disciplinary barriers, please hit "delete" now. And if you
respond to this long (16 kB) post, please don't hit the reply button
unless you prune the original message normally contained in your
reply down to a few lines, otherwise you may inflict this entire post
yet again on suffering list subscribers.
Education Week reporter Sean Cavanagh (2005) has cogently discussed
the laudable attempt by the NRC's Wilson & Bertenthal (2005) to give
some guidance to state officials on how to develop the science
achievement tests required by the No Child Left Behind (NCLB) Act in
the 2007-08 school year.
W&B-W&B-W&B-W&B-W&B-W&B-W&B-W&B-W&B-W&B-W&B-W&B
In response to the No Child Left Behind Act of 2001 (NCLB), "Systems
for State Science Assessment" explores the ideas and tools that are
needed to assess science learning at the state level. This book
provides a detailed examination of K-12 science assessment: looking
specifically at what should be measured and how to measure it.
Along with reading and mathematics, the testing of science is a key
component of NCLB and is part of the national effort to establish
challenging academic content standards and develop the tools to
measure student progress toward higher achievement. The book will be
a critical resource for states that are designing and implementing
science assessments to meet the 2007-2008 requirements of NCLB.
In addition to offering important information for states, "Systems
for State Science Assessment" provides policy makers, local schools,
teachers, scientists, and parents with a broad view of the role of
testing and assessment in science education.
W&B-W&B-W&B-W&B-W&B-W&B-W&B-W&B-W&B-W&B-W&B-W&B
It is heartening that, following the lead of Donovan & Pellegrino
(2002), and departing from the dismissal of the mass-testing use of
the "Force Concept Inventory" [Hestenes et al. (1992)] in previous
NRC reports [e.g., Fox & Hackermann (2003), Labov (2003), McCray et
al. (2004)]; Wilson & Bertenthal (2005) have recognized that
multiple-choice testing:
(a) CAN serve to measure student understanding in conceptually
difficult areas, and
(b) is therefore of value in large-scale testing of the effectiveness
of courses in promoting students' conceptual understanding, as has
been demonstrated by the pre/post testing carried out by physics
education researchers [for a review see Hake (2002a,b)].
On page 81-82, Wilson & Bertenthal (2005) write [my CAPS]:
W&B-W&B-W&B-W&B-W&B-W&B-W&B-W&B-W&B-W&B-W&B-W&B
Performance assessment is an approach that offers great potential for
assessing complex thinking and learning abilities, but multiple
choice items also have their strengths. For example, although many
people recognize that multiple-choice items are an efficient and
effective way of determining how well students have acquired basic
content knowledge, many do not recognize that they can also be used
to measure complex cognitive processes. For example, THE FORCE
CONCEPT INVENTORY (Hestenes, Wells, and Swackhamer, 1992) IS AN
ASSESSMENT THAT USES MULTIPLE-CHOICE ITEMS TO TAP INTO HIGHER LEVEL
COGNITIVE PROCESSES.
W&B-W&B-W&B-W&B-W&B-W&B-W&B-W&B-W&B-W&B-W&B-W&B
I hope that Wilson & Bertenthal (2005) will help to avoid the
possibility that California's Direct Science Instruction [Hake
(2004)] will not be propagated throughout the U.S. by a Dept. of
Education/NCLB-induced emphasis [Hake (2005a,b,c) on the memorization
of scientific facts, formulas, and algorithms.
However, the failure of Wilson & Bertenthal and the NRC's Board on
testing and Assessment (perhaps related to the fact that there are no
physical or life scientists among the membership of that Board
<http://www7.nationalacademies.org/bota/BOTA_Members.html>] to
discuss the extensive qualitative and quantitative research [Halloun
& Hestenes (1985a,b)] BY DISCIPLINARY SCIENCE EXPERTS that laid the
groundwork for the Force Concept Inventory (FCI), may lead to inept
state-developed multiple-choice tests. That amateurs cannot develop
valid and consistently reliable science tests of the caliber of the
FCI has been painfully demonstrated by the physics tests produced by
the California Standardized Testing and Reporting (STAR) Program [for
a discussion see Woolf (2005a,b,c) & Hake (2005d)].
Donovan, M.S. & J. Pellegrino, eds. 2003. "Learning and Instruction:
A SERP Research Agenda," National Academies Press; online at
<http://books.nap.edu/catalog/10858.html>.
FLAG. 2005. "Field-tested Learning Assessment Guide; online at
<http://www.flaguide.org/>: ". . . offers broadly applicable,
self-contained modular classroom assessment techniques (CAT's) and
discipline-specific tools for STEM [Science, Technology, Engineering,
and Mathematics] instructors interested in new approaches to
evaluating student learning, attitudes and performance. Each has been
developed, tested, and refined in real colleges and universities
classrooms." Assessment tools for physics and astronomy (and other
disciplines) are at <http://www.flaguide.org/tools/tools.php>.
Fox, M.A., & N. Hackerman, eds. 2003. National Research Council,
Committee on Undergraduate Science Education, National Academy Press;
online at <http://www.nap.edu/catalog/10024.html>.
Hake, R.R. 2002a. "Lessons from the physics education reform effort,"
Ecology and Society 5(2): 28; online at
<http://www.ecologyandsociety.org/vol5/iss2/art28/>. Ecology and Society
(formerly Conservation Ecology) is a free online "peer-reviewed
journal of integrative science and fundamental policy research" with
about 11,000 subscribers in about 108 countries.
Hake, R.R. 2005b. "Seven Reasons Why The NCLB Might Promote Direct
Instruction of Science in the U.S. and One Reason Why It Might Not,"
online at
<http://lists.nau.edu/cgi-bin/wa?A2=ind0504&L=phys-l&F=&S=&P=1107>.
Post of 4 Apr 2005 15:03:45-0700 to AERA-C, AERA-D, AERA-G, AERA-H,
AERA-J, AERA-K, AERA-L, AP-Physics, ASSESS, Biopi-L, Chemed-L,
EvalTalk, Math-Learn, Phys-L, Physhare, POD, and STLHE-L.
Hake, R.R. 2005c. "Will the NRC Switch the NCLB's Direct Science Instruction
Juggernaut to the Guided Inquiry Track?, online at
<http://lsv.uky.edu/cgi-bin/wa.exe?A2=ind0504&L=assess&T=0&F=&S=&P=2423>.
Post of 6 Apr 2005 11:42:21-0700 to AERA-C, AERA-D, AERA-G, AERA-H,
AERA-J, AERA-K, AERA-L, AP-Physics, ASSESS, Biopi-L, Chemed-L,
EvalTalk, Math-Learn, Phys-L, PhysLrnR, Physhare, POD, and STLHE-L.
Halloun, I. & D. Hestenes. 1985a. "The initial knowledge state of
college physics students," Am. J. Phys. 53:1043-1055; online at
<http://modeling.asu.edu/R&E/Research.html>. Contains the Mechanics
Diagnostic test, precursor to the widely used Force Concept Inventory
[Hestenes et al. (1992)].
Halloun, I., R.R. Hake, E.P Mosca, D. Hestenes. 1995. "Force Concept
Inventory" (Revised, 1995); online (password protected) at
<http://modeling.asu.edu/R&E/Research.html>. Available in English,
Spanish, German, Malaysian, Chinese, Finnish, and Russian.
Hestenes, D., M. Wells, & G. Swackhamer, 1992. "Force Concept
Inventory." Phys. Teach. 30: 141-158; online (except for the test
itself) at <http://modeling.asu.edu/R&E/Research.html>. For the
slightly revised 1995 version see Halloun et al. (1995).
McCray, R.A., R.L. DeHaan, J.A. Schuck, eds. 2003. "Improving
Undergraduate Instruction in Science, Technology, Engineering, and
Mathematics: Report of a Workshop" Committee on Undergraduate STEM
Instruction," National Research Council, National Academy Press;
online at <http://www.nap.edu/catalog/10711.html>.
Physicists/astronomers attending the workshop were Paula Heron,
Priscilla Laws, John Layman, Ramon Lopez, Richard McCray, Lillian
McDermott, Carl Wieman, Jack Wilson, and (believe it or not) even the
FLAG (2005) waving Mike Zeilik.
Wiggins, G. 1990. "The Truth May Make You Free, But the Test May Keep
You Imprisoned: Toward Assessment Worthy of the Liberal Arts," AAHE
Assessment Forum: 17-31; online at the Mathematical Association of
America (MAA) project "Supporting Assessment in Undergraduate
Mathematics" (SAUM) at
<http://www.maa.org/saum/> / "Getting Started With Assessment" where
"/" means "click on," or download directly at
<http://www.maa.org/saum/articles/wiggins_appendix.html>.