Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

Re: [Phys-l] Ratings of US High Schools - 2nd Try



My apologies if this appears twice. I transmitted it this morning but it has not yet appeared on the archives. Here's a second try:

If you reply to this long (18 kB) post please don't hit the reply button unless you prune the copy of this post that may appear in your reply down to a few relevant lines, otherwise the entire already archived post may be needlessly resent to subscribers.

*******************************
ABSTRACT: A problem with the "Mathews Challenge Index" (number of students taking AP or IB tests divided by number of graduating seniors) used by "Newsweek" to rate the merit of U.S. high schools is that AP and IB courses are not, in general, well aligned with established learning principles, as indicated in the NRC report "Learning and Understanding: Improving Advanced Study of Mathematics and Science in U.S. High Schools."
*******************************

Dennis Roberts (2006), in his post of 4 May 2006 to EvalTalk and EdStat titled "Ratings of US High Schools" wrote [bracketed by lines "RRRRRRR. . . ."; ellipses "..." in the original; slightly edited]:

RRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
Each year for the past few years, Newsweek has made a big splash with its annual rankings/ratings of high schools ... which can be found here:

<http://www.msnbc.msn.com/id/12532668/site/newsweek/>

A Washington Post Education writer, Jay Mathews, came up with a system that essentially does the following:

HS Rating = (# of HS students in 2005 who have taken Advanced
Placement or International Baccalaureate TESTS) / (# of graduating seniors) ...

This value is calculated ... then the ratings for 1000 HSchools are shown.

CLEARLY, the definition of a great high school is based on TWO things and two things only:

1. How many students TAKE AP and/or IB TESTS (passing is irrelevant)

2. How many students in the senior class graduated

So, the current values are for 2005 ...

I would be interested in ANY discussion ANY of you have related to this matter
. . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . .
RRRRRRRRRRRRRRRRRRRRRRRRRRRRRR

Mathews calls his rating index the "Mathews Challenge Index" [Mathews (2006a)] and attempts to justify its use as a gauge of high school merit in. e.g.:

(a) two articles "Why AP Matters: Test wars: Behind the debate over how we should judge high schools" Mathews (2006b) and "Four Steps to High School Greatness" [Mathews (2006c)]; and

(b) the transcript of a live talk on 3 May 2006, that appears at <http://www.msnbc.msn.com/id/12532668/site/newsweek/>
under "Web Exclusive." There Newsweek writes:

"This week, NEWSWEEK ranks America's best public high schools using a ratio devised by Washington Post Education Reporter Jay Mathews: the number of Advanced Placement or International Baccalaureate tests taken by all students at a school in 2004 divided by the number of graduating seniors. Mathews is also the author of 'What's Wrong (and Right) With America's Best Public High Schools.'. . .[Mathews (1999)]. . . He joined us for a Live Talk on Wednesday, May 3, to take your questions on NEWSWEEK's list of the best schools, the importance of AP tests, and what those not listed can do for better marks. Read the Transcript Below."

In my opinion, one of the problems with the Mathews method of rating high schools, at least insofar as physics is concerned, is that many AP physics courses simply mimic traditional passive-student introductory college courses.

Most of those subscribe to what Arnold Arons (1986) called "the relativistic model of instruction: based on the premise that, if one starts with an
E - N - O - R - M - O - U - S breadth of subject matter but passes it by the student at sufficiently high velocity, the Lorentz contraction will shorten it to the point at which it drops into the hole which is the student mind."

Such introductory physics courses are known to be relatively ineffective in promoting students' conceptual understanding of physics [see e.g., the reviews by Meltzer & Heron (2005) and Wieman & Perkins (2005)].

For a rather unfavorable critique (consistent with the above appraisal) of AP math and science and courses see "Learning and Understanding: Improving Advanced Study of Mathematics and Science in U.S. High Schools" [NRC (2002)]. According to the Executive Summary:

"the report presents results of a 2-year effort by a National Research Council (NRC) committee to examine programs for advanced study of mathematics
and science in U.S. high schools. The committee focused on the two most widely recognized programs in the United States, and the only two of national scope: Advanced Placement (AP) and International Baccalaureate (IB).

Expertise on the committee included scientist-researchers, secondary teachers of AP and IB, science and mathematics educators working on teacher education and issues of access and equity, cognitive scientists, and educational administrators. Panels of experts in the disciplines (biology, chemistry physics, and mathematics) also advised the committee. The four panel reports provided a critical basis for the committee's analysis and may be used independently of this volume. . . ."

The panel experts for physics were: Robin Spital, S. James Gates, David Hammer, Robert Hilborn, Eric Mazur, Penny Moore, and Robert Morse.

In the section "Analysis of AP and IB Programs Based On Learning Research," of the Executive Summary of the NRC (2002) report appears the following [bracketed by lines "NRC-NRC-NRC-. . . "]:

NRC-NRC-NRC-NRC-NRC-NRC-NRC-NRC-NRC-NRC-NRC-NRC
1. PRINCIPLED CONCEPTUAL KNOWLEDGE - Although the AP and IB programs espouse an emphasis on concepts and key ideas, this intention is largely unrealized in the sciences. Excessive breadth of coverage (especially in 1-year science programs) and insufficient emphasis on key concepts in final assessments contribute significantly to the problem in all science fields. Although emphasis on learning concepts and key ideas is more evident in mathematics, further improvement is needed, particularly in the assessments, which frequently focus on procedural knowledge at the expense of conceptual understanding.

2. PRIOR KNOWLEDGE - Except for mathematics, these programs do not specify clearly what prior knowledge is needed for success or help teachers to build on what students already know or to recognize student misconceptions. In all subjects, efforts to prepare students properly in the years preceding advanced study are often inadequate. Too many students, especially in physics, take a 1-year advanced course as their first course in the discipline - an inappropriate situation.

3. METACOGNITION - Advanced study can increase students' metacognitive skills, but many programs and courses do not help students develop these skills.

4. DIFFERENCES AMONG LEARNERS - AP and IB teachers who employ a variety of pedagogical approaches are likely to reach a broader range of learners. Using several sources of evidence of student progress also can provide a more accurate picture of what students know compared with any single measure, such as an examination. The single end-of year examinations and summary scores, as found in AP, do not adequately capture student learning.

5. MOTIVATION - Students have varied motives for enrolling in advanced study. Designing programs that are consistent with the findings of learning research can increase students' motivation to succeed in advanced study, encourage them to believe in their own potential, and increase the proportion of students who take and succeed in the course and final examinations.

6. LEARNING COMMUNITIES - Teamwork and collaborative investigation are especially important in advanced study. The breadth of course content and the generally short duration of laboratory periods in many schools may be inadequate for such activities. Better use of the Internet and technologies for collaborative learning is needed.

7. SITUATED LEARNING - Students need opportunities to learn concepts in a variety of contexts. The AP and IB programs currently do not emphasize
interdisciplinary connections sufficiently or assess students' ability to apply their knowledge in new situations or contexts. Additionally, advanced study courses might make better use of laboratory experiences by requiring students to plan experiments, decide what information is important, select experimental methods, and review results critically. These courses might also draw upon local resources (e.g., science-related industries) to give students experience with varied practices in mathematics and science.

Although AP AND IB PROGRAMS CURRENTLY ARE NOT WELL ALIGNED WITH LEARNING
PRINCIPLES (My CAPS), they can be revised with this research in mind. The resulting transformations are likely to make the programs more successful in enhancing deep conceptual learning and make them more accessible to additional students.
NRC-NRC-NRC-NRC-NRC-NRC-NRC-NRC-NRC-NRC-NRC-NRC

For discussion of AP course and the NRC (2002) report see Inside Higher Ed's David Epstein (2005a,b), and Hake (2005).

Richard Hake, Emeritus Professor of Physics, Indiana University
24245 Hatteras Street, Woodland Hills, CA 91367
<rrhake@earthlink.net>
<http://www.physics.indiana.edu/~hake>
<http://www.physics.indiana.edu/~sdi>


REFERENCES [Tiny URL's courtesy <http://tinyurl.com/create.php>]
Arons, A.B. 1986. "Conceptual Difficulties in Science," in Undergraduate Education in Chemistry and Physics: Proceedings of the Chicago Conferences on Liberal Education," No. 1, edited by M.R. Rice. Univ. of Chicago. p. 23-32.

Epstein, D. 2005a. "The New AP," Inside Higher Ed, 26 May; online at
<http://insidehighered.com/news/2005/05/26/ap>.

Epstein, D. 2005b. "Earning the AP Name," Inside Higher Ed, 15
August; online at <http://insidehighered.com/news/2005/08/15/ap>.

Hake, R.R. 2005. "The Role of Advanced Placement Courses," online at <http://listserv.nd.edu/cgi-bin/wa?A2=ind0508&L=pod&P=R5582&I=-3>. Post of 15 Aug 2005 13:34:29-0700 to AERA-J, AERA-H, Math-Learn, Phys-L, Physhare, PhysLrnR, & POD.

Heron, P.R.L. & D. Meltzer. 2005. "The future of physics education research:
Intellectual challenges and practical concerns," Am. J. Phys. 73(5): 459-462; online at <http://www.physicseducation.net/docs/Heron-Meltzer.pdf> (56 kB).

Mathews, J. 1999. "Class Struggle: What's Wrong (and Right) with America's Best Public High Schools." Three Rivers Press. Amazon.com information at <http://tinyurl.com/hf22m> .

Mathews, J. 2006a. "Jay Mathews's Challenge Index," Washington Post, May 1; online at
<http://www.washingtonpost.com/wp-srv/education/challenge/2006/challengeindex01.html>, or more compactly at <http://tinyurl.com/jo8bc>.

Mathews, J. 2006b. "Why AP Matters: Test wars: Behind the debate over how we should judge high schools," Washington Post, May 1; online at
<http://www.washingtonpost.com/wp-dyn/content/article/2006/05/01/AR2006050100399.html>, or more compactly <http://tinyurl.com/ptbl4>.

Mathews, J. 2006c. "Four Steps to High School Greatness," Washington Post, May 2; online at
<http://www.washingtonpost.com/wp-dyn/content/article/2006/05/02/AR2006050200567.html?sub=AR>, or more compactly <http://tinyurl.com/gd24n>.

NRC. 2002. National Research Council's Committee on Programs for Advanced Study of Mathematics And Science in American High Schools, co-chaired by Jerry Gollub and Philip Curtis, "Learning and Understanding: Improving Advanced Study of Mathematics and Science in U.S. High Schools," American Academy Press; online at <http://books.nap.edu/catalog/10129.html>. An Executive Summary is at <http://fermat.nap.edu/execsumm_pdf/10129.pdf> (968 KB).

Roberts, D. 2006. "Ratings of US High Schools," EvalTalk/EdStat post of 4 May 2006 21:16:10-0400 online at the EvalTalk/EdStat Archives <http://bama.ua.edu/archives/evaltalk.html>/<http://lists.psu.edu/archives/edstat-l.html>. One must evidently subscribe to EvalTalk/EdStat to access their archives, but it takes only a few minutes to subscribe by following the simple directions at
<http://bama.ua.edu/archives/evaltalk.html> /<http://lists.psu.edu/archives/edstat-l.html>."Join or leave the list (or change settings)" where "/" means "click on." If you're busy, then subscribe using the "NOMAIL" option under "Miscellaneous." Then, as a subscriber, you may access the archives and/or post messages at any time, while receiving NO MAIL from the list!

Wieman, C. & K. Perkins. 2005. "Transforming Physics Education," Phys. Today 58(11): 36-41; online at <http://www.colorado.edu/physics/EducationIssues/> / "Papers" (where "/" means "click on").