Re: Assessment

[John Clement <clement@HAL-PC.ORG>]

This is a very interesting question, and I would like to point out that
most schools are actually doing the same things and probably achieving
similar gain.  The real difference comes from the student population of
the typical school.  Most schools simply do the same things and then the
output is simply a function of the input.

I realize that many will not believe this assertion, but I would like to
point to the work of Shayer and Adey.  You can either read their
comprehensive book "Really Raising Standards" which you can get from the
library, as I am told it is out of print or go to the Innodata monograph
written by Adey.  It is available at.
http://www.ibe.unesco.org/International/Databanks/Innodata/inograph.htm

In England they found that if you graph the output scores vs the input
scores that they fall on the same line.  Their graphs adjust the scales
to linearize this line.  In their book and also in their original papers
in JRST early 90s you can see that the data clusters around this line
quite well.

They also show that an implementation of cognitive enhancement can raise
schools significantly above this line.

One this idea has been established, the idea of rating a school district
or individual school according to a fixed standard can be seen as quite
nonsensical.  Each school must be assessed according to the gain and not
according to the raw scores on any set of tests.

One of the top rated schools in Texas is Bellaire High.  The main reason
why it does so well is that it has a reputation for excellence.  This
reputation has attracted many students whose parents wish to get the
best possible education.  In other words it is a self fulfilling
prophecy.  However as far as I can see the pedagogy in math and science
is the same as most schools in the nation.  It does NOT include any of
the strategies that produce better understanding of either science or
math.

There are some indicators that point to slightly better education.
Studies that go back to the 70s are completely unambiguous in assessing
the effects of school size.  They find that problems with both faculty
and student increase as the number of students in a grade grows greater
than 150.  As a result massive HS with thousands of students will
generally experience more problems than ones with only about 600
students or less.  This particular number has recently been verified by
psychologists who have found that people can form about 150 close
attachments in a group.  Essentially massive schools will promote much
worse education in HS.

The problem of grade inflation certainly disguises what is really going
on in schools.  With standardized tests, the school preps the students
thoroughly so that they can do well on those tests, but the higher level
ability of the students usually suffers as a result.

The report A Nation at Risk certainly has some provocative ideas, but it
assumes that the actual quality of education has declined in the USA.
Gerald Bracey has a number of articles that debunks this idea.  I would
tend to agree that the quality of education is actually very similar to
what it was 50 years ago.  Actually, the number of things which have
been shoved into the middle and HS curriculum is much greater than it
was 50 years ago, but student understanding of basic concepts has
remained the same.  As far as I know there is no evidence that the
percentage of formal operational students has increased or decreased in
50 years.

This is a shame because since the 80s and early 90s various techniques
have been shown to be very effective in raising student ability to
reason.  To mention two such techniques "Instrumental Enrichment" (IE) ,
Reuven Feuerstein, and "Thinking Science" (TS), Shayer, Adey, & Yates.
IE can be explored by reading the 1980 book of the same name, and TS by
reading the above referenced Innodata monograph.  These two are actually
programs that can be bought and teachers can train in them.  A third way
of doing it is by using the learning cycle.  For a good reference on
that see Anton Lawson, "Science Teaching and the Development of
Thinking".

How would I assess the output of a school?  Have all students take the
Lawson test at the back of his book, and then compare.

Lacking that, you can find out how many teachers majored in the subjects
that they are teaching.  According to an analysis by ETS the better
trained the teachers, the better the output.  Unfortunately these sorts
of indicators do not assess the prevalence of pedagogy such as
Interactive Engagement (see Hake's studies).  By changing the paradigm
of education one can achieve much greater gains than by attempting
improvement in the factors that ETS has located.  Indeed pushing majors
in the subjects taught will actually decrease the pool of well trained
teachers unless the salaries are brought up to the national average for
equivalent industrial jobs for these specialties.  Texas has mandated
that all math teachers have to major in math.  I understand the output
of certified math teachers in TX has gone almost to the vanishing point.

Now to actually answer a question.  The usual measurement of school
district quality is according to standardized tests mandated by the
state plus such other factors as graduation rates, drop out rates ...
Graduation rates can be manipulated by increasing grade inflation.  Drop
out rates are also similarly manipulated.  The standardized tests have
already been addressed.  Each state often has a formula which is used to
rate schools as exemplary ...  This formula can and does place
nationally highly rated schools in low performing categories, and then
places so-so schools in the exemplary category.  You can probably go to
the school website to get some of the information, and to the state
education department to get more.

Is the nation at risk???  Yes, if you realize that 75% of graduating
seniors probably do not understand proportional reasoning.  However this
is the same risk it has always been.  The current national reforms will
not change that.  PER is nibbling at the edges of it, but the rate of
improvement is still small.  Other science education programs are also
doing some good things, but the number of teachers trained to use this
type of pedagogy is quite small.

Note, that students of educated parents will generally do well despite
the schools, and students of low SES parents will generally fall behind
also despite the schools.  The exceptionally endowed students (for
whatever reason) will generally forge ahead despite all obstacles.

John M. Clement
Houston, TX


> How does one assess the quality/proficiency of a school/school
district?
> My local school district says that it is well above average.  How
would
> that be measured?
>
> But I notice that the local middle school has more than 50% of its
> students
> on the honor roll.  Reminds me of Lake Woebegone!
>
> I caught a glimpse of a PBS program (Science Without Walls I think)
which
> mentioned a couple of reports:
>
> A Nation at Risk
>
> Science for All Americans
>
> Anyone know anything about these reports?  Quality?  Availability?
>
> TX