Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

Re: [Phys-l] Friendship trumps competance Was: Re:WHYVALUEADDEDTESTING IS A BUST.



I know that I said that I would not respond, but this had to be responded to.

I never equated minority with low income! I never used the word minority in that email. If you go to the website that I linked, you will see the quote:
"Economically Disadvantaged: 99%"

Not accurate? It's an exact quote.

Mike

----- Original Message ----- From: "John Clement" <clement@hal-pc.org>
To: "'Forum for Physics Educators'" <phys-l@carnot.physics.buffalo.edu>
Sent: Sunday, September 05, 2010 12:21 AM
Subject: Re: [Phys-l] Friendship trumps competance Was: Re:WHYVALUEADDEDTESTING IS A BUST.



Economically Disadvantaged: 99%
Maybe this doesn't equate perfectly with FRL (the criteria is not
defined), but that's beside the point when all we're doing is using the
measure to compare two schools. As long as we're comparing two schools in
the same state, the definition of ED will be the same.

You need to be careful with the statistics of what you claim. Just being
minority does not equate to low income levels needed for free/reduced price
lunch. Making this type of statement without evidence points to a possible
disregard of accuracy and reliability, and can lead the reader to doubt your
evidence. Also while it is true that the school cited did have 100% passing
one year in science, the other years are not close to that figure, so be
careful. This is known as cherry picking evidence and is invalid in a
scientific argument.


"If you are making an argument based on one outlier, then it should be
compared with another outlier."

I disagree. To show that poverty is not the cause of poor test scores at
high poverty schools, I would compare the poor school to the average of
all schools. A school that is far above average in percent poverty and
far above average in performance proves that there is not a causal
relationship between the two. If poverty was the cause of low
performance, then a poor school would have lower performance than ALL
affluent schools. If there is one exception to that, then poverty is not
the (sole) cause of the difference. I agree that there is a strong
correlation between poverty and low performance in most cases, but the
existence of outliers proves that it is not a causal relationship. AVID
produces a DVD of success stories of their students. There is one story
told by a homeless boy who goes back to school and ends up at Georgetown
University. Can't get much poorer than homeless. Did this student's
economics change? No, his support at school changed.


As to the relationship between poverty and poor performance you admit that
it is there. If you are comparing things you need to control variables. So
comparing an outlier with an average is not a valid comparison. Look at the
overall distribution and the STD for both types of populations. All
population statistics have outliers. A good example is that most racial
physical features vary more within a given racial population than between
populations. A single data point does not establish statistical validity.


"The distribution of students by grade is heavily weighted to the younger
grades and there seems to be a steady outflow of students because the
later grades have far fewer students."

Interesting, but not relevant to this discussion. The hypothesis being
tested is that poverty is the cause of poor performance in school. No
matter who these kids at Crawford elementary school are, they are poor . .
. and they are achieveing. We cannot answer your questions about this
pattern with the information that we presently have, but I will ask next
time I speak with their science coordinator. And as I stated, it's
irrelevant to the question being discussed anyhow. If there is one poor
student who is achieving, then the causal relationship is broken. If it
is not a causal relationship, then the problem can be fixed without
solving the poverty problem.

But it is relevant. Why is there an outflux? When you see things like that
they raise a red flag about a possible problem. The evidence from looking
at passing scores may be suspect. Have the lowest students been excluded by
getting rid of them somehow. Are lower level students dropping out or being
pushed out? If it is attrition then the rate is very high.


"One way is to have strategies to combat the poverty, and there are
studies that show this has a positive effect on education."

The studies on poverty say that the way to fight poverty is to close the
education achievement gap. A circular argument. The economy has crashed
in the last two years and there is no pattern of falling test scores
nation-wide as a result, another bit of evidence that poverty does not
cause low achievement. California's unemployment rate tripled over the
last two years, yet all of the measures of achievement increased.


No, it is not circular. The studies showed that when improvements were made
that resulted in less poverty, the scores rose. The argument is that this
shows that SES is one of the causes of poor performance. This was an
immediate and direct result. Education as a means for reducing poverty is a
longer term effect. The argument was simply an observation based on the
experimental studies, and is unrelated to your added observaton. As to the
effects of poverty, it is the fact that you have a low SES group compared to
the rest of the population. Indeed in all countries there are lower SES
groups which all have lower IQs than the general population. This has been
documented across countries including Japan, Belgium, the US... Now the
effect of improving education may result in feedback, which raises SES,
which raises education. As to unemployment, the effect of SES is strongly
psychological for a given population. Temporary unemployment should not
have an effect on this. The circularity argument is not rational.


There are 20 Language Arts, Math, Social Studies, and Science tests that
every single student in the state must take. 100% of those test scores
improved over the last 2 years of economic downfall. If you add up the
increase in percent proficiency of the 20 tests, they add up to 142
percentage points. The highest increase was on the 8th grade social
studies test where the percent proficient went from 36 to 47.


Notice that going from 36 to 47% proficient can be achieved by slight
changes in the average score because you may be on the steep slope of the
curve. I would like to know how much the average normalized score changed.

Explain that, Diane Ravitch!

"There is now a treatment for low working memory, and it can be expanded,
so why not use it?"

I'd love to hear more about that. But I'd also like to know what is meant
by "low working memory." Are Alzheimers and Dementia forms of low
working memory? If so, I would have thought that this announcement would
be headline news. But, I don't see what any of this has to do with
poverty and low performance. If you're making a case for low working
memory and low performance, it would make sense.

I am pointing out that there are treatments for problems and that these have
been found to be effective. Alzheimer's and dementia are not problems of
working memory but loss of long term memory. These are things that all
educators should be aware of.


"So I can admit that the cited schools are doing something right, but I
will not retract the idea that the high stakes testing is overall having
negative effects on education."

That's not really what we've been discussing, but I disagree with that
too. I'd guess that you've never worked at a highly diverse, highly
underperforming school. I taught at one for 10 years and spend my days
traveling around the county helping underperforming schools improve. If
it were not for high stakes testing, I'd be sitting in an office waiting
for the phone to ring while generations of poor students go on learning
nothing. I've seen amazing transformations at low-performing schools that
never would have happened if they had not been identified as low-
performing according to the state test scores. I worked with a school
last year that had an average school-wide 2.8% proficiency in mathematics
over the last 10 years. In this school of 1,700 tested students, that
means that fewer than 50 of them are proficient in math. And absolutely
nothing was being done about it. Last year, their district was put on the
list of districts in need of improvement so I began work
ing with them in October. Last year, the school was put on the national
"5% Persistently Underperforming Schools" in early May. So, I worked with
them for about 7.5 months in between these two events. For last year,
their math proficiency went up to 11%. They got on the underperforming
list because their API had not increased 50 points in 5-years. Last year
alone, I helped them increase 40 points. And we did not reduce poverty,
we improved collaboration and instruction. 11% proficiency is still not
great, but that is 125 more students proficient in math who would not have
been proficient if this school had not been identified by state testing as
underperforming. The results would have been better, but the district
selected the transformation model of improvement where all of the
administrators are removed and half of the teachers are transferred. All
of this occurred in the middle of testing and was a huge distraction.
When these teachers looked at their math data,
they literally responded, "That's just how our students are." They
would never have made these improvements if they were not forced to do so.
This is about the 10th school like this that I have worked with in the
last 3 years and they all share very similar stories. My office has
gotten schools out of program improvement and taken high-poverty schools'
APIs from low 600's to the low 900's (nearly highest in the county)
without EVER fixing poverty or changing the economy in the city. We did
it by improving instruction, increasing collaboration, improving
leadership, and focusing all of the systems at these schools on student
achievement.

The evidence for the high stakes testing as being detrimental comes from the
international studies. But notice what you have just said is not dependent
on high stakes. Publicizing the results of testing will put pressure on
schools without the need for the high stakes. So testing can be beneficial,
but high stakes testing is not. Notice I did agree that it is possible to
improve schools, but the evidence is still there that low SES contributes to
lower test scores. And notice I and the cited paper agree that improving
insturuction, increasing collaboration, leadership, and achievement are all
good goals.

We agree that schools can be improved, but this last story still showed that
the low SES school failed to achieve what a high SES school can. Ravitch is
right that we can not ignore SES. But the school can not do anything about
it. Other community services need to be built up to help the community in
tandem with school improvements if you want to get the highest achievement.
The two articles cited by BC did not claim that one could not improve
schools. The claim was Ravitch says that the current educational paradigm
is ignoring the effects of poverty.


>From the paper cited: "although standardized test scores of students are
one piece of information for school leaders to use to make judgments about
teacher effectiveness, such scores should be only a part of an overall
comprehensive evaluation."

Nobody in the education world has ever said that 100% of a teacher's
evaluation should be their students' test scores. I think you'd be hard-
pressed to find a single person in the education world who would make that
argument. I didn't have to read far to realize that the researchers were
just proving something that everyone already believes.


No but they are saying 50% should be based on these scores when the
reliability of them is in the 6 to 14% range, and then only if they are
properly normalized. This is from observed data about value-added.

I, for one, believe that (at least in California), the standardized tests
do a pretty darned good job of measuring what students learn. As such, I
have no problem with the idea that these test scores may be used as part
of teacher evaluations. But, I do not believe that 100% of the evaluation
should be test scores.

Name a percentage, but remember that 84%+ of value added testing is really
random chance. The statement of not 100% still allows anything from 99% to
1%. The authors of the paper did not say ignore value-added, but that it
should be a small percentage of the evaluation.


I'll leave you with one final question . . . What would you do if your
child's school published value-added test results and your child's teacher
was identified as very low-performing, worst in the school? If you say
"nothing" . . . I don't believe you. If you'd be bothered by it, then
deep down inside you truly believe that there is something to this
measurement. I would argue that everyone would and should be bothered by
it.

At one time I would have been bothered by it, but now I would take a
different view. I have seen too many teachers who look good on paper, but
are really not that good. My children went to a well regarded school, that
maintained its reputation by having a large number of National Merit
scholars, but that had poorly taught regular classes. Indeed I have been
fooled by schools and teachers into thinking they were better than they
really were. Since I now know that value-added assessment is a crap shoot,
I would not be as bothered by it. Most people would be bothered because
they do not know the facts about value-added testing. And value-added
testing is much less reliable when it is not corrected for SES. I would say
that the facts about value-added testing should be known, and that people
should NOT be bothered by low value added scores.

One thing that we should be aware of is that high or low value added scores
are often not attributable to a specific teacher. In my computer science
classes I raised SAT scores on the math section for some of my students.
But of course the higher value added would have been attributable to the
math teacher. Meanwhile I found that my physics students gained in thinking
skills, but the equivalent students who took only the same math, but not
physics did not show gain. Value-added scores are not valid or reliable
enough to significantly contribute to teacher evaluations. Any attempt to
scare people away or towards specific teachers on the basis of value-added
is simplistic and may penalize the innocent. The estimated average
reliability is only about 10%. The paper pointed out that value-added may
only be reliable over a long term, but even then it may not. Some teachers
are always assigned students with more problems, so their value-added will
always be low. The paper did agree that value-added is more reliable for
schools as whole. So the authors of the paper proposed that rewards should
be made for school wide improvements, which is actually in line with current
industry methods.

The following is not data, but it is an indicator of possible problems. My
children went to a well regarded private elementary school that sent many of
its graduates to high performing HS. They hated it. One teacher was
actually superior because she gave my son accommodations that he needed,
when another punished my daughter and required he to do things that were
both painful (mentally) and counterproductive. Finally in the last few
weeks she realized my daughter was dyslexic. She ignored all of the times
we told her this. Duh. So they went to a public school which could provide
accommodations and did much better. So I take the reputation of schools
with a very large grain of salt.

I have also had parents object to things I do because their view of what
constituted good teaching was not in the best interest of students. I also
had test scores which showed that these same students had gained more than
they would in conventional classes.

Unfortunately to judge what is actually going on with respect to education
you need more than just passing rates. Small changes in the average can be
magnified to large changes in passing rates depending on the average. So
the average and STD are necessary to make better assessments of what is
going on. And in addition one needs to know if the curve is approximately
bell shaped or bimodal, or worse. Finally the attrition rate needs to be
looked at, and which students are leaving.

Actually I did work for a while in a low performing HS. From what I could
see most teachers were doing the same things I saw being done in a private
school. But I did see the same stick the head in the sand mentality with
respect to curriculum problems that I saw in the private school. Most of
the teachers I worked with did not notice the outrageous misconceptions in
the materials which they were using. Then of course I saw money being
poured into equipment and perfectly good stuff being thrown away that might
have been useful to other schools.

Now what I am seeing is that students are coming into college with very poor
preparation and low thinking ability. The authors of the cited paper have
pointed out that this might be the result of the narrowed curriculum
promoted by high stakes testing. This has probably been observed by others
on this list. There is the distinct possibility that the current apparent
increase in education due to increasing state test scores is illusory and
that the narrowing of curriculum is producing more passive, poorer thinking
students overall. How can this possibility be disproved???? At present the
international evaluations may be pointing to this problem.

John M. Clement
Houston, TX

_______________________________________________
Forum for Physics Educators
Phys-l@carnot.physics.buffalo.edu
https://carnot.physics.buffalo.edu/mailman/listinfo/phys-l

__________ Information from ESET Smart Security, version of virus signature database 5423 (20100904) __________

The message was checked by ESET Smart Security.

http://www.eset.com





__________ Information from ESET Smart Security, version of virus signature database 5424 (20100905) __________

The message was checked by ESET Smart Security.

http://www.eset.com