Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

Re: [Phys-L] Indicators of quality teaching(Was:MOOC: EdxOffers Mechanics course by Prof.Walter Lewin)



That of course is a more difficult thing to do. It is easy to quantify
learning in one course,but to do it over a longer time frame has generally
not been done. The longest time frame is probably the research of Shayer
and Adey which benchmarked the scores in the equivalent of high school for
an intervention in middle school and showed a large delayed effect.
Feuerstein showed a large effect at the end of high school for an
intervention a number of years earlier.

But I recall that there are some studies which showed higher subsequent
grades, but I can't put my finger on them. Perhaps some others can.
Incidentally the Hellers show both high gain and also the ability to solve
more complex problems using their method.

John M. Clement
Houston, TX


Is there any research that shows there is any gain from
"gain"? Improvement between a pre and post test is one thing
- the claim being lecture courses produce less "gain". But
what about the performance of these same students a few years
out? Has a study been done that takes a group of science
majors in college and divides them into two groups - those
who had "high gain" type classes in HS and those who had more
traditional lecture-lab type courses in HS? Is there evidence
that the "gain" students actually show gain in terms of their
college physics grades?

Bob at PC

________________________________________
From: Phys-l [phys-l-bounces@phys-l.org] on behalf of John
Clement [clement@hal-pc.org]
Sent: Friday, June 21, 2013 8:19 PM
To: Phys-L@Phys-L.org
Subject: Re: [Phys-L] Indicators of quality teaching
(Was:MOOC: EdxOffers Mechanics course by Prof.Walter Lewin)

Anyone who has read the research and is familiar with Hake's
posts should know what gain is.
One standard way of specifying gain is the effect size. This
is standard in many education research papers. Everyone who
calls them selves a teacher, instructor, or professor should
learn about these sorts of things.

Effect size = (post - pre)/STD
An effect size of 1 is considered enormous and many studies
do not get effect sizes larger than .5. Many PER
practicioners get effect sizes greater than 1.

But this definition of gain has the problem that it is skewed
by the size of the pre-test and also is highly dependent on
class homogeneity. Just a straight post-pre has a large
dependence on the pre test. So Hake came up with Hake gain
or normalized gain.

Normalized Gain = (post-pre)/(max score - pre) This is
relatively independent of the pre test score and is how most
PER results are quoted. You can convert it to a percentage,
and that is how I like to quote it. It indicates what
percentage of unknown material was learned during the class.

The pre and post-test are the same, or they need to be
correlated. I have even given the same test 3 weeks apart
and gotten exactly the same student scores, so using the same
test is OK as long as there is a time lag and the test has
been disguised and never discussed. One researcher told me
that 2 weeks is enough elapsed time to give the same test.
Incidentally Laws et al have found that when a subject is
taught using PER that the post test scores continue to rise
for 2 weeks. This is in contrast to conventional teaching
where the test scores droop after the lesson.

Many PER practicioners get gain between 50% and 70%, while
the average standard traditional course gets only 0% to 25%.
Hake has shown indirectly that the average HS course only
gets around 12%. The latest study of gain vs teaching method
vs instructor shows that traditional teaching gets around 13%
or so. But instructors that used the research based method
achieved around 50%, independent of instructor.

I have shown that the normalized gain depends strongly on the
Lawson test score. Which means that gain is strongly
dependent on the thinking skills of the students. Phillips
and Colletta have also shown this.

Similar results come from courses other than mechanics using
conceptual exams. In addition similar results come from math
courses. The inquiry math course shows higher gain than the
traditional course. Data goes as far back as the early 90s.
So we have had the results for over 20 years, and still
people are not changing what they do in class. Also if you
want to know how well the student learn you have to measure
it using an evaluation that is standard, and can be compared
to other classes.

Incidentally energetic can actually be a deficit. When you
put stuff into a class to make it interesting, that is what
students look at and remember.
But they don't remember the point of the interesting stuff.
Mazur showed that when you put people in illustrations, the
students spend the most time looking at the people and not
the physics. So low key to the point may work better. But
we have absolutely no data either way. It is a belief that
energetic means better, but that may not be true. People
think they can just look at a class and see how well they are
learning, but that is not true. Remember that MDs were
practicing medicine by observation for millenia. They did
come up with some good conclusions as to what can be
effective, but they also came up with squirrely ones like
blood letting. It took research to get them to the current
state, and initially they resisted it by continuing to do
operations in their blood-stained prestigious uniforms. Even
now some MDs resist routine hand washing.

Observation may point to a possible effect, but in the end
you have to measure it. We do that in physics, so why not in
education?

As to lecture, I did point out that they may be more
effective when the lecturer and listener are on the same
page. But when the listener has to change or accommodate
their paradigm to understand the lecture, there is a huge
problem. Lectures are not useless, they are far from optimal.

John M. Clement
Houston, TX



Wow! there isn't a Wikipedia entry on 'gain' in this
context yet. Can
anyone provide a reasonably reputable reference so that we
can ensure
we are on the same page? That said, the electronics
definition of gain
is reasonable: "a measure of the increase in signal
amplitude produced
by an amplifier, expressed as the ratio of output to input."

So my current definition of academic gain is: "a measure of the
increase in student knowledge obtained in a measured learning
environment, such as an academic course, expressed as the ratio of
output to input."

Electronic gain is frequency dependent, and academic gain can be
measured for every student. However, since students control their
destiny, it is more appropriate to talk about the
collective gain of
the entire cohort of students.

I am currently contemplating how to *measure* academic gain
objectively. Pre & post-testing is in vogue, but the pre-test is
different than the post-test. The latter being significantly more
challenging than the pre-test. Maybe a few questions from
the pre-test
on the post-test? Your thoughts?


The surrounding neighborhoods are filled with these
businesses, many of which move to larger quarters or are
bought out by
larger companies for the patents they acquire.
Many professors start their own business when they retire or during
their tenure or consult for others. They can pull in
the most promising students to work for them. Other
universities have the same type of things going on. I am
just a bit
more familiar with this particular one.

I'm very leery of this word... "gain". One person's gain
is anothers mediocrity. I am very proud of a student who
might struggle with a D for the first two quarters and then
progress
to a C+ for the final grade. Is his gain less than the student who
started with a "B" and ended up the top student? I would say the
first student gained more even though the grades may not show it.
Suppose the best student lazed his way through the first
half of the year with low B's then started working and got
the A which
comes naturally for him. Contrast with the first student I
described
above with D's and then worked his butt off to get a high C. There
are a lot of intangibles that tests cannot consider, but in
real life
we cannot use them to "measure" gain.

On Jun 21, 2013, at 5:12 PM, Philip Keller wrote:


BC said:

Plenty of successful start ups from drop outs, also.

--------------------------------------------------------------


The media glamorizes the successful drop-outs. Steve Jobs
didn't need no stinkin' degree -- why do you? But consider
the odds.
What % of MIT grads develop successful businesses (and not just
"start-ups", a phrase which seems to give credit for
winning just by
starting the race)? And what % of drop-outs do the same? And how
many orders of magnitude separate those two figures?
_______________________________________________
Forum for Physics Educators
Phys-l@phys-l.org
http://www.phys-l.org/mailman/listinfo/phys-l

_______________________________________________
Forum for Physics Educators
Phys-l@phys-l.org
http://www.phys-l.org/mailman/listinfo/phys-l
_______________________________________________
Forum for Physics Educators
Phys-l@phys-l.org
http://www.phys-l.org/mailman/listinfo/phys-l



_______________________________________________
Forum for Physics Educators
Phys-l@phys-l.org
http://www.phys-l.org/mailman/listinfo/phys-l
_______________________________________________
Forum for Physics Educators
Phys-l@phys-l.org
http://www.phys-l.org/mailman/listinfo/phys-l