Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

Re: [Phys-L] Indicators of quality teaching (Was:MOOC: EdxOffers Mechanics course by Prof.Walter Lewin)

Anyone who has read the research and is familiar with Hake's posts should
know what gain is.
One standard way of specifying gain is the effect size. This is standard in
many education research papers. Everyone who calls them selves a teacher,
instructor, or professor should learn about these sorts of things.

Effect size = (post - pre)/STD
An effect size of 1 is considered enormous and many studies do not get
effect sizes larger than .5. Many PER practicioners get effect sizes
greater than 1.

But this definition of gain has the problem that it is skewed by the size of
the pre-test and also is highly dependent on class homogeneity. Just a
straight post-pre has a large dependence on the pre test. So Hake came up
with Hake gain or normalized gain.

Normalized Gain = (post-pre)/(max score - pre)
This is relatively independent of the pre test score and is how most PER
results are quoted. You can convert it to a percentage, and that is how I
like to quote it. It indicates what percentage of unknown material was
learned during the class.

The pre and post-test are the same, or they need to be correlated. I have
even given the same test 3 weeks apart and gotten exactly the same student
scores, so using the same test is OK as long as there is a time lag and the
test has been disguised and never discussed. One researcher told me that 2
weeks is enough elapsed time to give the same test. Incidentally Laws et al
have found that when a subject is taught using PER that the post test scores
continue to rise for 2 weeks. This is in contrast to conventional teaching
where the test scores droop after the lesson.

Many PER practicioners get gain between 50% and 70%, while the average
standard traditional course gets only 0% to 25%. Hake has shown indirectly
that the average HS course only gets around 12%. The latest study of gain
vs teaching method vs instructor shows that traditional teaching gets around
13% or so. But instructors that used the research based method achieved
around 50%, independent of instructor.

I have shown that the normalized gain depends strongly on the Lawson test
score. Which means that gain is strongly dependent on the thinking skills
of the students. Phillips and Colletta have also shown this.

Similar results come from courses other than mechanics using conceptual
exams. In addition similar results come from math courses. The inquiry
math course shows higher gain than the traditional course. Data goes as far
back as the early 90s. So we have had the results for over 20 years, and
still people are not changing what they do in class. Also if you want to
know how well the student learn you have to measure it using an evaluation
that is standard, and can be compared to other classes.

Incidentally energetic can actually be a deficit. When you put stuff into a
class to make it interesting, that is what students look at and remember.
But they don't remember the point of the interesting stuff. Mazur showed
that when you put people in illustrations, the students spend the most time
looking at the people and not the physics. So low key to the point may work
better. But we have absolutely no data either way. It is a belief that
energetic means better, but that may not be true. People think they can
just look at a class and see how well they are learning, but that is not
true. Remember that MDs were practicing medicine by observation for
millenia. They did come up with some good conclusions as to what can be
effective, but they also came up with squirrely ones like blood letting. It
took research to get them to the current state, and initially they resisted
it by continuing to do operations in their blood-stained prestigious
uniforms. Even now some MDs resist routine hand washing.

Observation may point to a possible effect, but in the end you have to
measure it. We do that in physics, so why not in education?

As to lecture, I did point out that they may be more effective when the
lecturer and listener are on the same page. But when the listener has to
change or accommodate their paradigm to understand the lecture, there is a
huge problem. Lectures are not useless, they are far from optimal.

John M. Clement
Houston, TX



Wow! there isn't a Wikipedia entry on 'gain' in this context
yet. Can anyone provide a reasonably reputable reference so
that we can ensure we are on the same page? That said, the
electronics definition of gain is reasonable: "a measure of
the increase in signal amplitude produced by an amplifier,
expressed as the ratio of output to input."

So my current definition of academic gain is: "a measure of
the increase in student knowledge obtained in a measured
learning environment, such as an academic course, expressed
as the ratio of output to input."

Electronic gain is frequency dependent, and academic gain can
be measured for every student. However, since students
control their destiny, it is more appropriate to talk about
the collective gain of the entire cohort of students.

I am currently contemplating how to *measure* academic gain
objectively. Pre & post-testing is in vogue, but the pre-test
is different than the post-test. The latter being
significantly more challenging than the pre-test. Maybe a few
questions from the pre-test on the post-test? Your thoughts?


The surrounding neighborhoods are filled with these
businesses, many of which move to larger quarters or are
bought out by larger companies for the patents they acquire.
Many professors start their own business when they retire or
during their tenure or consult for others. They can pull in
the most promising students to work for them. Other
universities have the same type of things going on. I am
just a bit more familiar with this particular one.

I'm very leery of this word... "gain". One person's gain
is anothers mediocrity. I am very proud of a student who
might struggle with a D for the first two quarters and then
progress to a C+ for the final grade. Is his gain less than
the student who started with a "B" and ended up the top
student? I would say the first student gained more even
though the grades may not show it.
Suppose the best student lazed his way through the first
half of the year with low B's then started working and got
the A which comes naturally for him. Contrast with the first
student I described above with D's and then worked his butt
off to get a high C. There are a lot of intangibles that
tests cannot consider, but in real life we cannot use them to
"measure" gain.

On Jun 21, 2013, at 5:12 PM, Philip Keller wrote:


BC said:

Plenty of successful start ups from drop outs, also.

--------------------------------------------------------------


The media glamorizes the successful drop-outs. Steve Jobs
didn't need no stinkin' degree -- why do you? But consider
the odds. What % of MIT grads develop successful businesses
(and not just "start-ups", a phrase which seems to give
credit for winning just by starting the race)? And what % of
drop-outs do the same? And how many orders of magnitude
separate those two figures?
_______________________________________________
Forum for Physics Educators
Phys-l@phys-l.org
http://www.phys-l.org/mailman/listinfo/phys-l

_______________________________________________
Forum for Physics Educators
Phys-l@phys-l.org
http://www.phys-l.org/mailman/listinfo/phys-l
_______________________________________________
Forum for Physics Educators
Phys-l@phys-l.org
http://www.phys-l.org/mailman/listinfo/phys-l