Homework (Was Measure of student understanding)

[John Clement <clement@HAL-PC.ORG>]

As far as the topic of homework and what works there is a significant body
of research that can be used to guide practice.  Also there are some
theoretical considerations that flow from this research.

Large scale studies for example using the TIMMS have shown that homework is
effective in improving scores on these tests.
However, studies at the level of the classroom have found that a significant
portion of homework is done incorrectly and this contributes to lower or
even negative learning gain.  The main problem is that when students get the
homework back they tend to concentrate on whether the answer is right or
wrong rather than the process they used to get the original answer.  This is
a well known problem that immediate feedback works much better than delayed
feedback.  Often students will just shrug off the bad grade and not really
go through the process necessary to understand the problem.  The original
thought that students put into the problem has evaporated by the time they
get the papers back.

An additional problem with homework is that it usually consists of simple
problems found at the end of the chapter.  These sorts of problems tend to
discourage thinking and encourage a formulaic solution.  When used heavily,
understanding is often depressed rather than enhanced.  This has been
attacked fairly successfully by the Heller's by having students do "rich
context" problems in groups in class.  Such problems are generally too
difficult to be done individually, but can be solved by a group.  Students
receive immediate feedback, and they are forced to use concepts rather than
formulas to solve the problems.  This method is considered to be a fairly
conventional approach compared to other PER methods.

Some very interesting methods have been developed which enhance the
usefulness of homework.  Web assignments can be automatically graded with
immediate feedback to the students.  The instructor is then given the scores
the next morning and can shape further instruction guided by good data.
This JIT or Just in Time pedagogy has shown good gain, and can be considered
to be an extension of conventional pedagogy.  The only problem with this
type of pedagogy is that it generally can not work well with rich context
problems.  In addition it is difficult to accurately grade non numeric
answers such as graphs, explanations, and pictures.  The latter have been
shown by research to be a vital component of understanding.

Homework can NOT be a substitute for hands on exploration in physics
courses.  Research by Lawson and others has shown that a learning cycle
approach of exploration, term definition, followed by application generally
works best.  Renner et al. have shown that all three are necessary for
better results, but the order can be changed for "formal operational"
thinkers.  Since all HS courses and most intro. college courses have a
significant fraction of "concrete operational" and transitional thinkers the
learning cycle approach is very important.  Other research by Renner has
shown that hands on exploration is vital for concrete operational thinkers
and they do not learn well from books.  This being said, Laws et al. have
found that homework when properly targeted can improve gain.  They supply
such homework as review exercises to be used after Workshop Physics and Real
Time Physics activities.  This would appear to be a much smaller effect than
is obtained by using their labs without the homework.

If we truly wish to improve instruction then we must have some way of
objectively measuring effectiveness.  With such measurements it is possible
to do experiments to test the effectiveness of various curricula changes.
FCI Gain is just such a measurement.  If we do not use objective
measurements then we are relying on anecdotal information.  While anecdotal
information is a reasonable guide to what might be going on, it has low
reliability.  There are some very good medical studies which seemed to show
good results from various procedures such as hormone therapy.  The anecdotal
reports were usually glowing.  However when controlled studies were done the
good effects sometimes evaporated or became very low.

Gain can be measured in many ways, but currently the FCI and FMCE are
currently the only measurements that we can use to compare mechanics
curriculum across courses.  Other tests have been developed for circuits and
thermo.  They show similar effects to the FCI and FMCE in that active
engagement curricula show much better gain than conventional teaching
methods.  Along with scores on such tests students have been interviewed to
see whether or not their understanding lines up with the tests.  The
interviews usually, but not always confirm that understanding lines up with
the test results.  I will agree that only the FCI is probably not by itself
a good indicator.  However it correlates well with other tests such as the
FMCE and improvements in other areas correlate with FCI scores.  Mazur
reports some gain in problem solving ability when he gets good FCI gain.
There is also data that shows that high FCI scores correlates well with
other evaluations of thinking ability.  Until we have better standard
evaluation tools the FCI and their brethren are necessary.  Incidentally the
research by Lawson and Renner did not involve any of the current physics
tests.

I have no quibbles with alternate methods of teaching as long as careful
objective measurements are being used.  Unfortunately most teaching does not
involve such methods to gauge its effectiveness.  There is a very extensive
research literature about science education and physics education in
particular.  All science teachers should be aware of what science education
research has found.  If you knew that your MD is not reading the current
journals in medicine would you continue to go to him/her?  I do not include
most articles published in either AJP or TPT as being good education
research articles.  While informative and even useful they generally do not
show objective measurements of how better learning resulted from the
particular reported activity.  Does your course have high effectiveness?  I
don't know, but do you know, and how do you know?  If you doubt the results
of PER then please do some experiments and report the research, or find some
good studies that are contrary to the PER studies.

One of the most important results to come out of science education research
is that large overall improvement is very difficult but not impossible to
get.  One factor alone is not adequate, but may be necessary.  For example
the steps in the learning cycle are common to improved learning (gain) and
seem to be necessary.  Combining this with carefully researched curricula
buys more learning.  No single factor such as homework can be the key to all
education problems.  I did not question anyone's course, just the idea that
graded homework could be the key to "true ""active physics.""". Has anyone
on this list experimented to see if there is a large difference in learning
between courses with graded or ungraded homework?

John M. Clement
Houston, TX



> It keeps coming back to--'Good Gain'.  The PER folks seem to measure
> everything against a couple diagnostic tests such as the FCI.
> What has yet
> to be shown (at least to me) is that the performance on such tests
> correlates to success as a physicist, a scientist, or a member of society.
> Until that can be done, then the teaching community has every right to be
> skeptical of any and all of the 50 or more major theories of learning.  I
> would prefer that those PER folk whose rhetoric tends to constantly be 'If
> you are not teaching the way we say, your courses are useless, you are
> useless, your students are learning nothing, ......." keep that
> rhetoric on
> the PER lists and off Phys-L and PhysShare.
>
> Rick
>
> *************************************************
> Richard W. Tarara
> Professor of Physics
> Saint Mary's College
> Notre Dame, IN 46556
> 219-284-4664
> rtarara@saintmarys.edu
>
> FREE PHYSICS INSTRUCTIONAL SOFTWARE
> www.saintmarys.edu/~rtarara/software.html
> NEW:  Photo Realistic Laboratory Simulations
> XP compatible updates now available.
> **********************************************************
> ----- Original Message -----
> From: "John Clement" <clement@HAL-PC.ORG>
>
>
> > Since teachers have been grading homework in conventional
> classes and have
> > not gotten much higher gain, this is probably not true.  Homework
> certainly
> > can have some effect, but it is small compared to the PER labs.
>  Lawson's
> > and other research has shown clearly that the learning cycle
> with hands on
> > experience is vital for students who are at the concrete operational
> level.
> > Work done in class is the most important aspect of achieveing good gain.
> > This has also been shown by the Heller's research.
> >
> > John M. Clement