Re: [Phys-l] BYU Virtual labs

["John Clement" <clement@hal-pc.org>]

> I've heard the Virtual ChemLab (the first/oldest of the suite) is very
> good
> and the propaganda says "there has been a 30 percent increase in student
> performance in organic chemistry labs due to the virtual lab."
>
> Also featured on PBS Teaching NOW!:  http://chemlab.byu.edu/node/41
It seems that the program does not play on my computer so I can not figure
out what that advertising is based on.  OK, so I looked at two of the
journal articles.  One reports that students "feel" more competent, but
there is little in the way of hard evidence about student learning and
understanding of chemistry.  Ok, it makes labs safer, and allows repeated
exploration, but the level of actual learning was not shown by strong
objective testing.

Statements like "We believe that because Virtual ChemLab removes the
kinesthetic details, students can remember, understand, and focus
on chemical theory better than they would be able to without the presence of
the simulation." do not give one confidence that there is actual
improvement.

The articles are short on statistics and long on opinions.  The use of
opinion surveys to see whether students like the labs is not very valid.
> From PER research we know that student gain scores often go up at the same
time that student "satisfaction" goes down.  There are no details about the
type of testing used or any gain scores given.  OK, they used quizzes and
conventional tests, but what do they measure?  The PER research generally
first constructs a research based test and then uses it to evaluate results.
The inorganic article used teacher made tests which are not the most
reliable.  The actual gains while probably real were not that great.  What
was the effect size or normalized gain????  Just statistical significance by
itself has little meaning.  A statistically significant result can still be
educationally insignificant.  Perhaps I missed the effect size so if someone
else finds it, please point it out to me.

I found the evidence not that convincing that the virtual labs were the
source of the gains.  Did they replace cookbook labs with better writeups?
If that was changed, then conventional labs might work better with just
better writeups.

I am fairly certain, that changing the course to a learning cycle format
with or without virtual labs would buy a huge amount in student learning.
Indeed Priscilla Laws said that some classes could use their methods without
any technology and still achieve gains.  So I see this as a technological
fix which ignores the big problems in education and fixes some of the
smaller ones.  It is not so much the technology used, but HOW it is used
that makes the big difference.  An example here is that the ILDs are often
known to be effective, even if the instructor just tells the students the
answers, without any technology or physical demonstrations.  Mazur uses a
method which may seldom include physical demonstrations, but has a strong
minds on approach.

John M. Clement
Houston, TX