Re: [Phys-L] testing all the /things that could happen/ ... was: "safe fails"

[John Denker <jsd@av8n.com>]

On 04/26/2017 03:38 PM, rjensen@ualberta.ca wrote:
> The idea of a "safe fail" is an experiment or exercise designed with a
> high likelihood of failure in the first (few) iteration(s). The
> metacognitive goals are for learners
>  * to recognize failure as intrinsic to science (and life)
>  * to realize that failure is not a reason for punishment 
>  * to be able to identify errors and failures
>  * to learn to troubleshoot methods
>  * to learn from failure, and not be afraid of it
>  * <others?>
>
> I'm looking for examples of experiments and exercises that integrate
> safe fails.


This is a super-important lesson, for reasons discussed below (*).


1) The poster child for this is a /maze/.

When running a maze, it is imperative to give up on one sub-goal after
another, if you want to make progress toward the main overall goal.

Here is an exercise.  It's an app that runs in your browser.  Free for
all.
  https://www.av8n.com/physics/glorpy-maze.html

It has a number of amusing features:
  a) You have no idea where the goal is, or even what it looks
   like, until you have found it.  This is closely analogous
   to how research works.
  b) The Wallbanger algorithm that you may have learned in
   grade school will *not* solve typical instances of this
   maze, and is not even an efficient approximate method.
  c) You can only obtain information locally.  If you want an
   overall view, you have to earn it by exploring.

For more details about maze-running and the analogy to how research
is done, see:
  https://www.av8n.com/physics/research-maze.htm


2) If you want something more chemistry-ish, ask people to identify
the marble or limestone chip in a pile of quartz and feldspar chips.
If you set up the scenario properly, it's really hard to tell just
by looking.  On the other hand, it is cheap and easy to grab rocks
at random and see whether vinegar makes them fizz.  You can adjust
the limestone/other ratio in the obvious way.

You may be able to grab suitable rocks from your back yard, but if
not, you can get them from the local building materials yard.  The
going rate for marble chips is double digits per ton.  If you want
a bucket of rocks for school, they may give 'em to you for free.
(Bring your own bucket.)


3) If you want an easy-to-understand authentic example where trial
and error is by far the best technique known, consider finding
large prime numbers (thousands of bits, or more).  The procedure
is to randomly pick a number in the right general range and apply
a bunch of primality tests.  If it fails (which it probably will),
pick another.  This is a 100% genuine real-world procedure, widely
used e.g. in modern cryptography.


4) There are many other stochastic algorithms, some with tremendous
practical applications:
  https://www.google.com/search?q=stochastic+algorithm
  https://www.google.com/search?q=simulated+annealing
  https://www.google.com/search?q=genetic+algorithm
  https://www.google.com/search?q=fuzzing


5) The connection between experimentation, randomness, entropy,
and pure information is particularly clear in the celebrated
"12 coins" puzzle:
  https://www.av8n.com/physics/twelve-coins.htm

and in the 20 questions game, if done right:
  https://www.av8n.com/physics/twenty-questions.htm


======================

Beware that terms such as "error" and "failure" are emotionally
laden.  There are plenty of students who believe that failure is
Wrong with a capital W, in the same way that stealing is Wrong.
It is somewhere between difficult and impossible to desensitize
students to this issue.  These feelings are stored in the same
place as religious convictions, so tread lightly.

My recommendation is to avoid such terms and instead to speak
of randomness, uncertainty, scatter in the data, width of the
distribution, exploring the maze, backtracking out of blind
alleys, et cetera.


(*) A great many students have been taught for years that "the
scientific method" requires getting invested in a specific guess
before doing the experiment.  This is of course madness;  the more
specificity the worse, and the more investment the worse.  Instead
one should systematically make a list of /things that could happen/.
The 12 coins puzzle and the 20 questions game help make this point:
There are no "fail" outcomes, just /things that could happen/.

It is super-important to make this point, but beware that you are
contradicting years of indoctrination.
    https://www.av8n.com/physics/scientific-methods.htm#sec-poster
One of the great perks of being a scientist is that you don't
have to guess and you don't have to fail;  instead you just
consider the various /things that could happen/.  Some students
will understand this immediately and joyfully, but others will
find it an uphill slog.


=====

Also:
It is impossible to formalize uncertainty, randomness, entropy,
information, etc. without a decent grasp of basic probability.
Beware that even though probability is a required subject in
typical math education standards at the grade school level on
up, very often it gets short shrift or less.  So you can plan
on doing some remedial instruction in this area.  An excellent
reference for the modern (post-1933) approach to probability
can be found in volume II of Apostol's _Calculus_ book (which
is well worth having for at least a dozen reasons).