Re: [Phys-L] From a Math Prof (physics BS major) at my institution ( math challenge)

[John Denker <jsd@av8n.com>]

On 02/25/2014 08:58 AM, Jeffrey Schnick wrote:
> I ran a Monte Carlo Octave function

Good!

> to investigate occurrences of sequences of 2 numbers in a row and, 
> occurrences of round numbers (10,20,30).  In a million sets of 21 
> rows of five numbers between 1 and 35 inclusive, I got 2784 cases in
>  which there were exactly 2 round numbers and only 133 cases in which
>  there were exactly 2 sequences of 2 numbers in a row.  In 0 out of a
>  million cases there were both 2 or fewer round numbers and 2 or 
> fewer sequences of 2 numbers in a row.  The first set given in this 
> thread met both of these conditions.

Those are meaningful numbers.  In general, stacking
subtest upon subtest is a way to come up with a more
sensitive overall test.

OTOH those numbers may be not quite as convincing as they 
might seem at first glance, for a couple of reasons.  For
starters, there is a "density of states" factor:
 -- The probability of getting the non-ideal number of
  round numbers is very small.
 -- However, the probability of getting exactly the
  ideal number of round numbers is also somewhat small.

Secondly, you can seriously fool yourself by stacking
subtest upon subtest if the tests have been /selected/
after you see the data.  Any number of people have been
wrongfully convicted on the basis of "statistical"
arguments of the form "what is the chance that a
person with the name Smith was walking down a street
named Jones while carrying a package of Skittles and
having the same blood-type as found at the crime scene."
Note the contrast:
 -- A /randomly/ selected stack of subtests will very
  soon rule out data set #2 as well as well as set #1.
 -- An /artfully/ selected stack of subtests can be used
  to "prove" anything you like, with very high "probability".

> Richard Tarara's hypothesis that the students that produced the 
> student-generated set of values would tend to shy away from
> sequences of numbers in a row, and mine that they would shy away from
> round numbers could be exactly wrong.

That is an excellent albeit negative point.  Similar words
apply to the Math Prof's intuition that a row of all primes
is a characteristically human foible.

To say the same thing another way, none of us here are
experts.

We can turn this into a positive as follows:  If somebody
wanted to pay me to become an expert, there are positive
steps I would take.  Among other things, I would collect
data from at least 1000 students and /find out/ what sort 
of foibles they exhibited.  That way I wouldn't need to
rely on uncalibrated intuition.

  Note that "intuition" is not /necessarily/ a bad thing.
  Kip Thorne once said that "Education is the process of
  cultivating your intuition."

> if someone offered you a million dollars to correctly pick the
> student-generated set

That's an interesting question.  I can answer with confidence,
because I have actually been in similar situations, usually
for somewhat lower stakes.

Here's one example:  I was in a business/economics/management
class with a bunch of other D-level managers.  The instructor
offered to pay $25.00 to anyone who could predict the outcome
of a "random" event with two possible outcomes.  The question
was, how much would we pay.  He opened the bidding at $10.00
and I immediately accepted.  Nobody else was willing to take
the bet at that price.  They all thought I was so stupid that 
I shouldn't even be allowed in the class.  They had all seen
too many "bar bets" where the guy offering the bet was going
to make money based on a counterintuitive outcome.

The "random" event involved tossing an upholstery tack
  http://www.kennedyhardware.com/images/P/t_17070.jpg
which might land "point up" or might land lying on its 
side.  I had no way of predicting what would happen based
on the physics.  I negotiated the terms of the deal to
make sure the instructor could not manipulate the outcome
/after/ I made my prediction.

So, he tossed his tack ... and then I stood up where
everybody could see me and tossed a coin.  I made my
prediction based on my coin toss.  The instructor about
fainted, because in many years of teaching the course
he'd never seen anybody do that.  He knew exactly what
I had done.  He made me explain it to the class.  It 
was the minimax strategy.  It guaranteed that I could 
not be punked.  It guaranteed that I had a 50/50 shot 
at the 25 bucks.  I had won the bet, which enhanced 
my credibility ... but I emphasized that even if I 
had lost, I would take the same bet again, as often 
as I possibly could.  (At this point I lost everybody
again.  If I had lost the bet, they would have taken
it as proof that nobody should take such a bet, ever.)

So here is my answer, possibly not the expected answer:
Given nothing more than the information in the original
post, *if* forced to choose, I would decide randomly.

The rationale is, there are too many ways of getting
punked.  For all I know, the "student" data set came 
from really smart students who had been trained to 
produce good-looking random numbers, while the
"lottery" data set had been /selected/ from years
of data to find some peculiar-looking stuff.

I know absolutely this is what I would do.  Been there,
done that, with lots of witnesses.

Under different conditions, for instance if I could
guarantee that I wasn't being punked, if I could be
sure that the guy providing the data was a friend
and not an adversary, *then* I would be willing to
look for messages hidden in the data.

  We have discussed this before in this forum:  The
  correct strategy for the Monty Hall game is very
  sensitive to whether you think Monty is your ally
  or your adversary.

It must be emphasized that /physics/ data is different
from numbers cooked up under the control of some
un-named person.  James Randi said that scientists
are easier to fool than children.  He knows what he
is talking about, since he's made a career of studying
how to fool people, in his capacity as both a professional
skeptic and a brilliant magician.

Scientists are easier to fool than children, because
they spend their lives dealing with data that comes
from trusted colleagues and/or directly from Mother
Nature, i.e. from sources that aren't trying to fool
you.

It must be emphasized that the data in question here 
is *not* physics data.

Also let's please keep in mind the original question:

On 02/17/2014 09:33 AM, Rauber, Joel wrote:

> > Can you tell which list is the student's list?

Given that question, one is allowed to answer no,
I cannot tell with any great confidence which is
which.

If you want to ask which would I choose given a
forced choice, you're allowed to ask that, but it's
different from the original question.

There are lots of angles to this:
 *) I agree that /sometimes/ there is a forced
  choice, and /sometimes/ one is forced to make
  decisions based lousy data, based on a very slim
  preponderance of the evidence.  I know how to do
  this.  For instance in a baseball game, the umpire
  has to call every pitch a ball or a strike, even 
  if it's a close call.  Similarly, if you are
  committed to landing at the Aspen airport, you
  have to use either runway 15 or runway 33;
  equivocation or vacillation is not an option.
 *) In other situations, there is overwhelming data
  and the decision is easy.
 *) In yet other situations, the evidence is so
  imperfect and the chance of being punked is so great
  that a nondeterministic strategy is in fact the
  correct strategy.  Note that imperfect evidence is
  not the same as /no/ evidence.  Example: Playing
  poker requires both understanding the odds /and/
  being somewhat unpredictable.  Ditto for real-world
  business and military strategy.
 *) Last but not least, a lot of things that look 
  like a forced choice really aren't.   In particular,
  if you can foresee a situation where you are forced
  to decide between two bad choices, or forced to make
  a choice based on lousy information where a wrong
  choice would be disastrous (lady or tiger), you
  should make sure you never get into such a situation.
  Example:  I am often asked what's the correct procedure
  if you stall the airplane 25 feet above the ground.
  Answer:  You need to make sure you never get into
  that situation.  There is no correct procedure once
  you get into that situation.  It is OK to stall 2500 
  feet above the runway, and it is OK to stall 0.025 
  feet above the runway, but if you stall 25 feet above 
  the runway Bad Things are going to happen, no matter 
  what you choose.  Plan ahead so you never get into 
  that situation.