Re: [Phys-L] Timing Statistic + Moiré patterns

[John Denker <jsd@av8n.com>]

On 11/14/18 10:46 AM, Paul Nord wrote:

> Thanks for the phase plot.  That makes it very easy to explain to my
> colleagues how particular values of angular velocity may show a large sigma
> and others may show small sigmas and be consistently high or low.

:-)

> In fact, the lesson is about conservation of angular momentum. 

OK!  This goal should be achievable.
Angular momentum is good.  Conservation is good.

> This is a lab for college freshmen taking either the college physics or the
> university physics course.  This statistical analysis is pretty far beyond
> them at this point.  [...] Our lab is designed to make students think critically about
> error analysis and how well any measured value describes a physical
> system. 

Here's my strategic suggestion:  Let's focus on the physics.
I renew my suggestion of having two disks on separate bearings
on the same axis, coupled by eddy-current damping.

1) Spin them both in the same direction.  We are sure that
 angular momentum is /conserved/ but that's not the same
 as /constant/ because some is obviously leaking across
 the boundary of the system.
   https://www.av8n.com/physics/conservation-continuity.htm

2) Then spin them in opposite directions, so that the main
 contribution is transfer of angular momentum from one disk
 to the other.

3) Distinguish real from apparent non-conservation.  In step
 (2) there will be some apparent non-conservation, so the
 question is how to explain it.  The simplest explanation is
 that angular momentum really is conserved, but there was some
 unobserved leakage.  The quantitative question is whether the
 unobserved leakage in (2) is consistent with the observed
 leakage in (1).

This does *not* require any fancy statistics.

Assuming the leakage is large compared to the roundoff error,
you don't need to obsess over the roundoff error.  So let's
check that assumption.

Observed deceleration = 17 degrees per second per second.

Eddy currents should be able to transfer most of 5000 degrees
per second from one disk to the other in 10 seconds.  The
roundoff error on the final phase is 1 degree, leaving the
velocity uncertain by 1 degree per centisecond, aka 100
degrees per second (unaveraged) or 10 degrees per second
(if you average the last 10 points).  So I think we're OK.

See the discussion of *correlated* averaging, in the next
section.

The more ambitious students should be encouraged to repeat
the experiment and blow on one of the disks, to get a
qualitative estimate of the significance of air currents.
That's not proof of anything, but it's a clue about what
you would need to do to improve the experiment.

== Correlations == 

> Most of their clients are high school teachers who
> would like a device which reports a reliable angular velocity in a small
> number of time steps.  Yes, for you and I it would be interesting to get
> precision time values for each digital step in the internal encoder.  But
> most people won't want to see that level of detail.  Is there something in
> between?  Perhaps precise times at a few uniformly-spaced positions in the
> rotation?

There are ways of surviving with the existing non-timestamped
data.  You can proceed as follows:

Start by using a decent model of roundoff error:
	φ_0 is uncertain by ±0.5 degrees, worst case.
	φ_1 is uncertain by ±0.5 degrees, worst case.
	φ_2 is uncertain by ±0.5 degrees, worst case.
	φ_3 is uncertain by ±0.5 degrees, worst case.
	et cetera

These are worst-case uncertainties, which means they are *not*
Gaussian distributed.  Also they are *not* statistically
independent.  In particular:

	(φ_1  - φ_0) is uncertain by ±1 degree, worst case.
	(φ_10 - φ_0) is uncertain by ±1 degree, worst case.

As a consequence, averaging the velocity over a group of
10 intervals does not cause the uncertainty to go down by
a factor of √10 but rather a full factor of 10.  That is
related to the fact that successive φ residuals (relative
to a straight-line model) are highly anticorrelated.

  This is very much contrary to a colleague's guess that
  was mentioned at the beginning of this thread.

In some sense this is helpful, because it makes it easier
to reduce the noise by a factor of 10.  You can implement
(using a spreadsheet) a simple moving average that gives
you a new average every centisecond, averaging over the
local 10-centisecond window.  (Averaging over toooo many
samples will break the straight-line approximation, so
don't get carried away.)

In another sense, we are still very much playing defense,
because most of the standard tools are not reliable.  In
particular, *every* method that uses anything resembling
least squares is invalid.  This includes most "fitting"
methods and "regression" methods.  That's because they are
predicated on Gaussian-distributed independent residuals.

  Explanation:  A Gaussian is roughly exp(-x^2).  If you have
  two /independent/ Gaussian events, the probability is
  exp(-x^2)*exp(-y^2) aka exp(-x^2 - y^2).  So maximizing
  the log probability is just minimizing the sum of the
  squares, nothing more and nothing less.  But if the events
  are not independent and/or not Gaussian, the whole thing
  is nonsense.

===========================================================
The rest is mostly in the category of bad ideas, either
because they are too complicated or just plain wrong physics.
But let's discuss them, for completeness and for background.

> In this lab we hope they discover that the error in the
> measurement of the radius of the disk contributes more to the error in the
> calculation of the moment of inertia than does the error in the measurement
> of the mass. 

Maybe that's achievable with this setup, or maybe not.  I'm
a couple of breakthroughs shy of knowing how to accomplish it.
It's a fine thing to "hope" for, and you can expect it to be
true based on theory ... but actually measuring it is likely
to be far beyond the reach of beginning-level students.

I am 1000% supportive of comparing calculated uncertainties to
actual measured uncertainties.  All too often, an uncertainty is
calculated and then /not/ compared to anything resembling ground
truth.  This allows all sorts of misconceptions to proliferate.

The problem is, doing it right is hard.  You've heard of signal-to-noise
ratio?  In this situation we are dealing with a noise-to-noise ratio,
which is a big problem when the number of data points (N) is small.
In the best of situations, the variances are distributed according to
a χ^2 distribution, which is quite broad when N is small.

What's worse, this is verrry far from an ideal situation.  The
statistical uncertainty is buried under roundoff error.  Roundoff
errors are not random.  That means all your intuition about
probability and uncertainty goes out the window, along with all
your mathematical tools.

>  The angular velocity measurements and their errors are used
> to calculate the angular momentum before and after a change to the moment
> of inertia.  After the full error propagation they calculate a Z-value and
> we ask them to state, in good scientific terms, whether angular momentum
> was conserved.  Generally these Z-values are between 0.5 and 1.5.  Though a
> few students have poor technique and touch the system while it's spinning.
> (The Z-value becomes smaller when they screw up at least one of the 5
> trials... and that's a whole other statistical discussion.)

As things stand, that's not a doable calculation.  Z-values and χ^2
values quantify the scatter relative to what it "should" have been,
provided all the sources of uncertainty were (a) random, (b) Gaussian,
and (c) recognized and accounted for.  In this situations, none of
those provisos is true.

> If I wanted to give the students a reasonable description of the
> capabilities of the Vernier transducer, is it reasonable to tell them: The
> precision of the measurement looks like +/- x but you shouldn't trust
> that.  We've studied the device more carefully and you should assume that
> the reported angular velocity measurement is really +/- y?

Well, if you did that, their Z-values would be closer to 100 or 1000
than to 1.

> Is there a recommendation for a modification to the internal magic of
> Vernier's software?

My recommendations are:

a) Vernier should timestamp their observations, rather than snapshotting
 the integer phase at preordained intervals.  I haven't taken apart the
 encoder, but I imagine it has a chip inside to capture the data and
 format it for transmission to the base unit.  A current-production
 ATMEGA328 chip contains an onboard microsecond timer, and sells for
 less than two bucks, which seems small compared to the price of the
 finished product ($169 after discount).

b) If the user doesn't like the timestamp approach, the higher-level
 software can implement a mode that applies the round() function to
 the t-values.  But it's hard to imagine anybody choosing this mode.
 Most people understand that ω = Δφ/Δt and there is no law that says
 every Δt has to be the same.

c) Or there could be a mode that uses interpolation to convert
 [analog time, discrete phase] to [discrete time, analog phase].
 But you need the high-resolution timestamped data as input to
 the interpolation.

d) Failing that, in an emergency you can use a HMM to infer what
 the phase must be doing between samples.  However, this is like
 eating broth with a fork.  Spoons are easier and better and
 ought to be readily available.

I am informed (off list) that Vernier has no intention of doing
any of the above.