[Phys-L] one variable at a time ... or not

[John Denker <jsd@av8n.com>]

Hi Folks --

Consider the famous "Twelve Coins" puzzle.  The obvious approach is
to weigh each coin separately, but this is very far from optimal.
Finding an optimal solution is an interesting challenge at the
middle-school level.

At the undergraduate level, one could compare Bevington's GRIDLS
versus GRADLS. The latter is far from optimal, but the former,
which changes only one variable at a time, is worse.

Therefore I find it quite strange to find statements in favor of
changing only one variable at a time, such as the appalling
references\1--6\ below, and many many others.

   The OVaaT approach is favored by non-experts, especially in
   situations where the data is plentiful and cheap.  It may be
   that the mental effort required to conduct a complex multi-
   factor analysis exceeds the cost of obtaining extra data, in
   which case OVaaT might make sense.

   OVaaT might make sense as a pedagogical /starting point/ but
   never as an ending point.

Meanwhile, OVaaT is vanishingly rare in professional experimental
and statistical situations.  Indeed, the fact that the data is
precious and needs careful analysis is probably /why/ the job was
assigned to The Professor and not to Gilligan.

Bottom line:  Please do not tell students -- or anybody else --
that scientists change only one variable at a time.


*** Appalling References:

\1\  http://ims.ode.state.oh.us/ode/ims/lessons/content/csc_lp_s05_ba_l07_i01_01.pdf
> This lesson helps students understand that it is best to test only
> one variable at a time

\2\  https://ed.sc.gov/scdoe/assets/file/agency/ccr/Standards-Learning/documents/ScienceInquirySupportGuide5-1.pdf
> *It is essential for students to* know that only testable questions,
> which are used to test one variable, are suitable for scientific
> investigations.
[emphasis in the original]

\3\  http://www.sdpb.sd.gov/KQInvestigator/documents/Standards.pdf
> change one variable at a time

\4\  https://quizlet.com/46903385/9th-grade-physical-science-flash-cards/
> Explain why an experiment should test only one variable at a time.
>
> Because you want to see how the experimental results change due to 
> only that one variable change

[In other words, that seems to be saying:
   Q: Explain why you should sit on a cactus.
   A: Because you want to sit on a cactus.

IMHO the logic doesn't seem very tight.]

\5\  https://quizlet.com/94595932/interactive-science-grade-5-chapter-1-lesson-2-flash-cards/
> True or false: A scientist should choose only one variable at a time 
> to change so that it is a fair test.
>
> true

\6\  _NSTA Pathways to the Science Standards_
_Second High School Edition_
https://books.google.com/books?isbn=0873552296  (page 44)
> Second, as teachers of science, we understand the most important
> principle of the experimental method: “Check one variable at a time.”
> This rule not only guides the simplest classroom experiments but also
> the most sophisticated statistical statistical techniques [.....]

[Even if that idea were true, it wouldn't rank as "most important".
And it's not true.  Not by a mile.]