Here's a first cut at a proper discussion of scientific
methods. Comments?
1) There is no such thing as "the" scientific method.
Science uses many methods. There will never be a pat
answer to the question "what is science". The very notion
that there could be a pat answer bespeaks an attachment
to rote learning that is incompatible with scientific
thinking.
2) One of the goals of science is to make useful predictions.
3) A scientific prediction does not need to be exact to
be useful.
4) Sometimes it is possible to make useful predictions,
and sometimes not. If you are asked to predict the
exact total shown on a particular roll of a pair of fair
dice, you will be wrong at least 5/6ths of the time.
But if you can get into a situation where the payoff is
greater than 6:1, you can make some useful predictions,
and you can make money on average.
5) Scientists use words like rule, law, hypothesis,
formula, and algorithm almost interchangeably, to
describe the process for making predictions (although
there are slight variations in connotations). Scientists
use the word theory to mean almost the same thing, with
the connotation of something grander, namely a _system_
of rules giving a relatively coherent overview of a
topic. Non-scientists use the word theory to refer to
a mere speculation. So it is best to avoid the word
theory when talking to non-scientists.
6) Mathematical results are validated by formality and
rigor: If A then B, according to the rules of logic.
Physical-science results may be validated by appeal to
experiment: We observe A. Generally science is a
complex lattice of facts and rules, combining observations
and logic.
7) Predictive rules generally have a limited domain
of applicability. To state the rule without stating
its limits of validity is improper.
8) An established rule may be refined. It may
be supplemented by other rules so as to extend the
domain of validity. It may be supplemented by exceptions
to improve the accuracy. However a rule with too
many caveats and exceptions is likely to be not only
inconvenient but unreliable. Occam's razor and all that.
9) Occasionally a rule may be supplanted entirely by
a simpler and better rule. (It is considered very poor
form to gripe about the imperfections in an established
rule, unless you've got something better to offer.)
10) An important part of scientific thinking is being
able to recognize non-scientific thinking. Examples
include:
-- elementary logic errors, such as circular reasoning,
non sequitur, and many others
-- selecting the data (it is not right to select
tendentious anecdotes from a mass of data)
-- other misuses of probability
-- appeal to authority
-- ad_hominem arguments
-- improperly weighted voting (a thousand pieces of weak
evidence should not outweigh one piece of strong evidence)
-- et cetera.