An experimental value A1 is to be compared with a theoretical A.
Suppose A=100 and A1=101. Is the difference significant? One
can NOT answer this question without knowing how accurately
A1 was measured. The A was presumably calculated with so
many digits of accuracy that the error is practically zero.
Now think about the chi-square method of deciding about the
significance of a discrepancy between the two distributions.
Suppose we compare observed o1,o2,o3 and o4 with the
theoretically expected e1, e2, e3 and e4. We know how this
is often done for n data points. First we calculate
chi_sqr=sum( (o(i)-e(i))^2 / o(e) ) <-- (i from 1 to n)
Then we compare this value with what is found in a statistical
table for a given number of degrees of freedom, Df=n-1, and
for a desired level of confidence, for example 0.95 or 0.99.
If the reference number is larger than what was calculated
from the above formula then we say differences are not
significant; otherwise we say they are significant.
Note that errors of measurements of observed quantities are
NOT specified. In other words we are making a conclusion
on the basis of an analysis which does not depend on sizes of
errors. There must be some hidden assumptions in this analysis.
What are they? It seems to me that such analysis is probably
not valid in many situations. Any comments?
I am asking because for the distribution of counts, (observed
versus Poissonian) the chi-sqr value is about 19 while the
reference value from a table is about 15. The r*dt=12.60*0.3
=3.78 does give me the smallest chi_sqr (as expected) but this
value is not as small as it should be to say that differences are
not significant at the level of confidence of 95%. So, strictly
speaking, I must say that the discrepancy is real. But I do not
think that this is true.
Let me add that the same kind of analysis on the two
distributions of waiting times (experimental and theoretical)
gives a very clear indication that the curves are practically
identical. How can it be? Both waiting times and countings
per fixed time intervals are obtained from the same
experimental data (7004 recordings). Without the chi-sqr
analysis I would not hesitate to say that experimental data are
in very good agreement with the theoretical Poisson distribution.
Ludwik Kowalski