Chronology Current Month Current Thread Current Date
[Year List] [Month List (current year)] [Date Index] [Thread Index] [Thread Prev] [Thread Next] [Date Prev] [Date Next]

Re: [Phys-l] riddle : entropy



Since BC has spilled the beans, or at least indicated where
the beans may be found, let me give my answer to the riddle.

First, a disclaimer: I absolutely do not mean to gloat
that I know the answer to this riddle. Depending on
details, you could easily have tricked me into giving
the wrong answer. This is not a contest and we're not
keeping score; please think of it as a shared learning
experience.

To say the same thing another way, this comes down to
an issue of being careful about the _range of validity_
of equation [1] (below). A lot of references (including
some of my writings!) have not been sufficiently careful.

As of now, you can find a fairly careful explanation of
the key ideas at
http://www.av8n.com/physics/thermo-laws.htm#sec-quantify-s
http://www.av8n.com/physics/thermo-laws.htm#sec-rho
but a week ago you couldn't.

Suppose we have a system that can be in either of two
microstates. Actually we have an ensemble of such
systems, and repeated measurements have observed the
system to be in microstate "A" with probability 1/3rd
and microstate "B" with probability 2/3rds.

Two questions:

1) What is the entropy of the system?

2) How sure are you that your answer is correct?

My answers are:
(1) the entropy is at least zero and at most ~0.92 bits.
(2) I can be certain that the entropy lies in that interval,
and I cannot be certain of any more-specific answer, based
on the given description of the situation.

The upper end of the solution-set is the "textbook" answer (at
least the introductory-level textbook answer) as several people
have noted. That is,
S = -(p_A log p_A + p_B log p_B) [1]
~ 0.92 bits

The lower end of the solution-set can arise in various ways.
Perhaps the simplest is to consider the *pure state*
|phi> = sqrt(1/3) |A> + sqrt(2/3) |B>

which has zero entropy. Repeated measurements in the
{|A>, |B>} basis give the observations mentioned in the
statement of the problem, but that is not the only possible
basis and those are not the only possible measurements.

The state |phi> can be described by the density matrix

[ 1/3 sqrt(2)/3 ]
[ ] [2]
[ sqrt(2)/3 2/3 ]

which, again, has zero entropy, in accordance with the
general formula
S := - Trace(rho log rho) [3]

This is also a "textbook" answer, but you might have to look
through graduate-level quantum mechanics books to find it.
Equation [1] can be seen as a corollary or special case of
equation [3], valid when the off-diagonal elements of the
density matrix vanish.

The nice thing about density matrices is that they permit
a basis-independent description of the situation. (Any
trace is independent of basis. This can be seen as a
consequence of the cyclic property of the trace.)

In the statement of the problem I communicated the diagonal
elements of the density matrix, but I was silent about the
off-diagonal elements. (I did give a warning that I hadn't
told everything!)

The high-entropy end of the solution-set is described by a
different density matrix, namely

[ 1/3 0 ]
[ ] [4]
[ 0 2/3 ]

This matrix is consistent with the statement of the problem,
but matrix [2] and everything in between are also consistent
with the statement of the problem.

============================================

You may be wondering whether this is relevant or worthwhile.

The quick-and-dirty approach is to take equation [1] as "the"
definition of entropy, and to restrict attention to the set of
problems where this gives the right answer. This is a huge
set of problems, but not quite the universal set.

1) From a pedagogical point of view, it is not unheard-of for
students to identify (and ask about!) situations where equation
[1] cannot be reliably applied, for example sp3 hybrid orbitals
or spin-echo situations. It's bad medicine to be relying on a
"definition" (such as equation [1]) that can be applied in two
different bases yielding two different answers. If you blindly
rely on equation [1] you will be faced with unanswerable questions.

Even if it is beyond the scope of the course to talk about
equation [3] in detail (or at all), it is nice to understand
that equation [1] is not the be-all and end-all. It is nice
to be able to explain at least qualitatively that
a) equation [1] is not invariant with respect to change of
basis;
b) the right answer /should/ be invariant with respect to
change of basis; and
c) there exists a generalization of equation [1] that deals
with this properly.


2) Not just for teaching, but for science and for life in general,
it is nice to know the limits of validity of the things we rely
on. This is /particularly/ important for scientific research:
it is hard to advance the frontiers if you don't know where the
frontiers are.