Re: Entropy (was very long)

[David Bowman <David_Bowman@GEORGETOWNCOLLEGE.EDU>]

Regarding some of John Denker's comments on my long post:

DB:
>   For instance, a thoroughly
> shuffled deck of cards has essentially the same thermodynamic entropy
> as a well-ordered deck (assuming, of course, both decks have the same
> temperature and are subject to the same external macroscopic
> environment).  OTOH, a cold deck of cards has less thermodynamic entropy
> than a warm one regardless of how the decks were shuffled or not.

JD:
> Shuffling the deck increases its entropy by a few bits.  Cooling the deck
> (in the usual way) decreases its entropy by something like 10^23 bits.

That is why I hedged my comment by saying the shuffled deck had
"essentially the same thermodynamic entropy as a well-ordered deck.
 ^^^^^^^^^^^

The weasel word was to account for a *possible* unobservable
contribution to the ~21st-23rd significant digit or so.  In fact, whether
or not the shuffled deck's entropy is greater or not than the unshuffled
one by a few bits depends on the details of the macroscopic
specification of the two decks.  For instance, if the actual
card sequence in the shuffled desk was included in its macro-level
description then the few bits needed to specify the sequence would
not contribute to the entropy.

> ...

DB:
>  The composite object(s) that has(have) this minimal possible
> complexity value over the whole ensemble is/are said to be the
> most-ordered object(s).  The difference between the actual complexity of
> each of the ensemble members and this minimal complexity value defines
> (by my definition of the term) the 'disorder' of each such composite
> object.

JD:
> That's an interesting definition, but it has bugs that I can't immediately
> see how to resolve.  Conceptually, the bug has to do with neglecting the
> internal disorder of the component subsystems.

Oh?

JD:
> As an illustration of this conceptual bug, consider a crystal built by
> assembling a vast number of micro-crystals.  If I do it right, the entropy
> of the whole is much less than the sum of the entropy of the parts (because
> knowing the microstate in one part let's you make predictions about the
> others).

I agree.

JD:
>       Similar remarks apply to a stack of 10^20 brand-new unshuffled
> decks of cards.

That's a lot of cards.

JD:
> AFAICT the suggested definition of disorder yields a
> strongly negative value in such cases;  I consider this a bug.

Not to my way of thinking.  It is true that complexity is not an
exactly additive measure over subsystems in the same way that entropy is
not exactly additive.  But I don't see what this has to do with
anything related to my definition.  My definition would consider the
complexity of the most ordered stack of 10^20 cards as much less than
10^20 times the complexity of one such card.  This goes for the
other arrangements as well.  This is because most of card-manufacturing
algorithm can be looped.  Almost all the knowhow for producing one card
can be used to manufacture the rest of the cards.  All that is needed
extra is a repeat count index and some specialized routines for making
the different patterns of face markings.  The complexity of a very
complicated arrangement of the stack would include all of the
complexity of making the cards in the first place *and* would include,
too, the complexity of specifying the arrangement of the stack, which
for most "random" such arrangements would require on the order of
~ log_2((10^20)!) bits of extra information to encode *if* we assumed
that the cards were all macroscopically distinguishable.  If we only
had 52 kinds of cards and didn't distinguish among various cards of the
same face value then the amount of this extra information would be
drastically reduced.  In either event it is only the amount of this
extra 'arrangement information' that constitutes the disorder in
the stack. The disorder measure subtracts off all the information
required to make the cards in the first place which is common to the
the most ordered stack and to all the other arrangements.  IOW, for
a given arrangement A of the stack

[disorder]_A = [total complexity]_A - [total complexity]_M   where

M is the most ordered arrangment over the ensemble of all possible
stacks.  It's complexitity is given by:

[total complexity]_M = MIN{A,[total complexity]_A}

Since [total complexity]_M is the minimum possible value over the
whole ensemble of A values, subtracting it from one particular member
of this ensemble will always result in a non-negative number.

I hope this helps explain further my definition for disorder.

David Bowman
David_Bowman@georgetowncollege.edu