\magnification=\magstep1 \pretolerance 2000 \baselineskip=14pt \overfullrule=0pt \catcode`@=11 \def\vfootnote#1{\insert\footins\bgroup\baselineskip=12pt \interlinepenalty\interfootnotelinepenalty \splittopskip\ht\strutbox % top baseline for broken footnotes \splitmaxdepth\dp\strutbox \floatingpenalty\@MM \leftskip\z@skip \rightskip\z@skip \spaceskip\z@skip \xspaceskip\z@skip \textindent{#1}\footstrut\futurelet\next\fo@t} \skip\footins 20pt plus4pt minus4pt \def\footstrut{\vbox to2\splittopskip{}} \catcode`@=12 \def\buildurel#1\under#2{\mathrel{\mathop{\kern0pt #2}\limits_{#1}}} \hfill November 1, 1994 %sent to Physics Today \vglue 24pt plus 12pt minus 12pt \centerline{\bf MICROSCOPIC REVERSIBILITY AND MACROSCOPIC BEHAVIOR:} \centerline{\bf PHYSICAL EXPLANATIONS AND MATHEMATICAL DERIVATIONS} \medskip \centerline{Joel L. Lebowitz} \centerline{Departments of Mathematics and Physics} \centerline{Rutgers University, New Brunswick, NJ 08903} \bigskip \bigskip \bigskip \bigskip \bigskip \noindent {\bf Abstract} \medskip The observed general time-asymmetric behavior of macroscopic systems---embodied in the second law of thermodynamics---arises naturally from time-symmetric microscopic laws due to the great disparity between macro and micro-scales. More specific features of macroscopic evolution depend on the nature of the microscopic dynamics. In particular, short range interactions with good mixing properties lead, for simple systems, to the quantitative description of such evolutions by means of autonomous hydrodynamic equations, e.g.\ the diffusion equation. These deterministic time-asymmetric equations accurately describe the observed behavior of {\it individual} macro systems. Derivations using ensembles (or probability distributions) must therefore, to be relevant, hold for almost all members of the ensemble, i.e.\ occur with probability close to one. Equating observed irreversible macroscopic behavior with the time evolution of ensembles describing systems having only a few degrees of freedom, where no such typicality holds, is misguided and misleading. \bigskip \bigskip \vfill \eject \line{\hfill\vbox{\hsize3.25truein\baselineskip10pt {\it ``The equations of motion in abstract dynamics are perfectly reversible; any solution of these equations remains valid when the time variable $t$ is replaced by $-t$. Physical processes, on the other hand, are irreversible: for example, the friction of solids, conduction of heat, and diffusion. Nevertheless, the principle of dissipation of energy is compatible with a molecular theory in which each particle is subject to the laws of abstract dynamics.''}}} \hfill W. Thomson, (1874)[1] \noindent {\bf Introduction} Given the success of the statistical approach, pioneered by James Maxwell, William Thomson (later Lord Kelvin) and made quantitative by Ludwig Boltzmann, in both explaining and predicting the observed behavior of macroscopic systems on the basis of their reversible microscopic dynamics, it is quite surprising that there is still so much confusion about the ``problem of irreversibility''. I attribute this to the fact that the originality of these ideas made them difficult to grasp. When put into high relief by Boltzmann's precise and elegant form of his famous kinetic equation and H-theorem they became ready targets for attack. The confusion created by these misunderstandings and by the resulting ``controversies'' between Boltzmann and some of his contemporaries, particularly Ernst Zermelo, has been perpetuated by various authors who either did not understand or did not explain adequately the completely satisfactory resolution of these questions by Boltzmann's responses and later writings. There is really no excuse for this, considering the clarity of the latter. In Erwin Schr\"odinger's words, ``Boltzmann's ideas really give an understanding" of the origin of macroscopic behavior. All claims of inconsistencies (known to me) are in my opinion wrong and I see no need to search for alternate explanations of such behavior---at least on the non-relativistic classical level . I highly recommend some of Boltzmann's works [2], as well as the beautiful 1874 paper of Thomson [1] and the more contemporary references [3-7], for further reading on this subject; see also [8] for more details on the topics discussed here. Boltzmann's statistical theory of nonequilibrium (time-asymmetric, irreversible) behavior associates {\it to each microscopic state of a macroscopic system}, be it gas, fluid or solid, a number $S_B$: the ``Boltzmann entropy" of that state [4]. This entropy agrees (up to terms negligible in the size of the system) with the macroscopic thermodynamic entropy of Clausius {\it when} the system is in equilibrium. It also coincides {\it then} with the Gibbs entropy $S_G$, which is defined not for individual microstates but for statistical ensembles or probability distributions (in a way to be described later). The agreement extends to systems in local equilibrium. However, unlike $S_G$, which does not change in time even for ensembles describing (isolated) systems not in equilibrium, e.g.\ fluids evolving according to hydrodynamic equations, $S_B$ typically increases in a way which {\it explains} and describes qualitatively the evolution towards equilibrium of macroscopic systems. This behavior of $S_B$ is due to the separation between microscopic and macroscopic scales, i.e.\ the very large number of degrees of freedom involved in the specification of macroscopic properties. It is this separation of scales which enables us to make definite predictions about the evolution of a {\it typical individual realization} of a macroscopic system, where, after all, we actually observe irreversible behavior. As put succinctly by Maxwell [9] ``the second law is drawn from our experience of bodies consisting of an immense number of molecules. \dots it is continually being violated, \dots, in any sufficiently small group of molecules \dots . As the number \dots is increased \dots the probability of a measurable variation \dots may be regarded as practically an impossibility''. The various ensembles commonly used in statistical mechanics are to be thought of as nothing more than mathematical tools for describing behavior which is practically the same for ``almost all'' individual macroscopic systems in the ensemble. While these tools can be very useful and some theorems that are proven about them are very beautiful they must not be confused with the real thing going on in a single system. To do that is to commit the scientific equivalent of idolatry, i.e.\ substituting representative images for reality. Moreover, the time-asymmetric behavior manifested in a single typical evolution of a macroscopic system distinguishes macroscopic irreversibility from the mixing type of evolution of ensembles which are caused by the {\it chaotic} behavior of systems with but a few degrees of freedom, e.g.\ two hard spheres in a box. To call the latter irreversible is, therefore, confusing. %This {\it typicality} is very robust---we can %expect to see unusual events, such as gases unmixing themselves, only if we %wait for times inconceivably long compared to the age of the universe. To %quote from Boltzmann response [insert] The essential qualitative features of macroscopic behavior can be understood on the basis of the incompressible flow in phase space given by Hamilton's equations. They are not dependent on assumptions, such as positivity of Lyapunov exponents, ergodicity, mixing or ``equal a priori probabilities,'' being {\it strictly} satisfied. Such properties are however important for the quantitative description of the macroscopic evolution which is given, in many cases, by time-asymmetric autonomous equations of hydrodynamic type. These can be derived (rigorously, in some cases) from reversible microscopic dynamics by suitably scaling macro and micro units of space and time and then taking limits in which the ratio of macroscopic to microscopic scales goes to infinity [10]. (These limits express in a mathematical form the physics arising from the very large ratio of macroscopic to microscopic scales.) Using the law of large numbers then shows that these equations describe the behavior of almost all individual systems in the ensemble, not just that of ensemble averages, i.e.\ the dispersion goes to zero in the scaling limit. Such descriptions also hold, to a high accuracy, when the macro/micro ratio is finite but very large. They are however clearly impossible when the system contains only a few particles. The existence and form of such hydrodynamic equations depends on the nature of the microscopic dynamics. In particular, instabilities of trajectories induced by chaotic microscopic dynamics play an important role in determining many features of macroscopic evolution. A simple example in which this can be worked out in detail is provided by the Lorentz gas. This consists of a macroscopic number of non-interacting particles moving among a periodic array of fixed convex scatterers arranged in the plane so that there is a maximum distance a particle can travel between collisions. The chaotic nature of the microscopic dynamics, which leads to an approximately isotropic local distribution of velocities, is directly responsible for the existence of a simple autonomous deterministic description, via a diffusion equation, for {\it typical} macroscopic particle density profiles of this system [10]. Another example is the description via the Boltzmann equation of the density in the six dimensional position and velocity space of a macroscopic dilute system of hard spheres [7], [10]. I use these examples, despite their highly idealized nature, because here all the mathematical $i$'s have been dotted. They thus show {\it ipso facto}, in a way that should convince even (as Mark Kac put it) an ``unreasonable" person, not only that there is no conflict between reversible microscopic and irreversible macroscopic behavior but also that, {\it for essentially all initial microscopic states consistent with a given nonequilibrium macroscopic state}, the latter follows from the former---in complete accord with Boltzmann's ideas. Boltzmann's analysis was of course done in terms of classical Newtonian mechanics and I shall use the same framework for this article. The situation is in many ways similar in quantum mechanics where reversible incompressible flow in phase space is replaced by unitary evolution in Hilbert space. In particular I do not believe that quantum measurement is a {\it new} source of irreversibility. Such assertions in effect ``put the cart before the horse''. Real measurements on quantum systems are time-asymmetric because they involve, of necessity, systems with very large number of degrees of freedom whose irreversibility can be understood using natural extensions of classical ideas [11], [13]. There are however also some genuinely new features in quantum mechanics relevant to our problem. First, to follow the classical analogy directly one would have to associate a macroscopic state to an arbitrary wave function of the system, which is impossible as is clear from the Schr{\"o}dinger cat paradigm [12] (or paradox). Second, quantum correlations between separated systems arising from wave function entanglements lead to the impossibility, in general, of assigning a wave function to a subsystem ${\cal S}_1$ of a system $\cal S$ in a definite state $\psi$ even at a time when there is no direct interaction between ${\cal S}_1$ and the rest of $\cal S$, and this makes the idealization of an isolated system much more problematical in quantum mechanics than in classical theory. These features of quantum mechanics require careful analysis to see how they affect the irreversibility observed in the real world. An in depth discussion is not only beyond the scope of this article but would also require some new ideas and quite a bit of work which is yet to be done. I refer the reader to references [11--14] for a discussion of some of these questions from many points of view. I will also, in this article, completely ignore relativity, special or general. The phenomenon we wish to explain, namely the time-asymmetric behavior of spatially localized macroscopic objects, has certainly many aspects which are the same in the relativistic (real) universe as in a (model) non-relativistic one. This means of course that I will not even attempt to touch the deep conceptual questions regarding the nature of space and time itself which have been much discussed recently in connection with reversibility in black hole radiation and evaporation [14]. These are beyond my competence and indeed it seems that their resolution may require new concepts which only time will bring. I will instead focus on the problem of the origin of macroscopic irreversibility in the simplest idealized classical context. The Maxwell-Thomson-Boltzmann resolution of this problem in these models does, in my opinion, carry over essentially unchanged to real systems. \bigskip \noindent {\bf The Problem of Macroscopic Irreversibility} Consider a macroscopic system evolving in time, as exemplified by the schematic snapshots of a binary system, say two different colored inks, in the four frames in Figure 1. The different frames in this figure represent pictorially the two local concentrations of the components at different times. {\it Suppose} we know that the system was isolated during the whole time of picture taking and we are asked to identify the time order in which the snapshots were taken. The obvious answer, based on experience, is that time increases from 1a to 1d---any other order is clearly absurd. Now it would be very simple and nice if this answer could be shown to follow directly from the microscopic laws of nature. But this is not the case, for the microscopic laws, as we know them, tell a different story: \ if the sequence going from left to right is permitted by the microscopic laws, so is the one going from right to left. This is most easily seen in classical mechanics where the complete microscopic state of an isolated classical system of $N$ particles is represented by a point $X=({\bf r}_1, {\bf v}_1, {\bf r}_2, {\bf v}_2, \dots, \ {\bf r}_N, {\bf v}_N)$ in its phase space $\Gamma$, ${\bf r}_i$ and ${\bf v}_i$ being the position and velocity of the $i$th particle. Now a snapshot in Fig.\ 1 clearly does not specify completely the microstate $X$ of the system; rather each picture specifies a coarse grained description of $X$, which we denote by $M(X)$, the macrostate corresponding to $X$. For example, if we imagine that the (one liter) box in Fig.\ 1 is divided into a billion little cubes, then the macrostate $M$ could simply specify (within some tolerance) the fraction of particles of each type in every cube $j$, $j=1$, $\dots$, $10^9$. To each macrostate $M$ there corresponds a very large set of microstates making up a region $\Gamma_M$ in the phase space $\Gamma$. In order to specify properly the region $\Gamma_M$ we need to know also the total energy $E$, and any other {\it macroscopically relevant}, e.g.\ additive, constants of the motion (also within some tolerance). While this specification of the macroscopic state clearly contains some arbitrariness, this need not concern us unduly here. All the qualitative statements we are going to make about the time evolution of macrostates $M$ are sensibly independent of its precise definition as long as there is a large separation between the macro and microscales. Let us consider now the time evolution of microstates which underlies that of the macrostates $M(X)$. They are governed by Hamiltonian dynamics which connects a microstate $X(t_0)$ at some time $t_0$, to the microstate $X(t)$ at any other time $t$. Let $X(t_0)$ and $X(t_0 + \tau)$, ~ $\tau > 0$, be two such microstates. Reversing (physically or mathematically) all velocities at time $t_0 + \tau$, we obtain a new microstate. If we now follow the evolution for another interval $\tau$ we arrive at a microstate at time $t_0 + 2 \tau$ which is just the state $X(t_0)$ with all velocities reversed. We shall call $RX$ the microstate obtained from $X$ by velocity reversal, $RX = ({\bf r}_1, -{\bf v}_1, {\bf r}_2, - {\bf v}_2, \dots , {\bf r}_N, -{\bf v}_N).$ Returning now to the snapshots shown in the figure it is clear that they would remain unchanged if we reversed the velocities of all the particles; hence if $X$ belongs to $\Gamma_M$ then also $RX$ belongs to $\Gamma_M$. Now we see the problem with our definite assignment of a time order to the snapshots in the figure: that a macrostate $M_1$ at time $t_1$ evolves to another macrostate $M_2$ at time $t_2 = t_1 + \tau$, $\tau > 0$, means that there is a microstate $X$ in $\Gamma_{M_1}$ which gives rise to a microstate $Y$ at $t_2$ with $Y$ in $\Gamma_{M_2}$. But then $RY$ is also in $\Gamma_{M_2}$ and following the evolution of $RY$ for a time $\tau$ would produce the state $RX$ which would then be in $\Gamma_{M_1}$. Hence the snapshots depicting $M_a$, $M_b$, $ M_c$ and $M_d$ in Fig. 1 could, as far as the laws of mechanics (which we take here to be the laws of nature) go, correspond to a sequence of times going in either direction. It is thus clear that our judgement of the time order in Fig. 1 is not based on the dynamical laws of evolution alone; these permit either order. Rather it is based on experience: \ one direction is common and easily arranged, the other is never seen. {\it But why should this be so}? \bigskip \noindent {\bf Boltzmann's Answer} \medskip The above question was first raised and the answer developed by theoretical physicists in the second half of the nineteenth century when the applicability of the laws of mechanics to thermal phenomena was established by the experiments of Joule and others. The key people were Maxwell, Thomson and Boltzmann. As already mentioned I find the 1874 article by Thomson an absolutely beautiful exposition containing the full qualitative answer to this problem. This paper is, as far as I know, never referred to by Boltzmann or by latter writers on the subject. It would or should have cleared up many a misunderstanding. I can only hope (but do not really expect) that my article will do better. Still I will try my best to say it again in more modern (but less beautiful) language. The answer can be summarized by a quote from Gibbs which appears (in English) on the flyleaf of Boltzmann's Lectures on Kinetic Theory, Vol. 2, [15] (in German): ``In other words, the impossibility of an uncompensated decrease of entropy seems to be reduced to improbability [16].'' This statistical theory can be best understood by associating to each macroscopic state $M$ and {\it thus to each phase point $X$ giving rise to $M$}, a ``Boltzmann entropy'', defined as $$S_B (M) = k \log \ |\Gamma_M|, \eqno(1)$$ \noindent where $k$ is Boltzmann's constant and $|\Gamma_M|$ is the phase space volume associated with the macrostate $M$, i.e.\ $|\Gamma_M|$ is the integral of the time invariant Liouville volume element ($\prod\limits^N_{i=1}d^3{\bf r}_i\; d^3{\bf v}_i$) over $\Gamma_M$. ($S_B$ is defined up to additive constants, see [4].) Boltzmann's stroke of genius was to make a direct connection between this microscopically defined function $S_B(M(X))$ and the thermodynamic entropy of Clausius, $S_{eq}$, which is a macroscopically defined, operationally measurable (up to additive constants), extensive property of macroscopic systems in {\it equilibrium}. For a system in equilibrium having a given energy $E$ (within some tolerance) volume $V$ and particle number $N$, Boltzmann showed that $$S_{eq} (E,V,N) = N s_{eq} (e,n) \simeq S_B(M_{eq}), \quad e = E/N,\; n = N/V,\eqno(2)$$ \noindent where $M_{eq} (E,V,N)$ is the equilibrium macrostate (corresponding to $M_d$ in Fig.\ 1). By the symbol $\simeq$ we mean that for large $N$, such that the system is really macroscopic, the equality holds up to terms negligible when both sides of equation (2) are divided by $N$ and the additive constant is suitably fixed. It is important that the cells used to define $M_{eq}$ contain many particles, i.e.\ that the macroscale be very large compared with the microscale. Having made this identification it is natural to use Equation (1) to also define (macroscopic) entropy for systems not entirely in equilibrium and thus identify increases in such entropy with increases in the volume of the phase space region $\Gamma_{M(X)}$. This identification explains in a natural way the observation, embodied in the second law of thermodynamics, that when a constraint is lifted from an isolated macroscopic system, it evolves toward a state with greater entropy. To see how the explanation works, imagine that there was initially a wall dividing the box in Fig.\ 1 which is removed at time $t_a$. The phase space volume available to the system without the wall is fantastically enlarged: If the system in fig.\ 1 contains 1 mole of fluid in a 1-liter container the volume ratio of the unconstrained region to the constrained one is of order $2^N$ or $10^{10^{20}}$, roughly the ratio $|\Gamma_{M_d}| / |\Gamma_{M_a}|$. We can then expect that when the constraint is removed the dynamical motion of the phase point $X$ will with very high ``probability'' move into the newly available regions of phase space, for which $|\Gamma_{M}|$ is large. This may be expected to continue until $X(t)$ reaches $\Gamma_{M_{eq}}$ corresponding to the system now being in its unconstrained equilibrium state. After that time we can expect to see only small fluctuations from macroscopic equilibrium---typical fluctuations being of order of the square root of the number of particles involved. It should be noted here that an important ingredient in the whole analysis is the constancy in time of the Liouville volume of sets in the phase space $\Gamma$. Without this invariance the connection between phase space volume and probability would be impossible or at least very problematic. Of course, if our isolated system remains isolated forever, Poincar{\'e}'s Recurrence Theorem tells us that the system phase point $X(t)$ would have to come back very close to its initial value $X(t_a)$, and do so again and again. But these Poincar{\'e} recurrence times are so enormous (more or less comparable to the ratio of $|\Gamma_{M_d}|$ to $|\Gamma_{M_a}|$) that when Zermelo brought up this objection to Boltzmann's explanation of the second law, Boltzmann's response [17] was as follows: ``Poincar{\'e}'s theorem, which Zermelo explains at the beginning of his paper, is clearly correct, but his application of it to the theory of heat is not. \dots Thus when \dots Zermelo concludes, from the theoretical fact that the initial states in a gas must recur---without having calculated how long a time this will take---that the hypotheses of gas theory must be rejected or else fundamentally changed, he is just like a dice player who has calculated that the probability of a sequence of 1000 one's is not zero, and then concludes that his dice must be loaded since he has not yet observed such a sequence!''\footnote {$^{a)}$}{It is remarkable that in the same paper Boltzmann also wrote ``likewise, it is observed that very small particles in a gas execute motions which result from the fact that the pressure on the surface of the particles may fluctuate''. This shows that Boltzmann completely understood the cause of Brownian motion ten years before Einstein's seminal papers on the subject. Surprisingly he never used this phenomenon in his arguments with Ostwald and Mach about the reality of atoms.} Thus not only did Boltzmann's great insights give a microscopic interpretation of the mysterious thermodynamic entropy of Clausius; they also gave a natural generalization of entropy to nonequilibrium macrostates $M$, and with it an explanation of the second law of thermodynamics---the formal expression of the time-asymmetric evolution of macroscopic states occurring in nature. \bigskip \noindent{\bf The Use of Probability} \medskip Boltzmann's ideas are, as Ruelle [6] says, at the same time simple and rather subtle. They introduce into the ``laws of nature'' notions of probability, which, certainly at that time, were quite alien to the scientific outlook. Physical laws were supposed to hold without any exceptions, not just almost always and indeed no exceptions were (or are) known to the second law; nor would we expect any, as Richard Feynman [3] rather conservatively says, ``in a million years''. The reason for this, as recognized by Maxwell, Thomson and Boltzmann, is that, for a macroscopic system, the fraction of microstates for which the evolution leads to macrostates with larger $S_B$ is so close to one (in terms of their Liouville volume) that such behavior is exactly what should be seen to ``always'' happen. As put by Boltzmann [17], ``Maxwell's law of the distribution of velocities among gas molecules is by no means a theorem of ordinary mechanics which can be proved from the equations of motion alone; on the contrary, it can only be proved that it has very high probability, and that for a large number of molecules all other states have by comparison such a small probability that for practical purposes they can be ignored.'' In present day mathematical language we say that such behavior is {\it typical}, by which we mean that the set of microstates $X$ in $\Gamma_{M_{a}}$ for which it occurs have a volume fraction which goes to $1$ as $N$ increases. Thus in Fig.\ 1 the sequence going from left to right is typical for a phase point in $\Gamma_{M_a}$ while the one going from right to left has ``probability'' approaching zero with respect to a uniform distribution in $\Gamma_{M_d}$, for $N$ tending towards infinity. Note that Boltzmann's argument does not really require the assumption that over very long periods of time the macroscopic system should be found in different regions $\Gamma_M$, i.e.\ in different macroscopic states $M$, for fractions of time {\it exactly} equal to the ratio of $|\Gamma_M|$ to the total phase space volume specified by its energy. Such behavior, which can be considered as a mild form of Boltzmann's ergodic hypothesis, mild because it is only applied to those regions of the phase space representing macrostates $\Gamma_M$, seems very plausible in the absence of constants of the motion which decompose the energy surface into regions with different macroscopic states. It appears even more reasonable when we take into account the lack of perfect isolation in practice which will be discussed later. Its implication for ``small fluctuations'' from equilibrium is certainly consistent with observations. (The stronger form of the ergodic hypothesis also seems like a natural assumption for macroscopic systems. It gives a simple derivation for many equilibrium properties of macro systems.) \bigskip \noindent {\bf Initial Conditions} Once we accept the statistical explanation of why macroscopic systems evolve in a manner that makes $S_B$ increase with time, there remains the nagging problem (of which Boltzmann was well aware) of what we mean by ``with time''. Since the microscopic dynamical laws are symmetric, the two directions of the time variable are {\it a priori} equivalent and thus must remain so {\it a posteriori} [18]. In particular if a system with a nonuniform macroscopic density profile, such as $M_b$, at time $t_b$ in Fig.\ 1 had a microstate that is typical for $\Gamma_{M_b}$, then almost surely its macrostate at both times $t_b + \tau$ and $t_b - \tau$ will be like $M_c$. This is inevitable: Since the phase space region $\Gamma_{M_b}$ corresponding to $M_b$ at some time $t_b$ is invariant under the transformation $X \to RX$, it must make the same prediction for $t_b - \tau$ as for $t_b + \tau$. Yet experience shows that the assumption of typicality at time $t_b$ will give the correct behavior only for times $t > t_b$ and not for times $t < t_b$. In particular, given just $M_b$ and $M_c$, we have no hesitation in ordering $M_b$ before $M_c$. If we think further about our ordering of $M_b$ and $M_c$, we realize that it seems to derive from our assumption that $M_b$ is itself so unlikely that it must have evolved from an initial state of even lower entropy like $M_a$. From an initial microstate typical of the macrostate $M_a$, which can be readily created by an experimentalist, we get monotonic behavior of $S_B$ with the time ordering $M_a$, $M_b$, $M_c$ and $M_d$. If, by contrast, the system in Fig.\ 1 had been completely isolated for a very long time compared with its hydrodynamic relaxation time, then we would expect to always find it in its equilibrium state $M_d$ (with possibly some small fluctuations around it). Presented instead with the four pictures, we would (in this very, very unlikely case) have no basis for assigning an order to them; microscopic reversibility assures that fluctuations from equilibrium are typically symmetric about times at which there is a local minimum of $S_B$. In the absence of any knowledge about the history of the system before and after the sequence of snapshots presented in Fig.\ 1, we use our experience to conclude that the low-entropy state $M_a$ must have been an initial prepared state. In the words of Roger Penrose [5]: ``The time-asymmetry comes merely from the fact that the system has been {\it started off} in a very special (i.e.\ low-entropy) state, and having so started the system, we have watched it evolve in the {\it future} direction''. The point is that a microstate corresponding to $M_b$ (at time $t_b$) which comes from $M_a$ (at time $t_a$) must be {\it atypical} in some respects of points in $\Gamma_{M_b}$. This is so because, by Liouville's theorem, the set $\Gamma_{ab}$ of all such phase points has a volume $|\Gamma_{ab}| \leq |\Gamma_{M_a}|$ that is {\it very much smaller} than $|\Gamma_{M_b}|$. This need not however prevent the overwhelming majority of points in $\Gamma_{ab}$ (with respect to Liouville measure on $\Gamma_{ab}$ which is the same as Liouville measure on $\Gamma_a$) from having future macrostates like those typical of $\Gamma_b$---while still being very special and unrepresentative of $\Gamma_{M_b}$ as far as their past macrostates are concerned. This sort of behavior is what is explicitly proven by Lanford in his derivation of the Boltzmann equation [7], and is implicit in all derivation of hydrodynamic equations [10]; see also [19]. To see intuitively the origin of such behavior we note that for systems with realistic interactions the domain $\Gamma_{ab}$ will be so convoluted that it will be ``essentially dense'' in $\Gamma_b$, so that any slight thickening of it will cover all of $\Gamma_{M_b}$. It is therefore not unreasonable that their future behavior, as far as macrostates go, will be unaffected by their past history. (This can be worked out completely for a model macroscopic system in which the (large) $N$ noninteracting atoms are each specified not by $({\bf r, v})$ but by ${\bf \sigma} = (\dots, \sigma_{-2}, \sigma_{-1}; \sigma_0, \sigma_1,\dots)$, a doubly infinite sequence of zeros and ones (equivalently a point in the unit square). Their discrete time dynamics is that of a shift to the left $(T{\bf \sigma})_i = \sigma_{i+1}$ (equivalently the baker's transformation). If we define ``velocity reversal'' by $(R{\bf \sigma})_i = \sigma_{-i-1}$ and the macrostate $M({\bf \sigma})$ by the $k$ values, $M({\bf \sigma}) = (\sigma_0+\sigma_{-1}, \sigma_1+\sigma_{-2}, \sigma_2+\sigma_{-3}, \dots , \sigma_{k-1}+\sigma_{-k})$ then a little thought shows that the future behavior of typical points in $\Gamma_{M_{ab}}$ is indeed as described above.) \bigskip \noindent {\bf Origin of Low-Entropy States} The creation of low-entropy initial states poses no problem in laboratory situations such as the one depicted in Fig.\ 1. Laboratory systems are prepared in states of low Boltzmann entropy by experimentalists who are themselves in low-entropy states. Like other living beings, they are born in such states and maintained there by eating nutritious low-entropy foods, which in turn are produced by plants using low-entropy radiation coming from the Sun. That was already clear to Boltzmann as may be seen from the following quote [20]: ``The general struggle for existence of living beings is therefore not a fight for the elements---the elements of all organisms are available in abundance in air, water, and soil---nor for energy, which is plentiful in the form of heat, unfortunately untransformably, in every body. Rather, it is a struggle for entropy that becomes available through the flow of energy from the hot Sun to the cold Earth. To make the fullest use of this energy, the plants spread out the immeasurable areas of their leaves and harness the Sun's energy by a process as yet unexplored, before it sinks down to the temperature level of our Earth, to drive the chemical syntheses of which one has no inkling as yet in our laboratories. The products of this chemical kitchen are the object of the struggles in the animal world''. Note that while these experimentalists have evolved, thanks to this source of low entropy energy, into beings able to prepare systems in particular macrostates with low values of $S_B(M)$, like our state $M_a$, the total entropy $S_B$, including the entropy of the experimentalists and that of their environment, must always increase: There are no Maxwell demons. The low entropy of the solar system is also manifested in events in which there is no human participation---so that, for example, if instead of Fig.\ 1 we are given snapshots of the Shoemaker-Levy comet and Jupiter before and after their collision, then the time direction is again obvious. We must then ask what is the origin of this low entropy state of the solar system. In trying to answer this question we are led more or less inevitably to cosmological considerations of an initial ''state of the universe'' having a very small Boltzmann entropy. To again quote Boltzmann [10]: ``That in nature the transition from a probable to an improbable state does not take place as often as the converse, can be explained by assuming a very improbable initial state of the entire universe surrounding us. This is a reasonable assumption to make, since it enables us to explain the facts of experience, and one should not expect to be able to deduce it from anything more fundamental''. That is, the universe is pictured as having been ``created'' in an initial microstate $X$ typical of some macrostate $M_0$ for which $|\Gamma_{M_0}|$ is a very small fraction of the ``total available'' phase space volume. In Boltzmann's time there was no physical theory of what such an initial state might be and Boltzmann toyed with the idea that it was just a very large, very improbable, fluctuation in an eternal universe which spends most of its time in an equilibrium state. Richard Feynman argues convincingly against such a view [3]. In the current big bang scenario it is reasonable, as Roger Penrose does in [5], to take as initial state the state of the universe just after the big bang. Its macrostate would then be one in which the energy density is approximately spatially uniform. Penrose estimates that if $M_f$ is the macrostate of the final ``Big Crunch'', having a phase space volume of $|\Gamma_{M_f}|$, then $|\Gamma_{M_f}|/|\Gamma_{M_0}| \approx 10^{10^{123}}$. The high value of $|\Gamma_{_M{_f}}|$ compared with $|\Gamma_{M_0}|$ comes from the vast amount of phase space corresponding to a universe collapsed into a black hole, see Fig.\ 2. I do not know whether these initial and final states are reasonable, but in any case one has to agree with Feynman's statement [3] that ``it is necessary to add to the physical laws the hypothesis that in the past the universe was more ordered, in the technical sense, than it is today...to make an understanding of the irreversibility.'' ``Technical sense'' clearly refers to the initial state of the universe $M_0$ having a smaller $S_B$ than the present state. Once we accept such an initial macrostate $M_0$, then the initial microstate can be assumed to be typical of $\Gamma_{M_0}$. We can then apply our statistical reasoning to compute the typical evolution of such an initial state, i.e. we can use phase-space-volume arguments to predict the future behavior of macroscopic systems---but not to determine the past. As put by Boltzmann [2], ``we do not have to assume a special type of initial condition in order to give a mechanical proof of the second law, if we are willing to accept a statistical viewpoint\dots if the initial state is chosen at random \dots entropy is almost certain to increase.'' \bigskip \noindent {\bf Irreversibility and Macroscopic Stability} Of course mechanics itself doesn't preclude having a microstate $X$ for which $S_B(M(X_t))$ decreases as $t$ increases. An experimentalist could, {\it in principle}, reverse all velocities of the system in Fig.\ 1b, and then watch the system unmix itself. It seems however impossible to do so in practice: Even if he/she managed to do a perfect job on the velocity reversal part, as occurs (imperfectly) in spin echo experiments [21] , we would not expect to see the system in Fig.\ 1 go from $M_b$ to $M_a$. This would require that {\it both} the velocity reversal and system isolation be {\it absolutely perfect}. The reason for requiring such perfection now and not before is that while the macroscopic behavior of a system with microstate $Y$ in the state $M_b$ {\it coming} from a microstate $X$ typical with respect to $\Gamma_{M_{\scriptstyle a}}$ is {\it stable} against perturbations as far as its future is concerned it is very {\it unstable} as far as its {\it past} (and thus the future behavior of $RY$) is concerned (see Figs.\ 3 and 4). (I am thinking here primarily of situations like those depicted in Fig.\ 1 where the macroscopic evolution is described by the stable diffusion equation. However, even in situations, such as that of turbulence, where the forward macroscopic evolution is chaotic, i.e.\ sensitive to small perturbations, all evolutions will still have increasing Boltzmann entropies in the forward direction. For the isolated evolution of the velocity reversed microstate, however, one has decreasing $S_B$ while the perturbed ones can be expected to have, at least after a very short time, increasing $S_B$. So even in macroscopically ``chaotic'' regimes the forward evolution of $M$ is in this sense much more stable than the backward one. Thus in turbulence all forward evolutions are still described by solutions of the same Navier-Stokes equation while the backward macroscopic evolution for a {\it perfectly isolated} fluid and for an actual one will have no connection with each other.) The above analysis is based on the very reasonable assumption that almost any perturbation of the microstate $Y$ will tend to make it more typical of its macrostate $M(Y)$, here equal to $M_b$. The perturbation will thus not interfere with behavior typical of $\Gamma_{M_{b}}$. The forward evolution of the unperturbed $RY$ is on the other hand, by construction, heading towards a smaller phase space volume and is thus untypical of $\Gamma_{M_{b}}$. It therefore requires ``perfect aiming'' and will very likely be derailed by even small imperfections in the reversal and/or tiny outside influences. After a {\it very short} time in which $S_B$ decreases the imperfections in the reversal and the ``outside'' perturbations, such as one coming from a sun flare, a star quake in a distant galaxy (a long time ago) or from a butterfly beating its wings [6], will make it increase again. This is somewhat analogous to those pinball machine type puzzles where one is supposed to get a small metal ball into a particular small region. You have to do things just right to get it in but almost anything you do gets it out into larger regions. For the macroscopic systems we are considering, the disparity between relative sizes of the comparable regions in the phase space is unimaginably larger. In the absence of any ``grand conspiracy'', the behavior of such systems can therefore be confidently predicted to be in accordance with the second law (except possibly for very short time intervals). Sensitivity to small perturbations in the entropy decreasing direction is commonly observed in computer simulations of systems with ``realistic'' interactions where velocity reversal is easy to accomplish but unavoidable roundoff errors play the role of perturbations. It is possible, however, to avoid this effect in simulations by the use of discrete time integer arithmetic. This is clearly illustrated in Figs.\ 3 and 4. The latter also shows how a small perturbation which has no effect on the forward macro evolution completely destroys the time reversed evolution. This point is very clearly formulated in the 1874 paper of Thomson [1]: ``Dissipation of energy, such as that due to heat conduction in a gas, might be entirely prevented by a suitable arrangement of Maxwell demons, operating in conformity with the conservation of energy and momentum. If no demons are present, the average result of the free motions of the molecules will be to equalize temperature-differences. If we allowed this equalization to proceed for a certain time, and then reversed the motions of all the molecules, we would observe a disequalization. However, if the number of molecules is very large, as it is in a gas, any slight deviation from absolute precision in the reversal will greatly shorten the time during which disequalization occurs. In other words, the probability of occurrence of a distribution of velocities which will lead to disequalization of temperature for any perceptible length of time is very small. Furthermore, if we take account of the fact that no physical system can be completely isolated from its surroundings but is in principle interacting with all other molecules in the universe, and if we believe that the number of these latter molecules is infinite, then we may conclude that it is impossible for temperature-differences to arise spontaneously. A numerical calculation is given to illustrate this conclusion.'' Thomson goes on to say: ``The essence of Joule's discovery is the subjection of physical phenomena to dynamical law. If, then, the motion of every particle of matter in the universe were precisely reversed at any instant, the course of nature would be simply reversed for ever after. The bursting bubble of foam at the foot of a waterfall would reunite and descend into the water; \dots Boulders would recover from the mud the materials required to rebuild them into their previous jagged forms, and would become reunited to the mountain peak from which they had formerly broken away. And if also the materialistic hypothesis of life were true, living creatures would grow backwards, with conscious knowledge of the future, but no memory of the past, and would become again unborn. But the real phenomena of life infinitely transcend human science; and speculation regarding consequences of their imagined reversal is utterly unprofitable. Far otherwise, however, is it in respect to the reversal of the motions of matter uninfluenced by life, a very elementary consideration of which leads to the full explanation of the theory of dissipation of energy.'' \vglue 24pt plus 12pt minus 12pt \noindent {\bf Boltzmann vs. Gibbs Entropies} The Boltzmannian approach, which focuses on the evolution of a particular macroscopic system, is conceptually different from the Gibbsian approach, which focuses primarily on ensembles. This difference shows up strikingly when we compare Boltzmann's entropy---defined in (1) for a microstate $X$ of a macroscopic system---with the more commonly used (and misused) entropy $S_G$ of Gibbs, defined for an ensemble density $\rho(X)$ by $$S_G (\{\rho \}) = -k {\textstyle \int} \rho (X) [\log \rho(X)]dX. \eqno (3)$$ \noindent Here $\rho(X)dX$ is the probability (obtained some way or other) for the microscopic state of the system to be found in the phase space volume element $dX$ and the integral is over the phase space $\Gamma$. Of course if we take $\rho(X)$ to be the generalized microcanonical ensemble associated with a macrostate $M$, $$\rho_M(X) \equiv \left \{ \matrix { | \Gamma_M |^{-1}, & {\rm if}\ X \in \Gamma_M \cr 0, \hfill & {\rm otherwise}\hfill \cr } \right. ,\eqno (4)\, $$ then clearly, $$S_G(\{\rho_M \}) = k\log |\Gamma_M | = S_B(M). \eqno (5)$$ Generalized microcanonical ensembles like $\rho_M(X)$, or their canonical version, are commonly used to describe systems in which the particle density, energy density and momentum density vary slowly on a microscopic scale {\it and} the system is, in each small macroscopic region, in equilibrium with the prescribed local densities, i.e.\ when we have local equilibrium [10]. In such cases $S_G(\{\rho_M\})$ and $S_B(M)$ agree with each other, and with the macroscopic hydrodynamic entropy. Note however that unless the system is in complete equilibrium and there is no further systematic change in $M$ or $\rho$, the time evolutions of $S_B$ and $S_G$ are {\it very} different. As is well known, it follows from the fact that the volume of phase space regions remains unchanged under the Hamiltonian time evolution (even though their shape changes greatly) that $S_G(\{ \rho \})$ never changes in time as long as $X$ evolves according to the Hamiltonian evolution, i.e.\ $\rho$ evolves according to the Liouville equation; $S_B(M)$, on the other hand, certainly does change. Thus, if we consider the evolution of the microcanonical ensemble corresponding to the macrostate $M_a$ in Fig.\ 1a after removal of the constraint, $S_G$ would equal $S_B$ initially but subsequently $S_B$ would increase while $S_G$ would remain constant. $S_G$ therefore does not give any indication that the system is evolving towards equilibrium. This reflects the fact, discussed earlier, that the microstate $X(t)$ does not remain typical of the local equilibrium state $M(t)$ for $t > 0$. As long as the system remains truly isolated the state $T_tX$ will contain subtle correlations, which are reflected in the complicated shape which an initial region $\Gamma_M$ takes on in time but which do not affect the future time evolution of $M$ (see the discussion at end of section on Initial Conditions). {\it Thus the relevant entropy for understanding the time evolution of macroscopic systems is $S_B$ and not $S_G$}. (Of course if we are willing to do a ``course graining'' of $\rho$ over cells $\Gamma_M$ then we are essentially back to dealing with $\rho_M$, or superpositions of such $\rho_M$'s and we are just defining $S_B$ in a backhanded way.) \bigskip \noindent {\bf Remarks} a) The characterization of a macrostate $M$ usually done via density fields in three dimensional space as in Fig.\ 1 can be extended to mesoscopic descriptions. This is particularly convenient for a {\it dilute gas} where $M$ can be usefully characterized by the density in the six dimensional position and velocity space of a single molecule. The deterministic macroscopic (or mesoscopic) evolution of this $M$ is {\it then} given by the Boltzmann equation and $S_B(M)$ coincides with the negative of Boltzmann's famous $H$-function. It is important to note however that for systems in which the potential energy is relevant, e.g.\ non-dilute gases, the $H$-function does not agree with $S_B$ and $-H$ (but not $S_B$) will decrease for suitable macroscopic initial conditions. As pointed out by Jaynes [24] this will happen whenever one starts with an initial total energy $E$ and kinetic energy $K=K_0$ such that $K_0 > K_{eq} (E)$, the value that $K$ takes when the system is in equilibrium with energy $E$. This can be readily seen if the initial macrostate is one in which the spatial density is uniform and the velocity distribution is Maxwellian with the appropriate temperature $T_0 = {2 \over 3} K_0/kN$. The temperature will then decrease as the system goes to equilibrium and $-H$ which, for a Maxwellian distribution, is proportional to $\log T$ will therefore be smaller in the equilibrium state when $T = T_{eq} (E) < T_0$. \noindent b) Einstein's formula for the probability of fluctuations in an equilibrium system, $${\rm Probability ~~ of ~~} M ~~~ \sim ~~~ \exp\{[S(M) - S_{eq}]/k\}$$ is essentially an inversion of formulas (4) and (5). When combined with the observation that the entropy $S_B(M)$ of a macroscopic system, prepared in a specified nonuniform state $M$, can be computed from macroscopic thermodynamic considerations it yields useful results. In particular when $S_B(M)$ in the exponent is expanded around $M_{eq}$, and only quadratic terms are kept, we obtain a Gaussian distribution for normal (small) fluctuations from equilibrium. This is one of the main ingredients of Onsager's reciprocity relations [25]. \bigskip \noindent {\bf Typical vs. Averaged Behavior} I conclude by emphasizing again that having results for typical microstates rather than averages is not just a mathematical nicety but goes to the heart of the problem of understanding the microscopic origin of observed macroscopic behavior --- {\it we neither have nor do we need ensembles when we carry out observations like those illustrated in Fig.\ 1.} What we do need and can expect to have is typical behavior. Ensembles are merely mathematical tools, useful as long as the dispersion, in the quantities we are interested in, is sufficiently small. This is always the case for properly defined macroscopic variables in equilibrium Gibbs ensembles. The use of such an ensemble as the initial ``statistical state" immediately following the lifting of a constraint from a macroscopic system in equilibrium at some time $t_0$ is also sensible, as long as the evolution of $M(t)$ is, with probability close to one, the same for all systems in the ensemble. There is no such typicality with respect to ensembles describing the time evolution of a system with only a few degrees of freedom. This is an essential difference (unfortunately frequently overlooked or misunderstood) between the irreversible and the chaotic behavior of Hamiltonian systems. The latter, which can be observed already in systems consisting of only a few particles, will not have a uni-directional time behavior in any particular realization. Thus if we had only a few hard spheres in the box of Fig.\ 1, we would get plenty of chaotic dynamics and very good ergodic behavior (mixing, K-system, Bernoulli) but we could not tell the time order of any sequence of snapshots. Finally I note that my discussion has focused exclusively on what is usually referred to as the thermodynamic arrow of time and on its connection with the cosmological initial state. I did not discuss other arrows of time such as the asymmetry between advanced and retarded electromagnetic potentials or ``causality''. It is my general feeling that these are all manifestations of the low entropy initial state of the universe. I also believe that the violation of time reversal invariance in the weak interactions is not relevant for macroscopic irreversibility. \bigskip \noindent {\bf Acknowledgments:} I want to thank Y. ~Aharonov, G.~Eyink, O. ~Penrose and especially S.~Goldstein, H.~Spohn and E.~Speer for many very useful discussions. I also thank the organizers of this conference for their kind hospitality. Various aspects of this research have been supported in part by the AFOSR and NSF. \bigskip \noindent {\bf References} \bigskip \openup-2\jot \parskip=6pt \item {[1]} \ W. Thomson, Proc. of the Royal Soc. of Edinburgh, {\bf 8} 325 (1874), reprinted in 1a). \item {[2]} \ a) For a collection of original articles of Boltzmann and others from the second half of the nineteenth century on this subject (all in English) see S.G. Brush, {\it Kinetic Theory}, Pergamon, Oxford, (1966). \item\item {b)} \ For an interesting biography of Boltzmann, which also contains many references, see E. Broda {\it Ludwig Boltzmann, Man---Physicist---Philosopher}, Ox Bow Press, Woodbridge, Conn (1983); translated from the German. \item \item {c)} For a historical discussion of Boltzmann and his ideas see also articles by M. Klein, E. Broda, L. Flamn in {\it The Boltzmann Equation, Theory and Application}, E.G.D. Cohen and W. Thirring, eds., Springer-Verlag, 1973. \item \item {d)} \ For a general history of the subject see S.G. Brush, {\it The Kind of Motion We Call Heat}, Studies in Statistical Mechanics, vol. VI, E.W. Montroll and J.L. Lebowitz, eds. North-Holland, Amsterdam, (1976). \item \item {e)} \ G. Gallavotti, Ergodicity, Ensembles, Irreversibility in Boltzmann and Beyond, J. Stat. Phys., to appear, (1995). \item {[3]} \ R. Feynman, {\it The Character of Physical Law}, MIT P., Cambridge, Mass. (1967), ch.5. R.P Feynman, R. B Leighton, M. Sands, {\it The Feynman Lectures on Physics}, Addison-Wesley, Reading, Mass. (1963), sections 46--3, 4, 5. \item {[4]} \ O. Penrose, {\it Foundations of Statistical Mechanics}, Pergamon, Elmsford, N.Y. (1970), ch. 5. \item {[5]} \ R. Penrose, {\it The Emperor's New Mind}, Oxford U. P., New York (1990), ch. 7. \item {[6]} \ D. Ruelle, {\it Chance and Chaos}, Princeton U. P., Princeton, N.J. (1991), ch. 17, 18. \item {[7]} \ O. Lanford, Physica A {\bf 106}, 70 (1981). \item {[8]} \ J.L. Lebowitz, Physica A {\bf 194}, 1 (1993). \item {[9]} \ J.C. Maxwell, {\it Theory of Heat}, p. 308: ``Tait's Thermodynamics'', Nature {\bf 17}, 257 (1878). Quoted in M. Klein, ref. 1c). \item {[10]} \ H. Spohn, {\it Large Scale Dynamics of Interacting Particles}, Springer-Verlag, New York (1991). A. De Masi, E. Presutti, {\it Mathematical Methods for Hydrodynamic Limits}, Lecture Notes in Math 1501, Springer-Verlag, New York (1991). J.L. Lebowitz, E. Presutti, H. Spohn, J. Stat. Phys. {\bf 51}, 841 (1988). \item {[11]} \ Y. Aharonov, P.G. Bergmann, J.L. Lebowitz, Phys. Rev. B {\bf 134}, 1410 (1964). D.N. Page, Phys. Rev. Lett. {\bf 70}, 4034 (1993). \item {[12]} \ J.S. Bell, {\it Speakable and Unspeakable in Quantum Mechanics}, Cambridge U. P., New York (1987). \item {[13]} \ D. D{\"u}rr, S. Goldstein, N. Zanghi, J. Stat. Phys. {\bf 67}, 843 (1992). \item {[14]} \ See articles by M. Gell-Mann, J. Hartle, R. Griffiths, D. Page and others in {\it Physical Origin of Time Asymmetry}, J.J. Halliwell, J. Perez-Mercader and W.H. Zurek, eds., Cambridge University Press, 1994. \item {[15]} \ {\it Vorlesungen {\"u}ber Gastheorie}. 2 vols. Leipzig: Barth, 1896, 1898. This book has been translated into English by S.G. Brush, {\it Lectures on Gas Theory}, (London: Cambridge University Press, 1964) . \item {[16]} \ J.W. Gibbs, Connecticut Academy Transactions {\bf 3}, 229 (1875), reprinted in {\it The Scientific Papers}, {\bf 1}, 167 (New York, 1961). \item {[17]} \ L. Boltzmann, Ann. der Physik {\bf 57}, 773 (1896). Reprinted in 1a). \item {[18]} \ E. Schr{\"o}dinger, {\it What is Life? And Other Scientific Essays}, Doubleday Anchor Books, New York (1965), section 6. \item {[19]} \ J.L. Lebowitz and H. Spohn, {\it Communications on Pure and Applied Mathematics}, XXXVI,595, (1983); see in particular section 6(i). \item {[20]} \ L. Boltzmann (1886) quoted in E. Broda, 1b), p. 79. \item {[21]} \ E.L. Hahn, Phys. Rev. {\bf 80}, 580 (1950). See also S. Zhang, B.H. Meier, R.R. Ernst, Phys. Rev. Let.. {\bf 69} 2149 (1992). \item {[22]} \ D. Levesque and L. Verlet, J. Stat. Phys. {\bf 72}, 519 (1993). \item {[23]} \ B.T. Nadiga, J.E. Broadwell and B. Sturtevant, {\it Rarefield Gas Dynamics: Theoretical and Computational Techniques}, edited by E.P. Muntz, D.P. Weaver and D.H. Campbell, Vol 118 of {\it Progress in Astronautics and Aeronautics}, AIAA, Washington, DC, ISBN 0--930403--55--X, 1989. \item {[24]} \ E.T. Jaynes, Phys. Rev. A{\bf 4}, 747 (1971). \item {[25]} \ A. Einstein, Am. Phys. (Leipzig) {\bf 22}, 180 (1907); {\bf 33}, 1275 (1910). L. Onsager, Phys. Rev. {\bf 37}, 405 (1931); {\bf 38}, 2265 (1931). \bigskip \noindent {\bf Figure Captions} \bigskip \noindent {\bf Fig.\ 1} How would you order this sequence of ``snapshots'' in time? Each represents a macroscopic state of a system containing, for example, two differently colored fluids. \bigskip \noindent {\bf Fig.\ 2} With a gas in a box, the maximum entropy state (thermal equilibrium) has the gas distributed uniformly; however, with a system of gravitating bodies, entropy can be increased from the uniform state by gravitational clumping leading eventually to a black hole. From Ref.\ [5]. \bigskip \noindent {\bf Fig.\ 3} Time evolution of a system of 900 particles all interacting via the same cutoff Lennard-Jones pair potential using integer arithmetic. Half of the particles are colored white, the other half black. All velocities are reversed at $t=20,000$. The system then retraces its path and the initial state is fully recovered. From Ref.\ [22]. \bigskip \noindent {\bf Fig.\ 4} Time evolution of a reversible cellular automaton lattice gas using integer arithmetic. Figures a) and c) show the mean velocity, figures b) and d) the entropy. The mean velocity decays with time and the entropy increases up to $t=600$ when there is a reversal of all velocities. The system then retraces its path and the initial state is fully recovered in figures a) and b). In the bottom figures there is a small error in the reversal at $t=600$. While such an error has no appreciable effect on the initial evaluation it effectively prevents any recovery of the macroscopic velocity. The entropy, on the scale of the figure, just remains at its maximum value. This shows the instability of the reversed path. From Ref.\ [23]. \end