In 1963 an MIT meteorologist revealed deterministic predictability to be an illusion and gave birth to a field that still thrives.


In classical physics, one is taught that given the initial state of a system, all of its future states can be calculated. In the celebrated words of Pierre Simon Laplace, “An intelligence which could comprehend all the forces by which nature is animated and the respective situation of the beings who compose it—an intelligence sufficiently vast to submit these data to analysis . . . for it, nothing would be uncertain and the future, as the past, would be present to its eyes.”1 Or, put another way, the clockwork universe holds true.

Herein lies the rub: Exact knowledge of a real-world initial state is never possible—the adviser can always demand a few more digits of experimental precision from the student, but the result will never be exact. Still, until the 19th century, the tacit assumption had always been that approximate knowledge of the initial state implies approximate knowledge of the final state. Given their success describing the motion of the planets, comets, and stars and the dynamics of countless other systems, physicists had little reason to assume otherwise.

Starting in the 19th century, however, and culminating with a 1963 paper by MIT meteorologist Edward Lorenz, pictured in figure 1a, a series of developments revealed that the notion of deterministic predictability, although appealingly intuitive, is in practice false for most systems. Small uncertainties in an initial state can indeed become large errors in a final one. Even simple systems for which all forces are known can behave unpredictably. Determinism, surprisingly enough, does not preclude chaos.





Figure 1. Edward Lorenz and the butterfly effect. (a) Lorenz, studying a computer-generated time series. (Photo courtesy of the Inamori Foundation.) (b) A close-up of Lorenz’s original printout from his discovery of the butterfly effect shows two time series generated with the same equations but with slightly different initial conditions. The series diverge exponentially with time due to sensitive dependence on initial conditions. (Adapted from ref. 9.)



A gallery of monsters

Chaos theory, as we know it today,2 took shape mostly during the last quarter of the 20th century. But researchers had experienced close encounters with the phenomenon as early as the late 1880s, beginning with Henri Poincaré’s studies of the three-body problem in celestial mechanics. Poincaré observed that in such systems “it may happen that small differences in the initial conditions produce very great ones in the final phenomena. . . . Prediction becomes impossible.”3

Dynamical systems like the three-body system studied by Poincaré are best described in phase space, in which dimensions correspond to the dynamical variables, such as position and momentum, that allow the system to be described by a set of first-order ordinary differential equations. The prevailing view had long been that, left alone, a conventional classical system will eventually settle toward either a steady state, described by a point in phase space; a periodic state, described by a closed loop; or a quasi-periodic state, which exhibits n > 1 incommensurable periodic modes and is described by an n-dimensional torus in phase space.

The three-body trajectories calculated by Poincaré fit into none of those categories. Rather, he observed that “each curve never intersects itself, but must fold upon itself in very complex fashion so as to intersect infinitely often each apex of the grid. One must be struck by the complexity of this shape, which I do not even attempt to illustrate,” as paraphrased in English in reference 4, page 414.

What Poincaré refused to draw is now widely known as a homoclinic tangle, a canonical manifestation of chaos having fractal geometry. (An image of the tangle can be seen in figure 4 of the article by David Nolte, Physics Today, April 2010, page 33.)

Poincaré’s results, independent findings by Jacques Hadamard, and experimental hints of chaos seen by their contemporaries were dismissed by many as pathologies or artifacts of noise or methodological shortcomings and were referred to as a “gallery of monsters.”4 It would take nearly another century for chaos theory to gain a lasting foothold.

A serendipitous discovery

In all likelihood, Lorenz was unfamiliar with Poincaré’s work when he began his foray into meteorology in the mid 1900s (reference 5, page 133). With undergraduate and master’s degrees in mathematics, Lorenz had served as a meteorologist in World War II before completing his doctoral studies in meteorology at MIT and joining the MIT faculty in 1955.

At the time, most meteorologists predicted weather using linear procedures, which were based on the premise that tomorrow’s weather is a well-defined linear combination of features of today’s weather. By contrast, an emerging school of dynamic meteorologists believed that weather could be more accurately predicted by simulating the fluid dynamical equations underlying atmospheric flows. Lorenz, who had just purchased his first computer, a Royal McBee LGP-30 with an internal memory of 4096 32-bit words, decided to compare the two approaches by pitting the linear procedures against a simplified 12-variable dynamical model. (Lorenz’s computer, though a thousand times faster than his desk calculator, was still a million times slower than a current laptop.)

Lorenz searched for nonperiodic solutions, which he figured would pose the biggest challenge for the linear procedures, and eventually found them by imposing an external heating that varied with latitude and longitude—as does solar heating of the real atmosphere. Sure enough, the linear procedures yielded a far-from-perfect replication of the result.

Having found the nonperiodic solutions of his model interesting in their own right, Lorenz decided to examine them in more detail. He reproduced the data, this time printing the output variables after each day of simulated weather. To save space, he rounded them off to the third decimal place, even though the computer calculations were performed with higher precision. What followed is best appreciated in Lorenz’s own words:

At one point I decided to repeat some of the computations in order to examine what was happening in greater detail. I stopped the computer, typed in a line of numbers that it had printed out a while earlier, and set it running again. I went down the hall for a cup of coffee and returned after about an hour, during which time the computer had simulated about two months of weather. The numbers being printed were nothing like the old ones. I immediately suspected a weak vacuum tube or some other computer trouble, which was not uncommon, but before calling for service I decided to see just where the mistake had occurred, knowing that this could speed up the servicing process. Instead of a sudden break, I found that the new values at first repeated the old ones, but soon afterward differed by one and then several units in the last decimal place, and then began to differ in the next to the last place and then in the place before that. In fact, the differences more or less steadily doubled in size every four days or so, until all resemblance with the original output disappeared somewhere in the second month. This was enough to tell me what had happened: the numbers that I had typed in were not the exact original numbers, but were the rounded-off values that had appeared in the original printout. The initial round-off errors were the culprits; they were steadily amplifying until they dominated the solution. (reference 5, page 134)


The butterfly effect

What Lorenz had observed with his model came to be known as sensitive dependence on initial conditions—a defining property of chaos. In phase space, the phenomenon has a distinct quantitative signature: The distance between any two nearby trajectories grows exponentially with time. Sensitive dependence is illustrated in figure 1b, one of Lorenz’s own plots, which shows the gradual divergence of two time series calculated using identical equations but slightly different initial conditions. That trademark behavior gives chaotic systems the appearance of randomness. But as Lorenz himself noted, the appearances are deceiving: At any given time in a random system, one of two or more things can happen next, as one usually assumes for the throw of a die; in chaotic systems such as Lorenz’s, outcomes are fully deterministic. (And strictly speaking, so are those of die throws.)

Lorenz realized that if the atmosphere were to behave like his model, forecasting the weather far in the future would be impossible. At a 1972 meeting of the American Association for the Advancement of Science, in a talk titled “Predictability: Does the flap of a butterfly’s wings in Brazil set off a tornado in Texas?” Lorenz used a butterfly as a metaphor for a tiny, seemingly inconsequential perturbation that could change the course of weather. The metaphor caught on, and sensitive dependence famously came to be dubbed the butterfly effect.

Given that computer simulations generally introduce round-off error at each time step—error that is amplified by chaos—one must ask whether Lorenz’s solutions can possibly provide reliable information about real chaotic trajectories. As it happens, they can, because of a property now known as shadowing: Although for any given initial condition the numerical trajectory diverges from the exact one, there always exists a nearby initial condition whose exact trajectory is approximated by the numerical one for a prespecified stretch of time. In the end, it is as if one had started from a different initial condition and calculated the trajectory exactly—a crucial result, given that numerical calculations came to be widely used in the study of chaotic systems. For example, the trajectories Lorenz calculated using the truncated variables were, in fact, just as representative of his model’s behavior as the original (as well as the exact) trajectories.

Lorenz first presented the results from his 12-variable model at a 1960 symposium held in Tokyo. At that meeting, he only briefly mentioned the unexpected effect of round-off errors; he believed those results belonged in a different paper. In retrospect, he was in little danger of being scooped—apparently, most of his contemporaries failed to recognize the broad significance of his findings. (Meanwhile, the work of other pioneers of chaos often went unappreciated; see reference 6 for Yoshisuke Ueda’s description of his frustration at the lack of appreciation of his 1961 analog computer observations of the “randomly transitional phenomenon,” later recognized as chaos.)

The Lorenz attractor
Go to section...


Lorenz published his serendipitous discovery in a March 1963 paper titled “Deterministic nonperiodic flow.”7 He had spent a significant part of his time since the Tokyo meeting looking for the simplest possible model exhibiting sensitive dependence on initial conditions, and he eventually arrived at a three-variable system of nonlinear ordinary differential equations now known as the Lorenz equations (see the box on page 31).

Like Poincaré’s three-body system, the Lorenz equations yield phase-space trajectories that never retrace themselves and that don’t trace out surfaces of integer dimension. Rather, typical trajectories tend to converge to, and then orbit along, a bounded structure of noninteger, fractal dimension known as a chaotic attractor. (See figure 2.)





Figure 2. The Lorenz attractor, as revealed by the never-repeating trajectory of a single chaotic orbit. The spheres shown here represent iterations of the so-called Lorenz equations, calculated using the original parameters in Edward Lorenz’s seminal work. (Spheres are colored according to the iteration count.) From certain angles, the two lobes of the attractor resemble a butterfly, a coincidence that helped earn sensitive dependence on initial conditions its nickname—the butterfly effect. An animated visualization of the attractor is availablehere.(Image courtesy of Stefan Ganev.)

more here

excellent copy and past job, reminds me of the film good will hunting!

is there an "unexcellent" way to copy and paste?

Well,
I apriciate Moe's posts. Very informative and thought provoking.

thnx, just an article about the 50th bday of the chaos theory, and how it came about...


but i would like to know what made it excellent, maybe because i included a bunch of hyperlinks? the pics? maybe the combination of the right and left clicks in the right spots?


i'm just glad i impressed him with my copy and paste skillz...

Studied it in school. The digital logic circuits gave the analog computer the ability to round up properly the left over integers. Making the study of string and kaos theory, true.

Well, maybe he's somewhat incoherently objecting to your forgetting to credit the original author properly. Or perhaps that you didn't add any comment as to why you personally thought this was a good thing to post.

I'm not complaining mind you, but I know plenty of people can get concerned about those two issues, especially if they suspect that the person posting has an unspecified ulterior motive in doing so.


I myself, while having tremendous respect for the brilliance of the mathematical modeling work involved with all this, take the greatest overall lesson to be, that most human beings don't recognize how complicated our world and our challenges really are. They tend to come up with overly simple "solutions" based on a short-sighted and ignorance-based set of incomplete assumptions, and then get carried away with how much fun it is to make self-righteous or snobbish declarations about whatever they are concerned with.

Because another way to look at thing like chaos theory, is to recognize that what it's really about, is the recognition that most of the time, we don't know near as much NOW, about what factors influence the future as we want to think we do.

maybe if they're doing the right math, some future events can be predicted with some accuracy... but that seems like trying to find a needle in a haystack, with all the variables that are present...

It reminds me of being in school and recognizing the beauty of Laplace Transforms. It has been recognized for decades that no initial state is absolute... at least not in the real world. Schrodinger's cat keeps jumping out of the box and biting you!

Can't it.......only if there are no more perks mountains spiecies plans gravity........and the universe........

no. it's when the cigar smokers leave the nest with newspapers and keys ...

Everything can be the source of interruption. Then how could make such kind of perfect environment for the the theory establishing. also if the theory needs kind of environment then where can be perfect on earth? Is it redigulars? in Budaha theory they call such kind of effects names 万 卍 then started know about why hitler was mad now? never try to moving the whole hill by your fingers.......XD but if some one try to do that like hitler should call extreme sport athlete.......

no such thing as a perfect environment, it's always about survival of the fittest...something better always comes along, and the environment isn't so perfect then...

like the tonado? kind of environmental scence should call the mass effects it's totally different from the butterfly's effect. the way of butterfly's effect is from the narrowest to the mass but the mass effects is on the opposite theory. So more clearly now? Like the building demolition even one slitly mistaken of gunpower point setting the process will be fail. although some time there are only few of column bracings within.....

A few years ago i first read about this theory & saw how it worked in practical life & at many times thought of about how amazing & accurate it was

perfect and right are different... perfect means no variables, the outcome will always be the same, where the variables of right can be high, and the outcome can be predicted within a tolerance... close, but not the same meanings...

whatever.........