Complexity | The Emerging Science At The Edge Of Order And Chaos by M. Mitchell Waldrop

I. Brief Summary

A compelling book about the scientific revolution of a complex adaptive system. The book takes the readers into the hearts and minds of scientific pioneers and their involvement with Santa Fe Institute. The Santa Fe Institute is an influential interdisciplinary research institute. Topics in this book include complexity, self-organization, emergence, order and chaos from the diverse fields such as economics, physics, computer science and biology. Waldrop does an excellent job of explaining non-linear systems. Both linear and non-linear systems co-exist, but non-linearity is fairly an emerging topic. Pick this book up and give it a go. I highly recommend it if you are interested in better models of reality, so that you can make better decisions.

II. Big Ideas

• Complexity is the science of emergence where many individual agents interact and the outcomes are difficult to predict.
• Complex adaptive system originated at Santa Fe Institute (SFI). Santa Fe Institute was formed in 1984. The SFI is an independent, nonprofit theoretical research institute dedicated to the study of complex adaptive systems.
• The theory of complexity manifests itself in “complex adaptive systems,” which are made up of many independent agents who interact and adapt to each other and to their environment, producing the phenomenon of emergence—a system behaving as more than the sum of its parts.
• A complex adaptive system is a system in which a large network of components with no central control exhibit complex behavior, sophisticated information processing, and adaptive learning.
• One of the important insights emerging from complexity theory is the concept of complex systems poised at the “edge of chaos” in a zone between rigid stability and chaotic turbulence. Computer modeling has become prominent in the search for patterns of complexity, all leading to the same observation of a balance point between order and chaos where complex systems come alive and produce emergent order.
• Existence of all this order at the edge of chaos would seem to fly in the face of the second law of thermodynamics, which declares process of decay from order into chaos. Waldrop argues that one of the important directions of complexity theory is toward a reformulation of the second law, to take into account the structure and trend toward greater, rather than less, complexity in the universe.
• Human agents are not all-knowing and do not always make profit maximizing decisions, nor is the economy always in a frozen state of equilibrium. Complexity theory offers a window into the phenomenon of adaptive behavior of agents and emergent properties, same as the apparent personality of financial markets.
• Spontaneous self-organization can be found all over nature. It is the most powerful force in biology and living systems operate at the edge of chaos. The characteristics of self-organization are positive feedback, increasing returns, lock-in (harder to change technology until something vastly better comes along), unpredictability and tiny events that have immense consequences all seem to be a re-requisite for life itself.
• Increasing returns are prominent when marginal costs are minimal.
• Innovation happens at the intersection of multiple fields. It does not happen in a vacuum.
• Domino effect occurs once tipping point hits, leads to cascades, and often winner-take-all systems.
• Reality is superior to elegant theory.
• Essence of science lies in explanation more than prediction. If the future is unpredictable, what do you do? You must learn to act without prediction.
• Everything is caught up in non-linear web of incentives, constraints and connections. The crucial skill is insight. The ability to see connections. Power lies in connections between exploitation (improving current conditions) vs. exploration (taking chances for greater reward).
• Complex adaptive systems can never reach equilibrium as new opportunities are constantly rising.
• Emergence is hierarchical—building blocks at one level combining into new blocks at a higher level. Hierarchies are one of the fundamental organizing principles of the world and is good for execution.
• Adaptive agents play game with its environment for fitness. It is feedback driven.
• Competition is much more essential than consistency. Competition and cooperation may seem antithetical but at some very deep level, they are two sides of the same coin.
• Following are the paradigms of spectrum:
  • Dynamic systems: order, complexity, chaos.
  • Physical systems (matter): solid, phase transition, liquid.
  • Computation: halting, undecidable, non-halting.
  • Life: static, life/intelligence, noisy.
• Life is based on a great degree on its ability to process and store information and then mapping it out to determine proper action.
• Life is not a property of matter per se, but the organization of that matter.
• “Aliveness” lies in the organization of the molecules and not the molecules themselves.
• There is no duality between man and nature, we are all part of this interlocking network.
• Increasing returns (on technology and molecules being a network of system as opposed to being a commodity):
  • When you look at economic history, as opposed to economic theory, he told Kauffman, technology isn’t really like a commodity at all. It is much more like an evolving ecosystem. “In particular, innovations rarely happen in a vacuum. They are usually made possible by other innovations being already in place. For example, a laser printer is basically a Xerox machine with a laser and a little computer circuitry to tell the laser where to etch on the Xerox drum for printing. So a laser printer is possible when you have computer technology, laser technology, and a Xerox reproducing technology. But it is also only possible because people need fancy, high-speed printing.”
  • Moreover, these technological webs can undergo bursts of evolutionary creativity and massive extinction events, just like biological ecosystems. Say a new technology like the automobile comes in and replaces an older technology, the horse. Along with the horse go the smithy, the pony express, the watering troughs, the stables, the people who curried horses, and so on. The whole subnetwork of technologies that depended upon the horse suddenly collapses in what the economist Joseph Schumpeter once called “a gale of destruction.” But along with the car come paved roads, gas stations, fast-food restaurants, motels, traffic courts and traffic cops, and traffic lights. A whole new network of goods and services begins to grow, each one filling a niche opened up by the goods and services that came before it.
  • But to Kauffman, this autocatalytic set story was far and away the most plausible explanation for the origin of life that he had ever heard. If it were true, it meant the origin of life didn’t have to wait for some ridiculously improbable event to produce a set of enormously complicated molecules; it meant that life could indeed have bootstrapped its way into existence from very simple molecules. And it meant that life had not been just a random accident, but was part of nature’s incessant compulsion for self-organization.

III. Quotes

• The royal road to a Nobel Prize has generally been through the reductionist approach.
• Why is it that simple particles obeying simple rules will sometimes engage in the most astonishing, unpredictable behavior?
• Like it or not, the marketplace isn’t stable. The world isn’t stable. It’s full of evolution, upheaval, and surprise.
• Predictions are nice, if you can make them. But the essence of science lies in explanation, laying bare the fundamental mechanisms of nature.
• Theoretical economists use their mathematical prowess the way the great stags of the forest use their antlers: to do battle with one another and to establish dominance.
• Everything affects everything else, and you have to understand that whole web of connections.
• If you have a truly complex system," he says, "then the exact patterns are not repeatable. And yet there are themes that are recognizable. In history, for example, you can talk about 'revolutions,' even though one revolution might be quite different from another. So we assign metaphors. It turns out that an awful lot of policy-making has to do with finding the appropriate metaphor. Conversely, bad policy-making almost always involves finding inappropriate metaphors. For example, it may not be appropriate to think about a drug 'war,' with guns and assaults.
• An adaptive agent is constantly playing a game with its environment. What exactly does that mean? Distilled to the essence, what actually has to happen for game-playing agents to survive and prosper? Two things, Holland decided: prediction and feedback.
• All these complex systems have somehow acquired the ability to bring order and chaos into a special kind of balance. This balance point—often called the edge of chaos—is were the components of a system never quite lock into place, and yet never quite dissolve into turbulence, either.
• In contrast to mainstream artificial intelligence, I see competition as much more essential than consistency. Consistency is a chimera, because in a complicated world there is no guarantee that experience will be consistent. But for agents playing a game against their environment, competition is forever. Despite all the work in economics and biology, we still haven't extracted what's central in competition. There's a richness there that we've only just begun to fathom. Consider the magical fact that competition can produce a very strong incentive for cooperation, as certain players spontaneously forge alliances and symbiotic relationships with each other for mutual support. It happens at every level and in every kind of complex, adaptive system, from biology to economics to politics. Competition and cooperation may seem antithetical but at some very deep level, they are two sides of the same coin.
• Look at meteorology, he told them. The weather never settles down. It never repeats itself exactly. It's essentially unpredictable more than a week or so in advance. And yet we can comprehend and explain almost everything that we see up there. We can identify important features such as weather fronts, jet streams, and high-pressure systems. We can understand their dynamics. We can understand how they interact to produce weather on a local and regional scale. In short, we have a real science of weather-without full prediction. And we can do it because prediction isn't the essence of science. The essence is comprehension and explanation. And that's precisely what Santa Fe could hope to do with economics and other social sciences, he said: they could look for the analog of weather fronts-dynamical social phenomena they could understand and explain.
• It will require the renunciation or sublimation or transformation of our traditional appetites: to outbreed, outconsume, and conquer our rivals, especially our rivals in other tribes. These impulses may once have been adaptive. Indeed, they may even be hard-wired into our brains. But we no longer have the luxury of tolerating them.
• Humanity is gravely threatened by superstition and myth, the stubborn refusal to recognize the urgent planetary problems, and generalized tribalism in all its forms.
• "Santa Fe approach": Instead of emphasizing decreasing returns, static equilibrium, and perfect rationality, as in the neoclassical view, the Santa Fe team would emphasize increasing returns, bounded rationality, and the dynamics of evolution and learning. Instead of basing their theory on assumptions that were mathematically convenient, they would try to make models that were psychologically realistic. Instead of viewing the economy as some kind of Newtonian machine, they would see it as something organic, adaptive, surprising, and alive. Instead of talking about the world as if it were a static thing buried deep in the frozen regime, as Chris Langton might have put it, they would learn how to think about the world as a dynamic, ever-changing system poised at the edge of chaos.
• an autocatalytic set was a web of transformations among molecules in precisely the same way that an economy is a web of transformations among goods and services. In a very real sense, in fact, an autocatalytic set was an economy—a submicroscopic economy that extracted raw materials (the primordial “food” molecules) and converted them into useful products (more molecules in the set). Moreover, an autocatalytic set can bootstrap its own evolution in precisely the same way that an economy can, by growing more and more complex over time. This was a point that fascinated Kauffman. If innovations result from new combinations of old technologies, then the number of possible innovations would go up very rapidly as more and more technologies became available.
• At the same time, Kaufmann discovered that in developing his genetic networks, he had reinvented some of the most avant-garde work in physics and applied mathematics-albeit in a totally new context. The dynamics of his genetic regulatory networks turned out to be a special case of what the physicists were calling "nonlinear dynamics." From the nonlinear point of view, in fact, it was easy to see why his sparsely connected networks could organize themselves into stable cycles so easily: mathematically, their behavior was equivalent to the way all the rain falling on the hillsides around a valley will flow into a lake at the bottom of the valley. In the space of all possible network behaviors, the stable cycles were like basins-or as the physicists put it, "attractors.”
• In non-linear systems-and the economy is most certainly nonlinear-chaos theory tells you that the slightest uncertainty in your knowledge of the initial conditions will often grow inexorably. After a while, your predictions are nonsense.
• This game analogy seemed to be true of any adaptive system. In economics the payoff is in money, in politics the payoff is in votes, and on and on. At some level, all these adaptive systems are fundamentally the same.
• It's people who like process and pattern, as opposed to people who are comfortable with stasis and order.
• To me, coming from applied mathematics, a theorem was a statement about an everlasting mathematical truth—not the dressing up of a trivial observation in a lot of formalism.
• It didn’t take very long for Arthur to realize that, when it came to real-world complexities, the elegant equations and the fancy mathematics he’d spent so much time on in school were no more than tools—and limited tools at that. The crucial skill was insight, the ability to see connections.
• Living systems are actually very close to this edge-of-chaos phase transition, where things are much looser and more fluid. And natural selection is not the antagonist of self-organization.
• If the origin of life had really been a random event, then it had really been a miracle.
• Generally treating free-market capitalism as a kind of state religion.
• You observe, and observe, and observe, and occasionally stick your oar in and try to improve something for the better. It means that you try to see reality for what it is, and realize that the game you are in keeps changing, so that it’s up to you to figure out the current rules of the game as it’s being played… you stop adhering to standard theories that are built on outmoded assumptions about the rules of play, you stop saying, ‘Well, if only we could reach this equilibrium we’d be in fat city.’ You just observe. And where you can make an effective move, you make a move...The idea is to observe, to act courageously, and to pick your timing extremely well. —Brian Arthur
• “The people doing math in biology were the lowest of the low,” he says. It was exactly the opposite of the situation in physics or economics, where the theorists are kings.
• We call our particles ‘agents’—banks, firms, consumers, governments. And those agents react to other agents, just as particles react to other particles. However, he added, there is one big difference: “Our particles in economics are smart, whereas yours in physics are dumb.” In physics, an elementary particle has no past, no experience, no goals, no hopes or fears about the future. It just is. That’s why physicists can talk so freely about “universal laws”: their particles respond to forces blindly, with absolute obedience. But in economics, said Arthur, “Our particles have to think ahead, and try to figure out how other particles might react if they were to undertake certain actions. Our particles have to act on the basis of expectations and strategies. And regardless of how you model that, that’s what makes economics truly difficult.”
• Unfortunately, the economists’ standard solution to the problem of expectations—perfect rationality—drove the physicists nuts. Perfectly rational agents do have the virtue of being perfectly predictable. That is, they know everything that can be known about the choices they will face infinitely far into the future, and they use flawless reasoning to foresee all the possible implications of their actions. So you can safely say that they will always take the most advantageous action in any given situation, based on the available information....The only problem, of course, is that real human beings are neither perfectly rational nor perfectly predictable—as the physicists pointed out at great length.
• Indeed, in their own minds, physicists are the aristocracy of science. From the day they sign up for Physics 101, they absorb the culture in a thousand subtle and not-so-subtle ways: they are the heirs of Newton, Maxwell, Einstein, and Bohr. Physics is the hardest, purest, toughest science there is. And physicists have the hardest, purest, toughest minds around.

We call our particles ‘agents’—banks, firms, consumers, governments. And those agents react to other agents, just as particles react to other particles. Only we don’t usually consider the spatial dimension in economics much, so that makes economics a lot simpler.”

• However, he added, there is one big difference: “Our particles in economics are smart, whereas yours in physics are dumb.” In physics, an elementary particle has no past, no experience, no goals, no hopes or fears about the future. It just is. That’s why physicists can talk so freely about “universal laws”: their particles respond to forces blindly, with absolute obedience. But in economics, said Arthur, “Our particles have to think ahead, and try to figure out how other particles might react if they were to undertake certain actions. Our particles have to act on the basis of expectations and strategies. And regardless of how you model that, that’s what makes economics truly difficult.”
• Unfortunately, the economists’ standard solution to the problem of expectations—perfect rationality—drove the physicists nuts. Perfectly rational agents do have the virtue of being perfectly predictable. That is, they know everything that can be known about the choices they will face infinitely far into the future, and they use flawless reasoning to foresee all the possible implications of their actions. So you can safely say that they will always take the most advantageous action in any given situation, based on the available information.
• In the natural world such systems included brains, immune systems, ecologies, cells, developing embryos, and ant colonies. In the human world they included cultural and social systems such as political parties or scientific communities....First, he said, each of these systems is a network of many “agents” acting in parallel. In a brain the agents are nerve cells, in an ecology the agents are species, in a cell the agents are organelles such as the nucleus and the mitochondria, in an embryo the agents are cells, and so on. In an economy, the agents might be individuals or households. Or if you were looking at business cycles, the agents might be firms. And if you were looking at international trade, the agents might even be whole nations....It is constantly acting and reacting to what the other agents are doing. And because of that, essentially nothing in its environment is fixed....If there is to be any coherent behavior in the system, it has to arise from competition and cooperation among the agents themselves.
• Second, said Holland, a complex adaptive system has many levels of organization, with agents at any one level serving as the building blocks for agents at a higher level. A group of proteins, lipids, and nucleic acids will form a cell, a group of cells will form a tissue, a collection of tissues will form an organ, an association of organs will form a whole organism, and a group of organisms will form an ecosystem.
• Complex adaptive systems are constantly revising and rearranging their building blocks as they gain experience. Succeeding generations of organisms will modify and rearrange their tissues through the process of evolution. The brain will continually strengthen or weaken myriad connections between its neurons as an individual learns from his or her encounters with the world. A firm will promote individuals who do well and (more rarely) will reshuffle its organizational chart for greater efficiency. Countries will make new trading agreements or realign themselves into whole new alliances.
• Finally, said Holland, complex adaptive systems typically have many niches, each one of which can be exploited by an agent adapted to fill that niche. Thus, the economic world has a place for computer programmers, plumbers, steel mills, and pet stores, just as the rain forest has a place for tree sloths and butterflies. Moreover, the very act of filling one niche opens up more niches—for new parasites, for new predators and prey, for new symbiotic partners. So new opportunities are always being created by the system. And that, in turn, means that it’s essentially meaningless to talk about a complex adaptive system being in equilibrium: the system can never get there. It is always unfolding, always in transition. In fact, if the system ever does reach equilibrium, it isn’t just stable. It’s dead.
• Trade with Japan is at least as complicated as chess. But economists will start out by saying, ‘Assume rational play.’”