Rules of engagement: The architecture of robust, evolvable networks

John Doyle (Caltech)

Abstract

The nature of complexity, robustness, and evolvability has been a persistent source of confusion both throughout science and between science and society. Recent broad progress could potentially help resolve much of this confusion, particular among those attracted to rigor in experiments, statistics, and mathematics. This talk will take a hopefully light-hearted look at the sources of confusion as well their potential resolution.

The technical basis of this talk is on the architectural and organizational principles of networked systems, building on insights about the fundamental nature of complex biological and technological networks drawn from three converging research themes. 1) With molecular biologyís description of components and growing attention to systems biology, the organizational principles of biological networks are becoming increasingly apparent. Biologists are articulating richly detailed explanations of biological complexity, robustness, and evolvability that point to universal principles. 2) Advanced technologyís complexity is now approaching biologyís. While the components differ, there is striking convergence at the network level of architecture and the role of layering, protocols, and feedback control in structuring complex multiscale modularity. New theories of the Internet and related networking technologies have led to test and deployment of new protocols for high performance networking. 3) A new mathematical framework for the study of complex networks suggests that this apparent network-level evolutionary convergence within/between biology/technology is not accidental, but follows necessarily from the universal system requirements to be efficient, adaptive, evolvable, and robust to perturbations in their environment and component parts.

One of the simplest but most important observations is that biological systems are robust and evolvable in the face of even large changes in environment and system components, yet can simultaneously be extremely fragile to small perturbations. Such universally robust yet fragile (RYF) complexity is found wherever we look. The amazing evolution of microbes into humans (robustness of lineages on long timescales) is punctuated by mass extinctions (extreme fragility). Diabetes, obesity, cancer, and autoimmune diseases are side-effects of biological control and compensatory mechanisms so robust as to normally go unnoticed.

RYF complexity is not confined to biology. The complexity of technology is exploding around us, but in ways that remain largely hidden. Modern institutions and technologies facilitate robustness and accelerate evolution, but enable catastrophes on a scale unimaginable without them (from network and market crashes to war, epidemics, and climate change). Understanding RYF means understanding architecture ó the most universal, high-level, persistent elements of organization ó and protocols. Protocols define how diverse modules interact, and architecture defines how sets of protocols are organized.

The Internet is an obvious and familiar example of how a protocol-based architecture facilitates evolution and robustness. The hourglass protocol ìstackî has a thin, hidden ìwaistî of universally shared feedback control (TCP/IP) between the visible upper (application software) and lower (hardware) layers. This allows ìplug-and-playî between modules that obey shared protocols; any set of applications that ìtalksî TCP can run transparently and robustly on any set of hardware that ìtalksî IP, accelerating the evolution of TCP/IP-based networks. Diverse applications use additional protocols to exchange objects such as web files, images, and email. In the hardware layer, signals are also exchanged via shared protocols and interfaces. Indeed, recent theory proves that with suitable versions of TCP/IP and network provisioning, an otherwise arbitrary (e.g. large and heterogeneous) network has guaranteed global dynamics that robustly optimizes aggregate application utility. (Similar results hold for flocking and oscillator synchronization.) Other theory shows that overall system robustness obeys conservation laws, suggesting RYF behavior can be managed but not completely eliminated.

Within a layer these protocols can be visualized as bowties with large fan-ins and -outs of energy, materials, and information via a thin `knotî of universal protocols specifying carriers, building blocks, or interfaces. Other engineered examples of bowties include networks which connect energy sources to consumers via carriers and standard socket-plug interfaces, sellers to buyers via money, fibers into diverse clothing via sewing, and raw materials to assemblies via standardized building blocks for advanced manufacturing. In these and the biologic examples below, the currencies, carriers, intermediates, precursors, plugs, packets, conserved residues, and interfaces in the bowtie ìknotî are highly constrained by protocols. Yet this shared universality allows robust adaptation and evolution to extreme diversity at the inputs (energy, sellers, fibers, materials, nutrients, receptors) and outputs (consumers, buyers, clothing, assemblies, products, effectors), as long as they have appropriate (and typically hidden) layers of feedback control.

The robustness and evolvability of the cell exemplify reliance on layers and protocols analogous to those in TCP/IP and other engineered architectures, but with wondrously different hardware components and more extensive and sophisticated use of feedback control. Perhaps the ultimate bowtie involves the universal, central protocols that transcribe and translate DNA to proteins via conserved codes, polymerases, and RNA intermediates. As in IP, feedback control of polymerases yields a functional network with regulated levels of macromolecules. A simple cell in a constant environment needs little more, but real cells add robustness to changing demands with a TCP-like feedback layer of allostery and post-translation modifications acting on faster time scales. These TCP/IP-like layers (the waist of a cellís hourglass) of control protocols constitute much of biochemistry texts.

In the microbial biosphere, genes that ìtalkî transcription and translation protocols can move by horizontal gene transfer (HGT), thus accelerating evolution in a kind of a ìbacterial internet.î Furthermore, proteins newly acquired by HGT or through duplication and divergence or domain rearrangements can work effectively because they can exploit additional shared protocols. For example, a bacterial cellís application layer consists of biochemical fluxes with various additional bowtie protocols such as core metabolism (nutrients to biosynthesis and energy supply via conserved precursors and carrier-based group transfers) and signal transduction (receptors to downstream effectors via conserved residue pairs). Thus selection acting at the protocol level could evolve and preserve this variety of shared and ultimately conserved architecture, essentially evolving evolvability.

All life and advanced technologies rely on protocol-based architectures. The evolvability of microbes and IP-based networks illustrate how dramatic, novel, dynamic changes on all scales of time and space can also be coherent, responsive, functional and adaptive, despite implementations that are largely decentralized and asynchronous. New genes and pathways, laptops and applications, even whole networks, can plug-and-play, as long as they obey protocols. Biologists can even swap gene sequences over the Internet in a kind of synthetic HGT. A related aspect of sophisticated architectures is that actuator signals (e.g. from intermediate fluxes and concentrations in core metabolism to cardiopulmonary rates and renal concentrating effects) have extremely high variability in order to keep other critical signals (e.g. from key metabolic products to blood oxygenation and pH, to core body temperature,) tightly regulated despite fluctuating supplies and demands. Such natural physiological variability can be a source of confusion if homeostasis is misinterpreted as implying that all signals, including actuators, are held nearly constant.

Typical behavior is fine-tuned with this elaborate control and thus appears boringly robust despite large internal and external perturbations and fluctuations. As a result, complexity and fragility are largely hidden, often revealed only by rare catastrophic failures. Since components come and go, control systems that reallocate network resources easily confer robustness to outright failures, whereas violations of protocols by small malicious or even random rewiring can be catastrophic. So programmed cell (or component) ìdeathî is a common strategy to prevent local failures from cascading system-wide. The greatest fragility stemming from a reliance on protocols is that standardized interfaces and building blocks can be easily hijacked. So that which enables HGT, the web and email also aids viruses and other parasites. Large structured rearrangements can be tolerated, while small random or targeted changes that subtly violate protocols can be disastrous. Thus another source of high variability is in the external behavior of RYF systems, where typical behavior is boringly robust but rare events can be catastrophically large.

Chaos, fractals, random graphs, criticality, and related ideas from statistical physics inspire a popular and completely different view of complexity, where behaviors that are typically unpredictable and fragile ìemergeî from simple and usually random interconnections among homogeneous components. The attraction is understandable because a variety of features, continually recycled in series of ìnew sciences,î greatly simplify modeling and analysis. For example, random internal rewiring has little effect. Tuning, structure, and component heterogeneity is minimal, as is system functionality, robustness, and environmental uncertainty. Perhaps most appealing is that ìmodularityî reduces to superficial self-similar patterns in ensemble averages, avoiding the complexity of protocols and architecture. Unfortunately, highly evolved and architecture-based RYF complexity is utterly bewildering when viewed from this perspective, which has led to widespread and remarkably persistent errors and confusion about complex systems of all kinds. There are striking parallels with irreducible complexity and intelligent design, which has even more successfully popularized persistent errors and confusion, though more narrowly targeting biology and evolution.

Particularly persistent errors surround the origin and nature of power laws. The statistical invariance properties of Gaussians and exponentials for low variability phenomena are well-known enough that their presence is little cause for surprise. But for high variability phenomena, power laws have even richer statistical invariants, and could rightly be called ìmore normal than Normal.î Thus their ubiquity in the abundant high variability of architecture-based RYF systems should be no surprise, and certainly no evidence on their own for any specific mechanism beyond the presence of such high variability.

One thing all can agree on is that the search for a deep simplicity and unity underlying natural and technological complexity remains a common goal. Fortunately, our growing need for robust, evolvable technological networks means the tools for engineering architectures and protocols are becoming more unified, scalable and accessible, and happen to include the more popular complexity views as extreme special cases within a much richer structure, organized, and dynamical perspective. These developments are bringing much-needed rigor and relevance to the study of complexity generally, including biology, but will not eliminate the need for attention to domain-specific structure and detail. Quite the contrary: both architectures and theories to study them are most successful when they facilitate rather than ignore the inclusion of domain-specific details and expertise.

Selected references

1. Csete M.E. and J.C. Doyle, (2004), Bow ties, metabolism, and disease, Trends in Biotechnology, Vol 22, Issue 9, pg. 446-450
2. H. El-Samad, H. Kurata, J.C. Doyle , C.A. Gross, and M. Khammash, (2005), Surviving Heat Shock: Control Strategies for Robustness and Performance, PNAS 102(8): FEB 22, 2005
3. Jin C, Wei D, Low SH, Bunn J, Choe HD, Doyle JC,et al (2005), †FAST TCP: From theory to experiments IEEE NETWORK 19 (1): 4-11 JAN-FEB 2005
4. J. Doyle and M. Csete (2005). Motifs, stability, and control. PLOS Biology, 3, 2005.
5. Doyle et al, (2005), The ìRobust Yet Fragileî Nature of the Internet, PNAS 102 (41), October 11, 2005
6. MA Moritz, ME Morais, LA Summerell, JM Carlson, J Doyle (2005) Wildfires, complexity, and highly optimized tolerance, PNAS, 102 (50) December 13, 2005;†,
7. L Li, D Alderson, JC Doyle, W Willinger (2006) Towards a Theory of Scale-Free Graphs: Definition, Properties, and Implications, Internet Math, Vol. 2, No. 4, 2006
8. H El-Samad , A Papachristodoulou, S Prajna, J Doyle, and M Khammash (2006), Advanced Methods and Algorithms for Biological Networks Analysis, PROCEEDINGS OF THE IEEE, 94 (4): 832-853 APR 2006
9. Kurata, H El-Samad, R Iwasaki, H Ohtake, JC Doyle, et al. (2006) Module-based analysis of robustness tradeoffs in the heat shock response system. PLoS Comput Biol 2(7): July 2006
10. M Chiang, SH Low, AR Calderbank, JC. Doyle (2006) Layering As Optimization Decomposition, PROCEEDINGS OF THE IEEE, Volume: 95 †Issue: 1 †Jan 2007
11. Martins NC, Dahleh MA, Doyle JC (2007) Fundamental Limitations of Disturbance Attenuation in the Presence of Side Information, IEEE Trans Auto Control, Feb 2007
12. JC Doyle, ME Csete, and L Caporale. (2007) An engineering perspective: The implicit protocols. In The Implicit Genome, Edited by L. Caporale. Oxford University Press, 2007.

Back to The First q-bio Conference.