Notes on software and processes for collecting, analyzing and acting on data

More: Related:

Complex Systems and Emergent Behavior: Most simply, a system is a collection of parts or pieces that interact, affecting each other over time. Each piece (more . . .)

Using Intuition: Decision Theory uses the mathematics of probability to guide decisions. But setting the odds and making sense of the numbers (more . . .)

About this site

Essays

Syndication available


tinderbox
Essays : Complex Systems and Emergent Behavior : Methods of Prediction

Systems theory can be more frightening than illuminating. If the simple act of linking one event to another can result in behavior that could never have been predicted from knowing the structure of the individual parts, then understanding systems seems like an endeavor unlikely to provide much that is useful in the real world.

Fortunately, based on our experience, we know this is not at all how the world is. While we don't know precisely what the weather will be like tomorrow, we can make a pretty good guess. Present and past measurements provide averages and trends that give us an even better guide to the future. And mathematical models of the atmosphere in which the complex interactions can reproduced, mathematically, in a computer get us even closer.

Of course there are unintended consequences when changes are imposed on a system. Try to change the behavior of a system in one way, the system may act in new, unforeseen ways. Build new public housing for low income families to replace the run down tenements in the decaying inner city? The concentration and isolation breeds more crime. And the public housing becomes a worse run down tenement over time.

Mathematical models may be feasible for big problems that need to be examined over an over. The atmospheric models are run multiple times a day, updated with new data as it becomes available. Structural models used to build jets and cars also benefit from constant refinement over time as new real measurement are made.

The data and assumptions needed for social systems are unique enough and hard enough to come by that good models are more rare. One approach is to model simple behaviors in conceptually reduced models of the system under examination. Macroeconomic problems can be modeled with a few agents in simple versions of large economies where there are millions of agents.

Simplification works best, perhaps, in highly fractal systems. If every part of the system is similar to every other part of the system and small parts of the system resemble big parts of the system then models of small parts predict the behavior of other small parts and the behavior of the system as a whole. Failing this, simplifying assumptions can render the exercise useless, providing completely inaccurate predictions of system behavior.

Using intuition is a way to simulate one complex system with another complex system. Imagining or guessing what will happen depends on having an accurate mental model of the system. But imagination is portable, fast and adaptable. Intuition is a faculty that should be developed by making predictions and tracking accuracy against reality. Learning.

What if you don't have the intuition? Or don't have the experience to have learned about the system? You can ask an expert for advice and he or she will give you some quick tips. Usually these tips come as rules or algorithms. "When the engine has been making a squealing noise and becomes hard to start", my mechanic said, "it's probably your alternator belt slipping. Try some belt dressing spray. If that doesn't work, bring it in and I'll have a look at it."

My mechanic's rules are specific to car systems. And many of the rules apply to Hondas but not Volvos. It would be useful to have rules about systems that can be generally applied when combined with knowledge of the system itself.

There's a large literature on leadership, management, organization and communication that consists of just such a set of rules that are intended to be widely applicable to systems composed of human agents. Perhaps some self help books are meant to help manage the system that is the mind of an individual. For example, Stephen Covey's 7 Habits of Highly Effective individuals can easily be recast into the language of systems theory.

I'm most impressed with Goldratt's Theory of Constraints because it is explicitly a simple, generalized way to deal with systems.

Copyright 2003 by James J. Vornov