Definition
Herbert A. Simon is one of the first who tried to formalize the notion of a complex system: * H. A. Simon, “The Architecture of Complexity”, Proceedings of the American Philosophical Society, vol. 106, iss. 6, pp. 467-482, 1962.
First of all, here is how he defines it:
« Roughly, by a complex system I mean one made up of a large number of parts that interact in a nonsimple way. In such systems, the whole is more than the sum of the parts, not in an ultimate, metaphysical sense, but in the important pragmatic sense that, given the properties of the parts and the laws of their interaction, it is not a trivial matter to infer the properties of the whole. » p. 467-468
According to Simon the idea of hierarchy (and therefore of architecture) is preponderant.
« By a hierarchic system, or hierarchy, I mean a system that is composed of interrelated subsystems, each of the latter being, in turn, hierarchic in structure until we reach some lowest level of elementary subsystem. » p.468
Nowadays this definition can be considered as a keystone of complex systems theory. To find the architecture, the dependencies between the subsystems, how they interact and interface is more than ever the purpose of complexity modeling.
Redundancy
Still, complex systems are redundant:there are only a few different kinds of subsystems, and these are nearly decomposable, i.e. for a tolerable description of reality only a tiny fraction of all possible interactions needs to be taken into account, last, the redundancy that is present but unobvious in the structure of a complex system can often be made patent by appropriate ‘recoding’.
Lloyd justifies his classification of complexity measures by saying some are redondant:
« This multiplication of measures has been taken by some to indicate confusion in the field of complex systems. In fact, the many measures of complexity represent variations on a few underlying themes. »
He develops a typology of complexity, which I mentioned in my last post: difficulty of description, difficulty of creation, degree of organization. Besides, he just mentions complex adaptive systems, for which there are models but no practical interfacing measures: - S. Lloyd, “Measures of Complexity : a non-exhaustive list”, Department of Mechanical Engineering, Massachusetts Institute of Technology, 2001.
So, research in complex systems is to be focused on the essential:
« There is now a growing body of evidence that the activity called human problem solving is basically a form of means-end analysis that aims at discovering a process description of the path that leads to a desired goal. The general paradigm is: given a blueprint, to find the corresponding recipe. Much of the activity of science is an application of that paradigm: given the description of some natural phenomena, to find the differential equations for processes that will produce the phenomena. » (Simon 1962, p.479)
Gell-Mann calls this feature crypticity and adds that the other way round, having difficulty finding proofs for a given theory has to do with logical depth.
« In the human scientific enterprise, we can identify crypticity roughly with the difficulty of constructing a good theory from a set of data, while logical depth is a crude measure of the difficulty of making predictions from the theory. »
- M. Gell-Mann, “What is complexity ?”, Complexity, vol. 1, iss. 1, 1995.
Language
As far as I read, Simon, Gell-Mann and Lloyd do not write much about language.
Gell-Mann states that algorithmic information content is no use to linguistic complexity:
« This property of algorithmic information content (AIC), which leads to its being called, on occasion, ‘algorithmic randomness’, reveals the unsuitability of the quantity as a measure of complexity, since the works of Shakespeare have a lower AIC than random gibberish of the same length that would typically be typed by the proverbial roomful of monkeys. »
Nonetheless, Herbert A. Simon gives an interesting example of hierarchy in what he calls ‘human symbolic production’: a book. In a way, the divisions he describes are the scope of discourse analysis.
« One very important class of systems has been omitted from my examples thus far: systems of human symbolic production. A book is a hierarchy in the sense in which I am using that term. It is generally divided into chapters, the chapters into sections, the sections into paragraphs, the paragraphs into sentences, the sentences into clauses and phrases, the clauses and phrases into words. We may take the words as our elementary units, or further subdivide them, as the linguist often does, into smaller units. If the book is narrative in character, it may divide into “episodes” instead of sections, but divisions there will be. » p.469-470
This does not mean linguists do not read Simon. On the contrary, with the growing importance of models considering language as a complex adaptive system, his work is perceived as a seminal one. Talmy Givón for instance refers to him to introduce his understanding of syntactic complexity, and links the notion of architecture to the structural parsing of sentences (more precisely to the parsing tree depth).
« The gist of Herbert Simon’s (1962) seminal work on the architecture of complexity is that increased complexity is, at the most general level, increased hierarchic organization; that is, an increase in the number of hierarchic levels within a system. And as Chomsky (1957) has made explicit, the syntax of a simple transitive clause can already be a 3-level hierarchy. » p.4
- T. Givon, The Genesis of Syntactic Complexity : diachrony, ontogeny, neuro-cognition, evolution, Amsterdam, New York: John Benjamins Publishing Co., 2009.