Collective Intelligence
- an article for the Berkshire Encyclopedia of the 21st Century
By Peter Hesseldahl

Collective intelligence is a form of cooperation that is not organized as a hierarchy around a central authority. Instead the intelligence to make useful decisions emerges from interactions and contributions from a network of participants that make independent decisions.
Even though each participant typically acts for selfish and locally determined reasons, the result of the combined actions of the participants can be of more benefit to everyone involved than the simple sum of what each participant could have achieved alone. Thus it appears that the total system of participants can posses an intelligence at a higher level than any individual participant.

Collective intelligence is a concept that has been gaining in importance and attention in recent years. The increasing connectivity between humans, not least through the internet, is creating the conditions for this type of interaction. More people are connected through more powerful and fast connections, and the result is a rapidly growing number of applications that aim at harnessing the collective intelligence of the users.

Looking ahead, collective intelligence can be seen as one aspect of a very large scale paradigm shift towards an understanding of the relationship between the individual and the collective that is very different from the prevailing cultural paradigm of the past several hundred years.

Beyond the paradigm of central power
Robert Wright, in his book Nonzero (2001), describes the evolution of life on this planet as a continued process of connecting matter and living organisms in ever larger and more complex systems. Wright argues that this trend is inevitable, since cooperation tends to give the participants an advantage which makes them better fit to survive.
Single cell organisms have joined forces to gain combined strength and the ability to specialize functions, eventually giving rise to complex creatures like humans which have proceeded to gather in increasingly large and complex societies and, in time, a globally connected civilization.

Culturally, humans have mainly seen society as the venue of a fight for survival and resources between individuals, families or clans - a zero-sum game in which one persons gain is another's loss.
Humans have organized around strong leaders, often kings with almost absolute powers over their subjects.

Thomas Hobbes, an English political philosopher, referred to this strong ruler as ”The Leviathan”, in his book of that same name in 1651.
In Hobbes view, mankind would descend into a ”war of all against all" in which life of man is ” solitary, poor, nasty, brutish, and short”, unless guided by a strong, sovereign ruler.

The rise of industrialism in the 1800's was a triumph of centralized, hierarchical organization, epitomized by the assembly line where masses of workers did exactly what had been planned by a relatively few with the knowledge and power to engineer the production.

The prevailing model in industrialism was a one-way process. The intelligence resided at the center, where products were designed and produced by a shrinking number of ever larger companies. Media was broadcast from the cultural elite to the masses.

Meanwhile, mankind proceeded to become ever tighter connected. Continents were connected, roads built, cultures were mixed and today we see our culture engaged in a rapid globalization, in which most of the planet's inhabitants are connected, interacting and interdependent. The connectivity is strengthened by many means: Highways, airports, common standards, international trade agreements and banking, the spreading of English as a second language, and formal international institutions such as the UN or the European Union.
In recent years, of course, the spread of broadband internet access and mobile telephony has strengthened the global connectivity immensely. It seems likely that the pervasiveness and power of information technology will only accelerate onwards as more humans are connected more of the time by faster connections. Furthermore, humans will become more intimately linked to our technology as we built up an infrastructure of countless everyday objects that will be equipped with sensors, processing power and the capability to communicate and coordinate with the rest of the digital infrastructure.

As connectivity grew, the concept of collective intelligence started emerging. In a sense, the Scottish economist Adam Smith was describing collective intelligence when he argued that the ”invisible hand” of a free market would be the most efficient way to distribute resources. In 1776, as Adam Smith wrote this in ”The wealth of Nations”, trade and production was firmly regulated by laws established by the monarchies and strong (non-democratic) governments.
Against this centralized control, Adam Smith argued that even though everybody in society acted in pursuit of self interest, the result of all of these individual decisions would be more robust and beneficial to society than if the government interfered with the free market.
In other words, Smith expressed a belief that the best decisions could arise by combining the local knowledge, analysis and intelligence of all participants - rather than relying on a central authority.

Although Adam Smith believed in self interest he also felt that humans should act out of “sympathy”; a moral obligation to the common good. He described this in his book ”The theory of moral sentiments” in 1759. Apparently Smith thought that acting in self interest and contributing to the common good were not contradictions.

The confidence in the ability of the masses to self organize and to manage better without rulers is the central thesis of the anarchist movement.
The anarchist and socialist thinkers were contemporaries and both movements were engaged in the various revolts against capitalism in Europe and Russia in the decades around 1900. However, anarchism should not be confused with the central control paradigm of socialism.
The word anarchy originally means ”no ruler”, and the argument of the anarchist thinkers such as of the Russian Piotr Kropotkin was that much of human misery and exploitation was the result of organizing around authority.

Technology as an extension of humans
During the 20th. century the idea of collective intelligence was coupled closely to the development of information technology.
In his book, Understanding Media: The Extensions of Man (1964), the Canadian media theorist Marshall Macluhan described technology as an extension of our nervous system. Equipped with video cameras, weather satellites and phones we can sense what's going on around the world - just as we sense our immediate surroundings through our eyes and ears.
The crucial difference is that these extensions are not personal, but shared among everyone connected to the media and communication networks. We all access the same sensory system, we draw upon the same enormous memory and the same processing of information.

The British author and film maker Peter Russell in his book The global brain (1983) drew the comparison between the interactions of neurons in a brain and the interaction of people and organizations connected through media. Russell's subsequent film, also titled The Global Brain, brought the idea of an emerging super-organism, greater than any single person or organization, into the popular realm.
In Peter Russell's view, the technological connectivity is linked to a larger, spiritual awakening on the globe.

James Lovelock, then a scientist at NASA, proposed the Gaia Hypothesis in his 1979 book ”Gaia: A new look at life on Earth”. Lovelock described the complete system of earth - the biosphere, the atmosphere, the oceans and the soil - as one living organism, capable of maintaining an optimal physical and chemical balance for life on earth to thrive.
Although Gaia does not have an explicit consciousness, the global organism acts intelligently to sustain favorable conditions.

James Lovelock drew on the principles of cybernetics which had been formulated by early computer scientists, most notably Norbert Wiener, who in his book ”Cybernetics” (1948) described how systems composed of simple programs such as feedback loops can self correct and maintain balances as if they were autonomous and somewhat intelligent.

By the 1980's the concept of collective intelligence was generating a steady stream of literature exploring its implications.
Metaman, a book by Gregory Stock from 1993, describes how human culture and the globe-spanning technological system it has created, has started to behave as a planetary super-organism, capable of dealing intelligently with problems that are common to all of humanity. In Stocks view, humans have merged with our own technology. We are so dependent on telephones, cars, infrastructure, pacemakers, medicine etc. that the relationship between an individual person and the technological infrastructure we share with everyone else can be compared to that of cells in an organism.

Geoff Mulgan in his book Connexity (1997) likens this merging of humans into a shared super-system to humanity growing up; becoming less self centered, able to think more long term and act in larger context.
As the founder of the British political think tank Demos and an advisor to Tony Blair's cabinet, Geoff Mulgan has since translated this view of human interdependence into concrete recommendations for policies that involve citizens more actively as co-creators of the public services they use - in that way distributing decision-making away from the central authority.

Emergence
Biologists have observed how collections of rather simple organisms can act together in remarkably intelligent ways.
Slime mould is an example: Slime mould consists of many, small individual mold cells, but despite each cell being very simple, a slime mold cluster is able to expand and contract and thus move in search of food.

Likewise, anthills and bee hives have no central planning authority and each individual participant follows very simple rules. Never the less, the whole system is able to deal effectively with a wide range of circumstances with a sophistication that far exceeds that of any single ant or bee.
The way birds flock in extremely efficient and highly dynamic patterns is a similar example - as is the behavior of a well functioning soccer team, where each player is taking rapid decisions independently but in the context of the whole team.

In ”Smart Mobs” (2003) the American writer Howard Rheingold reported how the flocking behavior could be observed in communities that connect via mobile phones or text messaging: Teenagers use their phones to arrange meetings and flocking around particular parties or clubs. Rheingold also describes the same ad hoc, decentralized way of organizing in political contexts. For instance massive protests, arranged by forwarding text message from citizen to citizen, led to political revolt against the president in the Philippines in 2001.

This phenomenon is called self organization or ”emergence” and it is a feature of so-called complex dynamic systems. Given a sufficient number of agents, and a sufficient number and speed of interactions between them, patterns may emerge that are qualitatively different from the sum of the individual agents.
This can be observed in all kinds of complex dynamic systems, ranging from weather systems, stock markets, traffic patterns, and living processes in general.

The evolutionary mechanism of differentiating through mutations and sex, and selection by letting the best fit survive to reproduce is an example of how adaptation and optimization can take place without anyone consciously acting to achieve it.

Complex dynamic systems and the ways in which complex patterns arise from the interactions among simpler elements are the focus of the relatively young science of complexity theory.
The Santa Fe institute in New Mexico has been the center for the cross disciplinary study of complex systems. It was established in 1984.

The book Emergence: From Chaos to Order (1998) was among the first descriptions of the mechanisms of emergence. It was written by the American scientist John Holland who has been a close associate of the Santa Fe institute. In his book, Holland described how what amounts to autonomous collective intelligent behavior can emerge even in very simple mathematical simulations run on computers.

In his book ”Out of control”, Kevin Kelly, an American journalist and editor, gave a comprehensive overview of findings within biology, mathematics, complexity science, robotics and many other fields that illuminate how advanced, intelligent behavior can emerge in a wide variety of systems. From a complexity theory point of view, collective intelligence is an emergent property and it is likely to emerge as the number of connections and interactions in a systems grows.
Kevin Kelly argued that this is exactly what is happening to global culture. Mankind and our technological devices are connecting through telephones and the internet to the point where a significant part of the earths population is participating in a realtime worldwide exchange of information.
As the connectivity extends further towards a global complex dynamic system, one would expect increasingly powerful examples of collective intelligence.

Web 2.0 - a conscious effort to enable collective intelligence
Kevin Kelly wrote ”Out of control” in 1995, just as the World Wide Web was about to fundamentally change our access to and exchange of information.
Ten years later, in 2005, the publisher Tim O'Reilly coined the term Web 2.0 to describe the way the World Wide Web was increasingly being used for collaboration and co-creation between thousands of participants - rather than as a traditional one-way medium to broadcast information and sell products. O'Reilly defined the concept of web 2.0 as "the design of systems that harness network effects to get better the more people use them."

Websites like Ebay, youtube, MySpace, Del.icio.us and Flickr are examples of this new type of service, in which the users collectively create the content, and the company that operates the service focuses on creating the tools and structures that enable the users to co-create and interact.
By uploading their own content, and by remixing, tagging, aggregating and filtering the contributions of others, users collectively process information - in a sense collectively thinking and constructing meaning.

Perhaps the best example is Wikipedia, the online encyclopedia that anyone can help edit. With a minimum of central planning, the voluntary, unpaid efforts of thousands has resulted in a rapidly evolving, self correcting and increasingly detailed collection of knowledge, freely available for anyone to use and contribute to.

The free or open source software movement is another example of how a non-commercial, loosely organized collaboration among participants that often don't even know each other can create results that can be competitive to and more robust than what large and wealthy hierarchical organizations produce.
The essay, The Cathedral versus the bazaar, by Eric S. Raymond, 1997 describes the work ethic, motivation and principles of cooperation among open source software programmers.
Raymond argues that a centralized organization is unable to respond as well and as quickly to the needs of the users of a service, whereas open source software is created by people who are driven by an individual wish to fix problems that they themselves are confronted with, or to improve programs by adding features that they themselves would like to have.

The fundamental principle behind free or open source development is that developers stand on each others shoulders. The open source developer furthers his goals by adding to and building upon the common pool of available features. If the code remains open for anyone to work on, corrections can be suggested and ideas can be improved.
Thus, the open source attitude challenges traditional notions of exclusivity and secrecy. If the goal is to maximize the collective intelligence rather than getting the individual player ahead relative to others, it is desirable to share information and to build as directly as possible on the creations and knowledge of others. Secrecy, in that sense, becomes an obstacle that slows down the realization of the collective potential; in effect ”dumbing down” the collective intelligence.

Although the technological connectivity seems to drive mankind towards a more bottom-up, networked and collective form of decision making, it is unlikely that the top-down command structure will disappear.
In circumstances that are relatively stabile and predictable a central ruler with a clear vision can achieve very specific goals much faster than a self organizing process. However, in a complex and changing context, it is unlikely that a central ruler can make intelligent choices, simply because no single player has an overview and understanding of the entire system.

Clash of dimensions
Arguably, the same connectivity and globalization that allows collective intelligence to emerge is creating issues of a nature that only collective intelligence can deal with.
The effects of human activity are global. Our technological systems - e.g. the internet or stock markets - are crucial to our civilization, yet they function without a central decisive authority. No single person or authority can understand, let alone control, them fully.
In particular, environmental issues like pollution, resource depletion and climate change require the cooperation and compliance to common rules and goals from most of the nations on the planet in order to be managed.

Environmental problems clearly illustrate an inherent clash between two dimensions: the interests of the individual against the interest of the whole. This conflict has been described by the American ecologist Garret Hardin as ”the tragedy of the commons”: Humans rely on resources that are shared and belong to no one. Even though these common resources are finite and depleting them would hit everyone, for each individual the immediate personal benefit from using more of the common resource tends to outweigh the long term danger of exhausting the whole system.
At the personal level it is not very attractive to reduce one's environmental impact by using the car less, or forgoing convenience to cut CO2 emissions. Even though the total system is clearly at risk, the effect on the greater goal of avoiding climate change seems negligible compared to the personal inconvenience.

This is the opposite of collective intelligence - collective madness, one might say. Thousands of individuals sharing their knowledge and thinking can create much more value than the simple sum of the individual contributions. But likewise, thousand or millions of people can each make decisions that are perfectly rational at the individual level, but at the statistical level, the combined result can be destructive to everyone.

Taking an optimistic view, one could hope that the capability to deal intelligently with issues at a global level could emerge due to the increasing connectivity which
- - Gives the individual a better sense of the consequences of his or her actions at a collective level
- - Exposes the individual to be credited and held accountable by others
- - Reduces the lag time between an individual action and the reaction of the total system

In short: When a system is coupled tighter, and the interdependence of the players increases, the difference between altruism and egoism is reduced. Overcoming the clash between the individual and the aggregate dimension could be an aspect of a collective intelligence emerging.

The Noosphere
Many philosophers have felt that the merging of humanity into a connected intelligent super-organism will be the inevitable result of evolution.
The Russian mineralogist Vladimir I. Vernadsky (1863-1945) speculated that the development of earth had three stages; first came the geosphere - the inanimate matter; rocks and minerals. Then the biosphere developed, and eventually a new realm of collective thought and consciousness will emerge. Vernadsky called it the Noosphere, employing the ancient greek term for “mind”; nous.

Pierre Teilhard de Chardin, a French Jesuit priest and philosopher saw the noosphere developing to reach an ”omega point”, at which evolution culminates in its total potential complexity and thus in the highest possible level of consciousness of the universe - a unified state, that de Chardin thought of as a re-uniting with God.
A somewhat similar concept is the Supermind, a future divine state of collective consciousness envisioned by the Indian guru Sri Aurobindo (1872-1950).

The singularity
A number of strict mathematicians and scientist have predicted the emergence of a super-human intelligence which would completely change the power structure on the planet; the singularity.
The American inventor Ray Kurzweil (born 1948) and the author and mathematician Vernor Vinge (born 1944) have argued that we may soon be reaching a point where the millions of computers connected through networks achieve a level of intelligence that exceeds the capability of the human brain. Contrary to human brains, this computer intelligence is developing rapidly, and therefore likely to develop beyond human intelligence at an enormous and accelerating speed - leaving humans behind as an inferior life form.

In his book, Robot: Mere Machine to Transcendent Mind (1998) the robot scientist Hans Moravec tried to describe how the nature of a machine intelligence would be different from a traditional human intelligence. These traits seem to be found generally in collective intelligences. Among Hans Moravec's speculations were that:
A machine intelligence would be networked, and thus not confined to or located at any particular geographical location.
It would be ad hoc, temporary and purpose driven; emerging by mobilizing available computing power to deal with particular issues and then dispersing to reconfigure with other processors to solve other problems.
The intelligence would not be individual or personal, it would exist only as a combination of many processors.


Further reading:
Wisdom of the crowds, James Surowiecki, 2004
Collective Intelligence: Mankind's Emerging World in Cyberspace, Pierre Levy, 1998
Information, a trilogy by Manuell Castells, 1996-1998