Wisdom of Crowds

Matt Barton's picture

Lately, I've been reading a very exciting book called The Wisdom of Crowds, authored by James Surowiecki. I don't usually write about books here at AA, but this one is just as good as Donald Norman's The Design of Everyday Things, another must-have if you do any reading whatsoever on technical topics.

Surowiecki's book looks at some amazing "coincidences" concerning how groups work successfully (or fail). For instance, if you get enough people to make guesses about how many beans are in a jar, then take the average, chances are it'll be stunningly accurate. However, Surowiecki doesn't claim that collectives always make the best decisions; there are a number of factors that play into it. For one, the individuals in the group need to be autonomous and not swayed by others in the group. For instance, imagine a group of 8 new scientists and 2 famous scientists. Assuming the 2 famous ones agree and try to influence the rest, the other 8 are very likely to acquiesce, assuming that if they disagree, it must be because they are wrong.

Diversity also plays into this; but not necessarily racial/cultural diversity. The idea is that it's better to have a group composed of people from different educational or technical backgrounds, since they will be able to bring different insights to the table. The key again is to have some way of aggregating the data so that the members don't change their answers based on how others respond. Surowiecki claims at one point that the old NASA was actually far superior to the modern one, since in the old days the employees came from different industries and academic/technical programs. Now their training tends to be more the same, with the employees going through the same or similar graduate programs, etc., then expected to conform. I really think Surowiecki is on to something here.

Surowiecki also talks about small groups and why they often fail. Again, small groups can be terrible because one or a few members tend to dominate the rest, inhibiting discussion and trying to silence opposition for the sake of a crappy consensus. I know I've been butting heads lately with several small groups of exactly this type; they have already made their decision, and their efforts at "discussion" boil down to getting everyone else in the group to agree with them and be silent. Unfortunately, these silencing efforts almost always lead to bad decisions, since there was probably a damn good reason other members of the group were not in agreement (but the leaders refuse to listen).

I've been conducting some experiments to test out Surowiecki's hypothesis. One is that I'm having my students anonymously predict the final grade of a few papers they heard in class. I asked them to predict the final score, assuming that the student would try to revise the papers accordingly. I haven't looked at them, of course, but after I grade the final papers I'll compile the averages and see how close they match. I plan to add something to the next round; anyone who guesses exactly right will get bonus points. That might provide a little more incentive to make a careful guess.

I haven't finished the book yet, but I'm very curious how all this will apply to wikis, multiplayer online games, rating systems, and the like. As you can see, it's almost impossible to read this book without discussing it with other people, so by all means--get a copy and let me know what you think!


davyK's picture
Joined: 05/21/2006
This is by no means a new

This is by no means a new idea Matt. I first came across it when I was studying for my MBA back in '98 or '99.

One cited example was prediction of American Football games. In a study the experimentors took a group of pundits, commentators etc. and amalgamated their predictions for a set of upcoming games - and the result was the average predictions computed from all of their predictions was very accurate - far more than any individual could have been.

The interesting point to note though is that when a sample of non-experts were tried they, or their combined predictions , were not accurate at all. It seems what is termed "competence" or a knowledge of the area under study is required for this to work.

This concept is used to talk about how open market forces are able to arrive at accurate stock prices for companies - reflecting the true "value" of a company, provided that the share dealers have a knowledge of the industry the companies work in. Of course you get problems when lots of dealers who don't have knowledge get involved, or other effects as you mention such as the influence of powerful members of a group.

Fascinating stuff.

Rowdy Rob
Rowdy Rob's picture
Joined: 09/04/2006
Thanks for the Heads Up!

This sounds like my kind of book. When I'm nearing a "reading phase," this will be at the top of my list.

qoj hpmoj o+ 6uo73q 3Jv 3svq jnoh 77V

Matt Barton
Matt Barton's picture
Joined: 01/16/2006
You're right, DavyK, it is

You're right, DavyK, it is not new, which is one reason the book is good--he tells you about all the studies so you could really delve much deeper if you were so inclined. I'm okay ATM with his summaries, though. :)

One thing he talks about is how experts can be bad if they are too limited or specialized. You really need outside information coming in to balance out things. For instance, a group composed of Bucks fans who knew little about other teams wouldn't be as good as a group composed of fans of various teams, since their combined knowledge would be more likely to hit on a good prediction.

davyK's picture
Joined: 05/21/2006
Yeah....the old "Groupthink"

Yeah....the old "Groupthink" effect can of course have adverse effects too.

Its fascinating to think about the power of the subconscious mind - especially when you take groups into effect too. There's a book called "Blink" which talks about the power of the mind and how we can make fast decisions based on really small amounts of info.

Makes you think about the the universal consciousness and if we really are individuals - I personally know a rather obscure author - Robert Barry - who talks about individuality being an illusion in his book "The theory of almost everything" that he wrote quite some time ago now.

Some people believe that phenomena such as remote viewing and other ESP type senses are really only the ability to tap into the universal conscience - maybe we are all part of one - maybe God does exist and we are all part of that one!

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.