The Wisdom of Crowds

This is a very interesting book that covers many complicated subjects related to group decisions vs. individual decisions. It touches on Game theory, behavioral economics, and Decision theory. However, it is not a treaty in any of these areas. The author keeps his observations at a 10,000 feet high level. So, the book instead of being a dry tome for mathematicians is actually a very entertaining book for laypersons. The author touches on the many aspects where crowds provide superior judgments than individuals alone. Our civilization quietly depends on many such favorable situations, These include the capital markets in general, and the stock market in particular. But, it also includes Nielsen ratings, polls, voting records. Most of the time, in all these circumstances the many give a better assessment, valuation, or judgment than the individual. The author is quick to point out that this is not always the case. Markets experience stock market bubbles where the collective judgment becomes euphoric. However, year in year out and over decades, the stock market (representing the many) beats the majority of the investment pros hands down. The author is fascinated by the emergence of "decision markets," including the Iowa Electronic Markets for betting on Presidential election outcome that has proven more accurate than the polls. Another such example is the tradesports website that does the same, including betting on sport events, media events, international politics events. These are perfect examples where the many gather their judgment through trading values thanks to the Internet. In general, it is uncanny how accurate these decision markets are. There is really something to the saying "put your money where your mouth is." In a nutshell, this is a very interesting and thought provoking book. It promotes democratic and decentralized decision making within corporations and government institutions. This makes good sense. The CEO or the President just can't hold that much information in their individual brains anyway. Instead, why not rely a lot more extensively on the collective wisdom generated by the aggregated sum of our own individual brilliance. Why not!

Crowds have a bad rap. Our opinion of collective decision making and behavior has been darkened by first-hand observation of events such as the stock market bubble and crash, and by classic works such as Extraordinary Popular Delusions and the Madness of Crowds, and The Crowd: A Study of the Popular Mind. That is unfortunate, argues James Surowiecki because, under the right circumstances, groups "are remarkably intelligent, and are often smarter than the smartest people in them." If nothing else, The Wisdom of Crowds should entertain you. It may not do much more than this if you are already well read in economics, complexity theory, decision analysis, organizational theory, social psychology, prospect theory and other fields. Few readers will have delved into all the relevant areas to any great extent, so most can expect to learn something new, interesting, and quite possibly useful. Surowiecki's wide-ranging gathering of sources to support his argument is a virtue, yet it's also something of a problem. The difficulty of knowing much about all the areas on which he draws makes it easy for him to pick and choose studies and arguments selectively. While many of his points are well made, the way he supports his case sometimes seems one-sided. In evaluating and supporting the idea of the wisdom of crowds, Surowiecki looks at how collective intelligence can be applied to three kinds of problems: Cognition problems (which have definitive solutions), coordination problems, and cooperation problems (which require self-interested agents to work together). The first half of the book sets out the theory, thoroughly and entertainingly illustrated by examples. These include the smarts of the audience on game shows, how to design an excellent search engine, why short selling is a good thing, and how a group finds a lost submarine. The second half of the book applies the ideas to show various ways in which people organize toward common goals in cases such as traffic, science, juries, committees, business organizations, markets, and democracies. Among the main points that may be useful to executives, Surowiecki emphasizes that for the crowd to be wise, it must be characterized by diversity of opinion, independence of members from one another, and a specific kind of decentralization, and there needs to be a good method for aggregating opinions. He stresses that the best collective decisions result from disagreement and contest, not consensus or compromise. While corporations often rely on experts, the book does well at challenging our confidence in expertise as compared to the average of the crowd. In the course of a discussion of the role of independence, we learn that to improve your organization's decision making you should ensure that decisions are made simultaneously rather than one after the other. Finally, I have to second Surowiecki's puzzlement at the apparent lack of interest by companies in using markets (such as decision markets) for corporate strategy and market research.

This is a very important book. James Surowiecki presents a wonderful spectrum of examples of how collective consciousness is superior to individual contributions to that consciousness. In the simplest example, Francis Galton, a British scientist, attended a country fair. He was curious in a weight-guessing contest to see how close the average of all guesses came in assessing the weight of an ox after it had been slaughtered and dressed. The meat was the prize for the closest estimate. He expected that the average of the 787 legible submissions would be considerably off the mark, because many people with no expertise whatsoever were participating in the hopes of winning. "Many non-experts competed," Galton wrote... in the scientific journal Nature, "like clerks and others who have no expert knowledge of horses, but who bet on races, guided by newspapers, friends, and their own fancies." The analogy to a democracy, in which people of radically different abilities and interests each get one vote, had suggested itself to Galton immediately. "The average competitor was probably as well fitted for making a just estimate of the dressed weight of the ox as an average voter is of judging the merits of most political issues on which he votes," he wrote. (p. xii) The average of all guesses was 1,197 pounds and the ox weighed 1,198 pounds. Surowiecki notes that many have expressed serious skepticism about the wisdom of groups of people. Notable among these have been Charles Mackay, a Scottish journalist, who wrote about the madness of crowds in 1841; Bernard Baruch, an early 20th century speculator; Henry David Thoreau; and Friedrich Nietsche. Surowiecki acknowledges that there are situations in which crowds demonstrate execrably poor wisdom, as in the crowds who egg on people to jump when poised for suicidal leaps to their death. Countering the skeptics and the dictates of simple logic as stated by Galton, Surowiedki, with a marvelous gift of pattern recognition, expands upon his original example, considering the wisdom of crowds in addressing various types of problems. He demonstrates repeatedly, in diverse situations, how the collective wisdom of groups of people outweighs the wisdom of any of the participants in the group - even the judgments of the most educated and expert participants in these groups. A lovely example is that of the US submarine Scorpion, which disappeared in the Atlantic with no known cause. ... Although the navy knew the sub's last reported location, it had no idea what had happened to the Scorpion, and only the vaguest sense of how far it might have traveled after it had last made radio contact. As a result, the area where the navy began searching for the Scorpion was a circle twenty miles wide and many thousands of feet deep. You could not imagine a more hopeless task. The only possible solution, one might have thought, was to track down three or four top experts on submarines and ocean currents, ask them where they thought the Scorpion was, and search there. But as Sherry Sontag and Christopher Drew recount in their book, Blind Man's Bluff, a naval officer named John Craven had a different plan. First, Craven concocted a series of scenarios - alternative explanations for what might have happened to the Scorpion. Then he assembled a team of men with a wide range of knowledge, including mathermaticians, submarine specialists, and salvage men. Instead of asking them to consult with each other to come up with an answer, he asked each of them to offer his bst guess about how likely each of the scenarios was. To keep things interesting, the guesses were in the form of wagers, with bottles of Chivas Regal as prizes. And so Craven's men bet on why the submarine ran into trouble, on its speed as it headed to the ocean bottom, on the steepness of its descent, and so forth. Needless to say, no one of these pieces of information could tell Craven where the Scorpion was. But Craven believed that if he put all the answers together, building a composite picture of how the Scorpion died, he'd end up with a pretty good idea of where it was. And that's exactly what he did. He took all the guesses, and used a formula called Bayes's theorem to estimate the Scorpion's final location. (Bayes's theorem is a way of calculating how new information about an event changes your preexisting expectations of how likely the event was.) when he was done, Craven had what was, roughly speaking, the group's collective estimate of where the submarine was. The location that Craven came up with ws not a spot that any individual member of the group had picked... The final estimate was a genuinely collective judgment that the group as a whole had made, as opposed to representing the individual judgment of the smartest people in it. it was also a genuinely brilliant judgment. Five months after the Scorpion disappeared, a navy ship found it. It was 220 years from where Craven's group had said it would be. (pp. xx-xxi) Cognition problems Surowiedki examines the unusual situation of the TV show, Who wants to be a millionaire? Contestants could walk away with a million dollars if they correctly answered 15 successive multiple-choice questions of increasing difficulty. Contestants could call upon a trusted outside advisor or on the TV audience (who responded by computerized votes). We might guess that logic would suggest that the smartest person contestants could pick ought to score better than the random collection of people sitting in a TV studio on a weekday afternoon. Well, guess again. The experts answered correctly 65 percent of the time, while the audience was 91 percent on target. Surowiedki reviews many research studies of guesses similar to Galton's original situation, such as estimating beans in a jar or ranks of items by weight. Invariably, the average of group guesses is closer to the actual number than the vast majority of individual guesses. In another example, gamblers' betting odds show that the public is extremely savvy, and those who set the odds are likewise very astute at guessing outcomes of events. What is even more fascinating is that a diverse group that includes experts and non-experts in fields relevant to a problem being addressed will usually do better than a group composed only of experts in the relevant field. He then expands to consider votes by public purchases and sales of shares on the stock market following the space shuttle Challenger disaster of 1986. Within minutes following the disaster, the prices of shares of contractors that could have been involved in causing the disaster dropped: Lockheed (ground support manager); Martin Marietta (manufactured the external fuel tank); Rockwell International (builder of the shuttle and its main engines); and Morton Thiokol (built the booster rocket). By the end of the day, the price of Thiokol had dropped 12 percent, while the other prices had each rebounded from 6 percent to 3 percent drops. It took six months to identify what caused the disaster (O-rings designed by Thiokol), but the wisdom of the stock market crowd was right on target on day 1 of the disaster. Detailed investigations (including scrutiny of possible insider trading) turned up no clues to how the public immediately identified the culprit. Surowiecki believes that the wisdom of crowds explains this unusual finding. He identifies four contributing components to this wisdom: diversity of information and opinions; individual participants' independence in their contribution to the guesses; decentralization of sources of knowledge; and aggregation of the individual opinions into a collective decision. Coordination problems The wisdom of groups of people is challenged when they must coordinate the opinions and actions of large numbers of people. There are situations in which it is very difficult to sort out how to achieve the maximum benefits from the inputs of individual group participants, as in factories with many separate steps in production lines. Surowiecki demonstrates that the wisdom of groups of workers can often overcome these potential difficulties in successful collaborations. Cooperation problems Trusting strangers is something we do all the time, without thought, particularly in commerce. Surowiecki discusses how such trust developed as international commerce developed, and presents various studies on how people will cooperate in market settings. The broader implications of the issues discussed in this book are far-reaching. Surowiecki makes a good case for a trust in democracy as a form of government, if the special interests of lobbying influences can be controlled. What I found of most interest was a hope in the collective wisdom of mankind to deal with the challenges of global heating. A serious deficiency in this book, however, is a total lack of consideration of intuition and collective consciousness - for which there is a major body of substantiating research. These constitute major further potential strengths in the wisdom of groups of people. A prime example is in the collective guesses that led to the location of the Scorpion. Another annoying deficiency of the book is the lack of an index.

Subtitled, "Why the many are smarter than the few and how collective wisdom shapes business, societies and nations." He starts out by quote a great curmudgeon -- H. L. Mencken: "No one in this world, so far as I know, has ever lost money by underestimating the intelligence of great masses of the plain people," then proceeds to prove him wrong. Replete with examples of how the aggregate wisdom of a group is greater than the smartest member, his first story is of a 1906 Exhibition where British scientist Francis Galton compiled the guesses on the dressed weight of a price ox by 787 attendees, only to find that the average guess was within 1 lb.! Surowiecki cites three basic problems that collective intelligence can address: Cognition - problems that have a definite solution (who will win the Superbowl?), Coordination - trying to get everyone on the same page (driving safely in heavy traffic) and Cooperation - getting self-interested, distrustful people to work together (paying taxes, dealing with pollution, etc.). Scores of case studies come from finding a lost submarine (a group of independent individuals' scenarios located the sub within 220 yards of where it went down from a search area that was 20 miles wide -- sadly, too late for the crew) to how Google finds what you want to the Challenger disaster (how not to do it, unfortunately). Surowiecki shows the importance of independence, diversity and private judgment to improve the results. Actually, too many experts or too much collaboration hinders more than helps. He shows that "the best collective decisions are the product of disagreement and contest, not consensus or compromise. An intelligent group . . . does not ask its members to modify their positions in order to let the group reach a decision everyone can be happy with." Interestingly enough, although communication is important in groups, too much communication reduces the results of the decision process. Experts can add to the problem in the same way. A fascinating book, rich with examples from business, industry, education, government, etc.

This is an interesting study - really, a collection of anecdotes - about how aggregations of diverse and independent individuals sometimes (how often?) come to better decisions than even the smartest individuals among them. But it is deeply flawed by the author's implicit definition of "rationality" as wealth maximization. Throughout the book, he claims that individuals defy the economists' assumption of rationality. No economist adopts such a narrow, impoverished, and obviously false definition of "rationality," according to which we could easily prove that no individual in the world, from Mother Teresa to the richest person in the world, is rational. (Richard Posner defines rationality as wealth maximization, but he's not an economist.) People seek to fulfill many many preferences that have nothing to do with wealth. This does not make them "irrational" in any economic or non-economic sense. Individuals satisfy the economic requirement of rationality so long as their preferences (at any given time) are transitive (if they say they prefer wearing long pants to shorts, they don't wear shorts when they have the option of wearing long pants). Their rationality may well be "bounded" by various cognitive and information constraints; they may use heuristic devices to make decisions, instead of acting as human calculators of costs and benefits. But none of that makes individuals "irrational." Indeed, given bounded rationality, reliance on heuristic devices is decidedly rational from an economic (transaction-cost-saving) point of view. The author's constant mis-application of the concept of economic rationality throughout the book is (very) annoying, but it does not negate his larger thesis. To the contrary, his thesis would be even stronger if the author recognized that groups can be smarter than the smartest individual among them, even though all everyone in the group is behaving in accordance with economic definitions of rationality.

Surowiecki's book focuses on the strengths and weaknesses of collective decision-making. The text covers a large number of issues rather superficially, and it will not satisfy those looking to dive into and master the underlying economic theories. But it makes an important point and can be read quickly. His thesis is that groups of people can aggregate information to produce surprisingly accurate decisions. The book starts by recounting a famous experiment, in which a 19th century Scottish scientist studied the results of a typical contest where county fair-goers guessed the weight of an ox. He found that, while many guesses were wide of the mark, the mean of the guesses was spot on -- and closer to the ox's actual weight than any individual guess. Apparently this phenomenon holds true for other such contests (how many jellybeans in a jar or how much a pumpkin weighs) -- and this ought to make one stop and think about the power of aggregating information from groups of ordinary people. Surowiecki relates the power of group thinking to other situations, ranging from the efficacy of market prices to the eery accuracy of the Iowa Electronic Markets (which forecast election results among other things). He contrasts these examples with the dangers of "group think," the consensual and wrongheaded approach to decision-making that plagues governments and corporations, the frequently poor quality of "expert" forecasts, and the volatility of security prices (he explains the theory of "information cascades" in a clear and easy-to-follow manner). The moral of the story: "Diversity and independence are important because the best collective decisions are the product of disagreeemant and contest, not consensus or compromise." It strikes me that this bit of wisdom could be usefully applied in government, business, investing, and many other walks of life; hence the value of the book, even if you skim it quickly.

Surowiecki's The Wisdom of Crowds documents and analyzes an extremely important phenomenon. When people guess at a question to which nearly no one knows the answer but most people can make a sensible guess (e.g., what proportion of the world's airports are in the USA; how many marbles can fit into a box that is a meter on each side) the average of a large group is nearly always more accurate than the guess of any member of that group. Moreover, the more people involved, the more accurate the average is. This phenomenon was first discovered by Francis Galton, Charles Darwin's first cousin. Throughout his life, Galton was obsessed with measurement and categorization. His study of fingerprints led to their use by the police to identify criminals. His study of twins revealed that biological heredity determines intelligence and temperament. He also worked out the coefficient of correlation, which is a basis of modern statistics. In 1906, when Galton was eighty-five and still as inquisitive as ever, he visited a country fair. One of the events was a contest to try to guess what the weight of an ox, which was on display, would be after it had been killed and dressed. In order to enter the contest, a person had to pay sixpence and write his guess, along with his name and address, on a piece of paper. After the contest was over, Galton borrowed the papers with the guesses. There were 787 papers in total. To his amazement, the average guess was only one pound less that the actual weight (1,198 pounds). That was closer than any individual guess. Yet, some of the participants in the contest were butchers and cattle raisers, who would be expected to do much better than the average. This phenomenon also applies to predictions of future events. That is why it is nearly impossible to make money consistently by betting on sporting events - because the odds are determined by the average of all bets, which is extremely accurate. The most striking illustration of this phenomenon is the otherwise inexplicable fact that, with very few exceptions, professional stock investors (managers of mutual funds, pension funds, etc.) do worse than the stock market indexes. Professional investors are highly intelligent people, who devote all their time to analyzing stocks and stock market trends; they have specially developed soft-ware and teams of assistants. But their analyses are not as accurate as the average analysis of all investors. However, for the wisdom of crowds to work, two conditions must be met (pages 10-11, 36-7). One is that the opinions of the individuals involved must be formed and expressed independently of the others. When decisions are made by groups meeting together, the group is often swayed by a consensus that seems to have formed or by one or two of its member who seem to have more expertise, or who express their opinions forcefully. Also, the individuals involved must have some knowledge of what is involved. For example, the average prediction of Indonesian peasants as to which team will win the National Football League championship would be worthless. In line with the second condition, the wisdom of crowds does not apply to areas of technical expertise. The average guess of a crowd as to the weight of an ox when it has been slaughtered and dressed is more accurate than the estimate of any butcher in the crowd. But the butchers in the crowd are more adept than any non-butchers at carving the ox, storing its meat, and preparing it for sale. Surowiecki offers a general explanation (pages 10-11) for why the crowds are wiser than any of their individual members and specific reasons (pages 34, 44-50) for why professional stock investors do worse than the stock indexes. I found the latter explanations more convincing than the former. Surowiecki also discusses (pages 236, 245-6) the obvious objection that the average prices of stocks, houses, and commodities often rise and fall sharply, without regard to the value of the assets involved. Such price swings do not occur with ordinary goods and services, such as televisions or haircuts; nor does a rise in price of ordinary goods and services induce more people to buy them. Average predictions of the results of sporting events or of elections are also immune from irrational price bubbles. In these cases, the accuracy of the predictions is decided unequivocally at a specific time in the near future. That keeps the crowd tethered to reality. But when the prices of stocks, houses, and commodities rise, that means that they can be resold at a higher price; and there is no point at which the bet is definitely over and the bettors have undeniably been proved right or wrong.

Suroweiki engrossed me from the beginning. Though this book appears to be a collection of anecdotes about how crowds often outthink the experts, it struck me as a blueprint for how decision-makers should harness the power of people. Thus it is a treatise on smart business and marketing, good government, and sound organization management. As a U.S. Army veteran, the author propelled me to thoughts on how the military could use its people's collective wisdom, something on which I have written extensively: Nine Weeks: a teacher's education in Army Basic Training Among the most relevant claims from the book is this cogent bit of logic: "To state the obvious, unless people know what the truth is, it's unlikely they'll make the right decisions. This means being honest about performance. It means being honest about what's not happening. It means being honest about expectations. Unfortunately, there's little evidence that this kind of sharing takes place....One of the things that gets in the way of the exchange of real information is the deep-rooted hostility on the part of bosses to opposition from subordinates. This is the real cost of a top-down approach to decision making: it confers the illusion of perfectability upon the decision makers and encourages everyone else simply to play along. What makes this especially damaging is that people in an organization already have a natural inclination to avoid conflict and potential trouble. It's remarkable, in fact, that in an autocratic organization good information ever surfaces. It's a book that anyone who has been around people should read.

I happened to pick this book, after I listened to a recommendation in a speech. Written in 2004, James argues that the crowd (provided there's diversity of opinion; independence; decentralization and aggregation) has the best wisdom, and is capable of making the best decisions. There are loads of references to famous authors, researchers and books all across the chapters. The fusion of cognition, cooperation and coordination among the members of the crowd is what filters out the outliers and brings the best possible decisions. For those working on policy and non-definitive decisions, this is a fantastic eye opener. Worth a read. I wish there was an updated version of the book available, relevant for today's world.

Un libro interessante e per certi versi scioccante che ci fa capire il potere delle masse. Lo consiglio anche a chi come me, da profano, vuole esplorare un po' il mondo della psicologia sociale.

Fascinating book that should be read by anyone caught up in the processes of 'decision-making' - voters as well as gamblers. Simply expressed, lucid insights which shed much light on the business of everyday life and politics. Excellent delivery service.

ここ数ヶ月の間に読んだマネジメント、統計、投資関連の複数の本(例えば“The Future of Management”[Hamel], “The Upside”[Slywotzky], “What Were They Thinking?”[Pfeffer], “Expert Political Judgment”[Tetlock], “Super Crunchers”[Ayer], “Black Swan”[Taleb],等)で幅広く引用されており、気になっていた本書を遅まきながら読んだ。 『一部の専門家やプロの判断よりも、知識や経験のレベルや領域もそれぞれの(専門家も含む)多くの人達による集合知(或いは全体の平均値)の方が正しいことが多い』という、ともすれば直感的には「えっ？ホント？」と疑わしく感じることを、多くの事例と実証研究を紹介しつつ説いている。但し、集合知が有効に機能する為には、とりわけ構成員の意見が多様であり(diversity)且つ他人の意見に影響を受けない (independence)状況が必要である旨強調している。 本書で取り上げられている例は、動物の体重や瓶の中のjellybeanの数、消息不明になった潜水艦の位置の推定から始って、プロスポーツの勝敗や大統領選候補者の指名予測、ハリウッド映画の興行成績の予測、自動車エンジンの発達等イノベーション、スペースシャトル事故、税金、株価形成、組織運営や経営の意思決定、等実に広範囲に亘る。 個人的には、なぜ本書が近年のマネジメントの方法論、とりわけ組織運営・リーダーシップ・イノベーション分野でのアプローチに大きな影響を及ぼしているのかにつき、大いに納得すると共に、experimental economics（実験経済学）やシミュレーション等が意思決定に応用されていく可能性に期待できると感じた。巻末注の関連文献も興味深いものが多い。

I actually bought this copy to give to one of the Directors of a large company that I advise. He was dabbling in consensual feedback as the core of a brainstorming session that I was in and I mentioned this book and then ordered it for him before he went back to the US. First comment - I love Amazon prime, because you can do this - think of a book and receive it so quickly. As for the book, it is something that sticks in your head and comes back to you at the most unexpected moments - even at my kid's Xmas fair I looked at the 'how many sweets in the jar' stall differently. Very insightful and useful at meetings where you can see the dangers of directed groupthink most clearly but also useful as a general way of thinking about problem solving. Highly recommended.