[This is the second post in a 5-part series, excerpted from my forthcoming book: Find it Fast: Extracting Expert Information from Social Networks, Big Data, Tweets, and More, 6th. Ed., CyberAge Books, Autumn 2015]
No doubt you've heard the phrase "The Wisdom of Crowds." Some deride the concept as just putting one's trust in an angry or ignorant mob; others dismiss it as a philosophy that ignores the value of the credentialed expert. And while a discussion of mob behavior or the value of the credentialed expert today would be an interesting one, it’s not what the wisdom of crowds is about.
James Suroweicki, a writer for The New Yorker is the person credited with coming up with the phrase. His book, which explored how and when large groups of people can be more accurate in answering factual questions or making decisions was in fact, titled, “The Wisdom of Crowds.” In the book Suroweicki looked at problems ranging from simple ones like guessing the number of jelly beans in a jar, to knowing the answer to popular trivia questions, to making predictions about future events, or coming up with solutions to large social problems and showed how large groups of people were more likely to come up with the right or best answer than a single individual, even if that person was an expert.
Suroweicki’s most important discovery, though, were his critical qualifiers outlining the four specific conditions that must be present for a crowd to act “wisely” (that is, be more accurate, solve a problem and/or come to better decisions). Those conditions were: independence, diversity, decentralization and aggregation.
Here’s what each of these means in practice:
Independence requires that each person in the crowd was able to cast a vote or make his or her decision independently--there was no knowledge or influence of the actions of other people (as can happen, for instance, on consumer rating sites where visitors can be influenced by previous ratings); diversity simply means that the voting or decision making group was not too much alike; decentralization means that the voting/decision making process was not centralized but done in a decentralization manner and aggregation means that there is some process for adding up and calculating the group's final answer from all of the individual responses.
But there is another important element to note here too: while these are the four conditions that must be present for the wisdom of crowds process to work, Suroweicki did not answer the question as to why crowds can be smarter than an individual. Those reasons were explained by Harvard Felix Frankfurter Professor of Law Cass Sunstein in his book Infotopia: How Many Minds Produce Knowledge. Sunstein, who is a prominent legal scholar and had served as the Administrator of the White House Office of Information and Regulatory Affairs in the Obama administration, explained in his book that the underpinnings of this phenomenon are mathematical and based on a key criteria: in order to be “wise”, at least 51% of a group must be composed of people that have the right answer. That mathematical principle is called “The Condorcet Jury Theorem”. It too has implications for when and how the wisdom of crowds principle works and when it does not as well.
Let’s examine this underlying principle a little more deeply.
The origin of the Condorcet Jury Theorem was in political science, where it was utilized to determine the probability of a given group of individuals coming to a correct decision in a legal case. The theorem states that for any given group, the more people that know (and then can express) the correct answer, the more likely the group will arrive at the right decision. However, for the group to end up with the right decision, a minimum of <50% of the group must have that correct answer; and the more people in the group that have the right answer, the higher the probability of the accuracy of the group. Expressed mathematically, the theorem is as follows:
The assumptions of the simplest version of the theorem are that a group wishes to reach a decision by majority vote. One of the two outcomes of the vote is correct, and each voter has an independent probability p of voting for the correct decision. The theorem asks how many voters we should include in the group. The result depends on whether p is greater than or less than 1/2:
The implication of this theorem though, is that it adds one more condition to Suroweicki’s list of four, and the fifth one is: in order for a group’s answers to be accurate, more than 50% of its members must know/have and provide the correct answer. And of course the corollary is true as well: when the group’s members are mostly wrong the group as whole will be wrong as well.
Next Monday we'll look at when it makes sense to apply the wisdom of crowds method for assessing accuracy to what one finds on the Web.
The origin of the Condorcet Jury Theorem was in political science, where it was utilized to determine the probability of a given group of individuals coming to a correct decision in a legal case. The theorem states that for any given group, the more people that know (and then can express) the correct answer, the more likely the group will arrive at the right decision. However, for the group to end up with the right decision, a minimum of <50% of the group must have that correct answer; and the more people in the group that have the right answer, the higher the probability of the accuracy of the group. Expressed mathematically, the theorem is as follows:
The assumptions of the simplest version of the theorem are that a group wishes to reach a decision by majority vote. One of the two outcomes of the vote is correct, and each voter has an independent probability p of voting for the correct decision. The theorem asks how many voters we should include in the group. The result depends on whether p is greater than or less than 1/2:
- If p is greater than 1/2 (each voter is more likely to vote correctly), then adding more voters increases the probability that the majority decision is correct. In the limit, the probability that the majority votes correctly approaches 1 as the number of voters increases.
- On the other hand, if p is less than 1/2 (each voter is more likely than not to vote incorrectly), then adding more voters makes things worse: the optimal jury consists of a single voter.
The implication of this theorem though, is that it adds one more condition to Suroweicki’s list of four, and the fifth one is: in order for a group’s answers to be accurate, more than 50% of its members must know/have and provide the correct answer. And of course the corollary is true as well: when the group’s members are mostly wrong the group as whole will be wrong as well.
Next Monday we'll look at when it makes sense to apply the wisdom of crowds method for assessing accuracy to what one finds on the Web.
RSS Feed
