[This is the third post in a 5-part series excerpted from my forthcoming book: Find it Fast: Extracting Expert Infor-mation from Social Networks, Big Data, Tweets, and More, 6th. Ed., (CyberAge Books, Autumn 2015) and the second of three on the Wisdom of Crowds, ]
How We Rely on The Wisdom Of Crowds
Last week we defined the wisdom of crowds phenomenon, and this week I will discuss how it should--and should not-- be used to determine if and when relying on the answers or decisions of a crowds is valid, it’s important to look at the various ways group views are now being collected on the social Web and apply the principle based on these varying conditions. In this way, we can understand the WOC principle not as something always and inherently “right” or “wrong” but more or less applicable and useful, on a spectrum, depending on the circumstance and type of answers being sought.
Here are examples of what I mean:
WOC conditions: Fully Present
It is rare that all of Suroweickis four conditions--as well as Sunstein’s fifth principle that more than 50% of the target group has the correct answer —are all present to help ensure that the group will in fact definitively be “smarter” than an individual expert. One circumstance where all factors could be present though is when a person or organization consciously and carefully creates a crowdsourced project--either on the public web or privately--to generate ideas or solutions and works ahead of time to ensure that each of Suroweicki’s four conditions are established ahead of time to tap into the knowledge of the crowd. A dilemma arises here though: how to ensure ahead of time that the requirement of the Condorcet Jury Theorem of <50% accuracy is represented in the group?
Companies sometimes do this kind of internal or external crowdsourcing via setting up brainstorming projects to spur innovation. For example, the Swedish appliance company Electrolux wanted to come up with innovative solutions to a variety of problems and deployed its internal digital network to hold a crowdsourced “innovation jam” sourcing 3,500 new ideas and 10,000 comments. In an interview, Ralf Larson, director of Electrolux’s online engagement told me that the entire company was invited to join, and that, “we had knowledgeable and skilled people to moderate and facilitate the event. Employees voted to select the top three ideas and several went immediately into the product development pipeline."
WOC conditions: Mostly Present
Google’s wisdom of crowds based PageRank algorithm for ranking Web pages works as well as it does because the voting—in this case expressed by linking--is done a) independently, b) by a diverse group of people, c) in a decentralized manner and d) integrates a method for aggregating the “votes” (Google’s own software). This is not to say that there are no attempts to manipulate the system via the creation of “link farms” to boost rankings or via other methods; but Google regularly monitors these and adjusts its system to try to damp down and prevent these gaming attempts to corrupt its process.
WOC conditions: Only Partially Present
Comments on Blogs and News Sites. These sites offer a platform where anyone an voice their views and their comments and opinions are very likely influenced by the comments made before them and so do not represent a WOC activity. Similarly, promoting (up and down voting) others’ comments or opinions on Facebook, YouTube, or news sites like Digg can be interesting for observing what is getting a group’s attention, or to see what news or sites are hot at the moment, but there is no real independence of action.
What about Wikipedia? Does Wikipedia qualify as a valid wisdom of crowds site? From a philosophical standpoint, Wikipedia certainly was created on the notion that the group knows the most, and integrates the fundamental Web 2.0 tenants that knowledge is best derived from many voices, emerges from an ongoing conversation, is always evolving and is not a static pronouncement from the head of a single expert. It also fulfills two of Suroweicki’s conditions: diversity and decentralization. However, there is not true independence of action (everyone sees everyone else’s edits); and there are a range of “rules” that contributors must adhere to (e.g. user edits need to reference published secondary sources; there must be neutrality of tone, etc.) so Wikipedia is kind of a special case.
WOC Conditions Mostly Not Present or Not Applicable
Web based or in person live discussion groups, meetings and forums. In these cases, a group of people (in person or online) share their thoughts together as a group, and may agree or disagree on an issue and sometimes try to come to a decision. This method for arriving at a decision or agreement, can either be a good one (say, if the facilitator knows how to bring groups to consensus) or possibly a bad one (if people are afraid to express their views due to fear, pressure or other negative forces in the group) but has nothing to do with the wisdom of crowds phenomenon. There is no independence of action (everyone knows what everyone else thinks or believes) nor is there decentralization of voting.
At its worst, group discussions can even turn into groupthink which can literally result in disaster. In fact the tragic case of the pressures that pushed NASA engineers to launch the Challenger on that cold Florida morning has become a standard case study used in schools to teach the perils of groupthink.
How We Rely on The Wisdom Of Crowds
Last week we defined the wisdom of crowds phenomenon, and this week I will discuss how it should--and should not-- be used to determine if and when relying on the answers or decisions of a crowds is valid, it’s important to look at the various ways group views are now being collected on the social Web and apply the principle based on these varying conditions. In this way, we can understand the WOC principle not as something always and inherently “right” or “wrong” but more or less applicable and useful, on a spectrum, depending on the circumstance and type of answers being sought.
Here are examples of what I mean:
WOC conditions: Fully Present
It is rare that all of Suroweickis four conditions--as well as Sunstein’s fifth principle that more than 50% of the target group has the correct answer —are all present to help ensure that the group will in fact definitively be “smarter” than an individual expert. One circumstance where all factors could be present though is when a person or organization consciously and carefully creates a crowdsourced project--either on the public web or privately--to generate ideas or solutions and works ahead of time to ensure that each of Suroweicki’s four conditions are established ahead of time to tap into the knowledge of the crowd. A dilemma arises here though: how to ensure ahead of time that the requirement of the Condorcet Jury Theorem of <50% accuracy is represented in the group?
Companies sometimes do this kind of internal or external crowdsourcing via setting up brainstorming projects to spur innovation. For example, the Swedish appliance company Electrolux wanted to come up with innovative solutions to a variety of problems and deployed its internal digital network to hold a crowdsourced “innovation jam” sourcing 3,500 new ideas and 10,000 comments. In an interview, Ralf Larson, director of Electrolux’s online engagement told me that the entire company was invited to join, and that, “we had knowledgeable and skilled people to moderate and facilitate the event. Employees voted to select the top three ideas and several went immediately into the product development pipeline."
WOC conditions: Mostly Present
Google’s wisdom of crowds based PageRank algorithm for ranking Web pages works as well as it does because the voting—in this case expressed by linking--is done a) independently, b) by a diverse group of people, c) in a decentralized manner and d) integrates a method for aggregating the “votes” (Google’s own software). This is not to say that there are no attempts to manipulate the system via the creation of “link farms” to boost rankings or via other methods; but Google regularly monitors these and adjusts its system to try to damp down and prevent these gaming attempts to corrupt its process.
WOC conditions: Only Partially Present
Comments on Blogs and News Sites. These sites offer a platform where anyone an voice their views and their comments and opinions are very likely influenced by the comments made before them and so do not represent a WOC activity. Similarly, promoting (up and down voting) others’ comments or opinions on Facebook, YouTube, or news sites like Digg can be interesting for observing what is getting a group’s attention, or to see what news or sites are hot at the moment, but there is no real independence of action.
What about Wikipedia? Does Wikipedia qualify as a valid wisdom of crowds site? From a philosophical standpoint, Wikipedia certainly was created on the notion that the group knows the most, and integrates the fundamental Web 2.0 tenants that knowledge is best derived from many voices, emerges from an ongoing conversation, is always evolving and is not a static pronouncement from the head of a single expert. It also fulfills two of Suroweicki’s conditions: diversity and decentralization. However, there is not true independence of action (everyone sees everyone else’s edits); and there are a range of “rules” that contributors must adhere to (e.g. user edits need to reference published secondary sources; there must be neutrality of tone, etc.) so Wikipedia is kind of a special case.
WOC Conditions Mostly Not Present or Not Applicable
Web based or in person live discussion groups, meetings and forums. In these cases, a group of people (in person or online) share their thoughts together as a group, and may agree or disagree on an issue and sometimes try to come to a decision. This method for arriving at a decision or agreement, can either be a good one (say, if the facilitator knows how to bring groups to consensus) or possibly a bad one (if people are afraid to express their views due to fear, pressure or other negative forces in the group) but has nothing to do with the wisdom of crowds phenomenon. There is no independence of action (everyone knows what everyone else thinks or believes) nor is there decentralization of voting.
At its worst, group discussions can even turn into groupthink which can literally result in disaster. In fact the tragic case of the pressures that pushed NASA engineers to launch the Challenger on that cold Florida morning has become a standard case study used in schools to teach the perils of groupthink.
RSS Feed
