Notes on The Wisdom of Crowds
Here are my notes on The Wisdom of Crowds by James Surowiecki
Overview
Under the right circumstances, groups are smarter than the smartest people in them, even if the group doesn’t contain expert members.
Experts are as likely to disagree as agree. Experts’ individual consistency is also 0.5. Experts overestimate the likelihood that they are correct – little correlation between self-assessment and performance. Therefore, however well informed and sophisticated and expert is, his advice should be pooled with that of others and the larger the group the better. So we should stop “chasing the expert” and ask the crowd. We continue to chase the expert because we assume average means least common denominator (Larrick and Soll) and we are fooled by randomness (Taleb).
Criteria for good collective decision making
The group must be big enough and diverse enough, the members must be forming opinions independently, and the group must be decentralized.
Diversity
To solve cognition problems you must 1) uncover alternatives and 2) decide among them. Diversity is needed for both 1 and 2. Diversity adds perspectives and weakens destructive characteristics of group decision making. A successful system recognizes losers and kills them quickly. Homogenous groups spend too much time exploiting and not enough time exploring (James March). Homogenous groups are susceptible to groupthink, willing to rationalize away counterarguments and often convinced that dissent is bad. Diversity is more important than individual intelligence but members must be somewhat informed. There has to be some information – can’t have a completely ignorant group.
Independence
Independence means members can’t be dependent upon one another for information and can’t be subject to influence from one another. Independence keeps mistakes from being correlated. Members can be biased/irrational without making the group dumber. The more influence members exert on each other, the more personal contact, the dumber the group will be. This is difficult to enforce because
- members want to learn from one another
- members are affected by environment/neighborhood/hierarchical position
- groups become more influential as they get bigger
- “It’s better for reputation to fail conventionally than to succeed unconventionally” Keynes
- Information cascades can occur when members make decisions in sequence rather than simultaneously. That is, the first deciders influence the subsequent even if they are wrong. If the subsequent start following the crowd then the cascade stops informing. (From The Tipping Point, cascades move via social ties – mavens, connectors, salesmen)
There also must be intelligent imitation not slavish.
- Intelligent: people stop imitating and learn for themselves when the benefits of doing so become high enough
- Slavish: people just keep imitating no matter what
To get intelligent imitation
- There needs to be initially a wide array of options and information
- Some members must value their own judgment ahead of group’s – overconfident people who go with their gut or systematically test and adopt
Corporations should incentivize employees to uncover and act on private information. (Blasi and Kruse, High Performance Work Systems).
Decentralization
Decentralization- specialization plus coordination. The best collective decisions are the product of disagreement and contest, not consensus or compromise. For a decentralized system to be intelligent there must be a means of aggregating all members’ inputs such as market prices or centralized decision makers, e.g., Linux. The risk of decentralization is that information won’t make it through the system to where it is most valuable. There must be a balance that allows individual knowledge to be specific and local (tacit) but also makes it globally useful.
People focus better on a decision when there are financial rewards attached to it so decision markets are often successful.
Examples
Problems can be classified as
- Cognition: when there are definitive solutions (Who will win the Super Bowl?) or there is a best possible answer (Where’s the best place to site this building?)
- Coordination: when members’ behavior must be coordinated (driving or finding a party)
- Cooperation: getting members to work together when self interest would dictate that they should not (taxes, dealing with pollution)
For coordination problems, independence is pointless since what one member is willing to do depends on what that member thinks other members are going to do. The El Farol problem shows that even in this case collective judgment can be good though it can result in many members not being satisfied. With traffic jams the diversity of drivers makes coordination difficult. Solution is more control: automatic highways with platoons of synchronized cars or driver assistance to keep cars evenly spaced.
Cultural conventions allow groups to organize without conflict, e.g., “first come first served,” queues – there is wisdom in conventions but many conventions can be very stupid. For example B movies and old movies cost the same as new. This convention is uncoordinated with moviegoers.
To solve cooperation problems, members need to adopt a broader definition of self-interest than maximizing short-term profits and they need to trust other members. There also needs to be a mechanism for preventing free riders since many people are conditional consenters – only cooperating because they believe that people who don’t will be punished. People exhibit strong reciprocity: willingness to punish bad behavior even when they get no material benefit. The evolution of capitalism has been toward more trust and transparency because the benefits of trust are immense.
· Science:
o People still prefer to work in proximity to colleagues but researchers who spend more time collaborating internationally are more productive.
o Scientists want recognition more than cash. We trust that allowing scientists to pursue self-interest yields better results than command and control.
o The blend of collaboration and competition works because of open access to information.
o The flaw is that most scientific work never gets noticed because famous authors get read more
Rules for Small Groups
Small groups can be good because they make people work harder and think smarter. Non-polarized small groups make better decisions than individuals. But small groups face many problems so there need to be rules for good decision making
o Discussions must have structure (ask each member for input) but not too much (one leader doing all the asking)
o Decision making must not begin with a conclusion. This makes it unlikely for new info to be incorporated. Don’t spend all the time talking about what everyone knows/agrees on
o Devils advocates must be encouraged
o Groups polarize through discussion (counter to common wisdom) because members try to maintain their place in the idea spectrum relative to the entire group. So if entire spectrum shifts right, member must shift right just to stay in the middle. To avoid polarization make sure the group has equal number of people with strongly opposed views.
o The order of speakers matters – earlier comments are more influential. So don’t choose earliest speakers on basis of status since that may not equate to more knowledgeable. The same applies to group members that talk the most.