Unintended-Consequences

In 2009, Air France Flight 447 was cruising at 35,000 feet over the Atlantic Ocean. The Airbus A330 was one of the most automated aircraft ever built. The pilots barely needed to fly it-automation handled almost everything. Then the airspeed sensors iced over. The autopilot disengaged. Control handed to the pilots. And in the next four minutes, three highly trained pilots flew a perfectly functional aircraft into the ocean, killing all 228 people aboard. ...

In 2006, Netflix announced a challenge: improve our recommendation algorithm by 10%, win $1 million. The Netflix Prize became one of the most famous machine learning competitions ever. Thousands of teams from around the world competed for three years. In 2009, team “BellKor’s Pragmatic Chaos” won. They’d built an algorithm that was 10.06% better than Netflix’s existing system. Netflix awarded the $1 million prize. The press celebrated the triumph of data science. ...

In 1847, Dr. Ignaz Semmelweis made a horrifying discovery at Vienna General Hospital: doctors were killing their patients. The maternity ward had a death rate of 10-35% from “childbed fever.” But here’s what was strange-the ward staffed by midwives had a death rate of only 4%. Mothers were literally begging not to be admitted to the doctors’ ward. Some women chose to give birth in the street rather than risk dying in the hospital. ...

In April 1967, a high school history teacher in Palo Alto, California, faced a difficult question from his students: “How could the German people claim they didn’t know about the Holocaust?” Ron Jones didn’t have a good answer. So he decided to show them. What started as a simple classroom demonstration became a terrifying social experiment. In just five days, Jones created a fascist movement so powerful that students were willing to betray their friends, enforce strict rules, and commit acts of violence-all in the name of the group. ...

In August 1971, Stanford psychologist Philip Zimbardo posted an ad in the newspaper: “Male college students needed for psychological study of prison life. $15 per day for 1-2 weeks.” Twenty-four mentally healthy, middle-class college students responded. They were screened, tested, and deemed normal, stable individuals. Zimbardo randomly assigned them to two groups: guards and prisoners. The experiment was supposed to last two weeks. It lasted six days. What Happened Zimbardo converted the basement of Stanford’s psychology building into a mock prison. The “guards” received uniforms, wooden batons, and mirrored sunglasses to prevent eye contact. The “prisoners” were given smocks, assigned numbers, and had their heads covered with nylon stockings. ...

It’s a hot summer afternoon in Coleman, Texas. A family is relaxing on the porch, playing dominoes and enjoying the fan. The father-in-law says, “Let’s drive to Abilene for dinner.” Nobody really wants to go. It’s 53 miles away in 104°F heat, in a car without air conditioning. But nobody speaks up. The wife says, “Sounds good to me.” The husband, not wanting to disappoint, says, “Sure, I’m in.” The mother-in-law agrees. ...

In the 1970s, the U.S. government mandated new safety features in cars: seatbelts, airbags, reinforced frames, and improved braking systems. The goal was simple: reduce traffic fatalities. Economist Sam Peltzman studied what actually happened. His findings were shocking: while driver deaths stayed roughly the same, pedestrian and cyclist deaths increased. Why? Because drivers felt safer-so they drove more recklessly. The Paradox The safer people feel, the more risks they take. ...

In 2003, photographer Kenneth Adelman was documenting coastal erosion in California. He took 12,000 aerial photographs of the coastline for the California Coastal Records Project. One of those photos happened to capture Barbra Streisand’s Malibu mansion. The photo had been downloaded exactly six times. Two of those downloads were by Streisand’s lawyers. Then Streisand sued Adelman for $50 million, demanding the photo be removed from the public website. The Backfire The lawsuit made headlines. Suddenly, everyone wanted to see the photo that Barbra Streisand was trying to hide. ...

In the early 1900s, British colonial India faced a venomous problem: too many cobras slithering through the streets of Delhi. The British government, determined to reduce the cobra population, came up with what seemed like a brilliant solution-offer a bounty for every dead cobra brought in. Initially, the program worked. People killed cobras and collected their rewards. The cobra population appeared to decline. Success! Or so they thought. The Twist Enterprising locals quickly realized they could breed cobras specifically to kill them and collect the bounty. Why hunt dangerous snakes in the wild when you could farm them at home? ...

In 1961, Yale psychologist Stanley Milgram placed an ad in a New Haven newspaper: “We will pay you $4.00 for one hour of your time.” Participants arrived at Yale’s psychology lab, believing they were taking part in a study about memory and learning. They were told they would be the “teacher.” Another participant (actually an actor) would be the “learner.” The teacher’s job: deliver electric shocks to the learner every time they answered a question incorrectly. ...