Why you should (sometimes) reward failure and punish success
The case for prioritising process over outcome
Imagine giving $1,000 to each of two investors. One earns 10%, one loses 10%. Who is the better investor?
That’s obviously a trick question; you don’t have enough information to judge. Maybe the first bought a lottery ticket and got lucky, despite it being a statistically terrible bet. Maybe the second has an outstanding long-term track record and was unlucky this time. You’d need to know more.
We face these types of situations every day as investors, managers, and even in our personal lives: we have a choice between two or more options with uncertain outcomes and limited information about the distribution of outcomes for each.
Here’s a typical business-world example. A CEO asks two senior leaders to each launch a new product. A year later, the first’s product has outperformed significantly. Should she get a promotion? Again, you don’t know enough.
What we’re really trying to answer with these questions is how we should distinguish between skill and luck when there’s a significant element of chance; and when organisations face decisions under uncertainty, how they should create incentives and cultures that tend to produce better decisions and therefore better outcomes over time.
One of the biggest differences I’ve seen between great and average companies is this: great companies teach and reward reliable processes and frameworks that predictably lead to good outcomes rather than blindly rewarding good outcomes that may have resulted from poor decisions.
One of Howard Marks’s legendary memos captures the idea well:
The first thing I remember learning at Wharton in 1963 was that the correctness of a decision can’t be judged from the outcome. Because of the randomness at work in the world and the unpredictability of the future, lots of bad decisions lead to good results, and lots of good decisions end in failure.
The professional poker player Annie Duke says something similar:
What makes a decision great is not that it has a great outcome. A great decision is the result of a good process.
And from investor Michael Maubossin:
Don’t confuse a good outcome with a good process. In probabilistic domains, the only way to win consistently is to emphasize process over outcome.
Amazon has done more than any operating company I’ve ever encountered to not only codify a rich set of processes and frameworks for decision making, but also to create a culture where leaders are judged less on the outcomes of their decisions and more on having followed Amazon’s decision-making processes and frameworks with intelligence and integrity.
Can you think of another company where, when a major new business or product failed, the leader would be celebrated and even promoted—because she accepted the challenge and pursued it with dedication and intelligence? I could give you a dozen such examples from my decade at Amazon.
To be clear, not all failures were celebrated at Amazon. Jeff Bezos made a clear decision between “good” and “bad” failures. A good failure was one where the team worked hard and made smart decisions on the basis of the information available to them at that point, and where the failure therefore taught us something new and important that we didn’t know—perhaps that there wasn’t customer demand for the product, or that there wasn’t enough margin in the value chain, or that the technology was too early. In contrast, bad failures (e.g., decisions that we could have known were poor in advance based on available information) were wasteful: due to poor execution, we had neither proven nor disproven the hypothesis. We’d have to try again with a better team to find out.
In fact, there are two distinct types of errors that companies can make when they fail to distinguish ex-ante decision-making processes (“ex ante” = before the fact; based on information and expectations at the time of decision) and ex-post outcomes (“ex post” = after the fact; judged by realized outcomes once uncertainty is resolved).
Punishing “good failures”: situations where a good process was followed but a bad outcome ensued due to bad luck
Rewarding “bad successes”: situations with a good outcome despite poor process
Amazon is rare in rewarding good failures. Punishing good failures has the pernicious outcome of dissuading leaders in an organisation from taking on risky projects, and Amazon’s culture of rewarding good failures was explicitly designed to encourage “intelligent risk-taking.”
But I think even fewer companies recognise the danger of rewarding bad successes, which is perhaps even more dangerous than punishing good failures. When you reward bad successes, you create perverse incentives that encourage a dangerous degree of risk, particularly when the individual reward (to the leader) for success is high relative to the cost of a failure to that individual.
This danger is persistent in financial markets, most notably in situations where investors are (a) gambling with other people’s money rather than their own, and (b) where the rewards for success are large. Such a creates extreme principal-agent risk because of the asymmetry between the agent’s personal upside (a multi-million dollar bonus for the trader who hits a homerun) and downside (perhaps taking on some hard-to-measure incremental risk with each bet that subtly increases the chance of a future blowup).
Jeff was particularly disdainful of businesses that he referred to as “picking up nickels in front of a steamroller”—i.e., situations where you make a small amount of money every day when things are going well (nickels), but risk losing everything when the inevitable crash comes (steamroller).
These aren’t just hypothetical risks. Behind most major financial blowups lies exactly this error of rewarding bad successes. Take AIG in the early 2000s: one division wrote hundreds of billions in credit default swaps on mortgage securities, earning steady fees and big bonuses. It looked safe until the 2007–08 crisis, when AIG needed a $180 billion bailout. Executives kept the (large) nickels; shareholders and taxpayers got run over by the steamroller.
Daniel Kahneman recommends that organisations create cultural bulwarks against rewarding bad successes; for example, that companies get used to saying something like, “Let’s not fall for the outcome bias. This was a stupid decision even though it worked out well.” He furthermore diagnoses the origins of this outcome bias (“outcome bias” = either of the two error types described above):
Hindsight bias has pernicious effects on the evaluations of decision makers. It leads observers to assess the quality of a decision not by whether the process was sound but by whether its outcome was good or bad [= outcome bias].
Amazon’s solution was to codify a set of processes and frameworks that leaders were expected to follow in reaching decisions; but equally importantly, to create a culture and a set of career incentives that rewarded such behaviors rather than narrowly focusing on the outcome.
I have seen some outstanding investors who combined this with Kahneman’s recommendation: for example, establishing post-investment reviews a year or two after the investment had been made, where assessments like, “We made money but we got lucky—this was a bad bet” as well as, “We lost money but it was a smart bet—we’d make that bet every time” are common.
The best organizations don’t just celebrate good outcomes. They build cultures and processes that reward good decisions—even when outcomes are bad—and guard against the greater danger of rewarding bad successes.
If you’d like to learn more, here are some recommended sources, going back to early thinkers in decision theory such as Howard Raiffa and Roland Howard who were writing on these topics in the 1960s, as well as most recent work.
Duke, Annie. Thinking in Bets: Making Smarter Decisions When You Don’t Have All the Facts. Portfolio/Penguin, 2018.
Howard, Ronald A. “Decision Analysis: Applied Decision Theory.” Proceedings of the Fourth International Conference on Operational Research, 1966, pp. 55–71.
Kahneman, Daniel. Thinking, Fast and Slow. Farrar, Straus and Giroux, 2011.
Marks, Howard. “Getting Lucky.” Oaktree Capital Management Memo, 16 Jan. 2014.
Mauboussin, Michael J. The Success Equation: Untangling Skill and Luck in Business, Sports, and Investing. Harvard Business Review Press, 2012.
Raiffa, Howard. Decision Analysis: Introductory Lectures on Choices under Uncertainty. Addison-Wesley, 1968.
Tetlock, Philip E. Expert Political Judgment: How Good Is It? How Can We Know? Princeton UP, 2005.
Tetlock, Philip E., and Dan Gardner. Superforecasting: The Art and Science of Prediction. Crown Publishers, 2015.
Great list. I would add the focus on inputs versus outputs. In my Retail days we were constantly told that focusing on inputs (information, selection, availability, price competitiveness and delivery speed) was better than obsessing on ouputs (sales and profit numbers): get the former wrong and the latter would eventually tank. As a result, "bad successes" were indeed punished - if your input metrics were poor, you got a lot of heat regardless on how well you were doing on outputs.
Chris, thank you for putting an extensive reading list. I am researching the topic and it is great to have more sources than just Kahneman and Dukes.
Would love to learn, what, in your view, were the good Amazon mechanisms on decision-making, beyond Right a Lot and one-way/two-day doors.