Foxes, Hedgehogs & Investors
I recently re-read Philip Tetlock and Dan Gardner’s book “Superforecasting”, one of the best practical guides to probabilistic thinking and overcoming cognitive biases I’ve come across. It’s rare to find a book so fun to read about so serious a topic: no less than the US Intelligence Community’s (USIC) efforts to improve its forecasting after the disastrous consequences of its assertion that Saddam Hussein was developing weapons of mass destruction. In an astounding moment of intellectual honesty, the USIC did so using an open tournament in which participants were asked questions like:
“Will the London Gold Market Fixing price of gold (USD per ounce) exceed $1,850 on 30 September, 2011?”;
“On 15 September, 2014, will the Arctic sea ice extent be less than that of 15 September, 2013?”;
“Will a foreign military intervene in Syria before December 1, 2014?”;
“Will either the French or Swiss inquiries find elevated levels of polonium in the remains of Yasser Arafat’s body?”; or
“Will there be an attack carried out by Islamist militants in [Europe] between 21 January and 31 March 2015?”.
Participants were then required to express their forecasts quantitatively as a number between zero and one (i.e. absolutely impossible and absolutely certain), with the highest scores going to those whose forecasts over time were both correct and expressed with confidence (i.e. were closer to either zero or one).
Those are not easy questions! And yet Tetlock and his team at The Good Judgement Project blew everyone out of the water with their accuracy. In particular, a group emerged which consistently scored higher than others - including USIC professionals with access to restricted information - and even improved their performance over time. These were the eponymous “Superforecasters”, the book’s ordinary yet extraordinary anti-heroes. As people, they had above average intelligence but weren’t necessarily geniuses; were highly numerate; were open-minded and not wedded to any views; had a ‘growth mindset’ (“[the belief] that your abilities are largely the product of effort - that you can ‘grow’ to the extent that you are willing to work hard and learn”); and showed ‘grit’ (“passionate perseverance of long-term goals, even in the face of frustration and failure”).
Together, the tournament’s rigour and these personality traits form one of the book’s key arguments: the way to improve in any endeavour is to measure our performance in an accurate and objective way; to cheerfully accept our mistakes; and to doggedly learn from them so as to be better the next time.
The book offers practical advice for long-term value investors but also poses some difficult questions. For a start, there is the importance of probabilistic thinking, and the Superforecaster’s toolkit for how to better assess and update probabilities over time:
Focus on what can reasonably be predicted;
Break problems into more tractable sub-components;
Strike a balance between ‘inside’ and ‘outside’ views;
Synthesise opposing views;
Update regularly but strike a balance between over- and under-reacting;
Be aware of your own biases;
Embrace a world with many shades of grey; and
As per above, keep a note of your thinking ex ante and conduct rigorous post-mortems on both your successes and your failures.
Tetlock and Gardner summarise these well in the book’s Appendix and online, though more in depth case studies would certainly help. What I liked most was the authors’ advice on structuring a problem, especially to pare assumptions down to first principles and then to frame the research process as a series of testable (falsifiable) hypotheses. This is a great way to promote intellectual honesty as well as to identify and target the most important research questions. These hypotheses must then be challenged constantly. As the authors say, “Beliefs are hypotheses to be tested, not treasures to be guarded”.
Drawing on Daniel Kahneman’s framework of System 1 and System 2 thinking, the authors warn of substituting the real underlying question for one our gut finds easier. For example, in the question above on Yasser Arafat, most participants (and pundits) spun their wheels pointing the finger at either Israel or rival factions within the PLA. In contrast, the superforecasters begun by looking up the half life of polonium. Investors often fall into similar traps, substituting questions like, “Is this security under-valued?” with “Do I have a strong gut reaction for or against this company, industry or management team?”; or “Do I really understand this business?” with “Are there lots of smart people talking about it?”. The most classic mistake is to substitute, “Is this business doing well?” with “Is its stock price going up?”.
The authors cite evidence that one simple fix to these knee-jerk reactions is to simply write your forecast or analysis down, wait a couple of weeks and then come back to it later with a cooler head (and more information). This is akin to doing your research before a company’s share price dives, or simply following it for some time before investing to get to know it better. They also suggest inverting the question to yield new perspectives, or tinkering with it by adjusting the time frame or another key variable to stress test your mental models, e.g. what would happen if the competitive environment changed.
The book is unequivocal in advocating the importance of teamwork, the evidence for which came from the outperformance of Superforecasters who worked in teams. The author’s explanation of this is that teams help us to better obtain more information - particularly information we might not have otherwise come across - and that more information ultimately leads us to more accurate decision-making. Teams effectively let us harness ‘the wisdom of the crowd’. But teams have their own dynamic and not all work well. I’m sure many of us have worked in an environment where it’s not OK to challenge the boss, for example. Groupthink is another risk. The solution is to create an atmosphere in which open and respectful debate is encouraged. A shared purpose can also act as a glue, while giving can engender surprising amounts of reciprocity. Personally, I am very grateful to my friends at ValueAsia for providing me with exactly this kind of sounding board. Because each of us invests independently, we’re also less likely to fall into the behavioural traps created when pay cheques are involved.
Where investing is different from the questions posed to the Superforecasters is the added dimensions of price, portfolio construction and risk. You and I can both be right and confident in our forecasts of Apple’s business but if you bought more stock at a lower price than I did, you will make more money and have a better result. Similarly, when the price is low, there is less risk. For example, if you buy a secondhand car for $1, it doesn’t really matter if your forecast of its condition was wrong and it later turns out to be a lemon.^ Regarding risk, Tetlock and Gardner acknowledge a criticism from investor and philosopher Nassim Taleb that the future is not Normally distributed, leading extreme events to occur more frequently than we might otherwise assume. I think this is a trenchant reminder of risk and the importance of staying humble. Similarly, questions in the tournament were independent of each other. An investment portfolio on the other hand might have many correlated bets, warranting a lower overall exposure even with a high level of confidence.
Successful long term investments tend not to hinge on a binary outcome like the questions posed in the book either. In reality, doing business is complex; markets are complex; and the world is complex. This is another of Taleb’s criticisms of the book: “what matters can’t be forecast and what can be forecast doesn’t matter”. I agree with this and always think back to when Buffett took control of a New England textile company in the 1960s. No one - not even him! - would have guessed that he would turn it into an insurance powerhouse and perpetual compounding machine. As discussed in an earlier post, I deal with this non-linearity in two ways. First, I prefer to invest in businesses with economic goodwill (a ‘moat’) as they tend to be more predictable. Second, I frame my work as the search for investments which on balance are likely to be worth significantly more than their current price, while unlikely to be worth significantly less. Compared to the traditional approach of estimating a single, fixed target price, thinking probabilistically and in ranges like this seems to me a more flexible and pragmatic approach which embraces the many ways to win (or lose), and the many futures which can possibly exist.*
However, Tetlock and Gardner made me realise that it can be hard for me to get good feedback because I generally don’t make explicit forecasts ex ante. And after all, as the authors say, “you can’t have clear feedback unless your forecasts are unambiguous and scorable.” Their suggestion is to therefore break open-ended, complex questions down into more tractable sub-questions which can be better defined and evaluated (so-called “Bayesian clusters”). For example, an important but vague question like, “Will there be another Korean War?”, could be broken down into a series of smaller, more precise questions regarding the likelihood and timing of missile launches, nuclear tests, cyber attacks or artillery shellings. The more yeses, the more likely the answer to the open-ended question will also be yes.
This technique could easily be used to more precisely define and assess a company’s individual value drivers as part of forming an opinion of its value as a whole. The authors write, “Another way to think of [this] is to imagine a painter using the technique called pointillism. It consists of dabbing tiny dots on the canvas, nothing more. Each dot alone adds little. But as the dots collect, patterns emerge. With enough dots, an artist can produce anything from a vivid portrait to a sweeping landscape.” I already do this implicitly so it shouldn’t be hard to do so explicitly, perhaps by expanding my pre-mortems into checklists for success and failure. I also recently started an investment journal and hope it will help me better avoid hindsight bias too.
One thing the book admits it can’t answer is how to get better feedback on long term forecasts. This is very, very relevant for long-term value investors; after all, how many times in our professional careers will we be there to get good feedback on a five- or ten-year long investment? More troubling, Tetlock’s earlier work shows the value of forecasts rapidly diminish as their time horizon extends beyond six months, likely due to the possiblity of ‘black swan’ events increasing over time. So are we deluding ourselves to even look that far out? I can’t answer conclusively but have a thought which leads me to think ‘maybe not’. %
The book references Isaiah Berlin’s metaphor of the Hedgehog and the Fox: “the fox knows many things but the hedgehog knows one big thing”. Hedgehogs have fixed beliefs and tend to confidently hold them regardless of contradictory evidence. They are the kind of people you see making bold declarations on CNN or CNBC. Foxes are not wedded to any idea, seek a wide range of views and always parse their thinking with arguments for and against. Because they say things like, “On the one hand… on the other hand… and on the third hand…”, they do not make for good TV. But the kicker is that Tetlock found that in general, foxes are far, far better forecasters than hedgehogs.
I think long-term investors are foxes who seek hedgehog-like bets. I think we do this because we need to anchor ourselves through the turbulence and complexity of the market. The key difference though is that unlike the Superforecasters, investors can choose which questions they answer. And we get to study history, watch consumer behaviour, analyse industry structure, interview managers etc. - all to gather evidence before we do. Jeff Bezos said it best when he recommended looking for what’s not going to change because that is much easier to predict. Buffett has said the same, like “the internet won’t change the way people chew gum”. More recently, Josh Tarasoff has explained that there is certainty even in change. For example, it’s all but guaranteed that in the next ten years, people will do more shopping online, not less. However, once we’ve used fox-like techniques to find a hedgehog-like idea that passes muster, we don’t just sit there sucking our thumbs; we aggressively try to tear it down.$
Tetlock and Gardner close the book with the admonishment to “try, fail, analyse, adjust, try again”. The same should be true for investing. I can’t recommend “Superforecasting” enough.
———————————————
^On the flipside, it can sometimes make sense to bet against your forecast when the expected return is greater the other way. There is a wonderful anecdote describing this in Nassim Taleb’s book “Fooled By Randomness”. At his investment bank’s morning meeting, Taleb said he thought the market would go up but that he was short (i.e. positioned for it to go down). The traders all nodded their heads in understanding while the salesmen harangued him for seemingly contradicting himself. Clearly the salesmen were not thinking about expected returns!
*You might also argue that I’m vulnerable to “thesis creep” but I think the evidence suggests good businesses benefit from business momentum and virtuous feedback loops. A more folksy way to describe this is that “good things happen to good people”. Stephen Wood wrote a good piece on this recently discussing his ‘wandering’ investment in Tripadvisor.
%A sceptic might say that long-term value investing itself is a big, hedgehog like idea begging to be tested like every other hypothesis. I would certainly agree given the breakaway, long-term success of investors like Jim Simons and Edward Thorpe. I would also be the first to acknowledge the weakness of Li Lu’s ‘Civilisation 3.0’ framework is that it can’t be falsified.
$ I often find myself sounding bearish about my own stocks; this is why!