Story time

02.22.2017 By , Kara Lilly

Our investment strategy is easy to remember: We look for good business models run by strong managers at reasonable valuations.

But how does one determine what makes a good business model or strong managers?

One intuitive approach is to study historically successful businesses and managers to try to tease out common themes. Books line the shelves enabling us to do just that. The Toyota Way, Good to Great, The Amazon Way, Creativity Inc., etc. are examples of popular titles which devote themselves to picking apart the histories of successful companies and managers.

While studying the stories of historical success is obviously an important part of learning, the approach also brings with it investing pitfalls to be aware of. One such pitfall is narrative bias.

Narrative bias involves deriving a general principle from the findings of a very specific/narrow instance of an outcome distribution. What does that mean?

Consider an extreme example. Assume we want to learn about the most reliable ways to earn a large amount of money in a short period time. After a quick study of people who achieved a rapid increase in net worth, we would find a high concentration of lottery-ticket winners amongst them. Thus, we might conclude erroneously that buying a lottery ticket is one of the best strategies for quickly accumulating wealth.

Now let’s consider a more real-life example of how not appreciating this bias can lead to problematic conclusions.

It is widely accepted that smaller schools tend to lead to better outcomes for students. This claim is, at least partially, based on studies which measured school performance by the percentage of kids scoring above a certain threshold. The studies found that the highest performing schools tended to be disproportionally represented—i.e., statistically significant—by smaller-sized schools. Assuming the studies’ were conducted properly, does it follow that smaller schools should, in aggregate, lead to better performance per child? Try to answer that question before continuing.

Turns out, not really. The result was the product of statistics. While smaller schools were overrepresented as the best performing schools, they were also overrepresented as the worst performing schools. Why? Because smaller schools have fewer students. Thus, their performance is more variable and the distribution is wider—and so they tend to sit more in the “tails” of the distribution of performance in schools.

So while there may be hard scientific evidence that smaller schools are statistically more likely than larger schools to outperform, it still doesn’t answer the key question at the root of the experiment: will smaller or larger schools produce better student performance?

What the small school example highlights is how you can conduct a sound statistical study and still wind up with an erroneous conclusion if you don’t understand the context properly, or if you don’t set up the experiment correctly. Just because you are couching your conclusions in the robes of science and statistics doesn’t mean the conclusion follows. Narrative bias focuses our attention onto one part of the outcome distribution. This a very useful lesson for the investor who is trying to understand something as complicated as what makes a management team “good.”

One of the ways we overcome this bias in our own research process is to emphasize and encourage the study of failed investments. You need to look at the full spectrum of outcomes to determine what lessons really make sense to apply.

In addition, we run statistical deep dives into the income distributions associated with the steps in our research process. In other words, we routinely look into what we are doing and if it is actually adding value. We do all sorts of things in our research team that we imagine are helpful—but just because you assume something adds value, doesn’t mean it does. Only by applying statistics, and applying them properly, can we avoid the narrative bias pitfalls which abound in the complex and murky world of performance attribution.

Our brains bias us towards stories. Stories are simple to consume and a powerful way to convey and learn a concept. But we must remember the potential pitfalls that story-time can bring in any research process. The careful, systematic (and boring) use of data and statistics is paramount for us as investors to manage our natural narrative biases.

You Might Also Like

1 Comment

  • Reply
    Donna dooher
    02.23.2017 at 7:40 am

    I’m not sure how I stumbled across your newsletter but I signed up. Investment is not a big part of my wheelhouse – yet, but I must say I enjoy your clever approach.

    Regards,
    Donna

  • Leave a Reply