Small Data (really small) and Expectation Management

Years ago, my wife and I went to see the movie Romancing the Stone. We were visiting a small town and it was the only movie playing. It was new in theaters and we didn't know anything about it.

We loved it. We told lots of people how good it was.

We loved it so much that we went to see it again about a month later. It was still good, but it wasn't great. We realized that we had zero expectations the first time - we were just looking for something to do - and high expectations the second time.

Years later, our friends were raving about My Big Fat Greek Wedding. They insisted that we see it. Really insisted. We were told that we needed to see it.

We finally went and it was a disappointment. It wasn't a bad movie, but no movie could live up the hyped reviews we heard.

Over the years we've referred back to those movies when we find ourselves reacting differently than expected. We recently went to a restaurant that someone close to us insisted that we try. It was disappointing. Then one of us said "I guess this was a Big Fat Greek Wedding instead of Romancing The Stone". It was actually a nice restaurant but it couldn't possibly live up to the expectations we were given

In related news, I just finished reading The Undoing Project by Michael Lewis. I learned some of this material in graduate school but, as usual, Lewis does a great job telling the story. The discoveries of Kahneman and Tversky explain our experience with the movies and the restaurant.

Small data, such as word-of-mouth reviews from a few friends, are poor statistical samples but people still give it significant weight in forming judgments. With social media, small data can get repeated and amplified so that it looks like much larger data and, again, people will give it significant weight in forming judgments.

We all want good reviews for our endeavors, but we should also want accurate reviews. What if I do good work, but my good work merely meets your expectations (or even falls slightly short). I'd rather be judged against an accurate expectation than an inflated one. However, in the world of small data where people aren't completely rational, I'm not sure how to make that happen.