Data Analytics vs Confirmation Bias

I've often said that corporate culture is the largest impediment to the effective use of data. The problem is simple. Organizations say that they are "data driven" but in practice, people embrace data that supports their prior conclusions and reject data that doesn't match.

We call this confirmation bias.

This morning, I was reading Peggy Noonan's column in the 10/7/2021 edition of the Wall Street Journal and came across this:

"I’m not a huge respecter of polls (only snapshots, not a measure of greatness or consequence) but when polls put numbers on what you’re sensing you pay attention."

Wow. I'd like to call it a "textbook" example of confirmation bias but I think it's beyond that. Few people are this self-aware regarding their own confirmation bias and, of those who realize it, even fewer will openly admit it.

I'm a fan of Noonan's writing. I applaud her honesty but I'm disappointed by her lack of trust in data.

As a long-term colleague and co-author often says - you have to be willing to let the data surprise you.

Survivorship Bias and Covid-19

One would think that a global pandemic that caused many of us to spend much more time at home would have resulted in great blogging productivity but that didn't happen. I haven't written anything in over a year. Instead during that time I:

• Took over the Chair's role in an academic department that lost 25% of its faculty less than a month before the school year started.
• Turned every class that I teach into either an online course or a hybrid (and got much better at quickly creating and editing videos),
• Published a co-authored paper on interdisciplinary teaching, and
• Moved over 400 miles to take a position with a new employer (go Wildcats!).

Oh well. Life gets busy. But some recent discussion I've had regarding covid-19 inspired me to create another post. Yes, another covid-19 post.

Ever since vaccines became available, there's been some debate on the role of natural immunity but there's been little actual data. For example, here's an excerpt from an October 19 Fast Company article.

-------------------------
One example: August 15, 2021 data showed cases at their peak for the period of time this tool’s data covers. On that day there were:

• Unvaccinated: 736.72 infections per 100,000 people
• Janssen-vaccinated: 171.92 infections per 100,000 people
• Pfizer-vaccinated: 135.64 infections per 100,000 people
• Moderna-vaccinated: 86.28 infections per 100,000 people

-------------------

I'm in the Janssen-vaccinated group, but I also had covid a couple of months before I was able to get the vaccine. What does "171.92 infections per 10,000 people" tell me about my risk of breakthrough infection? It's unlikely that the risk is the same for (Janssen-vaccinated/Recovered) and (Janssen-vaccinated/Never Infected). By ignoring the infected/recovered variable, these numbers aren't very useful.

Finally, in late August, a study came out of Israel claiming that natural immunity is even stronger than vaccine immunity. There is current debate on that study, but for the moment, let's assume that its findings are correct.

So what? Does this mean that you should try to get covid instead of a vaccine?

Statistically speaking - No.

Using this study to promote infection instead of vaccination commits a serious logical fallacy: Survivorship Bias. When you attempt to generalize from a dataset you need to think carefully about the population represented by your data and the population to which you wish to generalize. Survivorship bias occurs when the entities (people, airplanes, etc.) in your data set are systematically different from those that were eliminated from the data.

In the case of natural immunity, the data includes only those who quite literally survived the disease and excludes those killed by it. It should be no surprise that their current immunity is stronger than the immunity of those who died. It's entirely possible that survivors had stronger immune systems in the first place.

Therefore, the Israeli study cannot be used to recommend natural immunity over vaccination for the general population. 

So what can we say about the general population? 

If you have never had covid-19 and you are currently unvaccinated, then you have a choice:

1. Get vaccinated and face the side-effect risks.
2. Take your chances on getting covid and face the disease risks. 

For both of those decisions, the data is out there. Thankfully, the overall hospitalization/death rates of covid-19 are small. Still, the vaccine side-effects risks for most people are even smaller. In some demographic groups, the side-effects risks are much, much smaller than disease risks. 

Based on the data that I've seen, I recommend #1, but I respect your right to look at the same data and make a different decision based on your own medical situation.

 If you have recovered from covid-19 and you are currently unvaccinated, then you have a choice:

1. Get vaccinated and face the side-effect risks.
2. Take your chances on natural immunity.

This is a more difficult decision. The vaccine side-effect risks (#1) are still small. In the absence of data, I suspect that they're even smaller for the infected-recovered than for the never-infected but I think we have to assume that some risk is still there. On the other hand, we aren't sure what your reinfection risk is under #1 or #2. Either way, it's not zero.

Knowing what I know now, I would still choose vaccination. I consider the side-effect risks small and it's likely that the combination of my recovery and the vaccine is giving me even stronger protection now.

However, I see no reason to require vaccination for those who are infected-recovered. Without much stronger evidence to the contrary, they should be treated as if they were vaccinated**. 

Next, we'll have the booster issue. I can't draw any conclusion on boosters because I haven't seen any data the takes into account the difference between Vaccinated/Recovered and Vaccinated/Never-infected. This problem will not go away as long as studies and public policy continue to ignore natural immunity.

Summary: You should not seek out covid-19 in order to get natural immunity, but if you already survived covid-19 (thankfully) then your natural immunity needs to be considered.

==================

**In hindsight, I should have been denied a vaccine in March 2021. Vaccines were in short supply and many people wanted them. Those of us who already had covid should have been pushed to the back of the line. It wouldn't have hurt us to wait until June or July.

Millennials: Yes, this was predicted.

I guess I'm developing a habit of an annual post about millennials. My first was in October 2018 and my second was September 2019. 

In the first post, I talked about my personal experience with different generations of students and linked to an article about millennials. In the second, I didn't say as much and linked to another article.

The overall message was - millennials aren't wildly different than prior generations.

Well here's another article about millennials. Since it's in the Wall Street Journal, it might be behind a paywall. So here's the main point: millennials - who were supposedly going to reject traditional family housing and completely change urban living - are driving the 2020 housing market.

Yes, this was predicted. The millennials are, on average, marrying later and having kids later but they're still marrying, having kids, and buying houses.  

Gaiters - Are they bad or did the news jump the gun (again).

Yes, I suppose that this is another COVID-19 post but I haven't written about COVID since April and March. Also - one could argue that this post isn't really about COVID. It's really about press coverage of emerging research (maybe any research) and COVID just happens to be the context.

You've probably seen the headlines or heard the news that the neck gaiters people have been wearing as face masks might not be working. Six days ago the headline was pretty scary: "Wearing a neck gaiter may be worse than no mask at all, researchers find".

OK, they did say "may" which could imply some uncertainty, but the coverage that I saw was pretty negative on gaiters. In fact, my employer has banned them based on these reports.

Four days ago, the headline was a little less scary: "Some neck gaiters may be worse than not wearing a mask at all, study shows". Now it says both "may" and "some".

Three days ago, the headline shifted again: "The results of this viral mask study found gaiters weren't effective - but it that true?" That's a very different headline and the article includes a quote from one of the researchers (Brian Labus):

“People have really gone overboard with their interpretation of this study. The goal of the study was not actually to evaluate masks” 

What??? They weren't even trying to evaluate masks? You wouldn't know it from the headlines but they were trying to develop a low-cost method that could be used to evaluate masks. 

In fairness, I should point out that all three articles include a link to the actual research report. Unfortunately, when major news outlets report on research, very few readers click through to the actual research (did you click my link?). Instead, people count on the news story to accurately summarize the research. In this case, the news blew it and focused on a peripheral issue.

In my experience, it's not unusual for news reports about research to do a poor job of representing the research. Sometimes it's intentional but often it's just sloppy reporting.

That said, there was a peripheral finding with a small sample size for a particular gaiter. That's far from conclusive but it should be enough to raise concerns and encourage further research on gaiters. I hope that research happens soon and gets better reporting.

Oh, about that sample size I mentioned in the previous paragraph? I could comment on it, but I won't. You should click through to the actual research study and see for yourself.