How Olympic Judging Can Teach Us How to Examine Data

July 11th, 2024

At the Alliance, we enjoy watching the Olympics not only because it’s amazing watching athletes achieve incredible feats, but also because the Olympics present so many opportunities to explore Decision Education topics! Here, we’re exploring how accurately understanding data can make us better decision makers.

Why is it important to understand data?

We make judgments and decisions based on our interpretation of data. Is a certain car safe? Check the data. Should we undergo a certain medical procedure? Check the data. How likely is it that it will rain tomorrow? Check the data!

If our understanding of data is incorrect, this upsets the foundation for many of our decisions. So it’s important that we ask ourselves questions to make sure we are interpreting data accurately.

How can we make sure we’re understanding data correctly?

One factor we can focus on is making sure we are taking an “outside view” when it comes to our interpretation of data—that means taking a comprehensive look at all the relevant data rather than only the narrow “inside view” based on anecdotal evidence or the limited facts of the current situation. We can get an outside view by looking for base rates (the average occurrence of something based on historical data) to compare data to, checking the sample size of our data to make sure it’s large enough to be representative, and/or asking ourselves what information could possibly be missing from our data.

Was an Olympic judge biased?

To understand why an outside view is so important in making our judgments, let’s explore an example from the 2000 Summer Olympics.

At the 2000 Summer Olympics in Sydney, Mexican diver Fernand Platas lost the gold medal in the 3-meter springboard diving competition by a very narrow margin to Ni Xiong of China. The Chinese judge, Facheng Wang, gave Ni Xiong three of the highest scores on his last three dives, and some in the public accused Wang of being biased toward Chinese athletes. But is that conclusion accurate?

Let’s dive in by looking at the numbers. Below are the scores for Ni Xiong’s dive from all of the judges. As we can see, the Chinese judge’s score was the highest, and it was 0.64 points higher than the average score of 7.86.

So Wang is biased, right? Well, let’s look at data for all Chinese divers to see if there’s a trend for Wang. When we look at Judge Wang’s average discrepancy for all Chinese divers, we see his discrepancy is still +.17. That must mean he was being biased toward divers from his own country, right? If we were to stop right here, we might think so!

Except, here’s where we can trip up in our reasoning and come to incorrect conclusions if we don’t step back and look at the bigger picture. While we know Wang’s average discrepancy for Chinese divers, what is his average discrepancy for divers from all countries? Is it possible that Wang tends to give higher scores in general?

When we look at Judge Wang’s average discrepancy for all divers, we see his discrepancy is still +.17. That means there is no difference between his scores for Chinese divers and all other divers. His scores don’t reflect that he’s biased, just that he’s a high scorer!

Moreover, if we look at all the judging data from this event, we see other examples of potential bias that were not publicized. In fact, if we examine the American judge, McFarland, we see that he averaged a +0.234 discrepancy for American divers, and had an average discrepancy of +0.012 for non-American divers, suggesting a possible bias.

 

Learning from this example

This example illustrates why we need to understand data before jumping to conclusions or making decisions. Without comparing the context of data to get an outside view, many people falsely accused Judge Wang of bias toward his country’s athletes.

These same types of data misinterpretations can have a real impact on everyday decisions and larger life decisions that affect our finances, relationships, or well-being.

When we are better at understanding what data is showing us, we can be better decision makers. We hope you enjoy the Paris 2024 Summer Olympics, and just remember—before you yell at your TV that one of the judges may be biased, check the data!

For deeper dives into this topic, see below:

Nationalistic Judging Bias in the 2000 Olympic Diving Competition by John W. Emerson and Silas Meredith

Lecture for Wharton Moneyball Academy

Share this article to your favorite platform!

Check out our other blog posts

  • Researcher Feature: Dr. Brittany Shoots-Reinhard, Ohio State University

    We spoke with Dr. Brittany Shoots-Reinhard from Ohio State University about her research on decision-making and Decision Education.

  • Interview With Robin Gregory and Brooke Moore, Co-Authors of “Sorting It Out: Supporting Teenage Decision Making”

    In "Sorting It Out: Supporting Teenage Decision Making," authors Robin Gregory and Brooke Moore offer practical strategies from decision science to aid teenagers in developing critical decision-making skills.

Stay informed and join our mailing list