Lesson Plan

A Look at Base Rates

Grade Level: 6-12

Length: 45-60 minutes

DE Domain: CB TP

  • Students will understand the importance of considering frequencies and base rates.
  • Students will use frequencies and base rates to predict the likelihood of various possible outcomes to a scenario and communicate their predictions using subjective probabilities.

Understanding frequencies and base rates helps us to calibrate our thinking and make informed decisions. In this lesson, students explore the concept of a base rate, how it can inform their thinking, and guide their decision-making.

TP.3 – Using probabilistic thinking to evaluate evidence and truth claims, and to update beliefs

TP.5 – Use probabilistic thinking to weigh decision options and their possible outcomes

CB.8 – Recognize and actively resist the tendency to ignore measures of likelihoods and relevant data when making judgements

What to look and listen for: Are students able to articulate why a base rate is important, and the implications of the base rate neglect? Do students update their recommendation? Are they able to explain why? Are students able to weigh pros and cons of applying base rates in different scenarios?

Lesson:

Engage (15 minutes):

Introduce the concept of base rates to your students in two parts.

Please visit this teaching resource on Understanding Base Rates for details about these concepts.

Base rate: A number, usually represented as a percentage, that represents the prevalence of a characteristic in a population. It gives you information on how frequent or likely an event might be.

Base rate neglect: The tendency to ignore base rates.

Suggested questions:

  • Why are base rates important?
  • Why do you think there is a tendency to ignore base rates? How could that be harmful?

Then, watch the “Know Your Bias: Base Rate Neglect” video on base rates and base rate neglect and follow up with the questions below.

Suggested Questions for the Video:

  • What did you find surprising about the video?
  • What is a base rate and why was it important in this situation?
  • Consider if the base rate was less extreme, how should that change our view of the situation?
  • Can you connect this idea of base rates or base rate neglect to a situation you have encountered?

Apply (20 minutes):

Introduce probability estimates as a way of improving the quality of our decisions. Give the following scenario to your students. You might have them draw a diagram similar to the visuals in the taxi video to aid in their estimations of likelihoods when considering the base rate.

Scenario:

Let’s imagine that you are on a panel to decide whether to adopt facial recognition software at a busy airport (300,000 people walk through the Atlanta airport daily) to screen the public against an FBI watchlist. Let’s say that the facial recognition software you are researching claims a 99.7% success rate, meaning a false negative rate (not identified as being on a watchlist when the person is on a watchlist) of 0.3%. We will assume the false positive rate (wrongly identified as being on a watchlist) is also 0.3%.

Suggested Questions:

  • After reading the scenario, what is your initial recommendation?
  • Let’s assume 1 in 4,000 people are expected to be on the watchlist. How many people in this busy airport statistically are on the watchlist? [answer: 75 people]
  • How many people in total are flagged at this busy airport daily, considering that 300,000 people walk through the Atlanta airport? [answer: 975 people are flagged]
  • What’s the probability that someone who is flagged by the software is actually on the watchlist? [answer: 7.7% or about 8% – see detailed solution here]
  • What are the potential consequences for an individual who was flagged by the software? What could this mean for someone who is wrongly accused?
  • Revisit the first question. Has your understanding of the base rate shifted your thinking? Do you want to update your recommendation?

Reflect (10 minutes):

Connect the prior experience back to base rate and base rate neglect.

Suggested Discussion Questions:

  • What do you think are the implications of using just the 99.7% success rate in your consideration?
  • What changed when you considered the base rate?
  • The human identification of individuals has a much higher error rate than facial recognition. However, if we place our trust in the accuracy of the software, how does that affect our judgments of individuals who are flagged using this software?

Close by reviewing the big takeaways.

The following three principles can be used to improve the quality of judgements and choices you make. The considerations below are often attributed to “Bayesian Thinking.”

  • Remember to consider base rate probability when applicable. What is most likely? Estimating base rates is a skill that can be developed, even thinking about what is likely is helpful.
  • Consider how likely it is that your initial assumption is wrong. How much does knowledge of a base rate affect your assumptions?
  • Update your stance incrementally. Your understanding of the base rate might shift as you learn more about the world.

Differentiation:

Differentiation: If you are teaching advanced math students, consider adding a section in Apply where students work through the math in the taxi problem. Introduce them to Bayes Theorem and discuss when and why you need or don’t need to explicitly work through this equation.

Consider situations where using base rates is inappropriate – for example, when base rates would reinforce stereotypes by profiling marginalized populations, which could lead to harmful conclusions. Encourage students to think critically about why and how some base rates exist.

Optional extensions:

For Apply, repeat the likelihood of a flagged individual being on the watchlist if they were already flagged and rescanned by the software a second time. Do this again using a new prior base rate of 7.7% [answer: about 97%]

Helpful teacher resources:

Article: Why do we rely on specific information over statistics?

Article: Understanding Bayes Theorem (optional extension)

Stay informed and join our mailing list