insideout Get Pro

03 · Conditional probability

Free

Probability, lesson 3

Conditional probability

How the odds change once you know something.

A chance changes when you learn something, even though nothing about the dice or the cards has changed. This lesson is about updating a chance on news. You need the last lesson's rule: a chance is a share of equally likely outcomes.

By the end, you can:

  1. Update a chance on news by counting what's left.
  2. Multiply chances for cards drawn one after another.
  3. Find a chance by splitting on what you don't know.

Part 1. The chance given what you know

Update a chance on news by counting what's left.

Roll a red die and a blue die. The chance of a total of 7 is 6/36 = 1/6. Now someone peeks and tells you at least one die shows a 6. Every square where that's false can't be the roll any more. Cross those out, and count again among the squares left.

Example 1.1

A total of 7, given at least one 6

Roll a red die and a blue die. You learn that at least one of them shows a 6. What's the chance the total is 7?

  1. Keep the squares the news allows6 + 6 − 1 = 11

    A 6 on the red die is 6 squares, a 6 on the blue die 6 more. Six and six is in both, so take 1 away.

  2. Count the ones you want2

    1 + 6 and 6 + 1 are the only squares left that total 7.

  3. Divide by what's left2 / 11

    The squares are still equally likely. Only the bottom of the fraction changed, from 36 to 11.

Answer2/11, about 18%, up from 1/6.

234 567 345 678 456 789 567 8910 678 91011 78 91011 12 11223 3445 566 blue die red die At least one 6 11 squares left Total of 7 2 of them 2 / 11 about 18%
234 56 73 456 784 567 895 678 9106 789 1011 78 91011 12 11223 3445 566 blue die red die At least one 6: 11 squares left 2 of them total 7: 2/11, about 18%
Figure 1Given at least one 6, the 25 squares without a 6 are out. Of the 11 left, 2 total 7, so the chance moves from 6/36 to 2/11.

Why is this right? The news doesn't change the dice. It only tells you which squares are still possible. Those 11 were equally likely before the news, so they are equally likely after it, and the chance is a share of them.

Conditional probability

P(A | B), read "the probability of A given B", is the chance of A once you know B happened. Count A inside B. With chances instead of counts, divide. For the dice, (2/36) / (11/36) = 2/11.

P(A | B) = P(A and B) / P(B)

The common trap is swapping the two sides. Every lottery winner bought a ticket, so P(bought a ticket | won) = 1. Almost no ticket buyer wins, so P(won | bought a ticket) is tiny. The same two events give very different chances, because the bottoms of the fractions differ.

Your turn

A family has two children, and you learn at least one is a girl. What's the chance both are girls? (Each child is a girl or a boy with chance 1/2, independently.)

Four equally likely families: GG, GB, BG, BB. "At least one girl" rules out BB and leaves three, and one of them is GG: 1/3. If you'd learned that the older child is a girl, only GG and GB would be left, and the answer would be 1/2.

Part 2. One card, then another

Multiply chances for cards drawn one after another.

Cards are where this matters most, because every card you draw changes the deck. Draw two cards without putting the first back, and the second card's chances depend on what the first one was. So take the cards in order, and condition each one on the cards before it.

Example 2.1

Two aces in a row

Draw two cards from a shuffled deck, without putting the first back. What's the chance both are aces?

  1. First card4/52

    4 aces among 52 cards.

  2. Second card, given the first3/51

    After an ace, 3 aces are left among 51 cards.

  3. Multiply4/52 × 3/51 = 12/2,652 = 1/221

    Of the deals that start with an ace, 3/51 go on to a second ace.

Answer1/221, about 0.45%.

Why multiply? Picture many deals. The first card is an ace in 4/52 of them, and of those, the second is an ace 3/51 of the time. A share of a share is a product. Counting agrees: C(4, 2) = 6 pairs of aces among C(52, 2) = 1,326 pairs, and 6/1,326 = 1/221.

The multiplication rule for chances

The chance of both is the chance of the first, times the chance of the second given the first. It's the formula for P(B | A), turned around:

P(A and B) = P(A) × P(B | A)

Leave out the condition and you'd write 4/52 × 4/52 = 1/169, as if the first ace went back in the deck. That's right when you replace the card. Without replacement, the second card comes from what the first one left behind.

The rule follows the cards as they come, and counting looks at the finished hand. Both give 1/221. When one way gets messy, try the other.

Your turn

Draw two cards from a shuffled deck, without putting the first back. What's the chance both are hearts?

13 of the 52 cards are hearts, then 12 of the 51 left: 13/52 × 12/51 = 156/2,652 = 1/17. The answer 1/16 is 1/4 × 1/4, as if the first heart went back, and 12/51 is only the second step.

Part 3. Splitting into cases

Find a chance by splitting on what you don't know.

Guess first

Shuffle a deck and deal two cards face down. What's the chance the second one is an ace?

Before you look at anything, the second card is as likely to be any of the 52 as the first is, so it's 4/52 = 1/13. The long way agrees: (4/52)(3/51) + (48/52)(4/51) = 204/2,652 = 1/13.

It's tempting to want the first card before you answer. You don't need it. The first card was either an ace or not, never both, so split on it and work out each case.

Example 3.1

The second card

Deal two cards face down from a shuffled deck. What's the chance the second one is an ace?

  1. First card an ace4/52 × 3/51 = 12/2,652

    Then 3 aces are left among 51 cards.

  2. First card not an ace48/52 × 4/51 = 192/2,652

    Then all 4 aces are left among 51 cards.

  3. Add the cases12/2,652 + 192/2,652 = 204/2,652

    The cases can't both happen, and one of them must, so their chances add.

  4. Simplify204/2,652 = 1/13

    Exactly 4/52, the same as for the first card.

Answer1/13, the same as for the first card.

First card Second card 4/52 48/52 3/51 48/51 4/51 47/51 Ace Not ace Ace 4 × 3 = 12 Not 4 × 48 = 192 Ace 48 × 4 = 192 Not 48 × 47 = 2,256 Second card an ace: 12 + 192 = 204 of 2,652 = 1/13
First card Second card Deals 4/52 48/52 3/51 48/51 4/51 47/51 Ace Not ace Ace 12 Not 192 Ace 192 Not 2,256 Second card an ace: 12 + 192 = 204 204 of 2,652 deals = 1/13
Figure 2Each path through the tree is a case, and its chance is the product along it. The two orange paths end in an ace, and together they make 1/13.

The law of total probability

Split on what you don't know, into cases that can't overlap and that together cover everything. The chance of A is its chance in each case, weighted by how likely that case is:

P(A) = P(case 1) × P(A | case 1) + P(case 2) × P(A | case 2)

There's also a shortcut. Before anyone looks, nothing makes the second place in a shuffled deck different from the first, so each is an ace with chance 4/52. The trap is reasoning as if you knew the first card. A condition changes a chance only once you've learned it.

Your turn

A strategy trades every day. 80% of days are calm, and it loses money on 30% of those. 20% are volatile, and it loses on 60% of those. On what share of days does it lose?

Split on the kind of day: 0.8 × 0.3 + 0.2 × 0.6 = 0.24 + 0.12 = 0.36. The answer 45% averages 30% and 60% as if both kinds of day were equally common, and 90% adds them without weights.

Where you'll meet it

In interviews

Cards drawn without replacement, "given at least one" questions and the two-child puzzle come up often in quant interviews. Write down the equally likely outcomes, cross out the ones the news rules out, and count what's left. Say what you're conditioning on before you compute. Many wrong answers are right answers to a slightly different question.

On the job

Testing a trading signal is conditional probability. Say a stock rose on 520 of the last 1,000 days, so P(up) = 52%. A signal fired on 200 of those days, and the next day was up 116 times: P(up next day | signal) = 116 / 200 = 58%. If the two numbers matched, the signal would tell you nothing.

Conditioning by counting, in Python
from fractions import Fraction as F
from itertools import product

def chance(outcomes, event, given=lambda o: True):
    left = [o for o in outcomes if given(o)]  # cross out what the news rules out
    return F(sum(1 for o in left if event(o)), len(left))

dice = list(product(range(1, 7), repeat=2))
seven, six = (lambda s: sum(s) == 7), (lambda s: 6 in s)
print(chance(dice, seven, given=six), chance(dice, six, given=seven))  # 2/11 1/3

kids = list(product("GB", repeat=2))
print(chance(kids, lambda k: k == ("G", "G"), given=lambda k: "G" in k))  # 1/3
print(F(4, 52) * F(3, 51), F(13, 52) * F(12, 51))  # 1/221 1/17
print(F(4, 52) * F(3, 51) + F(48, 52) * F(4, 51))  # 1/13

chance does by brute force what the lesson does by hand: keep the outcomes that fit the news, then count. It's a quick way to settle a tricky wording on a small case.

Check yourself

Four questions from across the lesson. Try each one before you look back.

1

Which is bigger: the chance of at least one 6 given the total is 7, or the chance the total is 7 given at least one 6?

Both count the same 2 squares, 1 + 6 and 6 + 1. The first divides by the 6 squares that total 7, the second by the 11 squares with a 6. P(A | B) and P(B | A) share a top and have different bottoms.

2

Box 1 holds 1 red ball and 1 blue. Box 2 holds 3 red and 1 blue. Flip a coin to pick a box, then take a ball from it. What's the chance it's red?

Split on the box: 1/2 × 1/2 + 1/2 × 3/4 = 1/4 + 3/8 = 5/8. The answer 2/3 pours both boxes together, 4 red of 6, but each ball in the small box is more likely to be picked than one in the big box.

3

Draw two cards from a shuffled deck, without putting the first back. What's the chance at least one is an ace?

Count the opposite, as in the last lesson. No ace is 48/52 for the first card, then 47/51 for the second given the first. So 1 − (48/52)(47/51) = 33/221, about 14.9%. Squaring 48/52 puts the first card back, and 2 × 4/52 counts two aces twice.

4

Roll a red die and a blue die. You learn the red one shows 3. What's the chance the total is 7?

Only the 6 squares with a red 3 are left, and one of them, 3 + 4, totals 7: 1/6. This news changes nothing, because whatever the red die shows, exactly one blue face makes 7.

What to remember

  1. 1The chance given what you know

    News crosses out outcomes, so count what's left: given at least one 6, a total of 7 is 2 of the 11 squares left, or 2/11.

  2. 1The chance given what you know

    P(A | B) and P(B | A) share a top and have different bottoms: for the same 2 squares, 2/11 against 1/3.

  3. 2One card, then another

    For draws one after another, multiply each chance given the ones before: two aces is 4/52 × 3/51 = 1/221.

  4. 3Splitting into cases

    Split on what you don't know, weight each case by its chance and add: the second card is an ace with chance 12/2,652 + 192/2,652 = 1/13.

Next lessonBayes' ruleNext, Bayes' rule turns the chance of a positive test given an illness into the one you want: the chance of the illness, given a positive test. Start lesson 4

Probability overviewSpotted a mistake? Tell us.