09/10/25

More math. Boys, girls and Bayes, oh my.

Bayes' theorem

Since a vague disclaimer is nobody's friend: I have no formal training in mathematics, and all I know (or think I know) about Bayes' theorem is based on stuff that I found on the internet.

To learn about Bayes' theorem, this might be a good starting point:
Wikipedia: Bayes' theorem

And here's a Bayesian analysis of the Monty Hall problem (which was what got me interested in Bayes' theorem in the first place):
Wikipedia: Monty Hall problem - Bayesian analysis

More about Bayes' theorem:
Stanford Encyclopaedia of Philosophy: Bayes' theorem

The problem

This problem is used on several Wikipedia pages as an example of the application of Bayes' theorem:

"Suppose there is a school having 60% boys and 40% girls as students. The female students wear trousers or skirts in equal numbers; the boys all wear trousers. An observer sees a (random) student from a distance; all the observer can see is that this student is wearing trousers. What is the probability this student is a girl?"
Wikipedia: Bayes' theorem - Simple example

I know next to nothing about mathematics, and I found this problem very useful in understanding (at least some of) the reasoning behind Bayes' theorem.

The formula

Bayes' theorem gives us the following formula for problems like these:

P(A|B) = P(B|A) P(A) / P(B)

In our case, A means the child is a girl, B means that the child wears trousers and P(A|B) is the probability of A given B which is, in this case, the probability of a trouser-wearing child being a girl.

A visual representation of the variables:

	A (child is a girl)	not A (child is a boy)
B (child wears trousers)
not B (child wears skirt)

A visual representation of P(A|B), the answer we're looking for:

	A (child is a girl)	not A (child is a boy)
B (child wears trousers)	P(A|B)

Step one: determining P(B|A)

P(B|A) is the probability of B given A or, in our case, the probability of any girl wearing trousers.

	A (child is a girl)
B (child wears trousers)	P(B|A)
not B (child wears skirt)

This is what we know:

"The female students wear trousers or skirts in equal numbers; [...]"

This means that P(B|A) is 0,5.

Step two: multiplying by P(A)

P(A) is the probability of A or, in our case, the probability of any student being a girl.

	A (child is a girl)	not A (child is a boy)
B (child wears trousers)	P(A)
not B (child wears skirt)

This is what we know:

"[...] a school having 60% boys and 40% girls as students."

This means that P(A) is 0,4.

Multiplying P(B|A) by P(A) gives us P(A and B), the probability that A and B are both true or, in our case, the probability of any of the students being a trouser-wearing girl.

	A (child is a girl)	not A (child is a boy)
B (child wears trousers)	P(A and B)
not B (child wears skirt)

P(B|A) P(A) is 0,5 * 0,4 = 0,2.

Step three: dividing by P(B)

P(B) is the probability of B or, in our case, the probability of any student at the school wearing trousers.

	A (child is a girl)	not A (child is a boy)
B (child wears trousers)	P(B)
not B (child wears skirt)

This is what we know:

"[...] a school having 60% boys and 40% girls as students. The female students wear trousers or skirts in equal numbers; the boys all wear trousers.

Since P(B) isn't given, we'll have to determine it based on what we do know:

• P(B|A), the probability of any girl wearing trousers;
• P(A), the probability of any student being a girl;
• P(B|not A), the probability of any boy wearing trousers;
• P(not A), the probability of any student being a boy.

We can determine P(B) using the following formula:

P(B) = P(B|A) P(A) + P(B|not A) P(not A)

P(B) is 0,5 * 0,4 + 1 * 0,6 = 0,2 + 0,6 = 0,8.

Dividing P(B|A) P(A) by P(B) gives us P(A|B), the probability of A given B or, in our case, the probability of any child in trousers being a girl. And here's our solution:

	A (child is a girl)	not A (child is a boy)
B (child wears trousers)	P(A|B)

P(A|B) is 0,5 * 0,4 / 0,8 = 0,25.

09/10/22

We've got answers, but what is the question?

The return of Monty Hall

The Monty Hall problem in its best-known form:

"Suppose you're on a game show, and you're given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what's behind the doors, opens another door, say No. 3, which has a goat. He then says to you, "Do you want to pick door No. 2?" Is it to your advantage to switch your choice?"
Whitaker, Craig F. (1990). [Letter]. "Ask Marilyn" column, Parade Magazine p. 16 (9 September 1990), as quoted on the Wikipedia page about the Monty Hall problem.
Wikipedia: Monty Hall problem

What we know

The problem as it is worded gives us information about four different things:

• 1 The context. You're on a game show. You're given the choice of three doors. Behind one door is a car. Behind the other doors are goats (presumably there's one goat behind each door, though this isn't relevant to the problem).
• 2 You pick a door.
• 3 The host opens one of the two doors that haven't been picked by you. Behind it is a goat.
• 4 The host offers you the option to switch your choice to the one, remaining door.

What we don't know

The wording of the problem raises a number of questions:

• 1 Could the car be behind any of the three doors? Can the car or the goats be removed or moved to different doors during the game?
• 2 How do you decide which door to pick?
• 3 Will the host always open a door? How does he decide which door to open? Can he open the door that you picked (ending the game)? Can he open the door that has the car behind it (presumably invalidating the game)?
• 4 Will the host always offer you the option to switch? What if you don't care about the car but you really, really like goats?

Assumptions

In order to answer the question, we'll need to assume that a number of things will be true whenever the game is played. Based on the information that we have, none of the following assumptions seems too far-fetched:

• 1A The placement of the car behind the doors is random. Once you've picked your door, the car and the goats will not be moved to different doors and they will not be removed until the end of the game.
• 2A Your choice of doors is random.
• 3A The host will open a door after you've made your choice.
• 3B The host won't open the door that was picked by you.
• 3C The host won't open the door that has the car behind it.
• 3D When given a choice between two doors to open, the host will open one of them at random.
• 4A The host will offer you the option to switch.
• 4B You win by picking the door that has the car behind it.

The game

Based on these assumptions, we can chart everything that might happen in the game as follows:

Car	You	Host	Probability	Assumptions	Result of switching	Assumptions
1	1	1	0	3B and / or 3C	n/a	n/a
		2	1/18	1A and 2A and 3A and 3D	you lose	4A and 4B
		3	1/18	1A and 2A and 3A and 3D	you lose	4A and 4B
	2	1	0	3C	n/a	n/a
		2	0	3B	n/a	n/a
		3	1/9	1A and 2A and 3A	you win	4A and 4B
	3	1	0	3C	n/a	n/a
		2	1/9	1A and 2A and 3A	you win	4A and 4B
		3	0	3B	n/a	n/a
2	1	1	0	3B	n/a	n/a
		2	0	3C	n/a	n/a
		3	1/9	1A and 2A and 3A	you win	4A and 4B
	2	1	1/18	1A and 2A and 3A and 3D	you lose	4A and 4B
		2	0	3B and / or 3C	n/a	n/a
		3	1/18	1A and 2A and 3A and 3D	you lose	4A and 4B
	3	1	1/9	1A and 2A and 3A	you win	4A and 4B
		2	0	3C	n/a	n/a
		3	0	3B	n/a	n/a
3	1	1	0	3B	n/a	n/a
		2	1/9	1A and 2A and 3A	you win	4A and 4B
		3	0	3C	n/a	n/a
	2	1	1/9	1A and 2A and 3A	you win	4A and 4B
		2	0	3B	n/a	n/a
		3	0	3C	n/a	n/a
	3	1	1/18	1A and 2A and 3A and 3D	you lose	4A and 4B
		2	1/18	1A and 2A and 3A and 3D	you lose	4A and 4B
		3	0	3B and / or 3C	n/a	n/a

The numbers in the Car column represent the location of the car. The numbers in the You column represent the door picked by you. The numbers in the Host column represent the door opened by the host. The Probability column gives the probability for each combination of car location, door picked by you and door opened by the host. The leftmost Assumptions column lists the assumptions on which the probability is based. The Results of switching column lists, well, the expected results of switching, and the second Assumptions column lists the underlying assumptions.

Should you switch? The unconditional approach

When we add up all the odds in the table, we find a 6/18 (or 1/3) probability of losing by switching, and a 6/9 (or 2/3) probability of winning by switching.

Actually, we don't need the entire table to figure that out. All we need to know is the probability of a player picking the door with the car on their first guess. The 1/3 probability is based on the following assumptions:

• 1A The placement of the car behind the doors is random. Once you've picked your door, the car and the goats will not be moved to different doors and they will not be removed until the end of the game.
• 2A Your choice of doors is random.

If we prefer not to assume anything (at least not more than strictly necessary) we could also say that if P is the probability of a player picking the car on their first guess, then the probability of any player losing when they switch is P and the probability of any player winning when they switch is 1-P.

For switching to be a winning strategy, we also need to assume that the following applies:

• 3A The host will open a door after you've made your choice.
• 3B The host won't open the door that was picked by you.
• 3C The host won't open the door that has the car behind it.
• 4A The host will offer you the option to switch.
• 4B You win by picking the door that has the car behind it.

Now, it's important to keep in mind that what we have here is the overall probability - out of 6,000 players who play the game and switch, we may expect 4,000 to win and 2,000 to lose (assuming that P=1/3).

Should you switch? The conditional approach

Going back to the original problem:

"You pick a door, say No. 1, and the host, who knows what's behind the doors, opens another door, say No. 3, which has a goat. He then says to you, "Do you want to pick door No. 2?" Is it to your advantage to switch your choice?"

The problem isn't asking for an overall probability of 6,000 players winning the game by switching. The problem is asking about what you should do, as you're on stage with Monty Hall, looking at two closed doors and one goat.

Let's take a look at the situation described in the original problem: you've picked door 1, the host has opened door 3 and he asks you whether you want to switch to door 2.

Car	You	Host	Probability	Assumptions	Result of switching	Assumptions
1	1	1	0	3B and / or 3C	n/a	n/a
		2	1/18	1A and 2A and 3A and 3D	you lose	4A and 4B
		3	1/18	1A and 2A and 3A and 3D	you lose	4A and 4B
	2	1	0	3C	n/a	n/a
		2	0	3B	n/a	n/a
		3	1/9	1A and 2A and 3A	you win	4A and 4B
	3	1	0	3C	n/a	n/a
		2	1/9	1A and 2A and 3A	you win	4A and 4B
		3	0	3B	n/a	n/a
2	1	1	0	3B	n/a	n/a
		2	0	3C	n/a	n/a
		3	1/9	1A and 2A and 3A	you win	4A and 4B
	2	1	1/18	1A and 2A and 3A and 3D	you lose	4A and 4B
		2	0	3B and / or 3C	n/a	n/a
		3	1/18	1A and 2A and 3A and 3D	you lose	4A and 4B
	3	1	1/9	1A and 2A and 3A	you win	4A and 4B
		2	0	3C	n/a	n/a
		3	0	3B	n/a	n/a
3	1	1	0	3B	n/a	n/a
		2	1/9	1A and 2A and 3A	you win	4A and 4B
		3	0	3C	n/a	n/a
	2	1	1/9	1A and 2A and 3A	you win	4A and 4B
		2	0	3B	n/a	n/a
		3	0	3C	n/a	n/a
	3	1	1/18	1A and 2A and 3A and 3D	you lose	4A and 4B
		2	1/18	1A and 2A and 3A and 3D	you lose	4A and 4B
		3	0	3B and / or 3C	n/a	n/a

Once more, when we look at the odds in the table we find a 1/3 probability that you will lose by switching, and a 2/3 probability that you will win by switching.

In fact, we're talking about two distinct scenarios with a non-zero probability: if the car is behind door 1 you will lose by switching, and if the car is behind door 2 you will win by switching. The 1/18 probability of the car being behind door 1 is based on the fact that the host has a choice between opening door 2 and opening door 3, and on our assumption:

• 3D When given a choice between two doors to open, the host will open one of them at random.

Assumption 3D is irrelevant to the unconditional approach to solving the problem, but it is very much relevant to the conditional approach. If the host likes door 3 and will open it whenever possible, the probability that you will win by switching could be as low as 1/2. If, on the other hand, the hosts dislikes door 3 and will only open it when he has no other choice, the probability of winning by switching could be as high as 1.

09/10/18

Though I usually don't do this, I updated yesterday's post to correct a number of errors.

Monty Hall revisited

Get the all new and improved version here:
09/10/17

09/10/17

I have a cold, couldn't sleep, and ended up browsing the Wikipedia pages about the Monty Hall problem. The discussion on the Talk pages is longer and more heated than you might expect, and I think I actually learned something.

The Monty Hall problem

"Look. Here are three doors. Behind one of them is a car, and behind the other two are goats. Pick one. If it's the one with the car, you win. Otherwise, you lose..."

"So, you've picked your door. Are you sure? Good. Now, I'm going to open one of the other doors. Oh, look, it's a goat..."

"I'm going to give you a choice. You can either stay with the door that you originally picked, or you can switch. What will it be?"

Why you should switch

Let's say the following is true:

1. the show's host knows behind which door the car is
2. he will always open a door after you've made your pick, and the door that he opens will always have a goat behind it
3. if you have picked the door with the car, the host will open a random door that has a goat behind it
4. the host will always offer you the option to switch

In that case, you should switch.

Let's look at the possibilities:

Your choice	Host's choice	Probability	You switch?	Result
car	1st goat	1/6	yes	you lose
			no	you win
	2nd goat	1/6	yes	you lose
			no	you win
1st goat	2nd goat	1/3	yes	you win
			no	you lose
2nd goat	1st goat	1/3	yes	you win
			no	you lose

Now, look at what would happen if you would always decide to switch:

Your choice	Host's choice	You switch?	Result	Probability
car	1st goat	yes	you lose	1/6
	2nd goat	yes	you lose	1/6
1st goat	2nd goat	yes	you win	1/3
2nd goat	1st goat	yes	you win	1/3

If you switch, you have a 2/3 chance of winning. If you don't switch, you have a 1/3 chance of winning.

Let's look at this in a different way. The host opened one door, which has a goat behind it. Behind one of the remaining closed doors is a car and behind the other one is a goat. Switching will either take you from the door with the car to the door with the goat, or from the door with the goat to the door with the car. When you initially picked your door, you had a 1/3 chance of picking the door with the car, and a 2/3 chance of picking a door with a goat. Therefore, there is a 1/3 chance that switching will take you from the door with the car to the door with the goat, and a 2/3 chance that it will take you from the door with the goat to the door with the car.

Why it doesn't matter what you do

Now, let's say the following is true:

1. the show's host doesn't know behind which door the car is
2. he will always open a door after you've made your pick, and the door that he opens may have either a goat or a car behind it
3. the host picks a door at random
4. the host will always offer you the option to switch

In that case, it doesn't matter what you do.

Once again, let's look at the possibilities:

Your choice	Host's choice	Probability	You switch?	Result
car	1st goat	1/6	yes	you lose
			no	you win
	2nd goat	1/6	yes	you lose
			no	you win
1st goat	car	1/6	n/a	you lose
1st goat	2nd goat	1/6	yes	you win
			no	you lose
2nd goat	car	1/6	n/a	you lose
2nd goat	1st goat	1/6	yes	you win
			no	you lose

There's a 1/3 chance that you won't even be offered the option to switch, since the host will have opened the door that has the car behind it. Again, let's look at what would happen if you would always decide to switch whenever you are given the option:

Your choice	Host's choice	You switch?	Result	Probability
car	1st goat	yes	you lose	1/4
	2nd goat	yes	you lose	1/4
1st goat	2nd goat	yes	you win	1/4
2nd goat	1st goat	yes	you win	1/4

If you do get the option of switching, it doesn't matter what you do: you'll have a 1/2 chance of winning either way.

Why you shouldn't switch

Let's say the following is true:

1. the show's host knows behind which door the car is
2. he will always open a door after you've made your pick, and the door that he opens will always have a goat behind it
3. if you have picked the door with the car, the host will open a random door that has a goat behind it
4. the host will only offer you the option to switch if you've picked the door with the car behind it

In this scenario you will lose if you switch and win if you don't.

Confused yet?

Consider the following scenario:

1. the show's host knows behind which door the car is
2. he will always open a door after you've made your pick, and the door that he opens will always have a goat behind it
3. if you have picked the door with the car, the host will open the leftmost door that has a goat behind it
4. the host will always offer you the option to switch

Since the host is going to be displaying specific behaviour in this scenario, let's look at his choices first:

Your choice	Car	Host's choice	Probability
1st door	1st door	2nd door	1/9
	2nd door	3rd door	1/9
	3rd door	2nd door	1/9
2nd door	1st door	3rd door	1/9
	2nd door	1st door	1/9
	3rd door	1st door	1/9
3rd door	1st door	2nd door	1/9
	2nd door	1st door	1/9
	3rd door	1st door	1/9

Let's juggle that around a bit, so it becomes more obvious what the host's choices tell you about the whereabouts of the car:

Your choice	Host's choice	Car	Probability
1st door	2nd door	1st door	1/9
		3rd door	1/9
1st door	3rd door	2nd door	1/9
2nd door	1st door	2nd door	1/9
		3rd door	1/9
2nd door	3rd door	1st door	1/9
3rd door	1st door	2nd door	1/9
		3rd door	1/9
3rd door	2nd door	1st door	1/9

In three cases, your knowledge of the host's preferences will tell you where the car is. In each of these three cases you'll need to switch in order to win.

Let's look at what happens when you play the game, knowing what you do about the host's preferences:

Your choice	Host's choice	Car	Probability	You switch?	Result
1st door	2nd door	1st door	1/9	yes	you lose
				no	you win
		3rd door	1/9	yes	you win
				no	you lose
1st door	3rd door	2nd door	1/9	of course!	you win
2nd door	1st door	2nd door	1/9	yes	you lose
				no	you win
		3rd door	1/9	yes	you win
				no	you lose
2nd door	3rd door	1st door	1/9	of course!	you win
3rd door	1st door	2nd door	1/9	yes	you win
				no	you lose
		3rd door	1/9	yes	you lose
				no	you win
3rd door	2nd door	1st door	1/9	of course!	you win

There's a 1/3 chance that the host's actions will tell you where the car is, and switching will make you win. In all other cases it doesn't matter what you do.

You can optimise your chances of winning, even if you know nothing about the host's preferences, by always switching:

Your choice	Host's choice	Car	You switch?	Result	Probability
1st door	2nd door	1st door	yes	you lose	1/9
		3rd door	yes	you win	1/9
	3rd door	2nd door	yes	you win	1/9
2nd door	1st door	2nd door	yes	you lose	1/9
		3rd door	yes	you win	1/9
	3rd door	1st door	yes	you win	1/9
3rd door	1st door	2nd door	yes	you win	1/9
		3rd door	yes	you lose	1/9
	2nd door	1st door	yes	you win	1/9

Once again, switching gives you a 2/3 chance of winning.

So...

Switching often gives you the best chance of winning.

But consider the following scenario:

1. the host knows where the car is
2. if you've picked a door with a goat behind it, the host will open the door with the car and the game will be over. If you've picked the door with the car, the host will open a door with a goat
3. ...
4. the host will always offer you the option to switch (but actually you will only get the option if you've chosen the door with the car)

Or the following scenario, as discussed earlier:

1. the host knows where the car is
2. ...
3. ...
4. the host will only offer you the option to switch if you've picked the door with the car behind it

In both scenarios, you'll always lose if you switch. Also, if the host doesn't open a door before giving you the option to switch, the benefits of switching disappear (though switching may still be an effective strategy in specific situations).