Sign in to continue learning
Probability & Statistics for Machine Learning & Data Science
33 hours 26 mins
So remember I told you before that the normal distribution appears in a lot of places? Here's one that perhaps you didn't expect. Take a distribution, any distribution, it can be as skewed as you want. Now take a few samples, always the same number, and look at the average. And do this many times. And plot all these averages. Guess what you get? Yes, you get a normal distribution. No matter what distribution you started with in the first place. This is a fascinating result and it's one of the pinnacles of statistics. It's called the central limit theorem. So let's start with an example of a discrete random variable that you've seen a lot of time in this course. The probability of heads and tails is the same and the random variable X is the number of heads when a coin is flipped n times. So for one coin flip it's X equals 1 for the case of heads and X equals 0 for the case of tails. And that's a discrete random variable with this probability distribution over here. A half for X equals 1 and a half for X equals 0, where the number of heads is in the horizontal axis and the probability is in the vertical axis. Now what can we say about the probability distribution when the number of coin flips increases? Well, here's what happens with two coins, here's what happens with three coins, with four, and this is with ten. And as you can see this looks a lot like a Gaussian or a normal distribution. Notice that counting the number of heads in n coin flips is the same as adding n burden-only variables, where each variable takes the value 1 if you see heads and 0 if you see tails. This is an example of the central limit theorem that says that as you increase the number of variables you are adding the distribution of this sum. Your distribution looks more and more like the Gaussian distribution. Now recall that in here when we're tossing n coins with a probability of p, then the mean mu is np, which is n times the probability of heads, and the variance is np times 1 minus p, which is n times probability of heads times probability of tails. So for the case n equals 1 we have that the mean is n times p, which is 1 times 0.5, and that's 0.5, that's the case when we have a fair coin. And the variance is n times p times 1 minus p, which is 0.25. So let's calculate the mean and variance for all these examples. For the first one we have 0.5 and 0.25, for the second one it's 1 and 0.5, for the third one is 1.5 and 0.75, then 2 and 1, and for the tenth one is 5 and 2.5. Now what are these numbers? Well as n becomes sufficiently large we get a normal distribution where the mean is np and the variance is np times 1 minus p, just as expected.