WEBVTT
Kind: captions
Language: en

00:00:00.090 --> 00:00:04.650
Maximum like and estimation is one of the most&nbsp;
commonly used estimation principles in research.&nbsp;

00:00:04.650 --> 00:00:10.950
Understanding this principle why it's not&nbsp;
exactly necessary for you to do your analysis,&nbsp;&nbsp;

00:00:10.950 --> 00:00:17.730
it's very useful because sometimes when you get to&nbsp;
more complicated models, the estimation can fail,&nbsp;&nbsp;

00:00:17.730 --> 00:00:22.530
and to able to troubleshoot and fix the&nbsp;
problem you have to understand what the&nbsp;&nbsp;

00:00:22.530 --> 00:00:27.750
computer is trying to do, when it attempts&nbsp;
maximum likelihood estimates. To understand&nbsp;&nbsp;

00:00:27.750 --> 00:00:31.260
maximum likelihood estimation we need&nbsp;
to first understand a couple of concepts&nbsp;&nbsp;

00:00:31.260 --> 00:00:36.990
from probability. The first two concepts are&nbsp;
probability density and cumulative probability.&nbsp;

00:00:36.990 --> 00:00:43.200
When we have the p-value the p-value quantifies&nbsp;
the probability the cumulative probability&nbsp;&nbsp;

00:00:43.200 --> 00:00:48.510
what is the area under the curve here.
And the area quantifies how likely or&nbsp;&nbsp;

00:00:48.510 --> 00:00:54.870
how probable a value of less than minus one&nbsp;
is from a normal distribution center that&nbsp;&nbsp;

00:00:54.870 --> 00:01:01.200
one at zero and a standard deviation of 1.
Then the probability density tells us how&nbsp;&nbsp;

00:01:01.200 --> 00:01:07.800
probable a value of minus 1 is compared&nbsp;
to any other value of the distribution.&nbsp;

00:01:07.800 --> 00:01:15.270
So our probability cumulative probability tells us&nbsp;
what is the probability of getting a value or more&nbsp;&nbsp;

00:01:15.270 --> 00:01:23.460
extreme from the distribution, and probability&nbsp;
density tells us what is the probability related&nbsp;&nbsp;

00:01:23.460 --> 00:01:30.150
to others. So this is not any particular any&nbsp;
probability of any particular value, because&nbsp;&nbsp;

00:01:30.150 --> 00:01:37.500
the normal distribution has unlimited range, so&nbsp;
getting an exact value is pretty much impossible&nbsp;&nbsp;

00:01:37.500 --> 00:01:43.050
because there are so many different alternatives.
But this tells us the relative probability&nbsp;&nbsp;

00:01:43.050 --> 00:01:47.160
between this value compared to for&nbsp;
example this value or or that value.&nbsp;

00:01:47.160 --> 00:01:54.060
And we are not lot more likely to get the&nbsp;
value of minus 1 than a value of minus 2&nbsp;&nbsp;

00:01:54.060 --> 00:01:59.070
and we are more likely the value of 0 at them&nbsp;
which is the mean, than the value of minus 1.&nbsp;

00:01:59.070 --> 00:02:04.830
We also need to understand the probability&nbsp;
of independent events. And the independent&nbsp;&nbsp;

00:02:04.830 --> 00:02:11.370
events probability are also highlights why we&nbsp;
need to assume independence of observations.&nbsp;

00:02:11.370 --> 00:02:18.210
The idea of in the and the events is that, if&nbsp;
you have two events we have two dice here so&nbsp;&nbsp;

00:02:18.210 --> 00:02:24.690
we have died one the first die receives values&nbsp;
from 1 2 to 6 and we have the second die here&nbsp;&nbsp;

00:02:24.690 --> 00:02:31.200
receives values from 1 to 6 as well.
And these dice are independent,&nbsp;&nbsp;

00:02:31.200 --> 00:02:36.360
so throwing the first die doesn't&nbsp;
affect the throw of the second die.&nbsp;

00:02:36.360 --> 00:02:41.940
And therefore all possible 36&nbsp;
combinations are equally likely&nbsp;&nbsp;

00:02:41.940 --> 00:02:48.510
to occur because these are fair dice.
So if we have a value of 1 from the first&nbsp;&nbsp;

00:02:48.510 --> 00:02:55.950
die that is 1 out of 6, and to get a value&nbsp;
of 1 from the second die that is 1 out of 6&nbsp;&nbsp;

00:02:55.950 --> 00:03:03.930
again and the total probability of two ones is 1&nbsp;
out of 6 times 1 out of 6 which is 1 out of 36,&nbsp;&nbsp;

00:03:03.930 --> 00:03:09.180
as this figure illustrates.
So the probability of two&nbsp;&nbsp;

00:03:09.180 --> 00:03:14.250
independent events, is the product of&nbsp;
the probabilities of both events here.&nbsp;

00:03:14.250 --> 00:03:19.080
And this is an important principle&nbsp;
in maximum likelihood estimation.&nbsp;

00:03:19.080 --> 00:03:25.980
Let's now go on and explain the actual&nbsp;
estimation principle. Assume that we&nbsp;&nbsp;

00:03:25.980 --> 00:03:33.240
have a normal distribution here, which has&nbsp;
a mean of zero and standard deviation of 1.&nbsp;

00:03:33.240 --> 00:03:42.120
To get values of 2 3 and 4 from from this&nbsp;
population, they probability density for 2&nbsp;&nbsp;

00:03:42.120 --> 00:03:51.180
is 0.05 for 3 is 0.04 and for four it's&nbsp;
0.001 and the cumulative probability&nbsp;&nbsp;

00:03:51.180 --> 00:04:00.000
density is the product of these two values.
So the this value here is the product of that&nbsp;&nbsp;

00:04:00.000 --> 00:04:06.960
probability density and that probability density&nbsp;
this value here is the product of that probably&nbsp;&nbsp;

00:04:06.960 --> 00:04:12.600
the density plus that probability density.
We multiply these probability densities&nbsp;&nbsp;

00:04:12.600 --> 00:04:16.590
together because these are independent&nbsp;
observations from the same distribution.&nbsp;

00:04:16.590 --> 00:04:22.980
So the total probability density is the product&nbsp;
of these individual probability densities.&nbsp;

00:04:22.980 --> 00:04:30.870
So getting values of 2 3 &amp; 4 this probably&nbsp;
density quantifies the likelihood of those&nbsp;&nbsp;

00:04:30.870 --> 00:04:35.310
three values compared to any other&nbsp;
three values that we could get.&nbsp;

00:04:35.310 --> 00:04:41.820
So these probability densities are very&nbsp;
small numbers because there's one numbers&nbsp;&nbsp;

00:04:41.820 --> 00:04:48.900
it is typically more convenient to work with&nbsp;
logarithms and the when we work with logarithms.&nbsp;

00:04:48.900 --> 00:04:58.740
We extend our the table by having a the log here.
So we have the logarithm and then we have our&nbsp;&nbsp;

00:04:58.740 --> 00:05:03.570
the cumulative logarithm which&nbsp;
is the sum of these logarithms.&nbsp;

00:05:03.570 --> 00:05:10.950
So there are why we take a sum of these&nbsp;
logarithms is that the logarithm of us&nbsp;&nbsp;

00:05:10.950 --> 00:05:17.010
of a product is the sum of the logarithms of&nbsp;
those two components that go into the product.&nbsp;

00:05:17.010 --> 00:05:24.330
We also call these now likelihoods because we are&nbsp;
trying to estimate there are the mean of this.&nbsp;

00:05:24.330 --> 00:05:29.100
We no longer know the mean&nbsp;
of this normal distribution,&nbsp;&nbsp;

00:05:29.100 --> 00:05:37.230
rather we try to estimate it from the data.
So assuming that the mean is zero,&nbsp;&nbsp;

00:05:37.230 --> 00:05:43.230
then the likelihood of the data is this&nbsp;
small number here and the log likelihood&nbsp;&nbsp;

00:05:43.230 --> 00:05:50.220
which the logarithm of the likelihood is 117.
How maximum likelihood estimation works is&nbsp;&nbsp;

00:05:50.220 --> 00:05:57.720
that we try different values for the mean.
For example now mean is 0, we try 0.01 and&nbsp;&nbsp;

00:05:57.720 --> 00:06:05.790
we check if the log likelihood becomes larger.
If it does, then we know that the mean is in&nbsp;&nbsp;

00:06:05.790 --> 00:06:11.760
that direction, the actual maximum likelihood&nbsp;
estimate of the population mean. So we move&nbsp;&nbsp;

00:06:11.760 --> 00:06:17.070
the normal distribution sideways a bit&nbsp;
and we move it to the right this time.&nbsp;

00:06:17.070 --> 00:06:22.980
We can see here that the the probability&nbsp;
densities of all these observations increased,&nbsp;&nbsp;

00:06:22.980 --> 00:06:31.680
because the distribution is now closer&nbsp;
to these observations. And also the log&nbsp;&nbsp;

00:06:31.680 --> 00:06:39.030
likelihood became larger. We can still increase&nbsp;
the log likelihood or the cumulative likelihood&nbsp;&nbsp;

00:06:39.030 --> 00:06:46.500
by moving the distribution right a bit more,&nbsp;
so we would to write a bit more and uh the log&nbsp;&nbsp;

00:06:46.500 --> 00:06:51.600
likelihood became large or closer to zero.
This is nearly always a negative number,&nbsp;&nbsp;

00:06:51.600 --> 00:06:55.980
so increasing a negative number means&nbsp;
that the number gets closer to zero.&nbsp;

00:06:55.980 --> 00:07:05.040
And we can still move it right a bit to increase&nbsp;
the likelihood even more, and this is the&nbsp;&nbsp;

00:07:05.040 --> 00:07:10.080
maximum likelihood estimate, this is the maximum&nbsp;
likelihood that we can get by shifting the mean.&nbsp;

00:07:10.080 --> 00:07:15.630
So having a normal distribution&nbsp;
with standard deviation of 1 the&nbsp;&nbsp;

00:07:15.630 --> 00:07:21.030
we can't get any larger likelihood values.
We when we set the mean at zero that is the&nbsp;&nbsp;

00:07:21.030 --> 00:07:27.030
maximum likelihood if we received it right a bit&nbsp;
or left a bit then the likelihood would decrease.&nbsp;

00:07:27.030 --> 00:07:32.820
And the value of mean here that&nbsp;
maximizes the likelihood is called&nbsp;&nbsp;

00:07:32.820 --> 00:07:37.380
the maximum likelihood estimate.
The maximum likelihood estimate&nbsp;&nbsp;

00:07:37.380 --> 00:07:42.270
is found by maximizing the likelihood&nbsp;
function or the log likelihood function.&nbsp;

00:07:42.270 --> 00:07:49.950
We can also express the likelihood as a function&nbsp;
of the mean. So here is the likelihood function.&nbsp;

00:07:49.950 --> 00:07:53.940
We can see that the likelihood of getting&nbsp;
those three observations pretty much zero&nbsp;&nbsp;

00:07:53.940 --> 00:07:59.010
here and it's pretty much zero here,&nbsp;
so the likelihood Peaks only when&nbsp;&nbsp;

00:07:59.010 --> 00:08:03.990
we were very close to the correct values.
When we have the log likelihood, it looks a&nbsp;&nbsp;

00:08:03.990 --> 00:08:10.740
lot nicer curve because it actually goes to some&nbsp;
direction instead of being flat even if we are&nbsp;&nbsp;

00:08:10.740 --> 00:08:17.460
further from the actual correct population value.
So that's one reason why we use logarithms&nbsp;&nbsp;

00:08:17.460 --> 00:08:21.900
for maximum likelihood estimates.
So in practice we set some starting value&nbsp;&nbsp;

00:08:21.900 --> 00:08:26.850
let's say that that's zero for the mean.
Then we look which direction does the&nbsp;&nbsp;

00:08:26.850 --> 00:08:28.980
likelihood go up.
We see that it goes&nbsp;&nbsp;

00:08:28.980 --> 00:08:35.190
up when we go right, then we go right a bit&nbsp;
we try here, we calculate likelihood here,&nbsp;&nbsp;

00:08:35.190 --> 00:08:40.350
we check which direction we should go, we&nbsp;
calculated here and, then we discover that if&nbsp;&nbsp;

00:08:40.350 --> 00:08:44.850
we go further the likelihood starts to decrease.
Then we declare that this is our maximum&nbsp;&nbsp;

00:08:44.850 --> 00:08:49.290
likelihood estimate. So that's the basic&nbsp;
principle of maximum likelihood estimation.&nbsp;

00:08:49.290 --> 00:08:55.560
Trying to estimate mean of normal distribution&nbsp;
with standard deviation of 1 from the&nbsp;&nbsp;

00:08:55.560 --> 00:09:01.860
observations. In other videos I will use this&nbsp;
same principle for more complicated examples