Saturated Models and Deviance 

NOTE: In statistics, machine learning and most programming languages, the default base for the log() function is 'e'. In other words, when I write, "log()", I mean "natural log()", or "ln()". Thus, the log to the base 'e' of 2.717 = 1. ALSO NOTE: This video is about Saturated Models and Deviance as applied to Logistic Regression (and not ordinary linear regression). Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/

You know you've been watching too many StatQuest videos when you say "double bam" before he does.

dude wtf. why have I just discovered your channel? you have a gift to make things as simple as possible

I love the weird tunes you make during any sort of calculations.Besides needless to say your teaching is amazing!!!

10:17 BOOP BOOP BEEP BEEP BOOP BOOP BEEP BEEP BOOP BOOP BEEP BEEP HAHAHA had me dying

Here for the song :-) and because it was an unknown piece of jargon to me. Jargon clarified... BAM!

OMG HAHAHAHA You make statistics seem so easy and simple!!! I looooove this channel

I have been listening to your lectures as a review on my commute and I am mesmerized. I never made the connection between LRT and these methods.bThey are one in the same! just re-written differently. or where the negative or the product of 2 come from.

Another exciting Statquest! Thank you Josh

Why on Earth did I not have you giving this lecture in my course - I actually understood it! Thanks!

It's surprising to see t-test, linear model and anova roughly fit the framework of MLE, and likelihood ratio test. BAMMMMM

Hey Josh! Again, thanks for your awesome work as always! In this specific case, I got into more questions than answers, but after trying to find answers on my own I assume it's a conscious decision to not overly complicate things. Namely, you say that we "-2" in the log likelihood ratio is what makes it be chi-squared distributed. The origin of this seems to be the fairly complicate derivation in Wilks (1937), so I assume there is no quick intuitive way to answer why is this chi-squared distributed (i.e. in 13:42, which is the standard normal variable that's being squared?). In the same line, about why are we working with the log of the likelihoods, is it because the saturated likelihood is obtained by maximization and thus it is easier to work with the logs? If that's the case, I'm not clear on what dictates which densities are used for the likelihood computation. As you say, you used normal distribution here for illustration purposes, but with regards to the actual computation of the likelihoods, what distributions are used by the glm function for example? In logistic regression I would first say that it's a binomial but that's only conditional on the x_is and would demand one parameter per observation. And last but not least, the most simple question: why are Deviances and Likelihoods required in GLM in contrast to straight Sum of Squares and Residuals? Again, thanks a lot for your time!

I loved your video, super no-brainer examples, thank you.

The best RU-vid channel ever existed.

I cannot like this video enough.

How did you calculate the likelihood of univariate data?

10:17 12:23 Peak number-plugging-in sound effect! 👌

wow this opening song is great!

That's right Josh- here just for the song 😂 Seriously - your videos are super helpful!

Love your Bams and oan ouannn as much as your teaching. :)

Hi Josh, amazing videos. You are a godsend for visual learners and showing the concepts behind the math. I am a little confused about why we calculate the p-value for the R^2 the way we do. We have: Null Deviance ~ ChiSq(df = num params sat - null) Residual Deviance ~ ChiSq(df = num params sat - prop) and we use these distributions to calculate the p-values for the null and residual deviance. We also have: LL R^2 = (LL_null - LL_prop) / (LL_null - LL_sat) but we calculate the p-value via: null deviance - residual deviance ~ ChiSq(df = num param prop - null model) Can it be shown that (LL_null - LL_prop) / (LL_null - LL_sat) = null deviance - residual deviance and therefore R^2 ~ ChiSq(df = num param prop - null model) or am I missing something?

sir U r great. U r Ultimate. U r dangerous. lots of love from a Data Scientist 🥰🥰🥰

@2:34 How did you calculate the LL of the data?

Hi Josh. I don't understand how you got the likelihood values. Mathematically likelihood is just a probability.

You deserve a big bam. BAM!!!

Do you have video that about multinomial logistic regression . your video is the best in stat thanks a lot

I want to earn my PhD in BAM!!! Is there a thesis option?

BAM, BAM, what a great teacher

You nailed it, much appreciated

When you are calculating the likelihood of each of the null (2:39). proposed (3:18), and saturated (3:57) models, where do those raw values you are multiplying together come from? To me, it looks a bit like you have drawn several probability density functions (PDF) and looking at the value of a given PDF for each of the points. However, I know my interpretation is wrong because the values are often greater than 1, thus I am confused as to what those values are?

For my future revision. To work out R², we need Log Likelihood of: .Saturated Model (best possible) .Null Model (most lousy) .Proposed Model (fitting) To work out the p value, we need the Deviance. Residual Deviance = Deviance of Saturated (best possible) Model vs Proposed (fitted) Model = 2 * (LL_saturated_model - LL_proposed_model) = the value for chi stats to work out the p value Is it from Log［prob saturated model / prob proposed model ］² ? Null Deviance = Deviance of Saturated (best possible) Model vs Null (lousiest) Model = 2 * (LL_saturated_model - LL_null_model) Null Deviance - Residual Deviance = Yet Another Deviance = The Chi Sq value to work out significance between Null and Proposed model. BAM!!!

15:26 Why does multiplying the log likelihood's by 2 give a chi-sqaured distribution with degrees of freedom equal to the difference in the number of parameters of the proposed model and the null model.

Keep it up, Josh Star-scat-mer.

Really nice video, like all others! However, I'm still unsure about how we can disregard the saturated model in the logistic regression to find our R^2 when that could mean that our R^2 will be larger than 1. For example, that would happen in this example that you showed. I'm probably missing something, could you please explain?

Great explanation. Thank you sir!

learned a lot from your lessons, thx so much. BAM!!

(2:50) can anyone explain why the probabilities 1.5, 2.5 are greater than 1 ?

2:37 How do you come up with the numbers for the likelihood of the data?

17:06 Can someone explain why the likelihoods will be between 0 and 1? I thought likelihoods are greater than 0 but can be positive infinite.

@Josh Starmer. First of all thanks a lot. secondly, I am confused how you got your Saturated model likelihood. You multiplied 3.3 6 times. I understand 6 is no. of parameters. What is 3.3 here?

This video talks a lot about using chi-sq distribution, I have a general question about how distributions come about, because it seems people can design infinitely many different experiments to generate infinitely many distributions. Is chi-sq invented to measure p-value of R2 of logistic regression? If no, is it pure coincidence that this p-value can be seen from chi-sq, or the inventors of the 2(null dev-res dev) formula saw some useful properties of chi-sq and decided to use chi-sq, and invented the null dev - res dev formula? Is this formula tied to chi-sq only, or we can put the 21.34 value at 12:40 on the x axis of another dist that is not chi-sq too? Similarly, how/why was F distribution chosen to provide p-value of R2 for linear regression models?

I know it's weird but I was waiting for the jackhammer's sound XD

At 14:31, we have the p-value =0.002 which means that the proposed and the null model are statistically different but can't we infer that the result is by chance (connecting it with the root definition of p-value)

Is there any pre-requisite video before watching this? FInding it little complicated to understand.

@9:37 Hi Josh, can you explain why the Residual Deviance is defined as this form and what is the connection to chi-square test, or provide a reference link? Thanks.

(12:05) why the null deviance changed from "saturated model - ..." to "proposed model - ..." after greying out

Hey Josh, thanks for this video on saturated models and deviances! I'm wondering whether I'm missing some point in your video or whether there may be some slight confusion. On the slide at 14:22 it says that the p-value computed on the slide before for the difference between NullDeviance and ResidualDeviance (NullDev - ResDev) gives a p-value for the R-squared (R2). I can follow the argument up to the point that NullDev - ResDev has a chi-square distribution, since it is in fact a classical likelihood ratio, namely the same as 2*(LL(Prop model) - LL(Null model)). However, if we want to express the R2 in terms of deviances, R2 would be (NullDev - ResDev)/NullDev, i.e. we divide by NullDev. The distribution of this is not chi-square anymore and we would get a different p-value. Would you be able to clarify this?

How is the saturated model at 16:19 created? I thought lines can only go upwards from left to right, but there is a line going downwards from 1st blue to 3rd red dot. Also how are there 3 line segments? Are they created from 3 logistic regression models, with first 2 red dot + 1st blue dot creating 1st model, 1st blue dot + 3rd red dot creating 2nd model, 3rd red dot + last 3 blue dot creating 3rd model?

Thank you for this video! I have some questions after watching. How to create the curve in the proposed model and why are you choose this curve when below the 2-curve area to calculate the log-likelihood?

This one is difficult Do you plan to do other goodness-of-fit methods as well? I'm looking for a way to do counted R-square for conditional logistic regression in R

How do we know that mouse weight follows a normal distribution?

Is the Saturated Model always a model thats overfitted in machine learning lingus (16:30 in the vid)? If so, does it make sense to say we are looking for a proposed model that is close to the saturated model? OR is the term "close" the keyword, as we don't want an overfit, but just a very good model? Or are Overfit and Saturated Models something completely different, and not related to each other?

This got me confused. I've always thought of R^2 as a measure of how much better our model is at predicting the value of dependent variable given the values of features, but adding a saturated model to the equation makes it less intuitive for me. Of course, when we have the same amount of observations as dependent variables, we will maximize R^2, just because that's how math works (as they're in the same dimension you'll always be able to fit a geometric figure to all of them) but overfitting obviously is a malpractice. I don't get why we measure our proposed model's predictive ability by kinda comparing it to a saturated one (which is pretty useless).

4:11 How can the likelihood of the data given the distribution be larger than 1. I thought each specific likelihood was a probability (so

Thanks! The video was great as usual :)

Why do we use saturated model? Why do we use it as the basis for evaluating how good our model is? I can't comprehend the concept behind it! in the saturated model we consider a separate model for each point and obviously it results in overfitting doesn't it ??

3:20 The likelihood is greater than 1? Is it possible to have greater than 1 likelihood?

Can you explain what the log likelihood based R^2 actually represents? I know that R^2 in normal linear regression just represents the strength of correlation but what is it here?

Could someone explain why it is a chi-square distribution instead of a F distribution? I am confused because I saw there's a ratio of two chi-square distribution.

Question: when you calculate the likelihood based on two means at 03:20, how do you know which distribution to use? The arrows intersect both distributions, so for each data point we should multiply two likelihoods? Could you please clarify?

If I have a model contain four parameters with three variables as like concentration, times and their interaction, is the model with just theirs interaction as a variable and contain two parameters, is it a nested model?

Why is the LL at 4:00 3.3 for each curve?

why liklihood vaule can >1?? probability value >1?

How did you calculate log-likelihoods @16:10 and @16:33???

2:34 how do you get the likelihood of data? please explain.

Which models necessitate including the LL(saturated model) or equiv. to calculate R^2?

Oh My! I really love this man

How likelihood for the saturated model brings value-3.3 at time- 4:03

can somebody explain to me how the likelihood of data is calculated

Can you explain how you calculated the likelihood of the data please? If you are multiplying the probabilities then surely they must all be less than 1?

Hi Mr. handsome Josh. Is it possible to analysis for lottery pick 3. I have different theroy to lottery pick 3 and want to verify it.

R^2=0.45, means the proposed model does not fit the data! Is this right? (I think r-squared must be very small)

I do logistic regression in MAPLE 18. I want to learn another program to applied. I need an advice from you. How about R? I do not want to loss the time? please help me?

your explanation 6:20 is very confusing

did you really do that with your mouth??????

I have no clue what u talking about