A brief introduction to leverage and influence in simple linear regression. This video is about the basic concepts, and only briefly mentions numerical measures of leverage and influence near the end of the video.
I read a whole PDF of many pages from school, and still did not have a clue about this, but in only 7.13 minutes you clarified all of my questions. Thank you very much!
I really started to hate my schools educational system. All it takes is a 2 minutes of the 7 minute video for me the grasp the idea and here I am losing tons of hours staring at meaningless pdfs, shame on absolute zero efforts on teaching. I really appreciate your video! You saved my life and my interest to the statistics by posting this video 11 years ago.
Your videos are so clear and concise. I go back to them over and over to review and absorb statistical concepts. Your channel is the best on youtube for that. Thank you!
Once upon a time, I went to University to learn leverage and Influential points. And I became history then. This time it has perfectly stored in my brain. You are awesome.
Good visual explanation only you forget to mention that influence = leverage * discrepancy. Discrepancy is the distance of the Y value to the line. (Studentized residuals; Cook's distance) Anyway, keep up the good work!
Thanks for the feedback. I didn't forget to mention that -- I intentionally brought it up in only a loose way (describing it with a visual illustration), as at the point I cover this in an introductory statistics course students won't have been introduced to studentized residuals, Cook's distance, or measures of leverage. Formally describing Cook's distance, along with its expression as a function of the studentized residual and leverage, is far beyond the scope of this video. Cheers.
awesome video... 2 questions. 1) eliminating such high influential and leverage points can't this lead to overfitting problem ? 2. But if my managment want the estimates for those values (obserations) eliminated , what suggestions can I make ?
Every text book says that an outlier along X has more influence on the regression line. Hence such a point is called an Influential point. But the following example clearly shows that an outlier along Y has more influence than an outlier along X. Best fit line through (-1,-1), (0,0), (1,1) is y = x. Including an outlier along Y, the best fit line through (-1,-1),(0,0),(1,10) is y = 5.5x + 3. Including an outlier along X, the best fit line through (-1,-1),(0,0),(10,1) is y = .1487x - .446. Am I missing something? Thanks in advance for your help.
There's quite a bit to this, and it's tough to give a complete answer to this in a comment. First, points that are extreme in X have high *leverage*, but are not necessarily influential. If a high leverage point falls very close to the overall pattern, then it will not have much influence. If a high leverage point has a Y value that falls far from the overall pattern, then the point will be very influential. As I describe in the video, influence depends on both leverage and how extreme the Y value is. In the situation you describe, each of the changes result in a point that are extremely influential. If you plot it, you will see that (visually) the magnitude of their influence on the line is similar. However, if we calculate Cook's Distance (a commonly used measure of influence), point 3 has a *much* higher value in the scenario where X is also extreme. (In the original line, none of those points have any influence at all -- the regression line would be the same if any single point were removed.) One reason for this is that the variability of the estimators depends on the variance in X. (Very loosely, if the X's are spread farther apart, with greater variance, than it would be harder to pivot the line.) So, once the much lower variability of the estimators is taken into account, that point is much more influential.
This video is intended to be a light introduction to the topic, one appropriate for an introductory statistics class. The mention of the numeric measures of leverage and influence were intended to be just that, a mention. At some point, I'll try to get videos up explaining that part in greater detail.
