Man, thank you! I thought my school project was really doomed before I saw this, but with your explanation, I finally found a way to make sense of my project data. Once more, thanks a lot!
Thank you. Please subscribe and ask your friends to subscribe - our goal is to get to 100,000 subscribers by the end of 2021. To get even more help, subscribe to the numericalmethodsguy channel ru-vid.com, and go to MathForCollege.com/nm and MathForCollege.com/ma for more resources. Follow the numerical methods blog at AutarKaw.org. You can also take a free massive open online course (MOOC) at canvas.instructure.com/enroll/KYGTJR Please share these links with your friends and fellow students through social media and email. Support the channel if you able to do so at ru-vid.com/store
Thank you. Please subscribe and ask your friends to subscribe - our goal is to get 100,000 subscribers by the end of 2021. To get even more help, subscribe to the numericalmethodsguy channel ru-vid.com, and go to MathForCollege.com/nm and MathForCollege.com/ma for more resources. Follow the numerical methods blog at AutarKaw.org. You can also take a free massive open online course (MOOC) at canvas.instructure.com/enroll/KYGTJR Please share these links with your friends and fellow students through social media and email. Support the channel if you able to do so at ru-vid.com/store
Thank you. Please subscribe and ask your friends to subscribe - our goal is to get to 100,000 subscribers by the end of 2021. To get even more help, subscribe to the numericalmethodsguy channel ru-vid.com, and go to MathForCollege.com/nm and MathForCollege.com/ma for more resources. Follow the numerical methods blog at AutarKaw.org. You can also take a free massive open online course (MOOC) at canvas.instructure.com/enroll/KYGTJR Please share these links with your friends and fellow students through social media and email. Support the channel if you able to do so at ru-vid.com/store
THANKS A LOT SIR!!!! I was choking at the derivative part but you made it clear. I have watched some other videos of yours. All are great. You earned a like and a subscriber. Really huge thanks sir. I'll watch other videos of yours also. You're a really good teacher
Thank you. To get even more help, subscribe to the numericalmethodsguy channel, and go to MathForCollege.com/nm and MathForCollege.com/ma for more resources and share the link with your friends through social media and email. Support the site by buying the textbooks at www.lulu.com/shop/search.ep?keyWords=autar+kaw&type= Follow my numerical methods blog at AutarKaw.org. You can also take a free online course at www.canvas.net/?query=numerical%20methods Best of Learning Autar Kaw AutarKaw.com
Thank you. Please subscribe and ask your friends to subscribe - our goal is to get to 100,000 subscribers by the end of 2021. To get even more help, subscribe to the numericalmethodsguy channel ru-vid.com, and go to MathForCollege.com/nm and MathForCollege.com/ma for more resources. Follow the numerical methods blog at AutarKaw.org. You can also take a free massive open online course (MOOC) at canvas.instructure.com/enroll/KYGTJR Please share these links with your friends and fellow students through social media and email. Support the channel if you able to do so at ru-vid.com/store
Seems you can set a0 = 0, find a1 very easily, then deduct a0 also easily. Reason is the angle a1 of the straight line doesn't not change if all Yi are decreased by any constant. Also in the end we can verify that a1 = cov(x,y)/var(x) = cov(x, y-a0)/var(x) for any a0. This will simply the computations.
Do not know about setting a0=0. If we are minimizing with respect to a0, we cannot assume it to be zero. Simpler derivation should not be used to sacrifice logical explanation.
To get even more help, subscribe to the numericalmethodsguy channel, and go to MathForCollege.com/nm and MathForCollege.com/ma for more resources and share the link with your friends through social media and email.
Thanks good explanation. Question: why does the partial derivative in this case yield a 'minimum'? How do we know it's not a maximum? Is it because: SSR = (Y - a.o - sum(a.i*x.i))^2 is the multivariable function we're trying to minimize and since it's squared we assume it's parabolic and opens upwards? Therefore the solution to the first partial derivative = 0 is a minimum?
I'm sorry I misspoke when I placed the a.i and x.i in the sum. I was getting confused with multiple regression. Is solving multiple regression the same process? Just taking partial derivative with respect to each unknown variable and then solving the resulting equations?
SSR=sum(y_i - a_o - a_1*x_i)^2 where _ stands for subscript. First partial derivatives put=0 ONLY yield a possible location of local minimum or maximum (do not know yet, if it a local minimum, local maximum or inflection point). It has to be followed by a second derivative test to see if it is the location of a local minimum or a local maximum. The second derivative test shows it is the location of local minimum (see link below). Since the first partial derivatives equal to zero equations have only one solution and SSR is a continuous function of a_0 and a_1, it has to be the also the location where the absolute minimum occurs too. To see the complete math behind it, go here: autarkaw.org/2012/09/03/prove-that-the-general-least-squares-model-gives-the-absolute-minimum-of-the-sum-of-the-squares-of-the-residuals/ or look at the derivation and appendix of mathforcollege.com/nm/mws/gen/06reg/mws_gen_reg_txt_straightline.pdf
One cannot conflate the two items. What is shown is the derivation of the linear regression model. The least-squares linear regression method is to find the best fit straight line for given data. The straight-line regression model is found by minimizing the sum of the square of the residuals. " Minimization using partial derivatives" is the concept used to find the constants of the model. math.libretexts.org/Courses/University_of_Maryland/MATH_241/03%3A_Differentiation_of_Functions_of_Several_Variables/3.08%3A_Maxima/Minima_Problems
@@numericalmethodsguy Good answer. Also @numericalmethodsguy, this derivation was fantastic, thanks much. I'm now using regression models in electrical engineering (power distribution demand forecast models) and wanted to take a bit of a dive to understand where the coefficients for linear regression came from.
Thank you. Go to mathforcollege.com/nm/mws/gen/06reg/mws_gen_reg_txt_straightline.pdf to see how the second derivative test is done as it is not shown in the video.
@@natashawanjiru1018 The question is ill-posed. First, the kind of model should be defined - is it y=a0+a1*x? Is it y=a*exp(b*x)? If it is just the straight line, go to nm.mathforcollege.com/mws/gen/06reg/mws_gen_reg_txt_straightline.pdf and look at the derivation as well as the appendix.
ok I did inverted derivation. I take your equation for a0 and I make it equal to Crammer a0. So equation is true, but how did you hit on this idea ? :)
@@nD-ci7uw If you look at the equations, you already got a1 using Crammer's rule. You will get a similar looking expression to a1 for a0 by using Crammers rule. But how I get the expression for a0 is just by using equation (1) without Crammer's rule, that is n*a0+sum(xi)*a1=sum(yi), and writing a0 in terms of a1. Also, sum(xi)/n=xbar and sum(yi)/n=ybar.
One cannot conflate the two items. What is shown is derivation of linear regression model. The gradient descent method is to find the local minimum of any differentiable function. The least-squares linear regression method is to find the best fit straight line for given data. The straight-line regression model is found by minimizing the sum of the square of the residuals. The gradient descent method surely can be used to find the minimum of the square of the residuals.
Multiply equation (1) by sum of xi, and equation (2) by n. Subtract and you will get rid of a0 unknown. You will get the equation for a1. To find a0, simply use equation (1) and write it in terms of a1, sum of xi and sum of yi. You have already found a1. You can also look at the matrix form, and use Crammer's rule. See equation 9.8.5 and 9.8.6 of math.libretexts.org/Bookshelves/Precalculus/Precalculus_(OpenStax)/09%3A_Systems_of_Equations_and_Inequalities/9.08%3A_Solving_Systems_with_Cramer's_Rule
We cannot reduce each residual. If we reduce one, another will increase or decrease. When you have many points, it is hard to do that. So we as a next step say - let us add the residuals and add them up. Then make the sum as small as possible. We find that it is not a good criterion. The sum of the absolute residuals is also not a good criterion. Both these methods result in non-unique straight lines. Minimizing the sum of the squares of the residuals works. It gives a unique straight line as well.
If you look at the equations, you already got a1 using Cramer's rule www.chilimath.com/lessons/advanced-algebra/cramers-rule-with-two-variables/ or by using Gaussian elimination symbolically. You will get a similar to a1 looking expression for a0 by using Cramers rule. But how I get the expression for a0 is just by using equation (1) without Cramer's rule, that is n*a0+sum(xi)*a1=sum(yi), and writing a0 in terms of a1. Also, sum(xi)/n=xbar and sum(yi)/n=ybar.
You can simply use Gaussian elimination symbolically to get the solution. Give it a try - it won't hurt. Or use the cofactor method as explained here. www.nabla.hr/MD-SysLinEquMatrics2.htm
Thank you. To get even more help, subscribe to the numericalmethodsguy channel, and go to MathForCollege.com/nm and MathForCollege.com/ma for more resources and share the link with your friends through social media and email. Support the site by buying the textbooks at www.lulu.com/shop/search.ep?keyWords=autar+kaw&type= Follow my numerical methods blog at AutarKaw.org. You can also take a free online course at www.canvas.net/?query=numerical%20methods
Most welcome. Thank you. Please subscribe and ask your friends to subscribe - our goal is to get 100,000 subscribers by the end of 2022. To get even more help, subscribe to the numericalmethodsguy channel ru-vid.com, and go to MathForCollege.com/nm and MathForCollege.com/ma for more resources. Follow the numerical methods blog at blog.AutarKaw.com. You can also take a free massive open online course (MOOC) on Numerical Methods at canvas.instructure.com/enroll/KYGTJR and on Introduction to Matrix Algebra at canvas.instructure.com/enroll/J4BFME. Please share these links with your friends and fellow students through social media and email. Support the channel if you able to do so at ru-vid.com/store
