Corrections: 8:53 I meant to say "the remaining 198.7 people without the mutated gene". 14:27 I correctly wrote "If the tests worked as expected, 5% should have p-values less than 0.05". That is correct. However, when recording the voiceover, I said "5% should have p-values less than 0.5", which is not correct. NOTE: In statistics, machine learning and computer programming, the default base for the log() function is 'e'. Thus, throughout this video I use the natural logarithm, or log base 'e', to do the calculations. Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/
@@MM-jm1il I believe it is because the derivative of ln(x) is super simple, just 1/x. Likewise, the derivative of the exponential function, e^x, is e^x.
It is amazing how many math students who are advanced in math and algebra lack very basic foundation in statistics. Your materials are highly valuable. It’s like the Neanderthals discovering fire.
I've been excelling at math without understanding such fundamentals of statistics which I've been silently ashamed of. Your materials have significantly improved my understanding. Hats off to you sir. Thank you.
As a full-time machine learning engineer, I must say I love the animations and your clear concise explanations! Your videos really helped me understand the fundamentals to succeed in my field. Thank you
I think this is a good piece of trivia! I'm not 100% it is correct - I don't know off the top of my head when the first small bam showed up, but this could be it. :)
@@statquestthis is the first appearance of “small bam” to the best of my knowledge - I watched the videos from the oldest forward in the past month (😂) and I think I haven’t seen it’s appearance anywhere before this one. Now I feel like an 11 year old groupie girl 🤪
Brilliant stuff Josh.... I keeep coming back to your video again and again... only to find that there is more and more information which I missed to grasp earlier .. Loving Statquesting.
I wrote an R script to generate the histogram alluded to by Josh at 10:00. For people following along in R this might be of use, so I figured I'd share it. Thank you for your work Dr. Starmer :bow: # StatQuest: Odds Ratios and Log(Odds Ratios), Clearly Explained!!! Josh Starmer # Steps to reproduce the histogram for Wald's Test in R # Author: Evan Rushton # Date: 09/22/18 logOdds
This is so helpful, I have been struggling about log(odd ratio) for a while, the meaning of log part confuses me forever, just never get why log is necessary for the calculation, now I finally understand, thank you so much
Great video! I needed this refresher. Very clear and concise! A point of clarification: at approximately 14:18 the voice over says that "...5% should have p-values of less than 0.5." I believe you meant "less than 0.05" as the slide shows. I wanted to clarify for listeners.
Odds of something is itself a ratio (probability of it happening/probability of it not happening) The odds are usually odds of x given y (in this case the odds of getting cancer given that you do have a gene) Log odds ratio is a ratio of ratios (ratio of the log odds) but the given condition changes. This tells us if the “given” condition has an effect on the odds of x happening. Example: 3:40 Log odds ratio = (log odds of getting cancer | you have the gene) / (log odds of getting cancer | you don’t have the gene) This can tell us the effect of the given condition (given you have the gene vs given you don’t have the gene) on the odds of x happening (cancer vs no cancer)
would be great to have a video differentiating odds ratios from relative risk from hazard ratios. in terms of interpretation and when one is used instead of the other
I tried adding a "card' link to the Fisher's Exact test (this is a pop-up link that shows up at the right time in the video), but maybe something went wrong with it. Regardless, I added a link to the StatQuest in the description, and here it is as well: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-udyAvvaMjfM.html Happy StatQuesting!!!
Just to clarify for myself regarding fisher’s exact test - to calculate the p value we are calculating the sum of probabilities of getting >= 23 cases of cancers if we are choosing 140 (= 23 + 117) people from the total 356 (= 23 + 117 + 6 + 210) people, is my understanding correct? I have only used fisher’s exact test in school when sample size is really small and all the cases can be laid out in a table by hand, and have never done in using a software package. I know scipy has a fisher_exact function but admit I haven’t read its docs or used it. Is there an R package that you would recommend using for doing it? Thanks as always 😊
One other point this applies to a case where the dependant variable is categorical because when the dependant variables is continuous we would use an F value to determine whether the model is significant
small mistake 8 minute 50s :expected values for Chi-square should be in the second row (for no mutated gene) 17.8 - cancer and 198.2 for no cancer. It's easy to check without computing cause they should add up to integers totals under each column but anyway great video and thank you for your work
How did you get 17.8? When I multiply the number of people that do not have the mutated gene, 216, by the probability that someone will have cancer, 0.08, I get 17.3.
Here is the code in Python to generate the histogram at 10:10. But one thing I would like to raise is that it seems not necessary to have random sample size as you can notice that it would be cancelled out when calculating the odd ratios. import numpy as np import matplotlib.pyplot as plt n = 1000000 sample_size = np.random.rand(n)*100+300 #Sample size ~ uniform disturbution between 300 and 400 p1 = np.random.rand(n) #p1 = proability with cancer ~ uniform distribution between 0 and 1 p2 = np.random.rand(n) #p2 = probability with gene mutation ~ uniform distribution between 0 and 1 odd_ratios = (sample_size*p1*p2 / sample_size*(1-p1)*p2) / (sample_size*p1*(1-p2) / sample_size*(1-p1)*(1-p2)) log_odd_ratios = np.log(odd_ratios) plt.hist(log_odd_ratios,bins=np.arange(-10,10.5,0.5))
13:20 However, this is traditionally done using a standard normal curve (i.e. a normal curve with mean = 0 and standard deviation = 1). I am a little bit confused about the word "traditionally". Are you saying that the log(odds ratio) is not necessarily distributed according to standard normal distribution?
Ah. The log(odds ratio) is normally distributed, but the standard deviation is not always 1. Traditionally, you convert the distribution for the log(odds ratio) to a standard normal curve (mean = 0, sd = 1) by dividing by the standard deviation (which you calculate using the method of your choice). I say "traditionally", because that's how you had to do solve for the p-value back in the day. You had a table of values for a standard normal curve in the back of some book, and whenever you needed a p-value for a normal curve, you converted your normal distribution to a standard one (by subtracting the mean and dividing by the sd) so you could reference the table in the back of the book. These days, however, a computer can calculate the p-value for any normal curve, so it's not really important to transform to a standard normal distribution any more (however, everyone still does it).
@@statquest - Yes, I did. The video was really good so I wanted to practice it in R. I couldn't see if anyone else had already posted something like this. I'm happy to delete/edit if needed.
Hi Josh, thanks for uploading those lectures! I really appreciate that! I’m still confused on why we use z values in logistic regression. Is this the reason that we use log odds which has mean of 0 and symmetrically distributed? Or more fundamentally(?), because our response variables are ‘binomial’? Thanks so much!
I'd suggest that we use z-value because wald test uses it to calculate whether the variable (predictor) in logistic regression is zero (null hypothesis) or not (alternative hypothesis). If the variable's Z-value (according to Wald Test) in logistic regression is less than some value (for example -+1.96 which equals to 2 std. deviations) we can drop that variable from the logistic regression because it's useless. Hope my answer helps you after 2 years :D
Hi Josh, I just wanna make sure that I'm correct. Now you assume there is no relationship between gene and cancer, and the log(odd ratio) equal to 0 means no relationship at all. Then based on what you observed from an experience, the log odd ratio is 1.93, and the p-value is something extremely small given that there is no relationship, which means it is really rare to happen when there is no relationship. But it does happen, so we need reject that there is no relationship, and accept that there is a relationship between gene and cancer. Am I correct? really confused.
The small p-value tells us that it would be vary rare for random chance to give us the observed log(odds ratio). Thus, we reject that the observed log(odds ratio) is due to random chance. Does that make sense?
This video talks about odds ratios and log(odds ratios), not log odds, so that might be confusing you right there. If you want to learn about log(odds), see this StatQuest instead: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-ARfXDSkQf1Y.html
Which base are you using for the logarithm? The traditional base for statistics and machine learning and most programming languages is 'e', or the natural logarithm. So log_base_e ((2/4) / (3/1)) = -1.79.
Thanks for the awesome video. Just had a question as to how are Odds Ratios different from Relative Risk and when to use Odds Ratios and when to use Relative Risk ?
Hi Josh, at time stamp 13.59 you say that p-value that mutated gene does not have a relationship with cancer is 0.00005 which means that there is no relationship between mutated gene and cancer and the result of log(odds ratio) is not statistically significant. Now if I understand correctly, at time stamp 12.53, it is written that log(odd ratio) is statistically significant which means that mutated gene and cancer has significant relationship. Can you pse tell what am I missing here.
The p-value for the hypothesis that there is no relationship between the mutated gene and cancer is 0.00005. This means that we reject the hypothesis that there is no relationship between the mutated gene and cancer. To learn more about what p-values mean and how they are interpreted, see: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-vemZtEM63GY.html and ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-JQc3yx0-Q9E.html
I was taught that when I see the equation log(x) to assume the base of the logarithm is 10. If I want to do logarithm with base e, I have always written ln(x). It took me a while to figure out why my math didn't work out the same as in the video.
It's unfortunate that the conventions for what log() means are not consistent. In statistics, machine learning and computer programming, the default base for the log() function is 'e'. Thus, throughout this video I use the natural logarithm, or log base 'e', to do the calculations.
@@statquest Thanks a lot. I noticed this when I used the log function in R. I am reviewing statistical principles as I write my prospectus for my Master's thesis and your videos are extremely useful.
Hello Josh, I had a confusion at 12:02 in the video. You said that the - "Wald's test typically uses the estimated standard deviation" ; but in reality we replaced the histogram with the normal curve having standard deviation of the observed values i.e 0.47. Hence, shouldn't it be - "Wald's test typically uses the observed standard deviation" instead of the expected and then we have a normal curve of STD=0.47 at 12:02. Based on my understanding the expected standard deviation will come from the earlier matrix of expected values which we calculated and here we have used the observed values matrix for getting a STD of 0.47. Let me know if I am missing something, but I am kind of lost in this section of the video and could use your help. Thanks again for an amazing video.
Thanks a lot for your wisdom!!! I have one question, as the odds ratio in the example was = 6.88 and the p-value was = 0.00005, we can say that there is a relationship between the mutated gene and cancer? or in other words, that having a mutated gene increases the odds of having cancer?
p-value wording is super awkward. Technically we would just say that we reject the hypothesis that there is no relationship. However, there is still a small probability that there could be no relationship.
I love your vids. I don’t know where to begin when I look at the number of quality videos on your channel. Is there an organization of the videos that would help?
Hi Josh! I have a question. At 11:25, you mentioned that "it is more common to estimate the standard deviation from the observe values"-- Does that mean the histogram generated from 10:00 to 11:25 is only for illustration purpose? i.e., it is not part of the Wald test calculations, am I right? =)
Dear Josh,where can I see this statquest"EnrichmentAnalysis using Fisher's Exact Test and theHypergeometric Distribution"? I have a litttle confusing about it
Hi again, at 13:59 does it means: there is "no" relationship between mutated gene and cancer due to small effect size or small log(odds-ratio), which is although significant. Because I got confused at the result as odds ratio is positive in favor of mutated gene.
The small p-value tells us to reject the hypothesis that there is no difference. To learn more about how to interpret p-values in general, see: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-vemZtEM63GY.html and ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-0oc49DyA3hU.html
Hi Josh. Just started watching your videos. Currently going through your 'Statistics Fundamentals' playlist. In this video, at the 7:50 mark, you mention 'So, if the gene is not associated with the 140 people with the mutated gene...', shouldn't the assumption be that 'So, if we assume that cancer is not associated with the 140 people with the mutated gene...'. That's the only reason you used the expected value using the 'estimated' population probability of having cancer (0.08) to calculate the 'expected value' of the number of people having both the mutated gene (140) and cancer.
When you talk about smaller values you mean in the absolute value? So that oddsratio = -1 is just as good a predictor of oddsratio = 1, while 0.1 is a bad predictor?
Hi Josh, very great video, really appreciate your videos very much. I have a question regarding the estimation of standard deviation in the wald test. You said in the video that it is more common to estimate the standard deviation from the observed values. But I don't understand why the standard deviation is calculated by the square root of the sum of 1 over each observed values. May I ask for an explanation on that as I am quite confused on this part in this video. Thank you very much and sorry for the inconveniences caused.
Your music gives me Fitz and the Tantrums vibes, which I am enjoying. A question: In this example you used a 2 x 2 contingency table and calculated the odds ratio for the positive response (cancer with a mutated gene). In my case, lets say *hypothetically* I am tracking the rate at which surgeons perform 2 procedures, A and B, over the past 10 years.
I can calculate the odds of procedure A and procedure B and then find the odds ratio. In order to say the odds are changing, what statistic would I use?
Fisher's exact test and the Chi-square test could work well with this data (they would tell you that the proportions between procedures A and B change).
Hi Josh, at 10:24 how did you draw the sample for histogram? You iterate thru 325, each time you draw random to determine which of the 4 cell it should be? (yesyes, yes no, no yes, no no) ? from your step instruction, it looks like I can only determine all cancer column and all mutated row, but I cannot derive the whole 4 cells for matrix. Many Thanks!
For each sample, if the first random number is less than 0.08, then the sample has cancer, otherwise it does not. If the second random number is less than 0.39, then it is mutated, otherwise it is not.
@@statquestMany thanks! I figured out too and confirmed with some python code. one more question 14:14, what you mean by the content from this point on? You generated 10000 log(ratio) data points based on observed expected probability (0.08 and 0.39). And it's not standard normal. How you apply test on it? Previously in your content Fisher's and Chi Square and even Wald's they all work on 2x2 matrix, how you manage them to apply on a normal distribution? How did you exactly calc p-value for example Chi and Wald's based on your generated 10K log(ratio)s? Quite confused here. Please advice at your convenience, really appreciate it!
@@kevinshao9148 I generated a single matrix of values then then I applied the Fisher, chi-square, Wald's tests to it to get the p-values for the three tests. I then repeated the process, 10,000 times, and then calculated the percentage of p-values < 0.05 for each test.
Hi Josh we need a clearly explained statquest on Fisher Test, chi square test and wald test . It will be triple BAM!!! if hypergeometric distribution is explained Can you also show to a demo to do these using python notebook
At 13:50 of the video, the p-value that mutated genes does not have relationship with cancer is 0.00005. So does it mean that mutated genes has a strong association with cancer then? P.S: It would be great that each time when you give an example, you will have a conclusion to it, so it wouldn't be confusing. Great video!
You are correct - I should have had a more obvious conclusion to this example. Often I do, but I forgot in this case. The small p-value says the association isn't just due to chance or noise. How strong that relationship is, however, is determined not by the p-value, but by the log(odds ratio). In other words, if we had a small log(odds ratio) and small p-value, we would have a significant, but weak association. If you have a large log(odds ratio) and a small p-value, you have a significant and strong association.
To calculate the p-value, we estimate the mean and standard deviation of a normal distribution and then we the area under that curve to calculate the p-value.
How do you add multiple odds ratios? Let’s say a patient has multiple risk factors for developing schizophrenia, each risk factor with a 2 fold possibility. (Ie having an older father, smoking cannabis, birth complications). Would you add them all together or multiple them?
Unfortunately neither of those options are good (see stats.stackexchange.com/questions/12294/can-individual-odds-ratios-be-added-to-get-one-pooled-odds-ratio-to-compare-to-a ) If you have multiple factors, I think your best option is to fit a logistic regression model to your data. For details, see: ru-vid.com/group/PLblh5JKOoLUKxzEP5HA2d-Li7IJkHfXSe
Hi Josh - Many thanks for your videos. At 10:46 - This gives the matrix that did not depend on the relation between mutated gene and cancer. If there is no relation, matrix itself cannot be formed rite ? Because as per my understanding - in a matrix, the margin totals proportion cannot vary at any cost rite ? I have one sample where I'm stuck to form a matrix - 1. Sample size - 366 2. Random number for cancer b/w 0 to 1 - 0.19 3. Random number for mutated gene b/w 0 to 1 - 0.14 Also will log() result in negative number ? I could see only positive number output when I apply log(). Please clarify
When we make a matrix that is only dependent on the over all proportion of people with cancer and the overall proportion of people with the mutated gene, and not the known proportion of people with cancer AND the gene, then the matrix will not depend on the relation between the gene and cancer. Also, as you can see on the x-axis in the histogram, the log can give us negative numbers.
@@statquest thanks Josh...understood.. so matrix margin sum for random numbers should add to 325 rite both row wise and column wise instead of all 4 cells adding up to 325 or any number b/w 300 and 400 ?
@@KishoreKumar-fv2cx Because we are randomly deciding how many samples have cancer, the column and row totals will always be different. However, the total for the entire matrix will the number between 300 and 400 that you came up with at the start.
I'm a bit confused, in another video (in Spanish) they use the wald test being: OR-value of h0 i.e. 1 / se(OR), giving this 12.44 and a very low p value
People use odds all the time, as is (for example, most sports betting is done in terms of the odds). But if you only want to use them for odds ratios, that's ok.
I just have a question(basically two) for you Josh, I went through all of the videos of yours and they are literally clearly explained and It's quite easy to apprehend for me. But the thing is that after completing all of the videos it's hard for me to remember everything. I noted everything but seems like I tend to forget a lot of things. Is that natural? Or in that case, what should I need to do? I really want to be a machine learning engineer and researcher as well and I'll apply for fall22 for Ph.D. in the USA. So, it looks overwhelming and at the same time enthralling to learn new stuff.
Jot your own notes do help. Even if you just remember the rough idea later, it is much easier to pick up by reading the notes. Using some modern notes jotting software like Notion give you extra BAM.
I don't understand the standard deviation in the example. The log(odds ratio) stays the same regardless of the sample size, as the ratio stays the same, but the standard deviation does change with the sample size. Consider that you multiply all the above numbers by 100. 2300 cancer&mutant, 600 cancer&no_mutant, 11700 no_cancer&mutant, 21000 no_cancer&no_mutant. The odds ratio stays the same. (2300/11700) / (600/21000) = 6.88. However, the standard deviation goes from 0.47 to 0.047, which would indicated that we are 41 standard deviations away from the mean (=log(6.88)/0.047)?
That is correct. We calculating a standard error, which is a function of the sample size and the larger the sample size, the smaller the standard deviation. In other words, the larger the sample size, the more confidence we have that the ratio should be close to the population ratio.
@@statquest thank you for your response! Really amazing videos you have, you have a gift for explaining complicated topics! What would be the best way to calculate standard deviation of log(OR) distribution? SE * sqrt(sample size)?
Just a heads up Josh, The Fisher's Exact Test video is not in any of the playlists for your videos. I found it by googling it, but just in case folks don't think about that solution.
