If anyone's wondering where the values in slide 1 come from, I think this is right: For one roll ... (probability of rolling a six) = 1/6 (probability of NOT rolling a six) = 5/6 (probability of rolling a six) + (probability of NOT rolling a six) = 1/6 + 5/6 = 1 For 'x' number of rolls ... (probability of rolling a six 'x' times) = (1/6)^x (probability of NOT rolling a six 'x' times) = (5/6)^x (probability of rolling at least one six in 'x' rolls) = 1 − (probability of NOT rolling a six 'x' times) = 1 − (5/6)^x
Yes, that's exactly right! I also explain this on page 10 of Introduction to Biostatistics: github.com/FransRodenburg/Biostatistics-Book-Series You can access it for free.
Nice explanation. I have a question about multiple testing. In my data, my main hypothesis is that I want to check the difference of glucose between 2 groups of patients. However, I would like to compare the other clinical characteristics as well: Age, Gender, ... In that case, does the FWER increase ? Should I apply Bonferroni correction ?
Good question, it depends on the research question. If I'm guessing yours right, a short answer is that not everything needs to be assessed with a hypothesis test. Is it actually important for the research question whether the two groups differ in clinical characteristics, or are you simply trying to show that they are similar enough to compare their glucose levels in a fair manner? In case of the latter, I wouldn't use hypothesis tests for the clinical variables. Instead, just include a table of demographic variables of both groups (e.g., with mean and standard deviation for age, and counts of each sex). Another option is to use a regression model, where you have one with, and one without a set of potentially confounding variables, like so (R code): model1
Thank you for the explanation! However may I ask, at 8:25, for Bonferroni correction, shouldn't we devide 0.002 by 3= 0.0006? Because we divide p value by number of comparison which is 3 in the case
The multiple testing correction should always result in a *less* significant result. The correction in this case is to multiply the p-values, or (equivalently) divide alpha by the number of tests. Shown at 8:25 are the p-values, so these should be *multiplied* by 3. Hope that helps!
@@Frans_Rodenburg Dear Frans Rodenburg, thank you for your reply! :) Does it mean that for Bonferroni correction we simply multiply the p value with the number of test, whereas for FDR we use the Benjamini Hochberg equation p=Pi(k/i)? Therefore for p=0.02, k=3 (total of 3 p values), i= 3 (take the largest joint rank for ties in p value) p= (0.02)(3/3)= 0.02 Is that the reason why p=0.02 remains the same value after FDR correction? Please correct me if I misunderstand the calculation.
Which statement is correct? • The problem with multiple testing only arises if at least 1 test is significant • The problem with multiple testing only arises if all tests are significant • The problem in multiple testing arises when at least two results are significant • The problem in multiple testing arises when at least three results are significant
If nothing is significant before correction, then multiple testing correction will cause you to go from zero significant tests to zero significant tests. Can you answer the question with that information?
The problem of multiple testing isn't related to any particular result. The problem arised the moment you decided to run more than one test regardless of the outcome.
Because 1 - (1 - alpha / k)^k is approximately equal to, or smaller than alpha. Both methods ensure a maximum FWER of alpha. Some people prefer to show uncorrected p-values in their paper. In that case, you can simply work with a stricter alpha, by dividing it by k. In that case you only reject the null for p-values lower than alpha/k. This will give you the same conclusion of significance as when you keep your original alpha, but multiply all p-values by k.
