Confidence Intervals, Clearly Explained!!! 

BAM!!! One day I will make certificates. :)

@@statquest can I make one for you 👻

Oh! He will get your BAM certified

Thanks and good luck with Data Science! :)

how is it going man

@@Dupamine I am still just in the beginning, but I have just started in my first analyst job

@@joarvat hey I'm 20 rn and learning all bout statistics for data science. Is it worth it?

@@applepeel1662 I have got my first analyst job, and then I decided to go through the Data Science track with dataquest.io It's a great program.

bam!

bam! :)

Thank you! :)

Good luck with your interviews! Let me know how they go.

Hooray! :)

I'm glad I can help!

Nice! You're making great progress! :)

me too! this stuff is gold

@@samuelkellerhals5942 agree, it is good!

@@statquest double BAM!!

Thank you very much! :)

Good luck finishing your PhD! You can do it!!! :)

Wow, thank you!

Wow!!! Thank you very much!!! :)

That _was_ good, very clearly related. The introduction of the term 't-test' threw me however.

Awesome! Thank you very much! :)

Thank you! :)

bam!

Thank you! I'm glad I could save the world! I thought only Spider-Man could do that. ;)

Glad it was helpful!

Glad it was helpful!

BAM! :)

Good luck!!

Thanks!

Bam!

Thanks! :)

Glad you liked it!

Bam! :)

Thanks! :)

Thanks!

bam!

Glad it was helpful!

Thank you!

bam!

Go for it! :)

Same thing occurred to me!! If in a 95% confidence interval, the remaining 5% do not cover means right? If so, then how come its p-value is significantly different?

By definition p-value denotes a probability of (something other which is equally rare + something rarer than null hypothesis) happening. Here the null hypothesis is that the means for both male and female mice are from same population. But we already know that 95% confidence intervals of both male and female means don't coincide. So there is only one possibility left that less than 5% cases will have the possibility of their means coinciding. Which is why p-value is

:)

No problem 😊!

Thank you! :)

Thanks!

Thank you! :)

Thanks!

It's the same. However, we are arriving at the confidence interval differently and we need to make sure we don't confuse a bootstrapped mean for a population mean. The interval that contains 95% of the bootstrapped means is a 95% CI, and thus, if we repeated the process a bunch of times, 95% of the intervals calculated that way will contain the population mean.

Thank you! :)

Glad it was helpful!

You are welcome!

Yes!

Thanks!

For details on how I create my videos, see: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-crLXJG-EAhk.html

Thanks!

95% confidence intervals reflect the SEM, rather than the standard deviation of the raw data. For more details, see: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-A82brFpdr9g.html

Thanks! :)

Thanks!

I'm not sure how to do that with confidence intervals, but we could estimate it from the data by randomly selecting a female mouse and a male mouse and measuring the difference in mass. Do that a lot of times (if the dataset is relatively small - do it for every permutation of pairs - if larger, just do a lot of random sampling) and then plot a histogram of those differences and use the histogram to calculate the probabilities of getting differences between y and z.

You're welcome! :)

Thank you very much!

Thanks!

95% confidence intervals are not 1 or two tailed p-values, they are intervals. 95% of them will cover the true mean.

PI estimates range of RVs or some statistics of RVs. Its limits are not random (are not drawn from samples). For example for standard norm distribution ~99% PI for RV = mu+-3*sigma. And we must _know_ mu and sigma. On the opposite CI estimates range of not random but exact value - the population parameter (mu for example) though we don't know it's exact value. Its limits are drawn every time from different samples so they are random.

I think so too!

Thanks!

Thanks! :)

Thanks!

It depends on how you calculate it. If you are using bootstrapping, then your method is correct. If you are using a formula to approximate bootstrapping (so you are not using bootstrapping), then you have to appeal to the t-distribution (instead of the standard error). This is because for small sample sizes, the t-distribution is a little wider than a normal distribution, and that compensates for the fact that a small sample size means we have very limited knowledge of what is going on.

@@statquest you're a god sent Josh. Thank you 😄

Any interval that covers 95% of the bootstrapped means qualifies, but usually you select the one that is centered on the original mean.

Thanks!

For details on how I create the images, see: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-crLXJG-EAhk.html

There are lots of formulas for calculating confidence intervals. Conceptually, the easiest one to remember is bootstrapping, however there are lots of other formulas you can use. For details, see: www.statisticshowto.com/probability-and-statistics/confidence-interval/

@@statquest Thanks! I will check that out. Cheers

i think i've worked it out, i wasnt adjusting the margin of error in accordance with the sample size ie changing the t/z value, that is when you sample is really small the confidence interval becomes massive to account for that. Either way i would still like to hear your answer if you have time, thanks.

:)

I hope to do that in the spring.

The central limit theorem makes it possible to create confidence intervals for the estimate of the mean, but only the mean. In contrast, bootstrapping allows us to create confidence intervals for any statistic we want.

@@statquest Thanks! It makes so much sense

It suggests that the sample may come from a different population than the one you think you are collecting it from.

I'm not sure I understand your question. Are you asking about the distribution of the samples? (like, are you asking about the whether or not the data come from a normal distribution?)

Because of the central limit theorem, all means are normally distributed, so there is a closed form equation for all confidence intervals based on that and you don't need bootstrapping. In other words, you can calculate the CI with a single set of measurements. However, I believe the concept is easier to understand with bootstrapping.

@@statquest I see, thanks!

A p-value < 0.05, means, in general terms, that the result is probably not due to random chance. Thus, when we do linear regression, a small p-value tells us the the relationship between the independent and dependent variable is probably not due to random chance.

@@statquest Thanks Josh 😊😊

The bootstrap sample is always the same size as the original sample.

Thanks!

Keep in mind that the 95% confidence interval is just the interval that covers 95% of the means created with bootstrapping. And I believe that confidence levels just refers to the "95%" or "99%" - 95 and 99 are the levels that can then be converted into intervals.

@@statquest yes that's it. Thank you.

You might need to learn about hypothesis testing to understand the value of the confidence interval. Here's the link: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-0oc49DyA3hU.html

Thanks! :)

When you use bootstrapping, the 95% CI is any line that covers 95% of the bootstrapped means. So, take your pick! However, usually people center it over the estimated mean.

Traditionally, we center the 95% CI over 95% of the means, but you don't have to do it that way. You just need to cover 95% of the means.

3Blue1Brown has an excellent series on calculus. I also believe Khan academy has some good stuff.

Sure. The point, however, is that 0.05 is the usual threshold for making a decision about the hypothesis. So as long as we are < 0.05, we will reject the hypothesis.

The technical definition of a 95% confidence interval is that if we repeat the process a lot of times, calculating 95% CIs each time, 95% of the CIs we calculate will cover the true (population) mean. So, sure, sometimes we get outliers, and our CI is bad, but that is expected about 5% of the time we calculate a 95% CI.

@@statquest Thanks for responding, Josh ! Does this mean that while calculating the 95% CI, we are assuming that our point estimate (sample mean) is always 1.96 SD away from the population mean(mean of the sampling distribution) ?

no

@@statquest Thanks for responding again :). I'm having a lil bit of tough time connecting all the dots, sorry for the long questions ! If we are given the pop SD and use z stats to calculate 95% CI for mu, we say z=xbar-mu/(sigma/sqrt(n)) where z=1.96, xbar is the sample mean or point estimate and sample SD can also be computed. Based on the definition of z score, does this not mean that xbar is 1.96sd away from mu ? In any case, What is the intuition behind using this formula. Thanks in advance !

:)

Confidence intervals always have the same interpretation. If we repeated the procedure to calculate the CI a bunch of times, 95% of them would overlap the population mean.

Unfortunately, the language surrounding confidence intervals is very tricky. A 95% Confidence Interval should be interpreted like this: "If I re-do the exact same experiment a lot of times and use the exact same method to calculate the 95% confidence interval, 95% of the intervals will cover the population mean". If you're not familiar with the concept of a "population mean", check out my StatQuest on Population Parameters: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-vikkiwjQqfU.html

@@statquest Thank you very much for your reply! It is very helpful. I think if the sample size is large enough(12 is not large enough), the sample mean(x-bar) could be treated as population mean(miu). If based on the sample statistics, the 95% confidence interval is 170g-->230g, indeed I can get the density curve and make this statement: the weight of a random female mouse will fall in this range, with 95% confidence. Is this correct?

@@xsli2876 Unfortunately, that statement is not correct. Again, the Confidence Interval only tells us that if we repeated the exact same experiment and used the same method to calculate the confidence intervals, 95% of the intervals would cover the population mean. In other words, the confidence interval tells us about the "mean" and not individual measurements. If you want to make a statement about an individual measurement, like "there is a 95% chance that the weight of a random female mouse will fall within this range", then you need look at the distribution (probably normal distribution), and find the range that 95% of the values fall in. That is the range you are interested in. In math terms, the 95% CI is usually the mean +/- 2 times the standard error. In contrast, for individual measurements, a region where 95% of the measurements fall is the mean +/- 2 times the standard deviation. If you want to learn more about the standard error vs the standard deviation, check out: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-A82brFpdr9g.html

@@statquest Thank you so much. I got it. For individual measurements, a region where 95% of the measurements fall is the population mean(miu) +/- 2 times the population standard deviation(sigma). In real life, we don't know miu or sigma. However, if we have one large sample, we may use the sample mean(x-bar) as the approximate population mean, the sample standard deviation(s) as the population standard deviation.

@@xsli2876 Yes that is correct. If the sample size is not very large, you can use a t-distribution (which has fatter tails than the normal distribution) to compensate for the uncertainty in your measurements of the mean and standard deviation. However, as your sample size increases, the t-distribution converges to the normal distribution and then then you can just use the sample mean and sample standard deviation.

Here's another way to look at it: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-Xz0x-8-cgaQ.html

In real world you almost never know the exact population mean. You can sample the original data say 100 times more than Josh, but I can always oppose - why not 1000 times more - that would be more precise. And so on. So there is no solution until you sample the whole population - but this is impossible. That why the goal is - not getting the most precise assessment of population mean (it's impossible) but getting to know the boundaries where it lies with some % of confidence. And this is _possible_.

The 95% CI tells us that if we repeated the process of collecting data and calculating the CI, 95% of the CIs we calculate will overlap the true, population mean. They don't really tell us about future sample means.

@@statquest Thank you Josh. I know similar statement is already made in the comments below, but I was not able to follow. So I again watched the 'Population Parameters' and Confidence interval videos and all makes sense.

@@manikdhingra1606 Great! :)

At 3:22 I say that when using bootstrapping a 95% CI is an interval that covers 95% of the (bootstrapped) means. Now, if we made a lot of 95% CIs using this method, then 95% of them would contain the population mean. For more details on bootstrapping, see: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-Xz0x-8-cgaQ.html

@@statquest Thank you for the quick reply. It really separates you from the rest.

Thank you very much! I'm glad the videos are helpful. :)

Surprisingly, it can be any bar that covers 95% of the means.

@@statquest ah I see, here we are talking about CI itself, not the CI we using in N-distribution calculated by "std". (even in distributions CI can shift and change) Thank you!

@@stevequan7306 That's exactly right. There are different ways to calculate the confidence interval, and you end up with different intervals, but as long as you are covering 95% of the means (or at least doing that in theory), then you have a 95% CI.

@@statquest Hi Josh, Hi Steve, I had the same question as Steve. But I don't understand the answer/explanation. Why can CI be any range? Shouldn't it be the shortest range that cover 95% of the mean values?

@@karannchew2534 Why should it be the shortest?

It depends on the data and what you want from it. There's no one specific set of rules that fits all datasets.

This is the bootstrap approach to the frequentist CI. Since we are using bootstrapping, the interval covers 95% of the bootstrapped means and is equivalent to an interval that, if we repeated the experiment a lot of times and calculated the CI the same way, 95% of those CIs would cover the true mean.

@@statquest ❤️❤️❤️❤️❤️❤️. Thank you very much