This video was confusing to me at first but I think I'm understanding it more the second time watching it. I think the key for this video is to understand what each interval is guaranteeing is right 80% or more of the time. A confidence interval guarantees that, for each respective disease, 80% of the time or more you will correctly diagnose a first symptom for that disease.... i.e., if there are 100 people with the cold, you will correctly diagnose at least 80 of them as having a cold, and if there are 50 people with malaria, you will correctly diagnose at least 40 of them with malaria, etc.... With our groupings of diseases in confidence intervals, we are making sure that for at least 80% of every disease case out there, we are correctly picking the right disease/group of diseases for that person 80% of the time..... This is NOT to say that, for 80% of people who's first symptom is fever, we are picking the right disease. That is a credible interval. A credible interval says that for 80% or more of people who's first symptom is fever, we are correctly telling them what possible diseases they might have. This, conversely, does not guarantee that any specific disease get's 80% of its true cases diagnosed correctly. Imagine you run an NGO trying to diagnose diseases. You could say that the utmost important goal is to catch cases of Malaria... and you want to make sure that 80% of people or more who have malaria are actually getting diagnosed with malaria. Then you would use a confidence interval and say that if you have a fever or joint pain, we will say you might have malaria. Doing this would guarantee us to catch at least 80% (88% exactly) cases of malaria! If you have a fever, we would tell you it could also be flu or malaria, and joint pain could be a cold or food poisoning or malaria, but what's important is that by using this confidence interval, we would be telling 88% of patients who do truly have malaria that they possibly have it. Imagine a different scenario, where as leader of an NGO, you care deeply about correctly diagnosing patients with headaches... I don't know what but let's just roll with it. In this case, you would want some interval where given the knowledge of someone's first symptom (conditioned on the data), you would correctly diagnose that person's disease at least 80% of the time. This is fundamentally different! Using credible intervals, you are guaranteeing that, for each initial symptom someone tells you about, you correctly diagnose those symptoms 80% of the time. As a result, you may not end up telling 80% of people with malaria (or any other disease) that they could possibly have that disease.... the 80% certainty of the interval is referring to you correctly diagnosing 80% of each patient conditioned on them telling you their first symptom. In the case of people telling you their first symptom was a headache, you would say it's either a cold or malaria, and you would be right 80% of the time! Notice that with the confidence intervals, you only told people with a headache they had a cold. In this case, you would only correctly tell headache patients their possible diseases 68% of the time.... much below 80%. Your intervals are not concerned specifically with people with headaches, however.... you are guaranteeing that every disease, unconditioned on the data, has 80% coverage or more. Hope this helped someone - please correct me if I'm wrong
I would like to thank Ben for the excellent examples and explanations presented in the video, particularly the concise summary at the end. Below I try to add three points: 1. The frequentist school assumes the true value of which disease is contracted to be fixed, resulting in probabilities that can only be vertically summed to build a confidence interval. On the other hand, the Bayesian school models the true value to be a probability distribution, allowing for horizontal summation when building a credible interval. Here, we assume that the prior distribution (before reporting symptoms) of various diseases is equal, all 25% (see 20:14). 2. If colds are the only prevalent disease in the area and all other diseases are rare, the 80% credible interval for any observed symptom will be {cold}. In this situation, there will be a significant difference between the confidence interval and the credible interval, which echoes what is said in the video at 14:46. 3. Choosing flu over cold in the joint pain row when establishing a credible interval solely based on the criterion of "slightly exceeding 80%" is flawed. A more accurate guess that meets the criterion of "at least 80%" should be prioritized instead. The criterion used in the video seems unreasonable, considering the number of diseases being included remains three. Once again, thank you, Ben, for the valuable insights and explanations provided in the video.
I am a little confused. People tell me that Bayesian statistics is great because you don't have to choose an arbitrary cut-off value for the p-value. But in this case you still have to decide if you are interested in an 80 %, 95 % etc. credible interval.
Maybe it should be made more clear that, for the construction of the credible intervals, a uniform prior was used and that one could also choose different priors. Apart from that, nice video!
confidence interval: given a patient has certain disease, the doctor correctly diagnosed the disease 80% of the time. credible interval: given a patient diagnosed with the disease, the doctor correctly diagnosed the disease 80% of the time.. In a continuous example, frequentist assume the parameter is fixed, and 80% of the time, the confidence interval will capture the parameter; Bayesian assume the parameter is random, and 80% of the time, the parameter will fall in the credible interval. Frequentist assume there is only one true value of the parameter, the confidence interval they constructed based on the observed data are random. i.e. one true parameter, many confidence intervals. So 80% of the intervals contains the true value, or 20% of the time, the true value is outside the interval. Bayesian assume the parameter is random and has a distribution, the credible interval they constructed based on the data is fixed. i.e. parameter has many values with a probability density, but only one credible interval constructed based on the observed data. So 80% of the time, the parameter is inside the interval, or 20% of the time it's outside. Go back to the original example, the main difference is what the "80%" represents. Both the numerator is correct diagnosis. In confidence interval, 80% of the patients who have that disease get diagnosed. The denominator is total of people who has that disease. In credible interval, 80% of the patients who are diagnosed actually have that disease. The denominator is total of people who are diagnosed with that disease.
I *think* this is just a small error. There is another similar example on stackexchange, where the author consistently chooses the values with the highest probabilities to form the credible intervals: stats.stackexchange.com/questions/2272/whats-the-difference-between-a-confidence-interval-and-a-credible-interval
This part of the course was very tough particularly the last part of the lecture where you were comparing out the two intervals. I did not feel comfortable when you were adding up columns and rows to find out the summation for the interval. When you add up the rows and columns, in some cases, those numbers which were only included in one of the intervals were considered in your summation though in some other cases, you did not consider these data in the summation process. I did not find it out what can be the reason?! Best regards Thank your for your lecture series.
I think that it is quite useful for diagnosing a patient by using credible interval rather than confidence interval. because patients don't predict their disease but know their symptoms. Am I thinking right?
Just checking my intuition here, but are these concepts roughly analogous to those of precision and recall? If so which way round is it (i.e. is precision related to confidence interval or credible interval?)
In this example, i think we have to do a strong assumption. that is, patients have only one symptom. If patients suffer multiple symptoms, this example doesn't make sense. Am I thinking right?
@Ben Lambert, I don't understand how this example is supposed to make sense. Assuming that a patient could have multiple symptoms but only one of the diseases then shouldn't the columns NOT sum to 100%? Consider a disease where 80% of the patients have fever and 80% of the patients also have a headache.
You missed the fact that the variable is FIRST symptom. A disease cannot cause 80% of patients to have first symptom==fever and 80% first symptom==headache. This is of course a simplified problem, but the constraint that the probabilities add up to 100% makes sense. The categories are mutually exclusive and exhaustive.
I love most of the videos in this series, but this is asinine. It would have made way more sense to explain confidence vs credible intervals for continuous posteriors instead of some abstract discrete scenario
