In the video, I calculate just the upper bound at 95% and get my interval from the median. Then I subtract that same interval to get my 5% lower bound. (This is a very simple approach, there are more sophisticated approaches that provide more nuanced intervals). (It's the 95 percentile "line" of my data). - Hope that makes sense, waving my arms as I try to explain, take a look at the plots and try the numbers yourself, that helped me a lot.
@@serviwebm4644 Yes, in this case, it is a 90% range where predictions are "normal", but intervals don't always have to be symmetrical - you could have just a prediction interval at 95% and only want to know when it exceeds that with no concern about what happens at 5%. (you are only worried about really high predictions). The 95% is not a range here, but a point at which I want the interval to start at.
As I understand it, crucially, he calculated ABSOLUTE error values. You might be used to seeing raw error values which could be negative or positive. If that were the case, then yes, you would pick the 2.5% lowest and 2.5% highest values to get a 95% prediction interval. But for absolute values (that is, we don't care whether it is negative or positive), then the absolute size of the error being above 95% is all we care about.
how does this show calibration and Conformal Prediction? Its basically usiing quantile to create an interval and add the range. Where have we used Conformal Predicton methods here? Second, you have a calibration dataset but you haven't calibrated the model yet. It is predicting values directly from the model without undergoing calibration
The video shows how to use conformal methods to get a prediction interval. This is a simple approach and more sophisticated approaches are available. Not sure how familiar you are with conformal methods, but here is a great resource: github.com/valeman/awesome-conformal-prediction It also links to a slack group where you can have more detailed discussion on conformal prediction.
@@Rajistics again, what I am asking is this is just a quantile intervals. We haven't done calibration of probability model and don't think there is conformal prediction interval using VENN-ABER etc
The key here is we have a portion of the dataset we are using for calibration. That helps us calibrate the prediction intervals. There is nothing here about calibrating the model.
@@Rajistics I am not sure how does haviing a calibration dataset calibrates the model output without using a calibrator. You haven't used Platt Scaling or Isotonic Regression or even VENN ABER method from Conformal Methods to calibrate the output of the model. Just the calibration dataset won't calibrate the output without using a calibration function
@@ekansh I just want my prediction intervals calibrated, so if it crosses 90% it at 90%. I am not concerned about other ranges. I am not trying to calibrate the model overall. I just want a 90% interval that is calibrated.
