Machine Learning Fundamentals: Sensitivity and Specificity

Подписаться 1,2 млн

Просмотров 341 тыс.

50% 1

In this StatQuest we talk about Sensitivity and Specificity - to key concepts for evaluating Machine Learning methods. These make it easier to choose which method is best for your data.
For a complete index of all the StatQuest videos, check out:
statquest.org/video-index/
If you're not already familiar with the Confusion Matrix, check out the Quest: • Machine Learning Funda...
If you'd like to support StatQuest, please consider...
Buying The StatQuest Illustrated Guide to Machine Learning!!!
PDF - statquest.gumroad.com/l/wvtmc
Paperback - www.amazon.com/dp/B09ZCKR4H6
Kindle eBook - www.amazon.com/dp/B09ZG79HXC
Patreon: / statquest
...or...
RU-vid Membership: / @statquest
...a cool StatQuest t-shirt or sweatshirt:
shop.spreadshirt.com/statques...
...buying one or two of my songs (or go large and get a whole album!)
joshuastarmer.bandcamp.com/
...or just donating to StatQuest!
www.paypal.me/statquest
Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
/ joshuastarmer
Corrections:
7:28 My wording could be improved. The True Negatives for Troll 2 are the people who were correctly predicted to not like Troll 2.
#statquest #ml

Опубликовано:

2 авг 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 351

@statquest 4 года назад

Correction: 7:28 My wording could be improved. The True Negatives for Troll 2 are the people who were correctly predicted to not like Troll 2. Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/

@stefanosmandalas6599 4 года назад

Thanks for this comment, I was wondering during the video.

@LockyLawPhD 3 года назад

Thank you for all the great videos! I have learned a lot from you. Will we see a video on Partial Least Squares (regression / SEM) for ML? There are some videos on RU-vid but are all very confusing.

@statquest 3 года назад

@@LockyLawPhD I'll keep it in mind.

@freemind.d2714 3 года назад

And for those who wonder, yes, "Sensitivity " is "Recall"

@falaksingla6242 2 года назад

Hi Josh, Love your content. Has helped me to learn a lot & grow. You are doing an awesome work. Please continue to do so. Wanted to support you but unfortunately your Paypal link seems to be dysfunctional. Please update it.

@alejandrasevilla2237 4 года назад

StatQuest should be mentioned in my future Data Scientist Master Diploma. Seriously! Every time I get introduced to a new concept, I come here and.. BAM! it all makes sense.

@statquest 4 года назад

Awesome! Good luck with your Master Diploma. :)

@andyn6053 4 года назад

@@statquest I have learned more from watching your videos than from my teacher at the university.

@arvinflores5316 3 года назад

How's your Masteral now ma'am? I'm starting one too this 2021 :p

@buytfdip7712 Год назад

Same for me with my Bioinformatics master's! He's awesome

@geosaiofficial1070 Год назад

same thing for me. Thank you

@ShaileshDagar 4 года назад

Having trained myself on other StatQuest video, I predicted that I would understand this video and I actually understood this video, so that looks like a True Positive right there.

@statquest 4 года назад

BAM! :)

@littleKingSolomon 4 года назад

It isn't English that u r speaking. U r speaking ML

@nooreldali7432 Год назад

@@littleKingSolomon I read ML as machine language 😂

@ritikkhatri 4 года назад

This channel is a gem. Glad I found it!

@statquest 4 года назад

Awesome! Thank you very much. :)

@littleKingSolomon 4 года назад

Watched confusion matrix statquest, and I forgot to like. So I paused in the middle of this one, went back and liked then came back. Because u deserve more than just likes. But for now that's all I have to offer. God bless u and thanks

@statquest 4 года назад

Wow, thanks!

@nileshsuryavanshi8792 4 года назад

No one teaches like you, thanks for making such a playlist.

@statquest 4 года назад

Thank you! :)

@adsax1903 4 года назад

I listen to ur Quests at 2x speed and yet they are super uber crystal clear. god speed man

@statquest 4 года назад

Nice! :)

@alecvan7143 4 года назад

Your videos are always astoundingly clear and well explained! In fact, you've inspired me to the point of being the first RU-vidr I support financially!

@statquest 4 года назад

WOW! Thank you very much! I really appreciate the support. :)

@emmaning992 4 года назад

Your channel is honestly gold... I used to think stats is a mystery and it didn't make intuitive sense until I met your channel - now everything makes sense and I enjoy using them. Thank you for making these free and accessible, I really appreciate your effort, dedication and intelligence

@statquest 4 года назад

Thank you very much! :)

@DarkLordAli95 4 года назад

I only wish that I had found your amazing videos at the start of the semester. oh well, better later than ever, so BAM anyway!!

@statquest 4 года назад

Thanks! :)

@PunmasterSTP 4 месяца назад

Out of curiosity, how'd the rest of the semester go?

@aysegulkul1741 4 месяца назад

Size ne kadar teşekkür etsek az. Müthiş videolar gerçekten. Bu kadar iyi ders anlatan bir hocam olmamıştı hiç. Başarılarınızın devamını dilerim.

@statquest 4 месяца назад

Thank you!

@ankitsapra1595 3 года назад

This is my go to channel for any stats related knowledge. Thanks Josh for this exceptional channel👏👏

@statquest 3 года назад

Wow, thanks!

@amirmohammadjalili2676 3 года назад

Josh Starmer you're awesome and you're really helping the community so much. Many thanks to you sir

@statquest 3 года назад

Thanks! I'm glad you like some of my videos.

@ifmohammed6495 4 года назад

I really sad that there is no one support such that videos I really think you deserve more

@statquest 4 года назад

Thank you! :)

@nishantrao5600 3 года назад

BAM! That's an amazing video. I completely get sensitivity vs specificity now. Thanks!

@statquest 3 года назад

BAM!

@umanagaswamy6358 4 года назад

The prelude is absolutely funny :) And the content is amazing, thanks StatQuest 👌

@statquest 4 года назад

Thanks! :)

@douglasespindola5185 2 года назад

That's what I call a great explanation! TRIPLE BAM! Thanks, Josh! Greetings from Brazil!

@statquest 2 года назад

Muito obrigado!

@mohammadsaad7221 3 года назад

Why didn't I find this channel before?? It's a gem!!

@statquest 3 года назад

bam!

@gangcheng7322 4 года назад

you are the doctor who saves my time. many applause.

@statquest 4 года назад

Thank you! :)

@muthukumarantce 4 года назад

Always precise and clear video on intuitions. Thank you Josh :) Enjoying Double BAM!!!!!

@statquest 4 года назад

Hooray! :)

@sonang9495 4 года назад

easily understood, clearly explained, this channel is precious for newbies, Thanks ad

@statquest 4 года назад

Thank you very much! :)

@brpawankumariyengar4227 3 года назад

All the videos of this Chanel are very well made and are extremely good ….. Thank you so very much ….. I never knew I could get such incredible videos for free …. I will share this channel with All tomorrow.

@statquest 3 года назад

Thank you very much!

@oladamola8690 3 года назад

Josh, I must confess you’re the best !! I can’t thank you enough 🙌🙌

@statquest 3 года назад

BAM! :)

@janvichokshi4892 3 года назад

You explained it so simply ! Just loved this tutorial. Thanks Josh :)

@statquest 3 года назад

Glad you liked it!!

@haidyabugom3a133 6 месяцев назад

Great explanation! I would like to tell you that our Prof in the college uses your content and suggests it as a reference to understand ML. Thanks a lot for your effort! Really the best RU-vid channel .

@statquest 6 месяцев назад

Thank you very much!

@proggenius2024 3 месяца назад

I'm lucky to stumble on this channel before my final exam! How cool is your teaching...wow!

@statquest 3 месяца назад

Good luck on your exam!

@saravkumar8058 2 года назад

Wonderful Mate. Appreciate your work. Your explanation with example is top class!!! Happy that I found this channel. Great work. Thanks a lot for en lighting us on ML concepts.

@statquest 2 года назад

Awesome, thank you!

@saravkumar8058 2 года назад

@@statquest BAM!

@LorenceLiu 3 года назад

You are better than all of my undergrad and grad professors :D

@statquest 3 года назад

Thanks!

@MB-vd6hc Год назад

Thanks, Josh, you explained the concept very clearly.

@statquest Год назад

Glad it was helpful!

@RaynerGS 3 года назад

Amazing class. Way to go! Salute from Brazil.

@statquest 3 года назад

Muito obrigado!

@ejn2178 2 года назад

To be 100% honest I wasn't a big fan of the intro song hahaha but I gotta admit whenever I see new concept or having difficult time to understand topic while studying I come back to this channel. Great work!! It's sorta addictive now hahahaha

@statquest 2 года назад

Bam!

@pratikmishra162 4 года назад

I am really thirsty and watching statquest! Thank you sire.

@statquest 4 года назад

BAM! :)

@fabiof.deaquino4731 4 года назад

Quest on! Happy New Year! 👏👍👏

@statquest 4 года назад

Hooray! Happy New Year! :)

@xxxsamxxx130 Год назад

best explanation i've ever come across.. thanks a lot..

@statquest Год назад

Glad it was helpful!

@JerryWho49 4 года назад

You made again a great video! Thank you. Maybe you could have mentioned precision and recall (=sensitivity) as both terms are often used, too.

@statquest 4 года назад

Thanks! I'll put that on the to-to list.

@Hardson 4 года назад

Please, keep making these amazing videos!!!

@statquest 4 года назад

I will! :)

@Yibioa 4 года назад

Hey, can you make a video about Independant Component Analysis, and how this can be used for biological data, such as gene expression data? Btw, my favorite channel. You rock.

@adityasaxena3903 4 года назад

Amazing Content! Keep growing.

@statquest 4 года назад

Thank you very much! :)

@hi_10svideos86 4 года назад

before statquest path was like false positive for machine learning but after watching this channel my path become true positive.

@statquest 4 года назад

Bam! :)

@MajedDalain 4 года назад

such a great explanation and content. thanks a lot

@statquest 4 года назад

Thank you! :)

@prydefonbah4436 3 года назад

Gave practical application in a meaningful way.

@statquest 3 года назад

Thanks!

@ChituOkoli 4 года назад

I just discovered your videos yesterday and I subscribed immediately. Thanks a lot! Could you please make a video on precision and recall? Actually, what I would really like to see, more than just the calculations, is help understanding when sensitivity vs. specificity is more useful than precision vs. recall and vice versa. I imagine that a purely calculations video on precision and recall would be very similar to this one on sensitivity and specificity, but knowing when each statistic is more useful in real life applications would be more helpful to me. You touched on that very briefly in this video when you mentioned which is preferred (random forest for higher sensitivity in detecting heart disease versus logisitic regression for higher specificity in confirming that people don't have heart disease), but it would be helpful if you spend more time on those--to me, it is that kind of application that really brings statistics to life.

@ChituOkoli 4 года назад

@techoji No, they're not. That's why I'm asking for a separate video.

@barath961 3 года назад

Thank you so much for fantastic lessons :). Love from TN

@statquest 3 года назад

Bam! :)

@punchline9131 4 года назад

Great BAM for this video!!!!

@statquest 4 года назад

Thank you!!! :)

@neuodev 2 года назад

I was alwasy think of Sensitivity and Specifity as if they only can be calculated for 2x2 table. it was a good point from you to apply it on a 3x3 one Thanks!

@statquest 2 года назад

bam!

@chloegu4110 3 года назад

Great Explaination! The confusion matrix is really confusing... need to go over this video several times.

@statquest 3 года назад

bam! :)

@farzanehjmr7523 2 года назад

thank you very much for your clear and appealing explanation.

@statquest 2 года назад

Thank you!

@lightshinichi8056 4 года назад

This channel is awesome and content is so great ,

@statquest 4 года назад

Thank you so much 😀

@sanhitasaxena1542 9 месяцев назад

STATQUEST IS AWESOME!!

@statquest 9 месяцев назад

bam!

@harshkful 4 года назад

Amazing videos and explanation ! Thanks ..

@statquest 4 года назад

Thank you! :)

@hajarhajar8906 2 года назад

Thank you for saving my life!

@statquest 2 года назад

bam!

@zhspartan9993 7 месяцев назад

Your videos are awesome, thanks

@statquest 7 месяцев назад

Glad you like them!

@katwoods8514 3 года назад

Troll 2 is the best worst movie ever! Love that you included it as an example.

@statquest 3 года назад

BAM! :)

@AnkitSingh-fw7xy 4 года назад

More power to you! :)

@statquest 4 года назад

Thank you!

@thotayateeshchandrasai3828 2 года назад

So good!

@statquest 2 года назад

Thank you!

@prachinainawa3055 2 года назад

Woww! It's very much clear. I love it

@statquest 2 года назад

Thank you! 😃

@phuocnguyenlethanh3104 8 месяцев назад

I swear this guy release coolest music

@statquest 8 месяцев назад

bam! :)

@alfonsoreyes9811 3 года назад

Your vids are awesome!, thank you

@statquest 3 года назад

Thank you! :)

@dawnmarielagamba3165 2 года назад

FINALLY - I understand thank you!

@statquest 2 года назад

Glad it helped!

@technicalescola 2 года назад

Brilliant @StatQuest with Josh Starmer.. It is explicitly explained by You

@statquest 2 года назад

Thanks!

@t-ranosaurierruhl9920 4 года назад

Awesome!!

@statquest 4 года назад

Hooray! :)

@stevennye5075 4 года назад

well done!

@statquest 4 года назад

@maleehahaider8596 6 месяцев назад

Youre awesome Josh. Keep going! i have recommended your channel to all those bitten by sadistics... ahem! statistics!

@statquest 6 месяцев назад

Awesome! Thank you!

@sumitjoshi7904 4 года назад

BAM! i love your videos

@statquest 4 года назад

Thank you very much! :)

@aleksandrshipilo4467 4 месяца назад

Thank you for the lesson)

@statquest 4 месяца назад

Triple bam!!! :)

@pavoldzama4641 3 месяца назад

Hello, my suggestion for video I am missing on your chanell - Harmonic mean and F - score for recall and precision. Or at least when I searched your chanell I havent found video on this topic. Thanks for amazing work you are doing.

@statquest 3 месяца назад

Great suggestion!

@calvinfeng5357 4 года назад

I rarely see specificity as an evaluation metrics. It seems precision and sensitivity are more commonly used. However, it's very good to know this information!

@statquest 4 года назад

I should have a precision video out soon.

@calvinfeng5357 4 года назад

@@statquest That'd be great! I look forward to your newer videos!

@wolverine87833 3 года назад

BAMM!! this guy is a legend.

@statquest 3 года назад

Thanks!

@mostinho7 4 года назад

2:10 specificity example from 2x2 confusion matrix Sensitivity is true positives identified by the model / (true positives + false negatives) Specificity true negatives identified by the model / (true negatives + false positives) 4:50 using specificity and sensitivity to compare models Todo continue

@mehediazad1780 2 года назад

thank you so much.

@statquest 2 года назад

Bam! :)

@hoangminhthai4055 11 месяцев назад

This video is about the way to calculating sensitivity and specificity. Sensitivity = True positive /( True positive + False negative ) Specificity = True negative / ( True negative + False positive) For the case, we have more than two category, we will get these two numbers according to specific category

@statquest 11 месяцев назад

bam!

@sandipbhattacharjee3304 2 года назад

Thanks!

@statquest 2 года назад

WOW!!! THANK YOU VERY MUCH! BAM! :)

@sneezypete382 3 года назад

Josh Starmer, You rock!

@statquest 3 года назад

Thanks!

@bashiransari6258 9 месяцев назад

just awesome

@statquest 9 месяцев назад

Thanks again!

@parzynamea4701 2 года назад

now i know, when to use which classification model , lovely mates thank you

@statquest 2 года назад

bam!

@pranaymehta8471 3 года назад

Thanks for the video. Truly love it. Quick question1 - You mentioned that if correctly identifying negatives is more important, then we should put more importance on Specificity. But how do we decide which class is positive and which is negative. For instance, I am working on fraud detection problem and only 2% of transactions/examples are rejected. What would be my positive class and negative class in that case? Question 2 - Can we have a video on precision and recall ? I find it very confusing so as which Precision and Recall should I be computing when working with imbalanced classification problems. Should I compute Precision and Recall for Majority Class (accepted transactions) or should I compute these metrics for Minority Class(rejected transactions). Any insights on this would be really helpful.

@statquest 3 года назад

First, you have to decide what you want to detect. If you want to be sensitive to the rejected transactions/examples, then you need a method that is sensitive to the minority class. I talk about this towards to the end of my ROC video, so, for more information, see: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-4jRBRDbJemM.html

@liranzaidman1610 4 года назад

Fantastic

@statquest 4 года назад

Thank you! :)

@victoriaogden8322 2 года назад

What if you are hoping to predict something continuous - like the length of an object: Are sensitivity and specificity still used (perhaps by binning and tuning the sizes of bins)? I guess the metric moves from whether the model was right (True Positives and True Negatives) vs. wrong (False pos. and neg.) to how close the model performed to the true value. Would love to hear thoughts on what you would do to assess continuous models instead!

@statquest 2 года назад

When we predict something continuous, we usually use sum of residuals or mean squared residuals.

@ozonefriendlyman 2 года назад

Love you videos. Correction at 10:16 - The sensitivity for Cool as Ice not 17 / (17 + 175)

@statquest 2 года назад

nice!

@arjunkaarat9326 4 года назад

Hey Josh! Your videos are excellent. Would making similar videos on GLM and probability distributions need significant lift? Maybe it's in there already. I just came across this channel.

@statquest 4 года назад

I already have a bunch of videos about General Linear Models, as well as a bunch of videos on Logistic Regression (a type of Generalized Linear Model) and a few videos on statistical distributions. A full index is here: statquest.org/video-index/

@TheMangz1611 3 года назад

@statQuest Pls make a video on Precision as well - wth love from India

@statquest 3 года назад

Noted

@Pratik-Kedar 4 года назад

really nice concept clearance ..........! Keep it up , waiting for more content. BBBBBBBBBBBBAAAAAAAAAAAAAAAAAAAMMMMMMMMMMM

@statquest 4 года назад

Thanks! I have a lot of videos coming out this month. :)

@geraldoaugustoguto8442 4 года назад

Amazing Video. Thanks. What're relations between Precision/Recall and Specificity/Sensitivity?

@ShivamTiwari-on2kl Год назад

I have spent so much time on this and now I finally understand it. Sensitivity and Recall are the same. They tell us how many of the true values were truly predicted. Sensitivity tells us how many false values are correctly predicted. Precision tells us how many true values of all the true values are correctly predicted.

@ShivamTiwari-on2kl Год назад

I am sorry if I confuse you further.

@geraldoaugustoguto8442 Год назад

Thanks a million Shivam.

@Fieter_RMV 22 дня назад

thank youuu

@statquest 21 день назад

bam!

@superfaqer 4 года назад

Hey Josh, awesome videos! They help me understand the statistics and maths side of machine learning :) I have a question based on your example for the heart disease prediction; is it not possible to just use both models for predicting those 2 separate things? for example using the Random Forest for predicting heart disease and Logistic Regression for predicting free of heart disease? Or does that not make sense. Im farely new to ML and would appreciate an answer :) regards,

@statquest 4 года назад

Usually people use several different machine learning methods as an "ensemble" and make the final classification based on votes from each method. If you have a random forest predict if people have heart disease and logistic regression for predicting free of heart disease, then what do you do when the random forest does not predict that someone has heart disease and the logistic regression predicts that they do?

@superfaqer 4 года назад

@@statquest Thank you for the quick response :) I see what you mean, that could definitely be a problem. Appreciate the videos and your activeness! cheers mate.

@skumarr53 4 года назад

Thanks for the video. Can you make videos on Bayesian statistics?

@statquest 4 года назад

It's on the "to-do" list, but it won't be for a while.

@kimtae3446 4 года назад

Hi Josh, do you have a video about lift chart? Can't find it in your channel :(

@statquest 4 года назад

Not yet.

@tanphan3970 3 года назад

Hello Josh, Thank for directly explain about Sensitivity and Specificity. I read some term about Precision and Recall in Confusion Matrix. And I make confuse about using those concepts when choosing a algorithm for training. When do we take care Precision/Recall and Sensitivity/Specificity? Sorry if my question is in wrong in your flow! Thank you and have a good day!

@statquest 3 года назад

First see: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-4jRBRDbJemM.html

@vaggelisk9401 3 года назад

Hi, is there any difference in results if we change rows to Actual and columns to predicted? Great video btw!!!

@statquest 3 года назад

No.

@geoff1391 2 года назад

Hi Josh. Let's say one model was better at predicting Sensitivity, whereas another was better at predicting Specificity. Do you foresee any problems or downsides to data integrity to using different models, the model with the highest score respectively? For example, in your 2x2 confusion matrix @4:38, using your Logistic Regression model for Specificity measures and the Random Forest model for Sensitivity measures?

@statquest 2 года назад

Not that I know of off the top of my head.

@geoff1391 2 года назад

@@statquest Okay, that's good enough for me! Thanks Josh.

@sherryyu1936 4 года назад

I won't tell you, I am waiting for your neural network videos!!

@statquest 4 года назад

I hope to get to those in the coming spring. They are way behind schedule! (almost a year late). But better late than never. I'll start doing neural networks as soon as I'm done with XGBoost.

@Samurai_Jack__ 8 месяцев назад

Thank you for the amazing explanation as usual, I have a small question : what is the difference between this metric and precesion/recall metric ... when should we prefer one over the other? because my teachers just mentioned the latter metric

@statquest 8 месяцев назад

I talk about that in my video on ROC graphs: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-4jRBRDbJemM.html

@Samurai_Jack__ 8 месяцев назад

@@statquest Thank you so much

@bytesizedbraincog Год назад

Amazing one! How does the sensitivity and specificity metrics could be made actionable for more classes or categories (400 Classes) not sure which one to consider the most importance for bulk of classes

@statquest Год назад

See: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-4jRBRDbJemM.html

@umerbhikan576 4 года назад

you are amazzzzzzzzzzzzzzzzzziing

@statquest 4 года назад

Thanks! :)

@allaabdella4794 4 года назад

Thank you for the video, it's helpful. Can you please clarify the True Negatives ( 7:43). Why would we include 77 and 92?

@statquest 4 года назад

Because we are calculating specificity for Troll 2, everything is relative to Troll2. Because those people were correctly predicted to not love Troll2. This makes them "true negatives" with respect to Troll2.

@gauravchaudhari2828 3 года назад

I really liked illustrations and slides what software do you use ?

@statquest 3 года назад

I use keynote

@erick6381 2 года назад

I love you.

@statquest 2 года назад

@ankitsingh961994 4 года назад

Hi Josh, do you think it would add value to add metrics like Precision, Recall and Accuracy here, since we are talking about Confusion Matrix? I know Sensitivity and Recall are the same, just thought we can mention the term as well.

@statquest 4 года назад

I'll keep it in mind.

@weihangli5639 4 года назад

BAM!! It seems that author will reply every comment below his vedio, please prove me!

@statquest 4 года назад

@zeynabmousavi1736 4 года назад

What would be the definition of sensitivity and specificity in the case of non disease? Lets say we need to classify between two classes: class A and class B. Which one is positive which one is negative?

@statquest 4 года назад

You decide which one to treat as positive and which to treat as negative.