There is an error in your formula for Shapley values (compared to christophm's book and wikipedia). Most write the weight as (M choose 1, |S|, M-|S|-1) where S is some subset WITHOUT feature i. However, you are summing over subsets z' WITH feature i. Thus, the weight should be (M choose 1, |z'| - 1, M-|z'|).
This is a tour de force. A book I read with like-minded themes was also magnificent. "Game Theory and the Pursuit of Algorithmic Fairness" by Jack Frostwell
When you use summary force_plot to get an individual points contribution is the prediction shown the log of odds? If so how do I show the actual probability
Hi! Sorry didn't see that comment somehow. Did you have a look at this post: github.com/slundberg/shap/issues/963 ? For Tree Explainer you have an option to get probabilities, according to that.
Hi! Do you mean the bars wrapped around z? That comes from set theory in math and the notation stands for cardinality = the number of elements in the set. It doesn't mean the "abs" function. Is that what you were referring to? :)
@@DeepFindr yes, thank you for clarifying. One more doubt, if I put z' =M then the (M -|z'| -1)! becomes (-1) ! which is not defined. What am I missing here?
@@yashnarendra Hi, good catch! In the original formula of Shapley values this can never happen, because the sum is over subsets without the feature i. Therefore F will always be greater than S (with the notation of the original shapley formula in the paper). But you are right, this is not really reflected in the SHAP formula. However in the paper they state that they excluded |z'| = 0 and |z'| = M as both are not defined.
Amazing video again bro. It helped a lot! One question: Do you know the reference where I can find the proof of the 2nd theorem in the SHAP paper? I can't find it :(
Hey :) thanks! The supplement can be downloaded here: papers.nips.cc/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html Plus there is another link in a discussion on Github, which might be helpful as well: github.com/slundberg/shap/issues/1054 Hope this helps :)
Great Video. I have one question.... Can I use this SHAP model for Graph classification techniques, to see which nodes contribution is how much and kind of show a grad cam like effect? If you got any sort of resoyrces regarding this, please share. Thank you.
Does the classification threshold have an effect on shapley values? The default threshold in binary classification is 0.5, what if one changes it to 0.7, how to incorporate this in the shap library?
LIME is independent from the size of the Dataset. The only question is if the (Blackbox) model works on the Dataset. Can you maybe share some more details what makes you raise this question? :)
I have a very small dataset that surprising do well on prediction of 15 different class of identical species. In my black box model, I use three layers of feature extraction methods and finish off with one random forest model. I am not sure whether I can implement LIME for this situation because my black box were mostly feature extractions rather than ensemble of models.
Which feature extraction layers are you using? Is it trained end-to-end with the RF? It doesn't really matter what is happening inside your model. LIME is able to explain the input-output relation in a local area of a single prediction :)
