Softmax Regression (C2W3L08) 

so true

Actually , it was Richard Feynman who said it

no einstein said it, feynman applied it perfectly

This is a common logical fallacy: A implies B is not equivalent to B implies A.

Spotted a wild Jabril in his natural environment! *Attempts to say Hello*

Late reply but for anyone wonder. Yes you are correct, for prediction purposes it makes no difference (though still maybe you would like to see the "probabilities" generated), but you are correct that just the the max will be chosen as the "predicted solution". This however is not true for training. When training you need to be able to measure "how wrong" you were. This is where the softmax function comes in to give probabilities, which you can calculate loss from, and also calculate a derivative to update the weights.

Make a grid, let's say 100 by 100 points, and in each point calculate the activation. It's actually how he do this in programming exercises.

Thanks man! Then I suppose it's impossible to get a closed-form equation for the decision boundary? Can you please provide link to the video where he uses the method you described here?

please let me know if you got it

This makes definitely more sense to me. The same applies for the sum written in 5:00

yes :)

I believe that under the sigma function it should be i=4

just assumption

Kunhong YU sure, but i mean intuitively, what do they represent?

Similar to simple Logistic Regression, Softmax just adds more output units rather than one. For logistic regression, output unit is just a 1 dimensional vector computing input X's linear "score", if it's larger than zero, then its label is 1 and vice versa. Softmax is like training multiple binary classifiers simultaneously, for a sample, each element in Z is also a "score", largest score denotes that sample may have corresponding label.

@derrikbosse, the Z vector identified here has 3 arguments: W{L}, A{L-1}, and B{L}. W{L} is the vector of weights within the last layer of the network. A{L-1} is the vector of outputs from the penultimate layer of the network. B{L} is the bias vector from the last layer of the network. If none of this makes any sense, check out Professor Ng's earlier discussion of logistic regression, which is the simplest kind of neural network. That helped me make sense of this presentation: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-hjrYrynGWGA.html

Weight matrix for layer L , i.e the last layer.

colored-in-circles are actual training data/values.After training model on training data,model predicted all decision boundaries.so color of blocks are blocks predicted by model.

Actually I cal it for MR.NG hh