How Deep Neural Networks Work 

Yes

yes I agree but for simplicity sake he should have done a 0 to 1, 0 being black 1 being white and 0.5 being grey, because almost everyone follows that pattern, and for new learners its a bit harder for them to switch from thinking about -1 to 1 to 0 to 1

I feel the same, even though I still can't completely understand how weighting works :/

Tremor244 I am by no means an expert but weighting is what makes the network "learn" to be correct. By changing the weights it changes the output of each neuron and eventually the output of the network. If you tune the weights enough you will eventually get an output that is what it is supposed to be. i hope this helps

I don't work with neural networks but with other types of machine learning. But weighting is more or less the same in all these fields of mathematics. You want a system that, provided with an input (an image, for instance), achieves its classification as the output. For instance you have a scenery (input) and you want to know if it's from vacations at the mountains or at the beach (a classification, ie. the output). So you pass the image trough a set of filters: (1) does the image have umbrellas? (2) does it have clouds? (3) is there a lot of blue? (4) is there a lot of brown?, etc. If the image passes a specific combination of filters, there is a greater probability that the image is of a specific type (for instance, if the image (1) have umbrellas, (3) is blueish and isn't (4) brownish, it's more likely to be from the BEACH). But how much more likely? That's when the WEIGHTING comes into play. Through machine learning we want to calculate some coefficients (weights) that state a sort of likelihood of an image to pass a filter, given its type (for instance, if it has umbrellas there's a probability of 0.9 out of 1 (90%) that it is from the beach and not from a mountain, but if there's a lot of blue maybe only 0.6 of those images are from the beach, and so the WEIGHT IS LIGHTER. That means that, if the image passes a filter of COLOR BLUE it is likely to be from a BEACH, but if it passes a filter of UMBRELLAS it is EVEN MORE LIKELY). Weights, then, are a parameter of RELEVANCE of each of the selected filters to achieve the correct classification. So we make the machine learn from LOTS (thousands, perhaps) of images that we KNOW are from the beach or the mountain. One image from the beach has umbrellas, so the classification through the filters was correct and then the WEIGHT for the umbrellas is increased. But if there is an image of the mountains with umbrellas and the program says it's from the beach, the weight goes down for the umbrellas. When we did this with a lot of images, the weights are FINE TUNED to classify correctly most of the time (if the filters are any good... if we chose wrong filters from the beginning, then there's a chance the dictionary won't get any better even fed with lots of images. That could also happen if the training images are biased: ie. if they don't represent the real set of images that we want to classify). I hope this works better for you!

If you have had a course on linear regression, you will recognize weights are equivalent to parameters. They are just "free variables" you adjust in order to match inputs with outputs. In one-dimensional linear regression, the parameters are the slope and offset of a line, you adjust them so that the distance between the line and your points (your training examples) is the least. Neural networks use the same idea as statistical regression. The main difference is that neural networks use a lot of weights (parameters), and for this reason you have to care about overfitting. This in general does not happen in linear regression, since the models are way more parsimonious (use only a few parameters). The use of a lot of weights is also the reason why neural networks are good general approximators, the large amount of weights give them high flexibility. They are like bazookas, while statistical regression is more like a small gun. The point is that most of the times you need only a small gun. However, people like to apply neural networks to problems where linear regression would do a good job since NN are "sexier".

+Esteban Lucas Solari I want to let you know that I love you for everything you just wrote.

I legit cried a little bit. Thank you for this.

I'm very happy to hear it :)

I agree.Penny dropped for me today with this Video. Thank you so much Brandon

3blue1brown

Good to know that I'm not the only one who caught the logical mistakes. 9:14 Bottom second squash should be vertically inverted, if I'm not mistaken.

I had the idea of doing perceptual loss before I even knew the term for it, seems like it would work better for warp transforms and the like versus level transforms.

Alternatively, the edges are correct but the corresponding picture should be flipped. Regardless, the final step (output perceptron at the bottom indicating horizontal) works with either the white white edges or the black black edges scenario.

thanks :)

YES - thanks , I saw that tiny error too 🙂

Don't worry people I asked this guy if he was a physicist

@ thanks bro

Hey thanks! I really appreciate this. It's the highest compliment.

Thanks Mukul!

You are most welcome

I never understood backprop properly until this video...this was the light bulb

I'm really happy to hear that Bowbert. Thank you for the note.

Yes, there is a mistake.

It is true! Good catch both of you.

Nobody

:)

Muhammed Sahli's mother

SirNate I still remember most of my calculus and have notes from prior study 😜

lol

it goes without saying that I became a subscriber as well ;)

Artem Kovera lovely book , just downloaded. for the lazy people : amzn.to/2ntC9Zm

Same here. My vision is clear

I just enjoy numbers. Anything to do with them is a fantastic thing.

Give it a go! Although I would love to get a sticker for my laptop that says "Can't be that hard." I'll put it right above the one that says "What could possibly go wrong?"

I've been trying to implement the full network as seen (@ 25:30) in python but hit a roadblock. Have you done it?

I'm glad to hear it!

Thank you so much! I'm happy to hear how helpful it was, and it means a lot that you would send me a note saying so.

Another one!

Another one

another one :)

Another one! This is old :D

one more

Math, baby! It’s cool

Gavin Haynes please teach me how to start. im a beginner of this field. thank you

I haven't made one of my own yet, sorry for the confusion. The difference is that now, I understand how one works, so I can start structuring my information.

-------- Additional question is how does feature engineering play into training a CNN? Thank you.

Thanks Scott. I hope this one serves you and your class well.

Thank you! Ironically, I'm not a lecturer.

I must know calculas now. Yes!

Thanks Eddie. Will do.

Would it be better to train a very deep neural network in small chunks as opposed to just adding a lot of layers and letting back prop do the job?

Thanks! Google Slides 🙂

@@BrandonRohrer I love the fact that you explain how this neural network works by keeping it relatively simple and leaving out most of the jargon and abstraction without direct examples.

@@theultimatenewplayer9341 Thanks! That was exactly what I hoped to do.

That is such a good story. Thanks thắng.

Thanks on all counts

Thanks!