Just remember, the brain perceives things through a series of guesses. so with billions of neurons doing complex statistical analysis, nobody is as bad at math as they think :)
I'll never be able to articulate how wonderful it is to be learning this from a fellow black man. This is a blessing. Thank you infinitely my brotha. Thank you, you inspire me to be greater.
Neptune Productions small detail but “teach computers” is misleading. In captcha you label data to train models. A computer would be used to train the model or apply the model later, it doesn’t “learn” anything.
Yeah, it's kinda weird that the test to prove your human is training computers to be able to do the same thing. So there will be a point where Captcha will change/become more complex because computers can solve the current generation ones as well as humans can.
@@thomas.02 There are certain things that Humans are really just better at doing than computers are. Take for example AI in videogames. A human can probably execute a tactical manuveur, but they take time to process and plan. Computers can take instant action and rush the human before it has chance to think. (I'm referring to Total War: Atilla)
Crash Course is the place to be. I expecially love this series. This channel has inspired me to create my own channel. It is new and I would love to get some support/guidance on how to improve.
I find it strange that Alex Krizhevsky (as 12/20/19) doesn’t have his own page on Wikipedia nor does he appear in the Wikipedia pages for neural networks nor machine learning, yet his work is cited over 84,000 on google scholar.
AI is basically math. To understand, how backpropagation learning in neural nets works, you need to know your multidimensional calculus, chain derivation rule, and some undergraduate-level linear algebra. That's all there is to it.
wonderful. Just astonished to know that as we dive into the "deeper" parts of the neural network, we can understand less on what's happening behind the scene. It can be so abstract to the point that it evades our cognition.
This is a great series, thank you all for taking the time to make it! For future videos, could Jabril's audio be turned up just a bit more? Sometimes, the end of his sentences get quieter and it's harder to catch all the info. Thank you!
Speaking of Learning things fast 💟 AI is one such industry that requires constant learning.. Thanks for this video. Once you learn this technology it's so dynamic that if you don't catch up and be updated, your knowledge will get outdated
I believe that alphago and alpha zero put the grid in the chess or GO as one input neuron, in the GO they can make black piece as -1, none as 0, white as 1. In chess each piece can be assigned a single number.
Alpha Go is notable less for the basic architecture of its network, and more for the way it's trained. The reason why Go was considered such a problem was because its decision space is huge. Chess was comparatively simple. Chess is played on an 8x8 grid, which means it was feasible for a computer to calculate on the fly every single possible board state given the current state, and then it would simply pick the move that made it most likely to win. That's why computers beat grandmasters in chess during the 90's, almost 20 years before neural networks really took off. Go, on the other hand, is a 15x15 grid, and so the decision space was much to large for that kind of brute force calculation. A simple multi-layer perceptron like this episode shows wouldn't work for this either, partly because there simply isn't enough data. You would need move-by-move dissections of hundreds of thousands of games, minimum, and probably more in the millions or even possibly billions of games. They'll get to why this is the case next week when they talk about optimization methods. It also wouldn't work because you would have no way of telling the network what makes a move "good". The solution was to use a different method of optimization called a policy gradient, or more broadly, reinforcement learning. In a sense they simulated games of Go, with the computer playing against itself. At every turn, the network takes the board state as input, and then decides on an action to take - in the case of Go, which spot to put a tile on. It starts off making decisions randomly, but you update the weights on the various actions based on its performance - it plays out whole games, probably against itself, and gives a reward to the winning set of actions, and a punishment to the losing ones, weighting them up or down respectively. Over many, many games, the system learns a policy - what action to take given a board state. Importantly, it doesn't need to know anything about what moves were made to get you into this state, nor does it have to calculate future permutations of the board. Much like the image recognizer, it learns "if I see tiles in this configuration, this move is the most likely to win." In this way, it kind of provides its own training data. Alpha Go is far more complicated than this simple description of course, and if anyone knows its architecture better than me I'd be glad to hear more about it (I haven't read that paper) but that's the essence of it.
Maria Fedotova Glad it was enjoyable! I’m finishing up grad school on this topic now, and I had a seminar last semester that covered reinforcement learning extensively, so I find it super interesting!
Yes and no, input layers let you measure more, hidden layers allow the program to do more with the data and 'think about it' in a more complex manner. It also depends on how the weights have been calibrated as well, though. Hope I could help!
One thing that was glossed over (though understandably so because this is just the intro) is how the input features are used. What he described is a simple 'fully connected' layer that treats every input pixel as a separate feature and looks at arbitrary combinations of them. But this is actually not very robust against things like translation, i.e. if you trained it on images of dogs where the dogs were all centered in the picture and then you showed it the image of a dog in which the dog is off the side it probably wouldn't even recognize it as a dog because the starting location of each pixel is extremely important. What almost all image recognition algorithms use today are 'convolution layers'. Rather than training neurons that look at specific pixel they train 'filters' that are a small group of pixels that gets scanned across an image. So the specific pixels input into the filter are constantly changing but they're always going to be in the same positions relative to each other. This emphasizes relative positions of pixels over absolute positions and makes the whole algorithm a lot more robust as well as easier to train.
@@kevadu so they basically program the pixels to be scanned as one large 'pixel' and that changes based on what you are looking at, right? (Just trying to synthesize the info u said so I can make sure I understand it)
They didn't, but a bias can be thought of as an extra weight on each layer (or an extra neuron that's always inputting 1), so it's kind of captured by the simplified discussion of weights. Also, as someone who is a professional software engineer, who has a degree in computer science, and who took several in-depth machine learning courses at uni, let me tell you: I'm still stuck on backpropagation XD The overall idea is simple enough, but I have yet to be able to ever remember the math that goes into it.
IceMetalPunk yeah, I ended up just running two versions of the same network, and the one that is better gets cloned too the other with slight changes to the weights and biases. It works, but takes more computation and time (as cloning the original one with differences can make it worse). Also my network doesn’t have a limit where it can only be in between 1 and 0. I use outputs of different numbers to tell me what it’s saying. Ex output 0.5-1.4 means blank output 1.5-2.4 means blank etc.
@@element9224 That is known as neuroevolution, my dude :) As for the output, there's functionally not really a difference between 0-1 or 0-2.4; you can always project the range onto any other range proportionally :)
Hey! Jabril, want to thank you for such an informative and easy to comprehend lecture. But the only thing is that I didn't get that gist of the math imagery. Could you help me out?
why red green and blue and not red yellow and blue (or magenta, yellow and cyan blue)? and how do the neurons "distribute tasks"? how do they "decide" which neuron of the hidden layers focuses on what?
Tibetan Monks discovered this "Neural Network" long ago. Hundreds of them chant independently parts of a prayer so it is done in one sec. The problem is that god is deaf.
why can't the neurons within each hidden layer interact with each other? For example, if a neighbour neuron got a high number, that'd make another neuron (of the same layer) act differently. is that arrangement helpful or just unnecessarily complicating things?
Thomas Chow the network described in the video is called a feedforward network. The kind of network you’re talking about does exist and is called a recurrent neural network. It’s usually used for sequential data where as feedforward nets are used for one-off computations.
@@thomas.02 sequential data can come in a bunch of different forms. Usually it comes in the the form of a time series. For example you would use a feed forward net to classify images and you would use an RNN classify videos with each video being a stream of images over a period of time. You're right in that RNN's are likely used in self driving applications as data is captured in real time so to speak. Another place you might want to use an RNN is for something like speech to text where the number of words/syllables in a sentence can vary. Typically you don't want to use RNN's for simple classification as it's a bit overkill. You can make feedforward nets as big/complex as you want to approximate whatever function you're trying to map.
5 лет назад
What was the guy saying "nope, nope" with his head an closing the laptop, looking at ?
Contradiction. At 9:30 he says that AlexNet needed "more than 60 million neurons", but at 2:33 we can see the abstract of the paper and it says AlexNet used only 650,000 neurons.
@@IceMetalPunk if a psychology researcher wishes to carry out a study on human subjects, they have to pay those subjects. yet somehow, people literally mucking in on the hard graft of research should do it for free?
@@dylanparker130 The people who are "doing it for free" are the same people who are benefiting from it directly. In your analogy, it'd be less like having free subjects and more like having other psychologists help with their research --- something that happens all the time.
@@IceMetalPunk i don't believe the people who respond to crowd funding are typically fellow researchers, but rather interested amateurs if they were fellow researchers, why would they agree to help other researchers in this way? to do so would be to help the competition, unless they were credited with authorship, which they would not be in such cases
@@dylanparker130 First of all, many people in STEM fields are more concerned with advancing their field than trying to compete with others at the expense of their field. Secondly, whether you're an amateur programmer or a professional computer science researcher, advancing AI tech still helps you out. (Don't believe me? Take a look at Runway ML, a little software suite that's designed for amateur programmers, but lets you use tons of advanced machine learning algorithms and pre-trained networks without having to even understand the details of implementation if you don't want. Machine learning advancements aren't just for professionals and researchers.)
