i loved how you explained this topic in an understandable way. I learned the basic definition of Neural Networks through your video. I hope you continue making these types of videos in the future. I'll try and stay tuned :)
Thanks so much for the kind words! I used an open-source software called Synfig for this animation. It's good for small animations, however I would recommend a more sophisticated tool if you want to do anything more complicated.
@@Nurutomo I'm sorry about that. I really am. Sadly it seems that RU-vid is currently not supportive of this kind of format (videos with a length of only a minute don't seem to get promoted for me...). Thus I do not currently have the resources to produce another animation of this quality. If RU-vid decides to actually support videos like this a bit more, or if I find a way to acquire the resources through another source (e.g. sponsoring or patreon) I would of course be more than happy to produce more of them, especially since I personally really enjoy the idea of them. In the end, this was always meant to be sort of a 'pilot' episode in order to test how well the format is received.
@@SamuelArzt I loved this video and I would love if you could make more videos like it. I would be happy even if the animation is of lower quality allowing you to make a longer video so the RU-vid algorithm allows you to support the channel more. Please don't let RU-vid prevent you from creating more content!
HOLY SHIT MAN, I hardly understood certains concepts of neural network and right know, I'm stoned and still I could easly understand everything you said and make connections with others things I learn. In two words : GOOD JOB !
@@danielr. I'm just pointing the fact that he did a good job with his video, not asking for life advices. But thank you for caring about my health condition !
Thanks for the kind words. In regards to "what happened", I'd say 'life' 😅. But to be more precise, I think after finishing university, I enjoyed coding in my day job more than creating videos in my spare time. I'm a coder, not a video producer / editor after all. But I think I might come back to producing videos some day. I especially liked the 'explained in a minute' format and it might lend itself quite nicely to the now very popular "TikTok" / "Shorts" formats. Sadly, I can't promise any specific date for when I will get time again to produce more videos. I simply don't want to force it upon myself right now. I think that's the most honest answer I can think of for that question 😅
Thanks for the explanation!! It helps a lot to me :) Can i just ask one question? The number of elements in middle layers, I'm really wondering that it depends on the data i have or just chosen by me(ex-add numbers when i think it's important!)??
The middle layers, mostly called 'hidden layers', are completely up to the architect (i.e. up to you), both the amount of neurons per layer as well as the amount of layers. However, as you might have already guessed, there has already been a lot of research conducted on how to best choose the size of your hidden layers. In very very simplified terms, you could come up with this rule of thumb: if you choose too little neurons for your hidden layers the network might not be able to solve your problem. If you choose too many, your network might take too long to find the correct solution or get prone to overfitting (i.e. learns your training data perfectly, but fails at generalizing to new samples). The only layers dependent on your input data and the desired output are the input and output layers, i.e. the first and last layers.
@@SamuelArzt Ohmygosh Thankyou, sir!! Now i can understand whole contents in video. Actually it really bothers me for couple days.. you are the best :) !!!
Thanks! That's actually the next topic I have planned. Haven't got much time to create the animation for it right now, but it will definitely be coming some day ;)
That's an excellent question. The short answer is, in order to be able to solve more complex problems. The detailed answer is a bit more math-heavy than what I intended the video to be, but you are probably more interested in this one. Basically, the activation function enables neural networks to also approximate non-linear functions. You can also think of neural networks as *function approximators*. You probably know of the basic line equation y = k*x + d, where 'y' is the value on the y axis for a given value 'x' on the x axis and 'k' being the inclination and 'd' being the intercept. If you think in terms of neural networks you can also model this exact equation with a single neuron, where 'x' is the input of the neuron and 'y' is the output. The inclination 'k' is now the weight of the neuron and the intercept 'd' is the bias of the neuron. So you can obviously represent any line even without an activation function and even with only a single neuron. Similarly you can represent any *linear* function (for instance of the form y = k1*x + k2*x + k3*x + ... + d) by simply adding more neurons to the network. However, without an activation function, there is no way for the network to be able represent a function like y = x². With a non-linear activation function, however, neural networks are able to also approximate any non-linear function. In theory even a single hidden layer is sufficient to approximate any multivariat function (as already theorized by Kolmogorov in 1965) as long as it comprises enough hidden units. However, practical applications show that it is usually a lot easier to solve truly complicated tasks with deep networks rather than shallow ones.
I am wondering the same thing! How do we know what bias to set the neuron, or is that the part that is modified? I thought the script modifies the weights, not the bias?
Really digging the simplicity. We would like to use for an internal transformation training at our company. Requesting permission to use this copyrighted material
Thanks for the kind words. Feel free to use this video for internal purposes, under the sole condition of making sure that under no circumstances the author of the video might be assumed to be someone else, other than Samuel Arzt (e.g. by simply mentioning this channel either by name or by link).
I wish I could do more of these. Sadly, it wasn't as successful as I had hoped (as measured by the views in the first 3 months). Once I get more free time I would definitely like to continue this series though. Maybe doing a Patreon for it would help me to be able to invest more time, but I think I would have to have at least 3-5 videos of the series up before doing that. But please let me know, if you got any other ideas 😄 Just trying to be completely honest here.
I have some sort of request to you, please explain how the lines are made by algorithm, beacouse i cant find any article that would explain to me how do program does lines from point A to point B.
I assume by 'lines' you mean the connections between neurons. The lines are only a matter of visualization. In code they simple mean, that when a neuron needs to process it's current value, it takes the current values of those neurons connected to it. The connection weights can be simply stored in an array. Which neurons are connected with which other neurons depends on the type of neural network. In this video I explained 'fully connected feedforward' neural networks. Feedforward means that one layer always feeds input to the next, while no layer can feed a previous layer. Fully connected means that *each* neuron of one layer is connected to *all* neurons of the next layer. Or in other words: a neuron takes the current values of all neurons of the previous layer to calculate its own current value. Then there are also recurrent neural networks, convolutional neural networks (a variation of feedforward neural networks), etc.. There are also algorithms which remove / add connections between neurons, but in the standard case the fact that neuron A is connected to neuron B is predefined by the developer and stays like this until he decides to change it.
Wow, thanks for explaination how do neurons work, but i mean lines as connections of pixels (i skip that i didn't understand this about neurons too) i mean i dont know what is "Bresenham's line" and i think it could be explained in 1 - 15 minutes but *I* did not found any vid about this.
I think it is only hard for me to understand, each one guy who will think about this will understand it in "diffrent point of view" you know what I mean.
In your github Applying_EANNs you made a the ANN layers and all yourself!! Is it because C# doesn't have good libraries like python? or is there any other advantage to this? I am a student of Neural nets.
True, that's mainly because I did that project as a practice for getting to know Neural Nets. I think the only advantage of implementing it yourself is that you have to truly understand what you are doing. If you want a top performing and easy solution you are most likely better off to simply look for a library. You also have to keep in mind that this project was a very, very simple problem compared to actual research tasks. I have to admit, that I don't know of any well-known C# libraries for Neural Networks off the top of my head, but that is mainly because whenever you are looking into research problems, most of those will be using some scripting language such as python (bear in mind that the core of those libraries is most likely also written in something C or C++ like, or it even utilizes GPU-specific languages, which makes them a lot faster). I have heard of ML .NET which is rather new and is specifically designed to be used with C#, but I am not sure whether they also implemented Neural Networks yet. If I get some time, I would definitely like to try it though.
Samuel Arzt I did some research on ml with C# cntk is the library for it, just like python's tensorflow. Anyways liked your project a lot. It was one of the projects that got me interested in neural nets.
That's a very good question! It very much depends on the method you are using to train the network and the problem you are trying to apply it to. Probably the most simple method would be to initialize the weights of the network with random values (let's say in the range -1 to 1). The values are then tweaked through training until they lead to the desired outcome. You can image that as kind of trying random things at first and getting better through trial and error or supervising feedback. There are a lot of papers that studied how different initialization methods can help speed up learning or even lead to higher quality outcomes, but depending on what you are actually trying to achieve with the network, a random initialization might be completely sufficient.
I really appreciate your interest and understand any disappointment. That's true, almost two years. I always saw this episode as the pilot for a series. Unfortunately this video did not receive the attention I had hoped for. Not enough attention for me to be able to allocate as much time to this project in order to keep the standard of quality that I want to achieve. Sadly that's how this platform works. Views directly correspond to the amount of resources that you gain from a video. This doesn't mean that I will never come back to this format and create new videos for it. However, I will either require more free time or a way to adequately finance the time I put into this project. I could start asking for donations or start a patreon, but this always feels like begging to me and I also don't think my channel has enough of a following / enough content yet for this to work. I am still working on another machine learning project, which I hope to be able to upload a new video of soon. However that video will be more like the "Deep Learning Cars" project rather than an explainer video. I appreciate every single comment and every subscription on my channel. I will do my best to upload new videos in the near future.
Sure, I went full open-source and used an animation software called Synfig for the animations and ffmpeg to aggregate the rendered pictures into a video ^^ Back in 2017 Synfig was still quite buggy though and the animation preview was only playing at ~5 fps at the end even though the animation was not too complicated and only 1 minute long. If it is still in the same shape, I don't recommend it, but I would be surprised if it hasn't changed since then. So might be worth a try. Ffmpeg is great for keeping crisp high quality images, if you know how to use a command line and got some time to learn the commands (however, I am sure there are also a lot of UIs for it nowadays). Nonetheless, I have to admit that I am a programmer, not an animator, hence take everything I say in this context with a grain of salt 😉
Masterful job! If this excites you, then a book with akin content is something to look at. "From Bytes to Consciousness: A Comprehensive Guide to Artificial Intelligence" by Stuart Mills
Yeah... sorry about that, haha. Progress has been very slow, but I assure you that there progress has been made on that video! It is definitely the next video in this series that will be released ;)
I used Synfig for the animations Audacity for audio I wouldn't recommned Synfig though, it was very slow and laggy in the end and the whole production process was not very effective. However, it might very well be that the current version is a lot better now.
