Neural Networks from Scratch - P.5 Hidden Layer Activation Functions 

When pt 6 will be released ?

@@mayurpanpaliya He is waiting for you to buy the book haha

I am excited for part 6

@@bossragegamer4081 5 months now :(

6 months

Seriously! PUMPED encompasses all my feels as I follow along.

can you please explain how the activation point is getting changed by changing the bias? doesn't that flout the activation function which says y=x only when x>0?

@@Orchishman the bias here is essentially setting the activation point because y= max(0,max(0,-x+0.5)+0.48) will give you 0.48 for x being greater than or equal to 0.5 which serves as the lower bound for the function

@@Orchishman I created a graph so that you can play with the parameters and see for yourself how this is actually happening. I considered the first two rows and the last row of neurons only to make it simple (so 6 neurons in total in the hidden layer). I have numbered the neurons such that first neuron of first row has subscript 11, second neuron of first row has subscript 12 and so on (so, for example, the second neuron of the 8th row has subscript 82). Now to simulate the movement that is happening in the video, adjust the slider for the variable w_22 and see what happens with the plot. You will see where the area of effect of the second neuron comes into play. You can also adjust other sliders for weights and biases to see see their influence on the output. Here is the link: www.desmos.com/calculator/gruatlyner When you open the plot, the values of the weights and biases are the same as seen in the video till 17:01. I hope this helps.

ecpect something huge

Waiting also :), it will be worth the wait though for his quality.

Yes, please

I emailed Harrison about it, he says that he is finishing the draft (which is nearly complete) before continuing the series.

@@Nightmare-or2yd yayyyyy

It is all result of years of experiment, scientists just try to look for patterns.

Hi, Here's a link of my video where I've explained how relu helps in fitting lines to the data. ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-9t4TD5mcWSI.html.

For anyone that’s sees this in the future and agrees, this series generally balances practicality with understanding. I would heavily recommend also giving 3blue 1browns series on neural networks a look as that focuses far more on understanding and doesn’t really go into code that much.

For the spiral dataset, - features are the x-coordinates (x) and y-coordinates (y) of the points - In the code, there are 300 x and 300 y values associated with the 300 points - feature sets are the pairs (x, y) that fully define one point in the dataset - In the code, there are 300 feature sets - classes are the labels associated to the points - In the code, there are 3 classes defined by the colors - red, blue, green - and each feature set (x, y) corresponds to one of these 3 classes (with 100 points each) - samples are the combination of feature sets and classes that form the dataset - For example: (x = 0.2, y = -0.5, color = red) and (x = -0.5, y = -0.2, color=blue) are samples from the dataset Edit: Calling the function X, y = spiral_data(100, 3) creates samples belonging to 3 classes with 100 feature sets each. X (feature sets) is an array of shape (300, 2) and y (classes) is a vector of size 300.

Same here. That's the only thing so far in this series which confused me.

@@nishantsvnit Ahh so a "feature set" is essentially "the set of features which a sample has" but unlabelled?

@@iAmTheSquidThing You are right. But it is better to not call it "unlabeled" because that is a term used for feature sets that have no labels assigned to them (which was not discussed in the video). In the example in the video, all the feature sets have corresponding labels (i.e., the 300 x,y coordinates belong to one of the 3 colors). So to rephrase your sentence, you can say that feature set and label are the two components that make up a sample. If there is no label, the sample (or feature set) is called unlabeled. For more information on these terminologies, I would encourage you to see this: developers.google.com/machine-learning/crash-course/framing/ml-terminology#examples

@@nishantsvnit thanks

Yes😍

it IS good

same

@@sandeshtulsani1517 Same

Hey can u explain me what is an activation function atol

idk mate, probably not a good idea to put pressure on him to upload more

@@Hacker097 He is showing appreciation

Perhaps the author could help us out.... Hey @Daniel Optimizer Kukiela, please tell us about your optimizer!

Hello Daniel nice to see you here😂😂❤️

You are a legend man

How Does This Help Irene To SLAP ME"?" 👠😋😎

When he was saying optimizer, he was saying that the guy literally did it 'by hand'. So there is no optimizer, it was done by a human :P, if i understood correctly.

Lol PaderRiders interessiert sich für Nerdstuff XD

Thank you! Will do!

no stopped, sentdex is doin the draft first before uploading the new video i think

The link to the code is there in the description under "Spiral data function"

At 29:27 if you freeze the screen you can see it. I copied the text into Jupyter notebook and it worked.

aren't you the guy that was faster than light?

@@shauryapatel8372 yeah he was

He said expect it sometime between June 2022 and December 2038

@@OfficialRU-vid3 thts a long wait dude🙂

@@subratkishoredutta4132 Yes but Sentdex is a busy man... writing books, running a RU-vid channel, maintaining a website, did you know he is raising three different families on two different continents? (that last one is a secret)

@@OfficialRU-vid3 yes he is..

@@OfficialRU-vid3 that last one is wrong, he got a neural network to do the other two

Live success! Yay!

Why do you hate pep8?

@@kris10an64 it has unreasonable rules that shouldn't always apply. People are too strict with them. Raymond Hettinger himself agrees. Its suppose to make code more readable but its very flawed especially since python is already based on indentation. The 80 characters per line rule is the worst rule. If you have nested if blocks or if you like to work with lambda functions and iterators it can easily become long and well it can make the code blocky and ends up making it hard to read which is the very thing it was meant to avoid. In many cases following pep8 isn't the best option. Are you one of those pep8 absolutists? It also feels very restricting.

@@kris10an64 search for a video on youtube titled: "Beyond Pep8" where Raymond Hettinger talks about his dislikes of pep8 too and how some if its aspects are silly at best.

@@kris10an64 Here is how I like to code. Maybe it comes from me preferring C++ but these 2 functions can take 2 strings with hex values and xor them together: def stringXOR(a,b): return ('0' * len(a if a > b else b) + '%02X' % (int(a,16) ^ int(b,16)))[-len(a if a > b else b):] def stringAND(a,b): return ('0' * len(a if a > b else b) + '%02X' % (int(a,16) & int(b,16)))[-len(a if a > b else b):] Make that pep8 friendly and it looks like hell.

@@kris10an64 Here's another example. This is how I like to reverse a hex string: def byteFlop(hexstr): return ''.join(reversed([hexstr[y:y+2] for y in range(0, len(hexstr), 2)])) Show me a pep8 version that is better.

Working on the book atm. Videos after

@@sentdex sure No Problem!! I promise I wont learn Neural Nets from any other source Till you are back in business !! Warm wishes for the book !!

That's because he hasn't talked about backpropagation yet lol You should watch 1-2 videos of 3blue1brown on linear algebra just to understand how derivatives work, as that will increase your understanding immensely. Also, the math isn't that complicated as you usually just need to understand it once and then you can apply it globally to other network architectures as well, as they tend to operate on the same underlying principles.

You're very welcome!