What do you mean it’s not sparse? The convolution matrices are full of zeros. Check out this video: Matrix multiplication, signals, and convolutions ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-d2GixptaHjk.html
Hey Alfredo, Hope you are doing good. I'm binge watching this awesome lectures, and even though I know the concepts a little, there is still so much which I didn't know. Do you think the commoners can access the assignments that you give to students? Thanks!!
Hello Alfredo, hope you are doing well. I really enjoy your content and appreciate the effort that you put into the making the presentation. If it's possible,i would be nice if you can make a video on how actual research is done in this field. Let's say i got some idea, then do we perform rigorous mathematical formulation of the idea and try to come with with some mathematical solution to justify the idea. Or do we just implement the idea using deep learning libraries?🙂
Yeah, it goes the other way around. You play around with something. Usually nothing works. You try to understand the little that actually worked. You write down some math, describing what you did.
Thank you for sharing. I'v one question about the NNs on scrambled data. If I had to make a prediction I would have said we will have an accuracy about 15%, not more, thanks to the amount of pixel that can help to determine which digit is corresponding. So is it enough to get an accuracy of 83-85% or there is something else? I supposed that the fully connected neural network would have duplicate the filters, but there is no change with the scrambled data.
@@alfcnz Yes of course. If think my french explanation was not clear either. I would have assumed that with scrambled data, we would have had an accuracy of around 15%, not more (which is more than 10% thanks to the fact that by counting the number of pixels, the model can have an idea of which digit is the most probable). I have trouble understanding how the model can achieve as "good" results as 85% on scrambled data. Does the model count the number of pixels and determine it that way, or is there something else? I had assumed that in reality, the dense model would have worked like a ConvNet by learning the same kernels multiple times. Essentially, we would have had weight redundancy to get something similar to a ConvNet. Is it because of the lack of parameters in the dense network? If we had given a lot more parameters to it, would it have come back to having a ConvNet with weight redundancy to "simulate" the filter's movement? In french: J'aurai supposé qu'avec les données brouillées on aurait eu une précision d'environ 15% pas plus. (Ce qui est plus que 10% grâce au fait qu'en comptant le nombre de pixel le modèle peut avoir une idée de quel chiffre est le plus probable. J'ai du mal à comprendre comment le modèle peut avoir d'aussi "bon" résultats que 85% sur des données brouillées. Est-ce que le modèle compte le nombre de pixel et le détermine comme ça ou il y autre chose ? J'avais supposé qu'en réalité le modèle dense aurait fonctionné comme un ConvNet en apprenant plusieurs fois les mêmes kernel. En gros on aurait eu une redondance des poids pour avoir quelque chose de ressemblant à un ConvNet. Est-ce à cause du manque de paramètre du réseau Dense ? Si on avait donné beaucoup plus de paramètres à celui-ci, est-ce que ce serait revenu à avoir un convNet avec une redondance des poids pour "simuler" le déplacement du filtre ? Thank you
There's a lot going on in this question. First, let's address the fully-connected model. The model does not care if you scramble the input or if you don't. If smartly initialised, the model will learn *the same* weights but with a permutated order. That's why the model performance is (basically) the same before and after permutation. Until here, are you following? Do you have any specific question on this first part of my answer?
Hi Alfredo, do you know a course similar to yours but on RL ? By similar I mean a free class with lectures, recitations AND code; I don't refer to the elements that are only in this class, these are unique :) Thank you !
@@alfcnz Sorry, what I should have said is that "i" is the index of a channel. C1 is the first channel, C2 the second one etc. And for a specific kernel, C1 * w * h are different weights from C2 * w * h.
Good point! A perceptron assumes an Heaviside's step activation, which is clearly non differentiable. So, this name is frown upon in Yann's lab. (I used to be a former user of such term.)
