CNN: Convolutional Neural Networks Explained - Computerphile 

damn, I thought I was the only one who read that book

Dog Barksley which book?

forshadowing

Word Press Forms is all you need!

Me too, I enabled the subtitles, Any way its a great video

+Sirus done >Sean

Thanks

Thanks. Annotated links are pretty much always off in the future. At least for nerds?

+GoodWoIf Yep. Mike tends to focus on image analysis in his videos because he is, in fact, an image analyst, but CNNs can be used on pretty much any data set you can think of--as long as you have enough data to train them and a decent number of possible convolutions to apply.

RGB was just the example used here, I don't think he literally meant they use RGB to produce different versions of the image.

Heck, three of the convolutions might well convert from RGB to H, to S, to I, and/or some combination of them.

right, the information is exactly the same so it might not matter to the Net

The type of input matters because it helps when the input correlates strongly with reality. You could map every possible input to a pre-determined random representation, and it would probably have a lot more trouble, especially with generalization. It helps if similar inputs correspond to similar outputs.

+linkviii lol, I actually wrote a k means implementation just two hours ago (in Octave, because vectorization rocks) and compressed a png with it

+linkviii As simplistic as K means clustering is, I freaking love it. Probably *because* it's so simple, yet can be quite powerful depending on the task.

+linkviii That's a weird assignment! You should have used the C code from the stats package implementation ;)

+tady159 The same way, only instead of using (R,G,B) as the dimensions to analyze, it uses (T, F, V)--that is, it uses the volume of each frequency interval at each sample time. With that data, you can do similar things as mentioned in this video.

+IceMetalPunk I don't understand what (T F, V ) stands for, could you please enlighten me? Also I still don't get how to process arbitrarily long sound samples. How their length can the be normalized before being processed by a network?

tady159 As I said, I was using (T,F,V) to refer to the dimensions Time, Frequency, and Volume. So think of a 3D plot, where the dimensions are (x,y,z), except here the dimensions aren't spacial, they're samples. Keep in mind that sounds stored on computers, when uncompressed, are stored as discrete samples, not continuous waveforms (sometimes you can build a waveform up by connecting the samples, but that's interpolated). So to normalize it, you just break the sound into frequency intervals using a Fourier transform (smaller intervals = higher resolution), then for each interval, you divide every sample's volume by the maximum volume over all samples for that interval. Voila!

fake news

you better be quick - another few years and there won't be those things to survey...

Dick Hamilton Too true man! Let's hope we're wrong on this one prediction, right?

+greenrocket23 with you, there.

no

+Alcesmire Aha! Now that I know neural networks exist, I can start my Mathematics thesis with: "Let N be a neural network..."

+Alcesmire That's actually pretty awesome! It could seriously improve the lives of deaf people, especially seeing as how we're moving more and more into the whole voice-controlled, NLP virtual assistant world.

Frankly, I don't see the wisdom part. Sure, when you design a NN, you do have to scale it correctly to the problem and so on, but once you've got everything setup, the rest is magic.

Is'nt Nueral networks just math? Ive studied the backpropagation algo,stacked neural networks,etc..etc and the thing that struck me is that its all just math that you learn in an engineering course especially stats and linear algebra to solve equations. Why did you say its "witchcraft"?

Sreekar Nimbalkar Because NNs are large sets of linear or non-linear equations with dynamically generated coefficients that mean absolutely nothing to the designers of NNs, but somehow work. Hence witchcraft. I mean, I could basically simulate a NN on paper and it would still work, while I would still have no idea why it works.

YES!!!!!!!!! GRANT MONEY!!! PAY ME! This is an ACADEMIC!!!!!!!!!!! $$$$$$$$$$$$$$

this is the best comment i've ever read. I feel you bro. Lived it!!!!!!!

SaltyBrains I don’t get how that changes the meaning at all.

Another xkcd fan?

@@KnakuanaRka That changes meaning because a computer did it. Now you can automate that process and probably hundreds such processes. Automated object detection can be used in a multitude of processes and industries. Google it and surprise yourself.

i dont understand what you mean

2 deep 4 them + inception

not 3 deep 5 me anymore tho

+turarwanaa Lucky you, watched it twice and I still think he is a dark wizard.

+turarwanaa +J Simmons But doesnt it mean that its all just an optimization program that does the magic? The training images get run through the process, it produces a value. The settings are tweaked slightly, the result are compared and one is better than the other. Rince and repeat. Working with some optimization programs myself, the trick is in how the algorithm is programmed to make large or small tweaks to settings... Its like finding the tallest mountain in the area while blind... Does crossing the valley lead to a taller mountain or should you just go up hill? It all seems CPU horse power dependent to me o.o

+HexerPsy yes it's pretty much brute force, no magic. as with most things machine...

Thomas Gandalf It isn't anywhere near brute force either. The "magic" is in why neural networks work so well at all compared to other methods. At a low level it looks fairly similar to other optimization methods, but the structure of the network and how it abstracts is very important. It makes much less sense than a naive perspective suggests.

+Vulcapyro something that runs to iteratively tweak parameters of mathematical formulae until it finds the best possible solution, i.e. explores or exhausts the state space, is pretty much the definition of brute force.... granted CNN don't usually exhaust the state space but make informed decisions on which parameter set to try next

we need to go deeper

this is amazing

Moeシt wow

aren't we all?

I am a neural network inputting and outputting comments about neural networks watching videos about neural networks. The singularity is nigh.

fidelio your profile pic wins the internet for the day!

I guess if you applied the sharpen kernel it would make things more clearer

This is hilarious.

Google where is A visible date of Video oooplooopad inthisRU-vid

...because Computer Scientists are only allowed to reference one Operating System? I don't get it.

Linux is better for long(multiday) computations because it is more stable and uses less resources in the background; it also happens to make the project more reproducible because versions of linux don't become dysfunctional with time like windows. Even if a CS has windows on their personal, there is no reason to think they wouldn't use linux on there workstation.

agreed he is not capable of this but he doesn't care because he has WORK TO DO.

The best way of learning is building stuff that you want to understand😉

@@IgorRoztr absolutely agree. i wouldn't say i understand something unless i've built something like it

Hey... I tried to implement a cnn from scratch using python... But it is not working properly... I found few issues but not the solution for it.. I searched abt it in every website but couldn't find proper solution.. can u clarify my doubts, pls?

I jst want to clarify whether whatever ik is right or wrong and want to know the solutions for the issues

2 minutes paper with Dr. Gibberish

+abschussrampe Google is currently pushing them onto basically everything. I'm not sure that _all_ their services use them yet but increasingly they do. For the following they either have talked before about _planning_ to use this technology or they already use it directly in the services you may or may not love to use by them: Google Maps Google Car driving Google Now suggestions Google Search RU-vid Thumbnails RU-vid video suggestions Google Translate Google Photos DeepMind (the guys behind AlphaGo) Allo, their new messaging app probably lots more Other big guns who either talk about or already do use these: Apple Microsoft Facebook Amazon probably lots more Nowadays, if you are on the internet, chances are you are using a service that in one form or another relies on deep learning and convolutional networks. You can do an insane number of semi-cognitive tasks with them. They _do_ have their limits in their current form but development goes rapidly.

+abschussrampe They're used in quite a few places. One example I saw used a CNN to classify the activities occurring in a video--for example, to learn how to tell whether a video is of someone hiking, mountain biking, swimming, or canoeing. It could be expanded with more data sets to classify other types of activity, which in turn would allow our future AIs to understand what's happening around them instead of having to be told what's going on before knowing how to react.

+abschussrampe Self-driving cars (Autos, you might say) are essentially all implemented as systems of CNNs at this point, if you want a particular example that will likely significantly change our way of life.

If you're outside the EU/Canada, Facebook uses CNN for facial recognition to tag photos

Rest assured the government is using them to identify not only who you are but what you're doing. We're going from facial recognition to activity recognition.

+syawkcab "This is kind-of a follow up to Brais' video on deep learning"

OHHHH!

+Jerrod Milton You are correct! How they basically do this is instead of adjusting the weights of the network to get the correct output value, they change the pixels of the input image, so that the CNN predicts e.g. a cat with higher certainty.

+Karl Kastor Wait, this exists? Are there front-ends to those applications available?

Benjamin Philipp Google Deep Dream. People have done several implementations for this since the original paper.

Benjamin Philipp google Deep Dream. People have done several implementations for this since the original paper

Karl Kastor Tanks - I've since found Deep Dream, but thanks for coming back for me :)

Wowthatsfail Truth

+Christopher Willis really interesting way of looking at it.. what an exciting time to be alive!

+bibbly bobbly Thanks! Very interesting. I once read that the patterned hallucinations of LSD are probably caused by the acid disrupting the signal in our retina. Pretty interesting stuff.. Is that plausible in your opinion?

+bibbly bobbly thanks well said

As far as I know, the kernels that CNNs learn on the first hidden layer (without prior knowledge) are also very similar to patterns our visual cortex reacts to. So it's maybe even closer than you thought it is. I definitely find this quite fascinating :)

The part of the brain responsible for most mammalian intelligence (the neocortex) is considered hierarchical by many, sort of like neural networks. But it's not like there's a single layer of neurons in each level. Each level has millions of microcolumns of neurons (around 100 neurons per column), 3 to 9 or more layers (depending on how you count and the location in the hierarchy) with distinct properties, and multiple types of connections (inhibitory, excitatory, modulatory, different durations of effect, etc.) There are hundreds or thousands of underlying common characteristics in the neocortex alone, whereas neural networks have maybe one or two dozen underlying characteristics. I suspect some of that is just plumbing to deal with things like metabolism, but neural networks (except hierarchical temporal memory, which doesn't do anything the brain definitely doesn't do) are pretty unlikely to lead to brain-like intelligence. They're really useful, but the way they are designed is like trying to reinvent the computer by mimicking transistors and no other characteristics of computers.

+Jamie Twells Ah, they're there now - Thank you!

*****​ to be honest I wish RU-vid would get rid of annotations, the i and everything else that pops up over the videos. I just want to watch the video, annotations are too often abused and spammed to be useful. 

Idk how your game went, but you're the man.

How is that relevant?

qwertyuiopzxcfgh This is the internet, so to bash COD is relevant.

Jimmy De'Souza Try and watch the trailer for the new Call of Duty Infinity Warfare and see the likes and dislikes. Also watch the new Battlefield 1 trailer.

Transformers too