Geoffrey Hinton talk "What is wrong with convolutional neural nets ?" 

In a way, capsules are the encoder-decoder model applied to computer vision. Convolutions were constraining the network on how to encode an image. But capsules can evolve whatever model they want, to achieve an optimal encoding model. This is great progress. Encoder-decoder architecture has been applied successfully in NLP as well - see thought vectors which have achieved soa in MT

Very recommendable! Some of Hinton's critique of convolutional neural networks is worth pondering about

How would the transformation capability of the capsule be useful to recognizing plants/trees that don't have a fixed structure?

This might be my new favorite talk

That makes so much sense. I always felt it was weird and wrong that NN did not threat the objects in an image as constructs of their own and only a part of soemthign else as another level of interpretation.

The paper just came out. Today!

Question: How does linear transformation work? Answer : 21:27

At 58:00 it appears the model is being trained on the image in various orientations. Wouldn’t a CNN model perform as well if it were trained given the same orientation rotation? I do understand his linear transformation argument but not sure this would work any better.

Somehow this makes me proud to be UofT graduate student

Next time I see a shoe I'll think of matrix determinants lol. Fascinating how Hinton analyses NNs.

Similar concepts to Jeff Hawkins. Terms are slightly different. Would love to have these fellows cooperate and accelerate their work.

does anyone know is there any papers about capsule？

I've blogged about this post here: moreisdifferent.com/2017/09/hinton-whats-wrong-with-CNNs

Please please make the camera view static...

Question: Why is the squashing x|x|/(1+|x|^2) not x/|x| * tanh(|x|) ?

I noticed Hinton says he's going to take "Marian perspective" -- (not sure if I am spelling it correctly). does anyone know that that means?

This is very similar to the Recursive Cortical Network.

14:35 Carl Hewitt, the Carl Hewitt? The one of Planner and actor model fame?? Damn, it must be harder than it seems!

I'm curious on the psychology of that letter r. I didn't rotate it as Jeffrey said I performed a flip. Idk wether it's a spacial reasoning skill or what

I've been thinking along the same lines, but wasn't able to formulate a clear explanation of my thoughts. An excellent talk by Geoffrey Hinton.

True genius destroying the field he helped build

What does "EM" mean and what are "mini-columns"? It's so annoying when people use terms without defining them properly.

I like it! But.... does it work?

The tetrahedron problem solution is incorrect because the faces are of rectangular shape and they don’t fit each other. To be solvable, they must be squares. That’s why the MIT professor was right.

Does capsules really work and practically applicable?

At 28 minutes Dr. Hinton says "So I guess this is another piece of *an aitism*"?? Couldn't make out the word in asterisk. Did someone catch what was said?

In democratization of Deep Learning, people segmented it into CNN, RNNs and so on. These essentially are architectures. As they work, they also prevent people from thinking beyond them. In 'capsule' paper, Hinton has simply proposed a new architecture. Its not innovation, its just thinking from scratch. It was mystery to me as well, when reading text from natural scenes, how can there not be an algorithm which will read the position of each letter and simply combine. I guess that problem will now be solved which earlier had low accuracy.

It is very close to wave theory in cortex to organize associative memory in humans brain by Redozubov

as he said.....if a object lost part of itself. what will be happen ??? ex: a face lost a nose. which mean ...... a face isn't a face cuz missed part ???

i like it

Maybe "crowding" is why some people really hate crowds.

Hinton is burying an ax on MIT profs. ;-)

I solved the tetrahedron problem in minus 10 seconds before I even saw it. What I did was think about what shapes you could get if you sliced a tetrahedron. Either a triangle, or a square. Then I though, well a triangle would be too easy. So it must be the square. And the two shapes would be at right angles. Mind you I never been to MIT. I got the Africa one in 1/2 second too. Can I have a Nobel Prize. Thanks.

怼cnn

I'm an innate-ist. You only have to look at how a giraffe can walk after 1 minute of being born.

He talks too fast and indistinctly. I can't understand about 30% of what he says.

Hinton is an innatist at this point, but doesn't want to admit it.

AI is as useless as the relativity theory or as quantum physics.

Hinton was great... in 1988. I don't understand what progress he has made since then his ted talk was no different. I hope Google has helped him get more up to date as he is a very bright guy. The convnet feature detector and pooling is so primitive versus a holographic excitation model we use at noonean. "The fact that it works so well is going to make it hard to get rid of" - thats one of the smartest things Hinton has said. these simple backprop Convnet approaches are just wrong, slow performing, and dont manage large numbers of features well. They work for what Hinton and Google did with 256x256 bit planes. But show me all the faces in an image. show me all the triangles. nope it doesnt do that well at all. I do not think hintons approach can differentiate between a set of corners/angles and a triangle. It is very interesting that Hinton uses digits as his recognition sample which he has done now for 30 years, but other than Zero there is no notion of continuance. I would warn people to take Hinton with a grain of salt. He is not a great cognitive scientist as he does not follow brain model to design. Holographic paradigms with active reinforcement at the neural level withing a recognition plane, with scanning across a larger image, work better than convnets or Hintons approach. Rather than digits, I'd like to see just a solution for pure geometric shapes. Show me squares versus ovals versus triangles versus corners in a large scale photograph. Show me all the eyes vs noses vs mouths. show me all the faces with eyes/nose/mouth in the right spatial relationship to be a face. By working with digits, Hinton is just being too simplistic. Probabilistic models may suffice in that domain. His early research on solving orientation-less recognition was quite good. But I just don't see what he's added in 30 years. At Noonean, we use a edelman-pribram holographic model as our "capsule" not covnets.