Meta-Learning and One-Shot Learning 

Glad you like them :) I try to aim these things at people who are very interested but don't have enough background knowledge to read the literature directly. I figure, even for people who are more involved in the field, they can listen to my overview first and then actually look at the papers if they are interested.

Any kind of RNN is worth trying when it comes to meta-learning. However, for the particular problems I talk about in this video, training sequences are only 50 to 100 timesteps long. For such short sequences, I wouldn't expect uRNNs to outperform LSTMs in a significant way. Models like uRNN are optimized for extremely long sequences, not necessarily for fast high-bandwidth recall. The links in the description are a decent start for one-shot learning. I believe you can also use Google Scholar to see what cites those papers.

Technically speaking, I haven't declared a major yet. I was thinking about CS, but spending so much time in college "learning" something I am already good at seems like a waste. Either way, I disliked the college experience so much that it drove me to take this semester off--something most people don't do.

Funny, you just described my experience in a nutshell. I also took few semesters off before declaring a major in CS of recent. Its challenging but fun if you make it so.

The macheads101 twitter account is pretty much dead. I (Alex) have a personal twitter account @unixpickle, where I often tweet about ML.

Currently taking a semester off from college to work on ML.

macheads101 Within 20 years it might be some kids school that closed down for months, and then a student sits at home and figure out a total new paradigm for cognition and AI. Would not suprise me.

The language I use the most is Go (a language developed by google). Most people seem to use Python for deep learning these days.

Interesting, I was thinking it was the Hebrew Beit. I wish I remembered which alphabet I took it from (it's from Omniglot).

Long term planning/reasoning is kind of an open problem. If you want a machine to learn this behavior on its own, you need an environment that is complex enough (and long-running enough) to reward good planning. Also, our current learning algorithms aren't good at spanning long time dependencies. As far as communication is concerned, there is some work being done on multi-agent environments (e.g. some work at OpenAI on learning to communicate).

Thanks, but let me ask one more. I'm not too happy about how "deep learning" is mostly about CNNs these days, because their response is like single firing of reflexes. To me, CNNs are fixed equations with holes (paramers). Should we not have nodes that represent loop variables (change the form of the equation)? Is there some RNN where the firing signal travels conditionally back and forth, depending on the specific input, and the response comes out at a variable time?

Imagine making an AI agent playing Solitaire, but it's allowed to see only one card at a time. It has to emit arrow keys to navigate and see other cards. Does today's definition of NNs allow successful play Spider this way?

Having something learn the structure of external memory would be interesting to see. I almost feel like neuroevolution would be the best approach for that. As far as that "user" idea goes, I'm not sure how that would really differ from current memory augmentation models. If the user is a neural net that acts as a middleman between the controller and the memory, why couldn't one just say that the user is part of the controller? It's not uncommon to have a few neural net layers process controller output and turn it into more direct memory queries, but that's still considered to be a part of the controller. **EDIT:** when reading your comment, i mixed up what you called "user" and "memory controller". Same point applies, though.

I'm guessing you don't have a second friend who's obsessed with machine learning.

Nah the second friend talks too much about smash.

Wow Justin Burdge (if that is your real name)! You think your friend is hot shit? Well I have a friend who is hotter shit than your friend... Nurd!

I live for comment threads like this.