I love your work! Yours is the first online AI class I took ~7 years ago, even before I knew who you were! Thank you for being a constant teacher and inspiration.
Thank you for sharing these interesting ideas! As a junior software robotics engineer with keen interest in RL/Robotics Learning, your papers & materials have been an inspiration to me and an source of optimism in the field of robotics when I've questioned it. Excited and looking forward for more to come from you and Covariant! (And also thanks for the great Podcast!)
It feels like robots are still missing some sense of "why" in all these combined approaches. We have neural networks which can conceivably handle concepts of experience/curiosity (RL), and association (transformers, image word, etc.), but there's no model that can really handle a surprise, create a hypothesis for the surprise, and then test the hypothesis (or make new ones and test them) until it is satisfied with why that experience occurred. You could argue that a robot does not need to understand "why" something happened, but I think that until it does, it will be quite brittle and unable to handle a variety of edge cases or unexpected experiences without a considerable amount of human directed fine-tuning. All that said, perhaps I'm just summarizing the area of continuous learning. Hard to say.
Thank you for this presentation. The CURL and RAD paper really show the exiting potential of unsupervised self-supervised learning and data augmentation for robotics. A question: According to your statement in regards to the gap (16:20 -17:35 in results between image-based and state-based on the hard problems), do you feel (in those situations) it’s not possible to extract more useful information and close the gap, with maybe a different auxiliary task like a non-contrastive one? I feel like a generative model would learn more, but heard Aravind Srinivas mentioned that a reconstructive loss would potentially demand more focus (compared to the contrastive loss) over the RL task, which would in the end have a negative effect on the sample - efficiency. Are Generative model not right in this framework, as presented in the CURL paper?
Thanks for the amazing talk! I wonder what your opinion is about using RL methods in the animal world ? I've studied animal behavior from a neuroscience perspective (specifically bird song learning). Its a motor learning task (bird learns to sing like a model bird). The reward function used by birds is unknown. Similarly, there are other learning problems like monkeys learning how to manipulate new objects. Can we use modern RL methods to describe goal directed complex behavior ?
@Pedro Abreu indeed, it's likely something similar to CURL (or RAD or DrQ or SPR) could be used in audio space for this (whereas the original works are in image space); that said, there is still the other question even if how these birds (and other animals/humans) come up with their own reward functions, which is also really interesting to think about
Unfortunately the hardware required for such a general framework for embodied robotics is not yet there. We could use 5G + centralized cloud AI hardware which would result in some sort of costly & dangerous SkyNet.
definitely not looking for skynet; and more compute will always help; but I'd still be curious what kind of useful, fairly general systems we can build with today's compute if we were to follow the general ideas from the talk...
