I clicked the video thinking.... 😑 Oh look another "AI Expert". I invented LoRA.. Ok sir you have my attention. Hard to find real experts out here. Glad I clicked 😀.
"Huh, what's this guy's deal? Well, GFlowNets sound cool, let's check him out" "I invented LoRA, Prof. Bengio was my PhD Advisor and I'm currently an OpenAI Research Scientist" "Ah, so 'the real' is this guy's deal...gotcha". Easy subscribe.
Fantastic video! No hype just the intuition behind current research. I love this type of content because it sets the foundational thinking required to make sense of research papers (which are normally opaque to someone who isn't deep in the field of study)
Hey, I’m glad you’re on RU-vid! It’s great to have an expert in the field break down complex topics and intuition and also explain real world applications not just theory. I think it would be nice if you could have some summary pointers overlaid in the video or on a summary slide with you overlaid on it. Different people have different learning styles and being able to read helps.
The kitchen background is awesome. Just a better mic is all you need. Love to hear abou other reasings in addition to your lab's works. Do you accept visiting researchers by any chance 😊
So, articulating this in a way that makes sense to me: In traditional machine learning it's pretty common to treat the weights and/or biases (where relevant) as learnable parameters, in the sense that they'll be optimized by the chain rule to produce the lowest loss answer. We are not limited exclusively to modifying weights, though. You can introduce a low rank matrix (I wonder who came up with that? 🤔) instead of modifying the base weights, but you could also treat the prompt, or sampler parameters as learnable parameters, too, leading to interesting and occasionally useful results. In this case, it's taking it a step further, and training a sampler model, and optimizing it for "good pathways", which is clear to see in the example of reasoning; If one "pathway" leads to the model just giving an answer, that's bad reasoning and won't reliably be correct (Maybe you should call up one of your elementary teachers and apologize for not understanding why they made you show your work!), and if the reasoning doesn't follow a logical cadence ("How many rabbits are there in New York in the spring?" "Well, the sides of a Tie fighter have been hypothesized to be Ion thrust surfaces and solar panels...") that could also be recognized as a poor fit by the sampler model. It'd be very interesting to see how this technique compares to something like training a dedicated resolver model to take the best reasoning steps of 1000+ generations (scaling with test-time compute), because this looks very similar to my eyes, though I might be missing something, as I'm not a true "AI researcher" or the like.
Comment for the algorithm 😊 I'm just a layperson in this domain but still found the concept interesting and relatively clear to follow! Makes me wonder whether there is a finite population of reasoning pathways that can solve any logical problem, or if thats even a relevant question to ask. I'm not familiar with the last concept you introduced about maximum entropy.. sounds intresting! Thank you
I think the intuition is that solving logical problems is basically searching for a valid reasoning chain (the space of reasoning chains is practically infinite IMO, tho). the intuition for max-entropy RL is that we want to explore all kinds of chains before committing to a particular subset.
Sir, please make more videos explaining fundamental concepts as well as latest breakthroughs in AI field. Your style of explanation is neat and pin point. Everyone will get benefited out of these videos.
Awesome!!I am about to start my PhD in cs and stuff like this is way motivating and inspiring…keeping sharing your research through videos…I literally read the Lora paper…then watched your video then went back and read the paper again and then came back and watched the video again
Did you just casually explain what Yann LeCun vaguely hinted at? Where are all the cameras? Internet is a weird place, I'm watching the one-ring-to-rule-them-all being explained in under 10 minutes. So cool.
I found a way to do this at my company with just LLM's! I used LLM's to not only generate the process steps but also make a liminal/expansion map of the ideas themselves that could lead to those individual processing steps! The trick to make it run wasn't to fit to a "reward function" but to use the "reward function" for back propagation as the starting point to then determine what was needed from the user to retrieve and select the correct path!
Thanks for including subtitles with this video. I’m pretty sure (but not certain given you may have access to better models than regular Joes) that it’s been manually edited or written from scratch. If so, extra thank you!
Would love it if you did some implement from scratch videos in pytorch.. teaching us how to innovate, by teaching us how to implement ideas from scratch such as LORA, and Energy based approaches
i was trying to come up with something like this, in that i wanted to train a generator which would be the inverse of a classifier, and the classifier gave a score to how good a solution was drawn from some range. this looks miles and miles more sophisticated than what i was doing with tf and pytorch, but i definitely understand, at least on that level of abstraction, why such a development is necessary. i look forward to trying this, cheers.
what could be a powerful companion to the context of this video is a voice-centric personal assistant. It has enough of a "Context profile" on you to judge inflection, tone synthesize inference and intent with SOTA voice recognition tools just laying around. this would at least morph the zero-shot/few shot dilemma in prompting every query.
Very very impressive video Edward! As a startup founder in the Gen AI space, it's hard to find youtube channels that go further into the future than what we see in the market. Such video can certainly change someone's product roadmap :))))
Thank you so much for the video Edward. Its an inspo seeing you make videos and take off. Im currently a masters of science student in data science and Im always excited about NN architectures and new ML algos. Whenever easy, could you please share 1-2 tips for writing good research papers in ML? I recently started in a lab that does neuroscience-ML, and rlly want to make an impact there
Very interesting stuff, I'd love to use this algorithm to see how it works. Just a little note, one thing that would help a lot is reducing the reverb/cleaning your audio a bit. I'm sure there are a lot of ai tools online that can do it quick. Thank you for making this educational content!
Amazing explanation with clear explanation of the intuition. It would be great if you give us one more vedio on implementation or detail explanation of how Gflownet is used to enhance reasoning of LLMs (i.e the paper)
Just found your channel, I'm 100% impressed bro. Just a tip for the audio, if you don't have a good microphone, just use a free ai for speech enhancement, it works great. RU-vid ban the comments if you say the name of the AI, because it's also the name of a company apparently :/
thx for the feedback! i actually use a Rode VideoMic. (i'll fire the guy who processed the audio.) can you DM me the tool you know for audio processing? you can leave a message on my website edwardjhu.com 🙏
Please keep doing these videos explaining how state of the art developments on AI work. Hell you explained it so well I'll listen to any ai concept you decide to explain. Subscribed!!
Brilliant topic and explanation! Looking forward for your deep dives and if you like some observations or insights you have made on your saga as a researcher 😊
It is so cool that you made something so abstract to me that just magically works (lora). No idea how you even begin to understand the concept, let alone the code, of ai stuff.
Sir, with all due respect ,please don't take my question in the wrong way. How can a total newbie get started in AI ? What are the books you would recommend reading and the things Id need to do? Sir I really appreciate the clarity and intensity of your incredible thought processes. Love you sir.
This is exactly what I was thinking about a month ago, the combination of gflownets for active chain of reasoning in llms and I was about to call it out that this is how Q* from openai works and make a paper.. And what do you know, a bit later people from openai start talking about it :)
Here's a "Below are some new topics for you to discuss: - Grok-1 - Claude Opus - Gemini Ultra - Poe - Figure humanoid powered by OpenAI - Hugging Face - Sam Altman 7+ trillion - Google AI in iPhones - NVIDIA Disney robots Let me know if you need any help to choose the best topics for your content."
I believe you can bring a lot of wisdom to the table as a researcher, maybe you can talk about the future of AI based on your experience, like do you think AGI will get developed anytime soon, or some major problems the researchers and developers might run into that might hinder progress, or how soon will we reach some sort of ceiling?
Hello Edward, Thank you for this presentation. [Subscribed]. GFlowNet sounds pretty good after listening to your explanation. To be clear, are you saying that it includes discovery to prevent it from getting stuck in a local maxima? Also, If possible, please check your microphone. It sounds like it is partially under water with some echo.
Hmm for robotics, that may be good for learning to control the robots. You only have a few samples of live data. Most RL relies on big datasets and is a no go
Neat! If you've heard of this sampling as "quantilization," you might already know that the safety / "normalness" of the training distribution can get diluted if you do repeated sampling (as in search through a tree of possible sequences). Any plans to tackle this problem?
Is the very high level idea that if you could generate a large number of diverse samples which minimize a loss function you are increasing the number of different local minima you're sampling from, thus reducing the chance that your trained model hit a dumb, or extremely deep minima?
BTW, I have been working on a graphical way of building Ontology models that I am calling UniML (Universal Modeling Language) that can model the GflowNets process quite readily much like OWL or RDF graphs, but even more graphically. Something like this might help develop these nets in that it is a way to better visualize them. I made a number of introductory videos that I uploaded on my RU-vid channel that describe them in more detail into their what's and why's plus a Padlet virtual corkboard that takes a bit of of a closer look at them. UniML can model NNs as well, or just treat them as functions (Chips).
This might be dumb but I wonder how applicable can GFlowNets be in an object detection/instance segmentation context where the distribution created is based on the millions of images the model will train on and the "pathways" of the help the model be even more accurate
Hey there! Can we talk about Q*, the AGI developed by Sam Altman? I'm excited to discuss this with you, it's one of the most interesting topics for me after Sora.
Thank u brother, just a quick question, how do u get access to hardware? And how can a independent ai researcher access hardware and find interesting people in the domain, thanks🌹 mu pi was awesome too ❤
I get lost right at 3:45. If we’re sampling proportional to the reward function do we still not end up with the “hacky” maxing sample occurring the most? Is the idea that at this point humans take over and evaluate the that batch and can easily disregard the “hacky” result while still getting other useful results? In LLM outputs we already get log probabilities. Would this be a superior way of selecting the best choice given the log probabilities?
yes we will get a lot of the hacky samples but some samples might actually work (versus none will work if we maximize). also one can raise the temperature of the reward distribution to get more diverse samples.
Great video, but audio can be improved! I suggest you buy the DJI mics, Rode Wireless Pro, or HollyLark M2... Or find a different room to record in with less reflective surfaces. Those mics are small enough, good quality, and can clip on to your shirt.
So if I understand this correctly, this is about optimising the data that we feed into the model right? Taking the right distribution that allows max likelihood for a certain output we want?
This sounds exciting, however I still didnt really understand the problem this is trying to solve. Could it be used in rl like training a nn to play snake more sample efficient? I guess I am trying to say try to rephrase it like I am 10 or using a subject that I at some knowledge about. Thx!
that's cool, so if I'm understanding this correctly, you're training domain specific knowledge while maintaining the broader perspective of all the training data? that's cool. that's cool.
