Aligning LLMs with Direct Preference Optimization 

I'm learning a lot from these talks, thank you for having them.

The best content I have seen in a while. Enjoyed both the theory and the practical notes from both speakers! Huge thanks dlai for organizing this event

Excellent description between PPO and DPO! Kudos

Excellent topic. Well explained. One of the best videos on this subject I saw recently. Continue your goodwork😊

This was amazing thank you everyone. One thing though, if thats possible, it would be greatly appreciated if you could record in 1080p where the details/text on the slides are visible and easier to consume. Thanks a lot again

Great presentation, thank you very much!

cut to the chase - 3:30 questions on DPO - 27:37 practical deep-dive - 30:19 question - 53:32

great presentation. Congratulations.

muy bueno gracias

At 29:40 Lewis mentions an algorithm that requires fewer training samples, what is the name of it? I heard "data", but don't think that is correct. If anyone knows, would you mind replying?

I'm wondering why simple techniques, such as sample boosting, increasing errors for high ranked examples, or attention layer wouldn't work in place of RLHF. It seems like a very convoluted and inefficient way of doing a simple thing - which convinces me that I'm missing something :-)

Curious if you compared SFT on all data vs. training on completions only?

Interested

ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-QXVCqtAZAn4.html The details in the Llama3 paper seem to validate the claim that DPO works better than RL at scale.

How DPO works under the hood: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-Ju-pFJNfOfY.html