Unfortunately even the so called power users of LLM with their own RU-vid channels, always seem to have a small set of stock prompts, which get repeated with every new review. If LLMs were trained on these specific questions then they're going start appearing super intelligent! Things like 'why is the sky blue', or write a 'snake game in python' are hardly a test of machine intelligence, as all that is needed is to be trained in accurate code or factual data.
Excellent! I didn't know you before and I just stumbled upon your video while searching for material on prompting LLMs (for a local LLM project). You now have a new sub :)
Excellent review! Thank you. I really enjoyed this book and have been recommending it to people in order to help them set expectations about “AI” without excessive optimism or pessimism. Currently reading Deep Utopia; hope to see your review soon!
Thank you for the video, that was a great paper to go through. I find RAG research techniques have so much insight to how we can develop and identify our own cognitive impediments to our own judgement. The Comprehensiveness, Diversity of perspective, Empowerment and Directness is such a good mental model to use in our own human judgement.
great presentation Vivek. Some questions: - is graphRAG production ready ? if not, would it be difficult to upgrade RAG methods once we are in production ? - is there a RAG provider/stack that you prefer ? (datastax, pinecone, weaviate + a bunch of others who are all competing for attention) - what are your thoughts on LangChain vs LangGraph ?
can you pontificate on the combination of upcoming transformer inference ASICs with deep agentic workflows employing GraphRAG style strategies? Seems like we will be close to our personal assistants writing a PhD thesis in the background whenever we ask a question. SOHU is reporting 500,000 tokens per second with Llama3 70B....
Seems clear that for 'current events' rag is going to win, but for broader, domain specific themes or logic, how does fine tuning stack up? E.g. create code using our internal suite of APIs... If context is big enough, icl should be fine, but rag may miss some key docs based on semantic similarity alone... I guess... I should write a paper 😂
It's surprising to see ChatGPT achieving below 99% when Wolfram Alpha can basically answer anything just by having specific knowledge. It's also surprising to see that "playing" with the word prompt does anything at all yet alone give a better result. It makes no sense especially when we can clearly see from the research the information entropy is basically the same between prompts with and without extra steps.
wouldn't it cut to the chase to train an llm on your own data? theres your graph use one of these OpenAI's GPT-3/4 Hugging Face Transformers (e.g., GPT-2, GPT-3 via third-party providers) Google's T5 (Text-to-Text Transfer Transformer) Meta's BART and BlenderBot Anthropic's Claude every week update the llm summarization is the death of real data, better off one level of summarization? Just a thought!
same here brother. There are so many techniques, everyday I learn something new which is both good and terrifying ha. What stack are you thinking of using ? We are researching DataStax, Pinecone, Weaviate and are learning to build agents with LangGraph.
Thanks for sharing the paper. In my experience with using Llama3-8B, in my benchmark dataset, I noticed that LLM has learned an incorrect fact or in contradiction with my application. I tried to clarify that in the prompt, but noticed the LLM is actually quite stubborn, and lead to quite fragile responses, i.e. the LLM sometimes get it right sometimes get it wrong with minimal changes in the prompt, could be as small as adding spaces. I wonder if you have come across similar situation or papers that discuss this behavior. Thanks.
You ask vanilla questions if you can't un-cloak a machine response!. The reasoning is not there with language models, how stupid are people to not be able to ask the right questions? I call lies on these claims. Show the test as proof, I doubt you can or will show the actual test. This is absurd.
But why is the paper's fine tuning different than the original pre-training and alignment fine tuning that came before it. All expose the model to a mix of existing and new data...
You are correct -- in principle fine-tuning works the same way as pre-training (updating weights), so FT can be thought of as continued PT. Difference is in data used. One will FT when they have a domain-specific set of data that's very different from the PT data.
Just wanted to say I really appreciate your videos. Everything is short and concise and I love that you’re always using papers as the foundation for the conclusions. Keep it up!
Good brother 😊thanks for an excellent explanation , Peer 2 peer of BTC formula. I’ll eat this info into my brain 🧠 until my neurons starting a new circuit.😂.
