hi there, welcome to my channel, hope you enjoy it and are as having as much fun as i am.
my name is chris hay and i love code, software architecture and technology in general.
as you may have figured, i work for IBM (who i love working for) but all opinions are most definitely my own and do not represent the views of my employer
With the final running of the code, running the assembly hello world, the code still works if I don't link the SDK and libraries during the "ld" function (I can just do "ld hello.o -o hello -e _start"), and doing ./hello still works. Does that mean that the rest of the function linking the libraries and defining the main function is unnecessary? Genuine question, just trying to reduce the amount of complex code I'm not entirely sure I understand
Hi, how can I find the data template of llama 3.1 base model ? How can I prepare research papers and books for fine-tuning the base model in right data format ?
Someone please correct me if I'm wrong here. I think that neither of the examples you showed comes from reasoning. The order is flipped, they should first provide the reasoning and then the answer, not the other way around as in your examples. The models take all the tokens into account from the input and the output (generated up to that point). What is giving the right answer a better chance is if the previously generated tokens contain the reasoning steps. In your examples the previous tokens did not contain the reasoning steps as those were generated after the answer.
When you said "math" I thought you meant symbolic math, not arithmetic. Using an LLM to do arithmetic is pointless; a calculator does a far better job.
@@BlunderMunchkin symbolic math is coming but you have to start with a foundation…. but in order to do symbolic math, the llm still needs to know how to count
does seem a bit verbose for ADTs : you need to define the tag type and use a case of on it to get the different member variables ( similar to record case in pascal)
I don't have time to go through the whole video at this specific moment, but it seems to me that you came to a fairly similar answer to myself: LLMs are pretty strong at presenting data and handling noisy inputs, while traditional computer programs are pretty good at doing the math (numerical instability notwithstanding). One obvious opportunity that I'm not seeing in the first ten minutes (though I'll certainly allow it's possible that I'll have egg on my face after finishing), is that this seems like an efficient way to embed agentic function calling into a model; if the steps to solve the problem contain a call to a remote system with the equation as an argument, and the remote system can solve the equation, that seems a lot like a function call to my eyes. Beyond that, there's also probably some room to reverse engineer a problem with the synthetic generator LLM based on the equation and answer, in order to encourage semantic problem solving, as seen in certain benchmarks which have word problems encoding ideas best solved with mathematics. Overall, this is a super cool project, and is probably going to be very beneficial for people doing continued pre-training or experimenting with certain ideas like grokking. I'm pretty excited to have a hack at it myself.
Absolutely spot on, I cover later in the video that the same technique can be used for function calling for complex expressions and can also be used for teaching code generation etc
These videos are insanely awesome. I LOVE the verbosity. Most of the internet videos are a high level abstraction of "what is programming". This breakdown of "What is happening when we program" is what's lacking in engineering these days! Well done :)
@@chrishayukit seemed to me they are using clever prompt engineering with their “artifact” system in a way that resembles memory management and tool usage with the help of the massive context window. They must have also finetuned their models to support this syntax. Just crazy to think how the system message itself is able to help the AI with coherence and task management. All this seems fascinating as I’m trying to figure out why the Claude 3.5 sonnet is so good at code related tasks especially related to re-editing and updating the code compared to most other models. I can’t wait to see some open source models reach this level! Maybe finetuning and clever prompt engineering is all that is needed for now 👍
@@venim1103 i'll check out their system prompt... but i'm convinced they're using STaR backed by a reinforcment learning policy. the new mistral nemo model has followed this approach also. not checked out how they implemented artificat yet. but i'm convinced this is all now in the fine tune phase, hence these videos
Very good content, thanks! I was comparing models manually, and I'll integrate Nemotron into the eval. One off-topic question, the over imposed screen on top of your your video is a post prod edit or is there any software that let's you record the video like this? Thanks!
awesome glad it was useful. the overimposed screen effect is a post prod edit that i do. the way i set the lights, screen backdrop, combined with lumetric settings and use of opacity, allows me to achieve the effect
I don't like that very much. Why? I absolutely hate getting walls of text and code thrown at me for simple yes/no questions all the time! Both ChatGPT and Claude have this issue. So in the end It's just that you hardcode a system prompt like "think step by step" into your model and it's very hard then make it giving quick and short answers again. A hidden scratch pad is a good compromise but still slows down responses and could by achieved with a system prompt too. The system-prompt method could also include multiple agents or personas with different strengths to provide input. The best would be to also train the model to estimate the complexity of a question and then decide whether to do additional thinking or not. Also I've seen open weight models answering harder questions with just one or very few words correctly where others generated a text wall and still came to the wrong result. So whether an explicity step-by-step thinking is really required remains debatable. Obviously the chances for a correct answer increase the more relevant (!) information is in the context and that's all what CoT etc. actually does: pulling more information into context. Another similiar thing that I see doing Claude quite often and which I like is that it does summarizations before responding. If the problem is complex and there was a lot of back and forth the perceptions of it may diverge. Summarizations greatly help to create a synchronization point between the LLM and the user and then focus on the established and relevant intermediate results.
Claude is NOT as good as GPT, sorry it is not. When you ask it to update code at least the way it works for me, is it sends only a new block of code that it changed not the whole class, to keep the response size down, it is really limited in the length of its response compared to ChatGPT. With all these Claude videos, I have to believe RU-vidrs are getting paid to shrill for it. Also, the Artifacts only works on a very small sample size of types of code, so all this selling like its a viable thing is disingenuous.
GPT is better at many things especially narrative, q&a, summarization and generative content (see my video on multiheaded attention) but Claude is definitely better on code. Not being paid by anyone, you will notice that I switch off ads for all my vids and I never take sponsorships
Can you please give a tutorial on how to load private data, train/Rag/evaluate and deploy an open-source model on WatsonX or other online platform (AWS, Azure or Huggingface)? Many thanks! BTW Nemotron4 broke down this noon (PST), maybe due to too many users. I was in line 771 with a simple question. It gave out some sort of communication problem, after two minutes od waiting
I believe you are correct in that both Claude and Lllama 3 are finetuned using STaR generated dataset but this method still needs a ranker or human to mark the correct answers, whereas from what I have read online is that OpenAI's Q* is a combination of "A* search" algorithm combined with Q learning from reinforcement learning to self improve, where the model generates 100 different answers and picks the best answer to improve similar to AlphaCode2 from Deepmind.
It does not. There is no human marker needed. For example, you can use a series of prompts + the dataset to judge aspects of the answers with really well trained fine tuners. You can even train a model to predict the human evaluation, then you just need to human eval in a given domain until an evaluator model is ready. In addition, this incentivizes further investment in synthetic datasets. Finally - the best argument for this. Big model prunes dataset to make a small model - that prunes the dataset for the next big model repeat ad infinitum. the smaller model is cheaper and faster which means you can prompt more data for the next big one - which will make the next improved small model.
Some folks use human feed back with RL and some folks use synthetic at the end of the video I talk about how it could be done with a mixture of judges and I show how you could use Nemotron for your reward model. I will do a video on RL for this soon to cover the Q part
I still don't get how a pathing algorithm like A* can be utilized to find the best answer. I mean it's not like navigating some terrain with exactly known properties. Maybe it's a thing in the latent space? So actually the explanation that this is a modified version of the STaR approach seems to be more plausible but if so then again it doesn't seem to be such a big thing.
@@testalesQ* (Q-star) is a concept from reinforcement learning, a type of machine learning. In simple terms, it's a way to measure the best possible future rewards an agent can expect if it follows the optimal strategy from any given state. Think of it like a guide that tells you the best move to make in a game to maximize your chances of winning, based on all the possible future outcomes. Kinda like in chess.
Thanks, for another great video Chris. I've been through some LLM courses on Udemy but your channel is helping me to clear many doubts I have on the whole thing. I'm glad I found your channel. It's really the best on this subject. Congratulations. Marcelo.
Hi. I was looking for dozens of videos on how to make ReAct work on 7B models (to make a low cost Text to Sql solution) and the only video the answer my question so far is yours. Thank you. By the way, I'm from Rio and the current time is 09:42 AM
Lol, hello Rio, glad you like the example. Glad the video is useful, my back rule is to always go one level below and unveil the magic, glad it helped in this case
