Thanks a million to NordPass Business for helping me end my Shin Ramen speed run…and for sponsoring this video! Whatcha waiting for? Grab a 3-month free trial here nordpass.com/nicholasnord with code nicholasnord!
I started following quite a long time, I could see an incredible transition in video, quality and content. I will be always waiting to explore more new tech from you.
I've seen a bunch of your videos and this one has got to be the best. Such a complicated topic turned engaging + informative is no joke. Thanks Nicholas!
@@NicholasRenotte trying to create a Private expert on specific subject but i am seeing i will need to through fine tunninv to do what i want, i love the way you explain this tech, to me as a begginer is hard to realy understand what i am doing, aways watching your content to realy get an understanxing on what i am realy doing
How may RTX4090 required for Falcon 40B ? 160gb vRAM via RTX 4090 means 6-7cards , which is roughly $10k and still cheaper than 2 A100's. Using RTX for Stable diffusion already.
Have you been able to use VSCode connected to a remote jupyter instance that still allows Pylance to work? e.g., so you can make use of VSCode's nifty features like cmd/ctrl clicking to see a method definition, etc.
Can we not use model.save pretrained and save the model in the form of shards so that when device_map=auto is used accelerate would kick in and allow to offload the shards to disk and memory. I think that's why you were getting OOM errors.
What's the cheapest hardware we can find on the market to be able to run this? And I prefer it be local. Can someone help me order the required hardware for this?
Nicholas, thank you :) I have two questions. First, is it possible to use this on my personal laptop? I don't have a robust GPU; instead, I have a Mac M1 with 8 GB RAM. Second, could I train the pre-trained 40B model for a specific task in German? I'd like to use it for classifying sentences into labels. Is that feasible?
Realistically, no, you'll need a beast GPU to run it. Didn't work on my mac. You would probably need to fine tune it for that but you would need even more GPU compute to achieve that. But tbh, if it's just sentence classification there's much easier ways to do that, you could just use a small encoder only model and it would probably work well!
hey Nich thanks for u video but i have i have already devellope Q.A chatbot response and i try to implementation fontionnality , to make him read excel file using text generation, do u have any idea how an i implement this? the only i found is to use Langchain and openai API but i try to do withoug openIA API
Excellent video Nich. I am also exploring Falcon for my domain specific requirement using the concept of RAG with Langchain. But the model is taking too much of time to generate the result even after quantization. Do you have any suggestion on how to reduce the runtime? If I set max_length to less than 1000, model is unable to generate anything. Kind of stuck with the issue !!
On the math thing... is the word on the street that we're going to handle math just by increasingly larger parameter counts? Because that scares the crap outta me for engineering applications where the math becomes very technical and obscure. Almost like we need a separate ALU baked into the model to make math feasible on lightweight small parameter count models.
Mark my words, some new architecture will come out that will boost performance with dramatically smaller parameter counts. You're right though using a separate ALU could work as well, e.g. Langchain using Wolfram. Also, I can share some of the work our research teams are doing for efficient fine tuning and building smaller parameter efficient models!
I think they only do arithmetic tests to see how well the model can generalize. Like he said, people already use Langchain or the Wolfram plugin to do math properly.
@@LowestofheDead Yes, agree, but that's not going to accelerate us very far. It means you still have to be super specialized in mathematics to know how to use those tools. The promise of AI would be to get to a point where the AI model can use those tools to output highly mathematical solutions with simple prompts. For example, "imagine a geometrically correct STEP file assembly of a handheld drill"... then open it in Fusion360 and print or machine all of the parts. That is the next fundamental step change in this tech, IMO. "3D" images don't count, because they are not geometrical engineering files of reproducible physical objects.
Hello Nicholas sir, this video was really helpful to learn how to make own chatbot, it would be helpful if you make video upon how to use LLMs to perform classification using fine tuning techniques such as Zero shot & Few shots learning. Thanks.....
Really loving this going through technique , what an amazing video that's a lot . Also I have 2 questions.. 1) can I run falcon 40 instruct on Colab free version, which has Tesla T4 16GB ?! 2) can you make a video on Fine Tuning a Stable Diffusion model like sd2.1 or sdxl to make our own checkpoints ?! PS: really amazing video, thank lot❤
Answer to your first question: No. You cannot run falcon 40b-instruct on a Colab free version. Falcon 40b needs 85GB - 100GB of VRAM at 16-bit precision. Even with reduced precision down to 8-bit it still requires some 45 GB VRAM. At 4-bit precision, it requires 35 GB VRAM. You. need to load the entire model on to GPU memory (could be multiple GPUs).
This is so exiting and yet so demoralizing at the same time. Unless you have some real good understanding of coding and llm looks like an impossible task. And rightly so.. but I wish there were more effort out there to make this way more accessible to people that focused on other subjects, also because we may just end up with a lot of very superficial products .
Pas trop mal ton français et mieux que l'anglais de mes collègues 😜, as an american who has lived in France for more than 10 years, I've both said and heard a lot worse. Love the videos! so much great info and things to learn. Thanks so much for sharing 🙏
@@NicholasRenotte actually we have 2000 websites, if I’m trying in beautifulsoupe it’s taking one month it’s so long process…. Selenium also not working
guys I'll introducing programming helper the programming helper is so powerful and much better than OPENAI wat waiting for now since start 1990 is there Lua language in AI CHAT and many more
How can you overlay photo's? Open cv? I'm looking at just photo's that taken one after another. What I find interesting is just level of detail. At first glance they look like photo's taken in the 1950's until you hit the zoom button. There are thousands of stars and alot of stuff moving around in space. The just photo's you can see all the stuff. What I want to do I overlay 100 photo's of the same area and color everything that's not in all 100 photo's. See if we can discover new objects moving in space. Here's the starting photo jw0157126001_04201_00001_nis_trapsfilled.jpg the target is Antennea. These photos start around 790 in the list
