for a dummy like me trying to learn its too fast lol but I can always slow down and pause. his vids are so concise and short though, that's what's great, just simple do 1.->2.->3.
Bro I have windows laptop with intregeted Intel graphic and another is AMD, so how can I run it on only AMD? i changed the graphics settings for ollama to run on AMD still its not working that I can see through Task Manager
Although for completions they usually recommend a smaller model for speed, like deepseek-coder-1.3b. Have you tried both and which one performs better for completions?
I have already mentioned in the video about how you can use it with Groq. Also, You can use it with something like Phi-3 or other small models for CPU inference.
@@a2zcourse man it run great on my MacBook Air, maybe your using too large a model. It will gos slow for me if I try and load a 15GB model but 6GB it’s right away
suppe supper thanks alwys loved your vedios can you creat a video on autogen or auto gpt or ai agents using the power of this llm I think that will be very good please show us how to set up auto gen with local llm like this one please
You can run bigger models, it will fill your vram and, if necessary will use your cpu and ram to process the part that doesn't fit the vram. I have a GTX 960 4gb and I can run phi3:14b-instruct of 7.9gb and codestral:22b-v0.1-q2_K of 8.3gb. My specs: GTX 960 4gb, Ryzen 5 5600x and 16gb ram.
dude video request please... about keeping API keys on side bar of VScode so we don't have to reveal our API keys in any repo (because sometimes i mean all the time i forget to clean up API keys before i went repo commando)
I love your vids brother, They are short and to the point, only the info you need. Perfect for someone like me trying to learn all this when I should have done more in school besides play dragon trail on floppy.
Groq is only an inference API. The model you use via it matters for example Llama 3 or Mistral. Continue is only a Copilot for Coding not for searching the web.
@@AICodeKing got it, so can i combine those with something that can read docs from the web to understand edge cases that change daily like blockchain dev?
Idk dude, tried to get codeqwen to show me simple snake python code game and it couldn't show me one that is functional. The code was always either incomplete or it was garbage. GPT4 scored it first try so not sure how useful this is..... It would be nice though
GPT4 has 1.76 Trillion params. CodeQwen has 7 billion. Cannot compare the 2. The setup shown here has worked well for me for questions about the codebase or explaining code I didn’t write/understand, refactoring etc.
@@AICodeKing I started trying to learn code and all this when I have trouble turning on the tv lol but the only reason I am doing this is bc I see the writing on the wall, you think Microsoft wants to record your screen so you can have playback??? the US gov has free rain of all big systems. I just want my privacy and to be able to know what's really going on.
