Hey, great video, thanks to you I got it working. Is there a possibility to change the code, so that not the first modelfile is downloaded? I want to download Q4 Mistral for examble but the code gives Q2. I am pretty new to this, sorry if this is a silly question.
Thank you for reaching out !! Good question it is. In another video I have shared different ways to load the models, with Llama_cpp you can manually download the model of your choice and load that for inference.
Great one! Can you make a tutorial on, how can we finetune our custom dataset locally and use that finetuned model for getting domain-specific results locally?
Hi Joy, thank you for this video. What is the advantage of using ctransformers library versus other github libraries available such as OpenLLM? Is it just a matter of personal preference?
Hi, thanks for your quey, more than preference i would say understanding the core, openLLM is like a wrapper created using the basics that tried to capture.
@@joymaitra5414 I prefer simple as well. I will try with just ctransformers to see if that improves implementation. Right now, too many wrappers out there haha.
Thank you so much, this explanation is great! This really help me a lot but, i'm stuck at adding my own gguf models to my project. Like when i'm trying to add it my code didn't detect it and downloaded the other version of the model id. Can i download the models manually from Hugging Face than downloading it from the script? Because the file i downloaded from the script, is not even a gguf file or any type of that.
Yes, that is absolutely possible, you can manually download and provide to the local file path in LlamaCpp, for CTransformers also the same is possible
