In a world where there is an ocean of AI , LLMs related materials out there, with a lot of them often misleading, what you have done with your AI-Bootcamp repo is just...... incredible. You are a hero man. Thanks
tokenizer = AutoTokenizer.from_pretrained(r'/content/drive/MyDrive/trained_models/Dersat-2-llama-3.h5') model = AutoModelForCausalLM.from_pretrained( model_name, torch_dtype=torch.float16, device_map="auto", )# Load the model from the directory model.resize_token_embeddings(len(tokenizer), pad_to_multiple_of=8) model = PeftModel.from_pretrained(model,r'/content/drive/MyDrive/trained_models/Dersat-2-llama-3.h5') model = model.merge_and_unload() ## the error is : RuntimeError: Error(s) in loading state_dict for PeftModelForCausalLM: size mismatch for base_model.model.model.embed_tokens.weight: copying a param with shape torch.Size([128257, 4096]) from checkpoint, the shape in current model is torch.Size([128264, 4096]). size mismatch for base_model.model.lm_head.weight: copying a param with shape torch.Size([128257, 4096]) from checkpoint, the shape in current model is torch.Size([128264, 4096]). can you solve it ?
Hello, thanks for the great tutorial. I'm stuck at this section: "db = sqlite3.connect("example.db") db.enable_load_extension(True) sqlite_vec.load(db) db.enable_load_extension(False)" I'm doing everything but it says "OperationalError: The specified module could not be found." on sqlite__vec.load() side. Can you help me on that?
Hello Venelin, thanks for the video. Iface this problem when running the query: "I encountered an error while trying to use the tool. This was the error: StructuredTool._run() missing 1 required keyword-only argument: 'config'. Tool list_tables accepts these inputs: list_tables() - List the available tables in the database" any help here? Thanks
I’m trying to fine-tune the LLaMA 3.1 8B model without quantization, but when I try to use the pipeline with the unquantized model, I encounter this error: Trying to set a tensor of shape torch.Size([128256, 4096]) in 'weight' (which has shape torch.Size([128264, 4096])), this looks incorrect. Do you know why this is happening? maybe i'm using the wrong pad token idk. Thanks in advance
Possible - yes! It will be a project though, wouldn't go with building it from scratch. Checkout LangChain and/or LlamaIndex as starting points. Look at pgvector if you happen to use Postgres/Supabase. Also, there exist some premade solutions that work quite well. Btw, me and a few friends are building exactly such system that will be made available to the public, will make a video once it is released. Best of luck!
Great tutorial as usual😊. I have two queries: 1. You are using sqlite-vec for fast retrieval? Otherwise just store query and response in any db and using cosine similarity on the customer query fetch the top most relevant response. 2. Why are you using LLMs here, is it just bcoz to get the concise and short answer? Because LLMs might hallucinate in some cases which i think can be more harmful sometimes.
1. I use it because it is fast and easy to understand for this use case (simple RAG from scratch). You can use whichever vector store or database works for you. I prefer SQLite because it is an easy "run your database everywhere" solution. sqlite-vec works with SQLite and performs the similarity search for you. One general rule I follow is, "When the database can do something for you, let it do it" (as long as it is fast and simple enough). A great benefit of this approach is that your code often becomes simpler to read and understand. 2. The LLM is responsible for combining the context (retrieved questions and answers) and providing an understandable and relevant answer to the user's query. I completely agree with you that the LLM can sometimes produce incorrect responses, but it generally provides more conversational answers and takes into account the chat history. Thank you for your kind words!
If we apply the same approach to a dataset with another language like France or Arabic, will the approach change? Assume same columns structure, and names like you showed in the video.
So i guess this is another tutorial that ends after writing the basic thing with a "in the next video we'll make it do something" but that video never came.. I just want a single full start to end tutorial of a neural network....
Nice. What I am looking for though is a similar system for not just one document but one that is suitable to shat with a corpus of 1000s of documents in my archive, and - to be greedy - to create a relational map from the perspective of the query. Any hints?
Thank you in have watch several videos on llama fine tuning With lots of differences you are the best ! Especially anout dataset and how you formatted it
when i run trainer.fit(model,data_module) 54 has_train_dataloader = is_overridden('train_dataloader', model) 55 if not has_train_dataloader: ---> 56 raise MisconfigurationException( 57 'No `train_dataloader()` method defined. Lightning `Trainer` expects as minimum a' 58 ' `training_step()`, `train_dataloader()` and `configure_optimizers()` to be defined.' MisconfigurationException: No `train_dataloader()` method defined. Lightning `Trainer` expects as minimum a `training_step()`, `train_dataloader()` and `configure_optimizers()` to be defined how to fix
