The guy just makes a video of things that's available on the internet and doesn't even cite the source. You are just taking someone's code without 0 revisions and you make a video to increase views?
retriever = SelfQueryRetriever.from_llm( llm, vectorstore, document_content_description, metadata_field_info, enable_limit=True, verbose=True, ) this enable_limit will solve your problem, that retriever is returning incorrect responses.
really good question, i've been scripting it for phi 3.5 using swift. Having a hard time myself doing inferences. I've been able to export (there is an export script for swift) the model but struggling to make inference on my colab instance. Have you tried exporting and saving it on HF hub?
@@moslehmahamud They hid this line in those docs that I'm trying at the moment: CUDA_VISIBLE_DEVICES=0 swift infer \ --ckpt_dir output/qwen2-vl-7b-instruct/vx-xxx/checkpoint-xxx \ --load_dataset_config true --merge_lora true I think this creates a merged model that you can load in a script then like this: # Load the model model_checkpoint = "swift/output/qwen2-vl-7b-instruct/v2-20240919-150643/checkpoint-1200-merged" model = Qwen2VLForConditionalGeneration.from_pretrained(model_checkpoint, torch_dtype=torch.bfloat16, device_map="auto") # Load processor processor = AutoProcessor.from_pretrained("Qwen/Qwen2-VL-7B-Instruct")
Good Info mate!. Can you guide me how we can serve this quantized/fine tuned model as an API endpoint? I tried with vLLM ans Litserv but in both case they are not supporting the local model or our own finetuned model.
There are numerous other videos on RU-vid that are 20+ minutes in length. They have their merits, but yours is nice and straight to the point - with examples! Thank you!
hi Mosleh, great video! 💜 I am one of the maintainers of LitServe and would love to talk you. Please feel free to reach out (also I am sending an email)
Hi Mosleh, thanks for sharing this straight forward tutorial! I wonder how to use a custom dataset? I'm struggeling to find the path to latex-ocr-print, also on model scope
@@moslehmahamud folow up questions: how to create the custom datasets? I heard that creating synthetic dataset is faster than labelling itself? Hope there is easy to follow tutorial on it.
i have custom data but also with an history of a conversation and i want to fine tune qwen 2 with it. Where should i write it, in the input should i wirte the history of the conversation ? if so should i use a specifi format or should i make my own ? Maybe something like that : "User 1 : hello AI: Hello" ? i can't find any answers
