LangChain - Using Hugging Face Models locally (code walkthrough) 

Keep up the great work. And thanks for curating the important HF models that we can use as alternate for paid LLMs. When learning new tech, using the free LLMs can provide the learner a lot of benefits.

Thank you! I am a newbie in this area and your vid´s are helping me a lot to get a better picture of the current landscape.

Great tutorial. I need to train a model on some private company documents that aren't publicly released yet and this looks like it could be a big help to me. Subbed!!

Great video sam , i don't know how i missed this

Amazing video, everything was well explained, I needed it, thank you so much!

saved me from hours of debugging and research! thanks a ton

Nice video. Im trying to connect an LLM and use Unity 3d as my interface for STT and TTS with 3d characters. I just found a tool that enables connex to a LLM on huggingface which is how I discovered that you need a paid endpoint with GPU support to even run most of them, I kinda wish I found this video when you posted it. Very useful info.

Excllent tutorial, ad so weel explained. Thanks a lot.

Thanks for this video. You just earned a subscriber

If we cannot afford to get A100, what's the cheaper option you would recommend to run these? I understand the models differ in size also. Thanks Sam.

Hi Sam, Is there any way to generate question answer from a given text in a .txt file and save those questions answers in another .txt file with the help of free huggingface model?

Thanks for the video. Is there any way to get an example using the lang chain JavaScript library? I am new to this area, and I think many developers would have a node versus a python background

How can the ready-made projects on the platform be linked to Blogger blogs? I have long days searching to no avail

I tried to store the RU-vidDownloader loads in FAISS using HuggingFace Embeddings but the LLM was not able to do the similarity search. Colab finally ran into timeout. Can you share how to do this instead of using OpenAI? With OpenAI I had no issues but like to do it with HF Models instead e.g. Flan br

This is helpful because in some industries like banking or telcos, it's impossible to use open source things. So we need to host.

Thanks for the content!! Is there a way to run a HuggingfacePipeline loaded model using M1/M2 processors on Mac? How would one set that up?

I have a problem, when I use low_cpu_mem_usage or load_in_8_bit, I get an error about I need to install xformers , When I install xformers , I get an error I need to install accelerate, When I install accelerate, I get an error I need to install bitsandbytes, And so on: einops accelerate sentence_transformers bitsandbytes But finally, I got an error *NameError: name 'init_empty_weights' is not defined* I don't know how I can solve this error and why it happens, could you help me please?

i am using huggingface model chavinlo/alpaca-native however, when i use those embeddings with this model pipe = pipeline( "text-generation", model=model, tokenizer=tokenizer, max_length=248, temperature=0.4, top_p=0.95, repetition_penalty=1.2 ) local_llm = HuggingFacePipeline(pipeline=pipe) my output is always only 1 word long. can anyone explain this?

Thanks for the valuable series and highly informative. Can you provide some discussions on in-context learning(providing context/query), reasoning & chain of thoughts

Hello. Can you please make a video on how to use the SeamlessM4T HuggingFace model with langchain ? Particularly for text to text translation. I am trying to do some prompt engineering with the model using Langchain's LLMChain module. But it does not seem to work ...

Stupid question, so I'll take a link to another video/docs/anything. Which Python version, cuda version, pytorch is the best to use for this work? I see many using python 3.9 or 3.10.6 specifically. The pytorch site recommends 3.6/3.7/3.8 on the install page. Then the cuda version 11.7 or 11.8 - it looks 11.8 is experimental? Then when I look at my nvcc output its says 11.5, but my nvidia-smi says cuda Version 12.0 .. head explodes... I'm on Ubuntu 22.04. I will google some more, but if someone know the ideal setup.. or at least the it works setup.. I appreciate it!!! Thank you

How can I load the model for some time later, once I download it on the local drive

Than you for the awesome content, it would be very helpful if you make tutorial on how to use custom model with langchain embed it, so i want to train some documentations , so currently we can use open ai or other service APIs But it is very costly consuming their APIs, so can you teach how to do that locally please consider training a custom documentation of site, and it can answer from the documentation, more context aware and also history remember. Currently for that we depend on open ai APIs. So if it's achievable using local modal it would be very helpful

Please make a video on how to convert Safetensors to. GUFF format or format that can be used for Ollama? Thanks for these great AI videos!

how to fine tune this model so that it can follow instructions on data provided

Just curious, do you need to redownload the model everytime you run scripts like these? Is there a way to save the model and use it after it's been downloaded?

Hi sam , how do you feed your personal documents and query them and return response in a generative question answering format and not as extractive question answering , I am bit new to this library , I don't want to use Openai api keys please provide some guidance on using with open source llm models, thanks in advance!

Awesome video! Was wondering how I could use Langchain with a custom model running on sagemaker? Is that possible?

These are great videos Sam. I am using a Mac M1. Therefore, it is impossible to run any model locally. I understand this is because PyTorch has not caught up with M1 yet.

and how can I use big models from huggingface ? I can't load them into memory because many of them are bigger than 15gb, some of them are 130gb+ . Any thoughts?

Hello Sam,how do you solve Token indices sequence length is longer than the specified maximum sequence length for this model (2842 > 512). Running this sequence through the model will result in indexing errors. thank you inadavance

Thank for he work. Just let you know Loire Valley is in the north of France ;)

U a legend

Hi.. i would like to know that, Can we chain 2 models like a classification model and a pretrained model using langchain?

Hi! Do you find a way to load vicuna gptq version using this? I try your video with gpt neo 125M and it's working, but not vicuna gptq. Thank youu

what kind of cloab you need, becuase i am using $10 version with high ram and GPU on, and still cannotr run ValueError: A device map needs to be passed to run convert models into mixed-int8 format. Please run`.from_pretrained` with `device_map='auto'`

Hey, why it take time to response out? This needed of my CPU to be fast?

I'm a transformers user and I don't still get the point to learn this new library. Is just for very specific use cases?

how to do telegram chatbot with this

It would be nice to have someone do review fir noobies as there are so much to learn and it's hard to know we're to start from.

can you create for csv as input

hello and thank you very much for this video, on the other hand the problem is that I am not sure to have understood everything, I speak English badly, I am French

Would be useful if you explained what program your using, or what page your looking at, seems like waste of time if you don't know anything about the programs or what your doing 1/5

u are not doing it locally in this video.....

Ok, great tutorial, but as a French from Bordeaux, I am deeply disappointed by the answer of google about the best area to grow wine. Loire valley ? Seriously ???? Name one great wine coming from Loire, Google, I dare you. They are in the b league at best. The answer is obviously Bordeaux, I would maybe had accepted Agen (wrong) or even Bourg*gne (very very wrong). But Loire, it's outrageous and this answer made me certain that I will never use this cursed model.

What happens always with this setup langchain + HuggingFaceHub is that it only increments on 80 characters for each call, anyone else having this problem, I tried max_length: 400 and still same issue

Hello, Thanks for your video. for now it's : llm_chain = LLMChain(prompt=prompt, llm=HuggingFaceHub(repo_id="google/flan-t5-xxl", model_kwargs={"temperature":0.9, "max_length":64})) temperature must bu >0 and model : flan-t5-xxl

thx! helped a lot! but stuck at loading model...it says google/flan-t5-xl is too large to be loaded automatically (11GB > 10GB)....qaq