Chat with your PDF - Gen AI App - With Amazon Bedrock, RAG, S3, Langchain and Streamlit [Hands-On] 

You are welcome. I am glad you liked it.

Yes I don't think why not. You have 2 options: 1) If you are already using PostGreSql, then you can use pgvector extension to make it a vector data store. 2) If you do not want to use that, or not using PostGreSql and still want to vectorize your data and store in a vector index, you can certainly do that. Steps will be something like: 1. Let's assume you are using ElasticSearch as Vector store 2. Read from table(s) , sql query, get the data 3. Call an embedding model with the column(s) of data you need to generate the embedding for. 4. Create a JSON object with plain text and embedding data. 5. Make Api calls to Elastic search to index the data On querying side: 1. Convert the query into embedding using the same model that you used to create the embedding while indexing 2. Run similarity search on Elastic Search. I am going to make a tutorial on something similar soon. Stay tuned. I will add a comment here with the same when ready. Thanks

This error normally occur when the pdf reader is not able to parse pdf file. Could you try with a few different pdfs and see if it works. Thanks

It depends on the use case, data and computing resources at your disposal. For example, some embedding models outputs vectors of length 4096, some smaller ones 1536 or even 768. 4096 will have more hyperplanes I would assume than 768 but it will be more compute intensive to index and retrieve. There could be more reasons as well, like loss of accuracy etc.

Cost will be applied every time you use any model from bedrock (or openAI or any hosting platform). So you paid initial cost for Embedding your content and stored in a vector store. Every time a user queries, you are converting that query into a vector embedding (using the same model that you initially used) so a little cost is involved, then you make the search, find the similar document and send this all together to a LLM. There will be inferencing cost for input and output tokens from LLM at this point. I hope this clarifies. Thanks

You should put some debug (print) in the get_response method and see where the issue is. May be the LLM didn't respond with data. Thanks

@@MyCloudTutorials Thanks for the reply. I was using the same model for embeddings and the llm . I have updated that and its working now.

Its paid model. But it charges based on how many tokens you use. Thanks

Why was it rejected? Could you re-apply and mention that you are evaluating the model? If not, then you may need to use llama model but have to change the request / response accordingly in user.py

@@MyCloudTutorials Seems like once we provide the details regarding why we require the model...we can't edit those details again. Initially I mentioned random information in the company details, purpose etc... So I created another account and requested the model with appropriate purpose...then it was accepted

I have used PineCone and OpenSearch (ElasticSearch on AWS) for multiple documents indexing. Basically for similar topic, you want to create one index and append the embeddings & content in that index. I haven't used FAISS for such case, but if it does support then I will use one db file otherwise how will I load into one searchable index (I am not sure). I might try doing something and share if I find some solutions. Thanks

You can find it online, search for arthritis pdf. I downloaded from www.versusarthritis.org/media/22726/what-is-arthritis-information-booklet.pdf

Yes, Any embedding model should work. You have to use same embedding model for creating and querying. Once you containerize the application, you can run on AWS ECS (Fargate or EC2) or EKS. Thanks

​@@MyCloudTutorials thanks 👍

It depends on how many calls you make to bedrock for embedding, querying etc. S3 storage is cheap. Overall it costed me less than $0.50 for this, but your costs may vary. Thanks

Well, if you do not want to use RAG (like I showed in this video) then you will have to take a Foundation Model and train it with your data. So, yes, it's possible but it would require more resources, time etc (depending upon the base model you choose and your data). For continuously moving data, RAG works out better as you don't have to constantly fine tune the models as and when data is available. Another thing to think about is cost. Once you train and create a custom model, you have to host it somewhere (whether cloud or in data center) which will incur some cost. I hope it helps. Thanks

I am glad you liked the tutorial. About the case you mentioned: You can merge both admin and user application. A little different approach can be taken. I am assuming you have the user logged in (so you can get their id) You can ask user to upload the pdf and show them "processing" kind of spinner, while you generate the embeddings and create vector index. If you need to store their vector index, you can always make a subfolder in S3 with their userid and may be assign this processing a job For example, s3://// You may want to consider saving this meta information to a Database like DynamoDB or Mysql so you know where to load the index from for this particular user. You can ask them to upload to S3 and trigger an event that creates the index. There are several possible ways to solve this, it depends on the use case and business requirements. Please let me know your thoughts! Thanks

Langchain has Contextual memory. Please check this: js.langchain.com/v0.1/docs/modules/memory/ I will make a video soon, once ready, I will put a link in this comment. Thanks

I am glad you liked it.Thanks

@@MyCloudTutorials @palgorithm 0 seconds ago Hi, my request for anthropic claude model rejected. How to use that model then? I provided random details in the use case (company, website etc...)

Thank you. The source code is available in Github: github.com/mycloudtutorials/generative-ai-demos/tree/master/bedrock-chat-with-pdf

@@MyCloudTutorials thank you so much..i was able to download..but am actually getting strange errors on windows..is there anyway i contact you..not a developer per se...

Hi @KeerthanaPriyaDevaraj Do you have AWS Cli installed and setup with IAM credential that has access to Bedrock? If not, Please follow AWS documentation for installing AWS Cli from here: docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html If you already have an IAM user created, create access_key and secret_access_key in the IAM console and download the credentials. Use `aws configure` command to setup the CLI. Also, make sure that you have the models access enabled in the Bedrock console. I hope this helps. Thanks

Is possible lambda aws ?

Very much possible in Lambda. If you are using langchain with Lambda, you need to include that as dependency. I shall make a video on how to do it in lambda, no ETA as of now, but I will surely tag you when I post a video on that. Thanks

​@@MyCloudTutorialsI'm trying, but I have an error: "Unable to import module" for "langchain community", I generated a layer in the lambda function, but it doesn't work, I'm waiting for your video, thanks