Hey Farzad, I love this video and this is by far the best chat with database video out there. I've actually understood every bit of it however I have 1 small doubt. Whats the difference between using an agent and usnig an llm thats binded with tools (llm.bind_tools)
Thanks. Glad to hear it. In general LLM agents are responsible for making decisions. The decision can be to call a tool or it can be to choose the right agent. So, an llm that is bind with tools is also considered as an agent
Hi sir your videos are awesome. I would suggest to make a video of conversational rag with chat history trimming. Also integration of function calling, langraph etc. Because in companies we need these type of application.
Thanks. In this video I have explainedfunction calling and how to design graphs using langchain and langgraph. I also brifly discussed adding a memory to the system but it does not contain trimming
Excellent great video tutorial, thank you very much it is very helpful because I was building an agent with similar capabilities. To work with databases that have high cardinality, I used the example in the Langchain documentation where a similarity retrieval tool is created. I would like you to make a video about those cases.
I have 2 more questions: 1) Can we use any db like postgres or mysql etc? 2) Is there a way to limit the chat history that we pass to the LLM where instead of passing it all to the LLM, we pass the 5 most recent msgs or 10 most recent msgs. This way we can guarantee we won't exceed the context window and have our application break at some point
1. Yes 2. Yes there is a way. You can either design a custom chat history and pass it to the LLM or you can use langchain's memory just like the one that we used in the video. And you can control both of them in terms of how many Q&A pairs should be passed to the LLM
Is there a way to benchmark the performance of the Text to SQL systems on how accurate the sql queries are? I am having hard time to do this since RAG is involved. It completely depends on how good your RAG model is to generate queries. Most of the time business context is different from what resides in the database. e.g- a business may call it’s internal department as squads but database has a column called department. This could be solved using RAG. However, I want to get your insights on benchmarking the performance without using LangGraphs. Also, to your video- really good. But in reality, I think it would be good to create separate agents for each database and control the permissions of users on agents.
There's no easy way to measure the accuracy of SQL queries. If I wanted to do it, I'd create a small dataset with pairs of questions and their correct SQL queries. Then, I'd run those questions through my system, get the generated SQL, and use another LLM to compare the correct SQL with the generated ones and give them a score. You could also add some metrics to measure how similar the two queries are, but this is a custom method since there’s no standard way. It all depends on your project. If you need users to access multiple databases across different departments, this approach works well. But if you need to control user access to specific databases, your approach would be better. The right strategy always depends on the project's needs. Thanks for the great discussion!
Thanks. Well the primary agent should somehow understands which database too look for. So, it is either through the question itself that it can understand or througha comprehensive system role. If your users will not mention the right database, I'd suggest to explain what each database contains in the system role so the primary agent can understand which one to pick for each query.
Hi Farzad, what is the purpose of pair of ** around Music, Business and ALL in the system message definition under Strategy B? Also, I found the results are not always the same. if I clicked on the invoke function multiple times without changing any codes, sometimes the result returns all 6 as expected and sometimes only 3 table names returned...I'm concerned if consistency of the results will be a problem when we need to apply this to a real-life use case. Thoughts? Thanks.
Hello. When I write the system message, I usually tend to use markdown syntaxes. It is both for better human understanding and also LLMs understand markdown so I assume this sends them a more organized message. If the second strategy is not stable enough, go with the third strategy.
Hey Farzaad, Thank you for such a knowledgeable tutorial, I have two question here, 1. Is it possible to integrate product listings inside the chatbot? For example, if we build a chatbot for Amazon and a user searches for something like ‘I want to eat something low in carbohydrates and high in protein,’ can the chatbot display matching products and their listings? 2. If possible how we can?
Thanks. 1. It depends on the structure of your listings. 2. For instance in your example, if you have a column named `carbohydrates_rate` and it is filled with either `low` or `high` (or a percentage for example) and you have a column named `protein_rate` and again it is filled with `low` or `high`, then yes the agent can filter the table based on the correct values of these two columns and return the correct portion of the table.
There was a discussion about it in my Gothub: github.com/Farzad-R/Advanced-QA-and-RAG-Series/issues/7 I am not sure why it is happening. I cannot replicate it from my end but apparently it comes from the travel database. If you change the database the pipeline works fine
Hey so how should we approach a usecase where we need to combine knowledge from multiple sources before generating a SQL Query,like for eg:In this case we are finding info from multiple sources based on query,but what if we need a system where vector db's info is dependent on the sql db which helps the agent for generating better queries
You can still design systems where vectorDB's search result has an impact on the SQL query. But to give a more detailed strategy on the system design, much more information is needed. For instance, the type of the relationship (is it linear or does it have an asynchronous effect) is one of them.
Great tutorial ! Looking for google gemini based projects since long :( + JSON structured output format in langchain for Gemini based model without pydantic model
I only used the 7B Gemini model in one of my videos. ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-6dyz2M_UWLw.htmlsi=x2t4jH0UriIYFqUr But that is the open source version of Gemini.
Great explanation sir. Can you do a video on how we can create tools for any database that I can have. Here you have tools specified to databases. Can we do tool for large databases that is dynamic (new data and tables coming in most of the times)
Thanks. Well the approaches would be pretty much the same. The only part that needs to be properly adjusted is the data pipeline and the strategy that wants to pass the table names to the agents. It needs to be designed to be dynamic just like the nature of the data that is coming in
Hi Farzad, I got a quick question here...when I was using gpt-35-turbo, the function was not recognized, and the finish reason is "stop". While I switched to 4o-mini, the finish _reason is the same as what you showed, "function" call. Do you have any idea the reasons behind? Thanks.
Hello. Check the OpenAI's version and make sure it is updated. And also test your models with the notebook in which I discussed the custom agent design using function calling first. GPT-3.5 must have the capability to become an agent
@@airoundtable Thanks Farzad. Are you referring to api version? In both scenarios, 4o-mini and 35t-urbo, I was using "2024-02-15-preview". Btw, I was using Azure Open AI models due to security policies. I tried the custom agent function call as well and the result is the same (4o works but 3.5t does not work). The codes are as follows: response = client.chat.completions.create( model="gpt-35-turbo", # gpt-4o-mini functions=[abc_json], function_call="auto", messages=messages, temperature=0
@@airoundtable On another note, I read an article mentioning gpt-3.5-turbo-0613 deployment (from June 13th 2023?) might have some issues with functions....not sure if this is related....
@@terryliu3635might be. I haven't looked into it in that detail to see if all the models are capable of function calling. But all of the ones that I commonly use can do it
I have a doubt. If we have a rag pipeline and also a function calling pipeline can we integrate this two as a single chatbot by creating rag as tool and converting both tools into agent?
I didn't fully understand your point. But have a look at this project, it might be the one that you are looking for: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-1FERFfut4Uw.htmlsi=08ISQOb3IMb_18lF
@@airoundtable simply how to combine a conversational rag with tools . If a user ask a question How will the bot knows whether to retrieve data from vector store or do function calling
@@anandukc4709 That is simple, based on the user's question your LLM first decides whether to call the RAG pipeline or other tools. Then the final response goes back to the LLM to prepare the answer for the user. This is exactly what I showed in this video. Omit the SQL agents and tavily websearch and add any other tool that you have in mind. How does the LLM decides which tool to call? 1. By a good explanation through its system role 2. By a clear function description in for each tool 3. Optional: By providing few shot learning examples in the system role
@@anandukc4709 The LLM that is making the decision between choosing RAG or other tools is the agent. Just like the primary agent in this video. So, decision making is the most important role of that agent (LLM)
Thank you Farzad! Your videos are very helpful. I faced an issue in trying to run 'sql_agent_chain_steps.ipynb' notebook. When I run the cell with 'message = "How many tables do I have in the database? and what are their names?" response = chain.invoke({"question": message})'. I always get this error. Can you help me understand what could be the issue. Thanks again "TypeError: must be real number, not str"
Thanks. I am glad to hear it! It is hard for me to answer that question since The code works fine on my side. I'd start to break down the code and see where it is coming from. In most cases if you check the traceback, you'd see the exact line of the code that is throwing what error as well. If you couldn't solve it please feel free to open an issue on the repository and we will take it from there
sur it is a great project thank u... i hope you make a video for generating sql code from text (question) to query large database i mean the model has connection to my database schema ,tables ,columns , ..etc and based on my question it can generate the correct query without telling him the table name explicitly it should be clever to know the right table or the nearst table to the question ... i hobe you can make this video soon with fine tunning and RAG
Thanks. In this video, I introduced three strategies for connecting SQL agents to large databases. Give them a try and I hope they can help you solve the problem.
@@airoundtable Thanks , i will do this , ...iwant to ask you a question if u have a time to answer me i want to make this project with my self to improve my skills but i donot want to use Open ai models directly i want to fine tune an open source llm model like distilbert or llama3 ..etc on. QA dataset to generate sql code from natural language questions do i need the RAG or the fine tunning will be enough... I think I need it because when i pull the data from the database as a jason file i will transform it to vectors then store them in vector database and when i ask question the rag will agument the similar vectors to my question to the generator model to generate the final sql code ... I hope you can answer this question this will help me alot on my graduation project or provide me the. Right way to this approach Thanks alot i hope you the best
Hi. yes. There are multiple ways to do it. For instance, you can create a Python-based backend API using frameworks like FastAPI, Flask, or Django, and then connect this backend to your Next.js frontend via API calls. There are other ways to do it as well but it depends on the use case
@@airoundtable Hi, I tried using the FastAPI method using the Nextjs starter template, it works well on the localhost but I face issues for it running on the production instance. Its not able to connect to the python backend server, I am not a techie and don't have much idea about coding. But what I feel is that its looking for a server to run the python code fetching information from the front end part. Do you have any tutorial for this? Thanks for your help!
@@theaccountantguy Hmm, that is a tricky one. No, I don't have any tutorials on that problem but I bet there are plenty of conversations about it on software forums. My suggestion is to simplify things first and make a very simple call from the front end to the backend on your server and in the meantime solve all the networking and other connection issues. Once you made that first successful call then work on the main project. Hope you can find the solution.
