Fast and Simple Development - presents software development skills and techniques for super productive development focused on API, Microservices, Spring Boot, Java, MongoDB, Docker, React and AWS.
In this Channel I will shares Pro-Tips and Techniques for Java, Spring Boot, Microservices, Docker, AWS, MongoDB, SES, Twilio, Bit.ly and so much more
If you are want to grow your software development skills super first then you've come to the right place.
Overall I'II guide you to become a better software developer. If you are interest in this channel, make sure to Subscribe and click this notification button, So you never miss one of my videos! For sure, It's FREE!!
what do you think of zed instead of vscode? my input: it's very lightweight, but because it's quite new, not as full-fledged as vscode yet, but it does have "ai coding / copilot" implemented and supporting ollama also 🥳
Can you give me more information? I do not see a variable "context" in the code, "context" is returned in the lookup of data from the ChromaDB so if there is no context then I suspect there is no matching results from the search, maybe tell me the line number or share the code with me
@@federicocalo4776 That is part of the PromptTemplate so not a real variable, it is populated by the Retriever so line #70 should create this value in the retriever for you.
WINDOWS USERS INSTALL PROBLEM: "uvloop" says it doesn't work on windows remove it from requirements.txt it should install and seems to work anyways (I haven't tested the RAG part, only that the API call works)
I tried so hard to get lanchain_community to work but it wouldn't. I installed it and it would show in my env but it wouldn't work. I used 3.8 and 3.10 both and went through multiple doc and even took help of chatgpt for troubleshooting....It wouldn't work though. I am not sure why it wouldn't import langchain_community at all
At minute 23:00, I was getting an error with fastembedembeddings This was the fix There was a new version of langchain_community released 3 weeks ago. You need to roll back to the earlier versions. Run these commands and then it will work. pip3 uninstall langchain_community. pip3 install langchain_community==0.2.6. pip3 uninstall fastembed pip3 install fastembed==0.3.2
Bravo :) for my curiosity - is there any option to limit the answers only to provided content? Just to avoid irrelevant questions like "what is the best(...) / who is ()..." etc
The Video was awesome and prety helpful but can you cover the security point of view too like anyone with the IP and portnumber can access it So how can we avoide that?
I really love this video. I'm curious though the api call you make to elevenlabs, would it be possible to have it reach out to a new trained voice model or are you only able to call it locally on the package on the same server ollama is installed on?
Postman gives this error, can anyone help? <!doctype html> <html lang=en> <title>415 Unsupported Media Type</title> <h1>Unsupported Media Type</h1> <p>Did not attempt to load JSON data because the request Content-Type was not 'application/json'.</p>
Sorry for my written english, I just speak spanish: google. translate helped me : ) First Time Ever that I can finish a RAG project with chroma and llama3. Literally i saw dozens of videos y I was loosing my mind, because all videos out there are too complicated and langchain is huge and hard,(for me). Literaly I was loosing my mind. Thanks so much for took your time and give us this great tutor. For some reason the context of my raw_prompt is empty (I dont know) why and I am not getting answer... I am getting this message when creatae my retriever: /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/langchain_core/vectorstores.py:342: UserWarning: No relevant docs were retrieved using the relevance score threshold 0.1. I assume that for that reason context is not injected in raw prompt?
I tried adding a dockerfile and run in docker I am getting error fastembed.common.model_management:download_model:227 - Could not download model from HuggingFace: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/qdrant/bge-small-en-v1.5-onnx-q/revision/main (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)')))"), '(Request ID: ec0cb652-4ddc-4224-9caf-91eb57204534)')Falling back to other sources. any thoughts?
This is the typical hallucination problem, in the prompt you need to say "Only use the content provided" and also if the results from the retriever or zero length then I normally put up a message that says I could not find the content in the PDF.
I have video that I'm editing that has a full React UI, I built it so anyone can build a product with it if they wanted, I have the Python code now as a Microservice which makes it much cleaner to deploy in production as well as full logging.
Could you please tell me what's wrong. The context and sources of the document is not getting passed to the LLM {'input': 'What is the title of this document', 'context': [], 'answer': '[s][INST] Ah, let me check! * searches documents * Ah-ha! According to my search results, the title of this document is... "Technical Guidelines for Document Retrieval". [/INST]'}
Thanks for the tutorial. I had an error when saving the file in Windows , here is how itsolved: from pathlib import Path current_folder = Path(__file__).parent.resolve() save_file = str(current_folder) + "\\pdf\\" + file_name
FREE FREE FREE - All you have to do is host the service yourself. This is all Open Source, no fees for Ollama, Llama3, LangChain, ChromaDB, Flask or any of the Python Libraries. Keep watching for my Python Microservices that make all this even simpler to setup and use with a React Front End
thank you so much for this video, this is very helpful for me now. But i have some questions, how can i deploy this Flask app to server so no need to use 'localhost'
You can just create a venv on your Linux server, activate it and then pip install -r requirements.txt to install all the required dependencies. The just start as normal python 3 app.py and it will be running. You may need to open a port on your firewall on the server and then you can connect externally so maybe 10/10/10/25:8081/api for a connection. I do this all the time. I break things into smaller services (microservices) and have them run in a Docker container. You can load the Ollama on a GPU based server (see my video on this).
Thanks for the response, glad you enjoyed the video. Based on your comment I did research VectorDB, my initial concern is the lack of ongoing support from its GitHub, it appears the core is very old and it does not appear to be well maintained. One of the major issues with the functionality is that I do not see how to support distinct collections, so say you have a user and they upload a document (Bob has 3 PDF files) and another user then uploads a document (Sally uploads 5 PDF files) you would normally have something like a shared collection, a Bob collection and a Sally collection so the data is unique which is important for any Enterprise RAG application. I see no way to do this in VectorDB other than a unique instance which does not really work for me. I will continue to watch the project and if it progresses I will make some videos. Thanks for sharing!
@fastandsimpledevelopment thanks for the response what I meant was some vector dB being involved like chroma, pg vector etc ( mainly open source). Since u are dealing only with raw prompt. So maintaining context with be difficult if I use some paid model by chat gpt. As everything u have to sent the history in the chat which will be very expensive. So involving vector dB will make it cheaper. Anyways thanks for the response. And all the best. Hoping to see more videos from in future
@@ajaybiswal1 I do have a few videos using ChromaDB for Rag and also one for Chat History, I don't have anything using MongoDB which is what I use for production systems bug ChromaDB works well since it is based on SqlLite but you do need to turn off telemetry to keep everything private set anonymized_telemetry=False
Congratulations, this is really an exhaustive explanation on how to setup the necessary architecture for exposing AI, LLM based services from a private cloud. Thank you for sharing!
It worked, but when i run mvn install on the project, it also run tests, and during that time its giving error. Because it cannot decrypt the thing during test run. Any solution for this?
My first thought is to just skip the tests with the params -Dmaven.test.skip=true but maybe a more elegant solution would be to setup the decrypt to run in the tests, not sure exactly who to do that but I'm sure you can research this online
thank you very much. i would add that adjusting the hyperparameters `k`, `score_threshold` and the `PromptTemplate` custom instruction can make a huge difference to the answer - by changing these i got the system to stop producing useless, inaccurate verbiage and give short, accurate, useful answers. it was like comparing a 1B early LLM with GPT4o.
@@fastandsimpledevelopment to be specific this is what i have found so far to work reasonably well: PromptTemplate "Based on the following context, provide a precise, concise, and accurate answer to the query. Do not give a load of waffle and empty verbiage, just the actual answer. Do not use your own knowledge, only that in the text supplied. The answer is almost certainly in the given text, but it may require some intelligence on your part to piece together information in order to answer the question. Try hard; don't give up easily." "k": 5, "score_threshold": 0.5, if anyone is interested, i can give some example answers; my impression is that the LLM is the weak link - it's often not quite smart enough to piece together the right pieces of information, where a human would easily work it out. i plan to try out other LLMs and embedding algorithms, but at least the results are looking promising.
@@manihss i am away from my main desktop, but i will look as soon as i can. the prompt was something like: "keep the answer very short and don't give me a load of empty blather"! i think the score threshold was 5, but that may be erroneous.
There was so much OpenAI and almost none that really cover running Ollama locally, I uses this for a last company that has very private data, from HR information to Product Development, Jira and Confluence integration, no way could we use OpenAI and have them "Learn" all our IP content :)
We're about the same age. This is by far the best tutorial on the subject I've seen in a while. Thank you very much for your conscientiousness and dedication to quality! Cheers from Vancouver :)
