Zilliz is a leading vector database company for production-ready AI. Built by the engineers who created Milvus, the world's most popular open-source vector database, Zilliz is on a mission to unleash data insights with AI. The company builds next-generation database technologies to help organizations rapidly create AI/ML applications and unlock the potential of unstructured data. By taking the burden of complex data infrastructure management off of its users, Zilliz is committed to bringing the power of AI to every corporation, every organization, and every individual.
Headquartered in San Francisco, Zilliz has technologies and products that help 5,000+ organizations worldwide easily create AI applications in various scenarios, including computer vision, image retrieval, video analysis, NLP, recommendation engines, targeted ads, customized search, smart chatbots, fraud detection, network security, new drug discovery, and much more. Learn more at zilliz.com or follow @zilliz_universe.
Recently, I have had similar problems to those stated by " @josephroman2690 ' -- especially with the many convoluted and contradictory " llama-index" libraries --; HOWEVER, I will hereby "vow" to master Milvus Lite since 'Zilliz' has invested heavily in this relatively new and local form of Milvus and we desperately need alternatives to small VDB's like " chromadb " ! "
Amazing explanation I love the contetn. But i am not able to figure out a way to store the milvus vector DB locally so that i can load it and use it anytime in future. can you please guide me ?
it's an excellent vector database, but the problem seems to be when trying to set up the environment, always there's problem with version of libraries which starts with a chain of errors with this letting the user to surrender and abandon the idea of implementation in a full stack project
Hey I get this error in line 4: ContextualVersionConflict: (grpcio 1.62.2 (/usr/local/lib/python3.10/dist-packages), Requirement.parse('grpcio<=1.60.0,>=1.49.1'), {'pymilvus'}) can you please help me
Does ziliz already have an embedding model so that I don't have to embed the docs, can j just use ziliz to embed the pdf file for example, currently i am making embeddings using a embeding model from hugging face
Whoever the talking and answer I g questions should no be a person who integrates with the public to explain basic concepts. Human communication is NOT a skill set of his I am not going to allocate an hour to this. I will find the information I need that is presented to me in a clear, structured, concise, and easily digestible way that provides facts and not a lot of useless time-wasting fluff that comes with interviews and podcasts.” I need my hour to be productive not listening to a guy who defines words with definitions that include the word that needs to be defined.
Hellooooooo, super new to LLM's and Vector databases. I'm following along the tutorial but I was wondering if theres a GitHub Repo for this if possible?
Your team sure is taking liberties with some of these statements in the videos. Eg this is not the only database that can work with unstructured data. If you do any research you’ll find many options. It comes off as either naive or disingenuous. Not sure which.
@@yujiantang I guess he meant all those NoSQL DB's which can be used to STORE unstructured data. But they probably can't be said to WORK with unstructured data in the same way as vector DBs.
I am struggling with this from llama_index.vector_stores import MilvusVectorStore. Do you have a reference where MilvusVectorStore connection uses SSL. I have client key and pem but not sure how to use it. Not seeing any implementations. **kwargs has no examples with client_pem_path, client_key_path, ca_pem_path etc.
if you add new data to the vector db do you need to re-index the entire data? which could be very time consuming? is there some way to do rea-time updates?
i can't comment on how other vector databases work, but Milvus indexes your data in pre-defined segment sizes (default 512MB) and continues to index as you add data
No, you do not need to reindex all your data for HNSW. You can interatively add a new datapoint by: 1. Determining in which level you data point should be present, by draw a random number. 2. Finding to the k-nearest neighbours at the highest level you want to add your point to by querying the existing HNSW and connecting the neighbours to your new datapoint. (Likely you should also prune some existing connections to prevent creating very high degree vertices) 3. Decent to the next lower level and repeat the process. Do this until you reach layer 0. You have now added a new datapoint in log(n) time
an "index" is a mapping of a location to a data point (ie list[0]="Gerald-iz7mv" is the first "location" in a list), so an inverted index is a mapping of a data point to a location (ie list[Gerald-iz7mv]=0)
Great video. Can anyone please tell me what is the best strategy to regularly update contents in the RAG database. For eg. On monday there is a music festival whose info is stored in RAG. But I need to to store info about the next music festival happening three days after the first festival. How can I do that ?
it will be great if you do a series with different technologies like flask.. and build something on the frontend level.. I noticed that you have one with ruby on rails but there are other technologies that are a little bit more popular...
Undoubtedly, long context LLMs will not cover all the use cases where RAG is currently used for search. However, they will help a lot in tasks involving summarization, where - at the best of my knowledge - nowadays RAG and vector databases are not helping much. I’d like to hear your opinion.
