RAG From Scratch: Part 2 (Indexing) 

For efficient similarity searches

When performing RAG, you need to encode your input data into embeddings because that is what the LLM understands, it is from these embeddings that the model will perform decoding of the output and subsequently give you the result you've asked for. These embeddings need to be stored somewhere, they can be small or very large, some vector dbs can be free, opensource and be in memory like faiss and chroma, they can also be paid and hosted like pinecone

​@horyekhunley thanks for the insight

@@horyekhunley the process is a little different The LLM doesn't touch the embeddings. The embeddings are used to convert the documents to a form that can more quickly and accurately be compared to the question (which is also converted to an embedding). This is done by an embeddings model (in this example an embeddings model from OpenAI (referred to as OpenAIEmbeddings() in the code) is used. These embeddings and their associated documents need to be stored somewhere (in this example Chroma is used). This is the indexing phase. After the comparison between the embedding of the question and the stored documents, a subset of the documents which have a high similarity (in embedding space) with the question is given to the LLM. This is the retrieval phase. Finally, the LLM uses the returned documents and it's own knowledge to reason and given an answer to the user. This is the generation phase..

@@sepia_tone Still didn't quite get what's the role of indexing here. Relevant documents are retrieved based on the embeddings of the split documents and the embedding of the question. So what's indexing doing here?