Choosing the right Chunk Size for RAG 

I’d be interested in hearing more about free and local vector stores. I’d also like to hear more about different similarity measures.

About vectorstores: an overview of those floating around would nice for starting this topic. Chroma, redis, postgre, … what are the main differences and benefits of choosing one over the other. I really like, that you want to stay with local oss setups!

Big issues when it comes to unstructured info i.e. a pdf with boxes, tables and so on. Then chunk strategy is not adapted.

My primary use case for vector or RAG would be to get better coding assistance. Use case would be the closed source code libraries at work. Questions like "how do I fetch customer data from X system?", "show a example of batch job on Y customer type". The legacy codebase spans decades, has different levels of documentation, and often the original author is no longer working at my company any more.

Will be nice try with vector stores and different versions of llama 3

Great Video. Maybe Semantic or Agentic Splitting can improve the RAG results.

PgVector vote here.

So do you use agents, or do you generate RAG content; or can you just feed the documents into the LLM and instruct it to only reference the provided documents. Ive gotten very good responses from an LLM with the last mode and I didnt need to learn python to do it

Very interesting to watch and a lot to learn! Thanks, Matt. Comment: Is it not possible to make the evaluation of the outputs a bit more quantitative than qualitative? For e.g., can I spot the five things in the output, or mention of the 3 critical facts, etc. Of course this will mean we'd need to have a better set of RAG input as well as expected output. What are your thoughts?

This "top answers" script you reference ... I'd be interested in that script, if it is open source. I would like to evaluate some variables like chunk size, model, re-rank top-k, etc using this "top answers" technique.

I found it quite tricky to setup a reliable RAG! Pluse dealing with PDFs is a whole different animal :-/

🍸

That thumbnail kinda makes it look like an "I" instead of a "U". Just a warning 😅