The Power of Vector Databases For Knowledge Search 

thanks for the feedback! ooc are you not interested in vector databases in general, or you are you interested but just want to see things more from the perspective of the code? you may not have made it this far, but fwiw in this one the coding part starts at about 14:45. I think it's important for software developers to have vector databases on their radar and in their toolbox these days, be it Nuclia or something else.

@@codetothemoon I’m interested in learning I guess. I’m definitely interested in all things databases and would love to see a rudimentary implementation. I’d guess, like me, most of your viewers are rustaceans who are looking to enhance our understanding after reading the usual books.

Well he has to maintain the channel and provide us good content, and to be fair, Nuclia is a nice option for vectorDBing. Keep em coming, @codetothemoon !

Honestly, coming from web dev with javascript, I find these videos highly informative. Would happily sit through a bunch of ads for the comp sci info alone. That said, nuclia is actually useful, and I plan to work with it this weekend. Spent last weekend setting up private GPT only to realise the limitations, this looks so much more polished.

thanks for the kind words😎

nice! yeah it's definitely worth taking for a spin.

😆

thank you! Glad someone caught this, was worried the joke would slip past everyone 😎

I agree that the potential here is huge!

thanks so much for the kind words! I'd love to make such a tutorial - just need to figure out what the right medium for distributing such a thing would be. In the meantime, check out this video from Jeff of Fireship if you haven't already - he's got some pretty helpful tips - ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-N6-Q2dgodLs.html

@@codetothemoon Thanks, I'll check it out

I believe what you describe is roughly how it works. To your point, it seems like there'd be limitations of this approach - for example if part of a document was deemed relevant to answering a question, but additional context was provided elsewhere in the knowledge box that changes the meaning of the excerpt deemed "relevant". Maybe that scenario would yield an answer that isn't accurate. I haven't personally tested these boundaries myself, so I can't say for sure whether this issue exists and whether it is severe or not. It seems like LLMs that have accept massive prompts (tens of thousands of tokens) are an active area of research and would probably help if this problem does exist.

@@codetothemoon Yes I remember seeing a paper on models with over 10k tokens without much performance degradation. That would certainly help. Thank you for the reply, and really enjoy your videos!

If you think that knowing things without having to ask an llm is useless, then sure

How exactly do you use books...

if your goal in reading books is to answer specific questions that you already have, then I think the answer is probably yes. But the thing about getting answers to questions is that it's only helpful when there's a gap in your knowledge that you're aware of - I think a lot of the knowledge to be gained from reading books comes from filling in gaps in our knowledge that we didn't even know existed.

@@codetothemoon fair point

don't feel sorry for it! 🙃

my understanding is that it first does a semantic search of your entire knowledge box to find information that is likely relevant to answering the question, then feeds all the relevant information, along with the question, into an LLM. Nuclia says there are also some other clever techniques involved that they haven't gone over with me, but that's the high level idea.

nice! would love to hear how it goes.

🚀

thanks for watching!

thank you so much for the kind words! I definitely aim to make these as accessible as possible, I'm very happy whenever I hear that I've been successful 😎

it's indeed unclear whether "dark energy" is the right term - what do you think would be a better one?

@@codetothemoon They don't see this power - how did they define the color? :) God is Spirit. What color is God? :) Pontius Pilate, when asked about the truth, used the impersonal pronoun "what", this is a delusion of many pagans. They have karma and nirvana there - a fictitious impersonal god. God is, first of all, a Personality, and we, his small likeness, are also personalities. A man and a woman are one person consisting of two personalities. :) God also consists ... of Three personalities. Prayer of Jesus to the Father. 20 Neither pray I for these alone, but for them also which shall believe on me through their word; 21 That they all may be one; as thou, Father, art in me, and I in thee, that they also may be one in us: that the world may believe that thou hast sent me. 22 And the glory which thou gavest me I have given them; that they may be one, even as we are one: 23 I in them, and thou in me, that they may be made perfect in one; and that the world may know that thou hast sent me, and hast loved them, as thou hast loved me. 24 Father, I will that they also, whom thou hast given me, be with me where I am; that they may behold my glory, which thou hast given me: for thou lovedst me before the foundation of the world. 25 O righteous Father, the world hath not known thee: but I have known thee, and these have known that thou hast sent me. 26 And I have declared unto them thy name, and will declare it: that the love wherewith thou hast loved me may be in them, and I in them. (Jn.17:20-26)

@@codetothemoon invisible... the hand of God :) At the end of time, Jesus will restore the state of the universe to its original state - to nothing. :) We must understand the vanity of our worldview if we are based on what is temporary. Jesus reveals to us that which is eternal. Of course, we do not need to go to the extremes of pagan ascetic monks. God has given us everything for life and godliness. By mistake, 30 years ago, I wanted to become an Orthodox monk, did not eat meat, grew my hair. Then I began to systematically read the Bible. God revealed to me through the New Testament that all my religious monastic aspirations are a human delusion. There will be a Resurrection of the dead in eternal bodies for the Judgment of God. A believer in Jesus does not go to the Judgment of God - Jesus bore all the punishment. At the Judgment, the verdict is on those who do not submit to the meekest Lamb - Jesus. After that, God will create everything new - a new heaven and a new earth.

Take your meds

how do you choose which verses to post?

@@codetothemoon :D

@@martinsjoborg4950A fiery groundhog day awaits you, I feel sorry for you. Do not walk the path of the devil - it will be very sad there... all eternity...

@@codetothemoon Not a semantic search :) - there used to be a paper index on the Bible - "Symphony" is called in Russian. Then we started to distribute the Bible in doc format. Then there were electronic Bibles. I use "BibleQuote 5.0", there is a newer version - it's enough for me. I am looking for places familiar to me in parts of words or whole words. Paper "Symphony" previously allowed to find only the first word. You can already create a semantic search using machine learning. :)

yeah my guess is that they do that to throttle bot signups, maybe try using login with Github or Google Workspaces?

I know! I didn't notice this until after posting the video, was hoping it'd slip under everyone's radar, but you caught me.... 😎