Тёмный

I built an image search engine 

Beyond Fireship
Подписаться 389 тыс.
Просмотров 283 тыс.
50% 1

Learn how to use a vector database to build an image search engine powered by a deep neural network from scratch.
Full Image Search Engine Tutorial fireship.io/lessons/image-sea...
Weaviate weaviate.io
What is a Vector Database? • Vector databases are s...

Опубликовано:

 

7 апр 2023

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 357   
@etiennedilocker9729
@etiennedilocker9729 Год назад
Weaviate Co-Founder here. Thank you for featuring Weaviate and creating this awesome video. It covers a lot in just 6min and I think it's a really cool intro to the topic and really shows the power of Weaviate. Also really like that you hinted at more traditional db features such as filtering/sorting/etc. It's those that make Weaviate super powerful for real-life applications. Appreciate the blutness about not being a fan of the builder pattern in the JS/TS client. Would love to hear some more feedback from you and tap into the collective JS/TS experience of your followers to help us design a better , more TS-native API on the Weaviate client. The clients are versioned separately from the server, and we're totally cool with releasing a new major version of the client if it can help improve the DX for anyone that uses JS/TS in their daily work.
@user-gd4iu5uc2r
@user-gd4iu5uc2r Год назад
A problem I commonly face with the builder pattern is that it's unclear to me which parameters are mandatory and which are optional, so I may end up forgetting to set a mandatory parameter and run into nullpointer exceptions at runtime.
@asdfbeau
@asdfbeau Год назад
There's a small army of us all racing toward the same goal now (AI all the things), but for most of us it's a new world; having the 'traditional' features as a frame of reference is what we need to bridge the gap-thank you.
@davidf760
@davidf760 Год назад
@@user-gd4iu5uc2r agree. if it was a step builder this would not be a problem. You are only allowed to build after all mandatory params functions have been called.
@officialabdulrehman
@officialabdulrehman Год назад
Doesn't matter what the topic, JS somehow always gets involved lol
@ayushseth6950
@ayushseth6950 Год назад
True lol😂😂😂
@notsojharedtroll23
@notsojharedtroll23 Год назад
The real marvel of Computer Technology is Javascript at this point.
@ashwinalagiri-rajan1180
@ashwinalagiri-rajan1180 Год назад
This is a webdev project, JS will obviously be involved.
@parthsithvi21
@parthsithvi21 Год назад
That's why it's the GOAT the GOAT 🐐
@aaishikdutta290
@aaishikdutta290 11 месяцев назад
Anything that can be done in javascript will eventually be done in javascript.
@Kim-by5uy
@Kim-by5uy Год назад
we're definitely not tired of the AI videos. Great as always
@SeeWhatIs
@SeeWhatIs Год назад
Looks like made by Fireship
@Dev-Siri
@Dev-Siri Год назад
I was also going to write that xD
@compilererror
@compilererror Год назад
I find them boring, but Jeff can do whatever he wants -- and he will always have my undivided attention because he is the electronic prophet we have all been waiting for
@crowlsyong
@crowlsyong Год назад
Keep em' comin;
@upta
@upta Год назад
I mean, I'm tired of hearing about AI in general, but Jeff can make basically anything fun and interesting so I don't mind regardless of what he makes his videos about.
@DaveGouda
@DaveGouda Год назад
Would love a video comparing the different vector databases. Why did you choose this one over Pinecone?
@AnthonyBurback
@AnthonyBurback Год назад
Possibly due to Weaviate having a local docker container setup available (didn't see anything similar for Pinecone)
@wlockuz4467
@wlockuz4467 Год назад
Pinecone is closed-source and paid
@beyondfireship
@beyondfireship Год назад
1. Open-source and self-hosting 2. AFAIK pinecone requires you to vectorize your own data. For most that means paying for something like the OpenAI embeddings API. But with Weaviate we just ran our own ResNet locally.
@omarjimenezromero3463
@omarjimenezromero3463 Год назад
@@beyondfireship just what i expected, people running locally their own AI for their own purposes, i see more this common than pay for a bigger and better AI suscription where one can not manipulate or use for all.
@DouglasHewitt
@DouglasHewitt Год назад
Your content constantly impresses. As a programmer and an educator myself I'm amazed how you do so much and do it well.
@malikkissoum730
@malikkissoum730 Год назад
This is by far, one of the best videos you ever made. Keep us updated !! You channel could become the new tech news reporter in video format. Thanks for the good work
@bagboy5098
@bagboy5098 Год назад
just imagine Jeff life without Javascript
@xXxDerfoufixXx
@xXxDerfoufixXx Год назад
He will be eff
@vhaangol4785
@vhaangol4785 Год назад
@@xXxDerfoufixXx That's some next level programmer dad joke.
@notsojharedtroll23
@notsojharedtroll23 Год назад
​@@vhaangol4785 there are at least 3 jokes that make this a tor tier joke.
@darsh_shukla
@darsh_shukla Год назад
This can be also done, using CLIP model. And indexing vectors with KNN type in opensearch and using cosine similarity to find similar images. With the CLIP model you can search images by text and image both as input.
@sudiproy4413
@sudiproy4413 Год назад
Can you tell me what's the difference? Also pros and cons of both? You seem to know about these things and I'm new in AI
@deep.space.12
@deep.space.12 Год назад
I think the main difference is that, if you do naïve cosine similarity you probably have to compare it with all images in the DB, hence O(n). In HNSW it becomes O(log(n)) since the embeddings data is "somewhat sorted".
@Paarthri
@Paarthri Год назад
@@deep.space.12 opensearch also uses hnsw
@deep.space.12
@deep.space.12 Год назад
@@Paarthri thanks for the info
@ko-Daegu
@ko-Daegu Год назад
why would he use something worse thou ? (KNN) than a faster more efifcinat and scalable search algo like HSNW
@monabbar007
@monabbar007 Год назад
I love the vector data base, how the data is stored in form of coordination with direction
@BaldyMacbeard
@BaldyMacbeard Год назад
Yeah, vector databases have been around for quite a while, but the availability of open source vectorizer models made vector search more of a viable use case, so we're seeing more and more proper databases with easy to handle clients and all. Unfortunately, the biggest issue with vector search is not the database side of things. They're mostly all the same, relying on the same compression and distance computation algorithms. The big issue is vectorization. For some use cases like text or image similarity you can use huggingface models as-is. But for more business specific things, you'd have to train your own model and that is tough AF.
@amandahugankiss4110
@amandahugankiss4110 4 месяца назад
just fell over backwards watching every thing you said going over my head
@anasouardini
@anasouardini Год назад
More of those please.
@AliciaSykes
@AliciaSykes Год назад
Made it look so simple, I'm definitely trying this out this evening, thanks Jeff!
@notalanjoseph
@notalanjoseph Год назад
2:26 how to “run that command”? What command is he talking about?? Somebody please help
@AliciaSykes
@AliciaSykes Год назад
@@notalanjoseph - It's `docker-compose up` 😉
@bestactionclips9920
@bestactionclips9920 2 месяца назад
am getting an error that Meme class already exists while running the code. Buddy will you please help.
@himanshutripathi7441
@himanshutripathi7441 Год назад
Always remember Jeff, you are awesome and really helping Devs by introducing concepts. Stay classy cheers.🎉
@MMH919
@MMH919 Год назад
Love this kind of video. It gave me a lot of ideas
@RoyRope
@RoyRope Год назад
Got a kinda good idea for them, hopefully will get some time to play with them on work sometime soon. Good video as always!
@Dominik-K
@Dominik-K Год назад
This is super cool! Like how easy it is to do those tasks nowadays
@quarkleptons1036
@quarkleptons1036 Год назад
Amazing work.
@ogredev
@ogredev Год назад
Vectors are so 2022 Q4. Ive come up with trillion dollar concept. Text File Data. You will store all of your data in a single file where where each col will be separated by something like a tab code or comma and a row will be separated by a new line. Beta names are Tabbed Text Files or Comma Text Files. Reply if you’re ready to invest.
@lance3301
@lance3301 Год назад
You son of a bitch, I'm in!!!
@Yashss4617
@Yashss4617 Год назад
i think that`s what people call csv files, python even has extension to manage these type of files. they can also be opened in ms excel
@deep.space.12
@deep.space.12 Год назад
@@Yashss4617 r/whoosh
@-boiadeiro-
@-boiadeiro- Год назад
It's actually a good topic, storing and processing data is still most important thing
@abdulrafiuabdulazeez9924
@abdulrafiuabdulazeez9924 Год назад
There's absolutely nothing for you to apologise for. This is another great video. Please, keep 'em coming as always.
@wwc4168
@wwc4168 Год назад
This is amazing. Thanks.
@poulticegeist
@poulticegeist Год назад
Not tired of AI content at all. This is amazing. Thank you!
@brad1785
@brad1785 Год назад
I appreciate the slightly slower pace of this video.
@miquelvazquez4544
@miquelvazquez4544 Год назад
awesome tutorial !!!!!!
@0906WEST
@0906WEST Год назад
Man i friggin love fireship
@rj27thug78
@rj27thug78 Год назад
Really appreciated mention of not sponsored*
@lawrencemidwinter9416
@lawrencemidwinter9416 Год назад
That was a cool video. I am new to this game and while watching this I get a feeling iofwow that is really cool thing you have shown and I have no idea what he is doing but would luv to see more.
@SIMULATAN
@SIMULATAN Год назад
Yay, finally some docker on fireship 🎉
@walkieer
@walkieer Год назад
After seeing the video on vector databases on the main channel, I just knew this was what was next.
@johannjohannes7214
@johannjohannes7214 Год назад
Soon the own image boorus are going to be fantastic
@mgara514
@mgara514 Год назад
Loved this video, the the test image got me ROFL
@004307ec
@004307ec Год назад
😊nice video, I got an idea for my next project.
@arpitgarg6459
@arpitgarg6459 Год назад
Stop blowing my mind dude!! Too much power in our hands
@thuonglengoc6875
@thuonglengoc6875 Год назад
Great video! Could you make a video about how we can crawl data on a knowledge base, transfer to vector databas and search with it?
@alibahrami6810
@alibahrami6810 Год назад
Nice one!
@amigos786
@amigos786 Год назад
Thank God! Next videos will be on JS
@cedricschacht9445
@cedricschacht9445 Год назад
It's time for a ai driven Javascript framework that runs on the edge😂 And be sure not to mix stuff up, a js driven ai Framework, would essentially be the apocalypse
@crowlsyong
@crowlsyong Год назад
lol
@vmbgify
@vmbgify Год назад
I love how you used the buzzwords "ai", "driven", "js", "framework" and "edge" to mispell "disaster"
@MudHut67
@MudHut67 Год назад
​@@vmbgifyput down the vodka
@Darth_Bateman
@Darth_Bateman Год назад
Or the Singularity. . . ?
@FasutonemuMyoji
@FasutonemuMyoji Год назад
I miss the good ol' days when the most stupid overused buzzword was "the cloud", like we all pretended nobody before 2007 had seen the word WAN on literally any network topography chart from the previous 30 years on it and instead insisted they were super original and clever and came up with something totally new.
@nimitsavant3127
@nimitsavant3127 Год назад
loved it!
@thechoephix
@thechoephix Год назад
I like that the gif in the end was in Yellow stone. No reason.
@zeldaplayergl11
@zeldaplayergl11 Год назад
Seems Valid.
@meikyeon
@meikyeon Год назад
Very helpful and straightforward 。⁠◕⁠‿⁠◕⁠。
@bluelightning2k
@bluelightning2k Год назад
Please do a version of this for text similarity search
@0oEo0
@0oEo0 Год назад
Please keep making these videos. They are non-stop, but so is the pace of AI right now, so they're the only thing helping me feel like I'm staying current.
@GeorgeObregon
@GeorgeObregon Год назад
thank you
@MegaSuperCritic
@MegaSuperCritic Год назад
Damn, I had started to really rely on Fireship for my AI news!
@laptopuser5198
@laptopuser5198 Год назад
Top tier content found here.
@LibertyRecordsFree
@LibertyRecordsFree Год назад
Could you do a similar tutorial for a full beginner in programing (a bit slower, a bit more steps to explain each concepts), to create this image search database, to retrieve only copyright free image / cc0? Nice tuto thought 👌
@Dark_Peace
@Dark_Peace Год назад
You should've used Rektor for your database. I already invested in it. It's the next big thing.
@DougKulak
@DougKulak Год назад
Rektor? I hardly even knew her.
@gokudomatic
@gokudomatic Год назад
I find that personally very interesting. Next step I'm looking for would be to automatically tag images. About AI, I'm happy that Fireship recon that he's talking a hecka lot about AI. But he's right. It's the new trend and it's not going to leave anytime soon. I'd just hope we can talk about it in a more neutral way than an AIpocalypse or an AIrmageddon.
@pepperpeterpiperpickled9805
MORE AI videos, mate! Your videos are always the most informative AND entertaining on the subject
@mkhphp
@mkhphp Год назад
Can you do an episode about Medusa, the open source e-commerce js framework?
@empireempire3545
@empireempire3545 Год назад
mind BLOWN
@blacklistnr1
@blacklistnr1 Год назад
I thought "in the next 5 minutes" is a joke relating to the complexity of a search engine. Yet here I am 5 minutes layer knowing all the steps. Wild times.
@danielivanescobar6656
@danielivanescobar6656 Год назад
i was building a meme search engine for fun and this killed it, i might apply this implementation
@132_priyankrai5
@132_priyankrai5 Год назад
Nextjs thinking video
@canboraciner2256
@canboraciner2256 Год назад
Did you just b64'd different formats of images without converting them to a certain file format and then send them to the vector db? Congratz you just uploaded Spanish, English and French sentences to your db and queried if the English sentence will be the nearest match with the English sentence...
@sage_gamers
@sage_gamers Год назад
Man how did you learn all that? I mean how did you start your journey. Please make a video about your journey it would be very inspirational for many.
@heroe1486
@heroe1486 Год назад
Read the docs
@FaisalAfroz
@FaisalAfroz Год назад
@@heroe1486 That is old school , now you just feed the doc and generate summary using AI.
@lance3301
@lance3301 Год назад
@@FaisalAfroz Yes, this is the way.
@rumfordc
@rumfordc Год назад
when you get really popular, people approach you and tell you neat things in hopes you will make a video for them.
@beinyourguard
@beinyourguard Год назад
if the ai videos help us make our jobs easier, then keep'em coming
@CharmCaster
@CharmCaster Год назад
i um... kind of a beginner but never had trouble understanding your vids... this one was way beyond my lvl lol
@yowaimo890
@yowaimo890 Год назад
woah i never knew weaviate has its usefulness , i contributed in it for my gsoc but later switched to another org
@orion10x10
@orion10x10 Год назад
This is so cool, oh my god
@PlasmaUI
@PlasmaUI Год назад
hey fireship what font do you use for your videos? it looks pretty clean
@user-pm5wi1xg9i
@user-pm5wi1xg9i 8 месяцев назад
Thanks man! My company just changed my job role from a 'RoR developer' to an 'AI Engineer' and you have everything I need
@codemaniadev
@codemaniadev Год назад
Nice video I was wondering when js started supporting top level await 😢
@pontv9005
@pontv9005 Год назад
Considering storage resources, will that scale if you convert the image to a base64 string?
@as-qh1qq
@as-qh1qq Год назад
Yes, I did pause to view and read each meme
@earth9651
@earth9651 Год назад
Don't stop the AI vids. Your last video introduced me to vector databases which I used to easily solve a big, hairy, ugly business problem for my employer. Now they can make a lot more money and I get a pat on the back.
@seraphinduvolzairo5938
@seraphinduvolzairo5938 Год назад
😂
@mlnima
@mlnima Год назад
Please make a tutorial for vector DB and do the same with mongoDB at the same time
@GavHern
@GavHern Год назад
what if you took a text input and ran it through something like dalle to make it use a text input (which gets turned into an image under the hood)
@Dr.Cosmar
@Dr.Cosmar Год назад
Oof... I can follow this, but ill have no odea what I'm actually doing. Still, this was a very good way to get your point across.
@MistrzuShifu
@MistrzuShifu Год назад
My man, you just did Web 3.0 dirty. Blockchain is epic and I hope to see some videos about it too! 💪
@palmer1877
@palmer1877 Год назад
I'm trying to pick a vector database at the moment. Does anyone have any reason why you would pick, say, Pinecone over Weviate, or vice-versa?
@beyondfireship
@beyondfireship Год назад
Weaviate is open-source and can be self-hosted
@lulzmachineify
@lulzmachineify Год назад
Could this be used to build a "hotdog or not hotdog" matcher?
@AnthonyBurback
@AnthonyBurback Год назад
Not even a little bit tired of the AI videos - if anything I'd love to see more tutorials
@vaibhavverma4335
@vaibhavverma4335 Год назад
man i wanna do it
@TheMetadude
@TheMetadude Год назад
Good video but I really wish you would talk slower and add a longer gap ( maybe just a 0.5 to 1 second) between edit cuts. Please bear in mind that for many English is a 2nd language. It is so frustrating having to constantly remind the video that I don't complete most of yours despite the great content. It's great that you pack so much info but it needs to be at a pace that can be more easily followed
@Optable
@Optable Год назад
How can I reach you to discuss an app ive been building at pre dev stage with some of this in mind? I'm utilizing Gans with scikit learn and image on cloud to train the part of my app that will use ai for some valuable specific purposes, but this exact content here is what i needed for my leg heavy work to compile and filter all the exact images I need around defined search. I'd love to get your help, even if general on this for putting my pieces together. Your expertise on all the tools, systems, and use cases are such a great resource Ive had the pleasure of finding. Thanks
@kebman
@kebman Год назад
Oooooh can I also use this to find similar images on my own computer despite resolution differences?
@ansgkffx
@ansgkffx Год назад
Hello, i have veen interested in the thing that you don't love the builder pattern in many of your videos, can you explain why it is bad in your opinion, please?
@kardashevr
@kardashevr Год назад
JS churn -> AI churn -> panic attacks -> depression -> nirvana
@fahnub
@fahnub 8 месяцев назад
Yo this is cool
@Batselot
@Batselot Год назад
I wonder what kind of image similiarity algorithms can we use? And I assume this is completely free to use locally? I don't see any restrictions anywhere right now or I have missed it.
@rumfordc
@rumfordc Год назад
there are countless ways to measure image similarity depending on what aspects of images you care about. if you're given an image of black cat, is that more similar to a black bear or a grey cat? depends on the goals of your app.
@avgvstvs96
@avgvstvs96 Год назад
how do i build a chatbot based on recursively scraped documentation websites with embeddings and vector databases?
@davidcmpeterson
@davidcmpeterson Год назад
With the super AI we have these days it would be quite an easy step to implement just removing watermarks!
@8koi245
@8koi245 Год назад
Gotta love how he reads comments e
@ThysRoes
@ThysRoes Год назад
Can you train this on all my footage that I shot that I need indexed?
@tandyn
@tandyn Год назад
How do I make those visualisations that show the embedding stored on weaviate
@GSBarlev
@GSBarlev Год назад
Realizing you could use this to reverse Stable Diffusion. You feed it a prompt, it generates an image, and then your reverse image search returns the images it --stole-- "learned from" the most to create its output.
@neofox2526
@neofox2526 Год назад
how can i make the search more accurate? Its not even using alot of processing power, is there a way to just set everything to high so it can use all my processing power to make sure the result is accurate?
@JovenAlbarida
@JovenAlbarida Год назад
you have sample repo for this? :) thanks
@content_ai_
@content_ai_ Год назад
Why didn't you use elastic search?
@user-dg4ws2hy8r
@user-dg4ws2hy8r Год назад
We have to update the class name again and again in index.js otherwise it is giving me error that you have already used this class name can any one help me to get rid of this problem!
@kaushikiyer558
@kaushikiyer558 Год назад
I'm always just getting the same input image back as the result. What could be wrong?
@tari7555
@tari7555 Год назад
this is insanely good for AI Art
@thesanjithkumar
@thesanjithkumar Год назад
Can u make a video on pnpm
@AVX512
@AVX512 Год назад
Then use CLIP to generate descriptions of the images you have and you've just generated your own map of your images without manual labour
@DanielHernandez-bb9sj
@DanielHernandez-bb9sj Год назад
I'm not tired of the AI videos. Maybe a new channel: Fireship AI
@ScorgeRudess
@ScorgeRudess Год назад
this guy knows what he is talking about. Its like GPT but for images instead of tokenized letters, or words, or phrases, but is not generative. and does not have transforms xD but.. you get the point.
@yogpanjarale
@yogpanjarale Год назад
Where to host it?
Далее
PROOF JavaScript is a Multi-Threaded language
8:21
Просмотров 260 тыс.
I tried 8 different Postgres ORMs
9:46
Просмотров 381 тыс.
МАЛОЙ ШАНТАЖИСТ
00:34
Просмотров 78 тыс.
OpenAI Embeddings and Vector Databases Crash Course
18:41
15 futuristic databases you’ve never heard of
8:42
Просмотров 642 тыс.
What is a Vector Database?
8:12
Просмотров 45 тыс.
Never install locally
5:45
Просмотров 1,6 млн
How I deploy serverless containers for free
6:33
Просмотров 384 тыс.
10 Math Concepts for Programmers
9:32
Просмотров 1,7 млн
How AI 'Understands' Images (CLIP) - Computerphile
18:05
How Did I Not Know This TypeScript Trick Earlier??!
9:11
Motion Canvas is now Open Source!
4:17
Просмотров 780 тыс.