Тёмный

Can I Make A Search Engine From Scratch? 

Equalo
Подписаться 26 тыс.
Просмотров 107 тыс.
50% 1

I set out to make my own search engine. Yes there are already options like Google, DuckDuckGo, and Bing. But creating my own helps me better understand how they work, and I can make it function however I would like. I don't know if I will ever host this for the public to use. For now, it's just a project I'm working on. I would love to be able to implement complex queries and word associations, and maybe someday include image search as well.

Опубликовано:

 

5 окт 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 150   
@seraphimwiththecheese5880
@seraphimwiththecheese5880 4 года назад
Interesting video! I learned a lot about how search engines work. Keep it up!
@_equalo
@_equalo 4 года назад
Thanks Seraphim!
@busterdafydd3096
@busterdafydd3096 3 года назад
6:05 another fact about your search engine is that it will probably only target english pages and not the whole world's languages
@Foxtrot445
@Foxtrot445 3 года назад
Him: Hey google, how do you sell a child? Google: We have a Wikpedia article for that! 4:18
@Phiwipuss
@Phiwipuss 3 года назад
People also ask: How much is a kid worth on the black market?
@GamingFemboy
@GamingFemboy 3 года назад
@@Phiwipuss no just no
@Phiwipuss
@Phiwipuss 3 года назад
​@@GamingFemboy I got it from the video. -_-
@user-oas
@user-oas 3 года назад
@@luke-bookbear ?
@silverhoney6969
@silverhoney6969 8 месяцев назад
Yeah that was a weird example for the creator to use!?
@박민욱-h9d
@박민욱-h9d 4 года назад
Try word embedding. It basically changes strings to vectors and index said vectors. When you search you have to vectorize the search term and compare it with saved indexes. It should be faster than using strings to compare
@seankim8123
@seankim8123 3 года назад
Hi i have some questions
@АлексейГриднев-и7р
Better yet one can use proper database (like MySQL or SQL Server) to increase the search speed. If there's one table website URL and full text and other 2-3 tables with the most frequent words, bigrams, and trigrams per URL (ordered by frequency), that will be much more efficient than always relying on full-text seach, and it would also improve relevance.
@footiecyclo
@footiecyclo 4 года назад
Thanks Pewdiepie
@Fortnitesucks1-w2d
@Fortnitesucks1-w2d Год назад
Lol you are so funny i wish i was as funny as you🙄🙄🙄🙄🙄 Ur like really not funny
@StrangerHappened
@StrangerHappened 4 года назад
The lad is pretty adorable, I must say; an interesting content.
@burtmcgurt3584
@burtmcgurt3584 2 года назад
Awesome start! I am excited to see where you go with this!
@mrbushy7262
@mrbushy7262 4 года назад
I watched the video and thought you had 456k subs because it was so good 😅 you earned a sub with notifications.
@_equalo
@_equalo 4 года назад
Thanks mrbushy!!
@SlappyRB
@SlappyRB 3 года назад
in the first 1 minute, i already am enjoying this video
@АлексейГриднев-и7р
Great project! Are you aware of nltk package? It is capable of removing stop-words, stemming, word and word collocation frequencies, and so on. I believe that could help greatly with indexing.
@_equalo
@_equalo 3 года назад
Good point! I’ve used NLTK a little, but wasn’t confident enough with it to apply it to this project. Maybe it’s time for me to dig into it
@Snakeythepuppet
@Snakeythepuppet 11 месяцев назад
@@_equalowhat happened to your channel?
@manishbhati2722
@manishbhati2722 3 года назад
I want to know further about indexing. If possible, make next part of this search engine video.
@alisiddiquii
@alisiddiquii Год назад
Thanks for this interesting video, I've learnt about how search engine works after passing my exam
@vikinggeorge7007
@vikinggeorge7007 Год назад
The second I saw python I stopped the video to rethink my life
@moondev369
@moondev369 2 года назад
I'm working on something similar in machine learning. I'll let you know when i am through! Glad to see you workin do hard on it. You can do it!!!
@waynefilkins8394
@waynefilkins8394 2 года назад
That's probably the only way to compete these days. It would take so much time and manpower to build something like Google, but if you can incorporate machine learning into it, might bypass a lot of the stuff they had to do the slow way
@somewherenear3003
@somewherenear3003 3 года назад
Oh hey! I decided to make my own search engine this year back in 2020 too. Now is the time I'll be doing this project. I'll be sharing my progress on my channel.
@spreadItWide
@spreadItWide 3 года назад
did you start yet?
@Carambal81
@Carambal81 4 года назад
Your videos are very informative, I just learned how to sell my child! :P (@4:18)
@_equalo
@_equalo 4 года назад
Haha I’m always happy to help. Glad you found that part useful
@divyanshusah2809
@divyanshusah2809 3 года назад
@@_equalo 🤣🤣🤣
@sirrealsam
@sirrealsam 4 года назад
00:29 had me crack up, haha :-D
@wickederebus
@wickederebus 3 года назад
so, im guessing this project did not get a follow up video?
@AriJankelowitz
@AriJankelowitz 4 года назад
Great video with excellent music choice!
@_equalo
@_equalo 4 года назад
Thanks!
@myztartupjourney6772
@myztartupjourney6772 2 года назад
Equalo you should make a part 2 to this video!!
@PaAGadirajuSanjayVarma
@PaAGadirajuSanjayVarma 4 года назад
Good work bro.try to use hashing of words in a web page and store them in a hash table.I think it might increase it.search hashes instead of word to word
@_equalo
@_equalo 4 года назад
That’s a smart option! I’ll try to do a follow up video trying that and a couple other methods
@annuritv4617
@annuritv4617 4 года назад
I'm ready to help.
@ryanmacalandag5279
@ryanmacalandag5279 2 года назад
I'm also interested. I'm looking into building a search engine for a group of less than 200 related websites only. This is an insightful video. Hopefully you you post the code. Thanks
@zanjeev8654
@zanjeev8654 Год назад
Did you create your own search engine?
@Holleylifestyle
@Holleylifestyle 2 года назад
This was amazing. Gotta get to work. Thx for the great content.
@bruinebeerinhetblauwehuis
@bruinebeerinhetblauwehuis Год назад
Very interesting project from Michael Falk!
@prithivirajr7918
@prithivirajr7918 3 года назад
WHATS THE LEVEL OF YOUR SERCHENGINE NOW
@JamesScottGuitar
@JamesScottGuitar 3 года назад
How’s the project going now?
@meerachaturvedi9050
@meerachaturvedi9050 3 года назад
Its dead
@vihaankedia8134
@vihaankedia8134 3 года назад
could you send your source code for the search engine
@nyancat5140
@nyancat5140 3 года назад
Yeah, I was trying to do this by myself. A link to the code would be great!
@adamayala3906
@adamayala3906 4 года назад
Nice
@_equalo
@_equalo 4 года назад
Thanks Adam!
@av3stube480
@av3stube480 2 года назад
Was the number of links you parsed from the second batch of pages referring to the total links or the amount of unique links? It's pretty easy to think of an example of a set of Wikipedia pages that all link back to one article, or even two or more pages that all connect to each other and create endless loops. Of course, looking for changes in website contents is necessary, but avoiding crawling over the same pages too often should speed up the rate of expanding the database and reduce the strain on the hardware in the long term.
@dh2032
@dh2032 Год назад
you would have a process. search, in document, duplications, but you would still Identify if the are in deed duplications, and not just similar, and even if it was real duplications, it was still linking for a reason, unless it just the home button or something like that?
@movocode
@movocode 3 года назад
4:19 Did anyone see what he typed in the Search !! 🤣🤣🤣
@divyanshusah2809
@divyanshusah2809 3 года назад
Lol..🤣🤣🤣
@Raghuvaran13899
@Raghuvaran13899 3 года назад
"How to sell your child"🤣🤣🤣
@builder481
@builder481 2 года назад
Amazing video and also very funny lmao
@tiagotiagot
@tiagotiagot 3 года назад
How about using GTP-style "tokens" to encode both page contents and search keywords?
@snehashisbera8316
@snehashisbera8316 2 года назад
thanks bro, I Learned a lot today by practical way.
@sparrowEP
@sparrowEP Год назад
2:57 when u open OBS for the first time:
@marvelousmarvelxx3889
@marvelousmarvelxx3889 3 года назад
Subscribed!
@coyzee1
@coyzee1 3 года назад
LOL, 7.35 She gushed over him. Sorry about that. Very interesting, thanks for the vid.
@universenerdd
@universenerdd 4 года назад
From scratch just bothers me for some reason
@aa-qz2ej
@aa-qz2ej 3 года назад
Just subscribed, great content!
@mathcloud
@mathcloud 3 года назад
So where is your search engine? No link?
@ryxn3x
@ryxn3x 4 года назад
👍
@krissna9697
@krissna9697 Год назад
To create a search I'm a coumputer science student beginner .there are lot of fields in computer science major, in all of these which field do I specifically need to take to develope a search engine? please
@mawrahassan1973
@mawrahassan1973 2 года назад
I think this is the guy who has a channel named "what I've learned"
@OfficalOxy
@OfficalOxy Год назад
duckduckgo is getting popular
@subhakantasahoo9760
@subhakantasahoo9760 3 года назад
I searched for it after whatspp's new january privacy up-to-date 😃😃
@busterdafydd3096
@busterdafydd3096 3 года назад
5:10 it wouldn't be a bad idea to chuck the word recursion in there and state that that's what your crawling and parsing is doing
@AndrewsLorenzana
@AndrewsLorenzana 3 года назад
Hey, how is your search engine going?
@FilippoBerardo
@FilippoBerardo 9 месяцев назад
question: The search of pages can be done even with ip addresses, cycling every possibile number? Or only with links? Using links means you must have a list.
@melindamassey14
@melindamassey14 5 месяцев назад
Your choice of wiki topic to search????
@sebastianjohannes2316
@sebastianjohannes2316 3 года назад
What coding tool do you use to create the search engine?
@callofdutymobile1074
@callofdutymobile1074 2 года назад
More interested in the coding aspect you share source code or how to get started
@Plexversal
@Plexversal 3 года назад
9:10 lmfao that gif
@jasonfanclub4267
@jasonfanclub4267 2 года назад
Good content
@RageBird7200
@RageBird7200 Год назад
What about Bing Users?
@NickKartha
@NickKartha 3 года назад
Hope the project progressed beyond python scripts.
@iamanishkumar
@iamanishkumar 2 года назад
Why aren't you making another video?
@merrytantrimilleniatobing3740
@merrytantrimilleniatobing3740 2 года назад
Can you give us the tutorial to make that search engine, sir?
@phillipspodcast
@phillipspodcast 3 года назад
Good video mate
@Zz-ol5bx
@Zz-ol5bx 3 года назад
But won't this method be too much time consuming?is there any other way to make it faster or automate it bro
@ToniMartiAlbons
@ToniMartiAlbons 3 года назад
Nice video 👌
@busterdafydd3096
@busterdafydd3096 3 года назад
5:27 so now you start indexing. keywords in the websites links to websites
@namithshetty
@namithshetty 3 года назад
Bro go on title and headline base because then only you can get the information
@menopriezvisko2232
@menopriezvisko2232 2 года назад
try arango database for inverted index searching
@legixstudio6713
@legixstudio6713 3 года назад
7:45 is'nt scott morison the priminister
@sprinteroz2239
@sprinteroz2239 2 года назад
Is this on github or you keeping the code private?
@SankiShekher
@SankiShekher 3 года назад
😎My friends says - You can't compete... 👩My mother says - Never give up... 😇"I follow what my mother says"
@RahulSharma-oj4ik
@RahulSharma-oj4ik 3 года назад
PewDiePie's Got A Search Engine😀
@MoyaNandaoOfficial
@MoyaNandaoOfficial 3 года назад
Please upload for me the download link of your search engine. Thanks
@Ab-cj6gl
@Ab-cj6gl 3 года назад
i was planning to build something similar but ain't gonna happen 😂
@makersspace565
@makersspace565 3 года назад
petabytes=too expensive
@allanbenedict4558
@allanbenedict4558 3 года назад
Like it
@twansmith2622
@twansmith2622 2 года назад
Hello r u still working on this
@gspapp
@gspapp 3 года назад
link?
@madabtMSK
@madabtMSK 3 года назад
good try!
@dr.official9852
@dr.official9852 3 года назад
Wait a minute
@vhwjpzf1z0fi73a
@vhwjpzf1z0fi73a 2 года назад
Any news on this?
@SlumDawgSaint
@SlumDawgSaint 2 года назад
Did you make it, i want to make it and make it public and add free and zero tracking of people good gearh engine add free bias free...;)
@Randomynous01
@Randomynous01 2 года назад
Google no longer operates on the number of links that link back to it, but rather PRIORITIZATION
@pavannaidu759
@pavannaidu759 2 года назад
can you share the code
@apang1831
@apang1831 Год назад
"how to sell your child"
@busterdafydd3096
@busterdafydd3096 3 года назад
You didn't even paritially cover indexing in this video
@meerachaturvedi9050
@meerachaturvedi9050 3 года назад
Vedio is just a look
@meerachaturvedi9050
@meerachaturvedi9050 3 года назад
Indexing is a big process
@meerachaturvedi9050
@meerachaturvedi9050 3 года назад
Want to know
@raymondchiwade
@raymondchiwade 3 года назад
How can I contact you
@saidkauzu7831
@saidkauzu7831 4 года назад
I have an idea do you want listen?
@meerachaturvedi9050
@meerachaturvedi9050 3 года назад
Yup
@jessetate7601
@jessetate7601 3 года назад
Use folder and use letter to me the floder
@sparrowEP
@sparrowEP Год назад
"mom can we have google" "we have google at home" google at home
@RmbTfitness
@RmbTfitness 3 года назад
I laughed a lot... watching your video
@rylandsquires4886
@rylandsquires4886 3 года назад
Watch at 1.5x speed
@Naderium
@Naderium 3 года назад
420th like
@chris_the_nerd09
@chris_the_nerd09 8 месяцев назад
Opera is better than Google
@prudvi01
@prudvi01 3 года назад
duck gang
@ahsanabrar880
@ahsanabrar880 2 года назад
please share update.
@ahrorbekabdullayev2193
@ahrorbekabdullayev2193 3 года назад
I'd prefer wikipedia api
@TF-cn6oj
@TF-cn6oj 2 года назад
Update please
@marlonlopez4154
@marlonlopez4154 3 года назад
Can you email me really interested.
@elliotsearchengine8626
@elliotsearchengine8626 3 года назад
We did something different Check ✔️ Elliot Search
@prestonjudearnold2814
@prestonjudearnold2814 3 года назад
how
@mathquik1872
@mathquik1872 3 года назад
Are you on onlyfans?
@joshclaassen616
@joshclaassen616 4 года назад
Check out safenetwork.tech there are a few people pondering how to implement a search engine and you may enjoy the challenge.
@twistah
@twistah 3 года назад
Am I the only one who uses duckduckgo
@techample
@techample 2 года назад
Yes. Can't imagine someone not using Google😃
@rxkshan
@rxkshan 3 года назад
chuper
@ufrazor5543
@ufrazor5543 Год назад
chup dalle
Далее
КОТЯТА НАУЧИЛИСЬ ГОВОРИТЬ#cat
00:13
ЭТО НАСТОЯЩАЯ МАГИЯ😬😬😬
00:19
Why I Quit Scratch
10:18
Просмотров 24 тыс.
Run your own AI (but private)
22:13
Просмотров 1,5 млн
I built an image search engine
6:44
Просмотров 292 тыс.
I Made a FAST Search Engine
8:17
Просмотров 152 тыс.
How Many Potatoes Does It Take To Run DOOM?
16:59
Просмотров 3 млн
How Search Engines Treat Data - Computerphile
10:12
Просмотров 132 тыс.
Malware Development: Processes, Threads, and Handles
31:29