I just wanted you to know that after almost 10 years after abandoning programming (because of... reasons), your videos made me realize I want to learn and start practicing again. I did forget that programming was supposed to be fun.
I love how people say, in any programming "teaching" style of videos, that they'd never learned as much as the said video after 5 years of uni, or after 3 semesters of X discipline. One of my theories is that you learned a lot during those times but after the video it solidified a certain concept because of how the video is put together.
Some optimization tips: - Since you're lexing the file anyway, you can simply skip tags. Skip everything between ''. That way you can remove the XML parsing. - I believe at least with western languages (like English), words of 2 characters or less are often simply omitted for TF/IDF indexing. Cuts down on the has table size and generally doesn't loose any information.
As the absolute beginner I am, do I understand your comment correctly? Lexing: the process of reading the file Has table size: amount of entries in a "list" of some kind that contains everything that is supposed to be filtered out Thanks in advance
@@datpudding5338 I haven't watched the video (yet), but I think "has" was supposed to be "hash", becoming "hash table size.." which would make more sense to me, unless it's terminology over my head.
@@datpudding5338 the lexer splits the document into specific chunks that are easier to work with than the whole string. here it basically just splits the text contained in the XML along the spaces. but i think the term comes from parsing where it refers to splitting a string into predefined tokens like keywords and variable names. a hash table is a specific data structure that‘s very useful to quickly look up data in. but your extrapolation is correct, it‘s just a very fancy list with special properties.
какой приятный слегка заметный русский акцент у автора, спасибо за полезное видео, очень понравилось) what a nice slightly noticeable russian accent the author has, thanks for the useful video, I really liked it)
Came for Rust code, got philosophical thoughts about Internet. I was not disappointed :D And yeah, Rust is very well build and easy to use once you understand how it works. Traits are amazing.
25:50 Absolutely fricking loved what you said about turning off the internet and becoming one with the project you’re working on. It’s something called flow state, explained beautifully in the book Flow by Mihaly Csikszentmihalyi.
The story about computer speed and how programmers added delay to make people trust it, and how now it's the opposite lol made me laugh hard. I like these kinda videos, I miss my C years :/
I learn something new and useful every time I watch a stream. If it’s not about the main subject itself it’s usually just some random bit of bash or another arg for a commonly used command. How bout that
HolyFUck. There is no materials on how to implement a search engine on RU-vid. And here are you, and not in some shitlang, but in Rust!! I have very big expectations. Because whole playlist with several 2hour videos does NOT look like bullshit to me. It is serious SHIT. THANKS
55:26 - Well, I recently ported a legacy PHP image generator to JS (offloading the work on the client browser instead of the server). And I actually hardcoded a 1 or 2 seconds delay into the presentation of the result to actually indicate to the user, that work is indeed being done. 😂
I completely agree that rust is for smart enthusiastic people who like sending their time making really complicated abstractions. That's the problem with rust is that you waste your time building these abstractions. You send your time debugging your programming language knowledge rather then debugging the application.
One reason your program might be slow is because I don't think those functions that take in readers and writers do any buffering so you might get a decent speedup by wrapping your files into std::io::BufReader and std::io::BufWriter respectively
1:43:30 yeah, trying to serialize it to JSON is hard. If I were you I'd store it in an SQLite file. It'd be easy to write the query to add each entry in `tf_index` to the db and also straightforward to parse it later
54:05 my grandma was working as an accountant, she has started with abacus. When first mechanical calculators appeared in their bureau, she was double checking every calculation with an abacus. When first electronic calculators started appearing in their bureau, she was double checking them with the mechanical calculating machine. When first PC's were installed she started double checking the calculations on a calculator.
Hey Tsdoing, i love your skills and videos ,just one request is that when you do a video can you please add some study materials and references that we can look up before and the basic prerequisites needed to atudy to easily follow your chain of thought. Also some kind of announcement of your next video topic so that we can read it and be better prepeared
I think that Google Search is becoming shitty because nowadays there are lots of AI generated articles and people who specifically make it their JOB to bait Google results into their articles with SEO. But yeah, old Google was way better and I'm just waiting for a better search engine to come along.
@@matroqueta6825 it's just goodhart's law; "Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes."
27:00 Disconnecting from the internet for some time is quite brililant. I was disconnected recently but not by my own will, rather through circumstances. All I could do was work on my old projects and I couldn't actually download any libraries.
Result has a method called "expect" and it unwraps it but if there's an error, it prints out a message you give it, which is basically what you were doing with unwrap_or_else but in less code. If you want formatting, just call .expect(format!("ERROR: ..."));
45:25 After learning about the term "nerd sniping", I searched for the xkcd comic and spend about and hour on that website before returning to the video xD is there also a term for that?^^
Ryan Dahl created npm in C and then Deno in Go and rewrote it in Rust - and he said he will never ever start a new project in C. So here is the work ... and the opinion of a productive creator. Also I think there is a phase in every language were libs and frameworks etc. have to be created by smart enthusiastic people. Like in PHP - nobody would care if you start now a project like Symfony or Laravel. But back then it was huge and important.
Tf idf is pretty good. U can use a bit of extra stuff to make it ridiculously good. So bm25 changes the formula by a bit but it's the same core idea Modern stuff use ml embeddings but for a large query what u would usually wana do is run bm25 then take those results and rank them by the embeddings.
"I know this is not traditional in the rust community, but this is my project and i like to do things my way" 👏👏👏👏👏👏👏 i wholeheartedly agree! one thing i struggle with is feeling like i MUST do things exactly 1 specific way or i'm absolutely wrong even though it might still work. I understand the benefits of having everyone sticking to one established way of coding but Rust compiler is already strict enough, for good reason, don't restrict me even more on top of that.
I personally found it to be not THAT bad. It took me a while to become productive in rust since there is such a large shift in how you implement things. Certain approaches used in C is very hard and/or bad in rust. I would consistently spend my time googling on how I can work with certain data structures and then realize there already was a trait that gives me what I want etc. I think the greatest realization was when I finally understood that rust has a different paradigm to C (Rust is declarative compared to the imperative C). Truly understanding how to work with the borrow checker came with experience. My knowledge in C did help however since rust is sensitive about datatype sizes (see sized and unsized traits) and the knowledge from C and assembly carried over into it. Nowadays rust is clearly my favourite programming language, I've largely replaced C and C++ with Rust, Go and Python.
@@yeetusdeleetus Thank you for all info. Another question: Do you think you will miss a lot of C++ libraries while working in Rust? or Rust has most necessary libraries?
Windows has something similar (but not quite as good) if you hit Win key and Plus key at the same time (get out of it with Win + Esc), if you're on Windows
I wana make a lang that is like c but safe and functional. Like varibles are functions that return stuff. And if u need weird stuff u have something like a class or something where u can modify the intern varibles of the function
It sounds like you should not continue to torture yourself with rust. That's the vibe you give off. Rust is bad but I use it and complain and every 30s I must remind people of my superiority.
I asked AI about the color theme of your IDE: As an AI language model, I do not have access to real-time information about the current state of Tsoding's IDE or any other developer's IDE. However, based on some of Tsoding's past videos, he has used different color themes for his IDE, including a dark theme with a black background, white text, and colorful syntax highlighting for different programming languages. In some of his videos, he has also used a light theme with a white background and dark text. It is possible that he may have changed his IDE color scheme since then, but without any specific information, I cannot say for certain.
HUH non-idiomatic rust ???? you clearly didn't read da roulex , you now must shall be updating index for as much years as many clippy pedantic lints you have violated
also if you don't wan't to wait too long use sparse-registry with -Z sparse-registry flag or config in .cargo (will be in 1.68 version in march) can't send links but is's in rust blog
@@ArisenDrake What's so hard about fopen and checking/handling the error? It's what you do in Rust anyway. Do you think unwrapping is some magical statement that solves errors?
@@ElPikacupacabra The difference is simple: Rust ENFORCES error handling, while in C, someone could forget to do it. Humans do mistakes. Also, while this is subjective, Rust's API for this just looks nicer.
@@ArisenDrake Rust doesn't really enforce error handling. You can simply ignore errors and the program will panic. That's no different from C erroring out. Regarding your desire for syntactic sugar: sure, the dedicated mechanism makes it nicer, but you also pay with complexity. IMO, it's not worth it. It doesn't solve a significant engineering problem.
@@lame_lexem No, I mean the download button. Your previous videos are downloadable which is good because there's no wifi on the outside of my house. Your videos are good, bro.
@@RedstonekPL discord has automatic system that bans people because of strange or abusing requests to the api, betterdiscord and other things dont change how api work, but dear tsoding says he wants to use *custom* client, and things like cordless dont do requests identical to official client, and plus he is somewhat famous. ;)
