Software dependencies can be extremely helpful, but do you really need a whole framework for that? Today we'll be discussing software dependencies, and my approach to them. The site I showed: blog.transrigh... Discord: / discord
I built a PHP app 20 years ago before the advent of composer (PHP package manager). All dependencies were manually copied into the project and updated when needed. That project still runs to this day. How? A single compatibility.php file that re-implements deprecated functions that were removed from PHP over the years. That's it. The app has survived 4 major PHP versions and STILL WORKS! The compatibility script isn't even that big, coming in at less than 100 lines of code.
Many people greatly underestimate the true long-term costs of dependencies. It takes time to learn how to use them, often with incomplete or outdated documentation. It makes every update more fragile and setting up the project is more complex. It potentially makes you vulnerable to supply chain attacks. It makes you dependent on people you don't know to keep everything up to date. It makes troubleshooting harder. There are a few things were using dependencies makes sense (cryptography/security, anything with dates & timezones, Unicode stuff etc) but I know people who import an entire library just to get rid of a 15-line function and that's just frustrating.
It really is frustrating to deal with. This kind of mindset lead us to have like 2-4 competing phone number validators in a production code base I worked on. Like, we can just do it ourselves and drop the broken packages!
😂 bro u can't n shud t compare a language to a framework 🤦🤨 makes no sense and this coming from a JS hater!! That's like me comparing .Net to React or Angular where in .Net everything is spoon fed to the Dev's while React n Angular, Vite, Vue Next etc are all perfect examples of open-source gone wrong. Hated MS but then the others just became it so they all trash...even Rust 🤷
Worked at a place where a guy "with more experience" wanted me to use a library for a single datetime thing.. that I had already implemented myself... The library was even left behind - there were no future updates, and no further benefit to the project whatsoever. All to replace a few lines of code with "something simpler". Thankfully I don't work there anymore, or even with frontend for that matter. Absolute shitshow with people that think their idiocy is "best practices" and sign of competence. Derisive, I know, but this was my first job years ago, and he had YEARS of experience on me, could not formulate a reasonable case for using this library and yet insisted (blocked the PR). It led to actual issues dealing with the library itself, and he was in many other cases like this too. Run for the hills when people cannot argue their case from first principles and are not happy to do so if you ask for elaboration (if people ask you to do something, without being able to qualify what concrete problem you're solving). It's always a shitshow with those people, a ticking bomb waiting to explode.
Currently the Lazy types are still nightly since they have some discussion around type system funkiness around them but hopefully that'll be sorted out soon
@@cocacodes since rust 1.80 (which is stable since like a week) both LazyCell and LazyLock (which is probably what you want in this case) have been stabilized
I feel like brainlessly adding dependencies is a very frontend thing to do. Im 99% backend end only add a dependency if i absolutely have to which does only that thing that i need
It is something I see more commonly with more frontend oriented engineers, however I see this happen with the dependencies themselves as well. This in part is choosing the wrong tools for the job, but is also just the dependencies themselves needlessly depending on things.
And a dependency in the backend better do *exactly* what I need it to. The skepticism with which I read those docs… This app is important to me. I’m not just going to invite a bunch of other developers I don’t know into the project (which is what we’re doing when we import a dependency) without vetting first.
I see it plenty with backend engineers using JS, especially if JS wasn't their first language. I tell you, I am constantly screaming everywhere I go like a madman, "Don't install MomentJs." ... or lodash just to use _.forEach...
If you brainlessly add dependencies on the frontend your clients will definitely suffer, so I don’t get that logic.
Месяц назад
Nah don't worry, I've worked with lots of backend deva who do the same thing. Personally I have a shortlist of two or three libraries I use all the time, but for the most part I like to just write and reuse my own libraries.
hobby projects are nice because you can do exacly what you want. in an enterprise field it is usually the other way around. managers decide to select a tool that can do everything, which will come with a lot of dependencies
These things can happen in enterprise, but it requires more work. I personally was able to drive our choice of tooling at the company I previously worked at. I couldn't get anyone to rebuild the whole stack, but what we were building next I always put forward my input and had managers that trusted my experience. I know this isn't the experience everywhere though, and is more likely to happen when the software is the product, not for the product.
So basically you switched from a framework to using a new language with a few extra libraries. This video is not about reducing dependencies. Its about using the correct tool for the job. Edit: not hating, great video but I was expecting something different reading the title.
My argument ended up falling a bit more towards "use the right tool", but I feel the idea I wanted to show was that you don't need big and bold dependencies to do the job in the first place. The "right tool" could be argued for axum for instance as it will do the job, but it has a lot I really don't need and that costs me as a developer. It could be argued with that logic it isn't the right tool here to begin with, but that's harder to tell someone who likes and biases towards a tool.
I agree. The site was very simple, so using Angular was way overkill. It's fine to use Angular for it - I doubt the site performance was negatively impacted from having those dependencies - but if the goal is to cut down on those, then dropping Angular was the obvious first step. Angular could package all of the dependencies in a single package if they wanted though, so the number of dependencies feels like a poor metric. And obviously, if for some reason the site has to run without Javascript then finding another language is a requirement. A good reminder that you don't always have to grab the same hammer from your toolbox but perhaps a bit contrived as an example of dependency bloat.
Im gonna call the fact that prime will react to this, will mention htmx a couple times before the halfway mark, and when the video mentions htmx, prime will say "hmtx mentioned, pre watch"
FINALLY someone says what i have been thinking and arguing and practicing for so many years. Most of the time, your fancy library makes your project MORE complex, not LESS. There are a few things which are tricky and time consuming to write, and they are well-defined enough that you can just drop a library in and write a one-liner against it, but most of the time, doing things the so called "hard way" is actually SIMPLER. Nowadays, people don't use dependencies to get things done; instead the dependencies define how your project works! I feel like I am the last sane person in the assylum, or maybe I am the only insane person in the church. I don't know.
You are completely right when it comes to projects where you and you alone decide the scope. But a lot of software engineering (if not most) is not like that, you have to deal with varying budgets, priorities and complexity. People who recommend more complex tools that take more work to get to a solid foundation do this with this in mind. I can recall multiple times where the first version of an application I made with just used the bare minimum to get things done worked awesome, but eventually things change and you will have to add a features that you did not account for.
@@turolretar Oh yeah... I didn't realize it until now. We need those libraries to keep our billable hours up! We can get paid by the hour to make the libraries, and then get paid again for the extra time it takes to USE those libraries! OK. I get it now. Thank you.
I can see your general point, but for the sake of hosting a (static) website, why would you write any logic at all? Static site generators like hugo or astro do pretty much exactly what is needed. Add a template like blowfish, and you can have a very professional looking website with decent customizability, great SEO and responsive design in less an hour. Of course, without writing the actual content, which doesn't change in speed significantly either way. I think while your setup is cool, the easiest way to go from markdown files to a website are static site generators + your webserver of choice (I like Caddy because of sensible defaults).
I think it will be tbh. All the "hot new trends" tend to be great for a few specific enterprise level applications. But our mistake is thinking that that's what makes is good for our millions of small projects. The main idea that I hope is not lost: Use the right tool for the job. It's not about minimalism. It's not about convenience. It's about picking a tool that fits the use case. Some saying about square holes and round pegs or something, idk.
this is a good idea for a mini project. The moment you need anything more complex this aproach falls apart. I mean there so many things that may go wrong that a preconfigured server like actix/axum handles under the hood. You believe you don't need it until 1 guy takes down your server in a way that you did not handle. I also don't accept that re-writing half of axum because you only need that half is a mistake you are going to pay when you want to refactor or add any feature in a month and you don't remember how and why you designed your codebase that way. Also never under estimate the knowledge you gained the first time you wrote this project. The second time you write something its easier even if you change a lot of stuff.
@@atiedebee1020 not really, just some advice. This some advice from someone who is soing backend professionally for a while. Of course he is right, for a personal project this is ok but after a while maintaining this will be a pain.
@@Dimkar3000 i guess it highly depends on what you're developing. I've personally developed a loathing towards dependencies because of how often they get thrown around as the answer to a lot of problems.
@@atiedebee1020 I agree, some people take it too far, where they would take a dependency for a 3 line function. The oposite can be the same problem. I guess I am saying, never be afraid to use a library for something essential, I am pretty sure the actix/axum teams have done a lot of work in order to make there runtimes really good. You don't have to re invent the wheel but its ok if you want to do it for education or for fun.
I feel like comparing NPM to Cargo is pretty crude, Rust is much newer, documentation is way better and dependencies are in general, smaller and better maintained. Many large libraries are split into smaller crates (parallel compilation) so you *can* pick and choose what you use. Despite my short experience I also have to agree with the more complex projects, it's much easier to discover an existing solution in a full-package than to add your own patch every time you need something new.
can you articulate why you couldn't just use nginx + a static site generator? I've been hearing a lot about hugo lately also for static pages, remember that nginx and apache support simple macros that can be used for headers and footers, probably more as well if you look into it. Of course, if you're more comfortable with this then that's enough of a reason in itself.
I don't want to put myself into a box. I know I have ideas for things like comments in the future and know I will need tooling such as this to do it. I actually run an Nginx server, but it acts as a proxy server and routes it to my various hosts, which are all active servers. So it's also just how my servers are setup to work and I find it really easy to work with :)
@DestinyHailstorm It's almost like people have forgotten about server-side includes or what CGI was designed for. Running nginx to proxy to a static fileserver is just as bad as all that express and Angular to do basic webserver tasks. nginx could do everything you're doing with this Rust app with maybe 1 plugin... and you're already running nginx.
@@DestinyHailstormyou probably don't I have comments on a static website using Giscus. You could also make your main website with HTML and embed a tiny amount of Javascript/HTMX that communicates with a server that only deals with comments. That way it is a lot simpler, because you don't have to reinvent a static site generator
Reminds me of the time a consultant wrote something where 80% of the compiled program size was the pulled in dependencies for Log4J, when the thing did only log messages in about 2-3 places in total, where the destination of that log then had to be configured in an *additional* XML file for Log4J to work during deployment... So I deleted that lines, replaced it with a STDOUT/STDERR message dependent on the result (since all processing of that was already handled by the job-scheduler script anyway) , and removed the Log4J dependency. (Funny enough, that was about ~8 month before the Log4J security issue came up in 2022...)
Agree with the video. I picked Golang for that same reason and for the same kind of project. I wrote my custom static site generator using only the standard library (it even has standard templating language and the default HTTP router is great) with just 1 dependency being SQLite db driver. I also managed to do it entirely in two hours during a train ride. Since then I added go templ as a dependency for JSX-like components, and I'm building local editor server mode to let me edit my site content in the browser with htmx (it isn't linked in the built static version)
You should tell my browser to cache static assets from your site. Reloading the front page fetches like a megabyte of images from your server every. single. time.
I once wrote a pretty damn good M3U8 parser tailored for my specific use case in my open source project. After a few months, a user found a very obscure bug in a WIP feature of the parser that wasn't even exposed yet, but what really got me was how they lambasted me in the issue thread for *daring* to make my own parser instead of using one of the several premade ones on NuGet that haven't been updated for over 7 years and store everything in a Dictionary instead of object properties. I guess my point is that 3rd party dependencies aren't the gospel and its okay, if not more beneficial, to do stuff yourself sometimes.
@@rGunti Indeed. Writing the M3U8 parser taught me a lot about non-backtracking parsing, and I proceeded to use that knowledge in another project just 2 months later.
I've been a software developer for over two decades now, and a hobby programmer for over a decade before that, and I've never understood how people can seriously think it's a good idea to add a dependency to a library that itself has dependencies on 8 other libraries, some of which have dependencies of their own, just to use a single function of that library, which is 80 lines of code and could have been written by any moderately skilled programmer in less than an hour. There are so many things that could break here in the future, aside from the fact that installing your application on a system causes the system to install 20+ other packages just to satisfy dependencies that your code does not even use, they are just dependencies of dependencies of a small piece of code that your code does use, basically bloating up the system for no benefit at all.
The main reason why it works in rust is that it has ecosystem/mindset focuse on one library that can do it all, and if you want to drop some features you can do it via features flags. On the other hand in npm you don't have feature flags so you have to split it via packages aka deps to minimase size of the bundle
For example if you use more mature framework that has tree shake by default like Vue or Nuxt you can get almost all the features for exactly the same number of direct deps as in a rust project
Back in the days, adding a dependency was a huge matter that would need serious consideration. I have worked on several projects, where we have had to do major refactors, due to some insane dependency trees, for what, isolated, are minor things, but which, collectively, has been driving the design towards accommodation of these various apis, not towards what is the best long term. Then there is the security aspect of it all, which can also turn out to become a major liability, down the line. This is just one of many things, brought forward by modern web development, that is completely bonkers
when I first started coding 20 tears ago cross-site scripting was a tool for attack, now I come back and it's the recommended way of managing dependencies. The polyfil attack solidified my mistrust of cdns and I want to minimise deps in all my projects going forward. Why cdn google fonts in a project that uses 2 font-faces, one of which is installed on most systems for e.g.
@@Jason9637 My (limited and willing to be schooled) understanding is that that makes sure you have the same file that the cdn says it's served you, but doesn't guarentee the file itself isn't malicious. Let's say a major cdn gets infiltrated, and several major deps are compromised, is there currently any widely used safeguard against that or is it all based on trust?
@@paultapping9510 If they do get infiltrated, you'll have to trust that news of the attack comes out before you deploy. But signatures prevent cdns from changing the files without your approval
@@paultapping9510integrity ensures that the browser receives the version of the file *the webpage you're currently rendering* expects, not what the CDN says. It's the SHA sum of the resource embedded into the webpage EDIT: technically this doesn't guarantee that the file isn't malicious but it does guarantee you're getting the same resources as the site was developed for, so no direct supply chain attacks there
on one hand i can see how its an interesting way to teach yourself how to optimize things properly (or to love frameworks for not having to ever do that yourself) on the other most of the js frameworks are not shipping whole framework when building (and use tree-shaking for whatever is not used) and unless its a fun personal project it won't be worth the time spent implementing features that are already made for you
While i do agree with the idea of questioning if you really need a dependency: Serving static content or at least url parsing, should be handed off to a good library. There are so many vulnerabilities you can build with simple url parsing.
Rust is probably good if you want to write it once and forget, since it is less likely to have memory bugs, crashes, etc. I wouldn't use it if I wanted to add functionality to it the future, though.
It's easy to fall into this trap when you're using it to earn your bread and butter (like me). The mental load of *creating* the project is a lot lower. But all hell breaks loose once it comes to long term maintenance.
The reality of being a dev is that whatever you make will depend on something, if not a random node package, a cloud provider, if not that, something. I get wanting to slim your project down but beware, switching to rust, while raw doggin OS threads and ditching tokio/async: Is a great way to make your code extremely unmaintainable! Great for a small project though :)
The problem is this still happens even with feature flags. It helps the problem sure, but there are still very large and sprawling dependency chains on dependencies.
I totally feel this. I've just recently been trying to slim and trim my site, but also get something out the door quickly. I kept flip-flopping. I ended up staying with Astro (trying it out - cool tool but lots of extras), but I'll definitely be looking for my own super-slim, low-dependency, boring maintenance because it just works version.
I have honestly started to just think about building something that takes the principles I have in mind, and making it just 1 dependency with feature flags for all the different parts. I feel like there is a way to get that super slim, low dependency, boring maintenance kind of kit, but have it work pretty cleanly together in a uniform way. *Hmm*
@@diadetediotedio6918 I was more referencing the choice in his dependencies doing the one thing he needs rather than an overarching dependency with 20-40 sub dependencies in it where he won’t use 80% of the functionality.
@@jacoboneill3735 Oh, sure, this is true indeed. But it also means that you will end up with if you are building determinate types of software, which is something he goes against in the beginning of the video.
But pretty good video to show how easy you can create a website in Rust. I think, I didn't find all these dependencies when I wrote my website. I think I only used maud of these. Maybe I can improve my website with this.
Dependencies are great, people much better than me have implemented specific very useful algorithms and tools and I can just use them for what I need Having to code all the functionalities I need myself would take months or even years, and end up being not as good as these libraries Otherwise framework wise I'd say don't use Angular, it's bloated, slow and unintuitive. Go for React, or Vue for smaller projects, use Material UI and any regular website will be 100x faster to make. For your specific use case though, it's understandable you'd want something lighter, but lighter doesn't mean simpler to use
Dependencies are good in some cases, and I showed great examples of dependencies I picked to solve problems I had. You shouldn't be afraid to write code and get your hands dirty though. Of course be reasonable and know what you can do, but you'd be surprised how quickly you can move when you don't have to deal with your dependencies :)
@@ar_xiv I think part of this comes from the environment you learn to develop from. Personally, having started mainly in games, I feel this biases towards my ideas as they are almost requirements later on. No matter how shiny a dependency is, if it threatens the CPU too much it's getting the can.
Something like Angular or React is better off for large scale Enterprise level projects where many devs are working in the codebase. The cognitive costs can shared with a team, so once one problem is solved, its easy to share the aolution with other repos using the same framework. It's generally not recommended to use for personal projects unless you plan to learn the framework or grow the project, even then you want to reduce friction in development early on.
I think a better choice would have been using sveltekit, you'd have finished the project in 5 hours with 200 lines of code, with no JavaScript on the client, hot module reload, 5 seconds build time, and completely static build that you can host statically
@@DestinyHailstorm that still allows you to add non static content, you have the choice to make things dynamic or static, interactive or non interactive, with JavaScript or without, you have full freedom, if you have never seen sveltekit I highly suggest you to try it, it's incredibly lightweight too
`cargo-watch` and `just`. use cargo-watch to watch files and restart once file has saved and just for making callable scripts like package.json is a must in every project, not just Rust. Awesome video!
Unless I am missing something your data and templates seem to be static so you could SSG this. Then you only need to serve static files which you can do with a CDN or some static file server like nginx. The only dependency would be whatever you need to generate the static files + your provider or static file server of choice. No need for tiny_http even
That could be an interesting video idea. I think I would want to stream line it though, as there are some rough edges that should really be handled, and some oddly large dependencies in the mix. Perhaps I'll make a crate that wraps together a nice set of small dependencies and have some extra functionality to tie the loose ends.
I do the same for my website except I don't bother with actual programming and I use pandoc which does the markdown with highlighting and latex formulas. My entire "backend" is a few html templates and 20 lines of makefile to rebuild the pages. Pandoc itself has a lot of dependencies but the server doesn't need to rebuild so only a basic http server is required. In my opinion rust is overkill for something that will run once a month for a second.
I plan on expanding this idea as my videos go on. This site will end up being dynamic, and will need some actual infrastructure. Baking some pages and putting it behind something else to host it isn't the end idea, I want stuff to be more dynamic and show some cool ideas :)
That is one thing that made me leave PHP. For you to work on anything slightly more than minimal, like in the javascript world, you're recommended dozens over dozens of things you don't need to. You don't need a huge framework + a templating language on the language that can be used as templating to do what you want. But if you are learning, and then you run into a problem, people will ask you why you're not using them instead of helping you.
I don't know why so many people basically say fix your problem by replacing it with (Insert Thing), its honestly pretty frustrating as a developer and makes it just not fun to work on things anymore.
The variant where many dependencies were used was written in JS. And the variant with a low number of dependencies uses Rust. Sounds like the title should rather be "Why I Started Using Rust Instead Of JS (And Why You Should Too!)". I'm using Rust for years now and I never had issues with using too many, unneccessary or overly complex libraries in Rust.
In my short time working with Rust, I can tell you already I have run into it. It made my compile times worse, and it sucked to work with. While it isn't as big of an issue in the Rust ecosystem, it still applies to it. I would argue that even if Rust is better than JS in this respect, it still is worse than many other ecosystems, and as developers we should be looking to change our habits if we want sustainable software.
Broadly speaking it's easy to understand why having less dependencies is better than having thousands of them. This is all very understandable. But assuming you can only produce X lines of code in day, almost anyone can also see why those lines would be better spent not reinventing the wheel. As long as dependencies allow me to ship features faster, I will always prefer adding a dependency over doing a bunch of redundant completely wasteful work.
It'd be cool to see if it'd be possible to add an RSS feed to the site without another dependency, but frankly if there's something out there that does the job well and doesn't depend on 30 other things I think one more wouldn't hurt, but now I'm starting to sound like an addict.
I did a bit of thinking, it might be possible to add an RSS feed without another dependency. You'd just be generating it by smashing strings together though, so its not exactly guaranteed to generate valid data or anything. It would definitely work though. I'll think about it some more and maybe add support to my site just for fun.
It is, I just thought it was funny to pick on one of them XD Nuget has been the cause of many hours of needless debugging for me, granted same with NPM XD
Yes, this is one of the dependencies I use. It's called comraq, and I use syntect for syntax highlighting of code blocks. While it may be included in GoLang's standard library, this is not standard across the rest of the industry, and my focus here is more so on the principle, than exact tooling and libraries. I happen to enjoy Rust, and strive for safe and maintainable software, so it's why I chose Rust and the dependencies I did.
Man I still use static html for my personal web xd. Also it is funny how these new programmers think 95 dependencies is not too much. I mostly use c and I usually use what is already in system, or sometimes 1 dependency for networking, or some wrappers.
Why not Hugo or Astro to solve this "problem". I agree, JS dependency tree in a simple app is crazy. But a lot of Rust crates now-a-days have large dependency trees.
I didn't go into this in the detail I wanted to, but I mentioned a little bit about the amount of direct dependencies all of axum has (33+ IIRC), when combined I had 94 total dependencies that made it to build. Transitive dependencies are a real reason to not use a dependency!
Is it possible to share the code from the video? I just wanted to rebuild it to see how it all works. Unfortunately, there are still a few loose ends in my code that I can't quite figure out yet. For example, I can't find `templating::render_home()`.
I'm looking into making it publicly available. For now, please join my Discord and I can provide you some snippets and explanations, hopefully I can make this easier in the future.
I feel like you are asking a wrong question. "Should I add a dependency?" no, you should create a feature or an application that does what you need it to do. If you feel like there is a viable tool for a job, use it, if not write it. It's that simple. Nothing more, nothing less.
Yes, it does matter, especially because of the deep relation a lot of dependencies end up having. Sure, you might "shake off" some pieces, but that doesn't change you still have hundreds of transitive dependencies you need to keep track of and manage.
I recently did something similar with my blog. I rewrote it as a Web Assembly app in C# that is hosted on Github Pages. There are markdown files for each blog post and a JSON file that lists them all and their metadata. Any logic happens in the frontend with C#
I find it interesting that CS/SE classes the world over make a big deal about keeping code lean and minimizing the number of things that can go wrong with it, and yet this is such a rampant problem out in The Real World™. It reminds me of an old joke: "Noob programmers keep it simple stupid, intermediate programmers write general solutions that elegantly cover all possible use cases, and veteran programmers keep it simple stupid"
@@DestinyHailstorm To play devil's advocate, it feels cathartic when you try something fancy and it just works. To play the devil's devil's advocate, most of the time when you do that you've already wasted hours/days/weeks that you could have spent on other things lmao
I agree with the point but i feel like you chose a flawed way to illustrate it since the job could much better be done in a static site generator. Maybe I'm missing something though
This video doesn't fully tell you the scope and ideas of the project I have in mind. Yes, that would be a better solution, but I already know and have plans to go far beyond that. I hope as I put out more videos I can show this :)
Dude, im building my site using simple program i wrote in shell scripting language, and serve it as static files. I see you have static page as well. Why having a server if you can have something even simpler?
Talking of dependencies - Go would have been even lighter compared to Rust, using just the standard library. I know Go is boring and stupid easy compared to Rust, but it fits the bill for most web server stuff, leaving Rust to deal with systems programming.
I haven't tried out Go yet, I might need to give it a look in the near future. At the moment I've focused on using Rust since I have absolutely loved its compiler.
I do think the problem is not with dependency themselfs, dependencies are the bread and butter in business settings when things need being done yesterday for today and there's no time caring for bloat; the problem is with overbloated and overcomplicated frameworks that solve problems only big corpos have: like Angular. It's the exact same problem with things like kubernetes. it solves the core problem briliiantly and, if you think about it, clean way. But its' benefits are only visible if you have 20 people working with it; meaning that they're basically useless and only cause problems and waste of time for small developers.
Sorry you have too meny dependencies here things you should eliminate included the rust compiler. You'd be amazed how much easier & faster you can solve things by abandoning the idea you need to write software at all. An Apache server & server side templates can solve a wide variety of problems. It's these front end boozos retinventing the wheel or worse the time sharing operating system in because we have to do the backend & frontend in either python if they are data-scientists or Javascript if they are web devs. Reject Modernity Return to monke
