I think the take-home lesson from this is that LLMs produce bad code and, while you might get something that works well enough if that's all you need, if you're actually writing a benchmark that's supposed to measure optimal performance, getting an LLM to generate the code is pretty stupid...
@ nah! I didn’t request anything. All I do just want to express my thought on it. I don’t give a damm thing if he likes vue or not, just don’t like the way he tryna ignore Vue in the game even he knows Vue also was also falsely benchmarked initially but never talk, and that’s it. And it’s not just me, let’s look at other comments. Thanks
@@jatinkumar7287 read the room and the title. 😄 he's a react developer and has dabbled in a few other things. Vue not being on of those things isn't a travesty. i get wanting Vue to get more of a shine, but you're looking in the wrong places.
@@BurgerBurglar8964Theo defends React with all excuses possible "oh, but look, it's'just' half of the performance" meanwhile the only comment he does about Vue is "Vue looks good". I don't dislike him for that, but I found this funny. Maybe he's too deep in the rabbit hole.
Any framework that cooks its benchmarks like this is an immediate, giant, blaring red siren for me. Sketchy as fuck. You can't blame AI for that either. Any dev worth their salt reviewing those benchmarks is going to know the difference instantly. It's intentional.
why tf you are always ignoring anything related to Vue? vue creator was tweeting about it nuxt core team was involved there was a PR in the benchmark repo what else needs to happen to simply cover it the same way as you cover others?
@@naughtiousmaximus7853 I mean, we just sadly got to accept that Theo isn't some kind of tech reporter with some oath to treat each and every technology equal. If they don't have any interest in Vue, then why would they talk about it? Besides, Vue is great, don't get me wrong, but it isn't going to get any clicks since it isn't trendy anymore so the value here is little. What I can see here is just the fact that he has been more personally involved with the people behind these frameworks and that's why he likes to cover them, which is probably not the same for Vue. Nothing to do with Vercel or whatever
@@javierflores09 It's not about making content about it. It's about not deliberately ignoring it, if it's somehow related to the things Theo wants to make content about, like in this case, he entirely ignored that the initial score was wrong for Vue too.
@@bartek.igielski "deliberately ignore" is a rather harsh assumption to make. It could just be that he didn't see the initial rebuttal tweet from the Vue people like he had seen the others, that isn't ignoring but just not being aware of it. Sure, you could argue that Theo could've gone "if React and Solid guys said this then Vue's one must have been flawed too" and go on a crusade to find the flaws on the Vue benchmark, however as I said earlier, it isn't like he is a tech reporter so he has no obligation to do that, and I certainly wouldn't if I took no interest in the framework. Vue has rebutted their place in the benchmark as it should have and now the misconception is gone for the people who have seen the corrected benchmark. I am aware it can be frustrating that an influencer you follow doesn't do coverage of the framework but truth to be told, it isn't necessary anyway, with the amount of people that have pointed their fingers towards that benchmark, it wouldn't cause a dent to Vue's reputation in any sort of way as people are already dubious of all the results
This is such a goofball thing to measure. I love it. I think what I really learned is that LLMs can and will lead you down the wrong path and that it's really easy to deliver poorly optimized code in Node.
The react base component is generating a list of react nodes without adding a key to each element forcing an entire tree swap for the component in each re-render no?
What's really quick, is the speed at which a dev gets himself egg on the face by posting a half-assed test. You guys are really tolerant ; it's nice to see, because the damage could've been real. I'm sure Matteo learned his lesson ; benchmarking is more subtle than it seems.
To be fair I feel like we overuse JavaScript today to essentially spit out pretty basic html. There are of course situations where it makes sense but most of the time we should not be generating static html. It should just be served as is already. I'm not sure when writing basic html and css became such a naughty dirty thing.
Anyone else still suspicious of the results even after they were "fixed"? If all the first examples were AI generated and the person benchmarking them isn't actually equipped to write basic examples within the various frameworks why should we trust their framework is actually fastest, there can be plenty of other minor things destroying the performance of the others. Not just the most obvious dev mode vs prod mode, furthermore in the fixed svelte 5 example it was faster, but in the final benchmark fastify is once again fastest so what changes were made to the fixed svelte 5 example to make it slower or what improvements did they make to fastify to get the additional performance gains and if so why weren't those gains in the original example coming from the framework author themselves? This doesn't seem trustable at all, and extremely cherrypicked to make their framework seem better than the others when in reality it doesn't really seem to deliver in that regard.
that's benchmarking in a nutshell. i enjoy benchmarking but it should always be taken with a grain of salt. when there is a significant difference in performance, it will usually mean something, but this benchmark is not exhaustive enough and is too minimal.
Why does everyone say Theo ignores Vue? I guess they are just babbling without watching the entire video clip. Shame on them. 11:26 He said "Vue aws surprisingly fast..." 14:54 "But it's also crazy that Vue with similar capabilities is able to be so close to Fastify".
On garbage-collected languages, there are two major performance issues with arrays: 1. Every time you create a new array, it allocates on the heap, which is costly. 2. Every time ur array is elegible to be collected, it freezes your process, throttling it. Solution: Use array pools to reuse arrays, keeping them from being garbage collected and avoiding new heap allocations.
The same happens with Deno, the creators of Deno are obsessed with performance but seeing that even with their version 2.0 Bun continues to surpass them they prefer not to include it in their comparisons on their website. "Let's pretend that they don't exist"
@SilasDuarte-e9k that's how all benchmarks are because it's good for marketing. there is a faster runtime than bun called just-js. web frameworks do the same thing. they show comparisons across different languages and intentionally leave out faster languages or faster frameworks in other languages.
I think the way they handled being wrong is great. One thing though is that what the produced at first showcases the potential for letting AI write code.
Realistically when was the last time the framework performance actually remotely had an impact on your app? This never comes up in business apps in 99%+ of cases and the only time I've ever encountered it is screwing around for fun, messing around with 3D + webGL, etc. Unless a framework is perceptibly slow to users it doesn't really matter and it won't cost you any users or customers, but it will cost you money if you get sucked into premature optimization.
It's not that SSR performance doesn't matter. But you can scale always scale when your problem is server side. When your problem is client side your only solution is to optimize the code. We better see some real life examples thast would include all loading time to compare better.
It's the same story every single time and it's tiring. Each time a benchmark is posted it's almost always just disingenuous advertisement for a product/framework. You almost immediately have people in the replies pointing out flaws. I see this with runtimes (Deno vs Node vs Bun), I see this with frameworks, I see this with libraries. Literally none of this matters because it's possible to write crappy/slow software in any framework.
again, you could definitely get a quicker response from react if you used a stream? waiting for the entire request to finish would surely negate the rum stats associated with this being bad?
Theo what's your criteria for taking channel sponsors? Just wondering as I'm not really able to differentiate between if you're sponsoring for the sake of sponsoring or if also because it's a product you think would be valuable to you or your demographic.
Matteo fixed his tests and reposted a thoughtful article about everything. If you're shading him for this, you know nothing. He's not a famous framework author like RH, RC, DA, but in the wider NodeJS world, he's absolutely legendary. It's honestly unreal how much code he touches. Anyway, interesting video, glad Svelte had some redemption!
