Am i correct in presuming the only reason you're not gonna be billed for this is because you're sponsored? So if it was me i'd be getting billed for a DDOS attack? that's enough of a yikes for me tbh lol i'd rather it crash than it staying up and them charging me tenfold lol
Who has that much time to waste on a DDOS attack that gains them absolutely no benefits? It's clear that your tech stack handled the attack quite well, it didn't even cost you much at all. If you get enough views you'll probably even get money out of this video that they helped create. Someone really hated you, lol.
Either they bought botnet time and it costed them money directly; or they used their own botnet (instead of renting it to other people during this time), and that's money they could have earned but didn't. So the result is the same : it cost them money. Probably a decent amount. Which I find hilarious considering the ridiculous impact they had.
@@sardines7436 and good publicity for Vervel too, seeing how easy it was to handle the problem with them. Conspiracy theory plot twist : it's actually Vercel themselves who conducted the attack so that Theo make this video to further PR their services to his followers, for free.
We had a DDoS attack about a year ago where it was about ~10TB/minute and we are hosted behind cloudflare, so just couple clicks inside cloudflare panel (there is a button "we are under attack") and this attack is gone, next minute I checked where is it comes from and every single IP of attack came from outside of my country (No one wants to ddos from same country or your business since police could investigate it and attacker could end up in jail), so I did just enabled captcha for any request form outside of my country (since our business doesnt have international customers) and disabled "we are under attack" and never had an issues since then while they still trying (one year later). So may be something like cloudflare could help you.
You should make a dedicated video about DDoS-protection on the T3 stack, as clearly there is a possibility of creating unreasonable cost for the service provider and not everyone will have their bill refunded. You mention as a side note that you could put rate limiting on each route for a personal CloudFlare, maybe you expand on that and/or provide a package as framework for that.
RU-vid alg really loved this video, it showed it to me in the recommendations the minute it was published. That was with me not watching many of the previous videos.
This happened to me too! It might not have been targeted towards you because it happened to a test deployment of mine which didn't even have any real traffic. It was also on Vercel, and I get blocked pretty quickly (although support helped me get unblocked) Edit: My attack caused 462 GB-Hrs within like 20 minutes
The key is that you need to rate limit these attacks whether you use server or serverless. So this demonstrates serverless has tools to handle it. Ofc if you had a server and rate limited it could handle it too. Re one of your last statements, it doesn’t mean servers would’ve been worse. It means not rate limiting would’ve been bad, server or serverless
It is actually really cheap to buy residential proxies (pools with millions of IP addresses) and then use them to bombard requests to services. These residential proxies exist to enable scraping of SERP content as well as regular sites with hardened DDOS protections. Some residential proxy services also bypass recaptchas (using AI and sometimes even humans) for premium. Residential proxies have legitimate use cases but can be misused to create botnets too. That is what I am suspecting is happening here. They haven't actually paid for those 600 IPs. Rather, they are tapping into a pool of million IPs provided by residential proxy services.
Considering that one 10gbit server can (in theory) handle 1.5tb raw traffic (in 20min) I don't believe that this was a big DDOS attack. Also I believe that having multiple cheap vps with high bandwith automatically deployed when needed, would be probably way cheaper than vercel. Obv. the developer experience will be worse, especially when setting all the servers up or other cluster related issues occure.
Yes this was just 15 yr old some kid with a very small botnet. A pro would have used 10k IPs and Vercel would have to shutdown their dns for a period of time
This was not even that big of an attack. The traffic is literally less than 1 GB per second. If anything this was a skid attack which is further supported by them literally just loading one JS file over and over. This wasn't a DDOS attack, this was some kid trying out their $5 booter.
I mean, having a punchable face and arrogant personality is bound to provoke someone when exposing yourself to thousands of strangers. Even so, it takes some extra thick emotional issues to waste any amount of time and resources to get revenge on a parasocial relationship.
Did Vercel give more details? Such as if the IP addresses were all from the same IP block or dispersed across many, whether or not they were residential IPs, their own IPs, IPs from other cloud providers, etc, geolocation lookups of the IPs? All of this seems like it would be super useful to know about to prevent future attacks both for you and them
at what point does vercel consider the requests as a ddos attack do they use any tools? what happens if a tiny dev's app gets ddossed, would vercel refund 100% of the money by all requests that day? how long do they take to answer from the point where you're under attack to when the situation gets resolved?
after 1 minute according to their webpage. not great really given the number of requests you could be on the hook for at that point. and no, if they don't say they'll refund your money, assume they won't.
Love that the stack you are recommending is the one that you use for your stuff. I can imagine that the people behind this were just absurdly annoyed that you are recommending tools that don't fit their certificates or what they consider is the "right move".
Honestly Theo, now I am very relaxed about the decision of using T3 Stack and the services you recommend us. If even Chirp handled this insanity! Then we’re in safe hands as Solo-preneurs 😊
It's laughable how much the attacker likely spent vs what you incurred. Perfect example of mitigation, you can't stop it from happening, you have to make it too expensive for bad actors to continue.
@@name_less227 They do. It might not "cost" them anything in the literal sense. They didn't spend money most probably, if they own the botnet. BUT... usually when you own such a botnet, you can sell it, or rather rent it to people who want to conduct such attacks. So all in all, either their bought botnet time and is cost them money directly; or they used their own botnet instead of renting it, and that's money they could have earned but didn't. So the result is the same : attacking cost them money. Which I find hilarious considering the ridiculous impact they had.
I have a question. Are those people generating these attacks going to be held accountable or there are ways to generate botnets attack and get away with it
DDOS is my main concern with Serverless. With an nginx proxy you can get sub 1ms 503 responses in a DDOS and cap the number of requests per IP so that it doesn't touch your actual app code when it happens. So for a free or cheap server vs a Serverless platform, at least getting started out, I know someone can't rack up costs for me.
I can't imagine that someone decided to waste any significant ammount of money doing this. I'm wondering how they had access to 600 static ip addresses.
Absolutely hilarious that some people will spend thousands out of pure spite for absolutely nothing. Still, I can't help but be skeptical of these new cloud providers you're showcasing. When the 'growth' period ends and the 'taking profits' period begins, is it still going to be more attractive than AWS? We'll see!
How would you be dead if you had actual servers running this? Wouldnt it be better because then you wont get charged a lot of money? Thanks for helping a newb like me understand
"impossible to take down our services" my guy, you are challenging the wrong community here 😂, and for those that say it cost them money, ego doesn't care about money. We do things some time just to prove that we can, no need to hate you to do something like this (I'm not saying I'm even capable of this), but if i could, i wouldn't do it because hate, but ego or passion, and seems you already have enough of both
Does anyone have any examples of using upstash's rate-limiter with tRPC? Been using it more, and I haven't really gotten around the concept of how rate-limiting could be added to it. It most likely would be done via a middleware, but just putting up the flag for any existing repos that have it.
Considering how much this cost the attackers and how little it affected you, it had to be someone with disposable income. Seems like Elon wants to get his revenge after you told him how ads work.
Seems someone's back-end needs Rust Framework 😊 Edit: idk why my reply multiple times got deleted. so i am sorry, i cant explain due to no freedom of speech
If you weren't on the pro license and sponsored by vercel this might have been a different story. I can imagine a normal person would have to suck up the big fees or take down their application
It depends. Cloud providers more commonly than you might think cover ridiculous fees in case of an error or attack like this. Because they don't want to lose your business.
If you are a normal user on the free tier of vercel you would just put your application behind Cloudflare for free and let them handle the DDoS traffic.
huh, so vercel has no rate limitting by default? I would have expected a managed service to handle this, not have me set up my own edge middle ware (upstash?) It doesn't look good on their part
Is it me, but why are they are targeting static assets? I mean if you want to increase Theo's bill, DDoS the api route which has the upstash rate limiter as well? It will cost him 0.20 cents - 0.40 cent per 100k request for upstash and probably far more for serverless/edge functions on vercel. Thus sending 100 milion request will at least cost 1000 * 0.20 + 500 GB hours ( 5*40) = 400 dollar + rest of vercel
Tbh this attack probably came from a 15 yr old… this is not a serious attack tbh it’s very easy to rotate 10k residential proxies and force vercel to temporarily shutdown all ping services I could probably do it
You got lucky because you are publicly sponsored ... but this brings up a BIGGER point. When you pay for metered services, the providers NEED to indemnify you against DDOS attacks or other potentially ruinous events. This could be a huge selling point, because not all services have built in caps. As a matter of experience, I witnessed a peer who used AWS, his application while still in beta had a memory leak and AWS sent him a $13k bill. Insane! Risk avoidance is important to any business especially if there is no ceiling or price cap. As someone who uses these services this keeps me up at night.
big question here though... how much was the upstash bill? 😂 Really curious since we suffered a DDOS attack ourselves and are looking into some options
