Every time I try serverless it doesn't make any sense to me and just feels kinda of meh. links from video: mikhail.io/ser... mikhail.io/ser... mikhail.io/ser... #benawad #serverless ---- Follow me online: voidpet.com/be... #benawad
For image resize you can cache back to S3, most of your images are re-requested a lot in the same formats like list view thumbnails and so on (unless you have an extraordinary use case). After the image conversion, save the file to S3 with the format __.png or something and make your lambda look there first. If it hits, it's superfast and cheap. Thanks for the video, very funny and informative.
You could even configure CloudFront to check the S3 bucket first, and then ONLY fire the lambda on a cache miss. More performant and also lower cost. It would probably also make sene to add a retention policy that deletes generated images after a period of time.
The problem people have with new technology is , they want to use new technology like the old technology, that is just not how it works. You also might need to revisit the definition of serverless .
Agree, there are several uses cases where either you keep your function warm due high loads or you dont care about 1-2 second diferences. Als serverless is not only about scaling, but also about not having to manage and support a server
Yeah the benefits of serverless cover a whole range: * Speed of iteration * less a priori Setup * managed updates because one still needs to update Kubernetes, patch Docker images and Host OSes (also entails security implications) * maintainability by writing less code, because that script shown in the video would never work anywhere other than in Serverless because you’d need a http server like express, logging, alerting, error handeling, request validation etc
@@marcelofm80 Exactly. Serverless computing has been a godsend to me as someone who uses the cloud almost exclusively for LOB/infrastructure - I get to have isolated processes running 3D processing/rendering/filing/scheduling and all kinds of business housekeeping without having to fuck around with Laravel and create a whole consumer app/API project for a business domain that requires two services as I used to. Containers make it even better since I can reliably run legit 3D software in the cloud without having to depend on webserver shell calls, it sounds like magic sometimes.
Serverless doesn’t exist, and contrary to belief, it will have limited resources. There’s no magic here, it’s still maintained, still crashes, still needs updating. It just means your internal IT staff don’t have to do it. This concept has been around since the 90’s.
You might be expecting AWS Lambda too be a replacement for a static server but it's not. It's more for handlying async events that you don't needs results for immediately. Sure, you can bend it to pretend it's a replacement for an EC2 instance on a pay as tou go contract but that's not its purpose.
Also for something like that is hit rarely. Lets say u have an api that generates a CPU intensive report but is called rarely or less often compared to ur usual login calls. In that scenario, I would prefer setting up a lambda with giant RAM/CPU instead of setting up an x2.large on aws. People now days are abusing serverless thinking its cool. Sometimes, old school is always the way.
@@gsb22 Yep. One of the things I tell my development teams who practice resume driven development is they must tell me how moving to a recent technology solves their problems and if it's worth the cost of throwing away a decade of the Ops guys VM experience. There are some things that Lambda has done really well but it's all event based, totally async, or just makes more sense from a pricing model. Everything else is still EC2.
@@WunderWafer It does work for backends. Just not traditional ones. I use Lambda w/ S3 and DynamoDB for a lot of processes that aren't utilized enough to justify hourly EC2 costs or can afford the latency of startup. If we're talking an old school jar file requiring MySQL and NFS then I'm going to use EC2.
@@optymystyc Never had an issue with API Gateway either. I also use serverless framework. Hell, I added a lambda authorizer in no time flat and deployed it. Besides, he keeps talking about cold starts, but if your app is actually being used, its not really a big deal.
I'm using Serverless in production. Sure there are cons, but it is never as bad as some people describe. It has it's own purpose. It is not a replacement but an alternative.
@@pjcollazo8318 need APIs to post data to a SQL DB frequently. And, access the data infrequently either through a Dashboard UI or through exported CSVs. We had to deliver the product in a week, so it made perfect sense for us. It is very cost effective, and no headaches to maintain the infra.
@@RandomGuy-co7rq If you’re posting requests to a db that often then it would have made more sense to do the opposite and have an in-house database server.
@@slapmyfunkybass Lambdas don't store information. They process a task and return a result. They do not maintain any type of state. Storage of information is a separate topic unrelated to lambda. Lambda can communicate with any storage you desire. Its the people speaking about lambdas in relationship to a database server that everyone will listen to though. Lambda has nothing to to do with databases or storage of state. Lambdas are throw away functions much like an ec2 server *should be. Something that be tossed in the trash and regenerated. Something that easily be distributed, balancing out traffic but thrown in the garbage after and respun up moments later without affecting performance.
@@trustmebro703 depends on the use case, some never hit that kind of scale, you really have to be going 24/7 at a constant rate to make serverless cost-ineffective. plus there are many situations where having a cheap entry cost is worth the long-term trade-off of migrating your stack if you get big.
@@trustmebro703 That is true for literally everything though? Server computing is also cheap until you scale, cloud storage is cheaper until you scale, going from outsourced to in-house is just part of the natural evolution of a business. The fact that once you scale high enough it makes more sense to have your own infrastructure is not a flaw, it's the point. It shows that without cloud computing you might never have scaled at all because you wouldn't have been able to afford on-premise when that was the only option.
@@anthonyskrzypczak9437 Most will never hit that kind of scale, and the fact that you can become extremely profitable way before having to worry about the scale economies of owning your cloud infrastructure is nothing short of miraculous.
I love how with every video I watch I have 0 idea what I'm clicking into but Ben's meme language just clicks and I come out having learned something in a non-mind-numbing way
Despite all the double-speak he's a lot more straightforward than most tech personalities that say a bounch of empty words either for clout or to sell you something.
Two points you should consider 1) Provisioned concurrency makes perfect sense even in serverless architectures, because if you have a large complicated application decomposed into 10's if not 100's of lambdas then you can set the provisioning for each in isolation, this saves $$$ over having to scale a monolithic app. As for serverless vs Docker + Kubernetes there is very little difference as serverless is actually using these technologies under the hood, it's just abstracted away from the dev. If you go into the lambda settings you can see you can upload your own docker image to provide your own runtime even. 2) You don't even need to use provisioned concurrency to avoid cold starts in most cases. You can use a heartbeat method to pre warmup the function by hitting the endpoint and having it short-circuit and instantly return in anticipation before you are going to use it. You can even abstract that away using simple middleware to not convolute the function itself. That is not to say serverless doesn't have it's problems, though each one is something that can be solved once and not have to be thought about again. AWS all the way, as long as you do the leg work to get your infrastructure setup once, repeating the process actually becomes easier than using any 3rd party.
@@meamzcs That's true, AWS takes fiddling around with to get the entire infrastructure to run locally. I'm not sure about the vendor lock in as you can upload your own docker images to Lambda and your source is Version Controlled anyways. The only thing you are really buying into is the orchestration (Kubernetes part), it would be trivial to switch to Docker + Kubernetes for whatever reason.
@@DavidGraySailing Pretty easy if you only want to unit test, however becomes much more difficult if you want to locally run the entire stack. Also I believe the functions can only be invoked rather than deployed which means you can't run your application locally and trust me, your front dev's and testers will prefer to have a functioning stack locally than not in case of outages etc.
I think you answered your own question. Serverless scales 10x more easily and the only downside (cold starts) can be fully removed with provisioned concurrency.
Yes, but provisioned concurrency is much more expensive... that's true too... I think that like I said in another post, it's about the right tool for the right job. Containers can be a very powerful solution for some things, while lambdas excel at others.
@ Another good point serverless can save developer time as well (ultimately saving money). There is not a need to make developers maintain servers etc...
3 года назад
@@kylekim5861 if course that's one of the main advantages, along with scaling...
@@carlosjosejimenezbermudez9255 I've been reading a lot of comments on the cons of serverless and this is by far the only one that actually holds. Tbh it's not very hard to convert the code, but biggest con is really spending all the time to learn their infrastructure. There is a serverless framework out there, but unfortunately you have to pay money to maintain that.
Serverless is pretty good. It's basically microservices with more of a functional aspect. Great for data processing. Also like it more for distributed processes and pub/sub models.
I usually do the provisional thing, where i call my function every 15 minutes. Cause its only 96 calls per day which is 2880 calls per month of 2 million Free calls per month of gcp functions. That way my server is completely scalable and warm all the time.
Wait, this seems interesting, do you have a setInterval API call set up in the front-end? But even that means you need to keep the website open and connected to internet all the time...
My experience with Serverless is actually extremely positive and we use it extensively at my company. However it forces you to take a different approach to doing things otherwise you will find similar struggles as described above. To give a concrete example to the scenario mentioned in the video: I’d start with asking myself if you really need ALL possible resolutions for an image? What about w=2&h=10000 ? So what we found is that there exist a handful of reasonable defaults (e.g. 5-10). We would then do the resizing after the image was uploaded and put the resized images alongside the original. Then hook up CloudFront with Lambda@Edge to do the dynamic routing to the resized images in S3 or CloudStorage. This way you got a solution that is extremely cheap and incredibly fast (even faster than servers)
@@gogoloco4769 You would usually request content metadata in your Frontend which includes available Image sizes with keys like "thumbnail", "preview" and "original" and then you request the image size that you need. For gallery type apps you would usually have a fixed width which serves as your key.
Serverless is not just about scaling from zero to infinity, it's more about scaling between a to b, and a is not necessarily 0. If you think about that, serverless is pretty handy in case your b is too high and spikes at decent intervals. Apart from that, serverless also helps you create microservices without having to bother much about server maintenance.
Why does "production" automatically mean you have consistent traffic from the start? You won't. *Plenty* of requests will be hitting a cold function endpoint. Over and over, new users will be waiting on that cold start and bail your service, or have a perception of "slow". Additionally, if you have "a lot" of traffic, every request above the threshold that 1 function can handle will spin up a new function instance. There is no avoiding this problem, even when your "production" service is succeeding, you'll be hitting cold starts left and right. Serverless functions do NOT make sense for general HTTP request/response endpoints. Mostly they make sense for async event handling/background work, like webhooks or Firestore events etc.
if you don't have the skills to manage and maintain servers, sure cold starts won't be a big deal for you, but for your customers/clients? I don't think so. that half a second cold start will pretty much impact your throughput consistency. that's why ALB exists specifically to counter this issue, to not having to deal with scaling issues and cold starts.
You'll fail miserably trying to use serverless to build small webapp/API with constant load. It thrives when stream processing, batch processing, pub/sub and asynchronous treatments in general, when you need to be able to scale by a few orders of magnitude in the blink of an eye and caching is not possible. Yes those type of apps exist, but it's nowhere near the web frontend world.
I think you kind of missed the point. Severless isn't designed for applications where you're looking to optimize perf. It's for stuff where you don't want to deal with a ton of configuration, security, maintenance, etc. It "just works", and there are plenty of use cases, like some internal service for your company, and app's background telemetry, etc. where a 2 second call often isn't a big deal. This is the appeal IMO scalability is just a side benefit and if that's your biggest concern, there are better options for you
> "scalability is just a side benefit and if that's your biggest concern, there are better options for you" You mean just reaching for K8? Or other "better options"? I think another use-case is building small side-projects/prototypes and don't want/have the time to set up the infra for even a managed k8 cluster (after all, managed != no management/config at all). When all you need is a PoC or something nifty that you've worked on to show potential employers, it doesn't have to be fast and saves a lot of money since 99+% of the time it will just be sitting around, scaled down to 0, waiting for someone to demo it. Whereas with something like K8, it'd be wasting away a lot of server compute time (after all, K8 needs to have at least the control plane powered at all times).
@@mateowang6570 thats another good point. Often times serverless isnt even used in production, but even when its not that doesnt mean it cant be useful. For prototyping it can speed things up noticeably!
@@ThatsPety Yea, for sure. As for better options, did you just mean K8? Curious about any other ways you would approach scalability and high availability.
@@ThatsPety not used in production? You're kidding right? A lot of large companies implement serverless processes somewhere in their system. Check out this is my architecture by AWS in youtube
@@mateowang6570 oh btw yeah K8s was the main thing on my mind, and sometimes these providers often have a sort of "app" service that kind of sits in the middle. It does all the scaling for you like serverless, but you upload an entire build like a more traditional system
While I've never been a fan of serverless architecture , I feel like your argument against serverless here is basically nothing more than "There's 900 ms of initialization SOMETIMES!" ... Which if your use case for the function is just resizing an image before sending ... what's the practical limitation here? I could see the argument for this IF it was Sequential (If you have 20,000 requests simultaneously and each request has a 900 ms delay before the next request can execute, That's a practical problem), But the point of an Lambda function is that it's parallel , So this fundamentally won't be an issue in the first place. If you have 20,000 requests, And they're handled in parallel all within 1s, I don't think you have an 'actual' problem there. You have a "This is taking 0.8 Seconds when it SHOULD be taking 0.5 Seconds" problem because ... Computer scientists are strangely obsessed with shaving all the ms they can from anything they do ever, regardless of whether the impact can be seen with the naked eye. Granted, There are real-time applications where 'off by 500ms' can make a practical, real world, catastrophic difference, For example Vehicle/Aircraft operation systems (Self driving cars comes to mind), Medical Equipment (Think remotely operated surgical robot or anything exposing radiation or medication), Scientific Research Equipment / Programs etc. Injecting a patient with drugs for an extra 500ms could kill someone . Stepping on the brakes 500ms too late could kill someone . Taking 900 ms to resize an image for the webpage to save 3Mb of network traffic isn't going to kill anyone.
Absolutely agree. He didn't explain why the 900ms for the first request is really a problem. There is nothing mission critical that will fail because of those 900ms🤦🏾♂️🤦🏾♂️
There are several fundamental flaws with this statement. Unsure of your experience in development, so apologies if it sounds patronizing. Shaving ms is not a strange obsession. It's a legitimate requirement. This depends on the stage of the project of course. For example, if you're building a MVP or prototyping, then it doesn't matter. But in production, you have to consider UX, costs and other restraints or requirements. Obviously, there is a point when the cost-benefit is not worth it and you're just trying to squeeze blood from a stone. Determining acceptable performance by measuring risk to life (i.e. if it isn't going to kill anyone then it's okay) is a, let's say, strange metric to be using. The key metric is usability, e.g. what is comfortable to the user and this depends on use case. For example, a user may think 900ms may be okay for resizing an image, but totally unacceptable for populating a dropdown box. Although it would be amusing to watch a product manager to react to someone saying "it's not like the extra delay is going to kill anyone."
@@Sasataf hmm... I still don't see anything you have explained that makes 900ms for the first request a problem. This isnt a constant 900ms for every request... Just the first one.if those additional 900ms are a show stopper then you definitely have bigger issues in your system. Most times this due to some bad UI/UX implementation where slow network requests are not accounted for and the UI hangs without any loading signal given to the end user. Besides all this, there are serverless options that cost a bit more but get rid of this warm up issue. FYI We have all built production systems.
It definitely isn’t a strange obsession... as much as a 200ms change sounds small, it is very noticeable when you’re actually interacting with it. I work for a financial company and every year all the financial companies compete for the JD Power awards. It is a consistent trend that when response times go up (even if it’s only a few hundred ms) your JD Power rankings will suffer and when your response times go down your JD Power rankings will improve. This is true even for categories that aren’t directly tied to performance. Even if you’re not consciously recognizing a latency difference you will subconsciously. Making your website feel responsive and “snappy” will improve the user experience far more than you’d expect and that is impossible to do without fast APIs. In this case it may not be an issue. You should resize once and save the different sizes for distribution at a later date so it’s a one-time time sink. However, for APIs that get called on a more regular basis adding a few hundred ms to the response time would be completely unacceptable
I had a terrible experience with server-less when we decided our solution to slow start is to be periodically requesting our lambdas to keep them "warm". Some of the most insane programming I've done
Hmm.. Still think this is the wrong solution. You may as well provision a full on instance and azure supports that while maintaining the serverless nature of it. More importantly, if you are not running anything mission critical where someone will die, then an extra 900ms isnt the problem. Its probably a front end issue... Displaying some form of loading message etc. Serverless offers way more benefits than just getting rid of servers... Once you appreciate those benefits,you can not go back to procuring full instances
Yeah..our entire architechture is on aws and serverless having more than 1k lambdas and other services..thanks for the suggestion but every best practice or solution like provising u mentioned we habe imolemented it all :)
I don't think it's a serverless fault for poorly designed architecture and miss-uses. Serverless makes a lot of sense and it's very useful tool. SLS works really well in an event driven architecture where you process data asynchronously. Besides, it is a good choice for running applications on a budget and if fault tolerance is important.
@@XDTuber There is none because the queue system is not a replacement for sls and vice verse. You still need a compute unit to process queued jobs. AWS lambda functions are often used together with sqs, sns, and kinesis (in the aws environment). It's more about ec2 vs lambda vs ecs vs eks etc.
@@karolszymanowski518 Spot on. Would you rather run a ec2 cluster with loading balancing duplicating a bunch of crap. Would you rather manage a kube cluster with a bunch of convoluted configuration. Would you rather write a function in a familiar language and not worry about load balancing, scaling, kube architecture. Just write a node function push that out and have it available at scale. I rather use serverless framework to write functions in any language not worry about managing complicated configuration or legacy infrastructure requiring manual load balancing patterns / techniques.
The cold start is definitely something that really bothers most people when they start working with serverless, however, there are simple ways to overcome such complications like a keep-alive function, in AWS you can use a Cloudwatch alarm to send a request to your endpoints to that they continue to be warm forever. One of the main reasons to use serverless is the fact that you don't have to worry about server maintenance (OS updates), load balancing, etc...
Serverless is a new paradigm and you seem to be examining it from the perspective of a different paradigm. You sound like a monolith developer arguing about how bad microservices are in 2012.
Microservices suck tho. Just like OOP, it was a funny experiment, but it's time to put an end to it. I'll never get tired of saying how bad those two things are.
@@mjphillips6619 having worked with microservises in enterprise, it's a really nice fit for big applications that handle a lot of different business logic. It's very efficient development-wise if that matters, you can hammer out code like on a factory line. Debugging though is a pain.
You are joking, but after my latest blasphemies and insults about serverless in the last 2 months, I just decided (2 hours ago) my self to port 130 functions (they take 1 minutes to start) to a classic rest apis, You know what they suggest you in order to fix the slow start? To enable Premium features, which means to have 24hr running server, well, then, where's the gain with serverless?
@@luxxa Idk man, i think serverless is good enough for notifications and simple mobile apps. Maybe is not the right tool for your project. There is some SERIOUS limitations with things like firebase.
If you're looking at 'serverless' as Lambda functions, then yes, it makes sense that it doesn't make sense. However, Serverless is much more of hooking into the AWS (or GCP, etc) ecosystem to manage your infrastructure and you can take most advantage of it by understanding and working with event driven architecture. DynamoDB with Appsync, the AWS GraphQL solution, is a perfect example of how to make serverless work FAST. It connects directly to DynamoDB and you never have to worry about cold starts (or even internal http requests) and can get some CRAZY fast data requests. Within the Serverless ecosystem, you also have complete insight into what the different segments of your code is doing. Serverless event messaging (whether through SNS or Kinesis) is way better than a home solution and way cheaper than a managed solution. What many people think is 'serverless' is really just writing Lamdba Functions to respond to api-gateway events. When in reality, serverless is an entire ecosystem that you don't have to build or manage - Authentication, Events (almost every event imaginable), Block Storage, Data Storage, Message Queuing, Caching, and a host of other things. If you're building a webserver, yes. A standard webserver is a better idea. But if you're building a heavily integrated application that relies on event driven architecture and automation, serverless can save you tens of thousands of dollars.
TO SORT OUT THE COLD START ISSUE YOU NEED TO DO THE BELOW THINGS. 1. monitor how often you api get Hits. 2. you can use serverless warmup plugin 3. increase you lamda memory size.
I actually really enjoy serverless. Yeah it might not be the best in terms of performance, but it's pretty nice being able to write lambda functions and not have to worry about all of the other crap. I don't really want to spend my time reinventing the wheel by setting up and configuring servers. I've done that enough in the past decade. This just feels like the natural progression of back-end development -- let the IT guys handle the server maintenance and we write the application specific code.
Serverless is really good to use as glue between cloud services (take something out of RDS and put it in s3 for example). But to actually replace and application or service as serverless causes more problems than it solves. Yes, it is possible. And yes XYZ company did it one time. But it is more trouble than it is worth. Server OS' are so stable nowadays that there is already minimal overhead in your VMs, all you gain from serverless is other complexities like coldstarts.
Hey Ben. Provisioned functions are not about predicting your traffic but instead predicting the spikes that you may have. In your example you will always have 30 extra functions and you will have warm starts unless you have more than 30 new requests before the new instances are provision. On another note my favourite tool is Cloud run that is not Function as a service but Container as a service. Each container has a max of 80 connections and it solves the issue of having multiple functions that need to be warm eg. for Cloud Functions an API could have 20 cold starts where with Cloud Run only one because the container is reused!
It depends on the cost of having a server up and running 24/7. The Lambda container stays alive for about 15 minutes so it's warm. Depending on the traffic you got, in theory if you get 1 request each minute your Lambda should never go cold, also if it does get cold it is only impacting the first customer with the initial request and the rest will benefit as the Lambda is now warm.
I am an app developer and had no idea about backend infrastructure. Anyways with google search and chatgpt I built an app which does some processing on images through a firebase cloud function. And I am facing same issue. Some requests take insane amount of times like 5 seconds. Currently I have a traffic of around 150 users per day but if it scales I will need to get more into this. Thanks to Ben and youtube for recommending this
sometimes we just have to embrace the fact that cold booting is okay in certain situations. Like when I'm only building a really simple ecommerce site with the JAMstack and I'm only using like 4 different routes for checkout/order status stuff, in that case serverless seems more attractive that setting up a complete server.
Setting up a server is like 5min... install nvm (one command) and off you go. Serverless makes easy things easier. But makes complex things impossible.
@@batmansmk the hard part that serverless abstracts is not creating a server but keeping it up, which is way harder than writing three lines of code to listen on a port
@@batmansmk love your line at the end, it pretty much sums up my use case. If I just want a couple of services up that execute CRUD transactions, I don’t see the need of mantaining and setting up a VPS or cluster when i can just upload my services alongside my front end on Netlify. Once again it is just my use case and what I personally think it’s the easiest way around. Projects that require a bunch of services across multiple teams, faster response times as possible and overall a more robust architecture would get better benefits of a non-serverless implementation.
By the time we need more CPUs than what a single ec2 can provide, we are probably way past the simple CRUD on the product too. Serverless is a solution to a rare production problem. With S3 and RDS, maintaining a server takes less time than just watching one video on YT.
A proper engineer that tests and measures his services and makes decisions based on the results. I wish I had people like you working with me. Most of them don't do any of that.
The wonderful thing about lambdas is that everyone can be in a different language. If you want to run a program in java you can. If you want to run another in .net you can. If you want to run another in python you can. Another in node js fine. Even though they are each in a different language they are able to communicate with one other. Not only communication with one another but integration with aws itself. Most aws services expose events that can be interacted with using lambda. For example, you can run a lambda in response to an upload to an s3 bucket. You can run a lambda as a side effect to many different aws service actions.
I mainly use lambdas for scheduled tasks. If I only need a bit of code running a couple of times a day, I don’t want to pay for an EC2 instance to be on 24/7
If you are getting 30 requests all the time, then the first 30 cloud functions are gonna experience the cold start. Subsequent requests will be warm. So from my experience it is okay to use serverless in those situations. And the image resizer module would be a perfect candidate imo. However, the part where serverless sucks in my opinion is where you have to process long running tasks - say > 10 minutes. In such cases you have to figure out how to break up your logic into smaller pieces, communicate between them via a message queue etc.
Functions should be locked written in a nice and compiled language that starts quick, rust, zig, c++ that sort of thing. And serverless functions should only be used for those jobs that take more time to run than a connection setup costs you and that do not hit any storage. This is where most architecture goes wrong. Making devices for tasks that run in mere millisecond (100ms) is already the time it costs to connect and accept a TCP connection. So keep that locally, the network overhead is crippling you. Serverless functions that hit storage in parallel will cripple the performance of the storage. So either serially execute those stored by pushing them into a MQ and have a subscriber(s) pick them up and process them. Also be aware that there can be an order to these messages. You can’t update a record before it’s inserted. CRUD is inheritedly serial in nature.
Lambda and GraphQL is a great combination. I can’t speak for the other serverless providers, but I have had a great experience with Lambda and GraphQL. You have to think differently when using Lambda as an api. Minimize bundle size, declare as much as you can outside of the handler, etc. There’s a ton of factors that go into the actual speed of your response. If your Lambda is 20mb, that 20mb needs be downloaded from S3, mounted in the Docker container, and ran. It adds up. Using webpack is a huge boost to your bundle size. implementing tree shaking at work has helped me reduce Lambda sizes from 20-30mb to 200-400kb. Same functionality, but with the serverless first mindset. GraphQL really shines because you keep invoking the same Lambda over and over. Ours are in a vpc which causes longer cold starts, but keeps the container alive a lot longer, sometimes 2x the life of one outside the vpc. It all comes down to optimizing the code for the serverless platform. If you write static server style code, and throw it in a Lambda, you’ll have a bad time.
People dislike serverless because they don't know it well enough, but it's understandable, it has relatively poor ressources online and we need to learn lot of things to make something serious. But once you master it, all becomes easier
@@arvi8843 No, the Apollo Lambda package is over 1mb, and requires GraphQL as a peer dependency. I just use the plain ‘grpahql’ package. If the Lambda doesn’t need an sql db package like ‘pg’, the bundle size is usually 100-200kb.
Ben just wondering if that was a bad design choice. I would approach it differently by creating an image factory where it takes on (upload) the image process it via lambda call and generates 3 (or more if needed) different files based on the resolutions and it would add these files to an S3 bucket which you can use as CDN to consume the images from.
I spent 1 month trying to figure out what the heck should I do to deploy my small project. When I saw what can be done with AWS Lamda + API gateway, I realized how tough can be a month of your life, if you are not aware of simple things like serverless.
I personally like AWS.. I can't use anything from google cloud their UI sucks same as AWS but I feel there's more resources for AWS when I get lost... again lol I'm no expert with cloud or serverless so I go for which one I can survive on
It looks to me that your reasoning is too simplistic. There are other AWS services, best practices and techniques you can use alongside Lambda to reap the benefits. As any technology, there are pros and cons. You need to know where it makes sense to apply each. As you advance in your career you will see that there is more to a good architecture than these simple items you're comparing here.
we were using cloud functions as the main drivers, and it was a nightmare. even the warm starts aren't guaranteed because with traffic, some users will be moved over to new clusters. the main use case for these, as i see it, is to save costs on requests that aren't essential to the performance or presentation of your service. they can work for prefetching, uploads, and just having data handy for predictable user interaction, but even then cold starts would range from 3-20s for us sometimes. it was nuts
Out of curiosity, wouldn't it make more sense to create an endpoint that is designed to optimize your image then store it in something like S3 rather than re-optimizing an image on every single request? That seems like a lot of extra compute work that could be mitigated by using static assets more optimally. If you were pretty set on optimizing the flow as you detailed, you could also create an endpoint that just returns 200 and a blank string in the same lambda. With that you could setup a scheduler to hit that once per minute to keep your function always hot to your cold start timing.
Good point, but he would have to decide for each image the width and height statically and this is actually what he wants to compress dynamically. Maybe for some images the same dimensions are with a high probability requested and for those your optimisation is probably better.
@@romeostoll4056 You could still design the endpoint to accept parameters that would map to the expected or desired size. It's common to create a standard size to name convention and output not 1 but multiple versions of an image. For example to hyper optimize i might create a 2x, 1.5x, 1x, 0.5x, and 0.25x image pattern that outputs three versions of each image that have progressively better quality settings. In this way I can target 15 versions of the same static image programmatically. I get near instant loads on smaller lower quality images and I can choose to progressively load in images of increasing size and quality as needed to give a great user experience. If we extrapolate on the honey/molasses analogy Ben used, I COULD squeeze the bottle until I am shaking and maybe have some small success. However if I take the time to understand the problem fully a different approach like heating the bottle correctly will make the fluid flow more freely and thus give me largely optimal results. Also, interestingly to that analogy i can store the honey in a cold state when not in use and only heat it up as needed to access the contents.
Hi Ben For resizing, if you use S3, you can create S3 and then create S3 Lambda Trigger. Every time you upload to original S3, it can store image in other S3 as resized image. Lambda uses NodeJS ofc. Idk if it suits ur usecase, but its good approach.
@s brightThe best I found are from dynamodb book from Alex DeBrie. You need to pay extra to get examples, but you get like 6 boilerplate apps with serverless framework, nodejs and dynamo with best practices that are explained in the book. Best book I've ever read most likely.
Alright *cracks neck* so . Don't deploy via click click route , that sucks no one does that . 1) Use serverless framework to deploy they have templates for express js and everything . Most applications that I develop aren't bound to how the traditional function routes . I just use a middleware for the express library or if you're in golang you use an http adapter to bind the API gateway requests to something your framework can understand. 2) cold starts were an issue but aren't an issue anymore . First thing to do. Increase the ram size from the default , that usually decreases the cold starts by a good margin . Like a very good margin 3) If you know this is something on demand and don't want to deal with cold starts . You can use provision concurrency in AWS which will basically warm up x amount of lamdbas for you. Use this with caution as it comes with a cost . . 4) if you decide by any chance of yeah I wanna connect to rds or something . Do not just directly to it via your client ! If too many lambdas are spawned the connections will not be closed on time . You'll want to use rds proxy . This way you have a thread pool , and the time to setup a connection isn't as slow. For our work it brought down our api response times by half . And fixed our issue of scaling where it wouldn't choke the DB . 5) what are you writing an API that uses sql or relational dbs? If sql then go with golang . Otherwise use node for anything nosql . As the dynamodb solution works pretty good out the box for nodejs . . Reason why I say this, because with go you get the best of both worlds , no jit or anything . So low cold starts since you compile the binary . And second the runtime is as performant if not more than Java / c#. 6) API gateway is awesome , and you can setup custom authorizers ... Etc . But it comes at a cost if you have alot of customers . So get ready to fork up cash . 7) vpcs ! They make your lambda slower especially with rds , so make sure you use rds proxy or you'll have a bad time . 8) don't use Java for lambda with spring boot. Don't ever do that . Ever .
You are focusing too much on synchronous API calls. If you are supporting your API with serverless functions then yes - cold starts suck. If, on the other hand, you are using serverless for async operations in event-driven systems, where response times do not exist - then cold-starts do not matter that much. Try this example - API running on a kubernetes cluster (or ECS, whatever you like), at the point of receiving requests it publishes an event to SNS, you have SQS subscribed to this and Lambda triggering off that event, processing some data asynchronously. Or another one: API call -> kubernetes -> something saved into DynamoDB -> Lambda triggered from DynamoDB stream. Real world examples would be: user buys something in an online store, you do not need to generate an invoice "in real time" before showing them purchase summaary. A lambda asynchronously triggers of that event, generates the invoice and that gets emailed. THIS IS where serverless TRULY SHINES.
Well you do have a good point, one of the things many people advocate is that servereless functions should be small and fast so use cases such as the one you have pointed out are better handled by a convensional provisioned server. However use cases that are less computationally demanding (e.g. updating a specific user entity) and are performed many times by users can be consided candidates for serverless lambda functions. This has one advantage conventional servers will not be bombadered by large amounts of requests for small computational workloads and thus can be utilised for more computationally expensive use cases. Whilst serverless functions can do most of the simple grunt work. This is also very evident from the timeouts lambda functions have. Cloud providers do not want you to use them for demanding use cases, you have services like ECS/Fargate for that.
@@aj.arunkumar I know. But he's already offered two videos on these techs and both are good NOT like appreciating the one because of fan boy and ignoring the other, so that's why I asked!
That’s an interesting overview. I have experienced similar issues and ended up with writing a “keep-me-warm” lambda. Comparing performance and cost made this approach look better as opposed to “cold-start-it-is” one
In the past, I would have agreed with you on the two things you said in the first sentence. Today, I understand that there is sometimes an appropriate use of Serverless. For example for models or calculations. In my opinion less for routine things like image resizing service for customers
The cold start for go is unbelievable. I''m running an experimental project using go lambda. The cold start for go is beyond fast. I'm running go lambdas that interact with bonsai (elastic search), key spaces (cassandra), and write to s3. I write the entity (master) docs to s3 as json docs. I use elastic to index and search those documents to avoid the astronomic cost of searching s3 docs. Instead of using a relational database to manage the entity and also search it split it between multiple services. One service (s3) to store the json document (master) - instead of relational tables - one single json document. When the file is written to s3 kickoff a lambda in go to index that document inside elastic (bonsai). Not to mention the lambda code being written is so small to do all these things. Combine it with something like bazel run lambdas using any language within one single repo. All this combined with serverless framework. No need to manage some convoluted, expensive kube infrastructure. Instead everything scales without needing to write absurd config files to manage kube. The backend of the project facilities any language I can throw at it. I'm going to be adding dart to compile scss soon. With bazel, aws lambda, and serverless framework that is possible. Its feasible.
This video reminded me of another one where some guy tried to make the case that object oriented programming was bad, and his evidence were all examples of very simple procedural functions for which OOP was obviously overkill - which would be like saying cars are useless because bikes are much cheaper and perfectly enough for reaching a destination four blocks away. The image resizing thing is especially bizarre since the reason why image resizing is such a typical serverless use case is that it can be asynchronous, making the whole cold start rant irrelevant. So instead of proving a legitimate downside of serverless that example just made it seem like he doesn't even get the point of it. It's also kind of weird that these types are talking as though VPS-based computing is going anywhere? I don't doubt some devs may buy so much into the hype that they end up recreating a much less organized and responsive version of their full RESTful API on Cloud Functions and that is certainly silly, but serverless endpoints are fantastic for isolated pieces of functionality. I don't see why be bothered about an extra option when the previous one is still available.
@@DodaGarcia Was that the Brian WIll video "Object-Oriented Programming is Bad" by any chance? I am a pretty ignorant developer and that video started tilting me into functional programming, that sounds like a bit of context I could have missed from his argumentation. I would love to learn more about OOP but I haven't come across any solid intermediate deep-dives into it.
@@franksonjohnson Interestingly enough it wasn’t that one but another one by that same guy, called “object oriented programming is embarrassing” - I’ll watch the one you mentioned to see if he makes different points. When it comes to tilting though, there’s really no reason to choose between them because OOP expands on FP rather than replacing it. Functional programming has seen a big resurgence recently with serverless computing and the popularization of machine learning. It’s practical and intuitive in how it follows a linear set of instructions, and enough for a lot of uses. OOP is a way to design your code that mimics how in the real world things belongs to categories and share properties and behaviors within them. It’s usually the standard for any project that’s expected to grow in complexity over time since it allows for more organized and reusable code by way of those hierarchical inheritances. So it makes sense to get used to functional programming first and then familiarize yourself with OOP as needed, if nothing else because it’s how most external libraries are structured. If this was about choosing a first language to learn it would make more sense to talk pros and cons, but when it comes to programming paradigms, most developers aren’t in “team OOP” or “team FP” any more than handymen divide themselves between hammer fans and screwdriver fans. They just use the more appropriate tool for the job at hand. This is not specifically about OOP but my favorite development channel is @fireship, it offers a good mix of practical tutorials and language-agnostic design pattern lessons, some of which interview-oriented. (I’m sorry if any of this was already obvious to you, I don’t know how experienced you are and erred on the side of thoroughness.)
@@DodaGarcia No very helpful, thank you! I've always felt tension with the functional gospel on no side-effects and whatnot as a web developer, because pretty much any DOM manipulation is a side effect right? I like the handyman metaphor a lot.
"If your API is only good when the functions are warm..." I honestly don't understand the quibble about a 0.5 second cold start. First of all, serverless wasn't designed to be used for your high speed endpoints. It was meant to offload processing that takes a few seconds to a few minutes to execute. A bit of cold startup is meaningless if it takes 60 seconds to run the function. Using serverless for all the normal API endpoints is obviously for small personal projects and maybe as a first step in a startup. But even setting that aside, for a web page your time to first paint depends on your front-end bundle which will probably be coming from a CDN. Once that comes up, you can give a nice spinner while the API fetches whatever data is needed for the front page, and if you can cold start that first hit at less than two seconds or so, you're fine. If it takes longer than that for your endpoint to pull down the data needed, the 0.5 second cold start for a lambda isn't the part that really needs work, it's your function or maybe your DB. All that said, if you're using serverless for regular API functionality, then it obviously makes sense to go with whatever company has the lowest cold start times, so I really appreciate you including that in your video @3:43. That chart should really make it clear that lambdas are _not_ designed to be your basic GET endpoint. But since AWS has such low cold start times, why not use that for your toy project? And if you're doing serverless for the low cost benefits to a small project, then leverage DynamoDB which has a similar "only pay for what you use" model. Yeah, it's kind of a pain to learn, but that's why software engineers get paid the big bucks.
So yes, if you have consistent traffic you shouldn't be using serverless for your API, you should be using something that automatically provisions servers based on traffic needs. But that's above my pay grade, I just want to write Javascript without paying twenty to thirty bucks a month for hosting a program that nobody is using.
To get rid of cold starts I use Google Cloud Scheduler to hit my serverless function every minute. If each call is billed 100ms then it fits very well within the free quota.
I'm surprised you didn't talk about one of the worst parts of serverless to me: The developer experience. Testing locally is sometimes pretty much impossible, and deploying to a test environment to test in a team setting is often problematic.
FYI the Cold Start has its solutions, like *provisioned concurrency* in AWS. You pay extra, and function is always "warm". Serverless is the next big thing and IMO we're gonna see a huge shift to serverless in next 5 years. You write code and deploy it. Dont need to think about service infrastructure on daily basis.
3 things that will help with those cold starts: 1) bump up the memory. It might even be cheaper if your code doesn't run for as long. 2) compile with webpack 3) choose provisioned concurrency to keep your lambdas warm. Wish you'd dived a bit deeper, but I know it's a lot.
I think use cases really matters. I used functions as a way to make background processes that don't need to be done in real time. I would queue messages in a pub sub and functions picks them up and processes them for something like email or SMS.
In my view, Serverless provides other benefits than not having to deal with load issues. It also allows you not to spend too much time configuring the infrastucture... Of course, I'm not saying it's smart provisionning Functions ahead of time ofc, just correcting one point, in my opinion ofc. Btw, isn't there some wait to configure the "Keep Warm Period" ? Doesn't the "Keep Warm" period last long enought between each call? Thx for the vid!!
I think the first thing people get wrong about serverless is serverless == lambda, functions etc. Thats the wrong way to think about 'serverless'. I think this thinking has been pushed due to the popularity of frameworks like Serverless Framework. Serverless really just means you shouldn't care about managing your servers. This means load balancing, auto scaling etc are taken care for you. Auto scaling containers (ECS, Fargate, EKS, Swarm) is also 'serverless' ... Lambdas make decoupling services easy but that doesn't mean you should use them to like an actual racked machine server. I've managed monolithic servers the old school way and the serverless way (though usually more expensive) wins every single time. Serverless also includes managed Queues (SQS), Brokers, Databases (DynamoDB), cache memory (ElasticCache) etc .... Its too broad to just say meh. Even the K8s cluster you mentioned can be managed serverlessly.
I feel like sharing a serverless success story here. I'm using a Lambda function to process analytics streams from Segment.io which includes analytics around our search service (Meilisearch). To say the least, this thing handles some serious throughput as our app is designed to deliver results while typing without any debounce. Here's the thing though: our lambda functions are all written in Rust Lang, and so the binaries are incredibly tiny, there's a small resource footprint, and it's absurdly fast. Right now we're using the *least* amount of resources that's permitted as to basically make this perpetually free. Here are our stats over the past 3 months: Duration Maximum: 48.07ms (cold start, outlier) Average cold start: 43ms Duration Average: 4.31ms Duration Minumum: .84ms Allocated memory: 128mb Memory used: 31mb Binary size: 4mb So I wouldn't say serverless isn't attractive; it's just that the vast majority of languages are just ill-suited for it. Rust Lang on serverless however is a complete game changer. That said, even though I do user serverless for many use cases, I still prefer to deploy to K8 as then I can have an API that's really taking advantage of multithreading and connection pooling. We're using things like Rayon and Tokio block pretty much everywhere conceivable so multithreading really makes a huge performance difference.
