I'm in love with serverless Rust. My company is struggling so hard with GCP Cloud Run + Node.js coldstarts (3-10 second cold starts for our insane dependency tree). I went down the rabbit-hole, excited to introduce CF Workers, and also a possible prospect is Rust Lambdas.
Why do you need a huge microphone like that? These 2 guys ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-m1Isot9xoxU.html used some very good microphone .
It's actually easier when I'm at my desk to have a mic attached to the desk and plugged into my laptop than something I need to connect with Bluetooth and attach to my t-shirt. I have mics similar to this as well but it's just preference to have it attached to my desk
A really great video. I've used the 1st and 2nd option for securing data. I don't like the 3rd option as it is error-prone. Someone (DevOps) forgets to configure some permissions, and you end up data being available to more services than intended. You have really great videos and you deserve 10x times more views and subscribers
As usual, great video. In the second example, you could make a simple enhancement to fix yet another problem. If you have a key per user, say stored in a DB/kms/somewhere, you can encrypt the user's data on the bus with their own key. The consumer would also have access to that user-key. The advantages are that if you need to forget the user's data, you delete the key. This meets compliance requirements anywhere I've seen so far. Bonus - You'd also use their key to encrypt any sensitive data in the datastore itself as well.
@@sergeykichuk2586 there is an issue around data sovereignty. Many pieces of legislation like gdpr require the right to be forgotten. However, transactional systems often have a lot of referential integrity issues with that concept. If you can leave the data in place, but delete the key, you remove access to the data which satisfies the requirement.
@@sergeykichuk2586 it would be the producer who manages the key. The producer determines who has access to it. And as Derek points out the producer could choose to delete the key and remove all access to the data.
Thanks for the great video @serverlessjames I figure I must be doing something wrong. I can run dotnet build fine but when I attempt to run sam build --use-container --mount-with WRITE --build-image plantpowerjames/dotnet-8-lambda-build:8.0.100-arm64 it errors with: Error: DotnetCliPackageBuilder:Validation - Runtime dotnet8 is not supported Workflow DotnetCliPackageBuilder does not support value "False" for building in source. Using default value "True". my SAM version is SAM CLI, version 1.121.0 Any idea on the issue? Thanks!
@@DerekMurawsky I don't see the CDK supporting Rust in the near future, but I hope I'm wrong. At the moment the best solution I've found is to write CDK code in another language, and your functions in Rust. It doesn't need to be TS either. You could use any of the CDK supported languages for writing the CDK stuff.
Great to see you educating devs around CloudEvents :) One minor comment: the schemaref field also helps transports and consumers to use schema registry/decoders for events and not having to only rely on event type versioning.
Great video! I tried to use new dependencies that are used in my Controller but when I try then to deploy with cdk I get an error saying Resource handler returned message: Resource of type (arn) did not stabilize. Status Reason is An error occurred during function initialization. Is there something I need to know if I want to use other dependencies? I can nevertheless deploy the api locally and everything works. Thanks in Advance
Great tip. I remember hundreds of times in the real life when you expect the data to be not null due to the API documentation. But you get nulls. Your software should be prepared for this, have validation in place, etc. It will much more convenient to return 400 Bad Request with good explanation rather than 500 Internal server error with NullReferenceException
Hi James, thanks, amazing video and explanation! You've clarified me a lot the differences between Orchestration and Choreography in such scenarios. I'm wondering if the need of predefining the channel to publish the response back (from the payment service in that example) is due to having multiple orchestrators. If there is only one, would it be necessary to predefine it, or would just sending the correlationId be enough? Thanks in advance!
I think you'd probably want one responseChannel per orchestrator. The consumer of the message would always need to know where to send a message back to. Just having a correlationId wouldn't help them know what to do with it :) You could even make the responseChannel something like AWS API Gateway with a direct integration into SQS. So the consumer is just making HTTP API calls, but you have the freedom to change the underlying channel implementation.
Hi @@serverlessjames ! thanks for the response, If I'm not missing any of the parts of the puzzle, (and sorry if I'm not doing que question properly) I'm asking explicitly about the need of the orchestrator to specify the response channel in the message pusblished in the first step. If you have just one orchestrator in that case, can't the payment service already have a predefined channel where to leave the response messages along with the CorrelationId without the need for the orchestrator to tell you so? Thanks in advance again!
@@xaberueoh, i understand your point now. Yes, it absolutely could be predefined. But then if hte producer needs to change it, you need to make changes in 2 services.
@@serverlessjames that's true, your recommendation makes the services more autonomous to changes. thanks for taking your time to reply and clarify it !!
Just to make sure I'm connecting the dots. Are these orchestrators sometimes referred to as "Sagas" (thinking MassTransit). And in native AWS land, you could do request response using SQS temporary queues?
Kind of :) You would use an orchestrator with a Saga. The big difference with a Saga is that you would have a compensating action when things didn't work, to rollback the changes. That gives me an idea for another video though, covering the Saga pattern :) Thanks for the comment Brendon. Yes, you could absoloutely use a temporary queue. I'd prefer to use a single queue wired up to a Lambda function or something, and then use the correlationId to provide the mapping back to the right orchestrator.
Thanks. I’ve been wrestling with the best way to approach some long running background process that have a series of async steps in a serverless pattern. Looking at patterns there seems to be vague similarities with state machines and aggregate roots / event sourcing. Yet another place where the lines get blurry for me. Perhaps another video/talk idea.
The events and commands are the methods to communicate in an event driven application. Instead using requests/response it uses pubsub approach/pattern. With that , if you need transactions like ACID transactions in databases, you would need to use SAGA pattern (Coreography or orchestration) or any other method (2 phase commit ) using previous patterns.
James, it would be awesome if you publish a fully fledged course with an interesting architecture (that mimics some of real world examples), Rust + AWS services from scratch
Great feedback, thanks ☺️ I've got a bit of stuff in the pipeline. I'll add it to the backlog. This might be something I end up building on a livestream
Hi, I just now stumbled upon your video. Great work! I have one question: is it necessary to build different binaries for different endpoints or could you have all the endpoints in one binary and then pattern match on the path and http method to execute the right handler? Thanks 😊
You could definitely take the pattern matching approach, but actually if you are going to do that you're probably better off just running a web framework inside Lambda. Something like Axum/Actix. There's another video on my channel on how you can do that.
Great video. In an event driven system would the primary focus be on representing business events, but in addition you would possibly also still need some technical events as well?
Absoloutely, and great question! The technical events would definitely exist, they just wouldn't be driving all your functionality. The example I always think about is if you upload a file to AmazonS3. S3 is going to give you an S3ObjectCreated event (technical) but for integration purposes you'll probably transform that to an 'InvoiceFileUploaded' event, something that is meaningful to the business.
Based on this definition, no. The distinction is a technical event or a business event, I can think of maybe a few places were you might have crossover but definitely not always the case.
Excellent description of these technologies! It really gave insight into what to use when. Thanks for the tip and your talk at the ndc in oslo, really inspiring! 😊
There is a very generous free tier yes. Throughout all the RU-vid content and demos I've ever created I've never paid a penny for Lambda compute "The AWS Lambda free tier includes one million free requests per month and 400,000 GB-seconds of compute time per month," - aws.amazon.com/lambda/pricing/
Very clear breakdown! When using an event bus (not combined with a queue), does every instance of a consumer receive a message typically? Or is it load balanced to a single instance?
Great question. Typically an event bus is there to fan a single event out to lots of different subscribers. So yeah, if you had multiple instances of the same app subscribed directly to the bus with the same configuration I'd expect them all to receive the same events.
Hi James, Great video, trying to wrap my head around since lambda are stateless how would the website manage the session in lambda , also would lambda be able to ship javascript, images etc that is embedded into page ? Also if data should be post back to the page how that should be handled ? would it (api gw) support windows authentication.
