Тёмный

Sagas: Event Choreography & Orchestration (NServiceBus) 

CodeOpinion
Подписаться 84 тыс.
Просмотров 40 тыс.
50% 1

How do you handle business process and workflows in an asynchronous event driven architecture? You can use event choreography and orchestration with compensating actions to overcome the lack of distributed transactions or two-phase commits in a distributed system. Here's how Event Choreography & Orchestration work and examples using NServiceBus.
🔔 Subscribe: / @codeopinion
💥 Join this channel to get access to source code & demos!
/ @codeopinion
🔥 Don't have the JOIN button? Support me on Patreon!
/ codeopinion
📝 Blog: codeopinion.com/event-choreog...
👋 Twitter: / codeopinion
✨ LinkedIn: / dcomartin
Sample Code:
github.com/dcomartin/LooselyC...
0:00 Intro
0:44 Choreography vs Orchestration
2:13 Choreography Message Flow
4:31 Orchestration Message Flow
8:06 Compensating Actions
10:07 NServiceBus Saga
CodeOpinon: codeopinion.com
Twitter: / codeopinion

Наука

Опубликовано:

 

15 июн 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 85   
@CodeOpinion
@CodeOpinion 3 года назад
It's not an either/or with Choreography & Orchestration. You can use both in your system in different places. Where do you use Choreography & Orchestration?
@frank-michaeljaeschke4798
@frank-michaeljaeschke4798 3 года назад
From a technical point of view 2 examples came to my mind about choreography. Database XA transactions are a form of choreography too and Spring Ldap will, if used with Spring transaction handling, try to send compensating actions to Ldap server in case of rollback. But I think this are 2 very low level examples (only from technical side), as I think Sagas handling such things more high level from a business process view.
@RM-bg5cd
@RM-bg5cd 3 года назад
I had to deal with this a while ago. We had a critical choreography process a while ago that was giving us headaches. We managed to catch fresh hair and caught up to business demands, so we had time to refactor. We ditched the choreography for an orchestrator because it was becoming too difficult having to add new consumers if anything new came up, our new orchestration (or orchestrator) is the AWS Step Functions service. Night and day.
@rahulaga
@rahulaga 3 года назад
I think this is first video I came across which kind of explained the difference between command & events so beautifully. thanks !!
@vadimprudnikov9072
@vadimprudnikov9072 3 года назад
Thanks for the video! I would add that BPMN-based orchestrators can significantly help in complex business scenarios despite of they introduce some sort of coupling. Being able to see what's going on in the system from one central perspective is very helpful.
@gouki178
@gouki178 2 года назад
HI Derek. I am watching all your videos, thanks a lot to share this with us. Kind Regards from Argentina, Buenos Aires !!
@CodeOpinion
@CodeOpinion 2 года назад
Thanks for watching!
@BoJack32
@BoJack32 3 месяца назад
Tremendous video! Very well explained! Thanks!
@CodeOpinion
@CodeOpinion 3 месяца назад
Glad it was helpful!
@JosiahWarren
@JosiahWarren 2 года назад
Clear explanation. Thanks
@CodeOpinion
@CodeOpinion 2 года назад
You are welcome!
@burimcakolli3400
@burimcakolli3400 3 года назад
Thank you alot!
@GavSaysPogMoThoin
@GavSaysPogMoThoin 11 месяцев назад
Would love to get your take on routing slip choreography with MassTransit and/or NServiceBus. Keep up the great work.
@CodeOpinion
@CodeOpinion 11 месяцев назад
Good suggestion!
@robertolech6414
@robertolech6414 3 года назад
Hello, very informative and clean video, especially for beginners. Although I have stumped upon a problem, comparing to scenario from video (I like the last mixed approach very much, so let's focus on that): Let's say there is an Order Delete command (not the NServiceBus one) genereted by the user, order get's deleted which is an aggregate, order deleted event added to the queue with an outbox pattern, context is saved and order deleted event is published (the bussines logic doesnt makes sens in this example, but just hipoteticly). Then billing reacts with deleting some data in it's own context, which is succesfull and some blilling deleted event is published, but warehause buisness logic would result in publishing compensating event. Yet there is no way of restoring a deleted aggregate from Sales or changes made by performing a delete in billing context, because I have only some minor data in the events which are POCO objects or just only the ids. What to do in is this kind of situation, is there other way to approach this then 2PC? I thought about setting some flag IsDeleted, global filters and some background service to delete those with a flag once in a while, but what if the background service will performe cleanup when the order delete workflow is being processed? and also this seems like a overcomplicated solution and whatever approach I pick results in inconsistent data needing manual intervention. Would be very gratefull for any guidelines, thanks in advance.
@CodeOpinion
@CodeOpinion 3 года назад
If you need to "undo" something by a compensating action, then you need to be in a situation that you actually do it. Which means you can't actually delete the data because you need to restore it. As mentioned you could do a soft delete. Or maybe move the data to another datastore/table so you can restore it if needed. Ultimately in your exact example, you can't delete it, because you need a compensating action to restore it.
@user-qt6tn4xl2e
@user-qt6tn4xl2e 2 года назад
Thanks for video. What do you think about workflow tools , for example using temporal.io or AWS step functions to orchestrate business processes?
@CodeOpinion
@CodeOpinion 2 года назад
I really want to look into temporal.io more just haven't yet
@FrantisekSidak
@FrantisekSidak Год назад
Very useful piece. Thank you.👍
@CodeOpinion
@CodeOpinion Год назад
Thank you! Cheers!
@rollinOnCode
@rollinOnCode 2 года назад
when would you use choreography vs orchestration? and how do they relate to the "saga"?
@CodeOpinion
@CodeOpinion 2 года назад
I often times refer to "orchestration" because the term "saga" or "process manager" is unfortunately used interchangeably. Regardless, orchestration is how I described it. Having a centralized place that details a long running business process. When should you choose choreography or orchestration is really dependent on if you want logic centralized. It really is a trade off as I described it in this video.
@ramonennik2536
@ramonennik2536 10 месяцев назад
If the user should receive a notification or email for one or more of the events, would this be the place to also initiate sending such messages?
@NervousNab
@NervousNab Год назад
Cool, good to know that orchestrator that I found in project isn't doing what it's supposed to be doin.
@nicolasdb2497
@nicolasdb2497 2 года назад
Do you sometimes store the current state in a database? I'm not using .net so I don't know if changing `Data` does something behind the scenes. Thank you for all your videos
@CodeOpinion
@CodeOpinion 2 года назад
Yes, it's persisted. A messaging library like NSeviceBus or MassTransit provide the capability for various databases.
@pianoman1973
@pianoman1973 Год назад
Great tutorial ! Do you have a playlist of all the microservices related stuff ?
@CodeOpinion
@CodeOpinion Год назад
Not exactly microservices, but messaging specifically: ru-vid.com/group/PLThyvG1mlMzm2FyVpKDiU2c7VtrB2Zezg
@pianoman1973
@pianoman1973 Год назад
@@CodeOpinion Thanks a lot !
@thecodeman_
@thecodeman_ Год назад
Hi Derek, thank you very much for this video! Have some questions, would be great if you can answer. What you think about totally separated Orchestration in separated service (not "coupled" to Order Service side)? Would be that a good approach? Another thing, if we are using Orchestration with the NServiceBus and for example, Bill Service "calls" Warehouse Service (via commands through Orchestrator), what happens if Warehouse Service is down or something similiar? How should we handle this situation? Thank you!
@CodeOpinion
@CodeOpinion Год назад
I generally put the orchestration in usually the boundary that kicks of the event that starts it. However you can have multiple events/message start a saga. So yes ultimately it could be separate. If a boundary is down that is apart of the workflow, it just won't continue it until it's back up and processes the message, assuming it's going to send an even/reply that continues it.
@yonatandaniyel5640
@yonatandaniyel5640 10 месяцев назад
Can you explain the distinct differences between a Saga Orchestrator and a Process Manager? I am confused by how similar they seem to be and how different people whose articles/videos I am watching define them differently. Does a Process Manager , itself , manage the state of the service , whereas as an Orchestrator merely manage which messages to respond to a service based on the message it receives from a specific service? Basically, Stateful (Process Manager) vs Stateless (Saga)? As well would a Saga Orchestrator be asynchronous vs a Process Manager which would be synchronous?
@codewithkashif
@codewithkashif 3 года назад
Awesome article! To keep it simple In orchestration approach, Is it ok to not to use/publish message/events to invoke subsequent services instead make a http calls. Would you mind to guide the pros and cons of it!
@CodeOpinion
@CodeOpinion 3 года назад
Using orchestration with sync request/response has the same advantages/disadvantages as synchronous request/response vs async messaging anywhere. Async allows flexibility in availability. With sync requests, if the service is unavailable, what happens to your request and the overall business process?
@codewithkashif
@codewithkashif 3 года назад
​@@CodeOpinion Thanks for your answer. yes this is true -- "Async allows flexibility in availability" However sometime we are dealing with cost intensive services like typical example of saga is - travel app is trying to book a trip - so has to call following services: - Car rental - Booking a hotel - Airline reservation - ...and couple of email notifications to customer so here in this case we have to arrange service call in such a way which is less cost effective in case of any problem in between.. like canceling a car rental vs cancelling a flight reservation is always easy also we cannot undo a sent email (unless we send another email to disregard the previous one) So in such case it may be synchronous http calls and that state has to be maintained at orchestrator level....your thoughts please... Thanks again for your great support!
@CodeOpinion
@CodeOpinion 3 года назад
I think it really depends on the context and the operations that need to occur. In the above example, without knowing the actual domain and requirements, I'm having a hard time guessing why any of it would need to be synchronous. My naive view of it would seem like it would be more difficult to be synchronous. But I'm probably not thinking of the situation the same as you are.
@FilipBekic01
@FilipBekic01 2 года назад
Let's say we have orchestration with five events/commands. On initial command, we want to pass data that's required by last one. Basically, we have to drag data through all events until it reaches last event. How would you solve this issue? Maybe storing data somewhere else instead of passing through each event?
@CodeOpinion
@CodeOpinion 2 года назад
You don't drag/pass the data from a single boundary entry point. I have a video coming out about event carried state transfer that explains not to do that but have the appropriate boundaries apart of the process by having the client interact with the correct boundaries
@Lelis718
@Lelis718 2 года назад
Hello, Thanks for the video! I am wondering if the Events Handled by Saga are created only for the Saga Orchestration purpose. IE: Suppose that, in your example, we have received a CreateShippingLabel event from other part of our application (that is not the saga), How saga should ignore this event?
@CodeOpinion
@CodeOpinion 2 года назад
Orchestration can be done a couple ways. One by listening to more events that can have many consumers, such as I outlined or in a more async non-blocking request/response but still using messaging. Check out: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-6UC6btG3wVI.html
@officialgrasptechnology1804
@officialgrasptechnology1804 3 года назад
Do Micoservices have a get request? Cause if they do its not autonomy. I created a order sage to create a order but what if the customer needs to retrieve that order after creating it. 🤷🏾‍♂️ should I create a different service to aggregate customer orders. Whats your opinion thank you.
@CodeOpinion
@CodeOpinion 3 года назад
Ya you can expose data from a service to be used by a client. The client likely would be apart of the same service, logically.
@mabdullahsari
@mabdullahsari 2 года назад
An orchestrator is nothing but a good ol'' Actor model. TIL.
@FilipBekic01
@FilipBekic01 2 года назад
In orchestration case, event and command triggered by same event will usually have same attributes. Take OrderPlaced event BillOrder command as example. They both have OrderId integer. Now imaging there are more attributes, you'd have to literaly copy-paste objects all the time. Any suggestions how to overcome that?
@CodeOpinion
@CodeOpinion 2 года назад
Depends how you're using events. I mention it here more-so that the events will actually be slim and generally only contain IDs if you're using them as notifications for workflow. The later part of this video mentions it: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-IzBEbfSg0uY.html
@FilipBekic01
@FilipBekic01 2 года назад
​@@CodeOpinion Yeah, making lightweight events might be the right solution. Thanks for replying on every question here on RU-vid!
@kenjohnsiosan9707
@kenjohnsiosan9707 8 месяцев назад
Im quite confused about async way of sending events. which means one event can be sent without waiting for the other events to complete? or async in relation to other codes that are being processed, since it's not logical to process shipping when billing is not done yet. what does it mean Derek? thanks
@CodeOpinion
@CodeOpinion 8 месяцев назад
Async in a temporal coupling way. They are independent from each other.
@rcosta551
@rcosta551 2 года назад
Hi, I would like to know if makes sense to use Saga with Kafka. The idea is to create a topic to getting all streams named as topic_orchestrator and one consumer to deal with Saga. I would like to get your opinion.
@CodeOpinion
@CodeOpinion 2 года назад
Orchestration generally involves a consumer that is concerned with multiple different topics. Having the topics be specific for the orchestration doesn't really make sense to me. Meaning the stream/topics shouldn't be the focal point. Rather the focal point is the actual orchestration code that is concerned with multiple different streams.
@rcosta551
@rcosta551 2 года назад
@@CodeOpinion Thank you. If I understand, would be a single consumer to subscribe to multiple topics at the same time.
@andrewneeson
@andrewneeson Год назад
Is the advantage of Orchestration that now services don't have to listen to both Events _and_ Commands? (like the [Sales] services is having to do in the Choreography example) I.e. *Choreography* Services: Events + Commands in, Events out *Orchestration* Services: Commands in, Events out Orchestrator: Events in, Commands out *Edit:* How would either pattern handle commands that require a response. E.g. A `ViewOrder` command?
@CodeOpinion
@CodeOpinion Год назад
Yes, Orchestration provides a clear understanding of event/reply in, command out. Choreography has no central knowledge
@andrewneeson
@andrewneeson Год назад
@@CodeOpinion So if responses are required (such as ViewOrder) then pure Choreography is not an option?
@javadzoser
@javadzoser 3 года назад
after complete process how to send response to users? it's async but user need to get response after request.
@CodeOpinion
@CodeOpinion 3 года назад
Tell them it's processing, not that it's complete. If you want to call back after it's done, you can use the events with something like Websockets/SignalR to push to the client. Check out ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-Tu1GEIhkIqU.html
@javadzoser
@javadzoser 3 года назад
​@@CodeOpinion so we have to create other connection like websocket/signalr . it means limit connection.
@CodeOpinion
@CodeOpinion 3 года назад
Well if you're client is web it's not likely going to connect to the message broker.
@emmanuelogoma2595
@emmanuelogoma2595 2 года назад
You can send respose to client via text, sms or email if response it not required on the UI otherwise, you can use websockets, signalr of FCM
@DavidSilwal
@DavidSilwal Год назад
Can you do it with masstransit?
@CodeOpinion
@CodeOpinion Год назад
Yes
@DavidSilwal
@DavidSilwal Год назад
@@CodeOpinion expecting a video from you soon.
@mishamovdivar
@mishamovdivar 3 года назад
Thanks for sharing. I can't see how Choreography-based saga is decoupled, let me explain. Event based communication is decoupled because when Order Service publishes OrderCreated event, it has no expectation about any of it's listener, it doesn't even care if there is any listener at all, that is fundamental property of decoupled communication. With Choreography-based saga, you DO EXPECT a response from an event which you published (Either success event, like ShippingLabelCreated, or failure event, like BillingFailed). That is not a decoupled communication by any definition i'm aware of, that is just Request/Response (not the HTTP based, but message based via Reply of NSB for example) model done via events. Business transaction by definition implies the coupling between the participants, because of atomicity, you want all of them or none of them, this coupling cannot be hidden or eliminated and Orchestation-based saga is a clear example of that, it's just playing with word "Event".
@CodeOpinion
@CodeOpinion 3 года назад
Everything you wrote makes sense to me but implied for Orchestration, not Choreography. Choreography, you're just publishing events as you normally would without any idea there's a consumer or business-process for that matter. With Orchestration you are well aware of the command + event responses of the workflow.
@mishamovdivar
@mishamovdivar 3 года назад
@@CodeOpinion Not sure i'm following. If you've no idea that there's a consumer or business process, then why're you listening to `BillingFailed` event and cancelling an order, or why listen to `ShippingLabelCreated` to complete an order.
@CodeOpinion
@CodeOpinion 3 года назад
As you stated in the first comment, the publisher has no expectations about any listeners. At a conceptual level, you are aware of a workflow from a series of actions+events that flow from one service to another in an event driven way, however there is nothing centralized in code that defines that workflow.
@mishamovdivar
@mishamovdivar 3 года назад
@@CodeOpinion Yeah, so as an Order service you're aware that Billing service participates in transaction and it should charge a customer, as well as Shipping service is included and it should make a shipping label and you (as an Order service) are expecting them to do that, or fail to do so. Order service listens whether billing went well or it failed, or whether shipping went well or failed. So order service is aware of participants and what they're supposed to do. Even more, Order service knows that when it publishes an `OrderCreated` that's when Billing service should charge a customer, so it's not a natural dependency, that should be from Billing service to Order service, but vice versa also, right ?
@CodeOpinion
@CodeOpinion 3 года назад
If you're orchestrating, then yes.
@megaman2016
@megaman2016 Год назад
What happens if the warehouse throws an exception
@CodeOpinion
@CodeOpinion Год назад
Generally the message won't be marked as process/completed and will go through a retry/backoff cycle or be sent to a dead-letter queue. All depends how you want to handle failed messages.
@megaman2016
@megaman2016 Год назад
@@CodeOpinion great
@megaman2016
@megaman2016 Год назад
@@CodeOpinion oh and what happens if your messaging queue goes down?
@CodeOpinion
@CodeOpinion Год назад
Use an outbox. ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-u8fOnxAxKHk.html
@CodeOpinion
@CodeOpinion Год назад
Use an outbox. ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-u8fOnxAxKHk.html
Далее
13 Карт - Клоны в супе | 3 серия
11:12
My TOP Patterns for Event Driven Architecture
10:45
Просмотров 30 тыс.
Event Sourcing Example & Explained in plain English
18:23
A Beginner's Guide to Event-Driven Architecture
37:28
Просмотров 4,5 тыс.
CQRS & Event Sourcing Code Walk-Through
14:35
Просмотров 41 тыс.
don't throw your faulty fan
0:24
Просмотров 2,5 млн
Для фанатов SEGA MEGADRIVE - Anbernic RG ARC
14:23