Sometimes a user requests that all the data be deleted permanently, you can't just make another entry elsewhere that they requested to be deleted and capture data on that event because you will be violating their right to be forgotten. Sometime a delete is just a delete.
right-to-be-forgotten requests can be incorporated into the design of ledger based systems with appropriate encryption and a forgettable key per customer.
@@amlyo6722 I mean, he isn't exactly wrong. Forgetting an encryption key simply means it's "forgotten for the foreseeable future", not actually "forgotten". Someone 10 years from now could obtain a leaked dump of your database and decrypt it using, say, a quantum computer. It's not exactly a clear-cut issue.
A lot of your videos hit on a very important topic without actually explicitly getting into it. It's very important to have a competent product owner who can focus on delivering their team with meaningful acceptance criteria for actual business cases instead of things that look like CRUD operations. This is often a challenge with businesses that do a poor job aligning teams with business functionality and not technical considerations.
I can 100% support this mind set. I’ve worked in large scale and super small projects and in my experience I’ve started evolving away from a soft delete an instead use some kind of enumeration or status code id usually stored in a database of some sort to define that current status of data. I think a lot of the frustration caused here is generally because these type of thinking does require user interfaces and queries to be a bit smarter than we might like but I’ve found it’s usually worth it.
One notable exception: if your business wants to comply with European privacy laws and receives a request to delete PII for an individual, then deletion truly is in line with a business process.
Even when from a business standpoint you don't think of hard deletes, regulations like the GDPR and right to be forgotten laws would still requires systems to be able to hard delete. How do you handle such regulatory requirements?
Thanks for the video! Choosing between event sourcing and crud seems to be a difficult job. How should I know when to use which? And later down the line, when I realize that I should’ve gone with the other method, how should I make the change? Changing from event sourcing to crud seems pretty simple, as I already have all the data. But how should I change from crud to event sourcing? Or should I just event source everything to be safe?
It's not about CRUD vs Event Sourcing. You could be recording current state in the way I showed it with a document and having a collection that recorded when they were hired and terminated.
Expanding on the subject of soft-delete, do you believe that adding "created by", "created at", "updated by", "updated at", "deleted by" and "deleted at" columns to your database is redundant when you can grab that information from the event store?
Depends how those columns are being used, I suspect just for information/query/view purposes. If you're in CRUD mode, then yes. If you're not then you probably don't have those columns but more-so like a CancelledDate, EmploymentTerminatedDate, etc.
Very easy situation to get into. The most annoying repercussion I've come across is having to omit IsDeleted entities throughout the entire application, including navigation properties. If you're selecting from the db at the root entity level, it's very difficult to deal with in a large app.
Nah... you should cover GDPR where users has rights to wipe out their data. So you should wipe out all sensitive marked data. You can't keep it in original form.
Correct. This video has nothing to do with GDPR. There are many ways of handling, one of which is deleting data. If you need to delete data because of GDPR, sure. You could also encrypt data and lose the key. The point of the video is to illustrate to capture business concepts explicitly.
In that case could you just anonymize any identifiable fields in existing data (name, address, email etc.)? Would this satisfy GDPR whilst preserving most of the events?
Soft deletes are the bane of my existence. Got a system I’ve inherited where the soft delete flag means something different depending on the entity it’s on. And query/report performance is crap because it’s got to be included in every one
Yup! It's not really "deleted" it has some explicit meaning that is different in a different entity/context but it's implied with isDeleted. Not fun. Make the implicit explicit.
It would be helpful if you describe what the database schema and database changes would be for the evented version. It’s easy to see how soft delete works because you showed the schema with the IsDeleted column and how/when it is updated. But it’s not clear on what the db schema is and how data is managed in the evented version.
If your were event sourcing, the individual events define their own schema. All the events for a particular aggregate are persisted in a stream. This video might be helpful if you haven't watched already. ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-AUj4M-st3ic.html
And then GDPR became a thing for many of us, so we cannot just keep data around indefinitely. Eventually you will have to hard delete if it applies to your business.
So you would add data with more information (i.e. EmployeeTermination and EmployeeHiring) instead of toggling an isDeleted flag because the facts of what happened might be important.
Another interesting topic: does doing a hard delete, cascade delete other data? I believe cascade delete is evil; i think that if you want to delete a user in the code, you need to delete relevant data with code as well That way you do not need delete too much data. And if you dont delete all relevant data, you get an error when deleting the user
How do you handle mis-keyed data in a task based system? Let's say an employee moved, and you go through the workflow of entering a new address. You click save, the system kicks off an employee moved event and everything is great. Then you noticed you miskeyed their street name. Do you have another employee moved event? They didn't really move twice. Do you have an AddressCorrected event? At that point do you have to support task based AND crud based events for every piece of data?
The employee didn't move twice, but they also didn't move to the first address. I'd rather just create a second move and maybe add a field "reason" for moving where I can say "previous move error" under EmployeeAddedAddress event. Although I'm not against deleting an address, I must confess I'd stress out about the address use dependencies elsewhere. I'm just thinking off the top of my head. I don't really have an implementation example come to mind where I had to address this before
"Why store all these events and data taking up space?" - Because of auditing. "But I don't need that" - So be it, this is just a concern that might become important to your business later down the road. You need to know that this is a way of thinking about your system. And you don't have to go all in on event sourcing for this either.
That's was the same question I had! For example we do have accounts where all personal data has to be wiped or scrambled when they want to delete their account because of the GDPR. (So the other way around, we must delete the data by law)
The intent of the video is to illustrate that often there are business concepts to capture that soft deleting doesn't provide. Capture business concepts if applicable. If you need to delete data, delete it.
Then delete the data. The point of the video really is to capture business concepts explicitly when traditionally you would think about deleting or soft deleting data.
yean totally agree that events are more informative than crud, but I feel like the storage of events could be challenging. do you have any advice on that? let's say one entity is a row in the sql db, if its crud, no matter how many events happened in the domain, cuuud ends up with d, one row less for the table, however, it would be five events to stored if we do event sourcing.
You don't need to do event sourcing to capture the business concept. As I illustrated with the document example that's recording current state (not event sourcing) I had a collection that recorded the hiring and termination. If you were using tables, you'd probably have table with columns for HireDate, TerminationDate, etc. With a single record for their employment.
@@CodeOpinion makes total sense, I got it, so a row with termination date being empty/default value when created. Then update to fulfill that information when the corresponding event happened.
This specific example is unlikely to scale as I think a business is unlikely to hire and fire one person thousands of times, so just shoving the data in "EmployeeHistory" would probably suffice. That would apply whether you are using events/pub/sub or just an API/ORM, and for RDBMS or a NoSQL store. It is a concern if you are talking about something with a much higher cardinality. Say like clock in/out events for laborers. You might have several per day (breaks, etc) per employee, seeral thousands per year, and then it grows forever as long as someone is employed and working. You have to question then what the business case to read the data back out later is and how you will access it. It may make sense that data is stored by PayPeriod + EmployeeId, which roughly fixes your cardinality to maybe a few dozen or couple hundred no matter how many employees you have and how long they work at the company. To some extent, you could say Derek seems to be putting the cart in front of the horse, he's coming as a developer in a video presumably focused on providing information to other developers, but business should be the main driver to store this extra tabular dimension (history) instead of just a scalar or two (i.e. current employment status or hire/fire dates) It's a good thing to bring up from the dev side, though. There are a lot of really good talking points here, and knowing the difference between the scalar and tabular or event aggregation is good knowledge to have kicking around in your head. It's this sort of knowledge that separates someone who blindly completes a user story and someone who can help steer their business away from information black holes.
What if you Aggregate Root has a collection of items, and then somewhere in your code you write AggregateRoot.Collection.Remove(item)? In my case the collection is lets say an Assessment and the collection is Reviewers. But I want to remove one of the Reviewers of the Assessment. Lets say I am using EF and SQL database, What would be the best way to handle that?
I can think of several occasions where you should actually hard delete things. The first one that comes to mind is: If I click "Terminate My Account" on facebook for instance, Facebook should hard delete all of the media that I uploaded there. Now the question is: Will facebook actually delete it? I don't know. But should facebook hard delete it? Yes.
Just as a complement. I understand that it could exists some information that should be anonymized but still kept for integrity of references to some other data. But the majority of personal data should be hard deleted.
What about it? If you need to delete data, delete it. The point of the video is to capture business concepts if applicable. If you need to delete data, go ahead.
Doesn't this make your life as a developer much more difficult? For example we use soft delete, and in our EF database context we filter out based on IsDeleted == false as the default. Only when we explicitly requested deleted data is it returned from the API. This means we don't have to think about deleted data. If we went with your approach, we would have to have seperate logic for filtering out deleted records for every entity.
Absolutely would make your life more difficult if your focusing on crud. If your viewing things based on business concepts it's not difficult at all since the concept of deleting likely doesn't exist.
Well the point I'm trying to make is the word "delete" makes no sense. You're not deleting anything, you're capturing data related to a business concept (hiring/terminating).
Answering the question of whether to hard delete vs soft delete with the the notion of "adding" things doesn't make any sense. It still doesn't answer the question on whether you should use one or the other because the it's irrelevant to the question, with no alternative provided, and assumes extremely contextual business application requirements that do not apply universally! There's lots of cases where recording a chronological order of events is not the right solution. Not a great answer to the question IMHO.
You answered it yourself, its "extremely contextual". There's no single answer to the question. The point with the adding is there are more options beyond the soft/hard delete.
@@CodeOpinion That was my point, although I was thinking this video would be about hard and soft delete exclusively and when to choose between one or the other based on the title.
@CodeOpinion I've read the comments :D Please stop explaining purpose of this video to everyone. It's like riding a bicycle with inverted handlebars - your brain needs to adopt first.
@@CodeOpinion I disagree. You did perfect job! I've been watching your previouse videos when I was learning DDD approach and I had same problems - i was asking question from my limited context. It is so clear for me now and even funny :D
