This BIG Modeling Mistake Is Ruining Your Performance

Подписаться 51 тыс.

Просмотров 12 тыс.

50% 1

Get the source code: / amantinband
My courses on DomeTrain: dometrain.com/...
In todays video we'll talk about one of the biggest performance mistakes your app is likely making today. I've encountered this mistake in almost all applications I came across.
In this video we'll see what the problem is, how to fix it, and how significant it is in terms of performance.
Connect with me on 'em socials:
Twitter: / amantinband
LinkedIn: / amantinband
GitHub: github.com/ama...
Thanks for watching, don't forget to like & comment & subscribe! ❤️ 💻

Опубликовано:

27 сен 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 40

@pazzuto 8 месяцев назад

Most value from this video: "Create an object small enough that contains what's needed to make a business decision." I wrote that down! 😁

@yardeZ93 7 месяцев назад

1) You started displaying an implementation which is actually a good implementation, then you asked about the problem with it. There isn't any problem with it actually, at least without a context. What you provided is just an performance optimization. So you solution is actually not needed if the app isn't suffered from performance issues. 2) What about DDD, if your User class is an aggregate root, and you want to apply action on the reminders via different flow, you still need it fully in the User class, if not consistency can't be accepted. 3) Appreciate your videos, as always 🙌

@AndersBaumann 7 месяцев назад

I am complete agree on both points. If there is a performance problem then separate the write from the read. In general DDD and an ORM should only be used for writes. Use Dapper or full CQRS for reads.

@parlor3115 8 месяцев назад

Would it also be possible to have a reminderIdMap? I think this way, we'll be able to have a single source of truth for reminders on the user object.

@amantinband 8 месяцев назад

Yeah definitely

@quicoll14 7 месяцев назад

What if you need now to fetch the user with the Reminder object? Would you need to query it separately since your model does not have it together anymore? Or would you just use your DB models?

@EdwinMartin 8 месяцев назад

I like how you consistently write remidner 🙂

@amantinband 8 месяцев назад

😂

@georgehomorozeanu 8 месяцев назад

Hey Amichai, what drawing tool do you use? It’s not zoom it, right?

@amantinband 8 месяцев назад

Presentify (only available on MacOS though)

@Noceo 8 месяцев назад

Any particular reason why you don’t add a UserId to a reminder instead of the other way around?

@vivekkaushik9508 8 месяцев назад

User 'has' reminders not the other way around.

@Noceo 8 месяцев назад

@@vivekkaushik9508 I get that. Another way of saying that, would be that a reminder belongs to a user. Represented by a user id.

@pazzuto 8 месяцев назад

@@Noceo I guess he just didn't in this example, but yes, that's required, otherwise how would you retrieve the reminders for the user from the database.

@FFKangoroo 7 месяцев назад

@@pazzuto Hmm, to follow his approach through the reminderIds list. Which has some drawbacks. To store this User Object in a DocumentDB will probably provide issues sizewise as most have document size limits(see explanation further down.). In a relational DB it could be moddeled as a reference table (reminderId, userId) which would result in an N:M relation, which would probably not be what he wanted. So yes in his example he probably should have added the UserId to the Reminders except he wants to make them shareable. I personally think that what he describes is not a good solution, and he also pointed this out by himself with another example, look at users over time. The RemindersMap and the _remindersIds is still growing indefinitely the longer the app runs for a specific user. So when Fetching the User Object, it not only can get large in size but also the computation which is needed for the aggregation grows aswell. Which obviously can be mitigated by looking only into the future. But the future itself is indefinitely large. So size wise, and this is for what he claims to optimise, reminders are no good candidate to get fetched preemptively with the UserObject. Considering the possible absolute size they should be fetched/calculated at will, which will probably not throw OutOfMemory exception in some SuperUser Cases and be more responsive on startup, but which might be slower on per Operation basis, although this can then be optimized in the Storage Engine through proper indexing (baiscally what he tried to do in the application itself)

@syedjunaid7846 8 месяцев назад

Hi Amichai love your vids and your style of teaching! Could you share what drawing tool you use for the vids? Its like powertoys but its looks so good! :)👍

@amantinband 8 месяцев назад

Presentify (it's macOS only though)

@Fikusiklol 8 месяцев назад

Good point! But Im wondering to hear your opinion. What if you'd take DDD approach to mutate only 1 AR per transaction and eventual consistency is not an option at a first glance, aka 1) validate if you can add a reminder in the first place 2) create Reminder and publish an event 3) User picks it up and increments that count Would you lean towards idempotency to make it happen or something else?

@amantinband 8 месяцев назад

The single aggregate that should change in your example is the user. This is because the atomic decision is made by the user aggregate. Storing the reminder in the database can happen in a following transaction. Check out my clean architecture template, it implements exactly this using single AR per transaction and eventual consistency.

@Fikusiklol 8 месяцев назад

@@amantinband Just watched it, thanks. Cool stuff, but it really feels off for a User to produce another AR event. I was thinking more like User AR would validate and create Reminder AR using some internal static factory of Reminder class. So basically my thoughts are: 1) Why would a user produce another AR event? 2) What topic/queue would that event be published? Users or Reminders? Should be reminders, right? But again feels off to me. 3) What event type would Reminder class store in domainEvents? Its own or the one user created? 4) Would that Reminder's event even be published or not? 5) And if we add CQRS/ES on top of that it will become even more messy. Because initial event that would have been stored along with User AR would contain data that he (user AR) doesnt need at ALL to be recreated, but is needed for Reminder. Maybe because im not expereinced enough, thats why I asked. Thanks for the good content tho

@Linkario86 8 месяцев назад

Yeah our applications suffer terribly from this. It's old and the exuse of "historically grown" is used a lot. Data-Access is from hell and huge amounts of Data is joined and loaded many times. None of the original devs is around anymore and nobody really dares to touch it, and if you do, the superiors won't accept the change out of fear it breaks the app. It's a legit fear though, but it really does feel like we're stuck with this. Reason I wanna become an Architect is that I have seen terrible Applications in a whole line-up of companies I worked for, and since I can't trust others to make somewhat stable applications, I figured I'll do it myself. Long way to go, and a cargoship load to learn, but I'll get there and just keep going. It benefits me as a Developer either way already.

@nayanchoudhary4353 8 месяцев назад

Well, this makes sense for RDBMS. On the flip side, retrieval of data will require joins which can be slow. So, creation is fast, but retrieval is slow. 😊 In general, a computer spends 60% of it's time searching. So, retrieval should always be faster than creation.

@PeriMCS 7 месяцев назад

If reminders are stored inside User this is still join. This only complicates saving.

@vivekkaushik9508 8 месяцев назад

1st rule of programming - Never prematurely optimizer your code. 😅 I'll need another US for enhancement boss so that I can justify my man hours and charge you extra what could've been done in the first time only. 😂😂😂

@yardeZ93 7 месяцев назад

That's exactly what it told to my self when he asked about the problem with the initial code

@ibrahimkoz1983 7 месяцев назад

I think this is not very accurate. First, you violate liskov substition rule. AddReminder should accept only required fields, not the whole object. There is an atomic invariant between these objects, if you were to merge them into one document, then you wouldn't need to rely on a transaction primitive. Also, data access patterns should be noted.

@PeriMCS 7 месяцев назад

How about lazy loading? All Reminders would load only when you ask for them. And when you count it would count objects in dB without loading them. What I'm missing here is how you are going to save both User and Reminder now.

@kmcdo 8 месяцев назад

I just noticed that this guy Vims. (I just started learning, it’s one of my 2024 goals to really build that Vim muscle memory)

@amantinband 8 месяцев назад

Best investment you’ll ever make

@benatmartinez1365 8 месяцев назад

When did he use it?

@Vreth6 8 месяцев назад

He is using VsVim which is an extension to Vscode that lets you use Vim movements and commands.

@kmcdo 8 месяцев назад

@@benatmartinez1365 he’s using some Vim plugin in vscode I believe

@PeriMCS 7 месяцев назад

@@benatmartinez1365entire video. Vim plugin in VS code. There is Vim plugin for every IDE. I can't type anything without it.

@thibaultlesuisse8650 8 месяцев назад

Completely argee. That why I like to use my DbContext inside my Application layer instead of reusing the same repository and fetching way too much data. Using my DbContect directly in my application layer forces me to think about what data I really need for this exact business use case.

@Forshen 8 месяцев назад

Thank you so much! I see this so much as well. In my experience, I see a lot of project that use an ORM system do this. That just get the relation/child entities (so full select and maybe other child entities) and do a count or whatever b.s. performace hit with it. Instead just a simple query to the db. I always encourage to use the database because in most scenario's the database is a lot faster, but don't over use it like that. P.s. I am a query builder guy. I don't use EF core, i use sqlkata.

@fifty-plus 8 месяцев назад

I know that was just a trivial example but why have a public list of data on your record, that's also internal, and why not just use a check constraint so your application logic doesn't need to concern itself with transactional locking and concurrency? While the example was simple, I think it was too contrived to be realistically complex enough to illustrate a real scenario. I also wouldn't encourage using exceptions for control flow - it's not exceptional to consider the user has reached their daily threshold.

@amantinband 8 месяцев назад

Check out my videos on the topics you touched and you'll see I more than agree. This was just a trivial example to get the underlying idea across

@kevinlloyd9507 8 месяцев назад

How do you handle the argument that "You're duplicating data by storing the counts of reminders, when it can be derived from the data by querying the actual list of reminders. You should look to tune your database indexes instead."

@amantinband 8 месяцев назад

It all depends what you're optimizing for. Optimizing for database storage will lead to different decisions than optimizing for cost or performance