This section is present in the book designing data intensive applications. I highly recommend everyone to read it. There is lots of other concepts there also like how db handles transactions and instant backups etc.
Nice! So database system will take care of copying data to slaves, it is matter of configuration on database system level....what about routing read n writing request to the respective db node,would that b implemented on the application level via some gateway to filter n send to respective db i.e., read or write???eagerly curious :-))
I think inspiration for this video is from this book -> Designing Data Intensive Applications by Martin Klepman. Amazing book, I am halfway through it.
@@praveenkurapati691 I don't have a hardcopy, I read from a pdf available online. The benefit of that pdf is that you can take notes on a doc from pdf.
Good video for basic understanding but still lot of things to discuss. In real world, we would have a distributed environment . That means there would be multiple clients writing to multiple db nodes. We need to handle concurrent updates in such scenarios. We would also need a conflict resolving mechanism like vector clocks. Also writing to database itself involves multiple step. We write to commit logs, then memtables(inside ram) and then flush it to sstables (append only logs on disk). Compaction is a big topic and there are multiple ways of compacting data. For deletes, most dbs use a construct called tombstone.
I have a doubt. @18:00 in this video, we are telling if Active File size exceeds 100 MB we will create chunks and compaction will happen on chunks. But if in compaction Active File does not participate, then its could happen reader will not get data until Chunks does not gets created out of Active file. Am I missing anything ?
one question - how is this in-memory cache index built, what if machine restarted ? Or it's database engine responsibility to build this in memory DS when machine starts (more precisely when data base services in that machine starts)
Hey rachit, i am a 2021 passout currently working as a Software tester. I want to switch to SDE. Considering it would take me around next 6 months to be good at DSA , that would sum up 1 YOE as a tester. Would that create a problem for me in getting interview call for SDE profile? Or any other suggestion from your side about what should i do now? Thanks