5. Scale from ZERO to MILLION Users in Detailed | System design interview: Scale to 1million users 

### **Introduction:** - This video is part of a series on system design concepts, focusing on scaling from 0 users to 1 million users. - Covers various concepts like sharding, horizontal and vertical scaling, load balancing, caching, and messaging queues. ### **Steps for Scaling:** 1. **Single Server:** - Basic setup with a single server for the application, database, and client. - Suitable for the initial stage with zero users. 2. **Application and Database Separation:** - Introduces a separate layer for the application server, handling business logic. - Database server handles data storage and retrieval. - Enables independent scaling of both application and database. 3. **Load Balancing and Multiple Application Servers:** - Introduces a load balancer to distribute incoming requests across multiple application servers. - Load balancer provides security and privacy. - Ensures efficient handling of increased traffic by distributing workload. 4. **Database Replication:** - Implements a master-slave configuration for the database. - Master database handles write operations, while slave databases handle read operations. - Improves performance and provides redundancy in case of database failure. 5. **Caching:** - Utilizes a caching layer to store frequently accessed data in memory. - Application server checks the cache first before accessing the database. - Reduces database load and improves response time. - Uses time-to-live (TTL) to manage cached data expiry. 6. **Content Delivery Network (CDN):** - Uses a distributed network of servers to cache static content closer to users. - Reduces latency and improves website performance for users worldwide. - Handles requests for static content like images, videos, and JavaScript files. - In case of cache miss, CDN first ask neighbour CDN for the data then from origin DB. 7. **Multiple Data Centers:** - Distributes the application and database across multiple data centers. - Reduces load on individual data centers and improves reliability. - Enables geographically distributed user access with minimal latency. - Load balancer distributes requests to different data centers based on user location. 8. **Messaging Queues:** - Uses messaging queues to handle asynchronous tasks like sending notifications or emails. - Decouples tasks from the main application flow. - Improves performance and reliability by handling high-volume tasks efficiently. - Utilizes messaging platforms like RabbitMQ or Kafka. 9. **Database Scaling:** - **Vertical Scaling:** Increase the capacity of existing database servers (CPU, RAM). This has limitations and eventually reaches a ceiling. - **Horizontal Scaling / Data Sharding :** Split the database into multiple servers or shards, distributing data across them. This offers better scalability. - Splits data across multiple databases or tables based on a specific key. - Can be implemented through vertical sharding (splitting columns) or horizontal sharding (splitting rows).

0:00 - Introduction 1:19 - Single Server 2:28 - Application & DB Separation 3:50 - Load Balancer & App Servers 6:25 - Database Replication 9:01 - Cache Integration 10:40 - CDN (Content Delivery Network) 18:32 - Data Center Scaling 21:44 - Messaging Queues 28:16 - Database Sharding 34:19 - Conclusion

Hey shrayansh these system design videos are in hindi, If you create them in enlgish by creating separate Playlist that would be great, It would be helpful to understand many people like me

I seen LLD & HLD in language but couldn't grab it.... because there directly they were discussing on the solution. But the way you went from 0 -> N that was amazing. I worked in monolith application but couldn't understand how to approach to the best change..... Hats of to you....your teaching method.... More Power to you !!!

Honestly, this is one of the fine grained system design lectures I have ever seen! I paid for udemy courses, looked for freecodecamp videos, medium blogs, but this is the finest of all gems! Thanks a lot! You deserve a lot lot lot!

That is really cool to see such incremental approach. Appreciate this video. Could you please also make an extension of this video mentioning, Elastic Search Engine.

It's a great learning video. Thank you. I have seen in interviews and in other HLDs that they add queue to store requests when load increased too much, so that any request do not get missed. Could you please make a video on this concept explaining how this is done (whether the request thread wait on queue) and how response is send back in detail.

0:00 - Introduction 1:19 - Single Server 2:28 - Application & DB Separation 3:50 - Load Balancer & App Servers 6:25 - Database Replication 9:01 - Cache Integration 10:40 - CDN (Content Delivery Network) 18:32 - Data Center Scaling 21:44 - Messaging Queues 28:16 - Database Sharding 34:19 - Conclusion

Can you please make a separate detailed video on Messaging Queue? It will be really helpful.

Summary: 04:25 Scaling from 0 to 1 million users 08:50 Implementing Load Balancer 10:00 Database Replication to handle system failures 13:15 Using cache and CDN to improve performance and reduce database calls 17:40 CDN is a solution to reduce latency for global users 22:05 CDN enables advanced technology with features to optimize load on DB and increase security. 26:30 Messaging queue helps in asynchronous processing of heavy load operations. 30:55 Fanout, Topic, and Messaging Queue are important concepts in message exchange 35:14 Horizontal and vertical sharding are methods of dividing data in a scalable system.

Even in udemy also I observed that HLD & LLD courses are in hindi bro, please do start converting one by one video from hindi to english

I think what you mentioned about sharding here is not actually sharding its partitioning, multiple partitions can reside in the same shard also. Sharding is always on db and partitioning is on data content (rows and columns). Different instances of db deployed are called shards. CMIIW.

HERE WE CAN USE S3 Bucket as well to store uploaded data like images file etc etc

Amazing content☺ on LLD and HLD. I was finding difficulty in answering about the design patterns in the interview, but the roadmap of LLD and HLD explained here really helped.

CDN does not call to other cdns ..(neighbour cdn) a cdn service have internal edges / nodes seperated region wise 18:25

The CDN will be outside the datacenter right? then how will it scale for different geographical regions?

Hi Shrayansh , i had doubt in last part where initially we had master slave architecture , later on database scaling are we scaling only the master node or all the master slave both ? Thanks

Hello Question: How do data centre communicate with each other and avoid consistency in a system? Lets say at time t1 a write req goes to India DC(data centre), it then goes to US DC for replication. If this is a sync communication, user's write will get latency delay as sending ack back to user will take a while. To avoid latency, we choose an async comm btw data centres and series of events happened: 1. Since using async medium, Queue has data for US DC to be replicated. 2. Ind DC goes down 3. At time t2, t2 > t1, read req(read1) comes for USDC. At time t2, data is not replicated to US DC yet. 4. Say (read1) need to serve a heavy query result and supposedly we will have a cache miss. How to handle such case? PS: We can have multiple DATA CENTREs, but the crux here is read comes before replication to other data servers and primary/master(the server with latest write) goes down.

Can anyone Share notes with me for current and last 2 lectures.

in love with all the videos !! curiously waiting for the consistent hashing topic , please drop it soon 🙂

I don't know why the colleges don't take these topics for class.

i struggled... were you speaking hindu?

Bro, when can we expect English subtitles to this video?

It would be helpful if you provide some subtitles brother

What is the difference between master slave db concept and cqrs ?

Amazing flow of the video. Thanks :)

Nice explanation, Thank you!

Why dint you discuss threadpools in this discussion

Whats the use of vertical sharding, it is just splitting the table on the columns, but the number of rows remain same, which kind of means we are creating multiple tables from one table, doesn't that make the design even more complex?

Great content, keep it up You have earned a new subscriber

The best video :) Thanks Shreyansh

Please post videos in English

Maja aa gya , Love this video (y). Thank you so much

Very well explained...awsome!!!

Thanks for such amazing content bro. It would be great if you could please share resources like Books, blogs etc from where you learned all this , so that if someone wants they can go into greater details!!

so you are saying first step should be replication than vertical scaling .......... but I thinks it should be reverse.......

I was from an electrical branch, SO I didn't do that project, nor having learned of any computer science project. But now I am gaining it

Hey @ConceptandCoding, How consistent hashing will help to reduce the tree structured sharding, I mean every shard has different data, redirecting the request using consistent hashing to a wrong shard won't help right?

do you refer to alex xu?

Bhai you are working great. I must tell you. I always recommnd your channel to the people who wants to learn Lld and DP. One note : when you are recording DP comes up with real time use cases . And where can we implement these DP concept in the hld questions asked in interview. Like can we implement any dp in parking lot.

why there is a need of join query in sharding ? we have same columns in every tables , so why we have to do join ? we can simply make query on respective shards ?

Great Explanation

Please make a separate video on database scaling .

Question - Step 4, How does the requests from App servers farmed out to multiple dbs, do we need internal load balancer between App and db servers? Appreciate it if you could add some explanation notes. Great videos, love it.

Amazing content + highly recommended to watch it.

Very nice tutorial. Provided detailed information in very short time. thank you

Nice Video to get a summary of all concepts

Really good explanation. Thank you

Beautiful explanation. Thank you!

Liked the step-by-step scaling explanation. Pls make more such videos so that we can find them all here.

Hey , Thanks for a great content men. I have one doubt . I have few doubt thought. 1 - Scaling of db and scaling of app server is same right ? . I mean for both we use Horizontal scaling and Vertical scaling right ? only difference is implantation wise . 2 - Do you think Master slave is kind of horizontal scaling only bcz there we have replication of data(Slaves have same data as Master) and in case of app server we have replication of code on server and multiple servers are running them ? 3 - If we have master slave then what is the need of the Sharding (as it is also kind of horizontal scaling only) 4 - Why don't we have Load balancers in front of the Slaves in master slaves like we have in front of the App server ?

Hello Sir, I want to present my idea in Tata Imagination Challenge. Can you please guide me on the same? If possible, please make a video so that it will be helpful for all of us. Thankyou.

You are the best Tech Proffesional.

English would be very helpful

Thank you, really helpful video.

How to achieve Master slave replication in real time?

Question: Suppose if there is huge data in DB and there needs to be divided into two shards, Will both the shards have their dedicated node, or both the shards will live in single node ? Also can a single node/vm contains only one shard or it can contain multiple shards

agar hum sharding primary key ke base pr kre to jab bhi tree sharding problem hat jaegi ?

So clean and sophisticated explaination

I had a question, at your level like sde3 do you also have to work on frontend side also or more on backend architecture only

👍👍

Can you please make this video in English if you have free time?

Please speak in English or at least add english subtitles

Thanks for the great video, whats difference between Horizontal Sharding and Indexing?

Great video 😇.Hope there would have been some predefined steps to reach 1 million followers/subscribers 🥺

One of the best video on step-wise scaling!

I am watching this from New York but There are so many ads after every few minutes. Its so so frustrating. idk its by the author or location but it needs to be fixed. Content is good as always.

Heyy, can you please explain how data will be replicated between databases situated in different geo. location; like what is the connection type between them.

There is one concept of Service Discovery also. This should have also been added into the diagrams.

Beautiful explanation

crystal clear explanation

Nice work buddy. Can you help me with these doubts (as soon as you can, I have an interview scheduled, thanks in advance)::::> Suppose If I have a trading application, and want to reduce the latency for geographical users, lets say a user is in US and other is in Singapore(APAC), and third in london(EMEA), so I scale my appservers, make application replicas and place them in different geographies, so 1) where will I place the gateway? Gateway is generally 1 right? 2) For applications to interact with DB, where to place the db (will it be better to make distributed db replicas placed across geographies )? 3) what type of db should I use who can sync ups b/w replicas? (looks like consistency will take a hit since db sync up will take time) 4) How to handle concurrent orders( trade order may have 1 unit left, and all users are trying to book that 1 unit, db isolation may not work here bcos they are reading different replicas OR use active passive strategy where write is always on primary db, in that case where to put primary db as far uaway users will again face latency in write operatons)

Consecutive write and read operation for same data will result in failure if read is from replica even if replication gap is 3-5milli second

1)Who will take care of making slave DB master if primary fails..? Is that db cluster itself does the job? I believe replication is also taken care by db cluster by itself. 2)In general we give 1 url in datasource bean of springboot property file for a relational db. What would be URL in springboot application if there are multiple replicas dbs and how can we change primary in that case?

Keep up the great work:)

Hi can you pls share these kind of notes :) (onenote with us)

Great content! It would be helpful if you could include any references you used, such as articles, blogs, or books in descriptions

Beautifully explained words aren't enough for your effort taken, thanks! Also i have doubt pls reply, i wanted to know how long does it take for a single software engineer to implement HLD all these mentioned concepts in real time, how long does it take to scale an app? Lets assume it's a food delivery app

Does CDN have server instance or only static data ? where does the client request goes first ? - CDN or actual server

A humble request to speak in English

Great

Very Nice !!

Pls also start videos of spring boot framework.

Super

Best video I have ever seen. Thanks so much

Thank you for your service. Great job.

Where to learn LLD HLD from ? and how to scale it to millions of users

Great content 👍👍

good job young man

Started HLD today. Hope to finish hld and lld in a week. Wish me luck

This stuff is really really cool, and Thank you so so much for sharing your knowledge. Appreciated. :)