Is there a Limit to Number of Connections a Backend can handle?

Подписаться 412 тыс.

Просмотров 33 тыс.

50% 1

Someone asked me a question and I felt its interesting to make a video about, is there a limit to the maximum number of TCP connections a Client can make to the server? If there is what is it? and how does that make sense in all the configurations?
0:00 Intro
1:00 Is there a Max Connection Limit?
4:30 64K Connection Limit Explained
7:20 Max Connections on Reverse Proxies and Max Connections
14:30 How does Router get around Max Connections?
🏭 Backend Engineering Videos
• Backend Engineering (B...
💾 Database Engineering Videos
• Database Engineering
🛰 Network Engineering Videos
• Network Engineering
🏰 Load Balancing and Proxies Videos
• Proxies
🐘 Postgres Videos
• PostgresSQL
🚢Docker
• Docker
🧮 Programming Pattern Videos
• Programming Patterns
🛡 Web Security Videos
• Web Security
🦠 HTTP Videos
• HTTP
🐍 Python Videos
• Python by Example
🔆 Javascript Videos
• Javascript by Example
👾Discord Server / discord
Become a Member
/ @hnasr
Support me on PayPal
bit.ly/33ENps4
Become a Patreon
/ hnasr
Stay Awesome,
Hussein

Наука

Опубликовано:

1 июл 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 77

@BeALeaderAndAnInspir 3 года назад

Man, subscribing to your channel is one of the best decisions I have made in 2020. You have touched on the topics that a few people discussed in detail on RU-vid. Thanks, keep doing these great work.

@hnasr 3 года назад

I appreciate you Tien! So glad the content helps, let me know how I can do better

@syedaffan3071 2 года назад

Man I am just hooked to your channel! Can,t just have enought of it. Thanks for producing such quality content!

@ryan-bo2xi 3 года назад

Sir you just leveling up my confidence for life with your videos .. huge thanks ! Will surely applaud !

@sawza0313 2 месяца назад

I listen a lot of your explanations, and they help A LOT ! and I really really love how you use explain software engineering, and I really trust it! Some are new and some are like putting the missing puzzles at to place 🎉👏🏼👏🏼👏🏼👏🏼 Btw you look like a brother of mine, he’s civil engineer and very lever , like you 😊❤️

@youd87 3 года назад

One of the best channel RU-vid ever recommended to me. Thank you for all your efforts🙏

@hnasr 3 года назад

Thank you dear!!

@utikevin5987 3 года назад

Can you tell me other RU-vid channels that was recommended to you?...It may be helpful. Thanks

@aaronputra5578 3 года назад

If my parents let me choose between university or watching your RU-vid videos, I'm too dumb if not choosing your videos! I super love your contents, really. I love the way you share and teach me what are the problems, what are the fundamentals, how to solve them until you gave the solution. Thank you, man!

@DR-ee4wv 2 года назад

likes your all videos. A real backend engineer

@UgalabaguUgabagu 3 года назад

This video made life a bit simpler. Thanks!

@himanshutomar3512 3 года назад

Saw this video and listened to you, and immediately a like and a subscription 👍 You are awesome mate 🙂

@AllenLSY 3 года назад

We definitely like your videos. I have one suggestion: for the knowledges that can be better presented using diagrams or you need draw animations while expressing, get a ipad with apple pencil, draw on a sketch pad app, and record the screen. That way it's much more clear than waving your hands in the air.

@hnasr 3 года назад

Thanks for the tips! Appreciate you dear

@MehdiShojaei 3 года назад

Nice clarification, Thanks. Actually, the real problem comes from the maximum file descriptor (FD) limit, that must be configured for servers.

@hnasr 3 года назад

Mahdi Shojaei that is something I missed thanks for sharing!

@hackwithharsha5228 3 года назад

@@hnasr Thanks Hussien, Is it possible for you to explain file descriptor (FD).. I have done little bit of research.. not able to understand the context here.

@benjaminmella2407 2 года назад

Really good content, I appreciate your effort. Have a nice Day. Thanks

@sudharajamanickam7534 3 года назад

Thanks for the dedicated video on this topic..

@hnasr 3 года назад

Sudha Rajamanickam Thank you for watching!

@saimanojch 3 года назад

Thanks for vids dude, a correction though. At 13:50, neither adding more backends nor listening on multiple ports on the backend solve the issue that you are addressing (i.e the number of TCP connections that the reverse proxy can open with your backend) If the ports on the reverse proxy exhausted, if you add one more backed at this point of time, reverse proxy can't open an extra connection just because you added a backed, reason being ports are already exhausted. Even If the backend is configured to listen on multiple ports, it doesn't help reverse proxy to open one more connections once the ports are exhausted.🙂

@amlanbiswas4526 6 месяцев назад

Hey sorry @saimanojch, I am two years too late to answer this and since no one answered here, I am. @hnasr can correct me, if necessary. Each TCP/IP packet has basically four fields for addressing. These are: source_ip source_port destination_ip destination_port Now, when the number of backend servers increase, along with source_port, destination_ip changes too. So the TCP connection can be registered uniquely.

@familytamelo8140 Год назад

Thanks!

@namitpiriya 3 года назад

Great video. I am a beginner so didn't quite understand the last part. But rest was all good 😁. Keep making these videos great source of learning.

@ahmedtarek8792 3 года назад

Another great one!

@hnasr 3 года назад

Thank you Ahmed!

@Oswee 3 года назад

Hi. Idea for the topic. Build systems. Why do we need one. History. Shell, Ant, Maven, Buck, Bazel. When do you don't need any. When you could look into Bazel or Buck or any other. Personally i currently digging into Bazel. Seems quite powerful and mono-repo centric. Probably will try to adopt for my workflow as i am working on 7+ related projects simultaneously.

@anithaswaminathan996 2 года назад

You are awesome !!! Very interesting and you try to explain it in lame man terms - watching this gives the interest to learn more...

@ruhankhandakar 3 года назад

another great video.. thanx

@hnasr 3 года назад

Thanks Ruhan!

@Ooulu Год назад

This content is fire 🔥

@benjaminmella2407 2 года назад

Thanks for the explanation, I would like to know if there is a limit in a mysql database connections, I really appreciate your answer

@ishandhar2851 3 года назад

Hi Hussein, on the backend machine aren't we limited with file descriptor limits on the OS?

@dragon_warrior_ 2 года назад

Question: you said that once the reverse proxy hit the max limit it can't open more connection and to solve this we need more backend. Question is that if the reverse proxy is exhausted, then on connecting to the reverse proxy we will be waiting till a socket is open and if we are still waiting here on the front end of the backend that is the reverse proxy and haven't even reached to the backend, then how our problem is resolved by adding more backend?

@dexteraparicio6808 Год назад

With regards to limits at the backend, can you shed more info on what is the effect of /proc/sys/net/ipv4/ip_local_port_range? Thank you.

@ripudamank 3 года назад

Hundred things run in his mind when he says "when you make a request"

@vinny142 3 года назад

"One stiinki'n server" Not just "a" server, a *big* server. A *VERY* big server. Whatsapp is a bit of a utopian example because there are only a handfull of websites as big as whatsapp, the vast majority that us mere mortals have to deal with struggle to get 1Mln users, let alone 1M concurrent connections. About the proxies; a reverse proxy can ofcourse use as many internal IP's as it needs to, and each backend server can listen on as many internal IPs as it needs to, so there that 64k limit doesn't really exist there either. So a simple HAProxy setup would remove the TCP connections limit alltogether.

@hnasr 3 года назад

vinny142 that is a great point Vinny! So how would that work ? different network interfaces on different subnets? I haven’t played with this at all any resources? Cheers and thanks for those valuable info

@vinny142 3 года назад

@@hnasr No real resources off the top of my head but this support question does the same thing: discourse.haproxy.org/t/set-frontend-ip-as-source-ip-for-backend/320 What it boils down to is that you can instruct HAPRoxy to use a particular IP for outgoing traffic, so as long as that IP is defined on the NIC you can comunicate fro there to many backends and use all of the loadbalancer's IP's for all backends.Thre IP's = 60k connections,etc. Given that most hosted servers will have a private secondary network, and that IPv6 is getting more mainstream, this should solve the problem nicely.

@debugmedia 3 года назад

Is there a limit to in-memory caching? Because I have a websocket connection where I store names of the active users in an array and emit that array to whoever opens the website, So that way they can see who all arw online. But is that a good idea? Or should I use something like redis?

@hnasr 3 года назад

Debug Media that is a good question, if you exceed the memory the OS will take over and page excess memory to disk which can slow things down.. i would implement an LRU cache where you dumb all pages from the cache

@pulkitkedia8679 3 года назад

If we add multiple backend servers with only one Load balancer then also the same issue of 64K connections running out would arise right ? It is only with multiple load balancers we can increase the number of connections(more than 64k) right ?

@hnasr 3 года назад

Pulkit Kedia correct the load balancer can only open 64k connections in total from its side.you can avoid this limit by listening on multiple ip addresses on the load balancer. Its equivalent to installing multiple network cards on the load balancer machine so you get multiple nics and as a result multiple ip network addresses and you can listen to all of them and use them accordingly

@Flankymanga 3 года назад

13:59 Actually Hussein could another solution be to add more IPs to the single backend host? Like for example having a NIC with multiple IP addresses and backend is listening to both? While Proxy would have to have some load balancing functionality but the backend host will still be a single machine?

@hnasr 3 года назад

Flankymanga that is correct! The proxy can listen on multiple ip addresses .. i missed to talk about this , Someone else mentioned this in the comments

@sahukarinaveenkumar3188 3 года назад

Nice hair style 😊😊

@titidantete 2 года назад

ah... I thought 64k connection is limited to server, so basically if I run websocket server, even 200k connections from client connected to server then it's no problem right ? the problem might be if the client send more than 64k request at same time to server, is that correct ?

@StyleTrick 3 года назад

Great explanation. One question, how does a server establish a connection with a client? A server can handle millions of connections concurrently? After reading it seems like this optimisation is done at the kernel level?

@hnasr 3 года назад

thanks! the client is usually who initiate the connection not the server. I explain this here ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-bW_BILl7n0Y.html

@StyleTrick 3 года назад

@@hnasr I watched this, yes I understand the 3 way handshake and the client sends packets to the server in order to achieve this. My question was more around how the server can handle 3 million concurrent requests like WhatsApp. From what it seems, the server assembles packets together to form ONE TCP Connection and the amount of memory each connection takes is the bottleneck. Is the TCP connection ephemeral or reused in this case?

@pk-bn1hc 3 года назад

8:38 shouldn't it be layer 7 proxy/load balancer , bcoz as u have explained in your layer 4 7 lb video that layer 7 proxy initiates a new tcp connection with the backend server ?????pls explain ??

@hnasr 3 года назад

p k layer 4 and layer 7 proxy usually has two tcp connection. There is one special case of layer 4 proxy where it acts like a gateway (router) and in that case it is a single TCP connection

@hackwithharsha5228 3 года назад

@@hnasr I watched both your Layer 4 and Layer 7 videos to understand this point, Got it.. Thank you !!

@Flankymanga 3 года назад

6:08 it does make sense if you are doing DOS attack like slow lorris.

@hnasr 3 года назад

That is true, slow loris connections are long lived.

@hackwithharsha5228 3 года назад

@@hnasr As per wikipedia, Nginx and lighttpd server doesn't succumb to this attack. Not sure, how.. en.wikipedia.org/wiki/Slowloris_(computer_security)

@serhiihorun6298 3 года назад

I don’t understand who are those 5 guys who disliked your video. Seems some of your coworkers

@sharthakghosh970 3 года назад

How does client 2 open another set of 64k connections on the same server port ?

@hnasr 3 года назад

Sharthak Ghosh because client 2 has a different IP address and a whole set of a different local ports

@hnasr 3 года назад

This might help How Network Address Translation is used on Layer 4 Load Balancing and Port forwarding ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-RG97rvw1eUo.html

@sharthakghosh970 3 года назад

@@hnasr Makes sense

@athithyaraj1435 3 года назад

So at a moment a client can make 64k open connection with reverse proxy. What should be done to handle more than that with reverse proxy like nginx?. Clarify please

@hnasr 3 года назад

It is pretty bad idea to open 64k connections from the same client so I would want to know the use case (same IP address). I don't know anyone including big companies that hit that limit. But if you absolutely have to, nginx or any can listen to different IP addresses and put those behind a DNS as I explained in the video

@athithyaraj1435 3 года назад

@@hnasr Now I understand a bit more after your reply. From a single client , a reverse proxy can handle 64k connection which is more than required for any client server communication. Say , client A is engaged with reverse proxy with 64k connection (not a use case but curiosity) Now client B tries to connect with reverse proxy.. Question is : 1. What will happen if client A adds one more request to reverse proxy? Will it be in Q? 2.Say, multiple clients are having max 64k connection with reverse proxy.. can reverse proxy process a new client request without much performance issue?

@section9999 3 года назад

Hey man you mentioned that whatsapp uses some 3 million ip connections. Do you mind sharing where you get this information? It would be interestingbto see how some other companies are handling on thier backend.

@hnasr 3 года назад

Oh yeah sure I discuss this in length in the WhatsApp video here ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-vQ5o4wPvUXg.html you can find the resources in the description box

@section9999 3 года назад

@@hnasr cool man thanks so much! Your efforts are muuch appreciated. :)

@hnasr 3 года назад

❤️❤️

@pajeetsingh 3 года назад

Limit is 2^bits-used-to-represent-int -1

@barebears289 2 года назад

So, is there?

@rajeshkishore7171 3 года назад

sir layer 4 in your video ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-qYnA2DFEELw.html you talked about it wont terminate the connection , it will be the original tcp connection throughout but here I got confused as you mentioned layer 4 would terminate the connection and make a fresh connection to backend server?