if you read a little bit into the erlang language and erlang VM, you'll notice that it's specifically designed to scale paralelly. The Erlang VM is kinda like a supervisor managing thousands or millions of erlang-processes, which are kinda like mini-threads. If a single erlang process dies (all these erlang processes can run in 1 server-process, or across multiple processors/cores) the vm will notice and re-start that single instance. Erlang is designed with a let-it-crash philosophy, allowing processes to crash instead of handing weird edge cases with complex exception handling, and by just restarting the mini-process getting to a known safe state as quickly as posible.
Yo Hussein just want to say your content is really great man. As an early-stage/junior engineer these are the kind of conversations/lessons that you're lucky if you get to have once a week with veteran engineers. And you wear your expertise very humbly. An example to follow for sure. All the best!
well said man, I am also still a junior myself only having being in this field for 2 years, and these sort of "talks" by Hussein is gold for me. Like the technologies, anyone can easily learn and implement, but like those subtle things that hussein include in his videos can only come from being a veteran engineer.
Hi Hussein. I loved your db udemy course. Is it possible for you to make one on networking? I like your youtube videos but find a code little more coherent.
It's just my guess! may be they are using one connect feature in f5 Load balancer(connection aggregation) to speed up connection. But architecture sounds like very very well designed. Can't imagine amount of effort done by architect.
Dumb questions alert: 1. Is Linux Kernel's stock TCP stack that scalable or do they have their own customizations to it ? 2. Given that everything is E2E encrypted in whatsapp. I am surprised why CPU consumption is too low at that scale. Is it possible?
In the blog post linked, they show that they were using FreeBSD, not Linux. This post pre-dates E2E on WhatsApp. CPU usage should not matter for this anyway as the server can't decrypt the messages
Taking a page from storage systems, they might have servers running custom application kernel module which dedicated kernel threads mapping TCP data to user mapped memory. Every processing would be done in user space without a kernel context switch. Storage systems achieve really high performance using this approach.
Very close to what is actually, done. I was surprised by the video and how utterly the guy fails at how it could work and where the bottlenecks lie considering he has been working in the field for over a decade. Still respect the fact atleast he is asking interesting stuff. (Maybe I am extra salty as a company threw me out of interview/selection back in my college days because I tried to explain them how a single machine could handle 10 Million+ connections concurrently...) Btw this is 2022, and Epyc CPUs with 96+ cores exist and RAM capacity upto 12TB. And 100Gbe Nics are dime a dozen, my consumer grade system has 3 PCIE 4.0 ports to make that a possibility times 3. It may just do 25-50M concurrent connections these days with ease. Now let's discuss how, as I already displayed hardware is more than powerful enough. 1. The bottleneck is the kernel as you already have noted.(I think) 2. So just move to user space, and this is done by having a user space data plane that allows efficient and robust alternative to Linux networking stack(which is pretty shit at scale). 3. The most popular package is DPDK for it.(made by intel) 4. Next step is to get a network stack, they are dime a dozen as well all based on DPDK. Checkout f-stack made by Tencent peeps and it allowed 10M connections on Xeons back in 2013-15. 5. Last step your server, modern servers allows async and multi core programming and can easily handle millions of connections. I recommend Nginx. Apache is shit btw. As for storage peeps yeah they do love their user space dma buffers, tbh don't we all, I remember doing something similar for streaming in a game I was trying to make because I wanted to load a huge file(scenes/levels store as binary resource) as fast as possible. Fun fact: I have seen people use implementations for faster memory mapped io paths than what mmap and Linux kernel provide. Generally though the key idea is DMA and DMC with data intensive situations, for which Linux mmap is sufficient.
At 1:19 You said, in the middle east every single person uses WhatsApp - This statement looks factually very much incorrect because as per my knowledge, in the Middle East, especially UAE, WhatsApp, Slack, Discord and many such commonly used communication channel is banned!
Here is this blog that talks about actual software engineering stuff. Then there are others that just like to talk how coders are supposed to be and how coding is cool, their pride with being a supposed software engineer, showing off their offers and salaries from FAANG.Yeah, basically all BS except the work itself.
@@hnasr Here's what I made of your reply. They have multiple network interfaces installed on a server. Each one is listening to maximum possible number of ports. A server can listen to any number of client connection requests on a port as long as client IP/Port are unique. Is this understanding correct?
@@debkr yes, he's both technically sound and funny. I am also an instructor and a content creator and I would love to incorporate his creative style into my work.
wesome video. You're the best Hassein! Long time fan. Congrats on discovering OBS!! I remember when I made the switch. I played around for hours and hours. You should make a video on how it works: I'm convinced it must implement some level of virtualization, along with very ground-up methods of encoding and Interaction with the OS. It seems to act independently from the influence of other applications in a way? I'd be interested in what you come up with regarding the ins and outs of its design and what seems to be unique abilities. Great job as always.
Love you Hussein! Thanks for your videos, maybe you won't be ex tech lead slash millionaire but you bring a lot of knowledge to us and help tons of people to understand a tech. See you in the next video comment section!
I somehow tumbled upon one of your video. Boom, I subscribed and I became a fan of your content. I keep playing all your videos and learning a lot from those. Regarding this video I have a noob question. How do we even decide or come up with number of initial machines(I know we can scale up horizontal in later stage) we should use during our design phase?
Yo man, a friendly suggestion, please take it positively because I like some of the topics you talk about. Please stay on topic and narrate smoothly. Even 1.5x speed doesnt help to ignore you wavering onto irrelevant words/topics/humour/pauses. I still give this video a thumbsup. Thanks.
@@imacprousersam7306 man, why you always talk this way.. Just gratitude for this man. Then move on and do your own research. Erlang is pretty old shit, around for more than your age.
Yeah microservices are fucked up thing. But it's more fucked up when you create microservices that communicate via RPC and it becomes microlith instead of good decoupled microservices :)
You're good at this, I randomly found your channel when I was looking for an nginx tutorial but then subscribed as well, because, due to the quality, I expected your channel to be bigger than it currently is :)
This makes me happy since I do elixir which is build on top of erlang and has similar capabilities. The posibilities are just unlimited when you have such a powerfull langugae. Concurrent and parallel programming is very easy with functional languages like elixir/erlang since dnt have any shared state.
It is a stateful and long-lived, once the connections are created they are there. And the maximum number of connections are 3 million. We are not talking about requests which will be described as you said.
I might be wrong, but they can have 2 connections from frontend, both http for backend and Sockets tcp for backend, and then there would be a load balancer ip pointed to frontend, and that loadbalancer will be linked with all the servers backend that will redirect users to their local servers and also based on connection type. Also, they can also have multiple ports in one server as you says
Another nice video man.. I think you have been using the terms socket and connections interchangeably.I think normally server listens on a single socket but have multiple connections.You can be limited on socket by no of ports available,connections are more of property of cpus and memory and others
Siddiqui Sarik thanks for correcting me Siddiqui! Appreciate it. Correct, server listens on a socket on a port e.g 80. And connections to port 80 Are uniquely identified by source port/source ip
Hey, I'm relatively new to computer science. I was wondering how does a server support more than 65535 simultaneous connections? Maybe I am misinterpreting this.
@@jayjayma 127.0.0.0/8 is a full class A block so in addition to 127.0.0.1 you can assign 127.0.0.2, 127.0.0.3... to your local interface. (Up to over 16 million addresses if you need/want.) 😁
Do you even understand technology. Of course they are using a single port. When you don't understand technology why mislead young engineering. At the beginning you said probably they use some native protocol then you are talking about Layer 7 networking. Do you know your videos is just filled with just jargons. Please talk constructive. I am sure you don't understand technology.
Seriously too much bullshit. Don't do this please. Please i am requesting you. Probably every tech guy in the world knows it is based on Jabber XMPP protocol. Later yes they did a lot of tweaking. Either talk about their core architecture or talk about scalable network architecture. You have no idea what are you talking about.
Nice content once again. you were discussing most of the time. I felt, the actions which you make to explain was missed(Since, the screen was smaller). But, loved these discussions. Good to know about the design perspective. Always keep making videos. Thanks Hussein
