What is NUMA? 

"If anybody is gonna make Non-uniform Memory access exciting, I'm your guy" earned my like right then and there man...

Ma-ia-hii Ma-ia-huu Ma-ia-hoo Ma-ia-haa

Content found nowhere else! Thank you! :-)

*Wendel ECC-ing (Error Correcting) the crap out of himself in this video :D

Anyone else getting stuck on 2:54 no matter what?

How to make NUMA fun, step 1: be Wendell, step 2: you're already done

Having a NUMA system requires a bit of management to get the best performance, but once you get used to it, the pros outweigh the hassles.

Probably the few but I love watching these informative videos.

this is the first channel i consciously subscribed. it looks to me you know what your talking about and you produce serious videos with real info. well done.

Love it! This is high quality nerd content. And you can tell Wendell means business because of the awesome shirt.

Numa? Is it as bad as Ligma?

For the hiccup at 2:54 ----------- > Just change the quality of the stream from whatever you're running it at in the middle of the hitch. It will push through.

These kind of deep insights are the reason I am a fan of this channel

The R710 seems to be THE thing for home labs and learning, I see it in almost every server-related video on RU-vid. :)

I get it when Wendell explains NUMA latency using the old x2 socket Xeon server with 2 physical CPU slots and 2 separate I/O riser slots, 2 RAM banks slapped together with that Intel QuickPath, because when cpu1 has to ask cpu2 for some I/O or memory request because maybe the stuff it needs in on the other I/O riser/RAM bank, but I want to know would ANY latency be present on an x399 system if there is only ONE I/O (1 GPU on PCI-E x16 slot) connected directly to TR CPU and only ONE of those RAM banks were populated? TR then only has one of those on the motherboard it can use when running a game for example If using 2 GPU slots and all RAM slots were populated it is easy to understand why then latency would be introduced. Great video Wendell! You made NUMA easy to understand. I wish you would soon make a video about how to properly assign AMD Ryzen/TR CPU cores to a Virtual Machines in VMware Workstation and Hyper-V when virtualizing stuff for most optimized VM experience. EDIT: NVM I watched the video till the end :P seems it does not matter how many slots are populated if you do not limit the app or game to use specific cores on specific CCX.

NUMA NUMA node, NUMA NUMA NUMA node

love the graphics on the TV. but really thank you so much for the forum and this great videos.

Thanks for this video. I learned something today.

Yes, teach us Sensei. Class is in session.

off topic but i just installed a new ssd into my laptop (that was suppose to go into my future build but was just collecting dust - and yes after installing chrome i watched an level1tech vid ^^) but im amazed at how in many different aspects an ssd has given light to this dying laptop... no more stuttery video playback is the biggest change,

Wendell, You make superb videos! Keep up the great work, Sir!

This is an excellent video. Thanks for taking the time to properly explain NUMA.

Man, you’ve managed to clarify to the rest of us such a complex topic... you are a champion Wendell! Love your channel buddy, keep posting Threadripper stuff (maybe a comparison between 2990wx and 2790wx once released in Oct) 👍

Super informative video! glad I took the time to watch

Awesome timing. I was re-watching your Threadripper livestream from January regarding Looking Glass / VFIO, as I'm in the process of setting up such a system on my dual Xeon E5-2687W (very decent workstation chips, 2012). I'm hesitating between Arch and Fedora for the host OS... #nerdproblems

Very informative! thanks. Had some giggles along the way.

UMA mode doesn't just lie. It lies AND it also shuffles the memory so (for example), each block of 256 bytes alternates between NUMA nodes. This increases the chances that general memory allocations by the OS will cover both nodes and spread the memory load out. Some will have higher latencies, some will have lower latencies. On average it works out pretty well because latency is just one of several stall conditions that can slow a CPU down. The mechanism that makes UMA work reasonably well is this shuffling. Basically, one of the low address bits is exchanged with a high address bit in the physical address space. This shuffling appears to require symmetric NUMA. I have never been able to enable any sort of UMA mode on my 2990WX, and I'm guessing the reason is that the 2990WX has assymetric NUMA, where two of the four CPU dies have no memory connected to them at all. I guess the address bit shuffling hardware doesn't work for that particular situation. That is something AMD should fix. The 2990WX would benefit greatly from having a UMA mode. Your description of the spectre mitigation is ... well, it's actually pretty wildly off. Sorry. Though you got fairly close. First, the system overhead is not really an issue of switching between unrelated processes. There is no need to segregate processes to avoid that. There is overhead there, but it is not synchronous... it's no in the 'hot path' that would interfere with a program on a continuous basis and effect its performance. Where the spectre mitigation really hurts is when a user process does a system call. This is a synchronous operation, it always occurs on the same cpu as the process making the call. The syscall trampoline must issue the Spectre related MSRs to firewall the cache effects and these MSR's eat around 2uS (2000nS) of overhead when all is said and done. It's very nasty. The same context switch issue gets even worse when we are talking about VM's. VM's have to execute the spectre mitigation not only inside the VM, but the host also has to execute the mitigation when the VM punts out to the host (which it does for a lot of things, particularly if you can't map PCIe devices directly into the guest). The Spectre MSR's don't just clear the L1 cache. In fact, I'm pretty sure they don't clear the L1 cache at all. They primarily have to clear the branch cache and mess around with the call/return hardware cache in the cpu core. And they also have to firewall speculative execution from crossing the boundary. The Meltdown fix essentially requires isolating the MMU map between user and supervisor mode, which means that %cr3 has to be written to on the context switch (e.g. for a system call). Twice in fact, one to get into the kernel, and one to get out of the kernel. This overhead adds 100-300nS or so depending on the cpu. AMD is less vulnerable to Spectre and invulnerable to Meltdown. It is less vulnerable to Spectre because it uses full 64-bit address tags in its branch cache tags while Intel uses fewer address bits and XOR's the higher address bits into the lower address bits. This allows a user program (on Intel) to directly control the branch cache for memory locations in kernel memory space. And for Meltdown, Intel CPUs will speculatively read memory through the TLB while ignoring the U (supervisor/user) bit in the PTE, which allows a user program to mess with the L1 caches related to any kernel memory location. AMD's speculative reads honor the U bit and do not have this problem. So, basically, meltdown adds 100-300ns of overhead, and Specter adds 2000nS of overhead to any system call. This is also true for interrupts. That is a lot of overhead considering that the nominal overhead without the mitigations is typically only 70nS. -Matt

Very interesting video, thanks

Great timing! My next exam is about parallelcomputing and parallelprogramming. Which includes NUMA / UMA. This video was really great and helpful!

Great job explaining Nonuniform Memory Access.

that was pretty cool............... thank you wendel...........

This is a great video Wendell, and I think it's going t be an important consideration going in the future because AMD would not have been able to achieve high core counts without multiples dies and thus multiple NUMA nodes, and Intel isn't going to compete without following a similar path. I have been experimenting with my Skylake-X processor (7820X) and on some games, I get a vastly improved framerate by setting core affinity manually. But the Skylake-X is still on one node, so I'm not sure what the exact cause would be. It's more apparent with older titles that do not multithread well. WoW is a good example, and I get best framerates in that choosing two logical cores which are not on the same physical core (ie cores 0 and 2, or 13, 15 etc). I will have to test on GTA V. I don't know if it's do do with the userland/system level processes you mentioned. I was originally thinking it was due to Intel using a Mesh instead of the Ring Bus. I'd like to hear your thoughts on that. I love this sort of content, keep it up!

Vrei sa pleci dar nu ma, nu ma iei nu ma, nu ma iei, nu ma, nu ma, nu ma iei Nu Ma, Nu Ma Iei, nu ma, nu ma, nu ma iei Chipul tau si dragostea din tei Mi-amintesc de ochii tai ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-KmtzQCSh6xk.html

Who comes here after watching Linus Epyc video?

I run a 5950x. I'm going to try process lasso and or try modifying the core affinity for my games. I know that my system is not running numa but there is still some latency with spreading a task across multiple ccd's as far as I understand. Thanks for this.

very interesting! thanks Wendell!!

Using memory benchmarks I noticed a general improvement on the 1950X using UMA regarding peak bandwidth, I was unable to test latency

I couldn't help but notice DisplayCal on the monitor in the background. Doing some calibration? Nifty piece of software really.

Pure class, great work!

You make me regret not going forward with computer science after hurricane Katrina :/

Using ProcessLasso since 2015

This is a great Video Wendell. Could you take a deeper dive and explain why we would, or would not want to manage memory, cpu cores, for every situation to optimize performance?

Awesome, very good explanation.

I remember having major headaches just trying to not have threads stomp all over system process back in the day, and the only fix was just as aggravating as you could not set an affinity and have it stick. the instant the thread or process closes that affinity is forgotten the next time the thread or process starts up, so you basically had to manually set the affinity every single time you opened an app (on XP through win7). I've yet to even bother looking at win10 affinity, as it's a not worth the effort kind of thing at this point, and easy to just assume setting affinity on windows is more effort than it is worth. As for non-windows, well, yeah, remembering affinity has been a thing going way back, and it has always been a set it and forget it kind of thing, lol. I had thought about looking at the 'topo' of my FX8350 comp (win7), however, I hadn't had the time to do more than just think about it in passing. And I suspect I already know just how bad the M5A97 (rev original POJ) is laid out, lol. The top 16x slot will not post with a GPU over 50 or so watts, so the GTX1050ti had to go in the lower 4x slot (ouch), and I guess like the AM3-99 chipset (I forget the full name of it) that slot goes through the chipset. I don't game on the system, so it works well enough, lol. And I know the 4x PCIe 2.0 bus is NOT holding back the GPU, as the GPU has no prob going to 100% load with the bus never peeking over 50% usage from my testing some time ago. Great vid Wendal and Crew. B)

Very informative video, thankyou

this channel is absolute gold

Fantastic content, ty.

Loved the video! You know what you're talking about

Flush level1? All I can think about now is Wendell, Ryan, and Krista being flushed

15:38 - don't worry wendell, we won't flush you :P

Engagement!

verdict: you indeed made NUMA exciting (0:53) 👍 Thank you, Wendell

It was riveting!

How could he ever survive those long podcasts with the other guy. His depth of knowledge is great.

Good video, very informative.

at 2:54 the video is broken, thanks youtube

here's one of those report you talk about ;) mine I do that for an old game (torchlight 2) it already had stuttering problems on a 9900K (8c) but on 3960x (24c) it was way worse so I did the set affinity thing and only select threads 40-47 (4 cores+MT) and tada frametimes and fps go from a rollercoaster to pretty much an horizontal line, older Forza Horizon games on UWP/MS store also had that big problem as the game was decrypting the "secure" data of the game aka the textures and that in a open world game meant your core0 was at 100% use all the time basically don't know if they fixed that but it was a complaint often found in the officialk Turn10 forums, process lasso is also something game modders/tweakers have used for years

thats strange the video won't play past a little after 2:45 , just skipped a little ahead and its playing fine which I hope it still counts as a view in the system

09:46 Something else for the Windows users to download separately and install ... and then figure out how to keep up-to-date.

Great job!!! Please, next time try to address the the issue at hand in regard to network card and network performance... :-). Thank you 🙏

thanks Wendell

The big question: why doesn't MS allocate cores which aren't doing anyghing (sort of, no load) to games? Why put a part of the game its load on the same CPU core as Windows? That doesn't make any sense to me. I noticed that Wendell is running Ubuntu. Preparations for the PCIe passthrough on Ubuntu tutorial?

thanks was helpful... not a gamer but good to know when it comes to Sql Server.

haha I was just going through the best performance guide for VMware and never heard of NUMA even though it was mentioned to have it enabled and to use vNUMA never heard of it before hand - gonna score some pts with the boss

This dude does a good job of explaining things without boring the sh*t out of me.

numa is the coachella of it yeeiii

Clive Cussler National Underwater And Marine Agency Great adventure books!

1:43 Topology ≠ topography!

Hi Wendel, Can you please make a video about making a dual boot pc. I have tried to make a dual boot of Windows 10 and Linux mint, when i am installing Linux mint using Rufus as bootable it just stays on the linux mint logo. I have done this before on my old PC but i do not know what is wrong why it is not working anymore. I have x370 prime pro, 1700x, 16 gb ram and 1070 gpu. Please make a video of the step by step procedure and what programs are you using for this. Thanks.

Am I the only one having issues with the video? It just stops at 2:54

This is very interesting and should make the DIDs simulator EF2000 Tact-com run much faster

The NUMA Graph that you showed kinda reminds me of the "It's a Unix System" on jurassic park, except this is a real Unix-ish system xD

Enjoy the content man, can I just check something quickly though? So if all the cores are on one node, is NUMA not possible? Does it depend on there being a interconnect such as quickpath or more than one physical socket? Thanks man

Changing the CPU affinity sounds like drug/chemical’s affinity for the receptor it’s seated on. Flushing L1 cache sounds like taking a receptor antagonist while the receptor is occupied by an agonist. What process has the authority to “kick” off you off your set affinity ? Anything?

Let him talk about NUMA ...love your videos. Thanks for the schooling XD

Sweet I'm saved my 4K TV gets confused when I try to run 1080p, so I'm not losing anything sticking with 4K.

They still have 16MB of cache.. that's like.. the entire main memory of my first PC. it ran windows 95 in 16 meg :O

Powershell!

Only 5:00 in video already like clicking noise

hehehe. I clicked like because my 2009 Macpro Pro died (2 x 3.46GHz Xeons with 32 Gig ECC Memory). Now I'm back on my ~2008 core2 system, which I upgraded to a Quad core from South Korea on eBay, and maxxed the memory to 8gB. The 'old' workhorses you are showing made the internet way better. Time will tell if the newbies can do something awesome ;)

Hey a dell R710 ( well google search app) runs my works 90TB raw raid just fine !

I'm sure there is a song named numa,, I'm humming it now

Nice video

Hey I just got a workstation off ebay... the machine only POSTs when I disable NUMA... when I enable it, it's basically a dice roll if it POSTs and 100% doesn't get into the OS. What could be the issue?

What about nematodes?

But is there really any benefit to running numa mode on threadripper when you can just set affinity to cores within 1 die or 1 ccx, depending how many threads it benefits from? Even on ryzen 1600x setting affinity to threads within 1 ccx can improve framerate on some games. Also, if you want some more advanced tweaking, process hacker is program that can set affinity individually for every thread separately, but unfortunately you will have to do that every time manually for every thread when you open program.

The picture on the screen is all white during most of the video for me... Can you add pictures/screenshots of those in the description?

He’s saying it improves performance but then saying it increases latency

I love your explanations...lol

So the os scheduler is a better for dealing with specter and meltdown than any changes to the microcode? Obviously on lower core count processors this isn't the case, but for servers why is Intel responsible for fixing the issue? Please correct me if I'm misunderstanding this.

what is that software running on the big monitor? I want that :-)

I’m so confused so what’s better for latency no numa node or multiple

isn't flushing L1 cache related to resolving the meltdown issue ?

Ask Dirk Pitt...

I remember numa numa having a dance. Where is the dance?

Anyone knows why wendell mentioned flushing level 1 cache is bad? My guess is performance?

Intel server grade hardware also supports "UMA". I should say... you really can't make a NUMA topology become UMA by flipping a switch. The moment you have two CPUs with an IMC on each one you have NUMA nodes. What you can do, and both Intel and AMD support this, is setting node-interleaving on. This will present an UMA like topology to the OS (I'd rather call it Sufficiently Uniform Memory Architecture or SUMA, as some people do). This will only change the memory addressing to be interleaved, so when you access memory in sequence you will hit local memory, remote memory and so on. The latency will "look" uniform but it is not actually. I've read about workloads that prefer this predictable approach, but I've never seen a case where it is necessary in real life.

NUMA is like HEMA but sci-fi

engagement

👍