Тёмный

Rust Allocators and Memory Management 

Michael Mullin
Подписаться 2,9 тыс.
Просмотров 10 тыс.
50% 1

In this video I go over some basic Linux memory management concepts, and talk about the pros and cons of a few Memory Allocators in Rust.

Опубликовано:

 

30 окт 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 21   
@Hector-bj3ls
@Hector-bj3ls Год назад
I love a good "level 2" understanding of a subject.
@meanmole3212
@meanmole3212 4 месяца назад
it is good but wait until you get a taste of level 7
@meyou118
@meyou118 2 месяца назад
love this - glad i stumbled on this
@TehGettinq
@TehGettinq Год назад
Ahh a habs fan, fellow rust programmer and vim user. Delightful combination. Thanks for the video.
@seikatsu_ki
@seikatsu_ki Год назад
We yearn for serious topics, thank you for sharing with us your digging deep's journey!
@masmullin
@masmullin Год назад
Thank you. This is one of my favourite compliments I've received.
@ronlevi8298
@ronlevi8298 Месяц назад
Great video, super useful
@semigsedem
@semigsedem Год назад
Many thanks, I learned a lot. Just the right detail level for me I think :)
@masmullin
@masmullin Год назад
Glad it helped!
@CamaradaArdi
@CamaradaArdi Год назад
Really good video. Please do another one with the heap analyzer you mentioned
@irlshrek
@irlshrek Год назад
absolutely loving your content!
@mike-barber
@mike-barber Год назад
Nice one! Might be good to critique the system allocator on Alpine too, since it's not just glibc, and seems to perform quite poorly in some cases. Nice to have mimalloc and jemalloc available to work around it.
@masmullin
@masmullin Год назад
Oh, now I'm kicking myself. That's a really good idea
@mike-barber
@mike-barber Год назад
@@masmullin thanks! I think it could be quite interesting indeed!
@masmullin
@masmullin Год назад
Looks like musl (the standard libc of alpine) has a bespoke malloc implementation (elixir.bootlin.com/musl/latest/source/src/malloc/mallocng/malloc.c). This allocator is significantly slower than glibc (and jemalloc/mimalloc). The good news is that it's just as easy to replace the allocator in musl as it is with glibc. By switching to mimalloc+musl, the test application shown at the end of the video performs only about 4% slower than mimalloc+glibc (roughly on par with glibc alone), and musl alone is 38% slower than mimalloc+musl. jemalloc_perf+musl is the same as mimalloc+musl, but with the high memory initial overhead as seen with jemalloc+glibc.
@user-zq8bt6hv9k
@user-zq8bt6hv9k Год назад
Interesting, thanks for the work
@terrnnoo7007
@terrnnoo7007 Год назад
Didn't quite catch why wouldn't allocator give back 2559 dirty pages to OS if these 64 bytes are in use. Does allocator want us to free all requested memory to give those pages back or bcs we wrote data to these 10 Mb but freed only 9.9 Mb?
@masmullin
@masmullin Год назад
This is difficult to explain, sorry for the confusion. There's two types of allocation in Linux. One uses sbrk to move something called the break for the heap up and down. Think of the break like a line. In the case where you move the break up 10mb, either in one big jump, or many small jumps, then you use all of that 10mb, the you free all the memory other than the very top; the allocator cannot move the break back down because that very top is still being used. The other way to allocate is via mmap. If you use mmap by hand, you can grab a 10mb chunk of memory, use it, and the mark 9.9mb of that memory as DONT_NEED, I've not seen that sort of behaviour when an allocator uses mmap to grab memory and then give it to you via malloc/free. In the case where an allocator uses mmap, it will (hopefully) mark that 10mb chunk of memory as DONT_NEED when you are completely done with it. also, allocators try to be smart with mmap. Eg jemalloc will wait to mark an mmap as dont_need for some amount of time in case you ask for more memory.
@terrnnoo7007
@terrnnoo7007 Год назад
@@masmullin So if allocator uses sbrk syscall there is particular reason why 9.9 Mb isn't freed (bcs 64 bytes are located at top of the 9.9 Mb). But in case of mmap it seems like nothing prevents allocator from freeing 9.9 Mb if it wants so, bcs mmap doesn't increase brk segment address but instead giving us pages of memory somewhere. So is it true that 'Dirty Pages' are really possible only while using sbrk, bcs if allocator use mmap it can call free syscall on freed(by allocator API) memory pages?
Далее
OOP 2024: Rust Memory Management Introduction
57:05
Просмотров 1,6 тыс.
Ледник 1:0 Мужик
00:53
Просмотров 2,4 млн
Why does this Rust program leak memory?
35:24
Просмотров 58 тыс.
FA24 CSCI Review Session - CS111 Cunyet Akinlar Midterm 1
2:08:48
Rainer Stropek - Memory Management in Rust
59:48
Просмотров 12 тыс.
Solving distributed systems challenges in Rust
3:15:52
Просмотров 264 тыс.
When Zig Outshines Rust | Prime Reacts
23:31
Просмотров 143 тыс.
Unreasonably Easy Console Apps in Rust
1:54:16
Просмотров 102 тыс.
Constructors Are Broken
18:16
Просмотров 110 тыс.