Can't thank you enough for explaining the memory swap subject! Over 3 years using Linux as my daily driver I still found myself closing browsers, shutting down my virtual machines, even rebooting to address it (even with 32gb of ram). I now have the confidence to tackle this and really make use of my rig!
Hi Jay! Your videos are a godsend. As a career changer and a sort of a late-starter in IT, your content is pure gold to me. This video helped me fit some pieces better in place and I feel more confident with this topic now. Thank you so much!
I LOVE the way you and DJ Ware cover these kinds of topics- a great blend of more advanced topics that are useful for home desktop users as well as server and enterprise admins. I'm a compulsive tinkerer and videos like these help me learn ways to satisfy my tinkering/ optimization addiction.
I have an old original Raspberry Pi with 256 MB ram and 64 MB allocated to GPU. It runs headless and has Apache server installed along with ssh, dlna server, network scanner server and a vnc server. A desktop is installed. I very rarely need the desktop so both it and vnc server are disabled and only started if required. I did enable 512MB of swap as I think this may be needed when I do run the desktop and vnc server. I do have more modern Pi devices but old faithful has server me well and continues to do so.
In the scenario of hosts behind a load balancer, I would rather avoid swap slowness all together, and quickly remove that host as it stops taking requests reaching memory limitations, that I would see request latency increase, especially if your application is set to backoff/retry behind the scenes. This is the best argument against it imo, its not just about disk space being cheap.
Excellent content, thanks. In regard to allocating storage for swap-space, maybe you can elaborate on this: When a laptop is put to sleep/suspended I thought that the OS writes the contents of RAM to swap-space and back the other way when the laptop is awakened. So, in that use-case wouldn't a sufficient amount of swap be necessary? Sleep/suspend isn't relevant to servers/workstations but maybe it's worth mentioning.
Even though it may cause slowing down the server, any swap is better than no swap, especially running a database aplication. 30+ years of experience suggests that. Cheers,.
A m.2 at a good clock only moves 5000 mb/s at best and ram does 35500 mb/s. A sata ssd is around 500 mb/s. It's better to use the ram for video than hard drive. That's my theory is try not to use swap at all. But you need swap if you freeze your computer. The swap will slowly untangle it. So that is why the older thinking of twice the ram.
Is there a way, driver or something that would allow one to use storage as system memory? Say use a flash drive as part of system memory and a hard drive for swap. If I have 2 GB of ram and need 32+gb, setting a 64gb swap isn't going to help. How about a distributed shared ram?
Thanks for the video. Unfortunutely, my server uses swap because of the analysis dependencies. I have more than enough space, ram and storage but the analysis sometimes oversteps to swap.
Then you have enough memory demand to write dirty anonymous pages to the swap file, there's no harm in having idle unused daemons end up with saved state in swap rather than retain memory wasting it. It's when you have a high rate, due to insufficient memory causing thrashing and have the OOM killer terminate processes, possibly due to lack of swap allocation that you have a real issue. This swap is bad mantra is over simplistic, it's there to use memory better.
Well, it is stated that the swap is a bad thing because ot uses slow storage devices, but I'm not sure if it is still the case. We now have 7000 MB/s storage devices.
When I installed a 2TB SSD 16G mem in an old Toshiba laptop running Ubuntu I got 15sec boot time I set up 16G swap file, everything starts in seconds I can run multiple applications and I never ever need to use Htop or other system monitors.
I’ve got 15.5 GB RAM and the max thats being used is 3.8GB ( 2.1GB AVG). Swap is always used up to 3 GB ( 1.7 AVG). The performance of the applications I am hosting is not very good and I feel that is because it is not using enought memory even thou I’ve set 2.5GB for each ( total 7.5GB). For example right now in bpytop it shows the ram as 19% used ( 2.94 GB) ,81% free (12.6GB) , 79% cached (12.3GB) , 1% Free ( 163 MB) and the swap 328 MB used. So at this point I don’t understand swap since I am not running out of memory.
Hii please reply. I need help. I kinda set my swapsize to 2092 and now i can't remotely desktop connect to my Jetson Nano with Linux system. And I checked in it's SSH that the "system has been minimized". Is it because i set up it high? Did i broke my device???
test it out - assign 20GB of your disk for swap space, set the swappiness to 100 (use swap aggressively) and you will get the answer to your question very quickly :) basically you would think you have a machine with a lot of memory, while in fact everything will be extremely slow
The issue with swap memory is also about how much to spare. Now is 1 GB good enough or should it 50% of the RAM ? like I have a 16 GM RAM and partitioned 4 GB for swap in my SSD
It all depends on whether or not you plan to use hibernate, or "suspend then hibernate". If you do, swap should be slightly larger than how much RAM you have. If you're not using hibernate (and never plan to) then I usually just standardize on 2GB (4GB at the most).
Linux Crash Course helped me in improving Linux knowledge! Great thanks for your contribution and outstanding tutorials! On the other hand, I wanted to ask your opinion on WSL2. Did you have a chance to interact with it? Any thoughts from professional community on it's usage in day-to-day tasks?
Excellent video, easy to understand explanation of the essentials. However I only use desktops, I trust the defaults chosen by the kernel designers, so I never touch swapinness anymore :) I moved my applications to VMs and my Host OS is a minimal install of Ubuntu 22.04 LTS, my VMs run off OpenZFS 2.1.2. My Host OS has 2 swap partitions; first priority a 2GB Swap on a nvme-SSD (3400/2300MB/s) and second priority a 16GB Swap on HDD as an overflow swap, but intended more for future hibernation. I often see swap usages between 100MB up to 1500MB and I don't care, since I don't notice any negative effect on the responsiveness. Remember many programs have large parts, that are never used or only once during e.g. initialization, closing the program or in case of errors or exceptions. I see it as "unused and wasted" RAM too :) Those parts of the program will be swapped first (least frequently used), so you will not see any effect with respect to responsiveness. I limit the amount of RAM I assign to VMs, because I want to be able to run 3 to 4 VMs in the 16GB of my Ryzen 3 2200G. The VMs share the RAM with a 0.5GB to 3GB ZFS memory cache (L1ARC). So for Windows 11 Pro I use 4GB and for Linux VMs I use 1.5GB to 2.5GB. I have no complaints about responsiveness. Also there I see the SWAP is used extensively, but that swap storage is part of the virtual disk and that virtual disk is cached by the ZFS memory cache (L1ARC) of the Host OS. The L1ARC is lz4 compressed, so currently it stores 6.12GB of compressed stuff in 3GB physical space. Looking at the size of Linux after booting say 0.6GB to 1GB of uncompressed storage, the Host OS stores most of the used disk content of the 4 VMs in the L1ARC, including any used swap space. Consequently my VMs run from memory :) :) and even on the 14nm Ryzen 3 2200G (4C4T, OC 3.7GHz) it is fast. The boot time and initial program load times depend on the speed of the storage, so Xubuntu 22.04 LTS boots from nvme-SSD in 9 seconds and Xubuntu 20.04 LTS boots from two striped HDDs supported by a 90GB sata-SSD Cache (L2ARC) in 30 seconds. Booting Xubuntu 20.04 LTS for the second time takes 11 seconds, since now it largely boots from the L1ARC memory cache. Note that e.g. ext4 installations also extensively uses caching, but by default the cache is uncompressed and the algorithm is LRU. ZFS uses a more sophisticated method of caching, mixing LRU and MFU, while dynamically adapting the boundary between the two based on the dynamic results of both methods. My VMs use ext4, so I have some caching there too, but the available memory is relatively small, so caching is relatively often deferred to the Host level and the superior L1ARC.
Agreed, I have said myself "why argue about putting aside a 'tiny scratch' on a huge disk", I think some individuals just attach to ideas and 'get comfortable'. Swapping may be slow but it could be the best place to be when needed.
How do you calculate the total memory from the available memory, free and used, etc? It used to be you could do that in the past, but in this example there seems to be megabytes of missing ram.
I would love to see also tutorials that touch this harder topics (why its show less mem than actually physical mem, what are shared and cache plus free values differ from available, etc.). This concerns also other your videos/topics. As an Linux admin/user for years I already know basics, please consider to make videos for more advanced Linux user.
It is a compromise: It will make your system more "snappy", applications that didn't run will stay in RAM for longer and therefore be available more quickly. On the other hand, data of running applications will not be cached as well, so accessing data from disc will take longer and therefore your running applications will be slower. If you want to know more learn to use vmstat and how to interpret the values. This gives you the tool to fine-tune your applications.
I'll just give you this real example: you might have an app which is memory leaking, it will first consume all the ram, then it will make other processes memory to be pushed into swap. And then it's a matter of chance, what kind of error handler for memory allocation exceptions that buggy app will have. In vast majority of cases it will have something like "okay, we failed to malloc, we wait for a while and try again". Which in practical term means your system will become very unresponsive, because whatever memory will be freed, it will be immediately occupied by that single buster. And your chances to quickly ssh to that machine and kill the nasty app will be getting thinner and thinner, as now SSH daemon, and login/sh processess it spawns will be 100 times slower, as they'll be swapped to. So swap is bad not because of religious holywar. And to be 100% safe, have some resource manager set up on your system. And test how your system behaves, when some shitty process eats all the ram. It can be coded with just few lines of C code. And it will open your own eyes what a disaster it can be, if any of apps your server might be running will get some memory leak.
3:20 You have a gig "free" on that server because you don't understand what that 0.5G swap and vm overcommit_ratio are doing. The default for overcommit_ratio is 50, that's where 2.6 is coming from it's close enough to 4.5 * %50. This %50 is where the twice ram(because half of 3(1 + 1 * 2) is a little over 1 or use all of ram and some swap) for swap advice comes from, when ppl stopped following this advice... the kernel devs never changed overcommit_ratio and thus the old advice is still current. It's not that you have a gig free.... it's that you have a gig of ram the kernel will never commit... to things like running applications.
I wish every distro, whether it be for desktop or server, came with vim and htop. These 2 programs are my first downloads for any distro I install. Great content as always, keep up the great work!
Someone please tell if it's possible to install GCC 12.1 stable version on Ubuntu 22.04 or not. I've been struggling to update mine from 12.0.1 which is experimental. I just can't find a way to install 12.1 which is the latest version.
How about making a video about open source BIOS. If I want to install an open source BIOS in my computer replacing previous closed source BIOS what kind of challenges do I meet? Or, is that even possible?
It’s very possible to do, but depends on what model you have. Unfortunately, that particular topic wouldn’t translate well into a video because a small number of people would have the tools and appropriate hardware.
Hey friend I'm Jiejie from KAMRUI. We sell all kinds of MINI PC on Amazon. We are looking forward to presenting it to your audience through your professional presentation. Can I have a chance to work with you?
OK, I'm laughing my butt off. It looked to me like you used your mouse to quit htop, so I went to a terminal and opened it and clicked quit and it quit. I've used htop for I don't even know how many years and didn't know you could use a pointing device. I don't know that I'll do it again, but it sure made me laugh.
The reason too the free is lower than available is the buff/cache. The latter being alot like a RAM disk, but is changeable depending on memory demands.
You waste a lot of time saying the obvious and when it comes down to understanding the differences you just say "I'm going to stop my explanation right here". What's the point of making this video? Engagement only I guess
I like this format where you go through tools and features individually with their own videos. It works better for me than longer more wide-ranging videos. Thanks for the helpful content!
I've always included a swap file on distributions I've installed. Now, there is a different type called a page file. In the Windows world, the difference is a page file is dynamicly allocated as needed up to the limit you set, whereas a swap file is a static size. Page files are more disk management efficient because it only takes up as much space as it needs. But, contiguous space is not assured, which can really hurt performance. My first experience with paging and swapping was in 1985 on a VAX 785, where idle procecess would be swapped out to allow higher priority and active interactive processes access to memory (all 2 megabytes of it). We were running VMS 3.4, and the system administrator made the swapper a lower priority than he should have. The VAX (which serviced the 28,000 undergrad and graduate student population - even if you weren't a student that had a reason for an account, you had an account) was freezing. After about 3 days of troubleshooting, it was found that the swapper was swapping itself out to the page file. When something is paged out, it is put in a sleep state until the swapper swaps it back into memory. Well, if the swapper is put to sleep, how is the swapper going to swap itself back in? The system admin pit the swapper back at the hiest priority so it would never be swapped out. Thank you for reading my war stories of my leading a time sharing system.