@@Doriandotslash I have installed ubuntu 20.04 by selecting LVM (encryption). now I need to make a partition for my logical volume, please help? Thanks
So beautifully explained! And I used the word beautifully instead of clear because not only it makes it crystal clear but it does it in a way which makes you appreciate the beauty of the LVM feature! Thank you for this lesson, it is so much clearer what LVM is.
Thanks so much for this easy to follow tutorial. I knew how to extend on the server side but couldn’t figure out how to get it to work on the desktop side. Confirmed this works on Linux Mint 22 cinnamon ver. 6.2.9 Kernel 6.8.0-41-generic. I’m sure there is an easier solution but for the new Linux user this is great.
Just use symlinks/mount points. Done. No need to install/manage something. LVM is a thing when you do a server solution, like FTP/SSH backup storage, when you want to store a big amount of data on only one partition. On desktop it's quite useless, only creates a lack of reliability and management issues in any case. Keep it simple. The solution depends on a scenario.
Awesome video and thanks for the cheat sheets! This solved a problem that I have, but was unsure how to express as a question without looking like an idiot. Thank you! 🤩 BTW, I got here through the Linux Mint Resources Telegram.
Shrinking VGs are not that incredibly hard, but you certainly have to know what you are doing. You might need to reinstall the bootloader if the boot device is affected. Sometimes you may need to swap entire PVs where pvmove is your friend. In case you shrink an LV, the file system needs to be shrunk first which may require unmounting the FS that is problematic if that's your root FS. You need to boot another system (such as a Live CD) to pull that off or use a file system which supports online shrinking in the first place (such as BTRFS). I also recommend getting familiar with software RAID over LVM, because it's super-useful to add data redundancy on top of LVM benefits.
great vid. you could make it clearer at the beginning by stating what benefits you can get from lvm. Question: the partitions yoi created with lvm are then extended and spanning several physical disks which bring more failure points on the physical disk subsystem. Let's say that after extending your main disk partitions on another similar sized disk, you see that in the future both disks will be full and thus a new extension will be needed. BUT you decide then to move all the lvm volumes to a single big disk. Is it possible?
Yes, but that's not what I'm showing in this video. This is great for easy expansion when backups are already in place of course. But if I really needed reliable fault tolerance, I would probably do LVM over hardware-based RAID 5/6/10. But at this point I may not even bother with LVM depending on the requirements and purpose of the system.
Why is there no spanned volume option on Linux similar to Windows? I just want to JBOD SPAN 3 drives, LVM seems awfully complicated for something that requires 6 button presses in Windows.
Reducing a filesystem (other than root) is actually quite easy. First you have to reduce the file system size with resize2fs or something else if you are not using ext4. Reduce the file system to a safe size leaving a safe margin for lvreduce to operate. Next shrink the LV using lvreduce and the extend the filesystem to fill the entire logical volume with resize2fs again. This is not possible for the root file system because the filesystem must be unmounted. For the root filesystem you have to boot from another boot media. You can also move the extents making up a logical volume from one disk to another but that is an advanced technique. For added safety I use LVM over mdraid which has worked for me for ages. For simplicity I don't extend my VGs over multiple PVs, but mdraid makes it easy to swap one disk at a time to a larger one and once I have extended the underlying mdraid I can grow my filesystems using the appropriate LVM commands. When I was working with servers that had their storage on reliable external storage systems, extending volumes by adding new PVs was the norm.
Yes I’ve done all these steps many times. Booted off of USB, pvmove the data etc. only worked twice ever. I just wouldn’t recommend it at all personally. Too much risk of data loss.
I happen to have a mini notebook that I had to reinstall last spring because the current versions of the major desktops don't work on it anymore. I accidentally created a boot partition that is too small but that was most likely the default option in the installer. Since this particular laptop is generating quite large initial ramdisks, the boot partition cannot hold more than two kernels at once which is a pain in the "donkey". I was going to attach a USB drive and pmove my vg to the external drive, boot from a rescue image, and then (after resizing) move the vg back to the original disk. Bot your experiences have made me consider an easier and safer way. I'll just boot the rescue image and do the entire thing from there. The first thing I need to do is to reduce the size of the root lv since some of its extents are at the end of the vg. Thanks for you warning although I have never had any problems with pmove especially when running from the operating system that is using the vg's.
So if I'm going to build a new desktop computer, should I install it using LVM - just to be able to expand easily in the future? Also, are ther other downside besides shrinking not being a real option - like e.g. performance loss? Finally, is there _really_ no GUI-way to do this? (I don't mind the terminal, but it seems so odd, considering the capabilities for example of GParted)
If you want to easily expand in the future, yes. For me personally, I keep all my personal and project files in a separate partition which I regularly backup. So I don't mind just wiping and starting from scratch anytime. Performance for LVM is no different than using any other. There is slight overhead, but not noticeable. At the time of this video, no there's no official GUI to do this. But this also isn't something you need to do often, if at all. So it's hardly an issue most of the time.
I’ve watched a few if your videos as I’m researching returning to Linux after twenty years. They were so helpful and well explained that I had to subscribe. Many thanks.
If you use LVM for a non boot drive - like in the case of a DIY NAS - and you want to remove or replace a disk without data loss by redistributing the data across the remaining disks - can you do that safely? What about performance? Does the LVM do any striping or fragmentation to improve disk read performance?
Then you unfortunately lose the data. LVM is not redundant like a true RAID array. Like always, backups are very important. Even a RAID system can completely fail. This is why even though I run RAID arrays at work, we still do regular backups.
@@EduardoBattaglia Yes but that’s not what I’m demonstrating in this video. I personally don’t like using software-based raid either. If I need to use raid for a system then I want it to be hardware based and independent of the OS and software running.
Actually, it is LVM 2. :-) Great video. If you want redundancy, you add LVM on top of software RAID. It is EASY to change to larger disks if you have LVM. It is easy to enlarge a logical partition in LVM, and then expand the files system. You really doesn't need to think about what is on those physical disks. That is up to LVM. I usually NEVER extends the Logical volume to max of volume group. It is good to have some spare size. It is ALWAYS easier to expand a lv and filesystem. Much harder to reduce file system size then add size. Always remember to not reduce the file system size smaller then the underlaying logical volume/disk. So reduce in right order. And no, I never had any issue reducing size.
Good advice. I guess I haven’t had luck reducing for some reason, even after rechecking the proper steps and order. For redundancy though, I would personally only use hardware RAID since it cannot be affected by any software/OS/kernel failures or lockups.
@@Doriandotslash thanks. I prefers to use Softeware RAID, as a hardware that go bust you have to replace with the same hardware and firmware. Software RAID you can just move to another computer.
Btrfs RAID is not true RAID though. And Btrfs is only now starting to become more popular, like Fedora using it out of the box. The disadvantage is that snapshots in btrfs can be compicated to manage from an external boot environment. Which makes recovery and repairs difficult. There are definitely good thing with btrfs though, and I will also be covering it in the future!
@@Doriandotslash Software RAID isn't hardware RAID, but it's still RAID. It wouldn't make sense to call one thing "real RAID" and the other not, they're basically doing the same things, just in one case managed by software and in the other one by hardware. There also normally are other important differences of course, but those are not essential for the disambiguation.
The "software RAID isn't real RAID" thing probably comes from times before ZFS was a thing. Still didn't make much sense though, just because a solution is much inferior doesn't push it into another category.
I believe that most motherboards have built-in RAID controllers, and most RAID controllers support "JBOD" (just a bunch of disks). Assuming the above is correct (please let me know if I am mistaken), then I believe that you can use the hardware's RAID controller's JBOD option to add more disks, and it is probably easier than the LVM software option. It has been a long time since I purchased a motherboard (or new computer, which will have a new motherboard). So I am uncertain as to the inclusion of RAID controllers and the JBOD option.
You could do this yes, if you only need a basic setup. You can also put LVM across a RAID array. LVM is just easy to expand on the fly. And what I didn't cover is that you can also have multiple volume groups. So if you have 3 drives, you can have 2 VG's, where each VG takes up 1.5 disks.
I wonder how this compares to the performance of RAID 0 (if at all) or if you would have to use LVM over top of the RAID to get the best of both worlds. People think I'm crazy when I tell them I want to use RAID 0 but you can't deny the results on paper. It has the highest I/O performance and on a server that is being used for just backup, I don't care about redundancy as I already that covered on the client side.
I usually have duel boot along side windows. Now in case of ubuntu if I go for "something else" option (because I want to have separate partition for /home) I cannot have lvm.
So what about a disk failure? Does the volume gets down to the size of the other disk or the entire setup crash? The data on the surviving disk can be recovered or just gets corrupted? I never tried dividing volumes into different disk... 🤔 (yeah I know, backups, backups, backups 😅)
Pretty much yes. The important thing is to install as LVM from the start. So as long as a distro supports installing to LVM you can just follow these steps.
@@Doriandotslash thamks! Are you also doing reviews of different distros and their unique features? I've heard that Linux newbies should use Ubuntu or Mint or Zorin OS, but I've heard that Arch is the best option as it has no updates issues? Any opinion? Thanks.
@@TheJackal917 Arch can have issues, which is why I recently switched to Debian. I made a video about that. Check my channel as well. I have lots of distro reviews. And don't forget to subscribe for more in the future ;)
Thank you for your nice tutorial, I have installed ubuntu 20.04 by selecting LVM (encryption). now I need to make a partition for my logical volume, please any help highly appreciated. Thanks
@@Doriandotslash Thank you for your quick reply. I have installed Ubuntu 20.04 by selecting LVM (with Encryption). The default logical storage volume is about 923GB. I want to make this volume to 300GB(for data) and remain the other as it is. so how can I make a partition for that?
If you want to just clone a drive, and the new drive is the same size of larger than the one you're replacing, you can just boot from a USB and run "sudo dd bs=32M status=progress if=/dev/sdX of=/dev/sdY Now make sure the sdX is the device name of your old disk and sdY is the device name of your new disk. When complete, shut down, unplug old drive and boot up. Should work right off the bat, but of course make sure you've done backups. If you have issues, you can swap the new disk back to the old one and no harm done. There's a much longer way of extending groups/volumes to the new drive, moving data, and then removing the old disk, but the dd clone method I mentioned above should work and is pretty easy.
@@Doriandotslash OK, but to replace a failing drive using this method without affecting your system, would it have to be the same size as the dying drive? Would a larger capacity replacement throw off the LVM scheme, which now has more storage with the same number of drives?
If you need fault tolerance for very imporant data, or data that needs to be online 24/7, then RAID is what you need. LVM can also do RAID but it is software-based. In my experience, if you're doing RAID, use a hardware-based solution. If you're running a system where you might want to expand your storage in the future and you have backups in place, LVM is a good easy solution. Now I'm not saying in the first sentence that if you have RAID that means you don't need backups. You should always have a backup solution in place for important data.
@@Doriandotslash Cheers Dorian, followed your instructions and it works a treat, will save a lot time not having to move files around when the ssd fills up!
LVM has always confused me. Not because I don't understand how it works, but because I can't think of a scenario where it makes sense to add more than one physical disk to a volume group. The act of doing so doubles/triples/quadruples the risk of losing all your data (scaling with how many PVs you add to the VG), as well as taking away your ability to control which data goes on which disk (ie. if you want to keep backups on your slow drive and games on the fast one, or for any other reason), for the benefit of what - the ability to dynamically add more space to a single partition? Is that really worth the trade off, compared to just... making a new partition? I'm sure in the enterprise environment it can be very useful, but can anyone give me any use cases where it makes sense at home to have more than one disk in a VG?
You can place LVM on top of a striped RAID array for redundancy and protection against failed disks. You can also configure LVM itself to act as RAID as well, but I didn't get into that in this video. This would be good for people who deal with large amounts of data, like photo and video editors, or someone who has a ton of games maybe. It's especially useful for server application, where if you run out of disk space, you can just add more disks to your array, or add another entire array to the system.
With this method, yes, sort of. LVM configured like this provides no redundancy. But it may be possible to restore some files from the remaining disks but it would not be easy.
Yeah this is actually a really dangerous way to add storage to a machine. It's almost as bad as RAID 0, because if either drive fails you are going to loose data. Unlike RAID 0 though you won't lose ALL data hopefully. A much better option is to combine this with some form of RAID like lvmraid or MD RAID. Better yet just use a filesystem like ZFS or BTRFS that you can add drives too and has support for RAID and redundancy, and it also does check-summing to protect data integrity. Edit: lvm-raid is in fact base on MD RAID, though I suspect you can set them up independently if you really want to.
Yes and no. You can use a virtual environment to test out LVM's. You can also use LVM in a virtual environment and add virtual disks if you're unable to expand a VD due to physical space limitation. Or you can use LVM on the host so you can always expand space available for the VM's.
Holy shit this is my first time using Linux and I don't get it......at all I've used windows all my life n this seems really complicated or like it's for nerds.... no offense but ya I just want a regular OS and I liked the way ubuntu looks but wth
@@Doriandotslash No, to extend your pool with an extra _raidz_ . Your pool can exist of any number of raidz1, raidz2, mirror or single vdevs. In one vdev, like raidz2, you can only replace drives (optional with larger ones).
Isn't encryption the main thing you'd wanna use LVM for? To be honest, this just sounds useless to me. Maybe a short term solution, but I don't see it being viable in the long run. Sooner or later you will have to swap hard disks, and there's nothing you can do about that, so my question is: why even bother with LVM as a means of increasing storage space? From my perspective, you're simply adding more steps to achieve the same goal, and on top of that you're putting your data at risk. On the other hand, encryption is obviously extremely useful, since there's no reason you wouldn't want your data protected, other than convenience. Am I missing something here?
In a complete storage solution, you would also be running backups. I've managed systems that had triple backup solutions for extra redundancy, 2 to SAN arrays and 1 to tape. In the even you replace disks, you can do so and then restore all the data from backups. But now you're approaching enterprise solutions.
Yes but it was designed from a Unix volume manager, which was primarily used for servers that don't run GUIs. It does however have its uses for regular desktops as well. There were a couple of GUIs for it that never really took off.
@@Doriandotslash becaise of two reasons . hard to understand on not premoted by video. they may have fourms floating around, but heres the thing with that. No one is going to read them if no one has given it to them.
In a basic setup like this, yes. So make sure you have backups in place. But you can also setup RAID within LVM. Or, you can setup a RAID array and put LVM on it.
