Hi Chris. An addition ere: LVM doesn't necessarily have to use partitions. It can be built on any set of any block devices: partitions, raw drives, RAID devices, files, memory, etc. Things can go really crazy with it.
So if I have a second hard drive that has some data on it but I want to combine it with another one will that data be lost when I add it to a new group
@@InspiredInsights4U Yes. Basically, when you extend a partition, you'll also extend a filesystem to that drive, which practically means reformatting it.
Regarding the drawbacks you mentioned about LVM, one way to deal with them is to have a RAID layer below LVM. That way you used the logical drives from the RAID configuration instead of using "real" drives as PVs. Its very hard to lose data that way.
DUDE I have used Linux off and on for a decade, and started daily driving on all my machines a year ago. I'm glad I'm still finding cool stuff like this I didn't know. Thanks!
Thanks Chris for teaching me how to extend the LVM. I knew how to setup LVM's but never knew how to extend them with more drives. Another good thing about LVM are creating snapshots using Timeshift. If you have LVM configured using BTRFS it is super powerfull with creating snapshots.
Passing -r to the lvextend command will automatically resize the FS for the extended LV. So, in this case it would be "lvextend -l +100%FREE -r /dev/vg1/lv1" so then you don't need the final resize2fs call. Great video, thanks!
Three weeks ago I went totally Ubuntu. During installation I was offered to install using the LVM system. No idea what that was. But I said yes anyway.
This is how most people are introduced to it hehe. I know my introduction was doing a fedora install and I went "WTF... my home doesn't have all the space of my 120 GB drive!"
To this day I could not see it's use. I typically use one partition for /root and one for /home and the installation offered to set up lvm. No other options. Maybe make the installation offer to set up disc caching or some raid, but it just did lvm and I could not see any benefit of it, It's a click and forget thing. I assumed you could do something cool with it but ultimately when I wanted to have a system ssd and a 2x hdd raid 0 /home i went with mdadm because lvm is explained literally nowhere at all. Pointless to add it to the install and explain and do nothing with it.
@@wujekcientariposta Volume management doesn't have a lot of use for home users, but is very handy in an enterprise environment. For example, database files and their indexes can be separated into different logical volume groups and extended as required. System log files grow over time and if left unattended can cause issues. Being able to quickly extend a volume can help. Etc...
Wow, you're a lifesaver; thanks for making this video! I had like 200MB left in my /home directory and This was my only viable solution. I am not a storage expert and do not do this stuff on a daily, weekly, or even yearly basis.
You can resize btrfs filesystem on the fly too. You just need to use "btrfs filesystem resize max /mnt/xxx" (in most common configuration) or "btrfs filesystem resize :max /mnt/xxx" (if you spread the filesystem over multiple devices). It's safe to continue using the filesystem while it's being resized. Unlike most other filesystem resizing, btrfs can only be resized while mounted (i.e. it's always online resizing). And unlike resizing ext4 partition which only supports online extension, both shrinking and extending of btrfs can be done online.
Some small corrections. In partitioning the physical hd, ext4 is irrelevant. LVM needs the physical partitioned drive to be "lvm2 pv" on gparted and not "ext4". Also, now you do not need to use the pvcreate command because it just converted "ext4" to "lvm2 pv." So after gpated partitioned sdb1 to "lvm2 pv only type the following:" sudo vgextend Fedora_localhost-live /dev/sdc1 Note now "home" volume is an actual file on dev directory which is located in "/dev/Fedora_localhost-live/home." The "lvscan" command shows all the volumes. So more precise use of lvextend would be: sudo lvextend -l +100%FREE /dev/Fedora_localhost-live/home resize2fs /dev/Fedora_localhost-live/home Thanks for introducing lvm very to me as I need it for home, and for sure not for work ))))
What is up man? 👋🏻 Ever since i installed w10 (was on w7) i noticed my game (leage of legends) lagging more that before. I was like 🤔 Hmm. What went wrong??? Now after trying every single "Boost your game performance" and none of them worked, i started thinking. Zhen i remembered... Aha! 🤓 "Hey, was the problem maybe that I... urgh.. um was the problem that i, when i was installing windows 10, at one point i was asked if i would like to combine all storage on ma C drive ooor, keep it the way it was ( i had D and E, maybe F dunno). So, my questions are: 1. Can you explain me the difference of having all storage on one drive and having storage seperated to multiple drives 2. Is this the problem, is this why my lol is lagging even tho it didnt before, when i had all my drives (windows7). I really dont know, but im quessing it is cuuuuuz, everything ended up on my c drive which has the most things to do and my other drives arent being used at all. Am i right? Aaaand third question my, fellow youtuber 😉 3. Yo if my guess at 2. is true, Do you know if it is possible to get my drives back? Aaaand how 🙉 Yo, its like this If u help me, Imma give u a brofist 🤜🏻 Imma like the vid 👍🏻 En imma subscriiiibe ❤ Aight, peace ✌🏼 . . . Yo, crazy times right now, stay safe, hope u doing good 😷😤💪🏻
You have to use LVM if you do full disk encryption. If you use that, knowing these commands can help you get into your data if grub doesn't boot since you have to mount the LVMs to get in.
Since when? I've always done it but you should be able to use LUKS without an LVM partition. It might be more difficult if you have multiple partitions, but if you just had: /boot/efi -> EFI System Partition /boot -> Boot partition / -> Root partition It should work exactly like LVM on LUKS. The Arch wiki even has instructions for LUKS on a partition.
Oh man, doing full disk encryption on a bootable computer without using the installer or without LVM would be a pain. By putting it all under the LVM you can be sure your swap, home and root directories are encrypted. However, you are right that you can use LUKS on a drive, external drive, USB, etc., and I do that all the time for encrypted backups. Works really well.
Smac You think that's a pain? I had a setup like this once on a laptop. If you booted, it would load Windows. But if you booted with a USB drive I kept on my Keychain, it would boot into the rEFInd bootloader (because it's just better than Grub in every way IMO), and then if you selected Arch it would prompt for a LUKS password, it would use this password to decrypt an 8192 KB LUKS encrypted key file, and then use the decrypted key file and an external LUKS header to decrypt the LVM partition, then mount the USB drive at /boot, and mount the volumes. Then I could unplug the USB drive. The encrypted key file gave two factor authentication (something I had, the USB drive, something I knew, the password for the key file), and the LUKS header being external gave plausible deniability, as without it you couldn't prove the encrypted partition was encrypted and not just unallocated. That took me maybe a week to figure out, because I had to modify Arch's encrypt hook to handle it. It was the first "bare metal" Linux install I ever did, and I did it to see if I could after finding a guide to do a similar setup on Gentoo. I had a little shell script to automate the install, so I'd basically put that on the Arch live USB, move it to the laptop, run the script and reboot, if it didn't work, edit the script and start over. I still have that script on my Nextcloud I think.
@@praetorxyn I wanted to set something like that up. It really is super security. I wonder to what degree you can even tell a partition is there without the boot loader, is it entirely random without any headers? Thats good plausible deniability.
Smac As far as I understand it, I think you can tell it's a partition, if I remember right, but with an internal header you can tell it's encrypted. With an external header, there's no proof it's encrypted and not just empty space. I got the idea after reading about a situation where a court ordered a woman to decrypt something, but they were able to do it because they had a recording of her telling somebody that something they were looking for was in the encrypted place. Something like that. If they can't prove that a specific thing they are looking for is on your encrypted device, they can't order you to decrypt it. Sadly, this is all theoretical. In reality, it's more like this: xkcd.com/538/
All I want is a spanned drive like in Windows, but apparently its impossible in Linux, another reason why Linux is 500 years away from being a full fledged desktop OS. And no, people who do spanned drives, do not give a flying f*ck about data safety, thats what external backups are for and if the drive failed in a spanned drive, only the data on that drive would be gone, just like if it failed without a spanned drive.
The way you've explained it is analogous to using LVM as a software RAID 0. There are other options though, effectively similar to normal RAID. As a sample of where it gets used extensively: Synology and QNap use LVM to define their "RAID" setups. When you delve into the finer details you'll note they also define physical and logical volumes, even for their RAID 5/6 setups. Just like a RAID 0 is hardly ever recommended, I would also never recommend a LVM volume simply spanning across several disks. Same reason, if any one of those disks develop a problem you loose all of the data across all of them. I would definitely suggest anyone attempting LVM on data they don't absolutely hate, to look at lvmraid and adding parity to your LVM groups / volumes. And even after that, make backups, RAID (not even RAID 1) is also no substitute for a true backup.
@@ChrisTitusTech none of these are backups - just saying. might make your data last longer, but u r only putting of the innevitable loss - ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-l7-6m2cE6JM.html
My daily driver for the past 10 years or so has been FreeBSD, mainly because it has zfs volume management which is way more reliable than lvm. How are the equivalents - btrfs & zfs on linux getting on. Any time I've looked, it has been "experimental, should be ready soon", which is not something I would want to trust my data to. At the moment I have 4 x 10TB drives on my main machine giving me 25TB of usable space, and 4 x 6TB drives on my test-bed machine giving me 15TB of usable space.
Very well explained @Chris Titus Tech, that was just what I was looking for, thanks a lot for the effort! Just one doubt though, supposing I have a LVM, with multiple logical partitions, composed by 3 disks. If somehow I loose my motherboard or even one of the drives. How am I supposed to recover my files? I mean, what I should expect by plugging the two other drives in another PC setup?
I watched. Wow, I wish you would have been the Instructor that I had during one of the early Linux classes. How you explain it and simplify it out, is what more Instructors need to do. Also, the split screen -- with the commands in the background -- is incredibly helpful; also, as you work in the Terminal windows, is what I wish would have been presented when I was in courses for this. It would have helped drive it home. Most of our stuff was just walk through some tutorial steps, and it really didn't deepen the knowledge. It was rote learning, instead of "real learning" like this. Now, I feel more comfortable with disk organization. [Also, I plan on becoming a Patreon too for you.]
I likely misunderstood you when you said LVM is not used much. Actually. Many distros use LVM out of the box. Also, we've used LVM in the Enterprise for years. Great when you have planned ahead and need to grow LV in an emergency. Though I do miss ZFS from Solaris days. Now that was easy to use.
These days, I almost demand businesses store everything in ZFS. The only other storage I find acceptable is RAID 10, but there are a lot of requirements for RAID to be used, such as a separate controller that is a brand name and I buy everything in duplicate. Call me paranoid, but I've done several disaster recoveries and I just don't mess around with the less reliable stuff these days.
There is no need to run resize2fs after extending a logical volume. You can just add -r or --resizefs to the parameters of lvextend and then the resizing of the filesystem is done automatically. you eiter use mdadm to set up some kind of raid like raid 5 with 3 to 5 maybe six disks and raid 6 with more than 6 disks. Then you make physical volumes from the /dev/md device. If you have 2 disks you can do mirroring using lvm. LVM is used a lot in enterprise setups especially when the storage comes from some kind of NAS/SAN and is shared to VM's. It allows one to resize the storagage assigned to the vm host and then be flexible with the storage in the VM's. Resizing is used a lot; destroy and recreate is not used a lot; the basic rule is that data is only moved when there is no other way to do it. And if you need to move data you can always add a new volume to the volume group and then use pvmove to move the chunks of that volume group to the new volume and emptying the old disks. This can be done on-line without downtime. I'm sure stuff like this can be done with ZFS but ZFS is a userland process that can have serious performance problems. Also ZFS has some license issues with linux so it cannot be part of the kernel unlike lvm,
Really great video, Chris. I had largely avoided lvm, because I hadn't seen it all laid out in one place and wasn't exactly sure I knew what I would be doing. This makes the process accessible. Thanks much!
What is the difference between raid0 and LVM (in case if you lose a physical disk) will you lose all data? I used to use LVM a lot. But this thing is really bothering me
Yes, in this LVM configuration. Lose one disk and all data is gone. Backup! However you can use RAID with/within LVM for data security. You just need to set it up correctly from the start.
Check out "$ man inode" or a simple google search will satisfy all you need to know. Basically, it's just the building block of the file structure. It holds data about the file it points to such as filename, ownership, permissions, where to find the data blocks, etc.
You don't need to partition your disk for LVM. You can run it on a raw unpartitioned disk. Same with LUKS. I always do raw LUKS on an unpartitioned disk and then LVM on LUKS. Also, I prefer XFS. Sure, it can't shrink but it can grow and also dynamically grow its inodes. Plus, I still don't trust ext4 shrink. XFS performs better anyway with way more features like dedupe.
LVM sucks, parted sucks, volume/disk management on linux absolutely fucking SUCKS, it's written by geniuses for geniuses and I am a complete idiot. I am tired of reading pages and pages of documentation and watching tons of tutorials and everyone fails to explain the most basic things. Everyone will go from "yeah so this is a hard drive, plug into sata port" to "so yeah, LVCREATE AND YOU'RE SET."
I used to do the same thing. I would LVM everything including my backup folder which I symlinked to every reinstall. That is untill I bought an M.2 drive and suddenly half my sata connections disappeared including my backup LVM partition which was made up of three hard drives. At first I didn't know what had happened so I reluctantly erased and reinstalled my backup drive. I haven't used LVM since. But still I love LVM.Could you do a part explaining the snapshot process. I watched multiple videos explaining it but I still haven't got a clue what they were on about. Great video Thanks
on many motherboards, installing a m.2 drive will disable at least 2 sata ports. depends on which m.2 port you put it in. most boards have 1 m.2 wired to the cpu, and any more wired through the chipset, along with most or all the sata ports. if you install a m.2 into the slot wired to the chipset, sata lanes will be disabled. this is most likely why your drives disappeared.
It's nice video about LVM. In enterprise area for example at IBM where I've worked we used LVM on AIX and Linux servers also. Not just at IBM. Pretty easy to manage filesystems in this way. If you have a database or a log filesystem and not possible to delete anything, then LVM is a lifesaver for that DB or application. I save your guide. This is a very well documentation for linux. Thanks for!
I have a second SSD, instead of expanding, I would like to only have home and program files go to the second drive and not my OS drive. Is their a video with instructions for that? I have a fresh install of Ubuntu LVm 20.4 on a PCIE M.2 256gb. The second drive is SATA 1TB that already has files on it from my previous Ubuntu.
Agreed 👍 In light of zfs, it feels like LVM becomes a curiosity/legacy/oddity eventually. However, LVM might stay alive for sys-admins to release (reserved) storage to users, granted in an adversarial manner. I.e. keep doing it the way that they know how.
Lvm is too complicated (for very little benefit). Three "Primary" partitions and one "Extended" partition, (that can be subdivided into many partitions) should be enough for the average general purpose home user. I fear that potential linux users might decide not to use linux because of this kind of complexity. I set up a hard drive for lvm about 15 years ago. It took me several evenings to figure it out, and, after using it for a couple days, I took it down and have never had the slightest inclination to use it again. Make a video for Joe Sixpack that includes manual partitioning and a dead simple basic install. See how simple (yet flexible) you can make it. I'd make it myself, but, I'm way too old to be making youtube videos.
I beg to disagree. I have been using LVM professionally as a systems engineer since 1996, both under Unix (AIX) and Linux. Contrary to what you are saying, its benefits over the vanilla partition setup are enormous. I must admit however that it may be less useful for the standard home user who doesn't have to maintain loads of HD's and filesystems.
Thank you for your response. My concern is for the people who would like to use a "no nonsense" operating system, but give up switching to linux because of the "difficulties". Linux usage is extremely low in the general population, yet linux is now well over 20 years old. A big part of that is gaming, and "vendor lock-in", but there are a lot of people who just want internet for basic computing...email, web browsing, keeping records and inventory stats, pictures, music, etc. The old RedHat installer was easier to use than many of the more recent installers that default to lvm and encryption, and the UFI, EUFi confusion. It's great that linux offers "choice", but one of the choices should be simplicity.
And 640Kb memory should be enough for ANYBODY! :) I'm an old fart too my friend and I too can easily get away with ext4 filesystems on MSDOS partitions on a Legacy BIOS . That's how my current machine is set up. I've been doing it this way since my first Red Hat Linux 5.1 install in the mid 1990's. (Actually, it was ext2 back then.) However, it is useful for people to know that other options are available. Even learning the basic concepts without going ahead and implementing this in (and sometimes destroying) their own systems is good stuff. Hopefully, Joe Sixpack will eventually want to move beyond the basic setup of a minimally functional machine and delve deeper into the mysteries and black arts of computing. Anyway, breaking stuff is how we all learn best, isn't it?
@@dingokidneys My first linux install was RedHat 6 on a 2Gb partition on a 4Gb drive...dual boot with Win98. I think they call that "shoe horning". I didn't know a single person that was using linux. Twenty years later, I know 4 or 5 people using linux. That's not what I call ground swell of linux adoption.
Why isn't it used in business? LVM supports all the RAID levels. Plus, you can setup caching. A majority of my servers use a RAID 50 of HDD with a RAID 10 of NVME that act as a writeback cache to the RAID 50 of HDD. It allows me to get NVME performance for HDD price 99.999% of the time.
Hi Chris, great tutorial on LVM. It would be great to see you make a part2 that would cover resizing logical volumes or replacing a physical volume. One remark: you formatted the partition with ext4 which is not needed I think because the FS of the logical volume will be used; pv create also says it is wiping the ext4 headers (and making it an LVM partition type)
When setting up the partition, you need to identify the filesystem type it is to house. Just part of the required partition meta-data. This is then happily overridden by the LVM tools with the real filesystem.
@@dingokidneys That is incorrect, the only reason Chris needed to specify the filesystem is because he's using Gparted, which is a high level tool that combines both partition management and filesystem management. If you used a lower level partitioning tool like fdisk, then you would not need to set a filesystem. Also, AFAIK filesystem type isn't part of partition metadata, but rather filesystem metadata.
Hello Christ.. Great videos!!... Is it possible to create a Mirrored Volumes with LVM?... I have a WIndows PC with 256gb SSD as C: and added two 4gb partitions Miirrored since they will be used for backups. I would prefer to use Linux instead of windows.. Windows Partition Manager converted them to GPT and gave me an option to create New Mirrored Volume. Thanks in advance.
Has anyone here tried caching with nvme ssds? I’m playing around with one of our old servers and get worse r/w with dm-cache than with the an ancient raid controller alone. I have tried dm-cache and dm-writecache, writecache won’t let me activate the lv and shouts about memory and dm-cache halves r/w. 12 x 4 TB HW r6 1.7/2.2 r/w vs cached 0.6/0.3.
Now I have spent a month in Ubuntu. The only thing that went wrong once was when I rebooted and the reboot hung with a blank screen and my OS never appeared. So I put in a Linux live disk and, weirdly, it said that there was no prior OS there. Even though if I ran gparted there absolutely was my OS still there! I had set up as an LVM system install, something that I had never done before. I tried boot repair. That didn't change anything. So in the end I did a total reinstall of Linux. This time I used a normal Linux filesystem. There was no issue rebooting at all. So I think that LVM just does not work on my computer which is a bit old [2011]. Despite this I am happy that I went Ubuntu. More chance of joining the Moonies than of going back to Windows.
I installed with LVM but when running vgscan it says "reading volume groups from cache"... and that's it! what the heck?! I also don't know how to build new volume group since I cannot include /home into the group!
I am a bit curious, what happens if I use USB stick as LVM extension on a laptop and turn on the laptop while accidentally removed the USB? Will the machine boot normally after I plug the USB back and reboot?
Super gluing drives together, performance nightmare, lvm2 on raid0 or raid5, with f2fs filesystem for ssd/Nvme or xfs/Zfs for hdd should increase performance.
New comment /old vid... this did not work entirely as directed. Somewhere around "lvextend -l /dev/ sd#" the world essploded....but as always great tutorial.
This is good. I was trying to do that using KDE partition manager or even with Gparted, and I could not do it. Only this worked. Why though? Thank you!
Chris if we installed ubuntu without LVM and just with standard installation, than how can we do it from there, In this video you have already setup LVM and taught us that how to add and increase space, but in case we installed without LVM support, than how can we do that. Please
Hi, nice info! I was just about to add another disk to my system. I have some questions: 1. Can I mix SSD and HDD? Suppose that I already have a LVM partition on a SSD, can I extend the volume with the new added HDD? 2. If (1.) is possible, is it safe to do so? Any disadvantage of mixing drives? thanks ^^
After watching your video, i decided to check it. but not ready to test on my lap/Computer. so i purchased a dedicated server from a data center and its have 3 x 2 TB HDD. i setup the server with NO RAID. so i can use the three individual disk. Operating System used for testing was Centos and Ubuntu. 1. i untouched the sda disk 2. created single partition with sdb1 with 100% Storage. Around 1.7 TB Can use. 3. Created single partition with sdc1 with 100% Storage. Around 1.7. TB Can use. 4. i create physical volume successfully. pvs PV VG Fmt Attr PSize PFree /dev/sda5 kvmvpstorage lvm2 a-- 1.67t 1.67t /dev/sdb1 backup lvm2 a-- 1.82t 0 /dev/sdc1 backup lvm2 a-- 1.82t 0 root@rescue:/mnt# 5. successfully created volume group also vgs VG #PV #LV #SN Attr VSize VFree backup 2 1 0 wz--n- 3.64t 0 kvmvpstorage 1 0 0 wz--n- 1.67t 1.67t root@rescue:/mnt# 6. Finally i created 100% volumegrop to logical Volume lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert lv1 backup -wi-a----- 3.64t so i got the result of a total storage of 3.64 TB and mounted this lv1 to /backup But, after few hours, i restarted the server. unfortunaetly the server is not booting/working. On detailed inspection found that the newly created mounted lv was missing. so i logged the server via rescue mode and added the details or mount id in /etc/fstab. But i dont know how to remount it again via rescue mode. if you know it, please update it.
LVM is neat. I used two 256GB USB flash drives as one 512GB because can buy two for a way cheaper price then one 512GB USB flash drive. Today can get them for cheep price at lest for 512GB SATA SSD 3D NAND flash about $50.
What are exactly the Pros and Couns to use LVM? When you have to use it? I think is not so safe and you had many point of failures Hard to make reliable backups.
A mirror would not give him the additional space which was what he needed at that point. I also have no use for LVM at the moment but it is a useful technology to know something about.
hi Chris, good content !! i think zfs its the way to go ?! it is evolution of LVM, zfs combine LVM and partition tools, its very powerfull, already use with freenas box. I think the evolution path is LVM, XFS and now ZFS ... i ve miss someting ? thanks to put me on the right way !? :)
ZFS is great with the down-side that most Linux distros will not boot from a ZFS drive. You would have to compile a kernel yourself with ZFS support in-built. Ubuntu is to my knowledge the only distro which will boot from ZFS.
Depending on how you set up LVM you can end up with either a proper RAID setup or with JBOD (just a bunch of disks) the second being in essence what Chris set up. I think he said at the end that just extending volumes the way he did, while achieving the desired result of expanding storage, introduces risk in that you have two points of failure rather than the one you had before. Good backup is vital if storing any data that you care about on such a system. If you build a RAID incorporating redundancy (RAID1 or RAID5+) and then use LVM to allocate that space across your file systems, you should have a secure and flexible setup. Never the less, backup. BtrFS and ZFS essentially combine the functions of RAID and LVM (plus the filesystem), so you wouldn't use BtrFS or ZFS along with LVM or Linux software RAID.
Thanks for sharing the video. I followed your video and tried to increase to increase the swap partition to 8GB, but it's still showing 1GB and when i type lvscan here is the output i see ACTIVE '/dev/vgubuntu/swap_1' [
Super helpful comments section to accompany a pretty fine video 👍 1. Pass -r to lv extend; elim. resize2fs 2. Pvcreate doesn't need a f/s prerequisite or even a partition. Indeed pvcreate can ingest any block device such as a disk (partitioned or raw) or a partition (whether formatted with a f/s or not). Kindest regards, neighbours.
