Whats the faster VM storage on Proxmox 

For those interested, Wendell just did a "what were learned" review of Linus' (LTT) PetaByte ZFS drive failure - "A Chat about Linus' DATA Recovery w/ Allan Jude" - ZFS got another development boost (with more coming) as a result ..

I feel like i have learned more from this 8:55 second video than i have scouring forums for hours and piece mealing things together. thank you for the straight forward video.

One advantage of LVM over ZFS though, is that you can share it across hosts. If you have a cluster using shared iSCSI, FC or SAS storage (where every host sees the same disk) you can put LVM on that disk (on the first host, use vgscan on the rest), add it as shared LVM in the GUI and all other hosts see the same volume group. Allocate VM's out of that group, and it's easy and quick to do live migrations. ZFS cannot do this.

Thank you for an amazing, straight to the point, and concise video. I have actually been spending a lot of time trying to put together all the bits and pieces of what you managed to put into this fantastic video for a project of mine I shall be undertaking soon. Thank you for the time you put into collecting, and presenting all the benchmarks. You have a new subscriber.

Not gonna lie...I need a whole new vocabulary lol. Thanks for the explanations. I kind of dove face first into the world of Virtualization and wow do I need an adult. I bought some pc guts off a coworker for $500. 64 x AMD Ryzen Threadripper 2990WX 32-Core Processor, 64 Gigs RAM (forgot the speed/version), and an ASROCK MB. I threw in 24TB of spinning rust and now learning how to VM/setup an enterprise. End goal...stay employed lol. Thanks again for the help.

Finally a video that is straight to the point on what I wanted to know. Thank you

Great video. It is worth mentioning that it is possible to use the same ZFS pool to store all kinds of data (vdisks, backups, isos etc.). The user may create 2 datasets, and assign the first dataset as zfs storage and the second one as a directory.

Great video! Thanks

Thanks for a great video. I found ZFS to be the best way to go.

I’ve been trying to learn Proxmox the past couple days and this was SUPER helpful. Thanks a bunch man. Strait to the point and you explain your opinions on the facts presented.

Thank you for these videos. Very clear and answers the questions that come up as I'm listening. Satisfying!

Really love your content i hate that channels with way less info but just do flashy edits get the attention when the guys that know their shxt don't get the same views

I am getting to the point of realization how much Proxmox is not anywhere near ready to be competing with Vmware. The way administration works, the absolutely bad documentation and all the resources online are just so jank... Too bad. I'm considering even aborting switching my homelab over. I see no benefits even to just running the current ESXi without patches indefinitely.

btrfs changed since this video was made. it should be a way faster today. i wish for an updated video....

Ceph RADOS is definitely the way to go, I hope that the performance for BTRFS is improved in the future, I do not really care for RAID5 or 6 and prefer 10, 1 or none generally anyway. BTRFS send and receive is a killer feature. I prefer that BTRFS is licensed and in kernel, this make booting and recovery senarios based on BTRFS potentially better with some work on proxmox side. cross fingers for BTRFS.

I have a sql query taking 15 seconds long on a vm in proxmox stored on a nvme ssd. I created 3 other vm all running the same os but on different file system ext4,btrfs and zfs and installed only the mariadb server serving the same database but via tcp and i could not get the performance of the first initial vm. Any ideas why? I get close to performance with an lxc container only.

Hello dear EW, please can you again review Proxmox 8 with ZFS vs BTRFS performance?

5:00.. After many years, the BTRFS project is still considered unstable. Despite this, Synology uses BTRFS in its commercial products.

This was an excellent video. May the algorithm bless you.

Great video, thank you!

Informative. Thank you!

Nice content. Thanks for sharing. Is there any comparison or deep studying about centralized storage with iSCSI or fiber channel ?

If you use ZFS, does that mean you would not want hardware RAID on the physical server?

At 7:07 slight performance advantages? Performance is 3x faster with PCI-e passthrough than with ZFS or LVM. Although, I prefer ZFS as well for the flexibility.

No idea what was just said, but I’ll piece it together eventually 😂

Thank you once again for the excellent video and for sharing your knowledge with the community.

Hi. Is it possible to "pass" zfs-directory or zfs-subzfs to guest. The main idea is, that zfs is filesystem and guest needs to have own filesystem (e.g. ext4) which is overhead. So only the host should be doing filesystem operation, while guest would see as folder. I guess zfs would have to support some kind of server/client infratructure but without networking overhead...

Thank you for sharing your time and expertise with us. Insightful anf informative. Clear and precise.

Great video, thank you!

Nice presentation of what's available and pros/cons... good vid! Will stayed tuned for future content...thx.

I'm in the process of building an nvme Proxmox server using a combination onboard nvme drive w/ 4 x 2TB nvme in zraid 10 That said and based on your experience, would this be the optimal way to go for vm's? ps. having read multiple posts or comments on ssd wear, I remain a bit worried on my setup choice as I'd like to get the most out of my storage system without sacrificing the life of the devices - ie, 3 years would seem reasonable for a refresh imo

You are a smart Cooke. Thank you, this info is very helpful.

Thanks!

This is a fantastic primer on file storage for proxmox.

I can't get decent speeds on proxmox with my nvme or HDDs. I'm getting a max of 250Mb/s on a 4 hdd raid 5 array virtualized even with PCI passthrough but unvirtualized its 750Mb/s. Even my NVMe drive virtualized starts off at 800Mb/s then drops down to 75 - 200mb/s and fluctuates. I'm running the virtio SCSCI controller. Why are my speeds slow?

That statement you made about the layers you need to adjust individually is not reflected in any graphs anywhere, and thats a shame, because it clearly demonstrates _another_ main benefit of using ZFS over LVM ( yay, look BTRFS is way out in front, oh wait .. ) Not sure how to take the "ZFS fakes Proxmox cache setting", for testing non-cached it _is_ relevant, but that is _not_ a real world scenario, to the extent that you could attach a drive/device which has _no physical cache_ and ZFS will still happily cache that device, a more authentic real world scenario ( _if_ you could indeed find such a device). The _best_ part about ZFS, as Wendell showed and admitted, when your (especially raid) drive pool goes belly up, to the point software tools can not even help, you can still reconstruct original data by hand if need be, as _everything_ is there needed to achieve that .. BTRFS _might_ "get there in the end", as ZFS has had an extra 10 years of use, testing and development up its sleeve, but those BTRFS "features" that have not been "re-aligned" for years, means it's never going to be a practical solution, except in isolated cases, its better of being used for SD-Card filesystems, where it can extend the limited life span of the device (if setup correctly), and speed is already a physical issue (as long as you dont want to use said SD-Card on a windows system .. ). thanks for taking the time to do the review ..

Bro I've been researching this topic for a couple hours on and off each day. Thank you for just combining this information into one video.

How does ZFS compare to Ceph?

Ahh thanks! I was looking for just this kinda video.

great video, straight to the point! Confirmed my answer by :30 but stuck around to see how they all performed.

nice job

Thanks, very useful info in this video.

What file system do you recommend for a server whose storage needs keep growing at a high rate. LVM?

awesome overview and cleanly laid out.

Ceph has entered the chat

Well done! My data always resides on ZFS (FreeBSD, SmartOS, Linux).

I feel like there's a code in your blinking. Maybe Morse code?

And what about Ceph?

what about ceph

Can you still use ZFS for storage option if you didn't install ProxMox using ZFS format?

i don't quite understand the remark that ZFS can connect "to the physical disks and goes all the way up to the virtual disks" at 4:13. I mean, doesn't lvm/ext4 in proxmox provide the same? i'm trying to create a ubuntu VM with a virtual disk formatted as ext4, is this correct? if not, is there a demo showing the "better" way? thank you

What about draid?

big takeaway here is you want a nas with lots of ecc mem and zfs - a z440 with 256gb ram is about 1k making it a great deal

You need to increase the wait time in your blink-function.

Hello, I'm installing the new version for the first time on an HP server with 4 300G disks, I want to know the recommended option for using the disks, keep proxmox installed on a single disk and use the rest in pool zsf mode for the vms? What option do you recommend? Thanks Best Regards

Ext4 isn’t faster than Ext2 because it is a journaling file system like NTFS. Journaling File Systems have overhead from the journaling. Like it wasn’t good to have a LUKS on ext3 or ext4

I normally watch your videos in the background but actually focused on this vid today and noticed how much you blink which, no offense intended, i found a bit distracting.

Why blink so much?