Thanks man, your videos are just perfect nuggets of very useful information which are very hard to figure out otherwise. I hope more people appreciate all the time it takes to figure out such seemingly obscure quirks for such a short overall video. P.S. One thing I have noticed with HDD spin down and ZFS is when it needs to be spun back up for access it takes longer with the more HDDs you have in your pool as the HDDs are spun up sequentially as opposed to all at the same time with ZFS arrays, which can in itself be a disadvantage should you have a large pool of HDDs running ZFS.
Thanks and great video(s)! I wanted to mention (if anyway is interested) that one does not need to actually have a VM running Truenas let's say, with passed HDDs/LSI HBAs, which are then further shared over samba. Just having 2 pools in Proxmox as you suggested is enough. One for SSDs for the LXCs and one with HDDs for the big stuff. Then bind mounting various datasets and folders to the LCXs will not make the drives spin until those files need be accessed. Example: an LXC with Jellyfin with rootfs on SSD but with a bind mount points on the HDD for movies will spin up on playback.
Thanks for this as I am moving my media files (big, don't change much) to a Snapraid/Mergerfs (using xfs) on a Proxmox server and I wanted to spin those down when not accessed. The difference is I'm using SAS disks so I might need to use sdparm.
I've got an HP Ultrium LTO4 tape drive in my proxmox backup server and I've been playing with that for a few months. It would be interesting to see your take on LTO tape drives in proxmox backup server
This video was great for what i was looking for, the only question i have is regarding proxmox and truenas. How did you pass the harddrives into truenas nas and was it in truenas where you set the spindown commands or was that done on proxmox ?
There are multiples ways to passthrough drives to truenas in proxmox. The most ‘correct’ way is probably to passthrough the disk hba or controller via pcie passthrough. Then truenas will have full control over the disks and spin down. This will also require a separate controller for proxmox disks and truenas disks. You can also give truenas virtual disks via virtual disks or passing through a /dev/sda or other proxmox disk to truenas. This will have proxmox manage spin down and control of the drives in this case.
@@ElectronicsWizardry First of all thanks for the reply. Since I dont have an hba to pass through to the virtualized instance of truenas im running on my proxmox server I tried managing the hdd spindown from proxmox directly like you mentioned. It worked beautifully for around a week, the HDD would spin down after the set amount of time (2 hours), but for some reason that only worked for around a week I cant get it to spindown on that timer anymore. I changed nothing and I even made sure that no other VM or container was using that share but it still will not spin down, even when I force spin it down proxmox seems to wake it back up. Would you happen to have any pointers on where to look?
Looking for some advice please - I am running a proxmox PC that is drawing about 140 watts average at present - there are long periods in the day when it is not required, whats the best way to save energy without damaging or shortening the life of the PC
What hardware is your Proxmox PC using? With the hardware specs you can make a good estimate of whats pulling the power, and reducing the power of what draws the most power. Id probably setup spindown after 30min if HDDs are a large amount of power draw. Then they will be spundown when the server is idle, and there won't be an excessive number of cycles on the drive. Wake on lan could be setup if you want to save even more power. You could turn the PC off when your away, and have it turned back on when your ready to use the system.
I'm running 2 separate PVE nodes, each with a different NAS OS. One is TrueNas Scale and the other is Openmediavault. OMV has an internal spin-down command. Not so sure about Scale. At which layer should I execute the spin-down? At the Datacenter level of PVE or within the VMs themselves (Scale and OMV)?
The spin down and other power control commands for the drives normally need to be done on the host instead of the vm. The proxmox host won’t passthrough the commands given to the virtual disks to the physical disks storing data. The exception to this would be if your are passing a drive controller like a sas hba to a vm where the vm would now be in control of drive power commands.
could you make a video about how to virtualize unraid on proxmox this way you'll get best of both world if you have multiple hard drives with different sizes proxmox to handle vms and docker unraid to handle storage
I've already lost 2 seagate external hard drives of 4TB (they weren't iron wolf) that were very little accessed in 5 years, and the data written on them occupied much less than 2TB. I also had issues with unrecoverable data on SnapRaid. I'm thinking of doing the following, using 1 parity HDD, and 2 data HDDs, with SnapRaid, all ext4, and for greater security I want to have mirror HDDs for the data, leaving me with 5 HDDs (all WD Red). From my personal experience 3.5'' HDDs give less problems than 2.5'' HDDs, I don't like external HDDs because of the possibility of accidentally turning off the HDD when they are transferring data, so I was decided to use them as internal HDDs and hdparm (or set 15 minutes of stand by on Disks), however, I was afraid after seeing your warning about the wear of accelerating and decelerating them. Basically I will power up these mirror hard drives once every 7 days, they are simply for backup: what do you recommend?
I encounter a problem is that: when I constantly(every 1 minute) query "hdparm -C /dev/sdX", my seagate ironwolf drives won't spin down, as long as I stop querying this command they spin down properly. After they spun down, I query "hdparm -C" again, they won't spin up again. so I guess I will just trust them, and stop monitoring the spin down state. unraid use hdparm to spindown and monitor the drive state too, but they don't have the issue. it's weird.
run 2 independent nas with raid arrays - leave them on all the time- have a ups on each and also email notification and logwatch daily reports, the rest of the network is ssd or nvme - do run caching on nas and also with web servers (cache just static) - this can take off 90% of load on server and boost req and latency/web page speed #ab #hdparm #--assemble --scan
I typed /dev/sdb: issuing sleep command this is the output -bash: /dev/sdb:: No such file or directory your title is "How to setup HDD Spin down in Proxmox VE" I'm sorry but I don't see any directions of "how to setup" which latterly in your description do you have an other video that is this shows how to do it.. did I miss something?? its a how to video
you could turn off ext4 journaling to prevent drive spin ups_ but you'll risk data corruption in case of power failure ... tune2fs I guess is the command that does the job ..
servers are rarely idle - consider a web server that gets 10k hits a day which really isn't that much but you may as well just leave it on - a raid array is going to use like 5w/drive - it would take 10-20 drives to even equal 100w lightbuld - most electronics will fail with lots of off and on cycles vs just leaving devices on continuously...this does not apply to nvme obviously and nvme is the way to go - much lower latency and virtual price parity with regular ssd - nvme scales well over the network which is why you geally want to go to fastest networking you can manage - 2.5 is gud, 10 is better, 100 is best and 100bonded/bridged is very best - pls try a opnsense ha setup with some sbc - failover and link agg #triwan
other than doing a ha opnsesnse why not do content on the entire enchillada which consists of 2 sbc for opnsense 2 nas devices (can also be sbc) - sep nas with raid arrays for backup redundancy a sworkstation of some type can be 4th gen intel with some extra ram or something like ebay z440 - supports booting from nvme and bifurcatiion so you can run 4 nvme on one card a sniffer box - selks 6 may be better than selks 7 no docker needed...this will do 24/7 pkt cap for all lan devices - suricata/moloch 3 cluster nodes - qemu/kvm or proxmox - live migration, load balancing, mysql cluster (postgres cluster?) and netfs - you can spi9n up vm on ws to avoid quorum issues if one node fails 2 pihole on sbc, run recursively all told 11 devices but this system is what smb needs - with nvme and faster networking if possible - smb sector can easily scale - team it up with vps for r proxy - this is a good setup for builders to sell - build for 3k sell for 4k or something and build and ship it in a day - everybody is happy
Zfs is a pig with lipstick on anyway. Built a 5 drive 8tb zfs2. Hba all connected with sas. Brand new drives created it left it a few days to check it was stable. Transferred all the data I had to it. Left it a few days and all good so removed the external drive data on a different machine. 30 min later the whole array broke and cannot be recovered. I was just in the process of setting up a one drive sync in lxc to send folders through for a cloud backup. The drives still work fine a year later. But I will never use zfs again. I will stick with large drive and equal volume of one drive space.
@@ElectronicsWizardry the caching system is what makes zfs so great spinning rust is fine but can you serve up from cache w/o having to spin up the rust? it is sort of like using caching r proxy - you can hit the static cache and site will get served up - it won't even goto disk until there is some db req - it can improve perf by up to 90% #zil
