I'm new to this channel and was wondering... anyone else find this kinda, well, soothing? The steady, calm delivery works! Great video, really enjoyed the content.
I built a virtualization server using Proxmox a few years back using the same motherboard and a similar 2u case a few years back. Added an old quadro w/ PCIE passthrough for video transcoding and the thing works like a dream. It's been awesome a first "real" home server, running absolutely everything I need (with exception of a separate storage server) and has had pretty much zero downtime since building. The one thing I will say for anyone looking to use this board or similar (if you can find a deal on a several year old board), that passive heatsink really doesn't cut it if you aren't in a 1u case with loud high static pressure fans. I ended up designing a 3d printed fan mount that just pressure fits on the top of the heatsink, with a 20x40mm noctua fan blowing down on the heatsink. I haven't seen the CPU exceed 50c since installing (used to hit high 90s and throttle), and everything is still nice and quiet.
Hey Jay, the proxmox team added a way to apply network changes without needing a reboot. Click apply configuration button. However, I should note that it will ask you to install a package, which will need a reboot (but only one time after installing it). Now you can make future network changes without ever needing a reboot.
@@LearnLinuxTV Its worth noting that the apply changes will cause all running vm/ct to loose network status so you will need to restart them or just restart the host machine like you normally would
Thanks for the video. This is bit expensive. I have used Lenovo Tiny 93P got from ebay for US$125 and changed to SSD256BM and 16GB Ram. This set up cost me less than US200. It is core i5 Gen 2 cores 4 threads and work well for over a year running a) Debian-> Docker-> Home assistant to control 200+ devices running heaps of automations b) VM to run Blue Iris on Win7 with 10 cameras + Unifi controller for my unif network. I do have a HP Gen8 server with SAS drives (4)+ 8 Code (2 process) +24GB system configred proxmox but not running it as it uses over 250w and my Tiny PC uses less than 40w (my network rack use only 75w with Tiny 93P+ Synology 213 NAS+ Unif USG+ 2 PoE AC-Pros+ Modem+ Netgear Switch+ 3 Hubs for energy monitoring and home automatons). I am sure your system use more than 50w. So there is cheaper ways to get thing going and that does not need lot of power.
Wow this is fantastic I’m in the planning phase of upgrading a clients existing HP DL360 G8 servers. Now I’m thinking I may build them some. Also planning on moving them from ESXi to Proxmox. Thanks for sharing everything.
@@_TbT_ There is a model that does, but the one in the build in this video is no fan and he specifically says that in the video. With that said, I am sure the one case fan will work pretty well in this case.
Agreed - That chassis had room for one more fan at the front and it would be wise to add another couple elsewhere (even fixed together). Once those virtual machines start getting busy in real-world situations, processor temperatures rise exponentially, getting really hot - then you need multiple fans to ramp-up and push/pull all that heat out. It didn’t seem likely that was ever going to happen in that guys’ set up, though.
Nice server. You can do this on a budget if you go with used server parts from ebay. Or to do it cheaper and really low power, do it on a Raspberry Pi 4. The model with 4GB of RAM can host six or more containers. Under $100 and 5 watts. The Self-hosted podcast is a great source of info. And Rapberry Pi are the only way to build a cluster without going broke.
Yeah thats kind of what I am thinking of doing. Though I do find it annoying that most of the servers I see on ebay seem to use 2.5 inch hard dives, which does drive up the cost of storage a bit.
CaptRR rather looking for a complete server, consider looking for suitable MB, RAM, and CPU. Then put it in whatever case works for you. If you don’t need rack mount, and you want space for a lot of 3.5” drives, a Fractal Design 804 (8) or Define R6 (11) or R7 (14) are options. With the Define R6 and the R7 you will have to buy more drive sleds as I think they both only come with six sleds. And with all of them there’s room for three more 2.5” drives.
CaptRR look for SuperMicro X10 series motherboards. One generation back from current. Use DDR3 ECC RAM which can be had off eBay for about $100 for 32GB. Most models have 6 SATA ports but the X10SL7-f (mATX) has a built in 8 port LSI controller for a total of 14 SATA ports.
For arround 200-300$ you can buy Fujitsu PRIMERGY TX120 S3P. Small Form Factor, Xeon cpu (like ex. E3-1265L), ECC Ram, 4x2,5'disks bay, 2xGbps nic, remote access. I think its even better deal than HP Microservers.
Absolutely, and my power usage has gone down quite a bit since I've started lowering my power usage. I've been looking for cost-savings throughout the house when it comes to power.
I forgot low power AMD EPYC SoCs existed! Also, I can't remember when in the video you mentioned that ZFS isn't for everyone or something along those lines, but that reminds me of why I used BTRFS. It's hard for me to believe that I initially decided on hardware RAID when building my first custom NAS, but used BTRFS RAID instead, and when the secondary SATA controller on its mainboard died, I was still able to recover my data on another mainboard thanks to myself choosing to go with software RAID instead of doing it in hardware. That does mean I replaced the mainboard in my NAS, but because the rest of the previous mainboard still works, I was able to repurpose it in a different case with different components as a secondary always-on PC, that just like my NAS, runs regular Debian, but unlike the NAS, has a limited set of components in order to free up more system resources since that second system would just run one specific service, whereas the NAS runs multiple services and even a full Trinity session over a VNC connection.
@2:54 minor nit-pick: "order of magnitude", unless explicitly specified otherwise, is factors of 10. 11690 is not even one order of magnitude larger than 4338
@9:15 the IO-shield is not so much about airflow (notice that in your case you have a large vent right above it) but for EMI shielding. Also, if power consumption is of concern I'd rather use a PSU with Gold or Platinum rating. Especially under partial load those are much more efficient. Alternatively: dont oversize the PSU that much. If the system is only going to draw ~50W you're better off with a PSU in the 150-200W range, as it will run closer to its most efficient operating point (usually around 50-60% load). @30:45 the updates you get without key are those from the base Debian system
@@Illuminali4all Well, it does make a difference if - under partial load - the PSU has 50% (which many cheapo PSUs have below 30% load) or 80% efficiency. E.g. the server uses 100W - PSU1 (50%) will draw 200W from the wall, whereas PSU2 (80%) will only draw 120W.
I have a vmWare ESXi 6.7 virtualization server running since 2015 now which only consumpts 17W with 4 vms running simultaneously (2x win10, 2x ubuntu). (CPU i5-4590T, 8GB RAM, 2128 GB SSD Storage, 90W PicoPSU, Asrock Z87E-ITX Mainboard in an Kolink Satellite Mini-ITX case)
Enjoyed low energy virtualization and low energy storage server videos. Is there a reason not to do both in the same hardware? If not, maybe a video on a combined machine? Case and MB suggestions in particular.
No reason in particular, it all depends on what you want your homelab to look like. Your storage server can be a VM if you want. But then, you also have your storage on the same physical device as your VMs. In my case, my backups are on a separate storage server which is important to me. But everyone designs theirs differently.
Whoever may need this build guide; the part they need is exactly what happened @ 16:03 off-camera. The people that know how to do that part don’t need the rest of this video, so I’m not entirely sure who this video is for
So, now that you're almost 2 years in, how did you end up liking the AMD Epyc 3251? I liked this build and was thinking of using the board for a similar project. It's a nice combination of power efficiency, upper RAM limit, and compute. Anything new catching your eye?
I built a cluster of nearly identical servers to this almost six months ago. I need more RAM (64G isn't enough) and the crucial sticks used are dual rank, which kills the speed when four are used. I'm looking at two 2r8 64G sticks.
@@foxale08 Nice. I'll be doing the same very soon for the RAM. I ended up buying this board and built a Proxmox server as well. I hadn't been real familiar with it until your videos, and so far I really dig it. Running like a champ! I've had so many brown outs, surges, and moderate power loss in the area, so it's been great having confidence that my UPS keeps it up with minimal power consumption.
@@GilbertMendoza Not my videos, just providing helpful feedback lol. I personally use APC SMT2200's but am leaning towards SRT3000's or even a small APC Symmetra.
i have a passively cooled virtualisation host, at max it was able to run a minecraft server with 4-8 players and 276 Mods, now its just running a webserver and mailserver. Im Saving up for an epyc build, in the next 2-3 Months i hope my little HDD and SSD dont fail XD
Please note that usually when a server part (CPU, HBA, NIC...) comes with a passive heatsink it doesn't mean that this is enough but requires some amount of airflow to be properly cooled. I strongly suggest you to check the CPU temperature because if in idle it should be fine, I can almost guarantee that under load at 55W and without any airflow it's thermally throttling like crazy!
Noted. I will keep looking at it and take your advice if I notice any issues with that. Do you think the heatsink is easy to remove and replace, or is it proprietary? If it's necessary, I'll investigate this.
Watched a few of your vids today, i like your style. Id puff this joint with ya for sure. And respect on the retroarch boxes...i have one in each room as well =)
For common home use is enough to use old laptop with i5 or i7 and 16GB or even better with 32GB of ram. It will be much more cheaper and is a good start. You really do not need to spend 1K$ to start. Just make sure that the processor is normal M, H or HQ version. The ultra low power U versions are not very good for this task. Proxmox is definitely what you want to go for. Its easy to use free and stable. PS I am using i5-3230M with 8GB RAM and 500GB HDD and it is working 24/4 for a year with no downtime with 3 VMs and several containers and have still overhead for more...
VERY good point! Funny you'd mention this, I recommended using a laptop to a friend of mine recently, whom was concerned about power usage as his house has sporadic and unstable power. That's another use-case for a laptop as your VM server as well.
The sticker with the password is a consequence of a relatively recent law in California mandating use of more secure default security settings. Given manufactures have to use unique passwords there they might as well do it for everywhere.
I also did 55W 4 years ago, but i went for cheaper option. I am using ASROCK j1900 motherboard with 16GB of ram, the TDP of the CPU is 10W. To that i added 1tb HDD for proxmox and 3x 2tb for ZFS NAS, currently on openmediavault with sata passthrough. I have added a second NIC to access two networks because i did not wanted to go VLAN, and recently a 5th HDD, it's a 10tb drive for cctv and computer backups. I am waiting for mini pcie SATA card to attach it, currently it is on USB adapter, without the 10TB i measured it to be also 55W with 4 HDD drives. I am planing to replace it as soon as i will purchase a solar power installation with my HP z620 workstation with Intel Xeon 12 thread CPU and 16gb of ram. The cpu is 10 times weaker than yours but it was fine even for plex transcoding from 1080p. And i also run Proxmox ;). For network i went Mikrotik, with HEX S as a main router (because of it's 450mbit/s ipsec performance) and two RB951 series for WiFi and i am waiting for hAP ac2 for my main wifi since those two are meant to serve all the smart things in my house.
I run proxmox also and this video is great. However, i looked up the price of this board and wowza. It's not a cheap one! My proxmox runs a Ryzen 2400G and for my purposes it works exceedingly well! Furthermore, will the cpu run cool being passively cooled? Especially under load?
Great video. I am not sure if your power usage info is correct. 50W is very low and only a machine like Lenovo Tiny will come close to it. My Tint M93P with 16GB and 256GB SSD with no CD and one NIC uses about 40W. Core i5 4570T is rated 35W. How much power your Dell was using? Are your measurement power usage monitored from power plug. I have all my devices connected via Sonoff P2 plus with Tasomta -> Home assistant. I am planning to migrated to a Dell Optiplex 7040 Core I5-6500 rated 65w so based on what you are saying that low end SFF will use more power than users (In that I have Nvme drive). I do have HP ProLiant DL380 G7 Server 2x Xeon L5630 2.40GHz 24GB RAM 4x 72GB SA and has two power supply of 360W but I have not tested power usage. Could you kindly measure your actual power usage for the server as I cannot belive with cooling fans etc yours can be 50W
1000$?? Man... Are you serious? 1000$ for a homelab, when the market full of used Xeon processors starting from 15$ and whole servers staring from 150? I have an hp dl380g7 with two 6(12) core processors for even not a 200$, which runs 5-7 guests in a mixed unix-win environment and has a consumption of about 100W. 36gb of RAM, 2xHDD, SSD, raid controller - everything together. For an about 3-4 hundred I can build a top 2011v2 or even v3 Server with a same power requirements. 1000$ for a AMD 1xCPU solution.... Yeap... )))
Hello Jay, thank you for this content. Could you please share your approach for backup Proxmox server settings and how restore them in case of storage failure. Thank you.
Thanks Man. Very good vídeo. Already subscribed. Proxmox runs like a charm. I dont know some many people uses other things like xcp-ng or unraid. This was my First video of yours, and i dont know If you already make some about, but i like to view something ok CT for proxmox, like domain-controller, or file server....
@8:38 Although it's a passive heatsink, the system should have adequate airflow going through the heatsink. Chassis fans are required for that type of heatsink. You just can't run it hoping that passive airflow will be adequate.
I did have an issue though. I don't remember the error now. But when I looked it up, a lot of people were having the same issue. I agree that it shouldn't be a problem, but it was.
@@LearnLinuxTV I've also never had ZFS issues on non-enterprise equipment, either. I'm running pretty much purely consumer drives, with no errors. I *am* very incredibly careful to avoid SMR HDDs, which *will* cause major problems with basically any form of hardware or software RAID, though that's getting harder and harder every day without spending more and more per-drive/GB.
TDP = Thermal Design Power, which means the Thermal solution you need to use to cool the CPU, should be able to at least dissipate the number of watts listed. Recommending a better cooler capable of better cooling than that. Intel measures its TDP at base clock, while AMD measures at max boost clock.
Not trolling, genuinely want to understand the thinking here ... I'm not quite sure why the drive for low power here? - the cost difference shown for running those cpus was only $4 per year and the spend of $300+ on the new CPU doesn't seem to stack up as a cost driver. As comments call out, that board costs a fortune. So, I'm interested to know why the driver for low power usage here. If its eco friendly, suspect you could plant a lot of trees for $300. So if the old one was all good and working, not sure I get the push to swap it all out, but have low power as a major selection factor.
Low power usage was just one factor, not the entire reason. I also wanted something quiet that can be in the studio with me, benefit from some speed improvements, and overall just test what low-power is like in practice. I realize that it's not practical for everyone.
I wonder - How are you CPU temperatures? I just got the same board and seeing (what i think) is too high temperature of around 80c, I just ordered a "Noctua NF-A6x25 PWM, Premium Quiet Fan, 4-Pin (60mm, Brown)" and going to see if that will help. I tried a 4cm same fan and that reduced the temperature to around 32c
Some random Best Buy fan? Why not a Low Power Noctua that will last? Even those 2 extra ram sticks will increase power draw unless 4 channel memory will increase power per performance
@1:22 55W here is the TDP, which stands for Thermal Design Power (or Point). You can't use this as a power consumption/usage value. It's a completely different characteristic.
I've been waiting for the next gen asrock integrated motherobards with increased pci lanes for about 2 years now. could buy a super micro equivalent today, at x4 the budget
OmG where did you get the board, 700$ is not cheap - i build a ESXI VM Server for 500$ on Rycen Platform in the same formfactor with also Low Power Usage.
Ryzen 7 2700 has a much higher passmark score and costs $250, plus a motherboard (we'll say Gigabyte B450M as it seems to be the cheapest B450 with 4 ram slots, for comparison sake. It is bigger as i cannot find a single mini-itx with 4 ram slots) at $73. Undervolt it (as it's a 65watt TDP chip). and for $323 get around the same power consumption. The extras on the Supermicro board are nice, but i don't know if they are an extra $377 nice lol
It all depends on specific needs, of course. Considering it's an Epyc platform in mini-ITX form that supports 4 Registered ECC DIMM slots is already something usually unheard of. Going with Ryzen/Threadripper, you're only limited to Unbuffered DIMMs if you want ECC, plus you have to be careful in motherboard selection (I think ASRock stated that all their Ryzen boards support UDIMM ECC, but there are a small few of Gigabyte boards that might too). I don't think I've ever seen an AM4 mini-ITX board with more than 2 DIMM slots.
I feel you, in Germany I can only find it for roughly 550€ right now. Total deal breaker. With around 30ct(€)/kWh in cost for electricity I can still build something around a 2700/3700 on a AsRock board and let that run 24/7 for a veeeerrry long time...
@@TheRogueBro Here you can put DDR4 REG ECC. Total 512GB (maybe 1TB) in ITX format. B450 is max 64GB. I have 7 liter case with my test lab on it. I take it into backpack and go to the client. Those looks when I'm showing WHOLE environment concept of the future customer infrastructure with multiple of servers, storages, switches, firewalls and even workstations working taken from the backpack and put on the conference table is well worth the price.
I am looking to install Proxmox for the first time and rather confused about drives/storage. So far I think I need 3 types of storage. 1) The boot drive which could be a USB but thinking M.2 SSD. Not too sure how big though. 2) ISO and template storage. I have heard that can’t be local. Not sure why I can’t put ISO images on my M.2 boot drive. 3) VM hard drive. I am thinking a pair of NAS hard drives with ZFS/RAID 1. Any thoughts or suggestions? I am looking to run Docker, Kubernetes and other developer tools for learning.
Hi NewYorker. How are you doing? I do hope the Corona virus has skipped over your household and that of your close family. I must say that I follow your presentations as a matter of course. I am a retired old IT guy, with limited need. And that need is to enjoy watching talented people like yourself. Regards from Montreal, Canada
Hi man, nice! You blurred out the mac address of your nics from IPMI but not from proxmox :) Notice also that now netword does not need reboot if you use linux bridges, just install package ifupdown2 and the GUI of proxmox will apply all the changes without reboot(finally!!!)
The EPYC Embedded 3251 has a controller ethernet 4*10 GbE integradted but on the supermicro motherboard they have just put 4*1 GbE by adding an i350 ethernet controllers
Great build. Why not use a Noctua fan and a bigger case? You can increase your power consumption and number of threads as well by a tiny bit without affecting noise or electricity bills significantly.
Did nearly the same setup like you with AMD Ryzen 2000 CPUs, not much power consumption and samsung NVMe's. You should always buy the pro versions when there is a lot read/write activities on it. It is so powerful that you are able to run multiple Windows Servers (+ Domaincontrollers) without any issues. Great for smart comercials.
Now I know where my internet comes, from LearnlinuxTvs "internet ports". Awesome mainframe.. Todays small componet's are great but of course I like "real" servers, big ones! More noise and SAS is like scsi; making confusing situations when some things error oculus Colosseum kevyttoppapuku kurva happens..
I am not going to read off that entire model number, but it will be on your final exam. Memorize it just like your home address!
4 года назад
since this case doesn't have proper front to back airflow, cardboard air duct between front fan and motherboard should drop temperatures without adding more fans
i am using intel xeon e3-1230v2 using proxmox. Why should I use an external graphics card? so without a graphics card proxmox won't boot. can proxmox run without using external vga?
One if the best presentations I’ve ever seen for a straight up Proxmox build. But lordy! The pop up ads were annoying. There were nine in a 38 minute video. Seriously? I can appreciate the need to monetize. I’m completely good with that. But RU-vid is certainly doing you a disservice with the number they’re auto injecting. I almost abandoned your presentation twice because of the interruptions every 3-5 minutes.
How would you organize storage? I'm planning on have an ssd OS disk, an ssd disk for VM's, and a set spinning hdd's stolen from my Synology to manage with a virtualized TrueNAS server. From my reading, when you pass a controller through to a VM, it's the entire SATA controller. Meaning. Proxmox will no longer be able to use since the VM has control! I like the idea of a supermicro server, but I want to virtualize my gaming PC. When I looked at their site, I didn't see an offering that either lets me fit a standard atx mobo/GPU, or a complete system that has the z690 chipset for gen 12 i7 CPU. It would be great if they supported atx cards and GPU ( just have a riser card to fit a 2-3U chassis, but they don't yet. I could not find a single Xeon processor that people recommended for gaming, so I went with an Fractal R5 tower Case and MSI motherboard. It doesn't slide into my enclosure, but it sits on a shelf sideways, lol
How do they work out that yearly running cost of 9.13, I realise it may not be at 50w 24/7/365 but if it was that would be 438kw per year, which in the UK would be closer to 150dollars.. is electricity crazy cheap in the Us ?
umm, drives are not listed . i have a am3+ 8320 so with 32 gig you can run 12 vms on a 8 core ty for this. I want to run openmediavault(nas) ,ipfire,pi-hole,htpc will try with v7 of proxmox at some point as right now i only have 12gig ram, 1 ssd120gig,1 1tb media disk,320gig hd vm,iso
Hi. I built myself a proxmox home server with a AsrockRack X470D4U and a AMD Ryzen 5 1600 6x 3.20GHz (Passmark Score: 12512) with 32 GB of ECC memory and two Micron enterprise SSDs. It uses a 92W passive cooled PicoPSU and an external power converter built for LEDs. It runs at 25W when the OS is idling and about 40W under load. 😎😎😊
i have purchased this motherboard and am unable to use ipmi. you skipped over all of the jumpers and the actual ipmi configuration...not helpful. please go over all the bullshit of dealing with a supermicro board . thanks...yeah connecting ipmi is just stupid. you skipped over all of the connectivity and networking crap....as well as the jumper settings and utilizing the bcm and password that ships with this board...first level experience supermicro is a pile of crap...prove me wrong
That board is good if you need the power. But if you hardly load your machine, you could get down to 15W using a Fujitsu board and an i3 9100 with ECC. Intel is ahead when it comes to idle consumption....
Curious, I've been using Samsung SSDs (including 850) as my ZFS root & boot device (probably for years) without any errors reported. Both my current Proxmox servers are similarly (e.g. ZFS root on Samsung 860 pro 2 TB SSD) set up.
Thank you for such a good explanation on setting up a proxmox environment. I am trying to create a nextcloud instance for private cloud at home. Also want to be able to deploy VMs as I need them for testing out other Linux based systems and expand my knowledge on Linux as well. I set up a minipc with n4000 cpu, 4gb ddr4 ram and 1tb nvme ssd. I installed Proxmox on this machine using entire 1TB hard drive. I don't have any other storage device on this machine. But I realized that I cannot add any directory for ISO's and for other stuff. Because not having ISO directory, I cannot setup any VM too. I did some research on internet and came to a conclusion that I cannot add directory and new VMs, because of using my entire hard drive for Proxmox Debian installation. Am I right? I am going to try partitioning the hard drive per say 20gb for Proxmox Debian while conducting a fresh Proxmox installation. Am I on a right path, or totally going towards a wrong direction? Is there any way partitioning my hard drive without having fresh Proxmox installation. I am a total newbie, so please excuse my ignorance 🙂
Hi I’m looking to build home server lab nas system , are you still useing the same server or you found sum thing better? I’m looking for sum thing that don’t you too much power so my power bill don’t go though roof lol
for anyone watching this he does not cover the power supply hook up other than just two of the main cables to the motherboard. There is also no source of documentation on this matter. So make sure you gather the information you need before you buy the parts. Right now, I am looking to find out where and how he is using the red cables
You mentioned you finalized on a 1TB SanDisk Extreme Pro (NVME) but in yout install setup your using the samsung SSD Drive? So which are you using and installed proxmox with? Reason I ask, proxmox is failing to install at the the first stage of clean up root disks 0% complete. When using the NVME SSD drive as the target hard disk in the proxmox install. Basically, the install fails and the machine reboots.
The drive issue is due to the poor quality of proxmox. You need to; Mkfs the drives used. Then I used webmin to wipe and write the drives. Then click data centre and add lvm group, this will bring up the drive online. . I have noticed that they don't format well so you need. To keep a close eye on the installation as you may not get drive space and have to format once or twice and force it -f when doing the "blanking" of the drive following a proxmox installation . (It causes nightmares) you cannot do this from Proxmox as they don't have the tools and the drives just don't wipe /format in any standard way. If you have SATA drives then you can take them out and format them else where. Make sure it's a full format. Before trying to use them for anything.
I was the same then I lost my job and needed to reduce my electric bill. So I built a i7 3770 32gb ram on an intel motherboard. I basically run mirrored mx500 500gb for the VM storage mirrored mx500 1tb drives for my web server. And 5x 8tb zfs2 raid. So I was basically able to get Pfsense firewall Pi hole Open media vault Nas Debian Lemp stack web server Debian based Plex server with 1050ti for tanscoding It's high on the ram but the cpu hasn't ever gone over 30%. I literally had a server for each before so it's massively cut my energy use.
Yes, but it's a Very expensive Mobo, $750 , We can buy a complete IBM x3250 M4 Rack server with 4 500 GB SAS disks on it, redundant power source and 32gb Ram for that money even less, so Let me think about it...and still we need to invest a lot More to complete this server. No thanks.
