I have been looking for a video like this. It’s so helpful to see. Going through r/homelab can be so daunting even with the wiki. So seeing it all laid out is great. Thank you.
Really enjoying these videos as I build out services on my own lab. No formal computing g background so learning everything the hard way. Would be really interested in an overview of all of the firewalls you talked about as well as why you decided on sophos. I'm gravitating towards Pfsense for now primarily because of the great documentation as well as lots of how to videos.
Excellent video Tim. I liked what you said - "I am a developer trying hard to be a sysadmin".. This resonates very well with me. I have similar setup too but at far far smaller level.. Keep up the good work. I will now be watching your videos every week.
I've been in IT for about 25 years... grew up when bare metal was the only game in town. So I never really understood the need of VMs until I saw these home lab vids. It provided my ah-ha moment of why I should move my 12 core workstation from Virtualbox VMs to Proxmox VMs.
What a great comment! Thank you so much! I am glad this helped you! After you move to VMs, you might then find that Docker can help consolidate even more! Start with VMs though and then decide later if Docker might be a good fit for some of your services! Thank you again, so glad this helped you! ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-pxwUXJmAER4.html
This setup is SICK! But elegant. Everything that you utilize is sewn together very well. NICE to have a powerful couple of servers to get all these VMs and containers humming along seamlessly. Great vid!!!
As always, good stuff bro. FYI, I noticed you get better results in Windows if you pass the controllers thru via PCI for every device possible. I use to get freezing when I initiated a file transfer between my Windows VM and my FreeNAS VM via a quad gigabit adapter I had configured in a bond, until I passed through a single port PCI E network adapter I had laying around, solely for my Windows VM. Now it feels like a native setup.
One reason you need vlans is to separate your public facing VM's and containers from the rest of your network. If someone manages to hack into one of your public facing VM's/containers he/she will be able to use that as a starting point to attack the rest of your network. However if your public facing stuff is in a separate vlan, a hacker compromising one of your public VM's will be able to attack your other public facing stuff, but not the rest of your network.
Have to say I like the astronomy theme to your naming convention. Also I think I may have stumbled into an answer I was looking for inregards to my own home lab server rack setup. I was going to go with a NetApp disk shelf like the one you have and a Dell R710 that I already have. It has a Perc-6i controller rather than the fancier H700 and my question is if I could pass through an HBA card to Proxmox and still use the Perc-6i for local drives. Looks like I can. And that is a monster help and means I can run one less physical machine :) The First 7 minutes of this video alone have been incredibly helpful. Subbed, and now to finish watching this video.
Thanks! That's exactly what I do. I use my H700 RAID for all my VMs on Proxmox, then pass through my HBA to my FreeNAS install. ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-iva4DmOmSTc.html
Hey Tim, would love to see a series on how you setup your Sophos firewall...the hardware and software side of things....also how that can be pair with PiHole to make the most of both. Thanks!
Thanks for the tour! But now I have more questions haha. Curious how you perform backups for proxmox and why you chose VM over container. Would love to see videos on how you setup each service as well. Cheers.
Hi! Thank you! I actually prefer containers over VMs. VMs just end up being my Docker/Rancher host ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-pxwUXJmAER4.html
Hey Tim, awesome videos. I'm also a software engineer and need to build a homelab to for training purposes. I discovered your channel while researching the topic of building a homelab. I'm glad I found your channel, because it helped to clarify some open questions. I will start the coming weekend with the setup of my homelab. Two questions I have nevertheless. - Your setup also includes an ups, what did you have to configure for that? - Are the VMs that Proxmox manages LXC containers? - Is proxmox dependent on ZFS, or can it also work with a harware RAID 10 based on xfs? Somehow I have the impression that the discussions on this topic in context of homelabs strongly opinionated, almost religious. Is somewhat hard for a linux noob to figure out the way to go. Any I am curious how my journey in this topic will proceed
Impressive. Would love to see a break down of how you back all this up. Redundancy...etc. A lot of interconnected dependencies, would like to see your strat for how to rebuild and restore if/when it goes down from hardware failure and others.
I figured that you like the color gold! What about monitoring and notifying? Like Kuma Uptime? I also would like to see your firewall journey.
4 года назад
I am blown away by the content you have up on youtube. Do you have any notes or something to share about your garage door check and sending notifications?
I really could use that Wemo garage door setup... onetime I went to work and just happened to forget to close it. Luckily my neighbor let me know and shut it.
I'd say it's a bit dumb to have a DMZ without it being its own VLAN, since the point usually is to secure and separate in a sense those potentially exposed servers from internal servers and other systems. With separate subnets you don't have any security if a VM in DMZ gets exposed, since potential hacker could just add the subnets to it. Also without VLANs, they do hear each others broadcasts, which means there's extra clutter for all of the machines. Also you can't have DHCP as easily (without static leases). I personally use VLANs obviously in my server environment, but also even at home, where I've got separate VLANs for desktops, IoT (chromecasts, phones), NoT (Not-internet of Things), management (all networking gear have their mgt here), two internet connections, storage network, guest network, development network, VPN/tunneled network, etc. All in all, even at home I have around 15 VLANs. Many more at my homelabbity-environment which is actually 8U of co-location at my work. TL;DR: VLAN's are good and everybody should be using them.
jaketus thanks for the feedback. My DMZ is 100% virtualized and no other device physically connect to it and my router has a firewall between the two networks. May set up VLANs at some point. Thank you! I have nothing against them, just haven’t crossed that bridge yet.
Great Video Tim!!! I love the way you teach stuff. It's easy to understand, but somehow you manage to fit every aspect of a topic in one 20 min video. Awesome. I have a video idea for you: Since you already did a tutorial on Proxmox and are all about managing Kubernetes clusters with Rancher, why not make a video about Monitoring Proxmox with the built in "Export to External Metric Server" feature and setting up a database such as InfluxDB with some Visualization using Grafana. You could split it up into 2 parts, since databases and Grafana are a ratter big topic, but I'm sure you're gonna figure out some way to do it! Thanks I really appreciate you making these tutorials!
Thanks for making this video. It's super timely for me as well, since I'm setting up a rack very similar to yours and this setup seems like it would solve a lot of the issues I've been having with unraid.
Hi Tim, excellent video. I noticed in your Proxmox host that under “Hardware” for your VMs, you have the Machine Type set to i440fx. If you change this to q35, your VM performance will use significantly less resources
Any reason you don't use LXC Containers? I used to run most services as VM's or docker containers inside a VM but recently I'm using Proxmox LXC's more and more. Even running Docker inside an LXC is easy (just need to tick some features under Options), but I don't really like nesting (not sure it impacts performance though). Setting up a new LXC is super easy and fast, then controlling everything through the Proxmox interface and managing backups is also convenient.
Long story short, Docker is widely use for cloud computing, in the enterprise, and for software engineering. Also Kubernetes uses it too. Also, I like to keep my Proxmox clean and keep my containers on a guest vm so it’s portable and scalable. Hope that helps.
Wow nice coverage of your homelab! You fit the devops mentality perfectly. Curious how you're handling ingress and load balancing. Do you assign a cert with a DNS record, or are you using layer2 like metallb? I've been using rancher 2 with multiple loadbalancers mapped to DHCP reservations, which is a bit annoying, but gives me some firewall flexibility in routing my services. I should probably figure out multus and macvlan to have better layer 2 management.
Good question! I am not exposing my external services yet on Rancher 2 but on Rancher 1 (without kubernetes) I used Rancher's load balancer to do this.
I am actually doing a dissertation on network firewalls mostly virtualised firewalls such as the Sophos XG and how to optimise it within a enterprise enviroment
We were trying to track down a performance issue on some of our virtual machines at work and had a conversation with the people running VMware. They commented that we had over allocated CPUs for our machines given the CPU usage. It seemed counter intuitive at first until they explained that if we have allocated 16 cores to our VM then VMware needs to wait until it has 16 cores available before our processes could be run. Maybe allocating 24 cores to several VMs is not the best performance unless you really need them. Otherwise your virtual machines could be actually blocked waiting for CPU availability.
Great video! Thanks for introducing me to Rancher!! I have an, I guess, security related question: why are you running the docker daemon on a VM instead of an unprivileged LXC container? Is there any reason for this besides maybe that KVM has "complete" isolation? Thanks!
Update: I played around with it (Proxmox 7), and sadly came to the conclusion that - probably only kubelet - relies on certain kernel modules and mount points which are not available in unprivileged LXC containers ("cannot found mountpoint for cpu" I think was the error). I would either use privileged containers or VMs as hosts. If I were to do this at home, I'd choose a privileged LXC container. However, since I'm not doing this at home but rather on a dedicated server that has to be somewhat production-ready and open the least amount of possibilities for intruders, I should try it out with a couple VMs. Thank you anyways!
No LXC containers on Proxmox? Curious as to why you don't run docker from a container in Proxmox rather than a VM. Awesome video, I thought I knew everything about homelab until I found your channel...
Pretty cool setup! I have gone slightly differently - Ubuntu, ZFS and just plain Docker and deployment via Ansible. I have lost the ability to host vms/containers like iris, but it is also a way to go if you don’t build a cluster at home. Anyway checkout the frigate! It upgrades the ip cams to another level.
Just discovering your channel, great video very inspiring for me to build home server My current "server" is just an old laptop with ubuntu server. not much work just serving some docker container for file sharing, blog / notes, and ebook sharing.
RU-vid just suggested this video to me. I think I pretty much copy some of your infrastructure. Especially rancher: Right now, I'm still messing around manually with docker-compose and my amount of containers currently is at 29... Managing all via console is getting - well - annoying :D.
Great video, this like watching myself from a few years back when I was limited by a 14u rack.. now I have a 42u rack.. my advise, stick with a smaller rack :)
Thank you for this video Tim. Off the back of this I'm going to take a look at the Sophos XG Firewall. I'm currently using pfSense with the FauxAPI addon. This allows me to easily enable/disable firewall rules through Home Assistant. This allows me to disable access to the internet on the kids tablets when they should be doing homework! Do you (Or anyone reading this) know if the same thing can be done with the XG Firewall? There is probably a better place to ask questions like this. If so, where?
Thanks for sharing Tim. I'm a new subscriber... Can you do a video explaining why you favour proxmox? 👀 You may have covered this already, but curious how others come to this choice. Thanks!
Forgive my lack of knowledge if I am incorrect, but another resource suggested you need a minimum of 3 nodes to run a HA cluster in ProxMox; is that the case? It was stated by Craft Computing that each node votes on which is the primary or secondary and if you have only 2 they each vote for the other and the mechanism doesn't work.
I would love to know more about running your streams with a graphics card pass thru. I actually own a streaming pc just for me, but my wife started streaming and I would love to be able to let her use this pc to offload a bit of her PC stuff.
Great JOB for a home setup. How noisy the desk shelf you are using?? How many Power Supplies are you using? How long are you Virtualizing FreeNAS although it is not recommended???
not too noisy, I have redundant power supplies for my server, FreeNAS runs fine on virtual hardware if done right. They even have documentation on how to do it. Here's how ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-iva4DmOmSTc.html
Nice tour. Makes me wanna try proxmox again and I'm definately putting Rancher on my server now. Don't you have a huge power bill with all that equipment running? Or maybe electricity is just so much cheaper in the US? (I'm located in Denmark)
Sick setup! I have so many questions, but I'm gonna try to hit u up on twitch. I think the main question I would have is: Why you decided to virtualize FreeNas, and did you give FreeNas direct control of the hard drives (since that is usually recommended)? If so, how did you do that? If not, why? Great video!
What are the reasons you chose to pass through your LSI controller to TrueNAS as opposed to just creating the logical volume(s) in ProxMox and then just assigning that storage to the TrueNAS VM? I'm new to Proxmox so go easy on me :D Cheers for the video. Crazy stuff you got there.
Hi Tim, I'm new to your channel but I am addicted to it, thank you so much for taking the time to make videos! I know it takes lots of hours, I have a channel too. Anyways, I wasn't able to find a video on unifi on Docker, I had tried running it in my house and I wasn't able to get it to discover my 2 access points. If you can, could you make a video about it? Thank you again
I have a remarkably similar setup using an HPE DL380 Gen9 and a repurposed gaming system both running Proxmox. I would be interested to hear your thoughts about running them in a Proxmox cluster. I have gone back and forth but, without any great need for replication/high-availability (or a 3rd machine for quorum issues), am currently running them independently like you. That said, I think clustered servers offer some benefits for storage. What do you think?
hello tim can't wait your stream today i got a little question for you and i hope you can help me well i choosed proxmox as a graduation project (i know it's not a big shot project :p but i'm from morocco even hyper-Vis a BIG HUGE project) so my question is how can i have access to my proxmox management page from another network or another domain
I run unraid in a vm on my vmware cluster. Only thing you need to remember is that USB stick with the license bound to it. I just passthrough the usb to the vm
So, you dont feel that Layer 2 separation is necessary for your DMZ? Not that you want to discuss your security particulars, but are you comfortable opening that much of your LAN to the internet?
Hi! I will set up VLANS soon, but for now my DMZ is 100% virtual, nothing physical, and nothing on that network can communicate with anything on my trusted networks. It's a virtual network that only runs public services.
I used to have the SG-200 CIsco switch but it annoyed me way too much as the web-ui is really slow and i wasnt bothered enough to learned the commands. Making and configuring VLANS on this router/switch has been made way to complex. I ditched it for a Ubiquity Unifi device and honestly never looked back. Running the controller on a Raspberry Pi with 8 VLANS and a guest network, could not be more happy with our house infrastructure.
Curious on how your FreeNAS VM performs. I've been wrestling with a virtualized instance for a while, but have found the performance really poor. It has 64 GB of RAM and is mostly used for SMB sharing, and consistently slows down after a couple of days of use. I was thinking about running it bare metal just to rule out any possible virtualization issues, but that would require a lot of server and hardware shuffling. So I'm just sitting on a slow VM for now.
Mine runs great. I reboot mine only about every 60 days or so when there’s an update. How many users? Do you have a cache disk? This might help. Doubting it’s virtualization.
@@TechnoTim Thanks for the reply. I did try adding a cache disk, but performance was unchanged. I've been the only user so far, and my research send to point to either a physical fault in my network or some sort of bugginess with the SMB protocol. Good to hear that yours works - that gives me hope that I can get this working at some point!
@@TechnoTim Just wanted to do a follow-up since I did more testing. Got a server from work to play with for a couple of weeks and virtualized FreeNAS on it in a very similar setup to my home setup, and it runs great. So now I'm thinking I either have a bad disk, or the LOM on my server is bad. But viva la virtualization!
