In Proxmox you didn't use a second gateway but split out the interfaces and subnets. Are you simply running a flat subnet, is Proxmox supplying the DHCP somewhere? This made me have a lot of questions as a network engineer. I would expect a whole other interface set to be segmented from the primary interface. Unless it were a part of the same bridge. How do you perform bonding and use say lacp? I have not had a chance to play around with Proxmox yet. Wanted to RTFM a bit before I started and your class looked decent so I spent some time here. Will say I love it so far for starter information. Your firewall covering was nice. Thanks for the quality information.
@@bjornarsimonsen7592 I was also wondering how the vm got it's ip, what is serving dhcp? He must have connected the second port with a cable to another router but regretfully forgot to mention that.
You should have done an update to that linux VM at the end to show that it actually had net connection (which it hadn't). I cant get why you skipped that.
0:20 Networking is currently one of my least favorite topics right now, I've installed proxmox in a datacenter. i managed that the test-ct got an ip address but whatever i'm doing, it can't ping the Google DNS server. From the host, I can ping the DNS server with ease. Hope that Video helps me in that regard. And i have to say that i'm currently very upset about proxmox, for a software, which seems like to be designed like "have a gui for everything" the network configuration so far needed a lot of CLI so far. Seems not complete in that regard IMHO. So as you might notice: I'm very frustrated about that :( 10:07 "great", this video showed me also not what i needed to know: How do VMs reach the "outside". Cause they don't, they can't even Ping anything else besides the Host itself :(
Did you get it resolved? Could be an issue with the Proxmox firewall (if it's enabled), or there could also be a route issue going on inside Proxmox. If you've made a lot of CLI edits to the networking interfaces something probably got messed up along the way. This video is pretty introductory, and even though he mentioned VLANs at the beginning, he didn't cover them. I have lots of VLANs and went to other sources to understand how to set them up in Proxmox.
Excellent video. Hopefully we can see more of it. One question: how can i separate management interface and VMs through vlan aware or OVS. I did plenty of researching over internet about but still couldn't get to understand it completely. In cases where your proxmox server has only one nic how can I separate both interfaces?? Thanks in advance
Great videos Jay. May I suggest a topic for a future video in the Proxmox sessions? " web hosting from a VM in Proxmox". I have been searching for help on this topic. Nothing helpful for a novice exists at this time.
That dual port 10Gbps Network card, can you show how to setup LACP in Proxmox ??? for Best Practice please use 1Gbps Nic for Management and 10Gbps for share/external storage & VM.
Great video! I created a TrueNAS Scale VM inside of Proxmox VE 8 and I assigned a new bridge vmbr1 to it. However, I noticed that the MAC address of the port I assigned to the bridge and the MAC address seen inside of TrueNAS are different. Is this an inherent property of a bridge? How do you view the actual MAC address of the new bridge inside of Proxmox? I need this in order to set static IPs in my router...
Hi, how about accessing the internet on VMs from the second bridge you just created, Port-forwarding and so on? It's worth to show all the moving parts regarding networking in Proxmox. This is my suggestion.
Yeah, you need NAT or something similar to allow VMs from separated network to access Internet. I don't think this separation is worth except for mission critical systems. It is just another layer of complication that barks and bites you every time you want to do something useful. The argument "if someone breaches into your webserver VM, he can't access your VE" is strange; because if someone access your VE from your VM that means that your VE does not have proper security policies set up, which should not be the case.
you dont want to add an IP address to the bridge network that you setup for the VMs to use.. with that IP address defined... VMs can reach your pve webgui
Can you still access the webserver from your laptop in your management network now that it's on the vm network? Can the webservers still reach the outside world (for updates, for example?)
Thanks, just finished here on part 13 (only one server so skipped shared storage/HA etc. for now). Excellent information, superbly presented. Thank you.
I did exactly that but they don't have any wan access. It's for two windows vms to practice active directory. Ended up spending more time setting up the network than practicing for class 😢. Anyone got any tips or website to look at id appreciate it.
Good video,, but if possible can you expand on network on proxmox? How to do lacp bond and creating a trunk. So that we can have multiple vlans over the bond?
Hi, can you please do an updated video where OPNsense is used with 3 physical nic's one for management one for WAN and one for LAN that will then have vLAN's. Then be able to use the SDWAN feature in ProxMox 8.1? Thank you.
found the one gateway is an issue if you have 1g port and 10g port. I was trying to use management network on the 1g port and my 10g used for nas/san connection. I have no idea why, but I do not get even close to 10 speeds and read online that gateway has to be on the same physical nic to get full speed. (side note I have done all of the other steps with MTU and such).
I know little about networking and I want to learn it, but for some reason there is no hands-on course or tutorial teaching you how to learn networking just like building mobile apps or web apps; it is all theory which I don't like. Thank you Jay for showing us this.
When separating the networks like this is there anything special for the physical layer? Like would you have to connect 2 ethernet ports to that one node or is this done through the firewall/proxmox.
Great series. Thanks for your efforts! This is the one video I was looking forward to the most. I think there’s probably a lot potential for a further video on this topic.
Hey. I also like this video series, but as few others mentioned here, there are some confusions about the actual IP of the vmbr1. I presume those VMs/LXCs will require the GW, but there is no mention on it in the video. I'm also lacking any response from Jay in this comment sections. :-(
I find it easier to create a bond with 2 x 10Gb ports, configure the switch end as a trunk port and send all your vlans down it. You can create separate vlans on the Proxmox end for management, storage etc. Dont forget to config the vlan id for the guest. This also works well for XCP-NG but is a bit of a pain to get setup.
Create a vm-gw, enable IP routing on it. You can also use a fw such as pfsense. Add two nics on it, one connected to the internal network and one connected to the management network. Set the default routes on other vms to this gw
What i fI have two networks created on my router Mikrotik and both have DHSP servers and act as a gateway. Can I add a gateway for a second network too? And wouldn't that allow me to access Proxmox console though other networks too?
Why does the IP end in .0? You don't assign an actual IP address to the bridge? Doesn't the first IP address indicate the scope of the pool, just as the last indicates the broadcast? I'm confused because usually we assign an address to an interface in order for it to be reachable in the network.
Same question here. Adding to that: I have another problem. When I create a new bridge the management GUI transfers over to the new bridge.... I HAVE NO CLUE WHY! Any ideas?
Wow!... There's not one RU-vid Proxmox networking tutorial that covers the OVS aspect of networking. I've been searching forever and it seems that the creators of Proxmox tutorial videos either don't like it or don't know it. Why is that?
Thanks for this series. This video helped me find the answer to a problem that has been plaguing my server. I set it up on a desktop machine and all was working well to start. Then I decided to throw in a four-part NIC I had laying around because I thought it could come in handy at some point as opposed to sitting in a desk drawer. After a reboot, I realized I had no ability to connect to the server any longer. It took a bitvgorbthis noob to figure out that all the network ports were now set to manual start for some reason. I got eno1 set to start automagically and gave it the original static IP it had. Eh violá, I can get back in! Only now I find out that my servers have no network connection. I was stumped again, but knew I'd find the answer in this series somewhere. The part where you showed the original network bridge was tied to the port you had assigned during the proxmox install made me realize I needed to go tie my bridge 0 to the eno1. I'm gonna try it when I get home and see if that solves my problem. Thanks for the work you do. By the way, I love the mastering Ubuntu server book. It was very well written.
So, if I have only one network card (which is in my laptop), can I still make separate network with management network? I really new in networking (and quite bad in it) so mind if you could help me with this? thanks!
Is there a way to find out what port is for what card/device? I have a chassis with 4 internal ports and 2 add in cards with 2 ports each, how do I determine what's add in card 1 and card 2 or builtin? I currently have wires in all ports but proxmox only has the one active from setup, but I don't know what port/card it is. Secondary question (don't know what video to look for) how can I tell proxmox to assign bays x-z of a multibay chassis (sc846, single hba card to a sas2 expansion backplane) to a certain vm so regardless of what drives are in said bays they're passed through for example 0-11 to an unraid vm and 13-23 to a Truenas vm.
This may be a dumb question or misunderstanding on my part, but do you plug in both of your NICs using a lan cable? For me when I try this, I have a cable going to my mobo port, from which I can connect to my LAN, and then I have a card with another port that I am trying to use as a bridge. I was assuming that if I have the mobo port connected, that the bridge made to the card port would be able to work and any container built from that should connect to my LAN. Am I missing something? tldr: have two bridges, vmbr0 and vmbr1. Vmbr0 is plugged into a network switch and this allows me to hit the command GUI. Building a container from this lets me access the LAN and WAN. Vmbr1 is NOT connected via a cable and building a container from this does NOT connect to either LAN or WAN (it does hit the IP of the bridge itself but that is it).
Unfortunately my machine does not have two interfaces and I had to follow the video and write down the steps so when I manage to have several interfaces, I thought it was a very good video, congratulations! There is one small thing that I can't understand, if your PC has 172 and the Proxmox machines have 10, how can you connect without having display problems? That is, apart from this exercise, have you had to change the router's subnet mask or something so that you can connect to 10, right? I don't understand the point of having 2 different subnets and physically having a connection without having to modify the masks or routers. Great video!
I see some people are confused by the lack of gateway on the VM subnet. Yeah, I was too at first. The gateway is for the HOST OS, and not the VMs. You can only have one default gateway on Linux. For each GUEST/Container VM, they'll have their own default gateway, which would be, in this case, on his 10 dot (vm) network.
Thank you. I still struggle a bit with network in proxmox. So let say a vmbr is like a switch Why would I need to bind that linux bridge to a port (ensp ...) ? If gateway doesn't matter, how to make the VMs on one bridge get access to internet if I need to? I will be glad if I find answers to these questions. Thanks
@@digiground7613 The name says it all ... DEFAULT gateway, there's only 1. Consider a default gateway the front door of your house that opens up to the world. You can however have multiple gateways (just 1 default), like having a back door openening up to the world as well, just using a different road. Every OS uses its default gateway to send traffic over the internet, but that is defined in the OS or by DHCP, not on proxmox. Proxmox just has 1 default gateway (but you can define secondary (or more) gateways who will have a lower priority. This is for example used when providers have a fast and a slow link for backup. Imagine a computer has 5 network cards and every card has a default gateway defined. How would the OS keep track of any outgoing or incoming traffic when it's being sent over a different network card each time? Instead it just uses its one and only default gateway to send traffic over. But you can set local routing rules that overrule the use of the default gateway for certain traffic. This is where networking gets interesting... for the geeks anyway. vmbr just means that your network card is in bridged mode. It's like a layer or a mask you pull over enp5s0f0 where you can define the properties of the bridge, On a virtual host, you usually have several important network modes: - host only: this network connection only allows network traffic between the proxmox host and the VM - "normal" (i forget the name lol): Your proxmox host is going to behave as the router for this network card, using NAT. From the outside, this network is protected, like how your home router protects your private lan. - bridge: this mode gives "full access" to the network card by the VM, bypassing the VM host and exposing itself fully to the network. This scenario is the least restrictive but also the most dangerous to use as your VM sits wide in the open and requires additional security measures. It is a valid choice in most situations to avoid needing to do double-NAT, which can cause issues for some applications and services. Just be aware of the dangers and the necessity to implement additional security. Depending on your virtual software, you might see some more modes but these are probably the most used situations. And always keep in mind what you are talking about, a nic on a proxmox or a nic on a VM are very different things. You define the bridge in your host. IP address assignment, DNS, router/gatewat etc is all defined in the VMs OS. It has no clue what mode the NIC is defined as on Proxmox. The VM just knows there's a working network card for it to use.
Great video and guides - I did install proxmox but I think i need GPU acces sometimes to use on peertube and other stuf doing encoding how you make some guides for this also ;)
Nice video but couple comments: (1) You added a subnet on the vmbr1 instead of an IP address. When I configure a new network bridge I add an IP address that I can ping on that interface. On the switch side the port needs to be configured as Access Port on the specific VLAN that you want to use to make it work (and match the subnet configured on the vmbr1). (2) You can use the same vmbr0 or vmbr1 and enable VLAN aware and define the VLAN that the VM will use. In that case the uplink port on the switch needs to be configured as Trunk Port that allows to pass multiple VLANs.
he's trying to separate the VM network and the ProxMox web management. If a hacker got on the VM network, say your webserver, it would be harder to access the proxmox server itself.
You configured the vmbr1 with a subnet, why? I thought you need to set a static ip in ordner to give your host an ip adress in this network. What's the point of giving the proxmox interface a network instead of a static ip? Or is proxmox literally creating this subnet on that bridge?
So please correct me if I'm wrong, the management interface is now tied to the same ip-address that points to the gateway. Isn't this the very thing we're trying to avoid here? Edit: Also, my bridge automatically created during install. is "vmbr0", tied to "eno1". "eno2-4" are on the same network card. Is this bad from a security standpoint or is the physical separation of ports already enough?
As long as your firewall separates your vlans properly, the same gateway won't matter. As for the 2nd part of your question, this also boils down to your firewall. You can have separate vlans all on the same physical port and be completely secure as long as your firewall separates them from each other.
Hello, thank you and congrats for your really good and helpful videos! I am new to proxmox and i want to test it before start using it in production environment. But to do that, i would really need your advise, if that's possible. What i do now in my production environment, im using windows server as hypervisor, with only one physical network interface getting public ip via dhcp. Im using NIC Teaming to create an additional virtual network adapter so I can connect it to my provider's vSwitch, using vlan. When NIC Teaming is ready, i have two virtual adapters, and then I am creating two vSwitches inside Windows HyperV Manager Settings (one vSwitch for local ip - vlan and one vSwitch for public ip). That way my VMs have one public IP (which i buy from my provider) and one local IP using VLAN, so my VMs can connect with rest of my local network. What can I do to have a similar setup inside proxmox? Thank you in advance.
From where does it get the IP? Why didn't you define an IP for the bridge, but only a subnet instead? How does it know how to route the traffic without a gateway? Is there an actual cable connected to that extra port? Everything in this video is almost useless without this info.
I think it works that way: Since you didn't assign an valid IP-Adress, your interface can't be accessed. But traffic can still be routed through that interface. And since your VMs have configured the required gateway, the traffic can reach the suitable gateway.
Thanks for the valuable series. It is really appreciated. I am not sure if I lost something within the video. My question: How do you assign a static IP to a specific VM? I mean, my PVE has IP 192.x.x.100, by default my VM was assigned with 192.x.x.192 but I need that VM to become 192.x.x.80. Thanks for helping oaths.
Great video and series. Very helpful. One question to the community, I had issues with the bridge until I added a VLAN tag when creating the VM. Once I added the tag it started using the correct IP subnet. Is this typical or because how I setup the network from my router?
Random question, which is outside the scope of this video I think... but when it comes to best practice... should my idrac/iLO's be on a separate network than the management networks?
Hi - the way I see this, and I do this on our network is that ILO/iDRAC interfaces are in fact management interfaces and as such will fall into the management network IP addressing scheme.
This is a great series about proxmox. I have a small questions. I need to add bluetooth i mean motherboard`s bluetooth. I have hp prodesk 400 g6. Could you please help me. Thanks so much.
Everything is good but I can't understand why youtubers like you totally forget the IPv6 on their vidoes.. It's a shame that you only explain thins which already mentioned in Proxmox's documantation.. It will be great if you do something on IPv6 as well, specilally the routed network with IPv6
I highly DO NOT RECOMMEND the process shown in video! I tried twice and not only It did not work but I also lost connection to my management IP / bridged interface. I had to twice completely reinstall Proxmox sever. 😞
You have to understand the network that the ProxMox server resides on. You can't simply apply/change network IPs unless you do so. It sounds to me as that may have been your issue?
I'm not sure I follow the gateway explanation. You have a gateway. It is for Proxmox and the management network. You create another bridge without a gateway for the VMs. However, if the VMs need to get to the Internet, they are going to use the gateway for the management network? I thought the whole purpose was to have separate networks. In my lab, pfsense+unifi+proxmox, I built separate networks on pfsense and created bridges in proxmox with the respective vlans.Any VM connected to that bridge goes out the VLAN gateway I created in pfsense. Same as yours, I did not have to specify a gateway on the bridge but because that bridge is on that vlan, it got the dhcp scope with gateway for the vlan. Anyway, I guess we can all have it network differently. Thanks for the lesson.
