It could be awesome if XCP-NG could be compared with Proxmox, TrueNAS scale and Harvester. There are so mange new and exciting alternatives to "classic" virtualization (VMWare).
proxmox and truenas are nowhere close to the xcp-ng center if there was more people working on xcp-ng center it would be way better than vmware on all levels.
@@mranthony1886Yet to be proven, I want to leave VMware, but the reality is that as a business, I would be a fool to do so. You don't need to be a rocket scientist to use and setup VMware, I do it in 15-30mins, saying that I so want an alternative, and it's not like the cost of XCP-NG is any less than vmware and the learning curve is steep, to say the least, I just don't have the time to invest in that.
I love XCP-NG. I followed your guide over a year ago and did a trial on a basic 10 year old gaming PC at home and was impressed. Later I bought 2 used Dell R730s for my business and installed XCP-NG on both and its been in use in production since December with no problems and no regrets. It effortlessly runs multiple VMs on the Dells and its easy to move VMs from one host to another. Its also super-easy to import VirtualBox OVA machines - ideal for running testing / training labs.
Yes, but that doesn't take away from the sadness of it all in the slightest. I have worked with VMware for a couple of years, they have had a great, innovative culture with a couple of really cool guys I've been fortunate enough to have worked with. Yet another example how Broadcom is a shitty company that destroys everything it touches.
@@InterFelix I have been doing this for over 30 years. Companies come and go. I have changed vendor and technology many times. That part is easy. Currently it is a crowded marketplace - which is good for us because we have options.
VMWare will probably be nothing more than a nitch product in about 5 years once the big corps learn they can switch over as the competition makes it easier to do so. They will price themselves out of the market.
Even with Xen-Orchestra the administration of a XCP-ng (or Citrix Hypervisor) farm is not as smooth as with VMware and VCenter. I have worked on a 140 hosts Citrix Hypervisor farm for nearly 5 years and every time there is a patch or an update we all hope it will not break something. Hosts routinely lose the network configuration on a normal reboot, there is no central repository for the cluster configuration (the number of times we had to perform the emergency pool master switch procedure because the previous pool master would not come up again and we had to wipe it, reinstall it and join it to the cluster again!). Also, the performance of the VMs is substantially lower than equivalent VMs on VMWare. I have not used XCP-ng in a production environment (only tried it to compare to Citrix Hypervisor, but it lacked the features we needed, so we abandoned it), I sincerely hope it is better than the Citrix offering, but I doubt it: the command line tools are the same, agents are the same, patches are the same, drivers are the same, and it has less features. Compared to VMWare, XOSan is not comparable with VSAN, networking is way more primitive. I will say that passing through a GPU or using a vGPU is done better than in VMWare, simply because you do not pass a specific PCI ID device from a specific host (hence forget about live migration), but an instance of a graphics card and it is the pool master that does the link from the host the VM runs on to the actual piece of hardware: this allows for live migration, but only of Windows VMs (and only between hypervisors with the same version of the hypervisor, forget about it during a rolling upgrade of a pool), Linux VMs are out of luck (or they were the last time I checked). Honestly, if I had to consider an alternative to VMWare ESXi, I would go with RHEV or its open source counterpart, oVirt. It still would not be the same as ESXi, but a lot closer than XCP-ng for sure
Well RHV or RHEV ( can't remember which is the current name) is EoL/DOA 4.3 or 4.4 is the last version. They are moving customers over to Openshift Virtualization, 4x the headaches at 10x the price.
Would like to see a video covering migration from VMware to XCP-ng. It should be as simple as exporting each VM as an OVA and importing, but vCenter 6.7 appears to have removed the ability to do that. You can export to an OVF folder, but not an OVA file. I would like to avoid extra file conversions and command line utilities if possible.
We used ovf template export and it seemed to work well for our drivers. Didn't import the machine template. Remove VMware tools before export to ease the suffering. VMware converter also worked quite well
Tried multiple times over the years, it never worked: the virtual hardware in VMWare and in XCP-ng (or Citrix Hypervisor) are simply too different. Also, the OVA format used by the two is not even remotely similar, despite it supposedly being an open format. You would have better luck exporting the VMDK and converting it to VHD (ironically, using tools from KVM), then creating a new VM with the existing disk, but even this way is not painless as you have to then remove the old drivers and somehow inject the new ones into the disk. Going the other way (from XCP-ng to VMWare) is almost impossible
I migrated from ESXi to XCP-ng some years ago (whenever XCP-ng 7.6 was current) using the deprecated VMware converter to export the VMs. I seem to remember needing that to convert the vmdk disks to VHD. I was even able to migrate a Gentoo Linux based proprietary virtual appliance which was only supported by the vendor under VMware and hyperv and it worked flawlessly until I retired it.
VMware gave our licenses to another IT group in our city during some internal data migration. The other group didn't recognize them so they didn't renew them. We tried to renew but couldn't. They made us pay for them again even though it was their company that screwed up. We met with them several times, but all they wanted to do was sell us stuff. I really dislike them. The market needs better client focused companies than VMware.
Consider a classic lawsuit case gatherings from big VMware clients. This will teach them a lesson and penalized them with billions as damages to your going concern business and outright non-compliance of their license agreement.
To be fair, VMware =! VMware. VMware has a vast ecosystem of products that goes FAR beyond just a hypervisor (and its management). Large enterprises hooked on that ecosystem won't even remotely find an alternative in XCP-NG. Specifically when it comes to network virtualization and SDN (through NSX etc.), hybrid-cloud etc.
After Broadcom takes over they might change their tune. Broadcom RUINED Symantec for several years (support, ordering tanked), no idea how Symantec is doing lately because we got the hell out of Dodge and advised all of our customers to move to alternatives.
@@LAWRENCESYSTEMS I was going to ask the same thing. Proxmox is another great free alternative. Imo many viewers would like to know how it compares to XCP-NG. I'm also curious why have you picked XCP-NG instead of Proxmox when you started working with virtual environments?
Here's one key takeaway from the Broadcom earnings call: "And with this transaction, we are focused on serving all customers via a combination of our collective direct sales organizations, as well as through our important channel partners. When we look at the majority of our revenues, they are derived from the largest global multinationals, and they fit six key attributes. These customers are usually highly regulated and risk-averse with large and expanding IT budgets. They also have complex and heterogeneous environments that are built around hybrid and multi-cloud strategies."
XCP-ng and Xen Orchestra is a great replacement for ESXi + vCenter + Veeam for smaller environments. I have this in production in multiple places and it's very easy to grow the infrastructure as needs get more complex.
Would be interested to see a video from you on XOSAN, if you haven't already done one and I just missed it. Also interested to see what XCP-NG complementary alternatives might exist for VMware NSX.
I don't think they actually have a converged SAN product. it looks like XOSAN is no longer available pending XOSAN v2? Which they have been teasing for more than a year or 2. I waited until I no longer could, and just decided to go with ceph.
Xen Orchestra should be an out of box interface for XCP-NG. Should integrated or shipped together. What also about Cloudmin? It's not like a distro on its own. Can be installed and used with Webmin.
Is proxmox not a valid alternative? i have it in my homelab, and thought it was more used in corporates. any usecases differences for each of them? thanks!
He made a video about it I believe. Essentially it comes down to the philosophy and overhead. Proxmox in itself is a much heavier platform because under the hood is a near full Linux distro. XCP-NG is a very lightweight Linux distro striped to its bare minimum and built under the idea of a cluster of XCP-NG machines. In fact XCP-NG does not even come with a built in with a VMmanagement console/ UI front end. You install another super lightweight VM which contains Xen Orchestra. In other words, you can deploy another VM host in your network without Xen Orchestra and manage it with another instance, or even cloud. Proxmox on the other hand comes with all that weight built in. You can achieve almost, if not everything XCP-NG can do but at a greater effort. This barely scratches the surface, I'm still learning, but essentially it comes down to the philosophy behind what is a VM host supposed to do at its core, whereas Proxmox is a more traditional approach of having a robust Linux distro as a foundation with the VM functionality and management tools built on top. Anyone feel free to correct.
@@CmlDexter it cant be explained better, well done. I use proxmox at home and my learning curve was extreme, gpu passthrough, linux command line, filesystems, zfs, linux containers. My knowledge exploded within few months, like literally!
proxmox is more aimed at small and medium setups, with relatively small clusters and "converged infrastructure" aka hosts that have both storage and virtualization capability. Also it is soemwhat immature in many small things that matter more for a large cluster. For example if you turn off a host with powered off VMs on it, the HA does not move them to other hosts so they become unusable until it is restarted, or until you do manual commands from console to move config files around. Also afaik it has no load balancer (does not move VMs around to balance RAM and CPU usage).
another reason that plays in favor of vmware is that software used by enterprises is certified mostly on vmware and hyper-v. Alternative foss hypervisors are often not certified and that helps vmware stay baked in their existing customers
That is when we started looking at alternatives. It was a huge indicator they were looking to monetize (re squeeze) their existing customer base. We ended up moving our servers to AWS (corporate decision).
cloud providers charge you for vmware use based on memory used by your machines. And it's not that bad regarding the pricing and features you get. VCPA is name of the program IIRC
What's the difference between broadcom and a flounder? One is a nasty bottom feeder that scavenges mercilessly and digest and excrete everything that it encounters, the other is a fish.
Not exactly a fair comparison since the charts didn't include vmware's orchestration tool (vcenter), which adds some features that were missing from esxi alone, such as load balancing.
VMware is for large Enterprise Environment with focus on critical applications, Xen/Citrix XenServer is for medium Enterprise Environment with focus on windows, desktop and cloud applications, Proxmox is used in any Environment with more flexibility, but with more tweak on KVM and no special focus. KVM is a common hypervisor.
I don't think this was video was referring to the desktop VMware products. This was talking about enterprise level products like ESXi, which KVM and WSL do not exist in this space and are not alternatives. Many large companies are looking for an alternative to VMware since the takeover. I do agree with you that WSL is the best desktop solution.
@@vaprex yes I am aware of the KVM offerings for enterprises but it is not even close to the enterprise level of ESXi. Our team manages several OpenShift clusters (currently switching to Rancher) and have looked into KVM but there is no way that could replace our ESXi footprint. It's just not at that level.
The purchase hasn't happened yet and there is currently a 40-day shop around period in progress right now. I haven't heard of any large enterprises looking for alternatives to VMware. They might exist. I can't imagine anyone has started looking at this point, but again it's possible. In my almost 40-years in IT, and over 20 years with VMware and complementary products, there is no direct substitute. No one else comes close, and replacing a large VMware infrastructure would not only take years, but the total cost of the replacement would be more than any incremental licensing costs. I think there's a ton of FUD out there right now and nobody knows anything for sure.
VMware licensing already has a time bound licenses schema in place forever, side by side with the perpetual lics. I doubt there’s anything they have to do other than issue different license keys. I would actually welcome a way to tie keys back to the portal and validate they are current but that would put the Deloitte audit team out of work for VMware products. Hopefully they don’t try and engineer a citrix license server experience. Its also annoying when management won’t pay for support and don’t understand they are not entitled to upgrades and security fixes.Which a subscription model fixes… It also addresses Managements concern about capital costs vs expense depending, which I grant you changes every time you ask.
cool, so VMWare is now pulling what we call a "pro Citrix move", let's see how well this pans out. But I have suspicions this won't go well in the long run
Nope. I suspect once corperate IT realizes that competition is making it easier to switch, and realize they can easily save the millions to vmware, they will switch. And vmware will die like everything else BC touches.
XCP-NG is 5 times more expensive than VMware to run for a home-lab. XCP-NG Orchestrator is $924 a year for Starter or $6,600 for Premium with all feattures whereas VMUG advantage is $200 a year. The completely free version of Xen Orchestra is missing all snapshot features and practically the entirety of the backups system which is the biggest selling point of Xen Orchestra. If you don't offer training versions of software/tools than why would anyone risk putting business applications in jeopardy to learn a system?
XO is fully free if you compile it yourself, I have videos how to do it. lawrence.technology/virtualization/how-to-build-xen-orchestra-from-sources-on-debian-10-using-xenorchestrainstallerupdater/
You convinced me to move my home lab over about a year ago. We have a substantial ESX setup at work though so I'm curious what the price increases are going to do for us - might just push more stuff to the cloud instead of running it in the DC.
Its not quite that easy to shift over. In case of a VDI, you would need a working vSphere for Horizon Desktop provisioning. Unified access gateway appliances are not meant for Xen/XCP-NG, Integration with NSX and what not would be missed. Its got a vast community. Also the 600 customers VMware is calling out for are mostly Top Fortune 500 companies which pay in millions.
I've just finished migrate a server from ESXi 6... I've tryed XCP-ng a lot, but I had some issue with hardware compatibility on a very recent and powerful laptop... Such laptop was the only hardware available to keep the VMs running while i was playing around with the new hipervisor on the server. At the end I've installed Proxmox on that laptop and migrated a couple of VMs in half a day... Dam, I'd like to love XCP-ng and XO, but they don't like non-server hardware If only XO did support KVM...
I run an XCP-NG cluster for production, and it should be noted that, you need both XCP-NG (per host) AND Xen Orchestra license for your whole environment. Its not exactly cheap (not VM ware expensive, but still for 4 servers and XOA = $8k licensing per year). I will say though, that for the price, you would have to spend north of 40k to get the same feature set in VMWare.
Vmware is good but xcpng can be a good alternative only it has a 1.3gb ram need against way less then vmware needs so for low memory systems vmware would be a better option within the free tier
My vote goes to Proxmox who are doing it the proper way without a management VM that just waste memory (like Vmware and XCP-ng).The nodes are auto-clustered, you can management them easily and the interface is concise and to the point (like Vmware) wherare XCP-ng is too "webish" and you need a lot of "clicks" to do what you want and the interface is non-intuitive imo.
I still prefer XCP-NG, their XO management being a separate VM makes sense when you are scaling out to many many host or even many hosts at multiple locations. Their UI offers lots of tagging features for automation based on those tags which is great when you are dealing with a large number of VM's and their latest "Auto Restore" testing with the integrated backups is a great feature ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-zH5tQYLpfCk.html
Are the resources assigned to a management VM really such a burden on a virtualization farm nowadays? It usually is a pretty small VM... I mean, 8cores/32GB of ram are essentially scraps
@@dec0077sa Probably not, but the fact that the hypervisor doesn't provide all the tools to manage itself and clustered itself is just a business decision to sell more stuff and make more money. There is no logical reason to separate that fonctionnality from the hypervisor.
@@Traumatree Only XO is a separate license from the hypervisors. All other management tools (Vcenter, Xencenter, XCP-center, RHV-M, etc.) are included in the base license or available for free. Originally, for VMware, you could not host the vCenter on the farm itself for availability reasons; at some point (around ESXi 5.x?) it changed and you cannot have it outside of the farm because it is not an installer anymore but an appliance. Of course, tools that are "on top" of the base management are another matter entirely and could have a separate license, but the normal farm management is usually included
@@dec0077sa you could always host vCenter as a VM even back in the 2.5 days (as far as I go back) VMware didn’t recommend it until HA became a thing in 3.x and Hardware matured to have more cores and memory. Usually it was recommended to run vCenter on a physical node some folks including myself ran it on MSCS which was really a mistake and contributed to more downtime than uptime for vCenter
“Krause said CA and Symantec spent 29 percent of revenue on sales and marketing. Broadcom spends 7 percent” Basically gut vmWare and bleed out big customers who can’t leave.
I covered the basic features in this video and how they compare to VMware ESXI, XCP-NG is not a replacement for EVERY feature that VMware offers across their entire product line.
For a homelab do I put the NAS on the hypervisor? Or do I put the hypervisor on the NAS? Which free NAS software should I use for homelab with XCP-NG? Will this run on an AMD 4100 processor?
Agreed, still needing the Citrix management agent on my Windows Server VMs. They are getting closer, but still not there with an open source replacement.
Yup. The current xcp specific agent drivers has been in certification for almost 2 years. I would prefer not to have to use the Citrix drivers as they could be revoked at any point. Its not a show stopper, its just no great.
After working with xenserver many years ago I’ve vowed to only use VMware moving forward. Price increase be damned. Bullet proof stability, great support for the best backup system, veeam, little to no maintenance, and excellent support for hardware.
We used xenserver for some of our lab instances for cost savings and I completely agree with you, it was a pain to work with. That was about 7 years ago so hopefully there are a lot of improvements with xcp-ng? In my current space, we use Kubernetes in the cloud for everything so don't have a need for VMware anymore. Curious to see what most companies end up switching to.
when will this montly subscription start and will us home lab guys be affected and if so and i move my to xen and xpng do i have to recreate my vm's and virtual drives what do i do with all my data?
I found that some videos are very interesting but video about hipervisor isnt just right. Used xcp-ng and back to proxmox. Proxmox is difficult in some way but some features like changing network while vm is up and running and hard stop vm is what proxmox plus than xcp-ng...
So I have a question, I know xcp-ng and xen orchestra can do what esxi and vsphere does. However in VMware there is nsx-t whi h bring various features to vsphere like routing, bgp, ips/ids, distributed firewall, nsx intelligence, etc. Does xcp-ng and xen orchestra have something that can provide similar feature as nsx-t does?
NSX is a separate product from VMware vSphere and is a Software Defined Networking solution (SDN). NSX Advanced Firewall (which includes NSX-T Layer 7 Context Profile NSX-T Distributed IDS/IPS, NSX-T Identity Firewall) and NSX intelligence are add-ons which a customer must purchase in addition to NSX. VMware has many different products (vSphere, NSX, vRealize etc.) and add-ons which can be purchased separately or in bundles depending on a customer's needs. There are many SDN software vendors and SDN open-source projects which are comparable and even more feature rich and flexible than NSX. NSX is tightly integrated and requires VMware vSphere (specifically the vSphere distributed switch). NSX provides vSphere customers a path to deploy a SDN solution which can provide many operational and security benefits in their datacenters, cloud and edge networks.
I need to work out how to get VMWare Player - based VMs I run on my laptop(s) to something different. I have tried/failed to use Oracle VirtualBox (also free) and hated it. Does XCP-NG run on Windows 10 as well?
No, XCP-NG is a full operating system, type 1 hypervisor. What you are using on your laptops are type 2 hypervisors, so they run on top of the base OS (in simple terms). For playing around in Windows, you only really have Hyper-V, VMWare Player/Workstation and VirtualBox.
@@LAWRENCESYSTEMS Thanks for your quick reply. I was actually looking for a solution for networking because my home network is getting a bit outdated and learned from you about pfsense and unifi access points. I will most likely start to use them soon. I want to thank you already very much for your excellent YT channel with great info about these products. Looking forward to get accepted on your forum.
@@LAWRENCESYSTEMS Hey Tom, are there any good opensource VDI options, whether using XCP-NG or Proxmox? Is the only solution to us something like vSphere or Citrix running in a VM on XCP-NG or Proxmox if you need a VDI environment?
@@wcgi-us I never got why one need VDI... It's cost is twice the PC you need to run it anyway + massive overhead + complicated administration, resource hog, all for some automation that you can really get with proper tool on plain windows PC. Another solution for VDI are cloud offerings, cloud desktops, windows 10 in azure, aws workspaces and such. but they are costly if you need it at scale :( But VDI is costly anyway so....
@@LAWRENCESYSTEMS Thanks for the reply! Since Orchestrator can run as a VM on the hypervisor, is there some solid way to ensure the XO VM starts? I am just thinking back to the old days of vCenter where you *could* run it as a VM, but if it failed to start, you needed the built-in ESX(i) web UI (and XCP-ng doesn't have web UI built-in from what I understand). Sure, there's cmdline, but is there something better to ensure that XO VMs start?
Sorry for the n00b questions. I need to find a spare machine to test it out on. It looks like the XO page will talk with local servers. It's just very unlike standard VMware stuff...
EXT4 was supported since January 2019 as an optional driver and has been the default for local storage since version 8.1 came out in March 2020. Maybe you're thinking Citrix Xenserver?
Great info @Lawrence! Glad you brought the VMWare and broadcom information to public. Other people will be jumping ship, broadcom destroys companies. @Lawrence - What about oVirt? Unsure if you've used it or perhaps you can try it out. Ovirt 4.5 is latest release. Alternatively...... Oracle Linux and Oracle Virtualization Manager [which is still ovirt 4.4]...
When i concolidate disk on veeam that keep fatal error and call vm support do you know what that's funny? We one company forget to restart server for 562 daya ! 😂
VMWare's licenses don't actually ever expire. Once you have them it's good forever. The big difference is with subscription is you get updates, upgrades and support. This may change where the licenses do expire to bring it down to reduced functionality which I can't see it being a good thing in an enterprise environment. In fact I'd steer clear of that. I don't run a large vmware server farm at work but with 14 nodes for VMs and VSAN the cost adds up pretty quick. Right now we're 1.5 years into the 3 year subscription with them and once we're getting close to the end of the term I may go back to ProxMox or give XCP-NG a try long as it can support cluster to cluster over WAN. I can certainly look into ZFS style replication which I can now do in ProxMox for test environment at work.
Maximize your core count per vSphere license to reduce your CPU license costs with AMD 32 core CPUs and 1U servers. Also, ditch vSAN and/or HCI which is a rip off and go FC SAN or NVME/TCP SAN. SAN storage is so cheap/fast/small these days.
I am currently using ProxMox for my home lab and had a two cluster ProxMox with CEPH at work but had to switch to vmware over a year ago. Main reason for that is disaster recovery as CEPH do not support HA between clusters. Especially over the WAN. Vmware is able to do that via the use of replication on a schedule. We're halfway into the 3 year subscription with vmware so I may look into going back to ProxMox or look into XCP-ng. Just don't know if XCP-ng can support two clusters via WAN. With ProxMox I'd be using ZFS replication and skip CEPH entirely. When I had CEPH running which was great but if one of the nodes goes down due to a reboot it kills alot of the VMs. Maybe I will give CEPH another try on dedicated servers with more resources. CEPH is getting better over time.
@@Darkk6969 Running HA through WAN is a bad idea. IDK why you want to do that in the first place. DR and HA are used for different things. CEPH is very network intensive when scaling up. If you need to replicate VMs then use backup to a different storage. Or you can set up an asynchronous CEPH replication to another cluster. If you set up HA in Proxmox correctly your HA enabled VMs will start on any surving node. And if you have at least 2 replicas in CEPH the storage will always be available in case of a node failure. Proxmox HA wont work with less then 3 nodes as the surviving nodes wont be able to get quorum. I´m sure you have your reasons for your setup but it seems like you missed some key caveats.
@@TrannelNossnaj For production (depending on your hardware), is that not still significantly cheaper than even vSphere's current pricing? Unless you skip on backups, vSAN, vCenter, etc. or have a very small environment.
@@marcogenovesi8570 The problem with HyperV is that I can't find a way to connect to the VM without RDP (RDP has crap performance (30fps)) And yes VB also has very bad performance.
Is XCP-NG a good alternative to VMWare? Yes. The answer is yes. As much as I dislike Xen in general, VMWare is worse. VMWare is software from yesteryear, made by engineers from yesteryear, for engineers from yesteryear. VMWare is old and crufty.
As much as I like Tom XCP-NG is garbage, you may as well host your VMs Azure. With XCP-NG you cannot even make a vm on the hardware without first signing up to their SAAS GUI and they only allow backups to an XCP-ng pool. You are only exchanging one devil (ESXI) for an equally bad SAAS demon(XCP-ng)
I'm not completely sure you watch the video but you can compile Xen Orchestra from sources Without signing up ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-bq1iKO-0jWs.html