Did you try Talos? Did you consider using it? IMPORTANT: For reasons I do not comprehend (and Google support could not figure out), RU-vid tends to delete comments that contain links. Please do not use them in your comments.
I tried it on local virtual machines. I'm kind of a beginner though, and failed on two points: - wasn't able to set up nfs storage class (due to an os dependency missing, and being constant in nature I cannot install it) - wasn't able to set up a load balancer for external access to my apps (due to being a beginner, probably) Switched to k3s for now until I have more experience. Cool project, though.
@@m.sierra5258 I don't think there is a good use case that would justify Talos as a solution for local Kubernetes clusters, and questionable value as a replacement for managed k8s. It shines (or could shine) as a solution for self-managed Kubernetes (e.g., on-prem or those who do not want managed k8s services). That being said, storage and load balancers are painful to setup, no matter the self-managed solution one chooses. Those are one of the reasons I prefer managed k8s. It works out-of-the-box.
We have been using Talos our environment with 100+ workers for about a year and it's solved some pretty big problems we had with kubespray-deployed clusters.
Thanks for another great presentation Viktor. Many moons ago I was involved working with Openshift running on Fedora Atomic Linux. It seemed like a great idea at the time until it wasn't! The very thing it achieved security, read-only filesystem etc ended up causing a fair amount of frustration amongst those assigned to manage it. Having said that we was all finding our k8s wings back then ;-) I personally ended up settling on KOPS after employing a myriad of different day 1 deployment options, but will definitely take Talos for spin in my scratch environment.
stumbled upon talon back in 20.0 when they released 0.7, the idea and the project are really cool, I am planning to use it for my home lab project. Can't wait to see what they will do when they reach 1.0.0
12:29 is it a bit disingenuous to say you are not doing anything except creating a new node and it's intelligent enough to join the cluster, when you are passing in worker.yaml as the userdata? all the logic for adding the node to the cluster will be in that file.
@DevOpsToolkit So ClusterAPI, Crossplane and Talos seem like a likely combination for an IDP PaaS? CAPI can bootstrap Crossplane, and then Crossplane can bootstrap Talos, wouldn't this become a golden path to an IDP/PaaS?
I personally use kops for k8s cluster creations and I have created quite a few clusters so far with kops. Very convenient and product keeps improving over time. This one looks too much manual work to be honest. Kops even supports templates using the go template, very similar to helm, which makes provisioning multiple clusters consistent and easy. Also comes with addons, if you dont want to install certmanager, ingress controller, metric server ....
Very cool! Like CoreOS but it lacks of developing crd in order to expand kubernetes API and create objects like machineset and machineconfig to create nodes. Great video!!!! (As always!!!)
@@DevOpsToolkit That is exactly what Sidero Metal does - it's a cluster API provider for bare metal that manages the complete hardware/K8s lifecycle...
You have skipped ROSA (ocp on aws fully managed: masters and workers) and ARO (ocp on azure fully managed: masters and workers) that comes with CoreOS as hostOS .
Thanks. Talos could be great for Kubenetes on the Edge clusters... Harvester from Rancher uses K3OS that is a very tiny Linux distribution used basically for kubernetes.
I dont see the advantages of Talos over K3OS which is more mature. That ISO is up to 500MB already, indicative of its maturity. Talos is tiny (~ 70MB ) because there isn't much in it yet. So a bit like reinventing what is already out there, but their approach appears to be creating a control node where fleet of servers running Taloa can be managed from.
K3os is abandoned with the last release made over a year ago. Talos is tiny by design. It's meant to have only what is really needed to run Kubernetes as opposed to "traditional" operating systems. That is not an indication of maturity but a design choice.
@@andrewrynhard1926 giving up access to the underlying OS is one major tradeoff. Another would be having the ability to install agents and shipping system logs to a central syslog server (mandatory in highly regulated environments on-prem envs). But this is definitely next level immutable approach to K8s, for sure. Toil does shift a bit from using conf mgmt (Ansible) to managing OS images as code using Packer and whatever provisioner. Any progress on Longhorn ? Can you elaborate how Talos differs from K3OS? Is it the control node where talosctl runs and management talos fleet?
That's the problem that all similar tools designed to be on-prem first are facing. Since you do not know what the underlying infra is, you can it design the solution that works everywhere. That's why managed Kubernetes solution give much better first-rin experience. That being said, Talos should do better on that front. Since it uses images already created for specific vendors, those images should have basics like ingress and storage drivers baked in. The similar situation is with, let's say, Rancher RKE. It does not come with those either, and it should. So yeah, I agree with you. The first-rin experience should include more stuff instead of keeping them as addons.
@@DevOpsToolkit over the last couple of days Ive been trying to install kubernetes on bare metal and I still dont have a solution for volumes. Kubernetes itself has nothing out-of-the-box. So now Im looking into gluster. And to get the loadbalancing to work I had to get help from a github issue that told me to add a config on line 500 or so of some yaml file I needed to run. This stuff needs to be easier.
@@jurgen9568 I agree. It definitely needs to get easier, especially now that we experienced how good it can be when using managed Kubernetes services like GKE, EKS, AKS, etc.
@@jurgen9568 We used to have GlusterFS and it was always a pain. Switched to a professionell NetApp Storage solution with NFS volumes which also offers CSI via Trident. That was much more stable. Rook and Longhorn sound interesting but I haven't had a look yet
@@rodrigocc_rata Ah gotcha. Talos is a ~40mb squashfs and is unpacked in RAM. The core of Talos is all reproducible so it can be "ephemeral". There is one partition in Talos where data survives reboots and that is mounted at /var. On upgrades we wipe the disk entirely so we call the partition "EPHEMERAL". In this partition is where etcd, containerd, and Kubernetes store their data. Does that help?
@@andrewrynhard1926 yes, thanks! So, you got me more curious now. Two more questions if you don't mind: * How does Talos boots again if I reboot/power outage or something? Do I need to ipxe boot all the time? Or boot from a pendrive? * How do you upgrade? Do you reboot and start a new version of Talos or just do it without reboot?
@@rodrigocc_rata Good question. We still have a boot loader (grub). Talos is just a vmlinuz + initramfs.xz. So I suppose we still have another persistent partition but that is for the boot loader. We do have yet another partition that is writable but it is only for configuration of Talos itself.
