Great video. Straight to the point and informative. I really appreciate that you do not just go through the setup steps that you can find 1000 other places but you also answer how, when and why the agents scale, a lot of docs leave that info out.
Thanks! In my video I usually try to get beyond the docs that are readily available adding information that I have gained through experience or experimentation. I want to give the highest value I can to my viewers. And I’m very happy to know this came through in this video ☺️
Hi, can you please create a video on how to automatically scale our own hosted agents (fargate agents). The scheduling probably in the form of the lambda. Thank you so much, i need it asap.
hey. So the price of VMSS is the same as for normal Azure VMs... so when you create a VMSS you can select the size and therefore you know they price for a single agent. Things get a little more complex if you wanna know the overall cost per month... that would depend on the number of max agents you can scale to, and how much utilization you would have on the platform
Did u have to install first the deveops agent in the image and after that create the vms? or no need to manually install the agent? I didnt install it, and agents dosent show up in the pool...
No, I didn't have to install anything manually. AzDO does that for you. Just create the VMSS in Azure, and then create a new Agent Pool in AzDO selecting the VMSS as target
Could you please explain how I can change the basic image, that I use for VMSS? For example, if I need use the Windows-based agent and I need to install Azure CLI or .Net6 for my pipelines. I want to create an image with the pre-installed necessary tools and then use it for my VMSS.
Hi, you can create a custom Scale Set Template, that uses a custom image. And in the custom image you can install anything you want/need. You can read more about it here: docs.microsoft.com/en-us/azure/virtual-machine-scale-sets/virtual-machine-scale-sets-mvss-custom-image
Hi Friend! Graet video! I need to create it from a script. The only way I founded was creating with Rest API, but I can't connect the Instances with a Agent into the Agent Pool. Do you ever have a similar issue?
@@CoderDave Hi! Exact! But Terraform has no module for that. We already tried with REST API and although the scale set is generated and scaling is achieved, but on the AzDevops side no agent appears within the Agent Pool.
It is running jobs randomly in diff. machines, what if we have a pipeline that has some dependencies for Example if we have added a key vault task and we need key vault secret values for whole the pipeline then how we can manage it?
hey there. In case of key vault secrets, or other job-level environment variables, those are kept even between different machines. It's part of the featureset of Pipelines. If instead you need to use the output of one job into another one, then you have to use the upload artifact in job A, and use download artifact in job B
So we switched to vmss agents using the same specs hosted agents use docs.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops&tabs=yaml#capabilities-and-limitations but they are EXTREMELY slower (2-3x slower) than the hosted agents. Any ideas on what might be wrong or what we could do to improve them without breaking the bank?
Hey, thanks for reaching out. Not really sure why your agents are that slow... that's quite strange. One thing I can think of is disks... perhaps the hosted agents use SSDs (don't quote me on this :) ) or it could just be the Azure region... sometimes you land on a older cluster, but that shouldn't make the agents 2-3x slower anyway For me when I went with VMSS I;ve actually noticed the opposite, my agents were slightly faster due to no noisy neighbors...
If you want to do it in AWS (or anything else that is not Azure) then you'd need to do it "manually". You could for example use the containerized agents in K8S. But the scaling mechanisms may be on the actual utilization rather than on the build/release queue... so not that efficient. Or you can use something like KEDA which takes in consideration the queue: keda.sh/blog/2021-05-27-azure-pipelines-scaler/ Maybe I can make a video on this :)
That could have an impact on the operations that require connection to the instance; like for checkout, upload/download artifacts, etc… but I don’t expect it to be huge since the traffic between azure regions still travels on the Azure Backbone