Great video Matt. Is there a Q&A board for asking questions on Azure load test? The Microsoft Q&A board for Azure load testing is dead. I have several issues with my JMeter script that works well locally with no issues but chokes in Azure load test. The documentation is useless. There are no examples / documentation on how to ensure that cookies are extracted and passed in with every subsequent request after logging in. I have an HTTP cookie manager at the plan level in the JMX and that works great and I can test all areas of the website. But since the cookie extraction doesn't work, I can't get past the login process in Azure load test. No help to be found anywhere. Azure load test is not ready for prime time.
Thanks for watching! Sorry to hear the docs are falling short. Do you have an active subscription with a support agreement? It could be worth opening a service request via the Azure Portal. Otherwise, I believe the team moved from GitHub issues over to the VS developer community for load testing ideas, feedback, and issues. URL is below, I hope this helps! developercommunity.visualstudio.com/loadtesting
I haven’t come across that myself, sorry. There seems to be a bit of information about that error and JMX files. Have you gotten far with general troubleshooting or even with a LLM like ChatGPT?
I also got MalformedURLExpception. I used your sample directly and used it from git. It appears as if somehow environment variables are not picking up. Checked UDVs it is correct. Any settings in Azure to somehow let it pick?
Hi @Matt Another issue - Unable to upload JMeter script which works fine in local system While trying to upload the JMeter script in Test plan tab of Azure load testing service, it throws the following error “The uploaded JMeter script is invalid. Please remove it, and upload a valid script.” But the script is working fine in local system. Please let me know what would be causing this issue?
Hi there! Unfortunately I’ve got no idea, sorry. I’d recommend logging a support request with Microsoft, or reporting the issue in the community below (I noticed they redirect you here now from the Azure Load Testing GitHub repository). Hope this helps! developercommunity.visualstudio.com/loadtesting/report
Thankyou for posting the video. in the test criteria when i add 10 metrics in load test getting pass failure can't have 10 metrics points can you pls help on this
I love this video, it gives me a very clear understanding of Azure Load Testing. Thank you 🙂 PS: what application do you use to draw arrows and stuff while presenting?
Thanks for the feedback! Good question - I’m honestly not sure. I don’t think that functionality was there when this video was released, but it may be today. Alternatively leveraging LLMs to create the files is a great use of that technology.
Hi @vikashplayer, looks like this might be what you're after? techcommunity.microsoft.com/t5/apps-on-azure-blog/authentication-with-azure-load-testing-series-azure-active/ba-p/3718002
Hi. Apologies if this is a simple question, but I'm confused over the multiple Homepage lines on the graphs. There's a line for Homepage, a line for Homepage -0 and a line for Homepage -1. Can you explain what the difference is? I don't think you mentioned it in your video, but possibly I missed it
Hi there. From memory that might refer to the name of the client side metrics resource (the workers / engine), or maybe the name of the test that is being defined in the jmeter file. Edit: If you freeze the video @34:41, you'll see in the filters at the top that you can filter by "Requests" (currently set to "All"), and I believe "Homepage" is the name of the request being defined in the test, and is being incremented by the number of threads. Edit2: Ignore the above :D Did some digging and correct info is below in another comment.
@@MattAllford Thanks for your reply! I can filter the request like you said, that’s not a problem. It just throws me slightly when I can run a simple test through jmeter alone to measure response time to a URL and create a listener for a Response time graph with only one line for speed over time, but then when I upload the exact same jmeter test in Azure Load Testing and end up with 3 different lines on the graph as stated… Filtering is fine, but without knowing which line to trust it’s hard to know what to filter.
Hey @user-ht4tq2hx2c! Totally get what you are saying. So I did some testing, and I think that you get a number of different requests to filter on, based on the endpoint you are testing. For example, in my scenario where I was testing my own blog, the JMETER configuration was actually just going to the root of the website, using http (not https). But my website has a 301 redirect from http to HTTPS, and the JMETER configuration was told to follow redirects, so this ended up in "multiple requests", even though I was only testing one endpoint in the configuration. I found that if I changed the "HTTPSampler.protocol" configuration and specified HTTPS, the metrics in Azure Load Testing changed from showing 3 requests, to 1. You will get some insight in to what is happening with your requests if you download the results in CSV format from a test, but I suspect something similar is happening for you? Hope that helps.
@@MattAllford hi Matt, sorry for the late late reply I did what you said, specified https and it worked and gave one clear line instead of several. It makes a bit more sense now why there were multiple before. It’s still interesting to see the difference between Azure load testing and Jmeter alone when using the same plan. But I’m happy now anyways. So thanks for your help, your assistance was much appreciated 🙂
Thanks mate! I appreciate the comment and question, it made me go back and do a bit of digging! It’s now something I’ll make sure to point out in future presentations on Azure Load Testing as well
Yep, you should be able to do that by deploying Azure Load Testing in to your VNET - learn.microsoft.com/en-us/azure/load-testing/how-to-test-private-endpoint
Hi quick query: When the test engines are running will they bring down the performance of the actual app service where my app is hosted ? ( trying to understand if these load tests would hamper user experience using the website at the same time )
Howdy. Yes, this is a live load test against the target endpoint you provide when performing the test, so it will generate real load and therefore impact the performance. It's not a simulation, it is real load hitting the target. So you're right in thinking through whether you'd want to do this against, say, a production endpoint, or maybe have another environment that is used for load testing, with the same architecture as your production site, but doesn't have active users that will be impacted by the test.
The catch I see with creating a separate environment is paying for the 1 instance even when load testing is not being performed. In my research, the one way I've noticed this can be tackled with is by using Azure Container apps that have the ability to go down to 0 instance ( no charge ) when not in use, and an autoscaling can be put in place for it to scale to 1 instance or more once the requests start coming in ( the only issue I see here is the initial requests the load test would send would fail with a 503 as there wouldn't be an instance available to service them ? ). Not an ideal solution, however let me know if a different method comes to your mind. Thanks a bunch for the response, great video ! :)@@MattAllford
@@user-ht4mn2ud1oI’m glad you enjoyed the video! This is really one of those “it depends” kind of things, but I really do mean that. If the infrastructure isn’t too large and complex, maybe you can deploy it via pipelines using Infrastructure as Code, deploy the application, load test against that environment, and then delete the environment. This could all be done using pipelines such as Azure DevOps or GitHub Actions. A lot of customers I work with will typically have multiple environments, so they will usually have a test/dev/staging/uat/whatever environment, separate to production, and one of those environments can (usually) be leveraged to run load tests against. Or as you said, if the endpoint is running in infrastructure like serverless functions or container apps that could be scaled to 0, maybe you can have a pipeline that scales it up, potentially even pauses if the app needs a little bit of time to get “warm”, and then your load test could be invoked in that same pipeline.
Hi Matt, Thanks for this video. But I'm facing several issues while uploading jmeter script and running scripts with CSV data config. Could you pls help? Below are the two issues i'm facing. Test run failed due to error in test run configuration: I have a JMeter script with csv data config and added necessary plugins , but still im getting the same Failure error. The test seems to be executing for long time and later it shows as failed. “Test run failed due to error in test run configuration. Please try again. If the issue persists, please raise a support ticket along with the test run id.” Whereas if i run a sample script without any CSV data files, it is running and the test is passed. Why these differences? do we have configure anything else while running with CSV data file? I’m trying to create a framework for my organization using Azure Load testing service, is it reliable to create one? Please help I have double checked by disabling csv data config and hardcoded the param values. still it says executing. The difference between the script that is working vs script that is not working is, one uploads the file and other one is basic login script. Where is the issue here? Let me add another issue in a separate comment. Please do help me to debug this issue.
Hi there, and thanks for watching! Unfortunately I don’t have the capacity to help with issues like this. If there are quick things I know the answer to, I’m usually ok to try and help, but this one seems rather specific and I just don’t have the bandwidth sorry. What I would suggest is: Checking the docs on reading CSV data, make sure to follow the note about CSV headers - learn.microsoft.com/en-us/azure/load-testing/how-to-read-csv-data?pivots=experience-azp Take a look at the troubleshooting docs to see if there are any other logs that can help you pin point the issue - learn.microsoft.com/en-us/azure/load-testing/how-to-troubleshoot-failing-test?tabs=portal Try logging your issue on the Azure load test GitHub page - github.com/microsoft/azure-load-testing/issues Hopefully that helps!
@@MattAllford one kind request/help, do we need to add anything additionally in azure load testing service for JMeter script that uploads a document to website?
Hey there, thanks for watching! The jmx were shared, there is a link in the description :) Here it is again though - gist.github.com/mattallford/afd3da52a5adb2ad93c8462f477e39ca Hope that helps!
@@MattAllford oh thank you, i wasn't expecting a response, forget about such a fast response. I was struggling with key vault - get secret call. But I was trying that for a secret parameter unlike getting the header for function app. It does not recognise the key... However, if i use simple get environment call it does pull out the key vault secret and pass it as parameter to https request. Your http header with key vault call will be very helpful now with available jmx. Thanks 🙏
No worries at all! I'll always endeavor to reply to people where I can, that's what this is all about! Let me know how you get on - hopefully having access to the jmx files helps you figure out the key vault retrieval.