Etienne, thank you for publishing this content. I have found it to be very useful. Running Selenium in a pipeline is for me is a must and having used Selenium in the past I hadn't really got handle on getting it to work as you have shown, in a pipeline and within a hosted agent. This has been a great time saver - with your detailing not only the method of packaging the application but also the necessary steps in the pipeline to get the tests published, all invaluable. Thanks again.
@@TegaaaSolutions I was trying to perform "failed test case", were i can see in azure devops "Test" Tab (say e.g. 2 Pass, 1 Fail). So considering your example of google i couldn't perform, Requesting to please add a Failed test case video, whenever you get time. (I am basic learner in Selenium .Net Core)
Awesome video! Thank you for going through these steps. Do you have any good resources on how to better utilize Selenium for testing web applications? Thanks again!
Justin Miller do you specific questions the seleniumhq site has lots of information to author actual tests. If you have a specific problem I can maybe help you better
Hello. Thanks. In release it is sending me the error: "Dotnet command failed with non-zero exit code on the following projects : D:\ag3\_w 127\a\_PruebasSelenium_ASP.NET_Core-CI\drop\SeleniumDemo.dll in the Run Selenium Tests task
Hi , thanks for the explanation. But it is working absolutely fine for passed test cases. However, if one test case is failing ..the release is failing..how to deal with this?
Well it depends. The intent is that if you have tests that fail you should not continue with the release since there is an issue. You should validate why the test failed, is it the test? is it the application? if it's the test, then you should fix it and retry, if it's the app then you should fix the app. In either case, something is not well with your application and that release should be rolled back or rolled forward depending on how you deal with bug at your company.
Hello Etienne! In the 0:44 seconds of your video you ask us to click on new solution but I don't have this option, can you please help to enable it ? thanks
I am getting error like ##[error]Error: There was an error when attempting to execute the process 'C:\Program Files\dotnet\dotnet.exe'. This may indicate the process failed to start. Error: spawn C:\Program Files\dotnet\dotnet.exe ENOENT, any clue?
Have you installed dotnet core on your build server? The same version that you are running locally. You'll need to install the SDK for that version on your build server.
Thank you for the great video! I have a question that I got " Project file(s) matching the specified pattern were not found" message, how to fix it, please?
Hi Etienne, Nice video, I still have a question though, You could have "run" the test in the BUILD pipeline , after building everything right ? the extra step of the RELEASE is not mandatory if you just want to run the test and not actually deploying anything. is it right to say that ? Thanks
You can absolutely run the test in the build, BUT, Selenium tests (UI/Functional Tests) are usually run against a deployed instanced. So you normally would want to run this after you released. But if you are deploying inside the build then sure you could run those test there.
first of all - great video. I wonder if this a dotnetcore version issue but when i create the build and dotnet publish, the chrome driver exe doesn't go to the artifact staging directory directly, it's being copied to the source directory even when it says publish chrome driver in the project file
Alexandre Salomon hey Alex thanks, did you add the bit in the csproj to publish the chrome driver at publish time. Can’t remember exactly where it is in the video. I had the same issue before adding the bit to the proj
Etienne Tremblay hey yup its there, its actually executing that task it from what i see in the logs and copying the exe from the nuget dir. i basically had to use the artifact staging directory as the output location for the build for all the driver executables to be there in the final published artifacts
Great video! One question: Is this possible to run without the headless option? My tests require some "outside the browser" interaction so I think with the headless option some tests might fail.
Well you certainly can but your build server would need to run the agent in interactive mode and you’d need to keep a session alive on the build server with the build account to do it. Check docs.microsoft.com/en-us/azure/devops/pipelines/agents/agents?view=azure-devops&tabs=browser#interactive-or-service and docs.microsoft.com/en-us/azure/devops/pipelines/agents/v2-windows?view=azure-devops#run-interactively
You are correct Kat but selenium test usually need the application deployed in order to run so that’s why you usually run them as part of the deployment steps
Thanks for this initial setup, very clear. I do have some questions since I did get the tests to run, but they all fail with a chromedriver timeout error. Now our application requires you to log in via AD account. Which works locally, when I'm logged in with a Pulse VPN connection. How would I approach this? Would I need to change the way I log into the account or is there another way?
Hooo, ya connecting with Windows Creds (which are different then the account running the tests I presume) is a bit tricky, you need to launch the chrome driver with another user. Check this code as a start gist.github.com/tegaaa/b48a2d0b79808b4cbb05aae850326ffe
Thanks. Detailed explanation. I setup selenium c# test in Azure now, but unfortunately, when the release pipeline runs, getting error in as "##[error]Project file(s) matching the specified pattern were not found." unable to solve it, any help, will be highly appreciated. Thank you,
@@TegaaaSolutions Thanks for quick response. I using windows and first build pipeline to build, restore and publish ran successfully but the release pipeline with dotnet core job using vstest fails with above mentioned error..
@@bobbysiddhu1599 ok so it’s the same agent for both build and release then. Next thing is the test task parameters are they pointing to get all test dlls? Or is there a filter that does not match the name off your test dll...
Well since cordova is built on node.js, you can deploy your app to it's destination (is it a phone application or a web site) and then run your selenium tests from a browser (if web site). If it's a phone application you'd have to setup a subscription on Visual Studio App Center and then you can use the mobile test infrastructure. But I'm not sure if that supports Selenium or you'd need to write some new tests.
Hey Etienne, great tutorial! Could you maybe give me a hint about getting an error at the end of my test steps. The tests are running perfectly and will be published to the test section within the release, but however i receive an error which says: Total tests: 4. Passed: 3. Failed: 1. Skipped: 0. Test execution time: 50.6905 Seconds Error: The process 'C:\Program Files\dotnet\dotnet.exe' failed with exit code 1 Dotnet command failed with non-zero exit code on the following projects : d:\a 1\a\TestProject\drop\TestProject.dll
Thanks for this useful upload. I just use NUnit framework with dotnet core. I receive the same error while Releasing. It ran all my test cases but while completing the run test task it says the error above error. error Dotnet command failed with non zero exit code on the following projects. Also I checked the Tests tab, it says no Runs. Plz help me out here tried all possible hacks.. Thanks in Advance..
I have a run settings in my project and I can't run my test because of that, I try a lot of commands like "--Settings:Configuration/chrome.dev.runsettings" but I don't why the artefact can't find the path
@@TegaaaSolutions So, the problem wasn't that. I find the path and everything, but the problem was when to do in this way with .runsettings on release, the path couldn't be found, but if we do this in the build, everything goes fine and I have the same dashboard, graphics, and results like you, I don't find the reason to create and have this results just on release, I'll create an article or tutorial someday about it. If you can talk more about ping me here
Let's say I have one solution and two projects: My automation (with mapping and what not) and then a unit test project (the one with my tests). How would that work? I'm trying to use these steps and it seems I'm running into an error "project files matching the specified pattern were not found" and I'm assuming it's because I have two project files?
@@TegaaaSolutions Thank you very much for responding, I'm running through the video and when it comes to Azure, I'm not able to see the same setup in the video, they must have made changes since the video but I think the problem is that this project I'm doing is .net 4.6.1 and I'm assuming maybe that's where there is a problem? I'm still a bit new to Azure but essentially when I follow the video and go to create the release, it's saying that it fan't find the path to the .dll
@@TegaaaSolutions I can't tell you how much I appreciate the help. I went back and re-did it using .net core and it's getting way further but on the run selenium tests portion of the release it's kicking me an error "The process 'CProgram Files\dotnet\dotnet.exe' failed with exit code 1" have you ever seen this happen?
Moses A which agent are you using for the release part it looks like dotnet core is not installed on that machine. Also it may be that I was pointing to it but you sont need to now
Can you Create an asp.net web application which can run the selenium test you just created? For eg. We can go to this website and click on a button which says search for cheeze and on button click the script will be executed in azure and we will get the results in a textbox
Sorry for the late reply, I'm not sure I understand, you want an application that starts Azure DevOps Release Pipeline to run test and then get the result of the google query in the textbox? Not sure that makes sense...
While creating the "empty" but no empty repo, he picked the options to add a README and a .gitignore files. In any case, you should still be able to do all further steps without having those files. Cheers.
@@TegaaaSolutions I did some searching and got a solution for this, attached the code ChromeDriver(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location), chromeoptions); ChromeOptions chromeOptions = new ChromeOptions(); var driverPath = Path.Combine(Directory.GetCurrentDirectory()); var envChromeWebDriver = Environment.GetEnvironmentVariable("ChromeWebDriver"); if (!string.IsNullOrEmpty(envChromeWebDriver) && File.Exists(Path.Combine(envChromeWebDriver, "chromedriver.exe"))) { driverPath = envChromeWebDriver; } ChromeDriverService defaultService = ChromeDriverService.CreateDefaultService(driverPath); defaultService.HideCommandPromptWindow = true; driver = (IWebDriver)new ChromeDriver(defaultService, chromeOptions); driver.get(....)
