The cleanest way to use Docker for testing in .NET 

Respawn is getting a video next month. it’s pretty cool

I will have to test Respawn. I created my own solution. It uses database snapshots and the database is "seeded" from a .SQL script file. The first test create the database and make the snapshot; other tests revert back to the snapshot before running.

Why not just create a local image with static data already there, and run containers based on that image? I haven't tested this though

@@mohammadazhdari4469 It's a perfectly fine solution it just needs a bit more handholding because you have to keep it up do date

I have just finished implementing a similar approach, but to handle parallelism, we are using 1 db container instance per test assembly, then 1 Sql database per testFixture. We then override the connection string in the OneTimeSetUp of each TestFixture. It allows parallelism at the class level while avoiding too much RAM usage. ultimately I'll want to use only 1 container for all the assemblies, but it requires inter-process synchronization (Mutex) for making sure we create the container only once. We have also appended part of the hash from the list of migrations to the name of the container, so whenever the list of migrations is modified, a new container is bootstrapped. Another thing I noticed is it's best to leave the container running between runs (at least during local development), because otherwise it takes at least ~5 seconds to start on each run. One thing I found lacking with the TestContainers is the lack of support for already-running containers. Because of that, we had to implement some of the code manually and use a wrapper class to handle both cases. This issue explains the problem: github.com/testcontainers/testcontainers-dotnet/issues/506

Do you use the containers inside a CI/CD pipeline such as Azure DevOps? If so, how?

It would be good if you can write a post about how you do it ( custom abstactions for container and dbcontext that give us ability to create one or several databeses per any scope of tests)

you don't need custom code, you can use this overload (on the TestcontainersBuilder) and it'll choose a random port for you: .WithPortBinding(1433, assignRandomHostPort: true)

Assuming you're using EF Core and not Linq2Db, for example, you can use the DbContext to get the migrations and migrate if any migration is pending. Otherwise you can call EnsureCreated which will only work on a fresh database with no tables.

It depends on your execution model and whether you wanna run all the tests in parallel or in sequence

There are probably 4-5 alternatives that I’d use before I even considered RavenDb for any real world application

@@nickchapsas Eeks - that's pretty .. damning :(

@@nickchapsas Hey Nick. Clearly you are an advocate for C# (me too). I have yet to see a database client that comes even close to RavenDB's C# integration. My words truly cannot give enough justice to RavenDB. As a fellow C# developer, I highly recommend you to give it another try.

Not to mention the fact that I successfully launched a video game in production using RavenDB. Had no problems handling 17k+ concurrent users. I picked RavenDB for a good reason. The C# integration was above and beyond. Try the MongoDB client for 10 minutes and you will see how terrible it is.

@@nickchapsas could you name those alternatives? have you tried ravendb yet?

Can you say a bit on why it is so? What makes use of xUnit "nice"? :-?

@@RajaKajiev not XUnit, but Xunit Collections

@@PeriMCS ok, meditate on this I will (c) Yoda

Standard practice is to keep tests independent of each other

It depends on the usecase. I use both and in my integration testing course I show both usecases

Sure thing, you can either bake that on the docker startup level. Simply call the scripts towards the database your gonna run the tests against. You might wanna look at a library called Respawn to allow you to get back to a given state for every test

In this example it doesn’t create a new database for each test but rather one database for each test set which in this case it is a class. If you group your tests logically and do minimal clean up, you can run less databases like I show in the example. You can also run one and clean up after every test assuming you configured the tests to run one after the other. You control this fully and it is a balance between how much you wanna cleanup and how much you can parallelise

@@nickchapsas Yea, sorry, it is creating one database for each test set, not for every test, my bad but still it can be a lot when you have more and complicated tests. So, is there any option in the package to differentiate and control the created containers? For example, putting an attribute to use a naming instance ([ContainerName="Container1"] for example) over the tests to use an existing one or create a new one. Or should we do this manually?

To answer my own question, after couple of experiments, I think best way is to handle multiple setups and prevent creating new databases for every setup is to use the solution @Daniel Mackay gave (using Respawn).

The pipeline needs to support Docker but that's it. I've used it with TeamCity without any problem

You could use .EnsureCreated() instead of applying migrations. It's way faster

You won't really see a difference because of how memory is allocated on containers. It is the database itself which is the slow part, not the spinup of containers

Sure it can

Yes my integration testing course covers this in way more detail and it shown another way to do this for applications with a front end

@@nickchapsas now i just need to save to get the bundle course

I'm guessing it's somewhere in the DatabaseInitializer.cs file in the Database folder of the project.

You can also have it be in the ConfigureServices in the webhost builder in the WebApplicationFactory class

Sorry for that, it’s there now

Absolutely. I’ve been running this in my CI pipeline in TeamCity

If your Agent has docker up and running, it will run it without problem on Azure Devops.

Respawn is great for this I’m planning to make a dedicated video on the library

If you need to control docker from inside of a docker container, you can bind mount docker control socket into the container.

I've been using TeamCity for that, and in TeamCity it is possible to have docker running in the agent yes

@@nickchapsas Are you running the tests in the host? We have a problem where we run the tests while building the docker image (so in docker build there is a command that runs the tests) which makes it really hard to use this tool. Have you faced the same problem?

The container itself isn't the "heavy" thing. Its the data that you gonna need to seed into it that might take time. DB Instances are effectively as cheap as containers or at least you wouldn't see a visible difference.

A) in this case migrations run on startup and create the schema. They are usually very fast. If you have a very lenghty migration time, simply create a docker image with the schema and all the data pre-seeded and use that for testing. B) Yes, if you don't dispose the container in the dispose method it will stay up C) In C# there are multiple ways to interface with the DB so you control the transaction scope yourself. This example shows how you can create a clean DB every time so you don't have to worry about cleaning data.

B) .WithCleanup(false)

These are not unit tests, these are integration tests. It depends on the metal you are running on but with the ram we have today, you can run a lot of them before you have any problem especially with them being created and deleted as the tests are running

Sure thing. You can deal with it in multiple ways from a docker image that has them already run, to backup restore on startup to using a library like Respawn to go back to a snapshot

It's not for each test. It's for each test collection

You can run your migrations and seed data on startup or have a docker image with all the schema and data pre-seeded

@@nickchapsas thank you for the answer! I very much appreciate that. We tried migrating DB in a microservice itself, but when we scaled it to multiple instances, we had to lock migration somehow. And this adds unnecessary complexity. Also, we tried creating custom images, but this didn't work too. Because a script that creates schema starts at the same time as a DB server, so you can't wait for an open port, you should wait for the last migration applied, which is hard to determine. I'll research this lib, maybe it can help with my problem.

The code is available to my patreons

I think I got the idea - with this approach you don't need to think about managing test data in DB at all

Removing based on typeof will remove all the implementations. You'll usually just have one but there are cases where you might have more than one

btw nice video. lookin forward to swtich from SQLite inmemory to this and give it a try .

The container won't go down unless the test executes and the RAM is relative to your docker configuraiton

Which random ports? The ones that the library picks of you automatically? Those internally are checked for availability so it will never pick ports that are not available.

@@nickchapsas Ok then, thats cool.. nice video, thanks

You clearly not now why RDBMS are used and when to go to NoSQL. For you it's just SQL and NoSQL.

Be serious, where do you see vscode in the video?

First of all , he didnt teach docker at all. He presented a nuget package called "TestContainers" to simplify your integration tests. Second, there is no vscode, its called "Rider" from jetbrains. Finally one question, what is your point? Using vscode is not bad habbit. Use the IDEl, you feel comfortable with.