API Series Part 6 - ASP.NET Core 2.0 and Integration Testing

All code can be found on Github. See the series introduction for the list of posts in this series.

This is the first post on automated integration testing in this series but it will not be the last. In this first post we'll look at how to use the Microsoft.AspNetCore.TestHost package to run integration tests of our API. In future posts I want to take a look at SpecFlow (because I love it) and also scripting a short-lived test environment with its own short-lived database which is torn down after the test run.

Create an xUnit Test Project

First of all I created an xUnit test project.

AddXunitProject.PNG

I added a reference to the Govrnanza.Registry.WebApi project which is the application we are going to test. Then I added the Microsoft.AspNetCore.TestHost NuGet package.

Refactor Program.cs

Now we create a test fixture that contains the test server and HttpClient that we need to run the tests. The test server hosts our ASP.NET Core application, meaning we can serve HTTP requests without the need for a real host.

The first thing we need to do is refactor the Program.cs. Currently we have a method that returns an IWebHost instance but the TestServer class needs an instance of IWebHostBuilder.

Original

public static IWebHost BuildWebHost(string appRootPath, string[] args)
{
    var webHost = new WebHostBuilder()
        .UseKestrel()
        .UseContentRoot(appRootPath)
        .ConfigureAppConfiguration((hostingContext, config) =>
        {
            var secretsMode = GetSecretsMode(hostingContext.HostingEnvironment);
            config.AddGovrnanzaConfig(secretsMode, "REGISTRY_CONFIG_FILE");
        })
        .UseStartup<Startup>()
        .Build();

    return webHost;
}

Refactored

We now have a new method that returns an IWebHostBuilder that our TestServer will need.

public static IWebHost BuildWebHost(string appRootPath, string[] args)
{
    var webHostBuilder = GetWebHostBuilder(appRootPath, args);
    return webHostBuilder.Build();
}


public static IWebHostBuilder GetWebHostBuilder(string appRootPath, string[] args)
{
    var webHostBuilder = new WebHostBuilder()
        .UseKestrel()
        .UseContentRoot(appRootPath)
        .ConfigureAppConfiguration((hostingContext, config) =>
        {
            var secretsMode = GetSecretsMode(hostingContext.HostingEnvironment);
            config.AddGovrnanzaConfig(secretsMode, "REGISTRY_CONFIG_FILE");
        })
        .UseStartup<Startup>();

    return webHostBuilder;
}

We could not rely on the real code from the Program.cs and write our own copy for the integration test but I want to keep things DRY.

Modify our Startup.cs

When we run our tests with the TestServer, we don't want to run any Swagger related code. So we add a condition that prevents it running when in the Test environment.

public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
    StartupLogger.LogInformation("Configure started");
    if (!env.EnvironmentName.Equals("Test"))
    {
        app.UseStaticFiles();
        app.UseSwagger(SwaggerHelper.ConfigureSwagger);
        app.UseSwaggerUI(SwaggerHelper.ConfigureSwaggerUI);
    }

    app.UseMvc();
    StartupLogger.LogInformation("Configure complete");
}

Create our test fixture

Our test fixture performs the following actions:

  • Create the environment variables that the Registry service needs. Note that we need to adjust the paths relative to the execution context of the tests.

  • Call the GetWebHostBuilder method and pass the results to the TestServer

  • Instantiate an HttpClient that will make calls to the TestServer.

namespace Govrnanza.Registry.WebApi.IntegrationTests.Helpers
{
    public class TestFixture
    {
        public TestServer Server { get; set; }
        public HttpClient Client { get; set; }

        public TestFixture()
        {
            // We must configure the realpath of the targeted project
            string appRootPath = Path.GetFullPath(Path.Combine(
                            AppContext.BaseDirectory,
                            "..", "..", "..", "..", "Govrnanza.Registry.WebApi"));

            // set environment variables the application needs to read on start up
            Environment.SetEnvironmentVariable("ASPNETCORE_ENVIRONMENT", "Test");
            Environment.SetEnvironmentVariable("REGISTRY_CONFIG_FILE", Path.Combine(appRootPath, "devAppSettings.json"));
            Environment.SetEnvironmentVariable("REGISTRY_DB_PASSWORD_SECRET_FILE", Path.Combine(appRootPath, "InsecureSecretFiles", "RegistryDbPassword.txt"));
            Environment.SetEnvironmentVariable("REGISTRY_USE_DOCKER_SECRETS", "false");

            Server = new TestServer(Program.GetWebHostBuilder(appRootPath, null));
            var client = Server.CreateClient();

            Client = client;
        }
    }
}

Write our first test

We won't write a useful test now. I will do that later, using SpecFlow, for now we just want to validate that the text fixture works.

namespace Govrnanza.Registry.WebApi.IntegrationTests
{
    public class ApisControllerTests
    {
        private TestFixture _testFixture;

        public ApisControllerTests()
        {
            // ARRANGE
            _testFixture = new TestFixture();
        }

        /// <summary>
        /// This is a bad test, it is just to verify requests can be made via the TestServer
        /// </summary>
        [Fact]
        public async Task WhenGet_ThenReturnsOk()
        {
            // ACT
            var response = await _testFixture.Client.GetAsync("/api/v1/apis");
            var contents = await response.Content.ReadAsStringAsync();

            // ASSERT
            Assert.True(response.StatusCode == HttpStatusCode.OK, $"Expected OK but received {response.StatusCode}");
        }
    }
}

We can see the JSON returned when we inspect the response:

Our test passes.

TestPass.PNG

Conclusions

So we see that running tests against a simulated hosting of our API is pretty easy. The harder part of integration testing is the data and being able to run tests against a real database. Yes you can use Entity Framework in a In Memory mode but I advise against it. I have seen a few SQL bugs in EF Core 2.0 now where it generates the wrong query and you'll only discover those issues when you run your application against a real database.

If you really don't want to run against a real database then know that you are running a big risk and perhaps you should look at unit tests. However unit tests in the most part I believe are a waste of your time and effort. They should be reserved for complex algorithms rather than a general testing strategy. Only through integration tests can you get real confidence about your code, and that is what testing really is all about.

But running and maintaining integration tests is harder, so we'll be looking at changing that through automation and Docker. What you don't want is to share a staging or development database with your integration tests. Each automated test needs the freedom to create and destroy data and have complete isolation.

Stay tuned for more on integration testing.

See the series introduction for the list of posts in this series.