Database integration tests

Did you ever struggle while trying to write integration tests without mocking the third party? I know I did.

I’ve known about Martin Fowler since I was a junior developer. He is one of my favorite technology thought-leaders and I enjoy reading the thought-provoking articles that he publishes on his blog. In this post, I want to expand on his article about the test pyramid, in which he explains the difference between unit tests, integration tests and, UI tests. I’ll expand on the part about integration tests.

I want to show you an easy way to run integration tests against a database. With this goal in mind, I’ve created this java demo project. It illustrates the concept and is useful as a starting point for more complicated applications.

In the past, I would delay writing the integration tests. I dislike the idea of not having any integration tests. Fixing errors at runtime is not something that I want to do in Java. Meanwhile, I didn’t want to invest time on setting up a separate test infrastructure. And tests running against a mocked database, are not testing anything. The options seemed limited:

  • mocking the database interaction layer
  • connecting to an in-memory database
  • connecting to a local/remote test database

Regardless of the option I picked, I always had the feeling that the solution could be more elegant. My hesitation would turn into frustration as the application grew and the tests would become harder to maintain. Recently, I discovered a better option: connecting the tests to a database running in a Docker container.

In my last project, Vlad and Max showed me that using docker containers simplifies running integration tests against a database. No mocking, no complicated infrastructure, no tinkering with configuration files. As long as you can run Docker on the build machine, you can run the tests.

This is the elegant solution that I was looking for:

  • install and configure Docker on the build servers and the developer machines that will run the tests
  • use docker-maven-plugin to build a docker image of the database and start/stop a container based on the image
  • populate the database with the necessary data (use ‘/docker-entrypoint-initdb.d/’ or a database migration tool like flyway)
  • finally, use maven-failsafe-plugin to run the integration tests

I like this approach because the integration tests are portable and easy to debug. If the tests run on my laptop, they will run also on the build machine. If the tests fail on the build machine, the problem will also appear on my laptop.

I also like that the tests are repeatable: the plugin builds the Docker image from scratch before the tests start and removes it after the tests finish.

Finally, I like that this approach is transferable to other types of integration tests. It’s easy to replace the database container with a dockerized REST API (in a move towards contract testing) or with a message queue container.

The demo project is more than a typical “hello world” application. The integration test starts a docker MySQL container and makes a simple SQL query. You will notice that the docker-maven-plugin configuration is more complicated than necessary, given such a simple test. The reason: the code samples that I found online seemed trivial. I wanted to have a “bootstrap” project that I could reuse in real-life projects without the hassle of gluing everything from scratch.

This approach to writing integration tests has opened my eyes to the possibilities that exist today in the QA automation domain. The tools built on top of Docker seem varied enough to accommodate all test types. The number of plausible excuses for not having a proper CI pipeline is getting too low…

Maven plugins that I like

I’ve been through a couple of projects until now and I’m noticing that there are a couple of maven plugins that are useful, but are not that famous. I’ll list them below, together with a small explanation of why I find them useful.

Build Number Maven Plugin (org.codehaus.mojo)

Sometimes you want to expose the current build version of your application without necessarily updating the artifact version (when you’re iterating fast and using *-SNAPSHOT). I didn’t get to use this one, but I can image that it would be good to have in a /info endpoint.

Apache Maven Shade Plugin

Sometimes you need to package all your dependencies into a single jar file. If you’re using Spring-Boot, you’ll have an easier life if you use the spring-boot-maven-plugin. Most of the time I use it for small projects, where I want to keep everything compact.

License Maven Plugin (org.codehaus.mojo)

Sometimes you need to add a license header to all the files of your project or you’ll need to pack with you application a file with all the licenses of the 3rd party libraries that you’re using. I wish i would have known about it three years ago when I was updating THIRD-PARTY.txt files manually.

Versions Maven Plugin

Sometimes you want to update the libraries that you’re using to the latest version. This plugin makes managing that update a lot easier. You have good control over the strategy of the version advance. This plugin makes your life easier, assuming you have enough tests to make sure that the update didn’t break your application. I just discovered this plugin and I like that it makes the library update chore a breeze.

Apache Maven Enforcer Plugin

Sometimes you need to make it clear what versions of OS, Java etc the application can be built with (because of incompatibilities). I like it that the set of already-made rules is extensive.

Apache Maven Checkstyle Plugin

Sometimes you need to enforce the codestyle that the team has agreed to. A bonus is that it helps the new members of the team to pick up the code standards immediately. I have a love and hate relationship with this plugin, but overall I’m happy to see the same kind of code everywhere.

OWASP Dependency-Check Plugin

You always want to have an application that is as secure as possible. This plugin allows you to automatically check for the latest reported vulnerabilities. I knew about the OWASP Top Ten for a couple of years and I learned about this plugin 2 months ago. I’ll never stop using it.

One last thing: I’m new at writing posts. If you have any suggestions to make this post more clear, please write a comment.