Feb 22, 2023

How to write better unit test assertions

Checking for the right conditions in test assertions is important to produce valuable tests. So, what makes for good test assertions? How can we write better unit-tests assertions? I’ll discuss both of these questions in this blog.

Author

Diffblue Marketing

An assertion is a statement in Java that enables you to test the assumptions you have made in your code. It is simple to use by adding the assert keyword and giving it a boolean condition. Used the right way, it can significantly improve your unit tests and so, as part of our ongoing series of blogs on how to write unit tests for Java, I’m going to focus here on how to write better assertions..

Most unit tests are organized in 3 sections:

– Arrange: set up the environment and prepare a bunch of objects to run the unit under test

– Act: call the unit under test

– Assert: check that outputs and side effects of the unit under test are as expected

These three sections are often referred to as the ‘arrange’, ‘act’, and ‘assert’ phases of a test. Other names include ‘given/when/then’ or even ‘setup/exercise/verify’. The first two sections exercise the code on some relevant scenario. The assert section ensures that the code behaves as expected.

Assertions replace us humans in checking that the software does what it should. They express requirements that the unit under test is expected to meet.

Now, often one can write slightly different assertions to capture a given requirement. But, as many of us have noticed, those small differences can have a big impact on how robust, maintainable, or effective a test is. Checking for the right conditions in test assertions is important to produce valuable tests.

So, what makes for good test assertions? How can we write better unit-tests assertions?

Let’s look at one example. Assume that we test a Java method whose signature is:

List<Employee> getEmployees(Predicate<Employee> constraint)

This method accepts a predicate on Employee and retrieves from some database the list of all employees that match the given predicate.

We may want to check that the function retrieves all employees older than 25 when it is called with a predicate of the form “the age of an employee is greater than 25”. We could do so in a JUnit test like this one:

@Test
public void getEmployeesGreaterThan25() {

  // arrange
  Employee mark = new Employee ("Mark", 19);
  Employee nina = new Employee ("Nina", 26);
  Employee jules = new Employee ("Jules", 31);
 clearDatabase();
  insertEmployee(mark);
  insertEmployee(nina);
  insertEmployee(jules);
  Predicate<Employee> constraint = e -> e.getAge() > 25;

  // act
  List<Employee> result = getEmployees(constraint);

  // assert
  assertEquals(result.size(), 2);
  assertEquals(result.get(0).getName(), "Nina");
  assertEquals(result.get(0).getAge(), 26);
  assertEquals(result.get(1).getName(), "Jules");
  assertEquals(result.get(1).getAge(), 31);
}

We got five assertions in this test. At a first glance nothing seems wrong with them. They check that the function returns all employees older than 25. But, looking closely, these assertions actually check slightly more than that. They check that the result list always contains Nina in the first entry and Jules in the second one. This particular detail could change as we update the implementation of getEmployees, or could even be altered by external sources of non-determinism.

Alternatively, we could have written the following lines in place of those 5 assertions:

for (Employee e : result) {
  assertTrue(e.getAge() > 25);
}

This version has its own problems, too. We now check that every employee returned by the function is aged 26 or more. But we are not checking that all such employees will be returned. If getEmployees was modified to, say, always return the empty list, this test would still pass.

Assert the exact desired behavior; not more, not less

Good assertions check what is precisely requested to the code under test, as opposed to checking an overly precise or an overly loose condition.

It is easy to write overly precise assertions, but the resulting test is often not maintainable or might be affected by slight non-deterministic variations in the environment. Our first example above falls in this class.

Also, it is tempting to write overly loose assertions, but the resulting test might not catch regressions that it should catch. Our second example above should have prevented someone from implementing getEmployees as a function that always returns an empty list, but it didn’t.

One assertion, one condition

It is tempting to aggregate multiple checks in one assertion. In our example we could have written something like:

boolean first = result.get(0).getAge() > 25;
boolean second = result.get(1).getAge() > 25;
assertTrue(first && second);

If that assertion fails, we will not immediately know if it is because the first or the second employee. Obviously the assertion can be split in two assertions which, should they fail, give us a better clue about the cause:

assertTrue(result.get(0).getAge() > 25);
assertTrue(result.get(1).getAge() > 25);

Don’t write assertions which check conditions that you could split to multiple assert statements. Always check the simplest condition possible.

Any gain in speed when writing complex conditions is lost when someone needs to debug why such assertion fails. The true value of an assertion is often realized only when it fails.

Assert requirements, not implementation details

Write assertions that check requirements, not implementation details of the unit under test. You do not want to update a test just because the unit under test has changed for reasons unrelated to the test.

Think how someone else could implement the same unit and ensure that the assertions would still pass. Assertions that check implementation details are often overly precise, so that is another hint you might want to look for.

In our example above, we should assert that the list returned by getEmployees contains one employee with age 26 and one employee with age 31, in any order.

Assertion matchers concisely express precise conditions

Almost all modern testing frameworks come with a library for assertion matchers. Assertion matchers are an API that lets you write assertions by, somehow, structurally describing the condition that needs to be checked.

Matchers make it easier to precisely express the behavior to assert and provide much better debugging information when they fail. Try to use them as much as you can.

Here is how we should have written the assertions for our test:

MatcherAssert.assertThat(result, Matchers.containsInAnyOrder(nina, jules));

The classes MatcherAssert and Matchers come from the Hamcrest library, where you will find plenty of other ready-to-use matchers. It is also possible to write your own matchers.

Takeaways

Assertions replace us humans in checking that the software does what it should. They express the requirements that the unit under test is expected to meet.
Assert the exact desired behavior; avoid overly precise or overly loose conditions.
One assertion, one condition. Don’t aggregate multiple checks in one assertion.
Write assertions that check requirements, not implementation details of the unit under test.
Use assertion matchers as much as you can.

Now, if you feel that you spend too much time writing tests, there is another way. Diffblue Cover autonomously writes human readable, ready to use Java tests where the generated assertions capture the current behavior of the code. Give it a try and see how much time you can save writing unit tests for Java code!