Writing and running automated tests with every code change is considered best practice in the software industry. So, why not write automated tests in Java too? Whether you’re a junior developer just starting, or an experienced developer at a company that still uses manual testing, this guide will help you start writing Java unit tests quickly. It will also help you convince others to appreciate Java unit testing best practices. All you need is some Java and build system knowledge and some patience to get started. This guide will take you through a Java unit testing tutorial with JUnit and Mockito and point out Java unit testing best practices to improve test and code quality along the way.
What is unit testing in software and why write unit tests?
Unit tests are small, focused pieces of code written to verify the correctness of individual units of a software application. In Java, a unit typically refers to a single method or class. The primary goal of unit testing is to ensure that each unit of code works as expected in isolation from the rest of the application.
A unit test typically follows this structure:
- set up the test environment
- call the method being tested with specific inputs
- assert that the output or behavior matches the expected result.
Writing unit tests undoubtedly takes time, but they help developers:
- ensure code correctness
- catch bugs early—importantly before a customer does
- catch regressions—make sure earlier features continue to work as expected
- check that the code meets the requirements
- reduce QA workload and responsibility
- enable continuous integration
- facilitate test driven development (TDD).
Writing tests, especially unit tests, encourages developers to write modular, testable code and demonstrates that the code meets requirements. Developers often find bugs while writing tests, which prevents errors from reaching QA or customers, and thus saves time, effort, and money. Developers may also find that when writing tests, the code they have written does not meet the requirements or that the requirements they were given are unclear, allowing this to be addressed with management before the code is released.
When tests are maintained along with the code, they continue to provide Java developers with:
- a safety net for refactoring code
- a way to catch regressions
- easier debugging.
Well-tested code increases developers’ confidence in the code base, and in turn, they can develop features and make changes more quickly.
Types of tests in software testing
Here are some common types of software tests from low-level to high-level:
- Unit testing checks that the smallest building blocks of code are working as expected. Usually, each unit test checks a single case of a single method. Developers run and write unit tests day-to-day.
- Component testing focuses on whether the component delivers the required functionality overall, for example, the functionality of an entire module or service. Component testing tests how the units work together.
- Integration testing checks that dependent components work together as expected. Unlike component or unit testing, a change in any component can cause the test to fail.
- End-to-end testing verifies that the entire solution works as a user would experience it and tests multiple systems or services together. Testing is usually done using the user interface with custom scripts or automation tools.
All software tests have their place, but this guide will focus on unit testing, as unit tests are the tests that developers should be writing alongside their code day-to-day.
How to use JUnit to write Java unit tests
Prerequisites for unit testing with Java testing frameworks
Your project needs a build system to run tests effectively and to complete this JUnit tutorial. Follow the getting started instructions for Maven or Gradle if you need to. Most IDEs, such as IntelliJ Community Edition, can help you with this too. An IDE will make running tests easier.
The code in this guide was written using Java 11, Maven 3.9.9/Gradle 8.10.2 (Kotlin), and IntelliJ Community Edition 2024.1.4 with the dependency versions given in each section.
Example code for a quick JUnit tutorial
Imagine that you are working on a project and the requirements of the project state that the method average of the class DataAnalyzer should return the average of an array of integers. You wrote the following code:
package org.example;
import java.util.Arrays;
public class DataAnalyzer {
public static double average(int[] data) {
return (double) Arrays.stream(data).sum() / data.length;
}
}How can you write tests for the code to check for these requirements?
How to set up JUnit
First, you will need to introduce a test framework into the project. A test framework provides an easy way to write, organize, and execute tests. This guide uses JUnit 5 because it is the de facto standard Java test framework.
How to install JUnit in Maven or Gradle
To install JUnit 5 in Maven, add the following to your dependencies in the pom.xml:
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<version>5.10.0</version>
<scope>test</scope>
</dependency>or in Gradle add the dependencies and tasks:
dependencies {
testImplementation(platform("org.junit:junit-bom:5.10.0"))
testImplementation("org.junit.jupiter:junit-jupiter")
}
tasks.test {
useJUnitPlatform()
}How to organize test classes
Test classes are organized into packages, classes, and methods within the src/test/java directory that mirrors the structure of the project in src/main/java.
The class under test is called DataAnalyzer, so the test class will be called DataAnalyzerTest and contained in the file src/test/java/org/example/DataAnalyzerTest.java. The suffix Test in the test class name is a common convention and is the default pattern used by Maven and Gradle to identify test classes. Here is the empty test class:
package org.example;
class DataAnalyzerTest {
}How to write a unit test
When writing a unit test, you need to think about both the requirements and the edge cases of the code itself. This section will focus on the requirements first. Recall that the requirements state “the method average of the class DataAnalyzer should return the average of an array of integers.”
A single unit test is usually a method within the test class, so create a package-private void method for this test with a method name that describes what the test will do:
package org.example;
import org.junit.jupiter.api.Test;
class DataAnalyzerTest {
@Test
void testAverage() {
}
}The @Test annotation is a JUnit annotation that tells the build system or IDE that this is a test. If you are using IntelliJ or Eclipse, you will see that you can now run this method as a test with a green arrow to the left of the test method.
You need to call the method under test with suitable arguments. First, choose a small array with small values for the first test. The setup for a test is put under the “Arrange” comment. Then, call the method under test with the argument. The method call is put under the “Act” section. The comments help keep the test organized.
package org.example;
import org.junit.jupiter.api.Test;
public class DataAnalyzerTest {
@Test
void testAverage() {
// Arrange
int[] data = new int[]{0, 3, 6};
// Act
double result = DataAnalyzer.average(data);
}
}How to write JUnit assertions in a unit test
So far, the test is calling the method under test, but it is not yet checking the method output. JUnit provides assertions to do this. Assertions are used to check the output or side effects of the method under test. An assertion passes when the expected condition is met, and fails when it is not. Add an assertion on the output of average when called with the argument data, which is stored in the variable result.
package org.example;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
class DataAnalyzerTest {
@Test
void testAverage() {
// Arrange
int[] data = new int[]{0, 3, 6};
// Act
double result = DataAnalyzer.average(data);
// Assert
assertEquals(3, result);
}
}The expected output of the average method for this data is 3, so you can use the assertEquals method from JUnit to verify this. If the result is 3, the test passes, and if the result is not 3, it will fail. There are many assertions to choose from depending on your code that are not covered in the JUnit tutorial. For example: assertTrue, assertFalse, assertEquals, assertNotEquals. The full list is here.
How to run JUnit unit tests
You can run the test using an IDE or run a build system command in the terminal, with either mvn test -Dtest=org.example.DataAnalyzerTest, or gradle test –tests org.example.DataAnalyzerTest or ./gradlew test –tests org.example.DataAnalyzerTest depending on whether your project has a gradle wrapper or not. Once you run the test, it should pass. Congratulations—you’ve just written and run your first unit test!
To observe the test failing, you can change 3 in the assertion to another number and rerun the test. You should see an error message. Don’t forget to change it back.
How to write good unit tests
Good unit tests will pay dividends in the future, while poorly written ones may make it harder to pinpoint failures, so it is important to try to write the best unit tests you can the first time. Good unit tests:
- Cover all or most cases: As the writer of the code, you have to think of all the cases that could cause different or unexpected behavior of the method under test. These may be obvious from the software requirements, or not obvious at all. Many edge cases are discovered and fixed while writing unit tests.
- Isolate functionality: You want to make sure that your unit test is testing a single case of only the method under test as much as possible. Because code is complex and methods call other methods and have dependencies, make sure any called methods have their unit tests.
- Are fast: A unit test should run in milliseconds usually.
- Do not change from run-to-run and do not depend on the order they are run: Tests should not depend on global state. For example, you should not use random numbers in a test, set static variables, or depend on non-constant static variables.
- Have complete and specific assertions: Assertions should check the state of the return object, account for any side effects on arguments, and be as specific as possible.
How does the unit test for average stack up against these guidelines?
Cover all or most cases: The test covers only one case. Good unit tests consider the potential edge cases. Add a test for an empty array:
package org.example;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
class DataAnalyzerTest {
@Test
void testAverage() {
// Arrange <- new test case for an empty array
int[] data = new int[]{};
// Act
double result = DataAnalyzer.average(data);
// Assert
assertEquals(0, result);
// Arrange <- previous test case
data = new int[]{0, 3, 6};
// Act
result = DataAnalyzer.average(data);
// Assert
assertEquals(3, result);
}
}When this test is run, it fails. The output for the average with an empty array is NaN, not 0. This is not specified in the requirements. What should happen when the array is empty? One benefit of writing unit tests is that they can help clarify requirements that may initially seem reasonable – but are incorrect or incomplete – before the code goes into production.
To complete your test suite, you may want to consider test cases including negative numbers, maximum and minimum integer values, where the average is a non-integer, etc.
Isolate functionality: The test runs on the average method which doesn’t call any other methods in the code base, so it isolates functionality. However, when the first assertion fails, the rest of the test is not run. These should be two separate test cases. That way, if you need to make changes to the empty array test, you won’t have to change the passing test case. The method names should be chosen to be descriptive. Here is an updated unit test:
package org.example;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
class DataAnalyzerTest {
@Test
void testAverage_smallNumbers() {
// Arrange
int[] data = new int[]{0, 3, 6};
// Act
double result = DataAnalyzer.average(data);
// Assert
assertEquals(3, result);
}
@Test
void testAverage_empty() {
// Arrange
int[] data = new int[]{};
// Act
double result = DataAnalyzer.average(data);
// Assert
assertEquals(0, result); // <- fails
}
}
- Are fast: These tests are fast and should run in less than 100ms.
- Do not change from run-to-run and do not depend on the order they are run: These tests will not change from run-to-run because they do not depend on global state or random values. As an example of what not to do, do not use random numbers to construct the data argument, because that would change run-to-run.
- Have complete and specific assertions: The assertions are specific, but you can make use of a delta for assertions with doubles and floats, which might help prevent the tests from failing due to floating point arithmetic errors. You can also add a message that appears when the assertion fails, which helps clarify the purpose of the test. For example:
package org.example;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
public class DataAnalyzerTest {
@Test
void testAverage_smallNumbers() {
// Arrange
int[] data = new int[]{0, 3, 6};
// Act
double result = DataAnalyzer.average(data);
// Assert
assertEquals(3, result, 0.001, "The average of 0, 3 and 6 should be 3");
}
}How to use Mockito in unit tests
What is mocking?
Unit tests must construct the required arguments for the method under test first (the Arrange section). Many times this is straightforward, such as in the previous example which required only constructing an array of integers. However, it is not always possible or easy to construct and perform actions to call the method under test, and this is a case for using mock objects. A mock object simulates the behavior of a real object, and it can be controlled within the unit test.
For example, connecting to a database or interacting with the filesystem are unsafe operations and may have unintended side effects; these are good cases for using mocking. Instead of opening a real file and writing to it, the test can create a mock file, open it, and write to it. This does not affect the real file system and does not have side effects, but it simulates a real file for testing.
How to set up Mockito in Maven and Gradle
Mockito 5 is a commonly used Java mocking framework and is easy to get started with. Add the following to the dependency section for Maven for this Mockito tutorial:
<!-- https://mvnrepository.com/artifact/org.mockito/mockito-core -->
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>5.13.0</version>
<scope>test</scope>
</dependency>or Gradle:
testImplementation("org.mockito:mockito-junit-jupiter:5.13.0")How to mock a class and define method behavior
Consider a class GaussianDistribution that wraps around the Java Random number generator. The method generate scales and shifts the result of Random.nextGaussian and returns the result.
package org.example;
import java.util.Random;
public class GaussianDistribution {
private final Random random;
public GaussianDistribution(Random random) {
this.random = random;
}
public double generate(double mean, double standardDeviation) {
return random.nextGaussian()*standardDeviation+mean;
}
}Because the result is random, using GaussianDistribution in tests would be problematic. Randomness in tests would cause tests to change run-to-run. Mocking the class Random can help create stable, deterministic, and isolated tests.
To set up a mock for Random, use
Random mockRandom = mock(Random.class);Then define the behavior of the mocked class using Mockito.when and Mockito.thenReturn like this:
when(mockRandom.nextGaussian()).thenReturn(10.0);This tells the mock object to always return 10.0 when nextGaussian is called. Mocking provides a Random object that is deterministic to use in the test. Here is a possible test:
package org.example;
import org.junit.jupiter.api.Test;
import java.util.Random;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
public class GaussianDistributionTest {
@Test
void test() {
// Arrange
Random mockRandom = mock(Random.class); // <- Mock the class
when(mockRandom.nextGaussian()).thenReturn(10.0); // <- Define the behavior
GaussianDistribution gaussianDistribution = new
GaussianDistribution(mockRandom); // <- use the mock
// Act
double result = gaussianDistribution.generate(10, 10);
// Assert
assertEquals(110, result); // <- The result is deterministic
}
}
What are the benefits of using a Java mocking framework?
- Mocks can help isolate code for testing. In the example above, the mock removes the randomness of the Random dependency and focuses on testing the scaling and shifting in GaussianDistribution. It isolates the behavior of GaussianDistribution from Random.
Easy construction of complex or unknown dependencies. Some dependencies are difficult or impossible to construct for a unit test, such as central dependencies that require a complex setup or dependencies from a third party. Mocking allows you to continue writing tests in these cases if you know the behavior required from the dependency.
Mocking best practices
Mocking is defining the behavior of the mocked class twice – once in the actual class and once in the mocking setup. This can be problematic when changing or refactoring classes that are mocked in tests. Both the class and the mocks in every test may have to be updated. For this reason, it is recommended to mock stable interfaces only when necessary. In the example, Random is a stable class, so this is OK.
You might also find yourself needing to mock final or static classes and methods, or mocking constructors or void methods. Advanced mocking features that allow this are not always considered good practice and often require extra configuration of Mockito, which is not covered in this Mockito Tutorial. If you find yourself wanting to use these Mockito features, it is usually a hint of a bad code design. That said, not all code is within your control, so using these features is not bad in itself.
Further reading about Mockito
Mockito offers a whole suite of tools to help you write and verify mocks for unit tests. Detailed explanations are outside of the scope of this guide, but here is a brief list to help you get started reading the Mockito documentation to accompany this guide:
- Defining behavior: Stubbing
- Specifying generic arguments: Argument matchers
Assertions on mocked objects: Verifying method calls
How to write testable code
While this is not a guide to writing good code, thinking about SOLID principles can help developers write better code and enable better tests.
How to refactor hidden dependencies
When a class is responsible for constructing its dependencies, this can make the code hard to test. It is better to inject the dependency. This does not necessarily require a dependency injection framework, it simply means allowing the constructor (or setter) to provide the dependency. This will enable easy mocking, which is essential for dependencies that are databases, non-deterministic, or hard to construct.
Imagine you have written a service class UserService that depends on an existing repository UserRepository:
package org.example;
// You are unable to change this code
public class UserRepository {
public boolean isConnected() {
return false;
}
public String getUserById(int userId) {
if(!isConnected())
throw new IllegalStateException("Simulate connection error");
return "got" + userId;
}
}
package org.example;
// You have written this code
public class UserService {
public String getUserName(int userId) {
UserRepository userRepository = new UserRepository();
return userRepository.getUserById(userId);
}
}
You know you need to write a test for UserService.getUserName, but the call to userRepository.getUserById will always throw an exception because you cannot connect to the repository, meaning you can’t test past the initial if statement in the method.
It may seem hopeless because no matter what you try, you can’t mock it, but there is a problem with your UserService code. UserService constructs its dependency UserRepository. However, if you instead inject the dependency with the constructor, it can be mocked.
Here is the new UserService class and a test for it. Observe that the repository is now set by the constructor instead of being constructed in the method. You can now mock the isConnected method in UserRepository to test UserService.
package org.example;
public class UserService {
private final UserRepository userRepository;
public UserService(UserRepository userRepository) {
this.userRepository = userRepository;
}
public String getUserName(int userId) {
return userRepository.getUserById(userId);
}
}package org.example;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
public class UserServiceTest {
@Test
void testUserService() {
UserRepository userRepository = mock(UserRepository.class);
when(userRepository.isConnected()).thenReturn(true);
when(userRepository.getUserById(any(int.class)))
.thenCallRealMethod();
UserService userService = new UserService(userRepository);
assertEquals("got42", userService.getUserName(42));
}
}
How to refactor spaghetti code
It is hard to understand and test spaghetti code – code that lacks modularity, has tight coupling, poor naming conventions, or complex flow. Testing spaghetti code resembles end-to-end testing more than unit testing. A failure might tell you that something went wrong, but not where or why. Consider the poorly named method filter:
package org.example;
import java.util.ArrayList;
import java.util.List;
public class FilterNumbers {
public static List<Integer> filter(List<Integer> inputList) {
List<Integer> outputList = new ArrayList<>();
for(Integer number : inputList) {
if(number % 2 == 0) {
if(number > 0) {
for(Integer otherNumber : inputList) {
if(number.equals(otherNumber)) {
break;
}
}
if(!outputList.contains(number)) {
outputList.add(number);
}
}
}
}
return outputList;
}
}Can you figure out what the method filter does? It returns only the positive even integers in a list. Renaming the function to something more descriptive would help, as would breaking up the complex loops and conditionals into individually testable functions. In the revised code below, extractUniquePositiveEvenNumbers replaces the old filter method but serves the same purpose:
public static List<Integer> extractUniquePositiveEvenNumbers(List<Integer> inputList) {
return removeDuplicates(
extractPositiveNumbers(
extractEvenNumbers(inputList)));
}
public static List<Integer> extractEvenNumbers(List<Integer> inputList) {
return inputList.stream()
.filter(number -> number % 2 == 0)
.collect(Collectors.toList());
}
public static List<Integer> extractPositiveNumbers(List<Integer> inputList)
{
return inputList.stream()
.filter(number -> number > 0)
.collect(Collectors.toList());
}
public static List<Integer> removeDuplicates(List<Integer> inputList) {
return new ArrayList<>(new LinkedHashSet<>(inputList));
}@Test
void testExtractEvens() {
assertEquals(Arrays.asList(0, 2, 4, 6), FilterNumbers.extractEvenNumbers(Arrays.asList(0, 1, 2, 3, 4, 5, 6, 7)));
}
@Test
void testExtractPositive() {
assertEquals(Arrays.asList(1, 2), FilterNumbers.extractPositiveNumbers(Arrays.asList(-2, -1, 0, 1, 2)));
}
@Test
void testDeduplicate() {
assertEquals(Arrays.asList(1, 2, 3), FilterNumbers.removeDuplicates(Arrays.asList(1, 2, 2, 3, 3)));
}
@Test
void testExtract() {
assertEquals(Arrays.asList(2, 4, 6, 8, 10), FilterNumbers.extractUniquePositiveEvenNumbers(Arrays.asList(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11)));
}
When code is modular and readable, it is easier to understand the purpose of each unit and to test them in isolation. This also makes it easier to find errors. For example, perhaps you realize through the tests that you want the method extractPositiveNumbers to include 0. The tests show that it does not include 0.
How to refactor code that depends on global state
The use of global state can lead to unpredictable behavior in software. The unpredictable behavior may show up as randomly failing (or flaky) tests. Consider this simple class and its tests. The class Counter increments or decrements a static integer.
package org.example;
class Counter {
private static int count = 0;
public static void increment() {
count++;
}
public static void decrement() {
count--;
}
public static int getCount() {
return count;
}
}
package org.example;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
class CounterTest {
@Test
void testIncrement() {
Counter.increment();
assertEquals(1, Counter.getCount());
}
@Test
void testDecrement() {
Counter.decrement();
assertEquals(0, Counter.getCount());
}
}
These unit tests run and pass in the order they are written but will fail if they run in the reverse order because the variable count is static and is not reset between tests. Furthermore, they might fail if another test class also changes count. The tests may randomly pass and fail, causing developers frustration and to lose confidence in the unit test suite. Even worse, changes to the build system or configuration can change the order in which the tests are run suddenly making tests fail after years of running without issue.
It is better to rewrite the class as non-static, and then write tests that create a new instance for each test. However, you should consider the reason the class was written like this, and to make sure that refactoring the class doesn’t have any unintended side effects.
package org.example;
class Counter {
private int count = 0;
public void increment() {
count++;
}
public void decrement() {
count--;
}
public int getCount() {
return count;
}
}
package org.example;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
class CounterTest {
@Test
void testIncrement() {
Counter counter = new Counter();
counter.increment();
assertEquals(1, counter.getCount());
}
@Test
void testDecrement() {
Counter counter = new Counter();
counter.decrement();
assertEquals(-1, counter.getCount());
}
}
Now the tests can be run in any order and the tests will not interfere with each other or other tests.
What is test driven development (TDD) for Java?
Test driven development (TDD) is where unit tests are written one by one, and then the code that satisfies that single test is written. Not every developer or project uses TDD, but TDD can help identify dependencies, and plan interfaces and interactions in an incremental and well-tested way. Here is a quick summary to start trying out test driven development:
- Use JUnit and Mockito to write a single simple unit test for specific functionality. It should start simple, such as the constructor of the class or a call to a new method in an existing class. The classes and methods may not exist yet, but assume that they do. This helps define the interactions, classes, and method calls needed.
- Run the test. It should fail initially.
- Write just enough code so that the new unit test passes.
- Check for regressions by running the whole test suite.
- Take the time to refactor the code, improving the code quality, interfaces, and design.
- Repeat with a new test case that further expands the functionality to build the feature.
Test driven development is a disciplined art. While not for everyone, its existence further demonstrates that good code and good tests are interlinked.
How to maintain Java unit tests
Unit tests need to be maintained to stay relevant and useful, which does take time and sometimes may feel like a burden. Remember, tests are your friends and worth the effort; they are there for the benefit of the future of the codebase and you or any developers who work on it. If this is not the current mentality in your team, here are a few things you can think about to keep tests in good shape and working for you.
Run the tests with every code change. If the tests fail, either the code needs to be fixed, or the test, before the code is merged into the codebase. This might sound obvious, but any test suite is useless unless it is working and passing on the codebase.
Keep tests up to date with knowledge. Tests help codify the understanding of the codebase. Tests are best written when you understand the most about the code, usually at the time of writing the code, or at the time of modifying existing code or fixing a bug.
Treat test code like source code. Ensure that linters and format checks are run on tests too. Use good variable names and modularity, just as you should in source code.
Document the purpose of each test case. Sometimes what a test is testing is obvious, but other times it is not. Make sure that future developers understand what the test is doing and why these cases were chosen, just in case the test fails in the future.
Annoying tests should be fixed. Tests that fail randomly, are hard to understand, or take a long time to run should be fixed. Just like you refactor source code, you can refactor test code.
Why and when should unit tests run?
There are several reasons to write unit tests, but an important reason is to prevent regressions in the future, meaning avoiding making a change or fix that inadvertently breaks another part of the code base. Here are some key times to run unit tests:
Run them individually locally on your machine while you are writing code. You can run relevant unit tests on classes you are modifying to ensure you are not causing regressions or test failures as you develop. An IDE is a fast and efficient way to run individual tests.
Run them as part of your build process. Tests can and should be run through your build commands, for example, mvn install also runs mvn test. A benefit of unit tests is that they are usually fast enough to do so without slowing down the build too much, and if there are test failures, the software won’t build.
Run them with your code integrated into the code base. In a modern software environment, developers usually work on features concurrently, and the code base that is on your machine may be out of date with the current state of the software, especially for features that take a long time to build. Tests must be run with your code integrated into the current code base. The current industry best practice is to handle this with a Continuous Integration (CI) system, such as Github Actions, Gitlab, or Jenkins.
What is Continuous Integration (CI)?
Continuous Integration (CI) is the DevOps practice of integrating features or small chunks of code into a codebase often, as they become ready. Typically in CI, before the new code is integrated, the software is built and its tests are run on the code integrated into the current code base. If the build and tests pass, the developer would be allowed to put the feature up for QA or integrate the changes, but if the build or tests fail, the developer would not be allowed to integrate their code.
Setting up CI for your project is out of the scope of this unit test guide, but if your project is not running CI, it might be worth checking out the Getting Started guides for GitHub Actions, Gitlab, or Jenkins. A CI workflow does not need to be complicated, it can be as simple as running your build command that builds the software and runs the tests, e.g. mvn clean install or gradle clean build; however, setting CI up may require some DevOps experience and permissions.
How many unit tests should a codebase have?
This is a common question for new unit testers. There are many schools of thought about the answer, but the rule of thumb is “enough tests to ensure that serious regressions are not introduced as the code is modified.” So how is this measured?
For simple code, enumerating all of the cases is straightforward, but for more complex code, it can be daunting. A coverage tool, such as JaCoCo or the one provided in your IDE, can tell you approximately how well the tests you have written test the code.
What is code coverage?
Code coverage is a metric for measuring the goodness of a test suite. For example, if the class under test has 100 lines of code, the coverage might be 60 lines, meaning that when the tests are executed, 60 of 100 lines of your code are executed. If there is a regression in one of those 60 lines, the tests will likely fail. The other 40 lines of code are not executed and any regressions there will never be detected.
It’s not a perfect metric, but it is simple to measure and understand. High code coverage is an indication of sufficient tests, and low coverage is an indication of insufficient tests. High code coverage alone does not mean that the tests are good or will catch bugs; the tests may suffer from other flaws, such as not being consistent from run-to-run and failing randomly or having missing assertions, but still produce good coverage. However, low code coverage is usually a good indicator of insufficient tests, because it means your test suite simply does not execute the code.
It makes sense to measure the code coverage to help identify what code is untested, but not as a measure of the goodness of tests.
How to use JaCoCo
The easiest way to run your tests and see the coverage they give is by running the tests “with coverage” through your IDE, for example, IntelliJ or Eclipse. The code coverage tool JaCoCo can also be run as part of your build process by adding JaCoCo to Maven or adding JaCoCo to Gradle. This JaCoCo tutorial will add JaCoCo to Maven, but the conclusions are the same for any method you choose.
Add the following plugins, the JaCoCo Maven plugin and Maven Surefire:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.5.0</version>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.12</version>
<executions>
<execution>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>report</id>
<phase>prepare-package</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>This JaCoCo plugin defines two goals:
- Instrument the classes being tested
- Generate code coverage report.
The prepare-agent goal happens with mvn test, and the report goal requires running mvn jacoco:report afterward.
After running both commands, open the report from target/site/jacoco/index.html. You can drill down into the package and the classes to see the coverage, down to individual lines.
You can use the report and the line colors to help you find where to improve your tests and rerun the coverage report. For example, most of the classes are covered, but the report shows that UserRepository is not well-tested by the tests because of the mocking. Untested lines show up in red.
How to build a good Java unit test suite
The perfect test suite is one where if the test suite passes, there are no bugs in the software, and if it fails, it is easy to identify the failures, and nothing in between. This might sound simple, but it is a lofty goal. Given the time pressures that developers are often under, the question to ask is “What is good enough?”
What is a good enough unit test suite?
The quick and dirty answer and generally industry-accepted benchmark is that 80% or more code coverage from unit tests is good enough. Component, integration, and end-to-end tests will also form an important part of any test suite, which would add to the coverage too.
For a more considered answer, the next time you are writing code, ask these questions about your test suite:
- Does it help you ensure your code is correct?
- Does it help you catch regressions?
- Does it aid in releasing versions quickly and without errors?
- Does it prevent you from manually testing code changes?
If you’re answering yes to these questions, you’re on the right track. Keep it up! Good test suites need maintenance, so keep writing and updating tests. If you’re answering no to some or all of these questions, your test suite needs some work.
What to do if the unit test suite is not good enough?
Test suites are built incrementally, and they are best built alongside the code they test as part of Java unit testing best practices. If you’ve discovered your test suite is not good enough, at a minimum, write good unit tests for the code you are working on now, following the concepts covered in this guide. Similarly, the next time there is a regression or failure, write a test that demonstrates the failure, and proves that it is fixed after the fix is made. The time you will understand the most about the bug is when fixing the bug, so take the opportunity to codify your understanding in tests. You can use a coverage tool like JaCoCo to help identify gaps and get busy writing tests for the gaps if you can spend the time doing so.
Where to start if…
Very few developers are so lucky to start writing code from scratch. Most development work interacts with or maintains existing code. What should you do if your test suite is not good enough?
There are no tests. If there are no tests at all and you are afraid of breaking the software, you might be best off initially writing a few end-to-end or component tests. These kinds of tests cannot usually tell you where something went wrong, and are not comprehensive, but might tell you if something went wrong. They will provide you with a small safety net while you get busy refactoring code and writing unit tests. This is a great first step. From here, you can chip away at writing unit tests a little bit each day, or write tests for different parts of the codebase you touch.
The code is not unit-testable. If your code is not unit-testable and you’re convinced that you need unit tests, then you have some work ahead of you. To write unit tests, you will have to refactor the code as well. If you find yourself in this situation, it is best to incrementally refactor and test code as you touch it. It can take some time to gain the knowledge of how the code works and fits, and the best time to refactor it is when you have the most knowledge from actively working on it.
There is not good code coverage. When you spend considerable time trying to understand a method, class, or module, it is a good opportunity to write tests. Take the time to codify your new understanding in tests. It is easy to forget how code works as time goes on, but the tests will stay and well-written tests describe the code’s behavior. This benefits future developers too. If these habits are repeated for every piece of code you and your team touch, over time, the test suite will improve.
Java test automation tools
If you want to kickstart your testing writing process, then take a look at Diffblue Cover. It can write a suite of tests much faster than human test-writing efforts alone. It can also be integrated into CI to ensure new features are tested automatically with its AI unit testing tools for Java. The tests from Diffblue Cover may not represent all of your business logic but can give a good starting point. As Martin Fowler says: “Imperfect tests are better than perfect tests that are never written.”
