JUnit Best Practices

This page explains best practices related to JUnit. The overall best practices are to aim for unit tests that are:

• extremely fast - You're going to have a lot of unit tests and they are going to run frequently so they have to be lightning fast. I'm not talking a few seconds fast; rather, I mean milliseconds fast!
• extremely reliable - You want tests to indicate problems with production code. Tests should fail if and only if the production code is broken. Some have argued that tests that intermittently fail are worse than no tests at all!

Ensure that unit tests run completely in-memory.

For example, do not write unit tests that make HTTP requests, access a database, or read from the filesystem. These activities are too slow and too unreliable, so they are better left to other types of tests, such as functional tests.

Tests that read from the filesystem are too complicated to be considered unit tests:

• They need to configure the location of the current working directory and that can be complicated to get right on developer machines and build machines.
• They often require storing the file to be read in source control and it can be complicated to keep the item in source control up-to-date.

Do not skip unit tests.

There are a number of ways to skip unit tests, but you shouldn't use any of them:

• Do not use JUnit's @Ignore annotation.
• Do not use Maven's maven.test.skip property.
• Do not use the Maven Surefire Plugin's skipTests property.
• Do not use the Maven Surefire Plugin's excludes property.

Unit tests that are skipped provide no benefit, but still have to be checked out of source control and compiled. Instead of skipping unit tests, remove them from source control.

Aim for each unit test method to perform exactly one assertion.

There are two reasons to aim for each unit test to have exactly one assertion:

• When a unit test fails, it is much easier to determine what went wrong. If a failed unit test has three assertions, further effort is required to determine which one failed. If a failed unit test has only one assertion, no such effort is required.
• When a unit test performs more than one assertion, it is not guaranteed that all of the assertions occur. In particular, if an unchecked exception occurs, the assertions after the exception do not happen; instead, JUnit marks the unit test method as having an error and proceeds to the next test method.

Therefore, strive for one assertion per test method.

Use the strongest assertions possible.

Without strong assertions, unit tests provide nothing more than coverage. In other words, unit tests that lack strong assertions ensure that the production code does not throw an exception but when is the lack of an exception sufficient to determine that a method behaves correctly? Coverage is a worthy outcome of unit tests but we can do much better! In particular, we want our unit tests to ensure that our production code works correctly. Without strong assertions, our unit tests only ensure that our production code doesn't blow up in our face.

In order of decreasing strength, assertions fall into the falling categories:

• strongest assertions - asserting on the return value of a method
• strong assertions - verifying that vital dependent mock objects were interacted with correctly
• weak assertions - verifying that non-vital dependent mock objects (such as a logger) were interacted with correctly
• non-existent assertions

Do not verify how a mock logger was interacted with unless logging is critical to the correctness of the method being tested.

Consider writing unit tests for the following production method:

How should one test this method? A naive approach might be to test the method as follows:

However, this test is too brittle because it fails if the production code is changed in a non-functional way. For example, the test fails if the production code is changed so that log statements occur at a different level (say info instead of debug) or with a different message (say "Returning default value for 3" without the colon). You can eliminate the latter problem using a less strict assertion (verify(logger).debug(anyString())) but the test is still too brittle. You're better off not verifying any logger behaviour unless logging is critical to the correctness of a production method.

Use the most appropriate assertion methods.

JUnit has many assertion methods - assertEquals, assertTrue, assertFalse, assertNull, assertNotNull, assertArrayEquals, assertSame. Know the assertions in the latest version of JUnit and use the most appropriate one to have the most readable test code. For example:

• Use assertTrue(classUnderTest.methodUnderTest()) rather than assertEquals(true, classUnderTest.methodUnderTest()).
• Use assertEquals(expectedReturnValue, classUnderTest.methodUnderTest()) rather than assertTrue(classUnderTest.methodUnderTest().equals(expectedReturnValue))
• Use assertEquals(expectedCollection, classUnderTest.getCollection()) rather than asserting on the collection's size and each of the collection's members.

Put assertion parameters in the proper order.

The parameters to JUnit's assertions are:

1. expected
2. actual

For example, use assertEquals(expected, actual) rather than assertEquals(actual, expected). Ordering the parameters correctly ensures that JUnit's messages are accurate.

Use exact matching when using a mocking framework.

For example, avoid using Mockito's any* methods; prefer configuring and verifying exact parameter values.

Name unit tests using a convention that includes the method and condition being tested.

On my projects, we have used a naming convention of methodUnderTest_condition, where methodUnderTest is the method under test and condition is the condition being tested. For example, suppose we are writing unit tests for a method named encode that accepts an array of bytes and validates the bytes before encoding them. When combined with the best practice above (one assertion per test), we end up with test methods named as follows:

• encode_nullBytes
• encode_emptyBytes
• encode_tooFewBytes
• encode_tooManyBytes
• encode_rightNumBytes or encode_validBytes

Ensure that test classes exist in the same Java package as the production class under test.

When in the same package as the production class being tested, test classes can use package-private classes and invoke package-private methods. It has been my experience that high-quality code bases make extensive use of package-private classes and methods to hide implementation details.

Ensure that test code is separated from production code.

The default folder structure in a Maven project does this - production code exists in the src/main/java folder and test code exists in the src/test/java folder. Even if you don't use Maven, put test code and production code in different folders.

Do not print anything out in unit tests.

Unit tests are intended for consumption by the computer.  Developers may add print statements to their private working copy for debugging purposes, but there is typically no need to submit a unit test that prints. For a specific best practice about not printing stack traces, see below.

Do not initialize in a unit test class constructor; use an @Before method instead.

If a failure occurs during a test class constructor, an AssertionFailedError occurs and the stack trace is not very informative; in particular, the stack trace does not include the original error's location. On the other hand, if a failure occurs during an @Before method, all details about the failure's location are available. See this article for details.

Some teams allow test class members to be initialized in a constructor when those members are of a simple type such as String, but I personally don't ever initialize test class members. Instead, I always do one of the following:

• Assign values to test class members in a method annotated with @Before (preferred).
• Assign simple values to private static final members in the test class.

Note that this best practice also applies to members initialized upon definition. In other words, rather than this:

do this:

Do not use static members in a test class.

Static members make unit test methods dependent. Don't use them! Instead, strive to write test methods that are completely independent.

Do not write your own catch blocks that exist only to fail a test.

It is unnecessary to write your own catch blocks that exist only to fail a test because the JUnit framework takes care of the situation for you. For example, suppose you are writing unit tests for the following method:

Here we have a method that accepts an integer and returns an integer and throws an IOException if it encounters an error. Here is the wrong way to write a unit test that confirms that the method returns three when passed seven:

The method under test specifies that it can throw IOException, which is a checked exception. Therefore, the unit test won't compile unless you catch the exception or declare that the test method can propagate the exception. The second alternative is preferred because it results in shorter and more focused tests:

The JUnit framework will make sure that this test fails if any exception occurs during the invocation of the method under test - there's no need to write your own exception handling.

Do not write your own catch blocks that exist only to pass a test.

It is unnecessary to write your own catch blocks that exist only to pass a test because the JUnit framework takes care of the situation for you. For example, suppose you are writing unit tests for the following method:

Here we have a method that accepts an integer and returns an integer and throws an IOException if it encounters an error. Further suppose that we expect the method to throw an IOException if it is passed a parameter with the value nine. Here is the wrong way to write a unit test that confirms that the method behaves that way:

Instead of manually catching the expected exception, use the expected attribute on JUnit's @Test annotation.

The JUnit framework will make sure that this test passes if and only if the foo method throws an IOException - there's no need to write your own exception handling.

Do not write your own catch blocks that exist only to print a stack trace.

As we've already seen, it is a best practice for unit tests not to write anything. After all, unit tests are written to be consumed by the JUnit framework and JUnit doesn't care or monitor what gets printed. For example, suppose you are writing unit tests for the following method:

Here we have a method that accepts an integer and returns an integer and throws an IOException if it encounters an error. Here is the wrong way to write a unit test that confirms that the method returns three when passed seven:

The method under test specifies that it can throw IOException, which is a checked exception. Therefore, the unit test won't compile unless you catch the exception or declare that the test method can propagate the exception. The second alternative is much preferred because it results in shorter and more focused tests:

The JUnit framework will make sure that this test fails if any exception occurs during the invocation of the method under test - there's no need to write your own exception handling.

In test classes, do not declare that methods throw any particular type of exception.

Test methods that declare that they throw one particular type of exception are brittle because they must be changed whenever the method under test changes. For example, suppose you are writing unit tests for the following method:

Here we have a method that accepts an integer and returns an integer and throws an IOException if it encounters an error. Here is the wrong way to write a unit test that confirms that the method returns three when passed seven:

This test method is brittle. Imagine the method under test changed such that it could throw either an IOException or a GeneralSecurityException. In that case, we would have to change the test method for it to compile:

Instead, declare that the test method can throw any exception:

That way, the unit test is no longer brittle - it is impervious to changes to the production method's signature.

Do not use Thread.sleep in unit tests.

When a unit test uses Thread.sleep it does not reliably indicate a problem in the production code. For example, such a test can fail because it is run on a machine that is slower than usual. Aim for unit tests that fail if and only if the production code is broken.

Rather than using Thread.sleep in a unit test, refactor the production code to allow the injection of a mock object that can simulate the success or failure of the potentially long-running operation that must normally be waited for.

Do not attempt to test the timing of production methods that directly or indirectly call Thread.sleep.

Explanation to come

Test classes directly; do not rely on indirect testing.

Indirect testing occurs when a class is tested only because it is a dependency of another class. For example, suppose you have the following:

Here, class B is a dependency of class A.

We should test class B directly even though its methods are also invoked by the tests of class A. That means that if we name our test classes using a suffix of Test, we should have two test classes - ATest and BTest.

It's a bad idea to rely on indirect testing because we lose coverage of the dependent class if we change the implementation of the main class. In the example above, suppose we changed the implementation of class A so that it no longer depended on class B. The tests in ATest no longer invoke the methods of class B so we lose code coverage.