Things I’ve learned from writing a lot of unit tests

The following are a few observations I’ve learned from writing unit tests over the past decade and a half that you might not get from books or other blogs.

You should only test through the public contract

The first is a biggie and a reason why a lot of developers who try unit testing say it doesn’t work in any practical sense and then give up on it entirely as a development practice.

When writing a test you must write the test to test the public contract of the code in question.

If you directly test through any of the non-public methods then you will tightly couple your test to the implementation details. Unit tests (or any automated test) should not be testing implementation details but instead test outcomes in certain scenarios.

“Tests should be coupled to the behavior of code and decoupled from the structure of code” — Kent Beck

When we write tests that test code through the public contract then we are testing the behaviour of our code from the perspective of a consumer of our code. If the behaviour of our code changes then our tests should also change. If the internal structure of the code changes then our tests shouldn’t need to change.

But I really need to test an internal method…

There might be rare times when you really do need to test some internal code. I can strongly advise that after writing the tests and determining that the code works how you expected that you should probably delete the tests. Otherwise you will be leaving the problem of highly coupled tests (coupled to the internal structure of your code) for the next developer.

Do not fall into the trap of just making something publicly accessible so you can write unit tests for it. Be honest with yourself, is a particular type, method, property etc. really part of the public contract? Are consumers of your code really going to call it or should it be only internally accessible?

It should be very rare that you use a mocking frameworks to test certain methods were called (x number of times, with certain arguments etc.). Again because this causes your test to become tightly coupled to the implementation details of your code. These kinds of tests are testing the internal structure of the code not the overall behaviour.

In fact I have found over the years the better I have got at unit testing and coding in general the less I use a mocking framework in the first place! This leads me to my next observation …

Not all dependencies need to be mocked/stubbed

You may have some code that takes two dependencies (for example for a class this might be through the constructor). When you come to write a unit test for the class think about whether you should pass in the real implementation for any of the dependencies or a mocked/stubbed version.

Because you are interested in testing behaviours and outcomes through a public contract it could very well make more sense to pass in the real implementation.

Just because your code takes a dependency don’t automatically think you have to mock/stub it for your unit tests.

You should end up with a lot more unhappy path tests than happy path tests

Consider this simple code example:

public char GetChar(string str, int index)
{
   // get the character at index position from string
}

In this example a happy path test could be I pass in an index that is within the bounds of the string and the character is returned.

A unhappy path test is potentially every other single test that results in us not getting a character back from within the string.

For example what happens:

• if str is null?
• if str is empty (“”)?
• if index is a minus number?
• if index is greater than the length of str?

etc.

When you have thought of all of the possible scenarios for a piece of code you will very often find that you have many more unhappy path tests than happy path.

Code coverage is only really useful for one thing

Code coverage is a tool that only developers (who are writing unit tests) should be using. It has one primary use: to help developers spot scenarios for tests that currently have not been covered.

The code coverage metric is not a measurement of the quality of tests or the quality of the production code. It should not be a metric that is shared outside the developers realm, especially to managers for fear that they will wrongly assume that it is a good measurement of quality or progress.

Teams should also not be in the habit of setting arbitrary coverage targets (e.g. “80% by June!”). This does nothing except incentivise developers to game the system and write potentially poor or even useless unit tests. The only sensible target for coverage is in fact 100%, but this is rarely a number that any sensible suite of tests can hope to achieve.

Tests need to be small

If a unit test has gone over 10 lines of code then it certainly can no longer be considered small. Large tests aren’t just a lot harder to understand for anyone trying to read them but can be harder to maintain.

Most unit tests written these days adhere to the “Arrange, Act, Assert” (AAA) pattern. So lets see where we can gain test size improvements in each of these three areas:

Arrange

• The arrange (AKA setup) part of the test is usually the main culprit in test size bloat.
• Arrange code should be extracted to other methods where sensibly possible.
• Extracted methods can then often be moved to new classes, away from the test class and so making the test class itself smaller.
• Try using simple factory or builder patterns to create required entities for your tests.

Act

• An Act is the point of execution of a piece of code, in a certain scenario (determined by the Arrange) which will then expect a certain outcome (the Assert). Your unit test should have only one Act.
• If your test has more than one Act then you are very likely performing more than one separate test but within the one test method. Extract code out to other new test methods until your test has only one Act.

Assert

• Theoretically using the “Arrange, Act, Assert” pattern your test should only really have one Assert.
• However, in some scenarios it can be more practical to have multiple Assert statements. For example if you need to Assert a number of properties on a returned result from the Act.
• Again like in the Arrange, if you find yourself writing this code more than once then extract the set of Assert statements to a new method and call that instead.

Have a test name standard

Whether you are writing code in a team or on your own it can be a good practice to be consistent and standardise your unit test names. A good unit test name will always describe a behaviour of the code (i.e. an outcome in a certain scenario).

For example a simple standard for defining a unit test might follow a template such as:

[Test]
public void When<SCENARIO>_Then<EXPECTED_OUTCOME>()
{
   // test body
}

This simple standard above helps the developer to think about what the scenario and expected outcome is before they even attempt to write the actual test body. It also helps other developers, upon reading just the method name, to quickly understand what the test is about.

Some developers like to use the words “Given”, “Should” etc. in their test template. Over the years I have settled on “When” and “Then” simply because they convey adequate meaning and are short (test method names often have a tendency to be quite long).

Also note the use of “When” and “Then” in this case should not be confused with their use by the Gherkin (“Given/When/Then”) DSL.