5 Unit Testing Mistakes

When I first heard about unit testing using a framework like JUnit, I thought it was such a simple and powerful concept. Instead of ad hoc testing, you save your tests, and they can be run as often as you like. In my mind, the concept didn’t leave much room for misunderstanding. However, over the years I have seen several ways of using unit tests that I think are more or less wrong. Here are 5, in order of importance:

1. Testing algorithms together with coordinators. Algorithmic logic is easiest to test if it is separated from coordination code (see Selective Unit Testing – Costs and Benefits). Otherwise you end up with tests where you for example first have to submit a job through a job queue before the logic is tested. The job queue part only complicates things. Unless you are testing the job queue itself, break out the logic that would be executed when calling the run method, and test that logic separately. Both the code and the tests become much easier to write and manage that way.

2. Mocking too much. Perhaps the greatest benefit of unit tests is that they force you to write code that can be tested in isolation. In other words, your code becomes modular. When you mock the whole world around your objects, there is nothing that forces you to separate the parts. You end up with code where you can’t create anything in isolation – it is all tangled together. From a recent tweet by Bill Wake:  “It’s ironic – the more powerful the mocking framework, the less pressure you feel to improve your design.”

3. Not using asserts. Sometimes I see tests where an object is created, some methods are called, and that’s it. Maybe it is done in a loop, with some variation in creation or calling. However, nothing is ever checked using asserts. That misses the whole point – checking that the code behaves as expected. Sure, the code is run, but that’s it. If an exception is thrown, we would notice, but nothing else is verified.

4. Leaving print statements in the tests. I see this as a remnant from manual testing – you look at the values and decide if they are correct or not. But all checking should be done using asserts. If an assert fails, you will see it, because the test fails. When the test passes, nothing should be printed. Sometimes when developing the tests, it can be useful with print statements. But in that case add a flag, and turn printing off when checking in the tests.

5. Checking the log statements, not the result. Thankfully not common, but I have seen an otherwise very competent developer do this. Since it is the result of the method that matters, not what is printed in log, there can be errors in the code, and the tests still pass. Enough said.

The last 3 problems are all easy to avoid. The first 2 require more effort, but will result in code that is nicely separated. Happy unit testing!

9 responses to “5 Unit Testing Mistakes

  1. Mistakes 1 and 2 are good. A third common mistake is creating brittle tests that test too much algorithm implementation detail compared to behaviour of algorithms. It’s a hard balance to strike between adequately testing algorithms and creating sufficiently flexible tests. This is, at least, the mistake I most often make.

  2. i have seen unit tests that did not check anything because of 3. At my last place of work they were measuring test coverage, so most programmers were achieving 90%-100% test coverage without actually testing anything.

  3. Pingback: The Baeldung Weekly Review 8

  4. Hi Henrik, I think you might be interested in my (free) e-book devoted to worst/best practices of testing. You can find it at http://practicalunittesting.com – look for “Bad Tests, Good Tests”. Cheers!

  5. Pingback: Unit Testing Mistakes

  6. Pingback: Software Development Linkopedia March 2014 | Agile - ScrumXP

  7. Tomek, thanks for the interesting link

  8. Pingback: Bookmarks for March 10th | Chris's Digital Detritus

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s