We’ve tried to summarize some potential misconceptions as simple equations. Having been spun out of a major Software Quality Management company, we have seen how many of Europe’s leading companies have at one point or another focused on the wrong things or laboured under certain misconceptions.
Unit Testing ≠ Quality
Writing lots of unit tests says little about actual quality. What matters is writing the right unit tests. Blindly unit testing everything frustrates developers, requires more maintenance and leads to shortcuts.
Black Box Testing = Quality + Happiness
Black Box Testing done well leads to better code quality, less time writing unnecessary unit tests, more time writing functional code and, as a result, improved developer happiness.
1 Unit Test Case ≠ Adequate
When time is tight, we often see only one Happy Path unit test.
This is rarely adequate as there are typically many paths, some of them critical, that should be covered by tests.
If only the Happy Path is covered by unit tests, this shifts the testing responsibility higher up the testing pyramid (shift right to Integration and End to End tests). The net effect is that errors end up getting through to the customer.
A requirements led, black box approach is often the best way to deliver great quality.
Code coverage ≠ Quality
A lot of companies we see focus heavily on code coverage. But this says nothing about whether the system requirements are being properly tested and does little to guarantee quality.
Read our article Obsessed with Code Coverage?
White Box Testing ≠ Requirements Met
White box testing is very important in certain situations. However, white box tests can usually only be implemented by a developer as the functional code needs to be fully understood first.
Black box testing is, in most cases, a better way to go as it’s fully requirements led. This also allows the developer to shift responsibility for test definitions to Domain Experts, POs and QA Engineers who anyway best understand the requirements. As a result, developers will spend more time writing functional code and less time writing unit tests.
TDD = Black Box Testing
We often see companies not making this connection clearly. If you are following TDD principles then you are, by definition, performing black box tests and are focusing on requirements rather than just coverage.
As black box testing approaches are important for validation of the code (does the implemented code meet the requirements?), TDD and black box testing are essentially synonymous.
Doing Everything ≠ Doing the right things
Some people prefer to test everything and create extensive test case sets for every function and path that has been programmed. This is often unnecessary. A risk-based analysis helps focus the effort where it makes sense.
It’s better to write great tests on the important parts of your code than to write bad tests everywhere.