Sunday, January 16, 2011

What is testability?

What is testability?  My definition is:
Testability is the quality that allows a component to be easily tested in isolation.
There are two important parts to the above definition.  First, the component must be easily testable in isolation.  If something is able to be tested in isolation, but (for example) the test setup is difficult due to, say, excessive test setup or difficult mocking, then it's not testable.  That would be indicative of a testability problem such as high coupling and/or low cohesion.

The second important part is that the component must be testable in isolation. Testable has to mean something beyond 'able to be tested'.  By that definition, there's nothing that's not testable, since code can always be driven from the user interface or from a higher-level API.  If the definition is only 'able to be tested', then the concept loses all meaning.  So the ability to be tested in isolation is an important part of the definition.

In practice however, I use the term testability as an umbrella term for a group of interrelated qualities - loose coupling, high cohesion, maintainability, modularity, good design, readability, and (of course) the ability to easily test components in isolation.  Using testability as a stand-in for all of these qualities is not entirely accurate, but I find it useful.  

What makes testability suited for this purpose?  Because it's the closest thing that we have to an objective test for these qualities.  To a certain extent, something is either easy to test in isolation or not.  It's not a black-or-white.  But since testability is strongly correlated with all of the above qualities, it's the best that we have for a litmus test for the above qualities.

Strictly speaking, testability is really a side-effect of modularity.  The 'in isolation' part of my testability definition speaks to modularity (as does the 'easily' part, actually).  But I find it difficult to speak of modularity in practical and concrete terms.  However, given a chunk of code, I can easily discuss specifics about what makes it simple or difficult to test.  

So I tend to use the term testability as a general proxy for overall quality.  It is of course not strictly accurate, but it is very practical. 

4 comments:

  1. Does not *easily* imply *in isolation*?

    :)

    ReplyDelete
  2. I'm not sure I would entirely agree with that.

    It could be possible to test a unit in isolation, but for the test to require, say, a lot of mocks and difficult setup. In other words, the unit doesn't for example instantiate services inline, so it doesn't prevent substitution (and thus does not prevent testing in isolation). But it makes testing difficult because (likely) the class is doing too much [low cohesion].

    The isolation and the ease are definitely correlated though. But I've found that in discussing testability problems that a little extra emphasis doesn't hurt and is often helpful :)

    ReplyDelete
  3. "It could be possible to test a unit in isolation ... But it makes testing difficult because (likely) the class is doing too much [low cohesion]." - this is refuting the other implication - *in isolation* -> *easy* - and I agree of course :) My thesis is that if something cannot be tested in isolation then it is not easy to test.

    But yeah - extra emphasis - this is a good argument.

    ReplyDelete
  4. Good point, agreed. But I also think that if a component can be tested in isolation, then it is not necessarily easy to test. The isolation is necessary but is not always sufficient.

    The main places that I've seen this be a problem (in my own code and others) is in a class with many responsibilities. The collaborators are substitutable, but it's a pain to test because it's doing too much (i.e. not following the Single Responsibility Principle). Refactoring out a class for each responsibility usually does the trick.

    ReplyDelete