Monday, February 7, 2011

Non-DI code == spaghetti code?

A previous post describing how Dependency Injection differs from using a Dependency Injection container (largely in the context of static languages) contained the following table:

No DI ContainerDI Container
No DISpaghetti CodeService Locator Anti-Pattern
DIManual Dependency InjectionEnhanced Dependency Injection

Some took exception with the characterization of non-DI code as spaghetti code. My intent was not to be derogatory but was to succinctly describe the increased complexity of the control flow and the increased coupling of dependencies that occurs without dependency injection (due to the mixing of creation and logic concerns).  This post examines the use of this label and also contains an invitation to suggest an alternate metaphor for describing the structure of non-DI code in the context of an explanation of dependency injection.

Wikipedia describes spaghetti code as "a pejorative term for source code that has a complex and tangled control structure."  I hadn't intended to be uncomplimentary, but I do assert that the mixing of concerns caused by the lack of DI does indeed create a more complex and tangled flow.  Without dependency injection:

- creation concerns are mixed with the control flow (that is, with the application logic)
- object lifetime concerns are mixed with the control flow

Because creation instructions unrelated to the application logic are mixed in with the control flow, the control flow is (in my opinion) more complex and tangled without DI than with DI.  However, 'more complex and tangled than X' doesn't necessarily mean 'complex and tangled'.  So at what point does the mixing of concerns or additional complexity move from acceptable to spaghetti code?  Is the increased complexity in flow without DI enough by itself to merit term spaghetti code?  How many responsibilities can a unit of code have before turning into spaghetti?   How tight can couplings become before they are too tight?  I don't think that these questions can be answered outside of an individual's judgement within a particular context.  For my context in the original post of a high level comparison of the structural differences between DI code and non-DI code, I found the metaphor illustrative.

I also consider it more likely that without DI there will be increased flow complexity and dependency complexity due to reliance on global state (such as GoF-style singletons) and service locators.  Without a context-independent dependency management strategy like DI provides, I feel that these problematic dependency management strategies are more likely to be used (and hence mixed in with the logic).  This is of course not a given, however.

Beyond control flow concerns, I find it illustrative to use a spaghetti code metaphor in relation to object graphs.  When the dominant paradigm was procedural, it made sense to define spaghetti code strictly in terms of control structure.  But do people really write (for example) many goto statements these days?  Now that object-oriented programming is the dominant paradigm, high coupling between components seems to be a more common problem than, say, unstructured branching.  Of course, complexity in OO control flow could be considered a side-effect of high-coupling between objects since the objects in a system express the control structure.  Use of a spaghetti code metaphor is entirely appropriate in my opinion for describing high, unnecessary coupling among objects.

Is it possible to write good code without DI?  Of course.  People have been doing that for a long time and will continue to do so.  Might it be worth making a design decision to accept the increased complexity of not using DI in order to maximize a different design consideration?  Absolutely.  Design is all about tradeoffs.  But the point is that this should be a conscious decision informed by an understanding of what is being lost and what is being gained.

It is of course possible to write spaghetti code with DI too.  I did not mention this in the original post because it was about contrasting properly-applied DI with other alternatives and was not about describing poorly-applied DI.  My impression is that improperly-applied DI leads to worse spaghetti code than non-DI code.  It's essential to understand guidelines for injection in order to avoid creating additional dependencies.  Misapplied DI seems to involve more problems than not using DI at all.

I suspect that much of the antipathy for dependency injection stems from experience with poorly-applied DI.  Someone might start on a project where DI has been misapplied or where there has been 'container abuse' and then conclude that DI is nothing but hype and that its practitioners are a cargo-cult.  It would be understandable in that situation to not distinguish between the baby and the bath water, but it would be a mistake to ignore a valuable tool because of the result of its misapplication.  If someone attempts to use a lawnmower to paint a house, should they then conclude that lawnmowers have no value after viewing the result?

In any case, my purpose in using the term 'spaghetti code' was to illustrate the structural differences between a non-DI approach and a DI approach for those who may not yet understand the difference.  I think the metaphor does work in this context, but the term is indeed pejorative.  If you have a metaphor that illuminates this difference without being uncomplimentary, please leave a comment.  I'm interested in suggestions.

Tuesday, January 18, 2011

How to write testable code

This post is my attempt to summarize the high level basics of writing testable code.  My approach is heavily influenced by the work of Misko Hevery and is focused on creating loosely-coupled components using Dependency Injection (but not necessarily using a DI container).  This post describes how to structure your code to use Dependency Injection effectively.  If you keep these guidelines in mind, you’ll reduce your testing pain and have a nice, loosely-coupled code base.

Note that these guidelines mainly apply to static languages and mildly dynamic ones (such as, say, PHP).  Dependency management is also an important topic for highly dynamic languages, but different practices may be appropriate in those cases.

Writing testable code basically comes down to three guidelines:

Guideline 1: Categorize classes according to how clients should obtain instances of the class
Guideline 2: Follow the injection rules for each class category
Guideline 3: Structure your services according to the Single Responsibility Principle

Let’s examine each in turn:

Guideline 1: Categorize classes according to how clients should obtain instances of the class

There are two broad categories of classes for testability purposes:  newables and injectables.  A newable is a class that can just be instantiated (e.g. with the ‘new’ operator) as needed in code.  When a client (such as a test) needs an instance, it just ‘new’s the instance with the appropriate values.

An injectable is a class that should not be instantiated as needed in the code.  That is, instances should not be created with the new operator on an as-needed basis.  Instead, an injectable should be injected (typically via constructor injection) in its client [hence the name].  In other words, an injectable will be supplied to its client (usually as a constructor parameter) rather than having the client being responsible for constructing or locating this dependency.

However, these two base categories are too coarse to use as the only categories when creating a system, so I find it useful to classify objects into the pattern categories popularized by Domain Driven Design and then map these DDD patterns to the newable and injectable categories.  These patterns are:

Value Objects:  A Value Object is an immutable object whose responsibility is mainly holding state but may have some behavior.  A Value Object has no conceptual identity separate from its attributes (but this feature is not relevant for the purpose of testability).  Example of Value Objects might be Color and Temperature.  Value Objects are newables.  A client just uses the new operator to create on an as-needed basis.

Entities:  An Entity’s job is also mainly holding state and associated behavior.  Entities differ from Value Objects in that an Entity does have an identity separate from its contents (but again for testability this feature isn’t relevant).  Examples of Entities could be Account or User.  Entities are newables.  Just create them as needed in tests or production code (where the 'new's would typically be done in a Repository, which are very briefly discussed below).

For the purposes of testability, Value Objects and Entities are identical.  They are both newables.

Services: A Service performs an operation.  It encapsulates an activity but has no encapsulated state (that is, it is stateless).  Examples of Services could include a parser, an authenticator and a validator.  Services are injectables.

DDD also allows for the Factory and Repository class categories, but for the test purposes of testability, they behave just like Services, so I don’t discuss them separately in order to try to keep things simpler.

Guideline 2: Follow the injection rules for each class category

Each class category has rules for whether injectables are allowed to be injected or not.  Failure to follow those injection rules is one reason why tests can become ugly and why needless complexity can be added to systems.

As Misko Hevery states, a newable can only ask for other newables (or primitives) in its constructor, while an injectable should only ask for other injectables in its constructor.

A corollary to this rule is of course that newables should not be injected with injectables.  Why should newables not be injected with injectables? Because it will lead to pain for the client of the newable. And tests of course are clients, so it will lead to testing pain also.  Why?  Because injecting an injectable creates a dependency for the newable.  And if the newable has a dependency, then it can no longer be tested in isolation easily.  Every time a client wants to create this newable, it’s going to have to create this dependency also.  So every test that uses an instance of this newable is going to have to either create a mock for the injectable or use the real injectable collaborator.  In other words, the setup for any test involving this newable is now more complex.  The complexity can spiral out of control quickly since dependencies are transitive.  Instead of just being able to easily ‘new’ these collaborators,  the tests now have extra complexity because the rule to not inject into newables was ignored.

It’s not only tests that are more complicated but also production code.  Because it’s likely that a Factory will be needed to create the newable since the Service that it requires must be injected (i.e. the Service can’t be instantiated at the point of use).  Thus the newable is no longer really newable (i.e. able to be easily instantiated with the new operator).  This is needless additional complexity for the system.  Do yourself a favor and don’t inject Services into newables.

To state this in terms of our DDD categories,  Value Objects and Entities should never be injected with Services.  Only Services should be injected with other Services.

As noted above, injectables should only ask for other injectables in their constructors.  In other words, injectables should not be injected with newables.  However, there is one case where this rule can be broken without causing testing pain.  An injectable can be injected with a newable or a primitive value when the injectable uses the value as a configuration constant.  So, for example, you may have a Service that needs one URL when in production and different URLs for other environments (such as staging, beta, dev).  Then it is perfectly acceptable to inject this Service with a string instance or string literal for the URL as long as the Service treats this value as a constant.  In other words, the value serves to configure the Service at creation and can not be changed or substituted after that.  Such injection of a newable/primitive into a service will not cause any testability problems, so it is the exception to the rule.

Guideline 3: Structure your services according to the Single Responsibility Principle

This guideline is really just a way of emphasizing the Single Responsibility Principle when creating the Services in your system.  This guideline isn’t necessary for those already practicing good OOP, but I believe that it deserves special emphasis for Service design for two reasons.  First, a lot of testing pain results from ignoring it.  Second, in my experience, there is sometimes a great deal of resistance to the SRP and the idea of breaking a responsibility out into its own class is sometimes treated as heresy to commonly-practiced OO.  So I think that special emphasis of its importance is warranted when discussion writing testable code.

I informally categorize Services into two categories when I am developing:

Task Services: Task Services have a single functional responsibility.  They perform a single task such as, for example, validation, authentication, or parsing.

Controller Services:  Note that ‘controller’ isn’t used here in the sense of, say, an MVC controller but is used in the more general sense of the ‘Controller’ object role stereotype of Responsibility Driven Design.  The only responsibility of a Controller Service is to control and coordinate other Services.  Controller Services build a higher level of abstraction composed of other Services.  In GOOS language, Controller Services create a composite that is simpler than the sum of its parts.

Note that in Responsibility Driven Development, the object role stereotype of what I call a Task Service is ‘Service Provider’.  But since ‘service’ is overloaded between RDD and DDD, and ‘Service Provider Service’ isn’t very clear or helpful, I’m sticking with Task Service.  The important part isn’t the name but is instead to separate out functional responsibilities from the control and coordination responsibilities.

So why is it important to distinguish between Task Services and Controller Services when writing testable code?  Because mixing the responsibility of performing an operation with the responsibility of controlling and coordinating operations causes testing pain and complex test setups.

By isolating these responsibilities, each operation can be tested separately from the control of the operations.  If an operation is tested along with the control, then the operation will have to be tested in combination with other operations.  That is when ugly test setups happen.  Controller Services are easy to test because all they do is call other Services and have some conditional logic (such as for error values).  The Controller Services contain little functionality because they’re just rolling up other Services that contain the functionality.  So mock creation and use is easy in Controller Service unit tests.

Summary of testability for each category

Let’s look at each of our categories from a testing perspective:

Value Objects and Entities: Easy to use and easy to test.  Just ‘new’ as needed with the needed values.  Value Objects are particularly easy to test since they are immutable.

Task Services:  Easy to test because they’re stateless and their functionality is for a single behavior.

Controller Services: Easy to test because they are stateless and contain little functionality since they’re just aggregating other Services.

So each every class category is relatively easy to test in isolation.

Building a system out of these testable components gives you a great head start on creating a composable architecture while helping to reduce unnecessary testing friction.

In summary, when I have experienced testing pain personally or have seen it elsewhere, it is in general because of one or more of the following:

- a lack of use of Dependency Injection
- a misunderstanding of or misuse of Dependency Injection
- not separating classes cleanly into newables and injectables
- injection of injectables into newables
- inappropriate injection of newables into injectables
- disregard for the Single Responsibility Principle

So that’s my whirlwind tour of writing testable code.  I hope that I’ve helped to explain how some design mistakes can lead to ugly tests and testing pain and how you can produce loosely-coupled components with less testing friction.  If you're just beginning on the path to testability, I recommend these Misko Hevery videos as a great starting point for learning more about writing testable code:

The Clean Code Talks: Unit Testing
The Clean Code Talks: Don’t Look For Things

Monday, January 17, 2011

The relationship between Testability and Test-Driven Development

What is the relationship between Test-Driven Development and testability?  Is doing TDD sufficient to guarantee testable code?

In short, no. TDD is neither necessary nor sufficient for testability. You can write testable code without doing TDD, and doing TDD doesn't guarantee testable (that is, easy to test) code. Doing TDD without a clear understanding of how to write testable code results in unnecessary testing pain and doesn't yield the expected benefits.

TDD does help to produce more testable code by creating warning signs (that is, testing pain) when there are testability problems with the code. But TDD is a means to an end and is not an end in itself. Testability is the end. TDD helps to achieve the end, but it doesn't guarantee it.

Nat Pryce summed the relationship between TDD and testability perfectly in a tweet today about TDD and design:
TDD does not drive towards a good design, it drives away from a bad design. If you know what good design is, the result is a better design.
In the same way, TDD drives away from a lack of testability by creating pain for testability problems.  If you approach TDD with an understanding of how to write testable code, your testing path will be much smoother with more effective results. So how does one write testable code? That's the subject of the next post in which I will give testability guidelines and discuss how failure to follow the guidelines creates testing pain and friction.