Tuesday, January 18, 2011

How to write testable code

This post is my attempt to summarize the high level basics of writing testable code.  My approach is heavily influenced by the work of Misko Hevery and is focused on creating loosely-coupled components using Dependency Injection (but not necessarily using a DI container).  This post describes how to structure your code to use Dependency Injection effectively.  If you keep these guidelines in mind, you’ll reduce your testing pain and have a nice, loosely-coupled code base.

Note that these guidelines mainly apply to static languages and mildly dynamic ones (such as, say, PHP).  Dependency management is also an important topic for highly dynamic languages, but different practices may be appropriate in those cases.

Writing testable code basically comes down to three guidelines:

Guideline 1: Categorize classes according to how clients should obtain instances of the class
Guideline 2: Follow the injection rules for each class category
Guideline 3: Structure your services according to the Single Responsibility Principle

Let’s examine each in turn:

Guideline 1: Categorize classes according to how clients should obtain instances of the class

There are two broad categories of classes for testability purposes:  newables and injectables.  A newable is a class that can just be instantiated (e.g. with the ‘new’ operator) as needed in code.  When a client (such as a test) needs an instance, it just ‘new’s the instance with the appropriate values.

An injectable is a class that should not be instantiated as needed in the code.  That is, instances should not be created with the new operator on an as-needed basis.  Instead, an injectable should be injected (typically via constructor injection) in its client [hence the name].  In other words, an injectable will be supplied to its client (usually as a constructor parameter) rather than having the client being responsible for constructing or locating this dependency.

However, these two base categories are too coarse to use as the only categories when creating a system, so I find it useful to classify objects into the pattern categories popularized by Domain Driven Design and then map these DDD patterns to the newable and injectable categories.  These patterns are:

Value Objects:  A Value Object is an immutable object whose responsibility is mainly holding state but may have some behavior.  A Value Object has no conceptual identity separate from its attributes (but this feature is not relevant for the purpose of testability).  Example of Value Objects might be Color and Temperature.  Value Objects are newables.  A client just uses the new operator to create on an as-needed basis.

Entities:  An Entity’s job is also mainly holding state and associated behavior.  Entities differ from Value Objects in that an Entity does have an identity separate from its contents (but again for testability this feature isn’t relevant).  Examples of Entities could be Account or User.  Entities are newables.  Just create them as needed in tests or production code (where the 'new's would typically be done in a Repository, which are very briefly discussed below).

For the purposes of testability, Value Objects and Entities are identical.  They are both newables.

Services: A Service performs an operation.  It encapsulates an activity but has no encapsulated state (that is, it is stateless).  Examples of Services could include a parser, an authenticator and a validator.  Services are injectables.

DDD also allows for the Factory and Repository class categories, but for the test purposes of testability, they behave just like Services, so I don’t discuss them separately in order to try to keep things simpler.

Guideline 2: Follow the injection rules for each class category

Each class category has rules for whether injectables are allowed to be injected or not.  Failure to follow those injection rules is one reason why tests can become ugly and why needless complexity can be added to systems.

As Misko Hevery states, a newable can only ask for other newables (or primitives) in its constructor, while an injectable should only ask for other injectables in its constructor.

A corollary to this rule is of course that newables should not be injected with injectables.  Why should newables not be injected with injectables? Because it will lead to pain for the client of the newable. And tests of course are clients, so it will lead to testing pain also.  Why?  Because injecting an injectable creates a dependency for the newable.  And if the newable has a dependency, then it can no longer be tested in isolation easily.  Every time a client wants to create this newable, it’s going to have to create this dependency also.  So every test that uses an instance of this newable is going to have to either create a mock for the injectable or use the real injectable collaborator.  In other words, the setup for any test involving this newable is now more complex.  The complexity can spiral out of control quickly since dependencies are transitive.  Instead of just being able to easily ‘new’ these collaborators,  the tests now have extra complexity because the rule to not inject into newables was ignored.

It’s not only tests that are more complicated but also production code.  Because it’s likely that a Factory will be needed to create the newable since the Service that it requires must be injected (i.e. the Service can’t be instantiated at the point of use).  Thus the newable is no longer really newable (i.e. able to be easily instantiated with the new operator).  This is needless additional complexity for the system.  Do yourself a favor and don’t inject Services into newables.

To state this in terms of our DDD categories,  Value Objects and Entities should never be injected with Services.  Only Services should be injected with other Services.

As noted above, injectables should only ask for other injectables in their constructors.  In other words, injectables should not be injected with newables.  However, there is one case where this rule can be broken without causing testing pain.  An injectable can be injected with a newable or a primitive value when the injectable uses the value as a configuration constant.  So, for example, you may have a Service that needs one URL when in production and different URLs for other environments (such as staging, beta, dev).  Then it is perfectly acceptable to inject this Service with a string instance or string literal for the URL as long as the Service treats this value as a constant.  In other words, the value serves to configure the Service at creation and can not be changed or substituted after that.  Such injection of a newable/primitive into a service will not cause any testability problems, so it is the exception to the rule.

Guideline 3: Structure your services according to the Single Responsibility Principle

This guideline is really just a way of emphasizing the Single Responsibility Principle when creating the Services in your system.  This guideline isn’t necessary for those already practicing good OOP, but I believe that it deserves special emphasis for Service design for two reasons.  First, a lot of testing pain results from ignoring it.  Second, in my experience, there is sometimes a great deal of resistance to the SRP and the idea of breaking a responsibility out into its own class is sometimes treated as heresy to commonly-practiced OO.  So I think that special emphasis of its importance is warranted when discussion writing testable code.

I informally categorize Services into two categories when I am developing:

Task Services: Task Services have a single functional responsibility.  They perform a single task such as, for example, validation, authentication, or parsing.

Controller Services:  Note that ‘controller’ isn’t used here in the sense of, say, an MVC controller but is used in the more general sense of the ‘Controller’ object role stereotype of Responsibility Driven Design.  The only responsibility of a Controller Service is to control and coordinate other Services.  Controller Services build a higher level of abstraction composed of other Services.  In GOOS language, Controller Services create a composite that is simpler than the sum of its parts.

Note that in Responsibility Driven Development, the object role stereotype of what I call a Task Service is ‘Service Provider’.  But since ‘service’ is overloaded between RDD and DDD, and ‘Service Provider Service’ isn’t very clear or helpful, I’m sticking with Task Service.  The important part isn’t the name but is instead to separate out functional responsibilities from the control and coordination responsibilities.

So why is it important to distinguish between Task Services and Controller Services when writing testable code?  Because mixing the responsibility of performing an operation with the responsibility of controlling and coordinating operations causes testing pain and complex test setups.

By isolating these responsibilities, each operation can be tested separately from the control of the operations.  If an operation is tested along with the control, then the operation will have to be tested in combination with other operations.  That is when ugly test setups happen.  Controller Services are easy to test because all they do is call other Services and have some conditional logic (such as for error values).  The Controller Services contain little functionality because they’re just rolling up other Services that contain the functionality.  So mock creation and use is easy in Controller Service unit tests.

Summary of testability for each category

Let’s look at each of our categories from a testing perspective:

Value Objects and Entities: Easy to use and easy to test.  Just ‘new’ as needed with the needed values.  Value Objects are particularly easy to test since they are immutable.

Task Services:  Easy to test because they’re stateless and their functionality is for a single behavior.

Controller Services: Easy to test because they are stateless and contain little functionality since they’re just aggregating other Services.

So each every class category is relatively easy to test in isolation.

Building a system out of these testable components gives you a great head start on creating a composable architecture while helping to reduce unnecessary testing friction.

In summary, when I have experienced testing pain personally or have seen it elsewhere, it is in general because of one or more of the following:

- a lack of use of Dependency Injection
- a misunderstanding of or misuse of Dependency Injection
- not separating classes cleanly into newables and injectables
- injection of injectables into newables
- inappropriate injection of newables into injectables
- disregard for the Single Responsibility Principle

So that’s my whirlwind tour of writing testable code.  I hope that I’ve helped to explain how some design mistakes can lead to ugly tests and testing pain and how you can produce loosely-coupled components with less testing friction.  If you're just beginning on the path to testability, I recommend these Misko Hevery videos as a great starting point for learning more about writing testable code:

The Clean Code Talks: Unit Testing
The Clean Code Talks: Don’t Look For Things

41 comments:

  1. Hmm...

    To state this in terms of our DDD categories, Value Objects and Entities should never be injected with Services.

    Ok. I have a problem with this. (Not saying it's wrong, just trying to figure it out.) I have a Game entity that needs a ScoreEvaluator service. Right now, I'm injecting it; if I read you correctly, you're saying I should just new() it instead.

    Won't this hide the dependency? Wasn't this the point of Misko's posts - don't hide the dependencies? My Game entity still needs the ScoreEvaluator service, it's just that I've hidden that.

    ReplyDelete
  2. Hi Marcel,

    Thanks for being the first commenter on my blog. :)

    I wasn't trying to imply that an entity should directly new() a service. As you mention, that is undesirable because it's a testability killer.

    There are two options for how to handle this situation and still be testable. Both are equivalent in terms of testability, but one or the other might want more sense when modeling a specific domain:

    The first option is to pass the service as a method parameter to every entity method that requires it. So, for example, there might be a Game::score(IScoreEvaluator) method. This allows the score evaluation behavior to reside in the entity but avoids the testability problems that stem from injection of the service and from an inline new() of the service.

    The second option is to completely separate the service from the entity. In this case you would remove all score evaluation behavior from the entity. So your model for evaluation in the case would be a ScoreEvaluator service that scores Games rather than a Game that can score itself with the aid of a service (as in first option).

    Whichever option makes the most sense depends somewhat on the particulars of the domain.

    ReplyDelete
  3. Thanks for replying so quickly.

    So... why would the first option be better than injecting the service in the constructor? (I don't think the second option is valid in my case, as the service is scoring game *moves*, not the game as a whole... and I really want to keep track of both moves and their scores from the game object.)

    ReplyDelete
  4. Injecting a service into an entity is worse from a testability perspective because it forces creators of the entity to have an extra, unneeded dependency on the service.

    In other words, every component that needs to new() the entity now also has to create or obtain a service. Injecting the service into the entity forces the creator of the entity to have a dependency on a class that it never interacts with directly [that is, this is counter to the guidance of the Law of Demeter]. The system is more coupled than needed because of the unnecessary dependency.

    This applies to tests also, so that tests which involve an entity now either have to mock the service or have to use the real implementation. So test setup for tests involving the entity is also more complicated. It's no longer as easy to test the entity in isolation.

    If the service is passed as a method parameter, the entity remains a newable, and clients and tests can more easily instantiate the entity. There's no extra dependency at creation time.

    The only tests that have to worry about mocking the service are test for the methods that have the service as a parameter. And the mocking needs are simple in these cases. Thus it's easier to test the entity in isolation.

    I encourage you to read this post discussing this issue in more detail:

    http://misko.hevery.com/2008/09/30/to-new-or-not-to-new/

    ReplyDelete
  5. Weirdly enough, I had read that article before... but for some reason I missed that rule (don't require services from new-ables). Thanks :)

    ReplyDelete
  6. Excellent and well laid out post on some very important posts. There was a time when I had to search out the features of the two types of object ( newable and injectable ) I was very lucky because I had some very good mentors to explain ( or ridicule my silly questions to the point where the answer was considered the only logical way of looking at it ). Your post has made this distinction and the reasons for it very clear.
    In my testing ( and I came here by way of your comment on derik baileys post ) my pain, and it is considerable, comes from both an abuse of SRP and not really understanding the two different types of service ( task and controller ). Those may be the same thing though. anyway, this is one of those moments which people often have where they think, oh shit, I have to go rewrite all the code I've ever written ( in my case specifically my services which are over loaded ).
    I will conclude this post with a question. Do you not consider the Controller class in an MVC design as a controller service? you inject lots of crap into it and you shouldn't really do a whole lot of logic in it you should just delegate and return.
    thanks again,
    Raif

    ReplyDelete
  7. @Raif - thanks for the kind words about the post. A lightbulb went off in my head also when I encountered the newable/injectable concept (via Misko Hevery's blog), and it has been a highly useful lens through which to see when applying dependency injection.

    You're right, the task service and controller service distinction is really just a restatement of the SRP in a particular and more narrow context. But I have found that many have difficulty applying the SRP - it's just too counter to common OO practice. And even the idea of services themselves unattached to a 'noun' seems very difficult for some to swallow. So I find it helpful to call it out as a separate guideline for services (even though simply applying the SRP is sufficient).

    Yes, I would consider an MVC controller to be a controller service. While it may be doing a little more than controlling only services (for example, it of course could possibly interact directly with a view), it fits the bill in that its object role stereotype in RDD is definitely the Controller one. So it's possible that in some cases MVC controllers may not be 'pure' service controllers, but when looking at a system thought these lens, that's where they fit best. I was mainly trying to avoid confusion having readers think that I was *only* referring to MVC controllers with the 'controller service' concept.

    ReplyDelete
  8. I don't quite agree that Newables should be instantiated as needed since this would create a magic value. I understand that some DI frameworks cannot inject some kinds of classes - but this seems very awkward to me. For a longer version see: Is Newables/Injectables a fundamental distinction?

    ReplyDelete
  9. @zby: I had been trying to summarize a non-trivial subject and keep it short, so I didn't mention the case where it is acceptable to inject newables or primitive values into injectables. I think this case may address some of your concerns. I have gone ahead and added a paragraph in Guideline 2 to explain this case, as I don't want it to be a point of confusion. In summary, when a newable or a primitive is injected into an injectable and is used by the injectable as a configuration constant, it will not cause any testability problems. An example of such a value might be the port number mentioned in your post.

    Regarding your point that some DI frameworks can not inject some kinds of classes: it's not about what DI frameworks can inject but more about what is unnatural for them to inject. In other words, it's not about the capabilities of DI frameworks but about design. For example, if I need to inject, say, a Money newable, how would a framework know what value to create it with? 10.00? 3.87? 72,492.00? It doesn't know because the value relates to the use case and not to the application construction. Any attempt to inject it is going to create complications (like, for example, more dependencies such as unnecessary factories) compared to just instantiating the newable (a Value Object in this case).

    ReplyDelete
  10. Hmm - I think you made typo in that added paragraph.

    OK - so you agree that it is OK to inject newables as long as they are a kind of config like a port number to listen.

    Let's go to the second paragraph of your comment above. You say that the frameworks can inject Value Objects but they don't make it easy as a design choice - why is that? Why they discourage injecting for example a port number (that you agreed to be a good idea)? I don't really understand your example with Money - how is it different from a Port object?

    ReplyDelete
  11. Dependency injection is largely about separating your application construction (at startup) from application logic. So things that should be injected are the things that are known at startup (which are mostly domain and infrastructure services). A port number could be injected into one of those services as a configuration constant at startup appropriately since the value would be known when you construct your object graph.

    On the other hand, a Money value would most likely be need as a result of some user interaction. Perhaps calculating a price, displaying a value, converting a currency, entering a sell price, etc. Such an application logic value doesn't have anything to do with the application construction and can't be known at startup. DI separates application logic from application construction, and a Money value in these cases belongs firmly on the application logic side.

    The port value is used as a configuration constant at app startup, while, in the scenarios discussed, a Money object has to do with application logic and not application construction. How could a Money object be created at app startup for these scenarios? It can't, because it's not a configuration constant; it's part of the business logic and must be created at runtime.

    I imagine that most frameworks accomodate injecting a number for a port just fine. But the problems with, for example, the Money object doesn't really have anything to do with DI frameworks. Manual DI is no different. The difficulty is due to the attempt to put something on one side of the separation (application construction) that belongs on the other side (application logic) and is not caused by a DI framework. That's why I called it a design issue rather than a framework issue.

    Btw, I didn't spot a typo. Perhaps my brain auto-corrected for it as I read the paragraph :)

    ReplyDelete
  12. OK - so we agree that some Newables (those available at startup time) are Injectables. Some are not - but are all non-Newables always available at startup? I think this is not necessary - and this means that being a Newable have no relation to being an Injectable.

    Having just one factory that wires everything at app startup seems very constraining to me - even Hevery advices to use one factory per scope and in a web engine have, at least, one app level factory and one request level factory. The request level factory would inject stuff that is only available when the request is made. This sounds very logical for me - although I have hard time imagining the code doing this, I guess I'd need to write something to really feel how this can work.

    ReplyDelete
  13. I don't think of that case as some newables being injectables. Instead I think of it as a configuration constant being injected into an injectable or as an injectable taking a configuration constant as a constructor parameter. Although the configuration constant may have the same form as a newable (say, a string instance), it's not a newable in the sense of 'something that can be instantiated on-demand by clients'. Think of this case as an injectable being configured at creation with a constant rather than as a case of a newable being injected. The configuration constant and a newable-at-runtime may have the same form, but they have different purposes. Don't let the similarity of form mask the difference in the intent of usage. Thinking of the configuration constant as a special case will help to not let it cloud your understanding of the newable/injectable distinction.

    Your app factory can create the factories for the other scopes.

    Writing something will definitely help with understanding these concepts. It will probably take a few iterations for things to click. It did for me, at least. If multiple scopes throw you for a loop, you may want to start with a case where only a single scope is needed like a desktop app or PHP.

    ReplyDelete
  14. So what you say is that it is how a class is used that makes it a Newable not the class itself? What if you use the same class in different roles? Say a Date class - do you require that it is then split into two subclasses and the code that uses a date object as configuration used a different class then code that creates a date from user input?

    What if this Date (say the connection date/time) was saved into the Request Scope factory and then injected by that factory into the Request Scope objects? The factory creates it - so it is a Newable for it - but the Request Scope objects are injected with it.

    ReplyDelete
  15. I'm saying that, for this special case of configuring a service with a constant, although the newable is being *injected*, it is not an *injectable*. The injectable category refers to classes that should not be directly instantiated, while the newable category refers to classes that are intended to be instantiated at will. Even though the newable is being injected in this special case, that is not its general intended use. Even though it has a use case that is an exception to the general rule of not injecting newables, the general proper use of the newable is still for clients to directly instantiate it in the application logic as needed, so it is still best thought of as a newable rather than an injectable or as some sort of a newable/injectable hybrid. The special case is the exception and not the rule and does not change the newable into an injectable.

    ReplyDelete
  16. Care to show some concrete example of a newable that should not be injected? I like to have the rule to inject and the exception to not inject - not the way around :)

    If you create something and then use it extensively and pass it around a lot then either it can be created at the time the original object is created and then it should be injected independently of the type of that something, or you don't yet have all data needed to create it at that time. In that second case your object is spanning two scopes and the part dealing with this created data is not much object oriented and chances are that it would be better split into two (or more).

    So about the only exception from DI I see is when your newable is just a temporary value on the stack.

    ReplyDelete
  17. @zby - I didn't really follow much of that, so I'll just offer a few general comments...

    Re: injecting newables - I don't think of newables as being injected. As far as I'm concerned, I don't inject newables. Instead, I sometimes configure services with a value that may have the same type as a newable. It's important when writing testable code to keep the newable/injectable distinction clear. Thinking in terms of injecting a newable [even when that is what is happening] is only going to cloud understanding of that distinction imo (at least when starting to learn the technique].

    About your comment regarding a case of not having enough data to create an object: this is often a symptom of trying to inject a newable. For example, a service class (ill-advisedly) takes a newable parameter in its constructor but the newable value is not known at startup. Now because of that design decision, the service can't just be created at startup. Instead, a service factory is now required in order to create the service at runtime at some point after the newable value is available. So now all clients of this service, rather than just being injected with an instance of their direct collaborator [the service] now have to be injected with an indirect collaborator [the service factory] in order to obtain the collaborator that they really need [the service]. The code and design is needlessly more complicated because of the design decision to try to inject a non-configuration-constant newable.

    A better choice in this case would be to not inject a newable into the service but instead to create the service at startup without the newable and then:

    1) pass the newable as a (non-constructor) method parameter to the service or
    2) pass the service as a (non-constructor) method parameter to the newable

    Neither of those options creates any testability problems.

    ReplyDelete
  18. If your objects are created only with data that is available at application startup - then you code much of your project in procedural style (Managing Object Lifetimes). Sure there is always a trade-off between simplicity here or simplicity there - but I think a general rule would be to split object into parts operating on data in a specific scope.

    ReplyDelete
  19. @zby - I'm afraid that you've lost me. Creating a service at startup (or at the start any of the needed scopes) doesn't make anything procedural. Creating an authentication service or an Order Repository at startup has nothing to do with procedural programming. Initializing one of these services with, say, an URL does not make it procedural.

    But, yes, you do need a factory for each scope, and on that point we agree.

    ReplyDelete
  20. "Stateless service-classes are stateless. They are injectable, not newable ..." If I read you right?

    How about this?

    String s = new StreamReader().read(stream);
    DomainObj o = new DomainObjParser().parse(s);

    My service-classes StreamReader and DomainObjParser are stateless. I don't want to see them in the signature of my constructors nor do I want to see them anywhere.

    This is boilerplate, redundant and confusing:
    DomainObjWorker(stream, streamReader, modelObjParser)

    I want this:
    DomainObjWorker(stream)

    Providing both constructors, the full and the simple one, is also not an option. I can't stand having member variables of these two silly service-objects. They are stateless, why are they state of my class?

    The full constructor is useful if there is a need for switching service implementations .. or when one of the services is a singleton .. or other reasons I can't imagine right now.

    But it shouldn't be there for testing purposes only. Otherwise it's like programming for possible future requirements.

    If it is not an actual requirement leave it out.
    And if it is nowhere used, beside tests, take it out.

    ReplyDelete
  21. @Lars - how about

    DomainObjWorker( DomainObj o )

    ?

    @loose OK - if you accept the factory per scope rule - then we are at the same page. The argument about the procedurality of the code when you have only one factory (that is when create *only* at app startup) is in the article I linked to:
    http://misko.hevery.com/2009/04/15/managing-object-lifetimes/ . I understand that you agree with that

    So having covered this now let's assume we have a two factory web engine. Then the request scope factory will create objects like requestDate - that are value objects or even primitive types - and then pass them into the creator method of objects like Request or Handler or whatever. Don't you agree with this design?

    After reading your answer above I started to think that maybe you agree with the design - but when you pass a Value object (or primitive value) to the creator then you call it initialization and not injection.

    ReplyDelete
  22. This comment has been removed by the author.

    ReplyDelete
  23. A Handler might actually be a better example - because you could say that a Request is a Newable - but a Handler is not. It needs all the infrastructure, like a database, to serve the request. "Managing object lifetimes" is a good argument about why Handlers should be in Request scope.

    ReplyDelete
  24. The "rough spot" seem to be newables-that-could-be-injected-as-well: , I usually like - if the language supports that behaviour - to use an optional default value; this allows injections when the caller wants to override the default behaviour, leaving the class open for extension but closed for modification, and also making it clear what's an implementation details and what's a peer of that class - something that would risk to get obfuscated otherwise.

    ReplyDelete
  25. @Lars - It's unclear to me how you intended DomainObjWorker to be used. I'm not sure if it is:

    - a factory that produces DomainObj instances or
    - a domain object itself (such as what I would call an Entity) or
    - an object that handles a use case (such as a controller)

    Please let me know what you had intended as the responsibility for DomainObjWorker, and then we'll be able to discuss the testability of the design more specifically.

    ReplyDelete
  26. This whole theory is based on the *assumption* that you cannot test a class in isolation from any "newable" dependencies.

    The assumption is false, though. In Java (and probably all other modern programming languages) you *can* test any component in isolation, regardless of how it obtains instances for its dependencies. This is done through the use of suitable mocking/isolation tools. Examples: TypeMock (C#), Mocha (Ruby), Python Mock, PowerMock (Java), ...

    ReplyDelete
  27. @Alan - Good point. That can be a helpful approach for something like, say, a timeout value. When there are multiple default values though, I prefer setter injection. Multiple default values in a constructor can get messy in many languages when the value you want to override isn't the first optional parameter.

    However, when the value in question is an actual collaborator such as a service (rather than a newable), I don't care for a 'default implementation' approach because in my mind it clouds the separation of wiring and app logic that DI is trying to achieve in the first place. I don't believe that you were suggesting that approach at all though.

    ReplyDelete
  28. @Rogerio - Personally, I don't want to rely on a tool to provide the loose coupling for my system. I prefer to build the loose coupling into the system itself. However, I can see how in theory something like TypeMock perhaps might be useful as a sort of bridge to a better design when dealing with existing code that was not written with testability in mind (though I don't think that it would be my first approach). I do use mocking tools, but I prefer to use them on top of a loosely-coupled code base.

    ReplyDelete
  29. @zby - I think that you're on the right track. It's difficult to talk about multiple scopes [or if they are even needed] separate from the environment or framework. A lot of it depends on the environment (such as what - if anything - is kept around in memory between requests) and the specifics of the framework.

    ReplyDelete
  30. I am wondering how the blog author *defines* the concept of "loose coupling".

    For example, take the case of a class A which directly instantiates and uses a class B through B's public interface. Would this be considered *tight* coupling? AFAICT, conventional wisdom (from well known experts and book authors) is that it is a perfectly fine form of *loose* coupling.

    ReplyDelete
  31. As far as a definition of loose coupling, there's one in the header for the blog :-) I intended it as an explanation of the name of the blog rather than as a rigorous definition, but it works well enough.

    When classifying couplings between a client and a dependency, I think in terms of Misko Hevery's newable and injectable distinction. So my answer would depend on the dependency (that is, on class B). If B is an injectable, then I would classify it as a tight coupling; otherwise not.

    ReplyDelete
  32. I see. Classifying the coupling between a class A which instantiates an "injectable" dependency B as *tight* is problematic, however. According to the the original proponent of this concept, Larry Constantine, this would actually be considered the weakest form of coupling, "data coupling" (unless, of course, the criteria for a stronger form of coupling is met). In a more modern formulation (http://cc2e.com/File.ashx?cid=336, pages 100-102) of the concept, we have "simple-object coupling": "A module is simple-object coupled to an object if it instantiates
    that object. This kind of coupling is fine."

    To me, the coupling between A and B (whether B is injectable or newable) is loose if the implementation behind the public interface of B can be changed without requiring changes to A.

    ReplyDelete
  33. While I find those definitions of coupling interesting, I don't find them helpful for the purposes of writing testable code (which is my interest, at least in this post). For example, the latter definition would allow untestable behavior (such as use of global variables) to reside in B and still meet the requirement. Mutable global state could be used in B and could be changed in B without requiring a change to A, but it would still be untestable because of the mutable global state. So it may be necessary, but I don't find it sufficient for the purposes of testability. What's behind the curtain of B can affect the testability of A, and I find the newable/injectable distinction to be a practical way to distinguish these cases.

    One of the reasons that I like the newable/injectable distinction (and their injection guidelines) is that they provide a largely prescriptive set of guidelines that in my opinion lead to at least a good head start on testable code and a composable system when followed.

    ReplyDelete
  34. Well, I see those definitions of coupling as *the* definitions of coupling. I mean, we cannot intend to rewrite history here.

    I don't see why having mutable global state (or any other implementation detail) in B should affect the *unit*-testability of A. Obviously, it would make it harder to write *integration* tests that exercise both A and B at the same time (just as it would be harder to unit test A if *it* made use of global mutable state). If I fully isolate A from the implementation of B, on the other hand, I can easily write unit tests for A; and such isolation can be achieved purely in test code, rather than in a (necessarily more complex) combination of test and production code.

    ReplyDelete
  35. @Ronaldo da Silva - You've essentially got it right. I think that you're essentially asking how to put together newables and injectables to form a system. That is why I included the Domain Driven Design patterns (Value Objects, Entities, Services, Repositories, etc) in the post. Misko Hevery doesn't speak in terms of DDD, but I find the concepts useful in describing at a higher level how to make a system testable, and the patterns map remarkably well to newables and injectables in my opinion. So I think that it may help you to (also) think in terms of these concepts rather than just purely in terms of newables and injectables when you are trying to put the pieces together.

    For example, Entities [which fall in the newable category] will typically be instantiated within a Repository [an injectable]. A Repository provides the illusion of an in-memory collection of instances of an Entity. So you might have, for example, an OrdersRepository whose methods perform queries and 'new' Order instances using data gathered in the queries. So that is an example of an injectable/Repository (OrdersRepository) creating a newable/Entity (Order).

    The Order object might have some attributes which are Value Objects (which fall into the newable category). So you might have, for example, a CreditCard Value Object. So the Repository would need to 'new' a CreditCard to use in composing an Order object. That would be another example of an injectable (OrdersRepository) instantiating a newable/ValueObject (CreditCard).

    Another common example of an injectable creating a newable is that of a controller creating a Value Object. For example, you might receive user input in the form of an integer to represent a monetary amount. But in order to work in the language of the domain instead of in terms of primitives, the controller might 'new' a Money instance from the integer value and then use that when interacting with the model.

    Value Objects also often instantiate other instances of themselves. Since they are immutable, they need to provide new instances of themselves when a change in the value is needed (rather than modifying the existing instance). For example, a Money class might have an add(Money) method which returns a new Money instance containing the sum of the two Money instances. That demonstrates a newable instantiating a newable.

    Does that help to answer your questions?

    ReplyDelete
  36. Hi, here is a 'Newable entity' puzzle I cannot solve: How do you hook up an entity with an Observer if you are not allowed to pass the observer (which is an injectable service) to the constructor of the entity?

    My specific case is lazyloading (but I think the same applies to any other Observer required by an entity). I need my entities to transparently lazyload their properties from db (i.e. load the properties of an entity only when the client touches any getter or setter of the entity). The actual process of lazyloading is not the entites' responsibility, so it is encapsulated in a Lazyloader class. An entity thus needs to store a reference to the Lazyloader object, because the entity needs to notify the Lazyloader when a client touches any accessor method.

    Problems: a) the entity is not a newable any more (it needs the Lazyloader object in its constructor), b) we are breaking some ground rules here because we are keeping a reference to an injectable inside a newable. The question is, how do you keep the entity as a newable and enable it to interact with a service at the same time?

    As for the two obvious solutions: 1) I don't want a client to pass the Lazyloader object with every getter and setter call, 2) I don't want a client to first pass the entity to a Lazyloader before calling any setter or getter. Both these solutions would be way too tedious for the client (and usafe) and the lazyloading would not be transparent.

    I'd like to keep a clear distinction between my newables and injectables as you and Misko suggest (I've experienced way too much testing pain) but this problem with the lazyloader has been a show stopper for me so far. Any insight is greatly appreciated.

    ReplyDelete
  37. Hi David,

    I believe that what you need is the Repository pattern (as described by Fowler and later given prominence as a building block of Eric Evan's Domain Driven Design). A Repository is responsible for the persistence of an entity. The API that it provides to clients makes it seem as if provides an in-memory collection of entities. That is, it is a persistence-ignorant API. So, for example, you might have a UserRepository (or you might prefer to call it something like AllUsers) which might have, for example, a method to get a User by ID.

    The Repository is an injectable. The entities are still newables, but these instantiations of the entities take place inside of the Repository instead of directly being done by clients. And the entities can of course easily be new'd in tests when they are needed as collaborators.

    It appears that currently your entities have multiple responsibilities (both Information Holder [in RDD language] and persistence), and I think that this is the core of your design/testability dilemma. By separating the persistence responsibility into a Repository, I believe that you'll find it much easier to have a clean, testable design. This is a good practice (in most languages, at least) with or without a need for lazy-loading. If desired, the Repository can provide the lazy-loading capability by having it return proxies for entities rather than the entities themselves.

    ReplyDelete
  38. Hi LC,
    thanks for pointing me in the right direction! The Proxy pattern used for lazy loading seems to be the thing I am looking for. Can you recommend any tutorials/blogs on this topic? (I'd rather not reinvent the wheel...)

    ReplyDelete
  39. David -

    The pattern that you're looking for is more specifically called Virtual Proxy. This entry on Martin Fowler's POEAA site should be good enough as a starting point.

    ReplyDelete
  40. @LC - I've ordered PoEAA (I've been pondering this book for some time and now I have run out of excuses :) ).
    The Virtual Proxy is the thing, no doubt. It will make my entity a plain newable, which is what I am after.

    BUT:
    1) What is the proxy object then? Since it is a direct descendant of my entity, it is everything my entity is, thus it should be a newable (note that it has state), but at the same time it must be a service, ie injectable. Haven't we just pushed our problem one level down in the inheritance tree?

    2) How is the Virtual Proxy object supposed to be unit tested?

    (Yours and Misko's postings have been a real eye opener for me. Thanks for your good work!)

    ReplyDelete
  41. The virtual proxy doesn't necessarily have to derive from the entity. You could (in many languages) use an interface which the entity and the virtual proxy (independently) implement.

    In any case, you're right that the virtual proxy essentially has the same problem - it has multiple responsibilities (creation of the proxied instance and providing state). But that's the nature of the Virtual Proxy pattern - by definition, it has multiple responsibilities. But shifting the responsibility to the virtual proxy and the repository also serves to isolate the additional complexity to those classes (rather than all collaborators of the entities having to deal with the complexity). As you note, the entities will then be just straight newables, and their collaborators won't experience any unnecessary testing pain. Only the repository will have to have knowledge of the virtual proxy.

    The virtual proxy pattern isn't 100% pure in terms of the newable/injectable categories, but we need to be flexible and pragmatic also. The key of course when venturing outside of (any) guidelines is to know understand both what is being lost and what is being gained and to then decide if the tradeoff is worth it. To me, it seems like a very reasonable tradeoff when lazy-loading is a requirement.

    As far as unit testing the virtual proxy, using mocks is your best bet. In other words, you can verify that the calls to create the proxied instances are being made correctly and at the correct times and also verify that the proper delegation calls are being made.

    ReplyDelete