FIT without fixtures

During the Agile 2008 conference, Mike Stockdale organised a mini-session where he and Rick Mugridge presented some new features and ideas that they are working on at the moment. The session led to a very interesting discussion on whether we could produce a variant of domain adapter/domain fixtures that allows FIT to connect directly to most domain services and objects without the need for any fixtures.

The case against inheritance

FIT and its architecture started back in 2002 and heavily relies on inheritance as a way to extend the framework and integrate business domain code into the framework. That makes it hard to maintain the framework, which Mike pointed out, as lots of people depend on inner workings of the Fixture class. (I myself have complained a few times about his refactorings of the .NET test runner that broke my existing code). It also limits the options we have to integrate domain code with the framework. I often write a lot of boilerplate code to wrap domain objects into fixtures, since the integration layer has to extend from the Fixture class. Some options like target objects, system under test and domain adapters make it easier to connect domain objects directly to fixtures, but they require us to write lots of boilerplate code and we still have to use fixtures. It is hard to incorporate some generic test management functionality, such as starting and rolling back transactions or invoking the debugger, without changing the test runner or again extending fixtures.

Rich domain models and fixtures

Both FIT.NET and Java FitLibrary have started to move towards using a rich domain model (as in DDD) more effectively than with traditional mapping of each step into a fixture or a Dofixture method. Domain fixtures and domain adapters in FIT.NET and Java FitLibrary “explain” how to utilise rich domain objects. What we’ve noticed on the mini-session is that these domain adapters effectively do two things:

1. explain how to create and access domain objects (for setup and verification tasks)
2. explain how to find domain objects and execute methods on business services (action tasks between setup and verification)

In addition to that, it is interesting that FIT.NET and Java FitLibrary now have a mechanism to define meta-data about a whole test suite (suite configuration file in .NET and the suite fixture in FitLibrary).

Moving forward in that direction, we felt that we could lift the requirement to implement fixtures for a large majority of cases if the underlying business code is developed using a rich domain model and other DDD principles.

Integration with repositories

Most projects based on a rich domain model will now effectively use a repository to store and find objects, so the first benefit could be achieved by implementing support for a few popular repository method naming conventions instead of encapsulating it into a custom domain adapter/fixture. The suite meta-data could be used to select the appropriate naming convention and define the appropriate repository objects for domain objects. This could be used to prepare domain objects for the test or verify changes in domain objects later. For example, the traditional setup fixture becomes obsolete and could be replaced simply by a repository call. For example, this is a common set-up table:

If there is a repository defined for the Customer class, this table in the setup part of the test would be executed by instantiating a Customer object for every row, populating FirstName, LastName and Phone and calling CustomerRepository.Save(Customer c) method.

A very important new idea is to store the result of this operation directly into a symbol; we discussed the option to mark a column that contains the appropriate symbol key with a * or similar, but concluded that it would be better to just use the first column as the key name. So the previous table would create three Customer objects and store them automatically into symbols “Mike Smith”, “Mike Scott” and “Tom Cruise”. So this would effectively replace even Column Fixtures that are used to extract the result and store it into a symbol. The rationale behind storing objects automatically into symbols is that the objects prepared in the setup are most likely required later in the test (why else would they be created?).

If there is no repository, objects would just be created by instantiating the domain class and storing it into the symbol. If there is a repository, this would result in an additional call to the repository and the result of the repository call would be stored into the symbol. The reason behind that is to allow the repository to further amend the object if required by business rules.

If the first cell in the first row is not a class name, then it could map to a service method name or a repository method name. The corresponding method would get called by mapping the parameters directly and the result (if not void) would again be stored in the symbol. Repositories and services would be configured in the suite meta-data, so there would be no requirement to implement additional code.

In the verification part of the test, the same table (if the first cell maps to a domain class name) would act as a column fixture that retrieves the symbol value based on the first column, and then compares the other fields to actual domain object values retrieved from the symbol. (Possibly by going to the repository again to find an object by id if there is a repository, to support test-specific stuff). If the first cell in the first row maps to a method name, the method would be executed and the results would be compared to the table. A third option would work on repository finders and would also test for list length (similar to a list or set fixture). For example

would call CustomerRepository.FindAll() and compare the results with the table looking for missing or surplus customers as well.

would call CustomerRepository.FindActive() and so on.

Test-specific functionality

The acceptance sometimes do not list all the properties, to make tests more focused. If there is no fixture in between, the creation of objects in the repository might fail because of that. A solution for this case is to implement a decorator over the normal repository that adds the test-specific functionality (eg populates 10 remaining mandatory fields), and then using the test repository instead of the default repository in the suite meta-data configuration. This would also be the way to implement any other test-specific functionality. There is no way to avoid implementing this test-specific code, but this model would not require test-specifics to inherit from Fixture or require writing any other boilerplate code.

Talking to business services

One of the most important practices that DDD introduced is the ubiquitous language, a common jargon shared across all phases and participants of a project. One of the best practices for acceptance testing arising from that is using the same phrases for the same concepts in examples, acceptance tests and code. With this in mind, we really should not need any translation layers between the fixture tables and the domain code (both in objects and services). They should use the same jargon so we should be just able to glue them together by implementing a few naming conventions.

DoFixture and SequenceFixture support the concept of system under test, mapping calls directly to domain services. However, using SequenceFixture makes the test too technical and not really suitable for discussion with business people. Although this is relatively understandable:

It is much more natural to write this example as

DoFixture allows us to write scripts like this, but then requires the system under test to implement silly method name such as UserSendsInvitationToUserForRoom. No self-respecting programmer would create such a name in a business service.

Guessing the operation

Working on a few examples from the FIT book, we discussed how this link could be implemented better, for example creating something similar to DoFixture but that would map to more sensible method names, which would be written in normal domain objects and services. This would promote the idea of ubiquitous language even further, because it would suggest the correct service method or domain object method name. The final idea was to try out a few keyword combinations and find what the method name is, similar to service fixture but not requiring it to be the first keyword.

This could theoretically map to three methods: ChatService.Enter(User u, Room r), Room.Enter(User u) or User.Enter(Room r). If we have a ChatService in the suite meta-data configuration, then the algorithm could try to map keywords to either the service methods or class methods and look for the appropriate call. A possible pitfall is that this could lead to ambiguity. To promote the use of natural language, we would apply some basic transformations (such as stripping the final s from Enters), and ignoring some words (such as “a” or “on”). So we could use “Book a flight”, not forcing people to write “book flight”. Then:

would map to FlightService.BookFlight(Location from, Location to). The list of ignored words would probably also be configured on the suite level with some defaults.

The sentences could be even shorter, since the setup part would already have stored Ana, LOTR, LAX and JFK in symbols and we could extract the correct type.

We also discussed automatic object tree traversal, for example “Mike’s credit card number” would map to (Symbol(‘mike’)).CreditCard.Number.

Checking in natural language

Instead of

we would use the “is” keyword:

this would work very similar to “check” at the moment, meaning it will execute everything on the left how ever it is marked up, and compare the result to the cell on the right, but it reads much more naturally.

Tables or no tables

An interesting discussion followed after this on whether we need tables at all, or should we just try to recognise keywords. My personal opinion is that tables are good because they clearly identify what is a test script and what is just a description on the page. In any case, a bit of CSS tweaking can make the cells invisible.

Extension points

We also discussed the importance of providing a number of extension points or hooks for acceptance test execution, that would enable us to augment or modify the way tests are executed (eg run the test in a transaction and roll back, filter/debug on individual cells). The idea was to use something similar to filters (aspect-oriented) to attach to test runners and again encapsulate only test-specific code into that. The suite meta-data configuration file should allow us to specify filters that are applied to the whole test, to individual tables and individual cells (possibly with regex content matching to narrow it down).

Conclusion

With direct domain mapping to repositories and domain objects, setup and verification parts of most of the tests in a rich domain model project can easily be automated without writing fixtures if there is no test specific functionality. Smart keyword mapping will allow us to do the same for the middle parts of the test, where we mostly talk to domain services. Automatically storing objects from the setup part in symbols will make it easy to use those objects in the rest of the test. Using all that, we could effectively connect acceptance tests directly to rich domain models, without fixtures. Repositories and services would be configured in the test suite configuration file, so there would be no additional code required apart from genuinely test-specific code. Any genuinely test specific code could be encapsulated in test-specific repository decorators, and test-specific services, without the requirement to extend any class from the Fixture framework. Generic test-control functionality could be injected and reused across tests using configurable extension points, which should probably just implement a particular filter interface.

Challenges

Ambiguity seems to be the biggest challenge at the moment, and we need to work out strategies for avoiding ambiguity. Another challenge is to identify important naming conventions for repositories so that people can use this approach without changing the way they work at the moment.

An open question is how to divide the test clearly into three parts (setup, action, verification) while keeping them optional (eg support setup-verification or setup-action or action-verification). Rick’s domain fixture at the moment relies on a horisontal line (<HR/>) but this does not work with the standard FitServer. Possibly have some keywords (such as Given, When, Then from BDD) that stand as separate tables on the page.

Going forward

We experimented with relatively simplistic examples, and although this looks promising we need to try it out on some more complicated code. I promised to try to re-write some of my production tests using this model to identify potential pitfalls. If you did not give up reading this by now, then you are probably genuinely interested in the subject, so your feedback and ideas would also be greatly appreciated.