Everyone Else Is Doing It...

commons.testing

2007-04-01T11:59:00.000-07:00

There are lots of different testing tools and frameworks out there. Maybe too many. We end up having to use a whole bunch of different tools to test different parts of our projects. In working on LiFT, we've found that for various reasons we're currently tied in to JUnit, and it's quite a lot of work to create a version for TestNG or other frameworks, or to integrate our acceptance tests with jMock etc. Because of these problems we're looking forward to the release of commons.testing, a framework agnostic API for writing tests. This upcoming release was the cause of a lot of buzz at the SPA conference last week.

With its extensive XML configuration support, commons.testing also aims to remove the dependency on writing code, allowing tests to be written by non-programmers. Again very well aligned with LiFT's goals. Look out for more information on this all-encompassing framework soon.

Push Web

2007-02-11T12:32:00.000-08:00

It seems that as the amount of content on the web grows, it becomes less and less efficient to surf around and look for interesting things. Where previously people would have their lists of sites that they would visit regularly to catch up on updates, more and more people are now setting up subscriptions and feeds so that they don't have to take the time to go and look if there is anything new.

Rather than going out hunting for new content, we want new content to come to us. We want it to be a push rather than a pull model. However, even with RSS feeds and the like, at the moment we are still polling our favourite sites at regular intervals, it's just that we have a machine to do it for us. Although we appear to be getting content pushed to us, we aren't really.

I think that the next step for the web is some sort of real push messaging for content, so in my browser I just set up a list of things that I'm interested in, and when new stuff is available, and I'm connected, it is piped to my machine for me to read. One of the things I read recently which reinforced this feeling was Tibco's sponsorship of the development of DWR. DWR is a framework for working with AJAX in web applications, and Tibco one of the big name vendors of messaging products.

Also, with the increase in the numbers of mobile devices used for internet access, push would seem to be a good fit. Connectivity for a mobile device is at best intermittent, and currently if I want to look at the news headlines when I'm out of signal range, I'm stuck. With a push model, my device can be updated asynchronously as I walk around in and out of signal areas, so I should have some reasonably fresh data whenever I look. It would be a substantial change to the way that the web has tradionally worked to push content to the browser, but I can feel it coming.

New Year, New Calendar

2007-01-03T13:08:00.000-08:00

In planning our iteration's work at the beginning of this week, we used something simple but effective to focus on how many people were available, and who could work on what when. We drew up a simple calendar on a whiteboard showing when each person would be on holiday, and when people would be out of the office (at client meetings etc).

The effect that this had was that we were clearly able to think about estimating and committing to work suitable for the people available. For example, we can see that Raymond is not back from his Christmas break until Thursday. There were some tasks that he has the most relevant skills for, so our customer agreed that we should move these to next week, instead prioritising things that Kylie could start on, and pair with Raymond when he gets back. The graphical representation gave us something that clearly showed our resource constraints, and it clearly provided a basis for prioritisation that was easily accessible to the customer.

It was also easy for us to see how many person days we had available, and to know when the amount of work planned for the week was more than we could commit to. We also used the calendar to record release dates and other pieces of work that need to be done on fixed days. I like the simplicity and the focus that the calendar brings. I think we'll be keeping it.

Intentional Programming

2006-12-27T21:00:00.000-08:00

So, it's been a while since I last posted anything here. It's been a busy few months and I'm just finding a little bit of time to catch up in the Christmas break. One of the major things that happened in the last couple of months was XPDay which we organised in London. The conference sold out (for the sixth year running) and thanks to everyone who came along and participated it was a great event.

One of the sessions that I enjoyed the most at the conference was Awesome Acceptance Testing presented by Dan North and Joe Walnes. Dan and Joe presented a number of different approaches to acceptance testing, and talked about what acceptance testing means for different people in the development process - often these are the same people wearing different hats at different times - customers, analysts, testers and developers. They distilled the subject to five key themes. To do acceptance testing effectively, you need: automation, a harness, a vocuabulary, a syntax, and to be capturing intent.

This last point seems very important, and in fact links together a number of different themes at the conference, and a number of different open source projects that are currently under development in the area of testing. We are trying to build tests that express their intent. The clearer they are about how the system is supposed to behave, the more use they are as a specification, and the easier it is for customers to validate them. How do we know that we are writing the right tests unless we can check them with the customer?

Also at XPDay, Tamara Petroff and I ran a workshop session on literate testing. The idea here was for customers and developers to work together to create tests for a given scenario that expressed their intention clearly enough that there could be a discussion between customer and developer focussed on the test. However, we provided a structure which meant that the resulting tests were of a form that could be readily automated, making them of direct use to the development team. Our literate testing work has resulted in the release of the LiFT framework which allows tests to be written in Java that read as natural language. At the moment the framework focusses on the web domain, but one of the things that we experimented with in the workshop was how tests for different domains might be written while maintaining the same overall structure.

Groups worked together assembling tests usings decks of cards that we provided with various keywords printed on them. The also had some blank cards available to that they could extend the vocubulary when they needed to. This is important, as to express intent clearly, we need to be able to write our tests using the language of the domain that we are working in. If we are restricted to a fixed vocabulary by our test framework, this is very limiting, and often leads to tests being written that talk about the technology used to solve the problem (e.g. user interface widgets) rather than activity that the user is trying to carry out with the system.

One of our exercises involved writing acceptance tests for a report to be produced in the form of a spreadsheet. Most of the printed cards that we had provided were tailored to the web domain, so the teams had to produce more cards with words to describe the spreadsheet domain. From the results (in the picture above) we can see that the parts of the vocabulary to do with assertion could be reused from one domain to another, but the types of things present in each domain were different, and this is where new cards needed to be written. The pattern of printed and handwritten cards in columns struck me as highlighting the structure that is provided by the test framework. This helps to guide us towards creating tests that are automatable, but at the same time giving us the freedom in vocubulary to express our intent.

While we are discussing this topic, there are a couple of other projects worth mentioning. The assertThat syntax that we use in LiFT originally came from Joe Walnes, and was built into jMock. Not the matching/constraints part of jMock has been extracted into its own extensible library, Hamcrest. This has recently seen its 1.0 release and is well worth checking out.

One of the reasons for the release of Hamcrest was the upcoming (sometime real soon now, apparently) release of an RC1 version of jMock2. Having used jMock2 for a couple of weeks now, there are a few things that make it feel so much better than the original jMock. They're difficult to describe, but it's now much easier to write your test first and use your IDE to reactively complete/generate the code you need to make the test pass. jMock2 also uses Hamcrest, and focuses on providing a syntax that makes tests easy to read.

I can see intentional programming becoming a theme for 2007 (although I probably don't mean exactly the same thing that Charles Simonyi does).

PHP - the most postmodern of programming languages?

2006-10-29T13:44:00.000-08:00

Recently I've been doing a bit of maintenance work on a few websites written in PHP. I've never really been a PHP programmer, but I can read existing code and search for examples of things I want to do on the web. One way or the other I can usually get to something that works, although it might involve some trial and error, and I doubt that what I create is the most elegant solution to the problem. While doing this it struck me that I know very few people who would profess to be PHP programmers, but a lot of people who have "done some PHP" - many people who work with PHP in the same way that I do. You know enough programming concepts to envisage what should be possible, and you can copy, paste and edit existing code to come up with a solution.

This seems very much in line with some of the things that James Noble and Robert Biddle talked about in their paper Notes on Postmodern Programming. Postmodern programming takes the view that things are often not designed and implemented in a consistent way from scratch, but more often created by integrating and reusing existing software components, or bits of them, and glueing them together to make something that works. This is a topic that will be explored further at PoMoPro, the first international conference on postmodern programming, to be held in London on the 25th of November.

Although I can see that a lot of software projects do follow these themes of combining various existing parts to produce a whole, and that there is a certain satisfaction and pragmatism to rigging something up as fast as possible that does the job, I have to say that I am probably something of a modernist at heart. I have a better feeling about a project when the overall design is consistent and elegant. I prefer those clean lines. But I suppose that this is horses for courses, and in some cases the best thing to do is bolt together what you can find, in other cases you might be better served by creating an overarching design.

XPDay6 Registration Open

2006-09-27T22:28:00.000-07:00

Registration is now open for XP Day 6 in London on November 27-28. The programme is also now available on the website. There is a great set of sessions spread over two days on various aspects of agile development, from the hard core technical to the much more people focussed. Places are limited, and always sell out quickly, so make sure to register early.

LiFT: Literate Functional Tests go open source

2006-09-21T11:39:00.000-07:00

I've written a fair bit about literate testing in the past. With some support from my current employer Kizoom, the framework I have been working on has now been published as open source on java.net. We've called it LiFT.

Here is the LiFT page on java.net. Please try it out and let us know if you have any feedback.

For more background, see the video of the presentation Tom White and I gave at the Google London Test Automation Conference.

Thanks to all who have contributed code and ideas to get us this far. We hope there is a bright future for LiFT.

Google London Test Automation Conference

2006-09-17T19:23:00.000-07:00

The conference was held on the 7th and 8th of September in London. There was a wide range of talks over two days on various aspects of automated testing. In London, Google focus on mobile development and it was heartening to hear that they also think testing applications to be accessed through a large number of different devices is a hard problem, and that they don't have all the answers. There was an interesting talk on performance testing from Goranka Bjedov of Google. She advocated the use of open source tools like Grinder and JMeter. There were other interesting talks on testing ajax applications (Joe Walnes and Adam Connors), testing different browsers (Jason Huggins) and deployment environments (Steve Loughran), and testing wireless networks (Karl Garcia).

Tom White and I gave a talk about the development and use of our framework for Literate Functional Testing. This generated a lot of interest. Thanks to everyone for their comments and questions. We hope to be open sourcing the project soon.

All of the talks were recorded, and are now available online on Google video. The slides from the literate testing talk are also available here as a pdf file.

Conference on Test Automation - Registration Open

2006-07-13T13:52:00.000-07:00

Registration is now open for the Conference on Test Automation, to be held in London in September. The programme of speakers looks very interesting - I'm particularly looking forward to hearing what Rick Mugridge, Michael Ernst and Nigel Daley have to say. There's also a talk on testing on mobile handsets which sounds pretty relevant to things that I'm working on at the moment.

In perhaps typical style, Google have adopted a slightly unusual approach to the registration process for the conference. If you'd like to attend, simply draft a missive in 400 words or less describing "what you expect to learn at this conference and the specific issues you'd like the speakers to address". Post your application in the online form. Hope to see you there.

XP Day 6

2006-06-28T22:26:00.000-07:00

The details of this year's XP Day conference have been announced. The conference will take place on 27th-28th November, in London, UK. If the past years are anything to go by, this should be an excellent event. The style of the conference is such that most of the sessions actively encourage participation by the delegates, normally through some sort of interactive exercise. This is a great way to learn, and to get to know new people.

I am helping to organise the programme for this year's conference, and we are now looking for people to submit suggestions for sessions they would like to run. Full details of what we are looking for, and instructions for how to submit, can be found in the call for proposals. The closing date for submissions is August 14th, so get thinking.

Update - generics puzzles

2006-06-26T19:40:00.000-07:00

After my post yesterday, Tristan Allwood and Nick Cameron both wrote to offer some help. When you want a list of things that implement both interface A and interface B, the syntax that you need is: List<T extends A & B>. Yes, that's an ampersand. Ugly isn't it. To make matters worse, this syntax doesn't mix well with wildcards, so you can't write List<? extends A & B> . Instead you have to write <T extends A & B> void m(List<? extends T>) { ... } .

More generics puzzles

2006-06-25T20:42:00.000-07:00

This week I had a couple of interesting discussions with people about some particular points of programming with Java generics. The first case was that a developer wanted to write a method that processed a list of objects that implemented two interfaces. He wasn't concerned with the actual class of the objects, just that they implemented the two interfaces in question. This seemed to be a case for the use of wildcards, where we can specify a method as taking a paramater that is, say, a List<? extends A>. However, it doesn't seem to be possible to write something like List<? extends A, B>, even when A and B are interfaces rather than classes.

Some might say that a method should only act on an object through one of its interfaces, and that any processing using the other interface should be encapsulated in a separate method, but this is a matter of design, and it does not seem that it should be being enforced by the type system. Also, it seems unlikely that this is the reason that Java does not support this.

In the end, the best solution we could come up with was to create an interface C that extended A and B, and to make the objects to be processed implement this interface instead. The source of these classes happened to be under our control so we could do this, but it didn't seem like an optimal solution, and there are many cases where it could not be used.

The second case, was where a different developer was writing a unit test using jMock mock objects for the collaborators of the class under test. In this case, this meant that he needed to mock a generic class, as the class under test had a setter method that takes a particular instantiation of the generic. So we had code along the lines of:


class X {
  setK(K<String> k) {
    ...
  }
}

class TestX extends MockObjectTestCase {
  void test() {
    X x = new X();
    Mock mockK = mock(K.class);
    x.setK(mockK.proxy());
  }
}

The first thing to notice is that we cannot write mock(K<String>.class). See an earlier post for an explanation of why not. Therefore we have to mock K.class. First of all, we wondered whether this would work, especially as we were using the cglib variant of the jMock library, but in fact it did. The interesting part is that in the above code, we have to cast the result of mockK.proxy() in order for it to compile. The most obvious thing is to cast it to a K<String>. But, using Eclipse, we get a warning that says that our cast is "actually checking against the erased type". This is true, as after compilation the generic type information is not present, so there is no way to check this cast at runtime.

If we make the cast a bit less specific, and just cast to a K, then we get a different warning, saying that we need an unchecked conversion to conform to the signature of the method that we are calling. It seems there is no way we can win, and the only way to keep Eclipse happy is to add an annotation to the method to suppress the warnings - not ideal.

To some extent, these problems were caused by the fact that jMock is written using the Java 1.4 API, and so does not have inbuilt support for generics, but this is not the only cause, and it seems to be a common case that generic code is interwoven with older, non-generic code. These sorts of examples certainly show that the Java generics, and the supporting type rules, can be somewhat puzzling.

London Test Automation Conference

2006-06-21T22:31:00.000-07:00

A well-known search engine is hosting the London Test Automation Conference in September. Tom and I were pleased to be invited to present our experiences using the literate testing style that we have been working on recently. Hopefully it'll be an interesting event, although the full programme has yet to be announced, so we don't know who else will be there yet.

Carnival of the Agilists

2006-06-15T13:19:00.000-07:00

Kevin Rutherford has been putting together a fornightly round-up of agile topics being discussed in the blogosphere, which he calls Carnival of the Agilists. We got a mention in the current edition, thanks Kevin. There's also some interesting stuff from Liz Keogh and Joakim Karlsson amongst others.

Brian Marick's Sentence Style Tests

2006-06-14T14:53:00.000-07:00

Brian Marick has also been thinking about making tests more readable (also inspired by jMock). He mentions what Tom and I have been writing about (thanks Brian). His tests are perhaps more readable because a lot of the functions are written to explicitly test particular cases, e.g.


page.main_text.should_have_no_list_named(:visits). 
                   should_have_no_list_named(:audits). 
                   and_should_have_a_help_popup_named(:patient_display_page)

so the logic or quantification is combined with the noun he is trying to match, in the same statement. We have tried to construct more general building blocks, which means we tend to have rather more nesting of clauses. Also, Brian benefits from the fact that he is using Ruby rather than Java, so the syntax is somewhat less cumbersome.

Taking things into consideration

2006-05-31T14:45:00.000-07:00

I've been working on a new style of writing tests for a while. One of thing we're finding at the moment, is that you often want to narrow down the available objects to pick out a particular part, and then make a number of assertions about this same part.

For example, you might be modelling a garden, and you want to write some tests concerning a certain set of plants, maybe those that are yellow. Using the style that Tom White and I have discussed before, we could write something like:


assertThat(garden, has(flowers().coloured(yellow)));
assertThat(garden, has(flowers().in(flowerbed()).coloured(yellow)));

This checks that there exist some flowers that are yellow. But what if we want to check that all the flowers in the flowerbed are yellow? We've come up with the idea of a consideration. We use a considering(...) clause to refine the scope of our assertion, and have also created an assertThatAll() assertion to take into account the universal quantification. The results look like this:


considering(flowerbed());
assertThatAll(flowers(), areColoured(yellow));
assertThatAll(flowers(), areInBloom());

I think this will be subject to some refinement as we put it to use in more places, and we'll see whether or not it is useful. We've certainly found a few places so far where we want to check a property for a set of objects, and writing a loop isn't very in keeping with our literate style.

Tying Tests to Requirements

2006-05-24T14:39:00.000-07:00

On my current project, we've got a large requirements document, with a lot of numbered requirements. We want to be sure that we've covered all the requirements in our automated test suite. Rather than going through by hand checking off tests against the list, we had the idea of marking each test with its corresponding requirements, in the code.

As we're working with Java 5, it seemed appropriate to do this using Java annotations. We created an annotation that can be applied to test methods, parameterised by the number.

Even better, we were able to use the annotation processing tool to run over our test code, and produce a report of which tests test which requirements, and which requirements are not covered by any of our test cases. When I add a test, I can quickly regenerate the report. I have a feeling this is going to be very useful.

Literate Functional Testing

2006-05-24T14:30:00.000-07:00

For a while now, I've been working on some ideas about writing automated tests that are readable even for non-programmers. This makes it easier to write tests in language that is understood by users or customers, making sure that requirements are agreed.

I've been working on this with Tom White, who has written up some good posts on the subject, talking about our use of constraints, and our ideas about using anaphora, on his blog on java.net.

I really meant to write these up myself, but haven't had time for much blogging lately. Thanks to Tom for his articles.

On another note

2006-02-13T22:49:00.000-08:00

A couple of weekends ago I was pleasantly surprised to bump into Steve Freeman at a very English institution. Steve came up with a parallel between the way that orchestral players mark their parts in pencil, and the way people comment their code. With orchestral scores, each player and conductor has their own interpretation of the piece. They often mark their parts in pencil to remind them of changes of tempo, changes of key, or when the pages need turning quickly. As sets of parts get passed from one player or orchestra to another, the amount of pencil tends to increase, often obscuring the original information. It is normally a relief to see a nice clean part with little or no pencil, just the notes and directions as written by the composer. Removing the clutter gives time and space to read and process all of the information on the page, without confusion, contradiction or duplication of information.

People often encourage thorough commenting of code, to explain to other developers what it does, and how it works. Steve and I reckoned that we would much rather see clear, well written code with no comments, than the verbose and largely unhelpful comments that one often sees. Then when you do see a comment, you know to pay particular attention to it. We really want to see the notes underneath the pencil, not be distracted by scribbles.

Drawing Parallels

2006-02-13T22:30:00.000-08:00

Recently I've been reading a bit about Neuro Linguistic Programming (NLP). I haven't got too deeply into it, but it's generally a set of theories about the way that people think, communicate, learn and perceive the world around them. For example, some people think about things mostly in a visual way, whereas others prefer to use sounds or feelings to represent their thoughts internally. The idea is that by learning about the different representation systems, and how to tell which ones people favour, you can tailor your words and actions to communicate with them more effectively.

The particular book I am reading wraps this up in a self-help, being a more effective person, achieve your life goals type of way, but one thing struck me. The book describes the essence of NLP, and being an "effective person", as 1) know what you want; have a clear idea of your outcome in any situation, 2) be alert and keep your senses open so that you know what you are getting, 3) have the flexibility to keep changing what you do until you get what you want.

This seems a lot like what the agile development philosophy says. We have a set of stories that define where we want to get to; during development we monitor progress with daily standups, regular integration, or more empirically with techniques like Scrum; and we change the stories, the solutions, the code, the team etc, etc, until we get where we want to be.

Java Generics Niggle

2006-01-22T13:40:00.000-08:00

I've been using Java 5.0 in anger for a while now, and largely I like the extra type safety and (to some extent) expressiveness that the generic types provide, but there are some things that are somewhat annoying. I want to write a test with an assertion that checks the type of the object that I get out of an untyped map - the map is provided by a library class, and can hold different types of objects under different keys, so it can't be parameterised by type. I want to write an assertion something like this (using the JMock assertion syntax):


     assertThat(myMap.get(KEY_NAME), isA(List<String>.class));

The immediate problem is that this does not compile. List<String>.class is not valid Java 5.0 syntax. Trying various possibilities, it seems the best I can do is List.class. I also tried new ArrayList<String>().getClass(), which compiles but is unsatisfactory because it is too specific about the class that implements the List, and also, testing shows that the result of executing new ArrayList<String>().getClass() is just ArrayList, not ArrayList<String> as we might hope and expect.

What is going on here? The problem is that generics in Java are a compile time rather than a runtime artifact. When Gilad Bracha was deciding how to implement generics for Java, he decided to take an approach based on Pizza and GJ by Phil Wadler. The compiler takes the parameterisation of types given in the source code, and through a process of erasure uses them to insert the casts that would be needed if we were to write this code under Java 1.4. The effect of this is that in the bytecode, and therefore at runtime, there is no information about type parameters. Hence the problem I showed above, where we are trying to make assertions at runtime.

When Don Syme and Andrew Kennedy implemented generics for .NET, they took a different approach, updating the bytecode instructions in the CLR to allow the generic type information to be held and accessed at runtime. I haven't spent the time to find a neat and elegant way of performing the test that I want, but the documentation gives the impression that it should be possible to create a suitable library method. But I want this in Java, maybe in Mustang...

Spring AOP

2006-02-01T21:40:00.000-08:00

I've generally been a bit sceptical about the applicability of aspect-oriented programming (AOP). There seemed to be a lot of hype coming out of places like IBM about AspectJ, but I could never see this really taking off, as it is really a different language, rather than an extension to Java. Kevin Sullivan (et al) have an interesting paper which balances some of the benefits and some of the problems, using AOP to design software. After reading this I was still thinking that aspects might not be that useful.

However, I've recently been looking at the AOP support in Spring, which gives the programmer an API to use, rather than extra programming language constructs. Although this restricts what is possible to a certain degree, it does allow AOP to be used in "normal Java programs", especially as one of the tenets of Spring's design philosphy is to be non-invasive. I have managed to add two useful aspects to my current project, without changing the existing source code, but just by adding some new classes, and some configuration. Spring then weaves its magic to connect things up at runtime (using a lot of reflection and dynamic proxies behind the scenes). Regardless of this, I was pretty impressed.

In the application I am working on at the moment, we are calling a number of external web services, and we have been somewhat concerned with the performance of these. We wanted to measure how long calls to the web services take to complete. By using an aspect, we can write the profiling code in one place, and apply it across all of our web service calls. By configuring it in the Spring application context, we can also insert or remove the profiling code without having to alter the Java source.

I wrote the following simple class, which implements an interface from the Spring API, to implement some around advice. There's a lot of terminology used in AOP, for a quick guided tour, see this glossary.


public class SimpleProfilingAdvice implements MethodInterceptor {
   
    public Object invoke(MethodInvocation invocation) throws Throwable {
               
        StopWatch sw = new StopWatch();
        sw.start(invocation.getMethod().getName());
        Object returnValue = invocation.proceed();
        sw.stop();
       
        report(invocation, sw.getTotalTimeMillis());
        return returnValue;
    }

    private void report(MethodInvocation invocation, long ms) {
        // print out the timing info etc
    }
}

Then in the configuration, I just add:


   <bean id="webServiceFacade" class="WebServiceFacadeImpl" />
   
   <bean id="profiler" class="SimpleProfilingAdvice">

   <bean id="proxyCreator" class="org.springframework.aop.framework.autoproxy.BeanNameAutoProxyCreator">
       <property name="beanNames">
            <list><value>webServiceFacade</value></list>
        </property>
        <property name="interceptorNames">
            <list>
                <value>profiler</value>
            </list>                     
        </property>
    </bean>

Spring automatically creates a proxy for my webServiceFacade bean, which intercepts calls, and runs my advice instead (which in turn, proceeds to call the original method).

In the same vein, I took the ideas presented by Tom White in his article on using memoization for caching, and created a caching aspect. I could then chain this together with my profiling aspect to compare performance of the calls with and without caching.

More on Generics

2006-01-31T21:00:00.000-08:00

In a recent discussion on the Slurp list, Sun's pages on advanced use of generics came under some scrutiny.

Sun says:

Suppose we want to create a TreeSet<String> and pass in a suitable comparator, We need to pass it a Comparator that can compare Strings. This can be done by a Comparator<String>, but a Comparator<Object> will do just as well.

Slurp say:

We think introducing Object into a discussion about generics is bad... It would be better if the page had avoided Strings and Objects, and said:

"Suppose we want to create a TreeSet<Chicken> and pass in a suitable comparator. We need to pass it a Comparator that can compare Chickens. This can be done by a Comparator<Chicken>, but a Comparator<Animal> (where Animal is a superclass of Chicken) will do just as well. We COULD write the constructor as TreeSet(Comparator<? extends Animal> c) but it would be better to avoid mentioning a specific superclass of Chicken. Therefore TreeSet(Comparator<? super Chicken> c) is best."

Comparator<? extends Animal> is not quite good enough, as I can pass in a FreeRangeChickenComparator() which ranks chickens based on their happiness, an attribute not exposed in the Chicken superclass, but this is not really the point here. I agree that what Sun's page says is confusing. In most cases, a Comparator<Object> will not do "just as well", as it will likely return a different ordering.

If you're just using the TreeSet as an efficient storage structure then any order will indeed do, but if you want to iterate over the contents, then you likely want a particular total ordering, e.g. alphabetically by species, or average egg diameter...

TreeSets in general are slightly annoying, as they try to combine being a TreeSet<E extends Comparable>, with being a TreeSet<E> with a constructor taking a Comparator<? super E> (as we have seen). In order to support passing in a custom Comparator (which could work on any type), the set must be declared as TreeSet<E>, rather than TreeSet<E extends Comparable>.

This leads to a few loopholes. I can easily create a TreeSet<NotComparable>, where upon calling treeSet.add(new NotComparable()); (more than once, as you don't need to compare anything when you add the first element) causes a runtime ClassCastException. So this test compiles and passes:


public void testGenericTreeSet() {
     
    try {
        Set<NotComparable> ts = new TreeSet<NotComparable>();
        ts.add(new NotComparable());
        ts.add(new NotComparable());
     fail();
     } catch(Throwable t) {
         assertThat(t, isA(ClassCastException.class));
     }
    }
}

Generics are not being very helpful here, you'd hope that all this extra "type-safety" would allow this sort of thing to be flagged up at compile time, but making this class work in two ways, and in keeping everything backwardly compatible, there are some cracks between the floorboards.

Testing Controllers With Mocks

2006-01-19T23:11:00.000-08:00

One of the reasons for setting up this page was I wanted to write something somewhere about the new testing techniques I've been using over the past few days. I'm working on a Java web application, using a fairly standard MVC structure. Working on an XP team, I've always felt a little uneasy about not having a good way to build controllers test first. Typically in past projects I've tested controllers indirectly by using tools like HttpUnit to interact with the web front end, and exercise the view and controller components together. This approach has a number of drawbacks. In the case of a failure, it is difficult to determine whether the bug is in the view or the controller (or even the model). It is also necessary to develop the view and the controller in tandem, as one cannot be tested without the other.

On my current project, we are using Spring to provide the MVC framework. This allows us to make use of some of the other facilities provided by Spring to enhance our testing.

Spring provides support for unit testing controllers independently. This has a number of advantages:

We can develop controllers test-first.

Breakages in the unit tests point explicitly to problems in the controller, rather than problems in one of the combination of Model, View and Controller.

We can run the unit test outside of the container, meaning that we do not have to deploy within each code/test cycle.

The tests themselves run faster.

We can develop and test all of our controller code before developing any of the view components.

We can develop and test all of our controller code before creating the Spring application-context.xml, which makes the requirements for the contents of this file much clearer.

The difficulty we had had with unit testing controllers previously was that the methods in their public interface tended to take an HttpServletRequest and HttpServletResponse as parameters, and operate on these. When running a unit test, such objects are not normally to hand, and are difficult to construct. Spring aids this situation by providing the classes MockHttpServletRequest and MockHttpServletResponse in spring-mock.jar (note that these classes are not part of the core Spring jar). Here is an example of how we can use them in a JUnit test case.


import org.springframework.mock.web.MockHttpServletRequest;
import org.springframework.mock.web.MockHttpServletResponse;
import org.springframework.web.servlet.ModelAndView;

public MyControllerTest extends TestCase {

    public void testGettingIndexPage() throws Exception {

        MyController controller = new MyController();
        controller.setIndexView("index.jsp");

        MockHttpServletRequest request = new MockHttpServletRequest();
        MockHttpServletResponse response = new MockHttpServletRequest();

        request.setMethod("GET");

        ModelAndView modelAndView = controller.handleRequest(request, response);

        assertEquals("Should get index page", modelAndView.getViewName(), "index.jsp");
    }
}

This test shows that we intend our controller to respond to requests on the base url to display index.jsp. By writing the test first, we can see what we need to implement in our controller. To make this test pass requires only a very simple controller (which will be a subclass of one of the Spring MVC controllers).


import org.springframework.web.servlet.mvc.AbstractController;

public class MyController extends AbstractController {

    private String indexView;

    public void setIndexView(String viewName) { indexView = viewName; }
    public String getIndexView() { return indexView; }

    public ModelAndView handleRequestInternal(HttpServletRequest request, HttpServletResponse response) {
        return new ModelAndView(getIndexView());
    }
}

To make things a bit more interesting, we can add a test that passes in some parameters on the request, and expects an appropriate response, for example passing in a search query and expecting a results page back. Adding to our test case, we can write:


    public void testSearching() throws Exception {

        MyController controller = new MyController();
        controller.setSearchResultsView("results.jsp");
        MockHttpServletRequest request = new MockHttpServletRequest();
        MockHttpServletResponse response = new MockHttpServletResponse();
        request.setMethod("GET");

        request.addParameter("query","testing");

        ModelAndView modelAndView = controller.handleRequest(request, response);

        assertEquals("Should get results page", modelAndView.getViewName(), "results.jsp");        
    }

Again the intention of the test is pretty clear (we can now see that our testcase might benefit from factoring some common setup into a setup method, but we'll leave that for now), and again, we don't need to add much to our controller to make it pass.


public class MyController extends AbstractController {

    //  in addition to the fields and accessors from before...
    private String searchResultsView;

    public void setResultsView(String viewName) { searchResultsView = viewName; }
    public String getSearchResultsView() { return searchResultsView; }

    public ModelAndView handleRequestInternal(HttpServletRequest request, HttpServletResponse response) {

        String query = request.getParameter("query");
        if ( query != null ) {
            // perform the search
            return new ModelAndView(getSearchResultsView());
        } else {
            return new ModelAndView(getIndexView());
        }
    }
}

This shows a couple of very simple iterations of writing tests and implementation for a controller, you can go a lot further, making both the tests and the implementation more sophisticated. The main point is that we have built and tested the controller in a way that allows quick iteration, promotes clarity and good test coverage, and does not require a view component to be written to do so.

More Mockery

2006-01-20T21:36:00.000-08:00

In my last post I described using Spring's mock request and response classes to unit test controllers. The examples given were very simple. In real situations, controllers typically depend on other resources, often to provide data sources etc. These can also be troublesome to test with, and cause tests to be unpredictable and slow.

To combat this, and create fast, decoupled unit tests, I have been using JMock in tandem with the Spring mock library. This allows very expressive tests to be created, which run independently of any actual data source dependencies, and have predicatable results.

For example, say we are writing a controller to list the avaiable books in a library. We may have a Library interface, which we call to query an underlying database. Using JMock we can write a test like this:


public class LibraryControllerTest extends MockObjectTestCase {
    
    private static final String MODEL_NAME = "books";
    private static final String INDEX_PAGE = "index.jsp";

    private Mock mockLibrary;
    private LibraryController controller;
    private MockHttpServletRequest request;
    private MockHttpServletResponse response;

    protected void setUp() throws Exception {
        super.setUp();
        mockLibrary = mock(Library.class);
        
        request = new MockHttpServletRequest();
        response = new MockHttpServletResponse();
        request.setMethod("GET");
        
        controller = new LibraryController();
        controller.setLibrary((Library)mockLibrary.proxy());
        controller.setIndexView(INDEX_PAGE);
        controller.setModelName(MODEL_NAME);
    }

    public void testListingBooks() throws Exception {
        mockLibrary.expects(once()).method("getAvailableBooks").will(returnValue(new BookList()));
        ModelAndView modelAndView = controller.handleRequest(request, response);
        assertThat(modelAndView.getViewName(),eq(INDEX_PAGE));
        assertThat(modelAndView.getModel().get(MODEL_NAME), isA(BookList.class));
    }
}

Here I have set up a mock of the Library interface, and in testListingBooks set up an expectation that this mock will have its getAvailableBooks() method called once durnig the course of the test. I have also set a stub return value, so that when this happens, the mock will return a new BookList, so that the returned value is predictable. See the JMock documentation for more on setting up mocks with expectations.

Now we have a concise test that expresses what we want our controller to do. When we request the page, we expect it to call the Library, and put that information into the ModelAndView that is returned from handleRequestInternal().
To make this test pass, again requires only a simple controller.


import org.springframework.web.servlet.mvc.AbstractController;

public class LibraryController extends AbstractController {

    private String indexView;
    private String modelName;
    private Library library;

    public void setIndexView(String viewName) { indexView = viewName; }
    public String getIndexView() { return indexView; }
    public void setModelName(String name) { modelName = name; }
    public String getModelName() { return modelName; }
    public void setLibrary(Library lib) { library = lib; }
    public Library getLibrary() { return library; }

    public ModelAndView handleRequestInternal(HttpServletRequest request, HttpServletResponse response) {
        BookList list = library.getAvailableBooks();
        return new ModelAndView(getIndexView(),getModelName(),list);
    }
}

Another nice thing that we can do, is to test the case where there is a problem with the data source. We can set up an expectation on the mock Library that it will throw an exception when it is called, and then check that the controller gives an appropriate response, for example routing to an error page.


public class LibraryControllerTest extends MockObjectTestCase {

    private static final String ERROR_PAGE = "error.jsp";
    ...
    public void testErrorFromDataSource() throws Exception {
      controller.setErrorPage(ERROR_PAGE);  mockLibrary.expects(once()).method("getAvailableBooks").will(throwException(new DatabaseDownException()));
        ModelAndView modelAndView = controller.handleRequest(request, response);
        assertThat(modelAndView.getViewName(), eq(ERROR_PAGE));
    }
}

This sets up the mock Library to throw an exception when it is called, and checks that the view that we get back is the error page we expect. We can then complete the controller.


public class LibraryController extends AbstractController {
    ...
    // field, and getter and setter for errorPage here
    ...
    public ModelAndView handleRequestInternal(HttpServletRequest request, HttpServletResponse response) {
        try {
            BookList list = library.getAvailableBooks();
            return new ModelAndView(getIndexView(),getModelName(),list);
        } catch (DatabaseDownException dde) {
            return new ModelAndView(getErrorPage());
        }
    }
}

Again, writing the test first gives us a clear direction for what we want to do when we write the code, and we can be sure that the controller is behaving correctly when our tests pass, without having to construct any of the view components.