Saturday, September 25, 2010

Spec Style Unit Tests in C#

Spec style unit tests are unit tests that focus on making the specs clear, and then relating some test code to each part of the spec. The Specs Scala unit test framework is really good at that (e.g. check this earlier post), but I want to be able to do the same in C#.


Lets look at an example: A method for deciding whether a given year is a leap year. The rules for leap years are:


A leap year should
    be any year divisible by 400
    but not other years divisible by 100
    and all other years divisible by 4


In other words the above are the specs for the method. So I would like that to be clear from the unit test. I would also like the unit test to make it clear what test code corresponds to what part of the spec. Therefore I would like the spec to be part of the test, and the test code to be mixed right into that spec text. Something like:


"A leap year" should
{
  "be any year divisible by 400" asIn
{
// test code here
}
  "but not other years divisible by 100" asIn
{
// test code here
}
  "and all other years divisible by 4" asIn
{
// test code here
}
}


such that the spec is clearly visible as the strings, and such that the 'asIn' indicates that the code in the following scope tests the part of the spec right before the 'asIn'. To achieve something close to this I've made three simple extension methods on string: 'should', 'asIn' and 'andIn'. Each one of these extension methods take a delegate as an argument. That makes it possible to write this code:

   16             "A leap year".should(() =>
   17             {
   18                 "be any year divisible by 400".asIn(() =>
   19                 {
   20                     // Test code here
   21 
   22                 })
   23                 .andIn(() =>
   24                 {
   25                     // More test code
   26                 });
   27 
   28                 "but not other years divisible by 100".asIn(() =>
   29                 {
   30                     // Test code here
   31                 });
   32 
   33                 "and all other years divisible by 4".asIn(() =>
   34                 {
   35                     // Test code here
   36                 })
   37 

Because this is C# there I need to the dots and the arrows ( ()=> ) in there, but I still think the spec is pretty clearly visible. The extension methods themselves are very simple: They basically just execute the delegate passed to them, which means that all the test code snippets in the above will be executed. The complete code for the extension methods is:




    1 using System;
    2 
    3 namespace Specs
    4 {
    5     public static class SpecExtensions
    6     {
    7         public static void should(this string unitUnderTest,
    8                                   Action executableSpecification)
    9         {
   10             Console.WriteLine(unitUnderTest + " should");
   11             executableSpecification();
   12         }
   13 
   14         public static string asIn(this string readableSpecification,
   15                                   Action executableSpecification)
   16         {
   17             Console.WriteLine(@"    " + readableSpecification);
   18             return RunExecutableSpecification(
   19                     readableSpecification, executableSpecification);
   20         }
   21 
   22         public static string andIn(
   23             this string readableSpecification,
   24             Action executableSpecification)
   25         {
   26             return RunExecutableSpecification(
   27                     readableSpecification, executableSpecification);
   28         }
   29 
   30         private static string RunExecutableSpecification(
   31             string readableSpecification,
   32             Action executableSpecification)
   33         {
   34             executableSpecification();
   35             return readableSpecification;
   36         }
   37     }
   38 }


Combining that with NUnit, the test for my leap year checker becomes:




    1 using NUnit.Framework;
    2 
    3 namespace Specs.Test
    4 {
    5     [TestFixture]
    6     public class LeapYearTest
    7     {
    8         [Test]
    9         public void LeapYearSpec()
   10         {
   11             "A leap year".should(() =>
   12             {
   13                 "be any year divisible by 400".asIn(() =>
   14                 {
   15                     Assert.That(LeapYearChecker.IsLeapYear(0),
   16                                 Is.True);
   17                     Assert.That(LeapYearChecker.IsLeapYear(400),
   18                                 Is.True);
   19                     Assert.That(LeapYearChecker.IsLeapYear(800),
   20                                 Is.True);
   21                 })
   22                 .andIn(() =>
   23                 {
   24                     Assert.That(LeapYearChecker.IsLeapYear(-400),
   25                                 Is.False);
   26                 });
   27 
   28                 "but not other years divisible by 100".asIn(() =>
   29                 {
   30                     Assert.That(LeapYearChecker.IsLeapYear(100),
   31                                 Is.False);
   32                     Assert.That(LeapYearChecker.IsLeapYear(100),
   33                                 Is.False);
   34                     Assert.That(LeapYearChecker.IsLeapYear(200),
   35                                 Is.False);
   36                     Assert.That(LeapYearChecker.IsLeapYear(1000),
   37                                 Is.False);
   38                 });
   39 
   40                 "and all other years divisible by 4".asIn(() =>
   41                 {
   42                     for (int i = 0; i < 100; i += 4)
   43                     {
   44                         Assert.That(LeapYearChecker.IsLeapYear(i),
   45                                     Is.True);
   46                     }
   47                 })
   48                 .andIn(() =>
   49                 {
   50                     for (int i = 1600; i < 1700; i += 4)
   51                     {
   52                         Assert.That(LeapYearChecker.IsLeapYear(i),
   53                                     Is.True);
   54                     }
   55                 })
   56                 .andIn(() =>
   57                 {
   58                     for (int i = 1601; i < 1700; i += 4)
   59                     {
   60                         Assert.That(LeapYearChecker.IsLeapYear(i),
   61                                     Is.False);
   62                     }
   63                 });
   64             });
   65         }
   66     }
   67 

The specs are not as easily visible once all the code is there, but I still think its easy to find the specs. What do you think? Is there too much clutter of dots and arrows and stuff? Is the calling back and forth between the test and extension methods too strange? I'm on the fence. One part me really, really likes this. One part of me really, really doesn't.
Oh, and for completeness the code under test is:




    1    public static class LeapYearChecker
    2     {
    3         public static bool IsLeapYear(int year)
    4         {
    5             return year >= 0 &&
    6                 (year % 400 == 0 ||
    7                 (year % 4 == 0 && year % 100 != 0));
    8         }
    9     }

4 comments:

  1. very nice... I like the specs code, but the NUnit code seems to be cluttered. Should you be using camel casing in the SpecExtension?

    ReplyDelete
  2. @Dennis: Using camel casing would follow .NET standard practice, and is what I would usually do, but in this I chose not to, because I think that lower case 'should' and so on reads better right after a specification string. The point is to try to make code where the specifications inlined in that code read just like natural language, or at least close to.

    ReplyDelete
  3. This is nice. The spec style focuses your attention nicely on what you are trying to test. The syntax with the descriptive strings is something that you could share with business analysts and perhaps technically adept end users.

    There is also a good discipline here:

    1. Write your test in descriptive English that verified by application owner.

    2. Layout your code for the tests.

    3. Test, review, test, review until green lights.

    I'm going to adopt this style.

    ReplyDelete
  4. @dhrobbins: Thanks! Let me know how using this for real pans out...

    ReplyDelete