Monday, October 24, 2011

Frictionless .NET Web App Development with Nancy Part III - Introducing MongoDB

Continuing the URL shortener sample from thesse post: part I and part II, I'll go from (stupidly) storing the shortened URLs in a static variable, as was the case in the code in part I, to storing the shortened URLs in MongoDB. The reasons for choosing MongoDB for persistence in this sample (compared to the "traditional" NHibernate + SQL Server approach) are:
  • We will only store and retrieve one document at a time. Those are atomic operations in Mongo, so as far as this sample goes Mongo is fully consistent.
  • Mongo is simple to setup, and very simple to code against. For instance there is no O/R mapping layer at all.

In summary, Mongo is a nice frictionless option for this sample.

Enough talk, show me the code!

Decoupling interface from persistence
The first thing to do is to pull the persistence related code out of the Nancy module. We use the following couple of tests to drive that. They use a Moq mock to specify the interface to the persistence code; that is the UrlStore interface:

    1 namespace ShortUrlTest
    2 {
    3   using Xunit;
    4   using Nancy.Testing;
    5   using Moq;
    6   using ShortUrl;
    8   public class UrlStorageTest
    9   {
   10     private Browser app;
   11     private Mock<UrlStore> fakeStorage;
   13     public UrlStorageTest()
   14     {
   15       ShortUrlModule artificiaReference;
   17       //Given
   18       fakeStorage = new Mock<UrlStore>();
   19       app = new Browser(
   20               new ConfigurableBootstrapper(
   21                 with => with.Dependency(fakeStorage.Object)));
   23     }
   25     [Fact]
   26     public void app_should_store_url_when_one_is_posted()
   27     {
   28       //When
   29       app.Post("/",
   30         with =>
   31         {
   32           with.FormValue("url", "");
   33           with.HttpRequest();
   34         });
   36       //Then
   37       fakeStorage
   38         .Verify(store =>
   39                 store.SaveUrl("", 
   40                               It.IsAny<string>()));
   41     }
   43     [Fact]
   44     public void app_should_try_to_retrieve_url_when_getting_a_short_url()
   45     {
   46       //When
   47       app.Get("/shorturl",
   48         with => with.HttpRequest());
   50       //Then
   51       fakeStorage
   52         .Verify(store => 
   53                 store.GetUrlFor("shorturl"));
   54     }
   55   }
   56 }

A few interesting things go on in these tests.

Firstly, as stated above they specifify the interface UrlStore. They do so via the mock fakeStorage, and the calls to fakeStorage.Verify(...) which tells us that UrlStore should have two methods, SaveUrl and GetUrlFor.

Secondly we again use the Nancy.Testing.Browser type to call into the ShortUrl Nancy module. Remember that this goes though the whole Nancy pipeline, just like a real request from a browser. So we're genuinely testing the module from the outside. In the first test we do a HTTP POST, and in the second one we do a HTTP GET.

Thirdly in the constructor we're using another feature of Nancy.Testing; the ConfigurableBootstrapper. To understand this bit, a little detour is required. Nancy is very modular, everything in Nancy can be swapped for an alternative implementation via configuration. Furthermore Nancy apps uses IoC/DI from the get go. Out of the box Nancy uses TinyIOC, but that can (of course) be swapped with any other container - and NuGets exists for most (if not all) popular containers. The swapping out of things in Nancy apps is done through code, in a bootstrapper, which is a class in the app that implements INancyBootstrapper, and sets up the whole of Nancy and the app. To make this easy Nancy provides the DefaultNancyBootstrapper which can be used as the base class for app specific bootstrappers. Back to tests above. The ConfigurableBootstrapper, provides a simple way for tests to create specialized configurations through a fluent interface. The contructor in the code above registers the mock UrlStore as a dependincy, which means registering it in the TinyIOC container. In other words the parts of a the app that receives a UrlStore from the container will get the mock one when in the context of these tests.

But the UrlStore interface doesn't exists, so lets write it to make the tests compile:

    1 namespace ShortUrl
    2 {
    3   public interface UrlStore
    4   {
    5     void SaveUrl(string url, string shortenedUrl);
    6     string GetUrlFor(string shortenedUrl);
    7   }
    8 }

The tests don't run yet though, since our app does not use UrlStore at all. To make it do so, we change the application code to take a UrlStore in the module contructor, and use it for storage (particularly lines 7, 14 and 22):

    1 namespace ShortUrl
    2 {
    3   using Nancy;
    5   public class ShortUrlModule : NancyModule
    6   {
    7     public ShortUrlModule(UrlStore urlStore)
    8     {
    9       Get["/"] = _ => View["index.html"];
   10       Post["/"] = _ => ShortenUrl(urlStore);
   11       Get["/{shorturl}"] = param =>
   12       {
   13         string shortUrl = param.shorturl;
   14         return Response.AsRedirect(urlStore.GetUrlFor(shortUrl.ToString()));
   15       };
   16     }
   18     private Response ShortenUrl(UrlStore urlStore)
   19     {
   20       string longUrl = Request.Form.url;
   21       var shortUrl = ShortenUrl(longUrl);
   22       urlStore.SaveUrl(longUrl, shortUrl);
   24       return View["shortened_url", new { Request.Headers.Host, ShortUrl = shortUrl }];
   25     }
   27     private string ShortenUrl(string longUrl)
   28     {
   29       return "a" + longUrl.GetHashCode();
   30     }
   31   }
   32 }

And that makes the above tests run. But the end-to-end tests shown in part I don't run. The reason is they don't register a UrlStore with the container. They use the DefaultNancyBootstrapper, where no UrlStore is setup. To get past this we need to do a few things: Change the end-to-end tests to use an app specific bootstrapper, create an implementation of UrlStore that uses Mongo, and create an app specific bootstrapper. We'll start by driving out the Mongo implementation of UrlStore.

Shifting gears
To implement UrlStore against Mongo we need to shift TDD gears to using integration tests. The following tests, that drive the implementation of MongoUrlStore assume a Mongo server is running on localhost at port 27010:

    1 namespace ShortUrlTest
    2 {
    3   using System.Linq;
    4   using MongoDB.Bson;
    5   using MongoDB.Driver;
    6   using MongoDB.Driver.Builders;
    7   using ShortUrl.DataAccess;
    8   using Xunit;
   10   public class MongoUrlStoreTest
   11   {
   12     private string connectionString = "mongodb://localhost:27010/short_url_test";
   13     private MongoDatabase database;
   14     private MongoCollection<BsonDocument> urlCollection;
   16     public MongoUrlStoreTest()
   17     {
   18       //given
   19       database = MongoDatabase.Create(connectionString);
   20       urlCollection = database.GetCollection("urls");
   21         urlCollection.RemoveAll();
   22     }
   24     [Fact]
   25     public void should_store_urls_in_mongo()
   26     {
   27       //when
   28       var store = new MongoUrlStore(connectionString);
   29       store.SaveUrl("", "http://shorturl/abc");
   31       //then
   32       var urlFromDB = urlCollection
   33         .Find(Query.EQ("url", ""))
   34         .FirstOrDefault();
   36       Assert.NotNull(urlFromDB);
   37       Assert.Equal(urlFromDB["shortenedUrl"], "http://shorturl/abc");
   38     }
   40     [Fact]
   41     public void should_be_able_to_find_shortened_urls()
   42     {
   43       //given
   44       var store = new MongoUrlStore(connectionString);
   45       store.SaveUrl("", "http://shorturl/abc");
   47       //when
   48       var longUrl = store.GetUrlFor("http://shorturl/abc");
   50       //then
   51       Assert.Equal("", longUrl);
   52     }
   53   }
   54 }

These two test cases simply use the UrlStore interface, but also go directly to the Mongo instance to check that things are stored there. This is all quite straight forward, the one thing to notice though is that these test only assume that there is a Mongo server running, not that there is any particular database or collection present from the start. This is one of the ways Mongo is easy to work with, the databases and collections used in the code are just created for us behind the scenes. No friction.
To satisfy these tests we simply implement the UrlStore interface as follows:

    1 namespace ShortUrl.DataAccess
    2 {
    3   using System.Linq;
    4   using MongoDB.Bson;
    5   using MongoDB.Driver;
    6   using MongoDB.Driver.Builders;
    8   public class MongoUrlStore : UrlStore
    9   {
   10     private MongoDatabase database;
   11     private MongoCollection<BsonDocument> urls;
   13     public MongoUrlStore(string connectionString)
   14     {
   15       database = MongoDatabase.Create(connectionString);
   16       urls = database.GetCollection("urls");
   17     }
   19     public void SaveUrl(string url, string shortenedUrl)
   20     {
   21       urls.Insert(new {url, shortenedUrl});
   22     }
   24     public string GetUrlFor(string shortenedUrl)
   25     {
   26       var urlDocument =  
   27         urls
   28           .Find(Query.EQ("shortenedUrl", shortenedUrl))
   29           .FirstOrDefault();
   31       return 
   32         urlDocument == null ? 
   33         null : urlDocument["url"].AsString;
   34     }
   35   }
   36 }

The thing I really want to point out here is line 21. We just tell Mongo to store an anonymous object and the driver does the rest for us (i.e. serializes it to json). No O/R mapping code!
Now we have an interface for persisting things, our module uses that interface, and we have an implemtation of it against Mongo. The only thing remaining is wiring things up.

Shifting gears back the end-to-end tests
As mentioned earlier the end-to-end tests from part I don't run right now, to make them run we first change their setup to use an app specific bootstrapper:

   14    public BaseUrlSpec()
   15    {
   16      ShortUrlModule artificiaReference;
   17      app = new Browser(new Bootstrapper());
   18    }

We've changed line 17 to use Bootstrapper instead of DefaultNancyBootstrapper. The type Bootstrapper does not exists yet, so lets create it, and have it register MongoUrlStore with the container:

    1 namespace ShortUrl
    2 {
    3   using Nancy;
    4   using DataAccess;
    6   public class Bootstrapper : DefaultNancyBootstrapper
    7   {
    8     protected override void ConfigureApplicationContainer(TinyIoC.TinyIoCContainer container)
    9     {
   10       base.ConfigureApplicationContainer(container);
   12       var mongoUrlStore = new MongoUrlStore("mongodb://localhost:27010/short_url");
   13       container.Register<UrlStore>(mongoUrlStore);
   14     }
   15   }
   16 }

We use the DefaultNancyBootstrapper as a starting point, and just use one its hooks to do a little bit of app specific setup, namely registering MongoUrlStore. Using other overrides we could configure anything in Nancy. I really like that a lot.

Well that's it. All tests run again.

Updated code is a available on GitHub.


  1. Nice post - just one comment though, you shouldn't need to do the container.Register (although you can) - By default Nancy will automatically find and inject your dependencies for you, even if they're interfaces. All part of the Super Duper Happy Path (TM) ;-)

  2. @Steve: I cant seem to make that work: I get an exception from TinyIoC, because it can't resolve NancyEngine, which when inspecting inner exceptions turns out to be because UrlStore cannot be resolved, because MongeUrlStore cannot be resolved, because the string connectionString cannot be resolved.
    I tried registering the connectionString instead, but got the same exception.

    Still: This is all well within the Super Duper Happy Path as far as I'm concerned :-)

  3. Thank you Christian!

    I haven't grasped the ConfigurableBootstrapper until this post. Made a lot of sense now. And is also in the same fluent form as the rest of Nancy - Nice!

    One question - that variable you have in the Given-part of your tests (ShortUrlModule artificiaReference) - why is that needed?

    I guess that it is referenced and hence a part of my AppDomain but the strange thing is that when I removed it still worked fine - I still had a reference to the production code assembly though.

    In short - if I reference the production code assembly (the NancyApp if you like) why do I need that reference?

  4. @Marcus: You're right the artificial reference is unnecessary. It was there to ensure that the assembly under test was loaded, but since the interface UrlStore is also in that assembly it's already loaded.

  5. Excellent pieces. Keep posting such kind of information on your blog. I really impressed by your blog.