Monday, October 24, 2011

Frictionless .NET Web App Development with Nancy Part III - Introducing MongoDB

Continuing the URL shortener sample from thesse post: part I and part II, I'll go from (stupidly) storing the shortened URLs in a static variable, as was the case in the code in part I, to storing the shortened URLs in MongoDB. The reasons for choosing MongoDB for persistence in this sample (compared to the "traditional" NHibernate + SQL Server approach) are:
  • We will only store and retrieve one document at a time. Those are atomic operations in Mongo, so as far as this sample goes Mongo is fully consistent.
  • Mongo is simple to setup, and very simple to code against. For instance there is no O/R mapping layer at all.

In summary, Mongo is a nice frictionless option for this sample.

Enough talk, show me the code!

Decoupling interface from persistence
The first thing to do is to pull the persistence related code out of the Nancy module. We use the following couple of tests to drive that. They use a Moq mock to specify the interface to the persistence code; that is the UrlStore interface:

    1 namespace ShortUrlTest
    2 {
    3   using Xunit;
    4   using Nancy.Testing;
    5   using Moq;
    6   using ShortUrl;
    7 
    8   public class UrlStorageTest
    9   {
   10     private Browser app;
   11     private Mock<UrlStore> fakeStorage;
   12 
   13     public UrlStorageTest()
   14     {
   15       ShortUrlModule artificiaReference;
   16 
   17       //Given
   18       fakeStorage = new Mock<UrlStore>();
   19       app = new Browser(
   20               new ConfigurableBootstrapper(
   21                 with => with.Dependency(fakeStorage.Object)));
   22 
   23     }
   24 
   25     [Fact]
   26     public void app_should_store_url_when_one_is_posted()
   27     {
   28       //When
   29       app.Post("/",
   30         with =>
   31         {
   32           with.FormValue("url", "http://www.longurlplease.com/");
   33           with.HttpRequest();
   34         });
   35 
   36       //Then
   37       fakeStorage
   38         .Verify(store =>
   39                 store.SaveUrl("http://www.longurlplease.com/", 
   40                               It.IsAny<string>()));
   41     }
   42 
   43     [Fact]
   44     public void app_should_try_to_retrieve_url_when_getting_a_short_url()
   45     {
   46       //When
   47       app.Get("/shorturl",
   48         with => with.HttpRequest());
   49 
   50       //Then
   51       fakeStorage
   52         .Verify(store => 
   53                 store.GetUrlFor("shorturl"));
   54     }
   55   }
   56 }
   57 

A few interesting things go on in these tests.

Firstly, as stated above they specifify the interface UrlStore. They do so via the mock fakeStorage, and the calls to fakeStorage.Verify(...) which tells us that UrlStore should have two methods, SaveUrl and GetUrlFor.


Secondly we again use the Nancy.Testing.Browser type to call into the ShortUrl Nancy module. Remember that this goes though the whole Nancy pipeline, just like a real request from a browser. So we're genuinely testing the module from the outside. In the first test we do a HTTP POST, and in the second one we do a HTTP GET.


Thirdly in the constructor we're using another feature of Nancy.Testing; the ConfigurableBootstrapper. To understand this bit, a little detour is required. Nancy is very modular, everything in Nancy can be swapped for an alternative implementation via configuration. Furthermore Nancy apps uses IoC/DI from the get go. Out of the box Nancy uses TinyIOC, but that can (of course) be swapped with any other container - and NuGets exists for most (if not all) popular containers. The swapping out of things in Nancy apps is done through code, in a bootstrapper, which is a class in the app that implements INancyBootstrapper, and sets up the whole of Nancy and the app. To make this easy Nancy provides the DefaultNancyBootstrapper which can be used as the base class for app specific bootstrappers. Back to tests above. The ConfigurableBootstrapper, provides a simple way for tests to create specialized configurations through a fluent interface. The contructor in the code above registers the mock UrlStore as a dependincy, which means registering it in the TinyIOC container. In other words the parts of a the app that receives a UrlStore from the container will get the mock one when in the context of these tests.

But the UrlStore interface doesn't exists, so lets write it to make the tests compile:

    1 namespace ShortUrl
    2 {
    3   public interface UrlStore
    4   {
    5     void SaveUrl(string url, string shortenedUrl);
    6     string GetUrlFor(string shortenedUrl);
    7   }
    8 }

The tests don't run yet though, since our app does not use UrlStore at all. To make it do so, we change the application code to take a UrlStore in the module contructor, and use it for storage (particularly lines 7, 14 and 22):

    1 namespace ShortUrl
    2 {
    3   using Nancy;
    4 
    5   public class ShortUrlModule : NancyModule
    6   {
    7     public ShortUrlModule(UrlStore urlStore)
    8     {
    9       Get["/"] = _ => View["index.html"];
   10       Post["/"] = _ => ShortenUrl(urlStore);
   11       Get["/{shorturl}"] = param =>
   12       {
   13         string shortUrl = param.shorturl;
   14         return Response.AsRedirect(urlStore.GetUrlFor(shortUrl.ToString()));
   15       };
   16     }
   17 
   18     private Response ShortenUrl(UrlStore urlStore)
   19     {
   20       string longUrl = Request.Form.url;
   21       var shortUrl = ShortenUrl(longUrl);
   22       urlStore.SaveUrl(longUrl, shortUrl);
   23 
   24       return View["shortened_url", new { Request.Headers.Host, ShortUrl = shortUrl }];
   25     }
   26 
   27     private string ShortenUrl(string longUrl)
   28     {
   29       return "a" + longUrl.GetHashCode();
   30     }
   31   }
   32 }
   33 

And that makes the above tests run. But the end-to-end tests shown in part I don't run. The reason is they don't register a UrlStore with the container. They use the DefaultNancyBootstrapper, where no UrlStore is setup. To get past this we need to do a few things: Change the end-to-end tests to use an app specific bootstrapper, create an implementation of UrlStore that uses Mongo, and create an app specific bootstrapper. We'll start by driving out the Mongo implementation of UrlStore.

Shifting gears
To implement UrlStore against Mongo we need to shift TDD gears to using integration tests. The following tests, that drive the implementation of MongoUrlStore assume a Mongo server is running on localhost at port 27010:

    1 namespace ShortUrlTest
    2 {
    3   using System.Linq;
    4   using MongoDB.Bson;
    5   using MongoDB.Driver;
    6   using MongoDB.Driver.Builders;
    7   using ShortUrl.DataAccess;
    8   using Xunit;
    9 
   10   public class MongoUrlStoreTest
   11   {
   12     private string connectionString = "mongodb://localhost:27010/short_url_test";
   13     private MongoDatabase database;
   14     private MongoCollection<BsonDocument> urlCollection;
   15 
   16     public MongoUrlStoreTest()
   17     {
   18       //given
   19       database = MongoDatabase.Create(connectionString);
   20       urlCollection = database.GetCollection("urls");
   21         urlCollection.RemoveAll();
   22     }
   23 
   24     [Fact]
   25     public void should_store_urls_in_mongo()
   26     {
   27       //when
   28       var store = new MongoUrlStore(connectionString);
   29       store.SaveUrl("http://somelongurl.com/", "http://shorturl/abc");
   30 
   31       //then
   32       var urlFromDB = urlCollection
   33         .Find(Query.EQ("url", "http://somelongurl.com/"))
   34         .FirstOrDefault();
   35 
   36       Assert.NotNull(urlFromDB);
   37       Assert.Equal(urlFromDB["shortenedUrl"], "http://shorturl/abc");
   38     }
   39 
   40     [Fact]
   41     public void should_be_able_to_find_shortened_urls()
   42     {
   43       //given
   44       var store = new MongoUrlStore(connectionString);
   45       store.SaveUrl("http://somelongurl.com/", "http://shorturl/abc");
   46 
   47       //when
   48       var longUrl = store.GetUrlFor("http://shorturl/abc");
   49 
   50       //then
   51       Assert.Equal("http://somelongurl.com/", longUrl);
   52     }
   53   }
   54 }
   55 

These two test cases simply use the UrlStore interface, but also go directly to the Mongo instance to check that things are stored there. This is all quite straight forward, the one thing to notice though is that these test only assume that there is a Mongo server running, not that there is any particular database or collection present from the start. This is one of the ways Mongo is easy to work with, the databases and collections used in the code are just created for us behind the scenes. No friction.
To satisfy these tests we simply implement the UrlStore interface as follows:

    1 namespace ShortUrl.DataAccess
    2 {
    3   using System.Linq;
    4   using MongoDB.Bson;
    5   using MongoDB.Driver;
    6   using MongoDB.Driver.Builders;
    7 
    8   public class MongoUrlStore : UrlStore
    9   {
   10     private MongoDatabase database;
   11     private MongoCollection<BsonDocument> urls;
   12 
   13     public MongoUrlStore(string connectionString)
   14     {
   15       database = MongoDatabase.Create(connectionString);
   16       urls = database.GetCollection("urls");
   17     }
   18 
   19     public void SaveUrl(string url, string shortenedUrl)
   20     {
   21       urls.Insert(new {url, shortenedUrl});
   22     }
   23 
   24     public string GetUrlFor(string shortenedUrl)
   25     {
   26       var urlDocument =  
   27         urls
   28           .Find(Query.EQ("shortenedUrl", shortenedUrl))
   29           .FirstOrDefault();
   30 
   31       return 
   32         urlDocument == null ? 
   33         null : urlDocument["url"].AsString;
   34     }
   35   }
   36 }

The thing I really want to point out here is line 21. We just tell Mongo to store an anonymous object and the driver does the rest for us (i.e. serializes it to json). No O/R mapping code!
Now we have an interface for persisting things, our module uses that interface, and we have an implemtation of it against Mongo. The only thing remaining is wiring things up.

Shifting gears back the end-to-end tests
As mentioned earlier the end-to-end tests from part I don't run right now, to make them run we first change their setup to use an app specific bootstrapper:

   14    public BaseUrlSpec()
   15    {
   16      ShortUrlModule artificiaReference;
   17      app = new Browser(new Bootstrapper());
   18    }

We've changed line 17 to use Bootstrapper instead of DefaultNancyBootstrapper. The type Bootstrapper does not exists yet, so lets create it, and have it register MongoUrlStore with the container:

    1 namespace ShortUrl
    2 {
    3   using Nancy;
    4   using DataAccess;
    5 
    6   public class Bootstrapper : DefaultNancyBootstrapper
    7   {
    8     protected override void ConfigureApplicationContainer(TinyIoC.TinyIoCContainer container)
    9     {
   10       base.ConfigureApplicationContainer(container);
   11 
   12       var mongoUrlStore = new MongoUrlStore("mongodb://localhost:27010/short_url");
   13       container.Register<UrlStore>(mongoUrlStore);
   14     }
   15   }
   16 }

We use the DefaultNancyBootstrapper as a starting point, and just use one its hooks to do a little bit of app specific setup, namely registering MongoUrlStore. Using other overrides we could configure anything in Nancy. I really like that a lot.

Well that's it. All tests run again.

Updated code is a available on GitHub.

Sunday, October 16, 2011

Frictionless .NET Web App Development with Nancy Part II - Introducing a View Engine

In the last post I introduced a litte sample web app coded up with the Nancy web framework. The code presented in that post left a few things open for later enhancement. This post takes a stab one of those things: The part where a string.Format is used to create a bit of html, which is then just returned directly to the client, namely the POST action in this code:

 3   using System.Collections.Generic;
 4   using Nancy;
 5 
 6   public class ShortUrlModule : NancyModule
 7   {
 8         private static readonly Dictionary&amp;lt;string, string&amp;gt; urlMap = new Dictionary&amp;lt;string, string&amp;gt;();
 9 
10         public ShortUrlModule()
11         {
12             Get["/"] = _ =&amp;gt; View["index.html"];
13             Post["/"] = _ =&amp;gt; ShortenUrl();
14         }
15 
16         private string ShortenUrl()
17         {
18             string longUrl = Request.Form.url;
19             var shortUrl = ShortenUrl(longUrl);
20             urlMap[shortUrl] = longUrl;
21 
22             return ShortenedUrlView(shortUrl);
23         }
24 
25         private string ShortenUrl(string longUrl)
26         {
27             return "a" + longUrl.GetHashCode();
28         }
29 
30         private string ShortenedUrlView(string shortUrl)
31         {
32             return string.Format("<a id=\"shorturl\" href=\"http://{0}/{1}\">;http://{0}/{1}</a>", Request.Headers.Host, shortUrl);
33         }
34     }

Specifically the last method, 'ShortenedUrlView' is bad code: It's brittle, it doesn't produce valid HTML, and it's open for script injection. Therefore I want to separate that bit of view code from the module code: I want a view engine. Luckily Nancy supports a number of standard view engines(Razor, Spark, DotLiquid, NDjango) each of which are only a NuGet package away, but it also comes with its own appropriately named Super Simple View Engine, SSVE. For this sample SSVE will do, so let's just stick with that.

We add a file called shortened_url.sshtml to the Views folder and put this into it:

<html>
  <body>
    <a id="shorturl" href="http://@Model.Host/@Model.ShortUrl">
      http://@Model.Host/@Model.ShortUrl
    </a>
  </body>
</html>

As with Razor views the @ indicates code snippets. In the above we insert 'Host' and 'ShortUrl' properties from the view model in the view.

To use this we also need to modify the Nancy module code a bit. We need to return a View[] instead of simply a string:

   21         private Response ShortenUrl()
   22         {
   23             string longUrl = Request.Form.url;
   24             var shortUrl = ShortenUrl(longUrl);
   25             urlMap[shortUrl] = longUrl;
   26 
   27       return View["shortened_url", 
   28         new { Host = Request.Headers.Host, ShortUrl = shortUrl }];
   29         }
   30 
   31         private string ShortenUrl(string longUrl)
   32         {
   33             return "a" + longUrl.GetHashCode();
   34         }
   35 

We saw use of View[] last time, this time around we provide a model object along with the view name. This object is passed to the view code as the @Model.

That's it. Introduced a view engine. No friction.

The code for this sample is still available on GitHub. Now with the updates described here.

Monday, October 10, 2011

Frictionless .NET Web App Development with Nancy


This post is about a low ceremony approach to web apps in .NET. It's about just writting the app, without the framework getting in the way. It's about the same sort of feeling that Mogens Heller Grabe talks about with his notion of frictionless persistence with MongoDB. Only here it's about the web part.

I'll take you through a short and simple URL shortener sample app done with the Nancy web framework. A couple of more posts will probably follow on the subjects of view engines, persistence and hosting for this app. Also with an eye on low ceromony and frictionsless development.

What is Nancy? 
Nancy is a lightweight open source web framework in .NET. It has a declared goal of being the developers super-duper-happy-path. Nancy is lightweight in the sense that the number of concepts and the amount syntax you have to grok is minimal. That's why I like it, and that's why it provides the frictionless experience I'm after here.

First things first: The first red-green cycle
One of the first very nice things to notice about Nancy is that it supports TDD really well (much more so than ASP.NET MVC), so lets start looking at the sample by looking at the first test:

    9 public class BaseUrlSpec
   10 {
   11   [Fact]
   12   public void should_respond_ok()
   13   {
   14     var app = new Browser(new DefaultNancyBootstrapper());
   15     var response  = app.Get("/", with => with.HttpRequest());
   16     var statusCode = response.StatusCode;
   17     Assert.Equal(HttpStatusCode.OK, statusCode);
   18   }
   19 }

This is just an ice breaker test to get us going: It just tests that our app responds with an http 200 OK status code. There are nevertheless a couple of things to notice:
  1. Nancy.Testing provides us with the Browser class, which gives our tests an easy way to interact with the app as if they were a browser: The browser object allows for doing GET, PUT, POST and DELETE, PATCH and OPTIONS against the app, including setting up the body, the headers and the protocol. In the case of the above we're doing a GET over http to the relative URL "/", no body, no special headers.
  2. Nancy.Testing also provides a BrowserResponse type that allows for testing all sorts of things on the response from the app. The variable 'response' in the above has this type. Here we just check the status code, but in later examples we'll do a lot more.
There, that's the red part.

To make this run we only have to write this little snippet of Nancy based code:

    4   using Nancy;
    5 
    6   public class ShortUrlModule : NancyModule
    7   {
    8     public ShortUrlModule()
    9     {
   10       Get["/"] =_ => HttpStatusCode.OK;
   11     }
   12   }

Firstly this little piece of code shows the central concept of Nancy: It provides a DSL for responding to http requests. It does so by letting you specify what code should run on GET, PUT, POST, DELETE, PATCH and OPTIONS requests for different URIs. In the above we only specify one such action for one URI: GET for '/'. Secondly it shows another defining trait of Nancy: The framework really tries to make things easy for you, in this case illustrated by the fact we just return a 200 OK status code, but Nancy accepts that and turns it into a full response for us.

GETting the form
So returning a 200 OK does not make for much of an app. What we want here is a very simple URL shortener, so lets turn the front page of the app into a form with a field for the URL the user wants shortened and a submit button. Again we start with a test:

   28     [Fact]
   29     public void should_contain_a_form_with_an_input_field_for_a_url_and_a_button()
   30     {
   31       //when
   32       var baseUrlGetResponse = app.Get("/", with => with.HttpRequest());
   33 
   34       //then
   35       baseUrlGetResponse.Body["form"]
   36         .ShouldExist();
   37 
   38       baseUrlGetResponse.Body["input#url"]
   39         .ShouldExistOnce();
   40 
   41       baseUrlGetResponse.Body["label"]
   42         .ShouldExistOnce().And
   43         .ShouldContain("Url: ");
   44 
   45       baseUrlGetResponse.Body["input#submit"]
   46         .ShouldExistOnce();
   47     }

Here we see some more of Nancys test emphasis: The BrowseResponse type mentioned above provides a Body property that lets our tests search through the body of the response using CSS selectors and make assertions on the results using a set of fluent 'Should' style extensions. For instance the above asserts that there is exactly one input tag with the id 'url' and that there is one label tag which furthermore contains the text 'Url: '. This, I must sat, is just awesome, and is a very good level to test at: Just below the pixels of the UI, and just above the real http stack of the real web server.

To make this run we need only do slightly more than in the previous cycle: We alter the application code to this:

 4   using Nancy;
 5 
 6   public class ShortUrlModule : NancyModule
 7   {
 8         public ShortUrlModule()
 9         {
10           Get["/"] = _ => View["index.html"];
11         }
12   }

and add the following index.html file to a Views folder:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
   <html>
     <body>
       <form method="post">
       <label>Url: </label>
      <input type="text" name="url" id="url"/>
      <input type="submit" value="shorten" id="submit"/>
    </form>
  </body>
</html>

Now we see Nancy trying to make things easy again: Just indicate in the application code that you want to return a view by saying View["index.html"], and Nancy looks in your views folders, finds the view and off we go.

POSTing the form
Next step is reacting to a POST of the form we just made, by shortening the URL. Let's set up a test for that:

 49     [Fact]
 50     public void should_return_shortened_url_when_posting_url()
 51     {
 52       //when
 53       var baseUrlPostResponse = app.Post("/",
 54         with =>
 55         {
 56           with.FormValue("url", "http://http://www.longurlplease.com/");
 57           with.HttpRequest();
 58         });
 59 
 60       baseUrlPostResponse.Body["a#shorturl"]
 61         .ShouldExist().And
 62         .ShouldContain("http://");
 63     }

Here we see how the app object is used for sending a POST to the app, and along with it the appropriate form values; in this case just the url. This starts to show the flexibility and ease of use of the Browser type.

Considering how we've handled the GET request until now, handling the POST is as straightforward:

 3   using System.Collections.Generic;
 4   using Nancy;
 5 
 6   public class ShortUrlModule : NancyModule
 7   {
 8         private static readonly Dictionary<string, string> urlMap = new Dictionary<string, string>();
 9 
10         public ShortUrlModule()
11         {
12             Get["/"] = _ => View["index.html"];
13             Post["/"] = _ => ShortenUrl();
14         }
15 
16         private string ShortenUrl()
17         {
18             string longUrl = Request.Form.url;
19             var shortUrl = ShortenUrl(longUrl);
20             urlMap[shortUrl] = longUrl;
21 
22             return ShortenedUrlView(shortUrl);
23         }
24 
25         private string ShortenUrl(string longUrl)
26         {
27             return "a" + longUrl.GetHashCode();
28         }
29 
30         private string ShortenedUrlView(string shortUrl)
31         {
32             return string.Format("<a id=\"shorturl\" href=\"http://{0}/{1}\">http://{0}/{1}</a>", Request.Headers.Host, shortUrl);
33         }
34     }

This is all pretty straight forward. The Nancy parts of this are:
  1. The use of POST...which does just what you expect.
  2. The return of a string in the last method. That bubles all the way up, and becomes the repsonse back to Nancy. Again Nancy is nice to us, sets the content type to text/html and the status code to 200 OK. This is not the nicest code on my part, so I might just return to refactor it in a future post.
The above of course is not very clever about the shortening, and does not exactly persist the short URLs in a good permanent place. I'll return to the persistence in a furture post.

Redirect
Only one part is missing for a simplistic but functioning URL shortener: The app must redirect the shortened URLs to the original longer one.
Again lets start with the test:

 65     [Fact]
 66     public void should_redirect_to_original_url_when_getting_short_url()
 67     {
 68       //when
 69       var baseUrlPostResponse = app.Post("/",
 70         with =>
 71         {
 72           with.FormValue("url", "http://www.longurlplease.com/");
 73           with.HttpRequest();
 74         }).GetBodyAsXml();
 75 
 76       var shortUrl = baseUrlPostResponse
 77         .Element("a")
 78         .Attribute("href").Value
 79         .Split('/')
 80         .Last();
 81 
 82       //then
 83       app.Get("/" + shortUrl, with => with.HttpRequest())
 84         .ShouldHaveRedirectedTo("http://www.longurlplease.com/");
 85     }

We do the POST again, find the short URL with a bit of LINQ-to-xml and then do a GET to that. And then just assert that the redirect happened. Pretty straight forward except maybe for the Linq-to-xml part.

To make the last test run we add this to our application code:

 14  Get["/{shorturl}"] = param =>
 15  {
 16     string shortUrl = param.shorturl;
 17     return Response.AsRedirect(urlMap[shortUrl.ToString()]);
 18  };


which is an action for a whole set of URIs, namely the URIs matching the URI template "/{shorturl}". In the action we access the 'shorturl' part of the URI through the param.shorturl; all parts of the templatized URI will be availbe like that through the dynamic param. Nice.

That's it. We have a simplistic URL shortener. No friction.

The source is availble on github, and more Nancy information is avaible on the Nancy web site.