Sunday, February 12, 2017

Event-based Collaboration does not imply Event Sourcing


In my microservices book I talk about three styles of collaboration between microservices: Command, Query and Event based collaboration. On the book's forum I recently got a question that boils down to "if a microservice allows event-based collaboration, can all it's state always be recreated from those events?". I'm taking the liberty of re-framing this question as "if a microservice allows event-based collaboration, does it have to use Event Sourcing?" My answer is: No - you can, but you don't have to.
I will try to explain why below.

Event Based Collaboration

When we use a microservice architecture for a system, that system gets broken into lots - usually hundreds - of small narrowly focused services each of which handle one specific capability. In order to deliver a cohesive experience to the users using the applications built on top of all those microservices we have to make them collaborate - the alternative would be to expose the end user to that very fine grained break down into single capability services. That would make for an insane user experience.

One of the ways microservices can collaborate is through events: When something significant happens in a microservice it can choose to publish an event that other microservices can then react to however they need/wish to.

This is a powerful style of collaboration. Events allow for asynchronuous processing, for slow subscriberss to catch up with bursts at their own pace, for some microservices to be down for shorter periods and more. For these reasons event-based collaboration between microservices is quite often a better choice than command- or query-based collaboration. The point in this context, though, is more about what the events are. These events are things that are significant outside of the microservice publishing them. They are published in order to drive collaboration with other microserivces.

Internal Event and External Events

The events published to other microservices in order to drive collaboration are events that can - and should - be published regardless of how the publishing microservice is implemented. Referring to the figure above: The blue microservice publishes events to drive collaboration with other microservices, like the green and yellow ones. That has nothing to do with the inner workings of the blue microservice. The events published to other microservices are external events.
The blue microservice from the figure above can store its state however it wants. One of the ways it can chose to store its state is using Event Sourcing. Using Event Sourcing introduces another set of events: Ones stored internally in the microservice. These events to do not drive collaboration with other microservices, but they capture every little state change in the blue microservice.

I find it helpful to distinguish between these two types of events:

  • External events drive collaboration with other microservices. These are also sometimes referred to as integration events.
  • Internal events captures all state changes within the microservices and is the basis for event sourcing. These are also sometimes referred to as domain events.
Internal events are usually far more granular than external events. Internal events are also tightly coupled to the implementation details of the microservice, whereas external events are not.


Having established the difference between internal and external events it should be clear that one does not imply the other. A microservice can publish external events without using Event Sourcing. Likewise a microservice can use Event Sourcing without publishing any external events.

Sunday, January 29, 2017

"Microservices in .NET Core" has landed

My new book "Microservices in .NET Core  - with examples in Nancy" has been published. It's been available as an early access ebook for a long time, but now the final paper book and the final ebook is available.

This book is very much based on experience from working of real microservices systems. While the examples in the book are not lifted directly from any particular system they are inspired by real world systems. My rule has been that all the examples demonstrate something that could have been in one or more of the systems I have experience with.

With this book I have tried to show two things:

    1. How to design a microservices system.
    2. How to build such a system using a lightweight .NET based tech stack.

All but one chapter reflect these two levels, in that they start off explaining an aspect of designing a microservice system, and then go on to show how to implement it using my .NET tech stack of choice. This should help make clear which parts make sense regardless of tech stack and which are tech specific. At the end of the book  you should have the tools to both design a microservice system and implement it, as we have covered subjects like how to decide what belongs in which microservice, where data resides, how microservices collaborate, how to test such a system, how to make it robust and how gain insights into how it is doing in production.

The tech stack I use in the book is based on .NET Core, Microsoft's new cross platform version of .NET. On top of .NET Core I use the Nancy web framework and OWIN middleware, which forms a powerful yet simple stack. To compliment this I use various other OSS libraries where appropriate, like Polly and Dapper. The guiding principle in all the tech choices is that they should be simple to use - make the simple stuff simple, while also allowing to do more complex stuff without unwarranted hassle.

You can find the table of contents and a couple of sample chapter on the books page at Manning
Also, if you are interested in these things, I have a 3 day course around these things available.

Monday, November 7, 2016

Sharing and Caching Assets Across Single-Page BFFs

Over the last couple of posts I showed the single-page BFFs pattern (a variation of the Backend-For-Frontend aka BFF pattern where the BFF serves only one page in a web application), and how you will likely need to place single-page BFFs behind a reverse proxy.

In this post I will outline an approach to handling shared assets - like CSS or Javascript which is used on all or most of the pages in a web application built using a number of single-page BFFs.

When building a web application consisting of a number of pages, there is likely both some look-and-feel and some behavior that we want to share across all pages - buttons should look and behave the same, menus that should be on all pages, colors and fonts should be consistent etc. This calls for sharing some CSS and some Javascript across the pages. When thinking about handling these shared things across a number of single-page BFFs there are some competing requirements to take into consideration:

  • Clients - i.e. browsers - should be able to cache the CSS and Javascript as the user moves through the web application. The alternative is that the client downloads the same CSS and JS over and over as the user navigates the web application. That would be too wasteful in most cases.
  • Each single-page BFF should be individually deployable. That is we should be able deploy a new version of one single-page BFF without having to deploy any other services. If not the system soon becomes unmanageable. Furthermore developers should be able to run and test each single-page BFF by itself without having to spin up anything else.
  • I do not want to have to write the same CSS and JS in each single-page BFF. I want to share it.

In this post I will show an approach that allows me to achieve all three of the above.

First notice that the obvious solution to the first bullet - allowing clients to cache shared CSS and JS - is to put the shared CSS and the shared JS in two bundles - a CSS bundle and a JS bundle - and put the bundles at a known place. That way each single-page BFF can just address the bundle at the known place, as shown below. Since all pages refer to the same bundles the client can cache them.

The problem with this is that it violates bullet 3: Any given single-page BFF only works when both the global bundles are available, so for a developer to work on a single-page BFF, they need not only the BFF, but also something to serve the bundles. If we are not careful, this approach can also violate bullet 2: Since the single-page BFFs depend on the two global bundles, but they do not contain the global bundles themselves, each one has a hard dependency on another service - the one that serves the bundles. That means we can easily end up having to deploy the service that serves the bundles at the same time as a new version of one of the single-page BFFs.

Looking instead a bullet three first suggests that we should put the shared CSS and Javascript into a package - an NPM package, that is - and include that package in each single-page BFF. That means each single-page BFF has the bundles it needs, so developers can work with each single-page BFF in isolation. This also means that each single-page BFF continues to be individually deployable: When deployed they bring along the bundles they need, so there is no dependency on another service to serve the bundles. The problem now becomes how to allow clients to cache the bundles? When the bundle are included in each single-page BFF, each page will use different URsL for the bundles - completely defeating HTTP cache header.

Luckily, as discussed in the last post, the single-page BFFs are already behind a reverse proxy. If we put a bit of smarts into the reverse proxy we can include the bundles in each single-page BFF and still let every page use same the URLs for the bundles, like so:

The trick implementing this is to have the reverse proxy look at the referrer of any request for one of the bundles - if the referrer is frontpage, the request is proxied to the frontpage single-page BFF, if the referrer is the my account page, the request is proxied to the MyAccount single-page BFF. That means that the bundles loaded on the frontpage is the bundles deployed in the frontpage single page BFF. Likewise the bundles loaded on the my account page come from the MyAccount single-page BFF. But from the client perspective the bundles are on the same URL - i.e. the client is able to cache the bundles based on HTTP cache headers.

To sum up: Put shared CSS and JS bundles in an NPM package that all the single-page BFFs use, and making the reverse proxy use the referrer on requests for the bundles to decide where to proxy requests to allows clients the cache the bundles, the single-page BFFs to be individually deployable and developers work with each single-page BFF in isolation.