Wednesday, November 22, 2017

Event Sourced Aggregates Part 5: Anemic aggregate

In this 5th part in my series on event sourced aggregates I continue moving code out of the aggregate. In the last post I moved domain logic out of the aggregate and into the command handlers, making them smart enough to actually handle commands by themselves. In this post I will continue along the path of moving stuff out of the aggregate: I will move the remaining methods, namely the 'When' methods, out from the aggregate to a new abstraction - an aggregate projector. While this will achieve the goal set out in the first post of getting past the aggregate growing and growing over time, the new aggregate projector unfortunately will suffer from the same problem. In the next post I will take another approach moving the 'When' methods out and arrive at a better design, but first let's follow what I think is the most obvious path and see why that leads a bad design.

The aggregate is a projection

Taking a step back, what is the aggregate? At the datastore level it is a list of events. Together the events represent everything that has happened to the aggregate and, as we have seen, the current state of the aggregate can be recreated from the list of events. At the code level the aggregate is an object - it has some state and it has some behavior. At this point the only behavior it has left is the 'When' methods. The important bit is that the aggregate is an object. It's just an object. Likewise, in the code, different read models are just objects that result from projections over events. In that sense the aggregate is not different from a read model: It is an object that is the result of a projection over events.

Introducing the aggregate projector

Before I start refactoring lets take a look at how the aggregate looks right now:



The aggregate has some state represented by the properties on lines 3 and 4, and then some 'When' methods that make up the logic needed to perform the projection from the aggregates events to its current state.

Seeing that a new 'When' method will be added to the aggregate every time a new event is introduced - and new features will usually result in new events, in my experience - the aggregate still has the problem of growing big and unwieldy over time. So let's introduce another class that can do the projections:



This doesn't just work. First off the new 'UserAggregateProjector' cannot set the properties on the aggregate to anything. That can be fixed by adding internal setters to the aggregate, allowing the projector to access the setters, but disallowing access from outside the same project as the 'UserAggregate', which I expect to mean anything beyond commands, command handlers and events.
Furthermore the event replay done when fetching an aggregate must also change from calling 'When' methods on the aggregate to calling them on the 'UserAggregateProjector'. That means changing 'Aggregate' base class to this:



The changes are the introduction of the 'GetProjector' method on line 30 and the use of that new method in the 'Play' method, which now does reflection of the projector class to find the 'When' methods instead of doing it over the aggregate. The end result is the same: An aggregate object with the current state of the aggregate recreated by replaying all events.

Moving the 'When' methods has obviously also changed the aggregate, which now only contains state:



This is what is known as an anemic domain model, because it has no behavior. That's usually considered an anti-pattern, but I don't necessarily agree that it is; as argued above the aggregate is essentially a projection of the events, so I do not see why that object has to be where the domain behavior goes. As we saw in the 4th post of the series command handlers is a nice place to put domain behavior.

The projector violates Open/Closed principle


As a stated at the beginning of this post the design I've arrived at now is not good: The new 'UserAggregateProjector' suffers just as much from perpetual growth as the aggregate did before I moved the 'When' methods out of it. In other words the new projector violates the Open/Closed principle, which is what I am trying to get away from. So I have not solved anything, just moved the problem to a new abstraction :( Seems like I need to take another iteration, which I will in the next post.

The code for this post is in this branch.

Tuesday, November 14, 2017

Event Sourced Aggregates Part 4: Smart command handlers

In this fourth post in my series about event sourced aggregates I will continue work on the command handlers. In the 3rd post I did some cleaning up of the command handlers, cutting them down to very little code. In this post I will make the command handlers smart by moving domain logic from the aggregate to the command handlers.

Motivation

In the first and second posts of the series I outlined a typical C# implementation of event sourced aggregates and how that style of implementation leads to ever growing aggregates - every added features adds (at least) two methods to the aggregate: 
  • One for the domain logic. E.g. a 'ChangeUsername' method, that has whatever business logic there is around changing the username. If and when these methods decide a change to the aggregate state is needed they emit a domain event.
  • A 'When' method for any new events. The `When` methods perform all state changes on the aggregate.
The patterns I see in the implementations I come across is that there is a one-to-one correspondence between commands, command handlers and public domain logic methods on the aggregate. For example the pattern is that for a 'ChangeUsernameCommand' there is a 'ChangeUsernameCommandHandler' class and a 'ChangeUsername' method on the aggregate. I took almost all the plumbing out of the command handler in the last post and essentially left it at this:


which invokes the 'helper.Handle' method to get all the plumbing done and then calls the 'ChangeUsername' method to get the domain logic done. So in essence the command handler just delegates to the aggregate, but isn't the responsibility of the command handler to ... handle the command? I think it is. And handling the command means running the domain logic, so let's move that logic from the aggregate to the command handler.

Smart command handlers

In the second post I introduced the 'ChangeUsernameCommand` and the associated command handler and methods on the aggregate. In particular this `ChangeUsername` method on the aggregate:



which implements the domain logic for changing username. That is the logic I want to move to the command handler.
Moving the domain logic straight over the command handler, changes the `Handle` method on the command handler to this:



Now the command handler contains the logic for handling the command. Note that now the command handler also emits domain events - on line 7. This makes sense since this is still event sourced, so changes to the aggregate state are still done through events. The rest of the mechanics around events remain unchanged: The `Emit` method on the base aggregate still calls the 'When' method for the event and stores the event to the list of new events on the aggregate. Saving the aggregate still means appending the list of new events to the event store, and getting the aggregate from the 'AggregateRepository' still means reading all the aggregates events from the event store and replaying each one.

Having moved the domain logic out of the aggregate I have a slimmer aggregate, that only has the state of the aggregate and the 'When' methods. In the next two post I slim down the aggregate even further by moving the 'When' methods out.

The complete code for this post in this branch.

Monday, November 6, 2017

Event Sourced Aggregates Part 3: Clean up command handlers

This is the 3rd part of my series about event sourced aggregates. In the first post I outlined a typical implementation of event sourced aggregates, and in the second post I showed how that implementation leads to aggregates that grow bigger and bigger as features are added to the system. In this post I will clean up the command handlers from the previous post by moving some repeated code out them to a new abstraction. In the coming posts I will refactor between the command handlers, aggregates and event handlers to arrive at a design where the aggregate does not grow.

Repeated code in command handlers

In the last post we looked briefly at this implementation of a command handler:


Looking at the above the code at line 20 is - in a sense - where the ChangeUsernameCommand the is handled, because that is the only line that is about changing a username. All the other code in the command handler is about infrastructure; loading the aggregate, saving the aggregate and dispatching events. Moreover the code for loading and saving aggregates as well as the code for dispatching will be repeated in every command handler

Introducing a helper

To get past that repetitiveness and to cut back on the amount of infrastructure code in the command handler, we introduce this helper, where the loading of the aggregate, the saving of the aggregate and the dispatching of events is done:


The idea behind the CommandHandlerHelper is that the concrete command handler calls the Handle method with a handlerFunc, that does the business logic bit of the command handler. The handlerFunc is called at line 18, so the helper makes sure the infrastructure code is done in right order in relation to the business logic.

Cleaner command handlers

With the CommandHandlerHelper in place the ChangeUsernameCommand can be rewritten to use it like this:

This is a good deal simpler than the command handler code at the start of the post.


That's it for now. With this clean up in place we set for the next steps:

Tuesday, October 31, 2017

Event Sourced Aggregates Part 2: Where the mess starts

In the first post in this series I outlined a typical C# implmentation of  event sourced aggregates. In this post we add a second feature to the example from the first post. In doing so I wish to illustrate how that typical implmentation leads to violation of Open/Closed principle and ever growing aggregates.

The second feature

Once we have the code from the first post in place - that is: The infrastructure for sending commands to aggregates, raising events from aggregates, saving events and replaying events - and need to add a second feature, the way to do it is (to some extent) as outline in the first post: 
  1. Send a new type of command
  2. Implement a command handler for the new command
  3. Implement a new method on the aggregate with the new domain logic
  4. Emit any new events needed
  5. Implement new When methods for any new event types
Let's say we want to be able change a users username.
The command and the sending of of that command looks like this:



That's pretty straight forward and not too interesting, so let's move on to the command handler:



That's also pretty straight forward, but a little more interesting: Most of the the command handler code is something that will be repeated in all commands handlers. We will deal with this in the next post, where that repetition is pulled out into a helper class.
The next step is changes to the aggregate, where this is added:



This is still pretty straight forward. In fact everything needed to add this new feature was straight forward, so that is a good thing. The problem lies in the last two methods, the ones added to the aggregate.

Problem: An Open/Closed Principle violation

The aggregate just grew. In order to support changing the username we added 2 methods to the aggregate. That violates the Open/Closed principle, which indicates that it is a potential problem. In my experience, it quickly becomes a real problem because the aggregate grows relatively quickly and eventually becomes big and complicated, just like any other class that grows monotonically.

That's it for now. The next posts will:
  1. Make the command handlers smarter and clean up some repetitiveness
  2. Make the aggregate anemic in a naive way, leaving a stable aggregate, but introducing a new Open/Closed violation
  3. Make the aggregate anemic, events (a little) smart, and solve the Open/Closed violation

Tuesday, October 24, 2017

Event Sourced Aggregates Part 1: Outline of a typical implementation

This is the first post in a series of posts that takes its offset in a design problem I've encountered repeatedly with event sourced aggregates: They grow every time a feature is added. Nothing (almost) is removed from them, so over time they grow very big and gnarly. Why does this happen? Because typical implementations of event sourced aggregates violate the Open/Closed principle.
Through this series of post, I will show how event sourced aggregates violate Open/Closed and - as a consequence - tend to grow monotonically, and then show how we can address that by re-factoring away from the big aggregate towards a small and manageable one. Cliffhanger and spoiler: The aggregate will become anemic and I think that is a good thing.

The complete code from the series is on GitHub, where there is a branch with the code for the first two posts.

Event sourced aggregates

The context of what I am exploring in this series of posts is systems based on Domain Driven Design, where some or all of the aggregates are stored using event sourcing. Often these systems also use CQRS - very much inspired by Greg Youngs talks and writings.
Using event sourcing for storing the aggregates, means that the aggregate code does not change the state of the aggregate directly, instead it emits an event. The event is applied to the aggregate which is where changes to the state of the aggregate happens, but the event is also stored to a data store. Since aggregate state is only changed when an event is applied, the current aggregate state can be recreated by reading all the events for a given aggregate up form the data store and applying each one to the aggregate. The benefits of this approach are many (when applied to a suitable domain) and described elsewhere, so I wont go into them here.

A typical C# implementation

Typically implementations of all this follow a structure where requests coming in from the outside - be it though a client making a request to an HTTP endpoint, a message on a queue from some other service or something else - result in a chain that goes like this:
  1. A command is sent, asking the domain to perform a certain task 
  2. A command handler picks up that command, fetches the appropriate aggregate and triggers appropriate domain behavior. Fetching the aggregate involves replaying all the aggregates events (event sourcing!)
  3. A method on an aggregate is called, and that is where the actual domain logic is implemented
  4. Whenever the domain logic needs to change some state it emits an event (event sourcing, again) of a specific type
  5. A 'when' method for the specific event type on the aggregate is called and updates the state of the aggregate
  6. After the aggregate is done, all the events emitted during execution of the domain logic is dispatched to any number of event handlers that cause side effects like updating view models, or sending messages to other services.
To put this whole process into code, let's think of an example: Creating a user in some imaginary system. The first step is to send the create user command:


Next step is the event handler for the create user command. Note that in this example I use the MediatR library to connect the sending of a command to the handler for the command.


Note that most of what is going on here is the same as for other handlers for other commands: Pick the right aggregate and, after executing domain logic, save that aggregate and then dispatch whatever events were emitted.

On line 19 of the handler we call into the aggregate. The code in the aggregate looks like this:


At line 11 we call the Emit method. This is how most implementations I've seen work, and typically that Emit method is part of the Aggregate base class and looks something like this:


Notice how Emit calls Play which uses reflection to find a When method on the aggregate itself, and to call that When method. The When method is supposed to update the state of the aggregate and is also the method that gets called during event replay. More on that below. For now let's see the when method:


That's pretty much it, though there a few things I have skipped over a bit quickly: How the aggregate is fetched, how it is saved and how events are dispatched. I will not go into the event dispatching, since it is not relevant to the point I am making in this series, but the code is on Github, if you want to look. As for the other two bits - fetching and saving aggregates - lets start with how aggregates are saved:


As you can see saving the aggregate essentially means saving a list of events. The list should contain all the events that has ever been emitted by the aggregate. That is the essence of event sourcing. When it comes to fetching the aggregate, the list of events is read, and each one is replayed on a new clean aggregate object - that is the When methods for each event is called in turn. Since only the When methods update the state of the aggregate the result is an aggregate object in the right state. The Get method on the aggregate repository (which does the fetching) looks like this:


And the Replay method called in line 14 just runs through the list of events and plays each on them in turn, like this:


That pretty much outline the implementations of event sourced aggregates I seem to come across. 

That's it for now. The next posts will:

  1. Add a second feature and see how the aggregate starts to violate Open/Closed principle
  2. Make the command handlers smarter and clean up some repetitiveness
  3. Make the aggregate anemic in a naive way, leaving a stable aggregate, but introducing a new Open/Closed violation
  4. Make the aggregate anemic, events (a little) smart, and solve the Open/Closed violation

Tuesday, August 29, 2017

Free microservices ebook

In the early summer I helped Manning put together a short, free ebook about microservices. It's made up of chapters from other Manning books and as such it doubles as a sampler or appetizer for those books and as a short quick intro to some important topics relating to microservices.

You can get from here.

The books the chapters are from are:

Enjoy!


Tuesday, March 28, 2017

Talk video: "Consolidating Services with middleware" from NDC London 2017

Back in January I did a talk called at NDC London called "Consolidating Services with middleware", and the video of the talk has come out. Enjoy!



Consolidating Services with middleware - Christian Horsdal from NDC Conferences on Vimeo.

In case you are wondering what the talk is about, here is the abstract:
"Have many services? Writing new ones often? If so middleware can help you cut down on the ceremony for writting new services and at same time consolidate the handling of cross cutting concerns. But what is middleware? OWIN and ASP.NET Core both have a concept of middleware. What are they? How do they help? In this talk we will dive into the code, write some middleware and show how middleware helps you handle cross-cutting concerns in an isolated and re-usable way across your services. I'll compare and contrast the OWIN and ASP.NET Core middleware concepts and talk about where each is appropriate."

Sunday, February 12, 2017

Event-based Collaboration does not imply Event Sourcing

Abstract

In my microservices book I talk about three styles of collaboration between microservices: Command, Query and Event based collaboration. On the book's forum I recently got a question that boils down to "if a microservice allows event-based collaboration, can all it's state always be recreated from those events?". I'm taking the liberty of re-framing this question as "if a microservice allows event-based collaboration, does it have to use Event Sourcing?" My answer is: No - you can, but you don't have to.
I will try to explain why below.

Event Based Collaboration

When we use a microservice architecture for a system, that system gets broken into lots - usually hundreds - of small narrowly focused services each of which handle one specific capability. In order to deliver a cohesive experience to the users using the applications built on top of all those microservices we have to make them collaborate - the alternative would be to expose the end user to that very fine grained break down into single capability services. That would make for an insane user experience.

One of the ways microservices can collaborate is through events: When something significant happens in a microservice it can choose to publish an event that other microservices can then react to however they need/wish to.


This is a powerful style of collaboration. Events allow for asynchronuous processing, for slow subscriberss to catch up with bursts at their own pace, for some microservices to be down for shorter periods and more. For these reasons event-based collaboration between microservices is quite often a better choice than command- or query-based collaboration. The point in this context, though, is more about what the events are. These events are things that are significant outside of the microservice publishing them. They are published in order to drive collaboration with other microserivces.

Internal Event and External Events

The events published to other microservices in order to drive collaboration are events that can - and should - be published regardless of how the publishing microservice is implemented. Referring to the figure above: The blue microservice publishes events to drive collaboration with other microservices, like the green and yellow ones. That has nothing to do with the inner workings of the blue microservice. The events published to other microservices are external events.
The blue microservice from the figure above can store its state however it wants. One of the ways it can chose to store its state is using Event Sourcing. Using Event Sourcing introduces another set of events: Ones stored internally in the microservice. These events to do not drive collaboration with other microservices, but they capture every little state change in the blue microservice.


I find it helpful to distinguish between these two types of events:

  • External events drive collaboration with other microservices. These are also sometimes referred to as integration events.
  • Internal events captures all state changes within the microservices and is the basis for event sourcing. These are also sometimes referred to as domain events.
Internal events are usually far more granular than external events. Internal events are also tightly coupled to the implementation details of the microservice, whereas external events are not.

Conclusion

Having established the difference between internal and external events it should be clear that one does not imply the other. A microservice can publish external events without using Event Sourcing. Likewise a microservice can use Event Sourcing without publishing any external events.

Sunday, January 29, 2017

"Microservices in .NET Core" has landed

My new book "Microservices in .NET Core  - with examples in Nancy" has been published. It's been available as an early access ebook for a long time, but now the final paper book and the final ebook is available.


This book is very much based on experience from working of real microservices systems. While the examples in the book are not lifted directly from any particular system they are inspired by real world systems. My rule has been that all the examples demonstrate something that could have been in one or more of the systems I have experience with.

With this book I have tried to show two things:

    1. How to design a microservices system.
    2. How to build such a system using a lightweight .NET based tech stack.

All but one chapter reflect these two levels, in that they start off explaining an aspect of designing a microservice system, and then go on to show how to implement it using my .NET tech stack of choice. This should help make clear which parts make sense regardless of tech stack and which are tech specific. At the end of the book  you should have the tools to both design a microservice system and implement it, as we have covered subjects like how to decide what belongs in which microservice, where data resides, how microservices collaborate, how to test such a system, how to make it robust and how gain insights into how it is doing in production.

The tech stack I use in the book is based on .NET Core, Microsoft's new cross platform version of .NET. On top of .NET Core I use the Nancy web framework and OWIN middleware, which forms a powerful yet simple stack. To compliment this I use various other OSS libraries where appropriate, like Polly and Dapper. The guiding principle in all the tech choices is that they should be simple to use - make the simple stuff simple, while also allowing to do more complex stuff without unwarranted hassle.

You can find the table of contents and a couple of sample chapter on the books page at Manning
Also, if you are interested in these things, I have a 3 day course around these things available.