My talk from this years NDC Oslo isnow available on youtube, so in case you missed it: Enjoy.
Friday, December 16, 2022
Effective Testing in Microservice Systems talk from NDC Oslo
Friday, November 5, 2021
Recording of stream about collaboration between microservices
The other I was on the Manning Twitch talking about collaboration between microservices. You can see recording of stream below. I was there on occasion of the publication of the second edition of my Microservices book, which only just landed in hard copy a few days before. Much of the material in this talk appears in further detail in the book, though I did change it up a bit for the talk.
Hope you enjoy it!
Wednesday, February 10, 2021
Interview on Adventures in .NET
I recently had the pleasure of chatting with the folks on the Adventures in .NET podcast and now the episode is out. We talk about some of the things I consider essential to get right to have success with microservices - like getting the responsibilities right, having good observability, good automation and more. We also talked about when to use microservices and when not to. After all, one does not simply microservice. Anyway, the episode is out and you can find it here.
Tuesday, August 4, 2020
Interview about Microservices on the Cynical Developer Pordcast
Wednesday, January 8, 2020
"Lessons learned working remotely across timezones" video from BuildStuff 2019
Friday, October 11, 2019
Identifying and Scoping Microservices talk from DevConfPL
Friday, September 20, 2019
A pattern for synchronizing data from a legacy system to microservices
Abstract
A recurring need in the teams I work with seems to be to move and continually synchronize data from a legacy system - typically a monolith - to a new microservice based replacement. In this post I will outline a data synchronization solution that allows- The microservice side to catch up on historical data
- Synchronizes new data to the microservices as it is produced in the legacy system
- Lets us control the speed of synchronization and as a result the load on the legacy system
- Is repeatable, so the same data can easily be synchronized again if need be
Microservices need legacy data
When replacing an existing legacy system with a modern microservice based system teams (sensibly) tend to do so in steps following the strangler pattern. This means that for a while both systems run in production. Some functionality is handled in the legacy system and some in the microservices. At first very little is handled in the microservices but gradually more and more functionality is handled there. To support those small early steps implemented in the microservices data from the legacy side is often needed. For instance, in a back office system a team in the process of moving to a microservice architecture they might want to implement a new sales dashboard with microservices. To do so we will need order data and possibly other data too, but let's just focus on the order data for now. Orders are still being taken in on the legacy side, so order data is being produced on the legacy side. But the new sales dashboard needs both historical orders and new orders to work correctly.To make things more interesting let's say the legacy system and the microservices are in different data center - maybe the system is moving from on prem to the cloud as part of the microservice effort.
Solution: A data pump
A solution to the situation above is to implement a data pump that sends any updates to relevant data in the legacy database over to the microservices. In the example that means new orders as well as changes to orders.This solution has two components: A data pump which is deployed in the legacy environment and a data sink which is deployed in the microservices environment. The data pump tracks which data has already been sent over to the microservices and sends new data over as it is produced in the legacy system. The data sink simply receives the data from the pump and posts it onto a queue. This enables any and all microservices interested in the data - e.g. new or updated orders - to subscribe to such messages on the queue and to build up their models of that data in their own databases.
With the described solution in place any historical data can be sent over to the microservices. That may take a a while, especially if the legacy database cannot take to much additional load. In such cases the data pump can be throttled. Once the pump has caught up sending over historical data it will continue to send over new data in near real time.
If we put a bit of up front design into the data pump we can also support restarting the process - the pump tracks what it has already sent, resetting that tracking will make it start over. That's sometimes useful if we e.g. don't get the receiving mciroservice right in the first attempt, or if we introduce new microservices that also need data.
This is a solution I have seen used with success in several of my client's systems and that I think is applicable in many more systems too.
Monday, June 17, 2019
InMemoryLogger available on NuGet
If you follow arrange/act/assert in your tests the idea is to create the in memory logger in the arrange part either by directly newing it or adding it to the application under tests IServiceCollection. Then do whatever to the application under test in the act part, and finally inspect the properties that expose recorded logs on the InMemoryLogger in the assert part.
For more info check out the readme or grab InMemoryLogger from NuGet.
Wednesday, April 3, 2019
Interview on the .NET Core Show
Saturday, February 16, 2019
Building a self-contained .NET Core app in a container
- Build a self-contained .NET Core app.
- Not install the .NET Core SDK on the CI server
- Fit this into a build pipeline that was already containerized.
- First build a container using docker build
- The run the container with a local folder called "output" as a volume
This is a Dockerfile that allows this:
And the two commands needed to build and run the container are:
That builds the self-contained app into .\output\release\netcoreapp2.2\linux-x64\publish\
For context: In my case I needed to build DbUp as part of the build pipeline for a service that I run in a container. I want DbUp to be self-contained so I can run it during the deployment pipeline without needing to install the .NET Core runtime.
Sunday, May 6, 2018
Talk: Lightweight Microservice Collaboration using HTTP
Saturday, January 13, 2018
Event Sourced Aggregates Part 6: Smarter Events
In the 5th post I made an attempt at moving the 'When' methods out of the aggregate to get to a design where the aggregate does not violate Open/Closed. I did so by introducing a new abstraction - an aggregate projector - but that just ended up with the same Open/Closed violation that the aggregate suffered from originally. Therefore I take another approach in this post.
Smarter events
And the 'UsernameChangedEvent' looks like this:
Event replay revisited
The anemic aggregate
Wrapping up
Wednesday, November 22, 2017
Event Sourced Aggregates Part 5: Anemic aggregate
The aggregate is a projection
Taking a step back, what is the aggregate? At the datastore level it is a list of events. Together the events represent everything that has happened to the aggregate and, as we have seen, the current state of the aggregate can be recreated from the list of events. At the code level the aggregate is an object - it has some state and it has some behavior. At this point the only behavior it has left is the 'When' methods. The important bit is that the aggregate is an object. It's just an object. Likewise, in the code, different read models are just objects that result from projections over events. In that sense the aggregate is not different from a read model: It is an object that is the result of a projection over events.Introducing the aggregate projector
Before I start refactoring lets take a look at how the aggregate looks right now:The aggregate has some state represented by the properties on lines 3 and 4, and then some 'When' methods that make up the logic needed to perform the projection from the aggregates events to its current state.
Seeing that a new 'When' method will be added to the aggregate every time a new event is introduced - and new features will usually result in new events, in my experience - the aggregate still has the problem of growing big and unwieldy over time. So let's introduce another class that can do the projections:
This doesn't just work. First off the new 'UserAggregateProjector' cannot set the properties on the aggregate to anything. That can be fixed by adding internal setters to the aggregate, allowing the projector to access the setters, but disallowing access from outside the same project as the 'UserAggregate', which I expect to mean anything beyond commands, command handlers and events.
Furthermore the event replay done when fetching an aggregate must also change from calling 'When' methods on the aggregate to calling them on the 'UserAggregateProjector'. That means changing 'Aggregate' base class to this:
The changes are the introduction of the 'GetProjector' method on line 30 and the use of that new method in the 'Play' method, which now does reflection of the projector class to find the 'When' methods instead of doing it over the aggregate. The end result is the same: An aggregate object with the current state of the aggregate recreated by replaying all events.
Moving the 'When' methods has obviously also changed the aggregate, which now only contains state:
This is what is known as an anemic domain model, because it has no behavior. That's usually considered an anti-pattern, but I don't necessarily agree that it is; as argued above the aggregate is essentially a projection of the events, so I do not see why that object has to be where the domain behavior goes. As we saw in the 4th post of the series command handlers is a nice place to put domain behavior.
The projector violates Open/Closed principle
As a stated at the beginning of this post the design I've arrived at now is not good: The new 'UserAggregateProjector' suffers just as much from perpetual growth as the aggregate did before I moved the 'When' methods out of it. In other words the new projector violates the Open/Closed principle, which is what I am trying to get away from. So I have not solved anything, just moved the problem to a new abstraction :( Seems like I need to take another iteration, which I will in the next post.
The code for this post is in this branch.
Tuesday, November 14, 2017
Event Sourced Aggregates Part 4: Smart command handlers
Motivation
- One for the domain logic. E.g. a 'ChangeUsername' method, that has whatever business logic there is around changing the username. If and when these methods decide a change to the aggregate state is needed they emit a domain event.
- A 'When' method for any new events. The `When` methods perform all state changes on the aggregate.
Smart command handlers
In the second post I introduced the 'ChangeUsernameCommand` and the associated command handler and methods on the aggregate. In particular this `ChangeUsername` method on the aggregate:which implements the domain logic for changing username. That is the logic I want to move to the command handler.
Moving the domain logic straight over the command handler, changes the `Handle` method on the command handler to this:
Now the command handler contains the logic for handling the command. Note that now the command handler also emits domain events - on line 7. This makes sense since this is still event sourced, so changes to the aggregate state are still done through events. The rest of the mechanics around events remain unchanged: The `Emit` method on the base aggregate still calls the 'When' method for the event and stores the event to the list of new events on the aggregate. Saving the aggregate still means appending the list of new events to the event store, and getting the aggregate from the 'AggregateRepository' still means reading all the aggregates events from the event store and replaying each one.
Having moved the domain logic out of the aggregate I have a slimmer aggregate, that only has the state of the aggregate and the 'When' methods. In the next two post I slim down the aggregate even further by moving the 'When' methods out.
The complete code for this post in this branch.
Monday, November 6, 2017
Event Sourced Aggregates Part 3: Clean up command handlers
Repeated code in command handlers
Introducing a helper
Cleaner command handlers
- Make the command handlers smarter
- Make the aggregate anemic in a naive way, leaving a stable aggregate, but introducing a new Open/Closed violation
- Make the aggregate anemic, events (a little) smart, and solve the Open/Closed violation
Tuesday, October 31, 2017
Event Sourced Aggregates Part 2: Where the mess starts
The second feature
- Send a new type of command
- Implement a command handler for the new command
- Implement a new method on the aggregate with the new domain logic
- Emit any new events needed
- Implement new When methods for any new event types
That's pretty straight forward and not too interesting, so let's move on to the command handler:
That's also pretty straight forward, but a little more interesting: Most of the the command handler code is something that will be repeated in all commands handlers. We will deal with this in the next post, where that repetition is pulled out into a helper class.
The next step is changes to the aggregate, where this is added:
This is still pretty straight forward. In fact everything needed to add this new feature was straight forward, so that is a good thing. The problem lies in the last two methods, the ones added to the aggregate.
Problem: An Open/Closed Principle violation
That's it for now. The next posts will:
- Make the command handlers smarter and clean up some repetitiveness
- Make the aggregate anemic in a naive way, leaving a stable aggregate, but introducing a new Open/Closed violation
- Make the aggregate anemic, events (a little) smart, and solve the Open/Closed violation
Tuesday, October 24, 2017
Event Sourced Aggregates Part 1: Outline of a typical implementation
Through this series of post, I will show how event sourced aggregates violate Open/Closed and - as a consequence - tend to grow monotonically, and then show how we can address that by re-factoring away from the big aggregate towards a small and manageable one. Cliffhanger and spoiler: The aggregate will become anemic and I think that is a good thing.
The complete code from the series is on GitHub, where there is a branch with the code for the first two posts.
Event sourced aggregates
Using event sourcing for storing the aggregates, means that the aggregate code does not change the state of the aggregate directly, instead it emits an event. The event is applied to the aggregate which is where changes to the state of the aggregate happens, but the event is also stored to a data store. Since aggregate state is only changed when an event is applied, the current aggregate state can be recreated by reading all the events for a given aggregate up form the data store and applying each one to the aggregate. The benefits of this approach are many (when applied to a suitable domain) and described elsewhere, so I wont go into them here.
A typical C# implementation
- A command is sent, asking the domain to perform a certain task
- A command handler picks up that command, fetches the appropriate aggregate and triggers appropriate domain behavior. Fetching the aggregate involves replaying all the aggregates events (event sourcing!)
- A method on an aggregate is called, and that is where the actual domain logic is implemented
- Whenever the domain logic needs to change some state it emits an event (event sourcing, again) of a specific type
- A 'when' method for the specific event type on the aggregate is called and updates the state of the aggregate
- After the aggregate is done, all the events emitted during execution of the domain logic is dispatched to any number of event handlers that cause side effects like updating view models, or sending messages to other services.
- Add a second feature and see how the aggregate starts to violate Open/Closed principle
- Make the command handlers smarter and clean up some repetitiveness
- Make the aggregate anemic in a naive way, leaving a stable aggregate, but introducing a new Open/Closed violation
- Make the aggregate anemic, events (a little) smart, and solve the Open/Closed violation
Tuesday, August 29, 2017
Free microservices ebook
You can get from here.
The books the chapters are from are:
- My own microservices book
- Reactive Applications with Akka.NET by Anthony Brown
- The Tao of Microservices by Richard Rodger
- Docker in Action by Jeff Nickoloff
Tuesday, March 28, 2017
Talk video: "Consolidating Services with middleware" from NDC London 2017
Consolidating Services with middleware - Christian Horsdal from NDC Conferences on Vimeo.
In case you are wondering what the talk is about, here is the abstract:
"Have many services? Writing new ones often? If so middleware can help you cut down on the ceremony for writting new services and at same time consolidate the handling of cross cutting concerns. But what is middleware? OWIN and ASP.NET Core both have a concept of middleware. What are they? How do they help? In this talk we will dive into the code, write some middleware and show how middleware helps you handle cross-cutting concerns in an isolated and re-usable way across your services. I'll compare and contrast the OWIN and ASP.NET Core middleware concepts and talk about where each is appropriate."
Sunday, February 12, 2017
Event-based Collaboration does not imply Event Sourcing
Abstract
Event Based Collaboration
One of the ways microservices can collaborate is through events: When something significant happens in a microservice it can choose to publish an event that other microservices can then react to however they need/wish to.
This is a powerful style of collaboration. Events allow for asynchronuous processing, for slow subscriberss to catch up with bursts at their own pace, for some microservices to be down for shorter periods and more. For these reasons event-based collaboration between microservices is quite often a better choice than command- or query-based collaboration. The point in this context, though, is more about what the events are. These events are things that are significant outside of the microservice publishing them. They are published in order to drive collaboration with other microserivces.
Internal Event and External Events
The blue microservice from the figure above can store its state however it wants. One of the ways it can chose to store its state is using Event Sourcing. Using Event Sourcing introduces another set of events: Ones stored internally in the microservice. These events to do not drive collaboration with other microservices, but they capture every little state change in the blue microservice.
I find it helpful to distinguish between these two types of events:
- External events drive collaboration with other microservices. These are also sometimes referred to as integration events.
- Internal events captures all state changes within the microservices and is the basis for event sourcing. These are also sometimes referred to as domain events.




