Impressions from Software Architecture Open Space 2013

We ran the 2013 edition of the Software Architecture Open Space conference (SAOS) this past Thursday, and once again I am amazed how well the open space format works. Everybody there were both highly knowledgeable and keen to share. The combination is a goldmine of learning opportunity.

Topics
The topics covered this year were quite diverse. Ranging from the particulars of Docker.io to the most effective place in an organization for an enterprise architect to work.

Since there were up to 7 seven sessions running in parallel I did not participate in all discussions. Even so just looking at the topics I had the chance to discuss paints a broad picture: "Event Sourcing in practice", "Modeling in NoSQL DBs", "How to Build an Architecture that Enables Flow in Development", "Reactive Manifesto", "NoProcess Development", "Architects and Organizations" og "The Value of Contributing in Open Source".

Take Aways
What one takes away from an event like SAOS obviously vary a lot lot from person to person. Personally the main points I came away with were:

• I am not the only one that thinks that lean thinking can be applied to software architectures, not just organizations and processes. Hallelujah I am not alone :-P
  
• Confirmation that Continuous Delivery is slowly but steadily becoming mainstream
  
• A better understanding of the role of Enterprise Architecture in large organizations (and confirmation that EA is not for me)
  
• Lots of new good contacts
  
• Insights into concrete experience with introducing event sourcing to a team and which type of common hiccups to expect
  

Code and Lean

I've been toying around with an idea for a while. It's sort of half baked, but inspired by some good conversations about it at AAOSConf and GOTO I'm going to go out on a limb and sketch it out below.

Lean
Lean software development (LSD) has been around for a while. The Poppendiecks book took the teachings of Lean Manufacturing and turned them into a set of principles and tools for thought that made sense for software development processes. In contrast to most writings on software development processes the "LSD" book does not prescribe (or even describe) a particular process. Rather it describes a number of principles and a number tools, that can be used to analyse your current process in order to find out where it can be improved. LSD provides a framework for developing your own process that is efficient in your particular situation. The point is to continuously optimize the process towards faster delivery and shorter cycle times.

Software Architecture
What is a good software architecture? That's a hard question to answer. The following is an answer I've used for a few years now in connection with an architecture training course:

"A good architecture shortens the time between understanding user needs and delivering a solution. It does this by eliminating rework and by providing a de-facto standard framework based on the end user domain model." --James O. Coplien.

It's from Cope's Lean Architecture book, and it's about optimizing for cycle time and for fast delivery. But how? How does a software architecture help optimize for fast delivery? I think some of those tools for thought from LSD can help us out not only if we apply them to process, but also to the software architecture and to the code.

Value Stream Mapping in Code
First and foremost I think we can learn a lot about what makes a good software architecture by applying value stream mapping to our code bases. What I mean by this is: Map out what needs to be done in the code for a typical feature/user story/use cases - whichever is your unit of work - to be implemented. Then take a look at which steps takes about which amount of effort.
Note that when I say code here I include config-files, tests, deployment scripts, migration scripts and so on. Mapping this out gives us insights into which parts of the architecture needs to become better: Is most of the time spent dealing with a particularly hairy integration? Maybe that needs to be better isolated. Is a lot of time spent writing and rewriting data migration scripts? Maybe we aren't using the right persistence solution. Is it very time consuming to build out the GUI? Maybe that part of the code isn't testable enough. These are just made up examples. The point is that I think value stream mapping is a good tool to find those hot spots within the software architecture.

Seeing Waste
Another tool I think we can apply to the code is seeing waste: Basically this just says that we should be on the look out for anything that creates waste - in terms of effort or in terms of resources. Applying this to software architecture means asking questions like: Do we have methods or even whole classes or layers that just delegate to other parts the code without any additional logic? 'coz that's just wasteful. It takes time to write, it takes time to debug through, and it's takes (a little bit of) time to execute. All of that is waste. We then need to ask ourselves if that waste is justified by some other goal fulfilled by those delegations. If not they need to go.
Another useful question is if there are parts of the architecture that are only really warranted for a few features in the systems but impact all features. E.g. if 80% of the features in an application is basic CRUD, but we do all data access through repositories because there are 20% of cases where more than CRUD is needed. Then isn't it wasteful to funnel everything through those repositories? -Even simple code takes time to write and test. Any code that isn't clearly justified is wasteful.

Options Thinking
Options thinking - in my mind - applies pretty directly to code. Options thinking is about keeping options open while not just stalling. It's about finding ways to postpone decisions until more information is available without slowing down progress. This - in code - is done pretty directly with well designed abstractions. That's the obvious one. I also think techniques like feature toggles and a/b testing along with looking for minimal viable implementations support options thinking. These techniques can help with trying out more than one direction in parallel and getting real world feedback. If this is done quickly - e.g. by focusing on only doing minimal implementations - this allows us to postpone deciding between the different directions a little while and gather more information in mean time.

Pull Systems
Coding pull style: TDD. That's it.

Summing up these are not fully formed ideas. But now they are out there. Any feedback will be much appreciated.

Immediately Functional? -Really?

Following up on my last post I want to focus one point in the second decision diagram in that post. The diagram is a rule of thumb for deciding what you want a framework to implement for you, and it looks like this:

I want to focus on the "immediately functional" box. It says that you only want to use features in a framework that work in your context as they are. What does that mean? What if the feature needs to be configured just right to work in your context? Or needs to be extended through some extension point? Or needs to be modified slightly by cracking open the framework code? Or will work in the next release according to the frameworks roadmap? Lets look a these in turn.
Is configuration OK? Yes. Of course it is...But only to a certain point beyond which you drown, then you're usually better off building the whole feature by yourself. And keep in mind that frameworks that dont ask you to do a lot of configuration, while still implementing enough are (or course) preferable. Conventions are good at that.
Is extending through extension points OK? Yes. Of course it is. Iff its done through supported extension points. If you extend through something that isn't meant to be an extension point, you will probably have a hard time maintaining that extension, and would usually be better off building the whole feature specifically for your purpose.
Is tweaking the framework code itself OK? No, and not even if you're in a position to merge your changes back into the framework. Because even if in that case, you're now the maintainer of part of the framework, which is not what you set out to be, when you chose to use a framework. In this case you're better off building your own and maintaining that at your own pace, and according to your own needs.
Is waiting for a feature in the next release OK? It depends...as they say :-). It depends on when you absolutely need the feature, and on when you need to start developing on the feature. In other words it depends on whether you have reached the last responsible moment or not. The last responsible moment is the moment at which you have to make a decision.It not just any moment at which you can make a decision. That's likely too early. It's not simply the last minute, because at that point you dont necessarily have a choice anymore. It's the when you reach the point where you need to start developing the feature but still have time to think your choices through and make a decision. If you dont really need the feature yet, postpone to a later iteration, and decide on whether to use the framework for that feature or not, at that time. Maybe you can postpone long a enough for that new release of the framework to come out. At that point deciding whether to use the implementation in the framework or to build your own is probably easy. If you can't wait that long before starting developing the feature, you might still be able to postpone the framework vs DYI decision: Create a clear interface to what you expect the framework to implement in the next release, and then - when you have to - either use the framework to implement the interface or roll your own implementation. You might even hedge the your bets by defining the interface and simultaneously develop against the interface and rolling your own implementation of the interface. If the framework should be released with a suitable feature while your still working on your own implementation, you have the option to switch without throwing too much away, on the other hand if the framework doesn't release anything suitable you're well underway too.

It's about making the decision at right moment - which is at the last responsible moment because that is when you have the most, and best, information to base the decision on, while still maintaining the freedom to actually decide for yourself.