Chat with Mary Poppendieck on microservices

Earlier this year I attended a conference about Agile methods in Budapest that was a great opportunity to meet Lean and Agile-minded people and hear about lots of interesting and useful concepts. Clearly the biggest highlights were the keynote speech by Mary Poppendieck and the 30 minutes I could spend later on talking to her, starting first as a direct question during a coffee break, and continuing as a spontaneous group discussion in an ever growing circle of people.

The speech was an interesting combination of talking about scaling Agile organizations (probably this was the part that she was “ordered” to talk about), and some really cutting edge topics around Continuous Delivery and DevOps.

In our coffee-break discussion of course she emphasized how pull-based Lean systems are much better than having long backlogs and sophisticated planning activities. This one was not really surprising, given that she “invented” Lean Software Development in the first place. The interesting part was, however, when she started to talk about databases. In her theory, big central databases (no matter how fast and efficient or “distributed” they are) are not fully optimal, because they enforce tight coupling of business logic functions built on top of them. In the era of microservices, you can react to customer demand of new functionality (“pull”) fast enough only if the integration of your services within a big system that you have to do all the time is not overly complicated because of the multitude of hidden dependencies due to the database level coupling, but they can work quite independently, or at least the integration can be made very transparent restricting it to web service APIs. Of course data integrity should be kept, and for that, eventual consistency mechanisms exist, that can serve well a lot of senarios (although not all), and there are other integration challenges as well to be solved in a microservice setup. Btw, if you have some time to watch a great speech about microservices and their integration challenges, you can watch this video reording of a keynote by Jessica Kerr and Dan North (from another conference in Budapest earlier this year).

So no matter how good DevOps techniques you have, if your architecture is too tightly coupled, it can be a tough challenge to implement Continuous Delivery.