Agility in Software Engineering is not an Option

First word of caution: this blog is in the category ‘personal insights’: it is thus PERSONAL, it should be the start of a conversation, not the end… What is more useless than a blog with no comments?

Imagine you are working for a large enterprise software company: you are a manager of a team of say 100 persons, and one Monday morning, you receive a call from your manager saying: “Something important came up: you have one week to provide me with 20 names to work on this new topic!”. We will refer to this as the ‘Monday topic’.

Rings a bell? Not everything has to be that fast, big, or brutal, but as true as the sky is blue, these situations will happen: it could be because there is a customer escalation, because there was a bad analyst report, because a competitor announced a new stunning feature, because there is a new head or product, because a development manager has left and his team is in disarray…

How did we get here?

The first reflex is to ask: ‘but how did we get here?’. In the true spirit of agile methodologies, and the use of retrospectives, the first reaction is to try to understand why something so important was not considered high in the backlog on Friday and appeared on top on the next Monday. The truth is that backlog ranking is always subjective. You may try to organize committees between product managers to get a consensus, or rely on a single individual with charisma, or do customer surveys and users councils: None of these rankings is truer than the others because there is no absolute scientific truth about these rankings. Do not get me wrong here: you need a backlog ranking because you need to communicate something for the development teams to work in sequence if the workload is bigger than the available resource (it will always be the case). But a ranking is like any other plan: it may change, and some may argue that it should change (and it may even be because of incremental changes over months that the important ‘Monday topic’ was pushed to the bottom of the stack).

So, the practical advice is do not spend too much energy on understanding out you got there, there will be always unexpected triggering events, and black swans cannot be predicted, and they may not even be rare! Change is constant. The source of the change is not that much important.

How easy it is to handle that situation?

Now, you must engage with the development managers and the development teams. You must face a situation of forces pulling in different directions.

1. Most software organizations consider agility as a goal, so the reaction time to the ‘Monday topic’ is in your KPIs (Key Performance Indicators).
2. It is not as if your teams were not already working on some features expected by some customers, so the ‘Monday topic’ will choose some customers over some others. Of course, the Net Promoter Score of these customers will be lowered, and the NPS is also in your KPIs.
3. Switching to the ‘Monday topic’ will mean that you will have to stop developments that were going ahead, and it is likely that, when stopped, these developments will be half-baked, not reaching the viability that you were shooting for, and this leads to lower overall quality level, another of your KPIs.
4. Context switching will reduce productivity on a short term because of the knowledge transfer costs, and longer term because it may introduce dependencies (since you need to find the knowledge somewhere) and productivity is one of your KPIs.
5. Teams will feel less engaged because they have the feeling of losing their autonomy (which is a fact in our case), and not recognized for their domain knowledge on the topic they were working on before. The employee engagement will thus be reduced, and it is also one of your KPIs.

The way that your organization handles the fact that you (and your team leads) will have to lower 4 KPIs to be more agile is a clear sign of the overall maturity of your organization with respect to ‘agility’, and therefore you need a special leadership culture for agile. Agile Organization means much more than having scrum ceremonies or a Kanban board: it is a known fact and has generated the thinking around ‘Fake Agile’. Out of these 4 lowered KPIs, the most important is the employee engagement. How to manage that employee engagement? It boils down to the one agile currency: trust. You know that the ‘Monday topic’ is a disturbance is that this disturbance will have costs. It is important that you show that you know about these costs, and, eventually, that you tried your best to avoid it. Concerning the feeling of autonomy loss, it is worth remembering that autonomy of development teams does not mean that they live in the vacuum, there ARE pressures from the external world, and that developments teams should be as proud of having some domain knowledge as their ability to become knowledgeable on some other domains, knowing that the spectrum of these domains cannot extend to infinite. Finally, and this will be the subject of the next blog, you will have to watch the consequences of these disturbances to, in a way, check the costs.

Agility and Road-maps

The point on the satisfied customers and less satisfied customer is linked to the enterprise culture with respect to ‘commitments’ and thus road-maps. Road-maps describing features to be delivered for the next 3 or 4 quarters are not compatible with an agile world when things change fast. As noted in the agile methodologies, road-maps should focus on outcomes rather than outputs and talk in themes and epics rather than features: They must not stand for tactical plans. The large cloud providers understand this as noted in the road-map style of Microsoft Azure or the style of Amazon (a public example can found in Amazon blog) where the development themes are described under three labels: “shipped”, “in preview”, or “in development” (sometimes ‘in research” is added). When we talk about enterprise culture it even goes to the sales team which should not be trained to sell road-maps to win a deal. If the upper management is advocating both for agility and road-map adherence where road-maps are described as tactical plans, you know this organization is not setup for success.

Conclusion

In our world of large and complex enterprise software, there is nothing, absolutely nothing, that is carved into stone, and there is no absolute truth on what is needed by the customer enterprises. Therefore, agility is not an option: it is absolutely vital and mandatory. But changes will require a proper management of the customer (or internal stakeholders) dissatisfaction for the ones who were expecting something that will not happen (and thus leading to coming escalations), a proper monitoring of overall quality (since the developments that were stopped early will lead to quality issues and will eat some upcoming resources in the same development teams), a proper monitoring of the productivity loss (and do not mix activity with productivity), a proper monitoring and specific actions to elevate employee engagement. This will be exposed in a next blog.