So far in my Agile blog series I’ve talked about the reasons why you should adopt Agile for SAP, how to tackle resistance and negativity, and what steps you can take to help you get started.
What’s also vitally important is knowing what success looks like and how you might go about measuring it.
Most successful Agile transformations start small and are then rolled out to the wider organization, but that usually won’t happen until there’s empirical evidence that improvements are being delivered.
So how do you know what to measure? How can can you identify the things that are working and the areas that need to be tweaked?
Here are some metrics that I’ve found helpful both within our business and for the clients that we work with.
1. Business value
Generation of business value is a key reason to spend time and money on application development. How many ideas have you seen that have no effect on business performance? For me the answer would be ‘a lot’. So focus on the ones that will create most value.
Attributing business value to specific new features can help here. For example:
- What is the anticipated increase in sales revenue?
- Can you expect to gain more customers or market share?
- Will operational costs be reduced?
If you define your value metrics before you start, then post-delivery you’ll be in a better position to judge whether value is being generated by your investment (which as we know, can be significant in SAP).
2. Velocity / On-time Delivery
When you’re running an Agile process you should progress to using ‘story points’ as a means to estimate the size of requirements.
So look at the total number of story points you’re managing to deliver in each development sprint. How does that compare to previous sprints? Is this increasing over time?
During the sprint review take time to compare what was delivered with the number of stories and points you committed to in sprint planning. Assess whether promised delivery dates have been met. This will help you understand where estimation needs to be improved.
As your estimates improve you’re more likely be able to commit to deliver a realistic number of stories in each sprint. That will allow you to manage the expectations of your customers more effectively.
3. Quality
You can expect an increase in software quality when running Agile. That should translate into greater stability, which is vital for SAP applications. ‘Don’t break the system’ is probably the #1 development requirement!
Consider metrics such as
- How many testing defects are being found?
- How many production incidents are being raised?
- How stable are the applications and requirements that are delivered in each sprint?
- How many support tickets are being raised?
Smaller, more frequent releases ensure that issues are identified early, can be resolved with much less cost, and that the recovery time from failure is significantly reduced. This approach will make measurement of quality more straightforward as you build confidence in your processes.
4. Customer satisfaction
One of the key benefits of Agile is the fact that the business gets visibility of what’s being built much sooner than with other approaches, and thus can guide the process.
You’d therefore expect Agile to provide greater business and user satisfaction since what gets delivered is more likely to match what the end customer actually wants.
Monitor user feedback and get feature usage statistics so you can know when you’re getting it right.
You’ll also want to make sure that the business is involved in the development of user stories and acceptance criteria, and that they’re present in the sprint planning, standups and review meetings. That way you’ll get a much more direct read on whether they’re happy or not.
5. Visibility and Status
Agile promotes visibility and transparency so that it’s clear to everybody what’s going on.
There are many Agile tools that can help you to monitor project status - you should use them. Agree on metrics that show progress against a targeted plan, so the Project Management Office can have confidence that things are on track.
Be religious about time bookings and use the sprint burndown chart to report on the time spent and time remaining against the plan:
And use release charts to see whether progress is on track based on current performance so you can assess what the predicted completion dates are:
Measuring and sharing progress helps to get teams aligned with each other. The integrated nature of SAP means that there will be many dependencies across teams - it’s vital to have complete visibility so impacts and risks can be effectively managed.
6. Performance KPIs
Setting and then measuring specific performance KPIs for development and testing teams can be immensely insightful. There are lots of options, but here are some common metrics
- Velocity: Analyse the functionality that is being delivered by reviewing the number of transports and changes being deployed into every SAP system, and whether those were successful or in error;
- Cycle times: Investigate how long it takes to get requirements into production so you can see how efficient your development and testing teams are;
- Rework & waste: Understanding the amount of rework and waste in the process is a great barometer of quality and hidden cost;
- Work in progress: The amount of work in progress gives insight into the current on-going cost of development and testing activity;
- Approval times: Approval wait times can kill any process. Understand where approvals are being delayed so that these bottlenecks can be targeted.
Deploying tools to automate SAP change, approval and deployment processes will help your Agile approach to succeed by eliminating manual activities, delays and errors. Ask yourself how you’ll be able to extract the metrics you need - such as the ones above - from those tools.
Conclusion
Ultimately, businesses must have the ability and flexibility to change their SAP systems at the speed that matches the broader organisation. There must be confidence in the process, culture and tools if this is going to be achieved without compromising stability.
Agile development can deliver against these requirements, though constant measurement, learning and improvement are vital as part of the overall process. It will take time, patience and willingness for change to become properly embedded but tangible evidence of progress and success can help to motivate everyone involved, not to mention providing a solid business case for wider adoption.
If you’re looking to implement Agile for SAP, take a look at this eBook on some key steps you should take to get started.
- SAP Managed Tags:
