Take It Off the White Board, Execute Your Process and Measure Results

[Former Member]

Every company that embarks on a change control automation journey wants to deliver more and better change faster. How much more? What type of change? How much faster?

Metrics will be important if you want to measure success after deployment so you’ll need solid pre-deployment measures to compare against. Benchmarks will help you measure success after the deployment phase and articulate it precisely. You’ll better understand the volume of changes, the percentage of failures, and so forth, before and after, giving a solid indication of whether your processes have improved your outcomes or made them worse and where.

At this point the set of hard data points you established for your ROI baseline will help you monitor your system’s KPIs, or Key Performance Indicators, with accuracy. In terms of ROI, using this data as part of a larger set of systems’ KPIs will be doubly useful to ensure your ROI measures won’t slip.

Here are some examples of possible ROI data that can become KPI data after implementation:

• Number of Severity 1 emergency changes in a given period before and after implementation


• Number of change related production incidents in a given period before and after implementation


• Total number of changes promoted to production in a given period before and after implementation


• Mean time to migrate normal changes from development to production


• Mean time to migrate emergency changes from development to production


• Number of emergency incidents and time required


• Time required by resource level – senior Basis, SME team members, QA testing team member, release management team, etc.

Do take into account the cost of staff level involvement before and after installation. Some process and process automation initiatives require additional expert staff.

Your ROI data will feed into KPIs and SLAs, enabling you to monitor and measure whole system performance over a period of time. Following standard ALM methodology for continuous process improvement, measure all your SAP change control processes. Use your pre-defined KPIs to analyze the results and make further adjustments that improve performance, and repeat.

One of our very large retail customers made a case for optimizing change processes within their company. Their processes would not scale to meet growing demand, inconsistent landscapes led to invalid test results, quality suffered and impacted the business.

They identified some key thorny issues: conflicting parallel development (N+1), changes moving inconsistently across systems, and insufficient governance. Implementing Rev-Trac let them not only deliver more change (a 78% improvement over the previous year in just 5 months, 242% by year’s end) faster (61% of pre-approved changes now moved to production in less than a day).

How did they know this?

• They set a time period to collect new metrics on the resulting improvement delta


• They initiated a review board to guide their processes


• They analyzed their metrics after implementation to see what level of improvements they were realizing

They wouldn’t have been able to detail that improvement if they hadn’t planned ahead, regardless of their tool selection. They say it’s not just the tool, it’s the combination of the right people, tuned processes and leveraging the best tool.

One more thing - collect feedback from users during the deployment phase, to understand and meet unanticipated challenges in the new processes. Your users will thank you.