How to use the A-model to evaluate a SAP training programme
In an earlier post The A-model, a new way of thinking about measuring the effectiveness of training, I explained how the A-model allows you to measure the effectiveness of a training initiative, for instance whether your training in SAP software is effective. The A-model starts with a Problem, moves on to human Performance that will solve the Problem, and then measures a training Programme in the context of Performance and Problem.
IDC have suggested that the biggest training challenge is “to effectively describe the opportunity cost of spending a dollar on training as opposed to some other essential need”. I recently met Dr. Bruce Aaron, the inventor of the A-model and heard him speak at the Questionmark user conference, and I think it’s a really strong way of solving this problem. This post explains the A-model more:
Why is it called the A-model?
It’s easy to explain why it’s called the A-model. The model (shown left) traces the progress from Analysis & Design to Assessment & Evaluation. When following the A-model, you move from the lower left corner of the “A” up to the delivery of the Program (at the top or apex of the “A”) and then down the right side of the model to evaluate how the Problem has been resolved.
The key logic in the model is that you work out the requirements for success in Analysis & Design, and then assess against them in the Assessment & Evaluation phase.
Analysis & Design
During Analysis & Design, you define the measures of success, including:
- How can you know or measure if the problem is solved? For instance what business issue will the SAP software training solve, and how do you measure it?
- What performance are you looking for and how do you measure if it is achieved?
- What are the requirements for the Program and how do you measure if they are met?
It’s crucial to be able to do this to be able to do Assessment & Evaluation against what you’ve worked out in Analysis & Design. Assessments are useful in Analysis & Design – for example needs assessments, performance analysis surveys, job task analysis surveys and employee/customer opinion surveys.
Assessment & Evaluation of Program
A common way to evaluate the program is to administer surveys covering perceptions of satisfaction, relevance and intent to apply the Program in the workplace. This is like a “course evaluation survey” but you want to focus on all the requirements for the Program. Evaluation of program delivery therefore also includes other factors identified in Analysis & Design that indicate whether the solution is delivered as planned.
Assessment & Evaluation of Performance
The A-model suggests that in order to improve Performance, you identify Performance Enablement measures – enablers that are necessary to support the performance, typically learning, a skill, a new attitude, performance support or an incentive.
You may be able to measure the performance itself using business metrics like number of transactions processed or other productivity measures. Assessments can be useful to measure performance and performance enablers, for instance:
- Tests to assess knowledge and skill
- Observational assessments (e.g. a workplace supervisor assessing performance against a checklist)
- 360 degree surveys of performance from peers, colleagues and managers
You can deliver such assessments in many ways. One way is to use my company, Questionmark‘s assessment management system which can be used standalone or within SAP LSO.
Measuring whether the problem is solved
How you measure whether the problem is solved will arise from the analysis and design done originally. A useful mechanism can be an impact survey or follow-up survey, but there should also be concrete business data to provide evidence that the problem has been solved or business performance improved.
Putting it all together: the A-model
The key in the A-model is to put it together, as shown in the diagram below. You define the requirements for the Problem, the Performance, Performance Enablement and the Program. Then you assess the outcomes – for the delivery of the program, for the Performance Enablement and the Performance itself and then for the Impact against the business.
I hope you enjoyed this brief explanation of the A-model. For a more thorough explanation of the A-model, read Dr. Bruce C. Aaron’s excellent white paper available here (free with registration).
If you are delivering training on SAP software or elsewhere in the SAP ecosystem, and want to show that your training is cost-effective, the A-model is a strong route to do this.