Measure the impact of formal training
Companies are spending a lot of time, effort and money to develop the skills of their employees using internal or external formal training offerings. And the budget is anticipated to increase even more in the coming years.
But how does the training department know if a course has increased the skills of an employee? How can courses be identified which have had too little impact on employee’s development compared to their cost?
This ties into the trend that modern HR departments want to make better – more informed decisions in order to become a strategic partner of the business instead of an administrative function. To go after that objective the training department needs to identify the KPIs that drive training effectiveness. More challenging however – and therefore the focus of this post – is the collection of the data that is needed to calculate these KPIs.
To tackle the challenge HR departments often try to implement an evaluation process following the theory from Donald Kirkpatrick known as “Evaluating Training Programs – The four Levels”. Following this demand it’s not surprising that modern Learning Management Solutions have built in mechanisms to support these levels or a subset of them. But depending on what software has been chosen the scope might differ significantly.
For example the SAP Learning Solution only focusses on the first level of the Kirkpatrick model, which is the immediate feedback after the course, often referred to as ‘smile sheet’ or ‘happy sheet’.
As this is a major limitation for some of our customers SAP Consulting now offers an enhancement, which extends the training evaluation process by focusing on these requirements:
- Record the development need or expectation of the employee before the booking takes place
- Only allow employees to evaluate the course if there participation has been confirmed
- Re-evaluate the course in regards to the long-term impact and effectiveness
All of these enhancements have been tied to the course lifecycle commonly used within SAP Learning Solution. The following illustration shows the different stages used for classroom trainings:
Tier 1: Development Request
Instead of introducing an additional time-consuming process the Development Request is linked to the booking activity. That means that before a booking for a course can be placed by the employee or the supervisor a questionnaire needs to be filled out for the preferred course date.
Employee’s perspective:
If an employee selects the desired course instance in the learning portal the “request participation” / “book” button is inactive and the notification area indicates that there is no Development Request Form for the chosen course date.
To go ahead with the booking the user must create a Development Request document. On the one hand this document can work as either a business case in which the employee needs to state the need for the particular training activity. On the other hand it can record the employee’s expectations and goals regarding the course which can then help the instructor fine-tune the content.
However for administrative reasons it may be required to collect other data like accommodation or eating preferences. There is no reason why this can’t be covered during this stage as well. If that’s the case you just might label the stage differently.
After the document has been completed the “request participation” button will be available to finish the booking process.
Manager’s perspective:
Dependent on the company culture it might be common that the supervisors take care of the enrollment activity rather than the employees. To cover this need SAP Learning Solution provides a manager self-service called “manage participation” to allow your supervisors to prebook, book and cancel courses for their subordinates.
So obviously this services needed to be enhanced as well to include the “development request” stage. As you can see in the following screenshot the roadmap used for the booking procedure now features an additional stage where the manager can fill out the development request form on behalf of their employees.
If the supervisor is about to book more than one employee on the course, he’s not required to reenter the data into the development request form. The system simply creates multiple development request forms with the same content and links them to the corresponding employees.
Tier 2: Training Evaluation
The training evaluation is part of the standard functionality of SAP Learning Solution. The goal is to collect immediate feedback from the participants in regards to the course content, the instructor, the course location and so on. This questionnaire is made available via the learning portal and can be accessed from the “follow up” activities section by the employee. But one can argue about the implementation of this standard feature. By default the notification will pop up as soon as the course starts. This way all employees who have just booked the training are able to fill out the happy sheet, no matter if they actually attended the classroom training. Additionally only a small number of learners ever go to the learning portal proactively after the course started. The reason is that they simply don’t expect a do-to at this point in time.
Considering this reality the SAP Consulting team went after two objectives:
- Communicate to the employees that course evaluations are a mandatory part of the course lifecycle
- Alter the standard implementation to exclude employees who booked but not attended the course
The updated process works as follows: After the course has been conducted the course administrator / instructor starts the follow up process by confirming who of the participants booked for the training actually attended the course. Until this activity is completed the notification area in the learning portal will not show the “Perform Course Appraisal” link. In parallel this will send an E-Mail to the attendees that the training evaluation should be performed by them. Optionally the notification states that filling out the questionnaire is the prerequisite to receiving the certificate. In doing so the changes made to the standard procedure cover all issues raised earlier.
The follow up screen now indicates which participants have done the training evaluation and allows the course administrator to send a reminder via E-Mail to the employees who have not completed the process yet. Of course the process of reminding employees to fill out the questionnaires can be made automatic resulting in periodically generated notifications.
After the evaluations are completed by all the attendees the course administrator can mark the course as “followed up” which will send the certificate to the employee via E-Mail if desired.
Tier 3: Development Review
The Development Review form closes the evaluation cycle by providing another questionnaire for the participant after a defined timeframe (e.g. 180 days after the course ended). This form is used to record the long term success of the course and the impact it had on the employee’s development.
In contrast to the happy sheet where the learner is the only one capable of filling out the evaluation form the development review might require the input from other sources such as the supervisor. The technology used in this scenario provides the flexibility to pull in other persons as well.
No matter who is in charge for filling out the development review form, the system needs to take care of the communication. So after a specific milestone has been reached (X days after the course ended) the corresponding user(s) will be notified via E-Mail to access and complete the form.
For the employee a dedicated area has been created in the learning portal to surface pending development review forms.
Reporting
After all that data has been collected reporting can be done using the standard SAP ERP reports accessible via the dynamic information menu (LSO_PSV3). Because the enhancement leverages the flexibility of the PHAP forms no changes need to be made in order to report on that data. If required customers can export the results and visualize them in Microsoft Excel or extract the information into a BW-system to use the advanced analyzing capabilities.
Implementation and configuration options
In order to implement an evaluation process as described above several enhancements to your SAP Learning Solution system are needed. These enhancements have been packaged by SAP Consulting and can be imported into SAP ERP systems.
As the requirements might differ the solution includes multiple customizing tables similar to other IMG activities from SAP Learning Solution. For instance it can be determined which template is used for the selected stage or it is possible to differentiate the evaluation process based on the delivery method.
Course administrators have the option to override the templates on the course type level. This ensures that the enhancement is flexible enough to support the needs of many companies without massive changes to the code.
Obviously the evaluation templates will not be part of the configuration file as organizations have their own preferences regarding criteria and rating options. These will be built leveraging SAP standard functionality via PHAP_CATALOG.
In case you have any question or want to see the solution in action please reach out to Pelagia Papadopulu or me directly.
Kind regards,
Martin Mueller
@muellersmartin
P.S. Many thanks to Jenny and Daniel for putting this all together!
This post is also available on Workingtitle.
- SAP Managed Tags:
4 Comments
