I heard Frederick Stroebel from Sanlam Personal Finance in South Africa speak at the recent International SAP Conference for Financial Services and thought others would be interested in an interview on how Sanlam deal well with regulatory compliance whilst also training.
Frederick, what compliance challenges do you have?
Sanlam as a Financial Services Provider is required to comply with increasing regulatory requirements by civil authorities and also from the Financial Services Board (FSB) and the Financial Intelligence Centre (FIC), which oversees compliance in South Africa in the fight against crime, money laundering and terror financing. We need to produce annual, detailed reports for many different agencies.
The company has about 6,000 employees situated at Head Office and throughout South Africa as well as other parts of Africa (including Namibia, Botswana, Kenya and Ghana). Classroom training is not always possible as it would be a logistical nightmare to get every employee compliant.
How does training and testing people help?
We have two existing technologies: SAP Human Resources (HR) and SAP Learning Management System ( LSO). These two, together with Questionmark Perception, are used to train our employees to become compliant.
ELearning courses are developed and then published to our SAP LMS. At the end of each eLearning course, a Questionmark Perception assessment is available. SAP and Questionmark Perception are integrated, and once the employee has completed the eLearning course, his/her employee record on the SAP HR system is automatically updated. Immediately, statistical information is available and reports can be drawn and can be submitted to the civil authorities.
Within a given period of time, the employee has the luxury of doing the training at his desk in his own time. The entire exercise is paperless and is cost-saving. We can track progress of employees and a report can tell us immediately the number of completed versus uncompleted courses.
Do you run assessments just for compliance purposes or are there other business benefits?
We deliver assessments both for compliance and e-learning. It’s a combination of business requirements and legislation. We predominantly started it off thinking that the purpose would be for business reasons, but as soon as the business realized the value for regulatory compliance, we received more and more requests for that purpose.
Where does the training value come from?
We use it a lot when new products are launched or new processes are introduced. We would like to see that it becomes part of a continuous learning cycle, so people as part of their lifelong learning need to continuously up-skill themselves and go through new assessments. Or if qualifications have a certain duration, once a timeline has expired, they would do a re-assessment. It was interesting to read your interview about how How SAP and Questionmark software promote safety at one of America’s Nuclear Power Stations. We’re not exactly there, but in financial services, we need to continuously make sure people are aware of the latest legislation.
For instance, in our client contact centre we have about 700 people locally and about 1000 people country-wide. There are a lot of changes, in terms of process and product changes, and somehow they need to get that down to the people and ensure that the people have actually familiarized themselves with the information. And one way is by assessing them.
Do the regulatory authorities accept assessment reports directly?
Yes, we create a Coaching report for each individual from within Questionmark, and that is used to give the participant individual feedback to see which areas he/she needs to address. Our regulatory body for training in South Africa is SETA (Sector Education Training Authority) and we’ve submitted these reports to them, and they’ve audited these documents and have approved the use of that format in training. They are very strict in terms of compliance and governance. So where previously people had to write a full report by hand, it now makes it much easier – we just create a product assessment and provide an automated coaching report.
Frederick provided me a fragmentt of the Coaching report where someone (name not shown for confidentiality reasons) has recorded their commitment to honesty:
Do you also run surveys as well as tests?
Yes, we run a lot of surveys with the system, both for training evaluation purposes and for 360 degree evaluations and also as a very quick dipstick to determine specific views and to get input from people. Instead of sending out emails and everyone responding by email, they will ask us to put something in Questionmark Perception, and they get a combined survey report after it, which makes it easier to study.
One advantage of using Questionmark is that with the SAP system, we can only do surveys on people within the system. But we also have some people and clients who are not on SAP, and we can give them access to Questionmark Perception. For feedback after training, that is almost a standard in our training. As well as the basic reaction questions, we build in questions to ask about takeaways: What will you introduce back in your area? What will you do differently? We try to go further to see the value of the training, not just rating the facilitator, the venue and the food.
As well as speaking to Frederick Stroebel, who is Head of Learning and Development at Sanlam Personal Finance, I got some details from Mark Julius, a Training Technology Consultant at Sanlam Personal Finance.
Mark, how many assessments do you deliver a year?
We do about 30,000 to 40,000 assessments per annum. Most regulatory tests need re-taking every year. Right now for instance, we are in the middle of putting about 5,500 to 6,000 people through a FICA annual assessment, some of them in the company and some of them brokers and advisors.
How do SAP LSO and Questionmark work together?
Sanlam has been using Questionmark Perception for over 10 years now. Our employees like the user friendliness of the Questionmark system. The fact that the two systems integrate seamlessly makes it so much easier for SAP Administrators to schedule employees for any eLearning course as this happens from one point only.
And how do you link LSO with Questionmark technically?
We use SCORM from within SAP LSO, which works fine for us for now. We are looking at the Questionmark Connector for use with SAP for the future.
What are the key benefits that the SAP side of the equation gives you?
- Scheduling/booking of elearning courses takes place on SAP LMS system.
- We can record training interventions against qualifications and learning profiles of learners.
- Employee records are automatically updated on the SAP side.
- Reports are generated by race, gender and disability.
- Organizational structure-level reports provide Business Heads with feedback. regarding the progress of Learners – whether they are Competent or Not yet competent
And what key benefits does Questionmark give?
- We don’t need to schedule within Questionmark; it all hooks into SAP.
- Results are recorded against employee number.
- Detailed custom reports can be drawn per assessment per employee number.
- It gives trainers and facilitators access to Perception’s more comprehensive types of questions.
- Feedback to learners can be given at a detailed topic level.
It sounds like some people come into Questionmark Perception from SAP and some people come in directly. Is that right?
Yes. Mainly for our brokers and advisers who are not on our Sanlam network. They connect directly to the Questionmark Perception server; people in the company connect via our SAP Learning Management System. We have a preference to connect from SAP, because then reports match the organizational structure within Sanlam, but for our brokers and advisers, it’s great to be able to have them log in directly, as they are not in our SAP system.
And I see you use topic feedback in testing and assessment. Is this helpful to give diagnostic feedback at a level between question right/wrong and assessment pass/fail?
Yes, we set up our assessments so that at the end, the computer gives the participant a summary of the topics and what the score was per topic, so the participant can immediately see where they need further facilitation as well.
It is also valuable in providing feedback to the learner, where a facilitator sits with the learner. The facilitator can immediately determine from the coaching report where exactly the learner needs to go for re-training. We have done extremely well in terms of increasing our overall pass mark and per topic scores by using topic feedback. For example, for brokers and advisers, there’s an initial assessment they do, and because questions are in different topics, once they’ve taken the assessment, the facilitator can immediately see which type of training that person must go on.
Mark provided me with a snippet from a Coaching report showing that a participant had scored 80% on one topic and 58% on another topic:
Do you use simulations for skill tests as well as knowledge tests?
Yes, this is an area where Frederick is very keen for us to progress and do more simulations for skills testing. We currently use Adobe Captivate and connect it to Questionmark Perception with the Captivate question type.
What good practice advice would you give to people implementing a similar system?
Get the buy-in from your executive management. If they accept it, then everybody underneath them will be more comfortable with the proposal you put on the table. This is key to getting a successful implementation.
You must have a robust server, and also it matters where you put it if you need people inside and outside the company to access it – our assessment server is in our DMZ to permit this.
Frederick Stroebel then added:
Change management is key. Don’t just get the end users to buy into electronic assessment, but also get the current training and development facilitators to buy into the system and see the possibilities of it. Show that it is not just an electronic tool to get assessments out quickly, but also it helps you in report writing and you can get great information out afterwards.
When we started in Sanlam, people resisted this way of assessing. So we said, “Give us one of your assessments and we’ll do a trial for you”, and we did. It was a country-wide product exam.
Stakeholders said people are going to sit it all over the country and they are going to refer to their books. Secondly they are going to sit next to each other and they will ask each other the questions.
So we included a compliance first page which we do in all our assessments as a standard, where people testify that it’s their own work and they were not referring to reference books. Or in some cases we do let people refer to their own information and books (as they can in the real world), but just remind them that time keeps on ticking. So if you’ve not studied and you don’t know where to find things, it won’t help you a lot anyway.
Secondly we set up a data bank of questions, for instance in this test there were 300 questions in the bank but they only get 30 questions every time. So people can sit next to each other but they won’t get the same questions. In that way we addressed a lot of the resistance, by just demonstrating to them and showing that their pre-conceived ideas on electronic assessment were not valid.
Here is an example screenshot from one of Sanlam's tests:
