An analysis of the Certification Survey Results

The SAP Certification Survey is finally over. We started it on August 13th and let it run through different events we attended either in person or virtually. We got 500 responses on the day that SAP Inside Track Istanbul took place this last weekend. It was a fitting time to draw the survey to a close and on behalf of the Certification 5, I’d like to extend our thanks to all those who took the time to complete it. I also want to especially thank the 286 people who added comments of one kind or another. Comments provide the enrichment such surveys desperately need. So what did we conclude?

Earlier I said that almost from the get go we were seeing reasonably consistent answers to key questions all the way through. That remained the case right to the end. While it is always dangerous to project through to larger numbers I am reasonably confident that we would continue to see similar patterns within 1 Standard Deviation. 

Interpreting the results is another matter. As a first shot I’d argue that the demographics indicate we reached a broad spread of SAP expertise across all geographies. That should mean results are not particularly skewed in favor of location, experience or skillset. I believe the results fairly reflect the whole SAP population although there are bound to be nuances.

Our key questions centered around the value of multiple choice questions in the context of how SAP certifies skills. 47% of those who are certified did NOT believe MCQ validates practical skills. That is despite the fact 59% thought it would help their career prospects even if employers had not requested certification. 

As expected and based upon comments we were regularly seeing on certification blog posts, 63% of those who are not certified believe the ability to demonstrate practical experience is more important in the roles they undertake. 46% said that certification had not been asked for by their employing or contracting organization. 

It should not therefore be a surprise to find that only 29% felt certification was very important/important in staff recruitment.

At this point I would conclude that in broad terms, the survey bore out much of what we have seen anecdotally among the communities in which we participate and upon which the Certification 5 based its original whitepaper critique.

We wanted to establish whether proposed changes to the way SAP certifies would be welcome. In order to do that you first have to consider whether people believe there could be value in a fresh approach to skills certification. The professional and master levels are relatively new introductions but…

Alarmingly, 37% did not know there is a professional and master level program in operation. The message is clear: SAP has to do a better job communicating this topic through its various education and ecosystem channels.

Having said that, 74% felt there is value in assessing soft skills such as project management. At a personal level I consider this a vital element in ensuring project success yet this topic isn’t tackled through certification. Much will be made of this and there is clear potential for SAP to work alongside professional management organizations. 

Along with those who have deep SAP experience, we have long argued that testing for a practical problem would have significant value. 87% agree with that idea. However, 46% only think this is worthwhile IF such a test is perceived as enhancing their career. That is understandable given we are currently talking about something that has yet to feature in SAP certification. However, based upon discussions around this at TechEd and other events, we believe there is a lot of goodwill that SAP can leverage in making this a reality. 

One of our survey tests was to discover the extent to which people might have been swayed by the arguments presented in the original SAP Certification whitepaper. 63% said they had not read it. You can argue about the extent to which the other 37% skewed results assuming they were ALL in favor of our proposals. Here I go back to my opening remarks – while the survey was ongoing, we saw patterns of answer that did not change materially. I interpret that to mean that the impact our paper may have had on respondents was minimal. 

At this point I should restate that SAP Education has returned to us on this topic. While there are detailed points of discussion, there is broad agreement that the direction we propose is something SAP needs to take. I count that as a major milestone achieved through community effort. The next step is to get SAP to commit funding that will take these ideas forward.

Anyhoo…that’s how I view the results. How about you? Do you believe this is a fair interpretation or have I missed something fundamental? 

Endnote: If anyone would like a PDF of the results set out in chart format, please let me know.