Skip to Content
Author's profile photo Dennis Howlett

An analysis of the Certification Survey Results

The SAP Certification Survey is finally over. We started it on August 13th and let it run through different events we attended either in person or virtually. We got 500 responses on the day that SAP Inside Track Istanbul took place this last weekend. It was a fitting time to draw the survey to a close and on behalf of the Certification 5, I’d like to extend our thanks to all those who took the time to complete it. I also want to especially thank the 286 people who added comments of one kind or another. Comments provide the enrichment such surveys desperately need. So what did we conclude?

Earlier I said that almost from the get go we were seeing reasonably consistent answers to key questions all the way through. That remained the case right to the end. While it is always dangerous to project through to larger numbers I am reasonably confident that we would continue to see similar patterns within 1 Standard Deviation. 

Interpreting the results is another matter. As a first shot I’d argue that the demographics indicate we reached a broad spread of SAP expertise across all geographies. That should mean results are not particularly skewed in favor of location, experience or skillset. I believe the results fairly reflect the whole SAP population although there are bound to be nuances.

Our key questions centered around the value of multiple choice questions in the context of how SAP certifies skills. 47% of those who are certified did NOT believe MCQ validates practical skills. That is despite the fact 59% thought it would help their career prospects even if employers had not requested certification. 

As expected and based upon comments we were regularly seeing on certification blog posts, 63% of those who are not certified believe the ability to demonstrate practical experience is more important in the roles they undertake. 46% said that certification had not been asked for by their employing or contracting organization. 

It should not therefore be a surprise to find that only 29% felt certification was very important/important in staff recruitment.

At this point I would conclude that in broad terms, the survey bore out much of what we have seen anecdotally among the communities in which we participate and upon which the Certification 5 based its original whitepaper critique.

We wanted to establish whether proposed changes to the way SAP certifies would be welcome. In order to do that you first have to consider whether people believe there could be value in a fresh approach to skills certification. The professional and master levels are relatively new introductions but…

Alarmingly, 37% did not know there is a professional and master level program in operation. The message is clear: SAP has to do a better job communicating this topic through its various education and ecosystem channels.

Having said that, 74% felt there is value in assessing soft skills such as project management. At a personal level I consider this a vital element in ensuring project success yet this topic isn’t tackled through certification. Much will be made of this and there is clear potential for SAP to work alongside professional management organizations. 

Along with those who have deep SAP experience, we have long argued that testing for a practical problem would have significant value. 87% agree with that idea. However, 46% only think this is worthwhile IF such a test is perceived as enhancing their career. That is understandable given we are currently talking about something that has yet to feature in SAP certification. However, based upon discussions around this at TechEd and other events, we believe there is a lot of goodwill that SAP can leverage in making this a reality. 

One of our survey tests was to discover the extent to which people might have been swayed by the arguments presented in the original SAP Certification whitepaper. 63% said they had not read it. You can argue about the extent to which the other 37% skewed results assuming they were ALL in favor of our proposals. Here I go back to my opening remarks – while the survey was ongoing, we saw patterns of answer that did not change materially. I interpret that to mean that the impact our paper may have had on respondents was minimal. 

At this point I should restate that SAP Education has returned to us on this topic. While there are detailed points of discussion, there is broad agreement that the direction we propose is something SAP needs to take. I count that as a major milestone achieved through community effort. The next step is to get SAP to commit funding that will take these ideas forward.

Anyhoo…that’s how I view the results. How about you? Do you believe this is a fair interpretation or have I missed something fundamental? 

Endnote: If anyone would like a PDF of the results set out in chart format, please let me know. 

Assigned Tags

      You must be Logged on to comment or reply to a post.
      Author's profile photo Former Member
      Former Member
      Hi Dennis,

      Actually, I am not surprised by the results. I took part of this survey, and most of these results go according to my thoughts.

      I am certified but I have never seen these certifications as reflecting any specific skillset. I changed my job a few months ago, and none of the people interviewing me for the position ever asked a single question about the certs.

      The old system definitely needed an update. The introduction of the pro and master levels will definitely play a role into giving certifications more credit. But SAP has a long way communicating about them, and finding the appropriate way of assessing the skills (hard and soft), without being too much discriminatory and without losing relevance. Live practice during the test would be a great way of doing so.

      Anyways, this is a real great job. I drew SAP Education's attention on the flaws of their certification system, and hopefully, this might lead to some enhancements.

      Author's profile photo Tammy Powlas
      Tammy Powlas
      will be started, set up, who gets to participate.  This is a key next step, in my opinion.

      Thank you for mentioning too that more focus needs to be on project certification; as a Certified Project Management Professional I take the view that SAP projects don't fail due to the software, they just aren't following key project management principles.

      I thank you and the C5 for doing "the needful" and drawing attention to this topic.  You are right, SAP doesn't communicate/advertise enough about Certification.

      Great work by the Certification 5.


      Author's profile photo Simha R. Magal
      Simha R. Magal

      Do you have enough data to conduct an analysis by years of experience (and perhaps other variables) to see if there is a difference between professionals with some, significant, and no experience?

      As a professor in a UA school, I see tremendous value of certification for our students.  Given that graduates do not have much work experience (perhaps some internships), the certification is a differentiator. Some companies that recruit regularly from our program do indicate that certification is preferred, in their position descriptions.

      Now, with regard to whether the exam accurately measures skill or knowledge, I agree with the survey results.  For the certification that universities are allowed to offer (based on TERP10), memorization will result in passing the exam!  While the new exam is much improved (less memorization, more questions that test conceptual and integrative knowledge), it is still weak.

      Great job on the white paper and the survey. Definitely the right direction for improving the value of certifications.



      Author's profile photo Former Member
      Former Member

      As you might know, we showed a prototype called i-Mine at the DemoJam in SAP TechEd, Las Vegas, that can automatically analyze large quantities of data to bring out interesting insights, patterns and relations. (Check out*~*+YY8jqkSZqHeriOTRz0KP2CC86mEgYI=)

      I thought it might make sense to use i-Mine to discover some insights from data like the Certification Survey Results, to help you gather insights and also to make i-Mine a better tool.

      Please let me know what you think!


      Author's profile photo Dennis Howlett
      Dennis Howlett
      Blog Post Author
      What? Did you bother to look at the results table? Why are you pimping this service here? That's not what SCN is about.
      Author's profile photo Dennis Howlett
      Dennis Howlett
      Blog Post Author