Skip to Content
Author's profile photo John Kleeman

What do the PISA competency results for 15-year olds mean for the future of the SAP community?

Every 3 years, the OECD conducts a series of professional tests to measure the competence of 15-year olds around the world. This week, they’ve published the latest results from testing 470,000 people in 65 countries, in reading, maths and science.

Here is a table of the top performers:

image

As you can see, the top performance in the world is from the Shanghai province of China, and next come South Korea and Finland. The two largest countries where SAP employs people – Germany and the United States – do not get anywhere near the top.

SAP and SAP users do not of course recruit 15-year olds. But if we and others in the IT industry want to have a strong industry in the future, we need to be able to recruit competent people. And the 15-year olds of today are our new recruits tomorrow. We talk a lot about sustainability, mostly meaning resources and energy, but we also need to have sustainability of people.

SAP has employees in 120 countries and a very global culture, so I’m sure it can if it needs move operations around and remain successful. I am not suggesting that Germany and US and other places including my home country of the United Kingdom should be better than South Korea or Finland, but could they not be as good?

SAP and most of us in the IT industry want to recruit high performers. The graph below shows the percentage of high achievers in mathematics worldwide, showing the top two levels of a six level analysis by OECD:

image

This table seems particularly significant to me if companies are looking to employ the best people.  

 

For more information on the PISA test results, see the results section on the OECD website.

 

I’m an assessment professional and it’s great to see such an impressive set of test results. And also to see politicians and others seem to appreciate the value of the results. For a more assessment-orientated take on the results, see my blog entry on the Questionmark site which looks at some of the reasons why some countries do better than others – including by offering good quality assessments.

 

I think there’s also a lesson here for all of us. Find out the facts and measure our performance. Germany, the US, the UK seem successful, vibrant countries – three of the centers of the knowledge economy. But if our 15-year olds are not as strong, our future will not be as strong.

 

There may well be parallels in our organizations that assessments or other ways of measuring things could our organization realize the present and improve the future.

 

PS To help answer comments, here is the rest of the table above for the countries lower down the table:

image

Assigned Tags

      9 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Former Member
      Former Member
      this ranking is also good at showing two biggest omissions: India and Israel. i don't see SAP and the rest of IT industry avoiding those two countries in the future, but quite the opposite. obviously, small countries are at an advantage, too.

      @greg_not_so

      Author's profile photo John Kleeman
      John Kleeman
      Blog Post Author
      Good point.

      I added a PS to the blog to include the rest of the table, which includes Israel in a medium position. India is not included in the results, but I think some regions of India are planned to be included next time.

      Author's profile photo Ethan Jewett
      Ethan Jewett
      I think we have a long way to go from these results to a correlation with competitive performance of the tech sector (much less determining a causal relationship).

      Just examining it on an exception basis, it appears that some countries towards the top (Finland, New Zealand, Canada), while respectable, are not exactly tech powerhouses. Meanwhile, to take one example: Israel is below average, especially in mathematics, and is generally considered to have an excellent technical innovation industry.

      The graphs on your other blog are telling, in that the variable with the second highest correlation to performance on PISA appears to be the existence of a standards based examination regimen. In other words, taking tests makes people good at taking tests.

      This is potentially a classic example of a performance management and KPI creation snare: managing to KPIs that measure outputs rather than outcomes. What we want are not necessarily kids who are good at tests. We want to develop a culture of innovation, a thriving economy, and a healthy society.

      What would be really interesting to see would be an analysis of historical scores against economic, innovation, and social indices, with a 15-20 year offset (as these 15-year-olds would really start having an affect on the economy and society only once they were 25 or 30). If we can find a significant correlation between performance of 15-year-olds on these tests and the social and economic conditions 15 years later, then we would be in a better position to start making strategic investment decisions based on the results of these tests.

      Cheers,
      Ethan

      Author's profile photo John Kleeman
      John Kleeman
      Blog Post Author
      Ethan

      You make some good points. I'm not so sure on the tests helping people pass tests impact, but I entirely agree that there is a lot more to building a tech industry than basic knowledge in maths or science. And that innovation, teamwork and creativity are critical and not measured by PISA.

      But I still think it is a wake up call for education to do better.

      Author's profile photo Ethan Jewett
      Ethan Jewett
      So, to continue the thread of the thought, a question for you: Why should these PISA results motivate strategic business or education decisions?

      I'm not an expert in testing, so it's very difficult for me to tell if these results should be trusted as accurate indicators of *something* or if they are more like IQ tests which I understand don't seem to causally correlate to much at all. How can I tell? What do the results indicate now and further down the road?

      Author's profile photo John Kleeman
      John Kleeman
      Blog Post Author
      Ethan

      There is a lot of detail in the test data on the OECD website and I am only presenting a summary. The tests have been designed and analyzed by psychometric professionals and seem to be genuine and good tests of competency in maths, science and reading/related skills. They also seem to be accepted by education experts and politicians as meaningful.

      See for instance http://www.nytimes.com/2010/12/07/education/07education.html where it says

      “We have to see this as a wake-up call,” Secretary of Education Arne Duncan said in an interview on Monday. “I know skeptics will want to argue with the results, but we consider them to be accurate and reliable, and we have to see them as a challenge to get better,” he added. “The United States came in 23rd or 24th in most subjects. We can quibble, or we can face the brutal truth that we’re being out-educated.”

      So I think for strategic education decisions, these results are clearly useful.

      For strategic business decisions, there are lots of other factors. I suggest education outcomes will have a business impact over time. But you are right that Finland and New Zealand are not going to become technological powerhouses quickly.

      Author's profile photo Ethan Jewett
      Ethan Jewett
      Excellent, thanks for the further information. I'll have to spend some more time looking at the data, but I completely agree that it's likely these results are useful indicators of education quality, at least in some sense.

      It would be really interesting to try to correlate them to broader economic and societal factors, but I'm sure neither of us have the time!

      Author's profile photo Former Member
      Former Member
      What does PISA results mean for socialism?
      For climate change?
      For gold prices?
      ..
      ...
      ....
      For classical music?
      ..
      ...

      I am sure if I search the 'internets' enough I will find 'experts' expounding on these matters too.

      Author's profile photo Former Member
      Former Member
      will mean a lot if you have a teenager at home and try to pay for his or her education.