Putting the New Quiz Builder to the Test in SuccessFactors Learning

A lot of visibility has been placed on the new People Profile from SuccessFactors in the core SF platform/EC product with the b1508 release, and rightly so.  The new profile based on the latest SAP Fiori/UI5 design principles provides a more modern and efficient user-friendly interface when displaying employee profile details.  Not to be overshadowed by the new People Profile though is the new Quiz Builder functionality that is now part of SuccessFactors Learning.   The Quiz Builder is meant to replace (eventually) the legacy Plateau Question Editor (PQE) used in the LMS to build questions and exams.  For all the LMS admins out there, this could be one of the best updates in the LMS in a while, and there have been some rather nice features introduced as of late in their own right.  The PQE product though is an eye sore and a chore to deal with at times.  While the new People Profile in the SuccessFactors core platform might be the appointed star of the show, the bigger leap in terms of UX design improvement and usability is easily won by the Quiz Builder in the LMS (if this were a competition), at least in my opinion.  In this blog I explore what the new Quiz Builder brings to the LMS and what it means for the future of assessments in SuccessFactors Learning.

Quiz Builder vs. PQE – Why the Fuss?

I attended SuccessConnect 2015 in Singapore just before writing this blog and set in on a session given by the Director of Learning Products and Strategy at SuccessFactors for the APAC region.  In discussing the LMS roadmap and latest features in the system, you could tell that she was easily the most excited about what the new Quiz Builder brings to the LMS.  She also made a very interesting comment, and hopefully I paraphrase her correctly here. She stated that of all the usability testing that SuccessFactors has done on their products, the new Quiz Builder in SuccessFactors Learning has scored the highest among their testing groups. Hopefully that alone peaks your interest to continue reading?

First things first, a quick overview of what the Quiz Builder is in the LMS.  With this new tool, LMS administrators now have an alternative to creating and building quizzes to deliver to learners.  Previously this was done exclusively through the aforementioned PQE, where admins could create Questions and Exams for the same purpose. Let’s get clear on the semantics here now as this appears to be how SuccessFactors is going to refer to these going forward.  ‘Exam’ is the old term for assessments built through the PQE while ‘Quiz’ is the new term for assessments built through this new tool, termed the Quiz Builder.  This initial release of the Quiz Builder will allow for the creation of quizzes for the most commonly used question types (multi-answer, single-answer, and true/false).  Basic settings for the quiz as a whole can be specified as well, including passing percentage and quiz attempts.

If you picked up on a previous line in my opening paragraph here, I noted the new Quiz Builder is meant to replace the PQE at some point.  As with a lot of features SF introduces, a base level of functionality is rolled out initially and then you will see the subsequent releases bringing increased features related to the update in order to enhance the capabilities further (in the case of the LMS, see the recently introduced and progressively improved Programs and Open Content Network). With that in mind, the following points below should be considered.

• The Quiz Builder will serve as the basis for the replacement of the legacy Plateau Question Editor (PQE).
• Both the PQE and the Quiz Builder will be available for use for the foreseeable future. PQE will not be retired any time soon, especially not before the Quiz Builder provides all the features and capabilities that the PQE currently entails (more on the current gaps later).
• Subsequent releases after b1508 are planned to focus on supporting new question types and further usability enhancements.

Creating Quizzes in the New Quiz Builder

The new Quiz Builder can be accessed now under the LMS admin portal under the Content -> Quizzes section.  For those familiar with creating Programs in the LMS, creating Quizzes through the Quiz Builder will look much the same.  The quiz as a whole is summarized in one screen, providing a more complete overview of the quiz and questions utilized than ever was possible in the PQE.  The PQE itself is used to create questions for use in Exams, however, admins had to perform these two tasks separately (create questions vs. creating exams).  The new Quiz Builder brings the two concepts together, as they should be, and not requiring the two to be maintained outside of the central concept of the quiz.  The matrix below provides further details on the possible settings to be made for a quiz while the screen shot gives you an idea of what the UX looks like for managing a quiz.

Quiz Administration Overview

Quiz Settings

Of note from above, the capability to build quizzes and provide locale based text translations for quiz title/description, messages, and question/feedback is a HUGE new feature previously not possible in the PQE.  For clients who need to build quizzes in multiple languages, managing this in one quiz as opposed to numerous questions per language requiring separate exams per language is a much needed improvement from days past.

Building questions into the quiz is also much improved and more efficient in the Quiz Builder when compared to the PQE.  As previously noted within the PQE, questions had to be created independently (in a painful and inefficient editor to boot), then questions had to be pooled into Exam objects as well.  Within the Quiz Builder, questions can be created as part of the overall quiz directly from the Question maintenance screen and ordered accordingly.  With the new SAP Fiori/UI5 UX, the process of creating new questions, adding/adjusting question text and answer choices, and ordering the overall questions in the quiz is so much more smooth than the PQE that the LMS admins of old (back in the Plateau days) may just shed a tear or two during your first go with the application.

Question Management Within a Quiz

The experience only gets better when learners take the actual exams within the LMS as well. While the legacy exams would get the job done, they weren’t the prettiest things to look at or the most modern. The new Fiori UX principles and simplicity of the design does make the new quizzes a pleasure to run.  They are sleek and minimalist, keeping the focus on the assessment where it should be but at the same time not making the user feel like they have jumped into an early 2000’s web application in order to assess their understanding of the subject matter in question.

Examples of Questions Within Quiz (Learner)

Quiz Completion / Results (Learner)

Mind the Gap – Where the New Quiz Builder Lacks vs. PQE

As with most new features from SuccessFactors across the various modules, the new Quiz Builder is not meant to be a full replacement to the existing PQE across all capabilities, at least not yet.  There are some important gaps to consider presently with the b1508 release when comparing like-for-like against the PQE.  The list below is not comprehensive, but points out some of the key gaps to be aware of.  Keep an eye on the next few releases for SuccessFactors Learning as I expect many of these gaps to be eliminated as the tool matures.

• Adaptive learning and use of objectives
  • Questions in the new Quiz Builder are built directly within a quiz itself and cannot be assigned to Objectives within the LMS.
• Reuse of questions across quizzes
  • The PQE is focused on building a question bank so to speak, then assigning questions to objectives within the LMS and leveraging them where desired in Exams. This can allow admins to use questions/objectives across multiple exams.  Currently in the new Quiz Builder, questions are created directly within a quiz and unique to that quiz only and cannot be automatically reused as is in another quiz.
• Randomizing questions within quiz
  • While you can randomize the answer choices within questions in the new Quiz Builder, much like the PQE, you won’t be able to randomize the order of the questions within the quiz itself.
• Question variants and revisions
  • The PQE allows you to create different variants of a particular question that can then be randomized for which variant is displayed to a user when they take an exam. The Quiz Builder does not currently provide the capability to create different variants of a single question for use within a quiz.
  • Question revisions are also not supported currently in the new Quiz Builder. Within PQE you can update questions and save revisions of questions which comes in handy when an exam is active and being used.  Old question revisions can be maintained while new revisions can be used in future exams.  I haven’t played around with the Quiz Builder to the extent yet to see how/if it would allow for question revisions with respect to publishing quizzes in certain locales, so something to watch out for.
• Images in questions
  • PQE allows you to upload images to use for questions.  Currently the Quiz Builder does not support images or other media for use while presenting a question to a user.

The Future of Quizzes in SuccessFactors Learning

While yes there are functionality gaps with the new Quizzes in respect to the legacy PQE tool in SuccessFactors Learning, the UX and reduction in administration efforts for managing quizzes is a huge step forward for LMS admins.  The capabilities of the tool currently are enough to create general quizzes for a range of purposes, however you may need to wait on the next few releases if you wish to create more rich assessment experiences with the new quiz functionality.

My hope is SuccessFactors does not stop with the new Quiz Builder in the LMS once it reaches parity with the legacy PQE.  It would be great to see additional question types being supported which were not previously (matching for instance).  It’s not uncommon to see clients stick to building assessments in 3^rd party tools to create more robust assessment experiences overall.  If SuccessFactors could bridge the gap between the provided LMS Quiz functionality and those best of breed e-learning content authoring tools with respect to assessment functionality, that would be a great advancement for the LMS in the future.  Regardless, it is great to see new and improved LMS capabilities from SuccessFactors Learning in the new Fiori UX design, which I hope is only a precursor of things to come.

One last note, the Quiz Builder is not the only Fiori designed UX tool within the LMS to make a debut with the b1508 release.  SuccessFactors has also introduced a new Instructor View based on this design as well, which looks to be a great new tool for long overdue functionality for instructors that has been missing in the LMS.  A topic for another day perhaps, but not to be overlooked…

What are your thoughts on the new Quizzes in SuccessFactors Learning?