How to prioritize education production for Cloud Software using Agile Content Development

[jacek_konopelski]

Abstract

Cloud software by its nature is released via continuous deployment, a regular release cycle that keeps customers happy, but which sets a difficult pace for the production of corresponding enablement materials. Learning departments must develop a rapid decision-making process to keep up with software changes, to accommodate to market trends, and to adapt to organizational changes. In this article, I’ll introduce you to the Agile Content Development (ACD) framework which we developed at SAP and explain how to use the process of Content Backlog prioritization to make transparent quality decisions on education content production in a large enterprise.

The Challenge

Running an education team in a large enterprise in the IT industry means your educational material, videos, tutorials, classroom, or virtual training, must form a cohesive portfolio. You must release them in sync with the software, you must tailor it to the right audience and their needs. Your product is running to a continuous release cycle, you have resource constraints, and limited information about your audience. Being part of a big enterprise also means engaging with many stakeholders, Product teams, Customer and Partner Success organizations, Services, etc. They will come to you with their audience's needs, ideas, and opinions, all that to influence your decision on what assets your department should produce next. On top of that, you must adapt effectively to trends in eLearning and in your industry, as well as changes in your organization.

So, with so many moving parts, how do you decide on which educational asset to produce?

By introducing the Content Backlog and the prioritization process around it.

To understand it, you need to know a bit about the ACD first:

Introduction to Agile Content Development

If your software teams follow an agile development methodology, so should the adjacent departments, like learning. That’s what we did when we created the ACD framework. We borrowed concepts from the Agile Project Management framework (see DSDM) and the SCRUM Agile methodology and adapted them to fit into the development of educational material in a large enterprise. Read sandra.policht's blog post to find out more about the ACD.

In Agile Software Development, the Product Backlog is essentially a prioritizable(!) list of product features. This list is dynamic, each entry (feature) has estimated effort, and has measurable value to the customer.

In ACD, the Content Backlog is a prioritizable(!) list of all requests for content. It’s a living document, transparent to all internal stakeholders. Each content request is scoped to determine the estimated effort, and like its software development counterpart, has measurable value to the customer.

Maintaining the requests on the Content Backlog is the responsibility of a Product Owner (PO). Who, in the ACD world, is a person with an understanding of customers’ needs and what’s essential for them. PO owns the corresponding education portfolio and has a high-level understanding of product capabilities. His/her key responsibility is to continuously and proactively seek feedback from various sources and turn them into request on the Content Backlog. The sources usually are customers, product management, trainers, customer/partner success organizations and consultants, feedback from the company’s learning platforms, surveys, etc.

In a software enterprise with a vast software portfolio, a team of Product Owners (each PO having their distinct area of responsibility) is accompanied by a Portfolio Manager (PM) who owns the execution of the Content Backlog prioritization process; and by Project Coordinator (PC) who owns production estimates and governs the team of Scrum Masters (SM) to ensure the expected work-load doesn’t exceed team’s capacity. To understand the scope of all involved roles and their responsibilities in ACD, refer to Angelina Padarnitsas’ article. I will refer to these roles below. Keep in mind these roles diverge from their DSDM and SCRUM prototypes and that we invented a few new.

Prioritizing the Content Backlog

The prioritization process of Content Backlog is a methodical approach to cope with the competing requests and so many different factors/criteria to consider. The process consists of 4 steps which each PO executes to arrive at the optimal decision for the next production cycle; It goes as follows:

Step 1. Measuring against key criteria – for each request on the backlog, PO goes through the list of below criteria (not an exhaustive list but a solid start) and gives it a score (e.g. 0-10). The higher the total score on a request, the more important it is to address it in the next production cycle.

• Business & Product Strategy – are the most important criteria to consider. PO and their education material must follow it. What are the company focus areas? Is there a key customer segment, region, or audience? What’s coming with the next product release? What is the company’s flagship product? Is there a part of the product that should be emphasized? For example, the education department may have to support building a software-developer community and therefore focus production on the content for a technical audience and their preferred asset types. PO should prioritize accordingly (give a high score) only these content requests that are in-line with their company strategies, and thus give 0 scores to all other requests.


• Audience size – while addressing all relevant audience types is important, POs must prioritize proportionally to audience size and their importance. E.g. POs in a company selling software framework may prioritize the production of coding tutorials for partners’ developers over the production of end-user guides for business users. Requests addressing the major audience get a high score, and niche low.


• Portfolio gap – is each learning path fully covered with content? If an asset is missing in a learning path, especially at the beginning, and for the key audience, POs must prioritize accordingly (in this example a high score).


• Release stretch – if an asset (e.g. a Video) is relevant for the audience but becomes outdated, i.e. far behind the current software version – PO should give a high score to the corresponding request, low in other cases.


• Feedback – external and internal about content quality, topic accuracy: if negative and the topic is in demand, the PO should give the request a very high score.


• Timing – is there an upcoming event, hackathon, marketing campaign, or other promotional opportunities? PO gives higher scores to those highly exposes topics and assets.


• Product roadmap – by staying close to product teams, PO can identify which elements from the product roadmap may change (be postponed or dropped) and prioritize content requests accordingly: stable topic (higher score), volatile topic (lower score).


• Product adoption and revenue – is a product or module more in demand by customers than others? Content for a frequently used or sold product takes priority over others (higher score).


• Content adoption and revenue – a very popular free asset (tutorial, video), frequently scheduled or high-revenue paid asset (training, eLearning) take priority (higher score) over less consumed assets.

Step 2. Defining the scope – at this stage, each PO in their area has a prioritized list with high-score requests on top. What remains is to estimate the production effort and analyze the scope of the top requests.

• Adding estimates – maintaining the data of past projects is the best effort predictor of future projects – see the role of the Project Coordinator and Scrum Master. PO consults with Content Architect (CA) - who has detailed technical product knowledge - to assure estimates accuracy.


• Analyzing the scope – while detailed scope definition of each request is part of the content production (after Step 4. Below), at this stage PO determines what is the “must-have” and what is the “won’t have” from customers' point of view (see the MoSCoW method). This approach enables the PO to consider an alternative type of assets for the extend of required work. PO finds smarter alternatives to fulfill (at least partially) more resource-demanding requests, e.g., a video series instead of an extra day of training; or, covering a “should have” topic with a trainer demo instead of an exercise for students (lower system cost). At the end, each requests’ scope has been reduced to Minimal Viable Product but not less. Because POs own the corresponding education portfolio, they keep it clean of duplicates and outdated material. E.g. it’s at their discretion to decide, that a request for new developer training will be instead addressed by changing the content of existing assets in the developer learning path. They find smarter alternatives to fulfill the more resource-demanding requests.

Step 3. Deciding – at this stage, POs have all the necessary data at hand to make a quality decision on what to produce next. Thanks to the criteria scoring, they know the importance of the requests and the necessary scope of each (“must” vs. “won’t”). Thanks to the Project Coordinator, they know team capacity and how many of the high-score requests their team can address. Thanks to Content Architects, they understand the technical complexity, such as introducing demos or exercises (should there be any). POs start to see alternatives and focus on what’s essential. They can, with confidence, decide to reduce an asset scope without compromising the quality and effectively increase learner engagement with the content. At the end, PO has the content production roadmap ready.

Step 4. Communicating – PM together with POs communicate with stakeholders, presenting the portfolio production roadmap for the next production cycle, the decisions behind it, and the team’s capacity to produce the content. Presenting all gathered requests, considered criteria, and explaining the decision-making process, prevents disappointment of the more vocal stakeholders. Synergy opportunity also emerges here, other teams like pre-sales start to reuse and volunteer to co-produce assets together. After this step, the team starts the next content production cycle; but that’s a material for another article ?

The summary

ACD has served us well for the past several years and evolved along with our team to meet SAP’s growing set of products, audience, channels, and organizational complexity. What once was done by a single person now is split into several roles. Whatever the size of your organization, you will need a repeatable method to make quality decisions on content production. Next time you are under pressure of time and stakeholders, apply this prioritization process or even consider fully implementing the ACD. Let me know how your organization prioritizes and develops educational content.