Vision & Scope – Engineering of complex software systems requires more than just a good backlog
Imagine that you’re involved in the greenfield development of a new complex software system, starting from scratch. You have 100 backlog items and each individual item is written perfectly. The items follow all guidelines with respect to INVEST criteria and use the typical “as a ..”” template for user stories.
One of your backlog items might look like this:
User Story 48 – Sending confirmation for pricing legs
As a customer, I want to have a confirmation for each pricing leg that is created for one of my average pricing transactions, so that I know the current status of my average pricing transaction and can keep my systems up to date.
GIVEN a pricing leg is created WHEN the pricing leg is settled THEN a confirmation is sent.
Would you be able to start developing this backlog item? Of course not. You don’t have any contextual information about the projects, the industry, or the environment. You also don’t know the purpose of the project, and you have no information about out-of-scope items, the high-level architecture, or the non-functional requirements. By reading all 100 of the backlog items, you can only get a rough overview of the scope.
Many other perspectives are also relevant for the project. In addition to the developers involved, there may be a business case and a commercial view. You’ll have project managers with a view on time budget and quality. There’s a product owner responsible for backlog prioritization. Even in development, you might have front-end and back-end developers, architects, or new project members joining as the project progresses. You’ll never be able to start or complete the project successfully if you only have a list of 100 perfectly written backlog items as a basis.
Vision & Scope Document
Let’s turn that around: In addition to the product backlog, you need to design the system that you want to develop. This includes purpose, environment, high-level scope overview, out-of-scope topics, non-functional requirements, and high-level architecture. We develop this content in an innovation sprint and describe it in a document called the Vision & Scope Document. Here’s the document’s table of contents:
As you can see, some of the terms in the Vision & Scope Document differ slightly from the terms we’re using here. There are also additional details in the document. Let’s go through the document and take a look at the content section by section:
This section describes the environment of the system to be developed.
- Business Background: This may be information about the industry, the company, or a concrete business context of the project.
- User Descriptions: Describes the end-users of the system, perhaps as personas. It’s the starting point for a user-centric design.
- Project Stakeholders: Who is involved in the project? What are their roles and responsibilities?
- Business Processes: Outlines the business processes to be covered by the software or into which the software will have to fit in.
- Rules and Regulations: The industry or company might be subject to rules and regulations that must be considered in the project.
Opportunity for Innovation
In scenarios like start-ups, the project might cover the devlopment of an entire new business model in addition to software. In these cases, you can use this section to describe the market background, the planned business model, and potential business transformation. As mentioned in the beginning: the Vision & Scope Document covers more than just engineering-related roles. This section addresses business innovation roles and needs.
Vision and Business Value
This section deals with the purpose of the system that you want to engineer.
- Vision Statement: The vision statement should make clear why a company should invest in the proposed software and what concrete challenges it will address.
- Value Proposition: Describes the business value to be created by the developments. This might be monetary values or sustainability values, for example.
- Time-to-Value Aspects: Are there time-to-value aspects relevant for the project? For example, what would be the benefit of going live with a Minimum Viable Solution and adding other features later?
- Business Metrics: This is the link to the outcome based economy. A measurable value to drive the outcome towards the expected business value.
- Business Viability: In the context of business innovation, this defines the commercial case – that is, how new revenue is generated and the business impact.
- Scope Overview: High-level view of the planned scope, for example, at the epic level.
- Product Backlog: This is the link to the engineering backlog, which might be in Jira, for example.
- Nonfunctional Requirements: This section outlines all non-functional requirements relevant for the whole system. In particular, it deals with those relevant for defining the architecture, also called architectural characteristics.
- Data Protection and Privacy: Provides information on everything having to do the data protection and privacy requirements of the solution.
- Out of Scope: Describes the system boundaries and any aspects that are excluded from the scope of the solution.
- System Landscape and Architectural Overview: Describes the architectural surrounding of the planned solution and the planned high-level architecture.
- UI Information Architecture and Interaction Design: Presents the information architecture and, if already available, the interaction design. This can also include mock-ups.
- Prototype Information: If a prototype was built during the requirements engineering phase, you can document the results here.
- Architecture Decisions: Documents any architecture decisions taken.
Is That Still Agile?
Agile is a very important practice and mindset in software development. An engineering team must be able to adjust the backlog quickly to meet changing requirements. The Vision & Scope Document for a project should not change too much. So you might be asking yourself whether it’s still possible to use it in an agile context. And we also wanted to reduce waste and unnecessary documentation, didn’t we?
Let’s address these concerns one by one.
Agility doesn’t start with an agile backlog handling. Projects start with a requirements engineering phase. An innovation project might even start with understanding and refining the problem statement before designing a solution. You do this in very fast iterations, for example, in a Design Thinking format using sticky notes. That’s a very agile approach. In our case, this might be an innovation sprint.
An innovation sprint will provide a Vision & Scope Document and an initial backlog as results. At this point in time, the budget will be approved or, in case of co-innovation projects, the contracts will be signed based on these two documents. This also marks the end of some of the flexibility. The problem has now been defined, and the cornerstones of the solution have been designed. This explains the need to document the solution, in our case, in the Vision & Scope Document.
Now the project starts with its in-time, in-budget, and in-quality demands. Typically, only the backlog for the first sprints might be ready for development. The rest will be refined during development.
The Vision & Scope Document describes a solution that was designed through requirements engineering to solve a certain business problem. Thus, the document describes targets, context, and boundaries for the system to be engineered. It enables the development team to handle the scope in an agile way, changing and reprioritizing the backlog to reach the envisioned value proposition within the planned timeframe.
Vision, scope, and architecture can evolve over time, but there are limits to this flexibility. For example, a change of the vision, the environment, or the architecture can lead to additional costs or even a complete redesign of the solution. Thus, the solution defined in the Vision & Scope document enables agility on the way to realizing the solution, but also sets boundaries to flexibility.
Vision and Scope Document: A Central Part of Software Systems Engineering
As explained above, engineering of complex software systems requires more than a good backlog. We need to design and document the whole system that is envisioned, beyond the pure backlog.
Systems engineering is an interdisciplinary field of engineering and engineering management that focuses on how to design, integrate, and manage complex systems over their life cycles. At its core, systems engineering utilizes systems thinking principles to organize this body of knowledge. The individual outcome of such efforts, an engineered system, can be defined as a combination of components that work in synergy to collectively perform a useful function. (Wikipedia)
The Vision & Scope Document describes the cornerstones of the system to be developed: Its purpose, environment, high-level scope overview, out-of-scope topics, non-functional requirements, and high-level architecture. It brings together different perspectives of business, project management, engineering. Thus, it helps address viability, desirability, and feasibility throughout the entire software system’s life cycle.
| Editing: Ella Wittmann | Graphics: Emma Thoma | Content: Erik Scheid
thank you for such a wonderful write up. Its one of the core questions I have been battling with while evaluating retrospectively.
For massive programmes like implementing an erp or a business vertical software while build and delivery can be agile and iterative, the grasp on the vision, scope, high level design and the application integration and dependency stack is of utmost importance and should not be compromised. Delivery can be iterative and agile and would only be coherent when the baseline is strong and the teams are thoroughly onboarded.
Hi Erik (Nice first name by the way...)
I liked the read very much. thank you a lot. We have been in such situation lately pretty often, not exactly for a brand new topic, but for topics that are new to a team which is asked to help to accelerate developments on a domain for which they do not have subject matter expertise. That relates a lot to your proposal here, because this document must be written by the 'knowers': what we have seen also is that the culture of the domain can be transferred a lot faster when the edge cases (at least the ones that the 'knowers' know already) are somehow described, so this can be used to build up a good test strategy, so I would add a section in the solution scope chapter with 'known edge cases': this helps refining a lot the exact boundaries of what is in scope.