UX Research: What Customers Can Expect During a Si...
Technology Blogs by SAP
Learn how to extend and personalize SAP applications. Follow the SAP technology blog for insights into SAP BTP, ABAP, SAP Analytics Cloud, SAP HANA, and more.
This article clarifies what you as a customer can expect during a research site visit by SAP at your location. It focuses on user research methods frequently applied to help design and validate business software applications. Frames for the research activities can be co-innovation projects, Customer Engagement Initiative (CEI) projects, or any other collaboration.
In this blog post I will present the following user research methods in more detail:
Usability testing with existing prototypes (from low- to high-fidelity)
Validation workshops
Questionnaires
The appropriate user research method depends on the project status and the number and type of available end users and stakeholders. Ideally, SAP performs user research activities with multiple end users across companies to consider the needs and pain points of different user roles and perspectives. Please be aware that the methods and formats presented here are subject to modification.
All information that SAP collects on-site can have a direct influence on the design of applications. Therefore, you, and particularly your end users, can support SAP in building intuitive applications that lead to more efficient, effective, and satisfying work.
Through on-site user research, SAP can establish and moderate effective conversations with your end users, observe them interacting with a system, and get a better understanding of their needs and pain points. Also, SAP can understand the overall context of their work, such as their work space and surroundings, or why and when they need to communicate and/or collaborate with colleagues.
Before starting with user research, participants are informed that all results will be anonymized and that they cannot do anything wrong or break anything. Moreover, they are advised that they are not assessed at any time. Rather, SAP uses their feedback and observations to enhance product ideas. Of course, the participant can take a break and terminate a session at any time if they want.
Prerequisites
For all customer activities, a signed feedback agreement must be in place. The feedback agreement describes the principles of the working relationship with all involved parties. The most important principles include intellectual property, non-disclosure and customer commitment. If your company has not signed a feedback agreement yet, SAP will provide you with all relevant documents before the site visit.
Observations and interviews are usually conducted early in the design phase to better understand how end users work. All interviews (and observations) take place with one end user at a time and last 60 to 90 minutes. Preferably, the interview is conducted at the workplace of the end user. This ensures that the SAP interview team can see, for example, any kind of supporting material such as documents with frequently used information and can observe the end user’s interactions with the system. This helps the SAP interview team to understand how current systems are used and to identify opportunities for improvements. If allowed by the customer and the end user, the SAP interview team takes photos of the working space. Generally, interviews are conducted without further customer staff being present. This helps the interviewee to concentrate better and feel more comfortable talking about their tasks.
A SAP interview team usually consists of two or three persons: one moderator, one note taker, and/or one observer. The moderator leads the interview and asks questions according to an interview guide. The interview guide makes sure that all important topics are covered. The note taker listens very carefully to the end user and notes down everything to make sure that no information is lost. The observer (if present) focuses on interactions the end user performs with the system and/or any tools or devices. For example, the observer may note down the navigation steps taken to find specific information.
Usually, an interview begins with a short introduction of every person involved. After which, the moderator asks the end user questions about his or her background, e.g., job title and what the main tasks are. During the interview, the moderator asks the end user to provide details and to demonstrate important tasks in the system. This is to find out, for example, what data is needed at which point to fulfill certain tasks, what kind of problems occur, or why and when other persons need to be contacted.
Approximate number of participants: ≥2 per company
Background of participants: Frequently working in focus area
Estimated time investment: 60-90 minutes per end user
Focus groups are also typically applied at the beginning or during the design phase for business software applications. Here, a group of 6-8 end users are encouraged to actively think about and engage in discussions about a specific topic, such as, “What makes for a great user experience?” or “How can future interactions with a system look like in 2050?” Focus groups help to get a good overview of different opinions about one or more topics. This can be especially useful for exploring perspectives on future applications.
Depending on the goal of the focus group, participating end users can either come from the same or different backgrounds, e.g., the same user role or different user roles involved in one coherent process chain. A moderator from SAP guides the focus group through the discussions and provides triggers for new ideas and perspectives in the form of questions, photos, or even prototypes. Several note takers note down what is being said. If permitted, the focus group is also recorded.
Approximate number of participants: 6-8 end users
Background of participants: Interested in focus topic (with or without area knowledge)
Estimated time investment: 2 hours Design thinking (or similar) workshops can have different goals. Typically, workshops with design thinking formats comprise discussions and/or idea generating between SAP and customers on how to optimize existing or future processes. SAP applies such workshops, for example, to understand specific processes in companies, and identify gaps and/or opportunities for software solutions. Example topics can include optimizing applications for mobile usage, or the identification of use cases for machine learning, natural language processing, augmented or virtual reality.
SAP participants for such workshops can vary, but usually consist of a team of 2 to 5 persons: product owners and/or solution experts, researchers, designers, and sometimes development architects. Participants from the customer side are typically IT experts, team leads and experts for the business processes, end users of current solutions, and other stakeholders who are familiar with the topic. Depending on the topics to be discussed, groups might be split up and work in parallel or on different relevant topics to increase creativity and productivity.
Such workshops can be carried out with one or several interested customers at a time. The workshop formats can vary greatly. For workshops with a focus on understanding processes and closing gaps in existing scenarios, usually the customer(s) present their processes and describe relevant information. Any gaps or issues can be highlighted and discussed directly in the workshop, and can eventually be transferred into future scenarios and use cases that aim for a better support of end users.
For workshops with a focus on generating ideas, SAP usually introduces a topic for discussion. An example task to fulfill in the group(s) could be to identify relevant use cases and/or visualize one of them by creating a storyboard.
Approximate number of participants: Group: ≤ 12; at least 2 participants per customer
Background of participants: Involved in business processes or end user in focus area
Estimated time investment: ½ day – 2 days
Usability testing is applied when a user interface design is available in any form, such as scribbled on paper, visualized with low- or high-fidelity software prototypes, or as a running system. Sessions take place with one end user at a time. Again, a moderator leads the session, with at least one person observing the end user and noting down reactions and interaction steps with the prototype.
The advantage of paper prototypes is that the session with each individual end user can be very interactive: Changes to the prototype can be made immediately, e.g., by removing or crossing out unneeded information or adding data or functions that are necessary or desirable.
Usability tests can also be conducted with the help of software prototypes. Low-fidelity prototypes show a rough idea of what the SAP team has been thinking of in terms of presented information and functionality. Usually, such prototypes are not interactive but focus primarily on content. Open questions like “What do you see here?”, “What information is missing?”, “Which actions would you like to perform from here?” or “Where would you click in order to do xyz?” are asked to inspire end users to give feedback.
High-fidelity prototypes can look and feel like a live system, although there is no or little programming behind it. Such prototypes are usually used to validate how end users navigate and interact with a business application.
For a usability test, the moderator brings a test script for the end user. This script contains tasks that are relevant to the end user’s job role and that are performed regularly at work. During the usability test, the end user will often be encouraged to “think aloud.” This ensures an understanding of what the end user wants to do with a prototype, i.e., which actions to perform, where and how to search for information, or if anything is missing. A task that the user cannot successfully complete with the prototype indicates that either some information is missing or that the design is not intuitive enough.
Approximate number of participants: ≥ 2 per company
Background of participants: Frequently working in focus area
Estimated time investment: 45-90 minutes per end user Validation workshops are usually executed during or at the end of the software design process, after other research has been performed, such as usability tests. The validation takes place to discuss, for example, any changes in design ideas or complex contexts. For this approach, SAP does not test the prototype but rather shows how it is supposed to operate, i.e., what functions are available, and which use cases are supported. This method is usually applied to receive feedback from end users or stakeholders (such as IT experts or managers) on the presented designs and/or business processes.
From the SAP side, the team consists of a moderator and one or more note takers. All feedback is captured and noted down. This method does not test whether tasks can be completed with the prototype. Instead, it clarifies if the prototype is understood, how it is perceived, and whether all functions and end user requirements are represented, or if something is missing or unnecessary.
Approximate number of participants: ≥2 end users / stakeholders
Background of participants: Familiar with business processes in focus and/or frequently working in focus area
Estimated time investment: 1-2 hours
Questionnaires can be applied whenever users have gathered experience with an application, such as, after a usability test, or after having used the application for some time. Standardized questionnaires, such as SUS or UEQ, can, for example, measure the level of user satisfaction and compare it with industry benchmarks. Usually, a questionnaire just takes between 2 to5 minutes to conduct. The results of the questionnaire help the SAP team identify business applications that lack a desired level of satisfaction. Also, the results can be used to underline qualitative research results. With questionnaires, the level of satisfaction can also be tracked over time.
Approximate number of participants: As many end users as possible
Background of participants: Frequently working in focus area
Estimated time investment: 2-5 minutes per application
After the site visit
After the on-site visit, SAP consolidates all information collected from end users and participants within and across companies, shares these results in the development team to fuel discussions and enhance design ideas based on that feedback.
