Mixed Reality Solutions at SAP: What’s It All About?

The mixed reality solutions topic is gaining more traction every year, which is why SAP founded a dedicated team to explore the topic already back in 2016. Since then, our team has grown in size and expertise. We’ve shown various demos at customer events and released several videos showcasing our work and what’s to come. With this blog post, we’d like to give you a glimpse into our team and the topics that we’re currently working on.

Some definitions

In this blog post you’ll see lots of acronyms, so let’s quickly go over some of the most common terms in the world of mixed reality. Based on the reality-virtuality continuum, the composition of the real and virtual world can be separated into 3 dimensions:

• Virtual Reality (VR) is a simulated experience of a virtual world that is disconnected from the real world.
• Augmented Reality (AR) provides an overlay of virtual objects on top of the real world.
• Mixed Reality (MR), brings the real and virtual world together by allowing real and digital objects to co-exist and interact in real time.

Our topic areas

We cover the following skill sets to design and build mixed reality applications:

• Project and event management
• Design, use experience design & 3D-modeling (Blender, Adobe Substance)
• Development (C#, C++, JavaScript)
• 3D development (Unity)
• Video production (Adobe Premiere, After Effects)

And we are ramping up knowledge for all major VR/AR/MR devices, including:

• Mixed reality devices such as Microsoft HoloLens 2 or Magic Leap
• VR devices such as HTC Vive Pro or Oculus Quest 2
• Smartphones, Android and iOS devices

Our concepts

As there were no best practices for spatial UX design at SAP, especially in the business area, we started from scratch and developed several demo concepts around how our software might be used when AR becomes a standard. With hundreds of demos and very valuable feedback from customer events around the world, we are now ready to enhance current SAP applications by the capabilities of mixed reality.

Have a closer look at the interview from Alexander Lingg (Head of SAP User Experience) and Denis Schleier (Head of SAP Mixed Reality Solutions) to get a quick overview of our concepts:

https://youtu.be/qTlTZFiQ_Nc

SAP Future Office: User experience beyond the screen

With SAP Future Office, our team is exploring how to take everyday office interactions beyond the screen by integrating augmented reality into products such as SAP Ariba and SAP Concur. In case you missed it, Thomas Saueressig, Executive Board Member for SAP Product Engineering, recently published a blog post about SAP Future Office where he explains how we are creating an immersive experience with SAP’s intelligent spend systems using mixed reality. And don’t miss the video below:

https://youtu.be/KsCaZtk87ok

SAP Fiori launchpad with Microsoft HoloLens

If you are already familiar with the SAP Fiori launchpad, you’ll know that it acts as the central framework and entry point to most of SAPs web applications. With this project, we decided to develop a demo scenario to incorporate a mixed reality solution into the Transport Management truck load planning functionality directly from the SAP Fiori launchpad.

As most SAP applications are already web-enabled, we thought a good first step would be to catch the low-hanging fruits and simply enrich these web applications with virtual/spatial add-ons. To do this, we validated a technology for immersive websites called WebVR. It runs on all AR/VR devices with no need to develop a native application. All that’s required is a browser and the website will automatically recognize if you are on an AR/VR-enabled device and show the virtual 3D enhancements on the fly.

The successor of WebVR is WebXR, which is standardized by World Wide Web Consortium W3C . All major browsers are increasingly supporting this WebXR standard. Also the underlaying runtime technology OpenXR, standardized by Khronos Group Inc, has been increasingly implemented by companies such as HTC, Microsoft, or Oculus. We believe that WebXR and OpenXR will definitely have an impact on how mixed reality application will be developed and consumed in the future.

Below you can see the outcome of our case study to “SAP Fiori launchpad with Microsoft HoloLens”:

https://youtu.be/LVPFNazc8gs

Current PEO projects with AR

In our current Augmented Reality (AR) project, we are closely working with developers in SAP S/4HANA Manufacturing for production engineering and operations (PEO) to create an AR user experience on top of an existing SAP standard application. Some of our goals include:

• Enabling production workers to work hands-free on the shop floor.
• Interacting with the system using 3D visualization, voice control, hand gestures and other features to ensure quality and compliance for the parts being produced.
• Receiving required manufacturing data (such as work instructions) in real time from the backend system.
• Ensuring a real-time synchronization back to the SAP S/4HANA system to report all work order activities, used parts, serial numbers, etc.
• Enabling paperless work with up-to-date information always on hand

With the AR project in PEO, we aim to provide a solution that helps production workers execute their daily tasks in a more efficient way, supporting them with the latest software and hardware capabilities. Since we’re more familiar with working at a desk rather than at a workbench, it was a challenging task to figure out the best way  to position the holograms to help workers do their job. So we designed several virtual working environments to get into our customers’ shoes and create the best experience for them.

Our next steps will be to build the application and do some user research together with end users to make the app rock solid and bring clear benefits to all our customers once we release it to the world.

Have a look at one of our UX case studies below:

https://youtu.be/M2pW0EYgslQ

Other Teams working on AR/VR Solutions

SAP SE is a big company with around 100,000 employees worldwide. Although there are central departments like our team that dive deep into mixed reality solutions, every standard development starts their AR/VR activities based on the needs of their individual product. Stay tuned for part 2 of our blog post as we explore some of the mixed reality solutions coming from the XR Cloud area, AR truck load planning, and business scene editor.