Skip to Content
User Experience Insights
Author's profile photo Ulf Reinecke

Mixed Reality Solutions at SAP: What’s It All About?

The mixed reality solutions topic is gaining more traction every year, which is why SAP founded a dedicated team to explore the topic already back in 2016. Since then, our team has grown in size and expertise. We’ve shown various demos at customer events and released several videos showcasing our work and what’s to come. With this blog post, we’d like to give you a glimpse into our team and the topics that we’re currently working on.

Some definitions

In this blog post you’ll see lots of acronyms, so let’s quickly go over some of the most common terms in the world of mixed reality. Based on the reality-virtuality continuum, the composition of the real and virtual world can be separated into 3 dimensions:

  • Virtual Reality (VR) is a simulated experience of a virtual world that is disconnected from the real world.
  • Augmented Reality (AR) provides an overlay of virtual objects on top of the real world.
  • Mixed Reality (MR), brings the real and virtual world together by allowing real and digital objects to co-exist and interact in real time.

Our topic areas

We cover the following skill sets to design and build mixed reality applications:

  • Project and event management
  • Design, use experience design & 3D-modeling (Blender, Adobe Substance)
  • Development (C#, C++, JavaScript)
  • 3D development (Unity)
  • Video production (Adobe Premiere, After Effects)

And we are ramping up knowledge for all major VR/AR/MR devices, including:

  • Mixed reality devices such as Microsoft HoloLens 2 or Magic Leap
  • VR devices such as HTC Vive Pro or Oculus Quest 2
  • Smartphones, Android and iOS devices

Our concepts

As there were no best practices for spatial UX design at SAP, especially in the business area, we started from scratch and developed several demo concepts around how our software might be used when AR becomes a standard. With hundreds of demos and very valuable feedback from customer events around the world, we are now ready to enhance current SAP applications by the capabilities of mixed reality.

Have a closer look at the interview from Alexander Lingg (Head of SAP User Experience) and Denis Schleier (Head of SAP Mixed Reality Solutions) to get a quick overview of our concepts:

https://youtu.be/qTlTZFiQ_Nc

SAP Future Office: User experience beyond the screen

With SAP Future Office, our team is exploring how to take everyday office interactions beyond the screen by integrating augmented reality into products such as SAP Ariba and SAP Concur. In case you missed it, Thomas Saueressig, Executive Board Member for SAP Product Engineering, recently published a blog post about SAP Future Office where he explains how we are creating an immersive experience with SAP’s intelligent spend systems using mixed reality. And don’t miss the video below:

https://youtu.be/KsCaZtk87ok

SAP%20Future%20Office

SAP Fiori launchpad with Microsoft HoloLens

If you are already familiar with the SAP Fiori launchpad, you’ll know that it acts as the central framework and entry point to most of SAPs web applications. With this project, we decided to develop a demo scenario to incorporate a mixed reality solution into the Transport Management truck load planning functionality directly from the SAP Fiori launchpad.

As most SAP applications are already web-enabled, we thought a good first step would be to catch the low-hanging fruits and simply enrich these web applications with virtual/spatial add-ons. To do this, we validated a technology for immersive websites called WebVR. It runs on all AR/VR devices with no need to develop a native application. All that’s required is a browser and the website will automatically recognize if you are on an AR/VR-enabled device and show the virtual 3D enhancements on the fly.

The successor of WebVR is WebXR, which is standardized by World Wide Web Consortium W3C . All major browsers are increasingly supporting this WebXR standard. Also the underlaying runtime technology OpenXR, standardized by Khronos Group Inc, has been increasingly implemented by companies such as HTC, Microsoft, or Oculus. We believe that WebXR and OpenXR will definitely have an impact on how mixed reality application will be developed and consumed in the future.

Below you can see the outcome of our case study to “SAP Fiori launchpad with Microsoft HoloLens”:

https://youtu.be/LVPFNazc8gs

SAP%20Fiori%20launchpad%20with%20Microsoft%20HoloLens

Current PEO projects with AR

In our current Augmented Reality (AR) project, we are closely working with developers in SAP S/4HANA Manufacturing for production engineering and operations (PEO) to create an AR user experience on top of an existing SAP standard application. Some of our goals include:

  • Enabling production workers to work hands-free on the shop floor.
  • Interacting with the system using 3D visualization, voice control, hand gestures and other features to ensure quality and compliance for the parts being produced.
  • Receiving required manufacturing data (such as work instructions) in real time from the backend system.
  • Ensuring a real-time synchronization back to the SAP S/4HANA system to report all work order activities, used parts, serial numbers, etc.
  • Enabling paperless work with up-to-date information always on hand

With the AR project in PEO, we aim to provide a solution that helps production workers execute their daily tasks in a more efficient way, supporting them with the latest software and hardware capabilities. Since we’re more familiar with working at a desk rather than at a workbench, it was a challenging task to figure out the best way  to position the holograms to help workers do their job. So we designed several virtual working environments to get into our customers’ shoes and create the best experience for them.

Our next steps will be to build the application and do some user research together with end users to make the app rock solid and bring clear benefits to all our customers once we release it to the world.

Have a look at one of our UX case studies below:

https://youtu.be/M2pW0EYgslQ

AR%20project%20in%20PEO

Other Teams working on AR/VR Solutions

SAP SE is a big company with around 100,000 employees worldwide. Although there are central departments like our team that dive deep into mixed reality solutions, every standard development starts their AR/VR activities based on the needs of their individual product. Stay tuned for part 2 of our blog post as we explore some of the mixed reality solutions coming from the XR Cloud area, AR truck load planning, and business scene editor.

Assigned Tags

      6 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Greg Gipson
      Greg Gipson

      Ulf,

      Great blog!

      Is there anywhere I can go within the SAP landscape to see the roadmap for this initiative and any particulars with respect to utilization and development?

       

      Regards,

      Greg

      Author's profile photo Ulf Reinecke
      Ulf Reinecke
      Blog Post Author

      Hi Greg,

      thanks for your feedback!
      Not yet, but as soon as we have any updates, we will announce it.

      Kind regards,
      Ulf

      Author's profile photo Luis Andres Lara Graffe
      Luis Andres Lara Graffe

      Hi Ulf.

       

      Great Blog. I'm really interested in this topic. Do you know any tutorial to learn how to build VR applications using SAP Fiori?

       

      Best Regards and Take Care:

       

      Luis Lara.

      Author's profile photo Ulf Reinecke
      Ulf Reinecke
      Blog Post Author

      Hi Luis,

      thank you for your feedback!

      In regards to your question, there is atm no standard recommendation, how to enable SAP Fiori in VR.

      We are building in most cases our own UI (e.g. with the 3D engine unity and with frameworks like MRTK for Hololens development)

      And there are so many other approaches (Other 3D Engines, other UI Frameworks, other technologies like WebXR) which we are also looking at.

      Unfortunately based on the variety it`s hard for me to give you a recommendation, but any VR tutorial for your preferred 3D engine and for your VR Device is a good start.

      Kind regards,
      Ulf

      Author's profile photo Marco Favero
      Marco Favero

      Do you have any update about that? We are studing in order to offer some advanced solution in SAP to our customer. please contact me at mfavero@aimgo.live

      Author's profile photo Mark Trayford
      Mark Trayford

       

      This looks really interesting, I've been playing around with Unity and SAP in my spare time for the last year, as a SAP Integration consultant by trade it's fun doing some slight 'cooler' stuff. I've been  pulling data from SAP/BTP APIs into Unity. I've built a SAP CPI/Business Process monitoring tool and I've got a few other things in the pipeline. There is a lot of potential for some really interesting products.