Skip to Content
Author's profile photo David Chambers

Augmented Reality and IoT – Reality starts here!

Microsoft Hololens Hackathon

With the advent of PokemonGo, the impact of Augmented Reality (AR) was felt all around the world. Holograms as most of us Trekkie’s (Star Trek) know, is an absolute expectation of future Tech. To start off August with a bang, SAP and Microsoft jointly sponsored a Hololens Hackathon at the SAP D-Shop in Silicon Valley. The aim was to encourage ideation of innovative business scenarios using the Hololens – undoubtedly a great Mixed Reality (MR) technology feat. Five teams were set to compete and were given hands on experience to create a demo in 48 hours using the Microsoft Hololens and Unity as the development environment.

‘The Wesley Crushers’

The IoT Smart Connected Business Unit submitted a team who called themselves ‘The Wesley Crushers’.  The team consisted of David Chambers, Sharayu Nargolkar, Varun Hariharan, Pascal Crenzin, Jitendra Sharma from Development and Lalit Canaran in the Project Management role. Each team that entered the hackathon was also paired with a Designer from the Design and Co-Innovation Center (DCC) team, and ‘The Wesley Crushers’ were lucky enough to have Sarah Deng join their team and become an integral part of what became their overall solution.

The Hackathon Challenge!

Participants were given a deeper insight into the Hololens technology and what to expect in the MR environment by Microsoft experts present at the event. Teams were challenged  with ideating and solidifying great use cases while ensuring feasibility given the time and complexity. This was a challenge indeed for ‘The Wesley Crushers’,  but after several iterations, the team narrowed down to a topic which they could not only build in the limited timeframe, but also tell a larger overall business story. They played with various features of the Hololens like  Gaze, Gesture, Spatial Mapping, Voice Control among others, to envision how best to bring the idea to life. The team faced its share of challenges compiling individually developed components and features into one project space and projecting perfectly functioning holograms for the final demo.

The Use Case

The team chose to build a demo which showcases MR while having a strong Internet of Things (IoT) flavor. A typical scenario in IoT includes having to manage/repair a lot of smart assets. The challenge here is not only to locate a defected asset but also to locate the defect/point of failure on the asset. To address this challenge the team chose the use case of Asset Management – “Smart Maintenance”.

The business case makes use of sensor readings obtained from distinct points on the asset to reveal to the repairman, which part needs fixing. This is revealed when the repairman puts on the Hololens and looks at the asset, to see point(s) of failure. Once it is located, the repairman may examine the point by simply walking around, investigating the point of interest. A special feature of the solution is that it allows the Hololens wearer to carry out measurements in his environment with a simple tap of his finger at two distinct points. If need be, the holographic program allows the repairman to access an existing inventory of possible replacement parts, and even place an order for out-of-stock parts immediately.

The ‘Smart Maintenance’ Demo

The team highlighted current pain points in existing Smart Asset environments and provided an innovative solution that is possible with the use of a Hololens. The demo showed how the factory repairman, a newbie named Pascal who is wearing the Hololens, could navigate through many assets in a factory setting and  recognize the point of failure on a single unit, which is highlighted by a glowing indicator. He can take measurements of defected parts (say a broken pipe) with simple finger tapping gestures, and with a single voice command, he can open a holographic inventory of potential replacement parts. For all our SAP colleagues, you can check out the demo video here.

All teams presented their solutions to a judging panel comprising of SAP Executive Management, the Microsoft Engineers and representatives from Visual Enterprise, along with an encouraging audience. Everyone in attendance participated in an eccentric voting of the solutions that fit best in 3 categories: “Final Touchdown”, “The Bomb” and “Kick ***”. After the votes were tallied ‘The Wesley Crushers’ had won the “Final Touchdown” category, for the most complete performance, including the most appropriate business use case. The team described working with a Hololens as an extremely thrilling experience and an exciting learning opportunity – and they would do it all again!

IMG_1656 (002).jpeg

Co-authored by David Chambers and Sharayu Nargolkar.

Assigned Tags

      1 Comment
      You must be Logged on to comment or reply to a post.
      Author's profile photo Lazarus Mark
      Lazarus Mark

      Great Going Guys!!! The demo was cool. I liked the measurement technique and virtual inventory box!!