Technical Articles

Niladri Podder

September 17, 2019 4 minute read

Embedding Web Augmented Reality (WebAR) with Internet of Things

Introduction

Ever since the evolution of AR and VR has consistently produced new prospects to visualize objects in varied dimensions. My quest for establishing a “wow experience” for the customer has invariably taught me to include something intuitive on top of the existing services. I am working with customers who use various capabilities of SAP Leonardo Internet of Things, and whenever there is a demo, they are eager to know something the field engineers can use especially in Industry floors, site visits or live device health condition. Holo Lens, Oculus Rift, and other VR Glasses are there that deals with the issue, but site engineers rarely adopt the headsets and prefer for some mobile apps. We all notice that AR apps are complicated and consume a lot of mobile space and memory. With this limitation, Web Augmented Reality or WebAR has now come more prominent compared to the static AR apps that run with specific OSs.

A-Frame is a popular open-source web framework for building virtual reality experiences. It runs 100% in your web browser; this means no app to install! There is no need for a specific device, either e.g. Pixel or iPhone. It runs on all mobile platforms: Android, iOS11 and Windows mobile.

Scenario

In this blog post, we will use SAP IoT Device Management API to consume the live sensor data and publish it on WebAR. We will use a barcode that acts as a marker for the 3D image to superimpose on it.

Pre-requisite:

Internet of Things Service Cockpit
SAP Cloud Foundry
CF installed in CLI
Sensor Setup

Let’s Code.

First thing first, we should have access to the Internet of Things Service Cockpit.

Onboard a sensor by following this tutorial Send Data with MQTT and use Paho client to send data to the IoT cockpit. Visualize the incoming device data.

Navigate to the Useful Links tab on the left-hand side and go to API Docs > Device Management API

From the list of APIs we will only use:

Execute the above API with orderBy=timestamp desc, skip=0, and top=1 for which we will get a response like:

[
  {
    "sensorId": "sensor-id",
    "timestamp": 1566970427331,
    "measure": {
      "temperature": 23,
      "humidity": 45
    }
  }
]

Project Setup

Remember, this cloud foundry deployment depends on the static file and doesn’t have any environment-specific build packs.

Create a folder structure as mentioned below:

Edit the manifest.yml

applications:
- name:      <your App name>
  instances: 1
  memory: 256M
  disk_quota: 256M
  path:      src
  buildpack: https://github.com/cloudfoundry-community/staticfile-buildpack.git

Now, inside the src folder. Create a folder structure as mentioned below:

Create an empty file with the name Staticfile.

Download the latest .js files in the js folder:

aframe.min.js
aframe-ar.min.js
aframe-extras.loaders.min.js

Here is the index.html

<!doctype HTML>
<html>
    <head>
        <meta name="viewport" content="width=device-width, user-scalable=no, minimum-scale=1.0, maximum-scale=1.0">
    </head>
    <script src="js/aframe.min.js"></script>
    <script src="js/aframe-ar.min.js"></script>
    <script src="js/aframe-extras.loaders.min.js"></script>
	<meta name="apple-mobile-web-app-capable" content="yes">
	<meta name="mobile-web-app-capable" content="yes">
	<script>AFRAME.registerComponent('foo', {
      init: function() {

        var proxyurl = "https://cors-anywhere.herokuapp.com/";
        var url = "https://<endpoint from SAP IoT Device management API>";
          setInterval(async () => {
          const res = await fetch(proxyurl + url);
          const blocks = await res.json();
						console.log(blocks);
            this.el.setAttribute('value', blocks.temperature);
            }, 5000);

      }
    })</script>
    <bodystyle='margin : 0px; overflow: hidden;'>
        <!-- we add detectionMode and matrixCodeType to tell AR.js to recognize barcode markers -->
        <a-scene embedded arjs='sourceType: webcam; debugUIEnabled: false; detectionMode: mono_and_matrix; matrixCodeType: 3x3;'>
        <a-assets>
            <a-asset-item id="animated-asset" src="https://raw.githubusercontent.com/niladri30/Zipabox-Setup/master/models/thermometer/scene.gltf"></a-asset-item>
        </a-assets>

        <a-marker type='barcode' value='7'>
            <a-box position='0 0.5 0' color="yellow"></a-box>
        </a-marker>

        <a-marker id="animated-marker" type='barcode' value='6'>
            <a-entity
                gltf-model="#animated-asset"
                scale="0.08 0.08 0.08" >
            </a-entity>
			<a-text position="0 1 -1" color="yellow" foo></a-text>
			<a-text position="0.3 1 -1" color="yellow" value="C"></a-text>
        </a-marker>

        <a-entity camera></a-entity>
        </a-scene>
    </body>
</html>

Download the barcode 6 from this link and print it.

Deploy the application to SAP Cloud Foundry

Open the command prompt from the project folder and login to the cloud foundry and execute:

> cf push

Copy the URI form the deployed application.

Open the URL on a mobile browser and scan the barcode 6 to see the action.

Note:

For iOS use Safari.

For Android use Chrome.

Conclusion

This setup can be used for building any bespoke application and can be merged with any existing service. AR on top of IoT can be integrated into many use-cases to enhance visualizations. Considering this application to be used by field engineers, surveyors and supervisors for various industries can be helpful, as they can track the live status of the devices. Please check the below demo for the complete application usage.

3 Comments

You must be Logged on to comment or reply to a post.

Janos Noll

September 23, 2019 at 8:33 am

Hello, it might be me, but I cannot seem to find the link to the source code on this page?

Niladri Podder

Blog Post Author

September 23, 2019 at 9:21 am

Hi Janos,

The main code is index.html so you can copy the same from the blog spot. I have not deployed the complete project in Github, so as of now you can follow the steps mentioned in Project Setup.

Thanks