Skip to Content

While designing an IoT device, there are multiple options to connect them. Once of the considerations should be the protocol being used to chat to other devices or to a server.

Previously, I would have easily chosen to make the interface REST based and have done so with Jeenode project a few years ago. But REST is not ideal as it is designed around a simple request/response model. So you ask “did my account balance change” and the response is returned “no it did not”. So you check again a few minutes later, and get the same response. Sound like a silly example? Actually we’ve learned that it is a very real issue that many customers obsessively check throughout the day, as many as 60 times inflicting a load on back-ends that they weren’t designed for.

REST via HTTP also wasn’t really designed to work on mobile networks. HTTP on mobiles is a bit heavy, fragile and slow and drains batteries quickly.

/wp-content/uploads/2015/03/mqttorg_666557.png

Andy Stanford-Clark and Arlen Nipper invented MQTT to solve a problem they had: how to do reliable messaging over unreliable networks? In an industrial environment with computationally ”challenged” devices, with restricted power due to solar or battery power, on extremely low bandwidth and often brittle RF communications including satellite. It had to work reliably, it had to use very few computational cycles, consume trivial power and could not hog what little bandwidth there was. And it had to be pub/sub so to break the cycle of violence being inflicted by heavy unnecessary workloads and bandwidth consumption due to all the request/response polling-based monitoring & control systems that were in place at that time.

So Arlen & Andy developed a very simple, extremely efficient publish/subscribe reliable messaging protocol and named it MQ Telemetry Transport (MQTT). A protocol that enabled devices to open a connection, keep it open using very little power and receive events or commands with as little as 2 bytes of overhead. A protocol that has the things built in that you need for reliable behavior in an unreliable or intermittently connected wireless environments. Things such as “last will & testament” so all apps know immediately if a client disconnects ungracefully, “retained message” so any user re-connecting immediately gets the very latest business information, etc.

Stephen Nicholas did a fascinating apples to apples comparison of MQTT vs HTTPS on Android, 3G and WiFi which you can read here. The 3G results are quite interesting:

  • 93x faster throughput
  • 11.89x less battery to send
  • 170.9x less battery to receive
  • 1/2 as much power to keep connection open
  • 8x less network overhead

At this moment, MQTT is getting quite some tracktion and MQTT 3.1.1 has become an OASIS Standard with a royalty-free license for many years (read the OASIS press release here). On top of that, IBM has open sourced all of its MQTT source code via Eclipse.org including the new HTML5 MQTT over WebSocket JavaScript which enables MQTT in any HTML5 container including mobile browsers, desktop browsers, vehicle infotainment, consumer electronics. At the same time, open source brokers such as Mosquitto, client libraries for Java, Node.js, Python are getting increasingly mature. Also the firmware clients such as for Arduino and the ESP8266 are already very useful.

/wp-content/uploads/2015/03/75642172_2_1024x683_666558.jpgInitial versions of MQTT have been sending plain information over networks. In V3.1 you can also pass a user name and password with an MQTT packet of the protocol. Encryption across the network can be handled with SSL, independently of the MQTT protocol itself (it is worth noting that SSL is not the lightest of protocols, and does add significant network overhead). Additional security can be added by an application encrypting data that it sends and receives, but this is not something built-in to the protocol, in order to keep it simple and lightweight. These security features are especially meaningful when the IoT thing is running in e.g. a cloud scenario, in which the thing sends and receives data through the public internet.

SAP, eager to also claim its part in the IoT domain, unfortunately has not really shown any signs of offering MQTT on any of its services yet. It is currently being investigated, but nothing concrete has trickled down yet:

It is really a pitty that SAP is not supporting MQTT yet, as this would require a MQTT to HTTP bridge to collect messages from things into the SAP HANA Cloud. As the MQTT protocol is mature and secure enough to be used over the public internet, I think SAP should also support this IoT standard, especially given the fact that SAP wants claim a good chunk of the IoT domain. I could think of hooking up MQTT to e.g. SAP’s HCI, but perhaps this would also be a good business case to get ESP (Event Stream Processing) to run on the HANA Cloud Platform

To report this post you need to login first.

9 Comments

You must be Logged on to comment or reply to a post.

  1. John Patterson

    Hi Jan

    Very well written and informative thanks.

    I went to the IoT conference at SAPLabs prior to TechEd last year, it seems there are quite a few competing protocols in this space, at the conference each of the vendors got up and showed their solutions and talked about which protocol they used and why, for me as a developer there was very little signal amongst the noise, I felt I would need to do research on each and how they related back to a SAP solution. Thanks now i know a little more about MQTT.

    Cheers

    JSP

    (0) 
    1. Jan Penninkhof

      Hi John,

      You’re right, there are a few more contenders. This pages gives a very nice overview of the alternatives and the pros and cons:

      IoT Primer: IoT Protocol Wars: MQTT vs CoAP vs XMPP

      I guess it would have been fair to mention the other protocols as well, but when I was doing my research, I was looking for a protocol that would survive a cloud scenario. This would mean that on top of some basic requirements such as light-weight and pub/sub, it needed to support SSL, authentication and would preferrably be TCP (not UDP based).

      Because of that, I though only MQTT would fit the bill.

      Cheers,

      Jan

      (0) 
  2. Salvatore Castro

    Hi Jan,

    Just wanted to let you know that the SAP Plant Connectivity product (bundled with SAP MII and part of our overall SAP Connected Manufacturing strategy) is planned to support MQTT in the 15.1 release that is set for Q4 2015; so later this year.  There are some drawbacks to MQTT at the moment when compared to other protocols in that it’s completely unstructured so the target system will have to know how to manage the incoming data on it’s own.  IMHO, this standard has some room for growth but it’s an acceptable transport option…that’s why you see that PCo has a native SAP ESP streaming destination as this was well structured and easily secured.

    Sam

    (0) 
  3. Trevor Naidoo

    Fantastic overview Jan and a very important point you’ve highlighted. Yep, SAP are dithering again – or maybe just very poor communication from SAP in this regard. Personally feel SAP HCI could probably be first place to provide support for it (by way of an MQTT adapter that could be downported to SAP PI/O on premise) – This in itself could potentially open up a world of opportunities for customers with a big SAP backend footprint – or cloud footprint – or hybrid footprint.

    A personal pipe dream of mine is: SAP builds MQTT broker into ESP. Jokes aside, the momentum that MQTT is picking up can’t simply be ignored. Comparisons & IoT protocol wars aside, you’ve got to at least start supporting most popular IoT protocol first and work your way downwards, or is my thinking too simplistic?

    Trevor

    (0) 
    1. Salvatore Castro

      Trevor,

      What would be your business use case around having an MQTT interface into ESP, now braded as HANA Smart Data Streaming (SDS)?  As previously mentioned there are options with IoT Edge and also with SAP Plant Connectivity to translate various protocols and systems to natively push data into SDS so your streaming scenario may already be covered but without the need for MQTT in the middle.

      I think that your thinking may be too simplistic in that the structure of the data inside of the MQTT feed is hard-coded to the receiving application and not generic or open standard.  This makes it very difficult to say it’s the “best” IoT option or even the most popular.  I would argue that OPC-UA is more popular for industrial use cases over MQTT which is mainly focused on servicing telemetry based systems.  Also, it is my understanding that this protocol is really only a one-way feed and does not offer the option for synchronous queries or commands further limiting scenarios it can cover.

      Sam

      (0) 
      1. Trevor Naidoo

        Hi Sam, thanks for highlighting some things I wasn’t aware of.

        Was thinking out loud with the MQTT interface in SDS. Some of the business use cases I had in mind was related to sensor triggers (on devices / equipment inside the network, not on the edge). This with a backdrop of heavily integrated SAP environments.Sort of an early watch setup. An example, we have a scenario of a highly mechanized warehousing environment (literally robots traversing around the warehouse & tackling task / instructions that are fed to them via interfaces / integration. A delay of a couple of minutes (due to maybe an integration issue – or stock issue – into SAP EWM or SAP ECC) has a severe knock on effect in the Supply chain. It would be nice to be made aware of a slow down in operations (i.e. robot A is waiting inexplicably long at point C twiddling his thumbs) quite quickly through SDS (via MQTT client requests on these robots) before the integration monitoring teams pick it up through monitoring tasks at intervals. Was also thinking about it’s possible integration use cases in disparate environments (inside a network). From a systems perspective, used to identify trends of high latency network traffic issues, slow disk I/O, tables filling up too rapidly, high swapping, high CPU usage (triggering possible re-sizing discussions) etc. Sure some aspects could be covered through SAP Solution Manager and Wily Introscope but these diagnosis tools are mostly used retrospectively. My whole premise was around monitoring inside the network without burdening the network with heavy requests. 

        MQTT has it’s limitations like everything else in life :-). But if you throw in an HCI (or SAP PI/O) MQTT adapter (along with SDS detecting event trends if it has an MQTT interface) then you somewhat mitigate the interoperability concerns. If you look at IBM MQ & Apache’s ActiveMQ, they both offer native capabilities of automatically converting MQTT messages to JMS compliant messages – any middleware solution can take over from that point as a subscriber. The fire-and-forget pub/sub mechanism is immensely valuable from a landscape wide monitoring perspective. Tx

        Regards,

        (0) 
    1. Jan Penninkhof

      Hi Roberto, great to hear that SAP is also embracing MQTT. It’s very necessary for many IoT scenario’s, and a great step that it is available through HCI now 🙂

      Hope I’m not too demanding, but it would make this birthday present complete (it’s my birthday today) if SAP were to decide to package this as part of the IoT service as well. Do you think that could happen in the future?

      (0) 

Leave a Reply