Disclaimer: This is not a production or documented feature – its also more of a hijack than a hack  πŸ™‚

I have been hoping for the inclusion of Websocket support on the HANA DB platform for a while now, and I was a little disappointed it was not packaged in the SPS08 release. My goal when building apps (or products) is to make use of the core platform its running on as much as possible, I firmly believe that when convincing an IT department, or company, to implement a product or app, the first question is: “How much infrastructure does this need?”. This can often be a deal breaker and why I am such a big proponent of the HANA’s DB + XS App Server integration – it consolidates the requirements into a single investment. Having a Websocket technology built directly in XS can be an additional selling point which developers are starting to expect these days.

A little while ago I wrote a blog post on building a dashboard using the awesome Node.js package from Holger Koser, however I have really been wanting to use Websockets in the metric² platform since the get go. Some comments here and here are prime examples of my long lasting hope of seeing the technology being included in the XS Engine platform sooner rather than later. I recently had a little nudge again from John Patterson to dig back into the topic and did manage to hack something together. The most interesting part of this was that once I had it working, I was left wanting just a little more …

Firstly a little bit about Websockets and why I feel they important to the app/web development world …

Real-time: In the age of having blazingly fast DB’s, we need a UI & server side (In our case XSJS files) integration layer which can display the data to user with as little over head as possible. Web Sockets supports this by providing a low-latency, near real-time connection between a client and the server.

Light Weight: Currently we need to do multiple AJAX calls to our backend server (to the XSJS files) to either perform some complex logic or DML against our database. This is slow and also fairly network intensive as each request (and response) requires additional handshakes and the packet size is considerably larger than just the intended content inside the package. In Web sockets, the requests and responses are really just the content themselves.

Duplexity: Web Sockets by nature are Full-duplex, implying that we can send and receive data at the same time for multiple requests.

Persistence: Web sockets provide a “open” connection between the server and client after the initial upgraded HTTP request handshake has been made. This lets us perform multiple server side requests using the same connection. This also lets the server initiate a response without needing a request from the client side.

The importance of these 4 factors to the web development world and to HANA XS specifically is that this is the missing link for us to take Web applications to the next level.

[Cross-domain support is another great feature!] In this example I was able to successfully have the HTML + JS file local to my PC and execute it against my HANA XSWS service (via a public URL).

— 07/25/2014 — Chris Paine made a good point on potential Cross-domain security issues (see below). Keep in mind that these files are secured just like any of the XS content files, e.g. as long as your folder has a authentication requirement it will persist to this XSWS file as well.

So onto the more interesting HANA specific parts …

I initially realized that HANA XS was using Web Sockets in SPS06, when for some reason the XS Engine debugger was not loading correctly due to my user missing some permissions. After searching through the XS code I came across the folder where the debugger was saved and it included a interesting file with the suffix of xsws πŸ™‚ i.e. XSWebService. After doing more digging I found that Websockets were being loaded in the WebDispatcher config file and I was confident I could find a way to start using it for app development.

After spending some time trying to create my own HANA package with these file extension types I realized that the name is somehow being filtered and only this file, and more specifically, in this location can be executed, otherwise the XS Engine passes back a 403 (forbidden) – I was a little disappointed but it didn’t discourage me … and I decided I would simply re-purpose my Debugger.xsws file for my own needs πŸ™‚ After a quick backup, I was ready to do some coding …

Essentially, a xsws file is just like any xsjs file, with the exception that it has some socket code returning the responses versus your regular xsjs file. You can do things like $.import for additional libraries as well as perform $.sql functions. Here is a small snippet from the code over on Github.

Debugger.xsws



$.ws.onmessage = function (evt){
    handleIncomingRequest(evt);
}
$.ws.onerror = function (evt) {
    $.trace.debug("error on connection: " + evt.message);
    throw new Error(evt.message);
}
$.ws.onclose = function (evt) {
    $.trace.debug("connection closed, disabling debugger");
    debugObject.enabled = false;
    throw new Error("Close status " + evt.code + ":" + evt.reason);
}








And this is some of our Client side code making calls the xsws service:

App.js


// Create a new WebSocket. This works fine
  var socket = new WebSocket('ws://<ENTER YOUR HANA SERVER HERE>/sap/hana/xs/debugger/api/Debugger.xsws', 'xsCrossfire');
  // Handle any errors that occur.
  socket.onerror = function(error) {
    console.log('WebSocket Error: ' + error);
  };
// Send the message through the WebSocket.
socket.send(message);







As you can see – the code and requests are very simple and straight forward, in fact to me they are a little easier than jQuery + AJAX.

One caveat I did find was connection persistence in the event you have an error on the server side, the socket connection can break, in this case you would need a error handling state that attempted a reconnect before submitting any new requests.

A quick screenshot of the running test app i developed and how the server is sending persistence frames pings/pongs to validate the open connection along with the client request and server response.


If you are interested in trying this out on your test or dev instance I have posted the code on Github. Follow these simple instructions to get up and running …

1.) Using the Web IDE, open SAP -> HANA -> XS -> Debugger

2.) Make a backup of the Debugger.xsws file or simply comment out the contents.

3.) Paste the code into the file from Github

4.) Create the Websocket.html file and paste the contents of the Github file

4.) Create the app.js file and paste the contents of the Github file

5.) Open the Websocket.html file and enter any SQL statement

(Be sure you have the debugger security role)

As you can see from the files, the required methods for web sockets are really at a minimum and barely get in your way at all.

Conclusion

At the start of the article I mentioned I was left wanting a little more … this was mainly because since I have been wanting Web Sockets for such a long time, I realized that using it , alone its not really enough. In order for us to really take XS Engine to the next level, we also need to consider a “Publish/Subscribe” feature and a server side timer feature.

The Pub-Sub approach would essentially allow us to push data from the server side, on a specific occurrence of an event, much like a DB trigger. If a new row gets inserted, push the new inserted record to the registered subscriber.

The server side timer feature would allow us to create a server side timer (similar to a xsjob) which would persist and execute the server side function every x secs/mins/hours.

Overall I am pretty impressed with the opportunities Web sockets will bring to the XS Engine. I am hoping they will be included in the next release.

To report this post you need to login first.

16 Comments

You must be Logged on to comment or reply to a post.

  1. Chris Paine

    Hi Paul!

    great stuff!

    I’ve done a little playing with WebSockets too:

    OpenUI5 mobile app on SAP HANA Cloud Platform with d3.js and WebSockets and other cool stuff

    From what I learnt playing myself, however, I’d have to question one of your statements:

    [Cross-domain support is another great feature!]

    At the moment, as far as I can see, the cross domain support is actually a really BAD feature! it opens up a reasonable security hole in the application. Any additional content on your page (whether put there by you or not) has access to the WS and can start interfering with it.

    check out the blog post:

    http://www.christian-schneider.net/CrossSiteWebSocketHijacking.html

    you really need to implement some kind of alternate CORS origin checking into your WS subscriptions. Not that I did so myself, but I think any real enterprise use of this functionality should really consider it!

    It’s also worth considering some kind of protocol to ensure that messages are actually delivered when sent out. There are many case where fire-and-forget works, but many where it doesn’t.

    I hope that when XS fully supports websockets these sorts of things will be built in. Would be good if it supported STOMP, but as I’m not sure there are any server-side JS STOMP libs out there, might be hard! Would be cool for SAP to implement!

    Thanks for great blog post.

    Chris

    (0) 
    1. Paul Aschmann Post author

      Hey Chris,

      Thanks for comment – I think the Cross-domain is definitely a security concern and thanks for sharing your thoughts.

      Re: Fire-and-Forget – good point.

      Cheers, Paul

      (0) 
  2. Kel As

    No need for strong violence:

    $ vi xsengine.ini

    [httpserver]

    enable_experimental_websockets = true

    websocket_ping_interval = 60

    (sometimes SAP forgets to document good stuff)

    Enjoy.

    (0) 
    1. Thomas Jung

      There is a reason why we don’t document it: its not ready nor safe for external usage. With everything mentioned here, there is no intention by SAP for anyone outside the core team building HANA XS to use this. If you do, don’t expect any support from SAP.  That includes if you use it and it really messes your system up. My advice, it might be fun but your playing with fire. Don’t ever do it on any system you really care about.

      (0) 
      1. Kel As

        Thank you, Thomas, for your very fair warning. I guess the shorter version of it is right in  “_experimental_” part of the configuration key.

        However, I might object that if XS Websocket API is good enough for XS Debugger (which is very much in production and supported as far as I can tell), why be shy and obscure experimental features from people, even if they are not officially supported?


        As you can see from this thread, developers’ curiosity is a terrifying force of nature. An attempt to outsmart them will only insult their intelligence, which is definitely not what SAP wants to see happening.

        Sometimes things go another way around. For example, as of SP08, [graph_engine] parameter is already in the official configuration, but there is no documentation on AIS/WIPE, neither AIS Workbench is in public access.

        As in:

        Call WIPE(‘CREATE WORKSPACE uri:okay_you_got_my_strict_attention;’)

        On a constructive note: I’d be delighted if you could give any useful pointers on WIPE before SP09 hits the shelves.

        (0) 
        1. Thomas Jung

          There is a difference between a feature being used in a one specialized use case internally and allowing developers to use it themselves. First of all the support and testing required for those two use cases is very different.  As Paul correctly points out there is also no pub/sub model; so websockets as a technical feature alone without proper support in the programming model is limited value. When we finally do release the feature it must have a way to really use it in the programming model in a proper way.

          (0) 
          1. Kel As

            Agreed, but this is a completely different point altogether. What you’re saying is although web sockets are there, there is absolutely nothing useful one can do with them in XS before the database core catches up.

            Postgres offers LISTEN/NOTIFY scheme driven by triggers, an elegant and simple model. Something for your colleagues to look at!

            So, no luck with WIPE?

            (0) 
  3. Holger SchΓ€fer

    HI guys,

    any news on this?

    The first talk about WS finding their way into HANA was with SP7. Now with SP10 nothing has really changed.

    SPS 9x Hint !!!

    BTW: Iy yoiu want to play with ws, currently (SP9) i am also able to use a custom xsws file inside the sap hana package:

    /sap/hana/xs/debugger/api/MyWebSocket.xsws

    Make sure, that your user has the needed privileges and the param

    the param

    xsengine.ini/debugger/enbaled=true

    is set.

    You can also use the included UI5 lib to connect to the service

    jQuery.sap.require(“sap.ui.core.ws.WebSocket”); 

    var oWsConnection = new sap.ui.core.ws.WebSocket(

      ‘ws://YOURDOMAIN:8000/sap/hana/xs/debugger/api/MyWebSocket.xsws’,

      ‘xsCrossfire’

    );

    I think the debugger has moved into the API package and therefore the whole package will internally be used/dispatched for WebServices which does not force you to overwrite Debugger.xsws

    @Thomas

    Concerning use case without pub/sub.

    I connect from HANA via SDA to y MSSQL DB getting logged users.

    We have an iPad app showing users currently inside the company.

    Using WS the app does not need to periodically (1 Minute) has to call the model service, instead the xsws can check this and if there is a delta, the new data can be directly pushed to the client.

    But i agree (and i am hopefully still waiting) for the complete business case with pub/subscribe down to db level.

    Best regards,

    Holger

    (0) 
    1. Thomas Jung

      We stopped all work on XSWS (XS specific implementation of Web Sockets). If you use this functionality, please note it will not move forward into the future. With the work to change over to Node.js in progress, it was no longer necessary to build our own event/web sockets framework into XSJS. Once we have Node.js as the foundation of XS, you can use any number of reusable libraries such as socket.io.

      (0) 
      1. Holger SchΓ€fer

        Hi Thomas,

        this is an amazing news!

        This will make a lot of things much easier for us (image processing/thumbnails), etc.

        – Is there a planned release date, when we can expect to use node.js code from inside HANA?

        – Do you have a link with any additional information in this? (maybe spidermonkey will also be replaced by V8)

        – Will there by fileysystem (or package) access from inside node? the rest API is not that performant for a lot of tasks.

        I saw that you are currently writing the delta docs for SPS10. Are there some cool new features inside XSJS? I saw the support for boolean/arays in the academy videos, but the HANA Help is maybe currently not updates to use this datatypes on CDS also?!?

        If there is a special slot/link for this, please let me know.

        This is an amazing news…

        (0) 
        1. Thomas Jung

          No I don’t have a timeline I can announce.  I can only make roadmap statements that this is planned and that work is already well in progress. Yes we are switching from SpiderMonkey to V8 (actually we will have both running parallel for a while for backwards compatibility). 

          >Do you have a link with any additional information in this

          Just the HANA Roadmap available on SMP which only has high level bullet points. Don’t expect more detail before TechEd this year.

          >Do you have a link with any additional information in this

          Repository access will only be via APIs much like today.  But the repository also goes through some major changes itself that I can’t go into details on yet.


          >Are there some cool new features inside XSJS?

          There was really no significant new developments in XSJS in SPS 10 because all the development teams were already hard at work on the major changes we have coming further in the future.

          (0) 
      2. Kelaskin Vladimir

        Hey Thomas,

        “HANA going node” is probably biggest news since “SAP going HANA” – XSJS API was a stillborn and ignorant idea right off the bat.

        I can only hope that you guys will abandon REGI at some point and switch to git (although they don’t directly compare – it is a stretch to call REGI a version control system in the proper sense). Git would be a natural move after you switch the dev ecosystem to node.

        Functionality-wise, REGI is somewhere between CVS and SVN; it is also a shameless rip-off in terms of RAM consumption. From what I can tell, HANA core development itself is very much git-based – so why sell something you wouldn’t buy yourself? πŸ™‚

        (0) 
  4. Rusheel Jain

    Hi

    Thanks for the awesome tutorial.

    Is there native support for websockets in XS or “hack” is still the unofficial official way πŸ˜€ πŸ˜€

    Cheers!
    Rusheel

    (0) 
    1. Gregor Wolf

      Hi Rusheel,

      starting with HANA SPS 11 you have XSA where you can deploy node.js. So this should be the official way for Web Sockets. Thomas Jung can you confirm that for XS classic nothing was changed?

      Best regards

      Gregor

      (0) 
      1. Thomas Jung

        Correct.  We never opened up the websockets in XSC beyond the debugger because about the time we would have started that project XSA came along and all development efforts switch to it.

        (0) 

Leave a Reply