Skip to Content
Technical Articles
Author's profile photo Ralph Oliveira

SAP HANA Real Time Sentiment Analysis and Text Mining app

You want it you got it!

 

Finally we are publishing all HANA apps that all of us, B1 Solution Architects, developed to demonstrate how far you can go using SAP HANA!

 

If you are seeing this application for the first time, you didn’t go to one of the Business One Innovation Summits 2015 (Bangkok, Barcelona or Miami). Shame on you and you have to be there next year.

 

This application implements 2 interesting HANA functionalities:

 

1 – SAP HANA Text Analysis

 

This functionality allows us to handle “Unstructured Data” (in our case tweets). This feature classifies terms of a given input in several categories (People, companies, locations etc). Its is also possible to extract the whole meaning of an unstructured text and classify in a Positive or Negative Sentiment.

2 – SAP HANA Text Mining (Available on SPS09 only)

 

This feature works with a concept called Bag of words not only acting at a document level (tweet) but a whole set of documents (an entire table of tweets for example).

 

With those features we can classify documents relevancy for a given input. E.g. give me the tweets that are more relevant for the term “Steve Jobs” or suggest me terms that are related to it (“apple”).

 

Lets see the app running in this conceptual demonstration:

 

 

App Installation (SPS08 or higher required)

 

As Sally’s just explained on the video. This solution in composed by 2 applications:

 

You can find all installation files on Github Release

Or clone the repository

 

1 – Structurer One (The HANA App)

 

You just have to import the Delivery Unit (file .tgz) on your HANA System to have all the App structure set up. If you don’t know how to work with DUs, use this simple example of how to import it as a guide.

As by the time of this blog, XS Engine has no Text Analysis APIs, the DU import might fail the first time and only import part of the components. This ok.

After the import (even partially), on the development perspective of HANA Studio, check the software repositories and execute all the SQL commands that are listed on the file Summit15 > SQL > CreateIndexes.sql. These commands will Activate Text Analysis features on the app.

If you got the initial error, re-import the delivery unit again and now you should get no error messages.

 

http://i.imgur.com/iV3o1pF.png

 

 

 

The app will be at:

http://<yourserver>:8000/Summit15/UI/index.html

http://i.imgur.com/67zI9ou.png

2 – Tweets Retriever

 

To the app works completely, we need data. Of course you can input it manually, but the whole idea is to perform Real Time Analysis.

So, to gather tweets in real time we have a script written in Server Side JavaScript and runs on a framework called Node.JS.

You can install this framework in any OS and run the tweet retriever from your laptop for example.

 

This is a small script is responsible to listen the Twitter API and every time a new tweet comes up, it will be stored on a HANA Server.

 

To run it:

  • Follow this steps
  • Open the node script (twitterSummit.js) and fill your twitter keys and HANA server information.

 

 

Run the Tweets Retriever scripts with the commands

  • node twitterSummit.js

 

And it should work like this:

 

Final Considerations

 

This app is, off course, for demonstration purposes. It is 100% open source and can be enhanced or modified accordingly. It was developed under a didactical perspective, in a matter that you could have a comprehensive experience understanding each step.

 

One point to be emphasized it’s the amount of free third party resources that I used here to show you that, once working on HANA, we are free to work with a infinite of libraries and resources.
Examples:

 

SAP embraces and support (a lot) open source community. You can see several examples on SAP GitHub Repository. Here we are using the HANA Node driver that is a example of it and I bet you will hear more and more about this in a near the future.

 

Let me know what you think and don’t forget to follow me on twitter! Follow @Ralphive

 

Cheers!

 

Ralph

Assigned Tags

      9 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Former Member
      Former Member

      Dear Ralph,

      How are you!

      I'm wondering Is there any tutorial lessons about how to build such an application step bu step? that will be great helpful !

      Best Regards

      Joe

      Author's profile photo Former Member
      Former Member

      Amazing!

      Thanks for your job.

      Regards.

      Author's profile photo Ralph Oliveira
      Ralph Oliveira
      Blog Post Author

      Tks, Santiago!

      Now fork the code and implement it yourself 🙂

      Author's profile photo Former Member
      Former Member

      Dear Ralph,

      Hope you are well.

      I am not able to locate (brands.js, sapb1.js and twitterSummit.js) to fill twitter keys into Summit15 package which I imported from drop box.

      Could you please help me in this.

      Thanks,

      Yogesh Kumar

      Author's profile photo Ralph Oliveira
      Ralph Oliveira
      Blog Post Author

      Hi,

      both on Git repository and on the zip file, there is a folder called DataRetriever.

      StructurerOne/DataRetriever at master · Ralphive/StructurerOne · GitHub

      Author's profile photo Diana Rincon
      Diana Rincon

      Hi Ralph,

      Nice exercise, Can I replicate this on HCP? what type of privileges do I need to import the DU?

      Thanks,

      Diana

      Author's profile photo Ralph Oliveira
      Ralph Oliveira
      Blog Post Author

      The XS app can runs entirely on HCP.

      The twitter retriever runs on NodeJS not (yet) supported on this platform. So you still need a "NodeJs Server" can be your own laptop.

      To import the DU on HCP is the same method on Hana on premise.. You need the LM roles to access it: http://help-legacy.sap.com/saphelp_hanaplatform/helpdata/en/af/092a4dd0f2422193af9e9295ca18a0/content.htm

      Author's profile photo Diana Rincon
      Diana Rincon

      Greate Ralph, thanks a lot I will check on this!!

       

      Author's profile photo Gabriela López
      Gabriela López

      I´m having this error now ...

      Illegal vendor name "SAPB1_SOL_ARCHITECTS".

      lol