Skip to Content
Author's profile photo Robert Waywell

Using HANA Smart Data Streaming with the HANA Developer Edition

Please note that the HANA Developer Edition has been retired. HANA streaming analytics (formerly HANA smart data streaming) is now included in HANA Express. For information on getting started with HANA streaming analytics using HANA Express, the following blog post is a good starting point:

 

New in SAP HANA, express edition: Streaming Analytics

Assigned Tags

      32 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Svenja Brunstein
      Svenja Brunstein

      Hi Robert,

      thanks for this great overview.

      I configured the hosts file as you described above and opened the ports in our firewall. But when starting a streaming project from HANA studio, this message is logged into the console:

      Failed call to:https://vhcalhdbdb:65535/RPC2 (Failed to read server's response: Unable to tunnel through proxy. Proxy returns "HTTP/1.0 404 Not Found")

      Any idea on what might cause this?

      Thanks,

      Svenja

      Author's profile photo Robert Waywell
      Robert Waywell
      Blog Post Author

      In HANA Studio, under Window->Preferences->General->Network Connections what is the Active Provider set to?

      HANA Studio - Network Preferences.png

      Mine is set to "Native", you may need to set yours to "Direct" to skip the Proxy.

      Author's profile photo Svenja Brunstein
      Svenja Brunstein

      Yep, thanks!

      Although it still shows an error (Failed call to:https://vhcalhdbdb:65535/RPC2 (Failed to read server's response: Connect timed out)), I was now able to connect to our system 🙂

      Author's profile photo Robert Waywell
      Robert Waywell
      Blog Post Author

      That's good. The open ports for the project to run on are actually 65532,65533 and 65534. We had to move off of 65535 due to a port conflict. So in your project configuration if you switch port 65535 to 65532 then you should be good.

      I'll update the linked Command Port configuration document to be consistent.

      Author's profile photo Former Member
      Former Member

      Hi, I install Smart data streaming plug in in my HANA studio(windows),

      but I cannot find any input/output adapter in my ccl diagram.

      Do you have any idea about this issue ?

      NullAdapter.PNG

      Author's profile photo Robert Waywell
      Robert Waywell
      Blog Post Author

      Hi Xiaochuan,

      In order to succesfully install the plugins on Windows, you need to start HANA Studio with "Run as Administrator" privileges. The symptom of not seeing the adapters show up in the palette usually indicates that HANA Studio was not started with "Run as Administrator" privileges when you installed the plugin.

      You should be able to resolve the issue by:

      1) Restart HANA Studio with "Run as Administrator" privileges

      2) Uninstall the SAP HANA smart data streaming Plugin

      3) You will need to restart HANA Studio for the uninstall to complete. Rather than letting Studio automatically restart, you want to manually exist Studio and then manually restart it so that you can start Studio with "Run as Administrator" privileges.

      4) You can now re-install the SAP HANA smart data streaming Plugin and the elements should properly show up in the palette.

      Author's profile photo Former Member
      Former Member

      Thank you for your solution, which works perfect ~~

      Now I can see all adapter pallette items.

      Author's profile photo Former Member
      Former Member

      Hi Robert:

      I'm using SDS to process data stream via continuously querying external database.

      And I referred to below guide:

      http://help.sap.com/Download/Multimedia/zip-hana_options_sds/streaming_adapters_guide.pdf

      In this document, variable ?ESP_MSDATE_VALUE is required to set in my SQL.

      I'm confused with it, is the variable's name fixed to ESP_MSDATE_VALUE?

      And when I use it to query, error occurs, I checked log file project.trace as follows:

      [23934]{-1}[-1/-1] 2016-03-07 16:46:42.187000 e Streaming UNKNOWN(-0001) : [SP-3-100005] (1.220) sp(23866) Error executing SQL statement SELECT  COLUMN1, COLUMN2, COLUMN3 FROM "WXC_AUTO" WHERE COLUMN3 > ?ESP_MSDATE_VALUE. java.sql.SQLException: Missing IN or OUT parameter at index:: 1

      Author's profile photo Remi ASTIER
      Remi ASTIER

      Hello Xiaochuan,

      Did you specify adapter property msdateSequenceColumn to COLUMN3 ?

      Could you switch to text authoring and paste the code for the adapter ?

      Remi.

      Author's profile photo Former Member
      Former Member

      Hi Remi:   Here is my project CCL:

      CREATE INPUT STREAM InputStream1 SCHEMA (   

        C1 bigint ,    

        C2 string ,   

        C3 msdate

      ) ;

      ATTACH INPUT ADAPTER Generic_DB_Input1 TYPE db_in TO InputStream1 PROPERTIES   

        service = 'ds_oracle' ,    

        query = 'SELECT  COLUMN1, COLUMN2, COLUMN3 FROM \"WXC_AUTO\" WHERE   COLUMN3 > ?ESP_MSDATE_VALUE' ,     table = '\"\".\"WXC_AUTO\"' ,    

        pollperiod = 10 ,   

        seconddateFormat = '%Y-%m-%d %H:%M:%S' ,    

        msdateFormat = '%Y-%m-%d %H:%M:%S' ,    

        permutation = '' ,   

        msdateSequenceColumn = 'COLUMN3' ,   

        recordCounterColumn = '' ,   

        msdateSequenceInitValue = '2015-01-01 16:07:10' ,   

        recordCounterInitValue = '0' ,    

        blockSize = 5000 ,    

        reconnectAttemptDelayMSec = 1000 ,   

        maxReconnectAttempts = 1 ;

      Author's profile photo Remi ASTIER
      Remi ASTIER

      Ok.

      When written manually, the code is easier to read. Could you try this code :

      ATTACH INPUT ADAPTER Generic_DB_Input1 TYPE db_in TO InputStream1

      PROPERTIES     service = 'ds_oracle' ,

           query = 'SELECT  COLUMN1, COLUMN2, COLUMN3 FROM "WXC_AUTO" WHERE COLUMN3 > ?ESP_MSDATE_VALUE ORDER BY COLUMN3' ,

           pollperiod = 10,

           msdateSequenceColumn = 'COLUMN3',

           msdateSequenceInitValue = '2015-01-01 16:07:10';

      Author's profile photo Former Member
      Former Member

      Hi Remi:

           Unfortunately, it still cannot work with your code. And same error occurs which can be

      checked in trace file.

          Actually I'm puzzled with this msdateSequenceColumn, does it mean the database table column name or Stream/Window schema column name ?

      oracle.PNG

      Author's profile photo Remi ASTIER
      Remi ASTIER

      Try this, it should create the query automatically :

      ATTACH INPUT ADAPTER Generic_DB_Input1 TYPE db_in TO InputStream1

      PROPERTIES     service = 'ds_oracle' ,

           table = 'WXC_AUTO',

           pollperiod = 10,

           msdateSequenceColumn = 'COLUMN3',

           msdateSequenceInitValue = '2015-01-01 16:07:10';

      Author's profile photo Former Member
      Former Member

      It is not OK. The error is:

      Error binding parameters to SQL statement select * from WXC_AUTO. java.sql.SQLException: Invalid column index.

      Author's profile photo Former Member
      Former Member

      I guess the point is how to pass last query msdate to SQL clause, in which a placeholder

      ? locates.

      I didn't see any explanation about it.

      Author's profile photo Former Member
      Former Member

      I think it is a mistake in the SDS guide.

      I checked the file "./STREAMING-1_0/bin/i18n/server/esp-internal-adapters-errors.properties", and found

      捕获.PNG

      So it should be ?ESP_TIMESTAMP_VALUE, but not ?ESP_MSDATE_VALUE.

      Author's profile photo Robert Waywell
      Robert Waywell
      Blog Post Author

      Hi Xiauchuan,

      Did you try changing the parameter name in the project to ESP_TIMESTAMP_VALUE? If so, did that resolve your issue?

      Thanks

      Author's profile photo Former Member
      Former Member

      Hi Robert:

         I did and ESP_TIMESTAMP_VALUE did resolve my issue.

      Author's profile photo Peter Hrebik
      Peter Hrebik

      Hi Robert,

      I am having problem with setting up the Data Services in Eclipse:

      Screen Shot 2016-04-07 at 13.33.29.png

      I created the service as follows:

      Screen Shot 2016-04-07 at 13.34.13.png

      Screen Shot 2016-04-07 at 13.34.58.png

      What am I missing here? I am running SAP HANA Dev edition in AWS. Any advice much appreciated!

      Author's profile photo Robert Waywell
      Robert Waywell
      Blog Post Author

      Hi Peter,

      What you should be seeing in the Add Service popup menu in your 2nd screen shot is an option to add a HANA Service. One possible reason why you aren't seeing the Add HANA Service option would be that you got an older version of the SDS plugin but if you got the plugin from the Dev Edition VM that is pretty unlikely.

      A 2nd possibility is that you may have installed the plugin without first starting Eclipse with Run as Administrator permissions on Windows. In that case you would need to restart Eclipse with Run as Administrator permissions, uninstall the SDS plugin,  restart Eclipse again Run as Administrator permissions to complete the uninstall and then reinstall the plugin.

      Can you confirm if you also have the HANA Administration Console perspective installed in your Eclipse environment? I don't know of any dependencies on other perspectives, but the Admin Console is one that would be installed by default in HANA Studio and which you may not have installed yet in your base Eclipse environment.

      Thanks

      Rob

      Author's profile photo Peter Hrebik
      Peter Hrebik

      Hi Robert,

      thanks for reply. I am still struggling with this.

      I am running Eclipse on Mac. I found the Hana studio plugins on Hana Dev Edition instance in the folder /hana/shared/HDB/hdbstudio_update. Can you confirm, are these the plugins you have been referring to? I was unable to install them on Mac. Eclipse (4.4 Luna) simply won't "pick them up". I was able to install them on Windows machine (Eclipse 4.5 Mars):

      Screen Shot 2016-04-08 at 12.11.52.png

      I did of course run the Eclipse as Administrator.

      After plugins installation, following perspectives have been added:

      Screen Shot 2016-04-08 at 12.57.07.png

      Streaming development perspectives are not present.

      I tried to install all the HANA plugins from https://tools.hana.ondemand.com/mars:

      Screen Shot 2016-04-08 at 12.56.44.png

      Streaming perspectives are still not present.

      Any idea what I am missing here? I know I can get the SDS plugins from SAP Service Marketplace but that's what I did in Eclipse on Mac. The result was missing "Add HANA service" menu item!

      Author's profile photo Robert Waywell
      Robert Waywell
      Blog Post Author

      Hi Peter,

      Can you go to Help->Installation Details and show me the Installation Details tab?

      HANA Studion - Installation Details.png

      Author's profile photo Peter Hrebik
      Peter Hrebik

      Hi Robert,

      sorry for late reply. I've been away for some time. Please see attached screenshot. Thanks in advance for looking into this.Screen Shot 2016-07-05 at 12.13.14.png

      Author's profile photo Robert Waywell
      Robert Waywell
      Blog Post Author

      To make sure I'm on the same page - you still aren't seeing the perspectives listed in the Open Perspective dialogue, correct?

      HANA Studio - Open Perspective.png

      Author's profile photo Peter Hrebik
      Peter Hrebik

      I can see them fine:

      Screen Shot 2016-07-05 at 15.20.17.png

      Author's profile photo Robert Waywell
      Robert Waywell
      Blog Post Author

      When you select the SAP HANA Streaming Development perspective, does it open up? If so, then I think you have it working now.

      Author's profile photo Peter Hrebik
      Peter Hrebik

      Yes I get the SAP Hana Streaming Development perspective open ok, however I still cannot setup the Data Services (see my initial comment). I am not able to see the "Add HANA" service menu item on the right-click menu. I cannot see it on Windows nor Mac. Any idea?

      Author's profile photo Peter Hrebik
      Peter Hrebik

      Hi Robert,

      Ok, so I did update my Smart data streaming plugin to 1.10.12.v201605261044. I am still not able to see the "Add Hana service" menu item. I did terminate the SAP Hana Dev edition SPS10 and created new SAP Hana Dev SPS11 instance.

      1. I successfully added the system in the SAP HANA Administration Console
      2. I can connect to the system fine
      3. In the SAP Hana Streaming Run-Test perspective I've added new server connection (esps://<MY HOST>:30026):

        Host name: the IP of the SAP Hana Dev Instance

      Port: 33026

      Protocol: Native

      SSL: enabled

      Authentication: User/Password

                  4. In the SAP Hana Streaming Development perspective I am not able to see the "Add Hana service" menu item

      How can I connect to the streaming server? I tried to created ODBC service with the ODBC DSN: hanaservice with no success.

      Please help. I am having exactly same issue on Mac and Windows. The SAP Hana Dev instance comes with the streaming server already pre-installed, also the streaming server shows as running in the management console. Any idea what I am missing here?

      Author's profile photo Robert Waywell
      Robert Waywell
      Blog Post Author

      At this point it isn't obvious what is wrong. The fact that you can get as far as the pop up menu to add a data service shows that the plug in is installed and you are connected to the SDS server. We've already checked that you have the SP10 plugin and since you are connecting to the SP11 Dev Edition server we know that you are connecting to an SDS instance that provides the HANA Service feature.

      Since you have moved from the SP10 Dev Edition to the SP11 Dev Edition, you can try upgrading the SDS plugin to SP11. That would be recommended regardless as there are SP11 features that aren't supported by the SP10 SDS Studio plug in such as the machine learning functions.

      I will try to find time to test this with a plain Eclipse install to see if I can reproduce it.

      Author's profile photo Peter Hrebik
      Peter Hrebik

      Hi Robert,

      just to let you know I managed to solve my issue by uninstalling the Eclipse and instead of installing HANA SDS plugins into existing Eclipse IDE, I downloaded SAP Hana Studio 2 from SAP Marketplace. After installing SAP hdstudio 2 all works fine!

      Thanks

      Peter

      Author's profile photo Biswanath Sahu
      Biswanath Sahu

      I am looking for the Eclipse plugin for SDS.

      How can I download the plugin installation from /hana/shared/HDB/streaming of the Cloud developer edition to my local machine.

      Will really appreciate your help!!

      Thanks and Best Regards,

      Biswa

      Author's profile photo Tumulu Tripati Subudhi
      Tumulu Tripati Subudhi

      You can use Filezilla to connect to your AWS instance to download the plugin.

      Please note if the AWS EC2 instance SUSE Linus, then the user is root

      Please refer Connect to Amazon EC2 file directory using Filezilla and SFTP - Stack Overflow