Using HANA Smart Data Streaming with the HANA Developer Edition
Please note that the HANA Developer Edition has been retired. HANA streaming analytics (formerly HANA smart data streaming) is now included in HANA Express. For information on getting started with HANA streaming analytics using HANA Express, the following blog post is a good starting point:
Hi Robert,
thanks for this great overview.
I configured the hosts file as you described above and opened the ports in our firewall. But when starting a streaming project from HANA studio, this message is logged into the console:
Failed call to:https://vhcalhdbdb:65535/RPC2 (Failed to read server's response: Unable to tunnel through proxy. Proxy returns "HTTP/1.0 404 Not Found")
Any idea on what might cause this?
Thanks,
Svenja
In HANA Studio, under Window->Preferences->General->Network Connections what is the Active Provider set to?
Mine is set to "Native", you may need to set yours to "Direct" to skip the Proxy.
Yep, thanks!
Although it still shows an error (Failed call to:https://vhcalhdbdb:65535/RPC2 (Failed to read server's response: Connect timed out)), I was now able to connect to our system 🙂
That's good. The open ports for the project to run on are actually 65532,65533 and 65534. We had to move off of 65535 due to a port conflict. So in your project configuration if you switch port 65535 to 65532 then you should be good.
I'll update the linked Command Port configuration document to be consistent.
Hi, I install Smart data streaming plug in in my HANA studio(windows),
but I cannot find any input/output adapter in my ccl diagram.
Do you have any idea about this issue ?
Hi Xiaochuan,
In order to succesfully install the plugins on Windows, you need to start HANA Studio with "Run as Administrator" privileges. The symptom of not seeing the adapters show up in the palette usually indicates that HANA Studio was not started with "Run as Administrator" privileges when you installed the plugin.
You should be able to resolve the issue by:
1) Restart HANA Studio with "Run as Administrator" privileges
2) Uninstall the SAP HANA smart data streaming Plugin
3) You will need to restart HANA Studio for the uninstall to complete. Rather than letting Studio automatically restart, you want to manually exist Studio and then manually restart it so that you can start Studio with "Run as Administrator" privileges.
4) You can now re-install the SAP HANA smart data streaming Plugin and the elements should properly show up in the palette.
Thank you for your solution, which works perfect ~~
Now I can see all adapter pallette items.
Hi Robert:
I'm using SDS to process data stream via continuously querying external database.
And I referred to below guide:
http://help.sap.com/Download/Multimedia/zip-hana_options_sds/streaming_adapters_guide.pdf
In this document, variable ?ESP_MSDATE_VALUE is required to set in my SQL.
I'm confused with it, is the variable's name fixed to ESP_MSDATE_VALUE?
And when I use it to query, error occurs, I checked log file project.trace as follows:
[23934]{-1}[-1/-1] 2016-03-07 16:46:42.187000 e Streaming UNKNOWN(-0001) : [SP-3-100005] (1.220) sp(23866) Error executing SQL statement SELECT COLUMN1, COLUMN2, COLUMN3 FROM "WXC_AUTO" WHERE COLUMN3 > ?ESP_MSDATE_VALUE. java.sql.SQLException: Missing IN or OUT parameter at index:: 1
Hello Xiaochuan,
Did you specify adapter property msdateSequenceColumn to COLUMN3 ?
Could you switch to text authoring and paste the code for the adapter ?
Remi.
Hi Remi: Here is my project CCL:
CREATE INPUT STREAM InputStream1 SCHEMA (
C1 bigint ,
C2 string ,
C3 msdate
) ;
ATTACH INPUT ADAPTER Generic_DB_Input1 TYPE db_in TO InputStream1 PROPERTIES
service = 'ds_oracle' ,
query = 'SELECT COLUMN1, COLUMN2, COLUMN3 FROM \"WXC_AUTO\" WHERE COLUMN3 > ?ESP_MSDATE_VALUE' , table = '\"\".\"WXC_AUTO\"' ,
pollperiod = 10 ,
seconddateFormat = '%Y-%m-%d %H:%M:%S' ,
msdateFormat = '%Y-%m-%d %H:%M:%S' ,
permutation = '' ,
msdateSequenceColumn = 'COLUMN3' ,
recordCounterColumn = '' ,
msdateSequenceInitValue = '2015-01-01 16:07:10' ,
recordCounterInitValue = '0' ,
blockSize = 5000 ,
reconnectAttemptDelayMSec = 1000 ,
maxReconnectAttempts = 1 ;
Ok.
When written manually, the code is easier to read. Could you try this code :
ATTACH INPUT ADAPTER Generic_DB_Input1 TYPE db_in TO InputStream1
PROPERTIES service = 'ds_oracle' ,
query = 'SELECT COLUMN1, COLUMN2, COLUMN3 FROM "WXC_AUTO" WHERE COLUMN3 > ?ESP_MSDATE_VALUE ORDER BY COLUMN3' ,
pollperiod = 10,
msdateSequenceColumn = 'COLUMN3',
msdateSequenceInitValue = '2015-01-01 16:07:10';
Hi Remi:
Unfortunately, it still cannot work with your code. And same error occurs which can be
checked in trace file.
Actually I'm puzzled with this msdateSequenceColumn, does it mean the database table column name or Stream/Window schema column name ?
Try this, it should create the query automatically :
ATTACH INPUT ADAPTER Generic_DB_Input1 TYPE db_in TO InputStream1
PROPERTIES service = 'ds_oracle' ,
table = 'WXC_AUTO',
pollperiod = 10,
msdateSequenceColumn = 'COLUMN3',
msdateSequenceInitValue = '2015-01-01 16:07:10';
It is not OK. The error is:
Error binding parameters to SQL statement select * from WXC_AUTO. java.sql.SQLException: Invalid column index.
I guess the point is how to pass last query msdate to SQL clause, in which a placeholder
? locates.
I didn't see any explanation about it.
I think it is a mistake in the SDS guide.
I checked the file "./STREAMING-1_0/bin/i18n/server/esp-internal-adapters-errors.properties", and found
So it should be ?ESP_TIMESTAMP_VALUE, but not ?ESP_MSDATE_VALUE.
Hi Xiauchuan,
Did you try changing the parameter name in the project to ESP_TIMESTAMP_VALUE? If so, did that resolve your issue?
Thanks
Hi Robert:
I did and ESP_TIMESTAMP_VALUE did resolve my issue.
Hi Robert,
I am having problem with setting up the Data Services in Eclipse:
I created the service as follows:
What am I missing here? I am running SAP HANA Dev edition in AWS. Any advice much appreciated!
Hi Peter,
What you should be seeing in the Add Service popup menu in your 2nd screen shot is an option to add a HANA Service. One possible reason why you aren't seeing the Add HANA Service option would be that you got an older version of the SDS plugin but if you got the plugin from the Dev Edition VM that is pretty unlikely.
A 2nd possibility is that you may have installed the plugin without first starting Eclipse with Run as Administrator permissions on Windows. In that case you would need to restart Eclipse with Run as Administrator permissions, uninstall the SDS plugin, restart Eclipse again Run as Administrator permissions to complete the uninstall and then reinstall the plugin.
Can you confirm if you also have the HANA Administration Console perspective installed in your Eclipse environment? I don't know of any dependencies on other perspectives, but the Admin Console is one that would be installed by default in HANA Studio and which you may not have installed yet in your base Eclipse environment.
Thanks
Rob
Hi Robert,
thanks for reply. I am still struggling with this.
I am running Eclipse on Mac. I found the Hana studio plugins on Hana Dev Edition instance in the folder /hana/shared/HDB/hdbstudio_update. Can you confirm, are these the plugins you have been referring to? I was unable to install them on Mac. Eclipse (4.4 Luna) simply won't "pick them up". I was able to install them on Windows machine (Eclipse 4.5 Mars):
I did of course run the Eclipse as Administrator.
After plugins installation, following perspectives have been added:
Streaming development perspectives are not present.
I tried to install all the HANA plugins from https://tools.hana.ondemand.com/mars:
Streaming perspectives are still not present.
Any idea what I am missing here? I know I can get the SDS plugins from SAP Service Marketplace but that's what I did in Eclipse on Mac. The result was missing "Add HANA service" menu item!
Hi Peter,
Can you go to Help->Installation Details and show me the Installation Details tab?
Hi Robert,
sorry for late reply. I've been away for some time. Please see attached screenshot. Thanks in advance for looking into this.
To make sure I'm on the same page - you still aren't seeing the perspectives listed in the Open Perspective dialogue, correct?
I can see them fine:
When you select the SAP HANA Streaming Development perspective, does it open up? If so, then I think you have it working now.
Yes I get the SAP Hana Streaming Development perspective open ok, however I still cannot setup the Data Services (see my initial comment). I am not able to see the "Add HANA" service menu item on the right-click menu. I cannot see it on Windows nor Mac. Any idea?
Hi Robert,
Ok, so I did update my Smart data streaming plugin to 1.10.12.v201605261044. I am still not able to see the "Add Hana service" menu item. I did terminate the SAP Hana Dev edition SPS10 and created new SAP Hana Dev SPS11 instance.
Host name: the IP of the SAP Hana Dev Instance
Port: 33026
Protocol: Native
SSL: enabled
Authentication: User/Password
4. In the SAP Hana Streaming Development perspective I am not able to see the "Add Hana service" menu item
How can I connect to the streaming server? I tried to created ODBC service with the ODBC DSN: hanaservice with no success.
Please help. I am having exactly same issue on Mac and Windows. The SAP Hana Dev instance comes with the streaming server already pre-installed, also the streaming server shows as running in the management console. Any idea what I am missing here?
At this point it isn't obvious what is wrong. The fact that you can get as far as the pop up menu to add a data service shows that the plug in is installed and you are connected to the SDS server. We've already checked that you have the SP10 plugin and since you are connecting to the SP11 Dev Edition server we know that you are connecting to an SDS instance that provides the HANA Service feature.
Since you have moved from the SP10 Dev Edition to the SP11 Dev Edition, you can try upgrading the SDS plugin to SP11. That would be recommended regardless as there are SP11 features that aren't supported by the SP10 SDS Studio plug in such as the machine learning functions.
I will try to find time to test this with a plain Eclipse install to see if I can reproduce it.
Hi Robert,
just to let you know I managed to solve my issue by uninstalling the Eclipse and instead of installing HANA SDS plugins into existing Eclipse IDE, I downloaded SAP Hana Studio 2 from SAP Marketplace. After installing SAP hdstudio 2 all works fine!
Thanks
Peter
I am looking for the Eclipse plugin for SDS.
How can I download the plugin installation from /hana/shared/HDB/streaming of the Cloud developer edition to my local machine.
Will really appreciate your help!!
Thanks and Best Regards,
Biswa
You can use Filezilla to connect to your AWS instance to download the plugin.
Please note if the AWS EC2 instance SUSE Linus, then the user is root
Please refer Connect to Amazon EC2 file directory using Filezilla and SFTP - Stack Overflow