Skip to Content
Author's profile photo Andreas Schuster

SAP HANA Blockchain: Setup SAP HANA, express edition for HANA blockchain

This blog describes how to set up SAP HANA, express edition for use with SAP HANA blockchain. Although it focuses on HANA Express, most of the steps below apply to any full-size HANA database as well.

Check out the other parts of my blog series on SAP HANA blockchain as well to get a conclusive overview about HANA blockchain.

Part 1: SAP HANA Blockchain: An Introduction

Part 2: SAP HANA Blockchain: Technical Overview

Part 3: SAP HANA Blockchain: Setup SAP HANA, express edition for HANA blockchain

Prerequisites

Before we get started, a brief overview about the prerequisites and tools that we will be working with. You will need:

  • An instance of SCP Blockchain service on SAP Cloud Platform– either MultiChain or Hyperledger fabric.
  • The SAP HANA Integration Service, also on SAP Cloud Platform.
  • SAP HANA, express edition. Of course, any other SAP HANA database will work just as well.
  • SDI Agent, version 2.2.4 or higher.
  • SAP HANA Blockchain Adapter.

Step 1: SAP HANA, express edition

In this tutorial, I am using SAP HANA, express editionHANA express in short. HANA express is a leaner version of SAP HANA, but provides almost the same feature set as SAP HANA, with a generally lower memory footprint. Another huge advantage is that it comes free-of-charge for small installations. This means that you can try HANA Blockchain even if you do not have access to your own full-size SAP HANA database.

There are a few options to deploy HANA express. Virtual machine images and binary installers (for bare metal installations) are available from SAPs portal page. There are also Docker containers in the docker store (with XSA or without XSA).

The setup process of all options is well documented, so I will not re-invent the wheel here. I will instead provide links that will guide you through the setup process of the different options. A HANA-only installation (that is, without XSA) is sufficient, but make sure to select Smart Data Integration if you’re prompted.

Official Documentation

Virtual Machine

Binary Installer

Docker Container

Step 2: Data Provisioning Agent (Smart Data Integration)

If everything was set up correctly during the HANA Express installation, the Data Provisioning Server should already be enabled. If not, you can follow the steps described in the Installation and Configuration Guide for SDI.

The SPS 03 version of SAP HANA Express includes an SDI Agent as well. Unfortunately for us, it is version 2.2.3, while SAP HANA blockchain requires at least version 2.2.4. Therefore, you will need to update the agent to a more recent version. The section “Install the Data Provisioning Agent” in the SDI Installation Guide describes how this is done. In this tutorial, I am setting up the Data Provisioning Agent on the same host as SAP HANA Express.

Download the SDI Agent

You’ll need two files for this step. The SDI Agent itself and SAPCAR to unpack the Agent.

  1. Go to the SDI section on the SAP Support Launchpad.
  2. Navigate:
    • SAP HANA SDI 2.0
    • Comprised Software Component Versions
    • HANA DP Agent 2.0
  3. Download the latest version. For example: IMDB_DPAGENT200_03P_10-70002516.SAR

Extract the downloaded archive

  1. Download SAP CAR from the SAP Support Launchpad, e.g., SAPCAR_1014-80000935.EXE.
  2. Assign execution rights:
chmod +x SAPCAR_1014-80000935.EXE
  1. Extract SDI Agent archive
./SAPCAR_1014-80000935.EXE -xvf IMDB_DPAGENT200_03P_10-70002516.SAR

Install the new SDI Agent

  1. Create the installation directory:
sudo mkdir /usr/sap/dataprovagent
sudo chown <youruser> /usr/sap/dataprovagent
  1. Start the installation
HANA_DP_AGENT_20_LIN_X86_64/hdbinst

The output should look similar to the screenshot below. The data provisioning agent is installed to /usr/sap/dataprovagent by default.

Configure the SDI Agent and deploy the SAP HANA Blockchain adapter

After successful installation of a compatible SDI agent, the agent will need some configuration. To do so, you require a HANA database user with at least the system privileges ADAPTER ADMIN and AGENT ADMIN.

The configuration tool is available on the command line, as well as with a graphical user interface. The GUI version requires an X11 server if started remotely, as well as X11 forwarding enabled in your SSH client.

The screenshot above shows the command line version of the configuration tool. Execute the steps below and follow the instructions to get the data provisioning agent set up for HANA blockchain.

  1. Start the command line tool in configuration mode
/usr/sap/dataprovagent/bin/agentcli.sh --configAgent
  1. [Option 2] Start the agent.
  2. [Option 5] Connect the agent to your SAP HANA Express installation.
  3. [Option 6] Register the agent with your SAP HANA Express installation.
  4. [Option 8] Deploy the SAP HANA blockchain adapter as a custom adapter.
  5. [Option 7] Register the adapter.

Once this has be completed, move on to SAP HANA Studio or the Database Explorer to complete the configuration on the SAP HANA database side.

Step 3: Connect SAP HANA to SCP Blockchain

To finalize the configuration and make the SCP blockchain instance available to SAP HANA, we will first create a remote source and then map this remote source into a database schema via virtual tables. Persisting blockchain data into SAP HANA is achieved via a remote subscription that regularly polls the SCP blockchain service and persists new records in SAP HANAs column store.

Before you can do that, you will have to configure the SCP side of SAP HANA blockchain via the SAP HANA integration service. This is outlined in my previous blog post on SAP HANA blockchain.

More information on the individual steps is available in the official documentation.

3.1 Create Remote Source

The first step is to create a remote source in SAP HANA. To do so, you will need the following information. Most of it must be extracted from the service key of the SAP HANA integration service as described at the very end of my previous blog post.

Blockchain URL

e.g., https://blockchain-service.cfapps.eu10.hana.ondemand.com/blockchain

IMPORTANT: Append “/hanaintegration/api/v1” to the extracted URL

Access token URL

e.g., https://hanabccuv.authentication.eu10.hana.ondemand.com

IMPORTANT: Append “/oauth” to the extracted URL

Polling Interval Defines how often polling operations run to check for new data on the blockchain. Default is 5 seconds.
Proxy Host Only required if a proxy is needed for SAP HANA to connect to SCP Blockchain.
Proxy Port
Credentials Mode Choose “Technical User”.
Client ID e.g., “sb-7d165a60-…-420adfc9-f96e-4090-a650-0386988b67e0!b1836“
Client Secret e.g., “TJmM…Wsg=”

Creating the remote source can be done graphically in the Database Explorer, but it is also possible via SQL. A database user needs the CREATE REMOTE SOURCE system privilege to create a remote source. Examples of the various options are displayed below.

Adding a remote source in the Database Explorer

Adding a remote source via SQL

-- create remote source
CREATE REMOTE SOURCE "<NAME_OF_NEW_REMOTE_SOURCE>"
       ADAPTER "BlockchainAdapter"
       AT LOCATION AGENT "<NAME_OF_DPAGENT>"
CONFIGURATION
'<?xml version="1.0" encoding="UTF-8"?>
        <ConnectionProperties name="Connection">
                <PropertyEntry name="url">[Blockchain URL]/hanaintegration/api/v1</PropertyEntry>
                <PropertyEntry name="accessTokenURL">[Access token URL]/oauth</PropertyEntry>
                <PropertyEntry name="pollingInterval">[Polling Interval]</PropertyEntry>
                <PropertyEntry name="proxyHostName">[Proxy Hostname]</PropertyEntry>
                <PropertyEntry name="proxyPortNumber">[Proxy Port]</PropertyEntry>
        </ConnectionProperties>'
WITH CREDENTIAL TYPE 'PASSWORD' USING
'<CredentialEntry name="credential">
        <user>[Client ID]</user>
        <password>[Client Secret]</password>
</CredentialEntry>';

3.2 Create Virtual Table

Once the remote source is created, virtual tables are required to access the data it references. The virtual tables map the remote sources into a database schema on SAP HANA, from which they can be accessed by, e.g., SQL queries.

There are three types of tables:

  • Raw data: Information about blocks and transactions.
  • Ledger: Individual messages; allow access to the whole history of transactions that were committed to the blockchain.
  • Worldstate: Only the latest valid tuples that are known to the blockchain. A worldstate table looks and behaves in many ways just like a regular database table.Querying a “*_worldstate” virtual table always returns an empty result set. This behavior is intentional. To see the data, you will need to persist the data into SAP HANA first via a remote subscription

The virtual tables are created with a few clicks in the Database Explorer, as displayed below.

In my example I’m using a simple data model called INHABITANTS. The configuration of this data model in the HANA integration service (see also my previous blog) is displayed below.

Virtual tables can be named arbitrarily. In my case I chose

  • inhabitants.inhabitant for the ledger table, and
  • inhabitants.inhabitant_worldstate for the worldstate table.

3.3 Create Remote Subscription

The virtual tables we created in previous step are – as the name already suggests – only a virtual link to the actual data. This means that no data is actually persisted on SAP HANA, but the data is instead retrieved remotely from the blockchain with every access. Especially for large data volumes and performance-critical applications, this is most likely not the most efficient way of working with your data. The steps below describe how to setup a remote subscription to persist blockchain data physically in SAP HANAs column store.

Step 1: Create physical column store table, based on the virtual worldstate table.

-- create table for replicated data
CREATE COLUMN TABLE INHABITANTS_WORLDSTATE_REPLICA LIKE "inhabitants.inhabitant_worldstate";

Step 2: Create the remote subscription that pulls data from the blockchain.

-- create a remote subscription that pulls all data from the world state table
CREATE REMOTE SUBSCRIPTION INHABITANTS_WORLDSTATE_SUB
    AS (SELECT * FROM "inhabitants.inhabitant_worldstate")
    TARGET TABLE INHABITANTS_WORLDSTATE_REPLICA;

Step 3: Start queuing and distributing data.

-- start recording data
ALTER REMOTE SUBSCRIPTION INHABITANTS_WORLDSTATE_SUB QUEUE;

-- start distributing data
ALTER REMOTE SUBSCRIPTION INHABITANTS_WORLDSTATE_SUB DISTRIBUTE;

After executing these steps, the physical worldstate table (in this example INHABITANTS_WORLDSTATE_REPLICA) is kept in sync with the blockchain and can be read just like any other column store table.

Step 4: Testing SAP HANA Blockchain

The queries below can be used to test if everything was setup correctly.

DML:

Before we start, a few things to keep in mind.

  • DML statements will not change any transactions that were written to the blockchain before. Instead, they will append a new record with updated values to the blockchain.
  • DELETE statements will flag a record on the blockchain as deleted, but not physically remove the record.
-- insert new record
INSERT INTO "inhabitants.inhabitant_worldstate"
       ("SVNR", "firstName", "lastName", "birthTimestamp", "height", "married")
VALUES ('123456789', 'Simpson', 'Homer', '1956-05-12', '1.75', true);

-- update record
UPDATE "inhabitants.inhabitant_worldstate"
SET "firstName" = 'Abraham'
WHERE "SVNR" = '123456789';

-- delete record
DELETE FROM "inhabitants.inhabitant_worldstate" WHERE "SVNR" = '123456789';

SQL: Query on virtual ledger table

-- query entire transaction history from virtual table
SELECT * FROM "inhabitants.inhabitant";

If you have executed the previous DML statements in the displayed order (INSERT, UPDATE, DELETE), a SELECT on the virtual ledger table will lead to the result below.

SQL: Query on physical worldstate table

-- query worldstate from virtual table --> ALWAYS EMPTY BY DESIGN
SELECT * FROM "inhabitants.inhabitant_worldstate";

A query on the virtual worldstate table will always return an empty result set.

-- query replicated worldstate
SELECT * FROM "INHABITANTS_WORLDSTATE_REPLICA";

A query on the physical worldstate table returns only the latest version. Since we deleted the record before, this is in this case also an empty result set.

Of course, these are just some short and very simple examples on how blockchain data can be accessed by SAP HANA. Since all data is in regular column store tables, the whole feature set of the HANA column store is available for these tables. This includes all advanced analytic engines, like graph and geospatial, and the usage with, for example, Calculation Views.

Assigned Tags

      6 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Former Member
      Former Member

      Hi Andreas,

      great series and great timing with it. i just want to clarify one detail. are all the dependencies publicly available or are there any that require some kind of NDA or beta program sign-up?

      thank you, greg

      Author's profile photo Chan Jin Park
      Chan Jin Park

       

      Hi Andreas,

      Many thanks your insight for the detail step. I got stuck in [Option 5] Connect the agent to your SAP HANA Express

      2018-10-22 23:44:45,205 [ERROR] SocketConnector.read  - Failed to connect to server SAP DBTech JDBC: Cannot connect to jdbc:sap:

       

      Agent                    Status: Running
      Listener port: 5050
      Admin port: 5051
      Agent Name:
      HANA Connection          Server: xx.xx.xx.xx
      Port: 30015
      Protocol: TCP
      Agent Admin HANA user:
      ************************************************************

       

      does it require to change my DNS ?

      Author's profile photo Christophe Leske
      Christophe Leske

      Hello,

      there has been a silent addition of a parameter named "Block Fetchsize" to the Add Remote Source dialog I am wondering about. There is no documentation for it whatsoever.

      What does it refer to, and what is it for?

      Any insight would be much appreciated,

      Best,

      Christophe Leske

      Author's profile photo Former Member
      Former Member

      Hi Christophe,

      i don't have an answer for you, but i would investigate how this new parameter relates to the standard fabric command 'peer channel fetch'. i'm presuming that SAP may have added API's to allow fetching of more than one block, but this needs to be confirmed by the source, i.e. SAP themselves.

      in any case, the most recent syntax for the fetch as of 1.3 is as follows:

      peer channel fetch <newest|oldest|config|(block number)> [flags].

      just a hunch, anyway.

      thx, greg

      Author's profile photo Christophe Leske
      Christophe Leske

      Hello Gregory,

       

      thanks for the feedback, that does make sense... I still would like to get an official statement on this if possible.

      Author's profile photo Christophe Leske
      Christophe Leske

      I would also like to add that updating a record in either the replicated table or the virtual table does NOT create an additional block with the changes for me.

      There is no reapplication of changes done in SAP HANA to the blockchain.

      Example:

      current block count in my blockchain before doing any changes is 9797 Blocks.

      I do on the virtual table:

       

      Update “HL_5fcff071-51fd-4eed-8966-b6eb22c1f3fb-com-camelot-energy-market.ProductOffering_worldstate”
      set “buyerId”=”
      where “key”=’ffCoMStEfM’;

      Result:
      Statement ‘Update …’
      executed in 837 ms.

       

      I am not sure if this was right, so I redo this on the replicated table:

      Update "PRODUCTOFFERING_WORLDSTATE_REPLICA"
      set "buyerId"=''
      where "key"='ffCoMStEfM';
      
      Result:
      Statement 'Update "PRODUCTOFFERING_WORLDSTATE_REPLICA" set "buyerId"='' where "key"='ffCoMStEfM''
      executed in 17 ms - Rows Affected: 1
      
      
      

      I wait a couple of minutes, yet the blockchain's block count doesn't move - it is still at 9747. According to the article, the change in the table should produce a new block with the modifications:

      Quotation:

      “Before we start, a few things to keep in mind.

      • DML statements will not change any transactions that were written to the blockchain before. Instead, they will append a new record with updated values to the blockchain.
      • DELETE statements will flag a record on the blockchain as deleted, but not physically remove the record”

      I am sorry, but this is not what I see here, this feature seems to be broken. As of now, only unidirectional data exchange possible FROM the blockchain TO the SAP HANA database, but not the other way round. This severly limits the possibilities of the SAP HANA blockchain integration.