Introduction

SAP introduced a RESTful Web Service SDK in 4.X environments to interact with the BI platform and its components. In previous 3.X versions, you were tied into using the Java and .NET SDKs which were tedious to set up and correctly configure an environment for. By moving to a RESTful Web Service, SAP has unlocked the potential to communicate with your BI platform via any language of your choice, assuming it supports making HTTP requests. The SAP BusinessObjects RESTful Web Service SDK User Guide for Web Intelligence and the BI Semantic Layer does a fantastic job of documenting the particular URLs that can be accessed as well as the responses that they return.

While the documentation is thorough, it does not help you figure out how to leverage any one particular language to interact with your semantic layer. This document will show you how to do that using the Python programming language, while providing a theoretical web service implementation that will not only help you understand how to make requests, but should also give you a rough idea of how this could be implemented server side.

With that said, the web service implementation I’ve provided is for demonstration purposes only and will (almost assuredly) vary in some aspects from what you will encounter on the BOE platform. If there are any major differences please leave a comment or open an issue on the GitHub repository and I will do whatever I can to accommodate.

Requirements and Installations

You will need to have the latest version of Python 3.X installed on your system. You’ll also need to install the Bottle, lxml and request packages, which can be done using pip:

pip install bottle lxml requests

Pip comes pre-installed on Python versions 3.4 and up. If you are running on an older version you will need to install pip separately.

You’ll also want to download the code samples from my BOESDKParser GitHub repository. If you have git installed, you can clone the repository:

git clone git@github.com:WillAyd/BOESDKParser.git

Otherwise simply go to the link and download as HTML to your local system.

Optionally if you want easy export to commonly reportable file formats like CSV and Excel you should install the pandas library. Be sure to reference their installation instructions for details on how to do that.

Understanding the Application

Using the Sample Web Framework

The BOESDKParser provides two modules:

• sample_framework.py
• sdk_parser.py

The sample_framework module is a very lightweight web framework implemented using the Bottle package. Here’s some abbreviated code from that module:

from bottle import route, run, request, response

@route('/biprws/logon/long')
def login():
    return ('<attrs xmlns="http://www.sap.com/rws/bip">'
            '<attr name="userName" type="string"></attr>'
            '<attr name="password" type="string"></attr>'
            '<attr name="auth" type="string" possibilities="secEnterprise,secLDAP,secWinAD,secSAPR3">secEnterprise</attr>'
            '</attrs>')

run(host='localhost', port=8080)

So what exactly is this doing? When the module is loaded, it runs the “run” method from the bottle package, which is responsible for starting a web server with the provided host and port. In this case, once this module is loaded it will fire up a web server at http://localhost:8080/.

We’ve also defined a login function with the “@route(‘/biprws/logon/long’)” decorator. If you visit http://localhost:8080/biprws/logon/long while the server is running, Bottle will execute the login method, returning the XML in the function body. For more information on how that framework operates, be sure to check out the Bottle documentation!

You’ll notice there are two methods called login, with the other one looking as follows:

@route('/biprws/logon/long', method='POST')
def login():
    ...

There difference is the “POST” parameter being specified in the decorator. By default the decorator maps the “GET” method to the function underneath it. By specifying “POST” above, we can have two requests going to http://localhost:8080/biprws/logon/long mapped to two different functions, based purely on the type of HTTP method being used. You’ll see that this is exactly what the “To Log on to the BI platform” section in the SAP BusinessObjects RESTful Web Service SDK User Guide for Web Intelligence and the BI Semantic Layer asks you to do to get a logon token.

We’ll dive further into that in the next section as we review the sdk_parser module. For now, after you’ve looked at the sample_framework module go ahead and fire up your server from the command line by navigating to the module and running:

python sample_framework.py

If successful, you should see the following:

Bottle v0.12.12 server starting up (using WSGIRefServer())...
Listening on http://localhost:8080/
Hit Ctrl-C to quit.

Interacting with the Sample Web Framework

Importing and Initializing a BOESDKParser Instance

With a web server running, we’ll shift our focus to the sdk_parser module. Within that you’ll see a class definition for an object named BOESDKParser.

class BOESDKParser():
    def __init__(self, protocol='http', host='localhost', port='8080',
                 content_type='application/xml'):
        base_url = protocol + '://' + host + ':' + port
        self.bip_url = base_url + '/biprws'
        self.webi_url = self.bip_url + '/raylight/v1'
        self.sl_url = self.bip_url + '/sl/v1'
        self.headers = {
            'accept' : content_type
            }

The default arguments for the parser initialization match the defaults specified in the sample_framework. If you modified anything in that module before starting, be sure to account for that when initializing a BOESDKParser instance. You’ll also notice that the class builds the URLs described in the Default Base URLs section of the BOE REST API, with one for the BI Platform, one for the Web Intelligence Layer and one for the BI Semantic Layer.

Assuming you can stick with the defaults, let’s go ahead and import this module and create an instance of the BOESDKParser. Launch a Python shell in the same directory as this module and run:

from sdk_parser import BOESDKParser

parser = BOESDKParser()

Understanding the Logon Methods

If you’ve read through the SAP RESTful API documentation, you’ll notice that the first step you need to do to use the services is get a logon token from the server. The BOESDKParser implements this as follows:

    def _get_auth_info(self):
        return requests.get(self.bip_url + '/logon/long',
                            headers=self.headers)

    def _send_auth_info(self, username, password):
        '''Helper function to retrieve a log in token'''
        root = etree.fromstring(self._get_auth_info().text)
        root[0].text = username
        root[1].text = password

        return requests.post(self.bip_url + '/logon/long',
                             headers=self.headers,
                             data=etree.tostring(root))

    def set_logon_token(self, username, password):
        resp = self._send_auth_info(username, password)
        if resp.status_code == 200:
            root = etree.fromstring(resp.text)
            # Set logon token in headers
            self.headers['X-SAP-LogonToken'] = root[0].text
        else:
            # Crude exception handling
            raise Exception("Could not log on and set the logon token!")

The underscore preceding the first two methods is an indication that these methods are “private” and should not be called directly (although nothing in Python prevents you from doing so). The main entry point here is the set_logon_token method. When called with a username and password (we’ll be using “myUserName” and “myPassword”), it passes that information to _send_auth_info which itself calls _get_auth_info.

That method makes the first request going to http://localhost:8080/biprws/logon/long (again, assuming you’ve stuck with the default host / port). If you check back to the sample_framework module, you’ll notice that when called with GET that method returns XML, which looks as follows:

<attrs xmlns="http://www.sap.com/rws/bip">
    <attr name="userName" type="string"/></attr>
    <attr name="password" type="string"></attr>
    <attr name="auth" type="string" possibilities="secEnterprise,secLDAP,secWinAD,secSAPR3">secEnterprise</attr>
</attrs>

The result of _get_auth_info is sent back to the first line in _send_auth_info as a response object:

root = etree.fromstring(self._get_auth_info().text)

Here we access the “.text” attribute of the response object to focus on the XML contents (the request object comes with a lot of other information that we don’t need for this demo). We then use the etree class from the lxml package, calling it’s “fromstring” method to take that XML and convert into an object that we can more easily interact with.

        root[0].text = username
        root[1].text = password

The above two lines of code access the first and second child nodes of that XML, respectively, and sets their text to the username and password that are provided, which would yield something as follows:

<attrs xmlns="http://www.sap.com/rws/bip">
    <attr name="userName" type="string"/>myUserName</attr>
    <attr name="password" type="string">myPassword</attr>
    <attr name="auth" type="string" possibilities="secEnterprise,secLDAP,secWinAD,secSAPR3">secEnterprise</attr>
</attrs>

We need to send that information to get a logon token back. By POSTing to ‘/biprws/logon/long’ instead of using the GET method, we can access the second login function defined in sample_framework.py. That reads the request body and checks that the provided username is “myUserName” while the provided password is “myPassword.” If that is the case, it will return a logon token. Here’s how that all works (from sample_framework.py):

@route('/biprws/logon/long', method='POST')
def login():
    body = request.body.read()
    root = etree.fromstring(body)
    user, pw = root[0].text, root[1].text
    ...
    if user == 'myUserName' and pw == 'myPassword':
        return ('<attrs xmlns="http://www.sap.com/rws/bip">'
                '<attr name="LogonToken" type="string">COMMANDCOM-LCM:'
                '6400@{3&amp;2=5595,U3&amp;p=40674.9596541551,Y7&amp;4F=12,U3&amp;63=secEnterprise,0P&amp;66=60,03&amp;68='
                'secEnterprise:Administrator,'
                '0P&amp;qe=100,U3&amp;vz=SFY6agrLPxpfQBK1ZKYCwoBZKCbfsQm7VgWZFiH.RhM,UP</attr>'
                '</attrs>')

    # Fall back to a 401
    response.status = 401
    return 'Could not authenticate user with provided credentials'

This response makes its way back to the set_logon_token method, where it checks that the response returned successfully (i.e., with a 200 status code). Assuming so, our BOESDParser instance parsers out the logon token and stores it in its headers dict.

If the above seemed very complicated, don’t worry! The parser abstracts all of that for you, so after instantiating your object all you really need to do is call

parser.set_logon_token("myUserName", "myPassword")

and go on your merry way to the next section.

Retrieving a List of Universes

After logging on, you can use the get_universes method to pull a listing of sample universes from the sample web server. Here’s how that method is implemented:

    def get_universes(self):
        resp = requests.get(self.webi_url + '/universes', headers=self.headers)
        if resp.status_code == 200:
            root = etree.fromstring(resp.text)
            # Iterate over the children elements and convert them into a dict
            # of dicts
            univs = dict()
            for index, univ in enumerate(root):
                univs[index] = dict()
                for child in univ:
                    univs[index][child.tag] = child.text

            return univs
        else:
            # Crude Exception handling
            raise Exception(('Could not retrieve universes - have you set a '
                             'valid logon token?'))

What’s going on here? Well, first we make a call to self.webi_url + ‘/universes’, which expands out to http://localhost:8080/biprws/raylight/v1/universes. We also need to send our headers, which after having logged on should now look something as follows:

{
    'accept' : content_type
    'X-SAP-LogonToken' : 'COMMANDCOMLCM:6400@{3&2=5595,U3&p=40674.9596541551,Y7&4F=12,U3&63=secEnterprise,0P&66=60,03&68=secEnterprise:Administrator,0P&qe=100,U3&vz=SFY6agrLPxpfQBK1ZKYCwoBZKCbfsQm7VgWZFiH.RhM,UP'
}

The X-SAP-LogonToken is what the server will use to validate our session, so without it you will not be allowed to retrieve the universes that you want. With that however, we should get a response object that looks as follows:

<universes>
    <universe>
        <id>6773</id>
        <cuid>AXyRzvmRrJxLqUm6_Jbf7lE</cuid>
        <name>efashion.unv</name>
        <type>unv</type>
        <folderId>6771</folderId>
    </universe>
    <universe>
        <id>5808</id>
        <cuid>AUW2qRdU0IdPkyhlpZWrxvo</cuid>
        <name>Warehouse.unx</name>
        <type>unx</type>
        <folderId>5807</folderId>
    </universe>
</universes>

Just like before, we convert that XML response into an etree and iterate over the nodes. We are going to build out a dictionary to store that XML information, and to do such we enumerate the children of the root note (i.e. the two universe tags). We set our initial key in the dict to that enumerated value, and set its value to another dict mapping all the id, cuid, name, type and folderId information for that universe. Ultimately, that will yield something as follows:

{0: {'cuid': 'AXyRzvmRrJxLqUm6_Jbf7lE',
     'folderId': '6771',
     'id': '6773',
     'name': 'efashion.unv',
     'type': 'unv'},
 1: {'cuid': 'AUW2qRdU0IdPkyhlpZWrxvo',
     'folderId': '5807',
     'id': '5808',
     'name': 'Warehouse.unx',
     'type': 'unx'}}

With the explanation aside, let’s go ahead and just call the get_universes method and store its output in a local variable called univs. That alone will be a dict that you can use to access information, but if you’ve installed the optional pandas dependency you can easily convert that dict into a DataFrame.

univs = parser.get_universes()

import pandas as pd
df = pd.DataFrame.from_dict(univs, orient='index')

The DataFrame looks something as follows:

     id                     cuid           name type folderId
0  6773  AXyRzvmRrJxLqUm6_Jbf7lE   efashion.unv  unv     6771
1  5808  AUW2qRdU0IdPkyhlpZWrxvo  Warehouse.unx  unx     5807

Which we can easily export to CSV as such:

df.to_csv('some_file.csv')

Bringing it All Back Home

The above section has gone over in some detail how all of the methods here have been implemented. To sum it all up however, with the web server running we can “log on,” get a list of universes and optionally drop that list of universes to CSV all with the following lines of code.

from sdk_parser import BOESDKParser
parser = BOESDKParser()

parser.set_logon_token("myUserName", "myPassword")
univs = parser.get_universes()

# Optionally write out to a pandas DataFrame
import pandas as pd
df = pd.DataFrame.from_dict(univs, orient='index')
df.to_csv('some_file.csv')

Conclusion

Thanks for taking the time to read through this document. As mentioned, the items provided here may not perfectly replicate what you will see when communicating with a real BOE environment, but I hope that you find value in understanding at a high level how to make requests to a server via Python and also how the framework roughly interprets and responds to the requests that you make.

Should you have any ideas or thoughts on how to improve this document, or if there are any particular code samples you’d like to see spelled out (knowing this document only focused on pulling Universe information) please either leave a comment below or open an issue on the BOESDKParser GitHub repository page.