Tutorial for creating a simple Custom Data Source.
I recently developed a custom data source extension to access indicator data from the World Bank API. It was my first crack at the Design Studio SDK, and I thought I’d upload a tutorial blog to help those who are looking to create their own data extensions for the first time. I know there are already a few similar blogs out on the community already, so I am assuming no prior experience on the part of my readers. I am going to try and make it as detailed and simple as possible (while simultaneously keeping this post as short as possible).
The extension I have developed for this blog was the stepping stone to more versatile extensions which were used in an infographic which was presented at the SAP Analytics Infoday 2015 event on the 7th of July. You can find a video of the final infographic here.
Before we start, you should have the SDK Guide, and should already have added the SAP provided SDK extensions to your Eclipse workspace.
Before I get into the extension, let me tell you a little about the World Bank data, and how data can be accessed from their API.
World Bank APIs
The World Bank Indicators API lets you programmatically access more than 8,000 indicators and query the data in several ways, using parameters to specify your request. Many data series date back 50 years, and can be used to create interesting applications. The Indicators API has three types of response formats: XML, JSON, and JSONP. I have used JSONP format to circumvent the Same Origin Policy issue.
There are four basic arguments the developer needs to form their data request:
- Country/List of countries
- Year/Range of years
- Output format
Note: Find a more detailed description of the call structure here.
The figures below give an example of a sample request and its corresponding response.
Data Source SDK
Diving into the development of the extension, from a technical point of view, an SDK extension is an Eclipse plugin, which contains the following files.
- Contribution XML file
- Component CSS file
- Icon file
- Script Contribution file
- Additional Properties Sheet HTML file
Contribution XML File
The Contribution XML file specifies the SDK extension and all its extension components. SAP provides a documented XML schema definition file (sdk.xsd) that defines the format of the Contribution XML file. It is here where we provide Design Studio with information about our extension; its name, version, vendor. We also specify which files are included in the extension and where they’re stored.
Here is the contribution.xml for the extension described in this post.
<?xml version=“1.0” encoding=“UTF-8”?>
title=“Design Studio SDK Extension for World Bank REST API”
title=“World Bank Data Source”
tooltip=“Allows access to data from the World Bank API”
<property id=“countries” title=“Countries” type=“String”></property>
<property type=“String” title=“Range” id=“range”></property>
<property type=“String” title=“Indicators” id=“indicators”></property>
The extension is defined by the sdkExtension element. Its important attributes are:
- title: Title of the SDK extension.
- id: Specifies the ID of the SDK extension to avoid conflict with its own components and with other extensions. It is recommended that you give it the same, fully qualified names as the folder name of the corresponding Eclipse project.
- version: Provides the version of your extension. Now, if you’ve read the manual you’ll see that the method to create a new extension in Eclipse is to copy an existing one within the workspace and then modifying the newly-created extension’s identifiers. When you do this, pay attention to the version. If you create a new extension from an old one, do not adopt the latter’s version. Start from “1.0”.
- vendor: Provides the vendor name of your extension. You can mention your name here, or your employer’s name, or you can simply leave the value as “SAP”. Please note, that the values of the vendor attribute in the contribution.xml file and the Bundle-Vendor attribute in the MANIFEST.MF file should match. Should there be a discrepancy, Design Studio will not load the extension.
The component element is a child element of the sdkExtension element. The extension component is specified by this element. Its important attributes are:
- title: Title of the extension component.
- handlerType: Specifies the technology that implements this extension component. For custom data sources, this attribute must be specified as “datasource”.
This element is used to specify properties of the extension element. Properties can be thought of as variables that can be set at design time. As you can see from the contribution.xml file, the properties I’ve defined are the same as the arguments that are used to make up the data request. These properties allow me to modify the data request at design time.
The important attributes of the property element are:
- title: Title of the property. This is the textual representation of the property which can be viewed at design time.
- type: Type of the property. Specify one of the followingint, float, boolean, String, Text, ScriptText, Color, Url.
We are going to be exploring the second method, extending the DataBuffer class.
Extension Component Lifecycle
- Update all extension component properties using their setter/getter functions
The order in which these functions are performed depends on whether or not the component has already been rendered. The distinction is explained in the image below.
Note: It is not imperative to implement all of these functions. For instance if the developer leaves out the beforeUpdate() function, the SDK framework will move forward and start updating the property values.
Extending the DataBuffer class
This function is used to set the dimensions of a custom data source. The result of this function can be thought of as nothing more than the structure of a crosstab. We specify for each dimension whether its members will be in the rows or columns of the crosstab.
The syntax of the defineDimensions function is:
It has two arguments. The argument aoDimensions contains an array of JSON objects, where each object is a dimension. The argument oExternalMeasuresDimension is optional. We may either define the measure dimension externally using this argument, or we can configure one of the dimensions in the argument aoDimensions to contain measures. The attribute which is used to specify whether a dimension contains measures is containsMeasures, and it must be set to true.
Given below is an example of the defineDimensions function.
As you can see it does not contain the oExternalMeasuresDimension argument, and that the Indicators dimension is configured to contain measures.
This function is called to set the value of a single data cell of the custom data source. The syntax of the function is:
The argument aCoordinates is an array which is used to specify the coordinates of the next argument value. The array may contain either dimension member names or dimension member indices. Using the dimensions we defined in the example above, let’s assume that GDP of India for the year 2013 was $1861801615477.9, then the corresponding setDataCell call will be:
There are a few things to remember while using the setDataCell function
- The order in which we mention the members for each dimension must the same order in which the dimensions were defined.
When populating your data source with data cells, you must strictly follow this sequence: Set the data cells left-to-right first, then top-to-bottom. Adding data cells randomly (with respect to their coordinates) may lead to an unusual arrangement of data cells. Refer the images below to better understand of the order in which the data cells must be set.
This function is called to notify the SDK framework that the custom data source has been updated. Its syntax is:
If the optional argument bWillUpdateServer is true, then the SDK framework also notifies the server on the backend of the change. This function is called after all the data cells have been set.
Using the Extension in Design Studio
Step 1: Add a Chart component and a Simple Crosstab component into the Analysis Application, and then add a Custom Data Source.
Step 2: Configure the property values for the Custom Data Source
Step 3: Assign the Custom Data Source to both the Chart and the Simple Crosstab components.
Step 4: Run the Analysis Application.
Conclusion and Future Development
I have uploaded the Eclipse project for this extension to GitHub here which you can add to your own workspace and test on Design Studio.
My aim is to build up to a single, multipurpose data extension which can be used for the various types of charts used in this infographic
I hope this post was enough provide enough insight into the Data Source SDK to help you get started in creating your own custom data sources. Please feel free to ask any questions or leave any feedback in the comment section.
Update: I’ve uploaded Part two for this blog. It can be found here. I will be uploading Part three soon.