Skip to Content
Author's profile photo Shankar Narayanan SGS

Website to Visuals – Scraping Website into Data within SAP Lumira

Digital business demands dictate an ever increasing need for the extraction of information and useful deductions from live/ real time data on the web. Be it manufacturing or sales or any other department, they all require to base their decisions on real time data. Building visualizations from static data however, is very simple. But this does not provide a solution, for example, to a sales management executive who wants to visualize real time sales by product category. This is where there arises a need for a method for extraction and visualization of live data. While SAP Lumira helps best with the visualization, we need a method for extracting this live data into SAP Lumira. seems to be the best option to fit the requirement. It is free, easy to use and powerful with lot of features that we can use for extracting the data into SAP Lumira.  Read on to find out how to scrape a website and bring in live data for creating visualizations.


I had come across this great blog post: by Ronald Konijnenburg about using in Lumira to extract data from the web where you can follow to setup and expose Data as an API inside

We would need to have live data. Every time we need Lumira to fetch the updated data from the website and so the CSV method would not work. The DA extensions of SAP Lumira would help us connect to the Data source. So let’s build a DA extension for SAP Lumira. For our simplicity of understanding, we are taking the same WSOP data from


The process of DA extensions is simple enough to fetch the API and bring the data into Lumira. The data is converted into CSV and is then consumed by Lumira for Visualization. The extension is then installed into Lumira.

Figure 3: Print data

You can then build the extension into binary file and use it in Lumira.


You could check out the following Lumira – Open Source Data Access Extensions to learn about DA extensions and developing DA extensions for SAP Lumira.

Installing and Executing the Extension

1. Installation of the Extension is simple. Place the binary file in the Desktop\daextension folders located inside the SAP Lumira install folder.

Figure 4: installing extension

2. Open SAPLumira.ini and add the following two lines at the end

-Dhilo.externalds.folder=C:\Program Files\SAP Lumira\Desktop\daextensions


Figure 5: Editing SAPLumira.ini

3. Open a new Document inside SAP Lumira. Select External Datasource and click on Next

Figure 6: Accessing the External Datasource

4. Select the importio-da-extensions and click on Next

Figure 7: extension

5. You will be prompted for the URL+API key. This URL is obtained from when you save the Data source and extract the API.

Figure 8: URL+API key

6. You will be prompted for the number of pages to be fetched. Often websites have pagination that contains data across multiple pages. So you can set how many pages you want to fetch.

Figure 9: Page Number

7. You will be fetched with all the Data. This data can then be created as a Data source in Lumira.

Figure 8: creating dataset

Creating the Visualization

Here, we create couple of visualizations that give us an insight into the World Series of Poker rankings.

This tree chart shows the comparison of the number of bracelets and rings won by players in WSOP and the tag cloud displays player name by their WSOP ITM. These two charts give the viewer an idea of the players with maximum victories in Poker circuits and the World Series itself.

Figure 9: Tree chart- Bracelets vs Rings


The DA extension for is completed. You can feel free to download the code and the executable from my github Now Website to Visuals is just a button away with SAP Lumira and DA extensions.

Assigned Tags

      You must be Logged on to comment or reply to a post.
      Author's profile photo Former Member
      Former Member

      can we do this on SAP Lumira desktop free version? or need SAP Lumira Team server on SAP Lumira on HANA

      would scraping slow the transactional system (the customers web commerce site)

      by the way its very good effort from your side

      Author's profile photo Shankar Narayanan SGS
      Shankar Narayanan SGS
      Blog Post Author

      Hi Saee,

      It will work on Lumira Desktop. But since its DA extension it does not have support to Lumira Team server or Lumira Server for now.

      Scraping is done via normal HTTP request and that would not slow down the site since we are only scraping on demand when needed.


      Author's profile photo Former Member
      Former Member

      Dear Shankar,

      thanks for the reply, its nice extension to try out

      tryed to reach you from the mail id

      how could i reach you to discuss on this DA extension

      Author's profile photo Shankar Narayanan SGS
      Shankar Narayanan SGS
      Blog Post Author

      Hi Saee.

      Thanks for the feedback. you can comment below if you have any queries or connect with me via SCN messages. You could also check out my other blogs on SAP Lumira if you are interested.



      Author's profile photo Former Member
      Former Member

      how to send SCN pm messages to u ?