Skip to Content

Website to Visuals – Scraping Website into Data within SAP Lumira

Digital business demands dictate an ever increasing need for the extraction of information and useful deductions from live/ real time data on the web. Be it manufacturing or sales or any other department, they all require to base their decisions on real time data. Building visualizations from static data however, is very simple. But this does not provide a solution, for example, to a sales management executive who wants to visualize real time sales by product category. This is where there arises a need for a method for extraction and visualization of live data. While SAP Lumira helps best with the visualization, we need a method for extracting this live data into SAP Lumira. seems to be the best option to fit the requirement. It is free, easy to use and powerful with lot of features that we can use for extracting the data into SAP Lumira.  Read on to find out how to scrape a website and bring in live data for creating visualizations.


I had come across this great blog post: by Ronald Konijnenburg about using in Lumira to extract data from the web where you can follow to setup and expose Data as an API inside

We would need to have live data. Every time we need Lumira to fetch the updated data from the website and so the CSV method would not work. The DA extensions of SAP Lumira would help us connect to the Data source. So let’s build a DA extension for SAP Lumira. For our simplicity of understanding, we are taking the same WSOP data from


The process of DA extensions is simple enough to fetch the API and bring the data into Lumira. The data is converted into CSV and is then consumed by Lumira for Visualization. The extension is then installed into Lumira.

Figure 3: Print data

You can then build the extension into binary file and use it in Lumira.


You could check out the following Lumira – Open Source Data Access Extensions to learn about DA extensions and developing DA extensions for SAP Lumira.

Installing and Executing the Extension

1. Installation of the Extension is simple. Place the binary file in the Desktop\daextension folders located inside the SAP Lumira install folder.

Figure 4: installing extension

2. Open SAPLumira.ini and add the following two lines at the end

-Dhilo.externalds.folder=C:\Program Files\SAP Lumira\Desktop\daextensions


Figure 5: Editing SAPLumira.ini

3. Open a new Document inside SAP Lumira. Select External Datasource and click on Next

Figure 6: Accessing the External Datasource

4. Select the importio-da-extensions and click on Next

Figure 7: extension

5. You will be prompted for the URL+API key. This URL is obtained from when you save the Data source and extract the API.

Figure 8: URL+API key

6. You will be prompted for the number of pages to be fetched. Often websites have pagination that contains data across multiple pages. So you can set how many pages you want to fetch.

Figure 9: Page Number

7. You will be fetched with all the Data. This data can then be created as a Data source in Lumira.

Figure 8: creating dataset

Creating the Visualization

Here, we create couple of visualizations that give us an insight into the World Series of Poker rankings.

This tree chart shows the comparison of the number of bracelets and rings won by players in WSOP and the tag cloud displays player name by their WSOP ITM. These two charts give the viewer an idea of the players with maximum victories in Poker circuits and the World Series itself.

Figure 9: Tree chart- Bracelets vs Rings


The DA extension for is completed. You can feel free to download the code and the executable from my github Now Website to Visuals is just a button away with SAP Lumira and DA extensions.

You must be Logged on to comment or reply to a post.
  • can we do this on SAP Lumira desktop free version? or need SAP Lumira Team server on SAP Lumira on HANA

    would scraping slow the transactional system (the customers web commerce site)

    by the way its very good effort from your side

    • Hi Saee,

      It will work on Lumira Desktop. But since its DA extension it does not have support to Lumira Team server or Lumira Server for now.

      Scraping is done via normal HTTP request and that would not slow down the site since we are only scraping on demand when needed.