SAP Lumira MongoDB data access extension

The Variety Challenge of Big Data

In the Analytics space, the challenge of query speed is the one the BI tools were most frequently confronted with. The objective is always to get short enough response time to go with the analytical flow users are trying to accomplish. And sometimes running their analysis on very large tables, or on Big Data, could turn into a patience test, patience being a virtue we definitely seem to be short of in this mobile-I-need-it-now world… But focusing on this issue assumed already some work had been done to get those tables up and ready. I am not dismissing the topic of volume of data – this will be the topic of another blog post very shortly- but I would like to get back to the basic: how to access the data and nowadays we cannot assume the format of the data is always tabular.

MongoDB plugin for SAP Lumira

MongoDB is a good example of a non-traditional database: MongoDB stores data in documents, giving organizations more flexibility as they develop and iterate on their applications. Today there are already SQL connectors for MongoDB, the fastest-growing non-relational database as this db-ranking page is showing. For SAP Lumira specifically, you can download a Simba plugin from the SAP Marketplace and start using it to query the data stored in your MongoDB documents with SQL. These connectors do already enable organizations to use BI tools on MongoDB, but they require a flattening step that we are looked into to see if they could be improved and therefore enable more end-users. Indeed, the flattening of the document structure upfront forces the end-user to work with SQL and tables from the get-go while with a proper UI, this necessary step could be instead done only once the user knows what to acquire.

A dedicated data acquisition experience

By developing a process that is better adapted to the documents in MongoDB, we can improve the user experience, improve response time, and make BI more accessible to users by removing the need to know SQL entirely. Our idea was to present our users with a native interface that reflects the structure of MongoDB documents, one that highlights accessible data that they can begin analyzing. We’ve built a nodal interface that allows users to pick and choose fields they would like to bring into Lumira:

We have changed the acquisition experience so that the flattening of the source that traditionally would happen even before the user would have defined the query, is now done only once the user has decided what to acquire. This lets the analyst truly benefit from the richness of collections of nested data. This video shows a demo of how this prototype is working:

Try it out for yourself by simply downloading the prototype, and installing it in SAP Lumira through the brand new extension manager. And do let us know about your experience, we are curious to hear from you.

Best,

Paul