Earlier this year, we released SAP Lumira Server which provides the server-based complement to SAP Lumira running on the desktop. This product not only enables users to perform analysis on datasets of any size and share interactive storyboards based on them with others, but it now also allows users to create and edit stories online with just a browser. Two years in the making, SAP Lumira Server is a native SAP HANA application that is deployed directly on the appliance and therefore has a natural ability to work on complex or large data and leverage the full power of SAP HANA.
As the Product Manager for SAP Lumira Server, one of the most frequent questions I get is “Why does Lumira Server require SAP HANA?” Is this an artificial dependency created to sell more SAP HANA licenses and force customers to buy specific hardware configurations from SAP’s hardware partners or does SAP HANA truly give Lumira superpowers not possible any other way?
How Traditional BI Systems Work
To answer this question, we need to take a short diversion into BI architecture. Almost every BI solution in the last 20 years from every vendor works in roughly the same way: a BI engine issues queries to a reporting database and then pushes the result sets through a report definition to render a result. This creates a BI artifact such as a Crystal Report or Web Intelligence document that a user can interact with. When the report is refreshed, the same process is repeated all over again. BusinessObjects Explorer works slightly differently in that the current view is dynamically generated based on the user’s navigation context and it uses indexed data instead of the source database, but otherwise follows the same principles.
Now think of what’s happening in the background – the index and database I/O required to ensure access to the database is fast, the network and disk I/O required to get the results from the database to the BI Platform, and the CPU power and I/O required for the BI engines to churn through all that data, perform calculations, and actually render the report to the user are all factors as to why BI systems seem to always get bigger and slower as the data volume increases.
Figure 1: How Traditional BI Systems Work
Many modern BI solutions implement additional technologies like Web Intelligence’s “micro-cube” in memory to provide faster response times or Explorer’s Information Space indexes to allow guided navigation without the engine needing to re-query the database whenever the user drills. Effectively, the BI platform caches content outside the source database so that it can have faster, more direct access to the data. If this sounds similar to database technologies that use aggregates and pre-computed indexes to speed query times, you are on the right track.
Enter SAP HANA
You’ve likely heard that “SAP HANA is not a traditional database” at least once before. It isn’t just “in-memory” because it stores all data in RAM - it also performs operations on that data in-memory and in-place using highly tuned calculation engines. This is the not-so-secret sauce to SAP HANA and why indexes, aggregates, and caches are not typically required.
Now what if we could implement the BI system directly on top of such a set of engines and natively use their functions for business intelligence? This is exactly what the SAP HANA “eXtended application Services Engine” or “XS Engine” does for SAP Lumira Server. With native access to HANA’s computation engines, Lumira Server does much less work since intermediate data no longer needs to be extracted, interpreted, or calculated. Let’s take a simple example: a simple bar chart based on 100,000 rows of data in a traditional BI system may require a large data transfer from the database if the computations cannot be pushed to the SQL query. Lumira Server on the other hand has native access to HANA and can push most of its processing to the calculation engines which need to only return the metadata required to actually render the graph.
Figure 2: How Native HANA XS Applications like SAP Lumira Server Work
(Note this is very different than simply putting a traditional BI engine on the same machine as a traditional database – in this case the I/O between BI engine and database is still incurred, the row set still needs to be transmitted, and the BI engines must still process all rows to get the information required to render the graph. There is a small amount of efficiency in doing all network traffic on the same box but remember that network I/O is still incurred as the TCP/IP stack is nevertheless invoked for the database and BI system to communicate.)
Lumira Server: A HANA-Native BI Solution
Hopefully this sheds some light on why Lumira Server is different than traditional BI tools and how it uses SAP HANA as more than a database – it uses HANA’s computation engines and ultimately uses it as an application server as well. This “flattening of the stack” and the pushing of calculations to be as close to the data as possible results in huge savings in terms of CPU cycles and I/O overhead.
Does this mean that you need to adopt HANA as your database to benefit from Lumira Server? Not at all – it is true that Lumira Server operates on data stored within HANA, but that data could be published using Lumira Desktop or any technology that creates HANA Views (such as Sybase Replication Server, Data Services, or a large array of other options). In this scenario you can think of HANA using its memory as space for BI analytical working sets – the size of your HANA system needs to only be the size of the data that needs to be uploaded for exploration and analysis. Obviously if HANA was your authoritative source or data warehouse database as well, publishing or replication would not be required and you could operate on much larger and more complex datasets with even more efficiency.
Making SAP Lumira Server More Accessible
Unfortunately the need for SAP HANA can be a significant blocker for many customers who either do not have HANA already or even those who cannot allocate an existing HANA system to pair with their BI deployment. Many BI 4.x customers have told us that Lumira Server needs to be more accessible – either through lower cost hardware or perhaps even a configuration that does not require HANA at all.
Those who know SAP Lumira well will note that the desktop does not include a HANA engine yet can still perform most of the tasks as Lumira Server such as visualization and story creation through emulation (albeit in a single user scenario). Unfortunately we cannot just bring that emulation to the server side as it would be horribly inefficient in a multi-user scenario, but we are investigating some possible options on how we can make some type of enabling in-memory technology more accessible to our customers. This would obviously not perform as well as even a small HANA system, but could still be a viable option for some customers with smaller deployments because at its core, Lumira’s requirement is not SAP HANA itself, but what HANA can do for it.
More Details To Come
Some of you may feel that this detail isn't deep enough - and you are probably right - my first goal is to start getting my information out. I'm sure you all will help guide my future topics. :wink:
In the next installment, I’ll cover some of the most frequently asked questions about SAP Lumira Server’s integration with SAP BI 4, how it works, and how it doesn’t.
(Update: The next blog entry in the series is actually on Lumira Storyboards here: How Lumira Storyboards Work on SAP Lumira Server)
- SAP Managed Tags:
8 Comments
