How Real-Time is SAP’s Real-Time Data Platform ?

This blog post is 3 months in the making. I started writing it on the way back from Sapphire in May, and due to the manically hectic schedule that is called “my life”, never got to finish it and am now starting fresh.

In hindsight, I’m glad this happened, as between now and then the “To BW or not to BW” debate has been a hot topic. This blog post is not going to go down that road as I feel my peers have covered those points very well, and as Ethan Jewett pointed out so eloquently in a tweet two days ago “People (especially experts) tend to prefer tools that they understand.”

As I sit here on flight QF63 from Sydney to Johannesburg, I am pondering the sentiments and comments from all the attendees (customers, partners and SAP employees) of the Mastering BI with SAP Conference in Melbourne the past few days. The general message, from a Database & Technology (D&T) point of view, is still very much HANA for everything which is slightly disappointing. I was hoping that SAP would have started to embrace all the products in the Real-Time data platform with one integrated view for the customer instead of pushing the HANA message which I feel customers are getting a bit tired of.

Those that know me well often ask me if I go to all these conferences to surf or to attend the event. A surfboard is always part of my luggage, and the odd detour from the convention center to the ocean is always on the cards for me. The picture above represents my Nirvana. Warm tropical water and beautiful uncrowded surf. Throw in the wife and kids and this nerd is one happy man.

For my customers, I want them to have this Nirvana when it comes to accessing their data – one entry point, and not by using a BI4 universe, which kills your performance with large data sets. To put things bluntly, I feel that my customers should not really care where the data comes from. The words “hot and cold” data get thrown around loosely but this always leads to a large amount of ETL (Extract-Transform-Load) and users needing to be educated on which “entry point” they need to access their data based on the date range they are trying to access. One of my customers referred to this solution as “clunky” and I could not agree more!

I guess by now you are wondering what I am after… what is the “silver bullet”? I want my customers to have one entry point (the real time data platform) and by using slick and intelligent archiving mechanisms “under the hood”, they must be oblivious to where the data is stored without moving and duplicating data backwards and forwards between database technologies. In the “old days” we were lucky enough to work in batch driven systems, so reconciliation was pretty simple as there was always an end of day point. Those days are now long gone as many customers load data into their BI solution 24/7, with no end of day, which always makes reconciliation a challenge (and that’s putting it nicely). 

Fun Fact: We were fortunate enough to have 90 minutes with Hasso Platner and Vishal Sikka in May in Orlando and it was fascinating for me to hear (from Hasso Plattner) that the hot/cold concept of storing data was in the original design of R1, but due to the low volumes of data back then, everything was put in the “hot storage”.

So where are we today and how close are we to that Nirvana? As is the case in the current SAP eco-system, there are two distinct scenarios: BW or non-BW customers, so I am going to break down my opinions that way.

Many thanks to fellow SAP Mentor Ethan Jewett for checking my BW facts, as my hands-on experience is limited.

BW World

Things are actually looking very good for BW customers. Right now, there are two third party companies ,that I know of, that offer the Near Line Storage (NLS), namely:

1. PBS NLS: I have been exposed to this solution through my long-term love affair with Sybase IQ. The concept is excellent, where your read only data gets archived into a Sybase IQ data store. The benefits to the company are immediate and are as follows:
   1. Shrink the size of your native BW database, which ultimately leads to query performance enhancements
   2. Get the benefit of compression in Sybase IQ, dropping your data foot print
   3. The benefit of column based storage in Sybase IQ for fast query response times

          It almost sounds too good to be true typing this,  and by no means is the PBS solution cheap.

2.     SAND – I am less familiar with this solution, but thanks to fellow SAP Mentor, Sascha Wenninger, for his input, as they implemented this strategy at one of          his customers.

Ethan also let me know about this solution by IBM using DB2

The Future

By all accounts, native NLS is meant to be coming in one of the service packs of BW 7.3. This will allow BW customers, without paying the additional licenses to third party vendors, to have their truly hot data in HANA with the cold archived data in Sybase IQ. For me, this is an awesome solution, and once the price of HANA slowly drops over time, customers can increase the amount of the hot data in HANA.

One thing that does concern me, from a purely theoretical point of view, is that the BW application layer does most of the processing.  I understand why SAP did this, to ensure that they were database agnostic, but now that they are in the database game, there is a huge need to push the processing down to the database engines to really reap the benefits of the technology of HANA and SAP Sybase IQ. 

Enterprise Data Warehouse (EDW) / Non BW World

Many thanks to fellow SAP Mentor Josh Fletcher for checking my sanity on this section.

This is the area that is close to my heart and the one that I mainly focus on.  In this space, there are currently no options whatsoever (yet again I am not looking at a BI4 multi source universe). The EDW world is filled with ETL jobs and moving data backwards and forwards. The challenges brought in by the constant movement of data are vast, with reconciliation efforts often being quite tedious and time consuming and the “single version of the truth” is seldom met.

There is SAP Sybase Replication Server for replicating data from SAP Sybase ASE to SAP Sybase IQ. Replication Server has got a new “Real time data load” function that polls data up in replication server, and then bulk loads this into Sybase IQ. It is solid technology and works well, but all you are doing here is creating an operational data store (ODS) in your EDW by replicating the OLTP system sitting in SAP Sybase ASE.

I have such a clear vision for this space, and to me, it is clean, simple and has the potential for SAP to entrench themselves in many customers, and more importantly set themselves up for future opportunities.

During my conversations with the APJ SAP team over these last few days, the number of SAP BusinessObjects customers (2500) in the region got mentioned. Now, let’s assume that the majority of those are classic BOBJ customers and not BW ones.

If I was running D&T in APJ (doubt they would have me 😉 ) I would do all I could to make contact with those +- 2000 existing BOBJ customers and talk about Sybase IQ. Now I know a few of you might be sighing and say: Clint’s on about Sybase IQ again! But, wait, there’s more…………….

Similar to the BW space, I feel SAP need to offer the same NLS type solution for EDW customers. This way SAP can go out with confidence and start selling Sybase IQ into the EDW/classic BOBJ space immediately, and know that the HANA sales will come. As soon as SAP get this integration/archiving sorted out then, in my opinion,  SAP will be hard to beat. 

In my experiences, HANA is suffering in the EDW space (note I am based in EMEA) and when it comes down to the TCO argument with no special content all falls completely flat.

But, here is the Nirvana for both the customer and SAP – let’s also remember that SAP sell software! SAP need to approach all their EDW customers and get them to look at Sybase IQ. MANY customers out there are dying with performance of the traditional row based databases that are often seen under those classic BOBJ systems.

Customer Win

• Smaller data footprint due to compression
• Drastic query performance improvements
• Faster load times of data with lower latency 
• Not having to archive old data off due to performance constraints
• Lower TCO

I could go on and on but that will be another blog.

SAP Win

• They sell software and increase the D&T footprint. More importantly they sell their customers a tried and tested technology that will add huge and instant value to their customers. 

Let’s look 12-18 months down the line, and the HANA/Sybase IQ integration in the EDW space is out of beta and is now a slick solution. SAP can now approach the same customers with large data volumes, who I can pretty much assure you, are going to be happy with the SAP database technology (Sybase IQ), and look to start loading the hot data into HANA with seamless archiving into the already existing Sybase IQ instance

Customer Win

• Benefit of speed of in memory technology
• The ability to buy a small instance of HANA as an entry point and not worry about their users having multiple entry points into the data
• Continue to leverage their previous investment in Sybase IQ

SAP Win

• Sell more software
• A non disruptive way to offer their customers a cheap entry point to in memory technology
• A very scalable solution for customers being able to add on more and more HANA where needs be

I know the HANA message is strong right now at SAP; However, I honestly feel that the above message is one that will be more palatable to customers, and seeing that it is based on solid, scalable technology, will make SAP look good.

SAP is working hard at trying to change their image of long, expensive projects that often break the bank at customers. In my experience, within a few short weeks, you could replace a classic DBMS with SAP Sybase IQ, and amaze your customers with the benefits.

The key word for me in both my personal and business world is trust.  Customers are tired of the “HANA message” and in most cases it falls out of budget. If SAP look into their tool kit and see that SAP Sybase IQ is a great 2-4 year plan that will give customers huge value at a fraction of the price, it will buy them credibility and most importantly trust from their customers.

The investment won’t be a throw away, as their customers can use Sybase IQ as their archiving strategy to assist with long term (10+ years) trend analysis.

Don’t get me wrong; I am not a HANA hater. I think the technology is amazing and truly do feel that in memory is where everything is going to be in the next 5 years. The key is in how we get there!

I have just noticed that this blog post is over 5 pages – the joys of a  15-hour flight and post conference thoughts…

So to summarize:

BW folks: This is looking OK with a roadmap in place and third party tools for those less patient.

EDW folks: There is a real-time-data platform filled with amazing technology but right now the integration is either replication or ETL

SAP have been quite bullish in their statement of “being the 2^nd largest database vendor in the world by 2015”, and, perhaps walking the road with their customers, from disk based to in memory technology over the next few years, will work better than going straight to the HANA message upfront.