My Converting my lessons learned from Sapphire into compelling future architectures described a very broad architecture that emerged out of my experience at the Sapphire and included a variety of ideas – many of which were just examined in passing. There was one idea, however, that was so intriguing that I wasn’t really satisfied with just a single paragraph. I wanted to scrutinize it in more detail – I wanted to follow the rabbit hole and see where it took me. The idea in question concerned the potential of selling data via HANA in the Cloud.
Here’s how I described it in my last blog.
Just because a corporation is using HANA in the Cloud to store its data doesn’t necessarily mean that the corporation in question must develop its own applications that use this data. It is now common knowledge that SAP is creating a new store for its OnDemand offerings. Usually, this store is seen as selling applications in a variety of forms for a variety of platforms. What if this store also offered data that was stored in HANA in the Cloud.
At first, I thought that no one else had a similar idea – selling data via OnDemand platforms. Then, I started researching and found that there are actually a number of companies (usually called “data markets”) already active in this area. As a result, my focus changed from defining something non-existent to examining the competitors and learning from their offers. This comparison also allowed me to better understand how SAP fits into the broader OnDemand space and the impact that changes in this area might have on SAP’s brand image.
HANA in the Cloud
Note: In my quest to find an understandable definition of HANA in the Cloud, I’ve discovered a bit of confusion between the terms HANA in the Cloud and HANA AppCloud that appear to be used interchangeably by some in the ecosystem. In this blog, I’ve decided to use the term HANA AppCloud, because it more closely fits my focus. I’d recommend that SAP work to clear up this confusion in terminology before customers and influencers start making unfortunate assumptions regarding SAP’s future intentions in this area.
What is the HANA AppCloud?
Like other cloud platforms, the AppCloud will involve server farms coupled with a software fabric for elastic resource provisioning. HANA will run on top, allowing customers and ISVs to run analytics or build applications. [SOURCE]
It will allow customers to upload data to the vendor’s own cloud setup for processing, rather than deploy related infrastructure in-house” [SOURCE].
Although the quotes above focus on providing this platform for SAP customers and ISVs to create applications, it largely ignores the potential of SAP itself to offer applications in this environment. SAP already has experience providing such OnDemand applications (for example, SalesOnDemand) although these apps run in different environments (for example SAP’s Core Platform as a Service (PaaS) rather than the HANA AppCloud.
The two quotes also demonstrate that the environment includes more than just In-Memory technology but also includes more BI-related functionality (“run analytics”)” and also PaaS-related functions (“build applications”).
Note: This blog doesn’t look into the possible performance problems associated with certain use cases in this environment. See fellow SAP Mentor Vijay Vijayasankar’s blog regarding his concerns in this area (for example, the possible limitations associated with hosting Sales and Operations Planning activities on the HANA AppCloud).
It is also an assumption that existing applications can not and should not be moved 1:1 from OnPremise environments to the HANA AppCloud. In a recent blog, analyst Dennis Moore describes a similar perspective regarding HANA: “The longer term benefits of HANA will require new software to be written – software that takes advantage of objects managed in main memory, and with logic pushed down into the HANA layer”. Dennis’ comments concerning the necessity of changing apps are also relevant for transitions to this new OnDemand platform.
There is obviously a variety of applications that could be hosted by SAP in this new environment and one necessary requirement on the part of SAP is to identify those use cases that exploit the unique characteristics of this new platform. The identification and publication of such use cases are of value to SAP as well as to customers and partners trying to understand the potential of this new environment. Such descriptions should also be based on the realization that the HANA AppCloud is not perfect for every use case.
The data market described in this blog is one example of a SAP-created application that runs on the HANA AppCloud. This data market will, thus, be a Software as a Service (SaaS) offering albeit one that is based on the HANA AppCloud.
The role of the HANA AppCloud in SAP’s OnDemand Strategy
There are other similar SAP OnDemand offerings (Core PaaS, Edge PaaS) that are either present in the marketplace or currently being developed. What is the distinction between the HANA AppCloud and these other offerings?
Note: I’m on very shaky ground here, because current information about the HANA AppCloud is very limited.
I’m assuming that the architecture for the HANA AppCloud will based on the HANA architecture with support for multi-tenancy and usage monitoring. To learn more about the details of the offering, we’ll just have to wait until September when more information is expected.
Until September, however, the absence of concrete information regarding the HANA AppCloud is dangerous and threatens to disturb the crisp lines of SAP’s OnDemand strategy / architecture. Since many customers and influencers are still struggling to understand this strategy, smudges in the form of uncertainty and unclear marketing messages are unfortunate.
If you look at SAP’s architecture for these other PaaS offerings, InMemory technology is usually the foundation and described as the data storage layer. Where does the HANA AppCloud fit into this architecture? Is it a separate offering or just a new flavor of the existing offerings?
Figure 1 Integrated HANA AppCloud
Figure 2: Separate HANA AppCloud
Indeed, I find the use of the name “HANA AppCloud” dangerous, because it cuts into the “territory/functionality” provided by the other SAP PaaS offerings. Indeed, if you look at the one example of the a customer using the platform shown at the Sapphire – Medidata – [SOURCE] – you will see that Medidata has a SaaS application that is “built on” the HANA AppCloud. Since I have no idea if the application itself is deployed on the HANA AppCloud or just the involved data storage, it is difficult to make a concrete analysis of the role of the HANA AppCloud in this particular application. This confusion is indicative of the unclear messages surrounding the HANA AppCloud.
In my opinion, the HANA AppCloud should have restricted functionality and a focus on analytics rather than the broader focus of the other PaaS platforms. The resulting question concerning future distinctions between BIOnDemand and the HANA AppCloud is, at the current time, also unanswerable.
Since SAP’s other PaaS environments might also provide an excellent hosting environment for our data market, why is the HANA AppCloud the preferred option? For example, hosting the data in River would allow for the REST APIs to be created automatically. The particular characteristics of the data market lead to the conclusion that the HANA AppCloud would be optimal choice – the other potential candidates include a much wider functional foundation – our focus is on data and analytics and thus the fact that the HANA AppCloud concentrates on this same functionality suggests an ideal marriage – one that meets the requirements of our use case simply without complicating the required PaaS with unrelated and unnecessary features.
Background on Data Markets
Before we examine data markets in the context of the HANA AppCloud, we need some background information first.
The increasing accessibility and speed of Internet has enabled emergence of an online market where data providers publish their valuable information and from which customers subscribe and query data of all types. This virtual space in which the supply-demand forces exist and the exchange of information is incentivized by monetary gain is referred to as the data market
What motivates companies to contribute their data to such environments? The following quote focuses on geo-companies but is relevant for a variety of other companies as well.
The problem I see is the data market place concept is growing like wildfire as the go-to business model – especially for geo companies. The result is a proliferation of data silos that are increasingly proprietary and closed because that is where companies hope to extract payment. As we lock down data we are going against the grain of the Web itself. Data is a commodity and creates the greatest economic multiplier when we treat it as a public good.
This does not mean all data should be without cost, but it should be free to flow across the Web [SOURCE]
Note: The nature of data markets and the manner in which users interact with these information sources is very relevant to Gateway and the manner in which this offering from SAP evolves in the future. Since the Go-To-Market strategy for Gateway is still relatively immature, the lessons learned / experience from data markets should be used to jump start such efforts.
The presence of a large number of companies offering services in this area shows the value in participating in such arenas. The market contains a wide variety of players – more traditional companies such as Reuters and a number of start-ups (for example, Timetric and Kasabi) – that are active.
Of those companies that are presently in the marketplace, the one I find the most interesting is Microsoft with Windows Azure Marketplace DataMarket.
Windows Azure Marketplace features DataMarket, a new cloud-based service that provides a global marketplace for information including data, web services, and analytics. With DataMarket, content providers can make their datasets available to a wide audience around the world, subscribers can locate a dataset that addresses their needs through rich discovery, and developers can write code to consume the datasets on any platform. [SOURCE]
When I discovered this offering from Microsoft, I was immediately intrigued. An examination of why Microsoft had invested heavily in their data market (not only in terms of marketing but infrastructure as well) might provide interesting insights into the market for data services as well as the general market for OnDemand services
Microsoft uses the Windows Azure Marketplace DataMarket.as a way to promote their Azure platform. The data market demonstrates the potential of their OnDemand platform. This is also evident by the fact that “Windows Azure” is included the product name.
Furthermore, because DataMarket is built on Microsoft Azure® and runs in industry-leading data centers, you won’t need to make heavy investments in hardware. The service provides almost unlimited scalability and can guarantee high availability. And when you need to increase the size of your dataset, DataMarket scales smoothly with your requirements. [SOURCE]
The fact that vendors use vendor-developed SaaS applications to prove the worth of their PaaS environments is also one motivation in SAP’s activities as well. For example, the fact that CarbonImpact is based on River / the Edge PaaS as well as the creation of the LoB OnDemand applications on the Core PaaS platform allows SAP to demonstrate the value of these PaaS platforms. The Data Market from SAP could fulfill a similar function for the HANA AppCloud.
An apology: Some of this blog focuses on this Microsoft offering and I use it as an example that should be followed. I’m looking at this Microsoft technology in a general fashion. I’m definitely not a Microsoft Azure expert. Some may question whether a detailed analysis of a Microsoft offering is really appropriatein a blog focused on SAP but emulation is always the best example of flattery.
What are some of the characteristics of this platform that I found the most interesting?
The use of oData as a common interface for their data
The DataMarket from Microsoft uses consistent REST-based OData APIs across all datasets. Thus, developers can use one set of tools to access data from a variety of sources. This is similar to the concept behind Gateway.
There are other approaches to data market APIs – for example, community-developed APIs – that are also fascinating.
Each of those datasets has a minimum of 5 APIs. I say “minimum” because one of the core features of Kasabi is that we’re allowing the community to create their own APIs over datasets. In Kasabi the community can curate their own APIs onto the data we’re hosting. This means you can use an existing off-the-shelf stock API auto-deployed by the system itself, an API created by another community member, or create your own pathway into the data which can then be shared with others. [SOURCE]
A unified billing infrastructure
There are different billing options – subscribers choose pay-as-they go transactions, monthly subscriptions, or even enterprise volume licensing – in Microsoft’s offering that enable a wide variety of individuals to use the data market services on their own terms.
Here are some details of such possibilities:
[The data market] provides all the facilities a data provider needs to monetize the intrinsic value of carefully created datasets, which are subscribed by customers.
The dataset are priced as monthly subscriptions which are of two types. In the unlimited subscription type, the consumer is charged monthly for access to dataset and the transactions on the dataset are unlimited. For the limited subscription type, each month there is a pre-defined number of transactions that can be executed on the dataset. Each page of results returned from a query uses a single transaction and will count toward the transaction limit. [SOURCE]
I find the transaction-based billing especially interesting in that it is a relatively modern way of looking at such services and represents a break from typical licensing structures so often present in enterprise software. Such functionality might not only be relevant for our data market but for other applications based on the HANA AppCloud.
Integration into existing Information Worker tools
There are various Microsoft DataMarket add-ons for Microsoft Office products that enable normal Information Workers – not just developers – to easily access these services.
You can even use the DataMarket Add-in for Excel to discover, purchase, and use DataMarket datasets without ever leaving the familiar Excel environment—and then integrate your data with PowerPivot for Excel for rich, self-service business intelligence and Bing™ maps to use spatial datasets for quick, visual instant answers. [SOURCE]
Imagine the potential of users being able to integrate data from a HANA AppCloud-based data market into their daily work environment – it would democratize HANA data.
OnPremise HANA installations could provide this as well but with the restriction that employees are usually restricted to only analyzing the data of their own company. The potential of the HANA AppCloud would be the ability to access data from a variety of sources – many of them external.
What could motivate SAP to move into data markets
Increasing ROI for existing customers
As I mentioned in my last blog, one necessity of SAP’s strategy is to increase the ROI of the SAP installations in the landscapes of existing customers. One manner to do this is to find innovative ways to re-use existing data in such environments
One of the best descriptions of the potential of data markets for such customers companies is from Roger Mall of the Windows Azure Marketplace DataMarket product team.
New revenue streams
How does SAP earn its money from a data market? SAP earns money by taking a cut from the subscription fees earned by content providers.
I like the strategy of one data market called InfoChimps: “Posting data for sale on Infochimps is free! We only make money if your data is for sale and if it is purchased. Our standard commission rate is 30%”.
Indeed, today’s focus on open government with the need to publish data publically from government sources would be perfect set of customers for such environments (A fact that has not gone unnoticed by Microsoft).
Thus, there are a wide variety of potential content providers whose sale of data in the SAP data market could earn a great deal of money.
Providing data as part of Corporate Social Responsibility (CSR) activities
If you look at many of the other data market offerings, there is often a certain amount of data that is available free of charge. This is often data from the United Nations or other NGOs. SAP could promote its data market and offer participation in the data market to such entities at a negligible or no fee as part of its CSR activities.
What is SAP’s competitive advantages in this area
Existing user base
Many of the data market start-ups must identify and acquire new customers in a timely and lengthy process. As SAP often says, its software is in place of many of the world’s largest companies. If just a small percentage of these customers provide data for the data market, SAP would have a dominant role in the data market marketplace.
Developing methodologies to discover the appropriate data to put in the data market
Many of the data markets present in the marketplace (with this exception of Microsoft) don’t really have many content providers in their individual marketplaces. One problem that causes this dilemma is the difficulty in identifying which data would be successful in such environments. Therefore, SAP must develop methodologies (using market analysis and the industry-specific experience of SAP consultants) to discover which data would sell well in such a data market. Indeed, SAP might use the marketing muscle of its own sales organization as well as that of the associated partners to make sure that such offerings are successful.
Appropriate pricing structure for the data
Data markets are still a relatively young and dynamic market; therefore, the appropriate pricing of such data services is still open to much debate. There is a interesting article entitled “Pricing for Data Markets” from Avanish Kushal, Sharmadha Moorthy and Vikash Kumar which shows the difficulty in setting the right price for the data being offered.
Discovering technologies to effectively move data into the HANA AppCloud
Once the customer data has been identified that should be placed in SAP’s data market, there must be secure and efficient ways to move this data from the OnPremise SAP systems into the HANA AppCloud. Since mechanisms already exist to move the data from existing OnPremise systems into OnPremise HANA installations, I’m assuming that such tools (Sybase Synchronization tools?) should be able to be reused to some degree for our scenario.
What happens if SAP doesn’t decide to offer its own data market based on the HANA AppCloud. The architecture of the Microsoft Azure Data Market also permits the inclusion of 3rd party clouds. As the picture below demonstrates, this feature would allow that data from Gateway and the HANA AppCloud could still be sold via this platform.
Such an integration would mean a duplication of functionality between the SAP and Microsoft-based environments but would still permit the data to be sold in this environment.
Other Interesting Possibilities for Using InMemory Technology in OnDemand Offerings
This blog has focused on one particular use case that is based on one usage pattern of InMemory technology in OnDemand settings. There are, however, other ways that the resulting new features will impact SAP’s OnDemand offerings.
During a recent conversation between Vishal Sikka (CTO SAP) and bloggers where we discussed HANA, one topic was the general relevance of InMemory technology for cloud computing. I always focus on the impact of this technology on OnDemand applications. Vishal, however, focused more on the resulting revolution at the underlying infrastructure level– a shift away from the multitude of small servers in existing data centers to fewer larger servers based on in-memory technology.