There are significant innovations in the offering of SAP in the SQL Data Warehouse* (SQL DW) space. These innovations have taken place over the past few years, but together provide a solution with which you can build an SQL DW in a different way then you did before. I’m going to write a series of blogs about this. (See the end of the blog for my definition of SQL DW)
To be clear, for Data Warehousing, SAP offers two flavors: SAP Business Warehouse and SAP HANA SQL DW. You can also perfectly combine the two. However, in this blog I’m treating only the SQL DW topic.
The “Classic” SQL DW
SAP has been supporting the SQL DW for a quite a while already – mostly through products acquired in the past decade. SAP products like Data Services, Information Steward, and PowerDesigner served thousands of SAP Data Warehouse implementations for decades, and still do.
How does such a data warehouse work? Pick any supported database to run these standalone tools on, and you’re good to go. I’d say go for SAP HANA, but you’re free to choose otherwise as these tools support a large variety of RDBMS. And, you could replace any of the above SAP products by any 3rd-party vendor tool, as long as they support your database of choice. That’s why this approach is often called “best-of-breed.” I call this the “classic SQL DW,” contrasting it to the new tool set and approach that goes by the name of “SAP HANA SQL DW.”
So What’s New?
There have been significant changes for many data-focused products in the past two years, but these changes are often not described in a DW context. For a while now, you can build your SQL DW completely on the HANA platform, without the need for standalone tooling. The SAP HANA SQL DW approach allows you to do things not possible before. Below, I’ve describe the four changes I think are most relevant.
Although many DW functions were already available on HANA 1, several other features and integration between tools have been delivered with SAP HANA and the integrated XS Advanced. So SAP HANA 2 is definitely what you want to go for.
#1: Integration of All DW Functions on the SAP HANA Platform
All tools required to build the SAP HANA SQL DW are part of the SAP HANA platform. There’s no need for standalone tooling as in the classic SQL DW. In a next blog, I’ll describe these tools in more detail, but this is a short sum up of the major data warehouse functions of the platform and the component used for that function:
The main advantages of this platform integration I see are the ease of configuration and ease of development: the strong integration between the different components result in a minimization of tool-break. And although integrated, it still provides an “open” data warehouse, as all data can be freely accessed by tools outside of the platform, and freestyle SQL or programming still allow a developer to go beyond what the standard tools provide.
#2: Release of New DW Foundation Functions
The first DW Foundation tools were released for HANA 1, with the Data Distribution Optimizer and the Data Lifecycle Manager. With HANA 2 (SPS01) the Native DataStoreObject” was released, as well as the indispensable DW Scheduler. You will find them in the WebIDE, after you install the DWF plugin.
I’ll go into more detail on DWF in a next blog, but you could already check out the Native DSO in this youtube video series.
#3: Support for DevOps Processes
The XS Advanced platform and the WebIDE provide game changers for development and release processes.
First, each developer works in his own “container,” and therefore developers or development teams can easily isolate themselves from others. For customers where different teams develop and release on the same data warehouse, this reduces the number and severity of collisions.
Second, the WebIDE stores all development artefacts in GIT, an open source repository. It does not use the HANA repository, where design time objects were stored by default in classic HANA development. This gives you advanced version management like “branching”, which supports the isolated development mentioned above.
Third, the platform supports open source tools to release and deploy. Tools like Jenkins can be used to set up automated deployments and testing.
These technologies then enable you to work in a DevOps mode with Continuous Integration, Continuous Testing, and Continuous Deployment.
#4: Move to the Cloud
On the SAP Cloud Platform, SAP offers CloudFoundry. And guess what? On this cloud platform, you can run the same apps as on XS Advanced. For example, this is where you find the WebIDE as well. And this means that whatever you build on the WebIDE on premise, you can run it in the cloud too. Having said that, at time of writing not all DW features are available yet on CloudFoundry, and I haven’t tried it out yet. But this screenshot of the WebIDE on SAP HANA 2 SPS02 sure looks promising…
Besides the fact that this gives you the option to build a SQL DW in the cloud, this also provides you with an important benefit—whatever you build on premise now, can be leveraged when you would want to move to public Cloud. And that also goes for your data warehouse.
Conclusion
The integration of several tools onto the SAP HANA2 + XSA platform, and the release of DW foundation tools for that platform, allow you to drive your SQL DW as you never did before, with the platform supporting DevOps processes and cloud deployments. I hope to zoom in into these specifics in following posts.
See my next post on the SAP HANA Enterprise Architect Designer.
* My definition of “SQL DW”: Data Warehouse built with tools that primarily support the SQL language, built on a relational database. Even though the DW tools might hide generated SQL behind a user interface, developers need an understanding of SQL to proficiently use the tools. A SQL DW is characterized by its openness— it usually supports any access to data by any tool that supports SQL, as these requests go straight to the database.
Explore SAP's next generation data warehousing solutions.
To be clear, for Data Warehousing, SAP offers two flavors: SAP Business Warehouse and SAP HANA SQL DW. You can also perfectly combine the two. However, in this blog I’m treating only the SQL DW topic.
The “Classic” SQL DW
SAP has been supporting the SQL DW for a quite a while already – mostly through products acquired in the past decade. SAP products like Data Services, Information Steward, and PowerDesigner served thousands of SAP Data Warehouse implementations for decades, and still do.
How does such a data warehouse work? Pick any supported database to run these standalone tools on, and you’re good to go. I’d say go for SAP HANA, but you’re free to choose otherwise as these tools support a large variety of RDBMS. And, you could replace any of the above SAP products by any 3rd-party vendor tool, as long as they support your database of choice. That’s why this approach is often called “best-of-breed.” I call this the “classic SQL DW,” contrasting it to the new tool set and approach that goes by the name of “SAP HANA SQL DW.”
So What’s New?
There have been significant changes for many data-focused products in the past two years, but these changes are often not described in a DW context. For a while now, you can build your SQL DW completely on the HANA platform, without the need for standalone tooling. The SAP HANA SQL DW approach allows you to do things not possible before. Below, I’ve describe the four changes I think are most relevant.
Although many DW functions were already available on HANA 1, several other features and integration between tools have been delivered with SAP HANA and the integrated XS Advanced. So SAP HANA 2 is definitely what you want to go for.
#1: Integration of All DW Functions on the SAP HANA Platform
All tools required to build the SAP HANA SQL DW are part of the SAP HANA platform. There’s no need for standalone tooling as in the classic SQL DW. In a next blog, I’ll describe these tools in more detail, but this is a short sum up of the major data warehouse functions of the platform and the component used for that function:
- Compute and data store: SAP HANA database is the primary data store and compute engine. Of course, DW implementations might leverage secondary data stores and compute sources.
- Data ingest: accessing and importing source data is done with the Smart Data Integration (SDI), Smart Data Streaming (SDS), and Smart data Access (SDA). They support different ways of data ingestion, but are similar or same in use, with the primary purpose to provide data connectivity to and from the SAP HANA platform.
- Design of the data warehouse: Enterprise Architect Designer. Where in the classic SQL DW you would use SAP PowerDesigner to model your DW, EA Designer is the browser-based DW modeler that integrates with the SAP HANA platform.
- Development of the data warehouse: WebIDE with the DW Foundation add-on. The WebIDE app is where developers will spent most of their time, as activities like data definition, data transform, scheduling, and data preview are all done from here.
The main advantages of this platform integration I see are the ease of configuration and ease of development: the strong integration between the different components result in a minimization of tool-break. And although integrated, it still provides an “open” data warehouse, as all data can be freely accessed by tools outside of the platform, and freestyle SQL or programming still allow a developer to go beyond what the standard tools provide.
#2: Release of New DW Foundation Functions
The first DW Foundation tools were released for HANA 1, with the Data Distribution Optimizer and the Data Lifecycle Manager. With HANA 2 (SPS01) the Native DataStoreObject” was released, as well as the indispensable DW Scheduler. You will find them in the WebIDE, after you install the DWF plugin.
I’ll go into more detail on DWF in a next blog, but you could already check out the Native DSO in this youtube video series.
#3: Support for DevOps Processes
The XS Advanced platform and the WebIDE provide game changers for development and release processes.
First, each developer works in his own “container,” and therefore developers or development teams can easily isolate themselves from others. For customers where different teams develop and release on the same data warehouse, this reduces the number and severity of collisions.
Second, the WebIDE stores all development artefacts in GIT, an open source repository. It does not use the HANA repository, where design time objects were stored by default in classic HANA development. This gives you advanced version management like “branching”, which supports the isolated development mentioned above.
Third, the platform supports open source tools to release and deploy. Tools like Jenkins can be used to set up automated deployments and testing.
These technologies then enable you to work in a DevOps mode with Continuous Integration, Continuous Testing, and Continuous Deployment.
#4: Move to the Cloud
On the SAP Cloud Platform, SAP offers CloudFoundry. And guess what? On this cloud platform, you can run the same apps as on XS Advanced. For example, this is where you find the WebIDE as well. And this means that whatever you build on the WebIDE on premise, you can run it in the cloud too. Having said that, at time of writing not all DW features are available yet on CloudFoundry, and I haven’t tried it out yet. But this screenshot of the WebIDE on SAP HANA 2 SPS02 sure looks promising…
Besides the fact that this gives you the option to build a SQL DW in the cloud, this also provides you with an important benefit—whatever you build on premise now, can be leveraged when you would want to move to public Cloud. And that also goes for your data warehouse.
Conclusion
The integration of several tools onto the SAP HANA2 + XSA platform, and the release of DW foundation tools for that platform, allow you to drive your SQL DW as you never did before, with the platform supporting DevOps processes and cloud deployments. I hope to zoom in into these specifics in following posts.
See my next post on the SAP HANA Enterprise Architect Designer.
* My definition of “SQL DW”: Data Warehouse built with tools that primarily support the SQL language, built on a relational database. Even though the DW tools might hide generated SQL behind a user interface, developers need an understanding of SQL to proficiently use the tools. A SQL DW is characterized by its openness— it usually supports any access to data by any tool that supports SQL, as these requests go straight to the database.
Explore SAP's next generation data warehousing solutions.
- SAP Managed Tags:
5 Comments
