Solution Architectures On The Journey to Data Warehouse Cloud
There are many ways to get from a legacy data warehouse to a data warehouse in the cloud. Even more there might be a lot of details which have to be considered when moving from an on-premises environment to a cloud environment.
One methodology which was introduced by SAP is the so-called BW Bridge accelerator workshop. The main part of this workshop is called the discovery day. On the discovery day we get into an understanding of the customer’s as-is architecture, envision possible target architectures and discuss various transition architectures together with the customer. In this blog post we focus on the solution architecture. In a further blog we will discuss the business value of SAP Data Warehouse Cloud we evaluate in the BW Bridge Accelerator Workshop.
Meanwhile we executed this workshop several times and these findings shall be summarized here.
The following architecture shows a typical situation for an as-is architecture with these components:
- One or more legacy SAP BW systems, either 7.3 and higher or already BW/4HANA
- Reporting tools: Either from SAP like Analysis for Office, SAC, Business Objects or non-SAP like Power BI, Web Focus, XL Cubed
- Planning solutions from SAP like BI-IP, BPC or non-SAP solutions like TM1, HFM, …
- ABAP Developments
- Usage of Java stack for BI Launch Pad or scheduling (Information Broadcasting)
- Source systems from SAP ECC, either with standard S-API extractors or already within the ODP framework or ABAP CDS views. Non-SAP sources like relational databases or flat files.
- Mixed modelling with external SAP HANA calculation views.
- Add-Ons in the BW-ABAP Stack. An example might be DMIS if an SLT was used or SEM-BCS
In the sequel we discuss these points in detail and show what has to be considered and what are the right questions to pose.
ad 1. If a customer is still using BW 7.5 the usual question is “Shall I convert to BW/4HANA or shall I move right directly into SAP Data Warehouse Cloud (DWC)“. In order to decide this we take into account that BW 7.5 has end of maintenance at the end of 2027. So our recommendation with regards to the above questions would be to move right directly into SAP Data Warehouse Cloud (DWC) but under the prerequisite that this is possible until 2027. Otherwise a conversion to BW/4HANA could be necessary (or you go for extended maintenance of BW 7.5).
If a customer has already moved to BW/4HANA a DWC nonetheless will provide advantages and several quick wins if we think about the new possibility to connect to cloud data sources, see ad 6.
ad 2. Key criterium here is that with regards to frontends SAP Data Warehouse Cloud (DWC) allows access for SAC (web-based), SAP Analysis for Office (Excel-based) and SAC, add-in for Microsoft office beside ODBC-based non-SAP frontend tools like e.g. Power BI. In general it can be said that the migration of SAP frontend tool whose data source are BW queries is a two-step approach:
- First the BW query has to be transformed into a DWC model with the BW model transfer (semi-automatic)
- Second the reports have to be manually rebuilt on the DWC models.
This could mean a considerable effort. If a customer is just in the planning of an SAC migration then we would favor the deployment of a DWC in order to develop the new SAC reports already now on the right data sources and omit a further frontend migration project in the near future. Depending on the number of reports this migration will take some time and a hybrid architecture consisting of the legacy BW system and in parallel a DWC would be the recommendation here. Reports could be rebuilt one-by-one into DWC. This provides also some time buffer if features of BW queries cannot be migrated in a fully automated manner.
ad 3. For planning we will have the new SAC/DWC planning solution in 2023 but at the moment it is not yet quite clear how to migrate existing BI-IP or BPC solutions to the new SAC/DWC planning solution. Non-SAP planning will have to be replaced by SAC/DWC planning in any case. Also this is a scenario where a hybrid transition architecture would help until a clear way for planning is published.
ad 4. ABAP developments beside those occurring in user exits like e.g. expert routines in transformations are possible to be taken over into the ABAP stack of the BW bridge. It is also here that we saw that customers are willing to clean up legacy developments and replace native ABAP developments with something completely new.
ad 5. The Java stack is mainly used for a kind of BI portal but also for scheduling purposes as e.g. information broadcasting. Scheduling of reports cannot be taken over 1:1 into SAC. We come across a similiar requirement with customers using Business Objects as well: BO has also scheduling features which have no exact counterpart when trying to move away from BO.
ad 6. Data sources based on the ODP framework can be taken over to BW Bridge without any further efforts. Other SAP data sources and non-SAP data sources like e.g. relational databases have to be rebuilt in a way to be ingested directly into DWC. This means that data models which reside in the legacy BW have to be remodelled in DWC as SQL data warehouse which also means probably some effort.
But it is also at this point that something quite new enters the game and will provide a real quick win with DWC, namely cloud data sources. As we know SAP BW is a quite mature product developed during the last 20 years but mainly for SAP applications on-premises like ECC systems. There were also non-SAP sources like relational databases via dbconnect or flat files but SAP BW is definitely not really capable of connecting to cloud sources like e.g. SAP cloud LOB solutions as Success Factors, Ariba, Concur, …., but also non-SAP cloud products as e.g. all kind of cloud data lakes (Azure, AWS S3, GCP,…). And these cloud sources are definitely something which is a must to consider already now for every customer. SAP DWC offers many out-of-the-box connectors to such cloud sources.
Another quick win might be to replace flat file interfaces by their actual data producer: This means that instead of downloading the data from some tool and then upload them again into BW, try to ingest these data directly into DWC from the tool which produces the data.
Also when considering these cases we mostly end up with a hybrid transition architecture.
ad 7. There are several reasons why customers might use external HANA views which can be generated from BW infoproviders:
- Third-party reporting tools:
- Performance improvements
- Push transformation logic from BW transformations to calculation views
Whereas the first case (3rd party tools) needs a migration into the ODBC-based SQL spaces, the latter two have to be considered in detail case by case. Especially the last one which can include quite tightly interlocked data models which use both external and BW objects might become challenging to get translated into DWC. It shall also be mentioned at this point that DWC allows an integration of the HANA Development Infrastructure (HDI) such that HDI containers can be integrated into DWC modelling.
ad 8. Add-Ons have to be investigated case by case. The answer for SEM-BCS is usually that consolidation moves into group reporting in S/4HANA.
The Quick-Win Architecture
The first step when implementing a DWC should always be a quick win like e.g. connecting to a cloud data lake of a hyperscaler but also to cloud applications like Success Factors or Ariba. A third quick win might be to have a quick integration possibility with other machine learning solutions.
Let us briefly explain what we mean with the Virtual Layer and the Federation Layer within Data Warehouse Cloud. The Virtual Layer shall provide a stable interface for reporting tools whereas in the Federation Layer DWC models are created. The Virtual Layer is comparable to composite providers in SAP BW.
Ingest non-SAP Sources to DWC
Another scenario which is often found are non-SAP data sources or non-ODP SAP data sources of the legacy BW system. These data sources cannot by ingested into a BW bridge but they have to be ingested into DWC directly. However this needs some effort because the BW applications have to be analyzed and then the BW data models have to be re-built in DWC.
Find Replacements for Planning and Other Applications
Many customers use SAP planning solutions like BI Integrated Planning (BI-IP) or Business Planning and Consolidation (BPC) or non-SAP planning solutions. SAP’s strategic planning solution will be the integrated SAC-DWC planning solution (Planning-as-a-Service). It is also here that a solution is not straight forward – especially the migration path from the legacy planning solution to the new SAC-DWC solution. If a customer still uses the consolidation solution SEM-BCS the strategic answer would be migration to group reporting in S/4HANA
Get Rid of the Legacy Business Warehouse
Before the final step all reports having as data source a BW query have to be migrated to reports with a DWC view as data source. Remaining non-ODP data sources to SAP applications have to be replaced by ODP data sources.
The Final Architecture Option With DWC BW Bridge
After the final step the target architecture could look like this:
Resumee and Suggestion for Project Timeline
We saw that in most cases a hybrid transition architecture suites in most cases as a first step:
- Connect DWC to new cloud sources and federate them with legacy sources (e.g. from BW or ECC) to get a quick win as early as possible (Quick Win Architecture).
- Keep the legacy BW system as long as you need it to migrate reporting, planning and other features which are not yet fully supported by DWC. Try to do this until 2027, otherwise think early enough to plan a BW/4HANA conversion.
- The size of the legacy BW system will be continuously shrinked whereas on the other side the size of DWC grows.
- Think about leveraging BW bridge from the beginning and move your BW applications continuously to BW bridge during the whole transition cycle.
- Consider as an alternative also the option to re-implement the BW systems completely into the DWC core part as a greenfield approach. So no BW remains in the end at all: Neither as legacy on-premises nor as DWC BW bridge.
The following shows a proposal for a possible project timeline with the hybrid architecture ansatz and target DWC BW Bridge. Take into account that the actual length of the project will heavily depend on the used components of the legacy BW 7.5 system.
Check Plan B: A priori we do not suggest a migration to BW/4HANA before the BW Bridge migration. But it should be permanently checked whether the suggested migration schedule of BW 7.5 to BW Bridge is in time. Should this target be endangered, one should think in good time to achieve a migration of BW 7.5 to BW/4HANA before the end of maintenance of BW 7.5 at the end of 2027.
A lot of official information can be found in SAP notes
In the final architecture option I see 3rd party tools such as Tableau and WebFocus connecting to the DWC Virtual Layer. Where would the recently released DWC OData API fit in this architecture?
You could add PowerBI in the Final To-Be Architecture (see above) into the 3rd-Party box in the frontend layer (right below Tableaux and WebFocus which have to be seen merely as examples here). The arrow would also show to the Virtual Layer and the connection type would then be OData API (in a black oval shape).
HTH, BR, Ingo
Thanks Ingo. That was helpful.