This blog provides an overview about the BW Layered Scalable Architecture (LSA) the reference architecture for large scale BW DWHs/ EDWs and the process of customer adoption. Details about the LSA will be given in subsequent blogs.
For a first rapid walk thru the LSA please refer to SAP NetWeaver BW Layered, Scalable Architecture (LSA) for BI Excellence - Webinar Presentation
LSA Background
If your are going to build a house it goes without saying that in advance there is a process of making basic decisions about the look and feel, size and other important fundamentals of the new house before starting constructing the house. You decide on the architecture. As there are many aspects to observe with architecture you should consult an architect who knows about the principles of house-building. This is the more important the larger the manifoldness of your sometimes competing wishes. In short you are pretty aware that if you start constructing a house and the architecture blueprint does not provide a cellar the craftsmen will not create it and it will be impossible to introduce a cellar afterwards.
And now imagine building a house without any architecture blueprint.... probably your house will soon look like this:
Building a large, wide-spanning BW-managed Data Warehouse (DWH) - often addressed as Enterprise Data Warehouse (EDW) - is very similar. But it is unfortunately not so common to think about a stable, future-proof BW DWH architecture before starting the implementation. Using BW the consequences of not having an architecture blueprint are not as disastrous as compared to normal house-building because BW means a unification and standardization of data warehousing by simply using it.
But BW has no knowledge about your concrete conditions and preferences. Will it be a mid-range size BW DWH or a large one on global scale? Will it be a BW DWH for a business unit or a country or a wide-spanning, corporate one?
If these and other questions are not answered and fixed in a architecture blueprint (construction plan) before starting the implementation your DWH will most likely look like this house after a certain time despite the usage of BW:
Yes, it is a house - but likely too much individualism to meet your preferences and requirements with challenging conditions.
The challenging conditions of BI and Data Warehousing are well known: it is a highly dynamic, volatile and incremental world. From Data Warehousing perspective that means a steady growth of data and meta data caused by new BI applications and/or newly incorporated organizational units. Steady pressure to adapt and expand existing BI applications. Sooner or later it will become obvious for everyone that without standardization of data warehouse management described by an architecture blueprint the entire system will become more and more difficult and costly to administrate and operate. The same applies to development and maintainability of BI applications. In short keeping our service level agreements with respect to reporting availability, on time delivery (time to market), overall stability and performance will become extremely difficult.
This brings us back to house building best practices:
Large scale BW architectures should follow today's design & construction principles for large houses: standardized, scalable architecture, no squiggles, efficient usage of materials, earth quake save and a lot more.
The most popular conceptual standardization of data warehousing is known under the term Enterprise Data Warehouse (EDW) introduced by Bill Inmon. The most famous EDW demand is the propagation of a ‘single version of truth'.
Further key words of the EDW concept are:
- extract once & deploy many
- support the ‘unknown'
- controlled redundancy
- corporate memory
- historical complete, comprehensive, granular
- integrated view
- ...
The EDW conceptual offering became popular in the BW community soon after market introduction. Designing a BW DWH based on EDW principles today nearly is a standard requirement for large BW implementations - corporate/enterprise view on information as answer to globalization.
But as there is no copyright on the term EDW it showed that it was very difficult to streamline the way of introducing an EDW and to communicate on this topic. Furthermore customers focus with wide-spanning especially global BWs in addition on 24x7 operations, robustness, template development, corporate roll-out and of course overall scalability. These Topics relate very close to the BW functional offering thus they cannot be covered by the general EDW concept.
These circumstances drove the introduction of a standard for large and/or global-scale BW-managed Data Warehouses - the BW Layered Scalable Architecture (LSA).
The LSA
- Offers design and implementation standards
- Collects years of SAP- and market-experiences (best practice)
- Is built on EDW principles
- Provides a common terminology
or as a complete sentence:
LSA definition
The BW Layered, Scalable Architecture (BW LSA) describes the design-principles of service-level oriented, scalable, best practice BW architectures founded on accepted EDW principles.
As with all high-level definitions that sounds little concrete. Let's have a closer look at LSA principles.
LSA at a glance
We know that architecture is all about making decisions. Having this in mind leads us to the primary task of the LSA:
- The LSA shall help you to identify the areas you should think about and decide on when it comes to a large scale BW EDW.
We also learned that the point in time of reflecting and deciding is important. It is obvious that you cannot analyze all areas in advance (if you start building a house you do not need to decide on type of water taps). Doing so, you would probably never start implementing. Thus another important task of the LSA is helping you when to decide:
- The LSA proposes a rough sequence of topics you have to provide guidelines:
1. decide about the core structures of your BW - the Building Blocks
2. define project implementation standards based on your Building Blocks
3. define standards to administrate and operate your BW
In short the LSA is all about standardization of your future BW. The emphasis is on: where and how to standardize your architecture. The LSA calls these architecture topics ‘Building Blocks'. The LSA offers in addition a set of implementation best practices simplifying and standardizing the BI application development with respect to these building blocks framework and last but not least operation hints.
Building a normal house there are standards and best practices. Despite these standards and best practices the finished houses will look different because of different owner preferences.The LSA at Customer Side (Customer LSA)
The same is true if we look to the relation between the LSA and a customer BW architecture founded on LSA:
The LSA serves as reference architecture designing transparent, flexible, complete, comprehensive, customer BW architectures (Customer LSA).
The Customer LSA describes corporate standards for performant, maintainable, flexible BI applications and robust operations and administration.
As reference architecture it is unlikely that the LSA is implemented in its reference at customer side.
A concrete Customer LSA will always be something unique. Why it is like that? Why not accepting the LSA one to one? The simple truth is that customers are different. They are different with respect to their background, their targets and their priority of services thus the customer architectures (LSA) will be different.To give you an idea what I am thinking of let's have look at the diversity of starting conditions we may have: heterogeneous versus integrated master data, high operational process integration versus legacy, high central governance versus local autonomously organized, high DWH experience versus low one, high BW skills versus low one, high management sponsorship versus low one and so forth.
Or think about the diversity of customer expectations, which are very often driven by painful experiences.Depending on these experiences we observe a high variance of focus and willingness to invest in areas like:
- Availability i.e. operations conditions e.g. 24x7,time zone support e.g. report availability at 8 am (local time)
- Robustness, error-tolerance i.e. Data flows and persistency's (like DSO-Objects) should be resistible against all kind of errors e.g. protection against human errors
- Recovery e.g. fast (SLA) recovery without touching source system
- Application scalability & robustness (protection of investment) e.g. fast new-build of BI-applications without accessing Sources, manage seamless changes in OLTP world
- Scalability (performance)
- Reconciliation (audit trail)
- ....
Ok, if it is not meaningful to implement the LSA one to one, how does the process adapting the LSA at customer side look like?
How to Proceed Setting up a Customer LSA
As mentioned above there highly important topics defining a Customer LSA and less important ones.
This suggests an incremental approach defining a Customer LSA and indeed an incremental proceeding adapting the LSA to customer needs and preferences has proven to be successful.
1. Brainstorming phase - Create a first draft of Customer LSA
In this initial phase, which should not take too long (4 to 6 weeks are normally enough) the LSA building blocks are adapted with respect to the concrete customer situation and preferences. During this phase we strictly follow the 80:20 rule defining standards, which allow managing the majority of data. Don’t be anxious: the LSA allows expansions later on if the foundation is valid. Don’t lose yourself in the jungle of details and concerns. Decide on basic LSA implementation patterns. Remember building a house, you won’t invest in discussing the advantages of two- or three-layer windows before talking about the fundamentals.
2. Piloting phase - Create a productive BI application pilot
In this phase you build a productive BI application, from extraction to delivery, based on first draft of your Customer LSA. This can be either a migration (including redesign) of an existing application (preferable) or a build of a new application. This pilot follows multiple targets:
- Be a proof of concept
- Verify architecture
- Enable a learning curve
- Expand Customer LSA developing implementation and operations standards (s. above ‘perfect' Customer LSA)
- Spread knowledge (train)
A time frame of 1-2 month for this phase is realistic. During this phase the Customer LSA is further completed creating standards beyond the pilot scope.
3. Roll out and Perfect of Customer LSA
During roll out phase the transition of existing BI applications and new BI applications, a continuous expansion, refinement and verification of the Customer LSA will take place with emphasis on implementation and operation standards. Step by step the Customer LSA will become more and more complete.
4. Keep Customer LSA ‘up-to-date'
Architecture is a living thing. Latest technology should be leveraged. This means updating the Customer LSA.
It seems introducing the LSA means a lot of work. Yes it means efforts, like always: ‘there is no free lunch'. But these efforts definitely will pay back! It is also true that most customers have already a rich treasure of existing best practices and guidelines thus we do not start at a zero-level. These documents should be integrated into the Customer LSA after a compliance check. Also SAP materials like ‘How To Guides' should be referenced in the Customer LSA.
You will see in the following Blogs that the LSA does not mean something fundamentally new or that you can forget all you have learned.
The LSA just draws a holistic picture of the architecture for large BW DWHs streamlining best practices, theory and terminology.
More details in my Teched 2009 presentation: The Layered Scalable Architecture for SAP NetWeaver BW on a Corporate Scale’ – lecture BI302
May be a more general presentation from SAPSKILLS 2008 is also of value for you: SAP NetWeaver BI: Data Infrastructure und BI-Maturity – Data Marts, Data Warehouses (DWH), Enterpris...
(Don't become confused only the first two slides are in German :-))