/LSA100/ LSA Basic Flow –Layering Data & Logic
Background
With the new Data Flow Template feature it is now possible describing LSA standards directly in BW.
With BW 7.30 there comes a set of 10 LSA data flow templates as an initial offering.
This blog describes the LSA Data Flow Template LSA100 that offers basic yet general layering of data and flow logic.
For general information about LSA Data Flow Templates please refer to
Description
LSA100 offers basic yet general layering of data and flow logic. The propagation layer DataStore object provides reusable data for multiple InfoCubes, which serve different business needs. Reusability is achieved by strictly separating harmonization (e.g. for master data) and quality transformations from transformations that serve a specific business need (InfoCube). All InfoCubes obtain data solely from the propagation layer DataStore objects and never directly from extraction. The propagation layer therefore implements the EDW principle ‘extract once – deploy many’ and is the ‘single version of truth’ for all InfoCubes and the provided reports and analytics.
Picture 1: LSA100 conceptual view
Picture 2: LSA100 in BW 7.30
Target Group
Data flows should always be layered, regardless of the expected data warehouse size. LSA100 is applicable for BWs, which can be qualified as follows:
- Small, mid-range size
- No general predictable volume issues
- Limited geographical reach (time zones)
- Pragmatic customer orientation
- Single source system per DataSource
Implementation Details
- All InfoCubes obtain data from the propagation layer DataStore objects.
- Suitable for extractors that offer a delta load.
- Suitable for full loads if the loaded data offers a complete business picture (also reverse and deletion records!) – Propagation Layer DataStore objects calculate the delta image
- Transformations rules directly between InfoProviders, therefore without using InfoSources
Acquisition Layer
Accept extracted data 1:1. Add additional information to each record like source system, load time, organizational data (like company code) to make standardized administration and addressing of all records possible.
- Extracted data is stored in a PSA element
- The PSA element is defined at DataSource and source system level
- The DataSource should be comprehensive (offer all relevant process information)
- The load history is stored in a PSA element. Note the following impacts:
- No archiving/near-line storage (NLS) with PSA (volume!)
- History is stored in the regular data provisioning flow (performance!)
- Impossible to influence how history of loaded data is stored (1:1 from extraction)
Harmonization & Quality Layer
Apply data harmonization & data quality transformations (if any) between acquisition PSA elements and propagation layer DataStore objects.
- Data harmonization transformations if necessary
- Compounding of InfoObjects to handle multiple source systems that offer data for the same DataSource (an LSA data flow template that supports domains is recommended)
- Mapping of local keys/values to integrated keys/values (common coded)
- Standardized error handling/(integrity) checks via error DTP
- Adding additional information for each record
- 0SOURSYSTEM - Origin (possible part of Propagation Layer DSO key)
- Load time(stamp)
- Other technical characteristics, such as request ID (for ease of administration)
- Other customer defined useful, (stable) criteria to qualify/describe content of record that increases transparency for administrators and business user (assigning country/region for example)
- LSA data unification transformations (always)
Note: No business scenario-related transformations are allowed here! (This would reduce or prevent reusability of the data in the propagation layer)
Propagation Layer
The InfoProvider for the propagation layer is the DataStore object.
- Normal DataStore objects are preferable due to their functionality (calculation of delta images etc.)
- Business key is (part of) the DSO key
- No SID generation
- With unique business key (e.g. COPA) consider using standard DataStore objects with ‘unique data’ option. (This option can be switched on and off in process chains and thus enables single operations like on standard DataStore objects)
- With unique business key and large volumes write-optimized DataStore objects are another option
- The Propagation DataStore objects are related to a DataSource (s. naming – area part)
Business Transformation Layer
Data Mart specific transformations
- Lookups of master data/other transactional data
- Merge of data from different propagation layer DataStore objects can result in dedicated business transformation layer DataStore objects for reasons of synchronization.
Note: Apply business transformation rules only on top of the reusable propagation layer (DataStore Objects)
Reporting Layer
Normally InfoCubes – options:
- Standard InfoCube data hosted on RDBMS and usage of aggregates
- Standard InfoCube data hosted on RDBMS and BWA Index
- BWA InfoCube only (no RDBMS persistence – BW 7.30)
Virtualization Layer
Queries use only MultiProvider (Virtualization Layer) for reasons of transparency and flexibility