Data Loads from Data Stage to BW
The WebSphere DataStage Pack for SAP BW integrates non-SAP data into SAP Business Information Warehouse.
This pack populates the SAP warehouse with data from any source system:
- Enterprise data warehouses, complex flat files, mainframe legacy systems, customer systems, and supplier systems.
- Other enterprise applications, through the DataStage Packs for Siebel, PeopleSoft Enterprise and JD Edwards Enterprise One, and Oracle Applications
Using SAP’s standard business APIs, the DataStage Pack for SAP BW automates the process of connecting to an SAP source and selecting source data through metadata integration, data mapping, and direct data load into SAP BW.
This pack provides direct access to, and creation of, SAP BW metadata from the DataStage user interface. You can browse, select, create, and change SAP BW metadata objects such as Source Systems, Info Sources, Info Objects, Info Catalogs, and Info Packages.
The pack also helps you develop BW integration jobs from a single environment. DataStage Pack for SAP BW does not require pre-work in SAP BW before you can set up integration jobs.
The DataStage Pack for SAP BW includes the following interfaces:
Staging Business API (BAPI) interface
The BW load plug-in uses SAP staging BAPIs to load data from any source into SAP’s Business Information Warehouse (BW). Load events can be initiated from BW or DataStage. You can stream data into SAP BW without writing the data to disk during the process.
This stage has the following functionality:
- Create and modify SAP BW InfoObjects (Characteristics and Key Figures) based on column level metadata as represented in WebSphere DataStage Table Definitions, job links, and so forth.
- Create InfoSources that are comprised primarily of InfoObjects generated by the Load stage.
- Create InfoPackages to load data from the Load stage, including load requests initiated by the stage. (You can schedule the load using SAP BW.)
- Supports the use of job parameters for the source system and connection properties of the stage.
- Integrated with the InfoPackage third party selections for the SAP BW user interface to allow the display of WebSphere DataStage job parameters and possible values.
- Supports the creation of third party Source Systems.
- Integrates with the SAP BW GUI InfoPackage monitor to display the WebSphere DataStage job log from within the SAP BW GUI.
- Supports data streaming directly to SAP BW.
- Provides multithreaded RFC Servers to support simultaneous load requests.
- Supports loading of master data hierarchies.
- Supports SAP load-balancing capabilities
- Server jobs provide the ability to Reset and validate a job. The WebSphere DataStage Director Reset selection may resolve potential problematic job issues.
RESET: – Server: Checks for previously existing intermediary files – Deletes previously existing intermediary files – WebSphere DataStage Director log updated by runtime. The WebSphere DataStage Director Validate selection confirms job properties and BW connectivity without running the job.
VALIDATE: – Server: Checks Job Open Sequence – Checks all job properties – Checks BW connections – Checks Job Termination Sequence – WebSphere DataStage Director log updated by runtime
A GUI client selects a Source System and an InfoSource from the SAP BW system and displays the corresponding transfer structure into which you load data.
A runtime server stage processes data from an input link, sends its output directly to SAP BW using an RFC Server or to a text file, and generates an associated metadata file.
An RFC Server loads data into SAP BW. A separate instance of the RFC Server is constantly running for each Source System supported by IBM WebSphere DataStage.
An RFC Server Manager starts and stops the individual RFC Server instances, depending on which Source Systems are supported. Requests from SAP BW to load data from a WebSphere DataStage source system can be serviced only if a job produces the required data and metadata files. You can accomplish this by running the job independently before SAP BW makes the load request.
Note: You can also request that a SAP BW InfoPackage supply a job name as part of the load request itself. In this case, the RFC Server, upon receiving the request, runs the specified job to completion before loading the resultant data file into SAP BW.
Tuning load jobs
Change the following environment variables using the DataStage Administrator. The default timeout is 120 seconds. For BW Load variables (timeout is defined in seconds):
- ASC_BWLOAD_PCIPAK_TIMEOUT indicates how long the stage waits for a Process Chain to start an Info Package.
- ASC_BWLOAD_IPAK_TIMEOUT indicates how long the stage waits for the scheduled Info Package to become ready for loading.
- ASC_PROCESSCHAIN_START_TIMEOUT indicates how long the stage waits for a Process Chain to respond to the start request.