Technical Articles
Extract and Integrate Data from SAP S/4HANA Cloud using ABAP Pipeline Engine
Introduction
ABAP Pipeline Engine integration option provides direct communication from SAP S/4HANA cloud -> SAP Data Intelligence to extract and integrate the data. Repository with ABAP operators are pre-delivered by SAP in SAP Data Intelligence. ABAP CDS Reader operator reads and extracts the data using CDS View data extractors. The SAP S/4HANA Cloud Communication Scenario for ABAP Pipeline Engine is SAP_COM_0532 – enables the access to the CDS view extraction via ABAP Pipeline Engine(available with SAP S/4HANA CE 2002).
Example Scenario
- Analyse the different activities/steps of a business process
- Example process: Order2Cash
Source System: SAP S/4HANA Cloud
Target: Example scenario is to download the data in csv format from SAP Data Intelligence -> System Management tile. SAP Data Intelligence provides various options to extract and integrate data to other target systems e.g., Other SAP systems and 3rd Party Systems.
Example CDS View Data Extractors:
Data Extraction for Sales document Item
Billing Document Item Basic Extraction Data
Process Flow Diagram
Data Flow Diagram
Integration with SAP Data Intelligence via ABAP Pipeline Engine
SAP Data Intelligence example for extracting the data from SAP S/4HANA Cloud using ABAP Pipeline Engine integration option and saving the extracted data in a LOCAL storage.
Step 1: Create a Communication Arrangement using Communication Scenario “SAP_COM_0532” in SAP S/4HANA Cloud.
Step 2: Create a connection of type “ABAP” and Protocol: “WebSocket RFC”.
Step 3: SAP Data Intelligence Pipeline using ABAP Pipeline Engine operator.
Step 4: Configure the Cloud Data Integration Operator with example CDS view data extractor: C_SALESDOCUMENTSCHEDLINEDEX.
Step 5: Save and execute pipeline.
Conclusion
- Data is extracted from SAP S/4HANA Cloud using ABAP Pipeline Engine option
- File will be created in local storage
- Navigate to System Management from SAP Data Intelligence Launchpad->Files
- Expand folder Files under My workspace to find the created file with extracted data from SAP S/4HANA Cloud.
- Click on the icon “Show Actions” to down the load file into the local storage by using the option “Export File”.
Hello Venu,
I found your Blog today, and I guess it will be very unklikely that you read my question or have an answer... but I will give it a try.
I'm using SAP Datasphere, connected to S4, and my Dataflow which worked until last Friday now issues some unclear error messages.
I could not find any answer on a SAP Help page, nor in the support portal.. so any feedback is highly appreciated.
Can you tell me, where the APE (ABAP Pipeline Engine) has a monitor? Can you see the ABAP Sessions occupied by an extraction? Can sessions stuck there? Can I delete them manually somehow?
Feel free to share any page for those answers.
Thanks a lot, Martin
Hello Martin,
Apologises as the Dataflow has stopped working in SAP Datasphere.
On the connection management screen, if the connection is successful to SAP S/4HANA Cloud, I would recommend you to raise a incident to SAP to check the issue and resolve the error messages which you are facing in the Dataflow created in SAP Datasphere.
Thanks & Best Regards,
Venu
Hello Venu,
thank you for the reply, anyway.
Yes, I raised a ticket, which reached Dev. Support today.
However I would like to understand the ABAP Pipeline Engine more... do you know about other Blogs or Help pages?
I found this: https://blogs.sap.com/2021/11/30/sap-s-4hana-cloud-abap-pipeline-vs-cloud-data-integration-engine/
But it only states:
As for the ABAP Pipeline Engine (APE), it is mainly based on Websocket RFC, it is a SAP designed protocol and this API can only be used by SAP target systems...
So it consists of ABAP programs in S4 and it obviously can occupy ABAP sessions, but max. the number that is set via parameter: https://me.sap.com/notes/3281984/
At the moment, somehow one single data flow trigger thinks all sessions are occupied, even if not any other load takes place.
Will be interesting, what the root cause will be...
BR, Martin
Hello Martin,
The root cause for the issue can be clarified in the incident. As we are in the world of cloud computing, I'm afraid not all the aspects of ABAP sessions will be exposed compared to ECC/onPrem solutions.
As per your comment on 27th Nov 2023, you would like to understand more on these questions: "Can you tell me, where the APE (ABAP Pipeline Engine) has a monitor? Can you see the ABAP Sessions occupied by an extraction? Can sessions stuck there? Can I delete them manually somehow?" I will check and see whether these queries can be answered in a new blog or help document.
Thanks & Best Regards,
Venu