SAP DWC Live Connection Architecture and Open Connectors setup with Twitter
This blogpost is concentrated mainly on the explanation of Live connection architecture in SAP DWC and how to setup the live connections using Open Connectors with Twitter. Live connection setup helps business analyze the data sets appearing in Datawarehouse Cloud from multiple source systems, both SAP and non-sap.
Using SAP DWC you can implement hybrid landscape architecture for the organization under single umbrella. With the advancement of on-premise SAP BW systems on cloud applications SAP suggests using DWC for future reporting applications eg. SAC. SAP DWC integrates very easily with the SAP source systems but when it has to connect with non-SAP source systems then you need to setup additional parameters for it. Before setting up any connection in DWC please go through the Workspace concept once from developers point of view.
SAP DWC works as a single point of entry for all sets of data from different source systems and hence is super powerful from the data modeling perspective on Cloud which offers more advanced ETL capabilities. With its tight coupling with SAC which offers primarily data consumption, reporting and Financial Planning functionalities you can expose the analytical data models created in DWC for further reporting. At the backend DWC uses SDA (Smart Data Access) and SDI (Smart Data Integration) for virtual and physical data replications respectively.
SAP DWC Connections Compatibility Matrix for different connection types
DP Agent configuration is only needed to connect the source systems when physical replication of data is required in SAP DWC. And, SAP Basis admin needs to perform the configuration in Cloud Connector when virtual replication is required, eg. creating Data Flows in DWC using views. Cloud connector setup is done using BTP cockpit, a one time setup. The scenario remains same for setting up connections in mobile phones (iOS or android) / tablets. Implementing Cloud connector will help you bypass the firewall using a Tunnel connection. Direct connection has less data latency issues compared to the Tunnel connection.
Below diagrams show the overall connection architecture (simplified and detailed) which is needed to connect SAP systems with DWC.
All the connections in SAP DWC are allocated to a specified workspace. A workspace is a secure area created by DWC Administrator, in which members can acquire, prepare, and model data. The Administrator allocates disc and in memory storage to the space, set its priority, and can limit how much memory and how many threads its statements can consume.
Tip- Check the Prerequisites and Compatibility Matrix of the source systems needed to connect with SAP DWC.
Once you are logged in to DWC, navigate to the Connections from left hand side panel. Select the workspace where connection needs to be built. Pass on the mandatory parameters of source system and Cloud Connector.
Now Validate the connection by clicking on the Validate button in above screen. If no issues encountered then the connection will be successful and a prompt will appear.
If DP Agent is not setup then below screenshot should match with the connection parameters after successful connection.
Open Connectors- It can connect any non-sap system using APIs. The configuration for Open connectors is done via BTP cockpit. If you have access to the BTP cockpit then even Trial account can be used. Open Connectors empowers DWC and provides enormous capabilities to connect any non-sap source systems. In case SAP provided open connectors does not fulfill the requirements then custom Open Connectors can also be built.
Note that inside BTP cockpit, subscriptions need to be added to the Integration Suite.
Meanwhile, in the same browser login to the Twitter account. Using the cookies while performing next steps for configuring Open Connectors we need to authorize access to the Twitter account.
Create Sub-Account inside the main account & Add Instances and Subscriptions.
Click on Integration Suite in the above screenshot to login. Integration Suite is a complete package inbuilt in BTP cockpit to connect any Cloud/ API/ non-SAP products directly with SAP DWC.
Landing page will appear like below showing multiple connector options.
Please click on the required connector. Click on View All button to find other connectors. In this example I will connect Twitter with SAP DWC.
Since, I already logged in to my Twitter account in the same browser in a different tab, I was able to get the above popup to authorize my account.
Provide a name to the connector and click on Validate Connector. If no further issues encountered, you will find a success message and the Connector will show Yes under Enabled column.
Similar to Twitter, there are various other connectors readily available to connect with Live connection in DWC.
Still if you can’t find the connector in the above list, then you have option of creating your own connector.
Click on Connections tab and select Open Connectors. Integrate your SAP Open Connector Account in DWC to access the required source systems.
SAP DWC provides lot of integrity options to club SAP and Non-SAP data in a single platform to perform advanced Data Modelling. It all depends upon the business requirements and the reporting needs of end users to exactly identify the dimensions and measures needed to visualize in SAC. Similar to Power BI, SAP DWC has the capability to perform most granular level of data modelling, create relationships between multiple source tables and then visualize it in SAC. SAP DWC is purely a cloud application and can showcase n to the power of n reporting capability with highest level of performance.