Integrating Web services .xml files on HCI-DS
Consolidating customers’ data through HCI-DS using sources such as Web services and more specifically .xml encoded documents can be a requirement especially in some type of business where the data required to feed IBP (Integrated Business Planning) are not directly stored in SAP but more on a Website.
The most common way to merge and harmonize data within HCI-DS is to use the On-premise platform with data already saved on various modules like ECC, BW etc… but HCI-DS stands for Hana Cloud Integration and being on the Cloud this gives you the opportunity to introduce some new kind of data sets for instance Web services gathered from .xml files. In fact cloud to cloud and cloud to On-premise integration is permitted.
In this paper we will not go over the complete set up of the HCI Data stores but this is aimed to give you both the prerequisite and specification needed to read .xml documents.
Including .xml file format as source in HCI-DS requires couple of steps and one which is crucial is the .xsd file
An .xml file stored on the HCI server cannot be readable without an .xsd template file format.
Once you are ready to start designing data flows and you have your .xml file ready on the HCI server then you need to start converting this .xml into an .xsd file format. Few websites on Internet can convert and generate such a file but in case you need to manually change some part of it here are some explanations.
This file format can also be stored on the HCI server. By working on HCI-DS and IBP you should have access to the client HCI server otherwise you will need to work with the IT group organization in order to save and perform some of the steps explained in this paper.
As an example for the blog I have created an .xsd file named: XSD_Beelogix_file
Here are the main components you can find in the .xsd file.
The .xsd file is nothing more than a structure to the .xml document. This is summarizing all the parts the system will find while reading the .xml file
As I said before you can find generators on Internet to develop this .xsd file but in case you need to amend something be careful on typo as the system will refer to the .xsd file to create the XML Schema and then make it the template to read the .xml. An .xsd not in sync with the .xml will end up with an inconsistency when the mass read will occur.
Opening and integrating the .xml file
The screenshot below is representing the .xml file.
You can see the difference with the previous file .xsd when the .xml is openable straight on the Web. This file represents some data a customer could be asking to be included in IBP with some Customer #, SKU#, KG Ordered etc…
For the purpose of the blog I have built an .xml file with only 3 sets of data with some random data in it. The main goal of it is to show you what is part of the .xml file and how this will be showing in HCI-DS once read and converted into a format we can transfer over to IBP.
Building Dataflow and Output data
Once both files .xsd and .xml are stored in the HCI server we can go into HCI to set up the XML schema and build dataflow.
Building an XML schema is essentially giving HCI-DS the Root path to the .xml file and read the .xsd data structure.
Going onto your HCI-DS datastores, you can create XML schema. We will not cover Datastore set up in this article but this is really simple as it requires few clicks onto the HCI datastore already created onto the client machine.
Now I have the files ready on the HCI server and HANA Cloud Integration has the path to read my files format I can build dataflow.
I have created a Task within a Project and then created the Dataflow named XML_Beelogix_file.
The source in the dataflow is typically the XML schema we have previously created on the Datastore. As we are integrating Web services data coded as .xml markup language therefore the dataflow requires an XML transform query.
This XML transform query is represented in the screenshot below as the little white square with XML and in this example I called it Extract_xml_data
As I already mentioned, I did not design this dataflow to go onto a Key figure within IBP but more a simple flat file to make the process much easier for everyone. Targeting a flat file my dataflow requires a Target query in between my XML transform query and the flat file target. This Target query is actually a mapping of the Output columns you have gathered from the .xml file.
Something I would like to highlight in this blog and this is really important otherwise your dataflow won’t work properly and you will need to fix it before running it again.
Using XML markup language, you need to parameter some king of Iteration rule. While setting up your dataflow and when you go to your XML transform query then you will see a tab like this one:
Your XML transform query needs the Iteration to know exactly when to move onto the next dataset. Let’s say the Iteration are like a node within your .xml file
You can quickly set up these Iterations by clicking on the PROPOSE RULE the icon circled below on the left. The system will read your .xsd file and set up the Iteration maintained in it. Don’t forget this is a requirement to make your dataflow working properly.
When you dataflow is fully configured and all columns required in the target query are mapped then you can run your dataflow and see how HCI-DS goes and read the data stored in the .xml file onto the HCI server.
Here is the Output of the example I made for the blog. Comparing the data maintained in the .xml file earlier explained in this paper and the Output we can identify the 3 customers, SKUs etc…
The dataflow went to read the .xsd file to find out what was the structure of the file to read and then went over the .xml file to extract the data.
When implementing IBP the customers are sometimes having data stored on various websites and sometimes they have partnership with some of their clients and/or suppliers to get access to data maintained as a web language. Having the possibility of reading data saved on websites by integrating .xml files thru HCI-DS is a powerful tool and this gives both IBP a lot of prospect to answer customer’s needs and some kind of harmonization for the end-user working on IBP.