Overview on Customer Data Cleansing projects- involving multiple SAP Tools
Hi
In this blog I am going to share my experience while exploring and working on one of the Customer’s data cleansing and reporting project. This Project involves multiple SAP Technologies like :SAP BODS ,SAP Information Steward,SAP Business Objects(BO) , SQL Server(database,client’s requirement,Non SAP),
Such kind of projects have lots of data to be cleansed , put in other words, required to be in the desired format as the client needs.The Client will be sharing their requirements beforehand. Here we used SAP Information Steward to create various rules(involving Regex functions) , which are used in SAP BODS to cleanse and transform the data. Using SAP BODS we create various types of transforms which helps in cleansing the data process very smoother and easier. The data might be cleansed in various ways:Automated or Manual.However majority of the tasks are being carried by the automated cleansing process through the tools(Information steward and BODS).
SQL Server is the back-end database, being used at the client side to maintain their data.Hence the created tables in SAP BODS and SAP Information Steward are being stored on it.
The volume of data to be cleansed may vary across projects. In our case, we divided the data load and cleanse processes into 4 batch job load process (comprising 250 K data in each batch).
The project involved reporting as well. The customers wanted to display the amount of records cleansed,the changes which were affected to specific fields which were requested, count of records before cleanse and after cleanse and many more things, for which a universe fetching data from SQL server( which holds the cleansed data during each batch load) was created, and then using the Webi (Web-Intelligence) the reports were created.
Very soon ,will be adding few images describing the process flow.
Till Then Happy Learning 😀
Thank You
Hi Moumita
We are also doing similar data cleansing project using BODS and BOIS and source as SAP. We are lagging in reporting. Using BODS and BOIS we could cleaning but in reporting we dont know how to define the KPI's. So kindly share some process how it was followed in your project and post the screen shots of webI.
Thanks and Regards
Vijay Mukuthan
Hi Vijay
Your project sounds cool. But in our project we are using Business objects, as reporting tool(Webi and universe).Reporting side is much simpler. Regarding KPI's I can share overall picture - we have created the universe using UDT, where we have replicated the cleansed and uncleansed data coming from the backend and using those specific objects we are pulling the data on to the web-intelligence tool for creating reports.
For an example , if I have a table having details of customers adrs being affected before cleansing and after cleansing, we need to display them in report. Accordingly we would have multiple requirements in the project, so our seniors have created the universe , having the objects pertaining to cleansed records(cleansed records fields) and the objects which were uncleansed(before cleansing(mapping from DB to universe).There were certain fields which involving complex formulaes(related to Auditing), we have created objects in the universe itself(for those fields). And using all these we are using them on Webi.
For further ideas,i will reach to my senior developers and get solutions.
Thank You
Moumita