During one of my MDM implementation; the customer main pain point was current state of master data in multiple systems. We had to reconcile duplicates and perform data cleaning. As soon as I joined the project as MDM architect, I immediately made a suggestion to client to opt for BOBJ DS to solve their pain point of cleansing the Customer master data. Unfortunately for the client, the project was signed with no scope for any more licenses for existing/newly acquired products during the deal. Hence it was MDM team responsibility to cleanse the master data with the help of client.
It was simply one of the very difficult, expensive and time consuming exercises. Initially we had training more than 40 users for reconciling duplicates. However they were not able to provide their complete dedicated time and hence the deadline for removing duplicates seems to be slipping and requires immediate attention from management to keep it bring it back on track. These 40 odd users provided the training to their subordinates to help them out and hence the cleaning team size grew to more than 100 odd users. The total time provided to the users for removing all duplicates was 2 months and by simple calculation it became 100*40 = 400 man days. However after this mammoth, time and effort consuming exercise; the main concern was on the quality on the tasks performed by these users. The reason behind this was that even after touching all the users across geography, region; there is always scope of small missing links. Moreover for the client, this adds tremendous pressure on the business users who are mostly from sales team and have to spend considerable time on this exercise.
Interestingly after all this effort, there was still lot of scope left for data cleansing and even de-duplication. In the end it came back to client IT team to complete this task and that too was achieved by dividing data into multiple excel files since excel is unstable for large amount of data and not recommended. The best part was that client team was extremely dedicated to this and also had very serious understanding of this exercise. Somehow we managed to complete this data standardization, cleanup exercise and loaded the correct master data in new MDM system. The new implemented MDM system was helpful in preventing to repeat such an exercise in future having validations governance in place.
This experience helped me a lot in my next assignment where I suggested BOBJ DS along with MDM right from the beginning and client agreed for it. Both the consulting and client team reaped the benefit of BOBJ DS to standardize, cleanse the current master data from multiple systems which paved way to successful MDM implementation.
For more info :