Just in case you never heard this before the other 500 times I mentioned it, I used to be an ABAP programmer. If I look back at the programs I wrote - the maximum number of lines of code I wrote has been on conversion programs and interfaces.
The story goes like this - if you have seen the movie, you can sing along.
A bunch of functional consultants sit with the client business team and figures out the best way to configure the system. Then to make sure things work the way it is supposed to work - they will create materials and customers called TEST1, TEST2 etc. Then they create a bunch of sales orders etc that confirm that all is well. Every one is all smiles, and high fives are aplenty.
Enter the ABAP gang. They get a SHDB recording of the perfect transaction flow, and are pointed to the TEST sales order with TEST1 Customer and Test2 Material as example. Hours/days later, ABAPer comes out with his program that reads data from a file and commits a transaction exactly like the TEST sales order. More high fives, and every one in SAP team goes into hibernation while another team extracts data into a file that the ABAP program can read.
First time the program reads the output from external system, you get an error in salesorder that says order type GR is not defined. Analysis tells you the order type is not GR, but the program is reading the header text and trying to put the first 2 characters are put in order type field because legacy team ordered fields in a different order. Ah - no big deal, why bother to change the file - we will just tweak the ABAP program. Few minutes later, we find that all is well, and decide we are ready for next file with more records. All goes well, till we find that not all records in the new file has customer number filled - some only have customer name. Big deal - we will code a reverse look up. You get the drift - the smart ABAPer saves the world, 10 lines of code at a time.
The amount of time and money we spend on this is extremely high - and a lot of project failures can be traced back to this issue.
ABAPers are smart - and many have builf utility programs that help ease the pain of maintenance. However, ABAP's world is limited to make things right within SAP. The missing part is the source system. If we would profile the source system data and map it to target structures, almost all the problems that we face during development could have been found earlier.
It is not as if we don't have tools for this job - SAP, IBM and many other vendors have excellent products that do this. However, when project gets budgeted, data is one of the first aspects to get de-prioritized. The solution to all data problems is to throw more programmers at it. Very few projects make use of the information management tools. Granted this needs some capital investment in licensing and hardware - but would you rather not do this, and take the risk that your project will fail due to data issues?
Tools are just one part of the issue - we should also consider the people and processes around data. Most SAP projects will skimp on having roles in the project team to take care of data. And "Governance" is something you think of as a strange thing that lives only in powerpoint presentations. It is also not common place in most companies to have "owners" for data.
At a minimum, even if you want to solve all your data problems in ABAP - please try to move the data profiling part to an earlier phase of your project, like in blueprinting. Do not wait for development or testing phase to find you have a problem on your hands.
Companies spend a lot of time and money on fancy BI solutions, and a lot of those initiatives do not succeed in delivering business value. And almost always, it can be traced back to bad data. At a fraction of the cost, this data can be fixed early in its lineage, and these companies can reap great benefits. But then, we are used to being data deaf - the question is, are we going to do something about it?
