Cleanising BW Data Using Regular Expressions

Sometimes data in Source System is not checked for quality. For example, input data is not checked for non printable characters e.g. tabulation, carriage return, linne feed etc. If user copy and paste data into input fields from email or web page then non printable characters can be entered into the system causing BW data loading issues (not permitted characters). In case of master data quality issue must fixed immediately otherwise problem will become worse with every transaction where incorrect master data is used. In case incorrect of just information fields that are stored in DSO at document level, then data can be fixed in transfer rules.

What it takes is to correct data in transfer rule start routine using regular expression.

As you can see from the screenshot above I replace all character that are NOT BW allowed characters “%&()*+,-./:;<=>?_0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ_ AND NOT extra permitted characters ËÊÉÈÇÂÀÃÔÒÎÏÛÙ,. with space. Extra permitted characters are the one defined in trx. RSKC

Note: everything in [] defines a character set and ^ negates it e.g. actually what is not in the list gets replaced.

Prior to executing REPLACE statement SOURCE_PACKAGE field contains hex 09 (tabulation) character.