Most customers know the problem that invalid characters can be found in data records. Data load or DSO activation for char type InfoObjects can fail due to permitted or non-printable characters in BW.
There exist several solutions for this problem. Some customer uses function module calls in ABAP transformations for example RSKC_CHAVL_OF_IOBJ_CHECK or RSKC_ALLOWED_CHAR_GET. Other implement field routines with ABAP like
REPLACE ALL OCCURRENCES OF REGEX '[^[:print:]]' IN RESULT WITH 'replace_char' IGNORING CASE.
IF RESULT(1) = '!' OR return = '#'. RESULT(1) = 'replace_char'. ENDIF.
Now BW/4HANA has the option to push down transformation logic to HANA database, to reduce runtime of data loads. For custom transformations to be pushed down to HANA database, start, end and expert routine type need to be created as AMDP script (ABAP Managed Database Procedure script) instead of an ABAP based routine.
For more details on creation of AMDP Script based transformation you can refer to:
For the example we created a sample file with invalid characters ‘!’ and ‘#’ in record 2, 4, 5 and 6, and a non-printable character Tapstop in record 7.
We load the data in a field based write optimized ADSO and want it to be loaded and activated in a standard ADSO with InfoObjects and changelog.
To remove invalid and non-printable characters with an AMDP Script in a field routine, you can follow these steps.
Step 1: Select rule type routine for the transformation rule, see (1). A popup dialog asks for processing type, see (2). Choose AMDP script to create an AMDP script based field routine.
Step 2: an AMDP class will be generated with default method – PROCEDURE and default interface – IF_AMDP_MARKER_HDB. The BW Transformation framework opens the Eclipse based ABAP class editor.
Step 3: Enter your code for the field routine in the body of the method PROCEDURE.
You can use the following template. The code replaces the regular expression ([^[:print:]]|[#|!]) in source field “CHARACTERS” with replacement char ” and transfer the result in the target InfoObject “/BIC/ZSPECHAR”.
If you want to remove invalid characters at the first position only, you can use this template.
If necessary this can be extended with upper case conversion or similar.
After removing the characters in transformation our example data will be loaded without errors and the data activation in ADSO works well.
Here you can find some more information about SQL String function syntax and regular expression: