XSLT approach to enable FCC for deep structures (Part 1 – Deep XML to Flat File)
There have been multiple threads cropping up in the forum lately regarding FCC conversion for deep/nested structures. Unfortunately, the File Content Conversion (FCC) functionality (whether directly in the File/FTP adapter or through MessageTransformBean module) has not changed significantly for nearly 10 years since the days of XI 3.0 We are still stuck with standard FCC functionality that is only able to handle flat structures – all record-types/segment can only be at the same single level. However, in reality not all flat file structures conform to this “expected” pattern. We would have expected SAP to enhance the capabilities of the standard FCC functionality to cater for these commonly used deep structures, but that does not seem to be the case.
Therefore, all these years, there have been a myriad of custom solutions/workarounds to handle this. Some custom approaches are listed below:
- Graphical mapping – multi-step graphical conversion using intermediate structure. Example: XI/PI: Convert Flat File to Deeply Nested XML Structures Using Only Graphical Mapping
- Java mapping – full blown Java logic. Examples: Simplest way to Convert Flat File to Deeply Nested XML Structures Using Java Mapping and File Conversion using ‘Nodeception’
- Adapter module – handle the FCC conversion fully in a custom adapter module
- Seeburger BIC mapping – custom BIC mapping to handle flat file conversion (I think SAP’s B2B Add-on has a similar functionality for custom flat file conversion, but I have not had my hands on that Add-on yet to confirm)
In general, my preferred choice for custom development goes in the following order – Graphical mapping, Java mapping, XLST, adapter module. However there are certain areas where XSLT excels in, achieving robust transformation with minimal coding. Some of these are sorting XML and performing partial identity (1-1) transformation. The combination of the copy and apply-templates command allows for identity transformation without the need to individually specify field names.
In this two-part series, I will share the approach based on XSLT that was used in one of my previous projects to handle creation and parsing of deeply structured flat files. I have since tweaked the XSLT codes so that it is able to be used in any scenario with minimal changes to the code. Here are the scenarios that will be covered in each part of this series.
- Part 1 – flatten a deep structure XML prior to receiver FCC conversion
- Part 2 – deepen a flat XML after sender FCC conversion
Source Code and Explanation
The full source code, example input and output files are available in the following GitHub repository
The logic comprises of three XSLT template match sections.
1. Flatten segments with subsegments
This template uses *[*] to match any nodes that have subsegments (subnodes with child fields.) Once such a segment is matched, the immediate child fields are copied over 1-1, while the subsegments are brought up to the same level (flattened.) The logic then continues recursively.
<!-- Match any segment with child segments/fields --> <xsl:template match="*[*]"> <!-- (1) - Copy the current segment, then select attributes and child elements that are fields --> <xsl:copy> <xsl:apply-templates select="@* | *[not(*)]"/> </xsl:copy> <!-- (2) - Further select child elements that are segments --> <xsl:apply-templates select="*[*]"/> </xsl:template>
The following sample input and output shows that the subsegments are brought to the same level in the hierarchy as the parent segment.
2. Identity transformation for root element
This template matches the root element of the XML document and copies it 1-1 to the output.
Note: To reuse this XSLT code, the Root element need to be changed according to the root element name of the source input XML.
<!-- Match root element, copy details and and select child attributes and nodes --> <xsl:template match="*[local-name()='MT_Deep']"> <xsl:copy> <xsl:apply-templates select="@* | node()"/> </xsl:copy> </xsl:template>
3. Identity transformation for all other attributes and nodes
This template is similar to the one above and does a 1-1 transformation for all other attributes/nodes.
<!-- Match all other attributes (@*) and other node types (elements, comments) --> <!-- Copy the current element, and select child attributes and nodes --> <xsl:template match="@* | node()"> <xsl:copy> <xsl:apply-templates select="@* | node()"/> </xsl:copy> </xsl:template>
Note: In theory templates 2 and 3 can be combined into one by specifying an additional /* match for the root element. However, such logic does not provide the intended result when being executed on PI’s mapping runtime, although it works on Online XSLT Test Tool and the XML-Tools plugin of Notepad++. It will also work if usage of the SAP XML Toolkit is specified in the Operation Mapping, however this goes against SAP’s recommendation as documented in the following SAP Library link – XSLT Mapping – SAP Library.
I am unsure of the reason for the behavioral difference when it is executed on the XSLT parser of the JDK. Otherwise, the XSLT code would have been a generic logic that could have been reused in any transformation without any modification.
Here is a sample scenario where the requirement is to achieve the creation of a flat file with deep/nested structures.
The output file will be a shipment file which contains all the details for the shipment. Each shipment file can have multiple deliveries, each delivery in turn can have multiple orders, and each order multiple items. Each segment needs to occur under the parent segment, therefore indicating a deep/nested structure.
|Segment Name||Parent Segment||Type Indicator||Occurrence|
|Delivery||–||D||1 – unbounded|
|Order||Delivery||O||1 – unbounded|
|OrderText||Order||T||0 – 1|
|Item||Order||I||1 – unbounded|
This article will only focus on the sections that are required to enable this XSLT approach, therefore common steps for design and configuration will not be covered.
Target structure definition
The data type for the target structure will be defined as a deep XML schema. No flat DT definition that is required for FCC needs to be defined. Below is the sample deep XML schema representation of the output format.
Create a normal graphical message mapping to map the source structure to the deep target structure above.
Import XSLT code
Zip the XSL file and import it to ESR as an Imported Archive.
Typically the XSLT mapping will be used as the second step of the Operation Mapping. The first step will be the message mapping to map the source to target. Then the XSLT will flatten the final output.
|1||Message Mapping||<Message Mapping object created above>|
Below is a sample execution of the second step of the OM to flatten the deep structure in preparation for FCC. The result shows that the XSLT have flattened all the segments to the same level.
Note: The output of the XSLT mapping step is not a valid XML document when validated against the original data type definition, but it does not matter as the main purpose of the output is as an input to the FCC at the receiver channel.
The normal configuration steps are performed. At the receiver channel, specify the FCC parameters. The following screenshot shows the FCC parameters using MessageTransformBean on an SFTP channel.
After execution of the interface, we can see from the after FCC log version that the payload has been converted into a flat file format with deep structure.
Similarly, when the CSV file is viewed in Excel, we can see that all the subsegments are created in the correct order underneath their corresponding parent segments.
With a relatively simple XSLT logic, we can still achieve creation of flat file with deep structure whilst still using the standard FCC functionality. The benefits of this XSLT approach are:
- Easily reusable for other interfaces – just need to change the name of the root element
- The XSLT logic works irrespective of the number of levels of the deep structure
- No intermediate structure needs to be defined for flat XML representation of the file – this is required when using the Graphical mapping approach
- No changes required if fields of a segment are changed, i.e new fields, deleted fields, modified field name
In the continuation of this series, part 2 covers handling of deep structure FCC at the sender side.
I have an IDoc to File scenario. In receiver File adapter, I use FCC. It has five or six levels of hierarchies. Can this be used here? When I use this , do I still have to use FCC
Yes, this can be used for multiple levels of hierarchy, and you still have to use the standard FCC on the receiver channel.
This blog was very nice and useful.
Can any one explain the execution sequence of this XSLT Mapping Program.
and could you please explain the concept of Message Transform Bean and other Bean types also if there.
Thanks in Advance......
Any Suggestions on this...
Its a really good blog.Im new to PI and got to learn this concept well.
Im developing a Scenario in which the source is nested an dit repeats as well.
For example Data,Item,Details,Trailer.
Details is subnode of Item and they Keep repeating.
I achieved flattening of file but getting an issue in FCC.
Could you please help me with this?