Skip to Content
Technical Articles
Author's profile photo Eng Swee Yeoh

DeepFCCBean – The better FCC at meeting your deep (structure) needs! (Part 2 – Flat File to Deep XML)

Update 2 Oct 2018: New parameter endSeparator to define end of line separator for input file.

Update 4 Sep 2018: Now available in CPI as well.

Update 22 Jan 2016: New parameters missingLastFields and additionalLastFields to handle structure deviations between configured parameters and actual file content.

Update 26 May 2015: New parameter genericRecordType to cater for records with no key field value as shown in scenario 3. Previous scenario 3 renamed to Scenario 4.

Update 25 Mar 2015: DeepFCCBean has been refactored to be part of FormatConversionBean. Parameter conversionType replaced with converterClass. Note that some of the screenshots are not updated to reflect the new parameter converterClass.

Update 17 Mar 2015: Parameter indentXML replaced by indentFactor.

Introduction

This is the second part of the two-part series on achieving file content conversion (FCC) for deep structure with a reusable custom adapter module. The first part covered the conversion of deep XML to flat file at a receiver channel.

This second part will focus on conversion of flat file to deep XML at a sender channel.

Source Code

Refer to following blog on location of source code and/or EAR deployment file.

FormatConversionBean – One Bean to rule them all!

Prerequisite

  • Each substructure type must contain a field containing the key value to enable the parser to identify lines of this substructure type
  • Each substructure type must belong to one and only one parent substructure type. The parent can be either the root element or other substructure types defined

Usage of Module in Communication Channel

Module Processing Sequence

Number Module Name Type Module Key

<Depending on position of module in chain>

1) Asynchronous scenario,

Normally before the last module in channels

2) Synchronous scenario,

Before last module to convert request payload

After last module to convert response payload

Custom_AF_Modules/FormatConversionBean Local Enterprise Bean <Any Arbitrary Value>

Module Parameter Reference

Below are the parameters for configuration of the module for flat file to deep XML conversion (converterClass = ‘com.equalize.xpi.af.modules.deepfcc.DeepPlain2XMLConverter’). Certain parameters will automatically inherit the default values if it is not configured.

The parameters are separated into parameters that are common for this conversion type, and also parameters that are specific for field separator based or fixed length based conversion.

1) Common Parameters

Available only in PI

Parameter Name Allowed values Default value Remarks
converterClass

PI – com.equalize.xpi.af.modules.deepfcc.DeepPlain2XMLConverter

CPI – com.equalize.converter.core.DeepPlain2XMLConverter

Required field. Determines conversion class
encoding UTF-8 Defines encoding of output plain text file
documentName Required field. Document name of root element of XML output
documentNamespace Required field. Namespace of root element of XML output
recordsetStructure Required field. List of substructure names separated by comma
genericRecordType Name of substructure used as generic type. If configured, any lines that cannot be determined via key field determination will be treated as this generic type.
keyFieldName Required field. Name of the key field that must occur in all substructures
indentFactor Integer values beginning from 1 0 Determines the number of indentation spaces for each level in the XML output
rowOffset Integer values beginning from 1 0 Starting row to begin extracting content from (i.e. 0 = start from first row, 1 = start from second row)
trimContents Y, N Y Determines if all the leading and subsequent blanks for a field will be trimmed or not
endSeparator Defines default end separator for entire input file
<StructureName>.fieldNames Required field. Names of fields in this substructure
<StructureName>.keyFieldValue Required field. Key field value to identify this substructure
<StructureName>.parent Required field. Specifies the parent of this substructure. Value should correspond to a substructure in recordsetStructure or the value ‘Root’ to indicate that it belongs directly under the root element
<StructureName>.missingLastFields ignore,add,error ignore Behavior of conversion when there are less fields in content compared to configured fields in <StructureName>.fieldNames

  • ignore = missing fields not included in output
  • add = missing fields included with blank value in output
  • error = conversion terminated with error due to incomplete input
<StructureName>.additionalLastFields ignore,error ignore Behavior of conversion when there are more fields in content compared to configured fields in <StructureName>.fieldNames

  • ignore = additional fields not included in output
  • error = conversion terminated with error due to extraneous input
debug

Displays audit log messages of substructure and parent-child determination of each line.

WARNING: Use this only for debugging in non-productive systems

Available only in PI

messageLog pre, post Saves a log version of the message that is viewable in Message Monitor

  • pre = saves payload before conversion
  • post = saves payload after conversion

Available only in PI

logLocation Name of log version when messageLog is populated. Location defaulted to value in messageLog if logLocation not populated. Available only in PI

2) Field Separator Conversion Parameters

Parameter Name Allowed values Default value Remarks
defaultFieldSeparator Defines a default field separator for all substructures that do not have <StructureName>.fieldSeparator explicitly configured
<StructureName>.fieldSeparator Either <StructureName>.fieldFixedLengths or <StructureName>.fieldSeparator must be populated
<StructureName>.enclosureSignBegin Specify a string that acts as a text delimiter. Field separators within such texts are ignored
<StructureName>.enclosureSignEnd If the text delimiters for the beginning and end of the text are different, specify the text delimiter for the end of the text here. If it is not populated, it uses value specified in <StructureName>.enclosureSignBegin
<StructureName>.enclosureSignBeginEscape Specify a string that replaces the text delimiter if it occurs within a text that it delimits. When the text is transferred the string is replaced by the value specified in <StructureName>.enclosureSignBegin
<StructureName>.enclosureSignEndEscape Specify a string that replaces the text delimiter for the end of the text if it occurs within a text that it delimits. When the text is transferred the string is replaced by the value specified in <StructureName>.enclosureSignEnd. If it is not populated, it uses value soecified in <StructureName>.enclosureSignBeginEscape
<StructureName>.enclosureConversion Y, N Y
  • Y = the text delimiter is to be removed and/or the escape character is to be replaced when the files are transferred
  • N = the character is to be transferred unchanged

3) Fixed Length Conversion Parameters

Parameter Name Allowed values Default value Remarks
<StructureName>.fieldFixedLengths Integer separated by commas Either <StructureName>.fieldFixedLengths or <StructureName>.fieldSeparator must be populated. The number of lengths must match the number of fields listed in <StructureName>.fieldNames

Similar to standard FCC functionality, fieldSeparator parameter supports non-printable ASCII characters as follows:

  • characters encoded as hexadecimal values in the form of ‘0xHH’ (including quotation marks)

Example Scenarios

Here are some example scenarios of the behavior of the conversion based on different configuration options.

Scenario 1

3 substructures with 3 level deep structure

Field separator based conversion

Field separator – Order and Item using default comma. Delivery using tab (specified in hexadecimal)

Order uses ” as enclosure sign, and “” as enclosure escape

Item uses ” as enclosure sign but enclosure sign is not removed when transferred

Row offset provided to skip first line

Output XML indented

Module parameters

Parameter Name Parameter Value
converterClass com.equalize.xpi.af.modules.deepfcc.DeepPlain2XMLConverter
documentName MT_DeepPlain2XML
documentNamespace urn:equalize:com
recordsetStructure Delivery,Order,Item
keyFieldName Type
Delivery.fieldNames Type,DeliveryNo
Order.fieldNames Type,DeliveryNo,OrderNo
Item.fieldNames Type,OrderNo,ItemNo,Quantity
Delivery.keyFieldValue D
Order.keyFieldValue O
Item.keyFieldValue I
Delivery.parent Root
Order.parent Delivery
Item.parent Order
Delivery.fieldSeparator ‘0x09’
defaultFieldSeparator ,
Order.enclosureSignBegin
Order.enclosureSignBeginEscape “”
Item.enclosureConversion N
Item.enclosureSignBegin
rowOffset 1
indentFactor 4

Result

Input /wp-content/uploads/2015/03/in1_662853.png
Output /wp-content/uploads/2015/03/out1_662854.png

Scenario 2

4 substructures with 3 level deep structure

Fixed length based conversion

Leading and trailing whitespaces are not trimmed

Missing fields are automatically populated with blank

Output XML indented

Module parameters

Parameter Name Parameter Value
converterClass com.equalize.xpi.af.modules.deepfcc.DeepPlain2XMLConverter
documentName MT_DeepPlain2XML
documentNamespace urn:equalize:com
recordsetStructure Header,Delivery,Order,Item
keyFieldName Type
Header.fieldNames Type,FileName
Delivery.fieldNames Type,DeliveryNo
Order.fieldNames Type,DeliveryNo,OrderNo
Item.fieldNames Type,OrderNo,ItemNo,Quantity
Header.keyFieldValue H
Delivery.keyFieldValue D
Order.keyFieldValue O
Item.keyFieldValue I
Header.parent Root
Delivery.parent Root
Order.parent Delivery
Item.parent Order
Header.fieldFixedLengths 5,20
Delivery.fieldFixedLengths 5,10
Order.fieldFixedLengths 5,10,10
Item.fieldFixedLengths 5,10,10,10
trimContents N
indentFactor 4

Result

Input /wp-content/uploads/2015/03/in2_662870.png
Output /wp-content/uploads/2015/03/out2_662872.png

Scenario 3

Conversion where one and only one substructure type does not have key field to identify the type of record

Substructure Detail is configured as the generic record type

Sample test data for this scenario is from the following thread File Content Conversion – No Key Field in Detail Structure

Module parameters

Parameter Name Parameter Value
converterClass com.equalize.xpi.af.modules.deepfcc.DeepPlain2XMLConverter
documentName MT_DeepPlain2XML
documentNamespace urn:equalize:com
recordsetStructure Header,Detail,Footer
genericRecordType Detail
keyFieldName Type
Header.fieldNames Type,HeaderField
Detail.fieldNames Field1,Field2,Field3,Field4,Field5
Footer.fieldNames Type,TrailerField
Header.keyFieldValue HDR
Footer.keyFieldValue TRL
Header.parent Root
Detail.parent Root
Footer.parent Root
Header.fieldFixedLengths 3,11
Detail.fieldSeparator ,
Footer.fieldFixedLengths 3,9
indentFactor 2

Result

Input

HDRXXXYYY12345

MM8100422373,M631530008  ,1530008,031615,17

MN8100424411,M631530008  ,1530008,031615,22

XY8100422373,M631530008  ,1530008,031615,17

TRLXXZZZ4321

Output /wp-content/uploads/2015/03/scenario3_710809.png

Scenario 4

Finally for the fourth scenario, we will use the same input file that was used in the XSLT approach. Actual configuration and testing screenshots are shown.

Default field separator for all substructures is used.

A log version of the message payload before conversion is saved.

Module configuration on an SFTP sender channel.

/wp-content/uploads/2015/03/module_662093.png

The ‘plain’ log version shows the flat file before conversion.

/wp-content/uploads/2015/03/pre_662068.png

The converted payload is sent to a receiver proxy without further mapping. As shown below, the target payload has been converted to a deeply nested XML structure.

/wp-content/uploads/2015/03/proxy_662067.png

The audit log shows the trace of steps being executed by the module.

/wp-content/uploads/2015/03/log_662097.png

Conclusion

With the completion of this second part for DeepFCCBean, we now have a generic, reusable and highly configurable custom adapter module that is able to handle conversion of flat file to deep XML and vice versa!

Assigned Tags

      29 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Former Member
      Former Member

      Good one Eng.

      Author's profile photo Former Member
      Former Member

      Hi Eng Swee,

      Thanks for sharing the very useful information on Adapter Framework Modules

      When i trying to do the scenario 3 then it is throwing the error in channel monitor

      Error while sending message: Cannot send message to module processor com.sap.aii.af.lib.mp.module.ModuleException: Error during processing local bean: localejbs/Custom_AF_Modules/DeepFCCBean

      Error: com.sap.engine.services.jndi.persistent.exceptions720.NameNotFoundException: Path to object does not exist. First missing component is [Custom_AF_Modules], the whole lookup name is [localejbs/Custom_AF_Modules/DeepFCCBean].

      So my question is,

      Can we use this Custom Adapter Framework Modules without implementing them in the local system? If we need to implement them, then where we have to do this?


      And how can I overcome the  above error? Thanks in advance!!



      Regards

      Pavan

      Author's profile photo Praveen Gandepalli
      Praveen Gandepalli

      Hi Pavan,

      You need to use Custom_AF_Modules/FormatConversionBean, there are some changes to the modules please look at below blog for more details.

      FormatConversionBean - One Bean to rule them all!

      Regards,

      Praveen.

      Author's profile photo Eng Swee Yeoh
      Eng Swee Yeoh
      Blog Post Author

      Hi Praveen,

      Thank you for helping to point that out. I appreciate it. Looks like I need to update the info on my blogs so that the new name that should be used in the Module configuration is clearer.

      Pavan,

      Please use Custom_AF_Modules/FormatConversionBean as mentioned by Praveen.

      Rgds

      Eng Swee

      Author's profile photo Former Member
      Former Member

      Hi Eng Swee,


      is it possible to add a default endSeparator or <Structure>.endSeparator for file -> XML conversion?



      Best Regards,

      Thorsten

      Author's profile photo Eng Swee Yeoh
      Eng Swee Yeoh
      Blog Post Author

      Hi Thorsten

      Adding a default endSeparator would be possible. I'd probably can find some time to get it done and out by sometime next week.

      <Structure>.endSeparator would be a bit more tricky as it involves revamping the simple file parser into something more elaborate. I'll add it to the list of enhancements for this module in the future.

      I'm always up for others to collaborate on this - all the source codes are already publicly available. Let me know if you are interested.

      Rgds

      Eng Swee

      Author's profile photo Former Member
      Former Member

      Hi Eng Swe,


      a default endSeparator  would be fine and should be sufficient in most cases.

      I would be happy to support you with the module 🙂

      Best Regards,
      Thorsten

      Author's profile photo Eng Swee Yeoh
      Eng Swee Yeoh
      Blog Post Author

      Hi Thorsten

      I've never had to work with flat files with end separator other than the common CRLF/LF. Can you let me know some sample files that is different? This will help me design the changes accordingly.

      Rgds

      Eng Swee

      Author's profile photo Former Member
      Former Member

      Hi Eng Swee,


      the end seperator is working. The error is because of emty segments at the end of the line - i think.


      /wp-content/uploads/2015/12/error_843646.jpg

      When i add a value before the LF it works.

      The exception is:


      Caused by: javax.ejb.EJBException: ASJ.ejb.(Failed in component: sap.com/com.equalize.xpi.af.modules.app) Exception raised from invocation of public com.sap.aii.af.lib.mp.module.ModuleData com.equalize.xpi.af.modules.util.AbstractModule.process(com.sap.aii.af.lib.mp.module.ModuleContext,com.sap.aii.af.lib.mp.module.ModuleData) throws com.sap.aii.af.lib.mp.module.ModuleException method on bean instance com.equalize.xpi.af.modules.FormatConversionBean@13d0a9ea for bean sap.com/com.equalize.xpi.af.modules.app*xml|com.equalize.xpi.af.modules.ejb.jar*xml|FormatConversionBean in application sap.com/com.equalize.xpi.af.modules.app.; nested exception is: java.lang.NullPointerException: while trying to invoke the method java.lang.String.trim() of a null object loaded from local variable 'fieldValue'

      at com.sap.engine.services.ejb3.runtime.impl.RequestInvocationContext.proceedFinal(RequestInvocationContext.java:88)

      at com.sap.engine.services.ejb3.runtime.impl.AbstractInvocationContext.proceed(AbstractInvocationContext.java:166)

      at com.sap.engine.services.ejb3.runtime.impl.Interceptors_StatesTransition.invoke(Interceptors_StatesTransition.java:19)

      at com.sap.engine.services.ejb3.runtime.impl.AbstractInvocationContext.proceed(AbstractInvocationContext.java:179)

      at com.sap.engine.services.ejb3.runtime.impl.Interceptors_Resource.invoke(Interceptors_Resource.java:50)

      at com.sap.engine.services.ejb3.runtime.impl.AbstractInvocationContext.proceed(AbstractInvocationContext.java:179)

      at com.sap.engine.services.ejb3.runtime.impl.Interceptors_Transaction.doWorkWithAttribute(Interceptors_Transaction.java:37)

      ... 49 more

      Caused by: java.lang.NullPointerException: while trying to invoke the method java.lang.String.trim() of a null object loaded from local variable 'fieldValue'

      at com.equalize.xpi.af.modules.util.RecordTypeParameters.createNewField(RecordTypeParameters.java:201)

      at com.equalize.xpi.af.modules.util.RecordTypeParameters.extractLineContents(RecordTypeParameters.java:185)

      at com.equalize.xpi.af.modules.deepfcc.DeepPlain2XMLConverter.extractLineToFieldList(DeepPlain2XMLConverter.java:142)

      at com.equalize.xpi.af.modules.deepfcc.DeepPlain2XMLConverter.generateNestedContents(DeepPlain2XMLConverter.java:114)

      at com.equalize.xpi.af.modules.deepfcc.DeepPlain2XMLConverter.parseInput(DeepPlain2XMLConverter.java:68)

      at com.equalize.xpi.af.modules.FormatConversionBean.processModule(FormatConversionBean.java:14)

      at com.equalize.xpi.af.modules.util.AbstractModule.process(AbstractModule.java:49)

      at sun.reflect.GeneratedMethodAccessor2122.invoke(Unknown Source)

      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

      at java.lang.reflect.Method.invoke(Method.java:597)

      at com.sap.engine.services.ejb3.runtime.impl.RequestInvocationContext.proceedFinal(RequestInvocationContext.java:47)

      ... 55 more



      Author's profile photo Eng Swee Yeoh
      Eng Swee Yeoh
      Blog Post Author

      Hi Thorsten

      There might be a bug, not too sure yet.

      Can you do me a favor and open a new thread on this and provide the following:-

      - Screenshot of the module parameters

      - Sample file

      Rgds

      Eng Swee

      Author's profile photo Former Member
      Former Member

      Hi Eng Swe,

      we will find it with:

      Issue with DeepFCCBean

      Best Regards,

      Thorsten

      Author's profile photo Eng Swee Yeoh
      Eng Swee Yeoh
      Blog Post Author

      Hi Otto

      Thanks for the challenge to pick up adapter module with script support but I think I will pass at this moment - I've got too much on my plate to find time to look into it.

      Anyway, IMHO, I think if you already have such a solution, it would be more appropriate to share that yourself in the spirit of the SCN community instead of getting someone else to come up with the solution from scratch and share it.

      Rgds

      Eng Swee

      Edit: Hmmm... not sure why the original comment issuing the challenge was deleted.

      Author's profile photo Former Member
      Former Member

      Hi Eng,

      would it be possible to add a parameter for recordsets per message like in SAP module?

      Best Regards,
      Thorsten

      Author's profile photo Eng Swee Yeoh
      Eng Swee Yeoh
      Blog Post Author

      Hi Thorsten

      Thanks for your request. I had a look into the functionality in SAP and it is only allowed for the built-in FCC functionality in the adapter. For MessageTransformBean module, message splitting it not allowed.

      /wp-content/uploads/2016/03/split_907108.png

      One alternative is to use a similar approach as mentioned in AttachmentSplitterBean - Split attachments into child messages

      However, I need to spend some time exploring this. At the moment, for simplicity sake, the current converter does not have a "Recordset" level, so introducing that together with splitting will require significant effort and I can't estimate how long it'll take at the moment.

      Regards

      Eng Swee

      Author's profile photo Former Member
      Former Member

      Dear Eng Swee,

      Many thanks for your work, here is my gift to you …

      Regarding the Flat2XML-Conversion I have invested some time to re-implement com.equalize.xpi.util.converter.ConversionPlainInput  to handle any endSeparator instead of being fixed bound to newline (with the newLineReader). There was a “TODO” (also in the code remarked) still existing which I would like to fill now…

      It uses a java.util.Scanner to split the ‘lines’. This was needed for our implementation as our provider sends Messages with Key=Value&Key=Value… ampersand instead of new line.

      Here is the code which replaces your class:


      package com.equalize.xpi.util.converter;

      import java.io.IOException;
      import java.io.InputStream;
      import java.util.ArrayList;

      public class ConversionPlainInput {
      private final ArrayList<String> lineContents;

      public ConversionPlainInput(InputStream inStream, String endSeparator) throws IOException {

      java.util.Scanner scanner = new java.util.Scanner(inStream, “UTF-8”).useDelimiter(endSeparator);
      String theString = null;
      ArrayList<String> contents = new ArrayList<String>();

      while (scanner.hasNext()) {
      theString = scanner.next();
      contents.add(theString);

      }

      scanner.close();
      this.lineContents = contents;
      }

      public ConversionPlainInput(String string, String encoding, String endSeparator) throws IOException   {
      this(Converter.toInputStream(string, encoding), endSeparator);
      }

      public ConversionPlainInput(String string, String endSeparator) throws IOException {
      this(Converter.toInputStream(string), endSeparator);
      }

      public ConversionPlainInput(byte[] inputBytes, String endSeparator) throws IOException {
      this(Converter.toInputStream(inputBytes), endSeparator);
      }
      public ArrayList<String> getLineContents() {
      return this.lineContents;
      }
      }


       

      Additionally one has to replace and add few lines in public class DeepPlain2XMLConverter

      Add declaration:

      private String endSeparator;

      and

      gathering the value:

      this.endSeparator = this.param.getParameter(“endSeparator”, “\n”, true);

       

      Change the call of ConversionPlainInput because a parameter was added to

      this.plainIn = new ConversionPlainInput(inStream);

      this.plainIn = new ConversionPlainInput(inStream, this.endSeparator);

       

      By default newLine is still working as the default \n is exactly that, so the Module will work exactly the same way it did before without changing any parameter (Compatibility).

      We checked and it passed our tests.

       

      Maybe you can add this to your code one day.

      Kind regards Jörg

      Author's profile photo Eng Swee Yeoh
      Eng Swee Yeoh
      Blog Post Author

      Hi Jörg

       

      Thank you very much for your comment as well as the "gift" 😉 Appreciate your effort to complete the "TODO" 🙂

       

      I would like to ask that you issue a Pull Request to my GitHub repository so that we can collaborate on incorporating your changes.

      https://github.com/engswee/equalize-xpi-modules/pulls

       

      Regards

      Eng Swee

      Author's profile photo Peter Wallner
      Peter Wallner

      Hi Eng Swee,

      Thank you - it works perfectly on SAP PO 7.5. I actually prefer it to SAP's standard module MessageTransformBean because your module also gives proper error messages such as "MP: exception caught with message No. of fields in fieldNames and fieldFixedLengths does not match for record type = Addenda"

      Kind regards, Peter

      Author's profile photo Eng Swee Yeoh
      Eng Swee Yeoh
      Blog Post Author

      Thanks Peter. Glad to hear that it is working well and you are happy with the error handling 😉

      Author's profile photo Adarsh Dhingra
      Adarsh Dhingra

      Hi Eng,

      Could you please help me. I am using this in CPI. I have created Content Modifier as shown below:

      My Input file:

      But I am getting error as:

      Error Details
      javax.script.ScriptException: java.lang.Exception: java.lang.reflect.MalformedParameterizedTypeException@ line 19 in CallConverter.gsh, cause: java.lang.reflect.MalformedParameterizedTypeException
      Author's profile photo Eng Swee Yeoh
      Eng Swee Yeoh
      Blog Post Author

      Hi Adarsh

       

      Seems to work fine on my tenant.

       

      Can you try a simpler scenario (less parameters and/or other converterClass) and see if you are still getting the same MalformedParameterizedTypeException error?

       

      If yes, I'd suggest you switch to using JAR file from release 1.2.0.

       

      Regards

      Eng Swee

      Author's profile photo Adarsh Dhingra
      Adarsh Dhingra

      Hi Eng Swee,

      yes the jar from release 1.2.0 worked for me. Could you advise if the difference between 1.2 & 1.3 is related to CPI tenant Version. Anyways thanks a lot for your help.

      Author's profile photo Eng Swee Yeoh
      Eng Swee Yeoh
      Blog Post Author

      Hi Adarsh

       

      I changed a class from Java to Groovy after the CPI tenant's Groovy version was updated to Groovy 2.4.12 few weeks back. Not too sure why it's working on my tenant but not yours.

       

      There are no functionality changes between 1.2.0 and 1.3.0, just misc cleanup, so you are fine working with 1.2.0.

       

      Regards

      Eng Swee

      Author's profile photo Charles Golfman
      Charles Golfman

      Hi Eng Swee,

      Hope you are doing well.

      We have a requirement where we have to process the file as follows:

      As you can see this is the typical structure of N documents containing each Header/Items (1..N) but the line type instead of being in the line itself it is in the line above. Could this file be processed with this converter?

      Many thanks for your time,

      Regards.

      C.

      Author's profile photo Eng Swee Yeoh
      Eng Swee Yeoh
      Blog Post Author

      Hi Charles

       

      With the converter as it is right now, I would think it won't be able to process this file. There's no way to link the line type identifier with its following line(s). It is quite an unusual format.

       

      Regards

      Eng Swee

      Author's profile photo Charles Golfman
      Charles Golfman

      Many thanks Eng Swee,

      I was expecting that but wanted to make sure.

      Agree with you, quite an unsual way to format a flat file :).

      Regards,

      C.

      Author's profile photo Nithyamani Veeramohan
      Nithyamani Veeramohan

      Hi Eng Swee,

       

      I tried to convert plain (tab delimited) to xml in CPI tenant with the following settings. separator as tab is not working. I have tried with multiple options like 0x09, '0x09', \t, regular expression. But nothing seems to work. Can you help me here

      Content%20Modifier

       

      Thanks,

      Nithya

      Author's profile photo Eng Swee Yeoh
      Eng Swee Yeoh
      Blog Post Author

      Hi Nithya, please open an issue on GitHub, attaching your input payload and also the configuration (preferably in a text file), or a downloaded copy of the IFlow.

      Author's profile photo Dinesh M
      Dinesh M

      Hi Eng Swee,

      One query on the functionality of the FCC Bean. Is it possible to fetch only selected records from the file. Here is the example scenario

      Input:

      HDR,NameList
      DT,Dinesh,Mohan
      DT,Kalyan,Kumar
      DT,Ashwin,Ravi

      Output:

      (First Identifier fields like HDR, DT and Lastname is not needed in the output, Just need the namelist which has first name)

      NameList
      Dinesh
      Kalyan
      Ashwin

      Is there any parameter which can help us to achieve this?

      Thanks,

      Dinesh

       

      Author's profile photo Ahmad Shaik
      Ahmad Shaik

      Hi Eng Swee Yeoh,

      Need help to add recordsetname(Records) in FCC in SAP CPI. 

      Requirement is we are getting multiple fixedfield lengh file from SFTP server in that file getting multiple headers and line items. So we have to split and process one header and lineitems to SAP IDOC. Need your help on this.

      XML fiile in example.

      <?xml version='1.0' encoding='UTF-8'?>
      <Records>
      <Header>
      <ROW_ID>000</ROW_ID>
      <COMP_CODE>2213 </COMP_CODE>
      <VOUCHER_ID>NEXT </VOUCHER_ID>
      <HEADER_TXT>9771210 </HEADER_TXT>
      <DOC_DATE>2023/06/23</DOC_DATE>
      <VENDOR_SETID>BWS </VENDOR_SETID>
      <VENDOR_NO>0000000522</VENDOR_NO>
      <FIELD1>0000208889</FIELD1>
      <ADDRESS_SEQ_NUM> </ADDRESS_SEQ_NUM>
      <GRP_AP_ID>1000001727</GRP_AP_ID>
      <ORIGIN>ONL</ORIGIN>
      <OPRID>BATCH </OPRID>
      <VCHR_TTL_LINES>1 </VCHR_TTL_LINES>
      <ACCOUNTING_DT> </ACCOUNTING_DT>
      <POST_VOUCHER> </POST_VOUCHER>
      <DST_CNTRL_ID> </DST_CNTRL_ID>
      <VOUCHER_ID_RELATED> </VOUCHER_ID_RELATED>
      <GROSS_AMT>355.000 </GROSS_AMT>
      <DSCNT_AMT> </DSCNT_AMT>
      <USETAX_CD> </USETAX_CD>
      <SALETX_AMT>0.000 </SALETX_AMT>
      <SALETX_CD> </SALETX_CD>
      <FREIGHT_AMT>0.000 </FREIGHT_AMT>
      <DUE_DT> </DUE_DT>
      <DSCNT_DUE_DT> </DSCNT_DUE_DT>
      <PMNTTRMS>IMMED</PMNTTRMS>
      <ENTERED_DT> </ENTERED_DT>
      <CURRENCY>USD</CURRENCY>
      <RT_TYPE>CRRNT</RT_TYPE>
      </Header>
      <LineItem>
      <ROW_ID>001</ROW_ID>
      <BUSINESS_UNIT>2213 </BUSINESS_UNIT>
      <VOUCHER_ID>NEXT </VOUCHER_ID>
      <VOUCHER_LINE_NUM>1 </VOUCHER_LINE_NUM>
      <TOTAL_DISTRIBS>1 </TOTAL_DISTRIBS>
      <BUSINESS_UNIT_PO> </BUSINESS_UNIT_PO>
      <PO_ID> </PO_ID>
      <LINE_NBR> </LINE_NBR>
      <SCHED_NBR> </SCHED_NBR>
      <DESCR> </DESCR>
      <MERCHANDISE_AMT> 355.000 </MERCHANDISE_AMT>
      <ITM_SETID> </ITM_SETID>
      <INV_ITEM_ID> </INV_ITEM_ID>
      <QTY_VCHR> </QTY_VCHR>
      <STATISTIC_AMOUNT> </STATISTIC_AMOUNT>
      <UNIT_OF_MEASURE> </UNIT_OF_MEASURE>
      <UNIT_PRICE> </UNIT_PRICE>
      <SALETX_APPL_FLG> </SALETX_APPL_FLG>
      <USETAX_APPL_FLG> </USETAX_APPL_FLG>
      <FRGHT_PRORATE_FLG> </FRGHT_PRORATE_FLG>
      <DSCNT_APPL_FLG> </DSCNT_APPL_FLG>
      <WTHD_SW> </WTHD_SW>
      <TAX_CD_VAT> </TAX_CD_VAT>
      <VAT_RECOVERY_PCT> </VAT_RECOVERY_PCT>
      <BUSINESS_UNIT_RECV> </BUSINESS_UNIT_RECV>
      <RECEIVER_ID> </RECEIVER_ID>
      <RECV_LN_NBR> </RECV_LN_NBR>
      <RECV_SHIP_SEQ_NBR> </RECV_SHIP_SEQ_NBR>
      <MATCH_LINE_OPT> </MATCH_LINE_OPT>
      <DISTRIB_MTHD_FLG> </DISTRIB_MTHD_FLG>
      <TXN_CURRENCY_CD> </TXN_CURRENCY_CD>
      <BASE_CURRENCY> </BASE_CURRENCY>
      <CURRENCY_CD> </CURRENCY_CD>
      <SHIPTO_ID> </SHIPTO_ID>
      <SUT_BASE_ID> </SUT_BASE_ID>
      <TAX_CD_SUT> </TAX_CD_SUT>
      <ULTIMATE_USE_CD> </ULTIMATE_USE_CD>
      <SUT_EXCPTN_TYPE> </SUT_EXCPTN_TYPE>
      <SUT_EXCPTN_CERTIF> </SUT_EXCPTN_CERTIF>
      <SUT_APPLICABILITY> </SUT_APPLICABILITY>
      <WTHD_SETID> </WTHD_SETID>
      <WTHD_CD> </WTHD_CD>
      <VAT_APPL_FLG> </VAT_APPL_FLG>
      <VAT_APPLICABILITY> </VAT_APPLICABILITY>
      <VAT_TXN_TYPE_CD> </VAT_TXN_TYPE_CD>
      <NATURE_OF_TXN1> </NATURE_OF_TXN1>
      <NATURE_OF_TXN2> </NATURE_OF_TXN2>
      <VAT_USE_ID> </VAT_USE_ID>
      <BUSINESS_UNIT_GL> </BUSINESS_UNIT_GL>
      <ACCOUNT> </ACCOUNT>
      <ALTACCT> </ALTACCT>
      <OPERATING_UNIT> </OPERATING_UNIT>
      <DEPTID> </DEPTID>
      <PRODUCT> </PRODUCT>
      <PROJECT_ID> </PROJECT_ID>
      <AFFILIATE> </AFFILIATE>
      <VCHR_BLD_MATCH_FLG> </VCHR_BLD_MATCH_FLG>
      <BUSINESS_UNIT_RTV> </BUSINESS_UNIT_RTV>
      <RTV_ID> </RTV_ID>
      <RTV_LN_NBR> </RTV_LN_NBR>
      <DEST_CNTRY> </DEST_CNTRY>
      <SOURCE_FROM> </SOURCE_FROM>
      <SUPPLEMENTAL> </SUPPLEMENTAL>
      </LineItem>
      <Distribution>
      <ROW_ID>002</ROW_ID>
      <COMP_CODE>2213 </COMP_CODE>
      <VOUCHER_ID>NEXT </VOUCHER_ID>
      <VOUCHER_LINE_NUM>1 </VOUCHER_LINE_NUM>
      <DISTRIB_LINE_NUM>1 </DISTRIB_LINE_NUM>
      <GLCOMP_CODE>2213 </GLCOMP_CODE>
      <GL_ACCOUNT>650440 </GL_ACCOUNT>
      <ALTACCT> </ALTACCT>
      <STATISTICS_CODE> </STATISTICS_CODE>
      <STATISTIC_AMOUNT> </STATISTIC_AMOUNT>
      <QTY_VCHR> </QTY_VCHR>
      <JRNL_LN_REF> </JRNL_LN_REF>
      <OPEN_ITEM_STATUS> </OPEN_ITEM_STATUS>
      <DESCR> </DESCR>
      <AMT_DOCCUR>355.000 </AMT_DOCCUR>
      <BUSINESS_UNIT_PO> </BUSINESS_UNIT_PO>
      <PO_ID> </PO_ID>
      <LINE_NBR> </LINE_NBR>
      <SCHED_NBR> </SCHED_NBR>
      <PO_DIST_LINE_NUM> </PO_DIST_LINE_NUM>
      <BUSINESS_UNIT_PC> </BUSINESS_UNIT_PC>
      <ACTIVITY_ID> </ACTIVITY_ID>
      <ANALYSIS_TYPE> </ANALYSIS_TYPE>
      <RESOURCE_TYPE> </RESOURCE_TYPE>
      <RESOURCE_CATEGORY> </RESOURCE_CATEGORY>
      <RESOURCE_SUB_CAT> </RESOURCE_SUB_CAT>
      <ASSET_FLG> </ASSET_FLG>
      <BUSINESS_UNIT_AM> </BUSINESS_UNIT_AM>
      <ASSET_ID> </ASSET_ID>
      <PROFILE_ID> </PROFILE_ID>
      <FREIGHT_AMT> </FREIGHT_AMT>
      <SALETX_AMT> </SALETX_AMT>
      <USETAX_AMT> </USETAX_AMT>
      <VAT_TXN_TYPE_CD> </VAT_TXN_TYPE_CD>
      <VAT_INV_AMT> </VAT_INV_AMT>
      <VAT_NONINV_AMT> </VAT_NONINV_AMT>
      <BUSINESS_UNIT_RECV> </BUSINESS_UNIT_RECV>
      <RECEIVER_ID> </RECEIVER_ID>
      <RECV_LN_NBR> </RECV_LN_NBR>
      <RECV_SHIP_SEQ_NBR> </RECV_SHIP_SEQ_NBR>
      <RECV_DIST_LINE_NUM> </RECV_DIST_LINE_NUM>
      <COSTCENTER>221361146 </COSTCENTER>
      </Distribution>
      <Payment_Info>
      <ROW_ID>006</ROW_ID>
      <COMP_CODE>2213 </COMP_CODE>
      <VOUCHER_ID>NEXT </VOUCHER_ID>
      <PYMNT_CNT>1 </PYMNT_CNT>
      <BANK_CD> </BANK_CD>
      <BANK_ACCT_KEY> </BANK_ACCT_KEY>
      <PYMNT_METHOD> </PYMNT_METHOD>
      <FIELD1>21719800410117195 </FIELD1>
      <PYMNT_VCHR_PCT> </PYMNT_VCHR_PCT>
      <PYMNT_HANDLING_CD> </PYMNT_HANDLING_CD>
      <PYMNT_HOLD>N</PYMNT_HOLD>
      <PYMNT_HOLD_REASON> </PYMNT_HOLD_REASON>
      <MESSAGE_CD> </MESSAGE_CD>
      <PYMNT_GROSS_AMT>355.000</PYMNT_GROSS_AMT>
      </Payment_Info>
      </Records>