Skip to Content
Technical Articles
Author's profile photo Mandy Krimmel

Cloud Integration – ZIP and Compress Capabilities: Encoder, Decoder, Splitter, Aggregate

This blog describes the options for message zipping and compressing of messages in Cloud Integration. It explains the different flow steps and the respective configuration options.

ZIP and Compress Capabilities in Cloud Integration

In some integration scenarios messages have to be exchanged in a zipped and/or compressed format. Cloud Integration offers different flow steps for different use cases.

ZIP Encoder

The ZIP Encoder can be used to zip a single message during integration flow processing.

The ZIP Encoder can be found in the palette under Message Transformers -> Encoders -> ZIP Compression. This flow step has no further configuration parameters, it simply zips a message into a zip archive.

GZIP Encoder

The GZIP Encoder can be used to compress a single message with the gzip algorithm during integration flow processing.

The GZIP Encoder can be found in the palette under Message Transformers -> Encoders -> GZIP Compression. This flow step has no further configuration parameters, it simply compresses a message.

ZIP Decoder

The ZIP Decoder can be used to un-zip a single message from a zip archive during integration flow processing. Only one message may be contained in the archive, or else an error will be raised.

The ZIP Decoder can be found in the palette under Message Transformers -> Decoders -> ZIP Decompression. This flow step has no further configuration parameters, it simply un-zips a message from a zip message archive.

GZIP Decoder

The GZIP Decoder can be used to un-compress a single gzip-compressed message during integration flow processing.

The GZIP Decoder can be found in the palette under Message Transformers -> Decoders -> GZIP Decompression. This flow step has no further configuration parameters, it simply un-compresses a message.

ZIP Splitter (available with the update in May 2020)

The ZIP Splitter can be used to un-zip a zip archive containing multiple files. It splits them into single messages and processes them one after the other.

The ZIP Splitter can be found in the palette under Routing -> Splitter -> Zip Splitter. This flow step has two configuration parameters:

Using the File Name Pattern field, you can define a pattern for the files from the zip archive, which have to be processed. If you, for example, want to process only xml files from the archive and ignore the others, you can define *.xml. If no information is given in this field all files from the archive are processed.

Expressions such as ab*, a.*, *a*, ?b, and so on, are supported. Note that the expression * replaces no character or an arbitrary number of characters. The expression ? replaces exactly one arbitrary character.

Examples:

  • If you specify file*.txt as the File Name Pattern, the following files are processed: file1.txt, file2.txt, as well as file.txt and file1234.txt, and so on.
  • If you specify file?.txt as the File Name Pattern, the following files are processed: file1.txt, file2.txt, and so on, but not the files file.txt or file1234.txt.

By using the Stop on Exception field, you can define if the split process shall stop if there has been an error during the processing of one of the split messages or if the split process shall continue to process the next file from the archive.

ZIP Aggregate (available with the update in May 2020)

The ZIP Aggregate or ZIP Gather can be used to zip several messages coming from different branches or splits into a zip archive.

The ZIP Gather can be found in the palette under Routing -> Gather. In the Gather step you can select Zip as Aggregation Algorithm. This Aggregation Algorithm can be used for all Incoming Formats. The flow step has one configuration parameter:

In the File Name field you can enter a simple expression to create the file name for the files in the archive. For example, with ${header.id}.xml you can create the files with different names based on the header id. The header has to be set before the gather step. You need to make sure that the file name is unique within the archive, or else there will be an error during processing because multiple files cannot have the same name within a zip archive. If nothing is defined in the field, the header CamelFileName is evaluated. In case neither an expression nor the CamelFileName header is specified during runtime a unique file name is generated.

TAR Splitter (will be available with the update in June 2020)

The TAR Splitter can be used to un-tar a tar archive containing multiple files. It splits them into single messages and processes them one after the other.

The TAR Splitter can be found in the palette under Routing -> Splitter -> Tar Splitter. This flow step has two configuration parameters:

Using the File Name Pattern field, you can define a pattern for the files from the tar archive, which have to be processed. If you, for example, want to process only xml files from the archive and ignore the others, you can define *.xml. If no information is given in this field all files from the archive are processed.

Expressions such as ab*, a.*, *a*, ?b, and so on, are supported. Note that the expression * replaces no character or an arbitrary number of characters. The expression ? replaces exactly one arbitrary character.

Examples:

  • If you specify file*.txt as the File Name Pattern, the following files are processed: file1.txt, file2.txt, as well as file.txt and file1234.txt, and so on.
  • If you specify file?.txt as the File Name Pattern, the following files are processed: file1.txt, file2.txt, and so on, but not the files file.txt or file1234.txt.

By using the Stop on Exception field, you can define if the split process shall stop if there has been an error during the processing of one of the split messages or if the split process shall continue to process the next file from the archive.

TAR Aggregate (will be available with the update in June 2020)

The TAR Aggregate or TAR Gather can be used to combine several messages coming from different branches or splits into a tar archive.

The TAR Gather can be found in the palette under Routing -> Gather. In the Gather step you can select Tar as Aggregation Algorithm. This Aggregation Algorithm can be used for all Incoming Formats. The flow step has one configuration parameter:

In the File Name field you can enter a simple expression to create the file name for the files in the archive. For example, with ${header.id}.xml you can create the files with different names based on the header id. The header has to be set before the gather step. You need to make sure that the file name is unique within the archive, or else there will be an error during processing because multiple files cannot have the same name within a zip archive. If nothing is defined in the field, the header CamelFileName is evaluated. In case neither an expression nor the CamelFileName header is specified during runtime a unique file name is generated.

 

Assigned Tags

      15 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Maik Offerle
      Maik Offerle

      HI Mandy,

      good to see progress there 🙂

      Thanks for the update and regards

      Maik

      Author's profile photo Matthan Sienes Logronio
      Matthan Sienes Logronio

      Hi Mandy,

       

      Does this ZIP splitter works with .7z files?

       

      Thanks for the update..keep safe..

       

      Matt

      Author's profile photo Mandy Krimmel
      Mandy Krimmel
      Blog Post Author

      Hello Matt,

      no .7z files are not supported, only zip.

      Regards

      Mandy

       

      Author's profile photo Bart Truijens
      Bart Truijens

      Hello Mandy,

      Does the ZIP Decoder handle zip files which are secured with a password ?

      Regards

      Bart

      Author's profile photo Mandy Krimmel
      Mandy Krimmel
      Blog Post Author

      No, this is not supported.

      Best regards

      Mandy

      Author's profile photo Arun Maari Rajha K V
      Arun Maari Rajha K V

      Hi Mandy,

      Nice blog explaining the zip/unzip functionality.

      Please let me know if gunzip (.gz) can be extracted using GZIP Decompress?

      Thanks,

      Arun Maari Rajha

       

      Author's profile photo Mandy Krimmel
      Mandy Krimmel
      Blog Post Author

      Yes, the GZIP decoder can extract .gz files.

      Best regards

      Mandy

      Author's profile photo Arun Maari Rajha K V
      Arun Maari Rajha K V

      thank you for confirming!

      Author's profile photo Freek den Teuling
      Freek den Teuling

      Hello Mandy,

       

      Thanks for the nice blog.

       

      I have a use case where I need to send a GZIP containing multiple files via AS4. AS4 standard only supports GZIP. If I read correctly the ZIP Aggregate or ZIP Gather will not help me since they are not supporting GZIP, correct? Is there another way in CPI to bundle multiple files in a GZIP?

       

      Kind Regards,

       

      Freek

      Author's profile photo Mandy Krimmel
      Mandy Krimmel
      Blog Post Author

      No, there unfortunately there is no such configuration option available, you would have to use script to do this specific use case.

      Best regards

      Mandy

      Author's profile photo Freek den Teuling
      Freek den Teuling

      Thank you for the quick reply. I'll look into using a script.

      Kind Regards,

      Freek

      Author's profile photo Anderson Melo
      Anderson Melo

      HI Mandy, how are y doing?

      Congratulations for the blog and if you can help, i will be grateful.

      I have a very simple flow in SAP CPI, I use a "content modifier", add "Authorization" and "Content-Type", but it doesn't work. The zip file is not sent, it even appears for download in the trace, but it does not reach the server.

       

      Integration Flow -> Runtime configuration

       

      HTTPS -> Connection

       

      Content Modifier -> Message Header

       

      HTTP -> Connection

       

      The zipped file must go in the body of the submission, we can see it in the payload. Although, when it arrives at the endpoint we forwarded, the file does not exist in the body.
      Author's profile photo Mandy Krimmel
      Mandy Krimmel
      Blog Post Author

      Do I get it right, that the flow not uses the zip encoder at all? I would suggest to open a ticket with the question. This is not easily answered without all the details and a sample flow.

      Best regards

      Mandy

      Author's profile photo Anderson Melo
      Anderson Melo
      The flow starts receiving a zip in the body of the file, inside SCPI we handle the authentication part and forward the same zip in the body to the destination. Although, as I said, the destination acknowledges not receiving this file.
      
      
      Ícone "Verificada pela comunidade"
      Author's profile photo Mandy Krimmel
      Mandy Krimmel
      Blog Post Author

      The processing needs to be traced and checked in detail. Please open a ticket and attach the integration flow and the trace details.

      BR

      Mandy