Technology Blogs by Members
Explore a vibrant mix of technical expertise, industry insights, and tech buzz in member blogs covering SAP products, technology, and events. Get in the mix!
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

SAP CPI – SAP S4HANA (MDG and DMS ) integration with Amazon S3 and External API

rhviana
Active Contributor
‎03-02-2021 12:59 AM
Hello Folks,

One more interesting blog and sharing the knowledge and experience with you, basically let’s follow the topics and instructions during the reading.

Agenda:







  1. Introduction




  2. Scenario and Integration Perspective




  3. Amazon S3 Bucket service




  4. Scenario and Integration Perspective





      1. Challenge in this integration - In detail.






  5. Integration Flow in SAP CPI using AmazonWebService Adapter





      1. Scenario one: Send the data from S4/HANA ( SAP DMS ) to Amazon S3 Bucket Service.

      2. Scenario two: Send the payload (JSON format) with the details of the file that has been send t Amazon S3 Bucket to External API.






  6. Problems - Hexadecimal message and Gather Function to collect all message result (Update - 07/04/2021)




  7. SOAP UI Test







Introduction:



In below blog I would like to share how to integrate Amazon S3 Bucket service using the new adapter release by SAP.

In case that you still not have this adapter you can take a look in my previous blog - where I explain how to generate the AWS Header V4 signature via Groovy Script and using the HTTPS adapter.

SAP CPI – Amazon S3 Integration – Groovy Header Signature V4 – with HTTPS adapter


What I will not cover in the blog ?

  • The whole setup of the Amazon S3 service detail , you can easily check.

  • How to deploy the new adapter release by SAP.

  • ABAP codes to extract the binary file from SAP DMS

  • ABAP code to send the file from SAP MDG

  • Further details how the external system check the files in Amazon S3 service with the JSON file received in the API Service.

  • Storage of the exception





Amazon S3 Bucket service





An Amazon S3 bucket is a public cloud storage resource available in Amazon Web Services’ (AWS) Simple Storage Service (S3), an object storage offering. Amazon S3 buckets, which are similar to file folders, store objects, which consist of data and its descriptive metadata.

Basically the S3 bucket act on-cloud repository

S3 Bucket vs traditional SFTP:


I recommend you to read the advantages to think – GO TO CLOUD in your business, not only for storage perspective, please take look in the link below with the main differences between S3 Bucket (Cloud)  and the traditional on-premise SFTP server.

S3 vs SFTP

Important to mention that Amazon provides SFTP on-cloud also, we are not going to explore this topic in this blog.



Scenario and Integration perspective:



The scenario is the integration off S4HANA, once the change of state of Material happens, the synchronism with SAP MDG is automatically and the replication of possible document, pdf's, imagens and others are store in the SAP DMS..




  • Scenario 1: - Green Line





    • The details of the materials the documents ( pdf's, imagens and others)  is stored in the SAP DMS, a custom abap function was developed to extract those details in binary base64 and send to SAP CPI.

    • SAP CPI will receive using the SOAP-RM ( Asynchronous ), proceed with some transformations and forward to AWS S3

    • SAP CPI receive back the reply, if HTTP 200, return success to SAP DMS, if happens exception or HTTP different 200, return fail.





  • Scenario 2: - Black Line

    • After receive the status success from the scenario 1, the SAP MDG with custom function extract the details the internal table and send a custom XML to SAP CPI.

    • SAP CPI will receive using the SOAP-RM ( Asynchronous ) proceed with some transformations and send the JSON file to the API





  • Scenario 3: - Yellow Line.

    • The external system is responsible to check the file and compeer what is store in Amazon S3.







Scenario one: Send the data from S4/HANA ( SAP DMS ) to Amazon S3 Bucket Service.





As per of the clear description above, I will present you the Scenario 1 - Iflow in SAP CPI.

I will not cover the orientation about local integration process because it is related with the system responsible to store the logs of Success and Exception.

  • Steps in black a success message

  • Steps in red a fail message









Custom XML Model from SAP DMS:


As you can see in the XML could contains more than one file, type of file, folder to be store and many <LINES> of the BASE64 from the file.



Challenge in this integration





Send one single file per document as string to Amazon S3.

I need to spilt the files and concatenate all those lines in one single document, decode and send to Amazon S3 bucket.

 

To solve the challenge I used the:

  • Interacting Splitter ( to generate a new calls from <item> )

  • Content Modifier to generate properties dynamically that will be use per call

  • XSLT Mapping to exclude <MATERIAL>,<PASTA>,<NOME_ARQUIVO>,<CONTENT_TYPE> and just keep <item><LINE> and concatenate all content.

  • Decoding BASE64 Standard Function.


Interating Splitter - Configuration:



Content-Modifier - Configuration:



XSLT - Code:



This XSLT code is removing some tags and concatenating any value that comes inside <LINE> tag and check if the position is the last.



<xsl:stylesheet version="1.0"

 xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

 <xsl:output method="text"/>

 <xsl:strip-space elements="*"/>



  <xsl:template match="//item/MATERIAL"/>

  <xsl:template match="//item/PASTA"/>

  <xsl:template match="//item/NOME_ARQUIVO"/>

  <xsl:template match="//item/CONTENT_TYPE"/>



  <xsl:template match="/LINE">

  <xsl:value-of select="concat('',@value)"/>



  <xsl:if test="not(position()=last())">

    <xsl:text>&amp;</xsl:text>

  </xsl:if>

 </xsl:template>

</xsl:stylesheet>

Decoding BASE64 - No configuration needed, check the flow in detail below:



Details inside SAP CPI







Details behind







The result of those steps is basically send any messages, separate per type as string to Amazon S3, you will see more in details in the test sample further in this blog.

Configuration of Amazon adapter for SAP CPI, you can check in my previous blog but I will present here the details extract by the second step in the Content-Modifier

SAP CPI – Amazon S3 Integration – Groovy Header Signature V4 – with HTTPS adapter

Content Modifier to generate Dynamically the Directory, File Name and Content-Type per call, generated by the Integrating Splitter.






Scenario two: Send the payload (JSON format) with the details of the file that has been send t Amazon S3 Bucket to External API.





I will not describe in details this flow because the focus is in the first scenario, but basically here there are 4 local internal process - 2 for logging and 2 for call API with the URL dynamically based on the XML input, the responsible for this is the routing step.



Problems - Hexadecimal message and Gather Function to collect all message result





Another issue that comes up was that SAP DMS stantard function delivery the any time of file as Hexadecimal and not BASE64 to solve this I developed a groovy script.


Groovy steps:

  1. Exclude all tags from the result from XSLT

  2. Only delivery a pure string Hexadecimal from the document.

  3. DecodeHex

  4. EncodeBase64

  5. Return message


/**

* Name: Decode Hexa, Enconde HexaBase64, Return Base64 as String

* Creator: Ricardo Viana

**/

import com.sap.gateway.ip.core.customdev.util.Message

import org.apache.commons.codec.binary.Base64;

import org.apache.commons.codec.binary.Hex;



def Message processData(Message message) {

    // Get Body XML from <Item>

    def body = message.getBody(java.lang.String)

    // Get properties from the message

    def map = message.getProperties();

    // Replace XML tags to decodeHex only the string from <LINE>

    def item = body.replaceAll('<item>','');

    def fechaItem = item.replaceAll('</item>','');

    def content = fechaItem.replaceAll('<CONTENT>','');

    def fechacontent = content.replaceAll('</CONTENT>','');

    def line = fechacontent.replaceAll('<LINE>', '')

    def fechaline = line.replaceAll('</LINE>', '')

    def lineHexa = fechaline.replaceAll("\\s","")

    // Bytes decodeHex from string

    byte[] decodedHex = Hex.decodeHex(lineHexa.toString());

    // Bytes encodeHexBase64

    byte[] encodedHexB64 = Base64.encodeBase64(decodedHex);

    // Return String enconded Hexadecimal Base64 as a body ( payload )

    String encondedHexB64String = new String (encodedHexB64)

    message.setBody(encondedHexB64String)

  return message

 

}




Sample XML:



            <item>

               <CONTENT>

                  <item>

                     <LINE> HEXADECIMAL </LINE>

                  </item>

               </CONTENT>

               <CONTENT>

                  <item>

                     <LINE> HEXADECIMAL </LINE>

                  </item>

               </CONTENT>

               <CONTENT>

                  <item>

                     <LINE> HEXADECIMAL </LINE>

                  </item>

               </CONTENT>

		....

            </item>




Link of SAP CPI IDE Test:





SAP IDE with groovy

Site recommended:

This BASE64 Guru site helps me a lot to reproduce the same result with groovy, to valided if those functions works proper.

BASE64 Guru

Result after the groovy

HEXADECIMALHEXADECIMALHEXADECIMALHEXADECIMALHEXADECIMALHEXADECIMALHEXADECIMALHEXADECIMALHEXADECIMALHEXADECIMALHEXADECIMALHEXADECIMALHEXADECIMALHEXADECIMALHEXADECIMALHEXADECIMALHEXADECIMALHEXADECIMALHEXADECIMALHEXADECIMALHEXADECIMALHEXADECIMALHEXADECIMALHEXADECIMALHEXADECI






Gather Function:





I decide use the splitter, so for each message will generate one result, per message could generate 6 results more less, so I decide user Gather function with plain/text with values concatenate 1 or 2, but this is not going to be use in the end, because there is one groovy script generating the result using the value 1 (Success) and 2 (Fail) and IV_MATERIAL that comes from the property saved in previous content modifier.


So based on the success the Gather will received 000000 or 1111111 and with next Groovy I generate the message.



Groovy Script return:





I could not find a proper Camel to check fail of the Amazon Adapter and I decide use the CamelSliptComplete, in case of the process fail the status will be false or true, base on that the groovy generate the correct result, with this converged the exception handllying.
/**

* Name: Return SAP MDG with Success of Fail message using the CamelSliptComplete status

* Creator: Ricardo Viana

**/

import com.sap.gateway.ip.core.customdev.util.Message

import groovy.xml.MarkupBuilder



def Message failS3AWS(Message message) {

    processData(message)

    return message

}



def Message sucessoS3AWS(Message message) {

    processData(message)

    return message

}



def Message processData(Message message) {

    def xHeaders = message.getHeaders()

    def map = message.getProperties()

	def ex = map.get("CamelSplitComplete")

	Writer writer = new StringWriter()

    if (ex==true) {

        def indentPrinter = new IndentPrinter(writer, ' ')

        def builder = new MarkupBuilder(indentPrinter)

        builder.'ns0:ZMDG_RETORNO_ASSETS'('xmlns:ns0': 'urn:sap-com:document:sap:rfc:functions') {

                'ID_RETORNO' '1'

                'IV_MATERIAL' map.get('Material')

        }

    }

   else{

        def indentPrinter = new IndentPrinter(writer, ' ')

        def builder = new MarkupBuilder(indentPrinter)

        builder.'ns0:ZMDG_RETORNO_ASSETS'('xmlns:ns0': 'urn:sap-com:document:sap:rfc:functions') {

               'ID_RETORNO' '2'

                'IV_MATERIAL' map.get('Material')

        }

    }

    message.setBody(writer.toString())

    return message

}




SOAP UI Test





Here in this section I will explain you the test case and just overview of the Scenario 2 and nothing about Scenario 3 because is the responsibility of the external system.

In the call I will use the same sample of image of XML provide above, I can't post here because of the content of <LINE> is something really big.


Check the details of both files that will be store in the folder dinamically - documentacaoPDF / imagemPNG and names - TEST_BLOG_SAP.pdf and TEST_BLOG_SAP.png
 <item>

               <MATERIAL>6057099A</MATERIAL>

               <PASTA>documentacaoPDF</PASTA>

               <NOME_ARQUIVO>TEST_BLOG_SAP.pdf</NOME_ARQUIVO>

               <CONTENT_TYPE>application/pdf</CONTENT_TYPE>

               <CONTENT>

                 <item>

                   <LINE> XXXXXXXXXXX</LINE>

                 </item>

               </CONTENT>

 </item>

  <item>

               <MATERIAL>6057099A</MATERIAL>

               <PASTA>imagemPNG</PASTA>

               <NOME_ARQUIVO>TEST_BLOG_SAP.png</NOME_ARQUIVO>

               <CONTENT_TYPE>image/png</CONTENT_TYPE>

               <CONTENT>

                 <item>

                   <LINE> XXXXXXXXXXX</LINE>

                 </item>

               </CONTENT>

</item>




Trace in SAP CPI:







Interacting Splitter step 2x:







Before the XSLT that concatenate all values from <LINE> and produce one single string of image, or document, or any MIME type file.






After the XSLT - Base64 step 2x:



Result the concatenation of all base64 content from each <LINE> inside the <CONTENT>






S3 Bucket without folders:







S3 Bucket after the SAP CPI call with two folders - documentacaoPDF / imagemPNG







Let's check those files - PDF first - TEST_BLOG_SAP.pdf








Let's check those files - PNG first - TEST_BLOG_SAP.png








As you can see the PDF and Image extract from S4HANA ( DMS )  is store with success in the Bucket S3 Amazon.



Now the Scenario 2:

Based on the XML input from S4HANA ( MDG ) custom function, there will be a routing for different address in the API based on the data from the XML:

I will present a short version and simple explanation of the Scenario 3.
               <Id>794290</Id>

               <Name>TEST_BLOG_SAP.pdf</Name>

               <Version>AA</Version>

               <Part>000"</Part>

               <Path>/documentacaoPDF</Path>

               <EvBrand>SAP</EvBrand>

               <EvName>TEST_BLOG_SAP</EvName>

               <EvType>TEST</EvType>

The values from <EvBrand> and <EvType> will be used to generate the dynamically URL call to API and the values from <Name>, <Path>,<EvName> will be used for external company check in the bucket if everything is ok with the file.



I hope that you enjoy the read and what I presented in this blogs.

Kind Regards,

Viana.
Labels in this area
Popular Blog Posts