Skip to Content
Technical Articles
Author's profile photo Rajesh PS

CSV Multi-Part Form-Data Upload as Attachment using HTTP_AAE Adapter in SAP PI 7.5

Recently I had developed a IDOC to HTTPs Asynchronous scenario in SAP PI 7.5. There was a requirement integrating SAP ECC(client) and Amazon S3 (server) via EAI SAP PI 7.5.

In the server side, file name should be delivered as an Attachment in a csv multi form data. SAP has provided a Java method to read the attachment data and content type, content disposition, content ID, but there is no standard function to read the attachment file name.

The new Java HTTP_AAE adapter supports attachments, Multipart Documents, form submission. HTTP_AAE adapter also provides options to send HTTP Header Parameters, Query Parameters and HTTP URL parameters in the dynamic header of the PI message.

To read the form data attachment ‘name’ and ‘filename’ as Content-Disposition (content which is expected to be displayed inline in the browser, i.e. as a Web page or as part of a Web page, or as an Attachment, that is downloaded and saved locally).

Content-Type: multipart/form-data; boundary=—-WebKitFormBoundary7MA4YWxkTrZu0gW

——WebKitFormBoundary7MA4YWxkTrZu0gW

Content-Disposition: form-data; name=”file”; filename=”INV_20180119_110812.csv”

 

Below are the detail steps which explains in reading the API body parameters i.e. the key and value and sending the file name dynamically :

In the first graphical mapper(IDOC to XML) declare the Adapter Specific Message Attributes as an User-defined Functions for deriving the file name scheme as below and map to an field ‘fileName’.

Code:

public String getASMAFileName(String CREDAT, String CRETIM, Container container) throws StreamTransformationException{

String filename = “INV_” + CREDAT + “_” + CRETIM + “.csv”;

return filename;

1st Mapper:

In the second graphical mapper(XML to CSV) declare the Adapter Specific Message Attributes as an User-defined Functions for setting the mime type, specifying csv header fields, creating attachment as csv form data, passing the ASMA filename from first graphical mapper and header fields as an Input arguments to the user defined function and map to an field ‘file’.

Below is the standard Java method.

Code:

public void setContentType(String[] SKUId, String[] EANNumber, String[] Warehouse, String[] Quantity, String[] UOM, String[] Cost, String[] Entity, String[] TransactionType, String[] filename, ResultList rs, Container container) throws StreamTransformationException {

AbstractTrace trace = container.getTrace();

// field Names from your first mapping structure

String header = “SKUId,EANNumber,Warehouse,Quantity,UOM,Cost,Entity,TransactionType”;

String content = header + “\n”;

String mimeType = “application/vnd.ms-excel;charset=” + “\”” + “uft-8” + “\””;

String fName = filename[0];

for (int i = 0; i < SKUId.length; i++) {

// adjust the below line with your field names from first structure

content = content + SKUId[i] + “,” + EANNumber[i] + “,” + Warehouse[i] + “,” + Quantity[i] + “,” + UOM[i] + “,” + Cost[i] + “,” + Entity[i] + “,” + TransactionType[i] + “\n”;

}

trace.addInfo(content);

//Create attachment with CSV data

try {

GlobalContainer globalContainer = container.getGlobalContainer();

OutputAttachments outputAttachments = globalContainer.getOutputAttachments();

Attachment attachments = outputAttachments.create(fName, mimeType, content.getBytes());

outputAttachments.setAttachment(attachments);

} catch (Exception e) {

e.toString();

}

rs.addValue(fName);

 

2nd Mapper:

In the Integration Directory:

Coming to the Java HTTP_AAE Receiver communication channel, Under General tab the HTTP request details can be specified as multi-part form request and can be sent as an csv file attachment.

Tick ‘Set Form’ if the target server accepts a form-based submission. Enter the Main Payload Field Name to specify the name of the request parameter [here it is ‘file’ (name)] whose value contains the XI main payload.

Tick ‘Set Multipart’ if the HTTP request is a multipart request where one or more different sets of data form the HTTP body.

If you want the attachments in the request to be delivered to the target system, choose Keep Attachments.

Form-Based File Upload indicates that the HTTP POST request is form based and is multipart (RFC 1867). You use this scenario to select the XI message payload and XI main message attachments from a form-based file-upload.

Now Under Advanced tab, set the Additional Parameters as below it is used to submit the content type headers as ‘true’ (Incoming HTTP request content type headers).

Under Module Configuration tab, define the ‘PayloadSwapBean’ module name
(localejbs/AF_Modules/PayloadSwapBean) and declare the ‘swap.keyName’ as ‘content-type’ and ‘swap.keyValue’ as application/vnd.ms-excel;charset=”uft-8″.

PayloadSwapBean module will replace the application payload (Main Document) of the XI message that contains the business data with another payload that is appended to the XI message as an attachment. Note: Using PayloadSwapBean we cannot handle multiple attachments. This limits us to use when we are dealing with only one attachment.

swap.keyName is to enter a payload attribute such as payload-name or payload-description, or an attribute from the MIME header such as content-Disposition, content-description, and so on.

swap.keyValue is to enter the value that the attribute is to have. The MIME headers of the payloads are searched in order and the system takes the first payload that meets the criteria. This payload is swapped with the application payload.

TraceHTTP is to trace server-side HTTP communications and below are the following severity levels we can configure for parameter Trace HTTP:

Plain – The whole HTTP Request and response communication is traced(headers, body) in plain text.

Headers -Only HTTP headers are traced, both for the HTTP Request and response

Hex – A hex dump is traced  for the whole HTTP request and response communication in the hexadecimal format.

This will indeed dump the entire raw HTTP message(Both request and response) to the Logs in …/server#/j2ee/cluster/dispatcher/log/services/http directory.

Do note this feature is supported only PI 7.31 SP 14 and greater, PI 7.4, 7.5 SP 9 and greater.

 

Debug the HTTP traces in Nwa log Viewer application

Open NWA->Troubleshooting->Logs and Traces -> Log Viewer

Amazon S3 target server:

However, in this blog, I had solved this problem by manually creating Multipart request using JAVA Mapping since REST/SOAP Adapter is not having multipart form(request) options by default. In this way you can use plain HTTP_AAE adapter. Below is the URL to understand how a multipart request looks like. Please visit this URL:

https://www.w3.org/TR/html401/interact/forms.html#h-17.13.4.2

This blog tries to explain how a multipart form data request looks like, how it can be easily generated from XI and provides java code to achieve this scenario using HTTP_AAE.

Assigned Tags

      24 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Josmar Alarcon
      Josmar Alarcon

      Rajesh Thank you very much for your article, I have been very helpful, I would like to ask you if this UDF or the use of the OutputAttachments class can be applied for the sending of attachments in from-data in a receiver type REST 7.5 adapter

      thanks

      Author's profile photo Rajesh PS
      Rajesh PS
      Blog Post Author

      Thanks Josmar.

      Yes you can send the form data attachments using REST Adapter you will have to upgrade to latest PI package(>SP9 ). For more details please refer below link:

      https://blogs.sap.com/2019/04/30/rest-adapter-from-data-with-attachment-handling/comment-page-1/#comment-458661

      Author's profile photo Josmar Alarcon
      Josmar Alarcon

      thanks Rajesh!

      Author's profile photo Rajesh PS
      Rajesh PS
      Blog Post Author

      You're Welcome!

      Author's profile photo Josmar Alarcon
      Josmar Alarcon

      Hi Rajesh, I'm sending an xml in the front-data but in the log it appears that after attaching the document it takes the xml of the mapping, as if it were superimposing it:

      this is img from transformation 1 where this is transformating:

      then this is overlaying the mapping xml:

      So, I would like to know what I'm doing, I may not be doing well, I think it could be on the http channel because with the PayloadSwapBean, I should take what I'm loading in the set, right?

      Thanks for the help

      Author's profile photo Rajesh PS
      Rajesh PS
      Blog Post Author

      Hi Josmar,

       

      Could you please explain me your scenario. Is it iDoC to CSV forms?

      Author's profile photo Josmar Alarcon
      Josmar Alarcon

      the scenario is proxy to rest, where the rest API receives an XML in the from-data

      thanks

      Author's profile photo Rajesh PS
      Rajesh PS
      Blog Post Author

      HI Josmar,

       

      On seeing the error which states HTTP 403 Forbidden , it is an issue with the content and body params,

       

      Requesting you to check once again the mappers and channel configuration patiently.

       

      Before that execute once in POSTMAN take the code snippets and compare with the Http Traces generated in PI log viewer.

       

      Regards - Rajesh PS

      Author's profile photo Rajesh PS
      Rajesh PS
      Blog Post Author

      Hi Josmar,

       

      Yes, please check with the PayloadSwapBean. The content disposition is not appropriate.

      Ideally it is not replacing the application payload (Main Document) of the XI message that contains the business data with another payload that is appended to the XI message as an attachment.

      Author's profile photo Josmar Alarcon
      Josmar Alarcon

      Hi Rajesh.

      Attached below the PayloadSwapBean configuration of the adapter:

       

      and this is what the log shows me:

       

      Regards – Josmar Alarcon

      Author's profile photo Rajesh PS
      Rajesh PS
      Blog Post Author

      Hi Josmar,

       

      Please check in Advanced tab, set the Additional Parameters as mentioned below

      preferredContentTypeHeader as true.

       

      Also set Form and mention Main Payload Field Name and as well select  Multipart.

      Author's profile photo Josmar Alarcon
      Josmar Alarcon

      Hi Rajesh

      I tell you what worked but now I have a problem when sending a dynamic variable to send a dynamic token, you have some idea of how to do it from an HTTP_AAE adapter since a REST adapter is not taking me the configurations you named previously .

       

      Thank you very much for your help.

       

      Regards – Josmar Alarcon

      Author's profile photo Rajesh PS
      Rajesh PS
      Blog Post Author

      Hi Josmar,

       

      You can accomplish by using User defined function in graph mapper and map it root node appropriately.

       

      What kind of token is it?

       

      https://blogs.sap.com/2017/07/05/api-token-via-http-lookup-in-adapter-module/

      https://blogs.sap.com/2014/12/18/pi-rest-adapter-using-dynamic-attributes/

       

      Code looks like below:

      package com.xxx.aii.mapping;
      
      import java.io.InputStream;
      import java.io.OutputStream;
      import com.sap.aii.mapping.api.AbstractTransformation;
      import com.sap.aii.mapping.api.StreamTransformationException;
      import com.sap.aii.mapping.api.TransformationInput;
      import com.sap.aii.mapping.api.TransformationOutput;
      
      public class MessageMapping extends AbstractTransformation {
      
      	@Override
      	public void transform(TransformationInput transformationInput, TransformationOutput transformationOutput) throws StreamTransformationException {
      		try {
      			InputStream is = transformationInput.getInputPayload().getInputStream();
      			OutputStream os = transformationOutput.getOutputPayload().getOutputStream();
      
      			/**
      			 * TODO:
      			 * - copy payload to the TransformationOutput object
      			 * - retrieve token from the external server
      			 * - write the token to the header of the TransformationOutput
      			 */
      
      		} catch (Exception exception) {
      			getTrace().addDebugMessage(exception.getMessage());
      			throw new StreamTransformationException(exception.toString());
      		}
      	}
      Author's profile photo Philippe Addor
      Philippe Addor

      Hi Rajesh

       

      I’m not sure if I understand your scenario correctly. But if it’s just about sending a CSV (as the main payload in the form field called “file”), it would probably have been enough to use an adapter module to do the XML2CSV transformation of the payload.

      Ok, your solution of doing the CSV conversion in a UDF, add it as an attachment and then swap it back to the main payload is really creative and avoids writing and deploying an adapter module! But be aware that there is already an adapter module available by Eng Swee Yeoh: https://blogs.sap.com/2015/03/06/deepfccbean-the-better-fcc-at-meeting-your-deep-structure-needs-part-1-deep-xml-to-flat-file/

      Maybe it helps in the next scenario.

      Best,

      Philippe

      Author's profile photo Jonathan Ma
      Jonathan Ma

      Rajesh, thanks for the blog.

      I am struggling with the payloadswapbean. There is no documentation on how to use it. It is a rest attachment to file interface. Sender adapter is rest with attachment while receiver is file adapter. I am getting "swap no matching payload found"

      My question is: what exactly is Payload-Name and Attachment-1? Is the payload the main XML payload or is the payload the attachment on the receiving side(file adapter). What exactly does the swapbean do to compare the matching payload?

      I am using blog https://blogs.sap.com/2014/10/16/creating-file-name-from-soap-attachment1/ but it does not work.

       

      Author's profile photo Akram Habashi
      Akram Habashi

      Hi Rajesh

      Thank you for your article.

      I has a similar issue, to send CSV as an attachment within form-data as you mentioned

      However, the target is expects a JSON format.

      I used HTTP_AAE because we are in lower version in PI, REST adapter, does not have checkbox, of attachment.

      is it possible to use HTTP_AAE with JSON request ?

      Thanks

      Author's profile photo Rizwana Cheruvukatta
      Rizwana Cheruvukatta

      Hi Akram,

      I have a similar requirement to send Json payload as attachment. Could you please let me know if you have achieved this using HTTP_AAE adapter?

       

      Regards

      Rizwana.

      Author's profile photo Akram Habashi
      Akram Habashi

      hi rizwawana

       

      In PI, we were in middle of upgrade system, so we used another way.

      i have sent a CSV file as an attachment to JSON Request using Class CL_HTTP_CLIENT.

       

      Regards

      Akram

      Author's profile photo Vinit Patel
      Vinit Patel

      hi Akram,

      i am working on same requirement. i am not able to attach the file. i have the data in internal table. i want to know how to send that data as attachment using Class CL_HTTP_CLIENT.

      can you please share the logic you developed.

       

      Regards,

      Vinit Patel.

      Author's profile photo Akram Habashi
      Akram Habashi

      Hi Vinit

      this is the Code:

       

        cl_http_client=>create_by_url(   "Or By Destination
      EXPORTING
      url                lv_url    " URL
      *    ssl_id             = 'ANONYM'    " SSL Identity
      IMPORTING
      client             go_http_client    " HTTP Client Abstraction
      EXCEPTIONS
      argument_not_found 1
      plugin_not_active  2
      internal_error     3
      OTHERS             ).

      .....

        go_http_client->request->set_header_fieldname '~request_method'
      value 'POST' ).

      go_http_client->authenticate(.....

      go_http_client->request->set_header_fieldname '~request_uri'
      value iv_url )"#EC *

      CALL METHOD go_http_client->request->set_header_field
      EXPORTING
      name  '~server_protocol'
      value 'HTTP/1.0'.

      IF iv_file_type NE gc_hr_workspace_file.
      CALL METHOD go_http_client->request->if_http_entity~set_content_type
      EXPORTING
      content_type 'multipart/form-data'.
      ENDIF.

      CALL METHOD go_http_client->request->set_header_field
      EXPORTING
      name  'Accept'
      value 'text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8'.

      ......

       

        DATAlo_entity_part TYPE REF TO if_http_entity.
      DATAlv_file_name TYPE string,
      lv_len       TYPE i.
      DATAls_file_data TYPE string,
      ls_rec       TYPE string.

      lo_entity_part go_http_client->request->if_http_entity~add_multipart).

          lv_file_name 'form-data; name="sheet_file";filename="users.csv";'.

       

        CALL METHOD lo_entity_part->set_header_field
      EXPORTING
      name  'content-disposition'
      value lv_file_name.

      CALL METHOD lo_entity_part->set_content_type
      EXPORTING
      content_type 'application/text;charset=UTF-8'.

        "The whole file string separated by ',' between each value with cl_abap_char_utilities=>cr_lf. 

      lv_len strlengv_file_string ).  

      CALL METHOD lo_entity_part->set_cdata
      EXPORTING
      data   gv_file_string
      offset 0
      length lv_len.

      .......

       

        DATAlv_result TYPE string,
      lv_status TYPE i.
      DATAlt_nodes TYPE STANDARD TABLE OF ty_node.

      * Send Request
      CALL METHOD go_http_client->send
      EXCEPTIONS
      http_communication_failure 1
      http_invalid_state         2
      http_processing_failed     3
      OTHERS                     4.

      IF sy-subrc IS NOT INITIAL.
      CALL METHOD go_http_client->get_last_error
      IMPORTING
      code    lv_status
      message lv_result.

      CALL METHOD go_http_client->close.
      MESSAGE i118(swr).
      MESSAGE i000(zhr_global_dmz_mesWITH lv_result lv_status.
      RETURN.
      ENDIF.

      * Receive the Response Object
      go_http_client->receive(
      EXCEPTIONS
      http_communication_failure 1
      http_invalid_state 2
      http_processing_failed ).

      IF sy-subrc IS NOT INITIAL.
      CALL METHOD go_http_client->get_last_error
      IMPORTING
      code    lv_status
      message lv_result.

      CALL METHOD go_http_client->close.
      MESSAGE i118(swr).
      MESSAGE i000(zhr_global_dmz_mesWITH lv_result lv_status.
      RETURN.
      ENDIF.

      lv_result go_http_client->response->get_cdata).

       

      It works perfectly

      Regards

      Author's profile photo Rizwana Cheruvukatta
      Rizwana Cheruvukatta

      Hi Rajesh,

      We have a requirement of sending the json payload as attachment to Rest API. We are converting the xml payload to json and converting into attachment in java mapping and sending through REST  receiver adapter. Multipart/form-data is the content type we are using to send attachment. Support Attachment option in REST receiver channel is selected.

      we are using form data parameters file,filename,Simulation while posting from Postman and we are receiving 200 response in Postman.

      We are receiving 400 response code form the target in PI, as we are missing Filename parameter while sending attachment.

      I would like to know how to pass the form data parameters file and filename in Rest receiver channel.

       

      Regards

      Rizwana.

      Author's profile photo wang jinmu
      wang jinmu

      Hi Rizwana

      I have the same problem as you .

      I need to send a json file under the form-date tab .

      Can you tell me how do you achieved  it ?

      Regards

      Jin mu

      Author's profile photo Akram Habashi
      Akram Habashi

      Dear Rajesh

       

      Can anyone helps me with the REST PI (7.5)Receiver adapter.

      In Postman i have the following Code:

      POST /api/v4/sand/async-job HTTP/1.1
      Host: services.test.com

      Content-Length: 390
      Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW

      ----WebKitFormBoundary7MA4YWxkTrZu0gW
      Content-Disposition: form-data; name="file"; filename="/C:/Users/akramha/Desktop/result.csv"
      Content-Type: text/csv

      (data)
      ----WebKitFormBoundary7MA4YWxkTrZu0gW
      Content-Disposition: form-data; name="entity_type"

      CARS
      ----WebKitFormBoundary7MA4YWxkTrZu0gW
      Content-Disposition: form-data; name="file_format"

      CSV
      ----WebKitFormBoundary7MA4YWxkTrZu0gW

       

      I ticked the checkbox support attachments with (multiform/form-data)

      In the UDF, I am only writing the CSV file to a target element with name "file".

      AbstractTrace trace = container.getTrace();

      // field Names from your first mapping structure

      String header = "StoreNum,StoreName";
      String content = header + "\n";
      String mimeType = "application/vnd.ms-excel;charset=" + "\"" + "uft-8" + "\"";
      String fName = "test.csv";

      for (int i = 0; i < StoreNum.length; i++) {

      // adjust the below line with your field names from first structure
      content = content + StoreNum[i] + "," + StoreName[i] + "\n";
      }
      trace.addInfo(content);

      //Create attachment with CSV data
      try {

      GlobalContainer globalContainer = container.getGlobalContainer();
      OutputAttachments outputAttachments = globalContainer.getOutputAttachments();
      Attachment attachments = outputAttachments.create(fName, mimeType, content.getBytes());
      outputAttachments.setAttachment(attachments);
      } catch (Exception e) {

      e.toString();

      }

      result.addValue(fName);

      Do I need to insert anything about the boundary !!??

      In the CC module, i tried the SWAP BEAN & TRANSFER BEAN without any success, untill i noticed the boundary thing.

      Do i need to enter anything regarding the boundary in tab HTTP Headers ?

      I thought, that i need only to add these elements, (file name, entity type & file format ) only at the Module Tab !!

      can you please help ?

       

      Best Regards

      Akram

      Author's profile photo Friman Ramos
      Friman Ramos

      Hi Akram,

      Ihave a similar scenario, we get to form everything correctly when sending the attached file, but I get http 400 bad request, did you manage to solve your requirement?

      Regards,

       

      Friman.