Technical Articles
CSV Multi-Part Form-Data Upload as Attachment using HTTP_AAE Adapter in SAP PI 7.5
Recently I had developed a IDOC to HTTPs Asynchronous scenario in SAP PI 7.5. There was a requirement integrating SAP ECC(client) and Amazon S3 (server) via EAI SAP PI 7.5.
In the server side, file name should be delivered as an Attachment in a csv multi form data. SAP has provided a Java method to read the attachment data and content type, content disposition, content ID, but there is no standard function to read the attachment file name.
The new Java HTTP_AAE adapter supports attachments, Multipart Documents, form submission. HTTP_AAE adapter also provides options to send HTTP Header Parameters, Query Parameters and HTTP URL parameters in the dynamic header of the PI message.
To read the form data attachment ‘name’ and ‘filename’ as Content-Disposition (content which is expected to be displayed inline in the browser, i.e. as a Web page or as part of a Web page, or as an Attachment, that is downloaded and saved locally).
Content-Type: multipart/form-data; boundary=—-WebKitFormBoundary7MA4YWxkTrZu0gW
——WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name=”file”; filename=”INV_20180119_110812.csv”
Below are the detail steps which explains in reading the API body parameters i.e. the key and value and sending the file name dynamically :
In the first graphical mapper(IDOC to XML) declare the Adapter Specific Message Attributes as an User-defined Functions for deriving the file name scheme as below and map to an field ‘fileName’.
Code:
public String getASMAFileName(String CREDAT, String CRETIM, Container container) throws StreamTransformationException{
String filename = “INV_” + CREDAT + “_” + CRETIM + “.csv”;
return filename;
1st Mapper:
In the second graphical mapper(XML to CSV) declare the Adapter Specific Message Attributes as an User-defined Functions for setting the mime type, specifying csv header fields, creating attachment as csv form data, passing the ASMA filename from first graphical mapper and header fields as an Input arguments to the user defined function and map to an field ‘file’.
Below is the standard Java method.
Code:
public void setContentType(String[] SKUId, String[] EANNumber, String[] Warehouse, String[] Quantity, String[] UOM, String[] Cost, String[] Entity, String[] TransactionType, String[] filename, ResultList rs, Container container) throws StreamTransformationException {
AbstractTrace trace = container.getTrace();
// field Names from your first mapping structure
String header = “SKUId,EANNumber,Warehouse,Quantity,UOM,Cost,Entity,TransactionType”;
String content = header + “\n”;
String mimeType = “application/vnd.ms-excel;charset=” + “\”” + “uft-8” + “\””;
String fName = filename[0];
for (int i = 0; i < SKUId.length; i++) {
// adjust the below line with your field names from first structure
content = content + SKUId[i] + “,” + EANNumber[i] + “,” + Warehouse[i] + “,” + Quantity[i] + “,” + UOM[i] + “,” + Cost[i] + “,” + Entity[i] + “,” + TransactionType[i] + “\n”;
}
trace.addInfo(content);
//Create attachment with CSV data
try {
GlobalContainer globalContainer = container.getGlobalContainer();
OutputAttachments outputAttachments = globalContainer.getOutputAttachments();
Attachment attachments = outputAttachments.create(fName, mimeType, content.getBytes());
outputAttachments.setAttachment(attachments);
} catch (Exception e) {
e.toString();
}
rs.addValue(fName);
2nd Mapper:
In the Integration Directory:
Coming to the Java HTTP_AAE Receiver communication channel, Under General tab the HTTP request details can be specified as multi-part form request and can be sent as an csv file attachment.
Tick ‘Set Form’ if the target server accepts a form-based submission. Enter the Main Payload Field Name to specify the name of the request parameter [here it is ‘file’ (name)] whose value contains the XI main payload.
Tick ‘Set Multipart’ if the HTTP request is a multipart request where one or more different sets of data form the HTTP body.
If you want the attachments in the request to be delivered to the target system, choose Keep Attachments.
Form-Based File Upload indicates that the HTTP POST request is form based and is multipart (RFC 1867). You use this scenario to select the XI message payload and XI main message attachments from a form-based file-upload.
Now Under Advanced tab, set the Additional Parameters as below it is used to submit the content type headers as ‘true’ (Incoming HTTP request content type headers).
Under Module Configuration tab, define the ‘PayloadSwapBean’ module name
(localejbs/AF_Modules/PayloadSwapBean) and declare the ‘swap.keyName’ as ‘content-type’ and ‘swap.keyValue’ as application/vnd.ms-excel;charset=”uft-8″.
PayloadSwapBean module will replace the application payload (Main Document) of the XI message that contains the business data with another payload that is appended to the XI message as an attachment. Note: Using PayloadSwapBean we cannot handle multiple attachments. This limits us to use when we are dealing with only one attachment.
swap.keyName is to enter a payload attribute such as payload-name or payload-description, or an attribute from the MIME header such as content-Disposition, content-description, and so on.
swap.keyValue is to enter the value that the attribute is to have. The MIME headers of the payloads are searched in order and the system takes the first payload that meets the criteria. This payload is swapped with the application payload.
TraceHTTP is to trace server-side HTTP communications and below are the following severity levels we can configure for parameter Trace HTTP:
Plain – The whole HTTP Request and response communication is traced(headers, body) in plain text.
Headers -Only HTTP headers are traced, both for the HTTP Request and response
Hex – A hex dump is traced for the whole HTTP request and response communication in the hexadecimal format.
This will indeed dump the entire raw HTTP message(Both request and response) to the Logs in …/server#/j2ee/cluster/dispatcher/log/services/http directory.
Do note this feature is supported only PI 7.31 SP 14 and greater, PI 7.4, 7.5 SP 9 and greater.
Debug the HTTP traces in Nwa log Viewer application
Open NWA->Troubleshooting->Logs and Traces -> Log Viewer
Amazon S3 target server:
However, in this blog, I had solved this problem by manually creating Multipart request using JAVA Mapping since REST/SOAP Adapter is not having multipart form(request) options by default. In this way you can use plain HTTP_AAE adapter. Below is the URL to understand how a multipart request looks like. Please visit this URL:
https://www.w3.org/TR/html401/interact/forms.html#h-17.13.4.2
This blog tries to explain how a multipart form data request looks like, how it can be easily generated from XI and provides java code to achieve this scenario using HTTP_AAE.
Rajesh Thank you very much for your article, I have been very helpful, I would like to ask you if this UDF or the use of the OutputAttachments class can be applied for the sending of attachments in from-data in a receiver type REST 7.5 adapter
thanks
Thanks Josmar.
Yes you can send the form data attachments using REST Adapter you will have to upgrade to latest PI package(>SP9 ). For more details please refer below link:
https://blogs.sap.com/2019/04/30/rest-adapter-from-data-with-attachment-handling/comment-page-1/#comment-458661
thanks Rajesh!
You're Welcome!
Hi Rajesh, I'm sending an xml in the front-data but in the log it appears that after attaching the document it takes the xml of the mapping, as if it were superimposing it:
this is img from transformation 1 where this is transformating:
then this is overlaying the mapping xml:
So, I would like to know what I'm doing, I may not be doing well, I think it could be on the http channel because with the PayloadSwapBean, I should take what I'm loading in the set, right?
Thanks for the help
Hi Josmar,
Could you please explain me your scenario. Is it iDoC to CSV forms?
the scenario is proxy to rest, where the rest API receives an XML in the from-data
thanks
HI Josmar,
On seeing the error which states HTTP 403 Forbidden , it is an issue with the content and body params,
Requesting you to check once again the mappers and channel configuration patiently.
Before that execute once in POSTMAN take the code snippets and compare with the Http Traces generated in PI log viewer.
Regards - Rajesh PS
Hi Josmar,
Yes, please check with the PayloadSwapBean. The content disposition is not appropriate.
Ideally it is not replacing the application payload (Main Document) of the XI message that contains the business data with another payload that is appended to the XI message as an attachment.
Hi Rajesh.
Attached below the PayloadSwapBean configuration of the adapter:
and this is what the log shows me:
Regards – Josmar Alarcon
Hi Josmar,
Please check in Advanced tab, set the Additional Parameters as mentioned below
preferredContentTypeHeader as true.
Also set Form and mention Main Payload Field Name and as well select Multipart.
Hi Rajesh
I tell you what worked but now I have a problem when sending a dynamic variable to send a dynamic token, you have some idea of how to do it from an HTTP_AAE adapter since a REST adapter is not taking me the configurations you named previously .
Thank you very much for your help.
Regards – Josmar Alarcon
Hi Josmar,
You can accomplish by using User defined function in graph mapper and map it root node appropriately.
What kind of token is it?
https://blogs.sap.com/2017/07/05/api-token-via-http-lookup-in-adapter-module/
https://blogs.sap.com/2014/12/18/pi-rest-adapter-using-dynamic-attributes/
Code looks like below:
Hi Rajesh
I’m not sure if I understand your scenario correctly. But if it’s just about sending a CSV (as the main payload in the form field called “file”), it would probably have been enough to use an adapter module to do the XML2CSV transformation of the payload.
Ok, your solution of doing the CSV conversion in a UDF, add it as an attachment and then swap it back to the main payload is really creative and avoids writing and deploying an adapter module! But be aware that there is already an adapter module available by Eng Swee Yeoh: https://blogs.sap.com/2015/03/06/deepfccbean-the-better-fcc-at-meeting-your-deep-structure-needs-part-1-deep-xml-to-flat-file/
Maybe it helps in the next scenario.
Best,
Philippe
Rajesh, thanks for the blog.
I am struggling with the payloadswapbean. There is no documentation on how to use it. It is a rest attachment to file interface. Sender adapter is rest with attachment while receiver is file adapter. I am getting "swap no matching payload found"
My question is: what exactly is Payload-Name and Attachment-1? Is the payload the main XML payload or is the payload the attachment on the receiving side(file adapter). What exactly does the swapbean do to compare the matching payload?
I am using blog https://blogs.sap.com/2014/10/16/creating-file-name-from-soap-attachment1/ but it does not work.
Hi Rajesh
Thank you for your article.
I has a similar issue, to send CSV as an attachment within form-data as you mentioned
However, the target is expects a JSON format.
I used HTTP_AAE because we are in lower version in PI, REST adapter, does not have checkbox, of attachment.
is it possible to use HTTP_AAE with JSON request ?
Thanks
Hi Akram,
I have a similar requirement to send Json payload as attachment. Could you please let me know if you have achieved this using HTTP_AAE adapter?
Regards
Rizwana.
hi rizwawana
In PI, we were in middle of upgrade system, so we used another way.
i have sent a CSV file as an attachment to JSON Request using Class CL_HTTP_CLIENT.
Regards
Akram
hi Akram,
i am working on same requirement. i am not able to attach the file. i have the data in internal table. i want to know how to send that data as attachment using Class CL_HTTP_CLIENT.
can you please share the logic you developed.
Regards,
Vinit Patel.
Hi Vinit
this is the Code:
cl_http_client=>create_by_url( "Or By Destination
EXPORTING
url = lv_url " URL
* ssl_id = 'ANONYM' " SSL Identity
IMPORTING
client = go_http_client " HTTP Client Abstraction
EXCEPTIONS
argument_not_found = 1
plugin_not_active = 2
internal_error = 3
OTHERS = 4 ).
.....
go_http_client->request->set_header_field( name = '~request_method'
value = 'POST' ).
go_http_client->authenticate(.....
go_http_client->request->set_header_field( name = '~request_uri'
value = iv_url ). "#EC *
CALL METHOD go_http_client->request->set_header_field
EXPORTING
name = '~server_protocol'
value = 'HTTP/1.0'.
IF iv_file_type NE gc_hr_workspace_file.
CALL METHOD go_http_client->request->if_http_entity~set_content_type
EXPORTING
content_type = 'multipart/form-data'.
ENDIF.
CALL METHOD go_http_client->request->set_header_field
EXPORTING
name = 'Accept'
value = 'text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8'.
......
DATA: lo_entity_part TYPE REF TO if_http_entity.
DATA: lv_file_name TYPE string,
lv_len TYPE i.
DATA: ls_file_data TYPE string,
ls_rec TYPE string.
lo_entity_part = go_http_client->request->if_http_entity~add_multipart( ).
lv_file_name = 'form-data; name="sheet_file";filename="users.csv";'.
CALL METHOD lo_entity_part->set_header_field
EXPORTING
name = 'content-disposition'
value = lv_file_name.
CALL METHOD lo_entity_part->set_content_type
EXPORTING
content_type = 'application/text;charset=UTF-8'.
"The whole file string separated by ',' between each value with cl_abap_char_utilities=>cr_lf.
lv_len = strlen( gv_file_string ).
CALL METHOD lo_entity_part->set_cdata
EXPORTING
data = gv_file_string
offset = 0
length = lv_len.
.......
DATA: lv_result TYPE string,
lv_status TYPE i.
DATA: lt_nodes TYPE STANDARD TABLE OF ty_node.
* Send Request
CALL METHOD go_http_client->send
EXCEPTIONS
http_communication_failure = 1
http_invalid_state = 2
http_processing_failed = 3
OTHERS = 4.
IF sy-subrc IS NOT INITIAL.
CALL METHOD go_http_client->get_last_error
IMPORTING
code = lv_status
message = lv_result.
CALL METHOD go_http_client->close.
MESSAGE i118(swr).
MESSAGE i000(zhr_global_dmz_mes) WITH lv_result lv_status.
RETURN.
ENDIF.
* Receive the Response Object
go_http_client->receive(
EXCEPTIONS
http_communication_failure = 1
http_invalid_state = 2
http_processing_failed = 3 ).
IF sy-subrc IS NOT INITIAL.
CALL METHOD go_http_client->get_last_error
IMPORTING
code = lv_status
message = lv_result.
CALL METHOD go_http_client->close.
MESSAGE i118(swr).
MESSAGE i000(zhr_global_dmz_mes) WITH lv_result lv_status.
RETURN.
ENDIF.
lv_result = go_http_client->response->get_cdata( ).
It works perfectly
Regards
Hi Rajesh,
We have a requirement of sending the json payload as attachment to Rest API. We are converting the xml payload to json and converting into attachment in java mapping and sending through REST receiver adapter. Multipart/form-data is the content type we are using to send attachment. Support Attachment option in REST receiver channel is selected.
we are using form data parameters file,filename,Simulation while posting from Postman and we are receiving 200 response in Postman.
We are receiving 400 response code form the target in PI, as we are missing Filename parameter while sending attachment.
I would like to know how to pass the form data parameters file and filename in Rest receiver channel.
Regards
Rizwana.
Hi Rizwana
I have the same problem as you .
I need to send a json file under the form-date tab .
Can you tell me how do you achieved it ?
Regards
Jin mu
Dear Rajesh
Can anyone helps me with the REST PI (7.5)Receiver adapter.
In Postman i have the following Code:
POST /api/v4/sand/async-job HTTP/1.1
Host: services.test.com
Content-Length: 390
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW
----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="file"; filename="/C:/Users/akramha/Desktop/result.csv"
Content-Type: text/csv
(data)
----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="entity_type"
CARS
----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="file_format"
CSV
----WebKitFormBoundary7MA4YWxkTrZu0gW
I ticked the checkbox support attachments with (multiform/form-data)
In the UDF, I am only writing the CSV file to a target element with name "file".
AbstractTrace trace = container.getTrace();
// field Names from your first mapping structure
String header = "StoreNum,StoreName";
String content = header + "\n";
String mimeType = "application/vnd.ms-excel;charset=" + "\"" + "uft-8" + "\"";
String fName = "test.csv";
for (int i = 0; i < StoreNum.length; i++) {
// adjust the below line with your field names from first structure
content = content + StoreNum[i] + "," + StoreName[i] + "\n";
}
trace.addInfo(content);
//Create attachment with CSV data
try {
GlobalContainer globalContainer = container.getGlobalContainer();
OutputAttachments outputAttachments = globalContainer.getOutputAttachments();
Attachment attachments = outputAttachments.create(fName, mimeType, content.getBytes());
outputAttachments.setAttachment(attachments);
} catch (Exception e) {
e.toString();
}
result.addValue(fName);
Do I need to insert anything about the boundary !!??
In the CC module, i tried the SWAP BEAN & TRANSFER BEAN without any success, untill i noticed the boundary thing.
Do i need to enter anything regarding the boundary in tab HTTP Headers ?
I thought, that i need only to add these elements, (file name, entity type & file format ) only at the Module Tab !!
can you please help ?
Best Regards
Akram
Hi Akram,
Ihave a similar scenario, we get to form everything correctly when sending the attached file, but I get http 400 bad request, did you manage to solve your requirement?
Regards,
Friman.