Unix scripts in SAP PI/PO – Use cases

Dear All,

I would like to share my experience, working in Unix shell scripting in SAP PI, along with typical use cases.

A single Unix command or a sequential execution of commands, saved in a bash script file can be invoked from OS Command option in File adapter. This is an SAP provided option to leverage the powerful bash command arsenal.

1) SCP Command: SCP means Secure copy. It is used to transfer file from/to 3rd party sftp server to/from PI server.

scp sftpuser@host:<remote directory><remote filename> <PI NFS path>

2) SFTP Command: Since SCP command cannot be used to push/pull a file from a non-linux based system, the alternative command is SFTP. This command needs to be run in batch mode. Hence, we need to prepare a separate batch file, before invoking the SFTP command. Here, I have put the contents of the batch file in bold.

sftp -b <batchfile absolute filename in PI NFS server> sftpuser@sftphost

exit

sftp -b <batchfile absolute filename in PI NFS server> sftpuser@sftphost

lcd <processing directory in PI NFS server>
cd <target directory>
put <absolute filename>
exit

Here, the first instance of the execution with ‘exit’ as the batch parameter is used to check if the sftp server is available at the moment. It is a kind of ping test. PI tries to connect to the sftp server and logs out immediately thereafter. We can check the outcome based on the return code. Return code is 0 if the Connection was successful. (Return codes are checked after execution of most unix commands, before further processing is done. This is exactly the same as SY-SUBRC in ABAP).

The second instance of the command with 4 lines is the place where actually file transfer is achieved.

lcd -> local PI directory
cd -> remote directory in SFTP server
put / get – based on whether file is sent or picked.
Finally, exit is used to log out.

3) CURL Command: Used to push files to a 3rd party system, via a wide range of protocols like HTTP, FTP, IMAP, POP3, SCP, SFTP, SMTP, TELNET & FILE, directly from OS level. This is one of the most amazing bash commands.

curl –trace-ascii <absolute file name of trace file> -F <field-1> -F <field-2> -F <field-3> –max-time 900 –connect-timeout 900 -k http://<host>:<port>

Here, the parameter –trace-ascii is used to specify the location where curl command should write the trace. -F is used to emulate a form submitted via HTTP. These are optional fields, that could be used based on necessity. Then, I have put some timeout settings. Finally, we have -k parameter, which is used to push data in insecure manner (i.e. without any login required). Then, we specify the host and port details of the target server. We can further pass additional details such as user name & password, proxy server details, TLS version, based in requirement.

4) ZIP Command

zip -m -j <output folder>/$(basename -s .xml <target xml file name>).zip <receiver NFS channel path>/<target xml file name>

Here, <receiver NFS channel path>/<target xml file name> could be replaced with %F
What we are trying to accomplish here is, as the zip command picks the xml file placed by receiver channel and zips it, the base filename of zip file is set to be the same as the xml file, but extension will be changed from xml to zip. Finally, the zip file will be placed in another output folder (as per requirement). The parameters -m and -j are used to tell the command to delete the original xml file and also to not create nested folders in the output zip.

In our scenario, using payloadzip bean, we couldn’t maintain the same file name for zip file as well as the xml inside zip. Some blogs suggested creating a custom adapter module. However, we did this with unix scripting.

5) UNZIP Command

unzip <absolute filename in PI server> -d <NFS path to place the unzipped files>.

Here, -d <NFS path to place the unzipped files> is used to specify a different folder, other than the source location. Without this, the unzipped files will get extracted to the same location as the zip file.

6) MAILX Command: for sending email, along with attachment:

echo -e “Hello, \n \nThis is a test mail sent at $(date +”%m”)/$(date +”%d”)/$(date +”%Y”), from PI server using MAILX command.” | mailx -r <Sender mail id> -s “Test mail” -a <absolute file name> -c <CC email id> <To email id>

Hello, \n \nThis is a test mail sent at $(date +”%m”)/$(date +”%d”)/$(date +”%Y”), from PI server using MAILX command. -> This would be the email MIME body. The command will substitute current month/date/year.
Test mail -> This would be the email subject
absolute file name -> Here, we specify the complete file name, along with the path.
CC email id & To email id -> In this place, we specify the recipient mail ids
Sender mail id -> Sender mail id is mentioned here.

7) Base64 command: to encode / decode images, other files to plain text base64 format:

base64 SalesOrderdefect.jpeg >> encoded.txt
base64 –decode encoded.txt >> SalesOrderdefect_Output.jpeg

Here, the first base64 command is used to encode an image file and generate a base64 encoded text file. This could be sent to a target system or picked by another channel and xml tags could be added (flat file to xml), for further manipulation in PI. The second instance, which uses the –decode parameter is used to achieve just the reverse.

Further, following are some of the basic bash commands, which I am just mentioning by way of a summary.

8) Cp, mv, rm are used for copying, moving and removing the files within local server

9) Mkdir, rmdir – to create and make directory in server.

10) Echo command is used to display text is terminal / write text to a file.

11) Ne, eq for comparing strings and numbers. Ne is not equal to and eq is equal to operator.

12) touch – for creating a dummy file for testing purpose

13) Wget is another alternative for Curl, but with support for limited protocols

14) Grep – for searching text

15) Man – used in command line to see manual / documentation. Man means manual.

16) Chmod – to change folder / file permission

17) Find and replace: ${basetext//tofindtext/toreplacetext}

18) gzip / gunzip is another option similar to zip, but with some technical differences.

19) Cat is used for concatenate operation.

cat source_file-1 source_file-2 > target_file
to append the contents of file-1 & file-2 to new target file.

cat source_file-1 source_file-2 >> target_file
to append the contents of file-1 & file-2 to already existing target file.

Following are some useful tools for testing unix commands / standalone bash files:

• Powershell – Windows based bash emulator – handy only for single line commands,
• Mobaxterm – terminal client – useful to execute single line commands in PI server, by logging in to a remote session,
• Any Linux desktop version installed on Oracle virtual box – in local machine,
• Windows Subsystem for Linux (WSL).Note: There are many online emulators available. But, the shortcoming is that we cannot run file operation commands in those sites.

A few more points, before I conclude:

• A number of these functionalities can be achieved using java mapping. However, unix commands are quite easy to learn, adopt and implement – particularly for file operations. This saves a lot of development effort. Maybe java experts out here might differ.
• PI server needs to run on Linux based OS for these commands to work. Interesting, I have tried executing scripts on 3rd party server also (as against PI NFS server). For this to work, the SFTP server must be running a Linux based OS and also the user that we login must have execute permission.

Thanks for reading!