Skip to Content

Background

Just recently I was asked how to realize an automated file transfer between an on-premise system (e.g. SAP ERP) and an external business partner with an increased level of security including:

  1. Sending files over SFTP (FTP over SSH) to a FTP server of a business partner – so “security in transit”
  2. Authentication on the FTP server using certificate (instead of user/password).
  3. PGP file encryption – so “security at rest”. File signing was not required, but is supported as well

Since these security measures are all standard, there are numerous ways to realize this (scripts, MFT toolsets, etc.). Since we have NetWeaver PI already in place for 100+ integration scenarios, I did setup a such a case to verify if this scenario is sufficiently supported using the PI SFTP adapter and PGP module released in 2012.

I found plenty of blogs on SCN covering each individual aspect which helped me a lot to built up this scenario in a very short time. This blog shall combine the hard facts on the PI configuration required to achive such a secure file transfer setup and should be understandable by non-PI experts too.

Solution

To illustrate how PI fits in here, I used two UML/TAM diagrams.

Outbound – we are sending data in files to the business partner.

ERP triggers the file transfer to the FTP server. Depending on the inbound consumption schedule business partners’s side the target application data store is updated with a short time delay.

Flow outbound.jpg

Inbound – we are receiving data in files from our business partner.

Business partner’s application either push out new data in files right after the transaction is done or periodically (e.g. once a day). PI polls for new files periodically. If a new file is present, file is read and the content handed over to ERP.

Flow inbound.jpg

The asterisks indicate the required keys (either SSH or PGP) both parties need to have in order to authenticate against the FTP server and encrypt/decrypt files.

Side remark: File signing is also supported by the PGP module, but was not used in this particular scenario. In case of signing, also the other PGP key would be necessary, e.g. for sending files the own private key would be used for signing on top of the partners public key required for encryption.

Technical Details

Important aspects for implementation in PI

To realize such a scenario with PI, a few important aspects need to be considered here (If you are non-PI person and you are still reading this blog – RESPECT! – Now lets come to the beef):

  1. FTP logon using public key authentication: This is more secure than basic authentication using userid and password. To do such a thing, the business partner hosting the FTP service needs a SSH public key from you which the business partner will place inside the FTP account folder structure. This key has to match the private key the PI server later uses to logon. To get such a thing created, you need to use the SAP NW AS Java Key Store to create a key file. Since this cannot be used by a ssh server right away, you have to convert it using several open source tools. I found two options to do this: Option 1 and Option 2. I was successful with Option 2. The logon credentials in the PI communication channel then need to refer to the entry in the AS Java Key Store.
  2. PGP file cryptography: the PI PGP module can not only encrypt and decrypt files but also sign them. This is nicely explained in two blogs – one for encryption and one for decryption. Also see SAP Help Portal for more info.
  3. File naming: Since meaningful file names (after encryption) have to be like employees20130628.csv.gpg and the PI feature to place timestamps is only able to put it next to the last file extension and is probably to detailed (would look like employees.csv20130628-164411-969.gpg) you have to use Dynamic Configuration (DC) combined with variable substition. There is a great blog describing how you freely define parts of the file name, like in my case to set a suitable timestamp at exactly the right location in the file name.
  4. File Content Conversion (FCC): FCC as own settings tab is not available in SFTP adapter and therefore has to be done with using MessageTransformBean. Simple conversions using MessageTransformBean are nicely explained in the SAP Help Portal. On top of this, the execution of MessageTransformBean has to be embedded between Dynamic Configuration and Variable substitution.

Adapter Module Sequence

So the sequence to place files on a FTP server with SSH and PGP encryption using a PI communication channel is like this:
1) DynamicConfigurationBean: Read the timestamp from XI header and place it in a variable. Has to be done before step 2 otherwise the XI header is no longer accessible.
2) MessageTransformBean: Convert XML into a flat file.
3) PGPEncryption: The file stream is encrypted according the the PGP config (e.g. ASCII armored).
4) CallSAPAdapter (ModuleProcessorExitBean for SFTP): Finally the file name is put together (using variable substitution) with a meaningful timestamp and two file extensions like .csv.gpg added at the end and the encrypted file is transferred via SSH.

For the other direction it is exactly the other way around – except that Dynamic Configuration might not be required – depends on the case.I am considering to capture the filename of the files picked up and write it to the application log on in the target ERP system.

Benefit for IT and LOBs

By looking at all of the specifics above what you need to consider while realizing this with PI, you might be thinking “what an effort to do such a simple thing”. Compared to hard-coded scripts on top of PGP and SFTP – which can be triggered from ABAP – this looks very complicated. It is for the first time, as with many things you do for the first time. So what is the benefit?

On a technical level, there is never any file written on a (potentially less secure file share) neither in unprotected or protected format. Therefore such a file cannot be read, altered or used as a basis for an additional file placement. The sending application triggers PI to create and place a file on the business partner’s side in an automated way. This approach leaves little options to read or forge the data.

This means in case of very sensitive and critical data (e.g. payroll) it is imperative to reduce the risk of data manipulation. Even if manipulation still would happen, it better have to be re-traceble when and who made the change. PI supports both of these requirements.

Next Steps

So whats next? I am thinking of documenting this step by step with screenshots. Please comment if you like to see that on SCN.

To report this post you need to login first.

3 Comments

You must be Logged on to comment or reply to a post.

  1. Christian Braukmueller

    Hi Wolfgang,

    yes, i like that already and i’m curious for further details about the encryption part

    One interesting and very basic aspect of the inbound scenario is how you can catch only the newest files.
    Or better:   How to get only the files that you didn’t already got?

    You may rename/remove them after you ensured that you got the file and possibly were able to process it on your side.   ( That’s probably a similar agreement you already made with your partner system: 

       “How can i see that the file in the folder is ready to be taken?” (and not still be written by the partner)

        You might do something like the browsers do when doing a download of a file:

               Write <filename>.tmp and when you are done: rename it to <filename>

    I’m curious if these evergreens of file transfer are easy to manage the way you are doing your example scenario.

    But i won’t push you in a direction away from the encryption.  🙂   Plz, go on.

    Christian

    (0) 
    1. Wolfgang Hegmann Post author

      Hi Christian,

      thanx for your comment.

      1. How to get only the files that you didn’t already got? The default is to delete the file once PI successfully picked it up. Alternatively, PI archives it either on the remote SFTP server or on a local NFS share (see tab “Advanced”). AFAIK PI can also leave the file where it is and just remembers the file that it got processed already – not 100% sure about this one.
      2. How can i see that the file in the folder is ready to be taken? That is tricky if the outbound file creator is NOT using temporary filenames. But: PI has the feature to wait some time if it detects that the file is still being written. This is out of the box switched on. So instead of picking up the file, PI waits for 200 ms if the file size changes. If not, file is processed. Waiting time can be configured with “Interval (ms) for modification check”.

      For the PGP part the blogs I referenced above from Shabarish Vijayakumar are complete and very good to understand. I’ll see when I can write a part 2 of this blog.

      Best regards,

      Wolfgang

      (0) 
  2. Ramon Ruh

    Hi Wolfgang,

    This is great news. With this model, lots os FTPs can be done via PI. It will also show that companies can get out of the old way to do FTPs where there were a lot of restictions and poor visibility on monitoring and error control. Great that you put all this together!!

    And yes, the more we know about this, the better!

    Regards,

    Ramon

    (0) 

Leave a Reply