Skip to Content
Technical Articles
Author's profile photo Lukas Holthof

Cloud Integration – Connecting to FTP(S)-servers using the FTP Adapter

This blog post describes how to configure the connection and processing settings of the FTP sender and receiver adapter. This adapter will be available for SAP Cloud Integration customers with the 04-July-2020 release. This blog post describes the basic concepts as well as the configuration options.

Introduction to the FTP Adapter

The File Transfer Protocol was initially specified in 1971 and although encryption capabilities have been added to the specification in the meantime, the FTP/FTPS protocol is rated less secure than the SFTP protocol. Therefore, we recommend you to select the SFTP protocol as your first choice when setting up a new scenario which SAP Process Integration has been supporting with a dedicated SFTP Adapter for years. The new FTP Adapter should be used only for those scenarios in which you already have an existing FTP Server which you cannot connect to with SFTP.

In contrast to other protocols FTP uses two connections on two different ports between the client and the server, the control connection and the data connection. The connection mode specifies how the data connection is established. There are two different modes: active mode and passive mode. SAP Cloud Platform’s FTP adapter only supports passive mode.

FTP uses a control connection and a data connection

In the passive mode, the FTP adapter (which is always the client) sends a EPSV/PASV command to the server in order to receive the server’s data connection port. This port is within a port range specified on the server. The FTP server administrator should be able to lookup this range which will be required once you want to configure SAP Cloud Connector. For details regarding the Cloud Connector configuration see section “Cloud Connector Configuration” below.

The data transferred over the network can be of different types. The FTP adapter supports Binary and by default the FTP Adapter uses streaming.

The authentication is done either by username/password or by anonymous login. For both options the Integration Developer maintains a credential name in the adapter. The credential itself has to be created in the Security Material monitor as a User Credential artifact. This User Credential artifact contains the username and the password. For anonymous login use username = “anonymous” and an empty password. But this might deviate based on the requirements of your FTP server.

Transport Security can be achieved by setting an encryption parameter. If you choose “Plain FTP – no encryption” all transmission will be done in clear text. The default port for the control connection is 21. Two options are available to encrypt the communication:

  • Explicit FTPS: After the initial connection the client sends a “AUTH TLS” command to the server which initiates a TLS handshake (the FTP server must support TLS). The subsequent communication is encrypted. The default port for the control connection is 21.
  • Implicit FTPS: The connection to the FTP server is established as a TLS connection. In this case, usually a different port for the control connection is used, default 990.

Once you have selected an FTPS encryption, the server certificate needs to be trusted by SAP Cloud Integration. Ensure that you have uploaded the intermediate and root certificate to the SAP Cloud Integration keystore. Please check Manage Keystore Entries for how to upload certificates to the keystore. One option to get the server certificate and its chain is to execute the FTP Connectivity Test and download the files from the test result.

The FTP adapter performs a host verification which ensures that the control and data connection share the same host. You cannot have two different hosts.

Session reuse between control and data connection is only supported by the FTP adapter for connections using the SAP Cloud Connector.

 

FTP Sender Adapter: Setup & Examples

The FTP sender adapter can be used to read files from an FTP server. It is a polling adapter which means that it is scheduled to look for files once or periodically, according to a predefined pattern.

Find detailed information about all possible parameters in the Configure the FTP Sender Adapter documentation.

The following screenshots demonstrate a sample Integration Flow that reads all .txt-files from a /sender directory using explicit encryption and connecting via SAP Cloud Connector.

The Source tab contains general connection and file access parameters.

FTP%20Sender%20-%20Source%20Tab

FTP Sender – Source Tab

The Processing tab contains more advanced processing parameters. For example, the Read Lock Strategy ensures that the FTP adapter only polls the files if it has an exclusive read-lock on the file. Or the Post-Processing option which defines what should happen to the file after it has been processed successfully.

FTP%20Sender%20-%20Processing%20Tab

FTP Sender – Processing Tab

The Schedule tab allows you to specify when and how often each of your runtime nodes should perform a poll attempt.

FTP%20Sender%20-%20Schedule%20Tab

FTP Sender – Schedule Tab

 

FTP Receiver Adapter: Setup & Examples

The FTP receiver adapter can be used to write files to an FTP server. Either you use the FTP Receiver adapter together with an End Message step or a Send step.

Find detailed information about all possible parameters in the Configure the FTP Receiver Adapter documentation.

The following screenshots demonstrate a sample Integration Flow that writes a file.txt into directory /sendstep using explicit encryption and connecting via SAP Cloud Connector.

The Target tab contains general connection parameters as well as the directory and file name to be written.

FTP%20Receiver%20-%20Target%20Tab

FTP Receiver – Target Tab

The Processing tab contains more advanced processing parameters. For example, the Handling for Existing Files is set to Append in the sample which will add the Integration Flow’s payload to the end of an existing file.

FTP%20Receiver%20-%20Processing%20Tab

FTP Receiver – Processing Tab

 

Dynamic Setting of FTP Receiver Adapter Parameters

The FTP receiver adapter supports a wide range of dynamic configuration possibilities. The dynamic configuration can be used, for example, once you require to connect to different FTP servers from the same Integration Flow.

Text-based fields can contain dynamic input. You can configure the entry fields DirectoryFile NameAddressLocation ID and Credential Name dynamically by using a header (${header.abc}) or a property (${property.abc}) as shown below.

FTP%20Receiver%20-%20Dynamic%20Text%20Fields

FTP Receiver – Dynamic Text Fields

As shown in the screenshot above, the drop-down field Proxy Type has the option Dynamic.

If you choose Dynamic, you have to define the actual value in the respective SAP property.

Attribute					|SAP property			|Type				|Values
----------------------------|-----------------------|-------------------|---------------------------------------
Proxy Type					|SAP_FtpProxyType		|String				|‘internet’ and ‘onPremise’

If the property is not defined during runtime, an error is thrown.

Checkboxes, additional drop-down lists and integer fields are configurable dynamically if you define the values in predefined SAP properties. The following table shows the names of the properties for the different configuration options:

Attribute					|SAP property			|Type				|Values
----------------------------|-----------------------|-------------------|---------------------------------------
Timeout						|SAP_FtpTimeout			|int				|Values of type integer
Max. Reconnect Attempts		|SAP_FtpMaxReconnect	|int				|Values of type integer
Reconnect Delay				|SAP_FtpMaxReconDelay	|int				|Values of type integer
Automatically Disconnect	|SAP_FtpDisconnect		|boolean, string	|‘true’, ‘false’
Change Directories Stepwise	|SAP_FtpStepwise		|boolean, string	|‘true’, ‘false’
Create Directories			|SAP_FtpCreateDir		|boolean, string	|‘true’, ‘false’
Handling for Existing Files	|SAP_FtpAfterProc		|String				|‘Overwrite’, ‘Append’, ‘Fail’, ‘Ignore’
Flatten Filenames			|SAP_FtpFlattenFileName	|boolean, string	|‘true’, ‘false’

If a property is not set, the runtime uses the value defined in the channel.

 

FTP and SAP Cloud Connector Configuration

Most scenarios using the FTP adapter use the SAP Cloud Connector to establish the connection to corporate FTP servers. This allows you to connect via TCP to your on-premise FTP server, so no incoming ports have to be opened in your firewall which is often not allowed by your security policy. Refer to the Cloud Connector documentation for general setup details. The FTP adapter’s on-premise connectivity utilizes the SOCKS5 proxy supported in SAP Cloud Connector version 2.10 and higher.

NoteIf you have not already installed SAP Cloud Connector and connected it to your SAP Cloud platform account, with which you subscribed to SAP Cloud Integration, download SAP Cloud Connector from  SAP Development Tools page and follow its installation documentation.

The following information covers FTP adapter specifics only. If you are using encryption (FTPS), implicit or explicit, you must configure the Virtual Host equal to the Internal Host. This is required for a hostname verification to be successful which ensures that the server certificate is issued for the hostname it connects to. Otherwise the Integration Flow fails during runtime.

A sample configuration is shown in the following screenshot.

Usually, the Cloud Connector setup for a single server requires just a single entry in the “Cloud To On-Premise -> ACCESS CONTROL” screen. For the FTP connectivity, one entry for the control connection and one entry per port within the data connection port range is required. As mentioned above the FTP server administrator should provide this port range. As of now, the Cloud Connector does not support maintenance of ranges. Therefore, we require one entry per port which might require a lot of maintenance effort.

To mitigate this effort, SAP Cloud Connector offers two ways which are more convenient for mass creation of Access Control entries than creating them manually on the Administration page. The two options are:

  • SAP Cloud Connector API to create the Access Control entries
  • Import Feature for Access Control entries

The SAP Cloud Connector API to create the Access Control entries allows you to create a single entry with a HTTP POST request. So you would require some kind of automation for sending those POST request for a large number of ports. A single sample request looks like this, be aware to replace all masked values (“< >”). Refer to the API documentation for details.

curl --request POST \
  --url https://<SCC_host_and_port>/api/v1/configuration/subaccounts/<region_url>/<subaccount>/systemMappings \
  --header 'authorization: Basic <base64_username_password>' \
  --header 'content-type: application/json' \
  --header 'x-csrf-token: <cvrf_token>' \
  --data '{
    "virtualHost": "ftp-server.sample",
    "virtualPort": "55001",
    "localHost": "ftp-server.sample",
    "localPort": "55001",
    "protocol": "TCP",
    "backendType": "nonSAPsys"
}'

The other option is the import functionality on the administration screen for Access Control which accelerates the creation of a lot of entries. For this option, you would have to prepare one json file containing a backend entity per port. If you download one of your entries created manually, you get a template to fill out. A sample for possible content is shown below:

{
	"backends": [{
			"cloudHost": "ftp-server.sample",
			"cloudPort": "55001",
			"localHost": "ftp-server.sample",
			"localPort": "55001",
			"protocol": "TCP",
			"backendType": "nonSAPsys",
			"hostInHeader": "virtual",
			"authMode": "NONE",
			"resources": []
		}, {
			"cloudHost": "ftp-server.sample",
			"cloudPort": "55000",
			"localHost": "ftp-server.sample",
			"localPort": "55000",
			"protocol": "TCP",
			"backendType": "nonSAPsys",
			"hostInHeader": "virtual",
			"authMode": "NONE",
			"resources": []
		}, ...
	]
}

Once you established the connection successfully, be aware that the SAP Cloud Connector interrupts the connection after a timeout. We recommend to set the Automatically Disconnect option inside the FTP Adapter to true for connections using Proxy Type On-Premise.

Troubleshooting

For any connectivity issue I recommend checking the FTP Connectivity test first. The connectivity test will show you meaningful error messages. In addition it offers you the possibility to download the server certificate or to check your user permissions to the FTP server’s folders. Please check out the documentation for FTP Connectivity Tests.

FTP%20Connectivity%20Test%20-%20Error%20Message

FTP Connectivity Test – Error Message

The above screenshot is an example for an error message. This one is indicating that there is a Access Control entry missing for ftp-server.sample:21. The SAP Cloud Connector offers comprehensive logging as well, and this message is visible as well in the Audits screen with an entry: “Denying access to system ftp-server.sample:21”. In this case adding the required access control entry would fix the issue.

 

Conclusion

I hope this blog post helps you to understand the basic concepts of FTP/FTPS and enables you to smoothly configure the SAP Cloud Integration FTP Adapter. Besides certain limitations mentioned in the text, examples are shown for sender and receiver, I also described the dynamic configuration features as well as the SAP Cloud Connector configuration.

Assigned Tags

      11 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Yogananda Muthaiah
      Yogananda Muthaiah

      Nice blog and keep sharing a new way of working in CPI

      Author's profile photo Mohit Singhal
      Mohit Singhal

      Hello Lucas,

      My customer's S/4 HANA system is hosted in HEC and they have lot of restrictions to allow ftps access on port 21. Can we I use sftp adapter in sender side and ftps adapter on the receiver side. I am not sure whether this kind of scenario will work where both the parties are not willing to change the ftp protocols nor want to provide the access.

      Regards

      Mohit

      Author's profile photo Lukas Holthof
      Lukas Holthof
      Blog Post Author

      Hello Mohit,

      the adapters are independent from each other, so you can combine them as you like.

      Regards, Lukas

      Author's profile photo Mohit Singhal
      Mohit Singhal

      Hello Lucas,

      Thanks a lot for the feedback. I am able to configure the iflow in CPI using FTPS adapter as receiver. But when I setup the iflow with FTP/S adapter as a sender using internet , I get the error message that control IP and Data ip addresses are different when I use the default port 21.

       

      When I put data port explicitly , then I receive the below error message.

      nload=true&throwExceptionOnConnectFailed=true failed.   [CAMEL][IFLOW][EXCEPTION] : org.apache.camel.component.file.GenericFileOperationFailedException: File operation failed: Connection reset. Code: 0     [CAMEL][IFLOW][CAUSE] : Cause: java.net.SocketException: Connection reset

      I am try to read the encrypted files from the source , decrypt them and upload them into S4. The integration between S/4 and CPI is working as expected by using SAP CC.

      Please suggest , if this scenario is supported yet

      Regards

      Mohit

      Author's profile photo Lukas Holthof
      Lukas Holthof
      Blog Post Author

      Hi Mohit,

      the FTP Adapter does not support different addresses between control and data connection. Usually you can control on FTP Server side which hostname/IP is propagated as a response to a PASV/EPSV command. Please ensure this is the same you initiate the connection to.

      I am not getting what "When I put data port explicitly" means. Did you put it as part of the address? This will not work. The control port (usually port 21) has to be used to initiate the connection. The data port will be defined by the server as a response to a PASV/EPSV command initialed by the client (CPI) after the connection has been established.

      Regards, Lukas

      Author's profile photo Marek Zikmunda
      Marek Zikmunda

      Hello Lucas,

      thank you for this blog, I have though a question related to troubleshooting you mentioned here, we are facing same issues as per as your printscreen, though Access control is in place and TCP 21 reachable (+FTP accessible from server of CloudConnector) :

      Error Details
      org.apache.camel.component.file.GenericFileOperationFailedException: File operation failed: 227 Entering Passive Mode (XX,XXX,XXX,XX,199,35) Proxy error opening tunnel: Connection not allowed by ruleset. Check that a mapping to the target system is configured in Cloud Connector.. Code: 227, cause: java.io.IOException: Proxy error opening tunnel: Connection not allowed by ruleset. Check that a mapping to the target system is configured in Cloud Connector.
      Connectivity test is successful though when checking the option : Check Directory Access we are getting the error (as per as above error log from iflow). We are on 21 (FTPES / FTP) – Plain FTP, are there any specific configuration? SFTP(s) works fine…
      Regards,
      Marek
      Author's profile photo Lukas Holthof
      Lukas Holthof
      Blog Post Author

      Hi Marek,

      I guess you have missed some access control entries in SAP Cloud Connector. Please check if you have an entry for your port in the mentioned sample (XX,XXX,XXX,XX,199,35) -> 199*256+35 = 50979.

      You will also be able to find the denied request in SCC Audits View.

      For your reference, I created a Consulting Note for this: https://launchpad.support.sap.com/#/notes/2956711

      Regards, Lukas

      Author's profile photo rahul gupta
      rahul gupta

      Hi Lukas ,

      The note is not currently getting opened

      Regards

      Rahul

      Author's profile photo Lukas Holthof
      Lukas Holthof
      Blog Post Author

      Hi Rahul,

      please check again, it took a while until the note was released. It should be visible to you now.

      Regards, Lukas

      Author's profile photo Maik Offerle
      Maik Offerle

      It's working now 🙂 Thank you very much Lukas Holthof !

      Author's profile photo Madan Agrawal
      Madan Agrawal

      Hi,

       

      We are have added the port range 10000-65000 in Cloud connector and FTPS IIS server but we are facing an issue with error as 229 entering passive mode connection timed out. Please suggest what could be the issue.

       

      Error in CPI:

      org.apache.camel.component.file.GenericFileOperationFailedException: File operation failed: 229 Entering Extended Passive Mode (|||52131|)
      Read timed out (local port 50586 to address 127.0.0.1 (localhost), remote port 20004 to address 127.0.0.1 (localhost)). Code: 229, cause: java.net.SocketTimeoutException: Read timed out (local port 50586 to address 127.0.0.1 (localhost), remote port 20004 to address 127.0.0.1 (localhost))
      Error in Cloud Connector logs:
      ERROR#com.sap.core.connectivity.spi.processing.AbstractProtocolProcessor#tunnel-client-12-10#0xa4844ccc#Discarding packet as connection to backend has been unexpectedly closed
      ERROR#com.sap.core.connectivity.spi.processing.AbstractProtocolProcessor#tunnel-client-12-10#0x339c63e3#Unable to open connection to backend system: connection timed out: ftpserver/IPaddress:62766 io.netty.channel.ConnectTimeoutException: connection timed out: ftpserver/IPaddress:62766 at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:261) at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98)
      Thanks.