Skip to Content
Technical Articles
Author's profile photo Muniyappan Marasamy

Test Kafka scenarios using SAP CPI

This blog will help you to set up kafka server on cloud and test it with sap cpi. Recently SAP has released a new adapter called Kafka for sap cpi to integrate with Kafka. Please note that this blog is only for learning purpose, as I have not used this in productive systems.

1. Create Kafka Instance

I did a quick google search to check if there is any kafka trial server. I was lucky enough to find one which was pretty easy to set up and use.You can create your own kafka instance by subscribing.

https://aiven.io/

Once you are done with the service setup you will get a kafka cluster instance. Please make note of kafka connection information.

You have to create your topic. For example, I have created a topic called hello_world.

2. Test Producer and consumer using Java

First I have tried to check If I can produce and consume kafka messages using java as it was straightforward which will confirm that everyhing works with kafka server setup.

Follow this link for producing and consuming message using java.

Getting started with Aiven for Apache Kafka | Aiven Help Center

1.First download key and certificates and Create keystore and trust store using openssl.

2.Generate the private key and trust store.

Now create a java project and run first producer class after that run consumer class. Once you have run the consumer class, it will keep polling the topic.

You can check the number of produced messages and consumer groups.

sample java code.

3. Connect to Kafka server using SAP CPI

1. Connection Test

You can do the connection test first before deploying your flow, to make sure that connectivity is functioning. I have uploaded the key pair(.p12)file into the keystore.

Now you can do the connection test and you should be able to connect successfully.

Download the certificates and upload it into key store.

keystore

With this we are done with connection set up.

2. Receiver scenario (SFTP to kafka)

I have created a very basic SFTP to Kafka iflow for sending messages to kafka topic. You can also make it more simple by using time with content modifier to send some data to kafka instread of sftp sender.

 

Host and private key alias, topic name.

3. Sender Scenario (Kafka to SFTP)

 

4. Testing

File in sender folder

Records are read from kafka server and stored in sftp folder.

kept sftp sender to poll the files once a minute and send the data to kafka topic. kafka test receiver flow will send the data to kafka. kafka test sender flow will read the data from akfa and create file in sftp. messages are processed successfully.

consumer group with sender iflow id is created on kafka side and polls the records.

Hope this can help you set up kafka server and test with cpi.

Reference Links

  1. https://blogs.sap.com/2021/03/16/cloud-integration-what-you-need-to-know-about-the-kafka-adapter/
  2. https://help.sap.com/viewer/368c481cd6954bdfa5d0435479fd4eaf/Cloud/en-US/3e7b9952d37c4f208f73d29366a2bc61.html

Assigned tags

      1 Comment
      You must be Logged on to comment or reply to a post.
      Author's profile photo Sunil Mavachee
      Sunil Mavachee

      Great blog Mani. Thank you for sharing relatively new info.

      Appreciate your efforts.!

      Regards,