In a previous blog, our very own Jeff Wootton compared SAP HANA smart data streaming to the Apache Kafka message broker. The key takeaway of that blog post is that while there are certain similarities between the Kafka broker and HANA SDS, there is also a key difference that emphasizes the success of using these two technologies in conjunction with one another.

With just the Kafka broker, you move messages (unchanged) from a producer to a consumer. But bring smart data streaming into the picture and now you can take that raw data, apply business logic to it, transform it into meaningful and actionable information, and then send it to the consumer.

With the release of SP 12, HANA SDS now includes 10 adapters that work with the Kafka broker so you can put these two technologies to work for you. They are:

Adapter

What it does

Kafka Avro Record Input adapter

Obtains data in Avro record binary format from a Kafka server and publishes it to HANA SDS.

Kafka Avro Record Output adapter

Reads data from HANA SDS, formats it to Avro format, and writes it to a Kafka server.

Kafka CSV Input adapter

Obtains CSV data from the Kafka server and publishes it to HANA SDS.

Kafka CSV Output adapter

Reads data from HANA SDS, transforms it to CSV format, and writes it to a Kafka server.

Kafka Event XML Input adapter

Obtains event XML data from a Kafka server and publishes it to HANA SDS.

Kafka Event XML Output adapter

Reads data from HANA SDS, formats it to event XML format, and writes it to a Kafka server.

Kafka JSON Input adapter

Obtains JSON data from a Kafka server and publishes it to HANA SDS.

Kafka JSON Output adapter

Reads data from HANA SDS, formats it to JSON format, and writes it to a Kafka server.

Kafka String Input adapter

Obtains string data from a Kafka server and publishes it to HANA SDS.

Kafka String Output adapter

Reads data from HANA SDS, formats it to string format, and writes it to a Kafka server.

If you want to build a custom Kafka adapter using the adapter toolkit, you can use these new transporters and formatters:

Name

What it does

Kafka Input transporter

Receives string or Avro data from a Kafka server, and sends it to the next module configured in the adapter configuration file.

Kafka Output transporter

Obtains data from the previous module that is configured in the adapter configuration file and sends it to a Kafka server.

Avro Record to ESP formatter

Translates Avro record objects to AepRecord objects.

ESP to Avro Record formatter

Converts AepRecord objects to Avro record objects.

String to ESP column formatter

Passes a single input string to a single smart data  streaming column.

ESP Column to String formatter

Passes a single smart data streaming column to a  single input string.

The String to ESP column and ESP Column to String formatters are simple formatters with no additional properties aside from common ones to connect them from one module to the next. There are no additional details on these formatters in the documentation.

For detailed information on the rest of the new transporters and formatters, see the Transporters Currently Available from SAP and Formatters Currently Available from SAP sections in the SAP HANA Smart Data Streaming: Building Custom Adapters guide.

For detailed information on configuring and using these adapters, see the Kafka Adapter section in the SAP HANA Smart Data Streaming: Adapters Guide.

To report this post you need to login first.

Be the first to leave a comment

You must be Logged on to comment or reply to a post.

Leave a Reply