Skip to Content

Highlights of the HANA smart data streaming (SDS) SPS11 release:

  • Introduction of Machine Learning Algorithms for stream processing
  • Support for HANA System Replication
    • SPS11 introduces SDS support for HANA System Replication
    • SDS nodes will be created in the replica system
    • Deployed SDS projects will be copied to the replica
    • Streaming projects will require a manual restart to allow coordination with outside data sources
    • State information for running projects including log files for log backed windows are not replicated in SPS11. Support for this functionality is on the roadmap for a future release
  • Adapter Enhancements
    • Simplified Configuration of the SAP HANA Output Adapter
      • Older options that explicitly set the commit interval, number of events to be committed in a single batch and the number of worker threads to be used have been replaced by new parameters that set upper limits for each of these values that the adapter will then work within
      • The total number of parameters that can be set for the HANA Output Adapter has been reduced. The adapter now automatically adjusts the insert array size, the max queue size and the maximum length of time to buffer data before outputting it to HANA.
    • Custom Header Protection Support for the HTTP Client Output Adapter
      • This is an important new feature for streaming projects that integrate to an XS application. XS application enforce the use of a Cross-Site Request Forgery (CSRF) token.
      • With this feature, the HTTP Client Output Adapter can now automatically request and use a CSRF token
    • New Custom Adapter File Directory
      • SDS supports the development, integration and use of custom adapters to provide integration options to data sources not already supported by our ‘out-of-the-box’ adapters
      • Custom adapters built using the Adapter Toolkit can be integrated directly into the palette in the SAP HANA Streaming Development perspective in HANA Studio
      • To provide better organization of custom adapters, the custom adapter files are now stored in the STREAMING_CUSTOM_ADAPTERS_HOME directory
  • Streaming Web Service (SWS) Enhancements
    • In SPS 10, the SWS is configured, started and stopped as a separate process. While it was often run on the same host as the SDS server node, it still had to be manually started and stopped.
    • With SPS 11, the SWS is now managed as part of the SDS cluster and can be configured to automatically start when the SDS cluster starts
    • If you are upgrading from SPS10, then there are some one time manual migration steps required to migrate your current SWS settings from the old configuration file into HANA Cockpit
  • Data Service Enhancements
    • This is a security enhancement that provides greater control over who has access to specific data services. Prior to SPS11, any data service was visible across all workspaces on the SDS server. With SPS11 you can now choose to limit the data service availability to a single workspace, or continue to make it available across all workspaces on the server.

Incremental Enhancements Based on User Feedback

  • Added multi-threading support for custom row based formatters used in building custom adapters
  • Support for additional dynamic attributes for file/hadoop adapters
  • New methods have been added to the C, .NET and Java SDKs to process formatting errors with fixed decimal values
To report this post you need to login first.

4 Comments

You must be Logged on to comment or reply to a post.

  1. Jason Kelly

    Hi Robert,

    Thanks for the very informative post. There are lots of new features that I find most intersting and likely very useful.

    We currently have SAP ESP installed in our organisation. Is there a planned release for ESP that will contain a number of the same features? Or is the SDS product becoming increasingly different from ESP?

    Thanks,
    Jason.

    (0) 
    1. Robert Waywell Post author

      Hi Jason,

      Yes we are seeing some differentiation between SDS and ESP which reflects a greater investment in SDS as part of the integrated HANA platform. Another relevant factor is that the HANA platform is only supported on Linux while ESP is supported on additional platforms. This means that releasing the same new features on ESP requires additional additional development and QA effort over releasing on SDS only. That said, several enhancements are still shared between SDS and ESP.

      Are there particular new features that you are seeing in SDS that you would like to see in ESP?

      Understanding that you are using ESP today, do you also use HANA and is there the opportunity to move to SDS?

      Thanks

      Rob

      (0) 
      1. Michael Dutov

        Hi Robert,

        I apologize for the invasion, but I’m also very interested to understand the differences between ESP and SDS roadmaps.

        Our company uses BW on HANA and S/4HANA will be. At the moment we used old version of ESP (SP04) and I want to upgrade it. Currently we are testing the capabilities of SP10 and I can already see a forthcoming transition. As for the SP10 if I understood correctly ESP and SDS is very similar but what’s about the future releases?

        Regarding opportunity to move to SDS this question may appear on the agenda.

        Best Regards,

        Michael

        (0) 
        1. Robert Waywell Post author

          Hi Michael,

          While many enhancements and new features will continue to be shared between ESP and SDS, it is likely that there will also be some new features which are only available in SDS. In some cases, such as support for HANA System Replication the new feature is only relevant to SDS and isn’t relevant to ESP. In other cases where we are enhancing existing shared components such as the HANA Output adapter or the HTTP Client Output adapter in SPS11, the enhancements will continue to benefit both product packages. Then there will be features such as the machine learning algorithms which will be released for SDS only. In the case of the machine learning algorithms, they execution occurs within the streaming engine but still relies on integration with the HANA database to persist the models.

          As a general guideline we would recommend using SDS whenever the architecture includes HANA. ESP continues to be relevant in cases where HANA is not part of the environment.

          Thanks

          Rob

          (0) 

Leave a Reply