Note: this post has been updated to make it clear that it applies equally to both SAP HANA smart data streaming (SDS) and SAP Event Stream Processor (ESP) and also edited to improve clarity
Passing along some advice from the ESP engineering team:
To scale a Streaming+HANA system to handle high data rates through SDS/ESP into SAP HANA, assuming you have machine(s) with large number of CPUs and NUMA architecture:
1 . Create multiple streaming project instances each hosting one HANA output adapter instance and bind each project instance to a separate NUMA node on your machine(s). You can use “numactl –hardware” command to figure out how many NUMA nodes are available and what ranges of CPU cores are available on each NUMA node. Then you can use “taskset –c <range>” command to bind an esp server process to CPU cores that belong to only one NUMA node.
2. You need a way to split your incoming data between your multiple streaming project instances. If you use a splitter stream to split your incoming data then also use stream binding to send that split data to projects running on separate cluster nodes. Don’t just host everything on one streaming cluster node because you won’t be able to get optimal performance from all of your NUMA nodes if they all run the same projects. You want to have one NUMA node per streaming project described in point #1.
3. Try to keep each streaming project as simple as possible to reduce the number of threads it creates and leave as many CPU cores available as possible for use by HANA output adapter.
4. Configure each HANA adapter instance on those streaming projects with number of threads (threadCount) that gives you best performance. Probably somewhere in the range of 10 to 20. Make sure your bulkInsertArraySize is big enough to give you good performance but not so big that latency suffers beyond acceptable level. Somewhere between 100,000 and 1,000,000 is usually a good number. Increase the value for bufferAgeLimitMSec and idleBufferWriteDelayMSec timeouts to be at least 10,000 ms unless you really need the latency to be low.
5. Make sure you use the latest version of SDS/ESP, since we have continued to improve performance and scalability in each new version.
