Flow Builder Plus: the technical highlights

Flow Builder Plus, a.k.a. Flow Builder 2.0, is a new program which created as a successor of old Flow Builder (a.k.a. Flow Builder 1.0) since CE2111, and it is available since OP2021 SP1. For OP 2021 SP0, this program also delivered via Notes.

Then, what’s the improvement in Flow Builder Plus comparing to old Flow Builder?

Well, it is a big question which need be documented in business aspect and technical aspect, respectively.

See other posts which tried to document it from business aspect:

• Flow Builder Plus: the business insight, Part I | SAP Blogs
• Flow Builder Plus: the business insight, Part II | SAP Blogs

This post just highlight the technical advantages as following:

Huge performance Improvement, and no more HANA Out-Of-Memory issue.

The major pain point of Old Flow Builder is the performance, including both execution time and the runtime memory consumption.

Flow Builder Plus, put ‘a better performance and ensure a zero Out-Of-Memory issue’ as the first priority design target, implemented with following parts:

Firstly, Flow Builder Plus invents an auto-scaling program framework. This new auto-scaling program framework embeds ‘Packaging’ concept in all parts to avoid memory issue. The old Flow Builder using AMDP, and it may run into HANA Out-Of-Memory issue for the huge document chain if the statement limit reached, because it will load all related documents into memory. It is true that old Flow Builder also introducing ‘Packaging’ concept, but it only apply to the first step. Flow Builder Plus, using purely ABAP memory, apply the ‘Packaging’ concept in all steps to ensure the memory consumption is fully under control.

Secondly, The auto-scaling program framework integrates ‘Parallel Run Framework’ to accelerate the flows’ generation, which allows you to use your AS ABAP’s capacity to fast data processing. In current implementation, Flow Builder Plus using STPA. It can be adopted for other Parallel Run framework easily.

Thirdly, Flow Builder Plus introducing new locking mechanism to allow multiple Flow Builder instances running at the same client at the same time. The new locking mechanism benefits the performance side a lot.

For example, to process extremely large data volume, you can start several Flow Builder job with different company code, and each job using ‘Parallel Processing’.

Please be aware that Flow Builder Plus may cost longer time than older Flow Builder if Parallel Run is not enabled, the root reason is Flow Builder Plus will perform ‘Packaging’ in each step.

Some figures from LABS testing.

For execution times:

• Building Liquidity Analysis flows for 158 million document items (data volume of BSEG), with parallel task set to 30, package size set to 400 000, it cost 3525 seconds (less than 1 hour).
• Building Cash Position flows for 158 million document items (data volume of BSEG), with parallel task set to 30, package size set to 400 000, it cost 338 seconds (around 5.5 min) .

For memory consumption:

• Via DB02, ‘Expensive SQL Statement’ appears when Old Flow Builder running. Now, there is zero statement with Flow Builder Plus.
• HANA Statement Limit restrict old Flow Builder. That setting won’t impact Flow Builder Plus any more.
• Depends on HANA Statement Limit setting, there could be HANA Out Of Memory exception when old Flow Builder executing.
• The maximum memory cost (ABAP Server memory not HANA memory) via SM50 we observed is around 0.5GB (per session). Compare to the Out-Of-Memory issue in old Flow Builder, it normally cost more than 400GB.

Auto-healing mechanism and stronger data integrity.

Both Old Flow Builder and Flow Builder Plus generate flows coming from Accounting documents and MM documents. For accounting side, due to the complexity of accounting document chain as well as the real-time integration logic, the failure of Flow Builder will result in wrong figures. And it becomes a critical issue for Cash Position.

That’s second pain point we received for old Flow Builder.

Flow Builder Plus, introducing the auto-healing mechanism by defines a small program ‘Gap Filler’ which will try to ‘Fill the Gap’ automatically. That program will:

• Scans for missing flows
• Scans for obsoleted flows need be deleted
• Scans for unwanted delta entries

A light-weight job will be scheduled for that purpose to provide a stronger data integrity for FI flows part.

No more meaningless snapshot flows.

Old Flow Builder always generate snapshot flow each time when flows were rebuilt and the Snapshot was switched on. With this behavior, the data volume of our kernel table FQM_FLOW increasingly dynamically. That’s the third pain point we received.

Flow Builder Plus, also put the target ‘No more meaningless snapshot flows’ in the design phrase. Unlike old Flow Builder, Flow Builder Plus will generate flows but it will compare to existing flows, and it will generate snapshot flows, if and only if the generated flows are different with existing flows.

Old Flow Builder cannot perform such comparison because it lacks of ‘Packaging’ in the last step. Comparing million flows which generated in memory and million existing flows in database is nearly impossible. With the benefit of ‘Packaging’, Flow Builder Plus is able to perform the comparison before updating database.

Old Flow Builder, reports ‘XXXX flows generated’;

Flow Builder Plus, reports ‘XXXX flows generated, XX flows updated’;

If the accounting documents are the same and all relevant configuration are unchanged, ideally Flow Builder Plus reports ‘XXXX flows generated, 0 flows updated’.

With respect to the fact that accounting document chain is quite complex, Flow Builder Plus may still update a few flows, and we expect the updated ratio (updated flows / all generated flows) shall less than 1%.

An example:

• With Older Flow Builder, if one complex chain will generate 10 000 flows, and there will be 100 000 flows when Snapshot is on if that chain has been rebuilt 10 times.
• With Flow Builder Plus, process the same complex chain, ideally there are still 10 000 flows even Snapshot is on and the chain was rebuilt 10 times. With respect to the updated ratio 1%, there could be 11 000 flows if the chain has been built 10 times.

Easier to apply note: removed ‘Loading Class’ concept.

Old Flow Builder implemented with AMDP, and Loading Class concept is introduced to support extensibility (via HADI). SAP delivers the standard template, and customer need generate their own loading classes. Therefore, apply Notes or upgrading the systems, re-generate loading class is a mandatory step. And it will be a nightmare if the HADI class is missing in loading class – data rebuilding is a must.

That is fourth pain point we received.

Flow Builder Plus, which refactored as purely ABAP, which deprecates Loading class concept. No matter apply standard Notes or reimplement your extensibility code, there is no need to generate loading class.

Flexible way to stop Flow Builder job.

Another pain point of old Flow Builder is, stop Flow Builder’s job cost times and also impacts business usage. Due to the technical job of Flow Builder, normally the job will be stopped within 1 hours. And once Flow Builder was stopped, it impacts all company codes.

Flow Builder Plus, in another hand, controlled by the content in configuration (maintenance view VFQMC_ACT_FI), where you can set the ‘Liquidity Analysis Update Mode’ as ‘Deferred – Delta Table Update’. The Flow Builder job won’t triggered for that company code and the other company codes could be executed without any impaction.

Reduced conflicts between Manual Adjustment in CCFI and Flow Builder.

As introduced above, Flow Builder Plus introducing new locking mechanism to allow multiple Flow Builder instances running at the same client at the same time.

Old Flow Builder using the lock in client level;

Flow Builder Plus, however, using a new lock in company code level;

It will reduce the conflicts between Manual Adjustment in CCFI and Flow Builder Plus for sure.

Last but not least, Flow Builder Plus doesn’t require any data rebuild if you want to upgrade from old Flow Builder and without any business scope change. That’s another design target in Flow Builder Plus.

If you are interesting in Flow Builder Plus (currently available in OP2021 SP0 via Note and officially available in OP2021 SP1), please refer to

1. Collective Note 3097908 and,
2. Technical implementation Note 3111810
3. Trouble shooting Note 3144000

Any further question, drop us a mail please.