Technical Articles
Flow Builder Plus: the technical highlights
Flow Builder Plus, a.k.a. Flow Builder 2.0, is a new program which created as a successor of old Flow Builder (a.k.a. Flow Builder 1.0) since CE2111, and it is available since OP2021 SP1. For OP 2021 SP0, this program also delivered via Notes.
Then, what’s the improvement in Flow Builder Plus comparing to old Flow Builder?
Well, it is a big question which need be documented in business aspect and technical aspect, respectively.
See other posts which tried to document it from business aspect:
- Flow Builder Plus: the business insight, Part I | SAP Blogs
- Flow Builder Plus: the business insight, Part II | SAP Blogs
This post just highlight the technical advantages as following:
Huge performance Improvement, and no more HANA Out-Of-Memory issue.
The major pain point of Old Flow Builder is the performance, including both execution time and the runtime memory consumption.
Flow Builder Plus, put ‘a better performance and ensure a zero Out-Of-Memory issue’ as the first priority design target, implemented with following parts:
Firstly, Flow Builder Plus invents an auto-scaling program framework. This new auto-scaling program framework embeds ‘Packaging’ concept in all parts to avoid memory issue. The old Flow Builder using AMDP, and it may run into HANA Out-Of-Memory issue for the huge document chain if the statement limit reached, because it will load all related documents into memory. It is true that old Flow Builder also introducing ‘Packaging’ concept, but it only apply to the first step. Flow Builder Plus, using purely ABAP memory, apply the ‘Packaging’ concept in all steps to ensure the memory consumption is fully under control.
Secondly, The auto-scaling program framework integrates ‘Parallel Run Framework’ to accelerate the flows’ generation, which allows you to use your AS ABAP’s capacity to fast data processing. In current implementation, Flow Builder Plus using STPA. It can be adopted for other Parallel Run framework easily.
Thirdly, Flow Builder Plus introducing new locking mechanism to allow multiple Flow Builder instances running at the same client at the same time. The new locking mechanism benefits the performance side a lot.
For example, to process extremely large data volume, you can start several Flow Builder job with different company code, and each job using ‘Parallel Processing’.
Please be aware that Flow Builder Plus may cost longer time than older Flow Builder if Parallel Run is not enabled, the root reason is Flow Builder Plus will perform ‘Packaging’ in each step.
Some figures from LABS testing.
For execution times:
- Building Liquidity Analysis flows for 158 million document items (data volume of BSEG), with parallel task set to 30, package size set to 400 000, it cost 3525 seconds (less than 1 hour).
- Building Cash Position flows for 158 million document items (data volume of BSEG), with parallel task set to 30, package size set to 400 000, it cost 338 seconds (around 5.5 min) .
For memory consumption:
- Via DB02, ‘Expensive SQL Statement’ appears when Old Flow Builder running. Now, there is zero statement with Flow Builder Plus.
- HANA Statement Limit restrict old Flow Builder. That setting won’t impact Flow Builder Plus any more.
- Depends on HANA Statement Limit setting, there could be HANA Out Of Memory exception when old Flow Builder executing.
- The maximum memory cost (ABAP Server memory not HANA memory) via SM50 we observed is around 0.5GB (per session). Compare to the Out-Of-Memory issue in old Flow Builder, it normally cost more than 400GB.
Auto-healing mechanism and stronger data integrity.
Both Old Flow Builder and Flow Builder Plus generate flows coming from Accounting documents and MM documents. For accounting side, due to the complexity of accounting document chain as well as the real-time integration logic, the failure of Flow Builder will result in wrong figures. And it becomes a critical issue for Cash Position.
That’s second pain point we received for old Flow Builder.
Flow Builder Plus, introducing the auto-healing mechanism by defines a small program ‘Gap Filler’ which will try to ‘Fill the Gap’ automatically. That program will:
- Scans for missing flows
- Scans for obsoleted flows need be deleted
- Scans for unwanted delta entries
A light-weight job will be scheduled for that purpose to provide a stronger data integrity for FI flows part.
No more meaningless snapshot flows.
Old Flow Builder always generate snapshot flow each time when flows were rebuilt and the Snapshot was switched on. With this behavior, the data volume of our kernel table FQM_FLOW increasingly dynamically. That’s the third pain point we received.
Flow Builder Plus, also put the target ‘No more meaningless snapshot flows’ in the design phrase. Unlike old Flow Builder, Flow Builder Plus will generate flows but it will compare to existing flows, and it will generate snapshot flows, if and only if the generated flows are different with existing flows.
Old Flow Builder cannot perform such comparison because it lacks of ‘Packaging’ in the last step. Comparing million flows which generated in memory and million existing flows in database is nearly impossible. With the benefit of ‘Packaging’, Flow Builder Plus is able to perform the comparison before updating database.
Old Flow Builder, reports ‘XXXX flows generated’;
Flow Builder Plus, reports ‘XXXX flows generated, XX flows updated’;
If the accounting documents are the same and all relevant configuration are unchanged, ideally Flow Builder Plus reports ‘XXXX flows generated, 0 flows updated’.
With respect to the fact that accounting document chain is quite complex, Flow Builder Plus may still update a few flows, and we expect the updated ratio (updated flows / all generated flows) shall less than 1%.
An example:
- With Older Flow Builder, if one complex chain will generate 10 000 flows, and there will be 100 000 flows when Snapshot is on if that chain has been rebuilt 10 times.
- With Flow Builder Plus, process the same complex chain, ideally there are still 10 000 flows even Snapshot is on and the chain was rebuilt 10 times. With respect to the updated ratio 1%, there could be 11 000 flows if the chain has been built 10 times.
Easier to apply note: removed ‘Loading Class’ concept.
Old Flow Builder implemented with AMDP, and Loading Class concept is introduced to support extensibility (via HADI). SAP delivers the standard template, and customer need generate their own loading classes. Therefore, apply Notes or upgrading the systems, re-generate loading class is a mandatory step. And it will be a nightmare if the HADI class is missing in loading class – data rebuilding is a must.
That is fourth pain point we received.
Flow Builder Plus, which refactored as purely ABAP, which deprecates Loading class concept. No matter apply standard Notes or reimplement your extensibility code, there is no need to generate loading class.
Flexible way to stop Flow Builder job.
Another pain point of old Flow Builder is, stop Flow Builder’s job cost times and also impacts business usage. Due to the technical job of Flow Builder, normally the job will be stopped within 1 hours. And once Flow Builder was stopped, it impacts all company codes.
Flow Builder Plus, in another hand, controlled by the content in configuration (maintenance view VFQMC_ACT_FI), where you can set the ‘Liquidity Analysis Update Mode’ as ‘Deferred – Delta Table Update’. The Flow Builder job won’t triggered for that company code and the other company codes could be executed without any impaction.
Reduced conflicts between Manual Adjustment in CCFI and Flow Builder.
As introduced above, Flow Builder Plus introducing new locking mechanism to allow multiple Flow Builder instances running at the same client at the same time.
Old Flow Builder using the lock in client level;
Flow Builder Plus, however, using a new lock in company code level;
It will reduce the conflicts between Manual Adjustment in CCFI and Flow Builder Plus for sure.
Last but not least, Flow Builder Plus doesn’t require any data rebuild if you want to upgrade from old Flow Builder and without any business scope change. That’s another design target in Flow Builder Plus.
If you are interesting in Flow Builder Plus (currently available in OP2021 SP0 via Note and officially available in OP2021 SP1), please refer to
Any further question, drop us a mail please.
Hi Hongjun,
Thanks for this great explanation of thecnical improvements. If possible, want to know if when using FCLM_FLOWBUILDER_JOB i mandatory when we use the following configuration(for all company codes)
Accounting Scope:Cash Position Only
Liquidity Analysis Update Mode: Not Relevant
This, because with no job scheduled cash position is being moved when upload a bank statement file.
Thanks in advance,
Lucas
Hi Lucas,
It is true that Flow Builder Job is not necessary for 'Cash Position Only' case.
But it is recommend to setup the job to make the system 'technically readiness' for other account scopes. And it is up to the project's decision.
Thank you.
Best regards, Hongjun
Hi Hongjun,
Firstly, thanks for all the detailed cash management blogs, I've used many of them in the last couple of years!
Secondly, I have a question to ask:
We have recently upgraded to S/4 2021 FPS02 and we have not activated the Flow Builder Plus.
We noticed during testing that it seems like the exit class we setup in FCLM_FB_UTIL is no longer being called by the old Flow Builder (FCLM_FLOW_BUILDER). We had implemented the exit class as described in one of your other blogs: https://blogs.sap.com/2017/04/12/flow-builder-using-exit-to-fetch-hbkidhktid-for-accounting-documents-missed-those-information/
To fix we have implemented BAdI BADI_FQM_FLOW_ADJUST_CORE as described in note 3211174 - Flow Builder Plus: Reuse exit class implementation in Flow Builder Plus.
I would like to ask you if this is the expected behavior when still using the old Flow Builder?
And if yes, I would like to suggest that note "3128064 - Flow Builder: Technical Changes in S4CORE version 106" is updated to state that the exit class will no longer be called and the BAdI above must be implemented instead.
Many thanks,
Jack
Hi Jack,
If you didn't activate Flow Builder Plus, everything which work before for old Flow Builder shall be the same, means no changes (BADI implementation, configuration, etc.) are required from your side.
If you are seeking for support, raise an OSS incident to SAP, and our support team will help.
Thank you.
Best regards, Hongjun
Hi Hongjun,
That's what I expected but the exit class stopped working after we upgraded to the 2021 version and the only way we could find to resolve it was to implement the BADI_FQM_FLOW_ADJUST_CORE.
We also had to add new field FI_ITEM_TEXT to the exit class to fix a syntax error.
I will raise an OSS for clarification.
Thanks!
Jack
Hi Jack,
Okay. Meanwhile I think you can try Flow Builder Plus as you have system OP2021.
BTW, old Flow Builder will be depreciated since OP2022.
Thanks.
Best Regards, Hongjun
Hi Hongjun,
That's the long term plan of course but for this upgrade we decided to stay with the old Flow Builder to reduce the scope, risk and complexity of the upgrade. Flow Builder Plus is still relatively new and we see a lot of bug fixing notes being released so made the choice to wait until it's more mature and stable before switching.
We are also running CM on a sidecar system where we don't use liquidity item derivation, chain tracing, etc. so many of the benefits of Flow Builder Plus are not that relevant for our solution.
Br,
Jack
Hi Jack,
Now I got a clearer picture about your situation.
And it will be true that your HADI class won't be called any longer. Please refer to Note 3128064 for more detail. Old Flow Builder using the same logic for Cash Position part which won't support HADI.
Meanwhile regarding 'Flow Builder Plus is still relatively new and we see a lot of bug fixing notes being released', actually we released only 10 Notes in around 10 months, and we are purely sure it has a much better shape comparing to the old Flow Builder. It will also benefit for your case.
Thanks.
Best regards, Hongjun
Hi Hongjun,
thank you for this great blog!
I have small question - does the initial load of data to One Exposure have any differences using Flow Builder Plus compared to Flow Builder 1.0? Are the same programs/steps needed? In particular are these programs still relevant for initial load with Flow Builder Plus?
programs to prepare BSEG:
programs to load cash flows afterwards
Thank you in advance
Olga
Hi Olga,
Right, the program/step is the same for using Flow Builder Plus. Flow Builder Plus is targeted to replace Flow Builder 1.0.
Thank you.
Best regards, Hongjun