Skip to Content

Intro


Parallel processing is not a new concept, but one that is regularly overlooked when it comes to increasing ABAP performance. Why?

Your SAP system will (normally) have more than one process available at any given time. Still, most of us insist on using just one of them. This is a bit like a manufacturer relying on only one truck to bring the products from his plant to the shopping malls, when there’s a whole fleet of trucks just standing by!

Not only that, but most SAP systems spans more than one application server, each with a range of (hopefully) available processes. So, what are we waiting for?

h2. OK, so my program takes forever to execute. How can I put it on steroids?

In this blog, I’ll show a practical example for dealing with one of the dreaded father-and-son relations in the wonderful world of SAP FI: Accounting documents, tables BKPF-BSEG. Prowling your way through these tables can really take its toll, both on the system itself and your patience. Not to mention that of the customer breathing down your neck.

There are numerous other blogs and papers about parallel processing using RFC’s on SDN, and one of the best (if not the best) is Thorsten Franz’ blog Calling Function Modules in Parallel Universes.

I actually suggest you start there for some very interesting background info on the merits (and pitfalls) of using RFC’s. There’s also a link to Horst Keller’s blog series and the official SAP documentation. In addition, the excellent book “ABAP Cookbook” from SAP Press (by James Wood) outlines the principles of using asynchronous RFCs for parallel processing (as well as providing loads of other cool stuff for enhancing your ABAP skill set!) A highly recommended read.

Using asynchronous RFC’s without caution is not recommended, and you risk bogging down the system as well as running into errors that can be difficult to resolve. However, if you do decide to use parallel processing, the following might be a good starting point. I’ll try to keep things simple and explain every step along the way.

h2. The sample program

We’ll create a program to display info from BKPF/BSEG. The program will read all entries from BKPF (feel free to introduce your own selection criteria here, such as company code and/or fiscal year), and then retrieve all related entries from BSEG. This second step will be done by calling a function module via RFC, repeatedly, in parallel. We will try to balance the workload based on the number of documents in the tables, and the available processes on our SAP application servers.
Finally, we will examine the runtime analysis of the program and compare to a standard single process execution.

The test program basically consists of the following steps:

 

    • Call function module SPBT_INITIALIZE to find out how many available processes we can use
    • Split the number of documents into handy packages and call an RFC a number of times in parallel
    • Wait for the results from the called RFC’s and merge them all back into the final report

The program is fairly straightforward. It reads BKPF, tries to split the retrieved BKPF entries into nice “packages” based on the key fields BUKRS and GJAHR. These are the two main parameters for our RFC-enabled function module – we’re building range tables for them in order to facilitate our work. The idea is to pass these two as ranges to the RFC-enabled function reading BSEG, so that the number of documents passed to each call of the function is more or less consistent. Since the number of financial documents will vary with company codes and fiscal years, we cannot ensure a 100% even workload, but this is just an example.

Based on the available resources (number of processes for all application servers, which we find using function SPBT_INITIALIZE), we then start to kick off calls to the RFC-enabled function module. This is done a number of times in parallel, using the CALL FUNCTION STARTING NEW TASK… PERFORMING … ON END OF TASK. By using this feature, we ensure that the calling program executes a specific form whenever the control is passed back from the RFC “bubble” to the calling program (common sense states you should use object orientation, and thus specify a method, but for our example I find the classical procedural program better for illustration purposes).

 

What happens when the aRFC finishes, is the following:

    1. Control is passed back to the calling program
    2. The form (or method) specified in the CALL TRANSACTION statement (addition PERFORMING … ON END OF TASK) is called. This form enables you to retrieve any returning or exporting parameters from the called RFC, and use them in the main processing.

 

By splitting the workload into sizeable chunks, we can execute a multitude of workloads simultaneously, thereby reducing the total execution time to a fraction of the time traditionally used. In my example, I was able to run this report in less than 5% of the time it took running it in one single process.

The program and function module are presented below. I’ve done my best to insert comments in order to explain what’s going on, and hope you can use this as a template.

The function module has been created as an RFC function (check the Remote-Enabled Module box on the Attributes tab). Besides this, there’s nothing special about it.

&—-



*& Report  ZTST_CALL_ASYNC_RFC
*&
&—-



*& A small test program for calling function modules a number of times
*& in parallel.
*&
*& The program calls the RFC-enabled function ZTST_READ_BSEG – which
*& reads entries from table BSEG and calculates totals
*&
*& A log table, ZTST_RFC_LOG, is used for logging the progress, both
*& from within this program and the function itself.
*&
*& The program will launch the function on all available app servers.
&—-



report  ztst_call_async_rfc.


type-pools: abap.


types: begin of t_bkpf,
         bukrs  type bukrs,
         gjahr  type gjahr,
         belnr  type belnr_d,
       end of t_bkpf,
       tab_bkpf type table of t_bkpf.


types: begin of t_bseg,
         bukrs type bukrs,
         gjahr type gjahr,
         dmbtr type dmbtr,
       end of t_bseg.


types: begin of t_stat,
         bukrs type bukrs,
         gjahr type gjahr,
         count type i,
       end of t_stat.


types: begin of t_tasklist,
         taskname(4) type c,
         rfcdest     like rfcsi-rfcdest,
         rfchost     like rfcsi-rfchost,
         result      type char50,
      end of t_tasklist.


data:
      lv_max_processes type i,
      lv_free_processes type i,
      lv_number_of_processes_in_use type i,
      lv_started_rfc_calls type i value 0,
      lv_finished_rfc_calls type i value 0,
      lv_exception_flag(1) type c,
      lv_taskname(4) type n value ‘0001’,
      lt_tasklist type table of t_tasklist,
      lv_index type i,
      lt_bkpf type tab_bkpf,
      lv_lines_in_bkpf type i value 0,
      lv_records_pr_rfc type i value 0,
      lv_loop_pass type i value 0,
      lv_gjahr type gjahr,
      lv_belnr_start type belnr_d,
      lv_belnr_end type belnr_d,
      lt_results_total type ztst_bseg_results_tt,
      lv_total_so_far type i,
      lv_bukrs_range type ztst_bukrs_range,
      lt_bukrs_range type ztst_bukrs_range_tt,
      lv_gjahr_range type ztst_gjahr_range,
      lt_gjahr_range type ztst_gjahr_range_tt,
      lv_bseg type t_bseg,
      lt_results type ztst_bseg_results_tt,
      lv_stat type t_stat,
      lt_stat type table of t_stat,
      lv_sum type i.


field-symbols: type ztst_bseg_results_s.


parameters: p_para as checkbox default ‘X’.


start-of-selection.


  perform initial_selection.


  • The parameter P_PARA allows you to run in sequential mode for comparison purposes!

  if p_para = abap_true.


  • Start by retrieving number of maximum and available processes

    call function ‘SPBT_INITIALIZE’
      exporting
        group_name                     = ”
      importing
        max_pbt_wps                    = lv_max_processes
        free_pbt_wps                   = lv_free_processes
      exceptions
        invalid_group_name             = 1
        internal_error                 = 2
        pbt_env_already_initialized    = 3
        currently_no_resources_avail   = 4
        no_pbt_resources_found         = 5
        cant_init_different_pbt_groups = 6
        others                         = 7.


    if sy-subrc <> 0.
      message id sy-msgid type sy-msgty number sy-msgno
              with sy-msgv1 sy-msgv2 sy-msgv3 sy-msgv4.
    else.
      write : / ‘Max processes: ‘, 50 lv_max_processes right-justified.
      write : / ‘Free processes: ‘, 50 lv_free_processes right-justified.
      uline.


  • Clear the log table of old entries

      delete from ztst_rfc_log where taskname <> ‘0000’.
      delete from ztst_rfc_log where taskname = ‘0000’.


      lv_records_pr_rfc = lv_lines_in_bkpf / 1000.
      write : / ‘Number of estimated RFC calls: ‘, 50 lv_records_pr_rfc right-justified.
      uline.


      sort lt_bkpf.
      move 0 to lv_index.


  • Accumulate total number of BKPF per BUKRS and GJAHR

      loop at lt_bkpf assigning -count right-justified.
      endloop.
      uline.


      skip 2.
      write : / ‘Log messages during execution:’.
      uline.


  • Main loop – here, we loop at LT_STAT, which contains number of documents per company code
  • and year (BUKRS and GJAHR). We use these figures to build ranges for BUKRS and GJAHR, which
  • are then used when calling the RFC function module
  • If you do not need to programmatically calculate the number of entries for each RFC call,
  • for instance when processing a table sequentially, you can replace this logic with something simpler,
  • a.k.a. Loop at table, call RFC for each 1000 entries.

      loop at lt_stat assigning -count + lv_total_so_far ) < 1500.

  • Means we have previous entries in ranges, but < 1500 total (including the current record).
  • We add current record to previous ranges and run all of them.

            move ‘I’ to lv_bukrs_range-sign.
            move ‘EQ’ to lv_bukrs_range-option.
            move 0.   “If anything at all in previous ranges, run for these first & flush
              write : / ‘Calling RFC for previous range’, 60 lv_total_so_far right-justified.
              perform call_rfc. ” with previous ranges only
              refresh lt_bukrs_range.
              refresh lt_gjahr_range.
            endif.


  • Now, run RFC for BUKRS/GJAHR current record (which has more than 1000 in count)

            move ‘I’ to lv_bukrs_range-sign.
            move ‘EQ’ to lv_bukrs_range-option.
            move -gjahr to lv_gjahr_range-low.
            append lv_gjahr_range to lt_gjahr_range.
          endif.
        endif.


      endloop.


    endif.


  • That’s it! We’ve called our RFC a number of times (hopefully more than one).
  • Now, all that remains is to wait until all RFC’s have finished.

    wait until lv_finished_rfc_calls = lv_loop_pass.


  • Write the contents of TASKLIST to show which servers were used and how things went…

    skip 2.
    write : / ‘Result of RFC calls:’.
    uline.
    loop at lt_tasklist assigning -result.
    endloop.
    skip.


  • Booooring… sequential mode – for performance comparison only (try runtime analysis on each logic)

  else.


    loop at lt_bkpf assigning -gjahr.


        divide lv_bseg-dmbtr by 1000000.         ” To avoid overflow when summing up…
        collect lv_bseg into lt_results_total.
      endselect.
    endloop.


  endif.


  • Final touch: print the results of all our efforts.

  sort lt_results_total.


  skip 2.
  write : / ‘Results from BSEG:’.
  uline.
  write : / ‘Company’, 10 ‘Year’, 25 ‘Amount in mill.’.
  loop at lt_results_total assigning -dmbtr right-justified.
  endloop.


&—-



*&      Form  initial_selection
&—-



  •       text

—-



form initial_selection.


  select bukrs belnr gjahr from bkpf into corresponding fields of table lt_bkpf.


  describe table lt_bkpf lines lv_lines_in_bkpf.
  write : / ‘Number of records in BKPF:’, 50 lv_lines_in_bkpf right-justified.


endform.                    “initial_selection


&—-



*&      Form  call_rfc
&—-



  •       text

—-



form call_rfc.


  add 1 to lv_number_of_processes_in_use.


  • Note that it might not be a good idea to use ALL free processes, such as here.
  • Doing so might cause minor inconveniences for other users, or nasty phone calls
  • from Basis…. 🙂
  • A better idea would be to reduce lv_free_processes by, say, 5 before starting.


  if lv_number_of_processes_in_use > lv_free_processes.
    write : / ‘Waiting; number of processes > ‘, lv_number_of_processes_in_use.
    wait until lv_number_of_processes_in_use < lv_free_processes.
    write : / ‘Waiting over, number of processes =’, lv_number_of_processes_in_use.
  endif.


*
  call function ‘ZTST_READ_FINANCIAL_DOCS’
    starting new task lv_taskname
    destination in group default
    performing receive_results_from_rfc on end of task
    exporting
      im_taskname           = lv_taskname
      im_bukrs              = lt_bukrs_range
      im_gjahr              = lt_gjahr_range

  • Note that we are not using the IMPORTING parameter here;
  • instead it’s used when doing RECEIVE RESULT in form RECEIVE_RESULTS_FROM_RFC

    exceptions
      communication_failure = 1
      system_failure        = 2
      resource_failure      = 3.


  case sy-subrc.


    when 0.
      write : / ‘Started new task, task name ‘, lv_taskname.


      append initial line to lt_tasklist assigning -taskname = lv_taskname.


  • Retrieve the name of the server

      call function ‘SPBT_GET_PP_DESTINATION’
        importing
          rfcdest = -rfcdest
        exceptions
          others  = 1.


      lv_started_rfc_calls = lv_started_rfc_calls + 1.
      add 1 to lv_taskname.


    when 1 or 2.           “Communications failure

  • This could mean an app server is unavailable; no real need to handle this situation in most cases.
  • (Subsequent calls to the same server will fail, but the FM should run nicely on all available servers).


    when 3.                “No available dialog processes right now – wait!
      if lv_exception_flag = space.
        lv_exception_flag = ‘X’.
        write : / ‘No more processes available, waiting…’.
        wait until lv_finished_rfc_calls >= lv_started_rfc_calls.
      else.                “Second attempt
        write : / ‘Still no more processes available, waiting…’.
        wait until lv_finished_rfc_calls >= lv_started_rfc_calls.


        if sy-subrc = 0.   “Wait successful – processing continues
          clear lv_exception_flag.
        else.              “Wait failed – something is wrong with RFC processing. Aborting.
          write : / ‘No RFC calls completed – aborting processing!’.
          exit.
        endif.
      endif.
  endcase.


endform.                    “call_rfc
*
&—-



*&      Form  RECEIVE_RESULTS_FROM_RFC
&—-



  •       Called when we return from the aRFC.

—-



  •      –>VALUE         text
  •      –>(P_TASKNAME)  text

—-



form receive_results_from_rfc using value(p_taskname).


  • Note: WRITE statements will not work in this form!


  data lv_netwr type netwr_ap.
  data lv_netwr_num(18) type n.
  data lt_results type ztst_bseg_results_tt.


  lv_number_of_processes_in_use = lv_number_of_processes_in_use – 1.


  • Update the TASKLIST table, which is used for logging

  read table lt_tasklist with key taskname = p_taskname assigning -result = ‘Error in task execution’.
    endif.
  endif.


  • Receive the results from the RFC

  receive results from function ‘ZTST_READ_FINANCIAL_DOCS’
    importing re_results = lt_results.      ” <— receiving the result from the RFC!


  • Loop at partial results; include in our totals table

  loop at lt_results assigning function ztst_read_financial_docs .
*”—-


“Local Interface:
*”  IMPORTING
*”     VALUE(IM_TASKNAME) TYPE  NUMC4
*”     VALUE(IM_BUKRS) TYPE  ZTST_BUKRS_RANGE_TT
*”     VALUE(IM_GJAHR) TYPE  ZTST_GJAHR_RANGE_TT
*”  EXPORTING
*”     VALUE(RE_RESULTS) TYPE  ZTST_BSEG_RESULTS_TT
*”  CHANGING
*”     VALUE(CH_NETWR) TYPE  NETWR_AP OPTIONAL
*”—-



  • This is a function module used by program ZTST_CALL_ASYNC_RFC
  • for demo purposes. The idea is to show how to take long-processing
  • programs and split them up into parallel processes, by calling
  • RFC’s asynchronously. The calling program will call this function
  • module a number of times, then collect the results and process them.


  types: begin of t_bseg,
           bukrs type bukrs,
           gjahr type gjahr,
           dmbtr type dmbtr,
         end of t_bseg.


  data lt_bseg type table of t_bseg.
  data lv_rfc_log type ztst_rfc_log.
  data lv_filename type string.


  field-symbols h2. Structure for the exporting parameter (table)

Checking the system load


During program run, you can check your RFC’s with transaction SM66, which shows all processes across the application servers of your system.

h2. Final word: run time analysis

Try using the run time analysis on the program, both when selecting parallel mode and when un-checking P_PARA (which causes a normal select within the main program). The runtime analysis won’t show the additional load of the RFC modules running in parallel, but the total program execution time is far lower – and this, after all, is the main point of splitting a workload into separate parallel tasks.

Running in sequential mode (no parallel RFC’s):

!https://weblogs.sdn.sap.com/weblogs/images/252050646/img5.GIF|height=275|alt=Sequential run|width=617|src=https://weblogs.sdn.sap.com/weblogs/images/252050646/img5.GIF!

Running in parallel mode:

!https://weblogs.sdn.sap.com/weblogs/images/252050646/img6.GIF|height=199|alt=With parallel processing|width=626|src=https://weblogs.sdn.sap.com/weblogs/images/252050646/img6.GIF!

As you can see, the run time is dramatically reduced. Total system load may amount to the same (and should actually be slightly higher, with the overhead of the separate RFC’s), but it’s the total execution time that counts. Here, the execution time is roughly 10% when using asynchronous RFC’s as compared to a classical “one-in-all” process.

The above tests were run in a system with approximately 35.000 entries in BKPF, and 100.000 in BSEG.

h2. Words of warning (again)

A few words on transactional RFC’s: There are situations when you cannot use this technique. Commits cannot be performed inside an RFC – this would conflict with the session in which the main program is running. You can find more info on these topics by checking SAp help for RFC programming. However, for the larger part of processor-intensive developments, it is a technique that is sadly overlooked. I recommend everyone to give it a try, provided they follow the guidelines provided by SAP on the topic.

h2. Additional reading:

In addition to the blogs and resources mentioned at the start, the following is worth checking out:

Horst Keller: Application Server, what Application Server?

 

To report this post you need to login first.

18 Comments

You must be Logged on to comment or reply to a post.

  1. Michelle Crapo

    <Sigh>  Our basis people tend to frown on using all their App Servers on one program.  Although it might be fun to bug them a little, we rely on them too much.<br/><br/>That’s why I haven’t tried this.   Do you have any ideas on that?  Maybe it’s OK to do this, when the program is a fast one.  But that’s the purpose right?  Your program is long running.<br/><br/>Thanks for the answers to my questions – they may be found in the other blogs that I will get to shortly!<br/><br/>Michelle<br/>

    (0) 
    1. Trond Stroemme Post author
      Hi,

      I believe a decent approach would be to maximize the number of sessions to 3-5, based on the available number of sessions in the system. If you look at the program code, it investigates the number of free sessions by calling function SPBT_INITIALIZE, which provides max number of sessions and free sessions.

      I would advice – as a general principle – to limit your development to using 25% of the free sessions, but this is dependent on other factors as well, such as whether the development is run during high system loads (daytime, lots of users logged in, and so on…) Obviously, running during low-load periods would allow you to eat more free processes!

      By using SPBT_INITIALIZE and setting the limit well below the number of free/max sessions, I think you should end up being on the safe side. And, of course, this technique should be limited to the (hopefully very few) developments that really are both demanding on resources and mission-critical in terms of run time.

      Discussing and establishing a framework for these kinds of developments with Basis beforehand is anyway a good idea!

      Cheers,
      Trond

      (0) 
      1. Michelle Crapo
        Excellent!  I’ll have to try it.  I WILL talk with BASIS first.  They are my friends.  Especially when I break something in the system.  What I say?  It happens sometimes.  They weren’t real happy we did Web Dynpro before letting them know.  And you’d have to talk with a BASIS person to know exactly why that was.    I sort of remembered the answer I got.

        Thank you for the nice blog!  (And comment)

        Michelle

        (0) 
  2. Naimesh Patel
    Cool Post.

    Echoing what Michelle has said, you should also consider adding some more checks:
    * If you get the Exception from ‘SPBT_INITIALIZE’, you can wait for a second and try again. You can also restrict this check before giving up.
    * Prepare the data counts before starting the parallel processing. This would minimize the chances of getting the exceptions from ZTST_READ_FINANCIAL_DOCS.
    * Also add the fail safe handler: Add a wait statement to wait for say 60 seconds before all work process returns the result. You can add this after your RFC call. This should prevent your main process to hang when due to any reason the WP goes to dark.

    Regards,
    Naimesh Patel
    ABAP Help Blog</>

    (0) 
    1. Trond Stroemme Post author
      Hi Naimesh,

      thanks for your valuable comments! As this is an example, I did not focus too much on the logarithm to split the documents into suitable chunks before calling the RFC. Obviously, if there’s a combination of company code and year that contains tens of thousand of documents, it would be a good idea to split it into separate calls – but then you need another input parameter. Ideally, each process should have more or less the same workload – each programming scenario might require specific ways of deducing this load.

      As for the wait n seconds, there’s currently a wait statement after the loop, but it’s based on the difference between the called and finished processes. Adding a time-based clause is definitely a good idea, in case one of the WP’s decide to go AWOL… :-). You could possibly even do this dynamically by monitoring the processing times for each RFC (using time stamps?), and waiting for a situation-specific amount of time.

      Thanks again!

      (0) 
  3. Krishnendu Laha
    Thanks for the blog…It works like an warning bell for me…I almost forgot magic of RFC call (with caution :)) in same system…this one made me awake….nice one…
    (0) 
  4. Julian Phillips
    I notice you use RFCs to create the new threads – I don’t suppose there is any way to do this in the ABAP objects syntax. As a rule we are trying to move our programmers away from the usage of creating new function modules – this one seems to be the case that SAP only thought to allow parallel processing in the older procedural processing model… Java has threads – why not ABAP… I would love for someone to prove this wrong and show me the ABAP threading functionality…
    (0) 
    1. Michelle Crapo
      I can create an Enterprise service and multi-thread anything that an RFC does.   In the past we have used PI to create the XML that is passed to the object.  The proxy would push the data back to the PI system.

      I’m sure you could do the same thing without PI in between.  I know we slowed our system almost to a halt.  (Luckily our Test system)  We used every background process.  We needed some configuration and now use less processes.

      So we automatically used multiple processes.  We didn’t have to create them.  Yes, doing this in objects is possible.

      Make sense??

      Michelle

      I have some slides I can look up, if you are really interested.  I presented this at one of the techeds.  I think 2009.  So it isn’t at the top of my mind.  I remember one of the comments was could you do this without PI/XI.  The answer was yes.  it wouldn’t take me long to remember what I had there.

      (0) 
      1. Julian Phillips
        Hi Michelle,
          that sounds interesting – I think the whole point of this technique is to gain performance benefits – and if your ABAP Objects version does not incur too much overhead – then its definitely of interest I would think to the ABAP community at large.
        I think the concern of over grabbing too many processes is very genuine – and it what would be really good is to have a threading functionality inbuilt into ABAP Objects syntax – where the programmer does not need to worry about the number of work processes that their code is consuming – they should be able to rely on the ABAP runtime environment to limit it to something that isn’t gonna take down the server (in a similar way to Java). I guess this is all potential material for a new / upcoming version of ABAP.
        (0) 
  5. Fabio Pagoti
    This is a very important and not often used technique.

    It would be great to have along with this kind of blog some related projects in Code Exchange so community can familiarize with this.

    Very well done Trond. Thanks for sharing this.

    (0) 
  6. Shweta .
    Thanks for this post. This is one of the most important part which we should not ignore during our programming.

    Very well documented!

    (0) 
  7. Trond Stroemme Post author

    I’m truly sorry the guys at SDN SCN messed up the format of my blog. In addition, they’ve asked (all of us) to re-format and clean up – basically, doing their job. No, I’m not going to do that. I didn’t mess it up in the first place, see?

    (Yes, I’m slightly annoyed they did it. It’s a bit like having a colleague pouring salt on your dessert, when you’re not looking. It’s not even fun…)

    (0) 
    1. Arshad Ansary

      Hi Trond ,

      Excellent blog .Sad that new SCN format has messed up your code . But I am not able to get the logic for splitting up the lt_bkpf itab into sizeable chunks as this is not clear from the newly formatted code . Could you please mail me the sample code to my id ansaryarshad@gmail.com.

      Regards

      Arshad

      (0) 
    2. Bjorn-Henrik Zink

      Hi Trond,

      we should write a blog about “Blog and Comments eating Migrations …”. Half my blogs and the comments have been gone since the “migration”. I fully agree it is really not funny!

      Honestly, the number one feature of scn is contents. If you blow away the contents, then scn is nothing. Rethink. Redo.

      Cheers

      Björn-Henrik (Elvez)

      (0) 
  8. Zhaodong Wang

    Hi Trond,

    The topic is very useful.

    And, could you pleaes share the sample code?

    The above is messed up and couldn’t find coding/logic for ZTST_READ_FINANCIAL_DOCS.

    Thanks and Best Regards,

    Zhaodong

    (0) 

Leave a Reply