Environment sensitive Job Spawning
Inspiration
For faster results, we often execute programs in parallel using background jobs. Even though this might be a good idea in terms of processing efficiency, on many occasions this results in high utilization of the system resources and non-availability of background work processes for other key custom operations.
Solution
The below technique provides a controlled and conservative approach towards multiple/parallel job generation. With this technique, we can set a maximum utilization percentage and hence always reserve some work processes for other operations. The attributes like utilization threshold and delay can be input as a Parameter/Select-option in the program or can be maintained in selection variable table TVARVC or a custom table, but to keep it simple I am going to use this as constants in the program.
The key function that we use in this method is TH_GET_WPINFO which returns the details of the active work processes in the system. The function takes as input 2 optional parameters SRVNAME (Application server) and WITH_CPU. If the job processing has to be limited to a specific server on the system, these parameters can be used to limit the query and later submit the job to the specific server group. But if no restrictions need to be imposed in terms of servers, these can be left blank. The function returns the list of work processes in the system and is similar to SM50 screen (except for the fact that SM50 shows the work processes only from the server on which user is logged on to). Based on this information, we can determine the total and available work processes and the background work process utilization level. Based on the percentage, we can either generate the next job or wait for predefined amount of time and try again.
Assumptions
Utilization threshold : 60%
No. of jobs to be executed : 10
Wait before retry if utilization is too high : 10 seconds
Source Code
REPORT zcons_bgd_proc.
CONSTANTS:
“Assuming the utilization limit to be 60%
c_limit TYPE p DECIMALS 2 VALUE 60.
DATA:
“Assuming a total of 10 jobs need to be generated
v_jobs TYPE i VALUE 10,
“Assuming a wait time of 10 seconds to be inserted
” if utilization is above limit
v_delay TYPE i VALUE 10.
DATA:
lt_wplist TYPE TABLE OF wpinfo,
v_total TYPE i,
v_utilz TYPE i,
v_uperc TYPE p DECIMALS 2.
DO.
CALL FUNCTION ‘TH_GET_WPINFO’
* EXPORTING
* srvname = “Application server optional
* with_cpu = “CPU optional
TABLES
wplist = lt_wplist
EXCEPTIONS
send_error = 1
OTHERS = 2.
IF sy-subrc = 0.
“Retain details of background work processes only
DELETE lt_wplist WHERE wp_itype NE 4.
DESCRIBE TABLE lt_wplist LINES v_total.
“Delete the work processes in Waiting state
DELETE lt_wplist WHERE wp_istatus = 2.
DESCRIBE TABLE lt_wplist LINES v_utilz.
v_uperc = v_utilz * 100 / v_total.
“If utilization is within set threshold, proceed
IF v_uperc LE c_limit.
“<Generate background job>
v_jobs = v_jobs – 1.
IF v_jobs = 0.
EXIT.
ENDIF.
” If utilization crosses threshold, wait
ELSE.
WAIT UP TO v_delay SECONDS.
ENDIF.
ENDIF.
ENDDO.
Improvements
Please suggest alternate approaches you have used to handle similar situations at program level.
Hi Sreekumar,
The parallel processing solution provided by SAP (Calling a function module starting new task) takes care of prevention of overuse of resources with a few safeguard measures like...
(See Parallel Processing Jobs with Asynchronous RFC (SAP Library - Background Processing) )
Also we can implement a similar threshold like your TVARVC solution by setting up a new dedicated Server Group (See transaction RZ12) only for parallel processing. This new server group will utilize only a certain % of overall free WPs (say 70%), ensuring that the remaining % of WPs will always be left untouched to avoid gluttening the system.
This server group can be addressed in our programs using the keyword CALL FUNCTION STARTING NEW TASK DESTINATION IN GROUP.
There are other FMs like,
to monitor and report the available free WPs at runtime..
Check SAP Library - Background Processing for more info..
Valid point Yuvaraj and thanks for the detailed explanation. I also prefer Asynchronous RFC more often while building new solutions requiring parallel processing in real time.
But there can be cases where you cannot/need not use the Asynchronous RFC due to the nature of the logic to be executed. The above approach deals with background work process (night jobs) whereas Asynchronous RFC uses dialog work process (real time processing). Asynchronous RFC requires the logic to in a function module whereas this approach deals with background jobs (programs) that trigger child jobs for faster processing. So if the priority is to hold on to using the existing programs without introducing changes, reasons for that being long processing times or execution timelines (night jobs) then this approach might be useful. We can either use the existing SUBMIT or call function JOB_SUBMIT (if server group specification is required). The JOB_SUBMIT can use the RZ12 server groups if the job execution has to be limited to certain servers. By bringing the control to the program itself, it gives an opportunity for further customization. For instance, if a limited number of work processes (instead of percentage) have to be used for a program or a group of programs or if within the same server group different programs take different precedence and thereby different utilization thresholds which can be made dynamic based on the presence of each other.