Skip to Content
Technical Articles

Batch Job scheduling and Optimize Job Processing time

Introduction:

In this Blog, we will see how to execute a Program via a Batch Job and optimize the Batch Job execution time by splitting the number of records into different sets and executed each set via Multiple Batch Jobs.

The Number of records in each set and the Maximum number of jobs for this program that can be in Released / Active / Scheduled status can be entered via a Maintenance Table in TCode SM30.

I have considered a scenario to update Marketing Attributes in multiple Business Partners and provided pseudo code to update the Marketing Attributes in the Business Partners and pseudo code to schedule the program to update Marketing attributes as a Batch job and split the records to be processed in each Batch job and schedule several batch jobs accordingly. Using these pseudo codes developers can build their logic accordingly.

Steps:

  1. Define a Parameter Table with Two fields Parameter Name and Value and create a Maintenance View for this table and make entries in the Maintenance View for Number of records to be processed in each set of Batch Job and maximum number of batch jobs that can be in Released / Active / Scheduled status.
  2. In this example, I have provided pseudo code to update the Marketing Attributes in a Business Partner.
  3. The updation of Marketing attributes in the Business Partner will be done via Batch Jobs that would be scheduled via another report program. The number of Batch Jobs that would be scheduled is based on number of records to be updated and the entry in the Parameter table that holds both the Number of records to be processed in each set of Batch Job & Maximum number of batch jobs of this program that can be in Released / Active / Scheduled status.
  4. I have provided pseudo code for two programs that would help developers to build functionality accordingly:

         Batch Job Name: ZBP_ATTR_UPD_1, ZBP_ATTR_UPD_2 , ZBP_ATTR_UPD_3 etc.

Detailed Explanation of the Steps:

  1. Define a Parameter Table with Two fields Parameter Name and Value and create a Maintenance View for this table.

       Make entries in the Maintenance View for Number of records to be processed in each set of               Batch Job and maximum number of batch jobs that can be in Released / Active / Scheduled               status.

TCode: sm30

          Please Note:

  • Parameter JOB_SIZE indicates the number of records to be processed in each set of Batch Job.
  • Parameter MAX_JOB_COUNT indicates maximum number of batch jobs (for the Program ZBP_ATTR_UPD) that can be in Released / Active / Scheduled status.

 

2. Pseudo code to update the Marketing Attributes in a Business Partner

    Program: ZBP_ATTR_UPD.

    Logic:

  • Import the list of BPs to be updated and assign the various values to a work area and    insert into an internal table. The List of BPs to be updated is exported from another program ZSCH_JOB.
  • Use Function Module ‘CRM_MKTBP_SAVE_BP_LIST’ to update the Attributes into the  Business Partner.
*&---------------------------------------------------------------------*
*& Report ZBP_ATTR_UPD
*&---------------------------------------------------------------------*
*& In this report we update the Attribute values into Business Partners
*& and this report is scheduled via a Batch Job that is scheduled from
*& another Report Program ZSCH_JOB.
*&---------------------------------------------------------------------*
REPORT ZBP_ATTR_UPD.
* Data Declarations
************************************************************************
* In this section we give the Data Declarations
************************************************************************
TYPES :     BEGIN OF TY_BP_NUM,
              PARTNER      TYPE BU_PARTNER,
              PARTNER_GUID TYPE BU_PARTNER_GUID,
            END OF TY_BP_NUM.
DATA :      LT_ATTR_LIST  TYPE TABLE OF CRMT_MKTBP_BP_VALUES,
            LT_ATTR_VAL   TYPE TABLE OF CRMT_MKTBP_ATTRIBUTE_VALUES,
            LT_BP_NUM1    TYPE STANDARD TABLE OF TY_BP_NUM,
            LT_BP_NUM2    TYPE STANDARD TABLE OF TY_BP_NUM,
            LT_RETURN     TYPE TABLE OF BAPIRET2,
            WA_ATTR_LIST  LIKE LINE of LT_ATTR_LIST,
            WA_ATTR_VAL   LIKE LINE OF LT_ATTR_VAL,
            WA_BP_NUM1    LIKE LINE OF LT_BP_NUM1,
            LV_MEMORY_ID  TYPE CHAR10.
CONSTANTS : LC_X          TYPE C       VALUE 'X',
            LC_BUT000     TYPE TABELLE VALUE 'BUT000'.
* Assigning the Memory id
  lv_memory_id = 'ZMEMORY'.
*************************************************************************
* In this section we assigning the Attribute and Attribute value and
* Insert into the Internal table.
*************************************************************************
* Assigning
  WA_ATTR_VAL-ATNAME          = 'ZATTRIBUTE'.
  WA_ATTR_VAL-ATWRT           = 'VALUE_1'.
* Inserting into Internal table
  INSERT WA_ATTR_VAL INTO TABLE lt_attr_val.
* Clearing
  CLEAR WA_ATTR_VAL.
* Assigning
  WA_ATTR_VAL-ATNAME         = 'ZATTRIBUTE'.
  WA_ATTR_VAL-ATWRT          = 'VALUE_2'.
* Inserting into Internal table
  INSERT WA_ATTR_VAL INTO TABLE lt_attr_val.
* Clearing
  CLEAR WA_ATTR_VAL.
* Assigning
  WA_ATTR_VAL-ATNAME         = 'ZATTRIBUTE'.
  WA_ATTR_VAL-ATWRT          = 'VALUE_3'.
* Inserting into Internal table
  INSERT WA_ATTR_VAL INTO TABLE lt_attr_val.
* Clearing
  CLEAR WA_ATTR_VAL.
*************************************************************************
* In this section we import the list of BPs to be updated and assign
* the various values to a work area and Insert into an internal table
*************************************************************************
* Importing the List of BPs to be updated from Memory
  IMPORT LT_BP_NUM1 TO LT_BP_NUM1 FROM SHARED BUFFER indx(st) ID lv_memory_id.
* Looping at the Internal table
  LOOP AT LT_BP_NUM1 INTO WA_BP_NUM1.
*   Assigning to various fields of Work area
    WA_ATTR_LIST-PARTNER_GUID    = WA_BP_NUM1-PARTNER_GUID.
    WA_ATTR_LIST-PARTNER         = WA_BP_NUM1-PARTNER.
    WA_ATTR_LIST-ATTRIBUTE_SET   = 'ZATTRIBUTE_SET'.
    WA_ATTR_LIST-ALLOCVALUES     = lt_attr_val.
*   Inserting into the Internal table
    INSERT WA_ATTR_LIST INTO TABLE lt_attr_list.
*   Clearing
    CLEAR : WA_ATTR_LIST,
            WA_BP_NUM1.
  ENDLOOP.                      "End of LOOP AT LT_BP_NUM1 INTO WA_BP_NUM1.
*************************************************************************
* In this section we call the Function Module to Update the Attributes
* into the Business Partner.
*************************************************************************
* Calling a Function Module to update the attributes
  CALL FUNCTION 'CRM_MKTBP_SAVE_BP_LIST'
    EXPORTING
      IV_MSA                  = ' '
      IV_COMMIT               = LC_X                 "Constant LC_X is 'X'
      it_alloclist_bp         = LT_ATTR_LIST
      IV_CONVERT_VALUES       = LC_X                 "Constant LC_X is 'X'
      IV_OBJTYP               = LC_BUT000            "Constant LC_BUT000 is BUT000
      IV_SET_LOCK             = LC_X                 "Constant LC_X is 'X'
      IV_WRITE_ALE            = LC_X                 "Constant LC_X is 'X'
    TABLES
      ET_RETURN               = LT_RETURN.
*Committing
 CALL FUNCTION 'BAPI_TRANSACTION_COMMIT'
   EXPORTING
     WAIT                    = 'X'.
*Displaying the output
 IF sy-subrc = 0.
    WRITE 'Attributes successfully updated for the Business Partner'.
    LOOP AT LT_BP_NUM1 INTO WA_BP_NUM1.
      WRITE : / 'Partner Number Updated:' , WA_BP_NUM1-PARTNER.
      CLEAR WA_BP_NUM1.
    ENDLOOP.
 ELSE .
    WRITE 'Attributes not successfully updated for the Business Partner'.
 ENDIF.

3. Pseudo code Report Program to split the records to be updated into multiple sets depending on          Maximum number of records permitted in a Job and schedule Jobs for each set accordingly.

    Program: ZSCH_JOB

    Logic:

  • First select data from the database table that holds the Parameter values for Maximum             number of Records in a Job and maximum number of Jobs that can run at a time.
  • Then select data( that is Business Partners whose Marketing attributes needs to be updated ) from the database table and validate whether the number of records is more than (or) equal to the job size and accordingly pass the to a subroutine to schedule a Job to update the records. Also, delete the records that are being passed to the job(scheduled in the subroutine) from another internal table and use this set to schedule Job for final set of records.
  • In the subroutine validate for any Jobs of the Program ZBP_ATTR_UPD and if the                    number of jobs in Released / Active / Scheduled status are more than the Permitted                  Job Count then wait for 10 seconds and validate again. If the number of jobs in                          Released / Active / Scheduled status are less than the permitted Job Count then                      schedule New Jobs accordingly.
*&---------------------------------------------------------------------*
*& Report ZSCH_JOB
*&---------------------------------------------------------------------*
*& This Report Program gives a high level pseudo code to Split the records
*& to be updated into multiple sets depending on Maximum number of records
*& permitted in a Job and schedule Jobs for each set accordingly.
*&---------------------------------------------------------------------*
REPORT ZSCH_JOB.
*************************************************************************
* In this Section we do the Data Declarations
*************************************************************************
DATA : LV_JOB_NAME      TYPE BTCJOB,
       LV_JOB_COUNT     TYPE BTCJOBCNT,
       LV_MEMORY_ID(10) TYPE c,
       LV_ACTIVE_JOBS   TYPE INT4,
       LV_JOB_SIZE      TYPE INT4,
       LV_MAX_JOB_COUNT TYPE INT4,
       lv_index         TYPE INT4,
       LV_LINES         TYPE INT4,
       LV_LINES2        TYPE INT4,
       LV_SCH           TYPE C.
TYPES : BEGIN OF TY_BP_NUM,
          PARTNER      TYPE BU_PARTNER,
          PARTNER_GUID TYPE BU_PARTNER_GUID,
        END OF TY_BP_NUM.
TYPES : BEGIN OF TY_PARAM,
          CLIENT       TYPE MANDT,
          PAR          TYPE ZPAR_NAME,
          VALUE        TYPE ZPAR_VALUE,
        END OF TY_PARAM.
DATA : LT_BP_NUM       TYPE STANDARD TABLE OF TY_BP_NUM,
       LT_BP_NUM1      TYPE STANDARD TABLE OF TY_BP_NUM,
       LT_BP_NUM2      TYPE STANDARD TABLE OF TY_BP_NUM,
       LT_PAR          TYPE STANDARD TABLE OF TY_PARAM,
       WA_BP_NUM       LIKE LINE OF LT_BP_NUM,
       WA_BP_NUM1      LIKE LINE OF LT_BP_NUM,
       WA_BP_NUM2      LIKE LINE OF LT_BP_NUM,
       WA_PAR          LIKE LINE OF LT_PAR.
*************************************************************************
* In this Section we select data from the database table that holds
* the Parametre values for Maximum number of Records in a Job and
* Maximum number of Jobs that can run at a time. This values in this
* table can be updated via TCode sm30.
*************************************************************************
SELECT * FROM ZPAR INTO TABLE LT_PAR.
* Checking the value of sy-subrc
IF sy-subrc EQ 0.
*  Reading the Internal table
   READ TABLE LT_PAR INTO WA_PAR WITH KEY PAR = 'JOB_SIZE'.
*  Checking the value of sy-subrc
   IF sy-subrc EQ 0.
*     Assigning to a variable
      LV_JOB_SIZE      = WA_PAR-VALUE.
   ENDIF.                     "End of if sy-subrc EQ 0
*  Clearing
   CLEAR WA_PAR.
*  Reading the Internal table
   READ TABLE LT_PAR INTO WA_PAR WITH KEY PAR = 'MAX_JOB_COUNT'.
*  Checking the value of sy-subrc
   IF sy-subrc EQ 0.
*     Assigning to a variable
      LV_MAX_JOB_COUNT = WA_PAR-VALUE.
   ENDIF.                      "End of if sy-subrc EQ 0
*  Assigning values
   LV_INDEX         = 1.
   LV_MEMORY_ID     = 'ZMEMORY'.
*****************************************************************************
* In this Section we select data from the database table and call
* the Subroutine to schedule Jobs accordingly.We first validate whether
* the number of records is more than the job size and acordingly pass the
* to a subroutine to schedule a Job to update the records. Also, we
* delete the records that are being passed to the job from another
* Internal table and use this set to schedule Job for final set of records.
*****************************************************************************
*  Selecting from database table into an Internal table
   SELECT PARTNER PARTNER_GUID FROM BUT000 INTO TABLE LT_BP_NUM.
*    Assigning the Internal table
     LT_BP_NUM2[] = LT_BP_NUM[].
*  Looping at the Internal table
   LOOP AT LT_BP_NUM INTO WA_BP_NUM.
*     Inserting into the Internal table
      INSERT WA_BP_NUM INTO TABLE LT_BP_NUM1.
*     Checking if the Lines in the Internal table is more than the Maximum number
*     of records allowed in a Job
      IF ( ( LINES( LT_BP_NUM1 ) >= LV_JOB_SIZE ) ).
*         Looping at the Internal table
          LOOP AT LT_BP_NUM1 INTO WA_BP_NUM1.
*            Deleting from the Internal table
             DELETE TABLE LT_BP_NUM2 FROM WA_BP_NUM1.
*            Clearing
             CLEAR WA_BP_NUM1.
          ENDLOOP.            "End of LOOP AT LT_BP_NUM1 INTO WA_BP_NUM1.
*        Calling a Subrouting to Schedule Job
         PERFORM job_process USING lv_index.
*        Incrementing the Index
         ADD 1 TO lv_INDEX.
      ENDIF.                  "End of IF ( ( LINES( LT_BP_NUM1 ) >= LV_JOB_SIZE ) ).
*     Checking the records remaining to be processed
      LV_LINES2 = LINES( LT_BP_NUM2 ).
*     Checking if the number of records to be processed are less than the
*     Maximum number of records in a Job and if the Number of records to
*     be processed is not equal to Zero.
      IF ( ( LV_LINES2 < LV_JOB_SIZE ) AND ( LV_LINES2 NE '0' ) ).
*        Assigning
         LT_BP_NUM1[] = LT_BP_NUM2[].
*        Calling a Subrouting to Schedule Job
         PERFORM job_process USING lv_index.
*        Incrementing the Index
         ADD 1 TO lv_INDEX.
*        Exiting as All records have been sent to Job for processing
         EXIT.
      ENDIF.                  "End of IF ( ( LV_LINES2 < LV_JOB_SIZE ) AND ( LV_LINES2 NE '0' ) ).
*     Clearing the Work area
      CLEAR WA_BP_NUM.
   ENDLOOP.                   "End of LOOP AT LT_BP_NUM INTO WA_BP_NUM.
*  Checking if the variable has bee set
   IF LV_SCH = 'X'.
*     Displaying the Output
      WRITE 'Job Scheduled for Processing'.
   ENDIF.                     "End of IF LV_SCH = 'X'.
ENDIF.                        "End of IF sy-subrc EQ 0.
*&---------------------------------------------------------------------*
*&      Form  JOB_PROCESS
*&---------------------------------------------------------------------*
* In this subroutine we check if validate Jobs for this program based on
* status and we sc hedule Jobs accordingly
*----------------------------------------------------------------------*
*      -->P_LV_INDEX  This holds the Index Value
*----------------------------------------------------------------------*
FORM JOB_PROCESS  USING    P_LV_INDEX.
* Data Declarations
  DATA :      LV_PROGRAM TYPE SY-REPID,
              LT_JOBLIST TYPE tbtcjob_tt,
              LV_INDEX   TYPE CHAR10.
  CONSTANTS : c_running  TYPE tbtco-status VALUE 'R',
              c_unk_st   TYPE tbtco-status VALUE 'X',
              c_active   TYPE tbtco-status VALUE 'Z',
              c_aborted  TYPE tbtco-status VALUE 'A',
              c_sch      TYPE tbtco-status VALUE 'P',
              c_ready    TYPE tbtco-status VALUE 'Y',
              c_rel      TYPE tbtco-status VALUE 'S',
              c_fin      TYPE tbtco-status VALUE 'F'.
* Assigning the Program to a Variable
  LV_PROGRAM =  'ZBP_ATTR_UPD'.
*************************************************************************************
* In this section we validate for any Jobs of the Program and if the number of
* Active Jobs are more than the Permitted Job Count then we wait for 10 seconds
* and validate again. If the number of Active Jobs are less than the
* Permitted Job Count then we schedule New Jobs accordingly.
*************************************************************************************
  DO.
    CALL FUNCTION 'BP_FIND_JOBS_WITH_PROGRAM'
      EXPORTING
        ABAP_PROGRAM_NAME                   = LV_PROGRAM
      TABLES
        JOBLIST                             = LT_JOBLIST
      EXCEPTIONS
        NO_JOBS_FOUND                       = 1
        PROGRAM_SPECIFICATION_MISSING       = 2
        INVALID_DIALOG_TYPE                 = 3
        JOB_FIND_CANCELED                   = 4
        OTHERS                              = 5.
    IF ( ( SY-SUBRC <> 0 ) AND ( P_LV_INDEX <> 1 ) ) .
      RETURN.
    ENDIF.
*   Deleting Jobs that are not in a Particular status
    DELETE LT_JOBLIST WHERE NOT ( status = c_rel OR status = c_running OR status = c_sch ).
*   Getting the Count of active Jobs
    lv_active_jobs = lines( lt_joblist ).
*   Checking if the Number of active Jobs is less than the Maximum permitted Job count
    IF lv_active_jobs <= LV_MAX_JOB_COUNT.
*     Exiting the Loop to schedule the Job
      EXIT.
    ENDIF.                    "End of IF lv_active_jobs <= LV_MAX_JOB_COUNT.
*   Wait for 10 seconds and validate the Jobs again
    WAIT UP TO 10 SECONDS.
  ENDDO.                      "End of DO
* Assigning the avalue. This will be concatenated to the Jobname.
  LV_INDEX = P_LV_INDEX.
* Concatenating for the Jobname and Condensing
  CONCATENATE LV_PROGRAM LV_INDEX INTO lv_job_name SEPARATED BY '_'.
  CONDENSE lv_job_name NO-GAPS.
*************************************************************************************
* In this section we schedule the Job and Export the values to be used in the
* Program that is being scheduled as a Job.
*************************************************************************************
  CALL FUNCTION 'JOB_OPEN'
    EXPORTING
      jobname  = lv_job_name
    IMPORTING
      jobcount = lv_job_count
    EXCEPTIONS
      OTHERS = 1.
  IF ( SY-SUBRC <> 0 ).
    RETURN.
  ENDIF.
* Exporting
  EXPORT LT_BP_NUM1 to SHARED BUFFER indx(st) id lv_memory_id.
* Submitting the Program to be executed as a Job
  SUBMIT ZBP_ATTR_UPD  VIA JOB lv_job_name NUMBER lv_job_count AND RETURN.
* Closing the Job
  CALL FUNCTIOn 'JOB_CLOSE'
    EXPORTING
      jobcount  = lv_job_count
      jobname   = lv_job_name
      strtimmed = abap_true
    EXCEPTIONS
      OTHERS = 1.
* Checking the value of sy-subrc
  IF sy-subrc EQ 0.
*    Assigning
     LV_SCH = 'X'.
  ENDIF.                      "End of IF sy-subrc EQ 0.
  WAIT UP TO 5 SECONDS.
* Clearing
  CLEAR : LT_BP_NUM1 ,
          LT_BP_NUM1[].
ENDFORM.

 

Scenario 1:

List of Business Partners whose Marketing Attributes needs to be updated:

Before Execution, no Attributes present in the Marketing Attributes Tab of the Business Partners:

Parameter Table screenshot:

sm37 screenshot:

Now execute the Program ZSCH_JOB via the TCode for this Program ZSCH_JOB_TCODE.

Output:

Now check in SM37 for the Batch jobs:

Spool of Job ZBP_ATTR_UPD_1:

Spool of Job ZBP_ATTR_UPD_2:

Spool of Job ZBP_ATTR_UPD_3:

Business Partners have been updated as per below screenshot:

Now delete the Assigned attributes from the Business Partners for test data for Scenario 2.

Sample BP screenshot with assigned Attributes Set, Attributes deleted:

In a similar way delete all assigned Attributes Set, Attributes to all the Business Partners.

Parameter Table screenshot:

SM37 Screenshot before execution:

Now, execute the TCode ZSCH_JOB_TCODE to schedule the Jobs to Update the Business Partners.

Output:

SM37 Screenshot:

Spool of Job ZBP_ATTR_UPD_1:

Spool of Job ZBP_ATTR_UPD_2:

Screenshot of BPs after Execution. All the Business Partners have been updated with the Marketing Attributes:

Conclusion:

By Maintaining the values in the Parameter table the number of records to be processed in each Batch Job and Maximum number of Batch Jobs for this Program in specific status can be controlled. By using the above pseudo codes provided for the programs developers can build their logic accordingly to schedule Batch Jobs from report Programs and optimize Batch Job performance.

 

End of Document

 

 

5 Comments
You must be Logged on to comment or reply to a post.
  • Thank you for your contribution Bala Subramanian.

    Here are a couple recommendations for improvement:

    • The Report ZSCH_JOB should not be involved in the data selection process. It should only contain a framework to split records and create separate jobs based on customizing and/or input parameters.
    • I would therefore rather implement the functionality of ‘ZSCH_JOB’ as an Include or a Class that can be used from within any other main programm that requires parallelization of tasks.

     

    • That framework could also give the option of either scheduling parallel Jobs or starting parallel Function Module Calls with option STARTING NEW TASK.
    • The SUBMIT of the Report or the CALL FUNCTION call of the FM is done from within the main programme and the framework helps with setting up the parallelization, preparing the next parall process (job or task), closing/releasing the job, and optionally watching/waiting for the parallel processes to end (in order to evaluate the results in the main programm, if necessary).
    • During the setup of parallelization, the framework also needs to check whether the required or necessary amount of processes are currently available for a selected server or server group
    • If the main programme has a selection screen, an Include for selection parameters for the parallelization could also be provided by the framework, which would eliminate customizing tables for the setup of the parallelization.
    • When splitting up records, there should also be an option for grouping records together by group criterias, in case certain records have to be processed together in one process and shouldnt be split up across separate processes.
    • Dear Michael,

      Thanks for the suggestion..

      In this blog, my intention was to give a high level pseudo code ( with an example ) for developers to help them build their logic accordingly.

      I am writing another blog on optimization and I am incorporating more validations / checks that includes your comments.

      Regards,

      Bala

       

       

       

  • Check the SAP-standard transactions WLCPARC, WLCPAR which offer a similar functionality (external parallel processing) for almost any report.

  • This is very good blog, The explanation is very detailed and even a fresher can understand it easily.  I appreciate your efforts, keep write more blogs.