Purpose & Overview: We all know how to upload data into SAP but many of us not aware of best possible way to upload the data in to sap. Now, if you are dealing with small volume of data then there will be no problem but when it comes to huge volume of data then execution time is important.
So, what will be best possible way?
There are different solutions available. But SAP has provided wonderful solutions called ‘Parallel Processing’. There are two types of parallel processing available.
- With Release 3.1G, SAP offers a solution to the “short nights” problem: parallel-processed background jobs. Long-running SAP reports can now implement parallel processing, which lets them parcel out the work to be done to available dialog work processes in the SAP system and then collect the results.Parallel-processing is implemented with a special variant of asynchronous RFC. It’s important that you use only the correct variant for your own parallel processing applications: the CALL FUNCTION STARTING NEW TASK DESTINATION IN GROUP keyword.
- Another one is the best one and most effective one. SPTA Framework. Performance wise this is best possible parallel process. It takes minimum time to execute compare to other parallel processes.
My attempt is to provide a good example of both and give a comparison between two parallel processes.
Business Requirement: In this example for both the cases we have to upload data from legacy file to Z table in SAP.
Lets’ Start with Process 1:
Step1: First create a Z table: This is the table where we will update the data.
Step2- Create Function Module: We will create a function module, through which we will upload the data. Create import export parameter.
Step3: Function Module Coding:
FUNCTION zemp_data_upd1.
*"----------------------------------------------------------------------
*"*"Local Interface:
*" IMPORTING
*" VALUE(IM_DATA) TYPE ZEMP_DATA OPTIONAL
*" EXPORTING
*" VALUE(EX_RESULT) TYPE ZUPD_RESULT
*"----------------------------------------------------------------------
IF NOT im_data IS INITIAL.
INSERT zemp_data FROM im_data.
IF sy-subrc EQ 0.
ex_result-empid = im_data-empid.
ex_result-message = 'Updated'.
ELSE.
ex_result-empid = im_data-empid.
ex_result-message = 'Not-Updated'.
ENDIF.
COMMIT WORK.
ENDIF.
ENDFUNCTION.
Step4: Build the report
Report ZABHI_PARA_PROCESS_1.
TABLES: zemp_data.
TYPES:
ty_result TYPE zupd_result,
tty_result TYPE STANDARD TABLE OF zupd_result,
tty_emp TYPE STANDARD TABLE OF zemp_data.
DATA:
gt_emp TYPE tty_emp,
gt_result TYPE tty_result.
DATA: gv_snd_task TYPE i,
gv_ptask TYPE i,
gv_rcv_task TYPE i.
FIELD-SYMBOLS: <gfs_result> TYPE zupd_result.
SELECTION-SCREEN BEGIN OF BLOCK b1 WITH FRAME TITLE text-001.
PARAMETERS:
p_rfcgr TYPE spta_rfcgr OBLIGATORY MEMORY ID spta_rfcgr,
p_file TYPE rlgrap-filename.
SELECTION-SCREEN END OF BLOCK b1.
- INITIALIZATION.
* Not just anybody may execute this report
AUTHORITY-CHECK OBJECT 'S_ADMI_FCD'
ID 'S_ADMI_FCD' FIELD 'PADM'.
IF NOT sy-subrc IS INITIAL.
RAISE no_authority_for_report.
ENDIF.
AT SELECTION-SCREEN ON VALUE-REQUEST FOR p_file .
PERFORM f_get_file .
START-OF-SELECTION.
PERFORM f_sub_get_data USING p_file
CHANGING gt_emp.
DELETE FROM zemp_data WHERE empid NE space.
IF sy-subrc = 0.
COMMIT WORK.
ENDIF.
PERFORM f_sub_upload_data USING gt_emp
CHANGING gt_result.
END-OF-SELECTION.
WRITE: 'EMPLOYEE DETAILS'.
SKIP.
LOOP AT gt_result ASSIGNING <gfs_result>.
WRITE:/ 'Emp-Id:', <gfs_result>-empid.
WRITE: 'Status:', <gfs_result>-message.
ENDLOOP.
*&---------------------------------------------------------------------*
*& Form F_GET_FILE
*&---------------------------------------------------------------------*
* text
*----------------------------------------------------------------------*
* --> p1 text
* <-- p2 text
*----------------------------------------------------------------------*
FORM f_get_file .
CALL FUNCTION 'F4_FILENAME'
EXPORTING
program_name = syst-cprog
dynpro_number = syst-dynnr
field_name = ' '
IMPORTING
file_name = p_file.
ENDFORM. " F_GET_FILE
*&---------------------------------------------------------------------*
*& Form F_SUB_GET_DATA
*&---------------------------------------------------------------------*
* text
*----------------------------------------------------------------------*
* -->P_P_FILE text
* <--P_gt_emp text
*----------------------------------------------------------------------*
FORM f_sub_get_data USING p_file TYPE localfile
CHANGING p_gt_tab TYPE tty_emp.
DATA: ls_work_area TYPE tty_emp,
lv_filename TYPE string.
CLEAR lv_filename.
MOVE p_file TO lv_filename.
CALL FUNCTION 'GUI_UPLOAD'
EXPORTING
filename = lv_filename
filetype = 'ASC'
has_field_separator = 'X'
TABLES
data_tab = p_gt_tab
EXCEPTIONS
file_open_error = 1
file_read_error = 2
no_batch = 3
gui_refuse_filetransfer = 4
invalid_type = 5
no_authority = 6
unknown_error = 7
bad_data_format = 8
header_not_allowed = 9
separator_not_allowed = 10
header_too_long = 11
unknown_dp_error = 12
access_denied = 13
dp_out_of_memory = 14
disk_full = 15
dp_timeout = 16
OTHERS = 17.
IF sy-subrc <> 0.
* Implement suitable error handling here
ENDIF.
ENDFORM. " F_SUB_GET_DATA
*&---------------------------------------------------------------------*
*& Form F_SUB_UPLOAD_DATA
*&---------------------------------------------------------------------*
* text
*----------------------------------------------------------------------*
* -->P_GT_EMP text
* <--P_GT_RESULT text
*----------------------------------------------------------------------*
FORM f_sub_upload_data USING p_gt_emp TYPE tty_emp
CHANGING p_gt_result TYPE tty_result.
DATA: lv_lines TYPE i,
ls_result TYPE zupd_result,
ls_data TYPE zemp_data,
lv_msg(80) TYPE c,
lv_taskname TYPE numc10 VALUE '0',
lv_excp_flag TYPE flag.
FIELD-SYMBOLS: <lfs_emp> TYPE zemp_data.
CLEAR:ls_data,
ls_result,
gv_snd_task,
gv_rcv_task,
lv_lines.
DESCRIBE TABLE p_gt_emp LINES lv_lines.
*
LOOP AT p_gt_emp ASSIGNING <lfs_emp>.
gv_ptask = gv_ptask + 1.
MOVE <lfs_emp> TO ls_data.
CLEAR: lv_excp_flag.
DO.
ADD 1 TO lv_taskname.
CLEAR lv_excp_flag.
CALL FUNCTION 'ZEMP_DATA_UPD1'
STARTING NEW TASK lv_taskname
DESTINATION IN GROUP p_rfcgr
PERFORMING process_callback_prog ON END OF TASK
EXPORTING
im_data = ls_data
EXCEPTIONS
communication_failure = 1 MESSAGE lv_msg
system_failure = 2 MESSAGE lv_msg
resource_failure = 3 "No work processes are
OTHERS = 4. "Add exceptions generated by
*the called function module here. Exceptions are returned to you and you can
* respond to them here.
CASE sy-subrc.
WHEN 0.
ADD 1 TO gv_snd_task.
WHEN 1 OR 2.
CLEAR: ls_result.
MOVE <lfs_emp>-empid TO ls_result-empid.
MOVE 'Not_Updated' TO ls_result-message.
APPEND ls_result TO p_gt_result.
CLEAR ls_result.
WHEN 3.
lv_excp_flag = 'X'.
WAIT UNTIL gv_rcv_task >= gv_snd_task UP TO '1' SECONDS.
WHEN OTHERS.
CLEAR ls_result.
ENDCASE.
IF lv_excp_flag IS INITIAL.
EXIT.
ENDIF.
ENDDO.
ENDLOOP.
*--- wait till everything ends
WAIT UNTIL gv_rcv_task >= gv_snd_task UP TO 10 SECONDS.
ENDFORM. " F_SUB_UPLOAD_LINK
*&---------------------------------------------------------------------*
*& Form PROCESS_UPLOAD_LINK
*&---------------------------------------------------------------------*
* text
*----------------------------------------------------------------------*
*----------------------------------------------------------------------*
FORM process_callback_prog USING gv_task.
DATA: ls_result TYPE zupd_result.
gv_ptask = gv_ptask - 1.
RECEIVE RESULTS FROM FUNCTION 'ZEMP_DATA_UPD1'
IMPORTING
ex_result = ls_result
EXCEPTIONS
no_update = 1
OTHERS = 2.
gv_rcv_task = gv_rcv_task + 1.
APPEND ls_result TO gt_result.
CLEAR ls_result.
- ENDFORM. " PROCESS_CALLBACK_PROG
Step 5- Test the program: Activate the program and then execute it. Select server group from existing otherwise you can create a Z server group by executing transaction code RZ12 - RFC Server Group Maintenance.
Step 5: Execute it and see the result: Employee details in Ztable will be updated and the report is also generate.
*------------------------------------------------------------------------PROCESS 1 IS COMPLETE ----------------------------------------------------------------------------------------------------------------*
Lets’ Start Process 2:
Step1-Creation of Ztable: For this process also, we will use same table to update employee details:
Step2- Create Function Module: We will create a function module, through which we will upload the data. Create import export parameter.
Step3: Function Module Coding:
See the difference of two function module. In this FM no COMMIT WORK inside the function module.
FUNCTION zemp_data_upd.
*"----------------------------------------------------------------------
*"*"Local Interface:
*" IMPORTING
*" VALUE(IM_DATA) TYPE ZEMP_DATA OPTIONAL
*" EXPORTING
*" VALUE(EX_RESULT) TYPE ZUPD_RESULT
*"----------------------------------------------------------------------
IF NOT im_data IS INITIAL.
INSERT zemp_data FROM im_data.
IF sy-subrc EQ 0.
ex_result-empid = im_data-empid.
ex_result-message = 'Updated'.
ELSE.
ex_result-empid = im_data-empid.
ex_result-message = 'Not-Updated'.
ENDIF.
ENDIF.
ENDFUNCTION.
Step4: Build the report:
REPORT zabhi_para_process_2 .
*---- ZEMP_DATA - This table is getting updated through this process
TABLES: zemp_data.
TYPE-POOLS: spta.
* Define a type that contains ALL the input & output data
* needed for the RFC
TYPES:
ty_result TYPE zupd_result,
tty_result TYPE STANDARD TABLE OF zupd_result,
tty_emp TYPE STANDARD TABLE OF zemp_data.
DATA:
gt_emp TYPE tty_emp,
gt_result TYPE tty_result,
gv_repid TYPE syrepid,
gv_error_count TYPE syindex.
TYPES: BEGIN OF gty_rfcdata,
BEGIN OF importing,
emp_data LIKE gt_emp , " TYPE ty_emp,
END OF importing,
BEGIN OF exporting,
emp_result LIKE gt_result , "TYPE ty_result,
END OF exporting,
END OF gty_rfcdata.
FIELD-SYMBOLS: <gfs_result> TYPE zupd_result.
SELECTION-SCREEN BEGIN OF BLOCK b1 WITH FRAME TITLE text-001.
PARAMETERS:
p_rfcgr TYPE spta_rfcgr OBLIGATORY MEMORY ID spta_rfcgr,
p_maxtak LIKE sy-index DEFAULT '10',
p_start LIKE sy-index DEFAULT '1',
p_end LIKE sy-index DEFAULT '100',
p_file TYPE rlgrap-filename.
SELECTION-SCREEN END OF BLOCK b1.
- INITIALIZATION.
* Not just anybody may execute this report
AUTHORITY-CHECK OBJECT 'S_ADMI_FCD'
ID 'S_ADMI_FCD' FIELD 'PADM'.
IF NOT sy-subrc IS INITIAL.
RAISE no_authority_for_report.
ENDIF.
AT SELECTION-SCREEN ON VALUE-REQUEST FOR p_file .
PERFORM f_get_file .
START-OF-SELECTION.
PERFORM f_sub_get_data USING p_file
CHANGING gt_emp.
DELETE FROM zemp_data WHERE empid NE space.
IF sy-subrc = 0.
COMMIT WORK.
ENDIF.
gv_repid = sy-repid.
CLEAR: gv_error_count.
CALL FUNCTION 'SPTA_PARA_PROCESS_START_2'
EXPORTING
server_group = p_rfcgr
max_no_of_tasks = p_maxtak
before_rfc_callback_form = 'F_BEFORE_RFC'
in_rfc_callback_form = 'F_IN_RFC'
after_rfc_callback_form = 'F_AFTER_RFC'
callback_prog = gv_repid
EXCEPTIONS
invalid_server_group = 1
no_resources_available = 2
OTHERS = 3.
END-OF-SELECTION.
WRITE: 'EMPLOYEE DETAILS'.
SKIP.
LOOP AT gt_result ASSIGNING <gfs_result>.
WRITE:/ 'Emp-Id:', <gfs_result>-empid.
WRITE: 'Status:', <gfs_result>-message.
ENDLOOP.
*&---------------------------------------------------------------------*
*& Form F_before_rfc
*&---------------------------------------------------------------------*
* text
*----------------------------------------------------------------------*
* -->P_BEFORE_RFC_IMP text
* -->P_BEFORE_RFC_EXP text
* -->PT_RFCDATA text
* -->P_FAILED_OBJECTS text
* -->P_OBJECTS_IN_PROCESS text
* -->P_USER_PARAM text
*----------------------------------------------------------------------*
FORM f_before_rfc
USING
p_before_rfc_imp TYPE spta_t_before_rfc_imp
CHANGING
p_before_rfc_exp TYPE spta_t_before_rfc_exp
pt_rfcdata TYPE spta_t_indxtab
p_failed_objects TYPE spta_t_failed_objects
p_objects_in_process TYPE spta_t_objects_in_process
p_user_param.
DATA:
lv_package_size TYPE sytabix,
ls_task_data TYPE gty_rfcdata, " ***CUSTOM
ls_emp TYPE zemp_data, " ***CUSTOM
ls_failed_obj TYPE spta_t_failed_object,
ls_obj_in_process TYPE spta_t_pending_object.
FIELD-SYMBOLS:
<lfs_work> TYPE zemp_data.
* Delete list of objects in process
CLEAR ls_obj_in_process.
* Check if there are objects from previously failed tasks left ...
READ TABLE p_failed_objects INDEX 1 INTO ls_failed_obj.
IF sy-subrc = 0.
* Yes there are.
* Take first object and delete it from list of failed objects
DELETE p_failed_objects INDEX 1.
ls_obj_in_process = ls_failed_obj.
ls_emp-empid = ls_obj_in_process-obj_id(10).
APPEND ls_emp TO ls_task_data-importing-emp_data.
* Add list of objects that are about to be processed
* to list of "objects in process"
* so the task manager has that information
APPEND ls_obj_in_process TO p_objects_in_process.
ELSE.
* No there aren't.
* Take objects from regular input list of objects
* The number of objects that are processed at once is determined
* by the application. This sample coding here uses a dynamically
* determined package size (one 5th of remaining objects).
* In order to avoid extremly large or extremly small packages
* there is a maximum and minimum package size.
READ TABLE gt_emp ASSIGNING <lfs_work> INDEX 1.
IF sy-subrc IS INITIAL.
CLEAR ls_emp.
MOVE <lfs_work> TO ls_emp.
APPEND ls_emp TO ls_task_data-importing-emp_data.
ls_obj_in_process-obj_id(10) = ls_emp-empid.
DELETE gt_emp INDEX 1.
* Add list of objects that are about to be processed
* to list of "objects in process"
* so the task manager has that information
APPEND ls_obj_in_process TO p_objects_in_process.
ENDIF.
ENDIF.
* If there is (currently) nothing to do, clear the
* START_RFC field and leave the form.
* This informs the task manager that no rfc has to be started.
* If there are no more RFCs in process this also ends
* the processing of the task manager
* If there are still RFCs in process the BEFORE_RFC form
* will be invoked after each RFC has been received to give
* the application an opportunity to launch new RFCs that have been
* waiting on the RFC that was just received.
IF p_objects_in_process IS INITIAL.
CLEAR p_before_rfc_exp-start_rfc.
EXIT.
ENDIF.
* Convert the input data into the INDX structure
* that is needed for the RFC
CALL FUNCTION 'SPTA_INDX_PACKAGE_ENCODE'
EXPORTING
data = ls_task_data
IMPORTING
indxtab = pt_rfcdata.
* Inform task manager that an RFC can be started from the
* data compiled
p_before_rfc_exp-start_rfc = 'X'.
ENDFORM. "BEFORE_RFC
*---------------------------------------------------------------------*
* FORM IN_RFC *
*---------------------------------------------------------------------*
* Callback-Form invoked within the RFC *
*---------------------------------------------------------------------*
FORM f_in_rfc
USING
p_in_rfc_imp TYPE spta_t_in_rfc_imp
CHANGING
p_in_rfc_exp TYPE spta_t_in_rfc_exp
p_rfcdata TYPE spta_t_indxtab.
DATA:
ls_taskdata TYPE gty_rfcdata,
ls_data TYPE zemp_data,
ls_result TYPE zupd_result.
FIELD-SYMBOLS:
<lfs_data> TYPE zemp_data.
* Force synchronous update
* This is the most efficient method for parallel processing
* since no update data will be written to the DB but rather
* stored in memory.
* This statement must be reissued after each COMMIT WORK !!!!
SET UPDATE TASK LOCAL.
* Unpack RFC input data (that has been packed in the BEFORE_RFC form)
CALL FUNCTION 'SPTA_INDX_PACKAGE_DECODE'
EXPORTING
indxtab = p_rfcdata
IMPORTING
data = ls_taskdata.
*--- Begin processing of RFC
*------ Prepare your import data from -importing-workarea ( can define as per req )
CLEAR ls_data.
READ TABLE ls_taskdata-importing-emp_data ASSIGNING <lfs_data> INDEX 1.
IF sy-subrc EQ 0.
MOVE <lfs_data> TO ls_data.
ENDIF.
*------ Prepare your import data from -importing-workarea ( can define as per req )
CALL FUNCTION 'ZEMP_DATA_UPD'
EXPORTING
im_data = ls_data
IMPORTING
ex_result = ls_result
EXCEPTIONS
no_update = 1
OTHERS = 2.
IF sy-subrc <> 0.
* Implement suitable error handling here
ENDIF.
* Fill result tables
* This would include data for result lists, message handler etc.
* lS_taskdata-exporting-workarea = lS_taskdata-importing-workarea.
* Clear all data that is unnecessary for the AFTER_RFC form
* This keeps the amount of data transfered over the network
* small and makes the RFC more efficient!
REFRESH: ls_taskdata-exporting-emp_result[].
APPEND ls_result TO ls_taskdata-exporting-emp_result.
* Repack output data for AFTER_RFC form
CALL FUNCTION 'SPTA_INDX_PACKAGE_ENCODE'
EXPORTING
data = ls_taskdata
IMPORTING
indxtab = p_rfcdata.
* Don't forget to COMMIT your data, because if you don't, the
* RFC will end with an automatic rollback and data written to the
* database will be lost.
COMMIT WORK.
ENDFORM. "F_in_rfc
*&---------------------------------------------------------------------*
*& Form F_after_rfc
*&---------------------------------------------------------------------*
* text
*----------------------------------------------------------------------*
* -->P_RFCDATA text
* -->P_RFCSUBRC text
* -->P_RFCMSG text
* -->P_OBJECTS_IN_PROCESS text
* -->P_AFTER_RFC_IMP text
* -->P_AFTER_RFC_EXP text
* -->P_USER_PARAM text
*----------------------------------------------------------------------*
FORM f_after_rfc
USING
p_rfcdata TYPE spta_t_indxtab
p_rfcsubrc TYPE sy-subrc
p_rfcmsg TYPE spta_t_rfcmsg
p_objects_in_process TYPE spta_t_objects_in_process
p_after_rfc_imp TYPE spta_t_after_rfc_imp
CHANGING
p_after_rfc_exp TYPE spta_t_after_rfc_exp
p_user_param.
DATA:
ls_obj_in_process TYPE spta_t_pending_object,
lv_tabsize TYPE sytabix,
ls_taskdata TYPE gty_rfcdata.
DATA:
ls_emp TYPE zemp_data.
IF p_rfcsubrc IS INITIAL.
* No RFC error occured
* Unpack RFC output data and add RFC-results to global data,
* e.g. output list, message handler etc.
CALL FUNCTION 'SPTA_INDX_PACKAGE_DECODE'
EXPORTING
indxtab = p_rfcdata
IMPORTING
data = ls_taskdata.
APPEND LINES OF ls_taskdata-exporting-emp_result
TO gt_result.
EXIT.
ENDIF.
* Error handling
DESCRIBE TABLE p_objects_in_process LINES lv_tabsize.
IF lv_tabsize = 1.
* The failed task contained one object
* Inform task manager not to resubmit objects
p_after_rfc_exp-no_resubmission_on_error = 'X'.
* about the nature of the error.
READ TABLE p_objects_in_process INDEX 1
INTO ls_obj_in_process.
IF sy-subrc NE 0.
CLEAR ls_obj_in_process.
ENDIF.
ELSE.
* The failed taks contained several objects.
* Enable resubmission to process objects individually.
CLEAR p_after_rfc_exp-no_resubmission_on_error.
ENDIF.
ENDFORM. "AFTER_RFC
*&---------------------------------------------------------------------*
*& Form F_GET_FILE
*&---------------------------------------------------------------------*
* text
*----------------------------------------------------------------------*
* --> p1 text
* <-- p2 text
*----------------------------------------------------------------------*
FORM f_get_file .
CALL FUNCTION 'F4_FILENAME'
EXPORTING
program_name = syst-cprog
dynpro_number = syst-dynnr
field_name = ' '
IMPORTING
file_name = p_file.
- ENDFORM. " F_GET_FILE
*&---------------------------------------------------------------------*
*& Form F_SUB_GET_DATA
*&---------------------------------------------------------------------*
* text
*----------------------------------------------------------------------*
* -->P_P_FILE text
* <--P_gt_emp text
*----------------------------------------------------------------------*
FORM f_sub_get_data USING p_file TYPE localfile
CHANGING p_gt_tab TYPE tty_emp.
DATA: ls_work_area TYPE tty_emp,
lv_filename TYPE string.
CLEAR lv_filename.
MOVE p_file TO lv_filename.
CALL FUNCTION 'GUI_UPLOAD'
EXPORTING
filename = lv_filename
filetype = 'ASC'
has_field_separator = 'X'
TABLES
data_tab = p_gt_tab
EXCEPTIONS
file_open_error = 1
file_read_error = 2
no_batch = 3
gui_refuse_filetransfer = 4
invalid_type = 5
no_authority = 6
unknown_error = 7
bad_data_format = 8
header_not_allowed = 9
separator_not_allowed = 10
header_too_long = 11
unknown_dp_error = 12
access_denied = 13
dp_out_of_memory = 14
disk_full = 15
dp_timeout = 16
OTHERS = 17.
IF sy-subrc <> 0.
REFRESH: p_gt_tab.
ENDIF.
ENDFORM. " F_SUB_GET_DATA
Step 5 - Test the program:
Activate the program and then execute it. Select server group from existing otherwise you can create a Z server group by executing transaction code RZ12 - RFC Server Group Maintenance.
For this case one extra selection field has been taken for defining maximum task.
We will delete first from the table and upload the same file which has been used earlier.
Step 5: Execute it and see the result: Employee details have been updated.
*----------------------------------------------------------------------------- End of Process 2---------------------------------------------------------------------------------------------------------------------------*
Now we are going to compare which parallel process performance is better. For better comparison, take large volume of data because in small volume of data you will not find the difference.
Go to SE30 or SAT and Compare the result:
Process 1:
Execute it :
Process 2:
Result:
See the time difference is very less. For better statistics use large volume of data.
Reference: Sap standard report for parallel processing.
Suggestion: Please feel free to suggest.
- SAP Managed Tags:
9 Comments
