How does the extraction api work in case you are extracting data using a self defined function module with generic delta?
Sometimes, if your application or requirement makes it necessary, you have to create your own function module to extract the data from whatever sap system to bw.
Therefore you normally make a copy of the standard fm ‘RSAX_BIW_GET_DATA_SIMPLE’. To make this new function work for you, you have to realize the following.

1.) the fm gets called by the extraction api for at least 2 times. First time is for initialization and from the 2nd time on it is for data extraction.
2.) right after the last data package (somehow you need to make sure that it is the last one) you have to raise the exception no_more_data.

Example with lots of comments

FUNCTION RSAX_BIW_GET_DATA_SIMPLE.
*”————————————————————————–
*”*”Lokale Schnittstelle:
*” IMPORTING
*” VALUE(I_REQUNR) TYPE SRSC_S_IF_SIMPLE-REQUNR
*” VALUE(I_DSOURCE) TYPE SRSC_S_IF_SIMPLE-DSOURCE OPTIONAL
*” VALUE(I_MAXSIZE) TYPE SRSC_S_IF_SIMPLE-MAXSIZE OPTIONAL
*” VALUE(I_INITFLAG) TYPE SRSC_S_IF_SIMPLE-INITFLAG OPTIONAL
*” VALUE(I_READ_ONLY) TYPE SRSC_S_IF_SIMPLE-READONLY OPTIONAL
*” TABLES
*” I_T_SELECT TYPE SRSC_S_IF_SIMPLE-T_SELECT OPTIONAL
*” I_T_FIELDS TYPE SRSC_S_IF_SIMPLE-T_FIELDS OPTIONAL
*” E_T_DATA STRUCTURE SFLIGHT OPTIONAL
*” EXCEPTIONS
*” NO_MORE_DATA
*” ERROR_PASSED_TO_MESS_HANDLER
*”————————————————————————–

* Auxiliary Selection criteria structure
DATA: L_S_SELECT TYPE SRSC_S_SELECT.

* Maximum number of lines for DB table
STATICS: S_S_IF TYPE SRSC_S_IF_SIMPLE,

* counter
S_COUNTER_DATAPAKID LIKE SY-TABIX,

* cursor
S_CURSOR TYPE CURSOR.

* Initialization mode (first call by SAPI) or data transfer mode
IF I_INITFLAG = SBIWA_C_FLAG_ON.

* the coding here will be processed the first time the fm gets called.
* it is used to populate static or global variables and to check wether
* it’s called by the right datasource or not.
ELSE. “Initialization mode or data extraction ?
* this part will be executed from the 2nd call on
* First data package -> OPEN CURSOR
IF S_COUNTER_DATAPAKID = 0.

* in case it is for the first data package, range tabs gets filled, a cursor will
* be opened or a initial dataset will be read from database into a global internal table
* this has to be done here, with the first data package, to avoid getting the same data with
* each call of the fm.
* additionally you can check wether you get select-options for your ‘generic delta’-field or not,
* and, if it is provided, you are able to set a flag in order to do different selections
* based on this flag for delta or full load.
ENDIF. “First data package ?
* from now on records gets fetched from the database or gets read from the internal table
* and the return table e_t_data gets populated with the number of rows given by parameter s_s_if-maxsize.
* if the last record is populated to table e_t_data you need to raise the exeption no_more_data
* and you need to close the cursor, if you opened one.
* with each data package you also have to increase s_counter_datapakid.
ENDIF. “Initialization mode or data extraction ?

ENDFUNCTION.

If you forget to raise no_more_data, the extraction api will call the fm forever.

To report this post you need to login first.

46 Comments

You must be Logged on to comment or reply to a post.

  1. Arturo Sánchez

    Hi Siegfried,

    Your example is very good, but I don’t understand manage of delta load, this is a work of BW or I need to develop it in my FM? I was created a FM for generic extraction, with a Extract Structure view (and fill it a select condition tab with because a need to save DB space:
    table   field   operator Comparative Value
    EKPO     AEDAT     GE       ‘20051001’).

    The FM ran wrong because the delta is mayor than the initialization delta load. 

    I have only a difference than the standard FM this manage is correct? I only need the information by the last 62 days.

      data: f_extra = sy-datum – 62.
          SELECT (S_S_IF-T_FIELDS) FROM EKPO
                          WHERE AEDAT   >= f_extra.

    Can U send me an example code of a FM with a few fields?

    Thanks for your help.

    Regards,
    Arturo

    (0) 
  2. Mariem GHOUAIEL
    Hello,
    I have a different problem whith FM and DS :
    We use the 0MATERIAL_LKLS_HIER hierarchy datasource to load material hierachy in BW.
    I had to modify the function module CLBW_HIERARCHY_TRANSFER_LKLS that creates the hierarchy.
    Now, i need to create a new datasource based on the new function module (Z_CLBW_HIERARCHY_TRANSFER_LKLS).
    To do so, i created a generic datasource (with SBIW), a master data attributes datasource.
    I tried to put the same parameters that in 0MATERIAL_LKLS_HIER, but i didn’t find how to indicate that it’s a hierarchy and some other parameters (as you can see in RSA2 for 0MATERIAL_LKLS_HIER in the “semantic attributes” area).
    Also, i couldn’t run the new datasource in RSA3 because it didn’t show me the right selection fields.
    Then i tried BW10 but it generated a datasource for application component FI-SL-IO (internal name ROOTYPEHC) and with the extractor GSBH_HIERARCHY_TRANSFER_SETS, when what i need is a datasource for application component LO-IO (internal name 0MATERIAL) and with extractor Z_…
    Have you any idea form me ?
    thanks.
    MGH
    (0) 
  3. rishi malhotra
    Hi Siegfried,

    I am not able to manage delta loads in my FM.

    I need to have deltas for the last 3 days ( sy-datum – 3)
    I can compare it in my ‘where’ condition of my select statement.
    But how the program will know whether its a delta load or a full load.
    Can you help mer in this ?

    Regards,
    Rishi
    rishi_malhotra@infosys.com

    (0) 
  4. Uma maheswara Rao A
    i have a doubt…….. i have to make changes in FM of extractor. What the FM does give some values which are extracted into the item table…
    when i check or test the extractor  for a value of commorcial agreement say 717 it gives me all the values present in the table but i want only the
    distinct values of 717 in testing the extractor.
    (0) 
  5. Linda Brunton
    Hi Siggy,

    I am trying to create an FM in a CRM 5.0 system.  I have copied the sample FM RSAX_BIW_GET_DATA_SIMPLE.  When I try to change the E_T_DATA table parameter to my structure, I receive message FL069 “TABLES parameters are obsolete!”.  Have you encountered this problem ?

    Thanks and regards,
    Linda

    (0) 
    1. Siegfried Szameitat Post author
      Hi Linda,

      yes I already faced the same issue. But it is just a warnig and you can ignore it. I guess in the next future there will be a change here.

      kind regards

      Siggi

      (0) 
      1. Linda Brunton
        Thanks Siggi,
        I could not exit the tables tab once the warning message was displayed.  However, I have since discovered that you don’t need to specify the type spec and the associated type at all. You can just leave them blank and only specify E_T_DATA in the Parameter Name.  The extractor seems to work just fine.
        Cheers,
        Linda
        (0) 
  6. G P
    Hi –
    RSAX_BIW_GET_DATA_SIMPLE is assuming that the data source is a TABLE. I have a similar need wherein my function is pulling data from various sources & loading the net result into an internal table (structure). Do I have to follow the same methodology to feed my cube from this internal table ?
    (0) 
      1. Andrei Grechko
        did you figure it out?

        “Hi –
        RSAX_BIW_GET_DATA_SIMPLE is assuming that the data source is a TABLE. I have a similar need wherein my function is pulling data from various sources & loading the net result into an internal table (structure). Do I have to follow the same methodology to feed my cube from this internal table ?

            *
              Extraction via Function Module
              2007-03-06 06:49:31 Claes Widestadh Business Card [Reply]

              HI GP,
              I have the exact same problem, have you figured out how to manage this?”

        Thanks

        (0) 
  7. Nishu Gupta
    Do i need to copy all the includes dat are present in standard function module ..
    wen i m using rsa3 its giving error
    Error 6 in function module RSS_PROGRAM_GENERATE
    do let me know my code is

    FUNCTION Z_BIW_GET_DATA_SIMPLE.
    *”———————————————————————-
    *”*”Local Interface:
    *” IMPORTING
    *” VALUE(I_REQUNR) TYPE SRSC_S_IF_SIMPLE-REQUNR OPTIONAL
    *” VALUE(I_DSOURCE) TYPE SRSC_S_IF_SIMPLE-DSOURCE OPTIONAL
    *” VALUE(I_MAXSIZE) TYPE SRSC_S_IF_SIMPLE-MAXSIZE OPTIONAL
    *” VALUE(I_INITFLAG) TYPE SRSC_S_IF_SIMPLE-INITFLAG OPTIONAL
    *” VALUE(I_READ_ONLY) TYPE SRSC_S_IF_SIMPLE-READONLY OPTIONAL
    *” TABLES
    *” I_T_SELECT TYPE SRSC_S_IF_SIMPLE-T_SELECT OPTIONAL
    *” I_T_FIELDS TYPE SRSC_S_IF_SIMPLE-T_FIELDS OPTIONAL
    *” E_T_DATA OPTIONAL
    *” EXCEPTIONS
    *” NO_MORE_DATA
    *” ERROR_PASSED_TO_MESS_HANDLER
    *”———————————————————————-

    * Example: DataSource for table SFLIGHT

    TABLES: ZQUOTATION.

    * Auxiliary Selection criteria structure
    DATA : L_S_SELECT TYPE SRSC_S_SELECT.
    DATA : GIT_QTN type table of ZQUOTATION with header line .
    DATA : GIT_TAB type table of BBP_PDS_CND with header line .

    * Maximum number of lines for DB table
    STATICS: S_S_IF TYPE SRSC_S_IF_SIMPLE,

    * counter
    S_COUNTER_DATAPAKID LIKE SY-TABIX,

    * cursor
    S_CURSOR TYPE CURSOR.
    * Select ranges
    * RANGES: L_R_CARRID FOR SFLIGHT-CARRID,
    * L_R_CONNID FOR SFLIGHT-CONNID.
    *
    * Initialization mode (first call by SAPI) or data transfer mode
    * (following calls) ?
    IF I_INITFLAG = SBIWA_C_FLAG_ON.

    ************************************************************************
    * Initialization: check input parameters
    * buffer input parameters
    * prepare data selection
    ************************************************************************

    * Check DataSource validity
    CASE I_DSOURCE.
    WHEN ‘0SAPI_SFLIGHT_SIMPLE’.
    WHEN OTHERS.
    IF 1 = 2. MESSAGE E009(R3). ENDIF.
    * this is a typical log call. Please write every error message like this
    LOG_WRITE ‘E’ “message type
    ‘R3’ “message class
    ‘009’ “message number
    I_DSOURCE “message variable 1
    ‘ ‘. “message variable 2
    RAISE ERROR_PASSED_TO_MESS_HANDLER.
    ENDCASE.

    APPEND LINES OF I_T_SELECT TO S_S_IF-T_SELECT.

    * Fill parameter buffer for data extraction calls
    S_S_IF-REQUNR = I_REQUNR.
    S_S_IF-DSOURCE = I_DSOURCE.
    S_S_IF-MAXSIZE = I_MAXSIZE.

    * Fill field list table for an optimized select statement
    * (in case that there is no 1:1 relation between InfoSource fields
    * and database table fields this may be far from beeing trivial)
    APPEND LINES OF I_T_FIELDS TO S_S_IF-T_FIELDS.

    ELSE. “Initialization mode or data extraction ?

    ************************************************************************
    * Data transfer: First Call OPEN CURSOR + FETCH
    * Following Calls FETCH only
    ************************************************************************

    * First data package -> OPEN CURSOR
    IF S_COUNTER_DATAPAKID = 0.

    * Fill range tables BW will only pass down simple selection criteria
    * of the type SIGN = ‘I’ and OPTION = ‘EQ’ or OPTION = ‘BT’.
    select * from ZQUOTATION
    into table GIT_QTN .

    LOOP AT GIT_QTN .
    CALL FUNCTION ‘BBP_PDCND_GETDETAIL’
    EXPORTING
    i_p_guid = GIT_QTN-GUID1
    * I_P_KIND = ‘B’
    * I_OBJECT_TYPE = ‘BUS2000136’
    * IV_VERSION_TYPE = ‘ ‘
    iv_header_guid = GIT_QTN-HEADER
    * IV_WITH_DELETED_RECORDS = ‘ ‘
    IMPORTING
    ET_CONDITIONS = GIT_TAB
    * E_COM =
    * TABLES
    * E_T_DATA =
    .
    MOVE-CORRESPONDING GIT_TAB TO E_T_DATA.

    ENDLOOP.

    * LOOP AT S_S_IF-T_SELECT INTO L_S_SELECT WHERE FIELDNM = ‘CARRID’.
    * MOVE-CORRESPONDING L_S_SELECT TO L_R_CARRID.
    * APPEND L_R_CARRID.
    * ENDLOOP.
    *
    * LOOP AT S_S_IF-T_SELECT INTO L_S_SELECT WHERE FIELDNM = ‘CONNID’.
    * MOVE-CORRESPONDING L_S_SELECT TO L_R_CONNID.
    * APPEND L_R_CONNID.
    * ENDLOOP.

    * Determine number of database records to be read per FETCH statement
    * from input parameter I_MAXSIZE. If there is a one to one relation
    * between DataSource table lines and database entries, this is trivial.
    * In other cases, it may be impossible and some estimated value has to
    * be determined.
    * OPEN CURSOR WITH HOLD S_CURSOR FOR
    * SELECT (S_S_IF-T_FIELDS) FROM SFLIGHT
    * WHERE CARRID IN L_R_CARRID AND
    * CONNID IN L_R_CONNID.
    ENDIF. “First data package ?

    * Fetch records into interface table.
    * named E_T_’Name of extract structure’.
    * FETCH NEXT CURSOR S_CURSOR
    * APPENDING CORRESPONDING FIELDS
    * OF TABLE E_T_DATA
    * PACKAGE SIZE S_S_IF-MAXSIZE.

    * IF SY-SUBRC <> 0.
    * CLOSE CURSOR S_CURSOR.
    * RAISE NO_MORE_DATA.
    * ENDIF.

    S_COUNTER_DATAPAKID = S_COUNTER_DATAPAKID + 1.

    ENDIF. “Initialization mode or data extraction ?

    ENDFUNCTION.

    (0) 
  8. Bhaskar Rao Javvaji
    Hi,
    i am tyring to craete ZCOPY of “RSAX_BIW_GET_DATA_SIMPLE” . but if i try to check this one, its showing “SRSC_S_IF_SAMPLE” is unkmnown even after mention ‘SRSC’ type-pools.

    Thanks in  advance,
    Bhaskar

    (0) 
  9. Abhi Pandey
    Hi Siggy –
    I need to do a delta extraction based on AEDAT or ERDAT. If AEDAT is initial, then load delta based on ERDAT.  If AEDAT is not initial, then load delta based on AEDAT.  Is this possible via the delta-enabled generic extractor using FM.

    Thanks!!
    abhi
    abhi_999@hotmail.com

    (0) 
  10. Andrew Jabbitt
    Hi siggi,

    Thanks for a great weblog. I have been able to apply it successfully to extract long texts from table STXL.

    But I have a problem whenever the parameter: i_maxsize is greater than the size of the extracted dataset. It seems the ‘no_more_data’ exception closes the connection before e_t_data is received and I get 0 records.

    Any idea what I am doing wrong?

    Rgrds,
    Andrew

    (0) 
  11. Romina Canales
    Hi,
    Im new at this.. I am also trying to craete a COPY of “RSAX_BIW_GET_DATA_SIMPLE” . Its showing “SRSC_S_IF_SAMPLE” is unkmnown. I added:  Type-pools: SRSC. in the main program of the function group.

    Thanks for your help,
    Romina

    (0) 
  12. Shalini R
    Hi,
    I am in a situation whereby a generic extractor has to be created based on the table CDHDR and CDPOS(Cluster table) to track change document details.
    I guess a FM is an only way out.

    Do you have any suggestions on how to maintain a delta extraction on this?

    Thank you.

    (0) 
  13. Kenneth Hartman

    It seems that if I populate E_T_DATA priror to the “Fetch” statement I receive 0 records, but if I do it afterwards I go into an endless loop. Please see the code below and provide a sample or critique on where the code is incorrect.  <br/><br/> tables: pa0000.<br/><br/><br/><br/><br/>  data: l_s_select type srsc_s_select.<br/>*        t_pernr TYPE TABLE OF w_pernr.<br/><br/><br/><br/><br/>* Maximum number of lines for DB table<br/>  statics: s_s_if type srsc_s_if_simple,<br/><br/>* counter<br/>            s_counter_datapakid like sy-tabix,<br/><br/>* cursor<br/>            s_cursor type cursor.<br/>* Select ranges<br/><br/><br/>  ranges: l_r_pernr      for zbw_payroll_str-pernr,<br/>          l_r_fpbeg      for zbw_payroll_str-fpbeg,<br/>          l_r_fpend      for zbw_payroll_str-fpend,<br/>*          L_R_Z_P_FRDAT  FOR ZBW_PAYROLL_STR-Z_P_FRDAT,<br/>*          L_R_Z_P_FRTIM  FOR ZBW_PAYROLL_STR-Z_P_FRTIM,<br/>          l_r_chkdt      for zbw_payroll_str-chkdt.<br/><br/>* Initialization mode (first call by SAPI) or data transfer mode<br/>* (following calls) ?<br/><br/>  if i_initflag = sbiwa_c_flag_on.<br/><br/>***********************************************************************<br/> Initialization: check input parameters<br/>*                 buffer input parameters<br/>*                 prepare data selection<br/>***********************************************************************<br/><br/><br/><br/>    append lines of i_t_select to s_s_if-t_select.<br/><br/>* Fill parameter buffer for data extraction calls<br/>    s_s_if-requnr    = i_requnr.<br/>    s_s_if-dsource   = i_dsource.<br/>    s_s_if-maxsize   = i_maxsize.<br/><br/>* Fill field list table for an optimized select statement<br/>* (in case that there is no 1:1 relation between InfoSource fields<br/>* and database table fields this may be far from beeing trivial)<br/>    append lines of i_t_fields to s_s_if-t_fields.<br/><br/>  else.                 “Initialization mode or data extraction ?<br/><br/>***********************************************************************<br/> Data transfer: First Call      OPEN CURSOR + FETCH<br/>*                Following Calls FETCH only<br/>***********************************************************************<br/><br/> First data package -> OPEN CURSOR<br/>*    BREAK-POINT.<br/>    if s_counter_datapakid = 0.<br/><br/>* Fill range tables BW will only pass down simple selection criteria<br/>* of the type SIGN = ‘I’ and OPTION = ‘EQ’ or OPTION = ‘BT’.<br/>      loop at s_s_if-t_select into l_s_select where fieldnm = ‘PERNR’.<br/>        move-corresponding l_s_select to l_r_pernr.<br/>        append l_r_pernr.<br/>      endloop.<br/><br/>*      LOOP AT S_S_IF-T_SELECT INTO L_S_SELECT WHERE FIELDNM = ‘FPBEG’.<br/>*        MOVE-CORRESPONDING L_S_SELECT TO L_R_FPBEG.<br/>*        APPEND L_R_FPBEG.<br/>*      ENDLOOP.<br/><br/>      LOOP AT S_S_IF-T_SELECT INTO L_S_SELECT WHERE FIELDNM = ‘FPEND’.<br/>*        MOVE-CORRESPONDING L_S_SELECT TO L_R_FPEND.<br/>*        APPEND L_R_FPEND.<br/>*      ENDLOOP.<br/><br/>      LOOP AT S_S_IF-T_SELECT INTO L_S_SELECT WHERE FIELDNM = ‘CHKDT’.<br/>*        MOVE-CORRESPONDING L_S_SELECT TO L_R_CHKDT.<br/>*        APPEND L_R_CHKDT.<br/>*      ENDLOOP.<br/><br/>      loop at s_s_if-t_select into l_s_select<br/>                 where fieldnm = ‘Z_P_FRDAT’.<br/>        move-corresponding l_s_select to s_frdat.<br/>        append s_frdat.<br/>      endloop.<br/><br/>      loop at s_s_if-t_select into l_s_select<br/>                 where fieldnm = ‘Z_P_FRTIM’.<br/>        move-corresponding l_s_select to s_frtim.<br/>        append s_frtim.<br/>      endloop.<br/><br/>* Determine number of database records to be read per FETCH statement<br/>* from input parameter I_MAXSIZE. If there is a one to one relation<br/>* between DataSource table lines and database entries, this is trivial.<br/>* In other cases, it may be impossible and some estimated value has to<br/>* be determined.<br/><br/>      open cursor with hold s_cursor for<br/> select distinct pernr from pa0000<br/>          where pernr  in l_r_pernr<br/>          and   stat2  = ‘3’.<br/><br/>    endif.                             “First data package ?<br/><br/>* Fetch records into interface table.<br/>*   named E_T_’Name of extract structure’.<br/><br/>    fetch next cursor s_cursor<br/>              appending corresponding fields<br/>              of table e_t_data<br/>              package size s_s_if-maxsize.<br/><br/>    if sy-subrc <> 0.<br/>      perform fill_e_t_data tables e_t_data<br/>                          using s_counter_datapakid .<br/>      close cursor s_cursor.<br/>      raise no_more_data.<br/>    else.<br/>* Fill E_T_DATA<br/>*      PERFORM fill_data TABLES l_r_pernr e_t_data<br/>*                            USING s_counter_datapakid .<br/>      perform fill_e_t_data tables e_t_data<br/>                                using s_counter_datapakid .<br/>    endif.<br/><br/>    s_counter_datapakid = s_counter_datapakid + 1.<br/><br/>  endif.              “Initialization mode or data extraction ?<br/><br/>&—-


    <br/>&      Form  fill_e_t_data<br/>&–


    <br/>       text<br/>*


    <br/>      –>E_T_DATA               text<br/>*      >REF_COUNTER_DATAPAKID  text<br/>*


    <br/>form fill_e_t_data  tables e_t_data  structure zbw_payroll_str<br/>                       using ref_counter_datapakid.<br/><br/>  FIELD-SYMBOLS :

    (0) 
  14. Sameer Pathak
    Hi,

    I was wondering if creating multiple extraction FMs in the same function group, affects it’s working if every FM is run in parallel.

    Kindly comment.

    Regards,
    Sameer

    (0) 
  15. Manu Kapur

    Hi Siegfried,

     

    You example is quite helpful. I have a situation in which my FM has to fetch data from different tables and do some manipulation in the data before I send it to BW.

    I am using 4 different select queries to fetch data from different tables and then playing around with it.

     

    I am not using Open, Fetch and close Cursor. I am just populating the E_T_DATA structure finally with the desired data.

    This works fine in RSA3, but when we move it to BW to extract, it seems that the job is not finishing. I understand we need to close the cursor at the end, but in such a condition how do you open cursor , I mean on which select query should one open the cursor. 

     

    I tried opening the cursor on the first query and when I fetched data, it gave me cumulative records for each call which is wrong.

     

    Please advise.

     

    Regards,

    Manu

    (0) 
    1. Nageswara Reddy

      Hi Manu,

      Instead of opening and closing the cursor, did you try raising exception NO_MORE_DATA?

      Initially you were not using Open & Close. That means, you are sending the whole data in a single package. To make the system know that this is the last package, you should raise this exception.

      *if the last record is populated to table e_t_data you need to raise the exeption no_more_data

       

      regards,

      Nageswara

      (0) 
      1. Manu Kapur

        Hi Nageswara,

        I added the exception and the data in RSA3 seems to be fine. But in BW side, the IDoc which confirms the data is still in the warning state.

         

        Regards,

        Manu

        (0) 

Leave a Reply