Skip to Content

After I read this blog on Shared Memory , I got curious and gave a try, if this could be used in BW transformation routines.

Shared Memory in BW Transformation

Very often in a BW data load process, we may need to hit the DB tables for lookup in the start/end/expert routine. So here the same set of DB tables is hit as many times as the number of data packets in a data load. Apart from hitting the DB table several times, the same data is stored in internal tables across several instances of the transformation class (Data packet).This results in reduced performance by hitting the persistent layer and consumes more application layer memory by storing the same data in multiple instances.

We ever wonder, if this could be avoided by reading the persistent table, while first data packet loads and storing the data in internal tables defined in the global part of the transformation (CLASS-DATA),so that the data is visible other data packets as well. But this would work only if our DTP processing mode is ‘Serially in background processes’.

So, here comes the concept ‘Shared Memory’, by which the data can be stored in SAP memory once and could be read in the transformation routines for lookup.

What is Shared Memory?

            It is the concept of storing the data in SAP memory, for rapid data retrieval without the need to physically read the persistent database tables. It also allows us to apply certain business logics to the data, before it is stored. As the name implies, the data can be used across users or sessions. This makes this technique more flexible than the session memory concepts.

Illustration

            Let’s take a simple example: A transformation, where in the end routine, vendor master table is read repeatedly (for every data packet) from the database. In order to reduce the DB access, we decide to make use of shared memory objects, to store the data in SAP memory, instead of repeatly executing the ‘select statement’.

End routine code:

            For our illustration, the below end routines code will be fine tuned to access the shared memory.

*************************************************************************************************************

    TYPES: BEGIN OF ty_vendor,

            vendor     TYPE /bi0/oivendor,

            accnt_grpv TYPE /bi0/oiaccnt_grpv,

            addr_numbr TYPE /bi0/oiaddr_numbr,

            altitude   TYPE /bi0/oialtitude,

            bpartner   TYPE /bi0/oibpartner,

            city       TYPE /bi0/oicity,

            city_2     TYPE /bi0/oicity_2,

            country    TYPE /bi0/oicountry,

            dbduns_num TYPE /bi0/oidbduns_num,

            fax_num    TYPE /bi0/oifax_num,

            id_txnumb3 TYPE /bi0/oiid_txnumb3,

            id_xcpd    TYPE /bi0/oiid_xcpd,

            industry   TYPE /bi0/oiindustry,

            langu      TYPE langu,

            latitude   TYPE /bi0/oilatitude,

            logsys     TYPE rsdlogsys,

            longitude  TYPE /bi0/oilongitude,

          END OF ty_vendor.

    DATA:wa_vendor TYPE ty_vendor,

         it_vendor LIKE TABLE OF wa_vendor.

    FIELD-SYMBOLS:<fs_vendor> TYPE ty_vendor.

   

    SELECT

    vendor

    accnt_grpv

    addr_numbr

    altitude

    bpartner

    city

    city_2

    country

    dbduns_num

    fax_num

    id_txnumb3

    id_xcpd

    industry

    langu

    latitude

    logsys

    longitude

    FROM /bi0/mvendor

    INTO TABLE it_vendor

    WHERE objvers = ‘A’ AND

          country = ‘CA’.

    IF sy-subrc EQ ‘0’.

      SORT it_vendor BY vendor.

    ENDIF.

   

    LOOP AT RESULT_PACKAGE ASSIGNING <result_fields>.

      READ TABLE it_vendor

      ASSIGNING <fs_vendor>

      WITH KEY

      vendor = <result_fields>-vendor

                                  BINARY SEARCH.

      IF sy-subrc EQ ‘0’.

        <result_fields>-vendor       = <fs_vendor>-vendor.

        <result_fields>-accnt_grpv   = <fs_vendor>-accnt_grpv.

        <result_fields>-addr_numbr   = <fs_vendor>-addr_numbr.

        <result_fields>-altitude     = <fs_vendor>-altitude.

        <result_fields>-bpartner     = <fs_vendor>-bpartner.

        <result_fields>-city         = <fs_vendor>-city.

        <result_fields>-city_2       = <fs_vendor>-city_2.

        <result_fields>-country      = <fs_vendor>-country.

        <result_fields>-dbduns_num   = <fs_vendor>-dbduns_num.

        <result_fields>-fax_num      = <fs_vendor>-fax_num.

        <result_fields>-id_txnumb3   = <fs_vendor>-id_txnumb3.

        <result_fields>-id_xcpd      = <fs_vendor>-id_xcpd.

        <result_fields>-industry     = <fs_vendor>-industry.

        <result_fields>-langu        = <fs_vendor>-langu.

        <result_fields>-latitude     = <fs_vendor>-latitude.

        <result_fields>-logsys       = <fs_vendor>-logsys.

        <result_fields>-longitude    = <fs_vendor>-longitude.

      ENDIF.

      UNASSIGN <fs_vendor>.

    ENDLOOP.

    UNASSIGN <result_fields>.

*************************************************************************************************************

Steps for Defining Shared Memory Objects

        

     1.Create an ordinary class using transaction SE24.Define attributes (the data going to be held in the shared memory) and method for accessing and work with these attributes, in the end routine. This class is called the ‘root class’, since this will be used in step, while defining the memory area. More importantly, the class should be defined as ‘Shared Memory-Enabled’.

   /wp-content/uploads/2013/12/img1_337350.png

1.            2.Allocate a new memory are for our shared object. This could be done from transaction SHMA.

The next section will give us more information on how to do the above steps.

Step 1: Root Class

            As mentioned earlier, the concept of shared memory is based on memory areas, each related to a global area root class. The root class contains attributes and methods needed to be used in the end routine. The root class will need attribute to hold data you want store (in our example internal table storing the vendor data), as well as methods to manipulate the data. In our case, we will have methods to initialize the table with specific data from the persistent layer and to retrieve data from it.

Defining Attributes of Root Class

            First, we need to define data to be stored in our memory object. We need vendor data from vendor master data. So we simply create a table attribute as below.

   

     /wp-content/uploads/2013/12/img1_337350.png

I already defined a suitable table type ‘IT_VENDOR’ as below,

/wp-content/uploads/2013/12/img1_337350.png

     /wp-content/uploads/2013/12/img1_337350.png

Defining Methods of Root Class

            Now we need atleast two methods in our root class, one for initializing the attribute ‘X_IT_VENDOR’ with the data from the underlying DB table(Vendor Master data) and another method for retrieving this data from the memory object  in the end routine.

One more method (IF_SHM_BUILD_INSTANCE~BUILD), which is highlighted below is for ‘automatic pre-load’ of memory object, which we can discuss in further sections. But to make this method available in the class, an interface “IF_SHM_BUILD_INSTANCE” needs to be included in the class.

/wp-content/uploads/2013/12/img1_337350.png

     /wp-content/uploads/2013/12/img1_337350.png

Methods SET_VENDOR

            The SET_VENDOR method is used to retrieve vendor data from the DB layer(Vendor master data table).When our shared memory object is created for the first time, it will be empty, meaning we need to invoke this method to populate X_IT_VENDOR attribute.

In order to avoid reading the complete vendor master data into the shared memory, we will limit ourselves to specific country using the method attribute I_COUNTRY.

     /wp-content/uploads/2013/12/img1_337350.png

METHOD set_vendor.

  SELECT

      vendor

      accnt_grpv

      addr_numbr

      altitude

      bpartner

      city

      city_2

      country

      dbduns_num

      fax_num

      id_txnumb3

      id_xcpd

      industry

      langu

      latitude

      logsys

      longitude

      FROM /bi0/mvendor

      INTO TABLE x_it_vendor

      WHERE objvers = ‘A’ AND

            country = i_country.

ENDMETHOD.

Methods GET_VENDOR

            This method is used to retrieve data from X_IT_VENDOR attribute table of the object. This is the method we call from our end routine, instead of selecting records from the DB layer.

METHOD get_vendor.

  ex_it_vendor = me->x_it_vendor.

ENDMETHOD.

ex_it_vendor is the exporting parameter of this method

     /wp-content/uploads/2013/12/img1_337350.png

Methods IF_SHM_BUILD_INSTANCE~BUILD

                This method is for automatic pre-loading of our memory area. This method ensures that the memory area which is not yet initialized automatically started when an application tries to access it. This means that if we try to read vendor data from the shared memory, the memory area will be initialized on the fly, even if it is not initialized before. So there is no need of handling the exception ‘CX_SHM_NO_ACTIVE_VERSION’ in our end routine.

METHOD if_shm_build_instance~build.

    DATA:lcl_vendor_root TYPE REF TO zcl_vend_shared_obj,

         lcl_vendor TYPE REF TO zcl_tst_vendor.

    lcl_vendor = zcl_tst_vendor=>attach_for_write( ).

    CREATE OBJECT lcl_vendor_root AREA HANDLE lcl_vendor.

    lcl_vendor->set_root( lcl_vendor_root).

    lcl_vendor_root->set_vendor(

    EXPORTING

    i_country = ‘CA’

     ).

    lcl_vendor->detach_commit( ).

  ENDMETHOD.

The above code initializes the memory object with some vendor data. This is done by invoking the attach_for_write (lock the memory area for writing) of the memory area, followed by the instantiation of the root class. When this is done, the SET_VENDOR method of the memory object is called in order to fill the attribute table X_IT_VENDOR.

Step 2: Creating the Memory Area

            We need to create a shared memory area for our memory object; we can do this from transaction SHMA

/wp-content/uploads/2013/12/img1_337350.png

     /wp-content/uploads/2013/12/img1_337350.png

Here as highlighted above, we assigned our new class ‘ZCL_VEND_SHARED_OBJ’ as the root class and constructor class to the memory area. Also we have checked here ‘Automatic Area Creation’ to make sure that the memory object is automatically created whenever the invoked from end routine-if it does not exist already. We can keep rest of the properties as defaulted here (Please refer to SAP documentation on Shared memory for more details).

                Once the memory area is activated, a corresponding class is generated automatically (It has the same name as memory area).This class provides methods for establishing a handle for memory area.

     /wp-content/uploads/2013/12/img1_337350.png

Step 3: Using shared memory objects in end routine of transformation

            Using memory object in any program normally follows the following steps,

·         Create a reference to the shared memory area class

·         Call one of the ATTACH methods of the area class

·         Call any of the data manipulation methods of the area’s root class

·         Call a DETACH method to “free” the area handle

The ATTACH methods are used for requesting locks on the memory area. This is a prerequisite for accessing the related memory area. We normally use ‘attach_for_read’,’attach_for_update’ methods. ‘attach_for_write’ method is usually called only during the creation of memory object (as seen in the code for ‘IF_SHM_BUILD_INSTANCE~BUILD’ in the previous section).

The end routine code is now modified by replacing the select statement with the highlighted in red below.

    TYPES: BEGIN OF ty_vendor,

            vendor     TYPE /bi0/oivendor,

            accnt_grpv TYPE /bi0/oiaccnt_grpv,

            addr_numbr TYPE /bi0/oiaddr_numbr,

            altitude   TYPE /bi0/oialtitude,

            bpartner   TYPE /bi0/oibpartner,

            city       TYPE /bi0/oicity,

            city_2     TYPE /bi0/oicity_2,

            country    TYPE /bi0/oicountry,

            dbduns_num TYPE /bi0/oidbduns_num,

            fax_num    TYPE /bi0/oifax_num,

            id_txnumb3 TYPE /bi0/oiid_txnumb3,

            id_xcpd    TYPE /bi0/oiid_xcpd,

            industry   TYPE /bi0/oiindustry,

            langu      TYPE langu,

            latitude   TYPE /bi0/oilatitude,

            logsys     TYPE rsdlogsys,

            longitude  TYPE /bi0/oilongitude,

          END OF ty_vendor.

    DATA:wa_vendor TYPE ty_vendor,

         it_vendor LIKE TABLE OF wa_vendor.

    FIELD-SYMBOLS:<fs_vendor> TYPE ty_vendor.

    DATA:lcl_vendor_shmo TYPE REF TO zcl_tst_vendor,

                lv_exception TYPE REF TO cx_root,

               lv_text TYPE string.

   

    TRY.

        lcl_vendor_shmo = zcl_tst_vendor=>attach_for_read ( ).

      CATCH cx_shm_no_active_version

            cx_shm_read_lock_active

            cx_shm_change_lock_active

            cx_shm_exclusive_lock_active

            cx_shm_inconsistent

              INTO lv_exception.

    ENDTRY.

    IF NOT lv_exception IS INITIAL.

      CALL METHOD lv_exception->if_message~get_text

        RECEIVING

          result = lv_text.

      WRITE lv_text.

    ELSE.

      IF lcl_vendor_shmo IS NOT INITIAL.

        CALL METHOD lcl_vendor_shmo->root->get_vendor(

          IMPORTING

            ex_it_vendor = it_vendor

                           ).

      ENDIF.

      SORT it_vendor BY vendor.

    ENDIF.

      

LOOP AT RESULT_PACKAGE ASSIGNING <result_fields>.

      READ TABLE it_vendor

      ASSIGNING <fs_vendor>

      WITH KEY

      vendor = <result_fields>-vendor

                                  BINARY SEARCH.

      IF sy-subrc EQ ‘0’.

        <result_fields>-vendor       = <fs_vendor>-vendor.

        <result_fields>-accnt_grpv   = <fs_vendor>-accnt_grpv.

        <result_fields>-addr_numbr   = <fs_vendor>-addr_numbr.

        <result_fields>-altitude     = <fs_vendor>-altitude.

        <result_fields>-bpartner     = <fs_vendor>-bpartner.

        <result_fields>-city         = <fs_vendor>-city.

        <result_fields>-city_2       = <fs_vendor>-city_2.

        <result_fields>-country      = <fs_vendor>-country.

        <result_fields>-dbduns_num   = <fs_vendor>-dbduns_num.

        <result_fields>-fax_num      = <fs_vendor>-fax_num.

        <result_fields>-id_txnumb3   = <fs_vendor>-id_txnumb3.

        <result_fields>-id_xcpd      = <fs_vendor>-id_xcpd.

        <result_fields>-industry     = <fs_vendor>-industry.

        <result_fields>-langu        = <fs_vendor>-langu.

        <result_fields>-latitude     = <fs_vendor>-latitude.

        <result_fields>-logsys       = <fs_vendor>-logsys.

        <result_fields>-longitude    = <fs_vendor>-longitude.

      ENDIF.

      UNASSIGN <fs_vendor>.

    ENDLOOP.

    UNASSIGN <result_fields>.

     lcl_vendor_shmo->detach ( ).

The ‘detach’ method at the end, release the lock at the end of the routine on the memory object, imposed by the ‘attach_for_read’ method.

Step 4: Examining the Shared Memory Object

            The shared memory objected created can be checked from transaction SHHM.

     /wp-content/uploads/2013/12/img1_337350.png

  

The shared memory object is now in the SAP memory and it will be there until the SAP instance goes down. It could be deleted by selecting it and using the delete button.

In our case we can delete the shared memory object at the end of the DTP load using the method ‘FREE_INSTANCE’ of the class ‘ZCL_TST_VENDOR’.It will be created again, when the DTP run for the next time.

     /wp-content/uploads/2013/12/img1_337350.png

     /wp-content/uploads/2013/12/img1_337350.png

Vendor table data stored in shared memory:

     /wp-content/uploads/2013/12/img1_337350.png

Conclusion:

                I just did a test runs of the DTP with the end routine code included with ‘SELECT STATEMENT’ and ‘SHARED MEMORY’ separately. For 202 records, the first one took 19s and where as the later one took 12s.

SELECT from DB table:

     /wp-content/uploads/2013/12/img1_337350.png

READ from Shared Memory:

/wp-content/uploads/2013/12/img1_337350.png

The shared memory concept in BI transformation, apart from reducing the DB load and ‘application memory’ required to run a DTP, also lead to performance improvement of the DTP load. This could be more helpful in transformations, where a number DB tables are used for look up and in DTP loads involving more number of data packets.

I will keep explore this option of using shared memory and keep you guys posted.

Related Content:

Shared Object – SAP Help

Shared Object – Implementation

To report this post you need to login first.

24 Comments

You must be Logged on to comment or reply to a post.

  1. seshu P

    Hi,

    Good to know this feature in Sap BI/BW πŸ™‚ πŸ™‚ , Well Explained ℹ with Screen Shots 😎 .

    Regards,

    Seshu.P

    (0) 
  2. Mayank Jaiswal

    Hi Karthick,

    First of all thanks for sharing this info.Really good job.

    Only one issue which I can think of is when you have say millions of records in this case vendor master having millions of records then you are reading complete set of records from the master data table into Internal Table. Now you will be reading this huge table everytime.

    But on the other hand when we do Read from master table in the start or end routine we usually use For All Entries. So everytime you have smaller set of records in Internal table to read.

    I don’t know which one is better, but we should compare the performance when the number of records are really huge.

    What is your opinion on this please share.

    Regards,

    Mayank.

    (0) 
    1. Martin Maruskin

      I agree with Alexander, I think it is possible to achieve same thing declaring internal table in global part of routine. If I’m mistaken then you need to support your way of doing this with more results of performance tests. Also more data should be involved in the testing.

      Moreover seems this blog was inspired by SCN how-to document:

      How to Work with ABAP Shared Memory Objects

      Sequence of screenshots in blog just follows the how-to doc. In this case the how-to doc should be introduced as reference IMHO.

      To be clear I’m not saying it is wrong. I very much appreciate blog author’s effort I’m just saying this but for sake of objectivity.

      cheers

      m./

      (0) 
      1. Mayank Jaiswal

        Hi Alexander/Martin,

        I think you guys are missing one important point here. What Karthik is trying to explain is by using this method you would hit the database table only once through out the loading of request. Now, even if you declare Internal Table in global part of routine your database hit will be for each data pacakge ( usually 50K is the package size ) so if you have 2L records you will have four packages and hence there will be four DB hits.

        But by Shared Memory Conecpt, there will be only one hit and rest all the data packages can use the Shared Memory Object.

        I hope this explains your queries.

        Regards,

        Mayank

        (0) 
        1. Alexander Kutz

          Well even if you must use memory – wouldn’t it be easier to use Data Clusters / Export to Memory ID instead of developing the whole class setup?

          (0) 
          1. Benedict Venmani Felix

            hi Alex,

            It’s just an idea that it can be used in BW. Maybe the example he chose did not perfectly justify it for you but IMHO its something new that I have heard of and I think it is up to each one on how they implement based on their BW requirement.

            Thanks,

            Benedict

            (0) 
  3. Ramakrishnan Ramanathaiah

    Thanks Karthick. Nice document.  I am aware it will work in both BW on HANA and BW on traditional database.But if it’s HANA , every thing is in the in Memory . Need to check how it works on BW on HANA system.

    (0) 

Leave a Reply