Skip to Content

Abstract

SAP NetWeaver BW 7.30 introduces the concept of semantic partitioning. The following article explains how you can use semantically partitioned DataStores or InfoCubes to efficiently manage large data volumes and reduce the TCD/TCO.   

Motivation

Enterprise Data Warehouses are the central source for BI applications and are faced with the challenge of efficiently managing constantly growing data volumes. A few years ago, Data Warehouse installations requiring terabytes of space were a rarity. Today the first installations with petabyte requirements are starting to appear on the horizon.

In order to handle such large data quantities, we need to find modeling methods that guarantee the efficient delivery of data for reporting. Here it is important to consider various aspects such as the loading and extraction processes, the index structure and data activation in a DataStore object. The Total Cost of Development (TCD) and the Total Cost of Ownership (TCO) are also very important factors.

Here is an example of a typical modeling scenario. Documents need to be saved in a DataStore object. These documents can come from anywhere in the world and are extracted on a country-specific basis. Here each request contains exactly one country/region.

Figure 1

If an error occurs (due to invalid master data) while the system is trying to activate one of the requests, the other requests cannot be activated either and are therefore initially not available for reporting. This issue becomes even more critical if the requests concern country-specific, independent data.

Semantic partitioning provides a workaround here. Instead of consolidating all the regions into one DataStore object, the system uses several structurally identical DataStore objects or “partitions”. The data is distributed between the partitions, based on a semantic criterion (in this example, “region”).

Figure 2

Any errors that occur while requests are being activated now only affect the regions that caused the errors. All the other regions are still available for reporting. In addition, the reduced data volume in the individual partitions results in improved loading and administration performance.

However, the use of semantic partitioning also has some clear disadvantages. The effort required to generate the metadata objects (InfoProviders, transformations, data transfer processes) increases with every partition created. In addition, any changes to the data model must be carried out for every partition and for all dependent objects. This makes the change management more complex. Your CIO might have something to say about this, especially with regards to TCO and TCD!

Examples of semantically partitioned objects

Here you can set the semantically partitioned DataStores or InfoCubes (abbreviated to “SPO”: semantically partitioned object) introduced in SAP NetWeaver BW 7.30. It is now possible to use SPOs to generate and manage semantically partitioned data models with minimal effort.

SPOs provide you with a central UI that enables you to perform the one-time maintenance of the structure and partitioning properties. During the activation stage, the required information is retrieved for generating the partitions. Changes such as adding a new InfoObject to the structure are performed in the same on the SPO and are automatically applied to the partitions. You can also generate DTPs and process chains that match the partitioning properties.

The following example demonstrates how to create a semantically partitioned DataStore object. The section following the example provides you with an extensive insight into the new functions.

DataStore objects and InfoCubes can be semantically partitioned. In the Data Warehousing Workbench, choose “Create DataStore Object”, for example, and complete the fields in the dialog box. Make sure that the option “Semantically Partitioned” is set.

 

Figure 3 

 Figure 4

 

A wizard (1) guides you through the steps for creating an SPO. First, define the structure that are used to using for standard DataStore objects (2). Choose “Maintain Partitions”.

 

Figure 5

 

In the next dialog box, you are asked to specify the characteristics that you want to use as partitioning criteria. You can select up to 5 characteristics. For this example, select “0REGION”. The compounded InfoObject “0COUNTRY” is automatically included in the selection.

 

Figure 6

 

You can now maintain the partitions. Choose the button (1) to add new partitions and change their descriptions (2). Use the checkbox (3) to decide whether you want to use single values or value ranges to describe the partitions. Choose “Start Activation”. You have now created your first semantically partitioned DataStore object.

 

Figure 7

Figure 8

 

In the next step, you connect the partitions to a source. Go to step 4: “Create Transformation” and configure the central transformation using the relevant business logic.

 

Figure 9

 

Now go to step 5: “Create Data Transfer Processes” to generate DTPs for the partitions. On the next screen, you see a list of the partitions and all available sources (1). First, choose “Create New DTP Template” (2) to create a parameter configuration.

 

Figure 10

 

A parameter configuration/DTP template corresponds to the settings that can be configured in a DTP. These settings are applied when DTPs are generated.

 

Figure 11

 

Once you have created the DTP template, drag it from the Template area and drop it on a free area under the list of partitions (1). This assigns a DTP to every source-target combination. If you need different templates for different partitions, you can drag and drop a template onto one specific source-target combination.

Once you have finished, select all the DTPs (2) and choose “Generate”.

 

Figure 12

 

The last step is to generate a process chain in order to execute the DTPs. Go to step 6 in the wizard: “Create Process Chains”. In the next screen, select all the DTPs and drag and drop them to the lower right screen area: “Detail View (1)”.   You use the values “path” and “sequence” to control the parallel processing of DTPs. DTPs with the same path are executed consecutively.

 

Figure 13

 

Choose “Generate” (3). The following process chain is created.

 

Figure 14

  

Summary

In this article, you learned how to create a semantically partitioned object. Using the central UI of an SPO it’s now possible to create and maintain complex partitioned data models with minimal effort. In addition, SPOs guarantee the consistency of your metadata (homogenous partitions) and data (filtered according to the partition criterion).  

Once you have completed the 6 steps, you will have created the following components:

 

  • An SPO with three partitions (DataStore objects)
  • A central transformation for business logic implementation
  • 3 data transfer processes
  • 1 process chain
  
Further Articles

Are you interested in more SAP NetWeaver BW 7.30 news? Check out the SAP BW Developers SDN Blog Series Accompanying the BW 7.3 Ramp-Up Phase

Disclaimer

http://www.sdn.sap.com/irj/sdn/index?rid=/webcontent/uuid/b0b72642-0fd9-2d10-38a9-c57db30b522e

To report this post you need to login first.

8 Comments

You must be Logged on to comment or reply to a post.

  1. Sridhar Gande
    This SPO feature is really good, it will minimise the lot of effort and save time that we put in today…It is more than what I was expecting

    Regards,
    Sridhar

    (0) 
  2. Bala Prabahar
    The table level paritioning provided by DB vendors would offer all but one(errors due to invalid master data) advanatges of SPO. How many times one would run into data load errors due to invalid master data? If this occurs more frequently, wouldn’t it indicate some kind of design issue? I guess my point is not that SPO wouldn’t be useful;my point is that this wouldn’t be useful in all situations. I know several customers use MP(this is default choice) on time-period for one reason: if data load errors occur for the current period, data will still be available for other periods. This,IMO, is not good enough reason for implementing MP instead of table level partitioning. I am afraid SPO is probably going to be used more widely and might lead to increased complexity. I would like to know your thoughts.

    Thanks,
    Bala

    (0) 
    1. Alexander Hermann Post author
      Thanks for your reply, Bala!
      We’re recommending semantically partitioned data models not just because of the performance gains. Especially when thinking about time zone handling (report availability) or decoupled data loads semantical partitioning is a very helpful approach.
      But semantical partitioning and physical database partitioning aren’t mutually exclusive. When maintaining the reference structure of an SPO you can choose to use physical partitioning as well. For every semantical partition the physical criteria will be derived from the reference structure in that case. This enables you to take advantage of both approaches.

      Regards,
      Alexander

      (0) 
  3. Rajeshkumar Salecha
    Hi,

    As per mentioned in your blog like…

    Once you have completed the 6 steps, you will have created the following components:

    1) An SPO with three partitions (DataStore objects)
    2) A central transformation for business logic implementation
    3) 3 data transfer processes
    4) 1 process chain

    so hereby as what i understood we are just creating a semantic partitioned object. If this is correct i have queries as below.

    1. How is that we have just one central transformation and 3 DTP’s from same source using the transformations ?

    2. If it is just semantic partitioned, why do we have 3 DSO activation steps on just one DSO ? How will internally this 3 activation will work for the data to active table.?

    3. In case, if i need to use all data togather, do i use just this one dso or something else? Is there any other settings that allows me to merge all data togather for a combined view.?

    4. How flexible this kind of DSO would be when we use it in Multiprovider in a sense if we require only 2 regions in reporting ?

    and

    5. Can you please share what will be the Backend table names of each semantic partitioned DSO ?

    Hope i am not bugging you and this are questions just out of curiosity and further to update my knowledge.

    Thanks in advance and a good blog.

    Regards
    Rajesh

    (0) 
    1. Alexander Hermann Post author
      Hi Rajesh,

      thanks for your feedback!
      Actually a semantically partitioned object (SPO) is a new InfoProvider which is composed of existing BW objects. The important fact here is that you don’t have to model multiple e.g. DataStore objects but only one SPO. When activating the SPO it will automatically generate all dependent objects. The data model then looks like shown here:

      https://weblogs.sdn.sap.com/weblogs/images/32607/model.png

      The figure also explains your first question. Since the generated partitions are encapsulated by an incoming and outgoing InfoSource you have a central point for your transformation containing the business logic.

      Regarding your second question: since we’re generating three DataStore objects as partitions (like shown in the figure linked above) we’ll have three independent “request activations” – one for each DataStore.

      Reporting is possible on SPO level (you can create a query for an SPO) or on partition level – depending on your scenario. Queries are optimized for SPOs in terms of partition pruning (only partitions with requested data will be touched). In addition SPOs can be used in MultiProviders.

      The metadata for SPOs is stored in table RSLPO.

      Kind regards,
      Alexander

      (0) 
  4. Ed K
    HI,

    1) is spo possible for business content objects?
    2) how to install business content spo ?
    3)is there any pre-settings for ctrating spo?

    thanks,
    EDK…..

    (0) 
  5. Ashok Babu Kumili

    Dear Alexander Hermann.

    Great feature….. explained beautifully….

    Thanks for the screens they are extra ordinary…

    Great read.. Indeed, Very helpful… Awesome article to understand “Semantic partition in DSO”…. This is a brilliant way to go.

    Best regards -Ashok

    (0) 

Leave a Reply