Skip to Content

Bi Accelerator Index build

The BI accelerator (BIA) is built to be used as an appliance – a black box which is not to be questioned as to how it works.

This puts some people at unease when considering the following facts:

1. Are we sure that the job is being done ?

2. What happens if something goes wrong ?

and many such more….

To start with we have a lot of ways to understand how the BIA functions. Lets start with the BIA Index Initial fill job…

 

I have used the job log as it is .. excuse any formatting gaffes.. I have also maintained the timestamps to maintain sequence…

*The first step </u></p><p>is to switch off reporting for the cubes – this is necessary since statistics gets generated for the cubes and also the BIA will have unrestricted access to the cube and also the indexes till it is filled </p><p> 16.08.2008 01:00:11 Job started<br />16.08.2008 01:00:11 Step 001 started<br />(program RSDDTREX_AGGREGATES_FILL, variant &0000000000186, user ID )<br />16.08.2008 01:00:11  OFF<br />16.08.2008 01:00:14  OFF<br />16.08.2008 01:00:14  OFF<br />16.08.2008 01:00:14 INFOCUBE: <br />16.08.2008 01:00:14 Statistics UID of indexing job: ‘<br />4ATXNHLMXZGVUCD2WKYLY7CP9’ (RSDDSTATTREX/RSDDSTATTREXSERV)</p><p><u>Build of Master Data Indices</u> </p><p>Now the BIA proceeds to go and build the indices for the master data objects <br />namely the SID tables in the cubes. Here you can notice that the number of jobs <br />spawned for index builds for the SID tables is started. This setting of three parallel processes is set in the BIA configuration and can be changes if required. </p><p> 16.08.2008 01:00:18 Loading data to index BP1_BIC:SXXXXX (records ‘0000500000’, job ‘1’ )<br />16.08.2008 01:00:22 Loading data to index BP1_BIC:SXXXXX (records ‘0001000000’, job ‘2’ )<br />16.08.2008 01:00:22 Loading data to index BP1_BIC:SXXXXX (records ‘0001181769’, job ‘0’ )<br />16.08.2008 01:01:02 Total number of indexed reocrds in index ‘BP1_BIC:S<XXXXX>’ after COMMIT: ‘1,181,769’<br />16.08.2008 01:01:03 YUNBL_C04 OFF<br />16.08.2008 01:01:03 YUNBL_C04 OFF<br />After each index build the number of records indexed are indicated.</p><p><u>Key figure check</u> </p><p>16.08.2008 01:01:03 Caution: Key figure ‘/BIC/<KEY FIGURE>’ is of type “FLOAT”<br />16.08.2008 01:01:03 Caution: Key figure ‘/BIC/<KEY FIGURE>’ is of type “FLOAT”<br />16.08.2008 01:01:03 Caution: Key figure ‘/BIC/<KEY FIGURE>’ is of type “FLOAT”<br />16.08.2008 01:01:03 Caution: Key figure ‘/BIC/<KEY FIGURE>’ is of type “FLOAT”<br />16.08.2008 01:01:03 Caution: Key figure ‘/BIC/<KEY FIGURE>’ is of type “FLOAT”<br />16.08.2008 01:01:03 Caution: Key figure ‘/BIC/<KEY FIGURE>’ is of type “FLOAT”<br />16.08.2008 01:01:03 Caution: Key figure ‘/BIC/<KEY FIGURE>’ is of type “FLOAT”<br />16.08.2008 01:01:03 Caution: Key figure ‘/BIC/<KEY FIGURE>’ is of type “FLOAT”<br />16.08.2008 01:01:03 Caution: Key figure ‘/BIC/<KEY FIGURE>’ is of type “FLOAT”<br />Here any FLOAT Key figures in the cube are highlighted. You can ignore this because many times <br />key figures that are not related to your cube might appear.<br />16.08.2008 01:01:04 <CUBE> OFF   </p><p><u>Concurrent jobs that might be running</u> </p><p>16.08.2008 01:01:04 Lock for table ‘/BIC/SXXXX’. Job will be restarted again later<br />This message appears if any attribute change runs is in progresses and the BIA fill<br />will wait for the Attribute change run to complete. </p><p>16.08.2008 01:01:15 Loading data to index BP1_BIC:S<XXXXX><br />(records ‘0000833333’, job ‘1’ )<br />16.08.2008 01:01:20 Loading data to index BP1_BIC:S<XXXXX> <br />(records ‘0001666666’, job ‘2’ )<br />16.08.2008 01:01:26 Loading data to index BP1_BIC:S<XXXXX><br /> (records ‘0002499999’, job ‘3’ ) <br />16.08.2008 01:01:31 Loading data to index BP1_BIC:S<XXXXX> <br />(records ‘0003333332’, job ‘4’ ) <br />16.08.2008 01:01:37 Loading data to index BP1_BIC:S<XXXXX> <br />(records ‘0004166665’, job ‘5’ )</p><p>Here you can see five parallel jobs for the index build for an SID table.. </p><p><u>Parallelism</u> </p><p> 16.08.2008 01:08:20 Total number of indexed reocrds in index ‘BP1_BIC:SXXXX’ after COMMIT:<br /> ‘68,198,941’<br />16.08.2008 01:08:21 Parameter BATCHPARA during indexing: ‘3’<br />The number of batch processes used is also given in the job log.</p><p>16.08.2008 01:08:21 ’10’ different values in column ‘SID_0CALMONTH’ of table ‘/BIC/DYUNBL_C04T'<br />16.08.2008 01:08:21 WHERE clause: OPT =’BT’, LOW = ”, HIGH = ‘0000200806’ (field: ‘SID_0CALMONTH’)<br />16.08.2008 01:08:21 Program RSBATCH_EXECUTE_PROZESS successfully scheduled as job BITRP_4ATXO9JVXWGQMD4W4ZHAJCOI5 with ID 01082100<br />16.08.2008 01:08:21 WHERE clause: OPT =’BT’, LOW = ”, HIGH = ‘0000200809’ (field: ‘SID_0CALMONTH’)<br />16.08.2008 01:08:22 Program RSBATCH_EXECUTE_PROZESS successfully scheduled as job BITRP_4ATXO9JVXWGQMD4W4ZHAJCOI5 with ID 01082101<br />Here the Cube is partitioned on calmonth and the index accordingly looks for data in the same way<br />by partitions.</p><p><u>Fact Table Indexing</u> </p><p>Now the heavy duty loading of the fact table starts….<br /><br />16.08.2008 01:08:25 Loading data to index BP1_BIC:F<XXXXX> (records ‘0000051020’, job ‘1’ )<br />16.08.2008 01:24:13 Loading data to index BP1_BIC:F<XXXXX> (records ‘0022856960’, job ‘5’ )<br />16.08.2008 01:24:15 Loading data to index BP1_BIC:F<XXXXX> (records ‘0022907980’, job ‘5’ )</p><p>16.08.2008 01:25:04 Background process ‘2’ complete: number of indexed records ‘32,643’<br />16.08.2008 01:28:04 Background process ‘1’ complete: number of indexed records ‘28,209,394’<br />16.08.2008 01:28:04 Parallel background process for indexing table ‘/BIC/F<XXXXX>’ complete (’37’)  </p><p><u>Request ID Indexing*

To report this post you need to login first.

6 Comments

You must be Logged on to comment or reply to a post.

  1. Marc Bernard
    Hello Arun,

    thanks for the blog. It’s a nice explanation of the indexing job. You can find additional information on parallelizing the indexing job in SAP note 1161395.

    Indexing the Request ID table is of course necessary. However, this process must never lead to any inconsistencies or “unstable” system behavior that would require re-indexing. Please make sure you have implemented all corrections mentioned in SAP note 1162365.

    The sparse flag is used to optimize the BIA internal processes if a column in the BI data model contains only few distinct values.

    Regards
    Marc
    SAP NetWeaver RIG

    (0) 
    1. Arun Varadarajan Post author
      Marc,
      We are already on SP18 and release 47. However we had a critical failure of BIA when the following happened:
      A request was rolled back and then subsequent aggregate rollup failed for the BIA. We had just some days ago dropped the aggregates since BIA was there and reports came to s standstill since we could not fill the BIA indices during the day..

      But then managed to come out of it and BIA was rebuilt and everything is fine…

      Arun

      (0) 
  2. Amol Palekar
    Arun- Thanks for this informative blog.

    The time stamp suggests that the job ran for almost an hour and reporting was switched off during this period. This could definitely be a challenge for business critical reports which are accessed 24×7 (global scenario)!

    Regards,
    Amol

    (0) 
    1. Arun Varadarajan Post author
      Amol,
      This again is a cube with about 300 million records and used like crazy by users ( Has our critical sales data ) – but then we benefit from business hours which are not 24X7 and hence scheduled the same at night.

      Also this is the initial fill – not the aggregate rollup.

      Will try an put an explanation for the Aggregate rollup also soon.

      Also one mor thing I got to know is that the network speed between BIA and the network is a decider of the initial fill – thats why 1 GBPS is recommended ( Fiber ).

      Arun

      (0) 

Leave a Reply