Skip to Content

We have crossed another little milestone and we are more than 160 now on www.twitter.com/sapbwtweet. Many thanks for being there. Those of you who are still not there, be the part of group at www.twitter.com/sapbwtweet. We hope to put in lot more little tips and tricks there as we move forward and will continue with this weekly micro-blog / tweet.

 

Tweet#9 

 

Till now I have been trying to put some useful tip or trick in this block but this week I am not following the trend and putting a thought (not mine) across for discussion and your comments. 

 

This is what a friend of mine discussed while he was struggling with pretty high (critical and non-critical) volumes of data and critical and non-critical data was coming from same datasource.  Idea he proposed was –  

 

  1. Copy the standard datasource.
  2. Use both the datasources (standard and copied one) with more or less same transformations etc and same data target.
  3. Use different initialization for each datasource (one for critical data and another one for non-critical).
  4. Being two datasources, it shall have two delta queues and while extracting data reading data from delta queue shall be faster as delta queue data volume is less.
  5. Critical data upload can be prioritized.

 

Now the point of discussion is – Do you see any flip side to it?  

 

Needless to say try it on Sandbox, development system before attempting on production environment. 

 

See you next week!!!   

 

Disclaimer – Idea is to learn something in couple of seconds without going through lengthy blogs or articles. BW-Tweet hopes to exist with blogs and articles rather than replacing those. Also this is reproduction of concepts and in no way qualify as an original piece of work.

To report this post you need to login first.

10 Comments

You must be Logged on to comment or reply to a post.

  1. Rajeshkumar Salecha
    Hi Vikash,

    2 datasources of same type with same source will duplicate the read on R/3 and then into same target after loading the critical first – to DSO might be an issue.

    Let me know your view on this.
    Raj

    (0) 
    1. Vikash Agrawal Post author
      Thanks for your comment Raj.

      As records are distinct (but spread over two datasources) so no of calls for reading R/3 data shall remain same.

      You are right about DSO and using the same data target so scenario is modified to have separate data targets one each for one datasource.

      Thanks again

      Vikash

      (0) 
      1. Vijay Vijayasankar
        If underlying code is the same – then the only thing that is different between the original extractor and the copy is the selection of data, and delta inits.

        Copying the extractor is not an extensible solution. If there are 3 levels of criticality tomorrow instead of 2, will we copy a third time and maintain 3 separate extractors? 

        So I would think a BW side solution makes more sense. Get one common init into a DSO, and from there on – use deltas on DSO to propogate to various downstream systems. I assume we would have exhausted all source system optimizations before we go down this path on BW side.

        Or may be we can leave the non-critical data in source itself, and report on it virtually.

        (0) 
        1. Vikash Agrawal Post author
          Thanks for your comment Vijay.

          BW side solution sounds more logical, yes. But scenario is I shall be reading one delta queue for critical (may be few hundered records) and more non-critical data (may be millions), that means scanning millions of records even for few hundred cricial records.

          Any more thoughts please.

          Regards

          Vikash

          (0) 
      2. Rajeshkumar Salecha
        Again, adding to other commenter’s point if time is constraint along with requirement, then you can check the write-optimized DSO method than the standard DSO… that might save time and as what i belive is meant for such kind of scenarios.

        Well, do let me know what was the final approach you implemented and what are it’s results.

        (0) 
  2. Henry Janz
    wouldn’t it be better to use optimise SAPs standard extractor so it spawn multiple parallel jobs may be running slightly different cursors and then sending the data to BW using tRFC?

    Is the problem pureley on the extraction side? If so then this might be enough.

    If not then it becomes a SAP basis question on providing many DIA processes to process these incoming tRFCs.

    I do not think that creating sets of “parallel” datasources is the right way to go, e.g. how many will you have to create to make maximum use of the available processes on source and target system?

    Cheers
    Henry

    (0) 

Leave a Reply