The importance of setting the correct package size for SPM extractors, especially transaction data extractors, cannot be overstated. Here is an example from a real life customer scenario to prove my point. After changing the package size for invoice extractor from 40,000 to 1000, the load time for 1M records reduced from 10 hrs to 1 hr 30 min.
I am hoping I got your attention. So let us get into some specific details now.
What is the right package size?
Well there is no "one size fits all" number but in general for master data, recommendation would be 10,000-20,000 and for transaction data 1,000-5,000 works best. Especially for invoice extractors, IVBSIK and IVBSAK, we recommend that the size be under 3,000.
Wait. But isn't a higher package size better for performance since that will mean less DB hits?
Typically yes, but not always. In case of SPM transaction data extractors a lot of cross-reads (Invoice extractor read PO, Contracts, and Materials etc) are done after selecting from tables. If the initial package size is very large then all the internal tables becomes huge and search on them becomes time consuming. Smaller package size means that internal tables are smaller and processing is faster.
Ok that makes sense. But a package size of 1,000 still seems a bit extreme.
The package size determines the number of records which will be selected from first table for processing. That does not always correspond to the number of records which will be extracted as part of package. In invoice extractor the first selection table is BSAK/BSIK which only contains the supplier lines. But the number of records extracted corresponds to BSEG which has all the lines. So for a package size of 1,000 you can expect the number of records extracted to be 10,000 if the average number of lines per invoice is 10. So package size of 1,000 may actually mean 10,000.
So where can this setting be made?
In case you are using direct data source, this setting can be made for the infopackage on the SPM system with the menu option Scheduler ->DataService Default data transfer. For the flat file load this setting can be made in the table ZSA_FFCUSTTABL on source side and in table RSXAADMIN on the BI side.
I guess this blog should give you a good start on package size setting on SPM extractors. For further details on this setting and other performance improvement tips for extractors please refer to the documentation attached to the note 1239883.
