How To filter on the initial load & parallelize replication DMIS 2011 SP06 or higher
Hi SLT community,
This How To describe a new expert function released with DMIS 2011 SP6 wthat allows to filter directly in the source for the initial load to reduce the transfer volume (if you have anyway in mind to filter on the SLT system) and also the option for the definition of a parallelized replication.
Important: DMIS 2011 SP06 or higher
Use Case
Use Case: Filter Initial Load
You don’t want to load the entire source table but a filter on the SLT is not the optimal solution as most of the data is transferred to the SLT system and skipped here. In case you filter a significant part of a table it’s better to already limit the reader module of the initial load accordingly
Ensure that you use a reading type 4/5. Without it will not work.
Use Case: Parallelize Replication
You need to parallelize the replication of a table as there are high change rates and one parallel task can not catch up delta data within expected latency time.
Use Case: Parallelize Initial Load
In order to parallelize the initial load (more precisely,the access plan calculation which is required before the real data transfer can be done – the actual transfer will be automatically parallelized), there is another option availabe, which automatically determines the subsets to be processed by each of the parallel-running jobs. For details, see http://scn.sap.com/community/replication-server/blog/2013/09/26/how-to-improve-the-initial-load-by-row-id-approach
However, if you can define such subsets yourself, this might be more efficient, firstly because you save the first step to determine the subsets, and secondly because the access might be more efficient, if you define the subsets by only giving values and / or intervals for the first key fields rather than for all the key fields of the table.
Process
With DMIS 2011 SP6 a new table is available where Filter for the Initial Load and / or settings to parallelize the replication can be defined. The eintries in table DMC_ACSPL_SELECT have to be defined on the SLT systems
Fieldname |
Description |
---|---|
MT_ID |
Mass Transfer ID |
MIGR_OBJ_ALIAS |
Table Name Attention: in case of cluster tables, you need to specify:
So you need to maintain two records if the setting should be valid |
VALIDITY |
Validity of this setting |
ACS_PLAN_ID |
Access Plan ID |
FIELD |
Fieldname |
CURRNUM |
Line Number (0001 – 9999) to define multiple records for the same access plan ID |
LOGICAL_OPERATOR |
Logical Operator (allowed in Open SQL where clause) |
VALUE |
Filter / parallelization value for current field |
GATING_OPERATOR |
required if several records for the same field are defined |
The value has to be defined properly based on the internal format of the corresponding data type. Character-like field values have to be defined with single quotes. NUMC fields have to be defined with single quotes and leading zeros.
Process: Filter Initial Load & Replication
You can enter filter criteria for any field of the source table. If you enter mutliple filter criteria for one field you have to define a gating operator (AND / OR). If you define filter on multiple fields the filter from the indiviual fields will be concatenated with gating operator AND.
If you define a filter for the replication also a filter on the DB trigger is recommended as otherwise the changes that don’t fit to the filter criteria will not be handled and remain in the logging table.
Example 1: Filter on one field for a single value
Define filter criteria for table BKPF in mass transfer ID 001. Only those records should be loaded where GJAHR is greater or equal 2013. The filter criteria shall be relevant for initial load and replication
MT_ID |
MIGR_OBJ_ALIAS |
VALIDITY |
ACS_PLAN_ID |
FIELD |
CURRNUM |
LOGICAL_OPERATOR |
VALUE |
GATING_OPERATOR |
---|---|---|---|---|---|---|---|---|
001 |
BKPF |
1 |
00000 |
GJAHR |
0001 |
>= |
2013 |
The following where clause will be used for the select:
SELECT FROM BKPF |
Example 2: Filter on one fieldfor mutliple values
Define filter criteria for table BKPF in mass transfer ID 001. Only those records should be loaded where BUKRS is equal 0010 or between 1000 and 3000. The filter criteria shall be relevant for the initial load only
MT_ID |
MIGR_OBJ_ALIAS |
VALIDITY |
ACS_PLAN_ID |
FIELD |
CURRNUM |
LOGICAL_OPERATOR |
VALUE |
GATING_OPERATOR |
---|---|---|---|---|---|---|---|---|
001 |
BKPF |
2 |
00000 |
BUKRS |
0001 |
= |
‘0010’ |
OR |
001 |
BKPF |
2 |
00000 |
BUKRS |
0002 |
>= |
‘1000’ |
AND |
001 |
BKPF |
2 |
00000 |
BUKRS |
0003 |
<= |
‘3000’ |
The following where clause will be used for the select:
SELECT FROM BKPF |
Example 3: Filter on multiple fields
Define filter criteria for table BKPF in mass transfer ID 001. Only those records should be loaded where GKAHR is greater or equal 2013 and where BUKRS is equal 0010 or between 1000 and 3000. The filter criteria shall be relevant for the initial load only
MT_ID |
MIGR_OBJ_ALIAS |
VALIDITY |
ACS_PLAN_ID |
FIELD |
CURRNUM |
LOGICAL_OPERATOR |
VALUE |
GATING_OPERATOR |
---|---|---|---|---|---|---|---|---|
001 |
BKPF |
2 |
00000 |
GJAHR |
0001 |
>= |
2013 |
|
001 |
BKPF |
2 |
00000 |
BUKRS |
0002 |
= |
‘0010’ |
OR |
001 |
BKPF |
2 |
00000 |
BUKRS |
0003 |
>= |
‘1000’ |
AND |
001 |
BKPF |
2 |
00000 |
BUKRS |
0004 |
<= |
‘3000’ |
The following where clause will be used for the select:
SELECT FROM BKPF |
Process: Parallelize Replication
You can enter ranges to split the replication into parallel tasks. The number of defined ranges also defines the maximum parallel processing of the current table. How many jobs will finally procssing a table is defined by the overall number of jobs and the maximum number of parllel jobs defined for a table (can be defined in LTRC -> Data Transfer Monitor). If you define range criteria for the parallelization of the replication you need to define disjount subsets of the data to be processed by each of the replication jobs.
Example 1: Parallelize Replication
Split replication of table BKPF in mass transfer ID 001 into 4 ranges. The ranges are defined based on field:
Range 1: 0000 -> 2000
Range 2: 2000 -> 5000
Range 3: 5000 -> 8000
Range 4: 8000 -> 9999
MT_ID |
MIGR_OBJ_ALIAS |
VALIDITY |
ACS_PLAN_ID |
FIELD |
CURRNUM |
LOGICAL_OPERATOR |
VALUE |
GATING_OPERATOR |
---|---|---|---|---|---|---|---|---|
001 |
BKPF |
3 |
00001 |
BUKRS |
0001 |
<= |
‘2000’ |
|
001 |
BKPF |
3 |
00002 |
BUKRS |
0001 |
> |
‘2000’ |
AND |
001 |
BKPF |
3 |
00002 |
BUKRS |
0002 |
<= |
‘5000’ |
|
001 |
BKPF |
3 |
00003 |
BUKRS |
0001 |
> |
‘5000’ |
AND |
001 |
BKPF |
3 |
00003 |
BUKRS |
0002 |
<= |
‘8000’ |
|
001 |
BKPF |
3 |
00004 |
BUKRS |
0001 |
> |
‘8000’ |
The following where clause will be used for the select:
Range 1:
SELECT
FROM
BKPF
WHERE
BUKRS <=
'2000'
.
Range 2:
SELECT
FROM
BKPF
WHERE
BUKRS >
'2000'
AND
BUKRS <=
'5000'
.
Range 3:
SELECT
FROM
BKPF
WHERE
BUKRS >
'5000'
AND
BUKRS <=
'8000'
.
Range 4:
SELECT
FROM
BKPF
WHERE
BUKRS >
'8000'
.
Example 2: Parallelize Replication & Filter
Split replication of table BKPF in mass transfer ID 001 into 4 ranges as defined in example 1. In addition only those records should be loaded where GJAHR is greater or equal 2013.
MT_ID |
MIGR_OBJ_ALIAS |
VALIDITY |
ACS_PLAN_ID |
FIELD |
CURRNUM |
LOGICAL_OPERATOR |
VALUE |
GATING_OPERATOR |
---|---|---|---|---|---|---|---|---|
001 |
BKPF |
1 |
00000 |
GJAHR |
0001 |
>= |
2013 |
|
001 |
BKPF |
3 |
00001 |
BUKRS |
0001 |
<= |
‘2000’ |
|
001 |
BKPF |
3 |
00002 |
BUKRS |
0001 |
> |
‘2000’ |
AND |
001 |
BKPF |
3 |
00002 |
BUKRS |
0002 |
<= |
‘5000’ |
|
001 |
BKPF |
3 |
00003 |
BUKRS |
0001 |
> |
‘5000’ |
AND |
001 |
BKPF |
3 |
00003 |
BUKRS |
0002 |
<= |
‘8000’ |
|
001 |
BKPF |
3 |
00004 |
BUKRS |
0001 |
> |
‘8000’ |
The following where clause will be used for the select:
Range 1:
SELECT
FROM
BKPF
WHERE
GJAHR >= 2013
AND
BUKRS <=
'2000'
.
Range 2:
SELECT
FROM
BKPF
WHERE
GJAHR >= 2013
AND BUKRS >
'2000'
AND
BUKRS <=
'5000'
.
Range 3:
SELECT
FROM
BKPF
WHERE
GJAHR >= 2013
AND
BUKRS >
'5000'
AND
BUKRS <=
'8000'
.
Range 4:
SELECT
FROM
BKPF
WHERE
GJAHR >= 2013
AND
BUKRS >
'8000'
.
Enjoy this new features :smile:
Best,
Tobias
Hi Tobias,
Great blog, as usual 🙂
Couple of questions:
If I am already using DMC_SELSTRING for filtering at source, do I need to migrate to DMC_ACSPL_SELECT under SP06 or will the DMC_SELSTRING continue working?
Also, for Validity 1 (initial load and replication), given that the trigger remains unfiltered without other intervention, would it not be easier to just filter the initial load in the source (Validity 2) and then use IUUC_REPL_CONTENT for the replication filtering?
Last question is probably more for my understanding, but in your example for parallelized replication, you use table BKPF. Now, given we are talking latency and that a complete Finance document requires BKPF and BSEG, would you not also need to parallelize replication for BSEG in addition to BKPF? Or does it not really matter much due to the sequence of the packages being transferred i.e. there will always be a mismatch between BKPF and BSEG for a 'short' period of time?
Thanks
Kris
Hi Tobias,
Very nice Blog with some exciting information.
As mentioned by Kris, I've the same question:
Also, for Validity 1 (initial load and replication), given that the trigger remains unfiltered without other intervention, would it not be easier to just filter the initial load in the source (Validity 2) and then use IUUC_REPL_CONTENT for the replication filtering?
Thanks & regards,
Jomy
Hey,
here are the answers
1.) DMC_SELSTRING is also working after the update to SP06, but for new tables we recommend using DMC_ACSPL_SELECT, because it is easier to consume.
2.) Your are right, the best setup would be validity 2 to have a filter definition for the load + LTRS (new UI for IUUC_REPL_CONTENT) or direct on trigger lvl for the replication itself. At the moment if you would define validity 1, the trigger would write all entries into the logging table and our selcet on the logging tables would filter. The result would be that records will be stored in the logging tables without any further processing. We will work to have a clean report or the option to get the trigger coding accordingly adjusted. Hopefully with SP7.
3.) If you expect the same amount of data for BSEG in genreal, of cousre you could also parallelize. But in general there is no need. It is as you described, it do not really matter due to the sequence of packages.
Best,
Tobias
Thanks Tobias!
I will let you know how it goes...aiming to load COEP using DMC_ACSPL_SELECT...source will have close to 5 billion records. I have tested on a small system and it worked fine, so hoping that same will apply to the large volume 🙂
Looking forward top SP07 already!
Kris
Hi Tobias,
I tried this without the trigger options in ltrs. if no trigger option defined for the table the data will not be filtered during replication only initial load will be filtered.
HI Tobias,
Your blog is great and gave me good insights on filtering techniques in SLT.
I wanted to know, instead of writing codes for each and every table in LTRS for filtering data.
If we can write a generic code which will be applicable for all the tables for a particular connection between ECC and HANA using SLT
We are on DMIS 2011_1_731 SP07.
Hi guys,
We are on DMIS SP6 with notes up to correction 09 applied.
We tried several cases by filling table DMC_ACSPL_SELECT to either filter or parallelize the extracts. But none of these functions seem to work in our environment. For the table we do only simple filtering, the data gets extracted to HANA but no filtering is done. For the tables we tried to parallelize, the access plan gets calculated, but after that nothing happens, nothing loads.
So we are able to replicate tables the normal way, there are no ressource/job issues, but theses features doesn't work for us. Is there any other config part that is missing? Was this working for you the first time?
Thanks!
Christian
I'm really struggling to get IUUC_PRECALC_OBJ to work with BSEG in Oracle source database with DMIS 2011 SP06 up to Correction 11 applied. RFBLG is 177 million rows. I have NUMREC set to 35,500,000 to get 5 parallel calculation jobs in source. My reading type is 4. With this setup, some of the calculation jobs in source system complete but some fail with SAPSQL_INVALID_FIELDNAME after many hours of running. I thought SAP Note 1989212 "Shortdump in access plan calculation jobs, if reading type 4 is used for cluster tables" applied to my source system would help but it didn't. Now I'm not wondering if the correction contained in 1989212 is even right.
Do you think I should give up on IUUC_PRECALC_OBJ altogether and switch to DMC_ACSPL_SELECT? I'm thinking that SLT is having problems in the source while calculating boundaries and when it does I get SAPSQL_INVALID_FIELDNAME. Would I avoid a lot of that SLT logic with DMC_ACSPL_SELECT? Or should I stick with IUUC_PRECALC_OBJ and be more patient with SAP support?
Evert
SAP did release a fix for NUMREC of IUUC_PERF_OPTIONS in a new version of note 1989212 but we ended up switching to DMC_ACSPL_SELECT after deciding to both parallelize and filter the initial load.
Evert
In regards to DMC_ACSPL_SELECT not filtering, I think the DMC_ACSPL_SELECT filter logic is only used during the access plan calculation. So, if you choose a reading type (in transaction LTRS) that does not do an access plan calculation, then the table DCM_ACSPL_SELECT is never looked at by SLT. The default reading type "3" doesn't have do an access plan calculation for example and so DMC_ACSPL_SELECT isn't used. For me, with table BKPF, when trying to filter by GJAHR, the filter wouldn't work until I switched BKPF's reading type from "3" to "5".
Evert
Tobias/All,
Just to clarify, the entries in DMC_ACSPL_SELECT for initial load selection only when using multiple fields on one table. The blog documentation shows that Validity should be 1 for the first field and 2 for the second. Is this correct??
but for Example 2 you have Validity at 2 when using BUKRS only.
So given that Validity 1 states ‘Initial load + Replication’, do we still use it when the desired select is to be on initial load only?
Thanks
Kris
Thx Kris, it is a typo. Beer on this years d-code on my bill
Ha! That's an offer that's hard to refuse! Throw in some Jagermeister and we have a deal!
On a serious note
, the filtering works great now, however, I am finding a strange scenario (probably my fault) when I restrict a filter and replicate same table again. As an example:
BUT
The 'old table entries' in HANA are not dropped. So I get the filtered data for year 2014 and company code 1234...AND the old entries from the first replication where filter was only for Fiscal year.
As a workaround I have to manually drop the rows in the HANA table before starting the replication with the new filter.
I admit that maybe I am doing something wrong and will re-test this scenario again this week.
Thanks
Kris
Hi Tobias,
I have an issue with this it works fine in the load phase after that it turns the read type to 1 -initially it was set to 4 or 5 I have tested both - and filtering is not working any more did I missed any thing here?
do I have to use trigger filtering with this to work correctly?
how can I use the trigger filtering any reference for syntax ?
Thanks,
Mohamed.
Hi Tobias,
Nice blog.
Just one question , I am not able to put entries in table DMC_ACSPL_SELECT as I am not having developer access. Is there any way for this (like using LTRS transaction)?
Regards
Kartik
Hi Kartik,
so far there is no option to specifiy this settings in ltrs. This is planned for the future. But I cannot give you a due date.
Best,
Tobias
Hi Tobias,
As this post is nearly a year old I wanted to ask the question; are you aware if there is still no way to do this via LTRC? Is SE16 still the only route?
Thanks!
-Patrick
Also we are on DMIS 2011 SPS08 now. Thanks.
Ok I used SE16 since I didn't hear back and I had success! But I'm getting pushback from management for another alternate way to do this other than SE16 in production.
Hi Patrick.
Have a look at note 2191214 - Installation/Upgrade SLT - DMIS 2011 SP9.
In there you can see the following:
1. Corrections / Enhancements in DMIS 2011 SP9
Probably easier to got to SP9 than writing own front-ends
!
Cheers
Kris
Ok thanks Kris, will take a look.
-Patrick
Just read the note and looks like exactly what I needed to know. Hopefully we will goto SPS09 soon.
Thanks again.
Hi Kartik,
Have you tried to create entries through SE16 or SE16N? I maintain DMC_ACSPL_SELECT through SE16 without the need for Dev access...
Thanks
kris
Hi Kris,
Thanks for your comment.I just tried creating entries thorugh SE16 and it worked.
Regards
Kartik
I don't see a place in LTRC to set up the select statements you indicated the parallelize secitonof this blog. we have already load the table, but the replication is slow. please indicate how to add these select statments and does it have to be based off the prmiary key of the table (i.e BSEG, SL_GIRID).
As I described in the blog, it is an non-standard expert function :
The eintries in table DMC_ACSPL_SELECT have to be defined on the SLT systems
Best,
Tobias
I just maintained different ranges for a big table in DMC_ACSPL_SELECT. Do I have to stop replication --> restart initial load to get the parallel replication activated? I did not found any other documentation how to change from single to parallel load.
thanks
Bernd
Hello Bernd,
Yes. You will have to stop and start. But if you are already in replication mode, you don't need to start from initial load. You could maintain your ranges with validity = 3 in DMC_ACSPL_SELECT table and stop the replication and start it again. It would reset the migration object for the corresponding table with the updated ranges.
Thanks and regards,
Venkat.
Hello Venkat,
everything was working fine with parallel replication based on range of one column.
Now we have implemented DMIS2011 SP8. It should be possible to have also hash parallelism but I cannot find any documentation about how its configured.
Does someone knows how it works or where I can find detailed description how to configure SP8 features?
thanks and regards
Bernd
Hi Tobias,
I'm doing selective data load (day wise) for CE1XXXX by giving entry in dmc_acspl_select. After doing stop and start replication on CE1XXXX table only selective data is showing in target system and other data is getting deleted.
Please check the entries which I am maintaining below.
Regards,
Biswabrata Das
Hello all,
Tried the filtering, but does not work - it loaded all table. Values entered are
MT ID 076
MIGR OBJ VBAP
Validity 1
ACS PLan ID 00000
Field ERDAT
CURRNUM = 00000
LOGICAL OPERATOR >=
VALUE '20150101'
ERDAT has data element as DATS. I have tried (a) enclosing the value in single qoute and (b) without the quote - both does not work. I have done stop (which deleted all values) and then replicate - but still loaded all the source table entry.
Would appreciate help.
Thanks,
Terry
I have the same problem on another ECC table, have you found a solution?
Other filters on non-DATS data types works fine for me.
Thanks.
Filter works fine also for DATS on Sp9
Hi Terry,
How Did you resolved this issue ? I am also facing the same issue currently.
Thanks,
Somen
Did you find a solution?
I tried CDPOS and BSEG for initial load filtering and both do not work. All records are loaded. Maybe because they are cluster. I setup as Reading Type 5 per instructions.
Hi ,
I am looking to filter KONV table with the following criteria
KSCHL IN ('ZHCP','ZR00','ZWAS') AND ZAEHK = '01'
so i added the following interies in DMC_ACSPL_SELECT
003 KONV 1 00000 KSCHL 0000 IN ('ZHCP','ZR00','ZWAS')
003 KONV 1 00000 ZAEHK 0001 = '01'
003 KOCLU 1 00000 KSCHL 0000 IN ('ZHCP','ZR00','ZWAS')
003 KOCLU 1 00000 ZAEHK 0001 = '01'
do i still need to use trigger ?
would trigger work for cluster table fileds ?
Thanks,
Mohamed.
Hi Mohamed,
How Did you resolved this issue ? I am also facing the same issue currently.
Thanks,
Somen
Hi Tobias,
Hope you are doing great.
I have gone through your blog .
It is a nice blog and tried to implement it. But my load is not happening & Calculation is still in progress even though 24 Hrs has already gone. Still the calculation job in running in source ECC system.
I have a requirement to load the MSEG table with BWART = 'XXX' where I put the filter for 6 different values of XXX. MSEG contains total of 3 bn records in it.
I maintained the values in DMC_ACSPL_SELECT , maintained the Performance settings as well. Then I started the replication of MSEG. But I don't know why it is not working.
Could you please let me know .. Is there any other way to fulfill my requirement....
Thanks in Advance.
Biswamber
Hi Biswamber
For large tables where I need to use parallel init including parallel calculation job run, I have been using this one:
SLT Replication - BSEG and BKPF tables
Read the reply from Mahesh Shetty, this works for me every time.
What you also can try is using a viewtable with your filters and define the view table in LTRS to be used for Initial load. The calculation plan will use the view table. You still need to start the parallel calculation jobs.
Hope it will work for you.
Does SLT supports the filtering of Data, based on a common field from TWO different tables in the Source system ECC? Target is S4 HANA table.
Hello Tobias,
Currently we have a scenario in which we are having three systems ( 2 ECC & 1 BW ) replicating to native HANA system via SLT. Now we want to use the same SLT system to establishing the replication between ECC to BW system without distributing the initial setup.
Please let me know if it possible to replication the same ECC schema in two system at the same time one to Native HANA and other to SAP BW on HANA system.
Regards,
Tahir
Hi Tobias,
Thanks for the blog!
Could you please write examples for logical operators LIKE, IN, [] (between). How must be the content of Value column for these operators?
Thanks and regards,
Alfonso.
Hi ,
we are considering a SLT replication for BSEG, BKPF and there is a huge amount of data (billions documents)
Anyway we need only the open items (AUGBL blank), and S ledger documents, so from some billions the needed records can be reduced to few hundreds millions.
Now let's imagine that an SLT replica and filtering scenario works fine in delta.
Now my doubt is on how filtering works with delta.
Example.
Suppose last week I've replicated from ECC via SLT the document 0000001 and line item 010 which was an open item. So it came correctly into Hana because it passed the filtering criteria.
Let's now suppose that this week, the document, same line item, gets cleared. This means that now doc. 0000001 and line item 010 on ECC are now cleared in BSEG, but now this record do not satisfy my SLT filter criteria (because BSEG-AUGBL field is not blank but contains the clearing doc. number).
Will the SLT delta be still able to remove this cleared document from my Hana table ?
Or the record will be just filtered out during SLT load (because do not satisfy filtering criteria) and will never pair with the corresponding open item record which I loaded the week before in Hana ?
I am afraid this scenario will happen and I need to load all documents. Then I should use the filtering criteria in the calc. view for example.
Any help would be extremely appreciated.
Tnx in advance
Hi Tobias,
We are targeting on one of the big Z-table(transparent table) in our system which is like a copy of BSEG. it has nearly 1.8 billion records in it. So, we were trying the filter approach (DMC or ABAP code or both) including table partitioning from LTRS and parallel replication approach. Could you please help me in this? I tried below 6 approaches but did not receive any satisfactory results -
a.) Filter maintained under table DMC_ACSPL_SELECT for RYEAR >=2015 with Validity = 1 [= use for initial load and replication (filter load and parallelize replication)].
b.) Parallelism Criteria Set with parallel job as 5.
- Initial Load worked with filter approach for RYEAR >=2015.
- Delta Replication didn't work with filter approach for RYEAR >=2015.
- It replicated the delta entries to HANA for 2014 and 2017. Whereas, our aim was only to get the entries for >=2015.
a.) Filter maintained under table DMC_ACSPL_SELECT for RYEAR >=2015 with Validity = 2 [= use for initial load only (filter load)].
b.) ABAP code included.
c.) Parallelism Criteria Set with parallel job as 5.
- Initial Load worked with filter approach for RYEAR >=2015.
- Delta Replication didn't work with filter approach for RYEAR >=2015.
- It replicated the delta entries to HANA for 2014 and 2017. Whereas, our aim was only to get the entries for >=2015.
a.) Filter maintained under table DMC_ACSPL_SELECT for RYEAR >=2015 with Validity = 2 [= use for initial load only (filter load)] & Validity = 3 [= use for replication only (parallelize replication)].
b.) ABAP code included.
c.) Parallelism Criteria Set.
- Initial Load worked with filter approach for RYEAR >=2015.
- Delta Replication didn't work with filter approach for RYEAR >=2015.
- Delta replication entries were stuck in logging table and were not replication to HANA.
- ECC system started dumping with SAPSQL_PARSE_ERROR
a.) Filter maintained under table DMC_ACSPL_SELECT for RYEAR >=2015 with Validity = 1 [= use for initial load and replication (filter load and parallelize replication)] & Validity = 3 [= use for replication only (parallelize replication)].
b.) ABAP code included.
c.) Parallelism Criteria Set.
- Initial Load worked with filter approach for RYEAR >=2015.
- Delta Replication didn't work with filter approach for RYEAR >=2015.
- ECC system started dumping with SAPSQL_PARSE_ERROR
a.) Filter maintained under table DMC_ACSPL_SELECT for RYEAR >=2015 with Validity = 2 [= use for initial load only (filter load)].
c.) Parallelism Criteria Set with parallel job as 5.
- Initial Load worked with filter approach for RYEAR >=2015.
- Delta Replication didn't work with filter approach for RYEAR >=2015.
- It replicated the delta entries to HANA for 2014 and 2017. Whereas, our aim was only to get the entries for >=2015.
a.) Filter maintained using ABAP Code - Z_IMPORT_MODIFIED_ZZZA.
c.) Parallelism Criteria Set with parallel job as 5.
1.) More job Runtime for Calculation - Calculation job which were taking 60 secs to complete with DMC approach is not taking 300 to 400 secs to complete. Hence, more time for calculation.
2.) Performance Impact as per SAP.
Where Z_IMPORT_MODIFIED_ZZZA is as below –
SPAN {
font-family: “Courier New”;
font-size: 10pt;
color: #000000;
background: #FFFFFF;
}
.L0S31 {
font-style: italic;
color: #808080;
}
.L0S33 {
color: #4DA619;
}
.L0S52 {
color: #0000FF;
}
.L0S55 {
color: #800080;
}
.L0S70 {
color: #808080;
}
*&———————————————————————*
*& Include Z_IMPORT_MODIFIED_ZZZA
*&———————————————————————*
DATA: moff TYPE i,
mlen TYPE i,
mcnt type i,
match TYPE match_result,
tstamp TYPE char20.
if <WA_R_ZZZA>–RYEAR LT ‘2015’.
* skip record.
SKIP_RECORD.
else.
*do nothing for records >= 2015 and replicate to HANA
endif.
Hi Tobias,
I started to use the SLT few months ago, and now I´m trying to create a complex filter.
The table is LFB1, and in this table I want to filter data fron MANDT 100 (all) and MANDT 120 (After LIFNR 0002000100), but every time when I put it at LTRS I have a DUMP in sender system at the moment of replication.
There is any limitation, or it´s possible to create this filter?
Best regards,
Ivan
Before I run the calculation steps in parallel via MWBMON->Steps->Calculate Access Plan I need to wait for the Calculated column to turn to 'P' in the Data Transfer Monitor. This can take several minutes to get to this point and as far as I know there is no automated way to kick this off.
Does SAP have a solve to this? For example, define a default variant to use that gets triggered when the calculated state goes to 'P' mode.
Hi Tobias
please I discovered that it's not possible to set the filter on Cluster tables for process "Replication only" with DMIS 2011_1_731 sp13 .
The second row for the RFBLG table simply disappear from table DMC_ACSPL_SELECT.
What I did:
I added BSET in LTRC-> Advanced Replication Settings, using Reading Type 4 as it's a cluster Table; the BSET has a filter on the GJAHR field.
In Advanced Replication Settings you can just set the filter for process "Initial Load Only"
The if you try to add a new entry in Advanced Replication Settings for the RFBLG , with the same filtre on GJAHR for process 'Replication Only' and you save, the entry disappear as soon you do refresh or leave the transaction.
The same if you try to add directly the entry RFBLG in DMC_ACSPL_SELECT in SE16.
You do "save" and the entry is there, you go back and enter again the transaction and the item is gone.
So at the end in table DMC_ACSPL_SELECT I have only one record for the cluster table BSET, specific for the Initial Load process.
THen from LTRC-> Data Providing I try to run the 'Replication' for BSET table, and it worked fine, I mean the Initial Load has been performed using the filter I set in Advanced Replication Settings.
But then I realized that the Replication is happening really without keeping in consideration any filter on GJAHR field as expected.
So I'm wasting space on the target database for unwanted records.
How I can add the second row specific for the Replication Process and where ?
Maybe in DMIS 2011_1_731 sp13 there is another table ?
What I'm missing ? Or it's a bug ?
best regards and thanks in advance