API Data extraction patterns from Successfactors Employee Central for delta interfaces

Context

In this blog, I will try to explain the various data extraction mechanism that can be used to retrieve incremental/delta data from SuccessFactors Employee Central using the compound employee API. Based on the requirement of the downstream system,  the EC  SOAP API “CompoundEmployee” can be configured( with various parameters) to get the desired data.

Business Scenarios

Incremental or delta interfaces are the recommended approach ( compared to Full load interfaces) where only the “changed” data is sent down to interfaces. For organizations where there is a large set of employees, sending full data every time can have an adverse effect on the performance of the middleware and also for the target system.That’s why it is always best practice to send only the “changed” data to target systems.

I will try to explain with examples on how to extract the incremental and delta data.

It does NOT matter which middleware ( e.g SAP Cloud Platform Integration, Dell Boomi, SAP PO, Mulesoft etc.) one uses, the query and the underlying logic would remain same.

Delta w.r.t Employee Central Data model can categorised as 3 levels

1. Empoyee Level
2. Record Level
3. Field Level

1) Employee Level Delta

This pattern can be used to extract employees whose data (one or more field – does not matter which portlet or field ) changed post last successful interface execution.

Business Scenario

Let us consider SF has 100 employees and only 10 employees have changed since the last run. This query will fetch only those 10 employees. In this pattern irrespective of any change ( in any record) all the downstream applicable fields will be sent

Pseudo Query

select person,
personal_information,
address_information,
email_information,
phone_information,
employment_information,
job_information,
compensation_information,
paycompensation_recurring,
paycompensation_non_recurring,
direct_deposit,
national_id_card,
payment_information
from CompoundEmployee where 
last_modified_date>= lastSuccessfulRunDate

Here lastSuccessfulRunDate is variable(set in middleware) which stores the last interface run date.

2) Record Level Delta

This Pattern can be used to extract employees whose data (one or more field) changed post last successful interface execution and only changed records are needed.

Business Scenario

SF has 100 employees and only 10 employees have changed since the last run. This query will fetch only those 10 employees.Each employee has multiple record types( or Portlets) [ example – address, personal info, job, comp etc..). For each employee, the interface will map only such records which have changed post last run. Compare the value last_modified_field ( which exists in each portlet) with the lastSuccessfulRunDate in middleware.

Pseudo Query

Same as employee level delta query.

3) Field level Delta

This Pattern can be used to extract employees whose data (one or more field) changed post last successful interface execution and only changed fields(of any record) are needed.

Business Scenario

Considering the previous example(i.e the scenario described in Record Level Delta), if we go one level down i.e. individual fields for each record ( which have last modified date > = watermark) where the downstream system is interested to receive only fields which are changed then this extraction pattern should be used. Example – an employee’s personal information has been changed where the only field is changed ( marital status value changed from “Single” to “Married”).

Pseudo Query

select person,
personal_information,
address_information,
email_information,
phone_information,
employment_information,
job_information,
compensation_information,
paycompensation_recurring,
paycompensation_non_recurring,
direct_deposit,
national_id_card,
payment_information
from CompoundEmployee where 
last_modified_on > lastSuccessfulRunDate  

P.S-  Parameter  fromDate and toDate can also be used along with last_modified_on if period based delta is required.

In the middleware,

1) Compare the value last_modified_field ( which exists in each portlet) with the lastSuccessfulRunDate.

2) As a second step, in the mapping, map only those fields where there is a “previous” tag.

If you want to know on “Current and future dated data extraction using CompoundEmployee API” check here