SuccessFactors Recruitment Data Migration – A Journey

I have recently completed SuccessFactors Recruitment Data Migration for a client and I would like to put forward my experiences and lessons learnt in this blog post. Person reading this article must have an idea about SF Recruitment module. I have included the average time taken to load each Recruitment object. This time might vary depending on Data Centre’s performance at a given point of the day. Workaround per object for the system limitations like Internal Candidate Application load, Background Elements for Internal Candidates is also included.

Data Objects for Recruitment Load

The following data objects can be migrated to SuccessFactors system as part of Recruitment Load

1. Candidate Profiles
2. Candidate Background Elements
3. Candidate Tags and Candidate Tag Assignments
4. Candidate Attachments
5. Job Requisitions
6. Applications
7. Application Attachments

Jobs are scheduled in Provisioning to initiate each load. More details about setting up job and preparing data load files is available in Recruitment Handbook.

Manual Correction of Data

Recruitment data once loaded cannot be updated or deleted through the load process. Updating and deleting will be a manual activity and can be only done record by record. Hence a pre-load validation of data is very important to ensure that the data loaded in the system is accurate. Enough time must be allocated to perform a thorough check on the transformed and ready to load data.

Best Time to Migrate Data

The best time to load the data is during week days and business hours. SuccessFactors systems usually undergo maintenance during Friday-Saturday, and Saturday-Sunday midnight to early hours in the morning. The data load jobs must not run during this period as either the jobs are interrupted or the data will be loaded incorrectly.

Journey of Data Migration

I used the word ‘Journey’ as we did go through lot of hitches and learnt many lessons before loading the data in the production instance. At least 5 dry runs were initiated in multiple test instances to improve the data quality, load time and handle load errors before doing the final production load. The story of loading each Recruitment object goes as follows.

1. Candidate Profiles

As part of Recruitment load only External Candidates’ data can be migrated. Internal Candidates are users in the system. Their Candidate Profile gets created automatically whenever their users are created in the system.

While loading Candidate Profile data, the candidate’s CV can also be loaded. The system does not support to load the Cover Letter of the candidates. As per SuccessFactors documentation, the system supports loading CVs of type .doc, .txt and .pdf only. But, in reality the system supports more than the above 3 types. We could load .docx, .rtf, .jpg, .HTML, .xls .xlsx, .PNG as well.

Loading the Candidate Profile data is the most challenging and time consuming part in Recruitment. Our client had 80,000 Candidates to be migrated along with a CV for each candidate. After many trials and failed attempts we came up with the best number of candidate records per file. The CVs of the candidates must be distributed in different folders. The ideal number of records per file is 5000 and number of CVs per folder is 500. These numbers have also been recommended by SuccessFactors.

If the load file size is big, i.e. has more than 5000 records, the job stops abruptly after a while. To the consultant it seems that the job status is ‘In Progress’. But in the backend the program gets struck and doesn’t run. In such cases we had to rerun the job. Luckily the system does not create duplicate Candidate Profile records. It errors out saying candidate already exists. So there is no harm in running the same job any number of times.

If the number of CV files per folder is more than 500, the system times out while finding the correct resume. Hence the number successful candidate records loaded in the system will be very less. Another major issue we found, which is independent of the number of files in a folder, while loading candidates with their resumes is a weird error that says ‘Error in importing resume’. We could not figure out the reason as why this issue occurs. The error occurs even if the resume exists in the right folder and the correct folder path is mentioned in the load file. Also the filename has simple alphanumeric characters with an underscore. Sometimes the same records the fall out in previous run will load when the job is run for the second time. But this is not consistent either.

After all the unsuccessful attempts we have finally reached out to SuccessFactors to load all the Candidate Profiles (only). They loaded 2 files of 5000 records each at a time by running some script in the backend. This is the most efficient method of getting the profiles loaded in the system. The time taken to load data per file is approximately 3 hours. In spite of encountering the same error (Error in importing resume) the number of fallouts now is less than 1%, which is pretty good when compared to 60% – 80% fallouts when we tried through Provisioning.

On the contrary, if the Candidates are loaded without a resume, the success rate is 100%. Such a load can be carried out by the client/consultant independently without SuccessFactors help. Also the time taken is less than 30 mins for 3000 candidates.

2. Candidate Background Elements

Loading Background Elements is pretty smooth after completing the herculean Candidate Profiles task. Background Elements for External Candidates is part of the Recruitment load. There is no way to load the Background Elements for Internal Candidates through Recruitment. The workaround we got is to define the same Background Element in Succession Data model. Then load the data through Import Extended User Information in Admin Tools of the instance. This data is first loaded to Employee Profile and then automatically syncs with Internal Candidate Profiles. The sm-mapping for the background elements must be maintained in Candidate Profile XML template.

The only care to be taken is not to run the same job more than once. Unlike Candidate Profiles, running the same job multiple times would create duplicate records. We have also noticed that time taken to load 150000 Background Elements records in one file takes more than 5 hours. Hence we split the load file in smaller files of 30000 records each and then loaded all the small files at once in parallel. The load time then was less than an hour for all thefiles.

3. Candidate Tags and Candidate Tag Assignments

Loading Candidate Tags and Candidate Tag Assignments is straight forward. Did not face any issues but is supported only for External Candidates through the load. Reloading the same file more than once does not create any duplicates. The load time is less than a minute for 300 records.

4. Candidate Attachment

An additional document, besides CV, can be loaded against an External Candidate through Candidate Attachment Load. This additional document is visible only in the screen that is displayed when searching for a candidate in the Instance. It is only visible to the Recruiters or whoever has got permission to search for the Candidates. The Candidate themselves cannot see the document.

Through the Candidate Attachment Load only one document can be loaded for a candidate. Loading the second document for the same candidate replaces the first document.

The job takes around 15 mins to load 2500 attachments. We have stored 500 documents in each folder.

5. Job Requisitions

We did face a few glitches while loading Job Requisitions, but they were manageable. The major issue we had for which we didn’t know the reason, even till date, is for the error “Department is not mapped to values in the system.” We did a thorough check to make sure the department is assigned to the user and the department’s creation and assignment date start from 01/01/1900, but it still didn’t work. We had to load the Requisitions by blanking out the department. This field is maintained manually in all the Requisitions as part of data remediation. We faced a similar issue for 2 to 3 Locations and Divisions also.

Though SuccessFactors documentation does not explicitly state, the fields for Hiring Manager (hiringManagerName), Recruiter (recruiterName) and Coordinator (coordinatorName) are mandatory. The import fails if these fields are not filled in.

If Job Profiles are used to maintain the Job Descriptions in the Requisition, all Families, Roles and Job Profiles must be first maintained in the Instance before the Requisitions are loaded. If they maintained prior to the load, the system automatically links the Requisitions with the Job Profiles. Otherwise, it is going to be a manual activity to connect the Job Profiles with Requisitions. The data in the jobRole field must be maintained in <Job Family Name>|<Job Role Name> format in the load file.

If EC is implemented and users are also loaded in the system at the same time of Recruitment load, Job Requisitions load must wait until all the users, their organization data are imported in the system and HRIS sync job is completed. The reason for HRIS Sync job to be completed prior to the load is because only then the Department, Division and Location of an Employee is copied to Employee Profile. Recruitment recognizes only those Departments, Divisions and Locations that are assigned to users and populated in Employee Profile. If a Department is created in the system but is not assigned to any user, then Recruitment does not recognize that Department. The same holds true for Division and Location as well.

Another point to take care of before loading Requisitions is to comment out all the Team Recruiting (operatorTeam) fields in the Requisition XML Template. Uncomment the fields after the load gets completed. The import fails if the teams are present in the template.

The job takes less than 20 mins to load 300 Requisitions. No duplicates will be created if the same job runs twice.

6. Applications

The standard Recruitment load supports only External Applications and not Internal Applications. But to the Business, migrating Internal Candidate Applications is as crucial as migrating External Candidate Applications. We came up with the below workaround.

a. Prepare and load a file to load all the Internal Candidates that have applications as External Candidates through the Candidate Profile load. Make sure the email id for Internal and External Candidate remains the same. This will help later in sending notifications to the candidate during the recruit process.

b. Load all the applications for Internal Candidates who are being loaded as Externals.

c. Use Manage Potential Duplicates tool to merge the Internal Candidate with External Candidate. While doing so, the Internal Candidate Profile replaces the External Candidate Profile and all the applications linked to External Profile will now move to Internal Candidate.

The biggest win in this approach is that the client is able to get all the Internal Candidate Applications migrated. Another good news is that the after merging of Candidates, the candidate’s application is identified as an Internal one.

One of the disadvantages is that the merging step is a manual activity. It can be carried out as a part of data remediation process after the load. The time taken to merge does not take more than 2 minutes for each candidate, which is better than not having the applications at all.

Another minor issue is sometimes the Internal Candidates could not be searched under Manage Potential Duplicates. In such a case the person who does merging activity might have to perform an additional step to search for the Candidate under Recruiting -> Candidates tab. Open the Candidate Profile and click on the ‘Save’ button.

I agree that this is a little clunky process but with this limitation we do not have another choice. Our load process happened prior to 1508 release. With 1508 release, we have ODATA API available for Applications. It is worth exploring this option to load the Internal Candidate Applications for future loads.

The load time for Application is quick as well. On an average the load time is less than 20 mins for 3000 applications

7. Application Attachments

We did not load any attachments for Applications. My thought is to use this option to load the CV as part of application instead of loading CVs against their profiles due to the problems mentioned above.

At the end of this journey, we had a very successful data migration. It is a huge team work by the Data Team, Business, Process Analyst, Functional Consultants and of course SuccessFactors.