Extract data from SuccessFators to FileFormat (.csv) using HCIds

[nageshcaparthy]

In my previous blog(1) and blog(2) we have understood how to Install DS Agent on-premises  and the basic understanding of HCI Web UI. In this blog we shall see the process of creating a DataStore for different systems.

I have taken two systems, 1 Successfactors as a source and 2. Flat File as a target. I shall explain how to setup a project and connect the systems and do data transfer.

Successfactors per-requisite:

1. Admin access to SuccessFactors
2. Enable oData API in SuccessFactors

Creating a Datastore for SuccessFactors:

First we need to modify the HCI DS Agent for SuccessFactors Adapter. Open ConfigureAgent.bat and go to Configure Adapter, select the following as below:

I have removed all the details in Additional Java Launcher Options, this is used for Proxy setup. In my case, I do not have any proxy system. Save the settings and exit it. HCI DS Should restart successfully.

We have now successfully setup the HCI DS Agent to Communicate with SuccessFactors. Now we need to setup the SuccessFactors Account details. Log-in to HCI Web UI and click on Datastores and click on the + icon to add a new connection.

Give a name to your Connection Name, Select the other details as shown in the above figure. The endpoint URL should be in this syntax : 

https://xxxxxx.successfactors.com/sfapi/v1/soap?wsdl

Where xxxxx is your company system details. Enter the user name in this format username, eg: admin and enter your password. Leave it to default values.

Click on Save and click on Test next to the + icon to test your connections.

You should be able to see the successful confirmation message if everything is good. This message indicates that we are now ready to read the data from SuccessFactors. Now click on Tables tab and then click on Import Objects button. This will populate all the objects in the form of tables as shown below. Select the objects that you need for mapping and click on import.

I am selecting 1 object as shown for the demo. Once imported, you should be able to see the table name and its properties as shown below:

Now you are ready to consume the data from SuccessFactors.

Creating a Datastore for Filesystem:

Since we need two systems to transfer the data from source to destination, I have picked the simplest option as file system as it would be easy for demo. Now go to your HCI Agent On-Prem System and create a folder. Open Data Services Agent Configuration by using ConfigureAgent.bat file.  Click on “Configure Directories” and add the folder which you just created as shown below. In my case I have created a file folder FlatFiles.

Now click on Exit button. The Agent will prompt you for a restart. Click on Yes. Ensure you the agent is restarted successfully.

Now, that you have created a file path in the agent, now you can create a datastore in the HCI Web UI. Go to Datastores, create a new datastore and set the following details as shown below:

Click on Save and Test your connection once you enter the details. You may also add a SFTP in case you have a different system. Since Flat files do not have any format or table structure, we need to identify the source table and create the table structure with similar column names and data types. In my case I have picked a table called User from SuccessFactors.

Now lets create the fields/columns for the table mapping for SuccessFactors. Select your FileSystem and then click on File Formats as below: Click on Create File Format and select Create from Scratch. Enter the Table Name and the delimiter and other details as shown below and click on Ok.

To add a new column click on the Add button in the Columns tab. Enter the Name and datatype and length of the column and click on submit.

Repeat the above step for other fields that you need. Now that we have created two Datastores SuccessFactors and FileSystem we are now good to create a project and map the data to transfer the data.

Creating a New Project.

Go to Project Tab and click on New Project.

Enter the project name and description which is optional, click on Create and select Save and Create Task. Not to worry in you have selected the other option. Go back to your Project screen, select the project that you have created and then click on Create Task.

Enter the name for the Task, remove the tick from Use Template and click on next.

In this screen we have to select the Source, considering that we have no data in the FileSystem so select SFAD and click on next.  

Select the target as FileSystem and click on Save and Define Data Flow as shown below. You may also test the connection while doing this.

You should be able to see the screen below. Click on Add Target Object. You will be able to see the Table Name which was created in the DataStores. Select it and click on Create Data Flow.

Enter the Data Flow details as shown below and click Ok:

You should be able to see the screen below.

Target Query and Larning Activity are the details about Target. We need to map it to a source file. Now click on Source Table and drag it to the middle of the screen and select the table for which we selected from SuccessFactors. 

You should be able to see User Table on the screen, just connect the pointers from User to Target_Query as shown below:

Now double click on Target Query, you should be able to see the input column details and output column details. You can drag and drop on each of the line items or if the names are same you can just click on “Automap by name”. Once the fields are mapped you should see the Blue Link just before the Column Name as show below:

Close the window once the mapping is done. Click on Validate to see if there are any errors in your mapping. You should be able to see a validation successful message. We are now good to run the Task which is created. Select the task from your project window and click on Run Now. In case you need the Debug Information, select it and click on Ok.

You will be able to see the status message, the blue arrow indicate the Task is Running. Once its successful, you should be able to see a Green Square which indicate the Job is Successful. Click on Refresh to see the status.

Please ignore the project. I had to recreate the project due to some internal technical issue. 

If we go back to our Folder path, we see the data transferred.

In case you find a Red Symbol, select the Task and click on View History to see the error report.

Hope this document helps to setup a connection and run the tasks. Looking for your feedback and queries.

Regards,

Nagesh