Ariba master data extraction program

[vinita_kasliwal]

Introduction:

This blog talks about how to use the master data extraction program to read data from ECC to Ariba system. I also wrote another blog on how the integration works please read the blog here:

https://blogs.sap.com/2017/08/13/ariba-and-sap-srm-and-ecc-integration/

 

This blog talks about below topic areas:

• Master Data extraction program


• Sequence for Initial load consideration


• Customization of the master program


• Incremental load consideration


• Lessons Learnt


• Test cases for Master data integration

Master Data extraction program:

The standard program /ARBA/MASTER_DATA_EXPORT needs to be customized or you will keep wasting your time to find the implicit enhancement points which are too less and  not at the right places. So I just did what was the easier thing to do copied the entire program as a Z and then did the changes.When you run the program you see radio buttons like Procure to Order and Procure to Pay, Sourcing and contracts.

What needs to be run depends on what modules you have implemented in Ariba and a lot of the contents available to download is common among sourcing and P2P and P2O radio buttons However it still needs to be run twice as it updates specific modules in Ariba. So based on what you select you get options to ‘Choose Master data to export’ you get different master data options,

To understand better you need to  first extract company code from P2O and then extract company code again from sourcing. So even though it’s the identical company code master data which gets exported but it needs to go to different modules in Ariba like sourcing and contract management so you need to run the extractors twice. Generally procurement scenario  in Ariba consist of

P2P: Contracts Orders and Invoicing

P2O: Contracts only

You can also choose data to be sent as an initial load one time and as an incremental load. So you would need to run :

1. Sourcing and contracts


2. Procure to Pay/ Procure to Order (One of these depending on modules implemented in Ariba)

There are also connectivity options like :

• Direct connectivity: This is used to send the data to the Ariba system directly


• Integration Toolkit: This loads the file in a shared drive and Integration layer (PI/PO) should pick this up to send to Ariba.

Sequence for Initial load consideration:

It is advisable to follow a specific logic to ensure the correct data and in the right sequence gets uploaded to the Ariba system. It is advisable to fill Sourcing and contracts before as its an Upstream module in Ariba and the others are downstream

	Data	FileName	#	Delta Load
Sourcing and contracts	Company Code	CompanyCode	1
	Plant	Plant	2
	Purchase Org	PurchaseOrg	3
	Purchasing Group	PurchaseGroup	4
	Payment Terms	PaymentTerms	5
	incoterms	IncoTerms	6
		IncoTermsDescriptionLang
	Item Category	ItemCategory	7
		ItemCategoryDescriptionLa
	ERP Commodity Code	ERP Commodity Code	8
	Plant Purchase Org	PlantPurchaseOrgCombo	9
	Material Master	Material	10	X
		Material Plant Combo
		Material Lang
Procure to Order / Procure to Pay	Company Code	CompanyCode	1
	Plant	Plant	2
	Purchase Org	PurchaseOrg	3
	Purchasing Group	PurchaseGroup	4
	Plant Purchase Org	PlantPurchaseOrgCombo	5
	User	UserConsolidated	6	X
		GroupConsolidated
	Supplier	PurchaseOrgSupplierCombo	7	X
		RemittanceLocationConsolidated
		SupplierConsolidated
		SupplierLocationConsolidated
	Cost Center	CostCenter	8	X
	Internal Order	InternalOrder	9	X
	General Ledger	GeneralLedger	10	X
	WBS Element	WBSElement	11	X
		Company Code WBS Combo
	Asset	Asset	12	X
	Account Category	AccountCategory	13
	Account Category Field status	AccCategoryFieldStatusCombo	14
	ERP Commodity Code		15
	Currency Conversion	CurrencyConversionRate	16
	Taxcode	TaxCode	17
	Payment Terms		18

 

Customization of the master program

There may be some customization needed in order to either clean / correct the data being sent to Ariba. I am specifying some of the scenarios below:

To Change column Name or add additional column and populate data:

It  may require you to change some column name to make it as a the primary key for example instead of the user Id which is the default primary key for the user extraction client wanted to make email ID as the primary key. So, I had to change the name of the column for email as UNIQUENAME (Ariba understand UniqueName column as the primary key) for the user extraction file and assign it as value of email.  In /ARBA/FIELD_MAP the field BNAME (User ID) currently is mapped to Ariba field UNIQUENAME. I now want the email ID field to populate for UniqueName.

Lets have a look at how this was done.

1. In Table /ARBA/FIELD_MAP in SM30 I added an extra SAP column UNIQUENAME1 in SAP. Then name of the column on CSV would be UniqueName This value UNIQUENAME1 was assigned the email ID.

Also I changed the label for BNAME to UserId as only 1 field can have value as UniqueName which becomes the key in Ariba .

2. Add an append structure in the standard structure as seen below:

 



 

3. To assign value to this column we need to change code to read email of the user:

And in order to add value to this field I used  BADI  /ARBA/MASTER_DATA. This badi has all the user extraction data methods as seen below

   

 

Method  /ARBA/IF_EXP_MASTER_DATA~MODIFY_USER_DATA is called for each time the user record is extracted so I enter  my field UNIQUENAME1 and pass  Email address. Remember this is a structure so it will be called for all entries one by one.



However most of the methods like /ARBA/IF_EXP_MASTER_DATA~PUBLISH_USER_DATA have exporting /changing parameter as table. So you can loop at each record and delete if the Email ID is blank etc as this is our primary key so keeping it blank will create issues in Ariba. 

Now when you have the file extracted as a CSV it will have an additional column as UniqueName with the email ID of the user  when the Ariba program would be run.



 

Incremental load consideration:

Note that not all data can be set up as incremental load. In P2O/P2P scenario the incremental load consists of  below master data.

 



 

In Sourcing and contracts below master data can be scheduled to run for incremental data:



If the data which you wish the send to Ariba is not included in the incremental list above like product category, Pur grp, company code, pur org etc. then you need to either update that manually or send an initial load everytime as needed.

 

How would the system know what changes occured since last time and how to pick the delta load?

he program handles the incremental load based on a table /ARBA/INCR_DTTIM. Everytime an incremental load is run the table gets updated with date and time of the specific load which was run. THe next time system would automatically filter the data based on the last updated fill from the table for users, suppliers or other master data for which you plan to run the incremental load. After the load run the system automatically update the Date information and the next time for incremental load it references this to understand what has been the delta change in the system since the last run.

:

To ensure the data is sent to the right destination /ARBA/auth_param table should be populated. Ensure you have maintained the necessary parameters in the table /ARBA/ AUTH_PARAM so the ECC system points to the correct Ariba Realm

 

Lessons Learnt:

• Ensure you get a confirmation of the mandatory fields in Ariba which may not be mandatory in ECC hence always check if the extracted file from ECC got successfully updated in Ariba with the same number of records


• Data fails in Integration like email IDs have special character this needs to be handled by custom logic by creating email aliases etc.


• If number of error records are more than say 100 or something the entire file fails to load in Ariba. The data send needs to be corrected before you send the file again as it stops reading the entire file after a threshold of N number of failures is reached.


• If the extracted data load is huge it may cause your SAP screen to freeze so better run as a batch job


• In ‘Connectivity’ section If you do a direct connectivity it upload the data to the connected Ariba system else if you choose ‘Integration Tool Kit’ it stores the file in a local system at a specific location which you specify and then needs to be manually uploaded to Ariba. So for testing I used ‘Integration toolkit’


• You can also have automation using Integration tool kit where PI/PO picks this up and sends to Ariba. If that is the case then you may want to send the file to some other location for testing


• Not all data is available for incremental load so if you choose to create new product categories you need to update this manually in Ariba. Check the delta load available to update as seen in the last column in my table above


• When you run the batch job check user’s authorization as the user would need to have some specific authorization which involves accessing a local directory, creating a folder and placing the file in it


• If the jobs are Run for incremental load ensure that the datetime stamp gets correctly updated by the system in the table /ARBA/INCR_DTTIM. This is how the system knows which records to pick up for delta extraction.


• When you put filters for the load like user  you may use below files.
  
  
  
  • /ARBA/IF_EXP_MASTER_DATA~PUBLISH_USER_DATA
  
  
  • /ARBA/IF_EXP_MASTER_DATA~MODIFY_USER_DATA
  
  
  • /ARBA/IF_EXP_MASTER_DATA~PUBLISH_USER_GROUP
  
  
  
  


• Supplier data extraction Methods are separate for both initial and delta load


• /ARBA/IF_EXP_MASTER_DATA~PUBLISH_VENDOR


• /ARBA/IF_EXP_MASTER_DATA~PUBLISH_SUPPLIER_INCREMENT

For incremental load for users SAP does not provide any method in the BADI to modify the incremental load files for deletion like userdelete and usergroupdelete files.

These files User Deleted File and UserGroup deleted file are  sent to Ariba to deactivate those users who have left the organisation. However no method in the BADI is triggered for formatting these files  Hence I came up with a workaround as below. Overwrite the extracted user deletion and group_deletion file with the required data.

Note that in the deletion file only one column is updated the primary key using which Ariba system would know which user to deactivate.Since I had changed the primary key to email this was not populated in the deletion files being sent to Ariba and system would not know which user to deactivate. So I did below

Rename the file using below code where gv_fname is the file which was generated without the filter. So I am Overwriting  the same extracted file with additional information that I need to pass to the Ariba system. In my master data program I modify the gr_user_del file to add email and also rename the file to the same name as was used by Ariba so my file replaces  the original file created by the program.

*** Modify the deleted file as needed:

IF sy-subrc EQ 0. "update entry with email

 ls_user_del-uniquename1 =   ls_smtp-e_mail.

 MODIFY   gt_user_del FROM ls_user_del INDEX lv_index.

ENDIF.



***** overwrite the file

CONCATENATE gv_fname 'GroupConsolidated_Delete.csv' INTO lv_file_name.



CALL FUNCTION '/ARBA/DATA_CONVERT_WRITE_FILE'

    EXPORTING

      i_filename        = lv_file_name

      i_fileformat      = 'CSV'

      i_field_seperator = ','

      i_tabname         = '/ARBA/USER'

      i_encoding        = 'UTF-8'

      i_solution        = 'AR'

    TABLES

      i_tab_sender      =  gt_user_del

    EXCEPTIONS

      open_failed       = 1

      close_failed      = 2

      write_failed      = 4

      conversion_failed = 5

      OTHERS            = 6.

  IF sy-subrc <> 0.                                         "#EC NEEDED

  ENDIF.

 

 

Test cases for Master data integration should include:

1. Maintain a checklist to ensure all master data is loaded in Ariba


2. In addition to each of the master data also compare the count for each example 200 users sent from SAP should all be loaded in Ariba


3. Create Documents like contract, PO to ensure the system data for the users is correctly updated and you are able to create the documents successfully with multiple users.


4. If there are failures in loading the data to Ariba verify that  file format being sent to Ariba is being read correctly.


5. Check the system is reading the incremental load correctly with a small set of data for suppliers and users.


6. If there are issues check incremental files have the right format and changes which you applied to the initial file load whether they are needed or not for the incremental push.


7. Ensure blocked suppliers are updated correctly in the Ariba system and no more showing up


8. Ensure the users who have left the org are removed from the Ariba system


9. Check integration of the system is correct from and to ECC and SRM as applicable.


10. For currency conversion the same rate applies as shown in the ECC system


11. For UoM ensure the Ariba value translated to the ECC ANSI conversion units


12. Ensure there is a process documented for all manual activities like updating delta load manually if not set as a batch job e.g. for new product categories, cost center and WBS.


13. If you use SSO in your organisation ensure it works fine for the users when they login


14. There are email notifications being sent from Ariba ensure that SSO is enabled for them and they are being sent to the right recipients.


15. Ensure the approval rules are working correctly for creation as well as changes

 

Thanks for reading. I hope this document is helpful for those looking to find some help on using the master data extraction program.