November 30, 2017 5 minute read

SharePoint REST Batch Update Request using Java

Overview:

Here we will see

A business requirement, where employee’s pending item counts need to be updated in SharePoint.
Here we focus on ‘Batch Update‘ of items in SharePoint at a time.
In this blog, we will come to know,
- how we can perform batch update in SharePoint using a Java Program,
- same concept we can use in SAP-PI Java Map
Java Code functionalities will be as follows:
- Get all pending items from One-Source system’s REST service
- Update received items to SharePoint using one call i.e. Batch-Request
To perform Batch-Insert in SharePoint, following blog can be referred:
SharePoint Bulk INSERT using Java Program

Java-Code-Logic:

Get all employee’s pending count from “REST-Service-01” of one Source-System data provider
Next we need a list of all available employee’s in SharePoint against which we updated new pendingCount
For same, we call a SharePoint-REST say “REST-Service-02“, which returns all ’employeeId’ with respective ‘sharePointId’
‘sharePointId’ is unique number for each entry in SharePoint
Now create a new Json-input which will have elements ’employeeId’, ‘pendingCount’ from “REST-Service-01” result and matching employeeID’s ‘SharePointId’ from “REST-Service-02” result-string
Post merge of both results, new JSON input will be ready as follows: (here for example purpose, we try for 3 items at a time)

[
  {
    "employeeId": 213323,
    "pendingCount": 991,
    "sharePointId": 3435
  },
  {
    "employeeId": 231499,
    "pendingCount": 992,
    "sharePointId": 3436
  },
  {
    "employeeId": 205695,
    "pendingCount": 993,
    "sharePointId": 3437
  }
]

From above Json-Input string, we need to extract

“employeeId”,
“pendingCount”
“sharePointId”

which to be updated into SharePoint.

Next, extracted contents need to be framed in Batch-Update-Request-Format. This is been achieved using Java function SharePoint_BULK_UPDATE()

SharePoint Batch REST url is as follows:

https://ClientDomain.sharepoint.com/teams/SPdev/AlertsCount/_api/$batch

SharePoint-REST requires following request format for Batch-Update

--batch_47474c30-d001-4af7-d3be-66f16aae63c0
Content-Type: multipart/mixed; boundary="changeset_46a33441-e549-426b-b73d-bd1a67f07123"
Content-Length: 1403
Content-Transfer-Encoding: binary

--changeset_46a33441-e549-426b-b73d-bd1a67f07123
Content-Type: application/http
Content-Transfer-Encoding: binary

PATCH https://ClientDomain.sharepoint.com/teams/SPdev/AlertsCount/_api/web/lists/getbytitle('AlertCount')/items(3435) HTTP/1.1
Content-Type: application/json;odata=verbose
Accept: application/json;odata=verbose
If-Match: *

{"__metadata":{"type":"SP.Data.AlertCountListItem"},"Title":"213323","pendingCount":"991"}

--changeset_46a33441-e549-426b-b73d-bd1a67f07123
Content-Type: application/http
Content-Transfer-Encoding: binary

PATCH https://ClientDomain.sharepoint.com/teams/SPdev/AlertsCount/_api/web/lists/getbytitle('AlertCount')/items(3436) HTTP/1.1
Content-Type: application/json;odata=verbose
Accept: application/json;odata=verbose
If-Match: *

{"__metadata":{"type":"SP.Data.AlertCountListItem"},"Title":"231499","pendingCount":"992"}

--changeset_46a33441-e549-426b-b73d-bd1a67f07123
Content-Type: application/http
Content-Transfer-Encoding: binary

PATCH https://ClientDomain.sharepoint.com/teams/SPdev/AlertsCount/_api/web/lists/getbytitle('AlertCount')/items(3437) HTTP/1.1
Content-Type: application/json;odata=verbose
Accept: application/json;odata=verbose
If-Match: *

{"__metadata":{"type":"SP.Data.AlertCountListItem"},"Title":"205695","pendingCount":"993"}

--changeset_46a33441-e549-426b-b73d-bd1a67f07123--

--batch_47474c30-d001-4af7-d3be-66f16aae63c0
Content-Type: application/http
Content-Transfer-Encoding: binary

GET https://ClientDomain.sharepoint.com/teams/SPdev/AlertsCount/_api/web/Lists/GetByTitle('AlertCount')/Items?$orderby=Title HTTP/1.1
Accept: application/json;odata=verbose

--batch_47474c30-d001-4af7-d3be-66f16aae63c0--

Here, we require to generate unique-ID (GUID-like strings) for HTTP batch requests.

For example: following unique IDs (GUID) is been used for ChangeSetItems and BatchRequestItem:

47474c30-d001-4af7-d3be-66f16aae63c0 used for batch
46a33441-e549-426b-b73d-bd1a67f07123 used for each item

Java function “generateUUID()” generates universally unique identifiers.

	private static String generateUUID(){   	    
		//Generates a GUID-like string, used in HTTP batches
		//Generating unique IDs | The identifiers generated by UUID are actually universally unique identifiers.			
	    UUID idOne = UUID.randomUUID();
	    String idOne_Str =  String.valueOf(idOne);
	    return idOne_Str;
	}

Java function SharePoint_BULK_UPDATE() prepares Batch-Update request format as per above Json-input

	private static void SharePoint_BULK_UPDATE(String jsonCountStr) throws IOException{
		
		/*
		Following sample JSON format required for SharePoint Update function 
		[
  			{
    			"employeeId": 213323,
    			"pendingCount": 991,
    			"sharePointId": 3435
    			},
    			{
    			"employeeId": 231499,
    			"pendingCount": 992,
    			"sharePointId": 3436
    			},
    			{
    			"employeeId": 205695,
    			"pendingCount": 993,
    			"sharePointId": 3437
    			}
		]	
  		
  		SharePoint REST-URL to Update count:
  			https://ClientDomain.sharepoint.com/teams/SPdev/AlertsCount/_api/web/Lists/GetByTitle('AlertCount')/Items(3435)	
		*/
		
		//generate uniqueId for a batch boundary
		String batchGuid = generateUUID();
		
		//generate uniqueId for each item to be inserted
		String changeSetId = generateUUID();
				
		//Begin of: Prepare Bulk Request Format for SharePoint Bulk-Insert-Query ----------------		
		String batchContents = "";
		try {				
			//Parse the output-count JSON			
			JSONParser parser = new JSONParser(); 						//Jar file: json-simple-1.1.1.jar
			JSONArray jsonArr = (JSONArray)parser.parse(jsonCountStr); 	//Parse JSON-input to JSONArray
									
			//START: changeset to update data ----------
			String batchCnt_Update = "";
			for(int i=0; i<jsonArr.size(); i++){
				
				//Read JSON string values
				JSONObject jsonObj =  new JSONObject();
				jsonObj = (JSONObject) jsonArr.get(i);						//get ith array object				
				String empId  = (jsonObj.get("employeeId")).toString(); 	//get value of 'employeeId'
				String pCount = (jsonObj.get("pendingCount")).toString(); 	//get value of 'pendingCount'	
				String shpId  = (jsonObj.get("sharePointId")).toString(); 	//get value of 'sharePointId'
				
				//Start:create INSERT-Statement for one Item ...................				
				String request = "{\"__metadata\":{\"type\":\"SP.Data.AlertCountListItem\"}," +
								"\"Title\":\"" 			+ empId 	+ "\"," +
								"\"pendingCount\":\"" 	+ pCount 	+ "\"}";
			
				//SharePoint URL to insert one item		
				String endpoint_Update = "https://ClientDomain.sharepoint.com/teams/SPdev/AlertsCount/" +
						"_api/web/Lists/GetByTitle('AlertCount')/Items(" + shpId + ")";		
								
				batchCnt_Update = batchCnt_Update 
						+ "--changeset_" + changeSetId + "\n" 
						+ "Content-Type: application/http" + "\n" 
						+ "Content-Transfer-Encoding: binary" + "\n"
						+ "" + "\n"
						+ "PATCH " + endpoint_Update + " HTTP/1.1" + "\n"
						+ "Content-Type: application/json;odata=verbose" + "\n"
						+ "Accept: application/json;odata=verbose" + "\n"
						+ "IF-MATCH: *" + "\n"
						+ "" + "\n"
						+ request + "\n"
						+ "" + "\n";					
		        //END:create INSERT-Statement for one Item .........................
				/*
				//Break for-loop (for testing trying only 3 items)
	 			if(i == 302){
	 				break;		
	 			}
	 			*/
			}			
			//END:   changeset to update data ----------
			batchCnt_Update = batchCnt_Update + "--changeset_" + changeSetId + "--\n";

	        //create batch for creating items	        
	        batchContents = "--batch_" + batchGuid + "\n"
	        			+ "Content-Type: multipart/mixed; boundary=\"changeset_" + changeSetId + "\"\n"
	        			+ "Content-Length: " + batchCnt_Update.length() + "\n"
	        			+ "Content-Transfer-Encoding: binary" + "\n"
	        			+ "" + "\n"
	        			+ batchCnt_Update + "\n";
	        
	        //Start:create request in batch to get all items after update ---------
	        String endpoint = "https://ClientDomain.sharepoint.com/teams/SPdev/AlertsCount/_api/web/Lists/GetByTitle(\'AlertCount\')/Items?$orderby=Title";
	        batchContents = batchContents 
	        		+ "--batch_" + batchGuid + "\n"
	        		+ "Content-Type: application/http" + "\n"
	        		+ "Content-Transfer-Encoding: binary" + "\n"
	        		+ "" + "\n"
	        		+ "GET " + endpoint + " HTTP/1.1" + "\n"
	        		+ "Accept: application/json;odata=verbose" + "\n"
	        		+ "" + "\n";
	        //End:create request in batch to get all items after update -----------
	        
	        batchContents = batchContents + "--batch_" + batchGuid + "--";
	        	        
		} catch (Exception e) {
			System.out.println(e.getMessage());
		}		
		//End   of: Prepare Bulk Request Format for SharePoint Bulk-Insert-Query ----------------
		
		//Call SharePoint-REST to POST Items
		System.out.println(batchContents);
		SharePoint_BULK_POST(batchContents, batchGuid);
		
	}

Following Java functions SharePoint_BULK_POST() posts Batch request into SharePoint

	private static void SharePoint_BULK_POST(String batchRequest, String batchGuid) throws IOException{

		//Get SharePoint Access Token
		String accessToken = SharePoint_getAccessToken();
			
		//POST BulkRequest to SharePoint
		try {			
			String wsUrl = "https://ClientDomain.sharepoint.com/teams/SPdev/AlertsCount/_api/$batch";
		
			//Create HttpURLConnection
			URL url = new URL(wsUrl);
			URLConnection connection = url.openConnection();
			HttpURLConnection httpConn = (HttpURLConnection) connection;
				
			//Set Header
			httpConn.setDoOutput(true);
			httpConn.setDoInput(true);	
			httpConn.setRequestMethod("POST");			
			httpConn.setRequestProperty("Authorization", "Bearer " + accessToken);	
			httpConn.setRequestProperty("Content-Type", "multipart/mixed; boundary=\"batch_"+ batchGuid +"\"");
			
			//Send Request
			DataOutputStream wr = new DataOutputStream(httpConn.getOutputStream ());	
			wr.writeBytes(batchRequest);
			wr.flush();
			wr.close();
	
			//Read the response.
			String responseStr = "";
			if (httpConn.getResponseCode() == 200) {
				responseStr = "HTTP Response Code: " + httpConn.getResponseCode() + ", Batch records gets inserted successfully in SharePoint.";		
			}else{
				responseStr = "HTTP Response Code: " + httpConn.getResponseCode() +", Error while inserting Item. "+ httpConn.getResponseMessage();
			}
			System.out.println(responseStr);
		} catch (Exception e) {			
			System.out.println(e.getMessage());
		}			
	}

Following function helps to get SharePoint access token

	private static String SharePoint_getAccessToken(){
		/*
		This function helps to get SharePoint Access Token.
		SharePoint Access Token is required to authenticate SharePoint REST service while GET/POST
		
		SharePoint url to get access token is as:
		https://accounts.accesscontrol.windows.net/<tenantID>/tokens/OAuth/2
		
		This function requires below input related to SharePoint:
		1. client_id
		2. client_secret
		3. tenant_ID
		4. client_domain				
		*/
		
		String accessToken = "";
		try {	
			System.out.println("Get SharePointAccessToken");
			String wsURL = "https://accounts.accesscontrol.windows.net/"+ shp_tenantId +"/tokens/OAuth/2";
					
			//Create HttpConenction
			URL url = new URL(wsURL);
			URLConnection connection = url.openConnection(); 						
			HttpURLConnection httpConn = (HttpURLConnection) connection; 		
		
			//Set header
			httpConn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");		
			httpConn.setDoOutput(true);
			httpConn.setDoInput(true);	
			httpConn.setRequestMethod("POST");
		
			//Prepare RequestData
			String jsonParam =
							"grant_type=client_credentials" +				
							"&client_id="+shp_clientId+"@"+shp_tenantId +
							"&client_secret=" + shp_clientSecret +
							"&resource=00000003-0000-0ff1-ce00-000000000000/"+ shp_clientDomain + ".sharepoint.com@" + shp_tenantId;
		
			//Send Request
			DataOutputStream wr = new DataOutputStream(httpConn.getOutputStream ());	
			wr.writeBytes(jsonParam);
			wr.flush();
			wr.close();
		
			//Read the response
			String httpResponseStr = "";
			InputStreamReader isr = null;
			if (httpConn.getResponseCode() == 200) {
				isr = new InputStreamReader(httpConn.getInputStream());
			} else {
				isr = new InputStreamReader(httpConn.getErrorStream());
			}		
			BufferedReader in = new BufferedReader(isr); 	 		
			String strLine = "";
			while ((strLine = in.readLine()) != null) {
				httpResponseStr = httpResponseStr + strLine;
			}	
		
			//Extracting accessToken from httpResponseStr which is a JSON format string
			/*		
			Sample HTTP Response String(httpResponseStr) is as below:
			{"token_type":"Bearer","expires_in":"3599","not_before":"1509537628","expires_on":"1509541528","resource":"","access_token":"xyz"}
			 */			
			JSONParser parser = new JSONParser(); 						//Jar file: json-simple-1.1.1.jar
			JSONObject jsonObj =  (JSONObject)parser.parse(httpResponseStr);		
			accessToken = (jsonObj.get("access_token")).toString(); 	//get value of element	
			
		} catch (Exception e) {			
			System.out.println(e.getMessage());
		}			
		return accessToken;		
	}

Post update, following SharePoint Screen, where we can see our items

<<<<< Parent blog reference….. Integrate SharePoint using SAP PI

5 Comments

You must be Logged on to comment or reply to a post.

Antony FERMINUS

March 1, 2018 at 1:26 pm

Hi Dilip Kumar,

Thanks for the great article.

I am trying to follow the steps that you mentioned.

i get the result HTTP Response Code: 401, Error while inserting Item. Unauthorized.

Can you give some suggestions. where or how the Authorization should be adapted?

Thanks in advance

the result i got is below

batchcontents --JobsListItem_82b61736-2ae9-4b68-bc59-4ab55554380a
Content-Type: multipart/mixed; boundary="changeset_77d3c776-1bb1-4b6d-b165-9cf4366a15dc"
Content-Length: 515
Content-Transfer-Encoding: binary

--changeset_77d3c776-1bb1-4b6d-b165-9cf4366a15dc
Content-Type: application/http
Content-Transfer-Encoding: binary

PATCH https://domain/sites/TestHRJob/_api/web/lists/getbytitle('Internal%20Jobs')/Items(1) HTTP/1.1
Content-Type: application/json;odata=verbose
Accept: application/json;odata=verbose
IF-MATCH: *

{"__metadata":{"type":"SP.Data.Internal_x0020_JobsListItem"},"Title":"Title POTest2","POSTNAME":"POSTNAME Po-test","JDEPARTMENT":"HR"}

--changeset_77d3c776-1bb1-4b6d-b165-9cf4366a15dc--

--batch_82b61736-2ae9-4b68-bc59-4ab55554380a
Content-Type: application/http
Content-Transfer-Encoding: binary

GET https://domain/sites/TestHRJob/_api/web/Lists/GetByTitle('Internal%20Jobs')/Items?$orderby=Title HTTP/1.1
Accept: application/json;odata=verbose

--JobsListItem_82b61736-2ae9-4b68-bc59-4ab55554380a--
HTTP Response Code: 401, Error while inserting Item. Unauthorized

Regards,

Antony

Antony FERMINUS

March 8, 2018 at 11:57 am

Solved.

Correct permissions were given by Sharepoint.

Dilip Kumar Krishnadev Pandey

Blog Post Author

March 22, 2018 at 12:24 pm

Dear

Sorry for delayed response, I was too much busy in my current project....

Nice to here your problem gets sorted out.

Same issue I was also getting initially, then we figured out that our SharePoint <Client-ID> should have write permission with below syntax:

</AppPermissionRequests>

Thanks & regards

Dilip

Dilip Kumar Krishnadev Pandey

Blog Post Author

March 22, 2018 at 12:34 pm

Dear Anotny,

We can refer below link too for Write permission issue:

https://docs.microsoft.com/en-us/sharepoint/dev/solution-guidance/security-apponly-azureacs

Thanks & regards

Dilip...

Antony FERMINUS

May 4, 2018 at 9:10 am

Hi Dilip,

Thanks for the tips.

It will help more others.

Regards,

Antony

SharePoint REST Batch Update Request using Java

Overview:

Java-Code-Logic:

Assigned Tags