Google geocoding API for address check and normalization

SAP Hybris Cloud for Sales and Service has a feature to check postal address format, but has no OTB-feature to correct mistakes/misprints as well as the system has no feature for automatic determination of PO Box number based on city, street names and house number. Geo-coordinates of the address are also desirable to be determined. External geocoding service can help us. The following demo prototype shows how to use Google Geocoding API service in SAP Hybris Sales&Service .

I’m using Google Geocoding API (https://developers.google.com/maps/documentation/geocoding/intro?hl=en#language-param) but this example can be easily adapted for any REST-based geocoding service.

I also used SCN article «How to Parse a String in JSON Format in ABSL» (https://blogs.sap.com/2013/10/07/how-to-parse-a-string-in-json-format-in-absl/) by Thomas Schneider and priceless answer about URL encoding from Stefan Hagen in this topic https://answers.sap.com/questions/67032/url-encode.html.
Sincere thanks!

Step 1: Getting API-key.

First, you have to register as a developer in google and get API key. Here is a guide. It’s free of charge and you can send up to 2500 requests to Google daily. https://developers.google.com/maps/documentation/geocoding/get-api-key?hl=en

It is free of charge and you can send up to 2500 requests to Google daily (https://developers.google.com/maps/documentation/geocoding/usage-limits).

Step 2: Testing API via browser.

After you got the key and activated geocoding service it’s time to check the APi via your web-browser. My example is below, the address is written in Russian with misprints and mistakes. The API corrects errors and normalizes the form of address. Additionally it returns the result on any language you need (lang=EN parameter of the url).

Step 3: Service Configuration.

Create a solution and add Google geocoding service to it.

Choose REST Service

Add an URL https://maps.googleapis.com/maps/api/geocode/json

Choose «Create Communication Scenario» and define API key “key”

Click Next, then Finish and activate newly created service and communication scenario. In some cases you cannot add the API key name when you create he service – just finish the process, activate the service and add key name after that. Don’t forget to re-activate the service.

Then right click on the newly created scenario, choose «Manage Communication Arrangement» and create the new one based on arrangement «GOOGLE» we created before.

Define System Instance ID

Choose the Authentication method NONE and Enter the API Key you have got from Google.

Save and activate everything.

Step 4: Service Configuration.

As I told before, I am using some code examples from SCN with adaptation to my case. The big challenge here is that an Address BO is not available for extending. So, I used Customer BO and worked with default customer address in action Customer->After-Modify.

Create custom BO for JSON parser

import AP.Common.GDT as apCommonGDT;

businessobject JsonStructure_BO {

	element ID:ID;

	node Result [0,n] 
		{
			element ID : IntegerValue;
			element ParentID : IntegerValue;
			element NodeLevelID: IntegerValue;
			element Name : LANGUAGEINDEPENDENT_EXTENDED_Name;
			element Value : LANGUAGEINDEPENDENT_EXTENDED_Name;
		}
}

Add the code to Customer-AfterModify action.

The following code is the demo example, so, it must be optimized before using in production
Part 1: Running the service

import ABSL;
import AP.PlatinumEngineering;
import AP.Common.GDT;

// Communication details
var ScenarioName = "GOOGLE";
var ServiceName = "GOOGLE";
var HttpMethod = "GET";
var HttpResource = ""; //URL – File Name
// not required for this example
var ContentType = "";
var Body = "";
var HeaderParameter : collectionof NameAndValue; // Set URL Parameter


var URLParameter : collectionof NameAndValue;
var URLParameterEntry : NameAndValue;
URLParameterEntry.Name = "address";

if (this.CurrentDefaultAddressInformation.Address.DefaultPostalAddressRepresentation.IsSet()) // to check the adress is entered
{
	var originalCityName = this.CurrentDefaultAddressInformation.Address.DefaultPostalAddressRepresentation.CityName;
	var originalStreetName = this.CurrentDefaultAddressInformation.Address.DefaultPostalAddressRepresentation.StreetName;
	var originalHouseNumber = this.CurrentDefaultAddressInformation.Address.DefaultPostalAddressRepresentation.HouseID;
	URLParameterEntry.Value = URL.Encode(originalCityName.Concatenate(",".Concatenate(originalStreetName.Concatenate(",".Concatenate(originalHouseNumber))))); // address string to check via Google
}
else // nothing to normalize
{
	URLParameterEntry.Value = "";
}
URLParameter.Add(URLParameterEntry);

URLParameterEntry.Name = "language";
URLParameterEntry.Value = Context.GetCurrentUserLanguage().ToString();
URLParameter.Add(URLParameterEntry);

// Execute webservice call
var ws_result = WebServiceUtilities.ExecuteRESTService(ScenarioName, ServiceName, HttpMethod, HttpResource, URLParameter, HeaderParameter, ContentType, Body);

Part2: Parsing JSON answer.  I used SNC example (https://blogs.sap.com/2013/10/07/how-to-parse-a-string-in-json-format-in-absl/) with little modification

//JSON Parser;
var json = ws_result.Content;
var i = 0;
var i0 = 0;
var controlTag = ""; 
var controlTag2;
var lastControlTag;
var noOpenObjects = 0; // for checking check only
var noOpenArrays = 0; // for checking check only
var afterColon = false;
var string;
var result : elementsof JsonStructure_BO.Result;
var results : collectionof elementsof JsonStructure_BO.Result;
var ID = 0;
var parentID = 0;
var errorFlag = 0;
var blockId = 0;

if (ws_result.Code.Replace(" ","")=="200") // do only if return code is 200 ("OK")
{
while (i<json.Length())
{
 i = json.FindRegex("\\{|\\}|\\[|\\]|\"|:|,|\\d+|true|false", i);
 if (i < 0) {
  //raise Message.Create("E", "Parsing error (control tag not found)"); 
  break; // *** Error ***
 }
 lastControlTag = controlTag;
 controlTag = json.Substring(i,1);
 switch (controlTag) {
 case "{" , "}", "[", "]", ","  {
  i = i + 1;
  switch (controlTag) {
  case "{" {
   noOpenObjects = noOpenObjects + 1;
   parentID = ID;
   blockId = blockId+1;
  }
  case "}" {
   noOpenObjects = noOpenObjects - 1;
  // lookup for "grandparent ID" -> parentID
   if (!(parentID == 0)) {
    var partentIDtab = results.Where(n=>n.ID == parentID);
    if (partentIDtab.Count() == 1) {
     parentID = partentIDtab.GetFirst().ParentID;
    }
    else {
     //raise Message.Create("E", "Parsing error (parentID not found)"); 
	 errorFlag = 1;
	 break; // *** Error ***
    }
   }
  }
  case "[" {
   noOpenArrays = noOpenArrays + 1;
   parentID = ID;
  }
  case "]" {
   noOpenArrays = noOpenArrays - 1;
  // lookup for "grandparent ID" -> parentID
   if (!(parentID == 0)) {
    var partentIDtab = results.Where(n=>n.ID == parentID);
    if (partentIDtab.Count() == 1) {
     parentID = partentIDtab.GetFirst().ParentID;
    }
    else {
     //raise Message.Create("E", "Parsing error (parentID not found for array)"); 
	 errorFlag = 1; break; // *** Error ***
    }
   }
  }
  } 
  // close old node and open new one with exception list
  if ( (controlTag == "," && (lastControlTag == "}" || lastControlTag == "]")) ) {
   // no new node required
  }
  else {
   if( ID > 0 ) {results.Add(result);}
   ID = ID + 1;
   result.ID = ID;
   result.ParentID = parentID;
   result.Name = "";
   result.Value = "";
   result.NodeLevelID =  blockId;
  }
  afterColon = false;
 }  // end case "{" , ",", "}"
 case ":" {
  if (afterColon) {
   //raise Message.Create("E", "Parsing error (open colon)"); 
   errorFlag = 1; break; // *** Error ***
  }
  afterColon = true;
  i = i + 1;
 }
 case "\"" {
  i = i + 1;
  i0 = i;
  i = json.FindRegex("\\{|\\}|\\[|\\]|\"", i);
  if (i < 0) {
   //raise Message.Create("E", "Parsing error (control tag not found after quote)"); 
   errorFlag = 1; break; // *** Error ***
  }
  controlTag2 = json.Substring(i,1);
  if (controlTag2 == "\"") {
   string = json.Substring(i0, i-i0);
   if (!afterColon) {   // before colon -> name
    result.Name = string;
   }
   else {// after colon -> value
    result.Value = string;
    afterColon = false;
   }
  }
  else {
   //raise Message.Create("E", "Parsing error (quote not closed)"); 
   errorFlag = 1; break; // *** Error ***
  }
  i = i + 1;
 }
 case "t", "T" {
  if (!afterColon) {
   //raise Message.Create("E", "Parsing error (true/false)"); 
   errorFlag = 1; break; // *** Error ***
  }
  result.Value = "true";
  afterColon = false;
  i = i + 4;
 }
 case "f", "F" {
  if (!afterColon) {
   //raise Message.Create("E", "Parsing error (true/false)"); 
   errorFlag = 1; break; // *** Error ***
  }
  result.Value = "false";
  afterColon = false;
  i = i + 5;
 }
 default {
  // implementation for numeric (\d in regex)
  if (!afterColon) {
   //raise Message.Create("E", "Parsing error (numeric before colon)"); 
   errorFlag = 1; break; // *** Error ***
  }
  i = i + 1;
  i0 = i - 1;
  i = json.FindRegex("\\{|\\}|\\[|\\]|\"|:|,|true|false", i);
  if (i < 0) {
   //raise Message.Create("E", "Parsing error (numeric: end not found)"); 
   errorFlag = 1; break; // *** Error ***
  }
  controlTag2 = json.Substring(i,1);
  if (controlTag2 == "}" || controlTag2 == ",") {
   string = json.Substring(i0, i-i0);
   result.Value = string;
   afterColon = false;
  }
  else {
   //raise Message.Create("E", "Parsing error (numeric end not correct)"); 
   errorFlag = 1; break; // *** Error ***
  }
 }
 }

 // check
 if (noOpenObjects < 0 || noOpenArrays < 0) 
 {
  //raise Message.Create("E", "Parsing error (error during object/array processing) (1)");  // *** Error ***
  errorFlag = 1;
 }

} //while end


// final check
if (!(noOpenObjects == 0) && !(noOpenArrays == 0) && !(parentID == 0)) 
{
 //raise Message.Create("E", "Parsing error (error during final check"); // *** Error ***
 errorFlag = 1;
}
// End of JSON Parser

Part 3: Address data normalization. (for production purposes you have to add exception processing). Note, that different countries have different address formats and GOOGLE API returns the address a little different depending on the country. C4C has universal data storage format, so modify your code for country specific form. The code below is relevant for Russia and CIS countries.

var resultTab: collectionof elementsof JsonStructure_BO.Result;
var postalCode = "";
var streetName = "";
var townName = "";
var countryCode = "";
var houseNumber = "";
var lat = 0;
var long = 0;
var coordinates: GeoCoordinates;

if (this.CurrentDefaultAddressInformation.Address.DefaultPostalAddressRepresentation.IsSet()) // to check that an address BO exists
{
foreach (var r in results)
{
	switch (r.Name)
	{
		case "postal_code"
		{
			resultTab = results.Where(n=>n.NodeLevelID == r.NodeLevelID && n.Name == "short_name");
			if (!resultTab.GetFirst().IsInitial()) 
				{
					postalCode = resultTab.GetFirst().Value;
					if (postalCode.Length()>0)
					{
						this.CurrentDefaultAddressInformation.Address.DefaultPostalAddressRepresentation.StreetPostalCode = postalCode;
							
					}
				}
		}
		case "route" // street name
		{
			resultTab = results.Where(n=>n.NodeLevelID == r.NodeLevelID && n.Name == "short_name");
			if (!resultTab.GetFirst().IsInitial()) 
				{
					streetName = resultTab.GetFirst().Value;
					if (streetName.Length()>0)
					{
						this.CurrentDefaultAddressInformation.Address.DefaultPostalAddressRepresentation.StreetName = streetName;
							
					}
				}
		}
		case "administrative_area_level_2" // town name 
		{
			resultTab = results.Where(n=>n.NodeLevelID == r.NodeLevelID && n.Name == "short_name");
			if (!resultTab.GetFirst().IsInitial()) 
				{
					townName = resultTab.GetFirst().Value;
					if (townName.Length()>0)
					{
						this.CurrentDefaultAddressInformation.Address.DefaultPostalAddressRepresentation.CityName = townName;
							
					}
				}
		}
		case "country" // country code
		{
			resultTab = results.Where(n=>n.NodeLevelID == r.NodeLevelID && n.Name == "short_name");
			if (!resultTab.GetFirst().IsInitial()) 
				{
					countryCode = resultTab.GetFirst().Value;
					if (countryCode.Length()>0)
					{
						this.CurrentDefaultAddressInformation.Address.DefaultPostalAddressRepresentation.CountryCode = countryCode;
							
					}
				}
		}
		case "street_number" //house number from Google...
		{
			resultTab = results.Where(n=>n.NodeLevelID == r.NodeLevelID && n.Name == "short_name");
			if (!resultTab.GetFirst().IsInitial()) 
				{
					houseNumber = resultTab.GetFirst().Value;
					if (houseNumber.Length()>0)
					{
						this.CurrentDefaultAddressInformation.Address.DefaultPostalAddressRepresentation.HouseID = houseNumber;
					}
				}
			
		}
		case "location" // geolocation coordinates
		{
			resultTab = results.Where(n=>n.ParentID == r.ID);
			if (resultTab.Count()>0)
			{
				foreach (var rr in resultTab)
				{
					switch (rr.Name)
					{ 
						case "lat"
						{
						coordinates.LatitudeMeasure.content = Numeric.ParseFromString(rr.Value.Replace("\n","").Trim());
						//this.CurrentDefaultAddressInformation.Address.GeographicalLocation.GeoCoordinates.LatitudeMeasure.content = Numeric.ParseFromString(rr.Value);
						}
						case "lng"
						{
						//Trace.Info("A STR=",rr.Value);
						//Trace.Info("B STR=",rr.Value.Replace("\n","").Trim());
						coordinates.LongitudeMeasure.content = Numeric.ParseFromString(rr.Value.Replace("\n","").Trim());
						//this.CurrentDefaultAddressInformation.Address.GeographicalLocation.GeoCoordinates.LongitudeMeasure.content = Numeric.ParseFromString(rr.Value);
						}
					}					
				}
				if (this.CurrentDefaultAddressInformation.Address.GeographicalLocation.IsSet())
				{
					this.CurrentDefaultAddressInformation.Address.GeographicalLocation.GeoCoordinates.LongitudeMeasure = coordinates.LongitudeMeasure;
					this.CurrentDefaultAddressInformation.Address.GeographicalLocation.GeoCoordinates.LatitudeMeasure = coordinates.LatitudeMeasure;
				}
				else
				{
					this.CurrentDefaultAddressInformation.Address.GeographicalLocation.Create();
					this.CurrentDefaultAddressInformation.Address.GeographicalLocation.GeoCoordinates.LongitudeMeasure = coordinates.LongitudeMeasure;
					this.CurrentDefaultAddressInformation.Address.GeographicalLocation.GeoCoordinates.LatitudeMeasure = coordinates.LatitudeMeasure;
				}
				
			}
		}
	} // end switch

}//end foraech

}// end if

} // end "global" if

Step 5: Testing.

Here is the Customer address before normalization

The same object after – the address is in normal form (to be written by Russian speaking postal worker :)), PO box number is automatically added and geo coordinates are also determined.

I hope you have found some interesting ideas here :). Will appreciate any comments and new ideas!

Regards,
Alexey