• Update April 15, 2019:
  Include a enhancement to the UDF to work with HTTP and HTTPS simultaneously;
  Include a enhancement to the UDF to work with URL encoded and NOT encoded (it’s necessary comment/uncomment the line).

Today I’ll explain step-by-step how to calculate the signature to authenticate and download a file from the Amazon S3 Bucket service without third-party adapters.

Step-by-step Interface Flow

Request

In summary this interface receive download URL, Bucket, AccessKeyID, SecretAccessKey, Token and AWSRegion, a mapping calculate the signature with this information and sent to REST Adapter, the signature and anothers parameters are insert in HTTP header.

Some information for calculate the signature are provide another service, this post explain only how to calculate, but is possible implemented enhancements, for example, create a rest/soap lookup to get a Token and SecretAccessKey.

Response

The response is a file, and the REST Adapter don’t work with format different of XML or JSON, then you will need convert the file to binary, and this content are insert in a tag of XML. For this conversion I recommend a module adapter FormatConversionBean developed by @engswee.yeoh

Development and Configuration the Interface

Request mapping

For the request mapping you need create a two structures, one for inbound and another for outbound.

Inbound

Outbound

After create the structures for the request mapping (data type, message type, etc), you need create a message mapping.

Now you need to map the fields, pay attention to the next steps for configurations the roles.

Roles for Message Mapping

•  Fields XAmzSecurityTokenand Url are mapped directly….

• Field XAmzSha256 is mapped with a constant value e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 (this string is a hash of a null value)

• Field XAmzDate is mapped with a CurrentDate (format yyyyMMdd’T’HHmmss’Z’) function…

• Field ContentType is mapped with a constant value application/x-www-form-urlencoded…

• Field Host is mapped with a UDF or ConstantValue.

The Host is a result of concatenation of the Bucket + “.s3.amazonaws.com“,
so you can use a ConstantValue (eu01-s3-store.s3.amazonaws.com for example), which receives the bucket and returns the Host

public String Host(String inBucket, Container container) throws StreamTransformationException{
String vServiceName = "s3";
String vS3Host = vServiceName + ".amazonaws.com";

String vHost = inBucket + "."+ vS3Host;
return vHost;
}

• Field Authorizathion is mapped with a UDF.

In field Authorization you have insert the signature calculated with the UDF below.

public String AWS4Signer(String inURL, String inDate, String inBucket, String inAccessKeyId, String inSecretAccessKey, String inToken, String inAWSRegion, Container container) throws StreamTransformationException{
String vSignedHeaders="content-type;host;x-amz-content-sha256;x-amz-date;x-amz-security-token"; 
String vHashedCannonicalRequest="";

String vScope= inDate.substring(0,8) + "/" + inAWSRegion + "/s3/aws4_request";
String vCredential="";
String vSignature="";
String vSignatureKey="";
String vServiceName="s3";
String vS3Host=vServiceName+".amazonaws.com";
String vHost=(inBucket+"."+vS3Host).toLowerCase();

// Only URL NOT encoded
String vCanonicalURI = urlEncode(inURL.replace("https://"+vHost,"").replace("http://"+vHost,"").trim(),true);

// Only URL encoded
//String vCanonicalURI = inURL.replace("https://"+vHost,"").replace("http://"+vHost,"").trim();

String vCanonicalQueryString = "";
String vHTTPRequestMethod = "GET";
String vContentSHA256 = "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855";
String vContentType = "application/x-www-form-urlencoded";


String cannonicalRequest="";
String stringToSign="";

String outAutorization="";

MappingTrace importanttrace;
importanttrace = container.getTrace();

// Canonical Request

importanttrace.addInfo("***** Canonical Request ***** ");
cannonicalRequest = 
	vHTTPRequestMethod.trim() +"\n"+
	vCanonicalURI.trim()+"\n"+
	vCanonicalQueryString.trim()+"\n"+ 
	"content-type:"+vContentType.trim()+"\n"+
	"host:" +vHost.trim()+"\n"+
	"x-amz-content-sha256:"+vContentSHA256.toLowerCase().trim()+"\n"+
	"x-amz-date:"+inDate.trim()+"\n"+
	"x-amz-security-token:"+inToken.trim()+"\n"+
	"\n"+
	vSignedHeaders.trim()+"\n"+
	vContentSHA256.toLowerCase().trim();

importanttrace.addInfo(cannonicalRequest);

//Canonical Request Hashed

importanttrace.addInfo("***** Hashed Canonical Request ***** ");
vHashedCannonicalRequest = hash(cannonicalRequest);

importanttrace.addInfo( vHashedCannonicalRequest);

//String to Sign
importanttrace.addInfo("***** String to Sign ***** ");
stringToSign =
	"AWS4-HMAC-SHA256"+"\n"+
	inDate.trim()+"\n"+
	vScope.trim()+"\n"+
	vHashedCannonicalRequest.trim();

importanttrace.addInfo(stringToSign);

//Signing

importanttrace.addInfo("***** Signing  ***** ");

try{
byte[] kSigning = getSignatureKey(inSecretAccessKey, inDate.substring(0,8), inAWSRegion, vServiceName);
byte[] signature = HmacSHA256(stringToSign.toString(), kSigning);

vSignatureKey =  bytesToHex (kSigning).toLowerCase(Locale.getDefault());
vSignature = bytesToHex (signature).toLowerCase(Locale.getDefault());

} catch (Exception e) {
		throw new RuntimeException("Unable to compute hash while signing request: " + e.getMessage(), e);
}

outAutorization = "AWS4-HMAC-SHA256 Credential=" + inAccessKeyId + "/" + vScope + ", SignedHeaders=" + vSignedHeaders + ", Signature=" + vSignature;

importanttrace.addInfo("Siging Key: " + vSignatureKey);
importanttrace.addInfo("Signature: " + vSignature);
importanttrace.addInfo("Authorization: " + outAutorization);

importanttrace.addInfo(outAutorization);

return outAutorization;

}

You also need to create some methods, which will be used by UDF in signing.

public static String hash(String text) {
	String hashedString = "";
	try {
		MessageDigest md = MessageDigest.getInstance("SHA-256");
		byte[] hashedBytes = md.digest(text.getBytes("UTF-8"));
 
		for (int i = 0; i < hashedBytes.length; i++){
			hashedString += Integer.toHexString((hashedBytes[i] >> 4) & 0xf);
			hashedString += Integer.toHexString(hashedBytes[i] & 0xf);
	}
	return hashedString;
	} catch (Exception e) {
		throw new RuntimeException("Unable to compute hash while signing request: " + e.getMessage(), e);
	}
}

private static String bytesToHex(byte[] hashInBytes) {
        StringBuilder sb = new StringBuilder();
        for (byte b : hashInBytes) {
            sb.append(String.format("%02x", b));
        }
        return sb.toString();
}

 public static byte[] HmacSHA256(String data, byte[] key) throws Exception  {
         String algorithm="HmacSHA256";
         Mac mac = Mac.getInstance(algorithm);
         mac.init(new SecretKeySpec(key, algorithm));
         return mac.doFinal(data.getBytes("UTF8"));
}

public static byte[] getSignatureKey(String key, String dateStamp, String regionName, String serviceName) throws Exception  {
         byte[] kSecret = ("AWS4" + key).getBytes("UTF8");
         byte[] kDate    = HmacSHA256(dateStamp, kSecret);
         byte[] kRegion  = HmacSHA256(regionName, kDate);
         byte[] kService = HmacSHA256(serviceName, kRegion);
         byte[] kSigning = HmacSHA256("aws4_request", kService);
         return kSigning;
}

    public static String urlEncode(String url, boolean keepPathSlash) {
        String encoded;
        try {
            encoded = URLEncoder.encode(url, "UTF-8");
        } catch (UnsupportedEncodingException e) {
            throw new RuntimeException("UTF-8 encoding is not supported.", e);
        }
        if ( keepPathSlash ) {
            encoded = encoded.replace("%2F", "/");
        }
        return encoded;
    }

and import the packages…

com.sap.aii.mapping.api.*
com.sap.aii.mapping.lookup.*
com.sap.aii.mappingtool.tf7.rt.*
java.io.*
java.lang.reflect.*
java.util.*
java.lang.String.*
java.lang.Object
java.util.Locale
java.security.MessageDigest
java.security.NoSuchAlgorithmException
java.net.URLEncoder
java.io.UnsupportedEncodingException
java.math.BigInteger
javax.crypto.Mac
javax.crypto.spec.SecretKeySpec

After developed the UDF, its necessary configure with the inbound values.

Note: the format of CurrentDate is yyyyMMdd’T’HHmmss’Z’.

Now save and Activate the Request mapping.

Response mapping

The response mapping it’s simple and not necessary many explanation.

Configure the interface normally …

After created Request/Response Mapping, build the Operation Mapping and a Integrated Configuration normally. The Communication Channel can be of any type that is synchronous, but the Receiver must be to type rest and configured as below.

Receiver Communication Channel

Now you need configure the Receiver Channel, for this the values generates in request message mapping are storage in variables, and this variables are used in the communication channel.

Now the variables storage are used in the HTTP Header, here you configure how the canonical request is create.

It’s necessary configure the REST Operation, for this case the operation is GET.

And finally configure the module adapter FormatConversionBean to converte the file in b64string.

IMPORTANTE: The module adapter FormatConversionBean isn’t standard, and you need deploy if you have not already, for more information and download of module you can access here.

Save and active all objects, now we going test!

Running a test

Fill in all the fields correctly in the interface and call the created service, the response should be the file in b64string format.

If you analyze the log of request messages, the parameters are populated in the HTTP header and communication has succeeded (HTTP 200)

and the response (the file) is converted to b64 string.

That’s all! I hope I have collaborated and I am waiting for your feedback on this post.

References

How to Calculate AWS Signature Version 4
https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-header-based-auth.html

Module Adapter FormatConversionBean
https://blogs.sap.com/2015/03/25/formatconversionbean-one-bean-to-rule-them-all/

PI REST Adapter – Define custom http header elements
https://blogs.sap.com/2015/04/14/pi-rest-adapter-define-custom-http-header-elements/