Cloud Integration: Manually Sign / Verify XML payload based on XML Signature Standard

[CarlosRoggan]

SAP Cloud Integration offers iFlow steps for signing and verifying XML content according to the "XML Signature" standard. Nevertheless, there are use cases that require configurations that are not supported by the iFlow steps. Fortunately, in such cases we still can use Groovy scripts for manually sign or verify message payloads according to the requirements.
This blog post provides an entry point for creating Groovy scripts for signature creation and verification, based on the “XML Signature” standard.
This standard provides some benefits and flexibility specifically for xml content and was explained in my previous blog post. 
This blog post covers
🔸 SAP Cloud Integration on Cloud Foundry
🔸 Groovy / Java
🔸 org.apache.santuario library

Content

Prerequisites
Intro
1. Create Key Pair
2. Create iFlow
    2.1.  Create iFlow
    2.2. Upload library
    2.3. Groovy Script for Signing
    2.4. Groovy Script for Verification
3. Run
Appendix 1: Sample XML Payload
Appendix 2: Sample Groovy Script for Signing
Appendix 3: Sample Groovy Script for Verification
Appendix 4: Maven pom file

Prerequisites

🔹CPI
To follow this tutorial, access to a Cloud Integration tenant is required, as well as basic knowledge about creating iFlows.
🔹Maven
While not required, it is an advantage to have maven installed.
🔹Understanding the "XML Signature" standard  is not difficult when reading the previous blog post.
🔹For remaining open questions I recommend the Security Glossary.

Introduction

After going through the previous blog post, we’re well prepared for the hands-on tutorial in this post.

Scenario
In our Cloud Integration scenario, a sales order XML payload must be signed because it contains sensitive information (order data).
To make things easier, we hard-code the sample xml payload in the iFlow.
In a Groovy script, we sign only the subnode which contains the important info.
This is possible thanks to the “XML Signature” standard.
To prove that the new XML payload, which contains the signature part, can be verified, we add another Groovy script.
This one performs the signature verification.

Recap
The “XML Signature” standard allows to sign:
🔹The whole message (enveloping)
🔹A specific node, means the node itself plus its content (enveloped)

In our tutorial, we’re going for the second option.
The standard defines an additional XML tree that is inserted in the original document, next to the sensitive content.
This new XML tree has the root node <Signature>.

The sample payload
We’re keeping things as simple as possible, so we’re using this simple sample payload:

We want to sign the <order> node.
This ensures that neither the receiving customer nor the bought product are altered during message processing.
To be more precise:
The data might be altered, as it is not encrypted. But in that case, it would be detected, because the verification would fail.
Why we don’t encrypt it?
Because encryption is usually applied to secret content. In our example the content is not secret, as it doesn’t contain e.g. a credit card number.

Below screenshot shows the payload before and after signing:

The signing process
In the Groovy script, we’re using the Apache Santuario library, which provides an implementation of the “XML Signature” standard.
This means it is aware of the XML structure and the signing process that is defined in the standard.
It is also aware of the algorithms that are supported in the specification.

So what we have to  do in the code:
🔸Compose the additional xml tree, using the objects which are offered in the library.
🔸Provide the required  private key (public key optional).
🔸Choose the XML node that should be signed.
🔸Feed the signer with that node.

Note:
It is clearly recommended to use xPath for retrieving the xml node that should be verified.
The reason is that xPath helps to validate the xml structure of the whole document, as well.
However, to keep the sample code as simple as possible, I decided not to use xPath.
Please make sure to always use xPath in your productive code.

Disclaimer:
This blog post is not an official recommendation, this is not safe and not ready to be pasted into productive environment.
This is just a simplified tutorial to get everybody started.

1. Create Key Pair

There are multiple possibilities for creating a key pair. 
A simple way would be to use CPI, it's fine for following this tutorial.

Using the built-in functionality of CPI to generate a key pair has the following advantage:
The private key is generated at CPI, where it is needed.
The private key never leaves CPI (it is not possible to download it)
The private key does not need to be uploaded to CPI via net.
The disadvantage would be that this private key cannot be used locally, as it is not possible to  download it.

Ceate Key Pair in CPI

We go to keystore and choose "Create -> Key Pair"
We enter some data of your choice, e.g.
🔸Alias:  “demokeypair”
🔸CN: “demokeypair”
It will be used for creating the certificate.

 The alias name “demokeypair” will be used below in the Groovy scripts, in order to load the keys.

Note.
If you need examples of creating key pairs or certificate chains with OpenSSL, please refer my other blog posts.

2. Create iFlow

In this section, we’re creating a simple iFlow that does nothing but sign and verify a hard-coded xml-message.

2.1. The iFlow

Our iFlow will look as follows:

 Let’s quickly go through the configuration.

🔷Timer
We define a start event via "Timer" with default properties, i.e. run once
🔷  Content Modifier
We use a "Content Modifier" in order to hard-code a dummy xml-payload in the "Message Body".
It represents an incoming HTTP request (or similar). 
The content is copied from Appendix 1
🔷 Groovy Script1
The script used for signing.
The content is copied from Appendix 2
🔷 Groovy Script2
The script used for verification.
The content is copied from Appendix 3

2.2. The security library

In our Groovy code, we’re using the library Apache Santuario.
It is an implementation of the "XML Signature" standard, thus perfect for our needs.
The drawback: the library is not available in the CPI Java runtime.
We need to manually get hold of the .jar file and upload it to the iFlow.

2.2.1. Download the "xmlsec" jar file

A common place to find and download jar files is the "mavenrepository".  
We find our library at https://mvnrepository.com/artifact/org.apache.santuario/xmlsec
We choose a version (in my example: 4.0.2)
We click on "bundle" to download the jar file.
in my example the file name is xmlsec-4.0.2.jar.

Alternatively: use maven as described in Appendix 4.

2.2.2. Upload

To make the jar file available for our scripts in the iFlow, we open our iFlow.
To make sure that nothing is selected, we click on the background of the designer.
At the “Integration Flow” properties, we open the tab “References”, then click on “Add -> Archive”.
We browse to our jar file and confirm the dialog.
That’s it.

 2.3. The Groovy Script for Signing

Now let’s go through the code for signing xml payload according to the "XML Signature" standard.

Preparation

A brief look at the required packages:

import com.sap.it.api.ITApiFactory
import javax.xml.*
import org.w3c.*

We need some basic functionality for dealing with xml (not surprising in case of XML payload).
We cannot use native groovy xml-parsing, we need org.w3c because the apache library is based on it.
For the security related operations, we use some native Java packages:

import java.security.*
import javax.crypto.*

And finally, the protagonist, the XML Security implementation of Apache Santuario:

import org.apache.xml.security.*

Note:
I think it is definitely important to rely on a public implementation of the security standard and not to implement the single steps manually.
Reasons are the compatibility, flexibility of the standard, secure implementation, vulnerability-fixes that come with the library.

First thing we have to do is to initialize the apache library:

org.apache.xml.security.Init.init();

Key Pair

At the beginning, let’s retrieve the key pair, as both keys will be required in the script.
The key pair which is stored in the CPI keystore can be fetched using the CPI runtime API.

KeystoreService keystoreService = ITApiFactory.getService(KeystoreService.class, null) 
KeyPair keyPair  = keystoreService.getKeyPair(alias) 
return keyPair.getPrivate()

Parse xml string

Next first thing we have to do is to read the xml message and to parse it into a org.w3c.Document instance, with the help of our helper method:

Document document = convertToDocument(message.getBody(InputStream.class))

We need this document instance later.

It is our task to compose the <Signature> subtree, according to our needs and with the desired configurations.
As we know from the introduction blog post we need to specify the algorithms for signature, for digest, for canonicalization.

XMLSignature signature = new XMLSignature(doc, "", XMLSignature.ALGO_ID_SIGNATURE_RSA_SHA1)
rootElement.appendChild(signature.getElement())

Note:
The empty string parameter above is used to specify a base URI, in case that relative URLs are used in the document.
Above code places the <Signature> element below the root. We can do so, because we know that the <order> element is a child of root as well. What we want to do is to have the <Signature> subtree as sibling of the <order> node which we are signing.

The <Reference> element is composed implicitly, but we have to create the children, like <Transform> elements.

Transforms transforms = new Transforms(doc);
transforms.addTransform(Transforms.TRANSFORM_ENVELOPED_SIGNATURE);
transforms.addTransform(Transforms.TRANSFORM_C14N_EXCL_OMIT_COMMENTS);

KeyInfo
The <KeyInfo> element is optional.
It allows to send the public key along with the message. This makes it easier for the receiver to verify the signature.
However, if the receiver already has the public key, then it doesn’t need to be sent.
In our example, we have the public key in the keystore, so we don’t really have to send it.
On the other side, we want to showcase how it could be done.

Note:
Instead of the public key, we could also send the certificate, which contains the public key.
This has the benefit that the metadata of the certificate could be validated by the recipient.
Another option would be to send only the serial number (or similar) of the certificate, assuming that the receiver has the certificate already and only needs to find the right one and to validate some metadata.

signature.addKeyInfo(publicKey)

addDocument

Some explanation is required with respect to the xml element which should be signed.
In the corresponding method, this element has to be specified by a String which is a URI:

signature.addDocument("#id_1", transforms, MessageDigestAlgorithm.ALGO_ID_DIGEST_SHA256);  

This URI (first method parameter) can be relative (using the "#" notation) or it can point to any other xml, as specified by the XML Signature spec.
This is nice and flexible.
However, internally, it requires that the targeted element can be identified by an “ID”.
Defining an “ID” attribute for an xml element is not so simple:
We cannot just create an attribute with name “ID” or “id” or “Id”.
If we create such an attribute, it would be just a normal attribute with any name.
It has to be marked as a special “ID”-attribute.
This can be done with an XML scheme.
Or it can be done programmatically.
That’s what we’re doing in our sample, it seems the easier way.
We call our helper method:

setIdAttribute(doc, "order", "identifier")

which is implemented as follows:

def setIdAttribute(Document doc, String elementName, String idAttributeName) {
    Element orderElemToVerify = (Element)doc.getElementsByTagName(elementName).item(0);
    orderElemToVerify.setIdAttribute(idAttributeName, true);
}

Note that in your productive code this is probably not required.

Sign

Finally, the signature is created by providing the private key:

 signature.sign(privateKey)

All other configuration has been done before.

Note:
As usual, we're skipping all error handling, to make the code sample easier to read.

Output

That’s all for the signing.
At the end, we only need to convert the org.w3c.Document instance to a string and set it as message body of the iFlow.

Summary

🔹Prepare document
🔹Fetch Keys
🔹Compose <Signature> node
🔹Here: set ID attribute
🔹Sign the desired element

2.4. The Groovy Script for Verification

The verification script is much shorter because the library is in charge of finding the required elements within  the XML.
We only need to take care of properly initializing the lib.

Prepare XML Document

We  need to convert the XML message payload from string to org.w3c.Document:

Document document = convertToDocument(message.getBody(InputStream.class))

Again, we need to configure the attribute which we want to act as “ID”:

setIdAttribute(docToVerify, "order", "identifier")

Next, we have to get a hold of the element which represents the <Signature>, then wrap it in an XMLSignature object:

Element sigElement = (Element) docToVerify.getElementsByTagNameNS(Constants.SignatureSpecNS, "Signature").item(0)
XMLSignature verifySig = new XMLSignature(sigElement, "")

Note:
As mentioned, that code should be replaced by xPath!

For verification, we need the public key, so we fetch it from CPI keystore.
Note:
As per design, everybody should be enabled to verify a signature, that’s why the verification is done with a public key (not private key).

PublicKey publicKey = getPublicKeyFromKeystore("demokeypair") 

Note:
We’re making our lives too easy, you might think, by taking the public key from keystore instead of reading it from the XML.
The answer is yes, correct.
And the reason is that I’m planning to publish another tutorial for more use cases.

Finally, do the verification:

boolean verificationResult = verifySig.checkSignatureValue(publicKey)

The verification is a simple check method that returns a Boolean.

In our simple sample, we just print the result to the console.
In productive world, the iFlow should fail, an exception should be thrown, etc      

Little summary

🔹Find the <Signature> node in the XML payload
🔹Fetch the public key
🔹Configure an XMLSignature and run the check.

3. Run Scenario

At this point we’ve created a very simple iFlow with hard-coded xml-payload, signing script and verification script.
The scripts produce log output, such that we can view the message content before and after each step.

Finally, we can deploy the iFlow, I will be triggered automatically and we can view the results in the log at
Monitor -> Integrations -> Monitor Message Processing -> All Artifacts

Summary

After learning the “XML Signature” standard in the previous blog post, today we applied the learnings in java code, based on the Apache Santuario implementation of the standard.
The Santuario library knows the structure and the process of creating an “XML Signature”, we only need to collect the required information and configure the library objects:
For signing, we need:
🔹the signature mode, in our example “enveloped”
🔹the XML element to be signed
🔹the Canonicalization method
🔹the Digest alg: e.g. SHA256  (used to calculate a hash of the signed message)
🔹the Signature alg: e..g SHA256withRSA
🔹the Certificate or public key or metadata, required for verification, to be sent in the <KeyInfo> element

When it comes to verification, the library will extract all required info from the <Signature> element.
We only need to help to find it.
Also, we need to take care of the public  key, which may be included in the <KeyInfo> subnode, or we have it already and may want to compare the certificate metadata.
And again (for the last time):
The XML structure as a whole should be validated as well.
Therefore, xPath should be used

Links

Specification
W3C recommendation XML Signature: https://www.w3.org/TR/xmldsig-core/

Java library
Apache Santuario

Blogs
Intro Blog : Understanding the "XML Signature" standard.
Understanding the CMS (PKCS7) Standard.
The Security Glossary.

Appendix 1: Sample XML Payload

<SalesService>
   <order identifier="id_1">
      <customer>Joe</customer>
      <product>cat</product>
   </order>
</SalesService>

Appendix 2: Sample Groovy Script for Signing

import com.sap.gateway.ip.core.customdev.util.Message
import com.sap.it.api.ITApiFactory
import com.sap.it.api.keystore.KeystoreService

import java.io.ByteArrayOutputStream
import java.io.InputStream
import java.security.KeyPair
import java.security.Key
import java.security.PrivateKey
import java.security.PublicKey
import javax.xml.parsers.DocumentBuilder
import javax.xml.parsers.DocumentBuilderFactory
import javax.xml.transform.OutputKeys
import javax.xml.transform.Transformer
import javax.xml.transform.TransformerFactory
import javax.xml.transform.dom.DOMSource
import javax.xml.transform.stream.StreamResult

import org.apache.xml.security.signature.XMLSignature
import org.apache.xml.security.transforms.Transforms
import org.apache.xml.security.utils.Constants
import org.apache.xml.security.keys.KeyInfo
import org.apache.xml.security.utils.XMLUtils

import org.w3c.dom.Document
import org.w3c.dom.Element
import org.xml.sax.InputSource


/* the main method */

def Message processData(Message message) {
    
    def signedDocument = doSign(message)
    
    def signedDocAsString = convertDocumentToString(signedDocument)
    message.setBody(signedDocAsString)
    return message
}


/* Sign */

def Document doSign(message) throws Exception {
    org.apache.xml.security.Init.init()

    PrivateKey privateKey = getPrivateKeyFromKeystore("demokeypair")
    PublicKey publicKey = getPublicKeyFromKeystore("demokeypair")   

    Document doc = convertToDocument(message.getBody(InputStream.class))
    writeToLog (doc, message, "Before signing:\n")
    Element rootElement = doc.getDocumentElement()
    rootElement.normalize()
     
    // signature object
    XMLSignature signature = new XMLSignature(doc, "", XMLSignature.ALGO_ID_SIGNATURE_RSA_SHA1)
    rootElement.appendChild(signature.getElement())
    
    // set transforms
    Transforms transforms = new Transforms(doc)
    transforms.addTransform(Transforms.TRANSFORM_ENVELOPED_SIGNATURE)
    transforms.addTransform(Transforms.TRANSFORM_C14N_EXCL_OMIT_COMMENTS)
    
    // set the element to be signed: the <order> element
    //the <order> should be signed. The FWK finds it by "id", so we have to set the "identifier" attr as id-attribute (alternatively, do with scheme)
    setIdAttribute(doc, "order", "identifier")

    signature.addDocument("#id_1", transforms, Constants.ALGO_ID_DIGEST_SHA1)  // the URI references an elem by id via this notation

    //the <KeyInfo> section contains the public key, to be used by recipient for verification
    signature.addKeyInfo(publicKey)
    
    // sign
    signature.sign(privateKey)

    writeToLog (doc, message, "After signing:\n")
    return doc
}


/* Helper */

def Document convertToDocument(InputStream stream){
   DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance()
   factory.setNamespaceAware(true)
   DocumentBuilder builder = factory.newDocumentBuilder()
   return builder.parse(new InputSource(stream))
}

def String convertDocumentToString(Document document){
    ByteArrayOutputStream outputStream = new ByteArrayOutputStream()
    XMLUtils.outputDOM(document, outputStream)
    return outputStream.toString()
}

def setIdAttribute(Document doc, String elementName, String idAttributeName) {
    Element orderElemToVerify = (Element)doc.getElementsByTagName(elementName).item(0)
    orderElemToVerify.setIdAttribute(idAttributeName, true)
}

def PrivateKey getPrivateKeyFromKeystore(alias){
    KeystoreService keystoreService = ITApiFactory.getService(KeystoreService.class, null) 
    KeyPair keyPair  = keystoreService.getKeyPair(alias) 
    return keyPair.getPrivate()    
}

def PublicKey getPublicKeyFromKeystore(alias){
    KeystoreService keystoreService = ITApiFactory.getService(KeystoreService.class, null) 
    KeyPair keyPair  = keystoreService.getKeyPair(alias) 
    return keyPair.getPublic()    
}

def writeToLog(doc, message, text){
    StringWriter stringWriter = new StringWriter()           
    Transformer transformer = TransformerFactory.newInstance().newTransformer()
    transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes")
    transformer.setOutputProperty(OutputKeys.METHOD, "xml")
    transformer.setOutputProperty(OutputKeys.INDENT, "yes")
    transformer.setOutputProperty(OutputKeys.ENCODING, "UTF-8")
    transformer.transform(new DOMSource(doc), new StreamResult(stringWriter))

    def messageLog = messageLogFactory.getMessageLog(message)
    messageLog.addAttachmentAsString("Sign", text + stringWriter.toString(), "text/plain")

    return stringWriter.toString()
}

Appendix 3: Sample Groovy Script for Verification

import com.sap.gateway.ip.core.customdev.util.Message;
import com.sap.it.api.ITApiFactory;
import com.sap.it.api.keystore.KeystoreService;

import java.io.ByteArrayOutputStream;
import java.io.InputStream
import java.security.KeyPair;
import java.security.PublicKey;
import javax.crypto.KeyGenerator;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.transform.OutputKeys;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;

import org.apache.xml.security.signature.XMLSignature;
import org.apache.xml.security.transforms.Transforms;
import org.apache.xml.security.utils.Constants;
import org.apache.xml.security.keys.KeyInfo;
import org.apache.xml.security.utils.XMLUtils

import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.xml.sax.InputSource;


/* The main method */

def Message processData(Message message) {
    
    doVerify(message)
    
    return message
}


/* Verify */

def doVerify(message) throws Exception {
    org.apache.xml.security.Init.init();

    Document docToVerify = convertToDocument(message.getBody(InputStream.class))
    writeToLog (docToVerify, message, "Before verify:\n")

    //prepare the id-attribute
    setIdAttribute(docToVerify, "order", "identifier");

 	// compose the signature object
 	Element sigElement = (Element) docToVerify.getElementsByTagNameNS(Constants.SignatureSpecNS, "Signature").item(0); // TODO replace this with xPath
    XMLSignature signature = new XMLSignature(sigElement, "");
    PublicKey publicKey = getPublicKeyFromKeystore("demokeypair")   
    
    // run the verification check
    boolean verificationResult = signature.checkSignatureValue(publicKey);
    writeToLog (null, message, "Verification Result: " + Boolean.toString(verificationResult))
}

/* Helpers */

def setIdAttribute(Document doc, String elementName, String idAttributeName) {
    Element element = (Element)doc.getElementsByTagName(elementName).item(0);
    element.setIdAttribute(idAttributeName, true);
}

def PublicKey getPublicKeyFromKeystore(alias){
    KeystoreService keystoreService = ITApiFactory.getService(KeystoreService.class, null) 
    KeyPair keyPair  = keystoreService.getKeyPair(alias) 
    return keyPair.getPublic()    
}

def Document convertToDocument(InputStream stream){
   DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
   factory.setNamespaceAware(true);
   DocumentBuilder builder = factory.newDocumentBuilder();
   return builder.parse(new InputSource(stream));
}

def String convertDocumentToString(Document document){
    ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
    XMLUtils.outputDOM(document, outputStream);
    return outputStream.toString()
}

def writeToLog(doc, message, text){
    def messageLog = messageLogFactory.getMessageLog(message)
    if(doc == null){
        messageLog.addAttachmentAsString("Verify", text, "text/plain")
        return    
    }
    
    StringWriter stringWriter = new StringWriter()           
    Transformer transformer = TransformerFactory.newInstance().newTransformer()
    transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes")
    transformer.setOutputProperty(OutputKeys.METHOD, "xml")
    transformer.setOutputProperty(OutputKeys.INDENT, "yes")
    transformer.setOutputProperty(OutputKeys.ENCODING, "UTF-8")
    transformer.transform(new DOMSource(doc), new StreamResult(stringWriter))

    messageLog.addAttachmentAsString("Verify", text + stringWriter.toString(), "text/plain")
}

Appendix 4: Maven pom file

Create a maven project and enter the following dependency in the dependencies section.
Note that you might need to adapt the version.
Today, the current version is 4.0.2.

<dependencies>
    <dependency>
        <groupId>org.apache.santuario</groupId>
	<artifactId>xmlsec</artifactId>
	<version>4.0.2</version>
    </dependency>
</dependencies>

Run the command
mvn package
Maven will download the dependencies to the local repository at
C:\Users\joe\.m2\repository\org\apache\santuario\xmlsec\4.0.2\xmlsec-4.0.2.jar
From here it can be uploaded to CPI.