PGP Secret Keyring in CPI: The Lost Passphrase Recovery

Disclaimer

Material in this blog post is provided for information and technical features demonstration purposes only. The described technique might introduce security risks, if applied carelessly. In real-life scenarios, the technique shall be assessed and evaluated for practical applicability only in emergency cases and shall be applied only by authorized personnel.

A security incident report has been submitted to the SAP Product Security Response Team, and the recommendation that has been provided by the SAP team, is to ensure that relevant control measures are in place, as described in the corresponding blog post.

Intro

In SAP CPI, PGP keys are stored in two keyrings:

• public keys (used in content encryption and signature verification scenarios) are located in the public keyring (pubring),
• private key and signing key pairs (used in content decryption and signing scenarios) are located in the secret keyring (secring).

Both keyrings are maintained as corresponding security material artifacts in the CPI tenant.
While public keys are generally accessible, material contained in the secret keyring is protected with the passphrase. It shall be noted that CPI doesn’t use key specific passphrases (where each key in a secret keyring is protected with its own passphrase) – in contrast, a single common passphrase is used to secure entire content of the secret keyring.

A passphrase of a secret keyring – similarly to passwords and any other sensitive security material – shall be stored in a customer’s secure vault. Access (even read-only) to passphrases shall be restricted to a very few individuals who are responsible for generation of key pairs and maintenance of secret keyrings in CPI tenants. If a private key was compromised (for example, the key and the passphrase were leaked or stolen), such a key shall be revoked and a new key pair shall be generated as a replacement for a compromised key pair. Correspondingly, given all material contained in a CPI tenant’s secret keyring, is protected with a common passphrase, if the secret keyring and its passphrase are compromised, we shall treat such a security incident as an equivalent of all keys stored in the given secret keyring being compromised. If there are many key pairs stored in a secret keyring, the one might envision efforts that are required to revoke and replace compromised entries with newly generated ones.

Oops, we lost the passphrase of the secret keyring. Houston, we have a problem.

We are clear about importance of restricting access to private keys and secret keyring passphrases, and consequences of failing to do so. Now, what if we used to keep the passphrase in a protected and restricted vault, but due to some unforeseen circumstances, we accidently lost that passphrase?

CPI doesn’t offer standard tools to cope with such situations as loss of the passphrase of the secret keyring. A passphrase of a secret keyring is not recoverable – we cannot restore or reset it, if we only possess the secret keyring alone. CPI wis still able to use material that is already stored in such a secret keyring, but it will not be possible to update the secret keyring – for example, add a new key pair, remove or replace an existing key pair. As a result, loss of the passphrase is commonly treated in a similar way as if the passphrase would have been compromised: affected security material is revoked, new key pairs are generated and are used instead.

Recreation and replacement of the secret keyring?! We need a plan B. OSGi to the rescue!

In this blog post, I would like to demonstrate an alternative approach to replacing the affected secret keyring – that is, a method to programmatically retrieve the passphrase of the secret keyring from the CPI tenant.

CPI uses an OSGi container – Apache Karaf – as its application runtime. One of fundamental principles and core layers of the OSGi architecture is the service layer, that is about services provisioning, discovery and consumption. This is an important concept that enables modularity and a high level of abstraction and transparency, as deployed components (bundles) don’t need to be aware of implementation details of the required service – instead, they can search for the required service in the registry (where corresponding services are registered by service provider bundles), and consume capabilities offered by a relevant found service. A service consumer bundle doesn’t necessarily need to be aware of which other bundle provides the service – a scope of concern of the consumer is what capability / service it needs to use, if that service is available at runtime, and how the service can be consumed (how a consumer shall interact with the service to make use of it), but not what particular bundle provides that service and how the service is implemented.

CPI application runtime is equipped with a plethora of services – some of them are provided by core components of the application runtime, some others originate from the integration framework, SAP specific components, and from third-party or custom developments. Certain services are provided even by bundles that contain developed and deployed iFlows – even though this happens transparently to integration developers and doesn’t require any explicit configuration steps from them.

An OSGi framework (and correspondingly OSGi containers that are developed on top of that framework) offers unified and well-documented mechanisms on how services can be registered, discovered, filtered and consumed – these mechanisms are widely used by deployed bundles.

Keyring accessor is yet one of services provided by an SAP component that is registered at runtime startup and that is available at runtime to other components, including iFlows’ bundles. The keyring accessor service is accompanied by a number of accessor services for some other artifacts stored in the CPI tenant, but in this blog post, I will focus only on the keyring accessor. This service can be used to access the public keyring and the secret keyring – keyrings and their properties, keys contained in keyrings, etc. When exploring operations provided by this service, our attention can be attracted by a password getter operation – and that is right what we need here, as this operation can be used to access a passphrase of a keyring in plain text. Obviously, there is no sense to apply this operation for a public keyring, as a public keyring is not protected by a passphrase, but this is sensible when working with a secret keyring.

Implementation. It’s Groovy time.

After we set the scene, let’s now get to the sample implementation.

Given iFlows are contained in corresponding generated bundles at runtime, available OSGi services can be consumed by iFlow components – including script steps that can implement arbitrary custom functions and consume available services. The approach can also extend to adapters, as adapters are just another type of bundles. In the below demo, I use a Groovy script that is added to the iFlow, but generally speaking, the described approach can be applied in conjunction with alternative programming languages that are supported by CPI – such as JavaScript that we can use to implement a script and use it in a step in the iFlow. Alternatively, we can use Java and develop an adapter, which component will also have access to corresponding OSGi services.

Here, I will focus on the Groovy script function, as remaining parts of the technical implementation – addition of the Groovy script to an iFlow, usage of the Groovy function in a Groovy script step and configuration of an endpoint that can be used to invoke the iFlow – are fairly generic.

The entire logic that is required to use the keyring accessor service and retrieve the passphrase of the secret keyring, can be split into following steps:

1. Access OSGi framework bundle context. Given that CPI uses a shared single bundle context for all deployed bundles, access to that bundle context enables us to access any other deployed bundles or services registered in that context.
2. Acquire a service reference for the required service – keyring accessor – from the bundle context. In general, multiple bundles can provide implementations of the same service class – as a consequence, we might need to retrieve and iterate through all service references for the given service class, or apply some filter to retrieve only particular needed service reference(s). When it comes to a keyring accessor service registered by SAP, there is only one service reference to it in the CPI runtime, so we don’t need to apply filters or iterate through multiple service reference instances.
3. Possessing a needed service reference, acquire a service instance from the bundle context.
4. Invoke a required service operation – in the particular case, this is a get password operation applied to the secret keyring. The call returns the passphrase of the secret keyring in plain text. To keep this demo simple, output of the call will be issued to a response message body (payload).
5. Finally, release the earlier acquired and used service instance.

A complete code snippet of the Groovy function is provided below:

import com.sap.gateway.ip.core.customdev.util.Message
import com.sap.it.nm.security.KeyringAccess
import com.sap.it.nm.security.KeyringType
import org.osgi.framework.BundleContext
import org.osgi.framework.FrameworkUtil
import org.osgi.framework.ServiceReference

Message processData(Message message) {
    BundleContext ctx = FrameworkUtil.getBundle(Message).bundleContext
    ServiceReference keyringAccessSvcRef = ctx.getServiceReference(KeyringAccess)
    KeyringAccess keyringAccess = ctx.getService(keyringAccessSvcRef) as KeyringAccess
    message.body = keyringAccess.getPassword(KeyringType.PGPSecret)
    ctx.ungetService(keyringAccessSvcRef)
    return message
}

Note: in sake of simplicity and conciseness, the provided code snippet doesn’t implement error or exception handling.

End-to-end demo

For the purpose of an end-to-end demonstration, I created an iFlow that consists of a single step – a Groovy script step that uses the Groovy function provided above. The iFlow uses HTTPS sender connection – in this way, we can call it later using some HTTP client:

I also uploaded the secret keyring to the CPI tenant:

The secret keyring isn’t empty (I added a sample key to it) and is protected with the passphrase.

After the iFlow with the described Groovy script is deployed to the runtime, let’s call it. I send an HTTP GET request to the endpoint of the iFlow using Postman:

Note that response body contains some cryptic character string – here in the demo, V8rx0x7z1SP00QR. This is the passphrase of the secret keyring in plain text (and not its encrypted version / hash).

Should we want to validate this passphrase, we can do this with the help of various PGP tools. I use a combination of GnuPG (backend) and Kleopatra (frontend) that are a part of Gpg4win for this purpose.

After the secret keyring (secring.gpg) is downloaded from the CPI tenant, entries from it can get imported to a local keyring. If this operation is executed in an interactive mode (a command: gpg --import), you will be prompted to enter a passphrase when attempting to import entries from the downloaded secret keyring to a local secret keyring. If the entered passphrase is wrong, only public keys will be imported to a local public keyring, and an operation of importing entries to a local secret keyring will fail. If the operation is executed in a batch mode (a command: gpg --batch --import), you will not be prompted to enter the passphrase when importing keys.

One way or another, when keys got imported to a local keyring, we have several other methods to test the passphrase – for example:

• When attempting to change the passphrase of the secret key (a command: gpg --edit-key, followed by execution of a command passwd).
• When exporting the secret key (a command: gpg --export-secret-keys).

In both cases, you will be prompted to enter a passphrase, and if the entered passphrase is wrong, an operation will fail.

Outro

The approach that was described and demonstrated in this blog post might raise security concerns – and that is exactly what it is intended to trigger. One of reasons for me to write this blog post – besides sharing a practical method of recovering a lost passphrase of a secret keyring – was to emphasize importance of security considerations when working with CPI and to encourage you to reflect on following few points:

• The passphrase of the secret keyring is extremely critical in CPI – its comprometation or loss shall be treated as a serious security incident.
• There are no whitelisted / public APIs that would have been documented by SAP and that can be used by customers to reset the passphrase of the secret keyring, recover or retrieve it from the CPI tenant. On the other hand, there are some undocumented mechanisms to achieve that, and those mechanisms are accessible from components deployed to a runtime node.
• Taking the above into account, it is crucial to be mindful and cautious when granting access to the CPI runtime. Well-known and widely used tools (such as Groovy scripts in iFlows) can be used to avoid disaster (such as loss of the passphrase of the secret keyring), but also can be utilized to compromise certain aspects of tenant capabilities and turn into exploits, if fall into the hands of an attacker.