Skip to Content
Technical Articles
Author's profile photo Vadim Klimov

Execution of OSGi Shell Commands on CPI Runtime Node

Disclaimer

Material in this blog post is provided for information and technical features demonstration purposes only. The described technique might introduce CPI runtime node stability and security risks, if applied carelessly or if used to execute imprudent or unsafe OSGi shell commands.

 

Intro

SAP CPI runtime nodes run in an OSGi container and benefit from capabilities that underlying OSGi framework and OSGi runtime provide – such as modular system, components lifecycle management and accompanying runtime services, comprehensive ecosystem, to name a few. While it is normally and commonly not required for CPI developers to deep dive into OSGi architecture and OSGi runtime services, it is worth being aware of capabilities and tools that the runtime provides – for example, this can become useful when analyzing certain types of errors, or when developing advanced monitoring tools.

CPI uses OSGi runtime Apache Karaf that runs on top of OSGi framework Apache Felix. One of highly demanded and feature-rich tools that Karaf ships with, is an OSGi shell console. Commands that are accessible using this shell, allow execution of wide spectrum of monitoring and administration activities that are exposed by Karaf core components (instance administration, features and bundles management, user management, logs browsing, and so on), and can be complemented with additional commands that are provided by installed bundles (for example, when Apache Camel features and installed, OSGi shell commands that are specific to Camel framework, can be added). Below are very few screenshots of an OSGi shell console taken from a locally running Karaf instance:

An OSGi shell console is a very powerful tool and it is commonly accessed by Karaf administrators remotely using SSH protocol, but tenant access via SSH is not a feasible option for CPI customers. In absence of SSH access to a CPI runtime node, we can still benefit from OSGi shell commands, as corresponding functionality can also be accessed and consumed programmatically. A part of Karaf core bundles is Karaf Shell Core bundle that provides a corresponding service, which can be used to execute OSGi shell commands.

Ahead of reading this blog post further, please ensure that you are familiar with general concepts of modularization that are at the heart of OSGi framework. Description of OSGi concepts goes beyond scope of this post and will not be covered here, but corresponding knowledge (in particular, understanding of OSGi bundle context, bundle, service reference and service) is a prerequisite to make practical use of material described below.

 

Overview

In the demo, OSGi shell commands are going to be submitted in a request message body.
In sake of simplicity, the iFlow consists of only one step, which is a Groovy script step that implements the described approach:

The iFlow doesn’t check if the message body exists at all – it is assumed that an OSGi shell command is provided in the body of the submitted request, otherwise the iFlow shall not be called. Should this be a production-oriented scenario, such a condition should have been considered and handled appropriately.

Similarly, corresponding checks should be implemented to check if accessed objects exist (are not null) before invoking their instance methods to avoid NullPointerException runtime errors, but those are deliberately omitted in demonstrated code snippets to keep the demo focused and compact.

 

Implementation option 1: SessionFactory service of Apache Karaf Core bundle

Each integration flow that is deployed to a CPI runtime node, is represented as a bundle. Moreover, all bundles that are deployed to a runtime node, share the same bundle context. Given flexibility and extensive capabilities of Groovy scripting that can be embedded into an iFlow, it is possible to explore bundle context by accessing it from the iFlow, where such a Groovy script step is placed:

Bundle msgBundle = FrameworkUtil.getBundle(Message.class)
BundleContext context = msgBundle.bundleContext

The above code snippet consists of two steps:

  1. Get a bundle for the class that represents a message (com.sap.gateway.ip.core.customdev.util.Message) – FrameworkUtil.getBundle(),
  2. Get a bundle context of the bundle – Bundle.getBundleContext().

We got an object instance that represents a BundleContext (org.osgi.framework.BundleContext) – with its help, we can explore registered bundles, services that each bundle provides or bundles that use a corresponding service.

As it has been mentioned earlier, a Karaf Shell Core (org.apache.karaf.shell.core) bundle provides a relevant service that can be used to execute shell commands – namely, a SessionFactory (org.apache.karaf.shell.api.console.SessionFactory) service that can be accessed from the bundle context. Note that when creating a session, we are expected to provide input and output streams that will be used by the session. When using the session only for execution of OSGi shell commands, input stream can be null, and output streams are used to handle corresponding output and error streams:

ServiceReference<SessionFactory> sessionFactorySvcRef = context.getServiceReference(SessionFactory.class)
SessionFactory sessionFactory = (SessionFactory) context.getService(sessionFactorySvcRef)

Next, a Session (org.apache.karaf.shell.api.console.Session) can be created using earlier obtained SessionFactory.

ByteArrayOutputStream out = new ByteArrayOutputStream()
ByteArrayOutputStream err = new ByteArrayOutputStream()
Session session = sessionFactory.create(new ByteArrayInputStream(), new GroovyPrintStream(out, true), new GroovyPrintStream(err, true))

An OSGi shell command can now be passed to a session as a readable sequence of characters (java.lang.CharSequence) and executed by calling Session.execute():

String command = message.getBody(String.class)
session.execute(command)

After the command has been executed, its output can be redirected to a desired output destination – in the demo, standard output will be placed to a message body – and the session can be closed, unless further commands need to be executed or any other session related activities need to be performed within the same session:

message.body = out.toString()
session.close()

 

As a summary, let me put together all parts that have been described above, into a complete code snippet of the script. Some method calls have been chained to make code snippet more compact:

import com.sap.gateway.ip.core.customdev.util.Message
import groovy.io.GroovyPrintStream
import org.apache.karaf.shell.api.console.Session
import org.apache.karaf.shell.api.console.SessionFactory
import org.osgi.framework.BundleContext
import org.osgi.framework.FrameworkUtil

Message processData(Message message) {

    String command = message.getBody(String.class)

    ByteArrayOutputStream out = new ByteArrayOutputStream()
    ByteArrayOutputStream err = new ByteArrayOutputStream()

    BundleContext context = FrameworkUtil.getBundle(Message.class).bundleContext
    SessionFactory sessionFactory = (SessionFactory) context.getService(context.getServiceReference(SessionFactory.class))
    Session session = sessionFactory.create(new ByteArrayInputStream(), new GroovyPrintStream(out, true), new GroovyPrintStream(err, true))
    session.execute(command)
    message.body = out.toString()
    session.close()

    return message

}

 

Below are examples of some OSGi shell commands’ execution:

  • List installed bundles:

  • List installed features:

  • List services:

  • List events and their details:

 

Implementation option 2: ShellExecutor service of Neo Shell Karaf bundle

An approach that has been described earlier, uses a Karaf native service – in other words, it is based on “vanilla” Karaf capabilities. As it could have been observed, an implementation that is needed to execute OSGi shell commands programmatically, is not excessively complex, but it requires interaction with some lower-level objects – for example, streams (input and output) and session (handling of creation and closure). Among other CPI specific components, SAP provides a Neo Shell Karaf (com.sap.it.nm.plaf.neo:neo.shell.karaf) bundle that implements wrapper functionality on top of corresponding native Karaf functionality and complements it with some additional capabilities – such as whitelisting of certain OSGi shell commands, separation of commands into read-only and modify.

This implementation option is based on a similar approach as the earlier described option – namely, usage of a dedicated service that is obtained from the bundle context. Hence, the first step remains the same – we need to obtain a bundle context.

Next, in contrast to usage of a SessionFactory service provided by a Karaf Shell Core bundle, this implementation option makes use of a ShellExecutor (com.sap.it.nm.spi.diag.ShellExecutor) service provided by a Neo Shell Karaf (com.sap.it.nm.plaf.neo:neo.shell.karaf) bundle:

ServiceReference<ShellExecutor> shellExecutorSvcRef = context.getServiceReference(ShellExecutor.class)
ShellExecutor shellExecutor = (ShellExecutor) context.getService(shellExecutorSvcRef)

Finally, an OSGi shell command can be executed by calling ShellExecutor.execute() and passing command string as an argument, command output is returned as a string value. Note that we don’t need to create a session, handle output stream and close the session at the end – this all is handled behind the scene by wrapper functionality:

String command = message.getBody(String.class)
message.body = shellExecutor.execute(command)

 

As a summary, a complete code snippet of the script that implements the described alternative option, is compacted and provided below:

import com.sap.gateway.ip.core.customdev.util.Message
import com.sap.it.nm.spi.diag.ShellExecutor
import org.osgi.framework.BundleContext
import org.osgi.framework.FrameworkUtil

Message processData(Message message) {

    String command = message.getBody(String.class)

    BundleContext context = FrameworkUtil.getBundle(Message.class).bundleContext
    ShellExecutor shellExecutor = (ShellExecutor) context.getService(context.getServiceReference(ShellExecutor.class))
    message.body = shellExecutor.execute(command)

    return message

}

 

Below is an example of execution of an OSGi shell command to list installed bundles that has been demonstrated earlier using a SessionFactory service, but this time it gets executed using a ShellExecutor service:

 

It shall be noted that since a ShellExecutor service of a Neo Shell Karaf bundle implements support of only those OSGi shell commands that are whitelisted by SAP, it doesn’t support all commands that can be executed using a SessionFactory service of a Karaf Shell Core bundle. Whitelisted read-only and modify OSGi shell commands can be checked by accessing a corresponding enumeration (com.sap.it.nm.plaf.neo.shell.karaf.KarafOsgiCommmand) or getter methods associated with it – KarafOsgiCommmand.allReadOnlyCommands (for read-only commands) and KarafOsgiCommmand.allModifyCommands (for modify commands). In case of attempting to execute a valid OSGi shell command that is not whitelisted, NotSupportedCommand error (“Command is not a white listed command for the OSGi shell”) will be thrown. For example, and earlier demonstrated command that is used to display events (event:display), is not a SAP whitelisted OSGi shell command – as a result, the below error is returned when executing this command using a ShellExecutor service:

 

Outro

Two implementation options have been illustrated above. While both are based on the same concept and use the same underlying functionality of Karaf Shell Core, each has usage nuances that shall not be neglected and shall be taken into consideration.

A SessionFactory service of a Karaf Shell Core bundle provides a lower-level access to OSGi shell commands. A ShellExecutor service of a Neo Shell Karaf bundle provides a higher-level access to OSGi shell commands and introduces restrictions related to a list of accessible commands, but simplifies steps required to execute commands – it is just few code lines that are required to execute OSGi shell commands.

As a matter of interest, it is also worth mentioning that some other CPI specific components utilize functionality of a Neo Shell Karaf bundle. For example, it can be observed that one of Operations API commands – OsgiShellCommand – accesses functionality of a ShellExecutor. Demonstration of Operations API is out of scope of this post, but if you are interested in exploring Operations API and want to read more about it, check the blog post “CPI: Exploring the hiden and hideous /Operations url” written by Ariel Bravo Ayala.

Assigned Tags

      11 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Fatih Pense
      Fatih Pense

      Dear Vadim,

      Thank you for the clear, technical, and fun walk inside the internals of CPI.

      Best regards,
      Fatih

      Author's profile photo Vadim Klimov
      Vadim Klimov
      Blog Post Author

      Thank you, Fatih, glad you found this material interesting.

      Author's profile photo Sunil Chandra
      Sunil Chandra

      Hi Vadim,

      I always find your blogs exciting and detailed enough. It intrigues me for hands on immediately.

      Really appreciate your efforts exploring inner side of product - kind of reverse engineering.

      Regards,

      Sunil Chandra

      Author's profile photo Vadim Klimov
      Vadim Klimov
      Blog Post Author

      Sunil, thank you for this inspiring feedback. I hope this kind of information that shows what capabilities of underlying components can be used, and how they can be consumed, will turn to be useful for troubleshooting and monitoring. Although, I shall admit that some features that are accessible using those techniques, might turn to be dangerous - especially some of modify commands - so it is always a good idea to test as much as possible locally before executing such activities in the CPI tenant, and to consider side effects. After all, this is a toolbox - and tools can be applied in various ways in real life scenarios.

      Author's profile photo Raffael Herrmann
      Raffael Herrmann

      Hi Vadim,

      this is another great post by you (but to be honest I haven't expected anything else)! Thanks for sharing your results with us. While reading I already got some interesting ideas on how to use the shell... 😉

      Regards,
      Raffael

      Author's profile photo Vadim Klimov
      Vadim Klimov
      Blog Post Author

      I absolutely agree, with the (indirect) access to shell commands, it is possible to do truly a lot. And I guess such a creative person as yourself will immediately find a lot of use cases where shell access will make perfect sense and will bring value, so I'm looking forward to seeing your coming developments.

      Author's profile photo Vijay Kapuganti Kumar
      Vijay Kapuganti Kumar

      Hi Vadim,

       

      Another wonderful blog from your end !!!.

      When i try to use a command to stop the CPI bundle using the command

      “bundle:stop FB_Get_Post_Comments” has no effect. As i have understood it will uninstall the bundle in CPI. can you please check and correct if my understanding is wrong.

       

      Thanks and Regards,

      Vijay.

      Author's profile photo Vadim Klimov
      Vadim Klimov
      Blog Post Author

      I would question why do you want to stop/undeploy bundles in such a way in the CPI tenant? A reason for this question is, these operations can impact other bundles in case they are dependent on the bundle you stop/undeploy, so you shall be aware about if the bundle has any dependencies and what consequences will be if functionality provided by that bundle, is not accessible anymore in the tenant. For example, if the bundle relates to an iFlow, then I will suggest using Web UI to undeploy such a bundle by undeploying a corresponding iFlow.

      Next, I'm not sure if I understood a question correctly - do you want to stop or undeploy the bundle? These are two different operations for an OSGi runtime:

      • A command 'bundle:stop' is used to stop the bundle - the bundle becomes suspended in the runtime (and its capabilities will not be accessible), but it will not be undeployed from the runtime. If it happens later that you will want to bring bundle's capabilities back, the bundle can be started (resumed) with the help of a command 'bundle:start'.
      • A command 'bundle:uninstall' is used to uninstall (undeploy) the bundle - the bundle gets removed from the runtime, it cannot be started again (as it will not exist anymore at runtime), and if we will need to bring bundle's capabilities back, we will need to deploy the bundle again, for example, using a command 'bundle:install'. As a consequence, the bundle is also likely to get a new bundle ID after being uninstalled and installed again (opposite to bundle stop/start commands, where the bundle remains deployed at runtime and keeps its bundle ID).

      A reason that you don't see an effect when executing a command 'bundle:stop' (which can be checked by executing a command 'bundle:status' and checking what is a current status of a given bundle) is because this command - bundle:stop - is not available in a list of commands of a Karaf instance that is run by a CPI runtime node. Although both bundle:stop and bundle:uninstall are a part of standard Karaf shipment and are available via an OSGi shell console, they have been disabled in CPI. My guess about motivation behind it, will be that it could have been done for security purposes.

      There is a number of other standard Karaf OSGi shell console commands that you might find unavailable when trying to execute them using above described methods. To check which commands are available (and if executed commands are actually in that list of available commands), you can execute a command 'help' - or, for example, if you are only interested to see which bundle related commands are available, then you can restrict a search by executing 'help bundle:'.

      Author's profile photo Deep Kay
      Deep Kay

      Hello Vadim,

      Thanks for sharing these valuable information. Your blogs are of great help and handy and I had learned quite a lot from those.

      I have followed your blog Dark Side of Groovy and tried getting hold of some class jar (xx) for locally testing groovy in my Intelli J (of course following Eng Swee's blogs)

      When I try accessing the jar using below path using the bundle id and version approach you mentioned elsewhere I get error that there is no path/file exists.

      /usr/sap/ljs/data/cache/bundle${bundleId}/version${bundleVersion.replace(':', '.')}/bundle.jar

      I even tried to browse the file structure using different methods, but the /usr/sap/ljs/data/cache directory itself is not available.

      Is there recent change with the OSGi container in the way the libraries are stored ? Could you please guide me on how to download the JAR files for local development/test ?

      Thanks in advance!

       

       

      Author's profile photo Vadim Klimov
      Vadim Klimov
      Blog Post Author

      Hello Deep,

      Thank you for your feedback, I'm glad you find those materials helpful to you.

      Regarding the issue you got - I have just checked it in my CPI tenant, and couldn't replicate it: the /usr/sap/ljs/data/cache directory is available and cached bundles are contained in it.

      It might be helpful to check what directories are available for you in the tenant - for example, at the level of /usr/sap/ljs/data.

      Author's profile photo Deep Kay
      Deep Kay

      Hello Vadim,

       

      Thanks for taking time to test this and respond.

      Its interesting to know that your tenant has the directory structure with /usr/sap/ljs/data/cache.

      When I look at my tenant, the only directory inside /usr/sap/ljs is 'camel' and there is no 'data' directory.

      And inside /usr/sap/ljs/camel there are separate camel-tmp-* directories for each iflow.

      I am currently using a trial account. Could that be a reason for a different directory structure ? But browsing around I don't see a 'data' directory anywhere under root.

       

      Thanks