Disclaimer
Material in this blog post is provided for information and technical features demonstration purposes only. The described technique might introduce CPI runtime node stability and security risks, if applied carelessly or if used to execute imprudent or unsafe shell commands.
Intro and overview
Undoubtedly, a shell is one of essential tools in administrators' toolbox – shell commands are used in day-to-day administration and monitoring activities, and they play an important role in automations of various processes. When working with CPI, normally customers don't need to be concerned about underlying components of the infrastructure where the CPI tenant is hosted, and operating system level access isn't exposed to customers. On the other hand, putting responsibility segregation aspect of cloud service provisioning models aside for a moment, is it possible to retrieve information from lower levels of the CPI runtime and utilize the power of the shell? Can we have a terminal-like access to the CPI runtime node and execute shell commands on it using a command line interface similarly to a local shell? The answer is – YES. It shall be noted that this technique isn't what customers are encouraged to use – nevertheless, let us have a look at it from perspective of technical feasibility and see few examples of its usage.
The overall overview of components that are going to be involved in the end-to-end demonstrations in this blog post, is depicted on the illustration below:
Note: the proof of concept has been performed in the CPI trial tenant that was provisioned in the Cloud Foundry environment. Due to fundamental differences in the overall architecture and the securiy model of Neo and Cloud Foundry environments, installed packages, permissions and access rights of running processes may vary between those two types of tenants. As a result, some system commands that will be demonsrated in this blog post, might not be accessible with the described technique due to differences in installed packages and permission restrictions that are in place in corresponding provisioned tenants.
Background in Java: baseline versions
Java Virtual Machine (JVM) of a CPI runtime node comes with a plethora of techniques that allow it to interact with components of the runtime environment where the JVM is executed. A lot of such capabilities are abstracted by means of Java APIs, and some others are accessible using native interfaces.
In Java, it is possible to create a native child process of the operating system and execute a system command programmatically in the context of that newly created process. This functionality is a part of the Process API, which provides a relevant reflection of a native operating system process –
java.lang.Process.There are two most commonly used approaches to control a Process instance:
- Using
java.lang.ProcessBuilder. A ProcessBuilder helps customize process attributes – for example, environment variables, a working directory, a source of standard input, destinations for standard output and standard error – and finally the command that will be executed in the context of processes that will be created using this ProcessBuilder instance. After the ProcessBuilder instance is configured, we can callProcessBuilder.start()to create one or several Process instances and trigger execution of a required command. This method is preferred when a process needs to be started with a modified environment. - Using
java.lang.Runtime. A Runtime instance is a reflection of the runtime environment where the Java application is running. The application cannot create a new runtime, but it can access its runtime by callingRuntime.getRuntime(). We can then callRuntime.exec()to create a Process instance and trigger execution of a required command – the command and optional process attributes are passed as parameters in this method call. In fact, whenRuntime.exec()is called, it creates a ProcessBuilder instance and uses it to start a single Process instance.
The Process API can be used to execute commands that are recognized by a corresponding operating system where the JVM is executed. Since the CPI runtime node runs on a Linux-based operating system, examples of commands that will be illustrated in this blog post, are going to be Linux shell commands.
For example, if we want to retrieve and print information about the used operating system and its version (I will use a Linux shell command
lsb_release for this in the demo), we can achieve that using the following Java code snippet that illustrates the former approach based on usage of a ProcessBuilder (class and method declarations, as well as import statements are omitted to keep the code snippet more compact; conversion of an InputStream to a String is significantly simplified):ProcessBuilder builder = new ProcessBuilder();
builder.command("lsb_release", "-a");
Process process = builder.start();
InputStream input = process.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(input));
String output = reader.lines().collect(Collectors.joining("\n"));
System.out.println(output);A sample implementation for the same example, but using the latter approach based on usage of a Runtime, is provided below (class and method declarations, as well as import statements are omitted to keep the code snippet more compact; conversion of an InputStream to a String is significantly simplified):
Runtime runtime = Runtime.getRuntime();
Process process = runtime.exec("lsb_release -a");
InputStream input = process.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(input));
String output = reader.lines().collect(Collectors.joining("\n"));
System.out.println(output);From Java to Groovy: a one-liner version
Can we make above presented implementations more concise when migrating from Java to Groovy? Yes! The Groovy Development Kit provides enhancements for some classes that are a part of the Java Development Kit – for example, it allows execution of a command specified in a String, by calling
String.execute(). In fact, the runtime will execute the command using a Runtime instance that was described earlier, but using Groovy, we have an option to use a handy shortcut for it. Being combined with additional capabilities that Groovy offers for handling input and output streams, it leads us to a very compact implementation that can be expressed in just a single line of Groovy code:println('lsb_release -a'.execute().text)We can execute this locally using a Groovy shell and verify that the result will be exactly the same as if we would have executed the command more traditionally, in the shell. I’m going to use an Ubuntu distribution that runs in the Windows Subsystem for Linux (WSL) in the remaining part of the demo that involves a Linux shell, but the demonstrated realization can be adapted for use with other Linux distributions or the concept can be applied and implemented for other platforms – for example, it can take the shape of a PowerShell script for a Windows platform.
Command execution using a Linux shell:
Command execution using a Groovy shell:
Groovy meets CPI: a basic iFlow version
We got to know how we can create new native child processes and execute commands in Java/Groovy – we can even run those examples locally and use results of local commands execution in our Java and Groovy applications. But that alone isn't the immediate purpose of this blog post – we are here to make the CPI runtime node execute shell commands for us.
For that, let's embed the described sample implementation into the iFlow in CPI. In sake of simplicity, the iFlow consists of only one Groovy script step that holds the described implementation:
The script function that is used in the Groovy script step, is provided below:
import com.sap.gateway.ip.core.customdev.util.Message
Message processData(Message message) {
message.body = message.getBody(String).execute().text
return message
}In this first demo, commands are going to be submitted in a request message body (hence, we will be sending HTTP POST requests to the iFlow endpoint). The iFlow doesn't check if the message body exists at all – it is assumed that a shell command is provided in the body of the submitted request, otherwise the iFlow shall not be called.
After the iFlow is deployed to runtime, when the iFlow will be triggered and our sample implementation will be invoked, it will create a new native child process and make it execute the specified command – and given it will be a JVM of the CPI runtime that will process the call, a new native child process will be created in the runtime environment of the CPI runtime node, and the command will get executed on the CPI runtime node, and not on the local machine.
Let's use Postman and test the iFlow – clearly, you can also use any relevant HTTP client tool of your choice instead. I use the same shell command as earlier – but this time, it will get executed on the CPI runtime node:
Final notes on the server-side: an enhanced iFlow version
The provided one-liner implementation is indeed an extremely simplified version, as it doesn't customize process attributes, only processes standard output stream of command execution (doesn't handle standard error stream in case the child process ends with an error), doesn't implement error handling and so on. Moreover, the implementation above expects a command to be contained in the message body.
With the introduced enhancement, we make the iFlow a bit more fault-tolerant and safe. We also ensure that we don't leave the created child process unattended and that we kill the child process after some timeout, not to cause long-running child processes (in the demo, I use a timeout of 1 minute). Here, to keep the demo simple, a timeout value is fixed and settled in the script function, but that can get transformed into an externalized configurable parameter of the iFlow or even get submitted by the caller as an additional parameter to allow the caller adjust timeout depending on the command they submit for execution.
To extend an area of application of the iFlow, let's adjust it in such a way that the shell command is submitted in the URL query string, in the query parameter (I will use the query parameter
command for this). A reason behind using a query parameter instead of a message body is because it enables us to use HTTP GET requests to trigger the iFlow – which means, the setup on a client side will not require any additional API testing tools such as Postman, and we will be able to use a web browser to trigger the iFlow. In other words, we get a browser-based method to execute shell commands on the CPI runtime node.A corresponding Groovy function that implements described adjustments and enhancements and that replaces the previously used implementation in the Groovy script step of the iFlow, is provided below:
import com.sap.gateway.ip.core.customdev.util.Message
import java.nio.charset.StandardCharsets
Message processData(Message message) {
final String QUERY_PARAM_COMMAND = 'command'
final long COMMAND_EXEC_TIMEOUT_MS = 60000
String httpQuery = message.headers['CamelHttpQuery'] as String
if (httpQuery) {
httpQuery = URLDecoder.decode(httpQuery, StandardCharsets.UTF_8.name())
String command = httpQuery.tokenize('&').find { it.tokenize('=')[0] == QUERY_PARAM_COMMAND }?.minus("$QUERY_PARAM_COMMAND=")
if (command) {
try {
Process process = command.execute()
process.waitForOrKill(COMMAND_EXEC_TIMEOUT_MS)
message.body = (process.exitValue()) ? process.err.text : process.in.text
} catch (IOException | IllegalThreadStateException e) {
message.body = "Error while executing command\nCommand: ${command}\nError: ${e.message}\n"
}
} else {
message.body = "Error: Command has not been provided, check query parameter '${QUERY_PARAM_COMMAND}'\n"
}
} else {
message.body = "Error: Command has not been provided, check query parameter '${QUERY_PARAM_COMMAND}'\n"
}
return message
}We can still test the iFlow using Postman – or we can use a web browser for testing purposes:
With this, we conclude iFlow enhancements and leave it for now. We were successful to trigger it and execute shell commands on the CPI runtime node and receive results of commands execution back using HTTP clients with a graphical user interface – Postman and a web browser. But we aren't done yet with the client side of the demo: in the introduction, I mentioned we were aiming terminal-like experience when executing shell commands on the CPI runtime node. Using a Linux shell on a local machine to trigger execution of familiar shell commands on a remote Linux server (the CPI runtime node) – that might happen to be convenient. Let's get to the final element of the demo to settle that.
From GUI to CLI on the client-side: adding terminal to the mix
The developed iFlow exposes an HTTP endpoint, so we can send requests to it using some command line tool – here, I’m going to use cURL. For example, one of shell commands that we already observed earlier, can be triggered using cURL in the following way:
To improve usability of this approach and make it more user-friendly, let's create a Bash script and use it instead – the script addresses following aspects:
- Environment specific parameters are externalized and maintained in a configuration file that is used by the script. The file contains a list of key-value pairs for corresponding required parameters that are used to access the CPI runtime node and trigger the iFlow – in particular, a CPI runtime node address, a path to the iFlow endpoint, credentials.
- The shell command is passed as an argument to the script.
- After some basic input validations, the script uses environment specific parameters retrieved from the configuration file and the shell command submitted in the argument, to compose a request that is sent to the CPI runtime node using cURL.
- Outcome of cURL execution is issued to the standard output – this will contain a response produced by the iFlow and any errors that cURL might encounter or receive while calling the iFlow endpoint.
The script implements some basic input validations, but those checks are only to ensure that mandatory parameters aren't missing. The script provides no additional configuration (for example, usage of environment variables to specify location of the configuration file, verbose/silent modes) and doesn't implement any sophisticated input checks (for example, handling of additional arguments, parsing of the configuration file content and protection against malicious commands that can be contained in it) or advanced error handling, so please consider it as a simplified proof of concept used for demonstration purposes, and not as a production-ready reference example.
The script is provided below:
#!/bin/bash
config_file=~/.config/sap-cpi-env.conf
mandatory_params=("runtime_node" "api_path" "username" "password")
command=$1
[[ -z "$command" ]] && echo "Command is missing, check command line arguments" && exit 1
[[ ! -f $config_file ]] && echo "Configuration file $config_file does not exist" && exit 1
echo "Configuration file: $(realpath $config_file)"
. $config_file
missing_params_count=0
for param in "${mandatory_params[@]}"
do
[[ -z "${!param}" ]] && echo "Parameter $param is missing" && (( missing_params_count++ ))
done
[[ "$missing_params_count" -gt 0 ]] && echo "$missing_params_count mandatory parameter(s) is(are) missing, check configuration file" && exit 1
endpoint="https://$runtime_node/http$api_path"
echo "API endpoint: $endpoint"
echo "Command: $command"
echo -e "\n--------------------------------------------------"
curl --user $username:$password \
--get --data-urlencode "command=$command" \
$endpoint
echo -e "\n--------------------------------------------------"I placed the configuration file
sap-cpi-env.conf in the .config directory that can be found in the user's home directory. Location of the configuration file is maintained in the script (it can be replaced in the script or it can be alternatively externalized and stored in the environment variable), and sample content of the configuration file is provided below (username and password were replaced with placeholders here - in the configuration file, these parameters shall refer to valid credentials):runtime_node="22f65e1ftrial.it-cpitrial-rt.cfapps.us10.hana.ondemand.com"
api_path="/demo/shell-cmd"
username="{username}"
password="{password}"Next, to provide a simpler way of executing the script, I enabled a Linux shell to find this script when searching for executables (ensure that the directory where the script is located, is listed in the environment variable
$PATH) and made the script executable (for example, using chmod +x {script file name}).We are all set to go now. Let's now see the entire end-to-end demo in action and run a final series of tests – this time, with some more shell commands.
As it was in previous examples, let's start the final part of the demo from executing the already used command to get information about the operating system and its version using the command
lsb_release:Some more examples
More details about the system using the command
uname:Next, let's list available file systems using the command
df:We can also get familiar with information about the CPU using the command
lscpu:Let's check free and used memory using the command
free:Next, we take a glance at current processes using the command
ps:We can also browse content of directories using the command
ls:And as a final example for now, let's browse content of some files using commands
cat and tail:Outro
Above I executed and demonstrated very few examples of commands that help us become more familiar with the runtime environment where the CPI runtime node runs – obviously, there is a lot more to explore as we have just scratched the surface with these very few commands. I hope this already provides a glimpse of the technique and its usage.
The technique that was described in this blog post, is very powerful and allows to access and execute a lot of shell commands on the CPI runtime that can help browse directories, access content of files, get information about running processes, runtime environment workload and configuration, just to name a few. It shall be noted that since the user context which is used when creating and running such child processes in CPI is privileged, the technique shall be used mindfully and responsibly. Let me remind you here that if you get excited about applying this knowledge to practice in CPI tenants, please assess risks associated with executed commands carefully and do it at your own risk.
- SAP Managed Tags:
5 Comments
