Cloud-Native Lab #1 - 7 Ways to Define Environment Variables

[mariusobert]

In this first Cloud-Native Lab post, I'll show you seven different ways to define environment variables in the Cloud Foundry environment. Followed by this, I'll compare these approaches so you can find the one that suits your needs the most. In the end, I'll write a web app, using the UI5 web components, that lists all variables in a Fiori-style.

This post covers the role of environment variables in a cloud-native world and primarily how they are used in the SAP Cloud Platform. As environment variables are such a well-known concept, there are many ways to define them. I will explain the various techniques I personally use and compare them to each other to identify which options are suited for which use-cases.

UI5 Web Components list that visualizes environment variables

The veterans of operating systems

I assume there is no need to explain environment variables in-depth as most of you had to deal with them in one form or another already. So I'll just leave you with the first sentence from the Wikipedia article here:

An environment variable is a dynamic-named value that can affect the way running processes will behave on a computer. They are part of the environment in which a process runs.

I think this description fits it pretty well. I'm sure every developer already worked with the infamous path variable. The concept of environment variables has been introduced to the Unix-World in 1979 and shortly after also to the Windows-World. And while the idea is quite old, it is far from retirement. I would actually say that environment variables are going through their second adolescence.

I often hear that people refer to cloud-native platforms like Cloud Foundry and Kubernetes as cloud-native OS - and there is something to this. These new platforms rely on environment variables the same as traditional OS did. In the cloud-native-area where everything is containerized, environment variables are frequently used to inject configuration into the containers. They could declare connected services or contain information about the role or task of the container in a larger cluster. An example of the first one could that software modules check the environment variables for service credentials by default when the developer didn't enter them explicitly - as the GCP BigQuery client does. An example of the latter one is how CI/CD tools like CircleCI use them to pass the information on how many containers run in parallel. This lets each container know how many helpers there are to divide the workload among them.

One single (base-)container is often for many scenarios, which reduce the overhead of managing various containers with different configurations. It's fair to say that environment variables empower containers to be more reusable and therefore enable the entire cloud-native-mindset.

Environment Variables in SAP Cloud Platform

Environment variables in SAP Cloud Platform are, same as on other cloud platforms, used for configuration and binding of microservices. The Cloud Foundry environment creates new environment variables every time a service instance is bound to a microservice to inject the credentials into the app. Many cloud modules from SAP read these variables during startup and interpret them as the configuration.

The package that I work the most with is the Application Router. It uses environment variables for many configuration dimensions, whereby the most famous one is the configuration of the destination. You might know that destinations can be configured either via the cockpit, the service instance, or an environment variable. When the approuter is looking for a new configuration, it checks these three configurations one-by-one to find the right destination. In another configuration dimension, the approuter will search for the credentials of bound services to forward traffic to.

Another example of a package that relies on environment variables is xsenv. This utility package helps to load and read application configurations for bound services in the SAP Cloud Platform Cloud Foundry environment, SAP XS advanced model, and Kubernetes (K8S). On npmjs.org, you can see that almost all other npm packages by SAP depend on this single module. Therefore, it's no surprise that xsenv is the most popular npm package by SAP. xsenv combined the purpose of tools like dotenv and cfenv in one single tool that is already part of most node-based SAP projects. So there is no need to use additional packages, which would increase the size of your node_modules folder.

Defining Environment Variables

In this section, I want to explain a few ways how you can define environment variables. I'm sure this list is far from complete, but I think I mentioned the most common and most useful ones. In case I missed an approach, please let me know in the comments.

Shell variables

Using the native OS command is probably the most common approach:

set WindowsVariable=SOMETHING

#or

export UnixVariable=SOMETHING

Depending on your OS and the used shell, these commands may vary. In general, the variables that are defined like this are scoped for all processes started from the current shell. This approach can be combined with a shell profile to make the variables available permanently. Environment variables that are defined like this are only available on the local machine where they have been declared and are not persisted with git.

Process variables

You can also define environment variables only for a given process. For this, you need to prefix the command with the declaration.

UnixVariable=SOMETHING node index.js

Afaik, this only works in Unix environments. The variables are only available on the machine during the execution of the command. It is not possible to store the values with a version control system like git.

Package-based

The previous categories were OS-specific but independent of the used runtime platform. This category only applies to Node.js, but I'm sure there are similar ways to achieve the same result with Java or Python.

1. You can use packages like xsenv or dotenv to load environment variables from a text file when the application is started. This gives you the freedom to define variables with longer content like JavaScript arrays or objects. It makes sense to prefer xsenv over dotenv when you work in the SAP-context as the package is most-likely already used in your project.


2. Another useful package is cross-env, which can be used to bring the command-based approach from above to Windows systems too. This can be used to define environment variables that contain arbitrary Unicode-strings whereby long variables can be hard to read as this approach usually only uses a single line.
   

   {
   
     "scripts": {
   
       "start": "cross-env TS_NODE_COMPILER_OPTIONS={\\\"module\\\":\\\"commonjs\\\"} node some_file.test.ts"
   
     },
   
   }​

   
   

Both packages allow the usage of Unicode characters for the variables and are written to text files. This has the advantage that these environment variables can be stored in a version control system, and therefore, the variables can easily be shared among team members/a community. As a result, the variables are also available once the project has been deployed to another (cloud-based) environment.

Target Platform-based

The previous approaches always applied to the environment variables of the local machine of the developer. In this category, I will show you ways to define an environment variable that is ONLY available in the cloud environment. These instructions work on Cloud Foundry-based platforms only, but there are similar concepts for other runtime environments too.

1. You can define environment variables for a specific app in the app manifest (manifest.yaml). This works on all Cloud Foundry platforms.
   

   ---
   
   applications:
   
   - name: my-app
   
     command: node my-app.js
   
     env:
   
       CF_ENV: production

   
   These variables can also contain any Unicode character and be persisted with tools like git.
   
    


2. As you might know, SAP Cloud Foundry has a more powerful alternative to the app manifest, the MTA descriptor (mta.yaml). This file also has a particular property you can use to define environment levels for each application.
   

   ID: a.cf.app
   
   _schema-version: 3.3.0
   
   version: 0.0.0
   
   
   
   modules:
   
     #A cf app consuming the configuration
   
     - name: my-mta-managed-app-module
   
       type: application # value custom would do just as well
   
       path: "appBits.zip"
   
       properties: #module properties for CF Apps can be consumed as app environment variables at runitime
   
         MY_CF_APP_ENV_VARIABLE: "HELLO MTA"
   
         A_MORE_COMPLEX_VAR: "{
   
           \"can be an entire\":\"json object\",
   
           \"with\": [
   
             \"nested\": {
   
               \"elements\": \"${default-url}\"
   
               }
   
             ]"

   
   Have a look at this repository to find more mta examples.
   
    


3. You can also use a command-based approach to inject variables that shouldn't be persisted with the source code. Don't forget the restage the app once you added/changed an environment variable.
   

   cf set-env my-app CF_ENV production
   
   cf restage my-app​

   
   This approach will also work on any Cloud Foundry distribution.
   
   You don't necessarily need the CLI to add this type of variable. In the SAP Cloud Platform Cockpit, you can navigate to your app, select User-Provided Variables and Add Variable to add a new one. Both approaches are temporary, e.g the variables will be gone when the app is deployed another time.

 

Comparison

This table might suggest that tools like xsenv are the best way to load environment variables, but this doesn't mean they are suited for all use-cases. It could make sense to use another approach if certain environment variables should only be available in one runtime. Similarly, you might want to prevent that certain variables like credentials are saved in git and should not be shared among developers.

So you might also end up mixing different approaches in your projects.

	Platform	Scope	Trackable with git
Shell variables	local	shell/terminal	No
process variables	local	process (temporary)	No
cross-var	local and cloud	process	Yes
xsenv/dotenv	local and cloud	process	Yes
MTA.yaml	cloud	container	Yes
manifest.yaml	cloud	container	Yes
set set-env/cockpit	cloud	container (temporary)	No

Caution: Variables with the same name will override each other if multiple approaches are used in parallel. It is not possible to "merge" two values of the same variable.

Hands-on: Visualize Environment Variables with UI5 web components

To keep the hands-on simple, we're just going to define environment variables that don't do anything - they are just there. To visualize them, will build a simple server that exposes all defined variables in a list. As plain HTML5 lists (<ul>) are dull and ugly, we'll use UI5 web components to style the items familiarly.

0. Preparation

Before we get to the fun part, we need to install some tools which are mandatory for cloud development on SAP Cloud Platform (if you haven't done so already):

• CF CLI


• CF CLI MTA Plugin


• Install Node.js

1. Start the project

Create a package.json file to start the project. This file defines the used dependencies and the commands to build and start the project.

Express is used for the server, and the other dependencies, xsenv & cross-var, are used to define different environment variables. The other devDependecies are needed for the UI5 web components.

{

  "name": "cloud-native_lab_1",

  "scripts": {

    "build": "parcel build index.html",

    "start": "cross-env prefixVariable1= node index.js"

  },

  "dependencies": {

    "@sap/xsenv": "^3.0.0",

    "cross-env": "^7.0.2",

    "express": "^4.17.1"

  },

  "devDependencies": {

    "@babel/core": "7.10.5",

    "@ui5/webcomponents": "1.0.0-rc.7",

    "@ui5/webcomponents-fiori": "^1.0.0-rc.7",

    "parcel-bundler": "^1.12.4"

  }

}

2. Server exposing variables

Create a file with the name index.js and the following content.

const express = require("express");

const app = express();



require("@sap/xsenv").loadEnv();

app.get("/vars.json", (req, res) => {

  const vars = Object.keys(process.env).filter((name) => name.includes("Variable"));

  res.json(vars.map(prop => {

    return {

      name: prop,

      value: process.env[prop]

    }

  }));

});



app.use(express.static("dist"));



app.listen(8080, () => console.log("Example app listening on port 8080."));

This server will load and expose the variables and serve static content from the dist directory. Don't worry if you don't have such a directory yet, it will be created automatically during the build step of the web app.

Create a default-env.json file to define the variables that are loaded in line 4 of the server.

{

  "loadedVariable": ""

}

3. Web App using UI5 Web Components

Create an index.html page that contains the following markup. Can you spot the UI5 web components here?

<!DOCTYPE html>

<html lang="en">



<head>

  <title>UI5 Web Components</title>

  <meta charset="UTF-8" />

</head>

<script src="src/index.js"></script>



<body style="background-color: var(--sapBackgroundColor);">

  <ui5-shellbar primary-title="Environment Variables powered by UI5 Web Components"

    logo="https://www.sap.com/dam/application/shared/logos/sap-logo-svg.svg.adapt.svg/1493030643828.svg"></ui5-shellbar>

  <div style="max-width: 80rem; margin: 25px auto auto auto;">

    <ui5-title>Found Variables:</ui5-title>

    <br />

    <ui5-list id="myList" />

  </div>



</body>



</html>

This is almost all you need to do to create a simple website that leverages the UI5 web components. The only missing part is the necessary imports and the logic to render the list items manually. Create the srv/index.js file to take care of that.

import "@ui5/webcomponents/dist/List.js";

import "@ui5/webcomponents/dist/Title.js";

import "@ui5/webcomponents-fiori/dist/ShellBar";

import "@ui5/webcomponents/dist/StandardListItem.js";



fetch('/vars.json')

  .then(response => response.json())

  .then(variables => {

    const list = document.getElementById("myList");



    variables.forEach(variable => {

      const item = document.createElement("ui5-li");

      item.setAttribute("description", variable.value);

      item.innerText = variable.name;



      list.appendChild(item);

    });

  });

4. Run locally

Use npm (or alternatively yarn) to install the dependencies, build the static resources, and eventually start the application.

npm install

npm run build



# Unix

export exportedVariable=3   

prefixVariable2=2 npm start



# Windows

set exportedVariable 3   

npm start

Open http://localhost:8080/ to see the frontend in action. It should look similar to this.

Available environment variables in a local setup on a Unix system

5. Make deployable

Create the mta.yaml descriptor to define all parameters that are needed for the next step. This includes the runtime configuration of the platform as well as the build parameters to make sure the project is built before it's packaged.

ID: variables

_schema-version: 3.2.0

version: 1.0.0



modules:

  - name: variables

    type: nodejs

    path: .

    parameters:

      command: prefixVariable2= yarn start

      disk-quota: 256M

      memory: 128M

    properties:

      mtaVariable: "0"

    build-parameters:

      ignore: ["node_modules/"]

      builder: custom

      commands:

        - yarn install

        - yarn run build

6. Deploy

Run the following commands from the project root to deploy the app.

npx mbt build

cf deploy mta_archives/variables_1.0.0.mtar

Once the deployment has finished, you will find the URL of the app in the console output. Open the web app; you should see this result now.



You made it! You can see that you see different variables here. The "local variables" are gone, but you see additional "cloud variables" now,

Summary

In this edition, you have learned:

• About the importance of environment variables in the cloud area


• 7 options to define environment variables to choose from


• That environment variables can override each other when mixing approaches


• About SAP's npm packages and how they use environment variables


• How to build a web app using UI5 Web Components

Next Steps

• Learn more about SAP's npm packages (now on the public npm registry!)


• Learn more about the Application Router module specially


• Think about the explained options and find your favorite

About this new series

I prepared a short FAQ for you as you might have some questions about my new (and the old) post series.

Q: Why does CloudFoundryFun end and Cloud-native Lab start?

A: Lately, I often wanted to write about topics that would fulfill the core "Fun" idea of CFF, but it was not directly related to the Cloud Foundry environment, which is why I discarded the topics. With this new chapter, I can include all topics that are of interest.

Q: Does this mean that CF will be deprecated?

A: Absolutely not. CF is still the central platform for the extension and integration scenario of the SAP Cloud Platform.

Q: Does this mean CFF/CNL won't be about Cloud Foundry in the future?

A: Also, a definite "No" to this one. As you can see, this post is also about Cloud Foundry. It's just a name change for a new chapter. This will give me the chance to write about other cloud-native technologies from SAP, such as Gardener or Kyma, every now and then.

This was the first blog post of my bi-monthly series #CloudNativeLab. The name already says all there is to it; This series won't be about building pure business apps with cloud-native technology. I think there are already plenty of great posts about those aspects out there. Instead, this series rather thinks outside the box and demonstrates unconventional use-cases of cloud-native technology such as Cloud Foundry, Kyma, Gardener, etc.

Previous chapter/episode: CloudFoundryFun #12 – Create a tiny CAP project

Next episode: Cloud-Native Lab #2 – Comparing Cloud Foundry and Kyma Manifests