Technical Articles
Cloud-Native Lab #1 – 7 Ways to Define Environment Variables
In this first Cloud-Native Lab post, I’ll show you seven different ways to define environment variables in the Cloud Foundry environment. Followed by this, I’ll compare these approaches so you can find the one that suits your needs the most. In the end, I’ll write a web app, using the UI5 web components, that lists all variables in a Fiori-style.
This post covers the role of environment variables in a cloud-native world and primarily how they are used in the SAP Cloud Platform. As environment variables are such a well-known concept, there are many ways to define them. I will explain the various techniques I personally use and compare them to each other to identify which options are suited for which use-cases.
UI5 Web Components list that visualizes environment variables
The veterans of operating systems
I assume there is no need to explain environment variables in-depth as most of you had to deal with them in one form or another already. So I’ll just leave you with the first sentence from the Wikipedia article here:
An environment variable is a dynamic-named value that can affect the way running processes will behave on a computer. They are part of the environment in which a process runs.
I think this description fits it pretty well. I’m sure every developer already worked with the infamous path variable. The concept of environment variables has been introduced to the Unix-World in 1979 and shortly after also to the Windows-World. And while the idea is quite old, it is far from retirement. I would actually say that environment variables are going through their second adolescence.
I often hear that people refer to cloud-native platforms like Cloud Foundry and Kubernetes as cloud-native OS – and there is something to this. These new platforms rely on environment variables the same as traditional OS did. In the cloud-native-area where everything is containerized, environment variables are frequently used to inject configuration into the containers. They could declare connected services or contain information about the role or task of the container in a larger cluster. An example of the first one could that software modules check the environment variables for service credentials by default when the developer didn’t enter them explicitly – as the GCP BigQuery client does. An example of the latter one is how CI/CD tools like CircleCI use them to pass the information on how many containers run in parallel. This lets each container know how many helpers there are to divide the workload among them.
One single (base-)container is often for many scenarios, which reduce the overhead of managing various containers with different configurations. It’s fair to say that environment variables empower containers to be more reusable and therefore enable the entire cloud-native-mindset.
Environment Variables in SAP Cloud Platform
Environment variables in SAP Cloud Platform are, same as on other cloud platforms, used for configuration and binding of microservices. The Cloud Foundry environment creates new environment variables every time a service instance is bound to a microservice to inject the credentials into the app. Many cloud modules from SAP read these variables during startup and interpret them as the configuration.
The package that I work the most with is the Application Router. It uses environment variables for many configuration dimensions, whereby the most famous one is the configuration of the destination. You might know that destinations can be configured either via the cockpit, the service instance, or an environment variable. When the approuter is looking for a new configuration, it checks these three configurations one-by-one to find the right destination. In another configuration dimension, the approuter will search for the credentials of bound services to forward traffic to.
Another example of a package that relies on environment variables is xsenv. This utility package helps to load and read application configurations for bound services in the SAP Cloud Platform Cloud Foundry environment, SAP XS advanced model, and Kubernetes (K8S). On npmjs.org, you can see that almost all other npm packages by SAP depend on this single module. Therefore, it’s no surprise that xsenv is the most popular npm package by SAP. xsenv combined the purpose of tools like dotenv and cfenv in one single tool that is already part of most node-based SAP projects. So there is no need to use additional packages, which would increase the size of your node_modules folder.
Defining Environment Variables
In this section, I want to explain a few ways how you can define environment variables. I’m sure this list is far from complete, but I think I mentioned the most common and most useful ones. In case I missed an approach, please let me know in the comments.
Shell variables
Using the native OS command is probably the most common approach:
set WindowsVariable=SOMETHING
#or
export UnixVariable=SOMETHING
Depending on your OS and the used shell, these commands may vary. In general, the variables that are defined like this are scoped for all processes started from the current shell. This approach can be combined with a shell profile to make the variables available permanently. Environment variables that are defined like this are only available on the local machine where they have been declared and are not persisted with git.
Process variables
You can also define environment variables only for a given process. For this, you need to prefix the command with the declaration.
UnixVariable=SOMETHING node index.js
Afaik, this only works in Unix environments. The variables are only available on the machine during the execution of the command. It is not possible to store the values with a version control system like git.
Package-based
The previous categories were OS-specific but independent of the used runtime platform. This category only applies to Node.js, but I’m sure there are similar ways to achieve the same result with Java or Python.
- You can use packages like xsenv or dotenv to load environment variables from a text file when the application is started. This gives you the freedom to define variables with longer content like JavaScript arrays or objects. It makes sense to prefer xsenv over dotenv when you work in the SAP-context as the package is most-likely already used in your project.
- Another useful package is cross-env, which can be used to bring the command-based approach from above to Windows systems too. This can be used to define environment variables that contain arbitrary Unicode-strings whereby long variables can be hard to read as this approach usually only uses a single line.
{ "scripts": { "start": "cross-env TS_NODE_COMPILER_OPTIONS={\\\"module\\\":\\\"commonjs\\\"} node some_file.test.ts" }, }
Both packages allow the usage of Unicode characters for the variables and are written to text files. This has the advantage that these environment variables can be stored in a version control system, and therefore, the variables can easily be shared among team members/a community. As a result, the variables are also available once the project has been deployed to another (cloud-based) environment.
Target Platform-based
The previous approaches always applied to the environment variables of the local machine of the developer. In this category, I will show you ways to define an environment variable that is ONLY available in the cloud environment. These instructions work on Cloud Foundry-based platforms only, but there are similar concepts for other runtime environments too.
- You can define environment variables for a specific app in the app manifest (manifest.yaml). This works on all Cloud Foundry platforms.
--- applications: - name: my-app command: node my-app.js env: CF_ENV: production
These variables can also contain any Unicode character and be persisted with tools like git.
- As you might know, SAP Cloud Foundry has a more powerful alternative to the app manifest, the MTA descriptor (mta.yaml). This file also has a particular property you can use to define environment levels for each application.
ID: a.cf.app _schema-version: 3.3.0 version: 0.0.0 modules: #A cf app consuming the configuration - name: my-mta-managed-app-module type: application # value custom would do just as well path: "appBits.zip" properties: #module properties for CF Apps can be consumed as app environment variables at runitime MY_CF_APP_ENV_VARIABLE: "HELLO MTA" A_MORE_COMPLEX_VAR: "{ \"can be an entire\":\"json object\", \"with\": [ \"nested\": { \"elements\": \"${default-url}\" } ]"
Have a look at this repository to find more mta examples.
- You can also use a command-based approach to inject variables that shouldn’t be persisted with the source code. Don’t forget the restage the app once you added/changed an environment variable.
cf set-env my-app CF_ENV production cf restage my-app
This approach will also work on any Cloud Foundry distribution.
You don’t necessarily need the CLI to add this type of variable. In the SAP Cloud Platform Cockpit, you can navigate to your app, select User-Provided Variables and Add Variable to add a new one.
Both approaches are temporary, e.g the variables will be gone when the app is deployed another time.
Comparison
This table might suggest that tools like xsenv are the best way to load environment variables, but this doesn’t mean they are suited for all use-cases. It could make sense to use another approach if certain environment variables should only be available in one runtime. Similarly, you might want to prevent that certain variables like credentials are saved in git and should not be shared among developers.
So you might also end up mixing different approaches in your projects.
Platform | Scope | Trackable with git | |
---|---|---|---|
Shell variables | local | shell/terminal | No |
process variables | local | process (temporary) | No |
cross-var | local and cloud | process | Yes |
xsenv/dotenv | local and cloud | process | Yes |
MTA.yaml | cloud | container | Yes |
manifest.yaml | cloud | container | Yes |
set set-env/cockpit | cloud | container (temporary) | No |
Caution: Variables with the same name will override each other if multiple approaches are used in parallel. It is not possible to “merge” two values of the same variable.
Hands-on: Visualize Environment Variables with UI5 web components
To keep the hands-on simple, we’re just going to define environment variables that don’t do anything – they are just there. To visualize them, will build a simple server that exposes all defined variables in a list. As plain HTML5 lists (<ul>) are dull and ugly, we’ll use UI5 web components to style the items familiarly.
0. Preparation
Before we get to the fun part, we need to install some tools which are mandatory for cloud development on SAP Cloud Platform (if you haven’t done so already):
1. Start the project
Create a package.json file to start the project. This file defines the used dependencies and the commands to build and start the project.
Express is used for the server, and the other dependencies, xsenv & cross-var, are used to define different environment variables. The other devDependecies are needed for the UI5 web components.
{
"name": "cloud-native_lab_1",
"scripts": {
"build": "parcel build index.html",
"start": "cross-env prefixVariable1= node index.js"
},
"dependencies": {
"@sap/xsenv": "^3.0.0",
"cross-env": "^7.0.2",
"express": "^4.17.1"
},
"devDependencies": {
"@babel/core": "7.10.5",
"@ui5/webcomponents": "1.0.0-rc.7",
"@ui5/webcomponents-fiori": "^1.0.0-rc.7",
"parcel-bundler": "^1.12.4"
}
}
2. Server exposing variables
Create a file with the name index.js and the following content.
const express = require("express");
const app = express();
require("@sap/xsenv").loadEnv();
app.get("/vars.json", (req, res) => {
const vars = Object.keys(process.env).filter((name) => name.includes("Variable"));
res.json(vars.map(prop => {
return {
name: prop,
value: process.env[prop]
}
}));
});
app.use(express.static("dist"));
app.listen(8080, () => console.log("Example app listening on port 8080."));
This server will load and expose the variables and serve static content from the dist directory. Don’t worry if you don’t have such a directory yet, it will be created automatically during the build step of the web app.
Create a default-env.json file to define the variables that are loaded in line 4 of the server.
{
"loadedVariable": ""
}
3. Web App using UI5 Web Components
Create an index.html page that contains the following markup. Can you spot the UI5 web components here?
<!DOCTYPE html>
<html lang="en">
<head>
<title>UI5 Web Components</title>
<meta charset="UTF-8" />
</head>
<script src="src/index.js"></script>
<body style="background-color: var(--sapBackgroundColor);">
<ui5-shellbar primary-title="Environment Variables powered by UI5 Web Components"
logo="https://www.sap.com/dam/application/shared/logos/sap-logo-svg.svg.adapt.svg/1493030643828.svg"></ui5-shellbar>
<div style="max-width: 80rem; margin: 25px auto auto auto;">
<ui5-title>Found Variables:</ui5-title>
<br />
<ui5-list id="myList" />
</div>
</body>
</html>
This is almost all you need to do to create a simple website that leverages the UI5 web components. The only missing part is the necessary imports and the logic to render the list items manually. Create the srv/index.js file to take care of that.
import "@ui5/webcomponents/dist/List.js";
import "@ui5/webcomponents/dist/Title.js";
import "@ui5/webcomponents-fiori/dist/ShellBar";
import "@ui5/webcomponents/dist/StandardListItem.js";
fetch('/vars.json')
.then(response => response.json())
.then(variables => {
const list = document.getElementById("myList");
variables.forEach(variable => {
const item = document.createElement("ui5-li");
item.setAttribute("description", variable.value);
item.innerText = variable.name;
list.appendChild(item);
});
});
4. Run locally
Use npm (or alternatively yarn) to install the dependencies, build the static resources, and eventually start the application.
npm install
npm run build
# Unix
export exportedVariable=3
prefixVariable2=2 npm start
# Windows
set exportedVariable 3
npm start
Open http://localhost:8080/ to see the frontend in action. It should look similar to this.
Available environment variables in a local setup on a Unix system
5. Make deployable
Create the mta.yaml descriptor to define all parameters that are needed for the next step. This includes the runtime configuration of the platform as well as the build parameters to make sure the project is built before it’s packaged.
ID: variables
_schema-version: 3.2.0
version: 1.0.0
modules:
- name: variables
type: nodejs
path: .
parameters:
command: prefixVariable2= yarn start
disk-quota: 256M
memory: 128M
properties:
mtaVariable: "0"
build-parameters:
ignore: ["node_modules/"]
builder: custom
commands:
- yarn install
- yarn run build
6. Deploy
Run the following commands from the project root to deploy the app.
npx mbt build
cf deploy mta_archives/variables_1.0.0.mtar
Once the deployment has finished, you will find the URL of the app in the console output. Open the web app; you should see this result now.
You made it! You can see that you see different variables here. The “local variables” are gone, but you see additional “cloud variables” now,
Summary
In this edition, you have learned:
- About the importance of environment variables in the cloud area
- 7 options to define environment variables to choose from
- That environment variables can override each other when mixing approaches
- About SAP’s npm packages and how they use environment variables
- How to build a web app using UI5 Web Components
Next Steps
- Learn more about SAP’s npm packages (now on the public npm registry!)
- Learn more about the Application Router module specially
- Think about the explained options and find your favorite
About this new series
I prepared a short FAQ for you as you might have some questions about my new (and the old) post series.
Q: Why does CloudFoundryFun end and Cloud-native Lab start?
A: Lately, I often wanted to write about topics that would fulfill the core “Fun” idea of CFF, but it was not directly related to the Cloud Foundry environment, which is why I discarded the topics. With this new chapter, I can include all topics that are of interest.
Q: Does this mean that CF will be deprecated?
A: Absolutely not. CF is still the central platform for the extension and integration scenario of the SAP Cloud Platform.
Q: Does this mean CFF/CNL won’t be about Cloud Foundry in the future?
A: Also, a definite “No” to this one. As you can see, this post is also about Cloud Foundry. It’s just a name change for a new chapter. This will give me the chance to write about other cloud-native technologies from SAP, such as Gardener or Kyma, every now and then.
This was the first blog post of my bi-monthly series #CloudNativeLab. The name already says all there is to it; This series won’t be about building pure business apps with cloud-native technology. I think there are already plenty of great posts about those aspects out there. Instead, this series rather thinks outside the box and demonstrates unconventional use-cases of cloud-native technology such as Cloud Foundry, Kyma, Gardener, etc. |
Previous chapter/episode: CloudFoundryFun #12 – Create a tiny CAP project
Next episode: Cloud-Native Lab #2 – Comparing Cloud Foundry and Kyma Manifests
Hi Marius,
Nice blog as always!
One quick question, in the point "Target platform" do you think is a good idea to put all those variables in a MTA Extension descriptor? Is possible with the complex variable?
Thanks
Thanks, Javier.
This is a really good question! I guess you are asking because complex vars can make you mta file less readable and moving the variables to and extension descriptor could help you to get rid of the clutter, right? And I would absolutely agree with this statement. I think this would be a good way to solve the problem.
For completeness: There are also other ways to define variables that are only loaded on Cloud Foundry. E.g. you could add a "cloud-vars.json" file to the module and use xsenv to load it only when the production flag is set.
I think both approaches make sense, I would prefer the first one for simpler vars and the latter one for very complex ones.
yes you read my mind, my question is about the reason you mentioned.
What do you mean with add json file into the module? do you mean that you can add this json file in yaml file? or is a file that you define in your project and read with xsenv? I'm curious so please let me know 🙂
And other question that i have, what is the best way to store an environment variable that contains a sensitive information?
Thanks as always
Great!
So my suggestion would be that you have two JSON files, one that “all vars” and one only for the vars that should be used in production.
You can then make use of the “loadEnv” function of xsenv
So you could configure all vars as JSON files.
As you might have guessed, there are multiple ways to define vars with sensitive information. The simplest approach would be “cf set-env” from above, a more complex one would be the credentials store service.
Nice ! ?
Its perfectly clear to me and i’ll play with the credential store service to see how is works
Many thanks
Thank you Marius for this excellent blog post on Environment variables..
Nice blog post.
One question: You skipped the (for me) most obvious way to add a variable, using the Cloud Cockpit, navigating to your application - user-provided variables and just adding it there.
Doing it this way, would it remain there if I redeploy the app (using 'cf push') or are all variables cleared then?
Hi Mark,
a very good question! I forgot about this option and did not mention it. I added this option to the blog post above and edited the table to mark the options that only add "temporary" variables.
Under the hood, the cockpit is doing the same as the "cf set-env" (maybe it is even using this command) which means the variable will only be available temporarily until the next "cf push" or "cf deploy".
I hope this makes it more clear.
Hi Marius,
And assuming that I don't want to have backend component for such simple case. How to pass such variable to ui5 during build time? My starting command to run ui5 is "ui5 serve -o index.html". Is there a way to pass this var in similar way to "MY_ENV=test ui5 serve -o index.html" like it can be done for server side?
I'm not sure if I understand your question. When you start the application with "ui5 serve", you have a backend component.
And btw, you should only use "ui5 serve" during development and similar landscapes. This command shouldn't be used to actually host a UI5 app.