Developing Secure Applications in a Multicloud Environment: Python Code Sample Appendix
|This blog post series is about developing applications in a multicloud environment.
Any good? Post a comment, share on social media, and/or give a like. Thanks!
Developing Secure Applications on the SAP Cloud Platform
In this blog series, we will explore developing secure applications in a multi-cloud Cloud Foundry environment.
Think Global, Act Local
We start our project with the most basic Python web application.
Create a directory. for example, “sample” and save this code snippet as hello.py.
from flask import Flask app = Flask(__name__) @app.route('/') def hello(): return "Hello World"
We will use Flask to create us a web application. Flask is a micro web framework written in Python. For more information and a quick start, see flask.palletsprojects.com/en/1.1.x/quickstart/.
When we access the URL root (‘/’), the web server will return a string of text: feel free to ajdust.
Create Virtual Env
Before we deploy our application to the cloud, it can be helpful to first test it locally but this does mean we need to install some software. Feel free to skip this step and go to the next section.
For your OS, install Python, pip, and Flask. Flask comes with its own dependencies, like Jinja and Werkzeug. See installation for Flask platform instructions for Linux/macOS and Windows.
# Linux/macOS python3 -m venv sample source sample/bin/activate
# Windows py -m venv sample .\sample\Scripts\activate
The venv command creates a virtual environment (here named sample but you can use any name your want).
Next we run the Python package manager pip command to upgrade to the latest version as a best practice and then install Flask. For the ins and outs about pip, see the pip documentation)
pip install --upgrade pip pip install Flask
Flask comes with its own dependencies which are automatically installed: courtesy of pip.
Command venv has created the sample directory with bin, include, and lib. Command pip has installed the packages under lib.
Once we run the program, the executable bytecode will be created in __pycache__.
There are different ways how we can run our program. Define the FLASK_APP environment variable (use SET on Windows) and execute flask run.
export FLASK_APP=hello flask run
Note that without further instructions Flask runs on the loopback adapter (localhost at 127.0.0.1) and hence only accessible from your computer (might be a good idea at this stage).
We can pass the host parameter to specify a specific network interface (or all NICs with 0.0.0.0). We can pass command and variable as a single statement as well.
FLASK_APP=hello flask run --host=0.0.0.0 -p 1234
We can pass multiple environment variables with the command or, alternatively, store them in a .flaskenv file (although this requires python-dotenv to be installed).
Alternatively, we can also add host and port to hello.py and run the command using the python command.
When you are done with local development and testing, exit the environment with command:
SAP Cloud Foundry
For this tutorial we will be using the Cloud Foundry command-line interface, cf CLI, which you can install from GitHub: github.com/cloudfoundry/cli/blob/master/README.md.
We also assume that you have a trial or enterprise account on the SAP Cloud Platform. If not, see SAP Cloud Platform Developer Onboarding | Hands-on Video Tutorials for how to get started.
We continue with the server.py sample code from SAP Cloud Platform documentation:
The code is very similar to hello.py above. The main difference is that a port variable is defined using os.environ.get and for this we need to add an import statement. This would typically be used in a try block: see if it is free and if so, assign it. The if __name__ == line is a bit of boilerplate to indicate where our program starts but like the port assignment not really necessary for our sample app.
import os from flask import Flask app = Flask(__name__) port = int(os.environ.get('PORT', 3000)) @app.route('/') def hello(): return "Hello World" if __name__ == '__main__': app.run(host='0.0.0.0', port=port)
Push an App
To run the app on the SAP Cloud Platform, connect to your Cloud Foundry (trial) subaccount, org, and space.
cf api cf login | cf l cf target | cf t
To deploy the application to Cloud Foundry, we use the cf push command.
When we run this command as-is, without argument, the execution will fail with an incorrect usage message: we need to provide the app name or use a manifest.yml file.
When we run the command with only a name a staging error is returned:
No start command specified by buildpack or via Procfile.
App will not start unless a command is provided at runtime
According to the Cloud Foundry documentation, section Python Buildpack, the Python buildpack is used if a requirements.txt or setup.py file is detected in the root directory of the project.
Without these files, CF had no idea which runtime environment to provide.
Changing the file name server.py to setup.py solves the buildpack issue, however the script cannot be executed because the the Flask module is not found.
Application dependencies for a Python program are defined in the file requirements.txt. Unless a specific version is required, just the name suffices. This also applies to the dependencies of the dependency, Flask in our case. W could have specified all dependencies but could also leave this up to pip, the package manager.
# Flask==1.0.x Flask
When we run cf push myapp again we can see that the Flask (and its dependencies) are installed.
cf push myapp -c "python -m server"
The app is created with name = myapp, path = current directory; route = appname + cfapps + API domain.
Staging and Running Apps
The file is uploaded and the staging prepared by downloading the buildpacks.
First a container is created by a cell. The Python buildpack is downloaded and Python and pip are installed. Pip installs Flask and dependencies. The result is uploaded as droplet and the container stopped and destroyed.
The pesky warnings are unfortunate. These are caused by the buildpack. To get rid of them we need to create our own buildpack, a topic we return to below.
The illustration below shows the cf push process. A Diego cell is a virtual machine. One is used to stage the app and create the droplet using a container (first created, then destroyed). Another to run the app. For the details, see How applications are staged.
Side Note: Cloud Foundry was originally developped in Ruby. The component responsible for running the droplets was the Droplet Execution Agent (DEA). When Cloud Foundry was re-architected and rewritten in Go, DEA became Diego.
The same information is provided by the SAP Cloud Platform Cockpit. When we click the application route we are directed to our app.
The route needs to be unique in our domain and in shared environments the myapp route may already have been taken. We fix this below.
Note that our simple Hello World app allocates a fair amount of resources: 1024 MB memory and disk quota while only a fraction is needed.
Note that our basic app has allocated all available memory 1024 MB while only 15.9 is needed. The same applies to the disk quota.
Attributes and Manifests
We can control memory and disk allocation with the attributes -m and -k.
Random route generates a unique name for our app.
cf delete myapp cf push myapp -m 128M -k 256M --random-route -c "python server.py"
The update the app, we can simply run the cf push command again.
Should you want to clean up first, use the cf delete command.
The random route generated this time is myapp-execellent-warthog-by but it will be different when we delete and push.
Note that when you delete before push you will eventually run out of your random routes quota. To reset the counter, use command
As the cf CLI already indicated, instead of passing attributes on the command line we can also use a manifest.yml file.
Side Note: Initially the ML from YML was the same ML as in XML and HTML: Yet Another Markup Language (going back to the early days of Yahoo). Later the creators changed their mind and now it stands for YAML Ain’t Markup Language, a recursive acronym which only makes sense to very few people. However, the statement is correct. YAML is not a markup language but a configuration file language and one that is, like Python, senstive to identation. In other words, spaces have meaning.
When we enter our attributes in a manifest.yml file we need to do it likes this and exactly like this (although we can move the optional attributes around). For the full specs, see the App Manifest Attribute Reference.
--- applications: - name: myapp buildpacks: - python_buildpack path: . memory: 128M disk_quota: 256M random-route: true command: python server.py
Should the route be fixed, you can use the host attribute instead of random-route. This will need to be unique for the domain, as mentioned.
Path points to either the directory where the code is located or to a ZIP or JAR file with the code. As we have already seen, without this parameter the current directory is assumed; the . (dot) that is. In the manifest file above the path attribute is included but superflous as it contains the default value.
To try out the zip format, compress server.py and requirements.txt into an archive and provide this as path. For larger projects, this shortens the upload time.
The CLI looks for a manifest.yml in the current directory. When stored elsewhere, use the attribute -f with the path to the manifest is.
Courtesy of SAP
In the manifest above we explicitly set the buildpack we want to use. Cloud Foundry uses a buildpack to create a droplet (tarball or zipped archived stored in a blobstore), which is later used to run the app.
The cf buildpacks command lists all available buildpacks provided by SAP.
Latest and Greatest (?)
--- applications: - name: myapp buildpacks: - https://github.com/cloudfoundry/python-buildpack.git path: . memory: 128M disk_quota: 256MB random-route: true command: python server.py
The console output below shows a Cloud Foundry cell creating a container and download Go to build the app, then the Python Buildpack version 1.7.21. The version provided by SAP Cloud Platform is currently 1.7.17 (see output above).
The rest is the same: ‘go build’ installs python, pip-pop ( to manage requirements.txt files) and then runs the Python package manager pip to install Flask, etc.
A buildpack might contain several runtime versions. For example, buildpack 1.7.21 contains ten Python binaries from 3.5.9 (lowest) to 3.8.5 (highest). To specify the exact version to run, add a runtime.txt file to your project.
For Python buildpack release information, see github.com/cloudfoundry/python-buildpack/releases.
When we run our app locally, besides python server.py we showed that we could also start our app using an environment variable export FLASK_APP=hello.py with command python -m flask run.
Port numbers are configuration and configuration should not be in code. Let’s simplify our web app by removing the port.
Save as web.py.
from flask import Flask app = Flask(__name__) @app.route('/') def hello_world(): return 'Hello, World! (again)'
We can now remove the command from the manifest and add it to a file named Procfile (no extension).
web: FLASK_APP=web.py python3 -m flask run --host=0.0.0.0 --port=$PORT
When a Procfile is detected, the command attribute is ignored.
--- applications: - name: myapp buildpacks: - python_buildpack path: . memory: 128M disk_quota: 256MB random-route: true
User Provided Variables
Alternatively, we could have used the env attribute in the manifest for FLASK_APP instead of Procfile.
--- applications: - name: myapp ... command: flask run env: FLASK_APP: web.py
cf push -f manifest.yml -i 3
Docker (Bonus Track)
Docker and Diego
As we have seen, Cloud Foundry runs (Garden) containers inside virtual machines (cells). If you wish, you can also run Docker containers in a Diego cell. This is not relevant for our scenario and would require additional configuration to make this work with service instances (SAP HANA Cloud and XSUAA) but serves to clarify the inner workings of Diego: just another VM+Container architecture.
To test this out you need a local Docker installation, see www.docker.com for the downloads. We are going to assume you have done the Docker 101 tutorials and have fiddled with images and containers a bit and have an account on the Docker Hub.
For a step-by-step, see Deploy Application using Docker Container on SAP Cloud Foundry : 2020 by Kapil Verma.
First, ceate a Docker directory with a Dockerfile file.
# set base image (host OS) FROM python:3.8 # set the working directory in the container WORKDIR /app # copy the dependencies file to the working directory COPY requirements.txt . # install dependencies RUN pip install -r requirements.txt # copy the content of the local src directory to the working directory COPY src/ . # command to run on container start CMD [ "python", "./server.py" ]
We are using the same requirements.txt file as before and the orignal server.py file. To separate the configuration files from the source code, create a src file and move the server.py file.
Build the container with command
docker build -t python-tutorial .
This will build an image tagged as python-tutorial
- Download the Python image
- Change to the work directory /app
- Copy requirements.txt
- Run pip to install Flask and its dependencies
- Copy the source code (server.py)
- Run the command
To run the container locally, use command:
docker run -d -p 3000:3000 python-tutorial
Into the Cloud
Now that we have validated that our local container is working, time to upload the image to Docker Hub. For tag the image and push it to Docker Hub. Of course, Docker Hub is just one choice. Alternatively, you can use Azure, Google Cloud, AWS, etc. to store the image. For the documentation, see Deploying an App with Docker.
docker login --username=<your user> # note image ID docker images # tag the image <username>/<container>:<tag> docker tag c8d87c346a6a dvkempen/python-tutorial:cloudfoundry # push the container to the hub docker push dvkempen/python-tutorial
With the container already in the cloud, it is a small step for Cloud Foundry to host it. We only need to provide our APPNAME and docker image with credentials. Optionally, again, use a manifest to control memory, disk, and other attributes. This works the same as with the apps. As documented: push – Cloud Foundry CLI Reference Guide
cf push myapp --docker-image dvkempen/python-tutorial:cloudfoundry --docker-username dvkempen
As indicated but the CLI output, you can set a local environment variable CF_DOCKER_PASSWORD for the password.
Note that the staging phase is now much shorter as we no longer need to build the image.
Share and Connect
Enjoyed the blog? Post a comment, share on social media, and/or give a like. Thanks!
If you would like to receive updates, connect with me on
Denys van Kempen