Developing Secure Applications in a Multicloud Environment: Python Code Sample Appendix

[dvankempen]

This blog post series is about developing applications in a multicloud environment.

1. Cloud Foundry, Neo, and multicloud environments


2. Cloud Foundry, UAA, and XSUAA


3. Deploying Business Logic apps


4. Service Instance (SAP HANA Cloud)


5. XSUAA for Authentication


6. XSUAA for Authorisation

Any good? Post a comment, share on social media, and/or give a like. Thanks!

Hands-On Tutorials

Developing Secure Applications on the SAP Cloud Platform

In this blog series, we will explore developing secure applications in a multi-cloud Cloud Foundry environment.

This blog provides some additional information specific to the Python code sample

• Deploying Business Logic apps

Think Global, Act Local

Sample Code

We start our project with the most basic Python web application.

Create a directory. for example, "sample" and save this code snippet as hello.py.

from flask import Flask

app = Flask(__name__)



@app.route('/')

def hello():

    return "Hello World"

We will use Flask to create us a web application. Flask is a micro web framework written in Python. For more information and a quick start, see flask.palletsprojects.com/en/1.1.x/quickstart/.

When we access the URL root ('/'), the web server will return a string of text: feel free to ajdust.

Create Virtual Env

Before we deploy our application to the cloud, it can be helpful to first test it locally but this does mean we need to install some software. Feel free to skip this step and go to the next section.

For your OS, install Python, pip, and Flask. Flask comes with its own dependencies, like Jinja and Werkzeug. See installation for Flask platform instructions for Linux/macOS and Windows.

# Linux/macOS

python3 -m venv sample 

source sample/bin/activate

# Windows

py -m venv sample

.\sample\Scripts\activate

The venv command creates a virtual environment (here named sample but you can use any name your want).

Pip Install

Next we run the Python package manager pip command to upgrade to the latest version as a best practice and then install Flask. For the ins and outs about pip, see the pip documentation)

pip install --upgrade pip

pip install Flask

Flask comes with its own dependencies which are automatically installed: courtesy of pip.

Command venv has created the sample directory with bin, include, and lib. Command pip has installed the packages under lib.

Once we run the program, the executable bytecode will be created in __pycache__.

Flask Run

There are different ways how we can run our program. Define the FLASK_APP environment variable (use SET on Windows) and execute flask run.

export FLASK_APP=hello

flask run

Note that without further instructions Flask runs on the loopback adapter (localhost at 127.0.0.1) and hence only accessible from your computer (might be a good idea at this stage).

We can pass the host parameter to specify a specific network interface (or all NICs with 0.0.0.0). We can pass command and variable as a single statement as well.

FLASK_APP=hello flask run --host=0.0.0.0 -p 1234

We can pass multiple environment variables with the command or, alternatively, store them in a .flaskenv file (although this requires python-dotenv to be installed).

Alternatively, we can also add host and port to hello.py and run the command using the python command.

app.run(host='0.0.0.0', port=5000)

python hello.py

When you are done with local development and testing, exit the environment with command:

deactivate

SAP Cloud Foundry

Sample Code

For this tutorial we will be using the Cloud Foundry command-line interface, cf CLI, which you can install from GitHub: github.com/cloudfoundry/cli/blob/master/README.md.

We also assume that you have a trial or enterprise account on the SAP Cloud Platform. If not, see SAP Cloud Platform Developer Onboarding | Hands-on Video Tutorials for how to get started.

We continue with the server.py sample code from SAP Cloud Platform documentation:

• Developing Python in the Cloud Foundry Environment

The code is very similar to hello.py above. The main difference is that a port variable is defined using os.environ.get and for this we need to add an import statement. This would typically be used in a try block: see if it is free and if so, assign it. The if __name__ == line is a bit of boilerplate to indicate where our program starts but like the port assignment not really necessary for our sample app.

import os

from flask import Flask

app = Flask(__name__)

port = int(os.environ.get('PORT', 3000))



@app.route('/')

def hello():

    return "Hello World"



if __name__ == '__main__':

    app.run(host='0.0.0.0', port=port)

Push an App

To run the app on the SAP Cloud Platform, connect to your Cloud Foundry (trial) subaccount, org, and space.

cf api

cf login | cf l

cf target | cf t

To deploy the application to Cloud Foundry, we use the cf push command.

When we run this command as-is, without argument, the execution will fail with an incorrect usage message: we need to provide the app name or use a manifest.yml file.

The command syntax is returned. For the Cloud Foundry documentation, see the Cloud Foundry CLI Reference Guide. See also Pushing an App.

When we run the command with only a name a staging error is returned:

No start command specified by buildpack or via Procfile.
App will not start unless a command is provided at runtime

According to the Cloud Foundry documentation, section Python Buildpack, the Python buildpack is used if a requirements.txt or setup.py file is detected in the root directory of the project.

Without these files, CF had no idea which runtime environment to provide.

Changing the file name server.py to setup.py solves the buildpack issue, however the script cannot be executed because the the Flask module is not found.

Application Dependencies

Application dependencies for a Python program are defined in the file requirements.txt. Unless a specific version is required, just the name suffices. This also applies to the dependencies of the dependency, Flask in our case. W could have specified all dependencies but could also leave this up to pip, the package manager.

# Flask==1.0.x

Flask

 

When we run cf push myapp again we can see that the Flask (and its dependencies) are installed.

Start Command

However, a start command is still missing. We can pass this as --start_command or -c in short. This is the same command we used to execute the app locally.

cf push myapp -c "python -m server"

The app is created with name = myapp, path = current directory; route = appname + cfapps + API domain.

Staging and Running Apps

The file is uploaded and the staging prepared by downloading the buildpacks.

First a container is created by a cell. The Python buildpack is downloaded and Python and pip are installed. Pip installs Flask and dependencies. The result is uploaded as droplet and the container stopped and destroyed.

The pesky warnings are unfortunate. These are caused by the buildpack. To get rid of them we need to create our own buildpack, a topic we return to below.

After staging, a Cloud Foundry cell then powers up another container, this time to run the droplet.

The illustration below shows the cf push process. A Diego cell is a virtual machine. One is used to stage the app and create the droplet using a container (first created, then destroyed). Another to run the app. For the details, see How applications are staged.



Side Note: Cloud Foundry was originally developped in Ruby. The component responsible for running the droplets was the Droplet Execution Agent (DEA). When Cloud Foundry was re-architected and rewritten in Go, DEA became Diego.

 

The same information is provided by the SAP Cloud Platform Cockpit. When we click the application route we are directed to our app.

The route needs to be unique in our domain and in shared environments the myapp route may already have been taken. We fix this below.

Note that our simple Hello World app allocates a fair amount of resources: 1024 MB memory and disk quota while only a fraction is needed.

Note that our basic app has allocated all available memory 1024 MB while only 15.9 is needed. The same applies to the disk quota.

Attributes and Manifests

Attributes

We can control memory and disk allocation with the attributes -m and -k.

Random route generates a unique name for our app.

cf delete myapp

cf push myapp -m 128M -k 256M --random-route -c "python server.py" 

 

The update the app, we can simply run the cf push command again.

Should you want to clean up first, use the cf delete command.

The random route generated this time is myapp-execellent-warthog-by but it will be different when we delete and push.

Note that when you delete before push you will eventually run out of your random routes quota. To reset the counter, use command

cf delete-orphaned-routes 

Manifest

As the cf CLI already indicated, instead of passing attributes on the command line we can also use a manifest.yml file.



Side Note: Initially the ML from YML was the same ML as in XML and HTML: Yet Another Markup Language (going back to the early days of Yahoo). Later the creators changed their mind and now it stands for YAML Ain't Markup Language, a recursive acronym which only makes sense to very few people. However, the statement is correct. YAML is not a markup language but a configuration file language and one that is, like Python, senstive to identation. In other words, spaces have meaning.



When we enter our attributes in a manifest.yml file we need to do it likes this and exactly like this (although we can move the optional attributes around). For the full specs, see the App Manifest Attribute Reference.

---

applications:

- name: myapp

  buildpacks: 

  - python_buildpack

  path: .

  memory: 128M

  disk_quota: 256M

  random-route: true

  command: python server.py

Should the route be fixed, you can use the host attribute instead of random-route. This will need to be unique for the domain, as mentioned.

Path points to either the directory where the code is located or to a ZIP or JAR file with the code. As we have already seen, without this parameter the current directory is assumed; the . (dot) that is. In the manifest file above the path attribute is included but superflous as it contains the default value.

To try out the zip format, compress server.py and requirements.txt into an archive and provide this as path. For larger projects, this shortens the upload time.

The CLI looks for a manifest.yml in the current directory. When stored elsewhere, use the attribute -f with the path to the manifest is.

Buildpacks

Courtesy of SAP

In the manifest above we explicitly set the buildpack we want to use. Cloud Foundry uses a buildpack to create a droplet (tarball or zipped archived stored in a blobstore), which is later used to run the app.

The cf buildpacks command lists all available buildpacks provided by SAP.

Latest and Greatest (?)

However, you can download a more recent version from Cloud Foundry, if needed, or build your own. For the details, see Python Buildpack and Buildpacks.

---

applications:

- name: myapp

  buildpacks:

    - https://github.com/cloudfoundry/python-buildpack.git

  path: .

  memory: 128M

  disk_quota: 256MB

  random-route: true

  command: python server.py

The console output below shows a Cloud Foundry cell creating a container and download Go to build the app, then the Python Buildpack version 1.7.21. The version provided by SAP Cloud Platform is currently 1.7.17 (see output above).

The rest is the same: 'go build' installs python, pip-pop ( to manage requirements.txt files) and then runs the Python package manager pip to install Flask, etc.

Runtime

A buildpack might contain several runtime versions. For example, buildpack 1.7.21 contains ten Python binaries from 3.5.9 (lowest) to 3.8.5 (highest). To specify the exact version to run, add a runtime.txt file to your project.

python-3.8.x

For Python buildpack release information, see github.com/cloudfoundry/python-buildpack/releases.

Passing Parameters

Procfile

When we run our app locally, besides python server.py we showed that we could also start our app using an environment variable export FLASK_APP=hello.py with command python -m flask run.

Port numbers are configuration and configuration should not be in code. Let's simplify our web app by removing the port.

Save as web.py.

from flask import Flask

app = Flask(__name__)



@app.route('/')

def hello_world():

    return 'Hello, World! (again)'

We can now remove the command from the manifest and add it to a file named Procfile (no extension).

web: FLASK_APP=web.py python3 -m flask run --host=0.0.0.0 --port=$PORT 

When a Procfile is detected, the command attribute is ignored.

For more information, see Production Server Configuration (Cloud Foundry) and The Procfile (Heroku).

---

applications:

- name: myapp

  buildpacks:

    - python_buildpack 

  path: .

  memory: 128M

  disk_quota: 256MB

  random-route: true

User Provided Variables

Alternatively, we could have used the env attribute in the manifest for FLASK_APP instead of  Procfile.

---

applications:

- name: myapp

  ...

  command: flask run  

  env:

    FLASK_APP: web.py

Note that we can still pass attributes on the command line to supplement or overrule the manifest.

cf push -f manifest.yml -i 3

Docker (Bonus Track)

Docker and Diego

As we have seen, Cloud Foundry runs (Garden) containers inside virtual machines (cells). If you wish, you can also run Docker containers in a Diego cell. This is not relevant for our scenario and would require additional configuration to make this work with service instances (SAP HANA Cloud and XSUAA) but serves to clarify the inner workings of Diego: just another VM+Container architecture.

To test this out you need a local Docker installation, see www.docker.com for the downloads. We are going to assume you have done the Docker 101 tutorials and have fiddled with images and containers a bit and have an account on the Docker Hub.

For a step-by-step, see Deploy Application using Docker Container on SAP Cloud Foundry : 2020 by kachiever.

Dockerfile

First, ceate a Docker directory with a Dockerfile file.

# set base image (host OS)

FROM python:3.8



# set the working directory in the container

WORKDIR /app



# copy the dependencies file to the working directory

COPY requirements.txt .



# install dependencies

RUN pip install -r requirements.txt



# copy the content of the local src directory to the working directory

COPY src/ .



# command to run on container start

CMD [ "python", "./server.py" ]

We are using the same requirements.txt file as before and the orignal server.py file. To separate the configuration files from the source code, create a src file and move the server.py file.

Build

Build the container with command

docker build -t python-tutorial .

This will build an image tagged as python-tutorial

1. Download the Python image


2. Change to the work directory /app


3. Copy requirements.txt


4. Run pip to install Flask and its dependencies


5. Copy the source code (server.py)


6. Run the command

Run

To run the container locally, use command:

docker run -d -p 3000:3000 python-tutorial

Into the Cloud

Docker Push

Now that we have validated that our local container is working, time to upload the image to Docker Hub. For tag the image and push it to Docker Hub. Of course, Docker Hub is just one choice. Alternatively, you can use Azure, Google Cloud, AWS, etc. to store the image. For the documentation, see Deploying an App with Docker.

docker login --username=<your user>

# note image ID

docker images

# tag the image <username>/<container>:<tag>

docker tag c8d87c346a6a dvkempen/python-tutorial:cloudfoundry

# push the container to the hub

docker push dvkempen/python-tutorial

CF Push

With the container already in the cloud, it is a small step for Cloud Foundry to host it. We only need to provide our APPNAME and docker image with credentials. Optionally, again, use a manifest to control memory, disk, and other attributes. This works the same as with the apps. As documented: push - Cloud Foundry CLI Reference Guide

cf push myapp --docker-image dvkempen/python-tutorial:cloudfoundry --docker-username dvkempen

As indicated but the CLI output, you can set a local environment variable CF_DOCKER_PASSWORD for the password.

Note that the staging phase is now much shorter as we no longer need to build the image.

Share and Connect

Questions? Please post as comment.

Useful? Give us a like and share on social media.

Thanks!

If you would like to receive updates, connect with me on

• LinkedIn > linkedin.com/in/dvankempen


• Twitter > @dvankempen

For the author page of SAP PRESS, visit

• Denys van Kempen

Over the years, for the SAP HANA Academy, SAP’s Partner Innovation Lab, and à titre personnel, I have written a little over 300 posts here for the SAP Community. Some articles only reached a few readers. Others attracted quite a few more.

For your reading pleasure and convenience, here is a curated list of posts which somehow managed to pass the 10k-view mile stone and, as sign of current interest, still tickle the counters each month.

• Good Reads (my two cents)