Deprecation of Cloud Foundry Stack cflinuxfs3 and Migration to cflinuxfs4

Overview

SAP BTP, Cloud Foundry runtime uses stacks, a prebuilt root file system that in combination with application code and buildpacks is the base to build the application container. The stack in the SAP BTP, Cloud Foundry environment is based on a Linux system and is called cflinuxfs<x>.

Stacks receive regular updates to address security issues and vulnerabilities, which your applications can pick up through new releases and version updates of the SAP BTP, Cloud Foundry Runtime. To get information about version updates and other Cloud Foundry related topics, see What’s New for SAP Business Technology Platform.

Currently in the SAP BTP, Cloud Foundry environment, all applications run on a stack called cflinuxfs3, which is based on Ubuntu Bionic Beaver 18.04. This stack was deprecated in December 2022 and is planned to be removed from the SAP BTP, Cloud Foundry environment in May 2023 after support for Ubuntu Bionic Beaver 18.04 has ended.

The successor stack cflinuxfs4, based on Ubuntu Jammy Jellyfish 22.04, has been available on an opt-in basis since December 2022 and is planned to become the default stack in March 2023.

This change affects anyone who is running buildpack-based applications in the SAP BTP, Cloud Foundry environment.

To help you understand what is going to happen, we’ve created a simplified timeline of the planned changes:

Cloud Foundry Stack Migration Timeline

What Happens When cflinuxfs4 Becomes the Default Stack?

Once cflinuxfs4 is designated as the default stack, all new applications will use this stack automatically. Existing applications running in the SAP BTP, Cloud Foundry environment won’t be affected by this change, but it is recommended that you update them to use the new stack by following the guide below, as the old stack will be removed in May 2023 and your app can stop working in case it is not compatible with the cflinuxfs4 stack.

To start using cflinuxfs4 with existing applications, you can push the application and designate the stack manually using:

cf push <APP_NAME> -s cflinuxfs4

If you’re using the blue-green deployment method to push your applications, they are automatically deployed on cflinuxfs4 when it is declared the default stack. This only works if you delete the unused blue or green applications. For more information on blue-green deployment, see https://docs.cloudfoundry.org/devguide/deploy-apps/blue-green.html.

You can continue to use cflinuxfs3 for a transitionary period. This means you can also manually switch back to cflinuxfs3 if you notice problems using the new stack by using the “-s” command when pushing your application:

cf push <APP_NAME> -s cflinuxfs3

cflinuxfs3 will be permanently deleted in May 2023. Afterwards it is no longer possible to manually switch back to cflinuxfs3. For more information on stacks, see https://docs.cloudfoundry.org/devguide/deploy-apps/stacks.html#available-stacks.

Risks and Consequences

If the applications were running, the restart causes a short downtime. However, if your applications are incompatible with the cflinuxfs4 stack, it is possible that they stop working and cannot be restarted.

To maintain functionality and compliance with security standards in the SAP BTP, Cloud Foundry environment, you must migrate your applications to cflinuxfs4. We recommend that you start testing your applications with the new stack as soon as possible to be able to fix potential issues. Only applications that have been migrated to cflinuxfs4 before the force-migration will not suffer from downtimes.

To assist you, we have compiled a short guide that walks you through the required migration steps.

Migrate to the cflinuxfs4 Stack with the CF CLI

Prerequisite:

Make sure you are using a recent version of the Cloud Foundry command line interface (CF CLI). We recommend using version 8. For information about the newest version, see https://github.com/cloudfoundry/cli/releases.

1. Determine Your Current Stack

If you are unsure which stack your applications are currently running on, you can determine it by using the CF CLI. Run the command:

cf app <APP_NAME>

The resulting output displays information about the application you specified. The output also includes a line starting with “stack:” followed by the name of the stack currently in use.

If cflinuxfs3 is shown here, your application is still using the deprecated stack. If cflinuxfs4 is listed, your application is already using the new stack and no further action is required.

The CF CLI supports the use of third-party plugins. To check which stacks your applications are using, you can use the Stack Auditor CLI Plugin to list applications for each org that you have access to. To see all the applications in your deployment, ensure that you are logged in to the CF CLI as a user who can access all orgs. We do not recommend using the ‘change-stack’ command provided by Stack Auditor plugin. While it can change your stack, it does not provide logs or any information about what went wrong if the operation fails. It also does not work on ARM Macs and uses the deprecated Cloud Foundry API v2. However it works well if you want to find out which of your applications require migration. For more information, see https://docs.cloudfoundry.org/adminguide/stack-auditor.html. Note that this plugin is not supported by SAP.

Alternatively, you can use the jq tool and the CF CLI to query the CF APIs for this information. Note that the jq tool is also not supported by SAP. For more information, see https://stedolan.github.io/jq/.

Using the jq tool, you can run the following script to get an overview of the stack and buildpacks used for all applications within a Cloud Foundry org:

cf curl "/v3/apps?per_page=5000&include=space.organization" | jq '(.included.spaces | INDEX(.guid)) as $spaces | (.included.organizations | INDEX(.guid)) as $orgs | [ .resources[] | {app: .name, org:$orgs[$spaces[.relationships.space.data.guid].relationships.organization.data.guid].name ,space: $spaces[.relationships.space.data.guid].name , lifecycle} ]'

If you want to filter for cflinuxfs3 to display the applications that require manual migration, use:

cf curl "/v3/apps?per_page=5000&include=space.organization" | jq '(.included.spaces | INDEX(.guid)) as $spaces | (.included.organizations | INDEX(.guid)) as $orgs | [ .resources[] | select(.lifecycle.data.stack == "cflinuxfs3") | {app: .name, org:$orgs[$spaces[.relationships.space.data.guid].relationships.organization.data.guid].name ,space: $spaces[.relationships.space.data.guid].name , lifecycle} ]' 

2. Determine if Buildpacks Used by Your Applications Support the New Stack

With the CF CLI you can check for the available buildpacks and their stack support by running:

cf buildpacks

Buildpacks supporting cflinuxfs4 will list it in the stack column of the output. The SAP Java Buildpack does not list any stack in the output, but it does support cflinuxfs4.

If you’ve designated a specific buildpack version to be used, it may be necessary to update the buildpack version in the manifest.yml of your application. See https://docs.cloudfoundry.org/devguide/deploy-apps/manifest-attributes.html#buildpacks.

For more information on individual buildpacks, see:

• Staticfile Buildpack
• SAP Java Buildpack
• Java Buildpack
• Ruby Buildpack
• .NET Core Buildpack
• NodeJs Buildpack
• Go Buildpack
• Python Buildpack
• PHP Buildpack
• Nginx Buildpack
• R Buildpack
• Binary Buildpack

3. Deploy an Instance of Your Application Using the New Stack

To check that your application is working properly on the new stack, we recommend that you test it before proceeding with the migration. To do so, push your application a separate time using a new name and a separate route by running:

cf push <APP_NAME> -s cflinuxfs4

If your applications use or contain any precompiled binaries, it is possible that you must recompile them. An example of this are applications depending on binary libraries such as OpenSSL or Python, where cflinuxfs4 relies on newer versions than the ones provided by cflinuxfs3.

If everything worked well, you can continue with your productive apps and change their stack by simply defining it in the cf push command or in the Cloud Foundry manifest. To avoid an unwanted reversion to cflinuxfs3, ensure that your Continuous Deployment automation deploys your applications with cflinuxfs4 as well.

Note that after your application has been deployed using the new stack cflinuxfs4 once, your selection will be saved for future cf push operations. This means that whenever you push the application afterwards, the SAP BTP, Cloud Foundry environment will automatically select the new stack as destination.

Migrate the cflinuxfs4 Stack Using the SAP BTP Cockpit

Important: We identified an issue with the Cockpit switch to cflinuxfs4 and we are working to provide a fix. Please switch to cflinuxfs4 according to the CF CLI instructions for now.

Instead of using the CF CLI, you can switch from cflinuxfs3 to cflinuxfs4 and vice versa in the SAP BTP cockpit.

Prerequisites:

You need to have the Space Developer role assigned to you.

1. Determine Your Current Stack

You can find your current stack listed on the Overview page of your application in the SAP BTP cockpit. Here, the current stack is cflinuxfs3:

2. Determine if Buildpacks Used by Your Applications Support the New Stack

If the Change Stack button appears on the Overview page of your application, this means that the option to switch between cflinuxfs3 and cflinuxfs4 is available.

If you deploy a new application, it should use the new cflinuxfs4 stack by default. Just to make sure that’s the case, you can always check this information on the Overview page of your application.

3. Changing the Stack

To change your current stack, perform the following steps:

1. Navigate to the Overview page of your buildpack-based application.

For optimal performance, we recommend that you turn off temporarily the automatic refresh from the upper right-hand corner of the page. You can turn it back on after you change the stack.

2. Choose Change Stack.

3. Choose a stack from the drop-down list.

The currently selected stack in this example is cflinuxfs3. If you want to migrate to cflinuxfs4, select it from the drop-down list.

If you haven’t turned off the automatic refresh yet, please do so now.

4. Please read the instructions carefully. Then, check all the boxes.

5. Choose Save. Applying the change may take some time.

Important: If there’s an issue with the switch to cflinuxfs4 and the application stops working, you can revert to cflinuxfs3 by following the same steps. After that, please try to switch to cflinuxfs4 according to the CF CLI instructions.