Establish an automated continuous end-to-end development process on SAP Business Technology Platform
For the sake of readability, we often use SAP BTP as a short form of the complete “SAP Business Technology Platform”, Alert Notification as a short form of the complete “SAP Alert Notification service for SAP BTP”,“CTMS”as a short form of the complete “SAP Cloud Transport Management”,“CI/CD”as a short form of the complete“SAP Continuous Integration and Delivery service”
In this blog, you will learn more about the development process in the SAP Business Technology Platform, for which we will use the currently available DevOps BTP services, that give you agility when executing the end-to-end process. You will get the bigger picture of an automated development process. You will learn how to establish a sustainable development process across your 3-tier cloud landscape, including automated transporting, development and validation.
We will start by developing our code in the SAP Business Application Studio. With a simple commit and push to the central repository, throughout a configured webhook, this leads to a trigger in the SAP Continuous Integration and Delivery service. Which in turn would trigger a pre-configured pipeline from the provided pipelines catalog – “SAP Fiori in the Cloud Foundry Environment” (CI/CD pipeline for the development of SAPUI5/SAP Fiori applications in the Cloud Foundry environment). The application is built and deployed to the dedicated SAP BTP Subaccount and at the same time the release commit is transferred to the SAP Cloud Transport Management as an end of the Pipeline. The CTMS here gives you the opportunity to have more control over your deployment on QA and Prod systems. Anytime there is started or finished import from the CTMS you are going to get a notification from SAP Alert Notification, to a predefined email address or other dedicated channel of your choice.
Furthermore, SAP Automation Pilot is configured to perform a QA verification, whenever there is a successful import on the QA system. Also, the SAP Automation Pilot will push events to SAP Alert Notification for each status change of its execution and delivers the messages to us via the same channel
Architecture picture provided by Product Management
In my case, I configured a 3-tier development landscape, including 3 different cloud subaccounts. Each of them respectively represents Development, QA and Production system. The 3-tier setup is possible independent from the type of account you have. You can use your live, free tier or trial accounts. Provide the said subaccounts and match all the below prerequisites:
SAP Launchpad allows simplify access to business apps with a role based, personalized launchpad site. In our case we are using an HTML5 Application, so it is a must for us to have a subscription for it. Information in the official documentation:
The SAP BTP, Cloud Foundry runtime lets you develop and run cloud-native applications. We must have it in every SAP BTP Subaccount. For more information, follow the official documentation: https://help.sap.com/viewer/4287333baaa6413a8ece0a8ed1196af4/Cloud/en-US/25f37ca392064adaabb317c7513f1dc1.html
All development is managed in a central repository. In this blog, we will use a GitHub repository, where changes by all developers are pushed.
We will start the configuration with providing all the SAP BTP DevOps services that have to be enabled, to get the whole process working:
1. Create an instance of the SAP Business Application Studio and execute all the steps from “Subscribe to SAP Business Application Studio” tab in the official documentation:
2. Create an instance of the SAP Cloud Continuous Integration and Delivery service and execute all 3 tasks – Enabling the Service, Assigning Roles and Permissions and Enabling the API usage – as described in the documentation
3. Create an instance of the SAP Cloud Transport Management service and execute steps 1 and 2 from the “Initial Setup” tab – “Provide Access to SAP Transport Management” and “Set Up the Environment to Transport Content Archives directly in an Application” as described in the documentation:
4. Create an instance of the SAP Alert Notification Service and execute all the steps in the “Initial Setup” tab, following the official documentation:
5. Create an instance of the SAP Automation Pilot service and follow the link in the “Initial Setup” tab, in order to get access to the application, following the official documentation:
Note: If you are using a trial account, please consider that SAP Automation Pilot is not available there. If you want to use it, it`s available on live and in free tier.
If you have one or more of the services already configured, you can skip the corresponding steps above. Once we have all the needed services set up as described above, we can proceed to the configurations.
The following configurations in steps 6-7 are executed in the CI/CD service
6. You need to configure the service and create a Job. First, we will start by adding Credentials and the corresponding central GitHub repository. All the needed information can be found here: https://developers.sap.com/tutorials/cicd-start-fiori.html . Follow steps 2-3, you can double check step 1, if everything is completed in prior, when the service was created and configured.
Note: If you have more than one IDentity Provider (IDP) and you are getting the error message that you are missing privileges, make sure that you are using the correct user with the correct IDP, for which you have assigned the Role collection. If you still get the error message, refresh the session by relogging in.
7. Once we have added the repository and we have created the webhook, we will Configure the CI/CD service.
As a prerequisite of the Job creation, we should create some credentials.
1. Go into the Credentials tab and press ‘+’ to add new credentials. Select “Basic Authentication” type and enter username and password for authentication in the SAP BTP.
2. Again in the Credentials tab, press ‘+’ again to add new credentials. Select type “Service Key”. Go to SAP BTP -> Instances and Subscriptions -> CTMS instance create it, if you haven`t got it already) and copy the CTMS service key and paste it here. Select Save.
After we have created the needed credentials, we can proceed:
3.In the service UI, go to Jobs → select “+”, to create a new job.
4.Enter a Name of the Job, select the previously created repository, the name of the branch and for Pipeline: “SAP Fiori in the Cloud Foundry environment” from the dropdown menu.
5.Scroll down to Release tab and select the checkboxes to enable “Deploy to Cloud Foundry Space” and “Upload to Cloud Transport Management”. This would automatically transport and deploy your application.
6.“Deploy to Cloud Foundry Space”
-API Endpoint: Enter the URL of your SAP BTP, Cloud Foundry API Endpoint.
-Org Name: Enter the name of your Cloud Foundry organization. You can find it in the overview of your subaccount in the SAP BTP cockpit.
-Space: Enter the name of the Cloud Foundry space to which you want to deploy your application.
-Credentials: Select the credentials previously created in step 7.1 for authentication in SAP BTP.
7.“Upload to Cloud Transport Management”
-Node name: Choose a suitable Node name, that will be used later in the CTMS.
-Service Key: Select the previously added CTMS service key from the credentials.
8. Now we have the CI/CD configuration completed, and we will add the application to the SAP Business Application Studio
1. Go in the subaccount with the SAP Business Application Studio and open the application
2. Select “Create Space”, mark “Fiori Application” and click “Create”.
3. Once the space is ready, launch it and this is the starting window you get:
4.Select option 2 “Clone from Git”, provide the URL of the GitHub repository
Note: You can no longer use username and password to commit in GitHub. Since July 2021, you have to use GitHub access token, more information is available here : https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token
9. Currently we have the SAP Business Application Studio with the cloned repository and the next step is to configure the CTMS destinations
1. Go back in the SAP BTP Subaccount where the CTMS instance was configured and select the Destinations tab on the left.
2. We have to create an SAP Alert Notification destination in order to see the Perform Notification checkbox in the CTMS when creating Nodes. Follow the steps from the official documentation: https://help.sap.com/viewer/7f7160ec0d8546c6b3eab72fb5ad6fd8/Cloud/en-US/95d4fc79244f4c93b526a943eef1e274.html?q=notification
Note: The name must be exactly as it is shown in the documentation
3.Again here, on the destination tab, create destinations for the Quality and Production subaccounts. Select “Create” and fill the required fields:
New Destination for Prod:
Name: Name for QA destination, for example “DestQA”
URL: https://deploy-service.cfapps.<default-domain>/slprot/<myorg QA>/<myspace>/slp
Authentication: Basic Authentication
Username/Password for authentication for QA subaccount.
New Destination for Prod:
Name: Name for PROD destination, for example “DestProd”
URL: https://deploy-service.cfapps.<default-domain>/slprot/<myorg Prod>/<myspace>/slp
Authentication: Basic Authentication
Username/Password for authentication for Prod subaccount.
Note: When both destinations are created, check their statuses: Select the destination and click “Check Connection”. The status should be green, and the “Response” is “200 OK”. Check also the second one.
10. Transport nodes represent source and target end points of a deployment process – for example, a Cloud Foundry subaccount. For the 3-tier landscape in this blog, we will create 3 Nodes -Dev, QA and Prod. In this step, we will create and configure them and the two Routes they are representing:
We will start with the DEV Node first:
1. Select “Transport Nodes” tab on the left -> “+”. Select suitable name like “DEV”
2. Mark checkboxes “Allow Upload to Node” and “Perform Notification” and select OK.
The DEV node is the first entry point of the route, so we leave “Content Type” and “Destination” empty.
Now we are continuing with the creation of the other two Nodes for the “QA” and “Prod” destinations. Let’s proceed with the QA first:
3. Select the “+” again. Make sure that you use the same name you gave when creating the Job in the CI/CD step.
4. Mark the checkboxes for “Allow to Upload to Node” and “Perform Notification”
5. For the “Content Type”, select “Multi-Target Application” from the dropdown list
6. For the “Destination”, select the Destination for the QA system created in prior and select OK
The creation of the “QA” Node is completed,
1.Select the “+” and enter a suitable name for the Prod node
2. Mark the checkbox for “Perform Notification” and Leave unchecked “Allow to Upload Node”
3.For the “Content Type”, select “Multi-Target Application” from the dropdown list
4.For the “Destination”, select the Destination for the PROD system created in prior and select OK
After the three nodes are created, we must create two routes to complete the CTMS configuration:
7. Select “Transport Routes” from the left and then “+”.
8. Enter a suitable name, for example DEV_QA.
For “Source Node” select the “Dev Node” you have created and for “Target Node” select the “QA Node”.
9. Select the “+” once again and enter a name, for example QA_DEV.
For “Source Node” this time select the “QA Node” and for “Target Node” select the “Prod Node”.
To check if the routes are created correctly, go to “Landscape visualization” and you should see this:
11.Configure SAP Alert Notification, so we can receive notifications whenever an import is started or finished. Also, in this section, we will create a Service Key we will use in the next steps, while configuring the Automation Pilot.
1. Go back to the SAP BTP Subaccount where the Alert Notification instance is created and select “Instances and Subscriptions” -> Alert Notification instance.
2.In SAP Alert Notification, select Service Keys -> Create Service Key.
Follow the documentation for Service Key creation with Basic Authentication : https://help.sap.com/viewer/5967a369d4b74f7a9c2b91f5df8e6ab6/Cloud/en-US/80fe24f86bde4e3aac2903ac05511835.html
Make sure to note down the information from the Service Key, as you will need it shortly.
3. Now let’s create a Subscription. Select Subscriptions -> Create.
4. Enter name and description as you like and select Create.
5. Select ” + Create Condition”. On the next page, enter a name for the condition.
6. Select “EventType” equal to “TmsImportStarted”. Select Create.
Do the same procedure with the difference of the expected value to be “TmsImportFinished”. After that, select both conditions and click Assign.
7. Select “+ Create Action”. In my case, I used the email action and clicked Next – but feel free to select a different notification channel. Give a suitable name to the action and enter your email address and click create. Save the subscription.
Note: Here you can create an Action of your choice, for more Information: https://help.sap.com/viewer/5967a369d4b74f7a9c2b91f5df8e6ab6/Cloud/en-US/8a7e092eebc74b3ea01d506265e8c8f8.html?q=actions
When the Subscription is ready, it should look like this:
Note: If you are using a Trial account, the Automation Pilot is not available there and you will not be able to configure the automated testing, while still you’ll have the full pipeline configured. Therefore, you can skip steps 12 and 13 and move on to step 14.
12. Now let`s configure the Automation Pilot. Go back to SAP BTP and select “Instances and Subscriptions” and open the Automation Pilot application.
1. Select “Service Account” and create an account, follow the steps in the documentation: https://help.sap.com/viewer/de3900c419f5492a8802274c17e07049/Cloud/en-US/acf58b69e1484f8595e6b398687b08b3.html?q=service%20account
Note down the “Service Account” username and password for later use.
2. Enable Automation pilot for sending events to ANS following the documentation: https://help.sap.com/viewer/de3900c419f5492a8802274c17e07049/Cloud/en-US/e75533639c6d4193aa8a7e7420c25f8c.html?q=service%20account
3. Before creating an execution which you want to be triggered when the TmsImportFinished, you should create a command for a quality test you want to be triggered. In my case, it is a very simple validation test.
Customize your command as needed for the quality testing of your application
4. You need to create a “Build Event Trigger”, which will execute a command of your choice, when a trigger arrives from Alert Notification.
To do so, select “Execution” tab on the left and select “Build Event Trigger”
“Trigger type” has to be “Alert Notification” and for “Command”, you select the desired command that shall be executed. You can do other configurations, mappings or add additional features, but for the sake of simplicity, I will leave it like this.
In the “Event Trigger URL”, you will get an automatically generated URL, after all the editing of the event trigger is done. Copy it, so we can use it in the next step.
13. Go back in the ANS subscriptions and select Create.
1.Select a suitable name and click Create. Select the previously created condition TmsImportFinished and click Next.
2.Create Action and for the action type, select “SAP Automation Pilot”.
3.Select a suitable name and in the additional properties, paste the previously generated URL from step 12.4 and the username and password for the Automation Pilot Service Account.
Click Create and the Subscription is created.
It should look something like this:
4.Lastly, we need to create the third final subscription, so select Create again, give it a name and proceed.
5.Create a new Condition with “EventType” that contains “ExecutionStatusChange” and click create.
6.Select the email action (or the action of your choice) and save the subscription.
It should look something like this:
14. Check the working setup
Once you have all the configuration steps completed, let`s see if the process is working as expected.
First, for that reason, we should make a change in the code using the SAP Business Application Studio and push it to the central GitHub repository.
Second, open the CI/CD UI and navigate to the configured Job. You should see a commit that is in progress. You can observe the changes while the process is running.
Third, after the release phase is finished, your application should be running on the SAP BTP Dev Subaccount. Also, at the same time, in the CTMS, on the QA Node, should be the new commit which is waiting for the import.
Fourth, start the import and wait for the success status. Also, you must get a notification in the configured channel when the import is started and finished.
When the import is finished, the application with the new changes should be running on the SAP BTP QA Subaccount. At the same time in the CTMS, on the Prod Node, should be the new commit, waiting for the import.
Note: The following notifications will be delivered, only if you configured the Automation Pilot
Automation pilot is set to send alerts, whenever there is a status change of the execution, so we expect two more alerts:
The last step is to trigger the import in the Prod Node and after it`s finished to check if the new application version is running in the SAP BTP Prod Subaccount. Again, we expect to get 4 notifications when the second import is started.
This is how you can establish an automated continuous end-to-end development process using the currently available DevOps BTP services. Try it out on your own using the explained configurations or play around, so it can match your landscape specifics.
We would highly appreciate your feedback.