Neo to Cloud Foundry Migration

In this post, we will walk through the Migration of an Application in SCP (SAP Cloud Platform) Neo Stack to Cloud Foundry. The application being migrated is an internal application, and this activity was done purely from a research standpoint. There is no customer involvement nor business need for this port. The idea is to illustrate the path took on making this migration and discuss some of the common pitfalls that can be found while doing so. Cloud Foundry has some substantial changes compared to Neo, from how applications are provisioned, how services are consumed and how routing and authorizations are handled. For the sake of this write-up, a naive migration approach was taken. Substitute services were provided on the Cloud Foundry side and the application code was ported mostly as-is. For the most part, authorizations fall out of the scope of this post, however they were added in the ported code.

Contents

• Introduction

• UI5 Apps and Fiori Launchpad Migration

  • Serving the Application with Approuter

  • Serving Application with Portal and html5-apps-repo

    • How the Front-End was structured using html5-apps-repo

• Java Back-End Service Migration

  • How the Spring Services were Migrated

  • How the HANA DB was Migrated

• Open Points and Future Work

• Conclusion

Introduction

The application being migrated is an internal application from the IBSO Team in São Leopoldo, called Team Calendar. Its major functionality is to provide a full team view to our management with team members management, project allocations information as well as utilization reports. This application is used mostly by Management and Management support, however some functionalities, such as project allocation, ends up being responsibility of the team members. This makes Team Calendar’s features to be spread into eight different UI5 applications, used by both management and employees. Each one being a separate tile on a Fiori Launchpad, being:

• Team Calendar: For team members to view their teammates projects and edit their allocation to projects

• Booking Management: For management to maintain bookings

• Project Management: Maintain on-going projects

• Teams: To allocate team members to their teams

• Team Management: Create and manage teams

• Profile: View and edit your own profile with skills and roles

• User Management: Admin view of Users and their permissions

• Utilization Report: For generating utilization reports

All these tiles execute requests to the same /api endpoint, which is a Java application that uses Spring for the service layer and Hibernate as ORM. It uses version 7 of Java and Neo specific libraries. The persistence layer is an integrated HANA Database provided by Neo.

With that in mind, in our port we decided to use Portal Services, Spring Boot and Cloud Foundry’s HANA DB and Schema services. This posed a challenge not only to change the underlining configuration and provision files, but also some of the libraries being used by the original application.

In the following two sections, we’ll cover the specifics of the migration. On the first section, the Front-End is described and how a Fiori Launchpad was set for it on Cloud Foundry side. As for the second section, we describe the specifics of the Back-End service port and how HANA Database and Schema services are used to port the app.

UI5 Apps and Fiori LaunchPad Migration

In this section we’ll describe the steps taken to migrate the Front-End application from Neo to Cloud Foundry. This section is proposed in two different sub-sections. First, it’s described the naive approach and how a 1:1 port was proposed using approuter to simply “run” the application on Cloud Foundry. Secondly, we describe a proper way to migrate different UI5 applications and configure Fiori Launchpad tiles pointing to each of this apps. The later approach complies to the principle of having a Central Fiori Launchpad for customers. For that reason, the latter approach is the recommended one. However, it posed as a bigger challenge for the port as the provisioning of UI5 applications drastically changes from the HTML5 App on Neo by using html5-apps-repo services, together with Portal and deployer applications.

Not Ideal Approach: Serving the Application with Approuter

The initial analysis showed that in order to run the application as-is in Cloud Foundry, the only two things that needed to be provided were:

• Definition of routes and destinations (moving from neo-app.json to xs-app.json)

• Add a manifest.json for Cloud Foundry deployment

After providing the necessary configuration files, another point that was noticed was: how the Fiori Applications were going to be displayed and accessible to the Users; i.e. how to provide a Fiori Launchpad. In Neo this was solved by providing a list of apps in a fioriSandboxConfig.json. Apparently, this wouldn’t do the trick for a Cloud Foundry environment, specially because in Cloud Foundry we need to rely on different services to provide the Launchpad configuration and application hosting. The original approach taken (that was later discovered to be not optimal) was introducing approuter as a dependency and serve the UI5 applications with it, under an application level Fiori Launchpad.

These were the steps to do so:

1. Add the app router dependency:

npm install --save @sap/approuter

You need to have SAP’s npm registry configured for @sap packages in order for above command to work, if you don’t, run the following command before installing the dependency (more info here)

npm config set @sap:registry https://npm.sap.com

2. Add a start script to package.json

{
  "name": "team-calendar-ui",
  "dependencies": {
    "@sap/approuter": "^6.0.1"
  },
  "scripts": {
    "start": "node node_modules/@sap/approuter/approuter.js",
  }
}

3. Create the xs-app.json with routes mapping to destination

{
  "welcomeFile": "/src/index.html",
  "authenticationMethod": "none",
  "sessionTimeout": 30,
  "routes": [
    {
      "source": "^/src/(.*)",
      "target": "$1",
      "localDir": "src"
    },
    {
      "source": "^/test-resources/(.*)",
      "target": "/test-resources/$1",
      "destination": "ui5-sdk"
    },
    {
      "source": "^/resources/(.*)",
      "target": "/resources/$1",
      "destination": "ui5-sdk"
    },
    {
      "source": "^/api",
      "target": "/",
      "destination": "api"
    }
  ]
}

• src: direct to static files under the local /src folder.

• /test-resources and /resources: use the ui5-sdk destination. Which will later point to a UI5 CDN.

• /api: use the api destination. Which is our not (yet) implemented Back-End system.

4. Create manifest.yml with destination endpoints

applications:
- name: team-calendar-ui
  buildpacks:
    - nodejs_buildpack
  memory: 256M
  command: yarn start
  env:
    destinations: >
      [
        {
          "name":"ui5-sdk",
          "url":"https://sapui5.hana.ondemand.com/1.38.41/"
        },
        {
          "name": "api",
          "url":"https://team-calendar-backend.cfapps.sap.hana.ondemand.com/"
        }
      ]

manifest.yml is the deployment file for Cloud Foundry. This is where we defined what Cloud Foundry shall use for its destinations. The destinations needed for the Front-End application to run were the following:

ui5-sdk: this serves as static hosting of the UI5 lib. E.g. https://sapui5.hana.ondemand.com/1.38.41/resources/sap-ui-core.js. This is used by the Front-End application to fetch UI5 dependencies.

api: this is the deployed Back-End URL. Note that this snipped of code was taken after the full migration was done, so initially we had to deploy the Back-End system in order to have this here.

5. Fiori Launchpad Sandbox

In order to see the application running, we also wanted to provide a Fiori Launchpad (even if its a dummy one) to host all our 8 applications. In the index.html file configured as welcomeFile, the applications were mapped as follows:

window["sap-ushell-config"] = {
	defaultRenderer : "fiori2",
	renderers: { ... },
	applications : {
	  TeamPlanning : {
	    additionalInformation :"SAPUI5.Component=csc.ui.teamcalendar.team-planning",
		applicationType : "URL",
		url : "./team-planning/src/main/webapp/",
		description : "View and edit your Bookings",
		title : "Team Calendar"
	  },
...

6. Deployment

After the following points are configured:

• An application entry point with a Sandbox FLP (index.html)

• The destination for UI5 SDK and Back-End

• The routes mapping to the destinations

• approuter as the runtime for the Application

The application is ready to be deployed. This can be achieved by simply running the following command:

cf push

The command above will take all the applications defined in manifest.yml and deploy them.

Ideal Approach: Serving Application with Portal and html5-apps-repo

The steps above should be enough to take any UI5 application and make them run on Cloud Foundry. However, this approach seems fine for a single application but if there are multiple application (like in Team Calendar’s case), the idea of having a single approuter application with an app level Launchpad is not ideal. The reason for that, is because it defies the purpose of a Central Fiori Launchpad, with tile, catalog, roles and themes, across customer solutions (Standard and Custom).

Another possibility would be to wrap all the separated applications with approuter and run all of them separately. Once each would have its own endpoint and destination configuration, Portal Services can be used to create a catalog with tiles pointing to each of these applications. As much as this is a viable solutions it’s likely an overhead, specially because these applications are (ultimately) a bunch of static files.

For this reason, our Cloud Foundry environment offers a service called html5-apps-repo and a tool called ui5-deployer. These can be used, in junction, to deploy single UI5 application to a file storage (S3). This requires, however, significant changes on the configuration and deployment files of the Front-End application. HTML5 Application Repository Help Page.

How the Front-End was structured using html5-apps-repo

Changing the application to now work with html5-apps-repo requires even more tweaking. This blog post tutorial was used as a basis to create an UI5 Project with multiple apps on a Fiori Launchpad on WebIDE. In order to port our application to this format the following new services had to be used:

• Portal Service: Used to configure a Fiori Launchpad with its tiles.

• App Host and App Runtime: Needed to provide the static repository and runtime for the HTML5 Applications

Apart from these two services, there is an extra application that needs to be deployed:

• UI Deployer: Responsible for deploying any number of UI5 Applications into their static repositories

• FLP Content Deployer: Responsible for applying the Portal Configurations for the Launchpad Tiles

These were steps taken to do so:

1. Create a new Project in WebIDE using MTA Template

One important point is to also check the HTML5 Application Repository capabilities. This will create by default the approuter and the ui-deployer applications in the project. They are both needed to deploy and route to the HTML5 Apps.

2. Create the Launchpad and route to Portal home

A new “SAP Fiori Launchpad Site Module” named FLP and the generated approuter’s xs-app.json file was updated to point to Portal’s home page:

{
	"authenticationMethod": "route",
	"routes": [],
	"welcomeFile": "/cp.portal"
}

3. Import a single Team Calendar Tile to html5-apps-repo (Team-Display)

Now that the foundation of the UI5 apps is there, the next step was to try and port one of the original applications to this new format. So a new folder app was created, and the application folder was moved there. In order to have each application in a separated repository each application needs an xs-app.json and a manifest.json file. The problem with the original UI5 apps is that they had neither of those.

So the approach was: create a new HTML5 application through WebIDE, then import only the code from the original app. The reason for that was that WebIDE creates these files for you and updates the mta.yaml with the newly created application.

The repository ended up looking something like the following:

TeamCalendar/
  app/
  	team-display/
  	  webapp/
  	  	...
  	  	manifest.json
  	  Gruntfile.js
  	  package.json
  	  xs-app.json
  FLP/
  TeamCalendar_approuter/
  TeamCalendar_ui_deployer/
  mta.yaml

The next step was to see the application running on a FLP under a Team Display tile. WebIDE provides a graphical editor for the CommonDataModel.json file – which is the file where the Launchpad is configured.

Tile configuration (such as name, icon and intent) are configured on the manifest of the application itself. Which WebIDE provides and equally rich graphical editor.

5. Building and Deploying the MTA

There’s not much to say when it comes to building and deploying the MTA.

• Right click on the project and select build – this will create a mta_archives folder on your project and save the .mtar file there

• Right click on the .mtar and select deploy to SCP – after the job is completed, the services should be created and all the apps should be deployed to your Account.

It’s important to mentioned that the “deployer” apps (i.e. FLP and UI Deployer) are not apps that will remain running. They are a one-shot deploy and run. Your application page would look something like this:

Making the Front-End service communicate with Back-End does not differ much from the original port (the first half of this section). The destination configuration should be done on the approuter level, which is where all the requests will go thorough. This way, if any application tries to access /api, the destination configured on the approuter level would take care of redirecting the request to our running Java Back-End.

Java Back-End Service Migration

The original Back-End service for Team Calendar is a Monolith Java Spring application, using Hibernate to map the Entities to Database. So to port this application things were a little different, we had to battle the code a lot more than the infrastructure (different from the Front-End part). Since Cloud Foundry relies on Buildpacks to provision the applications, simply assigning a Java Buildpack is enough to have a Java application up and running. With the first analysis the major change points would have to be:

• Adapt the Java code to rely on Cloud Foundry libraries other than Neo ones

• Provide a HANA Database on Cloud Foundry and adapt configuration files/class

How the Spring Services were Migrated

The first step to this migration, was to create a Spring Boot application from Scratch and deploy it to Cloud Foundry (without the current Team Calendar code). This was enough to understand where the current application code is going to be placed. After the Spring Boot application is up and running, the second point would be to have a single endpoint “/Teams“. This allowed the port of a single slice of the application (rest api, controller, service and model) to the new Cloud Foundry version.

Luckily, someone published a blog post explaining how to setup a Spring Boot Application on Cloud Foundry using Hibernate and HANA. Which is exactly our scenario. This post was used as a basis to understand how to configure Cloud Foundry and the Spring application, as well as connect a HANA DB instance to it.

1. Adding the proper dependencies

Since we created a Spring Boot application, the pom.xml file was generated with the proper spring boot rest, JPA and web dependencies. Therefore, from the framework perspective the application was already covered. Now the old application relied on Neo SDK (neo-java-web-api), which needed to be changed for a couple of new Cloud Foundry libraries.

spring-cloud-cloudfoundry-connector: To connect with other services in a Cf environment

spring-cloud-cloudfoundry-hana-service-connector: HANA Connector for Spring Boot

spring-cloud-spring-service-connector: Data source implementation for Spring Data Connector

ngdbc: HANA Driver

2. Adding Teams endpoint

In a spring boot application, this was done by simply creating a dummy return on the /Teams/my endpoint using a Spring Servlet.

@RestController
@RequestMapping("/Teams")
public class TeamServlet {
    @RequestMapping(value = "/my", method = RequestMethod.GET)
    public @ResponseBody List<String> getCurrentUserTeams(HttpServletRequest request) {
        List<String> result = new ArrayList<String>();
        return result;
    }
}

3. Add manifest.yml

applications:
- name: team-calendar-backend
  buildpack: java_buildpack
  memory: 1G
  path: target/TeamCalendar-0.0.1-SNAPSHOT.war

Note 2: We have had problems with memory while starting the application, so we had to bump it to 1G. More information in this post

Once the application is deployed into Cloud Foundry, there should be a valid endpoint to it. This endpoint can be added to the destination environment variable on the Front-End (see above).

4. Migrating the service, controller and model files

After a dummy version of the Back-End application is up and running, the next logical step would be to migrate some of the core code of the original application. The idea is to continue with the /Team/my endpoint as the only migrated functionality, so all the code migrated was the coded necessary to this endpoint.

So one by one the @Controller , @Service and JPA @Entity classes were all moved into the new repository.

The only issue faced here was that the Spring Boot dependencies being used were Java 8+, this meant that some methods of the previous used JPA libraries had a different signature. This triggered a few changes in the original application code such as:

• findOne searches by ID had to be changed by findById

• Search results would no longer return null if nothing was found, instead an Optional<T> object was returned. This meant that most result != null had to be changed into optional.isPresent().

• save(List<T> multiple) was changed to saveAll(List<T> multiple).

How the HANA DB was Migrated

As stated above, most of the know-how on a Spring/Hibernate Cloud Foundry application was taken from this blog post, and sure enough how to configure the HANA database tables was directly taken form its part 2.

After the Database was created and the Schema service was provided, it was a matter of linking our newly created Java Application to this service, create a new DatabaseConfig class and the application properties file and Spring JPA libraries would do the rest.

CloudDatabaseConfig.java

@Configuration
@Profile("cloud")
public class CloudDatabaseConfig extends AbstractCloudConfig {

    @Bean
    public DataSource dataSource(@Value("${hana.url}")final String url,
                                 @Value("${hana.user}")final String user,
                                 @Value("${hana.password}")final String password,
                                 @Value("${hana.jdbc}")final String driver) {


        return DataSourceBuilder.create()
                .type(HikariDataSource.class)
                .driverClassName(driver)
                .url(url)
                .username(user)
                .password(password)
                .build();

    }
}

application-cloud.properties

spring.jpa.hibernate.ddl-auto=update
spring.jpa.hibernate.naming_strategy=org.hibernate.cfg.EJB3NamingStrategy
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.HANAColumnStoreDialect
spring.jpa.properties.hibernate.jdbc.lob.non_contextual_creation=true

hana.url = ${vcap.services.team-calendar-migration.credentials.url}
hana.user = ${vcap.services.team-calendar-migration.credentials.user}
hana.password = ${vcap.services.team-calendar-migration.credentials.password}
hana.jdbc = ${vcap.services.team-calendar-migration.credentials.driver}

Table migrations

After deploying the application.

cf push

The HANA Database Schema should be created with Hibernate tables migrated based on the entities definition, and sure enough, there it is:

After one vertical slice of the application is ported into Cloud Foundry, the remaining becomes easier. Porting models, utility and rest endpoint classes were done without much effort. Services and Controllers needed a slight code adjustment to fit the correct libraries (as stated before). All references to the authentication services were left commented in this initial port.

Open Points and Future Work

There are quite a few open points in our port that, if time (and interest) allow, we shall investigate and solve down the line. To name a few:

Authorizations were barely neglected during this first migration. As much as an xsuaa service was provided to require user login to open our UI5 applications, features on our Back-End such as creating Database Table Entries for users upon first login were skipped completely. They weren’t necessary for the first version of the application due to required code changes in our Back-End code since the libraries being used to extract the user from the session were specific to Cloud SDK (com.sap.security.um.*) .

Database Data Migration: Importing data from the original database was also not considered to balance the effort on a full migration.

UI5 Reusable Libraries: On the original code, there is a commons library created to provide controllers and abstractions for all the UI5 applications. As much as this worked fine on the approuter approach, it requires some tweaking to work with html5-apps-repo which was also skipped by the time of the writing of this post.

Break Down Services: Originally we wanted to attempt breaking the monolith apart and identify some cut points for multiple services to emerge from our Back-End application. Some were identified but not experimented with.

Conclusion

In this port, the steps to port an application from Neo to Cloud Foundry were described. We took a naive approach of doing so, by simply taking the code that is running on Neo and trying to make it run on Cloud Foundry as-is. For the most part, this was successful with minor changes in code. However, major changes were done regarding how the application is served and how the services communicated with each other.

Porting the Front-End application was a challenge for different reasons than the Back-End service. Our original application had to be adapted in order to fit the html5-apps-repo way of providing static HTML5 resources to Launchpad Tiles. There was also the need to create a deployer application and the Portal Configuration. It was also discussed the different ways available to provide UI5 applications in Cloud Foundry (e.g. using Approuter), and how they compare to each other. As for the Java Back-End service, there was not much when it comes to changes how the application is provisioned (as it remained a monolith). However, the connection to Database was drastically changed; together with its dependencies. There were two major porting points to the service: (i) fitting the original 1.7 code in our 1.8 Spring Boot template; (ii) providing the proper configuration files for Database connections.

While developing this port, a handful of points were identified as possibly different services, an example of that is a module in the original application for report generation. Another example is the separation of utilization and team structure both at the service level as well as persistence level. For those questions, we might address them in a follow-up post analyzing how a similar application could be structured with a more Cloud Native approach in mind.

References and Further Reading

• Setting Up Application Router

• HTML5 Application Repository Help Pages

• (Blog Post) Creating and Deploying an UI5 application on FLP using WebIDE

• (Blog Post) Fiori Application Programming for Cloud Foundry

• HTML5 Repository Cloud Foundry Plug-in

• Tutorial on how to develop/deploy an application with Spring and Hibernate to CF

• MTA PDF