Technical Articles
Neo to Cloud Foundry Migration
In this post, we will walk through the Migration of an Application in SCP (SAP Cloud Platform) Neo Stack to Cloud Foundry. The application being migrated is an internal application, and this activity was done purely from a research standpoint. There is no customer involvement nor business need for this port. The idea is to illustrate the path took on making this migration and discuss some of the common pitfalls that can be found while doing so. Cloud Foundry has some substantial changes compared to Neo, from how applications are provisioned, how services are consumed and how routing and authorizations are handled. For the sake of this write-up, a naive migration approach was taken. Substitute services were provided on the Cloud Foundry side and the application code was ported mostly as-is. For the most part, authorizations fall out of the scope of this post, however they were added in the ported code.
Contents
-
Introduction
-
UI5 Apps and Fiori Launchpad Migration
-
Serving the Application with Approuter
-
Serving Application with Portal and html5-apps-repo
-
How the Front-End was structured using html5-apps-repo
-
-
-
Java Back-End Service Migration
-
How the Spring Services were Migrated
-
How the HANA DB was Migrated
-
-
Open Points and Future Work
-
Conclusion
Introduction
The application being migrated is an internal application from the IBSO Team in São Leopoldo, called Team Calendar. Its major functionality is to provide a full team view to our management with team members management, project allocations information as well as utilization reports. This application is used mostly by Management and Management support, however some functionalities, such as project allocation, ends up being responsibility of the team members. This makes Team Calendar’s features to be spread into eight different UI5 applications, used by both management and employees. Each one being a separate tile on a Fiori Launchpad, being:
-
Team Calendar: For team members to view their teammates projects and edit their allocation to projects
-
Booking Management: For management to maintain bookings
-
Project Management: Maintain on-going projects
-
Teams: To allocate team members to their teams
-
Team Management: Create and manage teams
-
Profile: View and edit your own profile with skills and roles
-
User Management: Admin view of Users and their permissions
-
Utilization Report: For generating utilization reports
All these tiles execute requests to the same /api
endpoint, which is a Java application that uses Spring for the service layer and Hibernate as ORM. It uses version 7 of Java and Neo specific libraries. The persistence layer is an integrated HANA Database provided by Neo.
With that in mind, in our port we decided to use Portal Services, Spring Boot and Cloud Foundry’s HANA DB and Schema services. This posed a challenge not only to change the underlining configuration and provision files, but also some of the libraries being used by the original application.
In the following two sections, we’ll cover the specifics of the migration. On the first section, the Front-End is described and how a Fiori Launchpad was set for it on Cloud Foundry side. As for the second section, we describe the specifics of the Back-End service port and how HANA Database and Schema services are used to port the app.
UI5 Apps and Fiori LaunchPad Migration
In this section we’ll describe the steps taken to migrate the Front-End application from Neo to Cloud Foundry. This section is proposed in two different sub-sections. First, it’s described the naive approach and how a 1:1 port was proposed using approuter
to simply “run” the application on Cloud Foundry. Secondly, we describe a proper way to migrate different UI5 applications and configure Fiori Launchpad tiles pointing to each of this apps. The later approach complies to the principle of having a Central Fiori Launchpad for customers. For that reason, the latter approach is the recommended one. However, it posed as a bigger challenge for the port as the provisioning of UI5 applications drastically changes from the HTML5 App on Neo by using html5-apps-repo
services, together with Portal and deployer applications.
Not Ideal Approach: Serving the Application with Approuter
The initial analysis showed that in order to run the application as-is in Cloud Foundry, the only two things that needed to be provided were:
-
Definition of routes and destinations (moving from
neo-app.json
toxs-app.json
) -
Add a
manifest.json
for Cloud Foundry deployment
After providing the necessary configuration files, another point that was noticed was: how the Fiori Applications were going to be displayed and accessible to the Users; i.e. how to provide a Fiori Launchpad. In Neo this was solved by providing a list of apps in a fioriSandboxConfig.json
. Apparently, this wouldn’t do the trick for a Cloud Foundry environment, specially because in Cloud Foundry we need to rely on different services to provide the Launchpad configuration and application hosting. The original approach taken (that was later discovered to be not optimal) was introducing approuter as a dependency and serve the UI5 applications with it, under an application level Fiori Launchpad.
These were the steps to do so:
1. Add the app router dependency:
npm install --save @sap/approuter
You need to have SAP’s npm registry configured for @sap packages in order for above command to work, if you don’t, run the following command before installing the dependency (more info here)
npm config set @sap:registry https://npm.sap.com
2. Add a start script to package.json
{
"name": "team-calendar-ui",
"dependencies": {
"@sap/approuter": "^6.0.1"
},
"scripts": {
"start": "node node_modules/@sap/approuter/approuter.js",
}
}
This can be used to start a NodeJS server and host the UI5 application based on the xs-app.json
file we defined.
3. Create the xs-app.json with routes mapping to destination
{
"welcomeFile": "/src/index.html",
"authenticationMethod": "none",
"sessionTimeout": 30,
"routes": [
{
"source": "^/src/(.*)",
"target": "$1",
"localDir": "src"
},
{
"source": "^/test-resources/(.*)",
"target": "/test-resources/$1",
"destination": "ui5-sdk"
},
{
"source": "^/resources/(.*)",
"target": "/resources/$1",
"destination": "ui5-sdk"
},
{
"source": "^/api",
"target": "/",
"destination": "api"
}
]
}
-
src: direct to static files under the local
/src
folder. -
/test-resources and /resources: use the
ui5-sdk
destination. Which will later point to a UI5 CDN. -
/api: use the
api
destination. Which is our not (yet) implemented Back-End system.
4. Create manifest.yml with destination endpoints
applications:
- name: team-calendar-ui
buildpacks:
- nodejs_buildpack
memory: 256M
command: yarn start
env:
destinations: >
[
{
"name":"ui5-sdk",
"url":"https://sapui5.hana.ondemand.com/1.38.41/"
},
{
"name": "api",
"url":"https://team-calendar-backend.cfapps.sap.hana.ondemand.com/"
}
]
manifest.yml
is the deployment file for Cloud Foundry. This is where we defined what Cloud Foundry shall use for its destinations. The destinations needed for the Front-End application to run were the following:
ui5-sdk: this serves as static hosting of the UI5 lib. E.g. https://sapui5.hana.ondemand.com/1.38.41/resources/sap-ui-core.js. This is used by the Front-End application to fetch UI5 dependencies.
api: this is the deployed Back-End URL. Note that this snipped of code was taken after the full migration was done, so initially we had to deploy the Back-End system in order to have this here.
5. Fiori Launchpad Sandbox
In order to see the application running, we also wanted to provide a Fiori Launchpad (even if its a dummy one) to host all our 8 applications. In the index.html
file configured as welcomeFile
, the applications were mapped as follows:
window["sap-ushell-config"] = {
defaultRenderer : "fiori2",
renderers: { ... },
applications : {
TeamPlanning : {
additionalInformation :"SAPUI5.Component=csc.ui.teamcalendar.team-planning",
applicationType : "URL",
url : "./team-planning/src/main/webapp/",
description : "View and edit your Bookings",
title : "Team Calendar"
},
...
6. Deployment
After the following points are configured:
-
An application entry point with a Sandbox FLP (
index.html
) -
The destination for UI5 SDK and Back-End
-
The routes mapping to the destinations
-
approuter
as the runtime for the Application
The application is ready to be deployed. This can be achieved by simply running the following command:
cf push
The command above will take all the applications defined in manifest.yml
and deploy them.
Ideal Approach: Serving Application with Portal and html5-apps-repo
The steps above should be enough to take any UI5 application and make them run on Cloud Foundry. However, this approach seems fine for a single application but if there are multiple application (like in Team Calendar’s case), the idea of having a single approuter
application with an app level Launchpad is not ideal. The reason for that, is because it defies the purpose of a Central Fiori Launchpad, with tile, catalog, roles and themes, across customer solutions (Standard and Custom).
Another possibility would be to wrap all the separated applications with approuter
and run all of them separately. Once each would have its own endpoint and destination configuration, Portal Services can be used to create a catalog with tiles pointing to each of these applications. As much as this is a viable solutions it’s likely an overhead, specially because these applications are (ultimately) a bunch of static files.
For this reason, our Cloud Foundry environment offers a service called html5-apps-repo
and a tool called ui5-deployer
. These can be used, in junction, to deploy single UI5 application to a file storage (S3). This requires, however, significant changes on the configuration and deployment files of the Front-End application. HTML5 Application Repository Help Page.
How the Front-End was structured using html5-apps-repo
Changing the application to now work with html5-apps-repo requires even more tweaking. This blog post tutorial was used as a basis to create an UI5 Project with multiple apps on a Fiori Launchpad on WebIDE. In order to port our application to this format the following new services had to be used:
-
Portal Service: Used to configure a Fiori Launchpad with its tiles.
-
App Host and App Runtime: Needed to provide the static repository and runtime for the HTML5 Applications
Apart from these two services, there is an extra application that needs to be deployed:
-
UI Deployer: Responsible for deploying any number of UI5 Applications into their static repositories
-
FLP Content Deployer: Responsible for applying the Portal Configurations for the Launchpad Tiles
These were steps taken to do so:
1. Create a new Project in WebIDE using MTA Template
One important point is to also check the HTML5 Application Repository capabilities. This will create by default the approuter
and the ui-deployer
applications in the project. They are both needed to deploy and route to the HTML5 Apps.
2. Create the Launchpad and route to Portal home
A new “SAP Fiori Launchpad Site Module” named FLP and the generated approuter’s xs-app.json
file was updated to point to Portal’s home page:
{
"authenticationMethod": "route",
"routes": [],
"welcomeFile": "/cp.portal"
}
/cp.portal
our approuter route will redirect to whatever was deployed as Launchpad configuration. It’s important to note that the content of our newly created FLP
module is deployable Launchpad configuration, and once deployed it changes our portal services configuration to whatever is defined in CommonDataModel.json
. More on that later.3. Import a single Team Calendar Tile to html5-apps-repo (Team-Display)
Now that the foundation of the UI5 apps is there, the next step was to try and port one of the original applications to this new format. So a new folder app
was created, and the application folder was moved there. In order to have each application in a separated repository each application needs an xs-app.json
and a manifest.json
file. The problem with the original UI5 apps is that they had neither of those.
So the approach was: create a new HTML5 application through WebIDE, then import only the code from the original app. The reason for that was that WebIDE creates these files for you and updates the mta.yaml
with the newly created application.
The repository ended up looking something like the following:
TeamCalendar/
app/
team-display/
webapp/
...
manifest.json
Gruntfile.js
package.json
xs-app.json
FLP/
TeamCalendar_approuter/
TeamCalendar_ui_deployer/
mta.yaml
The next step was to see the application running on a FLP under a Team Display tile. WebIDE provides a graphical editor for the CommonDataModel.json
file – which is the file where the Launchpad is configured.
Tile configuration (such as name, icon and intent) are configured on the manifest of the application itself. Which WebIDE provides and equally rich graphical editor.
5. Building and Deploying the MTA
There’s not much to say when it comes to building and deploying the MTA.
-
Right click on the project and select build – this will create a
mta_archives
folder on your project and save the.mtar
file there -
Right click on the .mtar and select deploy to SCP – after the job is completed, the services should be created and all the apps should be deployed to your Account.
It’s important to mentioned that the “deployer” apps (i.e. FLP and UI Deployer) are not apps that will remain running. They are a one-shot deploy and run. Your application page would look something like this:
Making the Front-End service communicate with Back-End does not differ much from the original port (the first half of this section). The destination configuration should be done on the approuter
level, which is where all the requests will go thorough. This way, if any application tries to access /api
, the destination configured on the approuter
level would take care of redirecting the request to our running Java Back-End.
Java Back-End Service Migration
The original Back-End service for Team Calendar is a Monolith Java Spring application, using Hibernate to map the Entities to Database. So to port this application things were a little different, we had to battle the code a lot more than the infrastructure (different from the Front-End part). Since Cloud Foundry relies on Buildpacks to provision the applications, simply assigning a Java Buildpack is enough to have a Java application up and running. With the first analysis the major change points would have to be:
-
Adapt the Java code to rely on Cloud Foundry libraries other than Neo ones
-
Provide a HANA Database on Cloud Foundry and adapt configuration files/class
How the Spring Services were Migrated
The first step to this migration, was to create a Spring Boot application from Scratch and deploy it to Cloud Foundry (without the current Team Calendar code). This was enough to understand where the current application code is going to be placed. After the Spring Boot application is up and running, the second point would be to have a single endpoint “/Teams“. This allowed the port of a single slice of the application (rest api, controller, service and model) to the new Cloud Foundry version.
Luckily, someone published a blog post explaining how to setup a Spring Boot Application on Cloud Foundry using Hibernate and HANA. Which is exactly our scenario. This post was used as a basis to understand how to configure Cloud Foundry and the Spring application, as well as connect a HANA DB instance to it.
1. Adding the proper dependencies
Since we created a Spring Boot application, the pom.xml
file was generated with the proper spring boot rest, JPA and web dependencies. Therefore, from the framework perspective the application was already covered. Now the old application relied on Neo SDK (neo-java-web-api
), which needed to be changed for a couple of new Cloud Foundry libraries.
spring-cloud-cloudfoundry-connector: To connect with other services in a Cf environment
spring-cloud-cloudfoundry-hana-service-connector: HANA Connector for Spring Boot
spring-cloud-spring-service-connector: Data source implementation for Spring Data Connector
ngdbc: HANA Driver
2. Adding Teams endpoint
In a spring boot application, this was done by simply creating a dummy return on the /Teams/my
endpoint using a Spring Servlet.
@RestController
@RequestMapping("/Teams")
public class TeamServlet {
@RequestMapping(value = "/my", method = RequestMethod.GET)
public @ResponseBody List<String> getCurrentUserTeams(HttpServletRequest request) {
List<String> result = new ArrayList<String>();
return result;
}
}
/Teams/my
a valid endpoint for Team Calendar Back-End, and with this, the application can be deployed to cloud foundry and connected with the Front-End application.
3. Add manifest.yml
applications:
- name: team-calendar-backend
buildpack: java_buildpack
memory: 1G
path: target/TeamCalendar-0.0.1-SNAPSHOT.war
.war
file. For that reason, the pom.xml
file should also specify it’s packaging
property as war
.Note 2: We have had problems with memory while starting the application, so we had to bump it to 1G. More information in this post
Once the application is deployed into Cloud Foundry, there should be a valid endpoint to it. This endpoint can be added to the destination
environment variable on the Front-End (see above).
4. Migrating the service, controller and model files
After a dummy version of the Back-End application is up and running, the next logical step would be to migrate some of the core code of the original application. The idea is to continue with the /Team/my
endpoint as the only migrated functionality, so all the code migrated was the coded necessary to this endpoint.
So one by one the @Controller
, @Service
and JPA @Entity
classes were all moved into the new repository.
The only issue faced here was that the Spring Boot dependencies being used were Java 8+, this meant that some methods of the previous used JPA libraries had a different signature. This triggered a few changes in the original application code such as:
-
findOne
searches by ID had to be changed byfindById
-
Search results would no longer return
null
if nothing was found, instead anOptional<T>
object was returned. This meant that mostresult != null
had to be changed intooptional.isPresent()
. -
save(List<T> multiple)
was changed tosaveAll(List<T> multiple)
.
How the HANA DB was Migrated
As stated above, most of the know-how on a Spring/Hibernate Cloud Foundry application was taken from this blog post, and sure enough how to configure the HANA database tables was directly taken form its part 2.
After the Database was created and the Schema service was provided, it was a matter of linking our newly created Java Application to this service, create a new DatabaseConfig class and the application properties file and Spring JPA libraries would do the rest.
CloudDatabaseConfig.java
@Configuration
@Profile("cloud")
public class CloudDatabaseConfig extends AbstractCloudConfig {
@Bean
public DataSource dataSource(@Value("${hana.url}")final String url,
@Value("${hana.user}")final String user,
@Value("${hana.password}")final String password,
@Value("${hana.jdbc}")final String driver) {
return DataSourceBuilder.create()
.type(HikariDataSource.class)
.driverClassName(driver)
.url(url)
.username(user)
.password(password)
.build();
}
}
spring.jpa.hibernate.ddl-auto=update
spring.jpa.hibernate.naming_strategy=org.hibernate.cfg.EJB3NamingStrategy
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.HANAColumnStoreDialect
spring.jpa.properties.hibernate.jdbc.lob.non_contextual_creation=true
hana.url = ${vcap.services.team-calendar-migration.credentials.url}
hana.user = ${vcap.services.team-calendar-migration.credentials.user}
hana.password = ${vcap.services.team-calendar-migration.credentials.password}
hana.jdbc = ${vcap.services.team-calendar-migration.credentials.driver}
hana.url
, hana.user
…) are going to be injected in the CloudDatabaseConfig
bean.Table migrations
After deploying the application.
cf push
The HANA Database Schema should be created with Hibernate tables migrated based on the entities definition, and sure enough, there it is:
After one vertical slice of the application is ported into Cloud Foundry, the remaining becomes easier. Porting models, utility and rest endpoint classes were done without much effort. Services and Controllers needed a slight code adjustment to fit the correct libraries (as stated before). All references to the authentication services were left commented in this initial port.
Open Points and Future Work
There are quite a few open points in our port that, if time (and interest) allow, we shall investigate and solve down the line. To name a few:
Authorizations were barely neglected during this first migration. As much as an xsuaa
service was provided to require user login to open our UI5 applications, features on our Back-End such as creating Database Table Entries for users upon first login were skipped completely. They weren’t necessary for the first version of the application due to required code changes in our Back-End code since the libraries being used to extract the user from the session were specific to Cloud SDK (com.sap.security.um.*
) .
Database Data Migration: Importing data from the original database was also not considered to balance the effort on a full migration.
UI5 Reusable Libraries: On the original code, there is a commons
library created to provide controllers and abstractions for all the UI5 applications. As much as this worked fine on the approuter
approach, it requires some tweaking to work with html5-apps-repo
which was also skipped by the time of the writing of this post.
Break Down Services: Originally we wanted to attempt breaking the monolith apart and identify some cut points for multiple services to emerge from our Back-End application. Some were identified but not experimented with.
Conclusion
In this port, the steps to port an application from Neo to Cloud Foundry were described. We took a naive approach of doing so, by simply taking the code that is running on Neo and trying to make it run on Cloud Foundry as-is. For the most part, this was successful with minor changes in code. However, major changes were done regarding how the application is served and how the services communicated with each other.
Porting the Front-End application was a challenge for different reasons than the Back-End service. Our original application had to be adapted in order to fit the html5-apps-repo
way of providing static HTML5 resources to Launchpad Tiles. There was also the need to create a deployer application and the Portal Configuration. It was also discussed the different ways available to provide UI5 applications in Cloud Foundry (e.g. using Approuter), and how they compare to each other. As for the Java Back-End service, there was not much when it comes to changes how the application is provisioned (as it remained a monolith). However, the connection to Database was drastically changed; together with its dependencies. There were two major porting points to the service: (i) fitting the original 1.7 code in our 1.8 Spring Boot template; (ii) providing the proper configuration files for Database connections.
While developing this port, a handful of points were identified as possibly different services, an example of that is a module in the original application for report generation. Another example is the separation of utilization and team structure both at the service level as well as persistence level. For those questions, we might address them in a follow-up post analyzing how a similar application could be structured with a more Cloud Native approach in mind.
Hi Frederico,
Thanks for this blog.
Can you please tell me how we can move SAP Analytics Cloud from Neo to CF.
We have many models, stories, dashboards and data already built in our SAC system which is on Neo. I want to know how I can move all this without any issues.
Please share if you have any standard documentation for this.
Regards,
Madhav
Hy, I tried to create a project sapui5 and copied the webapp folder AND maked the 3 files and RUN but i cannot how to resolve the problems....its a great problem to migrate from NEO to CF . Can you help-me?
Please can you see this question https://answers.sap.com/questions/13138202/migrating-a-neo-html5-app-to-sap-bastudio-in-cloud.html