Technical Articles
Getting Grips on your Cloud Integration Migration from Neo to Cloud Foundry
Welcome screen displaying the step-by-step approach
Awesome! Isn’t it? When you finally decide to buy that new house. The moment you sign the agreement, the moment you tell your family and friends about it, the moment you get the keys! Until, that is, the moment you actually have to move, and panic takes over when realizing the time and effort it will take to pack everything in boxes, having to deal with the moving company and unpack everything again afterwards. If only this whole process could be made easier…
You probably feel the same way when your company decides to upgrade the existing SAP Cloud Integration landscape and leverage SAP Integration Suite as new end-to-end integration platform. Excitement about the new features waiting to be explored, question marks on how to fit a migration project in your already busy schedule …
Allow me to take some of the pressure off and introduce a new migration tool that will help you ‘move house’ from SAP Cloud Integration on Neo to SAP Integration Suite on Cloud Foundry.
Disclaimer: There is an official migration pack available via SAP Note 2937549, which leverages a collection of Postman scripts. It works great and is a perfect companion for smaller migration projects, but when it comes to more complex or selective migration projects it lacks certain flexibility. For this reason, we developed this alternative migration tool (scope of this blog), leveraging flagship SAP technologies such as the SAP Cloud Application Programming Model (CAP) and SAP Fiori, focusing on improving the migration process through easier configuration, execution and monitoring. This tool is not covered by SAP Support and is release as-is, as open source.
Why upgrading to SAP Integration Suite
Finding reasons to upgrade to SAP Integration Suite is a no-brainer: it’s fresh take on the traditional strengths of SAP Cloud Integration such as Process Integration and API Management, but also comes jam-packed with cool new features such as Open Connectors, Integration Advisor and Trading Partner Management. Increased SLA’s, virtually unlimited JMS capabilities and state-of-the-art partner adapters are some of the ‘under the hood’ advantages that come with it as well. To summarize: a no-brainer!
Looking for a more complete overview of SAP Integration Suite? Check out Patrick Kelleher’s SAP Integration Suite blog, or visit our SAP Integration Suite website.
How upgrading to SAP Integration Suite
When moving to a new house, you can do it yourself or you can get a moving company to help you. The same is true for ‘moving’ all your integration assets to SAP Integration Suite: you can manually export and import the artifacts, or you can get a free tool to help you.
Even though the conceptual approach is similar to the official migration tool, there are some additional advantages in this tool you can leverage in your migration project. So, let’s jump into the details and look at the top 5 advantages I see:
- It comes with a UI
- Flexibility in selecting individual items for a migration run
- Script analyzer warning about usage of changed environment variables
- Browse artifacts and rectify errors before executing a migration run
- Migrate configuration layers for multi-tier environments
Let’s explore each of these in a bit more detail.
1. It comes with a UI
Great, we can finally see what is going on! A UI generally lowers the learning curve of a tool and improves usability. It’s now easier to manage all your different connection settings, see what integration content sits in your tenants and have grips on which artifacts, security materials or configurations to migrate.
Screen to capture and test connection details of an integration tenant
2. Flexibility in selecting individual items for a migration run
When it comes to a migration project, there is no ‘one size fits all’, neither is it a ‘once off’ exercise. Some customers prefer a ‘big bang’ approach where all artifacts are migrated at once, others prefer a piece-meal approach, either by department, by project, or by functional area. Regardless of the approach you take, it’s important to be able to define the scope of the migration run.
To do this, the migration tool introduces the concept of a migration task. This defines which artifacts (integration packages, credentials, variables, etc.) need to be migrated from which source tenant to which target tenant. A migration task can be configured and saved, after which you can run it once or multiple times. A migration job will be generated for each run you do, which will monitor the progress, generate logs and hold the final outcome (succeeded / failed).
Select and configure the artifacts in scope of a migration run
3. Script analyzer warning about usage of changed environment variables
When using custom JavaScripts or Groovy scripts in Cloud Integration, you have the option to leverage environment variables (see SAP Help). They can give you information about the tenant’s name, the host name, port, etc. The way in which these work is slightly different in Neo and Cloud Foundry, so it is important to validate the correct syntax of a script after it has been migrated.
But how do you know which scripts you have to analyze? The migration tool will tell you. Each of the scripts that is migrated, either embedded scripts in an integration flow, or referenced scripts from a script collection, get quickly analyzed to see if environment variables are being used. If yes, the tool will generate a message so you can see which scripts need your attention after the migration.
See errors and warnings during a migration run
4. Browse artifacts and rectify errors before executing a migration run
There are many situations in which a migration will not be successful: items can be in draft state, SAP packages might not be up to date, you use specific security materials, etc. Hence it is worthwhile to do a pre-validation of your source tenant before executing a migration. For this, the migration tool does an initial download of the metadata of your tenant (only the metadata, not the actual content) so it can analyze all items and generate alerts for those items that could potentially pose an issue during a migration execution.
See errors and warnings from analyzing the tenant content
5. Migrate configurations for multi-tier environments
Following best practices of Cloud Integration, any system architecture would contain at least two or three integration tenants: a development tenant and a production tenant are the bare minimum. Some customers also add a test tenant, staging tenant, or even pre-prod tenant. Either way, there will be multiple tiers (tenants) available. Content is created in the development tenant, which is then transported through the stack until it arrives in production.
So how would you ideally approach a migration? You can start off by using the migration tool to do a dev-to-dev migration from your Neo development tenant to your Cloud Foundry development tenant. Once this is completed, you can leverage transport mechanisms such as SAP Transport Management on Cloud Foundry to promote it to production. However, there might be number ranges or specific configurations in your Neo test or production tenant that you also want to apply in your Cloud Foundry test or production tenant. This can be done via a special migration option which is called a ‘configuration only’ migration. Here the migration tool will no longer copy the integration flow itself but only the configurations.
Migration schedule for multi-tier landscapes
And there is more …
Sounds good? I bet it does, but there is even more:
- API access: anyone familiar with the SAP Cloud Application Programming Model (CAP) framework will know that applications are easily enabled for API access. So, we’ve enabled a bunch of APIs for you to automate some tasks, or plug this migration tool into your CI/CD pipeline.
- Custom coding option: to cater for very specific scenarios, there is a dedicated coding area which allows for custom logic to be applied during a migration (e.g., name changes, number range resets, etc.)
- Open source: to allow anyone to improve or extend the tool, it was decided to publish it as open source. So, if you have some great ideas for improvement, we can either work together on it, or you can venture on your own! No hard feelings!
Tool access
The tool is published on GitHub: https://github.com/SAP/migration-tool-for-cloud-integration
Any questions, suggestions or constructive feedback is always welcome, either as a comment on this blog (general topics) or as an ‘issue’ in the GitHub repository itself (technical topics). You can also post your questions or suggestions in the SAP Integration Suite community to discuss further.
Thanks,
Willem
This is super cool for Integration Suite folks can do migration much easier & transparent 🙂
Thank you for your feedback Yogananda. Much appreciated!
Excellent Willem !
Hi Willem,
Thanks for your super helpful tool.
I get "No .cds models found, please specify one as an argument" error while running 'cds deploy --to sqlite'.
Do you know if I am missing something here ?
Hi Senthil,
Thanks for your feedback. After cloning the project, make sure to navigate into the root directory of the project ("cd migration-tool-for-cloud-integration/") before doing the 'npm install' and 'cds deploy'. I will update the documentation to make this more clear.
Thanks
Willem
Thanks Willem for the response.
This time I get a different error
npm WARN config global `--global`, `--local` are deprecated. Use `--location=global` instead.
npm WARN old lockfile
npm WARN old lockfile The package-lock.json file was created with an old version of npm,
npm WARN old lockfile so supplemental metadata must be fetched from the registry.
npm WARN old lockfile
npm WARN old lockfile This is a one-time fix-up, please be patient...
npm WARN old lockfile
npm WARN tarball tarball data for sqlite3@https://github.com/mapbox/node-sqlite3/tarball/master (sha512-tUD7kpp82OxFSd93FTAta/h+vJVVQQAerG/QQmO3PtRJat1kSUwG/I38q+6ajwp4iG8/jKwCUkxnevo+kO8xNA==) seems to be corrupted. Trying again.
npm ERR! code EINTEGRITY
npm ERR! sha512-tUD7kpp82OxFSd93FTAta/h+vJVVQQAerG/QQmO3PtRJat1kSUwG/I38q+6ajwp4iG8/jKwCUkxnevo+kO8xNA== integrity checksum failed when using sha512: wanted sha512-tUD7kpp82OxFSd93FTAta/h+vJVVQQAerG/QQmO3PtRJat1kSUwG/I38q+6ajwp4iG8/jKwCUkxnevo+kO8xNA== but got sha512-xCWcDZTvIiZsZ/VMmx+wcs2rEfMCt1gOyrG1733WVYywgReh3o0SF069wqILPM0zbIGZNi1Qj6k/gOqAOUfTkw==. (3403153 bytes)
npm ERR! A complete log of this run can be found in:npm ERR! C:\Users\xyz\AppData\Local\npm-cache\_logs\2022-08-03T13_33_51_889Z-debug-0.log
Hi,
For technical issues, please raise an issue on the github itself, so we can discuss it there.
The error you receive is not related to the project. You are using a different NodeJS version, so you can just delete the 'package-lock.json' file and run 'npm install' again for your version.
Thanks Willem, It works like a charm now !
When is a migration from Cloud Foundry to Kyma planned?
Or will Cloud Integration stay let’s say for the next 10 years on CF?
Hi Stephan,
Thank you for your question. At the moment SAP Integration Suite is available on Cloud Foundry only, so no migration to Kyma is possible. The first component which will leverage Kubernetes is the Hybrid Deployment option on the roadmap for later this year: https://roadmaps.sap.com/board?PRODUCT=000D3A47875C1EDB98A8A910864AC24B&range=CURRENT-LAST#;INNO=901B0ED1A0641EDABE80AF561BFAC0F8
You can monitor the roadmap for adoption of Kubernetes in the future.
Is your question purely out of curiosity, or do you have specific scenario's/capabilities in mind which you think would become available with a Kyma implementation?
Hi Willem PARDAENS ,
Thanks for the blog! Interesting...
I have the tool up and running and wanted to configure my Cloud Foundry and Neo environments. Unfortunately, I receive this error
Any specific requirement about the service key on Cloud Foundry? I assume the service must be "Process Integration Runtime" and the plan is "integration-flow". Right?
And what about the OAuth client on the Neo environment? Which subscription should I use?
Many thanks.
Dimitri
Hi Dimitri,
Thanks for reaching out. The connection details to the different tenants has to be quite exact indeed otherwise it will not work. You can consult the step-by-step video tutorials in the /docs section to see how to configure each: https://github.com/SAP/migration-tool-for-cloud-integration/tree/main/docs
To your specific questions:
Hope this helps!
Thanks
willem
aaaaaah.... Salvation! Many thanks for the insights.
Time to get my hands dirty...
Hi Willem,
We were able to establish connections (MIG010) and performed readiness checks (MIG020) successfully. However, when we run MIG030 - Variables, we receive the following error:
We have the source api oauth client registered on tmn and all relevant roles assigned, target service instance and keys correctly configured (plan is api, admin and integ dev roles are assigned to the instance).
Also, when we try to run the MIG031 - Certificate to user mappings, we get the following error:
Appreciate any help/ inputs.
Thanks,
Krishna
Hi Krishna,
I'm not able to see the screenshots with your errors. Could you upload them again?
On a separate note: for support on the Postman migration package it would be best to open a ticket with SAP. This blog refers to a different tool.
Thanks,
willem