Skip to Content
Technical Articles
Author's profile photo Marius Obert

#CloudFoundryFun #10 – Partial Deployments to Cloud Foundry

In this tenth post in my CloudFoundryFun series, we will reuse an application that we build previously. But this time, we only deploy selected modules and resources of the application to Cloud Foundry.

Mono-repo vs Multi-repo

If you are coming from “vanilla” Cloud Foundry, you might be more familiar with cf push than with cf deploy. The first command uses a manifest.yaml descriptor as a source for information about the project. This descriptor defines the runtime of the applications and the service bindings, but not the service instance definitions. This means the service instances need to be created manually. If you’ve been following this series, you might have noticed that I’m a big fan of the mta.yaml project descriptor. Besides the runtime definition, this descriptor can be used to define build tasks. Besides, it contains all definitions of your modules (aka microservices) and resources (aka backing services). Overall, this descriptor makes it very easy to use the Mono-repo source control pattern for your project.

The mono-repo pattern means that your entire codebase lives in one single repository. Multi-repo, on the other hand, means that each module is being managed in a single source control repository. In this latter approach, all models can be referenced by a “master repo”. I don’t want to go deeper into the advantages and disadvantages of either approach as this would be enough content for another post. If you’re interested in this, I recommend this 8-minute long video from Uber Engineering on their experience with mono-/multi-repos.

Sometimes less is more

A possible downside to this mta.yaml approach is that the build process outputs a single file (the .mtar archive). This meant that we had to deploy all modules and resources for every single update. It’s no surprise that this takes more time than redeploying only one module.

Sometimes, you might just want to update a single module because you already know the other did not change. For a long time, I thought that I had to bite the bullet, but then someone pointed me to this comment on GitHub. It turns out that there already is an option in the cf deploy command which allows us to do partial deployments. ?

In this post, I want to show you how partial deployments work.

Hands-on

If you have read CFF8, you might remember this sample application from GitHub:

This application exposes a list of sample entities that represent cities. Each entity contains several properties, like name, region, and image. The user can navigate to a Fiori Object Page and replace the default image with a newly uploaded file. This file will be stored in an Azure storage account and the URL that references the image will then be stored in a table within HANA.

The project contains the following modules:

Application router module The entry point of the application redirects all incoming traffic to the following two microservices. This module also contains the source code of the Fiori Elements user interface.

The server module Connects to the HDI container and exposes the annotated OData service via HTTP

Uploader module Service to upload files to the Azure storage account that returns the URL to access the created resource.

Database module A Cloud Foundry tasks that will run once to set up the schema in the HDI container and to import the sample data. Once these steps are completed, the app will shut down and stop consuming memory and CPU quota.

Besides the modules, this sample app leverages two backing services:

SAP HANA service SAP Cloud Platform provides HDI containers to allow apps to efficiently store business data in a state-of-the-art in-memory database. This datastore is used for structured data.

Azure Storage Account This service is provided by Microsoft to store large files (also called Blobs). We will use this store for unstructured data. During deployment, this service will be provisioned by the Azure Service Broker.

Let’s assume we want to run this application in a Cloud Foundry environment where there’s no Azure service broker set up. E.g. we won’t be able to provision a storage account during deployment.

In this situation, we only want to deploy the components that we can test in any Cloud Foundry environment. Therefore, we not only skip the creating of the backing service but also skip the Uploader app (as its sole purpose is to connect to the Storage account):

This is the shortest hands-on of this series so far.

0. Prerequisites

Set up your local dev environment

  1. Clone the repository.
    git clone https://github.com/SAP-samples/cloud-foundry-cap-azure-cities​
  2. Build the .mtar archive that contains all modules and resources.
    npm run build:mta​
  3. Only deploy the modules we need.
    cf deploy mta_archives/city-explorer-demo-app_1.3.5.mtar \
      -m city-cap-router \
      -m city-cap-db \
      -m  city-cap-srv \
      -r city-hdi-container​
    
    #For Windows users
    cf deploy mta_archives/city-explorer-demo-app_1.3.5.mtar ^
      -m city-cap-router ^
      -m city-cap-db ^
      -m  city-cap-srv ^
      -r city-hdi-container​
  4. Test the application. (Please keep in mind that the image upload feature will fail as the upload app is missing)

 

Are you looking for more samples? The GitHub repo might also be very interesting to you!

Summary

In this edition you have learned:

  • What a mono-repo is
  • What differentiates a manifest.yaml file from a mta.yaml file
  • What benefits partials deployments bring
  • How do a partial deployment to Cloud Foundry

#CloudFoundryFun #11 – Integrate a React app in the Fiori Launchpad

About this series

This was the tenth blog post of my monthly new series #CloudFoundryFunThe name already says all there is to it, this series won’t necessarily be about building enterprise apps on Cloud Foundry. I think there are already plenty of great posts about those aspects out there. This series rather thinks outside the box and demonstrates unconventional or novel Cloud Foundry use-cases ?.

Assigned Tags

      16 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Pierre Dominique
      Pierre Dominique

      Hi Marius,

      Great stuff, as usual. Please keep sharing this kind of information!

      Pierre

      Author's profile photo Marius Obert
      Marius Obert
      Blog Post Author

      Thank you Pierre ?

      Will do!

      Author's profile photo Krishna Kishor Kammaje
      Krishna Kishor Kammaje

      Thanks, Marius for this topic.

      I am getting started with CAP, and I created and deployed a CAP app from the tutorial here. https://developers.sap.com/mission.cp-starter-extensions-cap.html. Each module has a manifest.yaml in this case. So do you mean this is a multi-repo? I can still have a git repository at the top folder level and I can have all the code in a single repository (isn’t it a mono-repo then?).

      I am not sure what is the advantage of using mta.yaml compared to the above approach. Great if you can explain.

      Regards

       

       

      Author's profile photo Marius Obert
      Marius Obert
      Blog Post Author

      Hi Krishna,

      this is a very good question!

      I also think that the line between both terms can be blurry depending on how you define a multi-/monorepo. I would say your example is technically a monorepo as all modules are managed in one single repository. But I don't think that the "code management" is the main purpose of the monorepo. This main purpose is the "deployment management" - and this part is not given in the tutorial you are referring to.

      The case you described is totally valid for beginners as it focuses on CAP. Later, when you build a bigger, more advanced application, you will notice that you actually have to maintain 3+ manifest.yaml files which all contain different runtime configurations, dependencies, version numbers and environment variables. When you change one module, you most likely have to change another module too, as they are interconnected. This means you will have to bump the version strings of both modules and make sure you deploy them at the same time. This can become even more complicated when you run multiple instances of each module.

      The mta.yaml is basically the combination of all manifest.yaml files in one single source of truth. This file describes the entire runtime environment of you application, as well as the backing services. With this approach, you only have to manage this content of this file and deploy ONE single packaged application. The management of the running modules will be handled by the Cloud Controller (a component of Cloud Foundry).

       

      I hope this helped. Please let me know if you have more questions.

      Author's profile photo Krishna Kishor Kammaje
      Krishna Kishor Kammaje

      The mta.yaml is basically the combination of all manifest.yaml

      That confirmed my assumption. Thank you. I agree that there might be scenarios in the future where I would end up updating multiple manifest.yaml.

      Unfortunately, the official documentation here https://cap.cloud.sap/docs/guides/deploy-to-cloud does not talk about the mta.yaml option at all. Does it mean that CAP is opinionated on using manifest.yaml instead of mta.yaml?

      Author's profile photo Marius Obert
      Marius Obert
      Blog Post Author

      Good observation. The documentation - same as CAP library itself - is still under development and that's why there is some information missing.

      I'm glad to tell you that CAP is not opinionated in that regard ?

      You can make use of the --mta option in the "init command" to create a mta.yaml file for you CAP project:

      cds init --modules db,srv --mta --insecure --db-technology hana bookshop

      I usually recommend this command for basic Cloud Foundry development as it already targets HANA as the DB. These options also turn off the UAA (no login needed) and use the mta.yaml approach over manifest.yaml files

      Author's profile photo Krishna Kishor Kammaje
      Krishna Kishor Kammaje

      Thanks for the clarification on the opinionated stuff. 🙂

      Just an observation. Building and deploying a mtar to the cloud took around 10 minutes for me. But building and deploying with package.yaml approach was pretty fast. I hope SAP is looking into it.

      Author's profile photo Marius Obert
      Marius Obert
      Blog Post Author

      Interesting observation. How much of the 10 minutes was used for building the mtar? This never takes longer than a couple of seconds for me to build the archive (it could be up to a minute if “npm install” needs to run for all modules and the connectivity is slow). If this takes too much time, something is wrong.

      The deploy sometimes takes a lot of time, we’re aware of that. In most cases, this is due to the fact that it also has to provision an HDI container (this task won’t be done when just “cf push”).

      Or did you mean that the “pure” staging and starting of the application (you’ll see it in the log when this phase is being handled) take significantly longer than a single “cf push”?

      Author's profile photo Krishna Kishor Kammaje
      Krishna Kishor Kammaje

      I looked further and here are the details:

      mtar file is around 95MB. (That is huge!. Is it common?)

      Build took 84 seconds

      Deploy took 12 minutes, out of which 10 minutes were for the upload itself. Deploying, staging and activating took around 2 minutes.

      According sppedtest.net, I have a download speed of 46Mbps, and upload speed of 6.6 Mbps. Windows 10, 32GB RAM.

      Author's profile photo Marius Obert
      Marius Obert
      Blog Post Author

      Hi,

      The bundle size of about 100mb is (sadly) common, as the archive contains all npm dependencies and the node_modules folders are huge. But you are right, 12 minutes are really too long.

      Let's try to break the various phases into parts:

      Build: The build should be way faster. I would assume that the build process runs "npm install" in each module folder. I usually go to each module (db, app, srv) and run "npm install" to pre-download the dependencies.

      This speeds up the "mbt build"-time significantly on every following invocation.

      Upload: I'm afraid I cannot explain why this takes that long. For me, the upload takes about as much time as a 100mb-upload usually takes. Do you see this delay all the time? Maybe it's a latency issue. It would be interesting to know if this changes when you switch between SCP regions. Unfortunately, this factor is hard to influence.

      Deploy: 2 Minutes of deploy-time coincides with my experiences as well.

       

       

      Author's profile photo Krishna Kishor Kammaje
      Krishna Kishor Kammaje

      Thanks Marius for the expansions. I hope the developer UX improves in coming days for CAP developers.

      Author's profile photo Holger Schäfer
      Holger Schäfer

      Hi Marius,

      great blog and i will try partial deploy options so far, because for me all whole deployment tooks also very long.

      Concerning your discussion with Krishna. I figured out for cap, that the whole node package dir and the lock file  needs to be excluded for my cap project. This is because using sqllite it will be compiled platform agnostic. I am developing under Win10 and it took my some time to figure out, that if i am doing mta deployment puting npm into mta, the target will not run in the cf environment.

      In general, i mostly used local builds only for testing purposes while our git pipe did the production build for ui5 etc. But what if you want to include npm libs that are platform agnostic?

      I figured out, that excluding npm packaged leads to a clean npm install on target after deployment which takes a long time to install all packages. The only local workaround would be a local dev env similar to cf one.

      After reading https://docs.cloudfoundry.org/buildpacks/node/index.html section OfflineEnvironment i figured out, that this is currently the case for most of the sap demos, but if you try to add sqlite to package dependency, deployment will fail. 

      I was currently thinking about mixture of cf push for dev updating while using cf deploy for initial or production deploy env, but i will give partial deploy -m a try.

      Regards Holger

      Author's profile photo Martin Koch
      Martin Koch

      Hi Marius

      thanks for the great article!

      Can you please be so kind and tell me what are the license requirements on SAP Cloud Platform to run the CAP productively ? Which HANA License is required and which application runtime license.

      Thanks!

      Kind regards

      Martin

      Author's profile photo Marius Obert
      Marius Obert
      Blog Post Author

      Hi Martin,

      thanks for your feedback.

      I'm sure you can understand that I cannot give you a definite answer to the question as I'm not a lawyer ?. What I can tell is that CAP is currently distributed under the SAP Developer License 3.1 which afaik doesn't have a legal requirement on a certain runtime or HANA license.

      The most straightforward option currently is using CAP on SAP Cloud Platform, Cloud Foundry with the Haas (HANA as a service) DB.

      Author's profile photo Martin Koch
      Martin Koch

      Thanks for your fast response!

      We are currently developing partner apps intended to be sold via the sapappcenter.com and therefore i wanted to check in advance if there is any special requirement. As we want to use features like fulltext search on Hana i guess we will need the enterprise edition.

      Kind regards

      Martin

       

      Author's profile photo Gregor Wolf
      Gregor Wolf

      Hi Marius,

      thank you for making me aware of this possibility. Unfortunately this options are not available for the SAP HANA xs command line. I've opened an improvement request to get this fixed: Provide xs command line option for partial deployments.

      Looking forward for some upvotes :-).

      CU
      Gregor