Skip to Content
Technical Articles
Author's profile photo tiago almeida

Reducing MTA build times – from 10 minutes to 30 seconds 😎

Introduction

Multi target applications (or MTA) are great. They allow us to package and fully describe an entire solution comprising of frontend applications, backend services (e.g. CAP services), database artefacts, required cloud services, etc. all in a declarative manner in a single, readable file.

You’re probably using them already as they’re used by default in SAP project templates. For example, when you start a new CAP project it comes with an mta.yaml.

With SAP’s mbt tool you can package the mta app into a deployable mtar file and deploy that into the cloud fairly easily.

All services are created and linked between themselves for you. This makes it much harder to have issues caused by updating certain parts of the live system without updating others.

The problem

One downside of using mta files and the mbt tool though is that build times can get very long.

It is not uncommon for your typical CAP project to take 10 minutes to build. All it takes is having many simple frontend apps, an approach that is natural if using Fiori elements for developing apps.

A common solution to this is to split your project into multiple mtas, maybe one mta for frontend and one for backend, or an mta per “function area” but in general that is something I avoid if possible as it negates the advantages of having mta files in the first place. I.e. forcing different inter-dependent components to be updated together.

So how can we reduce the build time of large MTA apps? In this post I’ll present two approaches  but would love to know of others. Please share in the comments below!

 

Parallelization

When you right click on the mta.yaml file of your project in BAS and choose “Build MTA Project”,

Menu

Menu

BAS is actually only running a single command:

mbt build -s '/home/user/projects/xyz-project'; sleep 2;

You might see this appear in the terminal at the top of the build process and if we run that command on the terminal, we’d start a build. Perfectly equivalent to the mouse right click option.

 

The mbt build tool is documented here and has a few additional flags. An interesting one is -j. . This option configures the number of Make jobs that can run simultaneously. It requires -m (mode) set to verbose. The full command would be something like this:

 

mbt build -s '/home/user/projects/xyz-project' -m=verbose -j=8; sleep 2;

 

What this does is the same build as before but starts multiple jobs at the same time. Instead of building one module at a time, sequentially, it builds J at the same time.
If your project has many modules (.e.g many frontend apps ) it can build up to 8 of those frontend applications at the same time.

The maximum number is 8. In an ideal world this would cut your project’s build time to 1/8th. However in practice you may get closer to 1/2 or 1/4 of the original build time.

There are 2 things I don’t love about this approach. One is that the output of the build jobs gets intermingled so its nearly impossible to read when a build issue occurs. The other is that this option is flagged as BETA in the documentation.

In my experience I’ve never seen it fail but you never know if it will produce a broken build. I use this when building locally in BAS but never when doing builds in a CI system.

 

Being lazy

The limitations of the -j option above got me thinking a bit deeper about the build process and how everything is packaged into a single mtar in a typical node.js CAP project. In general:

  • Each module is built independently – each ui app goes into its dist/app.zip file
  • All built modules are packaged in an mtar file

In the case of your typical CAP project with many UI apps the largest contributor to the overall build time are the UI applications, and in particular doing an npm install for every single one of them.

However, in the vast majority of situations when we’re doing a build, our apps haven’t changed. Maybe one or two have changed but not the majority of them. So why is the tool rebuilding every single app instead of using the intermediate results of the previous build?

 

The nice thing is we can customise how to build apps in an mta project.

 

In the mta.yaml we go from the default:

- name: mygreatuiapplication
  type: html5
  path: app/my_great_ui_application
  build-parameters:
    build-result: dist
    builder: custom
    commands:
    - npm install
	- npm run build:cf
    supported-platforms: []

to a custom script

- name: mygreatuiapplication
  type: html5
  path: app/my_great_ui_application
  build-parameters:
    build-result: dist
    builder: custom
    commands:
    - ../build.sh
    supported-platforms: []

 

What is this build.sh cuustom script? This shell script will be created in the /app folder of the project and it will:

  • calculate the most recently changed source file in that application
  • calculate the create/change date of the packaged application zip file
  • Perform npm install/npm run build only if the source files are more recent than the zip file that may exist in the dist folder.

Because of the above observation that in most builds only a small subset of files/apps have changes, this allows us to skip build for most apps, reducing the build time drastically.

Using this approach we see typical build times going down from 10minutes to around 30 seconds.

 

Note: I’m leaving build.sh as an interesting (BA)SH exercise to the reader 🙂 Here are a few tips:

  • get the name of the most recently changed file perhaps using find -type f and the -printf flag to also print changed timestamps.
  • Use sort/tail/cut or some awk magic to get the most recently modificated file from the above list.
  • Use bash if [A -nt B] to see if file A is “newer than” B.

 

Which techniques do you use to speed up MTA builds? Please share in the comments! Thanks

Take care

Assigned Tags

      5 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Volker Buzek
      Volker Buzek

      nice approach!

      And wouldn‘t it be cool if mbt would deliver this extremely useful optimized build feature out of the box?! Hard to comprehend from the outside why feature-related work on that tool has not happened for years

      Author's profile photo tiago almeida
      tiago almeida
      Blog Post Author

      Thanks Volker.

      Author's profile photo Jesse van der Velden
      Jesse van der Velden

      Are there any specific docs for this?

      Author's profile photo Jesse van der Velden
      Jesse van der Velden

      Hi Tiago, cool approach! I was actually thinking something similar about about caching the builds.

      How do you cache these zip files of the previous builds. Do you commit them in the repo or how does your CI/ CD look like actually?

      Author's profile photo tiago almeida
      tiago almeida
      Blog Post Author

      The dist/app.zip files stay there in BAS by default.

      We don't commit them to git repo. Technically you could but it's a build result so not advised to put in source control.

      On the CICD side, depending what you use you might be able to configure it to reuse the previous environment, in which case those dist/app.zip files will still be there.