The journey to Git-based ABAP development: Part 1

At Basis Technologies our tools are built in ABAP, so abapGit has been on our radar for a while and we’ve been fiddling with it for quite some time (if you’re not familiar with what abapGit is you can learn more here or here). But so far the only ‘productive’ use has been to periodically commit any changes done in development. This has some value for auditing (we can see who changed what and when or compare dev with production, down to the line of code) but it’s a far cry from the holy grail of developing like the rest of the world, with branching and CI pipelines.

However, we recently started the journey towards using abapGit for actual product development. We’ve seen a lot of potential in a Git-based approach for some time but getting it up and running successfully was always going to take some work, so we needed some sort of significant reason to get started. Recently we started planning a large piece of development which will affect most of our codebase and we think that with abapGit we might be able to make it much more efficient, but we are in the early stages. We’ve already learnt a lot but we’ve also hit a few bumps in the road.

Our goal is to adopt for ABAP on premises a workflow similar to what SAP recommends here for mta applications on cloud platform

The first step has been running SAP instances in containers. It’s not absolutely necessary, but since the plan is to give each developer their own personal dev environment, they’re an effective way to reduce costs and simplify onboarding: our system can spawn a brand new instance with about a minute of work and thirty minutes of waiting. For now they’re just a convenient way to quickly spin up and dispose of identical systems.

While not supported by SAP, it does work pretty well, but is quite different to how most people use containers:

• Images containing the database have bigger layers than what most registries accept (around 50 GB vs 10 GB for Amazon ECR). You can run your own to overcome the limitation but it’s one more server to manage. We opted for leaving the database in a volume which has advantages but makes creating new instances a bit more complicated
• The hostname has to be enforced
• Every service needs its own IP, as mapping different instances to different port numbers doesn’t really work

On the bright side none of this is a showstopper when you run it on your own machine. Looking properly into Amazon ECS and similar is on my to do list but what we have is good enough to start with.

Our plan is to:

• Develop in private boxes owned by developers 
• Commit to a git repository, using feature branches
• When development is ready
  • Run as many automated tests as reasonably possible. Go back if failed
  • Merge on a PC
  • Import on the developer’s laptop to validate it activates, fixing issues if needed
  • Pull the merge on a dev system where a transport will be generated
  • Run some other test/ consistency check
• Deploy downstream with transport based tools

Manual QA might sit downstream or after the automated tests.

Last year we demoed a prototype where this merge operation was automated via a Jenkins pipeline, and we definitely want to do that in production in the long run. But the first step is to get this working manually 

So we imported our codebase into one box and abapGit serialized to about 13000 files. Then we imported the same code on another, identical box. We will use one as private dev and one as the central dev system that feeds production

First surprise: abapGit found differences between the two as the import process didn’t completely fill the tables abapGit relies on and so it failed to pick up some includes. The ABAP repository tables didn’t list them, and they didn’t show up in SE80 until we reactivated the object, so we reactivated the whole thing on both sides, committed again and finally managed to get the 2 systems in sync.

We imported the code in the second box because pulling from abapGit from scratch doesn’t work. There are too many activation issues, many with circular dependencies. I guess for now we are stuck with transports plus mass activation to kick-start. Switching between branches usually works fine, so we will move forward with our journey towards CI.

We’re going to keep experimenting to see if abapGit can help the way we hope in the large development project we’re planning. The next step will be to do some development in the first box and then generate a transport on the second. This will then be merged on the main development track (we already have a system to detect conflicts with local developments).

I’ll be sharing our progress in a series of blogs like this one, so watch out for more over the next few weeks as we figure out whether abapGit can fulfill the potential we think might be there.

Continues in part 2