Skip to Content
Technical Articles
Author's profile photo Marcello Urbani

The journey to Git-based ABAP development: Part 1

At Basis Technologies our tools are built in ABAP, so abapGit has been on our radar for a while and we’ve been fiddling with it for quite some time (if you’re not familiar with what abapGit is you can learn more here or here). But so far the only ‘productive’ use has been to periodically commit any changes done in development. This has some value for auditing (we can see who changed what and when or compare dev with production, down to the line of code) but it’s a far cry from the holy grail of developing like the rest of the world, with branching and CI pipelines.

However, we recently started the journey towards using abapGit for actual product development. We’ve seen a lot of potential in a Git-based approach for some time but getting it up and running successfully was always going to take some work, so we needed some sort of significant reason to get started. Recently we started planning a large piece of development which will affect most of our codebase and we think that with abapGit we might be able to make it much more efficient, but we are in the early stages. We’ve already learnt a lot but we’ve also hit a few bumps in the road.

Our goal is to adopt for ABAP on premises a workflow similar to what SAP recommends here for mta applications on cloud platform

The first step has been running SAP instances in containers. It’s not absolutely necessary, but since the plan is to give each developer their own personal dev environment, they’re an effective way to reduce costs and simplify onboarding: our system can spawn a brand new instance with about a minute of work and thirty minutes of waiting. For now they’re just a convenient way to quickly spin up and dispose of identical systems.

 

While not supported by SAP, it does work pretty well, but is quite different to how most people use containers:

  • Images containing the database have bigger layers than what most registries accept (around 50 GB vs 10 GB for Amazon ECR). You can run your own to overcome the limitation but it’s one more server to manage. We opted for leaving the database in a volume which has advantages but makes creating new instances a bit more complicated
  • The hostname has to be enforced
  • Every service needs its own IP, as mapping different instances to different port numbers doesn’t really work

 

On the bright side none of this is a showstopper when you run it on your own machine. Looking properly into Amazon ECS and similar is on my to do list but what we have is good enough to start with.

 

Our plan is to:

  • Develop in private boxes owned by developers 
  • Commit to a git repository, using feature branches
  • When development is ready
    • Run as many automated tests as reasonably possible. Go back if failed
    • Merge on a PC
    • Import on the developer’s laptop to validate it activates, fixing issues if needed
    • Pull the merge on a dev system where a transport will be generated
    • Run some other test/ consistency check
  • Deploy downstream with transport based tools

 

Manual QA might sit downstream or after the automated tests.

 

Last year we demoed a prototype where this merge operation was automated via a Jenkins pipeline, and we definitely want to do that in production in the long run. But the first step is to get this working manually 

So we imported our codebase into one box and abapGit serialized to about 13000 files. Then we imported the same code on another, identical box. We will use one as private dev and one as the central dev system that feeds production

First surprise: abapGit found differences between the two as the import process didn’t completely fill the tables abapGit relies on and so it failed to pick up some includes. The ABAP repository tables didn’t list them, and they didn’t show up in SE80 until we reactivated the object, so we reactivated the whole thing on both sides, committed again and finally managed to get the 2 systems in sync.

We imported the code in the second box because pulling from abapGit from scratch doesn’t work. There are too many activation issues, many with circular dependencies. I guess for now we are stuck with transports plus mass activation to kick-start. Switching between branches usually works fine, so we will move forward with our journey towards CI.

 

We’re going to keep experimenting to see if abapGit can help the way we hope in the large development project we’re planning. The next step will be to do some development in the first box and then generate a transport on the second. This will then be merged on the main development track (we already have a system to detect conflicts with local developments).

 

I’ll be sharing our progress in a series of blogs like this one, so watch out for more over the next few weeks as we figure out whether abapGit can fulfill the potential we think might be there.

 

Continues in part 2

Assigned Tags

      20 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Jesús Antonio Santos Giraldo
      Jesús Antonio Santos Giraldo

      Nice blog and good look.

      Really looking forward for your progress on this.

       

      J.

      Author's profile photo Lars Hvam
      Lars Hvam

      nice

      feel free to open issues with the cyclic dependencies, sure that some parts of it is fixable

      Author's profile photo Marcello Urbani
      Marcello Urbani
      Blog Post Author

      Thanks

      Main issue is that the codebase is pretty big, I don't really know where the issues are, I just get thousands of errors and have no idea what the root cause is. Might try to reproduce on a small scale if I find the time.

      For now I will wait until I bump into a smaller case, as soon as we start using this workflow for real they should crop up sooner or later.

      Same for the indexes. Happens often in the wild, hard to recreate manually.

       

      Author's profile photo Jakub Filak
      Jakub Filak

      We workaround the problem with huge pulls by creating development docker images from "main dev" systems.

      Author's profile photo Marcello Urbani
      Marcello Urbani
      Blog Post Author

      We plan to do the same, but it's still a pain because it's not a great option for some scenarios, and because seeding with transports forces us to use non local packages which are a pain in the personal boxes

      We also plan a small utility to migrate the whole package tree to sort it, we should only need to use it once but don't like doing this sort of thing manually

      Author's profile photo Jakub Filak
      Jakub Filak

      I hope your utility will be based on abap-adt-api and will be open source.

      Author's profile photo Marcello Urbani
      Marcello Urbani
      Blog Post Author

      More likely to be an abap program, I see little value in running this out of the box you need it in.

      Another one I plan to use for this and will try to get my boss to open source is a code formatter.

      Our codebase is not consistently formatted, and while the SAP diff engine is good enough ignoring case and spacing the rest of the world will give you a lot of noise.

      Author's profile photo Jakub Filak
      Jakub Filak

      Only ABAP tools are not so easy to deliver, test and version (since ABAP platform has different goals and different benefits) -  hence, all my tools are written in non-ABAP if possible.

      Author's profile photo petiare nic
      petiare nic

      However, I think that it would be productive for ABAP only when a technical solution for local (active) branching is implemented.

      Author's profile photo Marcello Urbani
      Marcello Urbani
      Blog Post Author

      The idea is that every developer will have their own dev system and git repository, and only be able to affect the main one with pull requests.

      And will do most merge/rebase operations on their laptops, connected to both the main repo and the personal one, using their dev system to test the result

      I see a lot of challenges, but I think this one is basically sorted

      Author's profile photo David Encinas Fernandez
      David Encinas Fernandez

      What about customizing? How do you handle that with containers and abapgit only convering the development?

      Author's profile photo Marcello Urbani
      Marcello Urbani
      Blog Post Author

      That's the elephant in the room.

      We have 2 possible solutions to explore:

      1. use BC sets. Looks like the right thing to do, but we never tried
      2. do that in the TMS landscape
        1. move customising to preprod/prod with transports
        2. test it in preprod
        3. periodically release a 'project customising pack' transport to import in the personal systems

      Doubt they will work as smoothly as code but (2) is not as bad as it looks as we use our own software to sequence transports and prevent conflicts between local development and incoming git changes

      Author's profile photo David Encinas Fernandez
      David Encinas Fernandez

      I wonder how they do it on the cloud. Latest version on premises has already native supported fot Git in STMS but no support for customizing neither.

      https://help.sap.com/viewer/4a368c163b08418890a406d413933ba7/201909.000/en-US/f319b168e87e42149e25e13c08d002b9.html

      Author's profile photo Jakub Filak
      Jakub Filak

      gCTS should be capable of versioning of everything transportable.

      Author's profile photo David Encinas Fernandez
      David Encinas Fernandez

      Not according to note 2821718.

      Author's profile photo Marcello Urbani
      Marcello Urbani
      Blog Post Author

      Not according to the presentation Harald Kehr did in Vegas. I even asked him directly and confirmed it wasn't there yet / didn't have a release date

      Author's profile photo Lars Hvam
      Lars Hvam

      3. implement the functionallity in abapGit

      Author's profile photo Marcello Urbani
      Marcello Urbani
      Blog Post Author

      Yep.

      The main issue I see here is defining the requirement. More specifically assigning customising changes to a package.

      I think it can work reasonably well for most use cases with some customizing/coding, I'm skeptical on a zero configuration solution.

      For gCTS too btw

      Author's profile photo Alexander Tsybulsky
      Alexander Tsybulsky

      Cool, really cool.

      How do you solve the issue of data dictionary stuff that is missing in npl system by the way?

      Author's profile photo Marcello Urbani
      Marcello Urbani
      Blog Post Author

      Our application does transport management and doesn't need them.

      But we aren't using NPL, we installed our own box and replicate that