ABAP Unit tests, monitoring and integrating them into pipelines
Somewhere in the past, after a year practicing Clean Code and TDD as a team, we were able to cover around a third of our code base with automated unit tests. At some point though, I began to wander what wasn’t being covered by tests and began to notice we would often break some tests and not notice it. And although I could see an universe of tools to help write, execute and monitor tests in other technologies, I was not fully satisfied with what we had in ABAP.
Since ABAPGit came out, many open source solutions in ABAP started sprouting in Github. And now, after taking so much from the open source community I decided to give something back and hopefully motivate more teams to onboard their own journey towards the excellency modern software development demands.
Following some principles in continuous delivery and DevOps, we would from time to time deliver monitors of some of our applications and interfaces to help our support team to be pro-active and solve issues before they were reported.
However, we lacked something for ourselves, something that would help us nurture our code base to the better, find the corners that were not tested and allow us to be sure we were always green (as in all tests passed). Also, I knew having something visual every day would engage ourselves more into keeping things clean and green.
Thus, I decided to start working in a monitor just for that. And so, our ABAP Unit Dashboard, or ADASH, was born:
With it, we are able to navigate through our code base, understand which change broke what and how we are looking in terms of coverage. We can re-execute the tests on a specific package (or class) on demand too. With the Cli (see below), we are able to set group of packages so we can have different teams monitoring different domains. And at last, we can take a look at precisely which test has failed and why:
And all it takes to start the monitor is a
adash mon -s dev
provided that you had setup your dev system, of course ;).
The road to ABAP CI/CD pipelines continues
There might be some technical elements that still prevent the ABAP of achieving full continuous integration. But for continuous delivery, a lot has come up together in the past years. ABAPGit and ABAPLint are some examples on that direction. One thing that we were missing though, was integrating our tests in our internal Gitlab pipelines. The idea came from our front end developments, we had front end tests and would deploy the application to the backend if they passed, however we would do it regardless of the backend tests failing or not and so our CLI (Command Line Interface) came to be.
As the CLI was built using Node Js, it should be fairly easy to create pipelines that use it. And again, after setting it up, all it takes is a
adash testp yourPackage -s dev
to execute the tests remotly and get a quick feedback,
Bonus: A watcher
A fair warning
All is available and free through the ADASH Cli, keep in mind that this is by no means a commercial tool, nor one that reached (in my opinion) a V1.0 maturity. There are some bugs to fix, tests to implement (ohh the irony), and functionalities to add. I know summary tiles and vizchart graphs showing history of test and coverage over time are on the make, more than that, I was hoping to engage the community into collaborating to it.
This is but a tool, helping us to have a collective ownership of our code, and it runs the tests we wrote. It doesn’t, by any means, test things “automatically” or statically like ATC.
During a decade I would see and envy other languages ecosystems and communities, how vast their collaboration was and how easy it was to consume thousands of hours of other peoples work to greatly increase our own productivity. But, in the past 5 years I was able to see how fast our own ABAP community grew and how that started to become true to us too. UI5 came to the rescue in the front-end part and in the end all languages have advantages and disadvantages.
ABAP is renewing itself, as are we, who use it. XP Programming, Test Driven Development, Continous Delivery, Continuous Integration and DevOps are just some of the words that describes modern software development (and its lifecycle). And while its full adoption might be technically restricted with ABAP, there is huge difference of what we can achieve now and what we could achieve 20 years ago, and a lot is coming together thanks to this two way road called open source.
It is true that projects and products that use ABAP are generally restricted by rigorous processes and tools…
But, from my experience, the barriers to TDD (for instance) are more personal than technical or managerial. It is quite easy to find excuses and continue doing things the same way. It takes bravery to scrap years of function module experience to dive into OO, Testable Design and Test Driven development. ABAP for sure is ready for us to step up. Now we even have ABAP in the cloud! I hope projects like this inspire you to be better, as much as the open source projects I used inspired me to be better.
What do you think? Leave your comments below!
Hi Felipe, thanks for sharing your thoughts and work! The CI/CD topic has been on my mind for some time. As a “classic” ABAP developer it’s sometimes a big puzzle for me, but I am working on the overall picture 🙂 Your blog is a nice help.
As you wrote, you have to change your attitude and be open to new things. Go on a trip, so to speak. Get out of the habit and into small adventures
Greetings to Lyon
Small adventures are the best, we are lucky to work in software development, there is always something to do better, new ideas, technologies!
There is a lot of material, and I recommend this blog if you want to see some of the challenges in ABAP to fully achieve it 😉
Greetings to Saarbrucken! Been there once to visit the beatifull SAP Labs around the area.
I am interested in the "continuous integration is not possible in ABAP". As I understand it in other languages you develop your code on your local machine, and every so often you merge that code into the main system and the more often you do that the better.
A minimum of once a day is what I have often heard touted. We in ABAP world have difficulty getting our heads around that as there is only the main system as code is developed int one central repository as opposed to on local machines.
So in essence every time you press "activate" in ABAP you are attempting to integrate your new change with everything else.
Continous integration for sure is the biggest challenge. And you mentioned one of the reasons, is the existance of a single active version of any given program. Other systems are more permissive, or at least will not dump if you change a program in the middle of its execution, such is the case with ABAP. You need to choose specific moments to go to prod to avoid disrupting users. Large projects block systems for entire weekends or more, for instance. Of course, if you go often to prod, there will not be much to be disrupted. Even so, the max I was comfortable with was weekly releases during a short period of time where no users in the different timezones we support would be expected to be using our systems.
In many technologies, specially the ones that allow you build solutions as “microservices” you have the concept of “A/B” or “Blue/Green” instances. There is two different productive codes for a period of time. “A” is there, and then you randomly have “B” instances of your new code to be used instead, you monitor it closely and slowly let “B” instances take over and “A” instances to die and don’t be used anymore.
There is this blog explaining a bit more in depth the number of the issues. My only remark is that for me Continous Integration relates to your ability to deliver automatically into your productive systems, while Continous Delivery is about delivering them automatically for them to be integrated manually after. So you build your functionality, automate the tests for it, have a manual check and go. Next time, you are relying on your automated tests for whatever you tested and only confirm the functional correctness of the new functionality and rely on its automated tests next time. So no more NRT (Non regression tests) everytime you make a change.
As you will see in the blog, there are many issues in ABAP, as for example the amount of time it takes to activate everything if you change an structure that is used in other structures or tables, for example.
But mainly, a gigantic issue is the fact you cannot easily revert a change. On every “top notch” continous integration approach, even your database changes come as a “delta”, “add this field”, “change this setting”, when you revert you are back to your original state. In ABAP, sometimes we just live with a wrong change for the sake of not reverting database changes, it is chaotic and you often find yourself doing extra hours to fix it, and sometimes just praying for the backup because that small change you did ended up loosing data.
For me, this whole “activation” concept is also the may reason companies have expensive pre-production systems. In many landscapes is the only way you can ensure the changes you are doing (and the order you are doing them), are going to work and activate as they should in production.
Indeed, in any safe continous integration approch, is the ability to run your tests as close to production environment as possible, that gives you the ultimate trust that whatever is green on your tests will be green in production too. And I’ve heard of AWS services that could do that for you even with SAP, but never seen one and I imagine the costs are yet restrictive to be used to give the kind of fast feeback all of this (TDD,CI/CD…) is meant for.
So to resume, there is no ideal world for Continous Integration and ABAP just yet. But you can with a lot of hard work achieve continuous delivery. Automated Unit Tests in Dev -> Automated Integration tests in Quality (on a specific client for it) -> Deliver to pre-prod so you can do your final checks.
But you can only achieve it with small increments, if you are doing a full project, your tests are serving something else, not the continous delivery (or integration) of value.
Ahh...and how what I did relates to all of this. This continous delivery and integration are done through called "pipelines" that execute all the checks and tests needed for that code to move to the next stage. These pipelines usually triggered by your versioning system.
On this case, what I did is target for versioning systems like git. Oversimplifying: So you have your central repository somewhere (Github/Gitlab/SVN,etc) and you want every change that is pushed to the "quality" branch to be automatically tested and pushed to quality if green. For that, you have the CLI of this blog. That is able to run your test in the system you want, therefore allowing you to have a pipeline with them.
I would just like to thank you for a wonderful response. The term “continuous integration” often seems to mean different things to different people,and for ages now I have been trying to square a circle.
When I read about CI in Java or something the article is almost always about merging code from local machines onto a central system.
I knew CI could not mean that in ABAP, which is why I asked the question.
I am sure to a lot of ABAP developers what CI means in an ABAP is so obvious it doesn’t bear thinking about At school I was always taught “the only stupid question is the one you do not ask” so I asked it, and I am glad I did.
Could not agree more, the only stupid questions are the ones you don’t ask. Hmmm…that is why I don’t mind looking dumb every now and then.
And your question was far from stupid, maybe CI is a concept with multiflavored implementations. All converging to some general ideas and goals, but not on how…
Great to see this blog from you and thanks for writing it!
It's a productive addition to ABAP projects where TDD is being applied.
Thank you for the support!