A Practical Guide to DevOps for SAP ERP
If you like this post, you might enjoy the follow up. IaC enables us to deploy new systems with only two lines of code. https://blogs.sap.com/2021/12/17/infrastructure-as-code-devops-and-the-future-of-enterprise/
We’ve been applying DevOps for SAP to our product development and couldn’t be happier with the results. Below is a high-level overview of how we implemented DevOps for our team, and I hope it helps move the practice forward and gain adoption within SAP.
- Run SAP ECC in a Docker container and allow each developer to run an isolated system.
- Use abapGit to save ABAP code with a code repo such as GitHub.
- Leverage ABAP Unit to execute unit tests and measure coverage.
- Configure GitHub Actions to
- Run all tests and ensure minimum coverage
- Enforce syntax rules with abaplint
- Require peer review
- Spin up a “QA” system for the test team to use
- Code that makes it to the main branch is production-ready code
Feel free to comment if you have any questions.
Containers Change the Game
If you’re not familiar with containers, you can think of them similarly to virtual machines. A virtual machine allows you to run an operating system, and a container enables you to run a single program. In our case, the goal was to run SAP ECC in a container. One advantage of containers is that they are quickly replaceable. For example, you can break your system and start a new container in about three minutes. This is possible because containers are based on images. Images are like “snapshots” that store the system in a known state.
Each developer, functional resource, or tester can use containers to run their own SAP system. Running SAP in a container environment provides system isolation, so work never impacts others. Isolation is a massive advantage as you don’t have to worry about object locks, overwriting data someone else was using, or creating inaccurate results if your work depends on theirs.
There are several options to run containers, but one of the most popular is Docker. We built our containers using Docker mainly due to the large community and trusted name.
You have multiple images available to use:
- SAP ERP 3 years of data
- SAP ERP 1 month of data
- SAP ERP no data
- SAP S/4HANA 2020
You receive a requirement to format all the phone numbers in your SAP ERP system to match the country’s format for that customer. You start a container based on “SAP ERP 3 years of data” and write your first draft of the logic. You test it individually for 12 phone numbers, and it works. Then you decide to run it on all 125,000 customers and realize that it deletes every other phone number.
In a traditional scenario, this might make any developer panic. However, container isolation means you only impacted the current container you are running. You decide to remove that container and start a new one, again based on the image “SAP ERP 3 years of data,” so all of your customer phone number data is back to the original state after 3 minutes. Time to try again.
The Missing Source Code Manager
The keen reader might wonder, “what happens to my code if I remove a container?” Any changes you make to a container are wiped when you remove it. The new container is a replica of the image you use, introducing a new problem – how do we save our code?
abapGit to the rescue! abapGit is an opensource git client for ABAP. It allows us to use a tool such as GitHub for source code management. Instead of transports, we use abapGit to manage our code changes inside a container. There are countless tutorials on git and GitHub, so I will only cover the basics.
Git uses the concept of repositories which are very similar to SAP packages. Each repository corresponds to an SAP package. By default, your repository has one main branch considered the definitive branch. You can create feature branches to experiment with before committing it to the main branch. When you begin working on a feature or bug fix, you create a branch (e.g., my-feature-branch.) Any code changes you make are committed to this branch. The branch is now part of the repository and visible to others who have access. Now you can start a new container and pull your feature branch into the new container.
Before removing my container, I create a branch “nl-format-phone-numbers” and commit the changes. Then I can safely remove that container and start a new one. When the new container spins up, it will pull the main branch by default, but I can navigate to t-code ZABAPGIT and change the branch from main to the nl-format-phone-numbers branch. I can now pull my feature code and continue working where I left off.
Goodbye Test Team
As a developer, reducing the cycle of code, test, debug, repeat is crucial. The reality is that testing is difficult, and the feedback loop when humans are testing is far too long. Automated testing helps reduce this cycle and deliver better software.
Testing requires an organizational change. ABAP unit testing is not new in SAP but is rarely used. Using automation to enforce unit testing can train developers to follow the recommended practice. It’s too easy in a standard development workflow to skip the tests. Using GitHub Actions, you can make it a requirement to have adequate test coverage and tests that pass to merge a feature branch to the main branch.
Writing tests upfront is more work, but it’s well worth it in the long run. Tests don’t need to be overly complicated. The benefit is realized because they are run in aggregate by a machine and expose any dependencies across projects. If you do not have automated testing, a human needs to be aware of all the dependencies and manually test them when any change is made. This manual approach is not reasonable or sustainable.
The “Goodbye Test Team” is said tongue-in-cheek. In reality, we’re just shifting the test team’s responsibilities to help develop tests cases before development begins. The updated test process allows the developer to write code to test valid business scenarios and provide sufficient coverage. It also changes the flow so that the test team doesn’t need to perform the repetitive task that a machine can perform.
- Select 50 customer phone numbers at random and validate that the final format is correct.
- If reasonable, check at least one phone number from each country and validate the final format is correct.
- Check that the total number of entries with a valid phone number are the same before and after formatting.
Clean Code Only
Putting it all together leads to the final step before merging a feature branch to the main branch. This step uses GitHub Actions and is highly configurable, but we share a proposed strategy that can be used by organizations starting their DevOps journey.
GitHub Actions allows you to automate workflows. In our case, when a developer completes a feature, they will want to merge the code to the main branch via a pull request (PR). Before allowing this, we need to:
- Execute all unit tests and make sure everything passes and the minimum test coverage is met.
- Run abaplint to confirm the correct syntax is used.
- Require a code review from another developer.
- Spin up a “QA” system that the test team can use to run any manual tests required.
This entire process should only take a few hours. It provides a better separation of duties so that once a developer completes their development, it is unlikely the code will come back to them for additional revisions. This updated workflow allows all team members to focus more on their core responsibility instead of wasting time in the back and forth involved with the legacy process.
- I complete my feature branch, run my automated tests on my local environment, and commit my changes.
- I go to the GitHub website and create a PR that triggers the workflow.
- All my tests pass, but during the code review process, a fellow developer notices that my comments are not helpful and provides suggestions on how to improve them. They do not approve the request and instead request I implement the changes.
- I update my comments, commit my changes, and add a comment on GitHub, letting the reviewer know, “Thanks for the suggestions. Comments are now updated!”
- The reviewer approves the code.
- The test team runs some manual tests and finds nothing wrong. They approve the changes.
- GitHub has everything it needs and merges the nl-format-phone-numbers branch with the main branch.
- GitHub Actions triggers another action to pull the code in dev and create and release a transport.
- The transport is safe to move to PRD and moves automatically via a batch job at 2 AM.
Change is Hard
We treat the main branch as production. Once code makes it to the main branch it is production-ready. A transport can be automatically created and released in the development system. Since the code has been tested in a containerized version of QA, it is safe to move directly to production.
Of course, there are exceptions, but these can be handled as such and outside the standard workflow. For example, you must coordinate the process and manually move transports if a manual process is required.
As a starting point, it’s good to handle transports manually to provide confidence that this modern process works and does not disrupt production. If appropriate, you can move to more automation after establishing a certain comfort level.
Photo by Frank McKenna on Unsplash
Photo by Roman Synkevych on Unsplash
Photo by Rock’n Roll Monkey on Unsplash
Photo by Carl Heyerdahl on Unsplash
Photo by Suzzane D. Williams on Unsplash
Hi Nestor Lara
Thanks for lucid explanation on DevOps in SAP World. I am very new to this world.
One quick question, Any new green field Implementation with Rise with SAP Private Cloud, What is landscape configuration. SAP Provides Dev system in Container format and If in the project, 10 Functional Consultant and 10 Development Consultant work, can we assume 20 Containers of S/4HANA System will be provided as development systems?
Please provide little more details around this.
Thanks & Regards,
Hi, thanks for your interest in the blog post.
SAP does not support running in virtualized environments. Please see SAP Note 1122387 . They do offer ABAP Platform, dev edition via Docker Hub if you're looking to learn more about containers.
Thanks for your blog post Nestor!
Something I have a hard time wrapping my head around with DevOps for ABAP development is this "Each developer, functional resource, or tester can use containers to run their own SAP system."
What are the rough system requirements for this and how does it get "distributed" to both inhouse and external - and sometimes one-off - developers and consultants who might need it? At a guess, this isn't something which just a few people can use but will need to be used by basically everybody doing development work for a given system. I may well be wrong, but have a hunch that the logistics to set this up - apart from the relevant knowledge - are not easy (and will come at a price for the needed equipment).
We wanted complete isolation, so each dev runs their own VM (4 vCPU, 32768 RAM (MiB), and 500 GB SSD).
We developed Infrastructure as Code to deploy the same VM with all required software for any new team members. This process is one line of code, and it takes a few minutes to provision the VM and install all the dependencies.
The first time you run the SAP container, you need to pull it from the container registry, which is the slowest part of the process (it needs to download then extract the image). We try to keep wait times short enough to get another cup of coffee, but the initial setup is closer to a long lunch. Initial setup shouldn't happen that often, so the wait is acceptable.
Thanks so much for taking time to write this, Nestor!
My question is similar to what Bärbel Winkler asked. How exactly do you provision individual VMs? Since we're talking about full ECC with some data and not a barebones ABAP trial system, I'm guessing it's not installed on the developer's local PC, correct? Are you using something like SAP CAL? Sorry, I'm still not clear on this part from the response to the other comment.
How do you manage access to those VMs? (And I'm also curious about internal employees vs external consultants.) Even VM is obviously not completely free, so how are you managing the costs?
Also, how do you maintain "1 month of data" / "3 years of data" environments? Do you use some 3rd party solution to slice the data? Do you update it regularly? And if so, how?
Edit: just realized your other blog linked in this post had some more technical detail but in general, my previous questions remain.
Additional question: did you implement this at an actual SAP customer or just in your own team of an SAP partner (?). I suspect at any large customer the whole VM idea would be rather difficult to implement as you'd need to get many people to agree (management, security, Basis).
Hi Jelena! I'll write a post on infrastructure as code to better answer these questions.
We're currently only using it internally. The VM's are a few hundred USD per month but still much cheaper than people. 😁 It's like having your own personal tester that responds in minutes. I think the ROI is enormous, from personal experience.
As a developer, it's also more rewarding when you have quick feedback and know that your work is being used in production. Not sitting in QA purgatory.
As I mentioned in this post, there are still some unknowns, but I don't see any significant roadblocks left from what we've encountered on our journey.
Very interesting approach and beautiful way of using abapGit. I'm all for unit test, clean code and peer review. But let me challenge your approach a bit:
We have an unpredictable behavior of the system. Let's say, in some rare cases transaction doesn't set a delivery block (or it does even if it shouldn't). I need to have a very specific test case prepared to be able to debug and fix it (and then re-test it). This test case is too complicated to be built for developer, so I get a functional consultant to make it for me. Test case will be first used by me, then by tester. Such case was time consuming to prepare so I want to share it across local instances of the system, instead of reproducing it on each instance manually. Do you have a way of doing it in your landscape?
It depends on how the test case is generated.
We use an API, so if it's possible to recreate the scenario with JSON, you could push it to every system.
If that's not possible, multiple users can access the same container using their username/password,. They just need to add the SAP connection for the test case system.
I'm sure novel ways of solving this will come up as companies adopt this DevOps landscape.
Now I get it. You work on RESTful applications, so it's much easier to prepare test cases (interfaces/signatures are established - clear contract). In that case I see that your setup can work beautifully.
In case of other modifications, we would have to think about some automated way of distributing test cases, so that people who prepare them wouldn't have too much of extra work (and thus they wouldn't be against DevOps setup). This setup has to serve not only to developers, but also to testers and module consultants.
By the way, is this setup also used for customizing? If one person has to set up some customizing, the other has to change some basis settings and third one has to do coding, do they share one VM? Is there a way to merge customizings?
I agree the distribution of test cases should be easy to make sure the setup gains complete adoption.
One option is to have regular updates of the image (from PRD) as far as customizing. This makes sure the latest config is available. If anything is needed during the development phase, it has to be a manual process. Eventually, this can also be automated. Everything is code, even the clicks on the screen, so we can get to a state of full automation.
As there has been a very high interest in the topic in our DevOps community, I am happy to announce that Nestor will host an SAP Community Call together with his colleague Mary Murphy next week on February 9, 2022 - 10:00 AM EST/ 04:00 PM CET:
DevOps for SAP ERP | Boost your productivity with CI/CD pipelines (live on YouTube)
They will offer a presentation incl. a short demo and a long Q&A part, so bring all your questions with you!