This year I was host of a session at SAP Community day about “Integration Test and Integration Test Tools” in Munich. I already blogged about Integration Tests in ABAP Development und Tool Support before – now I want to cover the main ideas in test automatization using Domain Specific Languages. Testing Document Workflows
We perform integration tests to test software components in a group. A software component can be a single module, a part of an application or even an external system. Usually document processes are quite difficult to test because they consist of process chains like the following:
- An online or batch program triggers an outgoing document for a certain business object. Therefore difficult rule sets can be used. The correspondence request is written to a correspondence container.
- A batch program reads the container and creates files in the spool.
- The spool files are copied to an output management system.
- The output management system creates outgoing documents.
- These output management systems are given to a batch archiver.
- The batch archiver puts those document into an document management systems and generates information that are used to link the outgoing documents to the business object.
- A batch job scans those generated IDs and generates information to link the documents to the business objects (think of entries in the transparent table TOA01).
Of course this scenario can get even more complicated: A customer can use different output management systems or integrate others solutions (think of Records Management). But there are even more complicated processes within the SAP systems or external systems that might be missing within the development landscape. Therefore we have different objectives for tests:
- A Business Process Expert wants to simulate a certain scenario like integration of an additional external system.
- A tester or a developer wants to perform in integration test.
- A developer wants to run the complete process and emulate a certain tool like an output management system.
- A tester wants to run the complete process but also wants to check the content of the document automatically and wants to transform data streams into a readable format, too.
- A tester wants to simulate an certain error scenario – think of the failure of an external component.
A manual test with those objectives can be very time consuming so we need test automatization. Unfortunately we there no sufficient tools for that test scenarios within SAP standard because ABAP Unit wasn’t made for integration tests and it his hard to code business logic in CATT/eCATT scripts.
Using Domain Specific Languages for Test Automatization
When I had to find a solution for test automatization I decided to invent a Domain Specific Language (DSL) together with an interpreter that allows to run test cases. In my slides you’ll find a quite simple example for a test case together with some implementation details of the DSL interpreter.
The DSL helps me to create test cases for a specific domain of document workflows and output processes: Using the DSL it is easy to create master data that are necessary for certain test cases. But I also added commands for scanning data sources like the spool and for checking as well as transforming its content, for simulation of events of external components and last but not least assertions so that I could code the expected behaviour within my test case.
During my talk I was asked my how I invented that DSL. In fact this was quite a natural task: It evolved from a simple interpreter that calls static methods of certain classes and later I added more commands for checking the spool content, generating PDF documents using XSL-FO, the above mentioned assertions and even more.
Perhaps the DSL approach can be put in a more general context. If we test a BPEL process we do most of the time debugging. If we are lucky our tool supports simulation of a certain web service so that we can simulate a more complex behaviour say non-idempotent answers from a web service, wrong answers and even failures. Unfortunately manual tests take a lot of effort. If we have a test engine that run test cases we could generate the test cases for doing regressions tests, too.
Lessons Learned
In my opinion the DSL approach has some advantages but nevertheless manual testing is still necessary. The SAP standard already contains very powerful test tools like ABAP Unit and CATT/eCATT and I recommend to use them whenever it’s possible. But if it turns out the a certain tool from SAP standard doesn’t fit to your needs then I recommend write a new tool:
- At first I tried to create a general purpose test tool but this wasn’t accepted neither by developers nor testers. But if you restrict to a certain domain like outgoing documents or document processes in general then you have the chance for effective and efficient test support by creating support for certain domain aspects.
- Test tools should be developed by people who use them day by day. This guarantees simplicity as well as effectiveness of the tools. In my opinion testing (as well as developing) should be inspiring as well as challenging task and not a painful punishment. So be warned: Don’t let software bureaucrats create tools.
