Integration Tests and Integration Test Tools

Testing Document Workflows

We perform integration tests to test software components in a group. A software component can be a single module, a part of an application or even an external system. Usually document processes are quite difficult to test because they consist of process chains like the following:

1. An online or batch program      triggers an outgoing document for a certain business object. Therefore      difficult rule sets can be used. The correspondence request is written to      a correspondence container.
2. A batch program reads the      container and creates files in the spool.
3. The spool files are copied to      an output management system.
4. The output management system creates      outgoing documents.
5. These output management systems      are given to a batch archiver.
6. The batch archiver puts those      document into an document management systems and generates information      that are used to link the outgoing documents to the business object.
7. A batch job scans those      generated IDs and generates information to link the documents to the      business objects (think of entries in the transparent table TOA01). 

Of course this scenario can get even more complicated: A customer can use different output management systems or integrate others solutions (think of Records Management). But there are even more complicated processes within the SAP systems or external systems that might be missing within the development landscape. Therefore we have different objectives for tests:

• A Business Process Expert wants      to simulate a certain scenario like integration of an additional external      system.
• A tester      or a developer wants to perform in integration test.
• A developer wants to run the complete process and emulate a certain      tool like an output management system.
• A tester wants to run the      complete process but also wants to check the content of the document      automatically and wants to transform data streams into a readable format,      too.
• A tester wants to simulate an      certain error scenario – think of the failure of an external component.

A manual test with those objectives can be very time consuming so we need test automatization. Unfortunately we there no sufficient tools for that test scenarios within SAP standard because ABAP Unit wasn’t made for integration tests and it his hard to code business logic in CATT/eCATT scripts. 

Using Domain Specific Languages for Test Automatization

When I had to find a solution for test automatization I decided to invent a Domain Specific Language (DSL) together with an interpreter that allows to run test cases. In my slides you’ll find a quite simple example for a test case together with some implementation details of the DSL interpreter. 

The DSL helps me to create test cases for a specific domain of document workflows and output processes: Using the DSL it is easy to create master data that are necessary for certain test cases. But I also added commands for scanning data sources like the spool and for checking as well as transforming its content, for simulation of events of external components and last but not least assertions so that I could code the expected behaviour within my test case.

During my talk I was asked my how I invented that DSL. In fact this was quite a natural task: It evolved from a simple interpreter that calls static methods of certain classes and later I added more commands for checking the spool content, generating PDF documents using XSL-FO, the above mentioned assertions and even more.  

Perhaps the DSL approach can be put in a more general context. If we test a BPEL process we do most of the time debugging. If we are lucky our tool supports simulation of a certain web service so that we can simulate a more complex behaviour say non-idempotent answers from a web service, wrong answers and even failures. Unfortunately manual tests take a lot of effort. If we have a test engine that run test cases we could generate the test cases for doing regressions tests, too.

Lessons Learned 

In my opinion the DSL approach has some advantages but nevertheless manual testing is still necessary. The SAP standard already contains very powerful test tools like ABAP Unit and CATT/eCATT and I recommend to use them whenever it’s possible. But if it turns out the a certain tool from SAP standard doesn’t fit to your needs then I recommend write a new tool:

• At first I tried to create a      general purpose test tool but this wasn’t accepted neither by developers      nor testers. But if you restrict to a certain domain like outgoing      documents or document processes in general then you have the chance for      effective and efficient test support by creating support for certain      domain aspects.
• Test tools should be developed      by people who use them day by day. This guarantees simplicity as well as      effectiveness of the tools. In my opinion testing (as well as developing) should      be inspiring as well as challenging task and not a painful punishment. So      be warned: Don’t let software bureaucrats create tools.