The Detroubulator team has put together a set of best practices for using Detroubulator. We hope they will provide you some inspiration and help guide your use of Detroubulator.
No. 10: Share your test files
The value of your test files should increase as time passes. Every time a new test case is added in connection with changes or bug fixes, this increases the value of your tests. In order to reap the full benefit of unit testing, it is essential that every developer working in XI has access to all the test files. The most effective way to ensure this is to keep all the files in a shared directory or, better yet, in a versioning repository such as CVS or Subversion.
No. 9: Know your assertions
Assertions are the part of the test case that describes your expectations about the output of the mapping program. To write the best possible tests, you should understand how to use the different assertion types provided by Detroubulator. Here’s a brief rundown:
- Content assertion. This is the most frequently used type and describes what a particular part of the mapping output should look like. Content assertions use XPath expressions that refer to the output document.
- Exception assertion. This is used for test cases where you expect a mapping program to fail. In other words, this is used to describe a negative test.
- Validation assertion. To validate the mapping output against an XML Schema, use a validation assertion.
- External assertion. This last assertion type allows you to write any assertion you like in Java. Use this (only) when none of the other assertion types can be used to express what you are trying to test.
No. 8: Join the mailing list
The Detroubulator mailing list is the place to ask questions or give suggestions about Detroubulator. To join the mailing list, visit http://groups.google.com/group/detroubulator-users. The mailing list has a low traffic volume and is 100% spam free.
No. 7: Add comments to your test files
Test files are not just input to Detroubulator. They can also be viewed as executable documentation describing how your mapping programs should work. But while any given test case will seem intuitive and logical while it is being written, it may not seem obvious from someone elses point of view or even from your own a year later. Therefore it is always a good idea to comment the test cases as they are written. Comments should describe which part of the mapping program a test case is designed to test and why this is being tested.
No. 6: Tests are (almost) free – take advantage
Once a set of good test cases has been defined, it is possible to execute a comprehensive test of a mapping program in a very short time. Thus, the cost of re-testing is very low. Contrast this with manual testing, which takes an inordinate amount of time to obtain the same comprehensive coverage. It makes sense to make testing an integral part of your development process in order to catch problems as early as possible and make regression testing (almost) free.
No. 5: Mind the context
Message Mappings in XI are simple and easy to get started with. SAP has managed to deliver a tool which really does make simple mappings simple to implement. On the other hand, the complexity of the Message Mapping implementation rises quickly for less simple mappings. And automated testing becomes ever more important with rising complexity, due to the increased risk of introducing errors.
From experience, the concept of context in Message Mappings is one of the largest contributors to errors and misunderstandings. And with good reason: source element context (and the consequences of changing it) can be extremely difficult to understand. So it is a good idea to define good test cases for those parts of a Message Mapping where context is in play.
No. 4: Maintain your tests
The value of your test files increases over time, but a prerequisite for this is that they are actively maintained. This means that every time a mapping program is augmented or an error is corrected, a corresponding test case should be added to cover this change. If test files are not maintained, their value is diminished as you can no longer know how comprehensive your tests are.
No. 3: One input document is never enough
When you start to develop a new mapping program, you will typically have only one or a few input examples with which to run the program. This is fine as a start to gain an overall picture of how the mapping program will work, but it is far from enough for a comprehensive test.
To build a comprehensive set of tests for a non-trivial mapping program, you need to identify the critical areas on which to focus the test. These are the areas where you expect the mapping program might fail. After identifying these areas you need to generate input documents to exercise them. The number of combinations can grow quite large and at some point you do need to stop – you can never test absolutely everything. But one thing is certain, a single input document will never be enough for a comprehensive test.
No. 2: Choose your tests carefully
There will always be a limit to the amount of time you can spend on ensuring the quality of your mappings. By using Detroubulator, you can use the given time more effectively but it will, nevertheless, be limited. So it is important to focus your testing efforts on the areas where you will get the highest value from your tests.
One guide to this is to consider the strengths and weaknesses of the type of mapping program you are writing. We noted (in number 5 above) that context is one point of difficulty for Message Mappings, making this a good area to focus on when working with them. Conversely, there is usually little point in testing the transfer of a single header field of the source document into a header field of the target document. There is no reason to suppose that such a simple part of the mapping program is prone to fail in a subtle way, so there is little reason to spend part of your limited time testing this.
It is easy to write scores of test cases that do no fail, but the total value may be correspondingly meager. Good tests are those that actively seek to provoke failures!
No. 1: Make testing an integral part of your work
To maximize the benefit of automated testing, it is a good idea to make testing an integral part of your entire work process. In number 6 above, the point is made that testing should be an integral part of developing mapping programs, but it is equally important when dealing with errors.
Consider this scenario. An error report has just landed on your desk: a mapping program is failing in the production environment. Instead of immediately beginning to debug the program, you instead start by writing one or more Detroubulator test cases to reproduce the error seen in production. When executing your newly updated tests, the new test cases fail while all other tests (hopefully) pass.
Now you begin to correct the error and iteratively execute your tests. When all the test cases pass, i.e. the old and the new, you can be confident that you have corrected the reported error and that you have not inadvertently introduced any new errors in the process. As an added bonus the total quality and value of your tests has increased!
Note that this “test first” approach can be applied in connection with any mapping program changes, not just those initiated by an error report.