Domain Specific Language in ABAP
In my recent blogs
I have been exploring the central problem of how one computer program – or more to the point a set of several dozen programs comprising an entire solution – can fulfil its central purpose and yet at the same time cater for the myriad of legal and “we have always done it this way” requirements that each country in a multi-national company presents you with during an SAP rollout.
As a result I have been reading a number of articles by leading academics in the field of object orientated design – all from about 5 to 12 years ago which is an eternity in IT terms from such people as Robert C.Martin
a.k.a. “Uncle Bob” and the inspiration of today’s blog Martin Fowler http://en.wikipedia.org/wiki/Martin_Fowler
who wrote the book I used to get my head around the UML http://cgi.di.uoa.gr/~halatsis/Books_EDY/Book-Eng-01%20-%20Uml%20Distilled%20-%20Fowler.pdf
Both Robert Martin and Martin Fowler were founders of the “agile Manifesto” http://agilemanifesto.org/
What I try to do is to take an article about the art and practice of programming and then see if I cannot implement it straight away in an example program in ABAP before I forget everything I just read. Naturally all the examples in these articles are in Java or C++ or some other OO style language. My favourite thus far was the “design by contract” article from some time back written in a language called EIFFEL
which introduced concepts of pre-conditions and post-conditions which I promptly started using in my everyday work http://scn.sap.com/community/abap/blog/2012/09/08/design-by-contract-in-abap
They proved their worth on the very first day, just like my first experiments with ABAP Unit.
Some of these articles have a tendency to focus on problems that do not exist in an SAP environment e.g. the good old check-in / check-out / daily build procedure. Even the daily build was a radical idea once it seemed, in the days when programs took hours to compile, rather like SMARTFORMS when they first came out.
I presume those problems no longer exist but the articles suggest that once upon a time they were so severe that people were making decision decisions in their programs based on those technical limitations. I might ***** and whine about SAP but I get the sense that I do not appreciate just how good the development environment really is.
In any event, I read the introductory chapter of a book written by Martin Fowler about “Domain Specific Languages” http://www.informit.com/articles/article.aspx?p=1592379
and what appealed to me was that he described a core program that could be configured to respond to various sets of rules without changing anything in the core code. That is just the sort of thing I have been talking about. This is like the “open closed” principal
where you can make an existing program do something different without changing it (spooky!).
So, I thought to myself, the only way to prove to myself I have really understood this, and can thus make it work for me in SAP world, is to write an equivalent program in ABAP.
Anyone who has had to do an exam in mathematics will know that the answer is not actually that important i.e. you can get every answer wrong and still score 90%. Why is that you may ask? You get the vast majority of your marks by displaying your workings i.e. displaying how you went about solving the problem at hand i.e. showing how you think.
Thus, I will attempt to explain the methodology of how I implemented Martin Fowlers example in ABAP even if my end result is not optimal. Some people I used to work with would be horrified by the frivolous nature of his examples, but I love them, and if the underlying theory is sound I can’t see the problem. It is more interesting than talking about aircraft and flights all the time.
The whole idea of OO programming is to reflect the physical world inside your code, so you should be able to pick whatever scenario you want, be it the ever popular SFLIGHT model about passengers flying from place to place, or designing a defence system against zombies who crawl out of the grave to eat your brain. Some might say the latter is not actually a representation of the real world, but it’s clearly a big enough problem for the Prime Minister of Australia to give a warning speech about this the other day. Anyway, the point is, it doesn’t matter what you choose, but the colourful examples are more fun, and learning should be fun.
Just before I start with the diagrams and screenshots and the like, I would just like a quick recap on what we want.
The obvious one is code that can be changed easily when we move from country to country, but something quite important – at least in my company – is code that can be read and understood by the business analysts whilst they are debugging.
ABAP is quite a verbose language so it reads like English to a large extent, and so a lot of analysts are quite comfortable debugging, the problem appears to be that over the years people have become comfortable with following the traditional flow of a procedural program. I had an analyst tell me that he was able to debug anything a programmer wrote “as long as you don’t use that OO nonsense”. He actually used a much stronger derogatory term. I imagine some who hear that might start shouting “luddite” but I can see the point – if you debug VA01 after pressing <SAVE> for example, even though there is a lot of code, it is fairly clear what is going on. Trying the same trick with ME21N is a bit more difficult. It occurred to me that nowadays we have “layer aware debugging”. Could that possibly be used to avoid the low level “boiler plate” OO code and only show the bits of the code that relate to the business logic of the application at hand? I don’t know the answer to that one, if someone does then I am all ears.
So, the idea is to have code which can easily be read by a business expert, and looks just like the specification. Can we do that? To quote Bob the Builder and Barack Obama “yes we can”. Maybe they are one and the same person. You never see them together.
Off we go!
The first thing I had to do was to translate the Java code to the equivalent in ABAP, which is not the end of the world. I have programmed in BASIC when I was 14, and PASCAL when I was at university, everything’s the same really underneath the surface. The push towards Java at SAP might not be as strong as it was round about 2001, but the push towards OO in general marches ever onwards.
I would advocate everyone who programs in ABAP to learn more about Java or the Microsoft equivalent, it will help you more than you might imagine in your day to day work. A fine example is ABAP2XLS which looks a lot like what you might write in Excel.
It’s time to start with a pretty picture. In the Article the program is split into three layers like an onion, rather like the good old separation of concerns, the underlying structure, the behaviour in general and the rules for the application at hand.
If I was a consultant, I could charge a million dollars for changing that sentence into a picture like the one below.
There we go. Without that, you would never have known what I meant.
Now, if I grow up a bit for a second, the important thing there is that if I want to do this properly then I have to start with test (behaviour) driven development
The idea is that the very first thing you code is a test saying what the program should do.
As an excursus, if in a program all your classes are local, such as in the example program I am writing, then the problem you have is that the test class cannot access the private variables of class under test, which it naturally needs to do.
SAP claim they will fix this in the next version – ha ha ha ha – but in the interim, you need to declare a totally empty interface:-
CLASS lcl_gothic_security DEFINITION FRIENDS lif_unit_test.
CLASS lcl_test_class DEFINITION FOR TESTING
RISK LEVEL HARMLESS
That is a purely technical step you have to take to get this working.
Atomic … Now
The rule in the ABAP Programming Guidelines book by Horst Keller is to “modularise not atomise” but I find that if I am writing a test class or want to have the readable code I talked about earlier a lot of guidelines I would normally follow go out of the window. I am sure there is a happy medium.
Testing, Testing, 1,2,3
That is pretty much an exact match of the description in the article of how the program should behave in this instance.
Now we will step through the test, as if we were debugging the program. The above is a method of a test class so when you take the menu path PROGRAM > TEST > UNIT TEST it will get executed.
As can be seen I am trying to follow the structure of the behaviour driven development framework – which is called Gherkin as I understand it – almost everybody in IT gives things they invent really stupid names and then wonders why everyone laughs at us, you could invent the world’s best new concept or methodology but then if you decide to call it “flibble flobble” then it is not going to be taken as seriously as you might desire– but anyway we are trying to follow natural language and see if we cannot write the actual code such that it reads like the specification.
In the setup method we use “constructor injection” which means that since we are testing we don’t really connect to an external system so we have to use a mock / stub to simulate this.
Whenever I hear the term “mock” I am reminded of once when I was on a project and one of the top consultant women said “the purpose of a parallel run is to mock the current system” so I had to shout out “the current system is useless!”. She was not very impressed. Anyway, when you start the unit tests the setup method is run, and it goes like this….
I don’t really know what a state machine is when it is at home, or rather I think I understand the concept in the article but cannot quite see how this fits into this example. Regardless I am trying to replicate the example in the article as closely as I could, so I kept it in. We have a separation of the “model” which represents the business logic from the “controller” which does the actual work. There is no “view” in this example, but you can imagine we might add one so that someone sitting in a secret control room could see what the current status of the security system is.
The really important part is the method call to configure the security system. This is the domain specific language, where “domain” means the business that your company is in, in this example selling gotthic security systems. We’ll take a look at this first and talk about why this is good, and then get into the guts of how this actually is going to work in ABAP.
There was a very good blog recently about using macros wisely in ABAP
and that is what I have tried to do here, to try and separate the business rules from the “boiler plate” code i.e. code that just repeats itself again and again, just creating “noise” which makes the code harder to follow.
In this example we have “events” which are things a human being does such as switching on a light, and “commands” which are sent by the security system to unlock secret panels and the like, and “states” which say what point we are at in the process and what the system is waiting for the human to do next. So, in a given state the system is waiting for a human to do something, and when they do, it will respond by possibly sending a command, and changing state.
The above macros serve to create objects and for the events and commands that just sets some variables to say how the external system thinks of those events and commands i.e. in four digit codes. Now we define the possible interactions between the human and the security system.
As in the Java example, this is not very long at all. I have tried to name the methods such that it reads like natural English. I could probably have done a better job of that, but hopefully you get the idea. The point is that if a new rule comes along, like having to jump up and down on the bed three times, you add that logic here. It could be argued that you could read these rules out of a configuration table, and indeed that would work well, but then we have a value judgement as to whether that makes the rules more or less transparent to the casual observer, which is the business process expert in our case.
As mentioned earlier the idea is that if the person who wrote the functional specification looks at this piece of code and sees something that looks more or less exactly like what they wrote in the specification then they will be a happy bunny. In the example I gave earlier the business process expert who hated debugging OO code could get to this point and work out what was going on, and you would hope that any mistakes or omissions in the business rules would become glaringly obvious.
About 11 years ago I went on a training course in Australia about OO programming and the instructor said something I have heard many times since, and it was “with OO programming you can’t tell what the program does by reading the code”. Well, in this example, hopefully you can. In procedural programming you could clearly do the same but the flow of plain English would be interrupted – at least in SAP world – by assorted keywords such as PERFORM / USING / CHANGING.
I am going to go through each method in detail now, as I said before, this is all highly atomised, so each individual method is very small. For anyone interested in the minute detail of this, it would be a really, really good idea to read the article I mentioned earlier my Martin Fowler at this point. ABAP and Java are different in lots of respects but I have tried to keep things as close as I can, with the inevitable losses in translation. Try as I might there is no way I can be as succinct as he is, and I am working backwards from how he explained things.
When the security system reaches a certain state it can send zero tomany commands to the external system telling it to lock doors and what not. The obvious way to store this in SAP is via the good old internal table, a table of objects in this case.
By the way, did you notice how I have to manually write comments describing the signature? If the team at SAP were REALLY serious about wanting people to write things in an OO fashion you would think they would update the “pretty printer” to be at least as good in methods as it is in subroutines, but oh no. As the Beach Boys would say, if they were SAP programmers, wouldn’t it be nice to double click on a method definition and have the skeleton generated for you in the same way you can do this for procedures?
If you try to mention such things in “idea place” which I thought would be an ideal forum for that type of thinking, you get hit over the head with a frying pan and told “mother knows best”.
Leaving incoherent rambling aside, back in our security system, we next need to define how the state of the security system changes after an event triggered by a human.
I even have to manually write the name of the method tthe ENDMETHOD statement. Can you tell I am bitter about this? And I thought I was doing such a good job of hiding it.
In Java, the order of the day is often to have private attributes and the have SET and GET methods. In SAP you can make an attribute public and define it as read only, and that would make the code slightly more like English e.g. instead of EVENT->GET_CODE( ) you would have EVENT->CODE, I have kept to the Java convention here, in real life I could not have stopped myself making things more SAP like.
There is not much that would make you fall off your chair with shock inside the implementation for the TRANSISITION object.
The table of transistions is just an internal table with ideas above it’s station really. The last thing the configuration method does is to tell the “state machine” what events would send us right back to the initial state.
After the SETUP is complete, the ABAP unit test framework executes our test method, which just to remind you, goes as follows:-
If there were lots of test methods, which is usually the case, you should not make any presumption about which order they would run in, though I suspect it’s first to last, as each test should be self contained. So, I start the test by ensuring we are in the initial state.
I am sure all these one line methods would give some people heart attacks, though Robert Martin says you should decompose methods until you can decompose them no more.
There is a debate going on about whether to include technical details in variable names e.g. if they are objects (O) or elementary data objects (D) or structures (S) etc. and the scope of the variable e.g. object based (member of ) (M) or – shock horror – global (G) as opposed to local (L), this debate will go on forever, I am in favour of doing this, but that is just me. Horst Keller from SAP is dead against this. Anyway the result of HANDLE_COMMAND is to call a method to change the state of the system.
When you move from one state to another, naturally the current state changes, but in addition you may or may not have to execute some actions i.e. tell the external system to manipulate objects in the real world.
The implementation of sending the command to an external system doesn’t matter in this example, in the test class it is a stub, in real life it would generally be a proxy class that connects to PI which in run takes care of sending the message to the external system. I have to deal with PI every day of my working life and it has a tendency to make me want to burst into tears, but on the positive side it is a lot better than the Business Connector, and moving away from a dual stack system, SAP’s current direction in this area, sounds good to me. I am also lucky enough to have worked with Stephan Grube in Germany, one of the architects of the PI solution.
The remaining methods of the test class are just repeated calls to the “handle command” method, followed by an assertion that the actions have created the desired end state.
That is the test out of the way, once that passes we know the code works, but how would we run this if it was real?
This time we create an instance of the real external system as opposed to a mock.
This is just an example, I am sure there are better ways to implement this, for example you could push an external event into the SAP system via an inbound proxy from PI, that would be far more efficient than constantly polling.
I think that covers everything, to recap this is my, no doubt sub-optimal, attempt to replicate the example Martin Fowler gave about domain specific languages, but in ABAP as opposed to Java. The idea is to have the “guts” of the program i.e. how it is supposed to behave, all in one place and written in natural language, so a business process expert can have a look at that part of the code and easily tell if the program behaviour matches what was in the specification.
This all ties in with using behaviour driven development in the unit testing framework, as that too is supposed to be in natural language.
In theory, this should also make it easier to change the program behaviour.
As always, I welcome any feedback on this subject, I am sure the program lost a bit in translation, but hopefully I have understood the central premise.
I will just end by recapping all the definitions from my demonstration program, my experience has been that just pasting the code onto the SCN will mangle it beyond recognition, and you certainly cannot print it, so I will use screenshots.
In real life you would have Z repository objects for the table types, or even better, as of ECC 6.0 you can define table types inside Z classes and expose these to the outside world just as if they were Z table types.
The only difference between a command and an event is their semantic meaning to a human reading the program. From the computers point of view they are identical.
10/07/2013 – I have now read the WIKI definitions of “state machines”. I now know there are two types a MEALY state machine and a MOORE state machine. I am still none the wiser. Can someone explain these to me in ABAP terms, or if there is no equivalent, in terms an idiot like me can understand?
Other OO Blogs:-