Application Development Blog Posts
Learn and share on deeper, cross technology development topics such as integration and connectivity, automation, cloud extensibility, developing at scale, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 
rdiger_plantiko2
Active Contributor

Documenting with a Wiki

Since its beginning with Ward Cunningham's now-famous "WikiWiki" page c2.com (1995), a Wiki system can be defined as a collection of web pages with an "Edit" button, allowing everybody to alter or improve the page content.

You can try it with the WikiWiki which is still up and running - and, by the way, contains many interesting informations about programming paradigms, programming languages, design patterns, and similar stuff. For playing with the text formatting rules, there is a sandbox page. When you click "Edit", you get the content of the page in the "Wiki markup format" in which it is stored on the server. The markup format is very discreet, however. It should be so intuitive that you might not even need to read the TextFormattingRules page.  If you enter plain text, it will appear as such - so for 95% of the content, you need no special formatting rules at all. Slightly more complex formatting - titles, links, emphasizing, bullet lists - require some very simple markup rules. For more complex formatting - like tables - more elaborated markup rules are required.

Linking inside the Wiki has been simplified by the so-called "Wiki Words". If, in your text, you JoinCapitalizedWords, the system will automatically generate a link to that page. For external links - or for the few cases where this convention does not work well, there also is an explicit formatting rule for hyperlinks.

A Team TWiki

My SAP development team is using the enterprise collaboration platform TWiki since 2005, which sticks to  these original Wiki design ideas. The complete content is structured into webs and topics, where a topic means a single content page of the TWiki, and a web being a collection of topics, belonging to a common theme.

In our TWiki, we have (apart from the standard webs like TWiki, Main and Sandbox) only two custom webs:

  • A "ReUse" web, which is designed as a documentation repository for reusable parts of code,
  • and a "KnowHow" web, containing tricks and tips around the SAP systems simplifying our daily developments and support tasks.

For illustration: Here is a screenshot of our TWiki, displaying the topic for system monitoring.

Many function modules, reports, classes and even DDIC types, come with own documentation in the system. We didn't want to copy theses documentations into the TWiki (similar to duplication code, duplicating documentation is a bad practice) but rather looked for a way to connect the TWiki online to the SAP systems and to call off the documentation from there when the topic is requested by some user.

Merging SAP Documentation into TWiki Topics

This plan could be split up into three separate tasks:

  1. Write a transformer from SAP's internal  text format (ITF), better known as SAPscript, which is the common syntax for all documentation texts as well as for SAPscript forms, long texts of application objects, and much more, into TWiki's markup language (TML)
  2. Write an HTTP request handler (actually, a simple REST API) for getting documentation and code from an SAP system - and design an appropriate syntax for the request.
  3. In TWiki: Write a Plugin for emedding SAP documentation or SAP code into the current topic.

For illustration: This is SAPscript, the format in which documentation of code is written in the SAP system:

This is how the documentation is displayed in the SAP system itself (actually, it is using a converter to HTML):

In this blog, I am presenting a request handler transforming SAPscript docu to TWiki markup. When calling the request handler directly in the browser, the response text is "raw" TWiki markup, as follows:

The TWiki markup is merged internally into the TWiki topic (this is what the SapConnectPlugin does) - and the final result in TWiki will look similar to this:

The SAP Connect Plugin

The TWiki system is written in the Perl programming language. It provides many hooks for custom-written code: so-called plugins. It allows for adding content dynamically with the TWiki Variables. A TWiki variable is a special instruction interpreted at the time when a topic is requested. Syntactically, it is embraced by two percent signs. The TWiki variable is handled by its tag handler, usually assigned by name.

With our plugin, the SapConnectPlugin, a special TWiki variable like

%INCLSAPDOCU{"RE.ZZ_TEST"}%

is replaced by the documentation of the report ZZ_TEST in the connected SAP system. Since we have several development systems, I can choose a destination different from the default system:

%INCLSAPDOCU{"RE.ZZ_TEST" dest="D11"}%

A class may have not only a class documentation, but its components - attributes and methods - may be documented, too. With the special parameter expand I can the documentations of all subobjects to be included:

%INCLSAPDOCU{"CL.ZCL_TEST" expand = "true"}%

If I want to include the current code into the topic, I use a slightly different TWiki variable:

%INCLSAPCODE{"CO.ZCL_TEST.DO_TEST"}

will include the code of the method DO_TEST of class ZCL_TEST into the current topic.

If somebody prefers to offer the documentation only as link - this is possible, too:

%URLSAPDOCU{"CL.ZCL_TEST" expand="true"}%

This will provide a link to a special TWiki topic. If the user follows that link, he will be routed to that special TWiki topic (which basically contains a %INCLSAPDOCU% TWiki variable with dynamic content), which displays the documentation of the class ZCL_TEST in this case.

Transforming Documentation from SAPscript to TWiki

Like all the other wiki's, TWiki has a markup format which makes it easy to add plain text, while the more "decorated" your text becomes, the more special markup is required. Here are some simple markup elements:

  • Headings start with a sequences of three dashes, followed by 1 up to 6 plus signs. Headings will be collected automatically into a structured table of contents with the special TWiki variable %TOC%.
  • For links to a TWiki topic, there usually is no special notation necessary. You simply write the name of that topic. If this name is a "wiki word", i.e. includes camel case (like e.g. WikiWord), then the word will automatically be displayed as a link to that topic.
  • An enumeration item starts with a sequence of three spaces and an asterisk.
  • An ordered enumeration item starts with a sequence of three spaces, and then "1." (a digit and a dot).
  • To display a word in oblique font style, enclose it with underscores: _italic_ will render as italic.
  • To display it in bold style, enclose it with asterisques, like *bold*.
  • To display it in non-proportional style (the style that is normally used for code), enclose it with equal signs, like =DATA= .
  • Tabular data have to be enclosed by vertical bars. So | a | b | c | gives a table row with three cells, containing the letters a, b and c.

SAPscript, on the other hand, has different control sequences. A SAPscript line consists of a two-character prefix called the "paragraph format", and a 132 character text line, which may contain control sequences for text decoration (like <ZK>...</> for italic, <ZH>...</> for bold, <NP>...</> for non-proportional font), for symbol substitution ( like &NAME&, which will be replaced by the actual value vor the symbol NAME, looked up at various places), and for table-data (two commas work as separators between the cells of a table).

A translator is needed, transforming one markup dialect into the other. For the implementation, I am following the Builder Design Pattern.

The first actor is an event-based parser which detects a commonly used subset of the SAPscript markup, raising events for each paragraph format and for each text control sequence found. This is the class ZCL_ITF_PARSER. Actually, the name is not precise - it should better be called a lexer (or scanner), since it doesn't validate the input and doesn't send any kind of error messages. It simply crawls through the input text, detecting the different style and paragraph notations one after another and sending events for them, which can be used to manipulate the result string.

The result string is managed by a stack-type container for strings (actually a stringtab). This is necessary since SAPscript control sequences may be nested. The simple text parts (i.e. all the rest which is not a control sequence) are written directly into this container. Thus, if you run the parser without attaching any Transformer to it, the result will be the pure text content of the SAPscript document, with all control sequences stripped off.

ZCL_ITF_PARSER is a general component, which doesn't know about the target format. It's an expert on SAPscript only. Actually, we are using it for other purposes like form scanning, too.

To produce Twiki markup, we connect the parser with another class, ZCL_ITF_TO_TWIKI. This is the Transformer which actually produces the TWiki markup text. After having called the lexer's parse() method, it receives notifications on all SAPscript control sequences. The event handlers for these notifications are used to manipulate the result stream.

The following events are available:

  • Begin and End of a paragraph format like AS. These are two different events.
  • Begin and end of a character format (like ZH, which introduces an emphasized character sequence). Again, these are two events.
  • The Table Data event separates two cell contents in a table.
  • The VAR event notifies about a symbol embraced by two '&' signs. The symbol will be resolved, if it is found in the global symbol table TTDTG. In any case, the event handler can influence the resolution in an arbitrary way.
  • ITF Command, which is a line introduced with the special format /:
  • ITF Comment, which is introduced with the special format /*

Time for praising ABAP which has the Observer design pattern already implemented into the language core! We can use built-in statements like set handler and raise event, where in other languages a registration table has to be maintained explicitly. It seemed a good idea to me to use events as loose coupling between the parser and the actual consumer, performing the translation into the target syntax. Alternatively, of course, one could have used an interface with callback methods, passing the callback instance as parameter to the parse() method of ZCL_ITF_PARSER.

So how does the transformation from SAPscript to TWiki markup work in this scheme? At the core, there is a method transform() in class ZCL_ITF_TO_TWIKI, registering the handlers and starting the parse process:

method transform .

   data: lo_parser type ref to zcl_itf_parser,
         lv_text   type string.

   create object lo_parser.
   set handler :
     do_par_begin  for lo_parser,
     do_par_end    for lo_parser,
     do_tag_begin  for lo_parser,
     do_tag_end    for lo_parser,
     do_table_data for lo_parser.

   lo_parser->parse( exporting it_text = it_text
                     importing ev_text = ev_text ).
   do_at_end( changing cv_text = ev_text ).

endmethod.

During the execution of the parse() method, events are triggered. Here is an example: the parser notifies the transformer when a sequence of italic (ZK), of bold (ZH) or true type (NP) characters begins or ends. At this point, the event handlers merge TWiki's special markup for these sequences into the result:

method do_tag_begin.

   data: lv_tagname type string.

* Put tag name on stack
   lv_tagname = iv_name.
   append lv_tagname to gt_tagname.

   case lv_tagname.
     when zitf_tagname-np.
       io_text->add( ' =' ).
     when zitf_tagname-zh.
       io_text->add( ' *' ).
     when zitf_tagname-zk.
       io_text->add( ' _' ).
   endcase.
 
endmethod.

At the end, the writer contains the complete product of the transformation which will be passed back to the TWiki system.

The SapConnectPlugin on TWiki

All the software that is necessary to make this extension work can be downloaded from the public TWiki topic SapConnectPlugin, where I have documented the plugin from the TWiki perspective. On the ABAP site, these are the mentioned three classes and two interfaces:

NameDescription
ZCL_ITF_PARSERParser for SAPscript
ZCL_ITF_TO_TWIKITransformer SAPscript -> TWiki markup language
ZCL_TWIKI_CONNECTConnects to TWiki, serving the HTTP requests
ZIF_TWIKI_ADDITIONInterface for further extensions with other functionality
ZIF_WRITERInterface for writing to the result stream, needed by the ITF parser
ZITFA type-pool containing SAPscript-specific constants

Additionally, an extension written on the TWiki side is necessary - the actual plugin. This is written in Perl and available for download on the same topic.

This plugin might eventually become part of a future TWiki release.

Have fun!