Skip to Content

This blog is a prelude for a planned series of blogs on how to design web applications served by RESTful services on an ABAP Web AS. This first part is about a sufficiently generic XML format for carrying informations between client and server.

The ‘X’ in Ajax

It is almost forgotten that the ‘X’ in Ajax stands for XML. In the early times of the Ajax euphoria, XML has been thought as the general carrier format for messages. Ajax was planned as a pairing of “Asynchronous JavaScript” with XML. While “asynchronous JavaScript” addressed the XMLHTTPRequest object as the tool for data transfer, XML was thought to be the language for transmitting data between client and server.

But in those times, the browsers weren’t really prepared to easily parse, build and transform XML. XML quickly lost its terrain in favour of other data formats. Nowadays, the herds have moved on, favorizing JSON, and I join in the voices praising the advantages of this data format. Even ad hoc plain text formats are in fashion, built and parsed on the fly with basic string manipulation commands and regular expression search.

So Why Bother With XML?

First of all,  in the meantime we have mature XML tools on the client as well as on the server side. With a framework like Sarissa, we have all the XML power on the client: we can build and transform XML documents – and convert them into HTML fragments.

This is the second advantage: XML being conceptionally and notationally close to HTML, makes it easy to import XML fragments into the current HTML DOM. Also, the APIs for manipulating or traversing elements share many methods and properties like e.g. getElementsByTagName(), childNodes, firstChild, nodeType, and so on. This is particularly helpful with XHTML documents, but works equally well with proper HTML.

On the server side, it is easy to integrate an XSLT or ST transformation into an ABAP based web service: After having gathered all the necessary ABAP data for the response, you may throw them into a CALL TRANSFORMATION and directly receive an XML document in the desired target format. See below for an example.

The Ajax Response Format

In an Ajax based web application, there should be not too much overhead for the data transfer protocol. Employing a protocol like SOAP would clearly be overdone. On the other hand, it makes sense to design some simple cross-application container format for all the involved web services of a larger web application project.

In contrast to SOAP, the “enevelope” part of such a message should be reduced to an absolute minimum. Basically, just a root node, let’s call it <ajax>…</ajax> should be required – wrapping it all to meet the “One Root Element” criterion of XML well-formedness.

I found that, apart from the common root element, say <ajax>, it makes sense to have one particular defined child element <message>, containing a message accompanying the response.

Concerning the rest of the document, it is not necessary to impose any restrictive criteria. Virtually everything is possible – from nothing to a complex nested structure which fits best to a particular request.

The following may serve as an example:

<ajax>
  <message type="S" text="3 countries selected"/>
  <country value="ES">Spain</country>
  <country value="FR">France</country>
  <country value="IT">Italy</country>
</ajax>

We have a <message> element with a message type – the ABAP message types (Success, Information, Warning, Error, Abort, and X for failed assertions) serving for the allowed values – and an attribute for the message text itself. Following to this <message> element, there is a series of <country> elements.

Depending on the web service’s domain, we should be completely free to design the response format so that it fits best for transferring the relevant data to the client.

Another example could be the confirmation response after a new order has been created:

<ajax>
  <message type="S" text="Order saved"/>
  <order number="4711"/>
</ajax>

In such an example, the service could merely pass the new number of the created sales order to the client. The rest of the order data are on the client anyway, since the user had requested to create an order out of these data. So there is no need to transmit all these data back again from the server to the client.

There is one type of child elements which makes sense for a <message>: The service may indicate one or more of the query parameters (“fields”) that are connected to the message. So a list of <field> elements with obligatory name attribute (and an optional value attribute) may be included in the <message>. The web client could use this information to highlight an input field which had erroneous input:

<ajax>
  <message type="E" text="PACHOLKE is not a known user">
     <field name="user"/>
  </message>
</ajax>

The preceding example also illustrates that an Ajax Response may carry no application data at all.

But likewise, it may be the message that is omitted:

<ajax>
  <requisitionCode>1002</requisitionCode>
</ajax>

The Class of Ajax Responses

XML schema is a language for describing classes of XML documents. We can use it to give a formal description of all Ajax Responses, i.e. XML documents with the data structure exposed above:

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
<!-- Name of the root element is 'ajax' -->
  <xs:element name="ajax" type="ajaxResponse"/>
<!-- Content: [ <message>? , anyElement* ] -->   
  <xs:complexType name="ajaxResponse">
    <xs:sequence>
      <xs:element name="message"
                  minOccurs="0"
                  maxOccurs="1"
                  type="message"/>
      <xs:any processContents="lax"
              notQName="##definedSibling"
              minOccurs="0"
              maxOccurs="unbounded"/>
    </xs:sequence>
  </xs:complexType>
<!-- A <message> has a type and a text attribute
     It may contain <field> child elements --> 
  <xs:complexType name="message">
    <xs:sequence>
      <xs:element name="field" type="field" minOccurs="0"/>
    </xs:sequence>
    <xs:attribute name="type" type="msgType"/>
    <xs:attribute name="text" type="xs:string"/>
  </xs:complexType>
<!-- A field element has a 'name' and an optional 'value' -->
  <xs:complexType name="field">
    <xs:attribute name="name"  use="required" />
    <xs:attribute name="value" use="optional"/>   
  </xs:complexType>
<!-- msgType copies the allowed values auf ABAP's SY-MSGTY --> 
  <xs:simpleType name="msgType">
    <xs:restriction base="xs:string">
      <xs:enumeration value="S"/>               
      <xs:enumeration value="I"/>               
      <xs:enumeration value="W"/>               
      <xs:enumeration value="E"/>               
      <xs:enumeration value="A"/>               
      <xs:enumeration value="X"/>               
    </xs:restriction>
  </xs:simpleType>
</xs:schema>

This XML Schema describes all the rules mentioned above:

  1. The root element has the name “ajax”,
  2. It contains 0 or 1 <message> as children, and after that arbitrary many elements of whatever name and type, but no further <message> element.
  3. The message has an attribute type, containing one of the allowed values for an ABAP message type, and a text attribute for the message itself.
  4. The message may contain a sequence of <field> elements, denoting the fields that are connected to the message.

The only non-straightforward part in this XML scema document is the exclusion of further message elements in the <xs:any> element set. Actually, for implying this restriction, I use the attribute (thanks to Michael Kay for pointing me to this solution):

notQName = "##definedSibling"

This attribute of the <xs:any> element is precisely what’s necessary here. It excludes all the defined siblings (here: just <message>) from the allowed element names. While the rest of the schema would be a valid XSD 1.0 document, this attribute notQName came with XSD 1.1. I don’t see a way of describing the document structure purely with XSD 1.0 means.

Testing Document Instances

There is an online tool for Schema-based XML validation. This is very helpful for a quick test whether a certain XML document is a valid instance of a given schema.

However, when using the online tool with the above schema, you would be disappointed: The online service is still on XSD 1.0, it can’t interpret the schema. As far as I know, there is no online validator for XSD 1.1 (as of this writing).

However, if you have Java on your computer, you can install the Apache Xerces tools. On the Xerces download page, you can download the binaries for the current release (2.11). Be careful to choose a version containing the XSD 1.1 support (they are marked). Copy the eight JAR files contained in that package to the folder <Java>/jre/lib/endorsed, where <java> is the path to your Java SDK (or to your Java Runtime). If such a folder “endorsed” does not exist yet in /jre/lib: create it.

Once you have these JAR files in the JRE endorsed directory, the Xerces XSD 1.1 Schema validator can be used. There is a test class jaxp.SourceValidator coming with the distribution. You can use it from the command line. See here an example with an intentionally wrong XML document (it contained two message elements, where the AjaxResponse format, as explained, only allows one):

/wp-content/uploads/2012/12/validate_163344.png

We don’t need the XSD at runtime. But in any case it’s good to have a purely formal description of the exchanged data. For example, such a description proves useful for unit tests on the web services. Simple tests asserting that the output is an Ajax Response helps avoiding bugs when the services are extended or otherwise modified.

On the ABAP Side

I have designed some test services which I will further discuss in the following blogs.

One of these services retrieves, on a GET request, all the variants of a report with the given report name. Here is an example call:

http://bsp.mits.ch/buch/job/variants?repid=RSNAST00

When calling it, you will retrieve an answer like this:

<?xml version="1.0" encoding="iso-8859-1"?>
<ajax>
  <report name="RSNAST00">
    <variant name="I18730" text="WFMC send external address"/>
    <variant name="IMMNAST00" text="NAST send fax immediately"/>
    <variant name="SAP_SDB" text="Security data sheet dispatch"/>  
    <variant name="UXPD_NAST_ZMVN" text="Purchase Order Copy, MVN"/>
    <variant name="UXPD_NAST_ZTIN" text="Purchase Order Copy, ZTIN"/>
    <variant name="UXPY_NAST_NAB1" text="Yearly invoice, NAB"/>
    <variant name="VERA_001" text="Test scheduled RSNAST00"/>
    <variant name="ZBA1_20110708" text="Order confirmation, re-send"/>
  </report>
</ajax>

This obviously conforms with the above definition of an Ajax Response. It contains not more than the demanded data: The variant IDs, and their texts. The application will use precisely this response for filling a listbox, and this Ajax cycle will be triggered on any change of the field containing the report name.

To serve the request, a stateless web service in the ABAP Web AS is called. For the given report name lv_repid, it gathers the relevant data (VARID / VARIT) into an internal table lt_vars. If necessary, the message fields lv_message and lv_msgty are provided:

   lv_repid = server->request->get_form_field( 'repid' ).
   if lv_repid eq space.
 * Please enter a report name
     message e798(db) into lv_msg.
     lv_msgty = 'E'.
   else.
     try.
         call method check_for_variants
           exporting
             iv_repid = lv_repid
           importing
             et_vars  = lt_vars.
         if lt_vars is initial.
 * No variants found
           message i260(db) into lv_msg.
           lv_msgty = 'I'.
         endif.
       catch zcx_error into lo_ex.
         lv_msg = lo_ex->get_text( ).
         lv_msgty = 'E'.
     endtry.

   endif.

When all the ABAP data are determined, they are converted into XML data. On the ABAP side, this is accomplished by precisely one statement:

* Transform variants and/or message into Ajax Response format
    call transformation zvariants
         source
           variants              = lt_vars
           message           = lv_msg
           message_type = lv_msgty
           repid                   = lv_repid
         result
           xml lv_result.

Of course, the actual work of transforming is performed inside the XSLT transformation – here: zvariants.

In the following code of transformation zvariants, observe that it is possible to write the rules in a “stepdown” manner – beginning with the overall document structure, and then going down to the details one by one – as a composition of small templates with easy and single functions.

Also note that readability increases when using the curly brace notation for evaluation inside of attribute values of the target document:

<xsl:transform xmlns:xsl="http://www.w3.org/1999/XSL/Transform"                              xmlns:asx="http://www.sap.com/abapxml"                             exclude-result-prefixes="asx"                             version="1.0">

   <!--
      Gives all the variants of a report, in the Ajax Response format
      Data are taken from internal table VARIANTS
      MESSAGE and MESSAGE_TYPE contain a message (optional)
      -->

   <xsl:strip-space elements="*"/>

 <!-- Main template: Message and Variants -->
   <xsl:template match="asx:abap/asx:values">
     <ajax>
       <xsl:call-template name="message"/>
       <xsl:apply-templates select="VARIANTS"/>
     </ajax>
   </xsl:template>

 <!-- Message, if given -->
   <xsl:template name="message">
     <xsl:if test="not ( MESSAGE_TYPE = '' )">
       <message type="{MESSAGE_TYPE}" text="{MESSAGE}">
         <field name="repid"/>
       </message>
     </xsl:if>
   </xsl:template>

 <!-- Variants -->
   <xsl:template match="VARIANTS">
 <!-- Only if there are variants -->
     <xsl:if test="*">
       <report name="{../REPID}">
         <xsl:apply-templates select="*"/>
       </report>
     </xsl:if>
   </xsl:template>

 <!-- A single line of the variant table -->
   <xsl:template match="RSVARTXT">
     <variant name="{VARIANT}" text="{VTEXT}"/>
   </xsl:template>

 </xsl:transform>

In this form, such a request is served on a modestly tuned SAP system (actually, it is one of our team’s development systems) with an average ABAP response time of about 20 milliseconds, as the following screenshot from STAD shows:

/wp-content/uploads/2012/12/stad_163345.png

Exploiting the buffering for commonly used database resources, the gross share of the response time is CPU time – a stateless web service of this type will scale very well. In comparison, a request-response cycle of even a minimalistic Business Server Page requires a factor 5 – 10 higher ABAP response time. Also, a BSP navigation usually results in a reload of  the complete web page, whereas Ajax techniques can be used to exchange only small parts of the page, which additionally increases overall performance by reducing the client-side response time.

This is not to speak against Business Server Pages – actually, BSP and Ajax complement each other very well. BSP can be used to manage the application’s ressources (HTML, CSS, JavaScript, XML fragments, client-side XSLTs) and to generate the pages to be loaded from the server. As soon as they are loaded, further interaction can be handled by Ajax requests.

Conclusion

From the first spread of Ajax, XML was planned as data transfer format for the client-server communication. Although JSON is usually favorized today, there are good arguments for using XML: There are reliable tools on client as well as on server side to work with XML data. Also, the similarity of XML DOM to HTML DOM makes XML a good choice.

Once the decision is made to work with XML for Ajax-enabled web applications, it is useful to base all the services involved on a sufficiently general common Ajax Response format. Since Ajax is used for many small requests, such a format should be as loose and as minimalistic as possible. This blog detailed a proposal.

To report this post you need to login first.

5 Comments

You must be Logged on to comment or reply to a post.

  1. Uwe Kunath

    Thank you, Rüdiger for this blog post!

    One question that is still open for me, is how this issue could be solved:

    If we didn’t use GET requests but POST-requests instead, and don’t pass the generated bracket term in the URL, e.g.

    http://server:8080/sap/bc/bsp/leoym/bsp00/endpoint.htm

    the BSP runtime answers them with a generic form like this

    <html><head><style type=”text/css”>.middle  { vertical-align:middle; }</style></head><body onload=”document.forms[0].submit()”><form method=”post” action=”/sap(bD1lbiZjPTA3MA==)/bc/bsp/leoym/bsp00/endpoint.htm”></form><table border=”0″ width=”100%” height=”100%”><tr><td align=”center” class=”middle”>Your request is being processed</td></tr></table></body></html>

    A browser would be expected to display the form and retrigger the request with the bracket in the URL. If it was XMLHttpRequest, this wouldn’t work of course.

    Do you know any possibility to prevent this behaviour in BSP runtime? Of course we could follow the link in the generated form and resend the request manually as a browser would typically do it.

    But possible there is a better approach…

    Thank you,

    Uwe

    (0) 
    1. Rüdiger Plantiko Post author

      Uwe, thanks for your feedback – although it has nothing to do with the content of this blog… 🙁

      Actually, this is a frequently-discussed question. The “bracket part” that the ICF generates into the URL is necessary, since otherwise the system wouldn’t know on what client and with which language to perform the request. So why not include the bracket part in the URL, it doesn’t change anyway (it’s no session-id or something – it should only contain the base64 encoding of a term like “l=DE&c=565”, the language and client parameter.

      This thread  contains some further informations on this so-called URL mangling.

      In practice, you don’t need to worry about the redirect: Performing an XMLHttpRequest,  will automatically interpret the HTTP 30x redirection response, and will automatically forward you to the new URL. Usually, a framework waits until it finally receives an HTTP 200 status code, before triggering an event like “success”, on which you registered your custom code. See http://stackoverflow.com/questions/4168784/ajax-redirection-handling/4168800#4168800

      Regards,

      Rüdiger

      (0) 
    2. Rüdiger Plantiko Post author

      Uwe, I was now  able to reproduce your problem.

      It only happens

      • if you serve the request with a BSP (not with an own request handler, as I was using in the blog on implementing a REST API in ABAP ),
      • and when you send a POST request, instead of a GET request (which has to be possible for a REST API, of course).

      Indeed, the BSP request handler (CL_HTTP_EXT_BSP) has a special handling for POST requests that require an URL rewrite. The form you mention is produced in method ON_CHECK_REWRITE of class CL_BSP_RUNTIME (line 522ff. in our release):

      * go there!
         if c_request->get_header_field( ‘~request_method’ ) = ‘POST’.“#EC *.
           data: html type string.
           loop at ffs into ff.
      *      replace all occurrences of `”` in ff-value with `&quot;`.
             ffvalue = cl_abap_dyn_prg=>escape_xss_xml_html( ffvalue ).
             concatenate html ‘<input type=”hidden” name=”‘ ffname ‘” value=”‘ ffvalue ‘”>’ into html.
           endloop.
           concatenate ‘<html><head><style type=”text/css”>.middle  { vertical-align:middle; }</style></head>’
                       ‘<body onload=”document.forms[0].submit()”>’
                         ‘<form method=”post” action=”‘ l_location ‘”>’
                           html

      In this situation – POST requests, and using a BSP as request handler, I would suggest to use the correct mangling term containing the client and language to make it work (as I already stated in the previous reply).

      Actually, I don’t see the benefit in using BSP for implementing a REST API, as it is designed for creating pages to be looked at in the browser. What is the advantage in using it? Why not creating an own SICF handler class, as described in the mentioned blog?

      Regards, Rüdiger

      Remark: As another example of non-BSP request handler (instead of those described in the mentioned blog), see here the code for the handler that gets the variants of a report (  http://bsp.mits.ch/buch/job/variants?repid=RSNAST00 ): http://bsp.mits.ch/code/clas/zcl_job_repid_variants

      (0) 
      1. Uwe Kunath

        Thank you, Rüdiger, and sorry for the misunderstanding. Because of the section “BSP” of this blog post, and because it was tagged with “BSP”, I thought that you implicitly described BSP-Handlers. However, you stated clearly that you are talking about “stateless web service in the ABAP Web AS”.

        Thank you very much for the explanation of the mangling term – if it doesn’t change anyway, it could be included in the URL. That’s really good news.

        The reason why BSP handlers might be better than ICF-based handlers, is session handling. Of course, REST Services should be stateless, but this is often not possible or at least leads to performance issues for e.g. in hybrid mobile applications.

        Now, in the blog post you mentioned, you also included a section called “Session, Identity and Locking”, which is good news #2 today 🙂

        Thank you very much!

        PS:

        There are also other interesting approaches to REST in SCN like the python connector, which is optimized to leverage existing function modules

        http://scn.sap.com/community/scripting-languages/blog/2013/01/29/rest-prototyping-with-python

        (0) 
        1. Rüdiger Plantiko Post author

          Uwe, thanks for your reply.

          Working with state means that the REST API should require a session ID parameter, to be provided by the client web application. If that application is a BSP, the session ID can be extracted from the “sap-appcontext” cookie, as explained in my blog.

          Thanks for pointing my attention to the “Python connector” which opens yet another way for connecting an arbitrary web application server to the SAP system internally.

          With an own webserver like Apache in-between the SAP system and the client, one could also map some URL paths directly to the URL of a RESTful ABAP web service (using mod_rewrite or directives like ProxyPass), and the accesses can then be programmed in JavaScript on the client. This is near to the ideas I outlined here for XML and here for JSON as data transport layer.

          Regards, Rüdiger

          (0) 

Leave a Reply