Skip to Content

I don’t like getting into a lather when it comes to data and function integration. Rather than using SOAP, I prefer real web services, built with HTTP.

As an example of taking the RESTian approach to exposing your SAP data and functionality through services you can build with the excellent Internet Communication Framework (ICF) layer, I thought I’d show you how straightforward and natural data integration can be by using a spreadsheet as an example.

In my recent SDN article (published this week) …

Real Web Services with REST and ICF

… I presented a simple ICF handler example that allowed you to directly address various elements of CTS data (I prototyped it in my NW4 system so I thought I’d use data at hand, and build an example that you could try out too). For instance, you could retrieve the username of the person responsible for a transport by addressing precisely that data element like this:


http://shrdlu.local.net:8000/qmacro/transport/NW4K900007/as4text

The approach of making your SAP data and functionality first class web entities, by giving each element its own URL, has wide and far reaching benefits.

Take a programmable spreadsheet, for example. You’re managing transports between systems by recording activity in a spreadsheet. You’re mostly handling actual transport numbers, but have also to log onto SAP to pull out information about those transports. You think: “Hmmm, wouldn’t it be useful if I could just specify the address of transport XYZ’s user in this cell here, and then the value would appear automatically?”

Let’s look at how this is done. My spreadsheet program of choice is the popular Gnumeric, available on Linux. If you use another brand, no problem – there’s bound to be similarities enough for you to do the same as what follows. For background reading on extending Gnumeric with Python, you should take a look here.

With Gnumeric, you can extend the functions available by writing little methods in Python. It’s pretty straightforward. In my home directory, I have a subdirectory structure

.gnumeric/1.2.1-bonobo/plugins/myfuncs/

where I keep the Python files that hold my personal extended methods.

In there, in a file called my-funcs.py, I have a little script that defines a method func_get(). This method takes a URL as an argument, and goes to fetch the value of what that URL represents. In other words, it performs an HTTP GET to retrieve the content. If successful, and if the value is appropriate (it’s just an example here, I’m expecting a text/plain result), then it’s returned … and the cell containing the call to that function is populated with the value.

Here’s the code.

# The libs needed for this example
import Gnumeric
import string
import urllib
from re import sub

# My version of FancyURLopener to provide basic auth info
class MyURLopener(urllib.FancyURLopener):
    def prompt_user_passwd(self, *args):
      return ('developer', 'developer')

# The actual extended function definition
def func_get(url):
    urllib._urlopener = MyURLopener()
    connection = urllib.urlopen(url)
    data = connection.read()
    if connection.info().gettype() == 'text/plain':
        return sub("
$", "", data)
    else:
        return "#VALUE!"

# The link between the extended function name and the method name
example_functions = {
        'py_get': func_get
}

It’s pretty straightforward. Let’s just focus on the main part, func_get(). Because the resource in this example is protected with basic authentication (i.e. you have to supply a username and password), we subclass the standard FancyURLopener to be able to supply the username and password tuple, and then assign an instance of that class to the urllib._urlopener variable before actually making the call to GET.

If we get some ‘text/plain’ content as a result, we brush it off and return it to be populated into the cell, otherwise we return a ‘warning – something went wrong’ value.

We add the method definition to a hash that Gnumeric reads, and through the assignment, the func_get() method is made available as new custom function ‘py_get’ in the spreadsheet. (There’s also an extra XML file called plugin.xml, not shown here but described in the Gnumeric programming documentation mentioned earlier, that contains the name of the function so that it can be found when the spreadsheet user browses the list of functions.)

So, what does that give us? It gives us the ability to type something like this into a spreadsheet cell (split for readability):


=py_get('http://shrdlu.local.net:8000/qmacro/transport/NW4K900011/as4user')

and have the cell automagically populated with the appropriate data from SAP. You can see an example of this in action in the screenshot:

screenshot of py_get in Gnumeric

As you can see, being able to address information as first class web resources opens up a universe of possibilities for the use of real web services.

As a final note, I’ve submitted a SAP TechEd talk proposal. It’s titled:

“The Internet Communication Framework: Into Context and Into Action!”

If you’re interested in learning more about the ICF, and want to have some fun building and debugging a simple web service with me, you know where to cast your vote if you haven’t already. Hurry though – there’s only a few hours to go!

Thanks!

To report this post you need to login first.

20 Comments

You must be Logged on to comment or reply to a post.

  1. Piers Harding
    Thanks DJ, for some of the most entertaining and insightful (maybe even a riot) writing I have read for a while.

    “HTTP an Application Protocol” is an important leap that should be recognised from basic page serving applications, all the way through to Grid Computing and the Internet OS.

    SOAP Sucks! indeed 😉

    (0) 
  2. Benny Schaich-Lebek
    you know what betacam is, do you?
    And you surely know MacIntosh or even better NEXT?(I very well remember, when a bigheaded programmer told me that he never would need a mouse, as his desk doesn’t have space for such a stupid device).
    Or not so funny, that friend of mine who worked for years in an OS/2 shop and always told me nice stories how bad Windows was against OS/2- Until they closed down the whole consulting company.

    The train is already rolling and there is no way back on real applications as the standards we need are all based on SOAP. Don’t make this a religious fight, as nobody later will be thankfull to you.

    Regards,
    Benny

    (0) 
    1. DJ Adams Post author
      Hi Benny

      If I understand you correctly, with your comparisons to betacam (betamax?) and OS/2, are you saying that the world wide web (the biggest example of the application of the REST architectural approach) will become obsolete?

      I’m not making this into a religious fight. It does seem, however, by bringing the subject up in your comment, that you are trying to. Please don’t.

      Whatever happened to healthy debate and alternative viewpoints?

      Kind regards
      DJ

      (0) 
      1. Benny Schaich-Lebek
        Hi DJ,

        no,I’m not driving that direction rather then fighting it. The point here is that SOAP is the base for all business web service standards currently evolving, like ebXML and WSI and so on.

        That means the definition of a standard that tells us how to exchange actual business objects.
        And I have asked colleagues who know better: Theses standards are ‘bound’ to SOAP architectural. In other words: Fighting SOAP is fighting the foundation of a building that is already getting covered it’s roof.

        And now: what’s ‘unhealthy’ about this form of debate? I didn’t throw dirt, did I? I just feel like if somebody comes up telling my customers that there is a nicer, bigger, and more comfortable train on another platform driving the same way, it’s my duty to tell them that this train ‘might’ be behind the one already starting and that there is only a one lane track to the goal…

        Regards,
        Benny
        (Rhetorik is pepper for debates, btw)

        (0) 
        1. DJ Adams Post author
          Hi Benny

          I’m led both by a sense of duty and an effort to fight complexity. That’s why I point these things out to people.

          REST isn’t some new concept, not some new kid on the block. It’s merely the description (and application) of the mechanisms that make the world’s biggest web service (the WWW) tick.

          Regarding more business orientated web services, how about Amazon, up there as one of the top 3 web service providers right now. Amazon offer a SOAP interface and a REST interface for developers who want to use their API to build business applications. Guess what? Amazon said that 85% of their Web services traffic is via the RESTful interface.

          (Finally Best of SDN 2005).

          HTTP is an application protocol. If people want to tunnel a second application protocol inside of it, then I can’t stop them. But I can at least describe the alternatives.

          Kind regards
          DJ

          (0) 
    2. Piers Harding
      To say that you have to accept a technology (in this case SOAP) just because it’s the status quo or where the “train is heading” at the moment is not enough of an argument to support it.  Research is about pushing the boundaries, and taking fresh looks at old problems in an effort to continually strive to do better.
      I do accept that not all good ideas (like OS/2 at the time, or perhaps X500) get used to their full potential due to public acceptance problems, or other commercial agendas, but this certainly doesn’t have to always be the case.
      We only need to look to the long struggle of Open Source for many examples of technologies that have been consistently subjected to ‘FUD’, and other such dross, to see that a better idea can succeed over time – as the bad ones just can’t last the distance.
      I propose that SDN be treated like a research environment where it’s purpose is to exchange ideas, and put forth new solutions, that are up for serious, and critical peer revue, as opposed to shuting the door on a discussion for no other reason than ‘thats just the way it is now…’, or ‘thats the way its always been done …’.
      (0) 
      1. Benny Schaich-Lebek
        Jumping off a dead horse is not giving up on new ideas. If you look into webservice technology a little bit deeper, you’ll find that the train already is on heavy speed.
        Of course you always can introduce new technology, but then call it what it is. This look to me more like some new middleware concept.

        Regards,
        Benny

        (0) 
        1. Piers Harding
          REST is not a new principle – it is approximately as old as the Web itself.  Additionally it is a development philosophy as opposed to a middleware concept, so there is no “product” being pedalled here, rather a reintroduction of people to true depth of the HTTP specification itself, and the ways in which it can be exploited to its full potential.

          Regards,
          Piers.

          (0) 
  3. Brian McKellar
    Hi DJ, you are right that ICF provides a new power to retrieve data, well above RFC calls. It is quick and simple to use, and does the job well. I have written many such handlers over the years to test small things, and just fool around. I like you idea of “addressable data”. It will not take long until someone has the handler for /name_of_table/row_key! Now let my shutdown my OS/2 here and take the kids out. 🙂

    brian

    (0) 
    1. DJ Adams Post author
      Hi Brian

      You’re spot on about the ICF handlers. For a developer used to *SP technologies (ASP, JSP, even PHP) the BSP mechanism is a great ‘home from home’ that’s nice and familiar.

      What’s more, however, is that the extra ICF-based facilities offered at the level below BSP (or above ICM, depending on your viewpoint) are super-friendly and familiar to all those hackers out there who call mod_perl or mod_python (and other places) their home.

      /me switches back to his serial-connected VT320 to do some real work … 😉

      dj

      (0) 
  4. Karsten Bohlmann
    Hi DJ, criticism of SOAP for declaring its RPC-over-HTTP as “web services” is justified to a certain degree. But REST rests on HTTP _and_ XML. In REST-style web services, URI-addressable things evolve from HTML to XML; see http://www.amazon.com/webservices. Therefore, I find your “text/plain” examples misleading. Return something more complex than a single value, and you need XML. And in ABAP, you have a very powerful tool to do the necessary data / XML mappings (both as sender and receiver): Have a look at the CALL TRANSFORMATION statement. Or read our article: http://tinyurl.com/6ukwx

    Regards – Karsten

    (0) 
    1. DJ Adams Post author
      Hi Karsten

      Thanks for an interesting reply.

      However, I beg to differ on the point you’re making – web services do not always rest upon XML.

      Of course, that doesn’t mean that XML is used in plenty of circumstances – complex business documents, for example. Even I (;-) have built web services and used XML for the representations of data being transferred.

      But for example if I’m managing graphic files in PNG format via HTTP, I don’t base-64 encode the PNG data and send it in an XML envelope – that’s crazy. I use — directly — whatever MIME-type is suitable for the application. There’s data out there in the world that’s not a natural fit for XML. That’s why we have MIME-types (as I’m sure you know).

      The text/plain examples were deliberately simple; there’s only so much one can cover in an article without boring the reader to death. So I can see why you might think they’re misleading.

      But on the other hand, there’s nothing wrong with simplicity; indeed, right now, I’m building a web service where no payload data is used at all. It’s a boolean ‘is this valid?’ request made to a URL which is the address of a valid logical data object in a remote system. If the HTTP response code for the GET shows retrieval success (200), the object is available and therefore valid. If it can’t be found (404), it’s clearly not valid. That web service can be extended if required at a later date by supplying information in the payload.

      As you say, if XML _is_ used to support the payload, then yes, I agree – CALL TRANSFORMATION is indeed a powerful tool (I attended your TechEd tutorial on XML and XSLT in Basel last year – it was very good, thanks).

      But it’s a (unfortunately widespread) myth that web services always imply the use XML.

      One more thought – even for complex data structures, I know personally of one project at least that’s using YAML (http://www.yaml.org) to mark up the data, not XML.

      Kind regards
      DJ

      (0) 
  5. Alessandro Guarneri
    Hi DJ,
    This my first time reading stuff o’ yours. Really cool.
    I also like the idea of keeping it simple if you really don’t need to go complex. And SOAP surely is, always… In order to get e.g. a simple, single 40 bytes string you to have carry back and forth via http almost 1Kb of useless metadata…
    I love the REST option.
    I’ve been playing around with ICF since the early unwatchable 6.10 WAS release…! 😀

    And don’t you put the blame on faithful SAP guys if they all point at you as if you were cursing 😀
    They have invested (and still are) a lot on SOAP. Which indeed is good in 95% of cases given SAP inherent complexity, isn’t it?

    The real Peace-Of-Mind is to have the mind openness to use the right technology at the right time. No integralism allowed.

    Cheers,
    Alex

    (0) 
    1. DJ Adams Post author
      Hi Alessandro

      Thanks for the comment – it’s great to see interest in this post which is now over 5 years old and which covers a topic which is still as important today as it was then. A lot more important, in fact. If you’ve not heard it already, the eGeeks published a podcast about SOAP vs REST recently – it’s good listening:

      http://enterprisegeeks.com/blog/2009/08/27/egeeks-podcast-episode-27/

      Anyway, I’d just like to address one point you made about investment in SOAP being good given SAP’s inherent complexity.

      Why would you want to make something that’s complex even more complex? I’ve been using the ICF to build REST-orientated HTTP-based integration for a few years now and the benefit is that I’ve been exposing the complex machinery inside SAP … but in a simple and uniform way.

      Thanks
      dj

      (0) 
  6. Markus Kohler
    Hi DJ,
    Just right now found your blog post!
    You were certainly ahead of your time.

    To all the SOAP promoters,try to answer the following questions:

    How do you know how long the data you get returned from a SOAP call is valid? E.g. do you get any “meta-data” with your result that indicates how long the result can be cached?

    Will your results be cached by intermediate HTTP proxy servers?

    Will you be able to run a crawler (like Google) against your SOAP interface and index it to make it searchable?

    Regards,
    Markus

    (0) 
  7. Renald Wittwer
    Hi DJ,
    very nice to read the blog over 5 after you wrote it. And very funny to read the comments, especially from Benny Schaich-Lebek.

    I am playing around with Android and the development of Android. In this area REST and JSON are far more supported than SOAP. The reason is obvious, because of its lightweight and simplicity.

    REST is not dead! It will live beside SOAP. Both concepts have their advantages and their disadvantages. Nice if I have the opportunity to choose.

    Perhaps we are dinosaur, but: There’s life in the old dog yet!

    Best wishes
    Renald

    (0) 
    1. DJ Adams Post author
      Hi Renald, thanks for the comments. Yes, REST is certainly not dead. In a way, it’s never been alive, or at least is immortalised by the fact that it’s more an architectural approach than a toolset.

      I also just re-read the other comments here, and they made me smile. If nothing else, this debate raises passion as well as awareness.

      Cheers!
      dj

      (0) 

Leave a Reply