Skip to Content

Over the last year I’ve had various exposure to Netweaver Gateway; the usual “so what is this all about?” tinkering and self-teaching, actual project use and even training of junior consultants.

Prior to this I had been working on some R&D with a custom REST dispatch layer, i.e. doing RESTful SAP access without Gateway. The main bone of contention in the abstract design was the URI scheme – just what was the best way to structure resource paths and pass key values, parameters, etc. to the backend? In this respect REST is open to interpretation since it is not a protocol.

When it was announced that Gateway would use OData we were still some way off of getting a useable product, but it got me thinking: if I were using the OData protocol, how would I incorporate it into my dispatcher? It was pretty clear that an OData parser would be required to turn the URI scheme into something that the average non-webhead ABAP guy could understand.

With the arrival of Gateway Builder, it is now much less of an effort to put together the implementation layer of the data provider, and the model provider classes can pretty much be left alone. Now we have more time to delve into the OData interface and discover exactly what OData can do with ‘standard’ method results and, which OData scenarios require some extra understanding and ABAP effort.

To this end, I’ve been working through the OData URI conventions and comparing these to the Gateway OData implementation.

Basic entity addressing

OK let’s clear something up first. Always keep the cardinality 0:n in mind. Even if you know that an entity has a 1:1 cardinality, the OData protocol is taking  0:n as a baseline.  This makes sense as 0:n has to encapsulate all the other cardinalities.

The practicality of this means I should be referring to all entities in plural terms.

resource_path/service/foos

could return no foos, one foo or a whole basketful. That’s the point, it’s meant to work that way. Suppose I get several foos and want to reference one  (foo9) and navigate into it?

Well, I’ve actually got a navigation link to it in the result feed, so I don’t need to worry about that. Assuming I didn’t have that link, the logical addressing would be:

resource_path/service/foo(‘foo9’)

I’m referencing a single resource so I don’t need to bother with the entity set syntax now, do I…

WRONG!

The above would be thrown back by the OData parser telling me that the resource ‘foo’ does not exist.

I have to sideline English and logic here and return to my 0:n way of thinking. The correct syntax is:

resource_path/service/foos(‘foo9’)

Dammit – so I’ve got to go through the get_entityset method again? No, I don’t.

Although I am using the plural syntax, the simple addition of the key specifier section – adding  (‘foo9’) after the resource name – tells the parser that it needs to send this request to the get_entity method in my data provider.

Intrinsic System Query Options

The OData standard specifies a set of system query options that you can apply to any resource request; that is, the result will be often be different to the vanilla request on the same node.

Gateway implements some of these system query options intrinsically; provided that you have working GET methods for the entity and entityset you can expect the OData parser to transform the result for you.

Here are the intrinsic options.

Property extraction – single

Assume that the ‘foo’ entity has these simplified properties:

<EntityType Name=”foo” sap:content-version=”1″>

<Key>

<PropertyRef Name=”id” />

</Key>

<Property Name=”id” Type=”Edm.String” MaxLength=”10″ sap:label=”Foo ID” />

<Property Name=”name” Type=”Edm.String” MaxLength=”30″ sap:label=”Name” />

<Property Name=”category” Type=”Edm.String” MaxLength=”10″ sap:label=”Category” />

<Property Name=”price” Type=”Edm.decimal” Precision=”9″ Scale=”2” sap:label=”Price” />

<Property Name=”currency” Type=”Edm.String” MaxLength=”5″ sap:label=”Foo ID” />

</EntityType>

It is possible to extract just one of these properties as a result. The option can only be applied to a single entity URI.

resource_path/service/foos(‘foo9’)/category               //obtains the category of an entity identified as ‘foo9’

N.B. there are further formats of URI that can access single values at different levels of navigation – here I am just presenting the basic form against a simple primary entity address. I hope to revisit these later.

Property extraction – SETS ($select)

Sets of properties can be extracted, these are obtained with the $select query option. The option can be applied to any URI addressing an entity or a set.

resource_path/service/foos?$select=name,price,currency

resource_path/service/foos(‘foo9’)?$select=name,price,currency

// property section only contains request properties

<m:properties>
  <d:name>foo-nine</d:name>
  <d:price>380.00</d:price>
  <d:currency>gbp</d:currency>
</m:properties>

Instance count – ($count)

The $count option is similar to the Open SQL count in that it will return a count of the query results and no data (note that in the URI, $count sits behind a final ‘/’, not a ‘?’ operator like other options do!)

resource_path/service/foos/$count 

The $count option only makes sense when applied to an entity set address, but it will still work (sort of) on the access to a single entity, since it operates on the assumption of an overall 0:n cardinality.  I say ‘sort of’ because if the entity does not exist for the requested key, the count is still 1. That is semantically incorrect, since a client should be able to run an existence check on a key and determine true or false from the count result.

Why do we get a count of 1 when the key is wrong? It is technically correct because the response feed contains an empty property set after it comes back from the GET method. There doesn’t seem to be a way to make it return zero; if you modify the DPC class method to not transfer the data, you get an inner error reporting the resource is not found. That is “correct” but I’d rather not have inner errors used to report non-existence since they are logged as server errors.

One point to note on the above regarding performance : the property extraction/counting is done after the method call, applied to the results. Your method has no inkling that a reduced set of properties is being requested therefore it will always obtain the full property set. Until SAP change the design to transfer this level of request information to the backend, the ABAP layer will not be able to provide faster access to the data.  This is technically very similar to WDA contexts that are based on Dictionary structures – the whole set of columns are present in the background and the meta-model maps to those attributes used in the context, but in DB terms it’s always a full width access.

There are some further intrinsic query options; $expand and $links –  however they only have any value once you start navigating between entities. I’ll cover these in the next installment.

System Query Options requiring ABAP extensions

Now we come to the system query options that won’t work unless you provide some ABAP that anticipates and executes them.

Entity Set Paging ($top and $skip)

These options are at once the simplest to understand and possibly the hardest to implement within an enterprise service.

$top limits the amount of results from a set. This is a GOOD THING and should be mandatory in my opinion. Having an open query could be disastrous where the potential datasource is massive.

resource_path/service/foos?$top=200

Developing an entityset feed should always involve some kind of $top consideration. The value for the $top option is fed into the ABAP method via IS_PAGING-TOP.

You could design the ABAP logic to assign a default value for $top if the requester doesn’t set it, rather than having an unlimited retrieval, but I don’t believe that’s a good idea. How would the recipient know we had limited the results?  In short, the client should be required to send a $top value  and the acceptable maximum top value should have a limit . How this is implemented and enforced  is up to you guys

Thorny issues don’t stop with $top. So I got the top x entities, I want to get the next bunch.  In order to get a new collection, I can ask the server to skip n entries using the $skip option.

resource_path/service/foos?$skip=200

However that takes us back to the ‘how many?’ question posed by $top. There’s no point limiting the first query to 1,000 entries then letting the client skip past those and get a million.

No, $skip by itself is pretty useless, dangerous and other nasty things. Therefore another of Ron’s Rules is that any $skip still requires a $top value to be sent with it. Yes, you can do this – this is the first mention of the fact that you can combine some of these options. It also matches the paging concept as you expect a discrete number of rows per page.

resource_path/service/foos?$skip=200&$top=200

By the way, the value for the $skip option is fed into the ABAP method via IS_PAGING-SKIP – in case you hadn’t figured it out when you checked out its cousin ’TOP’.

As if that wasn’t bad enough, things are now complicated by the architecture. The requests are – by design – stateless as far as the server is concerned. A $top option in a request is not too bad and easily ABAPed.

$skip on the other hand – ewww! It is stateless, which means that the only way you can relocate your “logical cursor” to satisfy the request is to go back to entity 1 and read your way forward $skip entries. Other than adding a stateful component on the server side that is linked to the GET method I cannot see how this mechanism for traversing through paged data can be good for performance.

On top of this, the ordering of any logical pages has to be consistent so that you get the same instances in the same location in each page. That’s one for individual implementers to decide on…

…which brings us to $orderby. This allows us (via IT_ORDER in the method interface) to specify a sort order. Since OPEN SQL has an ‘order by’ clause you will probably be tempted to use that however I’d be really worried that I’d be missing something in these disconnected queries. I never use that clause anyway as I was warned it would place too much burden on the DB server.

Since there are other features of OData that can satisfy queries with better performing results, I would limit the use of $top and $skip to “document model”  type resources that would typically have a few dozen ‘pages’ but could still have a few thousand and work well.

Update June 2013: if you are serving your data from HANA, you do not have to worry so much about performance and reselection. In fact, $top and $skip are delivered out-of-the-box by standard data services that you create in HANA – Nice!.

Well I think I’ve gone on long enough about those three – mainly to get you thinking about the pitfalls those options can open up. There seems to be a pattern there – the URI scheme OData suggests that the backend requirements for something like $top are quite lightweight and therefore the implementation should be light. Quite often it’s the case that the simplest options invoke the heavier load on development and resources.

In my next blog I intend to look at progressing to more complex and advanced topics such as navigations and better query options.  

To report this post you need to login first.

13 Comments

You must be Logged on to comment or reply to a post.

  1. Prasanna Prabhu

    Hello colleagues,

    Just like in paging we get the value of “top” in IS_PAGING and then we fetch only “top” number of records, how to handle this “count” ? To which variable do we assign this value..?

    Regards

    Prasanna.

    (0) 
  2. Abhijit V Patil

    Hi,

    Thanks for the nice blog..

    With such a granular nature of odata services and that too combined with stateless processing.. I really wonder if it make sense to use them in scenarios other than which simply queries DB.. In our development (and think most of the practical scenarios), data access is done through APIs (RFCs) which provides complete hierarchical data for data model in one call..(to get the data for entity at third level, data has to be loaded beginning from the first entity.. I am talking of standard SAP solution). Now making call to it again and again when odata navigation is done from one entity to other (because of stateless processing) sounds too much of performance issue…

    Is there any solution to improve performance in such cases? e.g. cache data at query to first entity and using cached data when navigation happens to next levels?

    Regards,

    Abhijit

    (0) 
    1. Ron Sargeant Post author

      Hi Abhjit,

      You have a point, but it does not mean that the OData protocol is not suitable. I think the problem is that a lot of developers coming to GW/OData don’t change their mindset, especially in terms of using RFC functions. The best design pattern, in my opinion, is the outside-in pattern, where the OData model determines the implementation. A common ( and bad) practice is to take an RFC function and build a service out of it. While this is an accepted method of providing the data, it does depend on the data source and not all functions are equal. OData is by nature a discovery protocol, so loading all data that might be possible from a given root is not the way to be thinking about it.

      That said, there are a fair number of features that can help. Using $expand or $batch will enable the query to execute statefully, i.e. each access method will run within the scope of the call, thereby any data from a mass retrieval at the start could be spread around using a helper class.

      Fiori (UI2)  also has a new cache feature, so that entityies can be retained in the backend without reloading.

      If the service is coming out of HANA, the replication of effort will vapourise under the sheer power!

      Regards

      Ron.

      (0) 
      1. Abhijit V Patil

        Hi Ron,

        RFC is just an example. In many of the standard applications, in order to read data at nth level, you need to load complete set starting from the first level.. In our specific scenario, we work with SAP standard solution utilizing two dimensional versioning which complicates things further.. i.e. to read/change data, one need to load application (version) for a given date starting at first level. (which is expensive operation). there is no way to load nth entity alone.. and now loading version at every navigation is just not prudent..

        Without having capability of state-full processing, there will be many redundant operations.. Outside In thinking is also can’t help because of intrinsic nature of the SAP application..

        Thank you for the pointers regarding other features/option.. I will explore them if they can be helpful in improving performance…(HANA is distant dream for customer.. 🙂 )

        Will love to read more blogs and insights from you!

        Best Regards,

        Abhijit

        (0) 
        1. Ron Sargeant Post author

          Abhijit,

          Gateway is really a lightweight data provider and not a replacement for all that has gone before. It sounds like your backend datamodel is a heavyweight and actually requires a lot of work to transform it into something that works well in Gateway. I can speak from experience that building a backward-looking service is much more painful than a forward-looking one.

          I would have to assume here that you are not using an SAP UI, because there’s no real reason to use Gateway for such an application as this.

          regards

          Ron.

          (0) 
          1. Abhijit V Patil

            Hi Ron,

            True.. I see it will be difficult.. currently we use custom developed SPROXY services for web enablement of solution.. and evaluating possibility of developing odata services because advantages seems tremendous..

            SAP UI exists but not used widely as it makes sense to provide more user friendly UIs to the clients for self service through browser/ mobile..

            Thanks!

            Abhijit

            (0) 
  3. Jayesh Vorani

    Ron,

    Thanks for this post. Very useful.

    I was wondering if there are any good practices around naming conventions for the project names that we create using SEGW and what is the best way to group different entities in different projects given project name becomes part of the URI?

    Kind Regards,

    Jay

    (0) 
  4. Kalyan Chakravarthi

    Hi Ron,

    Nice Post.

    I have searched over SDN to find about how do we pass multiple value ranges from Query to SAP ODATA?

    But I have not found suitable answers so I am posting it here.

    If we need to pass a date parameter in Query which has a multiple range like  sales orders created date between 03/02/2014 to 05/07/2014.

    How do we phrase it in Query ?

    I tried as below but the IT_FILTER_SELECT_OPTIONS of /IWBEP/IF_MGW_APPL_SRV_RUNTIME~GET_ENTITYSET does not get filled up with the parameters

    How do we pass multiple values in Query?

    http://ctnhsapapp16.corp.ken.com:8000/sap/opu/odata/sap/ZCHAKRABK_MAINT_ORDERS_SRV/Maint_Orders?$filter=Maint_Plant eq ‘US19′ and B_st_dt gt datetime’2015-02-01T00:00:00’ and B_st_dt lt datetime’2015-02-28T00:00:00′

    Thanks in Advance.

    KC.

    (0) 
      1. Kalyan Chakravarthi

        Hi Ashwin,

        thanks for your reply,

        I have found solution for the query I posted, The Trick is correct placement of ( ) 🙂

        Please find the below URL for more clarification.

        http://ctnhsapapp16.corp.ken.com:8000/sap/opu/odata/sap/ZCHAKRABK_MAINT_ORDERS_SRV/Maint_Orders?$filter=Maint_Plant eq ‘US19′ and ( B_st_dt ge (datetime’2015-02-01T00:00:00′) or  B_st_dt le (datetime’2015-02-27T00:00:00’))

        http://ctnhsapapp16.corp.ken.com:8000/sap/opu/odata/sap/ZCHAKRABK_MAINT_ORDERS_SRV/Maint_Orders?$filter=Maint_Plant eq ‘US19′ and ( B_st_dt ge (datetime’2015-02-01T00:00:00′) and  B_st_dt le (datetime’2015-02-27T00:00:00’))

        By the above we can see B_ST_DT behaving as Range.

        Thanks,

        KC.

        (0) 

Leave a Reply