Show your cards, please!
Let me tell a short story.
Imagine a SAP customer having a webapplication using SAPUI5. The architecture is a typical transactional Fiori-like application.
Meaning: SAPUI5, connecting to SAP Gateway via OData. The services are implemented in the ECC backend via ABAP OO.
In our case leveraging the SAP PM/CS module. This is working perfectly fine. Like this:
HANA
Due to new requirements, HANA as platform came into scope as well. Searches do have to stay fast even if the amount of data will grow on our ECC system. Another reason was to be able to search in ‘legacy’ data. ECC Tables will now be replicated to HANA enterprise. Also the legacy (read-only) data will be loaded into HANA. Search services are implemented here via OData as well, in this case via xsodata. So a 2nd OData source come into play.
Pictured it in mind? Great! Let’s draw it.
So our new picture would ideally be like this.
SAP Gateway as our central access point for services. Meaning we can still to tracing/ logging as we are used to have with our initial architecture with the SAP Gateway to ECC setup. See response times, trace activity for a limited amount of time. And have a single and central access point for all our services. Ideally as well, since we have to deal in SAPUI5 with the ‘same origin policy‘ issue. Also for other applications later on which have to benefit from these services, can benefit from a single access point.
As Sheldon Cooper, would say:
All set… Or not?
External OData source
When we want to use SAP Gateway as our route to HANA, means we have to call an external OData source (HANA) via SAP Gateway.
Let me explain; Calling external OData services (in our case HANA) via SAP Gateway can be done, it is called OSCI. It is explained here. So we did. We run the wizard, and ….. it failed. “Metadata cannot be loaded”. It turns out a ‘http://’ deep down in the class is missing. No worries, when adding this via the debugger, luckily we could continue. Generation was successful. The metadata could now be loaded. All fine? Actually no, when testing our ‘new’ service via the gateway client, it failed. It had a different interpretation of ‘nullable’. Meaning all records with empty values failed, The default of Nullable is differently interpreted in Gateway. Nothing to worry as well, I guess we can fix that by making our definitions more explicit on the HANA side.
What was unexpected is, a complete new service is generated in Gateway. A completely new set of classes. Doing all the logic as you would have when generating a Gateway service using an ECC backend via an RFC. Meaning it parses the incoming request and check for validity against the metadata. Especially responses are completely parsed. Not only it takes time (we run 60.000 records through it, and it took >1,5 minute before a parse error came up). But parsing and validating the response would anyway be done in the source system, in our case HANA.
Sheldon is over-reacting when hearing this news.
Of course there are reasons for this, and there are great teams working at SAP, who must have a very good reason to implement it like this. In our scenario it is a bit unexpected, and behavior which is actually not required.
Our requirement would be fulfilled with having a proxy behavior. Proxying requests to HANA. With Gateway as proxy, but still enabling logging, tracing and response time measuring. A pure representation of a proxy is shown below.
What does Fiori do?
How does standard Fiori do this, when calling HANA OData services from a Fiori application. Is Gateway being leveraged here as a central point of access? Is it using OSCI?
This picture is directly from SAP documentation. The “Setup of SAP Fiori System Landscape with SAP HANA XS”. It describes the architecture for an analytical application. A SAPUI5 application consuming OData services from HANA.
What do we see? The webdispatcher. The good‘ol webdispatcher. This does mean it will do simple proxying, this also means it will not be consuming time in reading and validating and parsing the response. Great? Yes, sure it is acting a proxy. But remember, we want to have full tracing, and logging options. To be able to see if a request has failed. To turn on a trace for 2 hours and watch what happens (for a particular user). To quickly spot the services which have high response times, and possibly cause time-outs.
Solutions
We could implement our solution with bypassing SAP Gateway for HANA OData services. Since we are serving the SAPUI5 from a Java stack (Gateway is not our frontend server). We implemented a proxy here by using a Java proxyservlet, to route requests to HANA. However we do not have the routing and logging, handy tracing options and response times measuring in (one) place. It can be build and implemented here, sure, but it’s not standard.
It could be, that our future architecture will be a even more Fiori-like landscape. Gateway, ECC, HANA. The Gateway server being the frontend server, this stack running the Launchpad and offering the SAPUI5 applications. Meaning it might grow towards an architecture without the Java stack, this means we cannot do the proxying anymore here.
Roadmap
What is SAP’s roadmap here? In scenario’s with more OData sources being consumed in one or in more applications.
Is the webdispatcher the final solution, as in the Fiori analytical landscape? Monitoring and managing is less advanced here as the customer would prefer.
Is there another way to route and monitor and manage these different API’s via one solution. Will SAP Gateway get proxy functionality?
Or is this why SAP started a partnership with ApiGee?
Although it seems to be more focused on cloud based solutions, will this also be available for on premise usages as API management for all (SAP) OData sources?
Please SAP, show your cards!
Thanks for making it onto the end of this blog. Please share your comments!
I guess the winner (and the answer) is SAP HANA Cloud Platform. SAP wants to be Cloud company and HCP as Java based PaaS offering is the missing puzzle piece.
HCP has all the necessary capabilities to provide a solution.
hi,
Thanks for your comment. What specific functionality in HCP did you have in mind here?
Regards,
Wim
Hi Snoep,
Just a quick clarification on the above feel good blog.
When we expose HANA artifacts as "Stored Procedures" and consume them from ECC(NW 7.4) via AMDP(ABAP Managed Database Procedures - Introduction) could we break through this proxy/additional parsing overhead ? Then we might need only one OData source ?
I am just curious to listen from you...
Regards
Prabaharan
hi Praba,
Thanks for your comment. Sure this is a valid option as well, but actually we do want to push down as much to HANA as possible. Meaning, we rather not want to execute any application logic on the SAP Gateway ABAP stack. And let HANA generate the OData, via xsodata.
Cheers,
Wim
Hi wim,
Got your point. Is there any reason why you aren't opting to write application logic in gateway stack. i.e.,write AMDP inside a RFC in ECC to consume HANA artifacts and call them from gateway stack.
Just curious to know from you.
Regards
Prabaharan
While I've always been behind the "single access point" concept that Gateway was offering, in reality I think that term is becoming something of a compromise.
Initially, the job of Gateway was to provide data conforming to a standard data protocol (now it's OData, making it more friendly to non-SAP consumers). What this meant was the ability to aggregate access to various backends into a single access point whose chosen protocol was OData. The key point here was that those backends were very unlikely to be able to produce OData feeds directly, so Gateway acts as the parser in that equation.
When I first saw HANA producing XSData, I did think this could become a dliemma. Whilst very good at translating to-from OData from whatever form the source was in, GW could not actually converse in OData to OData fashion - very odd! OSCI has solved that to some extent, but you've discovered the overheads yourself.
Now I am rethinking the role of Gateway. It's really the single data access point for backends that can't provide OData directly. Where platforms can provide OData directly, why use Gateway, other than the authorisations, etc associated with single access points?
Use GW to consume data from ERP, CRM, etc and even SOAP by ABAP proxy.
HANA, BO Universe, PLM and others already have the capability to talk OData, why introduce unnecessary consumption brokers?
Which leads me to think that this is only solved by "SuperGateway", which is a proper single access point and only communicates with OData feeds, our current Gateway being one of them. SuperGateway would be a pipe rather than a parser. Maybe I'm oversimplfying, but the message 'keeping simple things simple' seems to ring less true the more we go down this road 🙁 I want it to be simple!
API Management might be a step towards SuperGateway - should be getting my hands on it soon so I will be looking at this angle for any sort of coverage.
Regards
Ron.
hi Ron,
Thanks for you extensive reply! And I'm completely with you on this route. Nice naming BTW, superGateway, this is exactly what it should do. I'm eager to hear more about the plans with API management, as what this SuperGateway is. Probably SAP can enlighten us, to what this might look like.
Apigee for example has the option of throttling, and this is an interesting functionality to what a superGateway can have as added value, besides logging and tracing.
Regards,
Wim
Hi,
I noticed something interesting a few days back while reading through the new features in the latest Support Packs.
See: What's New in Social Media Integration - UI Technologies - SAP_UI SP10 - SAP Library and: Configuring REST Tunnel - Social Media Integration - SAP Library
Quoted from the help documentation:
Could this possibly be the answer you are looking for? It seems to be a brand new feature so I haven't had a chance to play around with it yet but probably will in the next few weeks. Not sure if it offers performance and tracing capabilities.
Cheers,
Brad
hi Brad,
Nice find! This might solve the need for having proxyfunctionality on Gateway. It seems to be created for SAP Jam. But looking at documentation you could tweak it to proxy somewhere else. However, I don't see any documentation on the possibility of tracing and logging in here.
Cheers,
Wim
Does this provide any insight?
http://service.sap.com/saproadmaps
Click their Roadmaps link (above), then go to "Database & Technology", then scroll down and click the "SAP Gateway" Roadmap doc...
Is it possible that "Integration Gateway" is the answer? For example slides 35, 39-42
hi Michael,
Thanks! Interesting link you provided.
Sure integration gateway might be one of the options, either on PO or SMP. Although I question if it has the logging & tracing options as we would prefer.
Cheers,
Wim
Hi Wim,
great blog, and valid request to SAP. Wrt API Management, according to Joav Baly in this blog it is indeed the intention to also deliver this capability for on-premises: "SAP’s Apigee-powered API management will be available as a hosted solution on SAP HANA Cloud Platform, and SAP will also resell Apigee capabilities as on premises solution".
As I am struggling with same type of doubts/uncertainties about the diverse provided options to consume functionalities into mobile Apps, I wrote down my own impressions about the position of SAP Mobile Integration in a blog posting. "Sadly" it is within Dutch, but for you that will not be too much of a problem 🙂
Regards, William.
hi William,
Thanks and thanks for your comments. Slowly SAP is bringing API Management in to our attention. A couple of days ago, the SCN space regarding SAP APIs and API Management was made public. I wanted to initially post the blog here but couldn't yet. Luckily it's open now.
True I agree, mobile is one of the areas where this question is valid as well. My blog is broader actually and not specifically for a mobile solution.
Thanks again, and I succeeded in reading dutch 🙂
Regards,
Wim
Hello Wim,
very well written blog - thanks for the effort and also thanks for the good comments.
I see that there is a need to:
a) explain what is possible today - especially across the various product stacks and also having all the new technologies in mind (CDS, API Management, Gateway Java, HCP ...). What can help, what would make life more complex, ...
b) improve some aspects of end-to-end scenarios (integration, functionality) on our side.
Most colleagues are already in their christmas vacation - so I still owe you a good answer for the blog until 2015. So stay tuned...
Best regards,
Martin
hi Martin,
Thanks for your response. Glad you took the time to read the blog and write down a reply.
You touched the different answers I'm interested in, and would like have a response on. I'm looking forward to your answers!
Thanks,
Wim
Hi Wim,
the solution for the scenario that you depicted would be indeed SAP API Management.
Though it is technically possible to generate an OData Service using OSCI it is not the best way to achieve proxying an OData service since (as you found out) creates some overhead.
Monitoring of an OData service should be done in the platform it is running on. So an OData service published by the HANA XS engine is best monitored using the HANA XS platform. The monitoring and debugging capabilities of SAP Gateway have been build with the goal to best support the ABAP developer when it comes to debugging and error analysis.
Proxying an OData service is better done by solutions whose job proxying of data is.
One of these solutions is the good‘ol webdispatcher. But it doesn't bring any of the more advanced features to the table that you are looking at.
SAP API Management instead can do much more than just simple proxying. Here the read only OData service available through the XS engine and an OData service to perform the CUD operations on the same object in ERP are both published as one API.
SAP API management would now dispatch the read and CUD calls of an OData client accordingly and would allow you to have some own performance metrics. For example the number of SO's exceeding a certain amount of net value in a certain time frame.
SAP API management would also allow to differentiate between different clients. It would thus be possible to publish different functionalities for different clients using the same API. It would for example be possible to allow only read access for mobile clients while desktop UI's would have the option to perform updates on the data.
Together with my colleague Balakrishna Gottipati I am currently preparing a SCN document that will explain a few of the added values that SAP API Management brings to the table for SAP Gateway.
Best Regards,
Andre
hi Andre,
Thanks! Valuable input, our roadmap is getting more clear now. Although it is another system yet in the landscape, I do believe SAP API Management has the features we are looking for, and still does proper proxying. I am looking forward to your document on the added values of SAP Gateway combined with SAP API Management.
Regards,
Wim
Just curious to know the features that API management brings to the customers.
Regards
Prabaharan Asokan
Hi Andre,
Together with my colleague Balakrishna Gottipati I am currently preparing a SCN document that will explain a few of the added values that SAP API Management brings to the table for SAP Gateway.
Do we have any updates on this ? Thanks.
Regards
Prabaharan Asokan