OData Everywhere
We’re well into Day 1 at SAP TechEd 2012 in Madrid, and while SAP NetWeaver Gateway has already been mentioned in this morning’s keynote (even though the keynote was more Sapphire-focused than TechEd-focused), and is noted as an enabler in various conversations public and private, there’s a particular part of Gateway that is shining through as today’s story for me: OData.
Just now, I attended the SAPUI5 Q&A session with Tim Back and Oliver Graeff, where they presented a great overview of the libraries, tools and features of what is becoming an ever more popular platform for outside-in UI development. After all, it’s almost policy at SAP to use SAPUI5 for development projects, where appropriate. (“Where appropriate” means in many circumstances except probably heavy power user application UI paradigms). One of the key features of SAPUI5, and in particular the DataTable controls, is the ridiculously easy consumption of data. In particular, data made available by Gateway, in the form of OData. Sure, as I’ve noted before, SAPUI5 can consume arbitrary XML and JSON too, but the data exposed in the related, resource-oriented fashion by Gateway, OData in other words, is where the magic happens.
Start with controlled definition of resources, and the relationship between them, done in your systems of record using the IW_BEP backend Gateway component building Model and Data providers, either manually or using the Service Builder. Then expose those resources and relations to your UI developers using the core Gateway components (GW_CORE and IW_FND). Then, you’re off. Within no time you can start to see an application form around that data, with the right layer performing the right function with minimum friction. And that speed comes from the investment SAP has made in OData, an investment to make it all pervasive and all consumable.
So we know about Gateway being a key mechanism to expose OData for ABAP stack systems. Is there anything else? You bet. SAP HANA, full of data, can expose that data in an OData context. Use the magic of xsodata, create a definition marking a HANA table or view in a schema as an Entity, and *boom* you have a consumable OData service. And it doesn’t stop there. There are facilities in the NetWeaver Cloud to produce OData too.
What does all this mean? Well, to me it means two things. The first thing is that it means that Gateway has already been a great success. It Just Works(tm). I recently completed a customer project which went live earlier this year, and Gateway was a key component in the integration architecture. And after setting Gateway up and defining our entities and the relations between them, we moved up a layer in the stack and never really had to work hard on Gateway at all. It did exactly what it said on the tin. We started to use, and reuse, entities that we’d defined, in building out the features in the consuming application.
The second thing is how important your investment in Gateway is. Embrace Gateway and by definition you’re embracing OData. Before you know it you and your fellow developers are conversant in Entities, Entity Sets, Associations and Navigations (the relationships) — the building blocks of information in OData. And while this is a super end in itself, you’re also setting yourself up to move out into the cloud, and across onto HANA. Have a look at the speed with which you can put together an app that consumes data supplied to it from Gateway. And then consider you’re investing in that speed, and that speed across platforms.
Hi DJ,
thanks again for sharing your insights of Gateway! What I can't imagine right now, is how the security model works (I know security is boring in comparison of doing nifty apps 😉 ).
Where is proper point in this szenario to check authorization, whether the user in front of the client is allowed to consume this data? Does Gateway always has to forward user from client to appl.server?
Layers:
-1- Client [Browser|Mobile App]
-2- Gateway
-3- Appl.Server (ABAP stack + DB behind it)
For example we have a simple service to read table MARC from ABAP system. MARC has two keyfields: material and plant. Now I want to restrict this service that not every user can see data from all plants. To do a proper selection only on plants user is allowed to see I need to know in ABAP system which user request this data. So Gateway need to get this info from client as well. Am I right or totally wrong? What is the best way to implement such authorization questions and restrictions while using Gateway?
Forgive me if you've already shown this in one of your fantastic blogs/videos and I didn't catched it up.
--
Steffen
Finally catching up with conversations after a whirlwind two weeks on conference related activities.
You ask a good question and a very relevant one, Steffen. I had a brief conversation with Andre Fischer in Madrid on a very similar subject, and he mentioned SAML and other tech in his response. Rather than step on his toes, I'm going to leave this for Andre to answer (as he expressed an interest in doing so), although it's worth pointing out that (a) as developers we don't want to re-invent or re-code the existing authorisation concept (either in the backend or the frontend) and (b) the username is made available to the backend (IW_BEP) providers (except in the case of anonymous usage, obviously).
dj
Hi DJ,
But isn't exposing HANA through OData moving closer to the "ODBC for the web" idea an further from the idea of a resource representation of a true RESTful interface? Were you not the one that sold us all on the idea of REST and resource based modelling 😉 - and now abandoning it ?
Jumping to the topic of UI5, I worry that by adopting UI5 are we distancing ourselves from the huge number of non-SAP based developers that could and should consume OData in their app builds? Shouldn't we instead be building beautiful applications harnessing the huge amount of work that is occurring outside the SAP world - like jQueryUI for example. I can't see that UI5 can or will ever be able to keep pace with the open sourced / crowd sourced solutions out there. If we want to expand the SAP universe to the masses as per Jim's keynote, then the traditional SAP based (and I place UI5 in this space) developer will have to be in the minority? No? I tried to build an editable ECMAScript based table that could be populated by a JSON feed yesterday. Not possible (afaik) using UI5, but I found enough in the open source domain that I could build one myself - and it fully supports jQueryUI themeroller styling. Perhaps I just don't get it!
I also think that the possibility to build and run node.js apps should mean that the AJAX data consumption of the app can more easily be done without resorting to OData. Indeed in order to access many of the richest UI controls it's mandatory - I can't use OData with something like the dygraph library which can, however, consume Google Data Visualisation JSON format http://dygraphs.com/#gviz .
I love Gateway for its ability to open up and allow modelling of resources in an SAP system, I dislike that I can't do any content negotiation with it, but I can suffer that because it makes the consumption of data so much easier. Yet I would much prefer that it gave me some choices on how to consume that data - i.e. not just OData.
Anyway - love your post - it flows with the energy that you seem to habitually exude towards any new tech. Thanks for sharing - enjoy the rest of Sapphire/TechEd.
Cheers,
Chris
I had the same thought, but then noticed that DJ didn't mention 'REST' once in his blog. Exposing the domain model directly (though entities) isn't RESTful in any real sense of the term (in fact guys like Jim Webber would probably use some colourful language if anyone equated it with REST). But it sure is productive, and for a lot of use cases, that's probably okay. 🙂
Ah to sacrifice our ideals for the goddess of productivity 😉 aren't you supposed to be an "architect" Sasha - surely you can't be practical now too 😉 😉 (felt that deserved double sarcastic comment flag 😀 )
Just as long as no-one tries to sell me on the HANA Gateway exposure as being RESTful 😈 !
I'll have to invoke Eric Evans here on this and say that one has to accept that not all parts of any large system will be well-designed. And that's okay. Not everything has equal value obviously, so productivity will be very important for less-critical parts of a large system. More long-lived, continuously-evolving, mission-critical parts of a system which form part of the core of a company's domain (my guess is <5% of a system would fit this bill) would certainly benefit from a better approach, with hand-waving idealist architects all over it 😉 OData is probably not the best tool for that job due to its "ODBC for the web" heritage. So no sacrificing here, just less zealotry 😉
Agree with DJ and Sascha on this: no, it's not very RESTful, but yes, it does the job, and it does it with minimal cost and effort.
Don't use this if you are building your next big must-last-at-least-10-years system. Use it for the more nimble (is that the right term?) application (mobile app?) that you want out quickly, and which probably won't last more than a few years anyway. I think we're going more and more in this direction of throw-away apps, and for this category OData/Gateway is an excellent choice.
Just my 2ct
Surprisingly I agree with most of the comments so far.
Whilst OData has a lot of short term wins and is a good choice for the rapid development of disposable applications, a good strategy is one based on creating options, I am not seeing any alternatives.
When I read DJ's comments about the "magic of xsodata" it reminded me of the cake mix talk - Hibernate should be to programmers what cake mixes are to bakers: beneath their dignity.
JSP
That there's fighting talk! 😉
I do vaguely remember that cake mix / hibernate talk, which seemed to me at the time as somewhat disingenuous, while making some good points too (is that possible?)
When I bake bread, I don't use a bread mix, and sometimes even mill my own flour but I don't usually fall back to assember when I'm coding something. OK, sorry, now who's being disingenous 🙂
Seriously though, I think I might have used a misleading word ("magic") in my description of xsodata. For me, it's just a way of describing entities, their db sources, and the relationships between them.
dj
Disingenuous (or dissing genius) is a great way to describe that talk, nevertheless it is very passionate and entertaining, with a couple of key messages for the new breed of "Cake Makers".
wrt "Magic" my bad, I actually thought you were giving a backhanded compliment, I will take time to seek out and understand xsodata.
jsp
The idea of the 'throwaway app' was quite widespread, at least in the circles I talked within at SAP TechEd Madrid. I guess that might have been partly due to the spectacle filters I was wearing at the time.
There's a big contrast between the "10 year app" and the "18 month app" and there seems to be a focus on the latter at the moment, partly due to the novelty of SAPUI5. I don't see why SAPUI5-based apps should not be designed for the longer term. Perhaps it's partly due to the nature of where SAPUI5 comes from and also the changing substructures (e.g. jQuery libs) and superstructures (e.g. webkit browser engine) that may not look anything like today's versions ... in 5 years time. And these things, unlike the DIAG protocol and classic dynpros, are not in SAP's control.
dj
p.s. PBO/PAI forever!
Hi Chris
Thanks for an excellent set of points. As Sascha Wenninger and Fred Verheul have already pointed out, OData is certainly not REST, or vice versa. Yes, OData does have some RESTful qualities but ultimately comes from an Atom stable that focused on feeds and individually addressable items within those feeds (Atom Publishing Protocol etc), embracing HTTP for what it is (application protocol), rather than for what it can also do (transport, ahem, transfer data).
I can't help but wonder what might have happened if the challenge within SAP had been set: "Build a set of deliverable software components in the form of a REST framework". I managed to talk to Reiner Hammerich at SAP TechEd Madrid, who was involved in the early decisions to adopt OData as the application protocol (and by inference also the data format). One of the fundamental goals was to make it easy to consume SAP data and functions -- which with SAP's OData implementation it certainly is -- rather than to build a REST framework.
And I guess the challenge might be, bearing in mind REST is "only" an architectural style (as you know): What would such a framework look like? Would there be generators? What about URL templating? What else might be necessary for a deliverable and usable framework? Would those components be considered RESTful by us? (I'm thinking in particular about URL templates as is a bone of contention in some REST camps).
Your points on non-OData consumption by non-SAP developers is spot on. I don't think there's any easy answer but I certainly agree that there's SAPUI5, and then there's "the rest of the UI frameworks". But look at how much traction SAPUI5 is getting, admittedly inside of SAP rather than outside - once that takes hold, I can imagine the reach to SAP's customers' customers being enabled by apps that SAP's customers build; and being SAP customers, SAPUI5 is a viable, nay recommendable, candidate. Yes, there will always be non-SAPUI5 tools that are great (I built a simple jQTouch based app consuming SAP data in the UK & Ireland SAP User Group Conference last week for example) but are these tools more likely to be used by developers outside the "SAP shop" (i.e. by groups other than "SAP's customers"? (It's an open question, I don't know the answer).
Of course, it goes without saying that JSON is JSON is JSON. I'm just wondering whether (or how hard) I'm going to get clobbered by you lot by suggesting that you could always treat the JSON output from Gateway as non-OData 🙂 In the apps I build for pleasure, I take a JSON-based source, and my focus is on how to get to the data artifacts that I need, not whether the JSON format represents any particular higher-order protocol carrier 🙂
dj
Thanks for taking the time to reply DJ, I know what it is like in the post TechEd rush - I'm only just finding some spare time now - (admittedly if I hadn't been up until 3am each morning during TechEd Madrid keeping touch via web links and twitter, I might have got there quicker!)
I hear you on the OData != REST story, I just wish that the SAP mobile marketing machine would hear the same thing - as whilst Gateway is seen as providing that RESTful alternative for developers we are all going to be told to suck eggs and make do. But I shouldn't be such as glass half empty grump - it's certainly more that before! Especially with no additional costing to use it for existing users! (in the HCM space with ESS users, that's a BIG win!)
With UI5 I'm very scared that soon I'm going to be able to see a customer facing application delivered that is going to SCREAM at me "I WAS DEVELOPED USING UI5". I think that is the last thing that SAP customers want or need. It's like exposing WDA eRecruiting to the outside world (not a good idea). Customers should demand a more flexible and style-able set of UI elements. I should point out at this time that I very much dislike JQuery Mobile for the very same reason - every time I see a JQuery Mobile app it just shouts at me "HI I THINK THE iOS NATIVE DEVELOPMENT GUIDELINES ARE AWESOME" - the experience is styled on iOS and makes my Android fanboy heart sink each time I use it (I don't want back navigation to be on the top left of the screen please! I have a back button for that s**t).
If we encourage enterprise developers to adopt the UI5 tooling, my fear is they will never leave that mould and when the customer apps come, comes more UI5. And the land of beautiful user focused applications becomes a distant myth. At least in SAPland, in the land of faraway where the mobile developers that SAP desperately need to engage to start delivering on their mobile promises live, they don't speak UI5 and OData would be something that is suffered to speak to Microsoft systems, and they are still building beautiful apps. So I wonder why spend all the effort in alienating those folks, rather send the SAP community over the them to try and drag a few in.
If you thought you'd get away with suggesting OData over JSON is just JSON that is easy to parse, you've got another one coming! What about having to understand all that pagation stuff? if my result-set is large I'm going to have to set multiple queries to get it, and I'm going to have to talk OData to get that done... Nope, one needs an OData abstraction lib - like UI5 provides - and that's were I think it should have ended..(but that's more tied up in my first waffle).
Again, thanks for taking the time to respond to my rants!
Cheers,
Chris
Hi Chris
WRT to SAPUI5
I thought the same thing when i saw the latest version of SAPUI5 used JQuery Mobile, I really dislike applications which are targeted for iPhone/iPad on my Android devices.
However SAPUI5 caters for this, it renders specific to the device, see pic below. I have tested this feature out on both device types and the design is both effective and surprisingly responsive. For testing the different forms in a browser you use the parameter sap-ui-xx-fakeOS=ios/android.
Further to this at Innojam Bangalore which just passed, we witnessed a number of teams showcase their SAPUI5 applications where the themes did not look anything like SAPUI5, in fact some presentation it looked like the developer had adopted something similar to Twitter Bootstrap, I am not sure how they did it, especially given the time allocated, but I believe there is a SAPUI5 theme builder in the works.
JSP
Hi John, I still see a back button! 😀 - however it may have been styled to not look like a jQuery Mobile back! 😉 I've raised this before and am still probably in the minority (apart from that dude who runs facebook and a few others http://www.zdnet.com/facebooks-mark-zuckerberg-knocks-html5-in-favor-of-native-apps-7000004082/)- making a single solution across platforms is a mistake, users expect different ways of doing things, the single HTML5 based approach does not give best results (yet!) I expect that we will soon reach the point where a simple ECMAscript lib can help us render two/three completely different different UI experiences depending on the OS used and the size of the device. What's more we will only need to design one time and have the platform sort out the OS relevant styling (and a little bit better than the above pic)
However, and this goes to the point about consistent styling as well, it may be worthwhile for companies just to get stuff out there, use it, refine it or throw it away. Aggressive Agile development - release stuff to users fast, get feedback, try again. Using non-persistant HTML5 web apps gives by far the fastest deploy cycle you can hope for (each time user accesses, the solution is updated). By using a familiar UI people are much faster to adopt and build will be faster too. This is good, no?
I don't know, I just feel that it feels funny to me for SAP to be investing in an ECMAscript UI library when so many UI elements already exist and continue to evolve everyday. I've never seen SAP as being really good at UI design, but hopefully the whole design thinking idea might be mixing that up too...
Thanks for the extra detail - I'd heard that there was only very limited theming possible with UI5 - will be good to see more coming.
The Back button was added by the developer, i am not sure why, maybe to highlight the different themes. Testing some of the other sample mobile applications on my phone, it appears to me that the default behavior for Android navigation is very similar to Up vs Back like you get in apps like Gmail and Play.
Gotta love your rants, Chris, they make me smile (in a good way!).
One of the themes of your comment is styling, and in particular the worry that all the apps built with SAPUI5 will look the same. While I agree that the promise of HTML5 includes the ability to break away from the rigidity of any particular UI framework, I must admit to being quite taken with SAPUI5's Shell (https://sapui5.netweaver.ondemand.com/sdk/#content/Controls/UX3Controls/Shell.html). The reason is that it could be set to provide a core navigation familiarity for users. Not necessarily a casual user but perhaps the group of people in between the power user and the occasional consumer out there on the interwebs. With the workset and pane bar items, you get a consistency that we don't want to pour out with the rest of the bathwater. At least, that's my opinion 🙂
And as John mentioned also in reply to your comment, there's theming in SAPUI5. There's certainly theming stuff in the works and already you can theme SAPUI5 to make it almost unrecognisable. As I went round SAPTechEd in Madrid the other week, I'd spot apps being demo'd and would say to whoever was with me - "hey, is that SAPUI5". The response from both of us would often be "err, I'm not sure". Mostly it turned out that SAPUI5 *was* being used. For me that's a pretty cool indictment of the theming flexibility.
Oooh I almost forgot the part about JSON. You're right to call out on that, but I've messed around with pagination with Twitter's 'paginated' JSON responses, and unless I'm missing something (not had my coffee yet) I can't see there being a huge difference. But I agree that an OData abstraction lib is perhaps a good middle ground. There is some part of me that's slightly uneasy about saying that we need a lib, but I'll leave it at that for now. There's still some KoolAid left at the bottom of the bottle 🙂
dj
Thanks DJ, great reading your thoughts.
As I mentioned in reply to John, perhaps consistency is good for user experience support. I mean that's the reason I disable the ability for users to customise/hide/move anything in all my WDA apps. And I can't remember how many times I've said that by using a standardised technical approach support becomes easier. Was I lying? Not really, was I wishing there was a better way, certainly. Doesn't stop me from wishing however, that the apps I built were so good that users wouldn't need to call a support desk that would get confused as to why the button was on the right rather than the left of the screen. Perhaps in the next iteration after UI5 😉
I'm pretty sure that I could style UI5 to look how I liked - after all it's got classes for every little thing that I can address with CSS - but that does mean a lot of custom work currently. I look forward to more extensive theming capability!
I'm off to a non-SAP developer conference http://www.yowconference.com.au/ at the end of this week, where I doubt any will have heard of OData and certainly not UI5, will be interesting to ask a few people there what they think, as I think this is group (non-SAP developers) that SAP need to engage to really win in the mobile app space.
Hopefully I'll get my KoolAid bottle topped up too - and if not I'll just spend an hour or two with ATAlisdair Templeton and I should be good 🙂
That's great, and I think non-SAP conferences are almost essential for us SAP developers to remain rooted and open. Hope you have a great time!