Additional Blogs by SAP
cancel
Showing results for 
Search instead for 
Did you mean: 
0 Kudos

This is the second part of a 4-part blog series. Please also take a look at Part 1, Part 3 and Part 4.

In my last blog I presented the Customer Fact Sheet (CFS) scenario, mostly by referring to the ES Wiki. I built a composite application ("mashup") using the ES Webservices in PHP tightly coupled with Javascript. I promised to write more about this in the second part, so here we are.

Overview

I've used a bit of AJAX already in the last source, but now we'll spice it up: adding a background loader to display basic data in the result list and changing the ordinary tables in the CFS popup to 'live tables', which can be sorted and paginated.

I'll shortly cover a few reasons to use AJAX, then give you a quick introduction into the necessary Javascript (for those who aren't already masters of it). I'll show you how to load additional data in the background and one of two basic methods of creating a interactive table. The other method as well as some more details will be covered in the following blog.

Disclaimer: I use the term AJAX in the more general way here. While it’s name is based on XML and often the use of the XmlHttpRequest (XHR) object is implied, many sites use different ways of communication to achieve the Web 2.0 style user interface and the general consensus on the web is that AJAX now stands for what it can deliver and not only strictly from what it's name was created. In this example I won't use XML for the dynamic part.

Reasons to use AJAX

AJAX and "Web 2.0" allow for breaking the traditional web application rules, namely submit and redisplay completely, with waiting time in the user interaction during which not only nothing can be done but also the screen is often blank. Web 2.0 style applications are much more responsive and allow to view information (and interact with it) before everything has completed loading or during reloading of additional data (using asynchronous calls). This is a very modern user interface and while traditional web pages when used as applications were a huge step back in useability, this now changes.

Of course I'm not the first to do this, there is a huge amount of information about AJAX and Web 2.0 available in the net, both in terms of samples, instructions, howtos as well as ready to use small to big applications. Daniel McWeeney blogged about creating AJAX Applications in BSP about two years ago and the OpenAJAX Alliance has a whitepaper about AJAX in business environments. Thus I won't go into deep debates about the usefulness of AJAX but give some overview what's needed to know to create a CFS-like application.

Surfing or Diving?

It was only a matter of time until advanced tools and frameworks emerged which allows the easy creation of applications using high-level blocks of underlying functionality - like a live table - and sometimes hiding away all (client-side) JavaScript, offering their modules in the server-side language (like PHP) only.

I decided to not use those frameworks but do things by hand here. This was mainly because I wanted to present (and try for myself) the different ways how things can be achieved and give a more in-depth look at how it works instead of using the building blocks of a specific framework only. A secondary reason was that I had no prior experience with them and would have to test all of them first before deciding which one was most useful to be presented here. I'll give a short recap on what I found on the net, though, at the end of this blog.

One of the most important things is the architecture decision how much functionality is really shifted from the server to the client. On rich clients, both is needed, an intelligent client and a cooperating backend. But the amount of work (be it data processing or data storage) given to the two sides of "the internet connection" varies. The 'live table' is a good example for this: the data can be sorted and the display broken into pages. Where is the whole amount of data stored (not only the currently displayed page) and where does the sorting takes place?

This is actually a similar decision to what needs to be done way below near the DBMS backend: given scriptable databases with triggers and stored procedures, how much logic is implemented in the DBMS instead of the application layer? For this presentation level I'll present both different methods, (web)server-side and client-side sorting/caching of the table.

But first you'll need some basic understanding how Javascript (and thus AJAX) works. Please bear with me if you're already familiar, I might add some new insights. And if not, feel free to comment or create another blog for all of us 🙂

AJAX Basics

The modern interface is basically two things: JavaScript (introduced around '95 by Netscape, named after but having no other similarities to Java and now being standardized as ECMAScript) and the Document Object Model (DOM, a way of accessing the HTML page in Javascript, somewhat standardized after every browser vendor started to introduce such models) to allow dynamic web pages, interacting with the user without the need for server connections and, second, the XmlHttpRequest-object (or alternative technics) to make the backend connection in asynchronous background calls (commonly refered to as AJAX).

Ok, I said I won't use a framework here, but I do use a library - doing crossbrowser compatible code is no fun when done completely by hand, so I take the great Prototype library as a base. It is a spinnoff from what 37signals created for their Basecamp collaboration software (which also resulted in Ruby on Rails). There are several other libraries building upon Prototype, providing visual effects and higher level components. I use script.aculo.us here.

Prototype allows to refer to elements in the DOM in a very straightforward way: an element with the id label can be accessed as $('label') and its content or style can be read or changed in a crossbrowser safe way: $('label').innerHTML = 'This is a red label'; $('label').style.color = 'red'; For quick reference, here is a Prototype Cheat Sheet.

When working with Javascript, a debugger is extremly helpful. I recommend getting Firebug in combination with the Web Developer Toolbar, also recommended elsewhere on SDN here and here. Firebug allows you to inspect the whole DOM, view and debug the Javascript, jump directly from an error to the corresponding line of code, etc. It is extremly useful and I wouldn't want to develop any Javascript without it - although the latest version seemed to have a considerable speed impact on normal surfing. The Web Developer Toolbar offers a wealth of options to help with creating HTML and CSS by hand, including getting form details (with hidden fields or passwords displayed) or outlining elements like table cells - especially useful when dealing with hidden elements (like unselected tab pages).

The basic magic behind AJAX is the asynchronous call: a Javascript element can trigger an HTTP request to its originating server (or others, though some tricks are necessary) and get some results back without clearing the page or letting the user wait in the meantime. The result is then returned to Javascript (using a callback function) and can be displayed. Since there is always a layer of Javascript between the returned HTTP response and the user the actual data format used is completely flexible. Common formats are XML (which gave AJAX its name), JSON (a lightweight alternative to XML) and plain HTML (used to directly display it).

Although XML is a good generic data interchange format between different applications, JSON is getting more and more favoured in the web because it is a much simpler, smaller and easier to parse format (in fact it actually IS javascript and can be just fed directly into the interpreter using eval(), making the parsing a one-liner). Sidenote: by the W3C definition for a Web service only SOAP/XML (and in public perception more narrowly only SOAP-over-HTTP) is considered a real Web service. The basic idea is similar, though, and quite some APIs available for mashups are in fact providing JSON.

Making an AJAX call with Prototype

Using Prototype, making an AJAX call is a simple matter - the library handles the crossbrowser compability and adds some nice functionality, while still being low-level enough to allow for complete control over all activities.

A simple call to get some data might look like this (taken directly from the search query of the CFS):

    var url = 'cfsdata.php';
    var pars = 'action=custlist&' + query;
    var myAjax = new Ajax.Request(url,
        {
            method: 'get',
            parameters: pars,
            onComplete: doSearch_Result
        }
    );

url is the URL to call. Since HTTP is used this URL can be directly opened in a browser, which is a quick way of debugging the backend.

pars are the parameters. Depending on the given method (GET or POST) they are appended to the URL or set as headers - to make things easier they can just be given as if GET would be used. query holds the user-entered search query to be processed by the backend.

onComplete is a function reference. This function will be called when the call returned sucessfully. It has a single parameter, the response, which holds both request and response headers and data. The function can then parse the response and update the page. There are also optional functions for failures which I leave out for this demo.

Enriching the CFS customer list

The list of found customers only show their names (beside the ID), which is what the used Enterprise Service returns. It would be nice to have additional information - like address data and telephone number - directly visible in the result list, which gives a better overview than having to open the details popup. This information is returned by a service named CustomerBasicDataByID which must be called for each customer individually. E.g. after a search term is entered by the user, the program needs to first get a list of matching customers, then process this list in a loop to get the basic data for each customer.

The classic approach would be to place this loop on the web server side and only return the result list when all data is gathered. The disadvantage is that the user has to wait until every call has finished until he sees the first result. So I put this loop inside the client: the customer list is transmitted (and displayed) directly, and the basic data is added to the table as soon as it's available for each customer. To notify the user that there is still some loading in progress, an animated icon (I call it "fake progress icon", as it actually says nothing about the completeness apart from just being there 🙂 is added to each unfinished row, and a status area shows the progress in terms of fetched rows.

The table is updated each time a basic data response comes back to show this information. In Firefox this can be done easily with the innerHTML attribute for a table row: just replace it with the new row and the table will be updated. IE, however, prevents this for unknown reasons. So I simply replace the whole table with a new version, using the innerHTML of the surrounding div element. The last hovered on element is then highlighted again. A small flickering can't be avoided, though.

To correctly associate the returned result to the corresponding table row I added the row index (js_rowid) to the data returned - this is a parameter which is given to the web server and just returned unchanged, embedded inside the actual customer basic data coming from the Enterprise service. This is a working method, but has the drawback that the web server must be changed to pass this ID along. I'll present a better variant using function closures later. (The internal customer ID couldn't be used as for this to be generic we mustn't assume anything of the actual data to be special, like being a unique key. Also, the customer ID is a string, not a number, which would require searching for it in Javascript instead of just having it as an index in the array.)

Live Table, client-side data storage

A 'live table' means a table which contents are directly coming from a backend service and interactivly displayed. This especially means the ability to sort them (to each column) and to paginate them, i.e. only show a small part of it and let the user navigate. Let's do such a table - for starters I only implemented the sorting, and only to the two fields which are always present, ID and name. The other data is available only after the reloading finished and at least the demo data I used was only partially filled in here, but the principle of sorting is the same for each column anyway.

The OpenAjax Alliance describes in their whitepaper some basic architectures for two basic questions: should the final HTML be created on the server or on the client, and should additional data be kept on the server or the client?

If you create HTML by hand (i.e. not using framework-provided building blocks doing this for you), the most elegant way is the use of "unobtrusive DHTML". This means a normal table is created in HTML, without any special javascript attached to it directly, and it just 'magically' allows sorting. This can technically be done by giving the table a special CSS class which tags it as sortable and have a Javascript snippet change all tables of this sortable class so that the headers are clickable and the rows gets sorted then. Stuart Langridge created such functionality in this article and there are some other available on the net, too, so take a look if you are interested.

While this is a nice separation between the data and the functionality, especially if you create your HTML by hand (e.g. using a template system), this approach has some drawbacks. For one, the column type - which is important for correct sorting - can only be guessed. Second, additional information can't be stored here. Pagination is also hard to implement - where to store the additional data not currently being displayed? (this excellent implementation actually manages this) When using higher abstraction levels this might even become irrelevant: if the view states there is a table with some data then creating the HTML to form this table doesn't need to be done by hand or server-side code independent from the client-side Javascript. In fact the frameworks providing live tables create both at once, and thus mixing of the HTML (display data and structure) and Javascript (display logic) is done in the framework and nothing to worry about.

Not using the frameworks we'll do this by hand here. The first thing is to separate the data we receive from the AJAX call from the displayed table - i.e. the backend does not directly deliver HTML, but a JSON array and the client Javascript creates the HTML table on the fly. This is similar to what the OpenAJAX Alliance calls "Dual DOM", though I didn't actually create a second DOM but just a simple 2D array.

When the user selects the column-to-be-sorted-by, the whole table is sorted and recreated, using the same function also used when adding the basic data during background loading. The initial sort is also done through this function, as the Enterprise service returns the results unsorted and it wouldn't make sense to duplicate the sorting functionality in PHP. Note that my code treats the data type as strings, data type aware comparison functions can be found in the examples I linked above (for Javascript) or in the next blog (for PHP).

Advantages: since no further server requests are needed this is really fast - no network latency at all. Neither is a session on the server required. Pagination could also be implemented easily - just add a startrow and numrow parameter to the display function and provide some buttons for the navigation.

Disadvantages: the clients needs to hold the complete set of data (which may be quite huge) and wait for result display until it's completely loaded. This might be problematic as Javascript in a browser could become slow and instable much sooner than the web server.

Note: when returning data in PHP which is to be interpreted as JSON, make sure there is nothing else than correct JSON in the result, or else the Javascript will fail - probably even silently, hiding the error cause. Especially PHP errors must not be displayed, so adding the line error_reporting(E_ALL ^ E_NOTICE); is advised, as PHP tends to throw notices quite often even in non-critical cases.

Security

Again, this is a demo appliaction and for enterprise-ready productive systems a more throughout security analysis must be done. But I want to at least mention a few points here:

  • Putting something which comes from from our own server inside an eval() can be considered as reasonable secure - after all, the whole rest of the pages comes from there and an attacker able to influence the JSON-string most probably already has enough access to the system anyway. However, if you ever want to get data with JSON from an untrusted source run it through some security checks.
  • "Don't trust user input" is the most important rule in (web) security. I expect the Enterprise service to do its own security checking so it's ok to just pass the value along. However it shouldn't be returned to the browser without escaping, as this is a primary source for XSS attacks. In the search status I display "searching for...". This is done completely on the client from direct user input, so it should be rather secure.
  • A good rule is "treat every data coming from the user as tainted, but unmodified until it is really evaluated - then escape it". This is especially true since escaping for SQL queries is different than those to embedd data inside the HTML. Remember magic quotes in PHP: they are a source of confusion, not security, and should be stripped away - especially when handing the data over to the Enterprise services. See the getVal() function in util.php for this.
  • WS-S is not yet supported by PHP, but HTTPS can be used in an untrusted environment.

Quick Frameworks overview

As promised I'll add a few links here what I found in terms of frameworks. I've not (yet) used any of them, so I can merely give you some pointers and summarize what I read there.

Good starting points when searching for frameworks are this Inforworld article and the Wikipedia entry on Ajax frameworks. According to these sources, Dojo seems to be one of the best frameworks, followed by Yahoo which also tries to set a UI design standard. On the other end is Prototype the best approach for having tight control of the source and only using basic functionality. Googles approach is interesting, letting the developer code in Java and providing a kind of Java-to-Javascript-converter. Naturaly this is more suited for the native-java-coder, while MS Atlas is aimed at the .NET developers, especially since the IDE is necessary and it and relies heavily on the .NET backend.

Then there is also Rico or the Symfony project in the high level or the Javascript Toolbox which covers more the add-on aspect to existing sites, e.g. the 'sorting a table'. I'm sure there is quite a lot more out there.

Outlook

Now we have a very nice customer list (table) which can be sorted and gets updated with detail information in the background.

I split the AJAX topic in two parts, so in my next blog I'll cover some more advanced Javascript stuff:

  • Replacing the js_rowid with function closures
  • Adding tokens to discard outdated requests
  • Using the 'server-side dom' model to have data storage and processing (sorting 🙂 on the web server side
  • Table pagination and sorting of all columns with respect to the data type
  • Some thoughts on performance and graceful degration
Of course this blog is rather dry without the sources to actually take a look into and try it out for yourself. I had zip archives prepared for each blog part, containing the whole application (PHP, Javascript, Images) in the different stages. Unfortunately they have not yet been approved by SDN, so you have to wait a tad bit longer until you can take a closer look into what I did.
3 Comments