SAP HANA SPS 09: New Developer Features; XSDS (XS Data Services)

This blog is part of the larger series on all new developer features in SAP HANA SPS 09:http://scn.sap.com/community/developer-center/hana/blog/2014/12/02/sap-hana-sps-09-new-developer-features

Additional blogs on this topic:

• XS Data Services: A Native CDS Client and Query Builder for XS JavaScript
• XS Data Services: Working With CDS Entities
• XS Data Services: Building CDS Queries

In SPS 09, SAP introduces a major extension to the programming model in the form of a reusable XSJS Library called XSDS or XS Data Services. There are several goals for this new library and the impact that it has on the overall programming model.

General HANA artifact consumption

• Native JavaScript embedding of HANA database artifacts
• Structured API calls instead of low-level SQL Interface
• Less boilerplate coding (for example result set conversions)

Native CDS Consumption

• Import CDS entities as native JavaScript objects
• Understand CDS metadata for working with JavaScript objects

The origin of this extension to the programming model is that there were originally three separate projects/libraries built within SAP.  They focused on different aspects of data consumption within XSJS. They were an ORM (Object Relationship Mapper) library, a Query Builder, and a Relationship library.

While these three individual libraries all had value on their own; ultimately SAP decided to integrate them together into one single library and also build them onto the CDS/HDBDD foundation for even better reuse of information that already exists within the database layer.

CDS (Core Data Services)

Core Data Services are a cross-platform set of concepts and tools to define semantically rich data models.  Within SAP HANA we already have the development artifact HDBDD for using the CDS syntax to define tables (entities), their relationships and views that utilize those relationships. A central part of the CDS specification is the object-relationship mapping that links semantically meaningful entities to their technical representation as records in the HANA database.

The short description of XSDS then can be consider as the native CDS embedding for JavaScript in the XS application services layer of the SAP HANA Programming Model.

At its core, XSDS allows you to import and extend CDS entity definitions.  You simply supply the package path and file name of the HDBDD and the entity name during the import. The XSDS library reads the available metadata on types, keys and associations.  It also supports extensions, projections, and renaming of entity definitions.  It also supports CDS conceptions (like backlink and via entity associations) which aren’t yet supported by HDBDD definitions.

In this simple example we import the Employees entity from the EPM hdbdd artifact and the MasterData Context.

$.import("sap.hana.xs.libs.dbutils", "xsds");
var XSDS = $.sap.hana.xs.libs.dbutils.xsds;
var oEmployee = XSDS.$importEntity("sap.hana.democontent.epmNext.data", "EPM.MasterData.Employees");

oEmployee is a now a JavaScript which contains all this metadata which was already defined in the underlying CDS model. It also supports full CRUD transactionality via this JavaScript object.

Import Pre-Generation

However, this import of the metadata from the underlying model can be a relatively costly operation. Because the XSJS programming model is stateless this often creates the situation where you must re-import the CDS metadata for every single request.

For this reason the XSDS library also support pregeneration of imports in order to improve performance.  The library serializes the result of the import into a single reusable XSJSLIB file. Therefore instead of importing the entity from the HDBDD artifact you can use standard $.import feature of XSJS to utilize this prebuilt library definition.

Managed/Unmanaged

However XSDS isn’t limited to only using CDS Entities. The goal for XSDS was to provide a single solution for all XSJS applications.  This means being able to use the same access library even if your base tables are directly created in the catalog and not via CDS. This way you can decide between Managed or Unmanaged modes in the library.

Managed mode allows you to import entity definitions from catalog objects and from CDS definitions (as described above). It is best used when you need consistency in access. It functions as a lightweight ORM building JavaScript objects from catalog or CDS definitions. It supports navigation via associations, data consistency and a limited query ability.

Unmanaged mode, on the other hand, functions as a Query Builder to provide you with maximum flexibility and access to nearly the full HANA SQL Syntax but via a simplified query interface. Data consistency between entities, however, remains the responsibility of the application developer.

Now for more details, lets look at some syntax examples of both Managed and Unmanaged Mode.

Managed Mode

Earlier we saw how to import a CDS entity definition.  You can also import catalog object definitions as well:

var Demo= XSDS.$defineEntity("Demo", '"DEMOSP9"."DEMO_TABLE"');

In this case we supply the Schema and Table name from the catalog instead of the path to the HDBDD artifact.

Regardless of the source of the entity, we perform the same core operations on the entity after its imported into a JavaScript object.

We can create, retieve, update, and delete instances of the entity.

var post = Post.$get({ pid: 101 });

The result is a plain JavaScript object that can be programming against like any other JavaScript object.

if (post.Author.Name === 'Alice') post.Rating++;

Optionally the objects support lazy retrieval of associated instances to avoid “n+1 problem”.  The library also optimizes Join strategy to minimize Database queries.

You can write changes into the database explicitly.

post.$save();

The transaction manager tracks local changes for minimal updates to the database.

As far as instance manipulation goes we can retrieve a single instance by key:

var post = Post.$get({ pid: 101 });

You can then update an instance like this:

post.Author.Name = "Alice";
post.$save();

Or you can create a new instance using this syntax:

var user = new User({ Name: "Alice", Created: new Date() });
user.$save();

Deletion works similarly:

post.$discard();

Instance Management

As the above examples show the entity instances are managed. This means that instances are singleton objects with “personality”. Associated instances are stored by reference.

The benefits of this approach are that the runtime guarantees consistency.  You have a single consistent view on every instance enforced by the $save operation. You also have automatic association management across all relationship cardinalities (1:1, 1:n, and m:n).

The downside, on the other hand, is that we need additional overhead for instance management.  We are also limited to a subset of database functionality for instance retrieval.

Transaction Handling

The managed mode also has impacts on the transaction handling.  We receive the benefits of transparent database and transaction handling. With a minimal XSDS runtime, the library is still able to provide metadata processing, entity cache, and type conversion during the transaction. It still exposes and reuses as much underlying database functionality as possible (for example: Constraints checks).

XSDS uses a single Database connection and single transaction.  This makes it well suited for single-threaded, per-request execution model of the XS processing layer.  It supports auto commit, explicit commits, and rollbacks. It also supports connection configuration files (XSSQLCC).

Unmanaged Mode

We also have unmanaged mode for when you want a more convenient database interface but don’t want to give up the full database syntax.  For example here is the typical database join statement in XSJS code.

var rs = conn.prepareStatement(‘
SELECT a."Name", c."Text.Text"
  FROM "bboard.post" p
  JOIN "bboard.user" a ON p."Author.uid"=a."uid"
  JOIN "bboard.comment" c ON p."pid"=c."Post.pid"
  JOIN "bboard.user" ca ON c."Author.uid"=ca."uid"
WHERE a."uid"=ca."uid“').execute();
while (rs.next()) {
  rs.getInt(1); rs.getString(2);
}

There are several problems here which XSDS unmanaged mode addresses.

Leaking Abstraction:

• Column name can only be derived from the underlying table definition

Convenience:

• String based query interface
• Manual result set processing

Redundant Code:

• Joins could be derived from underlying metadata instead of hand written

For these reasons the XSDS library also supports a query library for Ad-hoc queries.  It allows you to construct general queries in a very JavaScript-like way:

var query = Post.$query().$where(Post.Author.Name.$eq("Alice"));
var results = query.$execute();

The result object still yields a plain JavaScript object as unmanaged values not an instance as of a managed object.

The Query Language used here has several features:

• Incremental query building for better readability/maintainability
• Operators like $project, $where, and $matching
• Full support for CDS path navigation
• Expression language based on CDS Query Language using fluent API
• Alternative, more concise JSON-like selection notation for restricted subsets

Here are some additional code examples to help you learn some of these benefits.

$project

Projections with CDS navigation expressions:

var res = Post.$query().$project({ Author: {Name: true},          
                  Comments: {Author: {Name: true}},
                  Title: "TitleOfPost"}).$execute();

The results of this query would look something like this:

[{             Author: {
                                Name: "Alice"
                },
                Comments: {
                                Author: {
                                Name: "Bob"
                                }
                },
                TitleOfPost: "First Post!"
}, ...]

$where

The library supports complex where conditions like the following:

var selfCommentingPosts = Post.$query().$where(Post.Comments.Author.pid.$eq(Post.Author.pid))
.$execute();

There are predefined operators of the expression language:

• $eq
• $ne
• $gt
• $lt
• $ge
• $le
• $like
• $unlike

But you can still escape to any SQL operator:

var somePost = Post.$query() .$where(Post.Rating.$prefixOp('SQRT').$gt(2)).$execute();

$matching

$matching allows selection using a template for the result.

var interestingPosts = Post.$query().$matching({
    Rating: {
        $gt: 2
    },
    Tags: {
        Name: "+1"
    }
}).execute();

**Note the Unmanaged version of $find and $findAll uses the same template language.

Incremental Query Construction

Incremental construction with immutable query objects:

// build query without DB call
var qBadPosts = Post.$query().$where(Post.Author.pid)
  .$eq(Post.Comments.Author.pid)));
// refine query
var qStarBadPosts = qBadPosts.$where(Post.Rating.$gt(4));

Explicit trigger:

// trigger actual DB calls
var badTitles = qBadPosts.$project(
                { Title: true, Author: {Name : true}}).$execute();
var recentBadTitles = qStarBadPosts.$project({ Title: true })
  .$execute();

JSON Expressions

Simple expressions:

Post.$find({ mandt: "001", pid: 101 });

Complex expressions:

Post.$find({ Title: { $like: "First%" },
             Rating: { $gt: 1, $lt: 3 },
 Created: { $lt: Date.now() – 3600 } });

JSON Expression Language Operators:

• $eq
• $ne
• $lt
• $le
• $gt
• $ge
• $like
• $unlike
• $null

Optimized Query Construction

The query construction allows for incremental query construction without database roundtrips.  Immutable query objects support sharing of subqueries. This results in just-in-time translation into plain SQL using needed joins.