Leveraging SAP HANA metadata to make it a “Trusted” Analytics platform – Part 2 of 2

In Part 1, we listed the key use cases that supports and increases the trust on HANA as an analytical platform leveraging HANA’s metadata. I list them here again for our reference:

1. Ability to draw the end-to-end lineage or trace the provenance of a data element from its system of use to its system of record or creation – This could be true for a platform consuming HANA (eg: BI/analytical tools) or for the platforms that are downstream consumed by HANA (eg: relational databases)
2. Ability to trigger governance processes and notify relevant stakeholders of change to a data element in the HANA platform
3. Push back to the HANA platform indicators that its data element is more traceable and trustable, as an outcome of the governance processes applied on them

We also looked at the first use case of drawing up the end-to-end lineage from the upstream systems/platforms to HANA or from HANA to its downstream systems and the benefits in brings.

Let us review the remaining two use cases as well.

One of the factor that improves ‘Trust’ on a data platform like HANA for the data consumers, is its ability audit changes to the underlying data structures. Business requirements being dynamic and hence the data needs, changes to the data structures are inevitable. Leaving alone the flexibility technology offers to implement these in a NoSQL database vis a vis SQL/relational database, auditing changes is key (a) to keep the data consumer informed of the changes (b) ensure security is not compromised (c) address regulatory needs of having an audit log of changes or (d) could be to ensure certain enterprise standards are adhered to.

Ability to capture changes on the HANA platform and triggering relevant governance processes on change then becomes essential. This is the second use case where HANA metadata can be leveraged to compare the version in the HANA platform against the version in the Data Governance(DG) platform and on change, trigger workflows that support notification to stakeholders, validation etc

It is definitely worthwhile to govern the HANA platforms from within the DG platform, looking at the lineage, doing impact analysis or triggering workflows. This is best suited for the DG teams or stakeholder with access to the DG platform. However, from an end user (of the HANA platform) perspective, it makes more sense to leverage the outcome of these DG processes right there in the HANA platform (without really logging into the DG platform).

This is where the third use case comes in. Ability to write-back / feed the outcome of the governance processes into the source HANA platform. For example, what if we are able to add a comment to a Catalog table or Catalog column based on the governed glossary definition for that object? Or use these comments to tag how ‘trustable’ the object is based on the assessment done within the DG platform say ‘Certified’, ‘Certified – Bronze level’ etc. I am not sure if HANA provides a means to update comments for the Calculation views yet but if this is available, tagging these analytical objects similarly will be huge boost to its reliability by the data consumers.

One bonus use case that also gets handled is the cataloging of the HANA objects! Data catalogs have become an essential component of a modern enterprise’s data architecture.  It is nothing but a catalog of an enterprise’s data sets, making it searchable and understand its contents and additional metadata, if available (typically likened to a product catalog of e-retailers like Amazon)

My experience has been to try out these use cases in Collibra, one of the leading DG platform. First step was to integrate the HANA platform with Collibra and load metadata of HANA objects (Catalog tables/views/columns/stored procs/functions as well as those of the calculation/analytic/attribute views, including their dependencies).

Then the load was setup for periodic refresh, which is where changes to the HANA platform could be monitored. Governance processes then were setup as Activiti-based workflows in the Collibra platform.

This surely is a big step towards governing the HANA platform and thus increasing its ‘Trust’ factor.