Metadata Management in a SAP Landscape
In this blog, we will explore how Metapedia and the native Metadata Integrators bundled with Information Steward can support you in managing your SAP landscape’s metadata.
In general, Information Steward Metadata Management collects metadata information from your enterprise systems, information such as:
- Attributes (name, size and data type)
- Structures (length, fields and columns)
- Properties (where it is located, how it is associated, and who owns it)
- Descriptive information (about the context, quality and condition, or characters of the data)
Information Steward then organizes metadata objects to allow you to:
- Browse, search and explore the metadata
- Understand relationships between different objects within the same source and across different sources
- Customize metadata objects and relationships with annotations and custom attributes and relationships
SAP BusinessObjects Enterprise and Some Basics on Data Impact/Lineage and Metapedia
In support of your SAP BusinessObjects Enterprise environment, the Metadata Management module of SAP Information Steward can discover metadata about universes, reports (including Crystal Reports, Web and Desktop Intelligence documents), dashboards, CMS folders and systems, server instances, application users and user groups and SAP BW systems. And, for each object, there is additional associated metadata. For example, for a universe, the associated metadata may include queries, universe classes and connections, objects and filters. For reports, metadata could includes universe objects, InfoObjects, queries, report fields, columns, variables, SQL expression fields, formula fields and running total fields.
Impact analysis will allow you to view the objects that are affect by data within a particular object. For example, the rather simple impact diagram above shows that the Calculate Totals report field impacts the Charts and Dials report. When you hover your mouse over an element in the impact diagram – in this case the Calculate Totals field – additional information about that metadata object appears.
The impact analysis for a universe object lists the objects that are affected by the data within the universe object. In this example, you can see that two reports are affected by the data in the “Revenue Amt Func” universe object, InvoiceSummary and Revenue. You can also take note of the report consumers, both users and user groups, that have permissions to access these particular reports. And, why is this important? Simple. It answers the question, what is the downstream impact if I change this universe or that query? And, this includes not only what does it impact, but also who and how many?
Data Lineage enables you to view a list of sources from which an object obtains its data. In this example, the report lineage diagram shows that the universe objects in the BOE Business Intelligence system came from tables that were loaded by the DS_Repo_RapidMarts Data Service data integration job. The dashed lines between each column name in BOE Business Intelligence and DS_Repo_RapidMarts systems indicates that the columns are the same (Same As relationship). And, you could explore the data lineage further to see the source database or business warehouse of the Data Services data flow.
Adding to the potential for insight, SAP Information Steward Metadata Management information can be accessed directly from within BI Launch Pad to view the data lineage of a Crystal report or Web Intelligence document, enabling direct access for report developers and consumers to understand where the data is coming from and how that data is being transformed.
And, not only can you help report developers and consumers to understand where the data is coming from, you can additionally instill a degree of trust in that data by allowing them to see how good the data really is. The lineage information provided via the BOE and Information Steward integration includes and highlights the quality scores of the specific data assets and allows the user to drill into those scores to see the details as to the data quality rules, failed data as well as profiling results, if these rules and results are available.
Metapedia terms can also be associated with report metadata objects.
And again, a link within the BI Launch Pad allow BI users to access the business terms that have been associated with a particular Crystal Report or Web Intelligence document directly from the BI Launch Pad. This promotes a common understanding of business concepts and terminology through Metapedia as your central location for defining standard business vocabulary (words, phrases, or business concepts). So, why might you want to start a Metapedia initiative? Well, think of your report/dashboard consumers, especially if those consumers areacross multiple lines of business. For example, let’s say Human Resources has created a new report that displays information about Org. Units, Dept. Units, Functional Area, Career Grades and Levels. Wouldn’t it be great if consumers outside of the HR team could gain access to the accepted and “HR-approved” understanding of each of the concepts represented on the report? Or, looking outside of the reporting world, think about a data migration project to bring two companies’ data assets together. As you sit down together and map common data elements, what a great asset to be able to capture the common understanding of the data – data that may be technically named differently – in business terminology and link that definition back to the disparate sources.
With the data migration example, so what if you want to expose your central repository of business terms to additional applications or locations to promote a common understanding across the two newly joined companies? Good news! Metapedia content can also be accessed via WebServices, which includes APIs that support searching the Metapedia repository terms, descriptions, authors, synonyms, categories, etc. Above is an example MS-Word Plugin created using the Metapedia WebService API (see the Information Steward Developer’s Guide for more information).
SAP NetWeaver Business Warehouse (SAP NW BW)
Okay, so we are going to dive back down into the metadata with a look at SAP NetWeaver BusinessWarehouse (SAP NW BW). Besides relational databases and data warehouses, BOE universes and reports can also assess data from SAP NW BW. However, the SAP NW BW is a “blackbox” for BOE BI users. It is not possible for them to see how the data crosses between the BOE BI and SAP NW BW environments. The Information Steward SAP NW BW metadata integrator removes the barrier between these two environments (BI to BW) by exposing the objects inside SAP NW BW environment and thus providing transparency and traceability needed. This allows questions such as, “If I change the definition of a specific SAP NW BW object, what universes or reports are affected?” or “From what SAP NetWeaver BW source does the universe obtain its data?” to be answered. The SAP NW BW metadata objects and relationships supported by the Information Steward SAP NetWeaver Business Warehouse metadata integrator are displayed in the light-blue boxes in the above diagram.
The Information Steward HANA metadata integrator has the ability to collect standard relational objects and information models in HANA. It collects all of the information about service instances, databases, packages, views, including Attribute Views, Analytic Views, and Calculation Views, tables, columns, measures, variables, etc. It also collects relationships between schemas, tables and columns as well as attributes and measures in your SAP HANA database sources. And, of course, all of the relationships upstream and downstream of your HANA instance.
SAP Data Services
SAP Data Services objects show up under the Data Integration category of Metadata Management. Metadata objects applicable for Data Services includes projects, jobs, work flows, data flows, datastores (source and target information), custom functions, table and column instances, etc.
With Data Services data lineage analysis, we can see how data from the source has been extracted, transformed and loaded into the target
In the example above, you can see the ETL process in action, following the data from report to source. Specifically, you can drill into to determine how LOS, or length of stay, was calculated and what source fields ultimately make up the PATIENT_NAME.
While analyzing the lineage of a Data Services integrator source column, you can view the Data Services Auto Documentation report for the associated data flow objects. The Auto Documentation report allows you to see another view of the ETL process.
If you click the data flow hyperlink, it will launch the Data Services Management Console and allow you to navigate to the dataflow details.
The Information Steward SAP PowerDesigner integrator is new with the Information Steward 4.2 release. With this out-of-the-box capability, users have access to all the metadata related to Power Designer, thus improving collaboration between Data Modelers and Data Stewards by extending impact and lineage analysis to the design models that are available in PowerDesigner. Once you have aligned the current-state (the operational view) with the architectural view, Data Stewards can then “inform” the Data Modelers where the root of quality concerns come from, informing architects so that they can address these quality concerns at the source as they design the next generation business applications. Data Stewards also have easy access to data quality rules and domains defined as part of these PowerDesigner models, which they can leverage to implement actual validation rules within Information Steward. In addition, the business terms defined in PowerDesigner can be integrated with Information Steward’s Metapedia so that all the business concepts are captured in a central location.
PowerDesigner metadata is collected for conceptual, logical and physical data models. In the image above, the left side shows the view in the PowerDesigner client and the right side shows corresponding objects in Information Steward. Note that the intent is not to replicate everything possible object from PowerDesigner to Information Steward. Only basic properties of conceptual and logical models are captured along with the relationships between the conceptual, logical and physical models. The connecting entity between PowerDesigner design time metadata and Information Steward operational metadata is the physical model, so that is where the focus is. Details about basic properties, physical diagrams, business rules, domains, references, tables, and server Instances are collected for PowerDesigner.
The above is an example of how the impact/lineage diagram shows up. In this example, the database was created using a script generated by PowerDesigner itself. On the BOE side, there was a universe built on top of that database, which is being used by the report. On the PowerDesigner side, there were domains and business rules that were associated with a few columns being used by the report fields. Hence, the lineage is shown as report > report fields > universe objects > PowerDesigner columns > domain/rules.
The Business Glossary in PowerDesigner is very similar to Information Steward Metapedia, so it is very easy to map concepts from one to another. You can import the content of PowerDesigner’s Business Glossary to Metapedia. If the glossary terms were associated with some other objects in PowerDesigner, that association is maintained in Metapedia as well.
So, what about the SAP Business Suite? Well, there is more work to be done here specific to Information Steward‘s Metadata Management capabilities. Currently, Information Steward gives you native connectivity to SAP ECC within Data Insight for data profiling and data validation. This gives you access to browse SAP ECC metadata down to the column level, similar capabilities exist within Data Services. This will also allow you to relate your Metapedia business terms to SAP ECC metadata (we covered this capability, object association, earlier). However, in terms of Metadata Management and the ability to discover objects and relationships all the way to the SAP Business Suite, this item is currently on our Information Steward roadmap (SMP Roadmaps, go to Database & Technology area). The goal is to provide complete metadata management for your SAP landscape, from data definition (via PowerDesigner) all the way to your operational systems and business processes (SAP ECC). Watch for more great capabilities to come with Information Steward’s Metadata Management!