The Value of Metadata in Integration
We live in a world where companies rely more and more on being able to make data-driven decisions, use data and information as competitive advantage and leverage AI and automation to increase their efficiency. Therefore many adopted an information and data strategy, that understands and treats data as a valuable asset. But if we look a bit on the more technical side, raw data on its own is not sufficient to achieve those goals. To work with data in a scalable and robust way, it needs to be made accessible via APIs with clear contracts, using protocols that the software development ecosystem can work with easily, which is already the case today. However, to make the data and APIs comprehensible to humans and machines, they must be annotated with metadata. The better the quality of our APIs and metadata is, the more value and automation possibilities we can get out of our raw data.
So far, this has been mostly done with a focus on technical metadata, for example, specifying the data type of a value (String, date, etc.). While we’re continuing to invest in better technical metadata, we also see the need to bring more business semantics to our metadata.
Data also becomes much more valuable if it is connected and integrated well. This is usually a big challenge, as most companies have a very heterogenous system landscape with many data sources that are not integrated by default. Metadata can also help understanding such complex system and data landscapes and aid the integration efforts.
For instance, the term "Capability" is used in SAP SuccessFactors, while "Qualification" is used in SAP Fieldglass. Although these terms are different, they essentially refer to the same business concept: a skill, competency, or certification possessed by a workforce individual. By assigning metadata with a straightforward semantic name, such as "Workforce Capability," and providing a clear semantic definition, these two business objects can be identified as capable of integration. In other words, they can be consolidated into a common abstraction and subsequently mapped to each other.
Adding Business Semantics
Currently, API metadata primarily describes the data structure and API interface, providing the technical understanding for proper API and data usage. The understanding of business semantics largely relies on the quality of the human-readable documentation.
Therefore, we aim to establish business semantics as type names, similar to how schema.org annotates field meanings, as shown in the Person example. At SAP, we establish a unified business semantics vocabulary used across our models as machine-readable metadata.
Advantages of Introducing Business Semantics
This addition of business metadata in our APIs offers numerous benefits:
- Greater Data Understanding: Metadata with detailed business meanings enhances the understanding of data managed by an API, providing crucial context for users.
- Faster Data-to-Value: Explicit business semantics enable IT to meet business department requests much faster. Adding business semantics to the metadata of an API provides a common language between IT and business departments. It enables IT to understand the business context and quickly respond to business requests because they can easily interpret the data's meaning and relevance. This mutual understanding promotes efficient communication, reduces misinterpretations and speeds up the implementation of business requirements.
- Facilitating Intelligent Tooling: Business metadata facilitates intelligent tooling like low-code/no-code tools, aiding business users to independently access necessary data and build applications. Furthermore, exposing metadata with detailed business semantics makes complex configurations easier to manage via user-friendly integration tools.
- Advanced Analytics and Machine Learning: Metadata enriches data analytics and machine learning with added business context for robust analyses.
SAP’s Journey of Enriching APIs with Metadata
With a legacy spanning five decades, SAP has led technological change, providing innovative solutions to enhance global business efficiency and profitability. SAP's extensive experience and industry insight has been crucial in defining the future of enterprise software.
This extensive journey in the tech industry has allowed SAP to accumulate a wealth of business and technical knowledge, which will now be leveraged to enrich APIs with business metadata to promote a deeper comprehension of data, driving more insightful decision-making. This move is not just a technical advancement, but a strategic one, aimed at accelerating the process of deriving value from data, simplifying integration, and facilitating sophisticated analytics and machine learning applications.
SAP’s Metadata Strategy
At SAP, we are in the process of developing a well-defined strategy for handling metadata, particularly with respect to business semantics, which can be broadly summarized in the subsequent diagram.
In the customer landscape (that represents the actual SAP applications and services the customer has running), SAP Unified Customer Landscape (UCL) serves as a metadata aggregator. It collects business metadata from various sources including SAP applications, partner, and third-party applications, as well as central metadata repositories provided by SAP. This metadata is then made accessible to metadata consumers, such as SAP Datasphere, SAP Business Application Studio (BAS), SAP Master Data Orchestration (MDO), and other SAP or third-party solutions.
The SAP Business Accelerator Hub serves as a comprehensive catalog that compiles metadata from various sources including SAP, partner applications, and central SAP metadata repositories. It functions as a static resource of documentation for key users and developers. It receives information from an environment that includes both SAP and partner applications. We refer to that environment as “Reference Landscape”. Within this landscape, the SAP Business Accelerator Hub functions as a metadata aggregator, collecting metadata from SAP, partner applications, and central SAP metadata repositories.
Current Status: A Reality Check
Various SAP solutions, such as SAP S/4HANA, already use an aligned protocol called "Open Resource Discovery" (ORD) to expose their metadata. This protocol is increasingly being adopted by more SAP applications.
The SAP UCL service and the SAP Business Accelerator Hub can already aggregate metadata from SAP systems and the capability to gather metadata form third-party applications is currently under development. Tools like SAP BAS or SAP BTP Cockpit already use information from SAP UCL, and more SAP applications will do so shortly.
SAP's central metadata repositories are increasingly utilized for dependable and consistent business metadata across major business lines, with entries managed via established governance processes.
We're currently developing new pages and connections between new and existing content for the SAP Business Accelerator Hub. This makes it easier for key users and developers to understand the relationships between business objects and available APIs, events, or data products.
The Integral Role of SAP One Domain Model (ODM) in the Journey SAP One Domain model (ODM) underwent a revision with a fresh perspective. Instead of only focusing on new master data models for Master Data Integration Service (MDI), ODM is now part of the bigger SAP metadata strategy.
The focus is now on enhancing and connecting existing data models for integration.
The new ODM standardizes how SAP expresses business semantics and relationships between business objects in APIs. We are starting with two principles that APIs must follow at the entity or main business object level to become ODM compliant.
Principle 1: Link data models and APIs to ODM Business Objects
An API / data model can be linked to an "ODM Business Object" that has a universally agreed-upon semantic definition within SAP. This definition ensures a unified and reliable understanding of business object semantics across various application models, including APIs, events, and data products. It aids in identifying similar semantic elements across different SAP applications.
Principle 2: Connect instances via unique ODM object identifiers.
An attribute is added, the so-called OID or ODM ID that defines the globally unique value for this object type across the customer landscape. The goal is to eliminate the need of point-to-point ID mappings between applications and to allow easy reconciliation of information for cross application data analytics.
In the above example, consumer C has an integration with two providers A and B. Both share cost centers with the consumer. Both have the same local cost center id 1803 but belong to different company codes. Consumer C needs no key mapping nor evaluation of company codes because providers A and B are using OIDs which are guaranteed to be unique in the landscape.
ODM is Incremental and Additive
Applications apply the ODM standards to existing data models of APIs, SAP MDI or events to improve the integration for our customers. For example, when an API 'CostCenter' is enhanced by adding an ODM.entityName annotation and an OID field as unique ID, it is not a truly new API but just a new version of the existing API. All existing integrations that use this API will still work but new use cases can make use of the new ODM elements.
Adding an ODM entity name annotation and an OID makes an API 'ODM compliant'. This is at first only needed for entities that are exchanged between applications. Since ODM entity names are used as type names on object OIDs and OID references, these OIDs link objects between applications and allow much easier cross-application analytics.
So far, we first focus on the linking (type and instance level) of entities between applications. In the future we will also do this on the field level.
So, in summary, ODM connects existing data models and objects (instances) between different applications by making small enhancements to existing models. This incremental approach applies the same way to master data (SAP MDI models), APIs, events etc.
Consequently, ODM itself now places much more emphasis on developing concepts, methods, and tools to connect existing models as a general capability that we can apply to the SAP’s Intelligent Enterprise scope now and any integration in the future. Even customers and partners will eventually be able to use the same methods and tools.
Outlook
SAP is progressively incorporating more standardization into API-, event-, and data product development. Plans include adding attribute-level business semantics to optimize customer integration and developer experience. We are at an early stage of this journey, but tangible results will be coming soon. We are excited about its potential to enrich our APIs and provide a smarter, more efficient developer- and user experience.
We'll keep you updated on our progress and insights through our blog. Stay tuned; we're just embarking on this exciting journey.
- SAP Managed Tags:
2 Comments
