Together with SAP Datasphere, SAP launched new partnerships to complete the picture of analytical data management and the idea of a Business Data Fabric.
Fig1: SAP Datasphere & Partnerships – Source: SAP, 2023 (slightly adapted)
I already gave an impression of the new partners in my blog “SAP Datasphere – Q&A and Partnerships“.
For the other Partnerships look also here:
As I want to focus here on Collibra I repeat what I have written:
With SAP Datasphere, SAP launches also a new Catalog functionality described in this blog "Unlock the full potential of your enterprise data with SAP Datasphere Catalog" by 537671, SAP.
Rise of Data Catalogs
The last years, I recognize a increasing interest in Data Catalogs due to different reasons. Here you can see typical motivations and challenges I see at organisations starting to or already implementing a Data Catalog:
Fig2: Motivation and challenges of Data Catalog initiatives
In general, the term Data Catalog have a clear focus for me but we also see a broad range of solutions in the market. We have very technical approaches typically being part of a larger offering like with Databricks Unity Catalog or AWS Glue, Data Catalog. Typically we do not find these Data Catalogs in market research as from Forrester or Gartner as the use case is limited and they are to specific to their context.
Market research is giving us a broad view of the market which results in a clear leadership team in the last years:
These catalogs enhance the technical metadata management with more business-oriented perspectives and functionality, supporting a strong data governance, data ownership and data literacy.
There are further offerings, which can be seen as very strong depending on the use cases, like IBM, Atlan, Precisely, erwin by Quest and others. What is typically not considered by market research but an interesting development are open source solutions like Amundsen, Datahub, Apache Atlas and others. In a 2020 study, Fraunhofer ISST recognized 60 solutions in the market, 70% US-based.
For the Collibra partnership there is the following announcement:
This is an interesting approach. Let's analyse it from two sides:
SAP Datasphere Catalog - starts now with mainly integrating metadata of SAP solutions like SAP Datasphere itself, SAP Analytics Cloud and SAP S/4HANA Cloud. I think in the future we will see a similar integration range as SAP Data Intellingence, Data Catalog has today, as it is announced that more and more SAP DI capabilities are integrated into SAP Datasphere. Collibra is used in many environments where SAP could be or could not be a source. From the strategic viewpoint it would be worth to say, if both solutions are in place, this partnership could make it easier to use the metadata in the specific context. So this exchange makes sense and enables possibly to find new ways to integrate and reuse metadata which is already collected, not necessarily harvesting metadata directly from the source several times. So we would have synergie effects and added value for SAP here.
Collibra Data Intelligence Cloud - For me Collibra is a Data Catalog solution with a lot of capabilities to adapt to user needs, different roles and use cases. It is a strong, business-oriented solution. As many Data Catalog solutions are challenged to connect to the application layer of SAP or other ERP-like software, they use, as many others, software by Sillwood Technology (Safyr). Possibly this partnership gives Collibra-users the change to easier and faster access of SAP metadata and enabling a better usage in this area, including SAP systems in a the corresponding processes and use cases.
At the end, we will see a combinded approach, which needs at least two Data Catalog solutions. As we see more and more distributed and heterogenous environments, this is possibly the way to go.
Collibra is a strong platform for data governance, data observability and data quality today. Again we have a partnership with one of the leaders in the market. It would be interesting to see if these cooperations grow into other vendors as the market is very much divided here and possibly partnership with vendors like Alation and Informatica could be the next step.
From what I know, it is not strategy for SAP Datasphere Catalog to represent a standalone enterprise Data Catalog solution. For a data fabric approach it makes sense to have these capabilities integrated to enable overview of all the systems managed by the data fabric in the future and enable more metadata-driven functionalities. But the vendors like Collibra are strong in this market and focussed on the Data Catalog use case, and I assume it is better to cooperate than to compete in this market.
Again, as for the other partnerships, I reccommend to be aware of building a strong data strategy first, as implementing a Data Catalog solution like Collibra could easily cost a lot of money and is strongly connected to a good data governance approach. Done right it can be a very helpful building block on your way to a data-driven company. Done the wrong way, you will just have another piece of expensive software in your company and wonder why data-driven not just happens.
This is just my opinion and current perspective. I’m happy to hear from you how you see these new partnerships in the context of SAP Datasphere?
Fig1: SAP Datasphere & Partnerships – Source: SAP, 2023 (slightly adapted)
I already gave an impression of the new partners in my blog “SAP Datasphere – Q&A and Partnerships“.
For the other Partnerships look also here:
- SAP Datasphere & Partnerships – Confluent
- SAP Datasphere & Partnerships – Databricks
- SAP Datasphere and Partnerships – DataRobot by fazam
- SAP and Google Cloud Expand Partnership to Build the Future of Open Data and AI for Enterprises (SAP News)
As I want to focus here on Collibra I repeat what I have written:
Collibra
If you look at market research like from Forrester or BARC for Data Catalog, Data Intelligence or Metadatamanagement, Collibra is typically on of the top three solutions in this area (typically together with Alation and Informatica or IBM).
SAP have some history with metadata management and already delivered these e. g. with SAP Information Steward, SAP Power Designer and also in other data and analytics solutions. SAP offers Data Catalog functionality within SAP Data Intelligence and build up more and more capabilities within SAP Data Warehouse Cloud.
So in a world, where data assets are distributed all over our company and an overview and understanding of our data is getting more and more important, a data catalog is clearly recommended and will become a cornerstone of the data culture within data driven companies.
Collibra lately expand its platform with data quality management and data observability capabilities together with an partner ecosystem.
With SAP Datasphere, SAP launches also a new Catalog functionality described in this blog "Unlock the full potential of your enterprise data with SAP Datasphere Catalog" by 537671, SAP.
Rise of Data Catalogs
The last years, I recognize a increasing interest in Data Catalogs due to different reasons. Here you can see typical motivations and challenges I see at organisations starting to or already implementing a Data Catalog:
Fig2: Motivation and challenges of Data Catalog initiatives
In general, the term Data Catalog have a clear focus for me but we also see a broad range of solutions in the market. We have very technical approaches typically being part of a larger offering like with Databricks Unity Catalog or AWS Glue, Data Catalog. Typically we do not find these Data Catalogs in market research as from Forrester or Gartner as the use case is limited and they are to specific to their context.
Market research is giving us a broad view of the market which results in a clear leadership team in the last years:
- Collibra Data Intelligence (vendor link)
- Alation Enterprise Data Catalog (vendor link)
- Informatica Cloud Data Catalog (vendor link)
These catalogs enhance the technical metadata management with more business-oriented perspectives and functionality, supporting a strong data governance, data ownership and data literacy.
There are further offerings, which can be seen as very strong depending on the use cases, like IBM, Atlan, Precisely, erwin by Quest and others. What is typically not considered by market research but an interesting development are open source solutions like Amundsen, Datahub, Apache Atlas and others. In a 2020 study, Fraunhofer ISST recognized 60 solutions in the market, 70% US-based.
What Collibra can do for SAP
For the Collibra partnership there is the following announcement:
"We are partnering with Collibra for bi-directional metadata exchange so that our customers can run end–to-end data governance with SAP and non-SAP data using SAP Datasphere Catalog and Collibra side by side."
Source
This is an interesting approach. Let's analyse it from two sides:
SAP Datasphere Catalog - starts now with mainly integrating metadata of SAP solutions like SAP Datasphere itself, SAP Analytics Cloud and SAP S/4HANA Cloud. I think in the future we will see a similar integration range as SAP Data Intellingence, Data Catalog has today, as it is announced that more and more SAP DI capabilities are integrated into SAP Datasphere. Collibra is used in many environments where SAP could be or could not be a source. From the strategic viewpoint it would be worth to say, if both solutions are in place, this partnership could make it easier to use the metadata in the specific context. So this exchange makes sense and enables possibly to find new ways to integrate and reuse metadata which is already collected, not necessarily harvesting metadata directly from the source several times. So we would have synergie effects and added value for SAP here.
Collibra Data Intelligence Cloud - For me Collibra is a Data Catalog solution with a lot of capabilities to adapt to user needs, different roles and use cases. It is a strong, business-oriented solution. As many Data Catalog solutions are challenged to connect to the application layer of SAP or other ERP-like software, they use, as many others, software by Sillwood Technology (Safyr). Possibly this partnership gives Collibra-users the change to easier and faster access of SAP metadata and enabling a better usage in this area, including SAP systems in a the corresponding processes and use cases.
Conclusion
At the end, we will see a combinded approach, which needs at least two Data Catalog solutions. As we see more and more distributed and heterogenous environments, this is possibly the way to go.
Collibra is a strong platform for data governance, data observability and data quality today. Again we have a partnership with one of the leaders in the market. It would be interesting to see if these cooperations grow into other vendors as the market is very much divided here and possibly partnership with vendors like Alation and Informatica could be the next step.
From what I know, it is not strategy for SAP Datasphere Catalog to represent a standalone enterprise Data Catalog solution. For a data fabric approach it makes sense to have these capabilities integrated to enable overview of all the systems managed by the data fabric in the future and enable more metadata-driven functionalities. But the vendors like Collibra are strong in this market and focussed on the Data Catalog use case, and I assume it is better to cooperate than to compete in this market.
Again, as for the other partnerships, I reccommend to be aware of building a strong data strategy first, as implementing a Data Catalog solution like Collibra could easily cost a lot of money and is strongly connected to a good data governance approach. Done right it can be a very helpful building block on your way to a data-driven company. Done the wrong way, you will just have another piece of expensive software in your company and wonder why data-driven not just happens.
This is just my opinion and current perspective. I’m happy to hear from you how you see these new partnerships in the context of SAP Datasphere?
- SAP Managed Tags:
