Intelligent Integration Landscape Operating Model in 2021 – Part 1
It has been an intense year for all of us at work and home adjusting to the new normal and many of us are still on furlough!
Covid has left a profound impact and long standing debt on economies but humans have never made more progress on any disease in a year than the world did on COVID-19 this year. Under normal circumstances, creating a vaccine can take 10 years. This time, multiple vaccines were created in less than one year. According to Bill Gates, These breakthroughs will make world better in 2021 than 2020.
The same has been true with my team as well, we never made great progress on new certifications and building useful tools outside projects as we had always been on a tight spot on the delivery of cutting edge SAP Cloud projects within very aggressive time scales.
I want to congratulate my team for not only completing TOGAF, Open SAP, Mule soft, AWS certifications but also building a prototype that will help organizations to shift to a Intelligent Integration Operating Model and say cheers to Ramesh Parashivamurthy Umasairam Reddibathina Ram B for building the prototype that we are hoping to release next year.
As humans, we are continuously striving to be better every year and it is the same for the “Integration World”!
I want to share my thoughts on how we can build intelligent integration operating model in 2021 and make life easier for integration support teams.
Before that, I want to wish you all HAPPY NEW YEAR with a small poem that captured my heart!
According to Gartner, The average business transaction now crosses 40+ different back-end systems, connecting new information and operationalizing it across the entire enterprise via API(S) can drive automation and process efficiency.
The right integration strategy allows organizations to cut complexity and drive tangible business value however IT teams often struggle to rally their organizations behind a new digital integration approach. A key reason for this is that we are not communicating the business value story of integration projects in a clear and compelling way that connects with the customer!
BIG ELEPHANT IN THE ROOM – DIGITAL TRANSFORMATON – INTEGRATION STRATEGY – IT PROJECT BUDGETS
TOP 7 CUSTOMER CHALLENGES
The explosion of 5G, container based deployment of apps, server less computing and IoT, Bigdata, Edge Computing has compounded the integration challenges for the customer as the integration now is not about connecting “apps” in a pre-defined network, “Integration is about connecting “anything” to “any other thing” deployed “anywhere” (space or earth or satellite or submarine) to seamlessly transfer ” any packet (messages, big data, streaming events etc)” to “any destination” without “any pre-defined boundaries”.
Whilst collaborating with customers to identify their pain point and shaping their data and integration strategy on ERP digital transformation projects , I have seen a consistent pattern of 7 challenges faced by customers summarized below:
1.Astronomical project costs to Integrate multiple cloud tools with complex on-premise legacy systems due to poor documentation of existing interfaces and lack of bird’s view of interfaces across landscape.
Ex: It takes us 3-5 months to discover the existing interface flows functionality and requirements compared to the time it takes to build , test and deploy these interface flows in a month.
2.Unable to reap Interface reusability(API) ROI: Organizations have multiple cloud and digital projects kicked off with no central governance resulting in projects building redundant interfaces
Ex: There is no central repository of interfaces where projects can check if they can re-use existing interfaces to accelerate the project interface requirements based on interface meta data parameters like fields, systems , protocols etc before commencing the build. I had never been on a project where client clearly articulated the data flows of existing integration landscape or provided me a list of As-Is interfaces with clearly defined functional and non-functional requirements.
3.No tools to assess process gaps in existing integration landscape to identify bottlenecks and modernize those interfaces that drive business value.
Ex: There is no central visibility on which interfaces and processes need to be re-designed to improve the operational efficiency and hence customers generally opt for lift and shift instead of innovation first approach during digital transformation. I would generally ask customers to give me a list of incidents that occurred in last 3 years but again customers find it very difficult to link the incidents to the actual interfaces. Ideally, a customer sentimental analysis of incidents linked to process and interfaces will help to identify the interfaces that needs to be re-designed to provide business value but no customer has ever been able to articulate it clearly as they don’t have tools.
4.Lack of Ability to estimate the rom costs of the complexity and costs of upgrading middleware due to poor documentation and understanding of existing interface flows…Do you want to try convincing business to upgrade 200 interfaces from SAP PO to CPI for million dollars to unlock innovation value?
Ex: Customers are often scared to upgrade their interfaces on existing platforms to digital innovation cloud platforms due to the costs i.e. they don’t have ability to identify interfaces which will provide business value when we upgrade them to digital innovation cloud platforms unless they do detailed discovery with SI partners which again is a significant cost to them.
5.No automatic Integration assistant to select the right tool for multi-cloud integration
Ex: SAP ISA-M provides guidelines on what SAP tools we need to choose based on integration scenario use case but this becomes very complex when we have applications deployed on multiple cloud platforms as each vendor will be looking to sell their integration platform for connecting to their applications. It may be easier if we can deploy IFLOWS as containers, but we can’t yet on many integration platforms. Please refer my blog where I discuss why integration strategy or roadmap is very critical before you start your digital journey.
6.No Real time incident assignment & reporting when an error occurs based on interface criticality
Ex: Ideally every customer and business unit within an organization may use different ticketing tools and also may have different ideas on what channels they want to use for sending alerts and how they want to group the interface alert reports. Ex: Customer A may want JIRA and Customer B may want Service Now. Customer A want to group alerts based on business criticality and country and receive one report at end of the day but customer B may want real time alerts only after message failed for 5 times.
7.Lack of Ability to analysing recurring integration error patterns across organizations , auto-learn from the errors and resolve issues
Ex: Customers don’t have ability to answer these questions without doing detailed investigation ..”What are the problem areas of my existing processes? What types of errors are impacting my revenues? What types of error patterns are occurring in each process ?What are my opportunities to auto fix or auto suggest the actions to resolve the recurring error patterns especially for the business critical interfaces?
The objective of the storyboard is to solve the above 7 challenges and build a Holistic Monitoring and Error Learning Framework that can be tailored to customer requirements for reporting errors into incident management systems using open connectors, recording interface meta data for all interfaces to promote re-usability , assess CR impacts and contact right teams to resolve the errors quickly, generating business friendly L1,L2,L3 operational dashboards, restarting messages.
The tool should provide following benefits to customer:
- Provides bird’s view of the customer integration landscape
- Provides ability for support teams to quickly assess the interface CR’s and implement the CR’s
- Provides support team ability to mitigate the risks of Interface Upgrades by providing an up to date bird’s view of all the interfaces in the customer landscape
- Provides ability for support teams to contact right teams and resolve errors quickly
- It provides metrics like Interfaces by Country, Interfaces by System, Interfaces by Business Criticality, Interfaces by Volume that helps organizations to reduce TCO and quantify which interfaces can bring business value when modernized in future
- Customizable Auto-restart and manual restart of messages
- Ability to create incidents on incident management systems or pro-active alert e-mails according to the customer’s choice of channel based on business criticality
- Provides ability to generate operational and performance dashboards that enables L1, L2, L3 Support Teams to make faster decisions for service improvements
- Provide ability to slice and dice the textual error logs using keyword tagging or text mining techniques to quickly identify the error root causes
- Provides ability to Enhance Monitoring Framework to suite Custom requirement ..Ex: No of auto-tries, raise incidents on Zendesk instead of JIRA or E-MAIL Alerts
Support User Journeys
The below table provides a snap shot of the Integration Support Team User Journeys that many customers are looking for improving in the future.
|Support User Journeys||Support User Type|
|As a Global Managed Integration Service support analyst, I want to monitor the interfaces and perform error trend analysis using a single graphical dashboard for CPI and SAP PO landscape and SAP API management and other on-premise and cloud integration platforms across the customer landscapes.||L1 User|
As a Global Managed Integration Service support analyst, I want to govern the interfaces across the landscape , approve and assess the impact and costs of the new interface CR’s quickly.
Additionally, I should be able to verify the CR’s against reusable interface metadata repository to assess if I can accelerate the CR delivery by re-using existing Interfaces in a different country or organization or business unit.
|As a Global Managed Integration Service support analyst, I want project teams to have the ability to record the meta data of the (new/extending existing) interfaces in a global repository before commencing the build and before handing over the interfaces to the BAU Teams.||L1 User|
|As a Global Managed Integration Service Support analyst, I want to be able to quickly refer the documentation of the interfaces and known issues and contact right people to resolve the errors quickly.||L1 User|
|As a Global Managed Integration Service Manager, I want to be able to view the historic interface error trends for all interfaces in the landscape and identify opportunities to improve the integration service levels.||L1 Service Manager|
|As a Global Managed Integration Service Manager, I want to be able to view the historic, current system performance of the interfaces by country, by process and by system to identify opportunities for reducing delays in business processes.||L1 Service Manager|
|As a Global Managed Integration Service Manager, I should be able to accurately estimate the costs of the IT upgrades of the Integration Service Landscape and identify the business critical interfaces for modernization that will bring business value across geographies.||L1 Service Manager|
|As a Global Managed Integration Service Manager, I want to view the historic trends of P1/P2 Incident trends and error patterns across the landscape to measure the integration service efficiency and identify improvement opportunities to ensure that P1/P2 incidents doesn’t occur in the future.||L1 Service Manager|
|As a Global Managed Integration Service Manager, I want to measure the overall external and internal organization sentiment of the Integration Service Performance by measuring the feedback on tickets.||L1 Service Manager|
|As a Global Managed Integration Service Manager, I want to see all interfaces that changed for a given time period.||L1 Service Manager|
|As a Global Managed Integration Service Manager, I want to do have a set of regression tests that I can run for any BAU changes or upgrades without re-inventing the wheel.||L1 Service Manager|
|As a Local AMS support analyst for my country and specific systems, I want to only monitor interface messages that are relevant to my country and receive alerts for business critical interfaces of my country and specific systems.||L2 User|
|As a Local AMS support analyst for my country and specific systems, I want to choose the service ticketing tools that are widely used by my country for raising interface errors based on the interface criticality.||L2 User|
|As a Local AMS support analyst for my country and specific systems, I should be able to restart the failed messages manually and have the ability to dynamically configure how many times an interface should auto-try during system outages/maintenance periods.||L3 User|
|As a Local AMS Technical Analyst, I should be able to identify the errors that occurred in each system previously in a given time period and identify the resolution steps taken to resolve the issues quickly||L3 User|
|As a Local AMS Technical Analyst, I should be able to see top 10 or 20 errors in a given time period for selected system||L3 User|
|As a Local AMS Technical Analyst, I should be able to see top 10 high volume interfaces in the landscape||L3 User|
|As a Local AMS Technical Analyst, I should be able to slice and dice the massive system error logs using keyword tagging or text mining techniques to quickly identify the error root causes||L3 User|
|As a Local AMS Technical Analyst, I want to deploy and un deploy communication channels/integration flows when source or target systems are going through maintenance or outages.||L3 User|
|As a Local AMS Technical Analyst, I should be able to see all technical interface objects that changed for a given time period and if possible I should be able to see the differences or code changes between versions.||L3 User|
Please check Part 2 of this blog for the solution and MVP architecture of the above idea board.