When Customers Raise the Alarm, the Mission Control Center Stays Calm
Translation of the article published January 09th 2019 in Neue Zürcher Zeitung into English prepared by SAP with permission of Neue Zürcher Zeitung (author Michael Rasch, original in German)
Europe’s largest software company, SAP, has more than 400,000 customers across the globe. So problems are bound to occur. When things get critical, customers are referred to the Mission Control Center and placed in the care of SAP’s “emergency doctors”.
“Houston, we have a problem.” This legendary communication, made during the failed lunar landing mission Apollo 13, is a good analogy for the calls for help heard several times a day in St. Leon-Rot. This quiet community – just a couple of miles from company headquarters in Walldorf in the southern German state of Baden-Württemberg – is home to software company SAP’s European Mission Control Center (MCC). When customers raise the alarm in the MCC, they’re not in life-and-death situations, despite the fact that the MCC is referred to within the company as the “emergency room”. But they are, in most cases, reporting production outages that could quickly cost them millions of euros. When SAP customers are referred to the Mission Control Center, it’s because they have declared their status as “business down”. There’s not a second to lose for Constanze Jansen, Head of SAP MCC for EMEA, and her team.
But panic does not break out in the emergency room at Europe’s largest software company. On the contrary: The MCC team, consisting of 16 men and women of varying ages, remains quiet and focused. They know full well that when customers’ emotions are running at their highest, they need to stay very calm. After all, it’s not unknown for customers to shout at MCC employees because a great deal of money – and perhaps even their jobs – are at stake.
So, the SAP employees also function as a kind of lightning conductor. But that’s what the “emergency doctors” at the MCC have been trained to do. They all receive training to hone their soft skills, says Jansen, because these are an essential supplement to their specialist expertise.
Some 35% of the “emergency doctors” work at the MCC full-time; the other 65% join them for periods of between four weeks and six months.
Most critical incidents can be solved within three hours; only a very few take days or even weeks to sort out.
They are for the most part “SAP MaxAttention Services” personnel who have practical experience and generally work at large customers. This approach to staffing the MCC is SAP’s way of gaining new expertise and a fresh, outside perspective. SAP often deploys its own employees to work on site at its 1,000 biggest customers. And they are of course also involved immediately when business down events occur.
Food industry: extra critical
Patients requiring emergency treatment at the MCC can potentially come from the entire spectrum of SAP customers. They might include an online sales platform operator, a bank with multiple IT systems, a car manufacturer with production lines, airline employees receiving training, or an oil platform in the ocean. Problems occurring in the food industry are among the most critical, because milk products, for example, are highly perishable. If production comes to a standstill, the affected company not only can’t deliver its goods, but, if the worst comes to the worst, its products could simply go off.
Sometimes, the cause of a business-critical disruption is fairly mundane – the hard disk might simply be full, for example. But the challenges are usually more multifaceted than that. After all, the SAP software does not run in isolation; it interacts across interfaces with other computer programs and components in a kind of Lego system.
Usually hardware problems
While the people fighting for a patient’s life in real emergency rooms are surgeons, anaesthetists, internal medical specialists, and so on, the SAP emergency room is staffed by database and cloud services specialists, support consultants, and SAP software experts. When larger-scale problems occur, experts with a mix of skills team up together. They are all required to have in-depth technical knowledge in a certain area and generally have at least five and often more than ten years’ SAP experience, says Jansen. Based in Walldorf, near Heidelberg, Germany, SAP operates in more than 180 countries and has over 413,000 customers worldwide.
That being the case, there are surprisingly few emergencies. On this particular visit to the MCC, there are only ten highly critical events showing on the mega video screens installed in the light and simply furnished room with a small space shuttle sticker on the door.
The countries affected include Germany, Australia, the United States, Japan, Brazil, China, and Singapore. Most critical incidents can be solved within three hours; only a very few take days or even weeks to sort out. In these cases, the customer has usually suffered hardware damage and spare parts may not be readily available. Ninety-five out of 100 incidents, says escalation manager Jansen, are problems caused by hardware, human error, or the side-effects of other IT processes, and do not require a software solution. SAP only has to make software repairs in 2% to 6% of cases.
SAP operates four MCCs worldwide; besides the one in St. Leon-Rot – with its supporting centers in Dublin and St. Petersburg – there are MCCs in Newtown Square, Pennsylvania, USA, in Brazil, and in APJ, with supporting centers in Australia, Japan, China, Korea, and India. The latter are linked up virtually. The MCC in China was the first to be established, back in 2012. The various centers collaborate using the follow-the-sun principle: Europe hands over to the United States and South America at 5 p.m., and they in turn hand over to Asia eight hours later.
The emergency rooms were a major leap forward for SAP in responding to highly critical customer incidents. Displaying incidents on mega screens increases employees’ motivation, says Wolfgang Eschenfelder, Global Head of the MCCs. The video wall creates a much greater sense of urgency than an e-mail in an employee’s Outlook inbox, he says.
The golden rules
If a problem cannot be solved in a particular time zone before the end of the working day, the emergency doctors pass on the patient’s medical notes to the team on the next continent. This is not, according to Jansen, a problem in itself, but it is not hard to see that there is a critical interface here. After all, language and culture play a different role from continent to continent, from country to country, and from company to company. So, it suits SAP fine that the global language of IT is English. Accordingly, says, Jansen, 99.9% of all processes are in English, even when German SMEs are referred to the MCC in St. Leon-Rot. Cultural differences or sensibilities can pose more of a problem if, for example, a Japanese customer does not wish to receive support from a Chinese support employee.
Assigned to the MCC is the Central Control Team, which not only keeps a watchful eye on extremely critical incidents, but also has to monitor several dozen other activities as well. This much smaller team is also very international: Its manager for the last year and a half has been a Brazilian, who came to Germany specifically to take up this role. As well as more monitors, what you also notice in this room are the “golden rules” of the MCC: “Truly understand”, “Deeply care”, and “Create legends” – and “Show empathy”, “Stay curious”, and “Ask questions”. The “touch-move” rule is also very important: Once you’ve touched a problem, you can’t just refuse it or pass it on without good reason. Which means you can’t shift responsibility to someone else, as happens much too often in many companies and departments.
The future of customer support
SAP attaches great importance to improving and enhancing its customer support. Some 4,000 of its 95,000+ employees work in this area. The company makes use of social media, too, such as WhatsApp groups for certain topics. SAP’s entry into cloud business a few years ago presented some very new challenges. Andreas Heckmann, global SVP and Head of SAP Support Delivery wants to increasingly bring support to customers in real time and thus end the process of opening tickets and sending instructions and questions back and forth. To achieve this, SAP is looking to technologies such as artificial intelligence (AI) and voice recognition. With more and more business processes running in real time, customer support needs to respond accordingly, says Heckmann, whose SAP career has included ten years working in China and who only returned to Germany just over a year ago. Tickets are intrinsically very slow, he says. SAP even wants to offer “self-healing” processes in the future, and it plans to equip its support organization with AI technology that identifies problems before the customer has even noticed them.
High seasonality in the MCC
Seasonal fluctuations are very noticeable at the MCC. While August is very quiet, due to the summer vacation period, it’s all hands-on deck as the year-end approaches. Companies are very active between October and mid-December. May and June, with their numerous long weekends spanning public holidays, are also an intensive time for the MCC employees, because long weekends are an ideal time for companies to implement their IT projects. As the year ends, the men and women of the MCC just have to wait a little longer before they enter into the festive season spirit.
But the team appears remarkably enthusiastic. The managers and employees really do give the impression of being eager to solve customer problems. Despite the stress they experience, many seem to feel it is an honour to work here. And if it does all get too much, there are green sofas to relax on in the corridor outside the MCC – green being a calming colour. That’s not just good for the employees, but also for the next SAP customer reporting that their “business is down”.