Business Process Mining: Automatic Extraction of Business Process; A Reverse Engineering Approach
In modern day world of business, enterprise systems (ERP) store relevant information from executed processes in some structured form. For example workflow stores start and completion of activities, SAP processes log all transaction information and CRM systems log interactions with customers etc. Business events are usually recorded by the system as event-logs. The information can be of great value if it could be analyzed properly, but this is where shortcomings of ERP system become clear.
Business process-mining can be used to analyze these logs and extract an explicit process model from it besides utilizing them for other type of analyses. The main goal of process mining is to build models without appropriate knowledge, based on sequence of events; one can look for presence or absence of certain patterns and deduce some process models from it.
Business process-mining techniques are now becoming available as graphical interface-driven software tools where processes can only be represented as a flow diagram (more as Visio format) but in near future may be they can also be manipulated as part of mining and process analysis. Process-mining is still a new field and state-of the art field/technology and is an expanding area.
One of the main purposes of business process mining is to compare and analyze the run/execution time analysis of process to its design time as explained in the figure below.
The SAP ERP business processes and best-practices are seen as the key intellectual property of SAP. However, these processes are hidden in the code and distributed over the landscape. A significant part of that code was/has been developed with a clear structure in mind; it is extremely difficult to extract this structure from the running system. For instance, if the code has been developed using given business process models, then the running system only gives very little clues, which one of these models is currently under execution. On the other hand, information about the actual deployment and runtime usage of the system can be extremely helpful for SAP customer and developers, because it provides valuable data on the system’s actual use and behavior, which in turn can be used as input to various analysis and optimization tasks. When it comes to analyze a process distributed over a landscape SAP systems are like black box to the non-SAP systems and vice versa. So far, several attempts have been made to analyze the processes spread over the landscape but not with significant success. It is important to mention over here that so far most of the attempts were/are made on the analysis of workflow or transactional data. In actuality/modern day business world a process starting in a system goes through several systems and several landscapes (i.e. through different customers/vendors depending upon the complexity of the process) and ends somewhere. Based on the current approach and methodology it is not easy (if not impossible) to achieve the task unless a proper technique and methodology is applied. Process-Mining is the method of distilling a structured process description from a set of real execution. The real executions are captured in an event-log. The event-log is the data basis for process mining. Thus, process mining can only yield to meaningful results if the log files contain sufficient information and if the events are logged properly. Only events captured in the log are available for process mining. If an event is not logged, it will not be part the mined process model. Logging is by all means crucial. The following assumptions are made.
Assume that is possible to record events such that
- Each event refers to a task (i.e., a well-defined step in the process)
- Each event refers to a process instance
- Events are totally ordered (i.e., in the log events are recorded sequentially, even though tasks may be executed in parallel)
These assumptions assure that each event can be properly associated to a process instance. If the log contains events from several different process types, an additional requirement is necessary, according to the experience
4. Each process instance refers to one process type or each task refers to one process type.
Special tools are available in the market to analyze the logs within SAP system that are capable of automatically extracting business process structures from runtime artifacts of the system, namely from event logs. The steady stream of events collected in log-files can be read by the process-mining tool/system that then generates structured process models out of these real executions using process mining techniques. The gained process structure is extremely useful, because it untangles the low-level, unreadable event log messages and transforms them into model-level, human-readable and graphical format. The extraction of business process structures has the potential to revolutionize analysis, root cause findings, maintenance, and optimization tasks within SAP.
The standard generic platform/tool EMMA/BPEM could be used to analyze the logs (all kind of logs). The IS-U product Business Process Exception Management (BPEM) is built on EMMA (Enhanced Message Management Analysis) generic framework for monitoring, analyzing and exception management of business processes based on the log messages. It is part of the SAP ERP system and SAP_ABA 7.1+. The tool collects messages of business processes, analyzes them, identifies exceptions based on customizing and provides functionalities to solve the problem. Although the main goal of it is to smooth the execution of business processes, it seems very interesting for process mining because of its capability to collect messages from various sources. EMMA can analyze messages from the application log, Job log, System log, IDoc log, Workflow log, RFC, XI, CRM Middleware, Front Office Processes and the best of all logs from any NON-SAP system as long as it can be displayed in the T100 format (the way SAP writes the log in the system; check my other blogs) .
Now, at this time I don’t want to get into the technical details of how we could make everything work etc. At this time I would like to focus to the possible results to show you that it is possible to achieve a 100% confirmed results with process mining. Instead of developing (reinventing) a tool for process mining we focused towards what is currently possible in the market. There are several different tools and methods available in the market where the tool ProM (development of a Dutch professor W.M.P. van der Aalst) seems the most flexible and capable of meeting our requirements. W.M.P. van der Aalst (http://is.tm.tue.nl/staff/wvdaalst/) is a world renounced leading specialist in the field of process-mining area.
Let’s take a quick look at the structure of the mined process. Each node of the mined processes represents one task type. The first line of the node description shows the WorkflowModelElement, i.e. the task type identifier and description. The second line is the EventType, which is not used here and always just set to “complete”. The third line is the frequency of this task type in the log. The nodes are connected with directed edges. Each edge represents a mined dependency relation between the two task types. The frequency of this relation is displayed next to the edge.
The business process model automatically generated out of event-logs can help to visualize actual executions of the business processes in the deployed ERP system and over the landscape. The information about the runtime behavior of business process, ERP system/s and landscape, in particular about the way the business processes have been performed can be an exciting achievement for SAP and customers. The potential of approach seems to be tremendous. Analysis, optimization and development of the business processes can be improved to the benefit of SAP’s customers and developers alike. Most important, is the fact that the key intellectual property of SAP, the business processes, is made visible and tangible, and thereby accessible for future development and evolution, and great advantage in the competition for the ERP market. The steady stream of events collected in log-files read by the system then generates structured process models out of these real executions using process mining techniques. The gained process structure is extremely useful, because it untangles the low-level, unreadable event log messages and transforms them into model-level, human-readable and graphical format. The extraction of business process structures has the potential to revolutionize analysis, maintenance, and optimization tasks within SAP:Analysis: the data can be used to uncover weak spots in the code by analyzing, which subtasks spend most time and resources; previously undetected critical resource bottleneck can be uncovered.Maintenance: Because the process structure of the system is made explicit by the system, the maintenance tasks become more transparent and can make use of the relationships between certain parts of the system that have been uncovered by the extraction of the business process structures.
Optimization with the acurate knowledge of the use of the ecosystem, business process can be optimized, algorithms tunes, and user interfaces polished resulting in a dramatic value increase for the user experience.
A further aspect is that each company faces the problem that goods and services are ordered by customers but do not produce any revenue until the goods and services are paid. The value chain from order to cash is critical in terms of way of execution as well as in terms of error-free execution. Therefore, it is obvious that running all the single business processes from order to cash in an error-free manner is one of the top goals of any organization. On top of running these processes as error-free as possible, it is quite important to take care of the situation if an exception/error occurs. It is a fact that exceptions will occur no matter how well a process is planned, the employees are trained, and monitored. The system can be used to monitor the business processes and their evolution by comparing the current executions of the processes with the process model generated from previous executions; Still a lot of work to be done on this very promising subject. Your comments are welcome and appreciated.
- SAP Managed Tags:
1 Comment
