What Is Complex Event Processing (CEP)?

What is Complex Event Processing?

What is Complex Event Processing (CEP)?It starts with the three Vs of big data. Most of the discussion is centered around the first V, Volume. Finding the best way to store terabytes, or even exabytes, of data is a thorny question.

Historical data is just the first building block of a big data strategy. Having a strategy to deal with the velocity and the variety of your data are key to becoming a real-time enterprise. Thats where Complex Event Processing (CEP) comes in.

A CEP Engine combines information from a variety of sources. It looks for patterns in these event streams and then responds in real-time. You could have a number of streams of information coming from employees, machines, or customers. Tying all of this incoming data together allows you to find complex patterns across different channels. Once a pattern is recognized the system can trigger a response.

How is CEP being used?

CEP is already a mature technology in a few specific markets, but it is poised to cross the chasm into a number of data driven industries. So far the widest adoption is in the financial industry. There’s a whole breed of Wall Street traders that use finely tuned complex event processing algorithms to trade at high frequency and high volume.

This requires scraping thousands of news articles a day and comparing them to even small movements in price to find micro events. Traders then create rules to buy or sell based on which micro events the CEP engine registers.

The next use case might surprise you: chances are you are driving a complex event processing system to work everyday. Cars nowadays are amazingly engineered systems with a lot of sensors. When the speed sensor shows the car suddenly slowing down, but the brake pedal hasn’t been pressed, and a tire sensor is suddenly showing low pressure, the car can trigger an emergency procedure to increase traction control, tighten the seat belts, and deploy an airbag.

All of this can happen faster than you can react and hit the brakes. The combination of signals from the tire, the brakes, and the speedometer defines a complex event. The safety processes are the logic that gets triggered when a complex event is registered.

Where will Complex Event Processing show up next?

There is a huge opportunity for experts who can work with CEP systems to understand what signals are most important. The opportunities for CEP are far ranging. Some of the most promising areas of innovation are the utilities, oil and gas, retail, and discrete manufacturing industries.