“The deluge of information stemming from social media, mobile devices and machine-generated devices,” is a definition of Big Data cited by 18 percent of 154 C-level executives at international companies in a recent SAP-sponsored poll. More than half indicated that Big Data was either a colossal expansion of transaction data or new technologies addressing that volume.
Whether your firm is conducting for sentiment analysis from among the deluge of social media, or looking for the nuggets of valuable information amongst the ever wider expanse of transaction data, Complex Event Processing (CEP) offers the technological foundation. My fellow Trading & Risk Technology blogger Domenic Iannaccone gave a primer on CEP in his Thursday webcast.
Investment banks around the world are preparing for 2012’s bleak second half, according to a Financial Times article Thursday. But that’s just the time for firms to invest in continuous insight, improved agility and easily deployable technology.
Flipping the Model
Traditional business intelligence solutions collect and consolidate data into warehouses, where it sits until users query and analyze it. This is a conventional way to backtest new models, but it can cost time and opportunity when trying to look at new data.
So CEP flips the model, storing the queries and letting data flow through them. Firms can rely on real-time data — as opposed to historical data, which may not reflect conditions in the real-time trading environment.
High-level real-time use cases for CEP include:
- Situation detection, or scanning live event streams for business critical situations
- Automated response, constantly adjusting business processes
- Stream transformation, turning raw data into actionable information
In his webcast, “Raise the Bar on Trade Execution with Real-time Liquidity Analytics,” Iannaccone discussed three of the benefits of CEP technology:
- Analyzes events as they occur: Empowers you to respond immediately.
- Develops applications quickly: Reduces your dependence on special programming.
- Deploys non-intrusively: Adapts to your existing data models.
No Sinking Sand with CEP
Firms that do not install a solid CEP foundation risk being swept away by the torrents of data that more tech-savvy firms have adopted. One example is Seattle-based data software provider Quantia Analytics, which relies on Sybase’s Event Stream Processor (ESP) as the platform for its turnkey market data enrichment service, TrueTick.
TrueTick helps users to drill down into real-time data, getting an instantaneous view of asymmetry via spider web-like charts (pictured left). Each arm represents a market (NASDAQ, BATS, etc.), and different colors indicate best price and ticks of additional spread.
“Where are your problem children?” asked Stephen Elston, Ph.D., managing director of Quantia during Iannaccone’s webcast. “In a small amount of screen real estate, you see the symbols that might involve some investigation — some check that something really weird isn’t happening in the market right now.”
Graphically representing many symbols, TrueTick uses colors to indicate a stock’s situation relative to others. Hotter colors (pictured right, upper left) have the most asymmetry, while cooler colors (lower right) have the least to help users identify a potential problem.
CEP technology updates the data every couple of minutes. Try competing with that using batch processing.
If you’re contemplating CEP technology implementation and would like to learn more, Iannaccone’s webcast is a good place to start. A recording is just became available here.
“Raise the Bar on Trade Execution with Real-time Liquidity Analytics” by Domenic Iannaccone
“Big Data of Growing Importance to Small Businesses: SAP” by Nathan Eddy
“Banks Fear for Second Half as Deals Fall” by Daniel Schäfer in London and Tracy Alloway