Why you Need Complex Event Processing

Posted by Neil McGovern, Senior Director of Marketing at Sybase on Feb 03, 2012

“Our currency is not AAA, and in recent months the performance of our government has not been AAA, but our debt is AAA.” — Warren Buffett in response to the Standard and Poor’s downgrade of U.S. treasuries on August 5th, 2011

It seems the markets agree with Warren Buffet on this one. A downgrade in an instrument usually results in a rush to unload said instrument. But the result in this case was a rush to safety, which still means U.S. treasuries, the very instruments downgraded. A spike in market volatility (as measured by the Chicago Board Options Exchange Market Volatility Index or VIX) lured high frequency traders (HFT) to the profits as if they were sharks on the trail of blood in the water.

The HFT practitioners thrive on volatility. Indeed, they benefit from it, as Tradeworx founder and CEO Manoj Narang pointed out. HFT cannot cause volatility, according to Narang, otherwise volatility rates would be perpetually high, maximizing HFT profits.


As latency rates decrease, a big weakness for HFT is time. Software-based HFT systems have given way to collocated hardware solutions, which has the trading strategies burned in. Time and effort required to change the systems skyrocket, yielding ultra-low latency trading systems with high latency change cycles.

So when there is a major market change, such as a downgrade of sovereign debt of a major economy or a flash crash, the systems are not prepared and take a long time to adjust.

New electronic monitoring systems will be able to adjust the underlying trading systems, but there would be two major limitations: the range of control they can exert, and the depth and complexity of their monitoring abilities. One solution to this is to allow human traders to monitor and control low level trading systems and adjust them to new market scenarios. However, just as electronic trading replaced slow humans on the trading floor, electronic monitoring systems would replace slow humans on the monitoring desk.


But there is hope for humans in the mix. Just as a flash crash can overwhelm the capacity of an HFT collocation system, alterations beyond capacity will overwhelm those monitoring systems. Therefore the people monitoring the monitors must be very smart (so they can see a wider range of possibilities than the monitoring systems) and be able to change the monitoring systems as quickly as possible.


Fortunately for trading firms, there is a class of technology that is very good at monitoring systems in real time, and adjusts very quickly if the need arises. These are event-driven systems based on complex event processing (CEP) technology. Event-driven systems (i.e. monitoring systems) also need to be very flexible, which is a large part of the value brought to the table by CEP technology, such as Sybase Aleri ESP.

Monitoring systems are very complex, relying on the skills of the trading experts to both make the right decisions and learn. Event driven platforms, such as monitoring systems, are very complex pieces of technology. So the first major advantage a CEP solutions provides is a user interface that shows the flow of data and the decision points in a visual way that traders can quickly understand.

Why make your traders learn a computing language to do their job? They won’t thank you for it!

The trader and the technologist can change the visual diagram quickly and easily as soon as they see that the monitoring system is incorrectly controlling the trading engines. The technologist can then add any detailed code to implement the new strategy, and roll out the changes.

Using this three level system — trading engine, monitoring system and human oversight, a trading firm can react to a major market change within hours, or minutes, instead of days or weeks.

Comment(s): 2

Posted on Feb 03, 2012 by Derek K

How plugged in should upper management be, Neil? I cannot imagine that the human overseer will have authority to do *anything* to remedy a situation gone awry. Likewise, I cannot imagine that a firm's upper management will be inclined to constantly oversee trading.

Posted on Feb 03, 2012 by Neil McGovern

Hi Derek Different firms will have different policies regarding the level of authority at different levels. A very sclerotic firm will not be able to become flexible, and will either survive (maybe I’m wrong and the best thing to do is leave everything alone) or will suffer losses in comparison to the market, taking risk levels into account, and the more flexible firms will thrive.

Post your comment

Your name *
Your E-mail
captcha Image
What code is in the image? *