Current Methods of Testing and Their Constraints

Historically, market participants have tested their algorithms and strategies in 3 ways:

  • ✺

    On Live Markets

  • ✺

    With Exchanges

  • ✺

    With Bespoke Simulators

The complexity of the markets and algorithmic development processes are now by far outstripping the testing infrastructure and operating models of most firms. Consequently, there are significant limitations of those historic testing paradigms.

Live Markets

  • It is no longer legal in certain regulatory jurisdictions (e.g. EU and APAC) to test your models in live markets.  Additionally, it is not best practice to test in live markets in agency trading (for clients) anywhere in the world.

  • Live market trading does not test for all eventualities that may occur in the market, and regulators need evidence of broader market testing against all possible market scenarios and legal obligations).

Exchange Testing

The testing experience that exchanges offer to their members has been, for so long, recognised by exchanges and members alike, as being sub-optimal.  These issues are broad ranging, but simple put, Exchange testing environments are not realistic and are overly simple.

  • Limited Exchange time-slots for testing:

    • With limited-availability testing windows, members’ day-to-day testing of connectivity and trading models is hindered. 

    • Member issues are more pronounced during a significant exchange upgrade and the member cannot test in a structured, high-available manner with the exchange. 

    • Market participants can be out of the market, slow innovation, or significantly delay exchange upgrades.

  • Lack of deterministic, repeatable testing

    • As with many exchanges, members lack determinism if multiple members concurrently use a single exchange simulation environment

    • Members without their own testing environments often have to share time-slots with other market participants and test against non-production venue systems or non-realistic data sets.

    • Lack of determinism is a major constraint to testing the efficacy of your trading infrastructure.

Algorithmic models are by their nature complex and have many moving parts to test to ensure that trading strategies execute in the market as expected and that trading infrastructures operate optimally.  Specifically exchange testing environments do not accommodate the testing of:

  • Latencies

    • Exchange simulations have no real way of tuning your strategies to the latencies of market participant’s infrastructure.  This gives market participants an unrealistic sense of where their trading strategies would execute in a moving market.

  • Single Exchange Strategies

    • Rather than integrated with the test, Exchange simulations often provide no realistic data or dynamic replay data which market participants can interact with (deterministically or at all).

  • Multiple Exchange Strategies

    • Where markets have a regulatory routing component (e.g. the US with RegNMS), or where market participants deploy sector trading (as is usual) across multiple exchanges or geographic regions, Exchanges currently offer no way for those market participants to test their strategies effectively.

  • Stress Testing

    • It is a regulatory requirement in every region that market participants are able to hand 2-4x times the velocity/volumes in the market.  No exchange provides the capability to test these market levels.

Bespoke Simulators

Many market participants use bespoke and generic market simulators to test their trading strategies.  However, there are many recognized constraints associated with this approach. They are:

  • Typically generic in nature

  • Not true representations of the market

  • Often just a reflection of what may have been expected in some (historic) market scenarios and not a true reflection of what will happen in every market scenario. 

  • Often don’t use the complete data/order/execution sets from a particular day/period, take a long time to set up and maintain, and often create potential downsides

Downsides:

  • Over-reliance on a generic, order-fill ratio model

  • Unrealistic trades without real market data feedback and live prices

  • The lack of market psychology in simulated data

Quantum solves for all of these issues and more, providing its clients with the flexibility and capabilities that market participants need to ensure that execution and regulatory testing processes keep up with today’s complex, ever-changing markets.