Do You Know Your System Quality Number?

Sep 24, 2021

Dr. Van Tharp developed the System Quality Number (SQN) as a simple method to rank trading systems.

This article covers all you need to get started:

  • The concept of R-multiples
  • The SQN formula
  • Obtaining your SQN using MT4 and QuantAnalyzer
  • Limitations of the SQN
  • Improving your SQN

Quantifying a strategy’s performance is a challenge that every trader faces.

In a previous post, I discussed MT4’s default backtest performance metrics. At the minimum, I recommend using the return/maximum drawdown ratio to get a measure of your strategy’s risk-adjusted performance.

The System Quality Number (SQN) is another popular metric that could help with your system evaluation.

Before discussing the SQN and its uses, we first need to understand R-multiples and R-expectancy.

Thinking in R-Multiples

All profits and losses should be related to your initial trade risk, R.

Your profits and losses should include transaction costs as well.

For example, if you decide to risk $100, but end up losing $200, your trade’s R-multiple is -2R. If you risk the same amount but instead have a $500 profit, your R-multiple is +5R.

Ideally, your losses should be no more than -1R. If you trade an illiquid market, slippage can often make you lose more than expected.

With the R-multiple of each trade calculated, you can then calculate your system’s R-expectancy. This is the average R-multiple across all the trades in your backtest.

Below I calculate the R-expectancy of a hypothetical system with 5 trades.

If you don’t know your trade risk beforehand, for instance if you use an indicator-based exit, you should use your backtest’s average loss as 1R. This will be done for the example below.

System Quality Number Formula

Here’s the formula for the SQN:

It is the ratio between the R-expectancy and its standard deviation, multiplied by the square root of the number of trades.

The expectancy measures your system’s profitability, while the standard deviation measures consistency.

If your individual R-multiples are closely clustered around the expectancy, your standard deviation will be low, improving your SQN.

Similarly, a large backtest sample size improves the statistical significance of your metrics, giving a higher SQN.

Dr. Tharp provides the following benchmarks, based on a sample size of 100 trades.

This particular sample size of 100 is important; I’ll come back to it when discussing the limitations of the SQN.

Calculating Your System Quality Number From MT4

MT4 doesn’t compute the SQN, so you’ll have to do some quick Excel work.

First, export your backtest report from MT4 and open it in Excel.

At the list of trades, you want to delete all rows with an empty Profit value. Excel’s Filter function works great for this.

MT4 logs all order modifications and deletions, but we are only interested in the trade outcome here.

Add a column to calculate the R-multiple for each trade. For simplicity, I’ll divide each trade profit/loss by the average losing trade.

Now you can calculate the R-expectancy and the standard deviation of the R-multiples.

Finally, using your number of trades and the formula above, you can obtain this system’s SQN of 3.85.

Obtaining Your System Quality Number in QuantAnalyzer

With QuantAnalyzer, you just need to load your MT4 report, and the SQN will be automatically calculated.

Limitations & Criticisms of the System Quality Number

Every metric has its benefits and drawbacks. I recommend using a combination of metrics to get a comprehensive assessment of your system.

Here are two areas where the SQN falls short:

1. Effects of Sample Size

In his book, Dr. Tharp mentions that a large sample size can overestimate the quality of your system.

This is because the SQN is proportional to the square root of the sample size.

To overcome this limitation, he proposes the following:

  • If you have fewer than 100 trades, simply use the formula above.
  • If you have over 100 trades, multiply the ratio between the R-expectancy and the standard deviation by 10. This assumes you have only 100 trades.

If you’re an algorithmic trader with a multi-year backtest, chances are the second point applies to you. Depending on your sample size, this approach can be extremely conservative.

Let’s illustrate this using the system above, which produces a SQN of 3.85 after 1167 trades.

If I used a sample size of 100 instead, the SQN plummets to 1.13!

Using Dr. Tharp’s benchmark table shown above, this has transformed the system from ‘excellent’ to ‘average.’

Here’s how the system’s SQN varies as sample size increases.

If you have a long-term backtest like the one above, you will probably overestimate your system’s quality. And if you artificially cap your sample size at 100, your system will become almost untradable.

Personally, the large impact of sample size on the SQN lowers its credibility. A system’s quality should not be affected by sample size; you are testing the same system after all. A larger sample only serves to make your metrics more reliable.

It makes more sense to apply the SQN when doing manual backtesting or forward testing, where sample sizes are generally small (well below 100).

In addition, arbitrarily capping your sample size will penalize systems that trade more frequently. Such systems are generally more efficient in making money, which is everyone’s ultimate aim in trading.

2. Effects of Large Winners/Losers

Recall that the SQN is inversely proportional to the standard deviation of your R-multiples, which measures the consistency of your individual trades.

A handful of abnormally large trade results will increase this standard deviation value, lowering your SQN.

Large losers will rightfully lower your SQN, but unfortunately large winners have the same effect. A large winner will increase the standard deviation more than it increases R-expectancy.

This can be problematic if you’re evaluating a trend following system, which usually contains the occasional large winner. These large winners create ‘fat tail’ trade distributions, which often make trend following profitable in the long run.

I’d prefer a metric that doesn’t penalize upside volatility.

To focus on downside volatility, you can amend the SQN formula to only include the standard deviation of losing trades. This is similar to how the Sortino ratio improves upon the Sharpe ratio.

How to Improve Your System Quality Number

Van Tharp mentions there’s a strong correlation between win rate and SQN.

Win rate should never be considered in isolation. Your average win and average loss also have a major impact on overall performance.

A good way to improve your win rate without affecting the rest of your system is to add entry filters. I discuss various time, trend and volatility filters here.

I suggest adding 1-2 filters at most, and if the system still fails to meet your expectations, it’s better to move on to another idea.

Continuing to modify a system with the express intent of improving your metrics can easily lead to overfitting. Systems with spectacular backtests usually deteriorate drastically in real-time.

Wrapping Up

The SQN is an innovative measure of your system’s profitability and consistency. It also tends to favour large sample sizes, which bodes well for your backtest reliability.

You can obtain it with some manipulation of your MT4 backtest report, but I suggest opening your report in QuantAnalyzer and having it calculated automatically.

Personally, I do not use the SQN because of the limitations explained above. Nonetheless, it can be useful when comparing systems with similar trading frequency. Alternatively, you can remove its reliance on sample size and simply compare the ratio of expectancy over standard deviation.

Like any other metric, it’s only meaningful if your system actually trades like it backtests. Try to backtest your system over different market conditions and if possible, apply the SQN to out-of-sample results.

If you’d like to learn more about the SQN, I recommend reading Van Tharp’s Definitive Guide to Position Sizing.

Are you a fan of Van Tharp’s SQN? Let me know in the comments!

Powered By

Development Platform

Forex VPS

FXVM Forex VPS

Popular Posts

No Results Found

The page you requested could not be found. Try refining your search, or use the navigation above to locate the post.

Make your money work for you!

Get promotions, trading ideas and strategy development tips delivered to your inbox!

Comments

5 Comments

  1. Massimo

    Am I wrong saying that if R is constant SQN result is independent from R ? (Found applying the formula to whaterever list of trades, but it is obvious..)
    If I am correct, how can it be a correct way to eveluate a strategy ?

    Reply
    • Wayne

      Hi Massimo, if you risk the same amount for each trade, R will be constant. But the trade outcome (R-multiple) will be different. This means different R-expectancy and standard deviation values, and a different SQN.

      Reply
  2. Mike

    I am a bit confused.
    I think the Strategy Quality Number is represented as “SQN” in StrategyQuant right?
    Not “STR Quality Number”

    Reply
    • Wayne

      they should be the same thing

      Reply
  3. Anonymous

    The solution is to evaluate the number of trades in the system vs. time, this is extremely important. If your system performs 2R on 100 trades a year, versus 1.5R on 500 trades, there’s a huge difference in performance (and annual return %). If you sample 100 trades in the 500 system it looks worse but actually outperforms the other over 3x…

    Reply

Submit a Comment

Your email address will not be published. Required fields are marked *

Trading Strategies

Ichimoku Trend Following Strategy

Ichimoku Trend Following Strategy

Like candlesticks, the Ichimoku indicator is a fine Japanese creation. Here I’ll explain how the Ichimoku is plotted, and use it to build a trend following strategy for the USDJPY.

Forex Trading With Volume – Does It Work?

Forex Trading With Volume – Does It Work?

Volume is seldom discussed in forex trading. But as a stock trader would tell you, volume provides insights into the underlying market sentiment. Here I’ll demonstrate the use of two simple volume filters to improve entry reliability.

What Is the Kaufman Adaptive Moving Average?

What Is the Kaufman Adaptive Moving Average?

The Kaufman Adaptive Moving Average is a unique indicator that automatically adapts to the market’s noise. Here I explain its inner workings and show you how to build a trend following strategy around it.

Awesome Oscillator Forex Trend Strategy

Awesome Oscillator Forex Trend Strategy

The Awesome Oscillator is one of Bill Williams’ fine creations. It is a simple yet effective momentum indicator that can be a valuable addition to your strategy. Here we’ll use it together with a moving average to fine-tune our trend following entries.

Dual CCI Indicator Forex Trend Strategy

Dual CCI Indicator Forex Trend Strategy

The Commodity Channel Index (CCI) is an underrated momentum indicator that is often overshadowed by the RSI and MACD. I’ll show you how to build a simple trend following strategy using a pair of CCI indicators!

What is Fixed Ratio Money Management?

What is Fixed Ratio Money Management?

Have you heard of fixed ratio money management? How does it compare to the popular fixed fractional approach? Here I’ll explain how fixed ratio works, and see how it stacks up against fixed fractional money management.

What Is the Kaufman Efficiency Ratio?

What Is the Kaufman Efficiency Ratio?

Avoiding false breakouts is a common goal among trend traders. The Kaufman Efficiency Ratio provides a simple method of quantifying a market’s noise, helping traders focus on the smoothest trends. Let’s add the efficiency ratio to a simple trend following strategy and see whether its performance improves.

What Is the QQE Indicator?

What Is the QQE Indicator?

The QQE is a mysterious indicator that sometimes pops up in trading forums. Does it deserve a place alongside the more traditional momentum indicators like the RSI and CCI? Let’s add it to a trend following strategy to find out!

Do Bollinger Bands + Candlestick Patterns Work?

Do Bollinger Bands + Candlestick Patterns Work?

Bollinger Bands are great at detecting overbought and oversold conditions. Let’s use them to develop a countertrend strategy, and then refine our entries using limit entries and candlestick patterns.

How Good Are The Bollinger Bands’ Trailing Stops?

How Good Are The Bollinger Bands’ Trailing Stops?

Trailing stop losses are a popular feature in many trend following systems. Bollinger Bands, the ever-popular technical indicator among retail traders, actually contain two inbuilt trailing stops. Are these any good? Let’s find out!

Bollinger Bands vs. Keltner Channels

Bollinger Bands vs. Keltner Channels

Bollinger Bands are ubiquitous in the trading world, but there’s less discussion about their lesser-known cousin, the Keltner Channels. Do these perform as well as the Bollinger Bands? Let’s compare them over a broad range of input parameters.

Strategy Development

No Results Found

The page you requested could not be found. Try refining your search, or use the navigation above to locate the post.

Live Trading

No Results Found

The page you requested could not be found. Try refining your search, or use the navigation above to locate the post.

Make your money work for you!

Make your money work for you!

 

Get trading ideas and strategy development tips delivered to your inbox!

Thanks for subscribing!

Pin It on Pinterest

Share This