Do You Know Your System Quality Number?

Dr. Van Tharp developed the System Quality Number (SQN) as a simple method to rank trading systems.

This article covers all you need to get started:

• The concept of R-multiples
• The SQN formula
• Obtaining your SQN using MT4 and QuantAnalyzer
• Limitations of the SQN
• Improving your SQN

Quantifying a strategy’s performance is a challenge that every trader faces.

In a previous post, I discussed MT4’s default backtest performance metrics. At the minimum, I recommend using the return/maximum drawdown ratio to get a measure of your strategy’s risk-adjusted performance.

The System Quality Number (SQN) is another popular metric that could help with your system evaluation.

Before discussing the SQN and its uses, we first need to understand R-multiples and R-expectancy.

Thinking in R-Multiples

All profits and losses should be related to your initial trade risk, R.

Your profits and losses should include transaction costs as well.

For example, if you decide to risk $100, but end up losing $200, your trade’s R-multiple is -2R. If you risk the same amount but instead have a $500 profit, your R-multiple is +5R.

Ideally, your losses should be no more than -1R. If you trade an illiquid market, slippage can often make you lose more than expected.

With the R-multiple of each trade calculated, you can then calculate your system’s R-expectancy. This is the average R-multiple across all the trades in your backtest.

Below I calculate the R-expectancy of a hypothetical system with 5 trades.

If you don’t know your trade risk beforehand, for instance if you use an indicator-based exit, you should use your backtest’s average loss as 1R. This will be done for the example below.

System Quality Number Formula

Here’s the formula for the SQN:

It is the ratio between the R-expectancy and its standard deviation, multiplied by the square root of the number of trades.

The expectancy measures your system’s profitability, while the standard deviation measures consistency.

If your individual R-multiples are closely clustered around the expectancy, your standard deviation will be low, improving your SQN.

Similarly, a large backtest sample size improves the statistical significance of your metrics, giving a higher SQN.

Dr. Tharp provides the following benchmarks, based on a sample size of 100 trades.

This particular sample size of 100 is important; I’ll come back to it when discussing the limitations of the SQN.

Calculating Your System Quality Number From MT4

MT4 doesn’t compute the SQN, so you’ll have to do some quick Excel work.

First, export your backtest report from MT4 and open it in Excel.

At the list of trades, you want to delete all rows with an empty Profit value. Excel’s Filter function works great for this.

MT4 logs all order modifications and deletions, but we are only interested in the trade outcome here.

Add a column to calculate the R-multiple for each trade. For simplicity, I’ll divide each trade profit/loss by the average losing trade.

Now you can calculate the R-expectancy and the standard deviation of the R-multiples.

Finally, using your number of trades and the formula above, you can obtain this system’s SQN of 3.85.

Obtaining Your System Quality Number in QuantAnalyzer

With QuantAnalyzer, you just need to load your MT4 report, and the SQN will be automatically calculated.

Limitations & Criticisms of the System Quality Number

Every metric has its benefits and drawbacks. I recommend using a combination of metrics to get a comprehensive assessment of your system.

Here are two areas where the SQN falls short:

1. Effects of Sample Size

In his book, Dr. Tharp mentions that a large sample size can overestimate the quality of your system.

This is because the SQN is proportional to the square root of the sample size.

To overcome this limitation, he proposes the following:

• If you have fewer than 100 trades, simply use the formula above.
• If you have over 100 trades, multiply the ratio between the R-expectancy and the standard deviation by 10. This assumes you have only 100 trades.

If you’re an algorithmic trader with a multi-year backtest, chances are the second point applies to you. Depending on your sample size, this approach can be extremely conservative.

Let’s illustrate this using the system above, which produces a SQN of 3.85 after 1167 trades.

If I used a sample size of 100 instead, the SQN plummets to 1.13!

Using Dr. Tharp’s benchmark table shown above, this has transformed the system from ‘excellent’ to ‘average.’

Here’s how the system’s SQN varies as sample size increases.

If you have a long-term backtest like the one above, you will probably overestimate your system’s quality. And if you artificially cap your sample size at 100, your system will become almost untradable.

Personally, the large impact of sample size on the SQN lowers its credibility. A system’s quality should not be affected by sample size; you are testing the same system after all. A larger sample only serves to make your metrics more reliable.

It makes more sense to apply the SQN when doing manual backtesting or forward testing, where sample sizes are generally small (well below 100).

In addition, arbitrarily capping your sample size will penalize systems that trade more frequently. Such systems are generally more efficient in making money, which is everyone’s ultimate aim in trading.

2. Effects of Large Winners/Losers

Recall that the SQN is inversely proportional to the standard deviation of your R-multiples, which measures the consistency of your individual trades.

A handful of abnormally large trade results will increase this standard deviation value, lowering your SQN.

Large losers will rightfully lower your SQN, but unfortunately large winners have the same effect. A large winner will increase the standard deviation more than it increases R-expectancy.

This can be problematic if you’re evaluating a trend following system, which usually contains the occasional large winner. These large winners create ‘fat tail’ trade distributions, which often make trend following profitable in the long run.

I’d prefer a metric that doesn’t penalize upside volatility.

To focus on downside volatility, you can amend the SQN formula to only include the standard deviation of losing trades. This is similar to how the Sortino ratio improves upon the Sharpe ratio.

How to Improve Your System Quality Number

Van Tharp mentions there’s a strong correlation between win rate and SQN.

Win rate should never be considered in isolation. Your average win and average loss also have a major impact on overall performance.

A good way to improve your win rate without affecting the rest of your system is to add entry filters. I discuss various time, trend and volatility filters here.

I suggest adding 1-2 filters at most, and if the system still fails to meet your expectations, it’s better to move on to another idea.

Continuing to modify a system with the express intent of improving your metrics can easily lead to overfitting. Systems with spectacular backtests usually deteriorate drastically in real-time.

Wrapping Up

The SQN is an innovative measure of your system’s profitability and consistency. It also tends to favour large sample sizes, which bodes well for your backtest reliability.

You can obtain it with some manipulation of your MT4 backtest report, but I suggest opening your report in QuantAnalyzer and having it calculated automatically.

Personally, I do not use the SQN because of the limitations explained above. Nonetheless, it can be useful when comparing systems with similar trading frequency. Alternatively, you can remove its reliance on sample size and simply compare the ratio of expectancy over standard deviation.

Like any other metric, it’s only meaningful if your system actually trades like it backtests. Try to backtest your system over different market conditions and if possible, apply the SQN to out-of-sample results.

If you’d like to learn more about the SQN, I recommend reading Van Tharp’s Definitive Guide to Position Sizing.

Are you a fan of Van Tharp’s SQN? Let me know in the comments!