Do You Know Your System Quality Number?

Sep 24, 2021

Dr. Van Tharp developed the System Quality Number (SQN) as a simple method to rank trading systems.

This article covers all you need to get started:

  • The concept of R-multiples
  • The SQN formula
  • Obtaining your SQN using MT4 and QuantAnalyzer
  • Limitations of the SQN
  • Improving your SQN

Quantifying a strategy’s performance is a challenge that every trader faces.

In a previous post, I discussed MT4’s default backtest performance metrics. At the minimum, I recommend using the return/maximum drawdown ratio to get a measure of your strategy’s risk-adjusted performance.

The System Quality Number (SQN) is another popular metric that could help with your system evaluation.

Before discussing the SQN and its uses, we first need to understand R-multiples and R-expectancy.

Thinking in R-Multiples

All profits and losses should be related to your initial trade risk, R.

Your profits and losses should include transaction costs as well.

For example, if you decide to risk $100, but end up losing $200, your trade’s R-multiple is -2R. If you risk the same amount but instead have a $500 profit, your R-multiple is +5R.

Ideally, your losses should be no more than -1R. If you trade an illiquid market, slippage can often make you lose more than expected.

With the R-multiple of each trade calculated, you can then calculate your system’s R-expectancy. This is the average R-multiple across all the trades in your backtest.

Below I calculate the R-expectancy of a hypothetical system with 5 trades.

If you don’t know your trade risk beforehand, for instance if you use an indicator-based exit, you should use your backtest’s average loss as 1R. This will be done for the example below.

System Quality Number Formula

Here’s the formula for the SQN:

It is the ratio between the R-expectancy and its standard deviation, multiplied by the square root of the number of trades.

The expectancy measures your system’s profitability, while the standard deviation measures consistency.

If your individual R-multiples are closely clustered around the expectancy, your standard deviation will be low, improving your SQN.

Similarly, a large backtest sample size improves the statistical significance of your metrics, giving a higher SQN.

Dr. Tharp provides the following benchmarks, based on a sample size of 100 trades.

This particular sample size of 100 is important; I’ll come back to it when discussing the limitations of the SQN.

Calculating Your System Quality Number From MT4

MT4 doesn’t compute the SQN, so you’ll have to do some quick Excel work.

First, export your backtest report from MT4 and open it in Excel.

At the list of trades, you want to delete all rows with an empty Profit value. Excel’s Filter function works great for this.

MT4 logs all order modifications and deletions, but we are only interested in the trade outcome here.

Add a column to calculate the R-multiple for each trade. For simplicity, I’ll divide each trade profit/loss by the average losing trade.

Now you can calculate the R-expectancy and the standard deviation of the R-multiples.

Finally, using your number of trades and the formula above, you can obtain this system’s SQN of 3.85.

Obtaining Your System Quality Number in QuantAnalyzer

With QuantAnalyzer, you just need to load your MT4 report, and the SQN will be automatically calculated.

Limitations & Criticisms of the System Quality Number

Every metric has its benefits and drawbacks. I recommend using a combination of metrics to get a comprehensive assessment of your system.

Here are two areas where the SQN falls short:

1. Effects of Sample Size

In his book, Dr. Tharp mentions that a large sample size can overestimate the quality of your system.

This is because the SQN is proportional to the square root of the sample size.

To overcome this limitation, he proposes the following:

  • If you have fewer than 100 trades, simply use the formula above.
  • If you have over 100 trades, multiply the ratio between the R-expectancy and the standard deviation by 10. This assumes you have only 100 trades.

If you’re an algorithmic trader with a multi-year backtest, chances are the second point applies to you. Depending on your sample size, this approach can be extremely conservative.

Let’s illustrate this using the system above, which produces a SQN of 3.85 after 1167 trades.

If I used a sample size of 100 instead, the SQN plummets to 1.13!

Using Dr. Tharp’s benchmark table shown above, this has transformed the system from ‘excellent’ to ‘average.’

Here’s how the system’s SQN varies as sample size increases.

If you have a long-term backtest like the one above, you will probably overestimate your system’s quality. And if you artificially cap your sample size at 100, your system will become almost untradable.

Personally, the large impact of sample size on the SQN lowers its credibility. A system’s quality should not be affected by sample size; you are testing the same system after all. A larger sample only serves to make your metrics more reliable.

It makes more sense to apply the SQN when doing manual backtesting or forward testing, where sample sizes are generally small (well below 100).

In addition, arbitrarily capping your sample size will penalize systems that trade more frequently. Such systems are generally more efficient in making money, which is everyone’s ultimate aim in trading.

2. Effects of Large Winners/Losers

Recall that the SQN is inversely proportional to the standard deviation of your R-multiples, which measures the consistency of your individual trades.

A handful of abnormally large trade results will increase this standard deviation value, lowering your SQN.

Large losers will rightfully lower your SQN, but unfortunately large winners have the same effect. A large winner will increase the standard deviation more than it increases R-expectancy.

This can be problematic if you’re evaluating a trend following system, which usually contains the occasional large winner. These large winners create ‘fat tail’ trade distributions, which often make trend following profitable in the long run.

I’d prefer a metric that doesn’t penalize upside volatility.

To focus on downside volatility, you can amend the SQN formula to only include the standard deviation of losing trades. This is similar to how the Sortino ratio improves upon the Sharpe ratio.

How to Improve Your System Quality Number

Van Tharp mentions there’s a strong correlation between win rate and SQN.

Win rate should never be considered in isolation. Your average win and average loss also have a major impact on overall performance.

A good way to improve your win rate without affecting the rest of your system is to add entry filters. I discuss various time, trend and volatility filters here.

I suggest adding 1-2 filters at most, and if the system still fails to meet your expectations, it’s better to move on to another idea.

Continuing to modify a system with the express intent of improving your metrics can easily lead to overfitting. Systems with spectacular backtests usually deteriorate drastically in real-time.

Wrapping Up

The SQN is an innovative measure of your system’s profitability and consistency. It also tends to favour large sample sizes, which bodes well for your backtest reliability.

You can obtain it with some manipulation of your MT4 backtest report, but I suggest opening your report in QuantAnalyzer and having it calculated automatically.

Personally, I do not use the SQN because of the limitations explained above. Nonetheless, it can be useful when comparing systems with similar trading frequency. Alternatively, you can remove its reliance on sample size and simply compare the ratio of expectancy over standard deviation.

Like any other metric, it’s only meaningful if your system actually trades like it backtests. Try to backtest your system over different market conditions and if possible, apply the SQN to out-of-sample results.

If you’d like to learn more about the SQN, I recommend reading Van Tharp’s Definitive Guide to Position Sizing.

Are you a fan of Van Tharp’s SQN? Let me know in the comments!

Powered By

Development Platform

Forex VPS

FXVM Forex VPS

Popular Posts

Laguerre RSI Trend Following Strategy

The Laguerre RSI attempts to improve the responsiveness of the regular RSI, whilst keeping whipsaw trades to a minimum. Let’s see how well it detects short-term pullbacks for a trend following strategy!

read more

What is Fixed Ratio Money Management?

Have you heard of fixed ratio money management? How does it compare to the popular fixed fractional approach? Here I’ll explain how fixed ratio works, and see how it stacks up against fixed fractional money management.

read more

Build a Diversified Portfolio With QuantAnalyzer

The ability to efficiently trade a diversified portfolio of strategies is one of the biggest advantages of algorithmic trading. Here we will use QuantAnalyzer’s Portfolio Master to build a portfolio consisting of high performing, uncorrelated strategies.

read more

What Is the QQE Indicator?

The QQE is a mysterious indicator that sometimes pops up in trading forums. Does it deserve a place alongside the more traditional momentum indicators like the RSI and CCI? Let’s add it to a trend following strategy to find out!

read more

Make your money work for you!

Get promotions, trading ideas and strategy development tips delivered to your inbox!

Comments

4 Comments

  1. Massimo

    Am I wrong saying that if R is constant SQN result is independent from R ? (Found applying the formula to whaterever list of trades, but it is obvious..)
    If I am correct, how can it be a correct way to eveluate a strategy ?

    Reply
    • Wayne

      Hi Massimo, if you risk the same amount for each trade, R will be constant. But the trade outcome (R-multiple) will be different. This means different R-expectancy and standard deviation values, and a different SQN.

      Reply
  2. Mike

    I am a bit confused.
    I think the Strategy Quality Number is represented as “SQN” in StrategyQuant right?
    Not “STR Quality Number”

    Reply
    • Wayne

      they should be the same thing

      Reply

Submit a Comment

Your email address will not be published. Required fields are marked *

Trading Strategies

What’s the Best Time to Trade Forex?

What’s the Best Time to Trade Forex?

The forex markets are open 24/5, but not all hours are created equal. Here I dissect my broker data to determine the best time to trade forex.

Forex Weekend Gaps: Can You Exploit Them?

Forex Weekend Gaps: Can You Exploit Them?

Have you noticed that forex weekend gaps usually reverse within 3 days? Here I’ll program a mean reversion strategy to exploit gaps over the last 18 years!

Money Flow Index: An Improved RSI?

Money Flow Index: An Improved RSI?

The Money Flow Index is sometimes called the volume-weighted RSI. Can it outperform the RSI in this trend following strategy?

Automated Bollinger Bands Squeeze Forex Strategy

Automated Bollinger Bands Squeeze Forex Strategy

StrategyQuant’s BBWR indicator is the perfect tool to detect a Bollinger Bands squeeze. Here I explain how it’s calculated, and use it to program a breakout strategy for the AUDJPY!

Should You Use the Kelly Criterion for Forex Trading?

Should You Use the Kelly Criterion for Forex Trading?

The Kelly criterion is a famous mathematical formula that attempts to maximize your long-term capital growth. In this post, I’ll apply it to a EURUSD breakout strategy and explain some of its potential shortcomings when applied to forex trading.

Can a Trading Pause Improve Your Trend Following Results?

Can a Trading Pause Improve Your Trend Following Results?

A temporary trading pause can improve your win rate if you’re trend following a volatile market. Here I’ll program a trading pause into a simple breakout strategy, and test its effectiveness on the Widow Maker – the GBPJPY.

Laguerre RSI Trend Following Strategy

Laguerre RSI Trend Following Strategy

The Laguerre RSI attempts to improve the responsiveness of the regular RSI, whilst keeping whipsaw trades to a minimum. Let’s see how well it detects short-term pullbacks for a trend following strategy!

How to Use the Supertrend Indicator

How to Use the Supertrend Indicator

Despite its cool name, the Supertrend indicator often seems to slip under the radar. Here I explain how it’s calculated, and combine it with moving averages to produce a simple trend following strategy.

Strategy Development

How to Get a Realistic Backtest Spread

How to Get a Realistic Backtest Spread

Your choice of backtest spread can certainly make or break a strategy. This post will show you how to study the intraday spread variations of your market, and suggest several ways to avoid paying ridiculous spreads.

Do You Know Your Strategy’s Optimization Profile?

Do You Know Your Strategy’s Optimization Profile?

Your strategy’s optimization profile often reveals its robustness, helping you select strategies that will remain profitable in live trading. Here I explain why an optimization profile is important, and how you can easily obtain one using StrategyQuant’s optimizer.

Which MT4 Backtest Report Metrics Should You Use?

Which MT4 Backtest Report Metrics Should You Use?

Understanding your backtest report is an essential part of being a successful strategy developer. Here I explain what the numbers mean, and how you can make use of each metric during strategy development.

Out-of-sample Testing Using Monte Carlo Simulations

Out-of-sample Testing Using Monte Carlo Simulations

Traders often use Monte Carlo simulations to estimate worst-case drawdowns, but did you know they can be used for out-of-sample testing too? This post demonstrates the use of StrategyQuant’s Monte Carlo simulator to randomize historical prices and strategy parameters, helping you select robust strategies for live trading.

How Many Trades Should Your Backtest Have?

How Many Trades Should Your Backtest Have?

We all want a large sample of trades in our backtests, but practical limitations such as data availability often get in the way. Here I’ll explain why 30 trades is insufficient, and how you can use standard error to quantify the uncertainty arising from a small sample size.

Build a Diversified Portfolio With QuantAnalyzer

Build a Diversified Portfolio With QuantAnalyzer

The ability to efficiently trade a diversified portfolio of strategies is one of the biggest advantages of algorithmic trading. Here we will use QuantAnalyzer’s Portfolio Master to build a portfolio consisting of high performing, uncorrelated strategies.

Strategy Optimization Using MT4

Strategy Optimization Using MT4

How do you improve your trading strategy in MT4? This post will show you how to optimize the entry and exit parameters for a moving average crossover strategy. Finally, an intraday time filter will be added to help avoid false breakouts.

Debugging & Backtesting Using MT4

Debugging & Backtesting Using MT4

With a fresh algorithm at your fingertips, how do you verify that it has been programmed correctly? This guide will show you how to use Metatrader 4’s visual backtester to debug and backtest your strategy.

Create Your Trading Algorithm in 15 Minutes (FREE)

Create Your Trading Algorithm in 15 Minutes (FREE)

Converting your trading idea into an algorithm is the first step towards reaping the benefits of automated trading. This guide will cover the creation of a simple moving average crossover algorithm, without any actual programming.

What Is Drawdown in Trading?

What Is Drawdown in Trading?

Are you getting a comprehensive assessment of your strategy’s downside? This post will discuss several methods to measure drawdowns, helping you build and select strategies that better suit your risk appetite.

How to Select the Best Trading Strategy Entry

How to Select the Best Trading Strategy Entry

With an abundance of technical indicators available, selecting your strategy’s entry conditions can be overwhelming. This post will illustrate a method to graphically compare the profitability of different entries.

Live Trading

What’s the Best Time to Trade Forex?

What’s the Best Time to Trade Forex?

The forex markets are open 24/5, but not all hours are created equal. Here I dissect my broker data to determine the best time to trade forex.

How to Find a Real Trading Guru

How to Find a Real Trading Guru

Every day I come across a trading guru offering educational content on the internet. Many of them speak of huge returns with minimal effort. Should these be trusted? Here’s some tips on how to separate the wheat from the chaff.

How to Enjoy Stress-Free Trading

How to Enjoy Stress-Free Trading

Trading is a great way to make some additional income, but not if you’re constantly pulling your hair out. Here I offer 7 tips to help make your trading profitable and stress-free.

How to Select the Best Forex VPS

How to Select the Best Forex VPS

A virtual private server (VPS) is a virtual computer that you can rent and access remotely. It provides a reliable platform on which to execute your forex strategies. This post will help you decide whether you need a VPS, and show you how to select an optimal VPS.

Make your money work for you!

Make your money work for you!

 

Get trading ideas and strategy development tips delivered to your inbox!

Thanks for subscribing!

Pin It on Pinterest

Share This