Do You Know Your Strategy’s Optimization Profile?

Apr 25, 2021

Your strategy’s optimization profile often reveals its robustness, helping you select strategies that will remain profitable in live trading.

Here I explain why an optimization profile is important, and how you can easily obtain one using StrategyQuant’s optimizer.

Here’s a familiar scenario:

After some hard work, you finally have a profitable backtest.

You know that for a certain combination of parameters, your strategy has been profitable over the historical period.

But if market conditions change, or you change your input parameters, will the strategy remain profitable?

Your strategy’s optimization profile can offer some answers.

Let’s dive into it.

What is a Strategy’s Optimization Profile?

An optimization profile simulates your strategy’s performance over the entire parameter optimization range.

Each data point in your optimization profile is the backtest result for that particular combination of parameters.

Let’s say you have a simple Donchian channel strategy where you:

  • Enter the market on a channel breakout
  • Hold the trade for a certain number of bars

You would have two optimizable parameters in this case – the Donchian channel lookback period and the number of bars in the market.

If you simultaneously optimize each parameter in the 10-100 range, you will get the following optimization profile.

The optimizable parameters are plotted on the horizontal axes, while the performance metric is on the vertical axis.

Why is the Optimization Profile Important?

The optimization profile reveals your strategy’s parameter stability. In other words, can it remain profitable when your parameters change?

Robust strategies remain profitable over a broad range of parameters. Such strategies are resilient to changing market conditions, and are more likely to remain profitable in live trading.

Ideally, you want to see an optimization profile with a high plateau, indicating stable performance across a wide range of parameters.

Conversely, an overfit strategy will only be profitable in a narrow parameter range.

Overfit strategies are constrained to random noise, which has no predictive value. Your historical backtests will be overly optimistic in this case.

In the next section, I’ll show you how to battle overfitting by quantifying your strategy’s parameter stability.

Plotting Your Strategy’s Optimization Profile in StrategyQuant

To demonstrate the use of StrategyQuant’s optimizer, I’ll use a GBPJPY trend following strategy that was automatically generated using genetic evolution.

You can download the complete strategy here.

The strategy captures breakouts by placing stop entries at the previous week’s opening price. An ATR-based stop loss and bar-based time stop are used for trade management.

There are only two optimizable parameters:

  • StopLossCoef: ATR multiple for the stop loss
  • ExitAfterBars: Number of bars used for the time stop

After backtesting from 2003-2020, the optimal values for StopLossCoef and ExitAfterBars were 3.1 and 154, respectively. This produced the following results:

Was this just a lucky combination of parameters? The optimization profile will tell us.

I loaded the strategy into StrategyQuant’s Optimizer module and configured the optimization range as shown:

What parameter ranges should you use?

Try to select values that are aligned with the premise of your strategy.

For example, if you’re a day trader, having a >100 lookback period for your indicator probably won’t make sense. Likewise, for trend following strategies where you should cut losses quickly, a stop loss over 5 ATR is excessive. 

With the optimization range selected above, the optimization profile will consist of 5986 different backtest runs. 

Let’s first visually examine the optimization profile.

And a top view for clarity:

The large stable region gives me confidence that the strategy has not been overfit.

It makes sense that the most profitable region is where the stop losses are small, and the time stops are large. You want to cut your losses quickly, while letting your profits run.

For simplicity, net profit was used as the performance metric. You can select other popular metrics like the profit factor or return/drawdown.

How about some quantitative measures?

Percentage of Profitable Optimizations

This is an important metric for parameter stability. I want at least 70% of my optimizations to be profitable.

Of the 5986 backtests, only 12 were losing, so I’m good here.

Average Profit

This is the average profit across all the optimization runs. If you want a metric that considers profitability together with robustness, this is a good choice.

At the minimum, you average profit should be positive.

I’ve got a $1000 average profit here, which translates to about 14000 pips over the 17-year backtest.

Uniformity of Profit Distribution

This counts how many times your profit flipped from positive to negative, or negative to positive.

For example, in the distribution below, this happened 3 times.

You want a small ‘flip count’, indicating your parameters are stable.

This is related to the percentage of profitable optimizations, although the same percentage can produce distributions of different uniformities.

Both profit distributions #1 and #2 above have 14 profitable runs.

But distribution #2 has 11 flips, while distribution #1 has only 1.

You want to see consistent and predictable patterns in your profit distribution. For example, having very small stop losses or time stops will result in overall losses. Profits will gradually increase as these parameters get larger.

If you get something erratic like distribution #2, your strategy is likely not robust enough.

For the strategy I’m testing now, there were 17 flips. This is a good result considering there were almost 6000 runs.

Spread of Profit Distribution

The performance of the top parameter set should be close to the average across the entire optimization space.

In StrategyQuant, the standard deviation of the profit is used to determine the spread of your profit distribution.

Ideally, I want my highest profit to be within two standard deviations of the average.

Here my highest profit of $1880 exceeds the two standard deviation limit of $1800. I consider this only a minor setback, especially since the rest of the results are good.

Plotting Your Optimization Profile Using MT4

If you use MT4 for strategy development, you can use its Strategy Tester for parameter optimizations.

The default optimization chart generated by MT4 is two-dimensional though.

You can export your results to Excel and create an array, with each cell corresponding to one combination of parameters.

You can then plot three-dimensional contours using the ‘3-D Surface’ option. Check out my Bollinger Bands vs. Keltner Channels comparison if you need a reference.

Tips to Prevent Overfitting

In an attempt to create a fantastic looking backtest, it is easy to keep adding/tweaking your strategy rules.

Overfitting is the result of excessive tweaking, whereby your strategy is designed to exploit random noise in your historical data.

When market conditions or input parameters change, you can expect strategy performance to suffer greatly.

You can minimize overfitting through the following measures:

  • Minimize the number of strategy rules

As the saying goes, ‘loose pants fit all.’ Having fewer rules ‘loosens the fit’ of your strategy to the market, and makes it more resilient in changing markets.

The strategy illustrated above used a simple weekly breakout entry condition, and had only two optimizable parameters.

Most authors agree that it’s best to have a maximum of 4 optimizable parameters.

  • Avoid simultaneous parameter optimizations

When constructing the optimization profiles above, simultaneous optimizations were used to quantify strategy performance over a broad parameter range.

But if you want to select optimal parameters for trading, try to optimize one parameter at a time. This makes it less likely to arrive at a ‘lucky’ parameter set that maximizes the fit between your strategy and historical data.

For example, optimize your entry lookback period first, then your stop loss size.

  • Lengthen your backtest period/Test on multiple markets

This will make your strategy more adaptable to different market conditions.

It’s better to have a ‘jack of all trades’ strategy since you never know what the future holds.

Trading a diverse portfolio of loosely fit, good-performing strategies is far better than trading a handful of highly optimized strategies.

Wrapping Up

I’m a strong believer of robustness testing during strategy development. These help you weed out overfit strategies before you put real money on the line.

Plotting your strategy’s optimization profile is a quick way to evaluate your strategy’s robustness.

You want a large percentage of profitable optimizations, together with a high plateau on your optimization profile.

Other popular robustness tests include walk-forward optimization and Monte Carlo simulations, although these tend to be more computationally demanding.

Do you find the optimization profile useful? Let me know in the comments!

    Powered By

    Development Platform

    Forex VPS

    FXVM Forex VPS

    Popular Posts

    Laguerre RSI Trend Following Strategy

    The Laguerre RSI attempts to improve the responsiveness of the regular RSI, whilst keeping whipsaw trades to a minimum. Let’s see how well it detects short-term pullbacks for a trend following strategy!

    read more

    What is Fixed Ratio Money Management?

    Have you heard of fixed ratio money management? How does it compare to the popular fixed fractional approach? Here I’ll explain how fixed ratio works, and see how it stacks up against fixed fractional money management.

    read more

    Build a Diversified Portfolio With QuantAnalyzer

    The ability to efficiently trade a diversified portfolio of strategies is one of the biggest advantages of algorithmic trading. Here we will use QuantAnalyzer’s Portfolio Master to build a portfolio consisting of high performing, uncorrelated strategies.

    read more

    What Is the QQE Indicator?

    The QQE is a mysterious indicator that sometimes pops up in trading forums. Does it deserve a place alongside the more traditional momentum indicators like the RSI and CCI? Let’s add it to a trend following strategy to find out!

    read more

    Make your money work for you!

    Get promotions, trading ideas and strategy development tips delivered to your inbox!

    Comments

    5 Comments

    1. Mike

      If you have 3 optimizable parameters vs. Net profit, can we still see the optimization profile?

      Reply
    2. Mike2

      If you have 3 optimizable parameters vs. Net profit, can we still see the optimization profile?

      Reply
    3. Mike

      I don’t understand how “Avoid simultaneous parameter optimizations” can avoid overfitting. Could you please elaborate? Parameter optimization profile will run the parameters sequentially. There is no difference between “simultaneous optimization” and “optimize one parameters at a time”, in essence both methods will fix (N-1)parameters and run one specific parameter until the iteraction ends.

      Reply
    4. Mike

      I think one good point to add in for avoding overfitting is “Don’t set unrealistically high performance metrics during strategy development”. This will force the generated strategies fit in the historical patterns, i.e., overfitting.

      Reply
    5. Mike3

      I think one good point to add in for avoding overfitting is “Don’t set unrealistically high performance metrics during strategy development”. This will force the generated strategies fit in the historical patterns, i.e., overfitting.

      Reply

    Submit a Comment

    Your email address will not be published. Required fields are marked *

    Trading Strategies

    What’s the Best Time to Trade Forex?

    What’s the Best Time to Trade Forex?

    The forex markets are open 24/5, but not all hours are created equal. Here I dissect my broker data to determine the best time to trade forex.

    Forex Weekend Gaps: Can You Exploit Them?

    Forex Weekend Gaps: Can You Exploit Them?

    Have you noticed that forex weekend gaps usually reverse within 3 days? Here I’ll program a mean reversion strategy to exploit gaps over the last 18 years!

    Money Flow Index: An Improved RSI?

    Money Flow Index: An Improved RSI?

    The Money Flow Index is sometimes called the volume-weighted RSI. Can it outperform the RSI in this trend following strategy?

    Automated Bollinger Bands Squeeze Forex Strategy

    Automated Bollinger Bands Squeeze Forex Strategy

    StrategyQuant’s BBWR indicator is the perfect tool to detect a Bollinger Bands squeeze. Here I explain how it’s calculated, and use it to program a breakout strategy for the AUDJPY!

    Should You Use the Kelly Criterion for Forex Trading?

    Should You Use the Kelly Criterion for Forex Trading?

    The Kelly criterion is a famous mathematical formula that attempts to maximize your long-term capital growth. In this post, I’ll apply it to a EURUSD breakout strategy and explain some of its potential shortcomings when applied to forex trading.

    Can a Trading Pause Improve Your Trend Following Results?

    Can a Trading Pause Improve Your Trend Following Results?

    A temporary trading pause can improve your win rate if you’re trend following a volatile market. Here I’ll program a trading pause into a simple breakout strategy, and test its effectiveness on the Widow Maker – the GBPJPY.

    Laguerre RSI Trend Following Strategy

    Laguerre RSI Trend Following Strategy

    The Laguerre RSI attempts to improve the responsiveness of the regular RSI, whilst keeping whipsaw trades to a minimum. Let’s see how well it detects short-term pullbacks for a trend following strategy!

    How to Use the Supertrend Indicator

    How to Use the Supertrend Indicator

    Despite its cool name, the Supertrend indicator often seems to slip under the radar. Here I explain how it’s calculated, and combine it with moving averages to produce a simple trend following strategy.

    Strategy Development

    Do You Know Your System Quality Number?

    Do You Know Your System Quality Number?

    The System Quality Number measures the profitability & consistency of your trading system. Here’s how to calculate your SQN and use it to improve your trading!

    How to Get a Realistic Backtest Spread

    How to Get a Realistic Backtest Spread

    Your choice of backtest spread can certainly make or break a strategy. This post will show you how to study the intraday spread variations of your market, and suggest several ways to avoid paying ridiculous spreads.

    Which MT4 Backtest Report Metrics Should You Use?

    Which MT4 Backtest Report Metrics Should You Use?

    Understanding your backtest report is an essential part of being a successful strategy developer. Here I explain what the numbers mean, and how you can make use of each metric during strategy development.

    Out-of-sample Testing Using Monte Carlo Simulations

    Out-of-sample Testing Using Monte Carlo Simulations

    Traders often use Monte Carlo simulations to estimate worst-case drawdowns, but did you know they can be used for out-of-sample testing too? This post demonstrates the use of StrategyQuant’s Monte Carlo simulator to randomize historical prices and strategy parameters, helping you select robust strategies for live trading.

    How Many Trades Should Your Backtest Have?

    How Many Trades Should Your Backtest Have?

    We all want a large sample of trades in our backtests, but practical limitations such as data availability often get in the way. Here I’ll explain why 30 trades is insufficient, and how you can use standard error to quantify the uncertainty arising from a small sample size.

    Build a Diversified Portfolio With QuantAnalyzer

    Build a Diversified Portfolio With QuantAnalyzer

    The ability to efficiently trade a diversified portfolio of strategies is one of the biggest advantages of algorithmic trading. Here we will use QuantAnalyzer’s Portfolio Master to build a portfolio consisting of high performing, uncorrelated strategies.

    Strategy Optimization Using MT4

    Strategy Optimization Using MT4

    How do you improve your trading strategy in MT4? This post will show you how to optimize the entry and exit parameters for a moving average crossover strategy. Finally, an intraday time filter will be added to help avoid false breakouts.

    Debugging & Backtesting Using MT4

    Debugging & Backtesting Using MT4

    With a fresh algorithm at your fingertips, how do you verify that it has been programmed correctly? This guide will show you how to use Metatrader 4’s visual backtester to debug and backtest your strategy.

    Create Your Trading Algorithm in 15 Minutes (FREE)

    Create Your Trading Algorithm in 15 Minutes (FREE)

    Converting your trading idea into an algorithm is the first step towards reaping the benefits of automated trading. This guide will cover the creation of a simple moving average crossover algorithm, without any actual programming.

    What Is Drawdown in Trading?

    What Is Drawdown in Trading?

    Are you getting a comprehensive assessment of your strategy’s downside? This post will discuss several methods to measure drawdowns, helping you build and select strategies that better suit your risk appetite.

    How to Select the Best Trading Strategy Entry

    How to Select the Best Trading Strategy Entry

    With an abundance of technical indicators available, selecting your strategy’s entry conditions can be overwhelming. This post will illustrate a method to graphically compare the profitability of different entries.

    Live Trading

    What’s the Best Time to Trade Forex?

    What’s the Best Time to Trade Forex?

    The forex markets are open 24/5, but not all hours are created equal. Here I dissect my broker data to determine the best time to trade forex.

    How to Find a Real Trading Guru

    How to Find a Real Trading Guru

    Every day I come across a trading guru offering educational content on the internet. Many of them speak of huge returns with minimal effort. Should these be trusted? Here’s some tips on how to separate the wheat from the chaff.

    How to Enjoy Stress-Free Trading

    How to Enjoy Stress-Free Trading

    Trading is a great way to make some additional income, but not if you’re constantly pulling your hair out. Here I offer 7 tips to help make your trading profitable and stress-free.

    How to Select the Best Forex VPS

    How to Select the Best Forex VPS

    A virtual private server (VPS) is a virtual computer that you can rent and access remotely. It provides a reliable platform on which to execute your forex strategies. This post will help you decide whether you need a VPS, and show you how to select an optimal VPS.

    Make your money work for you!

    Make your money work for you!

     

    Get trading ideas and strategy development tips delivered to your inbox!

    Thanks for subscribing!

    Pin It on Pinterest

    Share This