Do You Know Your Strategy’s Optimization Profile?

Apr 25, 2021

Your strategy’s optimization profile often reveals its robustness, helping you select strategies that will remain profitable in live trading.

Here I explain why an optimization profile is important, and how you can easily obtain one using StrategyQuant’s optimizer.

Here’s a familiar scenario:

After some hard work, you finally have a profitable backtest.

You know that for a certain combination of parameters, your strategy has been profitable over the historical period.

But if market conditions change, or you change your input parameters, will the strategy remain profitable?

Your strategy’s optimization profile can offer some answers.

Let’s dive into it.

What is a Strategy’s Optimization Profile?

An optimization profile simulates your strategy’s performance over the entire parameter optimization range.

Each data point in your optimization profile is the backtest result for that particular combination of parameters.

Let’s say you have a simple Donchian channel strategy where you:

  • Enter the market on a channel breakout
  • Hold the trade for a certain number of bars

You would have two optimizable parameters in this case – the Donchian channel lookback period and the number of bars in the market.

If you simultaneously optimize each parameters in the 10-100 range, you will get the following optimization profile.

The optimizable parameters are plotted on the horizontal axes, while the performance metric is on the vertical axis.

Why is the Optimization Profile Important?

The optimization profile reveals your strategy’s parameter stability. In other words, can it remain profitable when your parameters change?

Robust strategies remain profitable over a broad range of parameters. Such strategies are resilient to changing market conditions, and are more likely to remain profitable in live trading.

Ideally, you want to see an optimization profile with a high plateau, indicating stable performance across a wide range of parameters.

Conversely, an overfit strategy will only be profitable in a narrow parameter range.

Overfit strategies are constrained to random noise, which has no predictive value. Your historical backtests will be overly optimistic in this case.

In the next section, I’ll show you how to battle overfitting by quantifying your strategy’s parameter stability.

Plotting Your Strategy’s Optimization Profile in StrategyQuant

To demonstrate the use of StrategyQuant’s optimizer, I’ll use a GBPJPY trend following strategy that was automatically generated using genetic evolution.

You can download the complete strategy here.

The strategy captures breakouts by placing stop entries at the previous week’s opening price. An ATR-based stop loss and bar-based time stop are used for trade management.

There are only two optimizable parameters:

  • StopLossCoef: ATR multiple for the stop loss
  • ExitAfterBars: Number of bars used for the time stop

After backtesting from 2003-2020, the optimal values for StopLossCoef and ExitAfterBars were 3.1 and 154, respectively. This produced the following results:

Was this just a lucky combination of parameters? The optimization profile will tell us.

I loaded the strategy into StrategyQuant’s Optimizer module and configured the optimization range as shown:

What parameter ranges should you use?

Try to select values that are aligned with the premise of your strategy.

For example, if you’re a day trader, having a >100 lookback period for your indicator probably won’t make sense. Likewise, for trend following strategies where you should cut losses quickly, a stop loss over 5 ATR is excessive. 

With the optimization range selected above, the optimization profile will consist of 5986 different backtest runs. 

Let’s first visually examine the optimization profile.

And a top view for clarity:

The large stable region gives me confidence that the strategy has not been overfit.

It makes sense that the most profitable region is where the stop losses are small, and the time stops are large. You want to cut your losses quickly, while letting your profits run.

For simplicity, net profit was used as the performance metric. You can select other popular metrics like the profit factor or return/drawdown.

How about some quantitative measures?

Percentage of Profitable Optimizations

This is an important metric for parameter stability. I want at least 70% of my optimizations to be profitable.

Of the 5986 backtests, only 12 were losing, so I’m good here.

Average Profit

This is the average profit across all the optimization runs. If you want a metric that considers profitability together with robustness, this is a good choice.

At the minimum, you average profit should be positive.

I’ve got a $1000 average profit here, which translates to about 14000 pips over the 17-year backtest.

Uniformity of Profit Distribution

This counts how many times your profit flipped from positive to negative, or negative to positive.

For example, in the distribution below, this happened 3 times.

You want a small ‘flip count’, indicating your parameters are stable.

This is related to the percentage of profitable optimizations, although the same percentage can produce distributions of different uniformities.

Both profit distributions #1 and #2 above have 14 profitable runs.

But distribution #2 has 11 flips, while distribution #1 has only 1.

You want to see consistent and predictable patterns in your profit distribution. For example, having very small stop losses or time stops will result in overall losses. Profits will gradually increase as these parameters get larger.

If you get something erratic like distribution #2, your strategy is likely not robust enough.

For the strategy I’m testing now, there were 17 flips. This is a good result considering there were almost 6000 runs.

Spread of Profit Distribution

The performance of the top parameter set should be close to the average across the entire optimization space.

In StrategyQuant, the standard deviation of the profit is used to determine the spread of your profit distribution.

Ideally, I want my highest profit to be within two standard deviations of the average.

Here my highest profit of $1880 exceeds the two standard deviation limit of $1800. I consider this only a minor setback, especially since the rest of the results are good.

Plotting Your Optimization Profile Using MT4

If you use MT4 for strategy development, you can use its Strategy Tester for parameter optimizations.

The default optimization chart generated by MT4 is two-dimensional though.

You can export your results to Excel and create an array, with each cell corresponding to one combination of parameters.

You can then plot three-dimensional contours using the ‘3-D Surface’ option. Check out my Bollinger Bands vs. Keltner Channels comparison if you need a reference.

Tips to Prevent Overfitting

In an attempt to create a fantastic looking backtest, it is easy to keep adding/tweaking your strategy rules.

Overfitting is the result of excessive tweaking, whereby your strategy is designed to exploit random noise in your historical data.

When market conditions or input parameters change, you can expect strategy performance to suffer greatly.

You can minimize overfitting through the following measures:

  • Minimize the number of strategy rules

As the saying goes, ‘loose pants fit all.’ Having fewer rules ‘loosens the fit’ of your strategy to the market, and makes it more resilient in changing markets.

The strategy illustrated above used a simple weekly breakout entry condition, and had only two optimizable parameters.

Most authors agree that it’s best to have a maximum of 4 optimizable parameters.

  • Avoid simultaneous parameter optimizations

When constructing the optimization profiles above, simultaneous optimizations were used to quantify strategy performance over a broad parameter range.

But if you want to select optimal parameters for trading, try to optimize one parameter at a time. This makes it less likely to arrive at a ‘lucky’ parameter set that maximizes the fit between your strategy and historical data.

For example, optimize your entry lookback period first, then your stop loss size.

  • Lengthen your backtest period/Test on multiple markets

This will make your strategy more adaptable to different market conditions.

It’s better to have a ‘jack of all trades’ strategy since you never know what the future holds.

Trading a diverse portfolio of loosely fit, good-performing strategies is far better than trading a handful of highly optimized strategies.

Wrapping Up

I’m a strong believer of robustness testing during strategy development. These help you weed out overfit strategies before you put real money on the line.

Plotting your strategy’s optimization profile is a quick way to evaluate your strategy’s robustness.

You want a large percentage of profitable optimizations, together with a high plateau on your optimization profile.

Other popular robustness tests include walk-forward optimization and Monte Carlo simulations, although these tend to be more computationally demanding.

Do you find the optimization profile useful? Let me know in the comments!

    Want to develop a portfolio of automated trading strategies?

    Supercharge your strategy development with StrategyQuant

    Access 14-day FREE trial here!

    Get up to USD 300 discount!

    Strategies need improvement?

    Use QuantAnalyzer’s powerful analysis tools

    Try the FREE version here!

    Get 20% Discount here!

    FXVM Forex VPS
    Follow Us

    Popular Posts

    Laguerre RSI Trend Following Strategy

    The Laguerre RSI attempts to improve the responsiveness of the regular RSI, whilst keeping whipsaw trades to a minimum. Let’s see how well it detects short-term pullbacks for a trend following strategy!

    read more

    What is Fixed Ratio Money Management?

    Have you heard of fixed ratio money management? How does it compare to the popular fixed fractional approach? Here I’ll explain how fixed ratio works, and see how it stacks up against fixed fractional money management.

    read more

    Build a Diversified Portfolio With QuantAnalyzer

    The ability to efficiently trade a diversified portfolio of strategies is one of the biggest advantages of algorithmic trading. Here we will use QuantAnalyzer’s Portfolio Master to build a portfolio consisting of high performing, uncorrelated strategies.

    read more

    What Is the QQE Indicator?

    The QQE is a mysterious indicator that sometimes pops up in trading forums. Does it deserve a place alongside the more traditional momentum indicators like the RSI and CCI? Let’s add it to a trend following strategy to find out!

    read more

    Hey there, Wayne here! I’m on a mission to develop robust algorithmic trading strategies for the forex markets. Trading Tact is where I share my trading methods and insights.

    Follow Us
    Have a Question?

    Comments

    0 Comments

    Submit a Comment

    Your email address will not be published.

    Want to develop a portfolio of automated trading strategies?

    Supercharge your strategy development with StrategyQuant

    Access 14-day FREE trial here!

    Get up to USD 300 discount!

    Strategies need improvement?

    Use QuantAnalyzer’s powerful analysis tools

    Try the FREE version here!

    Get 20% Discount here!

    FXVM Forex VPS

    Recent Posts

    How to Find a Real Trading Guru

    How to Find a Real Trading Guru

    Every day I come across a trading guru offering educational content on the internet. Many of them speak of huge returns with minimal effort. Should these be trusted? Here’s some tips on how to separate the wheat from the chaff.

    read more
    How to Enjoy Stress-Free Trading

    How to Enjoy Stress-Free Trading

    Trading is a great way to make some additional income, but not if you’re constantly pulling your hair out. Here I offer 7 tips to help make your trading profitable and stress-free.

    read more
    How to Get a Realistic Backtest Spread

    How to Get a Realistic Backtest Spread

    Your choice of backtest spread can certainly make or break a strategy. This post will show you how to study the intraday spread variations of your market, and suggest several ways to avoid paying ridiculous spreads.

    read more
    Laguerre RSI Trend Following Strategy

    Laguerre RSI Trend Following Strategy

    The Laguerre RSI attempts to improve the responsiveness of the regular RSI, whilst keeping whipsaw trades to a minimum. Let’s see how well it detects short-term pullbacks for a trend following strategy!

    read more
    How to Use the Supertrend Indicator

    How to Use the Supertrend Indicator

    Despite its cool name, the Supertrend indicator often seems to slip under the radar. Here I explain how it’s calculated, and combine it with moving averages to produce a simple trend following strategy.

    read more
    How to Select the Best Forex VPS

    How to Select the Best Forex VPS

    A virtual private server (VPS) is a virtual computer that you can rent and access remotely. It provides a reliable platform on which to execute your forex strategies. This post will help you decide whether you need a VPS, and show you how to select an optimal VPS.

    read more
    Out-of-sample Testing Using Monte Carlo Simulations

    Out-of-sample Testing Using Monte Carlo Simulations

    Traders often use Monte Carlo simulations to estimate worst-case drawdowns, but did you know they can be used for out-of-sample testing too? This post demonstrates the use of StrategyQuant’s Monte Carlo simulator to randomize historical prices and strategy parameters, helping you select robust strategies for live trading.

    read more

    Hey there, Wayne here! I’m on a mission to develop robust algorithmic trading strategies for the forex markets. Trading Tact is where I share my trading methods and insights.

    Make your money work for you!

    Make your money work for you!

     

    Get trading ideas and strategy development tips delivered to your inbox!

    Thanks for subscribing!

    Make your money work for you!

    Make your money work for you!

     

    Get trading ideas and strategy development tips delivered to your inbox!

    Thanks for subscribing!

    Make your money work for you!

    Make your money work for you!

     

    Get trading ideas and strategy development tips delivered to your inbox!

    Thanks for subscribing!

    Make your money work for you!

    Make your money work for you!

     

    Get trading ideas and strategy development tips delivered to your inbox!

    Thanks for subscribing!

    Make your money work for you!

    Make your money work for you!

     

    Get trading ideas and strategy development tips delivered to your inbox!

    Thanks for subscribing!

    Make your money work for you!

    Make your money work for you!

     

    Get trading ideas and strategy development tips delivered to your inbox!

    Thanks for subscribing!

    Make your money work for you!

    Make your money work for you!

     

    Get trading ideas and strategy development tips delivered to your inbox!

    Thanks for subscribing!

    Make your money work for you!

    Make your money work for you!

     

    Get trading ideas and strategy development tips delivered to your inbox!

    Thanks for subscribing!

    Make your money work for you!

    Make your money work for you!

     

    Get trading ideas and strategy development tips delivered to your inbox!

    Thanks for subscribing!

    Make your money work for you!

    Make your money work for you!

     

    Get trading ideas and strategy development tips delivered to your inbox!

    Thanks for subscribing!

    Make your money work for you!

    Make your money work for you!

     

    Get trading ideas and strategy development tips delivered to your inbox!

    Thanks for subscribing!

    Make your money work for you!

    Make your money work for you!

     

    Get trading ideas and strategy development tips delivered to your inbox!

    Thanks for subscribing!

    Make your money work for you!

    Make your money work for you!

     

    Get trading ideas and strategy development tips delivered to your inbox!

    Thanks for subscribing!

    Make your money work for you!

    Make your money work for you!

     

    Get trading ideas and strategy development tips delivered to your inbox!

    Thanks for subscribing!

    Make your money work for you!

    Make your money work for you!

     

    Get trading ideas and strategy development tips delivered to your inbox!

    Thanks for subscribing!

    Make your money work for you!

    Make your money work for you!

     

    Get trading ideas and strategy development tips delivered to your inbox!

    Thanks for subscribing!

    Make your money work for you!

    Make your money work for you!

     

    Get trading ideas and strategy development tips delivered to your inbox!

    Thanks for subscribing!

    Make your money work for you!

    Make your money work for you!

     

    Get trading ideas and strategy development tips delivered to your inbox!

    Thanks for subscribing!

    Make your money work for you!

    Make your money work for you!

     

    Get trading ideas and strategy development tips delivered to your inbox!

    Thanks for subscribing!

    Make your money work for you!

    Make your money work for you!

     

    Get trading ideas and strategy development tips delivered to your inbox!

    Thanks for subscribing!

    Make your money work for you!

    Make your money work for you!

     

    Get trading ideas and strategy development tips delivered to your inbox!

    Thanks for subscribing!

    Make your money work for you!

    Make your money work for you!

     

    Get trading ideas and strategy development tips delivered to your inbox!

    Thanks for subscribing!

    Make your money work for you!

    Make your money work for you!

     

    Get trading ideas and strategy development tips delivered to your inbox!

    Thanks for subscribing!

    Make your money work for you!

    Make your money work for you!

     

    Get trading ideas and strategy development tips delivered to your inbox!

    Thanks for subscribing!

    Pin It on Pinterest

    Share This