Your strategy’s optimization profile often reveals its robustness, helping you select strategies that will remain profitable in live trading.

Here I explain why an optimization profile is important, and how you can easily obtain one using StrategyQuant’s optimizer.

Here’s a familiar scenario:

After some hard work, you finally have a profitable backtest.

You know that for a certain combination of parameters, your strategy has been profitable over the historical period.

But if market conditions change, or you change your input parameters, will the strategy remain profitable?

Your strategy’s optimization profile can offer some answers.

Let’s dive into it.

What is a Strategy’s Optimization Profile?

An optimization profile simulates your strategy’s performance over the entire parameter optimization range.

Each data point in your optimization profile is the backtest result for that particular combination of parameters.

Let’s say you have a simple Donchian channel strategy where you:

• Enter the market on a channel breakout
• Hold the trade for a certain number of bars

You would have two optimizable parameters in this case – the Donchian channel lookback period and the number of bars in the market.

If you simultaneously optimize each parameter in the 10-100 range, you will get the following optimization profile.

The optimizable parameters are plotted on the horizontal axes, while the performance metric is on the vertical axis.

Why is the Optimization Profile Important?

The optimization profile reveals your strategy’s parameter stability. In other words, can it remain profitable when your parameters change?

Robust strategies remain profitable over a broad range of parameters. Such strategies are resilient to changing market conditions, and are more likely to remain profitable in live trading.

Ideally, you want to see an optimization profile with a high plateau, indicating stable performance across a wide range of parameters.

Conversely, an overfit strategy will only be profitable in a narrow parameter range.

Overfit strategies are constrained to random noise, which has no predictive value. Your historical backtests will be overly optimistic in this case.

In the next section, I’ll show you how to battle overfitting by quantifying your strategy’s parameter stability.

Plotting Your Strategy’s Optimization Profile in StrategyQuant

To demonstrate the use of StrategyQuant’s optimizer, I’ll use a GBPJPY trend following strategy that was automatically generated using genetic evolution.

You can download the complete strategy here.

The strategy captures breakouts by placing stop entries at the previous week’s opening price. An ATR-based stop loss and bar-based time stop are used for trade management.

There are only two optimizable parameters:

• StopLossCoef: ATR multiple for the stop loss
• ExitAfterBars: Number of bars used for the time stop

After backtesting from 2003-2020, the optimal values for StopLossCoef and ExitAfterBars were 3.1 and 154, respectively. This produced the following results:

Was this just a lucky combination of parameters? The optimization profile will tell us.

I loaded the strategy into StrategyQuant’s Optimizer module and configured the optimization range as shown:

What parameter ranges should you use?

Try to select values that are aligned with the premise of your strategy.

For example, if you’re a day trader, having a >100 lookback period for your indicator probably won’t make sense. Likewise, for trend following strategies where you should cut losses quickly, a stop loss over 5 ATR is excessive. 

With the optimization range selected above, the optimization profile will consist of 5986 different backtest runs. 

Let’s first visually examine the optimization profile.

And a top view for clarity:

The large stable region gives me confidence that the strategy has not been overfit.

It makes sense that the most profitable region is where the stop losses are small, and the time stops are large. You want to cut your losses quickly, while letting your profits run.

For simplicity, net profit was used as the performance metric. You can select other popular metrics like the profit factor or return/drawdown.

How about some quantitative measures?

Percentage of Profitable Optimizations

This is an important metric for parameter stability. I want at least 70% of my optimizations to be profitable.

Of the 5986 backtests, only 12 were losing, so I’m good here.

Average Profit

This is the average profit across all the optimization runs. If you want a metric that considers profitability together with robustness, this is a good choice.

At the minimum, you average profit should be positive.

I’ve got a $1000 average profit here, which translates to about 14000 pips over the 17-year backtest.

Uniformity of Profit Distribution

This counts how many times your profit flipped from positive to negative, or negative to positive.

For example, in the distribution below, this happened 3 times.

You want a small ‘flip count’, indicating your parameters are stable.

This is related to the percentage of profitable optimizations, although the same percentage can produce distributions of different uniformities.

Both profit distributions #1 and #2 above have 14 profitable runs.

But distribution #2 has 11 flips, while distribution #1 has only 1.

You want to see consistent and predictable patterns in your profit distribution. For example, having very small stop losses or time stops will result in overall losses. Profits will gradually increase as these parameters get larger.

If you get something erratic like distribution #2, your strategy is likely not robust enough.

For the strategy I’m testing now, there were 17 flips. This is a good result considering there were almost 6000 runs.

Spread of Profit Distribution

The performance of the top parameter set should be close to the average across the entire optimization space.

In StrategyQuant, the standard deviation of the profit is used to determine the spread of your profit distribution.

Ideally, I want my highest profit to be within two standard deviations of the average.

Here my highest profit of $1880 exceeds the two standard deviation limit of $1800. I consider this only a minor setback, especially since the rest of the results are good.

Plotting Your Optimization Profile Using MT4

If you use MT4 for strategy development, you can use its Strategy Tester for parameter optimizations.

The default optimization chart generated by MT4 is two-dimensional though.

You can export your results to Excel and create an array, with each cell corresponding to one combination of parameters.

You can then plot three-dimensional contours using the ‘3-D Surface’ option. Check out my Bollinger Bands vs. Keltner Channels comparison if you need a reference.

Tips to Prevent Overfitting

In an attempt to create a fantastic looking backtest, it is easy to keep adding/tweaking your strategy rules.

Overfitting is the result of excessive tweaking, whereby your strategy is designed to exploit random noise in your historical data.

When market conditions or input parameters change, you can expect strategy performance to suffer greatly.

You can minimize overfitting through the following measures:

• Minimize the number of strategy rules

As the saying goes, ‘loose pants fit all.’ Having fewer rules ‘loosens the fit’ of your strategy to the market, and makes it more resilient in changing markets.

The strategy illustrated above used a simple weekly breakout entry condition, and had only two optimizable parameters.

Most authors agree that it’s best to have a maximum of 4 optimizable parameters.

• Avoid simultaneous parameter optimizations

When constructing the optimization profiles above, simultaneous optimizations were used to quantify strategy performance over a broad parameter range.

But if you want to select optimal parameters for trading, try to optimize one parameter at a time. This makes it less likely to arrive at a ‘lucky’ parameter set that maximizes the fit between your strategy and historical data.

For example, optimize your entry lookback period first, then your stop loss size.

• Lengthen your backtest period/Test on multiple markets

This will make your strategy more adaptable to different market conditions.

It’s better to have a ‘jack of all trades’ strategy since you never know what the future holds.

Trading a diverse portfolio of loosely fit, good-performing strategies is far better than trading a handful of highly optimized strategies.

Wrapping Up

I’m a strong believer of robustness testing during strategy development. These help you weed out overfit strategies before you put real money on the line.

Plotting your strategy’s optimization profile is a quick way to evaluate your strategy’s robustness.

You want a large percentage of profitable optimizations, together with a high plateau on your optimization profile.

Other popular robustness tests include walk-forward optimization and Monte Carlo simulations, although these tend to be more computationally demanding.

Do you find the optimization profile useful? Let me know in the comments!