How much optimization is enough

A very good question.

There is no easy answer.

There are experts in the industry who maintain that optimization is worthless. There are others who maintain that optimizing will reduce the predictive quality of your tests but increase your probability of future success.

My own view:

- Shrink the optimization issue by implementing an EA/strategy that is as simple as possible - so there are few parameters to optimize

- Create an optimization methodology for each parameter - these may be very different from each other; and some parameters you may choose to optimize once, periodically or not at all

- When you do decide to optimize a parameter, do it as a forward walk, considering whether the optimal value after each test remains close to being an optimal value in the next test period etc.

CB

I'll add a slightly different perspective to CB's very valid points... Let's say that you have a parameter which can range from 1 to 20, and which you're looking to optimize. You would/should expect to see a peak at some point in the range - e.g. 7 - with a steady improvement in the results as you go from 1 to 7 and a steady decline as you go from 7 to 20. If you don't see this, and instead you get all sorts of different results as your parameter varies, then it's a big red flag suggesting that your strategy is in fact random, and just happens by chance to have worked historically. 

(Unless, of course, you're one of the people who believes that there are "secrets" or "rhythms" of the market waiting to be uncovered - and that you, with a puny laptop at your disposal are more likely to come across such a thing than the millions of people who have already looked with much more impressive hardware.)

Unwittingly you have defined the problem quite well :)

The patterns of market action were quite different across the periods you mention

See http://investmenttools.com/equities/vix_volatility_index.htm for one of the main driving forces

Year 2008 thru til mid Nov was great for trending EA's, in fact in Q3 my sisters cat could have & I think did, make some $ by selling the downtrend on any yen cross

Since then we have had more different market patterns than some previous 5 year periods - with range trading and breakout methods largely winning over trending

So... you need to identify an 'X' factor that dictates & dynamically sets this parameter you mention

(Every time I put a fixed value as an extern, it now makes me nervous... OK, Slippage might be OK to set that way, but not StopLoss or TakeProfit)

This X factor could be the daily ATR, i.e. when its going up, use one value, when down another, etc

Or... and this is difficult for some to contemplate, stop the EA trading when ATR not in its good range for the EA

For a very useful indi giving moving averages of the D1 ATR and other good stuff, see http://www.selectfx.net/download/sfx_ind_3.zip

Good Luck

-BB-

Thanks BB for your comments.

The key point here to get correct X factor will will correctly capture the behavior of the market. This X factor indicator may not be indicating tend as such but the behavior of the market. All indicators, one wya or another, reflect the up and down trends and they may not reflect the "behavior" of the market. Thanks for the indicators, i will try to manipulate these and see these indicators helps to capture correctly the market behavior.

Is there any technical indicator which gives this behavior directly independant of uptrend and downtrend. Your comments will be valuable. Thanks again for your feedback.

Hi CB,

Thanks for your feedback. Your "forward walk" strategy is interesting. Forward walk strategy amounts to optimizimizing EA for short period for example for a month or so and apply the parameters obtained during th month over which it was optimized and apply to the next month. I tried to do that and looks like it doesnt work-out well for my EA. But I still think it is good idea and if this strategy works for any EA consistently it would be very good and will ensure that optimzation paramaters are "regular" in nature.

In my view, most EA's can benefit from optimization. But there are a few rules to respect:

.

1. If you work with ticks, your backtest won't ever be accurate anyway so optimization is useless. Working on past candles makes for more accurate backtests.

2. It's easy to go wild with optimization. Don't make a period of 6 months absolutely wonderful because you'll have optimized only for that period.

3. Go a couple years back up until 6 months ago and optimize. Use those settings for the last 6 months and then month to get an idea of how those settings are still performing. Going back 10 years should be for your curiosity and not for optimization.

4. Optimization is sometimes better after testing it on a live demo. If the EA is successful on backtests, run it on a demo to confirm. After that, you can run more backtests for optimization since you know now that backtest+livedemo is pretty accurate already for this strategy.

.

I might be forgetting a few things but this is what came to mind.

.

Jon

Jon, Thanks a lot!

Very helpful comments.

My pleasure, I sent you a PM as well by the way. I don't think you check them very often ;)