Need Help On Optimization Result - page 4

It is important to understand that the statistics computed and reported in the automated report are "time-series specific". Meaning they are literally only relevant in forecasting future trade characterstics provided the market itself has the same time-series characteristics....which for obvious reasons never actually happens.

You can really get hopelessly lost in trying to divine the future from the statistics of a backtesting report. At worst the results of backtesting are completely and utterly useless, at best if you have prepared the manner of backtesting correctly then you can generate some nuggets of data that allow you to speak to things that ought to be uncorrelated with the time-series used in the backtesting.

Just remember that you NOT dealing with a stationary process. Practically every statistic you may find yourself computing based on backtesting are irrelevant for providing indication of future results because the parent distribution is never fully sampled (it can't be because it doesn't exist yet, time creates more unsampled space) and the statistics of the distribution changes such as the average and standard deviation.

It is with this in mind that one is supposed to view Risk of Loss calculations as a "best case" result as the standard deviation is more likely to be wider in reality than what was generated from the limited sampling encountered during backtesting.

hello again :)

I changed the sample period from groups of 50 trades to per month computation and came with the following results :

assuming these values distribute normally ( which may not be the fact here )

1. Is there a way to factor skewness and kurtosis in the ROR computation ?

2. Is this really a normal distribution ?? If not how else can it be treated ?

A phrase you may not be familiar with is "therein lies the rub" which I suppose loosely translates to something like "the devil is in the details" in that once you come to realize the details that matter you then realize it is a devil to deal with.

Yes, you are assuming a normal distribution when in fact your results are not representative of a normal distribution.

By the way, totally a side-topic, but you may find that your histograms serve you better if you optimize the bin size.

Histogram Bin-width Optimization

I've implemented this code in MQL, I may have even uploaded it here if you check my posts. But I will say that if you decide to pursue it this is one of those things that you really need to dive into and teach yourself otherwise you won't really understand why an optimized bin-width histogram is helpful or special.

Back to your subject, the key point you have uncovered is that when you perform statistical analyses on your backtesting results oftentimes you will use statistics that are only rigorously true if your data are samples taken from a gaussian distribution. Where people tend to fail in their efforts at that juncture is testing this assumption, verifying that they have any legitimacy in applying normalized distribution statistics to their analyses.

At this point you reach a fork in the road...you can choose to pursue "statistically characterizable" results, discarding perhaps the seemingly optimal results for reasons that they do not conform to normalized distribution statistics, or you seek more generalized methods of analyzing your backtesting results such that the methods are robust and provide you with meaningful and useful metrics to forecast future results.

Here's an example of an analysis I performed from which I had my epiphany regarding the silliness of using normalized distribution statistics in my backtest characterizations:

The red-dots are datapoints, the solid green line is the best fit gaussian function to the red data points, the light blue line is the best fit generalized gaussian distribution function.

Are you mathematically inclined and not intimidated with pursuing the field of statistical analyses beyond that of the traditional gaussian-based distribution? Or are you more likely to find it not your passion and style and as such you would rather just discard and ignore these seemingly odd results and pursue characterizing the ones that conform to the more readily interpretable metrics?

At this point there is no consensus in which path you should choose, it is more a matter of personality and passion. Do what seems natural and easy.

I will second that and maybe refocus the thinking in the thread...
A sound strategy shouldn't need much optimization...?

If you are scalping, there should be an observed level of TP & SL that you will be going for
If grid-trading, the TP & SL are self-evident, as they are with range-trading
Swing trading will need ATR or Fibo based stops
Position trades will be too few to produce any meaningful stats on optimization, so...
Just what is it we are looking for?
If a strategy is not (actually) complete, is optimization really going to make up the difference?

FWIW

-BB-

@BarrowBoy "A sound strategy shouldn't need much optimization...?" Optimization can't make a good strategy from a bad one, but don't you rather think that optimization is made to find the settings that are revealing oportunities of the moment?