English

## strategy that "works" and then fails dramatically (overfitting / perceptron)

20
nina32 2016.10.06 17:17

i'm a little bit confused, maybe you can help me with some ideas?

##########################################################################################################################################

at first some words about the thread and me... i need to say that my english is not very good, so anyway, i hope you'll understand me ;)... my questions are more about
perceptrons and curve / over-fitting, i don't want to sell anything here and sorry but i also don't want to give the ea or any indicator away. so please don't ask me for that
and please don't offer any money or something else, feel free to create it on your own, that's the best way to learn - thank you. i just want to understand some things
and discuss about that and that's all. i'll explain some parts of the strategy and show some code, but only to explain some questions, i don't need a coder and i don't
want somebody that codes for me ;)

##########################################################################################################################################

the strategy

it is a simple buy only strategy, to determine current direction and to get a dynamic stoploss (close to the market - but depending on volatility) i'm using a
supertrend (own creation - not this cci thing), it's mean is a simple moving average. the deviation can be calculated in 2 ways, the iAtr() or the standard
deviation iStdDev() - like bollinger bands.

to enter the market i use the iAC() to identify dips in an uptrend. i split entry's to average the entry price, max 3 trades can be open at time, the stop is trailed on supertrend lower band...

now we need a filter to get the right points for entering the market...
double   W0 = (X0 - 100.00); // adx
double   W1 = (X1 - 100.00); // adx
double   W2 = (X2 - 100.00); // adx

double Perceptron(const string _sym ,
const int    _tfr ,
const int    _prd ,
const double _ex0 ,
const double _ex1 ,
const double _ex2 ,
const int    _i   ) {

/////////////////////////////////////////////////

double in2 = iADX(_sym , _tfr , _prd , PRICE_CLOSE , MODE_MAIN    , _i);

double in1 = iADX(_sym , _tfr , _prd , PRICE_CLOSE , MODE_PLUSDI  , _i);

double in0 = iADX(_sym , _tfr , _prd , PRICE_CLOSE , MODE_MINUSDI , _i);

/////////////////////////////////////////////////

return((in2 * _ex2) + (in1 * _ex1) + (in0 * _ex0));

/////////////////////////////////////////////////

}

as it is hard to create fixed rules for determine how strong a trend is and so on, i'm using 1 simple perceptron that has only (to prevent over-fitting) 3 inputs / weights, the iADX() values of the CURRENT OR HIGHER TIMEFRAMES.

i choose the iADX() because it is normalized to a fixed range (0-100) and it has 2 important features, DIRECTION AND STRENGTH.

during the optimization the generic algorithm of the backtester will brute-force the weights of the perceptron to get a valid setup and for the filter it checks every timeframe - THE CURRENT AND ALL HIGHER TIMEFRAMES to get a good setting - this is "HOW IT SHOULD WORK" ;)....

the entry with iAC() is only valid if the perceptron confirms it... to get the perceptron working i test the weights from 0-200 and map them internal to a -100 < 0 > 100 range.

example : backtester range for input X0 is 0 to 200 step 1 so w = (X0 - 100.00) maps it to -100 < 0 > 100 and finally i do (inp * w)

so at the end all 3 weights are adjusted from -100 to 100, the perceptron fires if its sum is > 0.0

##########################################################################################################################################

after optimizing the parameters i get a nice picture, there are many setups in profit, it looks not bad ? for testing purpose i only optimized it for 1 year, 01.01.2013 to 01.01.2014.

it looks like there are many setups in profit, for me it looks "stable"....

checking one of the good results in tick mode, 90% quality - 0 error - with more then 100 trades gives me a confirmation

Symbol
 EURUSD (Euro vs US Dollar) Periode 15 Minuten (M15) 2013.01.02 01:00 - 2013.12.31 20:45 (2013.01.01 - 2014.01.01) Modell Jeder Tick (präziseste Methode, die auf allen verfügbaren kleinsten Zeitrahmen basiert) Parameter InpMaPrd=10; InpUseLRMA=false; InpMode=1; InpDevPrd=60; InpMultiPer=1.5; InpPrcTFP=6; X0=114; X1=28; X2=116; InpTPMulti=6.5; InpTrailSl=true; InpCloseByDir=true; InpMagic=666;

Getestete Kerzen

sorry for the german language, i don't want to translate it all ;)

in this case the test is running from 01.2013 to 01.2014 entry in M15, InpPrcTFP = 6 means that we are using a perceptron adx filter based on PERIOD_D1.

so as you can see, for the moment it looks pretty good doesn't it? i just can stop here and sell it to stupid people, but i would never do that ;)

if you're experienced in back-testing and / or machine learning - you know that you can't trust it until you verify it...

so the next step is to let it run with the same parameter setup, but this time we let it run from 2013.01.01 to 2016.01.01.

the ea has never seen the year 2014 - 2015 before, it was optimized for only 1 year - 2013 and 245 trades.

and now you can see why you have to do this - last - but important step...

the ea is only operating on open prices, so i can speed up the process by using this...

we can clearly see that the performance goes down rapidly after leaving the optimized area,

we had 245 trades in 2013 that's the optimized trade count, all trades after 245 are trades in unkown area, we have not optimized it for that part...

so now i'm not sure... is it over - fitting or does it only need to be optimized every month / week ? how long should the optimizing period be ? 1-2-3 years?

can 1 simple perceptron be so powerful? with only 3 inputs - is it possible? or can 1 perceptron with so less inputs over-fit so much?

what do you think about using the iADX() as an input / filter in a perceptron? normally you have to say if plus_di > minus_di & adx > 30 it is a trend but that's stupid,

market's are changing and you will never have the right algo / limit in such calculations...

so the idea was to create a hybrid, a simple strategy filtered by a simple perceptron...