Different backtest results with same parameters (including spread etc)

 

Hello everyone,

 for a few months, I was trying to do some backtests and I realized something (I also search the forum but other topics are different than my problem): Yesterday I did some backtest, close my laptop etc and today I tried to run one of profitable results but with same parameter that result stopped out today.

 here is the thing:

same parameters

same date range (end date is also fixed like : 01-01-2016 and 30-04-2016)

same spread (I fixed it to 3)

same computer

same metatrader

same broker.

 As I said, it was a profitable result according to yesterdays backtest result but today, it is not! I tried many results and they all have different results today. what could it be ?

 

thanks. 

 

paranoyakX:

...

As I said, it was a profitable result according to yesterdays backtest result but today, it is not! I tried many results and they all have different results today. what could it be ?

Did you run your backtest being connected to the broker ?

If yes then some parameters can change in the trading environment, for example quote needed to convert into account currency, or swap or ...

Your data can also be changed by the broker.

 

Hello angevoyageur 

yesterday I wasn't connected to the broker but today I am. My account is a USD account and there is no any swap or commission charges at this account. Also, my ending date was 30/04/2016, I think it shouldn't be changing so differently. one parameter group listed  as 10K profit in 3 month for example, today it listed stopped out at first month of testing.

even if cause to that, how can I trust to results if it will be so different everyday. 

 
paranoyakX:

Hello angevoyageur 

yesterday I wasn't connected to the broker but today I am. My account is a USD account and there is no any swap or commission charges at this account. Also, my ending date was 30/04/2016, I think it shouldn't be changing so differently. one parameter group listed  as 10K profit in 3 month for example, today it listed stopped out at first month of testing.

even if cause to that, how can I trust to results if it will be so different everyday. 

Even if they are the same you should not "trust" them. It's just a backtest on past data, with no slippage/requote/latency, a fixed (very low) spread.

Anyway, the results should be the same, and there is obviously a reason if they are different. Up to you to see if you want to investigate or not.

Most obvious reason is a change in trading environment/data when you connected to the broker.

 

angevoyageur,

 

thanks for reply. of course trust is the wrong word, I hope I explained my self but there is a huge difference between backtest results. I will delete and re-install mt4 and try again. 

 

paranoyakX:

... I will delete and re-install mt4 and try again. 

It's completely useless.
 
angevoyageur:
It's completely useless.

Hi @angevoyageur, as you said it was useless and I still can not find the reason why the results are different. I re-run whole test (it takes 3 days to finish) and current results are still different than the before ones and they are different now too. I have no idea how to solve this. it seems every day I run the tests it gave different results. first I thought it could be "every tick model". I assume metatrader simulate the tick data because it has not have tick data and that could be the reason for different results. at a big bar, it is very important if simulated tick data go up first and than go down or vice versa. that will change results but It should not be so different I guess.

I do not  know what to do.

 
I assume you are somewhere mixing different time functions, TimeLocal instead of TimeCurrent and/or usage of GetTickCount. 
 

What is the currency pair you are testing and what is the currency of the deposit/balance?

This is important because, MetaTrader calculates conversion between currencies based on the current market values. The same problem is also in play when you have money management in use based on a %risk and you are using the TICK_VALUE to calculate this, where that value is actually the current market value and not a simulated one.

For example, if your are testing "GBP/JPY" but your deposit/balance is in "USD", then the profits/losses have to be converted into the balance currency and that is done by looking at the current conversion rate. Since the current market price is constantly fluctuating (during the week), every time you run the back test the results will be slightly different.

To avoid this, disconnect from the broker while tests are done and repeated so as to avoid variations in the conversion process or use a balance/deposit currency that is the same as the quote or counter currency in the pair so that the TICK_VALUE is constant.

PS! Just realised that "angevoyageur" said the same thing but in less words.

angevoyageur:

Did you run your backtest being connected to the broker? If yes then some parameters can change in the trading environment, for example quote needed to convert into account currency, or swap or ...

 

thank you very much for these answers,

 @Doerk, I didn't use any time or tick functions. I just open an order when markets reach to a price and than wait for close.

 @FMIC, your answer is really make me thinking. as you said  angevoyageur said this thing but as I said, I am new at this and couldn't understand it. so I run my code on Dow jones index. 1 point difference in dow jones makes 1 dolar profit or loss for 1 lot. also if the problem is this, then it should be just a slightly different from other runs. In my case, a combination has profit previous run but when run it now, it has a huge loss. about 4-5 hours ago, I run a combination and it was different previous one when run it now, it is still same with morning (4-5 hours ago). so it should be different from the morning one too, am I wrong ? 

 edit: my account currency is also USD. 

 
paranoyakX:

@FMIC, your answer is really make me thinking. as you said  angevoyageur said this thing but as I said, I am new at this and couldn't understand it. so I run my code on Dow jones index. 1 point difference in dow jones makes 1 dolar profit or loss for 1 lot. also if the problem is this, then it should be just a slightly different from other runs. In my case, a combination has profit previous run but when run it now, it has a huge loss. about 4-5 hours ago, I run a combination and it was different previous one when run it now, it is still same with morning. so it should be changed too, am I wrong ? 

I have never traded Indices but from your description then the TICK_VALUE should be constant since you have a USD deposit/balance.

So, the conclusion that I come to, is that maybe it has nothing to do with the broker or its data but maybe it has to do with the EA code itself. Maybe there are bugs in it that is causing the problem.

If you don't mind showing the code, we can try looking at it. Alternatively, if you have a coder friend or acquaintance that you trust, have them take a look at it and run tests of their own in order to see if they can find the problem.

Reason: