Skipping bars during backtest (a lot...!)

 

Hi, I created a simple EA using the automatic EA builder at eatbuilder.com, so I trust the general structure and logic of it is correct. Also, in my EA I need to check at the beginning of each new 5 min bar to see if it's time for a new buy or sell. In order to determine when a new bar happens, I found this function to check for "Newbar" at http://www.metatrader.info/book/print/34 (I assume it should be tried and true also). However, when I backtest my EA and look over the Journal entries at my Print statements I've inserted, I see that MT4 is skipping over a LOT of 5 minute bars. AT first I thought it MUST be something with my EA, so what I'd do is try an already built-in EA in MT4 - like the Moving Average one, and put this Print statement at the very beginning of the "void Start()" function:


Print("Timestamp for this tick ", TimeToStr(Time[0], TIME_SECONDS));


This SHOULD print out the timestamp for each and every tick as it loops around, right? Anyway, I backtested the Moving Average EA with this statement in it only from 2010/06/14 to 2010/06/19. Then I checked the Journal for the Print statements... there was BIG gaps of bars and ticks missing! For example, it would jump from 2010.06.15 03:45 right to 2010.06.16 17:34, with nothing in between!! And plenty more instances like that...this is crazy! I've been pulling my hair out all weekend trying to figure this out - I basically just wasted two whole days struggling with this.


Has anyone also experienced this, or can shed some light on it? I'm using an Alpari UK demo account (5 digits data).


Thanks!

Shawn

 

I've tested your above code and scenario. My observation is the log file is different depending on the speed of the tester. Also, sometimes it appears that it truncates the log file after the test is over.

 

Example: (my test #1 at high speed), Print("Timestamp for this tick ", TimeToStr(Time[0], TIME_SECONDS)); showed 5 minute open times during the process but at the end, when i scrolled down i could only locate timestamps for 2010/06/14. The 2010/06/18 stuff i saw earlier vanished.

 

Example: (my test #2 at high speed), Alert("Timestamp for this tick ", TimeToStr(Time[0], TIME_SECONDS)); this time I used Alert instead of Print and it shows about 5-hours worth of time stamps and then jumps to the next day.

 

Example: (my test #3 at slow speed), Alert("Timestamp for this tick ", TimeToStr(Time[0], TIME_SECONDS)); this time I used Slow speed instead. For 06/14 it was all there so half way through i decided to speed it up and it jumped from 6/15 to 6/18 with nothing in between.

 

Sounds like your history files are bad.

set tools -> options -> charts -> max bars in history = 999999999999, close the terminal, delete the history files for that pair, start the terminal, open a chart for that pair/time, page up to get all available history.

 
ubzen:

I've tested your above code and scenario. My observation is the log file is different depending on the speed of the tester. Also, sometimes it appears that it truncates the log file after the test is over.

Example: (my test #1 at high speed), Print("Timestamp for this tick ", TimeToStr(Time[0], TIME_SECONDS)); showed 5 minute open times during the process but at the end, when i scrolled down i could only locate timestamps for 2010/06/14. The 2010/06/18 stuff i saw earlier vanished.

Example: (my test #2 at high speed), Alert("Timestamp for this tick ", TimeToStr(Time[0], TIME_SECONDS)); this time I used Alert instead of Print and it shows about 5-hours worth of time stamps and then jumps to the next day.

Example: (my test #3 at slow speed), Alert("Timestamp for this tick ", TimeToStr(Time[0], TIME_SECONDS)); this time I used Slow speed instead. For 06/14 it was all there so half way through i decided to speed it up and it jumped from 6/15 to 6/18 with nothing in between.


Ubzen, I think this must be it... during a high speed test, the log file just omits certain items. That's pretty bad eh? When a person is debugging, the logfile is the ONLY thing you have - so if that is not accurate, you have nothing! This is a big big bug in MetaTrader.


WHRoesder, I'm pretty sure my data was good, I followed this procedure:

http://alansforexblog.com/2010/02/10/how-to-set-up-metatrader-history-data-and-get-90-backtesting-quality/


Thank You!

 

When checking the Journal, are you checking the Journal tab or are you opening the actual logfile to look at the content?

CB

 
cloudbreaker:

When checking the Journal, are you checking the Journal tab or are you opening the actual logfile to look at the content?

CB


I was simply checking the Journal tab CB... so the actual logfile is guaranteed to be 100% accurate? Where is it located exactly?


Thanks

Shawn

 

The full content of the Journal tab is in the logfile contained in the logs folder.

The full content of the Experts tab is in the logfile contained in the experts/logs folder.

CB

 
cloudbreaker:

The full content of the Journal tab is in the logfile contained in the logs folder.

The full content of the Experts tab is in the logfile contained in the experts/logs folder.

CB



Ahh okay, well that should make a big difference! How often do these files get overwritten CB?
 

They don't get overwritten, as such. The filenames are date-stamped.

CB

 
cloudbreaker:

They don't get overwritten, as such. The filenames are date-stamped.

CB


This should fix me up, thank you CB!
Reason: