Strategy Tester: fxt file becomes very large, backtest fails
Hi,
I've created a demo account and then I've disconnected MT and deleted the hst-files. Then I've imported external data into several markets such as EURGBP, XAGUSD, EuroStoxx50 and so on. Most of them work fine, but on several markets I've encountered a problem: the strategy tester takes extremely long to create the tickdata (fxt-file). The files becomes very large (around 15 Gb/year - normally the file has around 0,5 Gb/year).
The backtest also takes very long and at some point without any error-message it just finishes.
Does anyone know this problem and knows how deal with it?
I am guessing this data is not from a MT4 source...? Then it is very likely that the 'volume' associated with each bar is not the count of ticks (the convention used in MT4) but is the actual depth of market and may very well be ~thousands per bar. The Tester does not 'know' this and makes an FXT file with ~thousands of ticks per bar. Hence u have huge FXT files and very slow testing.
...at some point without any error-message it just finishes.
The tester can handle FXT files of up to 2GB in size. If they are any larger, the testing would finish after 2GB of tick data (I don't remember if in this case there are any errors in the log?).
The only solution to this problem is to change all volume info. IMHO, there is no point scaling down this 'volume' since it has nothing to do with the number of ticks per bar. So personally I just change all bars to have a flat volume of 4. This can be easily done with a script (HST file format is known). Note though, that these kind of sources should be used with care! Specifically, I would not rely on them for Testing any expert that is affected by M1 bar movements...
Do you already have such a script? If yes, would you post it here?
OMFG... I thought I'd finally be able to use volume info... But I'll try this. Thanks gordon!
Do you already have such a script? If yes, would you post it here?
Here ya go. It's a bit old and I haven't used it in a while, but I just checked and it seems fine.
Directions:
- Put all HST files in '\MT4\experts\files\' folder.
- Attach the script and set volume info.
- U can see script progress in the experts tab.
- After script is finished, close MT4 and move the files back to history folder.
- Designed for new HST file format. Older ones have a different header size...
- Assumes proper HST files.
- Although volume is a double, no point setting fractional part - as far as I know MT4 won't use it.
- I don't remember why I didn't make it process files in history folder directly... There might be a problem with that (or maybe I didn't think about it at the time).
Enjoy.
Then I exported ticksotry data to MT4 from 20110101 to 20140719 (only 4GB) and it worked well. Hence definitely the error is in the data, not EA. And the error is in the FXT-file, or MT4 or Tickstory, but definitely not in win 7/8/8.1 because they use NTFS with a file limit of 25 TerraByte! Please try to informe/aske for debugging this problem (Metaquotes or ticksory)!! As more user are asking as soon we will have a solution. Do any has any solution?
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
I've created a demo account and then I've disconnected MT and deleted the hst-files. Then I've imported external data into several markets such as EURGBP, XAGUSD, EuroStoxx50 and so on. Most of them work fine, but on several markets I've encountered a problem: the strategy tester takes extremely long to create the tickdata (fxt-file). The files becomes very large (around 15 Gb/year - normally the file has around 0,5 Gb/year).
The backtest also takes very long and at some point without any error-message it just finishes.
Does anyone know this problem and knows how deal with it?