Before I go any further, let me say that I have searched carefully using the forum search feature and using Google, and while I am aware that there are many threads over the years about unmatched data errors, NONE that I found addressed the issue I am facing.
Following advice in many of those posts, I have set up data for back testing a strategy designed to work on daily candles. I may expand this strategy later to work on smaller time frames too but at the moment my focus is on daily candles. The strategy combines some simple price-action related rules with some simple tick volume-related rules to determine entries, and uses a simple but very rigid risk management strategy.
As to the providence of my data. I run a process I wrote that downloads tick data continuously while the market is open using FXCM's Java API. I store the tick data to a database and use it to calculate one-minute bid and ask price data each week together with tick volume data. In this way I've been building up a database of one minute price data and tick volume since September 2012.
I took the bid data from this database for USDJPY for the entire period, including tick volume, and washed it through some home-grown programs I wrote that analyse the data for errors eg duplicate bars, bars where the high isn't the highest price / low isn't lowest price, etc. It also watches out for gaps in the data that don't occur at the weekend, and strips out pre-market data on Monday mornings (I am in Japan, market opens at 6am Monday in summer, 7am Monday in winter local time). I converted the time stamps on the data to Europe/Nicosia time zone, which is the time zone used by my broker, as it results in the market open being at midnight and thereby avoids Sunday bars.
I then followed the following steps:
If I omit step 14, ie don't turn the network back on, the unmatched data errors do not happen, even with repeated backtests, but then I get a warning in the log saying No connect to trade server, default environment will be applied, and I have no idea what "default environment" means and what bearing it has on my results.
I assume the problem is something to do with data MT4 is getting from the broker (not FXCM) where the volume data and/or price data on a daily time frame doesn't exactly agree with the data I've uploaded. What I don't understand is why it is looking, since I supplied all required data and am not trying to test outside the date range I loaded. Is there a way to prevent this clash, and if the only way to do so is to keep the network down, what implications are there of using the "default environment" ie what in precise terms does "default environment" mean?
Thanks for your help if you are still reading :)
Good to know "default" means "last used". That is a relief. I was afraid "default" would equate to "total bollocks".
On point 2 -- just let me make sure I have understood correctly, if you don't mind. Are we saying that on loading a chart (either a real one or a virtual one for the purposes of the Strategy Tester) the last 2048 candles' worth of data will be downloaded even if it has already been provided in the HST files? For a Daily chart that's 2048 days' worth of data that I uploaded in 1-min form and then translated to daily using the Period Converter, which is just getting trampled...
Aaactually I've just realised what might be going on here. Since I only started recording tick data and using it to construct my one minute bars in 2012, I only have volume-included history back to then (I have price history from FXCM back to 2002, but don't have volume data to go with it. That hasn't mattered until the EA I am testing now). On the daily chart that is 1000-odd bars. Less than 2048. Perhaps that is prompting MT4 to download the whole 2048 bars instead of only the 10 or so bars from where my uploaded data ends to the current day.
OK I'm starting to talk myself into believing that my situation is a little bit of a corner case. I guess I'll just have to backtest with the network disconnected for this EA for now (or test on smaller time frames!) or see if I can get my hands on more price history with volume... It's a major pain to switch data providers though...
To add comments, please log in or register