Dukascopy tickstory issue

 

I was beginning to implement volume in my back testing with some tick counting and categorising code(well I wanted to, but I then realised some fundemental flaws, such as assumptions about broker ticks and actual ticks).

Now I followed all of tickstory's instructions, however after doing so I started to get some weird results, for one I could not do multi-timeframe backtesting, as the chart could not get data from lower timeframes, I had to make my EA time frame independent and launch it on lower timeframes, results as normal. Until, I got a ridiculously good result (please see attached).

 

Now one aspect of my EA required high precision data in order to carry out backtests, so perhaps the presence of this data may be the cause, but I'm questioning the accuracy of the data, for one I read that dukascopy data had some very large swings (higher highs, and lower lows than some other brokers), erroneously of course. This and the lack of cross timeframe back testing at higher timeframes has made me questions some of my results.

 

Has anyone else had an experience with this time frame issue? How did you resolve it? Also dukascopy doesn't quote volume as the number of ticks but as the sum of contract sizes? 

Backtest 

 

tickdata-quotes put together tickvolume and spread into one number of the hst-files. You cant just use the bar's volume as it is. Birt's solution is:

long getTesterVol(long lng){ return( (long)((int)lng) ); }
long getTesterSpr(long lng){ return( (int)(lng >> 32) ); }
 
gooly:

tickdata-quotes put together tickvolume and spread into one number of the hst-files. You cant just use the bar's volume as it is. Birt's solution is:



Thank you very much. However the ticks thay MetaTrader receives, is its use not simply limited to determining the tick volume the broker is feeding to the platform (client). Let's say there is a price change caused by five hundred thousand contracts but we receive  5 ticks, and another price change is caused by two hundred thousand contracts however the broker processes this as 7 ticks, would this not undermine the use of tick volume as an indication of solid market activity/sentiment in relation to price changes?

On another note, have you had any problems with multiple timeframes in your backtests?
 

As far as I know the ticks do not have anything in common with the amount of money involved. It is just the number of prices the broker has received - or something like that.

Beside that the really big amounts of money are normally not visible!

I am about to publish an article about optimizing an indicator solely (without trading). For this I use (single) higher timeframes. For anything else I'll use only 1-min bars.

 
gooly:

As far as I know the ticks do not have anything in common with the amount of money involved. It is just the number of prices the broker has received - or something like that.

Beside that the really big amounts of money are normally not visible!

I am about to publish an article about optimizing an indicator solely (without trading). For this I use (single) higher timeframes. For anything else I'll use only 1-min bars.


Sorry when I said contracts I meant individuals buying or selling so, two hundred thousand individuals.

I wanted to use volumes as a form of comparing market activity(involvement) from one period to another and combine that with price action to determine sentiment, as I noticed from observation of dukascopy volume and price change spreadsheets, that there was some notable perhaps even reliable correlation.

It makes sense, the more people who believe the price should go up the more robust the sentiment, but the fact that ticks may not represent actual market involvement undermines the approach I was hoping to incorporate.

To your article, brilliant! I take it you use multidimensional arrays? How many dimensions do you use? Do you optimise multiple indicators at once?

By the way have you ever attempted a backtest with multiple timeframes?

I'm curious, how will optimisation work without RR taken into consideration? Do you have the RR recommended when the optimisation is complete?
 

Tick volume: Look here and google for more!

I remembered that there was an Metaquotes article about that - but I can't find it.

Thank you for our interest. I sent it in for review in November but may be the next weeks it'll be published.

No, I don't use multi dim. arrrays - only one indicator - I kept it simple.

I want to demonstrate how an answer of a special question to the market can be found without using a trading EA - just one simple question.
The main idea is just split a bigger problem (EA with trade management, higher and lower time frame indis for different purposes) into smaller pieces and solve them solely.

I choose: Is the market flat or trending and I use the ADX to check whether it does that trick or not and how good it might be on this job.

On this idea - I hope - anybody can now either develop a better solution for Flat-or-Trend with one's own set of indicators or ask a different question using the way that I developed for the flat-trend-question.

Be patient I am waiting now the 5th month ;)

Yes I tried MTF some time ago but I don't like to have such huge hst files and I don't like mt4 indicator concept (cant free the indicators and its memory, its always increasing - it seems not to be meant for started once and running forever - for what do I have a computer and its programs if I have to take care of them again and again?). So my base will be 1-min. Anything above that I calculate myself.

 

Ah great, I hope your pursuit accomplishes what you intend it to accomplish, for selfish reasons of course lol, as it would help me better understand trend formations if it did.

I trade mainly in trending markets, but I know traders who trade in all conditions, I always wonder how they do it, trading on fluctuations alone ( impressive and overwhelming). Spotting  macro and micro trends as well as macro and micro trend reversals or trend weakenings, of course you can't look for both macro and micro indications at all times as it can lead to over or under exposure, which  can distort strategies. Ideally one could might want heavy macro during trends and micro during flat markets. 

 Anyway, best of luck and thanks for your help. 

 

Another issue I found was my incorrect setting of the margin currency during export procedures, this has now been resolved. 

 

Good day. 

 

PS the link you provided was really helpful, if only MQL5 allowed imported ticks. 

 

Finally my article was published, you'll find it here.

I hope it helps you to find a better distinction between flat and trend markets.

 
MetaNt:

I was beginning to implement volume in my back testing with some tick counting and categorising code(well I wanted to, but I then realised some fundemental flaws, such as assumptions about broker ticks and actual ticks).

Now I followed all of tickstory's instructions, however after doing so I started to get some weird results, for one I could not do multi-timeframe backtesting, as the chart could not get data from lower timeframes, I had to make my EA time frame independent and launch it on lower timeframes, results as normal. Until, I got a ridiculously good result (please see attached).

Now one aspect of my EA required high precision data in order to carry out backtests, so perhaps the presence of this data may be the cause, but I'm questioning the accuracy of the data, for one I read that dukascopy data had some very large swings (higher highs, and lower lows than some other brokers), erroneously of course. This and the lack of cross timeframe back testing at higher timeframes has made me questions some of my results.

Has anyone else had an experience with this time frame issue? How did you resolve it? Also dukascopy doesn't quote volume as the number of ticks but as the sum of contract sizes? 


Sorry, to jump in so late in the discussion but I only started reading the thread now.

I too use TickStory and use Dukascopy data for all my back-testing. However, I have had no problem with back-testing with multi-time frame or with volume. There is however a little trick in order to accomplish this:

  1. When exporting from TickStory, make sure that the MetaTrader terminal is closed.
  2. Export both ".fxt" and ".hst" files and set it to export and include the Volume (don't suppress it because as of v1.7, it exports tick volume and not real volume).
  3. When the export is complete, make sure that the ".fxt" and the ".hst" files are set to read-only. Normally most people only set the ".fxt" files, but it is very import to set the ".hst" files as well, otherwise you can't do multi-timeframe.

The reason, is that when you run the back-test and access data of other time-frames, the Strategy Tester accesses the ".hst" files and not ".fxt". Arrays such as Close[], High[], Volume[] access the ".fxt" data, but functions such as iClose(), iHigh(), iVolume() accesses the ".hst" data. So, if you don't set the files to read-only and you also connect to the broker, the ".hst" files will have the data overridden with the broker data. This obviously causes havoc because price data no longer matches the ".fxt" and neither does the tick volume data.

That is all there is to it. If you do this, you will have no more problem with multi-timeframe or tick volume.

NB! There is however one more hick-up that has to do with the spread being set to Zero ( Ask = Bid ) in the strategy tester. I see from your report that you used build 950 which does suffer from this. So either use build 910 or follow the steps as described by TickStory (for v1.7.4) to workaround this. Otherwise, you will get unrealistic "good" results, because the spread being emulated as Zero, no matter to what spread value you set it.

 
FMIC:
....

NB! There is however one more hick-up that has to do with the spread being set to Zero ( Ask = Bid ) in the strategy tester. I see from your report that you used build 950 which does suffer from this. So either use build 910 or follow the steps as described by TickStory (for v1.7.4) to workaround this. Otherwise, you will get unrealistic "good" results, because the spread being emulated as Zero, no matter to what spread value you set it.

Do you know whether this influences the optimization?

Have you told this to the service desk - any reply?

 
gooly:

Do you know whether this influences the optimization?

Have you told this to the service desk - any reply?

Yes, this does affect optimisation. Although I have not personally contacted the service desk about this issue, I do believe that MetaQuotes are aware of it (see link below). However, since this only occurs when using externally generated data, they don't seem to be too worried about fixing it.

  • https://forum.mql4.com/70825/page4
So, you will have to use build 910 (as I do) or use the workaround as follows:

1. Launch MT4 as usual via the Tickstory launcher (probably also works if launched directly).
2. In the MT4 Strategy Tester, select your desired spread.
3. Select the “Control points” modelling type and press the ‘Test’ button. You can immediately stop the back-test once it has started. The selected spread will now take effect.
4. Select the “Tick” modelling method and continue back-testing as normal.
5. Repeat steps 2-4 whenever you wish to change the spread.

Reason: