tester: tick events vs bars vs bar states

New comment
 

The tester will make one final entry at the end of a backtest:  

AUDUSD,M1: 12233929 tick events (238505 bars, 9663402 bar states) processed in 4:39:18.345 (total time 4:39:22.807)

What does each part mean?

12233929 tick events,

238505 bars,

9663402 bar states

processed in 4:39:18.345 (this is the time from start it took to process).

Is there documentation that explains this?

Using mt4 b988 with TDSv2

 

Bar states mean number of bars in the FXT chart, bars is count in the HST file.

So far I believed, that the tick events were a subset of the bar states. So I am surprised you have it higher.

 
Ovo:

Bar states mean number of bars in the FXT chart, bars is count in the HST file.

So far I believed, that the tick events were a subset of the bar states. So I am surprised you have it higher.

When choosing "Every Tick" model, bar states (misleading name) are the count of the emulated tick data generated (or actual ticks contained in the FXT file), and it is absolutely guaranteed that bar states will be greater than bars, because obviously each bar contains many, many ticks (tick volume).

Tick events on the other hand, also a misleading name, is somewhat of a mystery to me.

 
FMIC:
When choosing "Every Tick" model, tick events are the count of the emulated tick data generated (or actual ticks contained in the FXT file), and it is absolutely guaranteed that tick events will be greater than bar events, because obviously each bar contains many, many ticks (tick volume).
Well, not many ticks, but the ticks can be aggregated to a single bar state, if they have the same quote. But still, I believe that the aggregated bar states (with volume > 1) are proceeded as a single event rather than multiple times.
 
Ovo:
Well, not many ticks, but the ticks can be aggregated to a single bar state, if they have the same quote. But still, I believe that the aggregated bar states (with volume > 1) are proceeded as a single event rather than multiple times.
Well, actually in my initial post a used the wrong name and have subsequently reworded my statement.
 
My tests all use 'every tick' and the fxt files are generated using Dukascopy tick data converted into fxt format.
 
4evermaat:
My tests all use 'every tick' and the fxt files are generated using Dukascopy tick data converted into fxt format.

 It does not give me a clue why there are more ticks (events) than quotes (bar states). Would you mind to upload here the start (appx 50kB) of your fxt file?

 
Ovo:

 It does not give me a clue why there are more ticks (events) than quotes (bar states). Would you mind to upload here the start (appx 50kB) of your fxt file?

I need the steps to splice the fxt file so that I can give you the correct portion (what program to use, what it should look like, etc).

 

I can say that I generate fxt file from csv2fxt script by Birt for use in TDS.

With TDSv2, the FXT file is created on the fly, and will be 0 bytes.  The ticks are fed to the FXT placeholder directly from a proprietary compressed database, similar to dukascopy compressed database, but with higher compression and pretty fast unzip.

 
4evermaat: With TDSv2, the FXT file is created on the fly, and will be 0 bytes. The ticks are fed to the FXT placeholder directly from a proprietary compressed database, similar to dukascopy compressed database, but with higher compression and pretty fast unzip.
Where can one find more information on TDSv2. Birt's EA Review website does not mention anything about it!
 
FMIC:
Where can one find more information on TDSv2. Birt's EA Review website does not mention anything about it!

He's on vacation.  He should be releasing the beta to the public by the end of the week or early next week.

 In a nutshell, he took the entire process of downloading tick data from dukascopy (or truefx.com), converting to mt4 FXT, and streamlined it.  

 Now you just open an mt4 terminal with a regular shortcut (no separate TDS launcher), and then select 'use tick data'.  Use variable spread in the tick data settings, and select the date/time range in tester.  At start of backtest, it will auto-generate hst files and load ticks directly from database at the same or slightly better speed.  Takes 40 seconds or so of startup time for first backtest of a pair, and then afterwards the tests start immediately.

 Old way:  spend hours downloading ticks (using Tickstory/SQ TDD) generating csv files, then using csv2fxt converter script, copy the fxt files to the "tester terminal", then launching TDS to launch terminal and run backtests. 

  New way: load up ANY mt4 terminal(s), TDSv2 adds an option to 'use tick data' (widen the tester window), adjust tick settings, run the tester.  40 second delay on first run of a particular symbol, otherwise it starts .

 


The longest part is initially importing your existing dukascopy database into TDS tick database (50-60% better compression), which will take about 25-40 hours uninterrupted.  Birt outdid himself here.  It will be a free upgrade to all existing customers.  

 
4evermaat:

He's on vacation.  He should be releasing the beta to the public by the end of the week or early next week.

In a nutshell, he took the entire process of downloading tick data from dukascopy (or truefx.com), converting to mt4 FXT, and streamlined it.  

Now you just open an mt4 terminal with a regular shortcut (no separate TDS launcher), and then select 'use tick data'.  Use variable spread in the tick data settings, and select the date/time range in tester.  At start of backtest, it will auto-generate hst files and load ticks directly from database at the same or slightly better speed.  Takes 40 seconds or so of startup time for first backtest of a pair, and then afterwards the tests start immediately.

Old way:  spend hours downloading ticks (using Tickstory/SQ TDD) generating csv files, then using csv2fxt converter script, copy the fxt files to the "tester terminal", then launching TDS to launch terminal and run backtests. 

New way: load up ANY mt4 terminal(s), TDSv2 adds an option to 'use tick data' (widen the tester window), adjust tick settings, run the tester.  40 second delay on first run of a particular symbol, otherwise it starts .

The longest part is initially importing your existing dukascopy database into TDS tick database (50-60% better compression), which will take about 25-40 hours uninterrupted.  Birt outdid himself here.  It will be a free upgrade to all existing customers.  

Thank you for the information! I will be on the lookout for when he releases it.
12
New comment
Reason: