mt4 strategy tester - BATCH MODE

Already been done. https://www.mql5.com/en/forum/138637

If you want to run the optimization on one pair, duration and then run another on some other pair and duration then the above link will be helpful.

If you want to run the optimization on one pair, duration and then run the same set of parameters on some other pair and or some other duration, then there is Testing (Optimization) Technique and Some Criteria for Selection of the Expert Advisor Parameters - MQL4 Articles, Please read this link or what I have below will not be very understandable.

I was going to publish in the code base of my implementation of the above article. I didn't like using Excel macros (especially since they're incompatable with the free Open Office equivalent that I use.) And I thought the implementation could be better. So here's my mini-report:

Starting with all the externals that will be optimized, e.g.:

extern int      Open.Bar01                      =  0;
extern double   Trade.UTC.Start                 =  0.0;
extern double   Trade.UTC.End                   =  0.0;
:

add two more (at the bottom)

extern int      Opt.Mode                        =  0;
extern int      Opt.Counter                     =  0;

Next add a mapping function between your externs and their names. Note: Opt.Mode/Opt.Counter is NOT included, only your int/double externs.

//+------------------------------------------------------------------+
//| Optimization functions                                           |
//+------------------------------------------------------------------+
void OptMapParameters(){
    OptMapToInt     ("Open.Bar01"               , Open.Bar01                );
    OptMapToDouble  ("Trade.UTC.Start"          , Trade.UTC.Start           );
    OptMapToDouble  ("Trade.UTC.End"            , Trade.UTC.End             );
    :

Then add a simple line to each of init(), start(), deinit()

int     init(){                                             OptInitialization(); ...
int     start(){                                if (Opt.Mode < 0)   return(0); ...
int     deinit(){                                               OptComplete(); ...

And add the routines below.

The procedure is this: Initial optimization is standard (Genetic, set the parameters start, stop, and step, Opt.Mode must be zero.) and optimize for a period (I used 1/4 of my downloaded history and repeated for 3 times.) The result is a file (tester/files/EANAME.opt) containing all the positive profit combinations. You can sort that list (using notepad2 for example,) it will not affect subsequent results, but note the number of the largest pass before hand.

Subsequent optimizations: Uncheck Genetic and all your variables (reset button.) Increment Opt.Mode by one each run. Set Opt.Counter to run from 1 to the number of passes in the previous mode and run on another time period. The result will be additional lines for those parameters that were profitable in BOTH periods (along with total profit.) You can repeat this step as many times as needed.

Here are the required routines:

//+------------------------------------------------------------------+
//| Multi-Optimization functions                                     |
//| WantedMode = Opt.Mode - 1 and WantedLine = Opt.Counter. Read the |
//| .opt file for the line WantedMode, WantedLine. That contains all |
//| the parameters that need to be modified for this run. Extract    |
//| them and continue.                                               |
//| On the OptComplete call, if the run was profitable, then I need  |
//| to append to the .opt file all the parameters plus Opt.Mode and  |
//| Opt.counter for the next group.                                  |
//+------------------------------------------------------------------+
double  optInitialAccountBalance, optInitialProfit;
int     opt.pass;
string  opt.parameters, opt.var.name, opt.var.value; // Export to OptMapToXXXXXX
void OptInitialization(){
    if (Opt.Mode > 0 )if( Opt.Counter < 1){
        Print("OptParameters: Mode=", Opt.Mode,
                                    " with invalid Opt.Counter=", Opt.Counter);
        Log(  "OptParameters: Mode=", Opt.Mode,
                                    " with invalid Opt.Counter=", Opt.Counter);
        Opt.Mode = -1;  return; }
    optInitialAccountBalance    = AccountBalance();
    optInitialProfit            = 0;
    opt.pass                    = 1;    // Assume .opt is empty or NSF.
    opt.parameters              = "";
        string  nameOPT         = WindowExpertName() + ".OPT";
    int         handleOPT       = FileOpen(nameOPT, FILE_CSV|FILE_READ, '~');
    if (handleOPT < 1){                 // .opt is NSF
        if (Opt.Mode > 0){  int GLE = GetLastError();
            Print("OptInitialization: FileOpen(", nameOPT,") Failed: ", GLE);
            Log(  "OptInitialization: FileOpen(", nameOPT,") Failed: ", GLE);
            Opt.Mode = -1;                                                  }
        else{   // Assume first pass of initial optimization. Externals are set
            opt.var.name    = "";       OptMapParameters(); // from the tester.
            Log("Pass=1 Opt.Mode=0 inputs: ", opt.parameters);
        }
        return;
    }
    #define TAB "\t"
    // Opt.Profit=xx<tab>Opt.Counter=yy<tab>Opt.Mode=zz<tab>PARAM1=Val1<tab>...
    string  parmLine    = "Opt.Counter=" + Opt.Counter + TAB
                        + "Opt.Mode="    +(Opt.Mode-1) + TAB,
            lastPass    = "Opt.Mode="    + Opt.Mode    + TAB,
            realParm    = "";
    while(true){    // Not EOF
        string line = FileReadString(handleOPT);    if (line == "") break;
        if      (StringFind(line, parmLine) >= 0)   realParm = line;
        else if (StringFind(line, lastPass) >= 0){
            int     passBeg = StringFind(line, "Opt.Counter=")+12,
                    passEnd = StringFind(line, TAB, passBeg);
            string  pass    = StringSubstr(line, passBeg, passEnd-passBeg);
            int     passNo  = StrToInteger(pass);
            if (passNo >= opt.pass) opt.pass = passNo + 1;  // Last plus one.
        }
    }   // Not EOF
    GLE = GetLastError();   FileClose(handleOPT);
    if (realParm == ""){
        if (Opt.Mode > 0){  Opt.Mode = -1;
            Print("OptInitialization: Error ", GLE, " reading OPT file for: ",
                                                                    parmLine);
              Log("OptInitialization: Error ", GLE, " reading OPT file for: ",
                                                                    parmLine); }
        else{   // Set opt.parameters for OptComplete() from tester values.
            opt.var.name = "";  OptMapParameters();
            Log("Pass=" + opt.pass + " Opt.Mode=0 inputs: ", opt.parameters);
        }
        return;
    }
    //Opt.Profit=$x.xUSD<tab>Opt.Counter=yy<tab>Opt.Mode=zz<tab>PARAM1=Val1<tab>
    int profStart   = StringFind(realParm, "$")+1,
        profEnd     = StringFind(realParm, TAB, profStart);
        string prof = StringSubstr(realParm, profStart, profEnd-profStart);
    optInitialProfit= StrToDouble(prof);

        int modeStart   = StringFind(realParm, "Opt.Mode=", profEnd);
    int paramStart  = StringFind(realParm, TAB, modeStart)+1;
    while(true){
        int paramEnd = StringFind(realParm, "=", paramStart);
        if (paramEnd < 0)   break;
        int valueStart  = paramEnd+1,               // Skip equal sign.
            valueEnd    = StringFind(realParm, TAB, valueStart);
        opt.var.name    = StringSubstr(realParm,paramStart,paramEnd-paramStart);
        opt.var.value   = StringSubstr(realParm,valueStart,valueEnd-valueStart);
        OptMapParameters(); // Calls OptMapToXXXXX() below
        paramStart = valueEnd + 1;
    }
    Log("Pass="+opt.pass + " Opt.Mode=", Opt.Mode," inputs: ", opt.parameters);
    return;
}   // OptInitialization
void    OptMapToDouble(string text, double& var){
// string   opt.var.name, opt.var.value;        // \ Import from
//          opt.parameters                      // / OptInitialization
    if (opt.var.name != ""){
        if (opt.var.name != text)   return;
        var = StrToDouble(opt.var.value);                                   }
    for(int d=8; d >= 1; d--){
        string value = DoubleToStr(var, d);                     // =1.60000000
        if (StringGetChar(value, StringLen(value) - 1) != '0') break;
    }
    opt.parameters  = opt.parameters + text + "=" + value + TAB;    // =1.6
}
void    OptMapToInt(string text, int& var){
// string   opt.var.name, opt.var.value;        // \ Import from
//          opt.parameters                      // / OptInitialization
    if (opt.var.name != ""){
        if (opt.var.name != text)   return;
        var = StrToInteger(opt.var.value);                                  }
    opt.parameters  = opt.parameters + text + "=" + var + TAB;
}
void OptComplete(){
    double  profit = AccountBalance() - optInitialAccountBalance;
    if (Opt.Mode < 0 || profit <= 0)    return;         // No entry for loosers.
    profit += optInitialProfit;
    int APPEND  = FILE_CSV|FILE_WRITE|FILE_READ;
    string  nameOPT = WindowExpertName() + ".OPT";
    int handleOPT   = FileOpen(nameOPT, APPEND, '~');
    if (handleOPT < 1){ int GLE = GetLastError();
        Print("OptComplete: FileOpen(", nameOPT,") Failed: ", GLE);
        Log(  "OptComplete: FileOpen(", nameOPT,") Failed: ", GLE);
        Opt.Mode = -1;  return;
    }
    FileSeek(handleOPT, 0, SEEK_END);
    // Opt.Profit=xx<tab>Opt.Counter=yy<tab>Opt.Mode=zz<tab>PARAM1=Val1<tab>...
    opt.parameters  = "Opt.Profit="  + CurrencyToStr(profit, "", 14) + TAB
                    + "Opt.Counter=" + opt.pass                      + TAB
                    + "Opt.Mode="    + Opt.Mode                      + TAB;
    opt.var.name = "";  OptMapParameters();
    FileWrite(handleOPT, opt.parameters);
    FileClose(handleOPT);
}   // OptComplete
/** Log
* send information to OutputDebugString() to be viewed and logged by
* SysInternal's DebugView (free download from microsoft) This is ideal for
* debugging as an alternative to Print(). The function will take up to 10
* stringa (or numeric) arguments to be concatenated into one debug message.
//*/
#import "kernel32.dll"
   void OutputDebugStringA(string msg);
#import
void Log(string s1,    string s2="", string s3="", string s4="", string s5="",
         string s6="", string s7="", string s8="", string s9="", string s10=""){
    if (IsDllsAllowed()){
        string out  = WindowExpertName() + ": "
                    + s1 + s2 + s3 + s4 + s5 + s6 + s7 + s8 + s9 + s10;
        OutputDebugStringA(out);
    }
}

I used the Log() function for output since you can't see Print statements during optimization.

Dear ubzen, WHRoeder - that's very much for your reply. Cheers.

I've updated the routines.

Again. Drop dots in variable names. Better formatting. Computing harmonic and geometric rates. The closer harmonic is to geometric, the more consistent the returns are.

I finally understood your code.

What is the function CurrencyToStr? I couldn´t found it.

opt.parameters  = "Opt.Profit="  + CurrencyToStr(profit, "", 14) + TAB
                    + "Opt.Counter=" + opt.pass                      + TAB
                    + "Opt.Mode="    + Opt.Mode                      + TAB;

I think you wanted to standardize the profit with a length of 14.

Thanks for sharing your optimizer method.

Hi WHRoeder,

I have tried to add this code, copied directly from the multipass.mq4, but unfortunately it does not compile. There seems to be missing functions, missing }, missing variable declarations. I have tried to fix what I can, but some of the error messages I am not sure about.

eg

'LogMe' - function not defined

'trToInteger' - function not defined

'Double_To_StringToString' - function not defined - it was DoubletoString() -but i think this is a reserved word now?

status.chart - undecalared identifier - i amended to status_chart

So instead i tried to use some of the code in the thread itself, which i managed to get to compile, but then no output file was created.

Would you be able to relook at the code?

I wanted to use your code as a comparison to Testing (Optimization) Technique and Some Criteria for Selection of the Expert Advisor Parameters - MQL4 Articles, which I have tested so far, to see which method I preferred, and which will be easiest to formulate a WFA test process around.

thanks

Hello WHRoeder,

I'm also very interested in your Optimization Routine but as already stated above, the attached file does not compile in MT4.

May I ask you to upload a version of the file that is compatible with MT4?

Thanks a lot in advance,

Marley