Searching for script creating more detailed report with more statistics after testing EA - page 2

New comment
 

I have a call function that does this. I call it in the deinit() of the EA so that it only runs at the end of the backtest. It doesn't consume much time and if you are doing an optimization it will append the csv file one row at a time with the stats.

Place all four files in the include folder.

In your EA put the following:

#include <Characterize Trades 2010.06.08.mqh>
Then augment your deinit so it includes the following:
int deinit()
   {
   if(IsTesting()==true || IsOptimization()==true)
      {
      string ParameterList=StringConcatenate("TakeProfitparameter =",TakeProfitparameter,"; StopLossparameter = ",StopLossparameter,"; ");
      // setup the stringconcatencate to create a text string with the parameters your EA is optimizing
      TradeAnalysis("MySuperEA",ParameterList,false); // edit the first field to reflect your EA's name
      }
   }

This will create a file in your tester/files folder with the following stats (in columns, one row per backtest run) regarding the trades made during the backtest:

Trade Span (days) - how many trading days did the backtest encompass (starting from first active trade open to most recent trade that was closed)
Total Trades - how many completed trades are included in the analysis
Net Profit ($) - net profit
IntegratedROL - aggregated Risk of Loss
µHPR% - average holding period return percent
stddev% - std deviation of the rate of return
AHPR$ - average holding period return in dollars
sHPR$ - std deviation of the return in dollars
SharpeRatio - Sharpe Ratio
µMAE ($) - average Maximum Adverse Excursion in dollars
µMAE (pips) - average Maximum Adverse Excursion in pips
µMFE$ - average Maximum Favorable Excursion in dollars
µExcessMFE$ - average Excess Maximum Favorable Excursion in dollars
µMAEminutes - average time (in minutes) to reach the MAE
µMFEminutes - average time (in minutes) to reach the MFE
Parameters List - this is the user-specified string intended to capture the relevant parameters used by the EA which resulted in the above listed stats

There are a ton of user notes to go along with the include files, please ask lots of questions. In the interest of not creating a 500 line post here I will not go over all the specifics in this initial post I am making on the topic but I will happily respond to any questions.

 
Hooray... Now thats what I'm talking about. Phillip you are a god_send man. I'm dropping everything else I'm working on to play with this. Lets see if I can get some #'s for that Booya. Yeah-ya :)
 

I apologize for the code in advance because I did not clean it up to make it any easier for a broader audience to use.

It does other things as well, computes the average time that a trade is open, the average time that the EA is "out of the market", etc. The histogram feature is pretty handy too if you are doing single runs and just want to create a deeper in-depth study of the resultant trades.

And of course it is a work in progress, still evolving, so don't be too surprised if you find some unfinished business in the bowels of the code.

 

Looks like I'm doing something wrong from the outputs.

Where should the TakeProfitparameter and  StopLossparameter variables be defined?

 

10:07:46 2010.07.29 23:59  BooYa EURUSD,M30: Total excursion trade count = 0 after start
10:07:46 2010.07.29 23:59  BooYa EURUSD,M30: Total excursion trade count = 0 after ActiveTradeTimeArray
10:07:46 2010.07.29 23:59  BooYa EURUSD,M30: Total excursion trade count = 0 after InActiveTradeTimeArray
10:07:46 2010.07.29 23:59  BooYa EURUSD,M30: Total excursion trade count = 1 after ProfitLossinDollarsArray
10:07:46 2010.07.29 23:59  BooYa EURUSD,M30: Total excursion trade count = 1 after MAEinDollarsArray
10:07:46 2010.07.29 23:59  BooYa EURUSD,M30: Total excursion trade count = 1 after TimetoMAEArray
10:07:46 2010.07.29 23:59  BooYa EURUSD,M30: Total excursion trade count = 2 after MFEinDollarsArray
10:07:46 2010.07.29 23:59  BooYa EURUSD,M30: Total excursion trade count = 2 after TimetoMFEArray
10:07:46 2010.07.29 23:59  BooYa EURUSD,M30: Total excursion trade count = 2 after ProfitLossinPipsArray
10:07:46 2010.07.29 23:59  BooYa EURUSD,M30: Total excursion trade count = 2 after MAEinPipsArray
10:07:46 2010.07.29 23:59  BooYa EURUSD,M30: Total excursion trade count = 2 after MFEinPipsArray
10:07:46 2010.07.29 23:59  BooYa EURUSD,M30: Total excursion trade count = 2 after HPRArray
10:07:46 2010.07.29 23:59  BooYa EURUSD,M30: Final excursion trade count = 2
10:07:46 2010.07.29 23:59  BooYa EURUSD,M30: AHPR = 0
10:07:46 2010.07.29 23:59  BooYa EURUSD,M30: sHPR = 0
10:07:46 2010.07.29 23:59  BooYa EURUSD,M30: SharpeRatio = 9999999
10:07:46 2010.07.29 23:59  BooYa EURUSD,M30: IntegratedROL = 1, µ = -100% and stddev = 0%
 

The ParameterList is just a documentation feature, you edit it to document whatever parameters you wish to have documented in the output csv file.

For example let's say your EA involves a MA cross and you are optimizing the number of candles to use for both moving averages (the ma_period parameter). So you would setup the string ParameterList to capture this info and pass it along to TradeAnalysis() routine.

Let's say you setup your ma_period variables as extern so they can optimized by strategy tester: 

extern int     SlowMAPeriod=81,FastMAPeriod=13;
Then you setup strategy tester to optimize both SlowMAPeriod and FastMAPeriod values spanning a range of say 50 to 100 in steps of 5 and say 5 to 25 in steps of five.

You want the specific paramter combo to be captured and documented along with the resultant trade statistics so you would setup ParameterList as follows:
string ParameterList=StringConcatenate("SlowMAPeriod =",SlowMAPeriod,"; FastMAPeriod = ",FastMAPeriod,"; ");

Now you will have the relevant info in your csv file. The string can be no longer than 64 characters. There is no point in capturing/documenting trade parameters that do not vary across the backtest space. If you always use 150pip stoploss then there is no need to document this in the csv file.

Depending on the specific criteria you use for your trade strategy you would setup ParameterList to suit your needs.

Another thing I do is create a string with my EA's name as well as set my histogram flag as an extern bool. For example I declare the following:

extern bool    Histograms=false;
string   EAName="Original BullBear Strategy 2010.06.08";

And then in my deinit() I call the following: 

TradeAnalysis(EAName,ParameterList,Histograms);
 

Awesome Stuff!! I've got the csv files. That B-Column in most of the CSV files, what does it represent? 

 

Depending on which features you have "enabled" there can be a slew of csv files created. If the file in question is "..._Trade_Characterization.csv" then the second column is the total number of completed trades during the testing time period (it intentionally excludes trades that were open at the end of the test time which strategy tester manually closes). If the file is one of the histogram files then the second column is frequency of occurrence, the first column is the histogram bin.

 

Cool, I'm particularly interested in that Time to MFE. Did you really discover MAE/MFE? LOL. No seriously did you? Anyways, I believe in the Statistic. Even if someone is running a scalper system, this could come in handy. When I was trying to create a scalper system. The most important question I had on my mind is how good is this Indicator in predicting the next 11-pips. I didn't want something which would feel like a random entry where I would have to use Huge stop_losses prying that the market would come back around to my direction. Nothing else IMO provide a better answer for that than MAE/MFE. Forex is not like most other Independent Events Betting where If you put $100 on the table then your losses/risk would be exactly $100. In Forex and most Trading, there are degrees of Risk and Reward which makes the math more fuzzy.

 

Anyways, you really have a good tool here. When do you think you'll have it completed and flexible for the Masses if that's your intension of course? I have to finish my studies on Arrays. But when I'm done I'm going to submit the statistics for the Booya on that thread referencing your tool (if thats ok with you). Last question, do I run the same curve fit risk trying to optimize MAE/MFE to capture every Peak and Trot. Oh 1-more, does MFE/MAE take into account taking profits too early. That's another form of leaving money on the tables. How do you feel about that?

 
Did you really discover MAE/MFE? LOL. No seriously did you?

My journey in forex has been one of reinventing many wheels, many times over, only to eventually come to find out the wheel was already discovered and well publicized in the formal financial circles decades ago.

I always took it to be proof of that which I already knew - I have no formal education in matters of finance so it comes as no surprise to me when I develop equations which I later find out to be the sharpe equation and so on - and confidence building as I can't be too far off the right path if I happen to be only a decade behind the footsteps of the leading edge guys.

Did I discover MAE/MFE? Sure, but I wasn't the first. For example I had no idea Rush had published similar code over 2yrs ago. It took me years of crawling on my hands and knees to just get to the point that some of these guys were at years ago, who knows what they have advanced to in the meantime.

When do you think you'll have it completed and flexible for the Masses if that's your intension of course? I have to finish my studies on Arrays. But when I'm done I'm going to submit the statistics for the Booya on that thread referencing your tool (if thats ok with you).


I don't really have any plans at the moment. Everything is a work in progress and in the summer months my rate of progress is extremely slow. Come Sept/Oct I have a few things I intend to iterate the code to include and if people are interested I would have no problem sharing the updated code.

In parallel, if things on my end are moving a little slow for your tastes then by all means feel free to take matters into your own hands and iterate the code to your liking. I shared it with expectations of it answering some questions regarding what can be done (it helps to know you aren't working towards the impossible) and possibly fostering some collaboration.

Last question, do I run the same curve fit risk trying to optimize MAE/MFE to capture every Peak and Trot. Oh 1-more, does MFE/MAE take into account taking profits too early. That's another form of leaving money on the tables. How do you feel about that?


Absolutely. You get into some extensions of the metrics, ratioing the excess MFE to the profit and so forth that give you a metric for ranking the quality of the profits versus the money left on the table. What you'd like to see is a strategy that gives you acceptable MAE and MFE (obviously subjective, as are expectations of acceptable rates of return, drawdown, value at risk, etc) within which you then go about downselecting based on risk of loss versus rate of return analysis. Put that autofilter feature to work in Excel since you have all the data captured by column for you in the csv file.

Borrowing from the world of finance, what we want to find ourselves doing is setting up our backtesting such that the resultant statistical analyses are performed within the backdrop of "risk-adjusted profits". Sharpe attempts to do this, rate of return divided by the standard deviation in those returns, but it misses the mark in terms of capturing the risk of loss that one can expect from the variance alone (we can get into this deeper if you like, but it is another topic).

 

Thanks for uploading this.

Sadly, I cant seem to get any csv file produced and when I compile my EA I get a number of Function warnings such as:Function "MathAvg2DArray" is not referenced and will be removed from exp-file,Function "MathStdev2DArray" is not referenced and will be removed from exp-file etc for a total of 5 functions. Could that be the reason I dont get the csv file ?

Otherwise, after I've added the code from your post above and changed the relevant bits to reflect my EAs name etc, the file compiles fine and the optimisation happens without a hitch.

Any suggestions would be appreciated.



123
New comment
Reason: