Random values in a loop - page 2

http://lmgtfy.com/?q=low+bits+on+random+numbers+are+nonrandom yields among many http://www.fourmilab.ch/random/ which states:

I've tried their ent.exe application on 500,000 samples of the low-order bytes from the MT4 random number generator, and I consistently get chi-squared percentages in the range 5-95 which they describe as not suspect. All the other measures (arithmetic mean etc) also look random.

On this basis, I think your "Never, EVER, use the low bits of random numbers" is wrong with regard to MT4.

Example output as follows:

C:\temp>ent mt4rand.txt
Entropy = 7.999608 bits per byte.

Optimum compression would reduce the size
of this 500000 byte file by 0 percent.

Chi square distribution for 500000 samples is 271.29, and randomly
would exceed this value 23.08 percent of the times.

Arithmetic mean value of data bytes is 127.3446 (127.5 = random).
Monte Carlo value for Pi is 3.147852591 (error 0.20 percent).
Serial correlation coefficient is 0.000529 (totally uncorrelated = 0.0).

double aRand = MathRand()/32768.0; // MathRand=0-32767, aRand=0-0.999969

Wouldn't MathRand()/32767.0 produce aRand 0 to 1.0?

That's how I do it.

This seems to depend on the specific random generator implementation (language, compiler, etc.)... There's no general rule here and jjc's test suggests that it's not necessarily as bad in MQL4.

^ I concur with gordon's surmising...after reviewing a handful of links provided by that google query the conclusion appears to be that one should be wary of the randomness of any given pseudo-random generator until it is determined to be sufficiently random.

The link pursued and examined by jjc provides the tools necessary to analyze and determine if one's pseudo-random generator is sufficiently random.

jjc's subsequent analyses of MT4's pseudo-random generator would appear to conclude that it is in fact sufficiently random and that one can use the lower bits with confidence.

WHRoeder do you not concur with this line of reasoning? If you do not concur can you please explain (assist us in understanding) where the discrepancy lies?

On subsequent runs of the code I have seen a few percentages in the 0-1 and 99-100 range, but the random number generation does look generally sound.

And if there were a problem with it, something like MathRand()/32768 wouldn't be the answer; that would obviously give you values unevenly spread (or spread in predictable bands) between zero and one. In addition, the page at www.fourmilab.ch isn't saying "don't use the lower bits". It's saying that the Unix random number generator is flawed in toto, with the slight possibility that they've only tested the low byte and that you therefore might possibly be able to get away with doing something like rand() >> 8.

FYI, my test script for generating input into ent.exe goes like this:

// Constants used in call to Win32 CreateFileA()
#define GENERIC_WRITE           1073741824
#define CREATE_ALWAYS           2

// Imports from Win32 for doing file writing (a) anywhere on disk and (b) in a way which
// makes it easy to write single bytes
#import "kernel32.dll"
   int CreateFileA(string Filename, int AccessMode, int ShareMode, int PassAsZero, int CreationMode, int FlagsAndAttributes, int AlsoPassAsZero);
   int WriteFile(int FileHandle, int & BufferPtr[], int BufferLength, int & BytesWritten[], int PassAsZero);
   int CloseHandle(int FileHandle);
#import


int start()
{
   // Array which holds the random number to be written to disk
   int Array[1];
   
   // Array used for receiving (and ignoring) the BytesWritten value passed back from CreateFileA()
   int IgnoreBytesWritten[1];

   // Initialise the random number generator
   MathSrand(GetTickCount());
 
   // Open the file to hold the list of bytes for passing into ent.exe
   int FileHandle = CreateFileA("c:\\temp\\mt4rand.txt", GENERIC_WRITE, 0, 0, CREATE_ALWAYS, 0, 0);
   
   // Generate 500,000 random bytes
   for (int i = 0; i < 500000; i++) {
      // Store a random value in the array which then gets written to disk
      Array[0] = MathRand();
      
      // Only write one byte - the lowest byte - from Array[0] to disk, i.e. the lowest byte of the random number
      WriteFile(FileHandle, Array, 1, IgnoreBytesWritten, 0);
   }
   
   // Close the file
   CloseHandle(FileHandle);
      
   // Done
   MessageBox("Done creation of mt4rand.txt"); 
}

THANKS!!

This was on my to-do list (making code like this) after reading your post original post on the matter way above, saves me a bit'o time.

While true, normally you don't want that.

A random int from 0-9 would be int(10*MathRand()/32768.0)

if you used 32767 then you would still get a random 0-9 except one time out of 32767 you'd get a 10.