how to read an csv-file into a string with the Windows-Functions?

 

Hi,

I know there are examples here usin the obsolet Windows-function like _open(), _read()...

But I think this could be troublesome due to ANSI and UTF8...

Anyway I want to read a csv-File which is admin. by LibreOffce and save LO save it as csv Notepad++ tells me this file is now ANSI.

After Windows-Copy fails every now and then to copy it into \Files (File is blocked while LockHunter tells me nothing is blocking nothing??) I want to read that file directly into a string to be splitted here after.

At the first glance it looks so easy but then I am stuck by the new (annoying) not convertible data types.

string readFileIntoString(string fName){

        if ( !existsFile( fName ) ) { 
                Print("ERROR File-Read does NOT EXIST, File: ",fName);
                SetUserError( ERR_FILE_NOT_EXIST ); return(""); 
        }
        
        int wHdl = CreateFileW(fName, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
        if ( wHdl == INVALID_HANDLE_VALUE ) { 
                Print("ERROR Win32: Invalid Handle #",kernel32::GetLastError());
                SetUserError( ERR_FILE_INVALID_HANDLE ); return(""); 
        }
        int bRead[1];
        int szFile = GetFileSize(wHdl, bRead);
        if ( szFile == 0 ) { 
                Print("ERROR File-Size == 0 ??  File: ",fName);
                SetUserError( ERR_FILE_NOT_EXIST ); return(""); 
        }
   
   int sz=0;   
   int rdBuff[];
   ArrayResize(rdBuff, szFile+1);
   string ret="";
        if ( !ReadFile(wHdl, rdBuff, szFile, bRead, NULL) ) {
                Print("ERROR Win32: Read ERROR #",kernel32::GetLastError(),"   of File: ",fName);
                SetUserError( ERR_FILE_READ_ERROR ); return("");
        }
    CloseHandle(wHdl);
    if(bRead[0]==0) {
                Print("ERROR Win32: Read ERROR #",kernel32::GetLastError(),"   of File: ",fName, " read 0 bytes, while File-Size = ",szFile);
                SetUserError( ERR_FILE_READ_ERROR ); return("");   
   }
   ret=CharArrayToString(rdBuff,0,bRead[0],CP_UTF8);
   return(ret);
}

The simple problem:

'rdBuff' - parameter conversion not allowed ..
// this is:
ret=CharArrayToString(rdBuff,0,bRead[0],CP_UTF8);

Windows requires an int buffer, MT4's

CharArrayToString()

an ushort buffer and both cannot be converted. Wonderful example to make live as complicated as possible.

BTW the Window-Function InternetReadFile() does work with ushort, but I heven't find neither FileRead reading into ushort buffer nor an MT4-function converting int.

Does anybody know what can I do?

Thanks in advance,

Gooly

 
what about ShortArrayToString ?
 
qjol:
what about ShortArrayToString ?

Not compatible:

string  ShortArrayToString(
   ushort  array[],      // array
   int     start=0,      // starting position in the array
   int     count=-1      // number of symbols
   );

the int I get cannot be casted to ushort: incompatible :(

BTW your function is pretty much the same as:

string  CharArrayToString(
   uchar  array[],              // array
   int    start=0,              // starting position in the array
   int    count=-1              // number of symbols
   uint    codepage=CP_ACP      // code page
   );

except the codepage.

But your link gives me that idea:

string  IntegerToString(
   long    number,              // number
   int     str_len=0,           // length of result string
   ushort  fill_symbol=' '      // filler
   );

Now I have to experiment of converting the char of a probaly ANSI-csv-file into a CP_UTF8-wide-string:

   int szFile = ArraySize(rdBuff), c=0;
   for (c=0;c<szFile;c++){ ret = ret+IntegerToString(rdBuff[c], ??, ??); }

Gooly

 
why all the trouble use CharToString
 
qjol:
why all the trouble use CharToString

Doesn't work either:

string  CharToString(
   uchar  char_code      // numeric code of symbol
   );

Windows requires and I get/have an int-array and (s.a.) an int cannot be casted into uchar:

parameter conversion not allowed

that's why I said once the massive inflation of data types is nothing but very annoying - there is NO profit for us..

 
can you please show the function #import and highlight what parameter are we talking about ?
 

ok here is the script.

I attached as well a simple csv-file named test.txt (csv cannot be attached) store in your \Documents.

Now try to to bring it into allLines[] line by line:

//+------------------------------------------------------------------+
//|                                                      readCSV.mq4 |
//|                        Copyright 2014, MetaQuotes Software Corp. |
//|                                              https://www.mql5.com |
//+------------------------------------------------------------------+
#property copyright "Copyright 2014, MetaQuotes Software Corp."
#property link      "https://www.mql5.com"
#property version   "1.00"
#property strict
//+------------------------------------------------------------------+
//| Script program start function                                    |
//+------------------------------------------------------------------+
void OnStart()
  {
//---
        string fName = "test.txt"; // attached, place directly in ..{user}\Documents 
        string g = "12345678 1 2345678 2 2345678 3 2345678 4 2345678 5 2345678 6 2345678 7 2345678 8 2345678 9 23456789";
        GetEnvironmentVariableW("USERPROFILE", g, StringLen(g));
        string CentreFolder = g+"\\Documents\\"; 
        string srcFile = CentreFolder + fName,
        strFile = readFileIntoString( srcFile );
        string arrFile[];
        int allLines = StringSplit(strFile,'\n',arrFile);


   
  }
//+------------------------------------------------------------------+



#import "kernel32.dll" //http://msdn.microsoft.com/en-us/library/windows/desktop/aa365467%28v=vs.85%29.aspx
   int  CreateFileW(string lpFileName, int dwDesiredAccess, int dwShareMode, int lpSecurityAttributes, int dwCreationDisposition, int dwFlagsAndAttributes, int hTemplateFile);
   int  CreateFileA(string lpFileName, int dwDesiredAccess, int dwShareMode, int lpSecurityAttributes, int dwCreationDisposition, int dwFlagsAndAttributes, int hTemplateFile);
   int  GetFileSize(int hFile, int& lpFileSizeHigh[]);
   int  ReadFile(int hFile, int &lpBuffer[], int nNumberOfBytesToRead, int &lpNumberOfBytesRead[], int lpOverlapped);
   int  CloseHandle(int hObject);
        int  GetEnvironmentVariableW (string lpName, string& lpBuffer, int nSize);
        //Attentione normal GetLastError() hast to be changed to ::GetLastError() !!!
        int  GetLastError(void); // use: Print("CopyFileW error#",kernel32::GetLastError()," mql5 error#",::GetLastError()); 
        // error codes: http://msdn.microsoft.com/en-us/library/windows/desktop/ms681382%28v=vs.85%29.aspx
#import

#define GENERIC_READ                     0x80000000
#define GENERIC_WRITE                    0x40000000
#define INVALID_HANDLE_VALUE                     -1
#define WinNULL  0x00000000
#define WinFILE_SHARE_READ 0x00000001
#define  OPEN_EXISTING   3
#define  FILE_ATTRIBUTE_NORMAL   128 // (0x80)


bool existsFile(string FileName) {
   int FileHandle = CreateFileW(FileName, 0, 0, 0, OPEN_EXISTING, 0, 0);
   if (FileHandle != -1) {
      CloseHandle(FileHandle);
      return (true);
   } else {
      return (false);
   }
}


string readFileIntoString(string fName){

        if ( !existsFile( fName ) ) { 
                Print("ERROR File-Read does NOT EXIST, File: ",fName);
                SetUserError( ERR_FILE_NOT_EXIST ); return(""); 
        }
        
        int wHdl = CreateFileW(fName, GENERIC_READ, WinFILE_SHARE_READ, WinNULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, WinNULL);
        if ( wHdl == INVALID_HANDLE_VALUE ) { 
                int WinErr = kernel32::GetLastError();
                Print("ERROR Win32: Invalid Handle ",kernel32::GetLastError());
                SetUserError( ERR_FILE_INVALID_HANDLE ); return(""); 
        }
        int bRead[1];
        int szFile = GetFileSize(wHdl, bRead);
        if ( szFile == 0 ) { 
                Print("ERROR File-Size == 0 ??  File: ",fName);
                SetUserError( ERR_FILE_NOT_EXIST ); return(""); 
        }
   
   int c,sz=0;   
   int rdBuff[];
   ArrayResize(rdBuff, szFile+1);
   string ret="";
        if ( !ReadFile(wHdl, rdBuff, szFile, bRead, NULL) ) {
                Print("ERROR Win32: Read ERROR: ",kernel32::GetLastError(),"   of File: ",fName);
                SetUserError( ERR_FILE_READ_ERROR ); return("");
        }
        CloseHandle(wHdl);
   if(bRead[0]==0) {
                Print("ERROR Win32: Read ERROR: ",kernel32::GetLastError(),"   of File: ",fName, " read 0 bytes, while File-Size = ",szFile);
                SetUserError( ERR_FILE_READ_ERROR ); return("");   
   }
   szFile = ArraySize(rdBuff); c=0;
   for (c=0;c<szFile;c++){ ret = ret+IntegerToString(rdBuff[c]); }
   int szStr = StringLen(ret);
   //ret=CharArrayToString(rdBuff,0,bRead[0],CP_UTF8);
   return(ret);
}

Try to de-comment

//ret=CharArrayToString(rdBuff,0,bRead[0],CP_UTF8);

you'll get an compiler error. And if you define rdBuff as ushort you will get again a comiler error:

'rdBuff' - parameter conversion not allowed     readCSV.mq4     88      23

Thank you MetaTrader for the brainteaser.

Goolly

Files:
test.txt  1 kb
 
have a look on winfile v600
 
you should use SHGetFolderPathW instead of GetEnvironmentVariableW
 

THANKS!! Now it works! I need to change just:

   int  ReadFile(int hFile, uchar& lpBuffer[], int nNumberOfBytesToRead, int &lpNumberOfBytesRead[], int lpOverlapped);
..
   uchar  rdBuff[];
..
   return( CharArrayToString( rdBuff,0,-1, CP_UTF8 ) );

(couldn't think of changing the imported Windows-finctions)

 
qjol:
you should use SHGetFolderPathW instead of GetEnvironmentVariableW

Why? It seems to be: "Deprecated. Gets the path of a folder identified by a CSIDL value".
Reason: