trying to embed the python interpreter -> crash

New comment
 
Hello,

i have a minimal example that will crash metatrader:

Code:
#import "python26.dll"
void Py_Initialize();
void Py_Finalize();
int PyRun_SimpleString(string code);
#import


int init(){
    Py_Initialize();
    PyRun_SimpleString("x=42");
    Py_Finalize();
   
    return(0);
}
From what i have read elsewhere using string in mql4 will pass a char* pointer to the function and thats exactly what this particular function wants. writing a dll with freepascal that takes the exact same argument (PChar) and passes it unmodifed to PyRun_SimpleString() will work.

Can anybody enlighten me why the above code will provoke a crash, although the argument passed to it is indeed a pointer to the string, exactly as required by the function? And why it will *not* crash when i define the *exact* same function in my own dll and just pass on the unmodified argument to the original function in python26.dll?

TIA,
Bernd
 

Functions imported from DLL into an mql4 program must provide convention __stdcall.

python26.dll not provide this convention.

 
Ilnur:

Functions imported from DLL into an mql4 program must provide convention __stdcall.

python26.dll not provide this convention.

but when using the same function from freepascal i have to import the function like this:


function PyRun_SimpleString(command : PChar) : longint; stdcall; external 'python26.dll';


if i leave out the stdcall keyword here then it won't work. the same is true when exportig functions from freepascal to metatrader.


This made me believe that python26.dll also uses the stdcall convention. Isn't the stdcall from pascal identical to the __stdcall from C? How many different calling conventions are there? I don't know enough C to be able to find the exact definition of the export of this function in the huge ammount of python sources and what it exactly means what is written there.


What convention does python26.dll use that forces me to import it with stdcall in pascal and still dont work in metatrader?

 
7bit:

but when using the same function from freepascal i have to import the function like this:


function PyRun_SimpleString(command : PChar) : longint; stdcall; external 'python26.dll';


if i leave out the stdcall keyword here then it won't work.


I don't understand, how it works from freepascal.  I don't know freepascal.


This made me believe that python26.dll also uses the stdcall convention.


The __stdcall convention is characterized by the following: the imported function will flush the stack by itself.

But functions from python26.dll don't flush the stack.  For example



From the disassembled listing we see that in function two parametres are passed.

Thus function does not flush the stack (instead of retn should be retn 8). Therefore I have assumed that python26.dll not provide __stdcall convention.

 
Ilnur:


From the disassembled listing we see that in function two parametres are passed.

Thus function does not flush the stack (instead of retn should be retn 8). Therefore I have assumed that python26.dll not provide __stdcall convention.


You speak assembler fluently, the last time i did assembler was back in the time when 6510 processors were built into home computers.


Since i cannot sleep well until i have fully understood what is going on i have one more question to you: Would you please be so kind and have a look into this dll with your disassembler: http://7bit.ath.cx/forex/code/py26/mt4/libraries/py26.dll and try to find out what is going on when the function PyExecute() is called?


It is defined as follows:

procedure PyExecute(code : PChar); stdcall;
begin
PyRun_SimpleString(code);
end;


the import of PyRun_SimpleString is also defined as stdcall.


according to your last posting the function should pop the stack before returning but since it also calls the external function with stdcall it should *not* pop the stack after the external function returns. but this would contradict your findings about the functions in pyton26.dll, however, this code seems to run fine.


Bernd

 
7bit:


Since i cannot sleep well until i have fully understood what is going on i have one more question to you: Would you please be so kind and have a look into this dll with your disassembler: http://7bit.ath.cx/forex/code/py26/mt4/libraries/py26.dll and try to find out what is going on when the function PyExecute() is called?

Sorry for my English. I'm russian and speak english badly :)


PyExecute() provide __stdcall convention. Look

The function pop the stack by itself (retn 4). Therefore this function should work with MetaTrader.

 
Ilnur:

Sorry for my English. I'm russian and speak english badly :)


PyExecute() provide __stdcall convention. Look

The function pop the stack by itself (retn 4). Therefore this function should work with MetaTrader.


Thank you for taking the time to care about my assembler-newbie like questions ;-)


but i have yet another question: the function in my dll calls PyRun_SimpleString() in python26.dll (which does not pop the stack itself as you showed me earlier) so my function should be responsible to restore the stack to the state it was before calling PyRun_SimpleString() after this call but before returning. But all i can see is the command leave. Does leave have something to do with it? (sorry, my x86 assembler knowledge is very limited)


(i call the external function in python26.dll with stdcall which should be wrong according to what i have learned so far, yet it seems to work, how can this be? Shouldn't it immediately crash because the return address is completely wrong when the stack is messed up?)


Thank you for your efforts,

Bernd



PS: what software are you using to disassemble the code?

 
7bit:

But all i can see is the command leave. Does leave have something to do with it? (sorry, my x86 assembler knowledge is very limited)


LEAVE remove data from the top of the stack while setting up a stack frame pointer EBP.


i call the external function in python26.dll with stdcall which should be wrong according to what i have learned so far, yet it seems to work, how can this be? Shouldn't it immediately crash because the return address is completely wrong when the stack is messed up?


I don't understand, how it works.  Probably freepascal restore the stack by itself. 


PS: what software are you using to disassemble the code?


The Interactive Dissassembler - DataRescue IDA PRO 5.2

 
Ilnur:


LEAVE remove data from the top of the stack while setting up a stack frame pointer EBP.



I don't understand, how it works. Probably freepascal restore the stack by itself.



The Interactive Dissassembler - DataRescue IDA PRO 5.2


thanks to your help i think i could now answer all my questions. I installed the free version if IDA and could do some more research.


The reason for my initial confusion was that i wrongly assumed that leaving out the stdcall keyword would produce cdecl calls, thus my initial doubt about cdecl being correct. I must explicitely specify cdecl, otherwise the FPC compiler will silently use pascal calling conventions which are the old Borland TurboPascal / Delphi standard and are completely incompatible with everything else, thus the crash when not explicitely specifying a calling convention.


when using cdecl i can see in IDA that FPC will insert code to pop the stack after the call, when using stdcall it does not generate this code. It seems that it is pure coincidence that it doesn't immediately crash with stdcall, somehow the stack must get restored before desaster can happen. cdecl will not crash also and i can be sure that the stack is handled properly.


Again i have learnd something ;-) this is the first time since more than 10 years that i try to produce (and fight with problems of) native machine code and such low level details, in the last years i only used very high level languages like python, cared about theoretic concepts like lambda calculus and functional programming but never ever wanted to go back so deep that i had to care about machine language and stack pointers ;-) (long forgotten the old days when i used to program a C64 with 6510 assembler, using the disassembler listing of the kernel and the circuit diagram of the mainboard as a reference)


Thanks again,

Bernd

 

I have now something workable: if you are interested: It can be found here:


http://sites.google.com/site/prof7bit/metatrader-python-integration

 
Thanks!
12
New comment
Reason: