How to System::Call DLL function with reference?
 

I have a DLL with an exported function with the following signature:

extern "C" __declspec(dllexport) HRESULT Run4( HINSTANCE, HINSTANCE, HINSTANCE, LPSTR, int, bool& )

How do I call this function and receive the 'bool&' back into the NSIS script?

The best I can come up with is:

System::Call "MyDLL::Run4( i 0, i 0, i 0, t '', *i .r4 ), i.r3" ? u

But the DLL segfaults when it tries to write to the boolean reference. I assume this means I'm incorrectly passing in the reference to $4. I've tried this:

System::Alloc 4
Pop $4
System::Call "MyDLL::Run4( i 0, i 0, i 0, t '', *i .r4 ), i.r3" ? u

And this:

System::Alloc 4
Pop $4
System::Call "MyDLL::Run4( i 0, i 0, i 0, t '', i .r4 ), i.r3" ? u

And this:

System::Call "MyDLL::Run4( i 0, i 0, i 0, t '', *i 0 r4 ), i.r3" ? u

But no effect. Can you see what mistake I'm making, or is what I'm trying to do not possible?

-david

PS: It works fine if I comment out the line assigning the value to the 'bool' in the DLL, so I do believe it's been narrowed down to this one issue.

Let's see:
Now converted to system syntax:
I don't have the dll, so I can't test it.

Alas, no luck. Every time it just segfaults when I try to write to the 'bool&' parameter. :(

Can't you use BOOL with TRUE and FALSE?
It's just an int with 0 or 1 then.

-Stu

Unfortunately I can't change the function signature for backwards compatability reasons. Were it up to me I'd just return 'true' or 'false' in the return value. Unfortunately, I'm stuck with a reference to a 'bool'.

Now, I *think* a reference looks just like a pointer on the stack, and is just treated special by the compiler. Thus I'd assume the standard pointer semantics of the System::Call would work. But unfortunately I can't get them too. Can you come up with any other variations I might try?

-david

Can you post the DLL?

Ok, I'll put together a DLL with the same function signature and post a link here. Thanks for all your help on this.

In the meantime, I've decided to temporarily drop the backwards compatibility requirement and go with a function that's:

HRESULT Run( )

Much simpler, works like a charm. However, now I've discovered another hangup: the DLL filename contains a hyphen, and that seems to muck it up. Specifically, my code is:

System::Call "$2::Run( ) i.r3 ? u"

So far as I can tell, this works great so long as $2 is a purely alphabetical name (eg, "MyDLL"). But a DLL name with hyphens (eg, "MyDLL-1.2-3") appears to screw it up.

Can you recommend any way to pass in a hyphenated DLL name?

Again, thanks for all your help with this.

-david

Extract it with a hyphen-less name. You can even use GetTempFileName.

Eureka! I figured out the filename issue. My DLL filename is of the following form:

MyDLL-X.YYY-ZZZ.dll

And I was calling as follows:

code:
System::Call "MyDLL-X.YYY-ZZZ::Run( ) i.r3 ? u"

However, the 'LoadLibrary' Win32 function only appends the ".dll" to the end when searching if the name passed in doesn't have an extension. And it assumed my DLL had the extension "YYY-ZZZ" and thus failed. The solution is to explicitly indicate the ".dll" extension in the System::Call as follows:

code:
System::Call "MyDLL-X.YYY-ZZZ.dll::Run( ) i.r3 ? u"

So, that fixes that up. Next I need to get the "bool&" function signature to work, but this lets me keep moving forward. Thanks!

-david