Old 6th November 2009, 12:02   #321
Anders
Moderator
 
Anders's Avatar
 
Join Date: Jun 2002
Location: ${NSISDIR}
Posts: 4,507
@Wizou: If you check my BgWorker plugin, you can see that I'm checking for kernel32::lstrcatW in the import table.

I also have some sample code in there that checks $pluginsdir, but like you said, you don't know the offset, so it has to use IsBadReadPtr() and there could be false positives.

Unfortunately, it turns out that writing hybrid plugins is a huge pain in the ass, and I can't recommend doing it on anything except very simple plugins

IntOp $PostCount $PostCount + 1
Anders is offline   Reply With Quote
Old 6th November 2009, 13:50   #322
jimpark
Senior Member
 
Join Date: Sep 2007
Posts: 204
For the next release, I plan on adding to pluginapi.h:

// True if NSIS is built with Unicode support.
bool NSISCALL IsUnicodeNSIS();

Also, Olivier noted that you could use IsWindowUnicode(hwnd). But this may not work on silent installers.

However, I would still encourage that the plugin writers use TCHARs or a strategy similar to it and compile and link with the right Unicode declarations and definitions. I know there is a temptation to try to generate a single binary that could work on both the ANSI and the Unicode NSIS variants but that seems a bit dangerous to me.

Unicode NSIS advocate -- http://www.scratchpaper.com for latest build and source.
jimpark is offline   Reply With Quote
Old 6th November 2009, 13:57   #323
Wizou
Senior Member
 
Join Date: Aug 2007
Location: Paris, France
Posts: 304
@Anders: IsBadReadPtr won't help you here as it will be valid memory range (.bss variables area), just you're not checking a valid Unicode 2nd byte.
Checking 'lstrcatW' import from main module is however a good idea

@jim park: I still wonder how you would create such a IsUnicodeNSIS, unless you pass special exec_flags_t value from NSIS to the DLL


If we have such a way for the DLL to detect the version of NSIS, I'm thinking about creating a new pluginapi.h/pluginapi.lib that could transparently convert variables and stack from ANSI or Unicode to the plugins choice depending on whether it is called from Unicode-NSIS or not
This would make plugins creator's life easier.
Wizou is offline   Reply With Quote
Old 6th November 2009, 13:58   #324
dariann
Junior Member
 
Join Date: Mar 2009
Posts: 2
@jimpark: What about turning on/off "messages about including files" ? NSIS Unicode is great tool and I ask You to add this option, please.
dariann is offline   Reply With Quote
Old 6th November 2009, 15:01   #325
Anders
Moderator
 
Anders's Avatar
 
Join Date: Jun 2002
Location: ${NSISDIR}
Posts: 4,507
Noooooooooo, we don't need a IsUnicodeNSIS() function or any kind of pluginapi helpers to do both.

It has already been done with the unicode layer for win9x, just use that.

I'm with Jim on this, plugin devs should use TCHAR and compile two versions. There is also my CallAnsiPlugin plugin for older stuff with no source.

@Wizou: well, I'm sure it can be done without checking the import table. The IsBadReadPtr usage was just a big hack and I never finished that type of detection.

IntOp $PostCount $PostCount + 1
Anders is offline   Reply With Quote
Old 6th November 2009, 20:25   #326
jimpark
Senior Member
 
Join Date: Sep 2007
Posts: 204
@Wizou: That's a good point. It's very difficult to write an IsUnicodeNSIS(). The only thing I could do is try validating the string (if anything) to see if it's good UTF16LE. Look in validateunicode.cpp which is in Source. You'll see a class there that can validate UTF16LE/BE as well as UTF8.

@Anders: I agree with you in that this API, even if accurate, is a bad idea. I'm pulling IsUnicodeNSIS() from pluginapi.h. It will NOT be in the next release.

Unicode NSIS advocate -- http://www.scratchpaper.com for latest build and source.
jimpark is offline   Reply With Quote
Old 10th November 2009, 13:50   #327
ChocJunkie
Senior Member
 
Join Date: Oct 2009
Location: Germany
Posts: 120
Does someone know why I'm always getting empty unicode files when I'm using:
code:
unicode::FileUnicode2Ansi "${_HDDS_countFile}" "${_HDDS_countFile}_tmp" AUTO
Pop $R0
${If} "$R0" != 0
# error handling
${EndIf}


The source files type is "UTF-16LE|UCS-2LE".

-- Problem solved ---

I was using Unicode plugin v1.1.
I've tried version 1.0 and everything is fine now.

Last edited by ChocJunkie; 10th November 2009 at 14:10.
ChocJunkie is offline   Reply With Quote
Old 15th November 2009, 22:39   #328
Anders
Moderator
 
Anders's Avatar
 
Join Date: Jun 2002
Location: ${NSISDIR}
Posts: 4,507
If we are going to have any chance of a merge, the examples and header files need to be 100% the same, meaning that both the current A and U builds need some changes:

*The ansi version needs to change its handling of &wXX so that XX is a count of characters and not bytes (and update the docs) (The unicode version already has this fixed)

*The unicode system plugin needs to default to the W version of functions when autodetecting (foo >> fooW and not fooA)


To reduce the NSIS_MAX_STRLEN confusion, we should probably make it really clear in the docs that NSIS_MAX_STRLEN is a count of TCHAR's (Or maybe even rename it). We should also add a predefine for the count of bytes, I suggest we name it NSIS_MAX_STRCB

Both versions currently fail this little test script

IntOp $PostCount $PostCount + 1
Anders is offline   Reply With Quote
Old 16th November 2009, 13:54   #329
jimpark
Senior Member
 
Join Date: Sep 2007
Posts: 204
Quote:
Originally posted by Anders
*The unicode system plugin needs to default to the W version of functions when autodetecting (foo >> fooW and not fooA)
We actually do default to the "W" version. Let me explain a little with how the "T" versions of the functions work for those who don't know.

We write in our code:

code:

::CreateFile(...);



When you look at the definition of CreateFile, you see that there really is no function called "CreateFile" but two functions called "CreateFileW" and "CreateFileA." This is true for MOST of the functions that have both the A and the W versions.

code:

#ifdef UNICODE
#define CreateFile CreateFileW
#else
#define CreateFile CreateFileA
#endif // !UNICODE



There are functions which only have one name, with no W or A prefixes.

So the logic that is in the system plugin currently is to try to load the function with the name as is, then failing that, try to add "W" for the Unicode build (or "A" for the ANSI build) and try again.

Unfortunately, for older string functions like lstrlen, the lib you are trying to load DOES have a lstrlen which is ANSI only. It's probably there for backward compatibility since it mimics standard library string functions. So for lstrlen, you need to specify kernel::lstrlenW. But as far as I've seen, it's only these string functions that look and behave like the standard library ones that have this problem.

A possible change could be that we always suffix with a "W" or "A" and see if that loads something and try that function first before we try the name of the function unmodified, but I'm uncomfortable with that since I'd rather give the programmer the benefit of the doubt and let him/her decide what s/he really wants to load.

I don't mind the plugin second guessing after failures but I don't want it to think it's smarter than the programmer.

Quote:
To reduce the NSIS_MAX_STRLEN confusion, we should probably make it really clear in the docs that NSIS_MAX_STRLEN is a count of TCHAR's (Or maybe even rename it). We should also add a predefine for the count of bytes, I suggest we name it NSIS_MAX_STRCB

Both versions currently fail this little test script [/B]
I agree that we should note that NSIS_MAX_STRLEN is in TCHARS. But going through the documentation, they all state that NSIS_MAX_STRLEN means a number of characters, not bytes. So I'm actually okay with the documentation as-is.

As for adding NSIS_MAX_STRCB, I think that's a great idea, although, it can easily be calculated by NSIS_MAX_STRLEN * NSIS_CHAR_SIZE. I like NSIS_MAX_STR_BYTES better as a name, though.

Unless the NSIS trunk has NSIS_MAX_STR_BYTES, it would be useless for me to add it just to the Unicode version as it's meant for portability across the two versions.

So perhaps we can request NSIS_MAX_STR_BYTES to be added to the NSIS trunk?

Unicode NSIS advocate -- http://www.scratchpaper.com for latest build and source.
jimpark is offline   Reply With Quote
Old 16th November 2009, 14:24   #330
Wizou
Senior Member
 
Join Date: Aug 2007
Location: Paris, France
Posts: 304
For your information, I'm starting to merge Unicode port of NSIS version into NSIS repository.

I will try to take the most of what has been discussed here into account.
However my plan is first to make it the most transparent possible for the user and the plug-ins developer.
Wizou is offline   Reply With Quote
Old 16th November 2009, 14:34   #331
jimpark
Senior Member
 
Join Date: Sep 2007
Posts: 204
That's excellent news! Is there an ETA on when this will be complete?

Unicode NSIS advocate -- http://www.scratchpaper.com for latest build and source.
jimpark is offline   Reply With Quote
Old 16th November 2009, 14:42   #332
Wizou
Senior Member
 
Join Date: Aug 2007
Location: Paris, France
Posts: 304
No ETA yet.. I just started the SVN branch I will work on..
I guess the first merge/port will be easy especially as your work will greatly simplify the effort.
But then there will be a phase of discussion/arbitration/tweaking/testing as to what should be the user & developer experience regarding this new version.
I will keep you updated on my progress.
Wizou is offline   Reply With Quote
Old 16th November 2009, 18:38   #333
Anders
Moderator
 
Anders's Avatar
 
Join Date: Jun 2002
Location: ${NSISDIR}
Posts: 4,507
well, if we don't look for "functionW" before "function", we are going to need some kind of define that is "A" or "W" and maybe a list of API's where it is required. IMHO that is the wrong approach, and the reverse is needed, they need to look for the char type specific version first unless a (new) option is provided in the system call

IntOp $PostCount $PostCount + 1
Anders is offline   Reply With Quote
Old 17th November 2009, 07:31   #334
Wizou
Senior Member
 
Join Date: Aug 2007
Location: Paris, France
Posts: 304
my opinion is to check whether the parameter type contains a "t", then we know we are facing an API which has two variant..
and only in this case, we start checking for 'functionW' or 'functionA' variant first.. if entrypoint is not found, then we check for 'function'

Last edited by Wizou; 17th November 2009 at 09:16.
Wizou is offline   Reply With Quote
Old 17th November 2009, 12:41   #335
jimpark
Senior Member
 
Join Date: Sep 2007
Posts: 204
@Wizou: the problem with that check is that if the user calls CreateFile() with 'w' in the Unicode version of NSIS, that's not wrong either. CreateFile should still resolve to CreateFileW.

Like I said, the only problems I've seen are the string functions that mimic standard library calls. There aren't that many and they are all lowercased like C standard library functions. If the person has programmed C at all, functions like strlen should jump out as taking char* and not wchar_t*. So it should raise a warning flag and the user should realize he should put a W at the end.

Unicode NSIS advocate -- http://www.scratchpaper.com for latest build and source.
jimpark is offline   Reply With Quote
Old 17th November 2009, 13:10   #336
Anders
Moderator
 
Anders's Avatar
 
Join Date: Jun 2002
Location: ${NSISDIR}
Posts: 4,507
Windows programmers don't think about that, I'm so used to just calling lstr*, I don't think about the char type. Whats wrong by defaulting to the W versions and only falling back if W does not exists or a option to disable it was set?

Do you have a single example of a function that needs to be called without W when running as unicode (meaning foo should be called with WCHAR* even if fooW exists)?

IntOp $PostCount $PostCount + 1
Anders is offline   Reply With Quote
Old 17th November 2009, 13:22   #337
Wizou
Senior Member
 
Join Date: Aug 2007
Location: Paris, France
Posts: 304
Quote:
Originally posted by jimpark
@Wizou: the problem with that check is that if the user calls CreateFile() with 'w' in the Unicode version of NSIS, that's not wrong either. CreateFile should still resolve to CreateFileW.
I don't agree with you. This is wrong.
If the user explicitely wants to call the Unicode version using 'w' as parameter type, he has to specify CreateFileW as the function name.

Summary:
* the user wants to call explicitely CreateFileA => he uses function name CreateFileA and parameter type 'm'
* the user wants to call explicitely CreateFileW => he uses function name CreateFileW and parameter type 'w'
* the user wants to call CreateFile whatever suits best according to NSIS variant => he uses function name CreateFile and parameter type 't'
* outside the presence of parameter type t, we can't suppose the existence of a A/W suffix, and it seems to me it would be wrong to try to add one (it would risk to reach another API which is unrelated)

Last edited by Wizou; 17th November 2009 at 14:17.
Wizou is offline   Reply With Quote
Old 17th November 2009, 14:10   #338
Anders
Moderator
 
Anders's Avatar
 
Join Date: Jun 2002
Location: ${NSISDIR}
Posts: 4,507
Yeah, I agree with Wizou

IntOp $PostCount $PostCount + 1
Anders is offline   Reply With Quote
Old 17th November 2009, 17:43   #339
jimpark
Senior Member
 
Join Date: Sep 2007
Posts: 204
That would be nice if everyone did that. But remember, when people read the Windows documentation, what do they read? The documentation says to use CreateFile(). It does not say to use CreateFileW() for Unicode. We're just lucky that Microsoft has been largely consistent with these W and A suffixes.

If you write C/C++, you will also be calling CreateFile() with wchar_t*. Because that's what the documentation tells you to do. You just have to make sure you define some macro like _UNICODE or UNICODE. If you read the fine print you get the "is implemented as CreateFileW() and CreateFileA()" but no example tells you to use these directly.

And remember, there's really no bearing on type-safety here. I can use any old function signature and as long as the name of the function matches, the System plugin will push your arguments, whatever types they may be, onto the stack and happily call the function. The arguments can be totally wrong types and have bogus values. So it's not even useful in that respects either.

Unicode NSIS advocate -- http://www.scratchpaper.com for latest build and source.
jimpark is offline   Reply With Quote
Old 17th November 2009, 17:58   #340
Pidgeot
Senior Member
 
Pidgeot's Avatar
 
Join Date: Jan 2002
Location: Denmark
Posts: 136
To the best of my knowledge, there IS no CreateFile-function, only CreateFileA and CreateFileW. CreateFile is #defined to CreateFileA or CreateFileW in WinBase.h, depending on whether or not UNICODE is defined.

As such, if the Sytem plug-in allows you to just write CreateFile, then it must either already be checking for *A, or it has some predefined mappings.
Pidgeot is offline   Reply With Quote
Old 17th November 2009, 18:02   #341
jimpark
Senior Member
 
Join Date: Sep 2007
Posts: 204
Quote:
Originally posted by Anders
Windows programmers don't think about that, I'm so used to just calling lstr*, I don't think about the char type. Whats wrong by defaulting to the W versions and only falling back if W does not exists or a option to disable it was set?

Do you have a single example of a function that needs to be called without W when running as unicode (meaning foo should be called with WCHAR* even if fooW exists)?
I believe there are calls in the Windows API where there is no ANSI counterparts so the "W" suffix version would fail. But I don't know of a case where the "W" suffix version would also be a different function. That doesn't mean that they don't exist or will never exist.

However, remember, this plugin architecture can call other libraries than those provided by Microsoft. And who knows what other people do.

Basically, as I mentioned before, it is wrong to second guess the user from the onset. Don't you hate it when your word processor "corrects" your typing when you type some acronym thinking it's SMARTER than you? I don't believe programs should behave that way. And I certainly don't want a programming tool to behave that way.

User: "I want to call Foo()."

Computer: "No, you don't. You want Bar(). Bar is nicer."

User: "No, I want Foo()!"

Computer: "Hey, I found Bar()! Look at that. It's a function that exists. Let's just call Bar()."

User: "I said, call Foo()!!!"

Computer: "Silly user. I called Bar() for you which is much nicer and that caused a booboo. You better go fix it."

What a nightmare!

Unicode NSIS advocate -- http://www.scratchpaper.com for latest build and source.
jimpark is offline   Reply With Quote
Old 17th November 2009, 18:04   #342
jimpark
Senior Member
 
Join Date: Sep 2007
Posts: 204
@Pidgeot: Yes, you are right. There is no CreateFile() function in the library you link to. There's a macro. But the documentation tells you to call CreateFile().

Unicode NSIS advocate -- http://www.scratchpaper.com for latest build and source.
jimpark is offline   Reply With Quote
Old 17th November 2009, 19:06   #343
Anders
Moderator
 
Anders's Avatar
 
Join Date: Jun 2002
Location: ${NSISDIR}
Posts: 4,507
Quote:
Originally posted by jimpark
That would be nice if everyone did that. But remember, when people read the Windows documentation, what do they read? The documentation says to use CreateFile(). It does not say to use CreateFileW() for Unicode. We're just lucky that Microsoft has been largely consistent with these W and A suffixes.

If you write C/C++, you will also be calling CreateFile() with wchar_t*. Because that's what the documentation tells you to do. You just have to make sure you define some macro like _UNICODE or UNICODE. If you read the fine print you get the "is implemented as CreateFileW() and CreateFileA()" but no example tells you to use these directly.
You will not be using wchar_t*, you will be using LPTSTR. t is the system plugins version of LPTSTR. The minute you start using w / wchar*, you already know you are working with unicode only. In 99.99% of cases, if there is a t, you need to look for the W/A versions first. This is the most compatible approach (And the only way to achieve A/U compatibility with existing code)

IntOp $PostCount $PostCount + 1
Anders is offline   Reply With Quote
Old 17th November 2009, 20:05   #344
jimpark
Senior Member
 
Join Date: Sep 2007
Posts: 204
@Anders: What you say is true. But if you are coding a Unicode only app, you will notice that people use wchar_t with CreateFile() and not CreateFileW().

Either way, I don't like the system to think it's smarter than the programmer.

Unicode NSIS advocate -- http://www.scratchpaper.com for latest build and source.
jimpark is offline   Reply With Quote
Old 17th November 2009, 20:45   #345
Anders
Moderator
 
Anders's Avatar
 
Join Date: Jun 2002
Location: ${NSISDIR}
Posts: 4,507
but we sort of have to be, in windows/C world, this stuff happens at compile time, for us it happens at runtime, and so, t is our clue that the user trusts us and expects us to the right thing

(OT: Why don't you two hang out in IRC, this discussion would go a lot faster that way)

IntOp $PostCount $PostCount + 1
Anders is offline   Reply With Quote
Old 17th November 2009, 20:50   #346
jimpark
Senior Member
 
Join Date: Sep 2007
Posts: 204
Unfortunately, my access to the internet is very limited. So I can't IRC. This is also what keeps me from being able to work on the NSIS repository directly (and hence that's why I haven't created a branch in the repository for my stuff here). So once the group actually takes my code into the repository, I can't work on it anymore. Anyway, that's my situation and will be for the foreseeable future.

Unicode NSIS advocate -- http://www.scratchpaper.com for latest build and source.
jimpark is offline   Reply With Quote
Old 17th November 2009, 21:04   #347
Anders
Moderator
 
Anders's Avatar
 
Join Date: Jun 2002
Location: ${NSISDIR}
Posts: 4,507
Well, there are HTTP IRC "clients" that work just fine, try mibbit

IntOp $PostCount $PostCount + 1
Anders is offline   Reply With Quote
Old 18th November 2009, 07:54   #348
Wizou
Senior Member
 
Join Date: Aug 2007
Location: Paris, France
Posts: 304
You are both right...
The user doesn't want to care or know about CreateFileA or W, he will just use CreateFile... and parameter type 't', and there will be no problem.

I think the few users that care to write their own System::Call (rather than copy/pasting them from somewhere) should know about TCHAR/LPTSTR and A/W variant.
Wizou is offline   Reply With Quote
Old 21st January 2010, 20:54   #349
gringoloco023
Member
 
Join Date: Nov 2009
Posts: 52
About recompiling plug-ins ?

Hi jimpark,

Lately, I've been studying some C, and got the hang of recompiling plug-ins. I've done 2 so far, and almost/basically got the registry plug-in done.
Using an NSIS .nsh script instead of the plug-ins just ended up being to slow

My question is about including <nsis/pluginapi.h>, if I just use this, the script is missing the pop- & push string/integer functions.

As I read topics about this issue in this forum, one stated I should include pluginapi.c, another stated I should use the pluginapi.lib.

Myself, I only got the scripts recompiled including <nsis/pluginapi.c>. Which includes pluginapi.h, anyway. And as I understand including Exdll.h, isn't necessary anymore?

Is this the way you advise me to compile Unicode NSIS plug-ins ???

Or should I implement pluginapi.lib some how ?
I only found this file in the installed version of Unicode NSIS, in the Examples\Plugins

I am using Microsoft Visual Studio 2008 express edition.

See my forum topic at : portableapps.com/node/21879
EnumINI.7z, is using the pluginapi.c
NewTextReplace, still uses it's own pop & push functions, but maybe I should change that one time.
gringoloco023 is offline   Reply With Quote
Old 22nd January 2010, 12:51   #350
jimpark
Senior Member
 
Join Date: Sep 2007
Posts: 204
It's very admirable of you to take the initiative. I'd advise that you download the source code for the Unicode NSIS and look at the Contrib folder. All of these are plugins. For example, a simple one to imitate is the Banner plugin and see what I did there to support Unicode.

Unicode NSIS advocate -- http://www.scratchpaper.com for latest build and source.
jimpark is offline   Reply With Quote
Old 22nd January 2010, 14:01   #351
gringoloco023
Member
 
Join Date: Nov 2009
Posts: 52
Done all of that a while ago. I needed the examples.

As well I got pluginapi.lib linked by VC++ now. So it's all fine ! But including pluginapi.c worked as well.

Finally I got an reply of the developer of the MD5dll plug-in, he will do this plugin soon !
gringoloco023 is offline   Reply With Quote
Old 23rd January 2010, 21:51   #352
Pawel
Moderator
 
Pawel's Avatar
 
Join Date: Aug 2004
Location: Poland
Posts: 506
Send a message via ICQ to Pawel
@Jim Park
Are you gonna compile NSIS 2.46 based Unicode build?

@All
Could someone with skills compile this ANSI NSIS plugins to UNICODE version?
1. CrcCheck.dll
http://nsis.sourceforge.net/CRCCheck_plug-in
2. Linker.dll
http://nsis.sourceforge.net/Linker_plug-in

-Pawel
Pawel is offline   Reply With Quote
Old 27th January 2010, 07:14   #353
tolan
Junior Member
 
Join Date: Nov 2006
Posts: 13
GJ with new version jimpark!

However I still get messages about BOMs with /V0 and /V1 switches.

Details here: http://forums.winamp.com/showthread....08#post2497708
Mr. kichik opinion: http://forums.winamp.com/showthread....55#post2498455

Hope it will be included some day
tolan is offline   Reply With Quote
Old 27th January 2010, 07:24   #354
shnur
Junior Member
 
Join Date: Oct 2008
Location: Novosibirsk, Russia
Posts: 8
Send a message via ICQ to shnur
nsisunz+CallAnsiPlugin

@Anders:
I use your plugin for unzip file to folder, but it consumes 2 gb memory, without result.
Can you show me how to unzip file?
shnur is offline   Reply With Quote
Old 29th January 2010, 12:05   #355
roy204
Junior Member
 
Join Date: May 2004
Posts: 10
Question NSIS Unicode with "m" and "t" System types

Hi Jim, Anders and others,

I have a question about NSIS Unicode (I'm using v2.45.1):

I have a DLL that I need to call with System::Call and some of the function prototypes in that DLL have parameters that are explicitly "char*" because the DLL expects some strings in UTF-8 format and also sometime return strings in UTF-8 format.

From what I understand I have to use the "m" type in these System:Call prototypes to explicitly specify char* but what happens when I actually call them and pass a NSIS variable or register?

From my testing it just works (incidentally this is probably because the strings I pass in contain only english characters) but consider if I did the following:

StrCpy $0 "some chinese characters"
System::Call 'SomeDLL:SomeFunction(m)i(.r0).r1? c'

Does the Unicode string contained in $0 get converted in any way before the DLL function is called or is it just called with a typecast to (char *) with whatever data is in $0?

Since my calls are working, I suspect it's being converted to multibyte, but how and when ?

I'm asking because I'm having trouble with this when doing it the other way around: i.e. the DLL returns some UTF-8 to me that I need to display in NSIS.

For example if there is a function in that DLL defined as:

int SomeDLLGetMessage(char *buffer);

Which expects you to pass a pre-allocated char* buffer of at least 1024 bytes in order to place a utf8 string in it...

...and if I call this function this way:

System::Call 'SomeDLL::SomeDLLGetMessage(m)i(.r0).r1? c'

I find that it completes and that $0 contains something that I am able to display with a simple MessageBox MB_OK "$0".

Since the DLL call returns UTF-8 I would have expected $0 to contain single-byte data and therefore to be mangled when I attempt to display it. However it displays (the utf8 chars are displayed "as is" but it displays nonetheless).

Is there some kind of conversion happening for me inside NSIS ? What is the proper way of doing this?

I'd be more than happy to convert the UTF-8 data i get back to Unicode myself by calling MultiByteToWideChar but it doesn't seem to work. When I do this, I am able to convert $0 to Unicode but when I try to display it, NSIS only displays the first character (as if it was treated like a single byte string and stopped at the first 0 byte)...

I guess I am confused at how this needs to be handled. Is there a way to call a DLL, retrieve a UTF8 string and convert it to Unicode to display it ? or do I need to have the DLL modified to return Unicode data instead ?

Sorry for the long post :-(

Cheers,
Damien.

Last edited by roy204; 29th January 2010 at 12:25.
roy204 is offline   Reply With Quote
Old 29th January 2010, 12:35   #356
Anders
Moderator
 
Anders's Avatar
 
Join Date: Jun 2002
Location: ${NSISDIR}
Posts: 4,507
yes, m will convert the buffer. In your case, you should allocate 1024 bytes with System::Alloc and pass it to SomeDLLGetMessage as i, not m. When the call returns, you can pass it to MultiByteToWideChar yourself (Utf8 is only supported on XP+ in MultiByteToWideChar)

IntOp $PostCount $PostCount + 1
Anders is offline   Reply With Quote
Old 29th January 2010, 12:46   #357
jimpark
Senior Member
 
Join Date: Sep 2007
Posts: 204
There is an implicit conversion happening with 'm'. In fact, it uses the system default codepage for the conversion. If the return value is also 'm', (ANSI codepage) it will convert it back to utf16.

Are you sure the API sets the buffer with a UTF8 string? Often times people mix up UTF8 and ANSI. UTF8 is a byte-based encoding of Unicode. UTF8, UTF16-LE, UTF16-BE, UTF32 are all Unicode but different encodings of it. ANSI is a different beast altogether requiring the codepage information to really figure out which 'character'* each of the codes represent. This means that 'm' does not do UTF8 to UTF16 conversion at all. This would ONLY work for English since the ASCII set is shared by all ANSI codepages and UTF8. So make sure that the API really does set the string with UTF8 encoding. (This may be unlikely since it requires work to actually get a UTF8 string in Windows.) But if it really is UTF8, what Anders suggests might be the way to do it. Otherwise, if it's ANSI, then 'm' is fine as long as you are okay with the assumption that the string returned is going to be in the system default codepage -- if it's not then it won't display correctly on the system anyway, so it's really not a big concern.

* not necessarily character but really 'code' since diacritics and other character modifying codes exist

Unicode NSIS advocate -- http://www.scratchpaper.com for latest build and source.
jimpark is offline   Reply With Quote
Old 29th January 2010, 19:42   #358
roy204
Junior Member
 
Join Date: May 2004
Posts: 10
Yes the DLL really does return UTF-8 but thanks to you guys I got it working !

I was breaking down the process in two functions, one to call the DLL and one to convert to Unicode and in the process the results were transiting into NSIS variables and therefore being converted all around. Now that I do both the DLL call and MultiByteToWideChar in the same block of code using allocated buffers and the 'i' type, everything is pretty :-)

Thank you guys you saved my friday ;-)

Cheers,
Damien.
roy204 is offline   Reply With Quote
Old 31st March 2010, 13:11   #359
Afrow UK
Moderator
 
Afrow UK's Avatar
 
Join Date: Nov 2002
Location: Surrey, England
Posts: 8,433
Hi Jim, there seems to be an issue with the myatoi and myatou in the pluginapi library (and possibly the myatoi_or) too. If the plugin is built as Unicode then they all work fine but they fail to parse a valid integer for multi-byte builds. Sometimes they return 0 or they return some hideously large number (like an unsigned has been negatived). Same goes for popint().

I will probably look into fixing it myself but I'd rather you had a look. If you want me to test, let me know.

Stu
Afrow UK is offline   Reply With Quote
Old 31st March 2010, 13:11   #360
Afrow UK
Moderator
 
Afrow UK's Avatar
 
Join Date: Nov 2002
Location: Surrey, England
Posts: 8,433
Hi Jim, there seems to be an issue with the myatoi and myatou in the pluginapi library (and possibly the myatoi_or) too. If the plugin is built as Unicode then they all work fine but they fail to parse a valid integer for multi-byte builds. Sometimes they return 0 or they return some hideously large number (like an unsigned has been negatived). Same goes for popint().

I will probably look into fixing it myself but I'd rather you had a look. If you want me to test, let me know.

Stu
Afrow UK is offline   Reply With Quote
Reply
Go Back   Winamp & SHOUTcast Forums > Developer Center > NSIS Discussion

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump