Unicode

[Anders]

@Afrow: the unicode merge has started, so whatever changes you make, add them to the official branch as well

[jimpark]

Stu, I'm looking at the code right now and I don't see anything that could cause the behavior you are seeing. But the behavior is in line with a Unicode string being fed into an ANSI version of myatoi. Is there a simple NSI file that would test the problem?

[Afrow UK]

I'll see if I can knock something up.

Stu

[Afrow UK]

Ok here is the problem being reproduced:

test_myatoi: Pushed: 999, Result: 0
test_myatou: Pushed: 999, Result: 0
test_popint : Pushed: 999, Result: 0
test_my_atoi: Pushed: 999, Result: 999
Completed

This only occurs when using the ANSI plugin build (Compile NSIS Script). Note that my_atoi is a different implementation.

I've attached the plugins and NSIS script.

Stu

[sealite]

Originally Posted by dariann

I have this problem too... It's very annoying... I want to turn off this messages, because they spamming my command window. How can I do it?

I observed that over time lots of users complained about the file encoding messages.

Please drop the command line spamming regarding file encoding or move them to /V3 this is my only desire in order to consider NSIS Unicode perfect.

I'm using NSIS Unicode for more than two years and my only complain is regarding this, so called feature. A good command line utility should not spam you with few pages of useless text.

This "feature" can make you miss potential warnings or even errors when compiling. Redirecting STDOUT is not an option.

Also, I hope that NSIS will switch to NSISu as default instead of keeping the ancient ANSI version.

[Wizou]

In my merging of NSIS Unicode in NSIS sources trunk, I removed most of these "File has a BOM"-type messages, keeping them only when it's a useful warning (no BOM).
So this will be solved in the future NSIS release that include my Unicode merge.

As for switching to Unicode installers as default, I'm thinking more about adding a command-line option (that can be set as default, like the compressor in MakeNSISw) and an equivalent attribute command (like SetCompressor), for choosing ANSI or Unicode target.

[Anders]

@Wizou: I posted a unofficial nsis installer ( http://forums.winamp.com/showthread.php?t=316842 ) that includes both ansi and unicode + a loader that tries to guess based on BOM or forced with command line param (Required hacked makensis.exe to allow different stub dirs) Once the unicode branch is fully merged, we could of course implement that functionality in a less hacky way (We need StubA/StubW and PluginsA/PluginsW folders and maybe for languages unless we are able to transform the ansi versions on the fly. Include dir should be shared (Some unicode fixes need to be added to make that come true IIRC))

[jimpark]

2.46 RC1 Is Out!

The Unicode port of NSIS 2.46 is out as a release candidate. Apart from the straight port, I've made some improvements.

New African languages are supported:
•Cibemba (ANSI and Unicode)
•Efik (Unicode only)
•Igbo (Unicode only)
•Malagasy (ANSI and Unicode)
•Yoruba (Unicode only)

As you can see, some of these languages are supported in my new ANSI build also. Please note that the translations aren't complete but "sufficient" as told to me by the translators so further enhancements may be forthcoming in a later release.

Also, the System calls to lstr* family of functions in the kernel32.dll has been modified so that there is no longer the need to always suffix with "W" the lstr* functions for the Unicode build. The problem stemmed from the fact that in the kernel32.dll, the lstr* functions such as lstrlen, had three callable versions: lstrlenW, lstrlenA and lstrlen. (This is different from other win32 functions which only have two callable versions: the A and the W suffixed calls.) However, in kernel32, lstrlen is the same as lstrlenA. The reason for this strangeness in kernel32.dll was presumably for backward compatibility. So an NSI script that used System::Call kernel32::lstrlen would get lstrlenA in both the Unicode and the ANSI build of NSIS! Please note, this only happens with kernel32::lstr* functions. But this has been remedied by special code in the System plugin that looks for calls to kernel32::lstr* functions and adjusts the calls according to the build of NSIS so that there is no longer any confusion. The TCHAR-like names for the functions should now work for all of the win32 calls which should make porting of the standard NSIS scripts to the Unicode version much easier.

I've also removed the "chatty" info messages regarding the Unicode encoding of the compiled NSI and NSH files. This was bothering some.

In my personal testing, everything looked decent. Please take some time to check if your project works well with it and let me know if you find any problems.

[jimpark]

Stu,

I've tried looking at your TestMyatoi zip file and found only one version of pluginapi.lib there. Since you are saying this fails for the ANSI, you must be using the Unicode version of the lib. If you are building for ANSI, you need the ANSI version of the lib. You can't link to the same lib for both ANSI and Unicode version of your plugin.

If this is something that is desired, I can create a unified lib with both ANSI and Unicode versions of the function existing in the lib with the pluginapi.h having #defines that map the different function calls to the right functions. This would basically mimic what win32 does.

[Anders]

Are you saying you have a hardcoded check for "kernel32::lstr"? How do you know those are the only functions that are like that? I still say that if "t" type is used, unicode versions needs to go FooW() > Foo() > Fail

[jimpark]

I'm pretty sure those are the only functions like that. Do the strings command on the various dlls and look for yourself. As for using 't' as the hint, would that work if the return type is what's different? It also won't work if the string type is actually a member of a struct and the pointer to the struct is what's being passed in. It's an interesting idea but I'm right now not sure if that is enough of a check.

[Anders]

@Jim: Yes, it would fail for structs. Not sure about return, I don't think you can have ...(...)t.s etc

Maybe we have to settle for some sort of flag. With the check for lstr, we can probably get by with your code until a problem actually shows up.

[Afrow UK]

Jim,

bother didn't think about that. If you could make a define that would be handy.

Stu

[jimpark]

Stu,

It is possible to do but probably not this release. It does require quite a bit of reworking for the pluginapi.h and I don't want to delay the 2.46 release any longer. But I'm sure 2.47 is around the corner and I will incorporate it then.

[Animaether]

Semi-on-topic...

First.. As I understand it, plugins written for the ANSI version of NSIS won't work in the Unicode version, and vice-versa.
Am I correct in that understanding - or is this limited to plugins that perform specific tasks?

Second.. As I understand it, the two compiles would typically result in two plugin files (Anders' BgWorker notwithstanding).
Am I correct in that understanding - or would this again be limited to plugins that perform specific tasks?

Just to use a particular example.. the GetVersion plugin by Stu (Afrow UK) has as part of its version notes "Better Unicode build support". However, the download only contains a single DLL.
Is there anything specific one might look out for in a plugin's source - if available - that would tell a person that the plugin it would build would be ANSI, Unicode, or Hybrid (a la BgWorker)?

The main reason I'm asking is because I'm working on compiling a list of plugins with some basic information - including whether it supports ANSI, Unicode or both - and GetVersion is the first one that made me realize that it might not be as trivial as simply checking if there's two DLLs (BgWorker makes its Hybrid method clear in both the wiki page and in the source).

[Anders]

You can pretty much assume that no other plugin is a hybrid, it is a big ass pain to do.

If you look at the GetVersion source, you see that it uses the TCHAR type and _T/TEXT macros for strings. This is a indicator that it supports both unicode and ansi output. If UNICODE & _UNICODE is defined when building, you end up with a unicode dll. (@Stu: GetProcAddress does not take a unicode string!)

If all you have is a compiled dll, open it in dependencywalker.com If a lot of the used functions in kernel32/user32 ends with A its ansi, or W for unicode.

[Animaether]

Cool - thanks Anders. I've used dependency walker before - just not that part of it.

Sounds like basically although the GetVersion plugin is Unicode-ready in terms of the source code, no compiled version is available in the official download at this time.

'll see about finishing that list later (fair bit later) and doing something with it in the wiki.

[Afrow UK]

Yeh I did make sure the source was ready for Unicode (except GetProcAddress, thanks Anders) but never built (I don't think we had Unicode NSIS when I wrote that plug-in). I will build a Unicode version when I have time - in fact quite a few of my plug-ins needs updating but I'm stuck for time at the moment.

Stu

[webtubbies]

Hi All

For my installer I use Anders' plugin "CallAnsiPlugin" with NSIS Unicode and everything's OK except with nsisXML by Wizou.

OutFile "test_xml_UNICODE.exe"

!define TempDir "C:\TEMP"
!define nsisXML "${TempDir}\nsisXML.dll"

Function .onInit
SetOutPath $INSTDIR
;Get All ANSI dll to the local machine
File /ONAME=${nsisXML} "${NSISDIR}\Plugins\nsisXML.dll"
FunctionEnd

Function test_xml
CallAnsiPlugin::Call ${nsisXML} create 0
CallAnsiPlugin::Call ${nsisXML} load 1 "sample.xml"
CallAnsiPlugin::Call ${nsisXML} select 1 '/main/child[@attrib="value2"]'
IntCmp $2 0 notFound
CallAnsiPlugin::Call ${nsisXML} getAttribute 1 "active"
DetailPrint "Attribute 'active' is $3"
CallAnsiPlugin::Call ${nsisXML} getText
DetailPrint "Tag <child> contains $3"
CallAnsiPlugin::Call ${nsisXML} parentNode
CallAnsiPlugin::Call ${nsisXML} removeChild
CallAnsiPlugin::Call ${nsisXML} save 1 "SampleU.xml"

notFound:
FunctionEnd

Section myFoo
Call test_xml
SectionEnd

Is there anybody to tell me what's wrong in my code please?

[Anders]

Originally Posted by webtubbies

Is there anybody to tell me what's wrong in my code please?

You are not checking that the nsisXML calls don't fail, if you did that you could maybe narrow down to the line that fails. What happens exactly, does it crash for example?

(There is also a chance that CallAnsiPlugin has a bug, it was never tested with this plugin)

[webtubbies]

Originally Posted by Anders

You are not checking that the nsisXML calls don't fail, if you did that you could maybe narrow down to the line that fails. What happens exactly, does it crash for example?

(There is also a chance that CallAnsiPlugin has a bug, it was never tested with this plugin)

Hi Anders,

Actually, the first call (create) seems to be OK and then there is a crash of the exe.

[Anders]

nsisXML has a unicode version in the zip file, why can't you use that?

[webtubbies]

@Anders:
Yeah you're right, I was using an older version. I'll try tomorrow the Unicode version.
Thank You for your time and sorry for the incovenience

[Anders]

I also found a varsync bug in CallAnsiPlugin, so I will update the wiki later today with the fixed version

[webtubbies]

Ok, the unicode nsisXML plugin works like a charm! thank You

[Wizou]

Originally Posted by webtubbies

Ok, the unicode nsisXML plugin works like a charm! thank You

you're welcome ;-)

[jimpark]

Unicode NSIS 2.46 is finally released!

[Pawel]

Jim, thanks!
-Pawel

[gringoloco023]

as well I should let you know about TextFunc.nsh
it misses support for UTF-16LE files.

I done a re-write of ${ConfigRead/Write} my self for now(see the attachment). I added the utf-16LE support within the same instruction by having it look for a BOM and handling the file accordingly. Hopefully all the other instructions can be done in a similar way.

[edit: I just realized my re-write was a little buggy, as I forgot to reset the variable what indicates if there is a BOM or not. I re-uploaded, but just see it as an example !]

[Wizou]

I'm done porting NSIS sources to Unicode. It is now possible to compile NSIS sources (from Sourceforge repository) either as ANSI or UNICODE.
I'm now working on making the Unicode version able to generate both ANSI & Unicode installer, so we can make a switch and have everybody use a single (Unicode) version of makensis.exe

[Afrow UK]

Very nice. On the plug-in side of things, will ANSI plug-ins be defunct now?

Stu

[Wizou]

No, because people wanting to target Windows 9x & such still need ANSI installer & ANSI plugins.
However I have created a new version of the plugin API, mostly compatible with the current version, that allows plugins to work with both ANSI & Unicode installer (even if the plugin itself is compiled as ANSI or Unicode), making the conversions transparently.
I will commit that soon to SVN.

[CrushBug]

Hold it. Are you saying that there could be an official release from the main NSIS page with both ANSI and Unicode versions in the same package?

If this is true, this is HUGE for us, as we are only allowed by our legal department to use the versions of NSIS that come from the main page/official site. We are not allowed to use other versions (Unicode NSIS from scratchpaper, etc.)

[Anders]

@CrushBug: Official SVN has working unicode build (It is pretty much the same code as the fork, at some point the merge will be complete hopefully)

[Pawel]

Originally Posted by Wizou

No, because people wanting to target Windows 9x & such still need ANSI installer & ANSI plugins.
However I have created a new version of the plugin API, mostly compatible with the current version, that allows plugins to work with both ANSI & Unicode installer (even if the plugin itself is compiled as ANSI or Unicode), making the conversions transparently.
I will commit that soon to SVN.

Wizou,
Is it possible to write src\Docs\src\*.but files with utf8 encoding (and Halibut could parse it and create output html files with default utf8 encoding, that HTML Help Help Workshop could compile to chm)?
Would be much simple for me, if it use this codepage, not default windows.
Could someone with skills look into it? Or, this is to much work, and it is not worth...
-Pawel

[Wizou]

I'm not much aware about Halibut, and your demand is quite independent from porting NSIS code to Unicode.. I guess I won't be doing that soon..
Maybe someone else will look into it..

[akatik]

In Unicode NSIS this plug-in works incorrectly.
http://nsis.sourceforge.net/IP_plug-in
Is there any other plug-in to get IPs of the computer or may be anybody could help with the solution for this plug-in?

[Afrow UK]

If the plug-in is not Unicode then you will need to rebuild it.

How about this? http://www.codeproject.com/KB/vbscript/ipaddress.aspx

Stu

[akatik]

Originally Posted by Afrow UK

If the plug-in is not Unicode then you will need to rebuild it.

Does rebuild mean anything more than compiling nsi-script via Make NSISW (Unicode) program? This plug-in is dll - how can I rebuild it?
And also I failed to understand how and what for to use a2u.exe program.

Originally Posted by Afrow UK

How about this? http://www.codeproject.com/KB/vbscript/ipaddress.aspx

Actually dll on C++ with code for determining IP-interfaces of the computer I'll create by myself. I was interested why former plug-in don't work.

[jpderuiter]

To get the IP address, have a look at the IpConfig plugin:
http://nsis.sourceforge.net/IpConfig_plugin