Debugging advice

[Yathosho]

i'm suffering over a problem occuring with my script and i'm having a difficult time pinpointing it. what this script does is locating and copying some files and using textreplace. in most cases it works without problems, but when a large number of files is involved (in my case 2530) the installer will freeze. to make things worse this does not happen always, just occasionally.

i'm looking for some hints on how to find the bug. i've already disabled various parts of the script, but without gaining any knowledge. all i can say is, this happens when working with a lot of files and i believe i did not have this problem when using other (slower) functions to replace text or to locate files. is there anything wrong with my textreplace calls? can they be optimized? any other tips on how to narrowing down the problem?

any help is greatly appreciated.

[{_trueparuex^}]

In cases like this I usually just add some messgeboxes to the code to find the point where things go wrong, but in this case Nsisdbg plug-in is probably much better choice.

With Nsisdbg plug-in use the nsisdbg::sendtolog call to track the progress of your script.

Edit: I just realized that the Nsisdbg plug-in download link is is dead. I uploaded the plug-in on my site.

[Yathosho]

i shall also note that i only get those problems on windows 7 (both 32 and 64 bit), not on windows xp.

[Yathosho]

thanks {_trueparuex^}, i actually tried all of this before - to no success. as i suspect the number of files to be responsible, i wonder if this could be a memory issue. i do have 8gb ram and enough space on all drives, so that should not be the case. i'm also not exceeding the string-length (i'm using the large string build), so i wonder where else i can look. what puzzles me, why does it work on xp in a virtual machine (with 2gb ram)

[Yathosho]

it appears i could find the problem in my script, so i wanted to resolve it in case someone runs into similar problems.

the problem appeared (occasionally, only on windows vista or later) when setting the outdir when the path existed already

code:
SetOutPath "$MyTempDir\fonts"

a simple IfFileExists fixed the issue, but i changed the code to this

code:
IfFileExists "$MyTempDir\fonts\*.*" +2

CreateDirectory "$MyTempDir\fonts"

in the following a couple of files were copied, so i didn't really need SetOutPath and changed it to CreateDirectory. for reasons unknown, it was IfFileExists that made the difference!

[Yathosho]

i continue to have this bug at a later point. again it's caused by a simple copy operation, just this time i can't figure out when and why it's happening (again it happens occasionally and only on vista/win7.) i have a test scenario which helps me testing this on 312 examples, but the result continue to be entirely random, though certain test-files tend to trigger the bug more often.

the line in question is this:

code:
CopyFiles /SILENT "$EXEDIR\ape\addborder.ape" "$MyTempDir\ape\addborder.ape"

the source-file is located in $PROGRAMFILES and is a dll with a different file-extension. the problem appears on elevated and usermode installers.

any clues, cause i'm clueless!

[Yathosho]

here are a couple of things i tried

1. extract the file from the installer rather than copying it -> works
2. copying the file using the full path as on my system -> works
3. checking if the source-file and the target-folder exist -> yes
4. checking if $EXEDIR is correct -> yes

$EXEDIR = C:\Program Files (x86)\whyEye.org\PimpBot\ape

CopyFile "$EXEDIR\ape\addborder.ape" "$MyTempDir\ape\addborder" -> crashes installer
CopyFile "C:\Program Files (x86)\whyEye.org\PimpBot\ape\addborder.ape" "$MyTempDir\ape\addborder" -> works

i'm clueless, please please help me!

[MSG]

What if you call API directly, instead of CopyFiles?

[Yathosho]

i tried StdUtils, which also results in a crash

[MSG]

Just a random thought: Have you tried this on other PCs? Win Vista/7 are very clunky with file copying. It's possible that bad RAM or a bad HDD causes it to blow.

[Yathosho]

i asked some friends to test it, but they didn't get any problems (well, i'd love to test it myself). i replaced my ram since this first came up and i tried various disks including ssd, harddrives and ramdisk (hence i'm using $MyTempDir instead of $PLUGINSDIR). i have another windows 7 (32-bit) running in a virtual machine (same problem) and also windows xp (where i NEVER get this problem.) i'm going to install windows 7 on vmware on my laptop and see what happens..

further, i've tested various nsis builds (standard and longstr)

it's interesting that there have been reports regarding CopyFiles and windows 7 on this forum, though i didn't find a case that really crashes the installer.

having found the exact line where this problem occurs, i wonder if it could still be caused by something else. not sure what i should be looking for.

[Yathosho]

same thing on the other computer

[MSG]

Originally Posted by Yathosho

$EXEDIR = C:\Program Files (x86)\whyEye.org\PimpBot\ape

CopyFile "$EXEDIR\ape\addborder.ape" "$MyTempDir\ape\addborder" -> crashes installer
CopyFile "C:\Program Files (x86)\whyEye.org\PimpBot\ape\addborder.ape" "$MyTempDir\ape\addborder" -> works

It shouldn't lead to a crash, but if $EXEDIR already has a trailing 'ape', your copyfile command is wrong.

[Yathosho]

Originally Posted by MSG

It shouldn't lead to a crash, but if $EXEDIR already has a trailing 'ape', your copyfile command is wrong.

just a typo in the post

anyway, i've figured out a workaround so i'm not going to break my head over this one. thanks for all the advice!

[{_trueparuex^}]

I think you have a heap corruption issue here. The crash.log shows it crashes in ntdll.dll with memory access violation which I think is probably result of heap corruption. If this is the case you probably also get crashes with exception code 0xc0000374 (STATUS_HEAP_CORRUPTION). And the cause for heap corruption would be most likely a third party plug-in.

PaR

[Yathosho]

Originally Posted by {_trueparuex^}

I think you have a heap corruption issue here. The crash.log shows it crashes in ntdll.dll with memory access violation which I think is probably result of heap corruption. If this is the case you probably also get crashes with exception code 0xc0000374 (STATUS_HEAP_CORRUPTION). And the cause for heap corruption would be most likely a third party plug-in.

PaR

hmm, so it's probably TextReplace as i've assumed before. i don't want to go back, as this plugin performs MUCH better than any functions or macros i've used before. i tried contacting instructor before to ask if he plans a unicode build, but i never heard from him since. so if plugin causes the crach (i don't want to blame the plugin, but i'm pretty sure it's causing this) it's probably never going to get fixed.

[Yathosho]

i commented out the two plugins in question each at a time (TextReplace and Locate) and it appears that either the locate plugin is buggy or my code using it produces the bug (will look into it!) in either case, the installer didn't crash once in 2,500 test runs.

the only let down is this: what happens if it's to blame on the plugin. it appears it's no longer maintained by the author

[DrO]

Originally Posted by Yathosho

the only let down is this: what happens if it's to blame on the plugin. it appears it's no longer maintained by the author

as the source is provided, you'd need to find someone to update / fix it for you or learn how to do it

-daz

[Yathosho]

Originally Posted by DrO

as the source is provided, you'd need to find someone to update / fix it for you or learn how to do it

you mean as a warm up for my go at avs 3.0?

[DrO]

something like that though i'd prefer a milkdrop 3 which supports avs presets

-daz