Big problem with big files...

[Moasat]

Doesn't cut it. I need the solid compression to get the 2.2GB down to about 700MB.

[Joost Verburg]

So non-solid gives no error but solid does? Maybe the system for solid compression can be changed so it will be able to compress more than 2GB of data. Having a compressed installer larger than 2GB won't be useful because most Windows file systems have a 2GB file size limit.

[Moasat]

I'm building it now but its taking quite a long time. I'm assuming that it will give no error since I can do this same process with zlib, but it won't generate a setup.exe that will fit on a CD. (Zlib ends up around 1GB IIRC) I know by using 7-zip manually that the 2.2GB of files will compress down to <700MB, so I know this would work if the datablock could grow beyond 2GB and then get compressed solidly via LZMA. I'm just at the mercy of the developers and if they feel this is an important enough feature to change/implement. I'm hoping they do.

[Joost Verburg]

LZMA non-solid is definately not the same as ZLIB. It will probably be a little larger than LZMA solid, but usually it won't make a huge difference.

[Moasat]

I'll know in a couple of hours. Besides this 2GB issue, I think the rest of it is fantastic. Along with the HM NIS Editor, it is very nice. I'm trying to think of all the things I need to make installers for so I can use this more. Problem is, I have many other installs that also require 2GB of data files.

[Moasat]

Here's the final report. It didn't quite make it.

EXE header size: 49664 / 34816 bytes
Install code: 19821 / 86427 bytes
Install data: 736102408 / -1999567682 bytes
CRC (0xA0163DCE): 4 / 4 bytes

Total size: 736171897 / -1999446435 bytes (-36.-8%)

[Joost Verburg]

Well, coverted to megabytes it's 702 MB.

Most 80 minute CD's have a capacity of about 703 MB, so it will probably work.

[Joost Verburg]

You can also increase the dictionary size a bit to make it fit.

[Koen van de Sande]

Perhaps the install data size variable shouldn't be signed to avoid negative numbers?

[Joost Verburg]

Yes, it should be unsigned.

[Comm@nder21]

ok, i'm getting this error:

Processed 1 file, writing output:
Internal compiler error #12345: deflateInit() failed(-1).

Note: you may have one or two (large) stale temporary file(s)
left in your temporary directory (Generally this only happens on Windows 9x).

i'm compiling few files (about 20), each one below 1 kb (only dummyfiles for testing), and some gfx. the whole installer is about 500 kb, with lzma.
the installer compiles correctly, while SetCompressorDictSize isn't used. everytime i use this setting, the error occurs (with any dictsize).
the nsis-script is running on a FAT32 partition with 3gb free space.
the partition with windows-xp has about 5gb free disk space, and the partition expecially for temporary files (including virtualram/swap) has another 3gb space, about 2gb free.
i'm using latest cvs-version of nsis.
why does this error occur at this configuration?

[Joost Verburg]

I can't reproduce it, please attach a script including the files. Btw, this is not related to this topic.

Did you recompile makensis.exe? Note that the LZMA source code has been updated today (it's smaller and faster) and needs some testing.

When compiling installers to release we always recommend using a stable version, however we really appreciate it that you help testing the development version

[Comm@nder21]

i'm always using the latest cvs, because, when u look at the changelog, it seems to be more stable (and more up to date - look at the language files) than the "stable".

also, this problem exists with some versions before.

and, yes, i think this is related to this topic, because i found this topic (and read all posts) when i searched for error #12345.

i don't know how to recompile makensis.exe, sorry.

hmm, i'll see, if i can attach an example.

[kichik]

You must have used a dictionary size which your computer can not handle. Remeber that SetCompressorDictSize takes the number of mega bytes to use and not the number of bytes.

All errors related to memory or failed compression are marked as error #12345.

[Joost Verburg]

Because you did not recompile makensis it is indeed not related to the new CVS source. I missed that line about SetCompressorDictSize.

[Comm@nder21]

oh, damn, it's in mbyte?
i thought, that was kbyte. kbyte would make more sense, i think.
but, ok, then this is the misstake, i entered 4096 as dict-size

edit:
so, does this command support floating numbers?

edit2:
worked with 'SetCompressorDictSize 8'

[kichik]

8 is the default. It doesn't support floating point numbers. I see no real need for that. In 7-zip all of the options are in mega bytes too...

[Thorondor]

Hi all,

my problem is this:
i've got about 1.5 GByte of files to install (multiple files; biggest not larger than 200 MB). I can only compile using zlib compression. When i try to use bzip2 or lzma I get this error:

Using lzma (compress whole) compression.

EXE header size: 49152 / 35328 bytes
Install code: (38548 bytes)
Install data: (1444031845 bytes)
Uninstall code+data: (61041 bytes)
Compressed data: Error: deflateToFile: deflate()=-3
Error - aborting creation process

I tried it with bigger dictsizes, but this didn't work. These results were with a dictsize of 48 MB (was desperate .

ps I use Windows2000 SP4 and NTFS drive with about 10 GB free.

I used the latest NSIS build (non-CVS)

[Joost Verburg]

A larger dictionary size makes compression more complicated, a smaller size makes it faster and reduces the memory requirements.

You can also download a special build from the website that uses non-solid compression.

[Thorondor]

Thanks non-solid and smaller dictionary size did the trick...

Sometimes Most of the time I get the same error with some of the files: deflate()=-3. The weird thing is if try compressing these files alone in NSIS I don't get an error message. Weirder still is that it's not always the same files. Sometimes the compiler stops on a file and sometimes not.
I'm not considering doing these files separate, I want one big file. Please help.

ps: I also tried compressing them outside of nsis with 7-zip and no problem

[kichik]

-3 means a memory error for bzip2 and an exception for lzma which usually means a memory error too. Are you sure you have enough free memory? Try setting a lower dictionary size, way lower than 48MB...

[Thorondor]

In the last run I used smaller dictionary sizes <8MB.
And I've got 576MB Physical memory of which usually about 50% is free. Total system memory is about 1.3GB with usually about 80% free. So it's not, not enough memory free.

[Joost Verburg]

8MB is still quite large, try 4MB.

[Thorondor]

I used 8, 4, 3, 2 and even 1.

[kichik]

What error code do you get for bzip2? Is it -3 too?

[Thorondor]

I haven't tried bzip2 in a while, but I'll try it next

[Thorondor]

I tried bzip2 and unfortunately I didn't get an error . The problem for me is solved, but I think there is something wrong with the lzma compression for big files (solid/non-solid) (or else I'm doing something wrong )

[kichik]

Last question before I perper a test which will give some more details. Have you used FileBufSize? If you have, what value? If you haven't, try setting it to 8.

[Thorondor]

I didn't use FileBufSize, but with it lzma-compression works like a charm. Even with a dictionarysize of 8MB.
Thanks for your help kichik and Joost Verburg.

(Looks like I WAS doing something wrong )

[kichik]

If setting it to a lower value, you surely had a memory problem. If not shortage of memory, then maybe corrupted memory? Worth a check...

[Thorondor]

Memory is fine, enough free and not corrupted. Checked it with multiple programs (the amount free and if it's corrupted). I think it purely was the size of the file-buffer. When I increased it I had no more problems.

[kichik]

I have added meaningful error strings in the latest version. Can you please test your script on makensis.exe from:

http://nsis.sourceforge.net/err.zip

and let me know what the error string is?

[Thorondor]

I can't tell you the error string, because there is none. *
I think you solved the problem without realizing it
I've tried multiple times, using different settings, even setting the dictionary size to 8, but no problems.
I'll keep on trying though. I'll post if I get an error.

*I mean I don't get any problems/errors anymore.

[SonicSD]

Hi!

I'm also having the size prob. I need to compress ~3 GB. I tried to compile a installer, but makensis
stopped at about ~2.1 GB. Ok I also tried it with the Non-Solid, but also stopped sometime.

What I want to know, would it be possible to compress the installation files externally with 7-Zip, and then add NSIS-Data additionally?
This would be a good workaround for now, until NSIS supports larger sizes.

Anyway NSIS is a great Installer, keep the good work up.
(using it since v1.98 )

SonicSD

[Joost Verburg]

You can use external data files and the 7-zip command line tool to decompress the files on the users system.

[SonicSD]

I did it that way, before NSIS got native lzma support.

But I thought that it might be possible to compress the data externally and make use of the native functions for decompression. (especially for the installing options )

SonicSD

[Joost Verburg]

NSIS does not support the 7-zip file format or any other archive structure.

If the compressed data size is below 2 GB, it should work with non-solid compression.