Winamp & Shoutcast Forums

Winamp & Shoutcast Forums (http://forums.winamp.com/index.php)
-   NSIS Discussion (http://forums.winamp.com/forumdisplay.php?f=65)
-   -   Big problem with big files... (http://forums.winamp.com/showthread.php?t=123323)

Moasat 13th January 2004 16:44

Doesn't cut it. I need the solid compression to get the 2.2GB down to about 700MB.

Joost Verburg 13th January 2004 17:13

So non-solid gives no error but solid does? Maybe the system for solid compression can be changed so it will be able to compress more than 2GB of data. Having a compressed installer larger than 2GB won't be useful because most Windows file systems have a 2GB file size limit.

Moasat 13th January 2004 20:08

I'm building it now but its taking quite a long time. I'm assuming that it will give no error since I can do this same process with zlib, but it won't generate a setup.exe that will fit on a CD. (Zlib ends up around 1GB IIRC) I know by using 7-zip manually that the 2.2GB of files will compress down to <700MB, so I know this would work if the datablock could grow beyond 2GB and then get compressed solidly via LZMA. I'm just at the mercy of the developers and if they feel this is an important enough feature to change/implement. I'm hoping they do.

Joost Verburg 13th January 2004 20:12

LZMA non-solid is definately not the same as ZLIB. It will probably be a little larger than LZMA solid, but usually it won't make a huge difference.

Moasat 13th January 2004 20:14

I'll know in a couple of hours. Besides this 2GB issue, I think the rest of it is fantastic. Along with the HM NIS Editor, it is very nice. I'm trying to think of all the things I need to make installers for so I can use this more. Problem is, I have many other installs that also require 2GB of data files. :(

Moasat 14th January 2004 20:11

Here's the final report. It didn't quite make it.

EXE header size: 49664 / 34816 bytes
Install code: 19821 / 86427 bytes
Install data: 736102408 / -1999567682 bytes
CRC (0xA0163DCE): 4 / 4 bytes

Total size: 736171897 / -1999446435 bytes (-36.-8%)

Joost Verburg 14th January 2004 20:37

Well, coverted to megabytes it's 702 MB.

Most 80 minute CD's have a capacity of about 703 MB, so it will probably work.

Joost Verburg 17th January 2004 15:25

You can also increase the dictionary size a bit to make it fit.

Koen van de Sande 18th January 2004 11:44

Perhaps the install data size variable shouldn't be signed to avoid negative numbers?

Joost Verburg 18th January 2004 12:52

Yes, it should be unsigned.

Comm@nder21 6th March 2004 19:29

ok, i'm getting this error:
Quote:

Processed 1 file, writing output:
Internal compiler error #12345: deflateInit() failed(-1).

Note: you may have one or two (large) stale temporary file(s)
left in your temporary directory (Generally this only happens on Windows 9x).
i'm compiling few files (about 20), each one below 1 kb (only dummyfiles for testing), and some gfx. the whole installer is about 500 kb, with lzma.
the installer compiles correctly, while SetCompressorDictSize isn't used. everytime i use this setting, the error occurs (with any dictsize).
the nsis-script is running on a FAT32 partition with 3gb free space.
the partition with windows-xp has about 5gb free disk space, and the partition expecially for temporary files (including virtualram/swap) has another 3gb space, about 2gb free.
i'm using latest cvs-version of nsis.
why does this error occur at this configuration?

Joost Verburg 6th March 2004 20:43

I can't reproduce it, please attach a script including the files. Btw, this is not related to this topic.

Did you recompile makensis.exe? Note that the LZMA source code has been updated today (it's smaller and faster) and needs some testing.

When compiling installers to release we always recommend using a stable version, however we really appreciate it that you help testing the development version :)

Comm@nder21 6th March 2004 20:51

i'm always using the latest cvs, because, when u look at the changelog, it seems to be more stable (and more up to date - look at the language files) than the "stable".

also, this problem exists with some versions before.

and, yes, i think this is related to this topic, because i found this topic (and read all posts) when i searched for error #12345.

i don't know how to recompile makensis.exe, sorry.

hmm, i'll see, if i can attach an example.

kichik 6th March 2004 21:11

You must have used a dictionary size which your computer can not handle. Remeber that SetCompressorDictSize takes the number of mega bytes to use and not the number of bytes.

All errors related to memory or failed compression are marked as error #12345.

Joost Verburg 7th March 2004 07:54

Because you did not recompile makensis it is indeed not related to the new CVS source. I missed that line about SetCompressorDictSize.

Comm@nder21 7th March 2004 09:29

oh, damn, it's in mbyte?
i thought, that was kbyte. kbyte would make more sense, i think.
but, ok, then this is the misstake, i entered 4096 as dict-size :)

edit:
so, does this command support floating numbers?

edit2:
worked with 'SetCompressorDictSize 8' :)

kichik 11th March 2004 17:57

8 is the default. It doesn't support floating point numbers. I see no real need for that. In 7-zip all of the options are in mega bytes too...

Thorondor 15th April 2004 18:03

Hi all,

my problem is this:
i've got about 1.5 GByte of files to install (multiple files; biggest not larger than 200 MB). I can only compile using zlib compression. When i try to use bzip2 or lzma I get this error:

Using lzma (compress whole) compression.

EXE header size: 49152 / 35328 bytes
Install code: (38548 bytes)
Install data: (1444031845 bytes)
Uninstall code+data: (61041 bytes)
Compressed data: Error: deflateToFile: deflate()=-3
Error - aborting creation process

I tried it with bigger dictsizes, but this didn't work. These results were with a dictsize of 48 MB (was desperate :).

ps I use Windows2000 SP4 and NTFS drive with about 10 GB free.

I used the latest NSIS build (non-CVS)

Joost Verburg 15th April 2004 18:20

A larger dictionary size makes compression more complicated, a smaller size makes it faster and reduces the memory requirements.

You can also download a special build from the website that uses non-solid compression.

Thorondor 17th April 2004 10:00

Thanks non-solid and smaller dictionary size did the trick...

Sometimes :( Most of the time I get the same error with some of the files: deflate()=-3. The weird thing is if try compressing these files alone in NSIS I don't get an error message. Weirder still is that it's not always the same files. Sometimes the compiler stops on a file and sometimes not.
I'm not considering doing these files separate, I want one big file. Please help.

ps: I also tried compressing them outside of nsis with 7-zip and no problem

kichik 17th April 2004 10:07

-3 means a memory error for bzip2 and an exception for lzma which usually means a memory error too. Are you sure you have enough free memory? Try setting a lower dictionary size, way lower than 48MB...

Thorondor 17th April 2004 11:03

In the last run I used smaller dictionary sizes <8MB.
And I've got 576MB Physical memory of which usually about 50% is free. Total system memory is about 1.3GB with usually about 80% free. So it's not, not enough memory free.

Joost Verburg 17th April 2004 11:06

8MB is still quite large, try 4MB.

Thorondor 17th April 2004 11:14

I used 8, 4, 3, 2 and even 1.

kichik 17th April 2004 11:17

What error code do you get for bzip2? Is it -3 too?

Thorondor 17th April 2004 11:19

I haven't tried bzip2 in a while, but I'll try it next

Thorondor 17th April 2004 11:58

I tried bzip2 and unfortunately I didn't get an error ;). The problem for me is solved, but I think there is something wrong with the lzma compression for big files (solid/non-solid) (or else I'm doing something wrong :))

kichik 17th April 2004 12:17

Last question before I perper a test which will give some more details. Have you used FileBufSize? If you have, what value? If you haven't, try setting it to 8.

Thorondor 17th April 2004 14:08

I didn't use FileBufSize, but with it lzma-compression works like a charm. Even with a dictionarysize of 8MB.
Thanks for your help kichik and Joost Verburg.:up: :up:

(Looks like I WAS doing something wrong ;) )

kichik 17th April 2004 14:27

If setting it to a lower value, you surely had a memory problem. If not shortage of memory, then maybe corrupted memory? Worth a check...

Thorondor 17th April 2004 17:58

Memory is fine, enough free and not corrupted. Checked it with multiple programs (the amount free and if it's corrupted). I think it purely was the size of the file-buffer. When I increased it I had no more problems.

kichik 8th May 2004 17:40

I have added meaningful error strings in the latest version. Can you please test your script on makensis.exe from:

http://nsis.sourceforge.net/err.zip

and let me know what the error string is?

Thorondor 10th May 2004 18:34

I can't tell you the error string, because there is none. *
I think you solved the problem without realizing it ;)
I've tried multiple times, using different settings, even setting the dictionary size to 8, but no problems.
I'll keep on trying though. I'll post if I get an error.

*I mean I don't get any problems/errors anymore.

SonicSD 22nd May 2004 12:40

Hi!

I'm also having the size prob. I need to compress ~3 GB. I tried to compile a installer, but makensis
stopped at about ~2.1 GB. Ok I also tried it with the Non-Solid, but also stopped sometime.

What I want to know, would it be possible to compress the installation files externally with 7-Zip, and then add NSIS-Data additionally?
This would be a good workaround for now, until NSIS supports larger sizes. :D

Anyway NSIS is a great Installer, keep the good work up.
(using it since v1.98 :D)

SonicSD

Joost Verburg 22nd May 2004 14:36

You can use external data files and the 7-zip command line tool to decompress the files on the users system.

SonicSD 23rd May 2004 15:18

I did it that way, before NSIS got native lzma support.

But I thought that it might be possible to compress the data externally and make use of the native functions for decompression. (especially for the installing options :D)

SonicSD

Joost Verburg 23rd May 2004 15:29

NSIS does not support the 7-zip file format or any other archive structure.

If the compressed data size is below 2 GB, it should work with non-solid compression.


All times are GMT. The time now is 06:26.

Copyright © 1999 - 2010 Nullsoft. All Rights Reserved.