Compression blocks

[CodeSquid]

The problem with huge installers using "whole compression" mode is the slow initialization time of the installer. I'm still using zlib compression for my installers because I hate to wait for installers (worst is the InstallShield + MSI combination).

The benefit of lzma + whole compression is the reduced installer size.
Zlib compression on the other side is the fastest, especially since every file gets compressed individually.


Now I had the idea of compression blocks, each consisting of one or more files.

For this I would introduce some new commands:
BeginCompressionBlock <name>
EndCompressionBlock <name>

Blocks can be nested in the sources (for use in macros with plugins for example). Inside a block, every instruction which causes a file to be added to the installer, will be added to the specified compression block.
If blocks with the same name already exists, any additional files will be appended.
If no block is open, a default block will be used.

So I could for example call BeginCompressionBlock at each section (and EndCompressionBlock at the end).
That way only the files for the sections which will be installed will be extracted. The remaining files won't be touched, as opposed to the current solid compression where those files still get extracted but discarded.

Hence it will also be possible to use seperate compression blocks for the install code/data and the plugins, thus dramaticaly reducing initialization time while still keeping a good compression ratio similar to old solid mode.

This would be a very useful feature for complex projects. Best example I can think of would be projects like XAMPP or other meta-projects.
Would it be very hard to implement this?

[kichik]

Starting with the shortest:

Would it be very hard to implement this?

Yes.

Next, a solution. If your installer loads slowly and shows the unpacking window, it's because your script is not optimally organized. I think the new ReserveFile descrption explains it best:

Reserves a file in the data block for later use. Files are added to the compressed data block in the order they appear in the script. Functions, however, are not necessarily called in the order they appear in the script. Therefore, if you add a file in a function called early but put the function at the end of the script, all of the files added earlier will have to be decompressed to get to the required file. This process can take a long time if there a lot of files. .onInit is one such function. It is called at the very beginning, before anything else appears. If you put it at the very end of the script, extract some files in it and have lots of files added before it, the installer might take a very long time to load. This is where this command comes useful, allowing you to speed up the loading process by including the file at the top of the data block instead of letting NSIS seek all the way down to the bottom of the compressed data block.

If your installer loads slowly and shows the verifying CRC window, it's because it's very large and CRCing it takes a while. Not much to be done there.

[Andy La Rubin]

When the script becomes big and hairy (and uses lots of macros) it might be hard to find/remember all the File commands, especially when the order of expanding macros is not clear.

Here's the method I used to optimize my script:

1. add line !define MUI_VERBOSE 4 into the script (you can remove it later). It'll cause all MUI macros to generate the needed output.
2. build the installer. Capture all makensis output into a file.
3. filter out all the lines except "File:" (grep is handy for it)
4. go through file list in reverse order and look for any files (plugin dlls, InstallOptions inis, MUI grafix, etc) which is used after whole bunch of files to be installed.
5. add all of these files as ReserveFile at the top of the script. Repeat procedure if necessary (to be sure that all ReserveFiles are placed before the main bunch)