Multithread compression and decompression in makensis

partyzone · 2nd June 2010, 22:31

Hi,

I have an automated script that produces an installer with makensis and the installer is run automatically for testing purpose. I noticed that only one core is in use during compression and decompression. I think that a multi-threaded compressor would be great for installer creation and for user installation as most of our modern computers have four cores...

What do you think of that?

MSG · 3rd June 2010, 04:57

If I remember correctly, there are some threads about this topic already.

Afrow UK · 3rd June 2010, 10:57

What compression are you using? LZMA uses multithreading.

Stu

partyzone · 3rd June 2010, 16:53

Sorry, it seems that I "multi threaded" the topic

Thanks for your fast answers!

I use the default algorithm. In the documentation, there is no mention of multi threading, and furthermore, the description seems to describe lzma as the slower one...

Extract:
"NSIS supports different compression methods, as explained here. ZLIB is the default compression method, which is fast and uses only a little bit of memory. LZMA is a good method for the creation of small installers for internet distribution. BZIP2 usually compresses better than ZLIB but not as good as LZMA, it is useful if you need lower memory usage or fast script compilation."

I'm a little bit confused...

Is it because the multithreading was added lately? Is it because even with multithreading, the compression is still slower? Should I benchmark for myself or I should pay confidence in the documentation?

Regards,

Afrow UK · 3rd June 2010, 18:27

http://www.7-zip.org/7z.html

Personally, I always use LZMA. The only downside to its use is slow compression. On the other hand it has extremely good compression ratios and fast decompression.

Stu

bytebucket · 9th July 2012, 14:27

we use LZMA, but on a multi-core build machine only one core appears to be utilized. Afrow UK indicated here that LZMA is multi-threaded, but that was a couple years ago.

Can anyone confirm whether or not LZMA is multi-threaded? It doesn't appear to be from what we can see when running with that option.

DrO · 9th July 2012, 15:14

lzma can support multi-threaded compression but only if the compressor is coded to do it and NSIS uses a heavily altered version which is single-threaded (and i cannot remember if the multi-threaded lzma libraries were around when the NSIS lzma support was added).

-daz

bytebucket · 9th July 2012, 18:30

ok, thanks for the clarification. so it sounds like there's no way currently to get multi-threaded compression with NSIS when building an installer? Do you know if there are any plans in this area to provide this?

bytebucket · 17th July 2012, 16:00

so are there any plans to add support for multi-threaded compression?

DrO · 17th July 2012, 16:18

i don't know. all i can suggest is looking through the bug / feature tracker and seeing if anything is in there but just being mentioned doesn't mean anything would happen (since it would need someone to find the time / inclination to work on it).

-daz

bytebucket · 18th July 2012, 18:29

where is the bug tracker? I can't seem to find a link for it anywhere on the site here.

Afrow UK · 18th July 2012, 19:10

http://nsis.sourceforge.net/Bug_Reports

Stu

2nd June 2010, 22:31	#1
partyzone Junior Member Join Date: Jun 2010 Posts: 2	Multithread compression and decompression in makensis Hi, I have an automated script that produces an installer with makensis and the installer is run automatically for testing purpose. I noticed that only one core is in use during compression and decompression. I think that a multi-threaded compressor would be great for installer creation and for user installation as most of our modern computers have four cores... What do you think of that?

3rd June 2010, 10:57	#3
Afrow UK Moderator Join Date: Nov 2002 Location: Birmingham, England Posts: 8,202	What compression are you using? LZMA uses multithreading. Stu Need an installer? http://www.afrowsoft.co.uk

3rd June 2010, 16:53	#4
partyzone Junior Member Join Date: Jun 2010 Posts: 2	Compression Algo Sorry, it seems that I "multi threaded" the topic Thanks for your fast answers! I use the default algorithm. In the documentation, there is no mention of multi threading, and furthermore, the description seems to describe lzma as the slower one... Extract: "NSIS supports different compression methods, as explained here. ZLIB is the default compression method, which is fast and uses only a little bit of memory. LZMA is a good method for the creation of small installers for internet distribution. BZIP2 usually compresses better than ZLIB but not as good as LZMA, it is useful if you need lower memory usage or fast script compilation." I'm a little bit confused... Is it because the multithreading was added lately? Is it because even with multithreading, the compression is still slower? Should I benchmark for myself or I should pay confidence in the documentation? Regards,

3rd June 2010, 18:27	#5
Afrow UK Moderator Join Date: Nov 2002 Location: Birmingham, England Posts: 8,202	http://www.7-zip.org/7z.html Personally, I always use LZMA. The only downside to its use is slow compression. On the other hand it has extremely good compression ratios and fast decompression. Stu Need an installer? http://www.afrowsoft.co.uk

18th July 2012, 19:10	#12
Afrow UK Moderator Join Date: Nov 2002 Location: Birmingham, England Posts: 8,202	http://nsis.sourceforge.net/Bug_Reports Stu Need an installer? http://www.afrowsoft.co.uk

3rd June 2010, 04:57	#2
MSG Major Dude Join Date: Oct 2006 Posts: 1,841	If I remember correctly, there are some threads about this topic already.

9th July 2012, 14:27	#6
bytebucket Junior Member Join Date: Jul 2012 Posts: 6	we use LZMA, but on a multi-core build machine only one core appears to be utilized. Afrow UK indicated here that LZMA is multi-threaded, but that was a couple years ago. Can anyone confirm whether or not LZMA is multi-threaded? It doesn't appear to be from what we can see when running with that option.

9th July 2012, 15:14	#7
DrO - Join Date: Sep 2003 Location: UK Posts: 22,265	lzma can support multi-threaded compression but only if the compressor is coded to do it and NSIS uses a heavily altered version which is single-threaded (and i cannot remember if the multi-threaded lzma libraries were around when the NSIS lzma support was added). -daz

9th July 2012, 18:30	#8
bytebucket Junior Member Join Date: Jul 2012 Posts: 6	ok, thanks for the clarification. so it sounds like there's no way currently to get multi-threaded compression with NSIS when building an installer? Do you know if there are any plans in this area to provide this?

17th July 2012, 16:00	#9
bytebucket Junior Member Join Date: Jul 2012 Posts: 6	so are there any plans to add support for multi-threaded compression?

17th July 2012, 16:18	#10
DrO - Join Date: Sep 2003 Location: UK Posts: 22,265	i don't know. all i can suggest is looking through the bug / feature tracker and seeing if anything is in there but just being mentioned doesn't mean anything would happen (since it would need someone to find the time / inclination to work on it). -daz

18th July 2012, 18:29	#11
bytebucket Junior Member Join Date: Jul 2012 Posts: 6	where is the bug tracker? I can't seem to find a link for it anywhere on the site here.

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode