Functions for large CD/DVD installers

[General Barron]

Hello all, I've been leeching here for a while, so I figured I'd make my first post a contribution.

I've made a couple functions which we use in our game's installer, and I think might be useful to many other people. I fully hope/expect that the more experienced NSIS users here will improve/clean up these rough functions.

Here was our situation, or why I made the functions:

------

-Our game was ~4 gigs in size, distributed on DVD and/or multiple CDs. So I needed to be able to span the installer across multiple discs.

-We can't pack it all into the NSIS installer because the data is too large. We have no need to compress all files anyway, since it makes the install take longer and we have extra room on the discs.

-We send out different versions of our game to various customers, which all contain different content. We also have frequent updates/additions to our content. So I wanted an easy way to adjust exactly what content files are being included in a specific installer, to reduce human error. I also don't want to recompile a huge installer just to add/change the content if we make a change to one small file.

-I wanted the progress bar to give a good estimate of the installer's progress. This is difficult due to the above problems, and the fact that the realProgress plugin can't use floating point numbers.

-We ran into lots of trouble with files not being correctly copied off of the disc onto the hard drive (especially with CD installers). Sometimes the game wouldn't run after the install, due to corrupted files, even though the files on the disc were fine.

------

So I wrote these two functions. What they do is:

-Copy or extract (7z archive) a file from the CD/DVD into the install directory.

-Ensure the file was correctly copied / extracted, without errors. If it there were problems, it retries. If there are still problems after the retry, it aborts the install.

-Increases the progress bar (requires RealProgress plugin) according to how much copy/extracting remains. (Example: you are copy/extracting a total of 100mb throughout your entire installer; you copy over 2.3 mb. Progressbar increases by 2%. You then extract a file that is 12.7mb uncompressed. Progress bar now increases by 13%.)

Below are the functions, as well as (poorly written) instructions for use. They were written very quickly so they are very rough and ugly to use, please forgive me for that. I am going to try to improve/clean them up, unless someone else does so:

code:


;=========================

;FILE EXTRACT / COPY FUNCTIONS

;By Andrew Barron

;=========================



;Macros to perform file copying and extraction from install media.

;Before use, the following must be set up ahead of time:

;

; 7za.exe (7-zip command line utility) must be in the temp directory (only for extract macro)

; Realprogress plugin progress bar must be started (can be anywhere)

;

; strcopy $total_data (total size of data [bytes] after all copy/extract macros [uncompressed size])

; strcopy $total_progress (total amount to increase progress bar by while total_data is copied/extracted [integer 1-100])

; strcopy $0 0 ;this needs to be initialized at 0 before your first use of the functions

;

; strcopy $copyFrom (source location to copy/extract from)

; SetOutPath (destination location to copy/extract to)



;To use the macros:



;!insertmacro copy file2.xyz (filesize in bytes)

;!insertmacro extract file1.7z (uncompressed filesize in bytes)

;----------------------------------------------------------------------------------



Var total_data

Var total_progress

Var copyFrom



;macro to copy a non-compressed file from folder A to folder B

;filename = name of file to copy (no path)

;filesize = size of file in bytes

!macro copy filename filesize

  StrCpy $1 ${filename}

  StrCpy $2 ${filesize}

  call file_copy

!macroend



;macro to extract one file from folder A to folder B

;filename = name of file to copy (no path)

;filesize = size of UNCOMPRESSED file in bytes

!macro extract filename filesize

  StrCpy $1 ${filename}

  StrCpy $2 ${filesize}

  call file_extract

!macroend



;----------------------------------------------------------------------------------





;function to copy a non-compressed file from folder A to folder B

;$1 = file to copy, $2 = file size (bytes)

Function file_copy

  DetailPrint "Copy: $1"

  nsExec::Exec 'xcopy /Y "$copyFrom\$1" "$OUTDIR"'



  ;check error level from copy

  Pop $0

  intcmp $0 0 +2  ;if copy was successful

  call copy_error ;if copy was unsuccessful



  call increase_progress

FunctionEnd



;function to extract a non-compressed file from folder A to folder B

;$1 = file to copy, $2 = extracted file size (bytes)

Function file_extract

  DetailPrint "Extract: $1"

  Push 0 ;push a flag on the stack

  

  attempt_extract:

  nsExec::Exec 'xcopy /Y "$copyFrom\$1" "$TEMP"' ;copy to temp folder on hdd first; extracting directly off disc is error-prone



  ;check error level from copy

  Pop $0

  intcmp $0 0 +2  ;if copy was successful

  call copy_error ;if copy was unsuccessful



  ;extract the file to proper directory

  nsExec::Exec '"$TEMP\7za.exe" e "$TEMP\$1" -o"$OUTDIR" -y'

  delete /REBOOTOK "$TEMP\$1"

  

  ;check error level from extraction

  Pop $0

  intcmp $0 0 extract_complete ;if extract was successful

  

  ;if there are errors and this is the FIRST attempt, re-attempt the copy / extract once

  ;if this is the second attempt and there are still errors, abort the install

  Pop $0

  intcmp $0 1 extract_failed ;if this was second attempt

  DetailPrint "Re-extract: $1"

  goto attempt_extract

  

  extract_failed:

  call extract_error



  extract_complete:

  call increase_progress

FunctionEnd



Function copy_error

  MessageBox MB_OK "Install error: could not copy file $copyFrom\$1. Harddrive may be out of space, or install media may be damaged."

  Abort "Install error!"

FunctionEnd



Function extract_error

  MessageBox MB_OK "Install error: could not extract file $copyFrom\$1. Install media may be damaged."

  Abort "Install error!"

FunctionEnd





;Helper function used in extract/copy functions. Used to smoothly increase the progress bar.

;$2 = how much data was just copied/extracted

;$9 = leftover progressbar increase that wasn't applied from last operation(s); must be initialized (set to 0) before the first copy / extract

Function increase_progress

  ;figure out how much of total copy/extract operation we completed.

  strcpy $0 $total_data

  math::script /NOUNLOAD "r0 += 0.1; r0 = r2 / r0"

  strcpy $1 $total_progress



  math::script /NOUNLOAD "r0 = r0 * r1 * 100"   ;find percentage amount to increase progress bar by

  math::script /NOUNLOAD "r0 += r9"       ;add in the leftover increase that wasn't applied from the last operation(s)

  math::script /NOUNLOAD "r9 = r0; r0 = flr(r0)" ;trim decimal off of the amount we will increase bar by; it is now an integer

  math::script /NOUNLOAD "r9 -= r0"    ;find the remainder after decimal; this will be left for the next time.



  ;check if we are above 1% to add to the progressbar

  math::script /NOUNLOAD "r1 = r0 > 0"

  IntCmp $1 0 done                       ;can't intcmp on $0 because it is formatted as a non-integer (has decimal)



  ;increase progress bar by the integer amount stored in $0. Do this by incrementing by 1 until we are out of percentage to add.

    increment:

  RealProgress::AddProgress /NOUNLOAD 1   ;add 1% to progress bar

  math::script /NOUNLOAD "r0 -= 1; r1 = r0 > 0" ;decrement $0 and check if it is still > 0

  IntCmp $1 0 done

  Goto increment                          ;if not, go back and do it again



    done:

FunctionEnd



;----------------------------------------------------------------------------------

Comments and improvements are very much welcome. I really would like to make it nicer to use.

I'd especially like to add automatic file size detection, so you don't have to write that in when using the macros, and maybe so it automatically uses 'addsize' for that file.

It also would be useful, at least for me, to have it just point at a directory on the disc, and then it automatically copies/extracts all the files in there. This way I wouldn't have to bother messing with file lists.

[Red Wine]

Interesting
This kind of installer has been asked several times from several users.
However would be nice if you post those functions at wiki where it's easy for everyone to find them.

[General Barron]

Originally posted by Red Wine
Interesting
This kind of installer has been asked several times from several users.
However would be nice if you post those functions at wiki where it's easy for everyone to find them.

Yes, well I was hoping for a bit of feedback/improvements first. But posting to the wiki is a good idea