Well, the data block is saved is a memory mapped file which means it creates a temporary file and maps needed parts to memory. This is done so the data block will be able to grow bigger than your memory. I don't really know what is the problem with Windows 9x, but I guess it has a bug which causes it not to delete those temporary files.
The data block can't be directly written to the output file because the headers size is not yet known and moving the data block around in the file will take a lot of time.
There is a size limit on the installer which is:
Installers can be as large as 2GB (theoretically -- when building on Win9x the limit seems to be around 500MB, however building on NT then installing on Win9x works with larger sizes)
You probably got that error because you crossed this limit.