This is very simple to do. Simply install your own files, then execute the other setup using ExecWait, then continue doing your own stuff.
File "Vendor Software.exe"
ExecWait '"$PLUGINSDIR\Vendor Software.exe"'
Rename "C:\VendorInstall\startup.exe" "C:\VendorInstall\Vendorstartup.exe"
Note that the user will need to go through the installation steps of vendor software.exe manually, unless that setup features a silent mode (something like ExecWait '"$PLUGINSDIR\Vendor Software.exe" /silent' ).
It sounds like you're new to NSIS. To learn NSIS, you should start by looking at NSIS\Example1.nsi and Example2.nsi. Learn what all the commands in those examples are for, and why they are used. You can use the command reference for this: http://nsis.sourceforge.net/Docs/Chapter4.html
Once you've done that, you will be well underway to building your own NSIS installer.