... but I don't know how to stop my installation
There's probably several ways to handle this, but here's a few of my suggestions:
Option #1 (assumes a custom page for QT not installed)
Display your custom "quicktime is not installed" page. Then, in that page's callback function, issue the QUIT command. This will stop the installation.
The quit command will NOT allow any of your other callback functions to run (such as .onInstFailed). If you require other cleanup functions to run when the installation quits, you'll need to wrap these instructions along with the Quit command inside your own function and then call your special function instead of just issuing a stand-alone quit command.)
Option #2 (assumes you'll use just a MessageBox):
Check for QuickTime during .onInit. If not installed, perhaps just display a message box letting the user know what's going on followed by the abort command. Probably something like this:
; *** [check for QuickTime here] ***
; Pretend $0 = "Y" if Quicktime is installed
StrCmp $0 "Y" QT_YES QT_NO
; QuickTime is NOT installed:
MessageBox MB_OK "QuickTime is NOT installed.$\rPlease install it."
; quicktime is installed:
; *** Continue your function here ***