Thanks a lot to Nunzio390 for the link to the "control and delete" component : that's what I was looking for. Works great! I hope they make bigger icons in the next version, though.
Another way to do this which I just found is to install a plugin "Linas.info Hotkeys for Winamp 3 - 0.4" at http://linas.info/
: you can configure hotkeys the way you like, and functions include "Delete current song from playlist;
Delete current song from disk! (File will be completely deleted without any warning!); ". I find it even easier to use than "control and delete" component.
As for Winamp2 (I use version 2.80), the function I was referring to can be found at the bottom of the playlist : click "rem" (second item from the left), then choose "rem misc", then "physically remove selected file(s)". If you click on that, a confirmation box says : "Using this command will remove all selected files permanently from disk. Are you sure you want to do this?".
I hope my explanations were clear enough in my first message : my mother tongue is not English (French!).