-- You're running WinAMP or another player for listening that does crossfading
-- when you switch from port 80 to port 81, what's left of the port 80 stream "volume+quality" will begin to get lower [fade out+remove silence] while the connection attempt is made on port 81, at some point it will die [when the buffer has run out ~90 seconds]. If you try port 81 again, you'll likely get nothing+the error message. You kind of answered your own question about the delay -- it's the buffer+crossfade that allows you to think that a connection to port 81 works, but it also allows you to continue listening long after you have disconnected.
You can "see" what's really happening if you look at the DNAS GUI while you do this -- in every case, you will not see a successful connection attempt when you use port 81 to listen, that connection will be refused by the DNAS, especially if a source is present.
add: Estimating delay
-- Roughly, a 128kbps MP3 is 1 MegaByte per minute -- the DNAS buffers about 1 MB of source regardless of bitrate, so the maximun source to DNAS delay for 128kbps would be 1 minute -- 64kbps is 500KB/min, so the delay would double to 2 min, at 32kbps and 250KB/min, you have a whopping 4 minute maximum delay. --- when your delay exceeds the maximum [you are behind] -- you will underrun. That's why a lower bitrate stream is less prone to skipping.