ADVANCED Streaming Issue Involving Missing Stream Titles
 

Hi all,

We have just made an arrangement with our datacenter to directly link our streaming source server to our frontline server where we have six SHOUTCast instances serving different streams. Each of the two boxes now has a second NIC with a cable linked between them for a crossover.

The IP stack of the source server's second NIC is:

IP: 10.0.0.1
SM: 255.255.255.0
GW: 10.0.0.2

The IP stack of the frontline server's second NIC is:

IP: 10.0.0.2
SM: 255.255.255.0
GW: 10.0.0.1

The frontline server's second NIC also has 10.0.0.3 - .9 setup as secondary IP's, the SHOUTCast server instances each have one of these configured as SrcIP's, and the source server points each of its six streams to one of these IPs.

This all works fantastic except for one annoying problem - the stream titles no longer appear whereas they have been fail-safe for the past four years. The source has titles enabled and no settings related to stream titles were changed anywhere. All SHOUTCast instances use an included .conf set with "TitleFormat=%s".

Any expert SHOUTCasters out there familiar with this issue?

PLEASE DO NOT REPLY TO THIS POST/THREAD ***UNLESS*** YOU KNOW ***EXACTLY*** WHAT WE ARE DOING, WHY WE ARE DOING IT, AND YOU ARE FAMILIAR WITH THIS PROBLEM.

Thank you in advance!

what does your shoutcast config look like?

thank you for asking. the key pieces of information were included above. could you be more specific, please?

Is this across all programs used to listen, or one specifically?
Windows Media Player does not support title streaming, neither does Winamp if you installed the MP3Pro plugin. Just want to make sure this can be confirmed with a clean install of Winamp

all progs. in fact we changed no config relating to stream titles prior to completing the server crossover networking arrangement last night.

we had no idea anything was amiss until we began getting listener complaints this morning and then observed it for ourselves.

are your configs bound to a specific ip?

each stream has its own SrcIP and DestIP

the streams are working exactly as they should and have hundreds of concurrent listeners. nothing is different except that the source box and the frontline box are using a crossover and the stream titles mysteriously stopped working. (critical details in the root post of this thread).

are you here to advertise your DnB station or do you have some idea of what is the problem here?

:rolleyes: nice attitude.. I was thinking that now that the machine has two nics and two ips, maybe the ips you are binding to are causing the issue.. but.. you aren't posting your config, you aren't giving any additional info on what you've tried (i.e. does it start working again when you pull the other nic)...etc.etc... so you you know what.. ill let you figure this out for yourself. :down:

the additional info on what we tried is that everything worked previously. there were always multiple IPs in this equation, the difference is they were public IPs previously and now they are loopback IPs.

nobody who has been doing this as long as you and I have are going to post their configs. save the attitude buddy. your station ads in this thread are much larger than your contribution here. if you want to make a suggestion, get to the point and lose the stupid fucking ads.

if the source server a windows box or *nix type ?

can you provide

route print

if windows or

netstat -rn

*nix

I have a sneaky suspicion it might a limitation on routing as the title updates are made using a URL call which is not in the bitstream sent for the audio, so perhaps the title update is going out the wrong interface.

BW

Thank you BW. Here it what you requested (public IP's obscured):

IPv4 Route Table
===========================================================================
Interface List
0x1 ........................... MS TCP Loopback interface
0x10003 ...00 30 48 2a 83 e7 ...... Intel(R) PRO/1000 MT Dual Port Server Adapte
r
0x10004 ...00 30 48 2a 83 e6 ...... Intel(R) PRO/1000 MT Dual Port Server Adapte
r #2
===========================================================================
===========================================================================
Active Routes:
Network Destination Netmask Gateway Interface Metric
0.0.0.0 0.0.0.0 10.0.0.2 10.0.0.1 20
0.0.0.0 0.0.0.0 2**.5*.138.1 2**.5*.138.43 20
10.0.0.0 255.255.255.0 10.0.0.1 10.0.0.1 20
10.0.0.1 255.255.255.255 127.0.0.1 127.0.0.1 20
10.0.0.3 255.255.255.255 127.0.0.1 127.0.0.1 20
10.0.0.4 255.255.255.255 127.0.0.1 127.0.0.1 20
10.0.0.5 255.255.255.255 127.0.0.1 127.0.0.1 20
10.255.255.255 255.255.255.255 10.0.0.1 10.0.0.1 20
6*.1**.4.112 255.255.255.248 6*.1**.4.114 2**.5*.138.43 20
6*.1**.4.114 255.255.255.255 127.0.0.1 127.0.0.1 20
6*.1**.4.115 255.255.255.255 127.0.0.1 127.0.0.1 20
6*.1**.4.116 255.255.255.255 127.0.0.1 127.0.0.1 20
67.255.255.255 255.255.255.255 2**.5*.138.43 2**.5*.138.43 20
127.0.0.0 255.0.0.0 127.0.0.1 127.0.0.1 1
2**.5*.138.0 255.255.255.0 2**.5*.138.43 2**.5*.138.43 20
2**.5*.138.43 255.255.255.255 127.0.0.1 127.0.0.1 20
2**.5*.138.255 255.255.255.255 2**.5*.138.43 2**.5*.138.43 20
224.0.0.0 240.0.0.0 10.0.0.1 10.0.0.1 20
224.0.0.0 240.0.0.0 2**.5*.138.43 2**.5*.138.43 20
255.255.255.255 255.255.255.255 10.0.0.1 10.0.0.1 1
255.255.255.255 255.255.255.255 2**.5*.138.43 2**.5*.138.43 1
Default Gateway: 2**.5*.138.1
===========================================================================
Persistent Routes:
None

had trouble getting the font set as "courier". see attached for text file.

so you have

2**.5*.138.43
10.0.0.1

as interface IPs ( just making sure we have clarity and I am reading it correctly )

this looks correct

but then

all the IPs 10.0.0.1,3,4,5 are routed via a loopback interface ? seems wrong


the above seem fine

this seem a bit wrong again, I assume the public facing IPs of the other servers, rather than the private ones ?

the rest seem fine

you could remove the routes for

10.0.0.1 255.255.255.255 127.0.0.1 127.0.0.1 20
10.0.0.3 255.255.255.255 127.0.0.1 127.0.0.1 20
10.0.0.4 255.255.255.255 127.0.0.1 127.0.0.1 20
10.0.0.5 255.255.255.255 127.0.0.1 127.0.0.1 20

as you have a catch all previously

6*.1**.4.114 255.255.255.255 127.0.0.1 127.0.0.1 20
6*.1**.4.115 255.255.255.255 127.0.0.1 127.0.0.1 20
6*.1**.4.116 255.255.255.255 127.0.0.1 127.0.0.1 20

also remove these ... possibly.

There is also a possible other solution, but would involve a local instance of Shoutcast on the source server, so updates would go locally and any relay would get the title in the bitstream, as is the case with relays.

BW

that was from the "source" server. attached is from the "frontline" server.

looks like a similar routing configuration on the frontline server.

I would investigate if there is a reason for the routes having been added, or possible remove one of them and see if that Shoutcast titles work there after ( might have to stop/start the encoder again )

BW

we can definitely try removing a route. which one would you recommend? a loopback or one of the public ones?

just tried pointing a test stream to a public IP set as a SrcIP rather than a loopback IP and get the same results. Very strange as this had worked previous to the crossover.

I setup a SC instance on the source, pointed a stream to it, and relayed off of it from the frontline server. The stream titles ONLY worked when using a public-facing IP address. the titles do not stream if a loopback IP is involved in the configuration in any way, whether the stream source is pointing to a loopback IP or relaying off of a loopback IP.

a shoutcast bug perhaps? although this could be a working solution to the problem, it is a bit superfluous involving more moving parts than necessary.

any other ideas or suggestions? the whole purpose of doing the crossover is sending the stream from the source to SC in a single hop. using public facing IPs involves having to send packets to the data switch which defeats the purpose of the crossover. there has to be something we're overlooking here.

i would remove all the private addresses that route to 127.0.01 from both sides

ie.

10.0.0.1 255.255.255.255 127.0.0.1 127.0.0.1 20
10.0.0.3 255.255.255.255 127.0.0.1 127.0.0.1 20
10.0.0.4 255.255.255.255 127.0.0.1 127.0.0.1 20
10.0.0.5 255.255.255.255 127.0.0.1 127.0.0.1 20

and

10.0.0.2 255.255.255.255 127.0.0.1 127.0.0.1 20
10.0.0.6 255.255.255.255 127.0.0.1 127.0.0.1 20
10.0.0.7 255.255.255.255 127.0.0.1 127.0.0.1 20
10.0.0.8 255.255.255.255 127.0.0.1 127.0.0.1 20

As they do not seem to do anything as the data is being routed to a loopback address.

BW

we don't have any settings explicitly routing 10.0.0.* IPs to 127.0.0.1, so it must be a default. do you know of any way we can override that in Win2K3?

use

route del

(should display help for the rest of the syntax)

to try and remove them.

These arent defaults, I have checked 2 Win2k3 servers, one has 3 NIC cards configured.

This could of course be completely the wrong path to solve the problem, but it does look odd, routing to a loopback address.

BW

thank you - seems like a pretty good suggestion but I'm sure I'm missing something here.
c:
i'm getting "route specified was not found" with
>route delete 127.0.0.1

>route print reveals that 127.0.0.1 is the gateway for all these 10.0.0.* IPs although 10.0.0.2 (source) and 10.0.0.1 (frontline) are set explicitly as the gatweays in the Advanced TCP/IP settings panels where the secondary IP addresses are set.

try

route del 10.0.0.1 255.255.255.255 127.0.0.1 127.0.0.1

BW

I have also just tried this on the server where 10.0.0.8 is a secondary IP and got "route specified was not found":

>route CHANGE 10.0.0.8 MASK 255.255.255.0 10.0.0.1

but yet, ROUTE PRINT clearly shows this IP, its mask and its gateway as 127.0.0.1

Is this safe? on one box, 10.0.0.1 is the IP address of the second NIC. on the other, it's the gateway.

okay, tried that and a route print shows it remains with 127.0.0.1

tried that same thing with route change and the gateway as 10.0.0.2 and get "route specified was not found". Windows weirdness.

could well be a red-herring, as I have checked on a couple of other windows boxes I have and XP shows the 127.0.0.1 behaviour ... so I would suspect it perhaps might not be the routing issue.

Very very odd though, perhaps a windows patch causes it .. not sure ...

BW

thank you again BW, nonetheless.

google searching this issue revealed that windows seems very sentimental about 127.0.0.1.

any other ideas? anyone else out there either? the crossover setup was a panacea to many networking issues that can occur with SC and stream titles are simply an indispensible feature....

thanks!

I would have thought if it was a routing issue, the stream would be affected too. Not only would you not get titles, but you wouldn't get audio either.

Here are some things to maybe try.

1) Send metadata manually to a DNAS just to see if it shows up. (Type the URL in a browser and hit "go")

code:
http://serverIP:serverPort/admin.cgi?pass=****&mode=updinfo

&song=Test%20Title&url=http://www.myurl.com

(You need the URL parameter too or it won't update)

If that works and the DNAS shows your test song title, then you're possibly looking at SAM misbehaving for some reason.

An interesting side-test would be to perform the above using a browser on the source machine, and also from a browser on the destination machine. Does it work in both cases?

If the title doesn't show up when you try that on either machine, then I'd guess it's a DNAS config problem. Seems unlikely, but I know on another forum you'd mentioned that TitleFormat=%s was in the config. Is that how it's always been? Have you tried commenting that out completely just to rule it out? It is commented out by default.

2) Perhaps try Bored_Womble's suggestion of a test DNAS on the *source* server and relay that to the production box (Assuming titles show up on the test DNAS on the source server)

Those are the only two suggestions I could think of, but maybe one of them might yield results... Hope it helps.

Sorry for the second post, but I just had a thought. I don't know how you confirm this, but what if SAM sends title updates like the method I outlined above?

The DNAS web could be listening only on the DestIP address, and maybe not listening on the SrcIP at all!

In that instance, it might be that the difference in networks between the SrcIP and DestIP addresses could be causing the trouble. If that's true, then the solution would probably be to set SrcIP=ANY and set those production instances to *relay* from private DNAS instances on the source server. With the 10.x.x.x private address as the source DNAS, windows routing will find them on the private NIC just fine.

I'd be curious if this is what's causing it... post back if you find out?

Hi DotMe,

Thank you for the suggestions. These all seem pretty good. The test in the original message worked. Commenting out TitleFormat does not make a difference.

I would like to try the second suggestion but we have six streams on different addresses, so I am not sure setting SrcIP=ANY would work since SC needs to know which stream it is getting its source. :(

We tried the SC instances on the source with the loopback addresses earlier today and it still did not work. It does not seem to work if there is any loopback address anywhere in the equation.

@moosifer

Might i suggest that next time when you post you pipe down your attitude a bit.

This forum is populated by fellow shoutcasters who volunteer as helpdesk for free software.

Comming in with things like

and

Drivehell asked you twice to provide more information.
You told him to get lost, while you provide that info if another forum member is asking the same thing.

are not exactly showing that you can behave yourself.
but are showing that you have absolutely zero skills in asking nicely.

Your style can be descibed as demanding while it now starts to go into the direction of a SAM problem.

And you know that SAM is not shoutcast.
www.spacialaudio.com

Next time when running into a problem at least try to show that you are not some punk bedroom dj that is using dad his bandwidth but that you are a fellow shoutcaster who needs a bit of assistance..

I didn't provide my config to ANYONE nor did I tell ANYONE to get lost. The issue at hand IS NOT verified to be a SAM problem either and looks VERY MUCH like a SHOUTCast bug.

Bedroom DJ? Dad's bandwidth? Before jumping to conclusions without fully reading a thread and jumping down someone's throat undeservedly, why don't you pipe down your own attitude?

The request for qualified responses is QUITE VALID in the Shoutcast forums. Being a Forum King, I'm sure you've noticed. After all, one just might get irrelevent responses like this...or even get another guy spamming the thread with ads to his station (which have since been removed).

You just proved my point with your reply

absolutely a bad attitude.

and you can't read either

I never told you names , i compared situations.


Since this forum is populated by fellow shoutcasters and it's no official helpdesk you have NOTHING to demand.
You should count yourselves lucky that somebody is willing to spend time to help you out.

I didn't prove anything. But what are YOU trying to prove - that you have NOTHING BETTER TO DO than post irrelevant replies to a thread you know NOTHING ABOUT HOW TO SOLVE?

If once in a year I have an issue that I actually need to come to these forums to post, it will ONLY be because I've already searched the web in general, searched these forums, worked on the issue myself, and have run out of solution attempts. You OBVIOUSLY don't know - even as a FORUM KING - how fluffy these threads get with two-cent replies and self-indulgent filler like your own reply here. This is a VERY technical issue I've run into and in the event another soul runs into this SAME ISSUE, wouldn't it be nice that the thread is filled with RELEVENT responses?

If you want to make a contribution to the same cause that you proclaim support, save your responses until you have something VALUABLE to contribute to the thread.

This is very likely a SHOUTCast bug as SC clearly has trouble working in a crossover configuration with loopback IPs. And if you don't know the first thing about any of that, well FINE. Stay OFF this thread.

hmm... have you tried packet sniffing?

That, to me, points at SAM. I run instances here with 127.0.0.1 in the encoder and it works great. But...

Nothing like a good challenge on a Saturday :)

I still suspect that one possible issue is that the IP used on the production machine to receive the audio is the SrcIP, but the IP used to receive titles might be the DestIP if SAM uses HTTP to send that info.

This is lengthy, but hopefully will make sense. I think at this point it's worth a try at least with stream 1.

1) Source Machine - set up a DNAS with SrcIP and DestIP *both* set to 10.0.0.1, port 7000, Public=Never (To prevent that server from advertising)

2) Create a SAM encoder with titles enabled, and point it to the source machine at 10.0.0.1:7000

Now for the million dollar question: Do you get titles on that DNAS after the next song change in SAM?

Answer No? Definitely a SAM Problem - open Spacial ticket.
Answer Yes? Next step below...

3) Production Machine - DNAS is set to relay with Public=Always (Assuming you want it in the YP), SrcIP=Any, DestIP=(Enter a public IP for listeners here). Portbase can be 80 or whatever you're using for the public to listen on. In the config...

SrcIP=Any
DestIP=aPublicIPGoesHere
Portbase=80 (example)
RelayPort=7000
RelayServer=10.0.0.1

That should create a successful "Stream #1" with titles.

If that works, repeating for the remaining streams should be a piece of cake. You would end up with

Source Server DNAS instances on:
10.0.0.1:7000 - Stream 1
10.0.0.1:7002 - Stream 2
10.0.0.1:7004 - Stream 3
10.0.0.1:7006 - Stream 4... etc

Production Server DNAS instances with:
Relay from 10.0.0.1:7000, available on public IP xx.xx.xx.x1:80 - Stream 1
Relay from 10.0.0.1:7002, available on public IP xx.xx.xx.x2:80 - Stream 2
Relay from 10.0.0.1:7004, available on public IP xx.xx.xx.x3:80 - Stream 3
Relay from 10.0.0.1:7006, available on public IP xx.xx.xx.x4:80 - Stream 4... etc

EDIT - The SrcIP=Any on the production server might create a "Port already in use" error. If that happens, set the SrcIP the same as the DestIP (The public IP address) because it doesn't matter. When you relay another DNAS, the SrcIP becomes irrelevent. It's not used. /Edit

Thank you for the additional tips, DotMe!

Yes, I will second that - except that Labor Day weekend plans will limit the three hours I've already spent testing this out this morning.... :(


Answer: Yes, it worked without a hitch.

Here is what I struggled with all morning - the test relay pointing to the SC instance on the source keeps coming back with "error creating socket". The SC on the source is using 10.0.0.10 for SrcIP and DestIP and the encoder points to the same. The relay (frontline server) SrcIP points to 10.0.0.10 and the DestIP is a public address. Most puzzling. All usual suspects have been verified, including AllowRelay=Yes and RIPOnly=No.

This exact same arrangement worked without a hitch yesterday and is now leaving me with only a hammer in my problem-solving toolkit, so for now I am going to go and enjoy some of my planned Labor Day weekend festivity. I will resume this tomorrow and post any newer findings.

Setting SrcIP and DestIP the same is what I would typically do when using public-facing IPs and I would point my stream source to the same. Can't really use SrcIP=ANY in this situation since we have multiple streams and have to bind each SC instance to each specific source. As far as relay settings go, I'd always just comment out SrcIP.

Once again, I'll pick this up again tomorrow unless somehow the beer I plan on drinking today makes me want to approach this again with a "renewed outlook" later tonight. :D

Thanks again DotMe!

Sorry for the delayed reply, DotMe, and thank you in advance for your continued participation with this thread.

Okay, so this also will not work for some reason just as mystifying as the absent stream titles.

Just to reiterate what is suggested and attempted:

* A relay was setup on the source with the encoder streaming to it directly via 10.0.0.11. The source relay DestIP is 10.0.0.11 and was tried with SrcIP as 10.0.0.11 and on another attempt with ANY.

* The public relay had RelayServer=10.0.0.11 and was set with RelayPort equal to the PortBase setting on the source relay. DestIP is a public IP address, which is irrelevant to this problem.

The new problem is that we get an error creating a relay socket whenever a loopback IP is used. This was attempted multiple times, even starting from scratch just for shits and giggles. Tracert from the frontline server to 10.0.0.11 successfully reaches it in one hop, as intended. Ping is successful.

So that we do not lose the subject of this forum post troubleshooting the failure of a theoretical solution (which probably reveals yet ANOTHER potential SHOUTCast bug, incidentally), suffice it to say that I cannot think of any other possibilities to get past this issue without nixing the crossover altogether, which is really not an option given that it was performed to improve the QoS from within the datacenter between the source server and the frontline server, bypassing routers.

Anyone have any other ideas?

I think it is safe to say in closing that this is a SHOUTCast bug on Windows Server 2003. I cannot imagine any further possibilities to correct this issue or to troubleshoot it. After all, SHOUTCast configuration couldn't be any simpler.

At any rate, if one is using SAM, a PAL script can easily be written to update the stream titles using the WebToFile method. This may not be a SAM problem after all, but at least a workaround for this SHOUTCast bug is available for those who use SAM.