User Agent Empty

[pjrhapsody]

Hi

I have read up across a number of posts about the stream rippers from Germany etc and it has been mentioned about the empty user agent. but it's not clear if there is an option to ban all those who don't supply a user agent.

My question is, is this possible to ban an empty user agent and if so what do we enter into the field ?

[djSpinnerCee]

the ban list can't handle this issue -- there are several ways that the user-agent will be blank, it could be NULL, one or more blank spaces, or the client may not include a user-agent header -- I don't think the dnas has a way to handle those special cases.

[pjrhapsody]

Thanks Spinner

I just thought that those wanting to steal your stream etc, if they don't identify themselves then it would be good to say, well you cant access the stream.

[dopelabs]

unless you are password protecting tune in access to your streams, you are providing anything you broadcast as free stuff to the world. if anyone can tune in, anyone can record the audio. plain and simple. cant really call it 'stealing' when your giving it away freely =].

[mattauckland]

I've actually just had a conversation with DrO on Twitter regarding this issue, as I'm experiencing the same issue, and he pointed out that you can block empty UA strings, using the following configuration option:

"Added 'blockemptyuseragent' configuration option to allow for preventing client connections without a user agent from connecting (note: some valid clients e.g. some hardware devices may not provide a user agent and enabling this may incorrectly block legitimate client connections)" ~ From the 2.2.2 changelog.

Just though that might be of interest.

[pjrhapsody]

Thanks for the heads up

Since i made my original post i have found that a web site was able to capture my stream information even though we hadn't published it anywhere, couldn't understand why we were getting logons from around the world.

Turns out that in the winamp dsp (Directory Tab)even with the box unticked information in the fields would be picked up by this web site directory of internet stations.

The web site would then show it as live the moment we started broadcasting. I removed all the old information and hey presto no more world wide logins.

For us this is important as we would only like people in the uK to benefit and listen through our web site app.

[sqgl]

DNAS v2.5.1 has "Block User Agent" button/link on the admin page. Perhaps this works on null agents too.

I also get a disproportionate number of null agents from Germany in particular. And how do we know they are people ripping? They might be bots though I cannot imagine what they gain from tuning in.

[neralex]

** Empty ** user-agents are not always bots or stream-rippers! There are a lot of apps/tools on different OS, which are providing incorrect user-agents but in the most cases they are real listeners. So be careful to block IPs with ** Empty ** user-agents.

Something like 'Winamp 5.50' is known for a popular choice of a faked user-agent. I had a simular issue in the last weeks with a Germany based streaming-directory which is collecting stream-URLs without permissions of the affected stations. They are running different bots with this old 'Winamp 5.50' user-agent to rip the streams in order to get playlists from stations with private servers and without public access to the stats-pages of DNAS. You can identify these bot on the long connection-time and mostly with a short view on the IP trace-route.

[sqgl]

Originally Posted by neralex

** Empty ** user-agents are not always bots or stream-rippers!

Thanks for pointing this out.

Originally Posted by neralex

Something like 'Winamp 5.50' is known for a popular choice of a faked user-agent. I had a simular issue in the last weeks with a Germany based streaming-directory which is collecting stream-URLs without permissions of the affected stations...You can identify these bot on the long connection-time and mostly with a short view on the IP trace-route.

In my case these WA5.50 semi-permanent connections belong to obscure ISP's:

• Hetzner (DE)
• Linode (which is simultaneously in DE and US depending on the lookup used)
• Zen (UK)

Though it seems inefficient, could it be that they are bots trying to attract the attention of the administrator simply to spread brand awareness? It reminds me of the way that bots will hit web pages so as to show up in the analytics pages - or are they attempts to boost their google rankings by being referenced? (although I suspect Google would not count such listings in their algorithm).

Originally Posted by neralex

They are running different bots with this old 'Winamp 5.50' user-agent to rip the streams in order to get playlists from stations with private servers and without public access to the stats-pages of DNAS.

My station is public but the XML page with the recent songs list is not (perhaps that is what you mean?).

[neralex]

If you have a public access to your xml-stats, everyone can grab it. The most directories are using this way and its ok, when it was set to public by the station itself.

But what I meant is, when all these stats are hidden and a then you noticed client-connections from IPs, which are hosted by pure server/webspace providers and the connection-time is longer than 12 hours without a disconnect, then I guess that is a bot.

For example Hetzner (DE) is a pure server/webspace-provider (one of the best, loyally!) but this is not a ISP, so all client-connections from this network are server-based. That means someone is using a dedicated server as proxy but in this case the connection will be closed normally after some hours. Or someone is running a 'bot' on a dedicated box.

In my case it were around 15 different IPs with the user-agent 'Winamp 5.50' from a network of a server-provider, which were connected more than 20 hours. I blocked each IP in my firewall and some days later the next IP from the same network does the same. So I started a trace-route-check and all my collected IPs are host-servers of a german company, which are selling mobile-apps and desktop-tools for listening and ripping radio streams. On their website they have a web based directory to listen to the streams and voila I found my private stations with a listing of the last played tracks. My info- and stats-pages are hidden/protected, my SHOUTcast servers are running in private-mode and I'm not listed in other directories like tunein & co. So the only way how they can grab a list of my played tracks is to listen to the streams. This is a very cheeky way and for sure financially driven.

[sqgl]

Originally Posted by neralex

For example Hetzner (DE) is a pure server/webspace-provider (one of the best, loyally!) but this is not a ISP, so all client-connections from this network are server-based.

Well spotted.

Originally Posted by neralex

That means someone is using a dedicated server as proxy but in this case the connection will be closed normally after some hours.

Goes on for days here.

Originally Posted by neralex

Or someone is running a 'bot' on a dedicated box.

How is this different from "using a dedicated server as proxy"?

Originally Posted by neralex

In my case it were around 15 different IPs with the user-agent 'Winamp 5.50' from a network of a server-provider, which were connected more than 20 hours. I blocked each IP in my firewall and some days later the next IP from the same network does the same.

I had a similar problem from an Amazon address. The range of IP's was huge (thousands of addresses which had to be explicitly added to the ban list since a wildcard in the final octet was not enough). I wrote to Amazon but they did not respond.

Originally Posted by neralex

So I started a trace-route-check and all my collected IPs are host-servers of a german company, which are selling mobile-apps and desktop-tools for listening and ripping radio streams. On their website they have a web based directory to listen to the streams and voila I found my private stations with a listing of the last played tracks. My info- and stats-pages are hidden/protected, my SHOUTcast servers are running in private-mode and I'm not listed in other directories like tunein & co. So the only way how they can grab a list of my played tracks is to listen to the streams. This is a very cheeky way and for sure financially driven.

Although I am listed at TuneIn and iTunes I was inspired by you and have now found that the Linode server connections to my station (both 128kbps and 96kbps) are listed at onlineradiobox.com and the 128kbps one even has the wrong station name (my station is RePlayScape in Australia but they called it LaReplay in Colombia instead, and stole an image from an LA arts project with the same name).

In each case, if I tune in via the rogue web site I show up as a listener on my own admin page so all they are doing is hoping to sell ads by getting listeners to tune in via their web site. I don't really care since I can spare the bandwidth, I play a station ID regularly and I don't carry ads. They may even find listeners for me.

None of this explains the null agent listeners unfortunately (the subject of the thread).

[neralex]

Originally Posted by sqgl

None of this explains the null agent listeners unfortunately (the subject of the thread).

Maybe not for you but maybe for all they know how a user-agent works.

Originally Posted by djSpinnerCee

the ban list can't handle this issue -- there are several ways that the user-agent will be blank, it could be NULL, one or more blank spaces, or the client may not include a user-agent header -- I don't think the dnas has a way to handle those special cases.

Originally Posted by neralex

There are a lot of apps/tools on different OS, which are providing incorrect user-agents

[sqgl]

Originally Posted by neralex

Maybe not for you but maybe for all they know how a user-agent works.

Qué?

Originally Posted by sqgl

the Linode server connections to my station (both 128kbps and 96kbps) are listed at onlineradiobox.com

The same Linode server IP address also is responsible for listing my tracks and carrying a 96kbps play button at www.radioways.de and I now notice that if I ban their proxy IP from listening, the playlist indeed stops updating... unless someone/I is/am tuned in via their web site in which case it gets the tracklist from that listener/me even if the listener/I hit the stop button (it just pretends to disconnect but continues monitoring for song titles). Clever!

Am yet to discover which site Zen is feeding my 64kbps stream but I guess it is unimportant, your theory looks verified enough.

[neralex]

Thanks for pointing radioways, I found my stations also on this website. But it seems I blocked their proxy because in the last weeks without to know from where it comes. I found your station also on the website which I meant: http://live.audials.com - I will send you a PM with some IPs from this 'service', which I collected in the last weeks.

[sqgl]

For anyone else reading this thread, the PM indicates we have exactly the same four zombie IP's connecting for long periods of time. I had two additional zombie IPs. Anyone else wanting to compare IP's can PM me.

[gulbis37]

I got the same issues,blocked IP,blocked subnet but Empty user agent pop up again with different IPs