Old 31st March 2016, 05:27   #1
pjrhapsody
Junior Member
 
Join Date: Jan 2016
Posts: 7
User Agent Empty

Hi

I have read up across a number of posts about the stream rippers from Germany etc and it has been mentioned about the empty user agent. but it's not clear if there is an option to ban all those who don't supply a user agent.

My question is, is this possible to ban an empty user agent and if so what do we enter into the field ?
pjrhapsody is offline   Reply With Quote
Old 1st April 2016, 00:13   #2
djSpinnerCee
Forum King
 
djSpinnerCee's Avatar
 
Join Date: Aug 2004
Location: Hollis, Queens/The Bronx, NYC
Posts: 3,399
Send a message via ICQ to djSpinnerCee Send a message via AIM to djSpinnerCee
the ban list can't handle this issue -- there are several ways that the user-agent will be blank, it could be NULL, one or more blank spaces, or the client may not include a user-agent header -- I don't think the dnas has a way to handle those special cases.
djSpinnerCee is online now   Reply With Quote
Old 1st April 2016, 07:15   #3
pjrhapsody
Junior Member
 
Join Date: Jan 2016
Posts: 7
Thanks Spinner

I just thought that those wanting to steal your stream etc, if they don't identify themselves then it would be good to say, well you cant access the stream.
pjrhapsody is offline   Reply With Quote
Old 10th April 2016, 04:19   #4
dopelabs
Senior Member
 
dopelabs's Avatar
 
Join Date: Oct 2006
Location: Silicon Valley
Posts: 470
Send a message via AIM to dopelabs
unless you are password protecting tune in access to your streams, you are providing anything you broadcast as free stuff to the world. if anyone can tune in, anyone can record the audio. plain and simple. cant really call it 'stealing' when your giving it away freely =].
dopelabs is offline   Reply With Quote
Old 29th July 2016, 00:13   #5
mattauckland
Junior Member
 
Join Date: Mar 2015
Posts: 39
I've actually just had a conversation with DrO on Twitter regarding this issue, as I'm experiencing the same issue, and he pointed out that you can block empty UA strings, using the following configuration option:

"Added 'blockemptyuseragent' configuration option to allow for preventing client connections without a user agent from connecting (note: some valid clients e.g. some hardware devices may not provide a user agent and enabling this may incorrectly block legitimate client connections)" ~ From the 2.2.2 changelog.

Just though that might be of interest.
mattauckland is offline   Reply With Quote
Old 29th July 2016, 12:53   #6
pjrhapsody
Junior Member
 
Join Date: Jan 2016
Posts: 7
User agent Empty

Thanks for the heads up

Since i made my original post i have found that a web site was able to capture my stream information even though we hadn't published it anywhere, couldn't understand why we were getting logons from around the world.

Turns out that in the winamp dsp (Directory Tab)even with the box unticked information in the fields would be picked up by this web site directory of internet stations.

The web site would then show it as live the moment we started broadcasting. I removed all the old information and hey presto no more world wide logins.

For us this is important as we would only like people in the uK to benefit and listen through our web site app.
pjrhapsody is offline   Reply With Quote
Old 22nd April 2017, 16:44   #7
sqgl
Senior Member
 
sqgl's Avatar
 
Join Date: Sep 2008
Location: Australia
Posts: 128
DNAS v2.5.1 has "Block User Agent" button/link on the admin page. Perhaps this works on null agents too.

I also get a disproportionate number of null agents from Germany in particular. And how do we know they are people ripping? They might be bots though I cannot imagine what they gain from tuning in.

Remixing listener requests of ambient tracks live http://RePlayScape.com
sqgl is offline   Reply With Quote
Old 22nd April 2017, 17:28   #8
neralex
Senior Member
 
Join Date: Mar 2011
Posts: 460
** Empty ** user-agents are not always bots or stream-rippers! There are a lot of apps/tools on different OS, which are providing incorrect user-agents but in the most cases they are real listeners. So be careful to block IPs with ** Empty ** user-agents.

Something like 'Winamp 5.50' is known for a popular choice of a faked user-agent. I had a simular issue in the last weeks with a Germany based streaming-directory which is collecting stream-URLs without permissions of the affected stations. They are running different bots with this old 'Winamp 5.50' user-agent to rip the streams in order to get playlists from stations with private servers and without public access to the stats-pages of DNAS. You can identify these bot on the long connection-time and mostly with a short view on the IP trace-route.
neralex is offline   Reply With Quote
Old 22nd April 2017, 19:16   #9
sqgl
Senior Member
 
sqgl's Avatar
 
Join Date: Sep 2008
Location: Australia
Posts: 128
Quote:
Originally Posted by neralex View Post
** Empty ** user-agents are not always bots or stream-rippers!
Thanks for pointing this out.

Quote:
Originally Posted by neralex View Post
Something like 'Winamp 5.50' is known for a popular choice of a faked user-agent. I had a simular issue in the last weeks with a Germany based streaming-directory which is collecting stream-URLs without permissions of the affected stations...You can identify these bot on the long connection-time and mostly with a short view on the IP trace-route.

In my case these WA5.50 semi-permanent connections belong to obscure ISP's:
  • Hetzner (DE)
  • Linode (which is simultaneously in DE and US depending on the lookup used)
  • Zen (UK)
Though it seems inefficient, could it be that they are bots trying to attract the attention of the administrator simply to spread brand awareness? It reminds me of the way that bots will hit web pages so as to show up in the analytics pages - or are they attempts to boost their google rankings by being referenced? (although I suspect Google would not count such listings in their algorithm).

Quote:
Originally Posted by neralex View Post
They are running different bots with this old 'Winamp 5.50' user-agent to rip the streams in order to get playlists from stations with private servers and without public access to the stats-pages of DNAS.
My station is public but the XML page with the recent songs list is not (perhaps that is what you mean?).

Remixing listener requests of ambient tracks live http://RePlayScape.com
sqgl is offline   Reply With Quote
Old 22nd April 2017, 20:03   #10
neralex
Senior Member
 
Join Date: Mar 2011
Posts: 460
If you have a public access to your xml-stats, everyone can grab it. The most directories are using this way and its ok, when it was set to public by the station itself.

But what I meant is, when all these stats are hidden and a then you noticed client-connections from IPs, which are hosted by pure server/webspace providers and the connection-time is longer than 12 hours without a disconnect, then I guess that is a bot.

For example Hetzner (DE) is a pure server/webspace-provider (one of the best, loyally!) but this is not a ISP, so all client-connections from this network are server-based. That means someone is using a dedicated server as proxy but in this case the connection will be closed normally after some hours. Or someone is running a 'bot' on a dedicated box.

In my case it were around 15 different IPs with the user-agent 'Winamp 5.50' from a network of a server-provider, which were connected more than 20 hours. I blocked each IP in my firewall and some days later the next IP from the same network does the same. So I started a trace-route-check and all my collected IPs are host-servers of a german company, which are selling mobile-apps and desktop-tools for listening and ripping radio streams. On their website they have a web based directory to listen to the streams and voila I found my private stations with a listing of the last played tracks. My info- and stats-pages are hidden/protected, my SHOUTcast servers are running in private-mode and I'm not listed in other directories like tunein & co. So the only way how they can grab a list of my played tracks is to listen to the streams. This is a very cheeky way and for sure financially driven.
neralex is offline   Reply With Quote
Old 23rd April 2017, 14:18   #11
sqgl
Senior Member
 
sqgl's Avatar
 
Join Date: Sep 2008
Location: Australia
Posts: 128
Quote:
Originally Posted by neralex View Post
For example Hetzner (DE) is a pure server/webspace-provider (one of the best, loyally!) but this is not a ISP, so all client-connections from this network are server-based.
Well spotted.

Quote:
Originally Posted by neralex View Post
That means someone is using a dedicated server as proxy but in this case the connection will be closed normally after some hours.
Goes on for days here.

Quote:
Originally Posted by neralex View Post
Or someone is running a 'bot' on a dedicated box.
How is this different from "using a dedicated server as proxy"?

Quote:
Originally Posted by neralex View Post
In my case it were around 15 different IPs with the user-agent 'Winamp 5.50' from a network of a server-provider, which were connected more than 20 hours. I blocked each IP in my firewall and some days later the next IP from the same network does the same.
I had a similar problem from an Amazon address. The range of IP's was huge (thousands of addresses which had to be explicitly added to the ban list since a wildcard in the final octet was not enough). I wrote to Amazon but they did not respond.

Quote:
Originally Posted by neralex View Post
So I started a trace-route-check and all my collected IPs are host-servers of a german company, which are selling mobile-apps and desktop-tools for listening and ripping radio streams. On their website they have a web based directory to listen to the streams and voila I found my private stations with a listing of the last played tracks. My info- and stats-pages are hidden/protected, my SHOUTcast servers are running in private-mode and I'm not listed in other directories like tunein & co. So the only way how they can grab a list of my played tracks is to listen to the streams. This is a very cheeky way and for sure financially driven.
Although I am listed at TuneIn and iTunes I was inspired by you and have now found that the Linode server connections to my station (both 128kbps and 96kbps) are listed at onlineradiobox.com and the 128kbps one even has the wrong station name (my station is RePlayScape in Australia but they called it LaReplay in Colombia instead, and stole an image from an LA arts project with the same name).

In each case, if I tune in via the rogue web site I show up as a listener on my own admin page so all they are doing is hoping to sell ads by getting listeners to tune in via their web site. I don't really care since I can spare the bandwidth, I play a station ID regularly and I don't carry ads. They may even find listeners for me.

None of this explains the null agent listeners unfortunately (the subject of the thread).

Remixing listener requests of ambient tracks live http://RePlayScape.com
sqgl is offline   Reply With Quote
Old 24th April 2017, 04:49   #12
neralex
Senior Member
 
Join Date: Mar 2011
Posts: 460
Quote:
Originally Posted by sqgl View Post
None of this explains the null agent listeners unfortunately (the subject of the thread).
Maybe not for you but maybe for all they know how a user-agent works.

Quote:
Originally Posted by djSpinnerCee View Post
the ban list can't handle this issue -- there are several ways that the user-agent will be blank, it could be NULL, one or more blank spaces, or the client may not include a user-agent header -- I don't think the dnas has a way to handle those special cases.
Quote:
Originally Posted by neralex View Post
There are a lot of apps/tools on different OS, which are providing incorrect user-agents
neralex is offline   Reply With Quote
Old 24th April 2017, 16:22   #13
sqgl
Senior Member
 
sqgl's Avatar
 
Join Date: Sep 2008
Location: Australia
Posts: 128
Quote:
Originally Posted by neralex View Post
Maybe not for you but maybe for all they know how a user-agent works.
Qué?

Quote:
Originally Posted by sqgl View Post
the Linode server connections to my station (both 128kbps and 96kbps) are listed at onlineradiobox.com
The same Linode server IP address also is responsible for listing my tracks and carrying a 96kbps play button at www.radioways.de and I now notice that if I ban their proxy IP from listening, the playlist indeed stops updating... unless someone/I is/am tuned in via their web site in which case it gets the tracklist from that listener/me even if the listener/I hit the stop button (it just pretends to disconnect but continues monitoring for song titles). Clever!

Am yet to discover which site Zen is feeding my 64kbps stream but I guess it is unimportant, your theory looks verified enough.

Remixing listener requests of ambient tracks live http://RePlayScape.com

Last edited by sqgl; 24th April 2017 at 17:53.
sqgl is offline   Reply With Quote
Old 24th April 2017, 20:08   #14
neralex
Senior Member
 
Join Date: Mar 2011
Posts: 460
Thanks for pointing radioways, I found my stations also on this website. But it seems I blocked their proxy because in the last weeks without to know from where it comes. I found your station also on the website which I meant: http://live.audials.com - I will send you a PM with some IPs from this 'service', which I collected in the last weeks.
neralex is offline   Reply With Quote
Old 25th April 2017, 18:08   #15
sqgl
Senior Member
 
sqgl's Avatar
 
Join Date: Sep 2008
Location: Australia
Posts: 128
For anyone else reading this thread, the PM indicates we have exactly the same four zombie IP's connecting for long periods of time. I had two additional zombie IPs. Anyone else wanting to compare IP's can PM me.

Remixing listener requests of ambient tracks live http://RePlayScape.com
sqgl is offline   Reply With Quote
Reply
Go Back   Winamp & SHOUTcast Forums > SHOUTcast > SHOUTcast Technical Support

Tags
ban, user agent

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump