PDA

View Full Version : Shoutcast with Awstats


deviantlinux
20th April 2011, 21:03
I wanted to share my experience as I searched Google a lot and couldn't find much help on getting Awstats to parse the w3c shoutcast log (for Shoutcast server v1.9.8).

I ended up writing my own Perl script to add the needed GET text for AwStats to parse the file. I've included it here. Simply run the perl script on your ShoutCast files before you run the awstats script. It will generate a second w3c file with the "_x" extension so awstats can use that and parse it. This is NOT a full howto on how to setup awstats; I assume you've used awstats before.

Perl Script (had to call it php code here on the forum for it to display correctly):

#!/usr/bin/env perl
#
# Parse ShoutCast v1.9.8 W3C log and append "GET" to each log line to pretend it's a web logfile
#
# Usage: perl sc_parse.pl -c /full/path/to/shoutcast/sc_w3c.log
#
# Written by Ryan Gehrig
#
use Getopt::Std;
our $opt_c;
getopts('c:');

# Open the log file
if (-e $opt_c)
{
print "Parsing log '$opt_c' ...\n";
open FILE, "$opt_c" or die "ERROR: Failed to open log file\n";
my @lines = <FILE>;

foreach(@lines)
{
# Ignore comments
if("$_" !~ /^\#/)
{
# Lose newlines
$_ =~ s/\n//;

# Write this line to new log file "sc_w3c.log_x"
open (MYFILE, '>>'.$opt_c.'_x');
print MYFILE "$_" . ' GET' . "\n";
close (MYFILE);
}
}
close(FILE);
}
else
{
print "ERROR: The specified log file doesnt exist. Exiting.\n";
}


Save it wherever as sc_parse.pl.


Awstats Config File (awstats.conf)

The important Awstats lines are as follows:

LogFile="/full/path/to/awstats/tools/logresolvemerge.pl /full/path/to/shoutcast/sc_w3c.lo* |"
LogType=S
LogFormat=c-ip c-dns date time cs-uri-stem c-status %other sc-bytes x-duration avgbandwidth %method
LogSeparator=" "


Cron Job (to update your stats every night at midnight)

In your cron job that you run every night, first run the perl script, then a few mins later run your normal cron for the awstats. For example:

# crontab -e

0 0 * * * /full/path/to/sc_parse.pl -c /full/path/to/shoutcast/sc_w3c.log
5 0 * * * /full/path/to/awstats/awstats.pl -update -config=myconfig.com > /dev/null 2>&1


This would run my perl script first, then update your awstats records. I hope this helps people, and of course feel free to modify the script for whatever you like. Input is appreciated.

Ryan

sondo
18th May 2011, 11:50
I tried using your perl script I managed to parsed the log but somehow when I run the awstats I found corrupted records, any idea to fix this?

deviantlinux
17th January 2012, 23:43
For some reason I can't edit my own post...there is a bug.

On line 30, where is says:

# Lose newlines
$_ =~ s/n//;

It should be a newline, not just an n:

# Lose newlines
$_ =~ s/\n//;

DrO
18th January 2012, 12:14
have made sure it's showing the correct thing (hence changing things from php or code to quote tags (not as pretty but it shows the correct things now).

as for not being able to edit, there's a 180 minute timeout which was met ~9months ago ;)

-daz

iccbroadcast
1st March 2012, 15:22
Hi forum...

Actually I'm struggling with awstats to get useful stats with quite succes thanks to this post.
Now I'm trying to go even further and let awstats to make statistics on the player used by listeners.

to do so , we should reply %other value in the log format string described above with %extra1 (or %extra2. and so...), so those strings on the logs can be used in the extra section of awstat config file.
Thoset fields in shotcast w3c_log correspond precissely to the player, although in a really ugly manner...
The trick is then, to create a new table, defined in the EXTRA SECTIONS (at the end of the config file) to something like:
ExtraSectionName1="Top used players"
ExtraSectionCodeFilter1="200 304"
ExtraSectionCondition1=""
ExtraSectionFirstColumnTitle1="player"
ExtraSectionFirstColumnValues1="extra1,([^&]+)"
ExtraSectionFirstColumnFormat1="%s"
ExtraSectionStatTypes1=HL
ExtraSectionAddAverageRow1=0
ExtraSectionAddSumRow1=1
MaxNbOfExtra1=20
MinHitExtra1=1

Actually te table is created and no error is reported by awstats, but the strings (ugly or not) do not appear...

So. I thing that, could we modify the sc_parse.pl so it replace those long, ugly strings to something more useable and troubleless?
for instance: we could set the script to replace any string starting with iTunes.... to simply iTunes, or to replace any string containing winamp by just winamp, and so with a reasonable list...

Since Im not a programmer, I wouldnt know how to do it at first glance, but giving an exaple on how to do it I could do the bulk task of identifying the strings and add them to de code