Old 20th April 2011, 21:03   #1
deviantlinux
Junior Member
 
Join Date: Apr 2011
Posts: 3
Shoutcast with Awstats

I wanted to share my experience as I searched Google a lot and couldn't find much help on getting Awstats to parse the w3c shoutcast log (for Shoutcast server v1.9.8).

I ended up writing my own Perl script to add the needed GET text for AwStats to parse the file. I've included it here. Simply run the perl script on your ShoutCast files before you run the awstats script. It will generate a second w3c file with the "_x" extension so awstats can use that and parse it. This is NOT a full howto on how to setup awstats; I assume you've used awstats before.

Perl Script (had to call it php code here on the forum for it to display correctly):

Quote:
#!/usr/bin/env perl
#
# Parse ShoutCast v1.9.8 W3C log and append "GET" to each log line to pretend it's a web logfile
#
# Usage: perl sc_parse.pl -c /full/path/to/shoutcast/sc_w3c.log
#
# Written by Ryan Gehrig
#
use Getopt::Std;
our $opt_c;
getopts('c:');

# Open the log file
if (-e $opt_c)
{
print "Parsing log '$opt_c' ...\n";
open FILE, "$opt_c" or die "ERROR: Failed to open log file\n";
my @lines = <FILE>;

foreach(@lines)
{
# Ignore comments
if("$_" !~ /^\#/)
{
# Lose newlines
$_ =~ s/\n//;

# Write this line to new log file "sc_w3c.log_x"
open (MYFILE, '>>'.$opt_c.'_x');
print MYFILE "$_" . ' GET' . "\n";
close (MYFILE);
}
}
close(FILE);
}
else
{
print "ERROR: The specified log file doesnt exist. Exiting.\n";
}
Save it wherever as sc_parse.pl.


Awstats Config File (awstats.conf)

The important Awstats lines are as follows:

Quote:
LogFile="/full/path/to/awstats/tools/logresolvemerge.pl /full/path/to/shoutcast/sc_w3c.lo* |"
Quote:
LogType=S
Quote:
LogFormat=c-ip c-dns date time cs-uri-stem c-status %other sc-bytes x-duration avgbandwidth %method
Quote:
LogSeparator=" "

Cron Job (to update your stats every night at midnight)

In your cron job that you run every night, first run the perl script, then a few mins later run your normal cron for the awstats. For example:

Quote:
# crontab -e
Quote:
0 0 * * * /full/path/to/sc_parse.pl -c /full/path/to/shoutcast/sc_w3c.log
5 0 * * * /full/path/to/awstats/awstats.pl -update -config=myconfig.com > /dev/null 2>&1

This would run my perl script first, then update your awstats records. I hope this helps people, and of course feel free to modify the script for whatever you like. Input is appreciated.

Ryan
deviantlinux is offline   Reply With Quote
Old 18th May 2011, 11:50   #2
sondo
Junior Member
 
Join Date: May 2011
Posts: 1
I tried using your perl script I managed to parsed the log but somehow when I run the awstats I found corrupted records, any idea to fix this?
sondo is offline   Reply With Quote
Old 17th January 2012, 23:43   #3
deviantlinux
Junior Member
 
Join Date: Apr 2011
Posts: 3
For some reason I can't edit my own post...there is a bug.

On line 30, where is says:

Quote:
# Lose newlines
$_ =~ s/n//;
It should be a newline, not just an n:

Quote:
# Lose newlines
$_ =~ s/\n//;
deviantlinux is offline   Reply With Quote
Old 18th January 2012, 12:14   #4
DrO
Winamp & SHOUTcast Team
 
DrO's Avatar
 
Join Date: Sep 2003
Posts: 26,563
have made sure it's showing the correct thing (hence changing things from php or code to quote tags (not as pretty but it shows the correct things now).

as for not being able to edit, there's a 180 minute timeout which was met ~9months ago

-daz

If you have issues with Winamp or still want to get it, ensure
you get v5.666 build 3516 and the required plug-in updates
DrO is offline   Reply With Quote
Old 1st March 2012, 15:22   #5
iccbroadcast
Junior Member
 
Join Date: Jan 2012
Posts: 31
Hi forum...

Actually I'm struggling with awstats to get useful stats with quite succes thanks to this post.
Now I'm trying to go even further and let awstats to make statistics on the player used by listeners.

to do so , we should reply %other value in the log format string described above with %extra1 (or %extra2. and so...), so those strings on the logs can be used in the extra section of awstat config file.
Thoset fields in shotcast w3c_log correspond precissely to the player, although in a really ugly manner...
The trick is then, to create a new table, defined in the EXTRA SECTIONS (at the end of the config file) to something like:
ExtraSectionName1="Top used players"
ExtraSectionCodeFilter1="200 304"
ExtraSectionCondition1=""
ExtraSectionFirstColumnTitle1="player"
ExtraSectionFirstColumnValues1="extra1,([^&]+)"
ExtraSectionFirstColumnFormat1="%s"
ExtraSectionStatTypes1=HL
ExtraSectionAddAverageRow1=0
ExtraSectionAddSumRow1=1
MaxNbOfExtra1=20
MinHitExtra1=1

Actually te table is created and no error is reported by awstats, but the strings (ugly or not) do not appear...

So. I thing that, could we modify the sc_parse.pl so it replace those long, ugly strings to something more useable and troubleless?
for instance: we could set the script to replace any string starting with iTunes.... to simply iTunes, or to replace any string containing winamp by just winamp, and so with a reasonable list...

Since Im not a programmer, I wouldnt know how to do it at first glance, but giving an exaple on how to do it I could do the bulk task of identifying the strings and add them to de code
iccbroadcast is offline   Reply With Quote
Reply
Go Back   Winamp Forums > SHOUTcast > SHOUTcast Discussions

Tags
awstats, parse stats, shoutcast, stats, v1.9.8

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump