Old 15th June 2004, 09:09   #1
EL-VIS2
Member
 
Join Date: Jan 2002
Location: Germany
Posts: 73
Analyzing and Manipulating Music Values

The most essential thing in AVS is to get the values of the music and setup the presets response to the music. Superscope, DynamicMovement, Texer, etc. offer almost unlimited space for coding mostly used for amazing effects but what about the music beyond simple beat and getosc/getspec?

I searched this forum for some sophisticated methods to analyze the music input but it is difficult to find and most of the time hidden in general coding discussions.

So I think we need a special thread on this particular theme collecting techniques for setting up music response.

Want to start with a try to react to the loudness of the music using a floating average of the music values over a certain amount of frames. Goal was to setup a var "speed" for example to be used for a rotation increasing and decreasing with the music loudness.

fdelta = number of frames used for floating average
bufstart = number of the first megabuf to store the values
speedf = factor to adjust the average found to your needs


INIT:
speed=0;fdelta=50;bufstart=50;speedf=1.0;

FRAME:
pt=bufstart-1;
loop(fdelta,exec2(assign(pt,pt+1),assign(megabuf(pt),megabuf(pt+1))));
assign(megabuf(bufstart+fdelta),getspec(0.05,0.9,0));
pt=bufstart;
vsum=0;
loop(fdelta,exec2(assign(pt,pt+1),assign(vsum,vsum+megabuf(pt))));
vsum=vsum/fdelta;
speed=vsum*speedf;
EL-VIS2 is offline   Reply With Quote
Old 15th June 2004, 10:06   #2
Rovastar
Moderator
 
Join Date: Jun 2001
Location: London, England
Posts: 3,632
Send a message via AIM to Rovastar
Interesting stuff using a buffer array to store the 'average'.

If feeling brave enter the world of beat detection and other methods in this thread.

http://forums.winamp.com/showthread....hreadid=102205

A bit MIlkDropy at times but we have a few other devolpers in there too from other plugins.

"Rules are for the guidance of wisemen and the obedience of fools"

Visuals - Morphyre www.Morphyre.com
Rovastar is offline   Reply With Quote
Old 15th June 2004, 10:20   #3
Tuggummi
Bin King
 
Tuggummi's Avatar
 
Join Date: Mar 2001
Location: Finland
Posts: 2,173
Well i mostly used this very simple method in Extra Dimension, which has it's flaws, but seems to work ok most of the time.

code:

per frame
now=gettime(0) ; sec=now-last ; fps=1/sec ; last=now ;
vol=vol*0.95+getspec(0.5,1,0) ;
speed=(min(3,vol)/fps)*0.1 ;
ts=ts*(1-sec) ; t=t+ts

per beat
ts=(rand(51)-25)*speed



Well first there is the whole speed fixed on fps line,
then it tries to figure out the current volume. The use of min function there is to prevent some sudden sound peaks that might make the dm/ssc/texer II/etc. spin extremely fast, i have tried to figure out the most common values & top values of the volume with the help of the debugging window and noticed that with this method the value of "vol" hardly goes beyond 3.

Well anyways, i know this is probably not what you mented or wanted, but this is currently working for me quite nicely.

I would gladly hear also what other people use for sound reaction in their presets

Texer Resources

Im retarded... err i mean retired!
Probably both...
Tuggummi is offline   Reply With Quote
Old 15th June 2004, 10:55   #4
EL-VIS2
Member
 
Join Date: Jan 2002
Location: Germany
Posts: 73
Tried your stuff tuggummi and it's working allright but the response is a little bit jumpy. The advantage of using a floating average (do not know if this is the correct english term for German "gleitender Durchschnitt") is that we get a flowing increase and decrease of the speed and we do not have to mind about short jumps in the volume. Also it is more "exact" since "vsum" will never become bigger than 1.

But I like the idea to have an eye on the fps. I think this could/should be implemented in my approach.
EL-VIS2 is offline   Reply With Quote
Old 15th June 2004, 14:49   #5
EL-VIS2
Member
 
Join Date: Jan 2002
Location: Germany
Posts: 73
Ok, here is the code for the floating average based on time/fps.

timebase = time in seconds for building the average
bufstart = first megabuffer to store the values
vsum = the average

The attached example shows 3 Graphs:
1. red = original values
2. yellow = values on a timebase of 0.2 seconds
3. green = values on a timebase of 2.0 seconds

What is interesting here is that Graph 2 and 3 nicely show the beats. Perhaps this could be used to build a custom beat detection.


CODE___________________________________________

INIT:
timebase=2.0;bufstart=50;

FRAME:
now=gettime(0);getit=above(now,timeout);
fps=if(getit,fct,fps);reg10=fps;
fct=if(getit,0,fct+1);timeout=if(getit,now+1.0,timeout);
fdelta=timebase*(fps+1);
pt=bufstart-1;
loop(fdelta,exec2(assign(pt,pt+1),assign(megabuf(pt),megabuf(pt+1))));
assign(megabuf(bufstart+fdelta),getspec(0.05,0.9,0));
pt=bufstart;
vsum=0;
loop(fdelta,exec2(assign(pt,pt+1),assign(vsum,vsum+megabuf(pt))));
vsum=vsum/fdelta;
speed=vsum*speedf;
Attached Files
File Type: zip el-vis_analyzer.zip (611 Bytes, 264 views)
EL-VIS2 is offline   Reply With Quote
Old 15th June 2004, 18:26   #6
UnConeD
Whacked Moderator
 
UnConeD's Avatar
 
Join Date: Jun 2001
Posts: 2,104
Actually you can keep an average value much more efficiently . Every frame, just add the new value to an accumulator, and subtract the oldest value from the accumulator.

UnConeD is offline   Reply With Quote
Old 16th June 2004, 08:03   #7
EL-VIS2
Member
 
Join Date: Jan 2002
Location: Germany
Posts: 73
Quote:
Originally posted by UnConeD
Actually you can keep an average value much more efficiently . Every frame, just add the new value to an accumulator, and subtract the oldest value from the accumulator.

I thought this is exactly what I am doing here.Is there a better way to manage the values?
EL-VIS2 is offline   Reply With Quote
Old 16th June 2004, 09:55   #8
[Ishan]
Major Dude
 
[Ishan]'s Avatar
 
Join Date: Mar 2003
Location: India
Posts: 1,156
Send a message via Yahoo to [Ishan]
i use the same method here, altho i dont think there is a better way....maybe....


Amateur AVS'er. All over again.

Last AVS Pack :
Resurgence

[Ishan] is offline   Reply With Quote
Old 17th June 2004, 12:12   #9
UnConeD
Whacked Moderator
 
UnConeD's Avatar
 
Join Date: Jun 2001
Posts: 2,104
"I thought this is exactly what I am doing here.Is there a better way to manage the values?"

As far as I can see, you're adding them together every frame, and even shifting them around all the time, rather than just overwriting the oldest value with the newest.

UnConeD is offline   Reply With Quote
Old 18th June 2004, 05:55   #10
EL-VIS2
Member
 
Join Date: Jan 2002
Location: Germany
Posts: 73
Quote:
Originally posted by UnConeD
As far as I can see, you're adding them together every frame, and even shifting them around all the time, rather than just overwriting the oldest value with the newest.

Ok, I agree the sum could be done without a loop like this:


vsum1=vsum1-megabuf(bufstart)+ megabuf(bufstart+fdelta);
vsum=vsum1/fdelta;

but when testing this I dicovered some little jumps in the average. I think because of the changing fdelta since this is related to timebase > fps which changes a little over time. So I have to average the fps too.

Still can't figure out how to dynamically keep and retrieve the oldest value without this loop
EL-VIS2 is offline   Reply With Quote
Old 21st June 2004, 15:03   #11
Synth-C
Junior Member
 
Synth-C's Avatar
 
Join Date: Aug 2003
Location: The Netherlands
Posts: 47
how about this method:

val=100;

fc1=(fc1+1)%val;fc2=val-fc1;
assign( megabuf(fc1), getspec(.1,.1,0));
tot=tot+megabuf(fc1)-megabuf(fc2);
avg=tot/val;


example preset:
Attached Files
File Type: zip synth-c -sound.zip (1.8 KB, 233 views)
Synth-C is offline   Reply With Quote
Old 21st June 2004, 22:11   #12
EL-VIS2
Member
 
Join Date: Jan 2002
Location: Germany
Posts: 73
Yeah thats very nice, synth, but there is a small bug. I do not think the results are 100% correct since you do not always subtract the oldest value. fc1 and fc2 are "getting close" ah... do not how to say this in english... like this example:

fc1: 48, 49, 50, 51, 52 ...
fc2: 52, 51, 50, 49, 48 ...

but I think this will fix the thing

fc1=(fc1+1)%val;fc2=(fc1+1)%val;
EL-VIS2 is offline   Reply With Quote
Old 22nd June 2004, 07:53   #13
EL-VIS2
Member
 
Join Date: Jan 2002
Location: Germany
Posts: 73
My first own beat detection

Thanx to synth-c for this great setup to build the average. Without this little bug it is working great.

Just added my fps detection to make the average run over time.

Rovastars hint on this beat discussion made me start a desperate try to build up a my own beat detection. Attached you'll find the result. Not too bad for a first try but of course still some things unsolved. The setup does no beat prediction at all since this is very dangerous ground. If prediction fails - a fact in all songs with changing bpm or difficult beats not detected by the setup - you'll get beats at unwanted points. So I think better rely on the pure input.

Difficult beats for this setup are for example a soft base drum behind a loud guitar solo. Don't know how to filter these out. Good prediction would help here but again: if it fails I think it is better to get no beat at all...

The graphs in the attached example show:

green => the average over time
yellow => the original input value
red => mybeat
white => beat found by AVS


The things behind this beat detection:

0. Getting the input value
1. Building a running average of the input value over a given amount of time
2. Comparing the input value with the average and stating a beat if certain conditions are true


Conditions/Assumptions:

- The input has to be a certain amount over the average to indicate a beat

- The next beat is only possible if the input has returned to/below the average

- The amount of the input to be over the average is intially set but must be dynamic too to suppress too fast beats and allow beats if no beats are found over a certain time

Dynamic adjustment of the amount of the input to be over the average

- To prevent too fast beats after a beat the next beat is suppressed by a linear decreasing additional dumping value over a short amount of time


- To get beats if no beats are found after a certain time the intial dumping of the input is lowered on the increasing time after the last beat until a beat is found.
Attached Files
File Type: zip el-vis_mybeat02.zip (1.2 KB, 254 views)

Last edited by EL-VIS2; 22nd June 2004 at 09:33.
EL-VIS2 is offline   Reply With Quote
Old 24th June 2004, 09:17   #14
EL-VIS2
Member
 
Join Date: Jan 2002
Location: Germany
Posts: 73
Here is a new version of my beat detection:

- Used different input "mval" to better detect bass beats

- v_dump adjusts dynamically on volume
Attached Files
File Type: zip el-vis_mybeat03.zip (1.3 KB, 236 views)
EL-VIS2 is offline   Reply With Quote
Old 24th June 2004, 12:10   #15
TomyLobo
Major Dude
 
TomyLobo's Avatar
 
Join Date: Feb 2004
Location: Germany
Posts: 580
hmm this has some problems with the trance music i'm listening to...
doesnt detect some of the beats

and in one song it doesnt detect beats at all (Usual Aspect - Mr Blue(Progressive Mix) that is)
maybe you should only watch the lower bands for beats...

If you can't say something nice, say something surrealistic.
TomyLobo is offline   Reply With Quote
Old 25th June 2004, 10:20   #16
EL-VIS2
Member
 
Join Date: Jan 2002
Location: Germany
Posts: 73
Quote:
Originally posted by TomyLobo
maybe you should only watch the lower bands for beats...
Yeah I know it is not perfect yet. Works well with a lot of songs but these lower beats are difficult. In mybeat03 I tried to concentrate on the lower bands but still not 100% OK. Perhaps check out lowering v_damp to almost zero.

A lot depends on the bands used for input. So I will do more experimenting with "mval".

What I wanted to get is a less hyperactive beat detection than the one AVS offers since in visualizations too fast beats sometimes ruin the effects. And skipping beats using custombeat setup will fail to get some of the essential beats in a song.
EL-VIS2 is offline   Reply With Quote
Old 25th June 2004, 16:40   #17
TomyLobo
Major Dude
 
TomyLobo's Avatar
 
Join Date: Feb 2004
Location: Germany
Posts: 580
one could skip fast beats, like colormap does
that is beats that are less than 0.1 sec from each other or something

If you can't say something nice, say something surrealistic.
TomyLobo is offline   Reply With Quote
Reply
Go Back   Winamp & Shoutcast Forums > Visualizations > AVS

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump