Old 17th November 2014, 19:56   #1
Nitorami
Major Dude
 
Join Date: Mar 2008
Location: Erlangen
Posts: 854
Perfect beat detection

Well, not quite, but almost.

Beat detection has been discussed quite a lot over the past 12 years; a thread search will give you lots of reading material, and there is even scientific research on it outside milkdrop.

Still, milkdrop presets, including my own ones, use a very simple form of beat reaction. Various schemes have been proposed by Rovastar, Krash, Geiss, and others. They all compare the actual volume to some sort of trigger level; if the volume exceeds the level, this is a beat. The difference in algorithms is just how this trigger level is determined. I spend a lot of time trying to optimise this, and in fact, after a few beers and repeating the same piece of music I was convinced my algorithm was the best ever devised; but the next day and with different music, it did not seem to perform that well at all.

Before the introduction of the ns-eel language extension some three years ago, no more was possible anyway: 30 variables, no loops, no address space. Although lacking the comfort of a structured high-level language, ns-eel is a huge improvement over the restrictions of the "old" milkdrop; unfortunately introduced at a time when most ambitioned programmers had already turned their backs on milkdrop.

Anyway, back to the topic: Proper beat detection cannot possibly be based on comparing an instantaneous value to some sort of smoothed value. This is not what you do on the dance floor - wait for a volume increase and then jump. Except on van Halen possibly.

Nevertheless, this sort of reaction works perfectly for most presets. More, normally we do not even want the "real" beat which is regular, and boring. An occasional reaction, though somewhat erratic, is ok, and certainly MUCH better than no reaction. If you happened to watch milkdrop without beat reaction at all (I remember a bug of that sort) you'll quickly feel a disappointment of sorts that something is wrong and does not fit.

Still, there has been a quest for the "perfect" beat detection even though it is questionable to what sort of presets it could be applied. Imagine presets that modulate the whole screen by altering dx, dy, zoom, rot - would you want this to happen regularly, predictably, in a fixed pattern exactly synchronised to the music ? I guess it would drive you nuts. On the other hand, the erratic volume triggered reaction we had so far can equally be annoying, when it keeps firing where it should not, or when it triggers off-beat. And in a club, you want the visuals to move strictly with the beat; I have learned this when I made simple line/dots presets to be shown on a projector: MUCH better when strict time is followed.

Check out the attached demo preset, which is still heavily under construction. It does not show you the beat, just a pre-stage, from which a beat trigger will be derived. The top curve (yellow) is the usual milkdrop volume signal based on bass, mid and treb, which used to be the base for all beat detection schemes so far.
The bottom green curve is my hilariously processed signal. After a short time, you will see a pattern in the green curve which is not normally evident in the yellow one, and cannot be derived from it by any sort of low pass filtering or similar. In many cases it is even possible to spot the rhythm, i.e. whether it is 2/4 or 4/4 (have not tried a 3/4 waltz yet). The reaction is not always 100% promising but close, at least for the sort of music I like; and I bet it will normally identify the beat before you have even noted there is one ;-) Problem remaining is to tell whether we have 70 or 140 beats per minute (BPM), but for a preset that should hardly matter. It can also tell whether it is good or not, and hold off if it feels that the quality of detection is poor. The demo does not do that.

Martin
Attached Files
File Type: milk martin - harmonic beat detection demo.milk (17.6 KB, 977 views)
Nitorami is offline   Reply With Quote
Old 17th November 2014, 22:24   #2
Victhor
Late skinner & Moderator
 
Victhor's Avatar
 
Join Date: May 2003
Location: Argentina
Posts: 1,606
martin like.. "martin - reflection on black tiles"??

PS: Can't get the preset to work..it just freezes on black screen (with a caption of the list and the previous preset if there was..), any advice on what to do?
Victhor is offline   Reply With Quote
Old 18th November 2014, 08:03   #3
Nitorami
Major Dude
 
Join Date: Mar 2008
Location: Erlangen
Posts: 854
Yes. The black tiles was one of my first presets. Rave from the grave.

And yes, you have an ATI video card, which does not work with certain shader versions. But it does not matter, we do not need this version. I used it once for a preset, and since then it lingers in a few presets and keeps reappearing whenever I copy from that. I should scrutinise my directory and delete all versions of PS4 to get rid of this fault.
Anyway, in the preset simply set all PS versions to 3 like this

PSVERSION=2
PSVERSION_WARP=2
PSVERSION_COMP=2

or just use the attached.
Attached Files
File Type: milk martin - harmonic beat detection demo.milk (17.6 KB, 505 views)
Nitorami is offline   Reply With Quote
Old 18th November 2014, 11:13   #4
Victhor
Late skinner & Moderator
 
Victhor's Avatar
 
Join Date: May 2003
Location: Argentina
Posts: 1,606
Now it works, thanks for that. Though most of what you said is chinese for me, I do notice the difference, helped with a schematic song I could easily tell the difference where the green "looked for" a pattern and stick to it (changing slightly from time to time or slowly "building" it again) while the yellow was more like "looking what it is" constantly and not really achiving a pattern or texture.
For me it seemed like the yellow was more accurate somehow, but I guess it's more useful to beat detection what you achieve with the green.

When I get back home I'll try with a not-so-schematic song..
Victhor is offline   Reply With Quote
Old 18th November 2014, 17:31   #5
Nitorami
Major Dude
 
Join Date: Mar 2008
Location: Erlangen
Posts: 854
I try to put this in other words.

The classic beat detection is event-triggered. Whenever there is a significant increase of volume, AND an adequate time has passed since the last beat -> this is a "beat". The preset will react to this somehow, e.g. by changing the rotation, zoom, position of patterns etc.

This feels "accurate" because it coincides exactly with simple features of the music you hear: the snare drum comes in and the preset reacts instantly, "on beat". But this is not necessarily the musical beat, and often it will not be regular. You see many peaks in the yellow curve, and it is not possible to decide which peak is just an odd noise, and which establishes the rhythm; in general, extra beats will be generated, or beats will be missing. The only criterion is the amplitude - if any noise is loud enough, this is recognised as a beat. For some songs, this works fairly well, but it fails utterly on others. But as said, failure is not necessarily a killer because the detected "beat" still coincides with a volume change, so you still feel some sort of correlation.

The harmonic detection tries a different approach. It does not bother about what happens in this moment but tries to analyse the underlying rhythm by observing several seconds of music, and synchronising on it. Once synchronised, it will provide a regular beat signal exactly in rhythm with the music, i.e. the beat repetition rate is correct and there are no extra or missing beats as with the old method. But, the beats may sometimes seem to have a time lag to what you feel as rhythm and this may feel "inaccurate": In fact this depends on the music; if the snare comes interlaced with the bass drum, then when exactly does the beat start ? This is psychoacoustics; if the algorithm interprets this different from you, you'll feel the beat lagging or ahead, BUT - it is still in rhythm. It is more like dancing; standing on the dancefloor, listening for a few moments, then start swinging.

It will take me a few days but I'll make a few sample presets to show you the difference. It can be quite amazing, but it needs presets fitting it.
Nitorami is offline   Reply With Quote
Old 18th November 2014, 17:45   #6
Sabine Klare
Forum Queen
 
Sabine Klare's Avatar
 
Join Date: Nov 2010
Location: Berlin, Germany
Posts: 2,492
Just downloaded the fixed version. I have analyzed both versions with Notepad++, the fixed version looks much better in the codes.

If scrutinising Your directory, also both orbit trap playground presets are in PS4, just checked the codes. They never worked on my PC & notebook, although they should. They were loaded, but nothing had happened, they simply stood still. Both versions use manyfish.jpg, which I have already since the beginning.

I am very curious for the few sample presets..

Sabine Klare Aka Sternenmaschinebine
Music, Art, Lyrics, Videos
AMBIENT... AMBIENT music forever...
Sabine Klare is offline   Reply With Quote
Old 19th November 2014, 15:50   #7
Nitorami
Major Dude
 
Join Date: Mar 2008
Location: Erlangen
Posts: 854
Sabine

The orbit trap does not work at a lower shader version. It generates (almost) still images of a modified mandelbrot set, and requires high iteration depth for the details. The code requires too many instructions to run at PS2 or 3. Sorry. The images are beautiful and you might like them for the sort of artwork you make, but it does not work on ATI cards. Attached a sample of the sort of pictures it generates :-)

BTW, manyfish.jpg is not really necessary, this is a code relict I forgot to delete.

Do not expect too much forthcoming here, all I currently plan is a simple presets to show the principle. I would hope that others tie the code into presets of their own but my hopes are suitably low. If anyone should consider a new version of milkdrop - the name Litheon springs to mind - I would love to assist integrating this sort of beat detection into the dll. This would be much better than in the presets. Well. Fat chance.

Martin
Attached Thumbnails
Click image for larger version

Name:	orbit trap sample.jpg
Views:	909
Size:	89.7 KB
ID:	51530  
Nitorami is offline   Reply With Quote
Old 19th November 2014, 16:44   #8
Sabine Klare
Forum Queen
 
Sabine Klare's Avatar
 
Join Date: Nov 2010
Location: Berlin, Germany
Posts: 2,492
Strange, that my PC has NVIDIA GeForce GT 230. I had the presets in Winamp on my previous Windows Vista Install, which was in 32-bit. My current Windows Vista Install is in 64-bit. I could re-add the presets for a new test.

My notebook has no NVIDIA. Was that AMD? I could turn on my notebook later and look there, but currently I am busy at my PC and updating my image collections after adding some more images (also replaced a terrible image by a better image).

The attached image looks more beautiful than what I saw once in the presets.

Edit: The image looks like "martin - mandelbrot orbit trap ocean style.milk", which has worked on both computer, although it is in PS4. Re-added now "martin - orbit trap playground.milk" and "martin - orbit trap playground - while loop.milk" in Winamp on my PC. They didn't look bad, although they came and then they stood still.

Sabine Klare Aka Sternenmaschinebine
Music, Art, Lyrics, Videos
AMBIENT... AMBIENT music forever...

Last edited by Sabine Klare; 19th November 2014 at 17:52.
Sabine Klare is offline   Reply With Quote
Old 19th November 2014, 20:06   #9
Amandio C
Senior Member
 
Join Date: Dec 2008
Posts: 400
Hello.

This a subject I find quite interesting, that we discussed some years ago.
I think a different approach would give much better results in terms of response of a music visualizer.
But it would require a huge amount of work and would infringe some copyright laws.

In simple terms, the visualizer would read, not the music file, but some kind of special, improved, midi file.
This file would not be heard but would serve only to feed the visualizer with data.
A midi file contains the partiture (notes and beats) of every single instrument and voice.

This software could perhaps be implemented by a company like Sony for the consoles, since they have a large music catalogue.
The "midi" files would be produced by them for the most important tracks of their catalogue and could be downloaded automatically by the console while playing audio CDs or Mp3s.

It would be a unique product for their clients, that could also be used in concerts, etc.

Just an idea.
Amandio C is offline   Reply With Quote
Old 22nd November 2014, 18:02   #10
Flexi
wellspring of milk
Major Dude
 
Flexi's Avatar
 
Join Date: Apr 2007
Location: 54.089866,12.11168,18.75
Posts: 2,058
Send a message via ICQ to Flexi
Perhaps, there's a ready-made web services available for sound analyzation. This wouldn't work for live streaming, but they also have a database for known tracks. I'm speaking of the Echonest API: http://developer.echonest.com/docs/v4
Flexi is offline   Reply With Quote
Old 22nd November 2014, 20:26   #11
Victhor
Late skinner & Moderator
 
Victhor's Avatar
 
Join Date: May 2003
Location: Argentina
Posts: 1,606
ANYTHING that could achieve something like T H I S would really be called a new generation visualizer and bring Winamp visualization again into the world spotlight.. now hold on, I think it's time to wake up..

..sorry for the slightly off topic (I needed to say it)..
Victhor is offline   Reply With Quote
Old 22nd November 2014, 20:55   #12
Flexi
wellspring of milk
Major Dude
 
Flexi's Avatar
 
Join Date: Apr 2007
Location: 54.089866,12.11168,18.75
Posts: 2,058
Send a message via ICQ to Flexi
Victhor, your linked video totally reminded me of this discussion: https://plus.google.com/u/0/+FelixWo...ts/bZNfZwedbA2
Flexi is offline   Reply With Quote
Old 22nd November 2014, 21:13   #13
Victhor
Late skinner & Moderator
 
Victhor's Avatar
 
Join Date: May 2003
Location: Argentina
Posts: 1,606
D'oh is the same video lol (there are 3 links in my post). I didn't understood much of what's being said, but I totally love the outcome and I'm actually collecting those Matthias videos (and the sort).
I would love Winamp to be able to play music while playing videos at the same time (yeh.. crazy no?)... that would be a nice way to have those kind of visuals along with music..
Victhor is offline   Reply With Quote
Old 23rd November 2014, 08:39   #14
Flexi
wellspring of milk
Major Dude
 
Flexi's Avatar
 
Join Date: Apr 2007
Location: 54.089866,12.11168,18.75
Posts: 2,058
Send a message via ICQ to Flexi
the other two links are the same?
In shadertoy, there's a 1D texture input available for a shader-based sound curve visualization: https://www.shadertoy.com/view/4sf3RH

Sorry Martin, i haven't said a word about your topic. I have just yet fetched a glimpse at the code. I hope a full quote is ok
Quote:
Originally Posted by Nitorami View Post
//############## THIS IS A CONSTRUCTION SITE #########

dt = min (1/fps,0.1) ; //Stability !
dec_m = 1-dt*4; dec_s = 1 - dt; dec_xs = 1-dt/6;
t0 = time - tstart;

bd_b = bass; bd_bass_ = bd_bass_*dec_m + (1-dec_m)*bd_b; bd_bass = bd_b-bd_bass_;
bd_m = mid ; bd_mid_ = bd_mid_ *dec_m + (1-dec_m)*bd_m; bd_mid = bd_m -bd_mid_;
bd_t = treb; bd_treb_ = bd_treb_*dec_m + (1-dec_m)*bd_t; bd_treb = bd_t-bd_treb_;

n = 0; BPM = bd_minbpm;
loop (bd_nres,
gmegabuf(50000+n*bd_recsz) = 0;
bd_arg = BPM/60*t0*6.283;
gmegabuf (bd_tab0+n*bd_recsz) = BPM;
BPM *= bd_finc;
n+=1);

bd_src=0;
loop (3, bd_exc = bd_bass*(bd_src==0) + bd_mid*(bd_src==1) + bd_treb*(bd_src==2);
oct = 0;
loop (bd_oct,
bd_slot = 0;
bd_pk = 0; bd_mean = 0.0;
loop (bd_spo,
bd_tabptr = bd_tab0 + (oct*bd_spo+ bd_slot)*bd_recsz;
n = bd_dat0 + (bd_nres*bd_src +oct*bd_spo+ bd_slot)*bd_recsz;
BPM = gmegabuf(bd_tabptr); bd_omega = BPM/60*6.283;
Feder = sqr(bd_omega); //Res.f = sqrt (Feder)/2pi (Hz)
gmegabuf (n+4) = gmegabuf(n+2); //alte pos
gmegabuf (n+1) *= 1-.5*dt * sqrt (BPM/300); //damping

EC_Steps = int (1 + BPM/50)*2;
loop (EC_Steps, // Euler-Cauchy steps
Kraft = bd_exc - Feder*gmegabuf(n + 2) ;
gmegabuf (n+1) += Kraft * dt/EC_Steps;
gmegabuf (n+2) += gmegabuf(n+1) * dt/EC_Steps;
);
beatcos = gmegabuf(n+1); beatsin = gmegabuf(n+2)*bd_omega;
quad = sqrt (pow(beatsin,2) + pow(beatcos,2));
gmegabuf (n) = gmegabuf(n)*0.8 + 0.2*quad;
bd_ampl = gmegabuf(n);
bd_mean += bd_ampl;
if (bd_ampl > bd_pk, bd_pk=bd_ampl, 0);
bd_slot +=1;
// Komb. bass,mid,treb
bd_slot = 0;
bd_qual = pow(bd_pk/bd_mean*bd_spo-1,1);
gmegabuf((bd_src*bd_oct+oct)*2) = bd_qual;
loop (bd_spo,
m = 50000 + (bd_slot + 0*oct*bd_spo)*bd_recsz;
n = bd_dat0 + (bd_nres*bd_src +oct*bd_spo+ bd_slot)*bd_recsz; //#2x berechnet
gmegabuf(m) += (bd_qual)*gmegabuf (n)/12;
bd_slot +=1;
oct += 1;
bd_src +=1;


//Get maxind
bd_slot = 0; maxind = 0; bd_pk = 0; bd_mean = 0;
loop (bd_spo,
m = 50000 + bd_slot*bd_recsz;
gmegabuf(m) = pow(gmegabuf(m),1);
bd_mean += gmegabuf(m);
if (gmegabuf(m) > bd_pk, bd_pk = gmegabuf(m); maxind = bd_slot, 0);
bd_slot += 1);
quali = bd_pk/bd_mean*bd_spo-1;

n = 0; loop (150, gmegabuf(50+n) *= 0; n+=1);

//KREISE
bd_src = 0; tsin = 0; tcos = 0;
loop (3,
oct = 0;
loop (bd_oct,
n = bd_dat0 + (bd_nres*bd_src +oct*bd_spo+ maxind)*bd_recsz;
bd_tabptr = bd_tab0 + (oct*bd_spo+ maxind)*bd_recsz;
BPM = gmegabuf(bd_tabptr);

im = gmegabuf(n+1);
re = gmegabuf(n+2) * BPM/60*6.283;
bd_cos = cos (BPM/60*6.28*time);
bd_sin = -sin (BPM/60*6.28*time);

//Winkel zurueckdrehen. Das sind nun die Winkel relativ zu cos(oct*time).
re0 = re*bd_cos + im*bd_sin;
im0 = -re*bd_sin + im*bd_cos;
bd_qual = gmegabuf((bd_src*bd_oct+oct)*2);
gmegabuf (50+oct*4) += bd_qual * (im0)*1;
gmegabuf (50+oct*4+1) += bd_qual * (re0)*1;
gmegabuf (50+oct*4+2) += bd_qual;

//Summe der besten Beitraege. Das ist der Referenzwinkel.
gmegabuf (100+0) += bd_qual * im0;
gmegabuf (100+1) += bd_qual * re0;

//Nun sollten wir die Differenzwinkel zur Ref berechnen. Und was damit ? Dann kann ich die
//Einzelwinkel auf Ref bringen - bringt doch nichts !
//Ich kann alle auf Null drehen, daraus sinusse erzeuge und addieren ->besserer Takt ?
re = (bd_cos*re0 + bd_sin*im0);
im = (-bd_cos*im0 + bd_sin*re0);
tsin += im*5;

oct += 1);
bd_src +=1);
q28 = tsin;

vol = bass_att + mid_att + treb_att;
vol_ = vol_*0.95 + 0.05*vol;
q29 = vol-vol_;

//Welche Oktave ???
bd_src = 0; maxo = 0; maxs = 0; maxa = 0;
loop (3,
oct = 1;
loop (bd_oct-1,
n = bd_dat0 + (bd_nres*bd_src +oct*bd_spo+ maxind)*bd_recsz;
bd_qual = gmegabuf((bd_src*bd_oct+oct)*2);

ampl = bd_qual / (1+oct*(maxind/bd_oct)*0);
if (ampl>maxa, maxa = ampl; maxs = bd_src; maxo = oct;, 0);
oct += 1);
bd_src +=1);

//Beat-Flash
testi = bd_nres*maxs +bd_spo*maxo + maxind;
if (bor (beat && (prog==0), abs(testi-test0)<2),test0 = testi,0);
n = bd_dat0 + (test0)*bd_recsz;
creep = gmegabuf(n+1);
beat = (creep>0) * (creepo <0);
beatct = if (change, 0, beatct+beat);
prog = (prog+beat)%8;
creepo = creep;
show = if (beat, 1, show*0.5);
q1 = (gmegabuf(n+1)/gmegabuf(n));
q2 = prog;



//Selbst bei Barbie Girl fehlt manchmal der Rhythmus in Band 2,
//obwohl das der gefuehlte Takt ist - >Problem mit Phasenkorrektur !
//stehen Phasen bei doppelter Frequenz in Relation ? NEIN.





//Problem: Fats Domino (1992) hat 55BPM -> Problem mit 110
//Problem: 1991 London Beat I been thinking about you - geht nicht - solved
//Problem: U96 - Love religion (1995) beginnt mit BPM < 20
//Porblem: adiemus (1995)
//Konflikt-Beat siehe Michale jackson b&W (1991)


q21 = bd_maxbpm;
q32 = aspecty;monitor = maxind;
Which variable can i use to trigger a random change in a set of variables like i used here

Quote:
Originally Posted by Flexi View Post
zoom = 1;
wave_a = 0;

startx = 0.3;
starty = 0.3;

a = 0.55;
b = 0.05;
d = 0.09;

q1 = aspectx;
q2 = aspecty;
q5 = a;
q6 = b;
q7 = d;

//lines below belong to the composite shader (flexis kaleidoscope)

//flexi's beat detection (made for breakbeats)
vol = bass*8 + mid*5 + treb*3;
m = m*0.97 + vol*0.08;
monitor = vol;
beat = above(vol,res)*above(vol,m)*above(vol,16);
diff = (1-beat)*diff + beat*(vol-res);
res = beat*(vol + m*0.04) + (1-beat)*(res - (0.1+diff*0.02)*60/fps);
res = max(0,res);

w = if(beat, rand(4096)/1024*asin(1),w); // on beat: randomize pos and angle of the kaleidoscope
xx = if(beat, 0.1 + rand(800)/1000, xx);
yy = if(beat, 0.1 + rand(800)/1000, yy);

//geiss' custom beat detection code: (fps-independent; quiet songs don't freak out)
decay_rate = pow(0.995, fps); // lower # = more hasty to declare a beat
min_att = 2.2; // lower # = quieter songs can declare beats
decay_to = 1; // lower # = more hasty to declare a beat
beat = bass/max(min_att,bass_att);
beat = max(beat, mid /max(min_att,mid_att ));
beat = max(beat, treb/max(min_att,treb_att));
beat = max( beat, (prev_beat-decay_to)*decay_rate + decay_to );
beat_level = (beat - prev_beat - 0.07)*24;
is_beat = above(beat_level, 0.5);
prev_beat = beat;

w = if(is_beat, w + (rand(256)-128)/1024, w); // on beat: random increment of pos and angle
xx = if(is_beat, xx + (rand(256)-128)/4096, xx);
yy = if(is_beat, yy + (rand(256)-128)/4096, yy);


q11 = sin(w); // w - angle of kaleidoscope
q12 = cos(w);
q13 = xx; // center of kaleidoscope window
q14 = yy;
q17 = sin(-w);
q18 = cos(-w);
I'm a bit helpless
Flexi is offline   Reply With Quote
Old 23rd November 2014, 11:26   #15
Nitorami
Major Dude
 
Join Date: Mar 2008
Location: Erlangen
Posts: 854
Haha, the editor changed the closing brackets of the code into smilies.

Flexi: At the moment this is just a pre-stage, meant to show how it can extract repeating patterns in the music. Better not try to use the code yet, I will come up in a few days with a version that provides one, or several, beat variables. I'd like to have a "trigger" beat as you say, to trigger events such as random variables; additionally I'd like to provide a linear variable in strict time, for rhythmic movements, e.g. lines rotating in rhythm.

Martin
Nitorami is offline   Reply With Quote
Old 23rd November 2014, 13:44   #16
Flexi
wellspring of milk
Major Dude
 
Flexi's Avatar
 
Join Date: Apr 2007
Location: 54.089866,12.11168,18.75
Posts: 2,058
Send a message via ICQ to Flexi
You make use of the gmegabuf too. What's the minimum index that is safe to use then?
Flexi is offline   Reply With Quote
Old 23rd November 2014, 14:49   #17
Nitorami
Major Dude
 
Join Date: Mar 2008
Location: Erlangen
Posts: 854
@Flexi: I started to move everything to the address space >100.000 to get it out of the way, but the current version also uses lower addresses starting at 0 and at 50000. Do not use it. I will get rid of that.

@Victhor: I agree this stuff looks fantastic. It is not possible in MD2 in principle because a Z Variable is not supported. I still hope for Litheon to come up with a new version, MD3. Unfortunately I cannot do it, I lack the programming skills and familiarity with C, Winamp API, DirectX API etc. Too much... Litheon ???
But as regards beat detection, there is none, this stuff just uses the volume signal with some triggering and smoothing, maybe some manual interaction.
Nitorami is offline   Reply With Quote
Old 23rd November 2014, 16:23   #18
Flexi
wellspring of milk
Major Dude
 
Flexi's Avatar
 
Join Date: Apr 2007
Location: 54.089866,12.11168,18.75
Posts: 2,058
Send a message via ICQ to Flexi
I'm really curious where this will go.

Basically, I could try and compile Milkdrop2, but I'm overbusy. I've changed my position and I'm working in the scientific/medical imaging field now. Also, i became a daddy for the second time earlier this year, and when I'm not writing WebGL experiments or remixing old presets, I'm currently working on this spline editor (to be finally integrated with a laser display): http://studio.sketchpad.cc/sp/pad/vi...qHCh2kh/latest

Then again, what I'd really like to see in winamp would be a freely customizable graph for multiple buffers and shader stages. Perhaps you could put the beat detection in one GPU analysis pipeline too.
Flexi is offline   Reply With Quote
Old 24th November 2014, 19:17   #19
Nitorami
Major Dude
 
Join Date: Mar 2008
Location: Erlangen
Posts: 854
Alright. Attached another simple demo. It is still somewhat buggy and chaotic, do not try to use this code. It shows:
  • The bar at the left border is the quality of detection (the algorithm thinks).
  • The white curve.... well, in short, it is the composite information extracted from bass, mid and treb. The peak indicates the detected BPM rate on the scale of green and blue dotted lines; the first green line marks 50 BPM, the second 100 BPM etc. up to 320.
  • The jumping square is the beat. If the quality drops below bd_minq, the algorithm falls back to a standard volume triggered beat reaction, indicated by the square turning red.
  • You should turn off browsers or other programs demanding CPU power in unpredictable intervals do do something crucially important such as annoying you with banner ads.
  • To force the standard beat reaction for comparison, simply increase the value of bd_minq at the top of the per frame section.

After having tested it a lot, I'll summarise the pros and cons of this beat detection.

Cons:
  • After the preset was started, synchronisation will take a moment. Not good for quick change of presets. It would be better to have the code in the visualiser dll itself, not in the preset.
  • The whole thing depends on a reasonably constant frame rate, because there is jitter between milkdrop time and song time. Again, this could be solved by integrating the scheme into the dll rather than the preset.
  • The beat may appear lagging and less direct than the standard scheme. It handles some songs better, others worse. If music has conflicting signals, i.e. a bass drum and snares interlaced, the algorithm may synchronise on the wrong one. I do not currently know how to overcome this.

Pros:
  • Once synchronised, the indicated beat will be very regular, there will be no missing or extra beats. With rhythmic music, this can be quite immersive (you will not experience that with THIS stupid preset though)
  • It will keep clocking over short pauses
  • It is relatively robust and not easily irritated by off-beat signals.
  • It is predictive, while the standard scheme has no clue when the next beat arrives.

Altogether, presets using this sort of detection should not use the beat to trigger something; this does not require strict time. Presets should exploit features the standard beat reaction does not have, mainly beat prediction.

Maybe this is best explained by an old idea I had when I started making presets: Imagine sitting in a roller coaster. You are looking on the rails in front of you. You can see the curve ahead, the coach accelerates, then you swing into the curve exactly when the beat arrives. The next curve comes in rhythm with the music etc. This requires a prediction of the next beat which is just not feasible with the standard scheme.

Martin
Attached Files
File Type: milk martin - harmonic beat detection demo II.milk (14.7 KB, 333 views)
Nitorami is offline   Reply With Quote
Old 25th November 2014, 08:14   #20
Flexi
wellspring of milk
Major Dude
 
Flexi's Avatar
 
Join Date: Apr 2007
Location: 54.089866,12.11168,18.75
Posts: 2,058
Send a message via ICQ to Flexi
a beat prediction could still go hand in hand with a classical energy threshold driven detection.

Just a heads up!

Flexi is offline   Reply With Quote
Old 2nd November 2015, 20:29   #21
Nitorami
Major Dude
 
Join Date: Mar 2008
Location: Erlangen
Posts: 854
A year is almost over

and it is time for a new attempt on beat detection.

It is very computational efficient because it uses fast memory moves for the delay chain, and needs no trigonometrics nor a lot of multiplications. In the sample attached, the majority of CPU power is used to draw the shapes.
The drawback is that it is very dependent on a constant frame rate.

Therefore if you try it out, please close other processes that interfere. This may be less of an issue on multicore processors, but on my single core a browser or virus scanner running in the background totally cocks it up.

The algorithm is not mature nor fine tuned yet but in overall seems to works quite well. Other than my previous attempts which tried to determine an absolute constant beat rate, this one reacts on volume and therefore feels rather "direct", but still uses resonators for preprocessing so it will keep clocking over short pauses. Just try it out.
Attached Files
File Type: milk martin - ringbuf beat detection 5.milk (11.5 KB, 408 views)
Nitorami is offline   Reply With Quote
Old 24th August 2017, 06:58   #22
hatecubed
Junior Member
 
Join Date: Sep 2016
Posts: 32
Hi Martin,

Not wanting to start a new thread, but, apologies for also reviving an old one here I'm a huge fan of your work, and my entire life (but more now than ever) has been an obsession with milkdrop - and beats - and I've been playing around with optimizing beat detection (with the trusty combination of linein://, my DJ mixer, and a HD projector!). I'm just curious if you've been thinking about this stuff at all over the last year or two? If so, I am curious to know your input/output plugin configuration... thanks to Winamp passing everything via the output plugin, every step of the chain can impact the sync of the beats and the vis.

I've got a lot to learn about this yet - hopefully posting here more can help keep things moving. I've been pleasantly surprised so far
hatecubed is offline   Reply With Quote
Old 24th August 2017, 18:44   #23
Nitorami
Major Dude
 
Join Date: Mar 2008
Location: Erlangen
Posts: 854
Hi hatecubed
I am using the absolute standard config, input as required, output out_wave, my presets are generally using the bass,mid,treb variables as input to the beat detection. If you are talking about latencies, I believe that the major issue here is the detection as such, not the output config.
And yes, I have continued tinkering with this, and come to a limit where I don't think it is worth putting more effort into it. If you are seriously interested, drop me a note.
Nitorami is offline   Reply With Quote
Old 29th August 2017, 20:54   #24
Nitorami
Major Dude
 
Join Date: Mar 2008
Location: Erlangen
Posts: 854
Hi Josh
following the PM, here are two presets for experimenting:

bass_mis_treb_over_time shows, from bottom to top, bass, mid, treb, and the sum of these. Each dot is a frame, you'll see that the peaks are sometimes fast, sometimes they take a few frames to reach the maximum.
Also shown (gray) is the threshold of the standard beat detection; a beat is detected when the white curve exceeds the threshold.

The other one is funky illusions with an adjustable delay in the beat detection. Press M while the preset is running to enter edit mode and select "edit preset initialisation code". You'll find the instructions there. To change the delay, simply edit and press Ctrl+Enter to accept the change. It will take immediate effect.
Attached Files
File Type: milk martin - funky illusions with adjustable beat delay.milk (19.6 KB, 209 views)
File Type: milk bass_mid_treb_over_time.milk (7.0 KB, 203 views)
Nitorami is offline   Reply With Quote
Old 30th August 2017, 07:52   #25
hatecubed
Junior Member
 
Join Date: Sep 2016
Posts: 32
Martin,

Excellent! Successfully downloaded. Many thanks for putting these together for me. I'll have a play with them as soon as possible and let you know how it all goes

(And a return PM is coming your way, naturally...)
hatecubed is offline   Reply With Quote
Old 30th August 2017, 14:02   #26
Sabine Klare
Forum Queen
 
Sabine Klare's Avatar
 
Join Date: Nov 2010
Location: Berlin, Germany
Posts: 2,492
Yep, with these 2 newest presets I will have 31968 presets in my both Winamps, and instead of 999 pages I will have 1000 pages in the Load-Menue. Of course also I myself want to see the presets. And I have to figure out, how many presets can I use in Winamp.

We have seen us already in some other threads, where I have said also something about my city. Hopefully I myself will have again some "Sunny Days" for going outside into parks and to the water, because I want to have some more perfect basic photos for some more random textures and desktop wallpapers, everything about nature, plants, stones, water. I want to share all my own works for the future Electronic Live Concerts and Techno Live Events.

I wish You good luck with the presets and with the VJing, hatecubed...

Sabine Klare Aka Sternenmaschinebine
Music, Art, Lyrics, Videos
AMBIENT... AMBIENT music forever...
Sabine Klare is offline   Reply With Quote
Old 30th August 2017, 18:22   #27
Nitorami
Major Dude
 
Join Date: Mar 2008
Location: Erlangen
Posts: 854
Sabine
these two are not meant to be used as presets, just experimenting. I am in a PM talk with hatecubed but the messaging system does not allow me to send attachments, so I put this here.

Martin
Nitorami is offline   Reply With Quote
Old 1st September 2017, 14:55   #28
Sabine Klare
Forum Queen
 
Sabine Klare's Avatar
 
Join Date: Nov 2010
Location: Berlin, Germany
Posts: 2,492
I didn't know that they were not be meant to be used as presets. I like to collect everything what I can get because also I myself manage some archives incl. ultra-archives with presets, textures, wallpapers. And I want to use very much presets which are not too ugly and annoying with their filenames. Also I have replaced some annoying and terrifying textures by other images which are not so terrifying for me.

Already in Summer 2013 I had discovered, that the messaging system does not allow a forums member to send attachments. And because the filesizes for attachments are limited, I have many .zip-folders on Mediafire, mostly with my own artist-works.

Sabine Klare Aka Sternenmaschinebine
Music, Art, Lyrics, Videos
AMBIENT... AMBIENT music forever...
Sabine Klare is offline   Reply With Quote
Old 2nd September 2017, 22:38   #29
xmuzack
Senior Member
 
xmuzack's Avatar
 
Join Date: Sep 2016
Posts: 110
Quote:
Originally Posted by Sabine Klare View Post
I like to collect everything what I can get.
Lol, I do the same thing, and then I scratch my head later, wondering what some presets were meant to do. I now know those were meant to be experiments, sharing of concepts or building blocks to make a better preset.

Last edited by xmuzack; 2nd September 2017 at 22:39. Reason: misspelled word
xmuzack is offline   Reply With Quote
Old 18th September 2017, 09:37   #30
Nitorami
Major Dude
 
Join Date: Mar 2008
Location: Erlangen
Posts: 854
Another test

Do not download this, it's just a test.
Attached Files
File Type: milk delayed beat test.milk (7.0 KB, 138 views)
Nitorami is offline   Reply With Quote
Reply
Go Back   Winamp & Shoutcast Forums > Visualizations > MilkDrop > MilkDrop Presets

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump