Old 27th August 2002, 10:44   #1
Rovastar
Moderator
 
Join Date: Jun 2001
Location: London, England
Posts: 3,632
Send a message via AIM to Rovastar
Beat detection.....again

I know that we cannot get perfect 'beat' detection.

Speaking to Gordon WIllaim the creator of the R2/Extreme plugin about implementing beat detection in his forthcoming R3...and failing.

Anyway here is how he tried to do it.

------------------
adds the bottom end of the spectrum together using a cos-based envelope.
Makes a running average of this value, and subtracts it so the bass value is centered on 0.
makes a running average of the standard deviation of the bass, and divides by this so the sound data is normally distributed around 0.
does if bass>myvalue then myvalue=bass else myvalue=average(bass,myvalue)
------------------------

ANy thoughts/comments on how to improve this........

"Rules are for the guidance of wisemen and the obedience of fools"

Visuals - Morphyre www.Morphyre.com
Rovastar is offline   Reply With Quote
Old 27th August 2002, 23:14   #2
ryan
not fucked, not quite.
(Forum King)
 
ryan's Avatar
 
Join Date: Feb 2002
Location: Tn
Posts: 8,798
Send a message via AIM to ryan
Unchained or Rozz might have an idea.... Not me though..Since most of the beat detection stuff I do is RANDOM

Why not just use a method simillar to the one Krash made?
ryan is offline   Reply With Quote
Old 28th August 2002, 05:59   #3
Krash
Major Dude
 
Krash's Avatar
 
Join Date: Jun 2001
Location: Sydney, Australia
Posts: 977
Porting my method across to another plugin would be doable, though I don't know precisely how bass and bass_att are calculated in milkdrop (I know how they work, but I don't know their formulas)

The basic premise of my beat detection is as follows:

First, we have to define in our minds what a "beat" is. My definition, is that in a vast majority of music, a beat is exhibited by a marked increase in the amount of bass in the sound wave (usually due to a bass or snare drum).
So the beat needs to be related to the (relative) level of bass.

Secondly, "beats" very rarely occur during quiet moments in the music (cuts, background, intros and outtros, etc). If they do, it is usually
due to a fade-in or fade-out of the music.
So the detection of a beat needs to be related to overall volume.

However, not every song has beats at the same rate, and they are not necessarily constant within a song.
So the detection of a beat needs to be semi-dependant on time - using an accumulative "beatrate" to estimate when the next beat should occur. Leniency to either side of this estimation allows for the beatrate to adapt to changes in tempo, as well as correct the occasional mistake.


So that's basically what my code does:
beatrate = equal(beatrate,0) + (1-equal(beatrate,0))*(below(volume,0.01) + (1-below(volume,0.01))*beatrate);
- this line resets the beatrate to 1 (1 beat per second) if the music drops to a very low volume, or upon initiation of the preset.

lastbeat = lastbeat + equal(lastbeat,0)*time;
- if a beat was detected on the previous frame, lastbeat is set to the current time (used for calculating beatrate)

meanbass_att = 0.1*(meanbass_att*9 + bass_att);
- running average of bass_att values for the last 10 frames.

peakbass_att = max(bass_att,peakbass_att);
- finds the peak bass_att value since peakbass_att was last reset.

beat = above(volume,0.8)*below(peakbass_att - bass_att, 0.05*peakbass_att)*above(time - lastbeat, 0.1 + 0.5*(beatrate - 0.1));
- beat is set to 1 IF the volume (calculated as 0.3*bass+mid) is above 0.8 AND bass_att is at least 95% of peakbass_att AND the time since the last beat is at least 50% of the beatrate (-0.1 - for reasons to be explained)

beatrate = max(if(beat,if(below(time-lastbeat,2*beatrate),0.1*(beatrate*9 + time - lastbeat),beatrate),beatrate),0.1);
- if a beat occurs, and it has been less than 2*beatrate since the last beat, calculate beatrate as the average time between the last 10 beats (running average). If it has been longer than this since the last beat, leave beatrate alone (maybe there was a long quiet bit in the music that we didn't pick anything up on, but the tempo remained the same). Also, prevent beatrate from going below 0.1 (600 beats per minute), as below this point the algorithm becomes hyperactive, and beats are detected almost constantly.

peakbass_att = beat*bass_att + (1-beat)*peakbass_att*(above(time - lastbeat, 2*beatrate)*0.95 + (1-above(time - lastbeat, 2*beatrate))*0.995);
- If a beat occurs, peakbass_att is set to current bass_att. Otherwise, peakbass_att is reduced slightly. If it is less than 2*beatrate since the last detected beat, the peakbass value is decreased a small amount. If it has been longer than this, peakbass is reduced drastically (maybe we have jumped to a section of music which still has distinct beats, but is much quieter). Remember that beatrate is unaffected if the time between beats is this long.

lastbeat = beat*time + (1-beat)*lastbeat;
- update lastbeat to the current time if a beat was detected.

peakbass_att = max(peakbass_att,1.1*meanbass_att);
- peakbass (one of our criteria for detecting a beat) should always remain at least slightly above the average bass level.


So that's my reasoning behind it all - it's actually pretty basic, once you get past the code and just see the logic - but it does work.
If someone knew what the calculation for bass and bass_att are, they could easily port this code to another plugin (like AVS or R2)

- Krash

Eighty-three percent of all statistical quotes are made up on the spot.
Krash is offline   Reply With Quote
Old 28th August 2002, 07:59   #4
che.0
Member
 
che.0's Avatar
 
Join Date: Jun 2002
Location: Prague, Czech Republic
Posts: 62
Krash your algorithm is really impressive.

My algorithm simply detects beat if mixture of bass_att (mostly) and treb starts exceeding an adaptive threshhold. Some day I'll try to somehow make it to detect and follow a beatrate ... maybe it'll work better then.
che.0 is offline   Reply With Quote
Old 28th August 2002, 21:23   #5
ryan
not fucked, not quite.
(Forum King)
 
ryan's Avatar
 
Join Date: Feb 2002
Location: Tn
Posts: 8,798
Send a message via AIM to ryan
I would Just like to say.....

My beat stuff is basically.... Just a variation of each of the 3 bass treb and mid.... The only thing about mine is that it reacts too much on some stuff and not enough on some.
ryan is offline   Reply With Quote
Old 29th August 2002, 00:54   #6
Rovastar
Moderator
 
Join Date: Jun 2001
Location: London, England
Posts: 3,632
Send a message via AIM to Rovastar
Unchained mate I know that getting a 'beat' is not your style (it cannot really be done and teh way you write you music detection code is very different) but if you were to do it how would you?

I am interested to see how you would attempt it.

"Rules are for the guidance of wisemen and the obedience of fools"

Visuals - Morphyre www.Morphyre.com
Rovastar is offline   Reply With Quote
Old 30th August 2002, 02:09   #7
unchained
Major Dude
 
Join Date: Jul 2001
Location: richmond, va
Posts: 639
Send a message via Yahoo to unchained
As I've said elsewhere, I feel that beat detection as traditionally conceived isn't possible even in principal. While my reasons for thinking this are more philosophical than scientific, they're still rather technical, and so I'll only attempt to outline them in brief. It is my hope that through this we can gain a better understanding of what is needed by a modern theory.

First, it would be well to clarify what I mean by the classic theory of beat detection; Krash and I seem to disagree strongly on what constitutes a beat, so I'll take his views and how I interpret them as a starting point. (The following is not meant in any way as a criticism of Krash's views, only as a warning against the uncritical acceptance of the theory they outline) On Krash's view, a beat has three essential characteristics:

1) in a vast majority of music a beat is exhibited by a marked increase in the ammount of bass. the beat needs to be related to the (relative) level of bass

2) beats very rarely occur during quiet moments in the music, and so must be related to the overall volume

3) not every song has the same beat rate, nor is it required that the beat rate be constant throughout the song

That these essentials are vague goes without saying. We are told that a beat is something that we must "define in our mind", the implication being from the start that they are somehow immune to analytical considerations. Nevertheless, let us attempt to submit them to further inspection:

- In the first case, what are we to understand by a "marked increase"? I assume that this marking is the same consideration as the parenthesized "relative" consideration, but no clue is given as to what this level is relative to, nor are we given to understand what this "vast majority" consists in, nor even a hint as to how it's to be identified. Certainly a song with less bass will have "relatively" less bass...something else must be at work here. Perhaps a resolution of these issues could provide a hint at a better definition.

- In the case of 2), we're again given a more-or-less vague qualification, rather than any condition by which a case is to be judged. It's hinted again that there's some sort of relative consideration to be taken into account, and this time our attention is directed towards the overall volume of the song. While the song (at least in milkdrop) isn't given to us in entirety, this "overall" must mean for us some sort of accumulated average...fair enough, but what type of specific relationship of the bass levels to this average are we to consider?

- 3) I support fully, in that it insists we're not begging the question. 3) asserts that there is in fact a beat, different for every song (and so we're not looking for a trivial constant), and that the beat changes WITHIN the song. This last would seem to be key amongst the criterion thus far given, in that it's the only one that gives us any assurance that the beat can be determined at a given point.

I hope I've been able to hint at the flavor of some of my thoughts on this issue. I can sum up by saying simply that on my view, not nearly enough reflection has been given to what we say when we state intuitively that a song has a beat. In particular, any possible strenghthening of 3) above, such that it assured us that there was not only something to find, but that it could be found by the means and methods at our disposal (treating such issues as our only being able to "see" the music as a series of dcrected point-instants from the start of the song towards the end) would be most desirable.
unchained is offline   Reply With Quote
Old 30th August 2002, 07:57   #8
Krash
Major Dude
 
Krash's Avatar
 
Join Date: Jun 2001
Location: Sydney, Australia
Posts: 977
in retrospect, I see that my definition of a beat (as written above) is pretty vague. But that's the process I went through when figuring out how to code my algorithm. If you want a succinct definiton of beat, it is similar to the definition of tempo.

In a piece of written music, the tempo is designated by how many crotchet notes will be played per minute. It's written down, and is a part of the music - for example, the 1812 overture will have pretty much the same tempo, no matter which conductor and orchestra is performing it. (Unless of course, the conductor is purposely altering the tempo for whatever reason)

A 'beat' will occur on some or all of those crotchet notes. A beat can occur on non-crotchet notes (off-beat triplets, syncopation, etc.) Most music (pop, r&b, rock, country, etc.) uses a basic 4/4 time signature, and so beats usually occur on crotchet notes.
The minimum number of beats per bar (of music) is determined by the rhythm. If a rhythm repeats itself once a bar, then you should have at least 1 beat per bar. If it takes two bars to repeat, you should have at least 1 beat per 2 bars.
During the time the rhythm is constant, beats should appear at the same place every bar (or series of bars).

The simplest example of a beat that I can think of is the 1812 overture - during the final part (mental block on the proper terminology) when the cannons are firing.

The cannons are beats. The strings in between are not beats.

I can't make it any simpler than that, and I'm in a hurry anyway =\

- Krash

Eighty-three percent of all statistical quotes are made up on the spot.
Krash is offline   Reply With Quote
Old 30th August 2002, 11:49   #9
mstress
Senior Member
 
Join Date: Aug 2002
Location: Florence (Italy)
Posts: 113
Send a message via ICQ to mstress
I think that Unchained arguments are really valid. But I prefer the approach of Krash who try to find a solution.
Probably noone will ever find the exact way to extract the right beat of every song. The very important thing is to try and draw near and near to solutions. This is called research for progress.
To make philosofy is interesting and useful but it's more useful to think a solution even if this is not perfect and probably it will never be.
Excuse me for my english . I don't want to offend anyone. It's just what I think!

mstress
mstress is offline   Reply With Quote
Old 30th August 2002, 13:09   #10
unchained
Major Dude
 
Join Date: Jul 2001
Location: richmond, va
Posts: 639
Send a message via Yahoo to unchained
I am agree! Sure, words have meanings and stuffs but isn't need for make happy ju-ju. Wall is big, and beat head just as fun as climb wall! Meeso clim-clam, jusa beeta-booh.

I was talking to the rabbits, mstress, not to the crows.
unchained is offline   Reply With Quote
Old 30th August 2002, 13:31   #11
Krash
Major Dude
 
Krash's Avatar
 
Join Date: Jun 2001
Location: Sydney, Australia
Posts: 977
something I forgot to put in my last post:

an ideal beat detection algorithm (IMO) would detect not beats per se, but instead the actual tempo of a track (eg 100 bpm). Once the tempo was detected (and it would need adjustment as the song progressed), the actual application of the code (in this case, a preset) could choose whether or not to perform an action at the tempo "pulses" (for lack of a better word), based on other (arbitrary) factors designated by the author (such as a certain time interval elapsed, a minimum volume, etc).

- Krash

Eighty-three percent of all statistical quotes are made up on the spot.
Krash is offline   Reply With Quote
Old 30th August 2002, 13:32   #12
mstress
Senior Member
 
Join Date: Aug 2002
Location: Florence (Italy)
Posts: 113
Send a message via ICQ to mstress

uhm......I cannot understand if you are joking me....
pheraps....boh....I don't understand english so well to catch on straight away.....
I cannot translate the real meaning of "I was talking to the rabbits, mstress, not to the crows.".....
boh..."Meeso clim-clam, jusa beeta-booh." ?!?!??!
probably my brain is too small for you......
mstress is offline   Reply With Quote
Old 30th August 2002, 14:26   #13
che.0
Member
 
che.0's Avatar
 
Join Date: Jun 2002
Location: Prague, Czech Republic
Posts: 62
Quote:
an ideal beat detection algorithm (IMO) would detect not beats per se, but instead the actual tempo of a track (eg 100 bpm)
Krash correct me if I'm wrong, but isn't the tempo derived from the number of beats per certain time interval? I can hardly imagine an algorithm that would detect tempo without detection beats first.
che.0 is offline   Reply With Quote
Old 30th August 2002, 18:45   #14
Zylot
Major Dude
 
Zylot's Avatar
 
Join Date: Jul 2001
Location: Pa, US(of)A
Posts: 803
Exactly, like, Paranoia Eternal is 200 bpm (Yes, i use DDR songs for examples cause they tell you the bpm right there, plus I'm just wacky like that)

Now, that means in each minute there's 200 beats.. right?
Not so easy...

Yer fergeting 1/8 beats, 1/16, 1/32, 1/64. they sneak in there, and if you managed some code that could detect the bpm of the song, first I'd conrgatz the living day lights out of you, for a job well done, but then I'd point out that these could screw with yer code quite a bit.

-------------
What do you wish for?
--Instrumentality
Zylot is offline   Reply With Quote
Old 30th August 2002, 20:05   #15
rabidhamster
Junior Member
 
rabidhamster's Avatar
 
Join Date: May 2001
Location: Bedfordshire, UK
Posts: 15
Send a message via ICQ to rabidhamster Send a message via Yahoo to rabidhamster
Hi. As Rovastar mentioned, we've been talking about this, with a vague amount of success. I had a quick read, and i'll try and outline what I do at present, and what i've tried/believe should be done.

Firstly, it seems that for scenes in R2 there are two very specific needs for beat detection. First, there's a pulse or something - where the scene moves out, or violently moves. This should just be the bass value taken from the spectrum and messed about a bit - like the bit of post Rovastar quoted. It seems to wory pretty well as-is, but maybe needs minor tweaking. In R3, scenes are pretty much all java, and i've made an object so you can specify the frequency range and get a value out that moves. You can do stuff like make the image shimmer on high treble, and bounce in and out on bass. Anyway, thats pretty sorted.

Secondly, there's stuff like the morphs, and dancers especially that really need a value that counts up over time, at the speed of the music. Kindof like the position along a bar in the music - I believe this is pretty much what Krash is aiming at. The ideal thing for me would be like a wheel rotating at one rev per beat - just a value between 0 and 1 maybe. I've tried this, with varying degrees of success, but nothing has really been good enough.

The problem it seems is isolating a 'beat'. It seems Krash's - and my - approach relies on being able to say 'yep. there's a beat on this frame' in code. I haven't had much success, since music varies so much, and there are little sub-beats in most music. The best approach I can think of so far is to do something linearly - to look at the bass value each scene, and use it whatever, rather than to try and analyse it.

My next port of call is to model what would effectively be a driven, damped harmonic oscillator - basically a weight on a spring, with some kind of air resistance. This kind of oscillator likes oscillating in a sine wave, which is fine for what we want. Moving the top of the spring up and down by the bass value means that hopefully the weight will move up and down with the main frequency of the music. Also, systems like that have a preferred frequency of oscillation - maybe set it up to be 80bpm, and it would automatically pick out the beat at around that value for the music. 60bpm would be 60bpm, but for 200bpm music it would probably pick out 100bpm, and just oscillate up and down every two beats.

Anyway, thats my take on it. If anyone has any questions, please let me know - i'm interested in anything people have to say - this seems to be a real problem area for anyone doing music vis, especially me
rabidhamster is offline   Reply With Quote
Old 30th August 2002, 21:07   #16
Rovastar
Moderator
 
Join Date: Jun 2001
Location: London, England
Posts: 3,632
Send a message via AIM to Rovastar
There are different schools of thought on this but a question I want to raise is do you think you could every get a very good 'beat' detection for all/most types of music without any human interaction.

I thought inintially no this is not possible at all. And I mentioned this to a computer programmer friend of mine explaining that this is 'the holy grail' of music vis stuff and unlikely to succeed. Now his response was to compare it to speech recognition stuff now this is a lot more complex than simply getting a beat that a human can do easily...and it is being done nowadays. So surely it is possible and maybe we need to think about this a bit more.

"Rules are for the guidance of wisemen and the obedience of fools"

Visuals - Morphyre www.Morphyre.com
Rovastar is offline   Reply With Quote
Old 30th August 2002, 21:46   #17
unchained
Major Dude
 
Join Date: Jul 2001
Location: richmond, va
Posts: 639
Send a message via Yahoo to unchained
One would think so, Rovastar, but I'm also intrigued by this method of "solving" without any thought whatsoever, and would like to understand its workings and mechinations.

Quote:
My next port of call is to model what would effectively be a driven, damped harmonic oscillator - basically a weight on a spring, with some kind of air resistance. This kind of oscillator likes oscillating in a sine wave, which is fine for what we want. Moving the top of the spring up and down by the bass value means that hopefully the weight will move up and down with the main frequency of the music. Also, systems like that have a preferred frequency of oscillation - maybe set it up to be 80bpm, and it would automatically pick out the beat at around that value for the music. 60bpm would be 60bpm, but for 200bpm music it would probably pick out 100bpm, and just oscillate up and down every two beats.
This loosely describes what I'm doing, save that I set up several oscillators with feedback between them.
unchained is offline   Reply With Quote
Old 30th August 2002, 22:32   #18
Rovastar
Moderator
 
Join Date: Jun 2001
Location: London, England
Posts: 3,632
Send a message via AIM to Rovastar
Intersting we all want a magic bullet but with the oscalition methods it appears more often then not that it is all analog or variable like where in the real world a 'beat' is digital or Boolean it's either 'On' or 'Off' there is no 0.6534 of a beat.

I feel it can get great music responsiveness but it will it be able to get a 'beat'. I wonder how we are going to 'convert' this anolog in a boolean 'beat'. If that makes sense.

I agree though more detialed looking into the workings is required.

"Rules are for the guidance of wisemen and the obedience of fools"

Visuals - Morphyre www.Morphyre.com
Rovastar is offline   Reply With Quote
Old 30th August 2002, 23:06   #19
unchained
Major Dude
 
Join Date: Jul 2001
Location: richmond, va
Posts: 639
Send a message via Yahoo to unchained
Nah, I just want pretty pictures on my screen. If one person in 10 or so sees it and goes "Wow, how'd you get it to sync up with the music like that?" I'm happy.

I could present as rigorous definition of a beat as you could like, as I'm sure could Krash or anybody else that felt like putting in the requisite thought. I'm certainly not trying to critique anybody's intuitive concept of a "beat"...certainly most have us some more-or-less clear idea of what we mean when we say a given song has a certain beat, and can determine the beat of a song when we hear it.

We don't have the luxury of a complete music theory. A universe of beats, rythms, notes, tempos, and their (we'll assume) well-known properties is simply too complex of an ontology for me to deal with.

Within milkdrop, from the "perspective" of my code as it's running, I have an incredibly simple ontology, it contains (roughly) three values and their spatio-temporal relations. With the operations available to me, I do my best to arrange things such that these relations define a topological surface, and my task in coding a preset then becomes merely to establish isomorphisms between that surface and the 2d space of my computer screen.
unchained is offline   Reply With Quote
Old 31st August 2002, 00:15   #20
che.0
Member
 
che.0's Avatar
 
Join Date: Jun 2002
Location: Prague, Czech Republic
Posts: 62
Quote:
I thought inintially no this is not possible at all. And I mentioned this to a computer programmer friend of mine explaining that this is 'the holy grail' of music vis stuff and unlikely to succeed. Now his response was to compare it to speech recognition stuff now this is a lot more complex than simply getting a beat that a human can do easily...and it is being done nowadays. So surely it is possible and maybe we need to think about this a bit more.
In the most of contemporary music, the instrument that sets the beat are drums. So I'd say if we were able to identify certain drums in the song (which is probably problem similar to finding certain words in the speech), we'd able to find the beat.
che.0 is offline   Reply With Quote
Old 31st August 2002, 06:53   #21
ryan
not fucked, not quite.
(Forum King)
 
ryan's Avatar
 
Join Date: Feb 2002
Location: Tn
Posts: 8,798
Send a message via AIM to ryan
I havent read so many big words since I.........Nevermind
ryan is offline   Reply With Quote
Old 31st August 2002, 11:44   #22
rabidhamster
Junior Member
 
rabidhamster's Avatar
 
Join Date: May 2001
Location: Bedfordshire, UK
Posts: 15
Send a message via ICQ to rabidhamster Send a message via Yahoo to rabidhamster
..."with the oscalition methods it appears more often then not that it is all analog or variable like where in the real world a 'beat' is digital or Boolean it's either 'On' or 'Off'"...

I think if this is what you want then you might as well just use R2's (and many other peoples) method, which just uses a smoothed version of the bottom part of the sound spectrum. You can do a pretty good job of saying 'there's a beat' just by looking at when it passes a set point - much like Krash. However, you do end up with beats that shouldn't really be there, and some quiet ones maybe missed out. Its much easier to have a value between 0 and 1 which is something like the likelihood of a beat there. A very bad example of 'Fuzzy logic' I guess. But everyone does that already.

What seems to be needed is a measurent of how far you are through a beat. I don't think R2's detection is all that noticibly bad for what I use it for - but there are things I can't do with my current detection mechanism - like dancers. For that I need to be able to get the BPM and phase of the music.

I think getting a BPM from the mass on a spring is infact stupidly easy. Think about whats happening when it oscillates at the right frequency - it will be 90 degrees out of phase with the beat. This means that at the top point of each bass note, the mass will pass through the equilibrium point on the spring. You can then use some really simple code to just measure the time between passes, and you have the bpm, phase, and can get a value that goes from 0 to 1 linearly which can be used directly with morphs and dancers and stuff.
rabidhamster is offline   Reply With Quote
Old 31st August 2002, 15:23   #23
Zylot
Major Dude
 
Zylot's Avatar
 
Join Date: Jul 2001
Location: Pa, US(of)A
Posts: 803
you know, reading all this.. I've totally got a weird ass plan for beat detection that's so crazy it just might werk...


I'll see what I can do for Sunday.

-------------
What do you wish for?
--Instrumentality
Zylot is offline   Reply With Quote
Old 31st August 2002, 18:06   #24
che.0
Member
 
che.0's Avatar
 
Join Date: Jun 2002
Location: Prague, Czech Republic
Posts: 62
When I read all this I implemented a small time-based code to my algorithm. Now it's more likely to generate a beat when it goes with interval of previously generated ones. Feel free to post comments.

One more idea also came to my mind - if someone would be able to get values of bass, mid and treb out of milkdrop to some graph, where we'll be able to match them with beats in the music, it might help to identify how exactly beats change their values in time.
Attached Files
File Type: zip che - --timed sidon.zip (1.1 KB, 593 views)
che.0 is offline   Reply With Quote
Old 31st August 2002, 19:24   #25
Boz88XJ
Member
 
Join Date: Jun 2002
Location: Walnut Creek, Ca
Posts: 89
Send a message via ICQ to Boz88XJ Send a message via AIM to Boz88XJ
che.0,
I had that exact idea earlier.
Well similar anyways, I tpold idiot about it like 2 weeks ago or so, anyways here was my idea:

A feature of Milkdrop, or a seperate plugin that works with Milkdrop, its a table or graph that per time graphs/charts different variables according to what you chose.

Sure, the per_frame monitor is ok, but, at its best, its limited. This feature would allow us, for instance, to chart the progress of different variables against eachother, as well as during time.

Basically Milkdrop would run in th ebackground, performing all the code and equations of a preset, and the chart/graph would list all the variables that you want it to at time(t).
I think this would be an awesome addition, maybe just not feasible, but awesome nonetheless.
Boz88XJ is offline   Reply With Quote
Old 1st September 2002, 07:52   #26
Krash
Major Dude
 
Krash's Avatar
 
Join Date: Jun 2001
Location: Sydney, Australia
Posts: 977
Quote:
Originally posted by Zylot
Yer fergeting 1/8 beats, 1/16, 1/32, 1/64. they sneak in there, and if you managed some code that could detect the bpm of the song, first I'd conrgatz the living day lights out of you, for a job well done, but then I'd point out that these could screw with yer code quite a bit.
1/8 beats and everything need to (if possible) be avoided in the detection algorithm. This can be done easily by limiting the time interval during which a beat can be detected. Of course, if the current estimation of beatrate is wildly inaccurate, limiting the "window of opportunity" in this way can have a negative effect in the long term.

But if a song is designated as 200bpm, we want to try and detect 200 beats per minute. If we detect 1/8th beats, rather than 1/4 beats, that's fine - we'll be detecting 400bpm, but at least it's still consistent, and matches the music. The same applies if we only end up detecting 1/2 beats - we'll be detecting 100bpm, but it still matches the music.

The problems arise when we end up detecting beats in strange places, and start calculating 167bpm - this will not match up with the music at all, and look pretty bad.

Quote:
Originally posted by Che.0
Krash correct me if I'm wrong, but isn't the tempo derived from the number of beats per certain time interval? I can hardly imagine an algorithm that would detect tempo without detection beats first.
You're thinking the right way. But as you may have noticed, different people have different personal definitions of "beat", whereas "tempo" is a musically-defined term. Oh, and tempo isn't derived from beats per time interval - it's the other way around. beats per time interval is defined by the tempo (it's the tempo which is written as part of a composition, not the beatrate)

- Krash

Eighty-three percent of all statistical quotes are made up on the spot.
Krash is offline   Reply With Quote
Old 1st September 2002, 09:14   #27
unchained
Major Dude
 
Join Date: Jul 2001
Location: richmond, va
Posts: 639
Send a message via Yahoo to unchained
I'm too lazy to look into it myself, but I'm sure that a substantial theory exists in which music is structured more along the lines of modern cognitive principals and methods. A beatrate is easy to hack together code for, and has nothing else going for it.

A beat is something used as an educational tool, something you gesture towards when you're trying to teach somebody how to sing, or dance, or to give them a general easy-to-grasp concept.

Nobody listens to Bach or Hendrix for the beat. Appreciation of a bassline, or a back-beat is a THEMATIC consideration, and it's accepted in modern music theory (and cognitive psychology generally) that the only appropriate metric relationships are inter/intra-thematic.

The idea to isolate instruments (bass/drum hits) is interesting, but would eventually founder on the same rocks as early attempt to recognize and emulate speech based on analysis into phonemes, and for the same reasons...they simply wouldn't have a broad enough lexical scope. Something more thematic, and (given the limitations of milkdrop) generative is in order.
unchained is offline   Reply With Quote
Old 1st September 2002, 09:19   #28
unchained
Major Dude
 
Join Date: Jul 2001
Location: richmond, va
Posts: 639
Send a message via Yahoo to unchained
Huh?
unchained is offline   Reply With Quote
Old 1st September 2002, 10:20   #29
ryan
not fucked, not quite.
(Forum King)
 
ryan's Avatar
 
Join Date: Feb 2002
Location: Tn
Posts: 8,798
Send a message via AIM to ryan
Quote:
Huh?
My thoughts exactly...

BTW Boz im not stupid just an idiot.. And I do
remember you telling me about that...I would post the log from it bum im too lazy to sift through a HUGE text file to find it.
ryan is offline   Reply With Quote
Old 1st September 2002, 15:31   #30
rabidhamster
Junior Member
 
rabidhamster's Avatar
 
Join Date: May 2001
Location: Bedfordshire, UK
Posts: 15
Send a message via ICQ to rabidhamster Send a message via Yahoo to rabidhamster
After I posted last I had a go at the spring/mass idea. When it crosses the 0 point going up it uses this to change the tempo (but over the course of a few beats). It seems to work well over a small range of tempos - I can make it work over a wider range by increasing the friction but then it gets easily swayed by short-term trends in the music. Anyway, it seems to work kindof. Not as much as i'd like though. - I have a plan for something else but need some time to look it up

I think to make sure you have a decent detection algo you're going to need an easy way to check the beat. I just use a rectangle that stretches from the bottom to the top of the screen depending on how far through the beat (or temp/whatever) it is. I'm not entirely sure about what you can do in milkdrop, but maybe you could set it to draw a line rotated by the amount through the music. Maybe even set the line to add, its color to the bass value of the music, and the background so it doesn't move the image at all so you end up with radar effect.

just an idea...
rabidhamster is offline   Reply With Quote
Old 14th November 2002, 16:10   #31
Rovastar
Moderator
 
Join Date: Jun 2001
Location: London, England
Posts: 3,632
Send a message via AIM to Rovastar
I found this

http://www.fmod.org/forum/viewtopic.php?t=1171

& from there to

http://users.esstec.be/adion/vb/

there is someones code (in VB yeah I can understand it ) for bpm detection.

There si not much o fthis stuff freely available onthe web but what your thoughts. (I know it is BPM and not we we are all about but maybe it will give a couple of ideas)

"Rules are for the guidance of wisemen and the obedience of fools"

Visuals - Morphyre www.Morphyre.com
Rovastar is offline   Reply With Quote
Old 13th April 2004, 20:51   #32
DMSS
Junior Member
 
Join Date: Apr 2004
Posts: 18
Hi everybody..

Very interesting threat, and i'm agree that such detection is not as easy as people think I think..

Well call me stupid or whatever you want to, but could some of you tell me what are the BPM for?, I mean what can i do when having that information? In which way will change the way a listen music or whatever...

Sorry about the ignorance, but asking is the only way to learn....

Cheers...
DMSS is offline   Reply With Quote
Old 14th April 2004, 01:11   #33
Entropy42
Junior Member
 
Join Date: Jul 2001
Posts: 13
Quote:
Originally posted by Zylot
Exactly, like, Paranoia Eternal is 200 bpm (Yes, i use DDR songs for examples cause they tell you the bpm right there, plus I'm just wacky like that)

Now, that means in each minute there's 200 beats.. right?
Not so easy...

Yer fergeting 1/8 beats, 1/16, 1/32, 1/64. they sneak in there, and if you managed some code that could detect the bpm of the song, first I'd conrgatz the living day lights out of you, for a job well done, but then I'd point out that these could screw with yer code quite a bit.
At least in that case, you have an integer ambiguity, which can be resolved if you have enough information.

In the worst case your guess for the tempo is a power of 2 times the actual beat. (Whether it is a negative or positive power is irrelevant) - since it's at least synced with the beats of the music.

Given additional restrictions (such as assuming BPM is between 50 and 250, and possibly only performing a beat-related action every interval T or longer), you can get reasonably accurate results.

There's a good article on beat detection at http://http://www.gamedev.net/refere...beatdetection/

Note that this is the article credited in projectM's sourcecode comments. So for an example of beat detection implemented in C, you might want to look at projectM's source. (For those that don't know, projectM is the MilkDrop reimplementation for XMMS that was recently announced by a pair of coders from CMU.)

To the person who was asking what the benefit of beat detection is - It doesn't affect your listening experience, but can greatly benefit your viewing experience when running a vis plugin, if the plugin can perform some sort of action in time to the music. The more linked the patterns on the screen are to the music you are listening to, the greater the effect a vis plugin will have on you. (i.e. the difference between just being "bleh" and "holy @#*)#$*@ that's amazing!")
Entropy42 is offline   Reply With Quote
Old 14th April 2004, 10:32   #34
Rovastar
Moderator
 
Join Date: Jun 2001
Location: London, England
Posts: 3,632
Send a message via AIM to Rovastar
Interesting article on beat detection there. I'll read in more detail later. I don't think there was an article like this a year and a half ago when this thread was first created.

I am not sure I would find the actual BPM all that useful. Geting the beat as an on/off variable yeah taht would be useful but the BPm. It would be handy for say the speed when moving down a tunnel or something. But I would still prefer this speed to be dynamic and constantly changing to the sound input. Rather than the static BPM value that is unlikely to change noticably in quick succesion.

"Rules are for the guidance of wisemen and the obedience of fools"

Visuals - Morphyre www.Morphyre.com
Rovastar is offline   Reply With Quote
Old 14th April 2004, 12:10   #35
rabidhamster
Junior Member
 
rabidhamster's Avatar
 
Join Date: May 2001
Location: Bedfordshire, UK
Posts: 15
Send a message via ICQ to rabidhamster Send a message via Yahoo to rabidhamster
I doubt having a value for BPM is all that useful, however producing a value that goes smoothly from 0->1->2->etc as the beat progresses is amazingly handy, and that can hopefully be derived once you know BPM.

Maybe not so much for milkdrop, but think of visuals where things have to blend together. Maybe a Point Morph, or a dancer. Having that value allows you to blend smoothly between objects (or poses), timed perfectly to the beat. It makes a hell of a difference.

I'll have a closer look at the article when i get some time. It looks quite useful. One method thats come to me recently and that i'll try out shortly is matching of the sound spectrum.

If you look at the voiceprint in the WinAmp tiny fullscreen plugin, you see that on beats there are clearly the same parts being repeated. The idea is basically to store a low-res version of this, and just compare the last 2 seconds of sound each frame. A high correlation = the beat being repeated.
rabidhamster is offline   Reply With Quote
Old 23rd April 2004, 11:37   #36
rabidhamster
Junior Member
 
rabidhamster's Avatar
 
Join Date: May 2001
Location: Bedfordshire, UK
Posts: 15
Send a message via ICQ to rabidhamster Send a message via Yahoo to rabidhamster
Ok. what follows is a description of something I had some luck with. All that comes out is a value thats higher when it thinks there's a beat.

So, every 1/20th sec, you:
1. FFT the wave data (maybe not needed)
2. Come up with maybe 3 values, which are different parts of the spectrum added together.
3. Put them into a buffer, where you've buffered maybe the last 5 seconds of values.
4. Do a box filter. This is basically something like:
for (x=0;x<N/2;x++) {
result[x] = 0;
for (y=0;y<N/2;y++) result[x] += data[x+y]*data[y];
}
5. Then multiply this box filter with the last few second's worth of data to get one single value.

If you look at the box filter, it detects the beat. Now you could work out BPM from this (but not phase). If you multiply this by the incoming data though, it emphasises things that happen in time with the beat.

It obviously needs some playing with, but it still works quite well.
rabidhamster is offline   Reply With Quote
Old 15th May 2004, 09:06   #37
cartlemmy
Junior Member
 
Join Date: May 2004
Posts: 1
I've read through the entirety of this thread, and it is all very interesting how broad a range of ideas for beat detection there is.
I am actually attempting to write a plug-in that can mix one track into the next using various techniques (ie if the BPM is close from one track to the next bring tempo down/up slighty on the first to match the next; Or if the tempo varies greatly use another technique, like a dramatic reverb effect of some shtuff.)
But anyhoo I realize that to do this is no easy task, especially since I have never written a plug in. I think the main thing that will have to be done is a "beat" mapping of the entire song.
Using an FFT I would split the song into 4-8 different bands, and map out all possible beats throughout the whole song. All the beats would be stored in an array with two (or possibly three) values: the band in which the beat was found in, and the sample location insdide the song. (and also probably the velocity of the "beat").
After all of these potential beats are mapped out an alogrithm would scan through all of the beats and toss out all beats that weren't at least close to the suspected beat grid (the "beat grid"'s resolution would change depending on the assumed fastest beat (1/16th or 1/32nd whatever would depend on the song...) It would also be nice to take into account songs with odd time signatures, but jeez I'm not sure how.
Well, I realize that to do all of this would be somewhere in between hell and a nightmare. But I guess my overall points are these:

-I believe that one of the main weakesses of beat detection is the absense of looking ahead in the song...and/or looking at the song in it's entirety.

-The beat of the song needs to be detected with an extremely comlex and "fuzzy" alogrithm. ie. kick drum is located in the lower (relative mind you) spectrum and normally provide the start of the beat. Snare/****/bongos/other intruments with recognisable attack [ie piano] are usaully strongest a little above the frequency of the kick drum. Then there is the high frequencies, that I think should not be overlooked, this is where some of the faster beats come in on the high hats/symbol work whatnot.

Hell, maybe I am dead wrong...but just my thoughts.
cartlemmy is offline   Reply With Quote
Old 7th June 2004, 18:50   #38
DoomSwitch
Junior Member
 
Join Date: Jun 2004
Location: US, North Carolina
Posts: 4
Send a message via AIM to DoomSwitch Send a message via Yahoo to DoomSwitch
Hello, I have been reading this thread. I stumbled upon it searching for the way that Milkdrop detects the beat it uses. (Not mentioned in help)

I know I am definitely inexperienced with this sort of thing, I have absolutely no experience with coding and you may totally disregard this post, but I would like to share a way that I think would be a new and good way to approach this seemingly huge problem.

First of all we know every song has its own style and rhythm with various instruments that produce various sounds to create what we know as a beat. Instruments could go from orchestral violins to a deep heavy beat of synthesized bass. All of this music has a style that currently humans were given a gift to interpret and take as a mood. Now we are trying to throw this seemingly simple task onto one of our greatest creations, the computer.

In order to take this incredible instinct and submit it in code this will have to take several thoughts of how we as people perceive music. We should take into account that music does not always come down to one constant instrument that creates a given "beat" but a pattern of many instruments that will create a timed rhythm. That, my friends, is what we essentially call music. In order to produce a code that will interpret music as we do, we would have to take into account all of these facts, and not one constant. If you listen to a song now, say it was orchestrated, you will notice one thing: that you are listing to music comprised of more than one note. You are not specifically looking for one certain instrument to carry or produce a beat, but a series of repetitive frequencies and tones that create a tune and rhythm. A code should be made to look for these patterns and tunes and take advantage of them.

I like how previously Che.0 mentioned that one of his friends used speech recognition as an example. Speech recognition uses a set of known words and phrases to interpret its medium. The user is subject to interfacing with the program with the predetermined phrases and word it knows. The user does have the ability to manually teach it a new set of words and so on. Applying this to the ReCoGnItIoN of music this would probably get good results. This would be a good approach; however, it would in due course produce a systematical set way of dealing with music. This method may work great, but it will not ultimately deal with music the way we do.

This recognition method may be further enhanced to freely interpret music and grasp almost any concept that has a repetitive musical OR rhythmic pattern. The way that today’s beat detection technology looks at things is in black and white. Currently beat detection looks for one certain nuance and applies that nuance over time. As you know, even though it seems that would work... most of the time it doesn’t. We take almost every task that we do simply everyday for granted, but its surely not. We, the people who create the music, look at it much differently than we tell our electronic counterparts to. Once we identify how we listen is the only time we can actually define that processES in code and see those processes verified as true through our computers.

If we could get computers to recognize music as we do, we could open the door to all kinds of new visualizations, processes, programs, auto-DJs, and other kinds of tasks not related to music. It would be one of the first steps of artificial observation.

I hope this post helped in some kind of way, if it didn’t please slap me with your comments... Any criticism would be appreciated.
DoomSwitch is offline   Reply With Quote
Old 14th June 2004, 05:07   #39
Feldspar
Junior Member
 
Join Date: Jun 2001
Posts: 6
Quote:
Originally posted by rabidhamster
If you look at the box filter, it detects the beat. Now you could work out BPM from this (but not phase). If you multiply this by the incoming data though, it emphasises things that happen in time with the beat.
Im not sure I get your description but from what I gather it is similar to a reasarch paper I saw that detected BPM (but not phase). The method was to take an entire song and compute the self-similarity matrix by convolving the entire fft spectrum - each element x,y in the matrix is the dot product of the fft vector at time index x and the fft vector at time index y. You end up with a grid-looking image with lines spaced according to the BPM. I cant find the paper I'm thinking of but here is one that uses something similar and it has a picture of what im talking about

http://www.ipem.ugent.be/mami/Public...rofuse2002.pdf

Another approach is to use the fft data to detect loud drum hits and such, and then perform some kind of comb-filter analysis to get a bpm and phase based on where the peaks in this signal are.

It is fairly complex but is also more robust to non-repetative musics than self-similarity methods are. There are some impressive results with this, check out this site and click results page for some examples.

http://web.media.mit.edu/~eds/beat/
Feldspar is offline   Reply With Quote
Old 14th June 2004, 10:50   #40
DoomSwitch
Junior Member
 
Join Date: Jun 2004
Location: US, North Carolina
Posts: 4
Send a message via AIM to DoomSwitch Send a message via Yahoo to DoomSwitch
Wow, That does a really great job
DoomSwitch is offline   Reply With Quote
Reply
Go Back   Winamp & Shoutcast Forums > Visualizations > MilkDrop > MilkDrop Presets

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump