Wiimote-reactive Milkdrop visuals

[indaVizz]

Hello developers, users, preset artists,

I just released the first version version of a Milkdrop mod that I think you'll like. I modified Milkdrop so that all the preset graphics is drawn only in response to you moving two Wiimotes in your hands. Move your right hand to the right, for example, and the fireball on the screen glows more blue. Move it faster forward, and it becomes hot white and zoomed in closer.

If you do it to the rhythm of the music (whether played by Winamp or from some other source -- an iPod in your room or a DJ in the nightclub), and if you hook up a projector to your PC, the feeling of the feedback when you flick the Wiimote and see the preset/Milkdrop instantly react is spectacular.

Here's the best part: it uses many existing Milkdrop presets unchanged. I disabled all sound analysis in Milkdrop, and replaced it with Wiimote readings. So the "bass" variable that a preset reads is really the acceleration on the X axis of the right Wiimote, "bass_att" is the X acceleration of the left Wiimote, "treb_att" is Z acceleration of left and right combined, and so on. In other words the stronger you move the Wiimote on the X axis, the stronger the "bass" the preset thinks is, and some shape on the screen gets bigger or greener or whatever in response.

I put on YouTube a demo of the software in action at

     http://youtube.com/watch?v=bCVF6EZFR3g

-- I'm still struggling to find the best way to describe this but I hope the video will make it clear. And fun, too.

You can download the software (predictably named Miilkdrop, packaged as a Winamp vis plugin) and setup instructions, from http://indaviz.com/download/. It's free, and I'll post the source code as well, once I clean it up.

I included in the installation hundreds of fantastic presets that react well to Wiimotes. I also modified one of them (Zylot's "Wisps") to directly read some new Wii-related preset variables I introduced.

There's other software out there that connects Wiimotes to visuals, for example via MIDI which a VJ-ing program reads. But everything I've seen looked too slow: for the feedback to feel good, both for you and the people watching, the image must be drawn no later than around 30-40ms (roughly the same time as one frame of video) after you move the Wiimote. Milkdrop is a speed demon that's up to the task.

Thanks for reading this, if you'd like to give it a shot I'd appreciate any feedback (and/or Wii-customized presets that I'd include in the next release).

Cheers,
Davor/indaVizz

[indaVizz]

... just noticed the URLs didn't come through, so here they are in the raw form:

YouTube video: youtube.com/watch?v=bCVF6EZFR3g

Download page: indaviz.com/download

[redi jedi]

I SOOOO have to check this out when I get home, Hope my bluetooth is compatable...

which version of milkdrop did you make this from?
1.04
1.04d
or on of the 1.05 betas?

I'm assuming its not 2 since I dont think the source is out there...

[indaVizz]

It's 1.04b I think, the only source I could find. (I SOOOO wish I had 2... :-) But even the 1.04 source is priceless.)

As for USB bluetooth, I can recommend the one I got from SparkFun Electronics for $17, link below. It worked for me on 3 different laptops, two of which had bluetooth built in but built-ins rarely work with Wiimotes. The SparkFun one was also recommended by a well-known Wiimote hacker (well known in the Wiimote hacking circles anyway), Johnny Chung.

You'll also definitely want two Wiimotes. Circuit City and the like usually have plenty in stock for $40 apiece even though you can't buy the Wii console itself. (I don't have one either.)

wiili.org/index.php/Compatible_Bluetooth_Devices
sparkfun.com/commerce/product_info.php?products_id=150

[redi jedi]

hey thats pretty fun... what are the available variables from the wii?

here's a quick remix of your test preset...
the left wiimote controls the zoom and the rot, tilt it to the sides to rotate and tilt foward and back to zoom, right one does the same as it use to, but for both "dots"

[indaVizz]

This is great!! I love it how you can change the depth of the tunnel when you want to, and then send the fireballs down so they "fall" longer. The feedback is awesome, the best so far.

Here's the list of all the variables:
indaviz.com/download/miilkdrop.html#8

wii_l_x, for example, is the acceleration on the X axis of the left Wiimote. The X axis is not in space but relative to the Wiimote -- it's in the direction and orientation of the left cursor button on the Wiimote. It's range is -3.0 to +3.0, close to 0.0 if the Wiimote is not moving (on its X axis). The Z axis is in the direction of the face of the Wiimote, and the Y axis is the direction of the up cursor button.

The roll and pitch (wii_*_r and wii_*_p) are the only ones that don't drop to 0 when you hold the Wiimote still so they can be used to change something slowly, like this depth and rotation of the Wisps' tunnel.

Were you able to connect the hardware already?

[indaVizz]

... here's a video of the remixed preset:

youtube.com/watch?v=AVqtB0tJkuo

[redi jedi]

ya hardware hook up just fine, already had a bluetooth adapter and the wiimotes... prolly have some more remixes for ya soon...

[Flexi]

holy crap! this is awesome!
and hey i've got all the necessary hardware (beamer, wiimotes, BT dongle)
and as you crawled my mind, about two weeks ago i've been to Berlin where i discussed this idea with some friends who want to organize a demo-party.
Someone has to call Ryan Geiss for the Milkdrop2 source . I want to dive into brilliant shader fractals with my wiimotes.
okay this is dreams of the future, but imagine a headtracking implementation..

...something very cool is going on...

[indaVizz]

I just saw 1.05 beta on redi jedi's site... don't know how I missed it. I'll take it and Wiify it, probably in a week or so.

As for a party, you can imagine 5-6 people holding a Wiimote each and each controlling a different part of the preset. Each would be in charge of visualizing a different instrument or vocals in the song, kind of like playing in a visual orchestra.

(Would be easy to make, Miilkdrop-wise, too, I'd just need to add vars wii_1_x, wii_2_x, wii_3_x etc, and you'd need a good preset to show it all well.)

[Flexi]

oh no, i can't find my bluetooth dongle - i fear this is the moment, where i have to tidy up my room

[indaVizz]

(A) buttons for effects

redi jedi,

here's a minor edit to show the use of the "A" buttons on wiimotes for effects ("B" buttons are for preset navigation):

replace
rot=-wii_l_r*wii_l_f
with
rot=-wii_l_r*wii_l_f*(1-wii_l_b)

so that if the left button (A) is pressed, it blocks the rotation. (i also added *.3 to reduce sensitivity.) wii_l_b is either 0.0 or 1.0.

and also replace
zoom=.9+sin(wii_l_p)*.1
with
zoom=.9+sin(wii_l_p*(1-wii_r_b))*.1
so that the right A button blocks the zoom.

i like it how it looks when i start with small visual changes and then expand as the music goes, using the most dramatic effects sparingly. in this case it's start with both zoom and rot blocked, just activating one dot, than another, and so on until at one point everything explodes at once.

thanks again btw, this remix is fantastic.

davor/indaVizz

[Flexi]

i borrowed my dongle to a friend and will get it back next weekend. say what's the physical maximum number of wiimotes you can connect to a pc resp. milkdrop?
My hypothesis: if you average all input of a serious number of 'pilots' and map it on the right selection of visualization, some very strange psychic effects will appear.
You know the Ideomotor effect? ( http://en.wikipedia.org/wiki/Ideomotor_effect )
I hope, steering the motion will end up in flowing and surfing the viz and finally in a perfect mindfuck. at least it's worth a try

[indaVizz]

a friend of mine suggested exact same thing...

i read somewhere that the maximum number of wiimotes per PC is 7 (some master-slave thing, part of the protocol). but there's nothing that says that you can't have another PC that reads 7 more, and so on, and they can all send updates the one running miilkdrop over the network. the averaging could be done by miilkdrop and given to presets as input vars (wii_avg_x etc.)

what would the preset look like?

[Flexi]

i would really love to see a trip through one of the chaotic "skin dot" presets, but... there's no go with Milkdrop 1.04/1.05.
I guess a really fast ride into a starfield should work fine too.
i will do my own experiments and give feedback!