No announcement yet.

Wiimote-reactive Milkdrop visuals

  • Filter
  • Time
  • Show
Clear All
new posts

  • Wiimote-reactive Milkdrop visuals

    Hello developers, users, preset artists,

    I just released the first version version of a Milkdrop mod that I think you'll like. I modified Milkdrop so that all the preset graphics is drawn only in response to you moving two Wiimotes in your hands. Move your right hand to the right, for example, and the fireball on the screen glows more blue. Move it faster forward, and it becomes hot white and zoomed in closer.

    If you do it to the rhythm of the music (whether played by Winamp or from some other source -- an iPod in your room or a DJ in the nightclub), and if you hook up a projector to your PC, the feeling of the feedback when you flick the Wiimote and see the preset/Milkdrop instantly react is spectacular.

    Here's the best part: it uses many existing Milkdrop presets unchanged. I disabled all sound analysis in Milkdrop, and replaced it with Wiimote readings. So the "bass" variable that a preset reads is really the acceleration on the X axis of the right Wiimote, "bass_att" is the X acceleration of the left Wiimote, "treb_att" is Z acceleration of left and right combined, and so on. In other words the stronger you move the Wiimote on the X axis, the stronger the "bass" the preset thinks is, and some shape on the screen gets bigger or greener or whatever in response.

    I put on YouTube a demo of the software in action at

    -- I'm still struggling to find the best way to describe this but I hope the video will make it clear. And fun, too.

    You can download the software (predictably named Miilkdrop, packaged as a Winamp vis plugin) and setup instructions, from It's free, and I'll post the source code as well, once I clean it up.

    I included in the installation hundreds of fantastic presets that react well to Wiimotes. I also modified one of them (Zylot's "Wisps") to directly read some new Wii-related preset variables I introduced.

    There's other software out there that connects Wiimotes to visuals, for example via MIDI which a VJ-ing program reads. But everything I've seen looked too slow: for the feedback to feel good, both for you and the people watching, the image must be drawn no later than around 30-40ms (roughly the same time as one frame of video) after you move the Wiimote. Milkdrop is a speed demon that's up to the task.

    Thanks for reading this, if you'd like to give it a shot I'd appreciate any feedback (and/or Wii-customized presets that I'd include in the next release).


  • #2
    ... just noticed the URLs didn't come through, so here they are in the raw form:

    YouTube video:

    Download page:


    • #3
      I SOOOO have to check this out when I get home, Hope my bluetooth is compatable...

      which version of milkdrop did you make this from?
      or on of the 1.05 betas?

      I'm assuming its not 2 since I dont think the source is out there...


      • #4
        It's 1.04b I think, the only source I could find. (I SOOOO wish I had 2... :-) But even the 1.04 source is priceless.)

        As for USB bluetooth, I can recommend the one I got from SparkFun Electronics for $17, link below. It worked for me on 3 different laptops, two of which had bluetooth built in but built-ins rarely work with Wiimotes. The SparkFun one was also recommended by a well-known Wiimote hacker (well known in the Wiimote hacking circles anyway), Johnny Chung.

        You'll also definitely want two Wiimotes. Circuit City and the like usually have plenty in stock for $40 apiece even though you can't buy the Wii console itself. (I don't have one either.)


        • #5
          hey thats pretty fun... what are the available variables from the wii?

          here's a quick remix of your test preset...
          the left wiimote controls the zoom and the rot, tilt it to the sides to rotate and tilt foward and back to zoom, right one does the same as it use to, but for both "dots"
          Attached Files


          • #6
            This is great!! I love it how you can change the depth of the tunnel when you want to, and then send the fireballs down so they "fall" longer. The feedback is awesome, the best so far.

            Here's the list of all the variables:

            wii_l_x, for example, is the acceleration on the X axis of the left Wiimote. The X axis is not in space but relative to the Wiimote -- it's in the direction and orientation of the left cursor button on the Wiimote. It's range is -3.0 to +3.0, close to 0.0 if the Wiimote is not moving (on its X axis). The Z axis is in the direction of the face of the Wiimote, and the Y axis is the direction of the up cursor button.

            The roll and pitch (wii_*_r and wii_*_p) are the only ones that don't drop to 0 when you hold the Wiimote still so they can be used to change something slowly, like this depth and rotation of the Wisps' tunnel.

            Were you able to connect the hardware already?


            • #7
              ... here's a video of the remixed preset:



              • #8
                ya hardware hook up just fine, already had a bluetooth adapter and the wiimotes... prolly have some more remixes for ya soon...


                • #9
                  holy crap! this is awesome!
                  and hey i've got all the necessary hardware (beamer, wiimotes, BT dongle)
                  and as you crawled my mind, about two weeks ago i've been to Berlin where i discussed this idea with some friends who want to organize a demo-party.
                  Someone has to call Ryan Geiss for the Milkdrop2 source . I want to dive into brilliant shader fractals with my wiimotes.
                  okay this is dreams of the future, but imagine a headtracking implementation..

                  ...something very cool is going on...
                  WebSocket Hub for Kinect SDK 2.0 with Milkdrop shader pipeline in VanillaJS and glsl
                  Codepen | Shadertoy | OpenProcessing | studio sketchpad
                  Twitter @ Google+ @ YouTube @ Facebook


                  • #10
                    I just saw 1.05 beta on redi jedi's site... don't know how I missed it. I'll take it and Wiify it, probably in a week or so.

                    As for a party, you can imagine 5-6 people holding a Wiimote each and each controlling a different part of the preset. Each would be in charge of visualizing a different instrument or vocals in the song, kind of like playing in a visual orchestra.

                    (Would be easy to make, Miilkdrop-wise, too, I'd just need to add vars wii_1_x, wii_2_x, wii_3_x etc, and you'd need a good preset to show it all well.)


                    • #11
                      oh no, i can't find my bluetooth dongle - i fear this is the moment, where i have to tidy up my room
                      WebSocket Hub for Kinect SDK 2.0 with Milkdrop shader pipeline in VanillaJS and glsl
                      Codepen | Shadertoy | OpenProcessing | studio sketchpad
                      Twitter @ Google+ @ YouTube @ Facebook


                      • #12
                        (A) buttons for effects

                        redi jedi,

                        here's a minor edit to show the use of the "A" buttons on wiimotes for effects ("B" buttons are for preset navigation):


                        so that if the left button (A) is pressed, it blocks the rotation. (i also added *.3 to reduce sensitivity.) wii_l_b is either 0.0 or 1.0.

                        and also replace
                        so that the right A button blocks the zoom.

                        i like it how it looks when i start with small visual changes and then expand as the music goes, using the most dramatic effects sparingly. in this case it's start with both zoom and rot blocked, just activating one dot, than another, and so on until at one point everything explodes at once.

                        thanks again btw, this remix is fantastic.

                        Attached Files


                        • #13
                          i borrowed my dongle to a friend and will get it back next weekend. say what's the physical maximum number of wiimotes you can connect to a pc resp. milkdrop?
                          My hypothesis: if you average all input of a serious number of 'pilots' and map it on the right selection of visualization, some very strange psychic effects will appear.
                          You know the Ideomotor effect? ( )
                          I hope, steering the motion will end up in flowing and surfing the viz and finally in a perfect mindfuck. at least it's worth a try
                          WebSocket Hub for Kinect SDK 2.0 with Milkdrop shader pipeline in VanillaJS and glsl
                          Codepen | Shadertoy | OpenProcessing | studio sketchpad
                          Twitter @ Google+ @ YouTube @ Facebook


                          • #14
                            a friend of mine suggested exact same thing...

                            i read somewhere that the maximum number of wiimotes per PC is 7 (some master-slave thing, part of the protocol). but there's nothing that says that you can't have another PC that reads 7 more, and so on, and they can all send updates the one running miilkdrop over the network. the averaging could be done by miilkdrop and given to presets as input vars (wii_avg_x etc.)

                            what would the preset look like?


                            • #15
                              i would really love to see a trip through one of the chaotic "skin dot" presets, but... there's no go with Milkdrop 1.04/1.05.
                              I guess a really fast ride into a starfield should work fine too.
                              i will do my own experiments and give feedback!
                              WebSocket Hub for Kinect SDK 2.0 with Milkdrop shader pipeline in VanillaJS and glsl
                              Codepen | Shadertoy | OpenProcessing | studio sketchpad
                              Twitter @ Google+ @ YouTube @ Facebook